Browsing by Author "Van Schalkwyk, Ane"
Now showing 1 - 1 of 1
Results Per Page
Sort Options
- ItemA conceptual framework for generating insights from log analysis in support of mHealth App evaluations(Stellenbosch : Stellenbosch University, 2022-04) Van Schalkwyk, Ane; Grobbelaar, Sara; Herselman, Marlien; Vermeulen, Euodia; Stellenbosch University. Faculty of Engineering. Dept. of Industrial Engineering.ENGLISH ABSTRACT: As the digital health domain, particularly mobile health apps (mHealth), continues to grow, considerable benefits exist for addressing resource-constrained healthcare challenges. Evidence supporting the use, implementation, and sustainability of the Apps are required to realise these benefits. Unfortunately, traditional Monitoring and Evaluation (M&E) methods are not suited to Apps’ fast-paced agile developments, resulting in a lack of the required evidence base. However, these Apps generate a plethora of log data that provide evidence of the user-technology interactions. Consequently, the log data could be used to support the needed M&E practices for mHealth Apps. Currently, there is a lack of structure and standards required for log analysis to contribute to M&E practices and move beyond descriptive statistics. This research project addresses this need by implementing a Design Science Research (DSR) approach to develop a conceptual framework. The VILA (Valuable Insights from Log Analysis) Framework, developed in this study, structures how insights can be generated from log analysis to support the M&E practices of mHealth Apps. It thereby contributes to the increased use of log data towards building the required evidence base for the support and sustainability of mHealth initiatives. The research design implemented is divided into two phases and five stages. Phase 1 (Theoretical component) incorporates two rigour cycles to inform the first design cycle. A preliminary review, two narrative overviews, an umbrella review, and a scoping review identify the necessary M&E, Digital Health, and Data Science concepts and categories to inform the conceptual framework. The existing M&E frameworks for mHealth Apps are identified and used to develop domain concepts where log analysis would be able to contribute. Thereby, the preliminary VILA Framework is developed based on the existing knowledge base. Next, the research design’s Phase 2 (Evaluation component) uses three stages to evaluate and refine the VILA Framework, thereby ensuring the practical value of the artefact. The VILA Framework is evaluated using two relevance cycles and one change and impact cycle according to the DSR approach. The first relevance cycle evaluates the efficacy, quality, and generalisability of the preliminary VILA Framework using expert reviews. The expert reviews consist of semi-structured interviews and a questionnaire with eight South African subject matter experts. The expert reviews confirm the quality and efficacy of the VILA Framework and highlight findings such as the need for a structured process, the challenges of log data, and the generalisability of the VILA Framework. During the second design cycle, the findings are translated into conceptual and structural improvements to develop the refined VILA Framework. The second relevance cycle applies the refined VILA Framework to a real-world case study to structure the log analysis of a South African App. The case study incorporates quantitative and qualitative data analysis, interviews, and four focus groups to evaluate the validity, utility, applicability, and sustainability of the VILA Framework. The case study outcomes include improvement suggestions and future recommendations for the VILA Framework and the company’s M&E practices. Thereby, the case study benefits the participating company and the researcher, establishing the value of the DSR approach and the VILA Framework. Finally, the case study findings are implemented to develop the final VILA Framework (design cycle 3). This research project contributes Level 1 and Level 2 knowledge contributions by providing the developed artefact (i.e., VILA Framework) and documenting its development and evaluation processes. Therefore, adhering to the pragmatic paradigm and DSR principles, the VILA Framework suffices the problem statement. It benefits researchers and practitioners who aim to standardise log analyses or improve the M&E of mHealth Apps. In addition, the project also highlights valuable research topics for future recommendations to improve or optimise the VILA Framework or research within this scope.