The interpretation and use of mixed methods research within programme evaluation practice
Thesis (MPhil (Sociology and Social Anthropology))--University of Stellenbosch, 2010.
The contemporary evaluation literature advocates for and recommends a pluralistic approach to programme evaluation, with some writers contending that the use of multiple and/or mixed methods for the practice is inevitable. The rationale for such an approach encompasses aspects of both the ‘technical’ and the ‘political’ requirements of evaluation practice. A review of evaluation research literature underscores the important role of mixed methods research towards realizing richer evaluation findings, and addressing the pragmatic, democratic and political facets of the evaluation practice. However, it is observed that there is a dearth of literature that focuses on how the use of a mixed methods evaluation approach facilitates the realization of richer conclusions or inferences about programme merit/worth. Thus, the overarching aim of the thesis is to establish how the perception and implementation of mixed methods research among evaluation practitioners influences the nature of inferences they make. This thesis aims at identifying patterns and relationships within and between conceptions and practices of mixed methods evaluation through a descriptive process. The selection of cases is therefore purposive and includes fourteen published evaluation articles on projects/programmes. An analytical framework is developed on the basis of a literature review on mixed methods research and background literature on evaluation research. This framework guides the qualitative content analysis of each case study and the cross-case analysis across the fourteen studies to identify common patterns. The findings reveal two prominent perspectives of mixed methods evaluation prevailing among evaluation practitioners. The first (labeled a ‘strong’ conception) has the intention of and places emphasis on the integration of the qualitative and quantitative components, with the primary objective of obtaining richer evaluation inferences. In this conception, the use of the methods and the data/inferences thereof are synthesized to achieve this goal. This conception is congruent with mixed methods purposes of: - ‘complementarity’ and ‘triangulation’ and is responsive to the ‘technical’ needs of evaluation. The second perspective (labeled a ‘weak’ conception) is silent about the integration of the respective methods or data/findings/inferences, qualifying the use of multiple methods and data in a single study as sufficing for a mixed methods approach. It resonates with justifications of mixed methods research that address issues of: - comprehensiveness, multiple view points, inclusiveness and democracy and seems more tailored to the ‘political’ needs of evaluation. The findings also reveal that the resulting multiple inferences from this ‘weak’ conception can weaken each other when contradicting or inaccurate qualitative and quantitative findings result, especially when the complimentary function of either method is not planned a priori. Therefore within the context of realizing richer and more valid evaluation findings/inferences, it is recommended that the purposes and qualification as mixed methods research of the second perspective be re-considered. It is apparent that in embracing the ‘political’ needs of evaluation practice, this conception seems to eschew the ‘technical’ requirements initially intended for a mixed methods approach. This has implications particularly for the mixed methods purpose of ‘expansion’ and rationales of pluralism, inclusiveness and democracy, which are seemingly popular within programme evaluation practice.