Implementation evaluation as a dimension of the quality assurance of a new programme for medical education and training

Date
2004-12
Journal Title
Journal ISSN
Volume Title
Publisher
Stellenbosch : University of Stellenbosch
Abstract
ENGLISH ABSTRACT: In this thesis, an ‘alignment approach’ to the quality assurance of medical curricula is developed and practically illustrated in the evaluation of a section of a new curriculum in undergraduate medical education and training instituted at the Faculty of Health Sciences of the University of Stellenbosch in 1999. The background of curriculum innovation at this institution during the 1990s is described, and the literature on the concepts of quality assurance is explored in higher education in general and in medical education and training in particular. The current focus on socially responsive curriculum renewal and accountability illustrates the need for this study. The empirical part of the study was conducted in two phases. The first phase consisted of a ‘clarification evaluation’. The planning of the new curriculum introduced in 1999 was analysed retrospectively through a study of the planning documents and interviews with leaders of the planning process. The results of this clarification evaluation are presented in the form of a ‘Logic Model’. The implicit theory of the curriculum, as represented by the Logic Model, was then evaluated regarding its consistency with trends in medical education. These trends were determined through a study of the literature on the subject published during the time of the planning of the curriculum. It was found that the planning of the curriculum was in line with most of the identified trends, but that it lacked detailed information on how the basic sciences and clinical skills training were to be addressed. This compromised the evaluability of phase I of the curriculum and of the clinical rotations1 by the method use in this study. Because of this, and also considering the time frame of this evaluation, phase I of the curriculum and the late clinical rotations were excluded from the second phase of the study. The aims identified for the curriculum during the process of clarification evaluation were also aligned with the document, The Profile of the Stellenbosch Doctor 2 . This indicates that the planning process of the curriculum was in line with its intended outcome.The second phase of the study consisted of an ‘implementation evaluation’ of phases II and III of the theoretical components and of the early and middle clinical rotations of the curriculum. Data for this implementation evaluation were collected from April 2002 to June 2003. Module chairpersons3, lecturers and students were used as sources of data for the evaluation of the theoretical phases. The perceptions of these groups regarding the implementation of phases II and III of the theoretical part of the curriculum were collected by means of questionnaires designed specifically for this study. For the evaluation of the clinical rotations, the results of the standard student feedback obtained by the Faculty of Health Sciences were used as a source of data for a secondary analysis. The study guides provided for each of the theoretical modules and the clinical rotations were also used as a secondary source for the analysis of data. The data obtained were then analysed by using the framework provided by the Logic Model. Following this, a judgment of the quality of the implementation of the curriculum was made. The planned curriculum was aligned with the practised curriculum by drawing up a ‘curriculum scoreboard’. It was found that alignment was adequately achieved for six of the identified aims, while the implementation of four of the aims was not aligned to the planning according to the criteria used in this study. The study illustrates that the methods of programme evaluation can be validly applied in the evaluation of a curriculum in medical education and training. The Logic Model enables an alignment between the planned and the practised curriculum, which can be used as a measure of the quality of a curriculum in terms of ‘fitness of purpose’. 1 See Addendum A for a diagrammatic overview of the curriculum. The curriculum was structured into three theoretical phases (phases I, II and III) and three clinical rotations (early, middle and late). 2 This document was drawn up during the initial phases of the planning process of the curriculum and regarded by the Faculty as a blueprint for the intended outcomes of the curriculum. 3 A module chairperson in the context of the Faculty of Health Sciences of the University of Stellenbosch is a senior faculty member responsible for the organisation and management of the modules presented as part of the curriculum in medical education and training.
AFRIKAANSE OPSOMMING: In hierdie tesis word ʼn ‘belyningsbenadering’ tot die gehalteversekering van mediese kurrikula ontwikkel en prakties op die proef gestel deur ʼn gedeelte van die nuwe kurrikulum vir voorgraadse mediese onderrig, wat in 1999 aan die Fakulteit Gesondheidswetenskappe van die Universiteit van Stellenbosch ingestel is, te evalueer. Die agtergrond van kurrikulumverandering in hierdie instansie gedurende die 1990’s word ondersoek, en daar word ’n oorsig gegee van die literatuur oor die konsepte van gehalteversekering wat op daardie stadium in die hoër onderwys in die algemeen en in mediese onderrig in besonder in gebruik was. Die huidige fokus op sosiaal responsiewe kurrikula en verantwoordbaarheid illustreer die noodsaaklikheid van ʼn studie van hierdie aard. Die empiriese gedeelte van die studie is in twee fases uitgevoer. Die eerste fase het bestaan uit ‘n ‘verklarende evaluasie’. Die beplanning van die 1999-kurrikulum is retrospektief geanaliseer deur die bestudering van die relevante beplanningsdokumente en deur onderhoude met leiers van die beplanningsproses te voer. Die resultate van die verklarende evaluasie is in die vorm van ʼn ‘Logika Model’ voorgestel. Die implisiete teorie van die kurrikulum, soos voorgestel in die Logika Model, is daarna geëvalueer ten opsigte van die ooreenstemming van die model met die tendense in mediese onderrig wat op daardie stadium geldig was. Hierdie tendense is nagespeur in die belangrikste literatuur oor die onderwerp wat in dieselfde tydperk as die beplanning van die 1999-kurrikulum gepubliseer is. Die bevinding was dat die beplanning van die kurrikulum in lyn is met die meerderheid geïdentifiseerde tendense, maar dat die basiese wetenskappe en opleiding in kliniese vaardighede nie in detail aangespreek is nie. Dit het die evalueerbaarheid van fase I van die kurrikulum en die kliniese rotasies4 deur die metode wat in hierdie studie gebruik is, gekompromitteer. Om hierdie rede, en met inagneming van die tydsraamwerk van hierdie evaluasie, is fase I en die laat kliniese rotasies nie in die tweede gedeelte van hierdie studie ingesluit nie. Die doelwitte van die kurrikulum wat gedurende die verklarende evaluasie geformuleer is, is ook met die dokument, Die Profiel van die Stellenbosch dokter 5, belyn. Dít het aangedui dat die beplanningsproses van die kurrikulum in lyn met die beoogde uitkoms daarvan is.Die tweede deel van die studie het bestaan uit ʼn ‘implementerings-evaluasie’ van fases II en III van die teoretiese komponente en van die vroeë en middel kliniese rotasies van die kurrikulum. Data vir die implementerings-evaluasie is vanaf April 2002 tot Junie 2003 ingesamel. Modulevoorsitters6, dosente en studente is as bronne van data vir die evaluering van die teoretiese fases gebruik. Die indrukke van hierdie groepe persone betreffende die implementering van die teoretiese fases is deur middel van vraelyste ingesamel wat spesiaal vir hierdie studie ontwerp is. Vir die evaluering van die kliniese rotasies is die resultate van die standaard studenteterugvoer wat deur die Fakulteit ingewin word, gebruik as bron vir sekondêre analise. Die studiegidse wat vir elke teoretiese module en die kliniese rotasies verskaf word, het ook as ʼn bron vir sekondêre data-analise gedien. Die data wat vir hierdie studie ingewin is, is deur middel van die raamwerk wat deur die Logika Model verskaf is, geanaliseer. Daarna is ʼn oordeel gevel oor die kwaliteit van die implementering van die kurrikulum. Die kurrikulum-soos-beplan is belyn met die uitgevoerde kurrikulum deur ’n ‘kurrikulumtelbord’ op te stel. Die bevinding was dat hierdie belyning voldoende bereik is vir ses van die geïdentifiseerde doelstellings van die kurrikulum, terwyl die uitvoering van vier van die doelstellings nie goed met die beplanning daarvan belyn was volgens die kriteria wat vir hierdie studie gebruik is nie. Hierdie studie illustreer dat die metodes van programevaluasie geldig toegepas kan word in die evaluering van ’n kurrikulum in mediese onderrig en opvoeding. Die Logika Model maak dit moontlik om die beplande kurrikulum met die uitgevoerde kurrikulum te belyn. Dit kan dan gebruik word as ’n maatstaf van die kwaliteit van ’n kurrikulum in terme van ‘geskiktheid vir doel’.4 Sien Addendum A vir ʼn diagrammatiese oorsig van die kurrikulum. Die kurrikulum is gestruktureer volgens drie teoretiese fases (fases I, II en III) en drie kliniese rotasies (vroeg, middel en laat). 5 Hierdie dokument is gedurende die vroeë fases van die beplanningsproses van die kurrikulum saamgestel en word deur die Fakulteit as ʼn bloudruk vir die beoogde uitkomste van die kurrikulum beskou.6 ’n Module-voorsitter in die konteks van die Fakulteit Gesondheidswetenskappe van die Universiteit van Stellenbosch is ʼn senior lid van die fakulteit wat verantwoordelik is vir die organisasie en bestuur van die modules wat as deel van die kurrikulum in mediese onderrig en opleiding aangebied word.
Description
Thesis (DPhil)--University of Stellenbosch, 2004.
Keywords
Educational evaluation, Medical education -- Aims and objectives, Quality assurance, Theses -- Sociology, Dissertations -- Sociology
Citation