Modelling the pre-assessment learning effects of assessment : evidence in the validity chain
Date
2012-10
Journal Title
Journal ISSN
Volume Title
Publisher
Blackwell Publishing Ltd
Abstract
OBJECTIVES We previously developed a
model of the pre-assessment learning effects of
consequential assessment and started to
validate it. The model comprises assessment
factors, mechanism factors and learning effects.
The purpose of this study was to continue the
validation process. For stringency, we focused
on a subset of assessment factor–learning effect
associations that featured least commonly in a
baseline qualitative study. Our aims were to
determine whether these uncommon associations
were operational in a broader but similar
population to that in which the model was
initially derived.
METHODS A cross-sectional survey of 361
senior medical students at one medical school
was undertaken using a purpose-made
questionnaire based on a grounded theory and
comprising pairs of written situational tests. In
each pair, the manifestation of an assessment
factor was varied. The frequencies at which
learning effects were selected were compared
for each item pair, using an adjusted alpha to
assign significance. The frequencies at which
mechanism factors were selected were calculated.
RESULTS There were significant differences in
the learning effect selected between the two
scenarios of an item pair for 13 of this subset of
21 uncommon associations, even when a
p-value of < 0.00625 was considered to indicate
significance. Three mechanism factors were
operational in most scenarios: agency; response
efficacy, and response value.
CONCLUSIONS For a subset of uncommon
associations in the model, the role of most
assessment factor–learning effect associations
and the mechanism factors involved were supported
in a broader but similar population to
that in which the model was derived. Although
model validation is an ongoing process, these
results move the model one step closer to the
stage of usefully informing interventions. Results
illustrate how factors not typically included in
studies of the learning effects of assessment
could confound the results of interventions
aimed at using assessment to influence learning.
Description
Publication of this article was funded by the Stellenbosch University Open Access Fund.
The original publication is available at http://onlinelibrary.wiley.com/journal/10.1111/%28ISSN%291365-2923/
The original publication is available at http://onlinelibrary.wiley.com/journal/10.1111/%28ISSN%291365-2923/
Keywords
Study skills -- Evaluation, Learning -- Ability testing, Education, Higher -- Research, Study skills -- Research, Educational tests and measurements -- Research
Citation
Cilliers, F. J., Schuwirth, L. W. T. & Van Der Vleuten, C. P. M. 2012. Modelling the pre-assessment learning effects of assessment: evidence in the validity chain. Medical Education, 46(11):1087-1098, doi:10.1111/j.1365-2923.2012.04334.x.