Investigating the construct validity of a developmental assessment centre

Brits, Nadia M (2011-12)

Thesis (MComm)--University of Stellenbosch, 2011.

Thesis

AFRIKAANSE OPSOMMING: Organisasies bestaan om skaars produksiefaktore te verander na bemarkbare goedere en dienste. Aangesien organisasies deur mense bedryf en bestuur word, is hierdie instellings grotendeels afhanklik van hul menslike produksiefaktor om hul hoofdoel te bereik, nl. om hul wins te vergroot. Organisasies poog om geskikte werknemers aan te stel wat sal voldoen aan die vereistes van 'n spesifieke pos of dit selfs sal oortref. In 'n werkswêreld wat konstant verander, vereis tegnologie en die kenmerke van die werkswêreld dat hierdie persone deurgaans ontwikkel word om by te bly met verandering. Personeelkeuring en –ontwikkeling is dus kritieke bedrywighede van die Bedryfsielkundige en Menslike Hulpbronpraktisyn. Die Takseersentrum is 'n gewilde meetinstrument wat dikwels gebruik word vir die doel van keuring of ontwikkeling. Hierdie gewilde assesseringsmetode word hoog aangeskryf vir sy vermoë om toekomstige werksprestasie te voorspel. Takseersentrums wat gebruik word vir keuring doeleindes, toon inkrementele geldigheid bo meetinstrumente van persoonlikheid sowel as kognitiewe vaardigheidstoetse. Al word takseersentrums internasionaal en hier in Suid-Afrika dikwels gebruik, word hulle ook dikwels gekritiseer op grond van die vraag of hulle werklik die dimensies meet wat hulle veronderstel is om te meet. Die konstrukgeldigheid van takseersentrums word dikwels bevraagteken aangesien lae diskriminante en konvergerende geldigheid, sowel as hardnekkige oefeningseffekte, navorsingsbevindinge oorheers. Hierdie vraag is die beweegrede vir die huidige studie. Die doel met hierdie studie is om die konstrukgeldigheid van 'n ontwikkelingstakseersentrum te ondersoek. 'n Geriefsteekproef is gebruik om die navorsing te doen. Die data is verskaf deur 'n private konsultasie maatskappy in die vorm van die takseersentrumtellings van 202 persone wat in 'n eendaagse sentrum geassesseer is. Die sentrum is ontwikkel vir 'n Suid-Afrikaanse bankinstelling en het drie hoofdoelwitte, nl. om kandidate te identifiseer vir die rol van 'n nuwe posbeskrywing, om werknemers na meer topaslike rolle te verskuif en om toekomstige ontwikkelingsgeleenthede vir alle deelnemers te verskaf. Twaalf vaardighede is deur vier verskillende oefeninge geëvalueer. Verskeie beperkinge is opgelê deur die aard van die geriefsteekproef deurdat die navorser geen invloed op die ontwerp van die takseersentrum gehad het nie. Die aanvanklike twaalf vaardighede kon nie afsonderlik ontleed word nie en moes gevolglik as subdimensies in hul onderskeie globale faktore gekombineer word. Dit het gelei tot vier enkeldimensie (ED) metingsmodelle wat eers ondersoek moes word om gesigswaarde van konstrukgeldigheid te bewys voordat oefeninge by die bestaande modelle gevoeg kon word. Die vier afsonderlike oefeninge is in een globale oefeningseffek saamgevoeg. As gevolg van die ontoereikende getal indikators in die datastel kon net twee van die vier ED-modelle oefeninge insluit en dit het gelei tot twee enkeldimensie-, enkeloefening-metingsmodelle (EDEO). Inter-itemkorrelatsies is in SPSS bereken, gevolg deur bevestigende faktorontleding van elke afsonderlike metingsmodel in EQS wat gebruik is om die interne struktuur van die dimensies te bestudeer. Met een dimensie as uitsondering, impliseer die uitslae van die CFA dat die indikators van die takseersentrum (d.w.s. gedragsbeoordelings) nie daarin slaag om die onderliggende dimensie te weerspieël soos dit veronderstel was om te doen nie. Nadat die saamgestelde oefeningseffek byvoeg is, het slegs een van die twee dimensies geloofwaardige uitslae met buitengewoon goeie modelpassing en parameterskattings wat dui op dimensie- eerder as oefeningseffekte. As gevolg van hierdie bevindings word die geldigheid van die ontwikkelingsterugvoer wat elke deelnemer na die evaluering ontvang het, ernstig in twyfel getrek. Met die uitsondering van een dimensie se resultate, bevestig die resultate van hierdie studie vorige navorsingsbevindinge.

ENGLISH ABSTRACT: Organisations exist by transforming scarce factors of production into goods and services. Since organisations are run and managed by people, these institutions are largely dependent on their human production factor to achieve their main goal of maximising profits. Organisations strive to appoint suitable employees who will meet, even exceed, the requirements of a particular job position. In a constantly evolving world of work, advancing technology and inherent features of the modern working environment necessitate ongoing development of these individuals in order to keep up with the changes. Personnel selection and development are therefore crucial activities of the Industrial Psychologist and Human Resource Practitioner. The Assessment Centre (AC) is a popular measuring instrument that is often used for either selection or development purposes. This popular method of assessment has received a great degree of praise for its ability to predict future job performance. ACs have also shown incremental validity over and above both personality and cognitive ability measuring instruments when used for selection purposes. Nevertheless, despite the frequent use of ACs both internationally and locally in South Africa, ACs have been widely criticised on the basis of whether they actually measure the dimensions that they intend to measure. The question has often been asked whether ACs are construct valid, since low discriminant- and convergent validity, as well as persistent exercise effects, seem to dominate research findings. This question serves as the driving force of the present study. The aim of this study is to examine the construct validity of a development assessment centre (DAC). A convenience sample was used to pursue the research objective. The data was received from a private consultant company in the form of the AC ratings of 202 individuals who were assessed in a one-day DAC. The DAC was developed for a South African banking institution and had three main purposes, namely to identify candidates who fit the role of a new job position, to reposition employees into more appropriate roles, and to provide future development opportunities to all participants. Twelve competencies were assessed by four different exercises. Several limitations were imposed by the nature of the convenience sample since the researcher did not have an influence on the design of the AC. The initial twelve competencies were not represented by a sufficient number of indicators and could consequently not be statistically analysed on an individual level. These dimensions therefore had to be used as sub-dimensions to be combined within their respective global (second-order) factors. This resulted in four single trait (ST) measurement models that had to be investigated first to provide face value of construct validity before adding exercises into the existing models. The four separate exercises were integrated into one global exercise effect. The insufficient number of indicators within the data set brought about only two of the four ST models to be examined for any existing exercise effects. The result was two single trait, single exercise (STSE) measurement models. Inter-item correlations were calculated in SPSS, followed by confirmatory factor analysis on each respective measurement model in EQS used to study the internal structure of the dimensions. With one dimension as the exception, the results of the CFA imply that the DAC's indicators (i.e. behavioural ratings) in each second-order factor, fail to reflect the underlying dimension, as it was intended to do. When adding the conglomerated exercise effect, only one of the two dimensions had plausible results with good model fit and parameter estimates that leaned towards dimension and not exercise effects. Based on these findings, serious doubt is placed on the validity of the developmental feedback provided to each participant after the completion of the DAC. With one dimension as the exception, the present study's results corroborate previous research findings on the construct validity of ACs.

Please refer to this item in SUNScholar by using the following persistent URL: http://hdl.handle.net/10019.1/18071
This item appears in the following collections: