This study used cognitive pretesting to assess what factors could be causing score differences between ethnic majority and ethnic minority applicants on role-plays. In a laboratory room, equipped with a video camera, cognitive interviews were conducted among 12 ethnic majority and 12 ethnic minority students to examine ethnic differences in the interpretation of the instructional texts of two role-plays. Four trained assessors independently rated the participants' videotaped responses. The assessors were asked to indicate whether the participants had understood the instructional text in general, the intended meaning of specific words or phrases, the problem, the job role, and the context of the situation. Ethnic differences were found for almost all of these aspects of the instructional texts (d-values varied between -0.02 and 0.89), indicating that the texts often did not activate the same concepts in the ethnic minority group as in the ethnic majority group. Ethnic differences in verbal ability explained only part of the ethnic differences in the interpretation of the instruction of the role-plays. Cognitive pretesting seems to be a valuable method to assess differences between ethnic majority and ethnic minority applicants in how they interpret test items and instructional texts.

Additional Metadata
Keywords Adverse impact, Cognitive pretesting, Majority/minority differences, Personnel selection, Role-play
Persistent URL dx.doi.org/10.1016/j.ijintrel.2013.12.008, hdl.handle.net/1765/67245
Journal International Journal of Intercultural Relations
Citation
Oostrom, J.K, & Born, M.Ph. (2014). Using cognitive pretesting to explore causes for ethnic differences on role-plays. International Journal of Intercultural Relations, 41, 138–149. doi:10.1016/j.ijintrel.2013.12.008