Although researchers often assume their participants are naive to experimental materials, this is not always the case. We investigated how prior exposure to a task affects subsequent experimental results. Participants in this study completed the same set of 12 experimental tasks at two points in time, first as a part of the Many Labs replication project and again a few days, a week, or a month later. Effect sizes were markedly lower in the second wave than in the first. The reduction was most pronounced when participants were assigned to a different condition in the second wave. We discuss the methodological implications of these findings.

effect sizes, judgment and decision making, nonnaïveté, open data, open materials, panel conditioning, repeated participation, research methods,
ERIM Top-Core Articles
Psychological Science
Rotterdam School of Management (RSM), Erasmus University

Chandler, J, Paolacci, G, Peer, E, Mueller, P, & Ratliff, K.A. (2015). Using Nonnaive Participants Can Reduce Effect Sizes. Psychological Science, 26(7), 1131–1139. doi:10.1177/0956797615585115