Recently, Brummelman, Thomaes, Orobio de Castro, Overbeek, and Bushman (2014: Study 3) demonstrated that inflated praise benefits challenge seeking of children with high self-esteem, but harms challenge seeking of children with low self-esteem. In the present paper, we examined the original data set on model-fit and prediction outliers according to various reasonable criteria and norms. Subsequently, we carried out a multiverse outlier re-analysis on the data of Brummelman and colleagues’ Study 3, employing the same analytical approach as the original authors did but excluding outliers. Out of the twelve re-analyses in the multiverse, six demonstrated that removing only a small number of outliers rendered the originally reported crucial interaction effect between self-esteem and type of praise non-significant and produced a sizeable reduction of the effect size. The present paper illustrates the use of reporting outlier analyses, which lies in allowing a critical evaluation of the empirical evidence and offering a more complete picture that enhances future studies in the field.

Additional Metadata
Keywords Challenge seeking, Inflated praise, Multiverse analysis, Outlier analysis, Self-esteem
Persistent URL dx.doi.org/10.1525/collabra.118, hdl.handle.net/1765/119603
Journal Collabra: Psychology
Citation
Verkoeijen, P.P.J.L, Polak, M.G. (Marike G.), & Bouwmeester, S. (2018). A practical illustration of methods to deal with potential outliers: A multiverse outlier analysis of study 3 from Brummelman, Thomaes, Orobio de Castro, Overbeek, and Bushman (2014). Collabra: Psychology, 4(1). doi:10.1525/collabra.118