The Machines Aren’t Taking Over (Yet): An Empirical Comparison of Traditional, Profiling, and Machine Learning Approaches to Criterion-Related Validation

Authors

Kristin S. Allen, SHLFollow
Mathijs Affourtit, SHLFollow
Craig M. ReddockFollow

DOI

https://doi.org/10.25035/pad.2020.03.002

Abstract

Criterion-related validation (CRV) studies are used to demonstrate the effectiveness of selection procedures. However, traditional CRV studies require significant investment of time and resources, as well as large sample sizes, which often create practical challenges. New techniques, which use machine learning to develop classification models from limited amounts of data, have emerged as a more efficient alternative. This study empirically investigates the effectiveness of traditional CRV with a variety of profiling approaches and machine learning techniques using repeated cross-validation. Results show that the traditional approach generally performs best both in terms of predicting performance and larger group differences between candidates identified as top or non-top performers. In addition to empirical effectiveness, other practical implications are discussed.

Recommended Citation

Allen, Kristin S.; Affourtit, Mathijs; and Reddock, Craig M. (2020) "The Machines Aren’t Taking Over (Yet): An Empirical Comparison of Traditional, Profiling, and Machine Learning Approaches to Criterion-Related Validation," Personnel Assessment and Decisions: Number 6 : Iss. 3 , Article 2.
DOI: https://doi.org/10.25035/pad.2020.03.002
Available at: https://scholarworks.bgsu.edu/pad/vol6/iss3/2

Corresponding Author Information

Kristin S. Allen

Kristin.Allen@shl.com