Criterion-related Validity of Forced-Choice Personality Measures: A Cautionary Note Regarding Thurstonian IRT versus Classical Test Theory Scoring

Authors

Peter A. Fisher, Wilfrid Laurier UniversityFollow
Chet Robie, Wilfrid Laurier UniversityFollow
Neil D. Christiansen, Central Michigan UniversityFollow
Andrew B. Speer, Wayne State UniversityFollow
Leann Schneider, Plum.io, inc.Follow

DOI

https://doi.org/10.25035/pad.2019.01.003

Abstract

This study examined criterion-related validity for job-related composites of forced-choice personality scores against job performance using both Thurstonian Item Response Theory (TIRT) and Classical Test Theory (CTT) scoring methods. Correlations were computed across 11 different samples that differed in job or role within a job. A meta-analysis of the correlations (k = 11 and N = 613) found a higher average corrected correlation for CTT (mean ρ = .38) than for TIRT (mean ρ = .00). Implications and directions for future research are discussed.

Recommended Citation

Fisher, Peter A.; Robie, Chet; Christiansen, Neil D.; Speer, Andrew B.; and Schneider, Leann (2019) "Criterion-related Validity of Forced-Choice Personality Measures: A Cautionary Note Regarding Thurstonian IRT versus Classical Test Theory Scoring," Personnel Assessment and Decisions: Number 5 : Iss. 1 , Article 3.
DOI: https://doi.org/10.25035/pad.2019.01.003
Available at: https://scholarworks.bgsu.edu/pad/vol5/iss1/3

Corresponding Author Information

Peter Fisher

fish0150@mylaurier.ca