The selection of a method for estimating the reliability of ratings has considerable implications for the use of assessments in personnel selection. In particular, the accuracy of corrections to validity coefficients for unreliability and test score bands are completely dependent on the correct estimation of the reliability. In this paper, we discuss how generalizability theory can be used to estimate reliability for test score bands with assessments involving ratings. Using selection data from a municipal entity, we demonstrate the use of generalizability theory-based compare the implications of its use in test score banding compared to the traditional approach.
Scherbaum, Charles; Dickson, Marcus; Larson, Elliott; Bellenger, Brian; Yusko, Kenneth; and Goldstein, Harold
"Creating Test Score Bands for Assessments Involving Ratings using a Generalizability Theory Approach to Reliability Estimation,"
Personnel Assessment and Decisions: Vol. 4
, Article 1.
Available at: https://scholarworks.bgsu.edu/pad/vol4/iss1/1