Publication Date
10-1-1981
Abstract
In tailored testing there are two commonly used methods of operation. These two methods on a maximum likelihood ability estimation procedure and a Bayesian ability estimation procedure (Owen, 1975). The first procedure estimates a subject's ability after each item using an empirical maximum likelihood technique. The ability estimate is then used to select the next item in such a way that the item information is maximized at that ability level (Birnbaum, 1968). With the second procedure, ability is estimated as the mean of the posterior ability distribution and items are selected to minimize the posterior variance of the ability estimate distribution, while assuming a normal prior distribution of ability. As these two methods of operation are substantially different, it is important to examine the quality of results from these two tailored testing procedures in order to make an educated decision in choosing which procedure to implement.
This research compared the two procedures on the basis of obtained ability estimates, obtained total test information, and reliability. However, since tailored tests need not be fixed in length, the first step in this research was to determine the optimal test length for each procedure. This is important because Reckase (1974) had found that continued testing beyond the point at which the ability estimate stabilizes may introduce bias in the ability estimate. This occurs because most of the appropriate items from the item pool for that ability level have been used and only inappropriate items are available. The determination of the appropriate test length was accomplished by using information and posterior variance. Once the test lengths for the two procedures had been obtained the ability estimates yielded by the two procedures at those lengths were compared. Also the total test information yielded by the two procedures were compared. The purpose of this study was to determine if a clearly preferred procedure would emerge, thus making the selection of a tailored testing system that much easier.
Recommended Citation
Rosso, Martin A. and Reckase, Mark D.
(1981)
"A Comparison of Maximum Likelihood and a Bayesian Ability Estimation Procedure for Tailored Testing,"
Mid-Western Educational Researcher (1981 - 1987): Vol. 2:
Iss.
3, Article 3.
Available at:
https://scholarworks.bgsu.edu/mwer_archival/vol2/iss3/3
