Loglinear smoothing for the latent trait distribution: A two-tiered evaluation
Abstract
This research focused on two main problems. The first problem concerns the use of loglinear smoothing for estimating the latent trait distribution, and how this may improve the recovery of item response theory (IRT) item parameters in marginal maximum likelihood estimation. The second problem concerns the robustness of applications of IRT item parameters to the error that occurs in item parameter estimation as a result of the misspecification of the latent trait distribution. To examine these problems, a two-stage Monte Carlo simulation study was conducted containing two tiers of evaluation. Tier I included an item parameter recovery study, which compared the use of a fixed normal specification, a nonparametric model, and loglinear smoothing models in item parameter recovery for three populations: normal, negatively skewed, and bimodal. The study design included a separate evaluation of this problem for the 1PL and 3PL item response models, for N=1,000 and a 50 item test, with the number of moments matched to the latent trait distribution, as well as the number of quadrature points used, as two varied factors. Tier II was a robustness study in which the item parameter estimates from Tier I were utilized to estimate conditional standard errors of measurement (CSEMs) and IRT true score equating functions. Estimation error that was manifested in Tier II was linked to error from Tier I using a series of analytic tools, including exploratory graphical analysis, Ward’s cluster analyses, and a qualitative case-study approach. Tier I results showed that there are instances where loglinear smoothing outperforms the nonparametric model, and that loglinear smoothing models matching 4 or 5 moments appear to be optimal in most cases. In addition, it is important to use an adequate number of quadrature points. For 50 items, 31 was optimal. There are diminishing returns associated with using many moments and/or many quadrature points. Tier II results showed a definite connection between errors at Tiers I and II. Error manifested in the CSEMs was significantly less than the error manifested in equating functions. Robustness criteria were suggested for use in item parameter recovery research. Root mean square errors (RMSEs) greater than 0.35 for a parameters, 0.15 for b parameters, and 0.06 for c parameters are associated with substantial error in CSEMs and equating functions.
Subject Area
Educational tests & measurements|Statistics
Recommended Citation
Casabianca, Jodi M, "Loglinear smoothing for the latent trait distribution: A two-tiered evaluation" (2011). ETD Collection for Fordham University. AAI3474125.
https://research.library.fordham.edu/dissertations/AAI3474125
