An option-based partial credit item response model
Abstract
Multiple-choice (MC) tests have been criticized for allowing guessing and the failure to credit partial knowledge, and alternative scoring methods and response formats (Ben Simon et al. 1997) have been proposed to address this problem. Modern test theory addresses these issues by using binary item response models (e.g., 3PL) with guessing parameters, or with polytomous IRT models. This dissertation introduces two new option-based partial credit models accompanied by a MC scoring rule. The two proposed models are an option-based partial credit IRT model and a generalized option-based partial credit model. The new scoring rule is based on a weighted Hamming distance between the option key and the option response vector. The test taker (TT)'s estimated ability is based on information from both correct options and distracters. These modifications reduce the TT's ability to guess and credit the TT's partial knowledge. The new models can be tailored to different formats, and some popular IRT models, such as the 2PL and Bock's nominal model, are special cases of the proposed model. The dissertation confirms that the option local independence model is the best model in fitting two GRE data sets, and that weighted Hamming distance scoring can provide partial credit to TTs. Simulation studies show that the benefit of weighted Hamming distance scoring is more obvious after fully excluding the effect of single selection MC items in the parameter estimation of multiple selection MC items. In supplementary results, an analysis of PISA data shows that the grouped Number Right scores are more nearly linearly and monotonically related to the underlying latent traits than the weighted Hamming distance scores. However, the weighted Hamming distance scores can provide partial credit, which is reflected in the form of a nonlinear monotone relationship between the weighted Hamming distance scores and the underlying latent traits. Low ability students benefited more from the weighted Hamming distance scores than the high ability students.
Subject Area
Statistics|Quantitative psychology
Recommended Citation
Bo, Yuanchao, "An option-based partial credit item response model" (2014). ETD Collection for Fordham University. AAI3715912.
https://research.library.fordham.edu/dissertations/AAI3715912
