Assessing item-level fit for the sequential G-DINA model

Abstract

.

Cognitive Diagnostic Models (CDMs) categorize examinees into latent classes with distinct profiles of attribute mastery based on their responses to the test items. Most of these models have traditionally focused on dichotomous responses (i.e., correct vs. incorrect) from multiple-choice items. The sequential process model (SPM) extends the applicability of CDM by modeling sequentially graded responses that stem from different item formats (e.g., constructed-response items). The SPM is flexible enough to accommodate a different response function for each category, which can also measure a distinct set of attributes. Empirical Q-matrix validation and category-level model selection have been already proposed for the SPM, which help identify the best specification for each category. However, absolute item-level fit has not been addressed for the SPM yet, preventing from determining whether such specifications are appropriate. The present study adapts three well-known item-fit statistics to the SPM and evaluates their performance through a simulation study. The results show that the statistics are usually conservative, but have an adequate power at detecting relevant misspecifications (i.e., those that provoke a substantial disruption in item parameter estimation). Practical implications and recommendations are provided.