Individual Score Validity: Using the Wariness Index to Identify Test Performance to Treat with Caution

This paper was presented at the 2011 NCME Annual Meeting in New Orleans, LA.

Test‐taker behaviors during the test should be considered part of the test‐taker’s performance. In this paper, an approach is offered in the form of a cautionary index first developed in 2010 and demonstrated with grade 4 mathematics CAT scores. The index is formed from the joint probability of observed responses under an IRT model and under a response time (RT) model. Details of the index, its salient operating characteristics, and its implementation demands are provided. Two empirical studies examine its parameter estimation procedures, extending the range of content and grade levels from the 2010 study, and the role that response correctness may play as the index evolves.


Resource Type

Focus Area

Recommended Audience