Measurement & scaling


What happens when test takers disengage? Understanding and addressing rapid guessing
How does rapid-guessing differ from solution behavior? Research provides insight into test disengagement and how disengagement should be managed in scoring.
By: Steven Wise, Megan Kuhfeld


Reconciling long-term education policy goals with short-term school accountability models
Schools are increasingly held accountable for their contributions to studentsā academic growth in math and reading. Under The Every Student Succeeds Act, most states are estimating how much schools improve student achievement over time and using those growth metrics to identify the bottom 5% of schools for remediation.
Topics: Measurement & scaling, Growth modeling, Student growth & accountability policies


In this study, we introduce those disengagement metrics for a policy and evaluation audience, including how disengagement might bias estimates of educational effectiveness. Analytically, we use data from a state administering a computer-based test to examine the effect of test disengagement on estimates of school contributions to student growth, achievement gaps, and summer learning loss.
By: Megan Kuhfeld, James Soland
Topics: Measurement & scaling, School & test engagement, Student growth & accountability policies


When computer-based tests are used, disengagement can be detected through occurrences of rapid-guessing behavior. This empirical study investigated the impact of a new effort monitoring feature that can detect rapid guessing, as it occurs, and notify proctors that a test taker has become disengaged.
By: Steven Wise, Megan Kuhfeld, James Soland
Topics: Measurement & scaling, Innovations in reporting & assessment, School & test engagement


Identifying disengaged survey responses: New evidence using response time metadata
In this study, we condition results from a variety of detection methods used to identify disengaged survey responses on response times. We then show how this conditional approach may be useful in identifying where to set response time thresholds for survey items, as well as in avoiding misclassification when using other detection methods.
By: James Soland, Steven Wise, Lingyun Gao


This paper briefly discusses the trade-offs involved in making such a transition, and then focuses on a relatively unexplored benefit of computer-based tests ā the control of construct-irrelevant factors that can threaten test score validity.
By: Steven Wise
Topics: Measurement & scaling, Innovations in reporting & assessment, School & test engagement


Predicting time to reclassification for English learners: A joint modeling approach
The development of academic English proficiency and the time it takes to reclassify to fluent English proficient status are key issues in English learner (EL) policy. This article develops a shared random effects model (SREM) to estimate English proficiency development and time to reclassification simultaneously, treating student-specific random effects as latent covariates in the time to reclassification model.
By: Tyler Matta, James Soland