School & test engagement
Educators need accurate assessment data to help students learn. But when students rapid-guess or otherwise disengage on tests the validity of scores can be affected. Our research examines the causes of test disengagement, how it relates to students’ overall academic engagement, and its impacts on individual test scores. We look at its effects on aggregated metrics used for school and teacher evaluations, achievement gap studies, and more. This research also explores better ways to measure and improve engagement and to help ensure that test scores more accurately reflect what students know and can do.
The relationship between test-taking disengagement and performance on MAP Growth retests
Educators sometimes ask: do students rapidly guess because they don’t know the answer to a question, or do rapid guesses reflect a lack of engagement with the test? Would a student’s scores improve if that student engaged more with the assessment and rapidly guessed on fewer items? Examining MAP® Growth™ test scores and levels of student test engagement for over 100,000 tests for which students retested within one day, the results showed that students’ test taking engagement often differed between the initial test and the retest.
Topics: School & test engagement
What happens when test takers disengage? Understanding and addressing rapid guessing
How does rapid-guessing differ from solution behavior? Research provides insight into test disengagement and how disengagement should be managed in scoring.
The impact of proctor notification when students disengage
How do you help make sure students give their best effort on testing day? New research on student test engagement shows how proctor notification can make a significant impact.
Topics: Measurement & scaling, Innovations in reporting & assessment, School & test engagement
This study uses an analytic example to explore whether metadata might help illuminate such constructs. Specifically, analyses examine whether the amount of time students spend on test items (after accounting for item difficulty and estimates of true achievement), and difficult items in particular, tell us anything about the student’s academic motivation and self‐efficacy.
Topics: School & test engagement, Math & STEM, Social-emotional learning
In this study, we examine the impact of two techniques to account for test disengagement—(a) removing unengaged test takers from the sample and (b) adjusting test scores to remove rapidly guessed items—on estimates of school contributions to student growth, achievement gaps, and summer learning loss.
Topics: Measurement & scaling, School & test engagement, Student growth & accountability policies
The (non)impact of differential test taker engagement on aggregated scores
Disengaged test taking tends to be most prevalent with low-stakes tests. This has led to questions about the validity of aggregated scores from large-scale international assessments such as PISA and TIMSS, as previous research has found a meaningful correlation between the mean engagement and mean performance of countries.
An information-based approach to identifying rapid-guessing thresholds
Although several common threshold methods are based on rapid guessing response accuracy or visual inspection of response time distributions, this paper describes a new information-based approach to setting thresholds that does not share the limitations of other methods.
