School & test engagement
Educators need accurate assessment data to help students learn. But when students rapid-guess or otherwise disengage on tests the validity of scores can be affected. Our research examines the causes of test disengagement, how it relates to students’ overall academic engagement, and its impacts on individual test scores. We look at its effects on aggregated metrics used for school and teacher evaluations, achievement gap studies, and more. This research also explores better ways to measure and improve engagement and to help ensure that test scores more accurately reflect what students know and can do.
The impact of test-taking disengagement on item content representation
Rapid-guessing can distort test scores and adversely affect measurement. New research shows how disengaged responses can also distort content representation.
Topics: Measurement & scaling, Innovations in reporting & assessment, School & test engagement
Validation and applications of rapid guessing to detect test taker disengagement
The International Association for the Evaluation of Educational Achievement (IEA) together with the Leibniz Institute for Research and Information in Education (DIPF) and the Centre for International Student Assessment (ZIB) offer an invitation to a three-day workshop on analyzing log file and process data from international large-scale assessments in education. The event will take place as an interactive webinar and online video conference on June 17th – 19th 2020.
Topics: School & test engagement
The impact of test-taking disengagement on item content representation
Rapid-guessing can distort test scores and adversely affect measurement. New research shows how disengaged responses can also distort content representation.
Topics: Measurement & scaling, Innovations in reporting & assessment, School & test engagement
This study investigates whether rapid guessing is a stable trait-like behavior or if rapid guessing is determined mostly by situational variables, and whether rapid guessing over the course of several tests is associated with certain psychological and background measures. We find that rapid guessing tends to be more state-like compared to academic achievement scores, which are fairly stable and that repeated rapid guessing is strongly associated with students’ academic self-efficacy and self-management scores.
Topics: Measurement & scaling, School & test engagement, Social-emotional learning
Are achievement gap estimates biased by differential student test effort?
New research shows that test effort differs substantially across student gender and racial subgroups. What does this mean for achievement gap estimates?
Topics: Equity, School & test engagement, Social-emotional learning
The relationship between test-taking disengagement and performance on MAP Growth retests
Educators sometimes ask: do students rapidly guess because they don’t know the answer to a question, or do rapid guesses reflect a lack of engagement with the test? Would a student’s scores improve if that student engaged more with the assessment and rapidly guessed on fewer items? Examining MAP® Growth™ test scores and levels of student test engagement for over 100,000 tests for which students retested within one day, the results showed that students’ test taking engagement often differed between the initial test and the retest.
Topics: School & test engagement
In this study, we introduce those disengagement metrics for a policy and evaluation audience, including how disengagement might bias estimates of educational effectiveness. Analytically, we use data from a state administering a computer-based test to examine the effect of test disengagement on estimates of school contributions to student growth, achievement gaps, and summer learning loss.
Topics: Measurement & scaling, School & test engagement, Student growth & accountability policies
