Working paper
The relationship between test-taking disengagement and performance on MAP Growth retests
2019
Description
Educators sometimes ask: do students rapidly guess because they don’t know the answer to a question, or do rapid guesses reflect a lack of engagement with the test? Would a student’s scores improve if that student engaged more with the assessment and rapidly guessed on fewer items? Examining MAP® Growth™ test scores and levels of student test engagement for over 100,000 tests for which students retested within one day, the results showed that students’ test taking engagement often differed between the initial test and the retest. Students who rapid-guessed less on retest tended to see score gains, sometimes greater than 10 RIT points, with the largest gains seen for students who initially showed the highest levels of rapid guessing. Students who rapid-guessed more on retest, in contrast, generally showed score declines. The findings provide evidence that that rapid guesses reflect a lack of engagement rather than a lack of knowledge, and support NWEA’s recommendation that educators should retest disengaged students. The findings also underscore the importance of increasing the student’s motivation before retesting to improve engagement and suggest that educators should be cautious if they do not retest students who show high levels of rapid guessing, since these scores pose a threat to the validity of the test score interpretations.
See MoreTopics: School & test engagement
Related Topics
This study compared the test taking disengagement of students taking a remotely administered an adaptive interim assessment in spring 2020 with their disengagement on the assessment administered in-school during fall 2019.
Topics: Equity, Innovations in reporting & assessment, School & test engagement
Six insights regarding test-taking disengagement
There has been increasing concern about the presence of disengaged test taking in international assessment programs and its implications for the validity of inferences made regarding a country’s level of educational attainment. In this paper, the author discusses six important insights yielded by 20 years of research on this and implications for assessment programs.
The impact of disengaged test taking on a state’s accountability test results
This study investigated test-taking engagement on a large-scale state summative assessment. Overall, results of this study indicate that disengagement has a material impact on individual state summative test scores, though its impact on score aggregations may be relatively minor.
Topics: Equity, Measurement & scaling, School & test engagement
Variation in respondent speed and its implications: Evidence from an adaptive testing scenario
The more frequent collection of response time data is leading to an increased need for an understanding of how such data can be included in measurement models. Models for response time have been advanced, but relatively limited large-scale empirical investigations have been conducted. We take advantage of a large data set from the adaptive NWEA MAP Growth Reading Assessment to shed light on emergent features of response time behavior.
Topics: School & test engagement, Innovations in reporting & assessment
A method for identifying partial test-taking engagement
This paper describes a method for identifying partial engagement and provides validation evidence to support its use and interpretation. When test events indicate the presence of partial engagement, effort-moderated scores should be interpreted cautiously.
Topics: Measurement & scaling, Innovations in reporting & assessment, School & test engagement
This study uses reading test scores from over 728,923 3rd–8th-grade students in 2,056 schools across the US to compare threshold-setting methods to detect noneffortful item responses. and so helps provide guidance on the tradeoffs involved in using a given method to identify noneffortful responses.
Topics: School & test engagement
A method for identifying partial test-taking engagement
This paper describes a method for identifying partial engagement and provides validation evidence to support its use and interpretation. When test events indicate the presence of partial engagement, effort-moderated scores should be interpreted cautiously.
Topics: School & test engagement
