Presentation

Do students rapidly guess repeatedly over time? A longitudinal analysis of student test disengagement, background, and attitudes

September 2020

By: James Soland, Steven Wise

Description

This session from the 2020 National Council on Measurement in Education virtual conference presents new research findings on understanding and managing test-taking disengagement. (Presentation begins at 22:55).

Considerable research has examined the use of rapid guessing measures to identify disengaged item responses.Ā  However, little is known about students who rapidly guess over the course of several tests.Ā  In this study, we use achievement test data from six administrations over three years to investigate whether rapid guessing is a stable trait-like behavior or if rapid guessing is determined mostly by situational variables.Ā  We do so using a Latent State Trait (LST) model (Prenoveau et al., 2011) displayed in Figure 1.Ā  In these models, the relative standing of an individual relative to her peers at each time point is decomposed into a trait factor that is free of situational or interactional effects and an occasion factor capturing the situational or interactional component. If an individualā€™s rank ordering is entirely consistent over time, then the variance of the observed variables is completely attributable to the trait factor. Within such models, we also examine whether rapid guessing over the course of several tests is associated with certain psychological measures, such as academic self-efficacy survey scores, and background variables like English learner status.

We find that rapid guessing tends to be fairly state-like compared to academic achievement scores, which are fairly stable.Ā  Further, we show that repeated rapid guessing is strongly associated with studentsā€™ academic self-efficacy and self-management scores.Ā  These findings have implications for detecting rapid guessing and intervening to reduce its effect on observed achievement test scores.

See More
See the presentation

Related Topics