Response time as an indicator of test taker speed: assumptions meet reality
Measurement: Interdisciplinary Research and Perspectives, 13(3–4), 186–188.
The growing presence of computer-based testing has brought with it the capability to routinely capture the time that test takers spend on individual test items. This, in turn, has led to an increased interest in potential applications of response time in measuring intellectual ability and achievement. Goldhammer (this issue) provides a very useful overview of much of the research in this area, and he provides a thoughtful analysis of the speed-ability trade-off and its impact on measurement. This author does not claim to understand why test takers tend to increase their time efficiency during MAP test events, although he notes that, having reanalyzed some of his own data, he had observed this phenomenon in data from 2 very different testing programs. How commonly this effect is present in test data is, for the time being, unclear. Whatever its nature, however, this increasing efficiency effect appears to (at least sometimes) represent an additional systematic factor influencing response time. And whenever this effect is present to a material degree, it threatens the validity of models that presume that item response time is systematically influenced only by item time intensity and the test taker’s ability and working speed. It confounds attempts to exploit a speed-ability trade-off, which is central to many of the models that Goldhammer discussed. If exhibited speed can change without a commensurate change in exhibited ability, the usefulness of response time in estimating a test taker’s ability is undermined. Hence, whenever an increased efficiency effect is present, it represents an inconvenient reality for response-time-based psychometric models.See More
This article was published outside of NWEA. The full text can be found at the link above.