MAP® Growth™ reporting has added two new student assessment engagement metrics:

• Percent of Disengaged Responses
• Estimated Impact of Disengagement on RIT

These metrics will inform educators about what percentage of items from a student’s test event were rapidly guessed, and what the estimated impact of that rapid guessing was on a student’s final RIT score. When combined with the notification system that alerts proctors when students are disengaged, these features should provide educators with actionable information about a student’s overall engagement during a test.

Here’s a quick snapshot of these two new metrics…

Percent of Disengaged Responses

When a student responds to a test item, the amount of time he or she took to respond can be compared to the response times from other students who answered the same item. This comparison allows us to determine if the student responded so quickly that he or she could not have reasonably read the question, considered the content, and attempted to correctly answer the question.

When this occurs, the student’s response is characterized as a rapid guess, or a “disengaged response.” Too many disengaged responses can impact the validity of a student’s test score. In general, the more disengaged responses in a test event, the greater the risk to test score validity.

For students who don’t have any disengaged responses, their report will currently show an “N/A” to indicate that they did not rapidly guess on test items. If students provide disengaged responses on less than 10 percent of items on a test, the impact on test score validity is generally minimal. If a student rapidly guesses on more than 10 percent of test items, retesting may be necessary. This decision can be informed by how much this disengagement has affected the student’s RIT score, which is the next new metric we’ll highlight.

Estimated Impact of Disengagement on RIT

This new metric quantifies the estimated impact of disengaged responses on a student’s score by re-estimating a student’s RIT score with all disengaged item responses removed, and calculating how much this estimate differs from the student’s demonstrated score based on all responses. This difference represents the estimated impact of disengagement on RIT.

If a student shows disengaged responses on 30 percent or more of test items, we recommend that the student be retested. This is because when this much rapid guessing occurs, it severely limits our ability to estimate a credible score for a student. The decision on whether or not to retest a student when the proportion of disengaged responses is between 10 percent and 30 percent is a matter of professional judgment for educators. In the guidance document below, we share some examples that may help you determine whether or not to retest students in this range. For additional NWEA recommendations and details on these two new MAP Growth reporting metrics, download our research report – NWEA Guidance on the Interpretation and Use of New Engagement Metrics in MAP Growth Reports.