Innovations in reporting & assessment
Emerging technologies allow for a variety of methods to assess students and report data specific to the needs of different stakeholders. These various approaches can result in assessments that are more engaging for students, along with reporting that provides more insightful, useful information for students, families, and educators.
Variation in respondent speed and its implications: Evidence from an adaptive testing scenario
The more frequent collection of response time data is leading to an increased need for an understanding of how such data can be included in measurement models. Models for response time have been advanced, but relatively limited large-scale empirical investigations have been conducted. We take advantage of a large data set from the adaptive NWEA MAP Growth Reading Assessment to shed light on emergent features of response time behavior.
By: Benjamin Domingue, Klint Kanopka, Ben Staug, James Soland, Megan Kuhfeld, Steven Wise, Chris Piech
Topics: School & test engagement, Innovations in reporting & assessment
A method for identifying partial test-taking engagement
This paper describes a method for identifying partial engagement and provides validation evidence to support its use and interpretation. When test events indicate the presence of partial engagement, effort-moderated scores should be interpreted cautiously.
By: Steven Wise, Megan Kuhfeld
Topics: Measurement & scaling, Innovations in reporting & assessment, School & test engagement
User guide for interpreting the NWEA state dashboard: Exploring the educational impacts of COVID-19
This guide explains the analyses, statistics, terms, and data included in the NWEA state dashboard, Exploring the Educational Impacts of COVID-19.
By: Greg King
Topics: COVID-19 & schools, Innovations in reporting & assessment
Using retest data to evaluate and improve effort-moderated scoring
This study investigated effortāmoderated (EāM) scoring, in which item responses classified as rapid guesses are identified and excluded from scoring, and its affect on score distortion from disengaged test taking.
By: Steven Wise, Megan Kuhfeld
Topics: Measurement & scaling, Innovations in reporting & assessment, School & test engagement
An intelligent CAT that can deal with disengaged test taking
This book presents varied applications of artificial intelligence (AI) in test development, including research and successful examples of using AI technology in automated item generation, automated test assembly, automated scoring, and computerized adaptive testing.
By: Steven Wise
Topics: Measurement & scaling, Innovations in reporting & assessment, School & test engagement
The impact of test-taking disengagement on item content representation
Rapid-guessing can distort test scores and adversely affect measurement. New research shows how disengaged responses can also distort content representation.
By: Steven Wise
Topics: Measurement & scaling, Innovations in reporting & assessment, School & test engagement
Predicting the difficulty and response time of multiple choice questions using transfer learning
This paper reports on whether transfer learning can improve the prediction of the difficulty and response time parameters for ā 18,000 multiple-choice questions from a high-stakes medical exam.
By: Kang Xue, Victoria Yaneva, Christopher Runyon, Peter Baldwin