Is teacher value added a matter of scale? The practical consequences of treating an ordinal scale as interval for estimation of teacher effects
Applied Measurement in Education, 30(1), 52–70.
Research shows that assuming a test scale is equal-interval can be problematic, especially when the assessment is being used to achieve a policy aim like evaluating growth over time. However, little research considers whether teacher value added is sensitive to the underlying test scale, and in particular whether treating an ordinal scale as interval might lead to erroneous conclusions about teacher quality. This article addresses the issue by estimating teacher value added, then applying extremely mild nonlinear transformations to the original scale and re-estimating the value added. Although by definition at most one of these scales can be equal-interval, all are treated as if interval-scaled when estimating value added. Results show that value added is sensitive to the scale used. While rank orderings of teachers do not change radically by transformation, even mild departures from the original scale can change a teacher’s odds of being considered high- or low-performing by a factor of 5.See More
This article was published outside of NWEA. The full text can be found at the link above.