Measuring the Quality of Classification Decisions

ReResearch Methods for Real Peoplesearch Methods for Real PeopleAs a Research Scientist at NWEA, I do research on the statistical methods and techniques that are used in educational research. My background is very technical and most of my day is spent wrestling with computer code. By developing, evaluating, and implementing advanced statistical methods, my work helps keep NWEA at the cutting edge of technology and contributes to the field’s overall body of work.

The true value of methodological research, however, is its positive impact on people actually doing educational research. When a new method is developed, the two main barriers to people actually using the method are:

(1) Lack of understanding of the method due to an overly technical presentation, and

(2) Not having an accessible implementation of the method

The solutions to these barriers involve presenting the statistical method in an approachable way and by providing software implementations. The ultimate goal is to guide educational researchers, who may not be comfortable with all the technical details, towards effective and appropriate use of the method.

To this end, I have recently published a paper in Practical Assessment, Research, and Evaluation, a peer-reviewed on-line journal, that non-technically introduces two methods to estimate Classification Accuracy and Consistency. Most assessments are used to classify examinees into categories (such as Pass and Fail). Classification Accuracy and Consistency provide a simple way to communicate the quality of these classification decisions. The article explains the practical differences in the methods and demonstrates their implementation with a common dataset with freely available computer software. You can read the full article here.

I’d love to hear from you. Please, reach out via Facebook or Twitter @NWEA.