Data Rule #1: Don’t Should on Yourself

Data Rule #1: Don’t Should on YourselfStudent growth is the expected outcome when using assessment data to inform instruction. Defining success metrics for students involves asking where a student should be in relation to state curriculum standards. Success metrics often also include reference to what is considered normal. For example, parents who want to assess their child’s height and weight development may consider what doctors and researchers indicate are normal child height and weight gain patterns, and assess their child’s growth against those trends.

Districts that use our Measures of Academic Progress® (MAP®) assessment often include the NWEA Normative Data as an additional reference for defining growth targets. The Normative Data is designed to show typical student performance on the MAP assessment for grades K-11, where typical is defined as the 50th percentile performance.

For example, the fifth grade reading norm for the fall testing term is a 205.7 RIT score, according to the 2015 norms. When comparing that data point to actual student performance, I often hear educators say that the 205.7 is where their students “should” be. I teach educators how to unpack that 205.7 during my workshops by translating the score to an instructional readiness skill, and comparing that skill to their state standards. More specifically, what reading skills does the state standard say a 5th grader should know? This comparison is crucial because the MAP assessment does not dictate what to teach, nor what a student should know.

The MAP assessment is designed to tell you what a student is ready to learn. In an ideal world, all students would enter their grade level with a readiness that includes mastery of skills and concepts taught in all the previous grades. However, the reality is that students bring their own unique learning experiences to the classroom. Some students will enter their grade level having already mastered the expected grade level standards, while others will have gaps in their learning and need to be brought up to grade level expectations.

So, does that 205.7 really tell you where a 5th grade “should” be in reading in the fall? Well, it depends. State standards reflect thinking on where students “should” be, so a best practice when using state or district standards along with the NWEA norms is to not “should” on yourself. Doing so can become messy. Here’s why:

Good is Personal.

  • There will always be high performers, low performers, and middle of the road performers on the MAP assessment. Knowing that the test is designed to indicate what a student is ready to learn, it becomes clear that the high performer will be ready to learn a different set of skills than the low performer. While
  • All growth can be deemed as good, growth for the high and low performer will be different.

“Should” is Relative.

  • The MAP is an adaptive assessment with questions that are targeted toward a student’s achievement level. The test differs for each student because the questions adapt to the student based on their answers, with correct answers yielding higher level questions and incorrect answers yielding lower level questions.  Each student has a different testing experience, so defining their “should” could easily become a matter of placing judgment on a student’s level of readiness.

So, “Who’s your data?” That question was meant to bring humor to a data analysis discussion by a participant in a workshop I recently facilitated, yet it reminded me that in this age of data-based decision making, knowing what the data mean is critical in order to effectively apply data to instruction with fidelity and fairness.