Using Norms to Answer the “What’s Next” Question

Using Norms to Answer the “What’s Next” QuestionAs NWEA prepares to release 2015 norms this summer, I am exploring the thoughtful use of norms in a series of four posts. The first post focused on understanding what norms are and how to judge the quality and usefulness of normative data. The second post explained what status norms are, how to interpret them, and how they may be used to make decisions about students. Here I will explore how NWEA leverages our norms data to link to instructional content. I will also briefly cover two other ways we link to content. The final post will discuss growth norms.

NWEA makes a concerted effort to connect MAP and MPG scores to instructional content. We have done this to help teachers answer the “what next” question: I have this great data, how can I apply it in the classroom and how can I support students and parents working at home?

Right now there are three ways to connect to instructional content – and with the release of Skills Navigator, there will be four. Perhaps you are already familiar with these resources, but if not follow the links to get the details of each program.

I will discuss briefly how all three work at a high level and how they use the student’s current RIT score and NWEA norms to make connections.

Creating Content Links

To create links to content, we need to sort instructional content along two dimensions:

  • Difficulty as measured by RIT scores
  • Topic as defined by Learning Continuum and/or standards

Basically, we are asking “what is this content about and how hard is it to learn?”

RIT to Resource, MAP to Khan Academy and some ICPs look at RIT scores through a normative lens. NWEA publishes a table of status norms showing Beginning-of-Year, Middle-of-Year and End-of-Year Means for each subject and grade level (2011 example). NWEA and some content providers use these mean scores to define grade-level RIT ranges to link scores to content that is already aligned to grade levels. I want to emphasize that in making these estimations of grade ranges, we are making assumptions that can limit the precision of the results. However, we are only using the estimations to suggest the most likely instructional content a student is ready to work on independently. This sort of estimating grade ranges would never be appropriate for classifying students or making any sort of higher stakes decision than pointing students to content. In this specific context, our estimations have proven to be useful. In a pilot study with Study Island, one of our instructional content providers, teachers have found our links to content based on grade-level ranges to point students to appropriate content. We have had some preliminary positive feedback on MAP to Khan Academy. In all these applications of RIT scores to content, the creator of the content creates the relationship between the content and grade-level standards or determines the appropriate grade-level for their content.

Other ICPs do not use the normative lens, but rather use the Learning Continuum to link to instructional content. The Learning Continuum offers learning statements already sorted by difficulty and topic, RIT range and MAP sub-goal. These ICPs simply link their content to the descriptions of content created by the learning statements. Again, it is the creator of the content, the ICP, who makes the connection between their content and the learning statements. A pilot study of this method of linking to content with Achieve 3000 in two Chicago schools had strong positive results that led to Chicago Publics Schools incorporating the MAP/Achieve 3000 learning paths into its summer school program.

With the release of Skills Navigator this summer, NWEA will offer a fourth way to connect our data to content. Skills Navigator is a classroom assessment and progress monitoring tool designed to identify for each student specific skills that need work. It offers open education resources (OER) content through a third-party, Knovation (best known for their Nettrekker product). NWEA is providing this access to curated OER content through Skills Navigator as a convenience for teachers. That content is not intended to replace a district’s choice for core and supplemental instruction. Though the OER resources are appropriate for addressing individual skills, taken together they do not provide the scope, sequence, breadth, depth and uniformity of approach found in core and supplementary instructional content. NWEA is committed to being content agnostic. We provide the assessment expertise and partner with various ICPs and other content developers to leverage their expertise in instructional design, content, and curriculum.