As assessment data is increasingly used in high stakes evaluations, a major area of discussion is the alignment between the assessment and the responsibility to teach content in accordance with the applicable content standards. While this is a very broad topic, answering the more specific question about how we align our MAP assessment to content standards is a good first step towards the overall answer. To do that I invited our English and Language Arts (ELA) content manager, Sarah Aaserude, and one of our ELA content specialists, Elizabeth Laskey, to be guest bloggers on the topic. Here’s the first of several posts on this topic. Enjoy.
Content alignment is a bit of a contentious issue these days. Teachers and administrators, students and parents, rightly want to be reassured that the questions, or items, being used in an assessment are carefully selected to match the standards and the curriculum required to be taught in the classroom.
Enter the various kinds of assessments–summative, fixed-form, interim/benchmark, CAT, unit tests, etc. Careful alignment is essential to each type. NWEA has been particularly attentive to what alignment means for a computerized adaptive growth measure like MAP.
MAP tests are adaptive so as to accurately measure students of all abilities. As a growth measure test, rather than a summative test, MAP’s goal is to provide accurate data about each student’s level of achievement, regardless of whether a student is performing “on grade level,” above level, or below level. Each test requires a very large number of items, or “item pool,” to be able to present each student with items that will provide detailed data about his or her level of achievement that educators can use to customize instruction.
When building an appropriate-sized pool of items that aligns to a set of standards, we do an initial evaluation of our existing item bank and carefully select items that align to the standards. We evaluate each item in the bank to determine if it matches a skill or concept in the standards and if it engages a student at an appropriate level of cognitive demand.
To ensure that the breadth and depth of the standards are addressed, we also create many new items to align to the standards. We start by examining each standard. Standards can be very specific with a singular category of cognitive demand and a very narrow focus of content (e.g., Use correlative conjunctions [e.g., either/or, neither/nor.]) They can also be very broad (e.g., Determine the meaning of words and phrases as they are used in a text, including figurative and connotative meanings; analyze the impact of a specific word choice on meaning and tone.) For those standards that are broad in their scope, the range of items needed to cover the range of content can be quite large. Often it may not be possible or practical to assess the entire scope of a standard with a single assessment item. Therefore, our Content Specialists analyze the standards and “unpack” them into their constituent skills, each of which describes a single objective from which a test item can be developed. For example, the vocabulary standard referenced above may be unpacked into five objectives:
- Determining the denotative meaning of words.
- Determining the connotative meaning of words or phrases.
- Determining the figurative meaning of words or phrases.
- Analyzing the impact of a specific word choice on meaning.
- Analyzing the impact of a specific word choice on tone.
Unpacking a standard into individual skills ensures that our assessments include items that address the full array of objectives embedded in a standard. In addition, isolating the individual skills in a standard allows us to develop items and item types that require students to engage at a standard’s intended range of cognitive demand categories. For instance, determining the figurative meaning of a phrase requires that students comprehend and process what they are reading. To analyze the impact of word choice on meaning, however, students must engage even more deeply, determining not only the meaning of a word but also connecting how the use of a particular word affects the overall meaning of a passage.
Aligning thousands of individual items to a set of standards is a major task in the alignment process, but it is also essential to consider alignment at the test level. In a future post, we’ll discuss test blueprints in the context of an adaptive test and the role they play in ensuring that during a test a student is presented with a set of items that appropriately covers the major goal areas of the standards. Feel free to leave a comment here if you have any questions or requests for a particular blog topic on content alignment!
