What’s the likely impact of the Smarter Balanced test on school’s proficiency rates?

smarter-balanced-test-imgWith students in schools throughout the country now taking the Smarter Balanced and PARCC assessments, there is considerable angst about the impact that college- and career- ready standards will have on proficiency rates. This is not surprising since in the states that implemented earlier versions of college- and career- ready assessments, proficiency rates generally dropped. The drops were almost entirely explained by higher passing scores that students were required to meet in order to be proficient, and were not reflective of an actual drop in student achievement. We published an article on this issue, entitled The Phantom Collapse of Student Achievement in the November 2014 edition of Phi Delta Kappan. That article shows, that while proficiency rates dropped on state tests due to higher cut scores, student scale scores on NWEA tests actually improved in our sample of schools in the same states.

This past winter, the Smarter Balanced consortium announced the cut scores for their new assessment. At the time of the announcement, the consortium also offered an estimate of the expected distribution of students across their four proficiency levels, based on the spring 2014 field test results. Their anticipated proficiency rates were as follows:

Table 1 – Estimated national proficiency rates for the Smarter Balanced assessment.

Data Table

 

 

 

 

 

 

 

 

 

 

Because NWEA administers tests that are aligned to each states’ standards and reported on a common scale, our researchers have the ability to link and compare the cut scores from Smarter Balanced and other state assessments to one another and assess the impact that changes in cut scores are likely to have on proficiency rates. We used this information to derive a preliminary estimate of the Smarter Balanced cut scores relative to NWEA’s scale and NWEA’s norms, which are derived from a nationally representative population of students. Because the population used to develop the Smarter Balanced proficiency rate estimates differs from NWEA’s norming population (the Smarter Balanced Consortium tests in 21 states, while our norms are derived from a representative sample of students from all U.S. states), we emphasize that our estimates are preliminary. We intend to update our estimate of the cut scores and this information, when official results are released by Smarter Balanced.

It is clear from the Table that proficiency rates in nearly all Smarter Balanced states will decline when results are announced this summer. This won’t represent a real decline however, as the higher cut scores associated with Smarter Balanced will be driving proficiency rates and not changes in the actual performance of students. To illustrate that issue, we developed this data visualization. (Note – The state cut score percentiles in the data visualization reflect the 2011 norms. States with NWEA linking studies that were published prior to 2011 reflect the 2008 norms, so they may have different percentiles. This does not impact changes in the proficiency rate calculations.)

data-visualization

This visualization allows you to compare, using achievement data from a representative school system, the estimated change in proficiency rates that may occur in most states if they were to move from their prior cut scores to the Smarter Balanced cut scores. We also show what the change in proficiency rate would be if the test did not change. What’s obvious when you explore the data is that while drops in proficiency rates will be significant for many schools, they won’t reflect an actual change in student achievement. In the case of our sample school system, you will see the opposite – achievement in most schools actually improved when the cut score was held constant. That’s why we’ve characterized this as a phantom collapse.

While most states are preparing the public for a decline in proficiency rates, it’s important for school systems to share data beyond their state test that shows how student achievement is generally continuing to improve even when proficiency rates are decreasing. We hope the illustration we’re offering here provides school boards, media, and the public at large a different lens to with which to view student achievement and helps correct the misperception that school performance collapsed when these new assessments were introduced.

Want to learn more? Check out Dr. Cronin’s webinar “Understanding how MAP & Smarter Balanced Data Can Help You Prep for Proficiency Shifts”