Last week, hundreds of you attended our webinar, “The 3 ways you could be sabotaging your assessment plan” (you can find the recorded version here). Since we couldn’t get to every question, we wanted to pose a few of them to our presenter, Lynne Kulich, PhD. Dr. Kulich is currently an account executive here at NWEA and formerly the director of curriculum and instruction in an Ohio district. There, she oversaw a major assessment system overhaul, and in our webinar, she shared some valuable lessons she learned during the process.
At a basic level, how did you get started with your assessment overhaul? I convened a cohort of teachers, principals, intervention specialists, instructional coaches, etc., who met at least once a month to conduct the audit. I ask the group members to check their passion and biases at the door. Just because you’ve ALWAYS used an assessment doesn’t mean it’s providing you with the most precise data. We created an assessment matrix to include each assessment given throughout the district, its purpose, students tested, the time of year, if accommodations were an option, the mode of delivery, etc. This assessment audit needs to be done objectively. You also want to be transparent about the purpose and process and communicate to all stakeholders. I sent memos summarizing our meetings, and I also presented updates to our Board of Education.
What did you find in your assessment audit? Did the audit surprise you or confirm what you already suspected? The assessment audit revealed that our students were severely over tested; our teachers and principals had more data than they could ever have analyzed; our students spent more time assessing than learning; and we were spending a large part of our district budget on numerous assessments for the same single purposes. The audit confirmed what I had suspected all along. Our students and teachers were experiencing assessment fatigue! In addition, we didn’t have cohesion amongst our elementary schools because some chose to give different assessments. Furthermore, we needed to reinvest our funds into professional learning so our teachers could implement fewer assessments and understand how to use the data with fidelity.
Who should be included in the assessment audit and evaluation process and why? Include representatives from all groups of stakeholders, and define the purpose and roles of each member. Include individuals like Technology Directors because their work is impacted by all your online assessments. I also included our Data and Research Director because I wanted to make certain we were looking beyond the surface of each assessment and really taking the validity and reliability aspects of the data into account. Our Data and Research Director could interpret and communicate the various alignment and linking studies to show, for example, that the MAP Growth assessment was providing the best data for us.
Did you experience resistance to the assessment review and planning process? What were the positive points you could make about it? Yes, there was some resistance. Our teachers were very passionate about certain assessments because they had become so comfortable giving them time and time again. Change isn’t always easy or welcome. However, we needed to ask ourselves if the assessment is what’s best for students and if the data is valid and reliable. Look beyond the surface and examine the research behind the data. No assessment is worth giving if the data is misleading. The assessment audit allowed us to eliminate unnecessary assessments and reinvest our time and resources to partner with NWEA. We trained our teachers, so they understood why we gave MAP Growth, and they were engaged with the reports and data. We created much needed consistency across the district, and our middle school teachers finally had common data to support their instructional decisions. We eliminated unnecessary assessments and created more time for instruction.
To learn more about assessment audits, check out Dr. Kulich’s recorded webinar or the assessment matrix tool she discusses here.