Partnering with schools and districts is the foundation of our Partners in Innovation research program (we’ve blogged about the program previously here and here). With the recent formation of the Innovation Lab at NWEA, our researchers can bring their projects to classrooms across the country where they work with students, teachers, and administrators in the field on testing new ideas and features in our assessment products.
This week, we sat down with researcher Meg Guerreiro of our Innovation Lab to check in on three projects she has in the field.
What is the embedded items project?
The embedded items project aims to explore embedding items within reading passages to improve the student experience and their performance. Students interact with a test item immediately after reading a section of text in which the evidence was presented.
What’s the purpose?
It’s to create an assessment that responds to different student needs in reading by letting students drive the interaction. The prototype accomplishes this by applying universal design principles and allowing for flexibility in student response, which models good instructional principles.
How is it going?
We’ve worked in two schools – in Gaston, Oregon and Gurnee, Illinois – with students providing their feedback. In Gaston, the students worked through the prototype, and we collected information on their user experience and ease of use. As a result, we improved the tutorial, moved around hotlinks, and altered some symbols, colors, and design. The kids in Gurnee tested the improved version – where there were embedded items and traditional items in reading passages. We’re analyzing the results now to see if there are any score differences.
What did the students have to say?
They seemed to like it! Here’s a sample of their feedback:
- “I like to answer it in the middle of the story. It was easier, and I got it done faster.”
- “I did not have to keep the thoughts in my head.”
Tell us about the augmented reality project.
This project explores the use of augmented reality within an assessment, specifically to measure higher depth of knowledge (DOK) levels within a performance-based assessment of the Next Generation Science Standards (NGSS). From a high level, the project aims to include augmented reality within an app where students must solve a problem based on a real-world phenomenon. Content will include measurement of the NGSS (grade 5 Physical Science), an historically difficult area to measure and will explore the measurement of critical thinking.
How is the project going?
In May, we took the working prototype to a school in Portland, where 10 fifth graders had the opportunity to work through it and provide feedback. We collected data on their user experience and plan to use their feedback to make changes in directions, layout, colors, and design.
What did the kids say? We’re guessing they liked the app!
Yes! Here’s a sample of their feedback:
- “It was fun to play around with, and I learned on the way, so I can learn and have fun at the same time using augmented reality.”
- “I liked just playing the game and learning new things about science.”
Tell us about the Collaborative Problem Solving project.
This prototype explores the measurement of Collaborative Problems Solving (CPS) capacity using human-human interaction, open response items, and collaboration between users. In this prototype, we aim to (1) create the framework of measurement, (2) create the platform of interaction that collects data for analysis, (3) create an item for interaction, (4) develop scoring criteria, and (5) assess data collected from the prototype.
This project is designed to create a platform and framework to identify, classify, and score communication and social capabilities involved in completing a CPS task. We believe this approach provides a novel means for measuring important aspects of this CPS dynamic.
What will it look like?
The CPS prototype will take the form of a two-player puzzle game delivered via web browser. In the prototype, each collaborator will use their own device with access to unique in-game resources and actions separate from their partner. Communication between collaborators will be possible through chat messages. To solve the puzzle, students will need to build a shared understanding of the problem, formulate a common plan of action, and coordinate to enact their plan. Data on the actions carried out by the players along with all communication will be captured and analyzed with the goal of providing actionable information to teachers to help them better facilitate the development of students’ collaborative problem-solving skills.
Have students tried it?
We are planning a usability study now to explore user experience, difficulties, and interactions between students and the platform, as well as interactions between student partnerships. This will provide useful information on necessary changes to make to the platform to support larger data collection.
That’s our update from the field! If you’d like your school or district to participate in one of our research projects, contact me to learn more.