A posterior predictive model checking method assuming posterior normality for item response theory
By: Megan Kuhfeld
This study investigated the violation of local independence assumptions within unidimensional item response theory (IRT) models. Bayesian posterior predictive model checking (PPMC) methods are increasingly being used to investigate multidimensionality in IRT models. The current work proposes a PPMC method for evaluating local dependence in IRT models that are estimated using full-information maximum likelihood. The proposed approach, which was termed as “PPMC assuming posterior normality” (PPMC-N), provides a straightforward method to account for parameter uncertainty in model fit assessment. A simulation study demonstrated the comparability of the PPMC-N and the Bayesian PPMC approach in the detection of local dependence in dichotomous IRT models.See More
This article was published outside of NWEA. The full text can be found at the link above.
Topics: Growth modeling
To describe the impact of the COVID-19 pandemic on students, researchers have translated test scores into months of learning to claim how many months/years students are behind in school. Despite its perceived accessibility, there are major downsides to this translation. To inform future uses by researchers and media, we discuss in this brief how to calculate this metric as well as its trade-offs.
The purpose of this technical appendix is to share more detailed results and to describe more fully the sample and methods used in the research included in the brief, Learning during COVID-19: An update on student achievement and growth at the start of the 2021-22 school year. We investigated two research questions:
- How does student achievement in fall 2021 compare to pre-pandemic levels (namely fall 2019)?
- How did academic gains between fall 2019 and fall 2021 compare to normative growth expectations?
To what extent has the COVID-19 pandemic affected student achievement and growth in reading and math, and which students have been most affected? Using data from 6 million students in grades 3-8 who took MAP Growth assessments in reading and math, this brief examines how gains across the pandemic (fall 2019 to fall 2021) and student achievement in fall 2021 compare to pre-pandemic trends. This research provides insight to leaders working to support recovery.
Using achievement data from fall and spring of grades K-8 for 840,000 students in 8,800 public schools, this study provides novel evidence on how achievement and growth differ between rural and nonrural schools. Rural students start kindergarten slightly ahead of nonrural students but fall behind by middle school. The divergence is driven by larger summer losses for rural students. In both rural and nonrural schools, Black–White achievement gaps widen during the school year.
New research examining academic achievement and growth of students in special education and their peers who were never in special education during each school year and summer in grades K-4 shows that students with disabilities grow as much or more academically during the school year than their peers without disabilities during some years, but that steeper summer learning losses for students with disabilities contribute to widening disparities.
By: Angela Johnson, Elizabeth Barker
This study compares within- and across-years academic growth for students who were ever in special education (ever-SPED) to students who were never in special education (never-SPED) in grades K-4. Ever-SPED students grew more in math and reading than never-SPED students during many school years, but lost more learning during every summer than their peers, leading to expanding disparities. These findings suggest that summer learning opportunities are crucial for improving educational outcomes for students with disabilities.
By: Angela Johnson, Elizabeth Barker
To avoid the subjectivity of having a single person evaluate a construct of interest, multiple raters are often used. While a range of models to address measurement issues that arise when using multiple raters have been presented, few are available to estimate growth in the presence of multiple raters. This study provides a model that removes all but the shared perceptions of raters at a given timepoint then adds on a latent growth curve model across timepoints. Results indicate that the model shows promise for use by researchers who want to estimate growth based on longitudinal multi-rater data.