How much learning have students lost due to COVID? Projections are coming in, but it’s still hard to say

New York City School Children Return To In-Person Classes
Second and 3rd grade students are monitored during in person learning at an art gallery turned learning pod on October 01, 2020 in New York City.  (Photo by Michael Loccisano/Getty Images)

There is good reason to fear that this spring’s school closures hurt students’ academic progress. But how much learning, exactly, did students lose?

On a national level, we don’t yet know. State tests were canceled last spring, and this year’s tests won’t be given for many months, if they happen at all

That’s prompted researchers to release their own projections of learning lossand they paint a grim picture. 

The nonprofit testing organization NWEA predicted that students started this school year having lost roughly a third a year in reading and half a year in math. CREDO, an education research organization, recently projected that the average student lost 136 to 232 days of learning in math, depending on their state. McKinsey, the consulting firm, predicts that by the fall of 2021, students will have lost three months to a year of learning, depending on the quality of their remote instruction. 

In the absence of actual data on student learning — which is only just starting to emerge from diagnostic exams — those estimates have been widely cited by state and federal officials. The projections, then, are influencing decisions about how to help students catch up, what schools and students need extra resources, and when to reopen school buildings. 

“I think people really latched onto it because it gave some certainty in a very uncertain world of what this could look like,” said Megan Kuhfeld, a research scientist for NWEA. 

But those projections aren’t ironclad. Here’s what they can tell us — and why educators and policymakers should use them carefully.

Projections of learning loss are just that.

Researchers projecting learning loss are not doing so with actual information about what happened after schools closed due to COVID, but with historical data and a host of assumptions — “statistically informed guesses,” says education researcher Paul von Hippel. That’s understandable: Perfect data doesn’t exist, so researchers are trying to fill that hole. 

But those same researchers say the projections shouldn’t be confused with actual learning loss. “They took on a kind of life of their own, beyond what I would have ever imagined, and are sometimes spoken about with far more confidence that we have in them as researchers,” said Kuhfeld. 

The projections rely on the assumption that students learned nothing (or worse) once schools shut their doors.

The backbone of both the NWEA and CREDO projections is the idea that students essentially went on an extended summer break when school buildings closed, and that remote learning was a total wash.

Become a Chalkbeat sponsor

That was certainly true for some students. Teachers, particularly in high-poverty areas, reported widespread absences during remote instruction, and some students became disconnected from school entirely. Many students lacked internet access, necessary devices, and critical in-person support, and the vast majority of teachers say students learned less than they would have in person.

“There is no dispute that the amount and quality of learning that has occurred since school buildings were closed has been deeply inferior,” writes Macke Raymond, the director of CREDO.

But is it fair to assume that the average student learned nothing — and in fact, lost learning as if it were summer? That’s less clear. Surveys of parents and teachers show remote schooling began fairly quickly after buildings closed, and most parents said their child’s school did a good job providing remote instruction despite the challenges.

Further complicating things is that the degree of “summer slide” that students experience in normal years is a matter of ongoing debate among researchers.

Some view CREDO’s projections of learning loss as implausibly large.

CREDO projects that in many states, students lost a full school year — 180 days or more — or more in math this spring. The organization describes its projections as “chilling.” 

But how could students have lost hundreds of days of learning from missing 60 or so actual days of in-person school? 

It has to do with how CREDO converts learning loss, measured in standard deviations, into “days of learning.” The approach is controversial among researchers. Some say it should be avoided because different ways of doing the translation can lead to wildly different results. 

In this case, some outside researchers say CREDO’s results are questionable.

“It doesn’t pass the smell test,” said Harvard education professor Andrew Ho of CREDO’s days of learning projections.

“It’s hard for me to believe that somebody lost a year in a quarter,” said Constance Lindsay, a professor at the University of North Carolina School of Education.

Become a Chalkbeat sponsor

McKinsey and NWEA also use variations of the days of learning approach, but come to more modest conclusions.

Raymond of CREDO said its projections are plausible because students were missing out on learning new things while buildings were closed and also forgetting concepts they’d already been taught. 

She compared it to learning but failing to retain a foreign language. “I don’t think it’s all that different to say a kid would lose all of his math,” she said.

Learning loss predictions for individual students are especially imprecise.

CREDO has gone a step further than others by creating an individual projection of learning loss for each student in 17 states. Raymond said this data has been shared with state departments of education, allowing officials to look up projected 2020 test scores for individual students and pass that data on to schools.

But pinpointing the numerous factors that might contribute to learning loss is even harder for individuals than it is for large groups. 

The British government faced backlash — and ultimately backed down — when it tried to use statistical projections in place of real final exams for high school students. CREDO warns that its estimates for individual student learning loss are imprecise and shouldn’t be used for high-stakes decisions.

“We have encouraged our state partners to not look at these as precise point estimates,” said Raymond. “Think of these as approximations.” 

The projections don’t take into account other factors, like COVID-related trauma, that may have hurt students’ learning.

What’s not included in the projections are some of the other ways COVID affected students, like trauma from family members being sick or dying, additional family stress and financial insecurity, and school budget cuts — all of which can affect students’ academic performance.  

Additionally, some projections focus solely on learning loss from building closures last school year. But many buildings — particularly in districts serving more students of color — remained closed as this year began. And even when buildings have opened, the quality of instruction may have declined as teachers, for instance, struggle to instruct students in person and virtually at the same time.

“We view this as a worst case scenario, because for most of our projections, we’re basically assuming that kids are getting no instruction in the spring,” said Kuhfeld. “But there could also be a much worse case scenario out there, which is that a child missed three months of school because they don’t have a device at a home, they’re facing parental job loss and family stress and housing instability all at the same time.”

Become a Chalkbeat sponsor

Wrong projections come with risks

If students’ academic challenges are understated, policymakers may not do enough to respond, missing chances to introduce ambitious programs like the widespread tutoring being adopted in England to make up for learning loss.

“If you’re over optimistic, you might get the idea that you don’t really need to do anything to compensate, which I think would be not a good decision,” said von Hippel, an education professor at the University of Texas at Austin.

On the other hand, overestimating learning loss could give school officials and teachers a skewed picture. Overestimates could be particularly concerning for individual students, if they lead to lower expectations or reduce students’ likelihood of being exposed to high-level academic content.

“You lost 180 days. What does that mean — you have to repeat a whole grade?” said Lindsay. “That seems kind of ridiculous.” 

Experts say that teachers should instead use diagnostic tests or exams they create themselves to figure out where their students are at. “Putting students back for remediation when they may not need it based on projections” would be unwise, said Kuhfeld.

“Teachers should be using the data that’s most close to the students,” she said. 

The Latest

The three-minute computer-generated video, created by an anti-abortion group, has drawn concern from major medical organizations and sparked controversy in state legislatures in Tennessee and elsewhere.

Chicagoans will vote for 10 of 21 members in the city’s first school board election. Here are six things you should know as the election cycle kicks off.

One proposes to improve accountability, though critics call it harmful. The other would open up a source of construction funds.

New York has been “a rather hostile political environment” for charter schools, Success CEO Eva Moskowitz said Wednesday.

Brooklyn state Sen. Zellnor Myrie recently introduced legislation for the state’s Office of Children and Family Services to conduct a feasibility study for universal after-school.

The Detroit school district last week approved extending the superintendent’s term to 2028.