School closures have been a common tool in the battle against COVID-19. Yet, their costs and benefits remain largely unknown. This column estimates the ‘learning loss’ that occurred when Dutch schools closed for eight weeks, using national exams that took place just before and after lockdown and similar data from previous years. On average, students lost out on a fifth of a year’s worth of learning. Losses were especially marked among those from disadvantaged homes.
School closures during the first wave of the COVID-19 pandemic raised concerns about how well children were learning out of school, and which groups were most likely to fall behind. According to the UN, some 95% of the world’s school population have been affected by now, constituting the largest disruption to education in history (United Nations 2020). Many observers have pointed to the potential harm in terms of learning progress, and the increased care burden on parents (Burgess and Sievertsen 2020, Ilzetzki 2020, Kuhfeld et al. 2020, Moroni et al. 2020). In countries like England, Germany, or France, leaders have opted to close bars and restaurants while letting schools stay open when going into a second lockdown. In other countries, stimulus packets are being directed at the economy while children are still being kept at home.
As the debate rages, reliable evidence on the costs of school closures has been hard to come by. There are several reasons for this. Data from schools are usually not collected and disseminated in real time, unlike data on economic activity, hospitalisations, or fatalities that have dominated the debate on the impacts of the pandemic (Chetty et al. 2020, Dong et al. 2020). Another challenge is inferential: test scores can fluctuate from one year to the next, so valid inference ideally requires data from before schools closed and after they reopened. We also need a relevant comparison group, and information to ensure that the two are similar in relevant respects. For example, if advantaged families keep their children from going back to school, average test scores may decline even if there is no loss of learning.
In this column, we provide new information on how children’s learning suffered when primary schools closed for eight weeks in the Netherlands (Engzell et al. 2020). We meet the above challenges by drawing on exceptionally rich data and nationally standardised tests that took place just before and after lockdown. This lets us implement a difference-in-differences design where we compare progress during school closures to that occurring in a normal year. We adjust for selection bias by matching on a large set of observables, but also with fixed-effects models that compare students within the same school and family. The Netherlands is especially interesting as a ‘best-case’ scenario with a relatively light impact in the first pandemic wave, a short lockdown, and world-leading rates of broadband access (Di Pietro et al. 2020).
Figure 1 Estimated learning loss during spring 2020 by parental education under various specifications
What happened when Dutch schools closed? Despite a light lockdown and excellent infrastructure for remote learning, our results are dire. Figure 1 shows that, across a range of specifications, primary school students lost on average 3 percentile points in the national distribution relative to a normal year, equivalent to 8% of a standard deviation. Measured as progress in a typical year, this implies that students lost out on a fifth of a year of quality-adjusted schooling (Azevedo et al. 2020). As this corresponds almost perfectly to the time that schools remained closed, the upshot is that the average student made little or no progress while learning from home. Worryingly, losses are particularly concentrated among students from less-educated homes – here, the learning loss is up to 55% larger than among their more advantaged peers.
Are these results a temporary setback that schools and teachers can eventually compensate? Only time will tell whether students rebound, remain stable, or fall further behind, but if no action is taken, small losses can cumulate into large disadvantages with time. What we do know is that these losses are in fact due to students learning less during the pandemic, and not due to unfamiliar testing environments. Performance on school tests that do not assess curricular content also declined, but to a much lesser extent than student progress on curricular subjects such as maths, reading, and spelling. This discrepancy suggests that actual knowledge learned is the main explanation for why students are performing worse when returning from lockdown than in a normal year.
To what extent are these learning losses driven by school closures over other detrimental implications of the pandemic? Exposure to stress and anxiety also increased during the lockdown, while children’s parents were more likely to face job loss and sudden economic hardship. The pandemic presents a unique historical event which makes it difficult to address these questions directly with data. Nevertheless, these concerns are arguably much smaller in the Netherlands than elsewhere – the country only suffered mild consequences during the first wave of the pandemic, and the government pursued a famous ‘intelligent lockdown’ relying on voluntary cooperation and allowing ordinary life to continue as much as possible (Tullis 2020).
As large parts of the world are heading into a second wave of the pandemic, it is vital to know how school closures impact on students’ progress, and be aware of the disproportionate damage to students from disadvantaged homes. With new school closures currently being discussed as a way to combat the pandemic, our study gives crucial input for decision-makers. The results are sobering: even in the ‘best-case’ scenario of a short lockdown and good infrastructure for remote learning, students learned little or nothing from home. Overall, these results highlight the urgency of meeting students’ educational needs and implementing measures to compensate for the progress already lost.
Azevedo J P, A Hasan, D Goldemberg, S A Iqbal, and K Geven (2020), “Simulating the potential impacts of covid-19 school closures on schooling and learning outcomes: A set of global estimates”, World Bank Policy Research Working Paper.
Burgess, S and H Sievertsen (2020), “Schools, skills, and learning: The impact of COVID-19 on education”, VoxEU.org, 1 April.
Chetty R, J Friedman, N Hendren, and M Stepner (2020), “How did COVID-19 and stabilization policies affect spending and employment?”, NBER Working Paper 27431.
Di Pietro G, F Biagi, P Costa, Z Karpinski, J Mazza (2020), “The likely impact of COVID-19 on education: Reflections based on the existing literature and recent international datasets”, Publications Office of the European Union.
Dong E, H Du, L Gardner (2020), “An interactive web-based dashboard to track COVID-19 in real time”, The Lancet Infectious Diseases 20(5):533-534
Engzell, P, A Frey, and M D Verhagen (2020), “Learning inequality during the COVID-19 pandemic”, SocArXiv.
Ilzetzki, E (2020), “The economic cost of UK school closures”, VoxEU.org, 5 August.
Kuhfeld, M, J Soland, B Tarasawa, A Johnson, E Ruzek, and J Liu (2020), “Projecting the potential impacts of COVID-19 school closures on academic achievement”, EdWorkingPaper, Annenberg Institute at Brown University.
Moroni G, C Nicoletti, and E Tominey (2020), “Children’s socio-emotional skills and the home environment during the COVID-19 crisis”, VoxEU.org, 9 April.
Tullis, P (2020), “Dutch cooperation made an ‘intelligent lockdown’ a success”, Bloomberg Businessweek, 5 June.
United Nations (2020), “Education during COVID-19 and beyond“, UN Policy Briefs.