Quantifying learning losses due to COVID-19 in Ecuador: A fixed-effects analysis of standardized test scores for elementary and middle school

Ponce Jarrín, Juan Elías; Intriago Armijos, Ruthy Vanessa

Cuantificación de las pérdidas de aprendizaje debido a la COVID-19 en Ecuador: un análisis de efectos fijos de los puntajes de pruebas estandarizadas para la educación primaria y secundaria

Ponce Jarrín, Juan Elías

Facultad Latinoamericana de Ciencias Sociales, FLACSO (Ecuador – Ecuador)

ORCID: https://orcid.org/0000-0002-1699-9934

jponce@flacso.edu.ec

NOTA BIOGRÁFICA

PhD. en Estudios del Desarrollo por el Institute of Social Studies de la Haya de la Universidad Erasmus. Exdirector de FLACSO Ecuador (2012-2020), profesor investigador del Departamento de Desarrollo, Ambiente y Territorio de esta universidad. Es experto en economía de la educación, pobreza y desigualdad, economía laboral y evaluación de impacto. Ha participado en algunas evaluaciones de impacto de programas sociales en Ecuador: Bono de Desarrollo Humano, Redes Amigas, Hilando el Desarrollo. Ha trabajado en proyectos de investigación a nivel latinoamericano.

Intriago Armijos, Ruthy Vanessa

Facultad Latinoamericana de Ciencias Sociales, FLACSO (Ecuador – Ecuador)

ORCID: https://orcid.org/0000-0003-4804-0194

rvintriagofl@flacso.edu.ec

NOTA BIOGRÁFICA

Máster en Economía del Desarrollo por la Facultad Latinoamericana de Ciencias Sociales (FLACSO) Ecuador. Ha ocupado importantes cargos en el Ministerio de Educación de Ecuador, donde ha dirigido investigaciones y políticas educativas. Su experiencia abarca la coordinación de proyectos sociales y el análisis de políticas públicas, con énfasis en la evaluación de programas educativos. Actualmente, es candidata a doctora en Economía del Desarrollo por FLACSO Ecuador y está especializada en estudios económicos y sociales. Ha publicado en revistas académicas y contribuido a libros sobre educación y desarrollo en América Latina.

ABSTRACT

Objectives: The COVID-19 crisis has led to extensive school closures, affecting education worldwide. This study investigates the impact of these closures on academic achievement in Ecuador, particularly for 4th-, 7th-, 10th-, and 12th-grade students. Methodology: A score-equating process was conducted to standardize test scores across different school years. The study spans four years and employs a fixed-effects regression model to analyze learning achievements. Results: The results indicate a significant adverse impact of the pandemic on academic achievements across all grades. On average, students’ academic achievements in 2022 were substantially lower than those in the pre-pandemic years, suggesting a near-complete loss of learning. Conclusions: These findings underscore the profound negative effects of the COVID-19 pandemic on education in Ecuador and emphasize the urgent need for effective interventions to address learning loss and mitigate long-term consequences.

KEYWORDS

Education; COVID-19; learning loss; academic performance; fixed-effects model.

RESUMEN

Objetivos: la crisis de la COVID-19 ha provocado extensos cierres de escuelas, afectando la educación a nivel mundial. Este estudio investiga el impacto de estos cierres en el rendimiento académico en Ecuador, particularmente en estudiantes de 4.º, 7.º, 10.º y 12.º grado. Metodología: se realizó un proceso de equiparación de puntuaciones para estandarizar los resultados de las pruebas a lo largo de diferentes años escolares. El estudio abarca cuatro años y utiliza un modelo de regresión de efectos fijos. Resultados: los resultados muestran un impacto adverso significativo de la pandemia en el rendimiento académico en todos los grados. En promedio, los logros académicos en 2022 fueron sustancialmente más bajos que en los años previos a la pandemia, sugiriendo una pérdida de aprendizaje casi total. Conclusiones: estos hallazgos destacan los profundos efectos negativos de la COVID-19 en la educación en Ecuador y subrayan la urgente necesidad de implementar intervenciones efectivas para abordar la pérdida de aprendizaje y mitigar sus consecuencias a largo plazo.

PALABRAS CLAVE

Educación; COVID-19; pérdida de aprendizaje; rendimiento académico; modelo de efectos fijos.

SUMARIO

1. INTRODUCTION. 2. PRE-COVID SITUATION IN ECUADOR AND EFFECTS OF THE CRISIS. 3. COVID-19 AND EDUCATION 4. DATA AND METHODOLOGY. 5. RESULTS. 6. CONCLUSIONS. AKNOWLEDGEMENTS. REFERENCES.

1. INTRODUCTION

The National Institute of Educational Evaluation (INEVAL) has been conducting standardized tests of learning and associated factors in Ecuador, since the 2013-2014 school year, both on a sample and, in some cases, on a census basis. This means that, in theory, Ecuador would have 9 comparable sets of learning data over the last 10 years. However, during this time, INEVAL made changes to the items evaluated and it cannot be guaranteed that the tests are equivalnt in terms of difficulty. This poses a challenge for the country in terms of assessing the impact of different education n policies, as comparability among different evaluations is required. Therefore, it is incorrect to directly compare scores between test takers who took the assessment on different test forms. The contribution of this study is that a score equating process was performed between the test forms, allowing us to estimate the effect of the pandemic on students’ academic achievements in the 4th, 7th, and 10th grades of General Basic Education and 3rd year of High School by using an econometric analysis based on assessments conducted in Ecuador before and after the school closures. Since the pandemic is an exogenous shock, comparing children’s achievements in the periods before the pandemic (2019) with their achievements in 2021 (after a year of the pandemic) allows us to assess the pandemic’s effect on children’s learning after standardizing the tests in different school years.

No study has addressed this topic in Ecuador by systematically documenting the country’s educational policy actions and the pandemic’s effect on educational variables, specifically academic achievements. In particular, no study has carried out a data equating process for its analysis. This gap in the literature is also apparent across Latin America, where studies on the impact of the pandemic on education are scarce. For this reason, this work represents a significant contribution to the literature and an important starting point for further research on educational policy during times of pandemics.

This paper is structured as follows: Section 2 provides an overview of the pre COVID scenario in Ecuador and the impact of COVID-19 on academic performance before, during, and after the pandemic; Section 3 covers the effects of the COVID-19 pandemic on education, addressing the concerns surrounding the prolonged closure of schools; Section 4 describes our data and methodology; Section 5 describes our findings on student achievement; and Section 6 concludes.

2. PRE-COVID SITUATION IN ECUADOR AND EFFECTS OF THE CRISIS

The closure of schools will impact educational achievements, enrollment, and attendance, and will have differentiated effects on vulnerable students. Educational issues that have been consistently present in Latin America, such as lagging and school dropout, will deepen and widen the educational and access gaps.

Most countries in the region opted for distance education initiatives according to their installed capacity: levels of connectivity, digital educational resources, printed material, television, etc. However, the starting conditions for many countries in this region are not optimal.

Before the onset of the COVID-19 crisis, the country faced certain economic, social, and political problems. Economically, GDP growth rates have already slowed since the second half of the 2010s. While the average GDP growth in the first half of the 2010s was 5.15% (between 2010 and 2014), that in the second half of the decade (between 2015 and 2019) was only 0.5%.

The COVID-19 crisis generated a dual shock in the Ecuadorian economy. On one hand, there was a supply shock due to the shutdown of most economic activities to prevent the spread of the virus. On the other hand, there was a demand shock due to a reduction in household income (and fiscal revenues). This reduced demand also led to a reduction in financial liquidity associated with the halt in economic activity, disruptions in payment chains, and losses of profitability and wealth. Market volatility also increased because of uncertainty regarding the intensity and duration of the pandemic and economic shutdown. These shocks represent something never seen in Ecuador’s economic history, resulting in an overall economic decline of 7.75% in 2020.

A similar picture was observed for the social indicators. For example, the incidence of poverty decreased in the first half of the 2010s and increased in the second half. In fact, between 2018 and 2019, poverty increased by 4 percentage points. Finally, with the COVID-19 crisis, poverty increased by seven percentage points, reaching levels similar to those at the beginning of the 2010s, essentially undoing all the progress made in poverty reduction between 2010 and 2017.

To analyze the social situation, it is crucial to review what happens to the employment levels in society. Before the COVID-19 crisis, in 2019, formal employment was 39%. The impact of the COVID-19 crisis on formal employment was significant, with a reduction of approximately nine percentage points. Another crucial element in observing the evolution of social conditions, both before and after the pandemic, is the Gini coefficient. Between 2010 and 2014, the Gini coefficient saw a 4-point reduction, dropping from 0.49 to 0.45. From 2015, the Gini coefficient increased, reaching 0.47 in 2019. With the COVID-19 crisis, the Gini coefficient increased by three points to 0.50, the highest value in the 2010s.

As we can see, the Ecuadorian economy and society were already facing problems before the COVID-19 crisis. The economy showed signs of stagnation, while poverty and inequality began to increase, and appropriate employment had already decreased. In this context of economic and social complexity, the COVID-19 crisis emerged, further exacerbating the situation. Macroeconomic and social indicators were drastically affected, representing one of the most complex crises in the country’s economic history. Alongside internal supply and demand shocks due to the halt in economic activity, the country faced an external shock manifested in the drop in oil prices, which directly affected fiscal revenues. Public policy, especially in terms of social investment, failed to respond to the needs of the COVID-19 pandemic. In the pandemic year, social spending, especially in health and education, decreased in per capita terms. On the other hand, debt servicing continued to increase permanently, even during the pandemic.

3. COVID-19 AND EDUCATION

The prolonged closure of schools has raised serious concerns about the effects of the crisis, including the potential increase in school dropout rates. According to UNESCO, it is estimated that around 24 million students, from preschool to university, were at risk of not returning to school in 2020 due to the educational disruption caused by the COVID-19 pandemic. Nearly half of these students are concentrated in regions such as South and Western Asia and sub-Saharan Africa.

University students have been particularly affected, in part, by the cost of education at this level. Additionally, early childhood education has been severely affected, while primary and secondary levels have approximately 10.9 million students at risk, including 5.2 million girls (UNESCO, 2020UNESCO (2020). How many students are at risk of not returning to school? https://unesdoc.unesco.org/ark:/48223/pf0000373992).

One study conducted by AzevedoAzevedo, J. P., Hasan, A., Goldemberg, D., Geven, K. & Iqbal, S. A. (2021). Simulating the potential impacts of COVID-19 school closures on schooling and learning outcomes: A set of global estimates. The World Bank Research Observer, 36(1). https://doi.org/10.1093/wbro/lkab003 et al. (2021)Azevedo, J. P., Hasan, A., Goldemberg, D., Geven, K. & Iqbal, S. A. (2021). Simulating the potential impacts of COVID-19 school closures on schooling and learning outcomes: A set of global estimates. The World Bank Research Observer, 36(1). https://doi.org/10.1093/wbro/lkab003 warns that approximately 7 million students from primary to secondary education may drop out of school due to household income loss. This scenario is especially concerning because once children leave education and engage in labor activities, it becomes extremely challenging to reintegrate them into the education system.

It is important to note that school dropout did not affect all students uniformly. This is expected to be much more pronounced in vulnerable populations, including those from low socioeconomic backgrounds, rural areas, ethnic minorities, migrants, students with disabilities, and girls.

Moreover, prolonged closure of schools has a significant impact on the quality of education, particularly in terms of academic achievement. For instance, simulation studies conducted by AzevedoAzevedo, J. P., Hasan, A., Goldemberg, D., Geven, K. & Iqbal, S. A. (2021). Simulating the potential impacts of COVID-19 school closures on schooling and learning outcomes: A set of global estimates. The World Bank Research Observer, 36(1). https://doi.org/10.1093/wbro/lkab003 et al. (2021)Azevedo, J. P., Hasan, A., Goldemberg, D., Geven, K. & Iqbal, S. A. (2021). Simulating the potential impacts of COVID-19 school closures on schooling and learning outcomes: A set of global estimates. The World Bank Research Observer, 36(1). https://doi.org/10.1093/wbro/lkab003 indicated that the extended closure of schools, as experienced during the pandemic, results in a decrease in the effective years of basic education that students achieve over their lifetime. This loss could amount to nearly a full year in the most pessimistic scenario, which is consistent with the school closures experienced in Ecuador.

For instance, EngzellEngzell, P., Frey, A. & Verhagen, M. D. (2021). Learning loss due to school closures during the COVID-19 pandemic. Proceedings of the National Academy of Sciences of the United States of America, 118(17), artículo e2022376118. https://doi.org/10.1073/pnas.2022376118 et al. (2021Engzell, P., Frey, A. & Verhagen, M. D. (2021). Learning loss due to school closures during the COVID-19 pandemic. Proceedings of the National Academy of Sciences of the United States of America, 118(17), artículo e2022376118. https://doi.org/10.1073/pnas.2022376118)Engzell, P., Frey, A. & Verhagen, M. D. (2021). Learning loss due to school closures during the COVID-19 pandemic. Proceedings of the National Academy of Sciences of the United States of America, 118(17), artículo e2022376118. https://doi.org/10.1073/pnas.2022376118 found that in the Netherlands, an eight-week lockdown resulted in a learning loss equivalent to 0.08 standard deviations, with students from less-educated homes experiencing greater setbacks. SchuurmanSchuurman, T. M., Henrichs, L. F., Schuurman, N. K., Polderdijk, S. & Hornstra, L. (2023). Learning loss in vulnerable student populations after the first COVID-19 school closure in the Netherlands. Scandinavian Journal of Educational Research, 67(2), 309-326. https://doi.org/10.1080/00313831.2021.2006307 et al. (2023)Schuurman, T. M., Henrichs, L. F., Schuurman, N. K., Polderdijk, S. & Hornstra, L. (2023). Learning loss in vulnerable student populations after the first COVID-19 school closure in the Netherlands. Scandinavian Journal of Educational Research, 67(2), 309-326. https://doi.org/10.1080/00313831.2021.2006307 found that among vulnerable student groups, school closures led to learning losses exceeding the actual closure duration, with a 2.47-month loss in mathematics and 2.35-month loss in reading comprehension.

Additionally, BertolettiBertoletti, A., Cannistrà, M., Soncin, M. & Agasisti, T. (2023). The heterogeneity of COVID-19 learning loss across Italian primary and middle schools. Economics of Education Review, (95), article 102435. https://doi.org/10.1016/j.econedurev.2023.102435 et al. (2023)Bertoletti, A., Cannistrà, M., Soncin, M. & Agasisti, T. (2023). The heterogeneity of COVID-19 learning loss across Italian primary and middle schools. Economics of Education Review, (95), article 102435. https://doi.org/10.1016/j.econedurev.2023.102435 revealed considerable learning loss among Italian students, with variations depending on subject and grade. ContiniContini, D., Di Tommaso, M. L., Muratori, C., Piazzalunga, D. & Schiavon, L. (2022). Who lost the most? Mathematics achievement during the COVID-19 pandemic. The BE Journal of Economic Analysis & Policy, 22(2), 399-408. https://doi.org/10.1515/bejeap-2021-0447 et al. (2022)Contini, D., Di Tommaso, M. L., Muratori, C., Piazzalunga, D. & Schiavon, L. (2022). Who lost the most? Mathematics achievement during the COVID-19 pandemic. The BE Journal of Economic Analysis & Policy, 22(2), 399-408. https://doi.org/10.1515/bejeap-2021-0447 analyzed the impact on mathematics skills in Italian primary school children, showing a negative impact on performance, particularly among girls and high-achieving children from low-educated parents.

JakubowskiJakubowski, M., Gajderowicz, T. & Patrinos, H. A. (2023). Global learning loss in student achievement: First estimates using comparable reading scores. Economics Letters, (232), article 111313. https://doi.org/10.1016/j.econlet.2023.111313 et al. (2023)Jakubowski, M., Gajderowicz, T. & Patrinos, H. A. (2023). Global learning loss in student achievement: First estimates using comparable reading scores. Economics Letters, (232), article 111313. https://doi.org/10.1016/j.econlet.2023.111313 estimated the global impact of COVID-19 on student reading using international achievement tests. They modeled the effect of school closures on achievement and found that scores declined by an average of 33 percent of a standard deviation, equivalent to more than a year of schooling.

BlaskóBlaskó, Z., da Costa, P. & Schnepf, S. V. (2022). Learning losses and educational inequalities in Europe: Mapping the potential consequences of the COVID-19 crisis. Journal of European Social Policy, 32(4), 361-375. https://doi.org/10.1177/09589287221091687 et al. (2022)Blaskó, Z., da Costa, P. & Schnepf, S. V. (2022). Learning losses and educational inequalities in Europe: Mapping the potential consequences of the COVID-19 crisis. Journal of European Social Policy, 32(4), 361-375. https://doi.org/10.1177/09589287221091687 studied educational inequalities across European countries, using data from the Trends in International Mathematics and Science Study (TIMSS) 2019. They found that educational inequalities between and within countries likely increased significantly because of the pandemic. The study revealed that the association between low achievement and disadvantage was strongest in countries such as Lithuania, Ireland, and Hungary, where pupils most exposed to learning losses during distance learning had already struggled prior to the pandemic. Conversely, countries such as Portugal and Denmark, with more robust education systems, demonstrated a greater capacity to counteract these inequalities.

In the United States, RelyeaRelyea, J. E., Rich, P., Kim, J. S. & Gilbert, J. B. (2023). The COVID-19 impact on reading achievement growth of Grade 3-5 students in a U. S. urban school district: variation across student characteristics and instructional modalities. Reading and Writing, (36), 317-346. https://doi.org/10.1007/s11145-022-10387-y et al. (2023)Relyea, J. E., Rich, P., Kim, J. S. & Gilbert, J. B. (2023). The COVID-19 impact on reading achievement growth of Grade 3-5 students in a U. S. urban school district: variation across student characteristics and instructional modalities. Reading and Writing, (36), 317-346. https://doi.org/10.1007/s11145-022-10387-y studied the impact of COVID-19 on reading achievement growth among Grade 3-5 students in an urban school district. They found significant learning losses during the 2020-2021 school year, with effect sizes of 0.54, 0.27, and 0.28 standard deviations for Grades 3, 4, and 5, respectively.

Overall, studies in developed countries suggest that learning loss is generally tied to socioeconomic factors, with students from disadvantaged backgrounds suffering the most from it.

In Africa and South Asia, studies have focused on learning gaps and educational inequalities exacerbated by the pandemic. WolfWolf, S., Aurino, E., Suntheimer, N. M., Avornyo, E. A., Tsinigo, E., Jordan, J., Samanhiya, S., Aber, J. L. & Behrman, J. R. (2022). Remote learning engagement and learning outcomes during school closures in Ghana. International Journal of Educational Research, (115), article 102055. https://doi.org/10.1016/j.ijer.2022.102055 et al. (2022)Wolf, S., Aurino, E., Suntheimer, N. M., Avornyo, E. A., Tsinigo, E., Jordan, J., Samanhiya, S., Aber, J. L. & Behrman, J. R. (2022). Remote learning engagement and learning outcomes during school closures in Ghana. International Journal of Educational Research, (115), article 102055. https://doi.org/10.1016/j.ijer.2022.102055 conducted a longitudinal study in Ghana, which highlighted the inequalities in remote learning opportunities between public and private schools. They found that public schoolchildren and those from low socioeconomic backgrounds performed significantly worse.

In South Asia, SinghSingh, A., Romero, M. & Muralidharan, K. (2023). COVID-19 Learning Loss and Recovery: Panel Data Evidence from India. Journal of Human Resources, 59(5). https://doi.org/10.3368/jhr.0723-13025R2 et al. (Singh, A., Romero, M. & Muralidharan, K. (2023). COVID-19 Learning Loss and Recovery: Panel Data Evidence from India. Journal of Human Resources, 59(5). https://doi.org/10.3368/jhr.0723-13025R22023)Singh, A., Romero, M. & Muralidharan, K. (2023). COVID-19 Learning Loss and Recovery: Panel Data Evidence from India. Journal of Human Resources, 59(5). https://doi.org/10.3368/jhr.0723-13025R2 reported substantial learning deficits among Indian students owing to prolonged school closures, though some recovery was observed through government interventions. Guariso & Nyqvist (2023)Guariso, A. & Nyqvist, M. B. (2023). The impact of the COVID-19 pandemic on children’s learning and wellbeing: Evidence from India. Journal of Development Economics, (164), article 103133. https://doi.org/10.1016/j.jdeveco.2023.103133 similarly found significant learning losses in Assam, India and noted that children lacking resources and parental support suffered the largest setbacks.

KilenthongKilenthong, W. T., Boonsanong, K., Duangchaiyoosook, S., Jantorn, W. & Khruapradit, V. (2023). Learning losses from school closure due to the COVID-19 pandemic for Thai kindergartners. Economics of Education Review, (96), article 102455. https://doi.org/10.1016/j.econedurev.2023.102455 et al. (2023)Kilenthong, W. T., Boonsanong, K., Duangchaiyoosook, S., Jantorn, W. & Khruapradit, V. (2023). Learning losses from school closure due to the COVID-19 pandemic for Thai kindergartners. Economics of Education Review, (96), article 102455. https://doi.org/10.1016/j.econedurev.2023.102455 presented empirical evidence from Thailand indicating significant learning losses among kindergartners during school closures, especially in mathematics and working memory. These losses were heterogeneous and varied according to the child’s gender, special needs, and caregiver education.

AngristAngrist, N., de Barros, A., Bhula, R., Chakera, S., Cummiskey, C., DeStefano, J., Floretta, J., Kaffenberger, M., Piper, B. & Stern, J. (2021). Building back better to avert a learning catastrophe: Estimating learning loss from COVID-19 school shutdowns in Africa and facilitating short-term and long-term learning recovery. International Journal of Educational Development, (84), article 102397. https://doi.org/10.1016/j.ijedudev.2021.102397 et al. (2021)Angrist, N., de Barros, A., Bhula, R., Chakera, S., Cummiskey, C., DeStefano, J., Floretta, J., Kaffenberger, M., Piper, B. & Stern, J. (2021). Building back better to avert a learning catastrophe: Estimating learning loss from COVID-19 school shutdowns in Africa and facilitating short-term and long-term learning recovery. International Journal of Educational Development, (84), article 102397. https://doi.org/10.1016/j.ijedudev.2021.102397 examined the impact of COVID-19 in several African countries and estimated that learning deficits for grade 3 students could lead to nearly three years of lost learning by grade 10. ArdingtonArdington, C., Wills, G. & Kotze, J. (2021). COVID-19 learning losses: Early grade reading in South Africa. International Journal of Educational Development, (86), article 102480. https://doi.org/10.1016/j.ijedudev.2021.102480 et al. (2021)Ardington, C., Wills, G. & Kotze, J. (2021). COVID-19 learning losses: Early grade reading in South Africa. International Journal of Educational Development, (86), article 102480. https://doi.org/10.1016/j.ijedudev.2021.102480 found significant short-term learning losses in South Africa, with grade 2 and 4 students losing between 57% and 70% of one year of learning compared to their pre-pandemic peers.

Finally, several studies have consolidated the research on learning loss due to the COVID-19 pandemic. HammersteinHammerstein, S., König, C., Dreisörner, T. & Frey, A. (2021). Effects of COVID-19 –related school closures on student achievement– a systematic review. Frontiers in psychology, (12), article 746289. https://doi.org/10.3389/fpsyg.2021.746289 et al. (Hammerstein, S., König, C., Dreisörner, T. & Frey, A. (2021). Effects of COVID-19 –related school closures on student achievement– a systematic review. Frontiers in psychology, (12), article 746289. https://doi.org/10.3389/fpsyg.2021.7462892021)Hammerstein, S., König, C., Dreisörner, T. & Frey, A. (2021). Effects of COVID-19 –related school closures on student achievement– a systematic review. Frontiers in psychology, (12), article 746289. https://doi.org/10.3389/fpsyg.2021.746289 provided a comprehensive overview of the general and differential effects of COVID-19-related school closures on student achievement in primary and secondary education. Their findings showed that school closures had a negative impact on student achievement, with younger students and those from low socioeconomic backgrounds being particularly affected. They also identified specific measures that could mitigate these negative effects, offering valuable insights into shaping educational policies in anticipation of future school closures.

Similarly, Moscoviz & Evans (2022)Moscoviz, L. & Evans, D. K. (2022). Learning loss and student dropouts during the COVID-19 pandemic: A review of the evidence two years after schools shut down [Working Papers 609]. Center for Global Development. https://www.cgdev.org/publication/learning-loss-and-student-dropouts-during-covid-19-pandemic-review-evidence-two-years, PatrinosPatrinos, H. A., Vegas, E. & Carter-Rau, R. (2022). An Analysis of COVID-19 Student Learning Loss [Policy Research Working Paper n.º 10033]. World Bank Group. https://hdl.handle.net/10986/37400 et al. (2022)Patrinos, H. A., Vegas, E. & Carter-Rau, R. (2022). An Analysis of COVID-19 Student Learning Loss [Policy Research Working Paper n.º 10033]. World Bank Group. https://hdl.handle.net/10986/37400, and UğraşUğraş, M., Zengin, E., Papadakis, S. & Kalogiannakis, M. (2023). Early childhood learning losses during COVID-19: Systematic review. Sustainability, 15(7), article 6199. https://doi.org/10.3390/su15076199 et al. (2023)Uğraş, M., Zengin, E., Papadakis, S. & Kalogiannakis, M. (2023). Early childhood learning losses during COVID-19: Systematic review. Sustainability, 15(7), article 6199. https://doi.org/10.3390/su15076199 conducted systematic reviews that identified key trends and common factors contributing to learning loss, including socioeconomic disparities, the digital divide, and inconsistent remote learning experiences. These studies indicate a significant increase in early childhood learning losses, suggesting that the pandemic’s effects were particularly severe for younger students and disadvantaged groups. Additionally, these reviews highlight concerns about the rise in educational inequality during the pandemic.

Tang (2023Tang, K. H. D. (2023). Impacts of COVID-19 on primary, secondary and tertiary education: a comprehensive review and recommendations for educational practices. Educational Research for Policy and Practice, 22(1), 23-61. https://doi.org/10.1007/s10671-022-09319-y)Tang, K. H. D. (2023). Impacts of COVID-19 on primary, secondary and tertiary education: a comprehensive review and recommendations for educational practices. Educational Research for Policy and Practice, 22(1), 23-61. https://doi.org/10.1007/s10671-022-09319-y and AguadedAguaded, I., Vizcaíno-Verdú, A., García-Prieto, V. & De-Casas-Moreno, P. (2023). The impact of post-pandemic learning loss on education development: A systematic review. Review of Communication Research, (11), 172-189. et al. (2023)Aguaded, I., Vizcaíno-Verdú, A., García-Prieto, V. & De-Casas-Moreno, P. (2023). The impact of post-pandemic learning loss on education development: A systematic review. Review of Communication Research, (11), 172-189. addressed the impact of the COVID-19 pandemic on educational practices and highlighted the need for more flexible and adaptable models. Tang’s review showed how remote learning influenced education at various levels, while Aguaded et al. emphasized the digital learning gap, pointing to the importance of improved media literacy and technology resources. These reviews, along with HammersteinHammerstein, S., König, C., Dreisörner, T. & Frey, A. (2021). Effects of COVID-19 –related school closures on student achievement– a systematic review. Frontiers in psychology, (12), article 746289. https://doi.org/10.3389/fpsyg.2021.746289 et al. (2021)Hammerstein, S., König, C., Dreisörner, T. & Frey, A. (2021). Effects of COVID-19 –related school closures on student achievement– a systematic review. Frontiers in psychology, (12), article 746289. https://doi.org/10.3389/fpsyg.2021.746289, serve as important references for understanding the broader context of learning loss during the pandemic and for guiding future strategies to address educational inequalities.

The literature review revealed common trends applicable to Ecuador. Socioeconomic factors play a crucial role in educational outcomes, with disadvantaged students being the most affected ones. In Ecuador, where the digital divide is significant, the pandemic has exacerbated existing educational inequalities. Prior to the pandemic, Ecuador experienced fluctuations in educational spending and performance, with a decline in per capita expenditure and varying enrollment rates. The impact of the pandemic on Ecuadorian education underscores the need for policies to address digital inequality and ensure educational continuity (Intriago, 2021Intriago, R. (2021). Revisión de mecanismos de adaptación de los sistemas educativos a nivel regional e internacional a la emergencia sanitaria y estudio sobre la adaptación de la gestión docente en el contexto de la crisis sanitaria generada por al COVID-19. Ministerio de Educación de Ecuador. https://educacion.gob.ec/wp-content/uploads/downloads/2022/02/Nota-tecnica-estudio-sistemas-educativos-practicas-docentes-en-emergencia-sanitaria.pdf).

The Ecuadorian educational system has adopted various policies to mitigate the impact of the pandemic, including flexible curricula, diagnostic and remedial programs, and teacher training. However, implementation challenges, particularly in the public sector, have highlighted the existing inequalities. The need for a comprehensive digital plan is evident in bridging the technology gap and ensuring equitable access to education (Intriago, 2021Intriago, R. (2021). Revisión de mecanismos de adaptación de los sistemas educativos a nivel regional e internacional a la emergencia sanitaria y estudio sobre la adaptación de la gestión docente en el contexto de la crisis sanitaria generada por al COVID-19. Ministerio de Educación de Ecuador. https://educacion.gob.ec/wp-content/uploads/downloads/2022/02/Nota-tecnica-estudio-sistemas-educativos-practicas-docentes-en-emergencia-sanitaria.pdf). Studies in developing countries have revealed similar challenges, emphasizing the need for investment in educational technology and policies to address disparities (Reimers, 2022Reimers, F. M. (2022). Learning from a pandemic: The impact of COVID-19 on education around the world. In F. M. Reimers (ed.), Primary and secondary education during COVID-19: Disruptions to educational opportunity during a pandemic (pp. 1-37). Springer. https://doi.org/10.1007/978-3-030-81500-4).

The digital divide exacerbates educational inequalities, with disadvantaged students facing severe barriers due to insufficient technology and support. This divide has led to decreased participation and learning quality, further deepening educational disparities. In response, some developing countries have implemented strategies such as radio programs and textbook distribution to address these challenges, but these measures have had varying levels of success (Reimers, 2022Reimers, F. M. (2022). Learning from a pandemic: The impact of COVID-19 on education around the world. In F. M. Reimers (ed.), Primary and secondary education during COVID-19: Disruptions to educational opportunity during a pandemic (pp. 1-37). Springer. https://doi.org/10.1007/978-3-030-81500-4).

The pandemic underscored the urgent need to address the digital divide and enhance the technological infrastructure in developing countries to ensure equitable and effective education. This experience highlights the critical need for investment in educational technology and policies to bridge these gaps for future resilience (Reimers, 2022Reimers, F. M. (2022). Learning from a pandemic: The impact of COVID-19 on education around the world. In F. M. Reimers (ed.), Primary and secondary education during COVID-19: Disruptions to educational opportunity during a pandemic (pp. 1-37). Springer. https://doi.org/10.1007/978-3-030-81500-4).

Educational resilience refers to the ability of students and educational systems to adapt and recover from adverse conditions such as economic shocks or natural disasters. Theoretical models of human capital formation suggest that crises such as the COVID-19 pandemic can disrupt the accumulation of human capital by increasing mortality rates, destroying infrastructure, and reducing household income. These disruptions often lead to a decline in educational investments, especially in low-income countries where the opportunity costs of education are higher during crises (BaezBaez, J., de la Fuente, A. & Santos, I. V. (2010). Do Natural Disasters Affect Human Capital? An Assessment Based on Existing Empirical Evidence [IZA Discussion Paper n.º 5164]. IZA. http://dx.doi.org/10.2139/ssrn.1672172 et al., 2010Baez, J., de la Fuente, A. & Santos, I. V. (2010). Do Natural Disasters Affect Human Capital? An Assessment Based on Existing Empirical Evidence [IZA Discussion Paper n.º 5164]. IZA. http://dx.doi.org/10.2139/ssrn.1672172).

Resilience in education can also manifest through adaptive responses that mitigate these negative impacts. A relevant case is the impact evaluation of an earthquake that hit the Ecuadorian coast on April 16, 2016. This study highlights how recovery efforts following the disaster led to unexpected positive educational outcomes for children in affected zones. Despite the initial devastation, students in these areas saw an increase in their grades compared with those who were not exposed. These improvements were linked to targeted recovery investments, infrastructure enhancements, and provision of additional educational resources (Pacheco, 2024Pacheco, D. (2024). Exploring the Impacts of Network Improvement Communities During Times of Crisis [doctoral dissertation]. Fordham University.).

Based on the literature review and specific context of Ecuador, a significant reduction in overall academic achievement due to prolonged school closures is expected. This hypothesis is supported by existing research, indicating that extended school closures lead to considerable learning losses. In Ecuador, where in-person classes have been suspended for over a year, the impact on student learning has been particularly severe. Unlike other countries, where school closures were shorter, the prolonged disruption in Ecuador’s education system led to significant learning losses, as evidenced by our analysis of standardized test scores.

Nevertheless, the concept of educational resilience offers a deeper understanding of its impacts. While significant learning losses are anticipated, the case of the 2016 earthquake demonstrates that targeted recovery efforts can lead to educational gains even in the face of substantial adversity. This suggests that while overall academic achievement may have suffered due to prolonged school closures, there could also be instances where resilience and adaptive measures within the education system mitigate some of these negative effects. Therefore, further research is needed to fully capture the complex interplay between crisis-induced learning losses and the potential for recovery and resilience in the Ecuadorian educational context.

4. DATA AND METHODOLOGY

In Ecuador, the use of standardized tests in the education system has experienced a notable increase in response to a societal push for educational accountability. These assessments aim for many students to take the same test and compare their individual scores to determine their relative positions.

Ser Estudiante (SEST) is a process that assesses the knowledge, skills, and abilities of students in the fields of Mathematics, Language and Literature, Natural Sciences, and Social Sciences at the Elementary, Middle, and Upper levels of General Basic Education (EGB), as well as at the high school level, based on the current curriculum and the Learning Standards issued by the Ministry of Education.

The selection was carried out through sampling, a representative portion of the total student population in the country was evaluated, and the obtained results were used to represent the entire population. Students were randomly selected and the evaluation results enabled us to generate information at the level of region (Costa-Galápagos and Sierra-Amazonía), area (urban/rural), and type of sponsorship (public, semi-public, private, and municipal).

The educational system in Ecuador has seen increased use of standardized testing in response to a societal push for educational accountability. These standardized tests were designed to be administered uniformly, with equivalent items, consistent instructions, and set time allocations for all students. However, the critical issue in the Ecuadorian case lies in the equivalence of the test items as multiple test forms with varying levels of difficulty are employed. Furthermore, significant changes in educational standards have rendered the 2017, 2018, and 2019 tests incomparable to the 2021 and 2022 tests, making direct score comparisons inappropriate.

To ensure valid comparisons of student performance across different years, especially in the context of standardized tests that vary in difficulty, it is crucial to use a technique known as “score equating”. Equating allows researchers to compare scores from different test forms and years in a way that accounts for variations in test difficulty and other factors that may affect score comparability.

Score equating is a statistical process used to create scores from different test forms in order to make them comparable. This is important when test forms differ in terms of difficulty or content coverage. The goal of equating is to ensure that scores reflect the same level of ability, regardless of the test form a student took. The process involves using “anchor items” –specific questions included in multiple test forms over consecutive years. Anchor items are intended to measure the same constructs and are repeated exactly in standardized tests across different administrations. These questions were designed to measure the same knowledge and skills across the test versions. Using these anchor items, we can assess how well different test forms align with each other. Anchor items serve as a common reference point, making it possible to adjust scores from various test forms so that they can be fairly compared.

The process begins with the selection of anchor items included in standardized tests across different years. These items must have identical wording and formats to ensure consistency in their measurements. For each student, the number of anchor items that were correctly answered was counted. This was compared to the total number of anchor items in the test form the student took. This ratio helps gauge how well the student’s performance aligns with the common benchmark set by the anchor items.

Scores from the different test forms were then adjusted based on the performance of the anchor items. This adjustment accounts for the varying levels of difficulty between the test forms. For example, if a test form is more challenging, the equating process will adjust the scores to reflect a comparable level of achievement, as if the same test form is less difficult.

In this study, the equating process was conducted by the National Institute of Educational Evaluation of Ecuador (INEVAL) at the researchers’ request and supervision. As mentioned, the ratio of correctly answered anchor items to the total number of evaluated anchor items provided a basis for making scores comparable among students and across years.

A key consideration was that not all students were administered the same number of anchor items, because of the block design of the tests. Therefore, the score used for comparison accounted for the number of anchor items on each student’s specific test form. Additionally, it is crucial to note that the anchor items alone do not represent the full range of each domain (e.g., Mathematics, Language and Literature, Social Studies, and Social Sciences).

Given these factors, it was not possible to conduct the equating process at the domain level across various sublevels (elementary, middle, upper, and third year of high school). However, it was feasible to perform the equating process by sublevel, which was applied to fourth grade (elementary level), seventh grade (middle level), and tenth grade (upper level) students in General Basic Education (EGB), as well as third-year students of the Unified General Baccalaureate (BGU). The academic cycles included in this study were 2016-2017 (2017), 2017-2018 (2018), 2018-2019 (2019), and 2021-2022 (2022). Owing to the absence of anchor items in the 2020-2021 (2021) cycle, no equating process could be performed for that year.

Thus, in this study, the equating process was applied to standardized test scores from 2017, 2018, 2019, and 2022, covering fourth-, seventh-, tenth-, and third-year high school students. By establishing an adjusted “anchor” score based on the proportion of anchor items each student answered correctly, normalized for varying numbers of anchor items, this study provides a method for valid comparisons of academic achievements across these years and assesses the impact of the COVID-19 pandemic on students’ performance.

As we have seen, the database used for this study has a two-stage sampling design, with educational institutions as the primary sampling unit and students as the secondary sampling and analysis unit. The geographical coverage of this study was national. The sample was selected probabilistically for each stratum at the institutional level. To stratify the sample, INEVAL uses three key variables: regime (Costa, Sierra), area (urban, rural), and type of institution (Fiscal, Municipal, Particular, Fiscomisional). This resulted in 16 strata representing all combinations of these variables.

After collecting information from the students in each educational institution considered in the sample, the data were cleaned and figures were cross-referenced with application reports to ensure that the number of participants and the number of records in the system matched. This study spans 4 years (2017, 2018, 2019, and 2022), making it a repeated cross-sectional analysis.

This study uses an econometric approach that employs a fixed-effects regression model. This method accounts for unobserved heterogeneity across schools and students, controlling for time-invariant factors. Errors are corrected using the “cluster” method at the school level, with appropriate weighting factors applied. The fixed-effects approach provides a robust analysis by focusing on within-school variations over time, offering a more accurate understanding of the impact of COVID-19 on learning outcomes. Its aim is to examine the relationship between a set of independent variables and the dependent variable “Anchor” for 4, 7, 10 and 12 grade (high school students). In this context, the exogenous shock caused by the pandemic during the 2020-2021 period becomes a crucial component of the analysis. In the context of this study, there are no endogeneity issues, because the pandemic is considered an exogenous shock.

The model can be represented as follows:

Ecuaciión

where “EquatedScore” represents the dependent variable of interest, “T” denotes the treatment, and “X” is a vector of control variables at the student, parent, and school levels. Alpha represents school fixed effects. Three specifications were used for this analysis. In the first specification, an OLS regression analysis is performed to assess the relationship between the dependent variable “Anchor” and the treatment (“T”) as the sole independent variable. “T” takes a value of 1 for the year 2022 and 0 for the years 2017, 2018, and 2019. This initial stage allows for an investigation of the impact of the treatment on “EquatedScore” without considering other factors.

To capture unobserved heterogeneity, the model incorporated fixed effects for school and time. These fixed effects account for factors specific to each school and variations across different years. The fixed effects ensure that the analysis controls for school-specific characteristics and any trends or shocks that may differ across time.

In the second specification, school-level controls are added, including variables such as area (rural or urban) and funding source (private, public, municipal, or mixed). We calculated the school infrastructure index as an input indicator for the school system. This index is a summary measure on a scale from zero to ten, calculated by adding variables indicating whether the school has the following: water, electricity, sewage, waste management, computer center for staff, computers for teachers, Internet, science laboratory, telephone, and cafeteria. In each case where the school has the analyzed input, the value is equal to one, and zero otherwise. If a school has all the analyzed components, it will have a maximum value of ten, and if it has none, it will have zero.

The third specification includes household-level controls, including the educational level of fathers and mothers. Additionally, a set of student-level variables were included: gender, employment status, language, and grade repetition. In all three regression specifications, errors were corrected using the “cluster” method at the school level. Weighting factors were applied in all the regressions.

Potential limitations of the analysis include the lack of representation at the domain level across various sublevels (elementary, middle, upper, and third year of high school), as well as potential problems with variations in the sample design throughout the period of analysis.

5. RESULTS

In this section, we present the results of the three specifications described above to assess the impact of the pandemic on the overall academic achievements of 4, 7, 10, and 12 grade students. First, we provided descriptive statistics, including both raw and adjusted scores based on anchor items. This allowed us to highlight the importance of the equating process used in this study.

Table 1. Academic Achievements: Adjusted and Unadjusted Scores

Year	Grade	Score	Adjusted score
2017	12	706.93	0.20
2018	12	698.81	0.48
2019	12	676.47	0.45
2022	12	688.53	0.06
2017	4	700.70	0.30
2018	4	700.00	0.29
2019	4	683.39	0.28
2022	4	674.23	0.06
2017	7	703.62	0.27
2018	7	706.13	0.26
2019	7	675.44	0.24
2022	7	693.58	0.06
2017	10	705.04	0.17
2018	10	705.33	0.19
2019	10	664.18	0.19
2022	10	703.74	0.05

Source: INEVAL databases.

Upon examining the original unadjusted academic test scores of the students across different years, we observed notable trends. Interestingly, there was a decline in scores between 2017 and 2019. However, it is important to note that by 2022, there was a significant increase in scores across all grade levels. It is imperative to acknowledge that these unadjusted scores represent a misleading picture. As is evident from the data, without adjustment, the scores seem to suggest an improvement post-pandemic, which is counterintuitive. Moreover, it is crucial to recognize that the values of both the adjusted and unadjusted scores are on different scales, making direct comparisons inviable.

Meanwhile, with the adjusted test scores, we observed the performance of students on academic tests across different years. There was a noticeable improvement in scores between 2017 and 2019 for the 10th and 12th grades, whereas a slight decrease was observed for the 4th and 7th grades. However, all grades experienced a significant decline in their scores by 2022. It is important to note that the adjusted scores represent the correct responses to anchor items divided by the total number of evaluated anchor items. For example, in the case of 12th grade, we evaluated 30 items that were repeated over the four years presented –a result of 0.1, indicating that three items were correct. This equating process helps to account for variations in test forms across different administrations and ensures fair comparisons. By adjusting for these variations, we aimed to provide a more accurate representation of students’ academic achievement over time. It is important to note that these data were sourced from the INEVAL database.

To further illustrate the impact of the pandemic on academic achievement, we have included two figures. The first compares the unadjusted scores of students in 2019 and 2022 across all grade levels. The second figure shows the adjusted scores for the same period, highlighting the importance of the equating process used in this study. We aim to visually represent how the equating process corrects the misleading impression of post-pandemic score improvements, ensuring a more accurate assessment of academic achievement over time.

Figure 1: Academic achievement: unadjusted scores

Figure 1: Academic achievement: unadjusted scores. Source: INEVAL databases.

Source: INEVAL databases.

Figure 2. Academic achievement: adjusted scores

Figure 2. Academic achievement: adjusted scores. Source: INEVAL databases.

Source: INEVAL databases.

With that in mind, the model results are presented below:

Table 2. Results for all grades

	esp1	esp2	esp3
4th grade
T	–0.218	–0.214	–0.215
	0.015	0.015	0.017
N	34 777	31 622	31 622
r2	0.435	0.443	0.445
7th grade
T	–0.199	–0.204	–0.198
	0.010	0.009	0.009
N	35 641	32 545	32 545
r2	0.374	0.381	0.383
10th grade
T	–0.163	–0.162	–0.157
	0.011	0.012	0.012
N	34 734	31 720	31 720
r2	0.381	0.388	0.390
12th grade
T	–0.352	–0.367	–0.380
	0.021	0.022	0.022
N	30 616	26 274	26 274
r2	0.678	0.687	0.687

Source: Own elaboration with data from INEVAL.

In all three specifications, the negative coefficients for variable “T” indicate an adverse impact of the treatment variable, representing the year 2022, in contrast to 2017, 2018, and 2019, on academic achievements in all grades.

For example, in 4th grade, the values –0.218, –0.214, and –0.215 represent the decrease in academic achievement scores in 2022 compared to the previous years. These values, reflecting the change in average scores due to the pandemic, suggest that, on average, 4th-grade students’ academic achievements in 2022 were approximately 0.22 points lower than in the pre-pandemic years, when the average adjusted scores before was 0.29 points.

Similarly, for 7th grade, the negative coefficients of variable “T” (–0.199, –0.204, –0.198) indicate a detrimental impact of the treatment variable, reflecting the year 2022 compared to 2017, 2018, and 2019 on academic achievement. On average, 7th-grade students’ academic achievements in 2022 were approximately 0.2 points lower than in the pre-pandemic years, when the average adjusted score was 0.26 before COVID-19.

For 10th grade, on average, the academic achievement of students in 2022 was approximately 0.16 points lower compared of the pre-pandemic years, when the average adjusted score was 0.19. For 12th grade, the treatment variable also has a negative impact on academic achievement. Therefore, on average, the academic achievement of 12th-grade students in 2022 was approximately 0.38 points lower compared of the pre-pandemic years when the average adjusted scores were 0.39.

These values reflect the change in average academic achievement score due to the pandemic. Adding the context given the pre-pandemic average achievement scores, these results imply that learning outcomes were reduced practically to zero, providing a clear interpretation of the magnitude of the impact.

6. CONCLUSIONS

The COVID-19 pandemic has presented unprecedented challenges to educational systems worldwide. In Ecuador, the abrupt closure of schools has resulted in an urgent transition to emergency remote learning, profoundly affecting the nation’s education landscape. This study seeks to fill a critical gap in the literature by examining the pandemic’s impact on academic achievement in the Ecuadorian context, specifically for students in the 4, 7, 10 and 12 grades of High School.

Notably, our analysis was contingent on addressing a significant challenge: the equivalence of standardized tests conducted in different years. The use of equating processes allowed us to make meaningful comparisons between the performance of students in the years preceding the pandemic (2017, 2018, and 2019) and the year 2022, which saw the educational landscape deeply influenced by the pandemic.

Our findings highlight the consistent negative impact of the pandemic on academic achievement. Regardless of the control variables introduced, the treatment variable (“T”), representing 2022, consistently exhibited a negative coefficient. The model results show notable trends across different grade levels: on average, students’ academic achievements in 2022 were approximately 0.38 points lower compared to the pre-pandemic years for 12 graders; 0.16 for 10; 0.2 for 7th; and 0.22 for 4 graders. These findings underscore the adverse effects of the pandemic on academic performance in various grades.

Moreover, it is important to note that, unlike in other countries where school closures were short-lived, in Ecuador, the suspension of in-person classes lasted for over a year. This prolonged break from traditional schooling has had a significant impact on student learning, with some experiencing nearly complete loss. Existing research on learning loss during the pandemic supports this, showing that, even in countries with shorter closures, there have been noticeable drops in academic performance. Therefore, our results highlight the urgent need for targeted and effective interventions to address learning gaps and minimize the long-term effects of the educational crisis caused by the pandemic.

The repercussions of this decrease in academic achievement are far reaching. As economic, social, and political challenges were already evident in Ecuador prior to the pandemic, the additional strain imposed by COVID-19 only exacerbated these issues. Notably, economic growth had already stagnated and the pandemic introduced a dual shock to the economy, leading to substantial economic decline.

The study’s robust findings highlight the urgent need for targeted policy interventions to mitigate the adverse effects of the COVID-19 pandemic on academic achievements in Ecuador. The significant decrease in academic scores across all grade levels demands immediate attention from policymakers. To address this unprecedented challenge, we propose the following policy considerations.

Targeted Educational Support Programs: implementing specialized support programs for students, especially those in 12th grade who experienced an average decrease of approximately 0.347 points, to bridge the learning gaps created by the pandemic.
Digital infrastructure enhancement: investing in and expanding digital infrastructure for remote learning to ensure continued education during unforeseen disruptions, avoiding the recurrence of the substantial negative impact observed in 2022.
Socioeconomic support: Recognizing the exacerbated effects on vulnerable populations and considering socioeconomic support measures to minimize disparities in academic achievement, particularly for students from low-income backgrounds.
Long-term educational resilience planning: Developing comprehensive, long-term resilience plans for the education system, including measures for standardized testing equivalence and adaptive strategies to navigate future crises without compromising learning outcomes.
Develop and implement frameworks for data-driven decision-making within educational institutions, track student performance, identify trends, and tailor interventions according to individual needs.
Ensure that educational resources are distributed equitably between urban and rural areas. Prioritize funding and support for underserved regions to address educational disparities intensified by the pandemic. This includes not only technological resources but also educational materials, teacher training, and infrastructure improvements.
Enhance community and parental involvement in education by providing resources and training to support students’ learning at home. Develop programs that facilitate communication between schools and families, ensuring that parents are equipped with tools and knowledge to support their children’s education effectively.
Improve teacher training programs to include strategies for effective remote and blended learning. Provide ongoing professional development opportunities that focus on innovative teaching methods, digital tools, and best practices for engaging students in diverse learning environments.
Robust monitoring and evaluation frameworks should be established to assess the effectiveness of educational policies and interventions. Regularly review and adjust strategies based on evidence and feedback to ensure continuous improvement and responsiveness to emerging needs.
Encouraging and funding research on educational practices and innovations that can address the challenges highlighted by the pandemic. Support initiatives that explore new teaching methods, technologies, and policy approaches to improve educational outcomes and resilience.

This study contributes to the literature by shedding light on the pandemic’s specific impact on Ecuador’s education system and emphasizing the need for tailored policy responses. It offers valuable insights into the educational implications of the COVID-19 pandemic in Ecuador and serves as a foundation for future research on educational policy during extraordinary circumstances. The negative impact on academic achievement underscores the urgency of targeted interventions to support students, especially those from vulnerable backgrounds, and bridge the educational gaps that have widened during this unprecedented global crisis. Future research should focus on evaluating the impact of these new interventions on student achievement.

ACKNOWLEDGMENTS

We thank the anonymous reviewers and editorial team of the journal for their suggestions and comments on the first version of this article.

REFERENCES

Aguaded, I., Vizcaíno-Verdú, A., García-Prieto, V. & De-Casas-Moreno, P. (2023). The impact of post-pandemic learning loss on education development: A systematic review. Review of Communication Research, (11), 172-189.

Alasino, E., Ramírez, M. J., Romero, M., Schady, N. & Uribe, D. (2024). Learning losses during the COVID-19 pandemic: Evidence from Mexico. Economics of Education Review, (98), article 102492. https://doi.org/10.1016/j.econedurev.2023.102492

Angrist, N., de Barros, A., Bhula, R., Chakera, S., Cummiskey, C., DeStefano, J., Floretta, J., Kaffenberger, M., Piper, B. & Stern, J. (2021). Building back better to avert a learning catastrophe: Estimating learning loss from COVID-19 school shutdowns in Africa and facilitating short-term and long-term learning recovery. International Journal of Educational Development, (84), article 102397. https://doi.org/10.1016/j.ijedudev.2021.102397

Ardington, C., Wills, G. & Kotze, J. (2021). COVID-19 learning losses: Early grade reading in South Africa. International Journal of Educational Development, (86), article 102480. https://doi.org/10.1016/j.ijedudev.2021.102480

Azevedo, J. P., Hasan, A., Goldemberg, D., Geven, K. & Iqbal, S. A. (2021). Simulating the potential impacts of COVID-19 school closures on schooling and learning outcomes: A set of global estimates. The World Bank Research Observer, 36(1). https://doi.org/10.1093/wbro/lkab003

Baez, J., de la Fuente, A. & Santos, I. V. (2010). Do Natural Disasters Affect Human Capital? An Assessment Based on Existing Empirical Evidence [IZA Discussion Paper n.º 5164]. IZA. http://dx.doi.org/10.2139/ssrn.1672172

Bartholo, T. L., Koslinski, M. C., Tymms, P. & Castro, D. L. (2023). Learning loss and learning inequality during the COVID-19 pandemic. Ensaio: Avaliação e Políticas Públicas Em Educação, (31), article e0223776. https://doi.org/10.1590/S0104-40362022003003776

Bertoletti, A., Cannistrà, M., Soncin, M. & Agasisti, T. (2023). The heterogeneity of COVID-19 learning loss across Italian primary and middle schools. Economics of Education Review, (95), article 102435. https://doi.org/10.1016/j.econedurev.2023.102435

Blaskó, Z., da Costa, P. & Schnepf, S. V. (2022). Learning losses and educational inequalities in Europe: Mapping the potential consequences of the COVID-19 crisis. Journal of European Social Policy, 32(4), 361-375. https://doi.org/10.1177/09589287221091687

Contini, D., Di Tommaso, M. L., Muratori, C., Piazzalunga, D. & Schiavon, L. (2022). Who lost the most? Mathematics achievement during the COVID-19 pandemic. The BE Journal of Economic Analysis & Policy, 22(2), 399-408. https://doi.org/10.1515/bejeap-2021-0447

De Medeiros Miranda, E. & Baum, D. R. (2024). COVID-19 learning loss and recovery in Brazil: Assessing gaps across social groups. Education Policy Analysis Archives, (32). https://doi.org/10.14507/epaa.32.8082

Engzell, P., Frey, A. & Verhagen, M. D. (2021). Learning loss due to school closures during the COVID-19 pandemic. Proceedings of the National Academy of Sciences of the United States of America, 118(17), artículo e2022376118. https://doi.org/10.1073/pnas.2022376118

Gore, J., Fray, L., Miller, A., Harris, J. & Taggart, W. (2021). The impact of COVID-19 on student learning in New South Wales primary schools: An empirical study. The Australian Educational Researcher, (48), 605-637. https://doi.org/10.1007/s13384-021-00436-w

Guariso, A. & Nyqvist, M. B. (2023). The impact of the COVID-19 pandemic on children’s learning and wellbeing: Evidence from India. Journal of Development Economics, (164), article 103133. https://doi.org/10.1016/j.jdeveco.2023.103133

Hammerstein, S., König, C., Dreisörner, T. & Frey, A. (2021). Effects of COVID-19 –related school closures on student achievement– a systematic review. Frontiers in psychology, (12), article 746289. https://doi.org/10.3389/fpsyg.2021.746289

Intriago, R. (2021). Revisión de mecanismos de adaptación de los sistemas educativos a nivel regional e internacional a la emergencia sanitaria y estudio sobre la adaptación de la gestión docente en el contexto de la crisis sanitaria generada por al COVID-19. Ministerio de Educación de Ecuador. https://educacion.gob.ec/wp-content/uploads/downloads/2022/02/Nota-tecnica-estudio-sistemas-educativos-practicas-docentes-en-emergencia-sanitaria.pdf

Jakubowski, M., Gajderowicz, T. & Patrinos, H. A. (2023). Global learning loss in student achievement: First estimates using comparable reading scores. Economics Letters, (232), article 111313. https://doi.org/10.1016/j.econlet.2023.111313

Kilenthong, W. T., Boonsanong, K., Duangchaiyoosook, S., Jantorn, W. & Khruapradit, V. (2023). Learning losses from school closure due to the COVID-19 pandemic for Thai kindergartners. Economics of Education Review, (96), article 102455. https://doi.org/10.1016/j.econedurev.2023.102455

Kuzmanic, D., Meneses, F., Valenzuela, J. P., Rodríguez, P. & Claro, S. (2024). Inequalities in learning loss during the COVID-19 pandemic in Chile: the significance of school effectiveness. School Effectiveness and School Improvement, 35(2), 95-115. https://doi.org/10.1080/09243453.2024.2328016

Lichand, G., Doria, C. A., Leal-Neto, O. & Cossi Fernandes, J. P. (2022). The impacts of remote learning in secondary education during the pandemic in Brazil. Nature Human Behaviour, (6), 1079-1086. https://doi.org/10.1038/s41562-022-01350-6

Lustig, N., Martínez Pabon, V., Neidhöfer, G. & Tommasi, M. (2023). Short and Long-Run Distributional Impacts of COVID-19 in Latin America. Economía LACEA Journal, 22(1), 96-116. https://doi.org/10.31389/eco.3

Maldonado, J. E. & De Witte, K. (2020). The effect of school closures on standardised student test outcomes [discussion paper series DPS20.17]. FEB Research Report Department of Economics. https://lirias.kuleuven.be/3189074

Monroy-Gómez-Franco, L., Vélez-Grajales, R. & López-Calva, L. F. (2022). The potential effects of the COVID-19 pandemic on learnings. International journal of educational development, (91), article 102581. https://doi.org/10.1016/j.ijedudev.2022.102581

Moscoviz, L. & Evans, D. K. (2022). Learning loss and student dropouts during the COVID-19 pandemic: A review of the evidence two years after schools shut down [Working Papers 609]. Center for Global Development. https://www.cgdev.org/publication/learning-loss-and-student-dropouts-during-covid-19-pandemic-review-evidence-two-years

Pacheco, D. (2024). Exploring the Impacts of Network Improvement Communities During Times of Crisis [doctoral dissertation]. Fordham University.

Patrinos, H. A., Vegas, E. & Carter-Rau, R. (2022). An Analysis of COVID-19 Student Learning Loss [Policy Research Working Paper n.º 10033]. World Bank Group. https://hdl.handle.net/10986/37400

Reimers, F. M. (2022). Learning from a pandemic: The impact of COVID-19 on education around the world. In F. M. Reimers (ed.), Primary and secondary education during COVID-19: Disruptions to educational opportunity during a pandemic (pp. 1-37). Springer. https://doi.org/10.1007/978-3-030-81500-4

Relyea, J. E., Rich, P., Kim, J. S. & Gilbert, J. B. (2023). The COVID-19 impact on reading achievement growth of Grade 3-5 students in a U. S. urban school district: variation across student characteristics and instructional modalities. Reading and Writing, (36), 317-346. https://doi.org/10.1007/s11145-022-10387-y

Schult, J., Mahler, N., Fauth, B. & Lindner, M. A. (2022). Did students learn less during the COVID-19 pandemic? Reading and mathematics competencies before and after the first pandemic wave. School Effectiveness and School Improvement, 33(4), 544-563. https://doi.org/10.1080/09243453.2022.2061014

Schuurman, T. M., Henrichs, L. F., Schuurman, N. K., Polderdijk, S. & Hornstra, L. (2023). Learning loss in vulnerable student populations after the first COVID-19 school closure in the Netherlands. Scandinavian Journal of Educational Research, 67(2), 309-326. https://doi.org/10.1080/00313831.2021.2006307

Singh, A., Romero, M. & Muralidharan, K. (2023). COVID-19 Learning Loss and Recovery: Panel Data Evidence from India. Journal of Human Resources, 59(5). https://doi.org/10.3368/jhr.0723-13025R2

Tang, K. H. D. (2023). Impacts of COVID-19 on primary, secondary and tertiary education: a comprehensive review and recommendations for educational practices. Educational Research for Policy and Practice, 22(1), 23-61. https://doi.org/10.1007/s10671-022-09319-y

Tomasik, M. J., Helbling, L. A. & Moser, U. (2021). Educational gains of in-person vs. distance learning in primary and secondary schools: A natural experiment during the COVID-19 pandemic school closures in Switzerland. International Journal of Psychology, 56(4), 566-576. https://doi.org/10.1002/ijop.12728

Uğraş, M., Zengin, E., Papadakis, S. & Kalogiannakis, M. (2023). Early childhood learning losses during COVID-19: Systematic review. Sustainability, 15(7), article 6199. https://doi.org/10.3390/su15076199

UNESCO (2020). How many students are at risk of not returning to school? https://unesdoc.unesco.org/ark:/48223/pf0000373992

Wolf, S., Aurino, E., Suntheimer, N. M., Avornyo, E. A., Tsinigo, E., Jordan, J., Samanhiya, S., Aber, J. L. & Behrman, J. R. (2022). Remote learning engagement and learning outcomes during school closures in Ghana. International Journal of Educational Research, (115), article 102055. https://doi.org/10.1016/j.ijer.2022.102055