COVID-19 is Poised to Exacerbate the Learning Crisis, Evidence from Long-term School Closures in Pakistan

written by Marla Spivack

The consequences of the COVID-19 pandemic and of the policies adopted to mitigate its spread will be drastic everywhere and particularly grim in low- and middle-income countries.1 As economies shrink, up to a decade of progress in poverty reduction could be undone. As health systems struggle, millions could go without treatment for preventable infectious diseases and childhood vaccination rates could plummet. And as schools are shuttered, millions of children will fall further behind in school with devastating effects on their outcomes later in life.

A leaky roof is a small problem on a sunny day and a big, urgent problem on a rainy one. The COVID-19 pandemic is not just a rainy day for education systems in low- and middle-income countries, it is a one-hundred-year flood. A new paper from the RISE Pakistan Country Research Team (CRT) shows that children in these countries are especially vulnerable since they are embedded in already dysfunctional education systems. The paper also shows that some of the worst damage to children’s long term learning from these closures may come after schools start up again. If children have fallen behind, they may never catch up even after they return to school.

New study of the learning effects of the 2005 Pakistan Earthquake has important lessons for COVID-19 related school closures

In the paper, RISE Pakistan Team members (Tahir Andrabi of LUMS and Pomona College, and Jishnu Das and Benjamin Daniels of Georgetown University) use a survey conducted four years after the 2005 Pakistan earthquake to measure how the disaster affected children’s learning and other human capital outcomes.

The 2005 earthquake was severe, with an epicenter in the Himalayan region, it registered 7.6 on the Richter scale, caused more than 80,000 deaths, and destroyed the vast majority of infrastructure in the area. As a result, children were out of school in the affected areas for an average of 14 weeks.

The researchers studied the effect of the school closures on children’s learning, and the findings have important implications for how we can expect the COVID-19 related closures to affect learning outcomes for children in education systems with similar constraints to Pakistan’s—and what can be done to mitigate these effects.

If countries reopen to business as usual, short-term school closures can produce outsized, long-run learning loss

The results of the paper add further evidence to the established finding that interruptions to human capital accumulation due to disasters can be severe. In this case, they show that the disaster can leave long-lasting scars on children, even when government interventions compensate households for the shock and facilitate a speedy economic recovery.

Four years after the earthquake, school enrolments had fully rebounded, infrastructure had been rebuilt, household incomes had rebounded, and adult’s health outcomes had returned to pre-quake levels, but children were still suffering the effects of school closures. Test scores of children in the affected areas put them 1.5 to 2 years behind their peers in unaffected. This lost learning could result in children earning 15% less in every year of their adult lives. 

Losses to human capital may well continue to accumulate further after children return to school, if they fall behind and are not able to catch up with the curriculum. Children fall behind, the over-ambitious curriculum races ahead, and children can’t catch up. In the aftermath of the earthquake, Andrabi et al find that school closures accounted for only 10% of the gap in test scores. Much more was lost after children returned to school. This is likely because children fell behind the standard curriculum during the closure period and then failed to catch up. This reinforces findings from RISE work in India which shows that when children fall behind in school they struggle to catch up later, due in part to overambitious curricula.

If steps are not taken to prevent children from falling farther behind when they return to school, the crisis will likely further exacerbate inequalities

Many commentators have noted that COVID is likely to exacerbate inequalities. The findings from the aftermath of the earthquake suggest that long term school closures can reduce inter-generational educational mobility. Andrabi et al found that in the aftermath of the earthquake children with educated mothers were fully able to fully catch up with their peers from unaffected areas. This finding is troubling considering pre-COVID work from the RISE Ethiopia CRT, which shows that first generation learners are already among the most disadvantaged in terms of learning outcomes.

Pedagogical approaches that focus on teaching children where they are will be critical to averting permanent learning deficits

The compounding of learning loss that was seen in the years following the earthquake in Pakistan suggests that assessing children when they return to school and teaching them from where they are will be vital to mitigating the long-term effect of school closures. The effectiveness of approaches like these often called teaching at the right level, have been well established. When they reopen, school systems should leverage approaches like these, and focus on helping children catch up and solidify their basic skills.

Reopening to business as usual would compound the disastrous effects of the COVID related closures. This catastrophe can be avoided if education systems embrace the opportunity to take on systemic changes, as part of their response to the crisis.

Footnotes

1 Incidentally, the May 23 edition also includes a laudatory and well deserved profile of RISE Nigeria CRT PI Leonard Watchekon and RISE implementing partner African School of Economics, which Watchekon founded.

This blog first appeared on the RISE blog.

Charting a new course: Education systems after COVID-19

Written by Dzingai Mutumbuka and Marla Spivack

An ordinary classroom in an African school.

We know that time away from school due to COVID-19 has undermined learning. Children are depending on education leaders – from high level officials to classroom teachers – to start planning now for a new focus on foundational skills. With bold action, and clear focus education systems can mitigate the long term effects of this crisis and set out on a new course towards sustainable improvement in learning. 

The global COVID-19 pandemic has upended our lives and our education systems. Education has been adversely impacted in two significant ways: schools have been closed, in some cases for a whole year; and economic production, the major source of education funding through budgets, has declined precipitously.  In Africa, where there was already a learning crisis characterized by millions of children out of school and for those enrolled completing primary education without minimum competency in literacy – 86% of children reach the end of primary school without basic literacy according to the World Bank – and numeracy, COVID-19 school closures have turned the crisis into a nightmare.

Insights from research on education systems suggest that by making a system-wide commitment to prioritizing foundational skills, assessing children’s learning levels when schools reopen, and adapting instruction to children’s learning levels, education systems can mitigate learning loss and even come back stronger than before.

Several features of the learning crisis set the stage for COVID-19 school closures to severely impact long-term learning outcomes. Learning profiles in African countries are flat, meaning that children acquire little new learning with each additional year in school. Many fail to master foundational skills early on and then struggle to keep up or catch up as the curriculum progresses. For children who learn little during school closures, catching up will be even more challenging. Learning levels in classrooms can vary widely and are likely to likely to increase in the aftermath of COVID-19 closures

COVID-19 closures are also poised to exacerbate learning inequality. The majority of children in African countries do not have access to virtual learning, but those that do are likely to be urban and better off. Better off children are also more likely to have parents who can support and supplement remote learning from the school system.  

Continue reading Charting a new course: Education systems after COVID-19

IPP Program Journey: Early Childhood Education in Brazil

Guest blog written by Beatriz Abuchaim

This is a blog series written by the alumni of the Implementing Public Policy Executive Education Program at the Harvard Kennedy School. Participants successfully completed this 7-month blended learning course in December 2019. These are their learning journey stories.

Screen Shot 2020-07-30 at 5.04.58 PM

A big headache. It was what I felt in the introductory class at Kennedy School. It was not my first experience at Harvard. I had taken a course in 2018 at Center on the Developing Child, but it didn’t have the same pressure I was feeling as a Public Policy Implementation student. My company was paying for me to be there and Harvard gave me a partial scholarship. While I was listening to Matt introduce the course to us, my painful head did not stop with mixed thoughts: “I should give my best to show I deserve all this investment. I feel so special to represent my country in such a selected group of people. I am worried if I will be able to implement my project”.

When I am overwhelmed with so many feelings my body complains with a migraine. Then I have to stop. It is a way to tell myself: take it easy. Breathe. Calm down. After the first night in pain in Cambridge, I could slow things down. During the week, I felt motivated by the professors and engaged with colleagues. Always feeling exhausted with so many assignments and tasks, but fulfilled. I came back home feeling empowered and secure. And missing my PDIA folks already.

My problem in a few words

Early Childhood Education (ECE) in Brazil currently covers 34% of 0 to 3-year-old population and 93% of 4 to 6-year-old population. These percentages represent eight million children enrolled in ECE.  The public sector is responsible for 70% of enrollments. In the past 10 years,  we have had a significant increase in the number of enrollments, but with budget limitations, so the quality of services may vary quite substantially nationwide. The municipalities, which are responsible for implementing  ECE, are struggling to improve service quality.

Although Early Childhood Education in Brazil has many problems regarding the quality of services, we still do not have a national assessment that could provide data about children’s development and learning environment. Without data, policy managers struggle to plan, improve and make decisions about ECE.

My problem is “lack of systematized information about ECE quality”. Working with PDIA and presenting my fishbone to as many people as I possibly could ended up with 12 causes and 16 subcauses for the problem.

Screen Shot 2020-07-30 at 5.10.28 PM

Here I will present three that were my entry points during this year, in other words, the causes I selected to try to address first:

  1.  The education area has criticisms and resistances about ECE assessment, mainly because the teachers are afraid that the results are used in a negative way: to punish them or stigmatize the children with low results.
  2. Municipal ECE managers are not used to working with data. They do not have access to systematized data, so they did not develop the skills to analyze results and use those to inform policies.
  3. There are no instruments adapted for the Brazilian context. Until now the assessments implemented in the country used translated instruments, which were not totally suitable for ECE in Brazil.

Continue reading IPP Program Journey: Early Childhood Education in Brazil

PDIA Course Journey: Local Problems, Local Solutions to the Indonesian Education Sector

Guest blog written by George Adam Sukoco Sikatan, Lanny Octavia, Sarah Ayu, Wahyu Setioko

This is a team of development practitioners who work for INOVASI and DFAT in Indonesia. They successfully completed the 15-week Practice of PDIA online course that ended in May 2019. This is their story.

Screen Shot 2019-12-06 at 12.07.15 PM

It is at last the final week of the course, and we say this full of gratitude and relief. None of us had anticipated just how intense and demanding this course was going to be, from the essential (and optional) reading, individual and groups assignments, to reflection exercises and graded discussions; needless to say they were onerous! At the same time, the abundance of knowledge was exciting and overwhelming.

Working in the public/development sector, in a large, populous country such as Indonesia, the 4 of us often come across bewildering, deeply rooted problems that seem just impossible to resolve. The PDIA approach shines a positive light on this situation and more importantly, confidence to overcome them. We learned to deconstruct a problem into smaller pieces and find the root cause using a relatively simple, yet powerful, tool namely the 3A analysis (Authority, Acceptance and Ability).

Another key takeaway from our group is the importance of reflective process to help us look into failures, challenges and feedback as opportunity to grow and construct (or when necessary, deconstruct all over). This methodology taught us to become better listeners, to arrive in a situation with an open mind instead of a will to impose external practices. These reflections and adaptations to the local context, allow us to remain relevant both to the problem that we are trying to solve and towards our beneficiaries.

This course also reminded us of the importance of collaboration and coordination with a broad range of stakeholders. We understand now that multiple perspectives, incentives and even interests are actually useful in defining problems and formulating solutions. Sharing a common goal at the beginning of the work had founded a sense of belonging and motivation for all team members, even when the time is hard and problem becomes more challenging.

Continue reading PDIA Course Journey: Local Problems, Local Solutions to the Indonesian Education Sector

PDIA Course Journey: Solving the Wicked Hard Problem of Education Quality in Indonesia

Guest blog written by Rina Arlianti, Stephanie Carter, Murni Hoeng, Siti Ubaidah Idrus, Susanti Sufyadi, Aaron W Watson.

This is a team of six development practitioners working for Indonesia’s Ministry of Education and Culture, the Tanoto Foundation, the Australian supported INOVASI program, and Australian Embassy, Jakarta. They successfully completed the 15-week Practice of PDIA online course that ended in May 2019. This is their story.

The Harvard BSC’s PDIA course has been an exciting journey for all of us. We began the course full of excitement and hope – with most of group members having not met before. We were one of four groups participating from Indonesia, all focused on the issue of education qualityOver the course of 15 weeks, we navigated the twists and turns of the PDIA process, putting key concepts to the test in the field of basic education in Indonesia.

Early on, once we had settled into our group dynamic, we settled on our problem statement:

Learning outcome quality in Indonesian primary schools is still low (low scores in international standardised student tests)

As we progressed, we gained several key insights and takeaways about our problem and the course. Through group discussion and debate, drawing on perspectives from working both within and outside the government system, we settled on the following six key sub-causes for low learning outcome quality in Indonesian primary schools:

  1. Measures of learning are weak (including the use of formative assessment, due to low teacher knowledge)
  2. Teaching/learning process is ineffective (with teachers lacking inadequate skills and knowledge of how to use learning media, to increase student engagement)
  3. Parents are already satisfied with the status quo (there is often low demand for changes to the system, as parents do not know what good teaching looks like)
  4. Lack of learning books for children (due to cumbersome book supply processes at the national level)
  5. Many teachers don’t use digital technology in classrooms (creating missed opportunity for enhanced learning)
  6. Policies that address education quality are not implemented well (and instead focus on physical infrastructure, or if they do exist, are not socialised well in a decentralised system)

Continue reading PDIA Course Journey: Solving the Wicked Hard Problem of Education Quality in Indonesia

PDIA Course Journey: Lack of Student Engagement in Bastar District, India

Guest blog written by Nikhilesh Hari, Poona Verma, Sadashiv N., Vijay Siddharth Pillai

This is a team of four development practitioners working for the PMRDF in India and an M.Phil student in the UK. They successfully completed the 15-week Practice of PDIA online course that ended in May 2019. This is their story.

Screen Shot 2019-10-20 at 9.11.54 PM.png

We began the course with a feeling that the approach which we are going to learn is going to be unique. As we progressed through the initial weeks, we realized that it’s a common sensical approach to solve problems. However we realized that the common sensical approach is rarely followed. We also realized while operationalizing the approach that it is not easy at all and requires a lot of perseverance.

Some of the key takeaways from this course are: Continue reading PDIA Course Journey: Lack of Student Engagement in Bastar District, India

PDIA Journey: The Mozambique School Lunch Initiative

Guest blog written by Cara Myers

Cara Myers is the Co-founder and Executive Director of the Mozambique School Lunch Initiative (MSLI).  She learned about the PDIA approach by taking two courses at the Harvard Kennedy School as part of her Master’s in Public Administration in International Development (MPA/ID) program. She then began applying more of the concepts directly with the MSLI team. This is her PDIA story.

It was March of 2016 and the rains had completely failed for a second year in southern Mozambique. Farming families had no crops. Children were missing school to dig up river roots to eat. Teachers were sending students home because they were “too hungry to learn anything.” Even in normal years, child malnutrition and poor school participation are major issues in Mozambique. This is one of those big, complex problems that is caused by a myriad of interrelated causes and sub-causes that are difficult to disentangle and prioritize.

Screen Shot 2019-09-23 at 4.36.28 PM

So there we were, myself, Talvina Ualane and Roberto Mutisse, all of us former colleagues who had worked together for a disaster relief nongovernmental organization in Mozambique in the past and felt deeply motivated to do something to help people affected by this crisis. But, where did we even begin?

We started with what we could do. This is one of the key aspects of the triple-A framework used in PDIA, which stresses that the space for change must include three key factors: authority, acceptance, and ability. PDIA also emphasizes moving to action quickly rather than taking a long time to try and plan everything out before starting to work. By deconstructing the problem into small, manageable bits, it creates points of entry whereby you can start addressing one of the causes or sub-causes of the problem and build the capacity to do more from there.

Continue reading PDIA Journey: The Mozambique School Lunch Initiative

The problem with ‘best practice’: using PDIA to find solutions for Indonesian education

Guest blog by Mark Heyward

Screen Shot 2019-09-19 at 9.48.24 AM

Much is made these days of doing development differently, of adaptive programming, and thinking and working politically. Devpolicy Blog featured a series of articles on this topic in September 2018. But do these approaches work?

One program that has embraced adaptive programming is Innovation for Indonesia’s School Children (INOVASI). The program, which began in 2016, is a partnership between the Australian Department of Foreign Affairs and Trade (DFAT) and the Indonesian Ministry of Education and Culture. It is being implemented by Palladium. INOVASI has adopted the problem-driven iterative adaptation (PDIA) approach to help Indonesian government partners find out what works to improve learning outcomes.

Continue reading The problem with ‘best practice’: using PDIA to find solutions for Indonesian education

Developing Country Education Systems that Learn

written by Lant Pritchett

Change is hard.  It is hard for individuals.  It is extra hard for organizations.  Change is especially hard for organizations when they have been successful.  Organizations often develop strategies, norms, and practices that are tailored to produce success in a particular activity or context.  When those strategies are successful, organizations have an especially difficult time to create and manage change that is not simply “more of the same, better.”

This is true even of large, successful, well-managed private sector organizations facing (organizational) life or death consequences.

The Big Store recounts how Sears, a veritable American retailing behemoth—accounting in the early 1970s for one percent of all US GDP—fell into a crisis and how incredibly hard it was to turn the organization around.  Even when people could see the organizational crisis it was often the people who were the very best at doing what made the organization a success in the past who were in top management positions—and hence those least likely to be able to recognize the need for, plan out, and lead change.

Continue reading Developing Country Education Systems that Learn

PDIA Course Journey: Tackling the problem of basic education in remote areas of Indonesia

Guest blog written by Annisaa Rachmawati, Agusti Padmanisa, Yossy Rachmatillah, and Senza Arsendy.

This is a team of four development practitioners working for an education program in Indonesia, INOVASI, that aims to find out ‘what works’ (and conversely what does not work) to improve student learning outcomes in literacy and numeracy in basic education. They are a multidisciplinary team of officers working in communications, program implementation, monitoring and evaluation, and operations unit. They successfully completed the 15-week Practice of PDIA online course that ended in May 2019. This is their story.

Screen Shot 2019-08-15 at 6.47.37 AM

The term PDIA is something that our team is familiar with, in fact it’s a buzzword we hear everyday at work. Our project uses PDIA as its underlying approach, yet there seems to be different interpretations and debates around how it should be translated into program implementation. Having observed this notion for a while, we decided to enroll in PDIA Online Course to learn rigorously about the approach. We were convinced that this course will equip us with practical knowledge to actually do what we preach in our project.

There are four principles which encompasses PDIA. First, we need to ensure that our intervention is “problem driven” instead of solutions driven. Second, we need to engage relevant stakeholders and create environment which allows for “authorization of positive deviance”. Third, we need to foster experiential learning through “iteration and adaptation”. Last, we “scale through diffusion” successful interventions for reform to be sustainable.

The problem we are trying to tackle is “early grade students in remote areas in Indonesia have difficulties learning to read”, a persisting issue our country has been struggling for decades despite the many efforts collectively put by the government, donor programs, and education practitioners. Policies and best practices (either locally nominated or externally imported) seem to be successful in a short period of time, deceiving us into thinking that we might have solved this problem for good. Not long after specific project or intervention is completed, the same problem reoccurred – leading us right back into capability traps. (Isomorphic mimicry alert!)

Continue reading PDIA Course Journey: Tackling the problem of basic education in remote areas of Indonesia