Complexities of multiple stakeholders in developing hydroelectricity in Pakistan

Guest blog by Masood Ul Mulk

I lead a public service organization (nonprofit) working in the northwest border regions of Pakistan, bordering Afghanistan, known for its turbulence, delivering development and humanitarian aid. The government presence is thin on the ground and service delivery in the region remains poor. Government policies change rapidly, as individuals and personalities change in government. Institutional culture is weak in the area and policy and implementation revolve around networks and social relationships. Conflict, space allowed to public service organisations, turf issues between civilian and military authorities, conservative culture, tribal values, sectarian divides, all add up to the uncertainty and complexity of working in the region. As practitioners we face the conflicting challenge of, on one hand meeting the needs of the poor and vulnerable communities in an uncertain and complex environment; while on the other hand satisfying policy makers and donors, who because of their training and accountability requirements design policy solutions which are rigid and linear to address these problems with little success.  For us the challenge is explaining to them the complex situation on the ground and the need for an iterative, adaptive and learning approach to address the complexity.  Reading about PDIA, had convinced me that exposure to the course on implementing public policy at Harvard will help me better understand where the policy makers and donors are coming from, and how I should be convincing them to adopt a radically different solution to the intractable problems on the ground which was based on responsiveness, iteration and learning. I also know that if I, a practitioner on the border regions of Pakistan, say this it will carry very little weight, but if I have the Kennedy School to back me up it will be a different proposition altogether. In this sense the course was of immense help to me and to my organization. It clarified concepts and gave me the tools to address such issues in a better way.

Continue reading Complexities of multiple stakeholders in developing hydroelectricity in Pakistan

Learning about Economic Growth in the Middle of a Pandemic

Guest blog written by Beenish Amjad

I was afraid that the onslaught of the pandemic would have devastating impact on a developing country like Pakistan which has a strong history of sluggish economic growth and long-standing need-base relationship with the IMF. The economy in 2019 was weakening and poverty (headcount poverty ratio 24.3) and inequality (Gini coefficient of 33.5) was rising. COVID-19 has not only pushed more people to the below-poverty line but also strained the Federal budget due to the substantial allocation to the Conditional Cash Transfer Program (Benazir Income Support Program now Ehsaas).

In the midst of the all this, I decided to take the Program of Leading Economic Growth, with Professor Ricardo Haussmann and Matt Andrews, to evaluate the barriers to the economic growth of Pakistan and learn about the strategies of overcoming them. In my opinion and as Professor Matt said, “we should focus on growth as means and not end”. The past 10 weeks journey, though strenuous, provided the learning experience of years. Throughout the program, wide and diverse range of topics were covered without compromising on the depths of the contents. These topics helped a great deal to understand those issues which have become the binding constraints to the economic growth. Several tools were taught to synthesize and breakdown these binding constraints to reach at the key issue hindering the growth. Some of the most useful tools included economic complexity index, fishbone diagram, PDIA and diagnostic tree. One tool which influenced me a great deal is “the Atlas of Economic Complexity”. Despite my previous education in Economics, I didn’t come across any tool which as user-friendly and so enormously comprehensive. It enabled me to witness the trajectory of the developments in the Exports and Imports of Pakistan and also to compare it with other countries. The snapshots of the export’s basket import basket and economic complexity ranking (along with the rational as explained by Professor Haussmann) was a thorough learning experience for me. 

Obviously, it is very difficult to highlight one issue which we consider as binding constraints to the growth. With the help of above tools, principles of binding constraint and case studies, I identified sluggish economic growth due to persistent current account deficit as economic challenge of Pakistan. With the help of further techniques, the problem was narrowed down to sluggish or no growth in exports.  

Continue reading Learning about Economic Growth in the Middle of a Pandemic

COVID-19 is Poised to Exacerbate the Learning Crisis, Evidence from Long-term School Closures in Pakistan

written by Marla Spivack

The consequences of the COVID-19 pandemic and of the policies adopted to mitigate its spread will be drastic everywhere and particularly grim in low- and middle-income countries.1 As economies shrink, up to a decade of progress in poverty reduction could be undone. As health systems struggle, millions could go without treatment for preventable infectious diseases and childhood vaccination rates could plummet. And as schools are shuttered, millions of children will fall further behind in school with devastating effects on their outcomes later in life.

A leaky roof is a small problem on a sunny day and a big, urgent problem on a rainy one. The COVID-19 pandemic is not just a rainy day for education systems in low- and middle-income countries, it is a one-hundred-year flood. A new paper from the RISE Pakistan Country Research Team (CRT) shows that children in these countries are especially vulnerable since they are embedded in already dysfunctional education systems. The paper also shows that some of the worst damage to children’s long term learning from these closures may come after schools start up again. If children have fallen behind, they may never catch up even after they return to school.

New study of the learning effects of the 2005 Pakistan Earthquake has important lessons for COVID-19 related school closures

In the paper, RISE Pakistan Team members (Tahir Andrabi of LUMS and Pomona College, and Jishnu Das and Benjamin Daniels of Georgetown University) use a survey conducted four years after the 2005 Pakistan earthquake to measure how the disaster affected children’s learning and other human capital outcomes.

The 2005 earthquake was severe, with an epicenter in the Himalayan region, it registered 7.6 on the Richter scale, caused more than 80,000 deaths, and destroyed the vast majority of infrastructure in the area. As a result, children were out of school in the affected areas for an average of 14 weeks.

The researchers studied the effect of the school closures on children’s learning, and the findings have important implications for how we can expect the COVID-19 related closures to affect learning outcomes for children in education systems with similar constraints to Pakistan’s—and what can be done to mitigate these effects.

If countries reopen to business as usual, short-term school closures can produce outsized, long-run learning loss

The results of the paper add further evidence to the established finding that interruptions to human capital accumulation due to disasters can be severe. In this case, they show that the disaster can leave long-lasting scars on children, even when government interventions compensate households for the shock and facilitate a speedy economic recovery.

Four years after the earthquake, school enrolments had fully rebounded, infrastructure had been rebuilt, household incomes had rebounded, and adult’s health outcomes had returned to pre-quake levels, but children were still suffering the effects of school closures. Test scores of children in the affected areas put them 1.5 to 2 years behind their peers in unaffected. This lost learning could result in children earning 15% less in every year of their adult lives. 

Losses to human capital may well continue to accumulate further after children return to school, if they fall behind and are not able to catch up with the curriculum. Children fall behind, the over-ambitious curriculum races ahead, and children can’t catch up. In the aftermath of the earthquake, Andrabi et al find that school closures accounted for only 10% of the gap in test scores. Much more was lost after children returned to school. This is likely because children fell behind the standard curriculum during the closure period and then failed to catch up. This reinforces findings from RISE work in India which shows that when children fall behind in school they struggle to catch up later, due in part to overambitious curricula.

If steps are not taken to prevent children from falling farther behind when they return to school, the crisis will likely further exacerbate inequalities

Many commentators have noted that COVID is likely to exacerbate inequalities. The findings from the aftermath of the earthquake suggest that long term school closures can reduce inter-generational educational mobility. Andrabi et al found that in the aftermath of the earthquake children with educated mothers were fully able to fully catch up with their peers from unaffected areas. This finding is troubling considering pre-COVID work from the RISE Ethiopia CRT, which shows that first generation learners are already among the most disadvantaged in terms of learning outcomes.

Pedagogical approaches that focus on teaching children where they are will be critical to averting permanent learning deficits

The compounding of learning loss that was seen in the years following the earthquake in Pakistan suggests that assessing children when they return to school and teaching them from where they are will be vital to mitigating the long-term effect of school closures. The effectiveness of approaches like these often called teaching at the right level, have been well established. When they reopen, school systems should leverage approaches like these, and focus on helping children catch up and solidify their basic skills.

Reopening to business as usual would compound the disastrous effects of the COVID related closures. This catastrophe can be avoided if education systems embrace the opportunity to take on systemic changes, as part of their response to the crisis.


1 Incidentally, the May 23 edition also includes a laudatory and well deserved profile of RISE Nigeria CRT PI Leonard Watchekon and RISE implementing partner African School of Economics, which Watchekon founded.

This blog first appeared on the RISE blog.

Applying the ‘triple A’ framework in Pakistan and Palestine: what we learnt about implementing reform

Guest blog written by Albert Pijuan and David Hoole

86 development practitioners at OPML have successfully completed the 15-week Practice of PDIA online course over the past two years. This is a story of how they are using the PDIA tools. 


At Oxford Policy Management, we have been building on and incorporating the lessons from the building state capacity course into our day–to-day work. As a company, we started drawing on the lessons and frameworks set out in this course on the back of Matt Andrew’s 2012 book, The Limits of Institutional Reform in Development. In doing so, we have explored how to apply problem-driven, iterative adaption (PDIA) in practice, and particularly some of the key frameworks. Chief among these is the ‘triple A’ framework of authority, acceptance, and ability. In this post we share some lessons and reflections we have drawn from applying this framework in separate projects in Pakistan and the Occupied Palestinian Territories (OPTs). Continue reading Applying the ‘triple A’ framework in Pakistan and Palestine: what we learnt about implementing reform

The Chief Minister Posed Questions We Couldn’t Answer

Guest post written by Jeffrey Hammer

I was recently at a conference in Lahore, Pakistan sponsored by the International Growth Centre where the keynote address was given by Shahbaz Sharif, the Chief Minister of the province of Punjab, Pakistan (100+ million people). While fun to see old friends and colleagues, the conference was a little depressing in the way it reflected the state of the development economics profession.

The Chief Minister posed serious questions that have traditionally been the bread and butter of the economics profession. Unfortunately, we are not even trying to answer them any more. The specific question was “Should I put more money into transport? Infrastructure (power, roads, water)? Law and order? Social services? Or what? And where am I going to get the money?” What questions could be more solidly part of the core of economics than these? Unfortunately none of these were even remotely the focus of the “evidence-based” policy making discussed.

Almost all of the cases analyzed were  single, simple policy “tweaks” that were, first of all, isolated from the broader market context in which they occurred and, second, had no conception of opportunity cost – what we would have to give up to pursue these things? We had an answer to “how to improve a public food distribution system” but even with a precise answer (to whether a tweak would work) we had no idea whether the substantial amount of money funding such a system is a good idea. Maybe the Chief Minister would be better off improving education or road networks or police or rural electricity. Some of these alternative policies could have more impact on food consumption than food distribution if we thought about how the world worked. Getting food to market securely (roads, better cold storage, trustworthy police and safe roads – this is Pakistan, which no one seemed to notice) may increase food availability much more than any tunnel-visioned food program Or not – maybe the food distribution system is better. We just don’t know. And none of us “experts” are trying to find out.

On spending priorities, what we need is the old fashioned notion of opportunity cost. “Evidence” now is “did something work?” meaning did it have any effect at all? or “can we get it to work a little better?” But the real question in such a resource-constrained economy is “does it work well enough to take money away from the power plant it prevented or any other thing money could have been used for?”  Or even, “is it better than leaving the money in private hands by not collecting the taxes to pay for it?” Besides not knowing the marginal welfare cost of taxation (anyone remember that?), we forget that poor people use their money for food, so the first-order effect of tax revenues is to make poor children hungrier. Is the benefit from secondary education or bicycles or the fertilizer subsidy so good as to impose this cost on these children?  We don’t know who ultimately pays taxes (when wages, for example, respond to indirect taxation) but it is likely that poor people, the majority of the population, pay at least some substantial share. And we don’t know how badly distorted the tax system is – in its very structure, not just in its administration. The incidence and efficiency loss of the whole structure of taxation are the first order answers the Chief Minister needs. No one studies these anymore.

When someone says “we should have more “X” because we have evidence that it works”, the response should be “compared to what?” What should we cut in order to promote your particular interest? My hobby horse these days is more sanitation in South Asia. I should have to defend it against (at least) a few alternatives.

It’s not like we have no basis for making this comparison. We usually try to determine which things the private sector (i.e. almost everyone – farmers, bicycle manufacturers and repairmen, truckers, shopkeepers, halal sausage casing makers) can be safely relied upon to produce, where it goes somewhat wrong (exactly how bad are private schools or doctors?) and where it is a flaming disaster such that the government is utterly indispensable. While we’ve all drawn the gap between public and private costs (or benefits) to help us talk about optimal Pigouvian taxes, when was the last time anyone tried to measure this one, central concept for valuing interventions in developing countries? Or in developed countries, for that matter? We look at enrollment rates (or even learning rates) but never ask “how much is this secondary education worth, and how much of that isn’t captured by the student?” Further, since there is no reason to think the number is the same in any two places, even if there were a couple of such studies, it wouldn’t make up the bulk of what we call policy-relevant research. And it’s not like it’s easy to do so we can’t just say “let the practitioner-types do the (routine) calculations”. There is nothing routine to it at all.

 In the conference, several research projects measured an effect (not an externality, not a welfare loss – just an effect) that could well be part of an almost completely private good with no serious market failures to speak of. Can it really be the case that date exporters genuinely didn’t know that packaging for export was available (and wouldn’t a phone call to either the exporters or the marketing wing of the packaging producers suffice)? Did football producers really need to know a better pattern for cutting pentagons out of leather when mechanized stitching (as the commentator on that paper noted) is swiftly changing the entire production process worldwide? Will the competition that is currently mechanizing allow firms to exist even with the 10% higher profits that a better pattern enables? And are policy makers (even with Ivy League economists as their advisors) really going to make better decisions than those producers or, much more importantly, the competitive forces in the economy?

My defense of my promoting sanitation is that I contrasted the value of health via providing public goods (sewer systems in cities) to spending on publicly provided health care (a rival and excludable service – I’m avoiding the “p” word, this being the sub-continent). I don’t know if I’ve cleanly identified the effect that I purport to have measured – whether open defecation without sewage in slums damages the health of its residents – but it makes sense, is tied to most peoples’ notions of the nature of public and private goods, and gives some evidence of an externality. One reason to avoid specifying which service should be sacrificed is to avoid fights. Even fairly convincing evidence that publicly provided healthcare is of questionable value can provoke uncomfortable arguments. But not even mentioning the opportunity cost of a proposed policy is irresponsible.

On collecting more taxes: this is, of course, a core government activity. Any way we can efficiently get more money into government coffers to support critical public services is to be applauded. But what we were treated to was a two-year experiment on something that looks like tax-farming (and indeed, was titled as such). Higher powered incentives to collect taxes? When you’re being watched?  Tax inspectors didn’t know an experiment was underway? Even if it was double blind (which it was not), can a two-year project using currently recruited tax inspectors (i.e., those that entered public service expecting to get a salary without having to work too much) anticipate what happens in equilibrium when everyone figures out how to make money from these high-powered incentives? That is, core government service or not, there is a labor market in which the people who this experiment purports to study operate. It is the nature of the long-run equilibrium of that market that is the proper level of analysis for policy purposes, not the behavior of the particular individuals who happen to have the job at present. As the commentator on that paper noted, the proposal looked like the medieval version of tax farming. But that scheme always deteriorated in time (longer than a two-year experiment would tell us) into an ugly system that brought down rulers.

The Chief Minister is a committed and capable man. With the recent elections behind him, he has the opportunity to actually accomplish things. He deserves much better support than we’re giving him.

This post originally appeared as a World Bank Blog.

BSC Video 9: Constructing Problems to Drive Change

Problems are key to driving change. In this video, Matt Andrews, uses two examples about HIV in Pakistan, to illustrate how constructing local problems using data can be used to mobilize stakeholders to search for solutions that ultimately drive change. You can watch the video below or on YouTube.

If you are interested in learning more, read Looking Like a State: Techniques of Persistent Failure in State Capability for Implementation and Escaping Capability Traps through Problem Driven Iterative Adaptation (PDIA).