We recently ran a PDIA course on climate change adaptation. Why?

written by Tim O’Brien


Leader of farming cooperative in central Sri Lanka that diversified into ginger production as drought increasingly hurt rice cultivation.

If you live in a developed country, odds are that you think about climate change as something that will harm future generations — your children or your grandchildren perhaps. But if you live in a poor country, chances are much higher that you think of climate change as a source of problems that are affecting you and your family today. Climate change may not be the most important problem for you if you live in a developing country, but odds are that it is making your problems worse.

The climate is changing globally, but vulnerabilities are faced locally, usually in ways that exacerbate existing development challenges. For most poor farmers, when and how the rain falls matters a great deal. But climate change is tending to affect seasonal patterns on which many farmers rely, paradoxically increasing the frequency and severity of both droughts and floods, often in the same places. In many poor cities, climate change is increasing water scarcity, flooding, landslides and overall risk of extreme weather events. For people without electricity in tropical countries, heat waves are increasingly deadly events. For communities that depend on fishing as a means of both income and food, ocean acidification is both an economic and health issue. For countries with weak infrastructure, limited budgets and undiversified economies, macroeconomic vulnerability to weather shocks continues to grow more severe. Continue reading We recently ran a PDIA course on climate change adaptation. Why?

PMRDF: an Innovative State Capability Experiment in India

written by Salimah Samji.

Anyone who has ever worked in India knows how hard it is to implement programs. The sheer size of the country makes it impossible for anyone but the government, who is the only one with infrastructure and reach, to provide public services to its citizens. Currently, every district administers typically 100+ development schemes. Each scheme has its own rules, regulations, reporting, and funding. Districts are continuously burdened with new schemes and more work without the requisite resources. They are often understaffed and have few (if any) skilled functionaries to be able to do their job. The reality is that 90% of the time is spent on routine day-to-day management leaving no time for strategic planning or experimentation.

To address the capacity and implementation gaps in backward districts which are isolated and less developed, the Ministry of Rural Development launched the Prime Minister’s Rural Development Fellowship (PMRDF) in 2012. The aim of the fellowship was to provide catalytic support to the district administration to help improve the delivery of programs and to develop a cadre of committed development professionals. The eligibility criteria was a minimum of a four-year degree from a recognized university. The fellows were to work for 2-3 years under the supervision of the District Collector, and paid INR 75,000 ($1,210) per month with a 10% increment each year subject to performance. That is a very generous amount. The rationale was to attract younger people who were passionate about development and willing to go to remote areas, but could not afford the low wages. In essence this was a way to bring in new blood into the system.

816 applicants were shortlisted from a total of 8,560 applicants for a group discussion and a personal interview. 156 fellows were selected to work across 82 districts in 9 states. Each state decided on the placement of fellows with 1-3 per district. Currently there are 137 fellows from the first batch and a second batch has just been selected. For more details see the TEDx talk.

Some of the challenges faced during the implementation of the 2012-13 PMRDFs include:
  • Building the trust of the States/District Collectors: When the program was initially announced, it was viewed suspiciously by some (i.e. was the center sending monitors to the states/districts?). Many meetings were held to explain and clarify the objectives of the program and to bring the states on board.
  • Ineffective use of the fellows: Depending on the District Collector, some fellows were welcomed and given a lot of responsibility and authority to experiment, while some others were used as glorified executive assistants – which is also understandable given the lack of human resources at the district level.
  • Lack of authority to issue instructions or to sanction funds: The fellows are not part of the government bureaucracy and therefore have no signing authority and cannot issue orders to lower level functionaries. As a result, they need to go to the District Collector with all those requests leading to time delays and inefficiencies. To address this, the state of Andhra Pradesh has delegated a budget head specifically for the fellows.
  • High monthly salary is attracting applicants who are not necessarily passionate about development: This has been an unintended consequence and they have tried to be more mindful in the interviews for the 2nd batch of fellows.
  • What happens after the fellows leave? One criticism is that they are building temporary capacity. To address this, the second batch will be offered one year of public service after their fellowship. However, this issue remains a challenge.
Here are some testimonies of the fellows, for more read their profile book or visit their blog.
  • “I still remember an incident from my initial days, when a village welcomed and garlanded me saying: We are receiving our first government official in our village after Independence.
  • “… working with the field staff in implementation of these schemes was not always easy. My efforts at convincing them and the higher authorities did not yield fruit every time. The convergence of different schemes has also been a challenge as each scheme has different criteria for beneficiary selection and process of implementation. However, struggling with these issues every day helped me understand the minutest details of these schemes and the complexities involved in the development sector.”
  • “This last one year as a PMRDF has helped me develop both personally and professionally. It has given me a chance to work and interact with people at every level of the hierarchy, starting with the panchayat secretary and going up to state officials.”
  • “The last one year has been a period of immense pleasure, satisfaction, frustration, learning and un-learning … There were difficulties: sluggishness, winning the trust of various actors and bringing them onto common platform to get the ball rolling, getting bogged down with too many operational formalities needing to continuously establish yourself within the system. I learnt that things will happen if there is strong will. I learnt of a number of unsung heroes who keep things going—unappreciated and unrecognized—but making a difference each day with their integrity.” This comment is a great illustration of Matt Andrew’s multi-agent leadership paper, Who really Leads Development?

The PMRDF is a work in progress with a lot of learning, iterating and adaptation. There are no easy solutions when you are working on implementation intensive programs. It will be interesting to see what other lessons are learned from this dynamic experiment of trying to solve capacity issues at the last mile of service delivery.

BSC Video 10: Specifying the Design Space

The design space of actual development projects is complex, granular, and nuanced. In this video, Lant Pritchett, uses a simple example of a design space for teacher training to illustrate this point. You can watch the video below or on YouTube.

If you are interested in learning more, read It’s All About MeE: Using Structured Experiential Learning (‘e’) to Crawl the Design Space.

BSC Video 5: Typology of Tasks by Capability Intensity needed for Implementation

It is important when thinking about building state capability, to first ask, what is the “type of problem” you are trying to solve? In this video, Lant Pritchett, provides a framework to determine the capability required for implementing development projects. He begins by asking whether your task is transaction intensive, followed by whether it is locally discretionary, to better understand if the nature of the task is logistics or implementation intensive. You can watch the video below or on YouTube.

BSC Video 4: Capability for Policy Implementation

Policy implementation requires agents of organizations who are responsible for implementation, to do the right thing, at the right time, and in the right place. In this video, Lant Pritchett, uses an example of delivering the mail and issuing driver’s licenses to illustrate this point. You can watch the video below or on YouTube.

If you are interested in learning more about the examples used in this video, read Letter Grading Government Efficiency, and Obtaining a Drivers License in India: An Experimental Approach to Studying Corruption.

What’s in a counterfactual?

written by Salimah Samji

I am amazed by people’s obsession with the counterfactual, and how evidence cannot exist without it. Why are people so enamored by the idea of ‘the solution’ even though we have learned time and time again that there is no one size fits all?

Is the existence of a counterfactual a sufficient condition? Why don’t people ask questions about the design and implementation of the evaluation? Specifically:

  • What are you measuring and what is the nature of your context: Where in the design space are you? Is your fitness landscape smooth or rugged? Eppstein et al. in Searching the Clinical Fitness Landscape, test two approaches (multicenter randomized control trials vs. quality improvement collaboratives where you work with others, learn from collective experience, and customize based on local context), to identify which leads to healthcare improvements. They find that the quality improvement collaboratives are most effective in the complex socio-technical environments of healthcare institutions. Basically, the moment you introduce any complexity (increased interactions between variables) experiential methods trump experimental ones.
  • Who is collecting your data and how: Collecting data is a tedious task and the incentive to fill out surveys without having to go to the village is high, especially if no one is watching. Then there are questions of what you ask, where you ask, how you ask, what time period it is, how long the questionnaire is, etc.
  • How is the data entered and verified: Do you do random checks? Double data entry?
  • Is the data publicly available for scrutiny?

And then there is the external validity problem. Counterfactual or not, it is crucial to adapt development interventions to local contextual realities, where high quality implementation is paramount to success. Bold et al. in Scaling Up What Works: Experimental Evidence on External Validity in Kenyan Education, find that while NGO implementation of contract teachers in Kenya produces a positive effect on test scores, government implementation of the same program yielded zero effect. They cite implementation constraints and the political economy forces in play as reasons for the stark difference. In a paper entitled, using case studies to explore the external validity of ‘complex’ development interventions, Michael Woolcock argues for deploying case studies to better identify the conditions under which diverse outcomes are observed, with a focus on contextual idiosyncrasies, implementation capabilities and trajectories of change.

To top it off, today’s graduate students in economics don’t read Hirschman (some have never heard of him!) … should we be worried?