written by Salimah Samji
The knee jerk reaction to building capacity is to organize more training workshops. These are taught by experts and held in fancy locations, with free-flowing food and refreshments. The attendees often do not include the front line workers who are ultimately responsible for implementation. In some cases attendees do learn new skills and are inspired to try them out, but they return to the same old organizations and systems – they are the only ones who changed. Then there is the case of technical advisors/assistants being parachuted into ministries to help build capacity. The reality however is that the job of these advisors is to perform functions and deliver results leaving no time to transfer skills or to build capability.
In a recent paper entitled, “Escaping the capability trap: turning “small” development into “big” development,” authors Campos, Randrianarivelo and Winning discuss the process used to build capacity in Burundi from 2006-2012. The World Bank Institute in collaboration with the government decided to launch a program whereby small initiatives would be launched to demonstrate quick and meaningful results that would in turn be showcased and used to generate political buy-in at cabinet retreats, ultimately expanding the program and sustaining a process of capacity building. They used Rapid Results Initiatives (RRIs) where small projects are created with measurable results in around 100 days. They began with a pilot of 2 RRIs in 2006 and had completed 246 RRIs by 2012. The authors consider Burundi’s experience as a “preliminary test” of the PDIA approach.
Initially the government was skeptical but agreed to conduct two small RRI pilots one in education and one in health. This process involved the entire chain of implementation actors and enabled these teams to learn what works and what does not. Team building and collaborative skills were necessary to ensure success.
- Education: The problem was that it took one year to deliver textbooks to village schools. The goal of the RRI pilot was to reduce this delay in a province to within 100 days. They delivered 25,000 textbooks within 60 days.
- Health: The problem was that there were few HIV/AIDS screenings of pregnant women in health care centers. The goal was to increase the number of screenings. They increased from an average of 71 per month to 482 in the first month, more than a six-fold increase.
The results of the pilots, as well as results from other African countries that had engaged in similar programs, were shared at a cabinet retreat. This helped create a shift in the mindset which allowed the methodology to be replicated in other ministries. In each retreat (4 in total) successful RRI experiences were shared, thus helping to build and sustain the authorizing environment. The results were impressive and include the suspension of payment to 728 ‘ghost’ individuals which saved the Ministry of Civil Service 60,000,000 Burundi Francs (US$ 470,000).
The authors quote the Ministry of Agriculture “leadership engagement from the top [the minister] right down to the base [even in decentralized provinces] encouraged a ‘spirit of results’ and mobilized people from all areas [the government administration as well as the local citizens] to work together to achieve what they wanted.” This is what Matt Andrews calls multi-agent leadership in his paper who really leads development?
Our recent untying development workshop had a session entitled new practice in action, which featured AGI’s work in Liberia, the Rapid Results Institute and the Innovations for Successful Societies at Princeton. You can watch the video here.
Bottom line: An RRI is one of many tools in the toolkit that espouses PDIA principles.
written by Matt Andrews
This is the last of the four common excuses that I hear about why PDIA cannot be done in development. If you are interested, you can read the first, second and third one.
Excuse 4: International development experts often tell me that PDIA is not possible because it implies that we are always muddling through. “How do I sell a muddled reform project ?”
I firmly believe that PDIA has its most value when we are in complex settings dealing with complex problems, where we don’t necessarily know the solutions or how to implement the solutions. In such situations one needs a process of finding and fitting relevant solutions that are unknown at first. This is where PDIA comes in and is useful .
If PDIA is used at the start of reforms in such contexts, one can find and fit solutions that have functional impact. A more traditional project process can be used once one knows (at least to some degree) what to do and how to do it. So you don’t need to muddle along forever… And PDIA is not about perpetual muddling – it is about structured, experiential learning.
written by Matt Andrews
This is the third of the four common excuses that I hear about why PDIA cannot be done in development. If you are interested, you can read the first and second one.
Excuse 3: International development experts often tell me that PDIA is not possible because it takes too long.
This is not true, especially when one considers the timing issues of traditional development projects. In my experience, many development projects last 5 years or so and deliver little more than new forms. These projects are then followed by a new project doing the exact same thing. Yes I know it sounds far fetched, but I see this in all countries and areas of development. So, if the counterfactual is 20 years of anti corruption reform in Malawi (that did not curb corruption) why say PDIA is too slow when it calls for crawling the design space for a year or two before we lock in a solution?
Second, traditional projects take ages to prepare (often one to two years). The preparation is largely passive, based on work in offices by project designers whose ‘product’ is the project design (not its implementation). PDIA offers much more with multiple active experiments and iterations in the context that lead to on the ground learning, capacity building, team and coalition building, experiential learning and active project design + quick wins. These gains far exceed those of many traditional project design phases, and yield projects that are already being implemented.
Iterative processes take multiple steps, but these are not necessarily long, can be much shorter than one step projects, and offers an opportunity for structured learning along the way. We need to use our time well and PDIA allows us to do this.
written by Matt Andrews
This is the second of the four common excuses that I hear about why PDIA cannot be done in development. If you are interested, you can read the first one.
Excuse 2: International development experts often tell me that PDIA is not possible because politicians will never support it.
Again, simply not true. It is true that many politicians will look for big projects promising large things. This is what I call signaling in my book and is a major constraint in many countries. I think it is facilitated by donors who offer large loans in response to big promises for best practice, which often leads to a ‘what you see is not what you get‘ situation. But my research shows that there are reforms that yield functional improvements in government. And studies of these reforms suggest that politicians can also welcome and support PDIA type processes that go beyond signaling. Where a locally felt problem exists, it is clear to me that politicians are often very interested in processes that promise real solutions. And many politicians are aware that these solutions need to emerge gradually so that they are properly authorized and capacitated.
So a PDIA approach of active and iterative engagement is not foreign or unwelcome in such situations. Indeed, I see many politicians creating a holding environment for such engagement. These politicians value the tight feedback loops and the rapid opportunities to learn and build capacity in their organizations. They also like the quick wins, especially when these wins feed into broader narratives about solving problems and promoting development. Indeed, tight iteration may overcome the time inconsistency problem we often see in reform (where politicians need results quicker than a large multiuser project can deliver). Read the Burundi post for an example.
written by Matt Andrews
Almost every time I give a presentation on PDIA (and I have given many), I hear excuses about why PDIA cannot be done in development. So, I’ve decided to set the record straight. I am writing a blog post and drawing a picture for each of the four most common excuses I hear. This is the first one.
Excuse 1: International development experts often tell me that they cannot do PDIA because they have to produce projects and project processes don’t allow the flexibility implied in PDIA.
This is simply not true. Every development agency I know of, has traditional project mechanisms that are rigid and foster disciplined process BUT every development agency also has instruments that allow experimentation and flexibility. The names of these instruments differ but common tools have (over time) included trust funds, learning and innovation loans, adaptable projects, and even some results based loans.
So, development experts can find tools to do flexible problem identification and active project design and implementation IF THEY KNOW THESE ALTERNATIVES EXIST AND TAKE THE EFFORT TO USE THEM. If they choose not to use these alternatives because they are risky, or hard, or different, that is one thing. But experts should stop saying that these alternatives do not exist. If you want an example, read the PDIA in Cameroon blog post.
In the spirit of Thanksgiving, I wanted to share the story of the image we chose for this blog – the Studley Tool Chest.
Designed by piano maker Henry O. Studley (1838-1925), this toolbox is about 40 inches by 20 inches when closed, and holds approximately 300 tools. Apparently, it is so heavy that it takes 3 strong people to put it up on the wall. He developed and added new tools, over the period of 30 years, adapting and ensuring that every tool fit snugly in its space. The craftsmanship is extraordinary and it remains in a class of its own. It has been exhibited at the Smithsonian National Museum of American History. Here’s a video if you are interested in seeing it.
written by Salimah Samji
I am amazed by people’s obsession with the counterfactual, and how evidence cannot exist without it. Why are people so enamored by the idea of ‘the solution’ even though we have learned time and time again that there is no one size fits all?
Is the existence of a counterfactual a sufficient condition? Why don’t people ask questions about the design and implementation of the evaluation? Specifically:
- What are you measuring and what is the nature of your context: Where in the design space are you? Is your fitness landscape smooth or rugged? Eppstein et al. in Searching the Clinical Fitness Landscape, test two approaches (multicenter randomized control trials vs. quality improvement collaboratives where you work with others, learn from collective experience, and customize based on local context), to identify which leads to healthcare improvements. They find that the quality improvement collaboratives are most effective in the complex socio-technical environments of healthcare institutions. Basically, the moment you introduce any complexity (increased interactions between variables) experiential methods trump experimental ones.
- Who is collecting your data and how: Collecting data is a tedious task and the incentive to fill out surveys without having to go to the village is high, especially if no one is watching. Then there are questions of what you ask, where you ask, how you ask, what time period it is, how long the questionnaire is, etc.
- How is the data entered and verified: Do you do random checks? Double data entry?
- Is the data publicly available for scrutiny?
And then there is the external validity problem. Counterfactual or not, it is crucial to adapt development interventions to local contextual realities, where high quality implementation is paramount to success. Bold et al. in Scaling Up What Works: Experimental Evidence on External Validity in Kenyan Education, find that while NGO implementation of contract teachers in Kenya produces a positive effect on test scores, government implementation of the same program yielded zero effect. They cite implementation constraints and the political economy forces in play as reasons for the stark difference. In a paper entitled, using case studies to explore the external validity of ‘complex’ development interventions, Michael Woolcock argues for deploying case studies to better identify the conditions under which diverse outcomes are observed, with a focus on contextual idiosyncrasies, implementation capabilities and trajectories of change.
To top it off, today’s graduate students in economics don’t read Hirschman (some have never heard of him!) … should we be worried?