Independent Evaluation of PDIA Application in Africa

In April 2017, we began our engagement with the Collaborative Africa Budget Reform Initiative (CABRI), an intergovernmental organization based in South Africa, to design and launch the Building Public Financial Management Capabilities (BPFMC) Program using the PDIA approach. In this program, CABRI’s member countries could apply for a team of 5-7 members to work on their locally nominated PFM problem for the period of 7-months. The training included online modules, an in-person framing workshop, virtual progress updates, and an experience sharing workshop at the end of the program. Each team was paired with a coach from CABRI who held regular check-ins to motivate and support them.

Teams from Ghana, Liberia, Lesotho, Nigeria, Sierra Leone, South Africa and The Gambia, participated in the inaugural program from May to December 2017. At the end of the program, the teams made significant progress: they developed a deeper understanding of the root causes of their problems; developed capabilities and confidence; financial reporting rates went up; budgets were prepared faster; and virements and arrears decreased. 

In April 2018, we began a second iteration, this time offering the program in both English and French. Teams from the Central African Republic (CAR), Côte d’Ivoire, Ghana, Lesotho, Liberia, and Nigeria, participated in the program. The Bill & Melinda Gates Foundation, a funder of the program, commissioned an independent evaluation to assess the application of the PDIA approach in the African context.

The evaluation included desk research, baseline and midline surveys of participants, interviews and observations from the framing workshop and experience sharing workshop, field visits to CAR, Lesotho, and Liberia, and interviews and observations from a review seminar, five months after the conclusion of the coaching support from CABRI.

Continue reading Independent Evaluation of PDIA Application in Africa

PDIA and PFM in Africa

written by Salimah Samji

We just held our final workshop for the Building PFM Capabilities in Africa Program, in collaboration with CABRI, in Johannesburg from December 11-13, 2017. Teams from Ghana, Liberia, Nigeria, Sierra Leone, South Africa, The Gambia, and The Kingdom of Lesotho, participated in this program. Continue reading PDIA and PFM in Africa

Building PFM Capabilities in Africa

written by Tim McNaught

For the past 30 years, governments across Africa have been implementing public financial management (PFM) reforms with mixed results. While budgets, laws and processes have improved, they are often not effectively implemented (Andrews 2010). Technical solutions, commonly copied from upper-income countries, do not always take into account the local context and can lead to capacity gaps and a lack of ownership when the project finishes and the consultants leave. Continue reading Building PFM Capabilities in Africa

World Bank uses PDIA in Sierra Leone

written by Salimah Samji

International development experts often tell us that they cannot do PDIA because the project processes within their organizations do not allow for flexibility. The truth however, is that all development agencies have some sort of instrument that does allow for experimentation and flexibility. Here’s an example of how a Pay and Performance project in Sierra Leone explicitly used PDIA principles.

Civil service reforms are complex in and of themselves. If you add, a lack of capacity to implement programs, multiple reporting lines, demoralized civil servants, a lack of coordination amongst key agencies, and a low-level of trust, the potential for success of such a reform decreases significantly. Recognizing this, the World Bank team decided to use the key principles of the PDIA framework with support from the Leadership for Results (LforR) program for their Pay and Performance Project in Sierra Leone. The rationale for this was to bring a broad range of stakeholders together and facilitate a process of collective problem and solution identification, as well as to introduce experimentation and adaptability during implementation.

They began with some short-term results-focused Rapid Results Initiatives (RRIs) in Year 0 and Year 1. The pilot was instrumental in building the confidence of the local civil servants by demonstrating that progress was possible in their context and gave them a sense of ownership. In addition, the short feedback loops facilitated rapid experiential learning about what results were actually achieved for both government and the World Bank staff – in PDIA terminology, we call this strategically crawling the design space.

Specifically, they used a two-pronged, learning-by-doing process, which included:

  1. Structured team coaching throughout the implementation process: A locally based rapid results coach who had an in-depth understanding of government and public sector reform was hired to provide support to teams on a daily basis. The coach:
    • Facilitated problem solving at multiple levels in the system with team-level work,
    • Helped create action plans by breaking a huge daunting task into smaller easier to digest chunks,
    • Motivated the teams despite the challenges, and
    • Created an opportunity for the teams to learn from each other and to see how their work fit within the larger picture.
  1. Facilitated leadership fora for dialogue: One-day strategic leadership convenings between leaders and implementation teams were held at critical points. These retreats served to review progress and learning, problem-solve, facilitate reflection, make strategic decisions, and course-correct where needed. In PDIA terminology, we call this maintaining the authorizing environment.

After 20 months of implementation (February 2014), they had several hard results. More importantly, there was stronger inter- and intra-agency collaboration and increased trust and communication. The teams actually had the capacity to do things themselves. The flexibility at the design stage allowed more politically and technically feasible solutions to emerge.

So, large bureaucracies can do PDIA and it doesn’t take forever. Bottom-line: the mundane matters and cannot be ignored for a project to succeed.

Roberto O. Panzardi and Kay Winning are in the process of publishing a paper with more details on this project. You can read about the preliminary results here.

 

Helping REAL Capacity Emerge in Rwanda using PDIA

written by Matt Andrews

What do you do if your government has been pursuing reforms for years, with apparent success, but your economy is still not growing? What do you do if the constraint seems to be the limited capacity of government organizations? What do you do if this capacity remains stubbornly low even after years of public sector reforms sponsored by outside partners and based on promising best practice ideas of fly-in-fly-out specialists?

A recent case study of work suported by the Africa Governance Initiative suggests an approach to just such a situation, faced by Rwanda in 2010. The approach is simple.

  1. Force your own people to look at festering problems in an up-close-and-personal manner, focusing on ‘why’ the problems persist instead of ‘what’ easy solutions outsiders might propose for the problems.
  2. Swarm the problems with new ideas emerging from those inside your government and from trusted outsiders committed to spending time adapting and translating their ideas to your context (instead of one-size-fits-all solutions coming from short-stay outsiders).
  3. Experiment in a step-by-step manner, actively, with the ideas, trying them out and seeing what works, why and with what kinds of nips and tucks.
  4. Learn. Yes, simply learn. Let people reflect on what they have done and absorb what made the difference and what they take from the experiences to carry to other tasks.

This is what I understand the Strategic Capacity Building Initiative (SCBI) was and is about. It is an approach to doing development that seems to have yielded some dynamic results in a relatively short period. The results are substantive, procedural, organizational, and personal. Farmer incomes are up after some of the experiments in the ‘pilot sites initiative’, for instance. The Energy Investment Unit has emerged as the focal point of a new process to increase energy generation and drive energy costs down. Perhaps more important, to me at least, is the fact that talented civil servants have done things they probably never dreamed they could. In my own language, they have become more adaptive—realizing that you can make a difference if you purposefully address real problems you face in an active, experiential and iterative manner. These young policy entrepreneurs and implementers will be in Rwanda for years to come, and hopefully long after SCBI ceases to exist as a program. They are the real success and legacy of the program.

I find this story line appealing. It tells of an approach to development that reflects the principles of problem driven iterative adaptation (PDIA). This SCBI approach is full of common sense but is also oddly revolutionary because it is such a contrast to the way development is commonly done (and, it seems, was done in some of these areas in Rwanda previously). The case shows that a locally problem driven, adaptive process works in complex developing country reform contexts and for this reason should be of interest to anyone working in development.

Not all is rosy and sweet in the story, however, which is true in all narratives on change and real functional reform—including all development narratives I think reflect the general principles of PDIA. There are a number of reflections on how hard it is to develop a real problem driven approach and allow flexibility in finding and fitting new solutions. The case notes that high level leaders demanded results particularly rapidly in some instances, for example, and this led to hurried action, mistakes and tension. The case suggests that this can be overcome with some common-sense ideas, like getting political authorizers to prioritize and ask policy people how long something will take and then agree on realistic time frames. In this respect, it also comments on the importance of focusing attention on a few key issues for deep dive attention rather than a slew of issues that ultimately only get a shallow look. As one Permanent Secretary notes, “Trying to do everything at the same time doesn’t work.” (Seems basic, doesn’t it, but this kind of mundane observation is one I see overlooked across the development agenda).

There are also hints at the importance of getting collaborative relationships right—with high-level authorizers and development partners engaging patiently and yet also expectantly with those in the policy and reform trenches. It seems there are real rewards when those at the top give those in the middle and even lower down the organization some structured space to prove their value. (Again, this seems basic, noting that ‘people’ really matter; but it is a vital observation in development where I thinking ‘policy’ is often seen as more important than people.) The importance of political coalitions and teams, incorporating outsiders and insiders, is also implicit and explicit and a vital take-away for those designing reforms and interventions. These coalitions and teams allowed natural coordination across boundaries (without having to change rigid organizational rules) and cross fertilization that is vital for the emergence of creative new policy ideas.

Beyond these ideas I was perhaps most struck by the lesson that this work is only achieved if people stick to it. It proved challenging and even uncomfortable to force politicians to prioritize, for instance, but this did not lead to the program falling apart. Instead, as the case notes, the SCBI team exerted “push back” on the system and made sure the prioritization was done. It proved hard to get the right people as well, but the SCBI team pivoted around this issue to get at least some of the right people and build on what they had. It proved tough to get disparate distributed agencies to work together and to even understand the importance of linkages, but the challenge was met with more determination. It was hard to take people through the uncomfortable process of problem analysis, where they interrogated existing processes to look for gaps (without jumping to quick but unlikely to work solutions). But the gap analyses went on nonetheless.

The literature on organizational change has a really academic term to describe the quality I think helped the SCBI reformers to stick to their guns when things got tough. It is ‘grit’ and it is vital for effective reform and change. It is the intangible thing that I think the SCBI story is really all about. It helped to keep the reforms going in the starting months that seemed slow and difficult, and it was what kept the growing SCBI team motivated when the actions they were taking were hard and constantly questioned as time consuming, demanding, politically uncomfortable, and (maybe even) downright impossible. Grit is what helps reformers turn setbacks into lessons, and what keeps reformers looking for the ‘right’ people needed to make something happen. It keeps people engaged in capacity building experiences that take time, personal sacrifice, and political capital. It is the magic ingredient behind real capacity building and change and is the one thing I hope other readers see in this case study, even with the other great practical ideas and embedded advice. The SCBI design and strategy was great, but it was the gritty commitment to make it work that really seems to have made the difference.

The lesson: Cultivate grit, don’t overlook it, as it is the key to capacity building success.