Independent Evaluation of PDIA Application in Africa

In April 2017, we began our engagement with the Collaborative Africa Budget Reform Initiative (CABRI), an intergovernmental organization based in South Africa, to design and launch the Building Public Financial Management Capabilities (BPFMC) Program using the PDIA approach. In this program, CABRI’s member countries could apply for a team of 5-7 members to work on their locally nominated PFM problem for the period of 7-months. The training included online modules, an in-person framing workshop, virtual progress updates, and an experience sharing workshop at the end of the program. Each team was paired with a coach from CABRI who held regular check-ins to motivate and support them.

Teams from Ghana, Liberia, Lesotho, Nigeria, Sierra Leone, South Africa and The Gambia, participated in the inaugural program from May to December 2017. At the end of the program, the teams made significant progress: they developed a deeper understanding of the root causes of their problems; developed capabilities and confidence; financial reporting rates went up; budgets were prepared faster; and virements and arrears decreased. 

In April 2018, we began a second iteration, this time offering the program in both English and French. Teams from the Central African Republic (CAR), Côte d’Ivoire, Ghana, Lesotho, Liberia, and Nigeria, participated in the program. The Bill & Melinda Gates Foundation, a funder of the program, commissioned an independent evaluation to assess the application of the PDIA approach in the African context.

The evaluation included desk research, baseline and midline surveys of participants, interviews and observations from the framing workshop and experience sharing workshop, field visits to CAR, Lesotho, and Liberia, and interviews and observations from a review seminar, five months after the conclusion of the coaching support from CABRI.

Continue reading Independent Evaluation of PDIA Application in Africa

BSC 2019: The Year in Review

written by Salimah Samji

Reflection is a key part of the PDIA iteration process and as I have done in previous years (20172018) here’s a look back at what we @HarvardBSC achieved in 2019.

Some highlights of the year include: training and engaging with 740 practitioners around the globe (incl. degree programs, executive education, online courses and direct policy engagements with governments); publishing 9 papers and 54 blog posts; activating our PDIA online course alumni community of practice; releasing a new 12-part podcast series on the Practice of PDIA; translating our content into Spanish and French; and last but not least … drum roll please … launching Harvard Kennedy School’s first blended learning Executive Education program Implementing Public Policy, designed to equip policymakers around the world with both the skills to analyze policies, as well as the field-tested tools and tactics to successfully implement them.

2020 promises to be another exciting year for us. Here’s a few things we have in store for you: releasing our PDIA Toolkit in French, Portuguese and Arabic; publishing blogs written by our Implementing Public Policy program alumni; launching our new long read podcast series; and sharing our experience on creating and sustaining communities of practice with you. To stay tuned, follow us on twitter, or subscribe to our blog and podcast.

Here’s a month by month playback of 2019.

January

BSC Faculty Director Matt Andrews chaired the executive education program entitled, “Public Financial Management (PFM) in a Changing World” at the Harvard Kennedy School. 47 PFM practitioners from 25 countries participated in this program.

PFM 2019
BSC collaborated with the Bloomberg Harvard City Leadership Initiative in their Cross-Boundary Collaboration Program held in New York City. Director Salimah Samji served as a City Team facilitator during this program.

Continue reading BSC 2019: The Year in Review

Why do we persist so long with a reform approach that does not solve problems?

written by Kate Bridges and Michael Woolcock

In Malawi, efforts at institutional reform have been numerous, earnest and longstanding. Since 1966, there have been more than three times as many World Bank projects with ‘institutional reform’ content  as there have been in any other thematic or sectoral category.

In a recent paper, we argue that these efforts have largely failed. Public scandals such as “Cashgate” – in which about US$ 32 million in government funds was misappropriated between April and September 2013 – are the tip of the iceberg: high profile cases reflecting a logic of corruption that remains unchallenged by reform efforts. Continue reading Why do we persist so long with a reform approach that does not solve problems?

My PDIA Journey

Guest blog by Awa Touray

39041761511_6d256ac4cb_k

The reality of public service is that you are often bogged down with routine tasks that don’t often allow you the room to innovate and initiate. So, in an environment that is very reactionary, the Problem Driven Iterative Adaptation (PDIA) project provided an exciting avenue to be proactive in tackling public financial management (PFM) problems. The team spent a 7-month long journey of discovery and learning in tackling the problem of high virements and arrears leading to a misalignment in the appropriated budget and spending. Continue reading My PDIA Journey

Getting things done: PFM reform in Africa

written by Tim McNaught

We recently wrote about the closing workshop we held in December, in collaboration with CABRI, for the Building PFM Capabilities in Africa program. There is now a summary event page on the CABRI website with all of the presentations from the seven participating country teams. During the workshop, each team presented on the progress they made on their self-identified problems over the previous six months, the lessons they learned, and their next steps.  Continue reading Getting things done: PFM reform in Africa

PDIA and PFM in Africa

written by Salimah Samji

We just held our final workshop for the Building PFM Capabilities in Africa Program, in collaboration with CABRI, in Johannesburg from December 11-13, 2017. Teams from Ghana, Liberia, Nigeria, Sierra Leone, South Africa, The Gambia, and The Kingdom of Lesotho, participated in this program. Continue reading PDIA and PFM in Africa

Building PFM Capabilities in Africa

written by Tim McNaught

For the past 30 years, governments across Africa have been implementing public financial management (PFM) reforms with mixed results. While budgets, laws and processes have improved, they are often not effectively implemented (Andrews 2010). Technical solutions, commonly copied from upper-income countries, do not always take into account the local context and can lead to capacity gaps and a lack of ownership when the project finishes and the consultants leave. Continue reading Building PFM Capabilities in Africa

Why many development initiatives have achievement gaps…and what to do about this

written by Matt Andrews

Yesterday I blogged about Hirschman’s Hiding Hand. As I interpret it, a central part of his idea is that many development projects:

  • focus on solving complex problems, and
  • only once they have started does a ‘hiding hand’ lift to show how hard the problem is to solve,
  • but because policy-makers and reformers are already en route to solving the problem they don’t turn away from the challenges, and
  • so they start getting creative and finding ways to really solve the problem. Initial plans and designs are shelved in favor of experiments with new ideas, and after much muddling the problem is solved (albeit with unforeseen or hybrid end products).

I like the argument. But why do I see so many development projects that don’t look like this?

I see projects where solutions or projects are introduced and don’t have much impact, but then they are tried again and again–with processes that don’t allow one to recognize the unforeseen challenges, and rigid designs that don’t allow one to change or experiment or pivot around constraints and limits. Instead of adjusting when the going gets tough, many development projects carry on with the proposed solution and produce whatever limited form is possible.

I think this is because many reforms are not focused on solving problems; they are rather focused on gaining short-run legitimacy (money and support) which comes through simple promises of quick solutions. This is the most rank form of isomorphism one can imagine; where one mimics purely for show… so you get a ‘fake’ that lacks the functionality of the real thing…

Let me use Public Financial Management (PFM) reforms as an example.

What problems do these reforms try to solve? Quite a few, potentially. They could try to solve problems of governments overspending, or problems of governments not using money in the most efficient and effective manner (and ensuring services are delivered), or of governments using money in ways that erode trust between the state and citizens (and more).

Now, let me ask how many reforms actually examine whether they solve these problems? Very few, actually. Mostly, reforms ask about whether a government has introduced a new multi-year budget or an integrated financial management system. Or a new law on fiscal rules, or a new procurement system.

Sometimes the reforms will ask questions about whether fiscal discipline is improved (largely because this is something outsiders like the IMF focus on) but I seldom see any reforms–or any PFM assessments (like PEFA or even the assessments of transparency) asking if services are better delivered after reforms, or if reforms enhance trust between citizens and the state. I don’t even see efforts to systematically capture information about intermediate products that might lead to these ‘solved problems’. For instance:

  • Do we have evidence that goods are procured and delivered more efficiently (time and money-wise) after reform?
  • Do we have any systematic data to show that our new human resource management systems are helping ensure that civil servants are present and working well, and that our new payment systems pay them on time (and do a better job of limiting payments to ghost workers)?
  • Do we have any consistent evidence to show that suppliers are paid more promptly after reforms?
  • Is there any effort to see if IT systems are used as we assume they will be used, after reforms?
  • Does anyone look to see if infrastructure projects are more likely to start on time and reach completion after costly project management interventions?
  • Do we have records to show that infrastructure receives proper maintenance after reform?
  • Is there any effort to see if taxpayers trust government more with their money?

This is a long list of questions (but there are many more), and I am sure that some reforms do try to capture data on some of them (if you’ve measured these in a reform, please comment as such…it would be interesting and important to know). Most reforms I have observed don’t try to do it at all, however, which was the focus of a recent discussion on the role of PFM and service delivery Time to Care About Service Delivery? Specialists from around the world were asked whether PFM reforms improve service delivery and the answer was “we think so…we expect so…we hope so…BUT WE CAN’T TELL YOU BECAUSE WE DON’T ACTUALLY ASK EXPLICIT QUESTIONS ABOUT THIS.”

My concern with this is manifold: (i) Does the failure to ask if we are solving the problems suggest that we as a community of reformers don’t really care about the problems in the first place? (ii) Does it mean that we will not be sensitive to the situations Hirschman speaks about when he discusses unforeseen challenges that undermine our ability to address problems (simply because we don’t focus on the problems)?  (iii) Does this also mean that we will not have any moments where we explore alternatives and experiment with real solutions that help to overcome hurdles en route to solving problems?

Unfortunately, I think the observations of gaps after reforms speak to all of these interpretations. And this is why many reforms and interventions do not end up solving problems. In these cases, we get the half-baked versions of the pre-planned solution…with no adjustment and no ‘solved problem’. PFM systems look better but still don’t function–so payments remain late, wages are unpaid to some and overpaid to many, services are not delivered better, and trust actually declines. Most worrying: we have spent years doing the reforms, and now need to pretend they work..and have no learning about why the problems still fester.

The solution (maybe): In my mind this can be rectified–and we can move towards producing more projects like those Hirschman observed–by

  • focusing reforms on problems, explicitly, aggressively, from the start;
  • measuring progress by looking at indicators of ‘problem solved’ (like improved levels of trust after PFM reforms) and intermediate indicators we think will get us there (better payment of contracts, more efficient procurement, etc;
  • regularly monitoring this progress;
  • being on the lookout for expected unexpecteds (things that we didn’t know about that make our initial solutions less impactful); and
  • being willing to adjust what we started with to ensure we produce real solutions to real problems–functional improvements and not just changes in form.

For more, read This is PFM which advocates a functional approach to thinking about and doing PFM reform.

This is PFM

The acronym PFM stands for Public Financial Management: But what does public financial management really mean?

Matt Andrews, Marco Cangiano, Neil Cole, Paolo de Renzio, Philipp Krause, and Renaud Seligmann have published a new paper on demystifying the concept of public financial management (PFM), drawing on their experiences as specialists in different contexts and with different views (from academia, the multilateral and bilateral development agencies, think tanks, government, and civil society). This paper provides an entry point for discussion on the constituent elements of PFM systems, how and why PFM reforms have emerged, and points to gaps for future attention. You can read it here.