State Capability Matters

written by Lant Pritchett

The Social Progress Index is a new attempt to gauge human well-being across countries that does not rely on standard measures like GDP per capita but rather builds and index of Social Progress from the ground up.  The Social Progress Index is an overall measure and then is divided into three measures:  Basic Human Needs, Foundations of Well-being, and Opportunity. 

The Building State Capability program focuses on new approaches to build the capability of governments and its organizations to carry out its functions—from making and implementing a budget to regulating the environment to maintaining law and order to educating children.

A set of natural questions to ask are:

  • Do countries with more government capability have higher levels of social progress?
  • Does the positive association of government capability and the various measures persist after also controlling for a countries GDP per capita and the extent of democracy?
  • How big is this connection?

The answers are:

  • Yes.
  • Yes.
  • Very big.

Table 1 reports a simple OLS multi-variate regression of the Social Progress Index and its three main components on (natural log) GDP per capita, one measure of state capability, the World Governance Indicator measure of Government Effectiveness, and the Polity IV measure of autocracy to democracy.  All of these are rescaled to 0 (the lowest country) to 100 (the highest country) so that the coefficients across the indicators can be compared.  In this scaling the regression coefficients say that a 1 unit change in say, WGI Government Effectiveness, is associated with a .39 unit change in the Social Progress Index or a .53 point change in the Opportunity index.

Table 1:  State capability matters for well-being

 

Main indices of Social Progress Index and its three components

(all rescaled to 0 to 100)

  ln(GDPPC)

(rescaled)

World Governance Indicators

Gov’t Effectiveness

(rescaled)

Polity

(rescaled)

R2
Social Progress Index Coefficients 0.50 0.39 0.13 0.92
t-stats 12.26 7.96 5.41
Basic Human Needs Coefficients 0.69 0.26 0.00 0.82
t-stats 11.06 3.48 0.05
Foundations of Wellbeing Coefficients 0.56 0.38 0.18 0.86
t-stats 8.46 5.81 5.90
Opportunity Coefficients 0.29 0.53 0.25 0.87
t-stats 4.68 8.91 8.67

The two questions are answered in Table 1 as these regressions say that, for a country of the same GDP per capita and with the same rating on democracy, an improvement in state capability is associated with large improvements in all four indicators of well-being (And these estimates are precise so that we can confidently reject that any of them are zero).

These affects are big, which can be illustrated in two ways.

  • First, there is a massive literature on the connection between GDP per capita (which measures average productivity of a country and hence is a crude indicator of the material basis available) and various indicators of well-being.  This literature tends to find very powerful correlations.  So it is interesting that improvements in government effectiveness are nearly as large as those in GDP per capita.  A one unit improvement in (ln) GDPPC is associated with SPI higher by .5 points and WGI-GE with .39 points (so 80 percent as large).  Interestingly, the impact of state capability is consistently and substantially larger than of POLITY’s rating of democracy.
  • Second, Figure 1 shows the association between the WGI government effectiveness measure and SPI, after controlling for GDP per capita and the POLITY rating of democracy.  This says “for countries with the same GDPC and POLITY how much higher would be expect SPI to be if government effectiveness were higher?”  As the graph shows moving from Venezuela’s capability (which is low for its GDP and POLITY) to Rwanda (which is high for its GDPPC and POLITY) would improve the Social Progress Index by over 20 points (which is the raw gap between say, Bangladesh (37) and the Dominican Republic (59) or between Indonesia (53) and Israel (75).

Of course this kind of data cannot resolve questions of cause and effect (as perhaps social progress or its components lead to greater state capability) but, to the extent these associations reflect a causal impact of state capability on well-being these are impressively large impacts and highlight the need for more attention to understanding not just how to promote economic growth but also how to build the capability of the state and its organizations.

Finding the Fringes of Formality: Organizational Capability in Street-Level Bureaucracies in Brazil

Guest blog written by Susana Cordeiro Guerra

Why is it that, despite the abundant resources invested and the largely favorable macroeconomic conditions that have prevailed until recently, middle-income countries have been unable to systematically deliver quality basic services, such as education and safety, to their citizens? Despite a wide variety of attempts to improve these crucial public services, results have failed to meet expectations.

Efforts to build state capabilities have often been influenced by the practice of the developed countries, traditionally especially the large Weberian bureaucracy model but increasingly in recent years models emphasizing less formal or strict approaches to bureaucratic performance and service delivery, such as those using the private sector as a rubric. Developing countries have applied various frameworks for improving service delivery and bureaucratic reform over the last 50 years – and yet there has been little to no significant convergence to developed country service provision levels (Pritchett 2013).

This is more than a puzzle. It has been a cause for revolt. Over the past few years, citizens have repeatedly risen in protest across the globe – notably in Brazil – to demand better service delivery and more efficient and fair government. What these protests highlight is actually a fundamental crisis of the state. If states cannot deliver better quality services in light of rising wealth, education, and expectations, can they sustain legitimacy?

This problem therefore calls for renewed scholarly and policy attention to how states can better perform these crucial functions, and thus to the performance of state bureaucracies. It also calls for novel approaches to how to resolve this problem. This dissertation project takes up this call by focusing attention on the too often neglected role of organizational performance and its role in improved service delivery by state bureaucracies. In particular, I focus on the under-investigated problems of organizational capability, its causes, and its relationship to positive organizational outputs in the context of “middle capability” countries.

I investigate the challenge of improving state capability by looking closely at the dynamics of Brazil, a paradigmatic and large middle-income country that has struggled with this very set of problems for a number of decades. In particular, I examine why there is variation in reform implementation in front-line bureaucratic units in three different sectors: education, policing and industrial policy. These sectors represent three different types of street-level (non-logistical) bureaucracies in a state of “middle capability” like Brazil. I chose case studies in each sector that have been deemed successes in reform implementation, but that actually exhibited tremendous variation in the management of the front-line service delivery units.

My question is: Why do some schools, police pacification units and innovation institutes do better than others? My hypothesis is that bureaucratic behavior is important to explaining this.

To research the question, I have drawn on semi-structured surveys, with both open- and close-ended questions to examine the behavioral patterns of managers (police commanders, school principals and innovation institute directors) of these front-line units. Having examined nearly 160 units across three sectors, I have found that purely structural explanations cannot account for this variation. For instance, I found units in the least likely places that were being very well managed while others in favorable settings that were not well managed.

So what accounts for this variation? I argue that an important part of this success under these conditions is related to how bureaucrats in middle management approach their responsibilities, and especially how they deal with fulfilling their responsibilities in light of the rules and protocols under which they operate. In particular, I hypothesize and have found evidence that the most successful are the middle-level bureaucrats who share a particular behavioral profile – a profile I refer to as operating “at the fringes of formality.”

The fringes of formality behavioral profile entails three main characteristics: middle-level agents who exhibit initiative, spend time on strategic rather than administrative or tactical functions and who operate in a particular way in the bureaucracy, husbanding and spending bureaucratic capital in a way that is innovative and results-oriented, but respects the rules and the interests of the organization. This differentiates such behavior both from model Weberian bureaucrats who strictly follow rules and protocols but also from jeitinho bureaucrats who simply seek convenient workarounds without reference to the interests of the larger organization or rules.

What do these three characteristics that describe the fringes of formality behavior mean in practice? Middle-level agents who have initiative show a strong sense of de facto autonomy and are energetic in the pursuit of solutions to organizational challenges within their appropriate sphere in the bureaucracy. Middle-level agents who spend their time largely on strategic functions are mostly involved in planning and abstract thinking, as opposed to administrative and routinized functions. Lastly, agents who are operating at the fringes of formality use their bureaucratic capital in a way that is useful and productive to the organization’s interests but tangential to the rules and protocols rather than strictly following them. In doing so, these agents are able to stretch and create space within rules without breaking them and in a manner that also benefits the organization as a whole.

I have found evidence that this behavior is present in organizations in Brazil, but also that it seems to be associated with positive administrative or intermediate outputs from the relevant level of their organizations. This, in turn, is associated with better organizational performance.

It is important to note that the aim of the project is to explain the variation in reform implementation across front-line administrative units by examining the relationship between behavioral profiles and intermediate organizational outputs. The aim of the project is not to examine whether given bureaucracies or organizational programs and initiatives lead to improved outcomes and overall performance in the sector. Rather, the aim is to identify the kinds of bureaucratic behavior that are associated with better bureaucratic performance in middle capability settings. Thus, the focus is on evaluating what causes bureaucratic competence, not with the evaluating programs themselves. Of course, while good programs are important to good public outcomes, so too bureaucratic competence is essential to effective public service provision.

Ultimately, the dissertation project has found a profile of behavior that seems to be associated with positive organizational outputs. This behavior is not the typical behavior that is commonly understood to be “proper” or “optimal” bureaucratic behavior. It is a behavior that actually somewhat deviates from the norm of what is considered “good” bureaucratic behavior. The upshot is that the behavior that actually works is one that is more bottom-up, more organic, and not the one that seems “the best” from a distance or in the abstract. In other words, there are practices currently being evolved or developed on the ground, within bureaucracies without top-down guidance or management that are working, even if they do not really conform to what is usually understood as best practice. These are the practices that can allow the middle-level manager to free their respective front-line unit of the middle capability trap and move to a more Weberian looking type of bureaucracy in the long run.

PDIA Notes 2: Learning to Learn

written by Peter Harrington

After over two years of working with the government of Albania, and as we embark on a new project to work with the government of Sri Lanka, we at the Building State Capability program (BSC) have been learning a lot about doing PDIA in practice.

Lessons have been big and small, practical and theoretical – an emerging body of observations and experience that is constantly informing our work. Amongst other things, we are finding that teams are proving an effective vehicle for tackling problems. We have found that a lot of structure and regular, tight loops of iteration are helping teams reflect and learn. We have found that it is vital to engage with several levels of the bureaucracy at the same time to ensure a stable authorising space for new things to happen. This all amounts to a sort of ‘thick engagement’, where little-and-often type interaction, woven in at many levels, bears more fruit than big set-piece interventions.

Each of these lessons are deserving of deeper exploration in their own right, and we will do so in subsequent posts. For now, I want to draw out some reflections about the real goal of our work, and our theory of change.

In the capacity-building arena, the latest wisdom holds that the best learning comes from doing. We think this is right. Capacity building models that rely purely on workshop or classroom settings and interactions are less effective in creating new know-how than interventions that work alongside officials on real projects, allowing them to learn by working on the job. Many organisations working in the development space now explicitly incorporate this into their methodology, and in so doing promise to ensure delivery of something important alongside the capacity building (think of external organizations that offer assistance in delivery, often by placing advisers into government departments, and promise to ensure a certain goal is achieved and the government capacity to deliver is also enhanced).

It sounds like a win-win (building capabilities while achieving delivery). The problem is that, in practice, when the implementers in the governments inevitably wobble, or get distracted, or pulled off the project by an unsupportive boss (or whatever happens to undermine the process, as has probably happened many times before), the external advisors end up intervening to get the thing done, because that’s what was promised, what the funder often cares more about, and what is measurable.

When that happens, the learning stops. And the idea of learning by doing stops, because the rescue work by external actors signalled that learning by doing—and failing, at least partially, in the process—was at best a secondary objective (and maybe not even a serious one). Think about anything you have ever learned in your life – whether at school or as an adult. If you knew someone was standing by to catch a dropped ball, or in practice was doing most of the legwork, would you have really learned anything? For the institutions where we work, although the deliverable may have been delivered, when the engagement expires, nothing will have changed in the way the institution works in the long run. This applies equally, by the way, to any institution or learning process, anywhere in the world.

The riddle here is this: what really makes things change and improve in an institution, such that delivery is enhanced and capability to deliver is strengthened? The answer is complex, but it boils down to people in the context doing things differently – being empowered to find out what different is and actually pursue it themselves.

In pursuing this answer, we regularly deploy the concept of ‘positive deviance’ in our work: successful behaviors or strategies enabling people to find better solutions to a problem than their peers, despite facing similar challenges and having no extra resources or knowledge than their peers. Such people are everywhere, sometimes succeeding, and depending on the conditions sometimes failing, to change the way things work – either through their own force of will, or by modelling something different. Methods to find and empower positive deviants within a community have existed for many years. But what if, by cultivating a habit of self-directed work and problem solving, it was possible to not just discover positive deviants but create new ones?

Doing things differently stems from thinking differently, and you only think differently when you learn – it’s more or less the definition of learning. Looked at this way, learning becomes the sine qua non of institutional change. It may not be sufficient on its own – structures, systems and processes still matter – but without a change in paradigm among a critical mass of deviants, those other things (which are the stuff of more traditional interventions) will always teeter on the brink of isomorphism.

We believe that positive deviance comes from learning, especially learning in a self-directed way, and learning about things that matter to the people doing them. If you can catalyse this kind of learning in individuals, you create a different kind of agency for change. If you can go beyond this and catalyse this kind of learnings in groups of individuals within an institution or set of institutions, and create a sufficiently strong holding space for their positive deviance to fertilise and affect others, then gradually whole systems can change. In fact, I’d be surprised if there’s any other way that it happens. As Margaret Mead put it, “Never doubt that a small group of thoughtful, committed, citizens can change the world. Indeed, it is the only thing that ever has.”

This is our theory of change. The methods we use – particularly the structured 6-month intensive learning and action workshop we call Launchpad – are trying above all to accelerate this learning by creating a safe space in which to experiment, teach ideas and methods that disrupt the status quo, and create new team dynamics and work habits among bureaucrats. By working with senior and political leaders at the same time, we are trying to foster different management habits, to help prevent positive deviance being stamped out. In doing all this, the goal is to cultivate individuals, teams, departments and ultimately institutions that have a habit of learning – which is what equips them to adapt and solve their own problems.

This does not mean that the model is necessarily better at achieving project delivery than other methods out there, although so far it has been effective at that too. The difference is that we are willing to let individuals or even teams fail to deliver, because it is critical for the learning, and without learning there is no change in the long term. Doing this is sometimes frustrating and costly, and certainly requires us gritting our teeth and not intervening, but what we see so often is agents and groups of agents working their way out of tricky situations with better ideas and performance than when they went in. They are more empowered and capable to provide the agency needed for their countries’ development. This is the goal, and it can be achieved.

 

 

PDIA: It doesn’t matter what you call it, it matters that you do it

written by Matt Andrews

It is nearly two years since we at the Building State Capability (BSC) program combined with various other groups across the developing world to create an umbrella movement called Doing Development Differently (DDD). The new acronym was meant to provide a convening body for all those entities and people trying to use new methods to achieve development’s goals. We were part of this group with our own approach, which many know as Problem Driven Iterative Adaptation (PDIA). 

Interestingly, a few of the DDD folks thought we should change this acronym and call PDIA something fresher, cooler, and more interesting; it was too clunky, they said, to ever really catch on, and needed to be called something like ‘agile’ or ‘lean’ (to borrow from approaches we see as influential cousins in the private domain).

The DDD movement has grown quite a bit in the last few years, with many donor organizations embracing the acronym in its work, and some even advocating for doing PDIA in their projects and interventions. A number of aid organizations and development consultancies have developed other, fresher terms to represent their approaches to DDD as well; the most common word we see now is ‘adaptive’, with various organizations creating ‘adaptive’ units or drawing up processes for doing ‘adaptive’ work.

‘Adaptive programming’ certainly rolls off the tongue easier than ‘Problem Driven Iterative Adaptation’!

Some have asked me why we don’t change our approach to call it Adaptive as well, others have asked where we have been while all the discussions about names and titles and acronyms have been going on, and while organizations in the aid world have been developing proposals for adaptive projects and the like (some of which are now turned into large tenders for consulting opportunities).  My answer is simple: I’ve made peace with the fact that we are much more interested in trying to work out how to do this work in the places it is needed the most (in implementing entities within governments that struggle to get stuff done). 

So, we have been working out how to do our PDIA work (where the acronym really reflects what we believe—that complex issues in development can only be addressed through problem driven, iterative, and adaptive processes). Our observation, from taking an action research approach to over twenty policy and reform engagements, a light-touch teaching intervention with over 40 government officials, an online teaching program, and more, is clear: the people we work with (and who actually do the work) in governments don’t really care for the catchy name or acronym, or if PDIA is clunky or novel or old and mainstream. The people we are working with are simply interested in finding help: to empower their organizations by building real capability through the successful achievement of results.

We thus seldom even talk about PDIA, or adaptive programming, or DDD, or agile or lean, or whatever else we talk about in our classrooms and seminar venues back at Harvard (and in many of our blog posts and tweets). Indeed, we find that a key dimension of this work—that makes it work—is not being flashy and cool and cutting edge. It’s about being real and applied, and organic, and relational. And actually quite nodescript and mundane; even boringly focused on helping people do the everyday things that have eluded them.

So, PDIA’s lack of ‘flash’ and ‘coolness’ may be its greatest asset (and one we never knew about), because it does not matter what PDIA is called…what matters is whether it is being done.

PDIA Notes 1: How we have PDIA’d PDIA in the last five years

written by Matt Andrews, co-Founding Faculty of the Building State Capability Program

We at the Building State Capability (BSC) program have been working on PDIA experiments for five years now. These experiments have been designed to help us learn how to facilitate problem driven, iterative and adaptive work. We have learned a lot from them, and will be sharing our lessons—some happy, some frustrating, some still so nuanced and ambiguous that we need more learning, and some clear— through a series of blog posts.

Before we share, however, I wanted to clarify some basic information about who we are and what we do, and especially what our work involves. Let me do this by describing what our experiments look like, starting with listing the characteristics that each experiment shares:

  • We have used the PDIA principles in all cases (engaging local authorizers to nominate their own problems for attention, and their own teams, and then working on solving the problems through tight iterations and with lots of feedback).
  • We work with and through teams of individuals who reside in the context and who are responsible for addressing the problems being targeted. These people are the ones who do the hard work, and who do the learning, and who get the credit for whatever comes out of the process.
  • We work with government teams only, given our focus on building capable states. (We do not believe that one can always replace failed or failing administrative and political bodies with private or non profit contractors or operators. Rather, one should address the cause of failure and build capability where it does not exist).
  • We believe in building capability through experiential learning and the failure and success such brings (choosing to institutionalize solutions only after lessons have been learned about what works and why, instead of institutionalizing solutions that imply ex ante knowledge of what works in places where such knowledge does not exist).
  • We work with real problems and focus on real results (defined as ‘problem solved’, not ‘solution introduced’) in order focus the work and motivate the process (to authorizers and to teams involved in doing the work).
  • We—the BSC team affiliated with Harvard—see ourselves as external facilitators of a process, and do not do the substantive work of delivery—even if the results look like they won’t come. Our primary focus is on fostering learning and coaching teams to do things differently and more effectively; we have seen too many external consultants rescuing a delivery failure once and undermining local ownership of the process and the emphasis on building local capability to succeed.

This set of principles has underpinned our experimental work in a variety of countries and sectors, where governments have been struggling to get things done. We have worked in places like Mozambique, South Africa, Liberia, Albania, Jamaica, Oman, and now Sri Lanka. We have worked with teams focused on justice reform, health reform, agriculture policy, industrial policy, export promotion, investor engagement, low-income housing, tourism promotion, municipal management, oil and electricity sector issues, and much more.

These engagements have taken different shapes—as we vary approaches to learn internally about how to do this kind of work most effectively, and how to adapt mechanisms to different contexts and opportunities:

  • In some instances, we have been the direct conveners of teams of individuals, whereas we have relied on authorizers in countries to act as conveners in other contexts, and in some interactions we have worked with individuals only—and relied on these individuals to act as conveners in their own contexts.
  • Some of our work has involved extremely regular and tangible interaction from our side—with our facilitators engaging at least every two or three weeks with teams—and other work has seen a much less regular, or a more light touch interaction (not meeting every two weeks, or engaging only be phone every two weeks, or structuring interactions between peers involved in the work rather than having ourselves as the touch point).
  • We have used classroom structures in some engagements, where teams are convened in a neutral space and work as if in a classroom setting for key points of the process (the initial framing of the work and meetings at major milestones every six weeks or so), but in other contexts we work strictly in the environments of the teams, and in a more ‘workplace-driven’ structure. In other instances, we have relied almost completely on remote correspondence (through online course engagements, for instance).

There are other variations in the experiments, all intended to help us learn from experience about what works and why. The experiments have yielded many lessons, and humbled us as well: Some of these experiments have become multi-year interactions where we see people being empowered to do things differently, but others have not even gotten out of the starting blocks, for instance. Both experiences humble us for different reasons.

This work is truly the most exciting and time consuming thing I have ever done, but is also—I feel deeply—the most important work I could be doing in development. It has made my sense of what we need in development clearer and clearer. I hope you also benefit in this was as we share our experiences in coming blog posts.

 

The “PDIA: Notes from the Real World” blog series

written by Salimah Samji

We are delighted to announce our new PDIA: Notes from the Real World blog series. In this series we will share our lessons from our PDIA experiments over the past five years, on how to facilitate problem driven, iterative and adaptive work . We will also feature some guest blog posts from others who are experimenting and learning from PDIA. We hope you will join us on this learning adventure!

Read the first blog post written by Matt Andrews here.

SearchFrames for Adaptive Work (More Logical than Logframes)

written by Matt Andrews

Although the benefits of experimental iteration in a PDIA process seem very apparent to most people we work with, we often hear that many development organizations make it difficult for staff to pursue such approaches, given the rigidity of logframe and other linear planning methods. We often hear that funding organizations demand the structured, perceived certainty of a logframe-type device and will not allow projects to be too adaptive.

In response to this concern, we propose a new logframe-type mechanism that embeds experimental iteration into a structured approach to make policy or reform decisions in the face of complex challenges. Called the SearchFrame, it is shown in the Figure below (and discussed in the following working paper, which also offers ideas on using the tool).

SearchFrame

The SearchFrame facilitates a transition from problem analysis (core to PDIA) into a structured process of finding and fitting solutions (read more about ‘Doing Problem Driven Work’). An aspirational goal is included as the end point of the intervention, where one would record details of ‘what the problem looks like solved’. Beyond this, key intervening focal points are also included, based on the deconstruction and sequencing analyses of the problem. These focal points reflect what the reform or policy intervention aims to achieve at different points along the path towards solving the overall problem. More detail will be provided for the early focal points, given that we know with some certainty what we need and how we expect to get there. These are the focal points driving the action steps in early iterations, and they need to be set in a defined and meaningful manner (as they shape accountability for action). The other focal points (2 and 3 in the figure) will reflect what we assume or expect or hope will follow. These will not be rigid, given that there are many more underlying assumptions, but they will provide a directionality in the policymaking and reform process that gives funders and authorizers a clear view of the intentional direction of the work.

The SearchFrame does not specify every action step that will be taken, as a typical logframe would. Instead, it schedules a prospective number of iterations between focal points (which one could also relate to a certain period of time). Funders and authorizers are thus informed that the work will involve a minimum number of iterations in a specific period. Only the first iteration is detailed, with specific action steps and a specific check-in date.

Funders and authorizers will be told to expect reports on all of these check-in dates, which will detail what was achieved and learned and what will be happening in the next iteration (given the SearchFrame reflections shown in the figure). Part of the learning will be about the problem analysis and assumptions underpinning the nature of each focal point and the timing of the initiative. These lessons will feed into proposals to adjust the SearchFrame, which will be provided to funders and authorizers after every iteration. This fosters joint learning about the realities of doing change, and constant adaptation of assumptions and expectations.

Readers should note that this reflection, learning and adaptation make the SearchFrame a dynamic tool. It is not something to use in the project proposal and then to revisit during the evaluation. It is a tool to use on the journey, as one makes the map from origin to destination. It allows structured reflections on that journey, and report-backs, where all involved get to grow their know-how as they progress, and turn the unknowns into knowns.

We believe this kind of tool fosters a structured iterative process that is both well suited to addressing complex problems and meeting the structural needs of formal project processes. As presented, it is extremely information and learning intensive, requiring constant feedback as well as mechanisms to digest feedback and foster adaptation on the basis of such. This is partly because we believe that active discourse and engagement are vital in a complex change processes, and must therefore be facilitated through the iterations.