Solving Complex Problems in Albany

Guest blog written by David Galin

Coming into this course, I was under the impression it was going to help me better understand the nuances of implementing policy from a roadmap that was created for every situation.  I was a bit nervous we would be taught a rigid set of procedures on how to implement policy – something that was made for every situation but really only worked for maybe one out of every ten, if we were all lucky.  Thankfully I was very wrong.  I learned about a theory that works for those situations that aren’t rigid and need meaningful analytical evaluation.  Situations where you need to think outside the box, need support from your authorizers, but also need to continually build a team.  Situations where you are not only implementing policy but in fact problem solving.  Read: messy, confusing, complex situations.

I learned there truly are a variety of issues – complex and complicated and everything in-between.  I think this is a principle that sometimes you think about in the back of your head but start to say to yourself you’ve overcomplicating the issue and that can’t be.  Turns out, it is.  I learned that you need to see things for what they are but also be willing to look past the first layer of the issue.  You need to unpack the problem.  People are very quick to hear what they think the issue is, and immediately try to come up with solutions.  Sometimes the issue isn’t complicated and that type of problem solving can work.  But sometimes the issue is so complex that you need to spend a significant amount of time unpacking the problem.  Taking the time to understand the hurdles in front of you and the hurdles that may be hidden beneath the surface, before developing a game plan. 

The other thing I learned is that PDIA is as much about relationships as it is about process.  Building relationships – before, during, and after iteration and implementation – is very important.  Having established relationships can cut down on the time needed to build them when trying to solve a complex problem, and helps foster a sense of trust – not only with your authorizers, but with your peers.

The entire process is designed to create a constant feedback loophelping you to review whether your potential solutions are working or not, but also to getting you working with other people, obtaining and re-affirming authorization from your superiors, and brainstorming additional methods to tackling an issue.  When it comes to our problem, we were able to learn that data-driven decision-making is optimal to use as part of PDIA.  Having data and being able to evaluate it before and after the feedback review helps to determine whether that iteration was successful or not.  We made progress narrowing down some of the core issues behind the perceived sub-optimal performance of See Click Fix, including no consistent methodology of using “acknowledged” vs. “closed,” and have also seen a decline in days to acknowledge and days to close as part of the expanded use of ipads as part of our improvements.

Continue reading Solving Complex Problems in Albany

EEP/Shiree: Using adaptive programming to monitor change in Bangladesh

written by Salimah Samji

How do you effectively monitor an 8 year, £83.5 million (around USD$135 million) challenge fund that partners with NGOs to improve the livelihood of 1 million beneficiaries? A daunting task indeed.

The Economic Empowerment of the Poorest (EEP/Shiree) program is a partnership between the UK Department for International Development (DFID), the Swiss Agency for Development and Cooperation (SDC) and the Government of Bangladesh (GoB), whose objective is to lift 1 million people out of extreme poverty by 2015. The fact that it is a challenge fund that has managing partners, consortium academic partners and NGO partners, all with many moving pieces, made it crucial to have agile decision making tools in order to respond to the needs of their beneficiaries in real-time. Traditional M&E methods of baseline, midterm and endline surveys were deemed insufficient.

The need for real-time measures and iterative decision making created the space for experimentation and innovation. Armed with authorization from their donors, the EEP/Shiree team set out to explore, experiment and create Change Monitoring System 2 (CMS 2). There are a total of 5 CMS tools which include in-depth life histories. They crawled the design space to find and fit solutions that would work in their context (pilot-test-adapt-iterate). Here is a summary of the three pilot phases:

  • Phase 1: Optical reader technology: They first created a simple survey for the NGOs partners to administer and fill out. The surveys were then digitally scanned. They quickly learned that this was too cumbersome a process and it took 2-3 weeks to receive the surveys. The time-lag was too long, they needed something more efficient.
  • Phase 2: Java enabled phone: Since mobile penetration is high, they partnered with mPower to develop a ten minute, monthly census survey on the phone. They equipped up to 20 field officers (the front line personnel who work at the field level with beneficiary households) with simple mobile phone devices that used the Bangla script for the survey. It was meant to be a 6 month pilot but it lasted for 1.5 years by which time they had scaled to 100 devices, with surveys and simple visualization. Convincing NGO partners as well as the visualization and the development of an in-house feedback loop mechanism took much longer than had been anticipated.
  • Phase 3: Android smart phone: The dropping costs of smart phones in the market (android phones were $60-70) created a lucrative option. The smart phone allowed greater flexibility (field staff just update the app on their phone), more functionality and accountability (GPS location of households, photos and voice recording verify that the beneficiaries are being met regularly). mPower also built a dashboard that allowed the comparison and served as a litmus test to identify red flags that required further investigation, ultimately allowing the NGO partners and EEP/Shiree to tailor recovery plans to the beneficiaries needs and changing context.

Slide1

After the trial-and-error and incremental adjustments over three pilot phases, EEP/Shiree deployed a full roll out of the system towards the end of 2012 (3 years later) with the use of smart phones. EEP/Shiree project partners have over 700 smart phones equipped with an Android operating system, internet connectivity and GPS capability, and have been monitoring over 100,000 households every month across Bangladesh as well as accessing information through an online visualization dashboard that is updated in real time.

Here are some of the challenges they faced:

  • Bringing NGO partners on board: The NGO partners were reluctant and viewed the collection of data as an imposition from above. Asking, “why do we have to do it?” and saying “we don’t have time.” They did not understand that the data and the dashboard could serve as a management tool for themselves. NGO partners were then involved in the design of the questions and were included in the process. It took approximately eight months for data collection to cross the 100,000 per month mark which has since been consistently met and represents most households.
  • Infrastructure constraints: Accessing the dashboard from some areas still face connectivity issues. De jure, every field officer is supposed to visit once a month but de facto not all of them do. The sheer scale of the program makes it physically difficult to monitor. While changing the survey questions is easy – you just download the new form on your phone – the back end dashboard change costs are high. Furthermore, by changing questions you lose the ability to compare across time.
  • Effective use of existing data: While the data is used to respond to the needs of the beneficiaries, very little predictive/trend analysis is done. The data is not used to challenge assumptions of what works and to continuously refine their understanding of the dynamics of ascents out of and descents into extreme poverty. This is partly because no one is responsible for this task and so it doesn’t get done.

Complex problems do not have clear solutions. The fact that the donors were flexible and created the space for experimentation and innovation allowing several pilots to be tested (all with good reasoning) is commendable. Throughout the process, EEP/Shiree and mpower co-designed CMS 2 and their continuous cycle of partnership led to a virtuous cycle of action. The leadership on both sides meet every 2-3 months to discuss what is working and what is not, which helps adapt process to technology and technology to process. Together they built a dynamic monitoring tool, proving that this can be done at scale. This is a far cry from the usual case of consultant comes, builds an MIS system and then leaves.