Results are like London buses - right now we have plenty

Anyone living in London knows the problem with London buses:  you wait for ages with not a bus in sight, but then they all come at once.   Doing programme monitoring is a bit the same.  We spend months designing forms, getting systems in place, clarifying terms, and waiting.  Then at the end of a programme, we are drowning in data, forms, heavy excel files.

The last 6 months of BIF have seen us crunch our way through a mound of data and qualitative feedback,  and generate a raft of new outputs about the results of the programme, and about aspects of how they are measured.

For most audiences, we just tend to flag the high level findings, always trying to put them into 2, 4 or 6 pages.   But members of this network probably are interested in the detailed results and in how we got them.     So here is a summary of what is now available, plus a few immediate reflections:

1.  The Logic of the Business Innovation Facility - this explains the rationale of the programme, which underpins the M&E process and the results we are now sharing. A set of diagrams, that builds up over half a dozen slides.

2. The full Portfolio Review, capturing the portfolio of 40 inclusive business in graphs, trends, and percentages, from top rated challenges, estimated turnover to reach to people at the BOP, and perception of BIF additionality.   It's a desk of 85 slides of glorious detail.  At our final event, someone queried our 'transparency'.  This is my answer. 

3.  Snapshot:  Review of the Portfolio, Results and Trajectories of Inclusive Businesses.  This is a 6-pager summary, highlighting the main findings of the portfolio review. 

4. Two spotlights focusing on results at the base of pyramid, what we looked for and what we found.    The first,  Understanding Impacts at the BOP looks across the portfolio, while another on emerging results focuses specifically on How is inclusive agribusiness benefiting companies and farmers?

4.  Spotlight: Ingredients and Results of Inclusive Business: Findings from the Pilot.   This goes beyond the detail of M&E results, to draw out wider findings and lessons from the 3.5 years. In includes a diagram  that I expect to see us and others re-use -  a pyramid showing that we think BIF-supported businesses are reaching those between $1 and $2 per day, plus other above and below this segment.

We have little hard data on the income or socio-economic profile of the customers or suppliers of most of the businesses.  The new data has come mainly from our recent 'deep dive' case studies, launched earlier this month.  The set of seven cases can be accessed here (not longer available).

The Spotlights are all short.  Yet to come, we will share a longer document explaining the BIF M&E system, and our final reports which underpin the findings we have been sharing at our events and webinar during December.   The best place to find all the final outputs, published and forthcoming, is here:(not longer available)

We've learnt a lot in producing these.  One is the vast amount of time it takes to first crunch, then interrogate the data, continually going back and forth between interesting findings, raw or corrected data.  Our team of three  (Carolin Schramm, M&E Manager, Adriano Scarampi  (M&E Officer) and myself spent a lot of timing reviewing and revising.    Another, of course, is the frustration of reporting company results after just a year or two since our intervention.  Most are on a course for profit, but not yet there, and data is rough.   If only the M&E could be repeated, one or two years from now, really good data would exist.

Another lesson is that few benchmarks exist.  One universal indicator we can report is the number of households at the Base of the Pyramid reached and expected to be reached.  Impact investors tend to report this too, although they tend to multiply by five/six for household size AND count all of the BOP reached by any investment in their portfolio: in BIF we calculated a share based on our level of additionality.   In other donor programmes, counting households or people reached is often not enough. As one person commented yesterday,  if that was the sole indicator, we would all do radio programmes.   But the donor programmes that calculate net earnings of the poor or net economic contribution tend to be those working with producers, where increased yield and income can be calculated.  The majority of the BoP reached by BIF-supported businesses gain access to energy, quality education, information.  We certainly to not have enough data to ascribe monetary values to that.   So when we report that a large share of the BIF £6mn budget went into technical support to 40 companies, who we realistically expect to reach 3.7mn households within 5 years (which is 2-4 years from now), of which about 1.5 million could be plausibly linked to BIF support, we do not know how to benchmark this with performance of others.   Or whether such crude deductions, that ignore much else in terms of input and impact, are worth it.

The final lesson, for now, is a very positive one.  We have struggled from the start with two challenges:  keeping the burden of M&E light on companies, and keeping commercial information confidential while maximising our knowledge outputs.  As strong relationships have been built, we have found our company partners surprisingly willing to relax on both.  Several have gone on to build on the M&E that we kicked off with them, and used it to help bring in other external funders/investors.  Very few indeed have asked us to edit any public material, even when we highlight challenges, investments or turnover.   While our biggest weakness probably remains our reliance on self-reported information from companies,  I am pleasantly surprised at how far we managed to go using their data and triangulating it with our own.