What should and shouldn't Development Finance Institutions be doing and reporting?

Development Finance Institutions (DFIs):  confident, uncertain, challenged, experienced, defensive, charting a new path?  All the above.  Resistant, curious? Perhaps.   Sure of how to track and demonstrate their results?  Only in small part.  Needing to change - well that's the fundamental question. How do DFI's stick to what they are good at, avoid what they are not good at, but keep up with changing times?

A fascinating workshop last week at the Overseas Development Institute brought together leaders from various development banks with a handful of policy-makers, researchers, NGOs, thinkers.   The not-so-simple challenges we tackled were: what development impact can DFI's deliver and how could that impact be measured?  The question is hugely relevant because (1) private finance is seen as crucial to deliver the SDGs and DFI's have the job of catalysing that; (2) some donors are putting more of their money into DFIs - for example DFID investing more in CDC in the UK  - so need clear strategy for results (3) new tools and mechanisms for DFIs to deliver greater development impact, whether through blended finance, higher risk tolerance, or development impact bonds - are mushrooming.

Before we get into the details of understanding impact, it was the implicit messages that struck me most. The development bankers in the room were strikingly clear:

• proud of what they do well - private sector investment  - and particularly the catalytic affects that their investments can create:  crowding in investment in a fragile economy or sector, building a financial sector, spurring growth post-conflict;
• clear on what DFI's cannot do, and should not be expected to do: tackling every aspect of poverty and development, working in small markets, creating rather than selecting new investments, tackling social challenges and short term problems, and stepping in where the problem is government failure not market failure;
• frustrated that DFIs are misunderstood by the development community;
• clear on the different mentalities between aid and investment: aid being planned, top-down, seeking to build solutions to specific problems;   investment comes from the bottom-up so investors are supply driven, seeking to add value deal by deal.

But they - and all of us - were uncertain how DFI's should respond to the growing calls for them to have greater development impact, to contribute to the SDGs, to take greater risks and seek more impact by incorporating concessional elements of finance.  The question of how DFIs should move into the bottom of the pyramid business space is just one sub-question within this framing.

In defining what the role of DFIs should be, I felt four fundamental questions at play: 

1. Just how far should DFIs go to adjust to new realities without becoming inefficient in using their own instrument?  It is a long-standing refrain that investors are better placed than others to assess and manage risk, assess structure and manage deals, to get businesses to grow and investment to flow.  But worse placed than others to tinker with other objectives or tools.  But DFIs cannot stand still.  Businesses have already moved:  utilities, health, education were all once thought of as state responsibilities, but the private sector has moved in.  More and more official development assistance is heading towards development banks, with 'blending' concessional and commercial finance the flavour of the day.  DFIs money and private finance are expected to help deliver the SDGs.  So move they must.  The question is how far can they move while keeping efficient and effective?
2. How tightly should DFI investment be targeted?  We heard plenty of real life examples of how every target or filter reduces the 'addressable market'.  If the highest environmental standards must permeate all aspects of the business, if a fixed percentage of entrepreneurs must be women, if products must be targeted at the BoP, if investment must be in post-conflict countries, then the addressable market shrinks and shrinks.  DFIs are unlikely to create new investments that meet the criteria, so they just filter others ones out. This is a clear strategy choice: is the aim a greater volume of DFI investment to catalyse a greater volume of private sector investment and growth of pretty much any kind? If so, leave the investors free to find the 'best' deals that need DFI finance.  Or is the aim to achieve much more specific types of investment, which are seen to have higher social benefit per deal but higher effort per deal and smaller total scale?   If so, define it, but recognise that other investments, perhaps perfectly viable investments that generate jobs but don't meet other policy objectives, get screened out.  The trade-offs are real, the choice will depend on whose capital is deployed and why.  
3. Is the main development benefit of DFI finance from their direct or indirect impact?  DFIs are generally good at reporting jobs created, government revenue generated, and some investment outcomes such as gigawatts generated or SMEs that grow.  Yet most of the enthusiasm for DFIs emerged when discussing their potential to leverage finance from others, to spur economic growth over a very long time scale, and to have a demonstration affect in the financial sector or a nascent industrial sector.  There is a clear tension here between what people believe DFIs achieve, and what they can demonstrate.
4. What can and should be done from the 'core business model' and in-line with DFI's normal return expectations, and what requires special structures, initiatives, models or finance envelopes?  This relates to a discussion at ADB in February, where we discussed DFI's roles in supporting inclusive businesses (commercial models with scalable systemic impact at the base of the pyramid).  Some DFIs such as IFC do it as part of their core business, with the same return expectations, but adapting risk assessment and staff responsibilities.  Others support it with more concessional and risk tolerant finance. 

Given that clarity is lacking on what exactly the development impact of DFIs should be, it's perhaps rash to offer tips for measuring that impact.  But in the context of fluidity, I have a few tips for making progress on the challenge of measuring DFI impact.

1. There are three reasons to measure impact of DFIs and they need different things.  The first is to prove to externals:  this generally requires credible information with a high degree of aggregation.  The second is to improve:  real-time disaggregated data can inform management of both social and commercial performance.  The third is to choose: use lessons from experience to prioritise alternative strategies going forward.  This needs detailed information with more comparability between deals.  The first should not be done at the expense of the other two.
2. Distinguish clearly between the direct and catalytic impacts of DFIs.  This means clarifying objectives of a deal at the start: some deals will be great for directly creating jobs, some for providing what other businesses need, such as energy or transport.  Other deals will have pretty ordinary direct impacts, but be important longer term for demonstrating the investability of a model, a sector, or an economy and so catalysing market change.  We should not expect every deal to perform on every count.   But in order to assess both types of DFI impact we need to know where to look for which and 'what counts as success' for any deal.  It is increasingly good practice among impact investors to clarify the 'routes to impact' of any deal for Investment Committee.  This should not only inform decision making but should permeate monitoring and judgements of success.
3. At deal level, what counts as 'good enough' impact? That can't be assessed without understanding the risk that is taken, resources invested and the financial return too.  Deals will vary in how these four are balanced, so they must be analysed together. Certainly most external reporting (I like to hope not all internal reporting) presents development impact aggregated, but in isolation from resources, risk and return, so means very little indeed.
4. It is time to get smarter at capturing the catalytic effects of DFIs.  I believe it is hugely difficult to quantify this and apply rigourous attribution, but very easy to do much better at qualitative tracking of leverage and demonstration affects. It needs decent baselines of investment and business behaviour at the start, and plausible tracking of change.  The debates on how to assess system change and additionality are mushrooming in other fields (such as market system programmes, see Further Information below) but those are perhaps not typical skills amongst DFI economists, investors and evaluation departments.  There is also more emerging evidence at the macro-economic level of the contribution of DFIs to productivity and growth (watch ODI for more on this, forthcoming).
5. Co-investment should probably be treated either as risk-mitigation or as leverage achieved, but not both.  If co-investors were going to invest anyway, fine. That simply reduces risk. If they put their money on the table partly because of the DFI, that can be perfectly credible, so long as it's clear how the DFI engagement changed the risk-return trade-off for others.  External reporting of total co-investment as total leverage totally miss this distinction.
6. It can be a mistake to always aim for comparability and aggregation.  Reality is complicated and sometimes complex details are valuable.  Where impacts can't be converted into numerical values that can be summed, I like to simply rank them (high medium low, or 1-4 or 1-5) so that we can then see the span of high to low and explore the reasons why. 
7. There are huge public policy questions to answer about the cost benefit of deploying official development assistance through DFIs.   Where this requires more monitoring and evaluation than can be afforded in a conventional DFI model, it will need grant funding or an adjusted model. It will also need several years to answer the questions. Meanwhile, sharing the questions as framed and the data as it builds will create trust and collaboration.
8. Whatever the logic, the metrics and the resources put into measuring results, they won't be enough without curiosity and collaboration across skill sets.   Investors are known for their confidence in why they do what they do and that they do it well.   The challenge its to maintain that confidence while opening up avenues of curiosity and bringing in a diverse set of skills that can tackle questions of long-term impact.

Finally, no such discussion would be complete without looking at additionality.  As one participant asked, is it the holy grail or grim reaper?  Trade-offs emerged again.  When decisions are based on 'could the private sector invest in this' it leads to false positives: perhaps 10 logistic deals are dropped because the private sector 'could'.  But only 3 happen, and 7 are left on the table.  The 7 - we were told - are the cost of 'tax-payer comfort' that DFIs are adding value.

Conventionally additionality is split into 'input additionality' (leveraging more input thanks to your own) and output additionality (creating more impact due to your involvement).  I prefer two more practical distinctions:

• additionality as a decision-making framework vs an M&E challenge.  To be frank, DFIs are wasting their money if they invest in something that would have happened just as well without them.  At investment stage, additionality is a crucial mind-set.  To be equally frank, as much money could be spent trying to measure additionality as could be spent on a deal, so perfect measurement is not worth aiming for, so long as the right processes are in place at investment stage.
• 'it would not have happened without' the investments vs 'it happened better, faster or at a bigger scale'.   These two types of additionality require different measurement techniques.

The first of these (it would not have happened) is probably due to a financing gap, or the 'pioneer gap' or other form of market failure.  The more risky and innovative models take time to be investible.  Buying down the risk, or financing the gap until they meet commercial criteria offers huge additionality. The problem is the risk of financing a deal that is just a bad venture, rather than a slow-burn or high-risk high-return one.  Whether it is the specific deal or catalytic affects in a sector that 'would not have happened anyway', the DFI input that achieves this is likely to be a first mover and first prover, taking significant risk. 

The second of these ('bigger better faster') can results from the investment, or the sector, expanding more quickly due to greater inflow of other investment,  or performing better due to higher standards of performance or management skills.   It is probably relatively easy to argue the case for this kind of additionality in many DFI deals - but harder still to plausible measure it.

The role of DFIs in the development space looks set to grow not wane.   The premise of ODI's workshop was that we need to be clearer on the scope and limitations of DFIs vis a vis other actors.   What should they do and what shouldn’t they?   The workshop was just one conversation along the way in this.   But the framing of the question also suggests that more work will need to be done making direct comparisons of DFI roles and roles of others which is even more ambitious, interesting and needed.

Further information

1.  Interesting reflections on the role of Development Banks in the specific inclusive business space were made at the Inclusive Business in Asia Forum, organised by the Asian Development Bank, in February.  This looked at how Development Banks can support business models with both commercial return and explicit contribution to solutions for the poor, with approaches varying from those that do so expecting and achieving their typical financial returns (IFC and ADB), to those that do so through a specific concessional envelope (European DFIs).   Different models for in-house structuring of team roles were compared. These are captured in the Forum report, here (not longer available).

2.  Work by others in other fields on assessing impacts on market systems or additionality of input to the private sector include:

• Impact Evaluation for Market Systems Programmes, and other material on the BEAM exchange site.  This is from the perspective of donor-financed programmes that aim to catalyse systemic change in how markets function.
• Demonstrating Additionality by the Donor Committee for Enterprise Development.  This is from the perspective of donor investments in companies, particularly through challenge fund mechanisms.
• Evaluating Ecosystem Investmentsby Omidyar and FSG.  This is from the perspective of venture philanthropy, investing in both non-profits and for-profits.

3. The discussion at the ODI roundtable was under Chatham House rules, and built upon a previous public workshop in April, also covered in a blog post: Why should donors subsidise the private sector?

4. Research on the role of DFIs has been done and continues to be done by ODI, led by Dirk Willem te Velde.