Editor’s Choice, June 2017: Do we really know if acceleration and advisory support make a difference? A new addition to the scattered evidence

'Pockets of rich data and huge gaps in our knowledge.'  That's my answer to the question of how much do we know about the effectiveness of acceleration and advisory support.

The array of services and organisations that provide advisory support to inclusive enterprises are young. Most providers are less than a decade old. So it's not surprising that the evidence base on whether this support has any impact is even younger and just emerging.

As more money and effort is put into providing 'more than money' support, so interest in real data has grown.  The volume of evidence has grown too, but in a very lumpy way.  Parts of the puzzle are being answered, but other parts are still empty blanks.

By far the biggest data set is focused just on accelerators (technically accelerators and incubators) not on other kinds of advisory support such as technical assistance from investors, challenge funds or specialised consultancies.

This data set comes from the Global Accelerator Learning Initiative (GALI) and this month's editor's choice is their second major report.  The first, looked at the results for ventures from 15 different programmes of Village Capital.  The new report, entitled Accelerating Start-ups in Emerging Markets: Insights from 43 Programs, focuses on the difference between acceleration in emerging and mature markets.

But even if your interest is simply how well acceleration works in emerging markets, irrespective of whether it is similar to developed markets, its data is useful.

What is most impressive about the GALI report is that it gets real data from enterprises that applied for acceleration, including those that were accepted and those that were not.  The latest report draws on a sample of 2,455 ventures that applied to 43 programs operating in 9 countries.   The core data comes from those accepted for acceleration (roughly 150 each, in emerging and developed markets) and those rejected (roughly 500 in each market).

Two key findings are:

1. There is a significant difference in performance between the accelerated ventures and those rejected.
2. There is less difference than might be expected between acceleration performance in mature and emerging markets.

The difference in performance between accelerated and rejected enterprises in emerging markets applies across their ability to grow revenue, increase employment, raise equity and take debt.   During the year of acceleration, on average, and adjusted for different venture size, accelerated enterprises grew their equity at 53% compared to 40% for rejected enterprises, and their debt at 52% compared to 9%. Their revenue grew at 39% compared to 26%, and their full-time employees at 43% compared to 30%.  So, across the piece, accelerated enterprises achieved growth rates in these indicators that are 13-14 percentage points higher, except for debt which is a huge 43% higher.   The findings on debt and equity are statistically significant at the p<.05 level, while on revenue and employees they are not.

The main limitation of the GALI approach, however wide the data set, is that it cannot prove causality.  There are likely characteristics of the applicants which led to them being accepted and led to their higher growth (confounding variables).  The extent to which the difference in performance can be attributed to these characteristics vs the acceleration input is unknown.

The GALI report provides much more detail on the comparison with mature markets, and why some assumptions that emerging market accelerators perform poorly have not been substantiated.   I leave you to read those well-summarised findings in the report, and instead want to put the GALI report in the wider context of 'what do we really know'.

These are the things that we can be pretty confident about in assessing the value of acceleration and other advisory support:

1. The growing data set from GALI, drawing on thousands of enterprises that were accelerated and were rejected for acceleration continues to show clear quantifiable differences between the two groups, particularly in the growth of their debt and equity. This emerges from a range of GALI reports to date, including the latest. However, the data is not able to show, at this point, whether accelerated enterprises achieve higher success rates long term.

2. An increasing number of organisations that have objectives to support inclusive businesses are investing a share of their money in technical and advisory support, rather than putting it into finance. Challenge funds and impact investors alike are setting aside a decent share of their pot for advisory support (10% may be typical for impact investors, it can be as much as 30% for a challenge fund).  This reflects their conviction that for their portfolio companies to scale, advisory support is needed rather than just more cash.  This is a strong theme of a forthcoming USAID report, assessing the landscape of advisory support for impact enterprises.

3. There is considerable qualitative information from enterprises that have used advisory support to good effect. Two blogs in our series this month from Reel Gardening and PharmaChk are among them.  And there is qualitative information from support providers who gather feedback, that their support is seen to add value.   Blogs from Technoserve and MDF are illustrative of this.

4. The role of advisory support is not always just to increase business growth and chance of success. It can be to increase the social impact of the business, and its ability to combine commercial and social return. Abigail Thomson, Program Director of the AAF Technical Assistance Facility, that sits alongside the African Agriculture Fund, provides just such an example, outlining how their support enabled Meridian Farm Services to pilot then mainstream a new approach to extension support for smallholders.    For this kind of input, success cannot be judged as increases in business growth, but increases in ability to combine steady growth with higher social impact.

5. Advisory support is highly varied in its quality as well as in its delivery. So, there is high value and low value support, and not enough yet known about what distinguishes these two, although providers are increasingly able to reflect and share good practice that they have learnt. More in the upcoming USAID report on technical assistance. 

Much advisory support is provided alongside financial support, whether grant, debt or investment. In such cases, it is likely impossible to disentangle the contribution of the two.  So our emerging data set on acceleration is likely to remain the strongest evidence base in this field.

And what are the most important things that we DON’T know?  I would flag two, based on the literature to date, and based on understanding the chain of logic for advisory support to look like this:

1. The more immediate results of advisory support: whether advise is actioned and incorporated, how valuable entrepreneurs rate it and whether capacity increases. The best providers of advisory support gather client feedback, but so far there are no tools to standardise, aggregate or compare this.  I've seen very little at all assessing how advisory inputs are actually actioned.  The issues of how clients value and adopt advisory inputs are more proximate to delivery of support than longer term changes in equity or revenue.  So while they don't provide a final answer, they suffer less from confounding factors.

2. Whether overall portfolio success and failure rates change thanks to advisory support. Impact enterprises are by nature risky. Anyone supporting them, whether with cash or technical support, expects a share of failures.  The driving motivation behind advisory support is that it enables more to succeed, faster.   But we have no benchmark for comparison.  We don’t yet know what failure rates might be 'normal', over what timescale, in which context.   While we will never have an exact counterfactual, to know what a specific business would have done with and without advisory support, over time we will build up better benchmarks of success rates.

Where does this leave us?  In my mind, we need to welcome and learn from the pockets of rich data that erupt, such as GALI.  But we also need to traverse the landscape, filling in some other troughs (such as client adoption), and to synthesis together different types of information, quantitative and qualitative, social and financial, against a backdrop of what I hope will be a growing evidence base about what counts as realistic expectations of performance in this sector.

This blog is a part of the June 2017 series on advisory support for inclusive businesses in partnership with USAID and the African Agricultural Fund’s Technical Assistance Facility, both of which deliver advisory support and have new analysis of it just launched (AAF’s TAF) and forthcoming (USAID).

Read the full series for more lessons from seven different providers of advisory support and stories of success from entrepreneurs.

 

NOTES

1. Accelerating Startups in Emerging Markets is the second major report from GALI, written in collaboration with Deloitte. It compares acceleration in emerging markets versus high-income countries. GALI is a collaboration of The Aspen Network of Development Entrepreneurs and Social Enterprise @ Goizueta, Emory University.
2. A recent report on the first five years of the AAF Technical Assistance Facility provides data on how business success of SMEs was increased, and business adoption of inclusive business models by larger corporates was achieved. For more information see blogs by Abigail Thompson, Anthony Kouderis, and Kate Diaz, on the AAF TAF report page.
3. A forthcoming USAID report on the landscape of advisory support included an extensive desk review of literature, plus interviews with 20+ providers. Interestingly it found a hidden wealth of insight on what works well, or not, in providing advisory support, some pockets of rich entrepreneur feedback, but relatively little else by way of data to add to the GALI data on accelerators.