Challenges and Recommendations for Aggregating Social Impact Data at Portfolio Level
Priscilla Boiardi, Knowledge Centre Director, European Venture Philanthropy Association
Venture Philanthropy Organisations (VPOs) want to aggregate impact information at portfolio level to manage better, i.e.:
- to improve the way they select investments
- to sharpen their investment strategy
- to take corrective actions when issues arise (i.e. when an investee is not performing as the rest of the portfolio)
Additionally, VPOs want to report on the impact achieved at portfolio level for marketing and fundraising purposes, i.e.:
- to show their own investor the impact they are achieving
- to use the information for additional fundraising
However, VPOs realise the importance of social purpose organisations (SPOs) setting their own objectives and impact measurement (IM) system, as the information collected by the SPO needs to be informative for the SPO’s management and improve the SPO’s operations. Additionally, aggregating data for reporting is technically difficult. The more customised the IM system, the more difficult it will be to aggregate data at portfolio level (and for reporting). It is therefore important to think of ways to show outcomes at portfolio level.
This tension between the bottom-up approach and the need for aggregation brings challenges that VPOs have tried to solve in the following four ways.
1. Align the Theory of Change of the VPO and SPO and have a couple of portfolio-wide outcome indicators
Making sure the Theory of Change of the investor and investee are aligned is crucial for an investment to be successful (and is a guiding principle in the investment selection phase). As the indicators used to measure impact are derived from the Theory of Change, it is possible for the VPO to find one or two outcome indicators that can be applicable and relevant for all investees. Hence, a solution is to have all investees reporting on one or two common indicators, and then have tailored indicators for each investment, depending on the specific management needs of the investee.
Example: In order to measure how well the VPO is achieving its own outcome objectives, Reach for Change is attempting to define overarching indicators that are linked to the investment themes within its investment strategy. For example, in Ghana, improving children’s education is the most important objective for Reach for Change. Beyond just counting how many children are helped by the specific programmes Reach for Change supports, it would be interesting to find concrete indicators that measure improvement in children’s education in Ghana and track progress over time (although attribution of Reach for Change’s investments will be impossible to define). If there is no progress, Reach for Change clearly would need to revise its strategy.
2. Define portfolio-wide Issues
Some organisations give their investees freedom to choose investment-specific impact indicators, but then define portfolio-wide issues (often ESG goals) that all investees have to report on.
Example: Investisseurs et Partenaires, a VPO based in France, for example believes there is value in tackling climate change issues at portfolio level, so supports all SMEs in the portfolio implementing actions to reduce their carbon footprint. The downside is a lack of flexibility in adapting the objectives to the specific needs of each investee. To overcome this it is important that the ESG approach and in particular the ESG action plans are co-developed with the investees and tailored to each company’s challenges and opportunities. With this approach it is possible to:
- analyse the individual and collective contribution of the investees in the portfolio to local development – and perform periodical reviews
- benchmark the performance of each investee against the performance of the portfolio and implement corrective actions if needed
3. Report on the change that happened
Organisations that support a high number of investees using different indicators to measure impact find it difficult to aggregate the results and measure impact at portfolio level. However, simplifications are possible by measuring the percentage improvement on the different indicators.
Example: For each specific investment, Reach for Change can track the progress the investee is making on achieving its customised outcome indicators. The degree of achievement of those indicators (e.g. 80%) will be a measure of success of the investment for Reach for Change, and can allow it to make a social “valuation” of its portfolio of investments by calculating an average success rate.
4. Annual survey with standardised response options
Some VPOs assess their impact at portfolio level by looking at how they have improved the way their investees work. An option is to use questionnaires with Likert scales. The standardised response options enables a VPO to aggregate the answers and to form a view at portfolio level of the impact the VPO has been making on the SPO and to assess how the SPO perceives its capability to deliver impact has changed thanks to the work with the VPO.
Example: This is the solution found by Reach for Change, a VPO in Sweden. Aggregated answers enable status snapshots of Reach for Change global, regional and local level portfolios, e.g. by making statements such as: “At the end of Q1, 40 % of our entrepreneurs had not yet identified a sustainable way to make money”. This provides an indication of where the entrepreneurs need the most support from Reach for Change, and how the VPO could most efficiently allocate its resources.
This blog is from a wider discussion document on the challenges of aggregation, collated and put together by Social Value UK. The whole document provides perspectives from a range of sectors, and can be downloaded in full from the Social Value UK website.