Context Matters: Measuring the impact of solar in rural India

When it comes to decision making, it is sometimes said that a good choice made in the wrong context can be just as bad as a poor one made in the right setting. Context is key. It’s even more critical when it comes to creating social impact. So why, then, do so many of us in the social impact space seem surprisingly indifferent to context? All too often we assume that just because something creates impact in one setting, it’ll work just as well in an entirely different one.

Our new research into the impact of off-grid energy access in emerging markets across multiple geographies, products, and companies is helping us understand when context matters and why.

Energy provision across East Africa and India

Two of the previously published studies from our energy impact series have focused on East Africa. This is a geography characterized by woefully inadequate access to grid-based electricity. Only a third of Kenyans, a fifth of Ugandans and a shocking sixth of Tanzanians have access to the grid. This widespread absence of traditional electrification means that the majority of households must rely on expensive and potentially dangerous alternatives, such as kerosene, to light their homes. So when households gain access to improved solar power, we have seen significant improvements in economic- and health-related social benefits.

By contrast, in India, more than three quarters of the population have access to grid-based electricity. This extension of energy has in part been driven by the country’s ambitious goal to electrify all households by 2020. While this level of coverage sounds like good progress given India’s size, it still leaves 300 million people — a number equal to the combined population of 18 of East Africa’s 19 countries — either in the dark or reliant on dirty sources of energy. Moreover grid-based energy access is neither especially reliable nor is it equitably dispersed. In many parts of the country, particularly in rural areas, it is often the quality of electricity provided rather than access that prevents households from reaping the full benefits of energy provision.

For example in the northeastern state of Bihar, where nearly 90 percent of its almost 100 million people live in rural areas, average energy consumption is barely a sixth of the national average. This is precisely the reason that Acumen investee Husk Power Systems, a mini-grid company providing solar-powered electricity to thousands of rural Indians, chose Bihar to set up shop. As is the case with many solar companies operating in places where the grid is failing, Husk competes with a wide range of alternative sources of energy such as kerosene, diesel and torches. For providers of solar energy, this landscape is not especially unusual, but the competition is especially intense in India—the country is the world’s sixth largest subsidizer of fossil fuels, including major government subsidies for kerosene.

Because of these differing circumstances, we were eager to investigate whether the positive impact of solar we had seen elsewhere would be equally effective in Bihar.

Researching the impact of solar in rural India

We set out to learn more about the impact Husk was having on various aspects of customers’ lives, such as their household assets, energy expenditure, nighttime activities, purchase of kerosene, and changes in children’s literacy and math skills. In partnership with researchers at J-Pal South Asia and with co-funding from LGTVP and USAID, we started a three-year, randomized controlled trial covering some 3,000 households across the East and West Champaran districts of Bihar. The researchers divided these households into three study groups — one with access to Husk’s services at full price, one with access to Husk’s energy services at a discount, and one that served as a control group with no access to Husk’s services.

Here’s what we learned:

Consumers of Husk Power are highly price sensitive. As part of the study, a quarter of the participants were offered a 50 percent discount on Husk’s monthly service price, from Rs. 200 ($3) to Rs. 100 ($1.50). In the areas where the price reduction was offered, demand for Husk’s services shot up from a two percent share of the market to an impressive 20 percent. As a result, the company is now looking at re-evaluating its entire pricing strategy. More broadly, households were more likely to purchase solar in areas dominated by diesel generators compared to those with a reliable grid.

Consumers of Husk Power use more electricity overall. Regardless of whether consumers of Husk’s services were offered a discount, their electricity usage increased significantly compared to the control group that did not have access. Families living in villages powered by Husk mini-grids saw a 38 percent increase in daily electricity use; those who received discounts increased their daily use by a whopping 77 percent. The researchers also saw double-digit increases in mobile phone usage, between 14 percent and 19 percent, respectively.

Kerosene purchase did not fall and total spending on energy remained unchanged. If there has been one consistent finding across Acumen’s surveying of thousands of low-income energy customers, it’s that when customers gain access to solar energy, kerosene usage falls dramatically (often to the point of total elimination). Much to our surprise, this fall in kerosene purchase failed to materialize for users of Husk’s mini-grids.

The reason for this is directly connected to the context in which Husk operates. In Bihar, kerosene is not readily available on the open market. Instead, families purchase an allotment, based on the size and poverty status of the household, from the government at a subsidized price. Anecdotal evidence suggests that families continue to take their full share of kerosene regardless of whether they use Husk’s services. They then find other uses for it, such as paint thinner or as a fuel additive, or they trade it on the black market typically in exchange for diesel power.

Better lighting had a small yet significant effect on educational performance. Unlike other studies on solar lighting’s impact on children’s study time, the researchers found that kids in homes with Husk Power Systems did indeed read more at night. As a result, those children saw an increase in scores by as much as 13 percent in both reading literacy and math.

The effect of context on social impact

These results, as compared to those from our studies in East Africa, paint a more nuanced picture of the impact of off-grid solar in India. Certainly, there are some highly encouraging findings, particularly those related to educational performance. However, the fact that overall spending on kerosene did not decrease — an almost universal finding elsewhere and something we at Acumen have historically considered a surefire sign of impact — suggests that the economic and environmental impacts of solar on households in rural Bihar may be lower than in East Africa.

The fact that context can play such an important role in impact suggests that for those serious about understanding and creating lasting social change it’s not enough just to assume that a solution, product or innovation will be impactful. This is true even where that assumption is predicated on significant prior evidence. Instead, in all but the rarest of cases, we need to investigate the impact we create with new data, direct from the end-consumers.

This should come as little surprise. Social transformation is complex, often personal and prone to change over time as societies themselves also change. This is not cause for despair, but rather a reminder of the importance of continual learning when it to comes to measuring social impact.