Spreadsheets are where data go to die

The data problem and opportunity

570 million smallholder farms, using less than two hectares of land, account for roughly 12% of the world's agricultural land (World Development, 2016). These farms supply around 30% of our global food commodities, but are hard to reach and often with little formal organisation. It is a challenge for larger food supply chain actors to engage with these smallholders: how can a rice processing company forecast how much smallholder grain will be available this season? how can an agri-retailer target small farmer consumers about whom they know little – how much fertiliser, seed and chemicals do they really need to improve the yields on their parcel of land? And what is the cost benefit of investing in data collection and management systems for such businesses to better understand their supply chain or their target market?

In their 2016 paper Big Data and Smallholder Farmers, Protopop and Shanoyan cite “the unprecedented opportunity for mass mobilization and aggregation of information and data on the smallholder farm level in developing economies”; between 2005 and 2015 mobile broadband use has increased (an incredible) thirty times. ‘Big data’ is data sets that are high in volume and with great variety, providing insights that were not previously available, and that enable timely decision-making for supply chain actors and policy makers. There is no question that the use of big data is already starting to transform agricultural supply chains. For example, with the advent of affordable electronic tablets, and increasing capacity and coverage of mobile data, options to use mobile apps to conduct smallholder surveys have opened up. The most rapid uptake has been in high value agri-sectors where firms already had their own teams of field workers who could switch from pen and paper to tablet. Streamlining data collection to manage smallholder suppliers for such businesses is a very obvious move – it is quicker, less prone to error, less time consuming, and therefore costly.

Creating survey tools for use on tablets is relatively simple using open source software – even incorporating drop down choices and fields that reduce entry errors. But what happens to the data sets at the other end? Most likely they will be downloaded into spreadsheets for analysis. This makes sense for a one-off exercise, but is not ideal if you want to be able to build up records year on year to really understand performance and preferences of smallholder suppliers or consumers. As one of my colleagues likes to say, “spreadsheets are where data go to die”. I like to refer to the differentiation between ‘data’ and ‘information’: When data are processed, interpreted, organized, structured or presented so as to make them meaningful or useful, they are called information. With tens of fields of data on thousands of individuals, turning this into usable information requires a database – in essence, a customer relationship management system.

Exploring potential benefits

This is where the opportunities for inclusive business get a lot more interesting. Not only can smallholder suppliers and consumers benefit from more targeted procurement and sales from agro-processors and agri-retailers, the information such companies collect can serve several other purposes:

• Firstly, as a platform for more directed agronomic advice, leading to better smallholder performance;
• Secondly, to build up a verified track record of production that may unlock credit opportunities for farmers who have no collateral, such as land ownership. Mercy Corps has, for example, developed a mobile platform AgriLife which works through farmer subscription and facilitates lending based on information collected by farmer-centric enterprises;
• Thirdly, for smallholder verified identities and track records to enable participation in traceable supply chains which offer greater market and price security and possibly ethical trading premiums.

These three benefits to smallholders are a win-win for all parties involved. Introducing working capital into the supply chain by improving smallholder crop yields, access to loans and better prices means agro-processors have access to more production inputs and agri-retailers have better informed and wealthier customers.

 Smallholdr business plans

In this regard, I am excited to be part of the team growing ‘Smallholdr’, a business that serves this opportunity to better capture and use information on smallholders for both commercial and development returns. Smallholdr deploys a mobile app and a web-based platform built on a semantic database with a proven track record. The system has been trialled in supply chain management and for outgrower schemes, and we are now rolling out our first retail-based solution for an agribusiness with over 14,000 farmer records. Not only do we see the benefits of our system to private sector clients, but also to NGOs and development programmes. Good quality information on farmers is essential for designing and monitoring performance for agricultural interventions; and as above, a database-driven system in this regard is far superior to ‘snapshot’ surveys. 

To build up records per individual farmer, there needs to be a means of identifying each individual - this can be national identity cards (where they exist), or an ID generated by a company / NGO, combined with a photo and a phone number. Once this has been set up, each time an extension worker meets a farmer, any new information collected is immediately linked to their profile in the database. We use QR codes when printed cards are issued to smallholders which can be scanned for identification and also can be used as loyalty cards which track sales linked to purchase for retail clients.

Not only does the Smallholdr system provide real-time information and analytics for our clients, it enables managers to design and assign specific surveys to their extension teams / field agents. This functionality is a huge boost to resource management challenges of a remote team. Moreover, extension workers have their own versions of the Smallholdr management tool on their tablet app, as well as access to all the farmer information they have collected, and any the opportunity to have any agronomic training materials (documents or even videos) that they might require.

Our Smallholdr application is one of many innovations that are seeking to use big data for commercial and development returns. As the sector grows, so will the volumes of data. Alongside the clear benefits of data sharing, particularly among development partners, there are also some challenges regarding data privacy and security and the underdeveloped legal infrastructure in the sector. With weak governance in data protection in many countries, the use and potential abuse of personal data becomes a risk for the future. At Smallholdr we will be following the progress of best practice in this area and providing our own guidance and processes for protecting information rights.

Another consideration is raised in Whose Data is it Anyway? by Adams and Thorp of the Acumen Lean Data team: as they rightly say, “We often collect data with little regard for the people behind the numbers”, and they advocate for ‘humanising’ the principles and practices of data collection. Interviewees should be shown respect as a minimum, and, without raising expectations, they should get a sense that they get something back for the information they share. However, this is harder to achieve using phone surveys, which are the preferred choice for Acumen’s Lean Data approach. This approach has limitations, not least because mobile phone ownership among low-income farmers is a long way from being the norm.

At Smallholdr, our ambition is that our app, in the hands of an informed extension officer, enhances relationships – and trust - between formal businesses or NGOs and small farmers. This trust is an essential building block to increased trading and performance in the supply chain, and to the virtuous cycle of increased yields and increased sales. If you share our vision of this opportunity and would like to discuss related ideas or potential collaboration, I would love to hear from you!

This blog is a part of the November 2017 series on data for inclusive business.

Read the full series for insights on how the data revolution could affect inclusive business. Will it bring an end to the uncertainty of business in Base of Pyramid markets? Can it straddle the development-business divide? Will the data drive spurred by the Sustainable Development Goals be useful to inclusive business?