For anyone who has ever stood in front of a classroom after delivering a lesson, the question “what did they learn, and what difference will it make?” has surely come to mind.

In many cases, this is a black box, including when it comes to traditional financial education, where in-person trainings are often delinked from the desired actions consumers will take – they may learn about interest rates now, but won’t need to borrow for a few months, or even years. This may be why recent analysis (see, for example, Fernandes [2014]) has questioned the behavioral change that traditional financial education programs can have.

But what if you could deliver the learning content at the moment in time it is most needed, and directly monitor not only how the content was accessed but its subsequent impact on actual financial behavior?

This was the premise behind an interactive SMS project for users of M-Pawa, Vodacom and Commercial Bank of Africa’s fully digital mobile money savings and credit product. The platform was run by Arifu, a personalized learning tool that provides customized learning content based on consumers’ preferences and responses. The project targeted farmers in rural Tanzania, as part of the Connected Farmers Alliance, who were receiving in-person trainings.

Using the farmers’ own feedback from initial user testing, CGAP, Arifu, TechnoServe, Vodacom and the Busara Centre for Behavioral Research developed a series of interactive SMS scripts that let farmers guide their own learning on their phones, and to do so when they wanted and on the content they wanted.

For example, those more interested in loans could learn how to check their loan limit or how to use a cost calculator tool; while those interested in savings could read a story of a farmer like them who saved on M-Pawa for their business or set their own personal savings goal.

The team also tested a range of different types of behavioral messages to drive uptake of the SMS content and different learning approaches – narrative, facts, introducing the Arifu name in the SMS as a personal learning guide to increase personalization of the experience – to see which messages worked best.

Overall, the results for the six-month pilot were striking. A total of 33,782 invitations was sent to farmers. The 2,862 farmers who accessed the Arifu learning platform saved at rates more than five times those of farmers who did not access the learning platform.

Similarly, farmers who accessed the Arifu learning content – regardless of the specific delivery method – took out larger loan amounts and repaid at higher rates than those who did not access the learning content.

On average, the more screens a farmer viewed, the more financial activity they had on M-Pawa. Even more interesting still, the treatment that introduced Arifu as their learning guide from the beginning had the highest impact on financial behavior – as shown in the graph on loan amounts. This shows how personification can help amplify digital learning. During pilot testing of SMS content with farmers, several farmers inquired about Arifu, asking questions such as “Arifu nii nani?” (Who is Arifu?) and frequently assuming it was another person communicating with them.

Besides the evidence this research provides on the power of timely, user-guided digital learning content, the research also highlights two key principles for consumer protection and responsible delivery of digital finance.

First, the use of well-built, interactive content about product features – including costs – helped increase use of the product and positive financial outcomes. While we have noted in the past how many digital lenders offer poor or insufficient disclosure of terms, these findings show how robust disclosure can be a good thing for providers as well as consumers (something also documented in CGAP’s digital credit experiment with Jumo in Kenya.)

Second, the interaction between the savings and borrowing aspects of M-Pawa in this experiment show how important it is to offer not only credit, but a place to store value. By driving up savings on M-Pawa, farmers were able to more accurately share their cash flow, and receive larger, more accurate loan amounts, leading to larger amounts borrowed and higher repayment rates for those who accessed the Arifu learning content.

We hope these results demonstrate the utility of enhanced consumer engagement, and that we see more approaches like this by digital financial service providers in the future, leading to increased consumer understanding, product use and benefit for consumer and provider alike.