Quality education benefits individual learners (the South African term for students in primary schools), economies, and societies at large through increased earnings and economic growth, better health choices, better-functioning institutions and public services, and more. South Africa, among other countries, has sought to better realize these benefits through increased spending on education. However, while South Africa’s education spending has increased in recent years, school performance metrics continue to lag expectations.

While many factors contribute to lower-than-desired performance metrics, lack of data for education administrators and teachers on school performance presents one significant challenge. Past research has shown that access to and use of accurate data on school performance can help improve education outcomes for learners. As a result, in the past decade, governments, multilateral agencies, NGOs, and other actors around the world have made concerted pushes to put a range of data into the hands of government officials, including education administrators and teachers. But, we know from behavioral science that access to data alone doesn’t guarantee it will be put into action that impacts people’s lives. In 2020, ideas42 partnered with the Michael & Susan Dell Foundation to determine how to remove barriers and enhance the impact of education data usage in South Africa.

As part of a larger set of initiatives targeted at improving South African educational outcomes, the Department of Basic Education (DBE) in partnership with the Dell Foundation, introduced an innovative data platform in 2013. The Data Driven Districts (DDD) Dashboard allows education administrators to positively impact learners through the ability to make data-driven decisions, from identifying and working with teachers with high absentee rates to identifying low-performing schools and developing targeted interventions to improve their performance. To support these decisions, the tool contains an online dashboard and downloadable reports with information on a variety of metrics, including learner and teacher attendance, test performance, and grade advancement. There are numerous instances where administrators are putting the dashboard to good use to make these types of decisions. For example, one curriculum official we spoke with uses the dashboard to identify low performing schools and subject areas within his district and offer targeted support based on the data. However, many administrators and teachers do not use the tool to the fullest extent.

While over 10,000 South African educational administrators and teachers access the tool each term, the DDD program aimed to further magnify impact by increasing uptake by the thousands of education officials who currently do not use the dashboard to allow for improvements in areas from earlier identification and response to decreasing performance in schools to improved allocation of resources among school.  We worked with the DDD implementation team and the Dell Foundation to apply a behavioral lens and explore ways to increase use of this tool by more education administrators.

When designing the dashboard, the DDD implementation team envisioned a tool that was both technologically innovative and user friendly for everyone in the education system who needed to access it. In partnership with the dashboard implementing program team, we explored the behavioral barriers that prevent educational administrators from using the DDD Dashboard efficiently and effectively. While data in the dashboard is relevant for educators at all levels, we focused specifically on education administrators at the circuit level (analogous to the district level in the United States). We looked to identify barriers that prevent them from using the data dashboard to formulate interventions for lower performing schools.

As education administrators use the dashboard, they go through several steps, which we identified through diagnosis interviews and a process audit of the dashboard. As users go through these steps, several behavioral barriers may arise along the way, and prevent successful use of the dashboard (see figure):

• Misleading social norms about who data tools are meant for and how many people use them depress use. In many cases, peer use looks much lower than it actually is because use of a tool is invisible (i.e. people don’t see their peers using it). This is especially prominent as people are working from home during school closures.
• In addition, people experience present bias. Educators are strapped for time and may overweight the near-term costs of learning the tool relative to the long-term benefits and time savings of mastering the tool.
• Decision makers also hold strong, outdated mental models and use default approaches when making decisions, which makes it difficult to integrate novel tools and prevents them from using data.
• Small hassle factors, such as forgetting their passwords, can further deter users when they’re getting started.
• Finally, users lack support tools to translate the data into action and follow through on implementation intentions after using the dashboard, such as reminders, planning prompts, and guidance.

The DDD Dashboard is consistently evolving based on real-time feedback. The DDD team incorporates input about what type of features best drive learner success and how these features can most easily be used when updating the dashboard. These findings will be incorporated into the dashboard itself as well materials and programs outside the dashboard, such as informational materials, to support better use of the dashboard and translation of insights, gleaned from the platform, to data-driven decisions.

Although this work focused on enabling education officials to access and apply learnings from the dashboard, many of the insights we captured are relevant to data use across different fields. While any interventions or changes to systems and practices should be adapted to suit their specific context, we believe that many of the barriers will be broadly applicable.

Based on both what we learned with this dashboard, and previous work encouraging South African municipal officials to use data when making policy decisions and responding to citizen requests, we’ve identified a set of common important barriers to account for, as well as broad recommendations for addressing them. Institutions, organizations, and government agencies can use these insights to help their decision makers follow through on using data systems that can significantly improve their decisions and relevant outcomes, but may face barriers to adoption:

• Decision makers have limited attention, which is already split over multiple tasks. New systems must capture attention, particularly by demonstrating how the tool will save decision makers time and effort while also improving their performance. Interventions can also showcase value through “jump start” reports that show key metrics and visuals in an appealing, concise format.
• Decision makers often face tremendous time scarcity. This scarcity makes them both more risk averse to approaches that could strain their already busy schedules and also causes them to tunnel, or focus on immediate tasks at the expense of long-term time saving and strategic opportunities. New data systems should clearly prove they will save time in the long run and also “create time” for users to learn the tool in the short term.
• Many decision makers have established habits that can be hard to break and are also averse to the risk of piloting unfamiliar approaches. Implementers of new data tools must carefully shift these habits or incorporate the new tools into habits. Stories and case studies about people who have successfully learned and incorporated the tool into their work can help users familiarize themselves with a new approach.
• Decision makers hold firm mental models about data tools. They often see these tools as difficult to learn and as being outside their responsibilities. Implementers should shift mental models to frame quantitative tools as a support tool, rather than a burden. Integrating data into day-to-day tasks (e.g., a browser extension tool that allows users to search for/integrate data when making a decision) and providing additional support for policymakers, such as interns or administrative assistants who can collect data, can also make leveraging data easier and less daunting.

As data tools, including data platforms, permeate into more areas, they have the potential to drastically improve decisions and outcomes. However, our findings suggest that it’s not enough to simple make data available to decision makers, it’s also crucial to design systems for use in the context of their work. Behavioral science reminds us that context matters. Those implementing data systems need to not only think through the data collection and display but also create the right environment for potential users to take it up and translate the data into informed actions.