HIGHLIGHTS

• Tobacco use is rising exponentially among adolescent girls in Ghana, particularly the use of shisha pipes.
• ideas42 worked to assess the impact and cost-effectiveness of a mass media campaign—developed by Good Business and Now Available Africa—on tobacco use and related norms in Ghana and to determine which interventions and behavioral insights are most important to influence smoking behavior.

The Challenge

Tobacco use is rising exponentially among adolescent girls in Ghana, particularly the use of shisha pipes [1]. There is limited evidence as to what works to prevent tobacco use among this population, and generating rigorous evidence could help illustrate the potential impact of media-based programming on tobacco-related behaviors. Additionally, conducting this evaluation through the lens of behavioral science could elucidate untapped behavioral pathways that could strengthen impact.

Our Approach

With funding from the Bill and Melinda Gates Foundation, we are working to:

1. Produced an independent assessment of the impact and cost-effectiveness of a mass-media campaign called SKY Girls, designed by the UK-based Good Business and implemented by Now Available Africa. The campaign sought to influence tobacco use and related norms.
   
2. Provided insights on the causal pathway of tobacco use, including which interventions and behavioral insights are most important to influence smoking behavior.

We first conducted formative research in the cities of Accra and Kumasi, interviewing teenage girls, their parents, and implementing partners. Our research sought to better understand the physical, social, and psychological contexts underlying tobacco use among Ghanaian girls and to explore the behavioral mechanisms and pathways by which girls are exposed or introduced to tobacco. 

We then identified key influencers within teenage girls’ context, including parents and male partners who influence both their perceptions of tobacco use and the opportunities they have to smoke. We found that girls with different social circles and contexts trigger opposing perceptions of the acceptability of tobacco use, with girls in very low or very high socioeconomic circumstances more likely to smoke—and to have friends who smoke—compared to their middle-class peers. 

Based on these insights, we conducted a cluster randomized controlled trial (RCT) in two regions in Ghana to assess the impact of the SKY Girls campaign on tobacco use and associated intermediate outcomes, including self-efficacy, use of refusal skills, and perceptions of tobacco users. The study included a panel survey of 9,000 adolescent girls over 12 months and incorporated responsive feedback mechanisms to provide implementers with actionable insights to boost the reach and engagement with their programming and to contextualize the study data.

Results

Reflecting on our experience as external evaluators, we have identified several insights and opportunities to enhance learning from rigorous impact evaluations:

• A behaviorally-enhanced theory of change aids evaluation design and programmatic decision-making. A theory of change often frames an evaluation by providing a coherent narrative for how a program’s activities are expected to generate impact. Insights from behavioral science can enrich theories of change and elucidate why and how programs generate results, as well as to consider external mechanisms and evidence-based pathways that may be relevant to the outcome but that may not have been conceived as in the purview of the program. A more detailed account of our learnings can be found in the X.

• Actionable, responsive feedback is important to implementers but context matters for choice of mechanism. Rigorous impact evaluations are essential to determining program effectiveness, yet they may not provide the feedback necessary for informing real-time decision-making and course corrections to maximize programmatic impact. Capturing feedback that’s both quick and valuable can be a delicate balance. Digital channels initially seemed promising given our young, urban population of interest, but we concluded that digital methods came with too many trade-offs. Ultimately, we chose another approach to help implementers boost engagement.

• Rigorous data collection provides opportunities for novel applications of data science to produce behavioral insights. As part of our secondary objective to provide behavioral insights on the pathways influencing smoking behavior, we sought to leverage the data collected for the evaluation to identify the most important variables for predicting initiation of smoking using machine learning (ML) techniques. We found considerable overlap between our formative research findings and the results of the ML analysis. A full write-up of these results is forthcoming.

Takeaways

Rigorous impact evaluations offer a powerful approach to guide the direction and scaling of social impact programs, but they often do not yield the valuable and actionable feedback program implementers need. Behavioral science is at the forefront of promising creative paths to better measurement, evaluation, and adaptive learning, and can help to make the findings of evaluations more actionable for implementers to improve programming and optimize impact.

Interested in our work applying behavioral science to global health? Email gh@ideas42.org or get in touch on X @ideas42 to join the conversation.

[1] Logo, D. D., Kyei-Faried, S., Oppong, F. B., Ae-Ngibise, K. A., Ansong, J., Amenyaglo, S. … Owusu-Dabo, E. (2020). Waterpipe use among the youth in Ghana: Lessons from the Global Youth Tobacco Survey (GYTS) 2017. Tobacco Induced Diseases, 18(May), 47. https://doi.org/10.18332/tid/120937