“The best laid plans of mice and men often go astray.” An 18th century poet like Robert Burns could scarcely have imagined something akin to the model that ideas42 and our partners at IntraHealth International designed to integrate family planning services with immunization days at health posts in Senegal, but he couldn’t have offered a better characterization of the challenges we faced implementing this intervention.
Our best laid plans
The months following childbirth are a critical time for women to adopt a family planning method and space out the births of their children, which has been shown to have long-term impacts on the health of both the mother and her children.
We know that over 90% of postpartum women in Senegal come to health facilities to access immunization services for their newborns. We also know that more than three in five Senegalese women have an unmet need for family planning. This gap in health services during the pivotal postpartum window – when health workers could be offering family planning to women when they are visiting the health facilities anyway and about to be fertile again – presented a key opportunity for ideas42 and IntraHealth International to design a program called Innovation Comportementale dans l’Integration (ICI) that integrates immunization services with family planning services.
Based on a thorough diagnosis of the behavioral barriers that prevent providers and clients from discussing family planning options during routine immunization services, we pursued an iterative design process in partnership with stakeholders from the Ministry of Health and Social Action to develop a three-pronged intervention, including:
- A series of referral cards that prompt health workers to discuss complementary family health behaviors with clients, including a cue for clients to set a specific time to speak with a provider about birth spacing.
- A series of mobile voice messages through which local actors act out a serial drama called théâtre mobile that dispels myths about family planning and gives clients timely information about important health topics during the postpartum period.
- A mobile training course for health workers delivered via interactive voice response (IVR) that builds and tests their competency in discussing immunization, child nutrition, and family planning with clients in this critical postpartum window.
Once the process was in place, we needed to measure how well this intervention was or wasn’t working. A rigorous impact evaluation had always been our intended capstone for this work. Although recognized as a promising high-impact practice by the World Health Organization, the integration of family planning and immunization services remains a largely untested proposition. Eager to gauge the impact of our diagnosis-driven model, we organized a cluster randomized controlled trial (RCT) at 78 health centers across two regions in the country. We distributed over 20,000 referral cards, recruited over a dozen trainers and data collectors, and facilitated trainings for nearly 200 health workers on incorporating our intervention materials into routine service delivery on immunization days at their health posts.
The plow levels a mountain
But just as we were about to launch the field test in April 2018, a labor dispute arose between public health worker syndicates and the Senegalese government, prompting a strike in health worker services. As the strike escalated in scope, we had to halt our implementation of the field test — after all, you can’t integrate family planning counseling with immunizations, when there are no immunization services taking place — and rethink how we were going to evaluate our behavioral model.
As a health worker later told us, “La montagne a accouché d’une souris,” or the mountain has given birth to a mouse – a French turn of phrase highlighting the contrast between our efforts in preparation for the field test and its underwhelming outcome. This experience will sound all too familiar to public health practitioners, as well as anyone working in contexts with multiple stakeholders: the best laid plans, carefully constructed over months or years, can often be thrown out by factors completely beyond our control, like a mouse’s nest felled by a farmer’s plow (or Robert Burns’ pen).
A pragmatic pivot
While we were unable to carry out a rigorous evaluation of this intervention, we were nonetheless determined to generate meaningful evidence on the potential for our family planning integration model, even without an RCT. So we pivoted from a quantitative impact evaluation to qualitative insights harvesting. This participatory research method allows the people directly involved with an intervention to report on the realities of implementation and identify strengths and weaknesses to government decision-makers.
To this end, we distributed open-ended journaux de bord diaries to health workers, the tutors who trained them, and regional health officials involved with our project, which prompted them to share their candid thoughts on the intervention. We then organized four workshops, during which nearly a hundred health workers and local government officials shared their experiences with the intervention (some are featured in the video above).
These journals and workshops culminated in a set of recommendations, such as optimizing access to our IVR messages in areas with low cell network coverage, and improving the reach of our monitoring tools.
But these qualitative tools also critically created a feedback channel between the front lines of health service delivery and the central government, allowing the health workers and regional officials to convey their thoughts on how to sustain this model on family planning integration after the strike concluded. And while local stakeholders shared opportunities for strengthening the intervention and continuing the implementation, national stakeholders also reflected on what kinds of evidence would be persuasive for the Ministry of Health to ultimately scale up the intervention, allowing us to gauge both what could be done and what needed to be done to return to the field with an even stronger intervention.
A new mountain arises
With the restoration of service delivery in public health facilities in November and the conclusion of national elections in February, we seized the opportunity to return to the field this spring. Propelled by the enthusiastic feedback from local stakeholders and the continued support of the Hewlett Foundation, we are launching a two-month pilot test this summer to demonstrate the feasibility of our model in new settings. With this second pilot test, we want to gauge if the referral card system is feasible and if our re-developed IVR system can works at various health posts in the Ziguinchor region, which is a new context from where we first tried to pilot the intervention.
It is hard to know to what degree this revival would have been possible had we not leveraged the journals and workshops to engage stakeholders during a prolonged period of uncertainty. As we discovered, the strike may have been a blessing in disguise, as it pushed us to engage with our stakeholders in new ways that allowed us to receive useful and candid feedback. An RCT may never have uncovered the same nuances. Using these qualitative methods in a moment of uncertainty allowed our intervention to continually evolve – even when implementation was impossible – and allowed us to forge and sustain connections with key decision-makers that ultimately allowed us to return to the field and implement the next iteration of our integrated service model.
While one goal of this renewed implementation is to generate support for an eventual impact evaluation, an important realization during our experience in Senegal was that multiple forms of evidence can be sufficient, and that a fixation on RCTs can obscure other approaches to evaluation that can achieve goals beyond evidence generation.
At ideas42, we always design our interventions with the high standard of an RCT in mind, but in the challenging contexts in which we work, we should keep ourselves open to pragmatic alternatives – be they quasi-experimental quantitative techniques or qualitative methods similar to those we pursued in Senegal.
Ultimately, switching to this pragmatic approach allowed our intervention to rise anew: a mouse that emerges from the ruins of its nest to build a new and improved home – one that might once again turn into a mountain. We are hopeful that this opportunity to return to the field will give us another chance to implement and evaluate an even stronger program that can provide thousands of Senegalese women with access to more opportunities to choose the right family planning service for them.
