Practical Application of a Relationship-Based Model to Engagement for Gene-Drive Vector Control Programs

The UCMI program objective and field site selection.

Our aim is to conduct an ecologically confined field trial of a genetically engineered mosquito with a low-threshold gene drive. We have selected a field site that maximizes prospects for success, minimizes risk, and serves as a fair, valid, and convincing test of efficacy and the impacts of a gene-drive product intended for large-scale deployment in Africa.^2.

The UCMI program believes that island sites are the ideal environment for achieving this aim. Applying the WHO guidelines and definition for an ecologically confined field trial,³ the UCMI conducted a rigorous evaluation of potential island field sites, which concluded that the island nation of STP would be an optimal site.²

The nation of São Tomé and Príncipe consists of two volcanic islands located 140 km apart in the Gulf of Guinea, roughly 250 and 225 km, respectively, off the northwest coast of Gabon. The larger island, São Tomé, has a total area of 854 km² and a human population of about 193,000, and Príncipe has a total area of 142 km² and a population of 7,324 people.⁴ It is a Portuguese-speaking country with a young population (about half of the population is under 18 years of age).⁵ Its remote location and small size and population limit the development and economic activities in the country, resulting in elevated poverty rates (15.6%), lack of employment opportunities, and government reliance on external financing.⁵

Human occupation of the STP is recent, having been initiated roughly 500 years ago with the first wave of Portuguese colonization.⁶ Malaria and its mosquito vector (A. coluzzii) were introduced into STP at about the same time.^7,8 Malaria is now endemic in the STP, and although it was considered to be in the pre-elimination phase in 2009, the current trend indicates otherwise.⁹ The number of malaria cases in the STP have been increasing each year since 2020, a nearly 30% increase in malaria cases from 2020 to 2021¹⁰ and a nearly 50% increase from 2021 to 2022.¹¹ The National Malaria Control Program in the STP reported a similar increase in the number of cases in 2023, and in the first 12 weeks of 2024, cases have been nearly 50% higher than they were in the same reporting period in 2023 (unpublished data provided by the National Malaria Control Program).

In late 2018, the UCMI program presented the government of STP with an evaluation of island sites and information about program technology and objectives. Discussions regarding a partnership with the Ministry of Health and the National Malaria Control Program began, and with this, a relationship was initiated between the UCMI and STP. The discussions surrounding partnership and collaboration included the subjects of field research and baseline data collection, capacity strengthening and training, and engagement.

Engagement for gene-drive research.

Advancements in the development of gene-drive technology for malaria control have accelerated the need for effective engagement practices and have stimulated discussion and debate surrounding best practices, recommendations, and guidelines for meeting this need. Gene-drive technology has the potential to spread and persist in nature, presenting important questions related to its scope and impact over space and time. This also means that the scale of human populations potentially affected may be quite large and, without opportunity to “opt out” of the intervention or participate in decision-making about its use, raises different sets of ethical and engagement questions.¹² In addition, public perceptions and attitudes about science and technology are dynamic, and they influence expectations and opinions about how the public ought to be engaged and involved. Although no industry standard yet exists to evaluate gene drive,¹³ numerous publications have offered conceptual frameworks and guidance for gene-drive developers in consideration of these questions.^3,12–32 In many of these publications, “co-development,” shared decision-making, and participatory development are articulated as important, if not essential, elements in the development and application of the technology. The recently published second edition³ of the WHO Guidance for Testing Genetically Modified Mosquitoes places importance on “a co-development approach that emphasizes authentic partnership and knowledge engagement” for community engagement and development of the technology in general. Other papers analyzing engagement practices and the obligations implicit in them have focused specifically on community consent for the use of the technology.^33–35 Inherent in these concepts is the call for developers to reflect on their empowered positions coming into a field site and the need to develop engagement programs that acknowledge this and move to shift power back to the end-user groups.

The UCMI approach to engagement.

We agree about the importance of establishing an approach to engagement that is inclusive—shaped by existing knowledge, partnerships, and collaborations. We also agree in the importance of end-user involvement in decision-making surrounding engagement. We define end users as the population who may be recipients of the technology, which, for the UCMI program, are the residents of the STP. End users include stakeholders and community members alike. We define communities as all who are impacted by project work, equaling all residents of the STP, and stakeholders as group(s) of individuals who are not only impacted by the project work but who may also be involved in governance surrounding the project and may also participate in defining the level of community acceptance required for this project. Decision-making and the level of involvement of one versus another is determined again by the existing governance and social structures established within the end-user population.

The UCMI program leaders established value-based beliefs and objectives in the initial stage of research that shaped the program’s approach to engagement. Included in the program mission is the stated importance of working in partnership and collaboration with stakeholders and community members in an ethical and transparent manner and to establish open dialogue and relationships of trust with our partners and collaborators. We believe that engagement frameworks and strategies should be developed on a case-by-case basis, reflective and inclusive of the perspectives, values, and knowledge of the local population where the technology is being developed.²⁶ We also believe that questions and definitions concerning community acceptance and consent should be answered by the end-user groups where the technology is being considered, rather than predetermined by external experts and academics not directly engaged in the work, and that a shift in power and decision-making is essential.²⁸ Establishment of a national engagement program developed by end users for end users provides a means for answering these important questions and a foundational starting point for assessment.

The UCMI model for engagement.

The UCMI has selected a model for engagement that supports end users in leading the development and implementation of the engagement program, strategy, and communication from start to finish. The goal of UCMI engagement is to ensure program transparency, share knowledge, and provide end users with what they need to make an informed opinion about whether to support the use of the technology or not. Fully integrating stakeholders and engagement practitioners from the STP in the development of all aspects of the engagement program reduces opportunities for preconceived and/or predetermined strategies developed by the research program or other external groups of experts from being imposed on end-user groups. It also increases opportunities for regular community member participation, building on existing relationships and pathways for communication and knowledge sharing.

Prior to selecting a field site or initiating engagement, the UCMI program consulted with an engagement practitioner with experience implementing the relationship-based model (RBM) for public health interventions. The model is one that has been tried and tested.^36–43 It is a model initially developed for transformation of healthcare delivery, articulating the importance of relationships between the providers and the recipients of care and the importance of involving the recipient in the decisions surrounding the delivery of care.⁴³ The concepts of the model are further advanced in The Nuka System of Care developed by the Southcentral Foundation in Alaska, shifting decision-making power entirely to the end users of the healthcare system, transforming care delivery to meet Native Alaskan values and needs.³⁶ The Nuka System of Care recognized that if a public health intervention is to be successful, it should be developed by the end user, reflective of their values and voices. This is particularly important when end-user groups historically have been recipients of inequitable power dynamics and decision-making. Southcentral Foundation offers a “core concepts” training focused on the importance of an RBM in engagement with people and commnuities.⁴⁴ Core concepts and guidelines for applying the model taught at the Southcentral training have been adapted and applied by the UCMI.

The model emphasizes the importance of end-user involvement in answering the how, who, what, where, and when questions surrounding the development of an engagement program. It recognizes the importance of relationships in the integration of existing knowledge, resources, strengths, and beliefs in decision-making.

The decision made by the UCMI to adapt the RBM to its program evolved from the program’s commitment to the establishment of trusting relationships with our field site partners and the need for integration of existing knowledge, practices, and perspectives into decisions surrounding engagement at the field site.

Establishing relationships through knowledge sharing between program and end-user groups to inform program practice, principles, and target goals and establish a balance of power was a primary aim of the program. Development of trust as a foundation of a relationship requires a lot of time, effort, patience, and resilience from everyone involved. This is a core concept in the RBM and the foundation on which the UCMI engagement strategy in the STP was developed.

Application of the RBM for gene drive.

The RBM offers a framework for engagement that acknowledges the critical importance of human knowledge and participation in the success and acceptance of public health interventions. The RBM can be applied at an individual level or a community or population level. The type of application and intervention plays a role in determining who should be involved and the size and scope of the engagement effort. In many cases, the RBM has been used for interventions that an individual could choose (or not) to use. In other cases, it has been applied in health delivery models to adjust how end users receive and access care,³⁵ in which case, not every individual may agree on the final decision or choose to give an opinion. The application is determined on a case-by-case situation. The importance of the application is that is provides all end users an opportunity to be involved in engagement and have access to knowledge and information that inform their choices, opinions, and decisions.

The question then remains, how do we scale this model up to engage with an entire nation and apply it to an intervention for malaria control that may not present each individual citizen with a yes or no choice about whether to use it or not? This is the question we attempted to answer here. The UCMI has taken the core concepts of the RBM and applied them to the development and implementation of a national engagement program for gene drive. This has been done in a phased approach, with each phase dedicated to establishing relationships of trust, knowledge sharing, and end-user involvement.

Initial stakeholder engagement: Establishing trust through shared understanding and goals.

The UCMI program established an initial collaborative relationship with the Ministry of Health (MoH) in the STP in 2019. This is the ministry responsible for developing the national malaria control strategy and the governance of external partners who wish to participate in the malaria control effort.

Inherent in the core concepts of the RBM is the acknowledgment of established systems and values. These include existing laws, policy, national priorities and processes, regulation and governance, and national beliefs (both historical and current) within the country. This complements the WHO’s guidance for the evaluation of genetically engineered mosquitoes, which states the importance of adapting to current governance rather than replacing it.³ It is this initial relationship, which established program understanding of the local governance, history, societal and political structures, and priorities, that directed all future UCMI engagement actions.

The program relationship with the MoH guided establishment of other important program relationships with stakeholders actively involved in the national malaria control and elimination effort. These stakeholders included the National Malaria Control Program, nongovernmental organizations, associations, the National Public Health System, the national university, government agencies, and representatives from international organizations (WHO and the United Nations Development Program) representing diverse interests and areas of expertise. Participation expanded further on the recommendations of the members of this group to include others from civil society and community leaders. This interdisciplinary group of individuals guided the development of the UCMI engagement program and will be referenced in this paper as the engagement stakeholder group (Table 1). This group further explored existing social values and priorities, expanding the program’s understanding of priority issues, and the common societal behaviors and responses reflective of shared values in the STP.

Establishing relationships with a large and diverse group of stakeholders takes time. This is an important reason why engagement should begin early, well in advance of initiating fieldwork. The UCMI program team conducted numerous information-sharing workshops, presentations, one-on-one meetings, virtual meetings, E-mails, and phone calls with stakeholders for nearly 2 years (between 2019 and 2021). This early engagement applied skills for healthy dialogue and communication, such as the use of open-ended questions (e.g., “How should we develop an engagement program?”) and active listening and reflections (e.g., “We are understanding that stakeholders want a national engagement effort; is this accurate?” followed up with open-ended question[s]: “How can we achieve this, and who should be involved?”). Several important questions (Table 2) were the focus of group discussions. Reports and written documentation of meetings and action items further reinforced important messages and information shared with the UCMI team and gave stakeholders an opportunity to further refine and clarify directions and perspectives until a decision point was reached (Table 2). These practices are used consistently by the UCMI team in our interactions with end-user groups.

Inclusion of end users in program development is an important RBM concept and is critical in establishing an engagement program that understands and respects the values and perspectives of communities and that builds on existing strengths and resources. It is essential in the development of project messaging and communication and in the mitigation of project misunderstanding and misinformation. The UCMI, under the direction of the MoH, collaborated with the engagement stakeholder group to 1) develop a phased communication plan and engagement strategy, 2) identify a local engagement team structure, and 3) develop terms of reference for engagement team members. This work was the result of the first phase of stakeholder engagement. Through establishment of shared understanding and goals, this important end-user group guided the early development of the UCMI engagement program.

Identification of a local engagement team: Building on existing strengths and resources.

The STP engagement stakeholder group identified a team that initially included national community health agents (CHAs). These CHAs provide community-based education and awareness activities for national health programs and work under the direction of the health delegation in the communities where they live. Most of the CHAs had previously conducted awareness programs about malaria in their communities, had received basic training in malaria education, and had a deep understanding of community composition, language, and values. Their inclusion in the UCMI engagement team was important for these reasons but, most importantly, because they had established relationships of trust and respect with community members and with their colleagues in the health system. They knew what to do and how to do it. The RBM tells us that as local health educators, as well as potential end users of the technology, these are the individuals who should be involved in engagement decision-making.

Also included in the UCMI engagement team are the health district malaria focal points and health delegates. The STP contains seven defined health districts, each of which has its own health delegate (medical doctor) who coordinates and manages the health activities and care delivery in that district, including the work of the CHAs. Health delegates are responsible for coordinating malaria testing, treatment, and communication in their district, and they work closely with the National Malaria Control Program in data reporting and surveillance. The malaria focal points liaise with the health district and the malaria control program to coordinate delivery of malaria control activities and education. They are often trained malaria control technicians, and they report directly to the health delegate in their district.

Several information-sharing workshops and planning meetings were organized with the health delegates and malaria focal points in 2021 to discuss project objectives, engagement budget, engagement program structure, and a general communication plan. The inclusion of these groups in the engagement program was critically important from the perspective of the RBM because it gave decision-making authority to those who are in the best position to make informed decisions. Over a period of 6 months, this group further developed and refined a phased engagement strategy and demonstrated that they had established relationships with the CHAs and communities in their districts, a deep understanding of the national health priorities and challenges, and a commitment to the success and safety of the members of the engagement team.

The engagement strategy included broad timelines and objectives for engagement consistent with larger UCMI program objectives and goals. These included a national engagement program; phased information sharing guided by assessments of knowledge, perception, and understanding; direct collaboration with the national health systems; and activities, messages, and materials directed and developed by a local engagement team. Phases of information sharing were identified as small building blocks, each building on the next set of concepts so that understanding of more challenging aspects of the technology (e.g., gene drive and population modification) could be achieved slowly over time. Initial phases of information were defined as general malaria facts, malaria transmission, malaria control in the STP, general UCMI program information and field research (e.g., mosquito collections, mosquito population studies), malaria transmission and parasite lifecycle, control methods and the impact on the mosquito and parasite, UCMI-modified mosquito, beneficial genes and how they interrupt the parasite life cycle, gene drive and how it works to modify the population, and how the modified mosquito works as a malaria control tool. These phases were established with the understanding that assessments of community understanding, perception, and knowledge would inform the focus of each phase and information would be developed to address the community feedback provided. In addition, health delegates and malaria focal points worked to revise the initial UCMI engagement budget, establish terms of reference for the CHAs supporting the UCMI engagement program, and determine a process for CHA selection. Discussions surrounding CHA time commitments to other health projects and available resources informed this work. These early discussions and group work evolved into what is now an engagement leadership team that has taken responsibility for management and assessment of CHA project work, leading capacity building and training activities, and the regular evaluation and planning of engagement activities.

The term of reference for a local STP manager for the engagement program was also defined. The UCMI team collaborated with a smaller group of engagement stakeholders to recruit and select candidates for this position. The selected candidate met all the criteria defined in the terms of reference, most importantly, 5+ years of engagement experience in STP communities; an advanced understanding of STP political and social structures, values, beliefs and culture; and exceptional communication skills.

The addition of a dedicated, full-time engagement manager in the fall of 2021 was a turning point in the establishment of the engagement team and strategy. The manager collaborated with the health delegates to identify a total of 55 CHAs for the UCMI engagement program. Available funding and estimated days of work/month determined this number. Estimations of CHA time were derived largely based on their previous experiences working with Global Fund malaria control projects that included a community education component associated with a particular intervention. There was historical knowledge about time required to conduct door-to-door engagement and community workshops in each community. District size and population determined the CHA number in each district.

Although the initial UCMI program budget included a robust engagement component, it was created using estimates and approximations. It was understood by the project funder that the project would require a second phase of funding for ongoing activities at the field site after establishment of agreements and partnerships, which was a consideration in the early development of the team. While revising the original budget, they also developed a projected phase two budget. This included additional funding for training, capacity strengthening, and increased time and for engagement activities as the phases of the engagement increased in complexity.

On review of the initial communication plan, which included engagement in schools, the manager recommended that the program partner with the Ministry of Education to integrate biology teachers into the program for successful engagement with students. A term of reference was developed, and in collaboration with the Ministry of Education, five biology teachers were initially selected to work with the UCMI engagement team. At the end of 2021, a full engagement team was established (Table 1). It is important to note that the roles and functions of the team members outlined in Table 1 were largely preexisting. Many of these positions (health delegates, teachers, malaria focal points, CHAs, and stakeholder group members) were not dedicated full-time to the project or direct employees of the UCMI. This is consistent with how the RBM works, integrating a new program with existing resources, values, and governance.

Team-based approach: End-user development of engagement plans and activities.

The RBM took a team-based approach that recognized that every member of the UCMI team in the STP (field research, laboratory research, advisors, etc.) plays a role in engagement.²⁷ Engagement activities are integrated and coordinated with the work of the local research teams. The engagement manager facilitates regular communication with the engagement stakeholder group, research teams, and other local advisors and consultants to ensure consistent communication and information sharing among all groups.

The RBM emphasizes relationships, shared understanding, and active participation in the development of community-based activities. This consideration and inclusion ensures that program activities meet community members where they are, leveraging their perspectives, knowledge, and expertise into the decisions involved in the development and implementation of the UCMI engagement program. This was the most important part of the initial work and is detailed in Table 2.

Decisions that shaped and determined the approach to engagement were made by local authorities and stakeholders through dialogue with one another and with community members. Decision-making processes were defined as the specific steps and actions taken to reach a decision end point. Decision-making power was defined as the group (or groups) who have authority to make end point decisions. Authority in many cases is determined by the existing structure of governance and hierarchy within the STP, and in some cases, it is determined by the terms or reference for specific leadership positions defined by these existing authorities or governing bodies.

Specific engagement activities, UCMI field research activities, and UCMI project planning are influenced by what is learned through engagement (Table 3). In the space of a publication, it is difficult to describe all of the project learnings from engagement with end-user groups; however, Table 3 is intended to illustrate some specific examples of how dialogue with the community shaped and defined UCMI project work in the STP.

Early dialogue with community members was intended to establish a baseline understanding of community knowledge and perspectives related to malaria, the UCMI program, and preferred methods of engagement. The engagement team determined that the best way to answer the question “What is the most effective way to conduct engagement?” was to ask the community members (Table 2). The team developed a quantitative assessment designed to gather information from communities across the country.

Community assessment 2021.

The initial assessment was developed with the engagement team, stakeholders, UCMI team, community leaders, and other public health professionals. The aim of the assessment was to gather baseline data about community knowledge, perspectives, and preferences related to malaria and malaria engagement in the STP. The assessment was conducted in November 2021 in 29 communities across all seven districts in the country. The number of communities included in each district was determined by the total population size in the district (larger number of communities for more populated districts), and specific communities within each district were randomly selected. Twenty individual assessments were completed in each community for a total of 580 responses. This number was determined by available resources (financial, human, and required transportation and supplies). Individuals were randomly selected from 20 different households, and an effort was made to assess an equal number of male and female participants. All individuals who participated in the assessment were over 18 years of age.

The results of the assessments were recorded, and the initial analysis was conducted by a University of STP student in collaboration with a University of California Davis administrative assistant and research assistant (Supplemental Table 1). Results were analyzed by the engagement manager, health delegates, and the UCMI team, and findings were shared and discussed with stakeholder groups involved in developing the assessment. The results demonstrated that each district had different perspectives on malaria and current malaria controls, but overall, nearly 80% of the population was concerned about malaria, 70% reported knowing little about malaria transmission, and preferred methods for receiving health information were through community meetings and digital media (TV/radio/Facebook).

An initial community engagement plan and district-specific activities were developed by the team to address the results of the 2021 assessment. Focused activities applied community-preferred methods of engagement and focused on education about malaria transmission. The CHAs shared the results of the assessment with members of the communities in January 2022 using visual summaries of the data specific to each district and community. The plan also included sharing general information about UCMI program objectives, team composition, and the initial field biology work and baseline data collection being conducted in partnership with the National Malaria Control Program. In addition, the CHAs identified a process for responding to community questions, concerns, and grievances to ensure that community feedback and opinions would be continuously integrated into the engagement planning processes.

Engagement plans for schools and stakeholder groups were developed using a similar process. Each plan tailored for the specific groups (e.g., primary schoolchildren, school faculty, community leaders, university students, government officials) was based on feedback and suggestions received during initial engagement activities with students and teachers in classroom presentations and with stakeholders during workshops and information-sharing events (Table 3).

Engagement plans for each group (communities, schools, stakeholders) included a written guide for the engagement team. These described goals and objectives during specific time frames that built one upon the other to ensure a shared understanding among all groups about the program research, goals, and objectives. The engagement manager played a critical role in coordinating collaboration and communication within the entire program team. The manager provided leadership and direction in the development and implementation of engagement activities coordinated within each stage or phase of program research. Bringing together the engagement stakeholders and the UCMI field biology research team ensured that engagement activities were fully integrated with the UCMI field biology research. The development process was realized through coordinated, integrated capacity strengthening to determine the most effective engagement activities to build understanding and acceptance. A detailed example is shown below using mark, release, recapture (MRR) studies conducted in the STP.

Engagement for MRR studies.

The UCMI program previously identified specific, standard practice field biology studies that needed to be done to provide information to national stakeholders that was essential in their eventual evaluation of the UCMI technology. Mark, release, recapture studies within the natural population of mosquitoes was one of these. The MRR studies evaluated the dispersal and size of the mosquito population.

The development process began by sharing the established protocol with stakeholders who would be involved in implementation and study design, including scientists, public health leaders, malaria program leaders, scientific advisors, and local engagement and entomology teams. The protocol was revised based on their feedback, and a shared set of goals for the study were established. Revisions included adjustment to specific sites where MRR studies were to be conducted based on accessibility and allocation of resources, time periods for studies based on historical weather data and local knowledge of weather patterns, and training and identification processes of volunteers assisting in the study. The final protocol was then sent to the STP Health Ethics Committee for evaluation and approval. The work in creating a study design and protocol with a multidisciplinary stakeholder group was possible because of the relationships of trust that had been previously established with these groups. Group members understood the program and the objectives and trusted that their opinions would be valued and integrated into the study design.

After the shared set of goals, design, and protocol had been established, the engagement team collaborated with the local field team and community leaders to develop an engagement plan for the study, including key messages and information that would be essential to share with community members and a training program for the engagement team specific for the study. Months prior to the study, local field and engagement teams implemented the training for the engagement team and conducted public consultations in communities where MRRs would be conducted. Comments and concerns from the communities (Table 3) were then used to further refine the study plan and to inform the engagement team in its development of activities leading up to the study.

The local engagement team developed a specific plan that included numerous awareness activities specific for the communities in and around the study, as well as communities distanced from the study. The goal was to inform the entire country about the study and the planned outcomes. An animated video and narration were created explaining the study methods, objectives, and measured outcomes and shared on national TV and radio and social media. Poststudy data and results were also shared using an animated video and through face-to-face awareness activities across the country.

This example demonstrates how inclusion of end-user groups in decision-making encourages relationships of trust and ensures shared understanding and program transparency. Together, the engagement stakeholders, field research team, and engagement team developed engagement activities that fully integrated with field research activities while adapting to the specific concerns and feedback of the end-user groups being engaged.

Developing an environment that supported the model.

Successful implementation of the RBM depended on bringing people together to share, collaborate, create, and participate in the program development. It also relied on establishing a system of gathering and responding in real time to feedback from the end-user groups being engaged. Establishment of an environment to support the work (encouraging good communication and providing safe spaces for open communication and sharing) is an essential component of the model. The UCMI program considered this as a foundational step in building the program and provided participatory communications training for staff, leaders, and researchers in the fundamental concepts of the RBM, as well as general communication skills and methods to support open dialogue, collaboration, and trust. The training was provided by a communications consultant with experience in training public health professionals and community groups to apply the RBM for program development projects.

The same training was developed for the local teams in the STP in partnership with the consultant and local STP public health professionals who had previous communications trainings and experience with health education programs. Provision of participatory communications training is an invaluable component for successful RBM implementation and has been a priority area of investment for the UCMI program. The engagement team has received quarterly, 3–5-day trainings over the last 18 months that included multiple interactive modules combining formal presentation, group work, practical exercises, and assessments of knowledge and understanding through pre- and posttests for each module. These trainings were codeveloped and delivered by local engagement stakeholders, malaria control stakeholders, educators, and social scientists. Every training included a communication and engagement module focused on supporting good communication and dialogue. Aims of the trainings included providing an environment for team building, information sharing, and capacity strengthening surrounding the complex research and science behind the project but, more importantly, the capacity to establish the same safe sharing environments in their communities where they conduct engagement activities. Training modules included UCMI program objectives and work, malaria biology, malaria transmission, control and treatment, engagement communication, RBM model, engagement methods and importance, communication skills with practical applications, responding to conflict and concerns, and development of engagement activities. To reinforce skills learned in training, the engagement team within each district met monthly and worked together to support good communication and information-sharing practices. Materials and messages for the monthly engagement activities in the communities were reviewed, and small group practice exercises were facilitated by the engagement manager within each district team.

Additional training and participatory team meetings and learning activities were conducted for members of the field and engagement teams and UCMI program leadership and administrators. Regular weekly meetings between the program engagement, field science, and administrative team leaders encouraged the active practice of open dialogue and sharing of information. This information was then transferred to the larger engagement team to help inform its development of program activity plans and materials. Monthly meetings with stakeholder groups provided an opportunity for the engagement manager to review materials (videos, booklets, posters, etc.) with a multidisciplinary team who provided advice and feedback. This ensured accuracy of content, and it promoted transparency and awareness about what information was being shared in the communities between all project stakeholder groups.

Continuous evaluation and quality improvement were important components of the UCMI program and the RBM, one that allowed the program to gather and respond to feedback and opinions of end-user groups, further expanding the opportunity for shared decision-making.²⁷ Engagement activities and personnel were regularly evaluated, and information from these evaluations were used to inform future program planning. Annual community assessments were conducted to evaluate engagement activities and methods as well as community knowledge and perceptions. Any response indicating understanding of less than 75% required adjustment, any response indicating utilization of an engagement tool or activity with lower than 50% was adjusted, and any response indicating misperception or misunderstanding of greater than 25% required immediate action. Annual exams and skill evaluations were conducted for CHAs and teachers within the engagement team. This included a written exam (passing was 80% or higher) and an oral exam where each individual was asked to conduct an engagement activity for a panel of engagement stakeholders and knowledge and skills were evaluated (80% ranking was required). Community health agents, teachers, and malaria focal points evaluated program trainings (presenters, information, and activities) to inform how trainings could be improved. Engagement leaders were evaluated annually by their peers and the people whom they supervised, materials for engagement were evaluated on a monthly basis by all members of the UCMI team and our STP collaborators, CHAs evaluated the communication skills of their peers in group practice and trainings, and regular social media and national media outlets shared contact information that people could use to communicate with the engagement team about questions, concerns, or doubts they had related to the UCMI program.

Information obtained through evaluations, assessments, surveys, and direct contact connected engagement program decision-making back to the end-user group that was most directly affected.