• View in gallery

    Schematic representation of a rolling malaria indicator survey (rMIS).

  • 1.

    O'Meara WP, Bejon P, Mwangi TW, Okiro EA, Peshu N, Snow RW, Newton CR, Marsh K, 2008. Effect of a fall in malaria transmission on morbidity and mortality in Kilifi, Kenya. Lancet 372: 15551562.

    • Search Google Scholar
    • Export Citation
  • 2.

    Ceesay SJ, Casals-Pascual C, Erskine J, Anya SE, Duah NO, Fulford AJ, Sesay SS, Abubakar I, Dunyo S, Sey O, Palmer A, Fofana M, Corrah T, Bojang KA, Whittle HC, Greenwood BM, Conway DJ, 2008. Changes in malaria indices between 1999 and 2007 in The Gambia: a retrospective analysis. Lancet 372: 15451554.

    • Search Google Scholar
    • Export Citation
  • 3.

    Kleinschmidt I, Schwabe C, Benavente L, Torrez M, Ridl FC, Segura JL, Ehmer P, Nchama GN, 2009. Marked increase in child survival after four years of intensive malaria control. Am J Trop Med Hyg 80: 882888.

    • Search Google Scholar
    • Export Citation
  • 4.

    malERA Consultative Group on Monitoring Evaluation and Surveillance, 2011. A research agenda for malaria eradication: monitoring, evaluation, and surveillance. PLoS Med 8: e1000400.

    • Search Google Scholar
    • Export Citation
  • 5.

    Rowe AK, 2009. Potential of integrated continuous surveys and quality management to support monitoring, evaluation, and the scale-up of health interventions in developing countries. Am J Trop Med Hyg 80: 971979.

    • Search Google Scholar
    • Export Citation
  • 6.

    USAID. External Evaluation of the Peru Continuous Survey Experiment (December 2007). Available at: http://www.ghtechproject.com/resources.aspx. Accessed July 5, 2011.

    • Search Google Scholar
    • Export Citation
  • 7.

    USAID. Nicaragua Health Program Evaluation (April 2008). Available at: www.ghtechproject.com/Attachment.axd?ID=1e06e02d-70bb-4671-b564-258f5ed70d26. Accessed July 5, 2011.

    • Search Google Scholar
    • Export Citation
  • 8.

    Bryce J, Steketee R, 2010. Continuous surveys and quality management in low-income countries: a good idea. Am J Trop Med Hyg 82: 360; author reply 361–362.

    • Search Google Scholar
    • Export Citation
  • 9.

    National Statistical Office (NSO) [Malawi] and ORC Macro, 2005. Malawi Demographic and Health Survey 2004. Calverton, MD: NSO and ORC Macro.

    • Search Google Scholar
    • Export Citation
  • 10.

    Roll Back Malaria. Malaria Indicators Surveys. Available at: http://www.malariasurveys.org/. Accessed June 2, 2011.

 

 

 

Rolling Malaria Indicator Surveys (rMIS): A Potential District-Level Malaria Monitoring and Evaluation (M&E) Tool for Program Managers

View More View Less
  • Malawi-Liverpool-Wellcome Trust Clinical Research Programme, Blantyre, Malawi; Liverpool School of Tropical Medicine, Liverpool, United Kingdom; College of Medicine, Blantyre, Malawi

Novel malaria monitoring and evaluation (M&E) tools are urgently needed to complement the current “gold standard” Malaria Indicator Surveys (MIS). Rapid up scaling of malaria control efforts is resulting in substantial reductions in malaria burden across sub-Saharan Africa. As transmission goes down, timely, accurate, sub-national, and district level burden estimates are needed to guide increasingly targeted control efforts in remaining hotspot areas. To test a novel district level M&E tool, we have conducted a continuous (“rolling”) MIS (rMIS) since May 2010 covering 50 villages in Chikhwawa district in southern Malawi, essentially adapting an existing cross-sectional evaluation tool into a continuous monitoring tool. Here, we report on our experience after completing the first full year of monthly data collection focusing on the methods, operational aspects, and estimated costs of rMIS in a programmatic setting. The potential applicability of this promising M&E approach for district-level program managers and control efforts is discussed.

Background

Increased funding and global political commitment have contributed to a rapid scaling up of malaria control efforts in recent years. As areas achieve high intervention coverage, considerable reductions in the burden of malaria are becoming apparent.13 In this rapidly changing landscape, transmission reduction and even malaria elimination is being reconsidered as a feasible long-term objective for control programs. To achieve this, the Malaria Eradication Research Agenda (malERA) group, among others, recently highlighted the need to develop: 1) tools to monitor short-term malaria trends to be able to respond to these changes in a timely manner, and 2) new sampling strategies at a sub-national level that could be widely incorporated into national monitoring & evaluation (M&E) programs.4

Currently, malaria M&E relies largely on national cross-sectional household surveys such as Malaria Indicator Surveys (MIS), Demographic and Health Surveys (DHS), or the United Nations Children's Fund (UNICEF) Multiple Indicator Cluster Surveys (MICS). Such surveys provide accurate national population-based estimates of malaria intervention coverage and/or malaria burden, but they are logistically demanding, expensive, conducted infrequently at intervals of 2–5 years, do not usually provide district or sub-regional estimates, and their “peak transmission” point estimates are subject to annual and seasonal variability in transmission. More timely, accurate, local level estimates could be useful to guide program managers on how to optimize their control efforts and resources. The use of continuous or intermittent “rolling” cross-sectional surveys at district level has recently been suggested as a potential tool to address this matter,5 and some countries have implemented this concept with continuous DHS and nutritional surveys,6,7 though we are not aware of any comparable examples from sub-Saharan Africa. Concerns have been raised however about their feasibility and sustainability as they require a commitment to data quality and long-term buy-in of health leaders and donors.8 In response to these suggestions, we report on our experience of conducting a year-long rolling MIS (rMIS) in southern Malawi. We focus on the methods, operational aspects, and estimated costs. The observed coverage and burden estimates will be reported separately.

Methods

Since May 2010, a “monthly” rMIS has been conducted in Chikhwawa District as part of a large randomized trial (Clinical Trial ID no: NCT01038063), to monitor the burden of malaria in an area of 50 villages within 400 km2 (population ∼30,000) covering approximately one-fifth of the entire district. Chikhwawa is located in Southern Malawi in the Rift Valley, 500 m above sea level, and has perennial malaria transmission with seasonal peaks between November and April. Malaria control interventions are rapidly being scaled up in this area including the distribution of long lasting insecticide-treated nets and the introduction of annual district-wide indoor residual spraying (IRS) in early 2011 (see Figure 1).

Figure 1.
Figure 1.

Schematic representation of a rolling malaria indicator survey (rMIS).

Citation: The American Society of Tropical Medicine and Hygiene 86, 1; 10.4269/ajtmh.2012.11-0397

To achieve a 10% precision in the prevalence of Hb < 8.0 g/dL at an estimated baseline prevalence around 25% (wet season), and assuming a design effect of 1.35,9 a confidence level of 95%, and a response rate of 80%, a total number of 609 children were required in the rainy season (i.e., ∼100 children to be surveyed each month). To assess differences between the rainy season from November to April, and dry season from May to October, every village (cluster-approximating census enumeration areas) was sampled once in each 6-month period (twice a year, once during the rainy season and once during the dry season). Households were randomly selected using a two-step sampling strategy. During each season, all 50 villages were randomly assigned to one of the 6 months (8 or 9 villages per month). Within each village, households were randomly selected from a list of households, with a probability proportional to village population size. This sampling frame was based on a census exercise of the entire study area conducted in November 2008. An adapted version of the standard national MIS evaluation package was applied (http://rbm.who.int/merg.html#MIS) to collect malaria impact and coverage indicators (see Table 1). If the person primarily responsible for health care decision-making was not at home during the first visit, up to two revisits were made to find them to collect information for that given household. Following informed consent, a household and individual under-five questionnaire was completed. A finger-prick sample was taken to determine anemia and asexual parasitemia (rapid diagnostic test [RDT] and malaria smear) in children under five. Children found to be parasitemic (by RDT) or anemic were treated as per national malaria treatment guidelines. If appropriate, children were referred to the nearest hospital for further assessment. All data was collected using personal digital assistants (PDAs) by study field workers.

Table 1

Summary of main impact and coverage indicators collected during the rolling malaria indicator survey (rMIS)

IndicatorAge groupDescription
Impact indicators
Prevalence of moderate to severe anemia6–59 monthsDefined as the proportion of under-fives with hemoglobin values < 8.0 g/dL
Prevalence of parasitemia6–59 monthsCollected using two methods: 1) proportion of under-fives with positive rapid diagnostic test, and 2) proportion of under-fives with asexual parasites on thick film
Coverage indicators
Coverage of bed netsProportion of households with at least one insecticide-treated net (ITN)
6–59 monthsProportion of under-fives sleeping under an ITN
Coverage of indoor residual spraying (IRS)Proportion of households protected by IRS
Coverage of prompt and effective treatment6–59 monthsProportion of under-fives with malaria/fever receiving treatment within 24 hours from onset of fever
Other indicators
Nutritional indicators6–59 monthsMid upper arm circumference (MUAC)

Results

Two survey teams (supervised by a senior field worker), each consisting of a nurse and a field worker sampled an average of 100 (range: 60–124) households and ∼60 under-fives each month during both the rainy and dry months, using a vehicle or motorbike for transport to the villages. Each monthly survey was scheduled to last for 5 working days (1 week). The average number of households sampled and completed varied by month, as it was sometimes difficult to find respondents and/or household members (under-fives), e.g., during the harvesting season. The average time to complete a household was 30 minutes, which mainly depended on the total number of under-fives sampled. The use of PDAs with embedded decision rules and skip patterns were similar to those used in the national MIS and guided decision-making (e.g., need for antimalarial treatment). The study teams were based at the district hospital, where the malaria smears were processed by a research microscopist and data were uploaded daily into a central database by a data officer. A quality control system ensured that staff personnel involved in the rMIS received regular standardized training (every 3 to 4 months) throughout the study period to conduct finger pricks for anemia and malaria parasitemia among under-fives. Complete results including the merged microscopy data were usually available for analysis within 1 or 2 weeks from completion of the survey.

The 2010 Malawi national MIS sampled ∼3,600 under-fives and was estimated to cost ∼$650,000 (courtesy of the Malawi National Malaria Control Program [NMCP]). Based on the standardized Roll Back Malaria budget template (available at: http://www.rollbackmalaria.org/toolbox/tool_CostingBudgetingMEplan.html), the field cost of our rMIS covering a full season (6 months), was calculated to range between $30,000 and $40,000 if the same Roll Back Malaria budget template assumptions were used.

Discussion

Our experience suggests that an rMIS provides a feasible, affordable, district-level M&E approach to assess “high priority districts,” with a small permanent team. There are several reasons that make the rMIS an appealing tool. First, by using a methodology that is consistent with current gold standard national MIS, the rMIS builds on the available and growing in-country expertise to conduct and analyze MIS surveys: 12 countries have completed MIS surveys over the past 5 years.10 As the rMIS uses the same standardized methodologies (e.g., same impact and coverage indicators, sampling strategies, questionnaires, standard operational procedures, etc.), results obtained from these surveys are directly comparable to their correspondent region-level (higher level) estimates obtained from national surveys. In addition, given that interventions are sometimes deployed at district level (and therefore not at regional or national level), estimates from targeted districts will be particularly helpful for monitoring progress of malaria control interventions in focus districts, which national surveys are unable to provide.

Second, rMIS are affordable and capable of obtaining quick results (within 2 weeks from completion of a survey) and therefore have the potential of being integrated into national M&E strategic plans such as those supported by the Global Fund. In addition, rMIS can act as a complementary M&E tool to the national MIS surveys (generally scheduled at ∼2–5 year intervals) by monitoring and informing intervention efforts in selected regions between national MIS surveys. This would give local and national program officers more feedback about impact of targeted control efforts in focus districts.

Third, rMIS are logistically simple, as they only require very small teams, and could potentially be conducted by a health management information system team consisting of 2–3 hospital district-based nurses in combination with 1–2 village health workers. This government set up would allow district level program managers to monitor their local progress. Although survey-specific data management skills would be required at district-level, the growing national expertise in conducting and analyzing national MIS surveys would enable in-country support.

Fourth, given the “rolling” nature of the surveys, a rMIS has the potential to monitor rapid changes in both malaria impact and coverage indicators. For example, district-wide IRS spraying in Chikhwawa started in February 2011 (see Figure 1). A rMIS covering this area could therefore be used to: 1) assess impact of interventions such as IRS by continuously monitoring areas before and after the intervention, and 2) identify IRS coverage gaps. The latter would be particularly helpful to local program managers as this information can be used to mobilize teams and resources rapidly to address deficiencies. The study presented here is based on an rMIS conducted monthly, however, the rMIS approach is completely flexible and can be adapted to different needs or capacities. For example, a rMIS could be conducted every 2 or 3 months to collect a subset of indicators at different times rather than collecting a full set of indicators on every survey. This could reduce the time for completing a household interview. The optimal frequency of surveys will depend on the estimates required, the impact of control campaigns, the available capacity, and the seasonal variation in malaria transmission intensity (if seasonal variations in indicators are to be assessed).

We are aware that our experience does not completely reflect how a rMIS would work in real life. This survey was conducted by a research team rather than the proposed government set up that may face more limited resources.8 In addition, our rMIS only covered ∼20% of the district and overall costs are likely to be higher that currently presented if an rMIS was to cover the whole district (although the logistics and management for a full district rMIS would be similar). Further costing exercises including district-level adaptations of the RBM budget template are required to fully assess the full economic and financial cost of this potential complementary M&E tool. In the short term, such a collaborative set up between NMCP and a research institution may provide a productive practical approach, which could help refine and adapt an rMIS M&E approach to local needs, support long-term high-quality data collection and capacity building, and help prevent overstretching of government staff and resources.5 In addition, such a partnership may allow substantial cost savings. A pilot of this collaborative approach will be explored with the Malawi NMCP.

In conclusion, our experience suggests that rMIS are a promising district-level M&E tool that could play an important role in monitoring short-term control progress. In addition, the rMIS approach could potentially contribute to the new set of novel tools urged by the MalERA group4 to apply “surveillance” as an intervention in areas where transmission is being reduced and cases of infection become increasingly more focalized by rapidly assessing malaria trends over time and place in all age groups. We are currently evaluating the expansion of rMIS to cover all age groups to identify suitable malaria impact indicators in older children and adults. Further evaluation of rMIS in other “high priority” districts in the country is being explored.

ACKNOWLEDGMENTS:

We thank the parents and children that participated in this study and the study team for their contribution to the data collection.

  • 1.

    O'Meara WP, Bejon P, Mwangi TW, Okiro EA, Peshu N, Snow RW, Newton CR, Marsh K, 2008. Effect of a fall in malaria transmission on morbidity and mortality in Kilifi, Kenya. Lancet 372: 15551562.

    • Search Google Scholar
    • Export Citation
  • 2.

    Ceesay SJ, Casals-Pascual C, Erskine J, Anya SE, Duah NO, Fulford AJ, Sesay SS, Abubakar I, Dunyo S, Sey O, Palmer A, Fofana M, Corrah T, Bojang KA, Whittle HC, Greenwood BM, Conway DJ, 2008. Changes in malaria indices between 1999 and 2007 in The Gambia: a retrospective analysis. Lancet 372: 15451554.

    • Search Google Scholar
    • Export Citation
  • 3.

    Kleinschmidt I, Schwabe C, Benavente L, Torrez M, Ridl FC, Segura JL, Ehmer P, Nchama GN, 2009. Marked increase in child survival after four years of intensive malaria control. Am J Trop Med Hyg 80: 882888.

    • Search Google Scholar
    • Export Citation
  • 4.

    malERA Consultative Group on Monitoring Evaluation and Surveillance, 2011. A research agenda for malaria eradication: monitoring, evaluation, and surveillance. PLoS Med 8: e1000400.

    • Search Google Scholar
    • Export Citation
  • 5.

    Rowe AK, 2009. Potential of integrated continuous surveys and quality management to support monitoring, evaluation, and the scale-up of health interventions in developing countries. Am J Trop Med Hyg 80: 971979.

    • Search Google Scholar
    • Export Citation
  • 6.

    USAID. External Evaluation of the Peru Continuous Survey Experiment (December 2007). Available at: http://www.ghtechproject.com/resources.aspx. Accessed July 5, 2011.

    • Search Google Scholar
    • Export Citation
  • 7.

    USAID. Nicaragua Health Program Evaluation (April 2008). Available at: www.ghtechproject.com/Attachment.axd?ID=1e06e02d-70bb-4671-b564-258f5ed70d26. Accessed July 5, 2011.

    • Search Google Scholar
    • Export Citation
  • 8.

    Bryce J, Steketee R, 2010. Continuous surveys and quality management in low-income countries: a good idea. Am J Trop Med Hyg 82: 360; author reply 361–362.

    • Search Google Scholar
    • Export Citation
  • 9.

    National Statistical Office (NSO) [Malawi] and ORC Macro, 2005. Malawi Demographic and Health Survey 2004. Calverton, MD: NSO and ORC Macro.

    • Search Google Scholar
    • Export Citation
  • 10.

    Roll Back Malaria. Malaria Indicators Surveys. Available at: http://www.malariasurveys.org/. Accessed June 2, 2011.

Author Notes

*Address correspondence to Arantxa Roca-Feltrer, Malawi-Liverpool-Wellcome Trust Clinical Research Programme, Blantyre, Malawi. E-mail: arfeltrer@mlw.medcol.mw

Financial support: We are grateful to the London School of Tropical Medicine and Hygiene, the Bill and Melinda Gates Foundation, and the Medicines for Malaria Venture for supporting this work.

Authors' addresses: Arantxa Roca-Feltrer, Malawi-Liverpool-Wellcome Trust Clinical Research Programme, Blantyre, Malawi, E-mail: arfeltrer@mlw.medcol.mw. David G. Lalloo and Dianne J. Terlouw, Liverpool School of Tropical Medicine, Liverpool, UK, E-mails: dlalloo@liv.ac.uk and dterlouw@liv.ac.uk. Kamija Phiri, Liverpool School of Tropical Medicine, Liverpool, UK and College of Medicine, Blantyre, Malawi, E-mail: kphiri@medcol.mw.

Save