• 1.

    World Health Organization, 2016. World Malaria Report 2016. Geneva, Switzerland: World Health Organization.

  • 2.

    Nabarro DN, Tayler EM, 1998. The “roll back malaria” campaign. Science 280: 20672068.

  • 3.

    World Health Organization and the United Nations Children’s Fund, 2015. Achieving the Malaria MDG Target: Reversing the Incidence of Malaria 2000–2015. Geneva, Switzerland: World Health Organization, 5.

  • 4.

    World Health Organization on behalf of the Roll Back Malaria Partnership Secretariat, 2015. Action and Investment to Defeat Malaria 2016–2030. Geneva, Switzerland: World Health Organization.

  • 5.

    Bradley DJ, 1991. Morbidity and mortality at Pare-Taveta, Kenya and Tanzania, 1954–66: the effects of a period of malaria control. Feachem RG, Jamison DT eds. Disease and Mortality in Sub-Saharan. Africa. The World Bank. Oxford, United Kingdom: Oxford University Press.

  • 6.

    Molineaux L, Gramiccia G, 1980. The Garki Project: Research on the Epidemiology and Control of Malaria in the Sudan Savanna of West Africa. Geneva, Switzerland: World Health Organization.

  • 7.

    Payne D, Grab B, Fontain RE, Hempel JHG, 1976. Impact of control measures on malaria transmission and general mortality. Bull World Health Organ 54: 369377.

    • Search Google Scholar
    • Export Citation
  • 8.

    Molineaux L, 1997. Malaria and mortality: some epidemiological considerations. Ann Trop Med Parasitol 91: 811825.

  • 9.

    Watt C, Dye C, 2000. Indicators to Measure the Impact of Malaria Control. Geneva, Switzerland: World Health Organization (WHO∕CDS∕RBM∕2000.22). Available at: http://whqlibdoc.who.int/hq/2000/WHO_CDS_RBM_2000.22.pdf. Accessed May 23, 2017.

  • 10.

    World Health Organization and Roll Back Malaria, 2000. Framework for Monitoring Progress and Evaluating Outcomes and Impact. Geneva, Switzerland: World Health Organization (WHO∕CDS∕RBM∕2000.25). Available at: http://www.rbm.who.int/cmc_upload/0/000/012/168/m_e_en.pdf. Accessed December 3, 2006.

  • 11.

    Remme JHF, Binka F, Nabarro D, 2001. Toward a framework and indicators for monitoring Roll Back Malaria. Am J Trop Med Hyg 64 (Suppl 1, 2): 7684.

    • Search Google Scholar
    • Export Citation
  • 12.

    de Savigny D, Binka F, 2004. Monitoring future impact on malaria burden in sub-Saharan Africa. Am J Trop Med Hyg 71: 224231.

  • 13.

    World Health Organization, United Nations Children’s Fund, 2005. World Malaria Report 2005. Geneva, Switzerland: World Health Organization, 1, 271–272 (WHO∕HTM∕MAL∕2005.1102).

  • 14.

    Bhattarai A, et al.., 2007. Impact of artemisinin-based combination therapy and insecticide-treated nets on malaria burden in Zanzibar. PLoS Med 4: e309.

    • Search Google Scholar
    • Export Citation
  • 15.

    Rowe AK, et al.., 2007. Evaluating the impact of malaria control efforts on mortality in sub-Saharan Africa. Trop Med Int Health 12: 15241539.

    • Search Google Scholar
    • Export Citation
  • 16.

    Kleinschmidt I, Schwabe C, Benavente L, Torrez M, Ridl FC, Segura JL, Ehmer P, Nchama GN, 2009. Marked increase in child survival after four years of intensive malaria control. Am J Trop Med Hyg 80: 882888.

    • Search Google Scholar
    • Export Citation
  • 17.

    Chizema-Kawesha E, Miller JM, Steketee RW, Mukonka VM, Mukuka C, Mohamed AD, Miti SK, Campbell CC, 2010. Scaling up malaria control in Zambia: progress and impact 2005–2008. Am J Trop Med Hyg 83: 480488.

    • Search Google Scholar
    • Export Citation
  • 18.

    Smithson P, Florey L, Salgado SR, Hershey CL, Masanja H, Bhattarai A, Mwita A, McElroy PD; Tanzania Malaria Impact Evaluation Research Group, 2015. Impact of malaria control on mortality and anemia among Tanzanian children less than five years of age, 1999–2010. PLoS One 10: e0141112.

    • Search Google Scholar
    • Export Citation
  • 19.

    Y, et al.., 2017. Framework for evaluating the health impact of the scale-up of malaria control interventions on all-cause child mortality in sub-Saharan Africa. Am J Trop Med Hyg 97 (Suppl 3): 919.

    • Search Google Scholar
    • Export Citation
  • 20.

    Hershey CL, et al.., 2017. Implementing impact evaluations of malaria control interventions: process, lessons learned, and recommendations. Am J Trop Med Hyg 97 (Suppl 3): 2031.

    • Search Google Scholar
    • Export Citation
  • 21.

    Thomson MC, Ukawuba I, Hershey CL, Bennett A, Ceccato P, Lyon B, Dinku T, 2017. Using rainfall and temperature data in the evaluation of national malaria control programs in Africa. Am J Trop Med Hyg 97 (Suppl 3): 3245.

    • Search Google Scholar
    • Export Citation
  • 22.

    Ashton RA, Bennett A, Yukich J, Bhattarai A, Keating J, Eisele TP, 2017. Methodological considerations for use of routine health information system data to evaluate malaria program impact in an era of declining malaria transmission. Am J Trop Med Hyg 97 (Suppl 3): 4657.

    • Search Google Scholar
    • Export Citation
  • 23.

    Ng M, Colson KE, Fullman N, Dwyer-Lindgren L, Achoki T, Schneider MT, Mulenga P, Hangoma P, Masiye F, Gakidou E, 2017. Assessing the contribution of malaria vector control and other maternal and child health interventions in reducing all-cause under-five mortality in Zambia. Am J Trop Med Hyg 97 (Suppl 3): 5864.

    • Search Google Scholar
    • Export Citation
  • 24.

    Florey LS, Bennett A, Hershey CL, Bhattarai A, Nielsen C, Ali D, Luhanga M, Taylor C, Eisele T, Y, 2017. Impact of insecticide-treated net ownership on all-cause under-five mortality in Malawi, 2006–2010. Am J Trop Med Hyg 97 (Suppl 3): 6575.

    • Search Google Scholar
    • Export Citation
  • 25.

    Hershey CL, et al.., 2017. Malaria control interventions contributed to declines in malaria parasitemia, severe anemia, and all-cause mortality in children less than five years of age in Malawi, 2000–2010. Am J Trop Med Hyg 97 (Suppl 3): 7688.

    • Search Google Scholar
    • Export Citation
  • 26.

    Thwing J, et al.., 2017. Declines in malaria burden and all-cause mortality following increases in control interventions in Senegal, 2005–2010. Am J Trop Med Hyg 97 (Suppl 3): 8998.

    • Search Google Scholar
    • Export Citation
  • 27.

    Eckert E, et al.., 2017. Impact evaluation of malaria control interventions on morbidity and all-cause mortality in Rwanda, 2000–2010. Am J Trop Med Hyg 97 (Suppl 3): 99110.

    • Search Google Scholar
    • Export Citation
  • 28.

    Rowe AK, 2009. The potential of integrated continuous surveys and quality management to support monitoring, evaluation, and the scale-up of health interventions in developing countries. Am J Trop Med Hyg 80: 971979.

    • Search Google Scholar
    • Export Citation
Past two years Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 315 138 12
PDF Downloads 128 64 4
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 

 

 

Assessing the Health Impact of Malaria Control Interventions in the MDG/Sustainable Development Goal Era: A New Generation of Impact Evaluations

Alexander K. RoweMalaria Branch, Division of Parasitic Diseases and Malaria, Centers for Disease Control and Prevention, Atlanta, Georgia

Search for other papers by Alexander K. Rowe in
Current site
Google Scholar
PubMed
Close

INTRODUCTION

Malaria remains a major cause of preventable death. The World Health Organization estimated that malaria killed 429,000 people (uncertainty range: 235,000–639,000) in 2015, with most deaths occurring in sub-Saharan Africa and among children under 5 years old.1 Nearly two decades ago, the Roll Back Malaria (RBM) partnership was created to help combat this plague. The initial targets were to halve malaria mortality from 2000 to 2010 and again from 2010 to 2015.2 In addition, the Millennium Development Goals (MDGs) included a target of reducing malaria incidence by 2015. With global incidence falling by an estimated 37% compared with 2000, this target has been achieved.3 Current targets, which correspond to the era of the Sustainable Development Goals (SDGs), are to reduce malaria incidence and mortality by 90% from 2015 to 2030.4 More than US$ 20 billion has been dedicated to the fight over the past decade, with an estimated US$ 2.9 billion spent on malaria control and elimination in 2015.1

Ambitious RBM and MDGs, along with the substantial funding that these goals have attracted, made it important to conduct impact evaluations to show how much malaria control efforts had actually reduced malaria mortality. However, impact evaluations are challenging because measuring malaria mortality directly (i.e., counting all malaria deaths) is not feasible in most of sub-Saharan Africa. The earliest impact evaluations in Africa were sub-national malaria control and research projects, such as those conducted from the 1950s to 1970s in Kenya, Tanzania, and Nigeria.57 The relatively small scale of these projects allowed for intensive, high-quality, community-based monitoring of child mortality and entomological measures of malaria transmission, which are generally not feasible for national-level programs.

Around the time of the establishment of RBM, a number of reports considered practical methods for evaluating the impact of national-level malaria control programs,813 which led to one of the first impact evaluations in the MDG era, in Zanzibar.14 In 2007, RBM’s Monitoring and Evaluation Reference Group published a framework for evaluating the impact of malaria control efforts on mortality in sub-Saharan Africa.15 The framework informed additional impact evaluations.1618

This supplement to the American Journal of Tropical Medicine and Hygiene includes nine contributions on the evaluation of the impact of malaria control interventions. The purposes of this supplement are to highlight the successes of malaria control efforts and present new methods for evaluating the impact of large-scale malaria control programs.

SUPPLEMENT OVERVIEW

Yé et al. have updated RBM’s decade-old framework for evaluating the impact of malaria control efforts.19 This expanded framework includes new features, such as examining high-risk subpopulations most likely to demonstrate improvement from intervention scale-up, using a national platform framework, and analyzing complete birth histories from national household surveys to characterize the association between exposure to malaria control interventions and all-cause child mortality (ACCM).

The article by Hershey et al.20 is a companion piece to the new RBM framework.19 The authors have reflected on their experiences with planning and conducting impact evaluations in nearly a dozen countries, and they offer practical lessons and recommendations that should influence and benefit future impact evaluations, such as ensuring country ownership, engaging and coordinating stakeholders and partners, tailoring evaluations to country contexts, and using a standard methodology and informative dissemination products.

Thomson et al. explore ways to use data on rainfall and temperature (two potentially important confounders of the association between malaria intervention scale-up and reductions in malaria burden) in the evaluation of national malaria control programs in Africa.21 They use a new quality-controlled dataset on rainfall and temperature and other climate information to assess the likely effect of climate on impact evaluations in 10 countries.

Ashton et al. present methods for using data from routine health management information systems (HMIS) for impact evaluations, with a focus on improving internal validity, reducing bias of impact estimates, and the identification of quasi-experimental designs that are particularly well-suited for HMIS data, such as interrupted time series and dose-response analyses.22 This article should be the go-to reference for analysts using HMIS data.

Ng et al. used multiple data sources from Zambia to produce district-level estimates of intervention coverage, contextual factors, and ACCM, and they then fit a model that estimated mortality trends as interventions were scaled-up.23

Two articles examined malaria control efforts in Malawi. Florey et al. evaluated the impact of the national scale-up of insecticide-treated nets (ITNs) on ACCM with two complementary methods: a retrospective cohort analysis of individual children and a district-level ecologic analysis—both of which controlled for numerous confounders.24 Hershey et al. evaluated the combined effect of all major malaria control interventions on a broader array of impact indicators: malaria parasitemia, severe anemia, as well as ACCM.25

In Senegal26 and Rwanda,27 the evaluations take advantage of widely varying malaria transmission levels to show greater programmatic impact in areas with higher malaria transmission. Such comparisons strengthen the causal link between malaria intervention scale-up and reductions in malaria morbidity and mortality. This approach is especially important because these countries experienced increasing coverage of non-malaria interventions (e.g., vaccines, breastfeeding, and improved water sources) and general economic improvements that likely contributed to ACCM reductions. The Rwanda evaluation is also notable for the use of decomposition modeling, which estimated the contribution of various factors (e.g., bed net ownership) to reductions in ACCM.

These studies convey three important messages. First, they demonstrate an increased sophistication in the use of national cross-sectional household surveys and of multiple data sources. Taken together, the articles represent a conceptual and practical guide for planning and executing a new generation of impact evaluations. This guidance might be of value for evaluating the impact of programs for other health conditions in low-resource settings. Second, the studies illustrate the substantial time and effort required to conduct a successful evaluation, including the engagement of many experts to accommodate an array of analytic approaches. It is heartening to see how these evaluations can help build in-country capacity on programmatic monitoring and evaluation.20 Third, the studies demonstrate the tremendous impact of malaria interventions.

CHALLENGES AND THE WAY FORWARD

Although this supplement includes some very fine work, the validity of future impact evaluations could be improved further, and to do so requires tackling several important methodologic challenges. First, in the context of large historical declines in ACCM in most countries, the counterfactual (what would have occurred in the absence of malaria control) needs to be better estimated. Second, programs and advocates would surely appreciate a clearer quantification of programmatic impact on malaria-related deaths, which would also require standard methods for calculating the uncertainty of impact estimates. Third, limitations exist for the high-quality household surveys that are used in all evaluations, such as recall bias and failure to collect the full range of data needed for a given analysis (e.g., data on ITNs discarded before the survey visit, retrospective ITN use by individual children, or immunization coverage among children who died).24 Fourth, better data are needed on the uptake and impact of case-management, which is a major pillar of all malaria control programs. Fifth, we need to find better ways of weaving practical impact evaluations into routine programs. Future impact evaluations should be anticipated, with needed data collected and analyzed prospectively, so the time to produce a formal impact evaluation is reduced. Continuous household and health facility surveys, for example, might allow for future impact evaluations to be done more quickly, updated regularly (e.g., every 1–2 years), and reflect the effect of scaling-up malaria case-management.28 Finally, despite improvements in many countries, the fact that routine HMIS data often still have poor (or unknown) validity and limited representativeness remains an important obstacle, especially for monitoring future progress toward SDGs. Weak HMIS data was a theme throughout the entire supplement. Timely, valid, and representative HMIS data are, of course, also useful for managing programs and essential for malaria elimination efforts. With improved HMIS data and optimal intervention coverage, and eventually the achievement of malaria elimination, we will finally know the precise proportion of malaria deaths prevented: 100%.

REFERENCES

  • 1.

    World Health Organization, 2016. World Malaria Report 2016. Geneva, Switzerland: World Health Organization.

  • 2.

    Nabarro DN, Tayler EM, 1998. The “roll back malaria” campaign. Science 280: 20672068.

  • 3.

    World Health Organization and the United Nations Children’s Fund, 2015. Achieving the Malaria MDG Target: Reversing the Incidence of Malaria 2000–2015. Geneva, Switzerland: World Health Organization, 5.

  • 4.

    World Health Organization on behalf of the Roll Back Malaria Partnership Secretariat, 2015. Action and Investment to Defeat Malaria 2016–2030. Geneva, Switzerland: World Health Organization.

  • 5.

    Bradley DJ, 1991. Morbidity and mortality at Pare-Taveta, Kenya and Tanzania, 1954–66: the effects of a period of malaria control. Feachem RG, Jamison DT eds. Disease and Mortality in Sub-Saharan. Africa. The World Bank. Oxford, United Kingdom: Oxford University Press.

  • 6.

    Molineaux L, Gramiccia G, 1980. The Garki Project: Research on the Epidemiology and Control of Malaria in the Sudan Savanna of West Africa. Geneva, Switzerland: World Health Organization.

  • 7.

    Payne D, Grab B, Fontain RE, Hempel JHG, 1976. Impact of control measures on malaria transmission and general mortality. Bull World Health Organ 54: 369377.

    • Search Google Scholar
    • Export Citation
  • 8.

    Molineaux L, 1997. Malaria and mortality: some epidemiological considerations. Ann Trop Med Parasitol 91: 811825.

  • 9.

    Watt C, Dye C, 2000. Indicators to Measure the Impact of Malaria Control. Geneva, Switzerland: World Health Organization (WHO∕CDS∕RBM∕2000.22). Available at: http://whqlibdoc.who.int/hq/2000/WHO_CDS_RBM_2000.22.pdf. Accessed May 23, 2017.

  • 10.

    World Health Organization and Roll Back Malaria, 2000. Framework for Monitoring Progress and Evaluating Outcomes and Impact. Geneva, Switzerland: World Health Organization (WHO∕CDS∕RBM∕2000.25). Available at: http://www.rbm.who.int/cmc_upload/0/000/012/168/m_e_en.pdf. Accessed December 3, 2006.

  • 11.

    Remme JHF, Binka F, Nabarro D, 2001. Toward a framework and indicators for monitoring Roll Back Malaria. Am J Trop Med Hyg 64 (Suppl 1, 2): 7684.

    • Search Google Scholar
    • Export Citation
  • 12.

    de Savigny D, Binka F, 2004. Monitoring future impact on malaria burden in sub-Saharan Africa. Am J Trop Med Hyg 71: 224231.

  • 13.

    World Health Organization, United Nations Children’s Fund, 2005. World Malaria Report 2005. Geneva, Switzerland: World Health Organization, 1, 271–272 (WHO∕HTM∕MAL∕2005.1102).

  • 14.

    Bhattarai A, et al.., 2007. Impact of artemisinin-based combination therapy and insecticide-treated nets on malaria burden in Zanzibar. PLoS Med 4: e309.

    • Search Google Scholar
    • Export Citation
  • 15.

    Rowe AK, et al.., 2007. Evaluating the impact of malaria control efforts on mortality in sub-Saharan Africa. Trop Med Int Health 12: 15241539.

    • Search Google Scholar
    • Export Citation
  • 16.

    Kleinschmidt I, Schwabe C, Benavente L, Torrez M, Ridl FC, Segura JL, Ehmer P, Nchama GN, 2009. Marked increase in child survival after four years of intensive malaria control. Am J Trop Med Hyg 80: 882888.

    • Search Google Scholar
    • Export Citation
  • 17.

    Chizema-Kawesha E, Miller JM, Steketee RW, Mukonka VM, Mukuka C, Mohamed AD, Miti SK, Campbell CC, 2010. Scaling up malaria control in Zambia: progress and impact 2005–2008. Am J Trop Med Hyg 83: 480488.

    • Search Google Scholar
    • Export Citation
  • 18.

    Smithson P, Florey L, Salgado SR, Hershey CL, Masanja H, Bhattarai A, Mwita A, McElroy PD; Tanzania Malaria Impact Evaluation Research Group, 2015. Impact of malaria control on mortality and anemia among Tanzanian children less than five years of age, 1999–2010. PLoS One 10: e0141112.

    • Search Google Scholar
    • Export Citation
  • 19.

    Y, et al.., 2017. Framework for evaluating the health impact of the scale-up of malaria control interventions on all-cause child mortality in sub-Saharan Africa. Am J Trop Med Hyg 97 (Suppl 3): 919.

    • Search Google Scholar
    • Export Citation
  • 20.

    Hershey CL, et al.., 2017. Implementing impact evaluations of malaria control interventions: process, lessons learned, and recommendations. Am J Trop Med Hyg 97 (Suppl 3): 2031.

    • Search Google Scholar
    • Export Citation
  • 21.

    Thomson MC, Ukawuba I, Hershey CL, Bennett A, Ceccato P, Lyon B, Dinku T, 2017. Using rainfall and temperature data in the evaluation of national malaria control programs in Africa. Am J Trop Med Hyg 97 (Suppl 3): 3245.

    • Search Google Scholar
    • Export Citation
  • 22.

    Ashton RA, Bennett A, Yukich J, Bhattarai A, Keating J, Eisele TP, 2017. Methodological considerations for use of routine health information system data to evaluate malaria program impact in an era of declining malaria transmission. Am J Trop Med Hyg 97 (Suppl 3): 4657.

    • Search Google Scholar
    • Export Citation
  • 23.

    Ng M, Colson KE, Fullman N, Dwyer-Lindgren L, Achoki T, Schneider MT, Mulenga P, Hangoma P, Masiye F, Gakidou E, 2017. Assessing the contribution of malaria vector control and other maternal and child health interventions in reducing all-cause under-five mortality in Zambia. Am J Trop Med Hyg 97 (Suppl 3): 5864.

    • Search Google Scholar
    • Export Citation
  • 24.

    Florey LS, Bennett A, Hershey CL, Bhattarai A, Nielsen C, Ali D, Luhanga M, Taylor C, Eisele T, Y, 2017. Impact of insecticide-treated net ownership on all-cause under-five mortality in Malawi, 2006–2010. Am J Trop Med Hyg 97 (Suppl 3): 6575.

    • Search Google Scholar
    • Export Citation
  • 25.

    Hershey CL, et al.., 2017. Malaria control interventions contributed to declines in malaria parasitemia, severe anemia, and all-cause mortality in children less than five years of age in Malawi, 2000–2010. Am J Trop Med Hyg 97 (Suppl 3): 7688.

    • Search Google Scholar
    • Export Citation
  • 26.

    Thwing J, et al.., 2017. Declines in malaria burden and all-cause mortality following increases in control interventions in Senegal, 2005–2010. Am J Trop Med Hyg 97 (Suppl 3): 8998.

    • Search Google Scholar
    • Export Citation
  • 27.

    Eckert E, et al.., 2017. Impact evaluation of malaria control interventions on morbidity and all-cause mortality in Rwanda, 2000–2010. Am J Trop Med Hyg 97 (Suppl 3): 99110.

    • Search Google Scholar
    • Export Citation
  • 28.

    Rowe AK, 2009. The potential of integrated continuous surveys and quality management to support monitoring, evaluation, and the scale-up of health interventions in developing countries. Am J Trop Med Hyg 80: 971979.

    • Search Google Scholar
    • Export Citation

Author Notes

Address correspondence to Alexander K. Rowe, Malaria Branch, Division of Parasitic Diseases and Malaria, Centers for Disease Control and Prevention, 1600 Clifton Road, Atlanta, GA 30329. E-mail: axr9@cdc.gov

Author’s address: Alexander K. Rowe, Malaria Branch, Division of Parasitic Diseases and Malaria, Centers for Disease Control and Prevention, Atlanta, GA, E-mail: axr9@cdc.gov.

Save