• View in gallery

    Comparing reported and surveyed coverage by year in 60 districts of Cameroon (2004–2011).

  • View in gallery

    Comparing reported and surveyed coverage by year in 38 districts of Uganda (2003–2011).

  • 1.

    Amazigo UV, Obono M, Dadzie KY, Remme J, Jiya J, Ndyomugyenyi R, Roungou JB, Noma M, Seketeli A, 2002. Monitoring community-directed treatment programmes for sustainability: lessons from the African Programme for Onchocerciasis Control (APOC). Ann Trop Med Parasitol 96 (Suppl 1): S75S92.

    • Search Google Scholar
    • Export Citation
  • 2.

    Katabarwa MN, Habomugisha P, Richards FO Jr., 2002. Implementing community-directed treatment with ivermectin for the control of onchocerciasis in Uganda (1997–2000): an evaluation. Ann Trop Med Parasitol 96: 6173.

    • Search Google Scholar
    • Export Citation
  • 3.

    Worrell C, Mathieu E, 2012. Drug coverage surveys for neglected tropical diseases: 10 years of field experience. Am J Trop Med Hyg 87: 216222.

    • Search Google Scholar
    • Export Citation
  • 4.

    Amazigo UV, Brieger WR, Katabarwa M, Akogun O, Ntep M, Boatin B, N’Doyo J, Noma M, Seketeli A, 2002. The challenges of community-directed treatment with ivermectin (CDTI) within the African Programme for Onchocerciasis Control (APOC). Ann Trop Med Parasitol 96 (Suppl 1): S41S58.

    • Search Google Scholar
    • Export Citation
  • 5.

    Yaya G, Kobangué L, Kémata B, Gallé D, Grésenguet G, 2014. Elimination or control of the onchocerciasis in Africa? Case of Gami village in Central African Republic [in French]. Bull Soc Pathol Exot 107: 188193.

    • Search Google Scholar
    • Export Citation
  • 6.

    Mas J 2006. Reduction in the prevalence and intensity of infection in Onchocerca volvulus microfilariae according to ethnicity and community after 8 years of ivermectin treatment on the island of Bioko, Equatorial Guinea. Trop Med Int Health 11: 10821091.

    • Search Google Scholar
    • Export Citation
  • 7.

    WHO, 1996. Community Directed Treatment with Ivermectin: Report of a Multi-Country Study. Geneva, Switzerland: World Health Organization, 80.

    • Search Google Scholar
    • Export Citation
  • 8.

    Boatin BA, Richards FO Jr., 2006. Control of onchocerciasis. Adv Parasitol 61: 349394.

  • 9.

    Katabarwa M, Habomugisha P, Eyamba A, Agunyo S, Mentou C, 2010. Monitoring ivermectin distributors involved in integrated health care services through community-directed interventions–a comparison of Cameroon and Uganda experiences over a period of three years (2004–2006). Trop Med Int Health 15: 216223.

    • Search Google Scholar
    • Export Citation
  • 10.

    Lawrence J, Sodahlon YK, Ogoussan KT, Hopkins AD, 2015. Growth, challenges, and solutions over 25 years of Mectizan and the impact on onchocerciasis control. PLoS Negl Trop Dis 9: e0003507.

    • Search Google Scholar
    • Export Citation
  • 11.

    Katabarwa MN, Habomugisha P, Eyamba A, Byamukama E, Nwane P, Arinaitwe A, Musigire J, Tushemereirwe R, Khainza A, 2016. Community-directed interventions are practical and effective in low-resource communities: experience of ivermectin treatment for onchocerciasis control in Cameroon and Uganda, 2004–2010. Int Health 8: 116123.

    • Search Google Scholar
    • Export Citation
  • 12.

    Richards FO Jr. 2001. The Carter Center’s assistance to river blindness control programs: establishing treatment objectives and goals for monitoring ivermectin delivery systems on two continents. Am J Trop Med Hyg 65: 108114.

    • Search Google Scholar
    • Export Citation
  • 13.

    Katabarwa NM, Mutabazi D, Richards FO Jr., 2000. Controlling onchocerciasis by community-directed, ivermectin-treatment programmes in Uganda: why do some communities succeed and others fail? Ann Trop Med Parasitol 94: 343352.

    • Search Google Scholar
    • Export Citation
  • 14.

    Salant P, Dillman DA, 1994. How to Conduct Your Own Survey. New York, NY: John Wiley & Sons.

  • 15.

    Dancey CP, Reidy J, Rowe R, 2012. Statistics for the Health Sciences: A Non-Mathematical Introduction. London, United Kingdom: SAGE.

  • 16.

    Katabarwa MN, Mutabazi D, 1998. The selection and validation of indicators for monitoring progress towards self-sustainment in community-directed, ivermectin-treatment programmes for onchocerciasis control in Uganda. Ann Trop Med Parasitol 92: 859868.

    • Search Google Scholar
    • Export Citation
  • 17.

    Katabarwa MN, Richards FO Jr., 2001. Community-directed health (CDH) workers enhance the performance and sustainability of CDH programmes: experience from ivermectin distribution in Uganda. Ann Trop Med Parasitol 95: 275286.

    • Search Google Scholar
    • Export Citation
  • 18.

    Baker MC, Krotki K, Sankara DP, Trofimovich L, Zoerhoff KL, Courtney L, Chowdhury D, Linehan M, 2013. Measuring treatment coverage for neglected tropical disease control programs: analysis of a survey design. Am J Epidemiol 178: 268275.

    • Search Google Scholar
    • Export Citation
  • 19.

    Guyer B, Atangana S, 1977. A programme of multiple-antigen childhood immunization in Yaounde, Cameroon: first-year evaluation, 1975–1976. Bull World Health Organ 55: 633642.

    • Search Google Scholar
    • Export Citation
  • 20.

    Cromwell EA, King JD, McPherson S, Jip FN, Patterson AE, Mosher AW, Evans DS, Emerson PM, 2013. Monitoring of mass distribution interventions for trachoma in Plateau State, Nigeria. PLoS Negl Trop Dis 7: e1995.

    • Search Google Scholar
    • Export Citation
  • 21.

    Dierickx S, Gryseels C, Mwesigwa J, O’Neill S, Bannister-Tyrell M, Ronse M, Jaiteh F, Gerrets R, D’Alessandro U, Grietens KP, 2016. Factors associated with non-participation and non-adherence in directly observed mass drug administration for malaria in the Gambia. PLoS One 11: e0148627.

    • Search Google Scholar
    • Export Citation
  • 22.

    Jin J, Sklar GE, Min Sen Oh V, Chuen Li S, 2008. Factors affecting therapeutic compliance: a review from the patient’s perspective. Ther Clin Risk Manag 4: 269286.

    • Search Google Scholar
    • Export Citation
  • 23.

    Whitworth JA, Maude GH, Downham MD, 1996. Clinical and parasitological responses after up to 6.5 years of ivermectin treatment for onchocerciasis. Trop Med Int Health 1: 786793.

    • Search Google Scholar
    • Export Citation
  • 24.

    African Programme for Onchocerciasis Control, WHO, 2009. Final Communiqué: Fifteenth Session of the Joint Action Forum, Tunis, Tunisia. Ouagadougou, Burkina Faso: WHO/APOC, 18.

    • Search Google Scholar
    • Export Citation
  • 25.

    WHO, 2016. Guidelines for Stopping Mass Drug Administration and Verifying Elimination of Human Onchocerciasis: Criteria and Procedures. Geneva, Switzerland: World Health Organization.

    • Search Google Scholar
    • Export Citation
  • 26.

    WHO/APOC, 2012. The WHO African Programme for Onchocerciasis Control: Progress Report 2012. The World Health Organization Year 2012 Progress Report. September 1, 2011–August 31, 2012. Ouagadougou, Burkina Faso: African Programme for Onchocerciasis Control.

    • Search Google Scholar
    • Export Citation
  • 27.

    WHO/APOC, 2010. Conceptual and Operational Framework of Onchocerciasis Elimination with Ivermectin Treatment. Ouagadougou, Burkina Faso: WHO/African Programme for Onchocerciaisis Control.

    • Search Google Scholar
    • Export Citation
  • 28.

    Burmeister H, Kayembe Kalambayi P, Koulischer G, Oladepo O, Philippon BA, Tarimo E, 2005. Report of the External Evaluation (APOC, WHO). Geneva, Switzerland: World Health Organization. Available at: https://apps.who.int/iris/bitstream/handle/10665/276000/JAF11.10-eng.pdf.

    • Search Google Scholar
    • Export Citation
  • 29.

    UN, 2017. World Population Prospects: The 2017 Revision, Key Findings and Advance Tables. New York, NY: Department of Economic and Social Affairs, Population Division.

    • Search Google Scholar
    • Export Citation

 

 

 

 

 

Comparison of Reported and Survey-Based Coverage in Onchocerciasis Programs over a Period of 8 Years in Cameroon and Uganda

View More View Less
  • 1 The Carter Center, Atlanta, Georgia;
  • 2 The Carter Center, Kampala, Uganda;
  • 3 Odza Center, Yaounde, Cameroon;
  • 4 Centre de Recherche sur les Filarioses et Autres Maladies Tropicales (CRFilMT), Yaounde, Cameroon;
  • 5 Emory University, Rollins School of Public Health, Atlanta, Georgia

Mass drug administration (MDA) with ivermectin must reach a high treatment coverage (90% of the eligible population) if onchocerciasis is to be eliminated. Questions have been raised as to whether reported treatment figures reaching such high coverage are reliable. Sample surveys are proposed as the method of choice for “validating” reported coverage figures. The purpose of this study was to compare the district-level MDA coverage reported by programs with contemporaneous surveys of randomly selected respondents living in those same districts. Over an 8-year period, 19,219 households were selected using multistage random sampling; 38,433 adult male and female heads of those households were asked about their recent ivermectin MDA treatment experience. District coverage reports were considered “accurate” if they fell within the 95% CIs determined by the corresponding district’s survey. Ninety-eight treatment rounds were evaluated over an 8-year period. Overall, the reported coverage of 96.5% (range: 68–100%) was significantly higher than the 92.5% surveyed coverage (range: 62.1–99.6%, 95% CI: 91.9–93.2%). However, only 20% of districts reported significantly higher coverage than surveys, 68% of district program reports were judged as accurate, and 12% of districts reported significantly lower coverage figures than their corresponding surveys. Eighty-eight percent of districts reported coverage ≥ 90% threshold for success, compared with 97% of surveys that included 90% in their 95% CIs. We conclude that when analyzed statistically at the district level, most surveys verified the reported coverage.

INTRODUCTION

Understanding the reliability of the reported treatment coverage in mass drug administration (MDA) programs is critical for all five Preventive Chemotherapy Neglected Tropical Diseases (PC-NTDs) programs.14 When the expected impact on disease prevalence and transmission is not attained after many years of MDA, poor treatment coverage is the first suspect, even if the official reports of the treatment coverage have been satisfactory.5,6 Conducting routine treatment coverage surveys has become an accepted approach to “validating” program reports, or the first step in diagnosing the reason for treatment failure before undertaking more expensive studies of potential epidemiologic and entomological factors that may be threatening progress.

Community-directed treatment with ivermectin (CDTI) for onchocerciasis control in Africa was launched by the African Program for Onchocerciasis Control (APOC) in 1996.7,8 For many years, the Carter Center–assisted programs in Cameroon and Uganda conducted CDTI monitoring surveys annually that included a question on coverage.9,10 The purpose of this study was to analyze these survey data by comparing them statistically with their corresponding district coverage reports. Our hypothesis, based on other reported experiences comparing surveyed with reported coverage, was that the treatment coverage officially reported by Cameroonian and Ugandan districts would be statistically significantly higher than surveyed coverage.3

MATERIALS AND METHODS

In this study, we compared 8 years of district-level treatment coverage reports with their corresponding district’s monitoring coverage surveys conducted within 2 months of the MDA exercise. The programs’ coverage goal was to reach ≥ 90% of the treatment eligible population (e.g., the population aged > 5 years).

In Cameroon, monitoring took place from 2004 to 2011 in West and North regions (although surveys were not conducted in the North in 2011 because of funding constraints). In Uganda, surveys were conducted from 2003 to 2011, except in 2008 because of time constraints. Surveys were conducted in 38 distinct districts. Cameroon surveys included 10–17 households per community, whereas Uganda surveys included 5–19 households. Uganda selected five households per community from 2003 to 2005 because of financial constraints.

District-level MDA coverage reports by the Uganda and Cameroon health system.

The official MDA coverage figures used in this study were compiled by the districts’ health services. Starting at the community level, the community-selected volunteers (known as community-directed distributors [CDDs]) kept a written household register in which each page was dedicated to a particular household. The household page would list each member’s name, age, gender, and treatment outcome, including the number of tablets of ivermectin ingested by that individual. Several years of treatment rounds could be recorded in different columns on a single household page. The treatment dose was based on height using a dosing pole, and then, treatment was directly observed by the CDD and immediately recorded in the register.9,11 At the end of the treatment round, the CDDs with their community supervisor would prepare the “roll-up” community summary form from the register (or from multiple registers if the community supervisor had several independently operating CDDs). The summary would provide the total community population, eligible population, number of persons treated, and ivermectin usage. Populations of most communities ranged from approximately 200 to 500 people. The completed community form was collected by a salaried ministry of health (MOH) worker based at the health facility responsible for providing care and outreach to that community. The MOH worker in turn completed the next level of summary reporting forms for those communities under his/her jurisdiction. The “roll-up” process thus continued for subdistrict, district, region (in Cameroon), and national levels. Each of these levels offered opportunities for inconsistencies or errors to occur.

We collected district-level treatment coverage figures for the years 2003–2011 for the districts and years that corresponded to the Carter Center–supported random monitoring surveys.9,11 District-reported coverage was calculated as a percentage: (the number of persons treated divided by the number of eligible people to be treated) × 100.12

Treatment coverage surveys.

All Carter Center–assisted districts were eligible to be sampled at random each year. A multistage random sampling process was used as described in previous published reports.2,9,11,13 Surveys were conducted a month after the submission of district reports, which was no more than 2 months after MDA completion. Communities were randomly sampled as follows: The district health services provided a list of all communities targeted for MDA with their corresponding population size. Communities then were randomly selected by the program statistician, and the number of persons to be interviewed in each was determined using the table for sample sizes.14,15 The selected community was visited and a list of households was generated from the community’s household registers. The first household to be interviewed was selected using a random number table and additional households were selected systematically with a computed interval per community.15 Between 5 and 19 households were selected per community. Interview teams were supervised by the program statistician or social scientist and two or three independent university staff, and district and central MOH program staff.

The two adult heads of household (male and female) were interviewed in every selected household. If household heads were not present, interviewers would return at least once to complete the interview. In the case where the team learned that one or both of the individuals selected for interview would be absent for more than 3 days, the interviewer would pick another randomly selected household to interview.

The questions used in the interviews had been previously field-tested to assure they were understood and elicited appropriate responses. When required, the questionnaire was translated into the local language, but in most cases, interviewers were comfortable translating English (Uganda) or French (Cameroon) to local languages during interviews when necessary.

The questions focused on seven key CDTI issues: 1) “Did you and your neighbors decide on the method of treatment (house-to-house or community center)?” 2) “Did you and your neighbors select the CDD?” 3) “Did CDDs demand monetary incentives from you?” 4) “Did you attend health education meetings?” 5) “Were you treated (with ivermectin)?” 6) “Will you be available to take the treatment next year?” and 7) “If not, why?”16,17 This study focused only on responses to the question on most recent MDA treatment (question 5). Surveyed coverage was reported as a percentage and calculated as (the number of heads of household who said they had been treated divided by the number of heads of households surveyed) × 100.

Data management and analysis.

The monitoring summary data sets were combined into a single data set and analyzed for the acceptance of treatment by adult heads of households by district and year. The data were entered in Epi Info 6.04d and analyzed in Epi Info 6.04d and Epi Info 7.0 (CDC, Atlanta, GA), and Open Epi (www.openepi.com). 95% CIs were calculated for the survey data using the following formula.
CI=100×(Estimate±1.96×6×Estimate×(1Estimate)Number surveyed).

A highly conservative design effect of six was applied to the variance estimate to account for the relatively small community sizes, the number of sampling stages, and the tendency for the MDA coverage to be relatively homogenous within communities.18 CIs were adjusted to a maximum limit of 100% and a minimum limit of 0%. CIs were not calculated around the reported treatments because these data were not the result of a sample.

Coverage surveys and their 95% CI results were matched with their corresponding reported coverage by district and year. The key outcome was to determine if the CIs from the coverage survey included the district’s reported coverage; if so, the reported coverage was considered to have been “verified” as being correct. If not, the reported coverage was considered to be either low (if it was below the lower 95% CI of the survey) or high (if it was above the upper 95% CI of the survey). The hypothesis of the study that district treatments are overreported would be supported if > 50% of coverage surveys had an upper 95% CI that excluded the coverage reported by the district.

The second key outcome was a comparison of the percentage of districts achieving the program’s coverage goal of 90% of the eligible population. To examine this outcome, we compared the number of surveys having ≥ 90% coverage within their 95% CIs with the corresponding district report achieving 90% in reported coverage. The χ2 test was performed to compare the proportion of districts including/attaining 90% and above by the measurement method.15 We also performed the same analysis on the mean (“point estimate”) district survey coverage result, without regard to the CI. All were tested with the Chisquarestatistic=(OrcErc)2Erc, where Orc is the observed frequency (percent treatment coverage) at level r for reported and level c for surveyed, and Erc is the expected frequency (percent treatment coverage) at level r for reported and level c for surveyed. The hypothesis of the study that district treatments are overreported would be supported if a statistically significantly lower percentage of coverage surveys achieved ≥ 90% within their CIs compared with district reports of ≥ 90% coverage.

Ethical approval.

The protocol for the monitoring surveys (that included both treatment coverage surveys and other CDTI indicators) was approved by the ministries of health. Emory University’s Institutional Review Board classified the study as “non-research” (a periodic program performance assessment). Oral consent was obtained first from community leaders and then from a meeting of community members held before interviews were conducted. Last, interviewees provided oral consent before the household interview. In all these meetings, leaders, community members, and participants were informed about the purpose of the study, that participation was voluntary, and that there would be no repercussions for giving particular answers or for refusing to participate.

RESULTS

A total of 38,433 persons in 19,219 households were interviewed in 98 surveys (Table 1). Sixty surveys took place in Cameroon in 23 health districts, in which 18,737 persons were interviewed. In Uganda, there were 38 surveys in 15 districts with 19,696 interviews. As per study design, the gender ratio was 1:1.

Table 1

Sample of districts, health areas, communities, households, and interviewees selected every year from 2003 to 2011 in Cameroon and Uganda

CountryYearNo. of districtsHealth areas/subcountiesCommunities selectedHouseholds per selected communityHouseholds interviewedInterviewedTotal sample size (n)
MaleFemale
Cameroon (60 surveys)2003
2004867147131,9121,9111,9123,823
2005858174111,8841,8831,8843,767
2006868191101,9611,9601,9613,921
2007831178111,9661,9651,9663,931
2008662117347347347694
200991131175335325331,065
201081240135135135131,026
2011552112255255255510
Subtotal602588031049,3719,3669,37118,737
Uganda (38 surveys)200363813456756756751,350
200453311055505505501,100
200553411056006006001,200
200642590191,7231,7231,7233,446
2007525115151,7251,7251,7253,450
2008
2009530105191,9501,9501,9503,900
2010521100151,5001,5001,5003,000
201131774151,1251,1251,1252,250
Subtotal38223838989,8489,8489,84819,696
Total984811,64120219,21919,21419,21938,433

Reported versus surveyed treatment coverage.

Aggregate results.

The overall mean coverage generated by either assessment method was excellent. Reported coverage was 96.5% (range: 68–100%) and surveyed coverage was 92.5% (range: 75–99.6%, 95% CI: 91.9–93.2%). Cameroon reported a mean of 95.0% (range: 68–100) and a surveyed coverage of 92.3% (range: 75–99%, 95% CI: 91.3–93.2%) (Figure 1, Table 2). Uganda reported a mean of 97.9% (range: 85–100%) and a surveyed coverage of 92.8% (range: 79–100%, 95% CI: 91.9–93.7%) (Figure 2, Table 3). Note that the upper 95% CI for all surveys combined was below the overall mean reported treatment figures by the programs, supporting the hypothesis that the reported treatment coverage is usually higher than the surveyed coverage.

Figure 1.
Figure 1.

Comparing reported and surveyed coverage by year in 60 districts of Cameroon (2004–2011).

Citation: The American Journal of Tropical Medicine and Hygiene 100, 5; 10.4269/ajtmh.18-0680

Table 2

Annually reported and surveyed treatment coverage by district in Cameroon (2004–2011) (n = 60)

YearReportsSurveys
DistrictPopulationTreatment goalTreatedCoverage (%)Sample sizeTreatedCoverage (95% CI)Statistically significant from reportsEstimate above (↗) or below (↘) reported coverage
2004Bamendjou28,60024,02421,62290.048446295.5% (90.9–100%)*
Batcham60,67650,96849,43997.047645896.2% (92%–100%)NS
Dschang72,91061,24461,244100.043142197.7% (94.2–100%)NS
Kekem27,32522,95322,49498.049047095.9% (91.6–100%)NS
Lagdo56,98747,86946,91298.050546391.7% (85.8–97.6%)*
Penka Michel63,09953,00346,11387.048845994.1% (88.9–99.2%)*
Poli40,14533,72231,69894.049939378.8% (70–87.5%)*
Tcholliré61,72851,85251,852100.045040389.6% (82.6–96.5%)*
2005Bamendjou29,56024,83022,19889.443842697.3% (93.5–100%)*
Bandja36,40030,57630,27099.047645695.8% (91.4–100%)NS
Batcham61,20051,40849,86697.047246398.1% (95.1–100%)NS
Kekem28,50023,94022,02592.044343698.4% (95.6–100%)*
Lagdo59,45049,93843,94588.048945292.4% (86.7–98.2%)NS
Penka Michel64,44054,13042,43878.447644693.7% (88.3–99%)*
Poli41,23034,63330,82489.048441184.9% (77.1–92.7%)NS
Tcholliré62,30052,33250,76297.048944791.4% (85.3–97.5%)NS
2006Bafang67,54056,73455,59998.049349199.6% (98.5–100%)*
Bandjoun88,96174,72771,29095.448046797.3% (93.7–100%)NS
Dschang165,501139,021137,07598.648247799% (96.7–100%)NS
Kekem32,84027,58627,31099.048147799.2% (97.2–100%)NS
Mbouda129,880109,099105,82697.049848898% (95–100%)NS
Poli64,90654,52149,12390.149744489.3% (82.7–96%)NS
Tcholliré90,84676,31171,04593.149144089.6% (83–96.2%)NS
Touboro137,425115,437102,73989.049946392.8% (87.2–98.3%)NS
2007Banja37,84031,78631,46899.048942887.5% (80.4–94.7%)*
Foumbot68,31957,38854,51995.048445894.6% (89.7–99.5%)NS
Massagam32,53027,32527,325100.048843589.1% (82.4–95.9%)*
Mbouda139,632117,291116,70499.549446293.5% (88.2–98.8%)*
Ngong34,36128,86328,863100.048842587.1% (79.8–94.4%)*
Rey Bouba80,43067,56146,41568.749639078.6% (69.8–87.5%)*
Santchou24,72720,77120,35598.049842885.9% (78.5–93.4%)*
Touboro149,061125,211120,20396.049442686.2% (78.8–93.7%)*
2008Baham43,16036,25436,254100.01179278.6% (60.4–96.8%)*
Bandja38,62032,44130,81995.012011394.2% (83.9–100%)NS
Batcham77,01664,69361,45995.010810395.4% (85.7–100%)NS
Dschang170,473143,197134,60594.012011394.2% (83.9–100%)NS
Foumban160,089134,475123,71792.01099587.2% (71.8–100%)NS
Kouoptamo45,77038,44738,06299.012011999.2% (95.2–100%)NS
2009Baham46,66039,19439,194100.01209075% (56–94%)*
Bandjoun98,64082,85882,858100.01149684.2% (67.8–100%)NS
Foumbot69,31058,22058,220100.011911395% (85.3–100%)NS
Kekem34,80029,23228,35597.011810084.7% (68.9–100%)NS
Mbouda141,820119,129114,36496.012011495% (85.4–100%)NS
Ngong35,60029,90429,904100.011510490.4% (77.3–100%)NS
Penka Michel74,00062,16060,91798.012011495% (85.4–100%)NS
Tcholliré110,94593,19480,14786.012010285% (69.4–100%)NS
Touboro156,600131,544130,88699.511911395% (85.3–100%)NS
2010Bandjoun112,40094,41694,416100.01068084% (48.8–100%)NS
Banja39,34033,04633,046100.0252175.5% (55.4–95.5%)*
Dschang171,230159,120159,120100.020019095% (87.6–100%)NS
Kouoptamo52,50044,10041,45494.0524076.9% (48.9–100%)NS
Lagdo64,25053,97043,17680.01049894.2% (83.3–100%)*
Penka Michel74,86062,88262,25499.012312097.6% (90.9–100%)NS
Poli46,83039,33736,58493.020820096.2% (89.8–100%)NS
Tcholliré116,80098,11287,32089.020819794.7% (87.3–100%)NS
2011Baham76,56064,31064,310100.0787393.6% (80.3–100%)NS
Bangourain44,07337,90334,11390.0787393.6% (80.3–100%)NS
Batcham85,60071,90471,904100.010410298.1% (91.6–100%)NS
Galim65,59156,40852,45993.010410399% (94.4–100%)*
Mbouda149,800125,832123,31598.014614498.6% (94–100%)NS

NS = not significant.

* Yes.

Figure 2.
Figure 2.

Comparing reported and surveyed coverage by year in 38 districts of Uganda (2003–2011).

Citation: The American Journal of Tropical Medicine and Hygiene 100, 5; 10.4269/ajtmh.18-0680

Table 3

Annually reported and surveyed treatment coverage by district for Uganda (2003–2011) (n = 38)

YearReportsSurveys
DistrictPopulationTreatment goalTreatedCoverage (%)Sample sizeTreatedCoverage (95% CI)Statistically significant from reportsEstimate above (↗) or below (↘) reported coverage
2003Adjumani166,954140,241139,96199.825022288.8% (79.2–98.4%)*
Kanungu45,31538,06537,18997.725024497.6% (93%–100%)NS
Kisoro20,79517,42516,40394.11008585% (67.9–100%)NS
Mbale175,365147,307147,307100.025024598% (93.7–100%)NS
Nebbi276,604232,347232,11599.925023092% (83.8–100%)NS
Sironko58,33148,99847,96997.925023493.2% (85.6–100%)NS
2004Kanungu46,44839,01637,76896.825024497.6% (93–100%)NS
Kisoro21,31517,90517,86999.81018584.2% (66.7–100%)NS
Mbale179,749150,989150,83899.925024397.2% (93.5–100%)NS
Moyo177,788140,069139,01999.325021284.8% (73.9–95.7%)*
Nebbi283,519238,156237,44199.724923996% (90–100%)NS
2005Kanungu47,60939,99238,87297.225024096% (90–100%)NS
Kasese98,11082,41279,69396.725024096% (90–100%)NS
Kisoro21,84818,35215,48984.420015879% (65.2–92.8%)NS
Mbale184,243154,764150,43197.225024196.4% (90.7–100%)NS
Moyo182,233143,571142,85899.525020983.6% (72.4–94.8%)*
2006Hoima100,60984,51283,66699.075046662.1% (53.6–70.6%)*
Kisoro22,39418,81115,98985.045040790.4% (83.8–97.1%)NS
Manafwa141,950119,238118,64299.574866889.3% (83.9–94.7%)*
Moyo186,789156,903155,64899.275072796.9% (93.9–100%)NS
Nebbi297,872250,212243,45797.374869893.3% (88.9–97.7%)NS
2007Kamwenge35,85630,11928,19193.645043797.1% (93.3–100%)NS
Kanungu48,23140,51439,17796.775073397.7% (95.1–100%)NS
Kasese115,47296,99694,66997.675073297.6% (94.9–100%)NS
Mbale41,83435,14134,61398.575074799.6% (98.8–100%)*
Moyo191,459160,826155,51896.775072596.7% (93.5–99.8%)NS
2008
2009Adjumani174,165146,299143,37398.075066488.5% (82.9–94.1%)*
Bududa142,548119,740116,98697.775073197.5% (94.7–100%)NS
Hoima140,229117,792111,66794.890082591.7% (87.2%–96.1%)NS
Kanungu50,79842,67040,70795.475074599.3% (97.9%–100%)*
Nebbi316,177265,589260,27798.075066388.4% (82.8%–94%)*
2010Amuru151,098126,922123,11597.075065387.1% (81.2–92.9%)*
Bushenyi130,855109,918105,85196.375072596.7% (93.5–99.8%)NS
Kabale27,60423,18721,63493.375070794.3% (90.2–98.3%)NS
Kasese130,585109,691106,40197.075072396.4% (93.1–99.7%)NS
2011Kamwenge42,45735,66435,02298.275072997.2% (94.3–100%)NS
Kasese126,785106,499104,90298.575071895.7% (92.2–99.3%)NS
Moyo165,550139,062130,85794.175068391.1% (86.1–96.1%)NS

NS = not significant.

* Yes.

District-level results.

The arrows in the far right columns of Tables 2 and 3 indicate the direction of a significant difference when the reported district coverage was outside of the 95% CIs of the sample survey conducted in that district. Only 20% of districts reported a coverage figure above the corresponding sample survey’s upper 95% CI (Table 4). Sixty-eight percent of district program reports fell within the 95% CIs of the corresponding year’s surveys, and so were judged as accurate. Twelve percent of districts reported lower coverage than determined by the survey (i.e., were below the lower 95% CI). Therefore, 80% of surveys demonstrated that district reports were either accurate or below surveyed coverage, disproving our hypothesis that district reports would be more frequently higher than surveyed coverage.

Table 4

Summary of comparison of reported coverage with surveyed coverage (N = 98)

CountryNo significant difference between reported and surveyed coverage (%)Survey below reported coverage (%)Survey above reported coverage (%)Total surveys conducted
Cameroon39 (65.0)12 (20.0)9 (15.0)60
Uganda28 (73.7)8 (21.1)2 (5.3)38
Total67 (68.4)20 (20.4)11 (11.2)98

Reported versus surveyed treatment coverage at the district level as related to the 90% treatment coverage goal.

Point estimates.

Overall 89% of districts reported coverage of ≥ 90%, which was considerably higher than the 68.4% of district surveys whose mean reached or exceeded the 90% goal. In Cameroon, 83.3% of districts reported reaching 90% coverage compared with only 66.7% of surveys, and for Uganda, 94.7% of districts reported reaching 90% or more coverage, whereas only 71.1% of surveys had such results. All these differences were highly statistically significant (P < 0.01).

Analysis based on CIs.

In contrast to survey point estimates, when 95% CIs were considered, reported results could not be distinguished from surveys. In this analysis, 96.9% of all survey CIs included 90% coverage, not significantly different from a reported 89% attainment of the 90% treatment goal (P = 0.258). In Cameroon, 96.7% of surveys included 90% in their 95% CIs (versus 83.3% of district reports, P = 0.198) (Table 2). In Uganda, 97.4% of surveys had ≥ 90% coverage in their CIs (compared with 94.7% reported, P = 0.811) (Table 3).

DISCUSSION

The prevailing opinion among assisting institutions and independent researchers is that the treatment coverage reported through MOH “roll-up” reporting systems is inaccurate and overstated–in other words, not to be trusted. More than 30 two-stage cluster household surveys carried out across Africa, the Americas, and Asia by the CDC and collaborators showed that figures from programmatic reports were likely to be higher than those from corresponding surveys.3 A study carried out in Cameroon reported the immunization program overestimating the vaccination coverage by 1–29%.19 In Plateau State, Nigeria, the reported treatment coverage in MDA for trachoma control was 76%, but only 60% of the respondents said they had been treated.20,21

Our results, by contrast, showed that, at the district level, the surveyed treatment coverage largely validated the reported coverage in Cameroon and Uganda onchocerciasis MDA programs when surveys were analyzed statistically. Only 20% of districts reported significantly higher coverage than surveys. Eighty-eight percent of districts reported coverage greater than the 90% threshold for success, compared with 97% of surveys that included 90% in their 95% CIs. These were surprising findings that did not support our original hypothesis (the prevailing opinion) that the reported coverage would be higher than surveyed coverage.

Our findings have several qualifications: 1) We would indeed have concluded that the reported coverage is higher than the surveyed coverage if we only considered country-level data, or if our analysis had been performed without considerations of 95% CIs. 2) We used a highly conservative design effect of six that resulted in expanded 95% CIs. The CI was especially important in the analysis of the 90% coverage threshold; an analysis of only survey means (“point estimates”) showed the reported coverage to be significantly higher than surveyed coverage. This pattern of lower point estimates in the surveyed coverage is most obvious in Figures 1 and 2. Knowing the true degree of clustering or other data about the statistical efficiency of the survey’s design could have tightened CIs and substantially changed the results.15,18 3) Our survey obtained treatment coverage data only from older heads of households, whereas the reported coverage was based on treatment data of eligible persons of all ages. That may have imparted a source of bias toward higher coverages if we assume older persons are more likely than teenagers and young adults to take treatment.22

The literature comparing the reported and surveyed MDA coverage in public health programs such as immunization and PC-NTDs is still insufficient. The challenge of attaining and maintaining believable optimal treatment coverage was a major concern for the APOC and still remains so.4 Studies on treatment compliance have been published, but have not compared reported and surveyed MDA coverage.14,15 A study that attempted to understand the use of reported immunization reports referred to as administrative data by district health services in Burkina Faso in 1999 applied a cluster survey method.16 Tally sheets were used in capturing immunization reports, and the population denominators were unknown. The results showed that administrative coverage estimates did not allow districts with moderate coverage to be distinguished from those with high coverage. Similar studies in Cameroon reported the immunization program overestimating vaccination coverage by 1–29%, whereas in Zimbabwe, the underreported coverage was 4–10%.17,18 Also, more than 30 two-stage cluster household surveys carried out across Africa, the Americas, and Asia by the CDC and collaborators showed that the reported coverage was likely higher than that surveyed.6 In Plateau State, Nigeria, the reported treatment coverage in MDA for trachoma control from village registers was compared with cluster survey results. Both did not attain the desired treatment coverage of 80%, but the reported coverage indicated 76%, whereas 60% of the respondents in the survey were treated.19 Although the study considered surveyed as more authentic and believable, both the reported and surveyed MDA coverage were deficient in their knowledge of the population involved.

Although the present study had a good grasp of the population involved as well as reported and surveyed treatment coverage, it was not a longitudinal coverage study. Communities and districts were surveyed randomly every year, and, therefore, trends over time in the same places were not measured. Districts were sampled on average only twice over the 8-year period and many Ugandan districts split into new districts over the study interval. It is necessary in future to survey selected districts over a period of time to determine if coverage trends are steady, improving, or decreasing. However, the assumption is that it would remain relatively stable and unaffected by migrations or other factors that may result in displacement of communities.

The reality is that reported and surveyed treatment coverage are equally important in understanding the dynamics and trends of community MDA programs, and both can provide important information needed to strengthen program implementation. Program data provide information on the overall scope of the effort in reaching all targeted villages (“geographic coverage”). Regular coverage surveys can verify the program reports and provide other specific information for program improvement.3,16,17 Coverage surveys are more beneficial when carried out immediately after the completion of the MDA exercise to minimize recall bias among respondents.18,23 In this study, surveys were conducted a month after the submission of district reports, which was within 2 months of MDA completion.

One reason for selecting districts randomly every year was to make it clear to all health workers that their respective areas were always potential candidates for validation of the reported treatment coverage and CDTI practice. This motivated workers throughout the program to always pay close attention to all the communities in their charge. In addition, we found that involving MOH workers in coverage surveys was important for their development, mentorship, and ownership of the coverage data, and, ultimately, improved MDA performance. On the other hand, having MOH personnel alone conduct coverage surveys could be a source of bias, given the understandable desire to show excellent (high coverage) outcome. To avoid such bias, the coverage surveys involved independent researchers and partners’ participation at every stage of sampling, training of interviewers, and close supervision in following set procedures and analysis. Interviews were conducted by locally trained interviewers from outside sampled communities and supervised by MOH staff, independent researchers from universities, and the program statistician and social scientist.

In 2009, APOC and its Joint Action Forum called for a programmatic shift in African programs from onchocerciasis disease control to transmission elimination.24 The APOC paradigm for onchocerciasis elimination in Africa included the imperative that all onchocerciasis elimination programs reach 100% of affected communities and achieve at least 80% therapeutic coverage (TC).2527 Therapeutic coverage is calculated as the number of persons treated divided by the total population, including children younger than five years who are ineligible for ivermectin.28 The 2015 United Nations estimated Cameroon’s population of children younger than five years comprised 16% of the total population, and 19% in Uganda.29 In this example, to reach the APOC goal of 80% TC, 95% of the eligible population would need to be treated in Cameroon and 99% in Uganda. These are staggering programmatic requirements that require almost perfect function. By contrast, the 90% coverage of the eligible population would be the equivalent of 76% and 73% of the total population in Cameroon and Uganda, respectively, a more reasonable target. Additional modeling studies are urgently needed to determine a more programmatically achievable total population coverage goal than 80% TC for the interruption of onchocerciasis transmission.

CONCLUSION

In this study conducted with data generated over an 8-year period by Cameroonian and Ugandan onchocerciasis MDA programs, district-level treatment coverage determined by sample surveys compared favorably with district-level MDA coverage reports when the survey 95% CIs were considered. Sixty-eight percent of district coverage reports were considered “accurate” because they fell within the 95% CIs determined by their year’s district sample survey. Similarly, 88% of districts reported reaching the goal of ≥ 90% coverage of the eligible population compared with 97% of surveys having 90% within their 95% CIs. CIs around survey estimates should be considered when validating district treatment reports.

Acknowledgments:

We would like to acknowledge all district health personnel who participated annually in monitoring exercises in Cameroon and Uganda between 2003 and 2011, and the community supervisors and community-directed distributors who treated their communities. This study could not have been successful without the participation of community members who were selected for the face-to-face interviews.

REFERENCES

  • 1.

    Amazigo UV, Obono M, Dadzie KY, Remme J, Jiya J, Ndyomugyenyi R, Roungou JB, Noma M, Seketeli A, 2002. Monitoring community-directed treatment programmes for sustainability: lessons from the African Programme for Onchocerciasis Control (APOC). Ann Trop Med Parasitol 96 (Suppl 1): S75S92.

    • Search Google Scholar
    • Export Citation
  • 2.

    Katabarwa MN, Habomugisha P, Richards FO Jr., 2002. Implementing community-directed treatment with ivermectin for the control of onchocerciasis in Uganda (1997–2000): an evaluation. Ann Trop Med Parasitol 96: 6173.

    • Search Google Scholar
    • Export Citation
  • 3.

    Worrell C, Mathieu E, 2012. Drug coverage surveys for neglected tropical diseases: 10 years of field experience. Am J Trop Med Hyg 87: 216222.

    • Search Google Scholar
    • Export Citation
  • 4.

    Amazigo UV, Brieger WR, Katabarwa M, Akogun O, Ntep M, Boatin B, N’Doyo J, Noma M, Seketeli A, 2002. The challenges of community-directed treatment with ivermectin (CDTI) within the African Programme for Onchocerciasis Control (APOC). Ann Trop Med Parasitol 96 (Suppl 1): S41S58.

    • Search Google Scholar
    • Export Citation
  • 5.

    Yaya G, Kobangué L, Kémata B, Gallé D, Grésenguet G, 2014. Elimination or control of the onchocerciasis in Africa? Case of Gami village in Central African Republic [in French]. Bull Soc Pathol Exot 107: 188193.

    • Search Google Scholar
    • Export Citation
  • 6.

    Mas J 2006. Reduction in the prevalence and intensity of infection in Onchocerca volvulus microfilariae according to ethnicity and community after 8 years of ivermectin treatment on the island of Bioko, Equatorial Guinea. Trop Med Int Health 11: 10821091.

    • Search Google Scholar
    • Export Citation
  • 7.

    WHO, 1996. Community Directed Treatment with Ivermectin: Report of a Multi-Country Study. Geneva, Switzerland: World Health Organization, 80.

    • Search Google Scholar
    • Export Citation
  • 8.

    Boatin BA, Richards FO Jr., 2006. Control of onchocerciasis. Adv Parasitol 61: 349394.

  • 9.

    Katabarwa M, Habomugisha P, Eyamba A, Agunyo S, Mentou C, 2010. Monitoring ivermectin distributors involved in integrated health care services through community-directed interventions–a comparison of Cameroon and Uganda experiences over a period of three years (2004–2006). Trop Med Int Health 15: 216223.

    • Search Google Scholar
    • Export Citation
  • 10.

    Lawrence J, Sodahlon YK, Ogoussan KT, Hopkins AD, 2015. Growth, challenges, and solutions over 25 years of Mectizan and the impact on onchocerciasis control. PLoS Negl Trop Dis 9: e0003507.

    • Search Google Scholar
    • Export Citation
  • 11.

    Katabarwa MN, Habomugisha P, Eyamba A, Byamukama E, Nwane P, Arinaitwe A, Musigire J, Tushemereirwe R, Khainza A, 2016. Community-directed interventions are practical and effective in low-resource communities: experience of ivermectin treatment for onchocerciasis control in Cameroon and Uganda, 2004–2010. Int Health 8: 116123.

    • Search Google Scholar
    • Export Citation
  • 12.

    Richards FO Jr. 2001. The Carter Center’s assistance to river blindness control programs: establishing treatment objectives and goals for monitoring ivermectin delivery systems on two continents. Am J Trop Med Hyg 65: 108114.

    • Search Google Scholar
    • Export Citation
  • 13.

    Katabarwa NM, Mutabazi D, Richards FO Jr., 2000. Controlling onchocerciasis by community-directed, ivermectin-treatment programmes in Uganda: why do some communities succeed and others fail? Ann Trop Med Parasitol 94: 343352.

    • Search Google Scholar
    • Export Citation
  • 14.

    Salant P, Dillman DA, 1994. How to Conduct Your Own Survey. New York, NY: John Wiley & Sons.

  • 15.

    Dancey CP, Reidy J, Rowe R, 2012. Statistics for the Health Sciences: A Non-Mathematical Introduction. London, United Kingdom: SAGE.

  • 16.

    Katabarwa MN, Mutabazi D, 1998. The selection and validation of indicators for monitoring progress towards self-sustainment in community-directed, ivermectin-treatment programmes for onchocerciasis control in Uganda. Ann Trop Med Parasitol 92: 859868.

    • Search Google Scholar
    • Export Citation
  • 17.

    Katabarwa MN, Richards FO Jr., 2001. Community-directed health (CDH) workers enhance the performance and sustainability of CDH programmes: experience from ivermectin distribution in Uganda. Ann Trop Med Parasitol 95: 275286.

    • Search Google Scholar
    • Export Citation
  • 18.

    Baker MC, Krotki K, Sankara DP, Trofimovich L, Zoerhoff KL, Courtney L, Chowdhury D, Linehan M, 2013. Measuring treatment coverage for neglected tropical disease control programs: analysis of a survey design. Am J Epidemiol 178: 268275.

    • Search Google Scholar
    • Export Citation
  • 19.

    Guyer B, Atangana S, 1977. A programme of multiple-antigen childhood immunization in Yaounde, Cameroon: first-year evaluation, 1975–1976. Bull World Health Organ 55: 633642.

    • Search Google Scholar
    • Export Citation
  • 20.

    Cromwell EA, King JD, McPherson S, Jip FN, Patterson AE, Mosher AW, Evans DS, Emerson PM, 2013. Monitoring of mass distribution interventions for trachoma in Plateau State, Nigeria. PLoS Negl Trop Dis 7: e1995.

    • Search Google Scholar
    • Export Citation
  • 21.

    Dierickx S, Gryseels C, Mwesigwa J, O’Neill S, Bannister-Tyrell M, Ronse M, Jaiteh F, Gerrets R, D’Alessandro U, Grietens KP, 2016. Factors associated with non-participation and non-adherence in directly observed mass drug administration for malaria in the Gambia. PLoS One 11: e0148627.

    • Search Google Scholar
    • Export Citation
  • 22.

    Jin J, Sklar GE, Min Sen Oh V, Chuen Li S, 2008. Factors affecting therapeutic compliance: a review from the patient’s perspective. Ther Clin Risk Manag 4: 269286.

    • Search Google Scholar
    • Export Citation
  • 23.

    Whitworth JA, Maude GH, Downham MD, 1996. Clinical and parasitological responses after up to 6.5 years of ivermectin treatment for onchocerciasis. Trop Med Int Health 1: 786793.

    • Search Google Scholar
    • Export Citation
  • 24.

    African Programme for Onchocerciasis Control, WHO, 2009. Final Communiqué: Fifteenth Session of the Joint Action Forum, Tunis, Tunisia. Ouagadougou, Burkina Faso: WHO/APOC, 18.

    • Search Google Scholar
    • Export Citation
  • 25.

    WHO, 2016. Guidelines for Stopping Mass Drug Administration and Verifying Elimination of Human Onchocerciasis: Criteria and Procedures. Geneva, Switzerland: World Health Organization.

    • Search Google Scholar
    • Export Citation
  • 26.

    WHO/APOC, 2012. The WHO African Programme for Onchocerciasis Control: Progress Report 2012. The World Health Organization Year 2012 Progress Report. September 1, 2011–August 31, 2012. Ouagadougou, Burkina Faso: African Programme for Onchocerciasis Control.

    • Search Google Scholar
    • Export Citation
  • 27.

    WHO/APOC, 2010. Conceptual and Operational Framework of Onchocerciasis Elimination with Ivermectin Treatment. Ouagadougou, Burkina Faso: WHO/African Programme for Onchocerciaisis Control.

    • Search Google Scholar
    • Export Citation
  • 28.

    Burmeister H, Kayembe Kalambayi P, Koulischer G, Oladepo O, Philippon BA, Tarimo E, 2005. Report of the External Evaluation (APOC, WHO). Geneva, Switzerland: World Health Organization. Available at: https://apps.who.int/iris/bitstream/handle/10665/276000/JAF11.10-eng.pdf.

    • Search Google Scholar
    • Export Citation
  • 29.

    UN, 2017. World Population Prospects: The 2017 Revision, Key Findings and Advance Tables. New York, NY: Department of Economic and Social Affairs, Population Division.

    • Search Google Scholar
    • Export Citation

Author Notes

Address correspondence to Moses N. Katabarwa, The Carter Center, 453 Freedom Parkway, One Copenhill Ave., Atlanta, GA 30307. E-mail: moses.katabarwa@cartercenter.org

Financial support: The Carter Center and Lions Clubs International Foundation provided financial support for the interviews. Partial support for the MDA program was provided by the African Program for Onchocerciasis Control, and all the medicines were provided by Merck & Co.

Authors’ addresses: Moses N. Katabarwa, Emily Griswold, Lauri Bernard, Paul Weiss, and Frank O. Richards, Health Programs, The Carter Center, Atlanta, GA, E-mails: moses.katabarwa@cartercenter.org, emily.griswold@cartercenter.org, lauri.bernard@cartercenter.org, pweiss2@emory.edu, and frank.richards@cartercenter.org. Peace Habomugisha, Edson Byamukama, and Annet Khainza, Health Programs, The Carter Center, Kampala, Uganda, E-mails: peace.habo@cartercenter.org, edson.byamukama@cartercenter.org, and annet.khainza@cartercenter.org. Albert Eyamba, Clinic, Odza Center, Yaounde, Cameroon, E-mail: eyamba09@hotmail.fr. Philippe Nwane, Health Program, Centre de Recherche sur les Filarioses et Autres Maladies Tropicales (CRFilMT), Yaounde, Cameroon, E-mail: philino07@yahoo.fr.

Save