Intermittent Preventive Treatment of Malaria in Pregnancy Coverage Estimates from Population-based Surveys: Reliability of Women's Recall Among Women with ANC Cards

Demographic and Health Surveys (DHS) and Malaria Indicator Surveys (MIS) are large, nationally representative household surveys conducted every 3 to 5 years.¹ These surveys rely on the participants’ ability to accurately recall receiving health services, including insecticide-treated net (ITN) use, intermittent preventive treatment during pregnancy (IPTp), and the vaccination history of children. The World Health Organization (WHO) recommends IPTp with sulphadoxine-pyrimethamine for pregnant women residing in most sub-Saharan African countries to protect them from malaria during pregnancy.² IPTp is administered monthly during the second and third trimesters of pregnancy during routine antenatal care (ANC) visits; at least three doses of IPTp are recommended during pregnancy.² Key indicators for assessing optimal IPTp coverage are the percentage of women receiving at least one dose (IPTp1+) and the percentage of women receiving at least three doses (IPTp3+).

IPTp coverage is a self-reported indicator in national household surveys; therefore, many of the studies of IPTp uptake cite recall bias as a potential limitation.^3–5 The risk of recall bias may lead to inaccurate estimates of coverage. To eliminate recall bias for vaccination coverage rates, it is standard practice in DHS surveys to verify a parent’s recall of their children’s vaccination history using the children’s medical card.⁶ However, this practice is not standardly used to assess interventions that protect pregnant women from malaria. Recall bias is a real concern for IPTp estimates because the data used to calculate IPTp coverage come from questions asked of women who experienced a live birth within the 2 years before completing the survey. This means it could have been as long as 2.5 years since some women received IPTp during the second trimester. Respondents’ health records, such as an antenatal care (ANC) card kept by women during their pregnancies, may be used as a second verification source to increase confidence in self-reported statistics; this practice is similar to the process used for self-reported immunization data in household surveys.⁷

This analysis assessed the validity of self-reported IPTp coverage in nationally representative household surveys by comparing interviewee responses based on recall with the interviewee’s written ANC card record. Several questions and a process of cross-referencing ANC cards were added to the 2017 Malawi MIS, the 2017–2018 Benin DHS, the 2011–2012 Tanzania HIV and Malaria Indicator Survey (THMIS), and the 2014 Ghana DHS.¹ Surveyors recorded the results of each woman’s recall and what was recorded on her ANC card using the survey questionnaire tool.

We compared the self-reported results to the ANC card data by testing specificity, sensitivity, kappa statistics, and area under the curve (AUC) of the receiver-operating characteristic (ROC) curve for each of these surveys assuming the IPTp frequency reported on the ANC card was the gold standard. The sensitivity represents the likelihood of a woman correctly reporting that she received IPTp during her most recent pregnancy. The specificity represents the likelihood of a woman correctly reporting that she did not receive IPTp. The kappa test was used to assess the overall agreement between the recall and the ANC card data. These statistics were calculated separately for each incidence of IPTp (one dose, IPTp1; two doses, IPTp2; and three or more doses, IPTp3+). For the purpose of this analysis, only women who had experienced live birth within the past 2 years before the survey, attended an ANC appointment at least once, received at least one dose of IPTp, and had an ANC card that was observed by the surveyor were included. Of the eligible women participating in the Benin survey, 69.2% presented an ANC card (N = 1,847); 46.8% (N = 873) participating in the Ghana DHS, 69.9% (N = 658) participating in the Malawi survey, and 47.3% (N = 997) participating in the Tanzania survey presented an ANC card. Table 1 shows the IPTp coverage according to background and sociodemographic characteristics of the women.

In the Benin 2017–2018 MIS and the Ghana 2014 DHS, the proportion of women with IPTp3+ was higher among the more educated and wealthier women. The proportion of women with IPTp3+ was also higher among women in urban settings than among those in rural settings who participated in the Ghana 2014 DHS. The proportions of women with IPTp3+ appeared to be more equitable among women who participated in the Malawi 2017 MIS and the Tanzania 2011–2012 MIS.

For the women who participated in the Malawi 2017 MIS, IPTp3+ coverage was higher among women who presented an ANC card compared with women who did not present an ANC card (49% versus 36%); however, IPTp1 coverage was lower among women who presented an ANC card (15% versus 22%). For the women who participated in the Tanzania 2011–2012 THMIS, IPTp3+ coverage was lower among women who presented an ANC card (4% versus 9%). For the women who participated in the Ghana and Benin surveys, no significant difference was apparent in IPTp coverage based on the ANC card availability.

The proportions of women who received IPTp1, IPTp2, and IPTp3+ according to self-report, ANC card data, both self-report and ANC card data, and the survey are shown in Table 2; however, these data are restricted to the subpopulation who presented ANC cards at the time of the interview. The IPTp3+ coverage was highest for women who participated in the Ghana and Malawi surveys and ranged from 47% to 55% and from 45% to 49%, respectively, depending on method of reporting information. The proportion of women with IPTp3+ in Benin ranged from 23% to 28%. However, the IPTp3+ coverage was lowest in Tanzania (range, 2–5%).

To test the validity of women’s recall of IPTp doses received, we calculated the sensitivity and specificity of IPTp1, IPTp2, and IPTp3+ by comparing self-report with ANC card data. Additionally, we calculated a kappa statistic to test the agreement between the two measures and the AUC of the ROC curve to test the individual validity of the self-report measure compared with that of the ANC card data (Table 3).

Across surveys, the sensitivity of women's ability to correctly report IPTp coverage ranged from 79% to 92% for IPTp1, from 79% to 96% for IPTp2, and from 73% to 100% for IPTp3+. The proportion of women who correctly reported that they did not receive IPTp (specificity) ranged from 88% to 99.6% for IPTp1, from 87% to 96% for IPTp2, and from 83% to 98% for IPTp3+. The agreement between self-reported IPTp coverage and coverage indicated on ANC cards, as measured by kappa scores, ranged from 0.69 to 0.85 for IPTp1, from 0.64 to 0.92 for IPTp2, and from 0.52 to 0.94 for IPTp3+. Although interpreting kappa scores can be somewhat problematic, the scores found during this analysis represent a moderate to high level of agreement on most scales.^8,9 The AUC of the ROC curve values were > 0.8 for all reported IPTp measurements from all surveys and > 0.9 for the majority, indicating an excellent to outstanding level of individual validity for the self-reported coverage estimates.^10,11

This analysis demonstrates that self-reported recall of IPTp is an adequate method of reporting IPTp with sulphadoxine-pyrimethamine doses received by women who had experienced a live birth within the 2 years before the survey using the standard protocols for household surveys such as the DHS and MIS. The analysis suggests that additional validation through external sources, such as ANC cards and a review of medical records, is not necessary. It has been found that using local context (i.e., a local word for iron folate) or a visual aid (photograph of a drug) can improve recall.^12,13 For example, because of the potentially long period of recall (up to 2.5 years), the accuracy of IPTp recall may be improved by providing visuals of sulphadoxine-pyrimethamine tablets during household survey interviews to help women recall the number of IPTp doses they received.

This study only includes data from four countries; therefore, the results presented here may not be representative of all countries implementing IPTp with different rates of coverage. Another limitation of this study was that the dataset was restricted to include only women who were able to present their ANC card to the interviewer; therefore, only 47% of the women in Ghana and Tanzania, 69% in Benin, and 70% in Malawi were eligible for this study. This group of women may be different than the group women who were unable or unwilling to present their ANC card at the time of the survey. Additional studies that use health facility records of IPTp administration to verify self-reported data could be considered to address this bias. Furthermore, the surveys used for this study covered a wide date range and a wide range of IPTp coverage. The Tanzania survey was conducted in 2011 to 2012, but the others were conducted more recently. The IPTp3+ coverage indicated by this early Tanzania survey was low; therefore, these data could have affected the sensitivity and agreement statistics. Finally, the potential for interviewer bias should be considered because it is possible that during the interviews, surveyors reconciled self-reported responses based on the data observed on the ANC card.

Despite these limitations, after validation using the ANC card data as the gold standard, the results of this study demonstrate that the self-reported IPTp coverage estimates indicated by these national household surveys are accurate. The results of this study support the validity of recall of IPTp indicated by large household surveys, thus suggesting that national malaria control programs can reliably assess their IPTp intervention coverage and make programmatic decisions based on survey data. Additionally, this analysis demonstrates that including additional questions and ANC card data verification may not be necessary and is not likely to significantly change IPTp coverage estimates.