Correcting for Verbal Autopsy Misclassification Bias in Cause-Specific Mortality Estimates

ABSTRACT. Verbal autopsies (VAs) are extensively used to determine cause of death (COD) in many low- and middle-income countries. However, COD determination from VA can be inaccurate. Computer coded verbal autopsy (CCVA) algorithms used for this task are imperfect and misclassify COD for a large proportion of deaths. If not accounted for, this misclassification leads to biased estimates of cause-specific mortality fractions (CSMFs), a critical piece in health-policy making. Recent work has demonstrated that the knowledge of the CCVA misclassification rates can be used to calibrate raw VA-based CSMF estimates to account for the misclassification bias. In this manuscript, we review the current practices and issues with raw COD predictions from CCVA algorithms and provide a complete primer on how to use the VA calibration approach with the calibratedVA software to correct for verbal autopsy misclassification bias in cause-specific mortality estimates. We use calibratedVA to obtain CSMFs for child (1–59 months) and neonatal deaths using VA data from the Countrywide Mortality Surveillance for Action project in Mozambique.


S2. Model comparison using the WAIC
We have two collected sources and types of data which need to be modeled differently. The COMSA data correspond to a marginal multinomial likelihood for the CCVA predicted COD, and the CHAMPS data correspond to a conditional multinomial likelihood of the CCVA predicted COD given the MITS COD. If we just used the COMSA data, which has no other COD information (like MITS-COD), to evaluate the WAIC, the CCVA algorithms would not be penalized for their high degree of misclassification, as evidenced in the CHAMPS cases, and the best WAIC would be obtained from the uncalibrated model for the CSMF which is the best fit to the COMSA VA data if we ignore misclassification.
However, the uncalibrated CSMF assumes that the models have perfect sensitivity and the CHAMPS data testifies for or against this assumption. Hence, the misclassification rates are critical to understand the relationship between the VA-COD and the MITS-COD, and the WAIC calculation needs to include the CHAMPS data as well. Thus in our case, the future data for WAIC are twofold -VA-COD predictions for nationally representative community deaths (COMSA data), which are modeled using both the true CSMF and misclassification rates, and VA-COD predictions for the CHAMPS data with MITS-COD provided, which are modeled using just the misclassification rates. If the CHAMPS data demonstrate large misclassification rates, the WAIC will be large thereby rightfully penalize the uncalibrated CSMF for the wrong assumption of perfect sensitivity.
To estimate the WAIC from the calibrated models, we use the MCMC draws of the CSMF and the misclassification rates to obtain the posterior distribution of the log-likelihood for every death in both sources of data.
To estimate the WAIC for the uncalibrated models, we first obtain draws from the posterior distribution of the CSMF by assuming perfect sensitivity. The posterior mean of this distribution is nearly exactly that of the uncalibrated CSMF estimate. The prefect sensitivity assumption for the uncalibrated model, ideally translates to all posterior draws of the misclassification matrix being the identity matrix. This however produces a WAIC of infinity immediately ruling out the uncalibrated model. To make the WAIC of the uncalibrated model more competitive to that of the calibrated model, we compute the former assuming model sensitivities of 95%, under the assumption that with sufficiently high (but not perfect) sensitivity, one would still be willing to accept the uncalibrated CSMF estimates.

S3. Implementation of CCVA algorithms
To run InSilicoVA, the data variables and values are converted from the VA questionnaire format to the openVA R package format. For example, the variable Id10019, which is sex in the VA questionnaire, is recoded i019a and i019b and response values for i019a are "Y" if male and "N" if female, and vice-versa for i019b. Missing, don't know, and refusal are coded "." This mapping process, for each of the hundreds of VA questionnaire variables, must be done manually and checked separately, for each data source. For InSilicoVA inputs we assume the prevalence of HIV and malaria to be high in Mozambique. To run EAVA, the VA questionnaire variables and responses remain in the same format as they were collected. The EAVA CCVA creates a database indicating a diagnosis is either present or absent, based on reported symptoms. Causes are then assigned for each age group based on a hierarchy.  [1] A. Datta, J. Fiksel, A. Amouzou, and S. L. Zeger, "Regularized Bayesian transfer learning for population-level etiological distributions," Biostatistics, Feb. 2020.