|
|
||||||||
| ABSTRACT |
|
|
|---|
| INTRODUCTION |
|
|
|---|
Although individual DSS sites may use different methods to capture births and deaths, events are likely to be missed no matter what method is used. Additionally, there may be differential ascertainment of vital events based on age, sex, geographic location, or season. A potentially effective means of assessing both the completeness of a DSS and differential ascertainment of demographic events is the use of capture-recapture methods. Capture-recapture techniques, first developed to estimate the size of wildlife populations, have been adapted to epidemiologic and demographic data to estimate the true number of some event of interest. By using the overlap between two or more independent lists of a particular event, capture-recapture techniques have been used to estimate the numbers of hard-to-reach populations,1214 the number of disease events,15,16 as well as to evaluate completeness of disease surveillance systems.17,18
We used a two-sample capture-recapture method to evaluate the completeness of two rounds of a DSS in western Kenya in identifying births and deaths among children from birth to 5 years of age (059 months old). The purpose of this exercise was both to determine the feasibility of using capture-recapture techniques to evaluate our data collection methodology, and to identify births and deaths among particular subgroups that were likely to be undercounted. We discuss here the strategies used to meet the underlying assumptions of a two-sample capture-recapture, the difficulties of matching individuals between lists in areas where birth and death certificates are not regularly obtained and spelling of names is fluid, and methods to reduce differential ascertainment of demographic events by age and sex.
| MATERIALS AND METHODS |
|
|
|---|
Mapping. All houses in the Asembo study area were mapped using a global positioning system between April and November 1999, and updated in 2001. Details of the mapping methodology have been described elsewhere.20,21 Between January and November 2000, only compounds in Gem, rather than individual houses, were mapped due to the size of the population. In both Asembo and Gem, each house was identified by a unique code comprised of village, compound and household identifiers. Data were entered into ArcInfo (Environmental Systems Research Institute, Redlands, CA), a geographic information system (GIS). The GIS databases were created independently and maintained separately from databases generated by the DSS.
The capture: DSS. The baseline census of the population was conducted in FebruaryApril and JulyNovember 1997 in Asembo and Gem, respectively. Demographic changes in the study population were then monitored through biannual enumerations of residents of the study area. Details of the methods are given elsewhere.22 Briefly, after all households and individuals were enumerated in the baseline survey, households were visited twice per year to record any migrations, births or deaths within the study area since the previous DSS round. New households were also registered at the time of each DSS round. Households were visited by local residents used and trained by the project as enumerators. The same unique household codes used in the GIS database were used to identify the homes of individuals enumerated in the DSS. Three names were recorded for each household resident: Christian name, traditional Juok (dhoLuo) name, and Juok name of the individuals father or husband. Dates of birth and/or death were obtained from the household respondent because neither birth nor death certificates are commonly obtained in this area. Preprinted census forms containing details of all residents alive and present at the time of the previous DSS round were used for identifying and updating individuals at each subsequent DSS round. All new births, as well as any new immigrants who satisfied residency requirements, were registered during each DSS round.
The list of births and deaths in children less than five years old occurring between March 1 and September 30, 2000 in Asembo, and those between January 1 and July 31, 2001 in Gem, served as the capture lists. Births and deaths in the Asembo DSS round were recorded in October 2000, while births and deaths in the Gem DSS round were recorded in August 2001.
The recapture. The recapture list is intended to be an independent register of the event of interest, such that the probability of an event being recorded on the first list is independent of the event being recorded on the second list. Due to the known undercount of neonates in civil registration data in Kenya,23 we chose not to use an existing list of births and deaths from civil registration data, but to create a recapture list by conducting a survey of 5,000 randomly selected households in both Asembo and Gem. To maximize independence between the list generated from the DSS and the recapture list, households registered in the Asembo GIS database served as the sampling frame for Asembo. However, because GIS mapping in Gem was conducted at the compound level, a household sampling frame for Gem was created by merging the GIS compound database with the DSS household database.
Based on the number of births and deaths reported in previous DSS rounds, a sample of 5,000 households in each study area was intended to yield approximately 100 deceased children and 250 new births over a seven-month interval. The recapture surveys were conducted by a different group of trained enumerators from those who had participated in the DSS. All new births and deaths in children less than five years old that occurred during the same interval covered by the DSS round were recorded during the corresponding recapture survey in the same manner as for the DSS rounds in November 2000 in Asembo and AugustSeptember 2001 in Gem.
Matching events between lists. Estimation of the total number of events occurring in a population requires the total number of events identified on each list separately, as well as the number of events identified by both. Matching newborn or deceased children between each DSS round and corresponding recapture survey was first done using a matching program created in SAS (Version 8.1; SAS Institute, Inc., Cary, NC). The program was first set to identify individuals present on both lists via exact matches on a childs three names, household identifiers, sex, dates of birth (and death), and two of the childs mothers names. Increasingly lenient SAS matching programs were also used to allow less strict criteria for matching individual characteristics between lists. However, because of the variability in spelling of names and poor date recall, visual matching of individuals between lists was ultimately undertaken using a preset algorithm. A match between newborns across lists required the following: two of three names of the child; same date of birth within two months; and at least one of the mothers Christian or Juok names. A match between deceased children required the following: two of three names of child; same year of birth; date of death within two months; and at least one of the mothers Christian or Juok names. A match between names did not require exact spelling. Because children in this area frequently spend time outside their compound of residence with extended family members, a match did not require the same household address. However, only 3% of all matches did not have the same village and compound.
Estimation methods.
We assumed that the probability that a vital event was missed by a DSS round was independent of the probability of it being missed by the corresponding recapture survey, and thus PB = PC x PR, where PB is the probability that an event was missed by both systems, PC is the probability that an event was missed by the DSS round, and PR is the probability that it was missed by the recapture survey. We estimated the total number of events missed by each DSS round and corresponding recapture survey using the Peterson maximum likelihood estimator for two independent samples,24,25 calculated as
![]() |
Because dates of birth and death were matched within two months, age at death was calculated as the month of death minus the month of birth. When a child matched between lists had conflicting dates of birth or death that resulted in the child falling into two different age categories, age was based on dates listed on the DSS database. Neonates are defined here as less than one month old.
To minimize the effect of heterogeneity, the total number of missing births and deaths at each DSS round were estimated using the sum of stratified data (by sex for births and by age and sex for deaths), as has been previously recommended.24 Ninety-five percent confidence intervals (CIs) were derived using a goodness-of-fit method described by Regal and Hook.28
The sensitivity of each DSS round in capturing births and deaths was estimated as the number of events identified by the DSS round divided by the total number of events estimated by the overlap between lists.
Ethical clearance. This research was approved by the institutional review boards of the Kenya Medical Research Institute (Nairobi, Kenya) and the Centers for Disease Control and Prevention (Atlanta, GA).
| RESULTS |
|
|
|---|
Overall, the distribution of deaths by age was similar between the Asembo and Gem DSS rounds and recapture surveys (Table 1
). While the distribution of deaths by sex was identical between the Asembo DSS round and recapture survey, the Gem recapture survey identified a higher proportion of male deaths compared with the Gem DSS round. The proportion of male and female births identified by the DSS rounds and recapture surveys was similar in both study sites.
|
We estimated that a total of 1,436 (95% CI = 1,4001,481) and 1,769 (95% CI = 1,6621883) births occurred during the periods covered by the Asembo and Gem DSS rounds, respectively (Table 2
). The sensitivities of the DSS rounds at identifying births were therefore estimated to be 88% and 77%, respectively. Sensitivity of the DSS rounds did not differ significantly according to the sex of newborns.
|
|
We investigated why some child deaths had been missed in the DSS rounds but were found by the recapture survey. Most (92%) of the deceased children missed by the two DSS rounds had never been registered in previous rounds as either new births or new residents, and thus were not present on the pre-printed census forms. There were no clear or consistent reasons why these children were left out of the DSS registration because in many cases, both their parents and siblings had been registered.
| DISCUSSION |
|
|
|---|
Although DSS perform very well in registering vital events over time, they are not 100% accurate and may ascertain events differentially according to demographic characteristics, geographic location, or season. We used a two-sample capture-recapture to evaluate the completeness of two rounds of a DSS in western Kenya and to determine if there was any bias in ascertainment of vital events by age and sex. The sensitivities of the Asembo and Gem DSS rounds in identifying child deaths (059 months old) were estimated to be 71% and 75%, while the sensitivities of the two rounds in identifying recent births were estimated to be 88% and 77%, respectively. In neither location did the DSS perform well in identifying neonatal deaths, and sensitivities generally increased with the age at which a child died. Additionally, female deaths were more likely to go unreported.
Although several studies have attempted to estimate the completeness of vital event registration systems,36,37 we found only one that has used a capture-recapture approach.38 As part of a large-scale ITN trial in Burkina Faso,8 Diallo and others used overlap between two concurrent demographic registration systems, a DSS and village-based vital registration, to evaluate the completeness of each method for ascertainment of births and child deaths. They found that the sensitivity of their DSS for identifying births and child deaths (<5 years old) was 94% and 88%, respectively. Similar to our results, they also reported that the DSS performed less well at identifying deaths among children less than six months old.
The use of a two-sample capture-recapture to determine the sensitivity of a DSS requires several assumptions. First, the population under study must be closed, meaning there can be no entries or losses of individuals during the study period. This assumption is unlikely to hold true in most human populations. However, by assessing DSS rounds that occur over relatively short time intervals within a well-defined area, and having recapture surveys that take place immediately after the conclusion of the DSS round, the likelihood of in- or out-migration among children and family informants is minimized.
Second, the two lists must be independent with respect to the probability of ascertainment. To maximize independence between lists, different interviewers were used to collect the data for the recapture surveys from those used during the DSS rounds. Additionally, households were selected for the recapture survey from a list of households created separately from the DSS database. Although it is impossible to identify any specific effects of dependence when using only two lists,24,25 we found no evidence to suggest negative or positive dependence between being identified by the DSS rounds and their corresponding recapture surveys.
Third, each individual/event in the population must have an equal probability of ascertainment, oftentimes referred to as a lack of heterogeneity. To limit the potential effect of heterogeneity, we report results from a stratified analysis. Because the probability of capturing births was affected by sex, and both sex and age affected the probability of capturing deaths, the estimated total number of newborns was based on the sum of estimates among males and females, while the total number of child deaths was based on the sum of estimates disaggregated by sex as well as age.
Last, it must be possible to match events between lists to use a capture-recapture method. Matching newborns and deceased children between lists proved extremely challenging in our study population. We found that using a computer algorithm to match individuals between lists using unique name and demographic identifiers was not useful due to the inconsistency of spelling and inaccurate recall of dates. We therefore matched children between lists by visual inspection on a case-by-case basis. This method was tedious and time-consuming, and would most likely be impracticable for frequent monitoring of a DSS.
There are several steps that could be taken to facilitate matching of individuals between DSS and recapture surveys. Distribution of study identification cards (IDs) would permit consistent identification of study subjects between the DSS and recapture samples. Study IDs have been used successfully in a number of DSS, but implementing them may depend on national laws, local customs, and linkage of services with the DSS, as well as logistics. An alternative to DSS ID cards could be the use of a name dictionary to standardize the spelling of common names. In areas without widespread registration of births and deaths, DSS study staff could also facilitate obtaining birth and death certificates to minimize the problem of date recall.
The finding of biased ascertainment by age and sex pointed to several potential means of improving DSS methodology. Identifying and monitoring pregnancies among women enrolled in this DSS, which was not done during the two DSS rounds included in this analysis, would increase the likelihood of identifying both births and neonatal deaths. Furthermore, the addition of an independent, real-time village reporting system may also improve the likelihood of identifying births and deaths of children that occur between DSS rounds. The problem of differential ascertainment of deaths by sex is not one that is easily solved. However, improving registration of births through monitoring of pregnancies would likely improve the registration of all children who are born and die between DSS rounds.
The results of this study should not be interpreted to suggest that child mortality rates obtained from a DSS with <100% ascertainment are necessarily incorrect. We found that nearly all (92%) individuals whose deaths were missed by the DSS were themselves missing from the population database. Our investigation of the children missed by the DSS could find no specific reasons for their absence besides random data collection errors. Therefore, a mortality rate derived from these data would be less precise than if all deaths and individuals had been registered, but likely would not be biased because there was no indication that the mortality rate among children missed completely by the DSS was different from those included.
Results presented here suggest that capture-recapture may prove to be a worthwhile tool for assessing completeness and differential ascertainment of vital events between subgroups of the population monitored with a DSS. In many developing countries, care will have to be taken to improve the facility with which names and dates can be matched. Information from capture-recapture analyses may then suggest changes to DSS methodology to improve registration of vital events in specific segments of the study population, thereby improving precision in measuring mortality rates overall.
Received February 4, 2003. Accepted for publication April 26, 2003.
Acknowledgments: We express our gratitude to the villagers of Asembo and Gem for their participation in this research. George Olang, James Kwach, Michael Onyango, Richard Otieno, and Maurice Ombok are thanked for their field and data management skills. We are grateful to the Centers for Disease Control and Prevention/Kenya Medical Research Institute administrative team for their support. We thank Dr. William A. Hawley for reviewing the manuscript, and the Director of the Kenya Medical Research Institute for his permission to publish this manuscript.
Financial support: This research was funded by the United States Agency for International Development.
Disclaimer: The opinions or assertions contained in this manuscript are the private ones of the authors and are not to be construed as official or reflecting the views of the U.S. Public Health Service or Department of Health and Human Services.
Authors addresses: Thomas P. Eisele, 3243 Dellwood Road, Cleveland Heights, OH 44118. E-mail: teisele{at}tulane.edu. Kim A. Lindblade, Daniel H. Rosen, Frank Odhiambo, John M. Vulule, and Laurence Slutsker, Centers for Disease Control and Prevention/Kenya Medical Research Institute, PO Box 1578, Kisumu, Kenya.
| REFERENCES |
|
|
|---|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |