| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
 |
ABSTRACT |
 TOP ABSTRACT INTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
|
INTRODUCTION |
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
70% of those with clinical illness.3–6 In many countries known to be at risk for JE transmission, the burden of disease is not known. The clinical presentation of JE cannot be accurately differentiated from other etiologies of meningoencephalitis; therefore, laboratory diagnostic confirmation is required for definitive diagnosis. Because affordable JE diagnostics are generally not available, cases of encephalitis and other severe neurologic illnesses that are truly due to JE virus may not be recognized or reported to public health officials. Limited awareness of disease burden means less chance to control this vaccine-preventable disease. To address this situation, the World Health Organization (WHO) has planned initiatives to support and build laboratory capacity to strengthen JE surveillance.7 Identification of one or more reliable, simple, and available diagnostic tests would facilitate this effort.
Although neutralizing antibody assays of paired serum samples are considered the gold standard for JE diagnostics, the capacity to perform such assays is rarely present in many affected nations. Instead, the simpler immunoglobulin M (IgM) antibody capture enzyme-linked immunosorbent assay (MAC ELISA) for cerebrospinal fluid (CSF) and serum has become the practical standard for the diagnosis of JE.2,8–10 "In-house" JE ELISA diagnostics are available at many research facilities; however, the performance of in-house tests in less sophisticated field laboratories has generally proven unsatisfactory (Nisalak A, unpublished data).
The ELISA format for confirmatory diagnosis of JE infection in Asia is attractive for several reasons. In the context of the goal of measles elimination, WHO has established a Measles Laboratory Network to strengthen laboratory capacity for measles testing.11 Because ELISAs are also widely used for measles diagnosis, the equipment and trained personnel to perform ELISAs already exist in many places and JE testing can be easily integrated. The time required to complete testing is relatively short, particularly with these newly developed commercial kits (which is a particular advantage over some in-house kits).
Although not commercially available at the time of this study, three standardized JE MAC ELISA diagnostic kits had been developed with the intent for commercial sale: the Japanese Encephalitis–Dengue IgM Combo ELISA Test manufactured by Panbio Limited, the Japanese Encephalitis IgM ELISA manufactured by InBios International, Inc., and the JEV CheX kit manufactured by XCyton Diagnostics Ltd.12–15 All three tests use a cell culture-derived recombinant particulate JE antigen; the Panbio test also uses recombinant dengue 1–4 antigens.
The Department of Virology, United States Army Medical Component–Armed Forces Research Institute of Medical Sciences (USAMC-AFRIMS), a regional reference laboratory for Asia designated as a WHO Collaborating Center, has an in-house JE MAC ELISA. This assay has been widely used across the region for many years and was shown to be very accurate when compared with JE hemagglutination inhibition assay results during the test’s development.16 The goal of this study was to evaluate the sensitivity, specificity, and usability of the three diagnostic kits against this regionally recognized JE MAC ELISA using specimens containing either JE IgM, dengue IgM, or neither. The inclusion of dengue IgM positive specimens was considered necessary because dengue virus co-circulates in many regions of Asia where JE virus is found, and the potential for cross-reactivity between flaviviruses is well recognized.3,4
|
MATERIALS AND METHODS |
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
Test methods. The AFRIMS in-house JE IgM ELISA was used as the reference standard for this evaluation. The AFRIMS JE IgM ELISA was run according to the standard operating procedures (SOPs) in duplicate.9,16,17 If a pair of AFRIMS test results was not concordant, then the initial results were discarded and the sample testing was repeated in duplicate. The same procedure was undertaken using the AFRIMS dengue ELISA. A second trained AFRIMS technician checked for proper test completion and sample use.
A single AFRIMS technician with > 17 years of experience performed all evaluation testing. This technician was blinded both to the manufacturer of each kit as well as to the classification of each sample. The technician thawed each sample only once prior to testing. AFRIMS stored all kits in the manner specified by the three commercial manufacturers in the package inserts. The AFRIMS technician ran the tests sequentially using each manufacturer’s written SOPs/ dilutions. Samples with equivocal results were retested with the reagents provided when indicated by the SOP. Duplicate testing was not undertaken for the test kits as this would not typically be undertaken when testing is conducted in the field. The technician classified the AFRIMS results as "JE infection," "negative," "primary dengue," or "secondary dengue" and classified the kit results according to the cut-off values provided by the manufacturer as "positive," "negative," or "equivocal" for JE (if indicated). The Panbio kit used a two-step testing algorithm that requires both JE and dengue IgM results to diagnose JE.
The AFRIMS laboratory technician also completed a questionnaire on the same day of testing regarding operational aspects of each kit and its ease of use. The protocol was approved by the institutional review board of the Walter Reed Army Institute of Research.
Statistical methods. For the purpose of this evaluation, the AFRIMS test results were considered in two categories, JE positive and negative. Samples with AFRIMS results of "JE infection" were labeled "positive" for JE infection and the remainder were "negative" for JE infection (i.e., those with results of "negative," "primary dengue," and "secondary dengue"). Equivocal results from test kits were coded negative for analysis of agreement, sensitivity, and specificity. Sensitivity was defined as the proportion of the samples with an AFRIMS result of "JE infection positive" that had a positive kit test result. Specificity was defined as the proportion of the samples with an AFRIMS result of "JE infection negative" that had a negative kit test result. Agreement was assessed using Cohen’s Kappa. Data were analyzed using SPSS version 11.5 (SPSS Inc., Chicago, IL), and quadratic 95% confidence intervals were obtained using Epi Info 6.04d (Centers for Disease Control and Prevention, Atlanta, GA).
|
RESULTS |
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
Quantitative analysis.
Comparing the results of the diagnostic kits from each of three manufacturers to the AFRIMS IgM ELISA results showed the Panbio kit had sensitivity and specificity of 89.3% and 99.2%, respectively (Figure 1
and Table 1); the InBios kit had sensitivity and specificity of 99.2% and 56.1%, respectively; and the XCyton kit had sensitivity and specificity of 96.7% and 65.3%, respectively. The agreement with the AFRIMS IgM ELISA was 90.4%, 45.6%, and 53.4% for the Panbio, InBios, and XCyton kits, respectively.
|
|
Table 2 provides a breakdown of the subclassification of results for each kit compared with the AFRIMS test results for each category of sample tested (i.e., JE infection, negative, primary dengue, and secondary dengue). The Panbio kit correctly identified all specimens that were JE IgM-negative/ dengue IgM-positive as negative for JE infection. The Panbio kit had 10.7% false negatives among the JE IgM-positive/ dengue IgM-negative specimens; of the 13 false negatives, using Panbio criteria, 6 were negative and 7 were equivocal. The InBios kit had 3.9% false positives among the JE IgM-negative/dengue IgM-negative samples; however, an additional 22.7% of these samples were classified as equivocal (and considered negative in this analysis). The InBios kit did not correctly identify any of the 111 JE IgM-negative/dengue IgM-positive samples as JE negative. The XCyton kit correctly identified 96.1% of JE IgM-negative/dengue IgM-negative samples as JE negative. However, the XCyton kit incorrectly identified 70.3% of the JE IgM-negative/dengue IgM-positive samples as JE positive.
|
100–200 samples required 3–3.5 hours of "hands-on" time and 5–8 hours in total to complete testing (Table 3). The technician estimated that for 20 samples it would require 1–1.5 hours and 4–6 hours of "hands-on" and total time, respectively. The AFRIMS IgM ELISA needs to be incubated overnight, therefore requiring 20 hours from start to finish.
|
). Ultimately, however, the technician did not prefer any one kit to the others. The technician also described issues that could affect ease of use in a peripheral laboratory. Each kit required components or procedures that, although easy to obtain or perform in a reference laboratory setting, might be challenging to obtain or conduct in a peripheral laboratory. |
DISCUSSION |
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
Use of diagnostics should be evaluated according to the desired endpoint of testing. If data are to be used for JE surveillance, then sensitivity and specificity need to be balanced with the needs of the system. Different diseases have differing diagnostic needs and health-system responses. Whereas sensitivity is important for viral infections when specific antiviral treatment options exist, such as herpes simplex, specificity may be more important where specific public health responses (e.g., mass vaccination) are needed to control disease, such as yellow fever. If the test is being used in a research setting then sensitivity and specificity can be pursued with additional testing in more sophisticated and expensive laboratory settings. In JE surveillance that is used for public health planning, improved sensitivity and specificity can be obtained by controlling the patient population that is tested (i.e., testing only those with AES), balanced with quality assurance and quality control mechanisms designed to address the needs of the system.
In this evaluation, a panel of well-characterized sera was used to assess the performance of each diagnostic kit. Serum samples, rather than CSF, were chosen for two reasons. First, laboratory confirmation for a case of JE in the WHO JE surveillance standards includes "presence of JE virus-specific IgM antibody in a single sample of CSF or serum, as detected by an IgM-capture ELISA specifically for JE virus." Second, although CSF is the preferred specimen for diagnosis, in some JE-endemic countries, often only a serum specimen is collected for diagnosis. For these reasons, assessment of kit performance with serum was considered important, as issues of cross-reactivity are likely to be more of a problem than in CSF. Currently, labeling on the Panbio and InBios kits allows only for use with sera; however, correspondence with both companies indicated that this was because CSF samples were not readily available for validating and optimizing performance.
The panel chosen for this evaluation included samples from patients with acute dengue virus infection. The problem of cross-reactivity when testing for antibodies to flaviviruses is a well-recognized phenomenon.3,4 In most JE endemic areas, co-circulation of dengue and other flaviviruses frequently occurs. The Panbio kit has an advantage when used in settings where dengue co-circulates. This was the only one of the three JE kits with at least 90% agreement to the AFRIMS standard when both JE and dengue IgM-positive samples were considered in the analysis. In this situation, sensitivity was good for the InBios and XCyton kits but lower for the Panbio kit. However, specificity was low for both the InBios and XCyton kits as a result of cross-reactivity with dengue antibodies.
The relative importance of dengue cross-reactivity is directly related to the application of the test in real clinical settings. If the test is used exclusively in patients presenting with an acute encephalitis syndrome, then issues with dengue cross-reactivity are less relevant. Although dengue infections occasionally cause encephalopathy, such infections are not considered to cause encephalitis. Therefore, when testing samples from encephalitic patients, dengue is not likely to be the pathogen, so the issue of dengue cross-reactivity is of less importance. There is a chance that if dengue virus was circulating in a JE-endemic area, serum samples only were collected, and the InBios or XCyton kits were used for diagnosis of an encephalitic patient, then a non-JE patient could be misclassified as JE. For example, if a child were to present with non-JE encephalitis but with a concurrent (possibly asymptomatic) or recent dengue infection, this child could be misclassified as JE with these two kits. However, this is not likely to be a frequent occurrence. It is likely that the positive predictive value (PPV) with any of the three kits is high when used in diagnosing persons with an acute encephalitis syndrome. All three kits showed specificity of 96% or above when dengue cross-reactivity was not a factor. This issue will be further evaluated in field trials of the kits.
For both the InBios and XCyton kits, altering the cut-off values would, of course, improve specificity, with an associated reduction in sensitivity. However, it was apparent that the problem of low specificity could not be overcome sufficiently in sample pools that include JE IgM-negative/dengue IgM-positive samples. A two-step testing algorithm in which dengue testing is performed, as the Panbio kit does, may be the only way to achieve both adequate sensitivity and specificity in a population where dengue-like illnesses (i.e., febrile illnesses) are tested for JE and dengue cross-reactivity is an issue.
More than one encephalitic flavivirus may be present at the same time as JE virus—for example, West Nile virus or Murray Valley encephalitis virus. These flaviviruses are closely related to JE virus and would likely cause false positives with any of the kits identified. This was not tested in this evaluation but may be investigated in the future. Differentiating between flaviviral infections is important, and until comprehensive and simple diagnostics are available for all flaviviruses, use of epidemiologic data and referral laboratories for quality assurance and quality control will be needed.
In considering use of diagnostic kits, sensitivity, specificity, ease of use, affordability, and availability are all important factors. Price could not be considered in this assessment as the kits were not available in the marketplace at the time of the assessment. Fortunately, equipment already exists for conducting ELISAs, and ELISA methodology is relatively simple, which will improve affordability and feasibility. However, cost will be an important factor to be considered as companies establish their public-sector pricing.
This study confirms the appropriateness of at least one commercial kit for use in any JE-endemic setting. This is an important step in meeting the demand for diagnostics from public health surveillance programs and in determining the real burden of JE in Asia. Forthcoming field evaluations will assess PPV in a patient population with encephalitis in an area known to have both JE and dengue fever, as well as assess usability in peripheral hospital laboratories. These evaluations will determine if more than one kit is appropriate for widespread use and will provide additional information on the accuracy and usability of these kits in a field setting.
Received December 22, 2006. Accepted for publication April 3, 2007.
Acknowledgments: The authors thank Khun Nawarat Charoensri and Khun Panor Srisongkram, USAMC-AFRIMS, for their performance of the tests; Dr. Ananda Nisalak of USAMC-AFRIMS for technical advice; and Dr. Martha Roper for her contributions to the analyses of the results.
Financial support: This work was funded by the Bill & Melinda Gates Foundation.
* Address correspondence to Julie A. Jacobson, Japanese Encephalitis Project, PATH, 1455 NW Leary Way, Seattle, Washington 98107. E-mail: jjacobs{at}path.org 
Authors’ addresses: Julie A. Jacobson, Susan L. Hills, and Jennifer L. Winkler, Japanese Encephalitis Project, PATH, 1455 NW Leary Way, Seattle, WA 98107, Telephone: +1 (206) 285-3500, Fax: +1 (206) 285-6619, E-mails: jjacobs{at}path.org, shills{at}path.org, and jwinkler{at}path.org. Mammen Mammen, Butsaya Thaisomboonsuk, and Robert V. Gibbons, Department of Virology, United States Army Medical Component–Armed Forces Research Institute of Medical Sciences, 315/6 Rajvithi Road, Bangkok 10400, Thailand, Telephone: +662 644 5644, Fax: +662-644-4760, E-mails: mammen.mammen{at}us.army.mil, ButsayaT{at}afrims.org, and Robert.Gibbons{at}afrims.org. Anthony A. Marfin, Division of Global Migration and Quarantine, Centers for Disease Control and Prevention, 2201 Sixth Avenue, Seattle, WA 98121, Telephone: +1 (206) 615-3600, Fax: +1 (206) 615-2481, E-mail: aam0{at}cdc.gov,
Reprint requests: Julie Jacobson, Japanese Encephalitis Project, PATH, 1455 NW Leary Way, Seattle, WA 98107, Telephone: +1 (206) 285-3500, Fax: +1 (206) 285-6619, E-mail: jeproject{at}path.org.
|
REFERENCES |
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
