• 1.

    World Health Organization, 2002. Annex 3: Burden of Disease in DALYs by Cause, Sex and Mortality Stratum in WHO Regions, Estimates for 2001. The World Health Report. Geneva: WHO, 192197.

    • Search Google Scholar
    • Export Citation
  • 2.

    Mauricio IL, Stothard JR, Miles MA, 2000. The strange case of Leishmania chagasi. Parasitol Today 16: 188189.

  • 3.

    Moreno J, Alvar J, 2002. Canine leishmaniasis: epidemiological risk and the experimental model. Trends Parasitol 18: 399405.

  • 4.

    Dantas-Torres F, 2007. The role of dogs as reservoirs of Leishmania parasites, with emphasis on Leishmania (Leishmania) infantum and Leishmania (Viannia) braziliensis. Vet Parasitol 149: 139146.

    • Search Google Scholar
    • Export Citation
  • 5.

    Gavgani AS, Mohite H, Edrissian GH, Mohebali M, Davies CR, 2002. Domestic dog ownership in Iran is a risk factor for human infection with Leishmania infantum. Am J Trop Med Hyg 67: 511515.

    • Search Google Scholar
    • Export Citation
  • 6.

    Dye C, Killick-Kendrick R, Vitutia MM, Walton R, Killick-Kendrick M, Harith AE, Guy MW, Canavate MC, Hasibeder G, 1992. Epidemiology of canine leishmaniasis: prevalence, incidence and basic reproduction number calculated from a cross-sectional serological survey on the island of Gozo, Malta. Parasitology 105: 3541.

    • Search Google Scholar
    • Export Citation
  • 7.

    Schnur LF, Jacobson RL, 1987. Appendix III. Parasitological techniques. Peters W, Killick-Kendrick M,eds. The Leishmaniases in Biology and Medicine. New York: Academic Press, 499542.

    • Search Google Scholar
    • Export Citation
  • 8.

    Solano-Gallego L, Morell P, Arboix M, Alberola J, Ferrer L, 2001. Prevalence of Leishmania infantum infection in dogs living in an area of canine leishmaniasis endemicity using PCR on several tissues and serology. J Clin Microbiol 39: 560563.

    • Search Google Scholar
    • Export Citation
  • 9.

    Cabral M, O’Grady J, Alexander J, 1992. Demonstration of Leishmania specific cell mediated and humoral immunity in asymptomatic dogs. Parasite Immunol 14: 531539.

    • Search Google Scholar
    • Export Citation
  • 10.

    Cardoso L, Neto F, Sousa JC, Rodrigues M, Cabral M, 1998. Use of a leishmanin skin test in the detection of canine Leishmania-specific cellular immunity. Vet Parasitol 79: 213220.

    • Search Google Scholar
    • Export Citation
  • 11.

    Ashford DA, Bozza M, Freire M, Miranda JC, Sherlock I, Eulalio C, Lopes U, Fernandes O, Degrave W, Barker RH Jr, Badaro R, David JR, 1995. Comparison of the polymerase chain reaction and serology for the detection of canine visceral leishmaniasis. Am J Trop Med Hyg 53: 251255.

    • Search Google Scholar
    • Export Citation
  • 12.

    Mathis A, Deplazes P, 1995. PCR and in vitro cultivation for detection of Leishmania spp. in diagnostic samples from humans and dogs. J Clin Microbiol 33: 11451149.

    • Search Google Scholar
    • Export Citation
  • 13.

    Reale S, Maxia L, Vitale F, Glorioso NS, Caracappa S, Vesco G, 1999. Detection of Leishmania infantum in dogs by PCR with lymph node aspirates and blood. J Clin Microbiol 37: 29312935.

    • Search Google Scholar
    • Export Citation
  • 14.

    Berrahal F, Mary C, Roze M, Berenger A, Escoffier K, Lamouroux D, Dunan S, 1996. Canine leishmaniasis: identification of asymptomatic carriers by polymerase chain reaction and immunoblotting. Am J Trop Med Hyg 55: 273277.

    • Search Google Scholar
    • Export Citation
  • 15.

    Valladares JE, Riera C, Alberola J, Gallego M, Portus M, Cristofol C, Franquelo C, Arboix M, 1998. Pharmacokinetics of meglumine antimoniate after administration of a multiple dose in dogs experimentally infected with Leishmania infantum. Vet Parasitol 75: 3340.

    • Search Google Scholar
    • Export Citation
  • 16.

    Rodriguez-Cortes A, Ojeda A, Lopez-Fuertes L, Timon M, Altet L, Solano-Gallego L, Sanchez-Robert E, Francino O, Alberola J, 2007. A long term experimental study of canine visceral leishmaniasis. Int J Parasitol 37: 683693.

    • Search Google Scholar
    • Export Citation
  • 17.

    Reed SG, Shreffler WG, Burns JM Jr, Scott JM, Orge Mda G, Ghalib HW, Siddig M, Badaro R, 1990. An improved serodiagnostic procedure for visceral leishmaniasis. Am J Trop Med Hyg 43: 632639.

    • Search Google Scholar
    • Export Citation
  • 18.

    Riera C, Valladares JE, Gallego M, Aisa MJ, Castillejo S, Fisa R, Ribas N, Carrio J, Alberola J, Arboix M, 1999. Serological and parasitological follow-up in dogs experimentally infected with Leishmania infantum and treated with meglumine antimoniate. Vet Parasitol 84: 3347.

    • Search Google Scholar
    • Export Citation
  • 19.

    Solano-Gallego L, Riera C, Roura X, Iniesta L, Gallego M, Valladares JE, Fisa R, Castillejo S, Alberola J, Ferrer L, Arboix M, Portus M, 2001. Leishmania infantum-specific IgG, IgG1 and IgG2 antibody responses in healthy and ill dogs from endemic areas. Evolution in the course of infection and after treatment. Vet Parasitol 96: 265276.

    • Search Google Scholar
    • Export Citation
  • 20.

    Rodriguez A, Solano-Gallego L, Ojeda A, Quintana J, Riera C, Gallego M, Portus M, Alberola J, 2006. Dynamics of Leishmania-specific immunoglobulin isotypes in dogs with clinical leishmaniasis before and after treatment. J Vet Intern Med 20: 495498.

    • Search Google Scholar
    • Export Citation
  • 21.

    Peng ZK, Simons FE, Becker AB, 1991. Differential binding properties of protein A and protein G for dog immunoglobulins. J Immunol Methods 145: 255258.

    • Search Google Scholar
    • Export Citation
  • 22.

    Barajas-Rojas JA, Riemann HP, Franti CE, 1993. Notes about determining the cut-off value in enzyme linked immunosorbent assay (ELISA). Prev Vet Med 15: 231233.

    • Search Google Scholar
    • Export Citation
  • 23.

    Bray RS, 1985. Immunodiagnosis of leishmaniasis. Chang KP, Bray RS, eds. Leishmaniasis. Amsterdam: Elsevier Science Publishers.

  • 24.

    Alberola J, Rodriguez A, Francino O, Roura X, Rivas L, Andreu D, 2004. Safety and efficacy of antimicrobial peptides against naturally acquired leishmaniasis. Antimicrob Agents Chemother 48: 641643.

    • Search Google Scholar
    • Export Citation
  • 25.

    Francino O, Altet L, Sanchez-Robert E, Rodriguez A, Solano-Gallego L, Alberola J, Ferrer L, Sanchez A, Roura X, 2006. Advantages of real-time PCR assay for diagnosis and monitoring of canine leishmaniosis. Vet Parasitol 137: 214221.

    • Search Google Scholar
    • Export Citation
  • 26.

    Livak KJ, Schmittgen TD, 2001. Analysis of relative gene expression data using real-time quantitative PCR and the 2(-Delta Delta C(T)) method. Method 25: 402408.

    • Search Google Scholar
    • Export Citation
  • 27.

    Böyum A, 1968. Isolation of mononuclear cells and granulocytes from human blood. Scand J Clin Lab Invest 21: 7789.

  • 28.

    Newcombe NG, 1998. Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med 17: 857872.

  • 29.

    Deeks JJ, Altman DG, 1999. Sensitivity and specificity and their confidence intervals cannot exceed 100%. BMJ 318: 193194.

  • 30.

    Simel DL, Samsa GP, Matchar DB, 1991. Likelihood ratios with confidence: sample size estimation for diagnostic test studies. J Clin Epidemiol 44: 763770.

    • Search Google Scholar
    • Export Citation
  • 31.

    Benjamini Y, Hochberg Y, 1995. Controlling the false discovery rate—a practical and powerful approach to multiple testing. J R Stat Soc, B 57: 289300.

    • Search Google Scholar
    • Export Citation
  • 32.

    Jaeschke R, Guyatt G, Sackett DL, 1994. Users’ guides to the medical literature. III. How to use an article about a diagnostic test. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA 271: 389391.

    • Search Google Scholar
    • Export Citation
  • 33.

    Christenson RH, 2007. Evidence-based laboratory medicine—a guide for critical evaluation of in vitro laboratory testing. Ann Clin Biochem 44: 111130.

    • Search Google Scholar
    • Export Citation
  • 34.

    Fischer JE, Bachmann LM, Jaeschke R, 2003. A readers’ guide to the interpretation of diagnostic test properties: clinical example of sepsis. Intensive Care Med 29: 10431051.

    • Search Google Scholar
    • Export Citation
  • 35.

    Feinstein AR, 1975. Clinical biostatistics XXXI. On the sensitivity, specificity, and discrimination of diagnostic tests. Clin Pharmacol Ther 17: 104116.

    • Search Google Scholar
    • Export Citation
  • 36.

    Fagan TJ, 1975. Letter: nomogram for Bayes theorem. N Engl J Med 293: 257.

  • 37.

    Deeks JJ, Altman DG, 2004. Diagnostic tests 4: likelihood ratios. BMJ 329: 168169.

  • 38.

    Maia C, Campino L, Methods for diagnosis of canine leishmaniasis and immune response to infection. Vet Parasitol 158: 274287.

  • 39.

    Rodriguez-Cortes A, Fernandez-Bellon H, Ramis A, Ferrer L, Alberola J, Solano-Gallego L, 2007. Leishmania-specific isotype levels and their relationship with specific cell-mediated immunity parameters in canine leishmaniasis. Vet Immunol Immunopathol 116: 190198.

    • Search Google Scholar
    • Export Citation

 

 

 

 

Leishmania Infection: Laboratory Diagnosing in the Absence of a “Gold Standard”

View More View Less
  • Unitat de Farmacologia Veterinària and LeishLAB–Servei d’Anàlisi de Fàrmacs, Departament de Farmacologia, de Terapèutica i de Toxicologia, Universitat Autònoma de Barcelona, Bellaterra (Barcelona), Spain; Servei Veterinari de Genètica Molecular, Departament de Ciència Animal i dels Aliments, Facultat de Veterinària, Universitat Autònoma de Barcelona, Bellaterra (Barcelona), Spain; Innoqua, Toxicology Consultants, Barcelona, Spain; Agencia Española del Medicamento, Madrid, Spain

There is no gold standard for diagnosing leishmaniases. Our aim was to assess the operative validity of tests used in detecting Leishmania infection using samples from experimental infections, a reliable equivalent to the classic definition of gold standard. Without statistical differences, the highest sensitivity was achieved by protein A (ProtA), immunoglobulin (Ig)G2, indirect fluorescenece antibody test (IFAT), lymphocyte proliferation assay, quantitative real-time polymerase chain reaction of bone marrow (qPCR-BM), qPCR-Blood, and IgG; and the highest specificity by IgG1, IgM, IgA, qPCR-Blood, IgG, IgG2, and qPCR-BM. Maximum positive predictive value was obtained simultaneously by IgG2, qPCR-Blood, and IgG; and maximum negative predictive value by qPCR-BM. Best positive and negative likelihood ratios were obtained by IgG2. The test having the greatest, statistically significant, area under the receiver operating characteristics curve was IgG2 enzyme-linked immunosorbent assay (ELISA). Thus, according to the gold standard used, IFAT and qPCR are far from fulfilling the requirements to be considered gold standards, and the test showing the highest potential to detect Leishmania infection is Leishmania-specific ELISA IgG2.

Introduction

Leishmaniases are endemic in 88 countries on four continents. In terms of global disease burden, leishmaniases are the third most important vector-borne disease, after malaria and lymphatic filariasis, causing 2.4 million disability-adjusted life years lost and 59,000 deaths in 2001.1 The causative agents are parasitic protozoa of the genus Leishmania. Visceral leishmaniasis is caused by species of the Leishmania donovani complex: L. donovani in the Palearctic and L. infantum (= L. chagasi2) in both the Palearctic and the Neotropical ecozones. L. infantum is the etiological agent of a widespread serious zoonotic disease that affects both humans and dogs. Dogs are considered the primary peridomestic reservoir host3,4 and controlling the dog infection rate reduces the incidence in humans.5 Sand flies from the genus Phlebotomus spp. in the Palearctic and Lutzomyia spp. in the Nearctic are the proven vectors of human and canine leishmaniases (CaL).

Prevalence and incidence of the canine infection are important parameters to determine the risk and the ways to control this reemergent zoonosis. The estimation of these parameters depends on the reliable identification of infected dogs.6 Parasitological examination is a very specific method but lacks sensitivity.7 Serological methods are more sensitive but may be less specific and the choice of the cut-off value may not be obvious.6 Moreover, seroprevalence underestimates the true prevalence of infection.8 Cellular immune response, as measured by lymphoblastogenesis, seems to be specially suited to detect current or previous infection in clinically healthy dogs.9,10 However, there may be no obvious cut-off. PCR has been confirmed to be a very sensitive and specific technique.1113 However, whether PCR is sensitive enough to detect all infected individuals is controversial due to conflicting results and the difficulties to choose the tissue to be sampled.11,14

The “gold standard” is the method, or composite of methods, giving results that are regarded as unequivocal classifications. Its use is a necessary prerequisite to examine the diagnostic utility of any test. The only true gold standard for classifying an animal as infected is the isolation of infectious agents or unequivocal histopathological criteria. Unfortunately, there is not such a gold standard for Leishmania infection. A way to validate diagnostic tests when lacking a classic gold standard is to take advantage of experimental infections. The strength of this method is that it not only measures the ability of the assay to detect infection, but also allows the kinetics of infection to be followed. In this setting everything is known about every sample collected and about the individual it is collected from. The aim of the present study is to assess the operative validity of several tests used in detection of Leishmania infection using samples from experimental infections, a reliable equivalent to the classic definition of gold standard.

Material and Methods

Samples and subjects.

Six healthy, 9-month-old, intact, female beagle dogs were used. Dogs were housed in indoor kennels (Isoquimen S.L., St. Feliu de Codines, Spain) with windows covered with delthamethrin-sprayed, double anti-mosquito-nets, and according to the Guiding Principles for the Care and Use of Animals, following the guidelines of the institution's ethics committee.

L. infantum strain MCAN/ES/92/BCN-83/MON-1 obtained from a dog that acquired the infection naturally and had not received treatment was passaged through hamsters. Parasites were then grown in Schneider's insect medium (Sigma, St. Louis, MO), supplemented with 20% fetal calf serum (Gibco, Paisley, Scotland) and 25 µg · mL−1 gentamicin (Sigma) at 26°C. Parasites in stationary phase were washed and resuspended in phosphate buffered saline (PBS) at 5 × 107 promastigotes per mL. Finally 1 mL was injected by the intravenous route to every dog.15

Samples were collected at monthly intervals beginning 3 months before the infection. One year after experimental infection, the dogs were euthanized and infection verified in all dogs at necropsy. Details of clinicopathological, parasitological, and immunological evolution of the infection have been described elsewhere.16

Enzyme-linked immunosorbent assay (ELISA).

ELISA was performed as described.1720 Polyclonal anti-dog immunoglobulin (Ig) G, IgG1, IgG2, IgA, IgM (Bethyl Laboratories, Montgomery, TX) and Protein A (ProtA) (ImmunoPure® Recomb® Protein A, Pierce Rockford, IL), which reacts with the Fc-region of Ig from several species including the dog,21 all conjugated to horseradish peroxidase and were used individually as secondary antibodies. The reaction was expressed in ELISA units (EU) in relation to a known positive serum, used as a calibrator and arbitrarily set at 100 EU. This calibrator serum, always the same, was included in all plates, and plates with an interassay variation > 10% were discarded. Sera from 32 healthy dogs living in a nonendemic region for CaL and proven to be parasitologically and serologically L. infantum-negative—tested by visualization of L. infantum organisms on bone marrow smears, by Leishmania DNA in blood and bone marrow, and by detection of specific antibodies—were used to set up a standard cut-off, established at mean + 2 SD.22

Indirect fluorescenece antibody test (IFAT).

The antigen was prepared from promastigotes of the L. infantum zymodeme MON-1 international reference strain MHOM/FR/78/LEM-75, and anti-Leishmania antibodies were detected using rabbit anti-dog IgG conjugated to fluorescein isothiocyanate (Sigma-Aldrich, Inc., St. Louis, MO). Samples were classified positive if promastigote cytoplasmatic or membrane fluorescence was observed at a serum dilution of 1:80 or higher.23

Quantitative real-time polymerase chain reaction (qPCR) Real-Time PCR.

qPCR was carried out as described.16,24,25 Briefly, 0.5 mL of peripheral whole blood (qPCR-Blood) or 0.1 mL of bone marrow aspirate (qPCR-BM) were used. DNA was extracted and diluted in milliQ water (1/10 for bone marrow aspirate and 1/5 for blood) and used for the qPCR. Primers (5′-AACTTTCTGGTCCTCCGGGTAG-3′ and 5′-ACCCCCAGTTTCCCGCC-3′) and the TaqMan MGB probe (5′-AAAAATGGGTGCAGAAAT-3′) were designed to target the kinetoplast minicircle DNA of L. infantum. The eukaryotic 18S RNA Pre-Developed TaqMan assay reagents (Applied Biosystems, Carlsbad, CA) were used as an internal reference of canine genomic DNA amplification to normalize L. infantum amplification for differences in DNA content or the presence of inhibitors. Amplification was performed in triplicate in 25 μL reaction mixture (TaqMan Universal qPCR Master Mix; Applied Biosystems). Thermal cycling profile was 50°C for 2 minutes, 95°C for 10 minutes, 40 cycles at 95°C for 15 seconds, and 60°C for 60 seconds. Quantitative analysis of L. infantum DNA amplification was performed by the comparative threshold cycle (Ct) method (2−ΔΔCt)26 and using a spiked sample with a known number of parasites/well as calibrator, allowing determination of the number of parasites in any qPCR sample, independently of the amount of DNA added or the presence of inhibitors.

Lymphocyte proliferation assay (LPA).

Peripheral blood mononuclear cells (PBMC) were isolated from 6 mL of heparinised venous blood samples using Ficoll–Hypaque (Histopaque 1077; Sigma, St. Louis, MO).27 Viable PBMC were resuspended at 2 × 106 cells per mL in RPMI-1640 medium (Gibco, Paisley, UK) supplemented with 10% volume/volume heat-inactivated fetal calf serum, 10 mM N-2-hydroxyethylpiperazine-N0-2-ethanesulfonic acid (Hepes buffer), penicillin (100 U/mL) and streptomycin (100 µg/mL). PBMC were cultured in triplicate in flat-bottomed 96-well-microtitre plates at a density of 2 × 105 cells per well in the absence or presence of leishmanial soluble antigen (LSA) or mitogen. Each well contained 100 µL of cell suspension plus 100 µL of either 20 µg/mL LSA, 10 µg/mL phytohemagglutinin (PHA), or medium alone. Cultures were incubated at 37°C in 5% CO2 for 3 days (control medium alone and PHA) or 6 days (control medium alone and LSA) and pulsed during the last 18 hours of culture with 10 μM of 5-bromodeoxyuridine (BrdU). Cell proliferation was assessed using a non-radioactive ELISA technique, according to the manufacturer's instructions (Cellproliferation ELISA, BrdU colorimetric; Boehringer-Mannheim, Germany).16

PBMCs from 20 healthy dogs that were Leishmania-negative—tested by visualization of L. infantum organisms on bone marrow smears, by Leishmania DNA in blood and bone marrow, by detection of specific antibodies, and by Leishmanin Skin Test—were cultured in triplicate under the same conditions described above to establish a standard cut-off.16 This was the mean difference between LSA wells minus medium alone wells plus 2 SD.22 Stimulation with PHA was used as a positive control. PHA proliferation values between 1.12 and 1.55 optical densities (95% confidence interval) for the mean difference between PHA and medium alone were considered positive.16

Data analysis.

Statistical analysis was performed using JMP version 7.0.1 (SAS Institute, Cary, NC). Receiver operating characteristics (ROC) curve analysis was performed using MedCalc 9.5.1 (MedCalc Software, Mariakerke, Belgium). Confidence intervals (CI) for sensitivity, specificity, and overall accuracy were produced with the Wilson score method28 to avoid exceeding the lower and upper limits acceptable for CI.29 Positive and negative likelihood ratios CI were calculated with the method described by Simel et al.30 A significance statistical level α = 0.05 was used, employing a rough false discovery rate (rFDR) to correct for multiple simultaneous comparisons.31

Results

Time to detect infection.

The mean time to detect infection differed significantly among tests (analysis of variance; P < 0.0006), ranging from a mean of 53 days for LPA to a mean of 273 days for IgA. Post-hoc multiple comparison analysis (Tukey-Kramer Honestly Significant Difference Test; P < 0.05) showed that two groups of means existed differing between but not within them. The first group, with the fastest tests to detect infection, contained LPA, IgG, IgG2, qPCR-Blood, ProtA, IgG1, and qPCR-BM; and the second group encompassed IgG, IgG2, qPCR-Blood, ProtA, IgG1, qPCR-BM, IFAT, IgM, and IgA (Table 1).

Measures of performance.

Measures of performance for the different tests evaluated are shown in Table 2. Sensitivity, specificity, and overall accuracy differed significantly among tests (Cochran's Q; P < 0.0001 for all three tests). The highest sensitivity was achieved by ProtA with 0.72, although multiple simultaneous comparisons showed that it did not differ statistically from IgG2, IFAT, LPA, qPCR-BM, qPCR-Blood, and IgG (rFDR; P > 0.0256). The highest specificity was achieved simultaneously by IgG1, IgM, IgA, qPCR-Blood, and IgG with 0.99, although they did not differ statistically from IgG2 and qPCR-BM (rFDR; P > 0.0256). The highest overall accuracy was obtained by qPCR-BM with 0.78, though multiple simultaneous comparisons showed that it did not differ statistically from IgG2, ProtA, qPCR-Blood, IgG, and IFAT (rFDR; P > 0.0256).

Maximum positive predictive value (PPV) was obtained simultaneously by IgG2, qPCR-Blood, and IgG with 0.99 although all tests evaluated were above 0.9, except IFAT and LPA. Maximum negative predictive value (NPV) was achieved by qPCR-BM with 0.70, followed by IgG2 with 0.57; all the other tests were below 0.50.

The best positive likelihood ratio (LR+) was obtained by IgG2 with 48.24, followed by qPCR-Blood with 37.00; all other tests were also above 10.00 except IFAT, ProtA, and LPA. The best negative likelihood ratio (LR−) was achieved by IgG2 with 0.35, but all tests showed ratios greater than 0.1. The highest diagnostic odds ratio (DOR) was achieved by IgG2 with 136.71; all other DOR were below 100. Finally, IgG2 showed the highest Youden's index with 0.64.

ROC curve analysis.

The area under the ROC curve ranged from 0.96 for IgG2 to an area of 0.56 for LPA. The area was different from 0.5 for all tests except for LPA (P = 0.2587). Multiple pairwise comparisons showed four groups of areas differing between but not within them (rFDR; P > 0.0256). The first group was composed solely of IgG2; thus, its area was the greatest among all. The second group contained IgG, IFAT, ProtA, IgA, and qPCR-BM. The third group encompassed IFAT, ProtA, IgA, and qPCR-BM. Finally the fourth group embraced qPCR-BM, qPCR-Blood, IgG1, IgM, and LPA (Table 3).

Discussion

A plethora of tests are available to improve diagnostic decision-making in Leishmania infection. A gold standard criterion must be present which allows discrimination of patients into two groups: one with infection and the other without infection. Ideally there should be no other difference between patients that may influence the test result.32 To the authors' knowledge this is the first study using a real gold standard, experimental infection, to assess the accuracy of tests used to diagnose Leishmania infection.

The time a test takes to detect infection is especially important for establishing prompt treatment. In this sense the better tests are LPA, IgG, IgG2, qPCR-Blood, ProtA, IgG1, and qPCR-BM, without statistical differences. However, LPA is useless from the point of view of diagnosing infection because, as explained below, its area under the ROC curve does not differ from 0.5.

The usual measures used to compare the performance of diagnostic tests are sensitivity, specificity, and, to a less extent, accuracy. Sensitivities using our gold standard model range from 0.72 for ProtA to 0.23 for IgM, and the statistical analysis includes the best performing group ProtA, IgG2, IFAT, LPA, qPCR-BM, qPCR-Blood, and IgG. Values are lower than those found in published reports, but sensitivities can only be compared when the same gold standard is used. Furthermore, when using our gold standard, the two most usual surrogates for gold standard in leishmaniasis, IFAT and qPCR, exhibit sensitivities at most equal to some ELISA-based tests. Specificities of the tests used in the present study are high, not different from 1 (95% CI) except for IFAT, ProtA, and LPA. Finally, accuracy ranges from 0.78 for qPCR-BM to 0.47 for IgM, with the most performing group composed of qPCR-BM, IgG2, ProtA, qPCR-Blood, IgG, and IFAT.

ROC curves are particularly useful for comparing the performance of tests. If one test shows a significantly larger area, then it is a better test for evaluating the outcome; if there is no significant difference, then the test does not differ in its ability to predict the outcome.33 Our results show that the test having the greatest, statistically significant, area under the ROC curve is Leishmania-specific IgG2 ELISA, so this test must be considered the better test for evaluating Leishmania infection. On the other hand, LPA must be considered a toss-up because its area does not differ from 0.5. According to the areas under the ROC curve, IgG2 can be classified as showing high accuracy; IgG, IFAT, ProtA, IgA, qPCR-BM, qPCR-Blood, and IgG1 as showing moderate accuracy; IgM, low accuracy; and LPA useless.34 The Youden's index is another commonly used single summary measure of overall diagnostic effectiveness that also combines sensitivity and specificity. This index, related to the ROC curve, shows again that the best test is IgG2 and the worst LPA.

Sensitivity and specificity of a test are the first-and-mandatory criterion for judging its usefulness, and of interest to public-health policymakers; however, these values offer poor translational information, not being enough to make decisions on a patient. Stated alternatively, the sensitivity and specificity—or its area under the ROC curve or Youden's index—of a test are population measures that look backward at results gathered over time,35 and looking at them it is impossible to conclude whether a patient has a certain condition. The clinician is not interested in population measures, but he faces the question: What is the probability of disease in individuals with a positive or negative test? In this sense, PPV and NPV are usually of keen interest because they address this question providing clinically relevant basis on which to compare the performance of the tests. The highest PPV are obtained by IgG2, qPCR-Blood, and IgG, all of them with 0.99, thus a positive in any one of these tests indicates almost certain infection. On the other hand, all evaluated tests showed lower NPV, qPCR-BM being the best with 0.70. Unfortunately, PPV and NPV are not stable characteristics of diagnostic tests as they are dependent on the prevalence of disease. Thus, the generalizability of PPV and NPV from one population to another is limited because of this dependence, even if test accuracy within the diseased and the non-diseased subgroups is constant across populations.

Another tool has been proposed with some practical advantages for the assessment and use of diagnostic tests: the likelihood ratio (LR). In LR sensitivity and specificity are usefully combined, but the LR goes beyond in the sense that LR deals precisely with comparing odds. Thus, the LR tells how many times more—or less—frequent is the result of a test in patients with the disease, compared with people without the disease. Furthermore, LR does not depend mathematically on prevalence, thus being more portable. In fact, the LR quantifies the increase in knowledge of the presence of the disease through the application of the diagnostic test. In our case, the test with the best LR+ and LR− is IgG2 with 48.24 and 0.35, respectively. By convention,32,36 marked changes in prior disease probability can be assumed in LR+ exceeding 10.0 and LR− below 0.1; and 2.0 and 0.5 have been suggested as the minimally useful values for LR+ and LR−, respectively. Thus, all tests evaluated are more or less useful according to their LR+ except LPA; and regarding LR−, IgG2, ProtA, qPCR-BM, and IFAT are the only useful tests. A unique advantage of LR is that using the Bayes’ theorem they adapt the sensitivity and specificity to individual patients. Thus, while the prevalence—pretest probability—of Leishmania infection in ecoregions around the Mediterranean basin is 67%,8 a positive result in IgG2 test increases this pretest probability by 32.99% giving a posttest probability of 99.99%; in turn, a negative result decreases the pretest probability by 25.26%, giving a posttest probability of 41.74%.36,37 In contrast, for example, a positive or negative IFAT implies a change of only 21% and 18%, respectively. Finally, the greater DOR, a single metric to combine LR+ and LR−, is obtained by an IgG2 that almost doubles the second one.

To sum up, LPA is very useful for studying cellular mediated immune response,16 but it is useless from a diagnostic point of view. The commonly used surrogate of gold standard for Leishmania infection IFAT38 is far from satisfying this criterion, and must be considered only a second choice for detecting Leishmania infection. Maybe the best scenario for qPCR is in determining the evolution of parasite load in the evaluation of drugs or vaccines24 or as an adjunct to serological test. Among ELISA tests, IgM performs poorly and its role in leishmaniasis remains to be elucidated; IgA is useful as a means of supporting the clinical assessment during treatment in clinical practice20 and in relation to the dissemination of the parasite,39 but only shows moderate accuracy as a diagnostic tool; finally, IgG isotypes are those showing the greatest interest in diagnosing Leishmania infection, especially IgG2.

In conclusion, in clinical practice the aim is to have a test to accurately identify the patient who has a certain condition. According to our gold standard model, experimental infection, the best suited test to detect individuals harboring Leishmania infection and to provide clinically useful information is ELISA based on polyclonal IgG2.

  • 1.

    World Health Organization, 2002. Annex 3: Burden of Disease in DALYs by Cause, Sex and Mortality Stratum in WHO Regions, Estimates for 2001. The World Health Report. Geneva: WHO, 192197.

    • Search Google Scholar
    • Export Citation
  • 2.

    Mauricio IL, Stothard JR, Miles MA, 2000. The strange case of Leishmania chagasi. Parasitol Today 16: 188189.

  • 3.

    Moreno J, Alvar J, 2002. Canine leishmaniasis: epidemiological risk and the experimental model. Trends Parasitol 18: 399405.

  • 4.

    Dantas-Torres F, 2007. The role of dogs as reservoirs of Leishmania parasites, with emphasis on Leishmania (Leishmania) infantum and Leishmania (Viannia) braziliensis. Vet Parasitol 149: 139146.

    • Search Google Scholar
    • Export Citation
  • 5.

    Gavgani AS, Mohite H, Edrissian GH, Mohebali M, Davies CR, 2002. Domestic dog ownership in Iran is a risk factor for human infection with Leishmania infantum. Am J Trop Med Hyg 67: 511515.

    • Search Google Scholar
    • Export Citation
  • 6.

    Dye C, Killick-Kendrick R, Vitutia MM, Walton R, Killick-Kendrick M, Harith AE, Guy MW, Canavate MC, Hasibeder G, 1992. Epidemiology of canine leishmaniasis: prevalence, incidence and basic reproduction number calculated from a cross-sectional serological survey on the island of Gozo, Malta. Parasitology 105: 3541.

    • Search Google Scholar
    • Export Citation
  • 7.

    Schnur LF, Jacobson RL, 1987. Appendix III. Parasitological techniques. Peters W, Killick-Kendrick M,eds. The Leishmaniases in Biology and Medicine. New York: Academic Press, 499542.

    • Search Google Scholar
    • Export Citation
  • 8.

    Solano-Gallego L, Morell P, Arboix M, Alberola J, Ferrer L, 2001. Prevalence of Leishmania infantum infection in dogs living in an area of canine leishmaniasis endemicity using PCR on several tissues and serology. J Clin Microbiol 39: 560563.

    • Search Google Scholar
    • Export Citation
  • 9.

    Cabral M, O’Grady J, Alexander J, 1992. Demonstration of Leishmania specific cell mediated and humoral immunity in asymptomatic dogs. Parasite Immunol 14: 531539.

    • Search Google Scholar
    • Export Citation
  • 10.

    Cardoso L, Neto F, Sousa JC, Rodrigues M, Cabral M, 1998. Use of a leishmanin skin test in the detection of canine Leishmania-specific cellular immunity. Vet Parasitol 79: 213220.

    • Search Google Scholar
    • Export Citation
  • 11.

    Ashford DA, Bozza M, Freire M, Miranda JC, Sherlock I, Eulalio C, Lopes U, Fernandes O, Degrave W, Barker RH Jr, Badaro R, David JR, 1995. Comparison of the polymerase chain reaction and serology for the detection of canine visceral leishmaniasis. Am J Trop Med Hyg 53: 251255.

    • Search Google Scholar
    • Export Citation
  • 12.

    Mathis A, Deplazes P, 1995. PCR and in vitro cultivation for detection of Leishmania spp. in diagnostic samples from humans and dogs. J Clin Microbiol 33: 11451149.

    • Search Google Scholar
    • Export Citation
  • 13.

    Reale S, Maxia L, Vitale F, Glorioso NS, Caracappa S, Vesco G, 1999. Detection of Leishmania infantum in dogs by PCR with lymph node aspirates and blood. J Clin Microbiol 37: 29312935.

    • Search Google Scholar
    • Export Citation
  • 14.

    Berrahal F, Mary C, Roze M, Berenger A, Escoffier K, Lamouroux D, Dunan S, 1996. Canine leishmaniasis: identification of asymptomatic carriers by polymerase chain reaction and immunoblotting. Am J Trop Med Hyg 55: 273277.

    • Search Google Scholar
    • Export Citation
  • 15.

    Valladares JE, Riera C, Alberola J, Gallego M, Portus M, Cristofol C, Franquelo C, Arboix M, 1998. Pharmacokinetics of meglumine antimoniate after administration of a multiple dose in dogs experimentally infected with Leishmania infantum. Vet Parasitol 75: 3340.

    • Search Google Scholar
    • Export Citation
  • 16.

    Rodriguez-Cortes A, Ojeda A, Lopez-Fuertes L, Timon M, Altet L, Solano-Gallego L, Sanchez-Robert E, Francino O, Alberola J, 2007. A long term experimental study of canine visceral leishmaniasis. Int J Parasitol 37: 683693.

    • Search Google Scholar
    • Export Citation
  • 17.

    Reed SG, Shreffler WG, Burns JM Jr, Scott JM, Orge Mda G, Ghalib HW, Siddig M, Badaro R, 1990. An improved serodiagnostic procedure for visceral leishmaniasis. Am J Trop Med Hyg 43: 632639.

    • Search Google Scholar
    • Export Citation
  • 18.

    Riera C, Valladares JE, Gallego M, Aisa MJ, Castillejo S, Fisa R, Ribas N, Carrio J, Alberola J, Arboix M, 1999. Serological and parasitological follow-up in dogs experimentally infected with Leishmania infantum and treated with meglumine antimoniate. Vet Parasitol 84: 3347.

    • Search Google Scholar
    • Export Citation
  • 19.

    Solano-Gallego L, Riera C, Roura X, Iniesta L, Gallego M, Valladares JE, Fisa R, Castillejo S, Alberola J, Ferrer L, Arboix M, Portus M, 2001. Leishmania infantum-specific IgG, IgG1 and IgG2 antibody responses in healthy and ill dogs from endemic areas. Evolution in the course of infection and after treatment. Vet Parasitol 96: 265276.

    • Search Google Scholar
    • Export Citation
  • 20.

    Rodriguez A, Solano-Gallego L, Ojeda A, Quintana J, Riera C, Gallego M, Portus M, Alberola J, 2006. Dynamics of Leishmania-specific immunoglobulin isotypes in dogs with clinical leishmaniasis before and after treatment. J Vet Intern Med 20: 495498.

    • Search Google Scholar
    • Export Citation
  • 21.

    Peng ZK, Simons FE, Becker AB, 1991. Differential binding properties of protein A and protein G for dog immunoglobulins. J Immunol Methods 145: 255258.

    • Search Google Scholar
    • Export Citation
  • 22.

    Barajas-Rojas JA, Riemann HP, Franti CE, 1993. Notes about determining the cut-off value in enzyme linked immunosorbent assay (ELISA). Prev Vet Med 15: 231233.

    • Search Google Scholar
    • Export Citation
  • 23.

    Bray RS, 1985. Immunodiagnosis of leishmaniasis. Chang KP, Bray RS, eds. Leishmaniasis. Amsterdam: Elsevier Science Publishers.

  • 24.

    Alberola J, Rodriguez A, Francino O, Roura X, Rivas L, Andreu D, 2004. Safety and efficacy of antimicrobial peptides against naturally acquired leishmaniasis. Antimicrob Agents Chemother 48: 641643.

    • Search Google Scholar
    • Export Citation
  • 25.

    Francino O, Altet L, Sanchez-Robert E, Rodriguez A, Solano-Gallego L, Alberola J, Ferrer L, Sanchez A, Roura X, 2006. Advantages of real-time PCR assay for diagnosis and monitoring of canine leishmaniosis. Vet Parasitol 137: 214221.

    • Search Google Scholar
    • Export Citation
  • 26.

    Livak KJ, Schmittgen TD, 2001. Analysis of relative gene expression data using real-time quantitative PCR and the 2(-Delta Delta C(T)) method. Method 25: 402408.

    • Search Google Scholar
    • Export Citation
  • 27.

    Böyum A, 1968. Isolation of mononuclear cells and granulocytes from human blood. Scand J Clin Lab Invest 21: 7789.

  • 28.

    Newcombe NG, 1998. Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med 17: 857872.

  • 29.

    Deeks JJ, Altman DG, 1999. Sensitivity and specificity and their confidence intervals cannot exceed 100%. BMJ 318: 193194.

  • 30.

    Simel DL, Samsa GP, Matchar DB, 1991. Likelihood ratios with confidence: sample size estimation for diagnostic test studies. J Clin Epidemiol 44: 763770.

    • Search Google Scholar
    • Export Citation
  • 31.

    Benjamini Y, Hochberg Y, 1995. Controlling the false discovery rate—a practical and powerful approach to multiple testing. J R Stat Soc, B 57: 289300.

    • Search Google Scholar
    • Export Citation
  • 32.

    Jaeschke R, Guyatt G, Sackett DL, 1994. Users’ guides to the medical literature. III. How to use an article about a diagnostic test. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA 271: 389391.

    • Search Google Scholar
    • Export Citation
  • 33.

    Christenson RH, 2007. Evidence-based laboratory medicine—a guide for critical evaluation of in vitro laboratory testing. Ann Clin Biochem 44: 111130.

    • Search Google Scholar
    • Export Citation
  • 34.

    Fischer JE, Bachmann LM, Jaeschke R, 2003. A readers’ guide to the interpretation of diagnostic test properties: clinical example of sepsis. Intensive Care Med 29: 10431051.

    • Search Google Scholar
    • Export Citation
  • 35.

    Feinstein AR, 1975. Clinical biostatistics XXXI. On the sensitivity, specificity, and discrimination of diagnostic tests. Clin Pharmacol Ther 17: 104116.

    • Search Google Scholar
    • Export Citation
  • 36.

    Fagan TJ, 1975. Letter: nomogram for Bayes theorem. N Engl J Med 293: 257.

  • 37.

    Deeks JJ, Altman DG, 2004. Diagnostic tests 4: likelihood ratios. BMJ 329: 168169.

  • 38.

    Maia C, Campino L, Methods for diagnosis of canine leishmaniasis and immune response to infection. Vet Parasitol 158: 274287.

  • 39.

    Rodriguez-Cortes A, Fernandez-Bellon H, Ramis A, Ferrer L, Alberola J, Solano-Gallego L, 2007. Leishmania-specific isotype levels and their relationship with specific cell-mediated immunity parameters in canine leishmaniasis. Vet Immunol Immunopathol 116: 190198.

    • Search Google Scholar
    • Export Citation

Author Notes

*Address correspondence to Jordi Alberola, Unitat de Farmacologia Veterinària and LeishLAB–Servei d’Anàlisi de Fàrmacs, Departament de Farmacologia, de Terapèutica i de Toxicologia, Universitat Autònoma de Barcelona, Bellaterra (Barcelona), Spain. E-mail: Jordi.Alberola@uab.cat

Financial Support: This work was financed in part by Projects BIO2004-03893 and AGL2008-00748, both for Jordi Alberola from the Spanish Government.

Authors' addresses: Alhelí Rodríguez-Cortés, Ana Ojeda, and Jordi Alberola, Unitat de Farmacologia Veterinària and LeishLAB–Servei d’Anàlisi de Fàrmacs, Departament de Farmacologia, de Terapèutica i de Toxicologia, Universitat Autònoma de Barcelona, Bellaterra (Barcelona), Spain, E-mails: Alheli.Rodriguez@uab.cat, Ana.Ojeda@uab.cat, and Jordi.Alberola@uab.cat. Olga Francino, Servei Veterinari de Genètica Molecular, Departament de Ciència Animal i dels Aliments, Facultat de Veterinària, Universitat Autònoma de Barcelona, Bellaterra (Barcelona), Spain, E-mail: Olga.Francino@uab.cat. Laura López-Fuertes, Innoqua, Toxicology Consultants, Barcelona, Spain, E-mail: llopezfuertes@gmail.com. Marcos Timón, Agencia Española del Medicamento, Madrid, Spain, E-mail: mtimon@agemed.es.

Reprint requests: Jordi Alberola, Unitat de Farmacologia Veterinària and LeishLAB–Servei d’Anàlisi de Fàrmacs, Departament de Farmacologia, de Terapèutica i de Toxicologia, Universitat Autònoma de Barcelona, Bellaterra (Barcelona), Spain, E-mail: Jordi.Alberola@uab.cat.

Save