INTRODUCTION
Dengue has a high impact on global health; it is endemic in approximately 130 tropical and subtropical countries. An annual occurrence of 390 million infections caused by the dengue virus has been estimated, with close to one million severe cases and 9,221 related deaths.1 This disease can constitute a public health emergency at the international level because of its rapid spread, the increase in the incidence, and the intensity of outbreaks with high human and economic costs.2 However, its diagnosis is difficult because of the unspecific nature of its clinical manifestations, which makes it almost indistinguishable from other febrile diseases, and the cross reactivity of diagnostic tests due to the co-circulation of other flaviviruses such as yellow fever, Japanese encephalitis virus, West Nile virus, and Zika. Inaccurate or delayed detection of dengue cases is associated with subsequent increased severity and mortality.3–5
Currently, the WHO classification of dengue with and without warning signs and severe dengue is available in clinical practice.6 The diagnostic capability of this classification has been evaluated in several retrospective and prospective studies showing high sensitivity (87–95%) but very low specificity (6–20%).7–11 The latter is partly explained by this classification having been developed for prompt identification of dengue complications and hence to inform treatment decisions but not for differential diagnosis.12 Several technologies are available for laboratory-based diagnosis of dengue; however, their sensitivity is affected by host and viral factors (such as days of fever, history of dengue, and viral serotype), and some may not be suitable for resource-limited settings.13 Reverse transcription polymerase chain reaction (RT-PCR) can confirm dengue in the first days of fever, but it takes several hours to yield results and requires specialized infrastructure. Enzyme-linked inmunosorbent assay (ELISA) tests that detect nonstructural protein 1 (NS1) of the virus or specific immunoglobulins IgM and IgG are less technically demanding than molecular methods, but the results still take a couple of hours and require equipment and highly trained personnel.14 Different point-of-care rapid tests that detect NS1 and/or IgM and IgG are commercially available, but they show variable sensitivities with up to 50% false negative results.15–18 All these factors limit the availability of accurate dengue laboratory diagnostic methods for use in both the clinical and public health contexts.5
Until now, different clinical scores or algorithms have been developed to diagnose dengue, but they have not completed the established validation processes that allow their adoption in routine medical practice.19–33 These clinical classification tools have been developed by researchers from different endemic countries where the availability of information of the affected population is an advantage. In addition, there exists the technological capacity and human resources to develop mHealth-type devices that facilitate the application of clinical algorithms. Colombia has more than 752 municipalities with endemic dengue transmission.34 Four dengue serotypes circulate, and the country has been affected by Zika and chikungunya epidemics. A network of dengue researchers (AEDES Network) was established in 2008 to conduct clinical and epidemiological studies and laboratory-based research aiming to contribute to sustainable arbovirus control in Colombia (http://www.redaedes.org). This is an appropriate context to address the problem of diagnosis of dengue with local research and development (R&D) capacities; therefore, this study aimed to develop and prospectively validate in endemic areas dengue diagnostic clinical algorithms.
METHODS
Study design.
The study was carried out in two phases. In the first phase, diagnostic clinical dengue algorithms were developed using existing data. Algorithms with a sensitivity > 70% in the first phase were considered suitable for prospective validation during the second phase (Figure 1). For the prospective validation, a quasi-experimental (non-randomized) single-group trial was conducted.35–37 An adaptive Bayesian design was used, which included intermediate analyses of the performance of the clinical algorithms and adjustments of these algorithms. Decision rules to either implement the planned adjustments or stop recruitment were pre-established. Intermediate analyses were performed to assess the sensitivity of each clinical algorithm, if it was < 70%, the algorithm was adjusted. The adjustments to the algorithms consisted in changing the variables and/or of their cutoff values. The stopping criteria were to reach a sensitivity of 95% and a specificity between 70% and 95%. These targets were based on an expected higher sensitivity of the clinical algorithms than that of the 2009 WHO dengue classification and an expected range of false positives. As the algorithms were under validation, they did not contribute to management decisions on study participants.
Study design for the development and prospective validation of dengue diagnostic clinical algorithms.
Citation: The American Journal of Tropical Medicine and Hygiene 102, 6; 10.4269/ajtmh.19-0722
Study design for the development and prospective validation of dengue diagnostic clinical algorithms.
Citation: The American Journal of Tropical Medicine and Hygiene 102, 6; 10.4269/ajtmh.19-0722
Study design for the development and prospective validation of dengue diagnostic clinical algorithms.
Citation: The American Journal of Tropical Medicine and Hygiene 102, 6; 10.4269/ajtmh.19-0722
Ethical approval.
This study was conducted according to the principles of the Declaration of Helsinki and the National Ethical Framework Resolution 8430 of 1993.38 It was reviewed and approved by the Human Ethical Review Institutional Committee of Universidad del Valle (No.144-016), Ethics Committee in Scientific Research of Universidad Industrial de Santander (No. 03989), and COMFANDI. Written informed consent from adult patients, informed consent from parents, and informed assent from children were obtained for all participants during the second phase. The trial was registered in ClinicalTrials.gov with number NCT04063774.
Development of diagnostic clinical algorithms.
For the development of clinical diagnostic algorithms, we used a multicenter cohort database provided by the AEDES Network in Colombia. This cohort study was conducted in healthcare institutions in four dengue endemic areas in Colombia between 2003 and 2011. Briefly, it recruited individuals aged between 5 and 86 years, with acute febrile syndrome of unknown origin of less than 96 hours of onset, and followed them until symptom resolution or death.39,40 Subjects were examined by dengue-trained physicians and underwent gold standard dengue diagnostic tests performed at research virology labs in Colombia. These diagnostics included a combination of in-house dengue IgM capture ELISA,41 Panbio Dengue IgM and IgG capture assays (Panbio®, Standard Diagnostics, Inc.), in-house nested RT-PCR based on the protocol of Lanciotti et al.,42 ELISA test for the capture of the antigen NS1 of dengue (Panbio, Standard Diagnostics, Inc., Suwon, South Korea), and the hemagglutination inhibition (HI) test.43 A case of dengue was defined by seroconversion (from negative to positive) or quadrupling the antibody titer in paired sera, positive RT-PCR, positive NS1, or HI titers ≥ 1:2,560. In the database, 1,130 met these criteria and were classified as dengue cases and the remaining 918 as non-dengue.
To identify key variables to be included in the diagnostic algorithms, we reviewed scientific articles, elicited expert opinions from Colombian clinicians and dengue scientists, and conducted a bivariate analysis of the database. From this initial step, 25 signs and symptoms were selected as informative for the construction of diagnostic algorithms. Similar signs and symptoms were grouped into categories to reduce the number of variables to a maximum of 12 considered as manageable in practice. The presence of at least one sign or symptom within each category indicative of dengue was coded as 1 and the absence of all as 0. Conversely, in the categories considered as indicative of non-dengue, the absence of signs and symptoms was coded as 1 and the presence as 0. Similarly, laboratory-based variables were explored, from which leukocyte and platelet counts were selected. Cutoff values for leukocyte and platelet counts were defined by expert consultation and by receiver operating characteristic (ROC) curve analysis. Dengue-related cutoff values defined by experts were leukocytes ≤ 4,500/mm3 and platelets ≤ 160,000/mm3, and values defined by ROC curve analysis were leukocytes ≤ 4,200/mm3 (sensitivity: 75.5%; specificity: 68.3%) and platelets ≤ 165,000/mm3 (sensitivity: 68.2%; specificity: 63.4%). Values less than or equal to the established cutoff were coded as 1 and those above as 0.
Clinical algorithms were constructed using the Bayes formula for both the discrete and continuous cases. Vectors of all possible combinations of categories of signs and symptoms were defined for all individuals in the database. Then, the predictive probability of each vector was calculated using the Bayes formula for events (discrete case), assuming conditional independence,44–47 and for continuous random variables (continuous case). The leukocyte and platelet counts were dichotomized using the defined cutoff values and then the binary variables were incorporated into the calculation of predictive probabilities, this time considering the sequential nature of Bayes’ theorem.48 Nonparametric ROC curves and Youden indexes were estimated with the predictive probabilities for all vectors and used to identify the cutoff value for each diagnostic algorithm that most accurately classified individuals as dengue or non-dengue.49 These cutoff values were applied to all subjects in the database to classify them as dengue or non-dengue, according to whether or not the corresponding predictive probability was at least as high as the established cutoff value for each algorithm.
Prospective validation of diagnostic clinical algorithms.
The prospective validation of the diagnostic algorithms was carried out between December 2016 and July 2018 in three dengue endemic areas in Colombia; Cali is located in the southwest of the country, whereas Yopal and Piedecuesta are located in the east-central region. Individuals of all ages who sought medical care because of fever of less than 15 days at primary or secondary level (Cali and Piedecuesta), and tertiary level (Yopal) of care institutions were included. Those in whom the treating physician clearly identified the cause of fever were excluded (e.g., urinary tract infection, skin infection, and pneumonia). Subjects were recruited consecutively on admission to the health services to reduce spectrum bias. Trained study physicians examined all participants, registered clinical variables, and applied the diagnostic algorithm to all participants using an app installed in a mobile device designed for this purpose (named “calculadora dengue” in Spanish or “dengue calculator” in English). A maximum sample size of 2,000 subjects was estimated based on an expected 95% sensitivity of diagnostic algorithms, 3% error, 95% CI, and 10% dengue prevalence.6,50 For the intermediate analyses, the sample size was estimated by modeling Bayesian effective sample size scenarios.51 Five intermediate analyses were performed with sample sizes of 177, 352, 530, 711, and 893, respectively. The final analysis included 1,039 observations. The adjustments consisted of dropping or adding clinical variables and new cutoff values for leukocyte and platelet counts according to the prior intermediate analysis or during updated literature reviews. For hemogram results, monocytosis (≥ 1,000/mm3), lymphocyte/neutrophil ratio (≥ 0.8), and new cutoff values for leukocytes (≤ 4,600/mm3) and platelets (≤ 186,000/mm3) were defined.
Laboratory analysis.
Blood samples were drawn from participants in the acute phase of disease after being examined by the study physicians (within 14 days of symptoms onset) and during the convalescent phase of disease (14–29 days after symptoms onset), and then sent refrigerated to the virology laboratory at Universidad del Valle in Cali, where dengue diagnostics were carried out by experienced personnel blinded to participants’ characteristics and clinical algorithm results. Part of the acute-phase venous blood sample was used for a complete hemogram test performed in automatized equipment in the corresponding healthcare institution. Dengue diagnosis was confirmed on the basis of one or more of the following criteria: 1) positive result of RT-PCR following an in-house method based on the protocol of Lanciotti et al.,42 2) positive dengue NS1 ELISA (Panbio, Standard Diagnostics, Inc.), or 3) seroconversion in dengue-specific IgM (Panbio, Standard Diagnostics, Inc.) or IgG (Panbio®, Standard Diagnostics, Inc.) antibodies from the acute phase to convalescent phase. An individual with positive results of the IgM or IgG ELISA in both the acute and convalescent phases, or with positive results in the acute phase but without convalescent phase results, was classified as a probable case of dengue. An individual with negative results in all the acute tests and without sample in the convalescent period was considered as undetermined diagnosis. An individual with negative results in all tests both at acute and convalescent phases, or did not meet the aforementioned criteria was classified as non-dengue. Equivocal results were repeated once and, if equivocal again, were not considered for the classification of the case. Primary and secondary infections were defined by a negative and positive IgG (Panbio Dengue IgG Capture ELISA assay) in the acute sample, respectively.
Statistical analysis.
A descriptive analysis of dengue and non-dengue cases in both the cohort database and trial participants was performed, followed by a bivariate analysis using Fisher’s exact test52 for categorical variables and the Wilcoxon rank sum test53 for continuous variables. During the first phase (development of diagnostic algorithms), the Bayes formula was applied44 in the discrete case using a dengue prevalence of 10%, whereas in the continuous case, a uniform (0,1) distribution of probability was used.46 For the calculation, 1,000 probability values were simulated, and the average predictive probability for each vector was obtained. Because of the relatively large sample size in the database, the reported final sensitivities and specificities with corresponding 95% CIs were estimated using the maximum likelihood method with sensitivity = true positive (TP)/(TP + false negative [FN]) and specificity = true negative (TN)/(TN + false positive [FP]) where TP, FN, TN, and FP are numbers of true positives, false negatives, true negatives, and false positives, respectively. Subjects with laboratory-confirmed dengue in whom the clinical diagnostic algorithm had a predictive probability greater or equal than the estimated cutoff value were TPs and those with probability less than the estimated cutoff value were FNs. Subjects classified as non-dengue in whom the clinical diagnostic algorithm had a predictive probability greater or equal than the estimated cutoff value were FPs and those with probability less than the estimated cutoff value were TNs.
During the second phase, sensitivity and specificity were estimated assuming that both were random variables with uniform (0,1) distribution, and the posterior estimates were used to obtain their corresponding 95% credible intervals. The prior distributions of the algorithms were not updated to be consistent with those obtained during the development stage. The estimates were obtained with a custom web application written using the R package “Shiny” in R (Rstudy, Boston, MA).54 The script is available in an online R repository under the name of “Bayesian classifier for discrete data using the beta distribution (BetaBsClassifier).”55 These estimates of sensitivity and specificity were used to calculate positive predictive values (PPVs) and negative predictive values (NPVs) and positive likelihood ratio (LR+) and negative likelihood ratio (LR−) as follows: PPV = (prevalence × sensitivity)/(prevalence × sensitivity + [1−specificity] × [1−prevalence]), NPV = (1−prevalence) × specificity/([1−prevalence] × specificity) + ([1−sensitivity] × prevalence]), LR+ = sensitivity/(1−specificity), LR− = (1−sensitivity)/specificity. The credible interval of each performance measure was estimated from 1,000 draws from its posterior distribution. For these calculations, indeterminate diagnostic results and missing data were excluded. All other analyses were performed in STATA 11 (StataCorp LLC, College Station, TX) and R.56,57 For the algorithm with the highest sensitivity, sources of heterogeneity potentially influencing its sensitivity were explored. Such sources included study location, age of participants (< 5, ≥ 5 years old), level of care (primary/secondary or tertiary), days of symptoms on inclusion (≤ 3, 4–5, ≥ 6 days), disease severity (severe or non-severe dengue), dengue serotype (DENV-1 or 2), and type of infection (primary or secondary).
RESULTS
Development of diagnostic clinical algorithms.
In the literature review and expert consultation, 23 signs and symptoms related to dengue were identified: arthralgia, headache, retroocular pain, myalgia, absence or presence of diarrhea, abdominal pain, hepatomegaly, vomiting, edema, chills, hyporexia, facial erythema, rash, hemorrhages, petechiae, absence of jaundice, irritability, insomnia, drowsiness, absence of rhinorrhea, absence of odynophagia, conjunctival hyperemia, and absence of cough. In addition to these, pruritus, altered consciousness, and any sign of bleeding (gingivorrhagia, epistaxis, hematemesis, and petechiae) were identified during analysis of the cohort database. Specifically, the cohort database had 1,130 dengue and 918 non-dengue cases, both with a median of 4 days of fever on admission. A significantly higher proportion of skin rash, bleeding signs, edema, hepatomegaly, and altered consciousness were found in dengue than non-dengue cases. Likewise, statistically significant differences were found in the median of the leukocyte and platelet counts, being both lower in the dengue group. Conversely, median of age, headache, hyporexia, sore throat, and rhinorrhea were more frequent in non-dengue than dengue cases (Table 1).
Bivariate analyses of the cohort database used for the development of clinical diagnostic algorithms
| Characteristics | Dengue n = 1,130 | Non-dengue n = 918 | P-value |
|---|---|---|---|
| Median years of age (range) | 16 (1–86) | 18 (1–81) | < 0.0001 |
| Median days of fever (range) | 4 (1–9) | 4 (1–11) | 1 |
| Headache (%) | 940 (83.2) | 835 (91.0) | < 0.0001 |
| Chills (%) | 935 (82.7) | 811 (88.3) | < 0.0001 |
| Hyporexia (%) | 602 (53.3) | 634 (69.1) | < 0.0001 |
| Sore throat (%) | 290 (25.7) | 387 (42.2) | < 0.0001 |
| Rhinorrhea (%) | 290 (25.7) | 454 (49.5) | < 0.0001 |
| Skin rash (%) | 406 (35.9) | 229 (25.0) | < 0.0001 |
| Pruritus (%) | 344 (30.4) | 211 (23.0) | < 0.0001 |
| Vomiting (%) | 490 (43.4) | 298 (32.5) | < 0.0001 |
| Altered consciousness (%) | 460 (40.7) | 175 (19.1) | < 0.0001 |
| Palpebral edema (%) | 227 (20.1) | 99 (10.8) | < 0.0001 |
| Hepatomegaly (%) | 174 (15.4) | 68 (7.4) | < 0.0001 |
| Bleeding signs (%) | 703 (62.2) | 476 (51.8) | < 0.0001 |
| Median leukocytes/mm3 (range) | 3,400 (1,100–45,400) | 4,800 (1,500–17,000) | < 0.0001 |
| Median platelets/mm3 (range) | 131,000 (11,500–734,000) | 199,500 (22,000–474,000) | < 0.0001 |
A total of 12 diagnostic algorithms were developed (Supplemental Table 1) from which, eight showed sensitivities above 70%. Three algorithms which included leukocytes and platelet counts and were developed using the continuous Bayes formula showed sensitivities above 80%. The algorithm of eight categories of symptoms and cutoff values of leukocytes ≤ 4,200/mm3 and platelets ≤ 165,000/mm3 reached the highest sensitivity (85.8%, 95% CI: 82.0, 89.5), followed by the algorithm of 12 symptoms plus the same cutoff values for leukocytes and platelets (81.3%, 95% CI: 78.3, 84.2). The latter also reached the highest specificity (88.2%, 95% CI: 85.6, 90.7) (Table 2, Supplemental Table 2).
Performance of developed dengue clinical diagnostic algorithms in the database cohort
| Dengue calculator | Sensitivity % (95% CI) | Specificity % (95% CI) |
|---|---|---|
| A8C | 72.2 (68.7, 75.6) | 60.3 (54.9, 65.6) |
| A8C_Y | 85.8 (82.0, 89.5) | 52.0 (43.2, 60.7) |
| A8C_E | 71.9 (67.2, 76.6) | 69.5 (61.5, 77.5) |
| A12D_Y | 76.0 (72.8, 79.3) | 72.4 (68.8, 75.8) |
| A12D_E | 74.5 (71.2, 77.8) | 69.5 (65.9, 73.1) |
| A12C | 75.7 (73.1, 78.2) | 81.6 (79.0, 84.1) |
| A12C_Y | 81.3 (78.3, 84.2) | 88.2 (85.6, 90.7) |
| A12C_E | 80.9 (77.9, 83.9) | 86.0 (83.3, 88.7) |
A8 = dengue calculator—eight categories of symptoms; A12 = dengue calculator—12 symptoms; D = discrete Bayes formula; C = continuous Bayes formula; E = leukocytes ≤ 4,500/mm3 or platelets ≤ 160,000/mm3; Y = leukocytes ≤ 4,200/mm3 and platelets ≤ 165,000/mm3.
Prospective validation of diagnostic clinical algorithms.
From December 2016 to July 2018, a total of 1,039 febrile subjects were included for the prospective validation of the diagnostic algorithms, of whom 25 were laboratory-confirmed dengue, 307 were classified as non-dengue, 514 as probable dengue, and 193 as undetermined (Figure 2). Among confirmed dengue cases, there were 12 primary and 13 secondary infections; eight were dengue without warning signs, 12 with warning signs, and five were severe. Dengue serotypes were identified in 13 cases (eight DENV-1 and five DENV-2). The trial ended because of the completion of the AEDES Network–funded program.
Study population selection and laboratory analyses in prospective validation. (In this study, Figure 2 was constructed considering consolidated standards of reporting trials (CONSORT) and standards for reporting of diagnostic accuracy studies (STARD), because it was a clinical trial of diagnostic tests. However, we want to highlight that there were several prototypes of clinical diagnostic algorithms of dengue under test and not a single “index test.”)
Citation: The American Journal of Tropical Medicine and Hygiene 102, 6; 10.4269/ajtmh.19-0722
Study population selection and laboratory analyses in prospective validation. (In this study, Figure 2 was constructed considering consolidated standards of reporting trials (CONSORT) and standards for reporting of diagnostic accuracy studies (STARD), because it was a clinical trial of diagnostic tests. However, we want to highlight that there were several prototypes of clinical diagnostic algorithms of dengue under test and not a single “index test.”)
Citation: The American Journal of Tropical Medicine and Hygiene 102, 6; 10.4269/ajtmh.19-0722
Study population selection and laboratory analyses in prospective validation. (In this study, Figure 2 was constructed considering consolidated standards of reporting trials (CONSORT) and standards for reporting of diagnostic accuracy studies (STARD), because it was a clinical trial of diagnostic tests. However, we want to highlight that there were several prototypes of clinical diagnostic algorithms of dengue under test and not a single “index test.”)
Citation: The American Journal of Tropical Medicine and Hygiene 102, 6; 10.4269/ajtmh.19-0722
Considering the 332 laboratory-confirmed dengue and non-dengue cases, the majority (> 80%) were > 15 years old. However, the dengue cases tended to have longer duration of fever on admission (4 versus 2 days) and to attend higher level facilities than non-dengue cases. Hyporexia, rash, signs of bleeding, altered consciousness, conjunctival hyperemia, leukocytes count ≤ 4,500/mm3, and platelet count ≤ 165,000/mm3 were statistically significantly more frequent in dengue cases. Retroocular pain, proximal arthralgias, and persistent vomiting were also more frequent in the dengue group but did not reach statistical significance. Only rhinorrhea and non-colic abdominal pain were more frequent in the non-dengue group but without statistically significant differences (Table 3).
Clinical characteristics of confirmed dengue and non-dengue cases included in the prospective validation of diagnostic clinical algorithms
| Characteristics | Dengue, n = 25 (%) | Non-dengue, n = 307 (%) | P-value |
|---|---|---|---|
| Gender | |||
| Female | 14 (56) | 166 (54.1) | 0.8 |
| Male | 11 (44) | 141 (45.9) | |
| Age (years) | |||
| < 5 | 4 (16) | 27 (8.8) | 0.2 |
| 5 and 15 | 0 | 25 (8.1) | |
| > 15 | 21 (84) | 255 (83.1) | |
| Level of care | |||
| Primary/secondary | 14 (56) | 288 (93.8) | 0.03 |
| Tertiary | 11 (44) | 19 (6.2) | |
| Median days of fever (range) | 4 (1–10) | 2 (1–13) | < 0.0001 |
| Signs and symptoms | |||
| Chills | 22 (88) | 265 (86.3) | 1 |
| Hyporexia | 25 (100) | 236 (76.9) | 0.004 |
| Headache | 22 (88) | 270 (87.9) | 1 |
| Retroocular pain | 12 (48) | 94 (30.6) | 0.07 |
| Myalgias | 21 (84) | 239 (77.8) | 0.61 |
| Proximal arthralgias | 14 (56) | 117 (38.1) | 0.07 |
| Persistent vomiting | 3 (12) | 11 (3.6) | 0.08 |
| Non-colic abdominal pain | 9 (36) | 159 (51.8) | 0.13 |
| Hepatomegaly | 1 (4) | 3 (1) | 0.27 |
| Skin rash | 7 (28) | 31 (10.1) | 0.007 |
| Pruritus | 4 (16) | 24 (7.8) | 0.15 |
| Rhinorrhea | 10 (40) | 180 (58.6) | 0.07 |
| Sore throat | 11 (44) | 152 (49.5) | 0.6 |
| Conjunctival hyperemia | 11 (44) | 70 (22.8) | 0.02 |
| Jaundice | 0 | 1 (0.3) | 1 |
| Edema | 0 | 2 (0.6) | 1 |
| Bleeding signs | 8 (32) | 37 (12) | 0.005 |
| Altered consciousness | 8 (32) | 46 (15) | 0.03 |
| Hemogram | |||
| Median leukocytes/mm3 (range) | 7,000 (1,100–15,860) | 7,180 (2,220–32,110) | 0.3 |
| Leukocytes ≤ 4,500/mm3 | 9 (36) | 40 (13) | 0.002 |
| Median platelets/mm3 (range) | 198,000 (29,000–414,000) | 238,000 (49,000–668,000) | < 0.0001 |
| Platelets ≤ 165,000/mm3 | 11 (44) | 42 (13.7) | < 0.0001 |
The first intermediate analysis was performed when a total of 177 participants were included of whom five were laboratory-confirmed dengue, 35 were non-dengue, 112 were probable, and 25 were undetermined. In this analysis, the highest sensitivity and specificity were 43% and 84%, respectively, with the algorithm that included 12 symptoms (headache, chills, hyporexia, odynophagia, rash, pruritus, vomiting, altered consciousness, edema, hepatomegaly, rhinorrhea, and bleeding signs) plus cutoff values of leukocytes ≤ 4,200/mm3 and platelets ≤ 165,000/mm3, both by the discrete and continuous Bayes formula (Table 4). Consequently, the clinical algorithms were adjusted, and their performance assessed in the next intermediate analyses. The adjustments included modified cutoff values of leukocyte and platelet counts and replacing symptoms (Supplemental Table 3). During the subsequent intermediate analyses, likelihood ratios were all close to 1, and the highest sensitivity (60.2%) was observed with the algorithm that included 12 symptoms (headache, chills, hyporexia, odynophagia, rash, pruritus, vomiting, altered consciousness, rhinorrhea, bleeding signs, arthralgia, and monocytes ≥ 1,000/mm3), leukocyte count ≤ 4,600/mm3, platelet count ≤ 186,000/mm3, and used the continuous Bayes formula. However, the specificity decreased to 37% (Table 4).
Performance of clinical diagnostic algorithms during intermediate and final analyses in the prospective field validation
| Analysis | Dengue calculator | Measure of diagnostic performance | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| True positive | False negative | True negative | False positive | Sensitivity % (95% CI) | Specificity % (95% CI) | Positive predictive value % (95% CI) | Negative predictive value % (95% CI) | Positive likelihood ratio (95% CI) | Negative likelihood ratio (95% CI) | ||
| Intermediate | |||||||||||
| 1 | A12D_Y | 2 | 3 | 30 | 5 | 43.0 (30.0, 55.1) | 84.0 (80.0, 88.2) | 28.0 (0.0, 53.8) | 91.0 (88.2, 97.6) | 2.7 (0.0, 0.5) | 0.7 (0.2, 1.2) |
| A12D_E | 2 | 3 | 28 | 7 | 43.0 (30.0, 55.1) | 78.0 (74.1, 83.2) | 22.0 (0.0, 43.8) | 91.0 (87.4, 97.5) | 2.0 (0.0, 7.0) | 0.7 (0.2, 1.3) | |
| A8C_Y | 2 | 3 | 28 | 7 | 43.0 (30.0, 55.1) | 78.0 (74.1, 83.2) | 22.0 (0.0, 43.8) | 91.0 (87.4, 97.5) | 2.00 (0.0, 7.0) | 0.7 (0.2, 1.3) | |
| A12C_Y | 2 | 3 | 30 | 5 | 43.0 (30.0, 55.1) | 84.0 (80.0, 88.2) | 28.0 (0.0, 53.8) | 91.0 (88.2, 97.6) | 2.65 (0.0, 0.5) | 0.7 (0.2, 1.2) | |
| 2 | A12C-Adj_1 | 3 | 4 | 56 | 21 | 45.0 (33.0, 83.2) | 72.0 (68.9, 75.7) | 13.0 (4.6, 27.3) | 93.0 (87.6, 96.4) | 1.6 (0.4, 3.4) | 0.8 (0.3, 1.3) |
| A12C-Adj_2 | 3 | 4 | 61 | 16 | 44.5 (33.0, 83.2) | 78.4 (75.6, 81.8) | 15.8 (5.8, 33.7) | 94.0 (88.6, 97.9) | 2.1 (0.6, 4.6) | 0.7 (0.2, 1.2) | |
| 3 | A12C-Adj_1 | 4 | 4 | 87 | 48 | 50.3 (39.0, 60.8) | 64.2 (61.5, 67.1) | 7.7 (3.9, 21.5) | 92.0 (85.9, 95.8) | 1.4 (0.4, 2.5) | 0.8 (0.3, 1.4) |
| 4 | A12C-Adj_2 | 7 | 6 | 130 | 66 | 53.6 (44.6, 62.1) | 66.2 (66.0, 68.5) | 9.5 (8.3, 22.4) | 95.6 (89.1, 97.4) | 1.6 (0.8, 2.6) | 0.7 (0.2, 1.1) |
| 5 | A12C-Adj_1 | 11 | 8 | 179 | 99 | 57.2 (49.5, 64.6) | 64.4 (62.4, 66.2) | 15.4 (9.6, 20.8) | 95.6 (90.0, 96.6) | 1.6 (1.0, 2.4) | 0.7 (0.3, 1.0) |
| A12C-Adj_5* | 11 | 7 | 117 | 154 | 60.0 (52.7, 67.4) | 43.2 (41.2, 45.3) | 10.6 (6.8, 14.0) | 94.2 (85.4, 95.8) | 1.0 (0.6, 1.5) | 0.9 (0.4, 1.5) | |
| A12C-Adj_7 | 11 | 7 | 100 | 171 | 60.2 (52.7, 67.4) | 37.0 (35.0, 38.9) | 9.6 (5.9, 12.9) | 93.3 (83.9, 95.1) | 1.0 (0.6, 1.3) | 1.08 (0.5, 1.7) | |
| Final | A12C-Adj_1 | 13 | 12 | 200 | 107 | 51.7 (32.0, 68.0) | 65.2 (59.9, 70.4) | 10.8 (6.8, 14.4) | 94.3 (92.1, 96.2) | 1.5 (0.9, 2.1) | 0.7 (0.5, 1.0) |
| A12C-Adj_5* | 16 | 8 | 129 | 171 | 65.4 (45.8, 83.3) | 43.0 (37.7, 48.7) | 8.4 (6.2, 10.8) | 94.0 (90.8, 97.1) | 1.1 (0.8, 1.5) | 0.8 (0.4, 1.3) | |
| A12C-Adj_7 | 16 | 8 | 112 | 188 | 65.4 (45.8, 83.3) | 37.4 (31.3, 43.0) | 7.7 (5.5, 10.0) | 93.1 (89.4, 96.8) | 1.0 (0.7, 1.4) | 0.9 (0.4, 1.5) | |
| A12C-Adj_9* | 14 | 10 | 136 | 164 | 58.0 (37.5, 79.2) | 45.4 (39.7, 51.3) | 7.4 (5.2, 10.3) | 93.1 (90.1, 96.2) | 1.1 (0.7, 1.4) | 0.9 (0.5, 1.4) | |
| A12C-Adj_10* | 13 | 11 | 130 | 170 | 53.7 (33.3, 70.8) | 43.4 (38.0, 49.3) | 7.1 (4.4, 4.5) | 92.1 (88.8, 95.2) | 1.0 (0.6, 1.3) | 1.1 (0.6, 1.6) | |
| A12C-Adj_11* | 14 | 10 | 135 | 165 | 57.6 (37.5, 79.2) | 45.1 (39.3, 50.3) | 7.8 (5.4, 10.2) | 93.0 (90.2, 96.0) | 1.0 (0.7, 1.4) | 0.94 (0.5, 1.4) | |
| A12C-Adj_12* | 16 | 8 | 120 | 180 | 65.4 (50.0, 83.3) | 40.1 (34.7, 45.7) | 8.0 (6.0, 10.2) | 94.0 (90.2, 96.8) | 1.1 (0.8, 1.4) | 0.9 (0.4, 1.4) | |
A8C = dengue calculator—eight categories of symptoms based on continuous Bayes formula; A12C = dengue calculator—12 symptoms based on continuous Bayes formula; A12D = dengue calculator—12 symptoms based on discrete Bayes formula; Adj_1 = leukocytes ≤ 4,600/mm3 and platelets ≤ 186,000/mm3; Adj_2 = leukocytes ≤ 4,600/mm3 or platelets ≤ 186,000/mm3; Adj_5 = replace edema by arthralgias + hepatomegaly by lymphocytes/neutrophils ratio ≥ 0.8 + leukocytes ≤ 4,600/mm3 and platelets ≤ 186,000/mm3; Adj_7 = replace edema by arthralgias + hepatomegaly by monocytes ≥ 1,000/mm3 + leukocytes ≤ 4,600/mm3 and platelets ≤ 186,000/mm3; Adj_9 = replace edema by arthralgias + hepatomegaly by lymphocytes/neutrophils ratio ≥ 0.8 + replace sore throat by retroocular pain + change rhinorrhea by absence rhinorrhea + leukocytes ≤ 4,600/mm3 and platelets ≤ 186,000/mm3; Adj_10 = replace edema by arthralgias + hepatomegaly by lymphocytes/neutrophils ratio ≥ 0.8 + replace sore throat by retroocular pain + change rhinorrhea by absence rhinorrhea + leukocytes ≤ 4,600/mm3 or platelets ≤ 186,000/mm3; Adj_11 = replace edema by arthralgias +hepatomegaly by lymphocytes/neutrophils ratio ≥ 0.8 + replace sore throat by retroocular pain + leukocytes ≤ 4,600/mm3 and platelets ≤ 186,000/mm3; Adj_12 = replace edema by arthralgias + hepatomegaly by lymphocytes/neutrophils ratio ≥ 0.8 + replace sore throat by retroocular pain + leukocytes ≤ 4,600/mm3 and platelets ≤ 186,000/mm3; CI = credible interval; E = leukocytes ≤ 4,500/mm3 and platelets ≤ 160,000/mm3; Y: leukocytes ≤ 4,200/mm3 and platelets ≤ 165,000/mm3.
One confirmed dengue and seven non-dengue cases had missing lymphocyte count data.
In the final analysis, the highest sensitivity (65.4%) and specificity (43%) was observed with the algorithm that included 12 symptoms (headache, chills, hyporexia, odynophagia, rash, pruritus, vomiting, altered consciousness, rhinorrhea, bleeding signs, arthralgia, and lymphocytes/neutrophils ratio ≥ 0.8), plus cutoff values of leukocytes ≤ 4,600/mm3 and platelets ≤ 186,000/mm3, and the continuous Bayes theorem. The highest specificity (65.2%) was observed with the algorithm that included 12 symptoms (headache, chills, hyporexia, odynophagia, rash, pruritus, vomiting, altered consciousness, edema, hepatomegaly, rhinorrhea, and bleeding signs), plus cutoff values of leukocytes ≤ 4,600/mm3 and platelets ≤ 186,000/mm3, and the continuous Bayes theorem; but its sensitivity was 51.7%. In all cases, the likelihood ratios were close to 1 (Table 4). When probable dengue cases were added to laboratory-confirmed dengue cases (n = 539), sensitivity decreased for all algorithms, and the highest sensitivity (56.4%) and specificity (40%) were observed with the algorithm that included 12 symptoms plus cutoff values of leukocytes ≤ 4,600/mm3 and platelets ≤ 186,000/mm3, named dengue calculator AC12-Adj_12 (Table 5).
Performance of diagnostic clinical algorithms during prospective validation including probable dengue cases as dengue.
| Dengue calculator | Measure of diagnostic performance | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| True positive | False negative | True negative | False positive | Sensitivity% (95% CI) | Specificity % (95% CI) | Positive predictive value % (95% CI) | Negative predictive value % (95% CI) | Positive likelihood ratio (95% CI) | Negative likelihood ratio (95% CI) | |
| A12C-Adj_1 | 170 | 369 | 200 | 107 | 31.6 (27.6, 35.2) | 65.1 (59.3, 70.4) | 61.4 (57.1, 66.2) | 35.1 (32.6, 37.0) | 0.9 (0.8, 1.1) | 1.0 (0.9, 1.2) |
| A12C-Adj_5* | 287 | 246 | 129 | 171 | 53.8 (50.0, 58.0) | 43.0 (37.3, 49.0) | 62.7 (59.8, 65.8) | 34.4 (31.0, 38.2) | 0.9 (0.8, 1.1) | 1.1 (0.9, 1.3) |
| A12C-Adj_7 | 298 | 235 | 112 | 188 | 55.9 (51.7, 60.2) | 37.4 (32.0, 42.7) | 61.4 (58.8, 63.9) | 32.3 (28.6, 35.9) | 0.9 (0.8, 1.0) | 1.2 (1.0, 1.4) |
| A12C-Adj_9* | 264 | 269 | 136 | 164 | 50.0 (45.2, 53.3) | 45.3 (39.3, 51.0) | 61.7 (58.3, 64.7) | 33.6 (29.9, 36.7) | 0.9 (0.8, 1.0) | 1.1 (1.0, 1.3) |
| A12C-Adj_10* | 265 | 268 | 135 | 165 | 50.0 (45.2, 54.3) | 45.0 (39.3, 51.0) | 61.6 (58.4, 65.2) | 33.5 (30.0, 37.2) | 0.9 (0.8, 1.0) | 1.1 (0.9, 1.3) |
| A12C-Adj_11* | 263 | 270 | 135 | 165 | 49.4 (45.2, 53.6) | 45.0 (39.3, 51.0) | 61.5 (58.1, 64.7) | 33.4 (29.8, 36.8) | 0.9 (0.8, 1.0) | 1.1 (1.0, 1.3) |
| A12C-Adj_12* | 301 | 232 | 120 | 180 | 56.4 (52.5, 60.4) | 40.0 (34.7, 45.7) | 62.6 (59.8, 65.2) | 34.1 (30.4, 37.7) | 0.9 (0.8, 1.0) | 1.1 (0.9, 1.3) |
A12C = dengue calculator—12 symptoms based on Bayes formula continues; Adj_1 = leukocytes ≤ 4,600/mm3 and platelets ≤ 186,000/mm3; Adj_5 = replace edema by arthralgias + hepatomegaly by lymphocytes/neutrophils ratio ≥ 0.8 + leukocytes ≤ 4,600/mm3 and platelets ≤ 186,000/mm3; Adj_7 = replace edema by arthralgias + hepatomegaly by monocytes ≥ 1,000/mm3 + leukocytes ≤ 4,600/mm3 and platelets ≤ 186,000/mm3; Adj_9 = replace edema by arthralgias +hepatomegaly by lymphocytes/neutrophils ratio ≥ 0.8 + replace sore throat by retroocular pain + change rhinorrhea by absence rhinorrhea + leukocytes ≤ 4,600/mm3 and platelets ≤ 186,000/mm3; Adj_10 = replace edema by arthralgias + hepatomegaly by lymphocytes/neutrophils ratio ≥ 0.8 + replace sore throat by retroocular pain + change rhinorrhea by absence rhinorrhea + leukocytes ≤ 4,600/mm3 or platelets ≤ 186,000/mm3; Adj_11 = replace edema by arthralgias +hepatomegaly by lymphocytes/neutrophils ratio ≥ 0.8 + replace sore throat by retroocular pain + leukocytes ≤ 4,600/mm3 and platelets ≤ 186,000/mm3; Adj_12 = replace edema by arthralgias +hepatomegaly by lymphocytes/neutrophils ratio ≥ 0.8 + replace sore throat by retroocular pain + leukocytes ≤ 4,600/mm3 and platelets ≤ 186,000/mm3; CI = credible interval.
One confirmed dengue and seven non-dengue cases had missing lymphocyte count data.
A higher sensitivity of the dengue calculator A12C-Adj_12 was observed in subjects included in Yopal; who attended a tertiary level of care institution compared with primary/secondary level; with ≥ 6 days of fever on admission; < 5 years old; classified as severe cases; and with DENV-2 serotype infections (Table 6).
Performance of the dengue calculator A12C-Adj_12 according to study population characteristics.
| Characteristic | Measure of diagnostic performance | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| True positive | False negative | True negative | False positive | Sensitivity % (95% CI) | Specificity % (95% CI) | Positive predictive value % (95% CI) | Negative predictive value % (95% CI) | Positive likelihood ratio (95% CI) | Negative likelihood ratio (95% CI) | |
| Location | ||||||||||
| Yopal | 10 | 1 | 6 | 13 | 84.4 (72.7, 100) | 33.4 (15.8, 52.6) | 42.3 (34.1, 52.3) | 78.8 (57.8, 100) | 1.3 (0.9,1.9) | 0.5 (0.0, 1.3) |
| Piedecuesta | 1 | 1 | 16 | 20 | 50.0 (–) | 44.4 (27.8, 61.1) | 4.7 (0.0, 10.6) | 94.1 (86.4, 99.0) | 0.9 (0.0, 2.2) | 1.1 (0.0, 3.0) |
| Cali | 6 | 6 | 98 | 147 | 44.5 (25.0, 75.0) | 41.0 (33.9, 46.1) | 3.9 (1.9, 6.3) | 94.2 90.2, 96.9) | 0.8 (0.4, 1.3) | 1.4 (0.6, 2.1) |
| Level of care | ||||||||||
| Primary/secondary | 6 | 7 | 114 | 167 | 46.6 (15.4, 69.2) | 40.6 (34.9, 46.3) | 3.5 (1.4, 6.1) | 94.3 (90.1, 96.5) | 0.8 (0.3, 1.2) | 1.3 (0.7, 2.1) |
| Tertiary | 10 | 1 | 6 | 13 | 84.4 (72.7, 100) | 33.4 (15.8, 52.6) | 42.3 (34.1, 52.3) | 78.8 (57.8, 100) | 1.3 (0.9, 1.9) | 0.5 (0.0, 1.3) |
| Days of fever on admission | ||||||||||
| ≤ 3 | 6 | 4 | 96 | 136 | 60.0 (30.0, 90.0) | 41.4 (34.9, 48.3) | 4.2 (2.6, 7.6) | 96.0 (91.7, 98.7) | 1.0 (0.5, 1.6) | 1.0 (0.2, 1.7) |
| 4 and 5 | 4 | 3 | 18 | 33 | 57.1 (28.6, 99.9) | 35.3 (23.5, 49.0) | 10.8 (4.1, 14.6) | 85.7 (77.9, 99.9) | 0.9 (0.4, 1.5) | 1.2 (0.0, 2.5) |
| ≥ 6 | 6 | 1 | 6 | 11 | 85.7 (57.1, 99.9) | 35.3 (11.7, 58.8) | 35.3 (23.9, 45.3) | 85.7 (58.5, 99.9) | 1.3 (0.8, 2.1) | 0.4 (0.0, 1.8) |
| Age (years) | ||||||||||
| 0–4 | 4 | 0 | 7 | 20 | 99.0 (–) | 25.9 (11.1, 44.4) | 16.7 (14.4, 21.2) | 99.0 (–) | 1.4 (1.1, 1.8) | 0.0 (–) |
| ≥ 5 | 12 | 8 | 102 | 148 | 60.0 (40.0, 80.0) | 40.8 (35.2, 47.2) | 7.5 (4.5, 9.5) | 92.7 (89.4, 96.6) | 1.0 (0.6, 1.4) | 1.0 (0.5, 1.6) |
| Dengue classification | ||||||||||
| Non-severe | 12 | 7 | 119 | 177 | 63.2 (42.1, 84.2) | 40.2 (34.0, 45.0) | 6.3 (4.1, 8.2) | 94.4 (91.1, 97.4) | 1.1 (0.7, 1.4) | 0.9 (0.4, 1.5) |
| Severe | 4 | 1 | 1 | 3 | 80.0 (40.0, 99.9) | 25 (0.0, 75.0) | 57.1 (29.5, 71.5) | 50.0 (–) | 1.1 (0.5, 3.2) | 0.8 (–) |
| Dengue serotype | ||||||||||
| DENV-1 | 2 | 2 | 120 | 180 | 48.0 (–) | 40.0 (34.6, 45.7) | 4.7 (0.0, 4.9) | 96.8 (92.6, 99.9) | 0.9 (0.0, 1.7) | 1.1 (0.0, 2.6) |
| DENV-2 | 7 | 1 | 120 | 180 | 85.6 (62.5, 99.9) | 40.0 (34.3, 45.3) | 7.2 (5.1, 8.6) | 96.8 (95.1, 99.9) | 1.2 (1.0, 1.8) | 0.8 (0.0, 1.0) |
| Type of infection | ||||||||||
| Primary | 9 | 4 | 120 | 180 | 67.2 (46.0, 92.3) | 40.0 (34.6, 45.3) | 4.8 (3.5, 7.6) | 96.8 (92.8, 98.9) | 1.2 (0.7, 1.0) | 0.8 (0.2, 1.5) |
| Secondary | 7 | 4 | 120 | 180 | 63.6 (36.4, 91.0) | 40.0 (34.7, 45.7) | 3.7 (2.9, 7.5) | 96.8 (91.8–1.7) | 1.1 (0.6, 1.5) | 0.9 (0.2, 1.7) |
A12C-Adj_12 = hyporexia, chills, headache, arthralgia, itch, rash, vomiting, altered consciousness, hemorrhagic manifestations, or retroocular pain, the ratio lymphocytes/neutrophils ≥ 0.8, leukocytes ≤ 4,600/mm3 or platelets ≤ 186,000/mm3, and as indicative of non-dengue rhinorrhea; CI = credible interval. * One confirmed dengue and seven non-dengue cases had missing lymphocyte count data.
DISCUSSION
The diagnosis of dengue based solely on the patient´s signs and symptoms is complex because of the wide spectrum of clinical characteristics and their low specificity. R&D of dengue diagnostic clinical algorithms using locally available data and resources could provide useful tools for routine care. This study developed and field tested several dengue diagnostic clinical algorithms, none of which reached the desired sensitivity (95%) and specificity (> 70%). The highest sensitivity was observed in algorithms that included parameters of the hemogram, of which leukocyte and platelet counts were most useful in both the development and validation stages (Tables 2 and 4). Leukocyte counts have been included in other diagnostic algorithms such as those reported by Acosta Torres et al.28 (cutoff of 6,500/mm3), Kumar32 (7,950/mm3), Tanner et al.27 (6,000/mm3), and Daumas et al.23 (7,500/mm3); as well as platelet counts by Gregory et al.58 (204,000/mm3), Diaz et al.19 (180,000/mm3), and Tanner et al.27 (193,000/mm3). Other parameters, such as lymphocytes/neutrophils ratio (≥ 0.8) and monocytes (≥ 1,000/mm3) could be further explored. Tanner et al.27 included lymphocyte count (> 0.58 × 1,000 cells/mm3) reaching 71.2% sensitivity, and Cucunawangsih et al.31 included relative monocytosis (> 9%) reaching 79% sensitivity. The discriminative capacity of the lymphocytes/neutrophils ratio and monocytosis could vary with days of symptoms, given that the underlying parameters do so.59 These findings support the use of the hemogram to improve the accuracy of the clinical diagnosis made by physicians in dengue-endemic areas.
Signs and symptoms such as mucosal bleeding and rash were included in all algorithms as indicative of dengue. Almost all the dengue diagnostic algorithms published to date have also included hemorrhagic manifestations and rash with sensitivities between 41% and 90%.21,22,25,30 In the prospective validation, the highest sensitivity (65%) and specificity (43%) was observed with algorithms that included the following variables as indicative of dengue: hyporexia, chills, headache, arthralgia, itch, rash, vomiting, altered consciousness, hemorrhagic manifestations, or retroocular pain, the ratio of lymphocytes/neutrophils ≥ 0.8, leukocytes ≤ 4,600/mm3 or platelets ≤ 186,000/mm3, and as indicative of non-dengue rhinorrhea. However, likelihood ratios were all close to 1, and hence, these algorithms are useful neither to confirm nor rule out dengue. As expected, the diagnostic performance of clinical algorithms was higher during the development stage than the field validation. A relative high sensitivity is expected when the same database is used to develop and assess the performance of diagnostic algorithms. Conversely, a lower value is expected when it is applied to a different population and when the occurrence of other febrile diseases varies in time.16,25 For example, the predictive probability of arthralgia or rash as indicative of dengue could decrease in the presence of chikungunya and Zika (which also manifest with varying degrees of arthralgia or rash, respectively) among the population included during the validation stage but not present in the population during the development stage, suggesting the need to keep clinical diagnostic algorithms updated.
One important consideration in the development and validation of diagnostic clinical algorithms is the gold standard used. Although we use a combination of dengue diagnostic reference laboratory tests, it was not possible to confirm or rule out dengue in all participants. Some limitations were unavailability of sera samples in the convalescent phase in 18% of the study population, and the acute sample being collected ≥ 5 days in 14% of participants which decreases the sensitivity of RT-PCR and NS1. Moreover, secondary dengue infections are difficult to confirm in endemic areas based on currently available ELISA IgM and IgG diagnostic assays.60–62 Misclassification caused by adding probable cases to those laboratory-confirmed is likely to explain the observed lower sensitivities compared with those estimated with only laboratory-confirmed dengue cases. Also, misclassification in dengue laboratory-confirmed cases due to cross-reaction with Zika-specific IgM and IgG could have occurred. Methods that considered an imperfect reference test would be needed to validate dengue diagnostic tools in routine care to overcome this limitation.63 In the present study, the clinical algorithms that achieved the highest accuracy were constructed with the continuous Bayesian formula. Others have developed dengue diagnostic clinical algorithms with Bayesian networks which reached 73% sensitivity64 and Bayesian naive systems yielding sensitivities between 72% and 99%.65–67 Bayesian approaches have the advantage that they reflect the “natural” diagnostic process and quantify the associated uncertainty in the diagnosis of dengue as a predictive probability. Precisely to reflect real-life conditions, the prior probability of dengue in the prospective validation was not fixed. Exploring sources of heterogeneity in test performance complement the validation of diagnostic tools. The dengue calculator showed relative high sensitivities (≥ 80%) in Yopal, in those attending tertiary level of care, ≥ 6 days of fever, children younger than 5 years, severe cases, or DENV-2 infections. Hence, these potential differences could suggest the need to develop and validate diagnostic algorithms in targeted subgroups or as modifiers in the score or the posterior probability. We did not find other studies that explore the heterogeneity of prospectively validated diagnostic algorithms, in spite of suggested differences in dengue clinical presentations.68
Generally, diagnostic tests and other health products are developed in high-income countries, and then transferred to countries where the disease is endemic. This technology transfer could result in performance and implementation deficiencies and decreased effectiveness.69,70 Technologies based on mHealth can be an alternative to support the diagnostic process in routine health care in resource-limited settings.71 During the prospective validation stage, the clinical algorithms were implemented in an app designed to be used on mobile devices such as tablets or smartphones. In its design, physicians who were representative of potential users of the technology contributed their knowledge on key characteristics such as usability and appearance. The research physicians who used the app considered it as adequate. One advantage of the Bayesian adaptive design is that it allows incorporating in the design a series of pre-established modifications in different elements of a study to handle the uncertainty typical of the R&D processes.72,73 This design includes multiple interim analyses to have more informed validations using the accumulated data and, hence, to perform trials with relative smaller sample sizes in a shorter time. Nevertheless, its implementation presents challenges in terms of the availability of infrastructure, resources, and logistical support required for managing information in real time. In the design of the present study, several biases were anticipated. First, febrile subjects who attended health institutions of different levels of care were included to address extrapolation bias. In spite of this, children appear to be underrepresented. To avoid verification bias, both the algorithm and the reference diagnostic tests were applied to all participants. Potential errors in the confirmatory tests were reduced by performing an independent blinded quality control by an expert in dengue and Zika diagnosis. On the one hand, the current study has some limitations such as not being able to perform the differential diagnosis with other arboviruses or other infectious diseases. There were 18% of participants who did not provide the convalescent blood sample, so their dengue status could not be determined. This occurred mainly in the most populated town, in children, and cases with mild symptoms or those who resolved relatively early, reasons that presumably reduced participants’ interest in contributing the convalescent sample. On the other hand, some advantages were the execution of the study under real-life conditions in the post-epidemic period of chikungunya and Zika, in children and adults from different geographical contexts, in different levels of care where dengue diagnostic clinical algorithms are expected to be implemented. To build on this work, further studies would include the development of clinical algorithms that allow inclusion of new clinical data that becomes available during patient follow-up and exploring the potential contribution of the current diagnostic clinical algorithms to reduce misclassification in the dengue surveillance system.
Acknowledgments:
We are grateful to all the scientific and administrative staff of the “Development and applied research to contribute to an effective and sustainable model of dengue intervention in Santander, Casanare, and Valle del Cauca” program and the Knowledge and Cooperation AEDES Network (Red Aedes) for their unconditional, timely, and professional support to conduct the study. We thank the managers and health personnel of all participating institutions Comfandi Torres, Alameda, and Calipso in Cali, and Clínica Piedecuesta, Hospital Local de Piedecuesta, and Hospital Regional de la Orinoquía in Yopal. We thank all research assistants Deici Narváez, Luisa Arias, Alejandra del Castillo, Juan Camilo Hernández, Javier Caicedo, Katherine Laurent, Lizeth Suárez, Liliana Soto, Katherin Quiñones, Gustavo Clemen, Diana Paredes, Liliana Cañón, Carlos Lucumí, Fernando Zamora, Tatiana Cortés, Leticia Rodríguez, Yadira Melo, and Mónica Consuegra for their hard work and contributions.
REFERENCES
- 1.↑
Stanaway JD et al. 2016. The global burden of dengue: an analysis from the Global Burden of Disease Study 2013. Lancet Infect Dis 16: 712–723.
- 2.↑
Castro MC, Wilson ME, Bloom DE, 2017. Disease and economic burdens of dengue. Lancet Infect Dis 17: e70–e78.
- 5.↑
Katzelnick LC, Coloma J, Harris E, 2017. Dengue: knowledge gaps, unmet needs, and research priorities. Lancet Infect Dis 17: e88–e100.
- 6.↑
World Health Organization, 2009. Dengue: Guidelines for Diagnosis, Treatment, Prevention and Control. Geneva, Switzerland: WHO.
- 7.↑
Gutiérrez G, Gresh L, Pérez MÁ, Elizondo D, Avilés W, Kuan G, Balmaseda A, Harris E, 2013. Evaluation of the diagnostic utility of the traditional and revised WHO dengue case definitions. PLoS Negl Trop Dis 7: e2385.
- 8.
Low JGH et al. 2011. The early clinical features of dengue in adults: challenges for early clinical diagnosis. PLoS Negl Trop Dis 5: e1191.
- 9.
Chaterji S, Allen JC, Chow A, Leo YS, Ooi EE, 2011. Evaluation of the NS1 rapid test and the WHO dengue classification schemes for use as bedside diagnosis of acute dengue fever in adults. Am J Trop Med Hyg 84: 224–228.
- 10.
Macedo GA, Gonin M, Pone S, Cruz O, Nobre F, Brasil P, 2014. Sensitivity and specificity of the World Health Organization dengue classification schemes for severe dengue assessment in children in Rio de Janeiro. PLoS One 9: e96314.
- 11.↑
Cavalcanti L, Mota L, Lustosa G, Fortes M, Mota D, Lima A, Coelho I, Mourao MP, 2014. Evaluation of the WHO classification of dengue disease severity during an epidemic in 2011 in the State of Ceará, Brazil. Mem Inst Oswaldo Cruz 109: 93–98.
- 12.↑
Alexander N et al. 2011. Multicentre prospective study on dengue classification in four south-east Asian and three Latin American countries. Trop Med Int Health 16: 936–948.
- 13.↑
Rodriguez-Manzano J, Chia PY, Yeo TW, Holmes A, Georgiou P, Yacoub S, 2018. Improving dengue diagnostics and management through innovative technology. Curr Infect Dis Rep 20: 25.
- 14.↑
Peeling RW et al. 2010. Evaluation of diagnostic tests: dengue. Nat Rev Microbiol 8 (Suppl 12): S30–S38.
- 15.↑
Costa VG, Marques-Silva AC, Moreli ML, 2014. A Meta-analysis of the diagnostic accuracy of two commercial NS1 antigen ELISA tests for early dengue virus detection. PLoS One 9: e94655.
- 16.↑
Buonora SN, Passos S, do Carmo C, Quintela F, de Oliveira D, dos Santos F, Hokerberg Y, Nogueira R, Daumas R, 2016. Accuracy of clinical criteria and an immunochromatographic strip test for dengue diagnosis in a DENV-4 epidemic. BMC Infect Dis 16: 37.
- 17.
Pan-ngum W, Blacksell S, Lubell Y, Pukrittayakamee S, Bailey M, de Silva HJ, Lalloo D, Day N, White L, Limmathurotsakul D, 2013. Estimating the true accuracy of diagnostic tests for dengue infection using Bayesian latent class models. PLoS One 8: e50765.
- 18.↑
Osorio L, Ramirez M, Bonelo A, Villar LA, Parra B, 2010. Comparison of the diagnostic accuracy of commercial NS1-based diagnostic tests for early dengue infection. Virol J 7: 361.
- 19.↑
Diaz FA, Martínez RA, Villar LA, 2006. Criterios clínicos para diagnosticar el dengue en los primeros días de enfermedad. Biomédica 26: 22–30.
- 20.
Díaz FA, Villar LA, Martínez RA, 2006. Indicadores tempranos de infección por dengue en niños. An Pediatr 64: 523–529.
- 21.↑
Ramos M, Tomashek K, Arguello DF, Luxemburger C, Quiñones L, Lang J, Muñoz-Jordan J, 2009. Early clinical features of dengue infection in Puerto Rico. Trans R Soc Trop Med Hyg 103: 878–884.
- 22.↑
Chadwick D, Arch B, Wilder-Smith A, Paton N, 2006. Distinguishing dengue fever from other infections on the basis of simple clinical and laboratory features: application of logistic regression analysis. J Clin Virol 35: 147–153.
- 23.↑
Daumas R, Passos S, Oliveira R, Nogueira R, Georg I, Marzochi K, Brasil P, 2013. Clinical and laboratory features that discriminate dengue from other febrile illnesses: a diagnostic accuracy study in Rio de Janeiro, Brazil. BMC Infect Dis 13: 77.
- 24.
Ho TS, Wang SM, Lin YS, Liu CC, 2013. Clinical and laboratory predictive markers for acute dengue infection. J Biomed Sci 20: 75.
- 25.↑
Chang K et al. 2009. Dengue fever scoring system: new strategy for the early detection of acute dengue virus infection in Taiwan. J Formos Med Assoc 108: 879–885.
- 26.
Vega Riverón B, Sánchez Valdés L, Cortiñas Abrahantes J, Castro Perza O, González Rubio D, Castro Peraza M, 2012. Clasificación de dengue hemorrágico utilizando árboles de decisión en la fase temprana de la enfermedad. Rev Cubana Med Trop 64: 35–42.
- 27.↑
Tanner L et al. 2008. Decision tree algorithms predict the diagnosis and outcome of dengue fever in the early phase of illness. PLoS Negl Trop Dis 2: e196.
- 28.↑
Acosta Torres J, Oller Meneses L, Sokol N, Balado Sariñas R, Montero Díaz D, Balado Sansón R, Sardiñas Arce ME, 2016. Técnica árboles de decisión aplicada al método clínico en el diagnóstico del dengue. Rev Cubana de Pediatr 88: 441–453.
- 29.
Tuan NM et al. 2015. Sensitivity and specificity of a novel classifier for the early diagnosis of dengue. PLoS Negl Trop Dis 9: e0003638.
- 30.↑
Fernández E, Smieja M, Walter SD, Loeb M, 2016. A predictive model to differentiate dengue from other febrile illness. BMC Infect Dis 16: 694.
- 31.↑
Cucunawangsih C, Dewi BE, Sungono V, Pratama N, 2015. Scoring model to predict dengue infection in the early phase of illness in primary health care centre. Arch Clin Microbiol 6: 2.
- 32.↑
Kumar MN, 2013. Alternating decision trees for early diagnosis of dengue fever. Available at: https://arxiv.org/abs/1305.7331. Accessed January 12, 2020.
- 33.↑
Hari Rao VS, Kumar MN, 2012. New Intelligence-based approach for computer-aided diagnosis of dengue fever. IEEE Trans Inf Technol Biomed 16: 112–118.
- 34.↑
Padilla JC, Rojas DP, Sáenz-Gómez R, 2012. Dengue en Colombia: Epidemiología de la Reemergencia a la Hiperendemia. Bogotá, Colombia: Ministerio de Salud y Protección Social.
- 36.
Berry DA, 2006. Bayesian clinical trials. Nat Rev Drug Discov 5: 27–36.
- 37.↑
Food and Drug Administration, 2010. Guidance for the Use of Bayesian Statistics in Medical Device Clinical Trials. Guidance for Industry and FDA Staff. Rockville, MD: Center for Devices and Radiological Health and Office of Surveillance and Biometrics.
- 38.↑
Ministerio de Salud de Colombia, 1993. Resolución No 008430 de Octubre 1993. Bogotá, Colombia: República de Colombia.
- 39.↑
Villar LÁ, Gélvez MR, Rodríguez JA, Salgado D, Parra B, Osorio L, Bosch I, 2013. Biomarcadores pronósticos de gravedad del dengue. Biomédica 33: 108–116.
- 40.↑
Caicedo DM, Méndez AC, Tovar JR, Osorio L, 2019. Desarrollo de algoritmos clínicos para el diagnóstico del dengue en Colombia. Biomédica 39: 170–185.
- 41.↑
Kuno G, Gómez I, Gubler DJ, 1991. An ELISA procedure for the diagnosis of dengue infections. J Virol Methods 33: 101–113.
- 42.↑
Lanciotti R, Calisher C, Gubler D, Chang G, Vorndam A, 1992. Rapid detection and typing of dengue viruses from clinical samples by using reverse transcriptase-polymerase chain reaction. J Clin Microbiol 30: 545–551.
- 43.↑
Clarke DH, Casals J, 1958. Techniques for hemagglutination and hemagglutination-inhibition with arthropod-borne viruses. Am J Trop Med Hyg 7: 561–573.
- 44.↑
Press SJ, 2002. Bayes’ Theorem. Subjective and Objective Bayesian Statistics. Hoboken, NJ: John Wiley & Sons, 41–72.
- 45.
Dawid AP, 1979. Conditional independence in statistical theory. J R Stat Soc Ser B Stat Methodol 41: 1–31.
- 46.↑
Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB, 2013. Bayesian Data Analysis. Boca Raton, FL: CRC Press.
- 47.↑
Tovar JR, 2015. Inferencia bayesiana e investigación en salud: un caso de aplicación en diagnóstico clínico. Rev Méd Risaralda 21: 9–16.
- 48.↑
Box GEP, Tiao GC, 1992. Bayesian Inference in Statistical Analysis. New York, NY: John Wiley and Sons, Inc.
- 49.↑
Ruopp MD, Perkins NJ, Whitcomb BW, Schisterman EF, 2008. Youden Index and optimal cut-point estimated from observations affected by a lower limit of detection. Biom J 50: 419–430.
- 50.↑
Brady OJ, Gething PW, Bhatt S, Messina JP, Brownstein JS, Hoen AG, Moyes CL, Farlow AW, Scott TW, Hay SI, 2012. Refining the global spatial limits of dengue virus transmission by evidence-based consensus. PLoS Negl Trop Dis 6: e1760.
- 51.↑
Pennello G, Thompson L, 2008. Experience with reviewing Bayesian medical device trials. J Biopharm Stat 18: 81–115.
- 52.↑
Fienberg SE, 2007. The Analysis of Cross-Classified Categorical Data. New York, NY: Springer Science+Business Media, LLC.
- 54.↑
Chang W, Cheng J, Allaire J, Xie Y, McPherson J, 2019. Shiny: Web Application Framework for R. R package version 1.3.2. Available at: https://cran.rstudio.com/bin/windows/contrib/3.5/shiny_1.4.0.zip. Accessed June 10, 2019.
- 55.↑
Méndez AC, 2019. Bayesian Classifier for Discrete Data Using the Beta Distribution (BetaBsClassifier). Available at: https://github.com/andresk159/BetaBsClassifier. Accessed January 12, 2020.
- 57.↑
R Core Team, 2013. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing.
- 58.↑
Gregory C, Santiago L, Arguello D, Hunsperger E, Tomashek K, 2010. Clinical and laboratory features that differentiate dengue from other febrile illnesses in an endemic area-Puerto Rico, 2007–2008. Am J Trop Med Hyg 82: 922–929.
- 59.↑
Chaloemwong J, Tantiworawit A, Rattanathammethee T, Hantrakool S, Chai-Adisaksopha C, Rattarittamrong E, Norasetthada L, 2018. Useful clinical features and hematological parameters for the diagnosis of dengue infection in patients with acute febrile illness: a retrospective study. BMC Hematol 18: 1–10.
- 60.↑
Tsai WY et al. 2017. Distinguishing secondary dengue virus infection from Zika virus infection with previous dengue by a combination of 3 simple serological tests. Clin Infect Dis 65: 1829–1836.
- 61.
Sa-Ngasang A, Anantapreecha S, A-Nuegoonpipat A, Chanama S, Wibulwattanakij S, Pttanakul K, Sawanpanyalert P, Kurane I, 2006. Specific IgM and IgG responses in primary and secondary dengue virus infections determined by enzyme-linked immunosorbent assay. Epidemiol Infect 134: 820–825.
- 62.↑
Chanama S, Anantapreecha S, A-nuegoonpipat A, Sa-gnasang A, Kurane I, Sawanpanyalert P, 2004. Analysis of specific IgM responses in secondary dengue virus infections: levels and positive rates in comparison with primary infections. J Clin Virol 31: 185–189.
- 64.↑
Sa-ngamuang C, Haddawy P, Luvira V, Piyaphanee W, Iamsirithaworn S, Lawpoolsri S, 2018. Accuracy of dengue clinical diagnosis with and without NS1 antigen rapid test: comparison between human and Bayesian network model decision. PLoS Negl Trop Dis 12: e0006573.
- 65.↑
Arafiyah R, Hermin F, Kartika I, Alimuddin A, Saraswati I, 2018. Classification of dengue haemorrhagic fever (DHF) using SVM, naive bayes and random forest. IOP Conf Ser Mater Sci Eng 434: 012070.
- 66.
Farooqi W, Ali S, 2013. A Critical Study of Selected Classification Algorithms for Dengue Fever and Dengue Hemorrhagic Fever. Frontiers of Information Technology (FIT), 2013 11th International Conference, Islamabad, Pakistan, 1: 140–145.
- 67.↑
Shaukat Dar K, Mehreen S, 2015. Dengue fever prediction: a data mining problem. J Data Min Genom Proteomics 6: 181.
- 68.↑
Leeflang M, Bossuyt P, Irwig L, 2009. Diagnostic test accuracy may vary with prevalence: implications for evidence-based diagnosis. J Clin Epidemiol 62: 5–12.
- 69.↑
Panerai RB, Peña J, 1990. Evaluación de Tecnologías en salud, Metodología para PaÍses en Desarrollo. Washington, DC: Pan American Health Organization.
- 70.↑
World Health Organization, 2010. Innovative Technologies that Address Global Health Concerns, Outcome of the Call: Global Initiative on Health Technologies. Geneva, Switzerland: WHO.
- 71.↑
Free C, Phillips g, Watson K, Galli L, Felix L, Wdwards P, Patel V, Haines A, 2013. The effectiveness of mobile-health technologies to improve health care service delivery processes: a systematic review and meta-analysis. PLoS Med 10: e1001363.
- 72.↑
Chow SC, Chang M, 2008. Adaptive design methods in clinical trials–a review. Orphanet J Rare Dis 3: 1–13.
- 73.↑
Food and Drug Administration, 2016. Adaptive designs for medical device clinical studies. Guidance for Industry and Food and Drug Administration Staff. Rockville, MD: Center for Devices and Radiological Health and Center for Biologics Evaluation and Research.