Sensitivity and Specificity of the Remote Evaluation of Therapeutic Response in Cutaneous Leishmaniasis Using Photographs from a Mobile Application

Alejandra Maria Del Castillo Centro Internacional de Entrenamiento e Investigaciones Médicas (CIDEIM), Cali, Colombia;

Search for other papers by Alejandra Maria Del Castillo in
Current site
Google Scholar
PubMed
Close
,
Maria del Mar Castro Centro Internacional de Entrenamiento e Investigaciones Médicas (CIDEIM), Cali, Colombia;
Universidad Icesi, Cali, Colombia;

Search for other papers by Maria del Mar Castro in
Current site
Google Scholar
PubMed
Close
,
Alexandra Cossio Centro Internacional de Entrenamiento e Investigaciones Médicas (CIDEIM), Cali, Colombia;
Universidad Icesi, Cali, Colombia;

Search for other papers by Alexandra Cossio in
Current site
Google Scholar
PubMed
Close
,
Jonny Alejandro García Luna Centro Internacional de Entrenamiento e Investigaciones Médicas (CIDEIM), Cali, Colombia;
Universidad Icesi, Cali, Colombia;

Search for other papers by Jonny Alejandro García Luna in
Current site
Google Scholar
PubMed
Close
,
Domiciano Rincón Universidad Icesi, Cali, Colombia;
Grupo i2t, Universidad Icesi, Cali, Colombia

Search for other papers by Domiciano Rincón in
Current site
Google Scholar
PubMed
Close
,
Ruth Mabel Castillo Centro Internacional de Entrenamiento e Investigaciones Médicas (CIDEIM), Cali, Colombia;
Universidad Icesi, Cali, Colombia;

Search for other papers by Ruth Mabel Castillo in
Current site
Google Scholar
PubMed
Close
,
Miguel Darío Prieto Centro Internacional de Entrenamiento e Investigaciones Médicas (CIDEIM), Cali, Colombia;

Search for other papers by Miguel Darío Prieto in
Current site
Google Scholar
PubMed
Close
,
David Esteban Rebellón-Sánchez Centro Internacional de Entrenamiento e Investigaciones Médicas (CIDEIM), Cali, Colombia;

Search for other papers by David Esteban Rebellón-Sánchez in
Current site
Google Scholar
PubMed
Close
,
Andrés Navarro Universidad Icesi, Cali, Colombia;
Grupo i2t, Universidad Icesi, Cali, Colombia

Search for other papers by Andrés Navarro in
Current site
Google Scholar
PubMed
Close
, and
Neal Alexander Centro Internacional de Entrenamiento e Investigaciones Médicas (CIDEIM), Cali, Colombia;
Universidad Icesi, Cali, Colombia;

Search for other papers by Neal Alexander in
Current site
Google Scholar
PubMed
Close
Restricted access

ABSTRACT.

Cutaneous leishmaniasis (CL) primarily affects people in remote settings with limited access to health services. mHealth tools offer an opportunity to overcome knowledge gaps about clinical response to treatment. We evaluated the validity of the Guaral+ST mobile application for the remote assessment of therapeutic response in patients with CL, through photographs of lesions captured with the app by community health volunteers. Patients with confirmed CL were followed at weeks 13 and 26 after completion of treatment to assess therapeutic response in two clinical settings in southwest Colombia. Direct evaluation of lesions performed by an experienced physician was considered the reference standard. Photographs of lesions taken by CHV or nurse assistants with the mobile app, were independently evaluated by three physicians to define clinical response. A summary measure of clinical outcome defined by the three physicians was considered the index test. Sensitivity, specificity, and positive and negative predictive values were estimated. Interrater reliability (kappa) was calculated. Among 53 participants with CL who had at least one follow-up visit, the sensitivity of therapeutic response evaluation through photographs taken with the Guaral+ST app, compared with direct evaluation by an expert physician, had high validity with sensitivity of 100% (95% confidence interval: 80.5–100%) and specificity of 97.2% (95% confidence interval: 85.5–99.9%). The chance-adjusted agreement (κ) was > 0.8, which is conventionally characterized as almost perfect. The high accuracy of the remote evaluation of photographs for the assessment of therapeutic response supports the use of mHealth tools for improving access to treatment follow-up for CL.

    • Supplemental Materials (PDF 1053 KB)

Author Notes

Address correspondence to Neal Alexander, Centro Internacional de Entrenamiento e Investigaciones Médicas, Calle 18 #122-135, Campus de la Universidad Icesi (Edificio O-CIDEIM), Cali 760031, Colombia. E-mail: neal.alexander@lshtm.ac.uk

Financial support: This study was supported by the Fogarty International Center of the National Institutes of Health (NIH), under award nos. D43 TW006589 and R21 TW009907; the National Institutes of Allergy and Infectious Diseases of the National Institutes of Health under award no. U19AI129910; and an International Master’s Fellowship from the Wellcome Trust awarded to J. A. G. (reference number 214641/Z/18/Z). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Authors’ addresses: Alejandra Maria Del Castillo and David Esteban Rebellón-Sánchez, Centro Internacional de Entrenamiento e Investigaciones Médicas (CIDEIM), Cali, Colombia, E-mails: amdelcastillo@cideim.org.co and derebellon@cideim.org.co. Maria del Mar Castro, Alexandra Cossio, Jonny Alejandro García Luna, and Ruth Mabel Castillo, CIDEIM, Cali, Colombia, and Universidad Icesi, Cali, Colombia, E-mails: mcastro@cideim.org.co, acossio@cideim.org.co, jagarcia@cideim.org.co, and rmcastillo@cideim.org.co. Domiciano Rincón and Andrés Navarro, Universidad Icesi, Cali, Colombia, and Grupo i2t, Universidad Icesi, Cali, Colombia, E-mails: domiciano.rincon@correounivalle.edu.co and nalexander@cideim.org.co.

  • 1.

    Organización Panamericana de Salud , 2020. Leishmaniasis. Informe Epidemiológico de las Américas. Available at: https://iris.paho.org/handle/10665.2/53089.

    • PubMed
    • Export Citation
  • 2.

    Gómez Gómez JL , 2021. Informe de Evento Leishmaniasis Cutánea, Periodo Epidemiológico XIII. Colombia, 2020. Instituto Nacional de Salud. Bogotá.

    • PubMed
    • Export Citation
  • 3.

    Pinart M , Rueda JR , Romero GAS , Pinzón-Flórez CE , Osorio-Arango K , Silveira Maia-Elkhoury AN , Reveiz L , Elias VM , Tweed JA , 2020. Interventions for American cutaneous and mucocutaneous leishmaniasis. Cochrane Database Syst Rev 2: CD004834.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 4.

    Palacios R , Osorio LE , Grajalew LF , Ochoa MT , 2001. Treatment failure in children in a randomized clinical trial with 10 and 20 days of meglumine antimonate for cutaneous leishmaniasis due to Leishmania viannia species. Am J Trop Med Hyg 64: 187193.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 5.

    Chrusciak-Talhari A , Dietze R , Chrusciak Talhari C , da Silva RM , Gadelha Yamashita EP , de Oliveira Penna G , Lima Machado PR , Talhari S , 2011. Randomized controlled clinical trial to access efficacy and safety of miltefosine in the treatment of cutaneous leishmaniasis Caused by Leishmania (Viannia) guyanensis in Manaus, Brazil. Am J Trop Med Hyg 84: 255260.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 6.

    Castro M , Martinez AJ , Cossio A , Jojoa SJ , 2014. Seguimiento de pacientes con leishmaniasis tegumentaria: experiencia en un centro de referencia. Infectio 18: 71.

  • 7.

    Olliaro P , Vaillant M , Arana B , Grogl M , Modabber F , Magill A , Lapujade O , Buffet P , Alvar J , 2013. Methodology of clinical trials aimed at assessing interventions for cutaneous leishmaniasis. PLoS Negl Trop Dis 7: e2130.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 8.

    Organización Panamericana de la Salud , 2017. Plan de Acción para fortalecer la vigilancia y control de las leishmaniasis en las Américas 2017–2022. Washington, DC: Pan American Health Organization.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 9.

    Wood CS , Thomas MR , Budd J , Mashamba-Thompson TP , Herbst K , Pillay D , Peeling RW , Johnson AM , McKendry RA , Stevens MM , 2019. Taking connected mobile-health diagnostics of infectious diseases to the field. Nature 566: 467474.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 10.

    Rubiano L et al., 2021. Adaptation and performance of a mobile application for early detection of cutaneous leishmaniasis. PLoS Negl Trop Dis 15: e0008989.

  • 11.

    Bossuyt PM et al., 2015. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ 351: h5527.

  • 12.

    Hajian-Tilaki K , 2014. Sample size estimation in diagnostic test studies of biomedical informatics. J Biomed Inform 48: 193204.

  • 13.

    Oviedo Sarmiento OJ , Castro MDM , Lerma YO , Bernal LV , Navarro A , Alexander NDE , 2021. Data management plan for a community-level study of the hidden burden of cutaneous leishmaniasis in Colombia. BMC Res Notes 14: 213.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 14.

    Ministerio de Salud y Protección Social , 2018. Lineamientos para la Atención Clínica Integral de Leishmaniasis en Colombia. Bogotá.

    • PubMed
    • Export Citation
  • 15.

    Ministerio de Salud y Protección Social , 2010. Guía de Atención Clínica Integral del Paciente con Leishmaniasis. Bogotá.

    • PubMed
    • Export Citation
  • 16.

    Bowen AC , Burns K , Tong SY , Andrews RM , Liddle R , O’Meara IM , Westphal DW , Carapetis JR , 2014. Standardising and assessing digital images for use in clinical trials: a practical, reproducible method that blinds the assessor to treatment allocation. PLoS One 9: e110395.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 17.

    StataCorp , 2019. Stata Statistical Software: Release 16. College Station, TX: StataCorp LLC.

  • 18.

    Seed PT , Tobias A , 2001. Summary statistics for diagnostic tests. Stata Tech Bull STB-59: 912.

  • 19.

    Reichenheim ME , 2004. Confidence intervals for the kappa statistic. Stata J 4: 421428.

  • 20.

    Fleiss JL , Cohen J , Everitt BS , 1969. Large sample standard errors of kappa and weighted kappa. Psychol Bull 72: 323.

  • 21.

    Cohen J , 1960. A coefficient of agreement for nominal scales. Educ Psychol Meas 20: 3746.

  • 22.

    Landis JR , Koch GG , 1977. The measurement of observer agreement for categorical data. Biometrics 33: 159174.

  • 23.

    Carrion C , Robles N , Sola-Morales O , Aymerich M , Ruiz Postigo JA , 2020. Mobile health strategies to tackle skin neglected tropical diseases with recommendations from innovative experiences: systematic review. JMIR Mhealth Uhealth 8: e22478.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 24.

    Nadri K , Shahmoradi L , Sajedi H , Salehi A , 2021. Content for self-care app for patients with cutaneous leishmaniasis: designing a mobile-based self-care app for patients with cutaneous leishmaniasis. Health Policy Technol 10: 8794.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 25.

    Ferreira da Silva PE , Junior G , Ambrozio RB , Salles Tiburcio Costa MG , Machado GB , Guimaraes de Carvalho SF , Jose de Oliveira E , Jorge DC , de Almeida Silva Teixeira L , 2020. LeishCare: a software designed for the management of individuals with leishmaniases. Am J Trop Med Hyg 103: 909916.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 26.

    de Vilhena Guimarães Sd S , de Pinho Barreiros M , Portal Pd SC , Pinto RB , do Rosário Carneiro IC , dos Santos VRC , 2021. Tecnologias móveis para auxiliar no manejo terapêutico de leishmaniose tegumentar: uma revisão integrativa. Research. Soc Dev 10: e46101320847e46101320847.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 27.

    Lee JJ , English JC , 2018. Teledermatology: a review and update. Am J Clin Dermatol 19: 253260.

  • 28.

    Shapiro M , James WD , Kessler R , Lazorik FC , Katz KA , Tam J , Nieves DS , Miller JJ , 2004. Comparison of skin biopsy triage decisions in 49 patients with pigmented lesions and skin neoplasms: store-and-forward teledermatology vs face-to-face dermatology. Arch Dermatol 140: 525528.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 29.

    Rubegni P , Nami N , Cevenini G , Poggiali S , Hofmann‐Wellenhof R , Massone C , Bilenchi R , Bartalini M , Cappelli R , Fimiani M , 2011. Geriatric teledermatology: store‐and‐forward vs. face‐to‐face examination. J Eur Acad Dermatol Venereol 25: 13341339.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 30.

    Levin YS , Warshaw EM , 2009. Teledermatology: a review of reliability and accuracy of diagnosis and management. Dermatol Clin 27: 163176.

  • 31.

    Devrim I , Duzgol M , Kara A , Caglar I , Devrim F , Bayram N , Apa H , 2019. Reliability and accuracy of smartphones for paediatric infectious disease consultations for children with rash in the paediatric emergency department. BMC Pediatr 19: 40.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 32.

    Navarro A , Rubiano L , Arango JD , Rojas CA , Alexander N , Saravia NG , Aronoff-Spencer E , 2018. Developing mobile health applications for neglected tropical disease research. PLoS Negl Trop Dis 12: e0006791.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 33.

    NHS England , 2015. Revisions to EU data protection laws could have a profound impact on the NHS. Information Governance Bulletin, 1–4.

    • PubMed
    • Export Citation
  • 34.

    Ferrándiz L , Moreno-Ramírez D , Ruiz-de-Casas A , Nieto-García A , Moreno-Álvarez P , Galdeano R , Camacho F , 2008. An economic analysis of presurgical teledermatology in patients with nonmelanoma skin cancer. Actas Dermosifiliogr 99: 795802 (English Edition).

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 35.

    Lim D , Oakley AM , Rademaker M , 2012. Better, sooner, more convenient: a successful teledermoscopy service. Australas J Dermatol 53: 2225.

  • 36.

    Whited JD , Datta S , Hall RP , Foy ME , Marbrey LE , Grambow SC , Dudley TK , Simel DL , Oddone EZ , 2003. An economic analysis of a store and forward teledermatology consult system. Telemed J E Health 9: 351360.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 37.

    Eminovic N , Dijkgraaf MG , Berghout RM , Prins AH , Bindels PJ , de Keizer NF , 2010. A cost minimisation analysis in teledermatology: model-based approach. BMC Health Serv Res 10: 251.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 38.

    Thomas J , Davey R , Peterson GM , Carson C , Walton SF , Spelman T , Calma T , Dettwiller P , Tobin J , McMillan F , 2018. Treatment of scabies using a tea tree oil-based gel formulation in Australian Aboriginal children: protocol for a randomised controlled trial. BMJ Open 8: e018507.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 39.

    Uribe-Restrepo AF , Prieto MD , Cossio A , Desai MM , Castro MDM , 2019. Eligibility for local therapies in adolescents and adults with cutaneous leishmaniasis from southwestern Colombia: a cross-sectional study. Am J Trop Med Hyg 100: 306310.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 40.

    Blanco VM , Cossio A , Martinez JD , Saravia NG , 2013. Clinical and epidemiologic profile of cutaneous leishmaniasis in Colombian children: considerations for local treatment. Am J Trop Med Hyg 89: 359364.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 41.

    Smith AH , Bates MN , 1992. Confidence limit analyses should replace power calculations in the interpretation of epidemiologic studies. Epidemiology 3: 449452.

  • 42.

    Meloan M , Castells P , 2017. Country Overview: Colombia. Available at: https://data.gsmaintelligence.com/api-web/v2/research-file-download?id=28999732&file=Country%20overview%20Colombia.pdf. Accessed May 9, 2022.

    • PubMed
    • Export Citation
Past two years Past Year Past 30 Days
Abstract Views 3747 1796 277
Full Text Views 198 25 0
PDF Downloads 140 22 0
 

 

 

 
 
Affiliate Membership Banner
 
 
Research for Health Information Banner
 
 
CLOCKSS
 
 
 
Society Publishers Coalition Banner
Save