• 1.

    Republic of Uganda Ministry of Health, 2015. National Village Health Teams (VHT) Assessment in Uganda. Available at: http://library.health.go.ug/publications/health-education/national-village-health-teamsvht-assessment-uganda. Accessed September 12, 2020.

    • Search Google Scholar
    • Export Citation
  • 2.

    Republic of Uganda Ministry of Health, 2010. Village Health Team Strategy and Operational Guidelines. Available at: http://library.health.go.ug/publications/health-education/village-health-team-strategy-and-operational-guidelines. Accessed September 12, 2020.

    • Search Google Scholar
    • Export Citation
  • 3.

    The Republic of Uganda Ministry of Health, 2010. Integrated Community Case Management of Childhood Malaria, Pneumonia and Diarrhoea: Implementation Guidelines. Available at: http://library.health.go.ug/publications/child-health-school-health/integrated-community-case-management-childhood-malaria. Accessed July 25, 2020.

    • Search Google Scholar
    • Export Citation
  • 4.

    Counihan H, Harvey SA, Sekeseke-Chinyama M, Hamainza B, Banda R, Malambo T, Masaninga F, Bell D, 2012. Community health workers use malaria rapid diagnostic tests (RDTs) safely and accurately: results of a longitudinal study in Zambia. Am J Trop Med Hyg 87: 5763.

    • Search Google Scholar
    • Export Citation
  • 5.

    Boyce MR, Menya D, Turner EL, Laktabai J, Prudhomme-O’Meara W, 2018. Evaluation of malaria rapid diagnostic test (RDT) use by community health workers: a longitudinal study in western Kenya. Malar J 17: 206.

    • Search Google Scholar
    • Export Citation
  • 6.

    Mukanga D, Babirye R, Peterson S, Pariyo GW, Ojiambo G, Tibenderana JK, Nsubuga P, Kallander K, 2011. Can lay community health workers be trained to use diagnostics to distinguish and treat malaria and pneumonia in children? Lessons from rural Uganda. Trop Med Int Health 16: 12341242.

    • Search Google Scholar
    • Export Citation
  • 7.

    Phiri TB, Kaunda-Khangamwa BN, Bauleni A, Chimuna T, Melody D, Kalengamaliro H, Sande JH, Nsona HK, Mathanga DP, 2016. Feasibility, acceptability and impact of integrating malaria rapid diagnostic tests and pre-referral rectal artesunate into the integrated community case management programme. A pilot study in Mchinji district, Malawi. Malar J 15: 177.

    • Search Google Scholar
    • Export Citation
  • 8.

    Harvey SA, Jennings L, Chinyama M, Masaninga F, Mulholland K, Bell DR, 2008. Improving community health worker use of malaria rapid diagnostic tests in Zambia: package instructions, job aid and job aid-plus-training. Malar J 7: 160.

    • Search Google Scholar
    • Export Citation
  • 9.

    Laktabai J, Platt A, Menya D, Turner EL, Aswa D, Kinoti S, O’Meara WP, 2018. A mobile health technology platform for quality assurance and quality improvement of malaria diagnosis by community health workers. PLoS One 13: e0191968.

    • Search Google Scholar
    • Export Citation
  • 10.

    Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG, 2009. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 42: 377381.

    • Search Google Scholar
    • Export Citation
  • 11.

    Miller JS 2018. Quality of care in integrated community case management services in Bugoye, Uganda: a retrospective observational study. Malar J 17: 99.

    • Search Google Scholar
    • Export Citation

 

 

 

Assessing Village Health Workers’ Ability to Perform and Interpret Rapid Diagnostic Tests for Malaria 4 Years after Initial Training: A Cross-Sectional Study

View More View Less
  • 1 Massachusetts General Hospital, Boston, Massachusetts;
  • 2 Harvard Medical School, Boston, Massachusetts;
  • 3 Global Health Collaborative, Mbarara, Uganda;
  • 4 Indiana University School of Medicine, Indianapolis, Indiana;
  • 5 Mbarara University of Science and Technology, Mbarara, Uganda;
  • 6 Bugoye Health Center, Bugoye, Uganda;
  • 7 Vanderbilt University School of Medicine, Nashville, Tennessee

ABSTRACT

Village health workers (VHWs) in Bugoye subcounty, Uganda, provide integrated community case management (iCCM) care to children younger than 5 years for malaria, pneumonia, and diarrhea. We assessed the longevity of VHWs’ skills in performing and reading malaria rapid diagnostic tests (RDTs) 4 years after initial training, comparing VHWs who had completed initial iCCM training 1 year before the study with VHWs who had completed training 4 years before the study. Both groups received quarterly refresher trainings. Trained interviewers observed 36 VHWs reading six mock RDTs each and performing an RDT as part of a larger skills assessment exercise. VHWs read 97% of mock RDTs correctly; of the 36 VHWs, 86% read all six mock RDTs correctly. Most VHWs scored either 12/13 or 13/13 on the RDT checklist (39% and 36%, respectively), with 25% scoring 11/13 or lower. For reading mock RDTs, VHWs in the first group (initial training 4 years before study) read 97% of mock RDTs correctly, whereas those in the second group (initial training 1 year before study) read 96% of mock RDTs correctly; the first group had a mean of 5.83 RDTs read correctly, compared with 5.77 RDTs read correctly in the second group (P = 0.83). For performing an RDT, the first group completed a mean of 12.0 steps correctly, compared with a mean of 12.2 correct steps in the second group (P = 0.60). Overall, VHWs demonstrated proficiency in reading RDTs accurately and performing RDTs according to protocol at least 4 years after initial iCCM training.

INTRODUCTION

Since 2001, Uganda has employed a national cadre of volunteer village health workers (VHWs), locally referred to as “village health teams,” with a broad range of responsibilities focused largely on health promotion and community mobilization.1,2 In some areas, VHWs also provide integrated community case management (iCCM) care to children younger than 5 years, in keeping with national policy.3 In this model of care, VHWs assess and treat malaria, pneumonia, and diarrhea using a clinical algorithm, which includes performing a rapid diagnostic test (RDT) for malaria on any child presenting with subjective fever and then interpreting the result of the test. Performing and interpreting an RDT correctly underpins VHWs’ ability to provide appropriate care for pediatric malaria. Inaccuracy of either aspect may result in lack of treatment of a potentially fatal disease or, less concerningly, overuse of artemisinin combination therapy. At the time of this study, VHWs in eight villages in Bugoye subcounty, Uganda, were providing iCCM care as part of a pilot program.

Several prior studies have assessed VHWs’ ability to maintain or improve their skills in performing and reading RDTs over time. One study in Zambia documented an improvement in performing RDTs over time, with the percentage of critical steps performed increasing from 87.5% at the 3-month follow-up to 100% at 6- and 12-month follow-ups.4 Performance in reading RDTs appeared generally stable over time, with over 90% read correctly at all time points, although in the subset of faintly positive RDTs, correct reading decreased from 89.7% to 76.7% at the 12-month follow-up.4 A study in Kenya found that completion of all “safety steps” as well as all steps in performing an RDT did not appear to differ between baseline (after training) and 1-year follow-up.5 This study assessed reading of RDTs at 1-year follow-up only and reported a sensitivity of 92% and a specificity of 97%.5

A number of other studies have assessed VHWs’ abilities at a single time point, usually shortly after training, overall demonstrating high levels of completion of correct steps in performing RDTs and correct interpretation of RDT results. One prior study in Uganda assessing RDT interpretation immediately after training reported that VHWs read 100% of RDTs correctly.6 A study in Malawi assessing health surveillance assistants’ ability to perform RDTs immediately after training found that 97% of RDTs were performed correctly.7 Another study in Zambia assessed both performing and reading RDTs and compared groups of VHWs with and without a job aid and formal training; those using a job aid with formal training completed an average of 90% of test steps correctly and read 93% of RDTs correctly.8 A study in Kenya used an automated RDT reader to assess both reading of RDTs and a subset of steps in performing RDTs; 92% of RDTs were performed correctly (for the subset of steps assessed) and read correctly.9

In this study, we sought to assess the longevity of VHWs’ skills in performing and reading RDTs over a longer time period, comparing a group of VHWs who had completed initial iCCM training 4 years before the study with a group completing initial training 1 year before the study.

MATERIALS AND METHODS

In this setting, VHWs are part-time volunteers who have attended at least some primary school and achieved basic literacy; most work as subsistence farmers. At the time of this study, there were two cohorts of VHWs in Bugoye subcounty: the first group (23 VHWs) had completed initial iCCM training (including training on how to perform an RDT) in 2013 (4 years before this study), whereas the second group (13 VHWs) had completed initial training in 2016 (1 year before this study). Initial iCCM training was similar for both groups and included several hours of training on performing and interpreting RDTs, including supervised practice performing an actual RDT. Both groups of VHWs also received quarterly refresher training sessions and periodic supervisory visits by program staff.

For this study, two trained interviewers visited VHWs at their homes or another agreed-on location to conduct a skills assessment exercise, which included interpretation of mock RDT results and performing an RDT. For the mock RDTs, there were six digitally created RDT results: two positive RDTs, two negative RDTs, and two indeterminate RDTs per VHW. The digitally created RDT results were presented to VHWs on color-printed laminated sheets; each VHW received the RDT results in the same order. Village health workers performed an RDT using a fake hand for the fingerstick sample collection, but otherwise using a real RDT kit (the use of a mock hand obviates the ethical need for the interviewer to interrupt if a critical step is missed). This program uses the SD BIOLINE Malaria Ag P.f/Pan rapid test (Abbott, FL). The exercise also included video and oral case scenarios and a brief qualitative interview; these results will be reported separately. Village health workers were given a small incentive at the end of the exercise.

The interviewer scored VHWs performing an RDT using a 15-point checklist already used by the iCCM program. However, two of the steps on the checklist (recording the patient’s name on the cassette and recording the result in the clinical register) were not included in the skills assessment exercise and thus not included in scoring, so performing the RDT was scored on a 13-point scale. Data were entered into a customized Research Electronic Data Capture database10 and analyzed in Stata version 15 (StataCorp, College Station, TX).

All VHWs provided written consent before participating in the skills assessment exercise. Ethical approval for this study was granted by the Partners Healthcare Institutional Review Board and the Research Ethics Committee at Mbarara University of Science and Technology.

RESULTS

All VHWs in both groups (a total of 36 VHWs) providing iCCM care in Bugoye subcounty at the time of the study agreed to participate in the skills assessment exercise; 53% are women, and 44% are younger than 40 years (to allow for collection of fully de-identified information, exact ages were not recorded given the small number of participants). Of the 36 VHWs, 31% completed primary school only and 69% completed some or all of secondary school.

Overall, VHWs read 97% of mock RDTs correctly; 100% of positive RDTs, 90% of negative RDTs, and 100% of indeterminate RDTs were read correctly (Table 1). Of the seven negative RDTs read incorrectly, two were read as positive, four were read as indeterminate, and one had no result recorded (reflecting either nonresponse by the VHW for that particular item or a record-keeping error). Of the 36 VHWs, 86% read all six mock RDTs correctly (Table 1).

Table 1

Village health worker performance reading mock RDTs

MeasureGroup 1 (4-year experience)Group 2 (1-year experience)Overall
%N%n%N
RDTs read correctly97134/1389675/7897209/216
Positive RDTs read correctly10046/4610026/2610072/72
Negative RDTs read correctly9142/468823/269065/72
Indeterminate RDTs read correctly10046/4610026/2610072/72
VHWs reading all six RDTs correctly8720/238511/138631/36

RDT = rapid diagnostic test; VHW = village health worker.

Most VHWs scored either 12/13 or 13/13 on the RDT checklist (39% and 36%, respectively), with 25% scoring 11/13 or lower (Table 2). The most commonly missed step was checking the RDT package expiry date, with only 58% of VHWs completing this step. In addition, 88% remembered to put on gloves, meaning that four VHWs neglected this step, and 86% correctly disposed of non-sharps garbage; for all other steps, over 90% of VHWs completed that step correctly (Table 2).

Table 2

Completion of correct steps in performing an RDT

Measure% Group 1 (4-year experience) n = 23% Group 2 (1-year experience) n = 13% Overall, n = 36
Overall scores (out of 13)
 VHWs with score of 13393839
 VHWs with score of 12304636
 VHWs with score of 11221519
 VHWs with score of 9 or 10906
VHWs’ completion of each step
 Assembles packet, buffer, swab, and lancet100100100
 Checks expiry date of test package615458
 Removes contents of test package100100100
 Puts on a new pair of gloves879288
 Selects the 4th finger of the left hand and cleans with alcohol9610097
 Uses sterile lancet to puncture finger1009297
 Discards lancet in sharps bin promptly9610097
 Collects blood with an enclosed capillary tube100100100
 Uses the capillary tube to blot blood in sample well9610097
 Discards capillary tubes/loops in the sharps box919292
 Dispenses buffer into the sample well9110094
 Waits 15 minutes before reading results9610097
 Disposes of other materials in non-sharps garbage839286

RDT = rapid diagnostic test; VHW = village health worker.

Performance was similar between the two cohorts of VHWs. For reading mock RDTs, VHWs in the first group (initial training 4 years before study) read 97% of mock RDTs correctly, whereas those in the second group (initial training 1 year before study) read 96% of mock RDTs correctly (Table 1); the first group had a mean of 5.83 RDTs read correctly, compared with 5.77 RDTs read correctly in the second group (P = 0.83; see Table 3). For performing an RDT, the first group completed a mean of 12.0 steps correctly, compared with a mean of 12.2 correct steps in the second group (P = 0.60; see Table 3).

Table 3

Overall comparison of village health workers with 4 years vs. 1 year of integrated community case management experience

MeasureGroup 1 (4-year experience)Group 2 (1-year experience)P-value*
MedianMeanSEMedianMeanSE
Number of mock RDTs read correctly (out of 6)65.830.1065.770.170.83
Number of steps completed correctly in performing RDT (out of 13)1212.00.231212.20.200.60

RDT = rapid diagnostic test.

By Wilcoxon rank sum test.

DISCUSSION

This study suggests that VHWs can read RDTs with high accuracy and largely adhere to protocol in performing RDTs, in line with prior research in this field.49 The more novel finding of this study is the evidence that VHWs’ performance remains high 4 years after initial iCCM training. Few studies have assessed VHWs’ long-term performance in providing iCCM care, which may be a particular concern for settings like Uganda in which VHWs are part-time volunteers whose long-term motivation may be affected by the many other demands on their time and whose performance over time may be affected by their lower test volume compared with full-time workers. Yet if iCCM is to function as a critical component of care for children younger than 5 years, then ensuring long-term quality of care is crucial. Prior studies of RDT performance and iCCM care more broadly have assessed a single point in time, often just after training, or short-term (e.g., 1 year) follow-up. Although the small study described here provides some reassurance that part-time volunteer VHWs can maintain their skills over time for the specific domain of RDT performance, broader iCCM care quality over time requires further study. Additional research may reveal that VHWs not only maintain but actually improve iCCM care quality over time as they gain experience, but in the absence of data, the prospect of deteriorating quality of care remains a concern. In addition, comparison of iCCM care quality over time among VHWs with different training backgrounds and payment or incentive structures may also help policymakers determine optimal training guidelines and workforce composition.

Of note, this study took place in a program that provides quarterly refresher trainings and periodic individual supervisory visits, so these results may not apply to VHWs who receive initial training without further refresher training and supervision. In addition, this program has maintained an uninterrupted supply of RDTs and medications and exists in a setting in which VHWs regularly evaluate patients with fever and perform RDTs11; thus, these results may also not generalize to setting in which VHWs use these skills more sporadically.

This study has a number of limitations. First, it evaluates a relatively small number of VHWs, and only six RDTs per VHW, which limits the ability to assess modest changes in performance over time. Second, we were unable to assess performance immediately after initial iCCM training as a baseline comparison, so this study does not provide any information about performance in the first year. However, as detailed earlier, several prior studies have assessed this question. Third, this study compares two different cohorts of VHWs, rather than assessing the same VHWs at different time points. Although both groups of VHWs included in this study received similar training and work in similar settings, if baseline performance between the two groups differed, then this study design could fail to detect a change in performance over time. Fourth, although the use of a mock hand, rather than a real patient, for VHWs to perform RDTs obviates the need to interrupt if a critical step is missed, the unfamiliarity of this piece of the assessment could also have affected VHWs’ performance. Fifth, because a real patient was not used, we did not assess the steps related to recording the patient’s name and information. Sixth, the use of mock RDTs for VHWs to read does not fully reflect real-world issues such as faintly positive results, so this study could overestimate VHWs’ abilities in reading RDTs. Finally, because VHWs knew they were being observed during these pre-arranged sessions, the Hawthorne effect may have increased or decreased performance relative to their normal clinical practice.

In conclusion, VHWs appear to maintain their abilities to read RDTs accurately and perform RDTs according to protocol at least 4 years after initial iCCM training. Further research with a larger number of VHWs and in more real-world conditions can help confirm these preliminary findings.

ACKNOWLEDGMENTS

We are grateful to all the village health workers for volunteering their time both for this study and, more importantly, to care for their communities. We also thank Sarah Masika for all her work in administering the program in Bugoye.

REFERENCES

  • 1.

    Republic of Uganda Ministry of Health, 2015. National Village Health Teams (VHT) Assessment in Uganda. Available at: http://library.health.go.ug/publications/health-education/national-village-health-teamsvht-assessment-uganda. Accessed September 12, 2020.

    • Search Google Scholar
    • Export Citation
  • 2.

    Republic of Uganda Ministry of Health, 2010. Village Health Team Strategy and Operational Guidelines. Available at: http://library.health.go.ug/publications/health-education/village-health-team-strategy-and-operational-guidelines. Accessed September 12, 2020.

    • Search Google Scholar
    • Export Citation
  • 3.

    The Republic of Uganda Ministry of Health, 2010. Integrated Community Case Management of Childhood Malaria, Pneumonia and Diarrhoea: Implementation Guidelines. Available at: http://library.health.go.ug/publications/child-health-school-health/integrated-community-case-management-childhood-malaria. Accessed July 25, 2020.

    • Search Google Scholar
    • Export Citation
  • 4.

    Counihan H, Harvey SA, Sekeseke-Chinyama M, Hamainza B, Banda R, Malambo T, Masaninga F, Bell D, 2012. Community health workers use malaria rapid diagnostic tests (RDTs) safely and accurately: results of a longitudinal study in Zambia. Am J Trop Med Hyg 87: 5763.

    • Search Google Scholar
    • Export Citation
  • 5.

    Boyce MR, Menya D, Turner EL, Laktabai J, Prudhomme-O’Meara W, 2018. Evaluation of malaria rapid diagnostic test (RDT) use by community health workers: a longitudinal study in western Kenya. Malar J 17: 206.

    • Search Google Scholar
    • Export Citation
  • 6.

    Mukanga D, Babirye R, Peterson S, Pariyo GW, Ojiambo G, Tibenderana JK, Nsubuga P, Kallander K, 2011. Can lay community health workers be trained to use diagnostics to distinguish and treat malaria and pneumonia in children? Lessons from rural Uganda. Trop Med Int Health 16: 12341242.

    • Search Google Scholar
    • Export Citation
  • 7.

    Phiri TB, Kaunda-Khangamwa BN, Bauleni A, Chimuna T, Melody D, Kalengamaliro H, Sande JH, Nsona HK, Mathanga DP, 2016. Feasibility, acceptability and impact of integrating malaria rapid diagnostic tests and pre-referral rectal artesunate into the integrated community case management programme. A pilot study in Mchinji district, Malawi. Malar J 15: 177.

    • Search Google Scholar
    • Export Citation
  • 8.

    Harvey SA, Jennings L, Chinyama M, Masaninga F, Mulholland K, Bell DR, 2008. Improving community health worker use of malaria rapid diagnostic tests in Zambia: package instructions, job aid and job aid-plus-training. Malar J 7: 160.

    • Search Google Scholar
    • Export Citation
  • 9.

    Laktabai J, Platt A, Menya D, Turner EL, Aswa D, Kinoti S, O’Meara WP, 2018. A mobile health technology platform for quality assurance and quality improvement of malaria diagnosis by community health workers. PLoS One 13: e0191968.

    • Search Google Scholar
    • Export Citation
  • 10.

    Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG, 2009. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 42: 377381.

    • Search Google Scholar
    • Export Citation
  • 11.

    Miller JS 2018. Quality of care in integrated community case management services in Bugoye, Uganda: a retrospective observational study. Malar J 17: 99.

    • Search Google Scholar
    • Export Citation

Author Notes

Address correspondence to James S. Miller, Massachusetts General Hospital Global Health, 125 Nashua St., Suite 722, Boston, MA 02114. E-mail: jsmiller@post.harvard.edu

Financial support: This work was supported by the Mooney-Reed Charitable Foundation, Massachusetts General Hospital Department of Medicine, Doris Duke Charitable Foundation, and Harvard Medical School Center for Primary Care.

Authors’ addresses: James S. Miller, Jessica Kenney, and Geren S. Stone, Massachusetts General Hospital, Boston, MA, E-mails: jsmiller@post.harvard.edu, jkenney4@partners.org, and gstone@mgh.harvard.edu. Rapheal Kisolhu Mbusa, Stephen Baguma, and Michael Matte, Global Health Collaborative, Mbarara, Uganda, E-mails: raphealmbusa176@gmail.com, bagumastephen3@gmail.com, and mmatte@mustresearch.org. Palka Patel, Indiana University School of Medicine, Indianapolis, IN, E-mail: patelpr@iu.edu. Moses Ntaro and Edgar Mugema Mulogo, Mbarara University of Science and Technology, Mbarara, Uganda, E-mails: ntaro2001@gmail.com and emulogo2000@gmail.com. Shem Bwambale, Bugoye Health Center, Bugoye, Uganda, E-mail: shembwambale@yahoo.com. Daniel Guiles, Vanderbilt University School of Medicine, Nashville, TN, E-mail: guilesdan@gmail.com.

Save