The Impact of a Fogarty International Center-Supported Tuberculosis Research Training Program in the Country of Georgia

INTRODUCTION

The global burden of tuberculosis (TB) is unevenly distributed with most high-burden countries being low- to middle-income.¹ Many of these countries lack the capacity to conduct the research necessary for developing innovative and locally relevant strategies to adapt new TB tools.² To help guide the research agenda for low- to middle-income countries, the World Health Organization (WHO) recently released “A Global Action Framework for TB Research.” A key tenet is the need to increase and retain high-quality TB researchers to achieve national and global TB elimination.³

The primary goal of the National Institutes of Health (NIH) Fogarty International Center (FIC) is to support global health research and scientists and to build partnerships among research institutions and investigators in the United States and abroad. The Global Infectious Disease (GID) Research Training Program (RTP) is a program administered by the FIC that strives to build sustainable research capacity in low- and middle-income countries. Although the impact of other FIC programs has been reported,^4,5 no specific studies have focused on the impact of GID RTPs. Given that the FY18 U.S. President’s budget proposes to eliminate the FIC and the expanding ability of infectious diseases to cross borders, it is important to evaluate the impact of global health RTPs.^6,7

In 2004, we initiated a GID RTP focused on TB research and research training in the country of Georgia where TB, including drug-resistant disease, has been a major public health problem. The health care partnership began in the Spring of 1992 when the American International Health Alliance received funding from the United States Agency for International Development to establish partnerships between US medical institutions and health care facilities in the former Soviet Union (including Georgia). An Emory faculty collaborated on this initiative in Georgia and facilitated contact between Emory Division of Infectious Diseases faculty members with interest in human immunodeficiency virus (HIV) and TB and institutions in Georgia. At the time of beginning our GID RTP in 2004, the prevalence of TB in Georgia was estimated at 143 cases per 100,000 persons and multidrug resistance (MDR) was emerging as a major problem.⁸ In addition, there was limited research capacity at the National Center for Tuberculosis and Lung Diseases (NCTLD) as demonstrated by only one English language TB research–related publication.⁹ Through our program evaluation, our goals were to assess the impact of a formal and collaborative RTP on building research infrastructure and capacity for TB control in Georgia.

METHODS

The Emory–Georgia TB RTP was funded by the FIC in 2004 and has undergone two competitive renewals in 2010 and 2015. The primary in-country partner is the NCTLD in Tbilisi, with additional partners including the National Centers for Disease Control, the International acquired immunodeficiency syndrome (AIDS), and Clinical Immunology Research Center (IAIDRC), and local universities. Applicants to the RTP were required to have a doctoral degree. The aims of the first training cycle (2004–2009) included building in-country TB research capacity and enhancing opportunities for multidisciplinary TB-related research. The aims of the second cycle (2010–2015) expanded on the initial aims and transitioned more research training to in-country and enhanced research ethics training.

All long-term trainees were provided with didactic and mentored TB-related research training of at least 1-year duration (Table 1). The didactic training for the first grant cycle consisted of Fogarty trainees coming to the United States to obtain a Master of Public Health (MPH) or Master of Science in Clinical Research (MSCR) degree at Emory University. The didactic training for the second cycle transitioned to a training model focused on distance-learning courses in biostatistics, data management, and scientific writing provided through video conferencing by Emory faculty and in-country MPH programs. Some trainees were selected to come to the United States for “sandwich” training to take advanced courses at Emory University. Each trainee had a Georgian- and U.S.–based mentor. Each trainee was expected to submit their work for publication. Mentors were expected to provide mentoring in research design, study logistics, data analysis, and scientific writing, and to communicate with their trainees at least once per month. An annual research conference was held once a year in Georgia beginning in 2010.

Each trainee received a monthly stipend to protect ≥ 50% of their professional effort and a laptop computer. Support was provided to attend an international scientific meeting if the trainee was the first author on an accepted TB-related abstract. In-country language training was provided to trainees to enhance their spoken and written English skills as needed. All trainees were also provided with ongoing mentoring by their U.S. mentor to help with career development and facilitate further research efforts (Table 1).

RESULTS

A total of 20 long-term trainees participated in the RTP between 2004 and 2015, including 5 (25%) from the first and 15 (75%) from the second training period. The median age was 29 years; the majority (65%) were female and most were (60%) employed at the NCTLD (Table 2). Despite most (N = 18) having MD degrees, only 2 (10%) were practicing clinicians. Among the 20 trainees, 19 (95%) successfully completed 2 years of formal research training. During training, nine trainees received a master’s degree including six at Emory University, one at the University of Albany, and two from universities in Georgia. Two trainees completed the Centers for Disease Control and Prevention field epidemiology training program in Georgia. Only 1 (5%) trainee did not remain in their home country after training. Four trainees received their PhD degrees at a Georgian university after Fogarty training and four former trainees are currently being supported in pursuit of their PhD (three in Georgia and one in Switzerland).

Publications.

There have been 65 peer-reviewed publications by trainees since they entered Fogarty training. Among the 20 trainees, 15 (75%) authored or coauthored at least one peer-reviewed publication after starting the program. The median number of peer-reviewed publications per trainee was six (interquartile range 2–14) and among the 15 trainees with a publication was 13 (interquartile range 4–15). The mean cost per publication was $26,573 which equated to approximately 36 publications per one million dollars of grant costs.

The first trainee-related publication was in 2008 and afterward there was a steady increase throughout the evaluation period (Figure 1). A trainee was a first or last author on 34 of 65 (52%) publications. There was a median of three Georgian authors and two Fogarty trainees per publication. There was a substantial number of publications (N = 26, 40%) with non-Emory collaborators with most of these (N = 19) occurring since 2014. The manuscripts were published in 32 different scientific journals with a median impact factor of 4.4 (Table 3). There was a total of 616 citations resulting from 49 articles with data available (Figure 2). Among the 15 trainees who coauthored a manuscript, 14 (93%) published their Fogarty training-related research project and for 12 of the 15 (80%) it represented their first peer-reviewed manuscript. Almost all the trainees (14 of 15, 93%) who coauthored a manuscript were a first author, and all have coauthored a manuscript in the last 2 years.

Publications of long-term Georgian Fogarty trainees who received formal training between 2004 and 2015 (N = 65 peer-reviewed publications among 15 Fogarty trainees). This figure appears in color at www.ajtmh.org.

Citation: The American Journal of Tropical Medicine and Hygiene 98, 4; 10.4269/ajtmh.17-0667

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Publications of long-term Georgian Fogarty trainees who received formal training between 2004 and 2015 (N = 65 peer-reviewed publications among 15 Fogarty trainees). This figure appears in color at www.ajtmh.org.

Citation: The American Journal of Tropical Medicine and Hygiene 98, 4; 10.4269/ajtmh.17-0667

• Download Figure
• Download figure as PowerPoint slide

Publications of long-term Georgian Fogarty trainees who received formal training between 2004 and 2015 (N = 65 peer-reviewed publications among 15 Fogarty trainees). This figure appears in color at www.ajtmh.org.

Citation: The American Journal of Tropical Medicine and Hygiene 98, 4; 10.4269/ajtmh.17-0667

• Download Figure
• Download figure as PowerPoint slide

Citations per year from publications of long-term Georgian Fogarty trainees. This figure appears in color at www.ajtmh.org.

Citation: The American Journal of Tropical Medicine and Hygiene 98, 4; 10.4269/ajtmh.17-0667

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Citations per year from publications of long-term Georgian Fogarty trainees. This figure appears in color at www.ajtmh.org.

Citation: The American Journal of Tropical Medicine and Hygiene 98, 4; 10.4269/ajtmh.17-0667

• Download Figure
• Download figure as PowerPoint slide

Citations per year from publications of long-term Georgian Fogarty trainees. This figure appears in color at www.ajtmh.org.

Citation: The American Journal of Tropical Medicine and Hygiene 98, 4; 10.4269/ajtmh.17-0667

• Download Figure
• Download figure as PowerPoint slide

Table 3

List of journals for all Fogarty trainee-related publications (N = 65) and impact factor of these journals

Number	Journal	Impact factor*
1	AIDS	5.02
2	AIDS Research and Human Retroviruses	2.10
3	Annals of Thoracic Surgery	3.70
4	Antimicrobial Agents and Chemotherapy	4.30
5	American Journal of Clinical Nutrition	6.93
6	BMC Genomics	3.73
7	BMC Infectious Diseases	2.77
8	Bulletin of the World Health Organization	4.94
9	Clinical Chemistry	8.01
10	Clinical Infectious Diseases	8.22
11	Clinical Nutrition	4.55
12	Emerging Infectious Diseases	8.22
13	Eurosurveillance	7.20
14	European Respiratory Journal	10.57
15	Genome Announcements	NA
16	Georgian Medical News	NA
17	Health and Human Rights	1.51
18	HIV Medicine	3.26
19	Infection Control and Hospital Epidemiology	3.55
20	International Journal of Infectious Diseases	2.53
21	International Journal of Mycobacteriology	NA
22	International Journal of Tuberculosis and Lung Diseases	2.47
23	Journal of Clinical Microbiology	3.71
24	Journal of Infections in Developing Countries	NA
25	Lancet Infectious Diseases	19.86
26	Lancet HIV	9.84
27	MBio	6.96
28	Nutrition	3.42
29	Open Forum Infectious Diseases	NA
30	PLoS One	2.81
31	Public Health Action	NA
32	Scientific Reports	4.26
–	Median impact factor (interquartile range)	4.4 (3.4–8.0)

AIDS = acquired immunodeficiency syndrome; HIV = human immunodeficiency virus; BMC = BioMed Central; NA = not applicable.

^*2016 impact factors obtained from InCites Journal Citation Reports.

Research capacity.

The amount of annual TB research funding at the NCTLD has increased from $5,000 in 2005 to $371,000 in 2016 (Figure 3). A current or former Georgian Fogarty trainee was either a site principal investigator (PI) or coinvestigator on more than 90% of all research funding obtained at the NCTLD. In 2017, the anticipated TB research funds are an estimated $1.5 million.

Tuberculosis (TB)-related research funding received by the National Center for Tuberculosis and Lung Disease, Tbilisi, Georgia (country) (2005–2016). Not including funds received from the Global Fund to Fight AIDs, Tuberculosis, and Malaria. This figure appears in color at www.ajtmh.org.

Citation: The American Journal of Tropical Medicine and Hygiene 98, 4; 10.4269/ajtmh.17-0667

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Tuberculosis (TB)-related research funding received by the National Center for Tuberculosis and Lung Disease, Tbilisi, Georgia (country) (2005–2016). Not including funds received from the Global Fund to Fight AIDs, Tuberculosis, and Malaria. This figure appears in color at www.ajtmh.org.

Citation: The American Journal of Tropical Medicine and Hygiene 98, 4; 10.4269/ajtmh.17-0667

• Download Figure
• Download figure as PowerPoint slide

Tuberculosis (TB)-related research funding received by the National Center for Tuberculosis and Lung Disease, Tbilisi, Georgia (country) (2005–2016). Not including funds received from the Global Fund to Fight AIDs, Tuberculosis, and Malaria. This figure appears in color at www.ajtmh.org.

Citation: The American Journal of Tropical Medicine and Hygiene 98, 4; 10.4269/ajtmh.17-0667

• Download Figure
• Download figure as PowerPoint slide

The majority of former trainees (N = 15, 75%) remain engaged in research with 14 involved in TB-related research. Ten (50%) trainees have been the PI and recipient of at least one external peer-reviewed research grant after training. The first investigator initiated TB-related randomized controlled trial in Georgia was led by five former Fogarty trainees in collaboration with Emory University and carried out during the observation period. One former trainee found and currently directs the NCTLD TB Research Unit, which employs five staff (including four former trainees) and oversees five ongoing TB clinical trials, including three led by former Fogarty trainees. Active ongoing TB randomized controlled trials include the Stream II (NCT02409290), NiX-TB (NCT02333799), and endTB (NCT02754765) studies.

Eight (40%) former Fogarty trainees serve as research mentors for a current Fogarty trainee. Five former trainees teach epidemiology and/or biostatistics courses at Georgian universities. In addition, one former trainee served as a course leader for the Structured Operational Research and Training Initiative operational research training course.¹⁰

DISCUSSION

Our NIH FIC-supported GID TB RTP helped establish a cadre of highly trained clinical and translational researchers and increased research capacity in Georgia. Through a combination of didactic research training, intensive mentored training, resource support, and protected time for research training, most (75%) of our 20 long-term Georgian trainees completed and published their research and have continued to be engaged in clinical and translational research. Fifty percent of our Georgian trainees have gone on to secure external funding for TB-related research as a PI. Additional accomplishments of our trainees and their institutions include the following: 1) an increasing amount of research collaboration among Georgian Fogarty trainees and other in-country researchers and international partners; 2) the increase in conduct of clinical research including randomized controlled trials; 3) a marked increase in research funding at our primary collaborating institution from nearly no research support to an estimated $1.5 million in 2017; and 4) the development of in-country research mentoring, led by former Fogarty trainees. Our performance metrics demonstrate the high return on investment of RTPs in low- to middle-income countries and their downstream benefits.⁷ Further evaluations such as ours will be beneficial in terms of identifying key components of RTPs that are successful across different settings and will help to develop optimal and standard methods to evaluate global health RTPs.

We used the publication of manuscripts as our main performance metric given the important role of publishing in disseminating scientific findings, its common use as a metric for RTPs, and its ease of measurement.^4,11 Overall, our former Georgian Fogarty trainees had 65 TB-related peer-reviewed publications during the evaluation period. The number of publications by Fogarty trainees and the impact factor, and citations of these publications suggest the success of our RTP in terms of scholarly productivity and indicate of the high-quality and breadth of research conducted. The majority of trainees (75%) published their Fogarty-related research project, were a first author, and for most, it represented their first peer-reviewed publication. Slightly more than half (52%) of our trainee publications were first-author publications. A unique aspect of our evaluation is the long-term follow-up of research trainees, which demonstrated the sustainability of our RTP. Maintaining contact with former trainees allowed for further collaborations between U.S. and Georgian investigators and continued mentoring and inclusion of former trainees in the leadership of the RTP. We believe these factors helped contribute to the high median number of publications per trainee, further publications after their training period including within the last 2 years, and by continued engagement in mentoring and research activities. These findings demonstrate the long-term benefits of a formal RTP and the potential jumpstart that a research publication can provide for an academic career.

Additional insights can be gained from the long time lag between initiating research training in 2004 and realizing scholarly output. This was likely due to multiple reasons, including the nonexistent research infrastructure at the time of initiating our RTP in 2004, an early training model where a few trainees received costly training in the United States, and the long duration required to complete clinical research and the publication process.¹² These delays in scholarly output are important in terms of performance evaluation as it stresses the need for long-term investments when determining whether efforts at building research capacity are successful. The increasing amount of research publications over time with non-Emory collaborators provided evidence that trainees were gaining academic independence and developing new productive collaborations. Publications with new collaborators can thus be used as one possible indicator of a sustainable research environment and may be useful in evaluating other RTPs.

The high rates of current engagement in research (75%) by former trainees, promotion to a leadership position (45%), and receipt of a peer-reviewed research grant (50%) are all important additional indicators of impact and key components of research capacity building. They demonstrate that most of our trainees went far beyond publishing their Fogarty-related TB research project to earn key leadership positions and obtain further funding to continue and expand their research activities. The development of a TB research unit at the Georgian NCTLD which is led and staffed by former Fogarty trainees is an excellent example of how sustainable research capacity building programs can develop when there exists a trained and talented group of researchers. The development of a research unit staffed by well-trained researchers (most through the Fogarty-supported program) along with key support and funding provided by the Georgian Ministry of Health and the Global Fund to Fight AIDS, Tuberculosis, and Malaria has made the NCTLD a highly desired site for externally supported research including randomized controlled trials, especially trials focused on MDR-TB, which is a major public health problem in Georgia.^13,14 The NCTLD is now participating in six ongoing randomized clinical trials and has increased their research funding 200-fold since the beginning of Fogarty-supported RTP. The indirect costs from these trials have been essential to support a research unit.

In the second cycle of funding, we worked on transitioning training to primarily in-country, which allowed us to markedly increase the number of long-term Fogarty trainees we could support. Trainees supported during the second cycle remained productive with regard to scholarly productivity and our other metrics. The technology for delivering high-quality distance learning including good internet connectivity is also readily available making the transition to an in-country training model very feasible for many settings, including Georgia.

Our experience also highlights a number of challenges in carrying out a RTP in a low- or middle-income country including the recruitment of clinician scientists, lack of protected time for research, limited in-country mentorship, and language barriers. The decline in clinician scientists has been documented in the United States, but has not been characterized in low- and middle-income settings.¹⁵ The high demands of clinical practice in Georgia along with a lack of a culture prioritizing and providing time for clinical research and lack of physician–scientist role models in Georgia are all factors that make it difficult for clinicians to engage in research. Low salaries also require most TB clinicians to perform additional clinical work outside their regular job. We have experienced similar issues in another ongoing Fogarty-supported GID TB RTP in Ethiopia, and we suspect these challenges are common to most low- and middle-income countries. The importance of clinician scientists in translating research findings into improvements in clinical care has been well recognized and necessitates the urgent need for innovative solutions to further their participation in research especially in low- and middle-income countries.^15,16

The lack of senior investigators who can serve as in-country role models to inspire and guide trainees toward a successful clinical research career has been a major challenge. We have found that former Fogarty trainees have become the best and most available in-country mentors for newly recruited trainees, and therefore, a major focus of our third cycle of Fogarty funding is mentorship training of former trainees. Mentorship training workshops have been implemented for both in-country mentors and current Fogarty trainees. The curriculum for mentorship training has been adapted from the US-based National Research Mentoring Network funded by the NIH.¹⁷ Some of our performance metrics such as the number of Georgian Fogarty trainee coauthors per publication and the 40% of trainees who are currently engaged in mentoring are promising early indicators of improved in-country mentorship capacity.

To be successful as a scientific investigator, fluency in English is required.¹⁸ The publishing of research manuscripts for nonnative English speakers places them at a disadvantage in getting their research published and thus recognized by the scientific community.¹⁸ The lack of English speaking and writing skills was considered an important barrier for two of our trainees who did not publish their Fogarty research project and this prompted us to enhance our efforts and training in English language skills both before and during research training. In addition to providing extensive mentoring in manuscript preparation and editing, we added a semester-long distance-learning course in scientific and grant writing during the second grant cycle and offered support for one-on-one in-country English language tutoring. Although these efforts have resulted in significant improvements in communication and writing ability and were a good investment, the English language barrier remains a major challenge for trainees to publish their work and write successful grants.

Lastly, our findings confirm the importance of RTPs in building research capacity in resource-limited settings and highlight the need to continue such programs. The current U.S. administration’s fiscal year 2018 budget proposal eliminates funding for the FIC and would end programs such as ours that are essential to building research workforce capacity in countries where infectious disease burden including drug-resistant TB is high.^19,20 Our high rate of publications per million dollars spent demonstrates a high return on investment and supports further funding for global health.⁷ In addition, our GID RTP program also provided invaluable research experiences in TB for U.S. clinical researchers including Emory University medical faculty, infectious disease fellows, internal medicine residents, and medical, PhD, and MPH students. An analysis of the experiences and productivity among this group of U.S. researchers including publication and funding metrics is ongoing and we expect this future report will demonstrate the many indirect benefits of our GID RTP for involved U.S. researchers.

In summary, our results show the direct and indirect benefits of a formal RTP in terms of scholarly output, career growth opportunities, and enhancing research capacity in a high-burden TB setting. More emphasis should be placed on training the next generation of TB researchers in low- and middle-income countries to create self-sustaining research communities where TB is most prevalent.