• View in gallery
    Figure 1.

    Pre- and post-fellowship self-assessment scores by competency. P value < 0.05 for change in self-assessed scores for all competencies.

  • 1.

    Drain PK, Primack A, Hunt DD, Fawzi WW, Holmes KK, Gardner P, 2007. Global health in medical education: a call for more training and opportunities. Acad Med 82: 226230.

    • Search Google Scholar
    • Export Citation
  • 2.

    Fogarty International Center, 2014. Global Health Program for Fellows and Scholars. Available at: http://www.fic.nih.gov/Programs/Pages/scholars-fellows-global-health.aspx. Accessed May 12, 2014.

    • Search Google Scholar
    • Export Citation
  • 3.

    Quinn TC, 2008. The Johns Hopkins Center for Global Health: transcending borders for world health. Acad Med 83: 134142.

  • 4.

    Stapleton FB, Wahl PW, Norris TE, Ramsey PG, 2006. Addressing global health through the marriage of public health and medicine: developing the University of Washington department of global health. Acad Med 81: 897901.

    • Search Google Scholar
    • Export Citation
  • 5.

    Haq C, Baumann L, Olsen CW, Brown LD, Kraus C, Bousquet G, Conway J, Easterday BC, 2008. Creating a center for global health at the University of Wisconsin-Madison. Acad Med 83: 148153.

    • Search Google Scholar
    • Export Citation
  • 6.

    Bryant JH, Velji A, 2011. Global health and the role of universities in the twenty-first century. Infect Dis Clin North Am 25: 311321.

  • 7.

    Anandaraja N, Hahn S, Hennig N, Murphy R, Ripp J, 2008. The design and implementation of a multidisciplinary global health residency track at the Mount Sinai School of Medicine. Acad Med 83: 924928.

    • Search Google Scholar
    • Export Citation
  • 8.

    Gladding S, Zink T, Howard C, Campagna A, Slusher T, John C, 2012. International electives at the university of Minnesota global pediatric residency program: opportunities for education in all Accreditation Council for Graduate Medical Education competencies. Acad Pediatr 12: 245250.

    • Search Google Scholar
    • Export Citation
  • 9.

    Bloomfield GS, Huffman MD, 2010. Global chronic disease research training for fellows: perspectives, challenges, and opportunities. Circulation 121: 13651370.

    • Search Google Scholar
    • Export Citation
  • 10.

    Martin IB, Jacquet GA, Levine AC, Douglass K, Pousson A, Dunlop S, Khanna K, Bentley S, Tupesis JP, 2013. Global health and emergency care: a postgraduate medical education consensus-based research agenda. Acad Emerg Med 20: 12331240.

    • Search Google Scholar
    • Export Citation
  • 11.

    Calhoun JG, Spencer HC, Buekens P, 2011. Competencies for global health graduate education. Infect Dis Clin North Am 25: 575592.

  • 12.

    Ablah E, Biberman DA, Weist EM, Buekens P, Bentley ME, Burke D, Finnegan JR, Flahault A, Frenk J, Gotsch AR, Klag MJ, Rodriguez Lopez MH, Nasca P, Shortell S, Spencer HC, 2014. Improving global health education: development of a Global Health Competency Model. Am J Trop Med Hyg 90: 560565.

    • Search Google Scholar
    • Export Citation
  • 13.

    Gruppen LD, Mangrulkar RS, Kolars JC, 2012. The promise of competency-based education in the health professions for improving global health. Hum Resour Health 10: 43.

    • Search Google Scholar
    • Export Citation
  • 14.

    Hagopian A, Spigner C, Gorstein JL, Mercer MA, Pfeiffer J, Frey S, Benjamin L, Gloyd S, 2008. Developing competencies for a graduate school curriculum in international health. Public Health Rep 123: 408414.

    • Search Google Scholar
    • Export Citation
  • 15.

    Pfeiffer J, Beschta J, Hohl S, Gloyd S, Hagopian A, Wasserheit J, 2013. Competency-based curricula to transform global health: redesign with the end in mind. Acad Med 88: 131136.

    • Search Google Scholar
    • Export Citation
  • 16.

    Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG, 2009. Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 42: 377381.

    • Search Google Scholar
    • Export Citation
  • 17.

    Holm S, 1979. A simple sequentially rejective multiple test procedure. Scand J Stat 6: 6570.

  • 18.

    StataCorp, 2013. Stata Statistical Software: Release 13. College Station, TX: Statacorp LP.

  • 19.

    Driessen E, van Tartwijk J, van der Vleuten C, Wass V, 2007. Portfolios in medical education: why do they meet with mixed success? A systematic review. Med Educ 41: 12241233.

    • Search Google Scholar
    • Export Citation
Past two years Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 330 141 14
PDF Downloads 127 45 2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 

 

 

The Development and Implementation of a Competency-Based Curriculum for Training in Global Health Research

Thanh G. N. TonDepartments of Neurology and Global Health, University of Washington, Seattle, Washington; Department of Pediatrics, University of Minnesota, Minneapolis, Minnesota; Department of Medicine (Infectious Disease), University of Washington, Seattle, Washington; Department of Tropical Medicine, Medical Microbiology and Pharmacology, John A. Burns School of Medicine, University of Hawaii at Manoa, Honolulu, Hawaii; Global Research, Education and Collaboration in Health (REACH) and Departments of Learning Health Sciences and Internal Medicine, University of Michigan, Ann Arbor, Michigan

Search for other papers by Thanh G. N. Ton in
Current site
Google Scholar
PubMed
Close
,
Sophia P. GladdingDepartments of Neurology and Global Health, University of Washington, Seattle, Washington; Department of Pediatrics, University of Minnesota, Minneapolis, Minnesota; Department of Medicine (Infectious Disease), University of Washington, Seattle, Washington; Department of Tropical Medicine, Medical Microbiology and Pharmacology, John A. Burns School of Medicine, University of Hawaii at Manoa, Honolulu, Hawaii; Global Research, Education and Collaboration in Health (REACH) and Departments of Learning Health Sciences and Internal Medicine, University of Michigan, Ann Arbor, Michigan

Search for other papers by Sophia P. Gladding in
Current site
Google Scholar
PubMed
Close
,
Joseph R. ZuntDepartments of Neurology and Global Health, University of Washington, Seattle, Washington; Department of Pediatrics, University of Minnesota, Minneapolis, Minnesota; Department of Medicine (Infectious Disease), University of Washington, Seattle, Washington; Department of Tropical Medicine, Medical Microbiology and Pharmacology, John A. Burns School of Medicine, University of Hawaii at Manoa, Honolulu, Hawaii; Global Research, Education and Collaboration in Health (REACH) and Departments of Learning Health Sciences and Internal Medicine, University of Michigan, Ann Arbor, Michigan

Search for other papers by Joseph R. Zunt in
Current site
Google Scholar
PubMed
Close
,
Chandy JohnDepartments of Neurology and Global Health, University of Washington, Seattle, Washington; Department of Pediatrics, University of Minnesota, Minneapolis, Minnesota; Department of Medicine (Infectious Disease), University of Washington, Seattle, Washington; Department of Tropical Medicine, Medical Microbiology and Pharmacology, John A. Burns School of Medicine, University of Hawaii at Manoa, Honolulu, Hawaii; Global Research, Education and Collaboration in Health (REACH) and Departments of Learning Health Sciences and Internal Medicine, University of Michigan, Ann Arbor, Michigan

Search for other papers by Chandy John in
Current site
Google Scholar
PubMed
Close
,
Vivek R. NerurkarDepartments of Neurology and Global Health, University of Washington, Seattle, Washington; Department of Pediatrics, University of Minnesota, Minneapolis, Minnesota; Department of Medicine (Infectious Disease), University of Washington, Seattle, Washington; Department of Tropical Medicine, Medical Microbiology and Pharmacology, John A. Burns School of Medicine, University of Hawaii at Manoa, Honolulu, Hawaii; Global Research, Education and Collaboration in Health (REACH) and Departments of Learning Health Sciences and Internal Medicine, University of Michigan, Ann Arbor, Michigan

Search for other papers by Vivek R. Nerurkar in
Current site
Google Scholar
PubMed
Close
,
Cheryl A. MoyerDepartments of Neurology and Global Health, University of Washington, Seattle, Washington; Department of Pediatrics, University of Minnesota, Minneapolis, Minnesota; Department of Medicine (Infectious Disease), University of Washington, Seattle, Washington; Department of Tropical Medicine, Medical Microbiology and Pharmacology, John A. Burns School of Medicine, University of Hawaii at Manoa, Honolulu, Hawaii; Global Research, Education and Collaboration in Health (REACH) and Departments of Learning Health Sciences and Internal Medicine, University of Michigan, Ann Arbor, Michigan

Search for other papers by Cheryl A. Moyer in
Current site
Google Scholar
PubMed
Close
,
Nicole HobbsDepartments of Neurology and Global Health, University of Washington, Seattle, Washington; Department of Pediatrics, University of Minnesota, Minneapolis, Minnesota; Department of Medicine (Infectious Disease), University of Washington, Seattle, Washington; Department of Tropical Medicine, Medical Microbiology and Pharmacology, John A. Burns School of Medicine, University of Hawaii at Manoa, Honolulu, Hawaii; Global Research, Education and Collaboration in Health (REACH) and Departments of Learning Health Sciences and Internal Medicine, University of Michigan, Ann Arbor, Michigan

Search for other papers by Nicole Hobbs in
Current site
Google Scholar
PubMed
Close
,
Molly McCoyDepartments of Neurology and Global Health, University of Washington, Seattle, Washington; Department of Pediatrics, University of Minnesota, Minneapolis, Minnesota; Department of Medicine (Infectious Disease), University of Washington, Seattle, Washington; Department of Tropical Medicine, Medical Microbiology and Pharmacology, John A. Burns School of Medicine, University of Hawaii at Manoa, Honolulu, Hawaii; Global Research, Education and Collaboration in Health (REACH) and Departments of Learning Health Sciences and Internal Medicine, University of Michigan, Ann Arbor, Michigan

Search for other papers by Molly McCoy in
Current site
Google Scholar
PubMed
Close
, and
Joseph C. KolarsDepartments of Neurology and Global Health, University of Washington, Seattle, Washington; Department of Pediatrics, University of Minnesota, Minneapolis, Minnesota; Department of Medicine (Infectious Disease), University of Washington, Seattle, Washington; Department of Tropical Medicine, Medical Microbiology and Pharmacology, John A. Burns School of Medicine, University of Hawaii at Manoa, Honolulu, Hawaii; Global Research, Education and Collaboration in Health (REACH) and Departments of Learning Health Sciences and Internal Medicine, University of Michigan, Ann Arbor, Michigan

Search for other papers by Joseph C. Kolars in
Current site
Google Scholar
PubMed
Close

The Fogarty International Center (FIC) Global Health Fellows Program provides trainees with the opportunity to develop research skills through a mentored research experience, increase their content expertise, and better understand trends in global health research, funding organizations, and pathways to generate support. The Northern Pacific Global Health Fellows Research and Training Consortium, which hosts one of the FIC Global Health Programs, sought to enhance research training by developing, implementing, and evaluating a competency-based curriculum that uses a modular, asynchronous, web-based format. The curriculum has 8 core competencies, 36 learning objectives, and 58 assignments. Nineteen trainees completed their 11-month fellowship, engaged in the curriculum, and provided pre- and post-fellowship self-assessments. Self-assessed scores significantly improved for all competencies. Trainees identified the curriculum as one of the strengths of the program. This competency-based curriculum represents a first step toward creating a framework of global health research competencies on which further efforts could be based.

Introduction

The need to address health inequities and the burden of disease in under-resourced regions of the world have generated increasing interest in global health among trainees, academic institutions, and funding organizations.15 This has, in turn, stimulated investments in educational programs to develop global health researchers.410 The Fogarty International Center (FIC) at the National Institutes of Health (NIH) is focused on training the next generation of global health scientists and clinical investigators through a wide variety of programs.2 One of these is the Global Health Fellows Program, which consists of a network of United States (US) academic institutions in collaboration with international host institutions that provide early-career health science professionals (e.g., physicians, scientists, veterinarians, and dentists) with an 11-month mentored research experience in a low- or middle-income country (LMIC).

The cornerstone of training in the Fogarty Global Health Fellows Program is a mentored research project in an LMIC setting. These mentored research experiences provide trainees with the opportunity to develop and hone their research skills. To become successful investigators, trainees must develop their content expertise while also understanding trends in global health research, funding organizations, and pathways to generate support for global health research. The establishment of a formal learning plan or curriculum to accompany the mentored research experience designed to address these important issues can ensure that trainees receive instruction and guidance in these areas. This may help trainees to make a successful transition to global health researchers. However, the development of curricula has been limited for several reasons. First, although competency-based curricula embedded within medical and graduate programs have emerged as an important educational framework in the field of global health,1,8,1115 a curriculum that is competency-based, focusing on what trainees are expected to do with what they are learning, has yet to be proposed for global health research. Second, trainees are often geographically dispersed, yielding insufficient numbers to justify hosting a local course and making participation in a real-time web-based tutorial challenging because of variances in time zone. Finally, mentors may lack the time and resources to ensure that trainees acquire the macrolevel skills necessary to be successful in global health.

The Northern Pacific Global Health Fellows (NPGHF) Research and Training Consortium sought to fill this gap in preparing future global health researchers. This paper describes the development, initial implementation, and early evaluation of a competency-based curriculum that trainees can pursue using a modular, asynchronous, web-based format.

Materials and Methods

Setting.

The NPGHF Research and Training Consortium comprising co-principal investigators from global health units at the University of Washington, the University of Michigan, the University of Minnesota, and the University of Hawaii, was established in 2012 with a 5-year grant from the FIC of NIH.2 This represents one of five consortia with a purpose to provide focused mentoring, training, and education in global health research to trainees over an 11-month period based at an LMIC setting where the consortium schools have active research programs. Applicants respond to an annual call for applications for positions as fellows if they have attained a doctoral degree (e.g., PhD, MD, DVM), or as scholars if enrolled in pre-doctoral training programs. Although primarily designed for US applicants to gain research experience in international settings, this program also has the goal of providing training to international applicants from LMICs. This paper is based on experience with the first-year NPGHF cohort, which included 19 trainees (5 US fellows, 8 international fellows, 2 US scholars, and 4 international scholars) in the 2012–2013 cycle. Nearly three-quarters (N = 14) were post-doctoral fellows, and 63% (N = 12) were from LMICs. In our initial cohort, fellows and scholars worked in China (N = 2), Ghana (N = 3), Peru (N = 3), Kenya (N = 4), Uganda (N = 5), and Thailand (N = 2).

Development of curriculum.

Development of the curriculum was led by the four co-principal investigators of the NPGHF Research and Training Consortium (J.R.Z., C.J., V.R.N., and J.C.K.), who all have extensive experience in conducting global health research and supervising global health trainees. Curriculum development was guided by several principles: (1) the content should support development of successful global health researchers, (2) experiential learning from actual doing in addition to critical thinking is essential in global health education,15 and (3) trainees are geographically dispersed, making collective learning opportunities difficult. Guided by these principles, the team reviewed existing literature on global health competencies and curricula, and held a series of weekly discussions to reach consensus on the core competencies for global health research, the learning activities to support development in each competency, and the process for delivering the curriculum. These discussions were also informed by colleagues at each of the institutions, who provided perspective from the disciplines of education (S.P.G.) and public health (T.G.N.T. and C.A.M.). The authors recognized that learners were coming from a variety of starting points, both here and abroad, which limited any assumptions that could be made as to what they already knew about research. For that reason, a more holistic approach to determine global health research curriculum and competencies was chosen, inclusive of aspects that would be common to all research pursuits.

Through this process, the principal investigators reached consensus on eight core competencies: (1) define global health and global health research, (2) know the major trends in global health, (3) know organizations that fund global health research and pathways to achieve support, (4) acquire research skills necessary for global health research, (5) communicate research proposals and results, (6) acquire and apply skills at managing research teams and processes, (7) understand responsible conduct of research and collaborative practice, and (8) understand professionalism, professional development, and cultivation of mentorship. Each competency is accompanied by two to seven learning objectives that guide trainees toward achieving competence in that domain (Table 1). Key learning activities and assignments requiring submission of documents, termed artifacts, by trainees were defined for each learning objective. Examples of activities include reading and writing assignments, creating presentations, drafting research proposals, attending conferences, and reflecting the principle of learning through doing. The curriculum was designed to be completed in a modular, asynchronous manner by trainees using a web-based platform, with learning supported through online lectures, participation in discussion boards, live facilitated web conferencing, and one-on-one meetings with mentors, reflecting the principle of distributed learning.

Table 1

Core competencies, learning objectives, and artifacts for global health research

Core competency Learning objectives Assignments (i.e., artifacts to be submitted to portfolio)
Define global health and explain what constitutes global health research A. Develop personal definition of global health. B. Explain concept of global health research. C. Cite common references, textbooks, and compendia on global health. A. If somewhere were to ask you, “what is global health,” how would you respond? Write a brief essay ≤ 1 page. B. Could any international research be considered “global health” research? Write a brief essay ≤ 1 page that states your own working definition of global health research. C. Write a brief essay ≤ 1 page responding to the following: if a student were to stop you and say “where can I get some information about the field of global health,” where would you refer them?
Know major trends in global health A. Have basic understanding of research themes around major disease and global health issues: HIV, TB, malaria, maternal and child health, neglected tropical diseases, chronic non-communicable disease, diseases in travelers and migrating populations. B. Explain what is meant by social determinants of health. C. Describe the concepts of population health and burden of disease. D. Describe concepts of capacity building and health system strengthening. E. Explain implementation science. F. Report on major meetings where global health research is discussed and the major journals where findings are published. A. Conduct a focused literature search in one of five context areas; provide a short (2–4 pages) essay on one of the listed content areas that describes key research, traditional interventions, or programmatic efforts, at least one controversial aspect within the content area, and literature in support of contrasting viewpoints. Develop presentation (≥ 10 slides). Critique assays from three trainees. B. Develop presentation (3–5 slides) describing social determinants of health that contribute to the content area chosen in A above. Write a short (2–4 pages) essay on social determinants of health that affect your area of research; mention at least two relevant interventions from literature. Critique essays or presentations from three trainees. C. Develop a presentation (3–5 slides) explaining the burden of disease for the focused content area in A above. Write a brief (1–2 pages) essay comparing the variance in the disease or problem that is the focus of your research in other countries or regions. Write a brief (1–2 pages) essay on the Millennium Development Goals and the forecast for reaching it by 2015. What are current obstacles and research questions that could address these obstacles? D. Provide a written example (< 2 pages) of a capacity building or health system strengthening intervention from at least one country or region, other than where you are working, that addresses the focused content topic of your research. E. Provide a written example (< 1 page) of how a scientific finding in the content topic of your research would be a good example of implementation science. In a brief (< 2 page essay), describe why implementation science is particularly important to global health and how some of your research findings might be further advanced through implementation science. F. Provide a list of at least three major global health meetings that include global health research that you would like to attend over the next 5 years (assuming no cost or travel barriers). Create a list of three to five journals (with impact factors) where your global health research could be published.
Know organizations funding global health research and pathways to achieve support A. Understand NIH structure and approaches to obtaining NIH funding for global health research. B. Describe other governmental funding entities and their approach to funding. A. Provide a 1-page summary of which NIH institute funds research in your area of interest. Include the institute's funding priorities and a brief listing of recent RFAs related to global health research to support your choice. B. Provide a 1-page description of a US or LMIC governmental funding agency other than the NIH that would support research projects in your area of interest. Provide funding priorities/RFAs/other documentation to support your choice.
Acquire research skills necessary for global health research A. Define a general research agenda and a more specific research study. B. Frame a research question. C. Perform a literature review on a research topic of interest. D. Write up a research stud with clearly stated hypotheses and study design. E. Design an efficient and useful data collection and management system using Excel or Access. F. Show proficiency at basic biostatistics and how to access input on more advanced methodologies. G. Develop a realistic timeline for proposal, including steps for submission to NIH. A. Write a 1-page summary of your thoughts on a general research agenda for your next 5 years and research studies that might allow you to achieve this agenda. B. Write up your primary research question for your project(s). C. Conduct a literature review relative to the research project proposal. D. Write a research study proposal with hypotheses and study design. E. Create a data collection instrument for your study that allows easy management and analysis of data. Create a database using Excel or Access for your study. F. Completion of online or in-person data management and biostatistics course. G. Develop a timeline specific to your proposal that would result in a successful submission for funding to continue your work.
Communicate research proposal and results A. Show proficiency at scientific writing. B. Show proficiency at scientific presentations (oral and poster). C. Provide meaningful critique of scientific writing and presentations. A. Write a succinct 2,000-character abstract based on your research for presentation at a national meeting and a draft manuscript based on your research findings that can be submitted to a high-impact peer-reviewed journal. Also, provide a brief essay (< 1 page) regarding the international norms and policies for the determination of authorship. B. Prepare a PowerPoint presentation of a project(s) for presentation at a talk or major meeting. Obtain feedback from peers on slide presentation. Prepare a poster presentation of a project(s) for presentation at a major meeting. C. Prepare written critiques of two scientific documents (e.g., papers, abstracts, grant proposals) assigned by the mentors. Ideally, this might include review of a paper a mentor was asked to review for a journal. Provide examples of written feedback provided to Fogarty peers.
Acquire and apply skills at managing a research team and processes A. Understand the important aspects of establishing and managing a laboratory or research unit. B. Understand the rationale, importance, and issues associated with development of biorepositories. C. Understand the supply chain in international procurement and how to create realistic budgets. D. Show the understanding of teamsmanship. E. Show issues related to conflict management. A. Write a 1-page description of the ideal research team that you would like to lead in 5 years. B. Write a 1-page description of a biorepository that you would like to have. C. Create a realistic budget for your project and submit for review. In a 1-page document, summarize some of the supply chain and procurement challenges that you have at your site/for your project. What would you do if a supply was unavailable or if your project was delayed by an unforeseen circumstance? Discuss your thoughts with your mentorship team. D. Cite an example of an exemplary team that you have observed. In 1 page or less, state what made them particularly successful. E. Cite an example of a situation involving a conflict. In 1 page or less, state how that conflict was managed. What was gained from the conflict and its resolution? Did the conflict involve intercultural issues?
Understand responsible conduct of research and collaborative practice A. Complete an IRB application for a study involving human subjects or animals. B. Explain concepts of research ethics and integrity. C. Understand how cultural differences between investigators or between investigators and their patients can impact research. A. Complete and submit an IRB application. Develop an informed consent form (or application for animal use). Critique two to three consent forms (or applications for animal use). Attend an IRB meeting, review minutes from the meeting, and prepare a 1- to 2-page essay reflecting on how the processes enhanced proper protection of the ethical use of human subjects or animals in research. B. Provide certification of completion of a research ethics course Collaborative Institutional Training Initiative. Prepare a 1- to 2-page essay on a case study of research fraud or poor research ethics relevant to the research in which you are involved. What were the pressures that contributed to the problem? What cultural or institutional practices could have been minimized this outcome? C. Write up a brief (< 1 page) description of how a cultural difference was handled either particularly well or poorly.
Understand professionalism, professional development, and cultivation of mentorship A. Understand the changing landscape of professionalism in an online, cross-cultural world. B. Networking and making the most of scientific meetings to further your professional career. C. Explain what good mentoring is, how you can get it, and how you can develop mentoring skills. A. Write a brief essay (≤ 1 page) describing the following questions. If a reporter were to ask you “why should we devote taxpayer money to studying [your area of research]?,” how would you respond? For US trainees, how would you defend conducting research with US taxpayer funding outside of the United States? Create a list of your social networking sites and types of people who have access to these sites (e.g., colleagues, friends, peers, patients). Create a 5- to 1-year personal development plan and comment on how you envision becoming a “change agent” to improve health in your area of interest. B. Prepare a reflection on networking at a scientific meeting attended. What is your “elevator speech” to succinctly convey your research interests in 1–3 minutes? Where were the best places to network (over abstract presentations, at meals, during plenary sessions)? Create a list of three to five individuals (names, positions, emails) who you would like to contact regarding research or professional development. Record a video of you presenting research findings or a didactic presentation. This should also include a reflection on your performance: what you did well and areas for improvement. C. Complete the mentoring plan and pact. Write a brief essay (≤ 1 page) stating your definition of what good mentoring entails and how you propose to proactively seek good mentoring. Participate in mentoring meetings and online/in-person mentor training events.

HIV = human immunodeficiency virus; TB = tuberculosis; RFA = request for applications.

Implementation.

A web-based template was developed to provide a means through which trainees could show competence in core competencies and interact with their US-based mentors while being geographically dispersed. Each trainee's established electronic portfolio is a personal website comprised of webpages dedicated to each core competency. Google Sites was selected as the platform for the electronic portfolio, because it is widely accessible across the consortium, is free to obtain for trainees, and can be maintained by the trainee after completing the fellowship, providing a portable collection of evidence of their global health research competence. Each portfolio contains a virtual file cabinet where trainees can upload and display their artifacts along with an interactive space for asynchronous discussion between trainees and their US-based mentors. Trainees were instructed to submit artifacts (i.e., completed assignments) to their electronic portfolio.

The complete curriculum, including assignments, was placed on a central password-protected internal webpage on the primary website for the fellowship (www.fogartyfellows.org). All trainees attended a virtual orientation to the curriculum and use of the portfolio through Adobe Connect. Because the curriculum was not implemented until the first cohort of trainees was, on average, 4–5 months into their fellowship, trainees in the initial cohort were encouraged to participate in the curriculum, but their participation was not a requirement of the fellowship. All trainees successfully established an electronic portfolio. In the few instances where access to specific websites was restricted, the program was able to upload artifacts on the trainee's behalf.

Determining the use and perceived value of the curriculum.

Trainees' participation in the curriculum was measured using two outcomes: (1) whether the trainee created a portfolio and (2) the number of artifacts that each trainee submitted. Counts of artifacts by each competency were entered into an Excel database. To assess trainees' perceived competence in each of 36 learning objectives, a survey instrument was developed by the co-principal investigators asking trainees to rate their self-assessment of competency in each learning objective using a 5-point Likert scale (1 point = strong disagreement to 5 points = strong agreement). Trainees were asked to complete this self-assessment at the beginning and end of their 11-month training period using REDCap electronic data capture tools hosted at the University of Washington.16 An end-of-program survey was also conducted, in which trainees offered their perspective on the strengths and limitations of the program by answering several open-ended questions. From this survey, quotes were abstracted related to the trainees' perceptions of the value of the curriculum.

Analysis.

Trainees' self-reported competence was determined in two ways: (1) scores for 36 learning objectives and (2) composite scores for each of eight core competencies, in which the scores for all learning objectives were averaged within each competency. Wilcoxon signed rank test was used to determine whether, on average, individual learning objective scores and composite competency scores differed statistically before and after participation in the fellowship. The statistical significance threshold for multiple testing was determined using the Holm method.17 The corrected overall critical P value for comparing eight core competency scores was 0.05; the critical P value for comparing 36 individual learning objectives was 0.002. Wilcoxon rank sum test was also used to assess whether the number of completed artifacts differed by trainee characteristics, such as fellow, scholar, international, or US. Data management and analyses were conducted in Stata, version 13.0.18 All tests were two-tailed. This study was approved by the University of Washington Human Subjects Division.

Results

Of 58 total artifacts, the number of artifacts completed by trainees ranged from 0 to 36. On average, trainees completed the most artifacts for the learning objectives: acquire research skills necessary for global health research and define global health and explain what constitutes global health research (Table 2). Trainees completed the fewest artifacts for communicate research proposals and results; and understand professionalism, professional development, and cultivation of mentorship. The total number of artifacts uploaded did not differ according to whether the trainees were fellows versus scholars, according to whether the trainee was from the US or an LMIC, or by gender (P > 0.05), but the sample size provided limited power to detect meaningful differences between groups.

Table 2

Participation in core competency curriculum as measured by the average number of artifacts uploaded by trainees to their electronic portfolio

Core competency Number of assignments (i.e., number of artifacts requested) Average number of artifacts uploaded Average percentage of artifacts completed
Define global health and explain what constitutes global health research 3 1.5 49.1
Know major trends in global health 16 1.8 11.5
Know organizations funding global health research and pathways to achieve support 2 0.4 21.1
Acquire research skills necessary for global health research 8 4.2 52.6
Communicate research proposal and results 8 0.9 11.8
Acquire and apply skills at managing a research team and processes 6 1.7 28.1
Understand responsible conduct of research and collaborative practice 7 2.6 36.8
Understand professionalism, professional development, and cultivation of mentorship 8 0.4 5.8
Total 58 13.6 23.3

All 19 trainees completed pre-fellowship self-assessments of competency. Of these, 17 trainees completed post-fellowship self-assessments. Pre- and post-fellowship scores by each core competency (i.e., the composite competency score) are presented in Figure 1. Values are listed in the subsequent results (mean [SD]). At baseline, trainees, on average, felt least competent in knowing organizations funding global health research and pathways to achieve support (3.16 [0.77]) and in acquiring and applying skills at managing a research team and processes (3.17 [0.71]), whereas they felt most competent in understanding professionalism, professional development, and cultivation of mentorship (3.79 [0.78]) and in acquiring research skills necessary for global health research (3.75 [0.67]). Self-assessed competency scores significantly improved for all core competencies (P < 0.05 for all eight comparisons), with the greatest gains observed for knowing organizations funding global health research and pathways to achieve support and knowing major trends in global health research.

Figure 1.
Figure 1.

Pre- and post-fellowship self-assessment scores by competency. P value < 0.05 for change in self-assessed scores for all competencies.

Citation: The American Society of Tropical Medicine and Hygiene 92, 1; 10.4269/ajtmh.14-0398

Pre- and post-fellowship results for each learning objective are presented in Table 3. At baseline, trainees felt most competent in their abilities to conduct a literature search (4.16 [0.60]), understand disease burden (4.12 [0.99]), and understand the ethics of human research (4.00 [0.74]). They felt least competent in understanding issues related to migrant health (2.26 [0.81]), neglected tropical diseases (2.33 [0.97]), non-NIH funding opportunities (2.58 [0.96]), biorepositories (2.68 [0.95]), and knowledge of the NIH structure (2.79 [0.98]). Scores improved on all items, with average improvements ranging between 0.47 to 1.41 points on the 5-point Likert scale. Learning objectives that showed greatest statistically significant improvements included identifying organizations that fund global health research (+1.41 [0.87]), defining pathways to pursue NIH funding (+1.24 [0.75]), understanding key concepts relating to research issues in migrating populations (+1.18 [0.91]) and neglected tropical diseases (+1.18 [0.63]), explaining the NIH (+1.12 [1.05]), and understanding how social determinants of health impact the trainees own research topic (+0.58 [0.87]). Other items on which trainees showed statistically significant improvements included understanding research issues in malaria; performing a literature search; showing proficiency in basic biostatistics, scientific writing, and scientific presentations; and describing research integrity.

Table 3

Results of pre- and post-fellowship self-assessments of competencies by year 1 fellows (2012–2013)

  Pre-test Post-test Change P value
N Mean SD N Mean SD N Mean SD
Global health
 Group health definition 19 3.74 0.73 17 4.53 0.63 17 0.76 0.97 0.007
 Explain global health research 19 3.58 0.69 17 4.47 0.62 17 0.88 0.99 0.004
Key topics in global health
 Research issues in HIV 19 3.68 1.06 17 4.24 0.83 17 0.53 0.80 0.018
 Research issues in TB 19 3.26 0.99 17 3.76 0.83 17 0.53 0.87 0.030
 Research issues in malaria 19 3.16 1.07 17 3.76 0.90 17 0.65 0.60 0.002
 Research issues in MCH 19 3.21 1.03 17 3.80 0.94 15 0.53 0.74 0.011
 Research issues in NTD 18 2.33 0.97 17 3.47 1.00 17 1.18 0.64 < 0.001
 Research issues in NCD 19 3.16 1.17 17 3.88 0.99 17 0.71 0.92 0.010
 Research issues in migrating populations 19 2.26 0.81 16 3.38 0.89 18 1.18 0.91 < 0.001
 Social determinants of health 19 3.79 0.98 17 4.41 0.61 17 0.58 0.87 0.017
 Social determinants of health impact on research topic 19 3.63 0.90 17 4.65 0.60 17 0.94 0.96 < 0.001
 Burden of disease for research topic 17 4.12 0.99 16 4.81 0.40 16 0.68 1.01 0.015
 Concept of capacity building 19 3.74 1.10 17 4.41 0.79 17 0.47 0.62 0.008
 Concept of implementation science 19 3.26 1.15 17 4.05 0.83 17 0.77 0.83 0.003
Funding opportunities
 Meetings where research topics are discussed 19 3.68 1.11 17 4.35 0.86 17 0.77 0.83 0.004
 Identify organizations that fund global health research 19 3.05 0.97 17 4.29 0.85 17 1.41 0.87 < 0.001
 Explain NIH 19 2.79 0.98 17 3.82 0.94 17 1.12 1.05 < 0.001
 Describe pathways to pursue NIH funding 19 2.58 0.96 17 3.71 0.85 17 1.24 0.75 < 0.001
 Define research agenda for next 5 years 19 3.68 0.82 17 4.47 0.51 17 0.83 0.88 0.003
Skills to conduct research
 Perform literature search 19 4.16 0.60 17 4.82 0.39 17 0.71 0.59 < 0.001
 Write up succinct study proposal 19 3.95 0.85 17 4.58 0.51 17 0.71 0.85 0.005
 Establish and use data management system 19 3.32 1.06 17 4.06 0.83 17 0.77 1.15 0.022
 Show proficiency in basic biostatistics 19 3.26 1.01 17 4.18 0.73 17 0.94 0.97 < 0.001
 Understand how to access expert opinion on statistical methods 19 3.63 0.83 17 4.18 0.73 17 0.47 0.62 0.011
Communicate research results
 Show proficiency at scientific writing 19 3.42 0.83 17 4.17 0.65 17 0.82 0.64 < 0.001
 Show proficiency at scientific presentations 19 3.42 0.84 17 4.24 0.66 17 0.82 0.73 < 0.001
 Provide meaningful critiques 19 3.63 0.95 17 4.11 0.85 17 0.53 0.62 0.007
Manage research
 Manage laboratory or research unit 19 3.21 0.98 17 3.88 1.11 17 0.76 1.20 0.128
 Establish and use biorepository 19 2.68 0.95 17 3.35 1.06 17 0.71 0.92 0.010
 Construct and manage research budget 19 3.47 0.77 17 4.29 0.47 17 0.88 0.78 < 0.001
Ethics and mentorship
 Design and apply ethical approach for human subjects 19 4.00 0.75 16 4.63 0.50 16 0.63 0.89 0.011
 Design and apply ethical approach for animal subjects 18 3.06 1.30 8 3.38 0.52 8 0.50 0.93 0.162
 Describe research integrity 19 3.63 1.01 16 4.44 0.51 16 0.94 0.85 < 0.001
 Explain cultural differences between investigators 19 3.58 0.84 15 4.40 0.64 15 0.80 0.86 0.005
 Explain cultural differences between investigators and participants 19 3.74 0.81 15 4.40 0.63 15 0.60 0.74 0.009
Professionalism and mentorship
 Obtain and cultivate effective mentorship 19 3.79 0.79 15 4.33 0.62 15 0.73 0.96 0.011

HIV = human immunodeficiency virus; TB = tuberculosis; MCH = maternal and child health; NTD = neglected tropical disease; NCD = non-communicable disease; NIH = National Institutes of Health.

The number of total artifacts across all competencies completed by trainees was correlated with a change in pre- and post-fellowship self-assessment scores across all competencies. Those who completed 19 or more of a total of 58 artifacts, on average, showed an increase in their self-perceived competence of 34.0 points compared with those who completed 5 or fewer artifacts, who reported an average increase of only 11.7 points.

When trainees were asked to list two strengths of the program, six trainees identified the core competency curriculum or the assignments on the portfolio as a strength of the program. In one instance, a trainee commented, “I really enjoyed the competencies. It gave structure to my year and also helped me measure my progress in a way that was not related to the final products like abstracts and publications.” When asked to list two ways to improve the program, one noted that “fellows should know beforehand the assignments attached so they can get them done in time.” Another trainee suggested that the program should “reduce the amount of online tasks.”

Discussion

We describe the successful development and implementation of a competency-based curriculum for global health research among trainees in the NPGHF Research and Training Consortium. All trainees participated in the curriculum by establishing an electronic portfolio, and all but one uploaded artifacts to their portfolios. Trainees, on average, also made significant gains in their perceived competence in all eight core competencies.

In terms of participation in the curriculum, the number of completed artifacts did not differ significantly by trainee characteristic; however, the proportion of completed artifacts did vary across core competencies. We speculate that several important factors may contribute to the different levels of participation in the curriculum across competencies. Compared with artifacts related to general topics, such as reflection papers on professionalism and professional development, those related specifically to the research project activities, such as conducting literature searches or preparing an Institutional Review Board (IRB) application, were more likely to be completed. Timing also seemed to be a factor associated with trainees' participation in the curriculum. Trainees received a timeline for completing core competencies and learning objectives. As a group, trainees completed a greater proportion of the artifacts for core competencies scheduled earlier in their fellowship. This was true even for competencies with artifacts that were more general and not directly related to their research projects, such as define global health and explain what constitutes global health research. Trainees may have become increasingly busy as their projects progressed over time, relegating the competency-based curriculum to a lower priority. An additional obstacle to participation may have been the complexity of the web-based curriculum and the number of online tasks that were required of trainees, which included accessing a password-protected central site, establishing and maintaining an online portfolio, and uploading artifacts through the web.

In terms of perceived gains in competence, the amount of gain reported by trainees increased with level of participation. Overall, trainees who completed the most artifacts showed greater improvement in self-assessed competence compared with those who completed the least number of artifacts. Trainees reported greater self-perceived gain for core competencies in which there was greater participation in the curriculum. For instance, understand responsible conduct of research and collaborative practice had the second largest gain in self-perceived competence and also ranked third in terms of the proportion of artifacts completed. In contrast, trainees reported smaller self-perceived gains in the area communicate research proposal and results, which also had one of the lowest completion rates for artifacts. The lower participation rate and observed improvements in this competency were likely because these artifacts required research projects to have been completed.

Variations in self-perceived gains across competencies were also likely correlated with trainees' initial self-assessments. Notably, trainees reported the smallest self-assessed improvements in core competencies for which they felt most competent at baseline: acquire research skills necessary for global health research. A high level of self-perceived competence provides less room for improvement on the five-point scale relative to competencies with lower baseline scores. For instance, trainees recorded the largest improvements for knowing organizations funding global health research and pathways to achieve support, which ranked the lowest in self-perceived competency by trainees as a group at baseline. However, with even modest levels of participation in the curriculum for this competency, they reported the largest gains in this competency. Although it is possible that trainees may have learned about funding sources through conducting their research projects, because their research projects did not require trainees to explore global health funding opportunities, we speculate that improvements were driven primarily by the curriculum.

Several obstacles were encountered in the implementation of this curriculum. First, the acceptance of the first cohort of trainees occurred shortly after the establishment of the consortium; therefore, the development of the curriculum occurred as trainees began their 11-month fellowship. The curriculum was not launched until trainees were, on average, 4–5 months into their fellowship. With 6–7 months remaining to complete both their research project and the curriculum, the observed level of participation likely represents an underestimate of what might be expected had the curriculum been introduced at the start of trainees' fellowships. Second, although distance learning through web-based technology is well-suited for LMICs and other resource-limited settings, it is dependent on internet accessibility. Although intermittent internet connectivity did not affect the asynchronous interaction between US mentors and trainees on electronic portfolios, there were country-specific website restrictions that prevented two trainees from being able to directly upload their artifacts.

Several additional limitations are notable. First, the curriculum and chosen competencies were arrived at through deliberate discussions among a select group of principal investigators with expertise in global health research. It is possible that other investigators or a more formal process would have resulted in different competencies. Second, the pre- and post-fellowship competency scores were self-reported. Although changes in self-assessed scores showed increased awareness among trainees in all competencies, they may not necessarily correlate with objective measures that define a standard level of competence. Third, the use of the uploaded artifacts as a proxy for participation in the curriculum is a limitation. It is possible that trainees completed more assignments than they uploaded to their portfolio for a variety of reasons, including lack of expertise with Google Sites, unreliable internet connections, or unfamiliarity with portfolios. Fourth, the relatively low mean completion rate of 25% for artifacts limits our ability to interpret the impact or effectiveness of the program. Fifth, the data came from a small sample of trainees representing the first year of the program within one consortium. As a result, we did not conduct any formal statistical testing comparing subgroups because of small sample size. More representative results may be obtained through larger samples across cohorts and consortia.

Despite challenges in implementation, these results suggest that an interactive, online, competency-based curriculum can be implemented across geographically dispersed trainees. The results further suggest the curriculum supports and complements the knowledge and skills that trainees gain through their mentored research experiences. For the first-year trainees, there were self-perceived improvements in all competencies, with notable improvements in competencies not covered by the research projects themselves.

Several important lessons were learned in developing and maximizing participation in this curriculum. Namely, helping trainees understand the relevance of artifacts important to global health research but not directly related to their projects is critical to increasing trainees' participation in the curriculum. Additionally, correct timing of the suggested completion of competencies both in terms of the constraints of an 11-month fellowship and the timeline for individual research projects is important to maximize participation in the curriculum.

Future efforts will focus on improving the implementation and evaluation of the curriculum. Research suggests that factors contributing to effective use of portfolios include clearly stated goals, integration with curriculum and assessments, and support through mentoring, including tutor support.19 All of these factors can be improved in our training program. In particular, greater participation of US-based and onsite mentors in providing feedback on trainees' progress in each of the competencies will be essential to increasing the relevance of the curriculum to the trainees. Incorporating objective measures to supplement self-perceived competence can be used to better triangulate the concept of competence. In terms of acceptability and willingness to participate, a better understanding of the trainees' experiences through surveys and focus groups could help improve the curriculum as well as increase participation.

The preparation of future global health researchers requires practical training in research skills as well as intentional definition and instruction of key competencies that should be emphasized over longer-term training programs. This competency-based curriculum represents a first step toward creating a framework of global health research competencies on which further efforts could be based.

ACKNOWLEDGMENTS

We thank Yolanda Thomas at Rambiss Consulting Group, whose support of the program has been instrumental.

  • 1.

    Drain PK, Primack A, Hunt DD, Fawzi WW, Holmes KK, Gardner P, 2007. Global health in medical education: a call for more training and opportunities. Acad Med 82: 226230.

    • Search Google Scholar
    • Export Citation
  • 2.

    Fogarty International Center, 2014. Global Health Program for Fellows and Scholars. Available at: http://www.fic.nih.gov/Programs/Pages/scholars-fellows-global-health.aspx. Accessed May 12, 2014.

    • Search Google Scholar
    • Export Citation
  • 3.

    Quinn TC, 2008. The Johns Hopkins Center for Global Health: transcending borders for world health. Acad Med 83: 134142.

  • 4.

    Stapleton FB, Wahl PW, Norris TE, Ramsey PG, 2006. Addressing global health through the marriage of public health and medicine: developing the University of Washington department of global health. Acad Med 81: 897901.

    • Search Google Scholar
    • Export Citation
  • 5.

    Haq C, Baumann L, Olsen CW, Brown LD, Kraus C, Bousquet G, Conway J, Easterday BC, 2008. Creating a center for global health at the University of Wisconsin-Madison. Acad Med 83: 148153.

    • Search Google Scholar
    • Export Citation
  • 6.

    Bryant JH, Velji A, 2011. Global health and the role of universities in the twenty-first century. Infect Dis Clin North Am 25: 311321.

  • 7.

    Anandaraja N, Hahn S, Hennig N, Murphy R, Ripp J, 2008. The design and implementation of a multidisciplinary global health residency track at the Mount Sinai School of Medicine. Acad Med 83: 924928.

    • Search Google Scholar
    • Export Citation
  • 8.

    Gladding S, Zink T, Howard C, Campagna A, Slusher T, John C, 2012. International electives at the university of Minnesota global pediatric residency program: opportunities for education in all Accreditation Council for Graduate Medical Education competencies. Acad Pediatr 12: 245250.

    • Search Google Scholar
    • Export Citation
  • 9.

    Bloomfield GS, Huffman MD, 2010. Global chronic disease research training for fellows: perspectives, challenges, and opportunities. Circulation 121: 13651370.

    • Search Google Scholar
    • Export Citation
  • 10.

    Martin IB, Jacquet GA, Levine AC, Douglass K, Pousson A, Dunlop S, Khanna K, Bentley S, Tupesis JP, 2013. Global health and emergency care: a postgraduate medical education consensus-based research agenda. Acad Emerg Med 20: 12331240.

    • Search Google Scholar
    • Export Citation
  • 11.

    Calhoun JG, Spencer HC, Buekens P, 2011. Competencies for global health graduate education. Infect Dis Clin North Am 25: 575592.

  • 12.

    Ablah E, Biberman DA, Weist EM, Buekens P, Bentley ME, Burke D, Finnegan JR, Flahault A, Frenk J, Gotsch AR, Klag MJ, Rodriguez Lopez MH, Nasca P, Shortell S, Spencer HC, 2014. Improving global health education: development of a Global Health Competency Model. Am J Trop Med Hyg 90: 560565.

    • Search Google Scholar
    • Export Citation
  • 13.

    Gruppen LD, Mangrulkar RS, Kolars JC, 2012. The promise of competency-based education in the health professions for improving global health. Hum Resour Health 10: 43.

    • Search Google Scholar
    • Export Citation
  • 14.

    Hagopian A, Spigner C, Gorstein JL, Mercer MA, Pfeiffer J, Frey S, Benjamin L, Gloyd S, 2008. Developing competencies for a graduate school curriculum in international health. Public Health Rep 123: 408414.

    • Search Google Scholar
    • Export Citation
  • 15.

    Pfeiffer J, Beschta J, Hohl S, Gloyd S, Hagopian A, Wasserheit J, 2013. Competency-based curricula to transform global health: redesign with the end in mind. Acad Med 88: 131136.

    • Search Google Scholar
    • Export Citation
  • 16.

    Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG, 2009. Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 42: 377381.

    • Search Google Scholar
    • Export Citation
  • 17.

    Holm S, 1979. A simple sequentially rejective multiple test procedure. Scand J Stat 6: 6570.

  • 18.

    StataCorp, 2013. Stata Statistical Software: Release 13. College Station, TX: Statacorp LP.

  • 19.

    Driessen E, van Tartwijk J, van der Vleuten C, Wass V, 2007. Portfolios in medical education: why do they meet with mixed success? A systematic review. Med Educ 41: 12241233.

    • Search Google Scholar
    • Export Citation

Author Notes

* Address correspondence to Joseph C. Kolars, 4122 Med Sci I, SPC 5624, 1301 Catherine St., Ann Arbor, MI 48109-5624. E-mail: jckolars@umich.edu

Financial support: This project was supported by National Institutes of Health Research Training Grant R25TW009345 funded by the Fogarty International Center, National Institute of Mental Health, National Heart, Lung and Blood Institute, Office of Research on Women's Health, and Office of Aids Research, and National Center for Research Resources/National Insitutes of Health Grant UL1 RR025014.

Authors' addresses: Thanh G. N. Ton, Departments of Neurology and Global Health, University of Washington, Seattle, WA, E-mail: thanhton@uw.edu. Sophia P. Gladding, Chandy John, and Molly McCoy, Department of Pediatrics, Division of Global Pediatrics, University of Minnesota, Minneapolis, MN, E-mails: gladd001@umn.edu, ccj@umn.edu, and mccoy019@umn.edu. Joseph R. Zunt, Departments of Neurology, Medicine, and Global Health, University of Washington, Seattle, WA, E-mail: jzunt@uw.edu. Vivek R. Nerurkar, Departments of Tropical Medicine, Medical Microbiology, and Pharmacology, John A. Burns School of Medicine, University of Hawaii at Manoa, Honolulu, HI, E-mail: nerurkar@hawaii.edu. Cheryl A. Moyer and Joseph C. Kolars, Global REACH and Department of Learning Health Sciences, University of Michigan, Ann Arbor, MI, E-mails: camoyer@med.umich.edu and jckolars@umich.edu. Nicole Hobbs, Department of Global Health, University of Washington, Seattle, WA, E-mail: hobbsn@uw.edu.

Save