Open educational resources on how to conduct randomised clinical trials: a protocol for a landscape analysis
============================================================================================================

* Kim Boesen
* Sarah Louise Klingenberg
* Christian Gluud

## Abstract

This is a protocol for the project entitled, “Open educational resources on how to conduct randomised clinical trials”.

## Introduction

A large part of conducted randomised clinical trials across fields, indications, and healthcare interventions are not informative for clinical practice and is considered waste.1-8 The three main reasons for clinical trials being uninformative are that they are done for the wrong reason, they are designed the wrong way, and/or their results are not analysed and reported adequately. 1-8 As Doug Altman put it in his famous essay on the scandal of poor research from 1994, “We need less research, better research, and research done for the right reasons”.9

For decades, systematic reviews and assessments of clinical trials across many different fields and specialties have reported issues with risk of bias, suboptimal trial designs, underreporting, selective outcome reporting, and publication bias, colloquially referred to as ‘research waste’.1-8 Recently, assessments of clinical trials’ informativeness encompassing indicators also of feasibility, transparency, and reproducibility, have emerged. One study of 125 diabetes, ischaemic heart disease, and lung cancer trials reported that only 26% fulfilled the authors’ four criteria of informativeness (clinical question, design, feasibility, reporting).10 Another similar study of 347 gynecology and obstetrics trials reported that only 10% fulfilled minimum half of the authors’ eight prespecified ‘usefulness’ criteria (clinical question, preceding systematic review, power, pragmatic, patient oriented, financially worth it, transparency).11 One main reason for this lack of ‘informativeness’ may be that it is not a priority during medical, graduate, and postgraduate training to learn about proper conduct of clinical trials, and access to freely available quality educational material may also be sparse.

We want to make a landscape analysis of publicly available online educational material on how to conduct interventional clinical trials, covering all aspects from the initial ideation to the design, conduct, and final reporting. Rather than focusing on traditional book or paper-based material, we will focus on free online material, colloquially known as open educational resources (OER),12, 13 and primarily in the ‘microlearning’ format,14, 15 i.e. short (or ‘bite sized’) concise videos, tutorials, or instructions. Such material is often used in a ‘blended learning’ setup combining online and face-to-face classes with video and written material and integrating various technologies.16-19 Within some medical specialties, particularly anaesthesiology and critical care medicine, extensive online communities with freely available educational material exist, known as free open access medical education (FOAM).20-22 Overviews and surveys of OER in various specialties, including pediatric intensive care,23 nephrology,24 radiation oncology,25 anaesthesiology,26 pediatrics,27 critical care,28, 29 otolaryngology,30 and medical physics31 report many available resources, including videos, podcasts, and tutorials. We will use this landscape analysis as a primer to build a comprehensive blended learning curriculum on randomised clinical trial conduct targeting clinicians, investigators, trialists, researchers, and anyone interested in the topic on how to make fair tests of treatments.32, 33

## Research objectives

1.  To identify publicly available educational material or open educational resources (videos and/or written material) about the conduct of randomised clinical trials.

2.  To describe and categorise existing material:
    
    1.  Curriculum, i.e. what content is covered?
    
    2.  Detail level, i.e. how thorough and comprehensive is the content?
    
    3.  Documentation, i.e. is the material evidence based and reference based?
    
    4.  Format, i.e. is the material written, videos, blended combination of video and written, or other combinations, and are there interactive elements, such a quizzes or questionnaire tests?

## Methods

Due to the unconventional topic, we will do (i) a conventional systematic search of databases of published literature, and (ii) we will also search other (unconventional) resources that one could expect researchers and clinicians to look for open educational resources on randomised clinical trials, such as YouTube, Google, and dedicated online learning portals.

### Eligibility criteria

We search for any freely available, instantaneously accessible, online course or educational resource, or massive open online courses (MOOC) as they may be referred to, covering any aspect of interventional clinical trial conduct. We do not apply restrictions on language, publication year, extent, and content.

#### (i) Conventional database searches

Our information specialist (SLK) will search MEDLINE ALL Ovid, Embase Ovid, Science Citation Index Expanded (Web of Science), Social Sciences Citation Index (Web of Science), Conference Proceedings Citation Index – Science (Web of Science), Conference Proceedings Citation Index – Social Science & Humanities (Web of Science), and Cumulative Index to Nursing and Applied Health Literature (CINAHL; Ebsco Host) using a compound search string consisting of keywords relevant to the field (Appendix 1).

We will merge and import the combined hits into the open-source artificial intelligence (AI) assisted screening program for systematic reviews, ASReview.34 One author (KB) will screen the hits in a one-step process (i.e. title/abstract and, when needed, full-text) to make an immediate final decision about inclusion or exclusion. We will apply an arbitrary stopping rule of 100 consecutive irrelevant hits, i.e. meaning that we will not necessarily screen all retrieved hits, but instead we will screen only those hits deemed most relevant by ASReview.35

For the ASReview machine learning algorithm to be able to rank and order the full number of retrieved hits, it must be trained (i.e. supervised learning) on a minimum of two relevant references (in our case, publications describing open educational resources on randomised clinical trials), and two irrelevant references. At our current stage, we are not aware of relevant references. One solution would be to train the algorithm on overview articles pertaining to OER in specific medical fields20-31 or to train it on articles describing specific OERs, such as online residency training programs.36-38 As a last resort, we can train it on those articles and screen all retrieved articles. However, this may not be feasible if the systematic database search returns a very large number of hits (e.g. >10.000 hits). We will decide our final strategy based on the number of retrieved hits and if we discover more eligible publications before screening.

#### (ii) Searching other resources

We will use a two-step search strategy by first doing ‘cold searches’ of general websites using expected key words to identify ‘seed references’ such as relevant material, videos, and resources. One author (KB) will use these keywords and terminology to subsequently stringently search the listed resources and others, if needed.

##### General website searches

We will search common websites to find seed videos and courses to guide our search, including YouTube ([https://www.youtube.com/](https://www.youtube.com/)), Google ([https://www.google.com/](https://www.google.com/)), and Bing ([https://www.bing.com/](https://www.bing.com/)). An example of a search string: “online learning material clinical trials”.

We know of one comprehensive resource already: the YouTube channel ‘GCP-Mindset’.39 The channel is maintained by the contract research organisation company GCP-Service International40 and they have published 350 videos (by 1 February 2024; not all relevant) on matters related to clinical trial conduct.

##### Online learning portals

We will search the most common and used online learning portals, including:

1.  **Coursera** ([https://www.coursera.org/](https://www.coursera.org/)) is one of the most popular and largest online learning portals with more than 5.800 courses. It was founded in 2012 by Stanford University but is now a for-profit company.

2.  **EdX** ([https://www.edx.org/](https://www.edx.org/)) has more than 4.000 courses available. It was founded by Harvard University and Massachusetts Institute of Technology (MIT) in USA in 2012 but it is now a subsidiary of the for-profit company 2U.

Portals with paid only material, such as Udemy ([https://www.udemy.com/](https://www.udemy.com/)), or restricted access, such as Udacity ([https://www.udacity.com/](https://www.udacity.com/)), will not be considered.

##### University portals

We will also search individual major US and European university eLearning portals. There might be overlap with the courses provided on Coursera and EdX.

1.  Stanford Online ([https://online.stanford.edu/](https://online.stanford.edu/))

2.  Harvard Professional and Lifelong learning ([https://pll.harvard.edu/](https://pll.harvard.edu/))

3.  MIT Open course ware ([https://ocw.mit.edu/](https://ocw.mit.edu/))

4.  SwissMOOC ([https://www.swissmooc.ch/](https://www.swissmooc.ch/)), is a collaboration of most Swiss universities’ online courses.

5.  Oxford University Online and distance courses ([https://www.ox.ac.uk/admissions/conBnuing-education/online-and-distance-courses](https://www.ox.ac.uk/admissions/conBnuing-education/online-and-distance-courses)).

6.  Cambridge Advance Online ([https://advanceonline.cam.ac.uk/](https://advanceonline.cam.ac.uk/))

##### Clinical trial units

We will map major Scandinavian, European, and international clinical trial units and clinical trial networks for relevant educational material. The following list may not be exhaustive:

1.  European Clinical Research Infrastructure Network (ECRIN) ([https://ecrin.org/](https://ecrin.org/)), and the national charters.

2.  Swiss Clinical Trial Unit Network ([https://www.scto.ch/en.html](https://www.scto.ch/en.html)), including individual CTUs, e.g. CTU Bern ([https://www.ctu.unibe.ch/](https://www.ctu.unibe.ch/)) and University of Zurich Clinical Trials Center ([https://www.usz.ch/en/clinic/clinical-trials-center/](https://www.usz.ch/en/clinic/clinical-trials-center/)).

3.  Danish trial units, e.g. the Open Patient data Explorative Network at Southern University Denmark ([https://open.rsyd.dk/](https://open.rsyd.dk/)), Aarhus University’s Clinical Trial Unit ([https://clin.au.dk/research/clinical-trial-unit](https://clin.au.dk/research/clinical-trial-unit)), and the centralised Good Clinical Practice (GCP) Unit ([https://gcp-enhed.dk/](https://gcp-enhed.dk/)).

4.  Clinical Studies Sweden ([https://kliniskastudier.se/](https://kliniskastudier.se/)) and it’s six regional websites.

5.  Clinical Trials Unit at Oslo University Hospital ([https://forskerstoVe.no/en/kliniske-studier](https://forskerstoVe.no/en/kliniske-studier)) and Clinical Research Unit at region Mid-Norway ([https://www.klinforsk.no/](https://www.klinforsk.no/)).

6.  Medical Research Council Clinical Trials Unit ([https://www.mrcctu.ucl.ac.uk/](https://www.mrcctu.ucl.ac.uk/)).

7.  University of Oxford Clinical Trial Units ([https://www.ctsu.ox.ac.uk/research](https://www.ctsu.ox.ac.uk/research)).

8.  Cambridge Clinical Trial Units ([https://cctu.org.uk/](https://cctu.org.uk/)).

9.  Stanford Clinical and Translational Research Unit ([https://med.stanford.edu/ctru.html](https://med.stanford.edu/ctru.html)).

##### Other online resources for clinical research conduct

If, and when, we identify other resources containing educational resources on the conduct of interventional clinical trials that do not fit the first four categories, they will be listed here.

Examples of such resources include:

1.  The UK Faculty of Public Health’s Health Knowledge portal ([https://www.healthknowledge.org.uk/](https://www.healthknowledge.org.uk/)).

2.  The European Communication on Research Awareness Need (ECRAN) Project ([http://ecranproject.eu/en](http://ecranproject.eu/en)), which is a European Commission project on communicating the importance of clinical trials.

## Discussion

The published literature on so called ‘microlearning’ is growing,12, 13 and even more so is the literature on open educational resources (OER) and free open access medical education (FOAM) across medical specialties.20-31 Even the concept of peer reviewed medical education videos has emerged.41 However, based on basic preliminary searches (Appendix 2) we expect some, but not many, relevant online resources on how to plan, launch, execute, analyse, and report randomised clinical trials. We expect videos and courses to be rather high-level, such as focusing on good clinical practice (e.g. the YouTube ‘GCP-Mindset’ channel34), whereas educational material focusing on the individual steps necessary to conduct a rigorous, reproducible, and reliable clinical trial may be scarce.

Material made available on Coursera and EdX are mainly produced by prestigious US universities, e.g. Johns Hopkins or Stanford, and this needs to be taken into consideration when evaluating the content and its global appeal and usefulness. Especially if the content focuses on local or regional legislation, it may not be applicable in other regions. Another more important limitation to open educational resources may be a lack of scientific rigour and quality, and content may be more or less evidence based.21, 42, 43 However, we do not intend to make a detailed qualitative assessment of the identified resources, which will be beyond the scope of this landscape analysis.

As a minimum we will make our full results available on the Copenhagen Trial Unit’s web site ([www.ctu.dk](http://www.ctu.dk)) as part of our forthcoming blended learning curriculum. If we judge the results to have appeal for a larger audience, we will also seek publication in a regular journal, to put our findings into a wider context.

## Data Availability

The complete dataset will be made publicly available alongside the final report.

## Conflicts of interest

KB and CG will develop a publicly available, freely accessible blended learning curriculum on randomised clinical trials. They have no financial or otherwise intellectual conflicts related to the topic.

## Data sharing statement

The complete dataset will be made publicly available alongside the final report.

## Funding statement

This project does not receive any funding.

## Acknowledgements

KB wrote the first draft to this protocol and KB, SLK, and CG revised it, and all agreed on the final version.

## Appendix 1

### MEDLINE ALL Ovid (1946 to the date of the search)

1.  (‘open access’ adj3 (education* or web-site* or meducation*)).ti,ab.

2.  (((online or video or digital or web-based) adj3 (resource* or material* or program* or education* or training* or course* or learning*)) or e-learning or ‘blended learning’ or ‘micro learning’).ti,ab.

3.  1 or 2

### Embase Ovid (1974 to the date of the search)

1.  (‘open access’ adj3 (education* or web-site* or meducation*)).ti,ab.

2.  (((online or video or digital or web-based) adj3 (resource* or material* or program* or education* or training* or course* or learning*)) or e-learning or ‘blended learning’ or ‘micro learning’).ti,ab.

3.  1 or 2

### CINAHL (Ebsco host; the date of the search will be given at review stage)

S3 S1 OR S2

S2 TX ((((online or video or digital or web-based) N3 (resource* or material* or program* or education* or training* or course* or learning*)) or e-learning or ‘blended learning’ or ‘micro learning’)) OR AB ((((online or video or digital or web-based) N3 (resource* or material* or program* or education* or training* or course* or learning*)) or e-learning or ‘blended learning’ or ‘micro learning’))

S1 TX ((‘open access’ N3 (education* or web-site* or meducation*))) OR AB ((‘open access’ N3 (education* or web-site* or meducation*)))

**Science Cita3on Index Expanded (1900 to the date of the search), Social Sciences Cita3on Index (1956 to the date of the search), Conference Proceedings Cita3on Index – Science (1990 to the date of the search), Conference Proceedings Cita3on Index – Social Science & Humani3es (1990 to the date of the search) (Web of Science)**

#3 #2 OR #1

#2 TI=(((online or video or digital or web-based) N3 (resource* or material* or program* or education* or training* or course* or learning*)) or e-learning or ‘blended learning’ or ‘micro learning’) OR AB=(((online or video or digital or web-based) N3 (resource* or material* or program* or education* or training* or course* or learning*)) or e-learning or ‘blended learning’ or ‘micro learning’)

#1 TI=(‘open access’ N3 (education* or web-site* or meducation*)) OR AB=(‘open access’ N3 (education* or web-site* or meducation*))

## Appendix 2

Initial pilot searches

### Coursera

Search words: “Clinical trials” Relevant courses:

1.  Johns Hopkins University - ‘Design and Interpretation of Clinical Trials’ [https://www.coursera.org/learn/clinical-trials](https://www.coursera.org/learn/clinical-trials)
    
    Six modules, rather short course. Voiced-over PowerPoint slides, no interactive elements or background material readily available.

2.  Johns Hopkins University - “Design and conduct of clinical trials” [https://www.coursera.org/learn/design-and-conduct-clinical-trials](https://www.coursera.org/learn/design-and-conduct-clinical-trials). Five modules, similar format.

### EdX

Search words: “Clinical trials”

Relevant courses:

1.  Stanford University - Thinking critically ([https://www.edx.org/learn/critical-thinking-skills/stanford-university-thinking-critically-interpreting-randomized-clinical-trials](https://www.edx.org/learn/critical-thinking-skills/stanford-university-thinking-critically-interpreting-randomized-clinical-trials)). Short overview of the topic, not about the whole process of making a clinical trial.

2.  MD Anderson - Certain Pragmatic trials ([https://www.edx.org/learn/research/the-university-of-texas-md-anderson-cancer-center-in-houston-certain-pragmatic-clinical-trials-and-healthcare-delivery-evaluations](https://www.edx.org/learn/research/the-university-of-texas-md-anderson-cancer-center-in-houston-certain-pragmatic-clinical-trials-and-healthcare-delivery-evaluations)).

Short overview of the topic, not about the whole process of making a clinical trial.

*   Received February 16, 2024.
*   Revision received February 16, 2024.
*   Accepted February 17, 2024.


*   © 2024, Posted by Cold Spring Harbor Laboratory

This pre-print is available under a Creative Commons License (Attribution 4.0 International), CC BY 4.0, as described at [http://creativecommons.org/licenses/by/4.0/](http://creativecommons.org/licenses/by/4.0/)

## References

1.  1.Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet 2009;374:86–9.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0140-6736(09)60329-9&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19525005&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F02%2F17%2F2024.02.16.24302873.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000267768800037&link_type=ISI) 

2.  2.Chalmers I, Bracken MB, Djulbegovic B, et al. How to increase value and reduce waste when research priorities are set. Lancet 2014;383:156–65.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0140-6736(13)62229-1&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24411644&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F02%2F17%2F2024.02.16.24302873.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000329579600031&link_type=ISI) 

3.  3.Ioannidis JPA, Greenland S, Hlatky MA, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet 2014;383:166–75.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0140-6736(13)62227-8&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24411645&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F02%2F17%2F2024.02.16.24302873.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000329579600032&link_type=ISI) 

4.  4.Salman RAS, Beller E, Kagan J, et al. Increasing value and reducing waste in biomedical research regulation and management. Lancet 2014;383:176–85.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0140-6736(13)62297-7&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24411646&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F02%2F17%2F2024.02.16.24302873.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000329579600033&link_type=ISI) 

5.  5.Chan AW, Song F, Vickers A, et al. Increasing value and reducing waste: addressing inaccessible research. Lancet 2014;383:257–66.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0140-6736(13)62296-5&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24411650&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F02%2F17%2F2024.02.16.24302873.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000330027500035&link_type=ISI) 

6.  6.Glasziou P, Altman DG, Bossuyt P, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet 2014;383:267–76.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0140-6736(13)62228-X&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24411647&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F02%2F17%2F2024.02.16.24302873.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000330027500036&link_type=ISI) 

7.  7.Glasziou P, Chalmers I. Is 85% of health research really “wasted”? BMJ Opinion 14 January 2016, [https://blogs.bmj.com/bmj/2016/01/14/paul-glasziou-and-iain-chalmers-is-85-of-health-research-really-wasted/](https://blogs.bmj.com/bmj/2016/01/14/paul-glasziou-and-iain-chalmers-is-85-of-health-research-really-wasted/).
    
    

8.  8.Zarin DA, Goodman SN, Kimmelmann J. Harms from uninformative trials. JAMA 2019; 322:813–4.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2019.9892&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=31343666&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F02%2F17%2F2024.02.16.24302873.atom) 

9.  9.Altman DG. The scandal of poor medical research. BMJ 1994;308:283–4.
    
    [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMDgvNjkyNC8yODMiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyNC8wMi8xNy8yMDI0LjAyLjE2LjI0MzAyODczLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

10. 10.Hutchinson N, Moyer H, Zarin DA, Kimmelmann J. The proportion of randomized controlled trials that inform clinical practice. eLife 2022; 11:e79491.
    
    

11. 11.Van’t Hooi J, van Dijk CE, Axfors C, et al. Few randomized trials in preterm birth prevention meet predefined usefulness criteria. J Clin Epidemiol 2023;162:107–17.
    
    

12. 12.Smith MS. Opening education. Science 2009;323:89–93.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6Mzoic2NpIjtzOjU6InJlc2lkIjtzOjExOiIzMjMvNTkxMC84OSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDI0LzAyLzE3LzIwMjQuMDIuMTYuMjQzMDI4NzMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

13. 13.UNESCO. Open educational resources. Not dated. [https://www.unesco.org/en/open-educational-resources](https://www.unesco.org/en/open-educational-resources) x(February 2024).
    
    

14. 14.Sankaranarayanan R, Leung J, Abramenka-Lachheb V, Seo G, Lachheb A. Microlearning in diverse contexts: a bibliometric analysis. TechTrends 2023;67:260–76.
    
    

15. 15.De Gagne JC, Park HK, Hall K, Woodward A, Yamane S, Kim SS. Microlearning in health professions education: scoping review. JMIR Med Educ 2019; 5:e13997.
    
    

16. 16.Cleveland-Ines M, Wilton D. Atabasca University, Canada. Commonwealth of Learning. Guide to blended learning. Chapter 1: Blended learning. Not dated. Available from: [https://openbooks.col.org/blendedlearning/chapter/chapter-1-blended-learning/](https://openbooks.col.org/blendedlearning/chapter/chapter-1-blended-learning/) x(Jan 2024).
    
    

17. 17.Rowe M, Frantz J, Bozalek V. The role of blended learning in the clinical education of healthcare students: a systematic review. Med Teach 2012;34:e216–21.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3109/0142159X.2012.642831&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22455712&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F02%2F17%2F2024.02.16.24302873.atom) 

18. 18.George PP, Zhabenko O, Kyaw BM, et al. Online digital education for postregistration training of medical doctors: Systematic review by the digital health education collaboration. J Med Internet Res 2019;21:e13269.
    
    

19. 19.Vallée A, Blacher J, Cariou A, Sorbets E. Blended learning compared to traditional learning in medical education: Systematic review and meta-analysis. J Med Internet Res 2022;22:e16504.
    
    

20. 20.Wong A, Wilkinson J, Malbrain M. Social media in critical care: what’s all the fuss about? 6 June 2018. [https://blogs.biomedcentral.com/on-medicine/2018/06/06/social-media-in-critical-care-whats-all-the-fuss-about/](https://blogs.biomedcentral.com/on-medicine/2018/06/06/social-media-in-critical-care-whats-all-the-fuss-about/) x(Feb 2024).
    
    

21. 21.Ting DK, Boreskie P, Luckett-Gatopoulos S, Gysel L, Lanktree MB, Chan TM. Quality appraisal and assurance techniques for free open access medical education (FOAM) resources: A rapid review. Semin Nephrol 2020;40:309–19.
    
    

22. 22.Chan TM, Stehman C, Gottlieb M, Thoma B. A short history of free open access medical education. The past, present, and future. ATS Sch 2020;1:87–100.
    
    

23. 23.Daniel D, Wolbrink TA. Online educational interventions in pediatric intensive care medicine. Front Pediatr 2023;11: 1127754.
    
    

24. 24.Larsen DM, Boscardin CK, Sparks MA. Engagement in free open access medical education by US nephrology fellows. Clin J Am Soc Nephrol 2023;18:573–80.
    
    

25. 25.Culbert MM, Brisson RJ, Oladeru OT. The landscape of digital resources in radiation oncology. Tech Innov PaBent Support Radiat Oncol 2022;24:19–24.
    
    

26. 26.Evans FM, Krotinger AA, Lilaonitkul M, et al. Evaluation of open access websites for anesthesia education. Anesth Analg 2022;135:1233–44.
    
    

27. 27.Barton J, Round J, Knight K. Paediatric free open access meducation (FOAM): behaviours, trends and implications. Postgrad Med J 2020;96:245–9.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTI6InBvc3RncmFkbWVkaiI7czo1OiJyZXNpZCI7czoxMToiOTYvMTEzNS8yNDUiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyNC8wMi8xNy8yMDI0LjAyLjE2LjI0MzAyODczLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

28. 28.Wolbrink TA, Rubin L, Burns JP, Markovitz B. The top ten websites in critical care medicine education today. J Intensive Care Med 2019;34:3–16.
    
    

29. 29.Kleinpell R, Ely EW, Williams G, Liolis A, Ward N, Tisherman A. Web-based resources for critical care education. Crit Care Med 2011;39:541–53.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/CCM.0b013e318206b5b5&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21169819&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F02%2F17%2F2024.02.16.24302873.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000287480000017&link_type=ISI) 

30. 30.Yang N, Hosseini S, Mascarella MA, et al. Identifying high quality medical education websites in Otolaryngology: a guide for medical students and residents. J Otolaryngol Head Neck Surg 2017;46:42.
    
    

31. 31.Prisciandaro JI. Review of online educational resources for medical physicists. J Appl Clin Med Phys 2013;14:4476.
    
    

32. 32.The James Lind Library. Available at [https://www.jameslindlibrary.org/](https://www.jameslindlibrary.org/) x(Jan 2024).
    
    

33. 33.Chalmers I. The development of fair tests of treatments. Lancet 2014;383:1713–4.
    
    

34. 34.ASReview: AI-aided open source systematic review soiware. [https://asreview.nl/](https://asreview.nl/)
    
    

35. 35.Van Dijk SHB, Brusse-Keizer MGJ, Bucsán CC, Palen JVD, Doggen CJM, Lenferink A. ArBficial intelligence in systematic reviews: promising when appropriately used. BMJ Open 2023;13: e072254.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMjoiMTMvNy9lMDcyMjU0IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDIvMTcvMjAyNC4wMi4xNi4yNDMwMjg3My5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

36. 36.Chang WY, Sheen STH, Chang PC, Lee PH. Developing an E-learning education programme for staff nurses: processes and outcomes. Nurse Educ Today 2008;28:822–8.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.nedt.2008.02.003&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=18384917&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F02%2F17%2F2024.02.16.24302873.atom) 

37. 37.Taylor FA, Nelson E, Delfino K, Han H. A blended approach to learning in an obstetrics and gynecology residency program: proof of concept. J Med Educ Curric Dev 2015; 2:JMECD.S32063.
    
    

38. 38.Markman KM, Brewer SK, Magnuson B. Developing an engaging and accessible clinical research training program for new investigators. J Clin Transl Sci 2022;7:e53.
    
    

39. 39.GCP-Mindset – All about clinical trials. [https://www.youtube.com/@GCPMindset/featured](https://www.youtube.com/@GCPMindset/featured) x(1 February 2024).
    
    

40. 40.GCP-Service International. [https://www.gcp-service.com/](https://www.gcp-service.com/).
    
    

41. 41.Seam N, Richards JB, Kritek PA, et al. Design and implementation of a peer-reviewed medical education video compeBtion: The best of american thoracic society video lecture series. J Grad Med Educ 2019;11:592–6.
    
    

42. 42.Khalid F, Wu M, Ting DK, et al. Guidelines: The do’s, don’ts and don’t knows of creating open educational resources. Perspect Med Educ 2023;12:25–40.
    
    

43. 43.Chan TM, Trueger NS, Roland D, Thoma B. Evidence-based medicine in the era of social media: Scholarly engagement through participation and online interaction. CJEM 2018;20:3–8.