A rapid review of the effectiveness of alternative education delivery strategies in medical, dental, nursing and pharmacy education during the COVID-19 pandemic ================================================================================================================================================================ * Judith Carrier * Deborah Edwards * Michal Tombs * Steve Riley * Ruth Lewis * Elizabeth Gillen * Alison Cooper * Adrian Edwards ## Abstract **Background** Education delivery in higher education institutions was severely affected by the COVID-19 pandemic, with emergency remote teaching developed and adapted promptly for the circumstances. This rapid review investigated the effectiveness of alternative education delivery strategies during the pandemic for medical, dental, nursing and pharmacy students, to help plan and adapt further education provision. **Methods** We included 23 primary studies in undergraduate education, all published in 2020-2021, no relevant UK-based or postgraduate studies were found. Included studies comprised 10 single cohort descriptive; 11 comparative descriptive; and two RCTs. There was considerable variability in terms of students, type of distance learning, platforms used and outcome measures. **Results** In medicine (n=14), self-reported competency and confidence, and demonstrable suturing skills were achieved through participating in remote learning. However, lower levels of knowledge were obtained by students who received virtual or blended learning compared to in-person teaching (low-very low confidence). Using bespoke interactive platforms in undergraduate medical training was superior to standard video (low confidence) or ‘textbook’ presentations (very low confidence). In dentistry (n=2), remote learning led to knowledge gained (low confidence), but self-reported practical and interpersonal skills were lower with remote rather than in-person learning (very low confidence). In nursing (n=3), remote learning, when compared to in-person, resulted in similar knowledge and self-reported competency levels (very low confidence) pre-COVID, but confidence was higher when learning or assessment was conducted virtually (low confidence). In pharmacy (n=4), virtual learning was associated with higher skills, but lower knowledge compared to in-person, pre-COVID; self-reported competency and confidence scores were similar between the two groups (very low confidence). **Conclusions** Remote teaching was valued, and learning was achieved, but the comparative effectiveness of virtual versus in-person teaching is less clear. Supplementary alternative or in-person practical sessions may be required post-emergency to address learning needs for some disadvantaged student groups. Keywords * Rapid review * COVID-19 * effectiveness * alternative education * undergraduate * medical and clinical education ## Introduction Education delivery in higher education institutions was severely affected by the COVID-19 pandemic, especially for healthcare students whose continuing education is imperative to maintain a well-educated healthcare workforce. Many courses transitioned to a period of remote emergency teaching,1–3 developed and adapted promptly for the circumstances, largely without prior contingency planning. For example the American Society of Plastic Surgeons announced free access to its online Education Network for all medical students with an interest in plastic surgery,4 whilst Ahmed at al5 suggested a range of online tools and resources that could be employed for online rheumatology education. In Jordan, distance e-learning was promptly engaged to maintain the continuity of medical education,6 and in Pakistan dental educators came up with innovative solutions to resume dental education remotely.7 Bakshi et al8 argued that whilst the COVID-19 outbreak disrupted the educational experiences of medical students worldwide, this was particularly significant in areas such as ophthalmology where structured education and clinical exposure had already declined. A shift to virtual education for nursing students in Iran highlighted some of the challenges faced by educators and students, such as lack of infrastructure, reduced readiness of educators and students for e-learning, and the time to prepare educational content,9 whilst educators in Canada10 emphasised the importance of continuing to engage nursing students online. Reviews have also highlighted the challenges in migrating to remote education11,12 which include poor knowledge of staff on how to deal with technology, poor internet connections and difficulty in transitioning content for online learning.11,12 By contrast, some students and staff report satisfaction with remote learning,2,13 especially when collaboration and engagement with peers is facilitated.2 A preliminary search of repositories specific to COVID-19 literature identified several existing reviews of alternative education delivery strategies for medical and healthcare students during the COVID-19 pandemic. The systematic review by Wilcha et al.3 looked at the effectiveness of virtual teaching for medical education and suggested that it was effective. However, searching was limited to two databases, including Google Scholar, and the review appears to have been conducted by one author with no critical appraisal conducted.3 Another systematic review by He et al.13 explored the use of synchronous distance education (videoconference or web conference, online classroom or virtual classroom) compared with traditional education for medical, dental, nurse, pharmacy students and other health science–related students.13 It found that there were no significant differences in terms of knowledge or skills but that satisfaction was rated higher for distance education.13 For nursing students, a scoping review by Jowsey et al.14 suggested that when delivered purposefully, blended learning (a mix of face-to-face and online study) can positively influence and impact on the achievements of students, especially when used to support distance education.14 However, none of the existing reviews specifically explored effectiveness of alternative education delivery strategies for medical, nursing, dental and pharmacy students, or allied health professionals during the COVID-19 pandemic, or provided a separate summary of the evidence for these disciplines. An initial scope of the evidence base for these healthcare disciplines identified a significant volume of primary research in the area for medical, nursing, dental and pharmacy students but very little for other healthcare disciplines including allied health professionals. We therefore conducted a rapid review of the effectiveness of alternative education delivery strategies that have been put into place for undergraduate and postgraduate medical, nursing, dental, and pharmacy students. ## Methods This rapid review was registered with the International prospective register of systematic reviews (PROSPERO) following the completion of the database searches, and study selection (Registration number: CRD42022304295). ### Eligibility criteria The inclusion criteria were informed by the PICO (Participants, Intervention, Comparison, Outcomes) framework (see Table 1). View this table: [Table 1:](http://medrxiv.org/content/early/2022/03/07/2022.03.04.22271892/T1) Table 1: Eligibility criteria ### Search strategy Searches were conducted across four databases: On the OVID platform: MEDLINE and Embase, on the EBSCO platform: CINAHL and ERIC, from December 2019 to 8th June 2021 for English language citations. An initial search of MEDLINE was undertaken (medicine or medical or nurs* or dental or dentistry or pharmacy or pharmacist) AND education* or train* or teach* or student* or undergraduate* or postgraduate* AND COVID* or coronavirus) followed by analysis of the text words contained in the title and abstract, and of the index terms used to describe the article. This informed the development of search strategies tailored for each information source ([additional material one](http://www.primecentre.wales/resources/RR/Clean/RR00004\_Supplementary_information_Healthcare_education.pdf)). The reference list of all included studies was screened for additional studies. ### Study selection process All citations retrieved from the database searches were imported into EndNote™ and duplicates and irrelevant citations removed and then imported to Covidence™ for study selection. Two reviewers dual screened at least 20% of citations using the information provided in the title and abstract using the software package Covidence™, resolving all conflicts. The remaining citations were then screened by a single reviewer, screening with categories of ‘include’ and ‘exclude’. To streamline the review process, the project team decided against a third category of ‘unsure’ and instead, where there was uncertainty about a citation, it was categorised as ‘include’ and the decision was made based on the full text. The full texts were screened for inclusion by one reviewer using a purposefully designed form which was piloted using approximately 10 manuscripts. One reviewer then screened full text manuscripts, and another reviewer checked all excluded manuscripts. ### Data extraction All demographic data were extracted directly into tables by one reviewer and checked by another. The data included specific details about the interventions, populations, study methods and outcomes of significance to the review question and specific objectives. A template for the data extraction process was piloted on manuscripts for each of the included study designs before use. All outcome data were extracted directly into tables by one reviewer and checked by another. ### Quality appraisal The methodological quality of all the research studies was assessed by one reviewer, and judgements verified by a second reviewer, using JBI design-specific critical appraisal tools ([https://jbi.global/critical-appraisal-tools](https://jbi.global/critical-appraisal-tools)). When a study met a criterion for inclusion a score of one was given. Where a particular point for inclusion was regarded as “unclear” it was given a score of zero. Where a particular point for inclusion was regarded as “not applicable” this point was deducted from the total score. All included studies were assessed using this method and their overall critical appraisal scores were calculated and are displayed for each study in Tables 2 and 3. For the full details of the critical appraisal scores see [additional material two](http://www.primecentre.wales/resources/RR/Clean/RR00004\_Supplementary_information_Healthcare_education.pdf). View this table: [Table 2:](http://medrxiv.org/content/early/2022/03/07/2022.03.04.22271892/T2) Table 2: Summary of critical appraisal scores from descriptive surveys View this table: [Table 3:](http://medrxiv.org/content/early/2022/03/07/2022.03.04.22271892/T3) Table 3: Summary of critical appraisal scores from randomised controlled trials ### Synthesis The data were reported narratively as a series of thematic summaries15 and presented separately for each health care discipline. Two RCTs were included in the review but there was insufficient homogeneity across the studies and therefore we were unable to perform a meta-analysis. ### Assessment of body of evidence The confidence in the synthesised findings was assessed by one reviewer and judgements verified by a second reviewer. The RCTs were assessed using the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach.16 Due to heterogeneity of the different interventions within similar settings, outcome data were only available for results from single studies and guidance was followed on undertaking GRADE for data of this type.17 Quantitative descriptive studies were assessed by applying the principles of GRADE.18 For further details of this processes see [additional material three](http://www.primecentre.wales/resources/RR/Clean/RR00004\_Supplementary\_information\_Healthcare\_education.pdf) and [four](http://www.primecentre.wales/resources/RR/Clean/RR00004\_Supplementary_information_Healthcare_education.pdf). Most findings in this rapid review were of low or very low quality and ratings are displayed for each study in Tables 2 and 3. This was mainly due to imprecision because of small sample sizes, and/or confidence intervals not being reported, and/or limitations because baseline levels of the outcome of interest not being controlled for, and/or lack of clarity of confounding factors. ## Results Of the 10,978 citations retrieved from our searches, 21 descriptive studies and two RCTs met our eligibility criteria. For details of the excluded studies see [additional material five](http://www.primecentre.wales/resources/RR/Clean/RR00004\_Supplementary_information_Healthcare_education.pdf). The included studies focused on undergraduate medical students (n=14), undergraduate dental students (n=2), undergraduate nursing students (n=3) and undergraduate pharmacy students (n=4). We did not find any studies that focused on postgraduate students, and research, that focused on clinically based postgraduate training, such as internships, were excluded. The flow of citations through each stage of the review process is displayed in a PRISMA flowchart, 19 see Figure 1. ![Figure 1:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/03/07/2022.03.04.22271892/F1.medium.gif) [Figure 1:](http://medrxiv.org/content/early/2022/03/07/2022.03.04.22271892/F1) Figure 1: PRISMA flow diagram ### Overview of evidence base for medical students Six pre-test / post-test designs20–25 and six post-test only descriptive studies.26–30 and two RCTs,31,32 provided evidence of the effectiveness of alternative education delivery strategies for undergraduate medical students during the COVID-19 pandemic (see Table 4). Most studies (n=7) were conducted in the USA.21–25,29,33 The remaining studies were conducted in Germany,26,27,31 Japan,20 South Korea,28 Switzerland32 and Greece.30 View this table: [Table 4:](http://medrxiv.org/content/early/2022/03/07/2022.03.04.22271892/T4) Table 4: Characteristics of included studies focusing on medical students These covered a wide range of both university and clinical based modules/ courses and included neurosurgery,21 surgical instruments, knot tying and suturing,24 digital histology,26 a residency preparation course,22 simulated patient consultations, documentation, and case presentation,27 simulated clinical experience in respiratory unit and general medicine,20 generic medical education,28 neuroanatomy,29 emergency medicine25,33 musculoskeletal system anatomy and neuroanatomy,30 the National Institutes of Health Stroke Scale,32 operative techniques and skills,31 and informed consent for surgical procedures.23 A variety of different online platforms was used to deliver synchronous learning; five used the Zoom video conferencing platform20,21,24,26,27 three used the University Supported Management Systems: CANVAS22,29 or Meducator,30 one used Microsoft teams,33 another Skype for business,30 and three did not specify the type of video communication software used.23,25,28 Other methods included neuroanatomical interactive virtual activities using “Digital Neuroanatomy” software,29 simulated patient encounters employing online MedEd Case X videos,33 and structural specimens replaced by photographs.30 Five studies also incorporated asynchronous elements using pre-recorded lectures23,28,30 or readily available podcasts.25,33 For one further study the course content (8 topics) was organised by 12 rising1 fourth-year medical students under supervision.25 The two RCTs used bespoke interactive online platforms31,32 and compared the outcomes to those students learning the same topic via a standard video format31 or textbook based preparation.32 Studies were conducted with students in their final year (Clerkship / Interns) (n=7),20,22,24,25,27,32,33 first year (n=2),29,30 second and third years (n=1),26 third year only (n=1),23 across all years (n=1),21 and a further two did not specify the year of study.28,31 Outcomes explored were confidence (n= 5),21,22,24,25,27 competency (n=2)20,23 and knowledge (n=6).26,28–30,32,33 Levels of competency, confidence and knot tying and suturing skills were found to have improved across the course of learning and a further study suggested that levels of competency were the same when learning was conducted virtually (2020) compared to in-person pre-COVID (2019). Evidence from RCTs showed that knowledge was greater when learning was conducted using bespoke interactive platforms with a standard video format reported during the COVID pandemic. Evidence from descriptive studies showed mixed results for knowledge, assessed and compared between cohorts at the end of virtual learning (2020) and in-person learning (2019). Four studies reported lower levels of knowledge for students in the virtual cohort and one further study found no difference. ### Overview of the evidence base for dental students Two post-test descriptive studies (see Table 5) conducted in Poland34 and Germany35 provided evidence of the effectiveness of alternative education delivery strategies for undergraduate dental students studying specific modules or courses in conservative dentistry with endodontics34 or operative dentistry35 during the COVID-19 pandemic. In one study, the teaching consisted of asynchronous online screencasts (screen-captured PowerPoint presentations with narrated audio) using Stud-IP (a source learning management system) and discussed via synchronous video meetings using the Zoom video videoconferencing platform.35 The other study used a blended learning approach using the Blackboard Collaborate platform.34 The outcome of interest across both studies was knowledge acquisition. Findings from both studies suggest that these alternative educational methods contributed towards knowledge and skill acquisition assessed through a self-assessment survey completed by fourth year students34 and through a final summative examination.35 However, the evidence suggests lower levels of knowledge for the subtopic of periodontology and lower levels of practical skills for 3rd year dental students when learning was conducted virtually compared to in-person. View this table: [Table 5:](http://medrxiv.org/content/early/2022/03/07/2022.03.04.22271892/T5) Table 5: Characteristics of included studies focusing on dental students ### Overview of evidence base for nursing students Three descriptive studies (see Table 6) conducted in Spain,36 Japan37 and USA38 provided evidence for the effectiveness of alternative educational delivery strategies for nursing students studying a specific module in human genomics,37 simulation in paediatric clinical practice38 and for the delivery of remote OSCEs with COPD patients36 during the COVID-19 pandemic. All three studies compared a group of students receiving a remotely delivered educational package with a group receiving standard, in-person education. In two studies the comparison groups were students from the previous, pre-COVID academic year, however, Weston and Zauche38 studied a cohort of students from the same academic year, 2019-2020, where half had received the standard educational package before the alternative version was introduced. Only one study used a pre-test / post-test design and thus compared results within as well as between groups.37 In this study, the conventional course was transferred to remote synchronous learning (narrative over PowerPoint), also uploading handouts and worksheets with no changes to content.37 Arrogante et al.36 used the virtual classroom platform Blackboard Collaborate to conduct OSCEs comprising eight simulated clinical scenarios with standardised patients. Weston and Zauche38 substituted virtual simulation using the i-Human platform to replace in-person clinical practice and simulation laboratory learning. Outcomes explored were competency (n=2)36,37 confidence (n=1),37 and knowledge (n=2).37,38 View this table: [Table 6:](http://medrxiv.org/content/early/2022/03/07/2022.03.04.22271892/T6) Table 6: Characteristics of included studies focusing on nursing students The evidence suggests that levels of competency were the same and levels of confidence were higher when learning or assessment was conducted virtually (2020) compared to in-person pre-COVID (2019). Knowledge improves regardless of whether the learning has been conducted virtually (2020) or in-person pre-COVID (2019). ### Overview of the evidence base for pharmacy students Four descriptive studies (see Table 7), all conducted in the USA, provided evidence for the effectiveness of alternative education delivery strategies for undergraduate pharmacy students studying specific modules or courses in integrated patient care,39 hypertension/drug information,40 advanced pharmacy experience,41 delivery of remote Objective Structured Clinical Examinations (OSCEs) for patient counselling, and taking a medical history42 during the COVID-19 pandemic. Two studies used a pre-test/post-test design,40,41 the remaining two reported a post-test only study design, with a comparison between the study population and an earlier (pre-COVID) cohort of students.39,42 View this table: [Table 7:](http://medrxiv.org/content/early/2022/03/07/2022.03.04.22271892/T7) Table 7: Characteristics of included studies focusing on pharmacy students In one study the teaching included remote synchronous learning,41 three studies used the Zoom videoconferencing platform,39,41,42 two studies used the University platform Blackboard Collaborate40 and one study also used the University Supported Management System: CANVAS.41 The outcomes of interest that were explored were competency (n=2),39,40 confidence (n=2),40,41 knowledge (n=2),39,41 skills (n=2)41,42 Evidence suggests competency outcomes improved across the course of learning and were similar when learning was conducted virtually (2020) compared to in-person pre-COVID (2019). Confidence was found to either improve across the course of learning or be the same for virtual (2020) compared to in-person pre-COVID (2019) learning. However, lower levels of knowledge were reported when learning was conducted virtually compared to in-person pre-COVID. The evidence suggests that, overall, students performed similarly between in-person (2019) and online (2020) OSCEs, although for some, skills performance was higher when students undertook these virtually. ## Discussion The findings of this rapid review are based on very limited evidence for dental (2 descriptive studies), pharmacy (4 descriptive studies) and nursing (3 descriptive studies) education. Only one finding from across all twelve of the descriptive studies that focused on medical education was rated as being of moderate quality. As expected, levels of knowledge, competency and confidence improved over the course of virtual learning. However, when results were compared to students who had completed in-person learning in the years before the COVID-19 pandemic, results were mixed. Most studies across the disciplines reported similar findings across all outcome variables suggesting that virtual learning produced similar results to in-person learning. To our knowledge this is the first rapid review of the effectiveness of alternative education delivery strategies for undergraduate and postgraduate medical, dental, nursing and pharmacy education during the COVID-19 pandemic. Previous systematic reviews showed online learning outcomes to be comparable to in-person learning. At the time of conducting this rapid review we were unable to locate any reviews that took an interdisciplinary approach. Given the potential overlap and value in sharing practices across the various healthcare educational contexts, we aimed to address this gap. Evidence from two RCTs showed that knowledge was greater when learning was conducted using bespoke interactive platforms compared with non-interactive formats, reported during the COVID pandemic.31,32 These findings concur with research conducted in the field prior to COVID-19, with three systematic reviews suggesting that pre-planned online eLearning for undergraduates in health professions is equivalent, possibly superior to traditional learning.43–45 Data from this rapid review indicated that the transition from traditional teaching into remote methods seemed to affect students’ performance at exams, particularly so for the practical based subjects in dentistry and medicine. It is recognised that emergency remote teaching and learning differs from planned online learning.46,47 Most remote teaching and learning that initially took place during the COVID-19 pandemic was not planned and was adapted promptly due to the emergency circumstances that presented. In addition, this new learning did not take into account the additional stress that e-learning can cause48 or incorporate strategies to increase social presence which Natajaran and Joseph49 argue is essential to improve student nurses’ satisfaction with online teaching. ### Implications for policy and practice Healthcare educators need to revisit the research base surrounding remote learning and consider this evidence when planning future online education. Whilst lessons learnt were quickly put into place, the COVID-19 pandemic brought issues to the fore that have long been debated in healthcare education: reduced clinical exposure, a move away from mass didactic education, and the need to ensure all healthcare students are provided with the skills and knowledge required to transition to competent caring health professionals with the ability to think critically and source and apply evidence to practice. With the increasing need for skilled healthcare professionals, policy makers need to consider how educational institutions can be provided with the resources required and how existing educators can be upskilled and supported to develop technology-enhanced learning experiences. Students from school entry age onwards need to be prepared for more online and blended learning experience which should include providing them with strategies they can use to support their emotional and psychological well-being, whilst accessing remote learning. Future research should investigate the effectiveness of blended learning approaches compared to more traditional education, in addition to investigating the views and perceptions of both students and educators and the barriers and facilitators to engaging effectively in blended learning. ## Limitations To complete the review rapidly a limited number of databases were searched, and further studies may have been identified if additional bibliographic databases had been used. Out of the 23 included studies none was conducted within the UK and the majority (n=21) were descriptive studies. All included studies focused on undergraduate not postgraduate education. Of these, 11 studies employed a pre-test/post-test design, and the remainder were post-test only evaluations. The two RCTs both used a test or examination to assess knowledge, but these evaluated two different interventions and therefore statistical pooling of data using meta-analysis was not appropriate. Furthermore, both studies had small sample sizes and poor response rates (75/158 and 44/58). Regarding the limitations of this review’s methods, the tool used for evaluating the confidence of the quantitative descriptive studies is an adaptation of GRADE and has not been approved by the tool’s originators. Finding well conducted comparative research proved challenging as not all educational researchers sign up to this experimental ideology when it comes to investigating teaching. Indeed, most published educational studies are small scale and qualitative in nature. There is, however, an agreement that there is a lack of high-quality studies to serve as models for future development in remote learning and teaching.50,51 We therefore suggest that studies that do apply the experimental approach should aim to enhance their research rigour in order for them to provide findings that can be synthesised more meaningfully. We also recognise the potential impact of the pandemic on resources and time, all of which would have likely impacted the quality of research. For this reason, we suggest that our rapid review provides a platform for further research that will consider the large body of literature that has emerged from the various fields of healthcare education since we conducted our review. ## Conclusions Remote teaching was valued, and learning was achieved, but the comparative effectiveness of virtual versus in-person teaching delivered in a pandemic is less clear. In addition, the available evidence is insufficient to demonstrate equivalence for student speciality groups and it is unclear whether planned remote teaching, rather than relying on emergency adaptation, would be more effective. For some healthcare students, academic achievement appears to decline when practical learning is insufficient, and this is something that must be addressed. However, this could be attributed to the sudden transition to online learning mid semester in which students did not have a chance to prepare or plan how they may need to adjust their own learning strategies. Moreover, teaching online requires a new skill set and educators may have had very little chance to upskill. It is therefore difficult to use the findings to inform future educational planning. Identifying which aspects of health education delivery are best delivered via a particular format or platform will be key to improving the efficiency of learning for organisations and accessibility of material for students. Time will tell as to the career progress of the students whose studies have been affected by COVID-19 with educators and regulators ensuring that health care professionals are supported in their learning and standards are maintained. Further research with robust methods to evaluate alternative education delivery strategies is needed to inform policy decision-making in this area. ## Data Availability All data produced in the present work are contained in the manuscript ## Data availability statement No data are associated with this article. ## Competing interests The authors declare they have no conflicts of interest to report. ## Grant information The Wales Centre for Evidence Based Care was funded for this work by the Wales COVID-19 Evidence Centre, itself funded by Health & Care Research Wales on behalf of Welsh Government. ## Extended data Additional material one: Full search strategies [http://www.primecentre.wales/resources/RR/Clean/RR00004\_Supplementary\_information\_Healthcare\_education.pdf](http://www.primecentre.wales/resources/RR/Clean/RR00004\_Supplementary_information_Healthcare_education.pdf) Additional material two Critical appraisal scores [http://www.primecentre.wales/resources/RR/Clean/RR00004\_Supplementary\_information\_Healthcare\_education.pdf](http://www.primecentre.wales/resources/RR/Clean/RR00004\_Supplementary_information_Healthcare_education.pdf) Additional material three: Tool for assessing the confidence of synthesised findings from quantitative descriptive studies [http://www.primecentre.wales/resources/RR/Clean/RR00004\_Supplementary\_information\_Healthcare\_education.pdf](http://www.primecentre.wales/resources/RR/Clean/RR00004\_Supplementary_information_Healthcare_education.pdf) Additional material four: Evaluation of confidence using GRADE [http://www.primecentre.wales/resources/RR/Clean/RR00004\_Supplementary\_information\_Healthcare\_education.pdf](http://www.primecentre.wales/resources/RR/Clean/RR00004\_Supplementary_information_Healthcare_education.pdf) Additional material; five: excluded studies [http://www.primecentre.wales/resources/RR/Clean/RR00004\_Supplementary\_information\_Healthcare\_education.pdf](http://www.primecentre.wales/resources/RR/Clean/RR00004\_Supplementary_information_Healthcare_education.pdf) ## Acknowledgements The authors would like to thank Assim Javaid for his contribution during stakeholder meetings to guide the focus of the review and interpret findings. In addition, thanks to Professor Jane Noyes for information regarding the adaption of the GRADE approach for quantitative descriptive studies. The authors would also like to express their gratitude to Maggie Hendry and acknowledge her contribution to the [Wales COVID-19 Evidence Centre report](http://www.primecentre.wales/resources/RR/Clean/RR00004\_Wales\_COVID-19\_Evidence\_Centre\_Rapid_Review_Healthcare_Education_August-2021.pdf). Maggie Hendry participated in study selection, data extraction, and quality assessment. ## Footnotes * 1 In the summer of an academic year, there are two “senior” classes (these are fourth year college students in America). The class that just graduated, known as graduating seniors, and the one that will be seniors, when fall comes around known as “oncoming senior” or “rising seniors.” * Received March 4, 2022. * Revision received March 4, 2022. * Accepted March 7, 2022. * © 2022, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/) ## References 1. Dedeilia A, Sotiropoulos MG, Hanrahan JG, Janga D, Dedeilias P, Sideris M. Medical and surgical education challenges and innovations in the COVID-19 era: A systematic review. In Vivo 2020;34:1603–11. [https://doi.org/10.21873/invivo.11950](https://doi.org/10.21873/invivo.11950). [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiaW52aXZvIjtzOjU6InJlc2lkIjtzOjE1OiIzNC8zX3N1cHBsLzE2MDMiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMi8wMy8wNy8yMDIyLjAzLjA0LjIyMjcxODkyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 2. NSW Health COVID-19 Critical Intelligence Unit. The impact of COVID-19 on clinical education and training. New South Wales: NSW Health COVID-19 Critical Intelligence Unit; 2020. 3. Wilcha R-J. Effectiveness of virtual medical teaching during the COVID-19 crisis: Systematic review. JMIR Medical Education 2020;6:e20963–e20963. [https://doi.org/10.2196/20963](https://doi.org/10.2196/20963). 4. Abi-Rafeh J, Azzi AJ. Emerging role of online virtual teaching resources for medical student education in plastic surgery: COVID-19 pandemic and beyond. Journal of Plastic, Reconstructive, and Aesthetic Surgery 2020;73:1575–92. [https://dx.doi.org/10.1016%2Fj.bjps.2020.05.085](https://dx.doi.org/10.1016%2Fj.bjps.2020.05.085). 5. Ahmed S, Zimba O, Gasparyan A. Moving towards online rheumatology education in the era of COVID-19. Clinical Rheumatology 2020;39:3215–22. https://doi.org/doi:10.1007/s10067-020-05405-9. 6. Al-Balas M, Al-Balas HI, Jaber HM, Obeidat K, Al-Balas H, Aborajooh E, et al. Distance learning in clinical medical education amid COVID-19 pandemic in Jordan: current situation, challenges, and perspectives. BMC Med Educ. 2020 Oct 2;20(1):341.Erratum in: BMC Med Educ. 2020 Dec 16;20(1):513. PMID: 33008392; PMCID: PMC7530879. BMC Medical Education 2020;20:513. [https://doi.org/10.1186/s12909-020-02257-4](https://doi.org/10.1186/s12909-020-02257-4). [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12909-020-02257-4&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33008392&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F07%2F2022.03.04.22271892.atom) 7. Haroon Z, Azad AA, Shariff AA, Aslam A, Rafiq S. COVID-19 era: Challenges and solutions in dental education. Journal of the College of Physicians and Surgeons 2020;30:129–31. [https://doi.org/10.29271/jcpsp.2020.supp2.129](https://doi.org/10.29271/jcpsp.2020.supp2.129). 8. Bakshi SK, Ho AC, Chodosh J, Fung A, Chan RVP, Ting DSW. Training in the year of the eye: the impact of the COVID-19 pandemic on ophthalmic education. British Journal of Ophthalmology 2020;104:1181–3. [https://doi.org/10.1136/bjophthalmol-2020-316991](https://doi.org/10.1136/bjophthalmol-2020-316991). [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MTI6ImJqb3BodGhhbG1vbCI7czo1OiJyZXNpZCI7czoxMDoiMTA0LzkvMTE4MSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIyLzAzLzA3LzIwMjIuMDMuMDQuMjIyNzE4OTIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 9. Tolyat M, Abolfazl Vagharseyyedin S, Nakhaei M. Education of nursing profession amid COVID-19 Pandemic: A qualitative study. Journal of Advances in Medical Education and Professionalism 2022;10:39–47. [https://doi.org/10.30476/jamp.2021.90779.1422](https://doi.org/10.30476/jamp.2021.90779.1422). 10. Dewart G, Corcoran L, Thirsk L, Petrovic K. Nursing education in a pandemic: Academic challenges in response to COVID-19. Nurse Education Today 2020;92:104471. [https://doi.org/10.1016/j.nedt.2020.104471](https://doi.org/10.1016/j.nedt.2020.104471). [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F07%2F2022.03.04.22271892.atom) 11. Moretti-Pires RO, de Campos DA, Zeno Junior CT, de Oliveira Junior JB, de Oliveira TB, de Oliveira DC. Pedagogical strategies in medical education to the challenges of Covid-19: Scoping review. Revista Brasileira de Educação Médica 2021;45:e025–e025. 12. Santos GNM, da Silva HEC, Leite AF, Mesquita CRM, Figueiredo PTS, Stefani CM, et al. The scope of dental education during COVID-19 pandemic: A systematic review. Journal of Dental Education 2021;85:1287–300. [https://doi.org/10.1002/jdd.12587](https://doi.org/10.1002/jdd.12587). 13. He L, Yang N, Xu L, Ping F, Li W, Sun Q, et al. Synchronous distance education vs traditional education for health science students: A systematic review and meta-analysis. Medical Education 2021;55:293–308. [https://doi.org/10.1111/medu.14364](https://doi.org/10.1111/medu.14364). [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F07%2F2022.03.04.22271892.atom) 14. Jowsey T, Foster G, Cooper-Ioelu P, Jacobs S. Blended learning via distance in pre-registration nursing education: A scoping review. Nurse Education in Practice 2020;44:102775. [https://doi.org/10.1016/J.NEPR.2020.102775](https://doi.org/10.1016/J.NEPR.2020.102775). 15. Thomas J, O’Mara_eves A, Harden A, Newman M. Synthesis methods for combining and configuring textual or mixed methods data. An introduction to systematic reviews, vol. Chapter 8. 2nd edition. London: Sage publications limited; 2017. p. 181–209. 16. Guyatt GH, Oxman AD, Vist GE, Kunz R, Fiack-Ytter Y, Alosno-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008;336:924–6. [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMzYvNzY1MC85MjQiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMi8wMy8wNy8yMDIyLjAzLjA0LjIyMjcxODkyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 17. Ryan R, Hill S. How to GRADE the quality of the evidence. Cochrane Consumers and Communication Group; 2016. 18. World Health Organisation. Communicating risk in public health emergencies. A WHO guideline for emergency risk communication (ERC) policy and practice. Geneva: World Health Organisation; 2017. 19. Page MJ, McKenzie JE, Bosstuyt PM, Boultron I, Hoffman TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. [https://doi.org/10.1136/bmj.n71](https://doi.org/10.1136/bmj.n71). [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE1OiIzNzIvbWFyMjlfMi9uNzEiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMi8wMy8wNy8yMDIyLjAzLjA0LjIyMjcxODkyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 20. Kasai H, Shikino K, Saito G, Tsukamoto T, Takahashi Y, Kuriyama A, et al. Alternative approaches for clinical clerkship during the COVID-19 pandemic: online simulated clinical practice for inpatients and outpatients-A mixed method. BMC Med Educ 2021;21:149. [https://doi.org/10.1186/s12909-021-02586-y](https://doi.org/10.1186/s12909-021-02586-y). 21. Martini ML, Yaeger KA, Kellner CP, Hadjipanayis C, Shrivastava R, Mocco J, et al. Student survey results of a virtual medical student course developed as a platform for neurosurgical education during the coronavirus disease 2019 pandemic. World Neurosurg 2021;28:28. [https://doi.org/10.1016/j.wneu.2021.05.076](https://doi.org/10.1016/j.wneu.2021.05.076). 22. Monday LM, Gaynier A, Berschback M, Gelovani D, Kwon HY, Ilyas S, et al. Outcomes of an Online Virtual Boot Camp to Prepare Fourth-Year Medical Students for a Successful Transition to Internship. Cureus 2020;12:e8558. [https://doi.org/10.7759/cureus.8558](https://doi.org/10.7759/cureus.8558). 23. Pang JH, Finlay E, Fortner S, Pickett B, Wang ML. Teaching effective informed consent communication skills in the virtual surgical clerkship. J Am Coll Surg 2021;233:64–72 e2. [https://doi.org/10.1016/j.jamcollsurg.2021.04.026](https://doi.org/10.1016/j.jamcollsurg.2021.04.026). 24. Qaranto BR, Lamb M, Traversone J, Hu J, Lukan J, Cooper C, et al. Development of an Interactive Remote Basic Surgical Skills Mini-Curriculum for Medical Students During the COVID-19 Pandemic. Surg Innov 2021;28:220–5. [https://doi.org/10.1177/15533506211003548](https://doi.org/10.1177/15533506211003548). 25. Rosenthal HB, Sikka N, Lieber AC, Sanky C, Cayon C, Newman D, et al. A near-peer educational model for online, Interactive learning in emergency medicine. West J Emerg Med 2020;22:130–5. [https://doi.org/10.5811/westjem.2020.12.49101](https://doi.org/10.5811/westjem.2020.12.49101). 26. Darici D, Reissner C, Brockhaus J, Missler M. Implementation of a fully digital histology course in the anatomical teaching curriculum during COVID-19 pandemic. Ann Anat 2021;236:151718. [https://doi.org/10.1016/j.aanat.2021.151718](https://doi.org/10.1016/j.aanat.2021.151718). 27. Harendza S, Gartner J, Zelesniack E, Prediger S. Evaluation of a telemedicine-based training for final-year medical students including simulated patient consultations, documentation, and case presentation. GMS J Med Educ 2020;37:Doc94. [https://doi.org/10.3205/zma001387](https://doi.org/10.3205/zma001387). 28. Kim JW, Myung SJ, Yoon HB, Moon SH, Ryu H, Yim JJ. How medical education survives and evolves during COVID-19: Our experience and future direction. PLoS ONE 2020;15:e0243958. [https://doi.org/10.1371/journal.pone.0243958](https://doi.org/10.1371/journal.pone.0243958). [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0243958&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33338045&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F07%2F2022.03.04.22271892.atom) 29. Nathaniel TI, Black AC. An Adaptive Blended Learning Approach in the Implementation of a Medical Neuroscience Laboratory Activities. Med Sci Educ 2021;29:1–11. [https://doi.org/10.1007/s40670-021-01263-5](https://doi.org/10.1007/s40670-021-01263-5). 30. Totlis T, Tishukov M, Piagkou M, Kostares M, Natsis K. Online educational methods vs. traditional teaching of anatomy during the COVID-19 pandemic. Anat Cell Biol 2021;25:25. [https://doi.org/10.5115/acb.21.006](https://doi.org/10.5115/acb.21.006). 31. Schmitz SM, Schipper S, Lemos M, Alizai PH, Kokott E, Brozat JF, et al. Development of a tailor-made surgical online learning platform, ensuring surgical education in times of the COVID19 pandemic. BMC Surg 2021;21:196. [https://doi.org/10.1186/s12893-021-01203-5](https://doi.org/10.1186/s12893-021-01203-5). 32. Suppan M, Stuby L, Carrera E, Cottet P, Koka A, Assal F, et al. Asynchronous distance learning of the National Institutes of Health Stroke Scale during the COVID-19 pandemic (e-learning vs video): Randomized controlled trial. J Med Internet Res 2021;23:e23594. [https://doi.org/10.2196/23594](https://doi.org/10.2196/23594). 33. Redinger KE, Greene JD. Virtual emergency medicine clerkship curriculum during the COVID-19 pandemic: Development, application, and outcomes. West J Emerg Med 2021;22:792–8. [https://doi.org/10.5811/westjem.2021.2.48430](https://doi.org/10.5811/westjem.2021.2.48430). 34. Nijakowski K, Lehmann A, Zdrojewski J, Nowak M, Surdacka A. The effectiveness of the blended learning in conservative dentistry with endodontics on the basis of the survey among 4th-year students during the COVID-19 pandemic. International Journal of Environmental Research and Public Health 2021;18:. [https://doi.org/10.3390/ijerph18094555](https://doi.org/10.3390/ijerph18094555). 35. Kanzow P, Krantz-Schäfers C, Hülsmann M. Remote teaching in a preclinical phantom course in operative dentistry during the COVID-19 pandemic: observational case study. JMIR Medical Education 2021;7:e25506. [https://doi.org/10.2196/25506](https://doi.org/10.2196/25506). 36. Arrogante O, López-Torre EM, Carrión-García L, Polo A, Jiménez-Rodríguez D. High-fidelity virtual objective structured clinical examinations with standardized patients in nursing students: An innovative proposal during the COVID-19 pandemic. Healthcare 2021;9:. [https://doi.org/10.3390/healthcare9030355](https://doi.org/10.3390/healthcare9030355). 37. Kawasaki H, Yamasaki S, Masuoka Y, Iwasa M, Fukita S, Matsuyama R. Remote teaching due to COVID-19: An exploration of its effectiveness and issues. Int J Environ Res Public Health 2021;18:1–17. [https://doi.org/10.3390/ijerph18052672](https://doi.org/10.3390/ijerph18052672). [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3390/ijerph18168831&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F07%2F2022.03.04.22271892.atom) 38. Weston J, Zauche LH. Comparison of Virtual Simulation to Clinical Practice for Prelicensure Nursing Students in Pediatrics. Nurse Educ 2020;07:07. [https://doi.org/10.1097/NNE.0000000000000946](https://doi.org/10.1097/NNE.0000000000000946). 39. Phillips BB, Palmer R, Chastain DB, Smith K, Bland CM. Impact of remote delivery on a pharmacists’ patient care process capstone course on the development of patient work-up skills. Journal of the American College of Clinical Pharmacy 2021;4:162–8. [http://dx.doi.org/10.1002/jac5.1376](http://dx.doi.org/10.1002/jac5.1376). 40. Cowart K, Updike WH. Pharmacy student perception of a remote hypertension and drug information simulation-based learning experience in response to the SARS-CoV-2 pandemic. JACCP Journal of the American College of Clinical Pharmacy 2021;4:53–9. [http://dx.doi.org/10.1002/jac5.1348](http://dx.doi.org/10.1002/jac5.1348). 41. Singh S, Roy D, Sinha K, Parveen S, Sharma G, Joshi G. Impact of COVID-19 and lockdown on mental health of children and adolescents: A narrative review with recommendations. Psychiatry Research 2020;293:113429. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.psychres.2020.113429&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32882598&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F07%2F2022.03.04.22271892.atom) 42. Scoular S, Huntsberry A, Patel T, Wettergreen S, Brunner JM. Transitioning Competency-Based Communication Assessments to the Online Platform: Examples and Student Outcomes. Pharmacy (Basel) 2021;9:52. [https://doi.org/10.3390/pharmacy9010052](https://doi.org/10.3390/pharmacy9010052). 43. George PP, Papachristou N, Belisario JM, Wang W, Wark PA, Cotic Z, et al. Online eLearning for undergraduates in health professions: A systematic review of the impact on knowledge, skills, attitudes and satisfaction. J Glob Health 2014;4:010406. [https://doi.org/10.7189/jogh.04.010406](https://doi.org/10.7189/jogh.04.010406). [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F07%2F2022.03.04.22271892.atom) 44. Liu Q, Peng W, Zhang F, Hu R, Li Y, Yan W. The effectiveness of blended learning in health professions: Systematic review and meta-analysis. J Med Internet Res 2016;18:e2. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F07%2F2022.03.04.22271892.atom) 45. Vallee A, Blacher J, Cariou A, Sorbets E. Blended learning compared to traditional learning in medical education: Systematic review and meta-analysis. Journal of Medical Internet Research 2020;22:e16504. [https://doi.org/10.2196/16504](https://doi.org/10.2196/16504). [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F07%2F2022.03.04.22271892.atom) 46. Hodges C, Moore S, Lockee B, Trust T, Bond A. The difference between emergency remote teaching and online learning. Educause Review 2020;27:1–12. 47. TASO. Online teaching and learning (post-entry). 2021. URL: [https://taso.org.uk/intervention/online-teaching-and-learning-post-entry/](https://taso.org.uk/intervention/online-teaching-and-learning-post-entry/). 48. Kabir H, Hasan MK, Mitra DK. E-learning readiness and perceived stress among the university students of Bangladesh during COVID-19: a countrywide cross-sectional study. Ann Med. 2021;53(1):2305–2314. doi:10.1080/07853890.2021.2009908n.d. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/07853890.2021.2009908n.d&link_type=DOI) 49. Natarajan J, Jospeh MA. Impact of emergency remote teaching on nursing students’ engagement, social presence, and satisfaction during the COVID-19 pandemic. Nursing Forum 2022;57:42–8. [https://doi.org/10.1111/nuf.12649](https://doi.org/10.1111/nuf.12649). 50. Daniel M, Gordon M, Patricio M, Hider A, Pawlik C, Bhagdev R, et al. An update on developments in medical education in response to the COVID-19 pandemic: A BEME scoping review: BEME Guide No. 64. Medical Teacher 2021;43:253–71. [https://doi.org/10.1080/0142159X.2020.1864310](https://doi.org/10.1080/0142159X.2020.1864310). [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/0142159X.2020.1864310&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F07%2F2022.03.04.22271892.atom) 51. Gordon M, Patricio M, Horne L, Muston A, Alston SR, Pammi M, et al. Developments in medical education in response to the COVID-19 pandemic: A rapid BEME systematic review: BEME Guide No. 63. Medical Teacher 2020;42:1202–15. [https://doi.org/10.1080/0142159X.2020.1807484](https://doi.org/10.1080/0142159X.2020.1807484). [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F07%2F2022.03.04.22271892.atom) 52. Singh H, Al Jammali Z, Bookman N, Ostrogorsky TL, McCracken C, Olstad S, et al. Zooming forward: An advanced pharmacy practice experience utilizing virtual case-based learning in response to COVID-19. Journal of the American College of Clinical Pharmacy 2021;4:184–94. [http://dx.doi.org/10.1002/jac5.1384](http://dx.doi.org/10.1002/jac5.1384).