A Bibliometric Analysis of Global Pediatric Emergency Medicine Research Networks ================================================================================ * Michael J Barrett * Stuart Dalziel * Mark Lyttle * Ronan O’Sullivan * for the Pediatric Emergency Research Networks (PERN) ## Abstract **Objective** During the last three decades newly formed pediatric emergency medicine (PEM) research networks have been publishing research. A desire of these networks is to produce and disseminate research to improve patient health and outcomes. To quantitatively analyze and compare the literature by PEM research networks globally through numeric and visual bibliometrics. **Methods** A bibliometric analysis of articles published from 1994 to 2019 (26 years) by authors from PEM research networks globally were retrieved using PubMed®, Web of Science™ (Thompson Reuters) and accessing individual research network databases. Bibliometric analysis was performed utilizing Web of Science™, VOSviewer and Dimensions. Research was quantified to ascertain the number of articles, related articles, citations and Altmetric attention score. **Results** A total of 493 articles were published across nine research networks in three decades. Pediatric Emergency Care Applied Research Network (PECARN) produced the most articles, citations, and h-index of all networks. We identified three main groupings of productive authors across the networks who collaborate globally. The gender of the first author was female in 46% of publications and the corresponding author(s) was female in 45%. A non-significant moderate positive correlation between the number of years publishing and the number of publications was identified. There was non-significant moderate negative association between the number of countries in a network and total publications per annum. **Conclusions** This study is the first bibliometric analysis of publications from PEM research networks that collaborate globally. The gender gap in first authorship compared to high impact medical journals and high impact emergency medicine journals is narrower. Exploring the relationships of numerical bibliometric indicators and visualizations of productivity will benefit the understanding of the generation, reach and dissemination of PEM research within the global research community. Keywords * Pediatric Emergency Medicine * Research * Bibliometrics * Biomedical Research/ Trends * Authorship ## Introduction Twenty seven years has passed since the first publication by a pediatric emergency medicine (PEM) research network (PEM Collaborative Research Committee (PEM CRC) of the American Academy of Pediatrics). 1 In the intervening years eight PEM research networks have united as a global body, the Pediatric Emergency Research Network (PERN), which consists of representatives from four of the six World Health Organization (WHO) regions. The desire of these networks is to produce and disseminate high-quality research to improve health and outcomes of acutely ill and injured children and youths throughout the world. 2-8 The global proliferation of research to massive proportions has expanded evidence, but has simultaneously made it difficult to gain oversight, by traditional methods, of the architecture of relevant research to cohere what is published on a given topic or specialty of interest. 9,10 Bibliometrics quantifies data for scientific output and has become increasingly important and valuable to researchers, research networks, investing stakeholders, funding agencies and policy makers. 9,11 The production of quantitative metrics can challenge bias tendencies in peer review and can facilitate further deliberation on the methods of achieving and optimizing the research missions of the PEM research networks.11 The aim of this study is to describe and analyze the literature by global PEM research networks through numeric and visual bibliometrics. ## Methods ### PEM Research Networks The networks included are Network for Research and Development of PEM in Latin America (Red de Investigacion y Desarrollo de la Emergencia Pediatrica Latinoamericana, RIDEPLA), Research in European Pediatric Emergency Medicine (REPEM), Spanish Pediatric Emergency Research Group (RISeuP/SPERG), Pediatric Emergency Care Applied Research Network (PECARN), Pediatric Emergency Medicine Collaborative Research Committee of the American Academy of Pediatrics (PEM CRC), Pediatric Emergency Research Canada (PERC), Paediatric Research in Emergency Departments International Collaborative (PREDICT) and Paediatric Emergency Research in the United Kingdom and Ireland (PERUKI). Each of these networks are affiliated with PERN. ### Search Strategy/Data Mining Publications were retrieved from Web of Science™ (Thompson Reuters, Toronto, Canada), Medline®/PubMed® and cross referenced from each Research Network (Table 1). A preliminary search of network publications identified 1994 as the first year of publication, which established the timeframe for the analysis of 1994-2019 (Last search 9 March 2020). Publications were identified by searching for the individual research network in the Authorship, Corporate Authorship, Abstract, or Title of the publication. First and corresponding authors were categorized based on author list ordering and correspondence details. Gender was determined by review of the given/first name. For articles that only listed the authors’ first initials, attempts were made to find the full first name through additional publication databases (e.g., PubMed), academic affiliate institutions and internet search engines (e.g., Google). This was manually cross referenced with publications retrieved from research networks in March 2018 and 2020. EndNote™ X8.2 software was used to remove duplicates and update references. View this table: [Table 1.](http://medrxiv.org/content/early/2021/04/20/2021.04.14.21255485/T1) Table 1. Search Strategy ### Toolkits for Bibliometric analysis Bibliometric analysis of research network publications includes the year of publication, research areas, types of documents, keywords, language, articles, authors, journals, and countries by PubMed®. Citations, citations per year, the number of citations per article and h-index were established in ISI Web of Science. The bibliometric mapping and visualization techniques are based on extensive bibliometric and information visualization literature. 12 This was conducted using VOSviewer (Leiden University, Belgium) ([http://www.vosviewer.com](http://www.vosviewer.com)). 13 Terms (Keywords and Authors) are in the 2D map in such a way that the distance between terms gives an indication of their relatedness. The relatedness of terms is determined by counting the number of times terms co-occur. The font size of a term is dependent on the number and strength of co-occurrence with other words in publications. Each of these terms occur a minimum of 5 times. The “overlay visualization” map returns a network that brings incidences of co-authorship or keywords and brings information of chronological order. This visualization is the foundation of the architecture of research that is represented by terms and can be used to assess trends in a research field. 9 Publications identified by the search in Table 1 were further analyzed by the Dimensions Artificial Intelligence program (Cambridge, MA, USA) ([https://app.dimensions.ai/](https://app.dimensions.ai/)). 14 This links publications and citations with grants, patents, clinical trials, datasets, and policy papers to deliver a more holistic view of the research landscape. Dimensions analysis produces the Altmetric Attention Score, which is a weighted count of the online ‘mentions’ a publication has received. This is a measure of research impact, with immediately available information on the reach and influence of an article, and an ability to track how the attention changes over time. 15 ### Statistical analysis Data are presented as median with interquartile range (IQR). Pearson’s correlation coefficient determined associations between continuous variables. Data were analyzed using Microsoft Excel 2016. Statistical significance was set at 2-tailed P <.05. ### Ethical Considerations This retrospective observational study did not involve human subjects, and ethical approval was not required. ## Results ### Publications Between 1994 and 2019, a total of 493 publications were published in journals by PEM research networks. The first publication was in 1994, the annual publication count had increased to 97 publications in 2019 (Figure 1), and PERN was established in October 2009 (blue arrow). Since 2004 the number of publications per annum has been steadily increasing. The number of publications per annum has surpassed the linear trendline of annual publications since 2016 (Figure 1). Full free text was available for 250 publications. The gender of the first author was female in 46% (227) of publications. The corresponding author(s) were female in 45% of publications. ![Figure 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/04/20/2021.04.14.21255485/F1.medium.gif) [Figure 1.](http://medrxiv.org/content/early/2021/04/20/2021.04.14.21255485/F1) Figure 1. Total annual publications across all networks and number of research networks Blue Arrow: Paediatric Emergency Research Network (PERN) inception in 2009 The range of publication counts by each group and related metrics is demonstrated in Table 2. There was a non-significant moderate positive correlation between the number of years publishing and the number of publications (Pearson’s coefficient R =.58; P=.12). PECARN had the highest number of publications, highest sum of times cited, and h-index, whilst PEM CRC had the highest average citations per publication. PEM CRC is the most mature network and RIDEPLA is the youngest network. A non-significant moderate positive correlation was observed between the number of years publishing and that network’s sum of citations (Pearson’s Correlation Coefficient R =.66; P=.07). There was non-significant moderate negative association between the number of countries in a network and total publications (Pearson’s Correlation Coefficient R = -.53; P=.17). View this table: [Table 2.](http://medrxiv.org/content/early/2021/04/20/2021.04.14.21255485/T2) Table 2. Research Network Publication Performance from 1994 to 2019 The relatedness of countries and keywords over time reveals initial establishment of the United States (dark green) in the literature with temporal establishment of Canada, Australia, New Zealand (light green) and more recently United Kingdom (and Ireland) and Spain (yellow) (Figure 2a). A similar temporal progression of publication keywords is noted with croup (dark green) progressing sequentially to bronchiolitis/asthma, meningitis, fever, sepsis/resuscitation, pain management, wounds and injuries, athletic injuries and more recently onto traumatic brain injuries (yellow) (Figure 2a). ![Figure 2a.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/04/20/2021.04.14.21255485/F2.medium.gif) [Figure 2a.](http://medrxiv.org/content/early/2021/04/20/2021.04.14.21255485/F2) Figure 2a. Temporal trends of the regional network output and progression of topics denoted by keywords from 2005-2019 The most productive authors are clearly identified by the size of the weighted circles (Figure 2b) and appear to be centered within distinctive groups of international authors (e.g., Kuppermann/Dayan, Babl/Lyttle/Borland/Bressan/Neutz and Freedman/Plint/Boutis). The multiple and extensive collaborations between authors and groups is clearly demonstrated by lines travelling between authors within and across separate groupings (Figure 1b). The temporal development of authors and their collaborations are clearly demonstrated by the “overlay visualization” in Figure 2b. ![Figure 2b:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/04/20/2021.04.14.21255485/F3.medium.gif) [Figure 2b:](http://medrxiv.org/content/early/2021/04/20/2021.04.14.21255485/F3) Figure 2b: Map of Authors and collaborations in Publications from 1994-2019 Legend: More productive authors are identified by the larger size of the weighted circles and linking lines reveal collaborations The most impactful research publications have been produced by PECARN, PERC and PEMCRC (Table 3) and these are the 3 longest formed research networks (Table 2). The most impactful publications have all been issued in the last three years. The median (IQR) year of most impactful publications was 2018 (2016-2019). The topics of these articles were concussion, status epilepticus, gastroenteritis, asthma, bronchiolitis, diabetic ketoacidosis, serious bacterial infection, and procedural sedation. Five of the top 10 impactful publications have been published in *JAMA* publications, four in the *New England Journal of Medicine* and one in *Pediatrics*. View this table: [Table 3.](http://medrxiv.org/content/early/2021/04/20/2021.04.14.21255485/T3) Table 3. Top 10 Cited and Impactful Publications from Research Networks The top cited publications have been produced by PECARN, PERC and PEMCRC (Table 3). These publications have been issued over the past 19 years. The median (IQR) year of top cited publications was 2009 (2006-2011). The topics of these publications were head injury, cardiac arrest, serious bacterial infection, bronchiolitis, and diabetic ketoacidosis. Four were published in the *New England Journal of Medicine*, two in *JAMA*, one each in *The Lancet, Canadian Medical Association Journal, Pediatrics* and *Pediatric Critical Care Medicine*. ## Discussion This study has examined and compared, through visual and numeric bibliometric analysis, PEM research network publications from 1994 to 2019. The annual product of research networks is increasing, and since the formation of PERN in 2009 the number of global annual publications has increased as has the number of research networks contributing (Figure 1). Five research networks, including PERN, have been established in the last decade, with RIDEPLA the most recently formed in 20167 (Table 2). Forty six percent of publications had a female first author and 45 % with female corresponding author. This is a much narrower gender gap in first authorship compared to high impact medical journals (37% in 2014) and high impact emergency medicine journals (24-45% in 2018).16,17. The most productive networks in terms of number of publications are based in North America, followed by Australia and New Zealand, Europe, and South America. In 2007, Wilson et al reported 58.5% of 14,000 emergency medicine articles were produced from the United States followed by the United Kingdom.16 We observed a moderate positive correlation between the number of years publishing and the number of publications. A moderate positive correlation was observed between the number of years publishing and that network’s sum of citations. Interestingly we identified moderate negative association between the number of countries in a network and average publications per annum of existence. Although this relationship is statistically non-significant, this negative correlation may reflect the additional challenges that multinational research networks undertake when conducting studies (e.g., heterogenic capacity to conduct research (i.e., randomized controlled trials), complexity of multi-country governance and research agreements, national legislation, and funding variability). The coalescing of these networks under the banner of PERN has facilitated extensive collaboration between groups of researchers and networks, as is clearly demonstrated in Figure 2b. More mature research networks may potentially provide considerable experiential expertise with respect to the five primary functions of a health research system (1. Governance and management; 2. Financing; 3. Knowledge generation; 4. Utilization and management of knowledge; and 5. Capacity development).17 Newer and younger networks potentially stand to gain from this expertise by developing in a shorter duration by learning and collaborating with more established networks. In bibliometrics current best practice is to use multiple indicators to provide a more robust picture.11 We identify the top-cited articles across all networks. All top-cited publications originate out of the 3 networks longest in existence. This may reflect a more mature network’s stage of development with the production of high-quality research. The h-index is a metric to evaluate research productivity and it varies by research field.18 In a study of worldwide emergency medicine researchers’ productivity, the h-index had the best coefficient of determination of productivity compared to number of published papers, impact factor or number of citations.18 The highest h-index was from the PECARN network closely followed by PERC, PEM CRC and PREDICT (Table 2). The Altmetric Attention Scores can be used to quickly identify which publications have been widely discussed and shared amongst academic and broader audiences.19 The top 10 research network publications by Altmetric scores may be one way to discern which articles are deemed important from a dissemination and knowledge translation perspective by assessing non-traditional uptake of the literature. The top scoring articles published help us recognize the quality of the works, discoveries, and trends steering PEM network research. Our paper has some important limitations. We conducted a visual and numeric analysis of citations and altmetrics from Pediatric Emergency Medicine Research networks affiliated to PERN. To evaluate articles, we had to develop a unique search strategy. We did not provide a dynamic overview of the current state of global PEM research for comparison. However, it is unlikely that this would provide additional material to the global overview that we provide. Publications anecdotally affiliated to a network could not be represented in this study. No one bibliometric measurement is an indicator of the quality of a research group, therefore multiple simple indicators were used to enhance the transparency of the analysis in line with best practice.11 We used the Web of Science for our bibliometric analysis. No bibliometric database is superior.20 Scopus, Medline, and Google Scholar are other available bibliometric databases. Citation analysis is based on the absolute number of citations that an article receives and many factors that affect citation rates such as older articles. We did not perform an analysis to correlate study funding or study design to research networks. ## Conclusions The formation of research networks has led to increased volume, as well as increased excellence determined by h-index and citations in PEM research. This study is the first bibliometric analysis of publications from PEM research networks that unite globally and provides insights into the performance of the literature produced. We have applied large-scale data analysis to PEM research network publications using numerical bibliometric indicators and visualizations of productivity. We have identified the gender gap in first authorship compared to high impact medical journals and high impact emergency medicine journals is narrower. Future research to explore these relationships of numerical bibliometric indicators and visualizations of productivity and the temporal trends will benefit the understanding of the generation, reach and dissemination of PEM literature within the global research community. ## Data Availability All data is available from the corresponding author upon reasonable request. ## Footnotes * **Funding Sources/Disclosures:** None * Received April 14, 2021. * Revision received April 14, 2021. * Accepted April 20, 2021. * © 2021, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/) ## References 1. 1.Losek JD. Characteristics, workload, and job satisfaction of attending physicians from pediatric emergency medicine fellowship programs. Pediatric Emergency Medicine Collaborative Research Committee. Pediatr Emerg Care. 1994;10(5):256–259. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/00006565-199410000-00003&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=7845849&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F04%2F20%2F2021.04.14.21255485.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1994PQ42400002&link_type=ISI) 2. 2.Mintegi S, Lyttle MD, Maconochie IK, et al. From cradle to adolescence: the development of Research in European Pediatric Emergency Medicine. Eur J Emerg Med. 2014;21(1):24–29. 3. 3.Klassen TP, Acworth J, Bialy L, et al. Pediatric Emergency Research Networks: a global initiative in pediatric emergency medicine. Eur J Emerg Med. 2010;17(4):224–227. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20523220&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F04%2F20%2F2021.04.14.21255485.atom) 4. 4.Pediatric Emergency Care Applied Research N. The Pediatric Emergency Care Applied Research Network (PECARN): rationale, development, and first steps. Acad Emerg Med. 2003;10(6):661–668. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1197/aemj.10.6.661&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=12782530&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F04%2F20%2F2021.04.14.21255485.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000183369000013&link_type=ISI) 5. 5.Babl FE, Borland M, Ngo PK, et al. Paediatric research in emergency departments international collaborative (predict). J Paediatr Child Health. 2005;41(11):614–615. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1440-1754.2005.00733_6.x&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=16398854&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F04%2F20%2F2021.04.14.21255485.atom) 6. 6.Mintegi S, Gomez B, de la Torre M. [Progress of the Spanish Pediatric Emergency Research Group (RISEUP-SPERG)]. Emergencias. 2015;27(3):208–209. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F04%2F20%2F2021.04.14.21255485.atom) 7. 7.Kohn-Loncarica GA, Fustinana AL, Jabornisky RM, et al. How Are Clinicians Treating Children With Sepsis in Emergency Departments in Latin America?: An International Multicenter Survey. Pediatr Emerg Care. 2019. 8. 8.Lyttle MD, O’Sullivan R, Hartshorn S, et al. Pediatric Emergency Research in the UK and Ireland (PERUKI): developing a collaborative for multicentre research. Arch Dis Child. 2014;99(6):602–603. [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MTI6ImFyY2hkaXNjaGlsZCI7czo1OiJyZXNpZCI7czo4OiI5OS82LzYwMiI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIxLzA0LzIwLzIwMjEuMDQuMTQuMjEyNTU0ODUuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 9. 9.Uddin S, Khan A, Baur LA. A framework to explore the knowledge structure of multidisciplinary research fields. PLoS One. 2015;10(4):e0123537. 10. 10.Rodrigues SP, van Eck NJ, Waltman L, Jansen FW. Mapping patient safety: a large-scale literature review using bibliometric visualisation techniques. BMJ Open. 2014;4(3):e004468. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMToiNC8zL2UwMDQ0NjgiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMS8wNC8yMC8yMDIxLjA0LjE0LjIxMjU1NDg1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 11. 11.Hicks D, Wouters P, Waltman L, de Rijcke S, Rafols I. Bibliometrics: The Leiden Manifesto for research metrics. Nature. 2015;520(7548):429–431. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/520429a&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25903611&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F04%2F20%2F2021.04.14.21255485.atom) 12. 12.Leydesdorff L. Katy Borner: Atlas of science: visualizing what we know: The MIT Press, Cambridge, MA/London, UK, 2010, US$20. Scientometrics. 2011;88(2):675–677. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21836764&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F04%2F20%2F2021.04.14.21255485.atom) 13. 13.van Eck NJ, Waltman L. Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics. 2010;84(2):523–538. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s11192-009-0146-3&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20585380&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F04%2F20%2F2021.04.14.21255485.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000278695500019&link_type=ISI) 14. 14.Hook DW, Porter SJ, Herzog C. Dimensions: Building Context for Search and Evaluation. Frontiers in Research Metrics and Analytics. 2018;3(23). 15. 15.Elmore SA. The Altmetric Attention Score: What Does It Mean and Why Should I Care? Toxicol Pathol. 2018;46(3):252–255. 16. 16.Filardo G, da Graca B, Sass DM, et al. Trends and comparison of female first authorship in high impact medical journals: observational study (1994-2014). BMJ. 2016;352:i847. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNTIvbWFyMDJfMTcvaTg0NyI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIxLzA0LzIwLzIwMjEuMDQuMTQuMjEyNTU0ODUuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 17. 17.Webb J, Cambron J, Xu KT, et al. First and last authorship by gender in emergency medicine publications-a comparison of 2008 vs. 2018. The American Journal of Emergency Medicine. 2020.