Practical Challenges for Commercial Enterprises in the Ethics Review Process for Digital Health Research: Document Analysis and Interview Study =============================================================================================================================================== * Katherine Yang * Henry W W Potts ## Abstract **Introduction** The rapid evolution of digital health interventions has created challenges in navigating the ethics approval process for commercial enterprises. Recognising the need for processes that balance ethical considerations with the specifics of digital health research, this study aimed to describe what happens when enterprises seek ethical review in the UK and propose strategies for a smoother process. **Methods** Inductive thematic analysis was conducted on thirty-two ethics review documents (29 to an NHS Research Ethics Committee, 3 to an ethics committee at a higher education institution) submitted by digital health developers with commercial sponsors and ten semi-structured interviews with digital health enterprise representatives. **Results** Ethics committees raised an average of 4.3 action points per submission. We identified five broad themes around committees’ concerns: ethical commitments in care; study design; digital health research peculiarities; data governance; document quality and completeness. Interviewees reported a range of experiences. Here, we identified six broad themes: submission and protocol revisions; the dynamic between parties; application time and procedures; acumen and practicality in digital health; support and guidance from RECs; enterprise expertise and resources. **Conclusion** We suggest strategies for applicants to achieve a favourable decision, such as evidence-based study designs and participant support for better inclusion and equity, and identified specific pitfalls to avoid, such as lack of justification for data governance procedures. We recommend that UK research ethics committees provide adapted guidance and foster collaboration through open communication and mutual understanding, to facilitate a smoother approval process in digital health research. Keywords * Ethical review * Ethics committees * Informatics * Commerce * Public-private sector partnerships * Digital health ## Introduction Digital health encompasses the use of a wide range of technology, including mobile apps, wearables, and telemedicine, to enhance health outcomes and services 1, 2. Recent years have witnessed accelerated developments in digital health technology, propelled by pandemic initiatives 3. While digital health interventions have vast potential, there is concern that they are infrequently and poorly evaluated 4. The UK has made strides to enhance the evaluation of digital health technology 5, 6, exemplified by the NICE Evidence standards framework for digital health technologies 7, 8. Research ethics committees (RECs, also known as Institutional Review Boards in the US) scrutinise the ethical aspects of research proposals and provide a formal opinion. Since the 1960s, RECs have safeguarded human subjects’ rights 9,10, adhering to codes of ethics from professional associations and educational institutes to guide standards 11, 12. Research ethics is not governed by a single legislation in the UK. However, obtaining ethical approval before commencing on a research study from entities like UK Research Ethics Service (RES) or higher education institutions (HEIs) is essential for journal publication and further implementation of results 13, 14. The NHS Health Research Authority (HRA) in England and equivalents in the devolved administrations collaborate on the Research Ethics Service (RES), overseeing multiple RECs across the UK, providing reviews for research involving NHS patients, staff, or facilities. This national system represents more centralisation than, for example, the North American context. Applications to RES RECs are made via a web portal, the Integrated Research Application System (IRAS) 15. Applications are assigned to a REC and reviewed during a meeting, where concerns are addressed, questions posed, and opinions given. A favourable opinion can be given, but the REC will often ask for amendments or further review 16. HEIs maintain their own RECs to review research by affiliates, when it falls outside the HRA’s scope. The specifics of the process and REC structure vary between institutions 17. Additionally, major corporations, like pharmaceutical firms, often maintain internal governance boards. While RECs traditionally focus on protecting participants and ensuring societal benefits 10, modern healthcare’s evolution and the advent of digital health technology introduced new ethical complexities, such as juggling privacy with the need for public health surveillance, data ownership concerns, consent validity, and disparities from algorithmic biases 18, 19. Balancing stakeholders’ interests at the crossroads of participants, researchers, and society 20, RECs are often vulnerable to being received as disrespectful, bureaucratic or obstructive 21. Past studies suggest that perceived unfairness might tempt researchers towards misconduct to “level the playing field”, leading to negative consequences 22. From the REC’s perspective, researchers might seem to be merely “checking boxes” for regulatory compliance rather than genuinely addressing ethical implications 23. A collaborative, respectful partnership between researchers and RECs benefits all parties involved, minimising unrealised research potential and promoting research quality and integrity 22. Recent times have marked a surge in industry-funded digital health trials 24, 25. Relevant and timely evaluation is crucial to keep pace with technological advancements. This study drew inspiration from unpublished work asking digital health developers their views on evaluation, which pinpointed ethics applications as a significant challenge (Paulina Bondaronek, pers. comm.). Enterprises often cite issues like data ownership, fair access, consent during data collection, and relationships with RECs as obstacles to meeting ethical standards. Over the past decade, research ethics guidelines have adapted to digital innovations, offering more up-to-date tools and guidance 26. Recent global guidelines underscore the values and principles of ethics in digital health research, highlighting the need to stay updated on ever-evolving study designs 27–31. However, a standardised framework tailored to address ethical concerns in digital health has yet to be developed 28. Researchers have critiqued the disconnect between current regulations and the actualities of digital health research, where blurring boundaries, such as between commercial and non-commercial work, cause uncertainties 32, 33. Even REC members report feeling uncertain about digital health research ethics 34. The ethical review process is often seen as confusing due to the absence of standardised procedures. While the majority of healthcare researchers report being confused by ethical ambiguities, only a few regularly consult RECs for guidance 35. Criticisms of RECs are common in research literature 36–38. While there are none in the context of digital health, numerous studies explored the ideal REC role, reinforcing the importance of ethical approval in research 21, 22. Previous studies have often limited their scope to a subset of digital health or a specific ethical challenge 18, 39, leaving a gap in a comprehensive examination of the ethical approval process in digital health to tackle practical challenges. The aim of this investigation was to identify and understand challenges encountered by digital health enterprises in the UK during the research ethics approval process. By analysing formal documentation outlining REC responses to submitted applications, the study determines common barriers. Interviews with enterprise representatives provide an understanding of their direct experiences and challenges faced. ## Methods The study had two parts in parallel: document analysis and interviews with representatives from enterprises. Participants could engage in either or both approaches. The study received ethical approval from the UCL Institute of Health Informatics REC (8-IHILREC). ### Ethics Application Documents Individuals submit an ethics application form via IRAS to RES RECs. These are discussed in the REC’s meetings, which are minuted. This can lead to a favourable opinion (the study can proceed), a favourable opinion with conditions (the study can proceed after certain conditions are met), a provisional opinion (where the applicant is asked to make revisions for subsequent review), or an unfavourable opinion. A response to the applicant with action points is produced. The opinion and action points to be communicated are detailed in the meeting minutes. Applications to RECs can be submitted as being of lower risk for a more streamlined application process, called Proportionate Review. HEI RECs may have similar arrangements. Digital health research ethics applications with a commercial sponsor reviewed by a RES REC in the UK between May 2020 and May 2023 were identified with the following keywords: wearable, informatics, digital health, mobile health, information technology, telehealth, telemedicine, personalised medicine. Drug trials were excluded. The keywords were refined in discussion with the HRA to ensure a practical number of applications could be identified. The search produced 33 applications. At the suggestion of the HRA, a Freedom of Information (FOI) request for completed application forms and ethics committee meeting minutes (summarising discussions between the REC members and applicants), provisional requirements, decisions and recommendations was made to the HRA and extended to the identified applicants. Of the identified applicants, 27 did not decline the FOI request and their anonymised documents were provided. Another five sets of documents, including ethics application forms and responses from RECs, were shared by interview participants (details below). Two of these were submitted to the HRA and three to an HEI REC in the UK. One document concerned a project involving one of the authors (HP). These documents were anonymised by KY. Combined, a total of 32 ethics applications submitted to various RECs across the UK and the committees’ responses were collected and analysed through thematic analysis to identify common challenges in ethical approval. ### Applicant Interviews Commercial enterprises were invited to semi-structured interviews if they met inclusion criteria of being registered and operating in the UK; conducting research in digital health; requiring ethics approval from an external REC instead of through an internal process; and having submitted a research ethics application in the past 5 years. We first approached digital health enterprises with whom a previous collaborative relationship existed. Interview participants were also recruited through advertisements posted by our personal accounts on LinkedIn and Twitter (although no responses came via Twitter). A snowball technique was used whereby identified participants could refer their peers. An invitation email outlining research objectives and procedure was sent to those who expressed interest, followed by the participant information sheet (PIS) and consent form if they were willing to participate. This produced six participants. In the process of obtaining ethics application documents through the FOI request via the HRA, several further participants were identified. These individuals agreed not only to share their application documents but also to participate in interviews. Four applicants corresponding to six sets of documents thus also participated in interviews. Overall, ten interviews were conducted. Before the interviews, consent was obtained from each participant, ensuring they were fully aware of the research objectives, the nature of their participation, the confidentiality of their information, and their right to withdraw at any point without consequences. The interviews lasted 30-60 minutes and took place on MS Teams or Zoom. Transcripts were produced using the platforms’ own functions, then verified against the recordings for accuracy. Identifiers in the transcripts were removed. The semi-structured interview topic guide, formulated based on past research and current study objectives, is in Appendix II. ### Document and Transcript Analysis Thematic analysis 40, 41 was conducted on the REC meeting minutes or decision letters, outlining the REC’s response to the application. Other information, including the study patient population, whether the REC was designated for medical device studies, whether the application was submitted for a low-risk proportionate review, the decision (favourable, favourable with additional conditions, provisional, or unfavourable opinion), and the actions required by the ethics committee, was also noted. The IRAS application forms were not coded, but were reviewed as needed to understand the research projects and committee comments. These forms served as a reference to how and if applicants addressed certain topics and to recognise effective practices that prevented common issues. A second, but complementary thematic analysis was conducted on the interview transcripts. Coding and subsequent analysis were done in NVivo 14 Plus. Initial codes were generated after reviewing the documents or interview transcripts, individually and separately, through a data-grounded inductive approach to produce rich, detailed themes. A passage could be coded to multiple codes. Related codes were clustered into potential themes, after which the themes were iteratively reviewed, edited, defined and named. The themes generated from the documents and the interview transcripts were compared and contrasted to identify overlaps. Cross-coding with a secondary coder was conducted for randomly selected documents to ensure consistency and validity. The primary coder’s codes and themes were provided as reference. The secondary coder had the freedom of including additional codes and organising the codes into subthemes as seen fit. The coders discussed conflicts or mismatches between the coding results, and all disagreements were resolved. ## Results ### Document Analysis Findings #### Application Characteristics About half of the ethics applications (17/32) received a decision with provisional opinions and four (13%) received favourable opinion. None received an unfavourable opinion. Eighty percent (25/32) of the studies involved non-healthy adult participants; five involved children/adolescents; two had healthy participants. The outcome of the discussions in the review meetings were categorised as: satisfactory responses, optional recommendations, or actions points needing amendments. Over all the applications, there was an average of 4.3 actions point (range 0-16). The applications given a provisional opinion had 4.9 action points on average compared to 1.8 for those receiving a favourable opinion. View this table: [Table 1](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T1) Table 1 Summary of key attributes of the ethics review meeting minutes. ### Common Concerns and Barriers We identified five main themes and 16 subthemes (Figure 1), detailed below. Example quotations are highlighted in tables. ![Figure 1](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/01/29/2024.01.28.24301885/F1.medium.gif) [Figure 1](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/F1) Figure 1 Themes and subthemes identified from ethics committee meeting minutes, detailing the concerns and queries raised. ![Figure 2](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/01/29/2024.01.28.24301885/F2.medium.gif) [Figure 2](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/F2) Figure 2 The themes identified from interviewee discussions of barriers and challenges encountered during the ethics application process. ### Theme 1: Ethical Commitments in Care View this table: [Table 2](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T2) Table 2 Summary of discussion topics and outcomes under the theme ‘Ethical Commitments in Care’. #### Inclusion and Equity The REC inquired about the availability of smartphones, tablets, internet access, video conferencing tools, and travel cost reimbursements, requesting provisions where needed and flagging omissions as action points to ensure equity. Withdrawing access to the service or device post-study was also identified as a risk for impeding participation and impacting subsequent care. Compensations that were inequitably low or varied between new and existing users were flagged for amendments. Concerns arose when software was limited to specific operating systems, necessitating justifications and future rollout plans. A commercial health app’s pre-existing license agreement placed undue demands on participants, leading to a modification request. For studies involving physical devices or software capturing physical features, the RECs inquired on the device’s adaptability to different body shapes and physical features. RECs emphasised the importance of inclusive questionnaire options beyond binary gender choices and sought justifications for exclusions based on BMI criteria. While including a broad spectrum of body types, sexes, and ages was highlighted, particularly in smaller sample sizes, these concerns generally did not lead to extensive amendments, as applicants often provided sufficient rationales for their design and participant selection choices. Translations or interpreters were often sought for non-English speakers. Data collection was advised to be inclusive, accommodating various measurement systems and international qualifications. Justification was requested for the inclusion or exclusion of special patient populations like impaired and pregnant patients. RECs encouraged the applicants to be “as inclusive as possible in their recruitment” when needed. Discussions on these complex ethical issues are lengthier than others, but were of a supportive and constructive nature. View this table: [Table 3](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T3) Table 3 Example quotations from meeting minutes on the ‘Inclusion and Equity’ subtheme, organised by codes. #### Participant Safety and Support About one fifth of the issues discussed surrounded public and patient involvement (PPI) in designing study protocols and developing intelligible materials. Most applicants had involved the public, while those who did not received either recommendations or action requirements to do so, depending on the suitability of the submitted information sheets and consent forms. Proactive PPI also alleviated committee concerns surrounding the burden of participation or possible frustration from using the technology. Other amendments to the recruitment protocol were requested to reduce patient burden or lessen the pressure on potential participants, including patients and staff, to ensure voluntary participation. Control groups and those interacting with participants also needed safeguarding. Whenever discussions about delivering questions of a sensitive nature or to sensitive participant populations arose, all applicants addressed them by presenting well-considered arrangements and justifications. Over a third of the applications faced queries regarding technology usage instructions. While some lacked clarity or were not user friendly, others, despite having clear guidelines in the application, had omitted them from participant information sheets. Such oversights consistently led to amendment requests. Depending on complexity, some studies needed face-to-face guidance on app usage and exercises, with helplines suggested for login troubles. High-risk devices also required laboratory tests and safety checks before approval. View this table: [Table 4](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T4) Table 4 Example quotations from meeting minutes on the ‘Participant Safety and Support’ subtheme, organised by codes. #### Conflict of Interest Although only affecting six out of the total 32 applications, nearly all issues surrounding conflicts of interest led to action points and unresolved queries. Applicants failed to include details in info sheets or adverts, including past and current affiliations with the sponsor, irrespective of their relevance to the study, and a clear explanation of the sponsor’s identity. One review requested confirmation that analysts were sponsor employees. The committee also requested that a researcher who is financially affiliated with the sponsor not be involved in consenting patients. When submitted to a HEI REC, there was heightened emphasis on delineating relationships and dynamics between researchers, listed organizations, study locations, and the approving institute. The REC also emphasised incorporation of institutional affiliation on participant-facing materials. View this table: [Table 5](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T5) Table 5 Example quotations from meeting minutes on the ‘Conflict of Interest’ subtheme, organised by codes. ### Theme 2: Study Design View this table: [Table 6](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T6) Table 6 Summary of discussion topics and outcomes under the theme ‘Study Design’. #### Sampling and Recruitment Issues arose with inconsistent and contradicting recruitment methods, such as recruiting from A&E for a study aimed at healthy volunteers, which would then call for a set of medical exclusion criteria due to their acute illnesses. There were gaps in the information in applications about age range, withdrawals, and justifications for excluding certain groups. Applicants had to provide additional consent forms and information sheets for carers, family, and non-team research staff, having initially overlooked their inclusion as participants. RECs also questioned the qualifications of clinicians assessing or recruiting patients, and scrutinized how well exclusion criteria were upheld when participants self-engaged. When researchers excluded patient groups due to their limited relevance to the study or its preliminary nature (not for strict CE marking), the REC found these reasons acceptable. Many applications lacked adequate protocol descriptions for sampling and recruitment, with verbal clarifications during the meetings frequently leading to action points. Common areas needing detail included the initial approach strategy, method of invitation and consent, and meeting formats (in person or virtual). Justifications were often required, especially regarding sample size determinations. Some applicants justified approaching more participants by referencing past trials and recruitment success rates. View this table: [Table 7](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T7) Table 7 Example quotations from meeting minutes on the ‘Sampling and Recruitment’ subtheme, organised by codes. #### Protocol Robustness The number of inquiries concerning the broader adequacy of the protocol was smaller than other subthemes, but often required intricate meeting discussions, demanding greater effort to address. Topics requiring justification for the protocol displayed a diverse range in both topic and nature, overlapping with subthemes mentioned previously. They spanned areas including the necessity for medical testing, potential technological burdens faced by participants, rationale for specific data collection, presumptions regarding positive trial outcomes, the design of interviews, feedback and results distribution, adherence to established care standards, and much more. Some issues even led to entire sections of the study protocol being removed. These multifaceted issues underscored the need for comprehensive reasoning and clarity in protocol development. The outcome of any inquiry—whether action was required or a response deemed satisfactory—was determined by the strength of the supporting rationale and the complexity of the presented issues. View this table: [Table 8](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T8) Table 8 Example quotations from meeting minutes on the ‘Protocol Robustness’ subtheme, organised by codes. #### Harm Prevention Most applications that encountered harm prevention issues (16/17) faced barriers related to unforeseen events such as non-adherence or emergencies. If participants failed to adhere to study guidelines or chose to withdraw, there had to be mechanisms in place to swiftly detect such instances. For high-risk patients, the REC advised maintaining constant contact to ensure early detection of any distress. Clear reaction protocols for unexpected events, like the identification of a risk factor or device failure, were needed. Typically, this would involve notifying the GP or a specialist, and relevant templates, such as a GP letter, and specified method of contact were requested. Participants should be informed of potential incidents and the related actions. RECs were asked about staff and clinician training if it was missing from applications. They stressed the need for proper training, whether through written instructions, sessions with company reps, or on-site experts. Where applicable, proper sterilisation or disinfection of study materials and clear listing of device contraindications to avert potential risks were emphasized. View this table: [Table 9](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T9) Table 9 Example quotations from meeting minutes on the ‘Harm Prevention’ subtheme, organised by codes. ### Theme 3: Digital Health Research Peculiarities View this table: [Table 10](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T10) Table 10 Summary of discussion topics and outcomes under the theme ‘Digital Health Research Peculiarities’. #### Necessity and Value of Research Goals RECs sought clarity on the necessity and value of research goals. These topics emerged because of nuances inherent to digital health interventions. A common theme among the RECs’ inquiries was the alignment between the stated research objectives and the proposed methods of assessment. Some study aims raised questions with unexpected chosen endpoints and a misalignment with traditional healthcare outcome measures. The potential value and broader impact of digital health interventions were also a point of focus. The committees also sought clarity on the practical implications of the research to ensure that research findings were meaningful and translatable. View this table: [Table 11](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T11) Table 11 Example quotations from meeting minutes on the ‘Necessity and Value of Research Goals’ subtheme, organised by codes. #### Remote Encounters There was a low frequency of concerns surrounding remote encounters, and none led to action points. However, these issues merit attention for the unique challenges posed by the shift to digital formats, particularly following COVID-19. The accuracy and consistency of remote assessments had to be confirmed, especially when comparing in-home versus hospital settings. When sensitive questions were posed, participants’ privacy during remote interviews had to be protected. Another point of interest was the potential distraction and supervision challenges during online interviews, particularly when young children were involved. View this table: [Table 12](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T12) Table 12 Example quotations from meeting minutes on the ‘Remote Encounters’ subtheme, organised by codes. #### Technical Issues and User Challenges Other than patient safety and support topics covered above, most action points in this category revolve around ensuring participants have a comprehensive understanding of the digital tools employed. Some questions around the mechanics and functionalities of a device or app were raised. View this table: [Table 13](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T13) Table 13 Example quotations from meeting minutes on the ‘Technical Issues and User Challenges’ subtheme, organised by codes. ### Theme 4: Data Governance View this table: [Table 14](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T14) Table 14 Summary of discussion topics and outcomes under the theme ‘Data Governance’. #### Data Storage and Access About 60% of applications faced queries around data management, with focus on data storage, access and transfer. When action points came up, most were not of dire impact. Amendments caused delays in the approval process, but were not frequently the primary drivers behind provisional opinions. Concerning storage on personal or company devices, applicants were prompted to de-identify data, clarify device ownership, and justify the use of unconventional devices like memory sticks and their safeguard measures. For physical storage, RECs asked about alignment with NHS standards and security measures at data storage sites. If not proposed, RECs requested that data should not be kept indefinitely and any unneeded data should be deleted. If not specified, RECs inquired about the handling of data when a participant withdraws or is unable to participate post-screening. Applicants also grappled with complexities such as removing data that had already been de-identified. View this table: [Table 15](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T15) Table 15 Example quotations from meeting minutes on the ‘Data Storage and Access’ subtheme, organised by codes. #### Data Transfer and Sharing RECs requested more information on who will be accessing data beyond the research team, such as during training and maintenance. Studies involving non-local collaborators often needed to clarify governance procedures and justify data storage on the cloud and transfers to the network or outside the UK. Details on how transfers between NHS servers, universities, and other collaborators would be kept safe were needed. Data transfer to commercial entities or third parties raised questions on access rights, GDPR compliance, international collaboration permissions, third-party data security, and the decision-making process for data storage locations, especially when HEI collaboration was involved. View this table: [Table 16](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T16) Table 16 Example quotations from meeting minutes on the ‘Data Transfer and Sharing’ subtheme, organised by codes. #### Anonymisation and Confidentiality RECs sought explanations for the use of identifiable information, such as a patient’s name on forms, questioning if codes or numbers could be preferable. RECs highlighted the importance of distinguishing between pseudonymous and anonymous. Detailed strategies of anonymisation and how confidentiality would be maintained were requested when missing. UK Data Service and ICO were recommended to applicants. View this table: [Table 17](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T17) Table 17 Example quotations from meeting minutes on the ‘Anonymisation and Confidentiality’ subtheme, organised by codes. #### Video and Audio Recordings Committees questioned the need and duration for video storage, asking for justifications. Often, when the applicant had no intention of recording, they would neglect to mention this in the application. This is shown by the frequent response assuring that no recording will be kept when committees sought clarification on whether interactions, like teleconferences, were recorded. The need to share recordings, especially with sponsors, was a concern. In one case, skepticism existed regarding companies deleting valuable research data. (The applicant, who was interviewed, elaborated on this point, feeling a sense of distrust and expressing frustration towards the REC, as discussed later.) View this table: [Table 18](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T18) Table 18 Example quotations from meeting minutes on the ‘Video and Audio Recordings’ subtheme, organised by codes. ### Theme 5: Document Quality and Completeness View this table: [Table 19](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T19) Table 19 Summary of discussion topics and outcomes under the theme ‘Document Quality and Completeness’. #### Insufficient Content and Detail The most common change needed was the addition or editing of information omitted from participant information sheets or consent forms, mostly covering topics from previous themes. These usually do not result in long discussions, but led to a high number of action points in over 80% of applications. Other than fixing simple omissions, many were a result of discussions covered under previous subthemes, requiring amendments on participant-facing documents. Some information included in the information sheet needed to be added to the consent form, and vice versa. Examples of common easy-to-fix omissions include ethics review information, contact details, and purpose of the study. Where applicable, applicants may be asked to include instructions with images or pictures of the device for visual guidance. View this table: [Table 20](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T20) Table 20 Example quotations from meeting minutes on the ‘Insufficient Content and Detail’ subtheme, organised by codes. #### Missing Required Documents The RECs frequently noted omissions in the documentation provided. On multiple occasions, the committee requested CVs of research team members in a standardised format and evidence of training. Applicants had often missed out documents like invitation letters and emails, adverts, device instruction sheets, specific protocols, GP letters, and forms that would be used. Additional consent forms or sections are also needed for separate participant groups. View this table: [Table 21](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T21) Table 21 Example quotations from meeting minutes on the ‘Missing Required Documents’ subtheme, organised by codes. #### Accuracy and Presentation Many changes were requested on wording and formatting. Changes in wording were usually for the purpose of conveying messages more accurately or increasing reader-friendliness with lay language. Formatting changes were requested to improve document clarity and readability. RECs insisted that authority-provided templates be used precisely as given. However, when using study team-created templates, some sections were found irrelevant, leading to action points for editing. For example, a checkbox for “Yes” or “No” was requested next to all optional items on the consent form. Changes requested during the meetings or oversight occasionally led to inconsistencies across different documents or versions not being up-to-date. Applicants were then required to remove irrelevant content to prevent inaccuracies and redundancies. RECs also pointed out grammar mistakes and typos. View this table: [Table 22](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T22) Table 22 Example quotations from meeting minutes on the ‘Accuracy and Presentation’ subtheme, organised by codes. #### Impact and Severity The issues in the subthemes Conflict of Interest, Anonymisation and Confidentiality, and Video and Audio Recordings were some of the most impactful. Although these subthemes cropped up in a limited number of studies, the vast majority of studies that encountered these barriers were provided with a provisional opinion. The least impactful subthemes were Sampling and Recruitment, Accuracy and Presentation, Insufficient Content and Detail, Protocol Robustness, and Technical Issues and User Challenges. Out of all the studies where these subthemes identified as issues, only about half received a provisional opinion. However, the vast majority of action points that came up fell into the former three subthemes. These issues consistently emerged but carried minimal weight against a favourable opinion. This is not to say that these barriers are not substantial, since action points delay the approval process. For example, requests to provide instructions on the technology are always required to be submitted as amendments. #### Comparison between RECs Perspectives and opinions can vary across different RECs. On one occasion, applicants received a request for an additional consent form and information sheet for young children from an HRA REC that provided an unfavourable opinion. They later sought a second review from a different HRA REC, who agreed that “these documents [could not] be used as it would be difficult to get a meaningful response” from such a young patient population and gave a favourable opinion with additional conditions. HRA RECs held meetings with the applicants, which allowed them to address some issues on the spot, whereas the HEI RECs replied to applications in writing, requiring the applicants to also respond in writing. The HRA REC meetings were documented in neutral language with an observatory tone, whereas the HEI REC documents were more question-oriented and less neutral in tone. However, the difference might stem from the HRA REC meeting minutes being a form of documentation and the HEI REC’s response being a medium that asks similar questions as would be posed in meetings (Table 23). Both RECs sought attribution, with the HRA RECs desiring proper acknowledgment for their ethical review and the HEI REC seeking visible affiliation through the inclusion of the university banner. View this table: [Table 23](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T23) Table 23 Example quotations from meeting minutes of HRA and HEI RECs compared. ### Interview Findings #### Enterprise and Study Characteristics Table 24 summarises the characteristics of projects discussed during the interviews. Some interviewees talked about multiple ethics applications for different digital health interventions. One interviewee had provided assistance with the application process but was unaware of the approval status at the time of the interview, having ceased involvement earlier. One interviewee from a UK-based office of a global enterprise shared their experience with a US institutional review board (IRB). View this table: [Table 24](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T24) Table 24 Summary of key attributes of interviewed enterprises and their projects. #### Common Barriers and Applicant Attitude Interviewees recognised the important role of ethical approval in safeguarding participants during the study and as further developments arise from the research. When asked about the biggest barriers encountered in the application process, we identified six themes from the interviewees’ responses, detailed below. #### Submission and Protocol Revisions Interviewees’ frequent reports of edits in wording, grammar, and clarity mirrored the document analysis. Though interviewees were surprised by the emphasis on minute details, many felt such changes had limited repercussions. View this table: [Table 25](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T25) Table 25 Example quotations from interviews with enterprise representatives on ‘Submission and Protocol Revisions’. Interviewees modified template wording, like consent forms and GDPR text. However, RECs favoured strict adherence to templates. Contrarily, one submission faced revisions for resembling a prior approved application. Some believed the lengthy REC-styled documents and perceived bureaucracy deterred patient involvement and site recruitment. View this table: [Table 26](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T26) Table 26 Example quotations from interviews with enterprise representatives on ‘Submission and Protocol Revisions’. Study design revisions presented challenges, with consent protocols and data governance perplexing to justify and mitigate. Interviewees sought clearer guidance. Table 27, example 6 revisits the scenario from Table 18, example 4, offering the enterprise’s perspective. The interviewee’s stance is elaborated in the following section. View this table: [Table 27](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T27) Table 27 Example quotations from interviews with enterprise representatives on ‘Submission and Protocol Revisions’. #### Dynamic between Parties Relationships between research teams, sponsors, REC, HEI, and external partners, while often supportive and collaborative, can be complex or hindering. Interviewees expressed hesitancy to engage with RECs and feeling a sense of distrust from the RECs. Table 28, example 1 shows the interviewee’s frustration following example 6, Table 27. View this table: [Table 28](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T28) Table 28 Example quotations from interviews with enterprise representatives on ‘Dynamic between Parties’. #### Application Time and Procedures Interviewees reported the ethics application and the steps leading up to it as time-consuming and effort-intensive. Many had not anticipated the length of the process, and those who did often sought to avoid ethics approval if possible. View this table: [Table 29](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T29) Table 29 Example quotations from interviews with enterprise representatives on ‘Application Time’. Interviewees encountered difficulties with the IRAS portal, and factors typically insignificant in traditional research became obstacles in digital health research. The relevancy of application questions was an international issue, and submitting even minor changes proved problematic and stressful. View this table: [Table 30](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T30) Table 30 Example quotations from interviews with enterprise representatives on ‘Application Procedures’. #### Acumen and Practicality in Digital Health Given the novelty of digital health research, RECs sometimes miss its nuances, leading interviewees to clarify technicalities and face unrealistic amendment requests. While acknowledging the importance of inclusivity, interviewees were frustrated by its practical challenges. Similar issues arose regarding data governance in the Submission and Protocol Revisions subtheme. View this table: [Table 31](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T31) Table 31 Example quotations from interviews with enterprise representatives on ‘Acumen and Practicality in Digital Health’. #### Support and Guidance from RECs Interviewees contacted the REC and HRA for support and consulted online resources published by the HRA during the application process. Multiple interviewees reported that the REC/HRA was unsure of the answer to their questions. The timeliness of responses varied across different RECs and experiences. View this table: [Table 32](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T32) Table 32 Example quotations from interviews with enterprise representatives on ‘Support and Guidance from RECs’. #### Enterprise Expertise and Resources Expertise in research ethics varied among enterprises, from novices to seasoned experts. Some had internal regulatory expertise while others relied on academic collaborators. While many faced similar challenges, enterprises with extensive ethics experience often reported a more positive or neutral application experience. There’s a recognised need for targeted resources; while HRA resources were appreciated, gaps were identified. Interviewees also consulted various healthcare and technology guidelines, frameworks from EMEA and the FDA, online resources published by HEIs, and internal enterprise materials. View this table: [Table 33](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T33) Table 33 Example quotations from interviews with enterprise representatives on ‘Enterprise Expertise and Resources’. #### Common Enablers Support, particularly through feedback from the REC and expertise accessed through various channels, was a primary enabler. PPI guidance bolstered protocol justifications and material development. One interviewee mentioned the benefits of connecting with local colleagues informally and advised an open mindset regarding reservations around sharing information: “Don’t be fearful. Just be careful.” Many interviewees emphasised the importance of a detailed protocol for team alignment and submission preparation, with templates and past examples being particularly valuable. #### Suggested Improvements Interviewees voiced desired changes regarding the challenges they faced during the application process, detailed below. View this table: [Table 34](http://medrxiv.org/content/early/2024/01/29/2024.01.28.24301885/T34) Table 34 Desired Changes Voiced by Interviewees Based on Challenges Faced, Organised by Associated Themes. #### International Experience An interviewee shared their experience with a commercial North American institutional review board (IRB), chosen for specific reasons by the enterprise. This IRB offers ethics reviews for institutions without their own boards and provides consulting and training services. The interviewee identified enablers including a user-friendly application portal, responsive customer support, and quick turnaround times. However, the primary challenge was the high application costs, including fees for initial submissions and amendments. ## Discussion We found a complex picture of the challenges in the digital health ethical landscape. The ethical review process demands considerable time and effort, irrespective of experience. For newcomers, it can be particularly daunting, with a steep learning curve. The findings emphasise the importance of providing detailed and high-quality application documents with well-developed study designs. Comprehensive protocols with arrangements for potential concerns are key to mitigating substantial amendment requests. Study designs and objectives should be justified with clear reasoning rooted in evidence, established guidelines, or PPI recommendations. A unified protocol embraced by the entire team is crucial, as is their preparedness for meetings where this protocol is presented. A participant-centric approach should be taken when preparing the application, making sure documents are accurate, up-to-date, avoid jargon, offer tailored support based on the intervention’s complexity and sensitivity, and allow participants adequate time and space for consent and engagement. While some barriers require greater attention, proactive effort can minimise post-submission delays. In research involving vulnerable patient groups, topics like inclusion criteria and protocol rigour invariably require in-depth discussions, often necessitating reference to regulations or care standards. Addressing such inherently multifaceted issues demands additional funding, extended discussions, and thoughtful deliberation. Substantial modifications to the study’s design may be required in the absence of robust justification. Mitigation strategies like PPI can be time-consuming, logistically complex, and yield diverse opinions. Proactively consideration of these complexities allows for setting realistic expectations and preventing surprises. More impactful topics like conflict of interest and data governance—common in research conducted by commercial enterprises—may call for more careful consideration and reflections for their influence on review outcomes. Drawing from interviewee insights, we encourage digital health developers to adopt a more receptive stance towards information dissemination, seek external expertise, and engage with HEIs or other relevant entities for collaborative undertakings, as diverse team expertise facilitates tackling complex issues. Satisfying ethical standards in research is a skill to develop. While ample guidance exists online, much of it is directed towards general research and remains in forms or on databases inaccessible to those outside the academic sphere 43, 44. Typically, industry professionals, like the interviewees, begin their search on RECs’ websites before turning to broader internet searches. Few possess the means or time to pursue literature research or entire books. The interviewees noted the absence of readily available guidance tailored to address specific queries unique to digital health. Researchers have expressed the desire for more explicit guidance and open communication from ethics committees 21. The HRA has developed resources for the public, including some applicable to issues identified in the findings 45. Yet, the challenge arises when supporting a cohort in a domain with diverse study designs and ethical considerations not addressed by traditional guidance. A recurring request among interviewees is the provision of example submissions in digital health showing what good and bad looks like. While there may be reservations about releasing explicit examples, offering guiding principles supplemented with practical solutions can be beneficial. This could manifest as: “If your study encompasses X, consider Y. Prior applicants have navigated this by implementing Z.” Many interviewees emphasised the value of receiving feedback prior to formal submission. While preliminary feedback throughout the submission process would be beneficial, if resources are limited, clearly conveying to applicants that the review’s purpose is constructive rather than decisive to the final outcome could alleviate some anxiety. Much of what has been highlighted so far is generalisable to all research. However, the digital health cohort in the UK faces unique challenges. Unlike some countries, the UK does not have independent IRBs that offer ethical review services for a fee 46, 47. While a few organisations or university departments occasionally make exceptions 48–50, these are not viable long-term solutions for digital health companies. Ethics governance in the industry has been described as the “wild west”. Large corporations may establish internal governance boards, but most digital health enterprises lack such capacity. If a study does not fall under the HRA’s purview, these enterprises face challenges in identifying alternative submission avenues. Some may forge affiliations with a HEI via collaborations, but others, especially those without academic backgrounds or networks, struggle. Calls in published literature advocate for fair access to ethical reviews, urging HEIs to broaden their view of ethics committees beyond just protective measures for their constituents 46, 51. The World Medical Association Declaration of Helsinki establishes the ethical imperative of timely public dissemination of findings 52. By sidelining enterprises from ethics reviews, a void is created, risking under-reviewed studies and potential risks to participants and hindering the publication of results. Though apprehensions exist regarding fee-based ethics review services, like perceptions of unfairness or contractual obligations 53, this research identifies an opportunity for RECs to weigh the merits of introducing paid services. The expressed readiness of enterprises to allocate funds for such services indicates a dual opportunity: enhanced access for digital health developers and a revenue stream to bolster REC services. It is worth noting that countries like the US have successfully adopted this model. While enterprises may navigate their way to ethical reviews via the HRA or a HEI, their external status introduces hurdles in seeking approval. The language and tone of the REC can foster a sense of mistrust, amplifying the divide between researchers and RECs, when a collaborative relationship should be cultivated. There is a particular dilemma in discerning when an intervention benefits the majority versus when to uphold standards of equity and inclusion. Many RECs operate on a voluntary basis, and members bring a variety of experiences and perspectives. Social science research attests to the variation in individuals’ views stemming from their backgrounds, perceptions, and affiliations 54, 55,. Amid calls for more standardised procedures and reflections on REC functioning, this study does not venture into these intricate debates. However, it can spotlight barriers emerging from the distinct nature of technology. The successful deployment and sustained operation of digital health interventions often require commercial backing. Most interventions found effective by research remain inaccessible to the general public 56. Numerous literatures highlighted the tech industry’s role in ensuring scalability and fostering sustainability through public-private partnerships 57, 58. Such collaborations are indispensable not only for advancing healthcare technology but also for ensuring sustainable implementation. A nuanced understanding of the practicalities underlying digital health development might foster a more accommodating stance on issues otherwise deemed untenable. Historically, the tech industry has faced criticism for its data-handling practices. This is particularly challenging in healthcare, where sensitive issues and vulnerable populations necessitate utmost caution. Yet, as the convergence of digital technology and healthcare becomes inevitable, there is a need for an open-minded understanding of technological development requirements. The interviewees’ perspective offers a valid consideration: recognising the company’s motivations and where user protection aligns with its interests can cultivate a cautious, yet unbiased, approach. Data management concerns did not carry heavy weight against favourable opinions, but were frequently mentioned. Recognising that much of digital health research is conducted by non-academic, non-NHS organisations, there is a need for greater adaptability towards data-handling practices unfamiliar to the HRA and HEIs. While reinforcing robust data governance remains central to participant safety, RECs are encouraged to approach without predisposed biases. The rise in research projects with a primary digital focus necessitates updating practices. Practically, this calls for revamping resources, such as templates and forms, to encompass diverse study types, consent methodologies, and data storage methods; generating resources that directly address the subtleties of digital health interventions; and augmenting REC expertise in the digital health realm. Omitting irrelevant sections or offering optional fields in the application form may mitigate applicant frustrations. Further, distinguishing between various digital health interventions or delineating between digital health technology and medical devices, followed by bespoke guidance, could offer significant utility. Several interviewees voiced apprehensions regarding later amendments. Given digital health’s dynamic nature and frequent iterations, it is essential to streamline the amendment process. Initial measures may include enhancing the user-friendliness of IRAS. Beyond this, applicants would benefit from mechanisms that assess and permit pre-specified changes without needing amendments. This entails devising an effective process and providing clear guidance on the boundaries of permissibility. Interviewees generally held an understanding attitude and acknowledge the importance of ethics reviews for ensuring safety, beneficence, and integrity. Those well-acquainted with research often describe the process as straightforward. However, many still reported hesitancy towards engagement, citing challenges and stressors encountered during the process. Beyond avoiding common pitfalls, a deeper understanding of the process could alleviate some concerns. A significant grievance voiced by interviewees is the prolonged wait for responses and the overall approval timeline. While addressing this directly might be challenging, more open communication about expected timelines could enable better planning and realistic expectations. Timeline guidance exists for other trial types 59, but digital health researchers might overlook them due to perceived irrelevance. As such, increasing transparency in the review process, detailing decision-making rationales, application approval rates, and the reasoning behind specific requests, could bolster confidence and counteract negative sentiments. This was an exploratory study. We chose to focus on industry perspectives in the choice of interview participants. We were not able to interview REC participants. We mostly reviewed NRES documents and did not achieve the same degree of HEI REC representation. Further examination into specific digital health subsets, such as artificial intelligence, is recommended to explore their unique intricacies. Future studies might also explore the amendment process. Many of our recommendations would need iterative development to ensure feasibility. ## Conclusion We have highlighted the challenges in obtaining ethics approval in digital health research, some universal and others unique to commercial enterprises. There are opportunities for applicants to prepare for a smoother experience, such as grounding their study designs in established evidence and guidelines, and in PPI recommendations. Concurrently, UK RECs can enhance the process by offering adapted guidance, expanding access, and adopting a tailored, non-biased approach. In light of this, both parties are encouraged to foster a collaborative relationship through open communication, flexibility in perspectives, and a deepened mutual understanding of the ethical landscape in digital health. We plan to develop these findings into accessible guidance. There is an open avenue for implementing and evaluating the proposed changes’ effectiveness. ## Conflict of Interest KY has no conflict of interest to declare. HP is a frequent applicant to research ethics committees. He provides reviews for the UCL Institute of Health Informatics Research Ethics Committee. He supported one application used in the analysis. Since this study was conducted and independent of the study, HP has started providing consultancy for one of the enterprises interviewed in the study. He provides, has in recent years provided or is planning to provide paid expert advice on digital health evaluation for Crystallise Limited, Flo Health Inc., Prova Health, Public Health England, and Thrive Therapeutic Software Ltd. He has PhD students in the field employed by or previously employed by, and with fees paid by AstraZeneca, Patients Know Best and BetterPoints Ltd. ## Funding statement This study did not receive any funding. ## Data availability Interview data and data not obtained by freedom of information requests is not available for secondary use. ## Acknowledgements We thank Paulina Bondaronek (UCL) and Hanjun Gu (UCL) for their assistance in the research process, and Matthew Darlison (UCL) for comments on an earlier draft. We thank the HRA, particularly Zoher Kapacee, Susannah Keeling, Ashley Totenhofer, Gemma Warren, Rebecca Evans, Jonathan Fennelly-Barnwell and Stephen Tebbutt. We also thank all the interviewees for their support and contributions to the data collection. ## Appendix– Semi-Structured Interview Protocol ### INTRODUCTION Recap of the purpose of the study and the interview. * - Our study aims to understand the challenges of the ethics application process for digital health research and projects that had to apply for ethical approval. * - We are looking to analyse the ethics application documents to understand the issue objectively, then interview the applicants about your experience with applying for ethics through the interview. * - The terms research, project, study, evidence generation, will refer to the project for which you applied for ethics. * - Outline of the interview sections. * - Informed consent to record. ### INTERVIEW #### Section 1: Background Information 1. Please can you confirm your role in X company 2. How big is the company? (Number of employees) 3. What was the location of the project you had to get ethics for? 4. When did the project take place? #### Section 2: Study Overview 1. Briefly, tell me about your company and the digital health product or intervention the project was on. *Suggested follow-up questions:* 1. Was/is the intervention/product available on the market? Is it available through the NHS (e.g. commissioned) or available directly to consumers (e.g. Available through app store)? 2. Was the project sponsored? If so, was the sponsor of the study from a commercial, academic, or another non-commercial source? 2. What were the ethical considerations associated with the project? Were there any that were unique or specific to the digital health context? #### Section 3: Experience of the Ethics Application (Including Barriers and Enablers) 1. Please describe your ethics application process, including events that occurred throughout the process. *Clarifying examples:* Filled out application, submitted, received feedback, altered procedure, received decision, etc. *Suggested follow-up questions:* 1. Was the project approved to go ahead in the end? 2. What was your expectation or understanding of the ethics approval process and **how did the actual process differ?** For example, you can also talk about how you felt about the process. 2. What do you think **went well** with the ethics application process? *Clarifying examples:* Please consider how you addressed the ethical issues, things you did during the application process, things the REC did that helped, and resources or help they provided, and anything else that stand out to you. 3. What were the **biggest challenges or barriers** you encountered during the ethics application process? #### Section 4: Knowledge, Resources and Capabilities 1. What **internal expertise** did you have in your company surrounding ethics in digital health 2. What **internal resources** did you have for the ethics application and surrounding ethical considerations in your evaluation? (e.g. financial, human capital) 3. What **external support and resources** did you have or used throughout the process (e.g. links to universities or organisations, government frameworks, funding, etc.) #### Section 5: Conclusion and Recommendations 1. In the context of digital health research/project/evidence generation, how would you like the ethics application process to change? 2. Is there anything that we haven’t talked about today on the ethics approval process in digital health that you would like to discuss? ### CONCLUSION * - Thank the participant for joining. * - Inform the participant that the final report will be circulated via email once produced. * Received January 28, 2024. * Revision received January 28, 2024. * Accepted January 29, 2024. * © 2024, Posted by Cold Spring Harbor Laboratory The copyright holder for this pre-print is the author. All rights reserved. The material may not be redistributed, re-used or adapted without the author's permission. ## References 1. 1.FDA. What is Digital Health? | FDA, [https://www.fda.gov/medical-devices/digital-health-center-excellence/what-digital-health](https://www.fda.gov/medical-devices/digital-health-center-excellence/what-digital-health) (2020, accessed 13 January 2023). 2. 2.Meskó B, Drobni Z, Bényei É, et al. Digital health is a cultural transformation of traditional healthcare. Mhealth 2017; 3: 38–38. 3. 3.Peek N, Sujan M, Scott P. Digital health and care in pandemic times: impact of COVID-19. BMJ Health Care Inform 2020; 27: 100166. 4. 4.Murray E, Hekler EB, Andersson G, et al. Evaluating Digital Health Interventions: Key Questions and Approaches. American Journal of Preventive Medicine 2016; 51: 843–851. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.amepre.2016.06.008&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F01%2F29%2F2024.01.28.24301885.atom) 5. 5.Greaves F, Joshi I, Campbell M, et al. What is an appropriate level of evidence for a digital health intervention? The Lancet 2018; 392: 2665–2667. 6. 6.Karpathakis K, Libow G, Potts HWW, et al. An evaluation service for digital public health interventions: User-centered design approach. J Med Internet Res; 23. Epub ahead of print 1 September 2021. DOI: 10.2196/28356. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2196/28356&link_type=DOI) 7. 7.Unsworth H, Dillon B, Collinson L, et al. The NICE Evidence Standards Framework for digital health and care technologies – Developing and maintaining an innovative evidence framework with global impact. Digit Health; 7. Epub ahead of print 2021. DOI: 10.1177/20552076211018617. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/20552076211018617&link_type=DOI) 8. 8.NHS England. The multi-agency advisory service (MAAS), [https://transform.england.nhs.uk/ai-lab/ai-lab-programmes/regulating-the-ai-ecosystem/the-multi-agency-advice-service-maas/](https://transform.england.nhs.uk/ai-lab/ai-lab-programmes/regulating-the-ai-ecosystem/the-multi-agency-advice-service-maas/) (2022, accessed 18 January 2023). 9. 9.Hedgecoe A. “A Form of Practical Machinery”: The Origins of Research Ethics Committees in the UK, 1967–1972. Med Hist 2009; 53: 331. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1017/S0025727300000211&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19584956&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F01%2F29%2F2024.01.28.24301885.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000270050100001&link_type=ISI) 10. 10.Gelling L. Role of the research ethics committee. Nurse Educ Today 1999; 19: 564–569. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1054/nedt.1999.0349&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=10808899&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F01%2F29%2F2024.01.28.24301885.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000083286300008&link_type=ISI) 11. 11.the British Psychological Society. Code of Ethics and Conduct, [https://www.bps.org.uk/guideline/code-ethics-and-conduct](https://www.bps.org.uk/guideline/code-ethics-and-conduct) (2018, accessed 14 January 2023). 12. 12.British Educational Research Association (BERA). Ethical Guidelines for Educational Research, [https://www.bera.ac.uk/publication/ethical-guidelines-for-educational-research-2018](https://www.bera.ac.uk/publication/ethical-guidelines-for-educational-research-2018) (2018, accessed 14 January 2023). 13. 13.Newson AJ, Lipworth W. Why should ethics approval be required prior to publication of health promotion research? Health Promot J Austr 2015; 26: 170–175. 14. 14.Graf C, Wager E, Bowman A, et al. Best Practice Guidelines on Publication Ethics: a Publisher’s Perspective. Int J Clin Pract Suppl 2007; 61: 1. 15. 15.HRA. Integrated Research Application System, [https://www.hra.nhs.uk/about-us/committees-and-services/integrated-research-application-system/](https://www.hra.nhs.uk/about-us/committees-and-services/integrated-research-application-system/) (2021, accessed 11 September 2023). 16. 16.HRA. Research Ethics Committee – Standard Operating Procedures, [https://www.hra.nhs.uk/about-us/committees-and-services/res-and-recs/research-ethics-committee-standard-operating-procedures/](https://www.hra.nhs.uk/about-us/committees-and-services/res-and-recs/research-ethics-committee-standard-operating-procedures/) (2022, accessed 11 September 2023). 17. 17.McCormick JB, Sharp RR, Ottenberg AL, et al. The Establishment of Research Ethics Consultation Services (RECS): An Emerging Research Resource. Clin Transl Sci 2013; 6: 40. 18. 18.Goodman KW. Ethics in Health Informatics. Yearb Med Inform 2020; 29: 26–31. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1055/s-0040-1701966&link_type=DOI) 19. 19.Petersen C, Subbian V. Special Section on Ethics in Health Informatics. Yearbook of medical informatics 2020; 29: 77–80. 20. 20.Tusino S, Furfaro M. Rethinking the role of Research Ethics Committees in the light of Regulation (EU) No 536/2014 on clinical trials and the COVID-19 pandemic. British Journal of Clinical Pharmacology 2022; 88: 40–46. 21. 21.Brown C, Spiro J, Quinton S. The role of research ethics committees: Friend or foe in educational research? An exploratory study. Br Educ Res J 2020; 46: 747–769. 22. 22.Keith-Spiegel P, Koocher GP, Tabachnick B. What Scientists Want from Their Research Ethics Committee. Journal of Empirical Research on Human Research Ethics 2006; 1: 67– 81. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1525/jer.2006.1.1.67&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19385866&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F01%2F29%2F2024.01.28.24301885.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000203006800009&link_type=ISI) 23. 23.Mcareavey R, Muir J. Research ethics committees: Values and power in higher education. Int J Soc Res Methodol 2011; 14: 391–405. 24. 24.Al-Durra M, Nolan RP, Seto E, et al. Nonpublication Rates and Characteristics of Registered Randomized Clinical Trials in Digital Health: Cross-Sectional Analysis. J Med Internet Res 2018;20(12):e11924 [https://www.jmir.org/2018/12/e11924](https://www.jmir.org/2018/12/e11924) 2018; 20: e11924. 25. 25.Micca P, Cruse CB, Shukla M. Health tech investment trends: How are investors positioning for the future of health? Insights into the quickly emerging health tech sector A report from the Deloitte Center for Health Solutions, [https://blogs.deloitte.com/centerforhealthsolutions/](https://blogs.deloitte.com/centerforhealthsolutions/). (2020). 26. 26.Cox S, Solbakk JH, Bernabe RDLC. Research ethics committees and post-approval activities: a qualitative study on the perspectives of European research ethics committee representatives. Curr Med Res Opin. Epub ahead of print 2022. DOI: 10.1080/03007995.2022.2115773. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/03007995.2022.2115773&link_type=DOI) 27. 27.FDA. Clinical Decision Support Software Guidance for Industry and Food and Drug Administration Staff, [https://www.fda.gov/regulatory-information/search-fda-guidance-documents/clinical-decision-support-software](https://www.fda.gov/regulatory-information/search-fda-guidance-documents/clinical-decision-support-software) (2022, accessed 13 January 2023). 28. 28.Almeida D, Shmarko K, Lomas E. The ethics of facial recognition technologies, surveillance, and accountability in an age of artificial intelligence: a comparative analysis of US, EU, and UK regulatory frameworks. AI and Ethics 2021 2:3 2021; 2: 377–387. 29. 29.Cornock M. General Data Protection Regulation (GDPR) and implications for research. Maturitas 2018; 111: A1–A2. 30. 30.ISO. ISO - ISO 27799:2016 - Health informatics — Information security management in health using ISO/IEC 27002, [https://www.iso.org/standard/62777.html](https://www.iso.org/standard/62777.html) (2016, accessed 13 January 2023). 31. 31.Jandoo T. WHO guidance for digital health: What it means for researchers. doi:10.1177/2055207619898984; 6. Epub ahead of print 8 January 2020. DOI: 10.1177/2055207619898984. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/2055207619898984&link_type=DOI) 32. 32.Marelli L, Lievevrouw E, studies IVH-P, et al. Fit for purpose? The GDPR and the governance of European digital health. Taylor & Francis 2020; 41: 447–467. 33. 33.Lievevrouw E, Hoyweghen I van. The social implications of digital health technology, [https://lirias.kuleuven.be/retrieve/547200](https://lirias.kuleuven.be/retrieve/547200) (2019, accessed 14 January 2023). 34. 34.Samuel G, Derrick G. Defining ethical standards for the application of digital tools to population health research. Bull World Health Organ 2020; 98: 239–244. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2471/blt.19.237370&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32284646&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F01%2F29%2F2024.01.28.24301885.atom) 35. 35.Nelson WA. Necessary competencies for ethics committees. Skilled internal consultants vital to addressing ethics issues. Healthc Exec 2013; 28: 46–8. 36. 36.Peute LW, Lichtner V, Baysari MT, et al. Challenges and Best Practices in Ethical Review of Human and Organizational Factors Studies in Health Technology: a Synthesis of Testimonies. Yearb Med Inform 2020; 29: 58–70. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F01%2F29%2F2024.01.28.24301885.atom) 37. 37.Steinbrook R. Protecting research subjects—the crisis at Johns Hopkins. N Engl J Med 2002; 346: 716–720. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJM200202283460924&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=11870258&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F01%2F29%2F2024.01.28.24301885.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000174062800031&link_type=ISI) 38. 38.Wald DS. Bureaucracy of ethics applications. BMJ 2004; 329: 282–284. [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMjkvNzQ2MC8yODIiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyNC8wMS8yOS8yMDI0LjAxLjI4LjI0MzAxODg1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 39. 39.Fiske A, Henningsen P, Buyx A. Your Robot Therapist Will See You Now: Ethical Implications of Embodied Artificial Intelligence in Psychiatry, Psychology, and Psychotherapy. J Med Internet Res 2019;21(5):e13216 [https://www.jmir.org/2019/5/e13216](https://www.jmir.org/2019/5/e13216) 2019; 21: e13216. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F01%2F29%2F2024.01.28.24301885.atom) 40. 40.Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006; 3: 77– 101. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1191/1478088706qp063oa&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=16426259&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F01%2F29%2F2024.01.28.24301885.atom) 41. 41.Fugard A, Henry PWW. Thematic Analysis. SAGE Research Methods Foundations: An Encyclopedia. Epub ahead of print 2019. DOI: 10.4135/9781526421036858333. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.4135/9781526421036858333&link_type=DOI) 42. 42.US FDA. Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence/Machine Learning (AI/ML)-Enabled Device Software Functions. Draft Guidance for Industry and Food and Drug Administration Staff. 43. 43.Greaney AM, Sheehy A, Heffernan C, et al. Research ethics application: A guide for the novice researcher. British Journal of Nursing 2012; 21: 38–43. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22240519&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F01%2F29%2F2024.01.28.24301885.atom) 44. 44.Remenyi D, Swan Nicola, Assem BVDen. Ethics protocols and research ethics committees : successfully obtaining approval for your academic research. 1st ed. Reading: Academic Publishing International Ltd., 2011. 45. 45.HRA. Consent and Participant information sheet preparation guidance., [https://www.hra-decisiontools.org.uk/consent/index.html](https://www.hra-decisiontools.org.uk/consent/index.html) (accessed 13 September 2023). 46. 46.Brown N. Research ethics in a changing social sciences landscape. doi:10.1177/17470161221141011 2022; 19: 157–165. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/17470161221141011&link_type=DOI) 47. 47.Beckwith M. Errors in Trials. In: Operation Innovation: How to Make Society Richer, Healthier and Happier. London: The Entrepreneurs Network, 2023, pp. 32–34. 48. 48.Independent researchers seeking ethics review. In: BPS Code of Human Research Ethics. British Psychological Society, 2021. Epub ahead of print April 2021. DOI: 10.53841/bpsrep.2021.inf180.12. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.53841/bpsrep.2021.inf180.12&link_type=DOI) 49. 49.The University of Sheffield. Reviewing for external organisations. Research, Partnerships and Innovation. 50. 50.Mitchels B. 7.1 Independent Review Panels (for research by independent practitioners). In: The Ethical Guidelines for Research in the Counselling Professions. Leicestershire: British Association for Counselling and Psychotherapy, 2019, pp. 73–74. 51. 51.Young C. Could Ethics Committees please welcome Independent Researchers? Oral Health and Dentistry 2017; 1: 180–182. 52. 52.World Medical Association. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA 2013; 310: 2191–2194. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2013.281053&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24141714&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F01%2F29%2F2024.01.28.24301885.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000327404400028&link_type=ISI) 53. 53.Pickworth E. Should local research ethics committees monitor research they have approved? J Med Ethics 2000; 26: 330. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToibWVkZXRoaWNzIjtzOjU6InJlc2lkIjtzOjg6IjI2LzUvMzMwIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDEvMjkvMjAyNC4wMS4yOC4yNDMwMTg4NS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 54. 54.Goffman E. Frame analysis: An essay on the organization of experience. Cambridge, MA, US: Harvard University Press, 1974. 55. 55.Tajfel H, Turner JC. The Social Identity Theory of Intergroup Behavior. Polit Psychol 2004; 276–293. 56. 56.Rogers MAM, Lemmen K, Kramer R, et al. Internet-Delivered Health Interventions That Work: Systematic Review of Meta-Analyses and Evaluation of Website Availability. J Med Internet Res; 19. Epub ahead of print 1 March 2017. DOI: 10.2196/JMIR.7111. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2196/JMIR.7111&link_type=DOI) 57. 57.WHO. Global diffusion of eHealth. Report of the third global survey on eHealth 2016; 154p. 58. 58.Free C, Phillips G, Watson L, et al. The Effectiveness of Mobile-Health Technologies to Improve Health Care Service Delivery Processes: A Systematic Review and Meta-Analysis. PLoS Med 2013; 10: e1001363. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pmed.1001363&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=23458994&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F01%2F29%2F2024.01.28.24301885.atom) 59. 59.HRA. Phase 1 clinical trials, [https://www.hra.nhs.uk/planning-and-improving-research/policies-standards-legislation/phase-1-clinical-trials/](https://www.hra.nhs.uk/planning-and-improving-research/policies-standards-legislation/phase-1-clinical-trials/) (2022, accessed 13 September 2023).