Apple Watch and Withings Evaluation of Symptoms, Treatment, and Rhythm in those Undergoing Cardioversion (AWE STRUCk): A Pragmatic Randomized Controlled Trial

Introduction

The pace of innovation for digital health technologies is accelerating. Many individuals now have access to personal digital devices that track activity, sleep, and weight. Some devices also monitor physiologic measures such as oxygenation, heart rate and rhythm, and blood pressure.^1-3 In response, the U.S. Food and Drug Administration (FDA) is working to establish a least burdensome regulatory framework for such technologies under its Digital Health Innovation Action Plan.^4,5 For software-based medical devices, the agency has initiated a Software Precertification (Pre-Cert) Pilot; a key element is collecting and interpreting post-market, real-world data about software performance.⁶ In September 2018, FDA cleared 2 software features for the Apple Watch (Apple, Cupertino, CA) through the De Novo medical device regulatory pathway: 1) to detect irregular heart rhythms likely to be atrial fibrillation⁷ and 2) to generate a single-lead ECG.⁸ However, several risks were identified, including: misinterpretation and/or over-reliance on the device, false negative results, and false positive results. Strategies to mitigate these risks were also identified.^7,8 Additionally, multiple other digital devices have come to market that can perform ECGs and offer irregular rhythm notifications. Since that time, Apple has released an oxygen saturation monitor, which was not cleared by the FDA.

Catalyzed by the coronavirus disease 2019 (COVID-19) pandemic, these digital health monitoring technologies have been rapidly integrated into clinical practice to support patient monitoring in lieu of in-person visits, thereby reducing possible patient and clinician exposure.⁹ Insurers also sometimes provide coverage for these devices. ¹⁰ These devices have inverted the traditional pyramid of scientific discovery.¹¹ Medical products (i.e., traditional new medical devices and drugs) are generally discovered and tested through scientific research that leads to regulatory approval and is disseminated to clinicians, who in turn inform patients of tests and treatments authorized for commercial use. While traditional medical devices and drugs can be marketed directly to patients, clinician prescriptions or clinical care are necessary for patient use.

In contrast, personal digital devices follow a different path. Many (or all) health and wellness-focused features are not reviewed or regulated by the FDA, while others are cleared if they are similar to previously cleared or approved medical devices. They are directly marketed to the public by manufacturers; patients consume marketing information and may purchase and use the devices without clinician oversight – or any direct interaction with the medical community. Patients may approach clinicians about using personal digital devices, but clinicians may not have scientific research available to guide them.¹¹ This issue occurs with the Apple Watch, which is widely advertised and now exceeds 30 million units in annual sales. However, there are limited data on the impact of commonly used health features such as heart rate measurement, irregular rhythm notification and electrocardiogram (ECG) features on patient quality of life¹² and healthcare utilization. Further, there have been limited evaluations of the accuracy of more novel health features such as the single lead ECG and pulse oximeter features.

Therefore, there is a need to assess the clinical impact of these devices through active surveillance to guide future labeling and risk mitigation strategies. While the Apple Heart Study reported positive predictive values for the irregular pulse notification² and a subsequent study of cardiac surgery patients provided data on accuracy of atrial fibrillation detection,¹³ it is unknown how such notifications affect patients. There has been concern expressed about smartwatches leading to patient anxiety.¹⁴ In fact, anxiety disorders associated with these devices have been documented.¹⁵ Further, there are limited data about the impact on healthcare utilization of Apple Watch heart rate, irregular rhythm notification, and ECG features. One single-health system study found higher overall healthcare utilization among patients with atrial fibrillation using personal digital devices, including the Apple Watch, as documented in clinical notes compared to a propensity-matched sample of patients without documentation of these devices; the authors concluded by calling for prospective, longitudinal, randomized data on health outcomes and healthcare use.¹⁶ Some clinicians have expressed worry about cascades of testing to confirm or rule out disease.¹⁴ One single-center study found that only 11.4% of patients had a clinically actionable diagnosis of interest after an irregular rhythm notification, although several tests (including Holter monitors, echocardiograms, and even computed tomography scans of the chest) were performed.¹⁷ Additionally, a small study has raised the possibility that Apple Watch heart rate measurements among patients with atrial fibrillation may sometimes be inaccurate;¹⁸ this is a known limitation of photoplethysmography-based heart rate monitors, given pulse deficit can occur in atrial fibrillation.¹ Therefore, further validation is needed.¹⁸

Finally, there is a need to evaluate the accuracy of the newer ECG and pulse oximeter features. The ECG feature has been compared to 12-lead ECGs in 2 studies, although one focused only on the QT interval in sinus rhythm¹⁹ and another included only healthy volunteers.²⁰ There are no published data about the accuracy of the oxygen saturation monitor.

Accordingly, we designed the Apple Watch and Withings Evaluation of Symptoms, Treatment, and Rhythm in those Undergoing Cardioversion (AWE STRUCk) trial, a pragmatic randomized controlled trial (RCT), to address these issues. This trial will provide an independent assessment of the impact of the Apple Watch heart rate, irregular rhythm notification and ECG features on 1) quality of life, 2) healthcare utilization, and 3) accuracy of measurements among patients undergoing cardioversion for atrial fibrillation at three geographically distinct academic medical centers in the U.S. By comparing the Apple Watch to a personal digital device without the ability to perform an ECG, offer irregular rhythm notifications, or measure pulse oximetry, we week to better understand the impact of these health features of the Apple Watch. In addition to informing physicians, patients, and policymakers, these data can inform future labeling and risk mitigation strategies that could impact millions of individuals.

Overall Study Design

AWE STRUCk is a prospective, open-label multi-center pragmatic RCT designed to evaluate the impact of patient use of the Apple Watch, focused on its accompanying health features of heart rate measurement, irregular rhythm notification, and electrocardiogram features on patient-reported outcomes and healthcare utilization at six months. Patients with a history of atrial fibrillation who are scheduled for cardioversion are randomized 1:1 to receive the Apple Watch Series 6 or the Withings Move (Figure 1). Patients and the public were not involved in study design.

• Download figure
• Open in new tab

Figure 1:

Study Flow Diagram

Data Acquisition

AWE STRUCk leverages a unique patient-centered health data sharing platform, Hugo, which is intended to overcome many of the limitations of traditional clinical trials.²¹ Hugo aggregates electronic health data for each patient from multiple sources, including electronic health records from hospitals and physicians’ offices, along with data from personal digital devices, by leveraging Blue Button technology and Application Programming Interfaces (Figure 2). Blue Button technology refers to functionality that enables individuals to download their health records.²² Hugo also supports direct patient surveys through text messaging or emails, enabling assessment of patient-reported outcomes and other information without requiring patients to meet directly with study coordinators after initial enrollment. All participants will be enrolled in this platform, through which they will receive near real-time access to their electronic health data from these multiple sources and then share these data with researchers.²³ No data, therefore, are directly obtained from health systems.

• Download figure
• Open in new tab

Figure 2:

Data Aggregated Using the Hugo Platform

Discussion

The AWE STRUCk pragmatic RCT will address important knowledge gaps about the impact of novel, widely used personal digital devices on quality of life and healthcare utilization. Despite the widespread and increasing use of these devices for both consumer and clinical applications, few studies have assessed their impact on these important outcomes. Some of the features are also marketed as health and wellness, without a requirement for FDA clearance. To our knowledge, no prospective studies of these technologies focused on these endpoints have been pursued independent of the device manufacturers; additionally, manufacturers have not conducted many studies in these areas and may lack incentives to do so.

In addition to the data generated, there are several strengths to this study. First, the use of a controlled design helps to isolate the impact of the cardiopulmonary features (heart rate, ECG, irregular rhythm notification, and pulse oximeter) from activity tracking. Second, this study employs a novel pragmatic digital paradigm of research in which patients partner with researchers to share their data; data do not come directly from traditional sources, such as health systems. This research paradigm is supported by the principle of individual agency over data.²¹ Third, recent policy changes by Medicare provide reimbursement for remote patient monitoring using personal digital devices and this study will provide information about the implications of these changes.^27,28

This RCT should also be considered in the context of multiple possible limitations. First, the study is powered to identify a difference of 8.8 in the AFEQT global score, which is greater than the minimal clinically important difference of 5,²⁵ due to the high costs of the devices provided to patients and the budgetary constraints of the trial. Second, the PROMs have been digitally adapted. The AFEQT questionnaire has been validated in paper format.²⁴ However, the vast majority of PROMs that are adapted to a digital format result in synchronous responses²⁹ and this approach is more accurate than clinician documentation.³⁰ Third, there is the possibility of incomplete ascertainment of events if patients use a different health system for care and do not connect these with Hugo. However, the Hugo platform allows patients to add data from multiple health systems (and other sources) at any time throughout the study, thereby facilitating complete clinical event ascertainment. Fourth, the irregular rhythm notification is not intended for use in individuals previously diagnosed with atrial fibrillation. However, the ECG feature is not recommended for users with arrhythmias except for atrial fibrillation and sinus rhythm (since the software can only distinguish these two heart rhythms)⁸. People with atrial fibrillation may be more likely to purchase this device than others to monitor their rhythm and correlate with symptoms. Indeed, in the Apple Heart Study, nearly 20% of first study visit participants reported a diagnosis of atrial fibrillation/flutter before enrollment, even though a history of atrial fibrillation was an exclusion criterion.² Few digital health studies have enrolled patients with clinical morbidities to examine outcomes; most studies have instead focused on healthy volunteers.³¹ Our study will address this limitation. Additionally, a study examining the impact of the Apple Watch on patient-reported outcomes and healthcare utilization requires an enriched population; otherwise, study duration would be many years with a population size multiple times larger than our trial, resulting in significant cost and delays in generating much-needed evidence.

Contributors

SD, ND, SV, AD, AB, GG, JF, JH, JP, JA, PN, JR designed the study. SD and JR wrote the first draft of the manuscript. GG is the overall study coordinator and TH, KA, and LE are individual site study coordinators. All authors revised the manuscript and approved the final version of the submitted manuscript. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. SD acts as the guarantor.

Funding statement

This project was supported by a research grant from the Medical Device Innovation Consortium (MDIC) as part of the National Evaluation System for health Technology (NEST), an initiative funded by the U.S. Food and Drug Administration (FDA). Its contents are solely the responsibility of the authors and do not necessarily represent the official views nor the endorsements of the Department of Health and Human Services or the FDA. While MDIC provided feedback on project conception and design, the organization played no role in collection, management, analysis and interpretation of the data, nor preparation, review and approval of the manuscript. The research team, not the funder, made the decision to submit the manuscript for publication.

Funding for this publication was made possible, in part, by the U.S. Food and Drug Administration through grant 1U01FD006292-01. Views expressed in written materials or publications do not necessarily reflect the official policies of the Department of Health and Human Services; nor does any mention of trade names, commercial practices or organization imply endorsement by the United States Government.

Competing interests statement

Dr. Dhruva reports research funding from the National Heart, Lung, and Blood Institute (NHLBI, K12HL138046) of the National Institutes of Health (NIH), from the Medical Device Innovation Consortium (MDIC) as part of the National Evaluation System for health Technology Coordinating Center (NESTcc), Food and Drug Administration (FDA), Greenwall Foundation, and Arnold Ventures. Dr. Shah has received research support through Mayo Clinic from the FDA to establish the Yale–Mayo Clinic Center for Excellence in Regulatory Science and Innovation program (U01FD005938), from the Centers of Medicare and Medicaid Innovation under the Transforming Clinical Practice Initiative, from the Agency for Healthcare Research and Quality (AHRQ, U19HS024075, R01HS025164, R01HS025402, R03HS025517), from the NHLBI of the NIH (R56HL130496, R01HL131535), from the National Science Foundation, and from the Patient Centered Outcomes Research Institute to develop a Clinical Data Research Network. Dr. Vemulapalli reports receiving funding support from the American College of Cardiology, Society of Thoracic Surgeons, and the NIH (R01 and Small Business Innovation Research BIR), FDA (through NESTcc), Abbott Inc, and Boston Scientific. He reports serving as a consultant or on the Advisory Board of Janssen, HeartFlow, the American College of Physicians, and Boston Scientific. Dr. Beatty reported former employment at Apple Inc. and holds stock in Apple Inc. Dr. Freeman reported salary support from the American College of Cardiology National Cardiovascular Data Registry and the NHLBI and consulting/advisory board fees from Boston Scientific, Medtronic, and Biosense Webster. Dr. Piccini reports receiving grants for clinical research from Abbott, American Heart Association, Association for the Advancement of Medical Instrumentation, Bayer, Boston Scientific, and Philips and serves as a consultant to Abbott, Abbvie, Ablacon, Altathera, ARCA Biopharma, Biotronik, Boston Scientific, Bristol Myers Squibb, LivaNova, Medtronic, Milestone, ElectroPhysiology Frontiers, Itamar, Pfizer, Sanofi, Philips, ResMed, and Up-to-Date. Dr. Noseworthy receives research funding from NIH, including the NHLBI and the National Institute on Aging [NIA]), AHRQ, FDA, and the American Heart Association (AHA). He is a study investigator in an ablation trial sponsored by Medtronic. Dr. Noseworthy and Mayo Clinic are involved in potential equity/royalty relationship with AliveCor. Dr. Ross received research support through Yale University from Johnson and Johnson to develop methods of clinical trial data sharing, from the FDA to establish Yale-Mayo Clinic Center for Excellence in Regulatory Science and Innovation (CERSI) program (U01FD005938), MDIC as part of the NESTcc, AHRQ(R01HS022882), NHLBI of the NIH (R01HS025164, R01HL144644), and from the Laura and John Arnold Foundation to establish the Good Pharma Scorecard at Bioethics International and to establish the Collaboration for Research Integrity and Transparency (CRIT) at Yale.