Quantified Flu: an individual-centered approach to gaining sickness-related insights from wearable data ======================================================================================================= * Bastian Greshake Tzovaras * Enric Senabre Hidalgo * Karolina Alexiou * Lukaz Baldy * Basille Morane * Ilona Bussod * Melvin Fribourg * Katarzyna Wac * Gary Wolf * Mad Ball ## Abstract **Background** Wearables have been used widely for monitoring health in general and recent research results show that they can be used for predicting infections based on physiological symptoms. So far the evidence has been generated in large, population-based settings. In contrast, the Quantified Self and Personal Science communities are comprised of people interested in learning about themselves individually using their own data, often gathered via wearable devices. **Objective** We explore how a co-creation process involving a heterogeneous community of personal science practitioners can develop a collective self-tracking system to monitor symptoms of infection alongside wearable sensor data. **Methods** We engaged into a co-creation and design process with an existing community of personal science practitioners, jointly developing a working prototype of an online tool to perform symptom tracking. In addition to the iterative creation of the prototype (started on March 16, 2020), we performed a netnographic analysis, investigating the process of how this prototype was created in a decentralized and iterative fashion. **Results** The Quantified Flu prototype allows users to perform daily symptom reporting and is capable of visualizing those symptom reports on a timeline together with the resting heart rate, body temperature and respiratory rate as measured by wearable devices. We observe a high level of engagement, with over half of the 92 users that engaged in the symptom tracking becoming regular users, reporting over three months of data each. Furthermore, our netnographic analysis highlights how the current Quantified Flu prototype is a result of an interactive and continuous co-creation process in which new prototype releases spark further discussions of features and vice versa. **Conclusions** As shown by the high level of user engagement and iterative development, an open co-creation process can be successfully used to develop a tool that is tailored to individual needs, decreasing dropout rates. Keywords * symptom tracking * COVID-19 * wearable devices * self tracking * citizen science * netnographic analysis * co-creation ## Introduction Patient-or participant-led research has been suggested to improve self-management capabilities [1] and provide ways to generate otherwise undone science [2], [3]. A particular subtype of participant-led research is *personal science*, which involves the use of empirical methods by individuals to pursue personal health questions [4]. Personal science is a distinct category of citizen science that has emerged from the Quantified Self community and its efforts to advance participant-led research [5], [6]. In personal science, practitioners almost always take the lead in all stages of the research process by definition [4]. Due to this high level of individual engagement and tailoring to individuals interests, personal science has the potential to deliver novel insights relevant for its practitioners [7] that can lead to an improved sense of agency and quality of life [8]. Furthermore, the insights and self-expertise generated by these types of participant-led processes has potential relevance for professional scientific research, both topically as a source of ideas, and methodologically as a source of tools, analytical approaches and workflows [9]. Wearable devices – from wristbands to smartwatches and other personalized, miniaturized on- and around-body devices – are frequently used by self-trackers. These devices are becoming more and more common and are used for a wide spectrum of well-being, fitness and health related purposes [10]. This is further facilitated by the fact that the number of sensors used in those devices is growing rapidly: In addition to accelerometers and gyroscopes to track physical activity, sensors to measure physiological signals such as heart rate, body temperature, respiratory rate, and blood oxygen saturation, that may correspond to health/sickness state of the human body [11], are now frequently found in wearables as well [12], [13]. Consequently, even outside the realms of *personal science* wearables have long been seen as promising tools to facilitate health-related monitoring and enable personalized medicine [14], [15], and have been proposed or used to monitor conditions as diverse as cardiovascular disease [16], [17], Alzheimer’s [18] and graft-versus-host disease [19]. In response to the COVID-19 pandemic, interest in using wearable technology for infection prediction and surveillance has increased [20]–[22]. Anecdotal reports from self-trackers suggested that wearables may provide evidence of COVID-19 infection [23]. During the first year of the COVID-19 pandemic a small number of studies appeared, highlighting that wearable devices, often along with self-reported symptoms, might indeed be used for the early detection of COVID-19 infections and to assess physiological symptoms [24]–[27]. The majority of these studies take a crowdsourcing-based approach, in which participants are invited to contribute by providing their own wearable data along with regular symptom reports and COVID-19 test results, as the main way of engaging individuals. The goal of the data collection process in these studies is to create big data sets to interrogate. In contrast, there have been only limited efforts that try to engage personal scientists in co-creating such symptom tracking efforts. Personal science practices are largely done in isolation, and the Quantified Self movement shows consequently limited knowledge accumulation so far [8]. To fill this gap we present a case study of *Quantified Flu* (QF), a project co-created by a community of personal science practitioners in response to the COVID-19 pandemic. We explore how the co-creation approach developed and was able to generate a citizen science platform prototype over a relatively short period of time, based on a combination of open source peer production principles [28] and co-creation approaches [29]. We investigate the unique benefits of this open and dynamic process in contrast to similar, traditional top-down research approaches, as well as the limitations of this case as a still ongoing collaborative effort. ## Methods ### Community co-creation process QF started out from a discussion on the monthly *Open Humans* community call at the beginning of the COVID-19 pandemic on March 10, 2020. Open Humans (OH) is a platform for empowering individuals around their personal data, to explore and share for the purposes of education, health, and research [30]. The community calls involved 83 individuals so far (until September 03, 2020), and the monthly calls are frequented by a mix of citizen science and personal science practitioners; usually around 10 individuals take part in each call. Following an initial brainstorming, the discussions and planning stages were continued through following community calls and a dedicated channel of the OH community Slack ([https://slackin.openhumans.org](https://slackin.openhumans.org)). Furthermore, over the evolution of the project other communities like the *Quantified Self* ([https://quantifiedself.com](https://quantifiedself.com)) and the *Open Covid19 Initiative* ([https://app.jogl.io/program/opencovid19](https://app.jogl.io/program/opencovid19)) were engaged and involved in different aspects for the development of the project. In parallel to 10 additional community calls between March 10 and September 03 2020, the main communication tool for the QF project was a specific Slack channel with a total of 146 subscribers and 50 active users over time (33% participation) with different levels of involvement. During this timeframe, this openly accessible channel gathered a total of 844 messages from these users, with a total count of 26,691 words (and 3,917 unique words). While the planning, coordination and social aspects of the co-creation process mainly took place on the project’s Slack channel, the technical collaboration and software development happened through GitHub and the git repository of the QF. Due to the iterative nature of the open source collaboration, no upfront requirement analysis was performed. Instead, the prototyping developed over time according to community discussions by iteratively exploring implementations. On GitHub, 7 contributors created a total of 316 commits since March 12 2020, leading to the technical prototype that is outlined below. The source code for the project is available under an open license at [https://github.com/openhumans/quantified-flu](https://github.com/openhumans/quantified-flu). ### Quantified Flu (QF): Technical platform implementation QF was developed as a responsive web application to facilitate use on a wide variety of devices and was implemented in Python/Django programming language. Users must be OH users, having an option of linking a wide range of available wearable devices from which physiological data (heart rate, body temperature, respiratory rate) will be imported into the OH platform; visualize past sickness/infection events (retrospectively) on the OH platform (present since the first prototype, launched on March 16, 2020) and engage in daily (prospective) symptom tracking (added in the second prototype, released March 24, 2020) (Figure 1). ![Figure 1:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/12/2021.03.10.21252242/F1.medium.gif) [Figure 1:](http://medrxiv.org/content/early/2021/03/12/2021.03.10.21252242/F1) Figure 1: Data- and user-flow in *Quantified Flu* #### User accounts, data storing & anonymization To enable rapid prototyping QF connected to the OH platform [30] as a backend to manage users and store user data. OH provides OAuth2-based APIs to authenticate users while keeping each user pseudonymous to the QF platform, as no personal identifiable information is transmitted. Instead, only a random 8 digit user identifier that is specific to the QF project is provided. Furthermore, OH provides APIs to access and store user data in their system through those identifiers, and gives methods for users to consent to sharing data from the OH platform with third parties such as QF. #### Wearables To further bootstrap the creation of the prototype, QF made use of the existing wearable integrations that OH already offered (Fitbit daily summaries, Fitbit intraday data resolution and Oura Ring). To facilitate usability QF also integrated those data import methods directly into the prototype, using OH as the data store for the wearable data. Furthermore, following community suggestions and ideation discussions, QF also added Google Fit (May 6, 2020), Garmin (June 11, 2020) and Apple Health (May 14, 2020) as additional supported wearable devices. Depending on the wearable, users can import and use their heart rate throughout the day, their resting heart rate, their body temperature and their respiratory rate in QF (see Table 1). View this table: [Table 1:](http://medrxiv.org/content/early/2021/03/12/2021.03.10.21252242/T1) Table 1: Wearables supported by *Quantified Flu* Unlike the other wearables integrated into QF, Apple Watch does not provide a web-based API to access and export data. Thus, a mobile iOS application was created to provide a link to QF. This application enables users to export their heart rate data collected by Apple Watch. The source code for this mobile application is available under an open license at [https://github.com/OpenHumans/qf-heartrate-apple-health](https://github.com/OpenHumans/qf-heartrate-apple-health). #### Symptom tracking Users can report symptoms through the QF website. Based on prior works [24]–[26] and community co-creation and feedback, QF implemented a list of 12 symptoms that are classified in respiratory, gastrointestinal and systemic symptoms (Table 2), allowing users to score those on a 5-point scale (1=light to 5=worst). Additionally, users can report fever measurements and use free-text fields for suspected origin of their symptoms, further symptoms and notes to put their symptoms into context. View this table: [Table 2:](http://medrxiv.org/content/early/2021/03/12/2021.03.10.21252242/T2) Table 2: Symptoms of sickness that users can monitor in *Quantified Flu* Users can opt-in to receive daily symptom report reminders that are sent through the anonymous OH email system at a user-selected time. Each email contains two links: (1) “reporting no symptoms”, single click that requires no further interaction of the user, and (2) “reporting symptoms”, taking users to the symptom report form. #### Data visualization To provide users with easy ways to facilitate understanding of their own physiological data, and potentially explore it in relation to their own symptom reports, QF used *D3*.*js* to create interactive visualizations. These visualizations present the evolution of the various physiological data points and put them into context of their symptom reports where available. ### Netnographic content analysis To investigate and analyze the co-creation process that led to the above prototype we performed a netnographic analysis of the communication process in the QF Slack channel. Netnography is an ethnographic technique broadly applied to the study of online communities [31] which allows to capture and reflect interactions in the community as an observational, inductive and unobtrusive approach. In particular, we examine how individuals engaged in the collaborative development of the QF platform in a case study setting [32]. For this part of data collection, one of the researchers (E.S.) developed a codebook combining key concepts of co-creation and collaboration in communities of practice (table 3). The codebook was cross-checked for validity by two other authors (B.G.T and M.B.). Following this, it was applied to the QF Slack channel *a posteriori*, without interfering in the community discussions. View this table: [Table 3:](http://medrxiv.org/content/early/2021/03/12/2021.03.10.21252242/T3) Table 3: Codebook for QF Slack communication content analysis This part of data analysis was used to determine the typology of messages regarding the co-creation of the QF platform, from idea to concept and to prototype [29], as well as other types of messages relevant from a communicational and empathy-needed dialogic process in communities of practice [33]. Each Slack message was assigned up to three top tags, based on the above codebook categories, depending on its text density and characteristics. The researcher’s (E.S.) assessments of types and categories of messages were afterwards reviewed and discussed by another co-author (B.G.T.), who was actively involved during the analysed co-creation process. ## Results We present the current prototype of the *Quantified Flu* platform (available at [https://quantifiedflu.org](https://quantifiedflu.org)), as result of the described technical development, before analyzing the co-creation process that led to it. ### Quantified Flu: The platform The QF platform gives personal science practitioners two main ways to explore their physiological wearable data in relation to infections: The **retrospective analysis** of prior events as well as an **on-going (prospective) symptom reporting**. #### Retrospective analysis Users can select a given date on which they fell sick and QF will – if available – extract wearable data for the 3 weeks prior to that date, as well as 2 weeks after the incident. Depending on the wearable (see Table 2), users are given the option to display different physiological variables over the 5 week time period and explore how they change over time. To facilitate the interpretation of changes and outliers in the graphs, both the first and second standard deviation are presented as well (see Figure 2 A). While users can add comments to retrospective events, detailed symptom reports are absent in this mode as most users do not have detailed records of the historic sickness events. The retrospective analyses were part of the first prototype of QF, launched on March 16, 2020. ![Figure 2:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/12/2021.03.10.21252242/F2.medium.gif) [Figure 2:](http://medrxiv.org/content/early/2021/03/12/2021.03.10.21252242/F2) Figure 2: **(A)** An example data visualization of an individual sickness incident that was reported on December 31, 2018. Plotted are the resting heart rate recordings as measured by a *Fitbit* and an *Oura Ring*. **(B)** An example of an on-going symptom report visualization: Top half gives a heat map of which symptoms were present in which strength and green boxes display user-provided free-text comments. The bottom half gives physiological data from wearables. #### On-going symptom reporting Users can also report currently experienced symptoms through QF at any moment in time by selecting symptoms and their experienced strengths from a list (see Table 3). This self-report is likely triggered by an email, as explained above (see methods). Following symptom reports users are automatically taken to their data visualization (Figure 2 B). On a wearable device data level this visualization provides the same details as the retrospective analyses do (see above). The on-going symptom reports were launched as a new feature in the second iteration of the prototype on March 24, 2020, following discussion and feedback by the community (c.f. Figure 4 below). ![Figure 4:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/12/2021.03.10.21252242/F3.medium.gif) [Figure 4:](http://medrxiv.org/content/early/2021/03/12/2021.03.10.21252242/F3) Figure 4: Distribution of message types over time with frequency of tags are given as 7-day rolling averages. Events 1-7 around QF development are given as vertical lines. Bar plots show the total number of tags per category. Additionally, this latter view aligns a heatmap of each daily symptom report to the wearable data timeline, allowing the identification of patterns within the reported symptoms themselves and for visual cross-comparisons between the physiological data and the symptom reports. Furthermore, users can access their comments for each symptom report from this visualization too, allowing them to understand the contexts in which they made those reports. ### Community usage A total of 190 personal science practitioners have engaged with QF between its launch on March 16, 2020 and December 22, 2020. The initial prototype of QF (in place until March 24, 2020) only offered the possibility to create analyses for retrospective sickness events. This feature was rarely used by the users: Only 24 users tried the feature, creating a total of 47 retrospective analyses. In total, 34 wearables have been linked by these 24 users. The prospective on-going symptom report feature has been launched on March 24, 2020. In total 92 users made use of this feature at least once, covering a range from a single symptom report being done up to over 300 reports for some members. Overall, 11,658 symptom reports were filed and 112 wearables have been linked to it, in the time between the feature’s launch and December 22, 2020. The distribution of user engagement for the whole period (Figure 3, Panel A) – as measured by the number of reports – shows an approximately linear relationship between the number of reports done and the user’s rank of activity. The reports with symptoms are also not equally distributed across all 92 users, with a sizable fraction of users having no or only a few reports that include symptoms while for some users symptom reports make up half or nearly all of the reports. Overall, the vast majority (91 %) were reports that included no symptoms. Of the 1,064 reports with symptoms, 16 % included explanatory notes or comments in addition to the standardized symptom reports. ![Figure 3:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/12/2021.03.10.21252242/F4.medium.gif) [Figure 3:](http://medrxiv.org/content/early/2021/03/12/2021.03.10.21252242/F4) Figure 3: Usage of QF as measured by ongoing symptom reports filed by users. **(A)** Users ranked by the number of symptom reports they have filed, broken down into whether symptoms were reported (blue) or not (red). **(B)** Number of symptom reports filed per day, values were averaged into a weekly rolling average. Looking at the number of symptom reports filed per day, we can observe a rapid rise in daily reports at the beginning of April 2020, reflecting the launch of the first prototype of the ongoing symptom reports. A second rise in daily reports happens starting in July 2020, leading to the numbers starting to stabilize at around 45 symptom reports filed per day (by on average 45±5 users per day) (Figure 3, Panel B). ### Community-based development A first overview of the four main categories of messages interchanged during the co-creation of the QF prototypes on its dedicated Slack channel (March 10 to September 03 2020) shows a relative balance in the topics of the online messages among the 1,171 message fragments that were annotated, with the *Prototyping* and *Socialization* being slightly more common than *Concept* and *Ideas* ones (see bar charts in Figure 4). On the level of the tags or sub-categories, the most frequent ones are *Support / coordination* (227), *Protocol / tool design* (109), *Technical issues* (107) and *Help developing* (106). We also investigated the more specific tags as defined in the codebook (Table 1) within each category over time (Figure 4, left). We observe that all four main categories as well as the individual tags are present over the whole time frame from early April to September 2020. Particularly messages regarding *Support / Coordination* are present throughout the whole time range. Other recurrent message types during the analysed timespan fall within the categories *Ideas, Concept* and *Prototyping*, highlighting the iterative design and co-creation process that was used to develop and improve the QF prototype over time. Importantly, the *Protocol / tool design, Mention to tool / wearable* and *Feature suggestions* categories – which are indicative of the co-creation process – do appear early on but remain active in bursts throughout the observed timespan, often following new releases of the QF prototype. Additionally, the *Help developing* and *Help testing* categories remain active over the whole duration of the prototype development, with the former showing a more constant activity (1.1±1.9 tags per day) while the latter appears in bursts (0.76±2.1 tags per day) around new feature releases. ## Discussion In this paper we present *Quantified Flu*, a co-created web-based project to enable personal science practitioners to engage with their own wearable data and visualize it in the context of when they are experiencing symptoms of potential infection. The spark that led to this community deciding to co-create a symptom tracking tool was the beginning of the global COVID-19 pandemic along with population-wide studies that made individuals wonder how useful their own wearables data might be for them. The goal of this work was to address two main questions: (1) As personal science practitioners, can we provide ourselves a way to generate self-expertise through building practical self-knowledge; and (2) Can methods of co-creation adapted from existing peer-production systems successfully support the development of collective knowledge in a community that is mostly concerned with individual discovery? By taking this perspective, this work is in contrast to the plethora of population level studies that are typically performed to evaluate the usefulness of wearable technology for the prediction of illness [20], [24]–[27]. The initial QF prototype that was developed focused solely on retrospective symptom tracking. While this feature was rarely used by users of QF (24 users), this initial version facilitated additional discussions about designing both the data collection protocol and extending the prototype (see Figure 4), leading to the creation of the on-going symptom reports as launched in the next QF iteration. This feature received a much higher attention of users, with a total of 92 people using QF for their own symptom tracking, delivering some first insight into the importance and potential benefits of early engaging potential users in a research design co-creation approach. Furthermore, digital or mHealth applications typically struggle with achieving continued use, as a large fraction of users drop out after a few interactions [34], [35]. In previous studies, only 2% of initial users show sustained use in the most extreme cases, with observational studies on average having a 49% dropout rate [36]. In contrast, around half of the QF users that engaged with ongoing symptom tracking do so on a regular basis, leading to 45 symptom reports per day on average (±5), Figure 3B), and over 50 users reporting more than 3 months of symptom reports, highlighting continued longitudinal use. Prior studies have found that users are more likely to continue using mHealth apps if there is a good fit between user and application [37], and a co-creation process with the future users – as applied for QF – can be a key way of achieving this fit. For some users, this continued engagement might furthermore be a sign that they experience regular or recurring symptoms, making them particularly interested in this specific kind of self-tracking. This is supported by looking at the number of reports that include symptoms, where a subset of users reports having symptoms frequently, with some users reporting symptoms in 40%, or extreme cases even 90% of the time (Figure 3A). Further evidence for this comes from the notes or annotations that users can submit along with their symptoms. Looking at the publicly shared notes we find examples like “*the cough is smokers cough. Because I’ve been smoking more since being out of work*” and “*I was deep cleaning the house…all the dust got my allergies going again*”, highlighting reasons for recurring symptoms. Furthermore, these annotations help to provide context to individuals and others that aim to re-use publicly shared data: While a severe case of coughing or nasal congestion might hint at an acute infection, they might also be unrelated as the annotations highlight. These contextual descriptions can be hard to formalize, potentially explaining why symptom-based diagnoses are hard to achieve in many cases [38], [39]. In parallel to the development of the QF prototype itself, we furthermore explored how a community-driven initiative can contribute to collectively creating the tools needed to build self-knowledge, by conducting a netnographic analysis of the main QF communication channel. Looking at the community communication there over time shows a marked overlap of the various phases of ideation, conceptualization and prototyping. While a greater number of interactions can be found in the initial phases, there is a sustained regularity later on, particularly in areas such as feature suggestions or design of the tool and protocol. In this sense, messages and interactions related to help with development throughout the whole process reflect a typology of continuous and iterative co-creation which is typical of collaboration processes in the development of open source tools [40]. This iterative co-creation process is also highlighted in the burst-like appearance of feature suggestions and protocol/tool design discussions, which are frequently appearing following the release of new features, suggesting that new releases spark further protocol refinements and feature ideas, which in turn lead to the QF prototype refinement. Importantly, this means that the protocol itself along with the concrete implementation remains in a stage of flux over a longer period of time compared to more traditional research design approaches. As a result, this type of collaborative approach is at odds with standard ethical oversight procedures for human subject research that require a precise pre-definition of the protocol and the role of the individuals, while a main feature of co-creation is that it is emergent and adaptive, making detailed pre-specifications impossible [41]. To fully take advantage of the benefits of co-creation in participant-led research it might be necessary to develop different models of ethical oversight that recognize the autonomy of participants [42], [43] in order to not discourage or stifle valuable forms of participant-led research [2]. Last but not least, it is also important to highlight how the other types of messages associated with communication in a community of practice context, which favor both online empathy and effective coordination, were produced in a prominent, constant and sustained way from the beginning of the co-creation process (see *Support/Coordination* in Figure 4). This mode of co-creation can be understood as an example of *uninvited citizen science* that relies on a shared set of values, a self-stabilizing communication infrastructure and a loosely defined co-produced knowledge object [44] (e.g. the QF prototype itself). In this way the development of the data collection platform itself is framed in a dynamic, bottom-up and adaptive way, similar to other open source and peer production experiences. ## Conclusions While QF is a project that is still at a prototype stage and with correspondingly small user base, the co-creation processes of the platform prototype described here represent an example of how the co-development of digital research objects, within the relatively new participatory paradigm of extreme citizen science [45], can be implemented following bottom-up, dialogic approaches and a high level of participant engagement. This aligns with the still scarce literature on what has been called do-it-yourself science or peer-to-peer science [46], [47], in which similar participatory approaches can offer an opportunity for early and sustained engagement from personal science practitioners in the collaborative definition of concepts, features and protocols for health-related digital platforms. ## Data Availability All data used in this manuscript comes from public sources and can be accessed under the links provided below. Additionally all software generated as part of this manuscript is freely accessible under open source licenses in the links provided below. [https://github.com/OpenHumans/quantified-flu](https://github.com/OpenHumans/quantified-flu) [https://quantifiedflu.org/public-data/](https://quantifiedflu.org/public-data/) [http://slackin.openhumans.org/](http://slackin.openhumans.org/) ## Authors’ Contributions BGT, KW, GW & MB initiated this study. All authors contributed to the design of the study. BGT, KA, LB, BM and MB performed the software development. BGT, ES, IB and MF performed data analyses and visualizations. BGT, ES and MB prepared the original draft of the manuscript. All authors reviewed and edited the manuscript before submission. ## Conflicts of Interest MB is the Executive Director of the Open Humans Foundation. BGT is the Director of Research for the Open Humans Foundation. ## Acknowledgements The authors thank all participants of the Open Humans community calls and the Quantified Self and OpenCovid19 communities for their input and support. We also thank all users of Quantified Flu for their support and feedback. Thanks to the Bettencourt Schueller Foundation long term partnership, this work was partly supported by the CRI Research Fellowship to BGT. This work was also supported by a microgrant of the OpenCovid19 initiative. KW is supported by the H2020 WellCo project (769765). ## Abbreviations OH : Open Humans QF : Quantified Flu * Received March 10, 2021. * Revision received March 10, 2021. * Accepted March 12, 2021. * © 2021, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution 4.0 International), CC BY 4.0, as described at [http://creativecommons.org/licenses/by/4.0/](http://creativecommons.org/licenses/by/4.0/) ## References 1. [1]. E. Chiauzzi, C. Rodarte, and P. DasMahapatra, “Patient-centered activity monitoring in the self-management of chronic health conditions,” BMC Med., vol. 13, no. 1, p. 77, Apr. 2015, doi: 10.1186/s12916-015-0319-2. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12916-015-0319-2&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25889598&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F12%2F2021.03.10.21252242.atom) 2. [2]. E. Vayena et al., “Research led by participants: a new social contract for a new kind of research,” J. Med. Ethics, p. medethics-2015-102663, 2015, doi: 10.1136/medethics-2015-102663. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToibWVkZXRoaWNzIjtzOjU6InJlc2lkIjtzOjg6IjQyLzQvMjE2IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMDMvMTIvMjAyMS4wMy4xMC4yMTI1MjI0Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 3. [3]. B. L. Allen, Y. Ferrier, and A. K. Cohen, “Through a maze of studies: health questions and ‘undone science’ in a French industrial region,” Environ. Sociol., vol. 3, no. 2, pp. 134–144, Apr. 2017, doi: 10.1080/23251042.2016.1220850. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/23251042.2016.1220850&link_type=DOI) 4. [4]. G. I. Wolf and M. De Groot, “A Conceptual Framework for Personal Science,” Front. Comput. Sci., vol. 2, 2020, doi: 10.3389/fcomp.2020.00021. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3389/fcomp.2020.00021&link_type=DOI) 5. [5].Grant & Wolf, “White Paper: Design and Implementation of Participant-Led Research in the Quantified Self Community,” Quantified Self, 2019. [https://quantifiedself.com/white-paper-design-and-implementation-of-participant-led-research/](https://quantifiedself.com/white-paper-design-and-implementation-of-participant-led-research/) (accessed Sep. 02, 2020). 6. [6]. M. D. Groot, M. Drangsholt, F. J. Martin-Sanchez, and G. Wolf, “Single Subject (N-of-1) Research Design, Data Processing, and Personal Science,” Methods Inf. Med., vol. 56, no. 6, pp. 416–418, Nov. 2017, doi: 10.3414/ME17-03-0001. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3414/ME17-03-0001&link_type=DOI) 7. [7]. E. J. Daza, K. Wac, and M. Oppezzo, “Effects of Sleep Deprivation on Blood Glucose, Food Cravings, and Affect in a Non-Diabetic: An N-of-1 Randomized Pilot Study,” Healthcare, vol. 8, no. 1, Art. no. 1, Mar. 2020, doi: 10.3390/healthcare8010006. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3390/healthcare8010006&link_type=DOI) 8. [8]. D. Lupton, “‘It’s made me a lot more aware’: a new materialist analysis of health self-tracking:,” Media Int. Aust., Apr. 2019, doi: 10.1177/1329878X19844042. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/1329878X19844042&link_type=DOI) 9. [9]. N. B. Heyen, “From self-tracking to self-expertise: The production of self-related knowledge by doing personal science,” Public Underst. Sci., vol. 29, no. 2, pp. 124–138, Feb. 2020, doi: 10.1177/0963662519888757. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0963662519888757&link_type=DOI) 10. [10]. J. E. Mück, B. Ünal, H. Butt, and A. K. Yetisen, “Market and Patent Analyses of Wearables in Medicine,” Trends Biotechnol., vol. 37, no. 6, pp. 563–566, Jun. 2019, doi: 10.1016/j.tibtech.2019.02.001. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.tibtech.2019.02.001&link_type=DOI) 11. [11]. J. Karjalainen and M. Viitasalo, “Fever and Cardiac Rhythm,” Arch. Intern. Med., vol. 146, no. 6, pp. 1169–1171, Jun. 1986, doi: 10.1001/archinte.1986.00360180179026. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/archinte.1986.00360180179026&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=2424378&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F12%2F2021.03.10.21252242.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1986C626200021&link_type=ISI) 12. [12]. A. Kamišalić, I. Fister, M. Turkanović, and S. Karakatič, “Sensors and Functionalities of Non-Invasive Wrist-Wearable Devices: A Review,” Sensors, vol. 18, no. 6, Art. no. 6, Jun. 2018, doi: 10.3390/s18061714. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3390/s18061714&link_type=DOI) 13. [13].1. H. Rivas and 2. K. Wac K. Wac, “From Quantified Self to Quality of Life,” in Digital Health: Scaling Healthcare to the World, H. Rivas and K. Wac, Eds. Cham: Springer International Publishing, 2018, pp. 83–108. 14. [14]. A. Clim, R. D. Zota, and G. Tinica, “Big Data in home healthcare: A new frontier in personalized medicine. Medical emergency services and prediction of hypertension risks,” Int. J. Healthc. Manag., vol. 12, no. 3, pp. 241–249, Jul. 2019, doi: 10.1080/20479700.2018.1548158. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/20479700.2018.1548158&link_type=DOI) 15. [15]. J. Zheng, Y. Shen, Z. Zhang, T. Wu, G. Zhang, and H. Lu, “Emerging wearable medical devices towards personalized healthcare,” in Proceedings of the 8th International Conference on Body Area Networks, Brussels, BEL, Sep. 2013, pp. 427–431, doi: 10.4108/icst.bodynets.2013.253725. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.4108/icst.bodynets.2013.253725&link_type=DOI) 16. [16]. A. Mizuno, S. Changolkar, and M. S. Patel, “Wearable Devices to Monitor and Reduce the Risk of Cardiovascular Disease: Evidence and Opportunities,” Annu. Rev. Med., vol. 72, no. 1, p. null, 2021, doi: 10.1146/annurev-med-050919-031534. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1146/annurev-med-050919-031534&link_type=DOI) 17. [17]. F. Sana, E. M. Isselbacher, J. P. Singh, E. K. Heist, B. Pathik, and A. A. Armoundas, “Wearable Devices for Ambulatory Cardiac Monitoring: JACC State-of-the-Art Review,” J. Am. Coll. Cardiol., vol. 75, no. 13, pp. 1582–1592, Apr. 2020, doi: 10.1016/j.jacc.2020.01.046. [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6MzoiUERGIjtzOjExOiJqb3VybmFsQ29kZSI7czo0OiJhY2NqIjtzOjU6InJlc2lkIjtzOjEwOiI3NS8xMy8xNTgyIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMDMvMTIvMjAyMS4wMy4xMC4yMTI1MjI0Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 18. [18]. L. C. Kourtis, O. B. Regele, J. M. Wright, and G. B. Jones, “Digital biomarkers for Alzheimer’s disease: the mobile/wearable devices opportunity,” Npj Digit. Med., vol. 2, no. 1, Art. no. 1, Feb. 2019, doi: 10.1038/s41746-019-0084-2. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41746-019-0084-2&link_type=DOI) 19. [19]. J. Tyler, S. W. Choi, and M. Tewari, “Real-time, personalized medicine through wearable sensors and dynamic predictive modeling: A new paradigm for clinical medicine,” Curr. Opin. Syst. Biol., vol. 20, pp. 17–25, Apr. 2020, doi: 10.1016/j.coisb.2020.07.001. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.coisb.2020.07.001&link_type=DOI) 20. [20]. B. Sen-Crowe, M. McKenney, and A. Elkbuli, “Utilizing technology as a method of contact tracing and surveillance to minimize the risk of contracting COVID-19 infection,” Am. J. Emerg. Med., vol. , no. , Jul. 2020, doi: 10.1016/j.ajem.2020.07.003. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.ajem.2020.07.003&link_type=DOI) 21. [21]. D. R. Seshadri et al., “Wearable Sensors for COVID-19: A Call to Action to Harness Our Digital Infrastructure for Remote Patient Monitoring and Virtual Assessments,” Front. Digit. Health, vol. 2, 2020, doi: 10.3389/fdgth.2020.00008. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3389/fdgth.2020.00008&link_type=DOI) 22. [22]. H. Jeong, J. A. Rogers, and S. Xu, “Continuous on-body sensing for the COVID-19 pandemic: Gaps and opportunities,” Sci. Adv., vol. 6, no. 36, p. eabd4794, Sep. 2020, doi: 10.1126/sciadv.abd4794. [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6MzoiUERGIjtzOjExOiJqb3VybmFsQ29kZSI7czo4OiJhZHZhbmNlcyI7czo1OiJyZXNpZCI7czoxMzoiNi8zNi9lYWJkNDc5NCI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIxLzAzLzEyLzIwMjEuMDMuMTAuMjEyNTIyNDIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 23. [23]. A. B. C. News, “Researchers investigate whether wearable apps could unveil hidden coronavirus cases,” ABC News. [https://abcnews.go.com/Health/researchers-investigate-wearable-apps-unveil-hidden-coronavirus-cases/story?id=69925541](https://abcnews.go.com/Health/researchers-investigate-wearable-apps-unveil-hidden-coronavirus-cases/story?id=69925541) (accessed Feb. 06, 2021). 24. [24]. G. Quer et al., “Wearable sensor data and self-reported symptoms for COVID-19 detection,” Nat. Med., pp. 1–5, Oct. 2020, doi: 10.1038/s41591-020-1123-x. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41591-020-1123-x&link_type=DOI) 25. [25]. T. Mishra et al., “Early Detection Of COVID-19 Using A Smartwatch,” medRxiv, p. 2020.07.06.20147512, Jul. 2020, doi: 10.1101/2020.07.06.20147512. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoibWVkcnhpdiI7czo1OiJyZXNpZCI7czoyMToiMjAyMC4wNy4wNi4yMDE0NzUxMnYxIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMDMvMTIvMjAyMS4wMy4xMC4yMTI1MjI0Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 26. [26]. A. Natarajan, H.-W. Su, and C. Heneghan, “Assessment of physiological signs associated with COVID-19 measured using wearable devices,” medRxiv, p. 2020.08.14.20175265, Aug. 2020, doi: 10.1101/2020.08.14.20175265. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoibWVkcnhpdiI7czo1OiJyZXNpZCI7czoyMToiMjAyMC4wOC4xNC4yMDE3NTI2NXYxIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMDMvMTIvMjAyMS4wMy4xMC4yMTI1MjI0Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 27. [27]. B. Smarr et al., “Feasibility of continuous fever monitoring using wearable devices,” Jul. 2020, doi: 10.21203/rs.3.rs-43914/v1. 28. [28]. G. Madey, V. Freeh, and R. Tynan, “THE OPEN SOURCE SOFTWARE DEVELOPMENT PHENOMENON: AN ANALYSIS BASED ON SOCIAL NETWORK THEORY,” p. 9. 29. [29]. E. B.-N. Sanders and P. J. Stappers, “Probes, toolkits and prototypes: three approaches to making in codesigning,” CoDesign, vol. 10, no. 1, pp. 5–14, Jan. 2014, doi: 10.1080/15710882.2014.888183. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/15710882.2014.888183&link_type=DOI) 30. [30]. B. Greshake Tzovaras et al., “Open Humans: A platform for participant-centered research and personal data exploration,” GigaScience, vol. 8, no. 6, Jun. 2019, doi: 10.1093/gigascience/giz076. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/gigascience/giz076&link_type=DOI) 31. [31]. R. V. Kozinets, “Netnography,” in The International Encyclopedia of Digital Communication and Society, American Cancer Society, 2015, pp. 1–8. 32. [32]. Z. Zainal, “Case Study As a Research Method,” J. Kemanus., vol. 5, no. 1, Art. no. 1, 2007, Accessed: Jan. 18, 2021. [Online]. Available: [https://jurnalkemanusiaan.utm.my/index.php/kemanusiaan/article/view/165](https://jurnalkemanusiaan.utm.my/index.php/kemanusiaan/article/view/165). 33. [33]. J. Preece, “Etiquette, Empathy and Trust in Communities of Practice: Stepping-Stones to Social Capital,” p. 9. 34. [34]. B. M. Bot et al., “The mPower study, Parkinson disease mobile data collected using ResearchKit,” Sci. Data, vol. 3, no. 1, Art. no. 1, Mar. 2016, doi: 10.1038/sdata.2016.11. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/sdata.2016.11&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26938265&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F12%2F2021.03.10.21252242.atom) 35. [35]. J. Torous, J. Lipschitz, M. Ng, and J. Firth, “Dropout rates in clinical trials of smartphone apps for depressive symptoms: A systematic review and meta-analysis,” J. Affect. Disord., vol. 263, pp. 413–419, Feb. 2020, doi: 10.1016/j.jad.2019.11.167. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jad.2019.11.167&link_type=DOI) 36. [36]. G. Meyerowitz-Katz, S. Ravi, L. Arnolda, X. Feng, G. Maberly, and T. Astell-Burt, “Rates of Attrition and Dropout in App-Based Interventions for Chronic Disease: Systematic Review and Meta-Analysis,” J. Med. Internet Res., vol. 22, no. 9, p. e20283, 2020, doi: 10.2196/20283. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2196/20283&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F12%2F2021.03.10.21252242.atom) 37. [37]. I. Vaghefi and B. Tulu, “The Continued Use of Mobile Health Apps: Insights From a Longitudinal Study,” JMIR MHealth UHealth, vol. 7, no. 8, p. e12983, Aug. 2019, doi: 10.2196/12983. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2196/12983&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=31469081&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F12%2F2021.03.10.21252242.atom) 38. [38]. S. A. Call, M. A. Vollenweider, C. A. Hornung, D. L. Simel, and W. P. McKinney, “Does This Patient Have Influenza?,” JAMA, vol. 293, no. 8, pp. 987–997, Feb. 2005, doi: 10.1001/jama.293.8.987. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.293.8.987&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=15728170&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F12%2F2021.03.10.21252242.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000227146400021&link_type=ISI) 39. [39]. A. F. Dugas et al., “Clinical diagnosis of influenza in the ED,” Am. J. Emerg. Med., vol. 33, no. 6, pp. 770–775, Jun. 2015, doi: 10.1016/j.ajem.2015.03.008. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.ajem.2015.03.008&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25827595&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F12%2F2021.03.10.21252242.atom) 40. [40]. W. Sack, F. Détienne, N. Ducheneaut, J.-M. Burkhardt, D. Mahendran, and F. Barcellini, “A Methodological Framework for Socio-Cognitive Analyses of Collaborative Design of Open Source Software,” Comput. Support. Coop. Work CSCW, vol. 15, no. 2, pp. 229–250, Jun. 2006, doi: 10.1007/s10606-006-9020-5. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s10606-006-9020-5&link_type=DOI) 41. [41]. F. Goodyear-Smith, C. Jackson, and T. Greenhalgh, “Co-design and implementation research: challenges and solutions for ethics committees,” BMC Med. Ethics, vol. 16, no. 1, p. 78, Nov. 2015, doi: 10.1186/s12910-015-0072-2. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12910-015-0072-2&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26573410&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F12%2F2021.03.10.21252242.atom) 42. [42]. A. Wiggins and J. Wilbanks, “The Rise of Citizen Science in Health and Biomedical Research,” Am. J. Bioeth., vol. 19, no. 8, pp. 3–14, Aug. 2019, doi: 10.1080/15265161.2019.1619859. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/15265161.2019.1619859&link_type=DOI) 43. [43]. A. D. Grant, G. I. Wolf, and C. Nebeker, “Approaches to governance of participant-led research: a qualitative case study,” BMJ Open, vol. 9, no. 4, p. e025633, Apr. 2019, doi: 10.1136/bmjopen-2018-025633. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMToiOS80L2UwMjU2MzMiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMS8wMy8xMi8yMDIxLjAzLjEwLjIxMjUyMjQyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 44. [44]. D. Mahr and S. Dickel, “Citizen science beyond invited participation: nineteenth century amateur naturalists, epistemic autonomy, and big data approaches avant la lettre,” Hist. Philos. Life Sci., vol. 41, no. 4, p. 41, Dec. 2019, doi: 10.1007/s40656-019-0280-z. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s40656-019-0280-z&link_type=DOI) 45. [45]. M. Haklay, “Citizen Science and Volunteered Geographic Information: Overview and Typology of Participation,” in Crowdsourcing Geographic Knowledge, Springer Netherlands, 2012, pp. 105–122. 46. [46]. F. Ferretti, “Mapping do-it-yourself science,” Life Sci. Soc. Policy, vol. 15, no. 1, p. 1, Jan. 2019, doi: 10.1186/s40504-018-0090-1. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s40504-018-0090-1&link_type=DOI) 47. [47]. A. Delfanti, “Users and peers. From citizen science to P2P science,” J. Sci. Commun., vol. 09, no. 01, Mar. 2010, doi: 10.22323/2.09010501. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.22323/2.09010501&link_type=DOI)