Consensus recommendations for measuring the impact of contraception on the menstrual cycle in contraceptive clinical trials
===========================================================================================================================

* Amelia C. L. Mackenzie
* Stephanie Chung
* Emily Hoppes
* Nora Miller
* Anne E. Burke
* Sharon L. Achilles
* C. Leigh Allen
* Luis Bahamondes
* Diana L. Blithe
* Vivian Brache
* Rebecca L. Callahan
* Alice F. Cartwright
* Kathryn B. H. Clancy
* Enrico Colli
* Amanda Cordova-Gomez
* Elizabeth E. Costenbader
* Mitchell D. Creinin
* Hilary O. D. Critchley
* Gustavo F. Doncel
* Laneta J. Dorflinger
* Alison Edelman
* Thomas Faustmann
* Christoph Gerlinger
* Lisa B. Haddad
* Julie Hennegan
* Cássia Raquel Teatin Juliato
* Simon P. S. Kibira
* Diana Mansour
* Andres Martinez
* Kristen A. Matteson
* Jacqueline A. Maybin
* Alexandria K. Mickler
* Kavita Nanda
* Chukwuemeka E. Nwachukwu
* Funmilola M. OlaOlorun
* Kevin J. Peine
* Chelsea B. Polis
* Carolina Sales Vieira
* Regine Sitruk-Ware
* Jennifer A. Smit
* Marsden Solomon
* Lisa M. Soule
* Douglas Taylor
* Elizabeth E. Tolley
* Olivia Vandeputte

## Abstract

**Objective** We sought to develop consensus recommendations for improved measurement of contraceptive-induced menstrual changes (CIMCs) in contraceptive clinical trials to build upon previous standardization efforts.

**Study Design** We undertook a virtual consensus-building process using a modified Delphi approach, incorporating aspects of Nominal Group Technique and the Jandhyala method. The process consisted of four half-day meetings, developing recommendations within five topical working groups, a series of consensus questionnaires to assess recommendation agreement on a 4-point Likert-like scale, and opportunities for input and feedback throughout the process. Recommendations required at least 75% consensus to be included.

**Results** Our interdisciplinary group—from 26 organizations and 13 countries in five global regions across academia, nonprofit research organizations, the pharmaceutical industry, and funding agencies— developed 44 consensus recommendations. These included recommendations on standardization, patient-reported outcome measure development aligned with current regulatory guidance, trial design, protocol development, participant recruitment, data collection, data analysis, and areas for exploratory research. Seven recommendations were deemed a priority by over 85% of our group.

**Conclusion** Using a formal consensus-building process, we reached high levels of agreement around recommendations for more standardized and patient-centered measurement of CIMCs in contraceptive clinical trials, now and in the future.

**Implications statement** Consensus recommendations on measuring bleeding changes and related outcomes in contraceptive clinical trials can improve reporting of standardized, patient-centered outcomes in future product labeling. These improvements can enable providers to offer more relevant counseling on contraceptives and permit greater comparability and data synthesis across trials to inform clinical guidance.

Keywords
*   bleeding patterns
*   contraceptive-induced menstrual changes
*   contraceptive clinical trials
*   menstrual cycle
*   patient reported outcome measures
*   consensus

## 1. INTRODUCTION

The use of contraception can cause changes in uterine bleeding patterns, uterocervical fluid, and uterine cramping, and it can impact how users experience menstrual and gynecologic disorders and symptoms. Collectively, these changes can be referred to as contraceptive-induced menstrual changes (CIMCs), although there are differing perspectives on this terminology (Figure 1). Experiencing CIMCs can both negatively and positively impact use dynamics, health, and wellbeing for contraceptive users, as well as the individual and sociocultural acceptability of contraception [1]. When users find CIMCs undesirable, negative consequences can include contraceptive dissatisfaction, reduced quality of life, increased burden around managing menstrual health, reduced sexual wellness, and potential unintended pregnancy if users discontinue contraception while still wanting to prevent pregnancy [1–4]. On the other hand, experiences of CIMCs users consider desirable can result in benefits, such as management of for menstrual and gynecologic disorders and symptoms, method satisfaction, improved sexual wellness, and reduced burden or costs of menstrual materials [1–6]. Which CIMCs users determine to be desirable or undesirable for them can vary widely and is influenced by individual preferences and norms and community-level norms—especially around menstruation, menstrual health, and sexual and reproductive health—as well as perspectives of partners and wider social and contextual factors [1–3,7,8]. However, broadly, users often prefer reduced bleeding and cramping, and dislike increased bleeding and unpredictability [1,2].

![Figure 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/04/05/2024.04.04.24305350/F1.medium.gif)

[Figure 1.](http://medrxiv.org/content/early/2024/04/05/2024.04.04.24305350/F1)

Figure 1. Terminology and definition of contraceptive-induced menstrual changes (CIMCs)
*Citation: Hoppes E, Nwachukwu C, Hennegan J, Blithe DL, Cordova-Gomez A, Critchley H, Doncel GF, Dorflinger LJ, Haddad LB, Mackenzie ACL, Maybin JA, Moley K, Nanda K, Vieira CS, Vwalika B, Kibira SPS, Mickler A, OlaOlorun FM, Polis CB, Sommer M, Williams KM, Lathrop E, Mahajan T, Rademacher KH, Solomon M, Wilson K, Wilson LC, Rountree L. Global research and learning agenda for building evidence on contraceptive-induced menstrual changes for research, product development, policies, and programs. Gates Open Res. 2022 Apr 19:6:49. doi: 10.12688/gatesopenres.13609.1.

Considerations for CIMCs have played a notable role in contemporary contraceptive research and development (R&D). Beginning in the 1950s with the preclinical and clinical research for the first oral contraceptive, a number of development decisions were made, at least in part, due to CIMCs. For example, mestranol and a regimen of hormone-free days were both added because researchers decided monthly withdrawal bleeding was the most acceptable to contraceptive users, as well as possibly other contemporary influential stakeholders [9–11]. These decisions, made decades ago, still impact the current contraceptive method mix and contraceptive R&D today. Throughout this contemporary development of new contraceptives, there have been several endeavors to standardize how CIMCs are measured and reported, which we summarize in Figure 2. Creinin and colleagues have recently reviewed this history and the measurement outcomes from that work in detail [12]; therefore, we focus on the contributors and processes of these previous efforts to highlight differences from the approach reported in this paper.

![Figure 2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/04/05/2024.04.04.24305350/F2.medium.gif)

[Figure 2.](http://medrxiv.org/content/early/2024/04/05/2024.04.04.24305350/F2)

Figure 2. Overview of efforts to standardize the measurement and analysis of CIMC data in contraceptive clinical trials
** Special Programme of Research, Development and Research Training in Human Reproduction* [23]

Current practices for measuring CIMCs in trials continue to have limitations, including burdensome data collection, a lack of uniform definitions provided to trial participants to understand key terms, and the use of patient-reported outcome (PRO) data without evidence of validation in line with current regulatory guidance on patient-focused drug development [24–27]. These limitations are particularly relevant for standard measurement of CIMCs in trials globally because—for topics like menstruation, with high levels of stigma—there are generally low levels of shared understanding and common references within and between communities and contexts [8,28,29]. In addition, how CIMC outcomes reported by trials for providers to use in contraceptive counseling of new methods are often not meaningful to contraceptive users to facilitate their informed method decision-making. Building upon previous standardization efforts—and in collaboration with many of the experts who contributed to this earlier work—we developed consensus recommendations to further improve how CIMCs are measured in contraceptive clinical trials now and in the future. Our consensus-building approach to address current measurement limitations focused on: (a) incorporating a variety of interdisciplinary and global perspectives, (b) considering current regulatory guidance around PROs, and (c) including all types of CIMCs, not just changes in bleeding patterns, to align with the Global CIMC Task Force’s comprehensive definition.

## 2. METHODS

Members of a scientific and technical team at FHI 360 engaged a group of interdisciplinary experts about participating in a consensus-building process to improve how CIMCs are measured in contraceptive clinical trials. Invitees had a range of expertise in topics (i.e., contraceptive clinical trials, family planning, the menstrual cycle, menstrual health, measurement and analysis approaches, and regulatory considerations) and disciplines (i.e., clinical research, social-behavioral science, biomedical research, basic science, and clinical practice). Experts were recruited within multiple sectors from members of the existing CIMC Global Task Force [1], contributors to previous standardization efforts, contributors to the literature in relevant fields, and suggestions from other experts. Among these experts, we convened a 13-member external planning committee from 12 organizations and 6 countries who volunteered to contribute to defining objectives for the expert consultation and refining the proposed consensus-building process.

Our objectives were: (a) to review and discuss limitations and strengths of current approaches for measuring and analyzing data on CIMCs in contraceptive clinical trials; (b) to come to a consensus on recommendations for improvements in CIMC measurement and analysis in contraceptive clinical trials that meet the needs of trial participants, researchers, sponsors, regulators, and future users; and (c) to identify important, related topics outside the scope of this expert consultation that warranted similar consideration. The second objective also included determining what research and evidence are needed to empirically evaluate the recommended improvements. Our consensus-building methodology was an amalgamation of the RAND/UCLA method, the Delphi method, the Modified Rand-Delphi method, the Nominal Group Technique, and the Jandhyala method because no single approach met all our needs [30–35]. We present extensive details on the methodology in the Supplementary Material, but briefly, the fully virtual consensus-building process, depicted in Figure 3, included four half-day meetings for all experts to join, recommendation development within five smaller topical working groups into which experts self-selected, and a series of consensus questionnaires around the scope and details of the recommendations.

![Figure 3.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/04/05/2024.04.04.24305350/F3.medium.gif)

[Figure 3.](http://medrxiv.org/content/early/2024/04/05/2024.04.04.24305350/F3)

Figure 3. Consensus-building methodology for developing recommendations on CIMC measurement in contraceptive clinical trials
**To establish a shared understanding on key information within our interdisciplinary group, including on contraceptive clinical trials, patient-reported outcome measures, sociocultural contexts and CIMCs, and inclusive language around gender and sexuality*

Each consensus questionnaire asked experts about their agreement/disagreement with recommendations on a four-point Likert-like scale (i.e., strongly agree, somewhat agree, somewhat disagree, strongly disagree) and included at least one open-ended question for additional input. One author (SC) completed qualitative analysis of open-ended questions. During the first two days of meetings, we decided topics would need 85% agreement (i.e., ‘strongly agree’ or ‘somewhat agree’) among experts to be considered within scope of this work, and specific recommendations within those topics would need 75% agreement among members of working groups to be included in our final list. The Office of International Research Ethics at FHI 360 determined the expert consultation protocol was research that was exempt from Institutional Review Board review.

## 3. RESULTS

### 3.1. Expert characteristics

A total of 57 experts were involved in the expert consultation from 30 organizations—across academia, nonprofit research organizations, funding agencies, and the pharmaceutical industry—and 14 countries in five global regions (i.e., Northern America, Europe, Africa, Latin America & the Caribbean, and Oceania, listed in order of frequency). Of those experts, 44 self-selected into working groups. Working group members represented over 625 combined years of experience—including nearly 70% with more than 15 years—in contraceptive clinical trials and other R&D, menstrual cycle and menstrual health research, social-behavioral research, contraceptive prescribing, and/or other relevant fields (Table 1). Most had doctoral training in either obstetrics and gynecology and/or research and worked either in academia or at nonprofit research organizations. Most experts self-identified as women, and about half were under the age of 50 and half were 50 or more years old. Almost all experts (88%) reported having either experienced CIMCs themselves or observed someone personally close to them experiencing CIMCs.

View this table:
[Table 1:](http://medrxiv.org/content/early/2024/04/05/2024.04.04.24305350/T1)

Table 1: Characteristics of experts in topical working groups developing recommendations on CIMC measurement in contraceptive clinical trials

### 3.2. Defining the scope for recommendation development

All 11 topics proposed in Questionnaire 1 reached consensus for inclusion (Table 2). Experts suggested another five topics that did not reach consensus during voting during the Day 2 meeting, but a number of elements of those topics were incorporated into other consensus topics. Topics determined to be out of scope are presented in Supplementary Table S1, aligning with the third objective of the expert consultation to identify important, related topics that warrant similar consideration outside of the present process. We converged the 11 topics into 3 cross-cutting themes and 5 topical working groups (Table 2). Working group scopes were: (1) **Eligibility**, enrollment requirements, and confounders; (2) Type, frequency, and format/mode of CIMC **data** collected; (3) Measurement of **acceptability**, impact on daily life, culture, and context; (4) Research agenda for developing CIMC **instruments**; (5) **Analysis** methodology, measures, and analyses standardization.

View this table:
[Table 2.](http://medrxiv.org/content/early/2024/04/05/2024.04.04.24305350/T2)

Table 2. Scope of topics for recommendations on CIMC measurement in contraceptive clinical trials, as determined via Questionnaire 1 and Day 2 meeting, and the corresponding topical working groups

### 3.3. Final Consensus Recommendations

We organized the final 44 consensus recommendations into six sections on: (a) *Standardization* in recommendations 1.1 to 1.4; (b) *Instruments for measuring CIMCs* in 2.1 to 2.3; (c) *Trial design, protocol development, and participant recruitment* in 3.1 to 3.8; (d) *Data collection* in 4.1 to 4.13; (e) *Data analysis* in 5.1 to 5.14; and (f) *Areas for exploratory research* in 6.1 and 6.2 (Table 3). We reached consensus on all recommendations, with nearly half (n=21) achieving greater than 90% consensus and six reaching 100% consensus (see Supplementary Table S2 for full consensus results). One aspect of one recommendation did not reach the consensus threshold of 75% by the end of our process (i.e., part of 5.3 on variables for analysis, details in Table S2), so it was not included. During the final stage of the consensus-building process (Figure 3), we identified priority recommendations based on the urgency and potential impact of their implementation (see Supplementary Table S2 for additional details on consensus and prioritization outcomes). This included seven recommendations that over 85% of experts deemed to be a priority and an additional 12 that at least 75% deemed a priority (highlighted by “*” or “†”, respectively in Table 3).

View this table:
[Table 3.](http://medrxiv.org/content/early/2024/04/05/2024.04.04.24305350/T3)

Table 3. Consensusi recommendations on CIMC measurement in contraceptive clinical trials

Our final recommendations include research approaches to establish the evidence for future improvement in CIMC measurement in trials (e.g., 2.1, 2.2 and 2.3) and recommendations investigators can implement in the meantime, while that research is being conducted (i.e., recommendations beginning with “presently…”, e.g., 4.1 through 4.13 and 5.2 through 5.12). As our aim was to build upon, and not replicate, previous work standardizing CIMC measurement in clinical trials, recommendations often incorporate concepts from previous criteria, especially the work of Creinin and colleagues [12].

Recommendations 1.3 and 5.2 draw heavily from these most recent criteria. Several recommendations also include and refer to regulatory guidance and best practices around developing measures for PROs like CIMCs.

Table 3. Consensus recommendations on CIMC measurement in contraceptive clinical trials See Table 3 at end of document.

## 4. DISCUSSION

Building upon previous approaches at standardization around CIMCs, our consensus recommendations were developed by virtually convening interdisciplinary experts from around the world to consider how to improve the current and future measurement and analysis of CIMC data in contraceptive clinical trials. The resulting 44 recommendations, including identified priorities, can serve as guidance to those implementing and funding clinical trials. Based on the thoughtful input throughout the expert consultation, there are a few important considerations we wish to highlight for those implementing our recommendations. We also note reflections in the Supplementary Material on our consensus-building methodology, which we propose as a model—for the field of contraception, wider sexual and reproductive health, and beyond—of how to successfully bring a large, interdisciplinary, and diverse group of people together from around the world to come to consensus on a topic that had eluded such previous agreement.

### 4.1 Reflections on developing recommendations

Although we ultimately reached consensus on almost every aspects of our recommendations, a few areas of contention emerged during the process. In some instances, differences occurred simply due to the interdisciplinary and global nature of our expert group and the distinct perspectives that inherently arise when convening those with different trainings, experiences, and areas of expertise. Often these differences emerged between those approaching the work from a biomedical perspective verses those working within social-behavioral research. All these differences and the ensuing conversations, however, were essential to thoroughly addressing a topic as multifaceted as measuring CIMCs in the context of clinical trials.

Some areas of contention proved especially challenging and often could be distilled down to differing perspectives along a continuum. On one end of the spectrum was a preference to recommend modest improvements to current approaches that would be feasible and straightforward to implement, aligning with past and present norms and existing systems; on the other end was a desire to push for larger, creative improvements aimed at innovating beyond current confines. In addition, we also encountered an innate tension in attempting to weigh the often-divergent perspectives of various stakeholders within the contraceptive clinical trial ecosystem, including trial investigators and other researchers, trial sponsors and funders, regulators, trial participants, and the future providers and users of the new contraceptives being developed in trials. Because we were developing recommendations for global use, balancing these different perspectives was even more challenging considering the wide range of contexts in which clinical trials are conducted, including within different healthcare systems, countries, and resource settings around the world. We describe a few examples of weighing these differing perspectives next.

Recommendations needed to be specific and technical enough to provide actionable guidance to trial investigators without being too burdensome to enact within the many constraints and competing demands of conducting a clinical trial. From this researcher perspective, there can be an appeal of smaller, incremental improvements to current practices that could be simple to implement with often-limited resources. This gradual approach could also be appealing from a sponsor and funder standpoint because it may not require increased trial budgets and may be more likely aligned with current and familiar regulatory strategies. For some experts, these types of modest changes within current norms and systems were of interest for very practical, realistic reasons founded in their many years of experience. Indeed, complex and/or numerous novel recommendations that may demand additional time and funding could result in delaying upcoming trials, as well as fewer, longer trials in the future. In addition, many experts were concerned our recommendations would be interpreted by funders or regulators as expert-endorsed mandates that should be required for every trial. Altogether, the broader impact of these types of additional burdens and constraints could be fewer new methods being developed, which could ultimately mean the contraceptive needs of some people and couples may not be met. To address the concerns our recommendations could hinder contraceptive R&D, we specify they should not be considered required for contraceptive trials to be funded, conducted, or reviewed by regulatory authorities. Rather, stakeholders should consider these as recommended approaches to achieving standardization and comparability across trials and to guiding future research for improvement, all with emphasis on identified priorities.

On the other hand, this convening of around 50 interdisciplinary experts from multiple global regions in a consensus-building process was a unique opportunity to push the contraceptive clinical trial ecosystem to consider innovative CIMC data collection and analysis approaches that could be more informative to providers in counseling on new contraceptives and more relevant to future users. Many experts endorsed this perspective, which is also aligned with current patient-focused drug development initiatives, other efforts at increasing patient engagement in clinical trials and regulatory decisions, and wider patient-centered outcome research [24–27,36,37]. As described in our recommendations, research will be needed to align CIMC measurement with best practices for PROs. Because this approach is novel for contraceptive trials—although increasingly common in other therapeutic areas—it will require close adherence to regulatory guidelines and engagement with regulators.

Although areas of contention persisted, our approach sought to balance the perspectives of all experts in the consensus-building process and stakeholders in the contraceptive clinical trial ecosystem in our final recommendations by focusing on the goal of improving the standardization and value of trial data on CIMCs. For example, this is the first time recommendations have included uterine cramping and pain, an important outcome for people who menstruate. By considering all CIMCs, not just changes in bleeding patterns, and including experts in menstrual health, heavy menstrual bleeding, patient-reported outcomes, social-behavioral researchers, and others, we were able to extend the boundaries of this work. One related limitation to our process we must acknowledge is the lack of direct input from clinical trial participants. Although we aimed to include their perspectives, as understood among our expert group, this does not replace explicit contributions from participants and their communities. To ensure the utility and relevance of the recommendations around future research a balanced and inclusive approach will need to continue, including using participatory-engaged research methodologies.

### 4.2 Implementing recommendations

As a starting point for implementing these recommendations, we have highlighted seven priorities: 1.1, 2.2, 4.1, 4.11, 5.2, 5.3, and 5.11 (listed separately in Supplementary Table S3). Recommendations 1.1, 2.2, and 5.2 also reached 100% consensus. Recommendation 1.1 calls for standardization and simplification of terminology. In support of this, 2.2 outlines best practices for making both terminology and measurement instruments simple, accessible, and patient-centered. Recommendation 5.2, also aligning with 1.1, lays out important terminology and definitions for use across trials, defining a bleeding episode, as well as duration, frequency, and volume of bleeding episodes. Recommendations 1.1 also calls for standardization in the type of CIMC data collected; therefore, 4.1 and 4.11 provide more detail on data collection, with 4.1 covering the primary outcomes of CIMCs that should be collected, including bleeding and spotting days and frequency, duration, and volume of bleeding episodes, and 4.11 outlining the acceptability and quality of life data to be collected. Finally, 5.3 and 5.11 outline some key variables and missing data that we recommend be reported for all contraceptive clinical trials.

Even implementing only these seven priority recommendations will come with challenges. We recognize the list of all recommendations in Table 3 is not particularly accessible or user-friendly to all. An important next step is to develop tools for stakeholders across the contraceptive R&D ecosystem, such as checklists and guidelines to make adopting these recommendations more straight-forward without imposing any unnecessary burden on the development of new contraceptives. It will also be valuable to tailor these resources to the types of research being conducted (e.g., Phase 1 vs. Phase 3, or clinical vs. social-behavioral research) and to target audiences (e.g., researchers, funders, providers, regulators).

### 4.3 Future implications

We call on stakeholders in the contraceptive clinical trial ecosystem to view these recommendations as new best practices, developed and endorsed by a wide range of experts in contraceptive R&D and related disciplines. We encourage those involved in the design of clinical trials to begin to incorporate these recommendations into their work as soon as possible. To achieve the full benefits of these recommendations, additional dissemination will be required. This publication is only the beginning of efforts needed to make these recommendations actual best practices amongst the contraceptive R&D community.

We also note contraceptives are not the only drug, device, or biologic to impact the menstrual cycle, yet such data are not routinely collected during clinical trials or during standard toxicology and pharmacodynamics studies, as other organ functions and vital signs are. The lack of these data was recently highlighted with the initial introduction of COVID vaccinations when vaccinated people who could or did menstruate experienced unanticipated bleeding changes [38–42]. We hope our work can be a guide and an invitation to develop recommendations for if, when, and how to measures changes to the menstrual cycle in more types of clinical trials and related research.

Broadly, our consensus recommendations can enable more standardized and patient-centered measurement and analysis of CIMC data in the near-term and continued improvement in the future, which can result in more accurate information in contraceptive product labeling on data that matter more to users. Ultimately, such efforts can allow for greater comparability across trials and better data synthesis in systematic reviews and meta-analyses that can inform clinical guidance. In turn, these improvements can enable providers to offer better counseling and users to make more informed decisions about their choice of method, which can improve the experiences of contraceptive users and better meet the contraceptive needs of people and couples.

### Supplementary material

1.  Supplementary Material file

2.  Supplementary tables

## Supporting information

Supplementary File [[supplements/305350_file02.pdf]](pending:yes)

Supplementary Tables S1-S3 [[supplements/305350_file03.pdf]](pending:yes)

## Data Availability

All data produced in the present work are contained in the manuscript.

## Acknowledgments

This expert consultation was made possible by the generous support of the American people through the U.S. Agency for International Development (USAID), provided to FHI 360 through cooperative agreement 7200AA20CA00016. The contents are the responsibility of FHI 360, the views expressed are those of the authors, and contents and views do not necessarily reflect those of any funder, including USAID, the National Institutes of Health, the United States Government, or the Reckitt Global Hygiene Institute Fellowship. ACLM, SC, EH, NM, AEB, RLC, AFC, ECC, LJD, AM, KN, MS, DT, and EET were supported but USAID cooperative agreement 7200AA20CA00016, with SC also supported by the Bill & Melinda Gates Foundation (OPP1200867) and the Eunice Kennedy Shriver National Institute of Child Health and Human Development (T32 HD52468) and MS also supported by the Bill & Melinda Gates Foundation (INV-045483). EC was supported by Exeltis, and CEN by USAID. LB, VB, KBHC, MDC, HODC, GFD, AE, LBH, JH, CRTJ, SPSK, DM, KAM, JAM, FMO, CBP, CSV, RSW, JAS, LMS, and OV received an honorarium supported by USAID cooperative agreement 7200AA20CA00016 that was declined by other authors. In addition, GFD was also supported by USAID Cooperative Agreement 7200AA20CA00019 provided to CONRAD/Eastern Virginia Medical School, JH was also supported by a Reckitt Global Hygiene Institute Fellowship, JAM was also supported by Wellcome Fellowship 209589/Z/17/Z, FMO was also supported by EVIHDAF, and RSW was also supported by The Population Council.

The FHI 360 team would like to recognize the additional contributions of the external planning committee members, whose feedback greatly improved the consensus-building process. These members are Amanda Cordova-Gomez, Anita Nelson, Bellington Vwalika, Carolina Sales Vieira, Chelsea Polis, Diana Mansour, Funmi OlaOlorun, Kathryn Clancy, Kristen Matteson, Lisa Haddad, Mitchell Creinin, Olivia Vandeputte, and Tanya Dargan Mahajan. We also appreciate the *ad hoc* advisory roles of Lisa Rarick on regulatory considerations, Marni Sommer on menstrual health, and Greg Kopf on preclinical and early pharmaceutical trials. We thank Leigh Wynne for input on virtual meeting engagement approaches and coordinating Working Group 5. We also appreciate feedback from Aurélie Brunie and Holly Burke on the manuscript. Natt Pladsri and Audrey Fratus provided important support in preparing for the virtual meetings and ensuring they ran smoothly to aid in successful discussion.

## Footnotes

*   * Authors in alphabetical order from here onward

*   **Acknowledgement of funding** This expert consultation was made possible by the generous support of the American people through the U.S. Agency for International Development (USAID), provided to FHI 360 through cooperative agreement 7200AA20CA00016. The contents are the responsibility of FHI 360, the views expressed are those of the authors, and contents and views do not necessarily reflect those of any funder, including USAID, the National Institutes of Health, the United States Government, or the Reckitt Global Hygiene Institute Fellowship. ACLM, SC, EH, NM, AEB, RLC, AFC, ECC, LJD, AM, KN, MS, DT, and EET were supported but USAID cooperative agreement 7200AA20CA00016, with SC also supported by the Bill & Melinda Gates Foundation (OPP1200867) and the Eunice Kennedy Shriver National Institute of Child Health and Human Development (T32 HD52468) and MS also supported by the Bill & Melinda Gates Foundation (INV-045483). EC was supported by Exeltis, and CEN by USAID. LB, VB, KBHC, MDC, HODC, GFD, AE, LBH, JH, CRTJ, SPSK, DM, KAM, JAM, FMO, CBP, CSV, RSW, JAS, LMS, and OV received an honorarium supported by USAID cooperative agreement 7200AA20CA00016 that was declined by other authors. In addition, GFD was also supported by USAID Cooperative Agreement 7200AA20CA00019 provided to CONRAD/Eastern Virginia Medical School, JH was also supported by a Reckitt Global Hygiene Institute Fellowship, JAM was also supported by Wellcome Fellowship 209589/Z/17/Z, FMO was also supported by EVIHDAF, and RSW was also supported by The Population Council.

*   **Conflicts of interest** AEB has received research funding mediated through JHU from: Merck, Chemo Pharmaceuticals, Dare Bioscience, and NICHD. KBHC received a small honorarium from Pfizer to participate in their Tales From the Uterus podcast series in the summer/fall of 2023. EC is an Exeltis employee. MDC has received speaking honoraria from Gedeon Richter, Mayne, OLIC, and Organon, served on Advisory Boards for Gedeon Richter and Mayne, has stock options with Femasys, and has consulted for Estetra SRL, Medicines360, and Organon. The Department of Obstetrics and Gynecology, University of California, Davis, receives contraceptive research funding for Dr. Creinin from Chemo Research SL, Evofem, Femasys, Medicines360, Merck, Sebela, and Sumitomo Pharma. HODC has received clinical research support for laboratory consumables and staff from Bayer AG (Paid to Institution) and has provided consultancy advice (Paid to Institution) for Bayer AG, PregLem SA, Gedeon Richter, Vifor Pharma UK Ltd, AbbVie Inc., Myovant Sciences GmbH, and Theramex. HODC has received royalties from UpToDate for an article on abnormal uterine bleeding. AE has received travel reimbursement from ACOG, WHO, CDC, and honoraria from Gynuity for committee activities. AE receives royalties from Up to Date, Inc. Oregon Health & Science University receives research funding from OHSU Foundation, Gates, Merck, HRA Pharma, and NIH where AE is the principal investigator. CG is an employee of Bayer AG. All other authors have no conflicts of interest to declare.

*   Received April 4, 2024.
*   Revision received April 4, 2024.
*   Accepted April 5, 2024.


*   © 2024, Posted by Cold Spring Harbor Laboratory

This pre-print is available under a Creative Commons License (Attribution-NoDerivs 4.0 International), CC BY-ND 4.0, as described at [http://creativecommons.org/licenses/by-nd/4.0/](http://creativecommons.org/licenses/by-nd/4.0/)

## References

1.  [1].Hoppes E, Nwachukwu C, Hennegan J, Blithe DL, Cordova-Gomez A, Critchley H, et al. Global research and learning agenda for building evidence on contraceptive-induced menstrual changes for research, product development, policies, and programs. Gates Open Res 2022;6:49. doi:10.12688/gatesopenres.13609.1.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.12688/gatesopenres.13609.1&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=35614964&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 

2.  [2].Polis CB, Hussain R, Berry A. There might be blood: A scoping review on women’s responses to contraceptive-induced menstrual bleeding changes. Reprod Health 2018;15:1–17. doi:10.1186/s12978-018-0561-0.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12978-018-0460-4&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 

3.  [3].Higgins JA, Smith NK. The Sexual Acceptability of Contraception: Reviewing the Literature and Building a New Concept. J Sex Res 2016;53:417–56. doi:10.1080/00224499.2015.1134425.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/00224499.2015.1134425&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26954608&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 

4.  [4].Higgins JA, Davis AR. Contraceptive sex acceptability: a commentary, synopsis and agenda for future research. Contraception 2014;90:4–10. doi:10.1016/j.contraception.2014.02.029.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.contraception.2014.02.029&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24792147&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 

5.  [5].Brunie A, Stankevitz K, Nwala AA, Nqumayo M, Chen M, Danna K, et al. Expanding long-acting contraceptive options: a prospective cohort study of the hormonal intrauterine device, copper intrauterine device, and implants in Nigeria and Zambia. Lancet Glob Health 2021;9:e1431–41. doi:10.1016/S2214-109X(21)00318-1.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S2214-109X(21)00318-1&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=34474001&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 

6.  [6].Rademacher KH, Sripipatana T, Danna K, Sitrin D, Brunie A, Williams KM, et al. What Have We Learned? Implementation of a Shared Learning Agenda and Access Strategy for the Hormonal Intrauterine Device. Glob Health Sci Pract 2022;10. doi:10.9745/GHSP-D-21-00789.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiZ2hzcCI7czo1OiJyZXNpZCI7czoxMzoiMTAvNS9lMjEwMDc4OSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDI0LzA0LzA1LzIwMjQuMDQuMDQuMjQzMDUzNTAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

7.  [7].Sekhon M, Cartwright M, Francis JJ. Acceptability of healthcare interventions: an overview of reviews and development of a theoretical framework. BMC Health Serv Res 2017;17:88. doi:10.1186/s12913-017-2031-8.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12913-017-2031-8&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 

8.  [8].Tan DA, Haththotuwa R, Fraser IS. Cultural aspects and mythologies around menstruation and abnormal uterine bleeding. Best Pract Res Clin Obstet Gynaecol 2016:1–13. doi:10.1016/j.bpobgyn.2016.09.015.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.bpobgyn.2016.09.015&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=27863914&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 

9.  [9].Speroff L, Darney PD. The history of contraception. In: Jensen JT, Creinin MD, editors. speroff & Darney’s Clinical Guide to Contraception. 6th ed., Philadelphia, Pennsylvania: Wolters Kluwer; 2020, p. online chapter.
    
    

10. [10].Drucker DJ. Contraception: a concise history. Cambridge, Massachusetts: Massachusetts Institute of Technology Press; 2020.
    
    

11. [11].Junod SW, Marks L. Women’s trials: the approval of the first oral contraceptive pill in the United States and Great Britain. Journal of the History of Medicine 2002;57:117–60. doi:10.1093/jhmas/57.2.117.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/jhmas/57.2.117&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=11995593&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000175051000001&link_type=ISI) 

12. [12].Creinin MD, Vieira CS, Westhoff CL, Mansour DJA. Recommendations for standardization of bleeding data analyses in contraceptive studies. Contraception 2022;112:14–22. doi:10.1016/j.contraception.2022.05.011.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.contraception.2022.05.011&link_type=DOI) 

13. [13].Snowden R. The statistical analysis of menstrual bleeding patterns. J Biosoc Sci 1977;9:107–20. doi:10.1017/s0021932000000511.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1017/s0021932000000511&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=845187&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1977CT92800011&link_type=ISI) 

14. [14].Rodriguez G, Faundes-Latham A, Atkinson LE. An approach to the analysis of menstrual patterns in the critical evaluation of contraceptives. Stud Fam Plann 1976;7:42–51.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2307/1964872&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=936255&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1976BJ37600001&link_type=ISI) 

15. [15].Belsey EM, Machin D, D’Arcangues C. The analysis of vaginal bleeding patterns induced by fertility regulating methods. World Health Organization Special Programme of Research, Development and Research Training in Human Reproduction. Contraception 1986;34:253–60. doi:10.1016/0010-7824(86)90006-5.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/0010-7824(86)90006-5&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=3539509&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1986E851500003&link_type=ISI) 

16. [16].Belsey EM, Farley TM. The analysis of menstrual bleeding patterns: a review. Contraception 1988;38:129–56.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/0010-7824(88)90035-2&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=3048871&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1988P685800002&link_type=ISI) 

17. [17].Machin D, D’arcangues C, Busca B, Farley TMM, Pinol A. Vaginal bleeding patterns — the problem and an example data set. Applied Stochastic Models and Data Analysis 1987;3:27–35. doi:10.1002/asm.3150030104.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/asm.3150030104&link_type=DOI) 

18. [18].Belsey EM, Farley TMM. The analysis of menstrual bleeding patterns: A review. Applied Stochastic Models and Data Analysis 1987;3:125–50. doi:10.1002/asm.3150030302.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/asm.3150030302&link_type=DOI) 

19. [19].Belsey EM, Carlson N. The description of menstrual bleeding patterns: Towards fewer measures. Stat Med 1991;10:267–84. doi:10.1002/sim.4780100210.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/sim.4780100210&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=2052804&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 

20. [20].Mishell DR, Guillebaud J, Westhoff C, Nelson AL, Kaunitz AM, Trussell J, et al. Combined hormonal contraceptive trials: variable data collection and bleeding assessment methodologies influence study outcomes and physician perception. Contraception 2007;75:4–10. doi:10.1016/j.contraception.2006.08.008.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.contraception.2006.08.008&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17161116&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 

21. [21].Mishell DR, Guillebaud J, Westhoff C, Nelson AL, Kaunitz AM, Trussell J, et al. Recommendations for standardization of data collection and analysis of bleeding in combined hormone contraceptive trials. Contraception 2007;75:11–5. doi:10.1016/j.contraception.2006.08.012.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.contraception.2006.08.012&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17161117&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 

22. [22].US Food and Drug Administration. Advisory Committee for Reproductive Health Drugs final summary meeting minutes 2007. [https://wayback.archive-it.org/7993/20170404050830/](https://wayback.archive-it.org/7993/20170404050830/)[https://www.fda.gov/ohrms/dockets/ac/07/minutes/2007-4274m1.pdf](https://www.fda.gov/ohrms/dockets/ac/07/minutes/2007-4274m1.pdf) (accessed December 6, 2023).
    
    

23. [23].World Health Organization. HRP at 50: The UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP) 2023. [https://www.who.int/teams/sexual-and-reproductive-health-and-research-(srh)/human-reproduction-programme/hrp-at-50](https://www.who.int/teams/sexual-and-reproductive-health-and-research-(srh)/human-reproduction-programme/hrp-at-50) (accessed December 6, 2023).
    
    

24. [24].US Food and Drug Administration. Patient-Focused Drug Development: Collecting Comprehensive and Representative Input: Guidance for Industry, Food and Drug Administration Staff, and Other Stakeholders 2020. [https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-focused-drug-development-collecting-comprehensive-and-representative-input](https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-focused-drug-development-collecting-comprehensive-and-representative-input) (accessed December 6, 2023).
    
    

25. [25].US Food and Drug Administration. Patient-Focused Drug Development: Methods to Identify What Is Important to Patients: Guidance for Industry, Food and Drug Administration Staff, and Other Stakeholders 2022. [https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-focused-drug-development-methods-identify-what-important-patients](https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-focused-drug-development-methods-identify-what-important-patients) (accessed December 6, 2023).
    
    

26. [26].US Food and Drug Administration. Patient-Focused Drug Development: Selecting, Developing, or Modifying Fit-for-Purpose Clinical Outcome Assessments: Draft Guidance for Industry, Food and Drug Administration Staff, and Other Stakeholders 2022. [https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-focused-drug-development-selecting-developing-or-modifying-fit-purpose-clinical-outcome](https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-focused-drug-development-selecting-developing-or-modifying-fit-purpose-clinical-outcome) (accessed December 6, 2023).
    
    

27. [27].US Food and Drug Administration. Patient-Focused Drug Development: Incorporating Clinical Outcome Assessments Into Endpoints for Regulatory Decision-Making 2023. [https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-focused-drug-development-incorporating-clinical-outcome-assessments-endpoints-regulatory](https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-focused-drug-development-incorporating-clinical-outcome-assessments-endpoints-regulatory) (accessed December 6, 2023).
    
    

28. [28].Johnston-Robledo I, Chrisler JC. The Menstrual Mark: Menstruation as Social Stigma. In: Bobel C, Winkler I, Fahs B, Hasson K, Kissling E, Roberts T, editors. The Palgrave Handbook of Critical Menstruation Studies, Singapore: Springer Singapore; 2020, p. 181–99. doi:10.1007/978-981-15-0614-7_17.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/978-981-15-0614-7_17&link_type=DOI) 

29. [29].Das M. Socio-cultural aspects of menstruation: An anthropological purview. East Anthropol 2008;61:227–40.
    
    

30. [30].Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR, Lazaro P, et al. The RAND/UCLA Appropriateness Method User’s Manual. Santa Monica, CA: 2001.
    
    

31. [31].Broder MS, Gibbs SN, Yermilov I. An Adaptation of the RAND/UCLA Modified Delphi Panel Method in the Time of COVID-19. J Healthc Leadersh 2022;14:63–70. doi:10.2147/JHL.S352500.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2147/JHL.S352500&link_type=DOI) 

32. [32].Beretta R. A critical review of the Delphi technique. Nurse Res 1996;3:79–89. doi:10.7748/nr.3.4.79.s8.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7748/nr.3.4.79.s8&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=27285530&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 

33. [33].Jandhyala R. Delphi, non-RAND modified Delphi, RAND/UCLA appropriateness method and a novel group awareness and consensus methodology for consensus measurement: a systematic literature review. Curr Med Res Opin 2020;36:1873–87. doi:10.1080/03007995.2020.1816946.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/03007995.2020.1816946&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 

34. [34].Horton JN. Nominal group technique. A method of decision-making by committee. Anaesthesia 1980;35:811–4. doi:10.1111/j.1365-2044.1980.tb03924.x.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1365-2044.1980.tb03924.x&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=7446921&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1980KG07600010&link_type=ISI) 

35. [35].Jandhyala R. A novel method for observing proportional group awareness and consensus of items arising from list-generating questioning. Curr Med Res Opin 2020;36:883–93. doi:10.1080/03007995.2020.1734920.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/03007995.2020.1734920&link_type=DOI) 

36. [36].Gnanasakthy A, Qin S, Norcross L. FDA Guidance on Selecting, Developing, or Modifying Fit-for-Purpose Clinical Outcome Assessments: Old Wine in a New Bottle? The Patient-Patient-Centered Outcomes Research 2023;16:3–5. doi:10.1007/s40271-022-00607-6.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s40271-022-00607-6&link_type=DOI) 

37. [37].Patient involvement in the development, regulation and safe use of medicines. Geneva: 2022. doi:10.56759/iiew8982.
    
    

38. [38].Lee KMN, Junkins EJ, Luo C, Fatima UA, Cox ML, Clancy KBH. Investigating trends in those who experience menstrual bleeding changes after SARS-CoV-2 vaccination. Sci Adv 2022;8:eabm7201. doi:10.1126/sciadv.abm7201.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1126/sciadv.abm7201&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=35857495&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 

39. [39].Edelman A, Boniface ER, Benhar E, Han L, Matteson KA, Favaro C, et al. Association Between Menstrual Cycle Length and Coronavirus Disease 2019 (COVID-19) Vaccination: A U.S. Cohort. Obstetrics and Gynecology 2022. doi:10.1097/AOG.0000000000004695.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/AOG.0000000000004695&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=34991109&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 

40. [40].Gibson EA, Li H, Fruh V, Gabra M, Asokan G, Jukic AMZ, et al. Covid-19 vaccination and menstrual cycle length in the Apple Women’s Health Study. NPJ Digit Med 2022;5:165. doi:10.1038/s41746-022-00711-9.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41746-022-00711-9&link_type=DOI) 

41. [41].Wesselink AK, Lovett SM, Weinberg J, Geller RJ, Wang TR, Regan AK, et al. COVID-19 vaccination and menstrual cycle characteristics: A prospective cohort study. Vaccine 2023;41:4327–34. doi:10.1016/j.vaccine.2023.06.012.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.vaccine.2023.06.012&link_type=DOI) 

42. [42].Kareem R, Sethi MR, Inayat S, Irfan M. The effect of COVID-19 vaccination on the menstrual pattern and mental health of the medical students: A mixed-methods study from a low and middle-income country. PLoS One 2022;17:e0277288. doi:10.1371/journal.pone.0277288.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0277288&link_type=DOI) 

43. [43].Mackenzie A, Chung S, Hoppes E, Mickler A, Cartwright A. Measurement of changes to the menstrual cycle: A transdisciplinary systematic review evaluating measure quality and utility for clinical trials. Submitted to PloS One 2023.
    
    

44. [44].USG General Services Administration. Federal plain language guidelines. 2023 n.d. [https://www.plainlanguage.gov/guidelines/](https://www.plainlanguage.gov/guidelines/) (accessed December 6, 2023).
    
    

45. [45].World Wide Web Consortium (W3C). Web Content Accessibility Guidelines 2023. [https://www.w3.org/standards/](https://www.w3.org/standards/) (accessed December 6, 2023).
    
    

46. [46].Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, et al. Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes (PRO) Measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health 2005;8:94–104. doi:10.1111/j.1524-4733.2005.04054.x.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1524-4733.2005.04054.x&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=15804318&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000227723000003&link_type=ISI) 

47. [47].Wild D, Eremenco S, Mear I, Martin M, Houchin C, Gawlicki M, et al. Multinational trials-recommendations on the translations required, approaches to using the same language in different countries, and the approaches to support pooling the data: the ISPOR Patient-Reported Outcomes Translation and Linguistic Validation Good Research Practices Task Force report. Value Health 2009;12:430–40. doi:10.1111/j.1524-4733.2008.00471.x.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1524-4733.2008.00471.x&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19138309&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000266467800008&link_type=ISI) 

48. [48].Eremenco S, Coons SJ, Paty J, Coyne K, Bennett A V, McEntegart D, et al. PRO data collection in clinical trials using mixed modes: report of the ISPOR PRO mixed modes good research practices task force. Value Health 2014;17:501–16. doi:10.1016/j.jval.2014.06.005.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jval.2014.06.005&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25128043&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 

49. [49].Byrom B, Gwaltney C, Slagle A, Gnanasakthy A, Muehlhausen W. Measurement Equivalence of Patient-Reported Outcome Measures Migrated to Electronic Formats: A Review of Evidence and Recommendations for Clinical Trials and Bring Your Own Device. Ther Innov Regul Sci 2019;53:426–30. doi:10.1177/2168479018793369.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/2168479018793369&link_type=DOI) 

50. [50].Vayena E, Blasimme A, Cohen IG. Machine learning in medicine: Addressing ethical challenges. PLoS Med 2018;15:e1002689. doi:10.1371/journal.pmed.1002689.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pmed.1002689&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 

51. [51].Mooney SJ, Pejaver V. Big Data in Public Health: Terminology, Machine Learning, and Privacy. Annu Rev Public Health 2018;39:95–112. doi:10.1146/annurev-publhealth-040617-014208.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1146/annurev-publhealth-040617-014208&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 

52. [52].Price WN, Cohen IG. Privacy in the age of medical big data. Nat Med 2019;25:37–43. doi:10.1038/s41591-018-0272-7.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41591-018-0272-7&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 

53. [53].Jeyaraman M, Balaji S, Jeyaraman N, Yadav S. Unraveling the Ethical Enigma: Artificial Intelligence in Healthcare. Cureus 2023;15:e43262. doi:10.7759/cureus.43262.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7759/cureus.43262&link_type=DOI) 

54. [54].Abràmoff MD, Tarver ME, Loyo-Berrios N, Trujillo S, Char D, Obermeyer Z, et al. Considerations for addressing bias in artificial intelligence for health equity. NPJ Digit Med 2023;6:170. doi:10.1038/s41746-023-00913-9.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41746-023-00913-9&link_type=DOI) 

55. [55].Munro MG, Critchley HOD, Fraser IS, FIGO Menstrual Disorders Committee. The two FIGO systems for normal and abnormal uterine bleeding symptoms and classification of causes of abnormal uterine bleeding in the reproductive years: 2018 revisions. Int J Gynaecol Obstet 2018;143:393–408. doi:10.1002/ijgo.12666.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/ijgo.12666&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 

56. [56].de Vet HCW, Terwee CB, Mokkink LB, Knol DL. Measurement in Medicine. Cambridge University Press; 2011. doi:10.1017/CBO9780511996214.
    
    

57. [57].Severy LJ, Newcomer S. Critical issues in contraceptive and STI acceptability research. Journal of Social Issues 2005;61:45–65. doi:10.1111/j.0022-4537.2005.00393.x.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.0022-4537.2005.00393.x&link_type=DOI) 

58. [58].Reeve BB, Wyrwich KW, Wu AW, Velikova G, Terwee CB, Snyder CF, et al. ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Qual Life Res 2013;22:1889–905. doi:10.1007/s11136-012-0344-y.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s11136-012-0344-y&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=23288613&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 

59. [59].Gutin I. In BMI We Trust: Reframing the Body Mass Index as a Measure of Health. Soc Theory Health 2018;16:256–71. doi:10.1057/s41285-017-0055-0.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1057/s41285-017-0055-0&link_type=DOI) 

60. [60].Bray GA. Beyond BMI. Nutrients 2023;15. doi:10.3390/nu15102254.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3390/nu15102254&link_type=DOI) 

61. [61].Gutin I. Body mass index is just a number: Conflating riskiness and unhealthiness in discourse on body size. Sociol Health Illn 2021;43:1437–53. doi:10.1111/1467-9566.13309.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/1467-9566.13309&link_type=DOI) 

62. [62].Deurenberg P, Deurenberg-Yap M. Validity of body composition methods across ethnic population groups. Forum Nutr 2003;56:299–301.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=15806909&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F05%2F2024.04.04.24305350.atom) 

63. [63].US Food and Drug Administration. Establishing Effectiveness and Safety for Hormonal Drug Products Intended to Prevent Pregnancy Guidance for Industry 2019. [https://www.fda.gov/regulatory-information/search-fda-guidance-documents/establishing-effectiveness-and-safety-hormonal-drug-products-intended-prevent-pregnancy-guidance](https://www.fda.gov/regulatory-information/search-fda-guidance-documents/establishing-effectiveness-and-safety-hormonal-drug-products-intended-prevent-pregnancy-guidance) (accessed December 6, 2023).
    
    

64. [64].Lin G, Rodriguez R. Weighted Methods for Analyzing Missing Data with the GEE Procedure. SAS Conference Proceedings: SAS Global Forum 2014, 2014.
    
    

65. [65].SAS Institute Inc. PROC GEE. SAS/STAT® 15.2 User’s Guide, Cary, North Carolina: SAS Institute Inc.; 2020.
    
    

66. [66].Fitzmaurice GM, Molenberghs G, Lipsitz SR. Regression Models for Longitudinal Binary Responses with Informative Drop-Outs. Journal of the Royal Statistical Society: Series B (Methodological) 1995;57:691–704. doi:10.1111/j.2517-6161.1995.tb02056.x.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.2517-6161.1995.tb02056.x&link_type=DOI) 

67. [67].Henriksen-Bulmer J, Jeary S. Re-identification attacks—A systematic literature review. Int J Inf Manage 2016;36:1184–92. doi:10.1016/j.ijinfomgt.2016.08.002.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.ijinfomgt.2016.08.002&link_type=DOI) 

68. [68].Chikwetu L, Miao Y, Woldetensae MK, Bell D, Goldenholz DM, Dunn J. Does deidentification of data from wearable devices give us a false sense of security? A systematic review. Lancet Digit Health 2023;5:e239–47. doi:10.1016/S2589-7500(22)00234-5.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S2589-7500(22)00234-5&link_type=DOI) 

69. [69].Gymrek M, McGuire AL, Golan D, Halperin E, Erlich Y. Identifying personal genomes by surname inference. Science 2013;339:321–4. doi:10.1126/science.1229566.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6Mzoic2NpIjtzOjU6InJlc2lkIjtzOjEyOiIzMzkvNjExNy8zMjEiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyNC8wNC8wNS8yMDI0LjA0LjA0LjI0MzA1MzUwLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

70. [70].Tyndall A, Du W, Breder CD. Regulatory watch: The target product profile as a tool for regulatory communication: advantageous but underused. Nat Rev Drug Discov 2017;16:156. doi:10.1038/nrd.2016.264.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/nrd.2016.264&link_type=DOI)