FDA and EMA clinical research guidelines: Assessment of trial design recommendations for pivotal psychiatric drug trials (Protocol)

Uncertain patient-relevant benefits of new drugs

Several recent reports have found that newly authorised medicines often have questionable or no added patient-relevant benefits compared to already available, older treatments.^1-6 In a cohort of 216 drugs approved by the German drug regulatory authority between 2011 and 2016, the independent Institute for Quality and Efficiency in Health Care (IQWiG) judged that 22 (10 %) new drugs had substantial benefits compared to already available treatments, while 125 (58 %) drugs had no proof of added benefits.¹ Cohort studies of oncology drugs approved by the European Medicines Agency (EMA)⁴ and the US Food and Drug Administration (FDA)⁵ found that most drugs were approved without evidence of benefits on overall mortality or quality of life. Similarly, an analysis of oncology drugs approved by the EMA between 2014 and 2016 reported that many pivotal trials underpinning drug approvals were of high risk of bias due to problems with the trial design and or trial conduct.⁶ It has been argued that the threshold for new drug approvals has been lowered to accommodate the pharmaceutical industry’s interests,⁷ and researchers have advocated for more pragmatic, patient-relevant trials.⁸

Field specific issues in psychiatry

Systematic reviews, particularly of antidepressants for depression^{9, 10} and of central stimulants for attention deficit hyperactivity disorder,^11-13 have highlighted problems of low generalizability of psychiatric drug trials, similar to those identified in pivotal oncology drug trials. Common methodological limitations in these trials are small sample size, short trial duration, restricted trial populations in terms of allowed psychiatric comorbidity, risk of withdrawal effects due to previous exposure to the drug of interest, and the use of surrogates and rating scales rather than patient-relevant outcomes.^9-13 In the German cohort of authorised drugs,¹ one out of 18 newly approved drugs (combined psychiatry/neurology) was judged to have substantial added benefits compared to already available treatments.

Drug regulatory agency guidelines on how to design pivotal trials

The FDA and EMA publish guidelines on how to design and conduct pivotal trials for new drug approvals. Whether there are important differences in the way FDA and EMA apply and enforce these regulatory guidelines is uncertain.

EMA describes their Clinical Efficacy and Safety Guidelines as follows: “The European Medicines Agency’s scientific guidelines on the clinical efficacy and safety of human medicines help applicants prepare marketing authorisation applications. Guidelines reflect a harmonised approach of the EU Member States and the Agency on how to interpret and apply the requirements for the demonstration of quality, safety and efficacy set out in the Community directives”¹⁴ and “The Agency strongly encourages applicants and marketing

authorisation holders to follow these guidelines. Applicants need to justify deviations from guidelines fully in their applications at the time of submission. Before that, they should seek scientific advice, to discuss any proposed deviations during medicine development.”¹⁴ The EMA research guidelines should therefore be understood as the minimal required standard for pivotal trial design within the European Union.

The FDA has also recently begun issuing guidelines for designing pivotal trials. The FDA describes these documents as, “Guidance documents represent FDA’s current thinking on a topic. They do not create or confer any rights for or on any person and do not operate to bind FDA or the public. You can use an alternative approach if the approach satisfies the requirements of the applicable statutes and regulations”.¹⁵

We want to systematically assess how these guidelines are developed and discuss the recommended trial designs from a clinical point of view.

Drug regulator guidelines

1. Guideline committee members’ conflicts of interest

The FDA and EMA guideline committee members and their declared conflicts of interest are not available in the publicly available documents on the websites. We will therefore file Freedom of Information Act requests to access the committee member lists and their declared conflicts of interest. We have previously filed a request for access to the EMA ADHD guideline committee, which we were granted. We will report the proportion of committee members with declared conflicts of interest for each guideline.

2. Stakeholder comments’ documents

3. Commenting phase on the draft guidelines

We will explore how the commenting phases on the draft guidelines are organised. We will assess whether stakeholders are actively or passively recruited to comment on the draft guidelines, e.g. if specific stakeholders are actively invited through invitations. We will assess:

• How long the draft versions are available for commenting.

• Whether there are any restrictions on who is allowed to comment, e.g. whether specific credentials or affiliations are required.

• Is the commenting phase advertised, e.g. on social media, in newsletters, or in relevant journals to make people aware that the draft guidelines are open for input.

We will search the FDA and EMA websites for information and contact the regulators directly for additional information, if necessary.

4. Trial design recommendations

For all included research guidelines, we will extract the trial design recommendations regarding four specific trial characteristics:

• Duration of follow-up, also after the randomised phase ended

• Exclusion criteria related to psychiatric comorbidity

• Previous exposure to the drug, or drug class

• Efficacy outcomes (primary and secondary)

We will describe each design recommendation in the context of how the drug (or similar drugs) is being used, outcome preferences by the patients or established gold standard outcomes in the COMET database, and how the trial recommendations may impact the external validity. See table 1 for a full description of how we will assess each trial characteristic.

Data extraction

One researcher (KB) will identify the research guidelines and stakeholder comments and extract outcome data. Another researcher will be involved when there is doubt about the inclusion of guidelines, stakeholder comments or data extraction.

Data analysis

We will report:

• Total number of guideline committee members on each guideline, and proportion (%) of guideline committee members with declared conflicts of interest.

• Total number of stakeholders, total number of stakeholder comments on each guideline, and the proportion (%) of comments from each stakeholder category: industry, not-industry but with industry-related conflicts, independent, and unclear relationship.

We will not apply specific statistics on the analyses of the commenting phase and the description of the trial design recommendations.

Strengths of the project

To our knowledge, this will be the first systematic and empirical assessment of the FDA and EMA guidelines on how to design pivotal trials. We have prespecified all outcomes in this protocol, which will be publicly available before the data extraction and data analysis begin. We believe it is an important question to address how two major drug regulators conceive their clinical research recommendations, if the guideline development is influenced by vested interests and whether the trial design recommendations may contribute to research waste.¹⁶

Limitations of the project

The number of preliminary identified FDA Guidance Documents is small compared to the cohort of EMA guidelines. We also note that not all EMA guidelines have available stakeholder comments. We depend on the FDA and EMA on getting access to the lists of guideline committee members and their declared conflicts of interest, since these documents are not publicly available. If we are not granted access to these documents within a reasonable time frame, we may have to skip this research question.

Timeline of the project

Data extraction and analysis will be conducted in January to March 2020. We expect to complete the project in early Spring 2020.