Participants’ perspectives and preferences on clinical trial result dissemination: The TRUST Thyroid Trial experience

Introduction

Patient and public involvement (PPI) is increasingly recognised as an essential component of clinical research. In the UK, the national advisory group supporting active public involvement in health services, public health and social care research (INVOLVE) defines PPI as ‘research being carried out ‘with’ or ‘by’ members of the public rather than ‘to’ ‘about’ or ‘for’ them’¹. In clinical trials, PPI has been defined as experimenting with participants instead of experimenting on participants². PPI may occur at any stage during the research process from priority setting and drafting study protocols right through to conducting the study, interpreting the end results and communicating and disseminating research findings^3,4. Research funders increasingly expect that PPI is prioritised and resourced within studies. This increasing expectation has heightened the risk of researchers carrying out ‘tick-box’ PPI rather than ‘meaningful’ involvement⁵. There are many moral and ethical arguments being made for PPI. Many believe that as citizens and taxpayers, members of the public have a right to influence research that is being funded by public money⁶. PPI researchers are also making pragmatic arguments for PPI and providing anecdotal accounts about how PPI can make research more relevant, accessible and acceptable to participants⁷. The ethical arguments are often seen as sufficient regardless of any pragmatic impact. However, PPI costs time and money, therefore pragmatic claims need scrutiny⁸. More substantive evidence is needed to evaluate the potential impact of PPI on the conduct and outcomes of research^5,9. In 2001, the need to establish if PPI leads to actual, rather than merely perceived benefits for research processes and output was identified. Over fifteen years later, this need remains.

In clinical research, the results of clinical trials have not traditionally been shared with clinical trial participants. A recent survey carried out on a large registry of health research participants, found that while 95.6% of respondents said researchers should always or sometimes offer the results to participants, only 33% of respondents actually received the results of studies in which they had participated¹⁰. An upcoming European Union Clinical Trial Regulation requires sponsors to provide summary results of clinical trials in a format understandable to laypersons, including participants¹¹. However, there is a lack of evidence on the most appropriate methods of sharing results with participants. Uncertainty persists around what information should be shared, how results should be shared and who should be responsible for sharing the results. Since the findings of clinical research often exist in a complex context of scientific exchange and debate, it is important that the information shared is accessible and relevant to participants¹². The increasing understanding of the importance of sharing research results with study participants is somewhat linked to a wider movement towards transparency in trials. This movement is largely promoted by initiatives such as SPIRIT, CONSORT and AllTrials. The SPIRIT Statement provides guidance to researchers to improve the completeness and quality of trial protocols¹³, the Consolodated Standards of Reporting Trials (CONSORT) Statement is an evidence based, minimum set of recommendations for reporting randomised trials¹⁴ and the AllTRials iniative calls for all past and present triasl to be registered and their full methods and summary results reported¹⁵. Some of these initiatives also include recommendations for disseminating results to research participants. For example, the SPIRIT statement states that study results must be released to participating physicians, referring physicians, patients and the general medical community¹³.

The Thyroid Hormone Replacement for Subclinical Hypothyroidism Trial (TRUST) was a multi-centre, double blind, placebo controlled, phase III clinical trial testing the efficacy of thyroxine replacement in subclinical hypothyroidism in older community dwelling adults¹⁶. The results of the TRUST trial were published in the New England Journal of Medicine on 3^rd of April, 2017¹⁶. This Study Within A Trial (SWAT) was conducted at the Irish TRUST trial site prior to and after publication of results. The aim of this SWAT was to investigate methods of disseminating trial findings to participants by using a PPI approach to identify, develop and evaluate a patient-based approach of receiving trial results.

Methods

Phase One: Identification and development of patient-based approach (qualitative and PPI phase)

The first phase of the study used a qualitative approach to iteratively identify and develop a patient-based approach to disseminate the results of TRUST trial. This was done in three separate stages: qualitative focus groups, a PPI group and an adult literacy review.

Phase Two: Dissemination of trial results (intervention phase)

The second phase of the study used a SWAT intervention to disseminate the results of the TRUST Thyroid Trial to trial participants. This was done using a prospective, randomised, single blind, parallel trial design. It is important to note that when the term randomisation is used, it refers to the allocation of patients to intervention/control within the SWAT and not the TRUST Thyroid trial. Irish TRUST participants were randomised to intervention or control groups using an online random number generator. The intervention group received the patient-based letter format (see Supplementary File 3: Final version patient-based results letter) and the control group received a copy of the TRUST results press release, which was made available by the lead study site on the TRUST Thyroid Trial Website (see Supplementary File 4: Standard results letter). Participants were blinded to their intervention group. One member of the research team was un-blinded in order to perform the randomisation and distribute the results of the trial. As they were un-blinded to perform these two important tasks, they were not involved in the data analysis or interpretation in any way.

Phase Three: Evaluation of patient –based approach (quantitative phase)

The third phase of the study used a quantitative patient understanding questionnaire to evaluate the patient-based approach to disseminating trial results. The questionnaire was developed in consultation with experts in the area of subclinical hypothyroidism and scale (questionnaire) development (PK and KMS). The early development of the questionnaire was guided by a consultation document, which accompanies the EU Clinical Trials Regulation No 536/2014²². This document highlights the information which should be presented to trial participants in the trial summary at the end of a trial. However, initial questionnaire items were modified to allow for psychometric testing. The final questionnaire contained 12 questions; six items were measured on a five point LIKERT scale, there were four multiple-choice questions and two vignettes. The first six items measured patients’ perceived understanding of results, the four multiple choice measured patients’ actual understanding of results by requiring them to select the correct answer. To further test participants’ understanding of the trial results, two vignettes describing two typical patient case studies of older adults with subclinical hypothyroidism were provided with a question on whether a doctor should prescribe thyroxine for the hypothetical patient described. The questionnaire was reviewed by the PPI group to assess content and face validity. It then underwent further review by NALA to ensure adherence to the national ‘Plain English’ standard. The final version of the questionnaire can be seen in Supplementary File 5: Patient understanding questionnaire.

The questionnaire was sent to all Irish TRUST participants (intervention and control group) one week after they received the results of the trial. A reminder questionnaire was sent to non-responders 3 weeks later.

Analysis. The primary outcome was the difference in levels of patient understanding between the intervention and control groups. This measured the impact of PPI on patient understanding of end of trial results. The psychometric properties and construct validity of the questionnaire were examined with exploratory factor analysis. Principal component analysis (PCA) was conducted on the six LIKERT scale items. Internal consistency of the questionnaire was investigated using Cronbach's alpha coefficient. Completed questionnaires were entered into SPSS software (version 24) and analysed using descriptive and inferential (Chi-square test and Fishers Exact) statistics. The researcher carrying out data input and analysis was blinded to the participants' allocation status.

Results

Characteristics of the trial participants stratified by participation in the different stages of the study are presented in Table 1.

Phase One: Identification and development of patient-based approach (qualitative and PPI phase)

Phase Two: Dissemination of trial results (intervention phase)

There were a total of 101 Irish TRUST participants randomised to the SWAT intervention. Trial participants from the PPI group (n = 3) were excluded from randomisation as they reviewed the content of the intervention method prior to the intervention. The intervention group (n=51) received the patient-based letter format (see Supplementary File 3: Final version patient-based results letter) and the control group (n=50) received a copy of the TRUST results press release, which was made available by the lead study site on the TRUST Thyroid Trial Website (see Supplementary File 4: Standard results letter).

Phase Three: Evaluation of patient-based approach (quantitative phase)

The overall response rate for the patient understanding questionnaire was 68% (69/101). The response rate for the intervention group was 74% (38/51) and the response rate for the control group was 62% (31/50). There were no significant differences in age, gender and education between those who returned the questionnaire and those who did not.

Post hoc power calculations showed that the study was underpowered to detect an effect. Power for each of the patient understanding components ranged from .01 to. 58.

Table 2 below shows the results of patients’ perceived understanding of the purpose and context of the TRUST Thyroid Trial. Due to low participant numbers across the five Likert responses, the questionnaire response bands have been contracted from ‘Strongly Agree’ and ‘Agree’ to ‘Yes, ‘Strongly Disagree’ and ‘Disagree’ to ‘No’ and ‘Neither agree nor disagree’ to ‘Neutral’. The results show that patients’ perceptions of understanding are similar between the intervention and control groups. Subgroup analysis showed patient’s understanding was not significantly impacted by age, gender or educational level.

Figure 1 shows patients’ actual understanding of the primary aim, side effect and results of the TRUST Thyroid Trial. Almost 82% (n=31) of the intervention group and 65% (n=20) of the control group correctly understood the primary aim of the TRUST trial (p=0.108). Almost 40% (n=15) of the intervention group and 36% (n=9) of the control group correctly understood the associated side effects of the active drug (p=0.734). In total 50% of the intervention group (n=19) and 58% of the control group correctly understood the results of the trial (p=0.504). There were no differences in patient understanding of trial results between the intervention and control groups.

Figure 1. Patient understanding of primary aim, side effect and trial result of the TRUST Thyroid Trial presented by group¹.

¹Patient understanding of primary aim, side effect and trial result was assessed using multiple choice questions.

In terms of patient understanding of hypothetical patient case studies, 43% (n=13) of the intervention group gave the correct answer to Vignette A; this was lower than the control group (62.1%, n=18, p=0.15). In total 77% (n=23) of the intervention group gave the correct answer to Vignette B, this was higher than the control group (66%, n=19, p=0.344).

Discussion

While PPI is increasingly recognised as an important element of clinical research, evidence on optimal methods and potential impact is lacking^4,9. Previous research conducted on the impact of PPI has largely focused on the experiences of participants and researchers²⁵ and on the research process in broad terms²⁶. In this study, our primary outcome was specific: a quantitative measure of patient understanding of trial results between those who received the patient-based approach and the standard approach. To our knowledge there has been no previous research conducted on the impact of PPI on patient understanding of trial results.

The involvement of clinical trial participants in this study offered insightful perspectives on the information needs of the study population in terms of receiving end of trial results. Study findings show that trial participants want to receive the results of the clinical trial in which they had participated. This is supported by much of the available literature on patients’ preferences of receiving results, with up to 90% of participants in previous studies reporting a desire to receive results²⁷. Focus group findings showed that participants felt that receiving results would provide an acknowledgement of their individual contribution to the trial. This finding complements previous commentaries about result sharing being an ‘ethical imperative or ‘moral obligation’. Fernandez et al. points out that many participants place their trust in science and researchers owe a debt to participants to fulfil their trust and recognise their altruism^12,28.

Unsurprisingly, findings also show that participants want to receive results that are accessible and easy to understand. In this study, the preferred format of receiving results was a letter posted to them directly from the TRUST trial. This preference is also consistent with the literature on patient preferences of receiving results. A previous study investigating prefrences of individuals taking part in a cardiac rehabilitation trial found that 80% of trial participants preferred to receive the results by post²⁹. The patient-based approach identified in this study was feasible for researchers to develop with significant involvement from trial participants and adult literacy experts.

Previous studies exploring participants’ reactions found that sharing trial results with participants can cause some negative impacts such as anxiety, anger, guilt, upset and confusion^30–32. As far as researchers in this study are aware, providing results did not cause any negative impacts. This may have been due to the fact that the TRUST trial had a low risk of morbidity or mortality compared to some of the other studies citing negative impacts. Both result methods contained the telephone number, email address and postal address of the research team and participants were urged to contact should they have any questions or concerns relating to the study. The research team did not receive any queries.

Previous systematic reviews highlight the lack of evidence on economic analysis of PPI and call for researchers to consider the costs of its implementation^26,33. As discussed previously research funders are increasingly demanding that PPI be carried out in research. However, the costs of PPI are often underestimated and can cause a significant financial burden on research project budgets^26,33–35. It is extremely important that researchers plan PPI at the grant proposal stage and estimate the costs appropriately. If these costs are not correctly estimated during the initial stages of developing research proposals, they may cause a financial burden on PPI partners.

Participants in this study were not paid for their time but were provided with a €20 voucher to cover travel expenses. When PPI is not the primary focus of a study, researchers do not consider the cost implications at the beginning of the study and are often tied with limited resources to carry out PPI^34–36. INVOLVE, the national advisory group supporting active public involvement in health services, public health and social care research in the UK, have recommended that PPI partners should be paid for their involvement³⁷. Despite this, existing research suggests that institutional difficulties make negotiating the mechanisms of paying participants very difficult³⁴. One study reported that in order for participants to be remunerated for their efforts, they needed to be registered as employees, a process that incurred much paperwork and time delays³⁴. This study outlines the cost of conducting PPI and includes a full breakdown of costs (see Supplementary File 6: Costs of conducting PPI). This breakdown provides a template to other researchers who plan to carry out and evaluate PPI as part of their research. It is important to note that not all costs associated with carrying out the study were included in this amount. For example, the only salary costed was that of the research assistant. The expertise provided by other members of the study team were not included in the total cost as they were being paid by the University or other research grants. The total cost of conducting this study was €8,049 which is not insignificant but should be considered in the context of the cost of large-scale trials.

Conclusion

Patient and Public Involvement (PPI) is advocated for every step of the trial process. We have demonstrated that it is feasible to involve PPI partners in the development of dissemination materials. Sharing clinical trial results with participants in a format understandable to laypersons will soon be a legal requirement¹¹. However, there is a significant lack of evidence as to the most appropriate methods of sharing results with participants. The study identified and developed a patient-based approach to disseminating clinical trial results for trial participants. Although, in this study PPI did not influence patients’ final understanding of results, it documents the process of conducting PPI within the clinical trial setting. This process may be useful for other trialists interested in conducting and evaluating the impact of PPI in clinical trials.

Ethics approval and consent to participate

The research was approved in Ireland by the Clinical Research Ethics Committee of the Cork Teaching Hospitals, UCC, Ref ECM 4 (t).

All participants provided signed informed consent to take part in the study.

Data availability

The raw data from this study cannot be sufficiently de-identified, and therefore are not publicly available. However, the data from the current study are available for further (collaborative) research purposes on reasonable request. Available datasets include transcripts from focus groups, field notes from PPI sessions and responses from the patient understanding questionnaire. To access the data, please contact the corresponding author (emmy.racine@ucc.ie) or the Principal Investigator (patricia.kearney@ucc.ie). Researchers must provide a written proposal on how the data will be used in research before access is granted.