Abstract
Background
With the aim of increasing benefits and decreasing harms, risk-based breast cancer screening has been proposed as an alternative to age-based screening. This study explores barriers and facilitators to implementing a risk-based breast cancer screening program from the perspective of health professionals, in the context of a National Health Service.
Methods
Socio-constructivist qualitative research carried out in Catalonia (Spain), in the year 2019. Four discussion groups were conducted, with a total of 29 health professionals from primary care, breast cancer screening programs, hospital breast units, epidemiology units, and clinical specialties. A descriptive-interpretive thematic analysis was performed.
Results
Identified barriers included resistance to reducing the number of screening exams for low-risk women; resistance to change for health professionals; difficulties in risk communication; lack of conclusive evidence of the benefits of risk-based screening; limited economic resources; and organizational transformation. Facilitators include benefits of risk-based strategies for high and low-risk women; women’s active role in their health care; proximity of women and primary care professionals; experience of health professionals in other screening programs; and greater efficiency of a risk-based screening program. Organizational and administrative changes in the health system, commitment by policy makers, training of health professionals, and educational interventions addressed to the general population will be required.
Conclusions
Despite the expressed difficulties, participants supported the implementation of risk-based screening. They highlighted its benefits, especially for women at high risk of breast cancer and those under 50 years of age, and assumed a greater efficiency of the risk-based program compared to the aged-based one. Future studies should assess the efficiency and feasibility of risk-based breast cancer screening for its transfer to clinical practice.
Figures
Citation: Laza-Vásquez C, Codern-Bové N, Cardona-Cardona À, Hernández-Leal MJ, Pérez-Lacasta MJ, Carles-Lavila M, et al. (2022) Views of health professionals on risk-based breast cancer screening and its implementation in the Spanish National Health System: A qualitative discussion group study. PLoS ONE 17(2): e0263788. https://doi.org/10.1371/journal.pone.0263788
Editor: Eugenio Paci, Centro per lo Studio e la Prevenzione Oncologica, ITALY
Received: July 3, 2021; Accepted: January 26, 2022; Published: February 4, 2022
Copyright: © 2022 Laza-Vásquez et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data consists of recorded group discussions and the corresponding transcriptions. Languages used were Catalan and Spanish. Data cannot be shared publicly because we did not ask participants' consent to share the recorded materials. Data are available from the The Drug Research Ethics Committee of the University Hospital Arnau de Vilanova at Lleida (contact via e-mail ceim.lleida.ics@gencat.cat) for researchers who meet the criteria for access to confidential data.
Funding: MR PI17/00834: Personalized breast cancer screening: assessment of its feasibility and acceptability in the National Health System. Instituto de Salud Carlos III and cofunded by Fondo Europeo de Desarrollo Regional (FEDER) ‘Una manera de hacer Europa’ https://www.isciii.es MCL PI18/00773: Collaboration of healthcare professionals to include shared decision-making in the breast cancer screening program. Instituto de Salud Carlos III and cofunded by Fondo Europeo de Desarrollo Regional (FEDER) ‘Una manera de hacer Europa’ https://www.isciii.es CLV Santander Program scholarship 2020: predoctoral fellow at the University of Lleida MJH-L: European Regional Development Fund (ERDF). European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 713679 from the Universitat Rovira i Virgili (URV). The funders did not participate in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Breast cancer is the most common cancer in women and overall. In 2020, more than 2.2 million new cases were diagnosed, representing 11.7% of all cancer cases. Breast cancer was the leading cause of cancer deaths in women with 68,000 deaths worldwide. In Spain, in 2020, the estimated numbers of incident cases and breast cancer deaths in women were 34,000 and 6,600, respectively [1].
Screening programs are one of the pillars in the fight against breast cancer, they aim to reduce mortality from this cause through early detection and treatment. However, there is evidence that screening may cause harm, e.g. false positive and false negative results or overdiagnosis of tumors that would otherwise not go on to cause symptoms or death [2], leading to negative consequences for women and economic consequences for health systems [3, 4].
Current population-based screening programs use age as the only condition to define the target population. With the aim of increasing benefits and decreasing harms, risk-based screening has been proposed as an alternative to age-based screening. Age, breast density, family history of breast cancer, previous benign lesions and genetic information have been the main risk factors used to define risk groups [5, 6]. Yanes et al. [7] showed that, in European populations, the addition of a PRS to the existing risk models improved their accuracy and predictive ability. Evans et al. [8] added a PRS and mammographic density to the Tyrer-Cuzick model, with the objective of enabling more targeted early detection/prevention strategies in population screening programs. The combined risk tool defined a low-risk group, about 30% of the total, such that cancers identified in this group were more likely to have a very good prognosis.
Although evidence of the effectiveness of risk-based screening from clinical trials is not available yet, a systematic review that included nine modeling studies and an observational study showed that with personalized screening, the gain in quality adjusted life years would be higher at a lower cost, compared with the standard age-based strategy [9]. However, evidence is lacking on feasibility and acceptance by the target population. French et al. [10] developed an automated system (BC-Predict) for offering an assessment of breast cancer risk and communication to women and health professionals. The study aimed to identify and resolve key uncertainties regarding the feasibility of integrating BC-Predict into the breast screening program in England. Pons-Rodriguez et al. recently published the protocol of a proof-of-concept study on feasibility and acceptability of personalized breast cancer screening in Catalonia (Spain) [11].
Although risk-based breast cancer screening seems promising, it is expected that its implementation as a population-wide screening will pose complex issues and will require the involvement of healthcare professionals. In addition, in a context of uncertainty where health professionals with clinical expertise have access to the best evidence and women have their experiences, values and preferences, it seems necessary to move from a paternalistic to a participatory style of care [12]. Shared decision-making would facilitate participatory decisions in clinical encounters where health professionals would explain, in a balanced way, the benefits and harms of screening [13] and provide risk-based screening recommendations.
Several studies have addressed the perspectives of health professionals on risk-based screening [14–19]. These works have described barriers such as the need for more evidence on effectiveness and efficiency [14], concerns about the capacity of the healthcare system to provide appropriate human resources, economic costs, lack of knowledge among healthcare providers [15], time constraints, low health literacy and language barriers to risk communication [18].
The organization of the Spanish National Health Service (SpNHS) is decentralized in 17 regions with screening guidelines set by the local governments and coordinated by the Network of Cancer Screening Programs and the Ministry of Health [20]. Most of the screening programs target women aged 50–69 years and perform biennial mammograms. However, opportunistic screening in women younger than 50 years is widely used, mostly in private consultations. Including women younger than 50 years in risk-based screening, so low-risk women are recommended to wait and high-risk women are screened, may improve the balance of benefits and harms, as mathematical models have shown [3].
The most recent recommendations of the Spanish Family and Community Medicine Society (SemFYC) state that the primary care physicians should provide objective and proven information, motivating women to make an informed decision about their participation in the screening program, and also should collaborate in the identification of women at high risk due to a family or personal history of breast cancer [21]. Therefore, studies exploring attitudes, experiences and opinions of healthcare professionals that may play a prominent role in implementing risk-based screening programs in our setting are needed.
The objective of the study was to explore the barriers and facilitators of implementing a risk-based breast cancer screening program from the point of view of health professionals, in the context of the SpNHS. The study is part of the DECIDO project, which aims to assess the acceptability and feasibility of offering personalized breast cancer screening and its integration into clinical practice [11].
Methods
Design and methodologic perspective
A socio-constructivist qualitative study on the barriers and facilitators of personalized breast cancer screening, in the Catalan Health Service, was conducted. Using a descriptive and interpretive study we intended to find out the meaning and interpretation of these barriers and facilitators, based on the meanings and explanations that health professionals attribute to them, according to their context, experience and reflection [22].
Context and participants in the study
Field work was carried out in the cities of Lleida and Barcelona (Catalonia, Spain) between April and November 2019. The informants were health professionals (including health service providers, managers and administrative workers) from primary care, breast cancer screening programs, hospital breast units, epidemiology units, and clinical specialties such as oncology, radiation oncology, and diagnostic imaging.
Health professionals were selected according to a theoretical sampling strategy aimed at ensuring heterogeneity and capturing their potential discursive diversity [23]. The selection criteria were: work area in relation to breast cancer (early diagnosis, cancer treatment, and primary care), field of expertise (nurse, midwife, general practitioner, specialist or health manager), and geographical health area (Lleida, Bellvitge Hospital/Catalan Institute of Oncology, Parc de Salut Mar, and Vall d’Hebron Hospital).
The participant search was carried out by the research team. We contacted the heads of the health care services to inform them about the study and ask them to inform and request contact consent from health professionals working with them. Professionals interested in participating received information about the study, dates of the discussion groups, and were invited to send a reply on their decision to participate. The sample included 29 health professionals whose characteristics are described in Table 1.
Data collection
Discussion groups were held to collect the participants’ perspectives. A discussion group consists of a meeting of seven to ten people who have been selected based on specific profiles according to the research objectives, to comment and debate on a series of topics, induced by a moderator. It is intended to reproduce a micro-social situation in the image of what would be a macro-social situation, through the interaction of the different profiles, as an effective way of extracting experiences, reactions, emotions and ideas according to Patton et al. [24].
The intention was to build a debate based on an initial provocation (e.g. is it possible to implement a risk-based cancer screening program in the SpNHS?). The discussion group technique is less directive than the focus group, where the moderator has a guide of questions for organizing a discussion that are answered in a timely manner [25]. We intended to build a joint and open conversation from the discursive confrontation between the participants [23]. The discussion group provided a presentation and guided discussion about barriers and facilitators of implementing a risk-based breast cancer screening program from the perspective of health professionals. The list of open-ended questions, presented in S1 Table, was developed from a literature review. Each discussion group had a moderator with extensive experience in qualitative research (ACC or NCB) and an observer from the research team (MR or MC).
Four discussion groups were held, two in Lleida and two in Barcelona. Discussion groups were held at the Medical School of the University of Lleida and the headquarters of the Catalan Institute of Health in Barcelona. The team previously discussed and agreed on how to conduct the groups. Before starting the discussion group, the objectives of the study were presented and the concepts of personalized screening and shared decision-making were introduced. A brief summary of a proof of concept study about the feasibility and acceptability of risk-based screening, carried out by the DECIDO team, was presented [11]. Informed consent and permission for audio recording of the session were obtained. Then, the discussion was started by the open questions through the guided discussion (S1 Table). The duration of the discussion was approximately 90 minutes. At the end of the group discussion ACC and NCB took notes. The sessions were audiotaped and transcribed verbatim by two transcriptionists, not members of the study team, and later reviewed by ACC and NCB. The research team decided to end the data collection when the information obtained was considered sufficient to answer the research questions [26]. Transcripts of discussion groups were not returned to the participants for comments.
Data analysis and interpretation
The textual corpus made up of the discussion group transcripts was analyzed by two researchers who performed a descriptive-interpretive thematic analysis [27]. The entire research team participated in a session to interpret the results.
The transcript analysis consisted of a) reading and rereading the transcripts; b) reviewing the field work notes and writing the first intuitions and pre-analytic thoughts; c) coding texts and extracting categories in a mixed way (inductive and deductive) considering the objectives of the study and Légaré’s taxonomy that categorizes barriers and facilitators as knowledge, attitudes and behaviors [28]; d) reconstructing the categories by performing a work of contrast with the texts and between the texts, concept maps and analysis of co-occurrences between the discursive profiles and the barriers and facilitators identified in the analysis, in order to achieve an explanatory framework of the meanings according to the different discourses. The analysis was carried out with the support of the Atlas-ti version 8.4 software [29].
Quality and rigor
To ensure the validity of the study, different strategies were taken into account: discussion of the suitability of the strategy to generate information, theoretical sampling to ensure discursive variability, conducting four discussion groups to reach information saturation, and triangulation in the analysis and discussion of the results by the entire research team [30].
The participation of the entire research team fostered reflexivity throughout the development of the study, by discussing prior knowledge and experiences and explicitly describing preconceptions that were considered in both the design and interpretation of the results [23].
Ethical aspects
The Drug Research Ethics Committee of the University Hospital Arnau de Vilanova at Lleida approved the study. All the informants, after having been informed of the objective of the study and the involved institutions, participated voluntarily. The sessions were recorded guaranteeing the anonymity and confidentiality of the data. Thus, the identities of the participants and the references that could identify them were anonymized.
Results
Fig 1 summarizes the visions of healthcare professionals on the implementation of risk-based screening for breast cancer. Barriers and facilitators have been grouped in four categories, related to 1) women; 2) health professionals; 3) the risk-based screening program; and 4) the health system. Some organizational proposals about how to implement a risk-based program are listed at the bottom of Fig 1.
Barriers and facilitators, related to women, for participating in personalized risk-based breast cancer screening
According to the informants perspective, the main barrier for women to participating in risk-based screening would be resistance to having fewer mammograms, for women at low-risk (Table 2). More than 30 years of promotional efforts to encourage participation in screening programs and intense media coverage of breast cancer have raised awareness of the disease. This can make it difficult to understand that a reduction in the number of mammograms can be beneficial, which may cause some women to reject the personalized screening recommendations and opt for using the private health system to ensure having annual or biennial mammographic exams.
Facilitators that could reduce the impact of this barrier focus on the benefits of a risk-based program compared to “one size fits all.” A risk-based program would entail a) an initial risk assessment and a potential start of screening at a younger age, something that is not possible now, within the public health system, but many women would like, and b) more frequent exams and preventive measures for high-risk women. The participants expressed the need to inform women adequately in order to increase acceptance and participation. They also considered a facilitator the proximity and the relationship of trust that women have with their primary care professionals (PCP). An active role for PCPs could involve informing and inviting women to participate in the program. Participants also mentioned the growing interest and proactivity of women in decision-making related to their health, which may be a consequence of the strong diffusion of breast cancer burden through mass media. This has raised awareness among women and has increased their interest in early detection of the disease.
Barriers and facilitators associated with health professionals, for being involved in personalized risk-based breast cancer screening
With regard to barriers, the informants mentioned that even though there is evidence in favor of risk-based screening, there would be resistance to shifting from a model that has been in use for more than 30 years to a new one that requires significant changes in current practice and policy (Table 3).
Professionals were aware of the difficulty involved in communicating risk to women or developing shared decision-making so that women could decide whether or not to participate in screening. Thus, professionals of diverse profiles stated that, despite the fact that the estimated breast cancer risk can be considered an objective measurement, there are multiple aspects that a woman will consider, such as individual experiences and their environment. Participants called attention to the fact that most professionals do not have training in these subjects, turning this circumstance into a barrier.
Another barrier raised by the informants is the differing views on breast cancer screening recommendations among health professionals, which could also be the case if personalized screening was implemented. That would result in women not being offered the established recommendations and could cause confusion.
Participants raised some facilitators to mitigate the impact of the previously identified barriers. Among them, provision of systematic training over time to professionals involved in the program, especially those working in primary care. They also highlighted the importance of coordination of the different levels of care (primary care, current screening program, hospital services) and receiving feedback and information on the program outcomes. An information system shared by all the actors and an adequate allocation of human and economic resources were also considered important facilitators. Finally, the informants suggested building on the learnings and experiences of other existing screening programs, such as cervical or colon cancer, in which PCP or even pharmacists have participated.
Barriers and facilitators related to implementing a risk-based screening program
The informants considered the uncertainty related to the lack of conclusive evidence on the effectiveness of risk-based screening and insufficient accuracy of risk measures to be an important barrier (Table 4). Some participants emphasized that more research is needed before implementing risk-based screening. This barrier may contribute to resistance to change, and possibly both barriers feedback mutually to one another. A second barrier, signaled mostly by PCP, arose in relation to the hypothetical scenario that the program burden fell mainly on them. The informants, in general, agreed that primary care should be the gateway to the screening program and also the setting where risk is communicated and recommendations are made. However, PCP expressed concern and some even rejection, due to lack of time and job instability, unless relevant organizational changes were made. Finally, the participants noted the high costs of measuring risks for a large number of women in the initial phase of program implementation.
On facilitators, informants stated the above mentioned benefits plus increasing accuracy in risk quantification, implementing shared decision-making, and allocating more screening resources to the high-risk group for a higher efficiency. These seemed like strong arguments for informants to justify the need to move toward risk-based screening.
Health systems barriers and facilitators, for implementing personalized risk-based breast cancer screening
Since the potential implementation of risk-based breast cancer screening will require extensive changes in current practice, the informants expressed that the change would not be possible without the commitment of health policy makers and an adequate budget allocation for program implementation and maintenance (Table 5). In fact, the extra amount of financial and human resources for the implementation of the program is an economic barrier. Another barrier mentioned was the set of limitations of the current information system regarding the real-time integration of screening data and clinical records, as well as the access of all health professionals involved in the screening program.
Considering the identified barriers, the facilitators that would reduce their impact are, on the one hand, the commitment of health policy makers to implement the program with its corresponding budget and human resources allocation, and on the other hand, a friendly and comprehensive information system that connects the different actors involved and facilitates communication between them.
Organizational proposals for the implementation of a personalized breast cancer screening program
The open questions were: How should the personalized screening program be organized? Who would do the different functions? There was no consensus on how to organize a personalized screening program and there was controversy on the role of professionals that work at the current screening programs and in primary care. However, the section has limitations as it was not deepened enough due to lack of time. The moderators were strict on the planned time in order to prevent the informants fatigue.
First, the informants discussed who should and how to make the first contact for informing and inviting women to the program (Table 6). They proposed that women be invited by primary care physicians and nurses in clinical encounters or by a breast cancer screening program by mail or phone. They also suggested that more time and adapted decision aids would be necessary for women with a low educational level and/or language barriers.
Second, they debated risk measurement and risk-based screening recommendations. The informants emphasized the importance of communicating individual risk in plain language, to improve understanding and decision-making. They proposed that primary care physicians and nurses communicate risk and make screening recommendations, given that most women have a trusting relationship with them. High risk should be communicated by physicians whereas moderate or low risk could be communicated by nurses. In fact, there was a general consensus on the role that nurses could play in personalized risk-based screening.
Third, the discussion dealt with who and how to perform the individual follow-up. Informants considered that, with a good IT system, it could be done either by the screening program or PCP. If primary care managed the follow-up, additional resources for this health level would be needed. For high-risk women, breast specialists should inform and follow them up.
Finally, the participants indicated the need for educational programs targeted to health professionals and women, including mass media campaigns for large audiences. They expressed their confidence in these interventions to reduce resistance to change and facilitate the acceptance and participation of women.
Discussion
This study explored, using a qualitative approach, the barriers and facilitators to the implementation of a risk-based breast cancer screening program as well as some organizational proposals for its implementation, from the point of view of a diverse group of healthcare professionals working in the SpNHS.
Barriers and facilitators related to women
The participants expressed concern about a reduction in the number of screening mammograms for low-risk women, a finding consistent with other studies. In the UK, Meisel et al. [31] explored attitudes towards modifying frequency of mammography screening based on genetic risk. They found that although 65.5% of the women supported the idea of varying the screening frequency, only 58.8% were willing to reduce mammographic exams if they were found at lower risk. Instead, 85.4% were willing to increase the frequency if at higher risk. A study of European women’s perceptions found that more intensive screening for women with above average risk was generally welcomed, but screening recommendations for the other risk categories were met with skepticism [32]. Another study, which assigned participants to one of four hypothetical breast cancer risk scenarios (low, average, moderate, or high), with subsequent screening and prevention recommendations, found again that women’s level of acceptance depended on their assigned risk category [33]. Only 13.1% of the women assigned to the low risk scenario found a 4-year screening interval acceptable, with 27.4% of these women opting for supplemental mammography screening outside of the national screening program.
Several authors have discussed the reasons for this barrier. In a qualitative study with North American women [34], personal acceptance of risk-based screening was mixed. While some believed that risk-based screening could reduce harms, others thought screening less often might result in missing a dangerous diagnosis, and many expressed concerns about the feasibility of risk-based screening and questioned whether breast cancer risk estimates could be accurate—either because women did not remember risk factors or were confused about how risk would be calculated. Some women also suspected that risk-based screening was motivated by a desire to save money rather than reduce harms. Dutch women described the role of perceived risk in the acceptability of personalized exams. They believe that if their estimated risk does not correspond to their perceived risk, they will be less likely to accept the screening advice [32].
In the UK, McWilliams et al. [35] explored healthcare policy decision-making stakeholders’ views on a low-risk breast screening pathway. Participants identified individual beliefs about risk and knowledge of breast cancer and screening as key factors that impact women’s responses to low-risk stratification. Among uncertainties that need to be resolved before implementation, they signaled accurate identification of low-risk women, gaining women’s acceptance, and having evidence of lack of harm. In a study with professionals, Rainey et al. [36] suggested that women will question their personal risk information and their assigned pathway of care (screening frequency or biomedical prevention) and they expected an increase in opportunistic screening, particularly from women at low risk. As some authors suggest, there is a need for a) educating women on the benefits and harms of risk-based screening and prevention, to facilitate acceptability and informed decision-making [16], and b) developing effective communication materials to minimize resistance to screening reduction for those at lower risk [31]. A research synthesis on factors associated with the acceptability of human papillomavirus HPV testing for cervical cancer found that strategies that increase women’s knowledge and efforts to increase health care providers’ awareness might also increase acceptability of wider screening intervals [37].
Unlike other studies [16, 38, 39], participants in our study did not consider the potential discrimination that women may suffer from the use of their individual risk data for employment issues or health insurance discrimination to be a barrier. The universal health coverage of the SpNHS with free access to health care may explain this result.
Our study participants identified several facilitators, related to women, for participating in a risk-based screening program. First, and consistent with Puzhko et al. [14], they considered that a risk-based screening program is beneficial for women, especially for those at a higher-than-population average level of risk and for those who have objective (e.g. family history) or subjective reasons to be concerned. Thus, for some women, knowing the risk would be reassuring and would reduce anxiety, which would facilitate their participation in the program and in shared decision-making. This, in turn, could reduce the inappropriate use of opportunistic screening [14] and increase the attendance at subsequent screening exams [40]. However, Puzhko et al. emphasized that women may not truly benefit from risk-based screening unless their health professionals understand the risks and benefits of screening and can interpret the results adequately [14]. Second, several studies suggest that a healthcare professional (i.e. primary care physician or nurse) should be involved in the invitation, risk stratification, risk communication and follow-up of women in a risk-based screening program [41]. As participants in our study mentioned, a relationship of trust with the health professional [14], as well as relying on the experience of risk communication acquired from other health conditions in preventive medicine [42], would facilitate a personalized approach [43]. Third, the increasing participation of women in decisions that affect their health represents an incentive to participate in a personalized screening program. Our results agree with those of other studies reporting risk knowledge as an opportunity to have a proactive attitude [32], by being aware of the impact that breast cancer could have on women’s lives [44]. Likewise, health professionals value the proactive approach to shared decision-making, as women can take control of some of their risk factors, take steps to reduce them and try to avoid breast cancer, all findings that are in line with women finding risk based screening acceptable [36].
Barriers and facilitators associated with health professionals
Our results showed greater knowledge and a more positive attitude on the part of professionals towards personalized screening when compared to a previous study conducted in 2016 [45]. In line with this view, Puzhko et al. [14] reported that health professionals acknowledged the substantial benefits of the risk-based program. However, our discussion groups, as others have reported, expressed concern about the critical view of health professionals as a result of the uncertainty in the evidence on the safety and cost-effectiveness of the risk-based approach [35], and the resistance to change of professionals, reluctant to modify their established routine.
According to our study, training is essential for successfully implementing a program. Our results are consistent with other studies that identified professionals’ concerns about lack of competence to communicate risk and subsequent screening recommendations effectively [14–16]. These authors suggested implementing educational programs that address these knowledge deficits, for each woman and risk category. Our study participants discussed how to implement training in the primary care setting, taking into account the diversity of settings (urban versus rural) and the mobility of health professionals. They also proposed strategies addressed at improving their training.
The increase in complexity of risk-based screening will require collaboration at multiple levels of care. Although Rainey et al. [16] suggested that the countries that already have a population-based screening program may experience fewer difficulties implementing risk-based screening, in our country it will be necessary to improve collaboration between primary care, population-based screening programs, and also specialized care for high-risk women.
Moreover, all discussion groups considered it essential that Health Authorities incorporate the personalized screening program into a health policy goal with adequate allocation of human and economic resources. Only then, dissemination of the personalized program, women’s risk assessment, risk communication and levels of care coordination can be guaranteed. Without a health policy commitment and budget allocation, professionals perceive that there will be resistance to change. These results are concordant with previous studies. Whereas Esquivel-Sada et al. [15] pointed out the lack of confidence of professionals in the public health system to provide an adequate allocation of human and economic resources, McWilliams et al. [35] stated that risk stratification is potentially profitable and consequently, further study of the cost-effectiveness around personalized risk-based programs will be necessary.
Barriers and facilitators related to implementing a risk-based screening program
As a barrier, participants mentioned the uncertainty around the effectiveness of risk-based screening. Their doubts relied on the lack of evidence of benefits from a risk-based model compared to the current age-based program. To advance in this area, the European Collaborative on Personalized Early Detection and Prevention of Breast Cancer (ENVISION) network [46] brings together several international research consortia, from 19 countries, working on different aspects of personalized early detection and prevention of breast cancer. The network highlights how risk stratification has improved with the use of comprehensive models incorporating genetic and epidemiological risk factors and mammographic breast density, which have shown an excellent calibration for the European, Hispanic and African-American population [7]. Recent studies report that PRS models derived from women with European ancestry for breast cancer risk generalized well for women with European and Latina ancestries and to a lesser degree to women with African ancestry where further studies with larger sample size are needed [47, 48]. Although research is still ongoing to assess the clinical utility of PRS for population screening programs, Yanes et al. point out that polygenic testing is already being implemented in specialist familial cancer clinics to provide additional information for women with family history and uninformative genetic test results [7].
On the one hand, despite the doubts expressed by professionals as a result of the lack of evidence of the effectiveness of risk-based screening, they recognized that it would provide greater accuracy in measuring individual risk. Thus, health professionals seemed to have a good predisposition towards risk-based screening. However, as Petrova et al. [17] propose it is essential that they know the results of the research and understand the statistics of early detection in order to avoid misleading and potentially harmful recommendations. Additionally, the general population should be exposed to media campaigns that inform them about the pros and cons of a risk-based program [42].
On the other hand, the barriers that most concern our professionals come from the organizational system, in particular the fear of an increase in the workload for PCP and the lack of economic resources. These impressions are based on the fact that the implementation of the personalized program involves a more complex organization and more agents involved [16] such as patients, providers, facilities, health care systems, and regional and national policy-making organizations [43, 49]. In addition, our study participants assume that there is an increase in costs derived from the genetic information in risk measurement and the time spent by professionals in communicating risk and shared decision-making. These costs would be largely offset by the increased effectiveness and efficiency of risk-based screening [3, 9, 50–53], as well as by greater well-being provided by fluid communication between women and professionals regarding risk monitoring [18, 54]. A reduction of professionals’ workloads could be achieved if women participated more actively (e.g. self-collection of saliva samples and self-completion of data) under a nurses’ supervision [43, 55]. In this line, our health professionals considered that their involvement would be facilitated if the current screening program were restructured, incorporating an adequate relationship with the health system and coordination with other levels of care.
Health systems barriers and facilitators, for implementing personalized risk-based breast cancer screening
Implementing personalized screening will require substantial organizational changes to the current healthcare model [16, 50]. Despite the need for health interventions to evolve when new evidence appears, substantial efforts will be required to abandon a screening program in use for more than 30 years and to introduce a new one that still presents uncertainties. Health professionals are used to changes in clinical protocols as new studies support them, but they are reluctant to face organizational changes whose dimension and effects they do not control. Rainey et al. suggest that countries that already have an organized, population-based screening program in place may experience fewer difficulties implementing a risk-based screening program [16], and professionals in our study consider that the experience with other risk-based interventions would reduce reluctance to change. Likewise, Bellhouse et al. [41] conclude that the integration of risk assessment and management tools into practice software or access to web-based applications would facilitate this change, as happened for cardiovascular risk assessment and management.
Professionals participating in our study indicated that risk-based screening would only be possible if the current IT tools were updated. Similarly, Puzhko et al. [14] emphasize the importance of integrating the risk calculator into the electronic medical record and Yi et al. [18] emphasize the need for developing electronic decision aids to enhance risk communication. Likewise, information systems must integrate electronic medical record and screening information in an easy and accessible way for all stakeholders. This would facilitate monitoring the risk of breast cancer overtime, accounting for potential changes in risk factors [16, 18].
Decision aids are needed to allow participatory health models such as shared decision-making [14]. Decision aids allow communication of the risks and benefits of screening in a balanced way, and work together to reach a decision about adherence to the program, based on clinical evidence and the patient’s informed preferences [16, 54]. Studies show that women participating in shared decision-making have a greater commitment to health decisions, since they feel more responsible for their actions. This would allow women to reduce barriers related to anxiety, decisional conflict, and to prefer more beneficial and less invasive options [28, 56, 57], that is, to better manage the expectations of a risk-based program [16]. We included shared decision-making in our discussion group guide as an innovative strategy in screening. We were interested in knowing the views of health professionals about their implementation, considering that some barriers that have been reported in the literature have their origin in professionals’ resistance to change [58].
Our study participants argued that the costs of updating the IT system and implementing shared decision-making, together with the aforementioned costs of genetic determinations and human resources for risk communication, will lead to budget increases. However, there is evidence in favor of risk-based screening being more efficient [3, 51–53], providing more resources to women at higher risk and fewer to those at lower risk. Harkness et al. [52] showed that compared to age-based screening, not offering screening to women in the lowest tertile of risk could improve effectiveness, cost-effectiveness, and decrease overdiagnosis, whilst resulting in a small excess of breast cancer deaths. However, Vilaprinyo et al. [3], in a modeling study, found that risk-based screening could reduce breast cancer mortality. Therefore, an economic evaluation of risk-based screening at the local level should be performed.
Given that both structural and organizational changes as well as an increase in economic resources will be required, the implementation of risk-based screening cannot be carried out without strong conviction from policy makers, health care providers, and regional and national authorities, to develop patient-centered medicine [49, 59, 60]. Therefore, declaration of intent is not enough, it is necessary to develop specific strategies for the application of risk-based screening [59, 60].
Finally, the evidence also supports the consensus reached in our discussion groups; primary care should be the gateway to the new program [14, 16, 41, 50]. However, among the professionals in our study, there was no agreement on which professionals should be in charge of risk communication and risk counseling, although nurses and family doctors were the most cited. In an European study [59], Dutch and Swedish women agreed that below average and average risk results can be relayed in a letter, while above average risk feedback should be done through a telephone or face-to-face consultation with a family doctor or a specialized nurse, with expert knowledge in the field. British women at below average to average risk were also satisfied receiving information on their risk in a letter, but women at moderate or high risk recommended the development of special women’s clinics operated by specialized nurses, radiographers, radiologists, and gynecologists, integrating breast and cervical cancer screening. British women also signaled a need to educate professionals on all aspects of risk-based screening and prevention in order to prevent the provision of conflicting information, and also a need for pathways and protocols to standardize interaction between primary and secondary care providers to avoid individual variation [59]. In line with this result, Culver et al. [61] and Bellhouse et al. [41] found that primary care physicians require more training in risk assessment and risk communication before facing complex scenarios in high-risk women. This evidence preserves some elements of the current screening program, such as the delivery of information by letter and telephone follow-up. However major changes are needed in several elements, such as information systems or risk communication protocols, for the implementation of a personalized risk-based screening.
Strengths and limitations
To our knowledge, this is the first study in Spain that intended to obtain a description of barriers and facilitators as well as a global assessment of the difficulties in implementing a risk-based breast cancer screening program, all from the perspective and experiences of health professionals working in the NHS. The views of the different stakeholders involved in the screening, diagnosis and treatment of breast cancer have provided valuable information to pave the way for the implementation of personalized breast cancer screening.
Our study has some limitations. First, collecting data from discussion groups may have produced a social desirability bias in the discourse [62]. However, the triangulation of information through observations and data with the participation of different professionals, in addition to contributing to a better understanding of the phenomenon, relativized this possible bias by corroborating the convergences, divergences and complementarity of data obtained in the discussion groups [63]. In addition, reflexivity was maintained during the research process with the participation of the entire research team and the use of information from field notes [64]. Second, the four discussion groups had a heterogenous profile that limited the saturation of specific perspectives or views. However, the research team considered that for most of the topics discussed the information obtained was sufficient to answer the research questions. Third, not all our findings may be generalizable to other health systems. Nonetheless, many of them may be transferable to health systems with population-based screening programs. Fourth, participants’ perspectives on perceived barriers and facilitators may have been influenced by the way the research team presented the most relevant changes in risk-based screening, compared to the current screening: risk measurement and communication and the corresponding screening recommendations.
Conclusions
Despite the barriers and limitations identified, study participants were supportive of the implementation of risk-based screening. They highlighted the benefits of risk-based screening, especially for women at high risk of breast cancer and those under 50 years of age. They stressed that allocating more resources to these women is not only beneficial for them, but also translates into a greater efficiency of the risk-based program compared to the current one.
However, the implementation of risk-based screening would pose a great challenge for the SpNHS due to organizational and administrative transformations, and also due to the need for commitment and political will. Other challenges come from the need for coordination between the different levels of care; improvements to information systems; education of health professionals on personalized screening, risk communication and shared decision-making; and dissemination of information through public campaigns, guaranteeing that it is accessible to the entire population. As facilitators of the implementation, the participants highlighted the experience of PCP in other screening programs; the relationship of trust and the closeness of these professionals with women; the awareness of women about breast cancer and early detection; and the increasingly proactive role of women in their health care.
More studies are needed that expand knowledge and understanding about the views of health professionals, health system administrators and those responsible for formulating public health policies, on the viability of risk-based screening for breast cancer and its future implementation; and also the acceptance of this strategy by women. Future studies should assess the efficiency and feasibility of implementing risk-based breast cancer screening in clinical practice in the SpNHS.
Acknowledgments
We thank the health professionals who generously and actively participated in the discussion groups. We also thank JP Glutting for reviewing and editing the manuscript.
The DECIDO Group
Members of the DECIDO Study Group: Pau Balaguer-Llaquet; Iván-David Benítez; Alexandra Bertran; Àngels Cardona; Cristina Cazorla-Sánchez; Núria Codern; Inés Cruz-Esteve; Carles Forné-Izquierdo; Marta Hernández-Andreu; María José Hernández-Leal; Edelmir Iglesias; Gisela Galindo-Ortego; Celmira Laza-Vásquez; Montserrat Llorens-Gabandé; Montserrat Martínez-Alonso; Hèctor Perpiñán; Anna Pons-Rodríguez; Mercè Reñé-Reñé; Montserrat Rué; Isabel Sánchez-López; Jordi Vilaplana-Mayoral.
References
- 1. Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2021;n/a. https://doi.org/10.3322/caac.21660
- 2. Marmot MG, Altman DG, Cameron D, Dewar J, Thompson SG, Wilcox M. The benefits and harms of breast cancer screening: An independent review. Br J Cancer. 2013;108: 2205–2240. pmid:23744281
- 3. Vilaprinyo E, Forné C, Carles M, Sala M, Pla R, Castells X, et al. Cost-effectiveness and harm-benefit analyses of risk-based screening strategies for breast cancer. PLoS One. 2014;9: e86858. pmid:24498285
- 4. Bitencourt AGV, Saccarelli CR, Kuhl C, Morris EA. Breast cancer screening in average-risk women: Towards personalized screening. Br J Radiol. 2019;92. pmid:31538501
- 5. Shieh Y, Hu D, Ma L, Huntsman S, Gard CC, Leung JWT, et al. Breast cancer risk prediction using a clinical risk model and polygenic risk score. Breast Cancer Res Treat. 2016;159: 513–525. pmid:27565998
- 6. Vachon CM, Scott CG, Tamimi RM, Thompson DJ, Fasching PA, Stone J, et al. Joint association of mammographic density adjusted for age and body mass index and polygenic risk score with breast cancer risk. Breast Cancer Res. 2019;21: 1–10. pmid:30611295
- 7. Yanes T, Young MA, Meiser B, James PA. Clinical applications of polygenic breast cancer risk: A critical review and perspectives of an emerging field. Breast Cancer Res. 2020;22: 1–10. pmid:32066492
- 8. Evans DGR, Harkness EF, Brentnall AR, Veen EM van, Astley SM, Byers H, et al. Breast cancer pathology and stage are better predicted by risk stratification models that include mammographic density and common genetic variants. Breast Cancer Res Treat. 2019;176: 141–148. pmid:30941651
- 9. Román M, Sala M, Domingo L, Posso M, Louro J, Castells X. Personalized breast cancer screening strategies: A systematic review and quality assessment. PloS One. 2019;14: e0226352. pmid:31841563
- 10. French DP, Astley S, Brentnall AR, Cuzick J, Dobrashian R, et al. What are the benefits and harms of risk stratified screening as part of the NHS breast screening Programme? Study protocol for a multi-site non-randomised comparison of BC-predict versus usual screening (NCT04359420). BMC Cancer. 2020;20: 1–14. pmid:32552763
- 11. Pons-Rodriguez A, Forné Izquierdo C, Vilaplana-Mayoral J, Cruz-Esteve I, Sánchez-López I, Reñé-Reñé M, et al. Feasibility and acceptability of personalised breast cancer screening (DECIDO study): Protocol of a single-arm proof-of-concept trial. BMJ Open. 2020;10. pmid:33361170
- 12. Coulter A, Jenkinson C. European patients’ views on the responsiveness of health systems and healthcare providers. Eur J Public Health. 2005;15: 355–360. pmid:15975955
- 13. Elwyn G, Frosch D, Volandes AE, Edwards A, Montori VM. Investing in deliberation: A definition and classification of decision support interventions for people facing difficult health decisions. Med Decis Making. 2010;30: 701–11. pmid:21088131
- 14. Puzhko S, Gagnon J, Simard J, Knoppers BM, Siedlikowski S, Bartlett G. Health professionals’ perspectives on breast cancer risk stratification: Understanding evaluation of risk versus screening for disease. Public Health Rev. 2019;40: 1–19. pmid:30858991
- 15. Esquivel-Sada D, Lévesque E, Hagan J, Knoppers BM, Simard J. Envisioning implementation of a personalized approach in breast cancer screening programs: Stakeholder perspectives. Healthcare Policy. 2019;15: 39–54. pmid:32077844
- 16. Rainey L, Waal D van der, Jervaeus A, Wengström Y, Evans DG, Donnelly LS, et al. Are we ready for the challenge of implementing risk-based breast cancer screening and primary prevention? Breast. 2018;39: 24–32. pmid:29529454
- 17. Petrova D, Mas G, Navarrete G, Rodriguez TT, Ortiz PJ, Garcia-Retamero R. Cancer screening risk literacy of physicians in training: An experimental study. PLoS One. 2019;14: 1–17. pmid:31269051
- 18. Yi H, Xiao T, Thomas PS, Aguirre AN, Smalletz C, Dimond J, et al. Barriers and facilitators to patient-provider communication when discussing breast cancer risk to aid in the development of decision support tools. American Medical Informatics Association, Annual Symposium proceedings AMIA Symposium. 2015;2015: 1352–1360. pmid:26958276
- 19. Kukafka R, Fang J, Vanegas A, Silverman T, Crew KD. Pilot study of decision support tools on breast cancer chemoprevention for high-risk women and healthcare providers in the primary care setting. BMC Med Inform Decis Mak. 2018;18: 1–12. pmid:29301576
- 20. Ascunce N, Salas D, Zubizarreta R, Almazán R, Ibáñez J, Ederra M, et al. Cancer screening in Spain. Ann Oncol. 2010;21: 43–51. pmid:20427360
- 21. Marzo-Castillejo M, Vela-Vallespín C, Bellas-Beceiro B, Bartolomé-Moreno C, Ginés-Díaz Y, Melús-Palazón E. Grupos de Expertos de Cáncer del PAPPS. Recomendaciones de prevención del cáncer. Actualización PAPPS 2020. Aten Primaria. 2020;52: 44–69.
- 22.
Denzin N, YS L. The SAGE handbook of qualitative research. Thousand Oaks: SAGE Publications, Inc; 2011.
- 23.
Zarco J, Ramasco M, Pedraz A, A P. Investigación cualitativa en salud (Cuadernos Metodológicos 58). Editorial CIS (Centro de Investigaciones Sociológicas); 2019.
- 24.
Patton MQ. Qualitative research and evaluation methods: Integrating theory and practice. Thousand Oaks: SAGE Publications, Inc; 2015.
- 25. Krueger RA. Analyzing focus group interviews. Journal of Wound, Ostomy and Continence Nursing. 2006;33: 478–481. pmid:17133134
- 26. Minayo M. Amostragem e saturação em pesquisa qualitativa: Consensos e controvérsias. Revista Pesquisa Qualitativa. 2017; 5: 1–12.
- 27. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3: 77–101. pmid:32100154
- 28. Légaré F, Ratté S, Gravel K, Graham ID. Barriers and facilitators to implementing shared decision-making in clinical practice: Update of a systematic review of health professionals’ perceptions. Patient Educ Couns. 2008;73: 526–535. pmid:18752915
- 29.
ATLAS.ti. Qualitative data analysis, version 8.4 [Internet]. Berlin, Germany: ATLAS.ti Scientific Software Development GmbH; 2019. Available: https://atlasti.com/
- 30. Calderón C. Assessing the quality of qualitative health research: Criteria, process and writing. Forum Qual Soc Res. 2009;10.
- 31. Meisel SF, Pashayan N, Rahman B, Side L, Fraser L, Gessler S, et al. Adjusting the frequency of mammography screening on the basis of genetic risk: Attitudes among women in the UK. Breast. 2015;24: 237–241. pmid:25708717
- 32. Rainey L, Jervaeus A, Donnelly LS, Evans D, Hammarström M, Hall P, et al. Women’s perceptions of personalized risk-based breast cancer screening and prevention: An international focus group study. Psychooncology. 2019;28: 1056–1062. pmid:30848018
- 33. Rainey L, Van Der Waal D, Broeders MJM. Dutch women’s intended participation in a risk-based breast cancer screening and prevention programme: A survey study identifying preferences, facilitators and barriers. BMC Cancer. 2020;20: 1–14. pmid:33023516
- 34. He X, Schifferdecker KE, Ozanne EM, Tosteson ANA, Woloshin S, Schwartz LM. How do women view risk-based mammography screening? A qualitative study. J Gen Intern Med. 2018;33: 1905–1912. pmid:30066118
- 35. McWilliams L, Woof VG, Donnelly LS, Howell A, Evans DG, French DP. Risk stratified breast cancer screening: UK healthcare policy decision-making stakeholders’ views on a low-risk breast screening pathway. BMC Cancer. 2020;20: 1–11. pmid:32698780
- 36. Rainey L, van der Waal D, Donnelly LS, Evans D, Wengström Y, Broeders M. Women’s decision-making regarding risk-stratified breast cancer screening and prevention from the perspective of international healthcare professionals. PLoS One. 2018;13: e0197772. pmid:29856760
- 37. Tatar O, Thompson E, Naz A, Perez S, Shapiro GK, Wade K, et al. Factors associated with human papillomavirus (HPV) test acceptability in primary screening for cervical cancer: A mixed methods research synthesis. Prev Med. 2018;116: 40–50. pmid:30172799
- 38. Hall AE, Chowdhury S, Hallowell N, Pashayan N, Dent T, Pharoah P, et al. Implementing risk-stratified screening for common cancers: A review of potential ethical, legal and social issues. J Public Health. 2014;36: 285–291. pmid:23986542
- 39. Lévesque E, Kirby E, Bolt I, Knoppers BM, De Beaufort I, Pashayan N, et al. Ethical, legal, and regulatory issues for the implementation of omics-based risk prediction of women’s cancer: Points to consider. Public Health Genomics. 2019;21: 37–44. pmid:30223261
- 40. Evans DGR, Donnelly LS, Harkness EF, Astley SM, Stavrinos P, Dawe S, et al. Breast cancer risk feedback to women in the UK NHS breast screening population. Br J Cancer. 2016;114: 1045–1052. pmid:27022688
- 41. Bellhouse S, Hawkes RE, Howell SJ, Gorman L, French DP. Breast cancer risk assessment and primary prevention advice in primary care: A systematic review of provider attitudes and routine behaviours. Cancers. 2021;13: 1–23. pmid:34439302
- 42. Lévesque E, Hagan J, Knoppers BM, Simard J. Organizational challenges to equity in the delivery of services within a new personalized risk-based approach to breast cancer screening. New Genet Soc. 2019;38: 38–59.
- 43. Silver E, Wenger N, Xie Z, Elashoff D, Lee K, Madlensky L, et al. Implementing a population-based breast cancer risk assessment program. Clin Breast Cancer. 2019;19: 246–253.e2. pmid:31072694
- 44. Woof VG, Ruane H, French DP, Ulph F, Qureshi N, Khan N, et al. The introduction of risk stratified screening into the NHS breast screening Programme: Views from British-Pakistani women. BMC Cancer. 2020;20: 1–9. pmid:32434564
- 45. Toledo-Chávarri A, Rué M, Codern-Bové N, Carles-Lavila M, Perestelo-Pérez L, Pérez-Lacasta M, et al. A qualitative study on a decision aid for breast cancer screening: Views from women and health professionals. Eur J Cancer Care. 2017; 1–11. pmid:28145105
- 46. Pashayan N, Antoniou AC, Ivanus U, Esserman LJ, Easton DF, French D, et al. Personalized early detection and prevention of breast cancer: ENVISION consensus statement. Nat Rev Clin Oncol. 2020;17: 687–705. pmid:32555420
- 47. Liu C, Zeinomar N, Chung WK, Kiryluk K, Gharavi AG, Hripcsak G, et al. Generalizability of polygenic risk scores for breast cancer among women with European, African, and Latinx ancestry. JAMA Netw Open. 2021;4: 1–12. pmid:34347061
- 48. Du Z, Gao G, Adedokun B, Ahearn T, Lunetta KL, Zirpoli G, et al. Evaluating polygenic risk scores for breast cancer in women of African ancestry. J Natl Cancer Inst. 2021;113: 1168–1176. pmid:33769540
- 49. Onega T, Beaber EF, Sprague BL, Barlow WE, Haas JS, Tosteson AN, et al. Breast cancer screening in an era of personalized regimens: A conceptual model and National Cancer Institute initiative for risk-based and preference-based approaches at a population level. Cancer. 2014;120: 2955–2964. pmid:24830599
- 50. Dent T, Jbilou J, Rafi I, Segnan N, Törnberg S, Chowdhury S, et al. Stratified cancer screening: The practicalities of implementation. Public Health Genomics. 2013;16: 94–99. pmid:23363703
- 51. Sun L, Yang L. Cost effectiveness of urban breast cancer screening programme in China. Value in Health. 2017;20: A583–A584.
- 52. Harkness EF, Astley SM, Evans DG. Risk-based breast cancer screening strategies in women. Best Pract Res Clin Obstet Gynaecol. 2020;65: 3–17. pmid:31848103
- 53. Khan SA, Hernandez-Villafuerte KV, Muchadeyi MT, Schlander M. Cost-effectiveness of risk-based breast cancer screening: A systematic review. Int J Cancer. 2021; pmid:33844853
- 54. Ozanne EM, Wittenberg E, Garber JE, Weeks JC. Breast cancer prevention: Patient decision making and risk communication in the high risk setting. Breast J. 2010;16: 38–47. https://doi.org/10.1111/j.1524-4741.2009.00857.x pmid:19889168
- 55. Hagan J, Lévesque E, Knoppers BM. Influence of organizational factors on implementation of a personalized approach to breast cancer screening. Sante Publique. 2016;28: 353–361. pmid:27531433
- 56. Bravo P, Contreras A, Perestelo-Pérez L, Pérez-Ramos J, Málaga G. Looking for a more participative healthcare: sharing medical decision making. Rev Peru Med Exp Salud Publica. 2013;30: 691–697. pmid:24448951
- 57. Légaré F, Adekpedjou R, Stacey D, Turcotte S, Kryworuchko J, Graham ID, et al. Interventions for increasing the use of shared decision making by healthcare professionals. Cochrane Database Syst Rev. 2018;7: CD006732. pmid:30025154
- 58. Hernández-Leal MJ, Pérez-Lacasta MJ, Feijoo-Cid M, Ramos-García V, Carles-Lavila M. Healthcare professionals’ behaviour regarding the implementation of shared decision-making in screening programmes: A systematic review. Patient Educ Couns. 2021; pmid:33581968
- 59. Rainey L, Van Der Waal D, Jervaeus A, Donnelly LS, Evans DG, Hammarström M, et al. European women’s perceptions of the implementation and organisation of risk-based breast cancer screening and prevention: A qualitative study. BMC Cancer. 2020;20: 1–11. pmid:32209062
- 60. Borgquist S, Hall P, Lipkus I, Garber JE. Towards prevention of breast cancer: What are the clinical challenges? Cancer Prev Res. 2018;11: 255–264. pmid:29661853
- 61. Culver JO, Bowen DJ, Reynolds SE, Pinsky LE, Press N, Burke W. Breast cancer risk communication: Assessment of primary care physicians by standardized patients. Genetics in Medicine. 2009;11: 735–741. pmid:19661809
- 62. Lincoln YS, Guba EG. But is it rigorous? Trustworthiness and authenticity in naturalistic evaluation. New Directions for Program Evaluation. 1986;1986: 73–84.
- 63. Morse JM. Critical analysis of strategies for determining rigor in qualitative inquiry. Qual Health Res. 2015;25: 1212–1222. pmid:26184336
- 64. Cypress BS. Rigor or reliability and validity in qualitative research: Perspectives, strategies, reconceptualization, and recommendations. Dimens Crit Care Nurs. 2017;36: 253–263. pmid:28570380