Abstract
Purpose Current clinical guidelines for genetic testing for Li-Fraumeni Syndrome (LFS) have many limitations, primarily the criteria don’t consider detailed personal and family history information and may miss many individuals with LFS. A personalized risk assessment tool, LFSPRO, was created to estimate a proband’s risk for LFS based on personal and family history information. The purpose of this study is to compare LFSPRO to existing clinical criteria to determine if LFSPRO can outperform these tools. Additionally, we gauged genetic counselors’ (GCs) experience using LFSPRO for their patients.
Methods Between December 2021 and March 2024, GCs identified patients concerning for LFS based on the patients’ personal and family history information. This information was entered into LFSPRO to predict the risk to have a pathogenic/pathogenic (LP/P) germline TP53 variant. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) was compared between LFSPRO and Chompret criteria. Select GCs were asked to fill out surveys regarding their experience using LFSPRO following their genetic counseling appointments.
Results LFSPRO’s sensitivity and specificity were 0.529 and 0.781 compared to Chompret’s respective 0.235 and 0.677. Additionally, LFSPRO had a positive predictive value (PPV) of 0.30 compared to Chompret’s 0.114. LFSPRO’s risk prediction was concordant with genetic testing results in 75% of probands. Eighty-one percent of GC surveys reported LFSPRO being concordant with the GC’s expectations and 75% would feel comfortable sharing the results with patients.
Conclusion LFSPRO showed improved sensitivity and specificity compared to Chompret criteria and GCs report a positive experience with LFSPRO. LFSPRO can be used to increase access to genetic testing for patients at risk for LFS and could help healthcare providers give more direct risk assessments regarding LFS testing and management for patients.
Introduction
Individuals with Li-Fraumeni syndrome (LFS) have an increased risk of developing cancer throughout their life. The most common cancers seen in LFS families are sarcomas, breast cancer, leukemia, brain tumors, melanoma, adrenal cortical carcinoma, among others1,2. LFS is caused by germline likely pathogenic/pathogenic variants (LP/P variants) in the TP53 tumor suppressor gene3,4. Individuals with these LP/P variants have a high lifetime risk of cancer, with an approximate 93% risk for women and 73% risk for men5.
The clinical management of families affected with TP53 LP/P variants remains a significant challenge and can create a clinical burden given these individuals may develop numerous cancers throughout their lifetime. Early cancer detection, as demonstrated by the 2011 comprehensive Toronto screening protocol, can dramatically improve cancer survival rates in children and adults6.
Additionally, it can be challenging to predict who will have a TP53 LP/P variant, because LFS overlaps with other hereditary cancer syndromes, and not all families with LFS meet established genetic testing criteria. Currently, risk prediction for LFS is based on four different clinical LFS classification schemes: Li-Fraumeni classic criteria, Birch criteria, Eeles Criteria, and Chompret criteria2,5,7,8,9,10. These schemes use combination of a probands cancer history, age of cancer onset, and family history of cancer in first and second degree relatives, to recommend genetic testing for those at risk for LFS. Although these schemes provide guidance to healthcare providers on which families are more suspicious for LFS, they do not provide a detailed risk assessment, which is often needed while providing genetic counseling to patients.
In cancer focused genetic counseling sessions, genetic counselors (GCs) often use risk prediction models to better define a patient’s lifetime cancer risk, determine eligibility for surveillance and chemo-preventative methods, and to provide probabilities of having certain hereditary cancer predisposition syndromes, such as hereditary breast and ovarian cancer syndrome and Lynch syndrome11. However, such risk models did not previously exist for LFS. Given the variable presentation of families with LFS, the number of cancers that overlap with LFS and other hereditary cancer syndromes, and the importance of identifying individuals with LFS to ensure they receive comprehensive cancer screening, a risk model was needed for GCs and other healthcare providers to better communicate risk to patients.
To address this gap in personalized risk prediction for LFS, an LFS risk prediction model has been created. Many steps are taken to move a risk prediction model from the development stage to the clinical tool stage (Figure 1). After a clinical problem has been identified, models are developed using initial research data sets. Since research data sets often differ from clinically ascertained data, particularly in the depth and breadth of information available, it is then important to train these models on clinical data. Once a model has been trained, it goes through a series of validations, both on internal and external validation sets. Following these validations, a series of research-based (RB) multi cohort and retrospective clinic-based (RCB) cohort validations are performed. Finally, as highlighted in this paper, a prospective clinical-based (PCB) cohort validation is completed. Each step in this process is of vital importance to create both a validated and clinically useful risk model.
Our LFS risk prediction model, LFSPRO, has successfully navigated these many important steps in the process of becoming a clinical tool. After initial development12, our tool completed a multicenter cohort validation13, which provided additional penetrance estimates for breast cancer, sarcoma, and other cancers in patients with LFS. This was incorporated into LFSPRO for more accurate cancer risk predictions. Following this, we validated the model using a single institution’s clinically ascertained data14. This data set differs from research data sets, as clinical data is obtained from a one-hour counseling session and may have less detailed personal and family history information. Although clinically obtained data often is not as robust as research data sets, risk models are developed to be used in a clinical setting and therefore need to be validated on less robust datasets.
Following these validations, our team developed the LFSPRO R package as well as its Shiny app, which takes this risk model and puts it into a user-friendly format to help identify the likelihood of an individual to have a LP/P TP53 variant15. The remaining limitation is the use of the LFSPRO R package in a clinical setting. It is important to evaluate the performance of this tool compared to already existing LFS tools, such as Chompret and Classic criteria7,9,10. This study aims to evaluate the concordance between the abilities of LFSPRO in predicting the likelihood of a proband having LFS compared to other standard clinical criteria which have historically been used to identify families at high risk to have LFS.
Methods
Study design
First, the LFSPRO R package required a user-friendly interface that could be easily accessed by GCs for quickly running LFSPRO. Our development team created the LFSPROShiny app to be used by study participants to easily input patient and family history information for LFSPRO to calculate their risk to have a TP53 LP/P variant as well as cancer-specific risks15. Family history information from MD Anderson Cancer Center’s (MDACC) electronic health records is input into the LFSPROShiny app for assessment. Outcome variables include whether the proband meets LFS Classic Criteria, Chompret Criteria, and whether LFSPRO predicts the proband to be a TP53 LP/P variant carrier. A threshold of 0.2 was set, where if a proband’s LFSPRO risk calculation is ≥0.2, they are classified as screening positive on LFSPRO, whereas a calculation of <0.2 is a negative prediction on LFSPRO. This threshold was previously suggested by a validation study of LFSPRO on multiple research datasets, and was further confirmed by a recently conducted validation on a clinical dataset that closely resembles the data collected in real genetic counseling sessions13,14. Although LFSPRO allows GCs to adjust this decision threshold if needed, the suggested value of 0.2 was used throughout our analyses.
Survey Development
To assess GCs experiencing using LFSPRO, we developed a Redcap survey that aimed to address four questions: 1) Is LFSPRO concordant to standard practice 2) Does LFSPRO provide additional information past standard genetic counseling 3) Would GCs have felt comfortable sharing results and 4) How would GCs utilize the provided risk predictions. GCs taking these surveys could select to complete a survey for 1) a patient undergoing TP53 germline testing, 2) a patient with a TP53 germline variant that wanted to utilize LFSPROs cancer risk prediction, or 3) a combination of TP53 germline testing risk assessment and cancer risk prediction. Surveys also included an area for free response, so GCs could provider further feedback on their use of LFSPRO.
Data collection
After receiving standard genetic counseling with an MDACC GC, patients identified by the GC as concerning for LFS were flagged in Progeny, an internal family history database. Patients were considered concerning for LFS if they meet Classic and/or Chompret criteria or if the GC who saw the patient determined TP53 genetic testing was appropriate based off their personal/family history (regardless of whether existing criteria was met). Examples of this may include individuals with an LFS-spectrum tumor diagnosed at an early age with a small or limited family history, individuals with somatic testing that revealed a potential underlying germline TP53 variant, individuals with an excess of multiple cancer primaries of any type at any age, and/or individuals with an extensive family history of childhood cancers, regardless of type. The study team then completed a chart review of all identified patients and evaluated them using LFSPRO. LFSPRO TP53 LP/P variant carrier risk, whether the patient meets Chompret and/or Classic clinical criteria, decision to test, and genetic test results were collected.
Four MDACC GCs who most frequently see patients concerning for LFS were invited to use LFSPRO on their concerning for LFS patients after completing their standard genetic counseling session. After each instance of running LFSPRO, these GCs completed a survey regarding whether their patient met existing LFS testing criteria such as Classic and Chompret criteria, whether LFSPRO predicted the patient to have LFS, whether the GCs intuition agrees with existing LFS testing criteria and LFSPRO predictions, whether they would have shared LFSPRO predictions with their patient, and how they would use the LFSPRO data with patients. A complete survey can be seen in Supplemental Materials.
Analyses
General patient information, such as the patient’s age, sex assigned at birth, cancer history, age at cancer diagnoses, and genetic testing results were gathered to be included in analysis. The sensitivity and specificity of Chompret criteria and LFSPRO were calculated as follows:
Thus, the sensitivity and specificity quantify the model’s ability to correctly identify TP53 mutation carriers and wildtypes among the respective groups. To provide a more comprehensive picture of model performance, we also compute the positive predictive value (PPV) and negative predictive value (NPV) as follows:
PPV and NPV are commonly used to evaluate how likely a person is to test positive or negative when a model provides a positive or negative prediction. A model that overly predicts positive results will have high sensitivity but poor PPV.
Lastly, we assessed the concordance of Chompret criteria and LFSPRO with actual testing results via hypothesis testing. We selected the Fisher’s exact test due to the small sample size, and tested the null hypothesis that there is no association between the model-based classifications (i.e., mutation or wildtype) and the genetic testing results. A small p-value would indicate sufficient evidence to reject the null hypothesis, hence suggesting the model is helpful in predicting TP53 mutations.
Results
A total of 140 probands were included in this concordance study. Out of these individuals, 89 were assigned female at birth while 51 were assigned male at birth. The average age of probands was 31 years old. Seventeen probands tested positive for a LP/P TP53 germline variant, with one having a mosaic germline variant (identified in two separate tissue types). Ninety-six probands tested negative for germline TP53 variants, with one of these having a TP53 variant identified at 13% variant allele frequency (VAF) but not identified in a second tissue source. Additionally, one individual in this group had a variant of uncertain significance in TP53 that was likely mosaic. Four individuals have pending genetic testing results, which includes patients who have not yet returned saliva samples, or those who are waiting for their next blood draw to complete testing. Twenty-three probands have unknown genetic testing results, as they were undecided regarding doing genetic testing, or declined genetic testing. In total, 113 individuals completed TP53 genetic testing. For a summary of proband characteristics based on genetic testing results, please see Table 1.
LFSPRO Concordance
Among individuals with germline LP/P TP53 germline variants (n=17), 14 had at least one primary cancer, while 6 had at least two primary cancers. Four of these individuals met Chompret criteria for genetic testing while nine were identified as having an LFSPRO result above 0.2. In individuals that tested negative for LP/P TP53 germline variants (n= 96), 87 had at least one primary cancer, while 9 had at least two primary cancers. Thirty-one of these individuals met Chompret criteria for genetic testing while 21 were identified as having an LFSPRO result above 0.2. Alternatively, 65 of these individuals did not meet Chompret criteria and 75 of these individuals scored less than 0.2 on LFSPRO. This results in a LFSPRO having a sensitivity of 0.529 (95% CI 0.278, 0.770) and specificity of 0.781 (95% CI: 0.685, 0.859) compared to Chompret’s respective sensitivity and specificity of 0.235 (95% CI: 0.068, 0.499) and 0.677 (95% CI: 0.574, 0.769). Among those who tested positive (n=17), the four individuals who met Chompret criteria also met LFSPRO criteria, and additionally, LFSPRO identified 5 individuals to be positive. Among those who tested negative (n=96), 49 individuals were predicted as negative by both criteria, whereas only 5 were predicted as positive by both. Among the remaining 42 individuals, 26 were predicted as negative only by LFSPRO and 16 were predicted negative only by Chompret. Overall, the paired difference between LFSPRO and Chompret criteria yields a gain in sensitivity with a median of 0.29 (95%CI: 0.04, 0.5) and specificity with a median of 0.11 (95%CI: 0.02, 0.22) by LFSPRO. Additionally, LFSPRO had a positive predictive value of 0.30 compared to Chompret’s PPV of 0.114 (Table 2 and 3). LFSPRO’s risk prediction was concordant with genetic testing results with a p-value of 0.01 while Chompret criteria did now show statistically significant concordance (p=0.6).
Genetic Counseling Survey Results
A total of 64 surveys were completed across four GCs. 57 surveys were completed after a genetic counseling session that focused on TP53 germline testing and LFSPRO variant risk prediction, whereas 4 were completed for a TP53 positive patients to determine cancer risk prediction, and 2 were completed as a combination of TP53 variant risk prediction and cancer risk prediction. Of the surveys completed, none of the probands met classic LFS criteria, 12 patients met Chompret criteria, and 46 did not meet Chompret criteria.
GCs reports that 70% of the time, LFSPRO was concordant with Chompret criteria, whereas 81% of the time LFSPRO was concordant with the GC’s experienced clinical judgment. For 77% of cases, GCs report that they offered genetic testing to a patient even if they did not meet standard germline testing criteria (Chompret or Classic criteria). There were 7 cases (16%) where LFSPRO provided additional support to the GC’s decision to offer testing when standard testing criteria was not met. Importantly, in 75% of cases, GCs reported that they would have felt comfortable sharing the results provided by LFSPRO with patients (Table 4).
Summary of concordance and usability measures of LFSPRO to standard genetic counseling sessions at MDACC
When asked how GCs would share LFSPRO results with their patients, the majority reports that they would either provide a generalized summary of the LFSPRO risk estimates (40%), or they would provide the patient with specific risk estimates (37%). Only 9% of surveys reported that they would not have used the information from LFSPRO during the session with their patient. Additional comments regarding how they would or would not use the LFSPRO risk data with patients were provided by genetic counselors in a free response section. Various quotes from these responses can be seen in Table 5.
Quotes associated with genetic counselor’s preferred methods of sharing LFSPRO results if they would have been available during the session.
Discussion
This study provides important and timely clinical usage information on the newly created LFSPRO risk assessment and cancer risk prediction tool. Our results show that the sensitivity, specificity, PPV, and NPV of LFSPRO exceeds that of Chompret criteria. Chompret criteria has long been used by GCs and other healthcare providers to help guide genetic testing recommendations for LFS. These guidelines, along with Classic LFS criteria, are included in the National Comprehensive Cancer Network’s (NCCN) genetic testing guidelines for LFS testing. In turn, many insurance companies use these standard guidelines to determine which patients will have their genetic testing covered by their insurance company. For many patients that do not meet these guidelines, financial barriers could prevent them from gaining access to genetic testing. The development of more personalized risk assessment tools with superior ability to predict patients at risk for LFS can open the door for more patients to qualify for genetic testing, and in turn, identify more individuals with LFS who would benefit from lifelong screening protocols.
In addition to showing an improved sensitivity and specificity in a clinical setting, this study also was able to assess whether LFSPRO aids GCs in their practice. Historically, GCs have been able to use standard genetic testing criteria to recommend testing or may recommend testing based on their own clinical judgement, even if certain criteria are not satisfied. Our study showed that 77% of GC surveys were for patients that did not meet standard LFS testing criteria, but the GC still recommended testing based on their own clinical judgement. LFSPRO supported this recommendation in 16% of these cases. Although there is still room for improvement between GC clinical recommendations and risk assessment tools, LFSPRO was able to further assist GC testing recommendations over standard criteria. It is important to recognize that nationwide, many patients receive genetic testing without a GC involved in their care, such as through their oncologist, surgeon, PCP, or other non-genetics providers. Therefore, having more sensitive and specific tools to predict LFS carriers, such as LFSPRO, may help to identify more patients for LFS testing that would not be identified without a genetics expert involved in their care.
Along with identifying individuals at risk for LFS, LFSPRO can provide GCs and other healthcare providers more specific risk assessments regarding a patient’s likelihood to test positive for LFS and/or their cancer risks over 5, 10, and 15 years. Current testing guidelines recommend genetic testing if criteria are fulfilled or do not recommend genetic testing if criteria are not fulfilled. However, these guidelines do not provide information on how likely a patient is to test positive. GCs may provide an estimated risk based on their own clinical judgment, however LFSPRO can provide more specific risk values to allow healthcare providers to counsel patients more effectively. This not only helps patients anticipate their results more appropriately but may help some patients make decisions regarding whether to pursue genetic testing.
Studies have shown that individuals who received personalized risk information made more informed health screening choices compared to those who received generic risk information16. Additionally, in a breast cancer setting, it has been shown that more accurate risk communication can help decrease general anxiety and breast cancer worry in patients who had high baseline anxiety17. We hope that providing patients with more accurate risk predictions related to LFS will not only help guide their testing and screening decision making but may also reduce the anxiety that commonly is associated with LFS.
Study Limitations
Our study highlights the improved sensitivity and specificity of LFSPRO over existing clinical testing guidelines, however LFSPRO still had multiple false positive and negative predictions. Reviewing these false predictions, 8 of our 21 false positives were for individuals that tested negative for a familial TP53 variant identified in a first degree relatives. LFSPRO predicted these individuals to have a ∼50% risk to test positive, which for this study counts as a screen positive. For the purposes of this study, the sensitivity and specificity could be impacted by these predictive testing negative results, so the true sensitivity and specificity may be higher in non-predictive testing cases. Two of the 21 false positives were found to have LP/P variants in other hereditary cancer genes, specifically CHEK2 and MSH6, which would lead to a strong cancer history that is taken as evidence for TP53 mutations by LFSPRO.
In the false negative category (n=8), one was confirmed to have a common founder mutation c.638G>A, which has been seen to have lower penetrance compared to other TP53 variants. An additional false negative was found to have a VAF <50% and therefore was classified as a mosaic result. Currently, LFSPRO is unable to further personalize risk estimates for low penetrant mutations or mosaic results. Therefore, patients with mosaicism or low penetrant mutations may be more likely to have an LFSPRO risk of <.20 and test positive on genetic testing.
Finally, LFSPRO creates risk prediction based on patient reported personal and family history information that is collected in a clinical setting. For families who have limited family history information, or in cases where a de novo TP53 variant is identified for an individual who has not lived long enough to develop cancer, LFSPRO may not accurately predict the risk to test positive. This is an important limitation that will need to be addressed in future versions of LFSPRO, and a limitation of our current clinical guidelines for TP53 testing as well.
Future Directions
Results from this study showed that GCs would feel comfortable using LFSPRO in a clinical setting and using the results to help guide their counseling sessions. It also showed that LFSPRO provided support for genetic testing in more cases where a GC recommended testing compared to standard clinical criteria. However, this study needs to be expanded to other healthcare providers, especially those at smaller institutions with limited experiences in LFS testing and LFS knowledge. Integrating LFSPRO into non-genetics clinics can help to increase the number of patients concerning for LFS that are offered testing. User testing for non-genetics providers will be vital to insure LFSPRO is easily understood by both those with genetics expertise and those without that training.
Importantly, further research regarding patient understanding of LFSPRO results, patient preferences related to personalized risk assessment versus standard guidelines, and whether personalized risk assessment for LFSPRO improves patient decision making or has any psychosocial benefit would be important.
Conclusions
In summary, this study shows the clinical use of a personalized risk assessment tool for Li-Fraumeni syndrome that has both improved sensitivity and specificity over standard clinical criteria. This tool also can provide healthcare providers and patients with specific risk estimates which could be used in clinic to guide genetic counseling sessions and help aid in patient decision making. Expanded use of this tool in clinics nationwide can ensure patients at risk for LFS are more appropriately offered genetic testing, identified as having LFS, and followed for high risk screening in hopes of reducing cancer mortality rates for these patients.
Footnotes
Research support: Our research is supported by the Cancer Prevention and Research Institute of Texas [RP200383], the National Cancer Institute [R01CA239342].