ABSTRACT
Purpose Understanding the value of genetic screening and testing for monogenic disorders requires high-quality, methodologically robust economic evaluations. This systematic review sought to assess the methodological quality among such studies and examine opportunities for improvement.
Methods We searched Pubmed, Cochrane, Embase, and Web of Science for economic evaluations of genetic screening/testing (2013-2019). Methodological rigor and adherence to best practices were systematically assessed using the BMJ checklist.
Results Across 47 identified studies, there was substantial variation in modeling approaches, reporting detail, and sophistication. Models ranged from simple decision trees to individual-level microsimulation, comparing between two and >20 alternative interventions. Many studies failed to report sufficient detail to enable replication or did not justify modeling assumptions, especially for costing methods and utility values. Meta-analyses, systematic reviews, or calibration were rarely used to derive parameter estimates. Nearly all studies conducted some sensitivity analysis, and more sophisticated studies implemented probabilistic sensitivity/uncertainty analysis, threshold analysis, and value of information analysis.
Conclusion We describe a heterogeneous body of work and present recommendations and exemplar studies across the methodological domains of (1) perspective, scope, and parameter selection, (2) use of uncertainty/sensitivity analyses, and (3) reporting transparency for improvement in the economic evaluation of genetic screening/testing.
INTRODUCTION
Genetic screening and testing for monogenic diseases can be used to establish a definitive molecular diagnosis in symptomatic patients, identify increased risk of disease in pre- symptomatic individuals, provide information about prognosis or management of rare disorders, identify other at-risk family members, and guide reproductive planning. If used appropriately, such analysis has the potential to reduce morbidity and mortality through disease prevention or early intervention, targeted treatment, and avoidance of inappropriate or ineffective treatment. However, genetic analysis and indicated downstream care for people who test positive can be costly for both the health system and the patient. Despite being rare (the most common affecting less than 1% of the population), molecular diagnosis of monogenic conditions can be highly useful from a clinical perspective. Currently, diagnostic genetic testing is recommended only to those meeting specific clinical criteria or after other clinical tests have failed to give a definite diagnosis. It may be cost-effective to identify and care for patients with monogenic conditions before symptoms manifest, especially for conditions with effective interventions that could improve clinical outcomes. Researchers are assessing the value of screening for clinically useful monogenic conditions within a broader population. Economic evaluations—including cost-consequence, cost-benefit, cost-utility, and cost-effectiveness analyses1—are critical for assessing the potential value of genetic screening/testing for specific applications. Over the last two decades, the number of such evaluations has increased rapidly.2, 3 Yet, the speed with which economic evaluations have been produced may be outpacing the field’s ability to disseminate and widely adopt best practices, as well as identify gaps where best practices have not been adopted.
High-quality methodological approaches to economic evaluations are essential for the appropriate interpretation and implementation of study findings. Despite the recent publication of several methodological recommendations for cost-effectiveness analyses in genetic medicine, study quality across disease areas has not been systematically reviewed.4–6 Importantly, there are methodological challenges unique to economic evaluations of clinical genetic screening and testing for monogenic disorders that deserve specific attention.7 Compared with other medical interventions that have more routinely been subjects of economic evaluations (e.g., pharmacoeconomics), the methodological tendencies of economic evaluators of genetic screening and testing programs may still be in formation.
This qualitative systematic review characterizes the methodological quality of recent economic evaluations of genetic screening and testing for monogenic disorders, spanning from birth to diagnosis. Throughout this review, we use the term “genetic testing” when referring to a clinical diagnostic setting in which a patient is at increased risk for a genetic disorder due to their personal and/or family history; we use “genetic screening” when the individual being screened is not known to be symptomatic of, nor at substantially increased risk for, such a condition. We emphasized this distinction given both the differing resources demanded of and health outcomes that may be experienced through either strategy and the field’s interest in evaluating screening programs. See Appendix 1 for more detail. The goal of this review is to improve the methodological quality of future economic evaluations to guide implementation of such studies. Where consistent methodological limitations were identified, we have provided recommendations and exemplar models.
METHODS
Search Strategy
This systematic review identified economic evaluations of genetic screening and testing for monogenic disorders, focusing on assays that seek to establish (or refine) a genetic risk or diagnosis. Included studies incorporated costs and health outcomes downstream from genetic testing and diagnosis. The review was registered with PROSPERO on July 2, 2019 (record number CRD42019141086). Studies that did not include complete economic evaluations (“the comparative analysis of alternative courses of action in terms of both their costs and consequences”1) or considered no health outcomes beyond diagnostic yield were excluded.8, 9 Studies of common variants and polygenic risk scores for complex diseases were excluded, as were studies of somatic variants or gene expression in tumors.10, 11 Pharmacogenetic screening was excluded, defined as testing for genetic variants primarily related to adverse reactions to drugs or drug metabolism.12 Genetic testing/screening specifically related to reproductive planning (pre-conception or pre-natal) was excluded.13 Systematic reviews and commentaries were also excluded.14 Additional search strategy details are included in Appendix 2.
Code Development
Qualitative codes reflecting methodological features of evaluations were developed using a top-down and inductive approach. Initial codes were adopted from the 1996 checklist developed by Drummond and Jefferson for the BMJ (hereafter: “BMJ checklist”), along with features highlighted in similar systematic reviews.15–18 While more recent checklists have been developed as guides for authors,19 the BMJ checklist was chosen given its emphasis on quality assessment by reviewers, its use in recent reviews of genetic evaluations18, 20 and its widespread use among similar systematic reviews published from 2010 to 2018.21 A full list of the codes and summary statement templates used can be found in Appendix 3.
BMJ Checklist and Qualitative Assessment beyond the BMJ Checklist
We used the 35 BMJ checklist items (hereafter: “items”) to assess included studies. Items were classified as not met (0), partially met (1), fully met (2), or not relevant (N/R). If relevant information was not contained in the primary publication or supplemental materials, but an appropriate citation was listed, we classified that item as “not available” (N/A). A detailed rubric was developed for each checklist item (Appendix 4). Average quality values were calculated for each question by summing the 0s, 1s, and 2s each article received across all studies, then dividing that sum by the number of items for which 0s, 1s, and 2s were possible (excluding N/R and N/A).
Additional items were created to track, in more detail, important article features which the BMJ checklist did not directly address but have been recommended in other authoritative guidelines (Appendix 5).3, 7, 28 These features did not contribute to average checklist values. During analysis, we grouped these additional items, along with select BMJ checklist items that we wished to highlight in more detail, into three distinct methodological constructs: perspective, scope, and parameter selection; the use of sensitivity/uncertainty analyses; and reporting transparency.
Review Process
Article coding and assessment began with a “primary coder” who applied qualitative codes and assessed items. Next, a “secondary coder” received the already-coded articles from primary coders and cross-examined articles to ensure codes were appropriately applied. Secondary coders independently assessed all 35 BMJ items, and were blind to the assessment given by primary coders. Conflicts were discussed and resolved between the two reviewers (KJ, IG, KH, KHL).
RESULTS
Study Characteristics
Of the 5727 records identified through database searches, 47 studies met inclusion criteria (Figure 1). Table 1 reports the main features of the 47 articles included in this review along with each article’s average quality assessment. Three genetic conditions constituted nearly half of all studies: Lynch syndrome (n=10), familial hypercholesterolemia (n = 7), and hereditary breast and ovarian cancer (n=14). A smaller set of studies considered maturity onset diabetes of the young (n=3), thrombophilia (n=2), or multiple conditions (n=2), and undiagnosed pediatric disorders (n=4). The setting of most studies was the United States (n=11), the United Kingdom (n=9), and Australia (n=9), with smaller numbers also conducted in Germany (n=4), both the United States and the United Kingdom (n=3), the Netherlands (n=3) and elsewhere (Spain: n=2; Poland: n=1; Norway: n=1; Malaysia: n=1; Italy: n=2; Taiwan, n=1; Singapore: n=1).
Table 2 presents the major model characteristics across all studies. Most studies utilized the combination of a decision tree with a Markov model (n = 17), though several used either just a decision tree (n = 11) or just a Markov model (n = 6). Five studies employed some form of individual-level simulation (e.g., microsimulation). Less than half of all studies (n = 18) compared only one alternative to “usual care”, which often consisted of the standard-of-care genetic or clinical testing/screening protocol. Most studies conducted cost-utility analyses (i.e., health outcomes are expressed in utility measures like QALYs or DALYs) (CUAs) (n = 26), with several conducting both CUAs and cost-effectiveness analyses (i.e., health outcomes are expressed in clinical measures like total diagnoses or deaths) (CEAs) (n = 10) and a limited number (n=6) conducting cost-consequence analyses. Three studies incorporated societal costs, the rest were strictly from either the healthcare sector or payer perspective.
BMJ Checklist Assessment
Some basic items from the BMJ checklist were fully met by nearly all studies, including “The research question is stated” (average value [AV]: 2) and “The primary outcome measure(s) for the economic evaluation are clearly stated” (AV: 1.98). Conversely, several checklist items consistently were not met or partially met by all studies, including “Quantities of resources are reported separately from their unit costs” (AV: 0.87) and “Details of the subjects from whom valuations were obtained are given” (AV: 1.10). Some items were consistently addressed by citing external sources but without an overview of the source material (N/A), such as “Methods to value health states and other benefits are stated.” Several of the cost-consequence analyses received an above average number of “N/R” assessments. Average values for each variable are presented in Table 3. While comparative assessment of studies is not the primary focus of our analysis and the BMJ checklist is not intended to produce a quantitative assessment, the distribution of values and the average value for each article is presented in Appendix 6 and Appendix 7.
Assessment of Key Methodological Constructs
Perspective, scope, and parameter selection
For studies that based effectiveness estimates for preventive interventions on several sources (n=25), roughly a third (n=7) presented a thorough evidence synthesis, which outlined how they identified the parameters used in their analysis. A systematic literature review was conducted and a formal meta-analysis was completed for important variables in only four articles (Appendix Table 5).29–32
Fourteen articles either conducted micro-costing or referenced prior micro-costing analyses, while the rest opted for a macro-costing approach. Furthermore, several studies (n=7) adopted costing information from other, similar cost-effective analyses without justifying the primary source of the costing data.
Of studies with a clearly stated perspective, all presented at least a healthcare payer or healthcare system perspective. Three articles also included components of a societal perspective; two of these studies incorporated lost labor productivity costs into overall costs and one conducted two separate analyses from either the healthcare sector or societal perspective. No studies incorporated non-medical benefits of genetic screening or testing, such as the personal utility of non-actionable genetic information or psychological benefits of negative test results. Studies that only examined carrier screening were excluded from the review, though two studies either incorporated costs associated with assisted reproductive technology use by parents after a child’s genetic diagnosis or DALYs averted by decisions to avoid having children with genetic disease.33, 34 One study included a discussion of the privacy implications of familial cascade testing,35 although privacy costs were not incorporated into their model.
No studies calibrate their model using real-world data. Two articles attempted some form of internal or external model validation, although this was not done to inform model parameterization but rather to confirm that model outcomes aligned with data used within the model and external values (e.g., known prevalence of disease).32, 36
Use of Sensitivity/Uncertainty Analyses
While all evaluations considered in this review conducted sensitivity analyses, the depth, breadth, and presentation of analyses varied widely. The majority of studies (n=33) conducted some form of one-way or two-way deterministic sensitivity analyses and 19 of such studies presented the results in the form of a tornado diagram. Among the 29 studies that included a probabilistic sensitivity analysis (PSA), nine displayed PSA results in ICER scatter plots, 23 presented cost-effectiveness acceptability curves, but only three presented uncertainty intervals for primary estimates. Twenty-one studies conducted at least one scenario analysis and eight studies conducted at least one threshold analysis. Only two value of information analyses were conducted, which included an expected value of perfect information analysis and an expected value of partial perfect information analysis for specific parameter groups (e.g., treatment costs, probability of cancer recurrence) (Table 2).
Reporting Transparency
Both the study question and answer to the study question were clear in all papers. Only two studies did not clearly report the discount rate for their analysis, though many studies (n=19) did not provide a proper justification of why their specific rate was selected. Similarly, most papers (n=39) clearly articulated the year and price information of their cost units but only about half (n=24) reported whether or how these prices had been adjusted for inflation or currency conversion.
All articles presented both disaggregated outcomes (such as total QALYs gained or total healthcare costs) as well as final ICER calculations. However, less than half of the studies (n=23) based the population size on a real-world population. Only one article disaggregated intervention costs into specific categories unique to genetic screening and nine studies disaggregated costing results based on the generic source of costs such as genetic sequencing, disease prevention, and disease treatment.
For studies that reported results in the form of either QALYs or DALYs (n=37), about half (n=16) presented the valuation method or study by which their utility values were generated and slightly more than half (n=19) reported the population from whom these values were generated.
Of the 19 studies that reported the results of one-way sensitivity analyses in the form of a tornado diagram, 9 had figures that did not indicate the direction of the associations between each parameter and the ICER. It was also unclear for several studies (n=10) why certain variables were ultimately selected to be included in deterministic sensitivity analyses (such as tornado diagrams) and not others. While more than half of the studies conducted a probabilistic uncertainty analysis.
DISCUSSION
Overview of Major Findings
This systematic review analyzed the methodological quality of 47 recent economic evaluations of genetic screening or testing for monogenic disorders across disease arenas. There was substantial variation in model sophistication and reporting quality. Most articles satisfied basic criteria for their presentation of parameter values, model design, and results as well as their implementation and interpretation of sensitivity/uncertainty analyses. A few studies achieved higher levels of sophistication or quality and can serve as exemplars for future work.32, 34, 37–41
Recommendations for Future Evaluations and Exemplar Cases
While uniformity of evaluation design and reporting should not come at the cost of analytic flexibility, the heterogeneity of quality assessed in our review suggests the importance of further training to develop high-quality economic evaluations of genetic screening/testing. Scholars are encouraged to reference one or more of the guidelines that have been published over the past 20 years; these guidelines demonstrate near-consensus on the key elements of an economic evaluation.19, 42 Within the last five years, several textbooks have also been published on the proper methodological approach to economic evaluation.3, 7, 28
Informed by our assessment and considering authoritative sources, we make several recommendations for future economic evaluations of genetic testing/screening. Our recommendations focus on three arenas that consistently caused difficulty for articles considered in our review (parameter selection, use of sensitivity/uncertainty analyses, and reporting transparency). Table 4 summarizes this discussion along with several exemplar cases from our review are provided to demonstrate recommended practices.
Perspective, Scope, and Parameter Selection
A central challenge in conducting any economic evaluation is employing expert judgement and the evidence synthesis needed to select or estimate parameter values for the model. A formal systematic review with or without meta-analysis should be attempted for parameter values that are especially influential, uncertain, or likely to change in different environments (e.g. as a consequence of policy decisions).3434432
Most economic evaluations of genetic testing/screening take a simplistic view of genetic analysis costs, often ignoring costs of implementation and patent outreach. For more realistic integration of the costs incurred by genetic testing/screening, micro-costing is recommended.7, 45 Micro-costing is especially important for analyses centered on changes in the way resources are delivered within a specific program or diagnostic odyssey, which is often the case for innovative genetic medicine programs.28 32Micro-costing may not be suitable for studies primarily concerned with nationally aggregated or long-run costs, and the importance of various sub- components may depend on the perspective.
Economic evaluations of genetic testing/screening should evaluate value across relevant stakeholders, including but not limited to payer and societal perspectives. Genetic analyses are unusual in that they has implications not just for the individual being tested but also for family members, who may or may not be covered by the same payer. For settings without a single- payer, including family members in models requires careful consideration of how and even whether cascade testing is relevant in a payer-perspective analysis. Moreover, it has been strongly recommended that economic evaluations report two standard reference case perspectives: one from the healthcare payer perspective (i.e. formal healthcare sector costs borne by third-party payers or paid for out-of-pocket by patients) and, in parallel, one from the societal perspective (i.e. including patient/family time costs involved in receiving an intervention and for self-management).3 Presenting a reference case from a particular third-party payer (e.g. the federal government, a single healthcare system, or a particular insurance company) can also be warranted, though care should be taken to consider whether the covered population is stable, especially when benefits may lag many years behind initial investments (e.g., crossing Medicaid and Medicare programs or attrition from insurance plans). Presenting both analytical perspectives in tandem clarifies how value may vary substantially among key stakeholders. 46
To account for the balance between the burden of screening and recovered productivity, future studies should refer to the “Impact Inventory” developed by the Second Panel on Cost- Effectiveness in Health and Medicine to guide which costs should be considered from either perspective.5 This resource was used by only one study in this review.32 For the specific context of genomic screening programs, Fragoulakis and colleagues outline several direct costs (e.g., healthcare payer costs) and indirect costs (e.g., patient productivity lost and family expenses) which may also serve as a useful guide. Examples of indirect costs modeled in reviewed studies included work productivity lost because of illness46 and physician visits.47 Screening for genetic conditions in the general population may also lead to non-health-related changes in utility resulting from new awareness of having a condition that either requires additional interaction with the health care system or cannot be addressed medically.
Model calibration is a process used in economic evaluations to improve the accuracy of parameters that cannot be directly measured, leveraging available data that can be matched with the model.48 Calibration efficiently searches the space of plausible parameter values to find those which optimize the model’s fit to real-world data.49 Calibration is not always necessary but should be used when it can reduce the amount of parameter uncertainty in the model, especially for the most influential or actionable model parameters.50
Authors should pay special attention to test performance in the model. True clinical sensitivity is extremely difficult to measure for most conditions, and categories of possible test results vary between diagnostic testing, family cascade testing, and population screening.51 The probability of further interaction between the healthcare system and patient will differ based on how these categories are reported. Evaluators should ensure that their modeling of test results accurately reflects both what is known about the clinical sensitivity and specificity of the genetic test and how that knowledge is translated into diagnostic protocols, which may vary across sites of implementation.
Use of Sensitivity/Uncertainty Analyses
Beyond reporting outcomes of a base case, economic evaluations of genetic testing/screening should identify and consider the impact of stochastic, parameter, and/or structural uncertainty as well as patient heterogeneity. Analyses should distinguish between variability in inputs that may affect outcomes (sensitivity analysis) and uncertainty in model inputs that may alter the uncertainty of model conclusions (uncertainty analysis).28, 52 As with all economic evaluations, it is challenging to estimate the collective impact that uncertainty within individual parameters will have on the uncertainty of overall model outcomes. We strongly recommend studies to conduct a Probabilistic Uncertainty Analysis (PUA) and to use the PUA results to clearly report the degree of uncertainty of estimates for primary outcomes of interest (i.e., confidence intervals). Given their likely dramatic impact on model outcomes, we recommend studies to consider the following parameters within their PUA: pathogenic variant prevalence (which depends on the target population and clinical scenario), probability of referral to genetic counseling and genetic testing uptake, likelihood of clinical outcomes (based on penetrance and expressivity of the condition), uptake/adherence and efficacy of interventions in symptomatic and pre-symptomatic individuals, morbidity and mortality in the absence of a genetic diagnosis, and cost of genetic analysis, implementation of interventions, and care used as part of post-result clinical interventions.
Studies should incorporate threshold analyses (a type of sensitivity analysis) to interrogate key parameters that may change in response to policy decisions, programmatic design, or other exogenous factors. Threshold analyses identify the minimum or maximum value for a given parameter that results in the intervention meeting willingness-to-pay thresholds. In the context of genetic testing/screening, this may fruitfully be applied to parameters such as the prevalence of the pathogenic variant being screened, with the assumption that programs could be developed to target populations with a critical prevalence rate (e.g., those with a clinical history suggestive of genetic disease). Threshold analyses could also determine the minimum rate of uptake for accepting genetic testing/screening or prophylactic interventions for screening to become cost-effective.5354 It is widely appreciated that genetic laboratory costs have fallen over the past decade, and there is speculation over whether testing prices will continue to fall or may even increase if testing companies capture greater control of markets. This value should also be strongly considered for threshold analyses. 55
As with any novel intervention, many parameters necessary to evaluate genetic testing/screening are fixed but unknown or uncertain. Some of these parameters, such as the prevalence of pathogenic variants in populations, could be studied using biobank or cohort studies and epidemiological research methods. Value of information analyses should be conducted to quantify the value of investing in research activities that generate additional evidence that lessens parameter uncertainty.3 This type of analysis informs what research is most valuable—essential information for researchers and funders.
Scenario analyses should be used to estimate structural uncertainty or to compare different intervention approaches in a model. In the context of genetic testing/screening, they could be used to consider alternative scenarios in which more energy is dedicated to certain sub- populations or the diagnostic pathway is slightly modified for these sub-populations. 3537,38The consistent incorporation of scenario analyses will not only make models more informative (by calling attention to particularly uncertain or variable parameters) but also improve methodological rigor as authors are forced to critically think about the specific questions that their model must be designed to address.
Reporting Transparency
The amount of content necessary to properly present an economic evaluation is often too much to fit in a single manuscript, prompting evaluators to reference secondary literature.21 When referencing secondary literature (especially for parameter estimation), summary information should be available within the main manuscript or appendix for readers to understand the context and methods behind the results produced from that literature. 56
When price transformations are necessary—either between different years or between different currencies—authors must be clear what year was used as the benchmark and what exchange rate was employed for the transformation. When adjusting for inflation, authors should use inflation rates unique to the medical industry.28 When relevant, inflation or cost adjustments should be specific to medical commodities that have increased or decreased in price relative to the rest of the industry (e.g., when a patented drug becomes available in generic forms).35Evaluators should also clearly identify when monetary amounts included in the model reflect price or cost estimates; we recommend accounting for all the associated costs of a medical good or activity.28 The cost of the same genetic analysis may also vary considerably depending on the equipment used, throughput level, and sequencing method; we recommend clearly identifying the sub-components of costs associated with the genetic analysis.
We strongly recommend disaggregating the outcome of cost-effectiveness analyses into total costs and total effectiveness. Disaggregation is especially useful if the size of the model population is reported and corresponds to a real-world population rate. This allows for population-wide health and economic impacts (e.g., a budgetary impact analysis) to be reported in addition to per-person cost and effectiveness. When expressing the total costs associated with any screening strategy, it is also recommended that authors report both total costs as well as costs disaggregated into relevant categories. This categorization provides a clear depiction of which aspect of genetic testing/screening is responsible for incremental cost differences. Detecting sources of incremental variation is especially important for a field such as genetics in which materials and activities are rapidly changing costs.4056–59
Alignment with Similar Systematic Reviews
Several recent systematic reviews of economic evaluations of genetic screening have been conducted for either specific populations or a more limited set of medical conditions. While prior reviews primarily covered older studies, were limited to specific genetic conditions, and were not as comprehensive as our own regarding methodological assessment, these reviews have identified many of the same limitations in economic evaluations our review has exposed. These include emphasizing the healthcare payer or health system costing perspectives over societal perspectives,60–63 dependence on macro-costing strategies and adopting costing estimates from other, similar studies,64 and limited or opaque use of complex sensitivity analyses.60, 65 The performance of our articles as measured by the BMJ checklist is also consistent with two recent reviews of economic evaluations of genetic testing that employed the BMJ checklist.20, 65 Both these reviews found that most studies failed to provide a rigorous description of how costs were derived, provided no description for how disparate sources were synthesized to establish effectiveness estimates, failed to appropriately adjust price or currency or report such adjustment, and had limited description of the valuation methods by which utility weights were calculated or characterizations of the population from which they were derived.
Study Limitations
There are several limitations to this systematic review. Firstly, our assessment mechanism gave equal weight to all items, implying that all items were of equal ease to achieve and of equal importance to the methodological quality of an article when important inequalities likely exist across both dimensions. To account for this limitation, we have focused our discussion on those items which we believe to be of greater importance to overall quality and have provided recommendations to facilitate ease of achievement. Secondly, this review does not consider the influence methodological limitations may have on the primary or secondary outcomes of studies. For instance, an opaque presentation of parameter derivation may complicate a reader’s ability to interrogate the integrity of a model, though these parameters may ultimately be the most appropriate leaving results unbiased. On the other hand, the lack of a PSA may indirectly hide the fact that primary outcomes are widely variable and cannot be interpreted with high confidence. Future research should consider which methodological features of an article may have the largest influence on outcomes. Thirdly, there is an abundance of methodological detail that went beyond the scope of this review, such as how well the structure of the model reflected the actual decision nodes within the healthcare system under study and whether a comprehensive selection of alternative strategies was considered for each model. This level of granularity is best suited for reviews with a much more limited scope than the one we conducted.
Conclusion
Economic evaluation of genetic medicine has been recently accelerating. Our review considered the methodological quality of such studies and demonstrated that, with notable exceptions, many studies fell short across several key methodological criteria. Improvements in these arenas highlighted above would enhance the extent to which outcomes can be understood, translated, and faithfully replicated. Renewed attention to the methodological design of future economic evaluations of genetic testing/screening is warranted. Future economic evaluations in this space should adhere to established guidelines and may benefit from considering the specific recommendations and exemplar articles identified in this review.
Data availability
All articles included in this review are accessible online, and the search terms used to query these articles can be found in the Appendix.
Data Availability
Data used to create this review is available upon request
Author Information
Conceptualization: K.H.L., K.S., I.G., K.J., J.S.B., K.H.; Data curation: K.S., I.G.; Formal analysis: K.J., K.S., K.H.L.; Funding acquisition: K.H.L., J.S.B.; Investigation: K.J., K.S., I.G., K.H., K.H.L.; Project administration: K.J., K.S.; Supervision: J.B., K.H.L.; Writing—original draft: K.J., K.S., K.H.L.; Writing—review and editing: I.G., K.H., J.S.B., K.J., K.S., K.H.L.
Ethics Declaration
This study was determined to be non-human subjects research by the University of North Carolina IRB.
Acknowledgments
We wish to acknowledge Dr. Gail Henderson for both her help with the design and scope of this systematic review as well as for funding early research assistant support through the Center for Genomics and Society at UNC (grant ID: 2P50 HG004488), a National Human Genome Research Institute (NHGRI)-funded center. We also wish to acknowledge Hailey James for developing our query search strategy, contributing to study planning, and conducting article screening. This study was partially supported by another grant from the support from the NHGRI at the National Institutes of Health (grant ID: 2U01 HG006487).
Footnotes
Conflicts of interest: The authors declare no conflict of interest.
References
- 1.↵
- 2.↵
- 3.↵
- 4.↵
- 5.↵
- 6.↵
- 7.↵
- 8.↵
- 9.↵
- 10.↵
- 11.↵
- 12.↵
- 13.↵
- 14.↵
- 15.↵
- 16.
- 17.
- 18.↵
- 19.↵
- 20.↵
- 21.↵
- 22.
- 23.
- 24.
- 25.
- 26.
- 27.
- 28.↵
- 29.↵
- 30.
- 31.
- 32.↵
- 33.↵
- 34.↵
- 35.↵
- 36.↵
- 37.
- 38.
- 39.
- 40.
- 41.↵
- 42.↵
- 43.
- 44.
- 45.↵
- 46.↵
- 47.↵
- 48.↵
- 49.↵
- 50.↵
- 51.↵
- 52.↵
- 53.
- 54.
- 55.↵
- 56.↵
- 57.
- 58.
- 59.
- 60.↵
- 61.
- 62.
- 63.↵
- 64.↵
- 65.↵
- 66.
- 67.
- 68.
- 69.
- 70.
- 71.
- 72.
- 73.
- 74.
- 75.
- 76.
- 77.
- 78.
- 79.
- 80.
- 81.
- 82.
- 83.
- 84.
- 85.
- 86.
- 87.
- 88.
- 89.
- 90.
- 91.
- 92.
- 93.
- 94.
- 95.