Predictors of Premature Discontinuation of Opioid Use Disorder Treatment in the United States
=============================================================================================

* Celia Stafford
* Wesley Marrero
* Rebecca B. Naumann
* Kristen Hassmiller Lich
* Sarah Wakeman
* Mohammad S. Jalali

## Abstract

Over the last few decades, opioid use disorder (OUD) and overdose have dramatically increased. Evidence shows that treatment for OUD, particularly medication for OUD, is highly effective; however, despite decreases in barriers to treatment, retention in OUD treatment remains a challenge. Therefore, understanding key risk factors for OUD treatment discontinuation remains a critical priority. We built a machine learning model using the Treatment Episode Data Set – Discharge (TEDS-D). Included were 2,446,710 treatment episodes for individuals in the U.S. discharged between January 1, 2015 and December 31, 2018 (the most recent available data). Exposures contain 32 potential risk factors, including treatment characteristics, substance use history, socioeconomic status, and demographic characteristics. Our findings show that the most influential risk factors include characteristics of treatment service setting, geographic region, primary source of payment, referral source, and health insurance status. Importantly, several factors previously reported as influential predictors, such as age, living situation, age of first substance use, race and ethnicity, and sex had far weaker predictive impacts. The influential factors identified in this study should be more closely explored to inform targeted interventions and improve future models of care.

## Introduction

Opioid-related deaths have risen dramatically since the 1990’s, with 49,860 deaths in the US in 2019 alone. With an estimated 1.6 million people suffering from an opioid use disorder (OUD),1 millions more lives are impacted beyond just victims of fatal overdose.2,3

Treatment for OUD, particularly medication for OUD (MOUD), is highly effective.4 Treatment success depends on adherence and retention in care.5 Research indicates that while there is variation by treatment type (e.g., MOUD or psychosocial treatment), OUD treatment of all kinds suffers from high rates of premature exit. In some cases, treatment discontinuation has been reported as high as 85%,6 although the average rate of exit hovers around 30%.7 Beyond the US; studies citing treatment discontinuity are numerous globally.8,9 Better retention in treatment requires an improved understanding of the characteristics and needs of target populations, and to date, the literature offers limited and conflicting evidence.

Two systematic reviews (published in 201310 and 20207) summarize several hundred studies on substance use disorder (SUD) treatment attrition. They show that literature on predictors of treatment retention, adherence, or discontinuation shares small sample sizes (e.g., median n=144 among studies included in the 2013 review), and limited scope and generalizability (e.g., faith-based treatment, or male populations only). Also, the reviewed studies considered few covariates, and primarily demographics (e.g.,11–14). Furthermore, where there is overlap on factors included across studies, the results are often conflicting. For example, 64 of the 122 studies included in the 2013 review examined the role of sex. Just 10 of those reported a statistically significant relationship, with five reporting male sex as a predictor of treatment discontinuation and five reporting female sex as a predictor of discontinuation. Less than 10% of studies investigated the relationship between retention and treatment specific factors such as the method, setting, and duration.10 In the 2020 review,7 a similar focus on age, sex, and education was evident. For example, 146 of the 151 studies examined the role of age, with studies generally concluding that age was not a significant predictor of premature treatment exit. The only participant characteristics associated with discontinuation were race, income, daily cigarettes smoked, and heroin and cocaine use. Since the publication of these systematic reviews, a 2021 study specific to retention in MOUD found that methamphetamine use, younger age, and homelessness were risk factors for treatment discontinuation.15

Despite the substantial body of literature on the opioid overdose crisis, the importance of treatment, and concerns about treatment retention,16 a large-scale analysis of demographic *and* contextual factors contributing to premature treatment exit has not been conducted. To address this gap, we developed a machine learning model based on millions of treatment episodes with a large holistic set of predictors. Our objective was to examine which factors best predict treatment attrition. Illuminating specific factors from this rich, national data source can help pinpoint factors that should be more closely explored to inform targeted interventions.

Additionally, though machine learning-based predictive models have been widely used in medicine,17 there is no big-data-based model utilized for understanding premature OUD treatment exit. We develop the first model of its kind and identify key factors and their relationship with premature OUD treatment exit.

## Methods

### Study Population

The cohort consisted of OUD treatment episodes included in the Treatment Episode Data Set-Discharge (TEDS-D)18—a national data system that contains records of individuals ages 12 and above derived from SUD treatment facilities. The national system includes facilities that receive any federal funding—varying by state, this can also include facilities such as private doctors’ offices.19 Annual data from 2015, until the most recent available year, 2018, were combined. Years prior to 2015 were excluded as they collected less data. Included were episodes where individuals reported use of heroin, or other opioids. No other inclusion or exclusion criteria were used.

We followed best practice observational study reporting guidelines.20 Complete datasets, code, and results are available on GitHub ([https://github.com/castaff/TEDS\_Treatment_Attrition](https://github.com/castaff/TEDS_Treatment_Attrition)) for review and reproducibility.17

### Factors and Factor Creation

To predict treatment discontinuation, we included 23 factors from TEDS-D that were collected at admission and excluded factors collected at treatment discharge. Included in this analysis are demographics, frequency of substance use, routes of administration, self-reported substances of use, treatment history, source of referral, and planned treatment type (i.e., whether MOUD is planned in the patient’s treatment).

By combining and recoding existing factors, we also created several additional variables. The first reflects minimum reported age at which the individual began using substances.21,22 The second presents the maximum frequency of nonmedical opioid use. A third variable was created indicating heroin use, and the fourth indicated any injection drug use.23 Finally, four binary factors were created indicating whether individuals used substances falling into the broad categories of stimulants, hallucinogens, sedatives, or tranquilizers (Table S1).

In total, 32 factors were included. A table of variables excluded and the reasons for exclusion can be found in Table S1, as well as a complete list of the variables included and their definitions. Missing data were imputed using a random forest-based multiple imputation approach.24

### Outcome

We dichotomized the reason for discharge to “dropped out of treatment” vs. all other reasons for treatment discharge. “Dropped out of treatment” includes clients who exited treatment for unknown reasons as well those who left against professional advice or were lost to follow up and discharged administratively. The outcome is non-missing for all treatment episodes.

### Machine Learning Analysis

We conducted a classification analysis, using a tree-based approach. With decision trees, what is made up for in interpretability is lost in accuracy as there can be large differences between trees based on changes in the data sampled for model training.25 This variance can be improved by building a “forest” of many trees. Differentiating it from bagged trees, a random forest selects parameters for each split from a random selection of *m* variables. This reduces correlation among individual trees. The majority vote is then taken of the predicted class from all trees.26

The data were divided into a training and a testing set, using a 75% and 25% split. We performed 10-fold cross validation on the 75% training set, and a final model was fit on the complete training set and performance assessed on the testing set. Missing data were imputed on the training set and the testing set independently to avoid data leakage.24 Additionally, models were fit based upon data stratified by year, as well as only on complete records (without imputation). Model performance was assessed with area under the receiver operating curve (AUC) and accuracy. In addition, permutation variable importance, which computes the decrease in model performance when a given predictor is shuffled, was reported. Finally, partial dependence plots were created for the top five influential variables. These plots depict the marginal effect a variable has on the predicted outcome of a given machine learning model.27 All analyses were performed using R statistical programming language version 3.3, using the tidymodels, ranger, vip, broom, pdp and missForest packages. Data preprocessing was conducted with dplyr.

## Results

Figure 1 presents an overview of study data. After excluding variables not collected at treatment admission and episodes without reported opioid use, 2,446,710 episodes were included in the analysis. Of these treatment episodes, 321,735 contained complete data. Most variables had missing values on less than 10% of the records, with just health insurance status, primary source of payment for treatment, days waited before entering treatment, primary source of income, and marriage status missing more than 20%. Overall, about 90% of observations were missing five or fewer variables. A complete summary of missingness can be found in Table S2.

![Figure 1:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/07/30/2021.07.26.21261080/F1.medium.gif)

[Figure 1:](http://medrxiv.org/content/early/2021/07/30/2021.07.26.21261080/F1)

Figure 1: 
Episode and variable inclusion and exclusion

### Descriptive Results

Table S3 presents a full listing of the characteristics of individual treatment episodes, stratified by premature treatment exit status. The vast majority (76%) of opioid-related treatment episodes were among white individuals. Of all opioid-related episodes, stimulants were the most often co-occurring substance reported (37%), followed by cannabis (21%) and alcohol (21%). A total of 698,161 (28.5%) opioid-related treatment episodes resulted in treatment discontinuation. The most represented age group among all episodes was 25-34, which accounted for 43% of all episodes and 43% of the episodes resulting in premature exit. Of episodes resulting in premature exit, 27% had no reported prior treatment episodes. Among episodes in which individuals discontinued treatment, 34% had planned to use MOUD at baseline, while 24% of those who did not exit treatment prematurely planned to use MOUD at baseline.

### Random Forest

We used 1,835,033 treatment episodes for training and 611,677 for testing (see Figure S1). The random forest classifier achieved a mean AUC of 75% with a standard error of 0.002 across the 10 folds cross-validation. On the unseen-by-the-model testing set, the random forest achieved an accuracy of 72% with an AUC of 70%.

Of greater relevance to treatment decisions is the relative importance of included factors in predicting premature treatment exit. A full ranking of important factors can be seen in Figure 2. The most influential predictor was service setting (e.g., inpatient, ambulatory, detox). Its exclusion from the model decreased accuracy by almost 4%. Besides service setting, geographic region, referral source, primary source of payment for treatment, and health insurance status each produced an accuracy decrease greater than 1.5%. Interestingly, low on the list of importance were reported heroin use, injection use, age, race, and ethnicity. Variable importance remained largely unchanged when the model was fit on individual years (see Figure S2).

![Figure 2:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/07/30/2021.07.26.21261080/F2.medium.gif)

[Figure 2:](http://medrxiv.org/content/early/2021/07/30/2021.07.26.21261080/F2)

Figure 2: 
Variable importance of the predictors of premature treatment exit

Figure 3 shows the marginal impact of different levels of each of the five most important variables on treatment dropout with partial dependence plots—service setting of 24hr freestanding residential detox, southern geographic region, Medicaid source of payment and insurance type, and criminal justice treatment referral were associated with the largest decreases in treatment retention.

![Figure 3:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/07/30/2021.07.26.21261080/F3.medium.gif)

[Figure 3:](http://medrxiv.org/content/early/2021/07/30/2021.07.26.21261080/F3)

Figure 3: 
Partial dependence plots of most influential variables

## Discussion

We used a machine learning model to predict premature OUD treatment exit using treatment episode data from TEDS-D. With 32 predictors spanning demographics, substance use habits, and treatment information, we built a model identifying factors most influential in determining whether an individual will exit treatment prior to completion. We found that treatment setting, geographical region, health insurance status, treatment referral source, and primary source of payment were the strongest predictors.

While we found that service setting was the most important predictor of attrition, only 3% of papers reviewed by Brorson et al did.10 Service setting options captured in TEDS-D include detox, short- and long-term inpatient care, and different forms of ambulatory care, each with differing implications for transportation, finances, and other factors. Though longer term and/or ambulatory care-based treatment modalities offer more chances to lose patients in the process of care, we found that patients are less likely to prematurely exit these. Certain treatments, such as MOUD, lead to better outcomes,4, 555 however there is little research showing that detox or residential care improves outcomes for individuals with OUD. Coupled with high attrition rates, it is clear that this is a costly and ineffective “revolving door” treatment model that bears re-evaluation. Thought should be dedicated to expanding long-term outpatient care potentially coupled with affordable housing rather than further investment in costly inpatient models.

Another notable feature is geographic region. Figure 3 highlights a meaningful difference in attrition across regions, and it is noticeable that exclusion of this variable from the model precipitates a drop in accuracy of over 3.5%. As shown in the second panel of Figure 3, likelihood of premature exit was highest in the South. Research indicates that this heterogeneity may exist due to infrastructural factors related to treatment access, social factors like stigma, and socioeconomic factors such as likelihood of working a manual labor job; a lot of this heterogeneity remains causally unexplained and is subject to more investigation.28

While socio-economic status and income are occasionally tested,10 our model is the first to include the primary source of payment for treatment. We find it to be the third most influential factor with highest rates of premature treatment exit in individuals paying with Medicare, Medicaid, and with other government payments. This connection bears further research – it is possible that programs accepting public payers differ from those accepting commercial insurance, or there could be ties to socioeconomic status and potential multicollinearity with social determinants of health or employment status. Simply knowing risk of treatment attrition is high among Medicaid enrollees provides an intervention opportunity.

Related to the primary source of payment, we also surfaced health insurance status as an important predictor. Medicaid stands out as a predictor of treatment attrition. Research indicates that Medicaid beneficiaries have a high prevalence of comorbidities and barriers to healthcare, including affordable and accessible transportation, that may contribute to this attrition.29 More recently, state Medicaid programs have attempted to remove insurance-based barriers (e.g., prior authorizations) to MOUD initiation; however, additional work to support treatment retention remains a clear need.

Referral source was also an important predictive factor, with referral from criminal legal settings associated with increased probability of premature treatment exit. Research indicates that 15% of deaths following release from prison are related to opioids and decreased tolerance thereof, and up to 65% of the US prison population may have an active substance use disorder.30 The Substance Abuse and Mental Health Services Administration and others have increased resources to support access to MOUD in prisons, and post-release; however, additional supports are needed. It is also apparent that coerced treatment, i.e., treatment mandated by the courts, may not be an effective strategy.31

With interestingly low impact on model accuracy were age and age of first use. Although age of first substance use has not been investigated in this context, there is evidence of its tie to development of OUD,21 and eventual admission to treatment.22 Most studies assess the impact of age on treatment exit, but just 36% of the studies investigating age found a significant relationship. Of these, 88% linked younger age to increased risk of premature treatment exit.7,10 Additional research on how substance use trajectories influence treatment initiation and retention trends is needed, as well as how treatment retention supports may differ across the life span.

In contrast with the many small cohort and retrospective studies on premature treatment discontinuation, this analysis utilized records from millions of substance use treatment episodes. While these data have been widely studied, research has focused on trends of substance use over time.32,33 We leveraged these data to study predictors of treatment discontinuation, taking advantage of a rich set of covariates and the 2,446,710 treatment episodes. This provided a larger sample size and the ability to explore a wider range of risk factors than any other study on the subject. Also differentiating this study, we utilized a supervised machine learning approach. This method offers a distinct benefit over previous analyses by considering possible interactions and multicollinearity among factors. Finally, random forest models are helpful in assessing nonlinearities which is challenging to parse out with classic regression-based models.

This study is subject to several limitations. First, TEDS-D is an observational dataset that relies on submission by individual states. Depending on year, several states do not submit data and are not included in totals. Nonresponse may be associated with higher rates of treatment discontinuation due to less treatment funding or substance use support which may lead to underestimates of rates of treatment exit. However, we would not expect this to alter the predictive relationships explored in our models. Second, TEDS-D has a minimum set of data elements that states are required to report on, including demographic and substance use factors. Factors outside of this minimum requirement can be missing. These missing data were imputed using a random forest and though the efficacy of this method has been described in detail (e.g., 34,35), imputation is not a perfect solution. We report our results on complete records (without imputation) in Figure S3 and the findings are overall similar to those with imputation.

TEDS-D also includes only facilities that are state licensed or certified. Because of differences in individual state policies, some states include private doctors’ offices and other private clinics, and some do not. While it is the most complete survey of treatment facilities, TEDS-D is an undercount both because of non-reporting and excluded private facilities. Additionally, there could be bias around the kinds of people that exit state-sponsored facilities vs those that might exit treatment prematurely at a private doctor’s office.

Reasons for treatment discharge not considered to be premature treatment discontinuation initiated by the patient included completion, termination by the facility and transfer to another facility or treatment program; a future analysis could consider any premature exit, whether initiated by patient or facility, as a failure. Finally, there is considerable variation in response to opioid use treatment. Because of this, there is great need to identify patients who will not respond well or are more likely to exit treatment prematurely. Although we identified key predictors of premature treatment exit, we cannot conclude that these are causal relationships. Still, these serve as important areas for future research to further explore specific causal mechanisms. Understanding the dynamics surrounding these key factors holds important clinical relevance for future treatment decisions and models of care.

## Conclusion

We demonstrated that a predictive model for premature OUD treatment exit can be constructed using a dataset of US adults receiving treatment for OUD at state-affiliated treatment facilities. Our results may help address varying likelihoods of treatment attrition across patients and modalities of treatment. The combination of effective treatment interventions with data on an individual’s risk level can help channel resources toward targeted mechanisms of attrition for specific patients.

## Data Availability

All data and analysis details are reported in the article.

[https://github.com/castaff/TEDS\_Treatment\_Attrition](https://github.com/castaff/TEDS_Treatment_Attrition) 

## Ethical approval

Not needed.

## Competing interests

Authors declare no competing interests.

## Supplementary Document for

### Supplementary Figures

![Figure S1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/07/30/2021.07.26.21261080/F4.medium.gif)

[Figure S1.](http://medrxiv.org/content/early/2021/07/30/2021.07.26.21261080/F4)

Figure S1. 
Data partitioning for cross validation

![Figure S2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/07/30/2021.07.26.21261080/F5.medium.gif)

[Figure S2.](http://medrxiv.org/content/early/2021/07/30/2021.07.26.21261080/F5)

Figure S2. 
Variable importance stratified by year

![Figure S3.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/07/30/2021.07.26.21261080/F6.medium.gif)

[Figure S3.](http://medrxiv.org/content/early/2021/07/30/2021.07.26.21261080/F6)

Figure S3. 
Variable importance of complete data records without data imputation

### Supplementary Tables

View this table:
[Table S1.](http://medrxiv.org/content/early/2021/07/30/2021.07.26.21261080/T1)

Table S1. 
TEDS-D variable exclusion

View this table:
[Table S2.](http://medrxiv.org/content/early/2021/07/30/2021.07.26.21261080/T2)

Table S2. 
Missing data summary

View this table:
[Table S3.](http://medrxiv.org/content/early/2021/07/30/2021.07.26.21261080/T3)

Table S3. 
Individual characteristics stratified by premature treatment exit status

*   Received July 26, 2021.
*   Revision received July 26, 2021.
*   Accepted July 30, 2021.


*   © 2021, Posted by Cold Spring Harbor Laboratory

This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/)

## References

1.  1.Substance Abuse and Mental Health Services Administration. Key substance use and mental health indicators in the United States: Results from the 2019 National Survey on Drug Use and Health. HHS Publication No. PEP20-07-01-001, NSDUH Series H-55 Web site. [https://www.samhsa.gov/data/](https://www.samhsa.gov/data/). Published 2020. Accessed.
    
    

2.  2.Kilmer B, Everingham S, Caulkins J, et al. What America’s Users Spend on Illegal Drugs, 2000–2010. RAND Corporation; 2014.
    
    

3.  3.Reuter P, Caulkins JP, Midgette G. Heroin Use Cannot Be Measured Adequately with a General Population Survey. Addiction.n/a(n/a).
    
    

4.  4.Wakeman SE, Larochelle MR, Ameli O, et al. Comparative Effectiveness of Different Treatment Pathways for Opioid Use Disorder. JAMA Network Open. 2020;3(2):e1920622–e1920622.
    
    

5.  5.Timko C, Schultz NR, Cucciare MA, Vittorio L, Garrison-Diehn C. Retention in medication-assisted treatment for opiate dependence: A systematic review. J Addict Dis. 2016;35(1):22–35.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/10550887.2016.1100960&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F07%2F30%2F2021.07.26.21261080.atom) 

6.  6.Smith EM, North CS, Fox LW. Eighteen-Month Follow-Up Data on a Treatment Program for Homeless Substance Abusing Mothers. Journal of Addictive Diseases. 1996;14(4):57–72.
    
    

7.  7.Lappan SN, Brown AW, Hendricks PS. Dropout rates of in-person psychosocial substance use disorder treatments: a systematic review and meta-analysis. Addiction. 2020;115(2):201–217.
    
    

8.  8.Berghofer G, Schmidl F, Rudas S, Steiner E, Schmitz M. Predictors of treatment discontinuity in outpatient mental health care. Social Psychiatry and Psychiatric Epidemiology. 2002;37(6):276–282.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s001270200020&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=12111033&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F07%2F30%2F2021.07.26.21261080.atom) 

9.  9.Basu D, Ghosh A, Sarkar S, Patra BN, Subodh BN, Mattoo SK. Initial treatment dropout in patients with substance use disorders attending a tertiary care de-addiction centre in north India. Indian J Med Res. 2017;146(Supplement):S77–S84.
    
    

10. 10.Brorson HH, Ajo Arnevik E, Rand-Hendriksen K, Duckert F. Drop-out from addiction treatment: a systematic review of risk factors. Clin Psychol Rev. 2013;33(8):1010–1024.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.cpr.2013.07.007&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24029221&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F07%2F30%2F2021.07.26.21261080.atom) 

11. 11.De Weert-Van Oene GH, Schippers GM, De Jong CA, Schrijvers GJ. Retention in substance dependence treatment: the relevance of in-treatment factors. J Subst Abuse Treat. 2001;20(4):253–261; discussion 263-254.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0740-5472(01)00160-X&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=11672639&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F07%2F30%2F2021.07.26.21261080.atom) 

12. 12.Sayre SL, Schmitz JM, Stotts AL, Averill PM, Rhoades HM, Grabowski JJ. Determining predictors of attrition in an outpatient substance abuse program. The American Journal of Drug and Alcohol Abuse. 2002;28(1):55–72.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1081/ADA-120001281&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=11853135&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F07%2F30%2F2021.07.26.21261080.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000173793500004&link_type=ISI) 

13. 13.Vendetti J, McRee B, Miller M, Christiansen K, Herrell J. Correlates of pre-treatment drop-out among persons with marijuana dependence. Addiction. 2002;97 Suppl 1:125–134.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=12460134&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F07%2F30%2F2021.07.26.21261080.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000179892800011&link_type=ISI) 

14. 14.Longabaugh R, Wirtz PW, Gulliver SB, Davidson D. Extended naltrexone and broad spectrum treatment or motivational enhancement therapy. Psychopharmacology (Berl). 2009;206(3):367–376.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s00213-009-1615-3&link_type=DOI) 

15. 15.Krawczyk N, Williams AR, Saloner B, Cerdá M. Who stays in medication treatment for opioid use disorder? A national study of outpatient specialty treatment settings. Journal of Substance Abuse Treatment. 2021;126:108329.
    
    

16. 16.NASEM (National Academies of Sciences E, and Medicine: Health and Medicine Division): Board on Health Sciences Policy: Committee on Medication-Assisted Treatment for Opioid Use Disorder. Medications for Opioid Use Disorder Save Lives. National Academies Press (US); 2019.
    
    

17. 17.Beam AL, Kohane IS. Big Data and Machine Learning in Health Care. JAMA. 2018;319(13):1317–1318.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2017.18391&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=29532063&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F07%2F30%2F2021.07.26.21261080.atom) 

18. 18.Health USDo, Abuse HSS, Studies MHSAOoA. Treatment Episode Data Set --Discharges (TEDS-D) In. Substance Abuse and Mental Health Data Archive 2015.
    
    

19. 19.Treatment Episode Data Set - Discharges. 2015-2018.
    
    

20. 20.von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med. 2007;147(8):573–577.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7326/0003-4819-147-8-200710160-00010&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17938396&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F07%2F30%2F2021.07.26.21261080.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000250437100007&link_type=ISI) 

21. 21.Jordan CJ, Andersen SL. Sensitive periods of substance abuse: Early risk for the transition to dependence. Developmental Cognitive Neuroscience. 2017;25:29–44.
    
    

22. 22.Strashny A. Age of Substance Use Initiation Among Treatment Admissions Aged 18 to 30. In: The CBHSQ Report. Rockville (MD): Substance Abuse and Mental Health Services Administration (US); 2013:1–9.
    
    

23. 23.Uusküla A, Raag M, Vorobjov S, et al. Non-fatal overdoses and related risk factors among people who inject drugs in St. Petersburg, Russia and Kohtla-Järve, Estonia. BMC Public Health. 2015;15(1):1255.
    
    

24. 24.Stekhoven DJ, Bühlmann P. MissForest--non-parametric missing value imputation for mixed-type data. Bioinformatics. 2012;28(1):112–118.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/bioinformatics/btr597&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22039212&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F07%2F30%2F2021.07.26.21261080.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000298383800017&link_type=ISI) 

25. 25.Breiman L, Friedman J, Stone CJ, Olshen RA. Classification and Regression Trees. Taylor & Francis; 1984.
    
    

26. 26.Breiman L. Random Forests. Machine Learning. 2001;45(1):5–32.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1023/A:1010933404324&link_type=DOI) 

27. 27.Friedman JH. Greedy Function Approximation: A Gradient Boosting Machine. The Annals of Statistics. 2001;29(5):1189–1232.
    
    

28. 28.Rigg KK, Monnat SM, Chavez MN. Opioid-related mortality in rural America: Geographic heterogeneity and intervention strategies. International Journal of Drug Policy. 2018;57:119–129.
    
    

29. 29.Akinyemiju T, Jha M, Moore JX, Pisu M. Disparities in the prevalence of comorbidities among US adults by state Medicaid expansion status. Prev Med. 2016;88:196–202.
    
    

30. 30.Binswanger IA, Blatchford PJ, Mueller SR, Stern MF. Mortality after prison release: opioid overdose and other causes of death, risk factors, and time trends from 1999 to 2009. Annals of internal medicine. 2013;159(9):592–600.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7326/0003-4819-159-9-201311050-00005&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24189594&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F07%2F30%2F2021.07.26.21261080.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000326769400003&link_type=ISI) 

31. 31.Parhar KK, Wormith JS, Derkzen DM, Beauregard AM. Offender Coercion in Treatment: A Meta-Analysis of Effectiveness. Criminal Justice and Behavior. 2008;35(9):1109–1135.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0093854808320169&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000258392800002&link_type=ISI) 

32. 32.Branham DK. Time-Series Analysis of the Impact of Prescription Drug Monitoring Programs on Heroin Treatment Admissions. Subst Use Misuse. 2018;53(4):694–701.
    
    

33. 33.Shuler MN, Wallington SF, Qualls-Hampton RY, Podesta AE, Suzuki S. Trend Analysis of Substance Abuse Treatment Admissions in New Orleans From 2000–2012: A Population-Based Comparison Pre- and Post-Hurricane Katrina. Substance Use & Misuse. 2016;51(12):1542–1554.
    
    

34. 34.Kokla M, Virtanen J, Kolehmainen M, Paananen J, Hanhineva K. Random forest-based imputation outperforms other methods for imputing LC-MS metabolomics data: a comparative study. BMC Bioinformatics. 2019;20(1):492.
    
    

35. 35.Pantanowitz A, Marwala T. Evaluating the impact of missing data imputation through the use of the random forest algorithm. arXiv preprint arxiv:08122412. 2008.