ABSTRACT
Background Substance use disorders (SUDs) represent a major public health risk. Yet, our understanding of the mechanisms that maintain these disorders remains incomplete. In a recent computational modeling study, we found initial evidence that SUDs are associated with slower learning rates from negative outcomes and less value-sensitive choice (low “action precision”), which could help explain continued substance use despite harmful consequences.
Methods Here we aimed to replicate and extend these results in a pre-registered study with a new sample of 168 individuals with SUDs and 99 healthy comparisons (HCs). We performed the same computational modeling and group comparisons as in our prior report (doi: 10.1016/j.drugalcdep.2020.108208) to confirm previously observed effects. After completing all pre-registered replication analyses, we then combined the previous and current datasets (N = 468) to assess whether differences were transdiagnostic or driven by specific disorders.
Results Replicating prior results, SUDs showed slower learning rates for negative outcomes in both Bayesian and frequentist analyses (η2=.02). Previously observed differences in action precision were not confirmed. Logistic regressions including all computational parameters as predictors in the combined datasets could differentiate several specific disorders from HCs, but could not differentiate most disorders from each other.
Conclusions These results provide robust evidence that individuals with SUDs have more difficulty adjusting behavior in the face of negative outcomes than HCs. They also suggest this effect is common across several different SUDs. Future research should examine its neural basis and whether learning rates could represent a new treatment target or moderator of treatment outcome.
1. INTRODUCTION
Substance use disorders (SUDs) have devastating mental and physical health consequences (1–5), and lead to large societal and financial burden (6). They also show high co-morbidity with depression and anxiety disorders, which can be both a cause and consequence of substance use (1–3). Yet, our understanding of the mechanisms that lead to substance use, and of those that moderate recovery and abstinence, remains poor. Given current relapse rates with existing treatments (7), a better understanding of these mechanisms may be crucial for identifying effective clinical targets; and they could also offer important insights regarding etiology and prevention.
Computational modeling offers a promising approach for characterizing these mechanisms in SUDs (8, 9), as it can disentangle processes underlying perception, learning, and decision-making that may contribute to these disorders (9, 10). A growing body of work has now applied this approach – highlighting several computational processes that differ from healthy participants and that may relate to vulnerability, severity, and/or treatment outcomes (11, 12). Much of this work pertains to deficits in “model-based” (i.e., goal-directed, prospective) control, which are thought to amplify habitual and impulsive choice (e.g., for recent reviews, see (12, 13)). Other studies have also highlighted deficits in computational mechanisms subserving perception (particularly with respect to internal bodily states; i.e., interoception (14, 15)) and reinforcement learning (e.g., see (16–18)). Thus, this approach has already been helpful in elucidating a number of mechanisms that may underlie SUDs.
Building on this body of work, we recently applied a computational approach to examine the way individuals with SUDs balance information-seeking and reward-seeking (19). Relative to healthy comparisons, modeling revealed slower learning rates for negative outcomes across multiple SUDs, as well as faster learning rates for positive outcomes and less precise (i.e., less value-sensitive) action selection. This pattern of results was also moderately stable when assessed in the same participants one year later (20). These findings suggested that substance use might persist despite painful life consequences due to slower learning from these negative outcomes. Returns to drug use after improvements in treatment could also be caused by imprecise action selection mechanisms (i.e., switching away from actions with positive outcomes). In the current pre-registered study (https://osf.io/u3sbg), we assess the reliability of these findings in a new, independent sample with similar characteristics to those in the original study. The same task and computational analyses were applied, and we report which results do and do not replicate. We subsequently extend these results by combining the two datasets, providing power to assess whether effects are driven by specific SUDs or co-morbid affective disorders or whether they represent a transdiagnostic marker across SUDs.
2. METHODS
2.1 Participants
Participants were sampled from the confirmatory dataset (N=550) of the larger Tulsa 1000 (T1000) study (21). The T1000 project recruited a longitudinal community sample with dimensional measures designed around the NIMH Research Domain Criteria framework. The sample in total consists of 1050 individuals, ages 18-55, recruited through radio, electronic media, word-of-mouth, and treatment center referrals. Participants were screened through various dimensional psychopathology measures, including Patient Health Questionnaire (PHQ-9 (22)) ≥ 10, Overall Anxiety Severity and Impairment Scale (OASIS (23)) >= 8, and/or Drug Abuse Screening Assessment (DAST-10 (24)) ≥ 3. Healthy comparisons (HCs) did not show elevated symptoms on any of the previous measures. Exclusion criteria for participants were: (i) positive tests for drug use, (ii) meeting criteria for psychotic, bipolar, or obsessive-compulsive disorders, (iii) a history of moderate-to-severe traumatic brain injury, neurological disorders, or severe or unstable medical conditions, (iv) active suicidal intent or plan, or (v) change in medication dose within 6 weeks. Full inclusion/exclusion criteria are reported in (21). The study was approved by the Western Institutional Review Board. All participants provided written informed consent prior to completion of the study protocol, in accordance with the Declaration of Helsinki, and were compensated for participation. ClinicalTrials.gov identifier: #NCT02450240.
Participants were divided into two groups: participants with one or more SUDs (with or without co-morbid anxiety/depression; N = 168) and participants without any mental health diagnosis (HCs; N = 99). Diagnoses were determined according to DSM-IV or DSM-5 criteria using the Mini International Neuropsychiatric Inventory (MINI) (25) and confirmed through clinical conferences with a board-certified psychiatrist. Table 1 provides summary statistics for demographic and clinical measures, as well as two-sample t-tests assessing group differences (and corresponding Bayes factors [BFs], calculated using the BayesFactor package in R (26)). Table 2 provides a summary of specific diagnoses within the SUDs group. Table S1.1 in Supplementary Materials 1 compares the demographic characteristics and clinical measures in this sample to those in our previous study (19). The only difference observed was that SUDs in the current sample had lower scores than the previous sample on the Wide Range Achievement Test reading score (WRAT) (27), a measure of premorbid intelligence (t(253) = 2.27, p = 0.02). This measure was included because of expected differences in baseline intellectual functioning relative to HCs.
Demographic and Clinical Characteristics
Lifetime DSM-IV/DSM-5 Psychiatric Diagnoses within SUDs
2.2 Three-Armed Bandit Task
Participants completed a standard three-armed bandit task (28). For 20 blocks of 16 sequential choices, they had to learn from trial-and-error which of three options was most likely to deliver a win (described in detail in our previous study (19); also see (20)). To maximize reward in each block, participants needed to strike a balance between sampling untested options (exploration) and capitalizing on the options that appeared most rewarding (exploitation). Figure 1 shows the task interface and provides additional details.
Upper Left: Screenshot of the task interface. In each block (N = 20), there were 16 trials where participants could choose the left, middle, or right options (buttons 1, 2, or 3 at the bottom). Each choice could lead to either a win (green circle) or a loss (red circle), which would appear above the chosen option. Left: A graphical depiction of the task model. Shaded circles denote inferred variables on each trial. White circles denote fixed model parameters. Arrows indicate dependencies between variables (see Table S1.2 for a detailed explanation). Right: Model equations for learning and choice. Reward probabilities were encoded in a matrix A; i.e., 
. Approximate beliefs about these reward probabilities, q(A), were updated within a matrix a; i.e., 
. This matrix included a Dirichlet distribution over reward probabilities, the concentration parameters (ai,j) of which were updated after each observation according to the specified learning rule (shown in the top-left box, which includes the learning rate η). Policy values were based on expected free energy (Gπ), which favors policies that are expected to maximize both reward (encoded in C) and information gain (i.e., the magnitude of expected change in beliefs about reward probabilities [q(A)] after a new observation). Randomness in action selection given approximate posteriors over policies, q(π), was controlled by an action precision parameter α (note that σ indicates a softmax function). All free parameters in the model are shown in red.
2.3 Computational Modeling
A detailed computational model description can be found in Table S1.2. In brief, learning and choice are implemented by a set of inference and decision rules drawn from the active inference framework, which is designed to capture information-seeking under uncertainty. A detailed description of the mathematical framework can be found in (29, 30). The task model includes a set of observations (e.g., wins/losses), the subjective value assigned to those observations, choice states (option 1, 2, and 3), the probabilities of each observation under each choice state (e.g., the reward probabilities), and policies (actions) controlling transitions between choice states on each trial.
Free parameters within the model offer mechanistic explanations for individual differences in choice behavior. These parameters are summarized in Table 3 and shown within the learning and choice equations in Figure 1.
Free Model Parameters
We used a standard variational Bayes algorithm (variational Laplace (31)) to find parameter estimates that maximize the probability of participant behavior under the model, while also minimizing overfitting with a complexity cost (32). We then performed Bayesian model comparison (33, 34) across a set of 10 nested model variants (shown in Table S1.3) to determine the best model. The estimates from this best model were then used in subsequent statistical analyses.
Results of parameter recoverability and model identifiability analyses are reported in previous work (20). Parameters show good recoverability across the region of parameter space describing participant behavior, and the best-fit model is identifiable within model comparison.
2.4 Statistical Analysis Procedure
2.4.1 Primary Replication Analyses
Statistical analyses replicate those in our previous study (19) and were pre-registered (https://osf.io/u3sbg). We first ran correlations to confirm how model parameter estimates were related to descriptive behavioral measures, including number of wins, mean reaction times (RTs) (after trimming using iterative Grubbs at a threshold of p < 0.01 (35)), and stay vs. shift behavior after wins vs. losses (i.e., whether participants shift to a new option or stay at the current option after a win or loss on the prior trial).
Our primary analyses then used parametric empirical Bayes (PEB) (36) to test general linear models with each model parameter estimate as the outcome variable, and using age, sex, baseline intellectual functioning (WRAT scores), and diagnostic group (SUDs and HCs) as predictors. PEB analyses allow incorporation of both posterior parameter means and variances when assessing these group-level models, as opposed to using the means alone as point estimates.
As in our prior study, secondary frequentist analyses (equivalent t-tests and multiple regressions) were also performed to assess group differences in the posterior parameter means. Supplementary model comparisons with these regressions, using Bayes factors (BFs) to compare the probability of the data under models with different possible combinations of predictors, were also performed (see note below Table 4).
Parameter values (mean and SD) and group comparison
For comparison to model-based results, these same analyses were also performed on the descriptive behavioral measures. PEB analyses were performed in MATLAB (https://matlab.mathworks.com/). All other analyses were performed in R (2018; www.R-project.org/).
Our previous study employed propensity matching (on the basis of age, sex, and WRAT scores; using the optmatch package in R [https://cran.r-project.org/web/packages/optmatch/index.html]), which we initially planned to do here. However, the characteristics of this sample, and particularly the minimal overlap in WRAT scores between groups, prohibited our matching algorithm from finding a suitable subsample. Consequently, we only report analyses on the full sample using linear models that included age, sex, and WRAT as covariates. However, due to issues that arose during data collection, 12 HCs and 48 SUDs did not have available WRAT scores. In order to capitalize on the full sample, while also assessing possible effects of baseline intellectual functioning, we therefore ran each planned analysis once in the full sample (without including WRAT scores) and once in the subsample with available WRAT scores (while including this measure as a covariate). Results of analyses in the full sample are reported in detail below. Results within the analyses repeated in the subsample with WRAT scores are provided in detail within Supplementary Materials 1 and 2, but we also note important findings here.
Given the heterogeneity within our SUDs sample, we also planned to assess potential differences in computational parameters between different SUDs – specifically, differences between individuals with opioid use disorders and stimulant use disorders. While there was a sufficient number of individuals with stimulant use disorders without co-morbid opioid use disorders (N = 109), the low number of individuals with opioid use disorders that did not have co-morbid stimulant use disorders (N = 10) was too small to permit this analysis. However, after completing all pre-registered replication analyses, we combined the exploratory and confirmatory samples to examine disorder-specific effects (see section 2.4.3 and 3.5).
2.4.2 Secondary Analyses
We ran correlations within SUDs to examine the relationship between model parameters and dimensional measures of symptom severity. Although not pre-registered, we also used t-tests to assess whether model parameters differed in medicated vs. unmedicated individuals in the SUDs group.
In addition to the 168 individuals with SUDs described above, the T1000 dataset also included 32 participants with an SUD diagnosis that had current symptom levels below the DAST ≥ 3 cutoff. These participants were not included in our primary analyses; however, they were used in some supplementary analyses described below. For more details about these additional participants, and comparison to the main sample, see Table S1.4.
2.4.3 Disorder-Specific Analyses in Combined Samples
After completing all pre-registered replication analyses, we combined the exploratory and confirmatory datasets (N = 468) to ask questions about narrower diagnostic groups, which the previous or current dataset alone provided limited power to answer. Using logistic regressions, we first asked whether model parameters could differentiate HCs from those who had each of the specific substance use or affective disorders listed in Table 2. These regression models included each of the five parameters as predictors of diagnostic status (i.e., coding HCs = 0 and those with the specific disorder in question = 1, removing all other participants). We also examined those with no co-morbid SUDs when available sample size allowed, but high levels of co-morbidity did not permit this in most cases (i.e., only in stimulant use disorders; N = 55). These were considered post-hoc analyses done primarily to provide additional insights regarding whether specific overlapping disorders might drive the confirmed difference between HCs and SUDs overall. As such, we did not correct p-values for number of regressions; but we note that a Bonferroni correction for 11 regressions would entail a threshold of p < .0045. In similar logistic regressions, we then asked whether model parameters could differentiate one disorder vs. others (i.e., removing HCs and coding those with vs. without the specific disorder in question equal to 1 and 0, respectively).
2.4.4 Predictive Categorization
Finally, to evaluate whether model parameters could predictively classify individuals into disorder groups, we trained logistic regression models (with the 5 parameters as joint predictors) on the exploratory sample and then assessed whether the trained models could accurately categorize individuals as either HCs or SUDs, both in general and for each specific SUD or affective disorder separately (i.e., HCs = 0, Disorder = 1 in each case, removing those without that the disorder being predicted). These predictive categorization analyses were performed using the glm function in R (https://www.rdocumentation.org/packages/stats/versions/3.6.2/topics/glm). Due to the different sample sizes per group, training data were also balanced using frequency weights. To assess performance, we report receiver operating characteristic curves (ROCs), reflecting the true positive rate (sensitivity) vs. false positive rate (1 -specificity) for different categorization thresholds, and the associated area under the curve (AUC). AUCs indicate how often a random sample will be assigned to the correct group with higher probability.
3. RESULTS
3.1 Model Validation
The winning model matched that found in our previous study (including action precision, reward sensitivity, separate learning rates for wins/losses, and insensitivity to information; Table S1.3), with a protected exceedance probability of 1.0. The average probability of participant actions under the model was .56 (SD = .10). The model assigned the highest probability to participant actions on 61% (SD = .10) of trials (note that chance = 33.33%).
Direct comparison of parameter values between the previous and current sample is provided in Table S1.5, with detailed results of associated linear models in Table S2.1 and S2.2 within Supplementary Materials 2. The parameter values in SUDs did not differ significantly between samples, nor were these values different between the two HC samples.
Relationships between model parameters and descriptive measures also replicated prior results (see Figure 2). Notable examples included: 1) number of wins was positively associated with action precision and reward sensitivity; 2) number of lose/shift choices was positively correlated with learning rate for losses; 3) stay choices were positively correlated with reward sensitivity and insensitivity to information; and 4) RTs were longer in those with slower learning rates for losses and faster in those with greater learning rates for wins, reward sensitivity, and insensitivity to information. The relationships with RTs are noteworthy, as model parameters are not fit to this aspect of behavior (i.e., the model itself does not simulate RTs).
Correlations (and associated Bayes factors) between model parameter values and descriptive behavioral measures.
3.2 Group comparison
Replicating our prior results, PEB models with age and sex as covariates found strong evidence for group differences in learning rate for losses (slower in SUDs; posterior probability [pp] = 0.93; see Figure 3). These models also revealed some evidence for differences in reward sensitivity (greater in HCs; pp = 0.82). Note, however, that this latter finding was not present in our prior study. Including WRAT reading scores (in the subsample for which they were available) did not change these results (see Figure S1.1). See the Additional Effects section within Supplementary Materials 1 for further effects observed in relation to age, sex, and WRAT scores.
The top-left panel depicts results from Parametric Empirical Bayes (PEB) analyses showing the posterior means and credible intervals for effects of group (when accounting effects of age and sex). Positive values indicate greater values in HCs relative to SUDs. Learning rates are estimated/displayed in logit space, while all other parameters are in log space. Results replicate the group difference in learning rate for losses found in our prior study. All other panels compare the means and standard errors for each parameter in our prior study (exploratory sample; 54 HCs, 147 SUDs) to those in the current study (confirmatory sample; 99 HCs, 168 SUDs), when separating parameters by group. Also shown are Bayes factors (BFs) evaluating the evidence for differences between the two samples (BF < 1 indicates greater evidence for the absence of a difference). BFs indicated that the data were between 2.6 and 7.1 times more likely under a model with no difference between samples. For an identical plot restricting analyses to participants with available WRAT reading scores (and including these scores as predictors in the PEB analyses), see Supplementary Figure S1.1, which shows the same overall pattern of results (and in some cases shows stronger consistency between samples; most notably with action precision values).
As further assessment of successful replication, we also compared parameter estimates from the exploratory sample in our prior study to those in the present study for each group. We further computed BFs from Bayesian t-tests for each parameter/group to assess evidence for differences between samples. As shown in Figure 3, these BFs ranged from .14 to .38, indicating that the data are between 2.6 and 7.1 times more likely under a model with no difference between samples. This provides strong evidence that the new sample successfully replicates the findings in our prior study. Analogous results for the subsample with available WRAT scores are shown in Figure S1.1.
3.3 Supporting frequentist analyses
In complementary frequentist analyses, initial two-sample t-tests for each parameter confirmed the group difference in learning rates for losses (t(265) = 2.26, p = 0.02, d = 0.28), but did not support differences in any other parameter. Table 4 reports results of subsequent regressions in the full sample analogous to the PEB models above, assessing whether age and sex could account for observed group differences. Equivalent Bayesian regressions were also used to identify the model (combination of predictors) for which the data provided the most evidence and how this compared to models either adding or removing potential group effects (see note below Table 4). Here we observed the same expected effects of group on learning rates for losses (lower values in SUDs).
Results of identical analyses on descriptive behavioral measures are reported in Table 5. We found greater wins in HCs than SUDs (p = 0.047; see Table 7 and Figure 3), as found in our prior study. We also found fewer win/stay choices in SUDs, which was not observed previously. As in our prior study, we also repeated these analyses for the first and second halves of each task block separately (i.e., first 7 choices vs. subsequent choices) to assess periods where exploration vs. exploitation would be expected to dominate, respectively (see Tables S1.8). This showed that the SUDs group only made fewer win/stay choices than HCs in the second half of each game (p = .02; and greater win/shifts, p = .04), while HCs only showed greater wins in the first half of each game (p = .007). Additional relationships observed between age or sex and these measures are reported in Table S2.3 and S2.4 and summarized in the Additional Effects section of Supplementary Materials 1.
Descriptive behavioral measures and group comparison
In most cases, similar results were seen in the subsample with WRAT scores (see Tables S1.6-8). The major exception was that this subsample did find higher action precision in HCs than SUDs, similar to our initial study. Evidence for the difference in learning rates for losses was also weaker in this subsample, but it was retained within the winning model in model comparison (BF = 2.93).
3.4 Relationships with symptoms
Relationships between model parameters and symptom measures in SUDs are reported in Figure S1.2. No significant relationships were observed, which failed to replicate the relationships observed in our previous study between PHQ/OASIS and information insensitivity, and between OASIS and reward sensitivity (BFs also provided moderate evidence against these relationships: 0.18 – 0.53).
When comparing the main sample of SUDs to the 32 diagnosed individuals with symptom severity below the cutoff of DAST ≥ 3, we found no differences in parameter values (see Table S1.4).
Comparison of those that were vs. were not on medication in the SUDs group revealed no significant differences for any model parameter (ts between −1.32 and .86, ps between .188 and .838).
3.5 Analysis of specific use disorders in combined samples
Summary statistics for computational parameters in specific disorder groups within the combined exploratory and confirmatory samples (N = 468) are shown in Table S1.9. Full results of the logistic regression models differentiating whether participants had each specific disorder relative to HCs are shown in Table S1.10. Accounting for other model parameters, these analyses found that action precision separately differentiated diagnostic status relative to HCs for those with each specific substance use and affective disorder (Wald z between −4.63 and −2.03, ps between < .001 and .042), with the exception of hallucinogen use disorder and PTSD. This was also true for those with stimulant use disorder without co-morbidities (z = −3.05, p = .002). Learning rates for wins separately predicted diagnostic status relative to HCs for those with stimulant use disorder (only marginally in those without co-morbidities; p = .079), opioid use disorder, sedative use disorder, major depressive disorder, and social anxiety disorder (z between −3.2 and −1.96, ps between .001 and .049). Learning rates for losses further predicted diagnostic status relative to HCs for those with the majority of specific disorders (including stimulant use disorders without co-morbidities; z between −3.57 and −2.08, ps between < .001 and .038), with the exception of hallucinogen use disorder, generalized anxiety disorder, panic disorder, or PTSD.
Full results of the logistic regression models predicting whether participants in the SUDs group had a specific disorder relative to other disorders are shown in Table S1.11. Accounting for other model parameters, these analyses found that greater action precision and faster learning rate for wins predicted diagnosis with stimulant use disorders vs. other disorders (z = 2.47 and 2.23, p = .014 and .026, respectively; and a marginal effect of learning rates for losses: z = 1.79, p = .073). Diagnosis with social anxiety disorder was also predicted by slower learning rates for wins relative to other disorders (z = −1.97, p = .049; and a marginal effect of learning rates for losses: z = −1.71, p = .09). Otherwise, model parameters did not differentiate between specific SUDs. Aside from social anxiety and learning rate for wins, they also did not indicate differences in SUDs with vs. without affective disorders for any model parameter.
The overall variance explained by the 5 parameters in significant logistic regression models ranged from 9% to 16% (based on pseudo-R2; for details, see Table S1.10). Figure 4 shows parameter values for those with each specific disorder and indicates which could be differentiated with the logistic regressions.
Comparison of HCs and subsets of individuals with specific SUDs/affective disorders within the combined exploratory and confirmatory samples (sample size per group is indicated within each bar; based on the groupings detailed in Table 2). Red stars indicate that, in logistic regressions, model parameters could predict whether individuals were HCs or had a specific disorder (i.e., removing individuals from analyses without the disorder in question; *p<.05, **p<.01, ***p<.001). Blue stars indicate that model parameters could further predict whether an individual had one disorder relative to other disorders (i.e., removing HCs from analyses). Statistical results are reported in Tables S1.10-S1.11. For a similar plot of general task performance (number of wins) by specific disorder group, see Figure S1.3. Legend: Stim. Only = stimulant use disorders without co-morbidities, Can. = cannabis use disorders, Op. = opioid use disorders, Alc. = alcohol use disorders, Sed. = sedative use disorders, Hal. = hallucinogen use disorders, SAD = social anxiety disorder, PTSD = post-traumatic stress disorders.
3.6 Predictive categorization
Results of predictive categorization analyses (ROCs, AUCs, accuracy, and confusion matrices) for specific disorders are shown in Figure 5. When tested on the confirmatory dataset, logistic regression models trained on the exploratory dataset varied in predictive classification accuracy (relative to HCs) between .55 and .66, depending on the specific disorder in question. AUCs ranged from .6 to .7, indicating poor to acceptable discrimination. When categorizing HCs vs. all individuals with SUDs together, accuracy = .63 and AUC = .66.
Results of logistic regression models trained on the exploratory sample and then tested on the confirmatory sample (classifying individuals as HCs vs. having a specific SUD or affective disorder). Receiver operating characteristic curves (ROCs) illustrate the true positive rate (sensitivity) vs. false positive rate (1 - specificity) for different categorization thresholds. Performance is quantified by associated area-under-the-curve (AUC) scores, reflecting how often a random sample will be assigned to the correct group with higher probability. Acceptable AUCs vary by application, but the following heuristic cutoff values have been proposed: 0.5 = No discrimination, 0.5-0.7 = Poor discrimination, 0.7-0.8 = Acceptable discrimination, 0.8-0.9= Excellent discrimination, and > 0.9 = Outstanding discrimination (37). Accuracy at a neutral threshold of 0.5 is also shown. Confusion matrices for each disorder (0 = HCs, 1 = Disorder) illustrate the numbers of accurate and inaccurate classifications for each group based on this criterion. Note that training data were balanced using frequency weights prior to training. Gen. Anxiety = Generalized Anxiety Disorder; Soc. Anxiety = Social Anxiety Disorder.
4. DISCUSSION
In this study, we found confirmatory evidence that substance users show selectively slower learning rates than HCs for negative outcomes. In contrast, previously observed differences in action precision and learning rates for positive outcomes did not replicate for most analyses in this new sample (with the exception of action precision in supplementary frequentist analyses accounting for WRAT scores). Also notable was the reduced reward sensitivity (suggesting greater directed exploration) observed within SUDs, which was not observed in our prior study.
Subsequent logistic regressions across participants in the exploratory and confirmatory samples showed that both action precision and learning rates for losses (and in some cases learning rates for wins) could jointly differentiate the presence of most specific substance use and affective disorders relative to HCs. This highlights the way a multi-dimensional computational phenotype can offer categorical information not available from summary statistics or a single parameter alone. In contrast, analogous logistic regressions found that most disorders could not be differentiated from one another, except for stimulant use disorders (which showed greater action precision and faster learning rates for wins relative to other disorders) and social anxiety disorder (slower learning rates for wins relative to other disorders). When considering co-morbid affective disorders, this general lack of differentiability further indicated, for example, that learning rates for losses and action precision did not differ in SUDs with vs. without MDD or anxiety disorders.
When logistic regressions were instead trained on the exploratory sample and tested on their ability to correctly categorize HCs vs. specific disorders in the confirmatory sample, performance was above chance but ranged from poor to acceptable discrimination across specific disorders. Thus, while the group differences we observed may offer important mechanistic insights, their potential clinical utility for diagnostic purposes should not be overstated.
Overall, these results provide strong evidence that the differences we observed in SUDs are transdiagnostic. They also provide added support for the hypothesis that individuals with SUDs adjust their behavior more slowly in the face of aversive outcomes, which could help explain continued drug use despite harmful effects. This is also supported by previous literature showing reduced sensitivity to negative outcomes (or to affective stimuli generally) in this population within other contexts (e.g., (18, 38–49)). Given this replication, there are now several future directions that should be considered. One important question, for example, is whether this difference is caused by substance use or whether it represents a pre-existing vulnerability factor. Answering this question will likely require longitudinal analyses of at-risk populations, as well as assessing potential relationships with pre-existing biological and psychological risk-factors. Another important topic to address will be whether this difference is affected by treatment and covaries with symptom severity. Randomized controlled trials assessing the effects of distinct treatments on learning rates will be necessary to shed light on this question. Additionally, neuroimaging studies designed to identify the neural correlates of these learning rate differences could highlight potential biological targets. Finally, it will be important to identify whether learning rates moderate treatment outcomes. In relation to this last point, we recently reported a 1-year follow-up study in our previous sample and unexpectedly found that slower learning rates for losses at baseline predicted greater symptom improvement in stimulant users at 1-year follow-up (20). Here we suggested that, despite potentially reducing learning from the harmful outcomes of drug use, this difference might also attenuate reactions to unpleasant aspects of the recovery process and facilitate adherence. However, as it was not an expected result, we aim to replicate this longitudinal effect in future work. If confirmed, it could indicate one concrete way in which measures of learning rate differences could be clinically informative.
Some remaining limitations are important to consider. First, our sample of SUDs was a heterogenous community sample from a particular region of the United States, the great majority of which were stimulant users, which may have limited our ability to detect effects specific to some substances and not others (with the exception of a moderate number of stimulant users that did not have co-morbidities in the combined sample), or to generalize findings to substance users in other regional or cultural contexts. It was also a treatment-seeking, abstinent sample, and thus may not generalize to non-treatment-seeking individuals. Many substance users also had affective disorders. However, parameters did not correlate with depression/anxiety symptoms and, with the exception of social anxiety disorder (slower learning rate for wins), model parameters in the combined sample could not differentiate SUDs with vs. without affective disorders. Finally, the correct interpretation is less clear for results that were significant in the previous or current sample, but not both (e.g., with respect to reduced action precision and reward sensitivity in SUDs). Future studies will therefore need to examine whether inconsistent results between samples are better understood as false positives in one sample or false negatives in the other.
In summary, this study lends added confidence to the generalizability of previous results suggesting slower learning from negative outcomes in SUDs and motivates a number of future directions to test the neurobiological basis of this difference and whether it might represent, for example, a pre-existing vulnerability factor, a predictor of treatment response, an objective severity marker, and/or a novel intervention target. These represent important next steps toward evaluating the clinical utility of this potentially important individual difference.
Software Note: All model simulations, model comparison, and parametric empirical Bayes analyses were implemented using standard routines (spm_MDP_VB_X.m, spm_BMS.m, spm_dcm_peb.m, spm_dcm_peb_bmc.m) that are available as MATLAB code in the latest version of SPM academic software: http://www.fil.ion.ucl.ac.uk/spm/. For the specific code used to build the three-armed bandit task model and fit parameters to data, see: https://github.com/rssmith33/3-armed_bandit_task_model.
Data Availability
All data produced in the present study are available upon reasonable request to the authors
Role of Funding Source
Nothing declared.
Contributors
Samuel Taylor performed analyses and wrote the initial draft of the manuscript. Claire A. Lavalley and Navid Hakimi performed analyses and edited the manuscript. Jennifer L. Stewart, Maria Ironside, Haixia Zheng, Evan White, Salvador Guinjoan, and Martin P. Paulus reviewed and edited the manuscript. Ryan Smith performed analyses, reviewed and edited the manuscript, and supervised the project. All authors have approved the final article.
Conflict of Interest
No conflict declared.
Supplementary Materials 1
Demographic Differences in Parameters for Participants between Exploratory and Confirmatory Samples
Elements of the generative model of the three-armed bandit task
Nested Models
Differences Between SUDs with vs. without DAST Scores Less than 3
Differences in Parameters for Participants between Exploratory and Confirmatory Samples
Additional Effects
PEB models in the full sample revealed strong evidence for a negative association between age and action precision and a positive association between age and reward sensitivity (pp = 1 in both cases). There was also strong evidence for a positive relationship between age and learning rate for wins (pp = 0.93). Including WRAT reading scores (in the subsample for which they were available) did not change these results (see Figure S1.1), except the association between age and learning rate for wins was not observed.
As in the PEB analyses, some effects of age were also observed in frequentist regressions. In the full sample, there was a positive association between age and reward sensitivity (t(263) = 4.29, p < .001), learning rates for wins (t(263) = 2.91, p = .004), and information insensitivity (t(263) = 2.38, p = .02), and a negative association between age and learning rates for losses (t(263) = 2.82, p = .005). For complete statistical results associated with all effects, see Table S2.3.
A similar pattern of results was also observed in the subsample with available WRAT scores for reward sensitivity, learning rates for losses, and information insensitivity (see Tables S1.6 and S2.4). Unlike in the full sample, analyses in the subsample with WRAT scores found a significant difference in action precision (p < .01), similar to that observed in our prior study. The group difference in learning rate for losses was not significant (p = .11) in this subsample. However, it was retained in the best model in the associated Bayes factor analyses (BF = 2.93).
Results of descriptive behavioral measures are reported in Table S1.7 for those with WRAT scores. Here found fewer win/stay choices in SUDs, which was not observed previously. As in our prior study, we also repeated these analyses for the first and second halves of each game separately (i.e., first 7 choices vs. subsequent choices) to assess periods where exploration vs. exploitation would be expected to dominate, respectively (see Table S1.8). The subsample including WRAT scores showed group differences in win/stay (greater in HCs) and win/shift (greater in SUDs) choices during both early and late choice. Unlike in the full sample, number of wins did not differ between groups.
In the full sample, there were positive associations between age and both win/stay (t(263) = 5.48, p < .001) and lose/stay (t(263) = 4.22, p < .001) choices, and negative associations between age and both win/shift (t(263) = 6.01, p < .001) and lose/shift (t(263) = 4.29, p < .001) choices. There were also effects of sex on both win/stay (greater in females; t(263) = 2.21, p = .03) and win/shift (greater in males; t(263) = 2.20, p = .03) choices, but no effects on lose/stay or lose/shift choices. The same pattern of results was also seen when including WRAT scores in the subset of participants in which they were available; WRAT scores were not associated with any variable (see Table S2.3 and S2.4).
Group Differences in Parameters for Participants with WRAT Scores
Descriptive Behavioral Measures for Participants with WRAT Scores
Choice Behavior in Early vs. Late Trials
Summary statistics (mean and SD) for parameters by specific disorder group in combined exploratory and confirmatory samples
Logistic regressions predicting specific disorder vs. HCs (N = 153) in combined exploratory and confirmatory samples
Logistic regressions predicting each specific disorder vs. all other disorders in combined samples
Supplementary Figures
The top-left panel depicts results from Parametric Empirical Bayes (PEB) analyses showing the posterior means and credible intervals for effects of group (when accounting effects of age, sex, and WRAT reading scores). Positive effects indicate greater values in HCs relative to SUDs. Learning rates are estimated/displayed in logit space, while all other parameters are in log space. Results replicate the group difference in learning rate for losses found in our prior study (posterior probability [pp] = 0.99). There was also some evidence for differences in reward sensitivity not observed previously (greater in HCs; pp =0.72). All other panels compare the means and standard errors for each parameter in our prior study (exploratory sample; 54 HCs, 147 SUDs) to those in the current study (confirmatory sample, including only individuals with available WRAT reading scores; 87 HCs, 120 SUDs), when separating parameters by group. Also shown are Bayes factors (BFs) evaluating the evidence for differences between the two samples (BF < 1 indicates greater evidence for the absence of a difference). As can be seen, the BFs range from.14 to .22, indicating that the data are between 4.5 and 7.1 times more likely under a model with no difference between samples, which supports the idea that the new sample successfully replicates the findings in our prior study. For an analogous plot when including all participants in the confirmatory sample, see Figure 3 in the main text.
Correlations between symptom measures (rows) and model parameters (columns). No significant associations were observed. This failed to replicate the relationships observed in our previous study between PHQ/OASIS and information insensitivity, and between OASIS and reward sensitivity.
Task performance (number of wins) divided by group across the combined exploratory and confirmatory samples. Each column includes all participants who had a particular diagnosis. Thus, there is partial overlap across groups (with the exception of 55 individuals who had stimulant use disorder without co-morbidities; second bar from the left). These descriptive results suggest that the significantly greater wins seen in HCs than SUDs overall reflect a similar pattern across individuals with different SUDs.
Supplementary Materials 2: Detailed Linear Model Statistics for All Effects
Table S2.1 and S2.2 below provide full statistics for all effects associated with results reported in Table S1.5 in Supplementary Materials 1.
Tables S2.3 and S2.4 below provide full statistics for all effects associated with results reported in Tables 4 and 5 in the main text and Tables S1.6 and S1.7 in Supplementary Materials 1.
Full Linear Model Statistics Comparing HCs Between Exploratory and Confirmatory Samples
Full Linear Model Statistics Comparing SUDs Between Exploratory and Confirmatory Samples
Full Linear Model Statistics (full confirmatory sample)
Full Linear Model Statistics (confirmatory sub-sample with WRAT scores)
Acknowledgment
This study was supported by the Laureate Institute for Brain Research.
Footnotes
Ethical approval: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent: Informed consent was obtained from all individual participants included in the study.
Funding: This work was funded by the NIGMS (P20 GM121312; PI:MPP) and the Laureate Institute for Brain Research.
Conflict of Interest: None of the authors have any conflicts of interest to disclose.
References
