Abstract
Elevated anxiety and uncertainty avoidance are known to exacerbate maladaptive choice in individuals with affective disorders. However, the differential roles of state vs. trait anxiety remain unclear, and underlying computational mechanisms have not been thoroughly characterized. In the present study, we investigated how a somatic (interoceptive) state anxiety induction influences learning and decision-making under uncertainty in individuals with clinically significant levels of trait anxiety. A sample of 58 healthy comparisons (HCs) and 61 individuals with affective disorders (iADs; i.e., depression and/or anxiety) completed a previously validated explore-exploit decision task, with and without an added breathing resistance manipulation designed to induce state anxiety. Computational modeling revealed a pattern in which iADs showed greater information-seeking (i.e., directed exploration; Cohen’s d=.39, p=.039) in resting conditions, but that this was reduced by the anxiety induction. The affective disorders group also showed slower learning rates across conditions (Cohen’s d=.52, p=.003), suggesting more persistent uncertainty. These findings highlight a complex interplay between trait anxiety and state anxiety. Specifically, while elevated trait anxiety is associated with persistent uncertainty, acute somatic anxiety can paradoxically curtail exploratory behaviors, potentially reinforcing maladaptive decision-making patterns in affective disorders.
Introduction
Persistent uncertainty and maladaptive avoidance are key maintenance factors in anxiety disorders and a major focus of psychotherapy (1, 2). While considerable progress has been made in understanding the neural and cognitive mechanisms associated with anxiety, underlying computational processes have only been examined in a limited number of studies to date (3, 4). For example, subclinical levels of trait anxiety have been associated with reduced flexibility in learning rates when there is a change in the stability of environmental statistics (5), and individuals with anxiety disorders show elevated learning rates in general (6), suggesting a belief that action-outcome contingencies often change unexpectedly. Both depression and anxiety have also been associated with elevated learning from punishment in particular (7). More recently, trait anxiety has been linked to a greater tendency to infer changes in the underlying causes of aversive outcomes during extinction learning – which may facilitate return of fear and reduce the long-term efficacy of behavioral therapies (8).
While such studies provide important insights into the learning processes that may contribute to depression and anxiety disorders, they do not fully account for avoidance or other behaviors driven by intolerance of uncertainty. There could also be multiple computational mechanisms underlying avoidance behavior, each representing distinct hypotheses and possible treatment targets. For example, individuals must often seek information to learn that a feared situation is tolerable. Avoidance prevents such learning, which can in turn maintain avoidance. This speaks to the explore-exploit dilemma (9, 10), which has begun to receive attention in psychiatry and substance use research (11–13). This dilemma reflects the need to judge whether one has sufficient information to maximize reward, or whether one should first seek more information. In avoidance, an individual may hold the confident belief that avoided situations are dangerous, while, in fact, exploration would allow them to learn otherwise. Yet, there are multiple types of exploration, and factors that could deter exploration, which have not been thoroughly evaluated.
To date, only a few studies have examined the relationship between affective disorders and information-seeking. For example, one study found that ‘directed’ exploration (DE) in a community sample was lower in those with higher anxiety and depression symptoms (14), while another study associated lower DE with greater somatic anxiety in particular (15). This type of exploration is strategically directed toward situations and actions for which an individual has had fewer past experiences. Further studies examining effects of traumatic/unpredictable childhood environments (16, 17) – which often correlate with affective disorders (18–20) – also suggest negative effects on DE. Other supportive work has shown that: higher stress/anxiety is associated with less exploration in a virtual-reality plus-maze (21), agoraphobia and anxiety sensitivity are associated with less exploratory behavior (22), and increases in cortisol in response to an acute stressor and scores on a chronic stress questionnaire are both associated with under-exploration in foraging tasks (23). Aversive arousal states are also generally known to reduce cognitive control network activity in neuroscience studies (24–28), consistent with reduced cognitive reflection tendencies seen in those who display lower DE (14). Notably, a distinct exploratory strategy – ‘random’ exploration (i.e., where choices become less reward-driven as a means of gaining information) – has not shown such associations.
Past research therefore suggests that anxiety may selectively reduce DE, consistent with maintained avoidance. Yet, these studies are largely correlational and have tended to focus on trait anxiety, and they have primarily investigated sub-clinical symptoms in community samples. There are also some reasons to expect relationships in the oppositive direction. Namely, the cognitive aspects of anxiety involve elevated uncertainty and worry, which could promote over-exploration. Indeed, intolerance of uncertainty could drive over-exploration as a means of continually attempting to reduce it. This is consistent with some work instead suggesting greater DE in those with higher trait/cognitive aspects of anxiety (29, 30) and increased exploratory behavior in depression (30, 31). Combined with the work reviewed above, this suggests the possibility that somatic anxiety might reduce exploration, while uncertainty-or worry-related cognitive anxiety could instead promote it.
Here we sought to build on this previous work by manipulating state anxiety with a somatic anxiety induction while individuals performed an explore-exploit task. By comparing exploration and learning with vs. without this induction, while also gathering information about trait anxiety and other clinical, cognitive, and affective dimensions, we aimed to disentangle the role of state- and trait-related anxiety and relationships with cognitive vs. somatic aspects of anxiety. By doing this in both healthy comparison (i.e., low-anxiety) and clinical (i.e., high-anxiety) groups, we also sought to clarify potential differences between subclinical variation in anxiety and that associated with psychopathology. We hypothesized that anxiety induction would reduce DE and that somatic vs. cognitive components of anxiety sensitivity might show negative and positive relationships with exploration, respectively. Given the mixed results described above with respect to trait anxiety, we did not have a confident directional hypothesis regarding group differences. As secondary aims, we also sought to reproduce the aforementioned relationships found between DE and both cognitive reflectiveness (14) and early adversity (16, 17), as these associations could offer additional explanatory power if present in our sample.
Methods
Participants
Data were collected as part of a larger, multi-visit study to investigate the cognitive and neural correlates of psychiatric disorders at the Laureate Institute for Brain Research (LIBR), with participants recruited from the community in and surrounding Tulsa, OK, USA. Clinical diagnoses were assessed by a licensed clinician according to the Mini International Neuropsychiatric Interview 7 (MINI; (32)). Healthy comparisons (HCs) were not previously diagnosed with, or treated for, any mental health disorder and had a score of < 8 on the Overall Anxiety Severity and Impairment Scale (OASIS; (33)). Individuals within the high-anxiety affective disorders group (iADs) had a current diagnosis of one or more anxiety or depressive disorders and scored ≥ 8 on the OASIS in a pre-visit screening. Any iADs who were taking psychiatric medications (e.g., SSRIs, SNRIs, etc.) were required to have a stable dosage for at least 6 weeks prior to study entry. No other neurological medications (e.g., antipsychotics, stimulants, or anticonvulsive medications) were allowed; nor were the following diagnoses: bipolar disorder, personality disorders, substance use disorders, eating disorders, schizophrenia, or obsessive-compulsive disorder. Recruitment aimed to match participants in the two groups by age, sex, and education level. Participants were asked not to consume any drugs (e.g., benzodiazepines, marijuana) or alcohol in the 48 hours preceding participation and were required to pass a drug panel during their study visit.
Seven individuals withdrew from participation due to discomfort with the anxiety induction paradigm (2 HCs, 5 iADs, all female; details in Supplementary Materials), leading to a final sample of 119 participants: 58 HCs, 61 iADs (see Table 1 below). A power analysis (using the wp.rmanova function in the WebPower package in R (34)) indicated that this sample size would afford 80% power to detect a medium effect size of 
for the effect of the anxiety induction.
It is important to note here that data from the healthy sample in this study was previously used for comparison to a different clinical population (methamphetamine use disorders) (45). Here, we instead focus on differences between this group and iADs.
Measures
Primary measures were chosen to assess state and trait anxiety (State-Trait Anxiety Inventory [STAI]; (35)) and to account for clinical symptoms of depression (Patient-Health Questionnaire [PHQ]; (36)). Our secondary aim of replicating prior work motivated inclusion of measures to test links between DE and both childhood adversity (Childhood Trauma Questionnaire [CTQ]; (37)) and cognitive reflectiveness (Cognitive Reflection Test [CRT]; (14, 16, 17)) – the tendency to ‘think things through’ before responding based on intuition. Detailed descriptions of each measure are included in Supplementary Methods. Descriptive information for all participants is shown in Table 1, as well as preliminary group comparisons. Additional sample demographics are included in Supplementary Table S1 and a diagnostic breakdown for iADs is shown in Supplementary Table S2.
Somatic Anxiety Induction and Sensitivity Assessment
To manipulate state anxiety, we utilized a previously established interoceptive (breathing-based) anxiety induction paradigm (38–40). In this paradigm, participants are asked to breathe through a silicon mask attached to a valve (Figure 1a) that allows application of different levels of inspiratory resistance, creating air hunger-related sensations that induce anxiety. To provide participants a chance to familiarize themselves with the paradigm, and assess baseline sensitivity, participants were first exposed to a series of resistances in ascending order (0, 10, 20, 40, 60, and 80 cmH2O/L/sec) applied for 60 seconds each. Participants were asked to rate their anxiety level after each exposure from 0 (“no anxiety”) to 10 (“maximum possible anxiety”). Other questions relating to subjective breathing difficulty, arousal, and other affective states were also presented (see Supplementary Methods).
A) Equipment used for anxiety induction including silicon mask with adjustable straps and single breathing port, and an example resistor used to create resistance on inhalation. B) Graphical depiction of the Horizon Task. Shown are the two horizon conditions (H1 and H6) at the last forced choice (red box) and the first free choice (green boxes). The H1 example also shows an equal information trial (2 outcomes for each choice), while the H6 example shows an unequal information trial (3 forced choices on the right, 1 on the left. C) Table outlining each of the trial type combinations, counterbalanced across the task.
This induction was later applied using a moderate resistance level (40 cmH2O/L/sec) during performance of one run of the behavioral task. This level of resistance was chosen because of prior work demonstrating its effectiveness and feasibility (41); we also confirmed that it induced moderate (roughly 5 out of 10) levels of anxiety within our clinical sample (Supplementary Figure S1).
Behavioral Task
Participants completed two runs of the Horizon Task (42), a previously validated task used to measure explore-exploit decision-making. During each task run, players repeatedly choose between two slot machines in 80 games. Each game involves either 5 or 10 sequential choices. The first four choices in all games are “forced”, meaning the player is told which option to choose (see Figure 1b). The remaining choices are “free”, where either option can be chosen. Trials with one free choice (H1) and those with six free choices (H6) appear an equal number of times throughout the task.
Forced-choice patterns create two different information conditions: equal, in which each option is chosen twice; and unequal, in which one option is chosen three times and the other is chosen only one time. The goal of the task is to maximize the number of points won by repeatedly choosing the option expected to provide higher point values on average. Point values were sampled (and rounded to the nearest integer) from a Gaussian distribution with a mean of either of 40 or 60 on one option (standard deviation of 8). Then, for each game, the other option had a mean point value difference of +/- 4, 8, 12, 20, or 30 from the first. The same pseudorandom sequences of forced-choice outcomes were shown to each participant, consistent with the generative means. All game dimensions (length, information type, mean difference, and better/worse side) were counterbalanced across the task (Figure 1c).
Participants wore the mask for each task run. In the no-resistance run, resistance was not added (0 cmH2O/L/sec). In the resistance run, a moderate resistance (40 cmH2O/L/sec) was consistently applied throughout the run. The order was counterbalanced across participants in each group. The first six participants played a version of the task with different outcome values than the rest in a small number of games (but still sampled from the same underlying distributions). To ensure this did not confound results below, this difference in task version was evaluated as a potential covariate.
Computational Model and Model Fitting
The computational model used here is outlined in Zajkowski, Kossut (43). We present this in detail in Supplementary Methods. Brief definitions of each computational parameter are provided in Table 2.
Statistical Analyses
Resistance sensitivity
To verify that the anxiety induction was effective, we examined affective responses to each resistance level in the initial exposure protocol. To this end, we estimated linear mixed effects models (LMEs) with self-reported anxiety (i.e., the single anxiety question on a scale from 0 to 10) as the outcome variable and resistance level and group as predictors (and their interaction).
Primary model-based analyses
Before addressing our primary hypotheses, we checked for outliers using an iterative Grubbs method (threshold: p<.01; using grubbs.test from the outliers package in R (44)). This resulted in one data point being removed for IB1 (in the resistance condition), two for DN6 in the equal information condition (one in each resistance condition), and two for RE (one in each resistance condition). Then, we estimated LMEs that included group, resistance condition, and their interaction as sum-coded predictors for each model parameter (HCs=-1, iADs=1; no-resistance=-1, resistance=1). All models predicting α∞ also accounted for α0 values. To rule out possible alternative explanations, we also re-ran these models including covariates to confirm that any group or resistance effects could not be explained by age, sex (Male=-1, Female=1), or task version (new=-1, old=1).
We then assessed continuous relationships between computational parameters and affective symptoms. We first tested separate models using PHQ scores or STAI Trait scores. These LMEs contained interactions with resistance condition and any covariates found to be significant predictors in the group models. Then, we included both symptom measures in the same models (and their interactions with resistance) to establish potentially independent effects.
Model-free task performance
To better interpret observed effects on task behavior and assess relationships between task performance and model parameters, we also performed additional supportive analyses examining overall accuracy on free choices (as measured by the number of times participants chose the option with the higher underlying mean reward). We first restricted analyses to first free choice and estimated an LME predicting accuracy based on horizon, information condition, resistance, and group as predictors. We also included possible three-way interactions between group, horizon, and information condition, and between group, horizon, and resistance (and associated two-way interactions). This allowed us to explore whether groups might show greater differences depending on horizon, information condition, or anxiety induction. As commonly observed in this task, we expected accuracy in H1 trials would be higher than in H6 trials (i.e., reflecting random exploration).
A subsequent model was run predicting accuracy across free choice trials in H6 (choices 5-10). We also tested three-way interactions between group, free choice number, and information condition, and between group, free choice number, and resistance (and associated two-way interactions) to explore whether groups might differ in the slope of accuracy improvement over time depending on information condition or anxiety induction.
To evaluate whether some parameter values might be considered more optimal than others, we then tested LMEs with parameters predicting accuracy on H6 free choice trials, excluding the first free choice to which model parameters were fit. As information bonus (IB) is only calculated in the unequal information condition, accuracy on the unequal information trials alone was predicted in these models, with resistance condition, group, free choice number, IB for H6 (IB6), and the interaction between free choice number and IB6 as predictors. Analogous models were used to test relationships between accuracy and the other model parameters within the relevant trial types (i.e., decision noise for H6 with equal information condition accuracy; learning rates and accuracy in both information conditions).
Secondary replication analyses
Additional LMEs were run to accomplish our secondary aim of replicating prior relationships found between DE and both early adversity (16, 17) and cognitive reflectiveness (14). First, separate LMEs were run predicting DE based on each subscale of the CTQ, while also including group, resistance condition, and their interaction as predictors (while controlling for known effects of age). Analogous LMEs were run instead using CRT scores as predictors.
Results
The Breathing Resistance Protocol Successfully Induced Anxiety During Task Performance
Anxiety ratings during the initial breathing resistance sensitivity protocol are shown in Supplementary Figure S1. Briefly, anxiety increased as the resistance level increased, anxiety ratings were higher in iADs than HCs, and iADs showed greater increases in anxiety compared to HCs as resistance levels increased. LMEs predicting self-reported anxiety scores at baseline and during both task runs showed the same pattern (Figure 2). See Supplementary Materials for full results.
Self-reported anxiety and STAI State at baseline and during each run of the Horizon Task (i.e., with and without the added breathing resistance of 40 cmH2O/L/sec). Results showed greater anxiety in iADs, greater anxiety with higher resistance level, and greater increases in anxiety in iADs than HCs when resistance was added. Boxplots show median and quartile values along with individual datapoints.
Directed Exploration was Elevated in Affective Disorders Patients, but Reduced by State Anxiety Induction
Plots depicting parameter values by group and condition are shown in Figure 3. In an initial LME predicting DE, there was a significant Group x Resistance Condition interaction 
, such that iADs showed higher DE in the no-resistance condition (estimated marginal mean [EMM]=5.69) than in the resistance condition (EMM=4.89; t(117)=-2.09, p=.039), while DE in HCs remained at lower values and did not change between conditions (t(117)=0.75, p=.456). Further exploration suggested this was driven primarily by greater IB values in iADs compared to HCs in H6 games within the no-resistance condition (see Supplementary Figure S2). Main effects of group or resistance were nonsignificant. In a subsequent model accounting for age, sex, and task version, this interaction remained significant 
. There was also a negative association with age 
.
Individual data points and density plots by group and resistance condition for each parameter. HCs showed stable DE values in both conditions. Those with ADs instead showed elevated DE in the no-resistance condition, which then decreased after application of the resistance. For illustrative purposes, the individual information bonus (IB) values underlying DE for each horizon condition (H1 and H6) are shown in the top-right (recall that DE corresponds to the difference in information bonuses between horizon conditions; DE = IB6 – IB1). As can be seen, the H6 information bonus is higher in the no-resistance condition than in the resistance condition for iADs only. RE did not differ by group or condition (bottom left). Learning rates tended to show lower values in iADs than HCs across conditions. As visible in the lower middle panel, α0 showed a bimodal distribution, which motivated the cluster-based analysis approach described in the text. Asymptotic learning rates (α∞) were significantly lower in iADs than HCs in both conditions (bottom right). Note that stars and Cohen’s d effect sizes correspond to the interaction between group and resistance for DE (upper left) and for the group difference in α∞ across resistance conditions (bottom right) in post-hoc comparisons.
No significant effects were found in LMEs predicting RE (Fs≤3.45, ps≥.067).
Because of bimodality in the distributions for α0 (see Figure 3), we used k-means clustering (46) across resistance conditions to divide participants into groups with high vs. low values and treated this as a 2-level categorical variable (low=-1; high=1). In logistic regression models predicting high vs. low α0 values (using glmer in the lme4 package; (47)), there were no significant effects before or after adding covariates (zs≤|1.60|, ps≥.110).
The LME predicting α∞ (accounting for α0) revealed a significant main effect of group (F(1,117)=7.73, p=.006, 
; see Figure 3), such that HCs (EMM=.33) had higher learning rates than iADs (EMM=.28, t(117)=-2.79, p=.006). The expected positive association with α0 was also observed 
. Both effects remained significant when accounting for covariates, and no additional effects were observed (Fs≤1.73, ps≥.192).
We also evaluated within-group associations with symptom severity. This was done to explore potentially differential effects of depression and anxiety, noting that associated measures (PHQ and STAI Trait) were highly correlated (rs=.83-.86; see Supplementary Figure S3). In models predicting DE in iADs alone (based on PHQ, STAI Trait, or both measures in a single model) no significant effects were found. However, marginal effects of resistance condition recapitulated the Group x Resistance interaction effects observed in our primary analyses above (Fs≥3.76, .05<p≤.058).
Within HCs, models predicting DE with either PHQ or STAI Trait showed no significant effects. However, when both symptom measures were included in the same model, there was a significant interaction between resistance condition and STAI Trait 
. This reflected a greater positive association between anxiety and DE in the no-resistance condition than in the resistance condition (no-resistance: estimated trend [ET]=.236; resistance: ET=.072; t(55)=2.34, p=.023), with a nonsignificant positive association with STAI Trait overall 
. PHQ instead showed a marginal negative association with DE that interacted with resistance 
, suggesting this negative relationship was stronger in the no-resistance condition (no-resistance: ET=-.484; resistance: ET=-.060; t(55)=-1.85, p=.070).
Equivalent analyses were also carried out for other model parameters, which suggested potential opposing associations between PHQ, STAI Trait, and RE when included in the same model (detailed in Supplementary Results).
Higher Levels of Directed and Random Exploration are Associated with Greater Task Performance
In LMEs predicting H6 accuracy (on choices 6-10) using each model parameter, we found main effects of learning rates (Table 3). Specifically, α0 predicted accuracy on equal information trials, while α∞ predicted accuracy on unequal information trials. Additionally, there was an interaction between IB6 and free choice number indicating that, in unequal information trials, those with higher IB6 showed steeper increases in accuracy from early to later choices. Similar results were found for decision noise when predicting accuracy on equal information trials.
We also tested an analogous model using DE as a predictor of free choice accuracy and saw comparable results to those seen for IB6 (i.e., main effect of free choice number and a significant interaction with DE; Fs≥9.69, ps≤.002). Similarly, results when using RE to predict equal information accuracy corroborated those found for decision noise (main effect of free choice: F(1,1057)=114.74, p<.001; RE x free choice: F(1,1057)=15.78, p<.001; main effect of group: F(1,117)=5.67, p=.019). Finally, motivated by the effect of resistance on DE observed above in iADs, we tested a model including a DE x Resistance Condition interaction term. This revealed a significant interaction (F(1,1089)=4.46, p=.035), such that greater DE was positively predictive of accuracy in the resistance condition (ET=0.002) but not in the no-resistance condition (ET=-0.001; t(1089)=2.11, p=.035). Jointly, these results suggest higher values for DE were more optimal.
Task Performance was Reduced in Affective Disorders in Specific Conditions
Secondary analyses of free choice accuracy indicated high performance across both groups (HCs: M=.84, SD=.13; iADs: M=.80, SD=.16; see Supplementary Figure S4). LMEs predicting first free choice accuracy (Supplementary Table S3) revealed significant effects of horizon type (greater accuracy in H1; F(1,823)=278.81, p<.001) and information condition (greater accuracy in equal information trials; F(1,823)=72.08, p<.001). The Group x Horizon x Information Condition interaction was also significant (F(1,823)=4.18, p=.041), indicating that HCs had significantly greater accuracy than iADs in H1/equal information trials and in H6/unequal information trials only (see post-hoc contrasts). Subsequent LMEs predicting accuracy across free choices in H6 showed significant effects of choice number (greater accuracy in later choices; F(1,2727)=201.30, p<.001), information condition (greater accuracy in equal information trials; F(1,2727)=51.37, p<.001), and a Group x Information Condition interaction (greater accuracy in HCs only in the equal information condition; F(1,2727)=8.93, p=.003).
Replication Analyses Confirm Associations with Early Adversity and Cognitive Reflectiveness
CTQ scores were greater in iADs than HCs (t(112)=5.31, p<.001). In separate LMEs predicting DE with each subscale of the CTQ, group, resistance condition, and their interaction as predictors (also controlling for age), and after correcting for multiple comparisons (p≤.01), there was a significant negative association with physical abuse (b=-.26; F(1,109)=6.96, p=.010, η2p=.06) such that exploration was lower in those who experienced greater physical abuse. This remained significant in iADs alone 
. No effects were observed for any other parameter.
CRT scores were lower in iADs than HCs (t(117)=3.05, p=.003). In an LME assessing the effect of CRT on DE across all participants, there was a significant positive association 
. The effect of CRT was directionally the same in iADs alone (and stronger in effect size), but only marginally significant due to smaller sample size 
. There were no significant effects of CRT on RE or α in either the full sample or in the clinical group alone (ps>.054). CRT scores positively predicted α0 cluster membership in the full sample (z=-2.91, p=.004; log-odds=-.41, CI=[−0.68,−0.13]), but not in iADs alone (z=-1.57, p=.115; log-odds=-.38, CI=[−0.86,0.09])).
Discussion
In this study, we compared decision-making behavior on an explore-exploit task in participants with and without clinically elevated levels of trait anxiety and depression. A somatic (interoceptive) anxiety induction was also used to allow dissociation of the influences of state vs. trait anxiety. We hypothesized that state anxiety induction would reduce direction exploration (DE) as a potential mechanism promoting avoidance behavior. Results were mixed, offering only partial support for our primary hypotheses. In particular, there was a significant interaction in which elevated DE in iADs at baseline was reduced by the anxiety induction, while DE in HCs remained stable between conditions. The elevated DE at baseline suggests more persistent and exaggerated uncertainty in iADs, similar to that observed in some previous work in non-clinical samples (29). The reduction in DE after anxiety induction also appears consistent with correlational work suggesting anxiety and depression are associated with less exploration (14), as well as with work showing that stress reduces exploration (21–23). It is also somewhat in line with a previous study showing lower DE in those with higher trait somatic anxiety (15). Thus, our results show partial consistency with findings of multiple previous studies and suggest a general differentiation between state and trait effects.
While the reduction in DE with state anxiety induction in iADs is consistent with our initial hypothesis, the expected decrease in HCs was not observed. This suggests the change in iADs could be viewed as a normalization toward healthy levels. On the other hand, higher levels of DE were also associated with steeper improvements in task performance over time, suggesting the anxiety induction had a maladaptive effect. This interpretation is also not entirely supported, however, as choice accuracy was not significantly different between resistance conditions, and task performance was marginally worse in iADs overall. This would therefore suggest greater exploratory behavior in the no-resistance condition did not in fact improve performance, or that there were other counteracting factors (e.g., differences in learning rates). Thus, differences in DE did not relate to optimality in task performance in a clear way.
The finding that iADs showed slower asymptotic learning rates across conditions may offer some additional explanatory insights. First, depression and anxiety symptoms (and particularly trait anxiety levels in the no-resistance condition) were negatively associated with asymptotic learning rates, while these learning rates were positively associated with task performance. This suggested that slower learning may have contributed to worse performance in iADs. Here, slower learning implies that beliefs remained closer to uninformative prior values; thus, confidence in the better choice would increase more slowly (similar to less flexible learning rates previously associated with higher anxiety; (5)). Within this task, slower learning rates might thus support a type of persistent uncertainty about the best course of action, with suboptimal effects on decision-making. Interpreting this result in light of previous findings is subtle, however, as faster learning rates are theoretically linked to greater uncertainty, and prior work has linked anxiety to both faster learning (6) and a greater tendency to infer changes in context (8). On the other hand, the type of uncertainty in these studies pertains to volatility, or how frequently environmental contingencies are expected to change. Thus, if anxious individuals believe the world is ever-changing, then learning rates should be high.
However, if “uncertainty” instead pertains to the estimated stochasticity of (i.e., noise in) the mapping from underlying states to observations, learning rate should instead be low, so as not to overfit beliefs to random outcomes (48). Thus, in the present task, slower learning in iADs could represent greater uncertainty about the informativeness of each outcome when inferring the underlying reward mean. This could offer a complementary means by which uncertainty is maintained in anxious individuals, consistent with the previous findings reviewed above. It could also relate to another recent study showing that individuals who experienced greater early adversity, itself a predictor of subsequent affective disorders (49–51), also showed a slower learning rate (16). It should be kept in mind, however, that our sample includes individuals with affective disorders, while most prior work has examined sub-clinical anxiety levels. Thus, some results between studies may not be fully comparable. This was suggested by our follow-up results in which, for HCs, DE showed potentially opposing positive associations with trait anxiety and negative associations with depression in the no-resistance condition (i.e., when both symptom measures were included in the same models), while no such relationships were found in the clinical sample.
In line with our secondary aims, we were also able to successfully replicate prior results showing higher DE in those with greater cognitive reflectiveness (14) and lower DE in those who experienced greater childhood adversity (physical abuse) and environmental unpredictability (16, 17). These findings suggest that less reflection on uncertainty and greater exposure to unpredictable early environments could each contribute to clinical differences seen here in exploration and learning. It is also worth noting that cognitive reflectiveness has been shown to improve with training (52–54). Thus, future studies could examine whether improving reflectiveness also optimizes exploratory behavior or reduces affective symptoms. Another consideration is that multiple previous studies have shown strong relationships between physical abuse in childhood and later avoidance behavior (55, 56). Results here suggest associated differences in DE could contribute to this avoidance behavior, perhaps especially in relation to somatic anxiety and physical abuse, potentially highlighting DE as a possible treatment target for those recovering from childhood trauma.
There are important limitations to consider. First, the breathing resistance only induced mild anxiety in HCs and moderate anxiety in iADs. While we intentionally chose a tolerable level of resistance for patients, it is possible that effects would be larger with a stronger induction approach. Additionally, the sample size was only moderate and may not have afforded sufficient power to detect some effects within the clinical group alone. Future research will also be needed to see whether results generalize to other explore-exploit tasks as well as tasks designed to distinguish learning rates in relation to volatility vs. stochasticity (48).
Conclusions
The results of this study suggest elevated directed exploration and slower learning rates in individuals with affective disorders, as well as a causal role of somatically focused state anxiety in reducing directed exploration in this population. It also replicates potential links to cognitive reflection and early adversity that may be of clinical relevance. These findings highlight potential computational mechanisms underlying both persistent uncertainty and maladaptive avoidance. If confirmed in future work, this could suggest potential benefits of treatments aimed at optimizing levels of information-seeking and belief testing in relation to current affective states as well as adjusting beliefs about the reliability of new experiences in revising expectations.
Data Availability
All data produced in the present study are available upon reasonable request to the authors.
Conflict of interest or competing financial interests
The authors have no competing interests to disclose.
Funding
This project was funded by National Institute of General Medical Sciences (P20GM121312 [R.S. and M.P.P.]) and the Laureate Institute for Brain Research.
Author contributions
NL and CAL took the lead in data analysis and writing the manuscript. RS designed the study, planned and oversaw analyses, and played a major role in writing the manuscript. ST, AEC, CMG, and TT assisted analyses and in data collection. All authors contributed to the manuscript.
Supplementary Methods
Supplementary Methods
Participant Withdrawals
The sensitivity assessment served not only as a way to obtain individual anxiety ratings to the breathing resistances, but also to allow initial exposure to the mask before participants began the Horizon Task. Seven participants who consented for the study withdrew either during or after the sensitivity assessment because of discomfort with the breathing mask. Of the 7 individuals who consented for this project and withdrew before completion due to tolerability of the breathing resistance, 5 were iADs and 2 were HCs. The iADs who withdrew were numerically older (M=44, SD=14.09) than those who did not and had comparable OASIS scores (M=8.5, SD=1.29) and numerically lower PHQ scores (M=8.25, SD=4.79). All 5 were diagnosed with MDD, 4 GAD, 1 PTSD, and 1 social anxiety disorder. Two were taking psychiatric medications. All 7 participants who withdrew were female. The small number of individuals who withdrew did not allow for formal statistical comparison to those who remained in the study.
Measure Details
State-Trait Anxiety Inventory (STAI-State/Trait)
The STAI consists of two sets of 20 items designed to assess state and trait anxiety, respectively. Respondents were asked to rate each item on a scale of 1 (“not at all” for state anxiety; “almost never” for trait anxiety) to 4 (“very much so” for state anxiety; “almost always” for trait anxiety) based on how much they agree or disagree with the associated statement. The STAI has shown good to excellent internal consistency (Cronbach’s alphas ranging from .86 to .96; (38)).
Overall Anxiety Severity and Impairment Scale (OASIS)
The OASIS is a 5-item self-report measure designed to assess anxiety-related impairment across various areas of life over the past week. Total scores are obtained by summing scores of individual items (0–4), ranging from 0 to 20, where higher scores indicate greater impairment. The OASIS has shown a high internal consistency, with a Cronbach’s alpha of .80 (39).
Patient Health Questionnaire (PHQ-9)
The PHQ is a screening tool for depression and includes 9 self-report items, which correspond to the diagnostic criteria of major depressive disorder. Each item statement is rated on a 4-point Likert Scale, ranging from 0 (not at all) to 3 (nearly every day) to describe the extent to which a depressive symptom has bothered them in the past 2 weeks. The total score reflects level of depression severity. The PHQ-9 has been found to show good internal consistency (α = .89; (40)).
Childhood Trauma Questionnaire (CTQ)
The CTQ short form (41) includes 28 self-report items that measure childhood abuse and neglect through five subscales: physical abuse, emotional abuse, sexual abuse, physical neglect, and emotional neglect. Each item is scored using a Likert scale, ranging from 1 (“never true”) to 5 (“very often true”), with relevant items summed to provide each subscale score. The total score is the mean of the subscale scores.
Higher scores indicate higher levels of childhood abuse and neglect.
Cognitive Reflection Test (CRT-7)
The CRT-7 (42) is comprised of 7 short questions that are designed to assess trait cognitive reflectiveness. Lower reflectiveness is operationalized as an individual’s tendency to provide an immediate intuitive answer that is incorrect, when more detailed reflection on the content of the question would lead to a less intuitive but correct answer. Example item: “Jerry received both the 15th highest and the 15th lowest mark in the class. How many students are in the class?” (intuitive answer: 30; correct answer: 29). Scores are based on the number of correct answers. The CRT-7 has shown acceptable reliability (α=.72; (42)).
Sensitivity Assessment
In addition to self-reported anxiety levels, participants also gave responses to a series of other related questions immediately after exposure to each breathing resistance during the sensitivity protocol (exact wording listed below). Self-Assessment Manikin items assessed happiness and excitement (1) rated on a 5-point scale, while the others were rated similarly to self-reported anxiety (11-point scale). See Figure S1 below for associated bar plots demonstrating ratings across resistance levels
“How calm or excited did you feel during loaded breathing?” (on the scale from 1 to 5)
“How happy or unhappy did you feel during loaded breathing?” (on the scale from 1 to 5)
“How difficult did it feel to breathe?” (0-no difficulty; 10-maximal difficulty you could tolerate)
“How much fear did you feel while breathing?” (0-no fear at all; 10-maximal fear you could tolerate)
“How unpleasant did it feel to breathe?” (0-not at all unpleasant; 10-maximal unpleasantness you could tolerate)
Computation Model and Model Fitting
Learning Model
The learning model is designed to reflect how the participant estimates the mean reward of both bandits during forced-choice trials. In order to estimate the mean reward values, the model, based on a Kalman filter (Kalman, 1960), assumes that rewards r are generated from a Gaussian distribution with a fixed standard deviation σr and a mean µ that differs between bandits and can change over time. Changes in the mean estimate are determined by a Gaussian random walk with mean 0 and standard deviation σd. Importantly, the dynamic mean assumptions made by this model do not reflect the true generative model of the Horizon Task (where means are constant throughout a game; i.e., σd = 0) allowing for the possibility of suboptimal learning as well as a recency bias in mean estimates.
For bandit i on trial t, estimation of mean reward 
is updated based on the reward prediction error (i.e., difference between the observed reward and prior mean), which is scaled by a time varying learning rate 
. This gives the following:
and
where
The learning rate, as shown here, is defined by uncertainty in: 1) the estimated mean of a bandit 
, 2) the drift in this mean over time (σd), and 3) the assumed standard deviation of the generative distribution (σr). If bandit i is not played on trial t, the estimated mean is assumed to stay the same, while the uncertainty in that estimate grows in accordance with σd:
The dynamic learning rate can be decomposed into two free parameters that are fit to individual behavior. The starting learning rate is defined by:
The “asymptotic” learning rate, or the value of 
if bandit i is chosen ad infinitum, is instead:
where
This reparameterization then constrains both learning rate values to [0,1].
Decision Model
After estimating the mean reward of each bandit, 
, the participant uses it to choose between two bandits probabilistically according to a logistic choice rule:
where ΔR is denoted as the differences in expected reward between two bandits and Δi is denoted as the differences in information between two bandits. Here, +1 is specified when the left bandit is more informative, −1 when the right bandit is more informative, and 0 when both bandits convey equal information. The decision process is characterized by three parameters: A (i.e., the information bonus), B (i.e., the spatial bias), and (i.e., the decision noise). Separate values of each parameter are estimated for each horizon (i.e., H1 and H6) and the spatial bias and decision noise parameters are further separated by uncertainty conditions (i.e., equal and unequal information), while information bonus is only calculated in the unequal information condition.
Model Fitting
A previously used hierarchical Bayesian model was used to fit each parameter to the behavior of each participant. In this approach to model fitting, it is assumed that each parameter for each participant is taken from the same group-level prior distribution, and is estimated using a Markov Chain Monte Carlo (MCMC) sampling procedure. Group-level prior distributions are parameterized by hyperpriors set to have broad, minimally informative distributions.
Model fitting was performed using the JAGS package (Plummer, 2003) to run Markov Chain Monte Carlo simulations in MATLAB (psiexp.ss.uci.edu/research/programs_data/jags/). This package draws samples from the posterior distribution given the observed behavioral data to approximate the posterior distribution over model parameters. Specifically, 1000 samples were generated from the posterior distribution over model parameters for each of 4 independent Markov chains (i.e., 4000 samples in total). We discarded the first 500 samples of each chain (i.e., burn-in = 500) that might impact the posterior distribution and generated posterior samples at a thin rate of 1.
Supplementary Results
Breathing Resistance Sensitivity
Anxiety ratings during the breathing resistance sensitivity protocol are shown in Figure S1. In the LME predicting anxiety including resistance level, group, and their interaction as predictors, all effects were significant (Fs≥5.77, ps<.001). Post-hoc contrasts indicated that 1) iADs had higher self-reported anxiety levels than HCs at each resistance level (ps<.013), 2) that anxiety at resistance levels 40, 60, and 80 were not significantly different from each other in HCs (ps≥.076), and 3) that each increase in resistance level resulted in a significant increase in anxiety for the iADs, but anxiety in HCs did not change from 40 through 80. These results indicate that anxiety induction was successful across participants, and that iADs showed greater sensitivity than HCs.
LMEs predicting self-reported anxiety scores at baseline and during both task runs (Figure 2) showed significant effects of group (F(1,117)=49.44, p<.001), resistance condition (F(2,234)=105.89, p<.001), and their interaction (F(2,234)=8.04, p<.001). Post-hoc analysis revealed that: 1) iADs (estimated marginal mean [EMM]=3.06) reported higher anxiety ratings than HCs (EMM=1.03; t(117)=7.03, p<.001); 2), anxiety ratings were highest during the task condition with the added resistance (EMM=3.20), followed by the task condition without the resistance (EMM=1.63) and baseline ratings (EMM=1.31; ps<.022); and 3) self-reported anxiety ratings did not increase from baseline to the no-resistance task run for HCs (EMMbaseline=0.53, EMMno-resistance=0.69; p=.440), but did for iADs (EMMbaseline=2.08, EMMno-resistance=2.57; p=.013).
Similar results were found for complementary STAI-State ratings. Here there were main effects of group (F(1,117)=54.93, p<.001) and resistance (F(2,234)=85.51, p<.001), but no interaction (F(2,234)=0.56, p=.570). Post-hoc contrasts indicated that iADs reported higher anxiety (EMM=44.9) than HCs (EMM=33.1; t(117)=7.41, p<.001) and that anxiety during the run with the added breathing resistance (EMM=44.3) was higher than anxiety at baseline (EMM=35.7) and when no resistance was added (ps<.001), but baseline was not significantly different from the no resistance condition (t(234)=1.73, p=.085).
Disentangling Depression and Anxiety for Other Parameters
In models predicting RE in iADs based on effects of depression (PHQ) alone or anxiety (STAI Trait) alone, only a negative effect of age was found (Fs≥3.92, ps≤.052). With PHQ and STAI in the same model, there was a significant interaction between PHQ and resistance condition (F(1,58)=4.40, p=.040). Post-hoc analysis indicated that this was driven by a stronger negative association between PHQ and RE in the resistance condition than in the no-resistance condition (resistance condition: ET=-.178; no-resistance condition: ET=-.016; t(58)=2.10, p=.040). The interaction between resistance condition and STAI Trait was also significant in this model (F(1,58)=4.67, p=.035). This reflected a positive association between anxiety and RE in the resistance condition and no relationship between these variables in the no-resistance condition (resistance condition: ET=.102; no-resistance condition: ET=.007; t(58)=-2.16, p=.035). There were also a marginal overall positive effect of STAI (F(1,57)=3.38, p=.071, b=.055) and negative effect of PHQ (F(1,57)=3.46, p=.068, b=-.098) driven by the above interactions.
When analogous models of RE were run in HCs, those containing only effects of depression (PHQ) or anxiety (STAI Trait) separately showed no significant effects. However, when both PHQ and STAI were included in the same model, results showed a marginal effect of STAI Trait (F(1,53)=3.30, p=.075) suggesting a possible positive association (b=.043).
In logistic regression models predicting α0 in iADs, no significant effects were found in separate or combined models. Analogous models in HCs found no significant effects when PHQ and STAI were included alone. However, in a model containing both symptom measures together, there were a couple noteworthy marginal effects. Namely, higher STAI Trait was suggestively associated with higher α0 (z=-1.93, p=.054; log-odds=-.23, CI=[−.47, .00])), and there was an interaction between STAI Trait and resistance condition (z=1.769, p=.077; log-odds=.22, CI=[-.02, .46]). This interaction suggested higher STAI scores predicted higher α0, but only in the no-resistance condition.
All analogous models predicting α∞ in iADs showed no significant effects. In HCs, there were similarly no significant effects in models including either PHQ or STAI alone. However, in a model predicting α∞ with both PHQ and STAI together, the interaction between PHQ and resistance condition was marginal (F(1,55)=2.85, p=.097), suggesting a positive association between PHQ and α∞ in the no-resistance condition (ET=.009), but a negative association in the resistance condition (ET=-.010; t(55)=1.69, p=.097). Additionally, the interaction between STAI Trait and resistance condition was marginal (F(1,55)=3.23, p=.078), suggesting a negative association between STAI and α∞ in the no-resistance condition (ET=-.001), but a positive association in the resistance condition (ET=.005; t(55)=-1.80, p=.078).
Finally, while main results of models predicting DE based on symptoms are reported in the main text, we provide some additional statistical details here in relation to secondary resistance effects. Namely, when predicting DE in iADs, specific statistics included marginal effects of resistance condition in the models with PHQ alone 
, STAI Trait alone 
, and both together 
.
Supplementary Figures
Participant ratings during the breathing resistance sensitivity protocol across 6 levels of breathing resistance (0 to 80 cmH2O/L/sec). Boxplots show medians and quartiles as well as individual datapoints.
Decision noise parameters separated by horizon, group, and resistance condition.
Post-hoc correlations between model parameters and self-report measures for the interested reader. Statistics below the diagonal indicate correlations with parameters in the resistance condition, while those above the diagonal are from the no-resistance condition. Note that the correlation between α0 and other variables were conducted using Spearman correlations due to non-normality.
Fraction of correct responses (i.e., option with higher generative mean) for each free choice in H1 and H6 trials. Color shades indicate task performance with (light) and without (dark) breathing resistance separately in the HCs (left/blue) and iADs (right/pink). Plots are also shown separately for equal (top) or unequal (bottom) information conditions.
Footnotes
A small formatting issue with regard to author names was corrected.
References
