Journal of Behavior Therapy and Experimental Psychiatry 64 (2019) 22–30

Contents lists available at ScienceDirect

Journal of Behavior Therapy and Experimental Psychiatry

journal homepage: www.elsevier.com/locate/jbtep

Difficulty in updating positive beliefs about negative cognition is associated T

with increased depressed mood
Keisuke Takanoa,∗, Julie Van Griekenb, Filip Raesb
a
Department of Psychology, Clinical Psychology and Psychotherapy, Ludwig-Maximilians-University Munich, Germany
b
Center for the Psychology of Learning and Experimental Psychopathology, University of Leuven, Belgium

A R T I C LE I N FO A B S T R A C T

Keywords: Background and objectives: Depressed people hold positive beliefs about negative cognition (e.g., rumination is
depression useful to find a solution), which may motivate those individuals to engage in sustained negative thinking.
Rumination However, in reality, rumination often leads to unfavorable outcomes. Thus, such beliefs create a large dis-
Memory crepancy between one's expectations and the actual outcome. Therefore, we hypothesized that this prediction
Reinforcement learning
error would be associated with increased depressed mood.
Q-learning
Methods: We observed how people update their positive beliefs about negative cognition within a volatile en-
vironment, in which negative cognition does not always result in a beneficial outcome. Forty-six participants
were offered two response options (retrieving a negative or positive personal memory) and subsequently pro-
vided either an economic reward or punishment. Retrieving a negative (rather than positive) memory was in-
itially reinforced, although this action-outcome contingency was reversed during the task. In the control con-
dition, positive memory retrieval was initially reinforced, although a contingency reversal was employed to
encourage negative memory retrieval.
Results: Model-based computational modeling revealed that participants who showed a delay in switching from
negative to positive (but not from positive to negative) responses experienced increased levels of depressed
mood. This delay in switching was also found to be associated with depressive symptoms and trait rumination.
Limitations: The non-clinical nature of the sample may limit the clinical implications of the results.
Conclusions: Difficulty in updating positive beliefs (or outcome predictions) for negative cognition may play an
important role in depressive symptomatology.

1. Introduction 1992). This normative positive bias could serve as a shield against
negative depressive processing, while preserving and promoting posi-
Persistent negative cognition, such as depressive rumination (Nolen- tive mental states (Mezulis, Abrahamson, Hyde & Hankin, 2004).
Hoeksema, Wisco, & Lyubomirsky, 2007), is recognized as one of the One critical issue with depressive cognition is that it is persistent
most important risk factors for depression. Rumination is defined as and uncontrollable; moreover, combined with a negativity bias it can
repetitively attending to one's feelings and problems, and is pre- progress into depressive rumination (e.g., Koster et al., 2011). The
dominantly characterized by excessive automatic negative thoughts “sticky” nature of depressive cognition can be partly explained by a
(Nolen-Hoeksema, 1991, 2000). Cognitive theories of depression em- persistent “positive” belief that negative cognitive activities lead to
phasize the role of biased emotional information processing, including beneficial outcomes. Papageorgiou and Wells (2001, 2003) have sug-
the preferential recall of negative compared to positive memories gested that people with depression tend to regard rumination as a
(Gotlib & Joormann, 2010; Mathews & MacLeod, 2005). A meta-ana- useful coping strategy to find a solution in difficult or otherwise dis-
lytic study has suggested that people with depression remember more tressing situations. These positive beliefs often appear at the beginning
negative than positive information. Conversely, non-depressed in- of an episode of persistent negative thinking, through which the in-
dividuals tend to possess a normative positivity bias, i.e., they recall dividual aims to minimize discrepancies between their expected (or
more positive than negative information (Matt, Vazques, & Campbell, desired) and actual states (Roelofs et al., 2007). This positive belief is

∗
Corresponding author. Department of Psychology, Division of Clinical Psychology and Psychotherapy, Ludwig-Maximilians-University Munich, Leopoldstr.13,
80802, Munich, Germany.
E-mail address: [email protected] (K. Takano).

https://doi.org/10.1016/j.jbtep.2019.02.001
Received 7 December 2017; Received in revised form 30 January 2019; Accepted 1 February 2019
Available online 02 February 2019
0005-7916/ © 2019 Elsevier Ltd. All rights reserved.
K. Takano et al. Journal of Behavior Therapy and Experimental Psychiatry 64 (2019) 22–30

consistent with the extant learning theory on depression, which has there was any evidence of associations between this updating difficulty
proposed that (a) negative, depressive thoughts are initially reinforced and depressive symptomatology.
by “rewarding” experiences, such as successful problem solving To test these hypotheses, we developed a laboratory model to cap-
(Watkins, 2008), and concern and compassion from others following ture one's persistent positive beliefs, or difficulty updating action-out-
negative self-disclosure (Coyne, 1976; Ramnerö, Folke, & Kanter, come predictions, regarding negative cognition. In this pursuit, we
2016); (b) the acquired negative thinking is maintained through an employed an emotional version of the probabilistic reversal learning
avoidance mechanism that motivates people to postpone overt actions paradigm (cf. Izquierdo & Jentsch, 2012). In a classic reversal learning
and avoid risking failure and humiliation (Watkins & Nolen-Hoeksema, task, participants have to choose between two options. One option is
2014). Kingston, Watkins, and Nolen-Hoeksema (2014) experimentally associated with a higher probability of reward, while the other is as-
manipulated rumination, which indeed increased justification for not sociated with a higher probability of punishment. Typically, over the
taking or avoiding overt actions. Avoidance can temporarily reduce course of the training period, participants learn which of the two re-
distress because depressed individuals perceive taking action to be even sponse options is more likely to result in a beneficial outcome. How-
more aversive than ruminating (Nolen-Hoeksema et al., 2007). How- ever, during the task, the learned response-reward contingency is re-
ever, withdrawing from aversive situations (or choosing rumination versed. To perform this task well, participants have to adapt their
instead of active problem solving) ironically results in fewer chances to response behaviors to the contingency reversal and switch to the other,
engage in potentially pleasant and rewarding activities and to interact more beneficial, response option.
with positive environmental stimuli (Ramnerö et al., 2016). Never- For the current “emotional” version of the task, participants were
theless, the acquired negative thinking is difficult to extinguish, as offered two response options, retrieving either a negative or positive
depressed people may maintain a higher “reward” expectancy for ru- personal memory. For example, if participants chose the “negative”
minating than for taking other types of actions (including other types of option, they were asked to retrieve a memory related to a given nega-
thinking, e.g., positive). tive word (e.g., “sad”). These personal memory recalls were probabil-
Importantly, negative cognitive activities are inherently aversive istically reinforced by an economic reward and punishment. Initially,
experiences as they arouse negative affect. Moreover, excessive use of participants were reinforced to retrieve a negative (rather than posi-
negative self-reference can lead to social rejection from others (e.g., tive) personal memory. However, in the middle of the task, the action-
Coyne, 1976). Given such unfavorable outcomes, people who hold outcome contingencies were reversed. As a result, negative memory
positive beliefs about negative cognition should often experience a retrieval was no longer the “correct” response, and was thus more likely
large discrepancy between their expected (positive) outcomes and the to be punished. Therefore, participants were supposed to switch to
actual (negative) consequences. For example, people may initially ru- “positive” memory retrieval, which was subsequently the correct re-
minate because they believe it will help them solve their current pro- sponse. To achieve this, participants had to update their beliefs (or
blems and thus, provide them with a sense of control and confidence. outcome predictions) according to the contingency reversal. However,
However, in reality, their ruminative thinking will only increase de- people who have difficulty updating their beliefs for negative memory
pressed mood and inactivity, and consequently impair their problem- retrieval would be more likely to experience increased levels of de-
solving capabilities (Lyubomirsky & Nolen-Hoeksema, 1995). pressed mood, as they continuously remember personal negative
In learning psychology, this discrepancy is referred to as prediction memories and receive punishment. As a control condition, we estab-
error (e.g., Schultz, 2016). Typically, people (or rational learning lished a positive-to-negative transition, in which positive memory re-
agents) attempt to minimize prediction error through learning or ac- trieval was more likely to be rewarded in the first half of the trials,
tion-feedback processes. If an agent is punished (or experiences a ne- while the contingency reversal reinforced participants to choose the
gative outcome) for a particular action, they will adopt a downward negative counterpart in the latter half. Because the resultant updating
prediction, and thus believe that such an action no longer leads to a delay can lead to persistent positive memory retrieval, the speed of
beneficial outcome. Consequently, the frequency with which they take updating is not associated with depressed mood.
that action will decrease. By updating their action-outcome prediction, The emotional reversal learning task is an experimental analogy of
people can optimize their behavior to maximize the benefits obtained persistent negative thinking and rumination because (a) retrieving ne-
by taking a certain action. However, regarding depression, this up- gative memories is part of rumination (e.g., Think about a recent situa-
dating function seems to be blunted, because depression and/or rumi- tion, wishing it had gone better; Nolen-Hoeksema, 1991); (b) a rumination
nation is associated with reduced sensitivity to reward (e.g., Pizzagalli, induction is known to facilitate the recall of negative past events (e.g.,
Jahn, & O'Shea, 2005) and with difficulty adapting a learned cognitive Nolen-Hoeksema et al., 2007); and most importantly, (c) insensitivity to
set to changing environmental contingencies (Davis & Nolen- changes in outcome is a central feature of habitual behavior, and ru-
Hoeksema, 2000; McAuliffe, Hughes, & Barnes-Holmes, 2014). mination is considered a mental habit that is difficult to oppose despite
Depressive rumination has core features of habitual behavior, which its negative consequences (Hertel, 2004; Watkins & Nolen-Hoeksema,
is known to be resistant and insensitive to changes in goals and (de) 2014). Indeed, depressed individuals “choose to ruminate”
valuation of outcomes (e.g., Watkins & Nolen-Hoeksema, 2014). The (Papageorgiou & Wells, 2004, p. 12) because they maintain positive
habit account of rumination provides an intriguing prediction; that is, beliefs about rumination (they expect better outcomes than if they
individuals with depressive symptoms maintain a relatively high re- would make alternative behavioral choices, e.g., thinking about posi-
ward expectancy (or positive beliefs) for rumination, even when ru- tive events). Another factor that promotes rumination is negative re-
mination is actually associated with unfavorable outcomes. Therefore, inforcement through the avoidance of alternative actions, which are
such individuals continuously experience a large discrepancy between often perceived as aversive and difficult, but are potentially pleasant
their predicted and actual outcomes for negative thinking. This re- and rewarding (e.g., Nolen-Hoeksema et al., 2007). It has been estab-
petitive prediction error would be associated with increased depressed lished that individuals with depressive symptoms prefer negative to
mood and a sense of helplessness, as people will find that their behavior positive self-reference when they have to choose between negative and
(or rumination) cannot control reinforcement. Thus, in the present positive valence (Takano, Iijima, Sakamoto, Raes, & Tanno, 2016), and
study, we tested the hypothesis that people with difficulty updating they also tend to experience difficulty updating their beliefs about their
their beliefs regarding negative cognitive processing (for example, ne- negative (vs. positive) self-reference when the outcome is variable
gative memory retrieval) would experience increased levels of de- (Iijima, Takano, Boddez, Raes, & Tanno, 2017).
pressed mood. This is likely due to a discrepancy between the actual Thus, we formulated the hypothesis that delayed belief updating
negative outcomes of such negative thinking behavior and their more would be associated with increased levels of depressed mood only in the
positive outcome expectations. Simultaneously, we explored whether experimental condition (which reinforces participants to switch from

23
K. Takano et al. Journal of Behavior Therapy and Experimental Psychiatry 64 (2019) 22–30

negative to positive memory retrieval), whereas this association would consists of five items. Each item was rated on a 5-point scale (1 = al-
not emerge in the control condition with the reversed reinforcement most never, 5 = almost always). Internal consistency for the RRS
schedule. The speed of belief updating was indexed using learning rates, brooding subscale in the current study was good (α = 0.71).
which are latent parameters estimated by the Q-learning model (e.g.,
Watkins & Dayan, 1992). A learning rate represents the extent to which 2.2.3. Profile of mood state – depressed mood subscale (POMS-D)
a participant updates their reward expectancy after experiencing an The POMS-D (Wald, 1984) was administered before and after the
unexpected outcome (i.e., prediction error). As a larger value for a ERLT to assess the level of depressed mood. The POMS-D consists of 8
learning rate indicates faster belief updating, we specifically expected a items rated on a 5-point scale (0 = not at all, 4 = very much). The
negative association between a learning rate and depressive symptoms POMS-D showed good internal consistency for the current data
in the experimental, but not control, condition. (α = 0.90).

2. Method 2.3. The emotional reversal learning task (ERLT)

2.1. Participants At the start of each trial, participants were asked to choose between
two options: “negative” or “positive” memory retrieval (Fig. 1 A; “va-
Forty-six participants (9 men and 37 women; mean age = 23.1, lence choice”), by pressing the “8” or “9” key. Participants were in-
SD = 5.6 years) were recruited from a large sample pool at the formed that if they chose the “negative” option, they would be asked to
University of Leuven, comprising university students and others in the retrieve a memory in response to a negative cue word in the following
Leuven community. No inclusion/exclusion criteria were used, except display and that, on the other hand, if they chose the “positive” option,
that all participants had to be fluent Dutch speakers. Participants were they would be asked to recall a memory in response to a positive cue
randomly allocated to one of two conditions, Negative-to-Positive word. Following this valence choice, participants were provided with a
(N = 23) or Positive-to-Negative (N = 23), which used different question containing an emotional cue word with the valence of their
learning schedules for the emotional reversal learning task (ERLT). choice. This question probed whether they could retrieve the specific
Groups were not explicitly matched based on gender, age, level of de- memory that they would need to retrieve based on the presented
pressive symptoms, or level of ruminative tendencies; no significant emotional cue word (e.g., “Can you remember an episode that made
differences between the conditions were found for any of these vari- you feel unhappy?” or, “Can you remember an episode that made you
ables (Table 1). Participants received monetary compensation (10 or 20 feel happy?”). If participants thought that they could retrieve a specific
euros) for their participation, depending on their performance on the memory in response to the presented cue word, they pressed the yes “8”
emotional reversal learning task. key. However, if they could not come up with a memory, they pressed
the no “9” key. Emotional cue words were randomly selected from a
2.2. Measures cue-word list, which consisted of 52 pairs of Dutch antonyms, such as
sad vs. happy and passive vs. active. All cue words used in the present
2.2.1. Beck Depression Inventory II (BDI-II) study were confirmed to have negative or positive valence in a pre-
Depressive symptoms were measured using the BDI-II (Beck, Steer, established database of Dutch emotional words (Moors et al., 2013).
& Brown, 1996; Van der Does, 2002b), which consists of 22 items re- After making the yes/no response for memory retrieval, participants
presenting typical depressive symptoms. Each item was rated on a 0–3 received feedback in the form of either a reward or a punishment (as
scale. The BDI-II showed good internal consistency in the current study +5 or −5 point tokens). The reward/punishment was provided prob-
(α = 0.90). abilistically depending on the participants' valence choices. Participants
were explicitly informed that: (a) one of the two valence options was
more likely to be associated with reward than the other option; (b) this
2.2.2. Ruminative Response Scale (RRS)
option-reward contingency could change during the task; and (c) if
The tendency to ruminate was assessed using the RRS brooding
participants indicated “no memory”, they would always receive pun-
subscale (Treynor, Gonzalez, & Nolen-Hoeksema, 2003; see also;
ishment. Participants were also informed that if the total amount of
Schoofs, Hermans, & Raes, 2010; Raes & Hermans, 2007), which
points that they acquired during the task exceeded a certain criterion,
they would receive an extra monetary compensation (10 euros) on top
Table 1
Means and SDs of questionnaires and task performances for the negative-to- of the base amount (10 euros). The researcher did not mention the exact
positive (NtP) and positive-to-negative (PtN) Condition. criterion to prevent participants’ reward devaluation during the task.
The current experiment had two conditions in terms of the re-
NtP (N = 23) PtN (N = 23)
inforcement schedule: Negative-to-Positive (NtP) and Positive-to-
Mean SD Mean SD Negative (PtN) condition (Fig. 1 B). In the NtP condition, participants
were initially reinforced to choose the “negative” option, which was
BDI-II 10.04 8.59 8.04 6.32 followed by a reward or punishment at the probabilities of 80:20% (i.e.,
RRS Brooding 10.17 2.87 10.13 3.49
the acquisition phase). After the halfway point (or the 26th trial), this
POMS-D pre 2.78 4.16 2.09 3.19
POMS-D post 2.17 4.74 2.00 2.86 contingency was reversed, as the “negative” option was more likely to
Performance on the ERLT be punished at the reward-punishment probabilities of 20:80% (i.e., the
Choice frequency of the “Negative” option reversal phase). Conversely, the PtN condition initially reinforced
Phase 1 (Acquisition, 1–13th trial) 0.59 0.25 0.23 0.18 choosing the “positive” option, and subsequently switched to the “ne-
Phase 2 (Acquisition, 14–26th trial) 0.72 0.21 0.17 0.16
Phase 3 (Reversal, 27–39th trial) 0.36 0.21 0.46 0.23
gative” option.
Phase 4 (Reversal, 40–52nd trial) 0.23 0.22 0.59 0.32 Participants were explicitly informed beforehand that they would be
Q-learning parameters tested on whether they actually retrieved their specific personal mem-
Learning rate (αreward) 0.35 0.12 0.32 0.12 ories during the ERLT. After completion of this task, participants un-
Learning rate (αpunish) 0.53 0.18 0.39 0.25
derwent two manipulation-check tasks: a recognition task and a de-
Exploitation (β) 3.99 1.83 3.53 1.35
scription task. In the recognition task, positive and negative words were
Note. BDI-II = Beck Depression Inventory-II; POMS-D Pre, Post = Profile of sequentially presented on a computer screen. These words were se-
Mood States Depression subscale, administered before and after the emotional lected from the antonymous word list used in the ERLT. Either the
reversal learning task; RRS = Ruminative Response Scale. negative or positive word was randomly selected from each of the 52

24
K. Takano et al. Journal of Behavior Therapy and Experimental Psychiatry 64 (2019) 22–30

Fig. 1. Schematic flow of the Emotional Reversal Learning Task (Panel A) and learning schedules in the Negative-to-Positive and Positive-to-Negative condition
(Panels B and C).

antonymous word pairs. For each presented word, participants were 2.5. Statistical analyses
asked to indicate whether they saw it during the ERLT. Because self-
reference and autobiographical encoding (e.g., Klein, Loftus, & Burton, Our statistical analyses consisted of two steps: (a) modeling trial-by-
1989) produce recall enhancement of encoded words, it was expected trial choice behaviors to parameterize the delay in updating outcome
that the words for which participants retrieved personal memories predictions, and (b) testing the association between the updating delay
during the ERLT would be more accurately recognized than words for and depressed mood. For the first step, we indexed the updating delay
which they could not retrieve a personal memory. In the description using a learning rate, which was estimated by fitting the Q-learning
task, participants were provided with 10 word pairs from the same model to individual participants' choice responses. As this parameter
antonymous word list. For each of the presented word pairs, partici- reflects the extent to which participants overwrite their old outcome
pants first had to indicate which word in the pair was displayed during predictions after experiencing a new actual outcome, lower learning
the task. Next, they were asked to describe the memory that they had rates indicate greater delays in updating outcome predictions. For the
retrieved. All participants provided written memories for this descrip- second step of our analyses, we tested the association between parti-
tion task (participants’ responses were not scored for statistical ana- cipants’ updating delay and depressed mood using a multiple regression
lyses). For all the tasks, instructions and stimuli were displayed on a model in which post-task depressed mood is predicted by an interaction
computer screen in Dutch, using E-prime 2.0 software (Psychology between the condition (NtP vs. PtN) and the updating-delay index
Software Tools, Pittsburgh, PA). (learning rate). In the NtP condition, individuals with low learning rates
should persistently choose to retrieve negative memories, and would
experience greater discrepancies between their predicted and actual
2.4. Procedure outcomes for negative memory retrieval. Therefore, we predicted that
the learning rate would be associated with increased levels of depressed
Participants were invited to the laboratory individually. On arrival, mood in the NtP condition, but not in the PtN condition, where low
they provided written informed consent. Participants completed a set of learning rate contributes to persistent positive memory retrieval.
questionnaires, including the BDI-II, RRS, and POMS-D, following
which they completed the ERLT.1 Immediately after the task, partici-
pants completed the POMS-D a second time, and subsequently engaged 2.6. Q-learning model
in the manipulation-check tasks. At the end of the experiment, parti-
cipants in the PtN condition received a mood-lifting manipulation Choice responses of individual participants were modeled using the
(Hepburn, Barnhofer, Williams, 2006; Nelis, 2014). All participants Q-learning technique (Daw, 2011; Sutton & Barto, 1998; Watkins &
were sufficiently debriefed and received monetary compensation for Dayan, 1992). Q-learning is an algorithm of reinforcement learning,
their participations. All study protocols were approved by an institu- which assumes that (a) a learning agent internally maintains outcome
tional review board. predictions for each response option (i.e., Q value), and (b) the outcome
predictions are sequentially updated after receiving an actual outcome
(reward or punishment). Based on the difference in Q value between the
1
In between the questionnaire and ELRT, participants engaged in a dot-probe (two) response options, the agent makes a choice, which further in-
task (MacLeod & Mathews, 1986), which was used as a filler task for a sub-
fluences the outcome predictions for the next trial. The Q updating
sequent self-verification manipulation (Giesler, Josephs, & Swann, 1996). Be-
policy is defined by the Rescola-Wagner rule, which is represented as
cause we did not find an effect of self-verification on the questionnaire re-
sponses and ELRT task performances, we do not report those results here.
follows:

25
K. Takano et al. Journal of Behavior Therapy and Experimental Psychiatry 64 (2019) 22–30

Qnwg (t + 1) = Qneg (t ) + αreward (R (t ) − Qneg (t )) between the condition (NtP vs. PtN) and learning rates. Although we
did not have good prior information for the expected effect size, a
or
power analysis with G*power (Faul, Erdfelder, Buchner, & Lang, 2009)
Qnwg (t + 1) = Qneg (t ) + αpunish (R (t ) − Qneg (t )) indicated that the required sample size was N = 26–55 (total for the
two conditions) to detect a moderate to large effect (f2 = 0.15–0.35)
where Qneg (t + 1) is the outcome prediction of the “negative” option under alpha = .05 and power = .80.
for trial t + 1. The outcome prediction is determined by the prediction
error, i.e., the difference between the actual outcome, R(t), and out- 3. Results
come prediction of the previous trial, Qneg (t). The prediction error is
weighted by a learning rate, αreward and αpunish, which defines the speed 3.1. Manipulation check
of updating the outcome predictions. We assumed different learning
rates for rewarded and punished trials, because it has been suggested Participants indicated that they could retrieve specific memories in
that the updating speed of an outcome prediction can differ across response to most of the negative and positive cue words (M = 86.8%,
different outcome values (Jahfari & Theeuwes, 2017). Similarly, the SD = 11.2%, for a total of 52 trials). However, it is possible that par-
outcome prediction for positive memory retrieval, Qpos (t), is also up- ticipants pretended to retrieve a memory just by pressing the “yes” key
dated by the Rescola-Wagner rule, with variable updating speeds for in the memory retrieval display. Therefore, as a manipulation check, we
rewarded and punished trials. tested the effect of autobiographical encoding on performance in a re-
The probability of choosing the “negative” option is represented by cognition task. It was expected that if participants actually retrieved a
a softmax function of the difference between the outcome predictions of memory for each cue word, their recognition performance would be
the two options: enhanced by the memory traces of retrieving a personal memory. We
1 found a slightly (but statistically significantly) higher recognition per-
Pneg (t ) =
1 − exp(−β (Qneg (t ) − Qpos (t ))) formance for the memory-recalled than not-memory-recalled words
(69.2 and 62.7%, respectively; χ2 = 5.60, p = .02, Cramer's V = 0.05).
where an exploration-exploitation parameter, β, reflects a learning
history, with a larger value indicating greater sensitivity to the differ- 3.2. Descriptive statistics
ence in outcome predictions.
When estimating the parameters, we fit the same Q-learning model Table 1 shows the means and standard deviations for the ques-
to the data for both the NtP and PtN conditions. It is important to note tionnaire variables, the choice frequencies of the “negative” option in
that the two learning rates have different meanings for each of these the ERLT, and the estimated Q-learning parameters. There were no
conditions. The learning rate for the punished trials (αpunish) influences statistically significant differences between the NtP and PtN conditions
the speed of decreasing reward expectancy for a chosen option; it mainly for BDI, RSQ, and baseline POMS-D scores, ts < 1.22, ps > 0.22. We
controls the updating processes for (a) the negative response option in calculated the mean “negative” choice frequency for each quarter of the
the NtP condition and (b) the positive option in the PtN condition after ERLT trials (i.e., Phase 1 and 2 = the first and latter half of the ac-
the contingency reversal. Therefore, αpunish in the NtP condition is of quisition phase; Phase 3 and 4 = the first and latter half of the reversal
particular interest, as it codes the speed of updating (or down-revising) phase) to ensure that the contingency reversal reinforced a decrease in
the outcome prediction for negative memory retrieval. “negative” choices in the NtP condition and an increase in “negative”
choices in the PtN condition. A one-way ANOVA for the “negative”
2.7. Parameter estimation choice frequency across the four phases indicates that participants
adapted their choice behaviors after the contingency reversal, F(1.3,
We used a Hierarchical Bayes (HB) estimation to obtain the optimal 28.8) = 19.45, p < .001, for the NtP, and F(2.1, 45.6) = 28.82,
parameter estimates of αreward, αpunish, and β for each participant. In the p < .001, for the PtN condition. More specifically, there was a sig-
HB approach, each parameter is assumed to follow multi-level dis- nificant increase in the number of “negative” responses from Phase 2 to
tributions, wherein parameters of individual participants were drawn 3 in the NtP condition, t(22) = 4.66, p < .001, dz = 0.97, and a sig-
from a group-level distribution shared by all participants. In our ana- nificant decrease from Phase 2 to 3 in the PtN condition, t(22) = 5.69,
lysis, each of the three free parameters was treated deterministically; p < .001, dz = 1.19.
however, we modeled their respective probit transformations (α′reward, The delay in switching from an old to new response was represented
α′punish, and β′) to follow a normal distribution with a group mean (μ) by a low learning rate for punished trials (i.e., αpunish). Supporting this
and standard deviation (σ): α′reward ∼ Normal (μα′ reward, σα′ reward), notion, the learning rate for punished trials was significantly negatively
α′punish ∼ Normal (μα′ punish, σα′ punish), and β' ∼ Normal (μβ′, σβ′). The correlated with the “negative” responses in Phase 3 (first half of the
group means and standard deviations also had priors, which were as- reversal phase) of the NtP condition, r = −0.56, 95%CI [-0.79,
sumed to be constant between the NtP and PtN conditions: μα′ reward, μα′ −0.20], p = .005, and with the Phase 3 “positive” responses in the PtN
punish, μβ' ∼ Normal (0, 1), and σα′ reward, σα′ punish, σβ' ∼ Uniform (0, condition, r = −0.78, 95%CI [-0.90, −0.54], p < .001. These sig-
1.5). These parameter settings were adapted from previous studies that nificant correlations suggest that the low learning rate for punished
used similar estimation approaches (Jahfari & Theeuwes, 2017; trials captures the delayed transition (a) from the negative to positive
Katahira, 2016; Steingroever, Wetzels, & Wagenmakers, 2013; Wetzels, response in the NtP condition (in which the negative response was
Vandekerckhove, Tuerlinckx, & Wagenmakers, 2010). This HB Q- punished in the reversal phase), and (b) from the positive to negative
learning model was implemented in Stan (Hoffman & Gelman, 2014; response in the PtN condition (in which the positive response was be
Stan Development Team, 2015). Three Hamiltonian-Monte-Carlo punished in the reversal phase).
chains were run to evaluate convergence, which showed that Rhats for
all parameters were close to 1 (indicating convergence to a stationary 3.3. Individual differences in the learning rate and depressive symptoms
distribution; Gelman & Rubin, 1992).
Subsequently, we examined the associations between the learning
2.8. Sample size calculation parameters and depressive symptoms. The results indicate that the
learning rate for the punished trials was significantly associated with
Our main hypothesis was tested using a multiple regression model, depressive symptoms, ruminative tendencies, and post-task depressed
in which post-task depressed mood was predicted by the interactions mood in the NtP condition (Table 2). As previously noted, the low

26
K. Takano et al. Journal of Behavior Therapy and Experimental Psychiatry 64 (2019) 22–30

Table 2
Correlations between learning parameters and depression measures.
Learning rate: Learning rate: Exploitation (β)
reward (αreward) punishment (αpunish)

Negative-to-positive condition (N = 23)

BDI-II −0.08 −0.47* 0.03
RRS Brooding 0.00 −0.43* −0.07
POMS-D Pre 0.26 −0.27 0.08
POMS-D Post 0.20 −0.49* 0.04
Positive-to-negative condition (N = 23)
BDI-II −0.03 −0.29 −0.12
RRS Brooding 0.30 −0.13 0.12
POMS-D Pre 0.39 −0.15 0.17
POMS-D Post 0.33 −0.11 −0.01

Note. BDI-II = Beck Depression Inventory-II; POMS-D Pre, Post = Profile of

Mood States Depression subscale, administered before and after the emotional
reversal learning task; RRS = Ruminative Response Scale. *p < .05.

Fig. 3. Mean Q values (reward expectancies) for the negative (Panel A) and
positive option (Panel B) as a function of trials for participants with high (i.e.,
BDI-II > 9) and low levels of depressive symptoms in the Positive-to-Negative
condition. Gray fields indicate standard errors.

learning rate for punished trials reflects the degree of delay in disen-
gaging from a previously reinforced response. To visualize this delay,
we depicted trajectories of outcome predictions (i.e., Q values) for
participants with higher and lower levels of depressive symptoms (split
by the mean of the BDI-II scores). As described in the Method section, Q
values reflect participants’ reward expectancies for negative and posi-
tive response options (i.e., the extent to which participants believe that
retrieving a negative or positive memory leads to a beneficial outcome).
Fig. 2 illustrates the Q trajectories in the NtP condition, which indicate
that people with high levels of depressive symptoms maintained high
reward expectancy for the negative option even after the contingency
reversal (Panel A), whereas individuals with low levels of depressive
symptoms decrease their reward expectancy rapidly after the con-
Fig. 2. Mean Q values (reward expectancies) for the negative (Panel A) and
tingency reversal (Panel A). Fig. 3 also depicts the Q trajectories in the
positive option (Panel B) as a function of trials for participants with high (i.e.,
BDI-II > 9) and low levels of depressive symptoms in the Negative-to-Positive
PtN condition, which indicate that the delay for people with high levels
condition. Gray fields indicate standard errors. of depressive symptoms is less evident; reward expectancy dropped to
the same level as for the less depressed group by the end of the reversal

27
K. Takano et al. Journal of Behavior Therapy and Experimental Psychiatry 64 (2019) 22–30

phase.2 Table 3
Multiple regression predicting post-task depressed mood (POMS-D).

3.4. Depressed mood and delayed updating of outcome prediction IVs Estimates SE t p 95% CI

BDI-II −0.01 0.07 −0.11 .910 [-0.14, 0.13]

Finally, we tested whether the low learning rate for punished trials POMS-D 0.68 0.14 4.88 < .001 [0.40, 0.97]
is associated with increased levels of depressed mood in the NtP con- Learning rate (reward) 1.08 5.86 0.18 .854 [-10.78,
dition. A multiple regression model was estimated, in which post-task 12.95]
depressed mood was predicted by the two learning rates, the condition Learning rate (punish) −0.28 2.74 −0.1 .919 [-5.83, 5.27]
Condition 3.25 2.78 1.17 .250 [-2.38, 8.87]
(dummy coded as NtP = 1, PtN = 0), and their interactions, after
Learning rate (reward) * 3.82 7.35 0.52 .607 [-11.06,
controlling for the pre-task levels of depressed mood and the BDI-II Condition 18.70]
score. All predictors except for the condition were standardized for ease Learning rate (punish) * −9.10 4.10 −2.22 .033 [-17.41,
of interpretation. The results revealed a significant interaction between Condition −0.79]
the condition and the learning rate for punished trials (Table 3), in- Adj R2 0.55

dicating a significant effect of the learning rate on post-task depressed

Note. N = 46. BDI-II = Beck Depression Inventory; POMS-D = Profile of Mood
mood in the NtP condition, Β = −9.38, 95%CI [-16.13, −2.63], State Depressed Mood Subscale.
t = 3.34, p = .008, but not in the PtN condition, Β = −0.28, 95%CI
[-5.83, 5.27], t = 0.10, p = .919. These results suggest that the delay in rumination (e.g., Kuo et al., 2012). Nolen-Hoeksema et al. (2007), for
updating outcome predictions is associated with increased levels of example, note how rumination is about using negative thoughts and
depressed mood, particularly when the reward expectancy for negative memories to make sense of current circumstances. Indeed, several
(but not positive) memory retrieval has to be decreased (as coded by scholars in the field of rumination have agreed that the sustained
αpunish in the NtP condition). processing of negative self-related material from memory, in order to
try to make sense of negative memories and current issues, is a central
4. Discussion ingredient of depressive rumination (e.g., see Nolen-Hoeksema et al.,
2007, p. 407).
We examined individuals’ difficulty with updating outcome pre- As an alternative interpretation, the association between the low
dictions for emotional memory retrieval, as an experimental analogy of learning rate and depressive symptoms may be due to a general deficit
persistent positive beliefs about negative cognition in depression (e.g., in the updating function (see also Dombrovski et al., 2010; Kunisato
Papageorgiou & Wells, 2001). As expected, the results showed that the et al., 2012), but not due to a valence-specific problem for negative
updating difficulty after receiving unfavorable outcomes (which is re- memory retrieval. There seems to be a small updating delay for people
flected by a low learning rate for punished trials) predicts increased with depressive symptoms also in the PtN condition (see Fig. 3B). More
levels of depressed mood when participants are reinforced to switch critically, the correlations between the learning rate and depressive
from negative to positive memory retrieval (i.e., NtP condition). In this symptoms were not statistically significantly different between the NtP
condition, participants with updating difficulty maintained their “po- and PtN condition (rs = −0.47 vs. −0.29, p = .51). This may be due to
sitive” belief (high reward expectancy or high Q value) that negative the small sample size in the current study, which was not large enough
memory retrieval results in beneficial outcomes, even when the reality to detect a significant difference between the two correlations. Related
of choosing a negative response option was a form of punishment. to this point, the correlational analyses may be underpowered due to
These individuals would repetitively experience a discrepancy between the small sample size in each condition. These results should be inter-
their expected outcome and the actual outcome for negative memory preted carefully as preliminary evidence, and future research (with a
retrieval, which might be associated with a sense of helplessness and better statistical power) needs to replicate the observed associations
depressed mood. Importantly, the updating delay was not associated between the learning rate and depressive symptoms.
with depressed mood when participants were reinforced to switch from Another important limitation is that our manipulation check tasks
positive to negative memory retrieval (i.e., PtN condition). This is likely (i.e., recognition and description tests) cannot completely eliminate the
because the updating delay in this condition increases the retrieval of possibility that participants did not retrieve their personal memories
positive memories, which might counteract the negative impact of the during the emotional reversal learning task. Because our primary in-
punishment feedback. terest was in the preference to (but not in the actual retrieval of) ne-
Another important finding is that individuals with higher levels of gative memories, we did not ask participant to report the contents of
depressive symptoms and depressive rumination have lower learning individual memories in the exact moments when participants indicated
rates for punished trials in the NtP condition. These significant corre- that they could retrieve a negative or positive memory. Thus, it should
lations imply that vulnerable people tend to maintain a persistent be noted that self-satisfying behavior (i.e., choosing “yes” without re-
“positive belief” that retrieving a negative memory leads to beneficial trieving a target emotional memory) may contaminate the results.
outcomes, even when it does not in reality. This tendency supports the Although the aforementioned limitations raise several alternative
argument that rumination is a mental habit (Ramnerö et al., 2016; interpretations (particularly for the statistical power due to the small
Verplanken, Friborg; Wang, Trafimow, & Woolf, 2007; Watkins & sample size), our data supports the association between difficulty in
Nolen-Hoeksema, 2014). In other words, negative thinking is less sen- updating outcome predictions for negative memory retrieval and in-
sitive to changes in action-outcome contingencies, which makes it dif- creased levels of depressed mood. Although the non-clinical nature of
ficult to oppose depressive rumination despite its negative con- our sample limits the clinical implications, our methodology (i.e., the
sequences (Hertel, 2004). However, it should be noted that rumination emotional reversal learning task) demonstrates a sound experimental
is not merely the memory recall of past negative episodes as we oper- analogy of persistent positive beliefs about negative cognition in de-
ationalized it here, although enhanced or biased recollection of nega- pression.
tive information is suggested to play an important role in depressive

2
Conflict of interest
The mean splitting of participants was only done for visualization purposes.
Following the dimensional view of psychopathology (Cuthbert, 2014), we
The authors declare no conflict of interest.
performed all our statistical analyses on continuous (and not on dichotomized)
measures (e.g., MacCallum, Zhang, Preacher, & Rucker, 2002).

28
K. Takano et al. Journal of Behavior Therapy and Experimental Psychiatry 64 (2019) 22–30

Acknowledgement negative thinking and interpersonal problem solving. Journal of Personality and Social
Psychology, 69, 176–190. https://doi.org/10.1037/0022-3514.69.1.176.
MacCallum, R. C., Zhang, S., Preacher, K. J., & Rucker, D. D. (2002). On the practice of
Keisuke Takano was supported by Japan Society for the Promotion dichotomization of quantitative variables. Psychological Methods, 7, 19–40. https://
of Science Postdoctoral Fellowships for Research Abroad and the doi.org/10.1037/1082-989X.7.1.19.
Alexander von Humboldt Foundation Postdoctoral Research MacLeod, C., & Mathews, A. (1986). Attentional bias in emotional disorders. Journal of
Abnormal Psychology, 95, 12–20. https://doi.org/10.1037/0021-843X.95.1.15.
Fellowship. Filip Raes was supported by the KU Leuven Research Mathews, A., & MacLeod, C. (2005). Cognitive vulnerability to emotional disorders.
Council grant PF/10/005. Filip Raes and Keisuke Takano received ad- Annual Review of Clinical Psychology, 1, 167–195. https://doi.org/10.1146/annurev.
ditional support from a Red Noses grant of the Research Foundation – clinpsy.1.102803.143916.
Matt, G. E., Vazquez, C., & Campbell, W. K. (1992). Mood-congruent recall of affectively
Flanders (FWO-Vlaanderen; G0F5617N). toned stimuli: A meta-analytic review. Clinical Psychology Review, 12, 227–255.
https://doi.org/10.1016/0272-7358(92)90116-P.
References McAuliffe, D., Hughes, S., & Barnes-Holmes, D. (2014). The dark-side of rule governed
behavior: An experimental analysis of problematic rule-following in an adolescent
population with depressive symptomatology. Behavior Modification, 38, 587–613.
Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Beck depression inventory (BDI–II) manual. https://doi.org/10.1177/0145445514521630.
San Antonio, TX: The Psychological Corporation. Mezulis, A. H., Abramson, L. Y., Hyde, J. S., & Hankin, B. L. (2004). Is there a universal
Coyne, J. C. (1976). Depression and the response of others. Journal of Abnormal positivity bias in attributions? A meta-analytic review of individual, developmental,
Psychology, 85, 186–193. https://doi.org/10.1037/0021-843X.85.2.186. and cultural differences in the self-serving attributional bias. Psychological Bulletin,
Cuthbert, B. N. (2014). The RDoC framework: Facilitating transition from ICD/DSM to 130, 711–747. https://doi.org/10.1037/0033-2909.130.5.711.
dimensional approaches that integrate neuroscience and psychopathology. World Moors, A., De Houwer, J., Hermans, D., Wanmaker, S., van Schie, K., Van Harmelen, A. L.,
Psychiatry, 13, 28–35. et al. (2013). Norms of valence, arousal, dominance, and age of acquisition for 4,300
Davis, R., & Nolen-Hoeksema, S. (2000). Cognitive inflexibility among ruminators and Dutch words. Behavior Research Methods, 45, 169–177. https://doi.org/10.3758/
nonruminators. Cognitive Therapy and Research, 24, 699–711. https://doi.org/10. s13428-012-0243-8.
1023/A:1005591412406. Nelis, S. (2014). When even a full glass looks half empty: A study of processes that dampen
Daw, N. D. (2011). Trial-by-trial data analysis using computational models. In R. Delgado, positivity in the context of depressionLeuven, Belgium: University of Leuven
E. A. Phelps, & T. W. Robbins (Eds.). Decision making, affect, and learning: Attention (Unpublished doctoral dissertation).
and performance XXIII (pp. 1–26). Oxford: Oxford University Press. Nolen-Hoeksema, S. (1991). Responses to depression and their effects on the duration of
Dombrovski, A. Y., Clark, L., Siegle, G. J., Butters, M. A., Ichikawa, N., Sahakian, B. J., depression. Journal of Abnormal Psychology, 100, 569–582. https://doi.org/10.1037/
et al. (2010). Reward/punishment reversal learning in older suicide attempters. 0021-843X.100.4.569.
American Journal of Psychiatry, 167, 699–707. https://doi.org/10.1176/appi.ajp. Nolen-Hoeksema, S. (2000). The role of rumination in depressive disorders and mixed
2009.09030407. anxiety/depressive symptoms. Journal of Abnormal Psychology, 109, 504–511
Faul, F., Erdfelder, E., Buchner, A., & Lang, A. G. (2009). Statistical power analyses using 101037/10021-843X.109.3.504.
G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Nolen-Hoeksema, S., Wisco, B. E., & Lyubomirsky, S. (2007). Rethinking rumination.
Methods, 41, 1149–1160. https://doi.org/10.3758/BRM.41.4.1149. Perspectives on Psychological Science, 3, 400–424. https://doi.org/10.1111/j.1745-
Gelman, A., & Rubin, D. B. (1992). Inference from iterative simulation using multiple 6924.2008.00088.x.
sequences. Statistical Science, 7, 457–511. Papageorgiou, C., & Wells, A. (2001). Metacognitive beliefs about rumination in recurrent
Giesler, R. B., Josephs, R. A., & Swann, W. B., Jr. (1996). Self-verification in clinical major depression. Cognitive Behavioral Practice, 8, 160–164. https://doi.org/10.1016/
depression: The desire for negative evaluation. Journal of Abnormal Psychology, 105, S1077-7229(01)80021-3.
358–368. https://doi.org/10.1037/0021-843X.105.3.358. Papageorgiou, C., & Wells, A. (2003). An emperical test of a clinical metacognitive model
Gotlib, I. H., & Joormann, J. (2010). Cognition and depression: Current status and future of rumination and depression. Cognitive Therapy and Research, 27, 261–273. https://
directions. Annual Review of Clinical Psychology, 6, 285–312. https://doi.org/10. doi.org/10.1023/A:1023962332399.
1146/annurev.clinpsy.121208.131305. Papageorgiou, C., & Wells, A. (2004). Depressive rumination: Nature, theory and treatment.
Hepburn, S. R., Barnhofer, T., & Williams, J. M. G. (2006). Effects of mood on how future West Sussex: John Wiley & Sons Ltd.
events are generated and perceived. Personality and Individual Differences, 41, Pizzagalli, D. A., Jahn, A. L., & O'Shea, J. P. (2005). Toward an objective characterization
801–811. https://doi.org/10.1016/j.paid.2006.03.022. of an anhedonic phenotype: A signal detection approach. Biological Psychiatry, 57,
Hertel, P. T. (2004). Memory for emotional and non-emotional events in depression: A 319–327. https://doi.org/10.1016/j.biopsych.2004.11.026.
question of habit. In D. Reisberg, & P. Hertel (Eds.). Memory and emotion (pp. 186– Raes, F., & Hermans, D. (2007). The revised version of the Dutch ruminative response scale.
216). New York, NY: Oxford University Press. Unpublished instrument.
Hoffman, M. D., & Gelman, A. (2014). The no-U-turn sampler: Adaptively setting path Ramnerö, J., Folke, F., & Kanter, J. W. (2016). A learning theory account of depression.
lengths in Hamiltonian Monte Carlo. The Journal of Machine Learning Research, 15, Scandinavian Journal of Psychology, 57, 73–82. https://doi.org/10.1111/sjop.12233.
1593–1623. Roelofs, J., Papageorgiou, C., Gerber, R.,D., Huibers, M., Peeters, F., & Artnz, A. (2007).
Iijima, Y., Takano, K., Boddez, Y., Raes, F., & Tanno, Y. (2017). Stuttering thoughts: On the links between self-discrepancies, rumination, metacognitions, and symptoms
Negative self-referent thinking is less sensitive to aversive outcomes in people with of depression in undergraduates. Behaviour Research and Therapy, 45, 1295–1305.
higher levels of depressive symptoms. Frontiers in Psychology, 8, 1333. https://doi. https://doi.org/10.1016/j.brat.2006.10.005.
org/10.3389/fpsyg.2017.01333. Schoofs, H., Hermans, D., & Raes, F. (2010). Brooding and reflection as subtypes of ru-
Izquierdo, A., & Jentsch, J. D. (2012). Reversal learning as a measure of impulsive and mination: Evidence from confirmatory factor analysis in nonclinical samples using
compulsive behavior in addictions. Psychopharmacology, 219, 607–620. https://doi. the Dutch Ruminative Response Scale. Journal of Psychopathology and Behavioral
org/10.1007/s00213-011-2579-7. Assessment, 32, 609–617. https://doi.org/10.1007/s10862-010-9182-9.
Jahfari, S., & Theeuwes, J. (2017). Sensitivity to value-driven attention is predicted by Schultz, W. (2016). Dopamine reward prediction error coding. Dialogues in Clinical
how we learn from value. Psychonomic Bulletin & Review, 24, 408–415. https://doi. Neuroscience, 18, 23–32.
org/10.3758/s13423-016-1106-6. Stan Development Team (2015). Stan: A C++ library for probability and sampling, version
Katahira, K. (2016). How hierarchical models improve point estimates of model para- 2.8.0. Retrieved from http://mc-stan.org/.
meters at the individual level. Journal of Mathematical Psychology, 73, 37–58. https:// Steingroever, H., Wetzels, R., & Wagenmakers, E. J. (2013). Validating the PVL-Delta
doi.org/10.1016/j.jmp.2016.03.007. model for the Iowa gambling task. Frontiers in Psychology, 4, 898. https://doi.org/10.
Kingston, R. E. F., Watkins, E. R., & Nolen-Hoeksema, S. (2014). Investigating functional 3389/fpsyg.2013.00898.
properties of depressive rumination: Insight and avoidance. Journal of Experimental Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge,
Psychopathology, 5, 244–258. https://doi.org/10.5127/jep.038013. MA: MIT Press.
Klein, S. B., Loftus, J., & Burton, H. A. (1989). Two self-reference effects: The importance Takano, K., Iijima, Y., Sakamoto, S., Raes, F., & Tanno, Y. (2016). Is self-positive in-
of distinguishing between self-descriptiveness judgments and autobiographical re- formation more appealing than money? Individual differences in positivity bias ac-
trieval in self-referent encoding. Journal of Personality and Social Psychology, 56, cording to depressive symptoms. Cognition and Emotion, 30, 1402–1414. https://doi.
853–865. org/10.1080/02699931.2015.1068162.
Koster, E. H., De Lissnyder, E., Derakshan, N., & De Raedt, R. (2011). Understanding Treynor, W., Gonzalez, R., & Nolen-Hoeksema, S. (2003). Rumination reconsidered: A
depressive rumination from a cognitive science perspective: the impaired disen- psychometric analysis. Cognitive Therapy and Research, 27, 247–259. https://doi.org/
gagement hypothesis. Clinical Psychology Review, 31, 138–145. https://doi.org/10. 10.1023/A:1023910315561.
1016/j.cpr.2010.08.005. Van der Does, A. J. W. (2002b). Handleiding bij de Nederlandse bewerking van de BDI-II.
Kunisato, Y., Okamoto, Y., Ueda, K., Onoda, K., Okada, G., Yoshimura, S., et al. (2012). [Manual of the Dutch version of the BDI-II]. Amsterdam, the Netherlands: Pearson
Effects of depression on reward-based decision making and variability of action in Assessment.
probabilistic learning. Journal of Behavior Therapy and Experimental Psychiatry, 43, Verplanken, B., Friborg, O., Wang, C. E., Trafimow, D., & Woolf, K. (2007). Mental habits:
1088–1094. https://doi.org/10.1016/j.jbtep.2012.05.007. Metacognitive reflection on negative self-thinking. Journal of Personality and Social
Kuo, J. R., Edge, I. G., Ramel, W., Edge, M. D., Drabant, E. M., Dayton, W. M., et al. Psychology, 92, 526–541. https://doi.org/10.1037/0022-3514.92.3.526.
(2012). Trait rumination is associated with enhanced recollection of negative words. Wald, F. (1984). De verkorte POMS [The shortened POMS]. Amsterdam: Onuitgegeven doc-
Cognitive Therapy and Research, 36, 722–730. https://doi.org/10.1007/s10608-011- toraalwerkstuk, vakgroep psychonomie, psychologisch laboratorium. Universiteit van
9430-7. Amsterdam.
Lyubomirsky, S., & Nolen-Hoeksema, S. (1995). Effects of self-focused rumination on Watkins, E. R. (2008). Constructive and unconstructive repetitive thought. Psychological

29
K. Takano et al. Journal of Behavior Therapy and Experimental Psychiatry 64 (2019) 22–30

Bulletin, 134, 163–206. https://doi.org/10.1037/0033-2909.134.2.163. Wetzels, R., Vandekerckhove, J., Tuerlinckx, F., & Wagenmakers, E. J. (2010). Bayesian
Watkins, C. J. C. H., & Dayan, P. (1992). Q- learning. Machine Learning, 8, 279–292. parameter estimation in the expectancy valence model of the Iowa gambling task.
Watkins, E. R., & Nolen-Hoeksema, S. (2014). A habit-goal framework of depressive ru- Journal of Mathematical Psychology, 54, 14–27. https://doi.org/10.1016/j.jmp.2008.
mination. Journal of Abnormal Psychology, 123, 23–34. https://doi.org/10.1037/ 12.001.
a0035540.

30