Behaviour Research and Therapy 151 (2022) 104054

Contents lists available at ScienceDirect

Behaviour Research and Therapy

journal homepage: www.elsevier.com/locate/brat

Towards personalizing cognitive remediation therapy: Examining

moderators of response for euthymic people with bipolar disorder
Dimosthenis Tsapekos a, *, Rebecca Strawbridge a, Matteo Cella b, c, Til Wykes b, c,
Allan H. Young a, c
a
Department of Psychological Medicine, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, UK
b
Department of Psychology, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, UK
c
South London & Maudsley NHS Foundation Trust, Maudsley Hospital, London, UK

A R T I C L E I N F O A B S T R A C T

Keywords: Background: Recent evidence suggests that cognitive remediation (CR) may reduce cognitive and functional
Bipolar disorder difficulties in people with bipolar disorder (BD). However, there is a limited understanding of whether, and
Cognitive remediation which, pre-treatment factors influence who will benefit from CR and this information could help to develop
Cognition
optimal therapy delivery. We aim to identify and examine baseline factors moderating post-treatment
Functioning
improvement.
Moderator
Personalized treatment Methods: This is a secondary analysis of data from a randomized controlled trial comparing CR (n = 40) to
treatment-as-usual (TAU; n = 40) in euthymic people with BD. Elastic net regression was used to identify patient
characteristics and baseline measures associated with post-treatment improvement in cognition, psychosocial
functioning, and goal attainment. We then tested the moderating effect of retained variables on each outcome
using multivariable linear regression.
Results: Despite lower baseline cognitive performance being associated with greater post-treatment changes in
cognition and psychosocial functioning, there was no evidence of treatment response moderation. CR effect on
goal attainment was larger for participants with better baseline cognitive performance, but this moderating effect
did not reach significance (p = 0.09). Those with more severe baseline subjective cognitive complaints (p = 0.03)
and more previously completed psychological therapies (p = 0.02) had also larger gains in goal attainment.
Conclusions: Treatment benefits in cognition and psychosocial functioning might not be affected by pre-treatment
factors and patient characteristics. However, baseline cognition and perceived deficits may influence the effect of
CR on achieving recovery goals. Therapy adaptations may be required to exert greater benefits for less responsive
patients.

1. Introduction recovery.
Cognitive remediation (CR) is a psychological therapy with well-
A substantial proportion of people with bipolar disorder (BD) expe­ documented and durable effects on cognitive and functional outcomes
rience impairments in cognitive processes such as attention, memory in people with schizophrenia (Wykes, Huddy, Cellard, McGurk, & Czo­
and executive functioning (Burdick et al., 2014; Cullen et al., 2016). bor, 2011). Similarities in the cognitive profiles between people with
These deficits are associated with real-world difficulties such as work schizophrenia and mood disorders prompted the application of CR
performance, psychosocial functioning, and quality of life (Brissos, Dias, treatment paradigms to mood disorders, with preliminary evidence
& Kapczinski, 2008; Tse, Chan, Ng, & Yatham, 2014; Wingo, Harvey, & indicating effects on cognition broadly comparable to those detected in
Baldessarini, 2009). The relevance of cognitive impairment for func­ psychotic disorders (Anaya et al., 2012). For people with BD, recent
tional difficulties highlights the need for evidence-based therapies tar­ reviews suggest promising effects on cognition and functioning,
geting cognition not only to improve cognitive abilities, but also to although these studies had methodological limitations (Bellani et al.,
enhance functional outcomes and promote long-term functional 2019; Tsapekos et al., 2020). Two recent randomized controlled trials

* Corresponding author. 103 Denmark Hill, London, SE5 8AZ, UK.

E-mail address: [email protected] (D. Tsapekos).

https://doi.org/10.1016/j.brat.2022.104054
Received 8 July 2021; Received in revised form 13 December 2021; Accepted 4 February 2022
Available online 7 February 2022
0005-7967/© 2022 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
D. Tsapekos et al. Behaviour Research and Therapy 151 (2022) 104054

(RCTs) examined CR in larger groups reporting medium-to-large 2. Materials and methods

cognitive benefits across multiple domains which were maintained at
follow-up (Lewandowski et al., 2017; Strawbridge et al., 2021), while 2.1. Study design
another RCT reported a medium-to-large effect of CR on executive
functioning which disappeared at follow-up (Ott et al., 2020). This is a secondary analysis of longitudinal data from the Cognitive
Despite increasing evidence on the CR benefits for people with BD, Remediation in Bipolar (CRiB) study, an RCT comparing CR to
some participants show no improvement and there is a limited under­ treatment-as-usual (TAU) in euthymic patients with BD (Strawbridge
standing about the patient characteristics associated with outcome et al., 2021). Compared to the main CRiB, this study includes a sample
variability. An uncontrolled study found reduced cognitive improve­ extended by 20 participants, randomized to CR (n = 11) or TAU (n = 9).
ment for patients with a history of comorbid anxiety disorders (Deck­ These participants, recruited under the same criteria following a
ersbach et al., 2010), while a secondary analysis of a functional recruitment extension, were not included in the primary CRiB analysis
remediation two trials showed greater verbal memory improvements for conducted according to the published protocol (N = 60) (Strawbridge
participants with poorer cognition at baseline (Bonnin et al., 2016) and et al., 2016), before all 80 participants had completed trial participation.
larger cognitive gains for those with higher premorbid IQ (Lewandowski The additional 20 participants are included in this study to increase the
et al., 2017). The examination of factors moderating CR outcomes has power of moderation analyses. Written informed consent was obtained
therefore not only been limited but is also contradictory. from all participants. All baseline assessment procedures were con­
Putative moderators of response have been examined more exten­ ducted prior to random group allocation. The trial was approved by the
sively in people with a diagnosis of schizophrenia. However, systematic City Road & Hampstead NHS Research Ethics Committee (reference
reviews suggest that the literature is still missing high-quality and well- 15/LO/1557).
replicated evidence on factors influencing CR outcomes for people with
schizophrenia (Biagianti, Castellaro, & Brambilla, 2021; Reser, Slikboer, 2.2. Participants
& Rossell, 2019; Seccomandi, Tsapekos, Newbery, Wykes, & Cella,
2019). Likewise, a recent large-scale meta-analysis of CR trials revealed Participants were recruited from primary and secondary services,
a moderating effect only for education, premorbid IQ, and baseline community mental health teams, online advertising and mental health
symptom severity, with more compromised participants receiving charities. All included participants had a DSM-5 diagnosis of BD, were
greater benefits in cognition and functioning (Vita et al., 2021). A better fluent in English, and aged between 18 and 65 years. The Mini Inter­
understanding of moderating factors is crucial to tailor and adapt CR national Neuropsychiatric Interview (Sheehan et al., 1998) was used to
treatment paradigms according to patient characteristics and needs confirm diagnosis and BD subtype. Participants were on stable psychi­
(Cella, Reeder, & Wykes, 2015a, 2015b; Wykes & Spaulding, 2011). atric medication and had been free of acute mood symptoms for ≥1
Personalization of CR through such adaptations may improve therapy month prior to inclusion, with euthymia defined as a score of ≤7 on the
delivery and maximize treatment benefits (Medalia, Saperstein, Hansen, Hamilton Depression Rating Scale 17-item (HAMD) (Hamilton, 1960) and
& Lee, 2018). Likewise, identifying whether certain patient subgroups Young Mania Rating Scale (YMRS) (Young, Biggs, Ziegler, & Meyer,
are more or less likely to benefit from CR may help clinical services use 1978) over the 1-month period. Participants with a neurological disor­
their limited resources (e.g., time, personnel) more efficiently. der, personality disorder diagnosis, abuse or dependence on alcohol or
Previous work seeking to identify factors moderating CR outcomes in illicit substances over the past six months were excluded.
people with schizophrenia has been limited by methodological issues,
such as not considering all potential contributing factors, investigating 2.3. Intervention
factors independently of other relevant variables, or problems with
multiple comparisons due to inadequate sample size (Seccomandi et al., 2.3.1. Cognitive remediation
2019). This may lead to a lack of evidence convergence and an increased Participants in the intervention arm received 12 weeks of therapist-
risk of false positive findings. Using traditional analytic methods (e.g., led CR focusing on strategy use, metacognition and transfer of cognitive
correlation, linear regression) to select response moderators might be skills to daily life activities and individual goals. The online software
contributing to these limitations. Considering multiple factors with ‘Computerised Interactive Remediation of Cognition – Interactive
these approaches can result in overfitted models which accurately Training for Schizophrenia’ (CIRCuiTS; https://www.circuitstherap
explain the variation of an outcome in a particular sample but have yinfo.com) was adopted for CR delivery. CIRCuiTS is a manualized CR
limited predictive value (Yarkoni & Westfall, 2017). Adopting more approach validated for patients with a diagnosis of schizophrenia
advanced analytic approaches may facilitate the identification of factors (Reeder et al., 2016, 2017).
moderating CR response. Elastic net regularized regression is such an Therapy was delivered by trained postgraduate psychologists with
approach allowing the examination of multiple variables while reducing supervision from an experienced clinical psychologist. Therapy
the model variance and minimizing the risk of overfitting or false pos­ comprised one-on-one sessions, either in person or remotely (e.g., video
itives (Zou & Hastie, 2005). call), and supplementary independent homework sessions, adjusted to
Here, we will explore baseline factors potentially moderating participant needs. The target for therapy engagement was 2–3 hourly
response to CR using data from a cohort of euthymic patients with BD sessions per week, aiming for a total of 30–40 sessions. A threshold of 20
taking part in a randomized trial (Strawbridge et al., 2021). Cognitive h of CIRCuiTS training was predefined as the minimum for treatment
and functional outcomes were assessed before and after the intervention completion. Further details in Supplementary Methods.
and putative moderators, such as sociodemographic variables, illness
characteristics and clinical symptoms, premorbid IQ, subjective com­ 2.3.2. Treatment-as-usual
plaints and objective cognition measures, were collected at baseline. Our Participants in both the active and the control group continued any
findings will generate hypotheses to be tested in future studies and treatments they were previously receiving, including medications and
provide initial indications about patient subgroups that are less likely to psychosocial interventions not explicitly targeting cognitive func­
respond to CR and may require adaptations on the way therapy is tioning, throughout the trial without any interference from the study
delivered. team.

2.4. Measures

Baseline measures and treatment outcomes were assessed blind to

2
D. Tsapekos et al. Behaviour Research and Therapy 151 (2022) 104054

treatment allocation. scores in the outcomes of interest. For each outcome, we computed
Cohen’s d (i.e., mean post-treatment difference between groups divided
2.4.1. Baseline assessments by pooled baseline SD) as an estimate of CR effect size.
Information on sociodemographic characteristics (age, gender, edu­
cation) and clinical variables (BD subtype, age of onset, illness duration, 2.5.2. Identifying and examining response moderators
current medications) was collected using a structured interview. Re­ Our analysis sought to identify and examine potential moderators of
sidual depressive and (hypo)manic symptoms were assessed using the the CR effect compared to TAU (Kraemer, Wilson, Fairburn, & Agras,
HAMD and the YMRS. The Hamilton Rating Scale for Anxiety (HAMA) 2002). We considered factors previously examined in research for peo­
(Hamilton, 1959) was used to evaluate anxiety symptoms. Premorbid IQ ple with schizophrenia and BD, as well as variables showing a significant
was estimated using the Test of Premorbid Function (TOPF) (Wechsler, association with baseline cognitive and functional measures in our
2011), and subjective cognitive complaints were examined with the cohort (16 variables, Supplementary Table 1). The list included socio­
self-report Perceived Deficits Questionnaire (PDQ) (Sullivan, Edgley, & demographic and illness-history characteristics, medication use at study
Dehoux, 1990). entry, measures of symptom severity, perceived cognitive deficits and
normative cognition at baseline (i.e., performance compared to general
2.4.2. Cognitive and functional outcomes of CR population norms). None of these putative moderators had missing data.
Four cognitive tests showing significant improvement between To identify potential response moderators for each outcome we first
groups in the original study were used for this analysis, assessing ran Pearson’s correlations to evaluate whether baseline variables were
different cognitive domains: individually associated with post-treatment outcome changes in the CR
group. We then performed model selection with elastic net regularized
• Processing speed, with the Digit-symbol coding from the Wechsler Adult regression (Zou & Hastie, 2005), using the GLMNET package in R
Intelligence Scale 4th edition (Wechsler, 2014) (Friedman, Hastie, & Tibshirani, 2010). Regularized regression is an
• Attention and working memory, using the Digit span (forward, back­ extension of linear modelling penalizing coefficient estimates to avoid
ward and sequencing) from the Wechsler Adult Intelligence Scale 4th overfitting. Elastic net selects predictor variables with a combination of
edition (Wechsler, 2014) the LASSO (Tibshirani, 1996) and the Ridge regression (Hoerl & Ken­
• Verbal memory, using the Verbal paired associates II (VPA2; delayed nard, 2000) penalties which enables variable selection and coefficient
free recall) from the Wechsler Memory Scale 4th edition (Wechsler, shrinking. It was applied with repeated 10-fold cross-validation to
2009) identify the optimal tuning parameters (alpha and lambda) correspond­
• Executive functioning, using the Hotel test (Manly, Hawkins, Evans, ing to the model with the minimum cross-validated prediction error
Woldt, & Robertson, 2002). (MSE). Cross-validation was repeated 10 times to take the average MSE
of the optimal tuning parameters and to minimize results variation. The
Raw scores from each test were transformed to age- and education- final model retained only predictors with non-zero coefficients.
corrected standardized scores (z scores; Mean = 0, SD = 1) using the Finally, we tested retained variables as moderators of the CR effect
test manuals. Higher scores reflected better performance for all tests. A relative to TAU, to examine whether the effect of baseline variables
global cognition composite score was calculated by averaging the z predicting improvement was specific to the CR group (Kraemer, 2016).
scores of individual tests. Moderation models were fitted for each outcome including the treat­
Psychosocial functioning was assessed using the Functional Assess­ ment group (CR/TAU), the putative moderator, and their two-way
ment Short Test (FAST) (Rosa et al., 2007), a validated scale designed to interaction term as predictors, while also controlling for the baseline
measure functional difficulties regularly reported by people with BD. score of the respective outcome, age and education (Kraemer et al.,
FAST evaluated six different domains of daily life functioning (i.e., au­ 2002). Moderation analysis was conducted with the PROCESS macro for
tonomy, occupation, cognition, financial issues, interpersonal relation­ SPSS which uses percentile bootstrapping (5000 repetitions) to estimate
ships, leisure time) with score reduction from baseline representing coefficients and confidence intervals for interaction effects (version 3.5)
greater levels of functional improvement. (Hayes, 2017). This analysis was conducted separately for each outcome
Attainment of personal recovery goals was examined using the Goal and was restricted to participants with complete post-treatment
Attainment Scale (GAS) (Turner-Stokes, 2009). GAS provided a sys­ outcome data. Missing data were assumed to be missing at random
tematic format for quantifying the extent to which participants achieved (MAR) and observed baseline variables (including outcomes) were
the expected levels of performance in their goals (defined at baseline examined as factors driving missingness (Jakobsen, Gluud, Wetterslev,
according to the needs and personal objectives of each participant) & Winkel, 2017). Any predictors of missingness were included in the
during the intervention period. Attainment was scored in a standardized analysis as covariates.
way across participants with higher scores indicating greater goal
achievement. 2.5.3. Power considerations
Power analyses were carried out using G*Power (version 3.1). Given
2.5. Statistical analysis the sample size after accounting for attrition (n = 72) and the number of
repeated measurements (i.e., two time-points), our study was 80%
Analyses were conducted using the R software (version 3.6, www. powered at an alpha level of 0.05 to detect post-treatment outcome
r-project.org) and SPSS (version 26; IBM, New York). All continuous differences equivalent to small or higher effect sizes (d ≥ 0.26) between
variables, including potential moderators and outcomes, were checked the CR and the TAU group. For moderation models, given the sample
for normality of distributions using the Shapiro-Wilk test and log size (n = 72) and the number of included predictors (n = 6), our analysis
transformation was applied to conform non-normally distributed was powered at 80% to detect small-to-medium or higher effect sizes (f2
variables. ≥ 0.08) for the interaction coefficient between the treatment and the
moderator, at an alpha level of 0.05.
2.5.1. Estimating treatment outcomes
Given that here we examine an extended sample compared to the 3. Results
original trial, we estimated the effect of CR versus TAU for post-
treatment cognitive and functional outcomes using repeated measures A total of 80 participants were randomized to CR (n = 40) or TAU (n
ANOVA models (estimating main effects of treatment group and time, = 40). Baseline characteristics for the whole sample and the two treat­
and a group × time interaction effect), which accounted for baseline ment groups are presented in Table 1. There were no missing data for

3
D. Tsapekos et al. Behaviour Research and Therapy 151 (2022) 104054

Table 1
Demographic and clinical characteristics of the treatment groups at baseline.
CR group (n = 40) TAU group (n = 40) Test statistic p value

Age (years), mean (s.d.) 41.8 (13.9) 42.6 (11.8) F = 0.08 0.78
Gender, n (%)
2
Women 30 (75.0) 27 (67.5) χ = 0.55 0.46
Men 10 (25.0) 13 (32.5)
Education (years), mean (s.d.) 15.8 (2.7) 15.9 (2.1) F = 0.03 0.87
Premorbid IQ (TOPF), mean (s.d.) 108.9 (7.3) 109.4 (7.3) F = 0.10 0.75
BD type, n (%)
Type I 26 (65.0) 27 (67.5) χ2 = 0.06 0.81
Type II 14 (35.0) 13 (32.5)
2
History of psychosis, n (%) 23 (57.5) 26 (65.0) χ = 0.47 0.49
Age of onset (years), mean (s.d.) 30.4 (12.5) 31.5 (10.9) F = 0.18 0.68
Diagnosis duration (years), mean (s.d.) 11.1 (10.2) 10.6 (7.4) F = 0.05 0.83
Number of hospitalizations, mean (s.d.) 2.5 (2.9) 2.4 (2.9) F = 0.07 0.79
Number of current medications, mean (s.d.) 2.3 (1.5) 2.6 (1.5) F = 0.79 0.38
Use of antipsychotic medication, n (%) 29 (72.5) 30 (75.0) χ2 = 0.07 0.80
Previous psychological therapies, mean (s.d.) 1.8 (1.1) 2.1 (1.9) F = 0.64 0.43
HAMD, mean (s.d.) 4.1 (2.6) 3.6 (2.5) F = 0.75 0.39
YMRS, mean (s.d.) 2.4 (2.3) 2.2 (2.4) F = 0.11 0.75
HAMA, mean (s.d.) 5.6 (4.8) 5.9 (4.1) F = 0.09 0.76
PDQ, mean (s.d.) 35.7 (14.9) 35.9 (12.9) F = 0.01 0.94
Global cognition, mean (s.d.) − 0.18 (0.63) − 0.27 (0.64) F = 0.42 0.52

Notes: BD: Bipolar Disorder; CR: Cognitive remediation; HAMA: Hamilton Anxiety Rating Scale; HAMD: Hamilton Depression Rating Scale 17 items; PDQ: Perceived
Cognitive Deficits; TAU: Treatment-as-usual; TOPF: Test of Premorbid Functioning; YMRS: Young Mania Rating Scale.
F-statistic: Statistic for One-way Analysis of Variance; χ 2-statistic: Statistic for Chi-squared test.

participant characteristics and baseline measures. All baseline variables functioning and large improvement in goal attainment for the CR group.
were comparable between the two groups. From the outcome measures,
only the GAS was missing for two participants at baseline (one per
3.2. Selection of moderators
group). Post-treatment data were obtained for 93% and 88% of partic­
ipants in the CR and TAU groups, respectively. No baseline predictors of
After adjusting the significance level for multiple comparisons
missingness were identified.
(corrected alpha = 0.003) no participant characteristics or baseline
measures were significantly correlated with any outcome changes
3.1. Intervention outcomes (Supplementary Table 2). Elastic net regression retained only poorer
normative cognitive performance as a potential moderator of improve­
Findings for the extended sample were in line with the primary CRiB ment in global cognition following CR. Lower baseline performance was
analysis, showing that CR significantly benefited treatment outcomes also predictive of post-treatment improvement for verbal memory
compared to TAU (Table 2). Adjusted mean differences between groups (Supplementary Table 3). No variables were retained for other indi­
corresponded to a medium effect size for global cognition and small-to- vidual cognitive domains.
medium effect sizes for individual cognitive domains. Between-group Improvement in psychosocial functioning was associated with
effect sizes indicated medium improvement in psychosocial poorer baseline global cognition, fewer education years, and higher

Table 2
Summary statistics and adjusted between-group mean differences (CR minus TAU) for cognitive and functional outcomes at post-treatment.
Outcomes CR group TAU group Adjusted mean difference (95% CI) F-statistic p d

Mean (SD) n Mean (SD) n

Cognition composite
Baseline − 0.18 (0.63) 40 − 0.28 (0.64) 40
Week13 0.35 (0.56) 37 − 0.12 (0.56) 35 0.45 (0.30, 0.61) 34.562 <0.001 0.71
Hotel test
Baseline − 0.50 (1.07) 40 − 0.40 (1.16) 40
Week13 0.59 (0.78) 37 0.00 (0.90) 35 0.64 (0.30, 0.98) 14.341 <0.001 0.58
VPA2
Baseline − 0.19 (1.12) 40 − 0.52 (1.08) 40
Week13 0.28 (1.00) 37 − 0.38 (0.96) 35 0.41 (0.12, 0.69) 8.138 0.006 0.37
Coding
Baseline − 0.21 (0.70) 40 − 0.29 (0.75) 40
Week13 0.16 (0.85) 37 − 0.20 (0.71) 35 0.33 (0.10, 0.57) 8.191 0.007 0.46
Digit span
Baseline 0.07 (0.66) 40 0.12 (0.78) 40
Week13 0.36 (0.72) 37 0.09 (0.64) 35 0.36 (0.13, 0.59) 9.995 0.002 0.50
FAST total score
Baseline 23.5 (10.1) 40 20.2 (9.5) 40
Week13 19.1 (9.5) 37 20.5 (9.8) 35 − 4.7 (− 2.6, − 6.9) 18.921 <0.001 0.48
GAS total score
Baseline 33.9 (4.1) 39 33.9 (4.4) 39
Week13 52.0 (9.9) 36 38.8 (7.7) 34 13.2 (9.3, 17.2) 44.926 <0.001 3.14

Notes: CR: Cognitive remediation; FAST: Functional Assessment Short Test; GAS: Goal Attainment Scale; TAU: Treatment-as-usual; VPA2: Verbal Paired Associates –
delayed free recall; d = Cohen’s d effect size (mean difference between groups divided by pooled baseline standard deviation).

4
D. Tsapekos et al. Behaviour Research and Therapy 151 (2022) 104054

residual depressive symptoms at baseline. These were the factors complaints at baseline (interaction β = 0.28, 95% CI: 0.12 to 0.53, p =
considered in the moderation analysis. For goal attainment, the vari­ 0.03, f2 = 0.09) and those who had completed more psychological
ables associated with improvement and considered as moderators were therapies in the past (interaction β = 0.29, 95% CI: 0.02 to 0.57, p =
younger age, female gender, more previous psychological therapies, 0.02, f2 = 0.08).
more subjective cognitive complaints and better cognitive performance Unlike cognition and functioning, CR benefits in goal attainment
at baseline (Supplementary Table 3). were more pronounced for participants with higher baseline cognitive
performance (Fig. 1). However, this moderating effect was not signifi­
3.3. Moderation of CR effects cant (interaction β = 0.48, 95% CI: 0.03 to 0.98, p = 0.09, f2 = 0.04).
Details for all GAS models in Supplementary Table 6.
No baseline factors predicting post-treatment improvement in
cognition and psychosocial functioning for therapy recipients moder­ 4. Discussion
ated the effect of CR compared to TAU (all p > 0.02; Supplementary
Tables 4–5). The effect of baseline factors was not specific to the treat­ This is one of the first studies to examine multiple pre-treatment
ment group and benefits for CR recipients compared to TAU were of a factors as response moderators following CR in euthymic people with
similar size across the range of each putative moderator. For baseline BD. Baseline cognitive performance was associated with post-treatment
cognitive performance, this is illustrated in Fig. 1. changes across outcomes for participants receiving CR. However, there
For goal attainment, baseline perceived deficits and previous psy­ was no response moderation for cognition and psychosocial functioning,
chological therapies significantly moderated treatment response with while this interaction only trended towards significance for goal
small-to-medium effect sizes (Fig. 2): the effect of CR over TAU was attainment. A moderating effect was detected for baseline subjective
greater for participants who reported more subjective cognitive cognitive complaints, with those who reported more severe deficits
showing larger improvements on personal recovery goals. Pre-treatment
level of cognitive performance and severity of cognitive complaints may
be useful patient characteristics to inform the personalization of CR for
people with BD.

4.1. Who benefits from CR?

Our results mirror those in schizophrenia suggesting that therapy

recipients with lower cognitive performance at baseline are more likely
to improve in response to CR (DeTore, Mueser, Byrd, & McGurk, 2019;
Rodewald et al., 2014; Tan et al., 2019; Twamley, Burton, & Vella,
2011). This relationship has been observed particularly for CR ap­
proaches similar to the one we used in this study. In contrast, CR ap­
proaches relying heavily on intensive, drill-and-practice training have
found better treatment response for those with higher cognitive per­
formance at baseline (Fiszdon, Cardenas, Bryson, & Bell, 2005; Kurtz,
Seltzer, Fujimoto, Shagan, & Wexler, 2009; Lindenmayer et al., 2017).
Baseline cognitive performance did not moderate CR effects relative to
TAU, despite affecting cognitive and psychosocial functioning
improvement for the treatment group (Fig. 1). Thus, we cannot exclude
that this improvement reflects a natural regression to the mean for lower
baseline scores. More pronounced gains observed in patients with poorer
cognition may be because these people benefit from greater “room for
improvement”, while those with milder or no deficits might be more
susceptible to ceiling effects (Miskowiak et al., 2017).
The absence of a moderation effect suggests that most euthymic
patients with BD may substantially benefit from CR compared to only
receiving the routinely available treatment (e.g., pharmacotherapy),
independently of their pre-treatment cognitive level. This contradicts
findings from functional remediation where a significant treatment ef­
fect on verbal memory was observed only for cognitively impaired
participants at baseline (Bonnin et al., 2016). The difference might be
explained by the characteristics of our therapy paradigm. CIRCuiTS
combines rigorous cognitive training with an emphasis on strategy use
and metacognitive skills to facilitate new learning, which might explain
why the intervention was able to benefit not only most severely
impaired individuals but also participants across the range of baseline
cognitive performance.
Achievement of personal goals was significantly moderated by the
level of subjective cognitive complaints at baseline, with greater
response for those with more pronounced complaints (Fig. 2). This is
Fig. 1. The effect of global cognitive performance at baseline on post-treatment
global cognition, psychosocial functioning and goal-attainment per treatment consistent with previous evidence suggesting larger improvements after
group. CR for patients with higher self-reported cognitive difficulties (Twamley
CR: Cognitive remediation; FAST: Functional assessment short test; GAS: Goal et al., 2011). A possible explanation is that these participants perceived
attainment scale; TAU: Treatment-as a therapy targeting cognition as more useful and were more likely to
-usual. value the input of CR than people with fewer subjective deficits. This

5
D. Tsapekos et al. Behaviour Research and Therapy 151 (2022) 104054

Fig. 2. Moderating effect of previous psychological therapies (Panel A) and subjective cognitive complaints at baseline (Panel B) on goal attainment per treatment
group.
CR: Cognitive remediation; PDQ: Perceived cognitive deficits; TAU: Treatment-as
-usual.

may have led to greater motivation to engage with the therapy. Like­ difficulties might reflect a poor level of metacognitive knowledge, one’s
wise, participants with greater previous experience with psychological awareness about their own cognitive problems (Cella et al., 2015a,
therapies were likely more familiar with the way these therapies work 2015b). Although metacognitive training is embedded in our CR para­
and so were prepared to engage with the therapeutic process. digm, patients with poor awareness of their difficulties may further
The association between baseline cognitive performance and CR ef­ benefit from additional strategy use and therapist input to prompt
fects on goal attainment was not significant. The direction of this effect metacognitive skills. Associating these therapy components with
differed from those of cognition and psychosocial functioning though, selected recovery goals might be useful for achieving greater benefits. In
with higher cognitively performing CR participants being more likely to addition, devoting more therapy time on transfer activities bridging
benefit (Fig. 1). The GAS is a personalized measure which represents a cognitive training with selected goals might be another adjustment to
treatment outcome different than traditional cognitive and functional facilitate goal attainment for these patients.
measures (Wykes et al., 2018). This might explain the inconsistency
between outcomes in relation to the role of baseline cognition. If
assessed in an appropriately powered sample, it is possible that this ef­ 4.3. Strengths and limitations
fect would have reached statistical significance. Our post-hoc power
analysis, given the estimated effect size, suggested that a sample size of Moderation analyses in this study were based on data from a high-
139 participants would be required for this effect to reach significance. quality randomized trial showing CR-related improvements in cogni­
We speculate that cognitively intact or high-performing patients might tive and functional outcomes for people with BD. Our cohort was in full
be more competent in persistently pursuing selected recovery goals clinical remission and treatment groups were balanced both in terms of
during the therapy, but this is yet to be evidenced. numbers and baseline characteristics. We used robust analytical ap­
proaches to select putative moderators and to estimate interaction
effects.
4.2. Research and clinical implications Our study had a number of limitations. We only included normative
performance in global cognition as a predictor of improvement. Future
Our study provides a framework for future, hypothesis-driven studies research will need to parse out the relative contributions of different
to investigate the moderating effect of baseline variables using larger cognitive abilities at baseline for cognitive improvement after CR
samples or aggregated datasets from multiple trials. As previously sug­ (Ramsay et al., 2018). In addition, we only examined baseline factors
gested, in the process of identifying CR moderators it is important to predicting changes in the CR group to identify response moderators.
consider for which outcomes patient characteristics are relevant, since Thus, we might have missed moderation effects driven by differential
these potentially require therapy adaptations to improve treatment changes in the control group. However, such effects cannot be inter­
benefits (Seccomandi et al., 2019). Based on our findings, improvements preted based on the impact of CR or inform therapy adaptations to
in cognition and psychosocial functioning do not appear to be affected improve outcomes.
by sociodemographic, illness-history, clinical or cognitive characteris­ Moderation analysis was not powered to detect small moderating
tics at baseline. For these outcomes, most people with BD can benefit effects (i.e., potential type II errors) due to the modest sample size.
without adaptations. However, effects of larger strength are more informative for potential
Attainment of personal recovery goals might be a more suitable therapy adaptations. Our sample consisted primarily of middle-aged
outcome to consider for personalizing CR. For example, patients with participants (mean age: 42 years old), with an average illness duration
less pronounced subjective complaints could benefit from therapy ad­ of approximately 10 years. Our findings might be representative of pa­
aptations, as they may underestimate their cognitive difficulties which is tients with an established BD diagnosis, but may not generalize to early-
common in people with BD (Torres, Mackala, Kozicky, & Yatham, 2016; stage BD patients. Finally, this was a complete case analysis, however we
Van Camp, Sabbe, & Oldenburg, 2019). One possible explanation is that had only a small percentage of missing data (>10%; only in dependent
the mismatch between subjective complaints and objective cognitive variables) and did not identify any predictors of missingness.

6
D. Tsapekos et al. Behaviour Research and Therapy 151 (2022) 104054

5. Conclusions Acknowledgements

The effect of CR on cognition and psychosocial functioning does not We are grateful to all CRiB study participants, service user repre­
seem to be influenced by pre-treatment patient characteristics, indi­ sentatives, and all students and researchers from Centre for Affective
cating that most people with BD will be able to benefit without therapy Disorders who contributed to the CRiB study. We also thank King’s
adaptations. For recovery goals though, individuals seem to respond to College Hospital Clinical Research Facility and King’s College Clinical
CR differently depending on baseline characteristics. Goal attainment Trials Unit, OPTIMA mood disorders service and SLaM Affective Disor­
might be a key outcome for progressing therapy personalization and ders Service.
improving treatment benefits through adaptations. Future, hypothesis-
driven studies are warranted to consolidate our knowledge on CR Appendix A. Supplementary data
moderators and evaluate whether tailoring CR according to baseline
characteristics would increase therapy benefits for attainment of re­ Supplementary data to this article can be found online at https://doi.
covery goals and other relevant outcomes. org/10.1016/j.brat.2022.104054.

Role of the funding source References

Anaya, C., Aran, A. M., Ayuso-Mateos, J. L., Wykes, T., Vieta, E., & Scott, J. (2012).
This paper represents independent research part-funded by the Na­
A systematic review of cognitive remediation for schizo-affective and affective
tional Institute for Health Research (NIHR) Maudsley Biomedical disorders. Journal of Affective Disorders, 142(1), 13–21.
Research Centre at South London and Maudsley NHS Foundation Trust Bellani, M., Biagianti, B., Zovetti, N., Rossetti, M. G., Bressi, C., Perlini, C., et al. (2019).
The effects of cognitive remediation on cognitive abilities and real-world functioning
and King’s College London. The views expressed are those of the author
among people with bipolar disorder: A systematic review: Special section on
(s) and not necessarily those of the NIHR or the Department of Health “translational and neuroscience studies in affective disorders”. Section editor, maria
and Social Care. DT would like to acknowledge that this scientific paper nobile md, PhD. This section of JAD focuses on the relevance of translational and
was supported by the Onassis Foundation - Scholarship ID: F ZO077-1/ neuroscience studies in providing a better understanding of the neural basis of
affective disorders. The main aim is to briefly summaries relevant research findings
2018–2019. AHY and TW would like to acknowledge their NIHR Senior in clinical neuroscience with particular regards to specific innovative topics in mood
Investigator award. and anxiety disorders. Journal of Affective Disorders, 257, 691–697. https://doi.org/
10.1016/j.jad.2019.07.059
Biagianti, B., Castellaro, G. A., & Brambilla, P. (2021). Predictors of response to cognitive
Contributors remediation in patients with major psychotic disorders: A narrative review. Journal
of Affective Disorders, 281, 264–270. https://doi.org/10.1016/j.jad.2020.12.011
Bonnin, C.d. M., Reinares, M., Martínez-Arán, A., Balanza-Martinez, V., Sole, B.,
DT, TW and AHY conceived the study idea and planned this study.
Torrent, C., et al. (2016). Effects of functional remediation on neurocognitively
DT and RS were involved in data collection. DT carried out the analysis impaired bipolar patients: Enhancement of verbal memory. Psychological Medicine,
and prepared the first manuscript with support from RS. MC, TW and 46(2), 291.
AHY supervised the project. All authors provided feedback, contributed Brissos, S., Dias, V. V., & Kapczinski, F. (2008). Cognitive performance and quality of life
in bipolar disorder. Canadian Journal of Psychiatry, 53(8), 517–524. https://doi.org/
to the final draft of the paper and approved the manuscript. 10.1177/070674370805300806
Burdick, K. E., Russo, M., Frangou, S., Mahon, K., Braga, R. J., Shanahan, M., et al.
(2014). Empirical evidence for discrete neurocognitive subgroups in bipolar
Ethics statement
disorder: Clinical implications. Psychological Medicine, 44(14), 3083–3096. https://
doi.org/10.1017/S0033291714000439
The authors assert that all procedures contributing to this work Cella, M., Reeder, C., & Wykes, T. (2015). Cognitive remediation in schizophrenia—now
comply with the ethical standards of the relevant national and institu­ it is really getting personal. Current Opinion in Behavioral Sciences, 4, 147–151.
Cella, M., Reeder, C., & Wykes, T. (2015). Lessons learnt? The importance of
tional committees on human experimentation and with the Helsinki metacognition and its implications for cognitive remediation in schizophrenia.
Declaration of 1975, as revised in 2008. All procedures involving human Frontiers in Psychology, 6, 1259. https://doi.org/10.3389/fpsyg.2015.01259
patients were approved by the City Road & Hampstead NHS Research Cullen, B., Ward, J., Graham, N. A., Deary, I. J., Pell, J. P., Smith, D. J., et al. (2016).
Prevalence and correlates of cognitive impairment in euthymic adults with bipolar
Ethics Committee (reference 15/LO/1557). disorder: A systematic review. Journal of Affective Disorders, 205, 165–181. https://
doi.org/10.1016/j.jad.2016.06.063
Deckersbach, T., Nierenberg, A. A., Kessler, R., Lund, H. G., Ametrano, R. M., Sachs, G., &
CRediT authorship contribution statement
Dougherty, D. (2010). Cognitive rehabilitation for bipolar disorder: An open trial for
employed patients with residual depressive symptoms. CNS Neuroscience and
Dimosthenis Tsapekos: Conceptualization, Investigation, Method­ Therapeutics, 16(5), 298–307.
ology, Formal analysis, Visualization, Writing – original draft. Rebecca DeTore, N. R., Mueser, K. T., Byrd, J. A., & McGurk, S. R. (2019). Cognitive functioning
as a predictor of response to comprehensive cognitive remediation. Journal of
Strawbridge: Investigation, Data curation, Writing – review & editing. Psychiatric Research, 113, 117–124. https://doi.org/10.1016/j.
Matteo Cella: Methodology, Writing – review & editing, Supervision. jpsychires.2019.03.012
Til Wykes: Conceptualization, Writing – review & editing, Supervision. Fiszdon, J. M., Cardenas, A. S., Bryson, G. J., & Bell, M. D. (2005). Predictors of
remediation success on a trained memory task. The Journal of Nervous and Mental
Allan H. Young: Conceptualization, Writing – review & editing, Disease, 193(9), 602–608. https://doi.org/10.1097/01.nmd.0000177790.23311.ba
Supervision. Friedman, J., Hastie, T., & Tibshirani, R. (2010). Regularization paths for generalized
linear models via coordinate descent. Journal of Statistical Software, 33(1), 1–22.
Hamilton, M. (1959). The assessment of anxiety states by rating. British Journal of Medical
Declaration of competing interest Psychology, 32(1), 50–55. https://doi.org/10.1111/j.2044-8341.1959.tb00467.x
Hamilton, M. (1960). A rating scale for depression. Journal of neurology, neurosurgery, and
psychiatry, 23(1), 56.
AHY reports paid lectures and advisory boards for the following Hayes, A. F. (2017). Introduction to mediation, moderation, and conditional process analysis:
companies with drugs used in affective and related disorders: AstraZe­ A regression-based approach. Guilford publications.
Hoerl, A. E., & Kennard, R. W. (2000). Ridge regression: Biased estimation for
neca (AZ), Eli Lilly, Lundbeck, Sunovion, Servier, LivaNova, Allegan, nonorthogonal problems. Technometrics, 42(1), 80–86. https://doi.org/10.2307/
Bionomics, Sumitomo, Dainippon Pharma and Janssen; consultant to 1271436
Johnson & Johnson and LivaNova; principal investigator on the Restore- Jakobsen, J. C., Gluud, C., Wetterslev, J., & Winkel, P. (2017). When and how should
multiple imputation be used for handling missing data in randomised clinical trials –
Life VNS registry study funded by LivaNova, on ESKETINTRD3004 trial
a practical guide with flowcharts. BMC Medical Research Methodology, 17(1), 162.
funded by Janssen, and the P-TRD trial funded by Compass; no share­ https://doi.org/10.1186/s12874-017-0442-1
holdings in pharmaceutical companies. RS reports a paid lecture from Kraemer, H. C. (2016). Messages for clinicians: Moderators and mediators of treatment
Lundbeck. TW is the creator of CIRCuiTS cognitive remediation soft­ outcome in randomized clinical trials. American Journal of Psychiatry, 173(7),
672–679. https://doi.org/10.1176/appi.ajp.2016.15101333
ware. DT and MC report no conflict of interest for this work.

7
D. Tsapekos et al. Behaviour Research and Therapy 151 (2022) 104054

Kraemer, H. C., Wilson, G. T., Fairburn, C. G., & Agras, W. S. (2002). Mediators and Strawbridge, R., Tsapekos, D., Hodsoll, J., Mantingh, T., Yalin, N., McCrone, P., &
moderators of treatment effects in randomized clinical trials. Archives of General Young, A. H. (2021). Cognitive remediation therapy for patients with bipolar
Psychiatry, 59(10), 877–883. https://doi.org/10.1001/archpsyc.59.10.877 disorder: A randomised proof-of-concept trial. Bipolar Disorders, 23(2), 196–208.
Kurtz, M. M., Seltzer, J. C., Fujimoto, M., Shagan, D. S., & Wexler, B. E. (2009). Predictors https://doi.org/10.1111/bdi.12968
of change in life skills in schizophrenia after cognitive remediation. Schizophrenia Sullivan, M. J., Edgley, K., & Dehoux, E. (1990). A survey of multiple sclerosis: I.
Research, 107(2), 267–274. https://doi.org/10.1016/j.schres.2008.10.014 Perceived cognitive problems and compensatory strategy use. Canadian Journal of
Lewandowski, K. E., Sperry, S. H., Cohen, B. M., Norris, L. A., Fitzmaurice, G. M., Rehabilitation, 4(2), 99–105.
Ongur, D., et al. (2017). Treatment to enhance cognition in bipolar disorder (TREC- Tan, S., Zhu, X., Fan, H., Tan, Y., Yang, F., Wang, Z., et al. (2019). Who will benefit from
BD): Efficacy of a randomized controlled trial of cognitive remediation versus active computerized cognitive remediation therapy? Evidence from a multisite randomized
control. Journal of Clinical Psychiatry, 78(9), e1242–e1249. https://doi.org/10.4088/ controlled study in schizophrenia. Psychological Medicine, 1–11. https://doi.org/
JCP.17m11476 10.1017/S0033291719001594
Lindenmayer, J. P., Ozog, V. A., Khan, A., Ljuri, I., Fregenti, S., & McGurk, S. R. (2017). Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the
Predictors of response to cognitive remediation in service recipients with severe Royal Statistical Society: Series B, 58(1), 267–288. https://doi.org/10.1111/j.2517-
mental illness. Psychiatric Rehabilitation Journal, 40(1), 61–69. https://doi.org/ 6161.1996.tb02080.x
10.1037/prj0000252 Torres, I. J., Mackala, S. A., Kozicky, J. M., & Yatham, L. N. (2016). Metacognitive
Manly, T., Hawkins, K., Evans, J., Woldt, K., & Robertson, I. H. (2002). Rehabilitation of knowledge and experience in recently diagnosed patients with bipolar disorder.
executive function: Facilitation of effective goal management on complex tasks using Journal of Clinical and Experimental Neuropsychology, 38(7), 730–744. https://doi.
periodic auditory alerts. Neuropsychologia, 40(3), 271–281. org/10.1080/13803395.2016.1161733
Medalia, A., Saperstein, A. M., Hansen, M. C., & Lee, S. (2018). Personalised treatment Tsapekos, D., Seccomandi, B., Mantingh, T., Cella, M., Wykes, T., & Young, A. H. (2020).
for cognitive dysfunction in individuals with schizophrenia spectrum disorders. Cognitive enhancement interventions for people with bipolar disorder: A systematic
Neuropsychological Rehabilitation, 28(4), 602–613. https://doi.org/10.1080/ review of methodological quality, treatment approaches, and outcomes. Bipolar
09602011.2016.1189341 Disorders, 22(3), 216–230. https://doi.org/10.1111/bdi.12848
Miskowiak, K. W., Burdick, K. E., Martinez-Aran, A., Bonnin, C. M., Bowie, C. R., Tse, S., Chan, S., Ng, K. L., & Yatham, L. N. (2014). Meta-analysis of predictors of
Carvalho, A. F., & Vieta, E. (2017). Methodological recommendations for cognition favorable employment outcomes among individuals with bipolar disorder. Bipolar
trials in bipolar disorder by the international society for bipolar disorders targeting Disorders, 16(3), 217–229. https://doi.org/10.1111/bdi.12148
cognition task force. Bipolar Disorders, 19(8), 614–626. https://doi.org/10.1111/ Turner-Stokes, L. (2009). Goal attainment scaling (GAS) in rehabilitation: A practical
bdi.12534 guide. Clinical Rehabilitation, 23(4), 362–370.
Ott, C. V., Vinberg, M., Kessing, L. V., Bowie, C. R., Forman, J. L., & Miskowiak, K. W. Twamley, E. W., Burton, C. Z., & Vella, L. (2011). Compensatory cognitive training for
(2020). Effect of action-based cognitive remediation on cognitive impairment in psychosis: Who benefits? Who stays in treatment? Schizophrenia Bulletin, 37(Suppl
patients with remitted bipolar disorder: A randomized controlled trial. Bipolar 2), S55–S62. https://doi.org/10.1093/schbul/sbr059
Disorders, 1–13. https://doi.org/10.1111/bdi.13021 Van Camp, L., Sabbe, B. G. C., & Oldenburg, J. F. E. (2019). Metacognitive functioning in
Ramsay, I. S., Ma, S., Fisher, M., Loewy, R. L., Ragland, J. D., Niendam, T., & bipolar disorder versus controls and its correlations with neurocognitive functioning
Vinogradov, S. (2018). Model selection and prediction of outcomes in recent onset in a cross-sectional design. Comprehensive Psychiatry, 92, 7–12. https://doi.org/
schizophrenia patients who undergo cognitive training. Schizophr Res Cogn, 11, 1–5. 10.1016/j.comppsych.2019.06.001
https://doi.org/10.1016/j.scog.2017.10.001 Vita, A., Barlati, S., Ceraso, A., Nibbio, G., Ariu, C., Deste, G., et al. (2021). Effectiveness,
Reeder, C., Huddy, V., Cella, M., Taylor, R., Greenwood, K., Landau, S., et al. (2017). core elements, and moderators of response of cognitive remediation for
A new generation computerised metacognitive cognitive remediation programme for schizophrenia: A systematic review and meta-analysis of randomized clinical trials.
schizophrenia (CIRCuiTS): A randomised controlled trial. Psychological Medicine, JAMA Psychiatry. https://doi.org/10.1001/jamapsychiatry.2021.0620
1–11. https://doi.org/10.1017/S0033291717001234 Wechsler, D. (2009). Wechsler memory scale–Fourth Edition (WMS-IV) technical and
Reeder, C., Pile, V., Crawford, P., Cella, M., Rose, D., Wykes, T., & Callard, F. (2016). The interpretive manual. Pearson Assessments.
feasibility and acceptability to service users of CIRCuiTS, a computerized cognitive Wechsler, D. (2011). Test of premorbid functioning. Bloomington, MN: UK version (TOPF
remediation therapy programme for schizophrenia. Behavioural and Cognitive UK).
Psychotherapy, 44(3), 288–305. Wechsler, D. (2014). Wechsler adult intelligence scale–Fourth Edition (WAIS–IV). San
Reser, M. P., Slikboer, R., & Rossell, S. L. (2019). A systematic review of factors that Antonio, Texas: Psychological Corporation.
influence the efficacy of cognitive remediation therapy in schizophrenia. Australian Wingo, A. P., Harvey, P. D., & Baldessarini, R. J. (2009). Neurocognitive impairment in
and New Zealand Journal of Psychiatry, 53(7), 624–641. https://doi.org/10.1177/ bipolar disorder patients: Functional implications. Bipolar Disorders, 11(2), 113–125.
0004867419853348 https://doi.org/10.1111/j.1399-5618.2009.00665.x
Rodewald, K., Holt, D. V., Rentrop, M., Roesch-Ely, D., Liebrenz, M., Funke, J., et al. Wykes, T., Huddy, V., Cellard, C., McGurk, S. R., & Czobor, P. (2011). A meta-analysis of
(2014). Predictors for improvement of problem-solving during cognitive remediation cognitive remediation for schizophrenia: Methodology and effect sizes. American
for patients with Schizophrenia. Journal of the International Neuropsychological Society Journal of Psychiatry, 168(5), 472–485.
: JINS, 20(4), 455–460. https://doi.org/10.1017/s1355617714000162 Wykes, T., Joyce, E., Velikonja, T., Watson, A., Aarons, G., Birchwood, M., et al. (2018).
Rosa, A. R., Sánchez-Moreno, J., Martínez-Aran, A., Salamero, M., Torrent, C., The CIRCuiTS study (implementation of cognitive remediation in early intervention
Reinares, M., & Ayuso-Mateos, J. L. (2007). Validity and reliability of the functioning services): Protocol for a randomised controlled trial. Trials, 19(1), 183. https://doi.
assessment short test (FAST) in bipolar disorder. Clinical Practice and Epidemiology in org/10.1186/s13063-018-2553-3
Mental Health, 3(1), 5. Wykes, T., & Spaulding, W. D. (2011). Thinking about the future cognitive remediation
Seccomandi, B., Tsapekos, D., Newbery, K., Wykes, T., & Cella, M. (2019). A systematic therapy–what works and could we do better? Schizophrenia Bulletin, 37(Suppl 2),
review of moderators of cognitive remediation response for people with S80–S90. https://doi.org/10.1093/schbul/sbr064
schizophrenia. Schizophrenia Research: Cognition, 100160. https://doi.org/10.1016/j. Yarkoni, T., & Westfall, J. (2017). Choosing prediction over explanation in psychology:
scog.2019.100160 Lessons from machine learning. Perspectives on Psychological Science : A Journal of the
Sheehan, D. V., Lecrubier, Y., Sheehan, K. H., Amorim, P., Janavs, J., Weiller, E., et al. Association for Psychological Science, 12(6), 1100–1122. https://doi.org/10.1177/
(1998). The mini-international neuropsychiatric Interview (M.I.N.I.): The 1745691617693393
development and validation of a structured diagnostic psychiatric interview for Young, R. C., Biggs, J. T., Ziegler, V. E., & Meyer, D. A. (1978). A rating scale for mania:
DSM-IV and ICD-10. Journal of Clinical Psychiatry, 59(Suppl 20), 22–33. quiz 34-57. Reliability, validity and sensitivity. British Journal of Psychiatry, 133, 429–435.
Strawbridge, R., Fish, J., Halari, R., Hodsoll, J., Reeder, C., Macritchie, K., et al. (2016). https://doi.org/10.1192/bjp.133.5.429
The cognitive remediation in bipolar (CRiB) pilot study: Study protocol for a Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net.
randomised controlled trial. Trials, 17(1), 371. https://doi.org/10.1186/s13063- Journal of the Royal Statistical Society: Series B, 67(2), 301–320. https://doi.org/
016-1472-4 10.1111/j.1467-9868.2005.00503.x