International Journal of Clinical and Health Psychology 24 (2024) 100446

Contents lists available at ScienceDirect

International Journal of Clinical and Health Psychology

journal homepage: www.elsevier.es/ijchp

Benchmarks for dialectical behavioural therapy intervention in adults and

adolescents with borderline personality symptoms
Julieta Azevedo a, b, c, *, Diogo Carreiras c, d, Caitlin Hibbs a, b, Raquel Guiomar c, Joshua Osborne b,
Richard Hibbs b, Michaela Swales a, b
a
School of Human and Behavioural Sciences - Bangor University, UK
b
British Isles DBT Training, UK
c
University of Coimbra, Center for Research in Neuropsychology and Cognitive and Behavioral Intervention (CINEICC), Portugal
d
Miguel Torga Higher Institute, Portugal

A R T I C L E I N F O A B S T R A C T

Keywords: Background: Dialectical Behaviour Therapy (DBT) is a multi-component cognitive behavioural intervention with
Benchmarking proven efficacy in treating people with borderline personality disorder symptoms. Establishing benchmarks for
DBT DBT intervention with both adults and adolescents is essential for bridging the gap between research and clinical
Borderline symptoms
practice, improving teams’ performance and procedures.
BPD
EQ-5D
Aim: This study aimed to establish benchmarks for DBT using the EQ-5D, Borderline Symptoms List (BSL) and
BSL Difficulties in Emotion Regulation Scale (DERS) for adults and adolescents.
DERS Methods: After searching four databases for randomised controlled trials and effectiveness studies that applied
standard DBT to people with borderline symptoms, a total of 589 studies were included (after duplicates’
removal), of which 16 met our inclusion criteria. A meta-analysis and respective effect-size pooling calculations
(Hedges-g) were undertaken, and heterogeneity between studies was assessed with I2 and Q tests. Benchmarks
were calculated using pre–post treatment means of the studies through aggregation of adjusted effect sizes and
critical values.
Results: DBT aggregated effect sizes per subsample derived from RCTs and effectiveness studies are presented,
along with critical values, categorised by age group (adults vs adolescents), mode of DBT treatment (full-pro-
gramme vs skills-training) and per outcome measure (EQ-5D, BSL and DERS).
Conclusions: Practitioners from routine clinical practice delivering DBT and researchers can now use these
benchmarks to evaluate their teams’ performance according to their clients’ outcomes, using the EQ-5D, BSL and
DERS. Through benchmarking, teams can reflect on their teams’ efficiency and determine if their delivery needs
adjustment or if it is up to the standards of current empirical studies.

Introduction includes four modes of treatment: individual psychotherapy, skills

training, phone coaching, and consultation for therapists (Linehan,
Dialectical behaviour therapy (DBT) is a multi-component and 2015; Swales & Heard, 2017). The skills are taught in a group format
integrative treatment that synthesises behavioural theory, principles of (and strengthened in individual therapy) and include modules on
Zen practice and dialectics (Linehan, 1993). This therapy was initially emotional regulation, mindfulness, distress tolerance and interpersonal
designed to treat people with chronic suicidality who had received a effectiveness. For less severe clients (without suicide ideation or active
diagnosis of Borderline Personality Disorder (BPD). It was the first self-harm), non-comprehensive skills training can be offered (which
treatment to consistently demonstrate clinical efficacy with this client ideally should still include a consultation team and phone coaching)
group and continues to be the most supported (Stoffers-Winterling et al., (Linehan, 2015; Valentine et al., 2015).
2012, 2022). BPD is a severe disorder characterised by emotional, interpersonal
DBT comprehensive treatment, also referred to as full-programme, and intrapersonal instability, impulsivity, self-harm behaviours and

* Corresponding author at: biDBT Training, Wrexham Technology Park, Croesnewydd Hall, Wrexham LL13 7YP, UK.
E-mail address: [email protected] (J. Azevedo).

https://doi.org/10.1016/j.ijchp.2024.100446
Received 21 September 2023; Accepted 27 January 2024
Available online 6 February 2024
1697-2600/© 2024 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-
nc-nd/4.0/).
J. Azevedo et al. International Journal of Clinical and Health Psychology 24 (2024) 100446

feelings of emptiness and abandonment (American Psychiatric Associ- RCT studies, to obtain a suite of benchmarks for teams to use to compare
ation, 2022; Biskin, 2015; D’Aurizio et al., 2023; Zanarini et al., 2011). against their own routine clinical practice outcomes.
Borderline symptoms usually begin in adolescence and are associated The current study aims to provide benchmarks for DBT treatment by
with negative affect and a high risk of suicide, which remains as high as meta-analysing data from quality RCTs and effectiveness studies deliv-
10 % over a 27-year course (APA, 2022; Videler et al., 2019). This sui- ering standard DBT (full-programme/ or stand-alone skills training) to
cide rate is partially due to emotional dysregulation (Mirkovic et al., adults and adolescents with BPD symptoms and that used the same in-
2021) and low levels of quality of life (IsHak et al., 2013). When struments to measure emotion regulation, health-related quality of life
assessing the efficacy of DBT, researchers have focused on decreasing and borderline symptoms.
borderline symptomatology, increasing skills use (e.g. emotion regula-
tion), as well as decreasing suicide and self-harm behaviour (Stoffers-- Methods
Winterling et al., 2022) and quality of life interfering behaviours, thus
looking for improvements in quality of life (Carter et al., 2010; Chakhssi Benchmarking methodology
et al., 2021; van Asselt et al., 2009).
Scientists, healthcare professionals, and decision-makers worldwide In terms of the steps to start benchmarking, we considered the plan
recognise that evidence-based clinical interventions do not translate described by Lloyd (2004) and the considerations of Bayney (2005),
easily into routine practice due to numerous implementation challenges beginning with the selection of the data collection method and estab-
(Eldh et al., 2017), resulting in a research-practice gap (Chorpita & lishing the measures to act as benchmarks. Furthermore, we reviewed
Daleiden, 2014; Gotham, 2006; Teachman et al., 2012). One recognised specific studies that benchmarked mental health interventions, adopting
barrier is the practitioner’s scepticism (Lilienfeld et al., 2013), partially their recommendations and structure (Delgadillo et al., 2014; Minami
relating to concerns about whether clients and therapists in RCTs are et al., 2007; Weersing & Weisz, 2002). Minami et al.’s (2008) proposal
representative of those in typical clinical settings (Hunsley & Lee, 2007). to withdraw benchmarks from clinical trials was followed, given its
Delgadillo et al. (2014) conducted a comprehensive examination of clarity and high citation rate in other studies. In order to review the
the widespread implementation of empirically supported treatments in published clinical trials, a thorough database search was conducted,
routine practice and assessing outcomes in everyday clinical settings. In followed by a meta-analysis of the collected data. This analysis included
doing so, they identified notable challenges associated with the mea- published studies using three selected measures (described below) to
surement, definition, and comparative analysis of clinical outcomes. For measure DBT intervention outcomes in adults and adolescents with
future benchmarking initiatives, the authors emphasised the importance borderline features so that DBT teams can use them to assess their per-
of comparing effect size estimates derived from condition-specific formance. Weersing and Weisz (2002) advise estimating benchmarks
measures with those obtained using more generalised distress mea- based on studies that used an intention-to-treat (ITT) approach; how-
sures. Furthermore, the authors emphasised the need for a nuanced ever, excluding studies without clear ITT information would have led to
approach to benchmarks, considering contextual, diagnostic, and pop- a substantial loss of data, mainly from effectiveness studies, hindering
ulation factors in specific settings. the establishment of robust and representative benchmarks. Hence, we
Benchmarking is an outcome assessment strategy used to assess decided to include studies that described only completers data, report-
clinical services (Eisen & Dickey, 1996) by evaluating patients’ data and ing the used method for all the studies.
clinical outcomes (Lovaglio, 2012). In clinical psychology, a benchmark
represents a standard of care or best practice for specific treatments or Selection of outcome measures
interventions, working as a reference point for comparison or mea-
surement. The key takeaway is that benchmarking fundamentally aims The considerations of Delgadillo et al. (2014) on selecting outcome
to enhance practices and processes, ultimately improving outcomes measures to benchmark were followed: weighing up the research evi-
through establishing care standards, identifying and delivering effective dence of the tools, sensitivity to change with the intended population,
treatments, and monitoring care quality (Lloyd, 2004). ease of administration and interpretation and measures that are free to
Only one study, to our knowledge, has established benchmarks for use and easily accessible.
DBT treatment in adult outpatients with BPD (Washburn et al., 2018). Minami et al. (2008) indicated that ideally, the selection of clinical
However, this study had strict inclusion criteria and focused exclusively trials for inclusion for a benchmark should utilise identical measures to
on RCT studies delivering DBT full-programme measuring depression, ensure they match in specificity and reactivity. Following this guideline,
anger and self-harm. Moreover, Washburn et al. (2018) aggregated we selected three measures to assess: health-related quality of life, which
studies that used different instruments to measure the above-mentioned in addition allows for clinical and economic appraisal; difficulties in
outcomes, which has limitations when drawing benchmarks since they regulating emotions, which is considered an important mechanism of
vary in their specificity and reactivity (Minami et al., 2008). Addition- change in DBT; and a disorder-specific measure for borderline
ally, considering their database search was done in 2016, a more symptoms.
up-to-date review is required, especially one including DBT skills groups
intervention and including adolescents. Finally, for a comprehensive Selected instruments and rational
team performance evaluation, it is crucial to employ measures assessing The EQ-5D-3 L (EuroQol Research Foundation, 2018), henceforth
health-related quality of life to inform cost-effectiveness and mecha- referred to as "EQ-5D" is a widely used generic measure of health status
nisms of change within the treatment. consisting of two parts. The first part (the descriptive system) assesses
The project was developed as part of an ongoing collaboration be- health in five dimensions (mobility, self-care, usual activities, pain/-
tween Bangor University, British Isles DBT Training, and NHS England, discomfort, anxiety/depression), each of which has three levels of
formerly Health Education England (HEE), to significantly increase the response (no problems, some problems, extreme problems/unable to),
number of trained DBT clinicians embedded in active DBT programmes. providing a health state profile. Each health state is assigned a summary
As part of the project, there was an aim to benchmark the clinical out- index score based on societal preference weights for the health state.
comes of teams as a means to assess the effectiveness of the training These weights, or utilities, are used to compute Quality-Adjusted Life
programme in producing a return on the training investment. Years (QALYs) in health economic analyses. Health state index for the
The first challenge faced was the lack of a body of evidence to draw United Kingdom ranged from 1 (’perfect’ health state), with higher
from. Due to a general lack of previous studies benchmarking the con- scores indicating higher health utility, to −0.543. EQ-5D’s negative
structs we were interested in assessing, there was a need to first establish range in the interval corresponds to health status ’worse’ than death,
benchmarks based on empirical studies, including both effectiveness and which has face validity in suicidal populations and can vary slightly

2
J. Azevedo et al. International Journal of Clinical and Health Psychology 24 (2024) 100446

between countries.1 Additionally, self-harm benchmarks are already available in Washburn

The second part of the questionnaire consists of a Visual Analogue et al.’s (2018) study.
Scale (VAS) on which patients rate their perceived health from 0 (the
worst imaginable health) to 100 (the best imaginable health). EQ-5D-3 L Search procedures
has revealed good test-retest reliability across studies ranging from 0.53
to 0.83 (Buchholz et al., 2018). The EQ-5D questionnaire is cognitively Our research team performed a literature search in September 2022
undemanding, taking only a few minutes to complete. A version for on the efficacy of DBT for the treatment of BPD. The search terms used
adolescents is also available (EQ-5D-3L-Y), with the same components were "dialectical behavior therapy" or equivalent (e.g., "DBT", "dialec-
and some adjustments to the language only. The anchor points for the tical behaviour therapy"), "borderline personality disorder" or related (e.
EQ-5D, with 0 representing death and less than 1 a state worse than g., "BPD", "borderline symptoms"), and "randomised controlled trial",
death, make the EQ-5D a perfect measure conceptually for treating a "systematic review" or similar (e.g., "RCT", "meta-analysis", "clinical
client group where a central feature is the desire to die by suicide and in trial"). The search was conducted in the following databases: PubMed/
whom suicide rates are high. The measure also aligns well with DBT’s MEDLINE, Embase, PsycINFO and Web of Science. The study selection
primary aim to create a life worth living (i.e., utility score is sensitive to was then performed in two phases. Firstly, through screening the titles
changes in psychotherapy for people with BPD (van Asselt et al., 2009). and abstracts, and secondly, the full texts, according to the inclusion and
The Difficulties in Emotion Regulation Scale (DERS; Gratz & Roemer, exclusion criteria. We also searched for systematic reviews or meta-
2004) measures difficulties in regulating emotions, which is considered analyses on DBT efficacy or effectiveness to ensure that we had
one of the core problems experienced by people with a diagnosis of BPD included all relevant studies. The search was repeated in May 2023 to
(Glenn & Klonsky, 2009). This instrument is one of the most widely used check if new studies had been published, but no additional articles met
to measure this construct (Sloan et al., 2017), and it has been used our inclusion criteria.
effectively to detect changes in interventions for BPD (Stoffers-Winter- We included only original articles published in journals with peer
ling et al., 2022). This self-report questionnaire assesses six components review and used the PICO (Population, Intervention, Comparison,
of difficulties in regulating emotions: nonacceptance of emotional re- Output) approach, which is depicted in Table 1. To include the articles in
sponses, difficulty engaging in goal-directed behaviour, impulse control our review the following characteristics were also confirmed:
difficulties, lack of emotional awareness, limited access to emotion
regulation strategies, and lack of emotional clarity. The 36 items are (a) RCT or effectiveness study design
rated on a 5-point Likert scale ranging between 1 ("almost never") and 5 (b) Conducted in public mental health and community outpatient
("almost always"). The DERS has shown good psychometric properties, settings
with high internal consistency, a Cronbach’s α of 0.93, and test-retest (c) Written in English, French, Spanish, Portuguese or German
reliability of 0.88. There is a short form with 18 items (DERS-SF; (d) Full text available (either in open access or through subscription -
Kaufman et al., 2016), and both versions are validated to use with adults database search from Bangor University and University of
and adolescents (DERS-36, Gratz & Roemer, 2004; Neumann et al., Coimbra)
2009; DERS-SF, Weinberg & Klonsky, 2009). (e) Good quality of the articles using the JBI critical appraisal tools
The Borderline Symptoms List (BSL; Bohus et al., 2009) is a self-report (Joanna Briggs Institute, 2017)
questionnaire with 23 items, which aims to assess BPD symptoms. This
scale’s items cover BPD diagnostic criteria (e.g., affective instability, A similar approach was used for RCTs and effectiveness studies, so
recurrent suicidal behaviour or threats, non-suicidal self-injury, and comparison or control groups were not accounted for comparison. It is
transient dissociative symptoms) and borderline-typical processes such also important to clarify that the studies selected reported that partici-
as self-criticism, emotional vulnerability, self-disgust and loneliness. pants engaged in pharmacotherapy and maintained the treatment pro-
Items are rated on a 5-point Likert scale from 0 ("not at all") to 4 ("very vided by their psychiatrist or GP (as usual in most psychotherapy-
strong"). This instrument was chosen because it has recently been focused studies). Participants were not required to remain on consis-
updated in line with the criteria in DSM-5 and was developed based on tent medication; therefore, medication changes may have occurred.
the experiences of both clinical experts and input from people with a A flow diagram with the identification and selection process is
diagnosis of BPD (Kleindienst et al., 2020). The BSL-23 has a depicted in Fig. 1. A total of 1253 studies were identified and uploaded
single-factor structure and excellent psychometric properties, with high into Rayyan (https://rayyan.ai/), a free web platform that helps expe-
internal consistency, a Cronbach’s α of 0.97, and test-retest reliability of dite the initial screening of abstracts and titles for systematic reviews
0.82 (Bohus et al., 2009). Moreover, it is an instrument tested on over (Ouzzani et al., 2016) and 589 duplicates were eliminated. We included
1000 adults with defined cut-off scores and severity levels (none or low, only original articles published in journals with peer review.
mild, moderate, high, very high, and extremely high) that facilitates A total of 627 studies were excluded after applying our criteria. We
score interpretation. In their study to propose severity levels, Kleindienst selected 37 studies for full-text detailed reading and quality assessment.
et al. (2020) specify that a score of 1.5 is typical for someone with a Of these, 21 were excluded for the following reasons: sample already
diagnosis of BPD but highly extreme compared to someone without a included in one of the selected studies (5 studies); modified DBT (for
psychiatric diagnosis. example, DBT Prolonged Exposure); different length or adaptation for
While many DBT outcome studies focus on suicidal behaviour and non-BPD populations (5 studies); necessary data not reported, namely
Non-Suicidal Self-Injury (NSSI), we decided against endeavouring to the outcomes M and SD (5 studies); low study quality (according to JBI
benchmark these outcomes for several reasons. First, studies use widely critical appraisal tools) (2 studies); mixed samples (4 studies).
varying measures of these constructs, making comparison difficult. The main features (e.g., study design, sample size) of the final 16
Second, the most reliable and valid measures of this construct used in the selected studies can be found in Table 2, of which 13 used adult samples
literature are typically interview-based and beyond the capacity of many and three adolescent samples. These studies were conducted in eight
routine clinical services. Finally, not all clients within a DBT service will different countries: USA (4), Ireland (3), Canada (3), Australia (2), UK
necessarily be suicidal or engaging in NSSI. These reasons make mea- (1), Germany (1), Netherlands (1) and Norway (1).
sures of these constructs less applicable to all clients in a service.
Assessment of studies’ quality

1
For detailed information per country consult - https://euroqol.org/eq-5d-i The full text of 37 studies was read thoroughly, and they were then
nstruments/eq-5d-3l-about/population-norms/ assessed in terms of methodological quality and quality of the report of

3
J. Azevedo et al. International Journal of Clinical and Health Psychology 24 (2024) 100446

data, as well as study bias with the JBI critical appraisal tools (Joanna

Note: M = Mean; SD = Standard Deviation; BPD = Borderline Personality Disorder; PTSD = Post-Traumatic Stress Disorder; DBT = Dialectical Behaviour Therapy; DERS = Difficulties in Emotion Regulation Scale; BSL =
Briggs Institute, 2017). The checklist for randomised controlled trials

• Skills interventions without the four skills modules: emotion regulation, distress tolerance, interpersonal
(13 items) was used for RCTs, and the quasi-experimental studies
checklist (non-randomised experimental studies; nine items) was used
for the effectiveness studies. The authors of the JBI tool do not provide a
cut-off for the scale, stating the result depends on the reason for using
the scale and how rigorous a user of the scale would like to be with the

• Different assessment timepoints, preventing comparison at baseline and post treatment

accepting/excluding criteria. We decided to accept RCTs above eight
(from a maximum possible score of 13), considering that some items
• Adapted interventions with different treatment length or frequency of sessions
result from methodological choices or blinding related parameters that
were not relevant considering the benchmark goal for routine practice.

• Studies which did not report pre-post M/SD from EQ-5D, DERS, or BSL.
• Concomitant psychotherapy other than DBT or enhanced treatment The checklist for quasi-experimental studies includes four items
related to having a comparison group, which depresses the score to a
maximum of five. Thus, provided the studies met the five quality
criteria, we included effectiveness studies without a control group.
• PTSD, patients selected based on BPD+other diagnosis.

Reliability was ensured by having two independent assessors who

rated the quality of the studies separately. Any discrepancies identified
were resolved through consensus discussions with all assessors, resulting
in the final presented score (see Table 2).

Data extraction process and calculation of benchmarks

The pre–post M and SD for DERS, BSL and EQ-5D, as well as the
sample size of the groups that received DBT, were retrieved from the
effectiveness, mindfulness.

papers. In regards to the EQ-5D, we found three studies which only re-
ported the EQ-5D VAS scores and not the utility scores. Therefore, it was
Exclusion Criteria

decided to contact the authors to request their datasets and their

• Older adults

permission to use their outcomes to generate benchmarks. The authors

Sinnaeve et al. (2018) and McMain et al. (2009, 2022) granted
permission to use their data and provided the necessary information and
databases at the beginning of 2023. Once all the necessary information
from the studies was gathered, an analysis for heterogeneity was per-
• Adolescents from 12 to 18 years of age with borderline symptoms, and adults with ages ranging from 18 to

• Health-related quality of life - EQ-5D difficulties in emotion regulation - DERS, and borderline symptoms list

formed, followed by aggregated benchmarks for different data group-

ings. We found different lengths for full-programme: some studies
• DBT Skills training only, with a minimum duration of 16 weeks to 24, using manualised treatment.

delivered the full-programme for 12 months, and some for 6 months. A

mean comparison was performed to search for significant differences
(with respect to the instruments we were interested in) for these two
programme lengths, and none were found in the outcomes of interest. A
recent McMain et al. (2022) study compared 6 and 12-month pro-
grammes and while differences were found in suicidality across time and
hospital admissions, no differences were observed in pre–and
• Within-group comparisons between baseline and post-treatment scores

post-outcomes regarding the variables of interest in our study. For that

• Focus on the intervention received and not the comparison group.

reason, we decided to group all the studies that delivered

• Dialectical Behaviour Therapy (full programme) 6 to 12 months

full-programme (6 M and 12 M), taking into consideration that the

6-month has the same content as the 12-month programme (the content
• Practitioners had to have been adequately trained in DBT.

of the programme is delivered across six months, and then it is repeated

in the following six to consolidate treatment gains). The studies were
then grouped according to the type of trial, the modes of DBT delivered,
PICO approach with inclusion and exclusion criteria applied.

the instrument they used and their age group.

Heterogeneity between studies was assessed with standard meta-
analysis statistical methods. The percentage of effect size variability
was evaluated using the I2 formula (values of 0 %, 25 %, 50 %, and 75 %
indicate no observed low, moderate, or large degrees of heterogeneity,
respectively). The Cochran’s Q-test was used to determine whether
statistical heterogeneity existed. To explore possible sources of hetero-
65 with BPD diagnosis.

geneity, subgroup analyses were performed according to the type of

Inclusion Criteria

treatment (full-programme vs skills programme), age (adolescent vs

adult) and trial (effectiveness vs RCT). To obtain pooled estimates and
Borderline Symptoms List.

95 % confidence intervals (CI) for the mean difference (MD) between

- BSL-23

post-treatment and baseline scores in each subgroup, a meta-analysis

was performed using Hedges’ g method (Hedges, 1981). Given that
significant heterogeneity was found, random-effects models (Higgins
I - Intervention
P - Population

Comparison
O - Outcomes

et al., 2009) were used with the restricted maximum-likelihood method

(DerSimonian & Laird, 1986). The inverse variance weighting approach
Table 1

PICO

was applied to assign weights to studies, giving less weight to smaller

C-

studies than larger ones. Forest plots were used to represent the pooled

4
J. Azevedo et al. International Journal of Clinical and Health Psychology 24 (2024) 100446

Fig. 1. Diagram flow of selected studies.

Table 2
Main features of selected studies with Dialectical Behavioural Therapy (DBT) intervention for adults and adolescents (N = 16).
Authors Study design DBT mode Age groups DBT Length (in Sample size at Gender Dropout ITT Outcome JBI
months) baseline % rate% measures appraisal
(F, M)

Walton et al. (2020) RCT Full- Adults 12 81 77, 23 35 Yes DERS 10/13
programme
Stiglmayr et al. Effectiveness Full- Adults 12 70 84, 16 40 Yes BSL 5/9
(2014) programme
Lyng et al. (2019) Effectiveness Full- Adults 12 37* 78, 22 21 No BSL 8/9
programme
Kuehn et al. (2020) RCT Full- Adults 12 66* 100, 0 20 Yes DERS 10/13
programme
Barnicot & Effectiveness Full- Adults 12 58 72, 28 47 Yes DERS 7/9
Crawford (2018) programme
Goodman et al. Effectiveness Full- Adults 12 11 82, 18 23 No DERS 6/9
(2014) programme
Kells et al. (2020) Effectiveness Full- Adults 6 100 71, 29 49 No DERS 5/9
programme
McMain et al. RCT Skills Adults 5 42 83, 17 31 Yes BSL, DERS 9/13
(2017)
McMain et al. RCT Full- Adults 12 90 90, 10 38 Yes EQ-5D 11/13
(2009) programme
McMain et al. RCT Full- Adults 12 240* 79, 21 30 Yes BSL, EQ-5D 10/13
(2022) programme
Heerebrand et al. Effectiveness Skills Adults 5 114 92, 8 27 No BSL 7/9
(2021)
Rizvi et al. (2017) Effectiveness Full- Adults 6 50 80, 20 32 Yes BSL, DERS 8/9
programme
Sinnaeve et al. RCT Full- Adults 12 42 95, 5 37 Yes EQ-5D 8/13
(2018) programme
Mehlum et al. RCT Full- Adolescents 5 39 87, 13 26 Yes BSL 11/13
(2014) programme
Gillespie et al. Effectiveness Full- Adolescents 6 152* 85, 15 22 No BSL 6/9
(2019) programme
Berk et al. (2018) Effectiveness Full- Adolescents 6 24 92, 8 8 Yes DERS 7/9
programme

Note. ITT = intention-to-treat; JBI = Joanna Briggs Institute.

*
Studies with two different samples (experimental + control) that received DBT intervention.

estimates visually within subgroups and across studies. The treatment–post-treatment data (means = M, and standard deviations =
meta-analyses of subgroups were conducted using the meta package SD) within the studies which used the same outcome instrument, to
(Balduzzi et al., 2019) in R statistical software version 4.1.0 (R Core calculate a single pre-treatment–post-treatment effect size estimate
Team, 2017). The statistical significance threshold was set at 0.05. (d+).
A two-step process was followed: first, we aggregated the pre- The second step involved aggregating each effect size estimate to

5
J. Azevedo et al. International Journal of Clinical and Health Psychology 24 (2024) 100446

obtain a single pre-treatment–post-treatment effect benchmark d + j for establish benchmarks, and those studies used either BSL (n = 2) or DERS
each outcome measure category (see formulas, Minami et al., 2008). (n = 1), with no studies using EQ-5D. From these studies, we decided to
Additionally, in order to establish that an effect size estimate obtained include both samples of adolescents (experimental and control group) of
from clinical settings is equivalent to efficacy benchmarks, previous Gillespie et al. (2019) because they differ only in the treatment length
literature suggested using a critical value which is dependent on the (16 vs 24 weeks), with no significant differences between groups in the
sample size of the clinical setting data. We adopted the minimum effect studies using the BSL. The four data samples retrieved from the three
size of dmin = 0.2 as the criterion for clinically significant differences studies with adolescents are presented in Table 6, aggregated per in-
between benchmarks and the treatment effect size estimates for a strument when possible. The aggregated effect sizes for BSL are pre-
range-null hypothesis test, (Minami et al., 2008) and we reported them sented in Table 7. We also calculated if from Berk et al. (2018) study,
only for the aggregated studies (RCT + Effectiveness). which used the DERS and revealed a large effect size (d = 1.095).
A total of 673 adult participants were included in these benchmark
calculations, 534 receiving DBT full-programme and 139 receiving Skills Discussion
group intervention. A total of 173 adolescents receiving full-programme
were also included. The interest in benchmarking psychological interventions is on the
rise. This seems to be driven by policy changes and requests from regular
Results service providers (Delgadillo et al., 2014; Moroz et al., 2020). As a
result, there is an increased emphasis on recognising the significance of
As the Q and I2 statistics for homogeneity indicated that effect size setting global standards for mental health systems. Providing bench-
estimates were heterogeneous, the reported benchmarks should not be marks for empirically supported treatments is essential to serve as a
considered an estimate of a single population parameter but rather the reference point for national implementation efforts. Currently, the main
mean of the effect sizes estimates (Shadish & Haddock, 1994). challenges when benchmarking any treatments are establishing com-
Results of the subgroup meta-analyses indicate that DBT significantly mon measures to benchmark against and the lack of consensus on what
improved both difficulties in emotion regulation and BPD symptoms and how to benchmark. This study therefore offers a proposal for com-
(see Figs. 2–4), for the full-programme (MD = - 34.62, k = 7, 95 % CI mon ground, providing benchmarks to assess standard DBT, based on
[−41.71, −27.53]; MD = −0.76, k = 9, 95 % CI [−1.02, −0.49], RCTs and effectiveness studies using our three selected instruments
respectively), versus skills modes (MD = −31.76, k = 2, 95 % CI [- 46.35, (EQ-5D-3 L, BSL and DERS). This will allow clinical services that are
−17.16]; MD = −0.81, k = 2, 95 % CI [−1.15, −0.48], respectively), and using these instruments to compare their performance against these
effectiveness studies (MD = −30.84, k = 5, 95 % CI [−38.37, −23.32]; standards. In mental health treatments, benchmarking can play an
MD = −0.77, k = 7, 95 % CI [−1.13, −0.41], respectively) vs RCTs (MD essential role in evaluating the effectiveness of the treatments being
= −37.69, k = 4, 95 % CI [−47.62, −27.76]; MD = −0.73, k = 4, 95 % CI offered to ensure outcomes are being compared to standards.
[−0.86, −0.60], respectively). When analysing adolescents and adults Even though DBT is now a widely used treatment for people with a
separately, subgroup analyses indicate that DBT was effective for both diagnosis of BPD, with substantial evidence of its efficacy (Gillespie
adolescents and adults, as measured by BSL (MD = −0.72, k = 3, 95 % CI et al., 2022; Stoffers-Winterling et al., 2022), when performing a broad
[−0.91, −0.54]; MD = - 0.78, k = 8, 95 % CI [−1.08, −0.47], respec- literature search through four widely used databases, it became clear
tively). For the DERS, only one study included adolescents, so it is only that a multiplicity of measures are in use with little common ground.
possible to pool the adults’ results, which were also significant (MD = After applying our inclusion criteria that aimed to find robust studies
−34.43, k = 8, 95 % CI [−40.96, −27.90]). For EQ-5D (USc and VAS), using our chosen measures (while delivering DBT full-programme or
subgroup analyses were not performed, given that all the studies found skills), we were only able to select 16 studies from the 664 studies
were RCTs, with adults, and delivered full-programme. The meta- identified. The small number of studies confirms the difficulties previ-
analyses showed that DBT full-programme applied to adults in an RCT ously mentioned by other authors, making it hard to compare their
design significantly improves both EQ-5D utility scores (EQ-5D US) and outcomes against the literature (Delgadillo et al., 2014).
perceived health (EQ-5D VAS; MD = −0.06, k = 4, 95 % CI [0.02, 0.09]; The retrieved studies served as the basis for aggregating outcomes for
MD = 7.31, k = 4, 95 % CI [3.52, 11.11], respectively). the BSL, DERS, and EQ-5D, enabling us to establish benchmarks for both
the full DBT programme and the skills programme for adults, as well as
Benchmarks for adults the full-programme for adolescents (using the BSL alone). Meta-analyses
were performed per instrument and per subgroup (by type of trial, age
The weighted pre and post-treatment (full-programme) M and SD of group and DBT mode). Results indicated improvements in emotion
the RCTs and effectiveness studies were aggregated per assessment regulation, a decrease in BPD symptoms, and increased health-related
measure (BSL, DERS and EQ-5D) and per type of study (RCTs and quality of life (EQ-5D). On the one hand, the significant heterogeneity
Effectiveness studies), as shown in Table 3. observed across most studies (except BSL in adolescents) emphasises the
The aggregated treatment efficacy benchmarks for DBT full-pro- need for caution when interpreting and generalising these calculated
gramme’s intervention are displayed in Table 4, grouped by instrument benchmarks. On the other hand, the subgroup analysis proves valuable
and type of trial, and for DBT skills intervention in Table 5 (only RCTs, as it reveals that DBT leads to significant improvements in the assessed
because there were no effectiveness studies), showing the overall ag- instruments across various contexts (consider the controlled and
gregation of the means and effect sizes of the studies, as well as the rigorous nature of RCTs vs effectiveness studies), modes (comprehensive
calculated critical value when possible. Additionally, in Figs. 5 and 6, it DBT vs skills only), and samples (adolescents vs adults). This implies
is possible to see the critical values and effect sizes by trial type for BSL that the studies used to derive these benchmarks are robust sources of
and DERS, according to the sample size estimations. For studies data. They provide substantial support for the efficacy of DBT in-
reporting skills-only intervention, only the BSL and DERS studies pro- terventions and validate the sensitivity of the chosen instruments to
duced benchmarks (in adults) because there were no studies using the changes within this population and treatment context.
EQ-5D. There were no significant mean differences either in pre- or post- Until this point, researchers and clinicians applying DBT interven-
outcomes between the effectiveness studies and the RCTs. tion with our selected instruments were limited to comparing their
findings with existing studies in isolation, without a clear understanding
Benchmarks for adolescents of how representative these studies were of the overall empirical data.
Our article provides aggregated metrics, allowing for an examination of
Only three studies with adolescents were possible to include to the means, standard deviations and effect sizes of RCTs and effectiveness

6
J. Azevedo et al. International Journal of Clinical and Health Psychology 24 (2024) 100446

Fig. 2. Forest-plots of the subgroup meta-analyses with DERS assessment tool.

7
J. Azevedo et al. International Journal of Clinical and Health Psychology 24 (2024) 100446

Fig. 3. Forest-plots of the subgroup meta-analyses with BSL assessment tool.

8
J. Azevedo et al. International Journal of Clinical and Health Psychology 24 (2024) 100446

Fig. 4. Forest-plots of the meta-analyses with EQ-5D-3 L assessment tool Utility Score (USc) and Visual Analogue Scale (VAS).

Table 3
Pre- and post-treatment outcomes for adult samples that received DBT treatment in the selected studies (full-programme).
DBT details n Pre-treatment M ± SD Pre-treatment interval Post-treatment M ± SD Post-treatment interval

Borderline Symptoms List (BSL)

Aggregated RCTs 6–12 months 240 2.25 ± 0.75 [1.50 - 3.00] 1.58 ± 0.98 [0.6 - 2.56]
Aggregated Effectiveness 6–12 months 67 2.11 ± 0.82 [1.28 – 2.93] 1.46 ± 0.93 [0.53 - 2.39]
Difficulties in Emotion Regulation Scale (DERS)
Aggregated RCTs 6–12 months 114 3.62 ± 0.61 [3.01–4.23] 2.42 ± 0.75 [1.67 - 3.17]
Aggregated Effectiveness 6–12 months 233 3.48 ± 0.67 [2.81–4.15] 2.61 ± 0.76 [1.89 - 3.41]
EQ-5D (Utility Scores and VAS)
Aggregated RCTs USc 6–12 months 271 0.65 ± 0.21 [0.44 - 0.86] 0.71 ± 0.20 [0.51 - 0.91]
Aggregated RCTs VAS 6–12 months 271 54.26 ± 33.33 [20.93–87.58] 61.42 ± 20.07 [41.35–81.49]

Note. M = Mean; SD = Standard deviation; DBT = Dialectical Behaviour Therapy; RCT = Randomised Controlled trial; VAS - Visual Analogue Scale.

Table 4 Table 5
Aggregated Benchmarks – DBT treatment efficacy benchmarks for adults (full- Aggregated benchmarks – DBT treatment efficacy benchmarks for adults (DBT
programme). skills).
Measure K N d+ σ2 CV Q p (Q) Measure K N d+ σ2 CV Q Q (p)

Aggregated treatment efficacy - benchmarks for RCT studies BSL-23 2 106 0.896 0.013 0.62 4.17 .041
BSL-23 2 240 0.70 0.006 – 0.02 .901 DERS 2 75 1.489 0.020 88.74 4.27 <0.05
DERS 2 114 1.25 0.013 – 1.31 .253
EQ-5D-USc 4 271 0.28 0.004 – 0.92 .819 Note. DBT = Dialectical Behaviour Therapy; K = number of samples included in
EQ-5D-VAS 4 271 0.30 0.004 – 0.16 .984 analyses; N = sample size; d = unbiased pre–post effect size estimate; σ2 = effect
Aggregated treatment efficacy - benchmarks for effectiveness studies size variance; CV = Critical Value; Q = test of homogeneity; p = significance;
BSL-23 4 117 0.882 0.011 – 33.10 < 0.001 BSL-23 = Borderline Symptom List; DERS = Difficulties in Emotion Regulation
DERS 4 233 1.247 0.008 – 9.51 <0.050 Scale.
Treatment efficacy benchmarks for RCTs + Effectiveness
BSL-23 6 357 0.824 0.004 0.67 23.39
SDpost-treatment= 0.2). To inform average means and standard deviations
<0.001
DERS 6 347 1.423 0.006 81.10 24.86 <0.001
EQ-5D-USc 4 271 0.28 0.004 – 0.92 .819 per type of trial at pre-treatment and post-treatment, consult Table 3.
EQ-5D-VAS 4 271 0.30 0.004 – 0.16 .984 Additionally, the following benchmarks for DBT skills for adults in
Note. DBT = Dialectical Behaviour Therapy; K = number of samples included in outpatient settings should be considered: BSL (d+ = 0.896; CV = 0.62);
analyses; N = sample size; d = unbiased pre–post effect size estimate; σ 2 = effect DERS (d+ = 1.489; CV = 88.74).
size variance; CV = Critical Value; Q = test of homogeneity; p = significance; Fewer studies were retrieved with adolescents, allowing only for
BSL-23 = Borderline Symptom List; DERS = Difficulties in Emotion Regulation aggregated benchmarks for standard DBT (full-programme) for BSL (d+
Scale; EQ-5D-USc: EQ-5D Utility Score; EQ-5D-VAS: EQ-5D Visual Analogue = 0.800; CV = 0.48).
Scale. It is important to take into consideration that, unlike the DERS and
BSL, which are measures with a normal distribution, in the case of EQ-
studies.for adults receiving DBT full-programme and skills alone, in an 5D utility score, mean scores and effect sizes need to be considered
outpatient setting. Thus, the following benchmarks for full-programme with caution, and a usable critical value was not possible to retrieve
should be considered regarding BSL (d+ = 0.824; CV = 0.67); DERS using the calculation proposed by Minami et al. (2008).
(d+ = 1.423; CV = 81.10) and EQ-5D (d+ = 1.423; Mpost-treatment = 0.71;

9
J. Azevedo et al. International Journal of Clinical and Health Psychology 24 (2024) 100446

treatment is offered under tightly controlled conditions. Even so, clini-

cians should not be dissuaded from striving to attain results similar to
those achieved in RCTs. Clinicians can consider these benchmarks as
guidance, considering that the demographic and diagnostic mix (for
example, baseline severity, socioeconomic status, and comorbidity) can
vary widely across services.
To develop more representative and generalisable benchmarks in the
future, establishing a further body of research using common in-
struments is a necessity, and we encourage researchers and DBT teams to
use the instruments suggested in this article. In treatments of other
mental health disorders, there has already been an effort to create a
unified protocol to assess the effects of given treatments (Allen et al.,
2008; Farchione et al., 2012). We believe this article makes a valuable
contribution by supporting the systematic use of these instruments in the
future. This, in turn, could facilitate a more consistent and standardised
Fig. 5. BSL effect size critical values by study type and data sample size. Note. way of assessing DBT treatments, and their effectiveness.
RCT = Randomised Controlled Trial; ES = Effect Size.

Limitations and future directions

The benchmarks established in this article offer several advantages
when reflecting on DBT delivery for, and treatment of, people with BPD.
In the future, we hope to be able to provide peer benchmarks in the
Firstly, benchmarks provide a standardised framework for assessing and
context of an ongoing project to benchmark teams delivering DBT across
evaluating progress throughout the therapy process and help set clear
and measurable therapeutic aims. Secondly, establishing benchmarks
Table 7
enables regular assessment and monitoring of client progress. By
Aggregated benchmarks – treatment efficacy benchmarks for adolescents (full-
comparing outcomes against predetermined targets, therapists can more
programme).
objectively evaluate the effectiveness of their interventions, adjusting
the treatment delivery and checking their adherence to the model if Measure K N d+ σ2 CV Q p (Q)

necessary, to deliver the best possible treatment. BSL-23 3 151 0.800 0.009 0.48 1.705 .426
Results from effectiveness studies potentially align more closely with Note. K = number of studies; N = sample size; d+ = unbiased pre–post effect size
the clients that clinicians may encounter in routine practice, whereas estimate; σ 2 = effect size variance; CV = Critical Value; Q = test of homogeneity;
RCTs ideally provide benchmarks for when the highest quality of p = significance; BSL-23 = Borderline Symptom List;.

Fig. 6. DERS effect size critical values by study type and data sample size. Note. RCT = Randomised Controlled Trial; ES = Effect Size.

Table 6
Pre- and post-treatment outcomes of adolescent samples that received DBT full-programme treatment in the selected studies.
DBT details n Pre-treatment Pre-treatment interval Post-treatment M ± SD Post-treatment interval
M ± SD

Borderline Symptoms List (BSL)

Aggregated RCT+ Effectiveness 16–24 weeks 151 2.18 ± 0.87 [1.31 – 3.05] 1.47 ± 0.79 [0.68 – 2.26]
Difficulties in Emotion Regulation Scale (DERS)
Berk et al., 2018 24 weeks 24 3.33 ± 0.61 [2.73 – 3.94] 2.56± 0.80 [1.75 – 3.36]

Note. M = Mean, SD = Standard-deviation; DBT = Dialectical Behaviour Therapy; n = sample size;.

10
J. Azevedo et al. International Journal of Clinical and Health Psychology 24 (2024) 100446

the UK and eventually be able to shed light on the similarities and dif- A smaller number of adolescent studies with higher heterogeneity
ferences between routine practice and the provided benchmarks. limited the development of definitive benchmarks for all the in-
Future RCTs and effectiveness studies should also seek to use an ITT struments, although teams can still use the findings for comparison.
approach and a standardised protocol to contribute to a common ground Subsequent research will focus on using these benchmarks in routine
for researchers and clinicians. This should include the utilisation of practice to support the development of a peer network focused on the
consistent assessment tools on a global scale. Such an approach greatly improvement of clinical outcomes and the development of peer
simplifies the subsequent analysis and comparative studies and is a benchmarks.
significant step towards benchmarking within the field of mental health.
Furthermore, looking at Devlin et al. (2020) chapter explaining Funding
different approaches to assess clinically significant changes using
EQ-5D, we believe the use of an anchor measure for BPD, which could This work was supported by the School of Psychology and Sport
account for a relevant improvement in this population, could inform Science of Bangor University, British Isles DBT-Training (biDBT) and
what would be a significant change. Future studies should investigate NHS England (formerly Health Education England (HEE).
the use of BSL-23 as a possible anchor measure along with EQ-5D, to
detect what could be consider a Minimum Important Difference (MID).
Pickard et al. (2007) suggested using half of a standard deviation to Declaration of competing interest
calculate MID when using EQ-5D in a sample of cancer patients while
selecting an anchor measure, and its methodology has been used in other The author have no conflict of interest to disclose.
clinical samples.
In terms of limitations, the studies we selected showed high het- Acknowledgments
erogeneity, which stresses the need to be conservative in our conclu-
sions. Moreover, we aggregated results from studies using different We thank the School of Psychology and Sport Science, Bangor Uni-
methodologies (ITT and completers) in order to establish more repre- versity, British Isles DBT-Training (biDBT) and NHS England (formerly
sentative benchmarks, however, this comes at the cost of some level of Health Education England (HEE) for funding and supporting this proj-
accuracy that could have been attained with a more homogeneous ect. We also thank the collaboration of Roland Sinaeve and Shelley
sample. Participants who did not complete a given study may be sys- McMain, who provided data from their studies, making a noteworthy
tematically different from those who did, their exclusion can alter contribution to the calculated benchmarks.
treatment effects, reducing the generalisability of the study’s findings.
Benchmarks for adolescents were not possible to establish for the EQ- References
5D or DERS, because no studies were found using EQ-5D and only one
study used DERS. The last can be used as a reference but not as a Allen, L. B., McHugh, R. K., Barlow, D. H., & Barlow, D. H. (2008). Emotional disorders:
benchmark. In addition, we aggregated data from studies which inves- A unified protocol. Clinical handbook of psychological disorders: A step-by-step
treatment manual (4th ed., pp. 216–249). The Guilford Press.
tigated programmes of different lengths, essentially treating them as American Psychiatric Association. (2022). Diagnostic and statistical manual of mental
equal in terms of outcomes. Whilst in terms of clinical outcomes at the disorders (5th ed., text rev.).
individual level this may be the case, it has a significant limitation. A Balduzzi, S., Rücker, G., & Schwarzer, G. (2019). How to perform a meta-analysis with R:
A practical tutorial. Evidence-Based Mental Health, 22(4), 153–160. https://doi.org/
programme that produces the same effect size in 6 months as a pro- 10.1136/EBMENTAL-2019-300117
gramme that is twice the length is twice as productive in health eco- Barnicot, K., & Crawford, M. (2018). Psychological Medicine Dialectical behaviour therapy
nomic terms provided other things remain equal (team size and v. mentalisation-based therapy for borderline personality disorder. doi:10.101
7/S0033291718002878.
training). As DBT is a team-based treatment arguably treating clinical
Bayney, R. (2005). Benchmarking in mental health: An introduction for psychiatrists.
outcomes at the individual level may not be the best approach, although Advances in Psychiatric Treatment, 11(4), 305–314. https://doi.org/10.1192/
it is the approach that clinicians are most familiar with. Few of the apt.11.4.305
Berk, M. S., Starace, N. K., Black, V. P., & Avina, C. (2018). Implementation of dialectical
studies that we reviewed and included in this paper systematically re-
behavior therapy with suicidal and self-harming adolescents in a community clinic.
ported on ’team’ as a variable or reported on any potential clustering of Archives of Suicide Research. https://doi.org/10.1080/13811118.2018.1509750
outcomes by site or in changes in outcome during the study due to Biskin, R. S. (2015). The lifetime course of borderline personality disorder. Canadian
learning effects in therapists, making it impossible for us to develop Journal of Psychiatry, 60(7), 303–308. https://doi.org/10.1177/
070674371506000702
benchmarks that incorporated a ’team’ factor. As we develop our peer- Bohus, M., Kleindienst, N., Limberger, M. F., Stieglitz, R. D., Domsalla, M.,
benchmarking platform we aim to address these important aspects of Chapman, A. L., et al. (2009). The short version of the Borderline Symptom List (BSL-
outcome in routine settings, in consultation with the teams using the 23): Development and initial data on psychometric properties. Psychopathology, 42
(1), 32–39. https://doi.org/10.1159/000173701
platform, by systematically developing team ’productivity’ measures, Buchholz, I., Janssen, M. F., Kohlmann, T., & Feng, Y. S. (2018). A systematic review of
that incorporate treatment length, team resource and skill, to bench- studies comparing the measurement properties of the three-level and five-level
mark against. versions of the EQ-5D. PharmacoEconomics, 36(6), 645–661. https://doi.org/
10.1007/s40273-018-0642-5
Carter, G.L., Willcox, C.H., Lewin, T.J., Conrad, A.M., & Bendit, N. (2010). Hunter DBT
Conclusion project: Randomized controlled trial of dialectical behaviour therapy in women with
borderline personality disorder. 10.3109/00048670903393621, 44(2), 162–173.
doi:10.3109/00048670903393621.
To assist teams delivering DBT in routine practice and to evaluate the Chakhssi, F., Zoet, J. M., Oostendorp, J. M., Noordzij, M. L., & Sommers-Spijkerman, M.
impact of a national training programme on clinical outcomes, we (2021). Effect of psychotherapy for borderline personality disorder on quality of life:
searched the literature to compile benchmarks for three measures of A systematic review and meta-analysis. Journal of Personality Disorders, 35(2),
255–269. https://doi.org/10.1521/PEDI_2019_33_439
outcome: the EQ-5D measuring health-related quality of life; the BSL, Chorpita, B. F., & Daleiden, E. L. (2014). Structuring the collaboration of science and
measuring borderline symptoms; and the DERS measuring difficulties in service in pursuit of a shared vision. Journal of Clinical Child and Adolescent
emotion regulation. We were able to compile aggregated benchmarks Psychology: The Official Journal for the Society of Clinical Child and Adolescent
Psychology, American Psychological Association, Division, 53(2), 323–338. https://doi.
for teams working with adults delivering both comprehensive DBT (BSL
org/10.1080/15374416.2013.828297. 43.
d+ = 0.824; CV = 0.67; DERS d+ = 1.423; CV = 81.10; EQ-5D d+ = D’Aurizio, G., Di Stefano, R., Socci, V., Rossi, A., Barlattani, T., Pacitti, F., et al. (2023).
1.423; M = 0.71±0.2) and stand-alone skills training (BSL d+ = 0.896; The role of emotional instability in borderline personality disorder: A systematic
CV = 0.62; DERS d+ = 1.489; CV = 88.74). Additionally, for teams review. Annals of General Psychiatry, 22(1). https://doi.org/10.1186/S12991-023-
00439-0
delivering DBT full-programme to adolescents with BPD features, Delgadillo, J., McMillan, D., Leach, C., Lucock, M., Gilbody, S., & Wood, N. (2014).
benchmarks were provided for BSL (d+ = 0.800; CV = 0.48). Benchmarking routine psychological services: A discussion of challenges and

11
J. Azevedo et al. International Journal of Clinical and Health Psychology 24 (2024) 100446

methods. Behavioural and Cognitive Psychotherapy, 42(1), 16–30. https://doi.org/ Lyng, J., Swales, M. A., Hastings, R. P., Millar, T., & Duffy, D. J. (2020). Outcomes for 18
10.1017/S135246581200080X to 25-year-olds with borderline personality disorder in a dedicated young adult only
DerSimonian, R., & Laird, N. (1986). Meta-analysis in clinical trials. Controlled Clinical DBT programme compared to a general adult DBT programme for all ages 18+. Early
Trials, 7(3), 177–188. https://doi.org/10.1016/0197-2456(86)90046-2 Intervention in Psychiatry, 14(1), 61–68. https://doi.org/10.1111/eip.12808
Devlin, N., Parkin, D., & Janssen, B. (2020). Advanced topics. Methods for analysing and McMain, S. F., Chapman, A. L., Kuo, J. R., Dixon-Gordon, K. L., Guimond, T. H.,
reporting eq-5d data (pp. 87–98). Springer International Publishing. https://doi.org/ Labrish, C., et al. (2022). The effectiveness of 6 versus 12 months of dialectical
10.1007/978-3-030-47622-9_5 behavior therapy for borderline personality disorder: A noninferiority randomized
Eisen, S. V., & Dickey, B. (1996). Mental health outcome assessment: The new agenda. clinical trial. Psychotherapy and Psychosomatics, 91(6), 382–397. https://doi.org/
Psychotherapy, 33(2), 181–189. https://doi.org/10.1037/0033-3204.33.2.181 10.1159/000525102
Eldh, A. C., Almost, J., Decorby-Watson, K., Gifford, W., Harvey, G., Hasson, H., et al. McMain, S. F., Guimond, T., Barnhart, R., Habinski, L., & Streiner, D. L. (2017).
(2017). Clinical interventions, implementation interventions, and the potential A randomized trial of brief dialectical behaviour therapy skills training in suicidal
greyness in between-a discussion paper. BMC Health Services Research, 7(17), 16. patients suffering from borderline disorder. Acta Psychiatrica Scandinavica, 135(2),
https://doi.org/10.1186/s12913-016-1958-5 138–148. https://doi.org/10.1111/ACPS.12664
EuroQol Research Foundation. (2018). EQ-5D-3L user guide. https://euroqol.org/publi McMain, S. F., Links, P. S., Gnam, W. H., Guimond, T., Cardish, R. J., Korman, L., et al.
cations/user-guides. (2009). A randomized trial of dialectical behavior therapy versus general psychiatric
Farchione, T. J., Fairholme, C. P., Ellard, K. K., Boisseau, C. L., Thompson-Hollands, J., management for borderline personality disorder. The American Journal of Psychiatry,
Carl, J. R., et al. (2012). Unified protocol for transdiagnostic treatment of emotional 166(12), 1365–1374. https://doi.org/10.1176/APPI.AJP.2009.09010039
disorders: A randomized controlled trial. Behavior Therapy, 43(3), 666–678. https:// Mehlum, L., Tørmoen, A. J., Ramberg, M., Haga, E., Diep, L. M., Laberg, S., et al. (2014).
doi.org/10.1016/J.BETH.2012.01.001 Dialectical behavior therapy for adolescents with repeated suicidal and self-harming
Gillespie, C., Joyce, M., Flynn, D., & Corcoran, P. (2019). Dialectical behaviour therapy behavior: A randomized trial. Journal of the American Academy of Child and
for adolescents: A comparison of 16-week and 24-week programmes delivered in a Adolescent Psychiatry, 53(10), 1082–1091. https://doi.org/10.1016/J.
public community setting. Child and Adolescent Mental Health, 24(3), 266–273. JAAC.2014.07.003
https://doi.org/10.1111/CAMH.12325 Minami, T., Serlin, R. C., Wampold, B. E., Kircher, J. C., & Brown, G. S. (2008). Using
Gillespie, C., Murphy, M., & Joyce, M. (2022). Dialectical behavior therapy for clinical trials to benchmark effects produced in clinical practice. Quality and
individuals with borderline personality disorder: A systematic review of outcomes Quantity, 42(4), 513–525. https://doi.org/10.1007/S11135-006-9057-Z/METRICS
after one year of follow-up. Journal of Personality Disorders, 36(4), 431–454. https:// Minami, T., Wampold, B. E., Serlin, R. C., Kircher, J. C., & Brown, G. S. (2007).
doi.org/10.1521/pedi.2022.36.4.431 Benchmarks for psychotherapy efficacy in adult major depression. Journal of
Glenn, C. R., & Klonsky, E. D. (2009). Emotion dysregulation as a core feature of Consulting and Clinical Psychology, 75(2), 232–243. https://doi.org/10.1037/0022-
borderline personality disorder. Journal of Personality Disorders, 23(1), 20–28. 006X.75.2.232
https://doi.org/10.1521/PEDI.2009.23.1.20 Mirkovic, B., Delvenne, V., Robin, M., Pham-Scottez, A., Corcos, M., & Speranza, M.
Gotham, H. J. (2006). Advancing the implementation of evidence-based practices into (2021). Borderline personality disorder and adolescent suicide attempt: The
clinical practice: How do we get there from here?. In Professional psychology: mediating role of emotional dysregulation. BMC psychiatry, 21(1), 1–10. https://doi.
Research and practice, 37 pp. 606–613) American Psychological Association. https:// org/10.1186/S12888-021-03377-X/FIGURES/2
doi.org/10.1037/0735-7028.37.6.606 Moroz, N., Moroz, I., & D’Angelo, M. S (2020). Mental health services in Canada: Barriers
Gratz, K. L., & Roemer, L. (2004). Multidimensional assessment of emotion regulation and cost-effective solutions to increase access. Healthcare Management Forum, 33(6),
and dysregulation: Development, factor structure, and initial validation of the 282–287. https://doi.org/10.1177/0840470420933911
difficulties in emotion regulation scale. Journal of Psychopathology and Behavioral Neumann, A., van Lier, P. A. C., Gratz, K. L., & Koot, H. M. (2010). Multidimensional
Assessment, 26(1), 41–54. https://doi.org/10.1023/B:JOBA.0000007455.08539.94 assessment of emotion regulation difficulties in adolescents using the difficulties in
Hedges, L. V. (1981). Distribution theory for glass’s estimator of effect size and related emotion regulation scale. Assessment, 17(1), 138–149. https://doi.org/10.1177/
estimators. Journal of Educational Statistics, 6(2), 107. https://doi.org/10.2307/ 1073191109349579
1164588 Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016). Rayyan-a web and
Heerebrand, S. L., Bray, J., Ulbrich, C., Roberts, R. M., Edwards, S., Services, D. N., et al. mobile app for systematic reviews. Systematic Reviews, 5(1), 1–10. https://doi.org/
(2021). Effectiveness of dialectical behavior therapy skills training group for adults 10.1186/s13643-016-0384-4
with borderline personality disorder. Journal of Clinical Psychology. https://doi.org/ Pickard, A. S., De Leon, M. C., Kohlmann, T., Cella, D., & Rosenbloom, S. (2007).
10.1002/jclp.23134 Psychometric comparison of the standard EQ-5D to a 5 level version in cancer
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009). A re-evaluation of patients. Medical Care, 45(3), 259–263. https://doi.org/10.1097/01.
random-effects meta-analysis. Journal of the Royal Statistical Society. Series A, MLR.0000254515.63841.81
(Statistics in Society), 172(1), 137–159. https://doi.org/10.1111/J.1467- R. Core Team. (2017). A language and environment for statistical computing.
985X.2008.00552.X Rizvi, S. L., Hughes, C. D., Hittman, A. D., & Oliveira, P. V. (2017). Can trainees
Hunsley, J., & Lee, C. M. (2007). Research-informed benchmarks for psychological effectively deliver dialectical behavior therapy for individuals with borderline
treatments: Efficacy studies, effectiveness studies, and beyond. Professional personality disorder? Outcomes from a training clinic. Journal of Clinical Psychology,
Psychology: Research and Practice, 38(1), 21–33. https://doi.org/10.1037/0735- 73, 1599–1611. https://doi.org/10.1002/jclp.22467
7028.38.1.21 Shadish, W. R., & Haddock, C. K. (1994). The handbook of research synthesis. Russell Sage
IsHak, W. W., Elbau, I., Ismail, A., Delaloye, S., Ha, K., Bolotaulo, N. I., et al. (2013). Foundation.
Quality of life in borderline personality disorder. Harvard Review of Psychiatry, 21(3), Sinnaeve, R., Van Den Bosch, L. M. C., Hakkaart-Van Roijen, L., & Vansteelandt, K
138–150. https://doi.org/10.1097/HRP.0B013E3182937116 (2018). Effectiveness of step-down versus outpatient dialectical behaviour therapy
Joanna Briggs Institute. (2017). The joanna briggs institute reviewers’ manual 2017. The for patients with severe levels of borderline personality disorder: A pragmatic
Joanna Briggs Institute. randomized controlled trial. Borderline Personality Disorder and Emotion
Kaufman, E. A., Xia, M., Fosco, G., Yaptangco, M., Skidmore, C. R., & Crowell, S. E. Dysregulation, 5(12). https://doi.org/10.1186/s40479-018-0089-5
(2016). The Difficulties in Emotion Regulation Scale Short Form (DERS-SF): Sloan, E., Hall, K., Moulding, R., Bryce, S., Mildred, H., & Staiger, P. K. (2017). Emotion
Validation and replication in adolescent and adult samples. Journal of regulation as a transdiagnostic treatment construct across anxiety, depression,
Psychopathology and Behavioral Assessment, 38(3), 443–455. https://doi.org/ substance, eating and borderline personality disorders: A systematic review. Clinical
10.1007/S10862-015-9529-3 Psychology Review, 57, 141–163. https://doi.org/10.1016/J.CPR.2017.09.002
Kells, M., Joyce, M., Flynn, D., Spillane, A., & Hayes, A. (2020). Borderline personality Stiglmayr, C., Stecher-Mohr, J., Meiβner, J., Spretz, D., Steffens, C., Roepke, S., et al.
disorder and emotion dysregulation. 7(3). doi:10.1186/s40479-020-0119-y. (2014). Effectiveness of dialectic behavioral therapy in routine outpatient care: The
Kleindienst, N., Jungkunz, M., & Bohus, M. (2020). A proposed severity classification of Berlin borderline study. Borderline Personality Disorder and Emotion Dysregulation, 1
borderline symptoms using the borderline symptom list (BSL-23). Borderline (20). https://doi.org/10.1186/2051-6673-1-20
Personality Disorder and Emotion Dysregulation, 7(1), 1–11. https://doi.org/10.1186/ Stoffers-Winterling, J. M., Storebø, O. J., Kongerslev, M. T., Faltinsen, E., Todorovac, A.,
S40479-020-00126-6/FIGURES/2 Jørgensen, S., et al. (2022). Psychotherapies for borderline personality disorder: A
Kuehn, K. S., King, K. M., Linehan, M. M., & Harned, M. S. (2020). Modeling the suicidal focused systematic review and meta-analysis. The British Journal of Psychiatry, 221,
behavior cycle: Understanding repeated suicide attempts among individuals with 538–552. https://doi.org/10.1192/bjp.2021.204
borderline personality disorder and a history of attempting suicide HHS Public Stoffers-Winterling, J. M., Völlm, B. A., Rücker, G., Timmer, A., Huband, N., & Lieb, K.
Access. Journal of Consulting and Clinical Psychology, 88(6), 570–581. https://doi. (2012). Psychological therapies for people with borderline personality disorder. The
org/10.1037/ccp0000496 Cochrane Database of Systematic Reviews, 2012(8). https://doi.org/10.1002/
Lilienfeld, S. O., Ritschel, L. A., Lynn, S. J., Cautin, R. L., & Latzman, R. D. (2013). Why 14651858.CD005652.PUB2. CD005652.
many clinical psychologists are resistant to evidence-based practice: Root causes and Swales, M. A., & Heard, H. L. (2017). Dialectical behaviour therapy - The CBT distinctive
constructive remedies. Clinical Psychology Review, 33(7), 883–900. https://doi.org/ features (2nd ed.). Routledge.
10.1016/J.CPR.2012.09.008 Teachman, B. A., Drabick, D. A. G., Hershenberg, R., Vivian, D., Wolfe, B. E., &
Linehan, M. (1993). Cognitive-behavioral treatment of borderline personality disorder. Goldfried, M. R. (2012). Bridging the gap between clinical research and clinical
Guilford Press. practice: Introduction to the special section. Psychotherapy, 49(2), 97–100. https://
Linehan, M. M. (2015). DBT® skills training manual (2nd ed.). Guilford Press. doi.org/10.1037/A0027346
Lloyd, R. C. (2004). Quality health care: A guide to developing and using indicators (1st ed.). Valentine, S. E., Bankoff, S. M., Poulin, R. M., Reidler, E. B., & Pantalone, D. W. (2015).
Jones and Bartlett Publishers. The use of dialectical behavior therapy skills training as stand-alone treatment: A
Lovaglio, P. G. (2012). Benchmarking strategies for measuring the quality of healthcare: systematic review of the treatment outcome literature. Journal of Clinical Psychology,
Problems and prospects. TheScientificWorldJournal, 2012, Article 606154. https:// 71(1), 1–20. https://doi.org/10.1002/JCLP.22114
doi.org/10.1100/2012/606154

12
J. Azevedo et al. International Journal of Clinical and Health Psychology 24 (2024) 100446

van Asselt, A. D. I., Dirksen, C. D., Arntz, A., Giesen-Bloo, J. H., & Severens, J. L. (2009). Washburn, M., Rubin, A., & Zhou, S. (2018). Benchmarks for outpatient dialectical
The EQ-5D: A useful quality of life measure in borderline personality disorder? behavioral therapy in adults with borderline personality disorder. Research on Social
European Psychiatry : The Journal of the Association of European Psychiatrists, 24(2), Work Practice, 28(8), 895–906. https://doi.org/10.1177/1049731516659363
79–85. https://doi.org/10.1016/J.EURPSY.2008.11.001 Weersing, V. R., & Weisz, J. R. (2002). Community clinic treatment of depressed youth:
Videler, A. C., Hutsebaut, J., Schulkens, J. E. M., Sobczak, S., & van Alphen, S. P. J. Benchmarking usual care against CBT clinical trials. Journal of Consulting and Clinical
(2019). A life span perspective on borderline personality disorder. Current Psychiatry Psychology, 70(2), 299–310. https://doi.org/10.1037/0022-006X.70.2.299
Reports, 21(7), 51. https://doi.org/10.1007/s11920-019-1040-1 Weinberg, A., & Klonsky, E. D. (2009). Measurement of emotion dysregulation in
Walton, C. J., Bendit, N., Baker, A. L., Carter, G. L., & Lewin, T. J. (2020). A randomised adolescents. Psychological Assessment, 21(4), 616–621. https://doi.org/10.1037/
trial of dialectical behaviour therapy and the conversational model for the treatment A0016669
of borderline personality disorder with recent suicidal and/or non-suicidal self- Zanarini, M. C., Horwood, J., Wolke, D., Waylen, A., Fitzmaurice, G., & Grant, B. F.
injury: An effectiveness study in an Australian public mental health servic. Australian (2011). Prevalence of DSM-IV borderline personality disorder in two community
and New Zealand Journal of Psychiatry, 54(10), 1020–1034. https://doi.org/10.1177/ samples: 6330 english 11-year-olds and 34,653 American adults. Journal of
0004867420931164 Personality Disorders, 25(5), 607–619. https://doi.org/10.1521/pedi.2011.25.5.607

13