Archives of Suicide Research, 21:113–126, 2017

Copyright # International Academy for Suicide Research

ISSN: 1381-1118 print=1543-6136 online
DOI: 10.1080/13811118.2016.1162242

Suicide Ideation Is Related

to Therapeutic Alliance in a
Brief Therapy for
Attempted Suicide
Anja C. Gysin-Maillart, Leila M. Soravia, Armin Gemperli, and
Konrad Michel

The objective of this study was to investigate the role of therapeutic alliance on sui-
cide ideation as outcome measure in a brief therapy for patients who attempted
suicide. Sixty patients received the 3-session therapy supplemented by follow-up
contact through regular letters. Therapeutic alliance was measured with the
Helping Alliance Questionnaire (HAQ). Outcome at 6 and 12 months was mea-
sured with the Beck Scale for Suicide Ideation (BSS). Therapeutic alliance
increased from session 1 to session 3. Higher alliance measures correlated with
lower suicidal ideation at 12 months follow-up. A history of previous attempts
and depression had a negative affect on therapeutic alliance. The results suggest
that in the treatment of suicidal patients therapeutic alliance may be a moderating
factor for reducing suicide ideation.

Keywords attempted suicide, suicidal ideation, therapeutic alliance, treatment engagement

Individuals who attempt suicide have a during the first year after the index episode
high rate of repeated attempts (Owens, (Heyerdahl et al., 2009). In view of the
Horrocks, & House, 2002; Sakinofsky, long-term risk of completed suicide in
2000). The repetition rates are highest in individuals with a history of attempted
the first months after the index attempt suicide (Carroll et al., 2014; Cooper et al.,
(Cedereke & Ojehagen, 2005; Monti, 2005; Jenkins, Hale, Papanastassiou,
Cedereke, & Ojehagen, 2003). In a recent Crawford, & Tyrer, 2002) effective inter-
meta-analysis (Carroll, Metcalfe, & ventions for people who have attempted
Gunnell, 2014) the estimated 1-year rate suicide have been given high priority
of repeated self-harm was 16.3%. (National Institute of Mental Health and
Patient-reported rates were higher (21.9%) the Research Prioritization Task Force,
than rates based on hospital records 2014). However, up to 50% of patients
(13.7%). Suicide attempts by overdosing do not attend follow-up treatment or drop
have a higher rate of reattempts compared out of treatment within 1 week (Granbou-
to other methods used. In a study from lan, Roudot-Thoraval, Lemerle, & Alvin,
Oslo, nearly 30% of patients hospitalized 2001; King, Hovey, & Brand, 1997; Monti
for poisoning repeated the poisoning et al., 2003). Self-harm patients who do

113
 Suicide Ideation and Therapeutic Alliance

not receive follow-up treatment have an hopelessness, and loss of self-esteem

increased risk of repeated self-harm (Crow- (Maltsberger, 2004; Shneidman, 1993).
der, Van der Putt, Ashby, & Blewett, 2004). Suicidality is a highly individual phenom-
So far, few treatment studies have been enon, with a strong biographical back-
able to demonstrate a significant reduction ground, and several authors have argued
of repeated suicide attempts (Arensman that, in order to increase therapeutic alli-
et al., 2001; Brown & Jager-Hyman, ance, treatment approaches should be
2014). Exceptions are the studies by Brown patient-centered and collaborative (Lizardi
et al. (2005), and, more recent, Rudd et al. & Stanley, 2010; Michel, Dey, Stadler, &
(2015). However, it appears to be difficult Valach, 2004; Michel et al., 2002; Rogers
to keep patients in treatment. Brown and & Soyka, 2004).
colleagues, in up to 24 sessions of cognitive Therapeutic alliance is a consistent
therapy supplemented with various clinical predictor of outcome in psychotherapy
elements introduced case managers to (Arnow & Steidtmann, 2014; Horvath,
ensure treatment adherence of ‘‘no-show’’ Del Re, Flückiger, & Symonds, 2011)
patients and medication compliance. and is therefore a key element of treat-
Rudd and co-workers in their cognitive- ment engagement. It has been character-
behavioral therapy of 12 individual sessions ized as ‘‘ . . . the active and purposeful
offered to active Army personnel, too, used collaboration between patient and thera-
case managers to track participants and to pist’’ (Gaston, Thompson, Gallagher, Cour-
assist with follow-up assessments. Rather noyer, & Gagnon, 1998). A recent
surprisingly, some studies found that mini- meta-analysis including 190 studies found a
mal outreach interventions after attempted significant correlation of r ¼ .28 between
suicide, such as contacting patients with the alliance and psychotherapy outcome
regular postcards or standardized letters, (Horvath et al., 2011). The effect of alliance
are associated with a reduction of suicides is increased when the treatment protocol is
(Carter, Clover, Whyte, Dawson, & D’Este, more structured, and when patients indicate
2007, 2013; Motto & Bostrom, 2001), high agreement with the therapist on goals
but findings have not been consistent and tasks (Arnow et al., 2014). Horvath,
(Beautrais, Gibb, Faulkner, Fergusson, & Gaston, and Luborsky (1993) distinguished
Mulder, 2010; Kapur et al., 2013). three universal aspects of therapeutic
A major barrier to the effective alliance: (a) The patient’s perception that
treatment of suicide attempters is poor the interventions offered are both relevant
treatment engagement, defined as com- and potent; (b) congruence between
mitment in the therapeutic process and the patient’s and the therapist’s expectations
active participation in a collaborative of the short- and medium-term goals of
relationship between therapist and therapy; (c) the patient’s ability to forge
patient (Lizardi & Stanley, 2010). One a personal bond with the therapist, and the
possible reason is that suicidal patients therapist’s ability to appear as a caring, sensi-
often do not feel understood by health tive, and sympathetic helping figure.
professionals, and that the focus of treat- We found only one treatment study
ment is not relevant for them (Michel, with suicidal patients investigating the
Valach, & Waeber, 1994; Treolar & effect of alliance on therapy outcome.
Pinfold, 1993). Clinicians tend to see sui- Bryan et al. (2012) used the Therapeutic
cide as a symptom of mental disorder Bond Scale to assess patient-rated evalu-
(Jobes, 2000), while for the suicidal ation of the treatment relationship, but
individual the central issue is the very found no association with the one suicide
personal experience of pain, anguish, related item in the Behavior Health

114 VOLUME 21  NUMBER 1  2017

 A. C. Gysin-Maillart et al.

Measure-20 as outcome measure. In an ear- conceptualization, safety planning, and

lier study we had reported that in a single continued long-term outreach contact.
assessment interview with patients who
had attempted suicide, patient-rated
therapeutic alliance measured with the METHODS
Penn Helping Alliance Questionnaire
HAQ (Alexander & Luborsky, 1986) was Procedure and Participants
increased when the interviewer used a nar-
rative interviewing technique (Michel et al., The study group represents the treat-
2004), based on a model of suicidal beha- ment arm of a randomized controlled trial
vior as a goal-directed action (Michel & on the effectiveness of ASSIP in reducing
Valach, 1997). This approach stands in suicidal behavior (Gysin-Maillart et al.,
contrast to the traditional medical model, 2016). The objective of the present study
in which suicidal behavior is usually seen was to investigate whether the quality of
as a symptom of psychiatric disorder. early therapeutic alliance in ASSIP was
Central to an action theoretical model is related to outcome during a 12-months
the notion that actions are understood as follow-up period. Suicidal ideation was
being carried out by agents, that is, by used as the outcome measure.
persons who are able to monitor their The study was conducted in a natural-
thoughts, emotions, and actions, and istic setting, with patients being assessed by
who have conscious access to their the duty psychiatrist in the emergency
reasons why they act in such a way. In a department of the University General
typical narrative interview the interviewee Hospital. Psychiatric consultation service
is considered the expert of his or her was provided by the medical staff of the
story. The interviewer’s primary task is University Hospital of Psychiatry Berne.
to be an attentive listener and to facilitate The duty psychiatrist informed patients
the patient’s narrative. about the study and asked for their written
We therefore hypothesized (1) that a consent to be contacted by an ASSIP thera-
therapy using a patient-centered, collabora- pist. The term ‘‘attempted suicide’’ is used
tive approach would enhance therapeutic according to Silverman, Berman, Sanddal,
alliance, and (2) that a stronger alliance O’Carroll, and Joiner (2007, p. 273), i.e., a
would be associated with better therapy ‘‘self-inflicted, potentially injurious beha-
outcome. Based on a model of suicidal vior with a nonfatal outcome for which
behavior as goal-oriented action we there is evidence (either explicit or implicit)
developed ASSIP (Attempted Suicide Short of intent to die.’’ In German speaking
Intervention Program), a highly structured countries, similar to North America, the
brief therapy consisting of three sessions, term attempted suicide does not include
followed by 2 years’ contact with regular habitual self-harm, which is characteristic
letters (Michel & Gysin-Maillart, 2015). for borderline personality disorders. At
The findings of the earlier study focusing the time of contact by the research team
on therapeutic alliance and narrative inter- the majority of patients had been admitted
viewing (Michel et al., 2004) prompted us to the psychiatric hospital or the crisis
to use a narrative interviewing style, which intervention unit. The responsible clinician
we complemented with a video playback was contacted before patients were given
session (Young & Valach, 2002). A major an appointment in order to assure the
focus of ASSIP lies in the development safety of patients for transport to the out-
of an early therapeutic alliance, combined patients department. Patients who had
with psychoeducation, a cognitive case been discharged after medical treatment

ARCHIVES OF SUICIDE RESEARCH 115

 Suicide Ideation and Therapeutic Alliance

were contacted directly and offered treat- ASSIP Therapy

ment. ASSIP was recommended to suicide
attempters as the usual procedure at the ASSIP brief therapy for patients who
University Hospital of Psychiatry, and over have attempted suicide is administered in
90% of the patients who fulfilled the three 60–90 minute sessions, ideally within
inclusion criteria could be allocated to the 3 weeks. A fourth session can be added if
study. Exclusion criteria for allocation to considered necessary. For details see
the study were language problems, current Michel and Gysin-Maillart (2015).
psychotic disorder, and dissocial person- First Session. A narrative interview in
ality disorders. Clinical diagnoses were which the patient is encouraged to tell the
made by the duty psychiatrists of the Uni- story behind the suicide attempt is conduc-
versity General Hospital and the University ted. The aim of the narrative interview is to
Hospital of Psychiatry, and were based on reach a shared understanding of the indi-
the ICD-10 Classification of Mental and vidual mechanism leading to suicidal beha-
Behavioral Disorders (WHO, 2014). vior in a biographical context, and to elicit
Written informed consent was obtained specific vulnerabilities and trigger events.
from all participants, as well as written The interview is video-recorded.
agreement for video-recording the narra- Second Session. Sequences of the video-
tive interview. Participants completed a recorded interview considered to be
set of questionnaires after the first therapy relevant for further elaboration are selected
session. For the follow-up evaluation part- and viewed on a screen, with patient and
icipants received the same questionnaires therapist sitting side by side. The aim of
after 6 and 12 months. Study procedures the video playback is the reactivation of
were approved by the local ethics commit- the mental state in the suicidal crisis in a
tee in accordance with the principles of the safe environment and the reconstruction
Declaration of Helsinki (Rickham, 1964). of the transition from a mental pain experi-
ence to the suicide action. Automatic
thoughts, emotions, physiological changes,
Therapists and contingent behavior are identified. At
the end of the session, patients are given
Treatment was provided by four a psychoeducative handout, which they
ASSIP-trained therapists. Two of them are asked to read and comment in writing,
(KM, psychiatrist; AGM, clinical psychol- and bring back to the next session. After
ogist and head of special clinic for suicidal the session the therapist prepares a draft
patients) were experienced clinicians, case conceptualization, which includes a
while two others (MM and SB, clinical formulation of the individual vulnerability
psychologists) with an average of 2 years in the context of the patient’s biography
of clinical practice after graduation were and the specific trigger event(s).
introduced to ASSIP and to the study Third Session. The patient’s feedback to
procedures during the study. Training of the handout is discussed. The written case
the ASSIP protocol including case super- conceptualization is revised together with
vision with the use of the video-recorded the patient. A list of long-term goals, indi-
narrative interviews was conducted by vidual warning signs, and safety strategies
AGM and KM. The clear structure of developed in a collaborative way. Summary
the three therapy sessions (see below) and safety strategies are printed and handed
provided a strict frame with tasks to be to the patient, with copies for the patient’s
completed in each session, securing adher- therapist and other relevant mental health
ence to the treatment protocol. professionals. The safety strategies are

116 VOLUME 21  NUMBER 1  2017

 A. C. Gysin-Maillart et al.

copy-pasted to a credit-card size ‘‘lepor- Beck Scale for Suicide Ideation. The Beck
ello’’ (a folded leaflet) and handed to the Scale for Suicide Ideation (BSS) is a
patient with the instruction to keep it the 21-item self-report instrument for measur-
wallet. In addition, patients are given a cri- ing the current intensity of the patients’
sis card with emergency phone numbers. specific attitudes, behaviors, and plans
Continued Outreach Contact. Patients are related to suicidal behavior during the past
sent letters over a time span of 2 years, week (Beck & Steer, 1991). Internal
every 3 months in the first year, and every reliability, test-retest stability and validity
6 months in the second year. The letters for the BSS have been established (Brown,
invite patients to give feedback about 2001; de Beurs, Fokkema, de Groot, de
how things were going. The content is Keijser, & Kerkhof, 2015). The severity
standardized, with an added remark refer- of suicidal ideation is calculated by sum-
ring to the personal safety plans developed ming the ratings of the first 19 items. Item
in the therapy sessions. Letters are person- 20 (prior suicide attempts) and 21 (severity
ally signed by the ASSIP therapist. If the of the suicide attempt) are not included in
therapist receives feedback from the the score. The sum scores range from 0
patients via e-mail or letters no further to 38 points. In the present study the Ger-
re-connection is made until the subsequent man version of the BSS was used (Fidy,
letter, unless patients appear to be in a sui- 2008).
cidal crisis, in which case the therapist con-
tacts them. Any information received from
the patient is usually referred to in the sub- Statistical Analysis
sequent letter.
Data were analyzed using SPSS
version 19.0 statistical software package.
Measures Sociodemographic data and outcome
measures were examined using t-test or
Penn Helping Alliance Questionnaire. The chi-square tests. Changes over time in
11-item self-rating questionnaire Penn alliance, suicidal ideation, and depression,
Helping Alliance Questionnaire (HAQ; were measured with paired t-tests. In
Alexander & Luborsky, 1986) is used to order to elucidate which factors most
evaluate the quality of the patient-therapist strongly predict higher suicide ideation
relationship. The HAQ has a good validity over the course of one year, we performed
for predicting outcome of psychotherapy a stepwise multiple linear regression analy-
(Fenton, Cecero, Nich, Frankforter, & sis using the HAQ, the BDI, and a history
Carroll, 2001). In the present study the of previous suicide attempts (PSA) as
German version by Bassler, Potratz, and independent variables. Where the Mauchly
Krauthauser (1995) is used. These authors test of sphericity indicated heterogeneity
found a satisfactory construct validity of of covariance, we verified repeated-
the German HAQ version in a study involv- measures results with Greenhouse-Geisser
ing 239 patients in a 12-week therapy. corrections. All tests were two-tailed and a
Beck Depression Inventory. The Beck probability of false rejections <0.05 was
Depression Inventory (BDI) is a 21-question considered statistically significant. All vari-
multiple-choice self-report inventory for ables were considered normally distributed
measuring the severity of depression (Beck (Kolmogorov-Smirnov test: p > 0.1, for all
& Steer, 1987). In the present study the variables).
German version of the BDI I was used Missing Data. In order to complete the
(Hautzinger, Bailer, Worall, & Keller, 1994). follow-up data hospital records were

ARCHIVES OF SUICIDE RESEARCH 117

 Suicide Ideation and Therapeutic Alliance

searched and involved health professionals TABLE 1. Demographic and Clinical

were contacted personally. The Last Characteristics
Observation Carried Forward (LOCF) N ¼ 60 (%)
method was used to handle missing data.
For each individual, missing data were Sex n (%)
replaced by the last observed value of that Female 36 (60)
variable. Male 24 (40)
Age (years)
Mean (SD) 36.5 (þ=  14.25)
RESULTS Time to first interview (days)
Mean (SD) 33.2 (47.88)
Participant Characteristics 1
Diagnosis (ICD-10)  n (%)
F1 10 (16.7)
Demographic characteristics, psychi-
F3 40 (66.7)
atric diagnosis, and history of previous
F4 (without F43.0 & F43.2) 3 (5.0)
suicide attempts for participants are shown
in Table 1. Of the 60 participants, 24 (40%) F43.0_F43.2 25 (41.7)
were male (mean age 39.42, SD 15.25), F6 7 (11.7)
and 36 (60%) female (mean age 34.58, SD Others 6 (10.0)
13.42). Thirty-three participants (55%) Attempted suicide method n (%)
were diagnosed with major depression, 8 Overdose 38 (63.3)
(13%) fulfilled the criteria for personality Cutting 8 (13.3)
disorder, 17 (28%) for acute stress reaction, Jumping 5 (8.3)
and 2 (3%) for substance abuse. Overdos- Other (hanging, shooting, 7 (11.7)
ing was the most frequent method used drowning)
(63.3%), while lethal methods, such as Previous attempts n (%)
hanging, shooting, and drowning repre-
0 33 (55.0)
sented 11.7% of all methods. Twenty-seven
(45%) participants reported previous 1 17 (28.3)
suicide attempts, 10 of them were multiple 2 and more (multiple) 10 (16.7)
repeaters. Three participants (5%) dropped Note. 1F1: Substance related disorders; F3: affective
out within the first 12 months of the disorders; F4: neurotic, stress-related, and somato-
follow-up. The reasons were: Psychotic dis- form disorders; F43.0 & F43.2: acute stress reaction,
order (1), refusal to return the follow-up adjustment disorder; F6: personality disorder; others:
F5, Behavioral syndromes with physical disorder and
questionnaires due to a change of ASSIP F9, Disorders usually first diagnosed in infancy,
therapist (2). The numbers of study parti- childhood, or adolescence.
cipants with complete follow-up data were,
at 6 months n ¼ 44 (73%), and at 12
months n ¼ 36 (60%).
(HAQ1) to the third session (HAQ2)
(HAQ1: 4.82  0.6; HAQ2: 5.19  0.6,
Therapeutic Alliance at First and Third t59 ¼ 4.854; p ¼ .001). Bivariate correla-
Session tions between patient-rated quality of
therapeutic alliance and previous suicide
Initial analysis employing paired t-test attempts revealed a negative association at
revealed a significant increase in the ratings the first session (r ¼ 0.34; p ¼ .008) but
of the Penn Helping Alliance Question- not at the third session (r ¼ 0.13;
naire from the first therapy session p ¼ .340). Thus, a history of previous

118 VOLUME 21  NUMBER 1  2017

 A. C. Gysin-Maillart et al.

TABLE 2. Spearman Correlation Between Outcome Measure: Suicidal Ideation

Suicidal Ideation at T2 and T3,
Therapeutic Alliance, Baseline
Scores of Depressive Symptoms and
Results of dependent t-test revealed a
Previous Suicide Attempts significant decrease in the scores of suicidal
ideation from baseline t1 (BSS:
BSS t2 BSS t3 0.39  0.45), to t2 at 6 months (BSS:
(N ¼ 60) (N ¼ 60) 0.30  0.5; t59 ¼ 2.185; p ¼ .033), and to t3
HAQ1 r ¼ 0.47; r ¼ 0.50; at 12 months (BSS: 0.25  0.44;
p ¼ .000 p ¼ .000 t59 ¼ 2.674; p ¼ .010).
HAQ2 r ¼ 0.32; r ¼ 0.36; Bivariate correlations (Table 2)
p ¼ .013 p ¼ .005
between patient-rated quality of therapeutic
alliance (HAQ1 & HAQ2) and suicidal
BDI baseline r ¼ 0.49; r ¼ 0.40;
ideation at 6 (BSSt2) and 12 months (BSSt3)
p ¼ .000 p ¼ .002
show a significant negative association, i.e.,
PSA r ¼ 0.59; r ¼ 0.56; higher ratings of therapeutic alliance at the
p ¼ .000 p ¼ .000 first and third session were associated with
Note. BSS: Beck Scale for Suicide Ideation (Beck & lower suicidal ideation at 6 and 12 months.
Steer, 1991); BSSt2: 6 months follow up; BSSt3: 12 Baseline depressive symptoms (BDIt1) and
months follow-up. HAQ: Therapeutic Alliance previous suicide attempts (PSA) correlated
Questionnaire (Alexander & Luborsky, 1986); with higher suicidal ideation (BSSt3) at
HAQ1: first therapy session; HAQ2: third therapy
session; BDI: Beck Depression Inventory (Beck &
one year follow-up.
Steer, 1987), BDI Baseline: first therapy session;
PSA: Previous Suicide Attempts. Predictors for Suicidal Ideation

Based on the association between

suicide attempts had a negative effect on therapeutic alliance, depressive symptoms,
the formation of a therapeutic alliance in and suicidal ideation, three multiple linear
the first session only. Further analysis regressions were conducted to analyze
revealed a negative association between the influence of each factor separately:
depression scores and the quality of therapeutic alliance, baseline depression,
therapeutic alliance at the first (HAQ1: and previous suicide attempts as predictor
r ¼ 0.37; p ¼ .003) and third session variables on suicidal ideation after 1 year
(HAQ2: r ¼ 0.27; p ¼ .036). as the dependent variable. First session

TABLE 3. Stepwise Multiple Linear Regression Analysis of Suicidal Ideation (BSSt3) at 12 Months
Follow-Upa

Variable R2 Corrected R2 DR2 DF DP Standardized ß T P

Step 1 .325 .301 .325 13.705 .000

BDIt1 .153 1.200 .235
PSA .475 3.725 .000
Step 2 .417 .386 .093 8.892 .004
HAQ1 .334 2.982 .004

Note. aModel: F ¼ 20.17, df ¼ 2, 59, p ¼ .000.

BSS: Beck Scale for Suicide Ideation (Beck & Steer, 1991, BSSt3: 12 months follow-up); BDIt1: Beck
Depression Inventory (Beck & Steer, 1987), first therapy session; PSA: Previous Suicide Attempts; HAQ:
Therapeutic Alliance Questionnaire (Alexander & Luborsky, 1986); HAQ1: first therapy session.

ARCHIVES OF SUICIDE RESEARCH 119

 Suicide Ideation and Therapeutic Alliance

therapeutic alliance (HAQ1) accounted for important to note that the HAQ
49.6% of the variance of suicidality expresses the patients’ own evaluation
(R2 ¼ .496; p ¼ .001), whereas depression of how much they felt understood and
(BDIt1), accounted for 40% (R2 ¼ .40; supported by the therapist.
p ¼ .002) of the variance. Previous suicide 2. An increase in the scores of therapeutic
attempts (PSA) explained 55% of the alliance in the ASSIP group between
variance (R2 ¼ .555; p ¼ .001). session one and session three. This is
To overcome the problem of consistent with results from psycho-
multicollinearity we conducted a linear therapy research, which suggest that
regression analysis with the correlated ratings of alliance in the third therapy
baseline measures previous suicide session are a good predictor of adher-
attempt (PSA) and depression at baseline ence to treatment and outcome
(BDIt1) as control variables in step one, (Saltzman, Luetgert, Roth, Creaser, &
and therapeutic alliance in step two (see Howard, 1976). We therefore assume
Table 3). PSA and BDIt1 explained that ASSIP therapists were successful
30.1% of the variance. In step two, 8.5% in creating a growing working relation-
of variation was added by the variable ship in the course of three sessions.
therapeutic alliance. 3. A negative effect of a history of pre-
vious attempts on the quality of the
therapeutic alliance in the first session.
DISCUSSION This effect disappeared in the third
therapy session. We therefore assume
The present study investigated the effect of that an initially skeptical attitude of
patient-rated therapeutic alliance in the these participants gave room to a more
Attempted Suicide Short Intervention Pro- positive attitude during the following
gram ASSIP, a specific, highly structured sessions.
brief therapy for attempted suicide, on 4. A negative effect of depressive symp-
the course of suicidal ideation after 6 and toms on therapeutic alliance in the first
12 months. We had hypothesized that a and the third session. This is not sur-
patient-centered, collaborative therapeutic prising, considering that participants
approach would enhance therapeutic with severe depression often find it
alliance, and that patient-rated alliance difficult to establish a therapeutic
would correlate with lower suicidal ideation alliance early in treatment. Furthermore,
at follow-up. higher baseline depression correlated
The main findings in the present study with more suicidal ideation after 6 and
are: after 12 months.
5. Regression analysis including thera-
1. A robust negative correlation between peutic alliance, previous suicide attem-
the patient-rated quality of the thera- pts, and depression revealed that a
peutic alliance in the first and third history of previous attempts had the
treatment session with suicidal ideation strongest effect on outcome. This is
in the BSS after 6 and 12 months, i.e., consistent with findings by Haw,
a stronger alliance was associated with Bergen, Casey, and Hawton (2007).
lower suicidal ideation after 6 and 12 Regression analysis suggests that
months. Obviously, therapeutic alliance therapeutic alliance and previous sui-
is determined by what both individuals, cide attempts may directly influence
therapist and patient, contribute to the outcome, while depression seems to
therapeutic process. However, it is have a negative effect on both.

120 VOLUME 21  NUMBER 1  2017

 A. C. Gysin-Maillart et al.

In summary, ASSIP, although a brief session are a good predictor of adherence

therapy, appears to initiate a therapeutic to treatment and outcome (Saltzman et al.,
interaction, increasing therapeutic alliance 1976).
from session one to three, with depression The patient information about the lim-
and previous suicide attempts acting as ited therapy goals may have been a further
confounding factors for suicidal ideation factor enhancing treatment engagement.
as therapy outcome. In our view it is Patients were informed that they had an
unlikely that the correlation between increased risk of future suicidal crises and
alliance and outcome is merely the that the therapy would not cure them from
expression of high alliance measures being suicidality, but that the aim was to reduce
a marker for low suicidality. Most of our the risk of further suicidal behavior.
study participants had overdosed. The dis- Furthermore, we assume that the regular
tribution of methods for attempting suicide letters may have acted as reminders of the
is in line with other studies (Horrocks, increased suicide risk, the warning signs,
Price, House, & Owens, 2003). Overdosing and the safety strategies developed in the
has been associated with higher rates of therapy sessions. To our knowledge, so
repeat attempts (Arensman, Corcoran, & far, no published treatment studies have
Fitzgerald, 2011), while high lethality meth- combined psychological interventions
ods have a higher risk of subsequent with subsequent ongoing contact through
suicide (Bergen, Hawton, Waters, Cooper, letters.
& Kapur, 2010; Horesh, Levi, & Apter, There are a number of limitations
2012; Runeson, Tidemalm, Dahlin, inherent in this study. The Penn Helping
Lichtenstein, & Langstrom, 2010). Alliance Questionnaire was originally
We can only speculate on the therapy developed to assess therapeutic alliance
process factors involved. ASSIP has been for psychotherapy with more than 25 ses-
devised as a therapy focusing on a sions (Morgan, Luborsky, Crits-Christoph,
patient-centered collaboration. We believe Curtis, & Solomon, 1982), and not for brief
that a strength of the ASSIP protocol are interventions such as ASSIP. Luborsky et al.
the key elements designed to promote (1996) have developed an improved ver-
‘‘ . . . the active and purposeful collabor- sion of the HAQ, which, however, focused
ation between patient and therapist’’ (Gas- even more on long-term therapeutic pro-
ton et al., 1998). As reported in an earlier cesses. We explained to the participants
study (Michel et al., 2004) a narrative inter- that, even if they had an ongoing therapy
viewing style was associated with better elsewhere, they should only indicate how
therapeutic alliance. We believe that the the felt after the interview with the ASSIP
initial narrative interview may set the basis therapist. The increase of alliance after
for a collaborative working alliance. In three sessions found in this study is con-
addition, ASSIP includes various thera- sistent with results from psychotherapy
peutic elements (video playback, psychoe- research, supporting the validity of the
ducation, safety planning, personalized HAQ ratings.
regular letters), which are all aimed at Because of the clinical nature of the
strengthening the collaborative therapeutic treatment study we felt that the first session
relationship. The finding that the scores should start with the therapists inviting the
of therapeutic alliance increased from ses- patients to tell their story. Therefore the
sion one (HAQ1) to session three questionnaires were completed immediately
(HAQ2) is consistent with results from after the conclusion of the first session.
psychotherapy research, which found that It could be argued that this poses a prob-
ratings of alliance in the third therapy lem to the baseline data. However, if the

ARCHIVES OF SUICIDE RESEARCH 121

 Suicide Ideation and Therapeutic Alliance

baseline was indeed biased by this pro- patients who attempt suicide do not differ
cedure, it is most likely that the experience in a number of variables generally known
of feeling understood by a caring therapist to be risk factors for suicide (Pompili
would have a positive effect on the scores et al., 2014). Therefore, BSS scores can be
of the baseline measures (suicide ideation taken as an indirect proxy for actual suicide
and levels of depression), and would there- risk.
fore reduce the treatment effect. The therapists in our study were not
Missing data regarding the BSS were involved in the clinical management of the
caused by non-response of participants patients. It is possible that this factor had a
during follow-up and are a frequent prob- positive effect on the development of the
lem of longitudinal studies. The Last therapeutic relationship. It will be interesting
Observation Carried Forward (LOCF) to see if our findings can be replicated in
method allows examination of trends over other clinical settings. Furthermore, although
time, but tends to underestimate the varia- the ASSIP therapy protocol is highly struc-
bility of results, in this case the BSS scores tured, setting clear tasks for each session,
at 6 and 12 months. In view of the decrease we do not yet know how much specific train-
of the BSS scores during follow-up LOCF ing is necessary for therapists to apply
would be expected to underestimate the ASSIP. So far, from our experience of train-
effect of alliance on outcome. We therefore ing therapists we have found that both,
consider LOCF an acceptable method for conducting a narrative interview, and formu-
our study. Missing data did not concern lating a patient-centered case conceptualiza-
HAQ ratings, which were collected tion, are elements that particularly need
immediately after session one and three. close supervision.
Therapist continuity was a problem In conclusion, the results suggest that
due to the maternity leave of the main therapeutic alliance may have a moderating
investigator (AGM): A considerable num- effect on outcome in the treatment of suici-
ber of participants were seen by one thera- dal patients. To further clarify the role of the
pist but received letters including the therapeutic relationship (and treatment
follow-up questionnaires signed by a differ- engagement) future treatment studies should
ent person. This is a practical problem typi- include measures of alliance. Patient-
cal for real-world treatment studies, which centered models of suicidal behavior and
must be expected to reduce treatment collaborative treatment approaches may be
effects. particularly promising. Special attention
It may be questioned how much a should be paid to patients with a history
decrease of suicidal ideation correlates with of attempted suicide and those with a
lower risk of further suicidal behavior. The depressive disorder. These individuals
BSS suicide ideation items are composed of appear to have more difficulties in establish-
three factors: Desire for Death, Preparation ing a therapeutic alliance with the therapist.
for Suicide and Actual Suicide Desire
(Steer, Rissmiller, Ranieri, & Beck, 1993).
A high proportion of people indicating sui- AUTHOR NOTE
cide plans (72%) move from plan to
attempt (Kessler, Borges, & Walters, Anja C. Gysin-Maillart, University Hospital
1999). Suicide ideation in patients who of Psychiatry, University of Bern, Bern,
attempted suicide has been associated with Switzerland.
persistent suicidal behavior (Pinninti, Steer, Leila M. Soravia, University Hospital
Rissmiller, Nelson, & Beck, 2002). Further- of Psychiatry, University of Bern, Bern,
more, patients with suicide ideation and Switzerland.

122 VOLUME 21  NUMBER 1  2017

 A. C. Gysin-Maillart et al.

Armin Gemperli, Department of Bassler, M., Potratz, B., & Krauthauser, H. (1995).
Health Sciences and Health Policy, Univer- Der helping alliance questionnaire (HAQ) von
sity of Luzern, Bern, Switzerland. luborsky. möglichkeiten zur evaluation des
Konrad Michel, University Hospital of therapeutischen prozesses von stationärer
psychotherapie. Psychotherapeut, 40, 23–32.
Psychiatry, University of Bern, Bern, Swit-
Beautrais, A. L., Gibb, S. J., Faulkner, A., Fergusson,
zerland. D. M., & Mulder, R. T. (2010). Postcard
We thank Millie Megert, who replaced intervention for repeat self-harm: Randomised
A.G-M. during her maternity leave, and controlled trial. British Journal of Psychiatry, 197(1),
Salome Bühler, who contributed with an 55–60. doi: 10.1192=bjp.bp.109.075754
ASSIP case. Beck, A. T., & Steer, R. A. (1987). BDI, Beck
Correspondence concerning this article depression inventory: Manual. New York, NY:
should be addressed to Konrad Michel, Psychological Corporation.
University Hospital of Psychiatry, Univer- Beck, A. T., & Steer, R. A. (1991). Manual for the
sity of Bern, Murtenstrasse 21, 3010 Bern, Beck scale for suicide ideation. San Antonio, TX:
Switzerland. E-mail: [email protected] Psychological Corporation.
Bergen, H., Hawton, K., Waters, K., Cooper, J., &
nibe.ch
Kapur, N. (2010). Epidemiology and trends in
non-fatal self-harm in three centres in England:
2000–2007. British Journal of Psychiatry, 197(6),
REFERENCES 493–498. doi: 10.1192=bjp.bp.110.077651
Brown, G. K. (2001). A review of suicide assessment
Alexander, L. B., & Luborsky, L. (1986). The Penn measures for intervention research with adults and
helping alliance scales. In L. S. Greenberg & W. older adults. Retrieved 18 September 2014, http://
M. Pinsoff (Eds.), The psychotherapeutic process: A ruralccp.org/lyra-data/storage/asset/brown-nd-
research handbook (pp. 325–366). New York, NY: 27cb.pdf
Guilford Press. Brown, G. K., & Jager-Hyman, S. (2014).
Arensman, E., Corcoran, P., & Fitzgerald, A. P. Evidence-based psychotherapies for suicide
(2011). Deliberate self-harm: Extent of the prob- prevention: Future directions. American Journal of
lem and prediction of repetition. In R. O’Connor, Preventive Medicine, 47(3 Suppl 2), S186–S194. doi:
S. Platt & J. Gordon (Eds.), International handbook of 10.1016=j.amepre.2014.06.008
suicide prevention (pp. 122). Chichester: Wiley & Brown, G. K., Ten Have, T., Henriques, G. R.,
Sons. Xie, S. X., Hollander, J. E., & Beck, A. T.
Arensman, E., Townsend, E., Hawton, K., Bremner, (2005). Cognitive therapy for the prevention
S., Feldman, E., Goldney, R., . . ., & of suicide attempts: A randomized controlled
Traskman-Bendz, L. (2001). Psychosocial and trial. JAMA, 294(5), 563–570. doi: 10.1001=
pharmacological treatment of patients following jama.294.5.563
deliberate self-harm: The methodological issues Bryan, C. J., Corso, K. A., Corso, M. L., Kanzler, K.
involved in evaluating effectiveness. Suicide and E., Ray-Sannerud, B., & Morrow, C. E. (2012).
Life-Threatening Behavior, 31(2), 169–180. doi: Therapeutic alliance and change in suicidal
10.1521=suli.31.2.169.21516 ideation during treatment in integrated primary
Arnow, B. A., & Steidtmann, D. (2014). Harnes- care settings. Archives of Suicide Research, 16(4),
sing the potential of the therapeutic alliance. 316–323. doi: 10.1080=13811118.2013.722055
World Psychiatry, 13(3), 238–240. doi: 10.1002= Carroll, R., Metcalfe, C., & Gunnell, D. (2014).
wps.20147 Hospital presenting self-harm and risk of fatal
Arnow, B. A., Steidtmann, D., Blasey, C., Manber, and non-fatal repetition: Systematic review and
R., Constantino, M. J., Klein, D. N., . . ., & Kocsis, meta-analysis. PLoS ONE, 9(2), e89944. doi:
J. H. (2014). The relationship between the thera- 10.1371=journal.pone.0089944
peutic alliance and treatment outcome in two Carter, G. L., Clover, K., Whyte, I. M., Dawson, A.
distinct psychotherapies for chronic depression. H., & D’Este, C. (2007). Postcards from the edge:
Journal of Consulting and Clinical Psychology, 81(4), 24-month outcomes of a randomised controlled
627–638. doi: 10.1037=a0031530 trial for hospital-treated self-poisoning. British

ARCHIVES OF SUICIDE RESEARCH 123

 Suicide Ideation and Therapeutic Alliance

Journal of Psychiatry, 191, 548–553. doi: 10.1192= (ASSIP). PLOS Medicine, 13(3), e1001968. doi:
bjp.bp.107.038406 10.1371=journal.pmed.1001968
Carter, G. L., Clover, K., Whyte, I. M., Dawson, A. Hautzinger, M., Bailer, M., Worall, H., & Keller, F.
H., & D’Este, C. (2013). Postcards from the edge: (1994). Beck-depressions-inventar (BDI). Bearbeitung
5-year outcomes of a randomised controlled trial der deutschen Ausgabe. Testhandbuch. Bern: Huber.
for hospital-treated self-poisoning. British Journal Haw, C., Bergen, H., Casey, D., & Hawton, K.
of Psychiatry, 202(5), 372–380. doi: 10.1192= (2007). Repetition of deliberate self-harm: A study
bjp.bp.112.112664 of the characteristics and subsequent deaths in
Cedereke, M., & Ojehagen, A. (2005). Prediction of patients presenting to a general hospital according
repeated parasuicide after 1–12 months. European to extent of repetition. Suicide and Life-Threatening
Psychiatry, 20(2), 101–109. doi: 10.1016= Behavior, 37(4), 379–396. doi: 10.1521=
j.eurpsy.2004.09.008 suli.2007.37.4.379
Cooper, J., Kapur, N., Webb, R., Lawlor, M., Heyerdahl, F., Bjornaas, M. A., Dahl, R., Hovda, K.
Guthrie, E., Mackway-Jones, K., & Appleby, L. E., Nore, A. K., Ekeberg, O., & Jacobsen, D.
(2005). Suicide after deliberate self-harm: A 4-year (2009). Repetition of acute poisoning in Oslo:
cohort study. American Journal of Psychiatry, 162(2), 1-year prospective study. British Journal of Psychiatry,
297–303. doi: 10.1176=appi.ajp.162.2.297 194(1), 73–79. doi: 10.1192=bjp.bp.107.048322
Crowder, R., Van der Putt, R., Ashby, C. A., & Horesh, N., Levi, Y., & Apter, A. (2012). Medically
Blewett, A. (2004). Deliberate self-harm patients serious versus non-serious suicide attempts: Rela-
who discharge themselves from the general hospi- tionships of lethality and intent to clinical and
tal without adequate psychosocial assessment. interpersonal characteristics. Journal of Affective
Crisis, 25(4), 183–186. doi: 10.1027=0227- Disorders, 136(3), 286–293. doi: 10.1016=
5910.25.4.183 j.jad.2011.11.035
de Beurs, D. P., Fokkema, M., de Groot, M. H., Horrocks, J., Price, S., House, A., & Owens, D.
de Keijser, J., & Kerkhof, A. J. F. M. (2015). (2003). Self-injury attendances in the accident
Longitudinal measurement invariance of the beck and emergency department: Clinical database
scale for suicide ideation. Psychiatry Research, 225(3), study. British Journal of Psychiatry, 183, 34–39. doi:
368–373. doi: 10.1016=j.psychres.2014.11.075 10.1192=bjp.183.1.34
Fenton, L. R., Cecero, J. J., Nich, C., Frankforter, Horvath, A. O., Del Re, A. C., Flückiger, C., &
T. L., & Carroll, K. M. (2001). Perspective is Symonds, D. (2011). Alliance in individual psycho-
everything: the predictive validity of six working therapy. Psychotherapy, 48(1), 9–16. doi: 10.1037=
alliance instruments. Journal of Psychotherapy Practice a0022186
and Research, 10(4), 262–268. Horvath, A. O., Gaston, L., & Luborsky, L. (1993).
Fidy, R. (2008). Psychologische suizidalita¨ts-diagnostik im The therapeutic alliance and its measures. In
internet. Zürich: Universität Zürich. L. L. N. Miller, J. Barber, & J. P. Docherty
Gaston, L., Thompson, L., Gallagher, D., (Eds.), Psychodynamic Treatment Research. New York:
Cournoyer, L., & Gagnon, R. (1998). Alliance, Basic Books.
technique, and their interactions in predicting out- Jenkins, G. R., Hale, R., Papanastassiou, M.,
come of behavioural, cognitive, and brief dynamic Crawford, M. J., & Tyrer, P. (2002). Suicide rate
therapy. Psychotherapy Research, 8, 190–209. doi: 22 years after parasuicide: Cohort study. BMJ,
10.1080=10503309812331332307 325(7373), 1155. doi: 10.1136=bmj.325.7373.1155
Granboulan, V., Roudot-Thoraval, F., Lemerle, S., & Jobes, D. A. (2000). Collaborating to prevent suicide:
Alvin, P. (2001). Predictive factors of A clinical-research perspective. Suicide and
post-discharge follow-up care among adolescent Life-Threatening Behavior, 30(1), 8–17.
suicide attempters. Acta Psychiatrica Scandinavica, Kapur, N., Gunnell, D., Hawton, K., Nadeem, S.,
104(1), 31–36. doi: 10.1034=j.1600-0447. Khalil, S., Longson, D., . . ., & Cooper, J. (2013).
2001.00297.x Messages from Manchester: Pilot randomised
Gysin-Maillart, A., Schwab, S., Soravia, L., Megert, controlled trial following self-harm. British Journal
M., & Michel, K. (2016). A novel brief therapy of Psychiatry, 203(1), 73–74. doi: 10.1192=
for patients who attempt suicide: A 24-months bjp.bp.113.126425
follow-up randomized controlled study of the Kessler, R. C., Borges, G., & Walters, E. E. (1999).
Attempted Suicide Short Intervention Program Prevalence of and risk factors for lifetime suicide

124 VOLUME 21  NUMBER 1  2017

 A. C. Gysin-Maillart et al.

attempts in the national comorbidity survey. prevention. Psychiatric Services, 52(6), 828–833.
Archives of General Psychiatry, 56(7), 617–626. doi: doi: 10.1176=appi.ps.52.6.828
10.1001=archpsyc.56.7.617 National Institute of Mental Health and the Research
King, C. A., Hovey, J. D., Brand, E., Wilson, R., & Prioritization Task Force. (2014). A prioritized
Ghaziuddin, N. (1997). Suicidal adolescents after research agenda for suicide prevention. An action
hospitalization: Parent and family impacts on plan to save lives. Rockville, MD: National Action
treatment follow-through. Journal of the American Alliance for Suicide Prevention. Retrieved from
Academy of Child and Adolescent Psychiatry, 36, 85– www.suicide-research-agenda.org.
93. doi: 10.1097=00004583-199701000-00021 Owens, D., Horrocks, J., & House, A. (2002). Fatal
Lizardi, D., & Stanley, B. (2010). Treatment engage- and non-fatal repetition of self-harm systematic
ment: A neglected aspect in the psychiatric care review. The British Journal of Psychiatry, 181(3),
of suicidal patients. Psychiatric Services, 61(12), 193–199. doi: 10.1192=bjp.181.3.193
1183–1191. doi: 10.1176=appi.ps.61.12.1183 Pinninti, N., Steer, R. A., Rissmiller, D. J., Nelson, S.,
Luborsky, L., Barber, J. P., Lynne, S., Johnson, S., & Beck, A. T. (2002). Use of the Beck scale for sui-
Najavitis, L. M., Frank, A., & Daley, D. (1996). cide ideation with psychiatric inpatients diagnosed
The revised helping alliance questionnaire with schizophrenia, schizoaffective, or bipolar
(HAq-II): Psychometric properties. The Journal of disorders. Behaviour Research and Therapy, 40(9),
Psychotherapy Practice and Research, 5, 260–271. 1071–1079. doi: 10.1016=s0005-7967(02)00002-5
Maltsberger, J. T. (2004). The descent into suicide. Pompili, M., Innamorati, M., Di Vittorio, C., Sher, L.,
The International Journal of Psychoanalysis, 85(3), Girardi, P., & Amore, M. (2014). Sociodemo-
653–668. doi: 10.1516=3c96-uret-tlwx-6lwu graphic and clinical differences between suicide
Michel, K., Dey, P., Stadler, K., & Valach, L. (2004). ideators and attempters: A study of mood dis-
Therapist sensitivity towards emotional life-career ordered patients 50 years and older. Suicide and
issues and the working alliance with suicide Life-Threatening Behavior, 44(1), 34–45. doi:
attempters. Archives of Suicide Research, 8(3), 10.1111=sltb.12051
203–213. doi: 10.1080=13811110490436792 Rickham, P. P. (1964). Human experimentation.
Michel, K., & Gysin-Maillart, A. (2015). Attempted Code of ethics of the world medical association.
suicide short intervention program ASSIP. A manual for Declaration of Helsinki. BMJ, 2(177).
clinicians. Göttingen: Hogrefe. doi:10.1136=bmj.2.5402.177
Michel, K., Maltsberger, J. T., Jobes, D. A., Leenaars, Rogers, J. R., & Soyka, K. M. (2004). ‘‘One size fits
A. A., Orbach, I., Stadler, K., . . ., & Valach, L. all’’: An existential-constructivist perspective on
(2002). Discovering the truth in attempted suicide. the crisis intervention approach with suicidal
American Journal of Psychotherapy, 56(3), 424–437. individuals. Journal of Contemporary Psychotherapy,
Michel, K., & Valach, L. (1997). Suicide as 34(1), 7–22. doi: 10.1023=b:jocp.0000010910.
goal-directed action. Archives of Suicide Research, 74165.3a
3(3), 213–221. doi: 10.1080=13811119708258273 Rudd, M. D., Bryan, C. J., Wertenberger, E. G.,
Michel, K., Valach, L., & Waeber, V. (1994). Under- Peterson, A. L., Young-McCaughan, S., Mintz, J.,
standing deliberate self-harm: The patients’ views. . . ., & Bruce, T. O. (2015). Brief cognitive-
Crisis, 15(4), 172–178. behavioral therapy effects on post-treatment
Monti, K., Cedereke, M., & Ojehagen, A. (2003). suicide attempts in a military sample: Results of
Treatment attendance and suicidal behavior 1 a randomized clinical trial with 2-year follow-up.
month and 3 months after a suicide attempt: A American Journal of Psychiatry, 172, 441–449. doi:
comparison between two samples. Archives of 10.1176=appi.ajp.2014.14070843
Suicide Research, 7, 167–174. doi: 10.1080= Runeson, B., Tidemalm, D., Dahlin, M.,
13811110301581 Lichtenstein, P., & Langstrom, N. (2010). Method
Morgan, R., Luborsky, L., Crits-Christoph, P., Curtis, of attempted suicide as predictor of subsequent
H., & Solomon, J. (1982). Predicting the outcomes successful suicide: national long term cohort
of psychotherapy by the Penn helping alliance rat- study. BMJ, 341, c3222. doi: 10.1136=bmj.c3222
ing method. Archives of General Psychiatry, 39(4), Sakinofsky, I. (2000). Repetition of suicidal behav-
397–402. iour. In K. Hawton & K. Van Heeringen (Eds.),
Motto, J. A., & Bostrom, A. G. (2001). A ran- The international handbook of suicide and attempted
domized controlled trial of postcrisis suicide suicide (pp. 385–404). Chichester: Wiley.

ARCHIVES OF SUICIDE RESEARCH 125

 Suicide Ideation and Therapeutic Alliance

Saltzman, C., Luetgert, M. J., Roth, C. H., Creaser, J., & Life-Threatening Behavior, 37(3), 264–277. doi:
Howard, L. (1976). Formation of a therapeutic 10.1521=suli.2007.37.3.264
relationship: Experiences during the initial phase of Steer, R. A., Rissmiller, D. J., Ranieri, W. F., & Beck,
psychotherapy as predictors of treatment duration A. T. (1993). Dimensions of suicidal ideation
and outcome. Journal of Consulting and Clinical Psychology, in psychiatric inpatients. Behaviour Research and
44(4), 546–555. doi: 10.1037=0022-006x.44.4.546 Therapy, 31(2), 229–236. doi: 10.1016=0005-
Shneidman, E. S. (1993). Commentary: Suicide as 7967(93)90090-h
psychache. The Journal of Nervous and Mental Disease, Treolar, A. J., & Pinfold, T. J. (1993). Deliberate
181(3), 145–147. doi: 10.1097=00005053- self-harm: An assessment of patients’ attitudes to
199303000-00001 the care they receive. Crisis, 14, 83–89.
Silverman, M. M., Berman, A. L., Sanddal, N. D., WHO (2014). The ICD-10 classification of mental and
O’Carroll, P. W., & Joiner, T. E. (2007). behavioral disorders: Clinical descriptions and diagnostic
Rebuilding the tower of Babel: A revised guidelines. Geneva: WHO.
nomenclature for the study of suicide and suicidal Young, R. A., & Valach, L. (2002). The
behaviors. Part 2: Suicide-related ideations, com- self-confrontation interview in suicide research.
munications, and behaviors. Suicide and Lifenotes, 7, 12–14.

126 VOLUME 21  NUMBER 1  2017

Copyright of Archives of Suicide Research is the property of Taylor & Francis Ltd and its
content may not be copied or emailed to multiple sites or posted to a listserv without the
copyright holder's express written permission. However, users may print, download, or email
articles for individual use.