J Psychopathol Behav Assess

https://doi.org/10.1007/s10862-017-9632-8

Intraindividual Dynamic Network Analysis – Implications

for Clinical Assessment
Sarah Jo David 1 1 1
& Andrew J. Marshall & Emma K. Evanovich & Gregory H. Mumma
1

# The Author(s) 2017. This article is an open access publication

Abstract A network analysis approach to psychopathology In understanding the etiology and maintenance of psychopa-
regards symptoms as mutually interacting components of a thology for a particular individual, it is important to assess
multifaceted system (Borsboom & Cramer, 2013). Although symptoms: a) longitudinally, in order to capture dynamic, in-
several studies using this approach have examined comorbidity teractive processes; b) idiographically, to collect relevant
between disorders using cross-sectional samples, a direct appli- empirically-derived information at the individual level includ-
cation of the network analysis approach to intraindividual dy- ing information that may be unique to that person and their life
namic relations between symptoms in a complex, comorbid circumstances; and c) comprehensively, such that all comor-
case has not been reported. The current article describes an bid symptoms and causal variables relevant for that individual
intraindividual dynamic network analysis (IDNA) approach to are assessed in order to fully understand the unique processes
understanding the psychopathology of an individual using dy- that are occurring within the individual (Haynes, O'Brien, &
namic (over time) lead-lag interrelations between symptoms. Kaholokula, 2011; Mumma, 2011). After briefly discussing
Multivariate time series data were utilized to create and exam- these three themes in assessment, we describe how the clinical
ine an intraindividual, lag-1 network of the partial, day-to-day assessment of an adult with complex comorbid symptoms
relations of symptoms in an individual with comorbid mood may benefit from a new approach to person-specific assess-
and anxiety disorders. Characteristics of the network, including ment: intraindividual dynamic network analysis (IDNA).
centrality indices, stability, dynamic processes between symp- The longitudinal study of individuals with psychopatholo-
toms, and their implications for clinical assessment are de- gy can inform and advance the conceptualization of comorbid
scribed. Additional clinical implications and future directions disorders and interrelated dimensions (e.g., anxiety and de-
for IDNA, including the potential incremental validity of this pression; Brown, Campbell, Lehman, Grisham, & Mancill,
assessment approach for empirically-based idiographic assess- 2001). Such studies can reveal whether level of distress in
ment and personalized treatment planning, are discussed. This one dimension predicts subsequent levels in that or other di-
person-specific IDNA approach may be especially useful in mensions. For example, several studies have found that level
complex and comorbid cases. of anxiety predicts subsequent levels of depression even after
adjusting for previous level of depression (Cole, Peeke,
Martin, Turglio, & Seroczynski, 1998; Kouros, Quasem, &
Keywords Network analysis . Intraindividual . Dynamic . Garber, 2013). As useful as such aggregate-level studies are,
Comorbidity . Personalized treatment planning the aggregated nature of the data analysis implies that conclu-
sions apply to the sample (or population), rather than the in-
dividual. Furthermore, the large intervals between waves (e.g.,
* Sarah Jo David months) that are typically used in many multi-wave designs
[email protected] may miss important processes occurring within shorter inter-
vals over time, such that results of longer-interval studies may
1
Department of Psychological Sciences, Texas Tech University,
have only general relevance for understanding the develop-
Psychological Sciences Building MS 2051, ment and time course of psychopathology within an
Lubbock, TX 79409-2051, USA individual.
 J Psychopathol Behav Assess

Comorbidity, the occurrence of more than one mental or be- of a multifaceted system (Borsboom & Cramer 2013).
havioral disorder in an individual, provides a challenge for re- Although most network analyses to date have used cross-
searchers and clinicians. Individuals with comorbidities are often sectional data, several studies have examined networks over
eliminated from randomized controlled trials (Rizvi & Harned, time. For example, Bringmann, Lemmens, Huibers,
2013) despite the high prevalence of comorbid disorders (Brown Borsboom, and Tuerlinckx (2014) utilized weekly time series
et al., 2001). The historical reluctance to include individuals with data to perform a NA on the dynamics of the Beck Depression
comorbidities in psychopathology and treatment research has led Inventory items (BDI-II; Beck, Steer, & Brown, 1996). In a
to a paucity of scientific evidence regarding the etiology and study using intensive ecological momentary assessment,
treatment of individuals with comorbid disorders. As a result, Bringmann et al. (2013) utilized a multilevel vector
clinicians are confronted with difficult decisions about whether autoregressive approach to examine mood-related networks
to treat multiple disorders simultaneously or sequentially, and in 129 participants using data collected over 12 days. In each
which symptoms to target and when for a particular client of these studies, however, all individuals made ratings on the
(Haynes et al., 2011; Persons, 2008). same set of items or variables. Thus, these aggregate-level
Assessment in randomized controlled trials typically involves NAs may be useful for studying the development and main-
standardized measures taken before and after the implementation tenance of symptoms as they typically occur within and be-
of the treatment. However, with increasing interest in personal- tween disorders and could be utilized to develop empirically-
ized treatment in medicine and clinical psychology (e.g., in adap- supported treatment interventions of general relevance to that
tive designs and Sequential Multiple Assignment Randomized population.
Trials) assessment may involve multiple outcome and mediation- In contrast, on a person-specific level, a network concep-
al variables that are repeatedly measured during the course of tualization can model the idiosyncratic symptom presentation
treatment (Barlow, Bullis, Comer, & Ametaj, 2013; Kazdin, of an individual with comorbid and complex forms of psycho-
2017; Lei, Nahum-Shani, Lynch, Oslin, & Murphy, 2012). As pathology. Moreover, the intraindividual, time-series network
a result, individuals may be assigned to sequential treatment of an individual may be useful for understanding the etiology
branches, thereby more closely adapting the treatment options and maintenance of psychopathology at an individualized
to the clinical needs of a particular individual. level and could be used for personalized treatment planning.
In contrast to these more recent developments, case formula- Although Borsboom and Cramer (2013) described numerous
tion approaches have historically generated an individualized or methods for analyzing time-intensive data and individual net-
idiographic understanding of the individual with treatment plan- works, their description was limited to a hypothetical
ning implications by hypothesizing intraindividual relations be- example.
tween causal and outcome variables (Eells, 2007; Haynes,
Mumma, & Pinson, 2009; Persons, 2008). For example, the
functional analytic clinical case model of Haynes and O’Brien Intraindividual Dynamic Network Analysis (IDNA)
(2000; Haynes et al., 2011) graphically displays the expected
relations between symptoms and causal variables as hypothesized This article describes IDNA, an approach to understanding the
in the formulation. Mumma and colleagues (Mumma, 2004, psychopathology of an individual using dynamic (over time)
2011; Mumma & Mooney, 2007a, 2007b; b; Mumma & Fluck, interrelations between symptoms. There are several potential
2016) have demonstrated how these hypothesized relations in a advantages to this person-specific, intensive longitudinal net-
cognitive-behavioral case formulation can be empirically tested work analysis approach.
or evaluated using intensive longitudinal data collection and First, the IDNA approach explicitly assesses the dy-
person-specific analyses. Results of a case formulation (CF) eval- namic relations between symptoms within an individual
uation can be used to develop an empirically-based personalized so that a clinician or researcher can empirically evaluate
or tailored psychological treatment plan that may address the which symptoms are relatively more central to an indi-
most salient symptoms or processes for a client. Thus, the inten- vidual’s distress or dysfunction as it presents over time.
sive multivariate assessment of intraindividual relations between Second, the IDNA approach can capture relations among
variables has substantial potential for developing CFs and per- symptoms that change over time within an individual, as
sonalized treatment planning, particularly with comorbid or com- opposed to relations assessed in larger-scale designs in
plex cases (Haynes et al., 2011; Mumma, 2011). which changes in level from one assessment to another
are evaluated based on the location of a person’s score
within the distribution of scores across the sample on
Network Analysis that occasion. Third, the IDNA approach provides a dy-
namic picture of comorbidity within the individual and
A network analysis (NA) approach to psychopathology as- can evaluate the extent to which symptoms of separate
sumes that symptoms are mutually interacting components disorders function as relatively distinct interacting
J Psychopathol Behav Assess

systems for that person, perhaps with symptoms that were adapted from the Mood and Anxiety Symptoms
bridge between them (Cramer , Waldorp, van der Maas, Questionnaire (MASQ; Watson et al., 1995). The
& Borsboom, 2010; Robinaugh, LeBlanc, Vuletich, & MASQ is a self-report measure of symptoms relevant
McNally, 2014), versus interact as symptoms within a to the tripartite model of depression and anxiety.
single dynamic system for that person. Thus, the IDNA Although Watson et al. (1995) condensed the original
approach described herein does not necessarily draw six subscales into a three dimensional factor structure
clear-cut boundaries between comorbid disorders within to better fit their large samples, the intraindividual di-
the person, but, instead, determines: a) how distinct the mensionality of these scales should not be assumed to
symptoms within a disorder are from symptoms within be the same (Molenaar, 2004). Thus, items from all
another disorder for that person, i.e., how well do the three s u b s c a l e s o f t h e G e n e r a l D i s t r e s s s c a l e :
symptoms cluster together dynamically for that person; Depression, Anxiety, and Mixed, were selected, as well
b) which symptoms are the core symptoms within the as items from the two bipolar subscales of the MASQ;
dynamic network or cluster of networks for that individ- Positive Affect and Anhedonia. Six to eight items from
ual; c) which symptoms (if any) are bridge symptoms each of these five subscales were included as symptoms
between these clusters; and more generally, d) how of distress for the present IDNA.
strong the dynamic interconnections are between the
symptoms in the network for that person. Procedure
This study utilized multivariate time series data to
create and examine the IDNA of symptoms for an indi- Sam was instructed to complete her IDQ at approxi-
vidual with comorbid mood and anxiety disorders. This mately the same time each day. She completed 90 daily
article responds to Borsboom and Cramer’s (2013) call to questionnaires over the course of 122 days, mailing
collect time-intensive intraindividual data and construct a re- each to the lab in a prepaid envelope. Twenty-two items
sultant idiosyncratic network depicting relations among symp- from this questionnaire that were used in an earlier
toms within a single individual. After describing the method- study involving confirmatory dynamic factor analysis
ology utilized to conduct the IDNA, the results for the case (Mumma, 2004) were used in the present study.
example are described. Finally, how this approach could be Descriptive statistics for these 22 items are in Table 1,
used to plan an individualized, tailored treatment plan for the which also shows the five scales they were placed in
most important symptoms for the individual is discussed. (Depression, Anxiety, Mixed, Anhedonia, Positive Affect) in
this earlier latent variable approach to understanding Sam’s
distress.
Methods

Participant Preliminary Analyses

The participant for the case example was a 44-year-old SAS statistical software (version 9.4; SAS Institute,
married female (herein referred to as “Sam”) who self- Cary, North Carolina, USA) and R (R Development
identified as Caucasian. According to the Structured Core Team 2010) were used for all statistical analyses.
Clinical Interview for DSM – IV (SCID-IV; First, Because the goal of the current study was to examine
Spitzer, Gibbon, & Williams, 1997), her diagnoses were the network of functional, occasion-to-occasion rela-
major depressive disorder, persistent depressive disorder tions, the items were detrended. Sam completed 90 dai-
(dysthymia), and social anxiety disorder. Scores on intake ly ratings over 122 days., Therefore, initial detrending
measures were as follows: Beck Depression Inventory models were tenth order so as to model possible linear
(BDI-II; Beck et al. 1996) = 38, Beck Anxiety Inventory and curvilinear trend as well as any monthly cyclicity in
(Beck & Steer 1993) = 18, and Beck Hopelessness the data.1 In addition to independent error models, first
Scale (Beck, Weissman, Lester, & Trexler, 1974) = 12. order autoregressive models were tested for each item to
These scores indicate severe depression, moderate anxi- assess for significant autocorrelation. As indicated in
ety, and moderate hopelessness, respectively. Table 1, although scores on most items did not require
detrending, five items required second order detrending
Measures and two required more complex polynomial detrending.

Individualized Daily Questionnaire (IDQ) The IDQ

consisted of both nomothetic and idiographic items assessing 1
A detailed example of the methods utilized can be found in Mumma (2004)
Sam’s cognitions and distress. The nomothetic distress items or may be obtained from the corresponding author.
 J Psychopathol Behav Assess

Table 1 Descriptive Statistics for Sam’s Final Detrended Items by Scale

Scale Item Label Mean* Detrending Minimum Maximum SD Skewness Kurtosis

Depression 1. Felt sad Sad 2.18 2nd −3.16 5.20 2.07 0.58 −0.10
2. Felt depressed Depressed 3.13 9th −2.68 2.79 1.25 −0.04 −0.66
3. Felt discouraged Discourage 2.07 None −2.12 5.88 2.34 1.24 0.94
4. Disappointed in myself Disappoint 2.36 2nd −3.52 5.51 2.67 0.55 −0.83
5. Blame myself for things Blame 1.36 10th −4.30 8.02 2.10 1.37 2.98
Anxiety 1. Felt nervous Nervous 1.08 None −1.13 6.87 1.86 1.73 2.31
2. Felt tense, “high-strung” Tense 1.53 None −1.54 7.46 2.43 1.37 0.74
3. Felt uneasy Uneasy 1.69 None −1.72 7.28 2.45 1.42 0.89
4. Unable to relax Un Relax 1.29 None −1.35 6.65 2.30 1.76 2.06
5. Felt “on edge”, keyed up On Edge 1.64 None −1.68 7.32 2.53 1.25 0.30
Mixed 1. Worried a lot about things Worry 1.76 None −1.76 5.24 2.05 0.85 −0.30
2. Trouble concentrating Trb Conc 1.64 None −1.63 5.37 1.97 0.87 −0.51
3. Felt confused Confused 0.70 None −0.70 5.30 1.25 1.76 2.86
Anhedonia 1. Felt nothing was enjoyable Not Enjoy 2.16 2nd −3.31 5.46 2.04 0.56 −0.12
2. Felt withdrawn from others Withdraw 2.24 None −2.24 5.76 2.44 0.59 −1.05
3. Nothing was interesting or fun Noth Fun 2.86 2nd −4.00 8.20 2.64 0.56 −0.10
4. Took extra effort to get started Extra Effort 3.62 None −3.62 6.38 2.76 0.18 −0.90
5. Felt slowed down Slow 2.96 None −2.96 7.04 3.18 0.53 −1.19
Positive Affect 1. Felt really “up” Up 2.38 2nd −3.45 3.67 1.75 0.13 −0.73
2. Felt really happy Happy 2.30 None −2.30 5.70 1.90 0.47 −0.43
3. Was having a lot of fun Fun 1.74 None −1.74 4.26 1.68 0.63 −0.49
4. Felt I had a lot of energy Energy 2.18 None −2.21 5.79 1.96 0.58 −0.40

Note. All residualized scales have Mean = 0 after polynomial detrending or, to facilitate comparisons, were set to 0 if not detrended. Subsequently
reported descriptives (Minimum, etc.) are on these scores. Detrending = the polynomial order of the best fitting model (e.g., 2nd = Week2 ).
Mean* = Mean for each item prior to detrending.

Statistical Analyses: Lag-1 Partial Correlation Matrix sparser network in which statistically unreliable param-
eter estimates are shrunk to zero. The corrected Akaike
Due to the number and distribution of missing observa- Information Criteria (AIC) was selected as the tuning
tions, neither vector autoregression (VAR) nor VAR- parameter to provide more conservative estimates of
based multiple imputation (iVAR; Liu & Molenaar the time series parameters given the ratio of estimated
,2014) could be utilized.2 Partial correlations were esti- parameters to sample size (Hurvich & Tsai 1989).
mated using dynamic time series multiple linear regres-
sion with maximum likelihood estimation in which each Centrality The network was further analyzed by calculating
symptom at time t was regressed on all other symptoms, three types of centrality indices: outdegree, indegree, and be-
including itself, at time t – 1. However, estimating a tweenness. Outdegree estimates how much information a
large number of parameters with relatively little data symptom sends directly to other symptoms (i.e., number of
can lead to spurious relations and large standard errors edges departing from the node), whereas indegree estimates
for the parameter estimates (i.e., the partial correlations). how much information a symptom receives directly from oth-
Thus, the partial correlation matrix was regularized er symptoms (i.e., number of edges arriving at the node).
using the least absolute shrinkage and selection operator Betweenness quantifies how much information passes
(LASSO; Friedman, Hastie, & Tibshirani, 2008) with through a given symptom by calculating the number of times
SAS PROC GLMSELECT. Regularization leads to a it lays on the shortest path between two nodes (Barrat,
Barthélemy, & Vespignani, 2008; Epskamp et al., 2012).

2 Community Structures Borsboom and Cramer (2013)

Multiple linear regression will yield similar parameter estimates to
those obtained using VAR but must be conducted separately for each theorized that comorbid disorders might manifest within
dependent variable. a network framework as distinct clusters for each construct
J Psychopathol Behav Assess

(i.e., disorder) with “bridge symptoms” (p. 97) connecting the originates: e.g., Discouraged) as a lagged incremental or
clusters. In complex networks, these types of clusters, or partial predictor of the outcome symptom (node to which
communities, may be identified as the within-cluster the edge points: e.g., Worry) over and above all other
nodes that relate more strongly to each other than those symptoms in the network. Note that this cross-lagged par-
outside of the community. To assess for such communi- tial relation is also controlling for autocorrelation: the
ties in the LASSO network, the spin glass method correlation of score on the same symptom at t – 1 and t
(Reichardt & Bornholdt, 2006) was utilized. (represented by the loop from Worry back to itself).
Modularity, a quantitative assessment of the number of Because these lag-1 relations are dynamic (yesterday to
edges within communities minus the number of edges today), it has been suggested that the edges provide a higher
one would expect to find using random data, was also degree of implied Granger causality than do concurrent rela-
estimated (Newman, 2006). Larger positive modularity tions (Bringmann et al., 2014; Granger, 1969). Granger (1969)
estimates suggest that the resulting community structures are suggested that, in time series analysis, predictive validity may
more reliable, whereas lower or negative modularity estimates be inferred when prior values of the independent variable are
suggest that separate communities may be spurious (Newman, shown to predict future values of the dependent variable.
2006; Reichardt & Bornholdt, 2006). Furthermore, this argument may be strengthened by control-
ling for the joint effects of other variables in the model via
partial correlations.
Results
Centrality of Symptoms
Of the possible 484 lag-1 relations, 433 were reduced to zero
by LASSO regularization. After regularization, one anhedonia Figure 2 displays the standardized results of the central-
symptom (“nothing was interesting or fun”) and two positive ity analyses for the regularized lag-1 partial correlation
affect symptoms (“felt really up”; “felt really happy”) network. Outdegree indices summarize how much infor-
displayed no functional dynamic relations to any other symp- mation a symptom sends directly to other symptoms.
toms in the network and were removed from the lag-1 partial For Sam, Tension (1.77), Worry (1.40), Trouble
correlation matrix. The 51 lag-1 functional relations are the Concentrating (1.31), and feeling Discouraged (1.22)
entries in the regularized lag-1 partial correlation matrix had the highest outdegree estimates, indicating that the
(see Appendix Table 2). level of these symptoms today had the strongest influ-
To construct the visual representation of the IDNA, ence on symptoms tomorrow, when controlling for the
the regularized lag-1 partial correlation matrix was en- influence of other symptoms.
tered into the qgraph package in R (Epskamp, Cramer, Indegree indices summarize how much information a
Waldorp, Schmittmann, & Borsboom, 2012). Figure 1 symptom receives directly from other symptoms. In Sam’s
shows the network of the 51 lag-1 functional relations lag-1 partial correlation network, Trouble Concentrating
contained in this matrix. The symptoms are represented (2.60) and feeling Uneasy (1.49) had the highest indegree
by nodes and the partial relations are represented by estimates. Three additional variables, Nothing Was
edges (i.e., line connecting two nodes). The thickness Enjoyable (1.04), Sad (0.95) and Extra Effort (0.87), also
of the edge indicates the strength of the relation, the had high indegree estimates compared to other variables in
color of the edge represents whether it is positive the network. Thus, when controlling for the influence of other
(green) or negative (red), and arrows represent direction- symptoms yesterday, the level of these five symptoms today
ality (i.e., the dynamic partial correlation at time t – 1 was most readily influenced by variability in other symptoms
(yesterday) to time t (today)). Nodes with stronger par- yesterday.
tial relations appear in the center of the network while Betweenness indices summarize the degree to which a
those with weaker associations appear on the periphery symptom lies on the shortest indirect path between two or
(Fruchterman & Reingold, 1991). more other symptoms (Opsahl, Agneessens, & Skvoretz,
For this network, edges represent the lag-1 partial cor- 2010). Symptoms with high betweenness may provide
relation between symptoms. For example, the arrow insight into the mechanisms or mediators that tend to fun-
pointing from Discouraged to Worry represents the rela- nel the dynamic flow of activation across symptoms in an
tion between Discouraged at time t – 1 (yesterday) and intraindividual network (Bringmann et al., 2013). For
Worry at time t (today), controlling for all other symp- Sam, Trouble Concentrating (2.62) and feeling Discouraged
toms, including Worry yesterday. By controlling for the (1.84) were frequently involved in the shortest lagged indirect
lagged influence of all other symptoms in the network, the connections between other pairs of symptoms within the net-
edge between two nodes represents the importance of an work. For example, Trouble Concentrating lies on one of the
originating symptom (node from which the edge shortest indirect paths between feeling Uneasy (an anxiety
 J Psychopathol Behav Assess

Fig. 1 Regularized Lag-1 Partial Correlation Network

symptom) and feeling that Nothing Was Enjoyable (a symp- communities within the network (Newman, 2006; Reichardt
tom of anhedonia). Worry (1.08) and feeling Uneasy (1.03) & Bornholdt, 2006). Thus, for Sam, the lag-1 relations be-
also had relatively high betweenness estimates. Betweenness tween symptoms generally regarded as indicative of “depres-
indices may also suggest possible “bridge symptoms” that sion” and “anxiety” do not appear to form distinct clusters, but
provide links between symptom clusters and explain the ar- rather, appear intermingled throughout the network.
chitecture of comorbidity in an individual, when indicated by
the community structure. Dynamic Processes in Sam’s Network

Community Structure In summary, the network analysis yields a dynamic pic-

ture of which symptoms are relatively more central ver-
The spin glass method (Reichardt & Bornholdt, 2006) was sus peripheral for Sam’s lag-1 dynamic partial relations
utilized to assess for distinct communities within the overall during the interval she made these daily ratings. Trouble
network. However, the modularity estimate of the resulting Concentrating played a central role for Sam in two re-
communities (−83.59) was negative. This indicates that any spects. First, (based on indegree, network strength, and
potential clusters are less cohesive than would be expected direction) Trouble Concentrating was greater today when she
using random data; overall, this suggests there are not distinct was Uneasy, Discouraged, and had Trouble Concentrating
J Psychopathol Behav Assess

Fig. 2 Centrality Plots for

Regularized Lag-1 Partial
Correlation Network

yesterday but, quite interestingly, was decreased mildly or A similar, but less strong, inferred feedback loop is present
moderately today when she was more Sad or Depressed, felt between Trouble Concentrating and Nothing Enjoyable.
like Nothing Was Enjoyable, Blamed herself, or when it took The stability or persistence of certain symptoms
more effort (Extra Effort) to get going yesterday, all when the through time is another important aspect of Sam’s lag-
effects of the other predictors yesterday were taken into ac- 1 partial correlation network. The strong persistence of
count.3 Second, (based on outdegree, network strength, and feeling Discouraged (indicated by the strong autoregressive
direction) when Sam had more Trouble Concentrating today loop indicating that yesterday’s score strongly predicts today’s
she reported greater anhedonia (Nothing Was Enjoyable), tak- score) and moderate persistence of Trouble Concentrating and
ing extra effort to get started (Extra Effort), feeling Slowed Worry, in combination with their relatively high centrality
Down, and having less Energy and greater Withdrawal tomor- indices, suggest that these three symptoms provide a basis
row. However, note the inferred feedback loop consisting of for stability in symptoms across time for Sam.
the positive partial lag-1 relation between greater Trouble
Concentrating today predicting Extra Effort tomorrow which,
in turn, predicts less Trouble Concentrating the following day. Discussion

3
The importance of the partial versus bivariate effect can be seen in Appendix This study utilized intensive, intraindividual, multivariate time-
Table 2 (which shows the partial correlations after regularization) and series data from an adult with comorbid mood and anxiety dis-
Appendix Table 3 (which shows the bivariate correlations for those relations orders to examine idiosyncratic relations among symptoms via a
retained after regularization). Note the sign reversal for yesterday’s Sad,
Depressed, and Nothing Was Enjoyable predicting today’s Trouble network analysis approach. Whereas a number of inves-
Concentrating between the bivariate and partial correlations. tigators have offered examples of different types of
 J Psychopathol Behav Assess

network analysis approaches utilizing cross-sectional or at time t, symptoms with high outdegree are possible key
hypothetical intraindividual data (Borsboom & Cramer, targets for individualized interventions due to their influence
2013), the present study expands upon the current liter- on other symptoms. Thus, Sam’s highest outdegree symp-
ature by using a network analysis approach to focus on toms, Tension, Worry, Trouble Concentrating, and feeling
the dynamic relations between symptoms, and across Discouraged, may be particularly useful targets for treatment
disorders, within a single individual. due to their dynamic influence on other symptoms in her net-
The lag-1 partial correlation matrix, when regularized work. Again note that the means over time for all four of these
and displayed as a network (Figure 1), provides a visual symptoms were very close to or within 1 SD below the mean
representation of the dynamic relations between symp- for the average rating of symptoms over time (See Table 1).
toms. Although these relations are not causal because Thus, as suggested above for Discouraged, the dynamic im-
the design was not experimental (Shadish, Cook, & portance of Tension, Worry, and Trouble Concentrating in
Campbell, 2002), they are based on temporal prece- influencing other symptoms a day later is not evident from
dence and imply a form of Granger causality. This their univariate descriptive statistics. That is, the IDNA pro-
empirically-derived representation of the dynamic rela- vides potentially important incremental validity for the case
tions between Sam’s symptoms provides a unique op- formulation.
portunity for a clinician to examine potential functional Interventions, including muscle relaxation and mindfulness
relations that exist in a complex case with comorbid exercises, to target Sam’s Tension Worry, and Trouble
diagnoses. Concentrating would likely have a positive impact on those
How might this empirically-derived network of dy- symptoms (Öst & Breitholz, 2000; Evans et al., 2008), as well
namic relations be interpreted in a clinically meaningful as on other symptoms in the network.
or useful way? The partial lag-1 network, such as the
one displayed in Figure 1, informs the clinician as to Indegree
which symptoms are most dynamically or temporally
influential in a complex symptom presentation. Therefore, this The indegree index summarizes how much information a
information can be utilized to guide treatment and focus inter- symptom receives directly from other symptoms. Thus, in
ventions in order to obtain a tailored or personalized interven- treatment settings, symptoms with high indegree may be use-
tion plan that is based on symptoms and relations that are ful indicators of change. For Sam, Trouble Concentrating, and
empirically most important for that particular person. feeling Uneasy, Sad, and that Nothing Was Enjoyable (symp-
Several principles are illustrated in the network displayed in toms with the highest indegree indices in her lag-1 partial
Figure 1. correlation network) are likely to be particularly relevant and
sensitive indicators of change in her overall network of
Stability symptoms.

First, at the center of the network, Sam feeling Discouraged Betweenness

today is a strong, positive predictor of her Worry and
Trouble Concentrating tomorrow. The strong, positive Betweenness indicates the degree to which a symptom
autoregressive loop for Discouraged indicates her feel- tends to funnel the flow of activation across symptoms.
ing Discouraged is relatively stable across time. Given Betweenness could provide insight into variables that
that Sam’s mean severity rating for Discouraged (2.07) mediate relations in an individual’s network. Such vari-
was very near the average for the 22 symptoms (2.04), ables, particularly if they have high outdegree, may be
the importance of Discouraged in terms of both influencing useful targets for intervention. For Sam, this would be
other symptoms and in terms of stability would not be evident Trouble Concentrating and feeling Discouraged. The po-
by simply examining the mean or variability in this rating. tential importance of these symptoms as mediators
Thus, targeting Sam’s discouragement in treatment may be would not be gleaned from their descriptive statistics
particularly effective in reducing other symptoms, such as (mean and SD of the ratings of each over time).
Sam’s Worry and Trouble Concentrating.
Comorbidity
Outdegree
Interestingly, Sam’s depression and anxiety symptoms
Because the outdegree index summarizes how much informa- do not form distinct clusters; but rather, appear intermingled
tion a symptom at time t – 1 sends directly to other symptoms throughout the network. This observation is supported by the
J Psychopathol Behav Assess

results of the community analysis, which suggested that the only persists on subsequent days but also predicts fur-
clusters in the network are less cohesive than would be ex- ther increases in Trouble Concentrating the following
pected using random data. The results for this specific day.
case, then, are contrary to Borsboom and Cramer’s Incremental validity in assessment, “the degree to which a
(2013) hypothesis that comorbid disorders might mani- measure explains or predicts some phenomena of interest,
fest within a network framework as distinct clusters relative to other measures”, is generally considered a neces-
with bridge symptoms connecting the clusters. This il- sary criterion for the addition of a new measurement instru-
lustrates the importance of empirically examining how an ment, or, as in the present case, assessment approach (Haynes
individual’s comorbid symptoms function and interact across & Lench, 2003, p. 457). At several points above, the advan-
time via an intraindividual dynamic network analysis tages of using the IDNA approach to display dynamic lead-lag
approach. relations (Figure 1) when combined with the various centrality
indices (Figure 2) in providing assessment-relevant informa-
Relations to Other Case-Formulation and Person-Specific tion over and above the mean of the scores over time
Longitudinal Approaches was emphasized. Specifically, the assessment importance
and potential treatment and measurement implications of
The visual representation of interrelations between vari- any of these symptoms (Discouraged, Tense, Uneasy,
ous symptoms relevant to a particular case has been Worry, and Trouble Concentrating) would not have been
previously suggested as a tool to systematize clinical evident from their univariate descriptive statistics. While
case formulation; for example, a functional analytic clin- the clinician should also include information on mean
ical case model (Haynes & O’Brien, 2000; Haynes et al., and trend in time series assessment as potentially im-
2011) or clinical pathogenesis map (Nezu & Nezu, 1989) portant information for the person’s case formulation,
may be used during case conceptualization. Mumma and col- the IDNA of Sam’s symptom scores provides prelimi-
leagues (Mumma, 2004, 2011; Mumma & Fluck, 2016; nary evidence for the incremental validity and utility of
Mumma & Mooney, 2007a, 2007b) have demonstrated how the IDNA approach.
intensive longitudinal ratings made by a client on idio-
graphic variables can be used to test the hypothesized Limitations and Future Directions
interrelations between variables in cognitive-behavioral
case formulations. Hypotheses are tested using person- Although the lag-1 partial correlation network offers
specific dynamic factor analysis, time-series regression, important information on dynamic symptom relations,
or correlations, to examine both concurrent and lead-lag several limitations of this type of intraindividual net-
relations. The network analysis approach described here- work should be noted. Because a lag-1 partial correla-
in is similar in that it uses the person-specific analysis tion network presents the unique effects of the predic-
of intensive longitudinal data (e.g., daily ratings) but dif- tors, the variance that is explained conjointly by multi-
fers in that functional relations are not necessarily hypothe- ple predictors (i.e., overlapping variance) is not evident
sized a priori. (Bulteel, Tuerlinckx, Brose, & Ceulemans, 2016).
Multiple symptoms may be functionally related – they
Advantages of the IDNA Approach could be either part of a functional stimulus class or a
functional response class (Haynes et al., 2011). If so,
One advantage of the network analysis approach de- their shared outdegree or indegree contributions could
scribed herein is that the dynamic relations between be evident in a lag-1 bivariate network but appear small
symptoms are estimated with corrections for chance, in the partial network, due to the small amount of
i.e., likely chance relations are re-estimated to zero unique variance explained. Therefore, some variables
(e.g., via LASSO regularization). A second advantage that are potentially important treatment targets may not ap-
of this approach is the possibility of statistically model- pear in a lag-1 partial correlation network due to statistical
ing a deepening severity of psychopathology or distress multicollinearity. Thus, we recommend examination of
(e.g., a “depressive spiral”) using the sequence of posi- the bivariate correlations for those predictors that are
tive lead-lag effects in the IDNA. For example, in retained after regularization (see Appendix Table 3).
Sam’s network, increased Trouble Concentrating today An important conceptual issue involves whether
may lead to feeling Slowed Down and that Nothing nodes represent potentially mutually interacting states
Was Enjoyable tomorrow, both of which lead to feeling or processes that are conceptually, behaviorally, or
Discouraged the next day. Feeling Discouraged then not neurofunctionally discrete versus similar response
 J Psychopathol Behav Assess

classes for that person. For example, Figure 1 shows an Treatment utility of IDNA assessment. To evaluate how
inferred dynamical feedforward loop involving Sam useful or helpful IDNA is for tailored treatment
feeling Nervous today leading to feeling On Edge to- planning, IDNA data could be collected for all pa-
morrow which in turn increases feeling Nervous the tients but only be made available to or utilized by
following day. However, it’s unclear for this individual some clinicians (e.g., one group receives the results
if these nodes are indeed distinctive states or processes, of the IDNA whereas the other does not).
slightly different descriptors of highly similar internal Comparisons in such a manipulated assessment de-
states, or related components of a more inclusive state sign (Hayes et al., 1987) might include idiographic
or process. as well as standardized outcome measures (e.g.,
Another important issue in network analysis is robustness Haynes, Mumma, & Pinson, 2009; Weisz et al.,
of the network. The network described herein displays a large 2011).
number of network connections; thus, some of these results Ecological momentary assessment. Important issues in
are likely to be false positives. Although regularization IDNA include the frequency of the dynamic assess-
methods decrease the likelihood of spurious findings, the de- ment, the rating interval (e.g., right now versus
velopment of parametric and nonparametric reliability analy- since the last rating), and the ecological or situa-
sis is warranted. ti on al em be dd ed ne ss of th es e as ses sme nt s.
The network analysis for this study was conducted Considerations for such decisions as applied to time
after the participant had completed treatment. Future series analysis and EMA have been described in a
studies could utilize the IDNA approach prior to and number of sources (e.g., Haynes, O’Brien, &
during treatment as a statistically-based decision-making Kaholokula, 2011).
aid for personalized treatment planning. Collecting mul-
tivariate time series data during treatment can assist in In conclusion, an IDNA using lag-1 partial correla-
adjusting interventions administered over time and may tions allows clinicians to visually examine, based on the
improve treatment outcome (Jacobson, Follette, & strength and direction of relation between symptoms,
Revenstorf, 1984). Future studies may examine out- which symptoms are central to the dynamic relations
comes based on interventions tailored to target individ- among symptoms for that individual. As with any
ualized symptoms and processes utilizing larger sample intraindividual analysis of time series data for a single
sizes with an experimental design (Hayes, Nelson, & participant, the results and inferences (e.g., as described
Jarrett, 1987; Nelson-Gray, 2003). above for Sam’s network) are not generalizable to other
Future research on IDNA in clinical assessment may con- individuals or even necessarily to dynamic symptom
sider several issues: interrelations within that person at different time periods
or situations in his or her life. Yet it is exactly this
Additional causal variables. Although the current study person-specific quality that contributes to the potential
did not include causal variables such as triggers, cogni- incremental clinical utility of the IDNA approach for
tions, or behaviors that may impact relations between personalized treatment decisions in a complex or comor-
symptoms, future studies could include such causal vari- bid case. This information may be utilized to guide
ables in the IDNA. Furthermore, future studies could in- treatment and focus interventions in order to obtain a tai-
clude transdiagnostic constructs and processes (e.g., in- lored or personalized intervention plan that is based on those
tolerance of uncertainty, anxiety sensitivity, perfection- symptoms and relations that are empirically important for that
ism) to examine their influence within an intraindividual person.
network and to obtain a more detailed depiction of poten-
tial functional variables contributing to an individual’s Compliance with Ethical Standards
distress.
Dynamic complexity and incremental validity of Ethical Approval All procedures performed in studies involving hu-
man participants were in accordance with the ethical standards of the
assessment. Larger sample research could estimate the institutional and/or national research committee and with the 1964
proportion of cases that an IDNA analysis, particularly Helsinki declaration and its later amendments or comparable ethical
when using the centrality indices of betweenness, standards.
indegree, and outdegree, provide incremental information
over and above simple data summaries (e.g., mean). Such Informed Consent Informed consent was obtained from all individual
participants included in the study.
research could evaluate if the extent of incremental valid-
ity varies with types of disorders, complexity of the case
Conflict of Interest Sarah Jo David, Andrew J. Marshall, Emma K.
(e.g., number of comorbid diagnoses), or other patient Evanovich, and Gregory H. Mumma declare that they have no
characteristics. conflict of interest.
Appendix
J Psychopathol Behav Assess

Table 2

Item Sad Depressed Discourage Disappt Blame Nervous Tense Uneasy Un relax On edge Worry Trb Conc Confused Not enjoy Withdraw Extra Effort Slow Fun Energy

Sad 0 0 0 0 0 0 0 0 0 0 0 −0.099 0 0 0 0 0 0 0
Depressed 0 0 0 0 0 0 0 −0.144 0 0 0 −0.054 0 0 0 0 0 0 0
Discourage 0.095 0 0.348 0 0 0 0 0 0 0 0.232 0.206 0 0 0 0 0 0 0
Disappt 0 0 0 0 0 0 0 0.06 0 0 0 0 0 0 0 0 0 0 0
Blame −0.084 0 −0.222 0 0 0 0 0 0 0 0 −0.104 0 0 0 0 0 0 0
Nervous 0 0 0 0 0 0 0 0 0 0.179 0 0 0 0 0 −0.196 0 0 0
Tense 0.227 0 0 0 0 0 0.107 0.227 0 0 0 0 0 0.075 0.113 0 0 0 0
Uneasy 0 0 0 0 0 0 0.059 0 0 0 0 0.322 0 0 0 0 0 0 0
Un relax 0 0 0.071 0 0 0 0 0 0.101 0 0 0 0 0 0 0 0 0 0
On edge 0 0 0 0 0 0.188 0 0 0 0 0 0 0 0 0 0 0 0 0
Worry 0 0 0 0 0 0.127 0.046 0.226 0.076 0 0.143 0 0 0 0 −0.094 0 0 0
Trb Conc 0 0 0 0 0 0 0 0 0 0 0 0.163 0 0.164 0.062 0.184 0.094 0 −0.048
Confused 0 0 0 0 0 0 0 0 0 0 0 0 0.088 −0.161 0 0.035 0 0 0
Not Enjoy 0.191 0 0.068 0 0 0 0 0 0 0 0 −0.057 0 0 0 0 0 0 0
Withdraw 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Extra effort 0 0 0 0 0 0 0 0 0 0 0 −0.132 0 −0.051 0 0 0 0 0
Slow 0 0 0.144 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Fun 0 0 0 0 0 0 0 −0.117 0 0 0 0 0 0 0 0 0 0 0
Energy 0 0 −0.014 0 0 0 0 0 0 0 0 −0.159 0 −0.178 0 −0.063 −0.074 0 0.101

Note. Regularized Lag-1 Partial Correlation Matrix. For each analysis, today’s score, the dependent variable, is the top row. Example: The partial correlation of today’s (t) Sad score with yesterday’s (t – 1)
Discourage score was 0.095 and with yesterday’s Tense score was 0.227, whereas the correlation of today’s Tense score with yesterday’s Sad score was 0 (after regularization).
Table 3

Item Sad Depressed Discourage Disappt Blame Nervous Tense Uneasy Un relax On edge Worry Trb Conc Confused Not Enjoy Withdraw Extra effort Slow Fun Energy

Sad 0 0 0 0 0 0 0 0 0 0 0 0.132 0 0 0 0 0 0 0
Depressed 0 0 0 0 0 0 0 0.214 0 0 0 0.077 0 0 0 0 0 0 0
Discourage 0.359 0 0.519 0 0 0 0 0 0 0 0.451 0.298 0 0 0 0 0 0 0
Disappt 0 0 0 0 0 0 0 0.285 0 0 0 0 0 0 0 0 0 0 0
Blame −0.052 0 −0.189 0 0 0 0 0 0 0 0 −0.006 0 0 0 0 0 0 0
Nervous 0 0 0 0 0 0 0 0 0 0.368 0 0 0 0 0 −0.272 0 0 0
Tense 0.424 0 0 0 0 0 0.369 0.439 0 0 0 0 0 0.172 0.298 0 0 0 0
Uneasy 0 0 0 0 0 0 0.365 0 0 0 0 0.377 0 0 0 0 0 0 0
Un relax 0 0 0.290 0 0 0 0 0 0.363 0 0 0 0 0 0 0 0 0 0
On edge 0 0 0 0 0 0.4 0 0 0 0 0 0 0 0 0 0 0 0 0
Worry 0 0 0 0 0 0.369 0.326 0.428 0.348 0 0.39 0 0 0 0 −0.247 0 0 0
Trb Conc 0 0 0 0 0 0 0 0 0 0 0 0.356 0 0.251 0.255 0.203 0.283 0 −0.258
Confused 0 0 0 0 0 0 0 0 0 0 0 0 0.259 −0.169 0 0.133 0 0 0
Not enjoy 0.426 0 0.368 0 0 0 0 0 0 0 0 0.079 0 0 0 0 0 0 0
Withdraw 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Extra effort 0 0 0 0 0 0 0 0 0 0 0 −0.113 0 −0.102 0 0 0 0 0
Slow 0 0 0.360 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Fun 0 0 0 0 0 0 0 −0.239 0 0 0 0 0 0 0 0 0 0 0
Energy 0 0 −0.277 0 0 0 0 0 0 0 0 −0.179 0 −0.265 0 −0.228 −0.262 0 0.292

Note. Regularized Lag-1 Bivariate Correlation Matrix.

J Psychopathol Behav Assess
J Psychopathol Behav Assess

Open Access This article is distributed under the terms of the Creative anxiety disorder. Journal of Anxiety Disorders, 22, 716–721.
Commons Attribution 4.0 International License (http:// https://doi.org/10.1016/j.janxdis.2007.07.005.
creativecommons.org/licenses/by/4.0/), which permits unrestricted use, First, M. B., Spitzer, R. L., Gibbon, M., & Williams, J. B. W. (1997).
distribution, and reproduction in any medium, provided you give Structured clinical interview for DSM–IV Axis I disorders.
appropriate credit to the original author(s) and the source, provide a link Washington, DC: American Psychiatric Association.
to the Creative Commons license, and indicate if changes were made. Friedman, J., Hastie, T., & Tibshirani, R. (2008). Sparse inverse covari-
ance estimation with the graphical lasso. Biostatistics, 9, 432–441.
https://doi.org/10.1093/biostatistics/kxm045.
Fruchterman, T., & Reingold, E. (1991). Graph drawing by force-directed
placement. Software - Practice and Experience, 21, 1129–1164.
References Granger, C. J. (1969). Investigating causal relations by econometric
models and cross-spectral methods. Econometrica, 37, 424–438.
Hayes, S. C., Nelson, R. O., & Jarrett, R. B. (1987). The treatment utility
Barlow, D. H., Bullis, J. R., Comer, J. S., & Armetaj, A. A. (2013). of assessment: A functional approach to evaluating assessment qual-
Evidence-based psychological treatments: An update and a ity. American Psychologist, 42, 963–974.
way forward. Annual Review Clinical Psychology, 9, 1–27. Haynes, S. N., & Lench, H. C. (2003). Incremental validity of new clin-
https://doi.org/10.1146/annurev-clinpsy-050212-185629. ical assessment measures. Psychological Assessment, 15, 456–466.
Barrat, A., Barthélemy, M., & Vespignani, A. (2008). Dynamical process- https://doi.org/10.1037/1040-3590.15.4.456.
es on complex networks. New York: Cambridge University Press. Haynes, S. N., & O’Brien, W. O. (2000). Principles of behavioral assess-
Beck, A. T., & Steer, R. A. (1993). Beck anxiety inventory: Manual. San ment: A functional approach to psychological assessment. New
Antonio: Psychological Corporation. York: Plenum/Kluwer Press.
Beck, A. T., Weissman, A., Lester, D., & Trexler, L. (1974). The mea- Haynes, S. N., Mumma, G. H., & Pinson, C. (2009). Idiographic assess-
surement of pessimism: The hopelessness scale. Journal of ment: Conceptual and psychometric foundations of individualized
Consulting and Clinical Psychology, 42, 861–865. https://doi.org/ behavioral assessment. Clinical Psychology Review, 29, 179–191.
10.1037/h0037562. https://doi.org/10.1016/j.cpr.2008.12.003.
Beck, A., Steer, R., & Brown, G. (1996). Beck depression inventory Haynes, S. N., O’Brien, W. H., & Kaholokula, J. K. (2011). Behavioral
manual (2nd ed.). San Antonio: The Psychological Association. assessment and case formulation. Hoboken: Wiley.
Borsboom, D., & Cramer, A. J. (2013). Network analysis: An integrative Hurvich, C. M., & Tsai, C. (1989). Regression and time series
approach to the structure of psychopathology. Annual Review of model selection in small samples. Biometrika, 76, 297–307.
Clinical Psychology, 9, 91–121. https://doi.org/10.1146/annurev- https://doi.org/10.2307/2336663.
clinpsy-050212-185608. Jacobson, N. S., Follette, W. C., & Revenstorf, D. (1984). Psychotherapy
Bringmann, L. F., Vissers, N., Wichers, M., Geschwind, N., Kuppens, P., outcome research: Methods for reporting variability and evaluating
Peeters, F., et al. (2013). A network approach to psychopathology: clinical significance. Behavior Therapy, 15, 336–352.
New insights into clinical longitudinal data. PLoS One, 8, e60188. Kazdin, A. E. (2017). Research design in clinical psychology (5th ed.).
https://doi.org/10.1371/journal.pone.0060188. Boston: Pearson.
Bringmann, L. F., Lemmens, L. M., Huibers, M. H., Borsboom, D., & Kouros, C. D., Quasem, S., & Garber, J. (2013). Dynamic temporal
Tuerlinckx, F. (2014). Revealing the dynamic network structure of relations between anxious and depressive symptoms across ado-
the Beck Depression Inventory-II. Psychological Medicine, 45, lescence. Development and Psychopathology, 25, 683–697.
747–757. https://doi.org/10.1017/S0033291714001809. https://doi.org/10.1017/S0954579413000102.
Brown, T. A., Campbell, L. A., Lehman, C. L., Grisham, J. R., & Mancill, Lei, H., Nahum-Shani, I., Lynch, K., Oslin, D., & Murphy, S. A. (2012).
R. B. (2001). Current and lifetime comorbidity of the DSM-IV anx- A 'SMART' design for building individualized treatment sequences.
iety and mood disorders in a large clinical sample. Journal of Annual Review of Clinical Psychology, 8, 21–48. https://doi.org/10.
Abnormal Psychology, 110, 585–599. https://doi.org/10.1037/ 1146/annurev-clinpsy-032511-143152.
0021-843X.110.4.585. Liu, S., & Molenaar, P. M. (2014). iVAR: A program for imputing miss-
ing data in multivariate time series using vector autoregressive
Bulteel, K., Tuerlinckx, F., Brose, A., & Ceulemans, E. (2016). Using raw
models. Behavior Research Methods, 46, 1138–1148. https://doi.
VAR regression coefficients to build networks can be misleading.
org/10.3758/s13428-014-0444-4.
Multivariate Behavioral Research, 51, 330–344. https://doi.org/10.
Molenaar, P. M. (2004). A manifesto on psychology as idiographic sci-
1080/00273171.2016.1150151.
ence: Bringing the person back into scientific psychology, this time
Cole, D. A., Peeke, L. G., Martin, J. M., Truglio, R., & Seroczynski, A. D. forever. Measurement: Interdisciplinary Research and Perspectives,
(1998). A longitudinal look at the relation between depression and 2, 201–218. https://doi.org/10.1207/s15366359mea0204_1.
anxiety in children and adolescents. Journal of Consulting and Mumma, G. H. (2004). Validation of idiographic cognitive schema in
Clinical Psychology, 66, 451–460. https://doi.org/10.1037/0022- cognitive case formulations: An intra-individual idiographic ap-
006X.66.3.451. proach. Psychological Assessment, 16, 211–230.
Cramer, A. J., Waldorp, L. J., van der Maas, H. J., & Borsboom, Mumma, G. H. (2011). Validity issues in cognitive-behavioral case for-
D. (2010). Comorbidity: A network perspective. Behavioral mulation. European Journal of Psychological Assessment, 27, 29–
and Brain Sciences, 33, 137–150. https://doi.org/10.1017/ 49. https://doi.org/10.1027/1015-5759/a000054.
S0140525X09991567. Mumma, G. H., & Fluck, J. (2016). How valid is your case formulation?
Eells, T. D. (2007). History and current status of psychotherapy case Empirically testing your cognitive behavioral case formulation for
formulation. In T. D. Eells (Ed.), Handbook of psychotherapy case tailored treatment. The Cognitive Behaviour Therapist, 9, 1–25.
formulation (pp. 3–32). New York: Guilford Press. Mumma, G. H., & Mooney, S. R. (2007a). Comparing the validity of
Epskamp, S., Cramer, A. O. J., Waldorp, L. J., Schmittmann, V. D., & alternative cognitive case formulations: A latent variable,
Borsboom, D. (2012). Qgraph: Network visualizations of relation- multivariate time series approach. Cognitive Therapy &
ships in psychometric data. Journal of Statistical Software, 48, 1–18. Research, 31, 451–481.
Evans, S., Ferrando, S., Findler, M., Stowell, C., Smart, C., & Haglin, D. Mumma, G. H., & Mooney, S. R. (2007b). Incremental validity of cog-
(2008). Mindfulness-based cognitive therapy for generalized nitions in a clinical case formulation: An intraindividual test in a
 J Psychopathol Behav Assess

case example. Journal of Psychopathology and Behavioral Reichardt, J., & Bornholdt, S. (2006). Statistical mechanics of com-
Assessment, 29, 17–28. munity detection. Physical Review E (Statistical, Nonlinear, and
Nelson-Gray, R. O. (2003). Treatment utility of psychological assess- Soft Matter Physics), 74, 016110. doi:https://doi.org/10.1103/
ment. Psychological Assessment, 15, 521–531. https://doi.org/10. PhysRevE.74.016110.
1037/1040-3590.15.4.521. Rizvi, S. L., & Harned, M. S. (2013). Increasing treatment efficiency and
Newman, M. J. (2006). Modularity and community structure in networks. effectiveness: Rethinking approaches to assessing and treating co-
Proceedings of the National Academy of Sciences of the United morbid disorders. Clinical Psychology: Science and Practice, 20,
States of America, 103, 8577–8582. https://doi.org/10.1073/pnas. 285–290. https://doi.org/10.1111/cpsp.12040.
0601602103. Robinaugh, D. J., LeBlanc, N. J., Vuletich, H. A., & McNally, R. J.
Nezu, A. M., & Nezu, C. M. (1989). Clinical decision making in (2014). Network analysis of persistent complex bereavement disor-
behavior therapy: A problem solving perspective. Champaign: der in conjugally bereaved adults. Journal of Abnormal Psychology,
Research Press. 123, 510–522. https://doi.org/10.1037/abn0000002.
Opsahl, T., Agneessens, F., & Skvoretz, J. (2010). Node centrality in Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and
weighted networks: Generalizing degree and shortest paths. Social quasi-experimental designs for generalized causal inference. New
Networks, 32, 245–251. York: Houghton Mifflin.
Öst, L. G., & Breitholz, E. (2000). Applied relaxation versus cognitive Watson, D., Weber, K., Assenheimer, J. S., Clark, L. A., Strauss, M. E., &
therapy in the treatment of generalized anxiety disorder. Behaviour McCormick, R. A. (1995). Testing a tripartite model: I. Evaluating
Research and Therapy, 38, 777–790. https://doi.org/10.1016/ the convergent and discriminant validity of anxiety and depression
S0005-7967(99)00095-9. symptom scales. Journal of Abnormal Psychology, 104, 3–14.
Persons, J. B. (2008). What is the case formulation approach to cognitive- https://doi.org/10.1037/0021-843X.104.1.3.
behavioral therapy? In The case formulation approach to cognitive- Weisz, J. R., Chorpita, B. F., Frye, A., Ng, M. Y., Lau, N., Bearman, S. K.,
behavior therapy (pp. 1–16). New York: Guilford Press. et al. (2011). Youth top problems: Using idiographic, consumer-
R Development Core Team. (2010). R: A language and environment for guided assessment to identify treatment needs and to track change
statistical computing. Vienna: R Foundation for Statistical during psychotherapy. Journal of Consulting and Clinical
Computing Retrieved from http://www.R-project.org. Psychology, 79, 369–380. https://doi.org/10.1037/a0023307.