OPINION

published: 28 June 2016

doi: 10.3389/fnhum.2016.00309

Stabilizing Constructs through

Collaboration across Different
Research Fields as a Way to Foster
the Integrative Approach of the
Research Domain Criteria (RDoC)
Project
Jacqueline A. Sullivan *

Department of Philosophy, Rotman Institute of Philosophy, University of Western Ontario, London, ON, Canada

Keywords: construct, DSM, experimental paradigm, integration, RDoC, validity

INTRODUCTION
More than 450 million people worldwide suffer from neuropsychiatric disorders and the numbers
continue to rise (WHO, 2016). In 2010, aiming to solve the global mental health crisis and
advance psychiatry toward a precision medicine approach, the US National Institute for Mental
Health (NIMH) initiated the Research Domain Criteria (RDoC) Project (Cuthbert and Insel, 2013).
Scientists at NIMH importantly recognize that understanding and explaining psychopathological
Edited by: phenomena requires input from different areas of science that investigate the role of different units
Mikhail Lebedev,
of analysis (e.g., genes, cells, systems) in the production of organism-level behavioral functions.
Duke University, USA
The RDoC matrix is put forward as a context for integrating results from these different sciences
Reviewed by: into a taxonomy of putatively valid constructs they purportedly share in common. It is intended
Elizabeth B. Torres,
to facilitate the development of “integrative psychobiological explanations” of those behavioral
Rutgers University, USA
Amilcar Silva Dos Santos,
functions designated by the constructs (Cuthbert and Kozak, 2013, p. 931; See also Sanislow et al.,
Hospital Vila Franca de Xira, Portugal 2010). Such explanations, by shedding light on the mechanisms of these functions, will enable
investigators to pinpoint viable targets for therapeutic intervention in cases in which these functions
*Correspondence:
Jacqueline A. Sullivan are disrupted.
[email protected] The RDoC project is still in its infancy and its proponents recognize that it has much room
for improvement (Casey et al., 2014). To date, it has been criticized for being “braincentric” and
Received: 05 May 2016 decontextualizing mental disorders from their bodily, social, and environmental contexts (e.g.,
Accepted: 08 June 2016 Whooley, 2014; Bernard and Mittal, 2015). Although proponents of RDoC claim that one of its
Published: 28 June 2016 crucial aims is to integrate various areas of science, an obstacle to this integration is the lack
Citation: of construct stability in psychology and neuroscience. In this article, I explain why stabilizing
Sullivan JA (2016) Stabilizing constructs is important to the success of the RDoC initiative and identify one measure for
Constructs through Collaboration facilitating such stability.
across Different Research Fields as a
Way to Foster the Integrative
Approach of the Research Domain THE RDoC MATRIX
Criteria (RDoC) Project.
Front. Hum. Neurosci. 10:309. The RDoC Matrix consists of a table in which findings from psychology and neuroscience may
doi: 10.3389/fnhum.2016.00309 be organized. Five broad domains of behavioral functioning are identified in the rows of the first

Frontiers in Human Neuroscience | www.frontiersin.org 1 June 2016 | Volume 10 | Article 309

Sullivan Stabilizing Constructs across Research Fields

FIGURE 1 | Schematic representation of different areas of science contributing collectively to construct stabilization, which facilitates explanatory
integration and the development of integrated valid constructs. Some examples of current RDoC constructs are identified in the figure in plain typeface.
Constructs currently not represented in the RDoC matrix and others to be added are identified in boldface.

column of the matrix: (1) positive, (2) negative valence systems, The constructs are essentially labels that are linked up with
(3) cognitive, (4) social processing, and (5) arousal/modulatory (1) experimental paradigms used to investigate the behavioral
systems. A selection of constructs designating some behavioral functions designated by those labels and (2) units of analysis that
functions currently under study across psychology and have been implicated in the production of behavioral functions
neuroscience are identified and classified under one of each designated by those labels. Although proponents of RDoC
of the 5 domains. For example, attention, perception, declarative remain vague about the details, at some point research findings
memory, language, cognitive control, and working memory are inputted into the matrix are supposed to result in integrative
classified as cognitive systems. Negative valence systems include psychobiological or mechanistic explanations that describe
the constructs of acute threat, potential threat, sustained threat, the physical components and processes that bring about the
loss, and frustrative nonreward. Each construct is also given a functions designated by RDoC constructs (Cuthbert and Kozak,
general definition. In most cases, these definitions make reference 2013). Knowledge about mechanisms is in turn supposed to
to neural, psychological, and behavioral processes associated foster the development of therapeutic interventions in cases
with the construct. For example, fear is characterized as in which behavioral functions are disrupted, with the aim of
involving “activation of the brain’s defensive motivational system restoring normal functioning in those individuals.
to promote” protective behaviors (i.e., neural processes), “a
pattern of adaptive responses to conditioned and unconditioned
threat stimuli” (i.e., behavioral/psychological processes) and OBSTACLES TO INTEGRATION
possibly “internal representations and cognitive processing” (i.e.,
psychological processes) (http://www.nimh.nih.gov/research- We should not expect integration to be an emergent feature of
priorities/rdoc/constructs/acute-threat-fear.shtml). the juxtaposition of a mass of research findings emanating from
The columns of the matrix reflect the fact that research psychology and neuroscience. The explanatory and conceptual
on domains of behavioral functioning spans multiple levels of integration that RDoC is supposed to deliver instead requires
organization from genes to cells to networks to behavior and intensive collaborative efforts on the part of investigators
includes multiple different areas of science (e.g., psychology, working in areas of science that investigate RDoC constructs.
systems neuroscience, and neurobiology). An additional column To understand why, a closer look at explanatory, conceptual
labeled “paradigms” is where experimental paradigms, standard and investigative practices in psychology and neuroscience is
procedures for producing, detecting and measuring behavioral relevant.
functions that correspond to the constructs in the matrix, Explanations in psychology have been characterized as
are placed. The Stroop Task is an experimental paradigm explanations by functional analysis (Fodor, 1968; Cummins,
historically used to investigate selective attention in human 1983). These explanations involve ascriptions of functions to
subjects. Fear-conditioning paradigms, in contrast, are used to organisms and the abstract identification of the sub-capacities
study fear in humans and non-human mammals. that bring these functions about without regard for anatomical,
How will RDoC facilitate progress in understanding structural, biochemical, or physiological facts about the brain and
disturbances in behavioral functioning? Investigators working nervous system. They often consist of box-and-arrow diagrams
in those sciences represented in the matrix use experimental where boxes stand in for psychological capacities and arrows
paradigms to produce, detect and measure instances of represent information flow through the system from stimulus
behavioral functions that correspond to RDoC constructs. inputs to behavioral outputs. An explanation of attention by

Frontiers in Human Neuroscience | www.frontiersin.org 2 June 2016 | Volume 10 | Article 309

Sullivan Stabilizing Constructs across Research Fields

functional analysis, for example, may describe a “short term paradigms. This ought to mean that intra-lab strategies are
storage” system that receives information from the periphery in place to ensure that experimental paradigms measure
and sends it through a “selective filter” that determines what the functions they are intended to measure, which ought to
information is received by systems downstream (Broadbent, contribute positively to construct stabilization. While this
1958). Explanations in neuroscience, in contrast, are described is true, inter-lab practices are not necessarily conducive to
as mechanistic insofar as they identify the physical parts (e.g., stability. For example, two investigators may be interested
systems, cells, molecules) and processes (e.g., activation, firing, in investigating the same function, but disagree about the
phosphorylation) that realize organism-level functions (Craver, most suitable task for this purpose. Since stimuli and task
2007; Bechtel, 2008). Part of a mechanistic explanation of demands may differ radically between tasks, it is difficult to
attention, for example, may describe activation of dopamine determine if the same component cognitive processes are
receptors and depolarization of medium spiny neurons in the involved in each task and whether they measure the same thing
nucleus accumbens of the basal forebrain. (Sullivan, 2009; Lilienfeld, 2014; Lilienfeld and Treadway, 2016).
A prerequisite for integrating explanations by functional Investigators often disagree about which component cognitive
analysis with mechanistic explanations is “connectability” processes are involved in a task and the behavioral data often are
(Nagel, 1961). More specifically, the terms designating cognitive compatible with multiple different explanations by functional
capacities in an explanation by functional analysis must have analysis.
roughly the same referents as the terms designating cognitive Although we may be optimistic that cognitive neuroscience
capacities in a mechanistic explanation. To refer back to will provide the fMRI or other brain data requisite for
the previous example, an explanation by functional analysis resolving such problems of underdetermination, it has its own
that contains the term attention ought to refer to the same troubles with respect to construct stabilization. Many cognitive
phenomenon as a mechanistic explanation that contains that neuroscientists do not aim to identify the component cognitive
term. Terms designating cognitive capacities are the common processes thought to be engaged in experimental tasks nor to
denominator between the two forms of explanation and determine how the variables manipulated in an experiment affect
satisfying the connectability condition requires that the terms these processes (Sullivan, 2014a,b). In fact, when evaluating or
designate the same thing. Otherwise, rather than explanatory comparing tasks it is more common for investigators to rely on
integration, terms in one area of science are eliminated and “intuitive judgments” about the processes involved rather than
replaced with those of another. look to “formal theories from cognitive psychology” for guidance
The RDoC task forces implicitly realize that satisfaction of the (Poldrack, 2010, p. 149).
connectability condition is required for explanatory integration, We also find rampant methodological pluralism in those
as is evidenced by the fact that they have sought to deploy areas of cellular and molecular neuroscience that investigate
strategies to stabilize RDoC constructs, while simultaneously cognition and behavior. Individual researchers vary experimental
acknowledging that the constructs are heuristics that may paradigms and protocols used to produce, measure and
warrant revision in light of future discovery (Cuthbert and Insel, detect behavioral functions. Yet even subtle differences
2013). However, stabilizing constructs requires more than a small in experimental protocols can impact the mechanisms
group of investigators selecting a set of terms that are used across productive of those functions, prompting uncertainty as to
different areas of science and providing them with definitions whether different laboratories are investigating the same
broad enough to accommodate different uses of these terms. As phenomenon. In some cases, investigators may be unclear what
RDoC’s creators acknowledge, it is not a project that will prove function they are discovering the mechanisms of (Sullivan,
ultimately successful if it isolates itself from those very sciences 2010), but given that they do not regard understanding
poised to shed light on the kinds of questions it is designed component cognitive processes as relevant to their mechanistic
to answer (e.g., Simmons and Quinn, 2014). RDoC’s success is explanatory goals, they see no need to look toward psychology
instead contingent on a large-scale revolution in the mind-brain for guidance.
sciences to collectively stabilize constructs so that conceptual
and explanatory integration are possible. One component of this
revolution has to be coordination across investigators working CONCLUSION
in the same and different areas of science to come to specific
agreement about (1) how to define terms designating behavioral If construct instability across psychology and neuroscience
functions, (2) what the best experimental paradigms for studying is as pervasive as these facts about practice suggest, then
a given behavioral function are, and (3) when two experimental stabilizing them for the purposes of conceptual and explanatory
paradigms may be said to produce, detect and measure roughly integration is going to require scientists engaged in research
the same function. relevant to investigating the domains of functioning identified
Do we encounter such coordination within or across in the matrix—including research on domains currently and
psychology and neuroscience currently? Let’s begin by problematically absent from the matrix, like motor functions
considering cognitive psychology. One of its paradigmatic (Bernard and Mittal, 2015)—to interact with each other in the
features is the importance placed on engaging in rigorous trenches to do hard work (Bilder et al., 2013). To date, RDoC has
task analyses to determine the component cognitive processes pointed to large-scale meta-analyses as a primary way forward.
operative when subjects are trained and tested in experimental However, insofar as meta-analyses abstract away from conceptual

Frontiers in Human Neuroscience | www.frontiersin.org 3 June 2016 | Volume 10 | Article 309

Sullivan Stabilizing Constructs across Research Fields

and experimental practices operative within and across different example http://www.rotman.uwo.ca/events-2/rethinking-the-tax
areas and laboratories in psychology and neuroscience, they will onomy-of-psychology-conference/). This is the RDoC model on
not yield valid constructs. Details about features of individual a larger scale and with greater inclusivity, but only by means
experiments matter for comparing data across laboratories and of broader collaborative efforts may we hope to ensure the
determining if the same capacities are under study and the same realization of RDoC’s positive aims.
mechanisms are operative. While amassing discordant evidence
under a set of common labels may result in testable hypotheses it AUTHOR CONTRIBUTIONS
will not directly shed light on real divisions in the causal structure
of the world. The author confirms being the sole contributor of this work and
RDoC’s success requires instead deliberate efforts across approved it for publication.
the relevant sciences and humanities to collectively stabilize
its constructs (Figure 1). One plausible way forward is to ACKNOWLEDGMENTS
create networks of investigators representing a diverse array
of perspectives on behavioral and psychological functions The author would like to thank two anonymous reviewers for
and regularly bring them together to facilitate discussions very helpful and extensive comments on this manuscript. It is
about what the relevant constructs are, how to investigate much improved from the original version as a result of their
them, how to stabilize them, and related issues (See for feedback and encouragement.

REFERENCES Mapping, eds S. Hanson and M. Bunzl (Cambridge: MIT Press),

147–159.
Bechtel, W. (2008). Mental Mechanisms: Philosophical Perspectives on Cognitive Sanislow, C. A., Pine, D. S., Quinn, K., Kozak, M., Garvey, M., Heinssen, R., et al.
Neuroscience. New York, NY: Taylor and Francis. (2010). Developing constructs for psychopathology research: research domain
Bernard, J. A., and Mittal, V. A. (2015). Updating the research domain criteria. J. Abnorm. Psychol. 119, 631–639. doi: 10.1037/a0020909
criteria: the utility of a motor dimension. Psychol. Med. 45, 2685–2689. doi: Simmons, J., and Quinn, K. (2014). The NIMH Research Domain Criteria (RDoC)
10.1017/S0033291715000872 Project: implications for genetics research. Mamm. Genome 25, 23–31. doi:
Bilder, R., Howe, A., and Sabb, F. (2013). Multilevel models from biology 10.1007/s00335-013-9476-9
to psychology: mission impossible? J. Abnorm. Psychol. 122, 917–927. doi: Sullivan, J. (2009). The multiplicity of experimental protocols: a challenge to
10.1037/a0032263 reductionist and nonreductionist models of the unity of science. Synthese 167,
Broadbent, D. (1958). Perception and Communication. London: Pergamon Press. 511–539. doi: 10.1007/s11229-008-9389-4
Casey, B. J., Oliveri, M. E., and Insel, T. (2014). A neurodevelopmental perspective Sullivan, J. (2010). Reconsidering spatial memory and the morris water maze.
on the research domain criteria (RDoC) framework. Biol. Psychiatry 76, Synthese 177, 261–283. doi: 10.1007/s11229-010-9849-5
350–353. doi: 10.1016/j.biopsych.2014.01.006 Sullivan, J. (2014a). “Is the next frontier in neuroscience a decade of the
Craver, C. (2007). Explaining the Brain: Mechanisms and the Mosaic Unity of mind,” in Brain Theory, ed C. Wolfe (New York, NY: Palgrave-MacMillan),
Neuroscience. Oxford, UK: Oxford University Press. 45–67.
Cummins, R. (1983). The Nature of Psychological Explanation. Cambridge, MA: Sullivan, J. (2014b). “Stabilizing mental disorders: prospects and problems,” in
MIT Press. Classifying Psychopathology: Mental Kinds and Natural Kinds, eds H. Kincaid
Cuthbert, B., and Insel, T. (2013). Toward the future of psychiatric diagnosis: the and J. Sullivan (Boston, MA: MIT Press), 257–281.
seven pillars of RDoC. BMC Med. 11:126. doi: 10.1186/1741-7015-11-126 WHO (2016). Mental Disorders [Fact Sheet]. Available online at: http://www.who.
Cuthbert, B., and Kozak, M. (2013). Constructing constructs for psychopathology: int/mediacentre/factsheets/fs396/en/
the NIMH research domain criteria. J. Abnorm. Psychol. 122, 928–937. doi: Whooley, O. (2014). Nosological reflections: the failure of DSM-5, the emergence
10.1037/a0034028 of RDoC, and the decontextualization of mental distress. Soc. Ment. Health 4,
Fodor, J. (1968). Psychological Explanation: An Introduction to the Philosophy of 92–110. doi: 10.1177/2156869313519114
Psychology. New York, NY: Random House.
Lilienfeld, S. (2014). The Research Domain Criteria (RDoC): an analysis of Conflict of Interest Statement: The author declares that the research was
methodological and conceptual challenges. Behav. Res. Ther. 62, 129–139. doi: conducted in the absence of any commercial or financial relationships that could
10.1016/j.brat.2014.07.019 be construed as a potential conflict of interest.
Lilienfeld, S., and Treadway, M. (2016). Clashing Diagnostic Approaches:
DSM-ICD Versus RDoC. Annu. Rev. Clin. Psychol. 12, 435–463. doi: Copyright © 2016 Sullivan. This is an open-access article distributed under the terms
10.1146/annurev-clinpsy-021815-093122 of the Creative Commons Attribution License (CC BY). The use, distribution or
Nagel, E. (1961). The Structure of Science: Problems in the Logic of Scientific reproduction in other forums is permitted, provided the original author(s) or licensor
Explanation. New York, NY: Harcourt, Brace & World. are credited and that the original publication in this journal is cited, in accordance
Poldrack, R. (2010). “Subtraction and Beyond: The Logic of Experimental with accepted academic practice. No use, distribution or reproduction is permitted
Designs for Neuroimaging,” in Foundational Issues in Human Brain which does not comply with these terms.

Frontiers in Human Neuroscience | www.frontiersin.org 4 June 2016 | Volume 10 | Article 309