Neurobiological Basis of Drug Reward and Reinforcement: Chapter Outline

16
Neurobiological Basis of Drug Reward

and Reinforcement
DAVID M. LOVINGER
CHAPTER OUTLINE
Introduction
agreed-upon definition of use disorders. Thus, it is important
Experimental Psychology Concepts of Reward and
to understand the neural mechanisms that contribute to pro-
Reinforcement
longed and maladaptive drug use.
Neurotransmitters and Neural Circuitry: Involvement in The Diagnostic and Statistical Manual of Mental Disorders,
Different Aspects of Reward, Reinforcement, and Addiction Fifth Edition (DSM-5),4 covers substance-related and addictive
Models of Drug Use and Drug Addiction disorders. It should be noted that addiction per se is not a diag-
nosis recommended in the manual, as the term substance use dis-
Actions of Addictive Drugs Within the Reinforcement and
order is preferred. Within this classification, the manual defines
Reward Circuitry
substance use disorder as a “…cluster of cognitive, behavioral,
What Drives Drug Use, Abuse, and Addiction: The Direction of and physiological symptoms indicating that the individual con-
Future Research tinues using the substance despite significant substance-related
problems.” Common features of these disorders are cognitive,
affective, sleep, and behavioral changes that center on use or ces-
sation of use of the addictive substance or action (sometimes
referred to as a habit). Central features of the use disorder are
Introduction risky drug use, craving, social impairment, continued use despite
negative consequences, and the possible negative consequences
Drug use disorders involve a number of factors including of cessation of drug use. All of these aspects of substance use
genetic and environmentally influenced predispositions, the and addictive disorders can be seen to relate to innate brain
actions of the drugs themselves, the immediate environment, mechanisms underlying processes referred to as reinforcement
and the neurobiological mechanisms that promote and support and/or reward. In this context, it is important to discuss cur-
drug actions and addiction. This chapter deals mostly with the rent ideas about the neural mechanisms of reinforcement and
latter aspect of drug use, abuse, and addiction, as we explore reward before discussing the impact of drugs of abuse on these
the ways in which the brain is built to adapt to environmental processes.
circumstances, and how these aspects of neural function can
promote the continued use and abuse of certain drugs and ulti-
mately promote disorders related to these drugs. We then con- Experimental Psychology Concepts of
sider the mechanisms through which drugs of abuse interact Reward and Reinforcement
with the brain systems that promote maladaptive drug use and
addiction. An important aspect of the use and abuse of a wide range of drugs
Drug use disorders have been defined in several different is their reinforcing properties. A reinforcer is defined in experi-
ways, most of which stress the habitual or compulsive nature of mental psychology as a substance or stimulus presented following
addictive behavior; the physical, psychological, and social dam- a behavior that increases the incidence of the behavior above base-
age produced by the behavior; and the trauma associated with line levels. As Skinner180 wrote:
cessation of the behavior. Drug use disorders also share many
features in common with disorders related to natural biologi- The operation of reinforcement is defined as the presentation of
cal drives (e.g., sex and food consumption), physical activities a certain kind of stimulus in a temporal relation with either a
(e.g., excessive exercise), and relatively benign drugs (e.g., caf- stimulus or a response. A reinforcing stimulus is defined as such by
feine) (discussed in Brunton et al.24 and Koob and Le Moal102). its power to produce the resulting change [in the response]. There
Drugs of abuse with harmful effects are the main focus of the is no circularity about this; some stimuli are found to produce the
present discussion. In addition, excessive use of drugs may lead change, others not, and they are classified as reinforcing and non-
to health and psychological problems even in the absence of an reinforcing accordingly.
193
Downloaded for Novin lib ([email protected]) at United Arab Emirates University from ClinicalKey.com by Elsevier on October 08, 2020.
For personal use only. No other uses without permission. Copyright ©2020. Elsevier Inc. All rights reserved.
194 PA RT I I I Visualizations and Workings of the Addicted Brain
It is worth highlighting the dual use of the term stimulus by Studies conducted over the last few decades have led to the
Skinner to refer to both the result of the action (the reinforcer), refinement of the concepts of instrumental conditioning, reward,
and a stimulus within the environment that can become associ- and reinforcement based on the role of the outcome produced
ated with the response and the reinforcer. One definition of rein- by a particular behavior in conditioning paradigms. Dickinson,
forcement, although not Skinner’s position, is that it involves Balleine, and others have shown that responses developed under
a strengthening of the ability of stimuli to elicit responses (the certain types of conditioning schedules will rapidly diminish if
so-called stimulus-response model81), while a somewhat looser the value of the outcome is decreased or if receipt of the outcome
definition is a strengthening of the ability of the environment in is no longer contingent on making the response.1,12,33 This learn-
general, including some neural activity within the animal itself ing of action-outcome contingencies is best achieved with training
and the animal’s past history in that environmental context, to schedules where the outcome is easily predictable and the prob-
elicit the response.48 ability of obtaining the outcome is enhanced with increased rates
The concept of reinforcement is best known from the work of responding (e.g., fixed or random ratio schedules). In this case,
of Konorski and Skinner on what is now called operant or the outcome has been termed to have a rewarding action, based
instrumental conditioning.9,180 However, the term reinforce- on its intrinsic value to the organism at the time of testing and
ment has also been used in the context of Pavlovian, or classi- association with the instrumental action itself.
cal, conditioning. One use of this term is that presentation of an In contrast, training with schedules where predictability is
unconditioned stimulus subsequent to the conditioned stimulus poorer and increasing rates do not increase probability of success-
reinforces the ability of the conditioned stimulus to elicit a con- ful outcomes (e.g., random interval schedules) produces respond-
ditioned response. This term has also been used to refer to the ing that is insensitive to outcome devaluation or noncontingent
effects of stimuli that predict the value of a rewarding stimulus presentation of the outcome.12,45,78 This stimulus-response type
presented prior to presentation of food or another naturally desir- of conditioning can also occur with extensive training using
able outcome.155,175,176 For example, in the paradigm used by schedules with higher predictability (discussed in Yin and Knowl-
Schultz,175,176 responding for food (licking) as well as neuronal ton219). In the case of stimulus-response learning, the association
activity related to stimulus presentation and responding can be is made between antecedent environmental stimuli and the subse-
measured. Dayan and Balleine provided a nice discussion of the quent response, with the outcome serving as a reinforcer regard-
distinctions between reinforcement in the context of Pavlovian less of the immediate value of this outcome to the animal. As you
and instrumental conditioning.42 can see, this is closer to the classical definitions of reinforcement
Two forms of reinforcement, termed positive and negative, favored by stimulus-response theorists, Donahoe, and perhaps
have also been postulated. Positive reinforcement refers to the Skinner.48,180 It should also be noted that White210 drew a simi-
process in which delivery of a desirable consequence increases lar distinction between reward and reinforcement, albeit with a
the incidence of the behavior. This is easily understood in the more traditional behaviorist emphasis on the definitions of these
context of the instrumental or operant conditioning paradigm terms. Other investigators have defined reward in terms of posi-
in which delivery of palatable food will increase bar pressing by tive reinforcement in combination with positive hedonic value,102
a rodent or key pecking by a pigeon.180 Negative reinforcement an idea that suggests more overlap between the two processes.
occurs when the performance of an action results in omission Although the separate definitions of reward and reinforcement in
or avoidance of an undesirable stimulus (e.g., foot shock), and this context may be debated, there is strong evidence for the two
the incidence of the behavior increases as a result of this learn- instrumental conditioning processes themselves. Thus the differ-
ing process.181 The initial phases of learning some skills, such as entiation of the roles of stimuli/environment and outcome in the
swimming, involve what might be termed negative reinforce- two different learning processes is important, and separate discus-
ment, as the skill helps to reduce the undesirable effects of the sion of reward and reinforcement in these contexts is useful.
environment. Many investigators do not subscribe to the idea Before we consider how reward and reinforcement contribute
that positive and negative reinforcement are distinct processes, to addiction it is worth discussing the adaptive purpose of these
as both types of reinforcement basically refer to something that neural systems. Behaviors that lead to enhanced survival and/or
increases the incidence of a given behavior. However, negative reproduction are necessary for propagation of genes and species.
reinforcement is a useful concept when measuring stimulus- Innate feeding, reproductive, and harm-avoidance behaviors
behavior relationships, as it describes a condition in which exist in all animals, but learning about features of the environ-
increasing behavior leads to omission/avoidance of a stimulus. ment is necessary to obtain the opportunity to express these
Learning in everyday life will often involve both positive and innate behaviors. Pavlovian conditioning is one such learning
negative reinforcement. process whereby performance of something approximating an
Two other terms that have come to be used in the con- innate or reflexive behavior can come to be elicited by stimuli
text of instrumental learning and addiction are punishment that were originally neutral with respect to predicting a partic-
and reward. Consideration of the conditions that promote ular outcome (e.g., obtaining food or avoiding harm). Instru-
cessation of behavior led to the definition of the undesirable mental conditioning adds another layer of sophistication to this
outcome as punishment,8,196 although the role of punishment process. Animals with this capacity can learn to perform new
has been hotly debated.182 The term reward was not so read- actions and new sets of actions to obtain a positive consequence
ily accepted by early behaviorists but has come into common or avoid punishment. Both types of learning have obvious adap-
use as a reference to the desirable outcome in an instrumental tive utility, as the animal can now integrate complex features of
learning paradigm. The terms reward and positive reinforce- the world and new behavioral strategies into maintaining safety,
ment are often used interchangeably, but as we will discuss, as well as the quest for food and mating partners. The power of
these terms can be used to refer to different processes that the neural mechanisms involved in reward and reinforcement
control instrumental learning of actions, including drug likely derives from this relationship to survival and reproductive
self-administration. success.126
Chapter 16 Neurobiological Basis of Drug Reward and Reinforcement 195
However, there is the possibility that reward and reinforce- called the nucleus accumbens that sits in the ventromedial region
ment mechanisms will not always be used for adaptive purposes. of the striatum. Neurons within the VTA also send dopaminergic
One such example is the phenomenon of self-starvation. Animals afferents to the prefrontal, orbitofrontal (insular), and cingulate
that are trained to perform an intracranial self-stimulation task, cortices, with more minor projections to other cortical regions
described later, will perform this task at the expense of sufficient such as the limbic cortical subregions.135
eating if access to food is time-restricted.166 Similar self-starva- The initial data suggesting that dopaminergic neuronal activity
tion is observed if animals are given the opportunity to run on is crucial for intracranial self-stimulation was later supplemented
a wheel when on a limited food access schedule.18,165 This par- by the finding that intracranial self-stimulation could be pro-
ticular form of self-starvation has been considered as a model of duced by stimulation within the ventral midbrain regions where
human anorexia nervosa,18 which itself is clearly an example of the dopaminergic neurons reside. Intracranial self-stimulation is
maladaptive behavior involving the brain systems we will consider. supported by stimulation in the VTA, as well as at subregions of
Stimuli that originally signal a positive outcome can change their the SNc.36,132,156 These studies did not rule out the possibility that
predictive value (a certain location may contain food at one time stimulation of fibers that originated elsewhere and passed through
and a predator at another). Furthermore, stimuli or substances the ventral midbrain contributed to intracranial self-stimulation.
that interact with the neural mechanisms involved in reinforce- Nonetheless, the combination of these findings with those find-
ment may come to have reinforcing value even when they are not ings that dopaminergic manipulations alter intracranial self-
coupled to a favorable outcome, or even when they are associated stimulation strongly implicated dopamine coming from ventral
with harmful results. Most drugs of abuse can act in this manner, midbrain neurons in the mechanisms that underlie reward and
and can lead to reinforcement of what we might call maladaptive reinforcement during this process.
behaviors. In the remainder of this chapter we will consider the The focus on dopamine in the context of reward and rein-
brain circuitry and cellular and molecular mechanisms involved forcement often overshadows the role of other neurotransmit-
in reinforcement. Consideration of this topic will also entail some ters. Indeed, dopamine is a modulatory neurotransmitter that
discussion of the experimental techniques used to uncover these in and of itself is not capable of strong excitation or inhibition
mechanisms. of neurons within this circuitry. Furthermore, there is evidence
indicating that dopaminergic transmission is not required for
certain aspects of behavior that are thought to involve reward or
Neurotransmitters and Neural Circuitry: reinforcement. For example, gene-targeted mice that lack dopa-
mine are still able to learn the location of food, but appear to
Involvement in Different Aspects of Reward, require dopamine to express the learned behavior.160 This find-
Reinforcement, and Addiction ing and similar data from other studies seems to indicate that
dopamine is necessary for the motivational aspects of reward
Although the concept of instrumental learning had begun to crys- seeking21,141 or incentive salience.17 In addition, there is evi-
tallize by the early 1930s mainly due to the work of Konorski dence that neurochemical lesions of the dopaminergic system
and Skinner,98,180,222 little was known about the neural circuits do not eliminate self-administration of drugs of abuse such as
involved in this behavior. Konorski and Divac both obtained evi- heroin and ethanol147,154 suggesting that dopaminergic trans-
dence from studies involving lesions of the caudate nucleus impli- mission may not be necessary for all of the rewarding or rein-
cating this part of the striatum in instrumental conditioning.47,98 forcing effects of drugs of abuse. Thus, we need to consider the
The discovery by Olds and Milner136 of intracranial self-stimula- role of other neurotransmitters in reward, reinforcement, and
tion provided an important clue as to the importance of at least addiction. An exhaustive description of all the neurotransmitters
one pathway within the basal ganglia. In the original intracranial involved in these processes is beyond the scope of the present
self-stimulation paradigm, the animal was implanted with an elec- chapter. Instead, the role of particular neurotransmitters with
trode that could stimulate fibers in the medial forebrain bundle. intriguing roles in the brain reward/reinforcement circuitry are
Investigator-initiated stimulation at this site led the animal to discussed.
repeat the behaviors that were ongoing at the time when the stim- Within the central nervous system, the neurotransmitters
ulus was delivered. Thus activation of this neural pathway was in glutamate and γ-aminobutyric acid (GABA) are responsible for
and of itself rewarding or reinforcing. It was later discovered that the majority of fast synaptic transmission.95 Glutamate directly
a key set of axons within the medial forebrain bundle supplied excites neurons via the activation of ligand-gated cation channel-
dopaminergic afferents to the forebrain regions known collectively type receptors, whereas GABA activates anion-preferring channels
as the striatum.36,150,214 This finding stimulated work on the role and generally has an inhibitory action. Both of these neurotrans-
of dopamine in brain mechanisms of reward and addiction that mitters have been implicated in brain mechanisms of reward, rein-
has continued to this day. forcement, and drug actions.50,60,94
The dopaminergic pathways in the brain are now well known. One approach that has been used to examine the role of
The somata of the majority of neurons that use dopamine as a glutamate and GABA in reward, reinforcement, and addiction-
neurotransmitter are concentrated in contiguous ventral midbrain related behaviors is blockade of receptors with specific antago-
structures called the substantia nigra pars compacta (SNc), or A-9 nists, usually injected into a specific brain area.39,41,179,186 These
nucleus, and the ventral tegmental area, or A-10 nucleus.95 The approaches have proven to be effective in altering behavior and
neurons in these two regions project to different striatal subregions have implicated certain subtypes of ionotropic glutamate recep-
and other forebrain targets. Neurons from the substantia nigra tors in reward and addiction-related behaviors. However, it is
pars compacta primarily innervate the dorsal striatum (the cau- sometimes difficult to discern the specific behavioral role of glu-
date and putamen nuclei in primates). In contrast, dopaminergic tamate and its receptors using antagonist blockade, as antago-
neurons from the ventral tegmental area (VTA) project strongly to nists of ionotropic glutamate receptor will almost certainly
the ventral portion of the striatum, particularly a striatal subregion decrease neuronal activity and disrupt circuit activity. Thus, the
antagonist effect may not necessarily reflect a need for activation Opioid neuropeptides are widely distributed in the brain,
of the receptor so much as the necessity of activity of a particu- including in the action/reinforcement/reward circuitry discussed
lar set of neurons. The opposite case often exists for GABAergic at present.95 These peptides, enkephalins, and endorphins, in par-
activity, as blockade of GABA receptors, GABAA receptors in ticular, are perhaps best known for their roles in analgesia. How-
particular, tends to increase neuronal activity and may stimulate ever, it is now well established that opioid peptide production
circuitry. For these reasons, much recent research on the roles and release is increased in response to stressful stimuli and other
of GABA and glutamate in reinforcement, reward, and addic- environmental challenges.53,149 In addition, brain opioid systems
tion has focused on the role of particular glutamate receptor and have been implicated in mechanisms of reward, particularly in
GABAA receptor subunit proteins. relation to food and drugs of abuse. Studies of food-related behav-
The ligand-gated ion channels that mediate fast excitatory iors generally indicate that opioid peptides signal something about
and inhibitory synaptic transmission are multimeric proteins that the hedonic value or desirability of the food, sometimes called
can be formed by numerous subunits and subunit combinations. “liking.”16,21,69,163 Opiate drugs that act as agonists at mu- and
Chronic exposure to addictive drugs alters the expression of par- delta-type opiate receptors are self-administered in instrumental
ticular ionotropic GABA and glutamate receptor subunits.27,89,105 paradigms (reviewed in Gratton63), and opiate agonists produce
Manipulating subunit expression can subtly alter receptor func- decreases in the threshold for intracranial self-stimulation.22,208
tion without eliminating receptor activity. This has allowed Opiate antagonists can also influence intracranial self-stimula-
investigators to explore the roles of these receptors in drug- and tion.209 These findings indicate that activation of the brain opioid
addiction-related behaviors without major disruption of the activ- system has rewarding effects.
ity of neurons within the reward/reinforcement circuitry. This line The brain endocannabinoid system has also begun to receive
of research has been boosted immensely by techniques for trans- a great deal of attention as a mediator of instrumental learning
genic receptor expression and gene-targeted receptor modification and addiction. Endocannabinoids are lipid metabolites that act
and disruption (i.e., so-called knock in and knockout techniques). on the cannabinoid receptors, the receptors originally discov-
Transgenic and gene-targeting mice that express higher or lower ered as mediators of the psychoactive effects of drugs such as
amounts of a desired receptor subunit are quite useful, as are mice marijuana and hashish.134 In the brain, endocannabinoid ago-
that express a slightly mutated version of a receptor. Viral-based nists act mainly through the cannabinoid-1 (CB1) receptor to
gene overexpression, often involving microinjection of constructs produce short- and long-lasting synaptic plasticity.109 The role
into specific brain regions, is also being widely used to enhance of the brain endocannabinoid systems in responses to a variety
protein function in neurons within reward/reinforcement cir- of drugs of abuse is a fascinating topic that has received a great
cuitry. Development of new techniques to alter receptor expression deal of attention in recent years, and this subject is discussed
or structure using the Talen and clusters of regularly interspaced later in this chapter. Recent studies using instrumental condi-
short palindromic repeats (CRISPR) techniques has enhanced the tioning techniques indicate that CB1 receptors play a role in
rate at which gene-targeted mice and viruses can be developed and the transition from action-outcome to stimulus-response (habit)
employed.80,87,124 Altering expression of the GluR1 alpha-amino- learning.58,64,72 Thus the endocannabinoid system may play an
3-hydroxy-5-methylisoxazole-4-propionic acid (AMPA) receptor important role in reinforcement-based instrumental learning. It
subunit in gene-targeted mice alters instrumental learning88,197 is not yet clear if alterations in dopaminergic transmission or
and reduces morphine dependence and sensitization.204 Altered effects on other neurotransmitter systems are involved in this
acute responses to drugs of abuse, as well as changes in ethanol habit-promoting effect of endocannabinoids. The CB1 recep-
tolerance and dependence, have been observed in mice in which tor is highly expressed throughout the brain circuitry thought
GABAA receptors have been altered by gene targeting of alpha2, to mediate instrumental conditioning.71,77 Within these circuits,
alpha5, and delta subunits.110,121,203 However, one caveat that must CB1 receptors are expressed on axon terminals of glutamatergic
be added to this discussion is that neuronal activity has not been and GABAergic neurons (reviewed in Lovinger109), and may well
measured in vivo in the animals used in these studies, and thus, regulate release of other neurotransmitters including catechol-
the extent to which subunit loss alters circuit activity has yet to amines.192 Thus there are many possible sites where endocannab-
be determined in any of the aforementioned experimental models. inoid-dependent synaptic plasticity may play a role in this type
The neuromodulatory transmitter serotonin (or 5-hydroxy- of learning and in addiction.
tryptamine) can influence the brain reward and reinforcement The foregoing discussion should make it clear that to better
circuitry, in part through actions on dopaminergic neurons.60,79 understand the neuronal mechanisms contributing to reward,
Serotonin can also influence goal-directed and habitual behavior reinforcement, and addiction we need to understand more
through its role in control of affect and in impulsivity. The likeli- fully the brain circuits involved in the control of actions and
hood of choosing new actions without strong outcome control the instrumental learning of actions and association of actions
has been linked to disorders of brain serotonergic systems. Impul- with stimuli. We must also gain a better understanding of the
sivity is responsive to some treatments aimed at the serotonergic roles of particular neurotransmitters and receptors in different
system.60,143 Because of the disregard for outcomes, impulsive parts of these circuits. The forebrain, in conjunction with the
responding may be a first step in the process leading to stimulus ventral midbrain, can be conceptualized as a series of paral-
control of behavior and maladaptive habits. Measures of impulsiv- lel cortico-basal-ganglia-cortex circuits that can also be serially
ity in animal models have been suggested to predict a pattern of interconnected (see Yin and Knowlton219 for review). The ulti-
addiction-like drug taking in rodents.15 Serotonin levels in several mate function of these circuits is to modify cortical and brain-
brain regions are elevated during administration of psychostimu- stem output to control the selection, initiation, and timing of
lant drugs such as amphetamine and cocaine, and there is evidence actions to produce effective integrated behaviors. Neurons and
that excessive serotonergic transmission contributes to the addic- synapses within these circuits can undergo plastic changes that
tive effects of these drugs, in addition to the well-characterized are thought to contribute to learning of new actions and associa-
role of dopamine in these processes.59,75,79,129,188 tion of actions with conditioned stimuli.
In an admittedly simplistic scheme, this circuitry can be sepa- cocaine seeking and taking will readily suppress seeking responses
rated into at least three parallel circuits219 (Fig. 16.1). (More when intermittent punishment is given, while prolonged expo-
circuits have been suggested,3,220 and undoubtedly further sub- sure to this paradigm reveals a subgroup of rats that will not show
divisions will emerge based on the complex afferent and effer- this punishment-suppression effect. Furthermore, rats allowed
ent connectivity of the striatum.73,82,185) Each of the circuits to orally self-administer cocaine continued to show instrumen-
consists of a cortical component, a striatal component, down- tal responses associated with the drug even after cocaine devalu-
stream basal ganglia components, and a thalamic component. ation.123 Thus evidence is developing that prolonged exposure
The sensorimotor circuit comprises the primary and second- to psychostimulants can lead to a shift from action-outcome to
ary sensory and motor cortices and the SNc, which project to stimulus-driven behavior.
the putamen (the dorsolateral striatum in rodents), which then The sensorimotor circuit involving the dorsolateral striatum
projects to the motor regions of the globus pallidus, ultimately appears to play a prominent role in stimulus-response or habit
influencing the ventral thalamus and closing the loop back at learning. In this circuit the neocortical components and the dor-
the sensory and motor cortices (see Fig. 16.1A). The associative solateral striatum process information about the relationship
circuitry involves similar connections between associative areas between stimulus presentation and response performance, with
of the cortex (including the prefrontal and parietal regions), the the dopaminergic inputs from the substantia nigra (and the ven-
SNc, the caudate nucleus (the dorsomedial striatum in rodents), tral tegmental area to some extent) providing a reinforcing signal
associative regions of the pallidum, and the mediodorsal and to promote the stimulus-response association.12 It has been sug-
ventral thalamus (see Fig. 16.1B). The limbic circuitry involves gested that the role of dopamine is required for the initial stages of
the limbic cortices (including not only neocortical prefrontal this association, but that behaviors become ingrained and resistant
and temporal areas, but also archicortical regions such as the to dopaminergic manipulations once the stimulus-response asso-
hippocampus and basolateral amygdala), the VTA, the ventral ciation is formed and habitual behavior is in place.211 Ultimately,
striatum/accumbens, the ventral pallidum, and the mediodor- output from the motor cortex and thalamus is important for
sal thalamus (see Fig. 16.1C). One can even consider connec- behavioral performance, and thus, this circuit can produce rela-
tions within the amygdala to have a similar organization, with tively straightforward throughput from sensory input to motor
the cortical component being the basolateral amygdala, the VTA output. Habitual responding has been postulated to contribute
projections providing the dopaminergic modulatory input, the to drug-taking behavior, such that when an individual is in the
striatal components being the central amygdala and bed nucleus proper environment with the drug available, the actions involved
of the stria terminals, and downstream targets leading ultimately in drug administration will be automatized and will often con-
to cortical outputs (see Fig. 16.1D). Evidence for interconnec- tinue regardless of the specific outcome of drug usage. There is
tions among the circuits at the level of striatonigral-striatal pro- emerging evidence that this sort of responding may contribute to
jections can help to coordinate the different systems.14,68,184,219 cocaine- and alcohol-related behaviors in rodents46,122,145 (see also
To fully understand reward and reinforcement-dependent learn- Sampson et al.169) However, to date the role of stimulus-response
ing and resultant behavioral output in the mammalian brain, it associations in drug administration and relapse has not yet been
is necessary to consider all of the components in this circuitry. thoroughly examined and fully dissociated from the stimulus-
Recent studies have begun to shed light on the role of these dependent forms of learning thought to be mediated by the limbic
different forebrain circuits in instrumental and Pavlovian condi- circuit (described in subsequent text).
tioning, and the ideas generated from these studies are now being Among the roles of the limbic circuit is the integration of
applied to examination of drug actions.12,49,57,219,220 Based on information for Pavlovian and instrumental conditioning in a
excitotoxic lesioning and local pharmacological manipulations, type of learning called Pavlovian-instrumental transfer.38,219,220 In
evidence has accumulated that the associative circuit involving the this circuit, limbic neocortical areas such as the ventral prefron-
dorsomedial striatum and associated circuitry, including the baso- tal cortex provide information relevant to task outcomes to the
lateral amygdala,139,140 has key roles in action-outcome learning. nucleus accumbens. The basolateral amygdala provides input on
Afferent inputs from the prefrontal cortex to neurons in the dor- reward and appetitive incentive value to the accumbens, where it
somedial striatum provide one source of input containing infor- is combined with the other cortical information. Dopaminergic
mation relevant to action selection, and the cingulate cortex may inputs to the basolateral amygdala, limbic neocortex, and ventral
provide input about discriminative stimuli.12 The dopaminergic tegmental area also provide information about reward value, while
input from the substantia nigra may provide information about the orbitofrontal cortex may provide information important about
reward value. The contribution of action-outcome learning to the relationship of particular stimuli to task outcomes.138 The role
drug taking is easy to conceptualize. Intrinsically rewarding effects of the hippocampus and other limbic cortical regions that proj-
of drugs likely control behavior even in recreational or social users ect to the nucleus accumbens is less clear. The net result is devel-
seeking the euphoric effects of cocaine and amphetamine or the opment of associations between environmental stimuli and task
anxiety-reducing effects of alcohol. Indeed, studies in rat indi- outcome (sometimes called stimulus-outcome learning), through
cate that cocaine-seeking behavior, measured as an instrumental which discrete stimuli gain control over particular instrumental
response normally associated with drug availability, is rapidly lost responses. In this way the Pavlovian association of the stimulus
with devaluation under certain conditioning regimens.137 It is transfers to the performance of the instrumental response. A role
not yet clear if action-outcome contingencies continue to drive for this type of learning within the context of addiction is easy to
drug seeking and self-administration after long-term drug use and postulate. It has long been thought that stimuli that are associated
in addicted individuals. It is tempting to speculate that addic- with, and predictive of, drug administration (e.g., needles, liquor
tion involves a shift in behavioral control from action-outcome/ bottles) can stimulate drug seeking and taking.57,99,162 Indeed,
reward to stimulus-controlled/reinforcement mechanisms such there is experimental evidence that this sort of cue-induced relapse
as those described in the next few paragraphs. Of interest, Pel- and drug craving can be induced in both humans and experimen-
loux et al.145 have shown that rats given limited experience with tal animals.19,30,31,99,162
SC PC
mPFC
DLS vTh
GP DMS md/vTh
SNc
SNr SNc
GP SNr
A B
Sensorimotor Cortical-Basal Ganglia Circuit Associative Cortical-Basal Ganglia Circuit
Hipp
BNST
mPFC ILC/PLC VLS
mdTh
vPal BLA
VTA HYP
VS/NAc VM CeA
VS/NAc
Limbic Cortical-Basal Ganglia Circuit VTA SNc
C D Amygdala-Basal Ganglia-Cortical Circuit
• Fig. 16.1 Schematic diagram of the reward/reinforcement circuits in the rodent brain. (A) The sensorimotor
circuit (shown in a parasagittal orientation) contains glutamatergic connections from the somatosensory and
motor cortices and the thalamus to the dorsolateral striatum (putamen equivalent in rodents); γ-aminobutyric
acid (GABA)ergic connections from the dorsolateral striatum to the substantia nigra pars reticulata and
motor pallidum (the direct and indirect pathways, respectively); GABAergic connections from pallidum to
nigra reticulata; GABAergic connections from the nigra reticulata to the thalamus; and glutamatergic thala-
mocortical projections back to the sensory and motor cortices. Dopaminergic inputs to the dorsolateral
striatum come from the substantia nigra pars compacta. (B) The associative circuit contains glutamatergic
connections from, for example, the medial prefrontal and parietal cortices, as well as thalamus, to the dor-
somedial striatum (caudate equivalent in rodents); GABAergic connections from the dorsomedial striatum to
the nigra reticulata and pallidum (direct and indirect pathways, respectively); GABAergic connections from
pallidum to nigra reticulata; and glutamatergic thalamocortical projections back to the associative cortex.
Dopaminergic inputs to the dorsomedial striatum come from the substantia nigra pars compacta. (C) The
limbic circuit contains glutamatergic connections from the limbic cortical and limbic areas such as the pre-
frontal cortex, hippocampus, and basolateral amygdala, as well as from thalamus, to the ventral striatum/
nucleus accumbens; GABAergic nucleus accumbens connections to the ventral mesencephalon and ven-
tral pallidum (direct and indirect pathways); GABAergic connections from ventral pallidum to ventral mesen-
cephalon; and glutamatergic thalamocortical connections back to the limbic cortex. Dopaminergic inputs
to the nucleus accumbens come from the ventral tegmental area. (D) The basolateral and central amygdala
connect to a variety of brain regions involved in motivation, reward, and reinforcement (shown here using
separate brain sections). The basolateral amygdala receives dopaminergic input from the substantia nigra
and ventral tegmental area, as well as glutamatergic input from the medial prefrontal cortex. Glutamatergic
efferents from the basolateral amygdala innervate the medial prefrontal cortex, central amygdala, dorsal
striatum, and nucleus accumbens. The central amygdala receives afferent input from the hypothalamus
as well as glutamatergic input from the basolateral amygdala. GABAergic efferent output from the central
amygdala innervates the hypothalamus, the bed nucleus of the stria terminalis, and several hindbrain areas
including the interstitial nucleus of the posterior limb of the anterior commissure, the SNc, the periaqueductal
gray area, the parabrachial nucleus the nucleus of the solitary tract, and the pedunculopontine nucleus (only
SNc projections are shown in this figure). Solid arrows depict connections within the plane of the single
brain section depicted, while dashed lines show connections that run out of the plane of the section. Black
arrows, glutamatergic corticostriatal, thalamostriatal, and BLA projections; green arrows, GABAergic striatal
and pallidal indirect pathway, and BLA projections; blue arrows, GABAergic striatal direct pathway afferents;
red arrows, glutamatergic thalamocortical afferents; brown arrows, dopaminergic nigral and ventral tegmen-
tal afferents; yellow arrow, hypothalamic projection to CeA. BLA, Basolateral amygdala; BNST, bed nucleus
of the stria terminalis; CeA, central amygdala; DLS, dorsolateral striatum; DMS, dorsomedial striatum; GP,
globus pallidus; Hipp, hippocampus; HYP, hypothalamus; ILC, infralimbic prefrontal cortex; mdTh, medio-
dorsal thalamus; md/vTh, mediodorsal and ventral thalamus; mPFC, medial prefrontal cortex; PC, parietal
cortex; PLC, prelimbic prefrontal cortex; SC, sensory cortex; SNc, substantia nigra pars compacta; SNr,
substantia nigra pars reticulata; VLS, ventrolateral striatum; VM, ventral mesencephalon; VS/NAc, ventral
striatum/nucleus accumbens; vPal, ventral pallidum; VTA, ventral tegmental area; vTh, ventral thalamus.
The characterization of the limbic circuit as the mediator of state via neuromodulatory regions such as the hypothalamus.167
reward and/or reinforcement is an idea that has captured the Output from the central amygdala is sent to the bed nucleus of
imagination of neurobiologists and addiction researchers.7,61,83,96 the stria terminalis, hypothalamus, and other subcortical regions,
However, it is now becoming clear that the circuitry that includes as well as to the substantia nigra and ventral tegmental area, where
the dorsal striatum has an equally important role in these pro- it can influence the circuitry involved in stimulus-response and
cesses (see Yin et al.220 for review). In addition to the studies men- stimulus-outcome learning.152,167 In addition, the amygdala has
tioned that implicated the associative and sensorimotor circuits in emerged as a brain region with important roles in conditioned
action-outcome and stimulus-response learning, there is also evi- responses related to the rewarding effects of drugs studied using
dence that dopaminergic innervation of the dorsal striatum plays the conditioned place preference task described in the following
important roles in instrumental learning. Stimulation of dopa- section.23,65 The amygdala interconnections with the bed nucleus
minergic neurons in the substantia nigra pars compacta supports of the stria terminalis, a subcortical nucleus with a striatal-like
intracranial self-stimulation, as mentioned earlier. Furthermore, organization (i.e., populated predominantly by GABAergic pro-
activation of substantia nigra neurons with intracranial self-stim- jection neurons), have generated a great deal of interest, as the
ulation–inducing patterns enhances learning and striatal synaptic bed nucleus of the stria terminalis has been implicated in the
plasticity.156 An elegant series of studies by the Palmiter laboratory actions of drugs of abuse as well as in drug self-administration and
indicate a key role for dorsal striatal dopamine in instrumental relapse.6,66,206
learning and performance. Using dopamine restored in the dor- Clearly, a better understanding of the brain regions in involved
sal striatum of mice that have been engineered to lack the neu- in learning and control of behavior involving reward and rein-
rotransmitter, these investigators have shown that food-seeking forcement is emerging. In addition, methodology is emerging that
and instrumental learning/performance were rescued.159,161 Thus will help define the roles of brain circuitry and circuit physiology
full neurochemical integration within the dorsal striatum is all in behavior, and to refine our behavioral models based on neuro-
that is needed for proper motivational signaling and instrumental scientific findings. One of the challenges in addiction research in
performance. This is not to say that the limbic circuitry does not the coming years will be to determine how the function of these
have reward-related functions, but rather that an intact limbic cir- brain circuits contributes to responses to drugs of abuse, maladap-
cuit may not be necessary for proper learning and performance of tive use of the drugs, and addiction.
a purely instrumental task.
In recent years, researchers have also focused on the circuitry Models of Drug Use and Drug Addiction
involved in generating undesirable effects that contribute to drug
taking and relapse, and the effects of the drugs themselves on this Examination of the neural basis of drug actions, drug use, and
circuitry (reviewed in Koob100 and Koob and Le Moal102). There is addiction has relied to a great extent on development of labo-
evidence for reduction in the positive hedonic effects of drugs after ratory animal models. A great deal of progress has been made
sustained self-administration, and negative consequences of drug with this approach. However, it has proven difficult to model all
use and withdrawal increase with repeated use and withdrawal.a aspects of drug actions and addiction. For example, how does
The amygdala and associated structures appear to have prominent one assess euphoria or craving in an animal that is incapable of
roles in this scenario. The amygdala has generally been thought of verbal self-report. Progress was slow at times for development of
as a brain region involved in the processing of information related reasonable models of self-administration for drugs such as alco-
to emotion, and the role of the amygdala in anxiety and responses hol, cannabinoids, and nicotine.111,170,183 Agreeing on a universal
to stress is widely known.107 However, one can also view the role of definition of addiction and developing an animal model thereof
the amygdala as providing a neural index of the incentive value of has also proven to be difficult. Nonetheless, several decades of
a particular stimulus or event.12,168,200 In this context, the amyg- research have led to the development of a variety of behavioral
dala plays roles in both reward and reinforcement processes as tests that assay various aspects of drug action, drug use, and
defined earlier. Furthermore, it is now clear that the structure we addiction (Table 16.1). These techniques continue to be refined
call the amygdala can be subdivided based on cytoarchitecture and and combined with new techniques for neuroscientific investiga-
afferent/efferent connections. Two well-characterized amygdalar tion to provide more complete information about relevant neural
subregions are the basolateral and central nuclei. The basolateral mechanisms. The following discussion will describe some of these
amygdala is an archicortical structure containing mainly gluta- animal models, with an emphasis on models of drug reward, rein-
matergic projection neurons and a small number of GABAergic forcement, and addiction.
interneurons. The basolateral amygdala innervates other structures A seemingly direct way to measure the reinforcing effects of a
within the amygdala, but also has connections with parts of the substance is to determine whether delivery of the drug itself will
prefrontal cortex, and the dorsomedial and ventral regions of the support learning or continued performance of a particular action
striatum.167 Input to the basolateral amygdala from areas such as or set of actions. This so-called self-administration paradigm has
the ventral tegmental area and the locus coeruleus provides infor- been used to examine the reinforcing actions of many drugs of
mation about arousal and motivational state152; thus, one possible abuse in a variety of animal models, and in general all of these
role for this brain region is to integrate information necessary for drugs have been found to support self-administration under at
a reward signal and relay that information to the associative cir- least one schedule of drug administration.57 Comparisons of self-
cuit involved in action-outcome learning. The central amygdala is administration in humans and laboratory animals have indicated
similar in cytoarchitecture to the striatum, having a large propor- similarities that auger well for the experimental use of these pro-
tion of GABAergic projection neurons.167 This structure receives cedures.142 The general procedure is to train the animal to press
excitatory input from the basolateral amygdala, neocortical, and a lever or nose-poke an object in order to receive the drug either
paleocortical regions, as well as information about motivational by oral, intravenous, or intracranial routes of administration.
Using the basic instrumental training schedule, animals can also
a References 56, 97, 100, 102, 108, 118. be tested in a final short extinction session in which no drug is
TABLE 16.1 Models of Drug Reward and In light of the previous discussion of reinforcement and
Reinforcement: Relation to Phenotypes reward, or action-outcome and habit learning, it seems impor-
of Human Drug Use, Dependence, and tant to evaluate which of these modes of behavior actually
drives drug self-administration. As mentioned in the preceding
Addiction.
text, investigators have found evidence that drugs of abuse pro-
Model Human Drug Use Phenotype mote habit learning37,46,122,145 (see also Samson169). Another
Simple operant self-admin- Hedonic value, liking/wanting
common variant of the drug self-administration procedure is
istration cue-induced drug-related responding and reinstatement of this
responding and/or drug taking. It has clearly been demon-
Devaluation Goal-directed versus habitual strated that cues signaling the opportunity to respond instru-
responding mentally and obtain a drug can come to elicit responding in the
Intracranial self-stimulation Hedonic value, anhedonia absence of the drug, and especially robustly when the drug has
threshold changes been omitted for long periods.19,93,177 This procedure involves a
component of stimulus-outcome learning or Pavlovian-instru-
Conditioned place prefer- Reinforcement, incentive sensitization,
ence/aversion resistance to negative outcome
mental transfer with the cue serving as the Pavlovian condi-
tioned stimulus. Indeed, this type of conditioning has become
Progressive ratio breakpoint Hedonic value, compulsivity pretty much the standard in instrumental self-administration
Behavioral cost Hedonic value, compulsivity procedures, as some explicitly paired cue, most often a light, is
included in most such studies. This may be one reason for the
Response persistence Compulsivity large number of studies implicating the aforementioned lim-
without drug bic circuitry in drug-seeking behavior, as this circuitry appears
Punished responding/ Compulsivity, resistance to negative to have important roles in stimulus-outcome learning. There is
pairing with undesirable outcome certainly some heuristic value to such studies in the context of
tastant human addiction, as it is easy to imagine how environmental
Cue-induced reinstatement Craving, incentive sensitization
stimuli that signal drug availability might trigger drug seeking
and relapse.
Secondary reinforcement Craving, habitual responding Other surrogate measures of the rewarding effects of drugs
Psychomotor stimulation/ Incentive sensitization of abuse have been developed. Drawing on the intracranial self-
sensitization stimulation paradigm discussed earlier, investigators have exam-
ined the ability of drugs of abuse to shift the threshold stimulus
Incubation Craving, incentive sensitization intensities needed to support self-stimulation. Several abused
drugs produce a leftward shift in the stimulus-response curve or
increase rates of responding, indicating that they enhance the
reinforcing properties of intracranial self-stimulation (see Wise215
available to see if they perform the operant behavior. This helps for review). Drugs with this sort of action include those that are
to assess the drug-seeking behavior without any interference from strongly self-administered such as cocaine and amphetamine, as
neural actions of the drug itself (e.g., depressant effects that reduce well as other drugs of abuse, although studies of ethanol have
rate of responding). Variations of this basic procedure include the yielded mixed results that might be explained by variables such as
use of secondary reinforcers (e.g., stimuli paired with the oppor- route of drug administration.10,103,125 This technique can reveal
tunity for drug self-administration that come to elicit behavior indirectly the rewarding or reinforcing effects of drugs, but thus
themselves),15,49 and use of a progressive ratio schedule in which far the emphasis has mainly been on the effects of investigator-
animals must increase their responses exponentially157 with each administered drugs and involvement of the limbic circuitry.
trial in order to continue drug delivery. In this latter procedure, It would be interesting to see this line of research extended to
the investigator assesses the breakpoint, which is the response include more self-administration/intracranial self-stimulation
requirement beyond which the subject will no longer work for studies and examination of different circuitry and component
the drug. This procedure can be used to determine the relative brain regions.
reinforcing efficacy of a particular drug. This approach has the Recent studies have focused on identifying behaviors that
advantage of direct measurement of the animal’s willingness to might be indicative of an addictive phenotype in experimental
use the drug. However, there are some drawbacks to self-admin- animals. One approach has been to develop a battery of tests
istration techniques. For example, self-administration leading to designed to measure continued drug seeking and taking under
high levels of drug in the brain that impair subsequent perfor- conditions where these behaviors become increasingly difficult and
mance of the actions needed for further drug taking (reviewed in costly. Deroche-Gamonet et al.44 have developed a three-test bat-
Hemby et al.70). Oral self-administration of drugs such as ethanol tery consisting of: (1) measuring the progressive ratio breakpoint
brings into play factors such as taste that affect the willingness of mentioned earlier; (2) measuring the persistence of instrumental
certain animals to ingest the desired drug.62 Use of instrumental responding on a previously cocaine-associated manipulandum
self-administration procedures also necessitates consideration of even when a signal indicates no drug availability; and (3) deter-
separate neural control of drug seeking and drug taking.15,162,170 mining whether cocaine self-administration will continue even
Nonetheless, self-administration procedures are arguably the most when associated with electric foot shock (a paradigm also used
direct measure of use and abuse, particularly given the variety of in Pelloux et al.145 and Vanderschuren and Everitt201). Of inter-
procedures that have been developed using instrumental para- est, Wolffgramm and Heyne217 and Petry and Heyman146 have
digms. Self-administration procedures also allow investigators to used a conceptually similar approach with alcohol. Wolffgramm
examine the effects of treatments on drug use in preclinical assays. and Heyne217 provided the alcohol in a solution with a normally
aversive tastant, and they found that this procedure decreased An alternative experimental approach is to examine key phe-
drinking in animals that had short-term alcohol drinking expe- notypic behaviors associated with drug use disorders and attempt
rience, while drinking was maintained at much higher levels in to determine the molecules, cells, and circuits that control these
animals that had been drinking alcohol for a long period (at least behaviors.91 For example, phenotypes such as excessive drug
9 months). Petry and Heyman146 steadily increased the behav- intake following abstinence, cue-induced reinstatement of drug
ioral cost necessary to obtain an alcohol-containing solution and self-administration, incubation of increased lever-pressing, or
found that rats with experience drinking alcohol maintained outcome-resistant drug intake can all be coupled with neuro-
their drinking despite the increasing cost, while similar effects physiological measurements and manipulation of particular brain
were not observed with palatable nutrient-containing solutions. circuits to better understand the neural underpinnings of different
These sorts of techniques are now being used to examine factors facets of the response to drugs of abuse and drug-seeking/taking
that predispose animals to uncontrolled/compulsive drug self- behaviors (see Table 16.1). The ultimate goal of this research is
administration. Everitt and coworkers15 have determined that to develop therapies aimed at these brain components to reduce
what appears to be impulsive responding in a five-choice serial harmful aspects of drug use disorders.
reaction time test is predictive of later abusive drug use in this One important phenotype is the rewarding effects of the drug
paradigm. This paradigm has been used by investigators to iden- itself. Investigators have long used Pavlovian conditioning to
tify subgroups of rats that are especially vulnerable to what might examine whether drug administration can be used to produce a
be termed addiction. It is interesting to note that only a rela- conditioned place preference in an animal.198 In this paradigm,
tively modest subgroup of rats given extensive self-administration the animal is given the drug paired with one of two or three
experience show maintained responding in the second and third chambers in an apparatus, and then tested later for location pref-
tests and also show high breakpoints in test 1.44 It is hoped that erence. In a final drug-free test, the animal is then free to choose
this approach will provide a powerful tool for identifying genetic, a location in which to spend the trial. If more time is spent in
neuronal, and circuit differences that contribute to enhanced sus- a particular location, this is thought to indicate that the drug
ceptibility to drug addiction. paired with this location has a preferred or rewarding effect. This
Although this approach has some face validity, it is not clear technique has the advantage that the animals are not subjected to
that a model of all aspects of addiction can be developed in drug intoxication at the time of testing, so there is little chance
rodents. Most rodent drug self-administration paradigms use of impairment of behavior by the drug itself. However, it must
operant responding for drug delivery, and often with intravenous be stressed that the location is a conditioned stimulus and not a
drug administration directly contingent upon the operant action. primary reward or reinforcer of any kind in this paradigm. Thus
The measurements in such experiments are generally number of the technique does not measure these functions per se, that is,
lever presses and number of drug infusions. However, because the animal does not have to repeat an action to obtain an out-
infusion will occur following the prescribed number of lever come, and thus, it is at best a surrogate measure of the underlying
presses there is no way to separate drug seeking (i.e., operant construct and one that may be subject to influence by proper-
responding) from drug taking (infusion). Thus it is unclear if the ties of the environment or drug that are not directly related to its
different manipulations are altering the operant responses or the reinforcing effects. Nonetheless, strong progress has been made in
drug control of these responses. Investigators have tried to sepa- identifying the neural mechanisms underlying conditioned place
rate the seeking and taking aspects in operant self-administration preference,22,64,74,178,179 and there is considerable overlap with
(SA) procedures using second order schedules and oral drug tak- mechanisms implicated in reward circuitry.
ing (especially for ethanol). The two aspects can be controlled The psychomotor stimulant effects of drugs have also been
separately, indicating a confound in interpreting effects of experi- proposed to provide a measure of drug reinforcement, reward,
mental manipulations in operant SA studies. Another problem is and addiction.216 Administration of many drugs will produce for-
the method of scoring in such studies. The protocol is designed to ward locomotion and it has been theorized that this represents an
identify the top scorers in a particular test within a given cohort operant approach response indicative of positive reinforcement
of a given rodent strain. This system generally ignores important by the drug.216 That forward locomotion is elicited by stimula-
genetic and cohort effects that influence behavior toward drugs tion of the medial forebrain bundle, the site where stimulation
of abuse. Indeed, C57Bl6J mice will more readily self-adminis- yields intracranial self-stimulation, was also advanced as evidence
ter several drugs of abuse in comparison to other mouse strains, that the mechanisms underlying this locomotion are linked to
and the genes that regulate these differences are being character- positive reinforcement. However, it is possible that locomotor
ized.43,106,130 Thus, even the top scorers from these other strains activation is merely an adjunct consequence of drug exposure and
may not reach the mean level achieved by mice from the more medial forebrain bundle stimulation. The circuitry that controls
self-administration-prone strain. It is difficult to see how one can performance of voluntary actions overlaps extensively with that
label mice as “addiction-prone” when there are large numbers involved in reinforcement, reward, habit formation, and addic-
of mice from another strain that show more severe drug-related tion. Thus, it is possible that drug actions produce separate effects
behaviors. There are additional problems in the case of alcohol, that both influence locomotion and drug reward or drug seek-
where self-administration often involves oral intake. In general, ing, but that these effects are separable. Indeed, elegant studies
it has been difficult to induce rodents to drink to the same blood showed just such a separation for regions of the ventral tegmental
alcohol levels and levels of intoxication achieved by humans.13 area and nucleus accumbens implicated in cocaine- and opiate-
New techniques have been developed for increasing rodent alco- induced locomotor stimulation and conditioned place preference
hol intake, and older techniques are being revisited.13 However, or self-administration.178,179 In addition, mice that lack dopamine
it is important to consider other animals that show excessive show a nearly complete loss of morphine-stimulated locomo-
alcohol intake, such as nonhuman primates.11,86 Overall, devel- tion but continue to show morphine-induced conditioned place
opment of a single rodent model that captures all aspects of addi- preference.74 Locomotor stimulation and reward/reinforcement
tion is an overly ambitious undertaking. can also be separated pharmacologically. In the case of alcohol,
stimulation of forward locomotion is inconsistent in rats, and responding has also been observed in alcohol-dependent animals
locomotor depressant effects are most often observed,54,113 but and is referred to as the “alcohol deprivation effect” (reviewed in
rats clearly show other signs of ethanol reward and reinforcement Spanagel189). Animals that have undergone conditioned aversion
(see Koob99,100 for examples). Furthermore, Risinger et al.158 and in which withdrawal is rapidly induced and paired with previ-
Sanchez et al.172 found differences in genetic factors underlying ously neutral stimuli show reinstatement of heroin self-adminis-
ethanol-induced locomotor stimulation and conditioned place tration and elevation of intracranial self-stimulation thresholds,
preference for ethanol. Thus, it is not clear that forward locomo- as if the aversive effects of withdrawal were reducing rewarding
tion is a good proxy for the actual reinforcing effects of the drug. drug effects while driving relapse.145 It is easy to imagine how
The idea of sensitization, an increase in frequency and intensity this withdrawal-relief model can explain relapse after full-blown
of a behavior elicited by a stimulus or treatment, has also figured symptoms have begun. However, it is not so clear that this model
prominently in models of drug abuse and addiction. Repeated can explain continuous drug use in the absence of withdrawal
administration of certain drugs of abuse, psychostimulants in par- sufficient to produce symptoms. The ability of this model to
ticular, elicits successively larger increases in locomotor activity in explain relapse long after the cessation of withdrawal symptoms
rodents. It has been speculated that this locomotor sensitization is also not as clear. Processes such as incubation, as lasting neu-
is a result of the underlying neuroadaptive processes that contrib- roadaptation that leads to greater drug seeking after prolonged
ute to addiction following repeated drug exposure. However, it abstinence, may more readily account for this type of relapse.205
is still not clear that the locomotor-stimulating effects are related Several brain regions within the associative and limbic circuitry
to reward or reinforcement per se, for the same reasons discussed have been implicated in the incubation process.25,148 In addition,
in the preceding paragraphs. In one sense, however, drug seeking lasting recruitment of drug effects on brain systems involved in
and self-administration must involve some form of sensitization, stress responding has been suggested to underlie the long-term
as these behaviors involve increases in responding elicited by the susceptibility to relapse.102
drug or drug-related environments or cues. One theory advanced The concept of negative reinforcement is also built into addic-
to account for this aspect of drug-related behavior is the incentive- tion theories based on the Opponent-Process idea.102,187 These
sensitization model.162 This theory provides a reasonable explana- theories essentially propose that net emotional state is the result
tion for the willingness of addicts to expend a great deal of energy of competition between emotions (e.g., elation vs. fear), and that
and engage in new behaviors to obtain drugs, and also can explain changes in the competitive balance over time can lead to changes
the greater motivation of animals to work for previously used in net emotion and behavior. Within the context of addiction
drugs in tasks such as the progressive ratio/breakpoint paradigm one can easily envision that the euphoric high achieved just after
mentioned in the preceding text. Other behavioral measures in administration of a drug like cocaine can dissipate and be replaced
laboratory animals provide evidence for enhanced incentive to by depression as the neurochemical effects of the drug wear off.202
seek and use drugs. For example, conditioned place preference With repeated drug use, the euphoria becomes less pronounced
and cue-induced reinstatement of drug seeking indicate that the as the depression is enhanced, and the user ends up taking the
motivational value of previously neutral stimuli is enhanced when drug to relieve the depression, which could be termed a nega-
these stimuli are associated with drugs of abuse (reviewed in Rob- tive reinforcement model. This process has been modeled with
inson and Berridge162). Thus although simple locomotor sensitiza- cocaine and heroin self-administration, and it was found that both
tion may provide only limited information about drug effects on intracranial self-stimulation thresholds and cocaine or heroin self-
the brain reward/reinforcement system, the concept of sensitiza- administration escalated after several cycles of self-administration
tion is important within this context. and withdrawal.2,97 Koob, LeMoal, and coworkers have extended
A role for negative reinforcement in addiction is also eas- these ideas to include the concept of allostasis117,213 in which the
ily conceptualized, and experimental models based on this idea emotional set-point resulting from the new balance of opponent
have been developed to provide information on important drug processes is altered toward a more depressed level with repeated
abuse-related phenotypes and neural mechanisms. Drugs such drug use.100,102 The addict ends up using the drug to maintain this
as benzodiazepines have known anxiolytic properties23 and thus new set-point, often relieving more adverse emotional symptoms.
reduce an aversive state. The psychostimulants produce acute Experimental models such as withdrawal-induced excessive self-
mood elevation that may provide temporary relief from negative administration have been used in conjunction with neurochemi-
affect (although these drugs are by no means effective antidepres- cal approaches to implicate the extended amygdala and associated
sants). Thus negative reinforcement may be a strong driving force brain regions in these allostatic changes and the accompanying
for acute drug use. negative reinforcement driving drug taking and relapse (reviewed
Relief of the negative symptoms encountered during drug in Koob100 and Koob and Le Moal102). Brain systems for respond-
withdrawal can also be characterized as a negative reinforcing ing to environmental stress and internal anxiety may provide the
component of addiction. Withdrawal following chronic use of aversive effects that interact with brain reward/reinforcement cir-
different drugs of abuse produces symptoms ranging from height- cuitry to drive drug use in these models.101
ened anxiety and irritability (benzodiazepines, alcohol) and dys-
phoria and depression (psychostimulants) to severe physiological
symptoms such as abdominal cramps (heroin).23,118 Withdrawal Actions of Addictive Drugs Within the
from drugs is associated with higher thresholds for intracranial Reinforcement and Reward Circuitry
self-stimulation in experimental animals, indicating dysphoria
associated with this state.101,174 Relief from these symptoms has The majority of abused substances act on specific molecular tar-
been postulated to drive relapse to drug use.100,102 Indeed, animals gets within the brain. These drug actions influence mechanisms
made dependent on drugs will increase self-administration and thought to be involved in addiction, usually by producing a direct
drug-related instrumental responding following drug withdrawal reinforcing or rewarding effect. Using the aforementioned addic-
(reviewed in Koob and Le Moal102). This sort of reinstatement tion models combined with neurochemical and pharmacological
techniques, investigators have discovered a number of molecu- The primary targets of almost all drugs of abuse are cell sur-
lar interactions produced by acute and chronic drug exposure face proteins that regulate synaptic transmission. The role of
that underlie the rewarding and reinforcing effects of drugs of neurotransmitter transporters as targets for psychostimulants has
abuse. The mechanisms that are easiest to appreciate are those already been discussed. Drugs such as nicotine, benzodiazepines,
triggered by the so-called psychostimulant drugs, such as cocaine barbiturates, and, to some extent, alcohol produce their actions by
and amphetamine. These compounds act directly on the protein altering the neurotransmitter receptors known as ligand-gated ion
known as the dopamine transporter to enhance the concentra- channels.24 These receptors mediate fast synaptic transmission in
tion of dopamine present in the synaptic cleft following release the brain and can directly influence the activity of neurons within
of the neurotransmitter.104 Cocaine is a competitive inhibitor of the brain reward/reinforcement circuitry. An assortment of other
dopamine transporter that reduces the reuptake of dopamine into psychoactive and addictive drugs, including opiates, cannabinoids,
neurons, while amphetamine can also increase release of dopa- and hallucinogens, produce their actions via G protein–coupled
mine directly through the dopamine transporter.92 It is easy to receptors.24 This class of receptors produces neuromodulatory
see how these molecular actions have rewarding and reinforc- actions of neurotransmitters by initiating or influencing intra-
ing consequences, given that dopamine has an important role in cellular molecular signaling pathways.24,95 The subtle changes in
both of these processes, and these drugs will enhance dopamine neuronal activity and gene expression produced by drug actions at
signaling. these receptors can have profound acute and long-lasting effects
Given that cocaine is thought to act primarily on the dopamine on the brain reward and reinforcement systems.
transporter, it is surprising that evidence has emerged for actions Opiate drugs produce their intoxicating and rewarding effects
of drugs of abuse that could be construed as indicating reinforc- through activation of mu, and to a lesser extent delta and kappa,
ing effects of cocaine even in the absence of the transporter.164,188 opiate receptors.35,114 Opioid peptides and their receptors have
However, it appears that blockade of serotonin reuptake indirectly also been implicated in the rewarding effects of drugs of abuse,
mediates this effect. Even more surprising is the finding that sero- including effects of nicotine, cannabinoids, and alcohol (reviewed
tonin mediates cocaine conditioned place preference in dopamine- in Contet et al.35 and Gaveriaux-Ruff and Kieffer55). There is
deficient mice.75 These mice show a conditioned place preference also considerable evidence that opiate receptor blockade reduces
for morphine.74 Thus, dopamine itself may not be necessary for rewarding and reinforcing effects of a variety of drugs of abuse, as
this drug-related learning. However, evidence that manipulations well as drug seeking and taking in animal models of addiction (see
that decrease the firing of dopaminergic neurons reduce condi- Boutrel20 for review). The opiate receptor antagonist naltrexone
tioned place preference in these dopamine-deficient mice suggest can also produce lower sensitivity to intracranial self-stimulation,
that another factor released by these neurons may have important but this is seen mainly with stimulation in the ventral striatum
roles in drug-related reward.75 and not with ventral tegmental area stimulation.209 Thus, there is
The actions of many other drugs of abuse have also been reason to believe that opioid peptides can mediate the reinforcing
suggested to involve dopamine, but in a less-direct manner. and rewarding effects of drugs of abuse, perhaps independent of
For example, opiates, nicotine, and alcohol all increase dopa- the actions of dopamine.
mine levels in regions of the brain such as the ventral and dorsal As mentioned in the preceding text, the cannabinoid drugs
striatum, and appear to do so by increasing the firing rate of produce their intoxicating actions mainly via brain CB1 receptors.
the dopaminergic neurons themselves (reviewed in Pierce and Delta-9-tetrahydrocannabinol, the main psychoactive ingredient
Kumaresan151). Hypotheses about the mechanisms underlying in cannabis-derived drugs, is a partial agonist of the CB1 receptor.
the reinforcing effects of these drugs of abuse have generally cen- Several synthetic delta-9-tetrahydrocannabinol analogs and other
tered on the idea that enhancement of dopaminergic transmis- CB1 agonists also produce intoxicating effects. Emerging evidence
sion leads to reward or reinforcement, and that all drugs of abuse indicates an important role for endocannabinoids and the CB1
work through this mechanism in one way or another. However, receptor in drug self-administration. Antagonist blockade or gene-
this idea is undoubtedly too simplistic given the evidence dis- targeted knockout of CB1 receptors decreases self-administration
cussed earlier that dopamine is not necessary for reward-related of a variety of drugs of abuse in animal models.112 Blockade of the
learning. Furthermore, dopamine does not appear to be necessary receptor decreases intake and operant self-administration of etha-
for self-administration and the reinforcing effects of all drugs of nol.32,207 Antagonists of the CB1 receptor also reduce increases
abuse, particularly for drugs with indirect effects on the dopami- in extracellular dopamine produced by several drugs of abuse,29
nergic system. Studies of the effects of dopaminergic lesions and and thus, endocannabinoids may alter the rewarding or reinforc-
dopamine receptor antagonists on ethanol self-administration ing actions of drugs of abuse via this mechanism.
have yielded mixed results in a variety of self-administration and Chronic exposure to drugs of abuse is thought to bring about
operant paradigms. D2 receptor antagonists and D1 antagonists neuroadaptive changes that alter the brain reinforcement/reward
and knockout of D1 and D2 reduce self-administration in some circuitry. Recent studies have focused on changes in the efficacy of
paradigms, but alcohol intake is not abolished after lesions of synaptic transmission and alterations in dendritic morphology in
the dopaminergic system or by other dopamine receptor antago- neurons within the ventral tegmental area and nucleus accumbens.
nists.151,154,171 The evidence for dopamine involvement in opi- Cocaine exposure has been demonstrated to produce changes in
ate self-administration and opiate-related reward/reinforcement the ratio of excitatory synaptic responses mediated by AMPA and
is likewise mixed, including evidence for lack of effect of dopa- N-methyl-d-aspartate–type glutamate receptors, a change thought
minergic lesions.147,151 Thus, it appears that dopaminergic trans- to reflect long-term plasticity of glutamatergic synapses.5,195,199
mission is not strictly necessary for alcohol and opiate intake. This sort of synaptic plasticity has been observed in recordings
Dopamine is probably not the final common pathway for drug from both nucleus accumbens medium spiny neurons and ventral
reward, reinforcement, and self-administration as was once tegmental area dopaminergic neurons.195,199 Even a single dose of
imagined, and the focus has now shifted to the importance of cocaine appears to produce this plastic change in excitatory trans-
the circuitry that is influenced by dopamine. mission.199 Other drugs of abuse, including ethanol, alter efficacy
of both excitatory and inhibitory transmission in the ventral teg- following chronic drug exposure could underlie loss of metabo-
mental area.119,190 Synaptic plasticity produced by drug exposure tropic glutamate receptor–mediated synaptic plasticity.52,67,173 In
may condition synapses within the reinforcement/reward circuits the case of alcohol, increased metabotropic glutamate receptor/
to enhance responses to subsequent exposure to drugs or drug- Homer expression may be part of a general increase in glutamater-
related stimuli. This could occur at the expense of using this cir- gic signaling after repeated drug administration and withdrawal,
cuitry to learn other, more adaptive, responses to environmental contributing to plastic changes in the circuitry that ultimately fos-
stimuli. ter increased drug seeking and self-administration.185 This idea is
Chronic drug exposure also alters long-term synaptic depres- consistent with the finding that a metabotropic glutamate recep-
sion mediated by endocannabinoids and the CB1 receptor. tor-5 antagonist decreases alcohol seeking and relapse in animals
Repeated exposure to delta-9-tetrahydrocannabinol for days elim- with self-administration experience.9
inates this form of long-term synaptic depression in the dorsal Roles in chronic drug actions and relapse for the presynapti-
striatum, hippocampus, and nucleus accumbens,76,116,131 and this cally localized metabotropic glutamate receptor 2 (mGluR2)
adaptation appears to involve decreases in the presynaptic actions are emerging from recent research. Early studies by Kalivas and
of CB1 receptors. Even a single exposure to delta-9-tetrahydro- coworkers indicated that increased extracellular glutamate pro-
cannabinol produces a similar action.115 A single treatment with duced by changes in transporter function leads to mGluR2
cocaine also eliminates endocannabinoid-dependent long-term activation following chronic cocaine exposure.127 This feedback
synaptic depression,52 and chronic ethanol exposure has been mechanism may help to limit hyperactivation of key cortical
reported to have a similar action in the striatum.218 Loss of endo- glutamatergic inputs that help drive drug seeking and taking.
cannabinoid-dependent long-term depression (LTD) is associated Indeed, recent studies support the idea that deficient mGluR2
with enhanced habit learning in mice.131 Given the role of CB1 activity contributes to increased seeking and taking of cocaine,
receptors in response to the drugs of abuse mentioned previously, ethanol, and nicotine.26,90,120,221 A positive allosteric modulator
it is possible that adaptations in endocannabinoid/CB1 function of mGluR2 shows promise for reducing drug seeking after chronic
underlie neuroadaptations that lead to altered reinforcing and exposure and abstinence in animal models.26,90
rewarding properties of many drugs of abuse. However, despite this wealth of knowledge, it is still not clear
Repeated exposure to cocaine, amphetamine, alcohol, and how the cellular and molecular changes brought about by drugs
other such drugs of abuse changes the dendritic structure of the of abuse contribute to overall changes in circuit function that ulti-
neurons in the prefrontal cortex, nucleus accumbens, and dor- mately lead to addiction. Addressing this topic will require more
sal striatum.40,85,128,163 Changes in spine density and dendritic sophisticated analysis of neuronal and circuit function in vivo,
branching are seen, with differential effects in different brain combined with continued work at the molecular, cellular, and
regions. These morphological changes are thought to lead to alter- behavioral levels. Clearly, this is a key direction for future research
ations in the ability of neurons within this circuitry to respond to on the neural basis of drug reinforcement and reward.
normal levels of synaptic input. Ultimately, this dendritic rear-
rangement could lead to less plasticity within the circuitry and
help to rigidify behaviors related to the addictive drug. What Drives Drug Use, Abuse, and Addiction:
There are also a host of cellular and molecular changes within The Direction of Future Research
the reinforcement/reward circuitry that have been related to drugs
of abuse (see Nestler133 for review), and there is simply not space Ultimately, the goals of research on drugs and use disorders are to
within this review to cover all of these changes. However, interest- gain a better understanding of how drugs act on the brain, and
ing information is emerging from examination of drug-induced to develop better approaches for minimizing and treating drug
changes in glutamatergic synaptic transmission within the cir- abuse-related problems. The neurobiological mechanisms dis-
cuitry of interest. Glutamate signaling through G protein–coupled cussed in this chapter indicate that brain mechanisms of reward
receptors called metabotropic glutamate receptors is one system and reinforcement are more complicated than most of us previ-
implicated in these drug-induced changes.67,193 The metabotropic ously imagined in terms of both the circuitry involved and the
glutamate receptors come in a variety of subtypes.34 The different underlying neurochemistry.
metabotropic glutamate receptor subtypes produce diverse cellular The interactions of drugs of abuse with these systems are like-
actions including inhibition and stimulation of neurotransmitter wise complex. It is likely that drugs engage multiple aspects of
release and activation of intracellular signaling pathways that con- the different circuits during the initial phases of exposure. Drug
tribute to changes in neuronal excitability and long-lasting plastic- actions on dopaminergic transmission and other aspects of the
ity of synaptic transmission.34 Group I metabotropic glutamate associative circuit will signal the positive hedonic value of the
receptors are implicated in neuroadaptations to abused drugs, as drug and establish the motivation to continue to take the drug.
well as drug self-administration.67,193 The expression and function At the same time, drug-associated cues and environmental con-
of metabotropic glutamate receptor-5 is downregulated following nections to drug availability will be signaled and learned through
chronic administration of cocaine and withdrawal, but upregu- the limbic circuitry, likely contributing to the sensitized incentive
lated by alcohol exposure (reviewed in Szumlinski et al.193). At the described by Robinson and Berridge.162 It is currently thought
same time, expression of the Homer protein that interacts with that the role of the sensorimotor circuitry in drug seeking and
this receptor and changes its signaling functions undergoes simi- taking develops rather slowly during the course of experience with
lar drug-related upregulation.191,193,194 The group I metabotropic drugs.49,57 However, effects on this circuitry of drugs such as eth-
glutamate receptors are implicated in forms of synaptic plasticity anol occur with acute and short-term chronic exposure.40,144,212
throughout the brain reward/reinforcement circuitry, including Furthermore, the rate of recruitment of this system may depend
endocannabinoid-dependent long-term synaptic depression.67 on the schedule and contingencies of drug availability, especially
Thus, cocaine-induced downregulation of metabotropic gluta- in relation to instrumental behaviors. Furthermore, engagement
mate receptor-5 in brain regions such as the nucleus accumbens of the sensorimotor circuitry likely develops in parallel with
activation of the other circuits, and there may be competition generality of these models to multiple drugs of abuse needs to be
between the different circuits for control of behavior.37 Ulti- demonstrated, and it will be important to determine if the same
mately the involvement of the systems will likely depend on the factors play a role in the development of compulsive use of differ-
pattern of recruitment of different circuit elements. However, ent drugs. It is very likely that no one model will be able to tell
there is emerging evidence that reinforcing effects of drugs ulti- us all that we need to know to understand maladaptive drug use
mately lead to involvement of the sensorimotor circuitry and and addiction. Thus it is important to continue to cultivate useful
establishment of drug-related habits.14,49,57,219 One intriguing models and develop new ones, always with an eye toward deter-
scenario is a shift from a purely stimulus-outcome–based mode mining the underlying neurobiology.
of responding to one that also includes stimulus-response–based In considering the role of neural circuitry in drug reward, rein-
actions. Thus a drug user may initially come to associate certain forcement, and addiction, a more inclusive approach will likely be
cues with the rewarding effects of a drug, and this may initially needed. Basic neuroscientific research is now revealing multiple
drive drug seeking. With continued drug exposure, the stimulus- parallel brain systems for control of different aspects of action pro-
response circuitry becomes progressively more engaged as drugs duction and action learning. Movement beyond the monolithic
reinforce the stimulus/environment control of drug seeking and concept of a single neural reward/reinforcement system is neces-
use. Indeed, the work of Everitt and colleagues appears to support sary. It is likely that these systems will have similar roles in behav-
such a transition from limbic to sensorimotor circuit control of ior directed toward drugs of abuse, and this must be considered
drug-related actions.14,49 It remains to be seen how action-out- in designing experiments aimed at determining the neural basis of
come–based learning plays a role in this scenario, but this form of drug seeking and use.
learning will almost certainly play a role in drug use and abuse in At the cellular and molecular levels, a number of changes
humans. One possibility is that action-outcome–based learning brought about by drug exposure and related conditioning have
may drive the acquisition of new behaviors that are designed to been described. However, little is known about the role of most of
locate and obtain drugs of abuse, providing another mechanism these changes in the development of maladaptive drug use and/or
through which incentive sensitization takes over brain function addiction. Consideration of the wide variety of neurotransmitters
to promote drug seeking and use. Brain mechanisms involved in and receptors that participate in normal and abnormal functioning
stress responsivity and production of aversive responses to drug of the relevant brain circuitry is especially important. Experimen-
withdrawal also become progressively more involved as drug use tal approaches designed to manipulate particular molecules within
and abuse continue, and these circuits likely interact with mecha- given cell types and circuits (e.g., local drug application, discon-
nisms of reward and reinforcement to promote relapse to drug nection analyses, opto- and chemogenetic techniques, and the use
use (Fig. 16.1). of sophisticated genetically manipulated mice) will play an ever-
The effects of stress on the prefrontal cortex are of interest in increasing role in our quest to determine which molecular and cel-
this regard. Studies showing that exposure to acute or chronic lular changes are important and which are merely epiphenomena
stress and corticosterone treatment stunts the dendritic morphol- or secondary to the truly causal changes. The powerful tools for
ogy of neurons in the medial prefrontal cortex,28,84,153 suggesting genetic and circuit manipulation, molecular analysis, examination
that the circuitry involving this brain region is impaired during of neurophysiology and neurochemistry at the in vitro and in vivo
stress. These dendritic stunting effects have been postulated to levels, and ever more sophisticated behavioral analysis in a variety
impair executive decision-making capabilities.162 One result of of organisms should allow investigators to make rapid progress in
this neurotoxic stress effect may be to drive behavior away from this area in the coming years.
goal-directed actions and toward habitual responding.51 Thus
stress may act on the reward/reinforcement circuitry at a number
of levels to promote inflexible drug seeking and use. Ultimately, References
drug use and abuse are initiated and sustained by a number of
intrinsic neural mechanisms including goal-directed behavior, 1. Adams CD, Dickinson A. Instrumental responding following rein-
environmentally stimulated instrumental behavior, habitization of forcer devaluation. Q J Exp Psychol. 1981;33B:109–121.
drug-related responses, and negative reinforcement/allostasis that 2. Ahmed SH, Kenny PJ, Koob GF, Markou A. Neurobiological evi-
promotes a return to drug use. dence for hedonic allostasis associated with escalating cocaine use.
Nat Neurosci. 2002;5:625–626.
Determining the relative roles of these different neural mecha- 3. Alexander GE, Delong MR, Strick PL. Parallel organization of seg-
nisms and neural circuits in drug use, abuse, and addiction will regated circuits linking basal ganglia and cortex. Ann Rev Neurosci.
require new avenues of research at the molecular, cellular, sys- 1986;9:357–381.
tems, and behavioral levels. Ultimately, an integrative approach 4. American Psychiatric Association. Diagnostic and Statistical Manual
that incorporates all of these levels of analysis is needed. Behav- of Mental Disorders V: Substance-Related and Addictive Disorders. 5th
ioral models of different phenotypes relevant to drug reward, ed. Washington, DC: American Psychiatric Publishing; 2016.
reinforcement, seeking, and taking need to be used with an eye 5. Argilli E, Sibley DR, Malenka RC, England PM, Bonci A. Mecha-
to determining the roles of the underlying changes in circuits, nism and time course of cocaine-induced long-term potentiation in
cells, and molecules. In using these models it will be necessary the ventral tegmental area. J Neurosci. 2008;28(37):9092–1000.
to bear in mind all of the possible processes that contribute to 6. Aston-Jones G, Harris GC. Brain substrates for increased drug seeking
during protracted withdrawal. Neuropharmacol. 2004;47(suppl 1):
drug use, relapse, and addiction, including goal-directed behav- 167–179.
ior, cue-related conditioning, avoidance of aversive consequences, 7. Atallah HE, Lopez-Paniagua D, Rudy JW, O’Reilly RC. Separate
incubation, and development of habitual actions. The develop- neural substrates for skill learning and performance in the ventral
ment of models that allow investigators to select animals that are and dorsal striatum. Nat Neurosci. 2007;10(1):126–131.
highly sensitive to compulsive drug use should provide the oppor- 8. Azrin NH, Holz WC. Punishment. In: Honig WK, ed. Operant
tunity to examine genetic, epigenetic, and environmental factors Behavior: Areas of Research and Application. New York: Appleton-
that have predispositional effects in these animals. However, the Century-Crofts; 1966:380–447.
9. Backstrom P, Bachteler D, Koch S, Hyytia P, Spanagel R. mGluR5 31. Childress AR, Mozley PD, McELgin W, Fitzgerald J, Reivich M,
antagonist MPEP reduces ethanol-seeking and relapse behavior. O’Brien CP. Limbic activation during cue-induced cocaine crav-
Neuropsychopharmacology. 2004;29:921–928. ing. Am J Psychiatry. 1999;156:11–18.
10. Bain GT, Kornetsky C. Ethanol oral self administration and 32. Colombo G, Orrù A, Lai P, Cabras C, Maccioni P, Rubio M,
rewarding brain stimulation. Alcohol. 1989;6(6):499–503. Gessa GL, Carai MA. The cannabinoid CB1 receptor antago-
11. Baker EJ, Farro J, Gonzales S, Helms C, Grant KA. Chronic alco- nist, rimonabant, as a promising pharmacotherapy for alcohol
hol self-administration in monkeys shows long-term quantity/ dependence: preclinical evidence. Mol Neurobiol. 2007;36(1):
frequency categorical stability. Alcohol Clin Exp Res. 2014;38(11): 102–112.
2835–284. 33. Colwill RM, Rescorla RA. Effect of reinforcer devaluation on dis-
12. Balleine BW. Neural bases of food-seeking: affect, arousal criminative control of instrumental behavior. J Exp Psychol Anim
and reward in corticostriatolimbic circuits. Physiol Behav. Behav Proc. 1990;16(1):40–47.
2005;86(5):717–730. 34. Conn PJ, Pin JP. Pharmacology and functions of metabotropic glu-
13. Becker HC, Ron D. Animal models of excessive alcohol con- tamate receptors. Ann Rev Pharmacol Toxicol. 1997;37:205–237.
sumption: recent advances and future challenges. Alcohol. 35. Contet C, Kieffer BL, Befort K. Mu opioid receptor: a gateway to
2014;48(3):205–208. drug addiction. Curr Opin Neurobiol. 2004;J14(3):370–378.
14. Belin D, Everitt BJ. Cocaine seeking habits depend upon dopa- 36. Corbett D, Wise RA. Intracranial self-stimulation in relation to the
mine-dependent serial connectivity linking the ventral with the ascending dopaminergic systems of the midbrain: a moveable elec-
dorsal striatum. Neuron. 2008;57(3):432–441. trode mapping study. Brain Res. 1980;85(1):1–15.
15. Belin D, Mar AC, Dalley JW, Robbins TW, Everitt BJ. High 37. Corbit LH, Janak PH. Habitual alcohol seeking: neural bases and
impulsivity predicts the switch to compulsive cocaine-taking. Sci- possible relations to alcohol use disorders. Alcohol Clin Exp Res.
ence. 2008;320(5881):1352–1355. 2016;40(7):1380–13837.
16. Berridge KC. Food reward: brain substrates of wanting and liking. 38. Corbit LH, Janak PH, Balleine BW. General and outcome-specific
Neurosci Biobehav Rev. 1996;20:1–25. forms of Pavlovian-instrumental transfer: the effect of shifts in
17. Berridge KC, Robinson TE. What is the role of dopamine in motivational state and inactivation of the ventral tegmental area.
reward: hedonic impact, reward learning, or incentive salience? Eur J Neurosci. 2007;26(11):3141–3149.
Brain Res Rev. 1998;28(3):309–369. 39. Cornish J, Kalivas P. Glutamate transmission in the nucleus
18. Boakes RA. Self-starvation in the rat: running versus eating. Span J accumbens mediates relapse in cocaine addiction. J Neurosci.
Psychol. 2007;10(2):251–257. 2000;20(RC89):81–85.
19. Bonson KR, Grant SJ, Contoreggi CS, et al. Neural systems 40. Cuzon Carlson VC, Seabold GK, Helms CM, et al. Synaptic and
and cue-induced cocaine craving. Neuropsychopharmacology. morphological neuroadaptations in the putamen associated with
2002;26:376–386. long-term, relapsing alcohol drinking in primates. Neuropsycho-
20. Boutrel B. A neuropeptide-centric view of psychostimulant addic- pharm. 2011;36(12):2513–2528.
tion. Br J Pharmacol. 2008;154(2):343–357. 41. Dalley JW, Laane K, Theobald DE, Armstrong HC, Corlett PR,
21. Brabano MF, Cador M. Opioids for hedonic experience Chudasama Y, Robbins TW. Time-limited modulation of appeti-
and dopamine to get ready for it. Psychopharmacology (Berl). tive Pavlovian memory by D1 and NMDA receptors in the nucleus
2007;191(3):497–506. accumbens. Proc Natl Acad Sci USA. 2005;102(17):6189–6194.
22. Broekkamp CLE, Phillips AG, Cools AR. Facilitation of self- 42. Dayan P, Balleine BW. Reward, motivation, and reinforcement
stimulation behavior following intracerebral microinjections of learning. Neuron. 2002;36(2):285–298.
opioids into the ventral tegmental area. Pharmacol Biochem Behav. 43. Deroche V, Caine SB, Heyser CJ, Polis I, Koob GF, Gold LH.
1979;11:289–295. Differences in the liability to self-administer intravenous cocaine
23. Brown EE, Fibiger HC. Differential effects of excitotoxic lesions between C57BL/6 x SJL and BALB/cByJ mice. Pharmacol Biochem
of the amygdala on cocaine-induced conditioned locomotion Behav. 1997;57(3):429–440.
and conditioned place preference. Psychopharmacology (Berl). 44. Deroche-Gamonet V, Belin D, Piazza PV. Evidence for addiction-
1993;113:123–130. like behavior in the rat. Science. 2004;305(5686):1014–1017.
24. Brunton L, Lazo J, Parker K. Goodman & Gilman’s the Pharmaco- 45. Dickinson A, Nicholas DJ, Adams CD. The effect of the instru-
logical Basis of Therapeutics. New York: McGraw-Hill; 2005. mental training contingency on susceptibility to reinforcer devalu-
25. Caprioli D, Venniro M, Zhang M, Bossert JM, Warren BL, Hope ation. Q J Exp Psychol. 1983;35B:35–51.
BT, Shaham Y. Role of dorsomedial striatum neuronal enembles 46. Dickinson A, Wood N, Smith JW. Alcohol seeking by rats: action
in incubation of methamphetamine craving after voluntary absti- or habit? Q J Exp Psychol B. 2002;55(4):331–348.
nence. J Neurosci. 2017;37(4):1014–1027. 47. Divac I, Rosvold HE, Szwarcbart MK. Behavioral effects of
26. Caprioli D, Venniro M, Zeric T, et al. Effect of the novel posi- selective ablation of the caudate nucleus. J Comp Physiol Psychol.
tive allosteric modulator of metabotropic glutamate receptor 2 1967;63:184–190.
AZD8529 on incubation of methamphetamine craving after 48. Donahoe J, Palmer D, Burgos J. The unit of selection: what do
prolonged voluntary abstinence in a rat model. Biol Pychiatry. reinforcers reinforce? J Exp Anal Behav. 1997;67(2):259–273.
2015;78(7):463–473. 49. Everitt BJ, Robbins TW. Neural systems of reinforcement for drug
27. Carlezon Jr WA, Nestler EJ. Elevated levels of GluR1 in the mid- addiction: from actions to habits to compulsion. Nat Neurosci.
brain: a trigger for sensitization to drugs of abuse? Trends Neurosci. 2005;8(11):1481–1489.
2002;25:610–615. 50. Feltenstein MW, See RE. The neurocircuitry of addiction: an over-
28. Cerqueira JJ, Pêgo JM, Taipa R, Bessa JM, Almeida OF, Sousa view. Br J Pharmacol. 2008;154(2):261–274.
N. Morphological correlates of corticosteroid-induced changes in 51. Ferreira ED, Sousa JC, Melo I, et al. Chronic stress causes fron-
prefrontal cortex-dependent behaviors. J Neurosci. 2005;25(34): tostriatal reorganization and affects decision-making. Science.
7792–7800. 2009;325(5940):621–625.
29. Cheer JF, Wassum KM, Sombers LA, et al. Phasic dopamine release 52. Fourgeaud L, Mato S, Bouchet D, Hémar A, Worley PF, Manzoni
evoked by abused substances requires cannabinoid receptor activa- OJ. A single in vivo exposure to cocaine abolishes endocannabi-
tion. J Neurosci. 2007;27(4):791–795. noid-mediated long-term depression in the nucleus accumbens. J
30. Childress AR, Ehrman RN, Rohsenow D, Robbins SJ, O’Brien Neurosci. 2004;24(31):6939–6945.
CP. Classically conditioned factors in drug dependence. In: Lowin- 53. Frederickson RC, Geary LE. Endogenous opioid peptides: review
son J, Ruiz P, Millman R, eds. Comprehensive Textbook of Substance of physiological, pharmacological and clinical aspects. Prog Neuro-
Abuse. Baltimore: Williams & Wilkins; 1993:56–69. biol. 1982;19(1–2):19–69.
54. Frye GD, Breese GR. An evaluation of the locomotor stimu- 76. Hoffman AF, Oz M, Caulder T, Lupica CR. Functional toler-
lating action of ethanol in rats and mice. Psychopharmacology. ance and blockade of long-term depression at synapses in the
1981;75:372–379. nucleus accumbens after chronic cannabinoid exposure. J Neurosci.
55. Gaveriaux-Ruff C, Kieffer BL. Opioid receptor genes inactivated in 2003;23(12):4815–4820.
mice: the highlights. Neuropeptides. 2002;36:62–71. 77. Hohmann AG, Herkenham JM. Localization of cannabinoid CB1
56. Gawin FH, Kleber HD. Abstinence symptomatology and psychi- receptor mRNA in neuronal subpopulations of rat striatum: a dou-
atric diagnosis in cocaine abusers: clinical observations. Arch Gen ble-label in situ hybridization study. Synapse. 2000;37:71–80.
Psychiatry. 1986;43:107–113. 78. Holman EW. Some conditions for the dissociation of consum-
57. Gerdeman GL, Partridge JG, Lupica CR, Lovinger DM. It could matory and instrumental behavior in rats. Learn Motiv. 1975;6:
be habit forming: drugs of abuse and striatal synaptic plasticity. 358–366.
Trends Neurosci. 2003;26(4):184–192. 79. Howell LL, Kimmel HL. Monoamine transporters and psycho-
58. Gerdeman GL, Schechter JB, French ED. Endocannabinoid sig- stimulant addiction. Biochem Pharmacol. 2008;75(1):196–217.
naling at striatal CB1 receptors is critical for the consolidation of 80. Hsu PD, Lander ES, Zhang F. Development and applications
stimulus-response memories. 2007 symposium on the cannabi- of CRISPR-Cas9 for genome engineering. Cell. 2014;157(6):
noids. Burlington, Vermont: International Cannabinoid Research 1262–1278.
Society; 2007:123. www.cannabinoidsociety.org. 81. Hull CL. Principles of Behavior. New York: Appleton-Century-
59. Giros B, Jaber M, Jones SR, Wightman RM, Caron MG. Hyper- Crofts; 1943.
locomotion and indifference to cocaine and amphetamine in mice 82. Hunnicutt BJ, Jongbloets BC, Birdsong WT, Gertz KJ, Zhong
lacking the dopamine transporter. Nature. 1996;379:606–612. H, Mao T. A comprehensive excitatory input map of the striatum
60. Goodman A. Neurobiology of addiction. An integrative review. reveals novel functional organization. Elife. 2016;28:5.
Biochem Pharmacol. 2008;75(1):266–322. 83. Hyman SE, Malenka RC, Nestler EJ. Neural mechanisms of addic-
61. Grace AA, Floresco SB, Goto Y, Lodge DJ. Regulation of firing tion: the role of reward-related learning and memory. Annu Rev
of dopaminergic neurons and control of goal-directed behaviors. Neurosci. 2006;29:565–598.
Trends Neurosci. 2007;30:220–227. 84. Izquierdo A, Wellman CL, Holmes A. Brief uncontrollable stress
62. Grahame NJ, Cunningham CL. Intravenous ethanol self- causes dendritic retraction in infralimbic cortex and resistance to
administration in C57BL/6 J and DBA/2 J mice. Alcohol Clin Exp fear extinction in mice. J Neurosci. 2006;26(21):5733–5738.
Res. 1997;21:56–62. 85. Jedynak JP, Uslaner JM, Esteban JA, Robinson TE. Methamphet-
63. Gratton A. In vivo analysis of the role of dopamine in stimulant amine-induced structural plasticity in the dorsal striatum. Eur J
and opiate self-administration. J Psychiatry Neurosci. 1996;21(4): Neurosci. 2007;25(3):847–853.
264–279. 86. Jimenez VA, Grant KA. Studies using macaque monkeys to address
64. Gremel CM, Chancey JH, Atwood BK, Luo G, Neve R, Ramak- excessive alcohol drinking and stress interactions. Neuropharmacol-
rishnan C, Deisseroth K, Lovinger DM, Costa RM. Endocannabi- ogy. 2017;122:127–135.
onoid modulation of orbitostriatal circuits gates habit formation. 87. Jinek M, Chylinski K, Fonfara I, Hauer M, Doudna JA, Charpen-
Neuron. 2016;90(6):1312–1324. tier E. A programmable dual-RNA-guided DNA endonuclease in
65. Gremel CM, Cunningham CL. Roles of the nucleus accumbens and adaptive bacterial immunity. Science. 2012;337(6096):816–821.
amygdala in the acquisition and expression of ethanol-conditioned 88. Johnson AW, Bannerman D, Rawlins N, Sprengel R, Good MA.
behavior in mice. J Neurosci. 2008;28(5):1076–1084. Targeted deletion of the GluR-1 AMPA receptor in mice dissociates
66. Grueter BA, McElligott ZA, Robison AJ, Mathews GC, Winder general and outcome-specific influences of appetitive rewards on
DG. In vivo metabotropic glutamate receptor 5 (mGluR5) antag- learning. Behav Neurosci. 2007;121(6):1192–1202.
onism prevents cocaine-induced disruption of postsynaptically 89. Jones S, Bonci A. Synaptic plasticity and drug addiction. Curr Opin
maintained mGluR5-dependent long-term depression. J Neurosci. Pharmacol. 2005;5(1):20–25.
2008;28(37):9261–9270. 90. Justinova Z, Panlilio LB, Secci ME, et al. The novel metabo-
67. Grueter BA, McElligott ZA, Winder DG. Group I mGluRs and tropic glutamate receptor positive allosteric modulator, ASD8529,
long-term depression: potential roles in addiction? Mol Neurobiol. decreases nicotine self-administration and relapse in squirrel mon-
2007;36(3):232–244. keys. Biol Psychiatry. 2015;78(7):452–462.
68. Haber SN, Fudge JL, McFarland NR. Striatonigrostriatal pathways 91. Kaffman A, Krystal JH. New frontiers in animals research of psy-
in primates form an ascending spiral from the shell to the dorsolat- chiatric illness. Methods Mol Biol. 2012;829:3–30.
eral striatum. J Neurosci. 2000;20:2369–2382. 92. Kahlig KM, Galli A. Regulation of dopamine transporter function
69. Hayward MD, Pintar JE, Low MJ. Selective reward deficit and plasma membrane expression by dopamine, amphetamine, and
in mice lacking beta-endorphin and enkephalin. J Neurosci. cocaine. Eur J Pharmacol. 2003;479(1–3):153–158.
2002;22(18):8251–8258. 93. Kalivas PW, McFarland K. Brain circuitry and the reinstatement of
70. Hemby SE, Johnson BA, Dworkin SI. Neurobiological basis of cocaine-seeking behavior. Psychopharmacology (Berl). 2003;168(1–2):
drug reinforcement. In: Johnson BA, Roache JD, eds. Drug Addic- 44–56.
tion and Treatment, Nexus of Neuroscience and Behavior. Philadel- 94. Kalivas PW, O’Brien C. Drug addiction as a pathology of staged
phia: Lippincott Raven; 1997:2–19. neuroplasticity. Neuropsychopharmacology. 2008;33(1):166–180.
71. Herkenham M, Lynn AB, Johnson MR, Melvin LS, de Costa BR, 95. Kandel ER, Schwartz JH, Jessell TM. Principles of Neural Science.
Rice KC. Characterization and localization of cannabinoid recep- New York: McGraw-Hill; 2000.
tors in rat brain: a quantitative in vitro autoradiographic study. J 96. Kelley AE. Ventral striatal control of appetitive motivation: role in
Neurosci. 1991;11:563–583. ingestive behavior and reward-related learning. Neurosci Biobehav
72. Hilário MRF, Clouse E, Yin HH, Costa RM. Endocannabinoid Rev. 2004;27(8):765–776.
signaling is critical for habit formation. Front Integr Neurosci. 97. Kenny PJ, Chen SA, Kitamura O, Markou A, Koob GF. Condi-
2007;1:1–12. tioned withdrawal drives heroin consumption and decreases reward
73. Hintiryan H, Foster NN, Bowman I, et al. The mouse cortico- sensitivity. J Neurosci. 2006;26(22):5894–5900.
striatal projectome. Nat Neurosci. 2016;19:1100–1114. 98. Konorski J. Integrative Activity of the Brain. Chicago: University of
74. Hnasko TS, Sotak BN, Palmiter RD. Morphine reward in dopa- Chicago; 1967.
mine-deficient mice. Nature. 2005;438:854–857. 99. Koob G. The neurobiology of addiction: a neuroadaptational view
75. Hnasko TS, Sotak BN, Palmiter RD. Cocaine-conditioned place relevant for diagnosis. Addiction. 2006;101(suppl 1):23–30.
preference by dopamine-deficient mice is mediated by serotonin. J 100. Koob G. Neurobiological substrates for the dark side of compulsiv-
Neurosci. 2007;27(46):12484–12488. ity in addiction. Neuropharmacology. 2008;56(suppl 1):18–31.
101. Koob G, Kreek MJ. Stress, dysregulation of drug reward path- 124. Mojica FJ, Montoliu L. On the origin of CRISPR-Cas technol-
ways, and the transition to drug dependence. Am J Psychiatry. ogy: from prokaryotes to mammals. Trends Microbiol. 2016;24(10):
2007;164(8):1149–1159. 811–820.
102. Koob G, Le Moal M. Addiction and the brain antireward system. 125. Moolten M, Kornetsky C. Oral self-administration of ethanol and
Annu Rev Psychol. 2008;59:29–53. not experimenter-administered ethanol facilitates rewarding electri-
103. Kornetsky C, Bain GT, Unterwald EM, Lewis MJ. Brain cal brain stimulation. Alcohol. 1990;7:221–225.
stimulation reward: effects of ethanol. Alcohol Clin Exp Res. 126. Moore BR. The evolution of learning. Biol Rev. 2004;79:301–335.
1988;12(5):609–616. 127. Moussawi K, Kalivas PW. Group II metabotropic glutamate recep-
104. Kuhar MJ, Sanchez-Roa PM, Wong DF, et al. Dopamine trans- tors (mGlu2/3) in drug addiction. Eur J Pharmacol. 2010;639(1–3):
porter: biochemistry, pharmacology and imaging. Eur Neurol 115–122.
Suppl. 1990;1:15–20. 128. Mulholland PJ, Chandler LJ. The thorny side of addiction: adap-
105. Kumar S, Fleming RL, Morrow AL. Ethanol regulation of gamma- tive plasticity and dendritic spines. Sci World J. 2007;7:9–21.
aminobutyric acid A receptors: genomic and nongenomic mecha- 129. Müller CP, Carey RJ, Huston JP, De Souza Silva MA. Serotonin
nisms. Pharmacol Ther. 2004;101(3):211–226. and psychostimulant addiction: focus on 5-HT1A-receptors. Prog
106. Kumar V, Kim K, Joseph C, et al. C57BL/6N mutation in the cyto- Neurobiol. 2007;81(3):133–178.
plasmic FMRP interactions protein 2 regulates cocaine response. 130. Mulligan MK, Ponomarev I, Boehm 2nd SL, et al. Alcohol trait
Science. 2013;342(6165):1508–1512. and transcriptional genomic analysis of C57BL/6 substrains. Genes
107. LeDoux J. The amygdala. Curr Biol. 2007;17(20):R868–R874. Brain Behav. 2008;7(6):677–689.
108. Leventhal AM, Kahler CW, Ray LA, Stone K, Young D, Chelmin- 131. Nazzaro C, Greco B, Cerovic M, et al. SK channel modulation
ski I, Zimmerman M. Anhedonia and amotivation in psychiatric rescues striatal plasticity and control over habit in cannabinoid tol-
outpatients with fully remitted stimulant use disorder. Am J Addict. erance. Nat Neurosci. 2012;15(2):284–293.
2008;17(3):218–223. 132. Nazzaro JM, Seeger TF, Gardner EL. Morphine differentially
109. Lovinger DM. Presynaptic modulation by endocannabinoids. affects ventral tegmental and substantia nigra brain reward thresh-
Handb Exp Pharmacol. 2008;184:435–477. olds. Pharmacol Biochem Behav. 1981;14(3):325–331.
110. Löw K, Crestani F, Keist R, et al. Molecular and neuro- 133. Nestler EJ. Molecular neurobiology of addiction. Am J Addict.
nal substrate for the selective attenuation of anxiety. Science. 2001;10(3):201–217.
2000;290(5489):131–134. 134. Nicoll RA, Alger BE. The brain’s own marijuana. Sci Am.
111. Maldonado R. Study of cannabinoid dependence in animals. Phar- 2004;291(6):68–75.
macol Ther. 2002;2:153–164. 135. Oades RD, Halliday GM. Ventral tegmental (A10) system: neu-
112. Maldonado R, Valverde O, Berrendero F. Involvement of the robiology. 1. Anatomy and connectivity. Brain Res. 1987;434(2):
endocannabinoid system in drug addiction. Trends Neurosci. 117–165.
2006;29(4):225–232. 136. Olds J, Milner P. Positive reinforcement produced by electrical
113. Masur J, Oliveira De Souza ML, Zwicker AP. The excitatory effect stimulation of septal area and other regions of rat brain. J Comp
of ethanol: absence in rats, no tolerance and increased sensitivity in Physiol Psychol. 1954;47(6):4194–4127.
mice. Pharmacol Biochem Behav. 1986;24:1225–1228. 137. Olmstead MC, Lafond MV, Everitt BJ, Dickinson A. Cocaine
114. Matthes HWD, Maldonado R, Simonin F, et al. Loss of mor- seeking by rats is a goal-directed action. Behav Neurosci.
phine-induced analgesia, reward effect and withdrawal symptoms 2001;115(2):394–402.
in mice lacking the μ-opioid receptor gene. Nature. 1996;383: 138. Ostlund SB, Balleine BW. Orbitofrontal cortex mediates outcome
819–823. encoding in Pavlovian but not instrumental conditioning. J Neuro-
115. Mato S, Chevaleyre V, Robbe D, Pazos A, Castillo PE, Manzoni sci. 2007;27(18):4819–4825.
OJ. A single in-vivo exposure to delta 9THC blocks endocan- 139. Ostlund SB, Balleine BW. Differential involvement of the baso-
nabinoid-mediated synaptic plasticity. Nat Neurosci. 2004;7(6): lateral amygdala and mediodorsal thalamus in instrumental action
585–586. selection. J Neurosci. 2008;28(17):4398–4405.
116. Mato S, Robbe D, Puente N, Grandes P, Manzoni OJ. Presyn- 140. Ostlund SB, Balleine BW. Theories of goal-directed behavior. In:
aptic homeostatic plasticity rescues long-term depression after Squire LR, ed. Encyclopedia of Neuroscience. Vol. 4. Oxford: Aca-
chronic Delta 9-tetrahydrocannabinol exposure. J Neurosci. demic Press; 2009:943–949.
2005;25(50):11619–11627. 141. Palmiter RD. Dopamine signaling in the dorsal striatum is essen-
117. McEwen BS, Wingfield JD. The concept of allostasis in biology and tial for motivated behaviors: lessons from dopamine-deficient mice.
biomedicine. Horm Behav. 2003;43(1):2–15. Ann NY Acad Sci. 2008;1129:35–46.
118. McGregor CM, Srisurapanont M, Jittiwutikarn J, Laobhripatr 142. Panlilio LV, Goldberg SR. Self-administration of drugs in ani-
S, Wongtan T, White JM. The nature, time course and sever- mals and humans as a model and an investigative tool. Addiction.
ity of methamphetamine withdrawal. Addiction. 2005;100: 2007;102(12):1863–1870.
1300–1329. 143. Pattij T, Vanderschuren LJ. The neuropharmacology of impulsive
119. Melis M, Camarini R, Ungless MA, Bonci A. Long-lasting poten- behaviour. Trends Pharmacol Sci. 2008;29(4):192–199.
tiation of GABAergic synapses in dopamine neurons after a single 144. Patton MH, Roberts BM, Lovinger DM, Mathur BN. Ethanol
in vivo ethanol exposure. J Neurosci. 2002;22(6):2074–2082. disinhibits dorsolateral striatal medium spiny neurons through
120. Meinhardt MW, Hansson AC, Perreau-Lenz S, et al. Res- activation of a presynaptic delta opioid receptor. Neuropsychophar-
cue inhibition of infralimbic mGluR2 deficit restores control macology. 2016;41(7):1831–1840.
over drug-seeking behavior in alcohol dependence. J Neurosci. 145. Pelloux Y, Everitt BJ, Dickinson A. Compulsive drug seeking by
2013;33(7):2794–2806. rats under punishment: effects of drug taking history. Psychophar-
121. Mihalek RM, Bowers BJ, Wehner JM, et al. GABA(A)-receptor macology (Berl). 2007;194(1):127–137.
delta subunit knockout mice have multiple defects in behavioral 146. Petry NM, Heyman GM. Behavioral economics of concurrent eth-
responses to ethanol. Alcohol Clin Exp Res. 2001;25:1708–1718. anol-sucrose and sucrose reinforcement in the rat: effects of altering
122. Miles FJ, Everitt BJ, Dalley JW, Dickinson A. Conditioned activity variable ratio requirements. J Exper Anal Behav. 1995;64:331–359.
and instrumental reinforcement following long-term oral consump- 147. Pettit HO, Ettenberg A, Bloom FE, Koob GF. Destruction of
tion of cocaine by rats. Behav Neurosci. 2004;118(6):1331–1339. dopamine in the nucleus accumbens selectively attenuates cocaine
123. Miles FJ, Everitt BJ, Dickinson A. Oral cocaine seeking by rats: but not heroin self-administration in rats. Psychopharmacology.
action or habit? Behav Neurosci. 2003;117(5):927–938. 1984;84:167–173.
148. Pickens CL, Airavaara M, Theberge F, Fanous S, Hope BT, Sha- 169. Samson HH, Cunningham CL, Czachowski CL, Chappell A, Legg
ham Y. Neurobiology of the incubation of drug craving. Trends B, Shannon E. Devaluation of ethanol reinforcement. Alcohol.
Neurosci. 2011;34(8):411–420. 2004;32(3):203–212.
149. Pfaff DW, Kieffer BL, Swanson LW. Mechanisms for the regula- 170. Samson HH, Czachowski CL. Behavioral measures of alcohol self-
tion of state changes in the central nervous system: an introduction. administration and intake control: rodent models. Int Rev Neuro-
Ann NY Acad Sci. 2008;1129:1–7. biol. 2003;54:107–143.
150. Phillips AG, Fibiger HC. Dopaminergic and noradrenergic sub- 171. Samson HH, Tolliver GA, Haraguchi M, Hodge CW. Alcohol self-
strates of positive reinforcement: differential effects of d- and administration: role of mesolimbic dopamine. Ann NY Acad Sci.
l-amphetamine. Science. 1973;179(73):575–577. 1992;654:242–253.
151. Pierce RC, Kumaresan V. The mesolimbic dopamine system: the 172. Sanchez FP, Dickenson L, George FR. Ethanol self-administration
final common pathway for the reinforcing effect of drugs of abuse? is genetically independent of locomotor stimulation in fast and
Neurosci Biobehav Rev. 2006;30(2):215–238. slow mice. Alcohol. 1996;13(1):79–84.
152. Pitkänen A. Connectivity of the rat amygdaloid complex. In: 173. Scheyer AF, Wolf ME, Tseng KY. A protein synthesis-dependent
Aggleton JP, ed. The Amygdala: a Functional Analysis. Oxford: Uni- mechanism sustaines calcium-permeable AMPA receptor trans-
versity Press; 2000:31–115. mission in nucleus accumbens synapses during withdrawal from
153. Radley JJ, Sisti HM, Hao J, Rocher AB, McCall T, Hof PR, McE- cocaine self-administration. J Neurosci. 2014;34(8):3095–3100.
wen BS, Morrison JH. Chronic behavioral stress induces apical 174. Schulteis G, Markou A, Cole M, Koob GF. Decreased brain
dendritic reorganization in pyramidal neurons of the medial pre- reward produced by ethanol withdrawal. Proc Natl Acad Sci USA.
frontal cortex. Neuroscience. 2004;125:1–6. 1995;92(13):5880–5884.
154. Rassnick S, Stinus L, Koob GF. The effects of 6-hydroxydopamine 175. Schultz W. Predictive reward signal of dopamine neurons. J Neuro-
lesions of the nucleus accumbens and the mesolimbic dopamine physiol. 1998;80:1–27.
system on oral self-administration of ethanol in the rat. Brain Res. 176. Schultz W. Getting formal with dopamine and reward. Neuron.
1993;623:16–24. 2002;36:241–263.
155. Rescorla RA, Wagner AR. A theory of Pavlovian conditioning: 177. See RE. Neural substrates of cocaine-cue associations that trigger
variations in the effectiveness of reinforcement and nonreinforce- relapse. Eur J Pharmacol. 2005;526(1–3):140–146.
ment. In: Black AH, Prokasy WF, eds. Classical Conditioning II: 178. Sellings LH, McQuade LE, Clarke PB. Evidence for multiple
Current Research and Theory. New York: Appleton-Century-Crofts; sites within rat ventral striatum mediating cocaine-conditioned
1972:64–99. place preference and locomotor activation. J Pharmacol Exp Ther.
156. Reynolds JN, Hyland BI, Wickens JR. A cellular mechanism of 2006;317(3):1178–1187.
reward-related learning. Nature. 2001;413(6851):67–70. 179. Shabat-Simon M, Levy D, Amir A, Rehavi M, Zangen A. Dis-
157. Richardson NR, Roberts DC. Progressive ratio schedules in drug sociation between rewarding and psychomotor effects of opiates:
self-administration studies in rats: a method to evaluate reinforcing differential roles for glutamate receptors within anterior and pos-
efficacy. J Neurosci Methods. 1996;66(1):1–11. terior portions of the ventral tegmental area. J Neurosci. 2008;28:
158. Risinger FO, Malott DH, Prather LK, Niehus DR, Cunningham 8406–8416.
CL. Motivational properties of ethanol in mice selectively bred for 180. Skinner BF. The Behavior of Organisms. New York: Appleton-Cen-
ethanol-induced locomotor differences. Psychopharmacology (Berl). tury-Crofts; 1938.
1994;116(2):207–216. 181. Skinner BF. Are theories of learning necessary? Psychol Rev.
159. Robinson S, Rainwater AJ, Hnasko TS, Palmiter RD. Viral res- 1950;57(4):193–216.
toration of dopamine signaling to the dorsal striatum restores 182. Skinner BF. Science and Human Behavior. New York: Macmillan;
instrumental conditioning to dopamine-deficient mice. Psycho- 1953.
pharmacology (Berl). 2007;191(3):567–578. 183. Slifer BL. Schedule-induction of nicotine self-administration.
160. Robinson S, Sandstrom SM, Denenberg VH, Palmiter RD. Dis- Pharmacol Biochem Behav. 1983;19(6):1005–1009.
tinguishing whether dopamine regulates liking, wanting, and/or 184. Smith AD, Bolam JP. The neural network of the basal ganglia as
learning about rewards. Behav Neurosci. 2005;119:5–15. revealed by the study of synaptic connections of identified neu-
161. Robinson S, Sotak BN, During MJ, Palmiter RD. Local dopamine rones. Trends Neurosci. 1990;13(7):259–265.
production in the dorsal striatum restores goal-directed behavior in 185. Smith JB, Klug JR, Howard CD, et al. Genetic-based dissection
dopamine-deficient mice. Behav Neurosci. 2006;120(1):196–200. unveils the inputs and outputs of the striatal patch and matrix com-
162. Robinson TE, Berridge KC. Review. The incentive sensitization partments. Neuron. 2016;91(5):1069–1084.
theory of addiction: some current issues. Philos Trans R Soc Lond B 186. Smith-Roe SL, Kelley AE. Coincident activation of NMDA
Biol Sci. 2008;363(1507):3137–3146. and dopamine D1 receptors within the nucleus accumbens
163. Robinson TE, Kolb B. Structural plasticity associated with exposure core is required for appetitive instrumental learning. J Neurosci.
to drugs of abuse. Neuropharmacology. 2004;47(suppl 1):33–46. 2000;20(20):7737–7742.
164. Rocha BA, Fumagalli F, Gainetdinov RR, et al. Cocaine self- 187. Solomon RL, Corbit JD. An opponent-process theory of motiva-
administration in dopamine-transporter knockout mice. Nat Neu- tion: 1 temporal dynamics of affect. Psychol Rev. 1974;81:119–145.
rosci. 1998;1(2):132–137. 188. Sora I, Wichems C, Takahashi N, et al. Cocaine reward models:
165. Routtenberg A, Kuznesof AW. Self-starvation of rats living in activ- conditioned place preference can be established in dopamine- and
ity wheels on a restricted feeding schedule. J Comp Physiol Psych. in serotonin-transporter knockout mice. Proc Natl Acad Sci USA.
1967;64:414–421. 1998;95:7699–7704.
166. Routtenberg A, Lindy J. Effects of the availability of rewarding 189. Spanagel R. Recent animal models of alcoholism. Alcohol Res
septal and hypothalamic stimulation on bar pressing for food Health. 2000;24(2):124–131.
under conditions of deprivation. J Comp Physiol Psychol. 1965;60: 190. Stuber GD, Hopf FW, Hahn J, Cho SL, Guillory A, Bonci A.
158–161. Voluntary ethanol intake enhances excitatory synaptic strength
167. Sah P, Faber ES, Lopez De Armentia M, Power J. The amygda- in the ventral tegmental area. Alcohol Clin Exp Res. 2008;32(10):
loid complex: anatomy and physiology. Physiol Rev. 2003;83(3): 1714–1720.
803–834. 191. Swanson C, Baker D, Carson D, Worley P, Kalivas P. Repeated
168. Salinas JA, Parent MB, McGaugh JL. Ibotenic acid lesions of the cocaine administration attenuates group I metabotropic glutamate
amygdala basolateral complex or central nucleus differentially effect receptor-mediated glutamate release and behavioral activation: a
the response to reductions in reward. Brain Res. 1996;742:283–293. potential role for Homer 1b/c. J Neurosci. 2001;21:9043–9052.
192. Szabo B, Schlicker E. Effects of cannabinoids on neurotransmis- 207. Wang L, Liu J, Harvey-White J, Zimmer A, Kunos G. Endocan-
sion. Handb Exp Pharmacol. 2005;168:327–365. nabinoid signaling via cannabinoid receptor 1 is involved in etha-
193. Szumlinski KK, Ary AW, Lominac KD. Homers regulate drug- nol preference and its age-dependent decline in mice. Proc Natl
induced neuroplasticity: implications for addiction. Biochem Phar- Acad Sci USA. 2003;100(3):1393–1398.
macol. 2008;75(1):112–133. 208. Weibel SL, Wolf HH. Opiate modification of intracranial self-
194. Szumlinski KK, Lominac KD, Oleson EB, et al. Homer2 is nec- stimulation in the rat. Pharmacol Biochem Behav. 1979;101:71–78.
essary for EtOH-induced neuroplasticity. J Neurosci. 2005;25: 209. West TE, Wise RA. Effects of naltrexone on nucleus accumbens,
7054–7061. lateral hypothalamic and ventral tegmental self-stimulation rate-
195. Thomas MJ, Beurrier C, Bonci A, Malenka RC. Long-term depres- frequency functions. Brain Res. 1988;462(1):126–133.
sion in the nucleus accumbens: a neural correlate of behavioral sen- 210. White NM. Reward or reinforcement: what’s the difference? Neu-
sitization to cocaine. Nat Neurosci. 2001;4(12):1217–1223. rosci Biobehav Rev. 1989;13(2–3):181–186.
196. Thorndike CL. Reward and punishment in animal learning. Comp 211. Wickens JR, Horvitz JC, Costa RM, Killcross S. Dopaminer-
Psychol Monogr. 1932;8:29. gic mechanisms in actions and habits. J Neurosci. 2007;27(31):
197. Todtenkopf MS, Parsegian A, Naydenov A, Neve RL, Konradi 8181–8183.
C, Carlezon Jr WA. Brain reward regulated by AMPA receptor 212. Wilcox MV, Cuzon Carlson VC, Sherazee N, et al. Repeated
subunits in nucleus accumbens shell. J Neurosci. 2006;26(45): binge-like ethanol drinking alters ethanol drinking patterns and
11665–11669. depresses striatal GABAergic transmission. Neuropsychopharmacol-
198. Tzschentke TM. Measuring reward with the conditioned place ogy. 2014;39(3):579–594.
preference paradigm: a comprehensive review of drug effects, recent 213. Wingfield JC. Anniversary essay: control of behavioural strat-
progress and new issues. Prog Neurobiol. 1998;56:613–672. egies for capricious environments. Anim Behav. 2003;66:
199. Ungless MA, Whistler JL, Malenka RC, Bonci A. Single cocaine 807–816.
exposure in vivo induces long-term potentiation in dopamine neu- 214. Wise RA. Catecholamine theories of reward: a critical review. Brain
rons. Nature. 2001;411(6837):583–587. Res. 1978;152(2):215–247.
200. Uwano T, Nishijo H, Ono T, Tamura R. Neuronal responsiveness 215. Wise RA. Addictive drugs and brain stimulation reward. Annu Rev
to various sensory stimuli, and associative learning in the rat amyg- Neurosci. 1996;19:319–340.
dala. Neuroscience. 1995;68:339–361. 216. Wise RA, Bozarth MA. A psychomotor stimulant theory of addic-
201. Vanderschuren LJ, Everitt BJ. Drug seeking becomes com- tion. Psychol Rev. 1987;94(4):469–492.
pulsive after prolonged cocaine self-administration. Science. 217. Wolffgramm J, Heyne A. From controlled drug intake to loss of
2004;305(5686):1017–1019. control: the irreversible development of drug addiction in the rat.
202. Van Dyke C, Byck R. Cocaine. Sci Am. 1982;246:128–141. Behav Brain Res. 1995;70:77–94.
203. van Rijnsoever C, Täuber M, Choulli MK, et al. Requirement 218. Xia JX, Li J, Zhou R, Zhang XH, Ge YB, RuYuan X. Altera-
of alpha5-GABAA receptors for the development of tolerance to tions of rat corticostriatal synaptic plasticity after chronic etha-
the sedative action of diazepam in mice. J Neurosci. 2004;24(30): nol exposure and withdrawal. Alcohol Clin Exp Res. 2006;30(5):
6785–6790. 819–824.
204. Vekovischeva OY, Zamanillo D, Echenko O, et al. Morphine- 219. Yin HH, Knowlton B. The role of the basal ganglia in habit forma-
induced dependence and sensitization are altered in mice defi- tion. Nat Rev Neurosci. 2006;7(6):464–476.
cient in AMPA-type glutamate receptor-A subunits. J Neurosci. 220. Yin HH, Ostlund SB, Balleine BW. Reward-guided learn-
2001;21(12):4451–4459. ing beyond dopamine in the nucleus accumbens: the integra-
205. Venniro M, Caprioli D, Shaham Y. Animal models of drug tive functions of cortico-basal ganglia networks. Eur J Neurosci.
relapse and craving: From drug priming-induced reinstatement to 2008;28(8):1437–1448.
incubation of craving after voluntary abstinence. Prog Brain Res. 221. Zhou Z, Karlsson C, Liang T, et al. Loss of metabotropic glutamate
2016;224:25–52. receptor 2 escalates alcohol consumption. Proc Natl Acad Sci USA.
206. Walker JR, Ahmed SH, Gracy KN, Koob GF. Microinjections 2013;110(42):16963–16968.
of an opiate receptor antagonist into the bed nucleus of the stria 222. Zielinski K. Jerzy Konorski on brain associations. Acta Neurobiol
terminalis suppress heroin self-administration in dependent rats. Exp (Wars). 2006;66(1):75–84.
Brain Res. 2000;854(1–2):85–92.

Neurobiological Basis of Drug Reward and Reinforcement: Chapter Outline

Uploaded by

Copyright:

Available Formats

Neurobiological Basis of Drug Reward and Reinforcement: Chapter Outline

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Neurobiological Basis of Drug Reward and Reinforcement: Chapter Outline

Uploaded by

Copyright:

Available Formats

16

Neurobiological Basis of Drug Reward

C D Amygdala-Basal Ganglia-Cortical Circuit

You might also like