Bogacz Cohen

Psychological Review Copyright 2006 by the American Psychological Association
2006, Vol. 113, No. 4, 700 –765 0033-295X/06/$12.00 DOI: 10.1037/0033-295X.113.4.700
The Physics of Optimal Decision Making: A Formal Analysis of Models of

Performance in Two-Alternative Forced-Choice Tasks
Rafal Bogacz, Eric Brown, Jeff Moehlis, Philip Holmes, and Jonathan D. Cohen
Princeton University
In this article, the authors consider optimal decision making in two-alternative forced-choice (TAFC)
tasks. They begin by analyzing 6 models of TAFC decision making and show that all but one can be
reduced to the drift diffusion model, implementing the statistically optimal algorithm (most accurate for
a given speed or fastest for a given accuracy). They prove further that there is always an optimal trade-off
between speed and accuracy that maximizes various reward functions, including reward rate (percentage
of correct responses per unit time), as well as several other objective functions, including ones weighted
for accuracy. They use these findings to address empirical data and make novel predictions about
performance under optimality.
Keywords: drift diffusion model, reward rate, optimal performance, speed–accuracy trade-off, perceptual
choice
This article concerns optimal strategies for decision making in It has been known since Hernstein’s (1961, 1997) work that
the two-alternative forced-choice (TAFC) task. We present and animals do not achieve optimality under all conditions, and in
compare several decision-making models, briefly discuss their behavioral economics, humans often fail to choose optimally (e.g.,
neural implementations, and relate them to one that is optimal in Kahneman & Tversky, 1984; Loewenstein & Thaler, 1989). For
the sense that it delivers a decision of specified accuracy in the example, in selecting among two items of comparable value and a
shortest possible time: the drift diffusion model (DDM). Further cash payment worth less than either, participants often opt for
definitions of optimality, via objective functions such as reward cash, possibly to avoid a harder decision between two more valu-
rate (RR), are given below, and we provide explicit solutions to the able but similar items (Shafir & Tversky, 1995).
speed–accuracy trade-off for the TAFC task. Such findings raise important questions: Is human decision
making always suboptimal? If it is not, under what conditions is it
optimal? When it is suboptimal, does this reflect inherent deficien-
Rafal Bogacz, Center for the Study of Brain, Mind and Behavior, cies in decision-making mechanisms or other sources of systematic
Princeton University, and Program in Applied and Computational Mathe-
bias? Answers to such questions may provide insights into under-
matics, Princeton University; Eric Brown and Jeff Moehlis, Program in
Applied and Computational Mathematics, Princeton University; Philip
lying neural mechanisms. However, to consider them, one must
Holmes, Program in Applied and Computational Mathematics, Princeton first describe optimal performance, against which actual behavior
University, and Department of Mechanical and Aerospace Engineering, can be compared.
Princeton University; Jonathan D. Cohen, Center for the Study of Brain, Optimality has long been a key principle in the physical sciences
Mind and Behavior, Princeton University, and Department of Psychology, (e.g., energy minimization and related variational methods), and it
Princeton University. has recently begun to see application in the life sciences, including
This work was supported by the following grants: National Institutes of behavioral ecology (Belovsky, 1984; Bull, Metcalfe, & Mangel,
Health Grant P50 MH62196, Department of Energy Grant DE-FG02- 1996; Dunbar, 1984) and neuroscience (e.g., Bialek, Rieke, de
95ER25238 (Philip Holmes), Engineering and Physical Science Research
Ruyter van Steveninck, & Warland, 1991; Levine & Elsberry,
Council Grant EP/C514416/1, National Science Foundation Mathematical
Sciences Postdoctoral Research Fellowship (held by Jeff Moehlis and Eric
1997; Olshausen & Field, 1996). Optimality is also a cornerstone
Brown), and the Burroughs-Wellcome Program in Biological Dynamics of standard economic theory and its application to human decision
and Princeton Graduate School (Eric Brown). We thank Peter Hu for making (e.g., the rational agent model and the principle of utility
collecting data in the experiment described in the section of the article maximization). However, this has focused on optimal outcomes
entitled TAFC Experiment and Fit of DDM. We thank Josh Gold for and not the dynamics of decision processes. Some theories based
providing us with the program to generate the moving dots stimuli and for on optimal performance have appeared in psychology as well (e.g.,
discussion. We thank Tyler McMillen for contribution to the section on Carpenter & Williams, 1995; Edwards, 1965; Link, 1975; Mozer,
multiple decisions and Mark Gilzenrat and Angela Yu for discussions and Colagrosso, & Huber, 2002). Anderson’s (1990) rational analysis
valuable insights and ideas that have driven the work reported in this
is perhaps the most fully developed example, and models based on
article. The full set of data from the experiment and Matlab codes allowing
comparison of decision-making models (together with online tutorial) are
it have illuminated cognitive functions including memory, catego-
available for download at http://www.cs.bris.ac.uk/home/rafal/optimal/. rization, and reasoning (Anderson, 1990; Oaksford & Chater,
Correspondence concerning this article should be addressed to Rafal Bo- 1998).
gacz, who is now at the Department of Computer Science, University of In this article, we adopt a similar approach with regard to
Bristol, Bristol BS8 1UB, United Kingdom. E-mail: [email protected] decision making, with an additional focus on how it may be
700
OPTIMAL DECISION MAKING 701
implemented in the brain. We do so within the context of a set of The Drift Diffusion Model (DDM) for Decision Making
highly simplified decision-making conditions, one in which a in the TAFC Paradigm
choice must be made between two responses based on limited
information about which is correct (that is, which will be re- In applying the DDM to the TAFC, we assume that the differ-
warded) and in which both the speed and the accuracy of the ence in the (noisy) information favoring each alternative is inte-
decision impact the total cumulative reward that is accrued. These grated over each trial and that a decision is reached when the
conditions are often referred to as the TAFC paradigm. resulting accumulated value crosses a critical threshold.
Discrete analogues of the DDM were first applied to decision
making in World War II, via the sequential probability ratio test
The Two-Alternative Forced-Choice (TAFC) Task
(SPRT), the theory being developed independently by Barnard
Choosing between two alternatives, even under time pressure (1946) and Wald (1947; cf. Lehmann, 1959). (For histories, see
and with uncertain information, is a simplification of many situa- DeGroot, 1988; Good, 1979; Wallis, 1980.) It was subsequently
tions, but we focus on it for several reasons. First, it is represen- shown that the DDM and its discrete analogue, a directed random
tative of many problems faced by animals in their natural envi- walk, describe accuracy and reaction times (RTs) in humans per-
ronments (e.g., whether to approach or avoid a novel stimulus). forming TAFC tasks (Laming, 1968; Ratcliff, 1978; Stone, 1960).
Pressures for speed and accuracy in such constrained situations In particular, DDM first passages account for a robust feature of
may have exerted strong evolutionary influences, thereby optimiz- human performance: RT distributions are heavily skewed toward
ing mechanisms. Second, a wealth of human behavioral data longer times (see Figure 1a).
generated since the late 19th century (e.g., Hill, 1898) has moti- More recently, neural firing patterns have been examined in
vated formal modeling of the dynamics and response outcomes in light of the DDM (e.g., Gold & Shadlen, 2002; Hanes & Schall,
TAFC tasks (e.g., Busemeyer & Townsend, 1993; LaBerge, 1962; 1996; Schall & Thompson, 1999; Shadlen & Newsome, 2001). For
Laming, 1968; Link, 1975; Link & Heath, 1975; Pike, 1966; example, Shadlen and Newsome (1996) have studied the activity
Ratcliff, 1978; Ratcliff & Smith, 2004; Ratcliff, Van Zandt, & of neurons in the lateral intraparietal (LIP) area (which code for
McKoon, 1999; Stone, 1960; Usher & McClelland, 2001; Vickers, gaze direction) in monkeys performing a TAFC task in which they
1970). Finally, neuroscientists can now monitor neuronal dynam- respond by saccading to one of two targets. Following stimulus
ics and assess their relationship to task performance. In many presentation, activity increases in neurons selective for both direc-
cases, neural and behavioral data are converging to support formal tions, with those selective for the correct response rising fastest on
models such as the DDM (e.g., Gold & Shadlen, 2002; Hanes & average (see Figure 1c). Critically, the time at which their firing
Schall, 1996; Ratcliff, Cherian, & Segraves, 2003; Schall, 2001; rate reaches a fixed threshold predicts the response.
Shadlen & Newsome, 1996, 2001). As noted above and elaborated below, the DDM is optimal in
TAFC task models typically make three fundamental assump- the sense that it is the fastest decision maker for a given level of
tions: (a) evidence favoring each alternative is integrated over accuracy. This assumes a fixed decision threshold, yet the model
time, (b) the process is subject to random fluctuations, and (c) the does not specify what this should be. Absent noise, thresholds have
decision is made when sufficient evidence has accumulated favor- no effect on accuracy. However, with noisy data, accuracy in-
ing one alternative over the other. A central question, to which we creases as thresholds and decision times (DTs) rise because there
return, is whether evidence for each alternative is integrated inde- is more time to average out uncorrelated noise. This produces a
pendently or whether the difference in evidence is integrated. Most speed–accuracy trade-off: Lower thresholds produce faster but less
current theories assume that the difference in evidence drives the accurate responding, whereas higher thresholds produce more ac-
decision. In neural models, differences can be computed by inhib- curate but slower responses. This speed–accuracy trade-off is
itory mechanisms, but theories vary in how inhibition is imple- critical for interpreting the results of most psychological experi-
mented, leading to different behavioral predictions. Several com- ments (Pachella, 1974; Wickelgren, 1977). For example, a manip-
parisons of theories with empirical data have appeared (e.g., ulation intended to influence task difficulty, and thereby accuracy,
Ratcliff & Smith, 2004; Ratcliff et al., 1999; Smith & Ratcliff, could instead simply induce a change in response threshold,
2004; Smith & Vickers, 1989; Usher & McClelland, 2001; Van thereby changing RTs.
Zandt, Colonius, & Proctor, 2000; Vickers, Caudrey, & Wilson, A number of different theories of how humans (or animals in
1971), but a systematic mathematical analysis that compares mod- general) set thresholds or otherwise manage the speed–accuracy
els with one another and with optimal performance is lacking. trade-off have been proposed (Busemeyer & Rapoport, 1988;
Our first goal is to conduct such a study. This is a key step if we Edwards, 1965; Gold & Shadlen, 2002; Maddox & Bohil, 1998;
are to decide which model best describes the data. We relate Myung & Busemeyer, 1989; Rapoport & Burkheimer, 1971). We
several existing models to a particular standard: the DDM (Lam- return to this in the second part of the article (A Theory of Optimal
ing, 1968; Ratcliff, 1978; Stone, 1960). We adopt the DDM as a Thresholds), where we extend the DDM to show how thresholds
reference because it is simple and well characterized (e.g., Smith, can be chosen to optimize performance according to various cri-
2000), has been proven to implement the optimal mechanism for teria and thereby make novel predictions.
TAFC decision making (e.g., Laming, 1968), and accounts for an
impressive array of behavioral and neuroscientific data (e.g., Gold Overview of the Article
& Shadlen, 2002; Hanes & Schall, 1996; Ratcliff, 1978; Ratcliff,
Gomez, & McKoon, 2004; Ratcliff & Rouder, 2000; Ratcliff, We have two primary goals: (a) to analyze the relationship of
Thapar, & McKoon, 2003; Schall, 2001; Shadlen & Newsome, the DDM to other decision-making models and (b) to address the
2001; Smith & Ratcliff, 2004; Thapar, Ratcliff, & McKoon, 2003). broader question of optimality, specifically identifying optimal
702 BOGACZ, BROWN, MOEHLIS, HOLMES, AND COHEN
using these as a reference for comparing models throughout the

a) 60
remainder of the article.
We then address the question of optimality. In the fourth section
Number of trials
40 (Optimal Performance of Decision Networks), we show that the

parameters optimizing performance of the other decision-making
20 models are precisely those for which the models reduce to the
DDM. We review several criteria for optimality in the fifth section
(A Theory of Optimal Thresholds), demonstrating that each im-
0
0.4 0.6 0.8 1 1.2 1.4 1.6 plies a unique optimal threshold and speed–accuracy trade-off and
Reaction time
illustrating their dependence on task parameters. We further show
b) 200 that the DDM implements the optimal decision-making procedure
MT left for all the criteria and predict novel patterns that should emerge in
150 MT right empirical data as signatures of optimal performance in each case.
firing rate
In the sixth section (Biased Decisions), we consider biased deci-

100
sions and rewards, in which one alternative is more probable, or
50
the reward associated with it more valuable, than the other. We
thereby explain some published neurophysiological data and make
0 new quantitative predictions about behavioral strategies in these
0 0.5 1 1.5 2
cases.
time from stimulus onset [s]
We finally discuss extensions to the DDM that might provide
c) 30
more complete accounts of decision-making behavior, arguing that
it offers an attractive framework for further theoretical analysis
20 and the design of empirical studies. Throughout, we restrict our
firing rate
formal presentation to the most important equations, and where

10 possible, we provide intuitive explanations. Further mathematical
LIP left details are available in Appendix A.
LIP right
0
0 0.5 1 1.5 2 Background
time from stimulus onset [s]
Figure 1. a: Sample reaction time distribution in two-alternative forced- Behavioral and Neurophysiological Data from the TAFC
choice task; based on data from a sample participant in Experiment 1
In a common version of the TAFC task, participants must
described in the section TAFC Experiment and Fit of DDM (trials in
identify the direction of a coherently moving subset of dots em-
condition D ⫽ 1). b, c: Cartoon of typical peri-stimulus time histograms of
neuronal activity during the moving dots task. The figure does not show the bedded in a random motion field (Britten, Shadlen, Newsome, &
actual data, but it is a sketch based on data described by Britten, Shadlen, Movshon, 1993). Critical parameters under experimenter control
Newsome, and Movshon (1993), Shadlen and Newsome (2001), and Schall include (a) stimulus fidelity or difficulty, which can be manipu-
(2001). Horizontal axes show time from stimulus onset. Vertical axes lated by varying the fraction of coherently moving dots; (b)
indicate firing rate. Representative firing rates are shown for stimulus with whether participants are allowed to respond freely or responses are
coherent leftward motion. b: Firing rate of neurons in the middle temporal cued or deadlined; and (c) the delay between response and the next
area (MT): Gray line represents a typical neuron selective for leftward stimulus.
motion, and black line for rightward motion. c: Firing rate of neurons in the In addition to their use in the study of behavior, TAFC tasks are
lateral intraparietal (LIP) area: Gray line represents a typical neuron
also used widely in neurophysiological studies, in which direct
selective for leftward saccades, and black line for rightward saccades.
recordings are made from brain areas involved in task performance
(e.g., Shadlen & Newsome, 1996, 2001). Figure 1b shows a
representation of typical firing rates observed in the middle tem-
parameters for TAFC tasks. We hope thereby to offer a unified poral area (MT) of monkeys trained on the moving dots task (MT
framework in which to pursue future modeling and quantitative is involved in motion processing). When a stimulus with coherent
studies of decision-making behavior. leftward motion is presented, the firing rate of an MT neuron
The article is organized as follows. In the second section (Back- selective for leftward motion typically exceeds that of one selec-
ground), we provide formal descriptions of the TAFC task, the tive for rightward motion (Britten et al., 1993)—the grey curve in
SPRT, and the DDM, specifying how key quantities such as error the figure is more often above the black one. However, both firing
rate (ER) and mean DT depend on the parameters characterizing rates are noisy, hence decisions based on instantaneous activities
the DDM. We then review five other decision-making models. We of MT neurons would be inaccurate, reflecting uncertainty inherent
analyze the relationship of these to the DDM in the third section in the stimulus and its neural representation.
(Relationships Among the Models), showing that all but one of Figure 1c shows activity patterns of neurons in area LIP (in-
them is a variant of it, at least in a limiting sense. We thereby volved in eye movement control): They clearly separate as time
clarify the relationships among several theories and their differing increases. LIP neurons are believed to integrate the noisy MT
predictions, facilitating further analysis and discussion. We illus- outputs over each trial, leading to more accurate decisions. Neural
trate this by fitting DDM parameters to empirical TAFC data and integration mechanisms have been studied extensively in the con-
text of control of motor processes (e.g., Aksay, Gamkrelidze, y1, y2, . . . , yn under H1 or H2, respectively. Hypothesis H1 (or H2)
Seung, Baker, & Tank, 2001; Cannon, Robinson, & Shamma, is accepted if the ratio of Equation 1 is less than Z (or greater than
1983; Goldman, Levine, Major, Tank, & Seung, 2003; Koulakov, Z), where Z is a constant determined by the desired level of
Raghavachari, Kepecs, & Lisman, 2002; Major et al., 2004; Seung, accuracy for one of the hypotheses.1 Neyman and Pearson showed
1996; Seung, Lee, Reis, & Tank, 2000). that, for fixed sample size n, setting Z ⫽ 1 delivers the most likely
hypothesis and minimizes the total error probability. Hence, this
procedure minimizes ER for fixed DT and thus is optimal for the
The Decision Problem
interrogation paradigm. (Here and throughout, we do not consider
Motivated by the above example, we formalize the TAFC de- any explicit cost for acquiring evidence.)
cision problem by postulating two neuronal populations whose The answer to Question b above is provided by the SPRT of
activities (firing rates) provide evidence for the two alternatives. Barnard (1946) and Wald (1947). Here, the goal is to decide as
We denote their mean activities during a given trial by I1 and I2 soon as a stream of incoming data reaches a predetermined level of
and assume that each experiences independent random fluctuations reliability. Again assuming that samples are drawn at random from
with the same standard deviation, c. The goal is to identify which one of two distributions with densities p1(y), p2(y), the running
of I1 and I2 is greater, and the optimality question becomes, for product of the likelihood ratios is calculated as in Equation 1, but
given signal and noise levels I1, I2, and c, what strategy provides now, observations continue only as long as the product lies within
the most accurate and fastest decisions on average? More pre- predefined boundaries Z2 ⬍ Z1:
cisely, there are two questions: (a) Which strategy yields the
p1n
lowest expected ER at a given (fixed) time allowed for decision, Z2 ⬍ ⬍ Z1 . (2)
and (b) which strategy yields the shortest expected RT for a given p2n
ER? Thus, after each measurement, one updates the likelihood ratio,
These questions correspond to two paradigms under which thereby assessing the net weight of evidence in favor of H1 over
TAFC tasks can be run. The first typically involves fixed-duration H2. When the ratio first exceeds Z1 or falls below Z2, sampling
stimuli, after which participants are expected to answer, usually on ends, and either H1 or H2 is accepted; otherwise, sampling con-
presentation of a signal to respond, thus constraining their RTs. tinues.2 The SPRT is optimal in the following sense: Among all
We refer to the hard-limit case, in which decisions are supposed to fixed or variable sample decision methods that guarantee fixed
be made instantly at the cue, as the interrogation paradigm. The error probabilities, SPRT requires on average the smallest number
second addresses a free-response paradigm under which partici- of samples to render a decision (Wald & Wolfowitz, 1948). Thus,
pants respond in their own time. Because, in the latter, both ERs for a given ER, SPRT delivers the fastest RT. In the first section
and RTs vary (participants implicitly choose a speed–accuracy of Appendix A (Probability Ratio Tests), we provide more precise
trade-off), one may assess optimality by asking which among statements and generalizations to decisions between alternatives
strategies that yield the same ER gives the shortest RT. These with unequal prior probabilities.
paradigms represent the extremes of a continuum of more general The SPRT is equivalent to a random walk with thresholds
deadlining tasks, in which responses can be made at any time corresponding to the alternative choices, as one can see by taking
before a fixed deadline. logarithms in Equations 1 and 2:
We now formalize the questions posed above, which both hinge
on determining whether the difference of the means I1 ⫺ I2 is p1 共y1 兲 p1 共yn兲
positive or negative. Let us denote by Y the random variable log Z2 ⬍ log ⫹ · · · ⫹ log ⬍ log Z1 . (3)
p2 共y1 兲 p2 共yn兲
corresponding to the difference in activity of two neuronal popu-
lations, where the activity of each represents accumulated evidence Denoting the logarithm of the likelihood ratio by I n, Equation 1
for one of the two alternatives. We suppose that successive sam- implies that I n is additively updated after each observation:
ples of Y within each trial are drawn from one of two probability
distributions with densities p1(y) and p2(y) with means ␮1 and ␮2. p1 共yn兲
I n ⫽ I n⫺1 ⫹ log (4)
Hence, we must decide which of the hypotheses H1 (I1 ⫺ I2 ⫽ p2 共yn兲
␮1 ⬎ 0) or H2 (I1 ⫺ I2 ⫽ ␮2 ⬍ 0) is correct.
The answer to Question a above is given by the procedure of (cf. Gold & Shadlen, 2002). The SPRT is therefore equivalent to
Neyman and Pearson (1933). To decide from which distribution a a random walk starting at I0 ⫽ 0 and continuing until I n reaches
random sequence Y ⫽ y1, y2, . . . , yn is drawn, we calculate the the threshold log Z1 or log Z2. Moreover, as discrete samples are
likelihood ratio of the sample Y given the hypotheses H1 and H2: taken more frequently and one approaches sampling of a contin-
p1n p1 共y1 兲p1 共y2 兲 · · · p1 共yn兲

⫽
1
. (1) The Neyman–Pearson procedure underlies many statistical tests used
p2n p2 共y1 兲p2 共y2 兲 · · · p2 共yn兲 in psychology to evaluate an experimental hypothesis on a fixed number of
experimental samples, including the t test.
Here, p1(yi) and p2(yi) denote the probabilities of each observation 2
In the context of statistical analysis of experimental data, the SPRT
yi assuming that hypothesis H1 or H2 is true respectively. The ratio would correspond to checking the condition of Equation 2 after each
p1(yi)/p2(yi) is greater than 1 if the observation yi supports H1 (i.e., sample collected and stopping the experiment if Equation 2 is not satisfied
if yi is more likely under H1 than H2) and less than 1 if it supports (i.e., a prespecified confidence level is reached) rather than waiting for a
H2. Because all observations are assumed independent, p1n and p2n fixed number of samples and then evaluating the hypotheses for signifi-
denote the probabilities of observing the sequence of observations cance (i.e., probability).
uous variable, the SPRT converges on the DDM, and the discrete We model the interrogation paradigm by asking if, at the interro-
log likelihood ratio I n becomes a continuous time-dependent vari- gation time T, the current value of x lies above or below zero. If H1
able x(t), after a change of scale. Details are given in Appendix A applies, a correct decision is recorded if x ⬎ 0 and an incorrect one
(Random Walks and the SPRT and The Continuum Limit of the if x ⬍ 0. The average ER is therefore the probability that a typical
SPRT).3 As shown below, in the section entitled DDM, this con- solution x(T) of Equation 5 lies below zero at time T, which is
tinuum limit yields explicit formulae and key quantitative predic- obtained by integrating the density p(x, T) of Equation 6 from ⫺⬁
tions. to 0:
In the following sections, we review six mathematical models of
冉冊冕
TAFC decision making, starting with the DDM as developed by y
A 1
Ratcliff (1978; Ratcliff et al., 1999). We then discuss the Ornstein– ER ⫽ ⌽ ⫺ 冑T , where ⌽共y兲 ⫽ e⫺共u / 2兲 du.
2
(7)
Uhlenbeck (O-U) model (Busemeyer & Townsend, 1993) because c
⫺⬁
冑2␲
it provides a link between the DDM and the biologically-motivated
models that follow (Ditterich, Mazurek, & Shadlen, 2003; Usher & (Here, ⌽ is the normal standard cumulative distribution function.)
McClelland, 2001; Wang, 2002). These employ different forms of In the free-response paradigm, the decision is made when x
inhibition to compute differences in signals associated with each reaches one of two fixed thresholds, positive or negative. If both
alternative. We also discuss the well-established and widely used alternatives are equally probable, the thresholds are symmetric
race model (LaBerge, 1962; Logan, 2002; Logan & Bundesen, (⫾z). Figure 2a shows examples of the evolution of x during
2003; Smith & Van Zandt, 2000; Smith & Vickers, 1989; Vickers,
separate trials (sample paths). Although on average x moves to-
1970). All six models are represented as simplified stochastic
ward the correct threshold, noise causes it to hesitate and back-
differential equations in which only essential elements are re-
track, and on some trials, it crosses the incorrect threshold.
tained. In particular, for ease of exposition and mathematical
Solutions of a first passage problem for the pure DDM yield
tractability, we describe only linearized systems, although many of
our observations extend to nonlinear models. Explicit comparisons simple expressions revealing how ER and mean DT depend on
among linear, piecewise-linear, and nonlinear (sigmoidal) models drift A, noise c, and decision threshold z (Feller, 1968; cf. Ratcliff,
show that linearized models often capture the key dynamics and 1978):
parameter dependencies (Brown et al., 2005; Brown & Holmes,
2001; cf. Usher & McClelland, 2001). 1
ER ⫽ 2 , and (8)
1 ⫹ e2Az/c
冉冊
DDM
z Az
As indicated above, in the DDM (Ratcliff, 1978), one accumu- DT ⫽ tanh 2 . (9)
A c
lates the difference between the amounts of evidence supporting
the two hypotheses. We denote the accumulated value of this (See Appendix A, subsection titled ERs, Mean DTs, and Reward
difference at time t by x(t) and assume that x ⫽ 0 represents Rates). Here, DT denotes the mean DT: the fraction of the RT
equality in the amounts of integrated evidence. We consider two associated with the decision process. We denote the remaining
versions of the DDM below, the first being a continuum limit of fraction of RT (e.g., due to sensory or motor processes unrelated to
the random walk model (Laming, 1968) that we refer to as the pure the decision process) by T0. Thus, RT ⫽ DT ⫹ T0. Because
DDM and the second or extended DDM, a generalized model in Equations 8 and 9 involve ratios of A, z, and c2, ER and DT do not
which drift rates and starting points may vary across trials (e.g., change if these parameters are scaled by the same constant. All the
Ratcliff & Rouder, 1998).
models reviewed here share this property.
In the pure DDM, we start with unbiased evidence and accu-
Because the variable x of the DDM is proportional to the log
mulate it according to
likelihood ratio, the DDM implements the Neyman–Pearson pro-
dx ⫽ Adt ⫹ cdW, x共0兲 ⫽ 0. (5) cedure (with Z ⫽ 1) for the interrogation paradigm and the SPRT
for the free-response paradigm. Hence, it is the optimal decision
(Biased decisions are treated in the sixth section, below.) In maker for both paradigms.
Equation 5, dx denotes the change in x over a small time interval The extended DDM (e.g., Ratcliff, 1978; Ratcliff et al., 1999)
dt, which is comprised of two parts: The constant drift Adt repre- includes two additional elements that improve its fit to experimen-
sents the average increase in evidence supporting the correct
tal data. Intertrial differences in stimulus fidelity or attention can
choice per time unit. In terms of the section above, The Decision
be modeled by allowing drift to vary by selecting A from a normal
Problem, A ⬎ 0 if H1 is correct for the trial in question, and A ⬍
distribution with mean mA and standard deviation sA on each trial
0 if H2 is correct. The second term, cdW, represents white noise,
(Ratcliff, 1978). Drift variability produces longer DTs for errors
which is Gaussian distributed with mean 0 and variance c2dt.
Hence, x grows at rate A on average, but solutions also diffuse due
to the accumulation of noise. Neglecting boundary effects, the
probability density p(x, t) of solutions of Equation 5 at time t is
normally distributed with mean At and standard deviation c公t 3
In Appendix A (sections Optimal Decisions for the Free-Response
(Gardiner, 1985): Protocol and Optimal Decisions Under the Interrogation Protocol), we also
provide direct arguments, independent of the SPRT, suggesting that the
p共x, t兲 ⫽ N共At, c 冑t兲. (6) DDM is optimal in all these senses.
a) b) c)
2.5
2 y2
1.5
Z
1
0.5
x
-0.5
-1 0 Z y1
-1.5
0 0.5 1 1.5 2 2.5
time
Figure 2. Examples of the time evolution of variables in decision models. a: The pure drift diffusion model.
Horizontal axis denotes time; vertical axis denotes the accumulated difference x between the evidence supporting
the correct and incorrect alternatives. The model was simulated for 100,000 trials using the Euler method with
timestep ⌬t ⫽ 0.01 and the following parameters: drift A ⫽ 1, noise c ⫽ 1, threshold z ⫽ 1. Each path
corresponds to one sample simulated decision process. The histograms outside the thresholds show proportions
of trials reaching the threshold after different intervals. b: An example of the evolution of the mutual inhibition
model, showing y1 and y2 as functions of time. c: The phase or state space of the mutual inhibition model.
Horizontal axis denotes the activation of the first decision unit; vertical axis denotes the activation of the second
decision unit. The path shows the decision process from stimulus onset (where y1 ⫽ y2 ⫽ 0) to reaching a
decision threshold (decision thresholds are shown by dashed lines). The mutual inhibition model was simulated
for the following parameters: I1 ⫽ 4.41, I2 ⫽ 3, c ⫽ 0.33 (parameters of the inputs correspond to those estimated
from the participant shown in Figure 5, via Equation 24), w ⫽ k ⫽ 10, Z ⫽ 0.4.
than correct responses (Ratcliff & Rouder, 1998, 2000),4 because dx ⫽ 共␭x ⫹ A兲dt ⫹ cdW, x共0兲 ⫽ 0. (10)
most errors occur on trials with low drift, which typically have
longer DTs. Secondly, the initial value x(0) can be chosen from a The rate of change of x now also depends on its current value, with
uniform distribution ranging from ⫺sx to sx (with sx ⬍ z). This a magnitude controlled by the additional parameter ␭, and x can
may reflect participants’ sensitivity to local frequency variations in accelerate or decelerate toward a threshold depending on the sign
presentation of specific stimuli that occur even when they are of ␭. In the following discussion, we again assume that A ⬎ 0 and
equally probable overall (e.g., Cho et al., 2002; Soetens, Deboeck, H1 is correct.
& Hueting, 1984) or premature sampling: the integration of noise First, suppose ␭ ⬍ 0. For x ⫽ ⫺A/␭ (⬎0), dx is zero on average;
from sensory neuronal populations before the stimulus onset this value of x corresponds to an equilibrium or fixed point for the
(Laming, 1968; Rouder, 1996). Starting point variability produces noise-free process. When x ⬍ ⫺A/␭, dx is on average positive, and
shorter DTs for errors than correct trials (Ratcliff & Rouder, 1998) x typically grows; when x ⬎ ⫺A/␭, dx is on average negative, and
because these errors occur more often on trials for which the x tends to decrease. The fixed point is therefore an attractor. More
process begins closer to the threshold for the incorrect alternative precisely, for this stable O-U process, the time-dependent proba-
and hence crosses this threshold with a relatively short DT. bility density of x converges to a stationary normal distribution
The expression for ER of the extended DDM in the interrogation centered on x ⫽ ⫺A/␭ with standard deviation c/公⫺2␭ (Gardiner,
paradigm involves an integral that cannot be analytically evalu- 1985; Holmes, Lumley, & Berkooz, 1996).
冉冊
ated, although it can be derived in the presence of drift variability
(see Ratcliff, 1978, where an equation for d⬘ [d-prime] as a A c
p共x兲 ⫽ N ⫺ , . (11)
function of interrogation time is also given). Expressions for ER ␭ 冑⫺2␭
and DT of the extended DDM in the free-response paradigm also
involve integrals that cannot be analytically evaluated, and so we Derivations and full expressions for the time-dependent distribu-
generally resort to numerical simulations, although asymptotic tion are given in Appendix A (section entitled Optimal Decisions
approximations are derived in Appendix A (see section titled ER Under the Interrogation Protocol).
and DT for the Extended Drift Diffusion Model in the Free- Now, suppose ␭ ⬎ 0. When x ⬍ ⫺A/␭ (⬍0), dx is on average
Response Protocol). An efficient numerical method for calculating negative, and x usually decreases, whereas for x ⬎ ⫺A/␭, it usually
ER and the distribution of DTs has recently been developed increases. Hence, x is repelled from the fixed point more strongly
(Tuerlinckx, 2004).
4
Ornstein–Uhlenbeck (O-U) Model Throughout this article, we assume that drift is fixed within a trial and
that drift variations occur only from trial to trial. The case of varying drift
The O-U model (Busemeyer & Townsend, 1993) differs from within the trial is analyzed in Brown et al. (2005); cf. Ratcliff (1980) and
the DDM in that a third term, linear in x, is added: Ratcliff & Rouder (2000).
the further away it is, and the mean of x accelerates away from x ⫽ a) Race b) Mutual inhibition
⫺A/␭ in an unstable O-U process. Thus, the corresponding prob- y1 y2 y1 y2
ability density has exponentially growing mean and variance, with Response
w
units:
no stationary limit. It is also described by the general expressions
developed in Appendix A (section entitled Optimal Decisions
Under the Interrogation Protocol).
Summarizing, ␭ ⬍ 0 causes attraction toward the fixed point
proportional to the distance of the current state from it, whereas
Inputs: I1±c I2±c I1±c I2±c
␭ ⬎ 0 causes repulsion. The intuition for these behaviors with
respect to decision making is as follows. In the stable case, all d) Pooled inhibition
solutions approach and tend to remain near the fixed point, which y3
c) Feed-forward inhibition
lies nearer the correct threshold, so they typically slow down w' w'
y1 y2 w w
before crossing it, corresponding to conservative behavior. The Response v v
unstable case corresponds to riskier behavior: Solutions on the units: y1 y2
u u
correct side of the fixed point accelerate toward the correct thresh-
old, giving faster responses, but solutions on the incorrect side
accelerate toward the incorrect threshold, possibly producing more
errors. For ␭ ⫽ 0, the O-U model simplifies to the pure DDM.
Busemeyer and Townsend (1993) proposed that rewards for Inputs: I1±c I2±c I1±c I2±c
correct responses should increase ␭, whereas punishments for
errors should decrease it. They also noted that negative ␭ produces Figure 3. Architectures of decision models. a: Race model (continuous
a recency or decay effect over the course of a trial because later version of Vickers, 1970). b: Mutual inhibition model (simplified from
inputs influence accumulated evidence more than earlier ones, Usher & McClelland, 2001). c: Feedforward inhibition model (simplified
whereas positive ␭ produces a primacy effect (earlier inputs have from Ditterich, Mazurek, & Shadlen, 2003). d: Pooled inhibition model
(simplified from Wang, 2002). Arrows denote excitatory connections; line
more influence).
with filled circles denotes inhibitory connections. In Panel d, discs con-
The ER expression for the O-U model in the interrogation
taining small open circles denote populations of excitatory neurons, and the
paradigm generalizes that for the DDM (Equation 7). For interro- disc with filled circles denotes populations of inhibitory neurons.
gation at time T, we have (Busemeyer & Townsend, 1992; derived
in Appendix A, Equations A88 and A92):
ER共T兲 ⫽ ⌽ ⫺ 冉冑 A
c
2(e␭T ⫺ 1)
␭(e␭T ⫹ 1)
冊
. (12) 再dydy ⫽⫽ II dtdt ⫹⫹ cdW
2
1 1
2
cdW
,
1
2
y1 共0兲 ⫽ y2 共0兲 ⫽ 0. (13)
We model the interrogation paradigm by assuming that at time T,

The expressions for ER and DT in the free-response paradigm
the choice is made in favor of the alternative with higher yi. In
were derived by Busemeyer and Townsend (1992); we give them
free-response mode, as soon as either unit exceeds a preassigned
as Equations A55 and A56 in Appendix A, along with asymptotic
threshold, the model is assumed to make a response. Again, we
approximations A64 and A65 that better reveal parameter depen-
assume equal probabilities of the two alternatives and hence equal
dencies.
decision thresholds y1 ⫽ y2 ⫽ Z (capital Z distinguishes this case
from the threshold of one-dimensional models, denoted by z).
Race (Inhibition-Free) Model Instead of deriving ER and DT expressions for the race model
here, we show how they relate to the ERs and DTs of the other
The DDM and O-U model are one-dimensional: A single inte- models in the section Optimal Performance of Decision Networks,
grator accumulates the net evidence. However, many models, below). Anticipating those results, we note that the race model
including physiologically-motivated ones, use separate integrators cannot be reduced to the DDM and is therefore not optimal in the
for the evidence in support of each alternative and are therefore sense described above.
inherently two-dimensional (or more). Here and below, we review Next, we describe three biologically-motivated models that re-
four such models, pictured in Figure 3, and describe when they can late more closely to the DDM and O-U models than to the race
and cannot be approximately reduced to one-dimensional descrip- model, although they also employ separate accumulators.
tions. We again simplify by considering only linearized models.
The simplest is the race model in which accumulators for each
Mutual Inhibition Model
of the two alternatives integrate evidence independently. At least
three discrete variants exist, differing in how time and evidence are Figure 3b shows the architecture of an abstract neural network
discretized: the recruitment model (LaBerge, 1962), the accumu- (connectionist) model described by Usher and McClelland (2001).
lator model (Vickers, 1970), and the Poisson counter model (Pike, We refer to this as the mutual inhibition model in the remainder of
1966). Here, we describe a continuous-time analogue (see Fig- this article to distinguish it from others to be discussed below. Its
ure 3a). We again denote the mean rates of incoming evidence by four units represent the mean activities of neuronal populations:
I1 and I2 and assume they are subject to independent white-noise Two input units represent populations providing evidence in sup-
processes of root-mean-square (RMS) strength c. The integrators port of the alternative choices (e.g., groups of left- and right-
y1 and y2 accumulate evidence according to movement-sensitive MT neurons; cf. the section Behavioral and
Neurophysiological Data From the TAFC, above), and two deci- where the crossed inputs imply that both noise sources influence
sion units represent populations integrating the evidence (e.g., LIP both accumulators. Here, u denotes the weight of feedforward inhib-
neurons). itory connections: In the version of Ditterich et al. (2003), these have
The decision units are leaky integrators with activity levels yi, the same weight as excitatory connections (i.e., u ⫽ 1). As in the
and each accumulates evidence from an input unit with mean mutual inhibition model, we assume that decisions are rendered
activity Ii and independent white-noise fluctuations dWi of RMS whenever the activity of either unit reaches the threshold Z.
strength c. The decision units also mutually inhibit each other via
a connection of weight w. Information is therefore accumulated
Pooled Inhibition Model
according to
再
Wang (2002) developed a biophysically based model of area
dy1 ⫽ (⫺ky1 ⫺ wy2 ⫹ I1 )dt ⫹ cdW1 LIP simulating individual spiking neurons rather than averaged
dy2 ⫽ (⫺ky2 ⫺ wy1 ⫹ I2 )dt ⫹ cdW2 , y1 共 0 兲 ⫽ y2 共 0 兲 ⫽ 0. (14)
representations of neural populations as in the abstract connection-
Here, k denotes the decay rate of activity (the leak terms ⫺kyi ist type models described above. The neural and synaptic proper-
cause the activity to decay to zero in the absence of inputs) and ties (e.g., membrane capacitance, leak conductance, etc., and decay
⫺wyi represents mutual inhibition. The parameters k, w, and those times of AMPA and NMDA currents) were based on anatomical
of the two models described below are all assumed to be non- and neurophysiological observations.
negative. The activity scale is chosen so that zero represents The model, shown in Figure 3d (cf. Wang, 2002), includes two
baseline activity in the absence of inputs, hence integration starts pools of neurons representing the alternative choices. These re-
from y1(0) ⫽ y2(0) ⫽ 0. ceive stimulus inputs and compete with each other as explained
The values of y1 and y2 describing the state of this model may below. At the population level, this architecture has much in
be represented as a point on a phase plane with horizontal and common with the mutual inhibition model (compare Figures 3b
vertical axes y1 and y2, and the evolution of activities during the and 3d), but they differ as follows.
decision process may be visualized as a path in this plane. Figure First, the decision neurons have self-excitatory recurrent projec-
2c shows an example corresponding to the individual time courses tions, denoted by v in Figure 3d. These are necessary to enable
of y1 and y2 of Figure 2b. individual neurons, whose membrane voltages decay on a milli-
The major simplification of the model presented here in com- second scale, to integrate information on the decision timescale of
parison to that of Usher and McClelland (2001) is the removal of hundreds of milliseconds. (They also allow the system to actively
nonlinearities. In Equations 14, the accumulation rates of y1 and y2 maintain stimulus representations, a property used to address
depend linearly on their present values. To account for the fact that short-term memory phenomena in other contexts.) Second, neu-
neural firing rates are never negative, Usher and McClelland rons representing the alternatives do not directly inhibit each other
assumed that y1 and y2 are transformed via a threshold-linear (because real excitatory neurons do not send inhibitory connec-
activation function f(y) with f(y) ⫽ y for y ⱖ 0 and f(y) ⫽ 0 for y ⬍ tions), but both populations excite a common pool of inhibitory
0; Equations 14 linearize this function by ignoring thresholding neurons via connections of weight w⬘, which then inhibit all
below 0. However, as noted above, in our analysis, yi ⫽ 0 corre- decision neurons equally via recurrent connections of weight w.
sponds to (positive) baseline activity so that yi ⬍ 0 need not Precise relationships between detailed neural network models
imply negative activity. As suggested by Cohen, Dunbar, and such as this and the DDM are elusive because it is not yet entirely
McClelland (1990), this sensitive (linear) range of the activa- clear when a population of neurons may be approximated by a
tion function is precisely where one expects decision units in noisy connectionist unit (although progress is being made in this
the focus of attention to operate. Furthermore, for suitable direction; e.g., Brunel & Latham, 2003; Ermentrout, 1994;
parameter values, Equations 14 yield good estimates of the ER Omurtag, Knight, & Sirovich, 2000; Renart, Brunel, & Wang,
and DT distributions of the fully nonlinear system, as shown 2003; Shelley & McLaughlin, 2002; X.-J. Wang, personal com-
elsewhere (Usher & McClelland, 2001; see Brown et al., 2005; munication, January 26, 2004; Wong & Wang, 2006). Here, we
Brown & Holmes, 2001, for explicit comparisons among the dynam- assume that such a reduction holds so that, denoting the activities
ics of fully nonlinear, piecewise-linearized, and linearized mutual of the decision groups by y1 and y2 and of the inhibitory pool by
inhibition models). y3, the model may be written as5
Feedforward Inhibition Model 再 dy1 ⫽ (⫺ky1 ⫺ wy3 ⫹ vy1 ⫹ I1 )dt ⫹ cdW1 ,

dy2 ⫽ (⫺ky2 ⫺ wy3 ⫹ vy2 ⫹ I2 )dt ⫹ cdW2 ,
dy3 ⫽ (⫺kinhy3 ⫹ w⬘(y1 ⫹ y2 ))dt.
(16)
Shadlen and Newsome (2001; cf. Ditterich et al., 2003; Ma- Again, as above (but unlike Wang, 2002), we have linearized all
zurek, Roitman, Ditterich, & Shadlen, 2003) proposed a feedfor- terms in these equations. As we show below in the section entitled
ward inhibition model for area LIP (see Figure 3c). It differs from Relationships Among the Models, this reduced or averaged net-
the mutual inhibition model in that the units are perfect integrators
with no leak, and they receive inhibition from crossed inputs rather
than inhibiting each other. The accumulation rates are independent 5
Some of the work referenced above indicates that the averaged syn-
of yi: aptic conductances are the relevant dynamical variables for the reduced
network equations; here, for consistency with the rest of the article, we
再 dy1 ⫽ I1 dt ⫹ cdW1 ⫺ u(I2 dt ⫹ cdW2 )

dy2 ⫽ I2 dt ⫹ cdW2 ⫺ u(I1 dt ⫹ cdW1 ) , y1 共0兲 ⫽ y2 共0兲 ⫽ 0, (15)
express Equations 16 in terms of the activities of the underlying popula-
tions (Wilson & Cowan, 1972).
work version of Wang’s (2002) model may be viewed as a more havioral data. Therefore, we begin, in the first subsection, with a
biologically realistic implementation of the Usher and McClelland description of a fit of the DDM to such data. The second subsec-
(2001) mutual inhibition model. tion shows that for particular parameter ranges, the mutual inhi-
bition model can be approximated by either the O-U model or the
Relationships Among the Models DDM. The third subsection describes the relationship between the
feedforward inhibition model and DDM. The fourth subsection
A number of previous reports have noted similarities between shows that the pooled inhibition model can also be approximated
the DDM and biologically-inspired connectionist and averaged by the DDM for certain parameter values but that there are subtle
neural network models (e.g., Ratcliff & Smith, 2004; Seung, 2003; differences in interpretation of the role of inhibition.
Usher & McClelland, 2001; Wang, 2002). In this section, we
extend those observations by conducting a formal analysis of the
relationships among the models, and we derive precise correspon- TAFC Experiment and Fit of DDM
dences among their variables and parameters. The relationships we To illustrate our analysis and verify its relevance, we must
find are summarized in Figure 4, which shows that, for appropriate choose representative parameter values. We do this using data
parameter ranges, all the models described in the previous section from an experiment examining human performance in a TAFC
except the race model can be reduced to the optimal DDM. similar to the one used by Shadlen and Newsome (2001) and
Recently, Ratcliff and Smith (2004) studied the specific rela- identifying the parameters of the DDM that best fit the behavior of
tionships among the DDM, O-U, race, and mutual inhibition a representative participant (with mean ER and RT that were
models. They used a model mimicry approach, in which they within one standard deviation of the means of full set of partici-
generated data by simulating one model, fitting a second to these pants). A complete description of the experiment will be the
data, and measuring the closeness of this fit. Our study comple- subject of a future report. Here, we briefly describe the experi-
ments these previous efforts. Although our analytical approach mental methods (a more detailed description of the experimental
limits us to simplified models, it allows us to explain why and for methods is provided in Appendix B) and our methods of parameter
which parameter ranges certain models are equivalent. estimation for the DDM, after which we return to the comparative
The relationships we find, summarized in Figure 4, in many analysis of models.
cases are similar to those that have been revealed by other ap-
proaches. For example, similar to Ratcliff and Smith (2004), we
find that the DDM and the O-U model with ␭ close to zero mimic Method
each other (as designated by the connection between these models Twenty adult participants performed a task in which they had to indicate
in Figure 4 with label ␭ ⫽ 0), the DDM does not mimic the race whether the predominant direction of movement of an array of dots on a
model (there is no connection between them in Figure 4), and the display screen was leftward or rightward by pressing a corresponding key.
mutual inhibition model can mimic the DDM, O-U, and race Task difficulty (i.e., motion coherence) was kept constant throughout the
models for different parameter values (arrows pass from the mutual experiment. Participants were paid one cent for each correct choice. After
inhibition model to the DDM, O-U, and race models in Figure 4). each response, participants were required to release the key, and there was
In the remainder of this section, we describe analyses of the a delay D before presentation of the next stimulus (D was kept constant
specific relationships among models. In several instances, we within each block but varied across blocks). On some blocks, an additional
present illustrative simulations using parameters fit to TAFC be- delay Dp was imposed after error responses (see below). Trials were
blocked by delay condition. This was manipulated to test predictions made
by the DDM (discussed below in the section entitled Optimal Thresholds
for Different Criteria in the Pure DDM). There were four delay conditions:
Pooled inhibition
(a) D ⫽ 0.5 s, (b) D ⫽ 1 s, (c) D ⫽ 2 s, and (d) D ⫽ 0.5 s and Dp ⫽ 1.5 s
kinh large 3D (in the first three conditions, Dp ⫽ 0).
kinh large Mutual inhibition Feedforward inhibition
w, w’ large w=0
v=k k=0
u=0 2D Estimating Parameters of the DDM
w+k large w+k large
w=k Race We estimated the parameters of the extended DDM using a
O-U u=1 1D combination of the weighted least squares fitting method (Ratcliff
& Tuerlinckx, 2002) and the method of Bogacz and Cohen (2004).
λ=0
Because we expected participants to adopt different decision cri-
DDM teria in different delay conditions (as described below in the
section entitled A Theory of Optimal Thresholds), we assumed that
Figure 4. Relationships among decision models. An arrow between two thresholds z differed across conditions. We further assumed that all
models indicates that the computations of the first model simplify to the other parameters of the DDM were the same for a given
computations of the second under conditions in the arrow’s label. The participant in all conditions because the stimuli were exactly the
horizontal dotted lines separate classes of models with different numbers of
same in all blocks (the motion of dots had the same coherence).
dimensions (one dimension [1D], two dimensions [2D], and three dimen-
sions [3D]), so that arrows crossing the dotted lines indicate simplifications
Thus, for each participant, we estimated the following parameters:
involving dimension reduction. A more detailed taxonomy of diffusion and mA (mean drift rate), sA (standard deviation of the drift rate), c
race models (distinguishing between different variants of these models) is (noise), sx (standard deviation of the starting point), T0 (non-
given in Figure 1 of Ratcliff and Smith (2004). DDM ⫽ drift diffusion decision-related response time), and z1, z2, z3, and z4 (decision
model; O-U ⫽ Ornstein–Uhlenbeck model. thresholds for the four delay conditions).
Following Ratcliff and Tuerlinckx (2002), for each delay con- ipant has a probability of making an error equal to pe and that there
dition d, we divided the RT distributions for correct and error trials are n trials in this condition. Then, the experimental error rate ERex
into five quantiles q: 0.1, 0.3, 0.5, 0.7, and 0.9, denoted as RTCthq,d , comes from the binomial distribution with mean pe and variance
RTEth ex ex
q,d , and RTCq,d , RTEq,d , with superscripts th and ex indicating pe(1 ⫺ pe)/n. We take ␻ER ⫽ n/pe(1 ⫺ pe). For simplicity, we
theoretical (extended DDM) and experimental statistics. We de- estimate pe as the mean ER of the participant averaged across all
note the ERs given by the extended DDM and observed in the delay conditions. This averaging across conditions is done to avoid
experiment by ERth ex
d and ERd respectively. dividing by zero in blocks in which the participant did not make
The subplex optimization algorithm (Rowan, 1990) was used to any errors and also because the differences in ERex across condi-
find parameters minimizing the cost function describing the tions for single participants are small in comparison to differences
weighted difference between ERs and RT distributions for the between participants. The estimation of variance for quantiles of
model and from the experiment (Ratcliff & Tuerlinckx, 2002): the RT distribution was computed using the method of Maritz and
Jarrett (1978).
冘冘
4 5
Figure 5 compares the ER and RT quantiles of the extended
Cost ⫽ 共␻ ER,d共ER thd ⫺ ER ex
d兲 ⫹
2
␻ RTC,d,q共RTC d,q
th
DDM fitted to experimental data from a sample participant by
d⫽1 q⫽1
plotting quantile probability functions, as in Ratcliff and Smith
(2004). The decision process of this participant exhibited variabil-
冘
5
ity of both drift and starting point (estimated parameter values: mA
⫺ RTC 兲 ⫹
ex 2
␻ RTE,d,q共RTE d,q
th
⫺ RTE d,q
ex 2
兲兲. (17)
d,q
⫽ 1, sA ⫽ 0.31, sx ⫽ 0.14, c ⫽ 0.33). The extended DDM fits the
q⫽1
essential properties of the data: In Figure 5a, the participant em-
In the above equation, ␻’s denote the weights of the fitted statis- phasized speed over accuracy; thus, the threshold was low (z1 ⫽
tics. We would like to choose the weight of a given statistic close 0.16, just outside the range of starting points sx ⫽ 0.14), and the
to 1/(the estimated variance of this experimental statistic), as effect of variability of starting point was prevalent, producing
described by Bogacz and Cohen (2004). For ER, this can be shorter RTs for errors than correct responses (Ratcliff & Rouder,
estimated as follows: Assume, for a given condition, that a partic- 1998). In Figure 5d, the participant emphasized accuracy over
D=0.5 D=1
a) 1
b) 1
0.8 0.8
Reaction time [s]
Reaction time [s]
0.6 0.6
0.4 0.4
0.2 0.2
0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
Probability Probability
D=2 D=0.5 Dpen=1.5

c) 1 d) 1
0.8 0.8
Reaction time [s]
Reaction time [s]
0.6 0.6
0.4 0.4
0.2 0.2
0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
Probability Probability
Figure 5. Fit of the extended drift diffusion model (DDM) to a sample participant in the experiment. Each
panel corresponds to one experimental delay condition labeled on the top of the panel. Circles indicate
experimental data: In each panel, the left column of circles corresponds to error trials, and the right to correct
trials. The horizontal axes show the probability of response (i.e., error rate for the left column and probability
of correct response for the right column). The vertical axes show the values of five quantiles of reaction time:
0.1, 0.3, 0.5, 0.7, and 0.9. Error bars indicate standard error. The approximately horizontal lines connect
predictions of fitted DDM for each of five quantiles. The following parameters of the extended DDM were
estimated: sx ⫽ 0.14, mA ⫽ 1, sA ⫽ 0.31, c ⫽ 0.33, z1 ⫽ 0.16, z2 ⫽ 0.19, z3 ⫽ 0.22, z4 ⫽ 0.26, and T0 ⫽ 0.37 s.
speed; thus, the threshold was high (z4 ⫽ 0.26), and the effect of inhibition, as does the strength of attraction toward the line and its
variability of drift was prevalent, producing longer RTs for errors location in the phase plane. Most of the interesting dynamics
than for correct responses (Ratcliff & Rouder, 1998). determining decisions occur along this line, and hence, we refer to
it as the decision line. Therefore, it is easier to understand these in
Mutual Inhibition Model, O-U Model, and DDM terms of new coordinates rotated clockwise by 45° with respect to
the y1 and y2 coordinates, so that one of the new axes is parallel to
Usher and McClelland (2001) observed that the dynamics of the the decision line. These new coordinates are shown in Figures 6b
difference y1 ⫺ y2 between activities of the decision units in the and 7a, denoted by x1 (parallel to the decision line) and x2 (per-
(linearized) mutual inhibition model are described by the O-U pendicular to it). The transformation from y to x coordinates is
model in general and, for certain balanced parameters, by the given by (cf. Grossberg, 1988; Seung, 2003)
DDM as well. Here, we provide an analytic interpretation of these
冦
observations and establish that when the decay and inhibition y1 ⫺ y2
parameters of the mutual inhibition model are equal and large, the x1 ⫽ ,
mutual inhibition model, O-U model, and DDM become equiva- 冑2
y1 ⫹ y2 (18)
lent. x2 ⫽ .
冑2
Reduction of the Mutual Inhibition Model to the Pure
Equations 18 derive from the geometry shown in Figure 7a: x1
DDM describes the difference between activities of the two decision
As illustrated above, in the section entitled DDM, the behavior units, whereas x2 describes the sum of their activities. The square
of the mutual inhibition model may be visualized by plotting states root of two in the denominators of Equations 18 is a normalization
on the phase plane. Figure 2c shows a representative path in state factor, included to ensure that y and x coordinates have the same
space: Initially, the activities of both decision units increase due to scale.
stimulus onset, but as the units become more active, mutual In deciding between two alternatives, it is natural that the
inhibition causes the activity of the weaker unit to decrease, and difference between the activities of the units selective for the
the path moves toward the threshold for the more strongly acti- alternatives should be a useful descriptor of the decision process.
vated unit (i.e., the correct choice). However, the new coordinates do more than merely emphasize this
To understand how these dynamics of the mutual inhibition point. They allow us to factor the two Equations 14 that describe
model are influenced by its two critical parameters (decay and the mutual inhibition decision process into two decoupled pro-
inhibition), Figure 6 shows its vector fields for three different cesses, separating the evolution of the difference in the activity of
ranges of these parameters. Each arrow shows the average direc- the two units (x1) from the change in their overall (summed)
tion in which the state moves from the point indicated by the activity (x2). If we can show that the latter has minimal impact on
arrow’s tail, and its length corresponds to the speed of movement the decision process, then we can reduce the description of this
(i.e., rate of change) in the absence of noise. In Figure 6, as for process from one that is two dimensional to a simpler one that is
most other simulations described in this article, we set I1 ⬎ I2; that one dimensional. As we show below, for certain parameters, this
is, we assume that the first alternative is the correct one (the one-dimensional description reduces to the DDM.
opposite case is obtained simply by reflecting about the diagonal To transform Equations 14 into the new coordinates, we first
y1 ⫽ y2). calculate the derivative (rate of change) of x1. Substituting Equa-
Note that in all three panels of Figure 6, there is a line (an tions 14 into the first of Equations 18, we obtain
eigenvector), sloping down and to the right, to which system states
are attracted: The arrows point toward this line from both sides.
The orientation of this line represents an important quantity: the
difference in activity between the two decision units. Note that the
dx1 ⫽
dy1 ⫺ dy2
冑2 冉
⫽ ⫺k
y1 ⫺ y2
冑2
⫹w
y 1 ⫺ y2
冑2
⫹
I1 ⫺ I2
冑2 冊 dt
evolution along the line differs for different values of decay and 1
⫹ 共cdW1 ⫺ cdW2 兲. (19)
冑2
a) Decay > Inhibition b) Decay = Inhibition c) Decay < Inhibition We assumed earlier that the noise processes for the input units are
(λ < 0) (λ = 0) (λ > 0)
independent. Because the standard deviation of the sum (or dif-
y2 y2 y2
ference) of two independent random variables is equal to the
square root of the sum of their variances, the noise process in x1
may be written
x2
1 冑c2 ⫹ c2
y1 y1 y1 共cdW1 ⫺ cdW2 兲 ⫽ dW1⬘ ⫽ cdW1⬘ . (20)
x1 冑2 冑2
Figure 6. Vector fields for the mutual inhibition model. In all plots, I1 ⫽ In Equation 20, dW1⬘ again denotes a noise process with mean
2, I2 ⫽ 1. Inhibition (w) and decay (k) have different values in different equal to 0 and an RMS strength of 1. Substituting Equation 20 and
panels. a: w ⫽ 0.5, k ⫽ 1.5. b: w ⫽ 1, k ⫽ 1. c: w ⫽ 1.5, k ⫽ 0.5. See text the definition of x1 from Equation 18 into Equation 19, we obtain
for discussion. Equation 21. Following analogous calculations for x2, we have
a) b)
y2
c) the system approaches the decision line faster (and often consid-
x2 D y2 erably faster) than it moves along it (e.g., see Figure 2c). More-
y2 Z B
Z
45°
z z
z over, if the denominator 公2(k ⫹ w) in Equation 23 is large rela-
x 2* A
Z tive to noise strength c, the states/solutions remain tightly bunched
y1 C y1 around this line as they continue to evolve. This is an important
0 x1
O Z y1
observation, as it indicates that the decision process divides into
two phases: an initial phase in which the activity of both units
increases quickly and there is rapid equilibration to a neighbor-
Figure 7. Geometry of the mutual and feedforward inhibition models. a:
hood around the decision line, followed by slower movement
Transformation from y to x coordinates. Small circle denotes a sample state
along the line, governed by an O-U process in which the difference
of the network. Labels y1 and y2 indicate the activation levels of decision
units. Labels x1 and x2 indicate the values of x (difference and sum) between the activities of the two units grows as one of them
coordinates for this state of the network. b: Thresholds for the balanced prevails and the other subsides.
mutual inhibition model. c: Thresholds for the feedforward inhibition Most relevant to the current discussion, when decay equals
model with u ⫽ 1. In Panels b and c, the grey arrow denotes the decision inhibition, the term (w ⫺ k)x1 in Equation 21 disappears. In this
line, or the line along which the diffusion occurs. Lines marked by Z denote case, the dynamics of x1 reduce to the pure DDM (cf. Equation 5).
the thresholds of the mutual and feedforward inhibition models. Labels A, The vector field for this case is shown in Figure 6b. When decay
B, C, D, and O are as used in the main text. and inhibition are both reasonably strong (as in Figure 6b), the
attraction toward the decision line dominates diffusion along it.
冉冊
Hence, typical paths migrate quickly toward the decision line and
I1 ⫺ I2 then move relatively slowly along (or near) it. In this case, the
dx1 ⫽ (w ⫺ k)x1 ⫹ dt ⫹ cdW1⬘ , and (21)
冑2 dynamics of the two-dimensional decision process closely approx-
冉冊
imate those of the one-dimensional DDM (see also Brown et al.,
I1 ⫹ I2 2005).
dx2 ⫽ (⫺k ⫺ w)x2 ⫹ dt ⫹ cdW2⬘ . (22)
冑2 Solutions to the full two-dimensional system (Equations 21–22)
are described by the joint probability density of x1 and x2. Because
Equations 21 and 22 are uncoupled; that is, the rate of change of
these are independent processes (neglecting threshold effects), this
each xi depends only on xi itself (this was not the case for the
is simply the product of the 2 one-dimensional Gaussian densities
decision units in Equations 14). Hence, the evolution of x1 and x2
for the O-U processes x1 and x2. Critically, when decay equals
may be analyzed separately, and in fact, each is described by an
inhibition and both assume sufficiently large values relative to
O-U process that depends on the parameters of k (decay) and w
noise strength, the mutual inhibition model’s behavior should
(inhibition).
closely resemble that of the pure DDM. We refer to a mutual
We first consider the dynamics in the x2 direction, correspond-
inhibition model in which decay is equal to inhibition as balanced.
ing to the summed activity of the two decision units. Equation 22
for the x2 process always gives a stable O-U process because ⫺k ⫺
w ⬍ 0 (recall that k and w are both constrained always to be Relating Parameters of the Mutual Inhibition Model to
non-negative). This corresponds to attraction to the line in all of the Pure DDM
the panels in Figure 6, implying that x2 approaches a limiting value In this section, we find the parameters describing the diffusion
as time increases. The rate of this (exponential) approach is ⫺␭2 ⫽ process along the decision line in the mutual inhibition model. As
k ⫹ w, and it is kept constant in the three cases of Figure 6 by above, the transformation is geometric and so requires the same
setting k ⫹ w ⫽ 2. From Equation 11, the stationary standard simple trigonometry used above. From Equation 21, we can see
deviation from the decision line in the x2 direction is equal to that the RMS noise of the approximating DDM is equal to c, and
the drift is given by
c
stdev共x2 兲 ⫽ . (23)
冑2共k ⫹ w兲 A⫽
I1 ⫺ I2
. (24)
Turning to the x1 process, Equation 21 involves a drift term that is
冑2
proportional to the difference between the inputs I1 and I2. As we Calculation of effective thresholds for the diffusion process re-
discussed above, in the section entitled Ornstein–Uhlenbeck (O-U) quires a little more work. If the density of solutions is sufficiently
Model, this process may be stable or unstable depending on the tight about the decision line, we need only find the points where
relative magnitudes of k and w, in a manner comparable to the this line reaches the thresholds y1 ⫽ Z and y2 ⫽ Z (D and C in
effects of the parameter ␭ in the O-U model. This dependence is Figure 7b). More specifically, we need to find the distances ⫾z to
due to the fact that the dynamics of x1 are described in Equation 21 those points from the intersection of the diagonal (x1 ⫽ 0) with the
by an O-U process with coefficient ␭ ⫽ w ⫺ k. When decay is decision line at point A. Because the diagonal and decision line are
larger than inhibition, then ␭ ⬍ 0, and there is an attractor for the oriented at ⫾45° to the y2-axis, the triangles ABC and ABD in
x1 dynamics (see above), as shown in Figure 6a. When decay is Figure 7b are isosceles, and the distances AC, AD, and AB are all
smaller than inhibition, then ␭ ⬎ 0, and there is repulsion from the equal to z. The decision line is located at the equilibrium for the
fixed point in the x1 direction, as shown in Figure 6c. The fixed stable O-U process:
point is a saddle in this case.
Because 兩k ⫹ w兩 (for x2 in Equation 22) ⬎ 兩w ⫺ k兩 (for x1 in I1 ⫹ I2
x*2 ⫽ . (25)
Equation 21) for all positive values of k and w, the average state of 冑2共k ⫹ w兲
Thus, again from Figure 7b and using the right isosceles triangle Similarly, under the above assumptions, the starting point of the
OBZ, the length of AB is equal to the difference of the lengths OB diffusion along the decision line is
and OA, or
y1 共0兲 ⫺ y2 共0兲
x1 共0兲 ⫽ ⫽ 冑2y1共0兲. (30)
z⫽ 冑2Z ⫺
I1 ⫹ I2
. (26) 冑2
冑2共k ⫹ w兲 Therefore, the relationship between parameters sx and sy is sx ⫽
公2sy .
Relating Parameters of the Mutual Inhibition Model to
The above relationships are verified in the simulations described
the Extended DDM in the next section.
The extended DDM includes variability of drift and starting
point across trials. To relate this model to the mutual inhibition ERs and DTs in the Balanced Mutual Inhibition Model
model, these two types of variability need to be incorporated into and DDM
the latter. Therefore, we define the extended mutual inhibition
As argued above, the balanced mutual inhibition model resem-
model in which the mean inputs to the integrators Ii are chosen on
bles the DDM. However, these models are not identical because
each trial from a normal distribution with mean mIi and standard
the mutual inhibition model describes a two-dimensional process,
deviation sI and the starting points of variables yi(0) are drawn
including movement along the x2-axis (corresponding to the
from a uniform distribution between ⫺sy and sy .
summed activity level of the decision units), whereas the DDM
First, let us consider the case of statistically independent I1, I2,
describes a one-dimensional process involving movement only
y1(0), and y2(0). In this case, the sum I1 ⫹ I2 differs from trial to
along the x1-axis (corresponding to the difference in activity lev-
trial, and because this sum enters Equation 26 relating the thresh-
els). As a way of assessing how well the DDM approximates the
olds of the mutual inhibition model and pure DDM, there is no
balanced mutual inhibition model, we compare ERs and DTs
deterministic relationship between the thresholds of the extended
produced by the extended versions of each model when their
mutual inhibition model and DDM (note, however, that if k, w 3
parameters are constrained according to the relationships derived
⬁, the second term in Equation 26 disappears, and the relationship
above, in the section entitled Relating Parameters of the Mutual
simply becomes z ⫽ 公2Z). Furthermore, the distribution of start-
Inhibition Model to the Extended DDM, and fit to experimental
ing points x1(0) projected onto the decision line x2 has a triangular
data described above, in the section entitled TAFC Experiment and
distribution (because it is given by the sum of two independent,
Fit of DDM.
uniformly distributed random variables), so the diffusion along the
As anticipated, the approximation of the extended mutual inhi-
decision line cannot be described by the extended DDM, which
bition model by the extended DDM is more accurate for larger
assumes uniformly distributed initial states.
decay and inhibition (k ⫹ w). This is due to the fact that as the
It is, however, straightforward to establish the equivalence be-
parameter ␭2 (⫽ ⫺[k ⫹ w]; see Equation 22) becomes more
tween the extended versions of the mutual inhibition model and
negative, attraction along the x2 direction toward the decision line
DDM if we make the following additional assumptions: I1 ⫹ I2 ⫽
is faster, and solutions become more tightly distributed around it
mI1 ⫹ mI2 ⫽ constant, and y1(0) ⫹ y2(0) ⫽ 0. The first assumption
(via Equation 23). This is evident in Figure 8, which compares the
states that the total input to the integrators is constant across trials
ERs and DTs of the extended versions of the mutual inhibition
(as in the Poisson counter model of Smith & Van Zandt, 2000).
model and DDM (with equivalent parameters). Figure 8a illus-
Although this assumption may not be satisfied exactly, adaptation
trates the case of high decay and inhibition, in which the solutions
in sensory cortices (Blakemore & Campbell, 1969) is known to
are tightly distributed about the decision line, and ERs and DTs of
reduce the neuronal response when the stimulus is strong and to
the two models are almost exactly the same. In the case of lower
enhance it when the stimulus is weak. The second assumption
decay and inhibition shown in Figure 8b, the solutions are more
states that the starting points of the two integrators are anticorre-
broadly distributed about the decision line, leading to greater
lated. Although as yet there is no physiological evidence for such
discrepancy between the ERs and DTs of the two models. Figure
anticorrelation in cortical populations, there are indications of this
8c illustrates the case in which the decay and inhibition are further
in the superior colliculus (R. Ratcliff, personal communication,
decreased so that most solutions do not reach the decision line
June 25, 2004).
before exceeding the threshold, resulting in substantially different
Under the above assumptions, I2 ⫽ mI1 ⫹ mI2 ⫺ I1, and hence,
ERs and DTs for the two models.
the drift along the decision line is equal to
Two additional points are worth noting with respect to these
I1 ⫺ I 2 2I1 ⫺ mI1 ⫺ mI2 results. First, movement toward the decision line may be facilitated
A⫽ ⫽ . (27)
冑2 冑2 by the high transient activity of sensory input neurons observed
just after stimulus onset, as illustrated in Figure 1b (Britten et al.,
Therefore, the relationships between the parameters are 1993; Schall, 2001). Smith (1995) also suggested the existence of
mI1 ⫺ mI2 the transient on the basis of psychophysical data. Second, if decay
mA ⫽ , sA ⫽ 冑2sI , (28) and inhibition are very low (or the signal is very large), then,
冑2 according to Equation 26, the threshold of the DDM may become
and the relationship between the thresholds is negative, a condition for which our reduction to one dimension is
ill defined.
mI1 ⫹ mI2
z⫽ 冑2Z ⫺ . (29) In summary, when decay and, in particular, inhibition in the
冑2共k ⫹ w兲 mutual inhibition model are very low, there is little competition
a) w=k=100 b) w=k=10 c) w=k=1

1 1 2
0.8 0.8
1.5
Reaction time [s]
Reaction time [s]
Reaction time [s]

0.6 0.6
1
0.4 0.4
0.5
0.2 0.2
0 0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
Probability Probability Probability
y2
y2
y2
0.5 0.5 0.5
0 0 0
-0.5 -0.5 -0.5
-0.5 0 0.5 y1 -0.5 0 0.5 y1 -0.5 0 0.5 y1
Figure 8. Similarity in error rates (ERs) and decision times (DTs) between the extended diffusion model and
balanced mutual inhibition model. Top panels show ER and quantiles of DT distribution for the extended drift
diffusion model (solid line) and extended mutual inhibition model (dashed line). In each panel, the left ends of the lines
correspond to error trials and the right to correct trials. The horizontal axes show the probability of response (i.e., ER
for the left ends and probability of correct response for the right ends). The vertical axes show the values of five
quantiles of reaction time: 0.1, 0.3, 0.5, 0.7, and 0.9. Approximately horizontal solid lines show the ERs and DTs of
the diffusion model fitted to condition D ⫽ 0.5, Dp ⫽ 1.5 of the participant from Figure 5 (these lines in all top panels
are the same as in Figure 5d). Dashed lines correspond to values obtained from simulations of the extended mutual
inhibition model (10,000 iterations for different values of decay and inhibition shown above panels), and all other
parameters are defined as in the text section Mutual Inhibition Model, O-U Model, and DDM to give equivalence with
the extended diffusion model. Specifically, mI1 ⫽ 1.42, mI2 ⫽ 0, c ⫽ 0.33, sI ⫽ 0.22, and sy ⫽ 0.096. Note that for
higher values of decay and inhibition, the match between the models improves. Note difference in scales in the top
of Panel c. Bottom panels show the states from 1,000 trials, at time t ⫽ T0 ⫹ 200 ms. Long-dashed lines show the
decision threshold. Short-dashed lines show the decision line.
冉冑冊
冦
between the two decision units: They integrate evidence essen- I1 ⫺ I2
tially independently, and hence, the mutual inhibition model can- dx1 ⫽ 共1 ⫹ u兲 dt ⫹ cdW1⬘ ,
2
冉冑冊
not be approximated by the DDM. However, when decay and I1 ⫹ I2 (31)
inhibition are greater, the balanced mutual inhibition model can be dx2 ⫽ 共1 ⫺ u兲 dt ⫹ cdW2⬘ .
2
approximated by diffusion along the decision line, as described by
the DDM. Thus, similar to the mutual inhibition model, the dynamics of the
feedforward inhibition model in the rotated coordinates can be
Feedforward Inhibition Model and DDM described by two uncoupled diffusion processes weighted by fac-
tors 1 ⫹ u and 1 ⫺ u. When we defined the mutual and feedfor-
In this section, we show that the feedforward inhibition model ward inhibition models, we implicitly assumed that the weight of
reduces to DDM when the parameter u describing the weight of connections from input units to the integrating (decision) units is
feedforward inhibition is equal to u ⫽ 1. equal to 1. Note that as the weight of the inhibitory connection (u)
To analyze the dynamics of the feedforward inhibition model, approaches the weight of the excitatory connection (1), the weight-
we convert it to the rotated coordinates as we did the mutual ing factor 1 ⫺ u diminishes, and thus, dynamics become slower
inhibition model. Transforming Equation 15 via Equation 18 (as along the direction of x2 relative to the direction of x1. However,
done above), we obtain unlike the case of the mutual inhibition model, in the feedforward
inhibition model given by Equation 26, there is no attraction the excitatory populations. If we assume that this rate is fast
toward the decision line. Thus, approximation by the one- relative to that of the excitatory populations7 (the effective decay
dimensional DDM depends on the mean and variance in the x2 rate of the excitatory population is decreased by the self-excitatory
direction remaining negligible for the duration of the decision task. connections; thus, this assumption is equivalent to kinh Ⰷ k ⫺ v),
If the excitatory and inhibitory weights are equal (u ⫽ 1, as they then inhibitory neurons may be said to have little memory of their
are in the model described by Ditterich et al., 2003), then the value previous inputs, closely tracking and thus approximating the value
of coordinate x2 remains equal to zero throughout the duration of
the decision process, and the entire process proceeds along the w⬘
decision line x1. In other words, if u ⫽ 1, the feedforward inhibi- y3 ⫽ 共y ⫹ y2 兲. (36)
kinh 1
tion model is exactly equivalent to the pure DDM. In this case,
diffusion occurs along the line shown in Figure 7c, and the Substituting Equation 36 into Equation 35 and regrouping terms,
relationships between the parameters of the pure DDM and the we obtain
feedforward inhibition model are the following:
冉冉冊冊
冦
I1 ⫺ I2 ww⬘ ww⬘
A 4 2 , c 4 2c, z 4 冑2Z, (32) dy1 ⫽ ⫺ k ⫹ ⫺ v y1 ⫺ y ⫹ I1 dt ⫹ cdW1 ,
冑2 kinh kinh 2
冉冉冊冊
ww⬘ ww⬘ (37)
where the left-side terms are for the DDM and the right-side terms dy2 ⫽ ⫺ k ⫹ ⫺ v y2 ⫺ y ⫹ I2 dt ⫹ cdW2 .
kinh kinh 1
are for the feedforward inhibition model. The relationships for the
drift and noise terms come from the first of Equations 31, and for A comparison of Equations 37 and 14 reveals that the connection-
the threshold from Figure 7c. ist model with the architecture of Figure 3d is equivalent to the
As in the previous section, we can define an extended feedfor- mutual inhibition model shown in Figure 3b with decay parameter
ward inhibition model, where we also assume I1 ⫹ I2 ⫽ constant equal to (k ⫹ ww⬘/kinh ⫺ v) and mutual inhibition ww⬘/kinh.
and y1(0) ⫹ y2(0) ⫽ 0. For u ⫽ 1, the relationships between the Therefore, pooled and rapidly equilibrated inhibition not only
parameters of the extended versions of the feedforward inhibition provides competition but also contributes to the decay of each
model and DDM are the following: unit’s activity. Hence, given the considerations of the above sec-
mI1 ⫺ mI2 tion, Mutual Inhibition Model, O-U Model, and DDM, the con-
mA 4 2 , sA 4 冑2sI, sx 4 冑2sy ,
冑2 nectionist model with the architecture shown in Figure 3d displays
the same dynamics in the x1 direction as the pure DDM when the
c 4 2c, z 4 冑2Z. (33) following balance among parameters holds:
Busemeyer and Diederich (2002) considered the following model k ⫹ ww⬘/kinh ⫺ v ⫽ ww⬘/kinh or k ⫽ v. (38)
combining mutual and feedforward inhibition:6
冋dydy 册 ⫽ 冋⫺k 册冋 yy dtdt 册

⫺w The dynamics in the x2 direction collapse to the decision line,
1 1
⫺w ⫺k exactly as for the mutual inhibition model, with a timescale pro-
2 2
portional to kinh/ww⬘.
冋
1 ⫺u
⫹ ⫺u 1 册冋II dtdt ⫹⫹ cdW
1
2 cdW 册 .
1
2
(34)
To summarize, the dynamics of the connectionist model with
pooled inhibition can be approximated by the pure DDM when
They have shown that as u ⫽ 1, the above model can be related to self-excitation exactly balances the decay of each decision unit’s
the O-U process (Busemeyer & Diederich, 2002; Diederich & activity (i.e., k ⫽ v), inhibitory strengths (w, w⬘) are sufficiently
Busemeyer, 2003). If such a combined model were balanced (k ⫽ high, and the decay of inhibitory units kinh is sufficiently large.
w), then as k and w increased and u approached 1, this combined Furthermore, paralleling the arguments of the previous section, an
model would be also increasingly well approximated by the pure extended pooled inhibition model may be defined and related to
DDM.
6
Mutual Inhibition and Pooled Inhibition Models The first square matrix in Equation 34 was denoted by S and the second
square matrix by C in Busemeyer and Diederich (2002).
In this section, we show that a connectionist model with pooled 7
The assumption that the decay rate kinh is relatively fast is plausible but
inhibitory connections can also be approximated by the pure will require validation in further physiological experiments. This rate
DDM. Specifically, we consider a connectionist model with the depends on the dynamics of synaptic currents afferent to the inhibitory
architecture shown in Figure 3d, the dynamics of which are de- population. Although, in the original model of Wang (2002), inhibitory
scribed by the following three (linearized) equations (we rewrite neurons receive a significant proportion of their input via NMDA receptors
Equations 16): (described by relatively slow time constants of approximately 100 ms;
再
Wang, 2002), the density of such receptors remains to be determined in
dy1 ⫽ (⫺ky1 ⫺ wy3 ⫹ vy1 ⫹ I1 )dt ⫹ cdW1 , area LIP. Current research indicates that inhibitory neurons in the hip-
dy2 ⫽ (⫺ky2 ⫺ wy3 ⫹ vy2 ⫹ I2 )dt ⫹ cdW2 , (35) pocampus do possess NMDA receptors (McBain & Dingledine, 1993), but
dy3 ⫽ (⫺kinhy3 ⫹ w⬘(y1 ⫹ y2 ))dt. Ling and Benardo (1995) have shown that the inhibition in the rat somato-
sensory cortex is not NMDA dependent. A parallel finding for the LIP
In the above equation, kinh denotes the decay rate with which the inhibitory neurons, showing that they are excited via rapidly decaying
pool of inhibitory neurons approaches firing rates determined by AMPA receptors, would support the present assumption.
the extended DDM.8 We do not consider the pooled inhibition decay and inhibition) is irrelevant. As a consequence, in contrast to
model further in view of its close relationship to the mutual free response, in the interrogation paradigm, the balanced mutual
inhibition model (for large kinh , cf. Equations 37). inhibition model achieves the same ER as the DDM irrespective of
the magnitudes of decay and inhibition. This is true even in the
Optimal Performance of Decision Networks extreme case of k ⫽ w ⫽ 0, corresponding to the race model in
which the decision units integrate independently of one another.
In this section, we identify parameter values that allow decision Similar conclusions hold for the feedforward inhibition model
networks to achieve optimal performance in the interrogation and (which minimizes ER in the interrogation paradigm for any value
free-response paradigms (the first and second sections immedi- of u) and the pooled inhibition model (if the dynamics in the x1
ately below, respectively). The third section below investigates direction are balanced by setting k ⫽ v, for any values of w, w⬘,
whether the ERs in decision models can be made arbitrarily small and kinh). Hence, the mutual inhibition model can in general be
if a sufficiently long time is allowed for the decision in the
approximated by an O-U process with parameter ␭ equal to the
interrogation paradigm or if the threshold is made sufficiently high
difference between inhibition and decay (i.e., ␭ ⫽ w ⫺ k), and in
in the free-response paradigm. The results of this section for the
the case of the interrogation paradigm, these are identical. There-
mutual inhibition model are summarized in Table 1. In particular,
fore, we focus exclusively on the O-U model and ask what value
we show that for the parameters for which decision networks
of ␭ optimizes performance.
reduce to DDM, they achieve optimal performance in both para-
First, we analyze the pure O-U model (without variability of
digms.
drift or starting point). In Appendix A (section entitled Optimal
Decisions Under the Interrogation Protocol), we calculate explicit
Highest Accuracy for Fixed DTs: The Interrogation expressions for the ERs for different interrogation times T and
Paradigm parameters A, c, and ␭, as already given above in Equation 12.
As described in the introductory section, the pure DDM is also These verify that the O-U process achieves its lowest ER for a
the optimal decision maker (i.e., is equivalent to the Neyman– fixed DT when ␭ ⫽ 0 (i.e., when it simplifies to the DDM). This
Pearson test) in the interrogation paradigm; that is, it is the most approximates the mutual inhibition model with decay equal to
accurate for a fixed DT. In this section, we show that the mutual inhibition. Figure 9a shows ERs achieved by the O-U process with
inhibition model achieves such optimality when it is balanced. different values of ␭ and different interrogation times. Usher and
It is worth noting here that in the interrogation paradigm, the ER McClelland (2001) noted that ERs depend only on the absolute
of the mutual inhibition model does not depend on the magnitude magnitude of ␭ (e.g., they are the same for ␭ ⫽ 0.2 and ⫺0.2 and
of decay and inhibition but only on their relative values. As we for 0.4 and ⫺0.4) and that the lowest ERs are achieved when ␭ ⫽
show below, this contrasts with the free-response paradigm, in 0. These observations are consistent with the analyses of Appendix
which performance depends additionally on the absolute magni- A (section entitled Minimizing ER for Unbiased Choices). In
tude of decay and inhibition. The reason is that in the interrogation particular, the fact that the lowest ER in the interrogation paradigm
paradigm, decisions are based solely on a comparison of activities is achieved by an O-U process with ␭ ⫽ 0 confirms the earlier
of the decision units and hence only on the value of the x1 observation that the mutual inhibition model achieves the lowest
coordinate. In effect, the interrogation paradigm allows the two- ER when it is balanced and approximates the DDM.
dimensional probability density to evolve and then, at the time of We now consider the extended O-U model, in which drift rate
interrogation, cuts it along the x2-axis and sums it in the negative and/or starting point are allowed to vary from trial to trial. Ap-
(error) and positive (correct) x1 half-planes. Therefore, the density pendix A (section entitled Optimal Values of ␭ for the O-U Model
of solutions about the decision line (that tightens under increased With Variable Drift in Interrogation Protocol) calculates the ER of
the O-U model with variability of drift but not starting point and
shows that for any mA, sA, c, and interrogation time T, the ER is
Table 1 minimized for ␭ ⫽ 0. Thus, the mutual inhibition model with
Conditions on Parameters k (Decay) and w (Inhibition) of the variability of inputs (but not starting points) achieves the lowest
Mutual Inhibition Model for Optimal Performance (Minimal ER ER in the interrogation paradigm when it is balanced (k ⫽ w).
in the Interrogation Protocol or Maximal Reward Rate for Free In contrast with variable drift rate, simulations of Figure 9c
Response) and for Vanishing ER in the Limit of Long Decision indicate that when starting point is allowed to vary, optimal per-
Time (Interrogation Protocol) or Large Thresholds formance of the O-U model (i.e., minimization of ER) may be
(Free Response) achieved for values of ␭ lower than zero. This may be intuitively
understood as follows: Values ␭ ⬍ 0 introduce a recency effect
Property No variability Variability of drift
(with recently arrived inputs influencing accumulated evidence
Optimal performance more than previous inputs or biases; see above section, Ornstein–
Interrogation k⫽w k⫽w Uhlenbeck (O-U) Model), which reduces the influence of the
Free response k⫽w k ⫽ w (suggested by
simulations)
k, w 3 ⬁ k, w 3 ⬁ 8
ER 3 0 The more-detailed firing rate model of Wong and Wang (2006), which
Interrogation k⫽w Not attainable includes specific synaptic time constants, indicates that Equation 38 may
Free response kⱖw Not attainable sometimes be violated. Depending on the overall stimulus strength, on
which the analogs of k, w, and so on, depend, both the stable and unstable
Note. ER ⫽ error rate. O-U cases of Figure 6 can occur.
a) interrogation b) free-response c)
0.5 1
λ = -6 or 6 λ = -6
λ = -3 or 3 λ=0
0.4 λ=0 0.8 λ=6
0.3 0.6
ER
ER
0.2 0.4
0.1 0.2
0 0
0 0.5 1 1.5 2 0 0.5 1 1.5 2
time [s] z
Figure 9. Error rates (ERs) for the Ornstein–Uhlenbeck (O-U) model. a: ER in the interrogation protocol
for decisions made at different times and for different values of ␭, according to Equations 8 and 12, with
A ⫽ 2.19 and c ⫽ 1 (parameters estimated from data from middle-difficulty condition of Participant S1 in
the experiment described in Usher & McClelland, 2001). b: ER in the free-response protocol. O-U models
were simulated for values of parameters corresponding to the evolution along the decision line in Figure 11;
that is, for A ⫽ 2.19, c ⫽ 1. O-U models with three different values of ␭ were simulated: ⫺6, 0, and 6,
shown by the three thick curves in different patterns (see legend). The value of the decision threshold is
shown on the horizontal axis. For each set of parameters, the model was simulated 1,000 times, and the
proportion of incorrect decisions is shown by solid lines. The thin line denotes the proportion of trials for
which the decision has not been reached by time t ⫽ 100 in the case ␭ ⫽ ⫺6. The ERs for ␭ ⫽ 0 and ␭ ⫽
6 very closely match the values predicted by Equation 8 and by Equation A55 in Appendix A, respectively
(although theoretical predictions are not shown for clarity of the figure). c: ER of the extended O-U model
with the variability of starting point in the interrogation paradigm. The model was simulated for the
following parameters: mA ⫽ 2.19, sA ⫽ 0, c ⫽ 1 (parameters estimated from data from middle-difficulty
condition of Participant S1 in the experiment described in Usher & McClelland, 2001), and sx ⫽ 0.5 at
interrogation time t ⫽ 0.1 s. For each value of ␭, the O-U model was simulated 20,000 times; the average
ER is shown by the solid line. Error bars show the standard error of the mean.
random starting point on the decision that is eventually made. This ␭ (perhaps via w), which would cause strong repulsion from the
suggests that when starting point varies, unbalanced versions of the fixed point and, thus, rapid movement toward one of the thresholds
mutual inhibition model may be optimal. However, this observa- used in the free-response paradigm. Such weight adjustments
tion needs to be considered with respect to the psychological could be achieved by a transient increase in gain, which might in
interpretation of variable starting points. Typically, starting point turn be mediated by release of neuromodulatory neurotransmitters.
is thought to reflect the influence of prior stimulus or response The locus coeruleus has been suggested as a possible element in
history on current processing as a result of priming effects and/or such a control circuit (Aston-Jones & Cohen, 2005; Brown et al.,
expectations based on estimations of stimulus and response prob- 2005; Brown, Gilzenrat, & Cohen, 2004; Gilzenrat, Brown, Aston-
abilities (e.g., Cho et al., 2002). In the section entitled Biased Jones, & Cohen, 2004; Usher, Cohen, Servan-Schreiber, Ra-
Decisions, below, we demonstrate that variations in the starting jkowski, & Aston-Jones, 1999), but further consideration of such
point are in fact optimal for all models when these variations mechanisms is beyond the scope of this article.
appropriately track systematic changes in stimulus frequency.
There, we return to a discussion of how to interpret the effects of Fastest Decisions for Fixed Accuracy: The Free-Response
variable starting point in fits of the DDM and related models to Paradigm
empirical data.
The analyses and simulations described here provide insight into As described above, the pure DDM achieves the fastest DTs for
the parameters of the mutual inhibition model that produce optimal a given level of accuracy, and in this sense, it is the optimal
performance in the interrogation paradigm under various condi- decision process for the free-response paradigm. Above, in the
tions. However, they do not explain how a biological system might section entitled Mutual Inhibition Model, O-U Model, and DDM,
actually make the decision at the required time in the interrogation we established that the mutual inhibition model’s behavior is
paradigm, that is, how the system determines the sign of x1 when similar to that of the DDM when decay equals inhibition and both
it has not yet crossed a specified threshold. A number of mecha- assume a sufficiently large value. Usher and McClelland (1995,
nisms for doing so can be imagined. For example, this could be footnote on p. 22) noted that their model also achieves its shortest
accomplished by setting a very high value of the decision threshold DTs for fixed accuracy when decay equals inhibition—that is,
during the trial and then lowering it rapidly or suddenly increasing when it best approximates the DDM. Similar observations were
made for the feedforward inhibition model. In this section, we that when the inhibitory and excitatory connections from the inputs
show that these relationships still hold when drift rate is allowed to are equal (at u ⫽ 1, as in the original model by Ditterich et al.,
vary across trials in these models. 2003), then the feedforward inhibition model achieves the same
In particular, we conjecture that when drift rate is allowed to DT predicted by the DDM (indicated by the dashed line). This is
vary but starting point is held constant, performance of the mutual expected because, as described above, in the section entitled Feed-
inhibition model is optimized when decay is equal to inhibition forward Inhibition Model and DDM, when u ⫽ 1, the feedforward
and both take large values (that is, when it approximates the inhibition model is exactly equivalent to the DDM and hence
DDM). To test this, we simulated the mutual inhibition model with achieves optimal performance. We do not discuss the feedforward
different values of decay k, while all other parameters were held inhibition model further in this article because it can be considered
constant. Specifically, the weight of inhibition9 was set to w ⫽ 10, equivalent to the DDM when u ⫽ 1.
the range of starting points was set to sy ⫽ 0, and the remaining When starting point is allowed to vary, simulations indicate that,
parameters were set by fitting the extended DDM to data from a as in the case of the interrogation paradigm, balancing decay and
sample participant (see Figure 5) and then determining equivalent inhibition may not be optimal (i.e., for fixed w ⫹ k, a minimum DT
parameters for the extended mutual inhibition model as described for fixed ER is achieved for w ⫽ k). However, we note again that
above in the section entitled Mutual Inhibition Model, O-U Model, this finding should be considered within the context of a psycho-
and DDM. For each value of decay, the threshold was set such that logical interpretation of the starting point and its relationship to
the ER of the network was 2%. Figure 10a shows the DTs stimulus and response frequencies, which we address below, in the
produced by this model. As expected, the shortest decisions are section entitled Variable Signal Strength and Adjustment of Drift
achieved when decay equals inhibition (i.e., k ⫽ 10). Rate.
Even when the parameters are balanced in this manner, the
mutual inhibition model only approximates the DDM because its Balanced Systems and Arbitrarily Small Errors
state merely approaches the decision line and remains with high
probability in its proximity, rather than moving exactly along the For the decision models considered in this article, increasing
line. As a result, sample paths cross thresholds Z over an interval interrogation time in the interrogation paradigm or threshold in the
along which the coordinate x2 takes different values. This is free-response paradigm leads to increased accuracy. This suggests
nonoptimal: The SPRT requires a single pair of threshold points. another sense in which these models may be optimal: We can ask
The larger decay and inhibition are, the closer the mutual inhibi- whether ERs can be reduced to arbitrarily small values in the
tion model is to the DDM (because attraction toward the line is models of these two paradigms.10 We first consider the case of
stronger). Hence, we expect that the balanced mutual inhibition drift being constant across trials (pure models). We show that in
model will achieve the shortest DTs for fixed ERs when decay and the interrogation paradigm, ERs can be reduced to arbitrarily small
inhibition are larger. This prediction is verified in the simulations values for the DDM and the balanced mutual inhibition model (i.e.,
shown in Figure 10b. The mutual inhibition model was simulated with decay equal to inhibition). In the free-response paradigm, this
with different values of decay k and inhibition w (maintaining is true for both the balanced mutual inhibition model and an
equality of the two), while all other parameters were held constant unbalanced variant in which decay dominates inhibition. Then, we
across simulations. For each value of decay, the threshold was set show that if drift varies across trials, arbitrarily small error cannot
such that the ER of the network was 10%. Figure 10b shows that be achieved by any of the models (results are summarized in Table
as decay and inhibition increase, DT decreases. Furthermore, the 1). The variability of starting point does not influence the ER as
mutual inhibition model’s DT approaches that predicted by the opti- threshold or interrogation time goes to infinity.
mal DDM for a 2% ER, shown by the dashed line in Figure 10b. Assuming collapse to a one-dimensional process for the long
In summary, the mutual inhibition model approaches optimal DTs in question, in what follows we consider only O-U processes
performance in the free-response paradigm when decay equals that approximate the mutual inhibition model (although, as dis-
inhibition and both increase in magnitude. Thus, as in the interro-
gation paradigm, optimality depends on the equivalence of decay 9
Note from Figure 9b that, for w ⫽ 10, the model performs far from fully
and inhibition. However, in the free-response paradigm, optimality optimally. We choose this value to illustrate the robustness of the property that
further depends on the absolute magnitude of these parameters. In balanced decay and inhibition improve performance—this fact holds even if
the extreme, when both decay and inhibition are zero (leftmost these quantities are not so large as to enforce collapse of the dynamics to a
point in Figure 10b), the DT in the mutual inhibition model single dimension. That is, balancing these parameters improves performance
corresponds to that of the race model (because, in this case, the even in regimes in which performance is not fully optimal.
10
mutual inhibition model becomes precisely the race model; see It must be noted here that humans rarely achieve perfect performance,
Figure 4), which has a slower DT than the balanced mutual even when a premium is placed on accuracy and participants are given
inhibition model (with any k ⫽ w ⬎ 0). unlimited time to respond (McElree & Dosher, 1989; Ratcliff, 1978; Reed,
Figure 10c shows similar results for the feedforward inhibition 1973; Usher & McClelland, 2001). This discrepancy with certain decision
model: DT for fixed ER ⫽ 2% and different weights (u) of the models has been cited as a challenge to their veracity (in particular, by
Usher & McClelland, 2001, with respect to the DDM). However, Ratcliff
inhibitory connections. The DT for u ⫽ 1 again corresponds to the
(1988) showed that the imperfect accuracy of human performance in long
DT of the race model; both leftmost points in Figures 10b and 10c time limits may be explained by making a simple, biologically plausible
correspond to the same value of DT (the small difference reflects modification to the DDM: adding boundaries limiting the maximum and
a simulation artifact due to finite sample size). The race model has minimum values of the diffusion variable x. We have confirmed this and
a slower DT than the feedforward inhibition model (for any u ⬎ 0), provided analytical estimates, but we do not treat this issue here, leaving its
as it does in comparison with the mutual inhibition model. Note further study for future work.
a) Mutual inhibition model b) Mutual inhibition model
0.5 0.5
Race
0.45 0.45
Decision Time [s]
Decision Time [s]

0.4 0.4
0.35 0.35
0.3 0.3
6 8 10 12 14 0 20 40 60 80 100 120
k k
c) Feedforward inhibition model d) Mutual inhibition model

0.435
0.5
Race 0.434
Reward rate [rewards/s]

0.45 0.433
Decision Time [s]
0.432
0.4
0.431
0.35 Diffusion 0.43
0.3 0.429
0.428
0 0.2 0.4 0.6 0.8 1 4 6 8 10 12 14 16
u k
Figure 10. Performance of mutual and feedforward inhibition models. During all simulations, the following
parameters were kept fixed: mI1 ⫽ 1.42, mI2 ⫽ 0, c ⫽ 0.33, sI ⫽ 0.22 (the same values as in Figure 8), and sy
⫽ 0. Panels a, b, and c show decision time (DT) for the threshold set such that error rate (ER) ⫽ 2%. a, b: DT
for the mutual inhibition model for different values of inhibition. In Panel a, inhibition is kept constant at w ⫽
10, and decay (k) is varied (shown on horizontal axis). In Panel b, inhibition is equal to decay (w ⫽ k), and they
are varied together. c: DT for the feedforward inhibition model as the weight of the inhibitory connections (u)
is varied (shown on horizontal axis). For each set of parameter values, the threshold was increased from zero in
steps of 0.01 until the model reached an ER less than or equal to 2%. For each value of the threshold, 10,000
trials were simulated. The error bars indicate standard error of mean DT estimation. d: Reward rate achieved by
the mutual inhibition model with optimal thresholds at different levels of decay. During simulations, the
following parameters were kept constant: w ⫽ 10; the delays between trials were equal to D ⫽ 2, Dp ⫽ 0. The
decay rate k is shown on the horizontal axis. For each set of parameter values, the threshold was chosen that
maximized the reward rate (by simulation), and the model was run for 100 simulated blocks of 10,000 trials each.
The lines on the graphs connect the mean reward rates achieved by the model, and the error bars show the
standard error across the blocks.
cussed above, we do not depend on this collapse for our O-U Clelland, 2001). In Equation 39, ⌽ denotes the normal standard
model to accurately describe the interrogation paradigm). cumulative distribution function, and A and c denote the drift and
standard deviation of noise as before. From this formula, it follows
Interrogation Paradigm that the ER can be arbitrarily close to zero only when ␭ ⫽ 0, that
is, when decay is equal to inhibition in the mutual inhibition
The minimum ER that can be achieved by an O-U model (i.e., model. Indeed, for ␭ ⫽ 0, there is a certain probability of error
after interrogation at infinite time) is given by: even for very long interrogation times, although this may be small
冉冑冊
ER ⫽ ⌽ ⫺
2A2
兩␭兩c2
(39)
for high drift rates (i.e., easy decisions).
To illustrate this result, Figure 11 shows the evolution of states
of the mutual inhibition model for three different ratios of decay
(from Equation 12 above via Equation A94 of Appendix A; and inhibition. These simulations represent moderately difficult
Busemeyer & Townsend, 1992; cf. Equation 10 in Usher & Mc- decision tasks (mean input to the first unit I1 ⫽ 6.1 [s⫺1], to the
a) Decay > Inhibition b) Decay = Inhibition c) Decay < Inhibition

(λ < 0) (λ = 0) (λ > 0)
4 4 4
y2
y2
y2
2 2 2
t = 0.5
0 0 0
-2 -2 -2
-4 -4 -4
-4 -2 0 2 y 4 -4 -2 0 2 y 4 -4 -2 0 2 y 4
1 1 1
4 4 4
y2
y2
y2
2 2 2
t=1
0 0 0
-2 -2 -2
-4 -4 -4
-4 -2 0 2 y 4 -4 -2 0 2 y 4 -4 -2 0 2 y 4
1 1 1
4 4 4
y2
y2
y2
2 2 2
t = 1.5
0 0 0
-2 -2 -2
-4 -4 -4
-4 -2 0 2 y1 4 -4 -2 0 2 y1 4 -4 -2 0 2 y1 4
Figure 11. Distributions of solutions to the mutual inhibition model with different values of inhibition (w) and
decay (k). Three mutual inhibition models were simulated. In all the models, the signal and noise from the
environment were the same, with I1 ⫽ 3 ⫹ 公2 2.19, I2 ⫽ 3, and c ⫽ 1 (parameters corresponding to the
parameters of the pure drift diffusion model A ⫽ 2.19; c ⫽ 1 estimated from data from middle-difficulty
condition of Participant S1 in the experiment described in Usher & McClelland, 2001). Dashed lines indicate
x1 ⫽ 0 axis, dividing correct and incorrect decisions in the interrogation paradigm, and decision lines are shown
solid. The models shown in the three panels differed in the ratio of inhibition to decay: a: k ⫽ 11.5, w ⫽
8.5 (␭ ⫽ ⫺3); b: k ⫽ 10, w ⫽ 10 (␭ ⫽ 0); and c: k ⫽ 8.5, w ⫽ 11.5 (␭ ⫽ 3). Each model was simulated
1,000 times, and the plots show the states (i.e., the activities of decision units) of the simulated models, with
each dot corresponding to one simulation. The horizontal axes denote the activations of the first (correct)
unit, and the vertical axes denote the activations of the second (incorrect) unit. The states of the models are
shown at three moments in time: top sections of panels for t ⫽ 0.5 s, middle sections of panels for t ⫽ 1 s, and
bottom sections of panels for t ⫽ 1.5 s. Many states have left the region plotted in the bottom-right (␭ ⬎ 0)
panel.
second unit I2 ⫽ 3 [s⫺1], magnitude of noise c ⫽ 1 [s⫺1]; values standard deviation in the x1 direction; however, the mean has now
corresponding to middle-difficulty condition of Participant S1 in moved far to the right so that a great majority of states are on the
the experiment described by Usher & McClelland, 2001). Fig- correct side of the line dividing decisions (x1 ⫽ 0; marked as
ure 11b illustrates the case for the balanced model (when decay dashed line). A critical property of the pure DDM is that the mean
equals inhibition). The top section of Panel b shows states of the of the probability density of its solutions eventually moves faster
model from independent trials after 0.5 s: They have already than the density spreads so that the probability of being on the
reached the proximity of the decision line but are still located close correct side of the line dividing decisions (x1 ⫽ 0) converges to 1
to their starting midpoint in the x1 direction. The middle section with time. Indeed, it follows directly from Equation 7 that as T
shows model states after 1.0 s. States have begun to diffuse along goes to infinity, ER approaches zero.
the decision line, and the mean of the density has moved toward Figure 11a shows the evolution of the states of the system when
the correct decision (i.e., rightward). The bottom section shows the decay is larger than inhibition (both parameters are modified by
state of the system after 1.5 s: The density of points has a greater 15% from their values in Figure 11b). The evolution along the
decision line is now described not by the diffusion process but by some fraction of solutions across the threshold for the incorrect
a stable O-U process (i.e., ␭ ⬍ 0; also see Figure 6a). The middle decision. This is shown in Appendix A (see Equation A58 and
and bottom sections of Panel a show that the distribution con- recall the definition of ã in Equation A28). It is interesting that this
verges to a steady state, with little difference between the states at limit of ER as threshold goes to infinity in the free-response
times 1 s and 1.5 s. The mean is influenced by the drift (note that paradigm precisely coincides with the limit of ER as DT goes to
the density is slightly shifted to the right, i.e., in the direction of the infinity in the interrogation paradigm when ␭ ⱖ 0 but that it does
drift), but it no longer moves: It has converged to the attracting not coincide in the case ␭ ⬍ 0.11
fixed point described above (see Figure 6a). Interrogation at future Figure 9b summarizes these results by showing how the ER of
times will result in essentially the same proportion of solutions the O-U process depends on the value of the threshold. Parameters
from the stationary distribution on each side of x1 ⫽ 0 and for the simulation in Figure 9b match those for evolution along the
therefore with an irreducibly finite number on the wrong side of decision line in Figure 11. For ␭ ⫽ 0, the ER converges to zero for
the x1 ⫽ 0 line contributing errors. large thresholds. For ␭ ⬎ 0, the ER converges to a finite value after
Figure 11c shows the evolution of model states for decay which increasing the threshold has negligible effect. For ␭ ⬍ 0,
slightly smaller than inhibition. Evolution along the decision line although the number of trials with errors decreases to zero for
is now described by an unstable O-U process (i.e., ␭ ⬎ 0; also see larger thresholds, on a high proportion of trials a decision is not
Figure 6c). The points that, because of noise, initially move to one reached within a fixed allotted time (shown by thin line) because
or the other side of the fixed point tend to remain on that side of expected DTs diverge to infinity. Figure 9b is in agreement with
the fixed point and be pushed rapidly away from it. Hence, no the finding of Busemeyer and Townsend (1993) that lower (more
matter how long the interrogation time is, there will again be a negative) values of ␭ lead to lower values of ER in the free-
finite fraction of trials on which an error is made. Remarkably, response paradigm (see left panel of Figure 9 in Busemeyer &
Equation 39 shows that this fraction depends only on the magni- Townsend, 1993).
tude of ␭ and not on its sign.
Figure 11 also illustrates the causes of the recency and primacy Lower Bounds on ER Due to Variability of Drift
effects in the O-U model discussed above, in the section entitled
Ornstein–Uhlenbeck (O-U) Model. When ␭ ⬍ 0, as in Figure 11a, If the drift in the DDM is selected at the beginning of each trial
the solutions fluctuate around the fixed point, and the effect of from a normal distribution with mean mA and standard deviation
inputs decays over time, so that the most recent inputs have the sA, then on a certain fraction of trials, the drift will be negative
biggest impact on the current state of the model. By contrast, when (Ratcliff, 1978). Hence, even as time goes to infinity in the
␭ ⬎ 0, as in Figure 11c, the fluctuations in input at the beginning interrogation paradigm or threshold is taken to infinity in the
of the trial strongly influence the response in the interrogation free-response paradigm, there is a nonzero probability of error
paradigm because, if a state of the model departs from the fixed equal to exactly the fraction of trials with negative drift, which is
point sufficiently, the repulsion from the fixed point is so strong
that it is likely to remain on the same side of the fixed point.
ER ⫽ ⌽ ⫺ 冉冊 mA
sA
. (40)
Free-Response Paradigm Intuitively, this follows from the fact that, for long times, drift
In contrast to the above results for the interrogation paradigm, always dominates the effects of noise or initial conditions in the
for free-response, the minimum achievable ERs depend not only DDM. In fact, as long as mA ⬎ 0, this value of ER is actually a
on the extent to which a model is unbalanced (i.e., on the absolute lower bound on ERs for any value of interrogation time or decision
value of ␭) but also on whether decay or inhibition dominates (i.e., threshold in both the extended DDM and the extended O-U model,
on the sign of ␭). As we show in Appendix A (Equation A59), if as shown in Appendix A (section entitled Lower Bounds on ER
␭ ⱕ 0, then, as the thresholds tend to infinity, ER tends to zero. Due to Variable Drift). For the O-U model, whether or not this
However, for ␭ ⬍ 0, as the thresholds are increased to achieve low bound is actually obtained depends on the task paradigm (interro-
ERs, DTs become excessively long (Ratcliff & Smith, 2004): The gation versus free response) and the value of ␭.
greater distance of the threshold from origin is now compounded The main results of Section 4 are summarized in Table 1.
by the fact that when ␭ ⬍ 0, there is an interior attracting point
(i.e., one that is closer to origin than the threshold; see Figure 11a). A Theory of Optimal Thresholds
This produces solutions that are confined, with exponentially de-
caying probability, near that attractor. (Recall from the section In any speeded decision task, one has to choose whether to be
entitled Fastest Decisions for Fixed Accuracy: The Free-Response faster or more accurate. In the DDM (and all of the related models
Paradigm, above, that, for fixed ER, when ␭ ⬍ 0, DTs are at least considered above), this speed–accuracy trade-off is determined by
expected to be longer than those of the DDM.) Thus, if the the value of the decision threshold. As we have shown, for any
threshold is set above a certain level, many decisions will not be
made within a given finite time. If the threshold is lowered to the 11
This observation could be used to adjudicate models that assume ␭ ⱖ
level at which these decisions can be made, there will be always a 0 (such as the DDM) versus ␭ ⬍ 0 by comparing performance in the
nonzero fraction of errors. interrogation paradigm at long interrogation times to performance in the
Finally, for the case ␭ ⬎ 0 (see Figure 11c), minimum achiev- free-response paradigm when strong emphasis has been placed on accu-
able ERs as thresholds tend to infinity are finite and given by racy. ␭ ⱖ 0 predicts comparable ERs, whereas ␭ ⬍ 0 predicts that they will
Equation 39. This is because the unstable process always forces differ.
given threshold (i.e., for any particular trade-off between speed Equation 41. These values can be inferred from fits to experimen-
and accuracy), the DDM is the optimal decision-making process. tal data (e.g., Mozer et al., 2002) or explicitly manipulated (as in
However, this begs the question, How should the threshold be set the case of Busemeyer & Rapoport, 1988). In contrast, assuming
to achieve optimal performance? The answer to this question that decision makers optimize RR does not require any further
depends, of course, on the criterion that is used to evaluate per- specification; that is, it is a parameter-free criterion.
formance, which in turn depends on the goals of the decision
maker. There have been a number of efforts to explain human Reward Rate
decision-making performance in TAFC tasks in terms of the op-
timization of specific criteria (e.g., Edwards, 1965; Gold & This can be defined as the proportion of correct trials divided by
Shadlen, 2002; Maddox & Bohil, 1998). Different criteria are the average duration between decisions, namely (Gold & Shadlen,
optimized by different thresholds, which in turn have different 2002),
implications for performance. In this section, we consider the
various criteria that have been proposed, the derivation of their 1 ⫺ ER
RR ⫽ (42a)
optimal thresholds, and their relationship to task parameters and RT ⫹ D ⫹ ER 䡠 Dp
performance.
We begin by reviewing the various criteria that have been 1 ⫺ ER
⫽ . (42b)
proposed in the literature, including Bayes risk (BR) and RR. In DT ⫹ T0 ⫹ D ⫹ ER 䡠 Dp
the second section, we find the thresholds that optimize these
criteria for the pure DDM, show analytically that there is a unique In Equation 42a, RT denotes total response time, D the interval
optimal threshold for each of these criteria, and examine how this between a correct response and the next stimulus (the response–
depends on task parameters (drift, noise, and the pace of the task). stimulus interval), and Dp an additional penalty delay introduced
In the third section, we show that, for each criterion, the DDM following an error (which may, of course, be set to zero).13
using the threshold optimal for that criterion outperforms all other Following the conventions used in previous sections, Equation 42b
decision processes. In the fourth section, we derive an equation decomposes RT into its two components: mean DT and T0, the
that specifies the DT for a given ER (and vice versa) under optimal time required for all non-decision-related (e.g., sensory and motor)
performance, thus defining a novel optimal performance curve. In processing. This formulation assumes that after each correct deci-
the fifth section, we apply our analysis to the extended DDM. sion, participants receive a reward and that after every response,
Finally, in the sixth section, we address the question of how the they have to wait a certain interval before the onset of the next
optimal threshold is discovered by the decision maker and the stimulus (and hence the possibility of making the next choice).
implications that this may have for performance. The BR and RR criteria differ in a number of important respects.
First, long DTs are penalized differently: BR assumes that there is
an explicit cost (e.g., financial) for each additional increment of
Criteria for Optimality stimulus observation time, whereas for RR, long DTs simply slow
down the rate at which rewards are received. The direct depen-
Bayes Risk
dence of the RR criterion on time also predicts different behavior
This criterion was first used by Wald and Wolfowitz (1948) in for different decision–stimulus intervals D and Dp (as is shown
proving the optimality of the SPRT. It assumes that decision later), parameters that do not enter into BR.
makers seek to minimize a cost function that is the weighted sum Most importantly, however, BR and RR have different implica-
of DT and ER:12 tions for the emphasis placed on reward versus accuracy. RR, by
definition, seeks to maximize reward. As we show further on,
BR ⫽ c1 DT ⫹ c2 ER. (41) optimizing RR may sometimes demand faster and less accurate
responding. If the experiment is configured exactly as described in
Edwards (1965) generalized this criterion in considering how
this section (i.e., participants are financially penalized for errors
human participants choose decision thresholds. He considered an
and time spent on decision), minimization of BR is equivalent to
experimental paradigm in which (a) observing the stimulus has a
maximization of reward. However, in general, BR allows that
cost (c1) that scales with the duration of the observation (partici-
accuracy may be favored over speed, even if this means receiving
pants lose c1 monetary units for each unit of observation time) and
less overall reward. A number of studies have suggested that
(b) there is a payoff matrix for different types of errors (c2). For a
decision makers do sometimes favor accuracy over maximizing
payoff matrix with no reward for correct responses and a penalty
reward.
c2 for errors, Edwards’s cost function reduces to BR. Edwards
Maddox and Bohil (1998) investigated participants’ choices in a
derived the optimal threshold for the DDM minimizing this cost
version of the interrogation paradigm in which one stimulus is
function (we consider these explicitly in the section entitled Op-
more frequent or more rewarded. They noticed that in this task,
timal Thresholds for Different Criteria in the Pure DDM, below).
Rapoport and Burkheimer (1971) extended Edwards’s theory, and
Busemeyer and Rapoport (1988) designed an experiment to test it, 12
For consistency, note that because DT is in units of time and ER is
finding that the participants chose decision thresholds close to dimensionless, the units of c2 must also be time. This implies the use of a
those minimizing the BR. characteristic time to normalize the weights.
The assumption that decision makers optimize BR requires 13
Such delays are often used in studies of nonhuman animals as pen-
further specification of the relative value that they place on speed alties for incorrect responses, to encourage accurate performance while
versus accuracy—that is, the values of parameters c1 and c2 in avoiding the use of aversive stimuli as punishment.
participants might choose the more frequent or the more rewarded Dtotal in the denominator of the second term. As expected, both RA
alternative even if the accumulated evidence at the interrogation and RRm reduce to RR when the accuracy weight c2 ⫽ 0 (Equation
time favors the other alternative. Let us define the decision bound- 43 with Dp ⫽ 0).
ary as the value of variable x in the DDM above and below which COBRA was introduced to explain the observation that decision
different alternatives are chosen (so far, we have assumed that the makers sometimes seem to favor accuracy over a strict maximi-
decision boundary is at 0). Maddox and Bohil suggested that zation RR. However, like the BR criterion, this requires the esti-
participants choose decision boundaries between the values max- mation of two parameters (c1 and c2). The question remains
imizing reward and those maximizing accuracy even when they whether it is possible to explain decision-making behavior using
are specifically instructed to maximize reward. the parameter-free RR criterion. We return to this question below,
The investigators offered three reasons why participants may after we derive the thresholds for the DDM that optimize the
tend to maximize accuracy: (a) To choose the boundary maximiz- various criteria discussed above.
ing reward, participants need to gain knowledge about the distri-
butions of the stimuli of different classes (i.e., varying reward Optimal Thresholds for Different Criteria in the Pure
value); (b) there is evidence that participants are more concerned DDM
about being accurate than about gaining modest rewards (Pitz &
Reinhold, 1968; Stevenson, Busemeyer, & Naylor, 1991); and (c) We begin by considering Edwards’s (1965) calculations of
the trial-by-trial feedback about accuracy is often equally or more thresholds minimizing BR, then derive the thresholds that optimize
salient than the feedback about the reward (in psychological ex- the three other criteria reviewed above, and examine the depen-
periments, participants are often informed if they have made an dence of optimal thresholds on task parameters. For simplicity,
error, and they may pay more attention to this information than to here we consider decisions produced by the pure DDM; in the
information about the reward). They also observed large individual section entitled Optimal Performance Curves for the Extended
differences in the emphases placed on reward versus accuracy DDM, below, we explore the effects of introducing drift rate and
among participants. On the basis of these findings, Maddox and starting point variability in the extended DDM.
Bohil (1998) proposed a model called competition between reward
and accuracy (COBRA), which postulates that decision makers Properties Common to the Criteria for Optimality
pursue a balance between reward and accuracy maximization.
First, we observe that thresholds optimizing the criteria of
Equations 41, 43, and 44 do not depend on parameters c1 and c2
Reward/Accuracy and Modified Reward Rate separately but only on their ratio (the values of the functions
The COBRA theory was proposed in the context of the inter- themselves, of course, do depend on c1 and c2). Therefore, we
rogation paradigm. Here, we extend it to the free-response para- define a parameter q describing the weighting of accuracy relative
digm. Furthermore, since Maddox and Bohil (1998) did not for- to speed or reward:
mulate an equation for the function being optimized, we consider c2
two different ways of doing so, each of which penalizes errors in q⫽ . (45)
c1
a different manner. The first is a reward/accuracy (RA) function
that is a weighted difference of RR and accuracy: Figure 12 shows thresholds that optimize the four criteria, com-
puted by seeking a minimum of BR and maxima of RR, RA, and
c2 RRm, respectively (as described further below and detailed in
RA ⫽ c1 RR ⫺ ER, (43)
Dtotal Appendix A). Each criterion corresponds to a row of panels, and
each column shows the dependence of the threshold on a particular
where Dtotal ⫽ D ⫹ T0 is the total delay between the end of one
task parameter while the others are kept fixed. Before focusing on
decision and the beginning of the next. We assume that penalty
individual criteria, we emphasize features common to all.
delay Dp (from the expression for RR in Equations 42a and 42b)
The leftmost column of Figure 12 shows that, in contrast to the
is set to zero because errors are penalized explicitly in the second
other parameters, the optimal threshold depends nonmonotonically
term of Equation 43. For c1, c2 ⬎ 0, maximization of this function
on the drift rate A. If the drift rate is equal to zero, the stimulus
emphasizes accuracy by negatively weighting errors (in the second
contains no information. In this case, the optimal strategy is to
term) in addition to positively weighting correct responses (via RR
guess immediately, and so, the optimal threshold is equal to zero.
in the first term). Note that inclusion of Dtotal in the second term
As the drift rate increases and information becomes available, it
allows the units [1/time] to be consistent.
becomes advantageous to integrate, and so, the optimal threshold
The second criterion, a modified RR (RRm), describes the con-
increases. However, as the drift rate becomes sufficiently high, it
dition under which correct responses are rewarded by c1 and errors
begins to dominate the noise, so that threshold can again be
are penalized by c2:
lowered until decisions can be made almost immediately.
c1 共1 ⫺ ER兲 ⫺ c2 ER The second column from the left shows the dependence of the
RR m ⫽ . (44) optimal threshold on noise (c). If c is close to zero, then the drift
DT ⫹ Dtotal
is not obscured by noise (i.e., the signal is strong) and decisions
Note that because errors are dealt with explicitly via the second can be made with little need to integrate information (i.e., average
term in the numerator, we again neglect the penalty delay, setting the signal) over time. Thus, the optimal threshold is close to zero.
Dp ⫽ 0. Also note that Equation 44 is closely related to Equation As noise increases, more integration is required to obtain an
43 but that it now includes the full interstimulus interval DT ⫹ accurate estimate of the signal, and thus, the optimal value of the
Dependence on drift A Dependence on noise c Depen. on delays D total Depen. on accuracy weight q
0.1 0.2 0.2
z minimizing BR Aq
0.15 4 0.15
0
0.05 0.1 0.1
q 0.05 0.05 A
0 4 0 0 4
0 1 2 0 1 2 0 1 2
0.4 1
1
2 ADtotal 0.4
∞
z maximizing RR
0.3 0.3
zA = c 2
0.2 0.5 0.2
0.1 D 0.1 A
total
2 0 2
0 0 0
0 1 2 0 1 2 0 2 4
0.4 1.5 0.4 0.35
z maximizing RA
0.3 0.3 0.3

1
0.2 0.2 0.25
0.5
0.1 0.1 0.2
0 0 0 0.15
0 1 2 0 1 2 0 2 4 0 1 2
0.4 2 0.4 0.3
z maximizing RRm
0.3 1.5 0.3

0.25
0.2 1 0.2
0.2
0.1 0.5 0.1
0 0 0 0.15
0 1 2 0 1 2 0 2 4 0 1 2
A c Dtotal q
Figure 12. The optimal thresholds for the pure drift diffusion model. Each row shows thresholds optimizing a
different criterion (indicated on the left side of the figure). Each column shows the dependence of the optimal
threshold on one task parameter (indicated on the top and bottom of the column), with other task parameters held fixed
at values chosen from the following: A ⫽ 1, c ⫽ 0.33 (corresponding to values estimated from a sample participant;
see Figure 5), Dtotal ⫽ 2 (corresponding to one of the delay conditions in the task performed by the participant), and
q ⫽ 0.5 (chosen arbitrarily). The values of the optimal thresholds were found numerically. Values at the
angles are equal to the rate of increase (i.e., the slope) of the optimal threshold function for the varied
parameter equal to zero and are derived analytically. The arrows indicate the values to which the optimal
threshold converges as the parameter on the horizontal axis goes to infinity. Panels are omitted where
criteria do not depend on those parameters. BR ⫽ Bayes risk; RR ⫽ reward rate; RA ⫽ reward/accuracy;
RRm ⫽ modified reward rate; Depen. ⫽ dependence.
threshold increases. Interestingly, as noise goes to infinity, the for responding requires that each response be more accurate. There
optimal threshold appears to converge to a constant (we prove this is no limit on the optimal value of the threshold.
for BR and RR below, and we conjecture that it is also the case for The rightmost column of Figure 12 shows that for greater
the other criteria on the basis of simulations). emphasis on accuracy (i.e., greater values of q), optimal thresholds
The third column from the left shows the dependence of the
optimal threshold on the total delay Dtotal. As Dtotal decreases
toward zero (i.e., the next stimulus is presented immediately after 14
For BR, RR, and RRm, the optimal threshold converges to zero as
the response, ignoring T0), the optimal value of the threshold also
Dtotal 3 0. However, for RA, the second term in Equation 43 can dominate
decreases toward zero.14 That is, it is optimal to make decisions as Dtotal decreases to zero, leading to a nonzero limiting value for optimal
immediately because there is no opportunity cost for errors. Al- threshold (not visible in Figure 12 as the minimum of zo[Dtotal] occurs for
though responding will be random (because no information has Dtotal ⬃ 0.01 for the parameters used there).
been accumulated) and therefore will produce a reward on only 15
Of course, in practice, this could not occur because Dtotal includes
50% of trials, the arbitrarily high frequency of trials serves to T0—the nondecision component of the response time, which includes the time
maximize reward.15 As the delay Dtotal increases, so too does the for stimulus encoding and motor response and therefore is necessarily nonzero.
optimal value of the threshold: An increase in the opportunity cost Consequently, the optimal threshold will always be nonzero (for A ⬎ 0).
rise, which can be expected because higher thresholds increase e2z̃ã ⫺ 1 ⫽ 2ã共D ⫹ Dp ⫹ T0 ⫺ z̃兲. (51)
accuracy.
Although this (transcendental) equation does not admit an explicit
solution in terms of elementary functions, it may easily be solved
Derivation of Optimal Thresholds for Individual Criteria numerically for z̃ to yield the optimal threshold (normalized by A),
The features above are general to all criteria. Here, we consider as a function ( f ) of ã and task delays as follows:
how optimal thresholds differ among criteria. z̃o ⫽ f共ã, D ⫹ Dp ⫹ T0 兲. (52)
Bayes risk. Edwards (1965) has shown that the BR is mini-
mized by the threshold z satisfying the following condition (Equa- First, we note that the optimal threshold does not depend on D and
tion 17 from Edwards, 1965, rewritten in the present notation; for Dp separately but only on their sum D ⫹ Dp (or actually on D ⫹
completeness, we derive this in Appendix A, Equation A76): Dp ⫹ T0). Hence, for example, the same value of the threshold is
optimal for a long intertrial interval but no penalty delay (e.g., D ⫽
2A2 4Az 2 s, Dp ⫽ 0 s) and for a much more rapid pace of presentation but
⫺ 2 ⫹ e⫺共2Az/c 兲 ⫺ e2Az/c ⫽ 0.
2 2
q (46)
c2 c with a corresponding increase in penalty delay (D ⫽ 0.5 s, Dp ⫽
1.5 s). Therefore, provided that decision makers optimize their
This transcendental equation does not have closed form solutions thresholds to maximize RR, they should have the same ER and
for z but can be solved numerically, as was done for the first row mean RT in both of these conditions.
of Figure 12. Furthermore, the terms can be rearranged so that Let us denote the sum of the three delays influencing the optimal
Equation 46 expresses equality between increasing and decreasing threshold together by Dtotal:
functions of z, thus demonstrating the uniqueness of its solution. It
is also useful to note that for low A, high c, or low q, Equation 46 Dtotal ⫽ D ⫹ Dp ⫹ T0 . (53)
simplifies to
The dependence of the optimal threshold on Dtotal (rather than D
Aq and Dp separately) is not a property specific to the pure DDM; it
z⫽ , (47) is true for any decision-making mechanism (e.g., an O-U model)
4
seeking to optimize RR, as shown in Appendix A (see the section
which gives the slopes and the limit indicated in the first row of Dependence of the Optimal Threshold on Experimental Delays).
Figure 12. We also note that when q ⫽ 0, the accuracy does not Multiplying Equation 52 by A gives the optimal absolute (nonnor-
influence the BR, and hence, the optimal threshold is equal to zero. malized) threshold in terms of the original parameters:
冉冉冊冊
Reward rate. This criterion is defined in Equations 42a and
A 2
42b in terms of ER and DT. The optimal threshold can be derived zo ⫽ Az̃o ⫽ Af , Dtotal . (54)
by reexpressing these in terms of the parameters of the DDM (drift, c
noise, and threshold) given by Equations 8 and 9 and then solving The second row of Figure 12 shows how zo depends on the
for the threshold that maximizes RR. Substitution of these equa- parameters A, c, and Dtotal (see Equations A38 and A39 of Ap-
tions into Equations 42a and 42b followed by some algebraic pendix A). In addition to the general properties of optimal thresh-
manipulation yields the expression olds described above (and shown in Figure 12), we can define the
冉冊
following quantitative characteristics of the optimal threshold for
z z ⫺共2zA/c2兲
1/RR ⫽ ⫹ D ⫹ T0 ⫹ D ⫹ T0 ⫹ Dp ⫺ e . (48) the RR criterion: (a) It increases logithmically as Dtotal goes to
A A infinity, (b) it converges to ADtotal /2 as c goes to infinity, and (c)
Note that this expression depends only on performance variables it reaches a maximum of c2/A (or 1/ã) as drift increases and then
and task parameters—that is, it does not rely on any criterion- decreases to zero as drift is increased further (see Equations
specific parameters. Note also that the parameters of the pure A38 –A39 in Appendix A). These relationships should be empir-
DDM (A, c, and z) do not occur independently and can be grouped ically testable.
into two new parameters. We define these two new parameters of Reward/accuracy and modified reward rate. The expressions
normalized threshold (z̃) and signal-to-noise ratio (ã) as follows: specifying the optimal threshold for these criteria are more com-
plex and are given in the Equations A78 and A84 of Appendix A.
z̃ ⫽
z
A
, and ã ⫽ 冉冊
A
c
2
. (49)
We note here, however, that there is a unique optimal threshold for
both of these criteria. This follows from the fact that they are both
sums of a function with a single maximum in z (the RR term) and
(The BR threshold condition in Equation 46 may also be written a function that is decreasing in z (the additional accuracy terms).
solely in terms of z̃, ã, and q.) Although ã is the squared signal-
to-noise ratio, we refer to it simply as the signal-to-noise ratio in The DDM Optimizes All Criteria Relative to Other
what follows. Note that the parameters z̃ and ã fully determine the Decision-Making Processes
ER and DT of the pure DDM (Equations 8 and 9).16 With these
new parameters, Equation 48 becomes Above, we showed that for a given set of task parameters, there
is a unique value of the threshold for the DDM that optimizes each
1/RR ⫽ z̃ ⫹ D ⫹ T0 ⫹ 共D ⫹ T0 ⫹ Dp ⫺ z̃兲e⫺2z̃ã. (50)
The above equation has a unique minimum (as shown in Appendix 16
Although the new parameters are ratios, they have units: z̃ has units of
A, in the section entitled Optimizing Reward Rates for Unbiased time, and ã has units of 1/time. (Below, we take the unit of time to be
Data) at which the RR is maximized, given by the solution of: seconds.)
of the criteria considered. We can also ask how, at optimal per- ate, emphasis on accuracy q). Each of these expressions describes
formance, the DDM compares with other models? Appendix A a relationship between ER and DT at optimality that we refer to as
(section titled Optimality of the Sequential Probability Ratio Test an optimal performance curve.
[SPRT]) shows that the optimality of the DDM (in the sense of Similar curves can be derived for the extended DDM (as we do
Wald, 1947, and discussed above in the section called The Deci- below in the section called Optimal Performance Curves for the
sion Problem) also implies that the DDM with optimal threshold Extended DDM). To the extent that the curves for different criteria
for a given criterion gives the minimum BR or the maximum RR, and models can be distinguished from one another, they can be
RA, or RRm achievable by any model. In fact, the DDM optimizes used to generate quantitative, empirically testable predictions con-
any criterion involving accuracy and speed that satisfies the broad cerning optimal performance under different decision-making
conditions specified in Appendix A (section titled Optimality of models and criteria.
the Sequential Probability Ratio Test [SPRT]). The derivation of optimal performance curves for the DDM
From these observations, we can further assert that among builds on the relationship between ER and DT for a given Dtotal ,
mutual inhibition models with optimal thresholds, the overall which can be deduced by rearranging Equations 8 and 9 to obtain
maximum RR will be achieved when decay is equal to inhibition the expressions for parameters z̃ and ã as a function of ER and DT:
and both are of a sufficient magnitude (see sections Background
and Relationships Among the Models, above). Figure 10d dem- DT
z̃ ⫽ , (55)
onstrates this point for a particular parameter set. By the same 1 ⫺ 2ER
冉冊
reasoning, the O-U model yields optimal RRs when ␭ ⫽ 0; the
1 ⫺ 2ER 1 ⫺ ER
calculations in Appendix A (section titled Optimizing Reward ã ⫽ log . (56)
2DT ER
Rates for Unbiased Data) demonstrate this explicitly. Further-
more, the exact Equation A57 and its approximate forms that
follow may be used to evaluate the extent to which RRs for the Bayes Risk
O-U model depart from optimality. Edwards (1965) showed that substituting the above equations
into the optimality condition of Equation 46 and rearranging terms
The Relationship Between ER and DT at Optimal yield a direct relation between ER and DT normalized by the
Performance weight of accuracy q under conditions of optimal performance
defined as minimization of BR (Equation 20 in Edwards, 1965):
The analyses described above indicate that for a given criterion
and set of task parameters, there is a single threshold that will 1 ⫺ ER
optimize performance of the pure DDM. This threshold defines a 共1 ⫺ 2ER兲log
DT ER
specific point in the speed–accuracy trade-off, that is, a specific ⫽ (57)
q 1 ⫺ ER 1 1
ER and DT that are optimal for a given criterion and set of task 2 log ⫺ ⫹
ER 1 ⫺ ER ER
conditions. In this section, we derive an expression for each
criterion that describes the optimal ER and DT as a function of task (cf. Equation A77 in Appendix A). The left side of this equation
parameters (signal-to-noise ratio ã, delay Dtotal , and, if appropri- expresses the ratio of DT to the parameter weighting accuracy
Figure 13. The optimal performance curves for the drift diffusion model. Horizontal axes show the error rate,
and vertical axes show the normalized decision time (DT), that is, DT divided by accuracy weight q in Panel a
and by total delay Dtotal in Panels b and c. a: Optimal performance curves for Bayes risk (BR). b, c: Optimal
performance curves for reward/accuracy (RA) and modified reward rate (RRm) criteria. Each curve corresponds
to different values of q ranging from ⫺0.2 (bottom curves) to 0.5 (top curves) in steps of 0.1. The thick line
(identical in Panels b and c) corresponds to q ⫽ 0; that is, the thick line is the unique optimal performance curve
for the reward rate.
(which, in this case, also has the units of time), whereas the right corresponding to maximum DT (i.e., the position of the peak),
side is a function of ER alone. Hence, the equation describes a whereas, for RRm, increasing q moves the peak to the right. These
relationship between ER and normalized DTs that we call an observations provide a framework for constructing and testing the
optimal performance curve (see Figure 13a). This relationship extent to which and under what conditions participants maximize
must be satisfied for any task difficulty level, provided that par- these different criteria for optimality.
ticipants set their thresholds to minimize the BR under the pure
DDM. As noticed by Edwards, when the decisions are very easy (ã Optimal Performance Curves for the Extended DDM
Ⰷ 1), both ER and DT converge to 0 (left side of Figure 13a);
when the decisions are very difficult (ã ⬇ 0), ER converges to 0.5 In this section, we compute the optimal threshold and optimal
and DT to 0 (the optimal strategy is to guess immediately; right performance curve for the extended DDM. For simplicity, we
side of Figure 13a). According to this analysis, the longest DTs consider only one criterion of optimality: RR. Optimal perfor-
should be about 0.136q and associated with an ER of about 13.5% mance curves for the other criteria can be derived similarly, and we
(peak of the curve in Figure 13a). have also computed them for the O-U model with ␭ ⱕ 0 (results
not shown here).
The variability in drift rate and starting point introduced in the
Reward Rate extended DDM have a direct influence on the optimal threshold for
Analogously, by substituting Equations 55 and 56 into the the different criteria. Figure 14a shows that as the variability of
condition for thresholds maximizing the RR (Equation 51), we drift increases, the optimal threshold first slightly increases, but
obtain the optimal performance curve for this criterion: then the information content of the drift becomes lower, and the
optimal threshold decreases. Numerical simulations show that as
冢冣
⫺1
DT 1 1 the variability of drift goes to infinity, the optimal threshold goes
⫽ ⫹ . (58)
Dtotal 1 ⫺ ER 1 ⫺ 2ER to zero because the drift ceases to carry any useful information and
ER log the optimal strategy is to guess. Figure 14b shows that as the
ER
variability of starting point increases, the optimal threshold also
The left side of the above equation expresses the ratio of time in the increases. Because the extended DDM assumes that the threshold
trial used on decision processes to the maximum intertrial interval in general must be higher than the range of starting points sx , the
(total response-to-stimulus delay on error trials), whereas the right optimal threshold is higher than sx . It should be emphasized that in
side is a function only of ER. Hence, the equation describes the Figure 14b, starting point variability reflects noise in the initial
relation between ER and normalized DT (as a fraction of maximum conditions of the integrator and not changing prior probabilities of
intertrial interval). This optimal performance curve is shown as the alternatives (the threshold maximizing RR when the starting point
thick black curve in Figures 13b and 13c. It has a shape similar to that correctly represents stimulus probabilities is discussed below, in
for BR, but the longest DTs are about 20% of the maximum interde- the section entitled Biased Decisions).
cision interval and are associated with an ER of about 18% (peak of Figures 14c and 14d show the optimal performance curves for
the thick curves in Figures 13b and 13c).17 the extended DDM for different values of Dtotal . These curves
were determined as follows: Parameters sA, sx , Dtotal , and mA were
Reward/Accuracy and Modified Reward Rate fixed, and c was varied to obtain the different points on the curves
(i.e., for each value of c, the optimal threshold was found numer-
In Appendix A (Equations A80 –A81), we derive analogous ically, and a point was added to the graph with corresponding
optimal performance curves for RA and RRm, respectively (Equa- values of ER and DT).
tions 59 – 60): Four observations can be made about these curves. First, the
DT E ⫺ 2q ⫺ 冑E2 ⫺ 4q共E ⫹ 1兲 shape of the curve depends on the value of Dtotal (unlike for the
⫽ , where pure DDM). Second, the variability of drift moves the optimal
Dtotal 2q
performance curve down, as would be expected, because it pre-
1 1 dominantly decreases the optimal threshold (see Figure 14a).
E⫽ ⫹ , (59) Third, the variability of starting point moves the optimal curve up,
1 ⫺ ER 1 ⫺ 2ER
ER log as would be expected, because it increases the threshold (see
ER
Figure 14b). Finally, with variability of starting point, the optimal
normalized DT for ER ⫽ 0 is not equal to zero. This is because the
冢冣
⫺1
1 q
⫺ extended DDM assumes that the threshold must be higher than the
DT ER 1 ⫺ ER 1⫺q
⫽ 共1 ⫹ q兲 ⫹ . (60) range of starting points, so it cannot be equal to zero, which is
Dtotal 1 ⫺ ER 1 ⫺ 2ER required for DT ⫽ 0. Figure 14d shows that if Dtotal is very low,
log
ER the optimal performance curve with variability of starting point
The right-hand sides of the above equations contain the additional becomes monotonically decreasing.
parameter q, hence they do not describe a single curve but rather
a family of curves for different values of q. As q approaches zero, 17
We recognize, however, that this relationship may not be valid for
both of these expressions converge to Equation 58 because, in this intertrial intervals exceeding certain values (e.g., intertrial intervals of
case, both criteria simplify to the RR. These families are shown in order of minutes) because, for very large Dtotal , the optimal threshold may
Figures 13b and 13c. Note that the RA and RRm criteria differ in be very large (see Figure 12a) and exceed the maximum biologically
their predictions: For RA, the value of q does not influence the ER allowed firing rate of decision neurons.
a) b)
∞
0.25 0.25
Optimal threshold
Optimal threshold
0.2 0.2
0.15 0.15
0
0.1 0.1
0 0.5 1 0 0.1 0.2
Drift variability Starting point variability
c) d)
s A =0, s x =0 s A =0, s x =0
Mean normalized decision time

Mean normalized decision time
0.25 s A =0, s x =0.14 0.25 s A =0, s x =0.14

s A =0.31, s x =0 s A =0.31, s x =0
0.2 0.2
0.15 0.15
0.1 0.1
0.05 0.05
0 0
0 0.1 0.2 0.3 0.4 0.5 0 0.1 0.2 0.3 0.4 0.5
Error rate Error rate
Figure 14. Optimal thresholds and optimal performance curves for the extended drift diffusion model (DDM).
a, b: Dependence of the reward-rate-maximizing threshold for the extended DDM on drift variability (Panel a)
and starting point variability (Panel b), with other parameters held fixed. The values of the varied parameters are
shown on horizontal axes, and the values of the fixed parameters were equal to sx ⫽ 0.14, mA ⫽ 1, sA ⫽ 0.31,
c ⫽ 0.33 (corresponding to values estimated from a sample participant; see Figure 5), and Dtotal ⫽ 2 (one of the
delay conditions performed by the participant). c, d: Optimal performance curves for the extended DDM,
obtained numerically by varying noise parameter c while other parameters were kept fixed: mA ⫽ 1, sx and sA
as shown in figure legend, Dtotal ⫽ 2 in Panel c, and Dtotal ⫽ 0.5 in Panel d.
Threshold-Setting Algorithms mine (or approximate) it using an adaptive adjustment procedure:

Different values of the threshold are sampled, the effects on a
Above, in the section Optimal Thresholds for Different Criteria criterion are observed, and the threshold is adjusted to optimize the
in the Pure DDM, we derived expressions for the thresholds that criterion.18 Several studies have examined this possibility.
optimize various criteria for a given set of task parameters. This
raises the question, How do decision makers identify this optimal
Review of Threshold Setting
threshold? In this section, we review published work addressing
this question, and we show that any adaptive threshold-setting Myung and Busemeyer (1989) performed an experiment using
model makes two predictions for a task in which participants the free-response paradigm19 in which participants were explicitly
maximize RR: (a) They will choose thresholds closer to optimal required to minimize BR. They tested predictions of two algo-
values for easier versus harder tasks, and (b) they will tend to
choose thresholds at higher than optimal values. 18
One possibility is that the knowledge of optimal threshold is Note that such an adaptive mechanism is likely to be required even if
the relationship of task parameters to optimal threshold is known. This is
intrinsic; that is, the decision maker comes to the task with com-
because unless the task parameters are explicitly specified at the outset,
plete knowledge of the relationship between task parameters (sig-
they must be discovered. This, in turn, requires sampling at some threshold
nal, noise, delay, and their variability) and the optimal threshold. that is most likely not optimal (because the optimal one is not yet known)
Although this may be possible for highly practiced tasks within a and that therefore must be adapted as the task parameters are discovered.
narrow domain of parameters, it seems unlikely more generally 19
In this experiment, participants were presented with successive “re-
and certainly for novel tasks. Here, we consider the more plausible sults of medical tests” until they made a choice of “diagnosis.” Thus, DT
(or at least more general) alternative that the decision maker has no (in the equation for BR) for this experiment corresponded to the number of
intrinsic knowledge of the optimal threshold but is able to deter- medical tests the participant observed before the diagnosis.
rithms for updating the decision threshold: (a) an error correction

model, which assumed that participants increase the threshold after z~o − ε z~o z~o + ε
an error and decrease it after a correct response, and (b) a hill- 0.9
RR (z~o + ε )
climbing model, which assumed that after receiving a reward, 0.85
participants make the same threshold modification as on the pre-
vious trial, whereas after an error, they make a threshold modifi- 0.8
RR (z~o − ε )
cation opposite to that on the previous trial (this algorithm imple- 0.75
Reward rate
ments gradient-based reinforcement learning). Analysis of the
experimental data suggested that both processes of threshold mod- 0.7
ification were in operation. During the course of the experiment,
0.65
participants’ thresholds converged close to the vicinity of the
optimal value, but convergence was slow and required hundreds of 0.6 a~a=14.1
trials. Later work of Busemeyer and Myung (1992) showed that hill a~a=9.1
climbing describes experimental data better than error correction.
0.55 a~a=4.0
Erev (1998) proposed a different reinforcement-learning algo- 0.5
0 0.1 0.2 0.3 0.4 0.5
rithm for adjusting the decision boundary in the interrogation Normalized threshold
paradigm. His algorithm did not involve gradients (i.e., differences
between decision boundaries on sequential trials) but assumed that Figure 15. The reward rate (RR) for the drift diffusion model as a function
(a) there is a finite set of decision boundaries from which partic- of the normalized threshold z̃ (shown on the horizontal axis) for D ⫽ 1 s, Dp
ipants choose, (b) participants store estimates of reward for each ⫽ 0 s, T0 ⫽ 0 s, and the values of ã corresponding to those estimated from
value of the decision boundary and update these estimates after three sample participants of the experiment described in the text section
entitled TAFC Experiment and Fit of DDM (the values are shown in the figure
each trial, and (c) the probability of choosing a certain decision
legend; ã ⫽ 9.1 corresponds to the participant whose performance was
boundary on a given trial is a function of this estimate. This algorithm
illustrated in Figure 5). z̃o indicates the optimal normalized threshold for the
is an extension of a reinforcement-learning algorithm described in case ã ⫽ 14.1, and ε denotes a small deviation. Note that underestimation of
chapter 2 of Sutton and Barto (1998) and was shown to account for the threshold by ε causes greater decrease in the reward rate than overestimation.
a large number of effects observed in the experimental data.
Maddox and Bohil (1998, 2001) and Bohil and Maddox (2003)
observed that participants in their generalized interrogation para-
rogation paradigm and suggests that it should generalize to the
digm task (see the section entitled Criteria for Optimality, above)
free-response paradigm: The easier the task, the closer the decision
chose decision boundaries closer to the value maximizing reward
threshold should be to its optimal value.
for stimuli that were easier to discriminate (d⬘ ⫽ 2.2) than for more
Furthermore and perhaps most interestingly, considering the
difficult stimuli (d⬘ ⫽ 1). They explained this fact by noticing that
effects of adaptive threshold adjustment may explain the frequent
the reward as a function of decision boundary is steeper for easier
observation that participants appear to favor accuracy over (or in
than for difficult stimuli and hence that learning of the criterion
addition to) reward maximization (Maddox & Bohil, 1998, 2001).
maximizing reward is more rapid and robust for easier stimuli.
From the perspective of pure RR maximization, this amounts to
threshold overestimation. However, this may reflect an adaptive
Threshold Adaptation and the DDM bias that is intrinsic to the threshold estimation procedure. Fig-
ure 15 plots RR as a function of threshold. Note that on the right
We now consider threshold adaptation in pure DDM to maxi-
side of the peak (denoting the optimal threshold, z̃o), the curve falls
mize RR. If a hill-climbing procedure (Myung & Busemeyer,
with a shallower slope than on the left side. Thus, overestimation
1989) or Erev’s (1998) method is used, this process is guaranteed
of the threshold by a small value ε gives higher RR than under-
to converge on the threshold maximizing the RR because we have
estimation by the same amount; that is,
shown that there is a single optimal threshold for a given set of task
parameters and that RR drops monotonically as the threshold RR共z̃o ⫺ ε兲 ⬍ RR共z̃o ⫹ ε兲. (61)
moves away from the optimal one (because the optimal threshold
z̃o defined by Equation 51 is the only threshold for which the deriv- Appendix A (see section Threshold Resulting From Learning)
ative of RR with respect to z̃ is zero; see Appendix A, Equation A33). shows that the above inequality is satisfied for all signal-to-noise
Recently, Simen, Cohen, and Holmes (2006) and Simen, Holmes, and ratios and experimental delays. That is, overestimation of the
Cohen (2005) proposed a different threshold-adaptation model that threshold is in general less costly than underestimation. Therefore,
converges very rapidly to the threshold maximizing the RR. Below, with any uncertainty of estimation, there should be a bias toward
we focus on predictions common to algorithms that iteratively con- threshold overestimation.20 Furthermore, if participants use a
verge to optimal thresholds, rather than on their distinctions.
Figure 15 shows the RR as a function of the normalized decision 20
These considerations suggest that we should in fact consider the
threshold (defined above, in the section entitled Optimal Thresh-
distribution of thresholds (rather than the single deterministic value) that
olds for Different Criteria in the Pure DDM, as z/A) for three maximizes RR, with the prediction that the mean of this distribution will be
different levels of signal-to-noise ratio. Note that the RR is a greater than the single deterministic value. The characteristics of this
steeper function of the threshold for higher signal-to-noise ratios. distribution will be determined by factors such as RR sampling error and
This provides an explanation of the observation made by Maddox other sources of noise, which are parameters that control the degree of bias
and Bohil (1998, 2001) regarding RR maximization in the inter- toward threshold overestimation.
a) b)
3
0.21
2
0.205
1
0.2
2
2x0 A/c
DT [s]
0
0.195
0.19 -1 Min ER
Min DT
Laming
0.185 -2 Link
VanZandt
0.18 -3
0 0.02 0.04 0.06 0.08 0.1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Starting point x 0 Π
Figure 16. Optimal starting point for biased decisions. a: Mean decision time ([DT] y-axis) for the biased-
probability drift diffusion model (DDM) at fixed accuracy (error rate [ER] ⫽ 2%) and different starting points
(x-axis). DTs were obtained numerically using Equations A44 and A45 from Appendix A for drift A ⫽ 1, noise
c ⫽ 0.33 (estimates from the participant shown in Figure 5), and probability of one of the alternatives ⌸ ⫽ 0.7.
Dashed line indicates position of the optimal starting point predicted by Equation 62. b: Dependence of the
starting point of the pure DDM (scaled; see vertical axis) on the prior probability ⌸ of the alternative with
positive drift A (horizontal axis). Solid line shows the starting point minimizing DT for fixed ER, obtained from
Equation 62. Dashed line shows the starting point minimizing ER for fixed threshold, obtained from Equation
63. Circles indicate starting points estimated from the experiment of Laming (1968) by Link (1975, Table 2),
squares indicate starting points estimated by Link from his experiment (Link, 1975, Table 6), and stars indicate starting
points estimated from the experiment of Van Zandt, Colonius, and Proctor (2000) by Ratcliff and Smith (2004, Table
8; the value of the drift was taken as the average of mean drifts for the two alternatives: [0.2161 ⫹ 0.3659]/2).
gradient-based learning algorithm to find the threshold, it is easier ities of the two alternatives (Carpenter & Williams, 1995; Ed-
to establish the sign of the gradient of RR for z ⬍ zo , where RR(z) wards, 1965; Link, 1975).
is steeper, than for z ⬎ zo , where RR(z) is shallower. Hence, it is Let us use ⌸ to denote the probability of the alternative correspond-
easier to discover that the threshold should be increased when z ⬍ ing to the upper threshold ⫹z being correct. Edwards (1965) showed
zo than to discover that it should be decreased for z ⬎ zo.21 These that the optimal decision process (SPRT) in this case is the DDM with
factors suggest an account, strictly in terms of RR maximization, the starting point not at x0 ⫽ 0, as for the unbiased case, but at a
of the often-observed bias toward threshold overestimation with- position x0 reflecting the odds of the prior probabilities of alternatives
out having to assume any additional criteria concerning accuracy (cf. Appendix A, section entitled Minimizing ER for Biased Choices):
such as RA and RRm.
c2 ⌸ 2A ⌸
x0 ⫽ log , or x0 ⫽ log . (62)
Biased Decisions 2A 1⫺⌸ c2 1⫺⌸
When one of the alternatives of the decision process is correct Figure 16a confirms that x0 given by Equation 62 is optimal by
more often than the other (e.g., one stimulus occurs more fre- demonstrating that, for a given threshold, it produces the minimum
quently in a TAFC task), then decision makers may use this DT for a fixed ER. Note that according to Equation 62, we recover
information about the prior probability of the alternatives to opti- x0 ⫽ 0 for ⌸ ⫽ 1⁄2 (the unbiased case). More generally, however,
mize their responses. In this section, we consider how this can be the optimal starting point is proportional to the log of the ratio of
addressed formally by the DDM. We begin by reviewing existing prior likelihoods for the two alternatives. Hence, as the probability
theories about performance under conditions of biased alternatives, of a given response increases, the optimal starting point moves
both when signal strength is kept constant and when it is varied toward the threshold corresponding to that response. Furthermore,
from trial to trial, and compare these with existing experimental
data. We then consider how asymmetries in the probability as well 21
We note that standard reinforcement-learning algorithms for learning
as the reward value of each alternative influence the optimal a single variable (chapter 2 of Sutton & Barto, 1998) also predict that
threshold, find the relationship of ER to DT for this threshold, and decision makers will set higher than optimal thresholds. For example, the
relate these theoretical results to existing experimental data. action-value method with soft-max selection (Sutton & Barto, 1998) would
build estimates of the values of RR for different thresholds on the basis of
Biased Probabilities and the Optimal Starting Point the rewards experienced and choose a new threshold on each trial, with the
probability being a monotonic function of the estimated RR for that
In this section, we review three theories of how the starting point threshold. Thus, from Inequality 61, the probability of choosing threshold
of integration in the pure DDM should depend on prior probabil- z̃o ⫺ ε would be lower than probability of choosing z̃o ⫹ ε.
this effect is amplified as the decision becomes more difficult (i.e., analogously P⫹兩⫹, P⫺兩⫺, and P⫹兩⫺. All these probabilities can be
the signal-to-noise ratio decreases). The dependence of the optimal directly estimated from the data (for example, P⫺兩⫹ can be esti-
starting point x0 on the prior probability ⌸ is shown by the solid mated as the number of trials in which participant chose ⫺
line in Figure 16b. The pure DDM with starting point given by while ⫹ was presented divided by the total number of trials in
Equation 62 minimizes ER in the interrogation paradigm (as which ⫹ was presented). Link showed that the value of the starting
shown in the Appendix A section entitled Minimizing ER for point chosen by the participant (scaled by 2A/c2) may be estimated
Biased Choices) and implements the SPRT in the free-response from data using the following equation:
paradigm, hence it optimizes all four reward-based criteria ana-
lyzed in the section A Theory of Optimal Thresholds, above.
Link (1975) considered a different criterion of optimality: min-
x0
2A 1
c2
⫽ log
2 冉
P⫺兩⫺ P⫹兩⫺
P⫺兩⫹ P⫹兩⫹
, 冊 (64)
imum ER for fixed decision threshold in the free-response para-
digm. Although this may seem similar to the criterion of minimum and he computed this quantity for his own experiments and those
DT for fixed ER that is optimized by Equation 62, Link showed of Laming (1968). Note that in deriving Equation 64, Link did not
that the optimal starting point in this case is halfway between zero assume any theory of how participants choose the starting point
and that of Equation 62, namely, but assumed only that responses were generated according to the
pure DDM; hence, the estimates from Equation 64 may be used to
c2 ⌸ 2A 1 ⌸ contrast predictions of Equations 62 and 63.
x0 ⫽ log or x0 ⫽ log . (63) In Figure 16b, we show that the starting points (scaled by 2A/c2)
4A 1⫺⌸ c2 2 1⫺⌸
estimated by Ratcliff and Smith (2004) and Link (1975) are closer
This dependence of starting point x0 on the prior probability ⌸ is to the predictions of Equation 62 than those of Equation 63 for all
shown by the dashed line in Figure 16b. The differential predic- three experiments, and for the data of Laming (1968), this differ-
tions of Equations 62 and 63 are contrasted with experimental data ence is significant (paired t test, p ⫽ 0.04). Such an outcome can
in the next section. be expected because the starting point of Equation 62 optimizes
Carpenter and Williams (1995) proposed a theory for the start- criteria directly based on reward (i.e., RR or other measures
ing point of integration explaining the dependence of DT on prior discussed in the section entitled A Theory of Optimal Thresholds,
probability ⌸. However, their model assumes two separate neural above), whereas the starting point of Equation 63 minimizes ER
integrators, each accumulating evidence for detection of one alter- for fixed threshold and does not optimize the RR. This result
native (in a manner similar to the race model). As we showed in provides support for the hypothesis that decision makers adapt
the section titled Background, above, this type of model cannot be parameters to maximize the experimental reward. Below, we do
reduced to the DDM, which is the optimal process for the discrim- not discuss the criterion optimized by Equation 63 further, and for
ination tasks that are the focus of this article; therefore, we do not simplicity, we refer to the starting point of Equation 62 as optimal.
consider the Carpenter and Williams model further.
Variable Signal Strength and Adjustment of Drift Rate
Experimental Tests of the Optimal Starting Point
In this section, we review the argument that when drift varies
Although the theories of Edwards (1965) and Link (1975) were across trials and one of the alternatives is more probable, optimal
proposed several decades ago, we are not aware of a systematic performance is obtained by biasing both starting point and drift.
comparison of their predictions with experimental data. In this We also verify the predictions of this theory with further experi-
section, we show that predictions of Edwards’s theory match better mental data from the literature.
than those of Link’s theory with results of three experiments Ratcliff et al. (1999) observed that decisions can be biased
performed by Laming (1968), Link, and Van Zandt et al. (2000) toward the more probable alternative within the DDM model in
and further analyzed by Ratcliff and Smith (2004). two ways (see Figure 32 of Ratcliff et al., 1999): (a) by moving the
To perform such a comparison, it is necessary to estimate the starting point toward the threshold for the more probable alterna-
parameters of the pure DDM (e.g., drift, starting point, and thresh- tive (as in Equation 62) or (b) by increasing the drift in the
old) from experimental data for tasks with biased alternatives. direction of the more probable alternative and decreasing it for the
Ratcliff and Smith (2004) estimated parameters for the extended other alternative. As shown by Edwards (1965; see the section
DDM (using methods similar to those described above, in the titled Biased Probabilities and the Optimal Starting Point, above),
section titled TAFC Experiment and Fit of DDM) from the data of the optimal strategy for tasks with constant signal strength is to
Van Zandt et al. (2000) and found that variability of drift sA and adjust the starting point and not the drift rate. However, if signal
starting point sx were an order of magnitude smaller than other strength (e.g., stimulus difficulty) varies among trials within a block,
model parameters (mA, z). Therefore, in what follows, we make the then a correction of the drift rate, in addition to that of the starting
simplifying assumption that this task can be described by the pure point, is required to achieve optimal performance (Yang et al., 2005).
DDM. Link (1975) made this assumption at the outset and used it Analyses of experimental data indicate that these different strat-
to develop the following direct method of parameter estimation egies are indeed used depending on the manipulations of signal
from data. Denote the alternative corresponding to the negative strength across trials. In an experiment by Van Zandt et al. (2000),
threshold in DDM by ⫺ and the other alternative, corresponding to participants made same– different judgments about pairs of letters,
the positive threshold, by ⫹. Denote the probability of choosing and the probability ⌸ of one type of stimulus was varied between
the alternative ⫺ if the stimulus ⫹ is presented by P⫺兩⫹ (i.e., this blocks. Recall from the section Experimental Tests of the Optimal
is the probability of an error if stimulus ⫹ is presented) and Starting Point, above, that for this experiment, the estimated
variability of drift sA across trials was an order of magnitude lower Endogenous Variations in Drift Rate and Starting Point
than mean drift mA. Ratcliff and Smith (2004) noticed that to (Extended DDM)
accurately describe the data of Van Zandt et al., it sufficed to vary
only the starting point x0 to fit conditions of different prior prob- In the preceding sections, we considered how parameters of the
ability ⌸ while mean drift mA remained constant (one value of pure DDM (starting point and drift rate) should be adapted to
parameter mA was used for different ⌸ conditions). By contrast, optimize performance in response to manipulations of exogenous
Ratcliff et al. (1999) reported that for their experiment, in which factors—namely, stimulus probability and strength. Furthermore,
difficulty was varied across trials within the same block, both we observed that there were specific optimal values of these
starting point x0 and mean drift mA had to be allowed to vary parameters for a given set of task conditions—that is, a fixed
significantly to fit performance in conditions of different prior adjustment of starting point for a particular stimulus probability
probability. Furthermore, Ratcliff (1985) estimated parameters of and a fixed adjustment of drift rate for a particular distribution of
pure DDM from data of Proctor and Rao (1983) and Proctor, Rao, stimulus strengths. These observations should be distinguished,
and Hurst (1984) and found that both starting point x0 and drift A however, from the fact that fits of the DDM to empirical perfor-
varied between conditions of different prior probability. Both mance are best when variability of drift and starting point are
experiments of Proctor and Rao and Proctor et al. involved the included in the model (Ratcliff et al., 1999). These concern en-
same task: to discriminate whether two strings of four letters were dogenous variability of parameters from trial to trial and should
the same or different where the strings differed by at most one therefore be distinguished from the fixed adjustments of parame-
letter. Although the experiments did not involve explicit variability ters that are optimal in response to exogenous factors such as
in difficulty across trials, it is reasonable to assume that, for asymmetric stimulus frequency and variable stimulus strength.
example, noticing a disparity between strings differing in the first Indeed, the variability of drift and starting point in the extended
letter is easier than for strings differing in the third letter, making DDM can improve fits to empirical data even when stimuli occur
such variability implicit for this task design. with equal frequency and fixed strength (Ratcliff & Rouder, 1998).
In summary, when asymmetries in the probability of alternatives In this case, such variability is clearly not optimal. In this section,
are introduced, adjustments in the starting point of integration are we discuss how such observations can be explained within the
sufficient to achieve optimal performance, and this appears to be optimality framework.
what participants do. However, when variations in signal strength One source of the variability of drift in the extended DDM may
(such as stimulus difficulty) are also present, then an adjustment in reflect variability of top-down attention. Although this necessarily
drift rate is also required, and again, there is evidence that partic- compromises optimality of task performance, elsewhere we have
ipants respond accordingly. argued that such variability may serve an adaptive function on a
These analyses have interesting implications for the neural broader scale, reflecting a trade-off between the value of optimiz-
network models described earlier and, in particular, for the optimal ing performance in the current task and exploring alternative
initial values of unit activities in the balanced mutual inhibition opportunities for reward (Aston-Jones & Cohen, 2005).
model. First, consider the case of constant signal strength across Similarly, it is possible that the variability of the starting point
trials. According to the first of Equations 18 (describing the reflects the function of other adaptive mechanisms operating on
relationship between the activities of the decision units in the incomplete knowledge. For example, if the participant is not told
mutual inhibition model and the variable x in the pure DDM), the the fixed probability with which each stimulus will occur, this
difference between initial activities of the decision units should be must be inferred from observations about the frequency of their
equal to 公2x0 (where x0 is given by Equation 62). This can be occurrence. Initial estimates will be subject to sampling error and
achieved if the decision unit corresponding to the more probable suggest starting points that are not appropriate for the true under-
alternative receives external input before the decision process. lying probability of the stimuli. As estimates of stimulus frequency
However, this input must cease when the signal appears and the are revised, the starting point will be adjusted. In addition to this
decision process starts (implementing a bias in the prior expecta- source of variability, there is substantial evidence that the decision-
tion but not the processing of information supporting that alterna- making apparatus is sensitive to the recent history of stimuli and
tive). Similar biases were used by Cho et al. (2002; although there responses, irrespective of knowledge about their overall probabil-
they were allowed to persist during processing) in a mutual inhi- ity. For example, several investigators have demonstrated priming
bition model to simulate the influence on RT and ER of changes in effects based on local sequences of stimuli and responses (even
the local probabilities of stimuli in a sequence (experimentally when participants have been told that stimuli occur at random) that
manipulated by controlling the frequency of repetitions and alter- can be explained in terms of simple mechanisms sensitive to local
nations in the sequence of stimuli). repetitions and alternations that bias responding in a manner com-
In the case of varying signal strength between trials, the analyses parable to a shift in the starting point of the DDM (e.g., Cho et al.,
above indicate that drift rate should also be modified. According to 2002). Although such mechanisms are not optimal for processing
Equation 28 (describing the relationship between mean drift in the truly random events, they may reflect adaptations that are well
extended DDM and mean inputs to the units of the extended suited to the processing of more natural stimuli that exhibit local
mutual inhibition model), the unit corresponding to the more sequential structure. The operation of such mechanisms could con-
probable alternative should, in addition to starting with an activity tribute to variability of starting point captured by the extended DDM.
closer to its threshold, also receive external input during the
decision process itself. However, the magnitudes of the input
Biased Probabilities and the Optimal Decision Threshold
before and after onset of the stimulus need not be equal. Below, we In this section, we show that the decision threshold of the pure
focus on the simpler case of constant signal strength across trials. DDM maximizing RR is little influenced by the probability bias ⌸,
but above a certain ⌸, the optimal starting point exceeds the 2⌸ ⫺ 1 ⌸

optimal threshold, implying that the best strategy is to always ⫹ 2⌸ log
1⫺⌸ 1⫺⌸
choose the more probable alternative. ãc ⫽ . (66)
2Dtotal
Edwards (1965) showed that there is a unique value of the
decision threshold for the pure DDM that minimizes BR and Note that, as for the optimal threshold, this expression depends
derived conditions that it must satisfy for different prior probabil- only on the total delay Dtotal and not individually on D or Dp.
ities of alternatives. Similar to the case of BR, there is a unique Similarly, the critical intertrial delay at which the starting point
value of the decision threshold maximizing the RR that satisfies reaches the threshold is given by
the following equation:
2⌸ ⫺ 1 ⌸
⌸ ⫹ 2⌸ log
1 ⫺ ⌸ 1 ⫺ ⌸
e2z̃ã ⫺ 1 ⫽ 2ã共Dtotal ⫺ z̃兲 ⫹ 共1 ⫺ 2⌸兲log . (65) c
Dtotal ⫽ . (67)
1⫺⌸ 2ã
This expression is derived in Appendix A (Equation A50). Note In other words, for any signal-to-noise ratio that is below ãc or
that when ⌸ ⫽ 1⁄2 , the last term of Equation 65 vanishes, and it intertrial delay shorter than Dctotal , integration does not improve
simplifies to the condition for the optimal threshold in the unbiased RR, and information about the biased probabilities alone is suffi-
case of Equation 51. As the probability of the more frequent cient for optimal decisions. Figure 18c plots a surface of critical
alternative ⌸ deviates from 1⁄2 , the optimal threshold decreases delays Dctotal as a function of ã (⫽ A2/c2) and ⌸. For delays shorter
slightly, as shown for sample parameters in Figure 17a. than Dctotal , optimal decision makers should immediately choose
The modest effect of ⌸ on the optimal threshold contrasts with the most frequent alternative without integrating. Note that Dctotal
its more profound effect on optimal starting point: As ⌸ grows, the decreases to 0 as ⌸ tends to 0.5 for all ã because, as we have
optimal starting point x0 increases without bound toward the shown, for unbiased alternatives (and all nonzero values of Dtotal ,
threshold (as discussed in the previous section and illustrated in A, and c), the optimal strategy requires some integration. Also note
Figure 17a). Above a certain probability bias ⌸, the optimal that Dctotal decreases as ã ([A/c]2) increases, reflecting the greater
starting point exceeds the optimal threshold. In this case, the value of integration for more informative stimuli. Figure 17b
strategy maximizing RR is to forgo integration altogether and shows ER as a function of ⌸, when x0 and z are set to their optimal
always choose the more probable alternative immediately upon values, and signal-to-noise ratio ã is fixed. As ⌸ increases, ER also
stimulus onset (i.e., DT ⫽ 0). That is, the task transitions from a increases (because thresholds come in; cf. Figure 17a) as long as
discrimination to a signal-detection task. The point at which this decision making still relies on integration. When x0 reaches the
occurs is determined by setting x0/A ⫽ zo/A ⫽ z̃o in Equation 65 optimal threshold (e.g., as ⌸ is increased), ER exactly matches that
and depends on ã and Dtotal . achieved by always selecting the more probable alternative,
The critical signal-to-noise ratio at which the optimal starting namely, limx03z̃o ER ⫽ 1 ⫺ ⌸. Then, after the switch to prior-
point reaches the optimal threshold is given by based decision making (i.e., signal detection), as ⌸ further in-
a) b) c)
0.2 0.07 0.2
x0 Π =0.5
zo Π =0.6
0.06 Π =0.7
Normalized Decision Time
0.15 Π =0.8
0.05 Π =0.9
x0 / zo
ER
0.1 0.04 0.1
0.03
0.05
0.02
0 0.01 0
0.5 0.6 0.7 0.8 0.9 1 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5
Π Π Error Rate
Figure 17. Maximization of reward rate for biased decisions. a: The optimal threshold zo (solid line) and the optimal
starting point x0 (dotted line) as functions of prior probability ⌸ (x-axis) for fixed parameters A ⫽ 1, c ⫽ 0.33 (values
corresponding to the data of Figure 5), and Dtotal ⫽ 1. There are no data points for ⌸ ⬎ 0.93 because for these
probability biases, the optimal strategy does not involve integration of input. b: Error rate ([ER] y-axis) as a function
of probability bias (x-axis) for fixed parameters A ⫽ 1, c ⫽ 0.33, and Dtotal ⫽ 1, and for optimal values of z and x0
shown in Panel a. c: The optimal performance curves generalized for biased decisions. The five curves correspond to
different prior probabilities ⌸. The horizontal axis shows ER; the vertical axis shows decision time normalized by
Dtotal . The right endpoint of each curve corresponds to immediate responding with ER ⫽ 1 ⫺ ⌸.
a) b)
0.08 0.08
Probability
0.06 0.06 Reward proportion
0.04 0.04
0.02 0.02
0 0
x0
x0
-0.02 -0.02
-0.04 -0.04
Probability
-0.06 Reward proportion -0.06
-0.08 -0.08
0 2 4 D 6 8 10 0.2 0.3 0.4 0.5 0.6 0.7 0.8
total Probability / Reward proportion
c) d)
5 5
4 4
Dctotal [s]
3 3
Dctotal [s]
2 2
1 1
0 0
5 5
4 0.9 4
0.8 0.8
3 0.7 3 0.7
0.6 (A/c)2 0.6
(A/c)2 2 0.5 Bias Probability Π 2 0.5 Biased reward r
Figure 18. Comparison of optimal decision strategies for biases in the probability (⌸) and reward proportion
(r) allocated to the two alternatives. a: The optimal starting point x0 as a function of Dtotal for fixed A ⫽ 1, c ⫽
0.33 (values corresponding to the data of Figure 5), and with ⌸ ⫽ 0.7, r ⫽ 0.5, for the solid line and r ⫽ 0.7,
⌸ ⫽ 0.5, for the dotted line. b: The optimal starting point as function of ⌸ (solid line) and r (dotted line) for
fixed A ⫽ 1, c ⫽ 0.33, and Dtotal ⫽ 2. c: Surface of critical total delays (Dtotal
c
) as a function of signal-to-noise
ratio ã and biased prior probability ⌸ for one of the two alternatives. d: Surface of critical total delays (Dtotal
c
)
as function of ã and biased reward proportion r allocated to one of the two alternatives.
creases, ER decreases. That is, the ER at the transition point is, DT

nonintuitively, the maximum ER achieved for any value of ⌸ z̃ ⫽ , (68)
g共⌸兲
(assuming optimal performance). These relationships make inter- 1 ⫺ 2ER ⫹
1 ⫺ ER
esting predictions regarding empirical performance, for example, log
ER
that ER should be jointly determined by the probability bias (⌸)
and Dtotal until it reaches the immediate response limit 1 ⫺ ⌸ where
(Equation 62). A study to test these predictions has been performed,
⌸
and data are currently being analyzed (P. A. Simen, personal g共⌸兲 ⫽ 共1 ⫺ 2⌸兲log , (69)
communication, August 23, 2006). 1⫺⌸
and
Relationship of ER to DT for Biased Stimuli g共⌸兲
1 ⫺ 2ER ⫹
1 ⫺ ER
Analogous to the unbiased case, we can also compute optimal log
ER 1 ⫺ ER
performance curves for RR maximization in the biased case by ã ⫽ log . (70)
2DT ER
reexpressing z̃ and ã as functions of ER, DT, and ⌸ (assuming that
participants set their starting points optimally according to Equa- Substituting the above equations into the optimal threshold con-
tion 62). Doing so (using Equations A46 and A47 of Appendix A), dition of Equation 65 and rearranging terms, we obtain a general-
we obtain ized optimal performance curve for the biased case:
1 ⫺ ER without integrating. This surface, found via a numerical search, is

共1 ⫺ 2ER兲log ⫹ g共⌸兲 shown in Figure 18d. Although we are unable to derive an explicit
DT ER
⫽ . (71) expression analogous to Equation 67 for Dctotal in this biased
Dtotal 1 ⫺ 2ER 1 ⫺ ER
⫹ log ⫺ g共⌸兲 reward case, we have verified numerically that Dctotal has the same
ER ER
dependence on signal-to-noise ratio ã for fixed reward bias r ⫽ 1⁄2
Figure 17c shows the optimal performance curves given by Equa- as it does for probability bias ⌸ ⫽ 1⁄2 . That is,
tion 71 for sample values of ⌸.
1
c
Dtotal ⬃ (74)
ã
Optimal Decisions for Biased Rewards
In this section, we describe the optimal starting point and critical in both cases. Comparison of Figures 18c and 18d reveals that the
delays for the case in which both alternatives are equally likely but critical delay Dctotal for a given signal-to-noise ratio and reward
one alternative is associated with a greater reward value than the bias r is longer than the critical delay for the same signal-to-noise
other. ratio and probability bias ⌸ equal to r (the surface in Figure 18d
For TAFC, this case can be described by allocating a fraction r lies above the surface in Figure 18c). We note that the starting
of reward for each correct response to one alternative and a point minimizing BR with biased rewards has been analyzed by
fraction (1 ⫺ r) to the other, where r lies between 1⁄2 and 1. For Edwards (1965).
example, when r ⫽ 1⁄2 , the two alternatives are equally rewarded
(i.e., unbiased); when r ⫽ 3⁄4 , the reward for the correct response Relationship to Neurophysiological Experiments and
to the first alternative is three times higher than that to the second New Predictions
alternative; and when r ⫽ 1, only the correct response to the first
alternative is rewarded. In this section, we compare the predictions of our theory with
This situation resembles that for biased probabilities, but simu- experiments by Platt and Glimcher (1999), who recorded neural
lations indicate that there is an important difference regarding the activity during biased decisions, and we make new predictions for
optimal starting point that yields maximal RRs. For biased re- modified versions of this experiment.
wards, the optimal starting point depends also on task delay Dtotal , The results of the section titled Biased Probabilities and the
unlike the case of biased probabilities (see the section entitled Optimal Starting Point, above, suggest that when the two alterna-
Biased Probabilities and the Optimal Starting Point, above) in tives in a TAFC task occur with different probabilities and diffi-
which starting points depend only on c, A, and ⌸. On the basis of culty is constant across trials, the activities of decision neurons
numerical results, we find that the optimal starting point of the associated with each alternative should be modulated according to
pure DDM for biased rewards appears to obey the following Equation 62 prior to stimulus presentation. That is, the prestimulus
relationships: activity of neurons representing a particular alternative should
depend on the probability of that alternative and its difficulty, as
c2 r described by Equation 62 and illustrated in Figure 16b. In contrast,
x0 3 log , for small values of Dtotal , (72)
2A 1⫺r activity during the decision-making period itself should follow the
usual DDM, unaffected by the relative probability of the alterna-
and
tives.
c2 r Platt and Glimcher (1999) trained monkeys to perform a TAFC
x0 3 log , for large values of Dtotal . (73) involving visual stimuli in which they indicated responses by
4A 1⫺r
making a saccade to one of two locations. The prior probability ⌸
Comparing the above equations with Equations 62 and 63, it is favoring one of the two alternatives varied across blocks of trials
apparent that for small Dtotal , the optimal starting point for biased from 20% to 80%. Platt and Glimcher observed that just before
rewards depends on r, exactly as the optimal starting point for stimulus presentation, the activity of neurons corresponding to a
biased probabilities depends on ⌸. For large Dtotal , however, the given alternative grew linearly with the probability of the alterna-
optimal starting point for biased rewards is halfway between zero tive being correct in that block of trials (see Figure 2b in Platt &
and the optimal starting point for biased probabilities (with equiv- Glimcher, 1999). Note that in Figure 16b, the dependence of the
alent values of ⌸ and r). Interestingly, this is the same dependence optimal starting point on outcome probability is close to linear for
as in Equation 63 (describing the starting point minimizing ER for probabilities ranging from 20% to 80% (see also Figure 18b).
fixed DT). In general, our simulations indicate that the optimal Thus, Equation 62 provides a reasonable description of the empir-
starting point for biased rewards moves monotonically as Dtotal ical observations under conditions of biased probability.
increases from the value given in Equation 72 to that of Equation Platt and Glimcher (1999) also varied the proportion of rewards
73. Figure 18a illustrates how optimal starting points depend on allocated to one of the alternatives from 20% to 80% on separate
Dtotal with fixed A, c, ⌸, and r for biased rewards versus biased blocks of trials (with the same intertrial intervals and task diffi-
probabilities; Figure 18b illustrates how the starting point varies culty). They observed that the activity of neurons corresponding to
with ⌸ versus r for fixed ã and Dtotal . a given alternative, just before stimulus presentation, also grew
As for biased probabilities, we may also derive a surface of linearly with the proportion of reward to the corresponding alter-
critical delays Dctotal for biased rewards that is analogous to that of native (see Figure 1b in Platt & Glimcher, 1999). Note that in
Figure 18c; for delays shorter than Dctotal , optimal decision makers Figure 18b, the dependence of the optimal starting point on r is
should immediately choose the most highly rewarded alternative also close to linear for probabilities ranging from 20% to 80%,
suggesting that these results can also be explained by Equations 72 with respect to all of these criteria and that, for a given set of
and 73. parameters (signal, noise, intertrial delay, and, where applicable,
Moreover, Platt and Glimcher (1999) reported that the average the emphasis on accuracy), there is a single optimal threshold. We
slope of prestimulus firing rate as a function of probability bias ⌸ examined how this threshold varies as a function of task parame-
was slightly larger than the slope of prestimulus firing rate as a ters, used this to derive optimal performance curves relating nor-
function of reward bias r (11 ⫾ 4.5 [SE] vs. 10.5 ⫾ 4.5 [SE]). malized DT to ER for all the criteria considered, and generalized
Although the difference is not significant, it is at least qualitatively these to the extended DDM, which allows for variability of starting
consistent with the pattern described in the section Biased Prob- points and drift rates.
abilities and the Optimal Decision Threshold, above. In particular, Finally, we extended our analyses to the case of biased proba-
note that in Figure 18b, the slope of the curve relating optimal x0 bilities and rewards. We showed that predicted effects conform to
to ⌸ is slightly higher than that relating x0 to r (the difference previously reported observations concerning behavioral perfor-
depends on Dtotal). mance (Link, 1975; Ratcliff & Smith, 2004), as well as the activity
Taken together, the results of Platt and Glimcher’s (1999) ma- of neural accumulators under such conditions (Platt & Glimcher,
nipulation of probability and reward suggest that the activities of 1999). We also computed optimal performance curves relating
neurons in area LIP just before response conform to predictions speed and accuracy for the biased probability case.
made by the DDM for RR optimization. To our knowledge, The main predictions of our theory consider the paradigm in
however, no experiments have yet been performed in which prob- which participants are required to maximize RR. They are as
ability and difficulty are simultaneously varied across blocks but follows: (a) The decision threshold should depend on task param-
kept constant within blocks. The DDM predicts that in such an eters in a specific way summarized in Figure 12. (b) Decision
experiment, the modulation of prestimulus activity of neurons threshold and behavioral statistics should depend on the sum Dtotal
involved in evidence accumulation (e.g., LIP neurons) should be of all task-related delays, as opposed to, for example, being dif-
higher for difficult decisions than for easier (see the section Biased ferentially influenced by error-related penalty delays. (c) For a
Probabilities and the Optimal Starting Point, above). Similarly, given ER, the DT achieved by participants should be proportional
we are not aware of any experiments that systematically manipu- to Dtotal , and their ratio DT/Dtotal should be equal to the value
late intertrial delay. As described above, the DDM predicts that given by the optimal performance curve (see Figures 13b and 13c,
prestimulus neural activity should be influenced by Dtotal for heavy line; Figure 17c). (d) In case of biased decisions, when the
biased rewards, but not for biased probabilities. delay is below the critical value Dctotal , participants should stop
integrating and always immediately choose the most probable or
General Discussion most rewarded alternative. Furthermore, for a given probability
and/or reward bias, the specific delay at which this occurs should
In this section, we summarize the theory developed in this vary with signal-to-noise ratio ã (see Figures 18c and 18d).
article and its main predictions, discuss further extensions of the
theory, and consider the significance of our findings for more Extensions
general issues concerning the mechanisms underlying decision
making and cognitive control. The theory of optimal performance in TAFC tasks developed in
this article has several natural extensions that may contribute to a
Summary broader understanding of the mechanisms underlying human de-
cision making.
This article began with a review of existing models of TAFC
performance and demonstrated that under reasonable assumptions Multiple-Choice Decisions
and for particular parameter values, all of these models (except the
race model) reduce to the simple DDM (Laming, 1968; Ratcliff, Perhaps the most important limitation of the current theory is
1978; Stone, 1960). In the neural network models, inhibition plays that it pertains to decisions involving only two alternatives. Even
a particularly important role, as this implements differencing of the simple decision processes often involve many more alternatives
competing sources of information that is central to DDM. We (e.g., retrieving an item from memory, searching a visual display
showed that suitable decay and inhibition values are consistent for a target item, etc.). Numerous procedures have been devised for
with data estimated from a typical participant in a TAFC task. A deciding among multiple alternatives (Eisenberg, 1991; Wetherill
long tradition of work has shown that the DDM is the optimal & Glazebrook, 1986), most involving combinations of SPRTs
decision strategy, insofar as it is the fastest to reach a fixed level between the various alternatives. Following the seminal work of
of accuracy or, alternatively, the most accurate for a given speed of Lorden (1977), Dragalin, Tartakovsky, and Veeravalli (1999,
response. The specific trade-off between speed and accuracy in the 2000) have shown that, in the limiting case of arbitrarily low ER,
DDM is determined by the choice of threshold. optimality comparable to that of the SPRT (i.e., the fastest re-
We reviewed two existing theories of how the threshold of the sponse for fixed accuracy) is achieved by two procedures known
DDM should be chosen to optimize performance (Edwards, 1965; as the multisequential probability ratio tests (MSPRTs). The first
Gold & Shadlen, 2002) using different criteria for optimality MSPRT procedure calculates for each alternative the ratio of its
(accuracy and RR). We also provided a formal interpretation of a likelihood and the sum of likelihoods of all alternatives consid-
theory that proposes that human participants use a weighted bal- ered; the decision is made whenever any of these ratios exceed a
ance of both criteria (Maddox & Bohil, 1998). We demonstrated certain threshold. Recently, Bogacz and Gurney (in press) demon-
that the DDM is the optimal TAFC decision-making mechanism strated that many aspects of the anatomy and physiology of a
circuit involving the cortex and the basal ganglia are precisely been closely associated with control. Within the framework of the
those required to implement this version of the MSPRT. The other DDM, we can think of control as the adjustment of these param-
MSPRT, often referred to as the max-versus-next procedure, com- eters to optimize performance. This formulation calls into focus
pares the alternative receiving the most evidence with the next- important questions: To what end are these parameters adjusted?
most supported alternative and terminates when the difference That is, what are the objectives of performance, and how does one
between these exceeds a fixed value (corresponding to the thresh- know when these are being met? The framework provided by the
old in the SPRT). The max-versus-next procedure has been im- DDM allows us to address these questions in a formally explicit
plemented in the word-identification model by Ratcliff and Mc- manner by defining an objective, or utility function, that the
Koon (1997) and provided a good fit to experimental data. individual seeks to maximize. We can then more precisely define
In contrast to the MSPRT, Usher and McClelland (2001) pro- control as the mechanisms that monitor performance by evaluating
posed that a multiple decision process can be modeled by a direct this utility function and adjusting parameters of the decision pro-
extension of the mutual inhibition model, in which each alternative cess to maximize its value. Although performance may never
inhibits and receives inhibition from all others (equivalent to a actually achieve the theoretical optimum, the ability to define it
multiple-choice extension of the pooled inhibition model). This provides valuable structure for constructing theories about the
implements a max-versus-average procedure, in which the evi- mechanisms involved in control and for generating detailed quan-
dence favoring the most supported alternative is compared with the titative predictions that can be tested in empirical studies.
average of the evidence in support of all other alternatives. Usher In particular, this framework allows us to formally identify and
and McClelland showed that this performed best among several characterize the factors involved in control—that is, the factors
alternative models. Recent analytical and simulation work by that influence an organism’s ability to optimally meet its goals.
McMillen and Holmes (2006) suggests that under certain condi- These factors fall roughly into three broad categories: (a) param-
tions (ER ⬃ 10%) the max-versus-average procedure closely ap- eters of the decision process that can be adjusted to optimize
proximates the performance of the MSPRT, although as ER ap- performance, (b) criteria used to guide such adaptations (i.e., the
proaches zero, it is significantly worse. Moreover, multiunit cost functions that the organism uses to determine how effectively
mutual inhibition models approximate such a procedure. Addi- it is meeting its goals), and (c) algorithms used to carry out these
tional work is needed to fully characterize the dynamics of this adaptations. This framework suggests specific directions for ex-
procedure. As this work proceeds, it will be interesting to consider tending the DDM to address factors in each of these categories.
how this compares with models that have been developed for Parameters. We have focused on TAFC tasks in which signal
multiple-choice decision making in domains such as memory strength (drift rate) remains constant over the course of a single
retrieval (e.g., Anderson, 1983; Diller, Nobel, & Shiffrin, 2001) trial. However, if signal-to-noise ratio varies within a trial, optimal
and lexical decisions (e.g., Wagenmakers et al., 2004) and with the performance requires adjusting the gain of the integrator (i.e., its
work of Roe, Busemeyer, and Townsend (2001). The latter gen- sensitivity to inputs), as suggested by classical signal-processing
eralized the model combining mutual and feedforward inhibition theory (Papoulis, 1977). We have begun to characterize the tra-
(described in Equation 34) to three alternatives and used it to jectories for dynamic gain adjustment that optimize performance
explain a number of fundamental empirical results described in in the DDM and various related neural network models (Brown et
multialternative preference literature. An important avenue for al., 2005).
future inquiry will be to extend the methods and analyses de- Work on dynamic gain adjustment has also identified and begun
scribed in this article to apply to the domain of multichoice to address an interesting tension between the optimality of a
decision making. single-layered neural network that implements the DDM (such as
the models considered in this article) and the reality that such a
Optimization and Cognitive Control network is almost certainly embedded in a multilayered system
within the brain (required, among other things, to translate the
The evidence reviewed in this article suggests that, at least under outcome of the decision process into overt behavior). Numerical
some conditions, human decision makers can adapt their behavior optimization studies suggest that dynamic gain adjustment, medi-
to approximate optimal performance. This ability can be consid- ated by brainstem neuromodulatory systems that are triggered by
ered as a simple but well-defined example of cognitive control. the outcome of the relevant DDM implemented in a particular
Broadly speaking, cognitive control is the ability to guide and layer, can recover a substantial fraction of the performance of a
adapt behavior to meet desired goals. The challenge, in psychology single-layered system (Brown et al., 2004; Gilzenrat et al., 2004).
and neuroscience, has been to operationalize these constructs in In the present article, we have assumed that the experimental
terms that lead to a precise understanding of the mechanisms delay D does not directly affect the parameters of the DDM
involved and can generate quantitatively detailed predictions re- beyond determining what their optimal values would be. However,
garding task performance. The DDM provides a framework for it is possible that D interacts with attentional effects and, hence,
doing so by defining a specific decision-making mechanism with the drift A. Delay D may also influence biases or variability in the
critical parameters that have a direct impact on performance: starting point (cf. the preparatory cycle in Brown & Holmes, 2001;
namely, the drift rate (and its variability), the starting point of Cho et al., 2002), as well as trial-to-trial threshold adaptation, as
integration (and its variability), and the decision threshold. These both of these may plausibly require a certain intertrial duration to
parameters can be interpreted in terms of psychological processes: take effect.
signal strength, stimulus difficulty, and the allocation of attention Another critical factor is attention. As noted earlier, assumptions
(drift); expectancy biases (starting point); and the speed–accuracy of linearity in processing can be justified by assuming that atten-
trade-off (threshold). Each of these psychological constructs has tion acts to place nonlinear integrators in the most sensitive and
approximately linear range of their response functions (e.g., Cohen tions in which reward delivery is delayed or information about
et al., 1990). This suggests that fluctuations in attention, both accuracy is not immediately available? For example, in most
between and within trials, could have a significant impact on human studies, payments based on performance are not made until
performance. Indeed, such effects have been exploited to model the end of the experiment. Other forms of feedback can serve as
the effects of attention on the dynamics of performance in a variety proxies for reward (e.g., a visual display of the dollar amount
of paradigms (e.g., Cohen, Servan-Schreiber, & McClelland, 1992; won). However, in many circumstances, people seem able to adapt
Grossberg & Stone, 1986; Servan-Schreiber, Bruno, Carter, & their behavior even when no explicit feedback is provided (e.g.,
Cohen, 1998; Sperling, Reeves, Blaser, Lu, & Weichselgartner, Botvinick, Nystrom, Fissell, Carter, & Cohen, 1999; Gratton,
2001). The present work provides a framework for analyzing the Coles, Sirevaag, Eriksen, & Donchin, 1988; Laming, 1979). Mea-
influence of such factors, identifying optimal protocols for atten- surements of brain activity (using both scalp electrical recordings
tional adjustment, and using these as a guide for the design of and functional magnetic resonance imaging) have produced strong
further empirical studies. For example, there are at least two ways evidence of neural signals that are responsive to both endogenous
in which attentional mechanisms could modulate processing: One and exogenous sources of information about performance (e.g.,
is by gain adjustment (along the lines suggested above); the other Carter et al., 1998; Falkenstein, Hohnsbein, Hoormann, & Blanke,
is by additive bias adjustment that, interacting with the nonlinear- 1991; Gehring, Goss, Coles, Meyer, & Donchin, 1993; Holroyd et
ity of the response function, can change the dynamic range of the al., 2004; Nieuwenhuis, Yeung, Holroyd, Schurger, & Cohen,
integrator. There has been considerable debate in the behavioral, 2004). In modeling work, we have begun to explore neurally
neurophysiological, and modeling literatures as to which of these plausible mechanisms for performance monitoring (e.g., Botvin-
better describes attentional influences. However, to our knowl- ick, Braver, Barch, Carter, & Cohen, 2001; Holroyd, Yeung,
edge, there is no analytic treatment of this issue within the opti- Coles, & Cohen, 2005; Yeung, Botvinick, & Cohen, 2004) that can
mality framework. The present framework allows for such a treat- provide internal indexes of performance and use these to adap-
ment, in terms of the influence on drift rate, that can bridge tively adjust behavior without access to explicit information about
between abstract characterization of behavior and neural imple- reward or accuracy. However, to date, such work remains discon-
mentation. Finally, as we noted earlier, periodic shifts in attention nected from more principled and formally rigorous considerations
may explain the variability in drift rate that has been observed of optimal performance. By relating models of performance mon-
empirically under a number of conditions. Such shifts of attention itoring to the ones examined in this article, it may be possible to
may reflect the effects of exogenous factors or a fundamental derive formal principles that better justify existing models and help
tension between the exploitation of opportunities for reward in the guide their further development.
current task and the value of periodically exploring the possibility Algorithms for adaptation. The foregoing considerations as-
of other opportunities for reward (e.g., Aston-Jones & Cohen, sume mechanisms for translating information about the outcome of
2005). performance into adjustments of parameters in the decision-
Criteria for optimization. Optimality theory requires that we making process that produce improvements. Above, in the section
specify an objective or cost function that defines what is being entitled A Theory of Optimal Thresholds, we suggested the pos-
optimized. This has the virtue of forcing us to be precise but the sibility that reinforcement-learning algorithms are a natural candi-
peril of being too narrow in our assumptions. In this article, we date given that they are designed to maximize reward (e.g., Simen
have considered four criteria involving reward and accuracy in et al., 2005, 2006). It remains to be determined, however, to what
various combinations. The importance of reward is plausible given extent such algorithms can reproduce detailed features of sequen-
that rewards are, by definition, a form of utility valued and actively tial adjustment effects that have been observed in the empirical
sought by the organism. The value of accuracy also seems self- literature (e.g., Botvinick et al., 1999; Gratton et al., 1988; Kerns
evident given that it is often a proxy for reward (especially when et al., 2004; Laming, 1979). It is also possible other mechanisms
time pressure is not a critical factor). However, these criteria have may be involved in behavioral adaptation. For example, we have
not taken account of other relevant factors, such as the potential shown that simple alternation and repetition detectors, using ex-
costs associated with behavior (e.g., uncertainty of outcome, en- ponentially decaying averages of prior response history, can pro-
ergy expenditure, or opportunity costs associated with the pursuit duce adjustments in starting points in a mutual inhibition model
of other types of reward). In particular, behavior that may seem that accurately simulate human RT and ER effects (Cho et al.,
suboptimal under the criteria and tasks considered here may in fact 2002). Similarly, Botvinick et al. (2001) used a simple exponen-
be optimal when richer criteria or task environments are consid- tially decaying average of response conflict to adjust attentional
ered. For example, the tendency to explore previously unrewarded influences and thresholds, simulating sequential adjustment effects
alternatives is not optimal in a static environment but could yield such as posterror slowing. An important goal for further work will
higher rewards in a changing environment (Sutton & Barto, 1998). be to determine whether these mechanisms converge on parame-
Such factors can readily be explored within the context of the ters that produce optimal performance and, if not, how mecha-
optimization framework and may help guide further empirical nisms that do so compare in fitting empirical data.
research (e.g., Aston-Jones & Cohen, 2005; Yu & Dayan, 2005).
Another important consideration is the source of information Conclusions
available to the organism about the outcome of performance and
used to adapt behavior. In the simplest case, earned rewards or We hope in this article to have contributed to the development
information about accuracy are immediately available and can be of a framework for the formal analysis of human decision-making
used directly by the mechanisms of adaptation to optimize the behavior. We have used this framework to examine how existing
parameters of the decision process. However, what about situa- models relate to the optimal TAFC decision-making process—the
DDM—and have extended the analysis of optimality to address November 11). Conflict monitoring vs. selection-for-action in anterior
maximization of reward and accuracy. This has generated insights cingulate cortex. Nature, 402, 179 –181.
into issues relevant to both psychology and neuroscience, as well Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10,
as their shared interests. For example, theories of optimal perfor- 433– 436.
mance make strong predictions about how human decision makers Britten, K. H., Shadlen, M. N., Newsome, W. T., & Movshon, J. A. (1993).
Responses of neurons in macaque MT to stochastic motion signals.
seeking to maximize reward should trade speed against ac-
Visual Neuroscience, 10, 1157–1169.
curacy—a phenomenon that pervades experimental research. At Brown, E., Gao, J., Holmes, P., Bogacz, R., Gilzenrat, M., & Cohen, J. D.
the neural level, our analyses make predictions about underlying (2005). Simple networks that optimize decisions. International Journal
mechanisms and how they may implement optimal performance. of Bifurcations and Chaos, 15, 803– 826.
Confirmation of these predictions would at once strengthen con- Brown, E., Gilzenrat, M., & Cohen, J. D. (2004). The locus coeruleus,
fidence in our framework and help adjudicate between competing adaptive gain, and the optimization of simple decision tasks (Tech. Rep.
neural network models of behavior. Most importantly, we believe No. 04-02). Princeton, NJ: Princeton University, Center for the Study of
that our approach offers hope that complex neural systems respon- Mind, Brain, and Behavior.
sible for behavior can, under reasonable assumptions, yield to Brown, E., & Holmes, P. (2001). Modeling a simple choice task: Stochastic
simpler, more abstract descriptions that are tractable to formal dynamics of mutually inhibitory neural groups. Stochastics and Dynam-
analysis and yet remain faithful to and can be used to make ics, 1, 159 –191.
Brunel, N., & Latham, P. (2003). Firing rate of noisy quadratic integrate-
predictions about behavior. Success in this effort promises a prin-
and-fire neurons. Neural Computation, 15, 2281–2306.
cipled understanding of the neural mechanisms underlying psy-
Bull, C. D., Metcalfe, N. B., & Mangel, M. (1996). Seasonal matching of
chological function and overt behavior akin to the one that has foraging to anticipated energy requirements in anorexic juvenile salmon.
emerged from physics relating atomic structure to the macroscopic Proceedings of the Royal Society, Series B, 263, 13–18.
properties of matter. Busemeyer, J. R., & Diederich, A. (2002). Survey of decision field theory.
Mathematical Social Sciences, 43, 345–370.
Busemeyer, J. R., & Myung, I. J. (1992). An adaptive approach to human
References decision making: Learning theory and human performance. Journal of
Experimental Psychology: General, 121, 177–194.
Abramowitz, M., & Stegun, I. A. (1984). Handbook of mathematical
Busemeyer, J. R., & Rapoport, A. (1988). Psychological models of de-
functions with formulas, graphs, and mathematical tables. New York:
ferred decision making. Journal of Mathematical Psychology, 32, 91–
Wiley.
134.
Aksay, E., Gamkrelidze, G., Seung, H. S., Baker, R., & Tank, D. W.
Busemeyer, J. R., & Townsend, J. T. (1992). Fundamental derivations from
(2001). In vivo intracellular recording and perturbation of persistent
decision field theory. Mathematical Social Sciences, 23, 255–282.
activity in a neural integrator. Nature Neuroscience, 4, 184 –193.
Busemeyer, J. R., & Townsend, J. T. (1993). Decision field theory: A
Anderson, J. R. (1983, April 1). Retrieval of information from long-term
dynamic-cognitive approach to decision making in an uncertain envi-
memory. Science, 220, 25–30.
Anderson, J. R. (1990). The adaptive character of thought. Hillsdale, NJ: ronment. Psychological Review, 100, 432– 459.
Erlbaum. Cannon, S. C., Robinson, D. A., & Shamma, S. (1983). A proposed neural
Aston-Jones, G., & Cohen, J. D. (2005). An integrative theory of locus network for the integrator of the oculomotor system. Biological Cyber-
coeruleus–norepinephrine function: Adaptive gain and optimal perfor- netics, 49, 127–136.
mance. Annual Review of Neuroscience, 28, 403– 450. Carpenter, R. H. S., & Williams, M. L. L. (1995, September 7). Neural
Barnard, G. A. (1946). Sequential tests in industrial statistics. Journal of computation of log likelihood in control of saccadic eye movements.
Royal Statistical Society, 8(Suppl.), 1–26. Nature, 377, 59 – 62.
Belovsky, G. E. (1984). Herbivore optimal foraging: A comparative test of Carter, C. S., Braver, T. S., Barch, D. M., Botvinick, M. M., Noll, D. C.,
three models. American Naturalist, 124, 97–115. & Cohen, J. D. (1998, May 1). Anterior cingulate cortex, error detection
Bender, C. M., & Orszag, S. A. (1978). Advanced mathematical methods and the online monitoring of performance. Science, 280, 747–749.
for scientists and engineers. New York: McGraw-Hill. Cho, R. Y., Nystrom, L. E., Brown, E., Jones, A. D., Braver, T. S., Holmes,
Bialek, W., Rieke, F., de Ruyter van Steveninck, R. R., & Warland, D. P. J., & Cohen, J. D. (2002). Mechanisms underlying dependencies of
(1991, June 28). Reading a neural code. Science, 252, 1854 –1857. performance on stimulus history in a two-alternative forced-choice task.
Billingsley, P. (1995). Probability and measure. New York: Wiley. Cognitive Affective and Behavioral Neuroscience, 2, 283–299.
Blakemore, C., & Campbell, F. W. (1969). On the existence of neurones in Cohen, J. D., Dunbar, K., & McClelland, J. L. (1990). On the control of
the human visual system selectively sensitive to the orientation and size automatic processes: A parallel distributed processing account for the
of retinal images. Journal of Physiology (London), 15, 237–260. Stroop effect. Psychological Review, 97, 332–361.
Bogacz, R., & Cohen, J. D. (2004). Parameterization of connectionist Cohen, J. D., Servan-Schreiber, D., & McClelland, J. L. (1992). A parallel
models. Behavioral Research Methods, Instruments, and Computers, 36, distributed processing approach to automaticity. American Journal of
732–741. Psychology, 105, 239 –269.
Bogacz, R., & Gurney, K. (in press). The basal ganglia and cortex imple- DeGroot, M. H. (1988). A conversation with George A. Barnard. Statistical
ment optimal decision making between alternative actions. Neural Com- Science, 3, 196 –212.
putation. Diederich, A., & Busemeyer, J. R. (2003). Simple matrix methods for
Bohil, C. J., & Maddox, W. T. (2003). On generality of optimal versus analyzing diffusion models of choice probability, choice response time,
objective classifier feedback effects on decision criterion learning in and simple response time. Journal of Mathematical Psychology, 47,
perceptual categorization. Memory & Cognition, 31, 181–198. 304 –322.
Botvinick, M., Braver, T. S., Barch, D., Carter, C., & Cohen, J. D. (2001). Diller, D. E., Nobel, P. A., & Shiffrin, R. M. (2001). An ARC-REM model
Conflict monitoring and cognitive control. Psychological Review, 108, for accuracy and response time in recognition and cued recall. Journal
625– 652. of Experimental Psychology: Learning, Memory, and Cognition, 27,
Botvinick, M., Nystrom, L., Fissell, K., Carter, C. S., & Cohen, J. D. (1999, 414 – 435.
Ditterich, J., Mazurek, M. E., & Shadlen, M. N. (2003). Microstimulation and temporal-order information in short-term memory. Memory & Cog-
of visual cortex affects the speed of perceptual decisions. Nature Neu- nition, 14, 451– 468.
roscience, 6, 891– 898. Hanes, D. P., & Schall, J. D. (1996, October 18). Neural control of
Dragalin, V. P., Tartakovsky, A. G., & Veeravalli, V. V. (1999). Multihy- voluntary movement initiation. Science, 274, 427– 430.
pothesis sequential probability ratio tests: Part I. Asymptotic optimality. Herrnstein, R. J. (1961, June 30). Stereotypy and intermittent reinforce-
IEEE Transactions on Information Theory, 45, 2448 –2461. ment. Science, 133, 2067–2069.
Dragalin, V. P., Tartakovsky, A. G., & Veeravalli, V. V. (2000). Multihy- Herrnstein, R. J. (1997). The matching law. Cambridge, MA: Harvard
pothesis sequential probability ratio tests: Part II. Accurate asymptotic University Press.
expansions for the expected sample size. IEEE Transactions on Infor- Higham, D. J. (2001). An algorithmic introduction to numerical simulation
mation Theory, 46, 1366 –1383. of stochastic differential equations. SIAM Review, 43, 525–546.
Dunbar, R. I. M. (1984). Reproductive decisions: An economic analysis of Hill, C. M. (1898). On choice. American Journal of Psychology, 9, 587–
Gelada baboon social strategies. Princeton, NJ: Princeton University 590.
Press. Holmes, P., Lumley, J. L., & Berkooz, G. (1996). Turbulence, coherent
Edwards, W. (1965). Optimal strategies for seeking information: Models structures, dynamical systems and symmetry. Cambridge, England:
for statistics, choice reaction times, and human information processing. Cambridge University Press.
Journal of Mathematical Psychology, 2, 312–329. Holroyd, C. B., Nieuwenhuis, S., Yeung, N., Nystrom, L., Mars, R. B.,
Eisenberg, B. (1991). Multihypothesis problems. In B. K. Ghost & P. K. Coles, M. G. H., & Cohen, J. D. (2004). Dorsal anterior cingulate cortex
Sen (Eds.), Handbook of sequential analysis (pp. 229 –244). New York: shows fMRI response to internal and external error signals. Nature
Mercel Dekker. Neuroscience, 7, 497– 498.
Erev, I. (1998). Signal detection by human observers: A cut-off reinforce- Holroyd, C. B., Yeung, N., Coles, M. G. H., & Cohen, J. D. (2005). A
ment learning model of categorization decisions under uncertainty. Psy- mechanism for error detection in speeded response time tasks. Journal of
chological Review, 105, 280 –298. Experimental Psychology: General, 134, 163–191.
Ermentrout, G. B. (1994). Reduction of conductance-based models with Kahneman, D., & Tversky, A. (1984). Choices, values, and frames. Amer-
slow synapses to neural nets. Neural Computation, 6, 679 – 695. ican Psychologist, 39, 341–350.
Falkenstein, M., Hohnsbein, J., Hoormann, J., & Blanke, L. (1991). Effects Kerns, J. G., Cohen, J. D., MacDonald, A. W., III, Cho, R. Y., Stenger,
of crossmodal divided attention on late ERP components: II. Error
V. A., & Carter, C. S. (2004, February 13). Anterior cingulate conflict
processing in choice reaction tasks. Electroencephalography and Clin-
monitoring and adjustments in control. Science, 303, 1023–1026.
ical Neurophysiology, 78, 447– 455.
Koulakov, A. A., Raghavachari, S., Kepecs, A., & Lisman, J. E. (2002).
Feller, W. (1968). An introduction to probability theory and its applica-
Model for a robust neural integrator. Nature Neuroscience, 5, 775–782.
tions. New York: Wiley.
LaBerge, D. A. (1962). A recruitment theory of simple behavior. Psy-
Gardiner, C. W. (1985). Handbook of stochastic methods (2nd ed.). New
chometrika, 27, 375–396.
York: Springer.
Laming, D. R. J. (1968). Information theory of choice-reaction times. New
Gehring, W. J., Goss, B., Coles, M. G. H., Meyer, D. E., & Donchin, E.
York: Wiley.
(1993). A neural system for error-detection and compensation. Psycho-
Laming, D. (1979). Choice reaction performance following an error. Acta
logical Science, 4, 385–390.
Psychologica, 43, 199 –224.
Ghosh, B. K. (1970). Sequential tests of statistical hypotheses. Reading,
Lehmann, E. L. (1959). Testing statistical hypotheses. New York: Wiley.
MA: Addison-Wesley.
Levine, D. S., & Elsberry, W. R. (Eds.). (1997). Optimality in biological
Gilzenrat, M. S., Brown, E. T., Aston-Jones, G., & Cohen, J. D. (2004).
and artificial networks? Mahwah, NJ: Erlbaum.
Locus coeruleus, adaptive gain, and the optimization of decision tasks
[Abstract] (Program No. 899.6). Retrieved from the 2004 Abstract Ling, D. S. F., & Benardo, L. S. (1995). Recruitment of GABAA inhibition
Viewer and Itinerary Planner on the Society for Neuroscience Web site, in rat neocortex is limited and not NMDA dependent. Journal of Neu-
http://sfn.scholarone.com rophysiology, 74, 2329 –2335.
Gold, J. I., & Shadlen, M. N. (2001). Neural computations that underlie Link, S. W. (1975). The relative judgment theory of two choice response
decisions about sensory stimuli. Trends in Cognitive Sciences, 5, 10 –16. time. Journal of Mathematical Psychology, 12, 114 –135.
Gold, J. I., & Shadlen, M. N. (2002). Banburismus and the brain: Decoding Link, S. W., & Heath, R. A. (1975). A sequential theory of psychological
the relationship between sensory stimuli, decisions and reward. Neuron, discrimination. Psychometrika, 40, 77–105.
36, 299 –308. Loewenstein, G., & Thaler, R. (1989). Anomalies: Intertemporal choice.
Gold, J. I., & Shadlen, M. N. (2003). The influence of behavioral context Journal of Economic Perspectives, 3, 181–193.
on the representation of a perceptual decision in developing oculomotor Logan, G. D. (2002). An instance theory of attention and memory. Psy-
commands. Journal of Neuroscience, 23, 632– 651. chological Review, 109, 376 – 400.
Goldman, M. S., Levine, J. H., Major, G., Tank, D. W., & Seung, H. S. Logan, G. D., & Bundesen, C. (2003). Clever homunculus: Is there an
(2003). Robust persistent neural activity in a model integrator with endogenous act of control in the explicit task-cuing procedure? Journal
multiple hysteretic dendrites per neuron. Cerebral Cortex, 13, 1185– of Experimental Psychology: Human Perception and Performance, 29,
1195. 575–599.
Good, I. J. (1979). Studies in the history of probability and statistics: Lorden, G. (1977). Nearly-optimal sequential tests for finitely many pa-
XXXVI. A. M. Turing’s statistical work in World War II. Biometrika, rameter values. Annals of Statistics, 5, 1–21.
66, 393–396. Maddox, W. T., & Bohil, C. J. (1998). Base-rate and payoff effects in
Gratton, G., Coles, M. G., Sirevaag, E. J., Eriksen, C. W., & Donchin, R. J. multidimensional perceptual categorization. Journal of Experimental
(1988). Pre- and poststimulus activation of response channels: A psy- Psychology: Learning, Memory, and Cognition, 24, 1459 –1482.
chophysiological analysis. Journal of Experimental Psychology: Human Maddox, W. T., & Bohil, C. J. (2001). Feedback effects on cost-benefit
Perception and Performance, 14, 331–344. learning in perceptual categorization. Memory & Cognition, 29, 598 –
Grossberg, S. (1988). Nonlinear neural networks: Principles, mechanisms, 615.
and architectures. Neural Networks, 1, 17– 61. Major, G., Baker, R., Aksay, E., Mensh, B., Seung, H. S., & Tank, D. W.
Grossberg, S., & Stone, G. (1986). Neural dynamics of attention switching (2004). Plasticity and tuning by visual feedback of the stability of a
neural integrator. Proceedings of the National Academy of Sciences, response bias in multiletter matching. Perception & Psychophysics, 35,
USA, 101, 7739 –7744. 464 – 476.
Maritz, J. S., & Jarrett, R. G. (1978). A note on estimating the variance of Rapoport, A., & Burkheimer, G. B. (1971). Models for deferred decision
the sample median. Journal of the American Statistical Association, 73, making. Journal of Mathematical Psychology, 8, 508 –538.
194 –196. Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review,
Mazurek, M. E., Roitman, J. D., Ditterich, J., & Shadlen, M. N. (2003). A 83, 59 –108.
role for neural integrators in perceptual decision making. Cerebral Ratcliff, R. (1980). A note on modelling accumulation of information when
Cortex, 13, 1257–1269. the rate of accumulation changes over time. Journal of Mathematical
McBain, C. J., & Dingledine, R. (1993). Heterogeneity of synaptic gluta- Psychology, 21, 178 –184.
mate receptors on CA3 stratum radiatum interneurons. Journal of Phys- Ratcliff, R. (1985). Theoretical interpretations of the speed and accuracy of
iology (London), 462, 373–392. positive and negative responses. Psychological Review, 92, 212–225.
McElree, B., & Dosher, B. A. (1989). Serial position and set size in Ratcliff, R. (1988). Continuous versus discrete information processing:
short-term memory: The time course of recognition. Journal of Exper- Modeling accumulation of partial information. Psychological Review,
imental Psychology: General, 118, 346 –373. 95, 238 –255.
McMillen, T., & Holmes, P. (2006). The dynamics of choice among Ratcliff, R., Cherian, A., & Segraves, M. A. (2003). A comparison of
multiple alternatives. Journal of Mathematical Psychology, 50, 30 –57. macaque behavior and superior colliculus neuronal activity to predic-
Moehlis, J., Brown, E., Bogacz, R., Holmes, P., & Cohen, J. D. (2004). tions from models of two choice decisions. Journal of Neurophysiology,
Optimizing reward rate in two alternative choice tasks: Mathematical 90, 1392–1407.
formalism (Tech. Rep. No. 04-01). Princeton, NJ: Princeton University, Ratcliff, R., Gomez, R., & McKoon, G. (2004). A diffusion model account
Center for the Study of Brain, Mind, and Behavior. of the lexical decision task. Psychological Review, 111, 159 –182.
Mozer, M. C., Colagrosso, M. D., & Huber, D. H. (2002). A rational Ratcliff, R., & McKoon, G. (1997). A counter model for implicit priming
analysis of cognitive control in a speeded discrimination task. In T. in perceptual word identification. Psychological Review, 104, 319 –343.
Dietterich, S. Becker, & Z. Ghahramani (Eds.), Advances in neural Ratcliff, R., & Rouder, J. N. (1998). Modeling response times for two-
information processing systems (Vol. 14, pp. 51–57). Cambridge, MA: choice decisions. Psychological Science, 9, 347–356.
MIT Press. Ratcliff, R., & Rouder, J. N. (2000). A diffusion model account of masking
Myung, I. J., & Busemeyer, J. R. (1989). Criterion learning in a deferred in two-choice letter identification. Journal of Experimental Psychology:
decision making task. American Journal of Psychology, 102, 1–16. Human Perception and Performance, 26, 127–140.
Neyman, J., & Pearson, E. S. (1933). On the problem of the most efficient Ratcliff, R., & Smith, P. L. (2004). A comparison of sequential sampling
tests of statistical hypotheses. Philosophical Transactions of the Royal models for two-choice reaction time. Psychological Review, 111, 333–
Society of London, Series A, 231, 289 –337. 367.
Nieuwenhuis, S., Yeung, N., Holroyd, C. B., Schurger, A., & Cohen, J. D. Ratcliff, R., Thapar, A., & McKoon, G. (2003). A diffusion model analysis
(2004). Sensitivity of electrophysiological activity from medial frontal of the effects of aging on brightness discrimination. Perception &
cortex to utilitarian and performance feedback. Cerebral Cortex, 14, Psychophysics, 65, 523–535.
741–747. Ratcliff, R., & Tuerlinckx, F. (2002). Estimating parameters of the diffu-
Oaksford, M., & Chater, N. (1998). Rational models of cognition. Oxford, sion model: Approaches to dealing with contaminant reaction times and
England: Oxford University Press. parameter variability. Psychonomic Bulletin & Review, 9, 438 – 481.
Olshausen, B. A., & Field, D. J. (1996, June 13). Emergence of simple-cell Ratcliff, R., Van Zandt, T., & McKoon, G. (1999). Connectionist and
receptive field properties by learning a sparse code for natural images. diffusion models of reaction time. Psychological Review, 106, 261–300.
Nature, 381, 607– 609. Reed, A. V. (1973, August 10). Speed–accuracy trade-off in recognition
Omurtag, A., Knight, B. W., & Sirovich, L. (2000). On the simulation of memory. Science, 181, 574 –576.
large populations of neurons. Journal of Computational Neuroscience, Renart, A., Brunel, N., & Wang, X.-J. (2003). Mean-field theory of
8, 51– 63. recurrent cortical networks: From irregularly spiking neurons to working
Pachella, R. G. (1974). The interpretation of reaction time in information- memory. In J. Feng (Ed.), Computational neuroscience: A comprehen-
processing research. In B. Kantowitz (Ed.), Human information process- sive approach (pp. 431– 490). Boca Raton, FL: CRC Press.
ing: Tutorial in performance and recognition (pp. 41– 82). Hillsdale, NJ: Roe, R. M., Busemeyer, J. R., & Townsend, J. T. (2001). Multialternative
Erlbaum. decision field theory: A dynamic connectionist model of decision mak-
Palmer, J., Huk, A. C., & Shadlen, M. N. (2005). The effect of stimulus ing. Psychological Review, 108, 370 –392.
strength on the speed and accuracy of a perceptual decision. Journal of Rouder, J. N. (1996). Premature sampling in random walks. Journal of
Vision, 5, 376 – 404. Mathematical Psychology, 40, 287–296.
Papoulis, A. (1977). Signal analysis. New York: McGraw-Hill. Rowan, T. (1990). Functional stability analysis of numerical algorithms.
Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Unpublished doctoral dissertation, University of Texas at Austin.
Transforming numbers into movies. Spatial Vision, 10, 437– 442. Schall, J. D. (2001). Neural basis of deciding, choosing and acting. Nature
Pike, A. R. (1966). Stochastic models of choice behavior: Response prob- Reviews Neuroscience, 2, 33– 42.
abilities and latencies of finite Markov chain systems. British Journal of Schall, J. D., & Thompson, K. G. (1999). Neural selection and control of
Mathematical and Statistical Psychology, 21, 161–182. visually guided eye movements. Annual Review of Neuroscience, 22,
Pitz, G. F., & Reinhold, H. (1968). Payoff effects in sequential decision 241–259.
making. Journal of Experimental Psychology, 77, 249 –257. Servan-Schreiber, D., Bruno, R., Carter, C., & Cohen, J. D. (1998).
Platt, M. L., & Glimcher, P. W. (1999, July 15). Neural correlates of Dopamine and the mechanisms of cognition: Part I. A neural network
decision variables in parietal cortex. Nature, 400, 233–238. model predicting dopamine effects on selective attention. Biological
Proctor, R. W., & Rao, K. V. (1983). Evidence that the same– different Psychiatry, 43, 713–722.
disparity is not attributable to response bias. Perception & Psychophys- Seung, H. S. (1996). How the brain keeps the eyes still. Proceedings of the
ics, 34, 72–76. National Academy of Sciences, USA, 93, 13339 –13344.
Proctor, R. W., Rao, K. V., & Hurst, P. W. (1984). An examination of Seung, H. S. (2003). Amplification, attenuation, and integration. In M. A.
Adbib (Ed.), The handbook of brain theory and neural networks (2nd Thapar, A., Ratcliff, R., & McKoon, G. (2003). A diffusion model analysis
ed., pp. 94 –97). Cambridge, MA: MIT Press. of the effects of aging on letter discrimination. Psychology and Aging,
Seung, H. S., Lee, D. D., Reis, B. Y., & Tank, D. W. (2000). Stability of 18, 415– 429.
the memory of eye position in a recurrent network of conductance-based Tuerlinckx, F. (2004). The efficient computation of the cumulative distri-
model neurons. Neuron, 26, 259 –271. bution and probability density functions in the diffusion model. Behav-
Shadlen, M. N., & Newsome, W. T. (1996). Motion perception: Seeing and ioral Research Methods, Instruments, and Computers, 36, 702–716.
deciding. Proceedings of the National Academy of Sciences, USA, 93, Usher, M., Cohen, J. D., Servan-Schreiber, D., Rajkowski, J., & Aston-
628 – 633. Jones, G. (1999, January 22). The role of locus coeruleus in the regu-
Shadlen, M. N., & Newsome, W. T. (2001). Neural basis of a perceptual lation of cognitive performance. Science, 283, 549 –554.
decision in the parietal cortex (Area LIP) of the rhesus monkey. Journal Usher, M., & McClelland, J. L. (1995). On the time course of perceptual
of Neurophysiology, 86, 1916 –1936. choice: A model based on principles of neural computation (Tech. Rep.
Shafir, E., & Tversky, A. (1995). Decision making. In E. E. Smith & D. N. No. PDP.CNS.95.5). Pittsburgh, PA: Carnegie Mellon University.
Osherson (Eds.), Thinking: An invitation to cognitive science (Vol. 3, Usher, M., & McClelland, J. L. (2001). On the time course of perceptual
2nd ed., pp. 77–100). Cambridge, MA: MIT Press. choice: The leaky competing accumulator model. Psychological Review,
Shelley, M., & McLaughlin, D. (2002). Coarse-grained reduction and 108, 550 –592.
analysis of a network model of cortical response. Journal of Computa- Van Zandt, T., Colonius, H., & Proctor, R. W. (2000). A comparison of
tional Neuroscience, 12, 97–122. two response time models applied to perceptual matching. Psychonomic
Simen, P. A., Cohen, J. D., & Holmes, P. (2006). Neural implementation Bulletin & Review, 7, 208 –256.
of decision threshold changes via reward rate estimates. Manuscript Vickers, D. (1970). Evidence for an accumulator model of psychophysical
submitted for publication. discrimination. Ergonomics, 13, 37–58.
Simen, P. A., Holmes, P., & Cohen, J. D. (2005). Threshold adaptation in Vickers, D., Caudrey, D., & Wilson, R. J. (1971). Discriminating between
decision making [Abstract] (Program No. 768.15). Retrieved from the the frequency of occurrence of two alternative events. Acta Psycho-
2005 Abstract Viewer and Itinerary Planner on the Society for Neuro- logica, 35, 151–172.
science Web site, http://sfn.scholarone.com Wagenmakers, E.-J., Steyvers, M., Raaijmakers, J. G. W., Shiffrin, R. M.,
Smith, P. L. (1995). Psychophysically principled models of visual simple van Rijn, H., & Zeelenberg, R. (2004). A model for evidence accumu-
reaction time. Psychological Review, 102, 567–593. lation in the lexical decision task. Cognitive Psychology, 48, 332–367.
Smith, P. L. (2000). Stochastic dynamic models of response time and Wald, A. (1947). Sequential analysis. New York: Wiley.
accuracy: A foundational primer. Journal of Mathematical Psychology, Wald, A., & Wolfowitz, J. (1948). Optimum character of the sequential
44, 408 – 463. probability ratio test. Annals of Mathematical Statistics, 19, 326 –339.
Smith, P. L., & Ratcliff, R. (2004). Psychology and neurobiology of simple Wallis, W. A. (1980). The Statistical Research Group, 1942–1945. Journal
decisions. Trends in Neuroscience, 27, 161–168. of the American Statistical Association, 75, 320 –330.
Smith, P. L., & Van Zandt, T. (2000). Time-dependent Poisson counter Wang, X.-J. (2002). Probabilistic decision making by slow reverberation in
models of response latency in simple judgment. British Journal of cortical circuits. Neuron, 36, 1–20.
Mathematical and Statistical Psychology, 53, 293–315. Wetherill, G. B., & Glazebrook, K. D. (1986). Sequential methods in
Smith, P. L., & Vickers, D. (1989). Modeling evidence accumulation with statistics (3rd ed.). New York: Chapman & Hall.
partial loss in expanded judgment. Journal of Experimental Psychology: Wickelgren, W. A. (1977). Speed–accuracy tradeoff and information pro-
Human Perception and Performance, 15, 797– 815. cessing dynamics. Acta Psychologica, 41, 67– 85.
Soetens, E., Deboeck, M., & Hueting, J. (1984). Automatic aftereffects in Wilson, H., & Cowan, J. (1972). Excitatory and inhibitory interactions in
two-choice reaction time: A mathematical representation of some con- localized populations of model neurons. Biophysics Journal, 12, 1–24.
cepts. Journal of Experimental Psychology: Human Perception and Wong, K.-F., & Wang, X.-J. (2006). A recurrent network mechanism of
Performance, 10, 581–598. time integration in perceptual decisions. Journal of Neuroscience, 26,
Spanier, J., & Oldham, K. B. (1987). An atlas of functions. Washington, 1314 –1328.
DC: Hemisphere Publication Services. Yang, T., Hanks, T. D., Mazurek, M., McKinley, M., Palmer, J., &
Sperling, G., Reeves, A., Blaser, E., Lu, Z.-L., & Weichselgartner, E. Shadlen, M. N. (2005). Incorporating prior probability into decision
(2001). Two computational models of attention. In J. Braun, C. Koch, & making in the face of uncertain reliability of evidence [Abstract] (Pro-
J. L. Davis (Eds.), Visual attention and cortical circuits (pp. 177–214). gram No. 621.4). Retrieved from the 2005 Abstract Viewer and Itinerary
Cambridge, MA: MIT Press. Planner on the Society for Neuroscience Web site, http://sfn.scholarone
Stevenson, M. K., Busemeyer, J. R., & Naylor, J. C. (1991). Judgement and .com
decision making theory. In M. D. Dunnette & L. M. Hough (Eds.), Yeung, N., Botvinick, M. M., & Cohen, J. D. (2004). The neural bases of
Handbook of industrial and organizational psychology (Vol. 1, 2nd ed., error detection: Conflict monitoring and error-related negativity. Psy-
pp. 283–374). Palo Alto, CA: Consulting Psychologist Press. chological Review, 111, 931–959.
Stone, M. (1960). Models for choice reaction time. Psychometrika, 25, Yu, A. J., & Dayan, P. (2005). Inference, attention, and decision in a
251–260. Bayesian neural architecture. In L. K. Saul, Y. Weiss, & L. Bottou
Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning. Cambridge, (Eds.), Advances in neural information processing systems (Vol. 17, pp.
MA: MIT Press. 1577–1584). Cambridge, MA: MIT Press.
(Appendixes follow)
Appendix A
Mathematical Details of Decision Optimization in Two-Alternative

Forced-Choice Tasks
In this appendix, we collect and describe mathematical results for first passage, boundaryless, and
reflecting-boundary drift diffusion problems. We consider both diffusion and Ornstein–Uhlenbeck
(O-U) models. The main tools are drawn from applied probability, the theory of stochastic ordinary
differential equations, and classical perturbation and asymptotic methods. Here, we focus on
continuous models in the form of stochastic ordinary differential equations, although we first
describe how these arise as limits of the discrete Neyman–Pearson and sequential probability
statistical tests, which are the optimal methods of deciding between two alternatives on the basis of
noisy accumulating data. It turns out that the optimal continuous processes admit rather simple exact
formulae for such behavioral observables as mean decision times (DTs) and error rates (ERs).
The first section, Probability Ratio Tests, reviews the optimality of the sequential probability ratio
test (SPRT) and describes how it becomes the drift diffusion model in the continuum limit. The next
section, Optimal Decision for the Free-Response Protocol, considers the free-response protocol, in
which participants make choices in their own time, and the third section, Optimal Decisions Under
the Interrogation Protocol, addresses the interrogation protocol, in which a decision is deferred until
a cue is presented. Finally, the fourth section, The Extended Drift Diffusion and O-U Models, gives
results for extended models in which drift rates and initial conditions are variable.
Probability Ratio Tests

Optimality of the Sequential Probability Ratio Test (SPRT)
The SPRT described in the main text is optimal in the following sense. Let P(rej Hi兩Hi) be the
probability that hypothesis Hi is true but rejected, i ⫽ 1, 2. Also, let Ei(N) be the expected value for
the number of observations required for a decision to be reached when hypothesis Hi is true, i ⫽ 1,
2. The following theorem was first proved by Wald and Wolfowitz (1948); a simpler proof was
given by Lehmann (1959; cf. Ghosh, 1970).
SPRT optimality theorem: Among all tests (fixed sample or sequential) for which P(rej Hi兩Hi) ⱕ ␣i , i ⫽
1, 2, and for which E1(N) and E2(N) are finite, the SPRT with error probabilities P(rej Hi兩Hi) ⫽ ␣i , i ⫽ 1,
2, minimizes both E1(N) and E2(N). Letting ER denote error rate, and DTSPRT (DTa , respectively) denote
the DT for the SPRT (some alternative, respectively) model, the SPRT Optimality Theorem implies that
DT SPRT 共ER兲 ⱕ DT a共ER兲, for all ER. (A1)
In the section entitled A Theory of Optimal Thresholds in the main text, we consider reward criteria
which depend on ER, DT, and other parameters. A corollary of the SPRT optimality theorem is that
the SPRT maximizes any reward criteria R ⫽ R(ER, DT(ER)) that is monotonically nonincreasing
with respect to DT, that is, for which ⳵R/⳵(DT) ⱕ 0. Indeed, suppose that ERSPRT (ERa , respec-
tively) gives the absolute maximum for R for the SPRT (alternative, respectively) model. Then,
max
letting RSPRT (Rmax
a , respectively) be the maximum value that R can take using the SPRT (alterna-
tive, respectively) model, we have
Rmax
a ⫽ R共ER a , DT a共ER a兲兲 ⱕ R共ER a , DT SPRT 共ER a兲兲 ⱕ R共ER SPRT , DT SPRT 共ER SPRT 兲兲 ⫽ RSPRT
max
.
(A2)
The first inequality follows from Equation A1 and the fact that R is monotonically nonincreasing,
whereas the second one follows from the definition of ERSPRT. Thus, under this simple condition
on R, the SPRT is able to achieve the maximum possible value for R for any decision strategy.
We now deduce the conditions for which the reward criteria described in the main text are
monotonically nonincreasing in DT and, hence, under what conditions one expects the SPRT to be
optimal for these criteria. Consider first the reward rate (RR) given by Equation 42. We find that
⳵共RR兲 1 ⫺ ER
⫽⫺ , (A3)
⳵共DT兲共DT ⫹ T0 ⫹ D ⫹ DpER兲 2
which is always negative because the denominator is always positive and ER ⬍ 1. Next, consider
the reward/accuracy (RA) function given by Equation 43. Here,
⳵共RA兲 ⳵共RR兲
⫽ ⫺c1 , (A4)
⳵共DT兲 ⳵共DT兲
which is negative whenever c1 ⬎ 0. Next, consider the modified RR given by Equation 44. Here,
⳵共RR m 兲 c1 共1 ⫺ ER兲 ⫺ c2 ER
⫽⫺ . (A5)
⳵共DT兲共DT ⫹ Dtotal兲 2
Because the denominator is always positive, we wish to determine when the numerator is negative.
This is just a linear function of ER passing through ⫺c1 at ER ⫽ 0 and with slope (c1 ⫹ c2).
Restricting attention to ER ⱕ 0.5, we find that RRm is monotonically nonincreasing for c1 ⬎ 0 and
c1 ⬎ c2. Finally, consider the Bayes risk (BR) given by Equation 41. Unlike the other reward
criteria, optimal performance minimizes BR, which, in the above framework, is equivalent to
maximizing 1/BR. We find that
⳵
⳵共DT兲冉冊1
BR
⫽
⫺c1
共c1 DT ⫹ c2 ER兲 2
, (A6)
which is negative for c1 ⬎ 0.
Thresholds for the SPRT

The thresholds Z1 and Z2 in the SPRT (Equation 2 of main text) are related to the ERs ␣1 and ␣2
as follows (Lehmann, 1959; Wald, 1947). Consider the set C1 of n-length sequences Y such that the
SPRT chooses H1 when Y occurs. That is, for any Y 僆 C1, p1(y1)p1(y2) . . . p1(yn) ⱖ
Z1p2(y1)p2(y2) . . . p2(yn). Integrating this inequality over all of C1,
p1 共C1 兲 ⱖ Z1 p2 共C1 兲, (A7)

where pj(C1) is the probability of making Choice 1 given that hypothesis Hj is true. By definition,
p1(C1) ⱖ 1 ⫺ ␣1, and p1(C2) ⱕ ␣2, so that
1 ⫺ ␣1
1 ⫺ ␣ 1 ⱖ Z1 ␣ 2 f Z1 ⱕ .
␣2
Similarly,
␣1
␣ 1 ⱕ Z2 共1 ⫺ ␣ 2 兲 f Z2 ⱖ .
1 ⫺ ␣2
The inequalities fail to be equalities because it is possible to overshoot the boundaries Z1 or Z2.
However, in practice, there is typically little penalty in assuming equality (Lehmann, 1959; Wald,
1947):
␣1 1 ⫺ ␣1
Z2 ⫽ , Z1 ⫽ . (A8)
1 ⫺ ␣2 ␣2
Note that the proof of the SPRT optimality theorem requires that Z2 ⬍ 1 ⬍ Z1 (Lehmann, 1959).
Using Equation A8, this condition becomes
␣1 1 ⫺ ␣1
⬍1⬍ . (A9)
1 ⫺ ␣2 ␣2
Thus, we require that ␣1 ⬍ 1 ⫺ ␣2. Now, 1 ⫺ ␣2 is the probability of choosing H2 when H2 is true.
Thus, the proof requires that the probability of choosing H2 when H1 is true be less than the
probability of choosing H2 when H2 is true. Similarly, it requires that ␣2 ⬍ 1 ⫺ ␣1, that is, that the
probability of choosing H1 when H2 is true be less than the probability of choosing H1 when H1 is
true. These are, of course, very reasonable conditions for a decision-making procedure. Wald (1947)
also gave approximate expressions for the expected numbers of observations Ei(N). Using Equation
A8, these may be written
(Appendixes continue)
冉
␣ 1 log
␣1
1 ⫺ ␣2 冊冉冊
⫹ 共1 ⫺ ␣ 1 兲log
1 ⫺ ␣1
␣2
冉冉冊冊
E1 共N兲 ⬇ , and (A10)
p1 共y兲
E1 log
p2 共y兲
共1 ⫺ ␣ 2 兲log 冉冊冉冊
␣1
1 ⫺ ␣2
⫹ ␣ 2 log
1 ⫺ ␣1
␣2
冉冉冊冊
E2 共N兲 ⬇ , (A11)
p1 共y兲
E2 log
p2 共y兲
where Ei(log( p1(y)/p2(y))) is the expected value of the argument when Hi is true, i ⫽ 1, 2.
We also note that the proof by Lehmann (1959) applies to the generalized case of decisions based
on biased data. Suppose that the participant has been told (or has deduced) that the probability of
drawing from the distribution p1(y) ( p2(y), respectively) is ⌸ (1 ⫺ ⌸, respectively). Then, the SPRT
continues as long as
⌸p1n
Z2 ⬍ ⬍ Z1 .
共1 ⫺ ⌸兲p2n
Equivalently, the thresholds Zj are simply multiplied by (1 ⫺ ⌸)/⌸; that is, observations are taken
as long as
1⫺⌸ p1n 1 ⫺ ⌸
Z2 ⬍ ⬍ Z1 . (A12)
⌸ p2n ⌸
Thus, if the upper alternative is more probable (⌸ ⬎ 1/2), both thresholds are shifted downward, and
vice versa.
Random Walks and the SPRT
Following Stone (1960), Laming (1968) applied the SPRT to a two-alternative forced-choice task.
In such a task, on each trial, a randomly chosen stimulus S1 or S2 is shown to the participant. The
participant is told to give a response Ri if he or she perceives stimulus Si , i ⫽ 1, 2. The response
made (which may or may not be correct) and the reaction time (RT) taken to make it are recorded
for each trial.
Laming (1968) modeled this procedure by supposing that decisions are made on the basis of
accumulation of information. Specifically, for each trial, the participant makes a series of brief
observations of the stimulus represented by the random sequence y1, y2, . . . , yn. The increment of
information gained from (independent) observation yr is defined to be
␦I r ⫽ log 冉冊 p1 共yr兲
p2 共yr兲
: (A13)
This is the log likelihood ratio (Gold & Shadlen, 2001, 2002), where pi(y) is the probability
distribution for y given that stimulus Si was presented, i ⫽ 1, 2. Implicitly, the participant has some
internal representation of p1(y) and p2(y). At the nth observation, the total information accumulated
is
冘冘冉冊
n n
p1 共yr兲
I ⫽ n
␦I ⫽
r
log . (A14)
p2 共yr兲
r⫽1 r⫽1
Observations continue as long as I2 ⬍ I n ⬍ I1, where I2 and I1 are constants. The response R2 (R1,
respectively) is made at step n if I n ⱕ I2 (I n ⱖ I1, respectively). Because, from Equation 1 of the
main text,
冉冊
I n ⫽ log
p1n
p2n
,
we see that the accumulation of information according to this formulation is equivalent to making
decisions using the SPRT with I2 ⫽ log Z2 and I1 ⫽ log Z1 (cf. Gold & Shadlen, 2002; Wald, 1947).
For example, if the desired ERs are ␣2 ⫽ ␣1 ⫽ ε, which is reasonable if the signals S1 and S2 are
equally salient, from Equation A8 we take
I2 ⫽ log 冉冊ε
1⫺ε
⬍ 0, I1 ⫽ log 冉冊 1⫺ε
ε
⫽ ⫺I2 ⬎ 0 (A15)
(cf. Laming, 1968). (The signs follow from the assumed inequality A9.) If we require equal ERs in
the case of signals of unequal salience, with ⌸ denoting the probability of S1, then, multiplying the
boundaries of Equation A8 and taking logs, we find:
I2 ⫽ log 冉 1⫺⌸
⌸ 冊冉冊
⫹ log
ε
1⫺ε
, I1 ⫽ log 冉 1⫺⌸
⌸ 冊冉冊
⫹ log
1⫺ε
ε
. (A16)
To maintain I2 ⬍ 0 ⬍ I1, we must select an ER lower than the smaller of ⌸ and 1 ⫺ ⌸.

Thus, from Equation A14, in logarithmic variables, the trajectory I n is a discrete-time, biased
random walk with initial condition zero: A new increment of information arrives, and the trajectory
is updated, as the timestep advances from n 3 n ⫹ 1 (recall that the increments ␦I r are assumed
to be independent and identically distributed).
The Continuum Limit of the SPRT

Hereafter, we treat the continuous-time limit I(t) of the process described above, in which
infinitesimal increments of information arrive at each moment in time. This limit must be taken with
some care to preserve the variability present in Equation A14. First, we recall some generalities
about the Wiener process W(t).
The Wiener process (often called Brownian motion) on the interval [0, T] is a random variable
W(t) that depends continuously on t 僆 [0, T] and satisfies the following (Gardiner, 1985; Higham,
2001):
• W(0) ⫽ 0.
• For 0 ⱕ s ⬍ t ⱕ T, W(t) ⫺ W(s) ⬃ 公t ⫺ sN(0, 1), where N(0, 1) is a normal distribution with
zero mean and unit standard deviation. Because the normal distribution is used, the process is often
referred as Gaussian.
• For 0 ⱕ s ⬍ t ⬍ u ⬍ v ⱕ T, W(t) ⫺ W(s) and W(v) ⫺ W(u) are independent.
In numerical simulations, the standard Wiener process is discretized with a timestep dt as
dW ⬃ 冑dtN共0, 1兲.
We now describe the continuum limit, which, up to an unimportant scale factor between timesteps
n and the continuous time t, proceeds as follows. Let the ␦I r have mean m and variance D2 (assumed
finite). Then, define the family (indexed by M ⫽ 1, 2, . . .) of random functions of t 僆 [0, T], where
T is some large time, as follows:
冘冘
k k
1 1
IM共t兲 ⫽ 共␦I r ⫺ m兲 ⫹ ␦I r, where k ⫽ Mt/T. (A17)
冑M r⫽1 M
r⫽1
Here, Mt/T is the largest integer smaller than Mt/T. Note that the first term of Equation A17 is
normalized by 1/公M and the second by 1/M, reflecting the different rates at which fluctuations and
means accumulate as the random increments are summed. For any M, IM(t) has mean mt/T and
variance D2t/T; for example, from Equation A14, I n has mean mn and variance D2n. Furthermore,
the Donsker invariance principle (see theorem 37.8 of Billingsley, 1995), together with the law of
large numbers, implies that as M 3 ⬁,
IM共t兲 f DW共t兲 ⫹ mt ⬅ I共t兲, (A18)

where W⵺ is a Wiener process and the convergence of the random functions I ⵺ is in the sense M
of distributions. In other words, the limiting process I(t) satisfies the stochastic differential equation
dI ⫽ mdt ⫹ DdW, I共0兲 ⫽ 0, (A19)

with boundaries I2 ⬍ 0 ⬍ I1. The drift m and variance D of the ␦I and hence of Equation A19
2 r
depend upon the distributions pi(y) (cf. Equation A13). For example, in the case of Gaussians
1 1
p1 共y兲 ⫽ e⫺共y⫺␮ 1兲 /共2␴ 兲 , p2 共y兲 ⫽ e⫺共y⫺␮ 2兲 /共2␴ 兲 ,
2 2 2 2
(A20)
冑2␲␴ 2
冑2␲␴ 2
with ␮1 ⬎ ␮2, we have
␦Ir ⫽ log 冉冊
p1 共yr兲
p2 共yr兲
⫽
␮1 ⫺ ␮2
␴ 2 yr ⫺
␮1 ⫹ ␮2
2 冉, 冊 (A21)
and if Si is presented, the expected value of yr is E(yr) ⫽ ␮i , and the variance is Var(yr) ⫽ ␴2. Thus,
taking expectations and substituting in Equation A21, we obtain
共␮ 1 ⫺ ␮ 2 兲 2
E共␦I r兲 ⫽ ⫾ ⫽ m, (A22)
2␴ 2
(the ⫹ applies if S1 is presented, the ⫺ if S2), and in both cases,
Var共␦I r兲 ⫽ ␮ 1 ⫺ ␮ 2 ⫽ D2 (A23)
(cf. Gold & Shadlen, 2001, 2002). If each increment ␦I r is derived from many subobservations (e.g.,
from different regions of the visual field or from large populations of neurons), this Gaussian
assumption is justified by the central limit theorem.
In the particular case of Equation A21 in which ␮1 ⫽ ⫺␮2 ⫽ A, ␴ ⫽ c, appropriate to tasks such
as the moving dots paradigm in which the alternative stimuli are of equal clarity (e.g., Schall, 2001),
the simplified form of Equation A21 implies that the accumulating information I n is simply a scaled
version of the running total of observations yr:
冘冘
n n
2A 2A def 2A
␦I r ⫽ y f In ⫽ ␦Ir ⫽ yr ⫽ y. (A24)
c2 r c2 c2 n
r⫽1 r⫽1
Assuming without loss of generality that S1 is presented, the yr has mean A and variance c2 so that,
in the continuous time limit analogous to Equation A17, the sequence of observations {yr} generates
a random process y(t), which satisfies the drift diffusion stochastic differential equation:
dy ⫽ Adt ⫹ cdW; y共0兲 ⫽ 0. (A25)
The logarithmic SPRT involving observations ␦I r is therefore equivalent to solving the first passage
problem defined by Equation A25 with thresholds y ⫽ z1 ⫽ (c2/2A)Ii and Ii as in Equations
A15–A16.
We emphasize that the constant drift stochastic differential equation A19 or A25 is a particular
limit of the discrete random walk occurring in the SPRT or Neyman–Pearson tests and that more
general stochastic differential equations, such as O-U processes, correspond to other (i.e., nonop-
timal) decision strategies. In the following sections, we analyze these stochastic processes in both
unconstrained (free response) and time-constrained (interrogation) contexts.
Optimal Decisions for the Free-Response Protocol
As suggested by the above discussion, we model the decision-making process for the two-
alternative forced-choice task as a one-dimensional drift diffusion process on the x-axis with two
(symmetric) thresholds x ⫽ ⫾z. The drift term represents the weight of evidence in favor of one
alternative; diffusion arises from unmodeled inputs, represented as white noise. We consider
diffusion and O-U models, both of which arise naturally in connectionist models of such tasks, as
described in the main text. In the free-response protocol, in which participants are free to respond
at any time after stimulus onset, we assume that a decision is made when the sample path first
crosses either threshold; thus, we have a first passage problem. Here, we present formulae for the
probabilities of first passage through the thresholds and of first passage times to either threshold, as
may be derived from the backward Kolmogorov or Fokker–Planck equation (as in Busemeyer &
Townsend, 1992; cf. Gardiner, 1985; Moehlis, Brown, Bogacz, Holmes, & Cohen, 2004). These in
turn represent expected error and percentage-correct rates and mean DTs. We also derive expres-
sions for the RR for these processes and show explicitly that the diffusion model allows higher RRs
than are possible from an O-U model.
The first section, The Drift Diffusion Model, presents ERs, DTs, and RRs for the drift diffusion
model and also finds the optimal threshold. The second section, The O-U Model, presents these
quantities for the O-U model and then finds the optimal value of ␭. The third section describes the
dependence of the optimal threshold on experimental delays that must be satisfied by any decision
maker. The fourth section describes modifications to the theoretical predictions arising from the fact
that participants must find the optimal threshold by learning. Finally, the fifth section derives
optimal thresholds and optimal performance curves for the alternative reward criteria discussed in
the text.
As in the main text, we define the RR to be the probability of a correct response divided by the
average time between responses. Allowing an imposed delay D ⱖ 0 between trials and an additional
penalty delay Dp ⱖ 0 after incorrect responses and denoting nondecision part of RT by T0,
1 ⫺ ER
RR ⫽ . (A26)
DT ⫹ D ⫹ T0 ⫹ DpER
We find it more convenient to consider 1/RR below. Similar analyses can be done for the other
reward criteria defined in the section entitled A Theory of Optimal Thresholds in the main text.
The Drift Diffusion Model

In this section, we explicitly calculate ERs, DTs, and RRs for the drift diffusion model. This
allows us to derive the crucial formula (Equation A33) for the unique optimal threshold in terms of
task delays and the signal-to-noise ratio. The parameter dependence implicit in Equation A33,
below, as discussed in the main text, leads to several experimentally relevant predictions about how
thresholds should vary with task delays and signal and noise strengths.
Consider the constant drift diffusion equation
dy ⫽ Adt ⫹ cdW, y共0兲 ⫽ y0 , (A27)

with thresholds ay ⫽ ⫺z , by ⫽ z. Without loss of generality, we take A ⬎ 0. Letting y ⫽ Ax and
defining
z̃ ⫽
z
A
⬎ 0, ã ⫽ 冉冊
A
c
2
⬎ 0, (A28)
Equation A27 becomes
1
dx ⫽ dt ⫹ dW, (A29)
冑ã
with thresholds a ⫽ ⫺z̃, b ⫽ z̃ and initial condition x0 ⫽ y0/A.
ERs, Mean DTs, and Reward Rates

Using standard techniques for calculating the statistics of the first passage of trajectories of
stochastic differential equations through boundaries (Gardiner, 1985; Moehlis et al., 2004; cf.
appendix of Busemeyer & Townsend, 1993), it is readily shown that
ER ⫽
1
1⫹e 2z̃ã ⫺ 再
1 ⫺ e⫺2x0ã
e2z̃ã ⫺ e⫺2z̃ã
, 冎 (A30)
DT ⫽ z̃ tanh共z̃ã兲 ⫹ 再 2z̃共1 ⫺ e⫺2x0ã兲

e2z̃ã ⫺ e⫺2z̃ã
⫺ x0 , 冎 and (A31)
1
⫽ z̃ ⫹ D ⫹ T0 ⫹ 共D ⫹ T0 ⫹ Dp ⫺ z̃兲e⫺2z̃ã
RR
⫹ 再 e2z̃ã(e2共z̃⫹x0兲ã ⫺ 1) 冎
(1 ⫹ e2z̃ã)[(D ⫹ T0 ⫹ Dp ⫺ z̃)(1 ⫺ e2x0ã) ⫹ x0 e2x0ã(1 ⫺ e2z̃ã)]
. (A32)
Note, in particular, that the ER (Equation A30) may be made as small as we wish for a given drift
and noise variance by picking the threshold z̃ ⬃ z sufficiently high.
These expressions allow biased initial data, as suggested by the unequal salience case of the
main-text section entitled Relationships Among the Models (cf. Equations A40 –A41, below), but
for the next subsection, we assume equal salience and unbiased initial data x0 ⫽ 0, in which case
the expressions in braces in Equations A30 –A32 vanish identically.
Optimal Thresholds for the Drift Diffusion Model

Suppose that we want to know the optimal value of z̃, say, z̃o , which maximizes the RR, or,
equivalently, which minimizes 1/RR. Setting
⳵ 1
⳵z̃ RR冉冊冏 z̃⫽z̃o
⫽ 0,
z̃o is found from
e2z̃oã ⫺ 1 ⫽ 2ã共Dtotal ⫺ z̃o兲, (A33)

where Dtotal ⫽ D ⫹ T0 ⫹ Dp. The left-hand side of Equation A33 vanishes at z̃o ⫽ 0 and is
monotonically increasing in z̃o , whereas the right-hand side is positive at z̃o ⫽ 0 (provided Dtotal is
non-negative) and is monotonically decreasing in z̃o. There is thus a unique solution to Equation
A33. Moreover, it is a minimum because, using Equation A33 to eliminate Dtotal , we have
⳵2
冉冊冏
1
⳵z̃2 RR z̃⫽z̃o
⫽ 2ãe⫺2z̃oã关1 ⫹ e2z̃oã兴 ⬎ 0. (A34)
Thus, RR is maximized with respect to z̃ at z̃ ⫽ z̃o(ã, Dtotal). We note that because the left-hand side
of (A33) is greater than 2z̃oã, it is necessary that z̃o ⬍ Dtotal /2.
It is useful to give limiting expressions for Equation A32 for small and large z̃. Taylor-expanding
Equation A32 in z̃ about z̃ ⫽ 0, we find that for small z̃,
1
⬇ 2共D ⫹ T0 兲 ⫹ Dp ⫺ 2ã共D ⫹ T0 ⫹ Dp兲z̃. (A35)
RR
On the other hand, using
lim z̃ ne⫺z̃ã ⫽ 0 (A36)

z̃3⬁
for any integer n ⱖ 0, we find that for large z̃,
1
⬇ D ⫹ T0 ⫹ z̃. (A37)
RR
Figure A1a shows a plot of 1/RR as a function of z̃, including the small and large z̃ approximations.
We note that even though the parameters in the drift diffusion first passage problem may be
reduced to two (z̃ and ã), it is still of interest to see how the threshold zo that maximizes RR depends
upon the original drift and noise root-mean-square parameters A, c, and on the delay parameters D,
Dp, T0. Figure A1c illustrates these dependences and also shows the approximations valid for small
and large values of the parameters:
A
zo ⬇ D , for large c, or small Dtotal , or small A, and (A38)
2 total
zo ⬇
c2
2A 冋2A2
册
log 2 Dtotal ,
c
for small c, or large Dtotal , or large A. (A39)
Biased Initial Conditions in the Drift Diffusion Model

In the event of unequal salience (⌸ ⫽ 1/2), rather than employing the asymmetric boundaries of
Equation A16, we may transform by letting y 哫 y ⫹ (c2/2A)log(⌸/(1 ⫺ ⌸)), and consider the
process A19 with biased initial data
y共0兲 ⫽
c2
2A
log 冉
⌸
1⫺⌸ 冊 (A40)
a) λ=0
b) λ = −0.1 λ = 0.1
40 40 40
30 30 30
1/RR
1/RR
1/RR
20 20 20
10 10 10
0 0 0
0 2 4 6 8 10 0 2 4 6 8 10 0 2 4 6 8 10
z̃ z̃ z̃
c) 3 20 3
Exact
2.5 2.5 Large
Small
15
2 2
o
z
o
1.5 10 1.5
o
z
1 1
5
0.5 0.5
0 0 0
0 20 40 D 60 80 10 0 0 5 10 15 20 0 1 2 3 4 5
total c A
d)
0.05 3 13.1
0.04 13
2.5
12.9
0.03
12.8
DT
1/RR
0.02 2
ER
12.7
0.01
12.6
1.5
0 12.5
−0.01 1 12.4
0 λ −0.5 −0.25 0 0.25 0.5 −0.5 −0.25 0λ 0.25 0.5
−0.5 0.5 λ
e) 2.2
2.15
0.08
2.1
RR(z̃o ( λ )) 2.05
z˜o ( λ )
2
0.0799
1.95
−0.1 −0.05 0 0.05 0.1 1.9

λ −0.1 −0.05 0 λ 0.05 0.1
Figure A1. Validity of approximations. a: 1/RR as a function of z̃ for the drift diffusion model with ã ⫽ 1, D ⫽
10, and Dp ⫽ 20. The solid line shows the exact result given by Equation A32, whereas the dashed lines show
the small and large z̃ approximations given by Equations A35 and A37. b: 1/RR as a function of z̃ for the
Ornstein–Uhlenbeck model for ã ⫽ 1, D ⫽ 10, and Dp ⫽ 20. Values of ␭ are shown above panels. Here, the
⫹s are from the exact formula (Equation A57), and the solid (dot-dashed, respectively) lines are results from
Equation A66 to O(␭2) (O(␭), respectively). The dashed lines show the small and large z̃ approximations given
by Equations A67 and A68. c: Threshold zo , which optimizes RR as a function of the original system parameters,
with left panel: A ⫽ 1, c ⫽ 1; middle panel: A ⫽ 1, Dtotal ⫽ 30; right panel: c ⫽ 1, Dtotal ⫽ 30. Approximations
for small (large, respectively) parameter values are shown as dashed (dotted, respectively) lines. d: Validity of
approximations for z̃ ⫽ 2, ã ⫽ 1, D ⫽ 10, and Dp ⫽ 20. Here the ⫹s are from the exact formulae (Equations
A55, A56, and A57), and the solid (dashed, respectively) lines are results from Equations A64, A65, and A66
to O(␭2) (O(␭), respectively). e: For fixed ␭, z̃o(␭) maximizes RR, giving RR(z̃o(␭)). Results for ã ⫽ 1, D ⫽ 10,
and Dp ⫽ 20, where z̃o(␭) is found from (⫹s) exact formula A57, (solid line) Equation A66 to O(␭2), and (dashed
line) Equation A66 to O(␭). DT ⫽ decision time; ER ⫽ error rate; RR ⫽ reward rate.
and symmetric boundaries
⫺z ⬍ 0 ⬍ z ⫽
c2
2A
log
1⫺ε
ε
. 冉冊 (A41)
Thus, when ⌸ ⫽ 1/2, the process starts closer to the threshold corresponding to the more probable
alternative.
As discussed at the end of the section entitled Probability Ratio Tests, above, the SPRT
allows for unequal prior stimulus probabilities by biasing the thresholds or, equivalently,
biasing the initial condition taken in the drift diffusion model (cf. Equation A40). We consider
this only for the (optimal) drift diffusion model (see Equations A27–A29). As above, we denote
the probability of stimuli S1 and S2 (drifts ⫾ A) as ⌸ and 1 ⫺ ⌸, respectively, and the ERs and
mean DTs as ER(⫾) and DT(⫾). The net ER and mean DT are then given by
NER ⫽ ⌸ER共⫹兲 ⫹ 共1 ⫺ ⌸兲ER共⫺兲, and NDT ⫽ ⌸DT共⫹兲 ⫹ 共1 ⫺ ⌸兲 DT共⫺兲. (A42)
We note that ERs and DTs for biased initial data, as given in Equations A30 –A31, apply to the case
A ⬎ 0 but that the reflection transformation x 哫 ⫺x takes sample paths of Equation A29 with A ⬎
0 to those for A ⬍ 0; we may thus simply substitute ⫺x0 for x0 in those expressions to obtain ER(⫺)
and DT(⫺), which yields
NER ⫽
1
1⫹e 2z̃ã ⫺ 再
1 ⫺ ⌸e⫺2x0ã ⫺ (1 ⫺ ⌸)e2x0ã
e2z̃ã ⫺ e⫺2z̃ã
, 冎 and (A43)
NDT ⫽ z̃ tanh z̃ã ⫹ 2z̃ 再 1 ⫺ ⌸e⫺2x0ã ⫺ (1 ⫺ ⌸)e2x0ã

e2z̃ã ⫺ e⫺2z̃ã 冎
⫹ 共1 ⫺ 2⌸兲x0 . (A44)
Substitution of the appropriate (optimal) biased initial condition (see Equation A40), which may be
written in the following form in the scaled variables,
x0 ⫽
y共0兲
A
⫽
1
2ã
log
⌸
1⫺⌸冉冊 N e2x0ã ⫽
⌸
1⫺⌸
, (A45)
then gives the following expressions for net ER and DT:
1
NER ⬅ , and (A46)
1 ⫹ e2z̃ã
NDT ⫽ z̃ tanh z̃ã ⫹

共1 ⫺ 2⌸兲
2ã
log
⌸
1⫺⌸ 冉
. 冊 (A47)
It is interesting to note that for fixed z̃ and ã, the ER takes the same value for all 0 ⱕ ⌸ ⱕ 1
and that DTs decrease symmetrically from the maximum at ⌸ ⫽ 1/2 to zero at ⌸ ⫽ 1/(1 ⫹
e⫾2z̃ã), for which values the initial condition falls on a threshold. Moreover, the biased initial
condition given by Equation A45 also maximizes the RR. Modifying the definition of Equation
A26 to read
1 ⫺ NER
NRR ⫽ , (A48)
NDT ⫹ D ⫹ T0 ⫹ DpNER
a lengthy calculation reveals that
⳵ ⳵
共NRR兲 ⫽ 共NRR兲 ⫽ 0, (A49)
⳵z̃ ⳵x0
with x0 given by Equation A45 and z̃ by the solution z̃o of
e2z̃oã ⫺ 1 ⫽ 2ã共Dtotal ⫺ z̃o兲 ⫹ 共1 ⫺ 2⌸兲log 冉 ⌸

1⫺⌸
. 冊 (A50)
We have checked (numerically) that this critical point is indeed a maximum. Hence, noting that
the final term in Equation A50 is even about ⌸ ⫽ 1/2 and strictly negative for ⌸ ⫽ 1/2, this
formula shows explicitly how optimal thresholds are lowered in choice tasks with stimuli of
unequal salience. Moreover, as pointed out in the section entitled Biased Probabilities and the
Optimal Decision Threshold in the main text, we may compute a relationship among the
salience ⌸, the total delay Dtotal , and the signal-to-noise ratio ã at which the optimal bias and
optimal threshold coincide. Setting xo ⫽ z̃o and replacing them in Equation A50 by means of
Equation A45, we obtain
ã ⫽
冋2⌸ ⫺ 1
1⫺⌸
⫹ 2⌸ log
⌸
1⫺⌸ 冉冊册 . (A51)
2Dtotal
Note that, as expected, Equation A51 gives ã ⫽ 0 for ⌸ ⫽ 1/2, and ã increases as ⌸ increases from
1/2 or Dtotal decreases with ⌸ ⫽ 1/2. For ã less than the value given by Equation A51, xo ⬎ z̃o ,
which implies that the more salient stimulus should be selected immediately upon presentation.
The O-U Model
In this section, we present explicit expressions for ERs, DTs, and RRs for the O-U model.
Additionally, we develop asymptotic approximations to these complicated expressions that are valid
for large thresholds and/or for small deviations ␭ from the drift diffusion model. These latter
approximations determine just how suboptimal RRs will become for systems that are not perfectly
tuned to achieve constant drift (e.g., for a mutual inhibition model with slightly imbalanced decay
and inhibition). We also use our explicit formula for RRs to provide arguments independent of the
SPRT to show that optimal RRs are achieved for the drift diffusion processes. In this section, we
restrict to equal salience stimuli ⌸ ⫽ 1/2 and unbiased initial data x0 ⫽ 0.
Consider the O-U process
dy ⫽ 共␭y ⫹ A兲dt ⫹ cdW, (A52)

with thresholds ay ⫽ ⫺z, by ⫽ z. Letting
A
y⫽⫺ ⫹ Ax
␭
and using z̃, ã as defined in Equation A28,
1
dx ⫽ ␭xdt ⫹ dW, (A53)
冑ã
with thresholds a ⫽ ⫺z̃ ⫹ 1/␭, b ⫽ z̃ ⫹ 1/␭.
ERs, Mean DTs, and Reward Rates

Suppose we have an unbiased initial condition, that is, y0 ⫽ 0 f x0 ⫽ 1/␭. Then, where
erf共z兲 ⫽
2
冑␲ 冕 z
0
e⫺t dt
2
denotes the error function,
erfc共z兲 ⫽
2
冑␲ 冕 z
⬁
e⫺t dt ⫽ 1 ⫺ erf共z兲
2
is the complementary error function (used below), and
D共z兲 ⫽ e⫺z
2
冕 z
0
2
et dt, (A54)
which is known as Dawson’s integral, it may be shown that (Moehlis et al., 2004)
冉冑
erf
ã
␭
冊冉冑冊
(1 ⫹ ␭z̃) ⫺ erf
ã
␭
erfc⵺
erfc共⫹兲
def
⫺1
冉冑冊冉冑冊
ER ⫽ ⫽ , (A55)
ã ã erfc共⫺兲
erf (1 ⫹ ␭z̃) ⫺ erf (1 ⫺ ␭z̃) ⫺1
␭ ␭ erfc共⫹兲
DT ⫽
冋冉冑
␭ erf
ã
␭
(1 ⫹ ␭z̃) ⫺ erf冊冉冑
2
冊册 ã
␭
(1 ⫺ ␭z̃)
⫻
冦冋冉冑冊冉冑
erf
ã
␭
⫺ erf
ã
␭
(1 ⫺ ␭z̃) 冊册
⫻ 冕冑 ã/␭共1⫹␭z̃兲
冑 ã/␭
冋冉冑冊冉冑
D(z)dz⫹ erf
ã
␭
⫺ erf
ã
␭
(1 ⫹ ␭z̃) 冊册冕冑 ã/␭
冑 ã/␭共1⫺␭z̃兲
D(z)dz
冧
冋册冋册
冦冧
erfc(⫺) erfc⵺ erfc⵺
⫺ D ⫺ ⫺ 1 D2
2 erfc(⫹) erfc(⫹) 1 erfc(⫹)
⫽ , and (A56)
␭ erfc(⫺)
⫺1
erfc(⫹)
冉冑冊冉冑冊
冢冣
ã ã
冉冊
erf (1 ⫹ ␭z̃) ⫺ erf
1 2 2 ␭ ␭
冉冑冊冉冑冊
⫽ D ⫹ T0 ⫹ D1 ⫹ D ⫹ T0 ⫹ Dp ⫺ D2
RR ␭ ␭ ã ã
erf ⫺ erf (1 ⫺ ␭z̃)
␭ ␭
冦冧
erfc⵺
冉冊
⫺1
2 2 erfc(⫹)
⫽ D ⫹ T0 ⫹ D ⫹ D ⫹ T0 ⫹ Dp ⫺ D2 . (A57)
␭ 1 ␭ erfc(⫺) erfc⵺
⫺
erfc(⫹) erfc(⫹)
Here,
erfc⵺ ⫽ erfc 冉冑冊 ã
␭
, erfc共⫹兲 ⫽ erfc 冉冑 ã
␭
(1 ⫹ ␭z̃) , 冊 erfc共⫺兲 ⫽ erfc 冉冑 ã
␭
冊
(1 ⫺ ␭z̃) ,
D1 ⫽ 冕冑 ã/␭共1⫹␭z̃兲
冑 ã/␭
D共z兲dz , D2 ⫽ 冕冑 ã/␭
冑 ã/␭共1⫺␭z̃兲
D共z兲dz.
Note that these formulae apply to both positive and negative ␭, via the definition of the error
function with imaginary argument:
2
冑␲ 冕0
z
et dt ⫽ erfi共z兲 ⫽ i erf共⫺iz兲.
2
Asymptotic Approximations
ER for large thresholds and ␭ ⬎ 0. If ␭ ⬎ 0, as the threshold z̃ ⬃ z is taken to infinity in
Equation A55, we have
lim ER ⫽
z̃3⬁
1
2 冋 1 ⫺ erf 冉冑冊册ã
␭
; ␭ ⬎ 0, (A58)
which only approaches zero as ␭ 3 0⫹. Therefore, in contrast to the constant drift (␭ ⫽ 0) case of
Equation A30, the ER always remains strictly positive if ␭ ⬎ 0, even as DTs increase to infinity.
ER for large thresholds and ␭ ⬍ 0. If ␭ ⬍ 0, we have
lim ER ⫽ 0; ␭ ⬍ 0. (A59)
z̃3⬁
Therefore, zero ERs are (asymptotically) possible for negative ␭ even when it is nonzero, in contrast
to the interrogation protocol, in which case zero ERs occur only when ␭ ⫽ 0 (see Equation A94,
below). Note, however, that as z̃ 3 ⬁ with ␭ ⬍ 0, DTs rapidly increase to infinity because the
probability density of solutions is exponentially confined within the thresholds, suggesting that such
a strategy is irrelevant to behavior.
ER, DT, and RR for small ␭. Equipped with the exact expressions (Equations A55–A57), in
principle we can numerically calculate ER, DT, and 1/RR for any values of the parameters z̃,
ã, ␭, D, T0, and Dp. However, the evaluation of the integrals in the formulae becomes difficult
for small ␭, which turns out to be the region of greatest interest. Also, it is difficult from the
exact formulae to see how ER, DT, and 1/RR vary with the parameters. To gain insight into
these issues, we now expand the expressions for small ␭. Note that we treat z̃ as an O(1) quantity
to obtain the following expansions.
First, we recall the expansion of the complementary error function for large z (Equation 7.1.23 of
Abramowitz & Stegun, 1984):
e⫺z
冉冊
2
1 3
erfc共z兲 ⫽ 1⫺ 2 ⫹ ⫹··· .
冑␲z 2z 4z4
This gives
冉 ␭ 3␭ 2
冊
冉冊
1⫺ 2 ⫹ ⫹···
erfc共⫺兲 1 ⫹ ␭z̃ 2ã(1 ⫺ ␭z̃) 4ã (1 ⫺ ␭z̃) 4
2
冉冊
⫽ e4z̃ã
erfc共⫹兲 1 ⫺ ␭z̃ ␭ 3␭ 2
1⫺ ⫹ ⫹···
2ã(1 ⫹ ␭z̃) 2 4ã2 (1 ⫹ ␭z̃) 4
冉
⫽ e4z̃ã 1 ⫹ 2z̃␭ ⫹
2z̃(z̃ã ⫺ 1) 2
ã 冊
␭ ⫹ O共␭ 3 兲, and (A60)
erfc⵺ 冉 ␭
1⫺
3␭ 2
⫹ 2⫹···
2ã 4ã 冊
冉冊
⫽ e2z̃ã⫹z̃ ã␭ (1 ⫹ ␭z̃)
2
erfc共⫹兲 ␭ 3␭ 2
1⫺ 2 ⫹ ⫹···
2ã(1 ⫹ ␭z̃) 4ã 共1 ⫹ ␭z̃)4
2
冉
⫽ e2z̃ã 1 ⫹ z̃(1 ⫹ z̃ã)␭ ⫹
z̃ 3 3
2ã 冊
(z̃ ã ⫹ 2z̃2 ã2 ⫺ 2)␭ 2 ⫹ O共␭ 3 兲. (A61)
We also use the following expansion for Dawson’s integral from Spanier and Oldham’s (1987)
Equation 42:6:6, valid for large z:
1 1 3
D共z兲 ⫽ ⫹ 3⫹ 5⫹··· .
2z 4z 8z
This implies that
z̃ z̃共1 ⫺ z̃ã兲 2 z̃共9共1 ⫺ z̃ã兲 ⫹ 4z̃2 ã2 兲 3

D1 ⫽ ␭⫹ ␭ ⫹ ␭ ⫹ O共␭ 4 兲, and (A62)
2 4ã 24ã2
z̃ z̃共1 ⫹ z̃ã兲 2 z̃共9共1 ⫹ z̃ã兲 ⫹ 4z̃2 ã2 兲 3

D2 ⫽ ␭⫹ ␭ ⫹ ␭ ⫹ O共␭ 4 兲. (A63)
2 4ã 24ã2
Using these formulae in Equations A55, A56, and A57 gives the approximations
ER ⫽ ER 0 ⫹ ␭ER 1 ⫹ ␭ 2 ER 2 ⫹ O共␭ 3 兲, (A64)

where
1
ER 0 ⫽ ,
1 ⫹ e2z̃ã
z̃e2z̃ã
ER 1 ⫽ 共z̃ã ⫺ tanh共z̃ã兲兲,
e ⫺1
4z̃ã
ER 2 ⫽ z̃e2z̃ã共⫺4z̃ãe2z̃ã共⫺1 ⫹ e2z̃ã兲 ⫹ 2共⫺1 ⫹ e2z̃ã兲 2 共1 ⫹ e2z̃ã兲 ⫹ z̃3 ã3 共⫺1 ⫹ e2z̃ã 兲共1 ⫹ e2z̃ã兲 2
⫺ 2z̃2 ã2 共1 ⫹ e2z̃ã ⫹ e4z̃ã ⫹ e6z̃ã兲兲/共2ã共⫺1 ⫹ e2z̃ã兲 2 共1 ⫹ e2z̃ã兲 3 兲, and
DT ⫽ DT 0 ⫹ ␭DT 1 ⫹ ␭ 2 DT 2 ⫹ O共␭ 3 兲, (A65)

where
DT 0 ⫽ z̃ tanh共z̃ã兲,
DT 1 ⫽ z̃ 再 tanh(z̃ã) ⫺ z̃ã
2ã
2z̃e2z̃ã
⫹ 4z̃ã
e ⫺1
[e2z̃ã(1 ⫺ tanh(z̃ã)) ⫺ (1 ⫹ z̃ã)] , 冎
DT 2 ⫽ z̃共⫺12z̃4 ã4 e2z̃ã共⫺1 ⫹ e2z̃ã兲共1 ⫹ e2z̃ã兲 2 ⫹ 9共⫺1 ⫹ e2z̃ã兲 3 共1 ⫹ e2z̃ã兲 2
⫹ 4z̃2 ã2 共⫺1 ⫹ e2z̃ã兲 3 共1 ⫺ e2z̃ã ⫹ e4z̃ã兲 ⫹ 24z̃3 ã3 e2z̃ã共1 ⫹ e2z̃ã ⫹ e4z̃ã ⫹ e6z̃ã兲
⫺ 3z̃ã共⫺1 ⫹ e2z̃ã兲 2 共3 ⫹ 13e2z̃ã ⫹ 13e4z̃ã ⫹ 3e6z̃ã兲)/共12ã2 共⫺1 ⫹ e2z̃ã兲 2 共1 ⫹ e2z̃ã兲 3 兲, and
1
⫽ A0 ⫹ ␭A1 ⫹ ␭ 2 A2 ⫹ O共␭ 3 兲, (A66)
RR
where
A0 ⫽ z̃ ⫹ D ⫹ T0 ⫹ 共D ⫹ T0 ⫹ Dp ⫺ z̃兲e⫺2z̃ã,
A1 ⫽ z̃ 冋 (1 ⫺ z̃ã)(1 ⫺ e⫺2z̃ã)
2ã
e2z̃ã ⫹ 1
⫺ Dtotale⫺2z̃ã ⫹ (Dtotal ⫺ z̃)z̃ãe⫺2z̃ã 2z̃ã
e ⫺1 冉冊册 , and
A2 ⫽ z̃共⫺6z̃4 ã4 共1 ⫹ e2z̃ã兲 2 ⫺ 3z̃ã共⫺1 ⫹ e2z̃ã兲共⫺5 ⫹ 2e2z̃ã ⫹ 3e4z̃ã ⫹ 4ãDtotal兲
⫹ 3共⫺1 ⫹ e2z̃ã兲 2 共3共⫺1 ⫹ e2z̃ã兲 ⫹ 4ãDtotal兲 ⫹ 6z̃3 ã3 共⫺1 ⫹ 4e2z̃ã ⫹ e4z̃ã ⫹ ã共1 ⫹ e2z̃ã兲 2 Dtotal兲
⫺ 4z̃2 ã2 共⫺共⫺1 ⫹ e2z̃ã兲 3 ⫹ 3ã共⫺1 ⫹ 2e2z̃ã ⫹ e4z̃ã兲Dtotal兲)/共12ã2 e2z̃ã共⫺1 ⫹ e2z̃ã兲 2 兲.

Here, the O(␭2) expressions were obtained using Mathematica. Notice that for ␭ ⫽ 0, Equations A64,
A65, and A66 are identical to Equations A30, A31, and A32. We verify the validity of these approxi-
mations in Figure A1b.
RR for small ␭ and large thresholds. It is also instructive to find approximations for 1/RR for
small and large z̃; to obtain these, we start with the expression for 1/RR valid to O(␭2), then consider
the limit in z̃. Using an analogue to Equation A36, we find that for small z̃,
1
⬇ 2共D ⫹ T0 兲 ⫹ Dp ⫺ 2ã共D ⫹ T0 ⫹ Dp兲z̃, (A67)
RR
as for the drift diffusion model (cf. Equation A35). For large z̃,
1
RR
⬇ D ⫹ z̃ ⫹ ␭
z̃
2 冉冊
1
ã
⫺ z̃ ⫹ ␭ 2 z̃ 冉
9(1 ⫺ z̃ã) ⫹ 4z̃2 ã2
12ã2 冊 3
␭ 2 z̃3
for z̃ 3 ⬁. 共␭ ⫽ 0兲. (A68)
3
The validity of these results is examined in Figure A1d.
Optimizing Reward Rates for Unbiased Data

We now show that (z̃, ␭) ⫽ (z̃o , 0), where z̃o is found from Equation A33, minimizes 1/RR, that
is, maximizes RR. First, using Equation A33, we have
⳵
冉冊冏
1
⳵z̃ RR 共z̃,␭兲⫽共z̃o,0兲
⫽
⳵A0
⳵z̃
冏 z̃⫽z̃o
⫽ 0. (A69)
Now, from Equation A66,
⳵
⳵␭ 冉冊冏
1
RR 共z̃,␭兲⫽共z̃o,0兲
⫽ A1 兩 z̃⫽z̃o
⫽ z̃oe⫺2z̃oã 冋 (1 ⫺ z̃oã)(e2z̃oã ⫺ 1)
2ã
e2z̃oã ⫹ 1
⫺ Dtotal ⫹ (Dtotal ⫺ z̃o)z̃oã 2z̃oã
e ⫺1 冉冊册 . (A70)
Solving Equation A33 for e2z̃oã and substituting into Equation A70, we find that all terms cancel, that
is,
⳵
⳵␭ 冉冊冏1
⫽ 0. (A71)
The point (z̃, ␭) ⫽ (z̃o , 0) is called a stationary point because conditions in Equations A69 and A71
are satisfied.
The condition for the stationary point (z̃, ␭) ⫽ (z̃o , 0) to be a local optimum involves the Hessian of
1/RR evaluated at the stationary point, that is,
冉冊冏冉冊冏
冢冣
⳵2 1 ⳵2 1
⳵␭ RR
2
共z̃,␭兲⫽共z̃o,0兲
⳵z̃⳵␭ RR 共z̃,␭兲⫽共z̃o,0兲
冉冊冏冉冊冏
H⬅ ⳵2 1 ⳵2 1 . (A72)
⳵z̃⳵␭ RR 共z̃,␭兲⫽共z̃o,0兲
⳵z̃2 RR 共z̃,␭兲⫽共z̃o,0兲
The stationary point is a local minimizer (maximizer, respectively) of 1/RR if H is positive

(negative, respectively) definite, that is, if H has two positive (two negative, respectively) eigen-
values. If H has one positive and one negative eigenvalue, then the stationary point is called a saddle
point, and it is neither a local minimizer nor a local maximizer.
From Equation A34,
⳵2 1
⳵z̃2 RR 冉冊冏共z̃,␭兲⫽共z̃o,0兲
⫽
⳵ 2 A0
⳵z̃2
冏 z̃⫽z̃o
⫽ 2ãe⫺2z̃oã关1 ⫹ e2z̃oã兴 ⬎ 0. (A73)
Using this, we conclude that if det(H) ⬎ 0, then H is positive definite. Using Equation A33 to
eliminate Dtotal ,
⳵2
⳵␭ 2 冉冊冏
1
⫽ 2A2 兩 z̃⫽z̃o
z̃o关15共e2z̃oã ⫺ 1兲 2 ⫺ 9z̃oã共e4z̃oã ⫺ 1兲 ⫹ 3z̃3o ã3 共e4z̃oã ⫺ 1兲 ⫺ 2z̃2o ã2 共1 ⫹ 10e2z̃oã ⫹ e4z̃oã兲兴

⫽ , and
6ã2 e2z̃oã共e2z̃oã ⫺ 1兲
⳵2
⳵z̃⳵␭ 冉冊冏1
⫽
⳵A1
⳵z̃
冏 z̃⫽z̃o
⫽
z̃o共1 ⫹ e2z̃oã兲共1 ⫺ e2z̃oã ⫹ z̃oã共1 ⫹ e2z̃oã兲兲
e2z̃oã共1 ⫺ e2z̃oã兲
.
From these expressions, after simplifying, the condition that det(H) ⬎ 0 becomes
0 ⬍ det共H兲 ⫽ z̃o共1 ⫹ e2z̃oã兲共15共e2z̃oã ⫺ 1兲 3 ⫹ 4z̃2o ã2 共e2z̃oã ⫺ 1兲 3
⫺ 12z̃3o ã3 e2z̃oã共1 ⫹ e2z̃oã兲 ⫺ 12z̃oã共e2z̃oã ⫺ 1兲 2 共1 ⫹ e2z̃oã兲)/共3ãe4z̃oã共e2z̃oã ⫺ 1兲 2 兲.

(A74)
For example, for ã ⫽ 1, D ⫽ 10, Dp ⫽ 20, and T0 ⫽ 0, we find that z̃o ⫽ 2.02115 and that
冉
3.9253 ⫺2.2487
H ⫽ ⫺2.2487 2.0351 , 冊 det共H兲 ⫽ 2.9317 ⬎ 0;
thus, (z̃, ␭) ⫽ (z̃o , 0) is a local minimizer of 1/RR, that is, a local maximizer of RR. For reference,
the eigenvalues of H for these parameters are 5.4194 and 0.5410. Although we have not been able
to prove that det(H) is positive for all possible parameter values, it is found numerically to be true
for all choices considered (Moehlis et al., 2004).
We can interpret these results as follows: Given ␭, it is possible to choose z̃o(␭)—say, by choosing
the threshold z—to maximize the RR; however, it is even better to first adjust so that ␭ ⫽ 0, then
choose z̃ to maximize RR. This is illustrated in Figure A1e, which also emphasizes that care must
be used in applying the O(␭) formulae.
Dependence of the Optimal Threshold on Experimental Delays

In this section, we show that the dependence of the optimal threshold on D ⫹ Dp (rather than D
and Dp separately) is true for any decision maker. Consider a decision maker whose ER(z) and RT(z)
are functions of the decision threshold z. As for the previous decision models, for the optimal
threshold, the derivative of 1/RR must be equal to zero. Using elementary calculations, we compute
the derivative
⳵
冉冊
1
⳵z RR
⫽
RT⬘共z兲 ⫹ ER⬘共z兲共D ⫹ Dp兲 ⫺ RT⬘共z兲 ER共z兲 ⫹ RT共z兲 ER⬘共z兲
共1 ⫺ ER共z兲兲 2
.
Note that the above derivative does not depend on D and Dp separately but depends only on D ⫹
Dp. Therefore, the optimal threshold must also depend only on D ⫹ Dp for any decision maker.
Threshold Resulting From Learning
In this subsection, we consider the case in which a small error of ε is made in estimation of the
optimal threshold z̃o. We show that overestimation of the threshold results in higher RRs than
underestimation of the threshold, that is,
RR共z̃o ⫹ ε兲 ⫺ RR共z̃o ⫺ ε兲 ⬎ 0. (A75)

From Taylor expansion, we obtain
RR⬙共z̃o兲 2 RR⵮共z̃o兲 3
RR共z̃o ⫹ ε兲 ⫽ RR共z̃o兲 ⫹ RR⬘共z̃o兲ε ⫹ ε ⫹ ε ⫹ O共4兲 ⫹ O共5兲, and
2 6
RR⬙共z̃o兲 2 RR⵮共z̃o兲 3
RR共z̃o ⫺ ε兲 ⫽ RR共z̃o兲 ⫺ RR⬘共z̃o兲ε ⫹ ε ⫺ ε ⫹ O共4兲 ⫺ O共5兲.
2 6
Because z̃o is the threshold maximizing RR, RR⬘(z̃o) ⫽ 0; hence,
RR⵮共z̃o兲 3
RR共z̃o ⫹ ε兲 ⫺ RR共z̃o ⫺ ε兲 ⫽ ε ⫹ O共5兲.
3
To prove Equation A75, it therefore suffices to show that RR⵮(z̃o) ⬎ 0. In the beginning of this
section, we noticed that instead of considering RR, it is more convenient to consider its inverse,
1/RR; let us denote it by f ⫽ 1/RR. Using the chain rule and the fact that f ⬘(z̃o) ⫽ 0, we compute
that
RR⵮共z̃o兲 ⫽
⳵3
冉冊
1
⳵z̃ f 共z̃o兲
3 ⫽⫺
f ⵮共z̃o兲
共f 共z̃o兲兲 2
.
Because the denominator of the above equation is positive, for small ε the inequality given by
Equation A75 is equivalent to f ⵮(z̃o) ⬍ 0. Elementary calculations show that
f ⵮共z̃o兲 ⫽ ⫺4ã2 e⫺2z̃oã共3 ⫹ 2ã共Dt ⫺ z̃o兲兲,

and substituting the condition for the optimal threshold (see Equation A33) into this equation, we
obtain
f ⵮共z̃o兲 ⫽ ⫺4ã2 e⫺2z̃oã共2 ⫹ e2z̃oã兲 ⬍ 0,

which proves Equation A75.
Other Reward Criteria
Here, we consider three further reward criteria discussed in the main text, deriving optimal
thresholds and optimal performance curves for each. We assume equal stimulus salience (⌸ ⫽ 1/2)
and unbiased initial conditions (x0 ⫽ 0).
Bayes Risk
The first criterion is the BR given by Equation 41 in the main text, which is a special case of the
criteria considered by Edwards (1965), who also obtained the following two equations. Because ER
decreases exponentially and DT grows faster than linearly with z̃, BR has a unique critical point, a
minimum. The optimal threshold z̃o (that minimizes BR for given ã, Dtotal , c1, and c2) is given by
substituting Equations A30 –A31 with x0 ⫽ 0 into Equation 41, differentiating with respect to z̃, and
setting the result equal to zero, which yields
e2z̃oã ⫺ e⫺2z̃oã c2
⫹ 2z̃o ⫽ def
⫽ q. (A76)
2ã c1
(As noted in the main text, the weight ratio q must have the units of time for consistency.)
Substituting for z̃ ⫽ z̃o and ã in terms of ER and RT from Equations 55–56, we then obtain the
following family of optimal performance curves:
DT
1 ⫺ ER
共1 ⫺ 2ER兲log
ER 冉冊
冉冊
⫽ . (A77)
q 1 ⫺ ER 1 1
2 log ⫺ ⫹
ER 1 ⫺ ER ER
Clearly, we require c2(q) strictly positive. The maximum of this optimal performance curve always
lies to the left of the corresponding maximum for RR (cf. Figure 13 in the main text).
The Reward/Accuracy Function
Next, we consider the RA function given by Equation 43 where we assume Dp ⫽ 0. Repeating

the above procedure and differentiating with respect to z̃ to find the optimal threshold, we obtain a
quotient whose denominator never vanishes and whose numerator vanishes if z̃ ⫽ z̃o with
关1 ⫺ e⫺2z̃oã ⫹ 2ã共Dtotal ⫺ z̃o兲e⫺2z̃oã兴共e2z̃oã ⫹ 1兲 2 c2 q

⫺2z̃oã ⫺2z̃oã 2 ⫽ def
⫽ . (A78)
2ãe 关z̃o共1 ⫺ e
2z̃oã
兲 ⫹ Dtotal共1 ⫹ e 兲兴 c1 Dtotal Dtotal
Because RR has a unique critical point (a maximum) and ER is monotonically decreasing as a
function of z̃, satisfaction of Equation A78 yields the unique maximum of RA for given set task
parameters. Substituting for z̃ ⫽ z̃o and ã, after some algebra, we obtain
1 ⫺ ER log 冉
1 ⫺ ER Dtotal
ER DT
⫺冊冉 1
1 ⫺ 2ER DT 冊
冉冊冉冊
⫽q . (A79)
1 ⫺ ER Dtotal 2
Dtotal
ER log 1⫹
ER DT
This quadratic equation in DT/Dtotal may be solved to give a family of optimal performance curves
parameterized by q, the relative weight of errors:
DT E ⫺ 2q ⫺ 冑E2 ⫺ 4q共E ⫹ 1兲
⫽ , (A80)
Dtotal 2q
where
冦冧
1 1
冉冊
E⫽ ⫹ . (A81)
1 ⫺ ER 1 ⫺ 2ER
ER log
ER
(See Figure 13b in the main text.) Observe that E is precisely the ER expression that appears in
Equation 58 for RR; indeed, as q 3 0 the right-hand side of Equation A80 converges on Equation
58, confirming that the negative square root is the appropriate choice. Moreover, computing the
derivative of the right-hand side of Equation A80 with respect to ER, we find that the peak in the
optimal performance curves falls at the same ER regardless of the value of the weight ratio q. This
value corresponds to
冤冉 1 ⫺ ER
冊冥 2
冉冊
ER log
dE 1 ⫺ ER ER 1
⫽ 0 f log ⫺2 ⫽ f ER ⬇ 0.174. (A82)
dER ER 1 ⫺ 2ER 1 ⫺ ER
The Modified Reward Rate
Finally, we consider RRm given by Equation 44, where we again set Dp ⫽ 0. Again substituting
for ER and RT and rearranging, we obtain
1 z̃共1 ⫺ e⫺2z̃ã兲 ⫹ Dtotal共1 ⫹ e⫺2z̃ã兲

⫽ ; (A83)
RR m c2
1 ⫺ e⫺2z̃ã
c1
differentiation with respect to z̃ then leads to the following condition for a critical point of RRm.
e2z̃oã ⫺ 1 ⫺ 2ã共Dtotal ⫺ z̃o兲 c2

⫽ def
q. (A84)
1 ⫺ e⫺2z̃oã ⫹ 2ã共Dtotal ⫹ z̃o兲 c1 ⫽
(Exactly as for RA, that this corresponds to a maximum of RRm follows from the fact that it is the
sum of a function with a single maximum and a function decreasing in z̃.) Finally, substituting for
z̃ ⫽ z̃o and ã, we obtain a third family of optimal performance curves:
冦冧
⫺1
1 q
⫺
DT ER 1 ⫺ ER 1⫺q
冉冊
⫽ 共1 ⫹ q兲 ⫹ . (A85)
Dtotal 1 ⫺ ER 1 ⫺ 2ER
log
ER
For q ⫽ 0 (c2 ⫽ 0, c1 ⫽ 0), this reduces to Equation 58 as expected.
Optimal Decisions Under the Interrogation Protocol
We now suppose that participants are interrogated at a fixed time after stimulus onset and are
required to respond as soon as possible after interrogation. The appropriate model is again a drift
diffusion model, but now, sample paths are assumed to evolve until interrogation, at which instant,
we interpret the probability of responses R1 and R2 by asking if a given sample path is closer to the
threshold at y ⫽ z or the threshold at y ⫽ ⫺z, respectively.
If no limits are imposed on the sample paths (i.e., they are allowed to evolve freely without
bound), this is the continuum analog of the (optimal) Neyman–Pearson test. This case is considered
in this section, in which the formula (Equation A92) for the ER as a function of model parameters
and interrogation time is developed. This formula demonstrates the qualitative difference between
long-time interrogations of the drift diffusion model, which yield vanishingly few errors, and
long-time interrogations of the O-U model, which always give nonzero ERs. We then explicitly
demonstrate that the drift diffusion model yields optimal ERs for all possible interrogation times and
model parameters (see the section entitled Highest Accuracy for Fixed DTs: The Interrogation
Paradigm, in main text).
Specifically, we evaluate the integrals of the probability distribution of solutions p(x, t) of the
forward Kolmogorov or Fokker–Planck equation for solutions to the appropriate stochastic differ-
ential equation between ⫺⬁ and 0 and 0 and ⫹⬁, respectively to evaluate the expected percentage-
correct rates and ERs. Note that a sample path may cross and recross either threshold multiple times
or cross neither during the interval before interrogation.
We again consider the O-U model of Equation A52:
dy ⫽ 共␭y ⫹ A兲dt ⫹ cdW. (A86)
Including the limit ␭ 3 0, this encompasses both the cases treated above. As shown in Section 3.2
of Busemeyer and Townsend (1992), the probability p(y, t)dy that a solution of Equation A86
occupies a point in [y, y ⫹ dy] at time t is
p共y, t兲 ⫽
1
冑2␲␯ 共t兲 2 冋
exp
⫺(y ⫺ ␮(t)) 2
2␯ 2 (t)
, 册 (A87)
with
A ␭t c2 2␭t
␮共t兲 ⫽ 共e ⫺ 1兲 ⫹ y0 e␭t, and ␯ 2 共t兲 ⫽ 共e ⫺ 1兲, (A88)
␭ 2␭
where we have assumed the (general, possibly biased) initial condition p(y, 0) ⫽ ␦(y ⫺ y0)
corresponding to starting all paths of Equation A86 at y(0) ⫽ y0. Note that the expressions of
Equation A88 for mean and variance hold for both positive and negative ␭ and that as ␭ 3 0, they
approach the corresponding expressions for the drift diffusion model:
␮共t兲兩 ␭⫽0 ⫽ At ⫹ y0 , and ␯ 2 共t兲兩 ␭⫽0 ⫽ c2 t. (A89)

Assuming that the upper threshold y ⫽ z represents the correct alternative, the probabilities of
correct and incorrect choices being made upon interrogation at time t are therefore
P共correct兲 ⫽ 1 ⫺ ER ⫽ 冕 0
⬁
p共y, t兲dy, and P共incorrect兲 ⫽ ER ⫽ 冕
⫺⬁
0
p共y, t兲dy.
(A90)
Evaluating these integrals, we obtain (cf. Section 3.2 of Busemeyer and Townsend, 1992)
P共correct兲 ⫽ 冕 ⬁
⫺共␮/ 冑 2␯ 2 兲
1
冑␲
e⫺u du ⫽
2
1
2 冋 1 ⫹ erf 冉冑冊册 ␮
2␯ 2
, and (A91)
P共incorrect兲 ⫽ ER ⫽
1
2 冋 1 ⫺ erf 冉冑冊册␮
2␯ 2
. (A92)
Inserting the appropriate expressions for ␮(t) and ␯2(t) into Equation A92, we see an important
qualitative difference between the drift diffusion and O-U models for asymptotically large values of
the interrogation time t. For the drift diffusion model (Equation A89),
lim ER ⫽ 0, (A93)
t3⬁
whereas for the O-U model (Equation A88),
lim ER ⫽ 1 ⫺ erf
t3⬁
冋冉冑冊册 A
c 兩␭兩
⬎ 0. (A94)
Thus, as discussed in the section entitled Interrogation Paradigm in the main text, the O-U model
gives nonzero ERs for infinite-time interrogations, whereas the drift diffusion model gives perfect
performance in this case.
Minimizing ER for Unbiased Choices

We first consider the case of unbiased initial data, appropriate to choice tasks with alternatives of
equal probability. Using Equation A88 with y0 ⫽ 0, the argument of the error functions in Equations
A91–A92 takes the form
␮
冑2␯ 2
def
⫽ f 共␭; A, c兲 ⫽
A
c 冑共e␭t ⫺ 1兲
␭共e␭t ⫹ 1兲
. (A95)
Because erf⵺ is a monotonically increasing function of its argument on (⫺⬁, ⬁), the probability of
making a correct decision will be maximized (and the ER minimized) by selecting the maximum
admissible value of this expression. The appropriate constraints are to fix A and t and maximize
Equation A95 over ␭ 僆 (⫺⬁, ⫹⬁). We claim that the unique (global) maximum is achieved for ␭ ⫽ 0:
lim f 共␭; A, c兲 ⫽
␭30
A
c 冑 t
2
. (A96)
To prove this, it suffices to show that ⳵f /⳵␭ ⫽ 0 at ␭ ⫽ 0 and that ⳵f /⳵␭ is strictly positive (negative,
respectively) for ␭ ⬍ 0 (␭ ⬎ 0, respectively). We compute
⳵f
⫽
⳵␭ 2c
A
冑 ␭共e␭t ⫹ 1兲
e␭t ⫺ 1 冋 2␭te␭t ⫺ e2␭t ⫹ 1
␭ 2 共e␭t ⫹ 1兲 2 册 def
⫽
A
2c
F共␭, t兲共2␭te␭t ⫺ e2␭t ⫹ 1兲, (A97)
where F(␭, t) is a bounded positive quantity for ␭ ⫽ 0, t ⬎ 0, and the expression as a whole vanishes
(⬃ ⫺␭t5/2) at ␭ ⫽ 0. The final quantity in parentheses in Equation A97 is positive (negative,
respectively) for ␭ ⬍ 0 (␭ ⬎ 0, respectively), as required. We conclude that the optimal strategy is
to set ␭ ⫽ 0.
Minimizing ER for Biased Choices

We now repeat the calculations allowing biased initial data and unequal prior probabilities for the
two alternatives. As in our review of SPRT in the first section of this appendix, let ⌸ denote the
probability of a stimulus corresponding to the upper threshold, with drift ⫹A, and let 1 ⫺ ⌸
correspond to ⫺A. Letting ER(⫾) ⫽ ER(⫾A, ␭, c, x0) denote the ERs in the two cases, the net ER
(NER) is
NER ⫽ ⌸ER共⫹兲 ⫹ 共1 ⫺ ⌸兲 ER共⫺兲. (A98)

ER(⫹) is given by substitution of the general (y0 ⫽ 0) expressions of Equation A88 into Equation
A92. To obtain ER(⫺), we observe that the reflection transformation y 哫 ⫺y takes sample paths
of Equation A86 with A ⬎ 0 to those for A ⬍ 0; we may thus simply substitute ⫺y0 for y0 in
Equation A88, obtaining
NER ⫽ 1 ⁄ 2 关1 ⫺ ⌸erf共f ⫹ 兲 ⫺ 共1 ⫺ ⌸兲erf共f ⫺ 兲兴, (A99)

where
f ⫾ ⫽ f ⫾ 共␭, y0 ; A, c兲 ⫽
A
c 冑
共e␭t ⫺ 1兲
␭t
␭共e ⫹ 1兲
⫾
y0
c 冑 ␭e2␭t
e2␭t ⫺ 1
, (A100)
and, for future use,
⳵f ⫾
⳵y0
⫽⫾
1
c 冑 ␭e2␭t def
e ⫺1
2␭t ⫽ ⫾⳵fy0, and
⳵f ⫾
⳵␭
⫽
A
2c
F共␭, t兲共2␭te␭t ⫺ e2␭t ⫹ 1兲 ⫾
y0
2c 冑␭
冉 e␭t(e2␭t ⫺ 1 ⫺ 2␭t)
(e2␭t ⫺ 1) 3/ 2 冊 def
⫽ ⳵f␭ 1 ⫾ ⳵f␭ 2. (A101)
To minimize the NER, we compute the partial derivatives of Equation A99:
⳵ 1
共NER兲 ⫽ 关⫺⌸ exp关⫺共 f ⫹ 兲 2 兴 ⫹ 共1 ⫺ ⌸兲exp关⫺共 f ⫺ 兲 2 兴兴⳵fy0, and (A102)
⳵y0 冑␲
⳵ 1
共NER兲 ⫽ ⫺ 关共⌸ exp关⫺共 f ⫹ 兲 2 兴 ⫹ 共1 ⫺ ⌸兲exp关⫺共 f ⫺ 兲 2 兴兲⳵f␭ 1
⳵␭ 冑␲
⫹ 共⌸ exp关⫺共 f ⫹ 兲 2 兴 ⫺ 共1 ⫺ ⌸兲exp关⫺共 f ⫺ 兲 2 兴兲⳵f␭ 2兴. (A103)
Setting Equation A102 equal to zero and using Equation A100, we obtain
exp关共 f ⫹ 兲 2 ⫺ 共 f ⫺ 兲 2 兴 ⫽
⌸
1⫺⌸
4Ay0 e␭t
f 2 ␭t
c 共e ⫹ 1兲
⫽ log
⌸
1⫺⌸
, 冉冊 (A104)
and substituting Equation A104 into Equation A103 and using the properties of ⳵f␭1 derived directly
below Equation A97, we conclude that
␭⫽0 and y0 ⫽
c2
2A
log冉 ⌸
1⫺⌸ 冊 (A105)
at the critical point.

To check that this is indeed a minimum, we compute the Hessian matrix of second partial
derivatives at Equation A105, obtaining
⳵2 2 2 A
共NER兲 ⫽ ⌸ exp关⫺共 f ⫹ 兲 2 兴共 f ⫹ ⫹ f ⫺ 兲关⳵fy0兴 2 ⫽ ⌸ exp关⫺共 f ⫹ 兲 2 兴 ,
⳵y20 冑␲ 冑␲ 冑2tc3
⳵2 2 2 Ay20 t3/ 2
2 共NER兲 ⫽ ⌸ exp关⫺共 f ⫹ 兲 2 兴共 f ⫹ ⫹ f ⫺ 兲关⳵f␭ 2兴 2 ⫽ ⌸ exp关⫺共 f ⫹ 兲 2 兴 , and
⳵␭ 冑␲ 冑␲ 4 冑2c3
⳵2 2 2 Ay0 冑t
共NER兲 ⫽ ⌸ exp关⫺共 f ⫹ 兲 2 兴共 f ⫹ ⫹ f ⫺ 兲关⳵fy0⳵f␭ 2兴 ⫽ ⌸ exp关⫺共 f ⫹ 兲 2 兴 . (A106)
⳵y0 ⳵␭ 冑␲ 冑␲ 2 冑2c3
The second variation is therefore positive semidefinitely, vanishing only along the line
y0 ⫽
c2
2A
log
⌸
1⫺⌸ 冉冊冋 1⫺
␭t
2
,册
and direct computations of NER (for all parameters we sampled) show that the point (see Equation
A105) is indeed a minimum along this line.
The Extended Drift Diffusion and O-U Models
In this section, we consider variable drift rates A and initial values x0. In the first subsection, we
derive ER and DT expressions for the extended drift diffusion model. Instead of explicitly writing
analogues for the extended O-U process, in the second subsection we argue that for an O-U process
in which only the drift varies from trial to trial, optimal performance in the interrogation protocol
still results from the choice ␭ ⫽ 0. Finally, in the third subsection, we give a simple argument that
determines a lower bound on the ER for both the extended drift diffusion and the extended O-U
models that holds for both the interrogation and free-response protocols.
ER and DT for the Extended Drift Diffusion Model in the Free-Response Protocol
Averaging Over Uniform Distribution of Initial Conditions
We now let the initial condition y0 be uniformly distributed in the range [⫺sx, sx] with probability density
再
1
p共y0 兲 ⫽ 2sx
0
⫺sx ⱕ y0 ⱕ sx
otherwise
.
Recalling the coordinate change x ⫽ y/A, this implies that x0 has probability density
再A
p共x0 兲 ⫽ 2sx
0
⫺sx /A ⱕ y0 ⱕ sx /A
otherwise
.
The average of ER(x0) from Equation A30 over such starting points is
1
冦 1⫺
A
2sx 冕 sx/A
⫺共sx/A兲
e⫺2x0ãdx0
冧
具ER典 x0 ⫽ ⫺ .
1 ⫹ e2z̃ã e2z̃ã ⫺ e⫺2z̃ã
Evaluating this integral and defining s̃x ⫽ sx/A, we find that
冦冧
1
共e2ãs̃x ⫺ e⫺2ãs̃x兲 ⫺ 1
1 4ãs̃x
具ER典 x0 ⫽ ⫹ , (A107)
1 ⫹ e2z̃ã e2z̃ã ⫺ e ⫺ 2z̃ã
and similarly, from Equation A31, that
冦冧
z̃
2z̃ ⫺ (e2ãs̃x ⫺ e⫺2ãs̃x)
2ãs̃x
具DT典 x0 ⫽ z̃ tanh共z̃ã兲 ⫹ . (A108)
e2z̃ã ⫺ e ⫺ 2z̃ã
Additional Averaging Over a Gaussian Distribution of Drift Rates

We now additionally allow the drift rate to be Gaussian distributed with mean mA and variance sA2 :
1 2
p共A兲 ⫽ e⫺关共A⫺mA兲 / 2sA兴 .
2
冑2␲s 2
A
The resulting averages are
具ER典 x0,A ⫽ 冕
⫺⬁
⬁
p共A兲具ER典 x0dA, and (A109)
具DT典 x0,A ⫽ 冕
⫺⬁
⬁
p共A兲具DT典 x0dA. (A110)
To carry out these integrals, we must first rewrite Equations A107–A108 to explicitly display their
dependence on A:
冤冦冧冥
c2
冕
(e2Asx /c ⫺ e⫺2Asx /c ) ⫺ 1
2 2
⬁
1 2
⫺关共A⫺mA兲 2 / 2sA兴
1 4Asx
具ER典 x0,A ⫽ e ⫻ 2 ⫹ dA,
⫺⬁
冑2␲s2A 1 ⫹ e2Az/c e2zA/c ⫺ e⫺2zA/c
2 2
(A111)
冤冦冧冥
2z zc2
冕冉冊
⫺ 2 (e2Asx /c ⫺ e⫺2Asx /c )
2 2
⬁
1 2
⫺关共A⫺mA兲 2 / 2sA兴
z zA A 2A sx
具DT典 x0,A ⫽ e ⫻ tanh 2 ⫹ dA.
⫺⬁
冑2␲s2A A c e2zA/c ⫺ e⫺2zA/c
2 2
(A112)
Defining the parameters â ⫽ mA2 /c2, ŝx ⫽ sx/mA, ŝA ⫽ sA/mA, and ẑ ⫽ z/mA, and changing variables
to q ⫽ A/mA, these expressions become
冤冦冧冥
1
冕 ⬁ (e2ãs̃xq ⫺ e⫺2ãs̃xq ) ⫺ 1
1 2
⫺关共q⫺1兲 2 / 2s̃A兴
1 4ãs̃xq
具ER典 x0,A ⫽ e ⫻ ⫹ dq, and
⫺⬁
冑2␲s̃2A 1 ⫹ e2ãz̃q e2ãz̃q ⫺ e⫺2ãz̃q
(A113)
冤冦冧冥
2z̃ z̃
冕 ⬁ ⫺ 2 (e
2ãs̃xq
⫺ e⫺2ãs̃xq)
1 2 z̃ q 2ãs̃ x q
具DT典x0,A ⫽ e⫺关共q⫺1兲 / 2s̃A兴 ⫻ tanh共ãz̃q兲 ⫹
2
dq. (A114)
⫺⬁
冑2␲s̃2A q e2ãz̃q ⫺ e ⫺ 2ãz̃q
Note that these expressions involve only the four combined parameters â, ŝx , ŝ A , and ẑ, rather than
the five original parameters. That is, analogous to the case for the pure drift diffusion model, only
the ratios of the other parameters relative to mA enter.
Approximations
The integrals of Equations A113–A114 cannot be evaluated explicitly but may be approximated
to second order using Laplace’s method (e.g., Bender & Orszag, 1978) to obtain
ŝ2A
具ER典 x0,A ⬇ 具ER典 x0兩 â⫽共m2A/c2兲 ⫹ ␬ , and (A115)
2 ER
ŝ2A
具DT典 x0,A ⬇ 具DT典 x0兩 â⫽共m2A/c2兲 ⫹ ␬ , (A116)
2 DT
where the first terms are, respectively, Equations A107 and A108 evaluated at â ⫽ mA2 /c2 and
the correction terms are given by the second derivatives of the terms in square brackets with
respect to q, evaluated at q ⫽ 1. Explicitly (where we drop the hats on a, z, and sx for clarity
of reading):
␬ ER ⫽ a2 z2 sech2共az兲 tanh共az兲 ⫹ 再 e2a(⫺sx⫹z)

2asx(⫺1 ⫹ e4az) 3
((⫺1 ⫹ e4asx)(⫺1 ⫹ e4az) 2
⫺ 8a3 sxe2asx(1 ⫹ 6e4az ⫹ e8az)z2 ⫺ 2a(⫺1 ⫹ e4az)(sx(1 ⫹ e4asx)(⫺1 ⫹ e4az)

⫺ (⫺1 ⫹ e4asx)(1 ⫹ e4az)z) ⫹ 4a2 e2a(sx⫹2z) (⫺2(sx2 ⫺ 3z2 )sinh(2asx)
⫹ (sx ⫹ z) 2 sinh (2a(sx ⫺ 2z)) ⫹ (sx ⫺ z) 2 sinh (2a(sx ⫹ 2z)))) , 冎 and

␬ DT ⫽ ⫺2z共⫺tanh共az兲 ⫹ az sech2共az兲共1 ⫹ az tanh共az兲兲兲
⫹ 再 e2a共⫺sx⫹z兲 z
asx(⫺1 ⫹ e4az) 3
(⫺3(⫺1 ⫹ e4asx)(⫺1 ⫹ e4az) 2 ⫹ 8a3 sxe2asx(1 ⫹ 6e4az ⫹ e8az)z2
⫹ 4a(⫺1 ⫹ e4az)(sx(1 ⫹ e2asx ⫹ e4asx)(⫺1 ⫹ e4az) ⫺ (⫺1 ⫹ e4asx)(1 ⫹ e4az)z)
⫺ 4a2 ea共sx⫹4z兲 (1 ⫹ e2asx)(⫺2(sx2 ⫺ 3z2 )sinh(asx) ⫹ (sx ⫹ z) 2 sinh(a(sx ⫺ 4z))
冎
⫹ 共sx ⫺ z兲 2 sinh(a(sx ⫹ 4z)))) .
Note that, as above, the terms in braces vanish in the case where initial conditions are concentrated
at the origin, that is, y0 ⫽ x0 ⫽ 0 (so that sx ⫽ ŝx ⫽ 0). Figure A2 demonstrates the validity of these
approximations for small ŝA.
Optimal Values of ␭ for the O-U Model With Variable Drift in Interrogation Protocol
In this section, we consider a version of the O-U model in which drift is selected at the beginning
of each trial from a normal distribution p(A) with mean mA and standard deviation sA. For simplicity,
we assume that the starting point x0 ⫽ 0.
As shown by Usher and McClelland (2001), according to Equation A87, the solutions of this
model at time t have normal distribution with the following mean and variance:
mA ␭t c2 2␭t s2A
␮共t兲 ⫽ 共e ⫺ 1兲, and ␴ 2 共t兲 ⫽ 共e ⫺ 1兲 ⫹ 2 共e␭t ⫺ 1兲 2 .
␭ 2␭ ␭
Hence, the probability of error at the interrogation time t is equal to
冢冑冣
mA ␭t
(e ⫺ 1)
␭
ER共t兲 ⫽ ⌽ ⫺ ,
c2 2␭t s2A
(e ⫺ 1) ⫹ 2 共e␭t ⫺ 1兲 2
2␭ ␭
where ⌽ is the standard cumulative normal distribution function.
To find the value of ␭ that minimizes ER(t), we calculate the derivative of ER(t) with respect to
␭ and find the following condition for this derivative to equal zero: e2␭t ⫺ 2␭te␭t ⫺ 1 ⫽ 0. This
condition is satisfied when ␭ ⫽ 0, so that this corresponds to a critical point in ER(t). Simulations
verify that this critical point is indeed a minimum.
Lower Bounds on ER Due to Variable Drift

In this section, we show that if drift rates are drawn from a Gaussian density p(A) with mean mA
⬎ 0 and standard deviation sA, then the ER at any interrogation time or, in the free-response
protocol, the ER for any threshold is always greater than or equal to
2.8
0.175
0.15 2.6
0.125 2.4
0.2 0.4 0.6 0.8 1 2.2

0.075
0.05
0.2 0.4 0.6 0.8 1
Figure A2. Error rate (ER) and decision time (DT) for the extended drift diffusion model with parameter values
â ⫽ 1, ŝx ⫽ 1, ẑ ⫽ 2. Solid lines are exact values from numerical integration of Equations A113–A114; dotted
lines are from second-order approximate formulae derived above.
ER b ⫽ 冕 0
⫺⬁
p共A兲dA
for both the extended drift diffusion and O-U models. To see this, first assume as usual that initial
conditions x0 are drawn from a uniform density q(x0) within [⫺sx, sx] and define r(A, x0) to be the
ER for fixed values of drift and initial condition for the model at hand. Then,
ER ⫽ 冕 sx
⫺sx
dx0 q共x0 兲冕⫺⬁
⬁
dAp共A兲r共A, x0 兲
⫽ 冕⫺sx
sx
dx0 q共x0 兲
冋冕 ⫺⬁
0
dAp(A)r(A, x0 ) ⫹ 冕 ⬁
0
册
dAp(A)r(A, x0 ) .
Note that the factors in the integrands are positive and that because mA ⬎ 0, p(⫺A) ⬍ p(A) when
A ⬎ 0. This gives
ER ⱖ 冕 ⫺sx
sx
dx0 q共x0 兲
冋冕 0
⫺⬁
dAp(A)r(A, x0 ) ⫹ 冕 0
⫺⬁
dAp(A)r(⫺A, x0 ) .
册
Next, we observe that r(A, x0) ⫽ 1 ⫺ r(⫺A, ⫺x0) due to the symmetry of the O-U and drift diffusion
models with respect to reflection around x ⫽ 0 and the definition of r as one minus the fraction of correct
responses. This yields
ER ⱖ 冕 sx
⫺sx
dx0 q共x0 兲
冋冕 ⫺⬁
0
dAp(A)r(A, x0 ) ⫹ 冕 ⫺⬁
0
dAp(A)[1 ⫺ r(A, ⫺x0 )]
册
⫽ 冕 sx
⫺sx
dx0 q共x0 兲
冋冕⫺⬁
0
dAp(A)r(A, x0 ) ⫹ 冕⫺⬁
0
dAp(A) ⫺ 冕
⫺⬁
0
dAp(A)r(A, ⫺x0 ) .
册
Because q(x0) ⫽ q(⫺x0), the first and last terms cancel, and the second term, together with the fact that
兰⫺s
sx
x
dx0q(x0) ⫽ 1, gives the desired lower bound.
Appendix B
Experimental Method
Participants M key (rightward motion) or the Z key (leftward motion) on a

standard keyboard (the mapping of keys to the right and left
These were 20 adults (9 men and 11 women) recruited via responses was not counterbalanced across participants). After each
announcements posted around the Princeton University campus. correct response, participants were informed by a short beep that
They were predominantly Princeton undergraduate and graduate the response was correct and that they had scored a point. There
students (average age: 20 years). Participants were paid one cent was no feedback after incorrect responses. After each response,
for each correct choice. To further increase motivation, partici- participants were required to release the key, and there was a delay
pants were informed that the one who earned the most would D before presentation of the next stimulus (D was kept constant
receive an additional prize of $100, which was awarded at the end within each block but varied across blocks). On some blocks (see
of the experiment. The experiment was approved by the Institu- below), an additional delay Dp was imposed after error responses.
tional Review Panel for Human Subjects of Princeton University,
During these delay intervals, the current score was displayed in the
and all participants expressed written consent for participation.
center of the screen.
Stimuli and Apparatus
Design
We used the same stimuli that have been used in other studies of
Trials were blocked by delay condition. This was manipulated to
decision making (e.g., Gold & Shadlen, 2003; Palmer, Huk, &
test predictions made by the drift diffusion model (discussed in the
Shadlen, 2005). The display was a field of randomly moving dots,
section entitled Optimal Thresholds for Different Criteria in the
all of which appeared within a 5° circular aperture in the center of
Pure DDM in the main text). There were four delay conditions: (a)
the screen. Dots were white squares 2 ⫻ 2 pixels (0.7° square)
D ⫽ 0.5 s, (b) D ⫽ 1 s, (c) D ⫽ 2 s, and (d) D ⫽ 0.5 s and Dp ⫽
displayed against a black background, with a density of 16.7
1.5 s (in the first three conditions, Dp ⫽ 0). Each block lasted 7
dots/degree2/s (6 dots per frame). On each trial, a fraction of the
min (block length was limited by fixing its overall duration rather
dots moved in a single direction over time, corresponding to that than by the number of trials completed within it). One block of
trial’s correct direction, while the remaining dots were randomly trials was run for each delay condition, except for Condition 3
repositioned over time. On each frame, 11% of the dots were (D ⫽ 2 s) for which two blocks were run to permit a sufficient
independently selected as the coherently moving dots and were number of trials for analysis (because, in this delay condition, trials
shifted 0.2° from their position for each 40 ms (3 video frames) lasted longer). Hence, in total, the experiment consisted of five
elapsed, corresponding to a speed of 5°/s (either leftward or blocks of trials randomly ordered across participants. Before the
rightward). The remaining dots were replotted in random positions start of the experiment, participants had three blocks of practice in
on each frame. The display was generated in MATLAB on a which they did not receive money for correct choices. Participants
Macintosh computer, using the Psychophysics Toolbox extension were informed about the number of blocks and their duration and
(Brainard, 1997; Pelli, 1997) and software written by Josh Gold. that blocks differed in delays but were not told the exact durations
of delays.
Procedure
Participants were instructed to gain as many points as possible Received July 9, 2004
by deciding, for each trial, whether the prevailing motion of the Revision received May 29, 2006
dots was left or right and to indicate their responses by pressing the Accepted June 5, 2006 䡲

Bogacz Cohen

Uploaded by

Copyright:

Available Formats

Bogacz Cohen

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Bogacz Cohen

Uploaded by

Copyright:

Available Formats

Psychological Review Copyright 2006 by the American Psychological Association

2006, Vol. 113, No. 4, 700 –765 0033-295X/06/$12.00 DOI: 10.1037/0033-295X.113.4.700

The Physics of Optimal Decision Making: A Formal Analysis of Models of

using these as a reference for comparing models throughout the

40 (Optimal Performance of Decision Networks), we show that the

In the sixth section (Biased Decisions), we consider biased deci-

formal presentation to the most important equations, and where

p1n p1 共y1 兲p1 共y2 兲 · · · p1 共yn兲

We model the interrogation paradigm by assuming that at time T,

Feedforward Inhibition Model 再 dy1 ⫽ (⫺ky1 ⫺ wy3 ⫹ vy1 ⫹ I1 )dt ⫹ cdW1 ,

再 dy1 ⫽ I1 dt ⫹ cdW1 ⫺ u(I2 dt ⫹ cdW2 )

D=2 D=0.5 Dpen=1.5

Reaction time [s]

a) w=k=100 b) w=k=10 c) w=k=1

Reaction time [s]

Reaction time [s]

-0.5 -0.5 -0.5

-0.5 0 0.5 y1 -0.5 0 0.5 y1 -0.5 0 0.5 y1

冋dydy 册 ⫽ 冋⫺k 册冋 yy dtdt 册

a) Mutual inhibition model b) Mutual inhibition model

Decision Time [s]

c) Feedforward inhibition model d) Mutual inhibition model

Reward rate [rewards/s]

0.35 Diffusion 0.43

a) Decay > Inhibition b) Decay = Inhibition c) Decay < Inhibition

0.2 0.5 0.2

0.3 0.3 0.3

0.3 1.5 0.3

Mean normalized decision time

0.25 s A =0, s x =0.14 0.25 s A =0, s x =0.14

Threshold-Setting Algorithms mine (or approximate) it using an adaptive adjustment procedure:

rithms for updating the decision threshold: (a) an error correction

but above a certain ⌸, the optimal starting point exceeds the 2⌸ ⫺ 1 ⌸

0.1 0.04 0.1

creases, ER decreases. That is, the ER at the transition point is, DT

1 ⫺ ER without integrating. This surface, found via a numerical search, is

Mathematical Details of Decision Optimization in Two-Alternative

Probability Ratio Tests

DT SPRT 共ER兲 ⱕ DT a共ER兲, for all ER. (A1)

which is negative for c1 ⬎ 0.

Thresholds for the SPRT

p1 共C1 兲 ⱖ Z1 p2 共C1 兲, (A7)

Random Walks and the SPRT

To maintain I2 ⬍ 0 ⬍ I1, we must select an ER lower than the smaller of ⌸ and 1 ⫺ ⌸.

The Continuum Limit of the SPRT

IM共t兲 f DW共t兲 ⫹ mt ⬅ I共t兲, (A18)

dI ⫽ mdt ⫹ DdW, I共0兲 ⫽ 0, (A19)

(the ⫹ applies if S1 is presented, the ⫺ if S2), and in both cases,

dy ⫽ Adt ⫹ cdW; y共0兲 ⫽ 0. (A25)

Optimal Decisions for the Free-Response Protocol

The Drift Diffusion Model

dy ⫽ Adt ⫹ cdW, y共0兲 ⫽ y0 , (A27)

Equation A27 becomes

ERs, Mean DTs, and Reward Rates

DT ⫽ z̃ tanh共z̃ã兲 ⫹ 再 2z̃共1 ⫺ e⫺2x0ã兲

Optimal Thresholds for the Drift Diffusion Model

z̃o is found from

e2z̃oã ⫺ 1 ⫽ 2ã共Dtotal ⫺ z̃o兲, (A33)

lim z̃ ne⫺z̃ã ⫽ 0 (A36)

for any integer n ⱖ 0, we find that for large z̃,