Risk-sensitive neurons in macaque posterior cingulate cortex

© 2005 Nature Publishing Group http://www.nature.com/natureneuroscience ARTICLES Risk-sensitive neurons in macaque posterior cingulate cortex Allison N McCoy1 & Michael L Platt1,2,3 People and animals often demonstrate strong attraction or aversion to options with uncertain or risky rewards, yet the neural substrate of subjective risk preferences has rarely been investigated. Here we show that monkeys systematically preferred the risky target in a visual gambling task in which they chose between two targets offering the same mean reward but differing in reward uncertainty. Neuronal activity in posterior cingulate cortex (CGp), a brain area linked to visual orienting and reward processing, increased when monkeys made risky choices and scaled with the degree of risk. CGp activation was better predicted by the subjective salience of a chosen target than by its actual value. These data suggest that CGp signals the subjective preferences that guide visual orienting. To survive and thrive, animals must make choices that relate internal states to the current environment. For example, choosing to pursue food or water depends not only upon available resources but also on whether hunger or thirst is greater. The decision to make a particular action thus depends on subjective needs and desires as well as any objectively measurable gains1–3. In addition to state-dependent variables such as hunger, thirst and even wealth, subjective biases also contribute to decision making. Since the 18th century, it has been known that people’s choices reflect reward uncertainty as well as reward value4. When confronted with two options of the same mean value but differing in uncertainty, both people and animals typically avoid choosing the uncertain, or risky, option5,6. The idea that subjective preferences guide decision making has since become a core concept in the decision sciences2,7. However, the impact of subjective preferences on neural mechanisms of decision making remains largely unexplored (but see ref. 8). The simplest economic models of decision making posit that rational choosers select the alternative with the highest expected value9,10. Recent neurophysiological studies of visual orienting decisions have demonstrated that neurons in several brain areas linking visual perception with eye movements also track target value11–15. These observations suggest that orienting decisions are computed, in part, by scaling neuronal responses by target value11,14,16. One question these observations raise, however, is whether reward modulation of neuronal activity in these brain areas reflects scaling by subjective value17, predicted reinforcement14,18 or motivation19. As people and animals often demonstrate strong attractions or aversions to options with uncertain rewards2,6,20, risk preference provides a promising behavioral framework for exploring neural mechanisms underlying decision making and offers a potential way to dissociate subjective value from objective rewards. Specifically, neurons participating in the decision process should be sensitive to subjective risk preferences, even when available options have the same objective value. To test this prediction, we recorded from single neurons in posterior cingulate cortex (CGp), a limbic area linking reward with spatial attention21,22 and orienting23,24. Two adult male rhesus macaques performed a visual gambling task in which they chose between two visual targets offering the same mean reward but differing in reward uncertainty (Fig. 1a). We found that monkeys preferred orienting to targets offering uncertain rewards, and neuronal activity in CGp reflected these risk preferences. Our data suggest that neuronal responses in CGp signal subjective spatial biases that guide orienting. RESULTS Behavioral risk preferences in monkeys Although numerous studies have sought to understand risk preferences in humans, birds and insects (reviewed in refs. 3,6), risk preferences in monkeys remain largely unstudied. Therefore, we first probed monkeys’ behavioral sensitivity to reward uncertainty in a visual gambling task. Shifting gaze to the ‘certain’ target resulted in 150-ms access to fruit juice; shifting gaze to the ‘risky’ target resulted in the random receipt of less than 150 ms on one-half of the trials and more than 150 ms on the other half of trials (mean ¼ 150 ms). The locations of the certain and risky targets, as well as the degree to which risky reward values deviated from the mean, were varied across blocks of 50 trials (Fig. 1a, lower panel). Here we define risk as the coefficient of variation (CV) of rewards associated with the risky target, a dimensionless measure of relative risk permitting direct comparisons with other studies3. qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Reward CV ¼ ðððx1
meanÞ2 +ðx2
meanÞ2 Þ=nÞ=mean 1Department of Neurobiology, 2Center for Cognitive Neuroscience and 3Department of Biological Anthropology and Anatomy, Duke University Medical Center, Box 3209, Durham, North Carolina 27710, USA. Correspondence should be addressed to M.L.P. ([email protected]). Published online 14 August 2005; doi:10.1038/nn1523 1220 VOLUME 8 [ NUMBER 9 [ SEPTEMBER 2005 NATURE NEUROSCIENCE Vertical amplitude where x1 and x2 are values for risky target and a b n is the number of risk values. Overall, both MR CS monkeys preferred the risky target and the T1 T1 frequency of choosing it increased systemReward CC Move atically with the degree of risk (Fig. 2a). T2 Fixation T2 Sensitivity to reward uncertainty is not Delay off Target on predicted by standard models of reinforceFixation on Time ment learning. Specifically, reinforcement learning models predict that the associative Fix strength of any stimulus for controlling behaTarget T1 c 20 Target T2 vior is determined by rewards delivered in Eye RF 25,26 T1 association with that stimulus . Because Reward average reward size was the same for risky Fix 0 Sample reward schedules and certain targets, such models would not 0.667 Reward CV 0.0667 0.1 0.167 0.333 0.5 predict a preference for one option over the Certain target 150 150 150 150 150 150 T2 other. To explore this surprising behavioral Risky target 140 160 135 165 125 175 100 200 75 225 50 250 –20 pattern further, we therefore examined the –20 0 20 50 trials Horizontal amplitude impact of prior reward outcomes on subsequent choices. We found that receipt of smal- Figure 1 Method for investigating risk sensitivity in macaque posterior cingulate cortex. (a) Visual ler-than-average rewards on the previous trial gambling trials were used to investigate risk sensitivity. On each trial, subjects initially fixated (± 1–21) a blunted the likelihood of choosing the risky central yellow LED (200–800 ms). Two peripheral yellow LEDs were then illuminated diametrically target on the next trial (Fig. 2b) but not nearly opposite the fixation LED (200–800 ms). The fixation LED was extinguished, cueing the monkey to shift as much as would be expected by standard gaze to either target (± 3–51) within 350 ms. Correct trials were rewarded with a 300-ms noise burst and reinforcement learning25. Moreover, monkeys juice. Lower panel: example of reward schedule. Mean reward size for each target was 150 ms; the range of reward differences for the risky target was 20–250 ms across blocks of trials. (b) Recording sites in behaved as if they overvalued relatively large posterior cingulate cortex (CGp), estimated by digital ultrasound imaging. Diagonal hatches indicate rewards. Our behavioral dataset was large approximate neuron locations within areas 31 and 23. Recording chamber projection (red box) and major enough to examine sequences of up to seven landmarks in the ultrasounds are indicated (CS: cingulate sulcus, horizontal limb; MR: marginal ramus; prior reward outcomes with statistical CC: corpus callosum). (c) Example of target geometry. One target was inside the response field (RF) while confidence. Analysis showed that monkeys the other was diametrically opposite the fixation point. Neuronal response is plotted as a function of significantly preferred the risky target even target location using an arbitrary color scale from blue to red (low to high firing rate). after a sequence of six smaller rewards that followed a large reward (F ¼ 5.709, P o 0.00001). These data make target when the monkey chose it: plain that the choices monkeys make depend not only on expected Vcertain ¼ Reward receivedcertain target + Riskcertain target ð2Þ reward value, as shown previously16, but also on reward uncertainty. We next examined the impact of both received rewards and risk experienced for prior choices on the probability of choosing the risky where the risk of the certain target was always 0. target using logistic regression27. We found that both the degree of risk We next estimated the subjective utility of the risky target Urisky on a and normalized reward value associated with the target chosen on the given trial (t) according to the following algorithm: previous trial biased the probability of choosing the risky target on the Urisky ðtÞ ¼ Sn ¼ 1 to i ½Vrisky ðt
nÞ
Vcertain ðt
nÞ  an ð3Þ following trial (logistic regression coefficients: target risk ¼ 2.768; target reward value ¼ 4.16 (risky), 3.378 (certain); all P-values | where an is the logistic coefficient for the difference in the experienced 0.001); moreover, including both risk and rewards received for prior value of the risky and certain targets lagged n trials. Multiple logistic choices significantly improved the explanatory power of the model over regression analysis of the probability of a risky choice as a function of any other single variable or combination of variables (Akaike’s Infor- the difference in experienced value for the two targets (Vrisky
Vcertain) mation Criterion (AIC)combined ¼ 14,629.66; AICtarget risk ¼ 15,476.45; on each of up to ten prior trials was used to derive the weighting factor AICrisky target reward ¼ 14,847.19; AICcertain target reward ¼ 18,079.97; all an. This analysis showed that the value function difference (Vrisky
other combinations, AICs 4 (AIC)combined). Vcertain) significantly influenced the probability of choosing the risky Next, we developed a simple model of monkeys’ subjective prefer- target at all lags up to five trials (AIClags1–5 o AICs for all other ence for the risky target—which we refer to here as subjective target combinations) but declined rapidly thereafter (Fig. 2c). As the addition utility—on the basis of the difference in the experienced value of of further lags did not significantly improve the model, i was set to 5 the risky and certain targets. Since the logistic regression analysis trials. Using the weighting factor an, our estimate of subjective target showed that both experienced risk and rewards received influenced utility, derived from prior choices and their associated rewards and risk, the probability of choosing the risky target on subsequent trials, we provided a good prediction of the probability of choosing the risky denoted the experienced value of the risky target Vrisky on each trial as target (logistic regression coefficient ¼ 5.2222, Wald statistic ¼ the sum of the risk and reward associated with that target when the 2,560.93, P o 0.00001). In animals and humans, preference for risky options has been monkey chose it: associated with impoverished physiological6,28 or financial2,29 status. Vrisky ¼ Reward receivedrisky target + Riskrisky target ð1Þ We therefore asked how the risk sensitivity of our monkeys was affected by their hydration status. In our experiments, access to fluids was Similarly, we denoted the experienced value of the certain target Vcertain limited during the week but freely available on weekends. Risk on each trial as the sum of the reward and risk associated with that sensitivity was therefore examined as a function of the day of the NATURE NEUROSCIENCE VOLUME 8 [ NUMBER 9 [ SEPTEMBER 2005 Response (Hz) © 2005 Nature Publishing Group http://www.nature.com/natureneuroscience ARTICLES 1221 1222 Coefficient for risky value Probability of novel choice Probability of risky choice Probability of risky choice Probability of risky choice week, as well as time during an experimental a 0.9 b 1.2 c 1.2 session, under the assumption that fluid bal1.0 1.0 ance would decrease over the course of the 0.8 0.9 0.8 week and increase as juice was consumed during each daily session. We found no effect 0.6 0.8 0.7 of day of the week (logistic regression coeffi0.4 0.7 cients, day of the week ¼ 0.005, P 4 0.97; 0.6 0.2 0.6 subject ¼ 0.23, P o 0.000001; risk ¼ 0.281, 0.0 0.5 Monkey Niko 0.5 P o 0.000001) or number of trials performed Monkey Broome –1.2 in each session (logistic regression coeffi0 50 100 150 200 250 300 10 9 8 7 6 5 4 3 2 1 0.0 0.2 0.4 0.6 0.8 1.0 cients: trial ¼ –0.00004, P 4 0.42; subject Risk (reward CV) Reward on previous trial (ms) Lag (trials) ¼ 0.22, P o 0.000001; risk ¼ 2.44, P o d 0.05 e 0.9 f 1.0 0.000001). Thus, monkeys showed consistent n = 2032 trials n = 2008 trials preferences for uncertain fluid rewards, and 0.8 0.04 0.8 these preferences were apparently independent of fluid balance. 0.6 0.03 0.7 One important question is whether any 0.4 nonlinearity in the computer-driven solenoid 0.02 0.6 controlling fluid delivery might explain why 0.2 monkeys preferred the risky option. In fact, 0.01 0.5 fluid reward size was a linear function of 0.0 100 200 300 0 Broome Niko 0.0 0.2 0.4 0.6 0.8 1.0 solenoid open time (volume ¼ 0.0026 + Solenoid open time (ms) Monkey Risk (reward CV) 0.001  (open time in ms); Fig. 2d). Thus, the risk preferences of our monkeys could not Figure 2 Monkeys prefer targets offering uncertain rewards. (a) Probability of choosing the risky target as be explained by an asymmetry in the size of a function of risk for each monkey. Risk preference increased with increasing risk (logistic regression rewards delivered across the range of values coefficients: Broome, 2.442, P o 0.0000001; Niko, 2.426, P o 0.0000001). (b) Monkeys discount tested. We also performed a control experi- low payoffs at the risky target. Probability of choosing the risky target is plotted as a function of reward received on the previous trial for both monkeys. Both monkeys were indifferent to the average reward size ment in which choosing the risky target was (150 ms) but systematically preferred the risky target after either small or large payoffs. (c) Influence of followed by delivery of larger-than-average rewards received and risk on current choice declines with time. Logistic regression coefficient for the rewards on one-third of trials and smaller- experienced value of the risky target, estimated as the sum of the reward and risk received for choosing than-average rewards on two-thirds of trials that target, plotted as a function of trial lag. (d) Juice volume varies linearly with solenoid open-time. (as compared with one-half larger-than- (e) Monkeys preferred the risky target despite receiving a net loss of juice. Probability of choosing the average and one-half smaller-than-average risky target as a function of risk for control experiment in which choosing the risky target resulted in a rewards in standard visual gambling trials). two-thirds chance of a lower-than-average reward and a one-third chance of a higher-than-average reward. (f) Monkeys were indifferent to targets that changed color. Probability of choosing novel colored target The certain target, as before, offered 150-ms when reward size was equal is plotted for both monkeys. Broome, n = 2,032 trials; Niko, n = 2,008 trials. access to juice on all trials, and the specific values of high and low rewards were unchanged from standard gambling trials. Choosing the risky target Neural correlates of reward uncertainty in CGp thus resulted in a net loss of juice compared with choosing the certain Having established the risk preferences of two monkeys performing a option, and this loss increased with increasing risk. Despite the fact that visual gambling task, we next examined the activity of single neurons in the expected value of the risky target now declined with increasing risk, posterior cingulate cortex (Fig. 1b) for evidence of similar risk sensitivity. Neurons in this area respond to the illumination of conmonkeys continued to prefer it (Fig. 2e). Another question these data raise is whether the observed risk tralateral visual stimuli30, after contraversive gaze shifts16,24,30 and after preferences might simply reflect a preference for the novelty, or reward delivery16, and the strength of these responses is modulated by variability, of rewards associated with the risky target. In other reward size and expectancy16. Because most CGp neurons respond words, perhaps a changing target or reward is simply more interesting selectively for a broad range of contralateral saccades, experiments were for monkeys. This is an important potential confound which we conducted such that one target was placed inside the response field of sought to address in a control experiment. Reward size was held the neuron under study while the other target was diametrically constant at 150 ms access to juice for both targets while novelty was opposite the fixation point (Fig. 1c). The response field was determined introduced by systematically changing the color of one of the targets during 100–400 standard mapping trials in which monkeys were asked during reward delivery. For the ‘monotonous’ target, the color of the to shift gaze to targets throughout the visual field. target remained yellow throughout the trial. For the ‘novel’ target, The activity of a single CGp neuron recorded during visual target color changed from yellow to green on half of the trials and from gambling trials is shown in Figure 3. Neuronal activity increased yellow to red on the other half of the trials. Notably, both targets were after movement onset, and this activity was modulated both by whether yellow until the delivery of reward such that any difference in the movement was into or out of the response field as well as whether luminance by color would not affect the monkeys’ choices. The those choices were for risky or certain rewards (Fig. 3a). During the locations of the novel and monotonous targets were reversed across epoch 200–400 ms after movement onset, firing rate was modulated by blocks of trials to mitigate any spatial bias in the monkeys’ choices. both movement direction and risk (Fig. 3b). Neuronal activity Neither monkey showed any preference for the novel colored target increased systematically with increasing risk, and this risk-related (Fig. 2f). Thus, monkeys’ preference for risky targets is unlikely to be modulation was stronger for movements in the neuron’s preferred explained by novelty alone. direction (Fig. 3c). Juice volume (ml) © 2005 Nature Publishing Group http://www.nature.com/natureneuroscience ARTICLES VOLUME 8 [ NUMBER 9 [ SEPTEMBER 2005 NATURE NEUROSCIENCE ARTICLES Firing rate (Hz) 15 0 BR030429 500 ms Movement onset b c 30 Into RF Out of RF Firing rate (Hz) © 2005 Nature Publishing Group http://www.nature.com/natureneuroscience 30 Figure 3 Posterior cingulate neurons are risk-sensitive. (a) Post-stimulus time histogram for a single CGp neuron aligned on movement onset. Points indicate average (± s.e.m.) firing rate measured in 100-ms bins. Firing rate was greater for choices into RF than out of RF, as well as for risky choices than for certain choices. Gray shaded box indicates 200-ms epoch analyzed in b,c. (b,c) Average firing rate of the same CGp neuron plotted as a function of choice (risky or certain) and reward CV for choices in the neuron’s preferred (black) and non-preferred (gray) directions. Firing rate systematically increased for risky choices as well as with increasing reward CV for movements in the preferred direction (b: t-test, t ¼ 5.55, df ¼ 375, P o 0.000001; c: multiple regression, r ¼ 0.230, df ¼ 375, P o 0.00003). Risky choices into RF Certain choices into RF Risky choices out of RF Certain choices out of RF 15 0 Certain 0.0 Risky 0.2 0.4 0.6 0.8 Risk (reward CV) Choice Neuronal activity was modulated both by which target was chosen and whether that target was associated with uncertain rewards. Overall, the activity of 22/41 (53.7%) studied neurons was significantly modulated by whether the risky target was chosen (14/25 neurons in monkey Broome and 8/16 neurons in monkey Niko). Across the population, average neuronal activity was greater for risky target choices than for certain target choices (ANCOVA: fixation epoch, F ¼ 3.957, P o 0.05; pre-movement, F ¼ 11.321, P o 0.001; post-movement, F ¼ 2.346, P 4 0.10), even when the effects of movement amplitude, latency, peak velocity and direction were removed statistically. We collected data from 39/41 CGp neurons in the same four conditions of risk. Population neuronal activity increased systematically with increases in risk throughout trials (Fig. 4; multiple regression: fixation epoch, rrisk ¼ 0.030, n ¼ 9,153, P o 0.005; premovement epoch, rrisk ¼ 0.048, n ¼ 9153, P o 0.00001; post-movement epoch, rrisk ¼ 0.047, n ¼ 9153, P o 0.00001), roughly paralleling the frequency of risky choices made by monkeys in these experiments (compare to Fig. 2a). Notably, the effect of risk on neuronal firing rates persisted as a tonic change throughout trials but was maximal when superimposed upon phasic, movement-related responses (Fig. 4c). Consistent with previous reports, the CGp population was also sensitive to whether the chosen target was in the neuronal response field, even while monkeys maintained central fixation before target onset (Fig. 4; fixation epoch: rdirection ¼ 0.056, n ¼ 9,153, P o 0.000001; pre-movement epoch: rdirection ¼ 0.067, n ¼ 9,153, P o 0.000001; post-movement epoch: rdirection ¼ 0.125, n ¼ 9,153, P o 0.000001). These data indicate that CGp neuronal activity is sensitive to reward uncertainty as well as to target choice. For any particular choice, either into or out of the response field, CGp neuronal activity varied with risk. Our data also indicate that the spatial selectivity of CGp neurons was enhanced by increasing risk: in high-risk blocks, the neuronal population more accurately discriminated movement direction, visualized NATURE NEUROSCIENCE VOLUME 8 [ NUMBER 9 [ SEPTEMBER 2005 here as a larger separation between black and gray lines (Fig. 4; posthoc Tukey’s honestly significant difference tests for saccade direction; fixation epoch: low-risk, not significant (n.s.); high-risk, P o 0.000001; pre-movement epoch: low-risk, n.s.; high-risk, P o 0.000001; postmovement: low-risk, P o 0.01; high-risk, P o 0.00000001). Thus, increasing risk did not seem to be associated with global or uniform changes in neuronal activity, but rather with selective enhancement of task-related neuronal activity. The enhancement of spatial sensitivity by risk is difficult to explain by global changes in arousal. Nevertheless, we sought to examine whether changes in neuronal activity associated with risky choices reflected changes in autonomic arousal. Correlates of autonomic arousal such as heart rate, galvanic skin response, and cortisol levels are elevated during gambling in humans31–33 and are attenuated in medial prefrontal lesion patients with poor impulse control in gambling tasks34. We therefore recorded the heart rates of both monkeys while they performed visual gambling trials in a separate set of experiments (monkey Niko: 4,471 trials; monkey Broome: 3,780 trials). Although there were fluctuations in heart rate over the course of behavioral sessions as well as upon block changes, we found no systematic effect of risk on heart rate (Fig. 5). We examined other potential behavioral correlates of arousal or motivation that might vary with reward uncertainty35. Reaction times were not significantly faster when monkeys made risky choices (F ¼ 2.7, P 4 0.1) but were significantly slower on trials after delivery of either unusually large or small rewards (F ¼ 13.8, P o 0.000001). Similarly, risky choices had no influence on peak saccade velocity scaled by saccade amplitude (ramplitude ¼ 0.519, P o 0.000001; rrisky choice ¼
0.00850, P 4 0.322), but peak saccade velocity was significantly higher after delivery of both the smallest and largest rewards a10 Firing rate (Hz) a Fixation b Pre-movement c Post-movement Into RF Out of RF 8 6 4 0.0 0.2 0.4 0.6 0.8 Risk (reward CV) Figure 4 Target risk enhances neuronal activity in CGp as well as sensitivity to movement direction. Plots of average (± s.e.m.) neuronal firing rate as a function of risk (reward CV) for the epochs after fixation (a), before movement onset (b) and after movement onset (c). Activity was greater for movements made into the RF and scaled with the degree of risk. 1223 ARTICLES 130 We therefore first computed the experienced values of the target in the response field (RF) and the target outside the response field (ORF) using the risk and reward outcomes associated with prior choices, as above. The experienced value of the RF target VRF and the experienced value of the ORF target VORF on a trial were computed as Monkey Niko Monkey Broome Heart rate (beats/min) 120 115 VRF ¼ Reward receivedRF + RiskRF ð4Þ 110 VORF ¼ Reward receivedORF + RiskORF ð5Þ The subjective utility of the target in the response field URF on a given trial t was then estimated according to the following algorithm: 105 100 0.0 0.2 0.4 0.6 0.8 1.0 URF ðtÞ ¼ Sn ¼ 1 to i ½VRF ðt
nÞ
VORF ðt
nÞ  an Risk (reward CV) Figure 5 Average heart rate does not increase with increasing risk. Average heart rate (± s.e.m.) measured at 12 Hz by means of pulse oximetry is plotted as a function of risk (reward CV) for both monkeys. (F ¼ 23.858, P o 0.000001). Thus, while the saccade metrics and choices of monkeys suggested that monkeys were, in fact, sensitive to risk, the effects of risk on firing rate are not readily explained by global changes in arousal. CGp neurons track subjective target preferences Despite the fact that the amount of reward received from choosing each target was equivalent over time, monkeys systematically preferred the risky target. Moreover, neuronal activity in CGp increased with the selection of risky targets. These data suggest that, under these conditions, the activation of CGp neurons reflects subjective biases for targets associated with uncertain rewards. If this is true, then firing rate on any particular trial should be more closely associated with subjective preferences for a particular target than with the actual rewards harvested by choosing it. We tested this hypothesis by first examining neuronal activity as a function of prior rewards received and, second, by computing an estimate of subjective target utility on the basis of the influence of both risk and reward received for previous choices27 and asking whether neuronal activity was related to this measure. First, we examined neuronal activity in the CGp population on each trial as a function of the size of the rewards delivered on the previous trial to ask whether neuronal activity reflected in any simple way the actual rewards received (Fig. 6). In all three measured epochs, firing rate discriminated between target choices into and out of the response field (fixation: F ¼ 29.951, P o 0.000001; pre-movement: F ¼ 42.226, P o 0.0000001; post-movement: F ¼ 148.152, P o 0.0000001). Moreover, firing rate was elevated when monkeys received rewards that deviated from the average, certain value (fixation: F ¼ 5.358, P o 0.0000001; pre-movement: F ¼ 7.022, P o 0.0000001; postmovement: F ¼ 5.649, P o 0.0000001). However, CGp neuronal activity did not distinguish between the lowest and highest rewards received on previous trials (post-hoc Tukey honestly significant difference tests, P 4 0.15 in all epochs). Thus, neuronal responses in CGp did not monotonically reflect the actual value of rewards received. We suspected that, instead, CGp responses for a particular target choice reflected monkeys’ subjective valuation of the target on the basis of their own internal preferences (compare with Fig. 2b). As a second test of this hypothesis, we computed a local estimate of subjective target preference, which we refer to as subjective target utility. Analysis of behavioral data, described above, showed that both the risk and reward value associated with prior choices roughly equally biased the probability of choosing the risky target on subsequent trials. 1224 ð6Þ where an is the logistic coefficient for the difference in the experienced value of the RF and ORF targets lagged n trials. Multiple logistic regression analysis of the probability of an RF target choice as a function of the difference in experienced value for the two targets (VRF
VORF) on each of up to ten prior trials was used to derive the weighting factor an. We found that differences in the experienced value of the two targets significantly influenced the probability of choosing the RF target at all lags up to ten trials (AIClags1–10 o AICs for all other combinations), so the weighting term an was therefore weighted on the basis of the logistic regression coefficients for each trial lag with i set at 10 trials. The utility of the target outside the response field was computed as the sign-reversed utility of the response field target. High target utility implied that the monkey frequently chose, and therefore preferred, a particular target in the past ten trials. These computations were performed across the entire dataset. We examined the percentage of cells in the population that were significantly correlated with the subjective utility of the response field target and whether the response field target was chosen, using a multiple linear regression analysis with movement latency, amplitude and peak velocity as co-regressors. The firing rates of 64% of cells were significantly modulated by target utility as well as target choice in any of twelve epochs examined (see Methods; Fig. 7). Modulations of firing rate by target utility and target choice, however, varied over time as well. Over 20% of the studied population of neurons showed a significant correlation between firing rate and subjective target utility during initial fixation before target onset (Fig. 7a), and the percentage of neurons with a significant correlation between firing rate and target utility gradually increased and peaked at around one-third of the a 12 Fixation b Pre-movement c Post-movement Into RF Out of RF Firing rate (Hz) © 2005 Nature Publishing Group http://www.nature.com/natureneuroscience 125 10 8 6 4 50 100 150 200 250 Reward on previous trial (ms) Figure 6 Population neuronal activity reflects monkeys’ preference for risky targets but not prior reward outcomes. Firing rate is plotted as a function of reward on previous trial for epochs after fixation (a), before movement onset (b) and after movement onset (c). Activity was greater for movements made into the RF as well as when relatively small or large rewards were delivered on previous trial. VOLUME 8 [ NUMBER 9 [ SEPTEMBER 2005 NATURE NEUROSCIENCE ARTICLES b 40 30 20 10 0 Target Movement onset onset Into RF 12 10 8 6 4 –0.5 0.0 0.5 1.0 1.5 2.0 2.5 Target utility Reward offset a b Behavior 0.9 Out of RF Post-movement activity 12 0.8 Firing rate (Hz) Target utility RF choice Probability of risky choice 50 Post-movement response (Hz) Cells significantly correlated (%) © 2005 Nature Publishing Group http://www.nature.com/natureneuroscience a 0.7 0.6 10 8 6 0.5 4 0 10 20 30 40 Trial within block 50 0 10 20 30 40 50 Trial within block Time during trials Figure 7 CGp neurons carry information about both target choice and subjective target utility. Multiple linear regression analysis was performed with utility, RF choice and saccade kinematic parameters as co-factors. (a) Percentage of cells with a significant correlation are plotted as a function of time during trials. (b) Population activity is correlated with subjective target utility. Firing rate measured after saccade onset is plotted as a function of target utility, estimated from the influence of risk and rewards received on target choices over the previous ten trials. population in the epochs after movement onset. In contrast, fewer than 5% of neurons showed a correlation between eventual target choice and firing rate during fixation before target onset, and the percentage of cells with a significant correlation peaked just after target choice and persisted through the end of the trial. These trends suggest a gradual temporal shift in the information carried by CGp neurons, initially favoring the subjective utility of available targets and later reflecting target choice and, to a lesser degree, target utility. Across the studied population of neurons, firing rate was positively correlated with target utility for all trials in which monkeys chose the target in the neuronal response field (Fig. 7b, black lines; post-movement epoch: r ¼ 0.146, P o 0.0000001) and negatively correlated with target utility when monkeys chose the target out of the response field (Fig. 7b, gray lines; post-movement epoch: r ¼
0.0893, P o 0.005). Activity in the CGp population was therefore well correlated with a measure of target utility, estimated from the monkeys’ history of choices, rewards received and associated risk. Note that black and gray lines overlap at low measures of target utility and diverge with increasing values, indicating that the ability of CGp neurons to discriminate target choice depended on the utility of those targets. If, as these data suggest, CGp neurons carry information about both the direction and subjective utility of movements monkeys make, an important question remaining is whether such activity causes—or reflects—the animals’ choices. To address this issue, we examined the time course of neuronal activity and the pattern of behavioral choices after a switch in the location of risky and certain targets. We found that the proportion of risky choices rose steeply from indifference (0.5) and reached a plateau within 15–20 trials (averaged across multiple values of risk; Fig. 8a). Similarly, neuronal activity in the 200–400 ms epoch after movement onset increased by approximately 50% and began to plateau within 15–20 trials from a block change (Fig. 8b). Piecewise linear regression analysis for the probability of a risky choice and neuronal activity as a function of time after a switch in the location of the risky target showed break points, respectively, of 19.4 (r ¼ 0.889, r2 ¼ 79.1%) and 24.6 (r ¼ 0.868, r2 ¼ 75.4%) trials, suggesting that neuronal activity in CGp closely followed changes in monkeys’ choices. This interpretation should be viewed with caution, however, as piecewise linear regression break points can be sensitive to the range of data included in the model. NATURE NEUROSCIENCE VOLUME 8 [ NUMBER 9 [ SEPTEMBER 2005 Figure 8 Both the frequency of risky target choices and neuronal activity gradually increase after block changes. Probability of risky choice (a) and neuronal activity (b) are plotted as a function of trial after a change in the location of the risky target. Behavioral preference for risky target and neuronal firing rate increased at a similar rate and reached asymptote within 15–20 trials. Taken together, our results indicate that a subset of neurons in posterior cingulate cortex carries information about the subjective utility of targets in the visual world. Indeed, neuronal activity in CGp mirrors the behavioral sensitivity of monkeys to risk. These data are consistent with a role for CGp in signaling the subjective salience of locations in the visual scene. DISCUSSION Economists, experimental psychologists and behavioral ecologists have long argued that decision making depends on the conversion of external variables into a common internal currency of value. Without such a common currency, neither animals nor people would be able to choose adaptively between apples and oranges, much less activities as disparate as eating and mating. While it is readily accepted that internal representations of value lie at the very core of decision making, most neurobiological studies of the decision process have manipulated objective value, typically by changing reward size or probability, because these factors are easily controlled and quantified. On the other hand, subjective value can be measured only indirectly, and attempts to correlate it with neuronal activity are, not surprisingly, rare14,17. The visual gambling task used in the current study affords a unique opportunity to examine neural activity under conditions in which subjective value, but not objective value, varied, as indicated by subjects’ choices. Although there was no apparent reason for monkeys to prefer one option or the other, they showed systematic preferences for targets offering uncertain rewards, much like hungry birds6,20,28,36, typical adolescents37–39 and people addicted to drugs40 or pathological gambling32. Moreover, preference for the uncertain reward increased parametrically with the coefficient of variation of reward or risk, in accord with some recent findings in humans3. Similarly, neurons in posterior cingulate cortex were also risksensitive, carrying information about both the direction of impending movement, as shown previously24,30, and the uncertainty of rewards associated with this movement. These findings both corroborate and extend the findings of a previous study16, in which posterior cingulate neurons were reported to be sensitive to reward size and predictability, manipulations that also presumably influenced subjective value. However, the previous study could not discern whether such modulations in neuronal activity reflected subjective preferences for larger or more surprising rewards. In the present study, the average reward value of each target was held constant, yet monkeys demonstrated clear 1225 © 2005 Nature Publishing Group http://www.nature.com/natureneuroscience ARTICLES preferences for one option over the other, thus permitting dissociation of subjective and objective value. Under these conditions, CGp neurons were sensitive to subjective target utility, consistent with recent findings in posterior parietal cortex, a premotor area that has been implicated in oculomotor decision making14,17. Recent studies have suggested that orienting decisions are computed, in part, by scaling neuronal responses by target value and then comparing them with a threshold41,42. Our data indicate that CGp neurons signal subjective biases for uncertain rewards (or, perhaps, the potential to receive a large reward) rather than objective target value. Thus, CGp appears to carry spatial information that is scaled by subjective preferences for particular patterns of reward outcome. Other investigators have shown recently that the responses of a population of midbrain dopamine neurons are selectively enhanced by reward uncertainty43. Such dopamine responses may indirectly facilitate neuronal activity in CGp through projections to anterior cingulate cortex, a major input to posterior cingulate cortex44. Several neuroimaging studies have also shown hemodynamic responses to outcome uncertainty in midline cortical areas such as anterior cingulate cortex, orbitofrontal cortex and precuneus45,46. Our data extend those findings, demonstrating a direct relationship between subjective preferences for uncertain rewards, or the opportunity to harvest a relatively large reward, and the activity of neurons thought to participate in the allocation of attention21,22. Neuronal activity in CGp reflects subjective preferences for risky target locations during fixation as well as before and after saccade onset, suggesting that this area contributes to visuospatial biases guiding orienting22,30,47. Our data are in sharp contrast with findings in posterior parietal cortex, where modulations by local fractional income14 or subjective desirability17 are found to emerge around the time of target onset and end after the eye movement. Therefore, posterior parietal cortex has been proposed to signal the relative value of potential eye movements17 but is unlikely to actually compute this value, as the signals seem to be ‘reset’ at the start of each trial14. In contrast, modulations in neuronal activity in CGp seem to persist across trials, and the timing of these modulations is consistent with a role in predicting and/or evaluating subjective value16,41. Thus, CGp may convey information about subjective target value that scales neuronal signals in parietal cortex. Our data also indicate that the spatial sensitivity of neurons in CGp is enhanced under conditions of risk or uncertainty. This result echoes a recent finding that the spatial selectivity of parietal neurons is greatest when target value is high14. Because both parietal and cingulate cortices have been implicated in the allocation of spatial attention, such enhancement may reflect heightened attention to regions of space with high subjective value48. We speculate that enhanced neuronal activity associated with risky rewards biases attention spatially, marking large payoffs as salient for guiding behavior48 and thereby favoring behavioral responses to risky targets. Such a link between risk preference, salience, attention and action has profound implications not only for oculomotor decision making, but also for why people and animals sometimes demonstrate irrational and even harmful preferences for risky behaviors. In conclusion, two monkeys were systematically risk prone when offered choices of targets associated with certain and uncertain fluid rewards. Posterior cingulate neurons were similarly risk sensitive, and their firing rates conveyed information about the direction of impending movements as well as the subjective utility of those movements. Moreover, the spatial sensitivity of CGp neurons was enhanced under conditions of high risk. Neurophysiological studies of risk preferences, as reported here, may serve as an important model for probing the 1226 neural processes that underlie pathological risk taking in individuals with addictions to drugs, sex, food or gambling. METHODS Surgical and training procedures. All procedures were approved by the Duke University Institutional Animal Care and Use Committee and were designed and conducted in compliance with the Public Health Service’s Guide for the Care and Use of Animals. Initially, a head restraint prosthesis and scleral search coil49 were implanted using standard surgical techniques30. Six weeks later, animals were habituated to head restraint and trained to perform oculomotor tasks for liquid rewards. A second surgical procedure was then performed to implant a stainless steel recording chamber (Crist Instruments) over posterior cingulate cortex at the intersection of the interaural and midsagittal planes. The chamber was kept sterile with regular antibiotic washes and sealed with sterile caps. Animals received analgesics and antibiotics after all surgeries. Behavioral techniques. Horizontal and vertical eye positions were sampled at 500 Hz (Riverbend Instruments) and recorded by computer (ryklinsoftware. com). Visual stimuli were LEDs (LEDtronics), which were illuminated to appear yellow, red or green to normal human observers, fixed on a tangent screen 144.78 cm (57 inches) from the animals’ eyes and forming a grid of points separated by 11, spanning 491 horizontally and 411 vertically. Behavioral datasets were collected for 12 sessions from both monkeys before physiological recording. A 300-ms broadband noise before juice delivery served as a secondary reinforcer on all correct trials. During visual gambling trials, one target was placed in the response field of the neuron under study, while the other target was placed diametrically opposite the fixation point. One target was associated with a ‘certain’ reward outcome of 150 ms access to juice on every trial, while the other ‘risky’ target was randomly rewarded with less than 150 ms on half of trials and greater than 150 ms on the other half of trials (mean ¼ 150 ms across trials). The locations of the certain and risky targets, as well as the coefficient of variation in reward for the risky target, were varied every 50 trials. Heart rate was measured at 12 Hz by pulse-oximetry (SurgiVet) on eight sessions for both monkeys. Two control experiments were performed for risk and novelty. First, for the risk control, visual gambling trials were as above with the following important difference: the risky target was associated with larger-than-average rewards on one-third of trials and smaller-than-average rewards on two-thirds of trials (as compared with one-half larger-than-average and one-half smaller-than-average rewards in standard visual gambling trials). Second, for the novelty control, reward size was held constant at 150-ms access to juice for both targets while novelty was introduced by systematically changing the color of one of the targets during reward delivery. The color of the ‘monotonous’ target remained yellow throughout trials, while the color of the ‘novel’ target randomly changed color from yellow to green on half of trials, and from yellow to red on the other half of trials. Microelectrode recording techniques. Single electrodes (Frederick Haer) were lowered under physiological guidance until the waveform of a single neuron was isolated. Individual action potentials were identified in hardware by time and amplitude criteria (BAK Electronics) and recorded by computer at 25 KHz. Neurons were selected for recording experiments on the basis of the quality of isolation and apparent task-sensitivity. Neuronal activity was first monitored during 100–400 single-target trials to identify the neuron’s response field and select appropriate target locations for subsequent visual gambling trials. Data were collected for 4 to 14 blocks of gambling trials for each neuron, depending on the duration and quality of isolation. Following some recording sessions, we confirmed the location of the electrode using a hand-held digital ultrasound device (Sonosite 180) placed against the recording chamber50. Ultrasound images taken in the sagittal plane showed that recordings were made in areas 23 and 31 in the cingulate gyrus and ventral bank of the cingulate sulcus, anterior to the intersection of the marginal and horizontal rami30. Analysis. Data were analyzed off-line using custom software (Eyemove, supported by K. Pearson, D. Sparks Laboratory, Baylor College of Medicine), which computed saccade direction, amplitude, latency, peak velocity and times VOLUME 8 [ NUMBER 9 [ SEPTEMBER 2005 NATURE NEUROSCIENCE © 2005 Nature Publishing Group http://www.nature.com/natureneuroscience ARTICLES of spike occurrence. For behavioral data, logistic regression was used to estimate the effects of experienced rewards and risk associated with prior choices on the probability of choosing the risky target. Statistics were computed using Statistica 6 or Matlab. Firing rates were measured for each trial during 12 200-ms intervals aligned with the time of an event during the trial: (i) 0–200 ms after the onset of fixation; (ii) 0–200 ms after the illumination of the eccentric target; (iii) 200–0 ms before target offset; (iv) 200–0 ms before movement onset; (v) 0–200 ms after movement onset; (vi) 200–400 ms after movement onset; (vii) 0–200 ms after the reinforcing noise burst; (viii) 200–0 ms before juice onset and (ix–xii) 0–200 ms, 200–400 ms, 400–600 ms and 600–800 ms after juice delivery. Analysis of firing rates focused on three 200 ms intervals in particular: 0–200 ms after the onset of fixation (fixation epoch), 200–0 ms before target offset (pre-movement epoch) and 200–400 ms after movement onset (post-movement epoch). Multiple regression was used to quantify the relationship between neuronal firing rate and target risk, independent of the effects of latency, amplitude and peak velocity of eye movements. In addition, neuronal activity was also analyzed as a function of subjective target utility, which was estimated by assuming that the experienced value of each target was the sum of both the received rewards and risk associated with prior choices. Experienced value for each target was incorporated into the model of subjective target utility by first multiplying this value by a weighting factor and then summing across up to ten previous trials. Weighting factors for the influence of experienced value at each trial lag were estimated by logistic regression of the effects of experienced target value on the probability of choosing the response field target. Aikake’s Information Criterion (AIC) was used to evaluate the inclusion of experienced target value for trials at different lags in the model. ACKNOWLEDGMENTS We thank G. Haghighian and J. Crowley for their contribution to the early stages of this work. We also thank S. Roberts for assistance in animal care and S. Huettel, J. Stowe, P. Glimcher, R. Deaner, J. Roitman, M. Bendiksby, S. Shepherd and A. Khera for valuable comments on the manuscript. Supported by the Klingenstein Foundation, the Duke Provost’s Common Fund and the National Eye Institute. COMPETING INTERESTS STATEMENT The authors declare that they have no competing financial interests. Received 5 May; accepted 18 July 2005 Published online at http://www.nature.com/natureneuroscience/ 1. Bateson, M. & Kacelnik, A. Starlings’ preferences for predictable and unpredictable delays to food. Anim. Behav. 53, 1129–1142 (1997). 2. Kahneman, D. & Tversky, A. Prospect theory: an analysis of decision under risk. Econometrica 47, 263–291 (1979). 3. Weber, E.U., Shafir, S. & Blais, A.R. Predicting risk sensitivity in humans and lower animals: risk as variance or coefficient of variation. Psychol. Rev. 111, 430–445 (2004). 4. Bernouilli, D. The Works (Birkhauser, Boston, 1758). 5. Rode, C., Cosmides, L., Hell, W. & Tooby, J. When and why do people avoid unknown probabilities in decisions under uncertainty? Testing some predictions from optimal foraging theory. Cognition 72, 269–304 (1999). 6. Bateson, M. Recent advances in our understanding of risk-sensitive foraging preferences. Proc. Nutr. Soc. 61, 509–516 (2002). 7. Von Neumann, J.V. & Morgenstern, O. Theory of Games and Economic Behavior (Princeton University Press, Princeton, New Jersey, 1944). 8. McClure, S.M. et al. Neural correlates of behavioral preference for culturally familiar drinks. Neuron 44, 379–387 (2004). 9. Arnauld, A. & Nichole, P. The Art of Thinking: Port-Royal Logic (Bobbs-Merrill, Indianapolis, 1662). 10. Stephens, D.W. & Krebs, J.R. Foraging Theory (Princeton University Press, Princeton, New Jersey, 1986). 11. Platt, M.L. & Glimcher, P.W. Neural correlates of decision variables in parietal cortex. Nature 400, 233–238 (1999). 12. Leon, M.I. & Shadlen, M.N. Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron 24, 415–425 (1999). 13. Kawagoe, R., Takikawa, Y. & Hikosaka, O. Expectation of reward modulates cognitive signals in the basal ganglia. Nat. Neurosci. 1, 411–416 (1998). 14. Sugrue, L.P., Corrado, G.S. & Newsome, W.T. Matching behavior and the representation of value in the parietal cortex. Science 304, 1782–1787 (2004). NATURE NEUROSCIENCE VOLUME 8 [ NUMBER 9 [ SEPTEMBER 2005 15. Lauwereyns, J., Watanabe, K., Coe, B. & Hikosaka, O. A neural correlate of response bias in monkey caudate nucleus. Nature 418, 413–417 (2002). 16. McCoy, A.N., Crowley, J.C., Haghighian, G., Dean, H.L. & Platt, M.L. Saccade reward signals in posterior cingulate cortex. Neuron 40, 1031–1040 (2003). 17. Dorris, M.C. & Glimcher, P.W. Activity in posterior parietal cortex is correlated with the relative subjective desirability of action. Neuron 44, 365–378 (2004). 18. Schultz, W. Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioural ecology. Curr. Opin. Neurobiol. 14, 139–147 (2004). 19. Roesch, M.R. & Olson, C.R. Neuronal activity related to reward value and motivation in primate frontal cortex. Science 304, 307–310 (2004). 20. Kacelnik, A. Normative and descriptive models of decision making: time discounting and risk sensitivity. Ciba Found. Symp. 208, 51–67 (1997). 21. Hopfinger, J.B., Buonocore, M.H. & Mangun, G.R. The neural mechanisms of top-down attentional control. Nat. Neurosci. 3, 284–291 (2000). 22. Small, D.M. et al. The posterior cingulate and medial prefrontal cortex mediate the anticipatory allocation of spatial attention. Neuroimage 18, 633–641 (2003). 23. Berman, R.A. et al. Cortical networks subserving pursuit and saccadic eye movements in humans: an FMRI study. Hum. Brain Mapp. 8, 209–225 (1999). 24. Olson, C.R., Musil, S.Y. & Goldberg, M.E. Single neurons in posterior cingulate cortex of behaving macaque: eye movement signals. J. Neurophysiol. 76, 3285–3300 (1996). 25. Rescorla, R.A. & Wagner, A.R. A theory of Pavlovian conditioning. Variations in the effectiveness of reinforcement and nonreinforcement. in Classical Conditioning II: Current Research and Theory (eds. Black, A.H. & Prokasy, W.F.) (Appleton-CenturyCrofts, New York, 1972). 26. Sutton, R.S. & Barto, A.G. Toward a modern theory of adaptive networks: expectation and prediction. Psychol. Rev. 88, 135–170 (1981). 27. Barraclough, D.J., Conroy, M.L. & Lee, D. Prefrontal cortex and decision making in a mixed-strategy game. Nat. Neurosci. 7, 404–410 (2004). 28. Caraco, T. White crowned sparrows (Zonotricha leucophrys): foraging preferences in a risky environment. Behav. Ecol. Sociobiol. 8, 820–830 (1983). 29. McGlothlin, W.H. Stability of choices among uncertain alternatives. Am. J. Psychol. 69, 604–615 (1956). 30. Dean, H.L., Crowley, J.C. & Platt, M.L. Visual and saccade-related activity in macaque posterior cingulate cortex. J. Neurophysiol. 92, 3056–3068 (2004). 31. Ladouceur, R., Sevigny, S., Blaszczynski, A., O’Connor, K. & Lavoie, M.E. Video lottery: winning expectancies and arousal. Addiction 98, 733–738 (2003). 32. Sharpe, L. Patterns of autonomic arousal in imaginal situations of winning and losing in problem gambling. J. Gambl. Stud. 20, 95–104 (2004). 33. Meyer, G. et al. Casino gambling increases heart rate and salivary cortisol in regular gamblers. Biol. Psychiatry 48, 948–953 (2000). 34. Bechara, A. & Damasio, H. Decision-making and addiction (part I): impaired activation of somatic states in substance dependent individuals when pondering decisions with negative future consequences. Neuropsychologia 40, 1675–1689 (2002). 35. Hikosaka, O., Sakamoto, M. & Usui, S. Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. J. Neurophysiol. 61, 814–832 (1989). 36. Marsh, B. & Kacelnik, A. Framing effects and risky decisions in starlings. Proc. Natl. Acad. Sci. USA 99, 3352–3355 (2002). 37. Kelley, A.E., Schochet, T. & Landry, C.F. Risk taking and novelty seeking in adolescence: introduction to part I. Ann. NY Acad. Sci. 1021, 27–32 (2004). 38. Doremus, T.L., Varlinskaya, E.I. & Spear, L.P. Age-related differences in elevated plus maze behavior between adolescent and adult rats. Ann. NY Acad. Sci. 1021, 427–430 (2004). 39. Chambers, R.A., Taylor, J.R. & Potenza, M.N. Developmental neurocircuitry of motivation in adolescence: a critical period of addiction vulnerability. Am. J. Psychiatry 160, 1041–1052 (2003). 40. Bickel, W.K., Giordano, L.A. & Badger, G.J. Risk-sensitive foraging theory elucidates risky choices made by heroin addicts. Addiction 99, 855–861 (2004). 41. McCoy, A.N. & Platt, M.L. Expectations and outcomes: decision-making in the primate brain. J. Comp. Physiol. A Neuroethol. Sens. Neural Behav. Physiol. 191, 201–211 (2005). 42. Gold, J.I. & Shadlen, M.N. Banburismus and the brain: decoding the relationship between sensory stimuli, decisions, and reward. Neuron 36, 299–308 (2002). 43. Fiorillo, C.D., Tobler, P.N. & Schultz, W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299, 1898–1902 (2003). 44. Vogt, B.A. & Pandya, D.N. Cingulate cortex of the rhesus monkey: II. Cortical afferents. J. Comp. Neurol. 262, 271–289 (1987). 45. Critchley, H.D., Mathias, C.J. & Dolan, R.J. Neural activity in the human brain relating to uncertainty and arousal during anticipation. Neuron 29, 537–545 (2001). 46. Dickhaut, J. et al. The impact of the certainty context on the process of choice. Proc. Natl. Acad. Sci. USA 100, 3536–3541 (2003). 47. Mesulam, M.M. Spatial attention and neglect: parietal, frontal and cingulate contributions to the mental representation and attentional targeting of salient extrapersonal events. Phil. Trans. R. Soc. Lond. B 354, 1325–1346 (1999). 48. Damasio, A. Descartes’ Error: Emotion, Reason, and the Human Brain (Avon, New York, 1995). 49. Judge, S.J., Richmond, B.J. & Chu, F.C. Implantation of magnetic search coils for measurement of eye position: an improved method. Vision Res. 20, 535–538 (1980). 50. Glimcher, P.W. et al. Application of neurosonography to experimental physiology. J. Neurosci. Methods 108, 131–144 (2001). 1227