Decision-making in a neural network model of the basal ganglia

HAL is a multidisciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Current Opinion in Neurobiology, 2001

Trends in Neurosciences, 2004

With the rapid accumulation of neuroscientific data comes a pressing need to develop models that can explain the computational processes performed by the basal ganglia. Relevant biological information spans a range of structural levels, from the activity of neuronal membranes to the role of the basal ganglia in overt behavioural control. This viewpoint presents a framework for understanding the aims, limitations and methods for testing of computational models across all structural levels. We identify distinct modelling strategies that can deliver important and complementary insights into the nature of problems the basal ganglia have evolved to solve, and describe methods that are used to solve them.

Movement Disorders, 2000

Computer simulation studies and mathematical analysis of models of the basal ganglia are being used increasingly to explore theories of basal ganglia function. We review the implications of these new models for a general understanding of basal ganglia function in normal as well as in diseased brains. The focus is on their functional similarities rather than on the details of mathematical methodologies and simulation techniques. Most of the models suggest a vital role for the basal ganglia in learning. Although this interest in learning is partly driven by experimental results associating the acute firing of dopamine cells with reward prediction in monkeys, some of the models have preceded the electrophysiological results. Another common theme of the models is selection. In this case, the striatum is seen as detecting and selecting cortical contexts for access to basal ganglia output. Although the behavioral consequences of this selection are hard to define, the models provide frameworks within which to explore these ideas empirically. This provides a means of refining our understanding of basal ganglia function and to consider dysfunction within the new logical frameworks.

Current Opinion in Neurobiology, 1997

Over the past year, a number of conceptual and mathematical models of the basal ganglia and their interactions with other areas of the brain have appeared in the literature. Even though the models each differ in significant ways, several computational principles, such as convergence, recurrence and competition, appear to have emerged as common themes of information processing in the basal ganglia. Simulation studies of these models have provoked new types of questions at the many levels of inquiry linking biophysics to behavior.

Behavioural brain research, 2009

2002

A large number of computational models of information processing in the basal ganglia have been developed in recent years. Prominent in these are actor-critic models of basal ganglia functioning, which build on the strong resemblance between dopamine neuron activity and the temporal difference prediction error signal in the critic, and between dopamine-dependent long-term synaptic plasticity in the striatum and learning guided by a prediction error signal in the actor. We selectively review several actor -critic models of the basal ganglia with an emphasis on two important aspects: the way in which models of the critic reproduce the temporal dynamics of dopamine firing, and the extent to which models of the actor take into account known basal ganglia anatomy and physiology. To complement the efforts to relate basal ganglia mechanisms to reinforcement learning (RL), we introduce an alternative approach to modeling a critic network, which uses Evolutionary Computation techniques to 'evolve' an optimal RL mechanism, and relate the evolved mechanism to the basic model of the critic. We conclude our discussion of models of the critic by a critical discussion of the anatomical plausibility of implementations of a critic in basal ganglia circuitry, and conclude that such implementations build on assumptions that are inconsistent with the known anatomy of the basal ganglia. We return to the actor component of the actor -critic model, which is usually modeled at the striatal level with very little detail. We describe an alternative model of the basal ganglia which takes into account several important, and previously neglected, anatomical and physiological characteristics of basal ganglia -thalamocortical connectivity and suggests that the basal ganglia performs reinforcementbiased dimensionality reduction of cortical inputs. We further suggest that since such selective encoding may bias the representation at the level of the frontal cortex towards the selection of rewarded plans and actions, the reinforcement-driven dimensionality reduction framework may serve as a basis for basal ganglia actor models. We conclude with a short discussion of the dual role of the dopamine signal in RL and in behavioral switching. q : S 0 8 9 3 -6 0 8 0 ( 0 2 ) 0 0 0 4 7 -3

Biological Cybernetics, 2001

We present a biologically plausible model of processing intrinsic to the basal ganglia based on the computational premise that action selection is a primary role of these central brain structures. By encoding the propensity for selecting a given action in a scalar value (the salience), it is shown that action selection may be re-cast in terms of signal selection. The generic properties of signal selection are defined and neural networks for this type of computation examined. A comparison between these networks and basal ganglia anatomy leads to a novel functional decomposition of the basal ganglia architecture into `selection' and `control' pathways. The former pathway performs the selection per se via a feedforward off-centre on-surround network. The control pathway regulates the action of the selection pathway to ensure its effective operation, and synergistically complements its dopaminergic modulation. The model contrasts with the prevailing functional segregation of basal ganglia into `direct' and `indirect' pathways.

HAL (Le Centre pour la Communication Scientifique Directe), 2021

Frontiers in Systems Neuroscience, 2014

Keywords: basal ganglia, dopamine, Parkinson's disease (PD), computational modeling, animal studies, human imaging studies, deep brain stimulation Frontiers in Systems Neuroscience www.frontiersin.org