Actor Critic

In order to overcome the difficulty of learning a global optimal policy caused by maximization bias in a continuous space, an actor-critic algorithm for cross evaluation of double value function is proposed. Two independent value functions make the critique closer to the real value function, and the actor is guided by a crossover function to choose its optimal actions. Cross evaluation of value functions avoids the policy jitter phenomenon behaved by greedy optimization methods in continuous spaces. The algorithm is more robust than CACLA learning algorithm, and the experimental results show that our algorithm is smoother and the stability of policy is improved obviously under the condition that the computation remains almost unchanged.

The purpose of this paper is to present a general mechanistic framework for analyzing causal representational claims, and offer a way to distinguish genuinely representational explanations from those that invoke representations for honorific purposes. It is usually agreed that rats are capable of navigation (even in complete darkness, and when immersed in a water maze) because they maintain a cognitive map of their environment. Exactly how and why their neural states give rise to mental representations is a matter of an ongoing debate. I will show that anticipatory mechanisms involved in rats’ evaluation of possible routes give rise to satisfaction conditions of contents, and this is why they are representationally relevant for explaining and predicting rats’ behavior. I argue that a naturalistic account of satisfaction conditions of contents answers the most important objections of antirepresentationalists.

There is a well-documented relationship between an infant's early ability to follow the gaze of an adult and their later receptive and expressive language. However, what is less well known is the degree to which a child's ability to learn new words through gaze following predicts later vocabulary knowledge. The current study explored this question by giving 18-month-old infants a word-learning task that required them to follow the gaze of a speaker in order to determine the referent of a novel word. Vocabulary measures were also taken when children were 18, 24 and 30 months old. Results showed that infants' scores on the word-learning task at 18 months were significantly related to their receptive and expressive vocabulary scores both concurrently and longitudinally. These results suggest that the ability to follow gaze direction to identify unfamiliar words' meanings is one of the key strategies used by infants to build their vocabulary.

This chapter explores the resolution of Quine’s word-learning conundrum: How young children identify the meanings of the words they hear in the utterances of expert language users. Various strategies are available to young word learners for solving this dilemma. The authors consider how the acquisition of each of these strategies might be underpinned by the same domain-general ability to detect and store associations between words and the environment in which they are heard. Drawing on examples of both socio-pragmatic and linguistic cues to words’ meanings, they explore how the young child’s early experiences with words might provide a platform for the discovery of more complex word-learning strategies, facilitating the learning of further new words. The chapter outlines specific predictions of the experience-driven model of the evolution of the toddler’s word-learning toolkit that might be explored through experimental and/or computational investigations.

Actor-Critic based approaches were among the first to address reinforcement learning in a general setting. Recently, these algorithms have gained renewed interest due to their generality, good convergence properties, and possible biological relevance. In this paper, we introduce an online temporal difference based actor-critic algorithm which is proved to converge to a neighborhood of a local maximum of the average reward. Linear function approximation is used by the critic in order estimate the value function, and the temporal difference signal, which is passed from the critic to the actor. The main distinguishing feature of the present convergence proof is that both the actor and the critic operate on a similar time scale, while in most current convergence proofs they are required to have very different time scales in order to converge. Moreover, the same temporal difference signal is used to update the parameters of both the actor and the critic. A limitation of the proposed approach, compared to results available for two time scale convergence, is that convergence is guaranteed only to a neighborhood of an optimal value, rather to an optimal value itself. The single time scale and identical temporal difference signal used by the actor and the critic, may provide a step towards constructing more biologically realistic models of reinforcement learning in the brain.

In this article, a simple CPG network is shown to model early infant walking, in particular the onset of independent walking. The difference between early infant walking and early adult walking is addressed with respect to the underlying neurophysiology and evaluated according to gait attributes. Based on this, we successfully model the early infant walking gait on the NAO robot and compare its motion dynamics and performance to those of infants. Our model is able to capture the core properties of early infant walking. We identify differences in the morphologies between the robot and infant and the effect of this on their respective performance. In conclusion, early infant walking can be seen to develop as a function of the CPG network and morphological characteristics.

This paper presents a computational model of the emergence of gaze following based on reinforcement learning. The model replicates the development of gaze following in human infants as observed by Butterworth and colleagues: while young infants follow gaze to targets within their field of view, only at older ages do they reliably follow gaze to targets outside their field of view. Also, while young infants sometimes stop gaze following at intermediate distracters along the scan path (but at the wrong depth) between the ...