An important part of human intelligence is the ability to use language. Humans learn how to use l... more An important part of human intelligence is the ability to use language. Humans learn how to use language in a society of language users, which is probably the most effective way to learn a language from the ground up. Principles that might allow an artificial agents to learn language this way are not known at present. Here we present a framework which begins to address this challenge. Our auto-catalytic, endogenous, reflective architecture (AERA) supports the creation of agents that can learn natural language by observation. We present results from two experiments where our S1 agent learns human communication by observing two humans interacting in a realtime mock television interview, using gesture and situated language. Results show that S1 can learn multimodal complex language and multimodal communicative acts, using a vocabulary of 100 words with numerous sentence formats, by observing unscripted interaction between the humans, with no grammar being provided to it a priori, and only high-level information about the format of the human interaction in the form of high-level goals of the interviewer and interviewee and a small ontology. The agent learns both the pragmatics, semantics, and syntax of complex sentences spoken by the human subjects on the topic of recycling of objects such as aluminum cans, glass bottles, plastic, and wood, as well as use of manual deictic reference and anaphora.
Four principal features of autonomous control systems are left both unaddressed and unaddressable... more Four principal features of autonomous control systems are left both unaddressed and unaddressable by present-day engineering methodologies: (1) The ability to operate effectively in environments that are only partially known at design time; (2) A level of generality that allows a system to reassess and redefine the fulfillment of its mission in light of unexpected constraints or other unforeseen changes in the environment; (3) The ability to operate effectively in environments of significant complexity; and (4) The ability to degrade gracefully— how it can continue striving to achieve its main goals when resources become scarce, or in light of other expected or unexpected constraining factors that impede its progress. We describe new methodological and engineering principles for addressing these shortcomings, that we have used to design a machine that becomes increasingly better at behaving in underspecified circumstances, in a goal-directed way, on the job, by modeling itself and i...
In this paper we propose a novel approach for the texture analysis-synthesis problem, with the pu... more In this paper we propose a novel approach for the texture analysis-synthesis problem, with the purpose to restore missing zones in greyscale images. Bit-plane decomposition is used, and a dictionary is build with bit-blocks statistics for each plane. Gaps are reconstructed with a conditional stochastic process, to propagate texture global features into the damaged area, using information stored in the dictionary. Our restoration method is simple, easy and fast, with very good results for a large set of textured images. Results are compared with a state-of-the-art restoration algorithm.
Recent theories of mindreading explain the recognition of action, intention, and belief of other ... more Recent theories of mindreading explain the recognition of action, intention, and belief of other agents in terms of generative architectures that model the causal relations between observables (e.g., observed movements) and their hidden causes (e.g., action goals and beliefs). Two kinds of probabilistic generative schemes have been proposed in cognitive science and robotics that link to a “theory theory” and “simulation theory” of mindreading, respectively. The former compares perceived actions to optimal plans derived from rationality principles and conceptual theories of others’ minds. The latter reuses one’s own internal (inverse and forward) models for action execution to perform a look-ahead mental simulation of perceived actions. Both theories, however, leave one question unanswered: how are the generative models – including task structure and parameters – learned in the first place? We start from Dennett’s “intentional stance” proposal and characterize it within generative theories of action and intention recognition. We propose that humans use an intentional stance as a learning bias that sidesteps the (hard) structure learning problem and bootstraps the acquisition of generative models for others’ actions. The intentional stance corresponds to a candidate structure in the generative scheme, which encodes a simplified belief-desire folk psychology and a hierarchical intention-to-action organization of behavior. This simple structure can be used as a proxy for the “true” generative structure of others’ actions and intentions and is continuously grown and refined – via state and parameter learning – during interactions. In turn – as our computational simulations show – this can help solve mindreading problems and bootstrap the acquisition of useful causal models of both one’s own and others’ goal-directed actions.
In order to have a robotic system able to effectively learn by imitation, and not merely reproduc... more In order to have a robotic system able to effectively learn by imitation, and not merely reproduce the movements of a human teacher, the system should have the capabilities of deeply understanding the perceived actions to be imitated. This paper deals with the development of cognitive architecture for learning by imitation in which a rich conceptual representation of the observed
Old printed photos are affected by several typical damages, due to age and bad preservation. &quo... more Old printed photos are affected by several typical damages, due to age and bad preservation. "Foxing" defects look like red-brownish spots onto the paper of the printed photo. Similar features can be seen in the digitized copies. In this paper we propose a set of low level descriptors to extract features from digitized photos affected by foxing. An image retrieval
This paper addresses the problem of automatic grasp synthesis of unknown planar objects. In other... more This paper addresses the problem of automatic grasp synthesis of unknown planar objects. In other words, we must compute points on the object's boundary to be reached by the robotic fingers such that the resulting grasp, among infinite possibilities, optimizes some given criteria. Objects to be grasped are represented as superellipses, a family of deformable 2D parametric functions. They can model a large variety of shapes occurring often in practice by changing a small number of parameters. The space of possible grasp ...
ABSTRACT We present a cognitive architecture whose main constituents are allowed to grow through ... more ABSTRACT We present a cognitive architecture whose main constituents are allowed to grow through a situated experience in the world. Such an architectural growth is bootstrapped from a minimal initial knowledge and the architecture itself is built around the biologically-inspired notion of internal models. The key idea, supported by findings in cognitive neuroscience, is that the same internal models used in overt goal-directed action execution can be covertly re-enacted in simulation to provide a unifying explanation to a number of apparently unrelated individual and social phenomena, such as state estimation, action and intention understanding, imitation learning and mindreading. Thus, rather than reasoning over abstract symbols, we rely on the biologically plausible processes firmly grounded in the actual sensorimotor experience of the agent. The article describes how such internal models are learned in the first place, either through individual experience or by observing and imitating other skilled agents, and how they are used in action planning and execution. Furthermore, we explain how the architecture continuously adapts its internal agency and how increasingly complex cognitive phenomena, such as continuous learning, prediction and anticipation, result from an interplay of simpler principles. We describe an early evaluation of our approach in a classical AI problem-solving domain: the Sokoban puzzle.
Congress of the Italian Association for Artificial Intelligence, 2005
In order to have a robotic system able to effectively learn by imita- tion, and not merely reprod... more In order to have a robotic system able to effectively learn by imita- tion, and not merely reproduce the movements of a human teacher, the system should have the capabilities of deeply understanding the perceived actions to be imitated. This paper deals with the development of a cognitive architecture for learning by imitation in which a rich conceptual representation of
2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2010
AbstractUnderstanding and efficiently representing skills is one of the most important problems ... more AbstractUnderstanding and efficiently representing skills is one of the most important problems in a general Programming by Demonstration (PbD) paradigm. We present Growing Hierarchi-cal Dynamic Bayesian Networks (GHDBN), an adaptive variant of the general ...
International Conference on Intelligent RObots and Systems - IROS, 2006
This paper presents the cognitive architecture Con-SCIS (conceptual space based cognitive imitati... more This paper presents the cognitive architecture Con-SCIS (conceptual space based cognitive imitation system), which tightly links low-level data processing with knowledge representation in the context of imitation learning. We use the word imitate to refer to the paradigm of program-level imitation: we are interested in the final effects of actions on objects, and not on the particular kinematic or dynamic
In order to have a robotic system able to effectively learn by imitation, and not merely reproduc... more In order to have a robotic system able to effectively learn by imitation, and not merely reproduce the movements of a human teacher, the system should have the capabilities of deeply understanding the perceived actions to be imitated. This paper deals with the development of a cognitive architecture for learning by imitation in which a rich conceptual representation of the observed actions is built. The purpose of the following discussion is to show how the same conceptual representation can be used both in a bottom-up approach, in order to learn sequences of actions by imitation learning paradigm, and in a topdown approach, in order to anchor the symbolical representations to the perceptual activities of the robotic system. The proposed architecture has been tested on the robotic system composed of a PUMA 200 industrial manipulator and an anthropomorphic robotic hand. The system demonstrated the ability to learn and imitate a set of movement primitives acquired through the vision system for simple manipulative purposes.
... 306 Antonio Chella, Haris Dindo and Ignazio Infantino of each detected object. Objects are mo... more ... 306 Antonio Chella, Haris Dindo and Ignazio Infantino of each detected object. Objects are modelled with superellipses, a natural generali-sation of ellipses. They can model a large variety of regular shapes by changing a small number of parameters (Jaklic, Leonardis, & ...
The paper deals with a system for simultaneous people tracking and posture recognition in clutter... more The paper deals with a system for simultaneous people tracking and posture recognition in cluttered environments in the context of human-robot interaction. We adopt no particular assumptions on the movement of a person nor on its appearance, making the system suitable to several real-world applications. The system can be roughly subdivided into two highly correlated phases: tracking and recognition. The
In order to have a robotic system able to effectively learn by imitation, and not merely reproduc... more In order to have a robotic system able to effectively learn by imitation, and not merely reproduce the movements of a human teacher, the system should have the capabilities of deeply understanding the perceived actions to be imitated. This paper deals with the development of a cognitive framework for learning by imitation in which a rich conceptual representation of the
The paper deals with an artificial-vision based system for simultaneous people tracking and postu... more The paper deals with an artificial-vision based system for simultaneous people tracking and posture recognition in the context of human-computer interaction. We adopt no particular assumptions on the movement of a person and on its appearance, making the system suitable to several real-world applications. The system can be roughly subdivided into two highly correlated phases: tracking and recognition. The tracking
IEEE Transactions on Systems, Man, and Cybernetics, 2005
The paper deals with the design and implementation of a visual control of a robotic system compos... more The paper deals with the design and implementation of a visual control of a robotic system composed of a dexterous hand and a video camera. The aim of the proposed system is to reproduce the movements of a human hand in order to learn complex manipulation tasks or to interact with the user. A novel algorithm for a robust and
We propose a system able to represent others' actions as well as to internally simulate their lik... more We propose a system able to represent others' actions as well as to internally simulate their likely continuation from a partial observation. The approach presented here is the first step towards a more ambitious goal of endowing an artificial agent with the ability to recognise and predict others' intentions. Our approach is based on the associative self-organising map, a variant of the self-organising map capable of learning to associate its activity with different inputs over time, where inputs are processed observations of others' actions. We have evaluated our system in two different experimental scenarios obtaining promising results: the system demonstrated an ability to learn discriminable representations of actions, to recognise novel input, and to simulate the likely continuation of partially seen actions.
An important part of human intelligence is the ability to use language. Humans learn how to use l... more An important part of human intelligence is the ability to use language. Humans learn how to use language in a society of language users, which is probably the most effective way to learn a language from the ground up. Principles that might allow an artificial agents to learn language this way are not known at present. Here we present a framework which begins to address this challenge. Our auto-catalytic, endogenous, reflective architecture (AERA) supports the creation of agents that can learn natural language by observation. We present results from two experiments where our S1 agent learns human communication by observing two humans interacting in a realtime mock television interview, using gesture and situated language. Results show that S1 can learn multimodal complex language and multimodal communicative acts, using a vocabulary of 100 words with numerous sentence formats, by observing unscripted interaction between the humans, with no grammar being provided to it a priori, and only high-level information about the format of the human interaction in the form of high-level goals of the interviewer and interviewee and a small ontology. The agent learns both the pragmatics, semantics, and syntax of complex sentences spoken by the human subjects on the topic of recycling of objects such as aluminum cans, glass bottles, plastic, and wood, as well as use of manual deictic reference and anaphora.
Four principal features of autonomous control systems are left both unaddressed and unaddressable... more Four principal features of autonomous control systems are left both unaddressed and unaddressable by present-day engineering methodologies: (1) The ability to operate effectively in environments that are only partially known at design time; (2) A level of generality that allows a system to reassess and redefine the fulfillment of its mission in light of unexpected constraints or other unforeseen changes in the environment; (3) The ability to operate effectively in environments of significant complexity; and (4) The ability to degrade gracefully— how it can continue striving to achieve its main goals when resources become scarce, or in light of other expected or unexpected constraining factors that impede its progress. We describe new methodological and engineering principles for addressing these shortcomings, that we have used to design a machine that becomes increasingly better at behaving in underspecified circumstances, in a goal-directed way, on the job, by modeling itself and i...
In this paper we propose a novel approach for the texture analysis-synthesis problem, with the pu... more In this paper we propose a novel approach for the texture analysis-synthesis problem, with the purpose to restore missing zones in greyscale images. Bit-plane decomposition is used, and a dictionary is build with bit-blocks statistics for each plane. Gaps are reconstructed with a conditional stochastic process, to propagate texture global features into the damaged area, using information stored in the dictionary. Our restoration method is simple, easy and fast, with very good results for a large set of textured images. Results are compared with a state-of-the-art restoration algorithm.
Recent theories of mindreading explain the recognition of action, intention, and belief of other ... more Recent theories of mindreading explain the recognition of action, intention, and belief of other agents in terms of generative architectures that model the causal relations between observables (e.g., observed movements) and their hidden causes (e.g., action goals and beliefs). Two kinds of probabilistic generative schemes have been proposed in cognitive science and robotics that link to a “theory theory” and “simulation theory” of mindreading, respectively. The former compares perceived actions to optimal plans derived from rationality principles and conceptual theories of others’ minds. The latter reuses one’s own internal (inverse and forward) models for action execution to perform a look-ahead mental simulation of perceived actions. Both theories, however, leave one question unanswered: how are the generative models – including task structure and parameters – learned in the first place? We start from Dennett’s “intentional stance” proposal and characterize it within generative theories of action and intention recognition. We propose that humans use an intentional stance as a learning bias that sidesteps the (hard) structure learning problem and bootstraps the acquisition of generative models for others’ actions. The intentional stance corresponds to a candidate structure in the generative scheme, which encodes a simplified belief-desire folk psychology and a hierarchical intention-to-action organization of behavior. This simple structure can be used as a proxy for the “true” generative structure of others’ actions and intentions and is continuously grown and refined – via state and parameter learning – during interactions. In turn – as our computational simulations show – this can help solve mindreading problems and bootstrap the acquisition of useful causal models of both one’s own and others’ goal-directed actions.
In order to have a robotic system able to effectively learn by imitation, and not merely reproduc... more In order to have a robotic system able to effectively learn by imitation, and not merely reproduce the movements of a human teacher, the system should have the capabilities of deeply understanding the perceived actions to be imitated. This paper deals with the development of cognitive architecture for learning by imitation in which a rich conceptual representation of the observed
Old printed photos are affected by several typical damages, due to age and bad preservation. &quo... more Old printed photos are affected by several typical damages, due to age and bad preservation. "Foxing" defects look like red-brownish spots onto the paper of the printed photo. Similar features can be seen in the digitized copies. In this paper we propose a set of low level descriptors to extract features from digitized photos affected by foxing. An image retrieval
This paper addresses the problem of automatic grasp synthesis of unknown planar objects. In other... more This paper addresses the problem of automatic grasp synthesis of unknown planar objects. In other words, we must compute points on the object's boundary to be reached by the robotic fingers such that the resulting grasp, among infinite possibilities, optimizes some given criteria. Objects to be grasped are represented as superellipses, a family of deformable 2D parametric functions. They can model a large variety of shapes occurring often in practice by changing a small number of parameters. The space of possible grasp ...
ABSTRACT We present a cognitive architecture whose main constituents are allowed to grow through ... more ABSTRACT We present a cognitive architecture whose main constituents are allowed to grow through a situated experience in the world. Such an architectural growth is bootstrapped from a minimal initial knowledge and the architecture itself is built around the biologically-inspired notion of internal models. The key idea, supported by findings in cognitive neuroscience, is that the same internal models used in overt goal-directed action execution can be covertly re-enacted in simulation to provide a unifying explanation to a number of apparently unrelated individual and social phenomena, such as state estimation, action and intention understanding, imitation learning and mindreading. Thus, rather than reasoning over abstract symbols, we rely on the biologically plausible processes firmly grounded in the actual sensorimotor experience of the agent. The article describes how such internal models are learned in the first place, either through individual experience or by observing and imitating other skilled agents, and how they are used in action planning and execution. Furthermore, we explain how the architecture continuously adapts its internal agency and how increasingly complex cognitive phenomena, such as continuous learning, prediction and anticipation, result from an interplay of simpler principles. We describe an early evaluation of our approach in a classical AI problem-solving domain: the Sokoban puzzle.
Congress of the Italian Association for Artificial Intelligence, 2005
In order to have a robotic system able to effectively learn by imita- tion, and not merely reprod... more In order to have a robotic system able to effectively learn by imita- tion, and not merely reproduce the movements of a human teacher, the system should have the capabilities of deeply understanding the perceived actions to be imitated. This paper deals with the development of a cognitive architecture for learning by imitation in which a rich conceptual representation of
2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2010
AbstractUnderstanding and efficiently representing skills is one of the most important problems ... more AbstractUnderstanding and efficiently representing skills is one of the most important problems in a general Programming by Demonstration (PbD) paradigm. We present Growing Hierarchi-cal Dynamic Bayesian Networks (GHDBN), an adaptive variant of the general ...
International Conference on Intelligent RObots and Systems - IROS, 2006
This paper presents the cognitive architecture Con-SCIS (conceptual space based cognitive imitati... more This paper presents the cognitive architecture Con-SCIS (conceptual space based cognitive imitation system), which tightly links low-level data processing with knowledge representation in the context of imitation learning. We use the word imitate to refer to the paradigm of program-level imitation: we are interested in the final effects of actions on objects, and not on the particular kinematic or dynamic
In order to have a robotic system able to effectively learn by imitation, and not merely reproduc... more In order to have a robotic system able to effectively learn by imitation, and not merely reproduce the movements of a human teacher, the system should have the capabilities of deeply understanding the perceived actions to be imitated. This paper deals with the development of a cognitive architecture for learning by imitation in which a rich conceptual representation of the observed actions is built. The purpose of the following discussion is to show how the same conceptual representation can be used both in a bottom-up approach, in order to learn sequences of actions by imitation learning paradigm, and in a topdown approach, in order to anchor the symbolical representations to the perceptual activities of the robotic system. The proposed architecture has been tested on the robotic system composed of a PUMA 200 industrial manipulator and an anthropomorphic robotic hand. The system demonstrated the ability to learn and imitate a set of movement primitives acquired through the vision system for simple manipulative purposes.
... 306 Antonio Chella, Haris Dindo and Ignazio Infantino of each detected object. Objects are mo... more ... 306 Antonio Chella, Haris Dindo and Ignazio Infantino of each detected object. Objects are modelled with superellipses, a natural generali-sation of ellipses. They can model a large variety of regular shapes by changing a small number of parameters (Jaklic, Leonardis, & ...
The paper deals with a system for simultaneous people tracking and posture recognition in clutter... more The paper deals with a system for simultaneous people tracking and posture recognition in cluttered environments in the context of human-robot interaction. We adopt no particular assumptions on the movement of a person nor on its appearance, making the system suitable to several real-world applications. The system can be roughly subdivided into two highly correlated phases: tracking and recognition. The
In order to have a robotic system able to effectively learn by imitation, and not merely reproduc... more In order to have a robotic system able to effectively learn by imitation, and not merely reproduce the movements of a human teacher, the system should have the capabilities of deeply understanding the perceived actions to be imitated. This paper deals with the development of a cognitive framework for learning by imitation in which a rich conceptual representation of the
The paper deals with an artificial-vision based system for simultaneous people tracking and postu... more The paper deals with an artificial-vision based system for simultaneous people tracking and posture recognition in the context of human-computer interaction. We adopt no particular assumptions on the movement of a person and on its appearance, making the system suitable to several real-world applications. The system can be roughly subdivided into two highly correlated phases: tracking and recognition. The tracking
IEEE Transactions on Systems, Man, and Cybernetics, 2005
The paper deals with the design and implementation of a visual control of a robotic system compos... more The paper deals with the design and implementation of a visual control of a robotic system composed of a dexterous hand and a video camera. The aim of the proposed system is to reproduce the movements of a human hand in order to learn complex manipulation tasks or to interact with the user. A novel algorithm for a robust and
We propose a system able to represent others' actions as well as to internally simulate their lik... more We propose a system able to represent others' actions as well as to internally simulate their likely continuation from a partial observation. The approach presented here is the first step towards a more ambitious goal of endowing an artificial agent with the ability to recognise and predict others' intentions. Our approach is based on the associative self-organising map, a variant of the self-organising map capable of learning to associate its activity with different inputs over time, where inputs are processed observations of others' actions. We have evaluated our system in two different experimental scenarios obtaining promising results: the system demonstrated an ability to learn discriminable representations of actions, to recognise novel input, and to simulate the likely continuation of partially seen actions.
Uploads
Papers by Haris Dindo