Academia.eduAcademia.edu

Cooperation in Corvids: A Simulative Study with Evolved Robot

2010, Artificial Life and Evolutionary Computation - Proceedings of Wivace 2008

June 9, 2008 15:15 WSPC - Proceedings Trim Size: 9in x 6in wiv 1 Cooperation in corvids: a simulative study with evolved robot Orazio Miglino, Michela Ponticorvo∗ , Davide Donetto Natural and Artificial Cognition Laboratory, Department of Relational Sciences, University of Naples ”Federico II” Naples, Italy Institute of Cognitive Sciences and Technologies, CNR, Rome ∗ E-mail: [email protected] www.nac.unina.it Stefano Nolfi Institute of Cognitive Sciences and Technologies, CNR, Rome Paolo Zucca Laboratory of Animal Cognition and Comparative Neuroscience, Department of Psychology, University of Trieste and Faculty of Veterinary Medicine, University of Teramo 1. Introduction Corvids (Corvidae) include various birds species characterized by high complexity in cognitive functions: they can be compared with primates both on brain relative dimensions and on social organization complexity. They are capable of long term cache recovery, tool manipulation and social reasoning and we can observe them in dyads as well as in small or large colonies. Corvids are also able to cooperate in order to obtain a goal. Cooperation has attracted attention from many scholars coming from different disciplines such as psychology, sociology, anthropology, economics, that study human behaviour and ethology, behavioural ecology and evolutionary ecology that are interested in nonhuman organisms interactions (Connor 1995; Sachs et al. 2004; Bshary and Bronstein 2005, Noe and Hammerstein 1994, 1995). In the last twenty years, as underlined by Noe (2006) in his quite recent review on the theme, many studies have been run on cooperation observing two conspecifics that could get a reward through cooperation, mainly addressed by ”three different motivations: (1) detecting the mechanistic ba- June 9, 2008 15:15 WSPC - Proceedings Trim Size: 9in x 6in wiv 2 sis of naturally occurring forms of cooperation; (2) analyzing behavioural strategies specific to cooperation; and (3) testing game-theoretical models” (Noe, 2006). In many cases these latter experiments study cooperation by using very abstract and artificial conditions to which animals, vertebrates in many cases, must be trained to, thus making it difficult to distinguish if dyads are ”coordinated trough communication or acting apart together”(ibidem). It seems therefore quite relevant trying to understand how communication allows dyads to cooperate indeed. This issue can be approached to in a comparative natural and artificial behavioural science (Ponticorvo et alii, 2006) in which artificial organisms, such as robots or simulated agents are compared with natural organisms. In our approach we use Artificial Intelligence and Robotics tools to build artificial systems that are able to simulate, at least for some aspects, animal or human behaviour. This methodology allows us to deal with theoretical goals, because the reconstruction, both in simulation or with physical artifacts, of a model about a certain phenomenon allows to verify synthetically its explicatory potential. In particular we use Evolutionary Robotics techniques(Nolfi and Floreano, 2000), allowing the robot-environment system to self-organize and then analyze how it came to a solution. This methodology, in recent years, has been widely applied to the emergence of communication(for a review see: Cangelosi and Parisi, 2002; Kirby, 2002; Steels, 2003; Wagner et al., 2003; Nolfi, 2005). In this paper we follow the theoretical and methodological approach used by Di Paolo (1997, 2000), Quinn (2001), Quinn et al. (2001), Baldassarre et al. (2002), Trianni and Dorigo (2006), Marocco and Nolfi (2007) with two main differences. First of all, these model are mainly ’idea models’, in which evolutionary techniques are used to reproduce very general phenomena, while we have worked on ’data models’: in this kind of models the artificial organisms accurately reproduce quantitative observations (behavioral indices) of animal behavior in well-defined experimental set-ups, as we are going to describe. The second difference is that our main focus is on cooperation: in other words we are interested in understanding how the emergence of communication leads to the emergence of cooperation. In the present study we propose a model to study how emerging communication leads to cooperation in the ’loose string’ paradigm derived from the most popular paradigm used in game-theoretical model: the Prisoner’s Dilemma, applied to comparative research (Ashlock et alii, 1996; Clements and Stephens, 1995; Gardner et alii, 1984). In the ’loose string’ task two June 9, 2008 15:15 WSPC - Proceedings Trim Size: 9in x 6in wiv 3 agents, for example corvids, must cooperate to obtain a reward, i.e. food, which is clearly visible, but not directly reachable. The dyad gets the reward if the two tips of a string are pulled at the same time. In the present study we model this task with artificial organisms to verify the emergence and maintenance of cooperation in artificial organisms. 2. Method 2.1. The loose string task In the loose string task two members of a dyad are trained to pull a string to reach a reward. In a first phase, the agents, for example, corvids such as rooks (see works by Ronald Noe and Christelle Scheid), are trained separately to pull the string which allows the bird the get the food by itself. In the cooperation testing phase, the two birds could get the reward only if they pulled the string at the same time. In this comparative study the researchers observed if the one bird pulled the string alone or two birds pulled it together, thus successfully accomplishing the task. We reproduced this natural experimental task with simulated and real robots. 2.2. Experimental setup The experimental setup involves two robots situated in an arena consisting of a square room where robots begin each trial and of a wide corridor with a target area and some landmarks in it. Once they’ve reached the target area, robots have to drive towards the same landmark. This task is derived by the ”loose string” task described above and represents a situation in which the robots should coordinate themselves/cooperate to get a reward. 2.3. The environment and the robots The environment consists of a 150x150 cm square arena joint with a 90x90 cm square arena both surrounded by walls (Fig.1). The corridor presents a target area and three landmarks whose positions are randomly assigned when the robots enter in the area. The robots are two e-Puck robots (Mondada and Bonani, 2007, see Fig.2) with a diameter of 7.5 cm provided with 2 motors which control the 2 corresponding wheels, 8 infrared proximity sensors located around the robot’s body, a VGA camera with a view field of 36 degrees pointing in the direction of forward motion and a LED ring June 9, 2008 15:15 WSPC - Proceedings Trim Size: 9in x 6in wiv 4 on the circumference. The camera and the LED ring can be used to send and receive signals. Fig. 1. The environment. There is a big square arena where the robots start their trials. When they are on the target area in the small arena three landmarks appears and robots have to reach the same landmark to get a reward Fig. 2. The e-Puck robot with 2 motors, 8 infrared sensors, a camera and the LED ring June 9, 2008 15:15 WSPC - Proceedings Trim Size: 9in x 6in wiv 5 2.4. The neural controller The neural controllers of each robot are provided with sensory neurons, internal neurons with recurrent connections, motor neurons and some signal neurons. These neurons allow to receive and produce signals that can be perceived by another robot. In Fig. 3 we present schematically the neural architecture we used in our simulation. Fig. 3. The neural network: in the sensory layer there are neurons that encode activation of infrared sensors and signal units; in the hidden layers there are 8 hidden neurons with recurrent connections; in the output layer there are two units that control wheels and one signalling unit 2.5. The evolutionary algorithm An evolutionary technique is used to set the weights of the robots’ neural controller. The initial population consists of 100 randomly generated genotypes that encode the connection weights of 100 corresponding neural networks. Each genotype is translated into 2 identical neural controllers which are embodied in 2 corresponding robots situated in the environment (i.e. teams are homogeneous). The 20 best genotypes of each generation are allowed to reproduce by generating 5 copies each, with 2% of their bits replaced with a new randomly selected value. The evolutionary process lasts many generations (i.e. the process of testing, selecting and reproduc- June 9, 2008 15:15 WSPC - Proceedings Trim Size: 9in x 6in wiv 6 ing robots is iterated 1000 times). The experiment is replicated 20 times each consisting of 4 trials with 4 different starting positions in the corners of the square room. We used the following fitness function, in pseudo-code, to evolve robots: If (two robots are on target area) { Landmarks appear; If (two robots are close to the same landmark and therefore close to each other) Fitness +=1; } 3. Results and Conclusions Results show that cooperation between robots is regulated by interaction between robots, with communication as a medium. In our simulative scenario the emergence of communication leads to a coordinated cooperation behavior that is somewhat similar to cooperation observed in natural organisms as corvids. We considered some behavioural indexes that were quite similar to the ones used in the work with corvids (private communication), namely if one robot entered the target area before the other. Then we observed whether one robot reached the landmark alone or if the two robots reached the landmark together and consequently if they successfully accomplished the task. The evolved dyads proved to be able to accomplish the task, using different strategies. Let’s now analyze the prototypical strategy of one the best performing dyad, e.g. that gets one of the highest fitness score. This good-performing dyad uses the following signals and behaviors. The first kind of signal that these robots use has the function to communicate if one robot has reached the target area. The range of this signal varies in different dyads (for these robots is in the range [0.6 to 0.8], while for some others is in the range [0.2 to 0.4]). This signal is strictly connected to the input pattern that can be perceived in the target area and induce the robot outside the area to get into it. In same trials, the two robots arrive inside the arena at almost the same time. The second kind of signal is used to effectively coordinate: when the robot produces and receives signals at the same time, meaning that both robots are on the target area, they move toward the landmark that they can see with the camera. This movement is often leading-following: one robot moves toward the landmark and the other follows it (note that the robot can perceive the other robot with the June 9, 2008 15:15 WSPC - Proceedings Trim Size: 9in x 6in wiv 7 same camera as the landmarks and can distinguish the front and the rear of its companion). When they are close to the landmark, where they receive the reward, they stay close to the landmark and close to each other. The robots also show an obstacle-avoidance behaviour that allows robots not to bump into walls and into each other and is produced when the frontal infrared sensors of the robot are activated. In this dyad, thus, the emergence of communicative signals that allow the exchange between robots, leads to the emergence of this simple form of cooperation. It’s worth noting that in the present study we analyze dyads of robots which are embodied and situated and that autonomously develop communication while they interact with a physical environment. This attempt to study the evolution of communication and cooperation through computational and robotic models has the important advantage to study how communication signals emerge from the grounding in robots’sensory-motor system and how cooperation can evolve and adapt to variations of the environment made up by physical environment and the other robot in the dyad. In this paper, we use Artificial Life systems as labs in which to test hypothesis to understand a general function, in this case cooperation comparing different living forms, biological organisms with artificial organisms. These preliminary results require a wider investigation in order to understand the artificial neural mechanisms underlying these behavioural modules that result in a cooperative behaviour. June 9, 2008 15:15 WSPC - Proceedings Trim Size: 9in x 6in wiv 8 Acknowledgements This work was supported by Cooperation in Corvids (COCOR), Collaborative Research Project (CRP) in the framework of the ESF-EUROCORES programme TECT (The Evolution of Cooperation and Trading). References 1. Connor, R. C. (1995). Altruism among non-relatives: alternatives to the Prisoners Dilemma. Trends in Ecology and Evolution, 10, 8486. 2. Sachs, J. L., Mueller, U. G., Wilcox, T. P., Bull, J. J. (2004). The evolution of cooperation. Quarterly Review of Biology, 79, 135160. 3. Bshary, R., Bronstein, J. L. (2005). Game structures in mutualistic interactions: what can the evidence tell us about the kind of models we need? Advances in the Study of Animal Behavior, 34, 59101. 4. Noe, R., Hammerstein, P. (1994). Biological markets: supply and demand determine the effect of partner choice in cooperation, mutualism and mating. Behavioral Ecology and Sociobiology, 35, 111. 5. Noe, R., Hammerstein, P. (1995). Biological markets. Trends in Ecology and Evolution, 10, 336339. 6. Noe, R. (2006). Cooperation experiments: coordination through communication versus acting apart together. Animal Behaviour, 71, 1-18 7. Ponticorvo, M., Walker, R., Miglino, O. (2006). Evolutionary Robotics as a tool to investigate spatial cognition in artificial and natural systems, . In Loula A.C., Gudwin R., Queirz J., Artificial Cognition Systems, Idea Group. (p. ). Hershley, PA 8. Nolfi, S., Floreano D. (2000). Evolutionary Robotics: The Biology, Intelligence, and Technology of Self-Organizing Machines. Cambridge, MA: MIT Press/Bradford Books 9. Cangelosi, A., Parisi, D. (Eds.) (2002). Simulating the Evolution of Language. London: Springer-Verlag 10. Kirby, S. (2002). Natural Language from Artificial Life. Artificial Life, 8(2):185-215. 11. Steels, L. (2003) Evolving grounded communication for robots. Trends in Cognitive Science. 7(7): 308-312. 12. Wagner, K., Reggia, J.A., Uriagereka, J., Wilkinson, G.S. (2003). Progress in the simulation of emergent communication and language. Adaptive Behavior, 11(1):37-69. 13. Nolfi, S. (2005). Emergence of Communication in Embodied Agents: CoAdapting Communicative and Non-Communicative Behaviours. Connection Science, 17 (3-4): 231-248. 14. Di Paolo, E.A. (1997). An investigation into the evolution of communication, Adaptive Behavior 6 (2): 285-324. 15. Di Paolo, E.A. (2000). Behavioral coordination, structural congruence and entrainment in a simulation of acoustically coupled agents. Adaptive Behavior 8:(1): 25-46. June 9, 2008 15:15 WSPC - Proceedings Trim Size: 9in x 6in wiv 9 16. Quinn, M. (2001). Evolving communication without dedicated communication channels. In Kelemen, J. and Sosik, P. (Eds.) Advances in Artificial Life: Sixth European Conference on Artificial Life (ECAL 2001). Berlin: Springer Verlag. 17. Quinn, M., Smith, L., Mayley, G., Husbands, P. (2003) Evolving controllers for a homogeneous system of physical robots: Structured cooperation with minimal sensors. Philosophical Transactions of the Royal Society of London, Series A: Mathematical, Physical and Engineering Sciences 361:2321-2344. 18. Baldassarre G., Nolfi S., Parisi D. (2003). Evolving mobile robots able to display collective behaviour. Artificial Life, 9: 255-267. 19. Trianni V., Dorigo M. (2006). Self-organisation and communication in groups of simulated and physical robots. Biological Cybernetics, volume 95, pages 213-231, 2006. 20. Marocco D., Nolfi S. (2007). Emergence of communication in embodied agents evolved for the ability to solve a collective navigation problem. Connection Science, 19 (1): 53-74. 21. Ashlock, D., Smucker, M. D., Stanley, E. A., Tesfatsion, L. 1996. Preferential partner selection in an evolutionary study of Prisoner’s Dilemma. Biosystems, 37, 99-125. 22. Clements, K. C., Stephens, D. W. 1995. Testing models of nonkin cooperation: mutualism and the Prisoner’s Dilemma. Animal Behaviour, 50, 527-535. 23. Gardner, R. M., Corbin, T. L., Beltramo, J. S., Nickell, G. S. 1984. The Prisoner’s Dilemma game and cooperation in the rat. Psychological Reports, 55, 687-696. 24. Mondada F., Bonani M. (2007). The e-puck education robot. http://www.epuck.org/ View publication stats