Vision Based Behavior Strategy to Play Soccer with Legged Robots

Vision Based Behavior Strategy to Play Soccer with Legged Robots Vincent HUGEL, Patrick BONNIN, Ludovic RAULET, Pierre BLAZEVIC, Dominique DUHAUT Laboratoire de Robotique de Paris 10, 12 avenue de l’Europe, 78140 VELIZY France E-mail : {hugel, bonnin, blazevic, ddu}@robot.uvsq.fr Abstract. This work deals with designing simple behaviors for legged robots to play soccer in a predefined environment : the soccer field. Robots are fully autonomous, they cannot exchange messages between each other. Behaviors are based on information coming from the vision sensor, the CCD camera here which allows robots to detect objects in the scene. In addition to classical vision problems such as lighting conditions and color confusion, legged robots must cope with “bouncing images” due to successive legs hitting ground. This paper presents the work achieved for our second participation. We point out the improvements of algorithms between the two participations, and those we plan to make for the next one. 1 Introduction It is not so easy to make robots play soccer in team. It is more difficult to design locomotion for legged robots than for wheeled machines But in both cases, « Vision » is essential to analyze the scene correctly, and therefore the current game action, in order to adopt the right strategy : go to the ball, go to the opposite goal with the ball, push the ball away of its goal, etc. Real soccer players would appreciate ! For the moment, the strategies seem to be selfish, but the level and the quality of the games increase. Robots are now able to adopt some reactive behaviors and to analyze the current situation so that it can adapt itself when it is changing. Reactivity is absolutely necessary when the robot is facing unpredictable events, for example if it loses the object it is tracking or if it falls down. Reactive behaviors are like reflexes, that make the machine react always in the same way when confronted to the same situation. Adaptability is different from reactivity since it results from a kind of reasoning capability. It is also necessary for robots to be able to analyze a real situation so that they can make up their minds as for what to do next. Here the quality of the analysis determines the suitability of the behavior selected. For instance, adaptability comes in the case where reflexes lead to deadlocks, the machine should switch autonomously to M. Veloso, E. Pagello, and H. Kitano (Eds.): RoboCup-99, LNAI 1856, pp. 74−84, 2000. Ó Springer-Verlag Berlin Heidelberg 2000 Vision Based Behavior Strategy to Play Soccer 75 a rescue behavior. Examples of adaptive behaviors can be slowdowns to avoid obstacles or turning round a ball to bring it back to the desired location, … This work, describes the whole strategy used to design behaviors of legged robots which are operating in a predefined structured world. The example of a soccer field is considered here. Section 2 outlines the improvements between the exhibition held in 98 in Paris and the first competition held in 99 in Stockholm. The improvement of the visual recognition system is detailed in section 3. Section 4 is devoted to the strategy used to play soccer games on a colored field. Finally, the last section presents future improvements in the design of behaviors for soccer robot players. 2 From Paris 98 to Stockholm 99 Previous work, achieved for the First Legged Robot RoboCup Exhibition, held in Paris in July 1998, stresses upon basic tasks of locomotion and vision [HBB98(a)]. The 98 prototype shown in figure 1 was used in soccer games. For locomotion purpose, we developed our own walking patterns, and transitions between the different patterns: forward motion, left and right turns and turn-in-place modes[HBB98(b)]. For vision purpose, we developed an-easy-to use interface to tune vision parameters of the color detection hardware available on the robot, and the embedded vision algorithms [BHS99]. By lack of time, a very simple strategy which consisted in pushing the ball was implemented. This strategy was not effective: our robots were pushing the ball in our goal during the play ! Considering the new design of the 99 prototype (figure 1) of AIBO, the first work was to adapt locomotion algorithms to new leg configurations. The most important problem came from the nails at the leg tips. They are necessary for a « like a dog » design, but they limit the possible positions of the feet during the walk. Embedded vision algorithms do not need adaptation. They take advantages of the new CCD camera, and the new processor : the processing rate increases from 15 to 30 images per second. Fig. 1. 98 and 99 Prototypes of AIBO The improvements of the algorithms from the Paris 98 version to the Stockholm 99 one concern the vision module : embedded algorithms as well as the interface to tune parameters (see § 3). Moreover, two real game strategies have been implemented : behaviors for the goalkeeper and for the attacker / defender (see § 4). 76 3 3.1 V. Hugel et al. Vision Improvement No Coherent Behavior Possible without Reliable Information It is impossible to make the appropriate decisions of action, that is to say to make the robot switch to the correct behavior, without reliable information coming from the vision module. Reliable information means identifying the scene element correctly: ball, beacons (landmarks), goals, players. Positions of scene elements are always correct. All the behaviors we describe here to illustrate the consequences of the color confusions result from observations during the games in Stockholm. We do not think that they were implemented by teams voluntarily ! The confusion between the orange ball and red players made robots follow red players instead of going to the ball : our robots had this behavior during the training play, we corrected this problem for the competition, but we saw other robots with this behavior. The confusion between the orange ball and the shadows on both sides of the yellow goal made robots stay part of a game inside the yellow goal. During part of a game, two attackers of the opposite team were staying in our goal on each side of our goal keeper, meanwhile our attackers went with the ball.... A robot stayed many minutes in front of a beacon, blocked by the inclined wall of the soccer field. From a practical point of view, it is possible to avoid color confusion, by choosing correctly the threshold values of the Color Detection Hardware. But it is not enough, results of the Color Detection hardware must be carefully processed in order to reject false pixel classifications. 3.2 More Autonomous Interface Since our first participation, we developed a practical interface, aimed at tuning the threshold values of the Color Detection Hardware in the best way possible. This interface is designed for the following tasks: 1. It can display a large number of images (between 10 and 20 in practical cases), taken from different points of view of the soccer field, and their segmentation (*) results (cf. figure 2) 2. It allows to select or to remove areas of the images representing different scene objects (like the ball, the goals, the beacons, the players) viewed from several locations, under various lighting conditions. These areas can be chosen « automatically » by pointing regions on the segmentation results (improvement of this last participation), or « by hand » (as in the first version) by determining a box. 3. It computes the threshold values for each color template in the YUV color space from a set of image areas representing the same color. We add, in this new version, the possibility to process the threshold values, to make the results of the color Vision Based Behavior Strategy to Play Soccer 77 detection more independent of the positions of the scene object and of the robot on the soccer field. 4. It checks the coherence between color templates : a color pixel may belong to one color template at most, so in other terms, there is no intersection between color templates. If this condition is not satisfied, it is possible to apply again processing from step 2. An « error message » informs the user about confusion between colors. 5. It processes the YUV color images and displays the results of processing : the identified connected components (representing ball, goal, beacon, player). If results are not satisfying, it is possible to apply again procedures of step 2. Finally, it writes the file containing the threshold values, which will be loaded into the robot. This interface includes other tasks : it is in fact a more general tool for color image analysis. (*) To be able to determine automatically a region in the image on which color parameters will be computed after, we develop a segmentation method, adapted version of the multispectral cooperative segmentation, specially developed for military applications (NATO project) of sensor fusion (Radar, Visual and Thermal Infra Red radiations) [BHP95] et [BHP96]. YUV homogeneous regions are extracted from YUV edges. In fact, the parameters of homogeneity were too strictly tuned: the ball and the goal are divided into several (really homogeneous) regions, and it is possible to select individually each of them, to avoid color confusions. Using this automatism, the tuning of the threshold values is easier to perform by the operator, and more precise because the regions on which thresholds are computed are selected by a segmentation of good quality, which gives better results than the human operator. Fig. 2. Image and its Segmentation Results After lots of experiments, we find a strategy for the choice of the regions on the segmented images, which allows for a correct tuning of threshold values. 78 V. Hugel et al. 3.3 Low Level Filtering The previous described method was tested systematically using all coherent values of camera parameters : the mode, (4 modes : indoor, outdoor, fluorescent and user, according to the temperature of light, are available), the minimal and maximal gains. Then we choose the best combination by looking at object color appearances in several images for each set of parameters values, and by trying to find some regions in the image to tune parameters without detecting incoherence. Some « control » images, results of the Color Detection hardware, with the best combination of camera parameters and with the tuning previously described (see § 3.2) are taken. In fact, the color confusion was noticed in only a few points which are generally isolated. So, a low level filtering, « similar to an Opening Procedure » using an isotropic 3 by 3 centered neighborhood was implemented to remove with success this wrong information (cf. figure 3). Fig. 3. The Results of the Color Detection Hardware, and the « Low Level » Filtered Image But this filtering has a drawback : the maximal distance of detection of scene elements like the ball and the beacons decreases, which is a problem for the absolute localization of robots on the soccer field. The choice we made results from a tradeoff : good identification (necessary including this filtering) / large distance detection of scene element, and seems to be good. It is better to be sure of the extracted information, in order to design a good strategy. Vision Based Behavior Strategy to Play Soccer 4 4.1 79 Behavior strategy to play soccer Influence of locomotion on behavior Locomotion is not omnidirectional. To change direction, the robot must compute the best suited turning circle and switch to a turning pattern as soon as possible. If the turning circle is below the threshold, the machine should stop and switch to the turnin-place mode in the desired direction. Therefore, if the supervision module needs to plan the trajectory of the body within a world reference frame, it must take these characteristics into account. Moreover, it must incorporate the possible delays for a new walking pattern to trigger. A supervision module was tested on simulation and gave satisfactory results. The problem appears when trajectory planning is carried out on the real machine. In spite of the careful design of walking patterns, some leg slippage, drifts and falls occur sometimes. Slippage is due to the nature of the interaction between the ground and the leg tip, which is changing. Since the robot does not master these interactions, drifts cannot be avoided over a long distance (4 or 5 times the length of the body). And it may happen that the robot falls down when it bumps into walls or other objects in the scene, one of the reason is that the locomotion is open-loop controlled and that the machine does not know how to deal with these situations. By adding some closed loops using exteroceptive data from force sensors, gyrometers and accelerometers, the machine should be able to detect collision, loss of balance, etc., and then react in real time to avoid falling down. In the worse case where falls cannot be avoided, the machine should be aware of it and capable of recovering by itself. Basic recovery behaviors have been implemented for real, thanks to its three-axis accelerometer, the robot can detect on which side it has fallen and chose the right standing up procedure. When confronted to real situations, absolute trajectory planning does not provide good results. This is because it relies on localizing procedures, therefore on the vision recognition system. Absolute localization is not possible while moving. The strategy adopted by our team is to use relative positioning, which can be used in real time. As precise positioning is not required, it does not matter if local motion is not accurate, the most important thing is to reach the goal. Therefore, in function of the landmarks captured while moving, the trajectory of the body is being corrected in real time by varying the turning circle. 4.2 Influence of vision on behavior The vision recognition system has to deal with changing lighting conditions and “bouncing images”. The first difficulty can be overcome using color adaptive algorithms. Color thresholds could be changed online after analyzing the results on the field. However, the vision system cannot cope with the second difficulty. It often occurs that an object is detected in the first image acquisition and lost the time after. 80 V. Hugel et al. Since it is not always possible to track an object in the successive images captured, it is essential to memorize the object during a certain amount of time. If the object is not refreshed until the time-out expires, the robot should switch to some different behavior. The first behavior consists in searching for the object lost. If it fails, and if the robot is completely lost, the next step consists in beginning an absolute localization procedure. Since it takes a lot of time, this behavior should only be called in critical situations. In other cases, relative positioning is preferable. For instance, on the soccer field, each time the robot can spot the ball, one goal or the other, or one of the landmarks surrounding the field, it can deduce some information and adopting a more thoughtful behavior. 4.3 Two kinds of behavior This section illustrates the strategies used to make quadruped robots play soccer successfully. Behaviors are separated into two groups. The first group includes basic behaviors that do not need analysis of the situation. The second group contains more high level behaviors that are based on interactions with the environment. 4.3.1 Basic primitives Two kinds of basic primitives are described here. · The first kind regroups behaviors that can be seen as reflexes. Reflexes are always triggered when the robot is confronted to the same situation. The duration of such a behavior is limited. It is a kind of quick reaction to a similar event. Examples of reflexes are all the recovery procedures in case of fall. Another reflex is the one thanks to which the robot turns its head towards the last position of the object lost. It can help save searching time. · The second type of basic behaviors are not limited in time. In fact, these behaviors need time to reach the objective set by the supervision module. For instance, ball Fig. 4. Online trajectory correction using landmarks captured Pole (landmark) Ball Robot Trajectory followed by the robot searching and ball tracking illustrate this kind of behaviors. In ball searching, the goal consists in finding the ball again. In ball tracking, the objective is to get Vision Based Behavior Strategy to Play Soccer 81 closer to the ball. These behaviors are basic ones since no further analysis of the situation is required. 4.3.2 Higher level behaviors involving situation analysis Four kinds of high level behaviors have been designed to play soccer. · The first behavior consists in curving the trajectory in function of the pole (landmark) captured. The objective in the case of soccer game consists in bringing back the ball to the opponent field. Every time a landmark is spotted, it is memorized, and the turning circle is corrected in such a way that the robot goes to the ball from the right or the left side, depending on which side of the field the pole is situated. The robot must have the ball in its field of sight or have it memorized, and have captured a landmark in the same time. The turning circle of the current trajectory depends on which pole is detected. The machine must therefore analyze the situation before triggering the behavior. In soccer game, this behavior is used to defend its own goal or to attack the opponent one. In defense mode, the robot can head for the ball located in the own field and bring it back by moving behind it. In attacking mode, the robot can bring the ball back from the corners towards the opponent goal. Figure 4 illustrates this behavior. · The second behavior is used by players to attack the opponent goal. In this case, the robot tries to align the ball and the center of the adversary goal along its longitudinal axis. To trigger this behavior, the robot must spot the ball and the opponent goal, the ball should be close enough. To this purpose, the robot uses the Opponent goal Pole Pole Ball Robot Fig. 5. Attacker behavior in front of the opponent goal angle of the ball and the goal computed by the vision module. However, memorization should be carefully tuned. Refreshment rate must be sufficient otherwise the robot could miss the ball. Figure 5 shows the role as an attacker. 82 V. Hugel et al. · The third behavior consists in defending the own goal when the situation is very dangerous, that is when the robot sees the ball and its own goal in the same time. When all conditions are met, the robot starts to get around the ball. The movement Own goal Pole Pole Ball Robot Fig. 6. Turn-round-the-ball defense. Successive turn-in-place and forward motion sequences around the ball. is a sequence of straight line and turn-in-place motions, as shown in figure 6. During the procedure, the robot does not turn the head. If it loses the ball, the machine switches to turn-in-place motion until it sees the ball again, then it moves straight forward until losing the ball, and so on. Normally the robot makes a turn of approximately 180°, it can therefore push the ball to the opponent field. · The last behavior is specially designed for the goalkeeper. First, the robot directs its head to the ball. If the ball gets close, the goalie aligns its body to the ball. Once the ball gets closer, the robot moves towards the ball to strike it and push it away. After that, it searches for its goal, goes to it. It detects the walls of its goal thanks to its infrared proximity sensor, and begins then to search for the opponent goal by turning in place. Once it has found it, it tries to spot the ball again. The procedure can restart. In this behavior, the robot has to estimate the distances to the ball and to the goal. Experiments must be carried out to tune them since they depend on the lighting conditions. 4.4 Behavior interaction In the case of soccer, the interaction between the different behaviors is easily managed. In fact, some behaviors are naturally preemptive over others. The direct Vision Based Behavior Strategy to Play Soccer 83 attack of the goal has a higher priority than others. If it were the behavior that curves the trajectory, the robot could miss the ball and the goal. In case of defense it is the turn-round-the-ball behavior which preempts all the others. This is obvious since it refers to the most critical situation. It can be seen as a rescue or last chance behavior to prevent the other team from scoring. Figure 7 summarizes the behavior interactions. Ascending priority Recovering procedures Turn-round-the-ball defense Direct attack of the opponent goal Ball tracking + trajectory bending Evolution of the behavior Ball tracking Ball searching Absolute localization attempts Fig. 7. Scale of behaviors priorities. Example of soccer game. The experience brought by soccer experiments taught us that behaviors should not be limited to classic basic ones such as turn to the right, turn to the left, go to a specific location, stop to localize, achieve a fixed number of steps, and so on. It is better designing some adaptive behaviors that interact with the environment. Decisions should be made while moving, and relative positioning should be used most of the time. 5 Conclusion The experiments carried out on quadruped robots playing soccer allow us to test behaviors in real situation. It helps us become aware of real problems of implementation. Vision and locomotion skills should be taken into account in the design of behaviors. A strategy that combines reactive and adaptive behaviors should give satisfactory results, provided that the interaction between both is well managed. We are satisfied by the improvements we added between Paris 98 and Stockholm 99. Regarding Vision, we will try to increase the maximal distance of scene element detection without decreasing the quality of identification, which is essential. This point is necessary in order to guarantee a reliable absolute localization, on which we are currently working. 84 V. Hugel et al. Another research subject is to embed an automatic color segmentation, like the one we use for the interface, to modify the threshold value of the color detection hardware. This will confer the vision system a better adaptation to inhomogeneous illumination. In the first series of experiments (Paris 98 and Stockholm 99), communication between robots was not possible. However the next version of quadruped prototypes might be provided with wireless communication links. Behaviors could then be improved as robots could know the relative or absolute positions of others. Information could then be crossed to increase the accuracy of positioning. Another way to improve the overall behavior is to use gyrodometry. This technique consists in using gyrometer data locally to correct drifts from odometry. The robot can then estimate its displacement with enough precision over a few more steps. 6 Acknowledgement The French LRP team would like to thank SONY, which provided four robots, a development environment, and very cooperative technical support. 7 References [BHP95] P.Bonnin, B.Hoeltzener, E.Pissaloux, A new way of image data fusion : the multispectral image cooperative segmentation, IEEE Int Conf on Image Processing ICIP95 Washington, October 1995 [BHP96] P.Bonnin, B.Hoeltzener, E.Pissaloux, The multi-spectral cooperative segmentation : How to Perform simultaneously image segmentation and image data fusion, Defence Optics's 96 (NATO Conference) Royal Military Academy Bruxelles Belgique 3-5 juin 1996 [BSH99] P Bonnin, D.Solheid, V.Hugel, P.Blazevic, Software Implementation of of a Real Time System inboard a mobile and autonomous robot for the SONY RoboCup Challenge Vision Interface 99, Mai 99, Trois Rivières Canada, pp 172-179 [HBB98(a)] V.Hugel, P.Blazevic, JC bouramoué, D.Solheid, P.Blazevic, D.Duhaut, Quadruped Robot Guidede by Enhanced Vision System and Supervision Modules in RoboCup-98 Robot Soccer World Cup II, M.Asada, H.Kitano (eds) [HBB98(b)] Vincent HUGEL, Patrick Bonnin, Jean-Christophe Bouramoué, Pierre BLAZEVIC, Integration of vision and walking skills together with high level strategies for quadruped robots to play soccer, Advanced Robotics, The International Journal of the Robotics Society of Japan, Special Issue on RoboCup.