At the Origins of Methodism: Susanna Wesley 

Generating Diverse Behaviors of Evolutionary Robots with Speciationfor Theory of Mind Si-Hyuk Yi and Sung-Bae Cho Dept. of ComputerScienceYonsei University 50 Yonsei-ro,Seodaemoon-gu, Seoul 120-149,Korea L h e s h y @ s c l a b . y o n s e i .a c . k r , s b c h o G c s . y o n s e i . a c . k r Abstract Theory of Mind (ToM) is the ability to read anotherperson's mind. To apply ToM in robots, robot should read the intention from target. However, it is difficult to read target'sintention directly. Robot usesthe sensorsto measure distance from target becausedistance is the feature to read target's intention. Neural network has been widely used to control the robot for generatinga diversespeciation.It has beenlessexploredin behavior-based robotics.Speciation usually relies on a distancemeasurethat allows different from the robot to target to be compared.In this paper, we proposednovel measureto generatediverse behaviorsof a robot with speciationfor ToM. It includessome distance measuresuch as Euclideandistance,cosine distance,arctangentdistance,and edit distance.It generatesdiverse behaviors of the robot by neural network for ToM. The proposedmethod has been experimentedon a real e-puck robot platform. Keywords: Theory of mind, EvolutionaryRobotics,Robot Controller,Speciation, DistanceMeasure. Introduction A theory of mind (ToM) is the ability to understandother's thoughtsand feelings. Primatesand great apeshave the ability to read behavior.For example,they can figure out behaviorby readinggestures,intention movements,and facial expressionsof emotions [1]. In case of robot environment,they use the sensorssuch as infrared, camera,and sound becausethey cannotdirectly read target'sintention.The distance is the most widely used information for trajectory,direction,and strategy.In a preypredatormodel, if a predatorreadsthe prey's behaviorstrategyfrom trajectory,it can easily hunt on discoveredroute. The robot needsa controller which can make the proper behavior patternsbecausethe method of measuringdistancecan generatediversepatterns.The distance,therefore,is a fundamentalfeaturefor robot. Evolutionary Robotics (ER) is a promising methodology,intendedfor the autonomous developmentof robots, in which their behaviorsare obtainedas a consequenceof the structuralcouplingbetweenrobot and environment[2]. That is, robots get primarily inputted through the sensorsas an environment,and then their behavior is determinedby evolution of neuralnetworks.It is a method to generatea behavior which interactsbetweenrobot and environment[3]. L . T . B u i e t a l . ( E d s . ) :S E A L 2 0 1 2 ,L N C S 7 6 7 3 , p p . 4 9 1 * 5 0 0 , 2 0 1 2 . O Springer-VerlagBerlin Heidelberg2012 492 S.-H. Yi and S.-B. Cho G Speciation is incorporated to the evolution process in order to obtain a diverse set of solutions using single algorithm. Although speciation is a kind of evolutionary computation, it has been less explored in behavior-basedrobotics. Speciation usually relies on a distance measurefrom own robot to target [4]. In this paper, we proposed novel measureto generatediverse behaviors of a robot. Measure combines in Euclidean distance, cosine distance, arctangent distance, and edit distance.The purpose of measureis to generatediverse behaviors of the robot by neural network for ToM. The proposed method has been implemented on a real epuck robot platform. encoding. The robots utilize 1 edit distance for behaviorbase graph and each node labeleds of a robot. To use this value among the neural networks. NEAT in reflecting trajectory trajectory from anotherbehavi out an obstacle. Table 1. Approaches of distance measure Author Distance measure method Year S. Luke t8l Euclidean distance Genetic Programming r996 L. Spector [9] Euclidean distance Genetic Programming 2005 L. Trujillo [10] Edit distance NEAT 2008 T.-S.Lee [ 11] Euclidean distance. Angle distance Proposedalgorithm 20r0 2 RelatedWork 2.1 ToM Applications in Simulation There are related works on ToM based application in simulated environments. Kaliouby et al. proposed a mind-reading machine that recognizes person's discrete mental statesfrom video of the facial expression t5l. It is modeled on a Bayesian network. Kondo et al. used the ToM in carrying a stick task for the cooperation of two computer programs. They use the neural network for modeling [6]. Takano et al. applied ToM to complex agent-basedsimulations for collision avoiding system [7]. These works focus on modeling on simulated environment. In this paper, we implement on the real robot platform for ToM. 2.2 Speciation with Distance Measure Table 1 shows related works using other distance measure for robot. In speciation, one of the best approaches is the Neuro-Evolution of Augmenting Topologies (NEAT) method, a specialized GA that evolves a population of ANNs with variety topologies [4]. Drchal et al. describe a simulation of autonomousrobots controlled by recurrent neural networks t121. HyperNEAT algorithm is evolved through indirect ProposedMethod Fig. 1 shows the overview of 1 model generatesthe number of rates changed each network bv light using each neural network to the destination. It analyzesI some distance each time. The n among the estimatedfitness valu 3.1 Neural Network Control Controller designed by neural r light such as food, prey, fodder, and 2-outputs. Inputs are IR sen tion of light from a robot. Outpu the robot's direction and rate of estimation designed that how er distance to measurefrom a destin (*,,*n, - x(last)1) DiverseBehaviors Robots Generating of Evolutionary 493 encoding.The robots utilize 180 degreeswide sensoranay. Trujillo et al. employed edit distancefor behaviorbasedspeciation[10]. The environmentis partitionedinto a graph and each node labeledstring.The fitnessvalue is calculatedby the stringtrack of a robot. To use this value, controller adopts some featuresusing fitness sharing among the neural networks. It representsrelatively more diverse speciationthan NEAT in reflecting trajectory.It is, however, difficult to measuresimilarity in the trajectoryfrom anotherbehavioralspacewhich cannotgeneratediversepatternswithout an obstacle. Table 2. Specificityof eachdistancemeasure Measurement Specificity Euclideandistance Various movement pattern Cosinedistance Wide and lensth 0f route Arctansentdistance Wide and lengthof route Edit distance o,,.*",.;;;," '" ,rr* ,,r,. ProposedMethod Fig. 1 shows the overview of proposedmethod using the evolution algorithm.The model generatesthe number of p with initializationin neural networks.And it generates changedeach network by mutation operation.The robot finds a goal such as light using eachneuralnetwork.We apply to fitnesssharinghow exactlynetworkgets to the destination.It analyzeswhole robot trajectory with location through sum of some distanceeach time. The model conductsevolutionbasedon the hishestvalue amongthe estimatedfitnessvalues. 3.1 Neural Network Controller Controller designedby neural networks.Goal of neural network is to conquerthe light such as food, prey, fodder, and home. Neural networksare madeup of 2-inputs and 2-outputs.Inputs are IR sensorvalue that can know a relative distanceand direction of light from a robot. Outputsare speedof wheels.The controllercan determine the robot's direction and rate of movementusing speedof each wheel. The fitness estimationdesignedthat how exactly networksto find light. We use the Euclidian distanceto measurefrom a destinationof the robot last seneration. (t) Gr S.-H. Yi and S.-B. Cho Initislieatiom Sffspring gsneff,tifir: {usingmutation) The eq. (4) which appliesfou are decided according to the r must consider as the most imp ment pattern. Cosinedistance and length of diversified paffe goal when generationproceed cosine distance must be a low r tory. So its weight has anything is like eq. (5). Gcncrating rsbot pancrn$ Fitnesscvnluatiun Euclidear: dixtance Hdit *listanctl d(i, j Iflrypqr:,:!, .." 'A :': :l l 3.3 r..{|t* si ne rldstance) ,;.4 i,. Physical Distance J' Arctangent distance As the feature of the trajectory the physical distance differenc network. Because the pattern c tance is long. Cosinedistance Fig. 1. Overviewof proposedmethod d(t,i)"urttd.ian = ZT3 3.2 Behavior Based Fitness Sharing A speciation is a method to maintain the diverse behavior. In this paper, we employ fitness sharing which ffansforms fitness value for maintaining a diverse behavior. In this respect, the distance between objects is more proper than other measurements.It is a method to prevent local optimized solution in dense individual who located near distance. fl is fitness value of network i, and fitness sharing value firepresents fitness divided by the sum of sharing function value in eq. (2). .. fi tf , i. -- Xlrsh(a(i,l)) Angle Distance When sharing fitness, we can u behavior has an influence width method. Firstly, cosine distanc each step. d(i,i)rorin" = ZTi - (d(i''i)/o't'"'")o [t + C3d(i,j)arctan+ C4d(i,j).ori.," + C2d(i,j)editdistance d(i, j) = Crd(i, j)eucliaian 3.4 e) (3) " :r*:);rdshare where p is the size of population, oshare is sharing radius. Fitness sharing is conducted when the objects locate within the sharing radius. d(i, j) representsthe distancefrom network i to j. To generate a variety pattern of the robot, we apply the confroller to distance measure such as Euclidean distance, cosine distance, arctangent distance, and edit distance.Based on these measures,we designed confroller evolution speciation using the fitness sharing. As a result, we can find out the feature as Table 2. To combine thesemeasureslinearly, conffoller can generatea pattern that applied the characteristic of each measure. sn(a1;,;)) = The Euclidean distancebetwee mation of each distanceaboutr diverse trajectory, but not make (4) AI BI ft Fl s= Fig.2.Mapis divided by 4x4g GeneratingDiverse Behaviors of Evolutionary Robots The eq. (4) which applies four distance measureis as follows. Ct, Cz, C3, and Ca are decided according to the weight among each distance measure.Euclid distance must consider as the most important factor due to generatingthe diversity of movement pattern. Cosine distanceand arctangentdistanceis used for changingthe width and length of diversified pattern.In caseof the cosine distance,robot cannotfind the goal when generationproceedsover the specific generation.For the reason,weight of cosine distancemust be a low value. In caseof edit distancecan changewhole trajectory. So its weight has anything value. Therefore,the weight of eachdistancemeasure is like eq. (5). d(i, j) : Cr l 3.3 Cz Physical Distance As the feature of the trajectory used by fitness sharing,there is a pattern becauseof the physical distance difference. It can keep the diversity of the trajectory made by network. Becausethe pattern changesfor finding the goal despite the physical distanceis long. | /' '\ &lL,J )eucuaian : ymAX-StePs L'rt=o (x(n) t - x(n) i)' + (y (n) t - y (n) i)2 (6) The Euclidean distancebetween neural networks i and j can be calculatedby summation of each distanceabout robot's movement coordinateson step.It can make the diversetrajectory, but not make dramaticallythe width. 3.4 Angle Distance When sharing fitness, we can use behavior of angle by features.Angle difference of behavior has an influence width of the trajectory. It can be measuredby two kinds of method. Firstly, cosine distance calculates cosine value at location of behavior on eachstep. (! - cosq - rT:;-steas (! d(t,j),o,t - DT:{-stePS #H " 0) t s=D1 Cl C2 C3 83 A3 A4 Fig.2. Map is dividedby 4x4 grid,andgivesa letter.Coloredpartsaremappingroute. f.l. r g 3 B p S.-H. Yi and S.-B. Cho G Cosine value has from 0 to 1. The cosine value has the closer 1, angle has the closer 0 degree.Then the angle location of two points is closer than a previous point. Cosine distancevalue of neural network i and j can be calculated by eq. (7). After calculating the angle difference of robot location of each step and summing each value, conffoller gets the angle of distance in the ftajectory. Secondly, arctangent distance calculates distancefrom the mobile robot to a goal on each behavioral step. Arctangent value is where time position is located in light. Thus distance from robot to goal can calculate like eq. (8). subY(n), -subX(n), = subY(n)t = subX(n)t - (lgoar - Y@)) (xsoat - x(n)i) (lsoat - Y@)i) (xsoat - x(n) i) (8) where n is stepped when the robot move in, that is, means time. Location of light defines(xsoar,lgoat).In n step, location robot reachesby network i and 7 define each (x(n)t,y@))and (x(n);,yfu)).Arctangent distancecan calculatedifference such as eq. (9). d(i, j)"urudian : larctan(subY(n)i/ subx(n)) uT:;-""o' -arctan(subY(n) 1/subx(n) )l '7n\ \Y) Arctangent value of neural network between i and j is an absolute value of difference from a goal in n step. The robot is moved by j network and arctangent value from a goal in n point by i network. It meansthat the smaller this value size is, the closer two points are located in angle by goal. That is, features of angle values are similar characteristicfrom goal. Although features of angle feature to change width and evolves some generation,robo If it moves any close place sensitively decided. 3.5 Edit Distance To avoid the same movemen distance of networks, it canbe t The edit distanceis an algo from original string to wante strings is the operatornumbertr Map of behavioral spaceis d area called cell. When the robc at letter string. An exampleof grid, and gives a letter by each string value, 's', is determine work evolution by fitnesssharin Behavior route found no si samepattern of route appearsat Experiment 4.1 Experiments Settings Experiment proceedsin real ro form. Size of the map defines I lated by 1cm and mark (x, y) f point of the robor set up (0, 0). works, sets 20. And the parame tively. We tested featuresof eac physical distance, angle distanc of each distance measure thro Table 3. 4.2 Tracking Environment To track robot patterns,we use a track consists of multiple infrar markers like Fig. 4. The markerI markers that can be identified b1 patched on the robot body are p placed on the robot body suit in a Fig. 3. Tracking system environment GeneratingDiverseBehaviorsof EvolutionaryRobots 497 Although features of angle don't generatevariously a form of robot pattern, give feature to change width and distance of a same trajectory. After cosine distance evolvessome generation,robot cannotfind a goal becauseof using only anglevalue. If it moves any close place at the starting point, distancebased on the angle is sensitivelydecided. 3.5 Edit Distance To avoid the same movementroute, by using the edit distanceand measuringthe distanceof networks,it can be the distancemeasureof fitnesssharing. The edit distanceis an algorithm to edit given two stringsby the minimum value from original string to wanted string. In other words, edit distanceof given two strings is the operatornumber to transfera letter to anotherletter. Map of behavioralspaceis divided by plaid form, and give a specificletterby each area called cell. When the robot passesthe cell, controlleraddsthe letter of this area at letter string. An exampleof the processis shown in Fig 2. Map is divided by 4x4 grid, and gives a letter by each part. Other color parts are a route of the robot. Letter string value, 's', is determinedby this method. The controller conductsneural network evolutionby fitnesssharingof edit distanceusing this value. Behavior route found no significant differences.However, we confirm that the samepattern of route appearsat different location. Experiment 4.1 Experiments Settings Experimentproceedsin real robot environmentwhich is used an e-puckrobot platform. Size of the map defines 120cm in width and length. The Coordinateis calculated by lcm and mark (x, y) form. The location of light is (100, 100) and starting point of the robot set up (0, 0). The value of p, which meansa numberof neuralnetworks, sets20. And the parameters Cl, C2,C3, and C4 are 10, 2, l, and0.5, respectively. We testedfeaturesof eachdistancemeasuresuchas the maximum weighteach physicaldistance,angle distanceand edit distance.And controllerreflectsall features of each distance measure through various trajectoriesto generate. Details see Table 3. 4.2 Tracking Environment To track robot patterns,we use a motion capturesystemcalled "Optitrack".The Optitrack consistsof multiple infrared cameraslike Fig. 3. And the systemdetectsthe markerslike Fig. 4. The markerbasedsystemgenerallyusessphericalretro-reflective markersthat can be identified by the cameras.In this experiment,the markerswhich patchedon the robot body are provided by the NaturalPointCompany.Markersare placedon the robot body suit in a configurationthat is definedby the software. F i1 h h *t gi s b f t5 *1 frl s. g a' 498 S.-H. Yi and S.-8. Cho '"t ,t/'n Fig. 4. Markers for detection on e-puck robot platform (left), marker based tracking software (right) 4.3 ,:, i :: i it.,::: i i r: :i: .: :' ,, : i 1 iii i:i: r :. lr r i i l : ' : ',: : ::; !i:iii:: I i i 1 Result Controller was evolved for 20 generationson the real e-puck platform. It represents three types of patterns-right, left, and straight like Fig. 5. An analysis of the generated pattern is shown in the Table 3. We can see the different from in simulation results which 1000 generations generated. In point of success,robot got more the goal in simulation. The real robot environment can vary the value of various external features called reality gap. The light has high uncertainty, which is not always same value as input in similar place. Confroller can generateother values from uncertain input value despite samelocation. Table 3. Analysis of generatedpatterns Spin None Forward Direction Loop CW Turning CCW Tight loose 58.33 4r.67 40 60 Success: 60Vo(in shnulation,92.3) 5 8 .3 3 16.67 25 67.67 33.33 Failure : 40Vo(in simulatian,0.T) 37.5 25 37.5 60 40 (a) Fig.5. Resultof rypica 5 Conclusion In this paper, we proposeda m robot by using speciatedneura terns are important features to robot behaves according to thr robust behavior strategy. Fea arctangent distance,and edit d tures. We applied Euclidian dir arctangent distanceand edit dir propriate cosine distancefor tr diverse trajectories to reflect al platform. We also can gener turning. In future work, we plan to t environment such as moving ta Acknowledgment. This resean Program for Brain Sciencethro funded by the Ministry of Educ GeneratingDiverseBehaviorsof EvolutionaryRobots 499 i',.,, {, i*. rft g;F (a) (b) (c) Fig. 5. Resultof typical patterns(a) Go straight,(b) keepright, (c) keep lel't Conclusion In this paper,we proposeda methodto generatediversebehaviorsof the evolutionary robot by using speciatedneural network basedon behavior.Generatedbehaviorpatterns are important featuresto read target's mind or intention basedon ToM. If the robot behavesaccordingto the proper patternof target's intention,can generatethe robust behavior strategy. Featuresof a robot are physical distance,cosinedistance, arctangentdistance,and edit distance.We set up standarddistancebasedon this features.We applied Euclidiandistanceto measurefor diversity of the trajectory.Next, arctangentdistanceand edit distanceare usedbasedon moving area.Finally, we appropriatecosinedistancefor trajectoryrange size.In the experiment,we confirm that diversetrajectoriesto reflect all characteristics of eachdistancemeasurein real robot platform. We also can generatethe patterns'combination of spin, direction, and turning. ln future work, we plan to extendthe model to generatevariouspatternsin harsh environmentsuch as moving targetor obstacle. Acknowledgment. This researchwas supportedby the Original TechnologyResearch Programfor Brain Sciencethroughthe NationalResearchFoundationof Korea(NRF) fundedby the Ministry of Education,Scienceand Technology(2010-0018948). 500 S.-H. Yi and S.-B. Cho References Improving Gender 1. Brune, M.: "Theory of Mind" in Schizophrenia: A review of the Literature. Schizophrenia Bulletin 3l(1), 2142 (2005) 2. Zagal, J.C., del Solar, J.R.: Combining Simulation and Reality in Evolutionary Robotics. J. of Intelligent Robotics and Systems (2007) 3. Nolfi, S., Floreano, D.: Evolutionary robotics: The biology, intelligence, and technology of self-organizing machines. Bradfor B ook (2004) 4. Trujillo, L., Olague, G., Lutton, E., Vega, F.F., Dozal, L., Clemente,E.: Speciationin behavioral space for evolutionary robotics. J. of Intelligent & Robotic Systems 64(3),323- Abbas R Dept. < roayaei@' 3s1(2011) 5. Kaliouby, R.E., Robinson, P.: Mind reading machines: Automated inference of cognitive mental states from video. In: Int. Conf. on Systems, Man and Cybernetics, pp. 682-688 (2004) 6. Kondo, K., Nishikawa, I.: The role that the internal model of the others plays in cooperative behavior. In: Proc. of Int. Workshop on Robot and Human Interactive Communication, pp. 265-270 (2003) 7. Takano, M., Arita, T.: Asymmetry between even and odd levels of recursion in a theory of mind. In: Proc. of ALIFE X, pp. 405411 (2006) 8. Luke, S., Spector, L.: Evolving teamwork and coordination with genetic programming. In: Proc. of Conf. on Genetic Programming,pp. 150-156 (1996) 9. Spector, L., Klein, J., Feinstein, C.P.M.: Emergence of collective behavior in evolving populations of flying agents. Genetic Programming and Evolvable Machines 6(1), 111- Abstract. This paperatt algorithms. We use loca extract a set of feafures; algorithm searchesthe s maximizes genderclassi sifier with hand tunedpu rithm reducesthe feature curacy with 10070reliab using aJl I42 features. Keywords: FeatureSubs rzs(200s) 10. Trujillo, L., Olague, G., Lutton, E., Vega, F.F.: Behavior-basedspeciationfor evolutionary robotics. In: Proc. of Conf. on Genetic and Evolutionary Computation, pp. 297-298 (2008) 11. Lee, T.-S., Eoh, G.-H., Kim, J., Lee, B.-H.: Mobile robot navigation with reactive free spaceestimation.In: Int. Conf.Intelligent Robots and Systems,pp.1799-1804 (2010) 12. Drchal, J., Koutnik, J., Snorek, M.: HyperNEAT controlled robots learn how to drive on roads in simulated environment. In: Proc. of Congress on Evolutionary Computation, pp. 1097-1092 (2009) I Introduction The face is an important biom analyzing faces is a challengi fully performing this task all psychology,and security[1]. I on ethnicity, identity, age,ge investigatesa new approachtc Gender plays a significant [3]. Gender classificationis tt given image containsa pictun images has received much aff search engine retrieval accura interfaces (adjusting the softw over, gender classification ca since it may halve the number images of both genders,befo double the speedof facerecog Like other image classifica then apply our classifier. Fron kinds of methods. First, we ci tures [6]. Second,subspace0 as Principal Component Anal L.T. Bui et al. (Eds.):SEAL 2012,LNr @ Springer-Verlag Berlin Heidelbergi