2007 IEEE International Conference on FrE8.

4
Robotics and Automation
Roma, Italy, 10-14 April 2007

Analysis of People Trajectories

with Ubiquitous Sensors in a Science Museum
Takayuki Kanda, Masahiro Shiomi, Laurent Perrin, Tatsuya Nomura, Hiroshi Ishiguro, and Norihiro Hagita

elsewhere [7]. Liao et al. also used locations obtained via

Abstract This paper reports a study that estimated visitor GPS with a relational Markov model to discriminate loca-
positions, visiting patterns, and inter-human relationships at a tion-based activities [8]. Choudhury and Pentland developed
science museum using information from RFID readers. In the a wearable device called a sociometer to visualize peoples
science museum, we exhibited humanoid robots. Visitors were
invited to wear RFID tags to interact with the robots. Visitor
peer relationships based on vocal communication activity [9].
behavior was simultaneously observed using 20 RFID readers, Scant previous research, however, has revealed to what
distributed throughout the entire floor, that roughly measured extent we can identify peoples behavior by studying their
the distances of nearby tags. We integrated the outputs from trajectories. Although Sparacino classified visitors into three
all RFID readers to estimate visitor trajectories that were used pre-defined types based on trajectories obtained through
to analyze three perspectives: space, visiting patterns, and wearable sensors [10], the user types were defined in advance,
relationships. Regarding space, we identified crowded and
uncrowded areas. We found several typical visiting patterns,
such as busy, selective, and greedy, and have been de-
such as visited at every exhibit and directly going to robot area. veloped in museological study with time-consuming obser-
We also identified atypical visiting behavior. Regarding rela- vation by researchers [11].
tionships, for example, we estimated 68% coverage of We are particularly interested in exploring in an exhibition
group-member relationships with 91% reliability. environment such as a museum, where many people remain in
an environment over an extended period of time and walk and
I. INTRODUCTION look around. In such an environment, we believe that peoples

R ECENT developments in ubiquitous computing and

robotics enable us to integrate these fields to produce an
idealized human-computer interface called the network robot
trajectories provide information that can be used to promote
activities in the environment. For example, we expect that
people will move around to see what they are interested in;
system. In this context humanoid robots serve an informative thus, the amount of time a person remains at a certain position
role with support from ubiquitous sensors. There are several may reflect his/her interest in a particular exhibit.
previous research works in this area, such as projects that used This paper addresses to what extent we can estimate peo-
RFID tags to identify interacting children to further promote ples activities in a museum space from their trajectories
interactions with robots [1,2]. However, what a single robot observed by ubiquitous sensors. We conducted a 25-day
can recognize is limited, even if many RFID tags are distrib- field trial in a science museum involving humanoid robots,
uted throughout the environment on the people within it. A whose presence offered considerable temptation to visitors
strong need exists to acquire a greater understanding of hu- to participate in the field trial. During the 25 days, 8,091
man activities through the use of ubiquitous sensors. visitors participated in the field trial by wearing RFID tags and
Several research efforts have also been conducted into interacting with the robots. We chose RFID as a positioning
recognizing human positions and trajectories using ubiqui- device since RFID tags are easy to carry and we can observe a
tous sensors [3-6]. Ubiquitous computing technology is number of tags at the same time. Analysis of participant tra-
gradually being used to analyze peoples activities. For jectories revealed conclusions from three different perspec-
example, Eagle and Pentland developed a Bluetooth-based tives: space, visiting patterns, and relationships.
device attached to a mobile phone that enables us to recog-
nize such routine activities as being at home, in an office, or II. DATA COLLECTION

A. Field trial at a science museum

Manuscript received September 15, 2006. This research was supported
by Special Coordination Funds for Promoting Science and Technology of The study was conducted in conjunction with a field trial
the Ministry of Education, Culture, Sports, Science and Technology with humanoid robots at the Osaka Science Museum in
(MEXT), Japan.
Japan. This subsection briefly explains the field trial (See [2]
All authors are with the ATR Intelligent Robotics and Communication
Laboratories, 2-2-2 Hikaridai, Keihanna Science City, Kyoto, Japan, e-mail: for details). Figure 1 shows a floor plan of the museum space
[email protected]). In addition, M. Shiomi and H. Ishiguro are with Osaka that was used. It is approximately 84 x 42 m. Visitors entered
University, Suita City, Osaka, Japan

1-4244-0602-1/07/$20.00 2007 IEEE. 4846

 FrE8.4

Robovie

Robovie M
exit
talking robots
entrance

Exit
Figure 4: RFID reader, antenna attached to ceiling, and tag

exit desk reception desk

Entrance
Figure 1: Fourth floor of Osaka science museum

42 [m]
RFID reader
Robot with RFID reader
84 [m]
Figure 2: Interaction scene with a humanoid robot, Robovie-II Figure 5: Placement of RFID tags
would assume that the visitor had tried it. The robot might say,
Yamada-san, thank you for trying magnetic power. What
did you think of it? Participants enjoyed such interaction
with robots, particularly when personally greeted by name.
Visitors not wearing RFID tags were also free to interact with
the robots outside of experiencing such greeting behavior.
Figure 3: Interaction scene with other humanoid robots The other Robovie-II and one of the Robovie-M explained
using the elevator shown at the bottom left of the plan, and at about exhibits behaved as if they were talking (Figure 3, left).
the reception desk they were invited to participate in the field The remaining RobovieM did not provide guidance but
trial. Participants were asked to wear RFID tags. If a visitor instead interacted with visitors by using their names from
decided to participate, he/she completed a registration form visitor tags and registered information, saying goodbye,
whose information was stored in a database server. Visitors and asking visitors to return their tags (Figure 3, right). The
were free to look around the museum as they wished. For this other Robovie-II was also equipped with an RFID reader and
purpose, 400 RFID tags were prepared; at most 172 RFID the remaining Robovie-M utilized a nearby RFID reader
tags were simultaneously active (worn by visitors). embedded in the environment. In the figure, the boy is show-
Four humanoid robots, RobovieII and RobovieM, were ing his RFID tag to the robot so that the robot will call his
placed in the environment at positions R and M shown in name. This behavior was frequently observed during the field
Figure 1. The robots had three roles. One RobovieII guided trial. Finally, participants returned RFID tags at the exit desk
people to museum exhibits while it is moving around, where near the entrance and completed a questionnaire about their
its position is revised by infrared sensor attached to the robot impressions of the robots (Details were explained in [12], a
and ceiling. This robot can engage in such behavior as field survey about impressions and anxieties).
handshaking, hugging, and the rock-paper-scissors game, as B. RFID tags and readers
shown in the left of Figure 2. It can also guide people to four Active-type RFID tags and readers were used (Figure 4).
kinds of exhibits, as shown in Figure 2 on the right. When We installed 18 RFID readers in the floor (Figure 5). RFID
bringing visitors to the telescope, the robot says, I am tak- tags were embedded in a nameplate (5 cm in diameter) so
ing you to an exhibit, please follow me! It then approaches that participants were easy to carry. Each RFID tag peri-
the telescope and explains it. odically transmitted its ID, which was received by each of
Robovie-II is equipped with an RFID reader (the detail of the readers. The reader has eight degrees of attenuation (a
which are explained in section 2.2). When it detects an RFID mechanism that increases electrical resistance to weaken
tag worn by a participant, it uses the registered information received radio signals) that reduces the maximum gain of the
stored in the database to promote interaction with visitors, receiver by 12.5% with each step. This enables us to esti-
such as greeting them by name. Robovie-II also used the mate the rough distance between tags and readers. Detection
amount of time that visitors spent near particular exhibits to areas were affected by the antenna positions of the RFID
judge whether visitors had used the exhibits. For example, readers and signal reflections from walls. We measured the
when an RFID-tagged visitor had lingered around the mag- detection area of each RFID reader to adjust the distance
netic power exhibit more than a predefined time, the system parameters for each attenuation parameter. Each RFID

4847
 FrE8.4

and calculate weight t using equations (1) and (2).

(n )
reader changed attenuation parameters from five to eight at
0.75 second intervals. In addition, it costs 0.25 seconds to We evaluated the accuracy of estimated positions with
stabilize detection in each range. Thus, each reader can this method. Several people walked around the floor simu-
observe the distance of tags every 4 seconds. lating typical visiting behavior of museum visitors. We
C. Position estimation with multiple RFID readers compared their real trajectories and estimated trajectories.
The error in the estimated positions was approximately 2.8 m.
We integrated the output from all 20 RFID readers (two
We believe that there were several sources of errors including
attached to the robot and 18 installed to the environment) to
the reflection of and instability of radio signals and the mov-
achieve approximate position estimation of each RFID tag.
ing of RFID tags due to the movement of subjects.
Figure 6 shows the outline of the estimation method, which
is based on the Markov chain Monte Carlo method [13]. It
III. ANALYSIS OF TRAJECTORIES
projects the probability of the tag position to a map image of
the floor. The size of the map image is 1024 x 512 pixels; We gathered 8,091 visitor trajectories during the 25-day
thus, the area of each pixel represents 8 cm2. The system field trial1. Since 49,091 people visited the science museum,
calculates the probability of each tag existence in each pixel approximately 16.5% participated in the field trial. Partici-
by the detected signal strength of each RFID reader. The pant IDs, trajectories (sequence of positions), and ques-
probability of tag existence is distributed equally to areas (x, tionnaire answers were stored in a database in real-time.
y) that satisfy the following two equations: Here, we define the basic symbols used in the paper. Since
( x cx A ) 2 + ( y cy A ) 2 > rA ( N Aa 1) 2 , (1) observations from each RFID reader are asynchronous, we
i
( x cx A ) 2 + ( y cy A ) 2 < rA ( N Aa ) 2 , (2) need to synchronize every visitor position. Pt i denotes the
positions of visitors in the X-Y coordinates, where i denotes
where (cxA, cyA) denote the position of RFID reader A in
visitor ID and ti denotes observed time. The position data for
the X-Y coordinate. rA (n) denotes the pre-measured radius
person i is updated every time one RFID reader detects the tag
for RFID reader A within which it can detect RFID tags
of person i; thus, the duration between ti and ti+1 is not constant.
with attenuation parameter n. N Aa is signal strength for
It mostly ranged within 0 to several seconds. The position of
RFID tag a (minimum attenuation parameter within which
visitor i at time t is linear interpolation, calculated as:
A could detect a). The system adds the probabilities of tag
P i (t ) = Pti + (1 ) Pti ,
existence when areas overlap and normalizes the probability i i +1
where = ( t t i ) /( t i +1 t i ) and t i <t< t i +1 . (If the duration
of each tags existence calculated by each RFID reader.
between t i and t i +1 is larger than 30 seconds, we treated
Signal strength detection Position estimation P i (t ) as a missing data point.)
N Aa =5
reader
A N Ca =7 rB (6) RFID reader
The trajectories of visitors P i are defined as:
reader
C
B
Probability of
P i = ( P i (t i ), P i (t i + t ), P i (t i + 2 t ),L, P i (t f )) ,
tag A C
a reader
D N Da =8
rA (5) rC (7)
tag existence
(dark area has
where ti and tf denote the start and end time of observations.
D
reader
B N Ba =6
high probability) We set t as 4 seconds based on the detection frequency of
rD (8) Estimated position
of tag a
an RFID reader. We refer to P i as a trajectory hereinafter.
58
Strength High low
A. Space
Figure 6: Outline of position estimation method
1) Use of space
To decrease observation noise, it incorporates the previ- We analyzed all 8,091 trajectories to identify how the
ous position into the estimation of the next tag position in the space was used. Figure 7 shows the distribution of positions
following process (This is a standard process of the Markov where participants remained. Red (darker in grayscale im-
chain Monte Carlo method [13]): age) represents positions where visitors remained for
(1) Acquire weighted sample set {( st(n1) , t(n1) ), n = 1,..., N } at amounts of time of more than average plus 2 (standard
time-step t-1 from a previous iteration. st 1 and t 1 denote deviation). If the amount equals the average, the software
(n) (n)

the n-th sampling point (X-Y coordination) and its weight at colors the position pink. If less than average minus 2, it
t-1 (that is, the probability of tag existence at position st 1 ).
(n)
colors it white. Two green triangles and 18 green circles rep-
(2) Calculate cumulative weights ct 1 as
(n)
resent the locations of RFID readers. The following findings
ct(01) = 0, and ct(n1) = ct(n11) + t(n1) , (3) were produced regarding visitor movement in the museum:
and select sample set {st'(n1) } by iterating the following steps: - The upper left area was crowded because this is where the
(a) Generate a random number r [0,1] , uniformly distributed. robots and child-friendly exhibits were placed, such as
(b) By binary subdivision, find the smallest j for which ct(j1) r .
(c) Set s = st 1 .
'( n ) ( j)
1
(n ) The trial was actually performed for two months, but since we changed
(3) Measure observation features at new sample position st , locations of the RFID readers after one month, we only used the data from
the second month (Aug. 7 to Aug. 31, 2003).

4848
 FrE8.4

robots force of repulsion weight comparison num. of cluster s.d.

arch bridge
(robot area)
pulley 10 2.94
chaotic pendulum
magnetic 20 1.73
power
30 1.56
centrifugal planets 40 1.30
force
telescope gravitation 50 1.14
historical
lets touch materials tools 60 0.98
historical
tools
Figure 9: Distribution based on parti- Table 1: Various par-
chemical
elements tioning titioning sizes
a robot
the Earth
m). This also affects visualization. As Figure 8 shows, a
Figure 7: Distribution of positions where participants lingered larger k value complicates the interpretation of results.
Considering these issues, we chose a k value of 30.
Figure 9 shows the distribution of spaces based on spatial
partitioning in relation to the amount of time participants re-
mained. Our software assisted visualization by representing
crowded areas with darker colors. If the number of points
(a) k=20 (b) k=30 ( P i (t ) ) included in the partition equals the average, the software
colors the area middle gray. If more than average plus 2, it
colors it black. If less than average minus 2, it colors it white.
B. Visiting patterns
Based on the estimated trajectories, we analyzed how
(c) k=40 (d) k=50 people visited the museum exhibits. In this section we in-
Figure 8: Spatial partitioning with different k values troduce a method of extracting typical visiting patterns, which,
lets touch materials and magnetic power activities. as a result, helps us retrieve atypical visiting behavior.
- In the upper right, the region around pulley and arch 1) Preparation: State chain models
bridge activities was crowded while the area around We analyzed trajectories based on the state chain model
chaotic pendulum was not as crowded. illustrated in Figure 10. That is, we converted the trajectory
- The right region around planets and historical tools based on X-Y coordination, P i , to a sequence of states, S i ,
activities was not crowded. based on spatial partitioning. It is defined as,
However, two limitations must be considered in the context Si = U {s t = n | p An } ,
of these findings: p P i

- Positions in the areas that were not surrounded with RFID where An is the partition the point in trajectory p belongs.
readers were inaccurate (particularly, in the planets area).
- Our method did not accurately estimate positions if partici-
pants are positioned upon exhibits because some exhibits are
partly made of iron, such as centrifugal force and gravita-
tion, which offered people rides to experience the force.
2) Spatial partitioning Figure 10: State chain model
To analyze the use of space, we conducted spatial parti-
tioning, which was also used in the following trajectory We calculate the distance between two state chains, S i
analysis to reduce calculation cost. A k-means algorithm and S j , by using a DP matching method, which is identical
was applied to all trajectories to divide the museum into k to the comparison of strings known as the Levenshtein dis-
areas. Figure 8 and Table 1 show the results of spatial par- tance. Figure 11 compares trajectory works. Here, we set
titioning with different k values. As Figure 8 shows, areas in the distance between partitions as the distance between the
which visitors remained for significant amounts of time, centers of the partitions. Thus, insert and delete opera-
such as the robot area, are represented by small clusters. This tion costs partition distance plus a constant parameter, which
reflects the center of balance of each cluster; a more represents the tradeoff cost between time and space.
crowded point is likely to be the center of a cluster, and an
uncrowded point is likely to be the periphery of a cluster.
Regarding appropriate k value, there is a tradeoff. If the
number of clusters is large, we can gather more precise
information about trajectories, but these results will be more (a) two trajectories (b) comparison of state chains of trajectories
influenced by errors of position estimation (error is about 2.8 Figure 11: Comparison of trajectories based on DP matching

4849
 FrE8.4

k S.D. calc. costs S.D.

22
2 20.9 40903.7
3 18.2 17144.3 20

4 17.5 10362.9 18

5 14.6 7278.8 16

6 14.7 7208.0 14

7 12.8 9493.3 12
8 14.1 8789.4 10
9 12.4 6449.9 1 2 3 4 5 6 7 8 9 10 k
10 12.2 6691.7 (1) Directly going to robot area (2) Going around and staying at robot area
Table 2: Standard deviation Figure 12: Illustration of stan-
and calculation costs [sec.] dard deviations

This state-chain representation reduces calculation cost.

For example, we compared calculation cost based on raw
trajectory P i and state chain S i for retrieving typical vis-
iting patterns (details explained in 3.B.3) with a k-means
clustering method from 28 trajectories. The state-chain
method costs 0.53 sec while the raw-trajectory-based (3) Went around backwards (4) Visited at every place
method costs 9.56 sec. Thus, using the state chain is eighteen
pattern 5
times faster. We cached the calculation of distance between
partitions in the state-chain-based method (that is, insert, pattern 4
delete, and substitute costs in DP matching), which greatly
improved the calculation speed. Moreover, state chain is pattern 2
used in the visualization of visiting patterns, which is also its pattern 3
advantage. We will demonstrate it in the following section. pattern 1
2) Identification of typical visiting patterns
We classified trajectories with a k-means method to (5) Stayed for long time Distribution of visiting patterns
identify typical visiting patterns. Although 8,091 trajectories Figure 13: Five typical behavior patterns of visitors
were obtained, some were corrupted due to observation (1) Directly going to robot area: 346 people
noise and error. Occasionally, the system failed to estimate People directly went to the robot area (green triangles
the position of people, which resulted in missing points in displayed in the figure around the slightly crowded area)
the trajectory. We extracted 5,102 trajectories longer than 5 from the reception desk clockwise and interacted with the
minutes whose missing duration is less than 10%. robots for a short time. After that, they went to the exit
Predicated on the trajectory distance based on the s- desk. In the figure, no arrows and no colored areas ap-
tate-chain comparison, we clustered these state chains with peared around the bottom right and upper right areas,
the k-means method. Table 2 and Figure 12 show the rela- which represents that they did not go there.
tionship between the number of clusters and standard de- (2) Going around and staying in the robot area: 1024 people
viations. Calculation times are done with a 3 GHz Pentium IV People moved counter clockwise, which is the suggested
computer. Since standard deviation largely dropped until k=5, direction from the entrance; but they rarely stayed at the
we interpreted visiting patterns at k=5. Of course, we can exhibits and only stayed around the robot area. This caused
analyze behavior patterns at any k value; a larger number k the visualization output of arrows in the upper right.
will result in more detailed separation of visiting patterns. (3) Went around backwards: 728 people
Figure 13 shows the visualized output of the global trends People directly went to the robot area from the reception
of each visiting pattern at k=5. In the figure, the frequency of desk, which resembles pattern 1. After that, they looked
remaining in a particular location is represented as color and around the exhibits in a counter clockwise fashion.
transitions between adjacent areas as arrows. If the fre- (4) Visited at every place: 2726 people
quency is more than average plus 2, it is colored black. If less People almost stayed at every place, particularly at the
than average minus 2, it is colored white. The arrow from Earth exhibit; some stayed at the exhibits in the upper
partition i to j indicates that (Nij Nji) is larger than a threshold right area and the robot area.
(here, set as 0.2) where Nij indicates a transition from i to j. If (5) Stayed for long time: 278 people
(Nij Nji) is near the threshold, the software outputs it as blue; Similar to pattern 4, but these people stayed in the floor for a
and if around the threshold plus 2.0, the software indicates this long time (2488 to 7059 seconds) lingering at many exhibits.
using red (brighter color in grayscale). As shown in Figure 13, The relationships among the five patterns are also illustrated
the following five typical patterns are retrieved at k=5: (See in the bottom right of Figure 13. It represents the similarity
Figure 7 for information of exhibit locations.) of participant trajectories by using a spring model, which is

4850
 FrE8.4

based on the distance between each trajectory as a spring registered people during their stay. For example, if there are
force. Due to the heavy calculation of the spring model, we 10 other registered visitors during a registered persons stay,
only used 10% of the trajectories. We can see the relation- the crowdedness for the person is calculated as 10.0. We
ships among the visiting patterns. For example, pattern 2 and compared upper quartile (crowdedness of more than 78.6,
4 are connected together while pattern 1 and 5 are placed in 1275 visitors) and lower quartile (lower than 20.4, 1275
the both ends: which means they are very different. visitors), but the difference was also small.
3) Identification of non-typical visiting behaviors
Based on trajectory distance from the closest cluster
center, we retrieved atypical visiting behavior, as shown in
Figure 14. Figure 14(a) is one notable atypical visiting
behavior, whose distance to the closest cluster center was
34.0. As shown in Table 2, the standard deviation of distance
from the closest cluster center was 14.6 at k=5; therefore,
34.0 is a large value. This person left the field trial by re- Low High
turning his tag to a staff member who immediately per- Figure 15: Interest in robots
formed his exit process at the robot area. As a result, his
trajectory disappeared. In the cases of (b) and (c), the
participants only stayed around the entrance, which is un-
usual as people usually tried to interact with the robots. In the
case of (d), the person stayed around the floor for 6776 sec
while the average length of stay was 1990 sec.

Low: less than 20.4 visitors High: more than 78.6 visitors
Figure 16: Crowdedness of environment

(a) Disappeared (distance=34.0) (b) Only stayed around entrance (34.0)

Adults (1331 visitors) Children (3771 visitors)

Figure 17: Adult-child attribution

(c) Only stayed around (d) Started to visit in counter clockwise

the Earth and planets (29.8) and stayed for long time (23.7)
Figure 14: Examples of non-typical visiting behaviors
(Path materialized by a ranging from red (darker color) to green (lighter).)

4) Analysis of attributions Without a child in groups (201 visitors) With a child in groups (4901 visitors)
We used a visualization technique to analyze how peo- Figure 18: Whether accompanying a child
ples interests and attributions affected their visiting be-
havior. Figure 15 shows the results for attributions with We also analyzed participant attributions; that is, whether
respect to the robots. At the exit desk, some participants a participant is an adult or a child. Figure 17 shows the
filled out questionnaires that asked about their interest in the results of adult-child comparisons, which revealed a slight
robots on 1-to-5 scale [12]. Figure 15 on the left is the visiting difference of staying time around the robot and planet areas.
pattern of people who answered low interest, and Figure 15 on Moreover, we only retrieved people who came in with an
the right is the visiting pattern of people who answered high adult. Figure 18 shows the comparison of adults without a
interest. As the figure shows, the difference was very small. child in the group and people in the group including a
Figure 16 illustrates the museums crowdedness as an child. It shows a large difference in the visiting of the robot
environmental factor affecting visiting patterns. From reg- and planets areas. A comparison of the left figure with the
istration information, we calculated the crowdedness of right shows a smaller staying time around the robot area and
each visitors environment, based on the average number of larger staying time around the planet area when without

4851
 FrE8.4

1.0 parameters (TTH). Since the number of group relationships

T TH =0.6 0.5
0.4 among all visitors was fairly small, we focused on the ap-
0.8
0.3 propriateness (coverage and reliability) of the estimated
relationships instead of the rate of correct classification
reliability
0.6
0.2
0.4 (precision and recall in web search techniques), which are
0.1
0.2
defined as follows:
0.0
Coverage = correct relationships in estimated relationships
0.6 0.7 0.8 0.9 1.0
all correct relationships
Reliability = correct relationships in estimated relationships
coverage

Figure 19: Estimation of group relationship estimated relationships

(Numbers besides the line represents threshold TTH. For example, at TTH Here, correct relationships denotes the group relation-
=0.4, it estimated 89% of group relationships with 81% reliability) ships gathered from registration forms, and estimated rela-
tionships are given by our estimation method. The line in the
figure has several points corresponding to different TTH.
There is obviously a tradeoff between reliability and cov-
erage that is controlled by TTH . As a result, for example, our
method estimated about 68% coverage of the relationships
with 91% reliability (at TTH =0.6) and 89% coverage with
81% reliability (at TTH =0.4). In other words, this result also
indicates that in the museum, 89% of visitors spent 40% of
their time with their group members. Figure 20 shows an
Figure 20: Example of estimated (left) and real (right) group example of estimated relationships and real relationships.
relationship: circles represent a person, and links between
persons represents a relationship
IV. DISCUSSIONS AND CONCLUSION
children (left figure) than with children (right figure). Thus,
it suggests that visitor behavior largely depended on the A. Summary of findings
presence of children who usually enjoyed interaction with
This paper revealed the extent to which we can obtain
the robots very much.
information from peoples trajectories in a science museum
C. Relationships using RFID tags. We found the following:
A previous work demonstrates a robots capability of es- 1. We obtained 5,102 clean trajectories of visitors within 2.8
timating childrens friendship based on measuring a period m distance error using 20 RFID readers placed in the 84 x
of simultaneous stay between two children [14]. Similarly, 42 m environment during the 25-day field trial.
we estimated the friendliness of people based on co-visiting 2. The trajectories enable us to see how the space was used,
behavior in the science museum. Since visitors registered their such as which areas were crowded and uncrowded. For
names with the group members (we limited registration forms example, the Earth and robot areas were crowded while
to one per group), we define members of a group as the visi- the planets area was not so crowded.
tors on the same registration form. We defined the estimated 3. The trajectories enabled us to recognize typical visiting
relationship between persons A and B (Group (A, B)) as: patterns. For example, people in visited at every place
Group (A, B) = if (TAB / min(TA, TB)> TTH), pattern lingered around the Earth exhibit for a while,
TA = if (obs (A) ) t, quickly passed the planets exhibit area, stayed around the
TAB= if (obs (A) and obs (B) and dist (A, B)<DTH ) t , pulley area, and then interacted with the robots.
where obs (A) becomes true only when any RFID reader ob- 4. The trajectories enabled us to identify participants with
atypical behavior. For example, it detected a person who
serves the ID of person A, dist (A, B) is calculated as
only stayed around the Earth exhibit near the entrance.
| P A (t ) - P B (t ) | at time t, and if () becomes 1 when the logical
5. We revealed to what extent peoples trajectories are related
equation inside the parentheses is true (otherwise 0). We pre-
to their subjective impressions and attributions. There is a
pared two thresholds TTH and DTH; threshold TTH is for the ratio
stronger correlation between trajectories and attributions,
of simultaneous interaction time, and threshold DTH is for the particularly whether the group consists of only adults.
distance which the system uses to assume A and B are together. 6. We estimated group membership based on trajectories.
After this process, a graph-based refine of estimation was
conducted to maintain the consistency of group relationships. We believe that these demonstrate the effectiveness of the
In our trial, we set t to four seconds (observation frequency of method in inferring complex visiting behavior in the museum.
an RFID reader) and DTH to five meters. B. Contribution to the method of analyzing trajectories
Figure 19 indicates the estimation results with various This study demonstrated one successful method of ana-

4852
 FrE8.4

lyzing peoples trajectories obtained by ubiquitous sensors formed participants that the recorded and obtained informa-
and an RFID tag system. The method consists of analysis tion would be carefully managed and only used for research
function and visualization support function. Regarding the purposes. We only provided RFID tags to visitors who com-
analysis, it converts trajectories to state chains based on pleted a consent form. When a system that employs this
partitions, calculates similarities between two trajectories technology is used, people must have a choice to participate,
based on the DP matching method, and makes clusters of which balances privacy concerns and benefits from system.
thousands of trajectories with the k-means method. It also
E. Limitations
accurately estimates group-membership between visitors by
observing co-visiting behavior. Regarding the visualization This study was conducted in a science museum where
support, the software displays staying times and transitions humanoid robots were exhibited. Thus, limitations exist
based on partitions, our software enabled us to visualize the about the degree to which we can generalize its findings. We,
following: trajectories, how the space was used, typical and however, expect to find similar trends in other exhibition
atypical visiting patterns, and the relationships between environments, since the presence of specific robots can
trajectories and attributions. probably be generalized as other popular exhibits.
To explore optimal algorithm will be our future work. Our
method is combination of available straight-forward meth- ACKNOWLEDGMENT
ods. The k-means clustering with DP matching worked well We wish to thank the staff at the Osaka Science Museum
but its calculation cost is heavy. Several methods analyze for their highly appreciated cooperation. We also with to
time-sequence data; however, note that an appropriate al- thank people at ATR for their cooperative support: H. Te-
gorithm also depends on the quality of position estimation. rauchi, T. Shibata, K. Hayashi, M. Kakio, T. Tajika, F. Ya-
In our case, position estimation based on RFID resulted in maoka, D. Eaton, T. Tasaki, K. Sugahara, and B. Noorani.
2.8 m error. Although we tried to apply a method based on
Bayesian clustering with a Markov-chain model, it failed REFERENCES
due to the noisiness of state transitions. [1] T. Kanda, T. Hirano, D. Eaton, and H. Ishiguro, Interactive Robots as
Social Partners and Peer Tutors for Children: A Field Trial, Human
C. Implication for network robot system Computer Interaction, Vol. 19, No. 1-2, pp. 61-84, 2004.
[2] M. Shiomi, T. Kanda, H. Ishiguro, and N. Hagita, Interactive Humanoid
The developed method is a powerful tool for developing
Robots for a Science Museum, ACM 1st Annual Conference on Hu-
network robot system (or, ubiquitous robotics), since it man-Robot Interaction (HRI2006), 2006.
enables to recognize capability of visitors activity (visiting [3] J. Letchner, D. Fox, and A. LaMarce, Large-Scale Localization from
pattern) and relationship, particularly in a museum envi- Wireless Signal Strength, Proc. of the National Conference on Artificial
Intelligence (AAAI-05), 2005.
ronment. For example, since this method enables us to clas- [4] D. Schulz, D. Fox, and J. Hightower, People Tracking with Anonymous
sify peoples visiting patterns, we will be able to efficiently and ID-sensors Using Rao-Blackwellised Particle Filters, Int. Joint
prepare the interactive behavior of robots. The robot may Conf. on Artificial Intelligence (IJCAI-03), 2003.
[5] A. Madhavapeddy, and A. Tse, A study of Bluetooth propagation using
talk more about robotics to the people in the directly going accurate indoor location mapping, Int. Conf. Ubiquitous Computing
to the robot visiting pattern, and it may talk more about the (Ubicomp2005), pp. 105-122, 2005.
science museum to people in the stayed for long time [6] H. Yan and M. J. Mataric, General Spatial Features for Analysis of
Multi-robot and Human Activities from Raw Position Data, IEEE/RSJ Int.
visiting pattern. Identifying atypical visiting behavior can Conf. on Robotics and Intelligent Systems (IROS02), pp 2770-2775, 2002
also contribute to optimizing robot behavior, because the [7] N. Eagle, and A. Pentland, Reality Mining: Sensing Complex Social
person whose behavior is atypical may need special services. Systems, Personal and Ubiquitous Computing, online first, Nov., 2005.
[8] L. Liao, D. Fox, and H. Kautz, Location-Based Activity Recognition
The estimation of group relationships can also contribute to
using Relational Markov Networks, Int. Joint Conf. on Artificial Intel-
promoting human-robot interaction. For example, the robot ligence (IJCAI-05), 2005.
may promote conversation among visitors who are within [9] T. Choudhury and A. Pentland, Modeling Face-to-Face Communication
estimated group relationship by talking such as Are you with using the Sociometer, Int. Conf. Ubiquitous Computing, 2003
[10] F. Sparacino, The Museum Wearable, Proc. of Museums and the Web, 2002.
friends today? In other words, we can prepare several types [11]B. Serrell, The question of visitor styles. Visitor Studies: Theory,
of robot behavior according to the detected visiting patterns, Research, and Practice 7(1): 48-53, 1996
but utilizing estimated information about peoples behavior [12]T. Nomura, et al, Questionnaire-Based Research on Opinions of Visi-
tors for Communication Robots at an Exhibition in Japan, Int. Conf. on
and relationships is one future research area. Human-Computer Interaction (Interact 2005), 2005.
[13] M. Isard, and A. Blake, CONDENSATION - Conditional Density Propa-
D. Privacy issues gation for Visual Tracking, Int. Journal of Computer Vision, Vol. 29, 1998.
Privacy is a concern that may arise from this study. As a [14]T. Kanda, and H. Ishiguro, Reading human relationships from their
scientific investigation, we conducted this study to identify interaction with an interactive humanoid robot, Int. Conf. on Industrial
and Engineering Applications of Artificial Intelligence and Expert
what we can estimate from peoples trajectories. It was never Systems (IEA/AIE), pp. 402-412, 2004.
our intention to force every visitor to wear RFID tags and
make their private experience public. In this study, we in-

4853