Health-Status Monitoring Through Analysis of Behavioral Patterns
Health-Status Monitoring Through Analysis of Behavioral Patterns
Health-Status Monitoring Through Analysis of Behavioral Patterns
1, JANUARY 2005
Abstract—With the rapid growth of the elderly population, there ability of basic sensors to detect a person’s behavioral patterns
is a need to support the ability of elders to maintain an independent needs to be examined.
and healthy lifestyle in their homes rather than through more ex- The primary objective of this paper is to examine whether a
pensive and isolated care facilities. One approach to accomplish
these objectives employs the concepts of ambient intelligence to system of basic motion sensors could detect behavioral patterns
remotely monitor an elder’s activities and condition. The Smart- and, thus, to provide the foundation for an ambient intelligence
House project uses a system of basic sensors to monitor a person’s approach to elder care. A mixture-model framework was used
in-home activity; a prototype of the system is being tested within to develop a probabilistic model of behavior and was tested on
a subject’s home. We examined whether the system could be used data from the University of Virginia’s SmartHouse system. The
to detect behavioral patterns and report the results in this paper.
Mixture models were used to develop a probabilistic model of be- results were then compared to a user log to provide validation of
havioral patterns. The results of the mixture-model analysis were the patterns. The effect of analyzing behavior during work and
then evaluated by using a log of events kept by the occupant. off-days separately was also examined.
Index Terms—Ambient intelligence, human behavior models, The remainder of this paper is organized as follows.
mixture models. Section II describes the background of the SmartHouse project
and related literature. Section III describes the mixture-model
analysis methodology. Section IV presents the results of the
I. INTRODUCTION mixture-model analysis. Section V compares the mixture-model
II. BACKGROUND
the 85 population is expected to increase from 4.2 million in
2000 to 8.9 million in 2030 [1]. With this proliferation of the The Medical Automation Research Center (MARC) at the
elderly population also comes an increased need for services for University of Virginia established a SmartHouse project to eval-
the elderly, including assisted-living facilities. However, many uate the ability to use a system of various sensors to provide
elderly desire to stay in their own private residences for as long monitoring health checks. A prototype of the system was in-
as possible, and thus methods are needed to allow them to do so stalled in the residence of a volunteer subject to examine the
safely and at reasonable costs. use of such technology. This SmartHouse system consists of
One possible method to help enable elders to live indepen- a series of motion detection and on–off switches. Eight mo-
dently is to employ the concepts of ambient intelligence by tion-detection sensors have been installed, one in each room of
installing remote monitoring technologies in elders’ homes. the house (bedroom, bathroom, office, living room, kitchen, and
The technologies could alert relatives, caregivers, or health-care laundry room/back door area), one at the front door, and one
personnel of any change in an elder’s normal activity pattern. in the shower. These sensors fire whenever there is movement
The monitoring technologies should maximize the privacy of within the sensor’s area of coverage. The switches are installed
elders while still providing information of any problems or primarily in the kitchen and indicate actions such as the opening
deviations from normal habits. As a result, simple and inex- of a kitchen cabinet or the microwave. Sensor readings are col-
pensive motion-detection sensors are likely preferred to more lected continuously and consist of the sensor and the time at
invasive technologies such as video recording. However, the which the sensor readings were recorded.
There have been several other research projects that have in-
Manuscript received February 5, 2004; revised June 11, 2004 and July 6, vestigated the use of various sensor technologies on monitoring
2004. This work was supported in part by the Medical Automation Research of daily activity [4], [5], [13]–[15], [18]. Data-analysis tech-
Center, University of Virginia. This paper was recommended by Guest Editor niques have included using plots and histograms to observe pe-
G. L. Foresti.
T. S. Barger was with the Systems and Information Engineering Depart- riodic patterns and infer activities [15], [18], comparing sensor
ment, University of Virginia, Charlottesville, VA 22904 USA. She is now readings to a daily activity log [13], [14], and using neural net-
with Northrop Grumman Corporation, Chantilly, VA 20151 USA (e-mail: works to predict how long a person will spend in or out of
[email protected]).
D. E. Brown is with the Systems and Information Engineering Department, a single-room environment based on a succession of previous
University of Virginia, Charlottesville, VA 22904 USA (e-mail: brown@vir- times spent in and out of a room [5]. Other research efforts have
ginia.edu). examined cyclical behavioral patterns, using probability esti-
M. Alwan is with the Medical Automation Research Center, University of
Virginia, Charlottesville, VA 22904 USA (e-mail: [email protected]). mates to detect deviations from the normal amount of time a
Digital Object Identifier 10.1109/TSMCA.2004.838474 person spends in a room in an hour [16].
1083-4427/$20.00 © 2005 IEEE
BARGER et al.: HEALTH-STATUS MONITORING THROUGH ANALYSIS OF BEHAVIORAL PATTERNS 23
The subscript specifies a particular cluster, while is a where is the maximized mixture likelihood of the
random vector for the observations. The mixing proportion is model, is the number of parameters estimated in the
with for all k and . The term repre- model, and is the number of observations [6].
sents the Gaussian density function of a particular cluster [3].
Each cluster can be represented as a Gaussian model of type C. Application of Mixture Models
The EM algorithm was applied to the training set data to de-
velop the mixture-model parameters and assign observations in
(2) the training set to clusters. The algorithm was run separately
on each sensor type. The number of clusters was selected by
The cluster has dimensions and is centered at mean with choosing the model with the highest BIC value. The -step of
the variance-covariance matrix, , determining the scaling the algorithm was then used to assign the observations in the test
properties [2]. For the data used in this analysis, is a vector set to the clusters. This process was repeated for the subset of
of the means of the time of day , length of time in the room data corresponding to each individual cluster. For a cluster to be
, and activity level attributes further subdivided, the BIC value for two or more clusters must
exceed that for one cluster. This iterative step helps prevent dif-
ficulties caused by clusters of different sizes.
(3)
After using mixture models to divide the data into clusters,
the clusters representing potentially significant event types must
The covariance matrix can be decomposed into the form: be determined. Some groups may consist of simply random
with the matrix determining the orien- events or several different types of underlying events that may
tation of the cluster, the matrix describes the shape, the not be part of a consistent pattern of behavior. For example,
scalar determines the volume, and the superscript denotes if 50-s events in the kitchen occur at about the same density
the transpose of the matrix [2]. The orientation, shape, throughout the day, they may all be placed in one group. Such
and volume were allowed to vary between the clusters for the events could represent many types of underlying activities from
analysis. walking through the kitchen on the way to the back door, to get-
The maximum likelihood criterion was used to determine the ting a quick snack, to checking on dinner. Random events are
mixing proportions and density parameters. The objective was assumed to most likely be those of short time lengths that do
then to maximize the log-likelihood of the parameters given the not occur consistently at the same time of day.
data. This is represented as Groups were considered to be significant if they met either
a length or time-range criterion and occurred on at least 25%
of the days. To determine whether the length of time was sig-
(4) nificant, the mixture-model-clustering algorithm was applied to
the mean length of time values for all groups. The groups were
where denotes the set of parameters [3]. split into two clusters with groups in the cluster with the longer
The expectation maximization (EM) algorithm was applied lengths of time considered to be significant. A similar approach
to calculate the parameter estimates and the assignment of ob- was used to determine whether the range of start times was sig-
servations to groups that maximized the log-likelihood [6]. This nificant. The range of start times was calculated for all groups,
algorithm consists of an iterative two-step process. When ap- and mixture modeling was used to divide the groups into two
plied to mixture modeling, the -step involves computing the clusters. The groups in the cluster with the smaller time ranges
probability that observation belongs to group given the cur- were considered to be significant.
rent parameter estimates and assigning each observation to the
group for which the probability is the highest. D. Analysis of Work and Off-Days
if belongs to group There may be significant differences in behavioral patterns
(5) between days when the subject works and days when the subject
otherwise
is off and, thus, before trying to discover patterns, it was impor-
In the -step, the log-likelihood criterion is used to compute tant to first categorize the days into work and off-days. However,
the parameter estimates given the current set of assignments of the subject works a variable schedule, and whether a day is actu-
observations to groups ally a work or off day is not known; thus, unsupervised learning
methods must be applied. The main indication of a work day is
(6) a lack of activity in the house during the daytime hours. As a
result, the number of motion sensor firings during the daytime
The number of groups can be determined by comparing the hours could be used to classify the days. However, even on work
Bayesian information criterion (BIC) value for different num- days, there could be some sensor firings during the day due to
bers of groups with occurrences, such as the opening of the front door or a sensor
firing because of an outside event such as a door slamming in the
basement apartment. Also, on off-days, errands might be run,
(7) leaving several hours without activity. Hence, classifying the
BARGER et al.: HEALTH-STATUS MONITORING THROUGH ANALYSIS OF BEHAVIORAL PATTERNS 25
days could not be simply based on examining the data for an 8-h TABLE III
period of no activity. Instead, a clustering approach was applied SIGNIFICANT CLUSTERS THAT MEET EVALUATION
CRITERIA FOR COMBINED DATA
to the hourly counts of the number of motion-sensor firings. This
allowed for grouping the days with low activity without setting
an arbitrary cut-off for the number of sensor firings in a partic-
ular hour that can occur.
The number of motion-sensor firings was calculated for
each discrete hour, and agglomerative hierarchical clustering
using Ward’s method was applied to the hours from 7 am.
to 7 pm. This method begins by placing each observation in
its own cluster. Clusters are then iteratively merged until all Another metric involves comparing the parameter values
clusters are combined. Ward’s method attempts to minimize of the mean length of time and mean time of day properties
the within-cluster variance by merging the clusters with the as calculated by the mixture-model algorithm ( and
minimum sum of squares [17]. ) to the estimated values of the parameters from the
This clustering method divided the days into two primary test set clusters ( and ). The differences can be
clusters with 37 days being classified as off-days and 28 identi- normalized by dividing by the parameter standard deviations to
fied as work days. The mixture-model analysis was then applied provide an estimate of consistency between the properties of
separately to the work and off-days and the combined data set. the test set clusters and the original model
Results of the mixture-model analysis on these sets were com-
pared and are described in the next section. (12)
(13)
IV. MIXTURE-MODEL RESULTS
A. Metrics
B. Results From the Combined Data Set
Several performance metrics were used to evaluate the results
of the mixture-model analysis. One performance metric for each The data were grouped into 139 clusters. The mean uncer-
individual cluster is the uncertainty values for the assignment of tainty level for each observation in the training set was 0.1016
the observations to the cluster. For each cluster, the probability and the standard deviation was 0.1468. For the test set, the mean
that the observation belongs to the cluster was calculated. The uncertainty level for each observation was 0.1087 and the stan-
uncertainty is then one minus this probability value. Let rep- dard deviation was 0.1509. For the training set, 72.76% of the
resent the uncertainty for cluster , represent the number of uncertainty values were less than 0.20 while 73.08% of the test
observations assigned to cluster , and the probability that set observations had uncertainty values less than 0.20. The per-
observation belongs to cluster cent of values exceeding 0.50 uncertainty was 1.90% for the
training set and 2.14% for the test set.
Of the 139 clusters, 44 were determined to be significant. Fif-
(8) teen of these 44 clusters exceeded a mean uncertainty level of
0.20, indicating a relatively high level of uncertainty for some of
A second metric compares the estimated probabilities of a the clusters for the training set data. These clusters were eval-
cluster in the training and test sets. For example, based on an uated using the test set. Comparing the percentage of days on
event space consisting of the different clusters, the which a cluster occurred in the training and test sets showed
estimated probability of event A is the number of occurrences of that many of the clusters showed a consistent rate of occurrence;
event divided by the total number of event occurrences however, 19 had percent errors exceeding 0.20. For the measure
representing the percentage of days on which at least one obser-
vation in the cluster occurred, there were also 19 clusters with
(9) percent errors exceeding 0.20. When comparing the time of day
attribute between the test set and model parameters, one cluster
The probability of event for the training and test sets can then differed from the mean parameter by more than one standard
be compared using a percent error metric deviation while six additional clusters differed from the mean
parameter by more than one half the standard deviation. For the
(10) length of time parameter, one cluster differed from the mean pa-
rameter by more than three times the standard deviation. All of
Similarly, the probability that event occurs at least once a day the other clusters had an estimated mean within one standard de-
can be compared between the training and test sets. In this case, viation of the mean parameter value with four clusters exceeding
the probability of event occurring on a day was estimated by one half of the standard deviation. Similar to the training set, the
dividing the number of days on which event occurred mean uncertainty levels for clusters in the test set included both
by the total number of days in the data set low and high levels of uncertainty with the mean uncertainty ex-
ceeding 0.2 for 15 clusters. Table III summarizes the number of
(11) clusters that met each of these criteria.
26 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART A: SYSTEMS AND HUMANS, VOL. 35, NO. 1, JANUARY 2005
TABLE IV TABLE V
COMPARISON OF OFF, WORK, AND COMBINED MODEL RESULTS SIGNIFICANT CLUSTERS THAT MEET EVALUATION CRITERIA FOR OFF DAYS
TABLE VI
Of the significant groups, 12 had uncertainty levels under 0.20 SIGNIFICANT CLUSTERS THAT MEET EVALUATION CRITERIA FOR WORK DAYS
for both the training and test sets, had error rates of less than 0.20
for the percentage of observations and percentage of days mea-
sures, and were within one standard deviation of the parameter
mean for the time of day and length of time parameters. These 12
groups include four bedroom clusters, three office sensor clus-
ters, one living room cluster, one front door sensor cluster, one
laundry room cluster, and two kitchen clusters.
customized PDA, where several event types, as well as the lo-
C. Effect of Evaluating Work and Off-Days Separately
cation of the activity, could be selected. The subject was asked
When the off and work day data were analyzed separately, to log activity entries in as close a time proximity to the activity
the mean uncertainty for both the training and test sets was less event as possible. The PDA automatically stamps the entry with
than for the combined data set, showing an improvement in per- the correct date and time. The log consisted of 448 entries where
formance. The mean uncertainty for the work data at 0.034 56 one of 26 activity types was selected. These entries were com-
was especially low. Furthermore, the standard deviations of the pared to the significant clusters that met all of the evaluation
parameter values for the separate off and work day models were criteria for the combined, off-day, and work-day data sets.
in general smaller than for the combined data. As a result, de-
The clusters were compared to the log-entry events by
spite the smaller number of observations, there were a similar
noting for events of the same sensor type whether the time the
number of significant clusters and significant clusters that met
log-event was recorded occurred within one standard deviation
all the evaluation criteria. Table IV summarizes the comparison
of the mean start and end time for a cluster. For the combined
between the off-day, work-day, and combined data models.
data, 42.19% of the log-entry events occurred within one
The results of the off, work, and combined data sets were
standard deviation of the 12 clusters meeting all the evaluation
further compared on each of the evaluation criteria. The results
criteria. This percentage was 41.07% for the off-day clusters,
showed a significantly higher rate of errors in the percentage of
22.54% for the work-day clusters, and 59.15% when the off-
observations measure for the separate off and work day models.
and work-day clusters were combined. When clusters from all
Only 32.5% of the significant clusters for the off-day data and
three data sets were used, the percentage was 66.29%. This
28.6% of the clusters for the work-day data had percent errors
percentage included all bedroom and front-door entries 75.0%
less than 0.2 for the measure. This compares to 56.8% for the
of the office entries, 64.2% of the kitchen entries, 45.3% of the
combined model. Another measure where the error rates were
bathroom entries, 37.5% of the living room entries, and only
higher for the separate models was in the comparison of param-
5.4% of the laundry room entries. These results help validate
eter values to the test set mean time and mean length values.
that the results of the mixture-model analysis represent many
For the combined model, there was only one cluster for both at-
of the patterns of one event type that actually occurred.
tributes where the difference exceeded one standard deviation.
The clusters were then compared to the activity log to de-
In comparison, for the work-day data set fourteen observations
termine which clusters may potentially represent an activity.
exceeded a difference of one standard deviation for the time of
Of the 12 clusters from the combined data, nine represented at
day parameter and ten observations exceeded a difference of one
least 25% of the occurrences for at least one activity type. The
standard deviation for the length of time parameter. The uncer-
corresponding activities included sleep, waking-up, changing
tainty metric was the main one that showed improvement from
clothes, computer use, television use, and meal activities. For
the combined model to the separate models. Tables V and VI
the off-day data, seven of the clusters corresponded to an ac-
show the number of significant clusters that met each evalua-
tivity with the activities including all of the previous types ex-
tion criteria for the off and work day data sets.
cept television use plus toilet use. The work-day data had the
lowest number of clusters corresponding to activities at six. The
V. COMPARISON TO USER LOG
activities included sleep, changing clothes, toilet use, entering
An activity log from the subject was collected for a period the house from the front door, and eating. These results show
of 37 days that did not correspond to the days used in the mix- that most of the significant clusters correspond to actual event
ture-model analysis. The log was created by the subject using a patterns.
BARGER et al.: HEALTH-STATUS MONITORING THROUGH ANALYSIS OF BEHAVIORAL PATTERNS 27
VI. CONCLUSION [13] M. Ogawa, S. Ochiai, K. Shoji, M. Nishihara, and T. Togawa, “An at-
tempt of monitoring daily activities at home,” in Proc. 22nd Annu. EMBS
The results of this analysis demonstrate that a remote mon- Conf., Chicago, IL, Jul. 23–28, 2000, pp. 786–788.
itoring system with inexpensive motion sensors, such as the [14] M. Ogawa and T. Togawa, “Monitoring daily activities and behaviors at
home by using brief sensors,” in Proc. 1st Annu. Int. IEEE-EMBS Special
SmartHouse, can be used to detect behavioral patterns. The Topic Conf. Microtechnol. Med. Biol., Lyon, France, Oct. 12–14, 2000,
mixture-model results show that there are clusters that occur pp. 611–614.
consistently over time with low classification uncertainty. [15] M. Ogawa, R. Suzuki, T. Izutsu, T. Iwaya, and T. Togawa, “Long term
remote behavioral monitoring of elderly by using sensors installed in or-
Such clusters represent behavioral patterns of one event type. dinary houses,” in Proc. 2nd Annu. Int. IEEE-EMBS Special Topic Conf.
Comparison of these results to the user activity log shows Microtechnol. Med. Biol., Madison, WI, May 2–4, 2002, pp. 322–325.
that even though the data are from different time periods the [16] G. Virone and N. Noury, “A system for automatic measurement of cir-
cadian activity deviations in telemedicine,” IEEE Trans. Bio-Med. Eng.,
significant clusters accurately represent many of the recorded vol. 49, no. 12, pp. 1463–1469, Dec. 2002.
events. Identifiable events include sleep behavior, changing [17] J. H. Ward, “Hierarchical grouping to optimize an objective function,”
clothes, bathroom/toilet use, leaving/returning home, and meal J. Amer. Stat. Assoc., vol. 58, pp. 236–244, 1963.
[18] A. Yamaguchi, M. Ogawa, T. Tamura, and T. Togawa, “Monitoring be-
preparation, which constitute the majority of the activities of havior in the home using positioning sensors,” in Proc. 20th Annu. Int.
daily living that are used in functional assessments performed Conf. IEEE Eng. Med. Biol. Soc., vol. 20, 1998, pp. 1977–1979.
by healthcare professionals. Evaluating the work and off-days
separately helps to reduce the uncertainty in classification of
observations and provides a method for detecting event patterns
that may be specific to work or off-days. Tracy S. Barger received the B.S. and M.S. degrees
For future application, a SmartHouse-type system could be in systems engineering from the University of Vir-
ginia, Charlottesville, in 2003.
adapted to meet the needs of the individual user. Additional sen- In 2003, she began working as a Systems Engineer
sors could be used and placed where needed to monitor spe- for Northrop Grumman, Chantilly, VA.
cific concerns. The analysis methodology described in this paper
could then be utilized to provide a baseline characterization of
the user’s activity pattern, which could be refined through in-
teraction with the user, and provide a reference for detecting
behavioral changes. As such, the results described here provide
the basis for turning the current SmartHouse technologies into
ones with greater intelligence that can adapt the environment to
user needs and conditions. Donald E. Brown (A’86–SM’90–F’01) received
the B.S. degree from the United States Military
Academy, West Point, NY, in 1973, the M.S. and
REFERENCES M.Eng. degrees in operations research from the
[1] A Profile of Older Americans (2001). [Online]. Available: University of California, Berkeley, in 1979, and the
www.aoa.gov/aoa/stats/profile Ph.D. degree in industrial and operations engineering
[2] J. D. Banfield and A. E. Raftery, “Model-based Gaussian and from the University of Michigan, Ann Arbor, in
non-Gaussian clustering,” Biometrics, vol. 43, pp. 2–18, 1992. 1985.
[3] G. Celeux and G. Govaert, “Gaussian parsimonious clustering models,” He is a Professor and Chair of the Department
Pattern Recognit., vol. 28, pp. 781–793, 1995. of Systems Engineering, University of Virginia,
[4] B. G. Celler, E. D. Ilsar, and W. Earnshae, “Preliminary results of a Pilot Charlottesville. His research is statistical learning
project on remote monitoring of functional health status in the home,” in and predictive modeling.
Proc. 18th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., 1996, pp. 63–64.
[5] M. Chan, C. Hariton, P. Ringeard, and E. Campo, Smart House Automa-
tion Syst. Elderly Disabled, pp. 1586–1589.
[6] A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood
for incomplete data via the EM algorithm,” J. Royal Stat. Soc., Series B, Majd Alwan (S’94–M’96) received the B.S. degree
vol. 39, pp. 1–38, 1977. in electrical engineering from Damascus University,
[7] C. Fraley and A. E. Raftery, “How many clusters? Which clustering Syria, in 1988, the M.S. degree in control engi-
method? Answers via model-based cluster analysis,” Comput. J., vol. neering with distinction from Bradford University,
41, no. 8, pp. 578–588, 1998. Bradford, UK, in 1992, and the Ph.D. degree in
[8] , “MCLUST: Software for Model-Based Cluster and Discriminant intelligent robotics from the Imperial College of
Analysis,”, Tech. Rep. 342, Nov. 25, 1998. Science Technology and Medicine, University of
[9] G. L. Goodman, “Detection and classification for unattended ground London, London, UK, in 1997.
sensors,” in Proc. Inform. Decision, Control, Adelaide, Australia, 1999, From 1988 to 1992, he was a Design Engineer with
pp. 419–424. the Higher Institute of Applied Sciences and Tech-
K
[10] J. A. Hartigan and M. A. Wong, “A -means clustering algorithm,” nology (HIAST). After obtaining his Ph.D. in 1997,
Appl. Stat., vol. 28, pp. 100–108, 1979. he returned to HIAST as a Lecturer and Researcher. In addition to lecturing on
[11] T. Kohonen, “The self-organizing map,” Proc. IEEE, vol. 78, no. 9, pp. fuzzy control, control and logic systems, and digital circuits, he directed several
1464–1479, Sep. 1990. projects and consulted with many departments, organizations, and companies
[12] C. P. Mathews and K. Warwick, “Using cluster analysis techniques to on various instrumentation, automation, and computerization issues. In the be-
improve machine reliability,” in IEE Colloquium Intell. Meas. Syst. Con- ginning of 2002, he joined the Medical Automation Research Center, University
trol Applicat., Apr. 4, 1995, pp. 4/1–4/3. of Virginia as a Manager of the Robotics and ElderCare Technologies Program.