A

paper
PRESENTATION ON

Blue Eye
Monitoring HumanOperator System
-_-_-_

Compiled By :
Name: SRIDHAR NANDIPATI
III-ECE
e- mail:[email protected]

Name: santosh meenavalli

III-ECE
e-mail:
[email protected]

BLUE EYES
ABSTRACT:
Blue Eye technology aims at creating
computational Machines with perceptual
and sensory abilities like those of human
beings. The Blue Eye system has hardware
with software loaded on it. Human error is
still one of the most frequent causes of
catastrophes and ecological disasters. The
main reason is that the monitoring systems
concern only the state of the processes
whereas human contribution to the overall
performance of the system is
left
unsupervised.
Since
the
control
instruments are automated to a large
extent, a human operator becomes a
passive observer of the supervised system,
which results in weariness and vigilance
drop. Thus, he may not notice important
changes of indications causing financial or
ecological consequences and a threat to
human life. It therefore is crucial to
assure that the operators conscious brain
is involved in an active system supervising
over the whole work time period.
Blue Eyes - The system developed
intended to be the complex solution for
monitoring and recording the operators
conscious brain involvement as well as
his
physiological
condition. This
required designing a Personal Area
Network linking all the operators and the
supervising system. As the operator using
his sight and hearing senses the state of
the controlled system, the supervising
system will look after his physiological
condition.Blue Eye system can be applied
in every working environment requiring
permanent operator's attention for it. The
hardware comprises of data acquisition
unit and central system unit. The heart of
Data acquisition unit is ATMEL 89C52
microcontroller. Bluetooth technology is
used for communication and coordination

between the two units. Blue eye system

can be applied in every working
environment, which requires permanent
operators attention. Blue eye system
provides technical means for monitoring
and
recording
human
operators
physiological condition. A blue eye is a
project aiming to be a means of stress
reliever driven by the advanced,
technology of studying the facial
expressions for judgment of intensity of
stress handled.
In totality blue eye aims at adding
perceptual abilities which would end up in
a healthy stress free environment and can
be applied in every working environment
requiring permanent operators attention.
INTRODUCTION:
Blue Eyes system provides technical
means for monitoring and recording
the
operators
basic physiological
parameters. The most important parameter
is saccadic activity, which enables the
system to monitor the status of the
operators visual attention along with head
acceleration, which accompanies large
displacement of the visual axis. Complex
industrial environment can create a danger
of exposing the operator
to toxic
substances, which can affect his cardiac,
circulatory and pulmonary systems. Thus,
on the grounds of plethysmographic signal
taken from the forehead skin surface, the
system computes heart beat rate and blood
oxygenation.The B lueEyes system checks
above parameters against abnormal (e.g. a
low level of blood oxygenation or a high
pulse rate) or undesirable (e.g. a longer
period of lowered visual attention) values
and triggers user-defined alarms when
necessary. This paper is about the
hardware,
software,
benefits
and

interconnection of various parts involved

in the blue eye technology.
Toward this end, the Blue Eyes aims at
creating computational devices with the
sort of perceptual abilities that people take
for granted Blue eyes is being developed
by the team of Poznan University of
Technology& Microsoft. It makes use of
the blue tooth technology developed by
Ericsson.
Why its named Blue Eyes?
Blue Eyes emphasizes Bluetooth
technology and the movements of the
eyes. Bluetooth provides reliable wireless
communication
whereas
the
eye
movements enable us to obtain a lot of
interesting and important information.

What Is Blue Eyes?

BLUE EYES is a technology, which
aims
at
creating computational
machines
that have perceptual and
sensory abilities like those of human
beings.

It provides technical means for

monitoring and recording
humanoperators physiological condition. The
major parts in the Blue eye system are
Data Acquisition Unit and Central System
Unit. The tasks sensor and sending it over
the wireless connection, to deliver the
alarm messages sent from the Central
System Unit to the operator and handle
personalized ID cards. Central System
Unit maintains the other side of the
Bluetooth connection, buffers incoming
sensor data, performs on-line data analysis,
records the conclusions for further
exploration and provides visualization
interface of the mobile Data Acquisition
Unit
are
to
maintain
Bluetooth
connections, to get information from the
sensor and sending it over the wireless
connection, to deliver the alarm messages
sent from the Central System Unit to the
operator
and handle personalized ID
cards. Central System Unit maintains the
other side of the Bluetooth connection,
buffers incoming sensor data, performs online data analysis, records the conclusions
for further exploration and provides
visualization interface.

The basic idea behind this technology is

to give computer human power.
For example, we can understand humans
emotional state by his facial expressions. If
we add these perceptual abilities
to
computers, we would enable them to work
together with human beings as intimate
partners.

PARTS
OF
TECHNOLOGY:

BLUE

EYE

The main parts in the Blue eye system are

1. Data Acquisition Unit

2. Central System Unit

Data Acquisition Unit (DAU)
Data Acquisition Unit is a mobile
part of the Blue eyes system. Its
main task is to fetch the physiological
data from the sensor and to send it to the
central system to be processed. To
accomplish the task the device must
manage wireless Bluetooth connections
(connection establishment, authentication
and termination). Personal ID cards and
PIN codes
provide
operator's
authorization.
Communication with the operator
is
carried on using a simple 5-key keyboard,
a small LCD display and a beeper. When
an exceptional situation is detected the
device uses them to notify the operator.
Voice data is transferred using a small
headset, interfaced to the DAU with
standard mini-jack plugs.

1. Physiological data sensor

Jazz multisensor

To provide the Data Acquisition Unit with

necessary physiological data we decided to
purchase an off- shelf eye movement
sensor Jazz Multisensor . It supplies raw
digital data regarding eye position, the
level of blood oxygenation, acceleration
along horizontal and vertical axes and
ambient light intensity.
Eye movement is measured using direct
infrared oculographic transducers. The eye
movement is sampled at 1kHz, the other
parameters at 250 Hz. The sensor sends
approximately 5,2kB of data per second.
2. Hardware specification
We
have
chosen
Atmel
8952
microcontroller to be the core of the Data
Acquisition Unit since it is a wellestablished industrial standard and

provides necessary functionality (i.e. high

speed serial port) at a low price. The figure
below shows the other DAU components .
Since the Bluetooth module we received
supports
synchronous
voice
data
transmission (SCO link) we decided to use
hardware PCM
codec to transmit
operators voice and central system sound
feedback. The codec that we have
employed reduces the microcontrollers
tasks and lessens the amount of data
being
sent
over
the
UART.
Additionally, the Bluetooth module
performs voice data compression, which
results in smaller bandwidth utilization
and better sound quality. Communication
between the Bluetooth module and the
microcontroller
is carried on using
standard UART interface..

The simple keyboard is used to react to

incoming events (e.g. to silence the alarm
sound) and to enter PIN code while
performing authorization procedure.

The alphanumeric LCD display (two 8character lines) gives more information of
incoming events and helps the operator
enter PIN code

3. Microcontroller software specification

The LED indicators show the results

of built-in self-test, power level and
the state of wireless connection.

ID card interface helps connect the

operators personal identification card to
the DAU. After inserting the card
authorization procedure starts. In the
commercial release a cryptographic
processor should be used instead. Each ID
card is programmed to contain: operators
unique identifier, device access PIN code
the operator enters on inserting his ID card
and system access PIN code that is used on
connection authentication. The operators
unique identifier enables the supervising
system to distinguish different operators.

All the DAU software is written in 8051

assembler code, which assures the highest
program efficiency and the lowest
resource. At a low-level design stage we
modeled the software using a state
diagram.
Such a diagram facilitates implementation,
debugging and testing phases.
Central System Unit (CSU)

Central System Unit hardware is the

second peer of the wireless connection.
The box contains a Bluetooth module
and a PCM codec for voice data
transmission. The module is interfaced to
a PC using a parallel, serial and USB
cable. The audio data is accessible through
standard mini jack sockets.
To program operator's personal ID cards
we developed a simple programming
device. The programmer is interfaced to
a
PC using serial and PS/2(power
source) ports. Inside, there is Atmel
89C2051 microcontroller, which handles
UART transmission and I2C EEPROM
(ID card) programming.
In this section we describe the four main
CSU modules (see Fig. 1): Connection
Manager, Data Analysis, Data Logger and
Visualization.
1. Connection Manager
It is responsible for managing the wireless
communication between the mobile Data
Acquisition Units and the central system.
The Connection Manager handles:
communication with the CSU hardware
searching for new devices in the covered
range

physiological condition. The separately

running Data Analysis Module supervises
each of the working operators. The module
consists of a number of smaller analyzers
extracting different types of information.
Each of the analyzers registers at the
appropriate Operator Manager or another
analyzer as a data consumer and, acting as
a producer, provides the results of the
analysis. An analyzer can be either a
simple signal filter (e.g. Finite Input
Response (FIR) filter) or a generic data
extractor (e.g. signal variance, saccade
detector) or a custom detector module.
As we are not able to predict all the
supervisors
needs,
the custom
modules are created by applying a
supervised machine learning algorithm
to a set of earlier recorded examples
containing the characteristic features to be
recognized.
In the prototype we used an improved
C4.5 decision tree induction algorithm.
The computed features can be e.g. the
operators position (standing, walking and
lying) or whether his eyes are closed or
opened. As built-in analyzer modules
we implemented a saccade detector,
visual attention level, blood
oxygenation and pulse rate analyzers.

establishing Bluetooth connections

connection authentication
incoming data buffering
sending alerts
2. Data Analysis Module
The module performs the analysis of
the raw sensor data in order to obtain
information
about the operators

The saccade detector

registers as an eye movement and
accelerometer signal variance data
consumer and uses the data to signal
saccade occurrence. Since saccades are the

fastest eye movements the algorithm

calculates eye movement velocity and
checks physiological constraints.
The visual attention level analyzer uses as
input the results produced by the saccade
detector. Low saccadic activity (large
delays
between subsequent saccades)
suggests lowered visual attention level
(e.g. caused by thoughtfulness). Thus, we
propose a simple algorithm that calculates
the visual attention level (L ): L =
100/t , where t denotes the time (in va va s
10 s1 0 seconds) occupied by the last ten
saccades. Scientific research has proven
that during normal visual information
intake the time between consecutive
saccades should vary from 180 up to 350
ms. This gives L at 28 up to 58 units. The
values of L lower than 25 for a longer
period of time should cause a va v a
warning condition. The following figure
shows the situation where the visual
attention lowers for a few seconds. The
Pulse rate analyzer registers for the
oxyhemoglobin and deoxyhemoglobin
level data streams. Since both signals
contain a strong sinusoidal component
related to heartbeat, the pulse rate can be

To test the possibilities and performance

of the remaining parts of the Project Kit
(computer, camera and database software)
BlueCapture had been created . The tool
supports capturing video data from various
sources
(USB web-cam, industrial

calculated measuring the time delay

between subsequent extremes of one of the
signals
TOOLS USED
During the implementation of the DAU
there was a need of a piece of software to
establish and test Bluetooth connections.
Therefore a tool had been created
called BlueDentist . The tool provides
support for controlling the currently
connected Bluetooth device. Its functions
are:
local device management and
connection management.

camera) and storing the data in the MS

SQL Server database. Additionally, the
application performs sound recording.
After
filtering
and
removing
insignificant fragments (i.e. silence) the
audio data is stored in the database.
Finally, the program plays the recorded
audiovisual stream. The software was used
to measure database system performance
and to optimize some of the SQL
queries. Since all the components of the

application have been tested thoroughly

they were reused in the final software,
which additionally reduced testing time.

at captain bridges

A simple tool was created for recording

Jazz Multisensor measurements. The
program reads the data using a parallel
port and writes it to a file.

Data security - This system implies data

security which is require in the modern
network system.

ADVANTAGES
Prevention from dangerous incidents
Minimization of ecological consequences
financial loss a threat to a human life
BlueEyes system provides technical means
for monitoring and recording
Human-operators
physiological
condition. The key features of the system
are:
visual attention monitoring (eye motility
analysis)
physiological condition monitoring
(pulse rate, blood oxygenation)
operator's position detection (standing,
lying)
wireless data acquisition using Bluetooth
technology
real-time user-defined alarm triggering
physiological data, operator's voice and
overall view of the control
room recording
recorded data playback
BlueEyes system can be applied in every
working environment requiring permanent
operator's
attention:
at power plant control rooms

at flight control centers

Only registered mobile devices can

connect to the system
Bluetooth connection authentication &
encryption
Access rights restrictions
Personal
encryption

and

physiological

data

DISADVANTAGES
Doesnt predict nor interfere with
operators thoughts.
Cannot force directly the operator to
work.
APPLICATIONS
1.It can be used in the field of security &
controlling, where the contribution of
human operator required in whole time.
2. Engineers at IBM's office:smarttags
Research Center in San Jose, CA, report
that a number of large retailers have
implemented surveillance systems that
record and interpret customer movements,
using software from Almaden's BlueEyes
research project. BlueEyes is developing
ways for computers to anticipate users'
wants by gathering video data on eye
movement and facial expression. Your
gaze might rest on a Web site heading, for
example, and that would prompt your
computer to find similar links and to call
them up in a new window. But the first

practical use for the research turns out to

be snooping on shoppers.
3. Another application would be in the
automobile industry. By simply touching a
computer input device such as a mouse,
the computer system is designed to be able
to determine a person's emotional state.
4. Current interfaces between computers
and humans can present information
vividly, but have no sense of whether that
information is ever viewed or understood.
In contrast, new real-time computer vision
techniques for perceiving people allows us
to create "Face-responsive Displays" and
"Perceptive Environments", which can
sense and respond to users that are viewing
them. Using stereo-vision techniques, we
are able to detect, track, and identify users
robustly and in real time. The results of
our research will allow the interface
between computers and human users to
become more natural and intuitive
FUTURE TREND
What can we do with blue eyes
technology?
It has the ability to gather information
about you and interact with you through
special techniques like facial recognition,
speech recognition, etc.
It can even understand your emotions at
the touch of the mouse.
It can verify your identity, feel your
presence, and start interacting with you.
The machine can understand what a user
wants, where he is looking at, and even
realize his physical or emotional states.
It realizes the urgency of the situation
through the mouse.

For instance if you ask the computer to

dial to your friend at his office, it
understands the situation and establishes a
connection.
It can reconstruct the course of operators
work.
CONCLUSION
BlueEyes need for a real-time monitoring
system for
a human operator. The
approach is innovative since it helps
supervise the operator not the process,
as it is in presently available solutions.
This system in its commercial release will
help avoid potential threats resulting from
human errors, such as weariness,
oversight,
tiredness
or
temporal
indisposition. In future it is possible to
create a computer which can interact with
us as we interact each other with the use of
blue eye technology. It seems to be a
fiction, but it will be the life lead by
BLUE EYES in the very near future.
Ordinary household devices -- such as
televisions, refrigerators, and ovens -- may
be able to do their jobs when we look at
them and speak to them