BLUE EYES TECHNOLOGY

CHAPTER 1

INTRODUCTION
Imagine yourself in a world where humans interact with computers. You are sitting in
front of your personal computer that can listen, talk, or even scream aloud. It has the ability to
gather information about you and interact with you through special techniques like facial
recognition, speech recognition, etc. It can even understand your emotions at the touch of the
mouse. It verifies your identity, feels your presents, and starts interacting with you.

You ask the computer to dial to your friend at his office. It realizes the urgency of the
situation through the mouse, dials your friend at his office, and establishes a connection. The
BLUE EYES technology aims at creating computational machines that have perceptual and
sensory ability like those of human beings. Employing most modern video cameras and
microphones to identifies the user’s actions through the use of imparted sensory abilities. The
machine can understand what a user wants, where he is looking at, and even realize his
physical or emotional states.

The U.S. computer giant, IBM has been conducting research on the Blue Eyes
technology at its Almaden Research Center (ARC) in San Jose, Calif., since 1997. The ARC
is IBM's main laboratory for basic research. The primary objective of the research is to give a
computer the ability of the human being to assess a situation by using the senses of sight,
hearing and touch. Animal survival depends on highly developed sensory abilities. Likewise,
human cognition depends on highly developed abilities to perceive, integrate, and interpret
visual, auditory, and touch information. Without a doubt, computers would be much more
powerful if they had even a small fraction of the perceptual ability of animals or humans.
Adding such perceptual abilities to computers would enable computers and humans to work
together more as partners. Toward this end, the Blue Eyes project aims at creating
computational devices with the sort of perceptual abilities that people take for granted. Thus
Blue eyes are the technology to make computers sense and understand human behavior and
feelings and react in the proper ways.

DVR & Dr. HS COLLEGE OF TECHNOLOGY 1

 BLUE EYES TECHNOLOGY

1.1 PARTS OF BLUE EYES TECHNOLOGY

The major parts in the Blue eyes technology are of two types. They are,

1. Data Acquisition Unit.

2. Central System Unit.

The tasks of the mobile Data Acquisition Unit are to maintain Bluetooth connections, to get
information from the sensor and sending it over the wireless connection, to deliver the alarm
messages sent from the Central System Unit to the operator and handle personalized ID
cards.

Central System Unit maintains the other side of the Bluetooth connection, buffers incoming
sensor data, performs on-line data analysis, records the conclusions for further exploration
and provides visualization interface.

Fig 1.1. Block Diagram of Blue eyes technology

DVR & Dr. HS COLLEGE OF TECHNOLOGY 2

 BLUE EYES TECHNOLOGY

1.1.1 DATA ACQUISITION UNIT

Data Acquisition Unit is a mobile part of the Blue eyes system. Its main task is to
fetch the physiological data from the sensor and to send it to the central system to be
processed. To accomplish the task the device must manage wireless Bluetooth connections
(connection establishment, authentication and termination). Personal ID cards and PIN
codes provide operator's authorization.

Communication with the operator is carried on using a simple 5-key keyboard, a

small LCD display and a beeper. When an exceptional situation is detected the device uses
them to notify the operator. Voice data is transferred using a small headset, interfaced to the
DAU with standard mini-jack plugs.

Fig 1.1.1. Block Diagram of Data Acquisition Unit

DVR & Dr. HS COLLEGE OF TECHNOLOGY 3

 BLUE EYES TECHNOLOGY

The Data Acquisition Unit consists of the following components, they are

 ATMEL 8952 microcontroller

 Bluetooth module – supports synchronous voice data transmission.
 PCM CODEC – used to transmit operator’s voice and central system sound feedback.
 UART (Universal Asynchronous Receiver Transmitter) – used for communication
between Bluetooth module and microcontroller (1152000 bps).
 MAX232 – level shifter.
 Alphanumeric LCD display.
 LED indicators.
 ID card Interface.
 Jazz Multi-sensor – used to capture the data of facial movements and gives them to
microcontroller for further processing.

The main features of Data Acquisition Unit are,

 The weight of the DAU is less.

 DAU runs on batteries, so it needs less power for working.
 Easy to use, does not disturb the operator working.
 ID cards are used for operator authorization.
 Voice transmission using hardware PCM CODEC

1.1.2 CENTRAL SYSTEM UNIT

Central System Unit hardware is the second peer of the wireless connection. The box
contains a Bluetooth module (based on ROK101008) and a PCM codec for voice data
transmission. The module is interfaced to a PC using a parallel, serial and USB cable. The
audio data is accessible through standard mini-jack sockets.

To program operator’s personal ID cards we developed a simple programming device.

The programmer is interfaced to a PC using serial and PS/2 (power source) ports. Inside,
there is Atmel 89C2051 microcontroller, which handles UART transmission and I2C
EEPROM (ID card) programming.

Blue Eyes software’s main task is to look after working operators’ physiological
condition. To assure instant reaction on the operators’ condition change the software

DVR & Dr. HS COLLEGE OF TECHNOLOGY 4

 BLUE EYES TECHNOLOGY

performs real time buffering of the incoming data, real-time physiological data analysis and
alarm triggering.

Each component in the central unit is having its own purpose. Connection manager is
used to maintain the wireless communication with the data acquisition unit. It establishes the
authentication, buffering of data and the alertness among the operators. Data analysis module
is used to perform the analysis of the physiological data from the users to that of the data
stored in the database.

The saccade detector, pulse rate analyzer, custom analyzers are three different analyzers used
in the data analysis module. Visualization module is a direct interface between the user and
system. The operator’s physiological condition can be viewed through it. Data logger is the
place where all the analyzed data are stored for further processing.

The Blue Eyes Central System Unit comprises of several functional modules:

 Connection Manager
 Data Analysis
 Visualization module
 Data Logger

Fig 1.1.2. Block Diagram of Central System Unit

DVR & Dr. HS COLLEGE OF TECHNOLOGY 5

 BLUE EYES TECHNOLOGY

Connection Manager:

Connection Manager is responsible for managing the wireless communication between the
mobile Data Acquisition Units and the central system. The Connection Manager handles:

 Communication with the CSU hardware

 Searching for new devices in the covered range establishing Bluetooth
connections.
 Connection authentication.
 Incoming data.
 Buffering sending alerts.

Data Analysis module:

Data Analysis Module performs the analysis of the raw sensor data in order to obtain
information about the operator’s physiological condition. The separately running Data
Analysis module supervises each of the working operators.

The module consists of a number of smaller analyzers extracting different types of

information. Each of the analyzers registers at the appropriate Operator Manager or another
analyzer as a data consumer and, acting as a producer, provides the results of the analysis.
The most important analyzers are:

 Saccade detector — monitors eye movements in order to determine the level of

operator’s visual attention.
 Pulse rate analyzer — uses blood oxygenation signal to compute operator’s pulse rate.
 Custom analyzers — recognize other behaviors than those which are built-in the
system. The new modules are created using C4.5 decision tree induction algorithm.

Visualization module:

Visualization module provides a user interface for the supervisors. It enables them to watch
each of the working operator’s physiological condition along with a preview of selected
video source and related sound stream. All the incoming alarm messages are instantly

DVR & Dr. HS COLLEGE OF TECHNOLOGY 6

 BLUE EYES TECHNOLOGY

signaled to the supervisor. The Visualization module can be set in an off-line mode, where all
the data is fetched from the database. Watching all the recorded physiological parameters,
alarms, video and audio data the supervisor is able to reconstruct the course of the selected
operator’s duty.

The physiological data is presented using a set of custom-built GUI controls:

 A pie-chart used to present a percentage of time the operator was actively

acquiring the visual information.
 A VU-meter showing the present value of a parameter time series displaying a
history of selected parameters’ value.

Data Logger Module:

Data Logger Module helps in storing the data from the database for processing. It stores all
the monitored data taken by the data acquisition unit in the form of data logs based on the
time basis. All the monitored data can be found in the data logger module. The Central
System unit uses the data logger as the memory component for storing and accessing data,
that acts as input reference for the processing.

The main feathers of the central system unit are

 Connection management
 Data processing
 Visualization
 Data recording
 Access verification
 System maintenance

For the monitoring and maintenance of the blue eyes technology we need three groups of
engineers. The three groups of engineers are needed for the better functioning of any blue
eyes technology. They are,

 Operator
 Supervisor
 System Administrator

Operator:

DVR & Dr. HS COLLEGE OF TECHNOLOGY 7

 BLUE EYES TECHNOLOGY

The operator is a person whose physiological parameters are supervised. The main works are

a). Authorization in the system

b). Receiving alerts

Supervisor:

The supervisor is the person responsible for analysing operator’s condition and performance.
The supervisor is like a second person in-charge.

System Administrators:

The main job of the system administrators is to perform the maintenance and performance
measurement. The system administrators also do the system deliver tools for adding,

a). New operator’s to the database.

b). Defining alarm conditions.

c). Configuring logging tools.

d). Creating new analyser modules.

2 AIMS OF BLUE EYES TECHNOLOGY

The main aims of the design of the blue eyes technology are

 To design smarter devices

 To create devices with emotional intelligence
 To create computational devices with perceptual abilities

The idea of giving computers personality or, more accurately, emotional intelligence” may
seem creepy, but the technologists say such machines would offer important advantages.

De-spite their lightning speed and awesome powers of computation, today’s PCs are
essentially deaf, dumb, and blind. They can’t see you, they can’t hear you, and they certainly
don’t care a whit how you feel. Every computer user knows the frustration of nonsensical
error messages, buggy software, and abrupt system crashes. We might berate the computer as
if it was an unruly child, but, of course, the machine can’t respond. “It’s ironic that people

DVR & Dr. HS COLLEGE OF TECHNOLOGY 8

 BLUE EYES TECHNOLOGY

feel like dummies in front of their computers, when in fact the computer is the dummy,” says
Rosalind Picard, a computer science professor at the MIT Media Lab in Cambridge.

A computer endowed with emotional intelligence, on the other hand, could recognize when
its operator is feeling angry or frustrated and try to respond in an appropriate fashion. Such a
computer might slow down or replay a tutorial program for a confused student, or recognize
when a designer is burned out and suggest he take a break. It could even play a recording of
Beethoven’s “Moonlight Sonata” if it sensed anxiety or serve up a rousing Springsteen
anthem if it detected lethargy. The possible applications of “emotion technology” extend far
beyond the desktop.

A car equipped with an affective computing system could recognize when a driver is feeling
drowsy and advise her to pull over, or it might sense when a stressed-out motorist is about to
explode and warn him to slow down and cool off.

Human cognition depends primarily on the ability to perceive, interpret, and integrate audio-
visuals and sensor information. Adding extraordinary perceptual abilities to computers would
enable computers to work together with human beings as intimate partners. Researchers are
attempting to add more capabilities to computers that will allow them to interact like humans,
recognize human presents, talk, listen, or even guess their feelings.

The main aim of the blue eyes technology is also to provide the data security that is collected,
which is because the data that is collected is more sensitive so we have to secure the data that
is collected by the blue eyes technology. The blue eyes technology provides data security by,

 Only registered mobile devices can connect to the system.

 Bluetooth connection authentication.
 Bluetooth connection encryption.
 Access right restrictions.
 Personal and Physiological data encryption.

DVR & Dr. HS COLLEGE OF TECHNOLOGY 9

 BLUE EYES TECHNOLOGY

3 TRACKS USED IN BLUE EYES TECHNOLOGY

Our emotional changes are mostly reflected in our heart pulse rate, breathing rate, facial
expressions, eye movements, voice etc. Hence these are the Parameters on which blue
technology is being developed. Making computers see and feel Blue Eyes uses sensing
technology to identify a user’s actions and to extract key information. This information is
then analysed to determine the user’s physical, emotional, or informational state, which in
turn can be used to help make the user more productive by performing expected actions or by
providing expected information. Beyond making computers more researchers say there is
another compelling reason for giving machines emotional intelligence. Contrary to the
common wisdom that emotions contribute to irrational behaviour, studies have shown that
feelings actually play a vital role in logical thought and decision making.

Emotionally impaired people often find it difficult to make decisions because they fail to
recognize the subtle clues and signals—does this make me feel happy or sad, excited or
bored?

That help direct healthy thought processes. It stands to reason, therefore, that computers that
can emulate human emotions are more likely to behave rationally, in a manner we can
understand. Emotions are like the weather. We only pay attention to them when there is a

DVR & Dr. HS COLLEGE OF TECHNOLOGY 10

 BLUE EYES TECHNOLOGY

sudden outburst, like a tornado, but in fact they are constantly operating in the background,
helping to monitor and guide our day-to-day activities.

Picard, who is also the author of the ground breaking book Affective Computing, argues that
computers should operate under the same principle. “They have tremendous mathematical
abilities, but when it comes to interacting with people, they are autistic,” she says. “If we
want computers to be genuinely intelligent and interact naturally with us, we must give them
the ability to recognize, understand, and even to behave’ and express emotions.”

4 AFFECTIVE COMPUTING

The process of making emotional computers with sensing abilities is known as affective
computing. The steps used in this are:

 Giving sensing abilities.

 Detecting human emotions.
 Respond properly.

The first step, researchers say, is to give ma-chins the equivalent of the eyes, ears, and other
sensory organs that humans use to recognize and express emotion. To that end, computer
scientists are exploring a variety of mechanisms including voice-recognition software that
can discern not only what is being said but the tone in which it is said; cameras that can track
subtle facial expressions, eye movements, and hand gestures; and biometric sensors that can
measure body temperature, blood pressure, muscle tension, and other physiological signals
associated with emotion.

DVR & Dr. HS COLLEGE OF TECHNOLOGY 11

 BLUE EYES TECHNOLOGY

In the second step, the computers have to detect even the minor variations of our moods. For
e.g. person may hit the keyboard very fast either in the happy mood or in the angry mood.

In the third step the computers have to react in accordance MJ with the emotional states.
Various methods of accomplishing affective computing are:

 AFFECT DETECTION.
 MAGIC POINTING.
 SUITOR.
 EMOTIONAL MOUSE. Etc.

5 TECHNOLOGIES UNDER BLUE EYES TECHNOLOGY

There are many technologies that have become a cutting edge technology, in our daily life.
Some of them are,

1) AFFECT DETECTION:

This is the method of detecting our emotional states from the expressions on our face.
Algorithms amenable to real time implementation that extract information from facial
expressions and head gestures are being explored. Most of the information is extracted from
the position of the eye rows and the corners of the mouth.

2) MAGIC POINTING:

MAGIC stands for Manual Acquisition with Gaze Tracking Technology. A computer with
this technology could move the cursor by following the direction of the user’s eyes. This type
of technology will enable the computer to automatically transmit information related to the
screen that the user is gazing at. Also, it will enable the computer to determine, from the
user’s expression, if he or she understood the information on the screen, before automatically

DVR & Dr. HS COLLEGE OF TECHNOLOGY 12

 BLUE EYES TECHNOLOGY

deciding to proceed to the next program. The user pointing is still done by the hand, but the
cursor always appears at the right position as if by MAGIC. By varying input technology and
eye tracking, we get MAGIC pointing.

3) SUITOR:

SUITOR stands for Simple User Interface Tracker. It implements the method for putting
computational devices in touch with their users changing moods. By watching what we page
the user is currently browsing, the SUITOR can find additional information on that topic. The
key is that the user simply interacts with the computer as usual and the computer infers user
interest based on what it sees the user do.

4) EMOTION MOUSE:

This is the mouse embedded with sensors that can sense the physiological attributes such as
temperature, Body pressure, pulse rate, and touching style, etc. The computer can determine
the user’s emotional states by a single touch. IBM is still Performing research on this mouse
and will be available in the market within the next two or three years. The expected accuracy
is 75%.

One goal of human computer interaction (HCI) is to make an adaptive, smart computer
system. This type of project could possibly include gesture recognition, facial recognition,
eye tracking, speech recognition, etc. Another non-invasive way to obtain information about
a person is through touch. People use their computers to obtain, store and manipulate data
using.

In order to start creating smart computers, the computer must start gaining information about
the user. Our proposed method for gaining user information through touch is via a computer
input device, the mouse. From the physiological data obtained from the user, an emotional
state may be determined which would then be related to the task the user is currently doing
on the computer. Over a period of time, a user model will be built in order to gain a sense of
the user’s personality.

The scope of the project is to have the computer adapt to the user in order to create a better
working environment where the user is more productive. The first steps towards realizing this
goal are described here.

DVR & Dr. HS COLLEGE OF TECHNOLOGY 13

 BLUE EYES TECHNOLOGY

6 EMOTION AND COMPUTING

6.1 INTRODUCTION:

Rosalind Picard (1997) describes why emotions are important to the computing community.
There are two aspects of affective computing: giving the computer the ability to detect
emotions and giving the computer the ability to express emotions. Not only are emotions
crucial for rational decision making but emotion detection is an important step to an adaptive
computer system. An adaptive, smart computer system has been driving our efforts to detect a
person’s emotional state.

By matching a person’s emotional state and the context of the expressed emotion, over a
period of time the person’s personality is being exhibited. Therefore, by giving the computer
a longitudinal understanding of the emotional state of its user, the computer could adapt a
working style which fits with its user’s personality. The result of this collaboration could
increase productivity for the user. One way of gaining information from a user non-
intrusively is by video. Cameras have been used to detect a person’s emotional state. We
have explored gaining information through touch. One obvious place to put sensors is on the
mouse.

6.2 THEORY:
DVR & Dr. HS COLLEGE OF TECHNOLOGY 14
 BLUE EYES TECHNOLOGY

Based on Paul Elman’s facial expression work, we see a correlation between a person
emotional state and a person’s physiological measurements. Selected works from Elman and
others on measuring facial behaviours describe Elman’s Facial Action Coding System
(Elman and Rosenberg, 1997).

DVR & Dr. HS COLLEGE OF TECHNOLOGY 15

 BLUE EYES TECHNOLOGY

Fig 6.1. Showing the psychological measurements

One of his experiments involved participants attached to devices to record certain

measurements including pulse, galvanic skin response (GSR), temperature, somatic
movement and blood pressure. He then recorded the measurements as the participants were
instructed to mimic facial expressions which corresponded to the six basic emotions. He
defined the six basic emotions as anger, fear, sadness, disgust, joy and surprise. From this
work, Dryer (1993) determined how physiological measures could be used to distinguish
various emotional states. The measures taken were GSR, heart rate, skin temperature and
general somatic activity (GSA). These data were then subject to two analyses. For the first
analysis, a multidimensional scaling (MDS) procedure was used to determine the
dimensionality of the data.

6.3 RESULT:

The data for each subject consisted of scores for four physiological assessments GSA, GSR,
pulse, and skin temperature, for each of the six emotions (anger, disgust, fear, happiness,
sadness, and surprise) across the five minute baseline and test sessions. GSA data was
sampled 80 times per second, GSR and temperature were reported approximately 3-4times
per second and pulse was recorded as a beat was detected, approximately 1 time per second.
To account for individual variance in physiology, we calculated the difference between the
baseline and test scores. Scores that differed by more than one and a half standard deviations
from the mean were treated as missing. By this criterion, twelve score were removed from the
analysis. The results show the theory behind the Emotion mouse work is fundamentally
sound.

DVR & Dr. HS COLLEGE OF TECHNOLOGY 16

 BLUE EYES TECHNOLOGY

7 MANUAL AND GAZE INPUT CASCADED (MAGIC) POINTING

7.1 INTRODUCTION:

This work explores a new direction in utilizing eye gaze for computer input. Gaze tracking
has long been considered as an alternative or potentially superior pointing method for
computer input. We believe that many fundamental limitations exist with traditional gaze
pointing. In particular, it is unnatural to overload a perceptual channel such as vision with a
motor control task. We therefore propose an alternative approach, dubbed MAGIC (Manual
and Gaze Input Cascaded) pointing. With such an approach, pointing appears to the user to be
a manual task, used for fine manipulation and selection. However, a large portion of the
cursor movement is eliminated by warping the cursor to the eye gaze area, which
encompasses the target.

Two specific MAGIC pointing techniques, one conservative and one liberal, were designed,
analysed, and implemented with an eye tracker we developed. They were then tested in a
pilot study. This early stage exploration showed that the MAGIC pointing techniques might
offer many advantages, including reduced physical effort and fatigue as compared to
traditional manual pointing, greater accuracy and naturalness than traditional gaze pointing,
and possibly faster speed than manual pointing.

In our view, there are two fundamental shortcomings to the existing gaze pointing techniques,
regardless of the maturity of eye tracking technology

First, given the one-degree size of the fovea and the subconscious jittery motions that the
eyes constantly produce, eye gaze is not precise enough to operate UI widgets such as
scrollbars, hyperlinks, and slider handles Second, and perhaps more importantly, the eye, as
one of our primary perceptual devices, has not evolved to be a control organ. Sometimes its

DVR & Dr. HS COLLEGE OF TECHNOLOGY 17

 BLUE EYES TECHNOLOGY

movements are voluntarily controlled while at other times it is driven by external events.
With the target selection by dwell time method, considered more natural than selection by
blinking, one has to be conscious of where one looks and how long one looks at an object. If
one does not, look at a target continuously for a set threshold (e.g., 200ms). The target will
not be successfully selected.

Once the cursor position had been redefined, the user would need to only make a small
movement to, and click on, the target with a regular manual input device. We have designed
two MAGIC pointing techniques, one liberal and the other conservative in terms of target
identification and cursor placement.

The liberal MAGIC pointing technique: cursor is placed in the vicinity of a target that the
user fixates on. Actuate input device, observe the cursor position and decide in which
direction to steer the cursor. The cost to this method is the increased manual movement
amplitude.

The conservative MAGIC pointing technique with "intelligent offset" To initiate a pointing
trial, there are two strategies available to the user. One is to follow "virtual inertia:" move
from tie cursor's current position towards the new target the user is looking at. This is likely
the strategy the user will employ, due to the way the user interacts with today's interface. The
alternative strategy, which may be more advantageous but takes time to learn, is to ignore the
previous cursor position and make a motion which is most convenient and least effortful to
the user for a given input device.

The goal of the conservative MAGIC pointing method is the following. Once the user looks
at a target and moves the input device, the cursor will appear "out of the blue" in motion
towards the target, on the side of the target opposite to the initial actuation vector. In
comparison to the liberal approach, this conservative approach has both pros and cons. While
with this technique the cursor would never be over-active and jump to a place the user does
not intend to acquire, it may require more hand-eye coordination effort. Both the liberal and
the conservative

MAGIC pointing techniques offer the following potential advantages,

 Reduction of manual stress and fatigue, since the cross screen long-distance cursor
movement is eliminated from manual control.

DVR & Dr. HS COLLEGE OF TECHNOLOGY 18

 BLUE EYES TECHNOLOGY

 Practical accuracy level. In comparison to traditional pure gaze pointing whose

accuracy is fundamentally limited by the nature of eye movement, the MAGIC
pointing techniques let the hand complete the pointing task, so they can be as accurate
as any other manual input techniques.
 A more natural mental model for the user. The user does not have to be aware of the
role of the eye gaze. To the user, pointing continues to be a manual task, with a cursor
conveniently appearing where it needs to be.
 Speed. Since the need for large magnitude pointing operations is less than with pure
manual cursor control, it is possible that MAGIC pointing will be faster than pure
manual pointing.
 Improved subjective speed and ease-of-use. Since the manual pointing amplitude is
smaller, the user may perceive the MAGIC pointing system to operate faster and more
pleasantly than pure manual control, even if it operates at the same speed or more
slowly.

The fourth point wants further discussion. According to the well accepted Fits' Law, manual
pointing time is logarithmically proportional to the A/W ratio, where A is the movement
distance and W is the target size. In other words, targets which are smaller or farther away
take longer to acquire.

For MAGIC pointing, since the target size remains the same but the cursor movement
distance is shortened, the pointing time can hence be reduced. It is less clear if eye gaze
control follows Fits' Law. In Ware and Michelin’s study, selection time was shown to be
logarithmically proportional to target distance, thereby conforming to Fits' Law. To the
contrary, Silber and Jacob [9] found that trial completion time with eye tracking input
increases little with distance, therefore defying Fits' Law.

DVR & Dr. HS COLLEGE OF TECHNOLOGY 19

 BLUE EYES TECHNOLOGY

Fig 7.1. Experiment task: point at paired targets

In addition to problems with today’s eye tracking systems, such as delay, error, and
inconvenience, there may also be many potential human factor disadvantages to the MAGIC
pointing techniques we have proposed, including the following:

 With the more liberal MAGIC pointing technique, the cursor warping can be
overactive at times, since the cursor moves to the new gaze location whenever the eye
gaze moves more than a set distance (e.g., 120 pixels) away from the cursor. This
could be particularly distracting when the user is trying to read. It is possible to
introduce additional constraint according to the context. For example, when the user’s
eye appears to follow a text reading pattern, MAGIC pointing can be automatically
suppressed.
 With the more conservative MAGIC pointing technique, the uncertainty of the exact
location at which the cursor might appear may force the user, especially a novice, to
adopt a cumbersome strategy: take a touch (use the manual input device to activate
the cursor), wait (for the cursor to appear), and move (the cursor to the target
manually). Such a strategy may prolong the target acquisition time. The user may
have to learn a novel hand-eye coordination pattern to be efficient with this technique.
Gaze position reported by eye tracker Eye tracking boundary with 95% confidence
True target will be within the circle with 95% probability The cursor is warped to the
boundary of the gaze area, along the initial actuation vector Previous cursor position,
far from target Initial manual actuation vector.
 With pure manual pointing techniques, the user, knowing the current cursor location,
could conceivably perform his motor acts in parallel to visual search. Motor action
may start as soon as the user’s gaze settles on a target. With MAGIC pointing
techniques, the motor action computation (decision) cannot start until the cursor
appears. This may negate the time saving gained from the MAGIC pointing
technique’s reduction of movement amplitude. Clearly, experimental (implementation
and empirical) work is needed to validate, refine, or invent alternative MAGIC
pointing techniques.

DVR & Dr. HS COLLEGE OF TECHNOLOGY 20

 BLUE EYES TECHNOLOGY

7.1.1 ADVANTAGES:

 Reduction of manual stress and fatigue.

 Practical accuracy level.
 A more natural mental model for the user.
 Faster than pure manual pointing.
 Improved subjective speed and ease of use

7.1.2. DISDVANTAGES:

 Liberal approach is distracting when the user is trying to read.

 The motor action computation cannot start until the cursor appears.
 In conservative approach, uncertainty of the exact location prolong the target
acquisition time.

7.2 IMPLEMENTING MAGIC POINTING:

We programmed the two MAGIC pointing techniques on a Windows NT system. The

techniques work independently from the applications. The MAGIC pointing program takes
data from both the manual input device (of any type, such as a mouse) and the eye tracking
system running either on the same machine or on another machine connected via serial port.
Raw data from an eye tracker cannot be directly used for gaze-based interaction, due to noise
from image processing, eye movement jitters, and samples taken during Saccade (ballistic
eye movement) periods.

The goal of filter design in general is to make the best compromise between preserving signal
bandwidth and eliminating unwanted noise. In the case, of eye tracking, as Jacob argued. The
eye information relevant to interaction lies in the fixations.

DVR & Dr. HS COLLEGE OF TECHNOLOGY 21

 BLUE EYES TECHNOLOGY

8 EYE TRACKER

Since the goal of this work is to explore MAGIC pointing as a user interface technique, we
started out by purchasing a commercial eye tracker (ASL Model 5000) after a market survey.
In comparison to the system reported in early studies this system is much more compact and
reliable. However, we felt that it was still not robust enough for a variety of people with
different eye characteristics, such as pupil brightness and correction glasses.

Fig 8.1.1. The liberal MAGIC pointing technique

Fig 8.1.2. The conservative MAGIC pointing technique with “intelligent offset”

DVR & Dr. HS COLLEGE OF TECHNOLOGY 22

 BLUE EYES TECHNOLOGY

Fig 8.2. Bright and Dark pupil images

From the figure, we can see that Bright (left) and dark (right) pupil images resulting from on-
axis and off-axis illumination. The glints, or corneal reflections, from the on- and off-axis
light sources can be easily identified as the bright points in the iris.

We hence chose to develop and use our own eye tracking system. Available commercial
systems, such as those made by ISCAN Incorporated, LC Technologies, and Applied Science
Laboratories(ASL), rely on a single light source that is positioned either off the camera axis
in the case of the ISCANETL-400 systems, or on-axis in the case of the LCT and the ASL
E504 systems.

Eye tracking data can be acquired simultaneously with MRI scanning using a system that
illuminates the left eye of a subject with an infrared (IR) source, acquires a video image of
that eye, locates the corneal reflection (CR) of the IR source, and in real time
calculates/displays/records the gaze direction and pupil diameter.

Once the pupil has been detected, the corneal reflection is determined from the dark pupil
image. The reflection is then used to estimate the user's point of gaze in terms of the screen
coordinates where the user is looking at. An initial calibration procedure, similar to that is
required by commercial eye trackers.

9 ARTIFICIAL INTELLIGENCE SPEECH RECOGNITION

DVR & Dr. HS COLLEGE OF TECHNOLOGY 23

 BLUE EYES TECHNOLOGY

It is important to consider the environment in which the speech recognition system has to
work. The grammar used by the speaker and accepted by the system, noise level, noise type,
position of the microphone, and speed and manner of the user’s speech are some factors that
may affect the quality of speech recognition .When you dial the telephone number of a big
company, you are likely to hear the sonorous voice of a cultured lady who responds to your
call with great courtesy saying “Welcome to company X. Please give me the extension
number you want”. You pronounce the extension number, your name, and the name of person
you want to contact. If the called person accepts the call, the connection is given quickly.
This is artificial intelligence where an automatic call-handling system is used without
employing any telephone operator.

9.1 TECHNOLOGY:

Artificial intelligence (Al) involves two basic ideas. First, it involves studying the thought
processes of human beings. Second, it deals with representing those processes via machines
(like computers, robots, etc). Al is behavior of a machine, which, if performed by a human
being, would be called intelligent. It makes machines smarter and more useful, and is less
expensive than natural intelligence.

Natural language processing (NLP) refers to artificial intelligence methods of communicating

with a computer in a natural language like English. The main objective of a NLP program is
to understand input and initiate action. The input words are scanned and matched against
internally stored known words. Identification of a key word causes some action to be taken.
In this way, one can communicate with the computer in one's language. No special
commands or computer language are required. There is no need to enter programs in a special
language for creating software.

9.2 SPEECH RECOGNITION:

DVR & Dr. HS COLLEGE OF TECHNOLOGY 24

 BLUE EYES TECHNOLOGY

The user speaks to the computer through a microphone, which, in used; a simple system may
contain a minimum of three filters. The more the number of filters used, the higher the
probability of accurate recognition. Presently, switched capacitor digital filters are used
because these can be custom-built in integrated circuit form. These are smaller and cheaper
than active filters using operational amplifiers.

The filter output is then fed to the ADC to translate the analogue signal into digital word. The
ADC samples the filter outputs many times a second. Each sample represents different
amplitudes of the signal .Evenly spaced vertical lines represent the amplitude of the audio
filter output at the instant of sampling. Each value is then converted to a binary number
proportional to the amplitude of the sample. A central processor unit (CPU) controls the input
circuits that are fed by the ADCS. A large RAM (random access memory) stores all the
digital values in a buffer area. This digital information, representing the spoken word, is now
accessed by the CPU to process it further. The normal speech has a frequency range of 200
Hz to 7 kHz. Recognizing a telephone call is more difficult as it has bandwidth limitation of
300Hz to 3.3 kHz.

As explained earlier, the spoken words are processed by the filters and ADCs. The binary
representation of each of these words becomes a template or standard, against which the
future words are compared. These templates are stored in the memory. Once the storing
process is completed, the system can go into its active mode and is capable of identifying
spoken words. As each word is spoken, it is converted into binary equivalent and stored in
RAM. The computer then starts searching and compares the binary input pattern with the
templates, t is to be noted that even if the same speaker talks the same text, there are always
slight variations in amplitude or loudness of the signal, pitch, frequency difference, time gap,
etc. Due to this reason, there is never a perfect match between the template and binary input
word. The pattern matching process therefore uses statistical techniques and is designed to
look for the best fit.

The values of binary input words are subtracted from the corresponding values in the
templates. If both the values are same, the difference is zero and there is perfect match. If not,
the subtraction produces some difference or error. Smaller the error, the better to match.
When the best match occurs, the word is identified and displayed on the screen or used in
some other manner. The search process takes a considerable amount of time, as the CPU has
to make many comparisons before recognition occurs. This necessitates use of very high-

DVR & Dr. HS COLLEGE OF TECHNOLOGY 25

 BLUE EYES TECHNOLOGY

speed processors. A large RAM is also required as even though a spoken word may last only
a few hundred milliseconds, but the same is translated into many thousands of digital words.
It is important to note that alignment of words and templates are to be matched correctly in
time, before computing the similarity score. This process, termed as dynamic time warping,
recognizes that different speakers pronounce the same words at different speeds as well as
elongate different parts of the same word. This is important for the speaker-independent
recognizers.

10 APPLICATIONS OF BLUE EYES TECHNOLOGY

DVR & Dr. HS COLLEGE OF TECHNOLOGY 26

 BLUE EYES TECHNOLOGY

Some of the applications of the blue eyes technology are,

 The blue eyes technology can be used in automobile industries. The emotional state of
the driver can be determined through the computer input device.
 The voice recognition system can be used in making airline and hotel reservations.
Reservation and cancellation of tickets, enquiries about the schedule can be simply
stated.
 Physiological condition monitoring.
 Used in military purposes. The machineries can be controlled by voice input
commands.
 Pilots instead of using their hands can give commands to the computer through
microphones.
 Household devices like televisions, refrigerators, ovens can do their jobs while we
speak to them.
 Radiologist scanning hundreds of X-rays, ultra sonograms, CT scans and
simultaneously dictating conclusions to a speech recognition system connected to
word processors. The radiologist can focus his attention on the images rather than
writing the text.
 One of the main benefits of speech recognition system is that it lets user do other
works simultaneously. The user can concentrate on observation and manual
operations, and still control the machinery by voice input commands.

11 CONCLUSION

DVR & Dr. HS COLLEGE OF TECHNOLOGY 27

 BLUE EYES TECHNOLOGY

The blue eyes technology paves the way for a simplified life through a more delicate
computer device. It is a very advanced system which avoids potential threats resulting from
human errors varying from weariness, oversight, tiredness, etc. Thus bluetooth provides
wireless communication and the movements of the eye enables us to collect a lot of
information about the user.

The identification of facial expressions has a lot of applications such as, medical applications
as pain detection, monitoring the depression and further more. The day when this technology
would be used at home is not far away. Even though this makes the people lazy, it is going to
be a technological forecast. The BLUE EYES technology ensures a convenient way of
simplifying the life by providing more delicate and user friendly facilities in computing
devices. Now that we have proven the method, the next step is to improve the hardware.
Instead of using cumbersome modules to gather information about the user, it will be better to
use smaller and less intrusive units. The day is not far when this technology will push its way
into your house hold, making you more lazy. It may even reach your hand held mobile
device. Any way this is only a technological forecast.

12 REFERENCES

 Meblin Babu, T. D. Subha, “Emotion Sensory World: A Review on Blue Eyes

Technology”, Department of Electronics and Communication, Holy Grace academy

DVR & Dr. HS COLLEGE OF TECHNOLOGY 28

 BLUE EYES TECHNOLOGY

of engineering, Thrissur, Kerala, India. Journal of Artificial Intelligence Research &

Advances, ISSN: 2395-6720(online), Volume 3, Issue 2.
 Swati International journal on Emerging Trends in Technology, Science and
Upcoming Research in Computer Science. DAVIM, Fariabad. Apr 2015.
 Zelinsky A, Heinzmann J. Real Time Visual Recognition of Facial Gestures for
Human Computer Interaction. In Proc. of the Int. Conf. on Automatic Face and
Gesture Recognition. 1996; 351–356p.
 Suvam Chatterjee, Haoshi. A Novel Neuro Fuzzy Approach to Human Emotion
Determination. Digital Image Computing Techniques and Application (DICIA), 2010
International Conference.
 https://www.reasearch.idm.com (online), blue eyes technology, emotion mouse.
 Levin J.L, An eye controlled computers.
 Silbert.L and R.Jacob, The advantage of eye gazing interactions.
 Colin ware, Harutune H.Mikaelian, An evalution of eye tracker.

DVR & Dr. HS COLLEGE OF TECHNOLOGY 29