See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/322486774

Smart Eye-tracking system

Conference Paper · January 2018

DOI: 10.1109/IWAIT.2018.8369701

CITATIONS READS

27 9,824

3 authors, including:

Aniwat Juhong
Michigan State University
14 PUBLICATIONS 102 CITATIONS

SEE PROFILE

All content following this page was uploaded by Aniwat Juhong on 14 January 2018.

The user has requested enhancement of the downloaded file.

 Smart Eye-Tracking System

Aniwat Juhong T. Treebupachatsakul and C. Pintavirooj

Department of Biomedical Engineering Department of Biomedical Engineering
Faculty of Engineering, King Mongkut’s Institute of Faculty of Engineering, King Mongkut’s Institute of
Technology Ladkrabang Technology Ladkrabang
Bangkok, Thailand Bangkok, Thailand
[email protected] [email protected]

Abstract— This project is a smart eye tracking system which is

designed for people with disabilities and elderly people. The This project aims to develop a prototype of an smart
concept of this research is to apply eye movement to control eye tracking system that can controlled appliances, electric
appliances, wheelchair and communicate with caretaker. This wheelchair and communicate with caretaker via send message
system comprises four components, imaging processing module, to smartphone.
wheelchair-controlled module, appliances-controlled module and
SMS manager module. The image processing module consists of II. RESEARCH METHOD
webcam and C++ customized image processing, the eye
movement image is captured and transmitted to Raspberry Pi
microcontroller for processing with OpenCV to derive the This system comprises four components including imaging
coordinate of eye ball. The coordinate of eye ball is utilized for processing module, appliances-controlled module, wheelchair-
cursor control on the Raspberry Pi screen to control the system. controlled module and SMS manger module. The imaging
Besides the eye movement, the eye blink is applied in this system processing module is the main module of this system that start
for entering a command as when you press Enter button on with webcam that installed on eyeglass as show in figure 2 (a)
keyboard. The wheelchair-controlled module is a cradle with two
to capture the eye images and transmit to Raspberry Pi for
servos that can be moved to two dimensions and also adaptable to
other wheelchair joysticks. This system also remotely controls image processing which is based on C++ with OpenCV to
some appliances and communicate with caretaker via send obtain position of eye ball and define eye blink. The eye
message to smartphone. movement is also employed as the cursor control on the
Raspberry Pi screen eye blink, is used for entering command,
therefore these feature can be used to control several
Keywords— eye tracking, wheelchair, people with disabilities,
applications.
elderly people, Raspberry Pi, image processing.

I. INTRODUCTION (i) Circular Hough transform

The circle Hough transform (CHT) is algorithm to detect
Presently, wheelchairs became important for elderly and
circle, First the edge of circle will be found by using Hough
disabled persons. The electrical wheelchairs are available in
gradient method after that, in each pixel on the edge of this
market mostly controlled by using a joystick control system.
circle will be created other circle so the position that have the
In some case such as totally paralysis person in amyotrophic
most pixels accumulation from this circle creation is center of
Lateral Sclerosis (ALS) and Parkinson disease, it may be very
circle which we would like to detect as figure 1. Because of
difficult or impossible for such patient to use such type of
the shape of eye ball is circle hence we can apply this
system. There are several researches attempt to develop other
algorithm to detect it.
techniques for controlling electric wheelchair for those patients
to improve quality of their life. Such as brain-controlled
wheelchair [1], The brain signal (Electroencephalography,
EEG) is captured by electrode cap that is placed on the user’s
scalp then this signal translated into movement commands but
it have many disadvantages such as spend a lot of time for
setup the system, slow response, expensive, etc. Voice
controlled wheelchair [2], patients speak command to move the
wheelchair but the drawback of voice controlled wheelchair is
the low immune to noisy environment that can bother system
to incorrectly respond.
Figure 1 Circle Hough transform Algorithm [3]
 (a) (b)
Figure 2 (a) webcam that installed on eyeglass as (b) eye ball detection based
(a) (b)
Figure 4 (a) Cursor control on the Raspberry Pi Screen (b) wheelchair-
on circle Hough transform
controlled

(ii) Eye blink detection

(iii) Eye position Control
The eye blink detection is used to simulate the enter
The eyeball position is used to control the mouse cursor
keyboard. To detect the eye blink, the number of black pixel
position. Algorithm for eyeball position is as follows.
in the pupil area is measured. The change from high to low of
the dark pixel number is designated for the eye-blink
(I) Determine the average position of eyeball. The average
detection; i.e. the enter key.
position, denoted as AVGX and AVGY are the average of
position of eyeball in relaxing position, i.e. eyeball looking
forward

(II) Detect the current position from Hough circle detection.

Computer the deviation of current position from average point
denoted the deviation as x and y.

(III) Determine the motion direction of eyeball as

Turning left if -60<x<30
Turning right if 30<x<60
Turning up if -30<y<-50
Turning down if 30<y<50
Figure 5 shows the four motion directions.

Figure 3 The system overview of the smart eye tracking system

The webcam is connected to Raspberry Pi microprocessor

that not only performs digital image processing but it also
interacts with other module including appliances-controlled, (a) (b)
wheelchair-controlled and SMS mange module. Appliances
controlled module, another Arduino is used to receive
command from Raspberry Pi by wireless communication, the
Arduino will turn relay on or off according to the command of
Raspberry Pi, this relay is used to turn Appliances on or off. In
wheelchair control, we designed cover of wheelchair joystick (c) (d)
that can move wheelchair joystick in two dimension with two Figure 5 Eyeball motion direction
servos as show in figure 5. In SMS mange module, The
WeMOS D1 mini is used to send the massage to Blynk
Application [4], it also receives command from Raspberry Pi.
III. EXPERIMENTAL AND RESULT

The eye movement and blink were used to controlled user

interface for smart eye tracking system as show in figure 4.
The movement of the cursor will be moved following eye
movement. The result to control wheelchair movement, turn
appliances on or off and communicate with caretaker as show
in figure 6, 7 ,8 respectively.
 (iv) Comparison with the commercial eye tracking system

We have compared the purposed system with commercial

eye-tracker modules- the Gazepoint GP3 Eye tracker. The
composition table is shown in table 1.

(a) Stop (b) Forward

(a) (b)
Figure 9. (a) Gazepoint GP3 Eye tracker module (b) Eye tracking by
Gazepoint GP3 Eye tracker.
(c) turning right (d) turning left
Figure 6 the wheelchair-controlled module (2 dimensional rotating Table 1 Comparison with commercial module
stage)
Eye tracking by camera Eye tracking by
installed on eye glass Commercial eye tracker
module
Fast Respond Satisfied Outstanding
Low cost Lower $20 At least $700
contactless eye
tracking system
Specification
CPU 900 MHz or faster, Intel core i7 or faster, 8 GB
1GB RAM RAM

Screen size Compatible with all Compatible with 24″

(a) (b) display displays or smaller
Figure 7 (a) turn on the lamp (b) turn off the lamp
Table 1 : Comparison Table

(v) Testing
To test the tracking performance, we design the phantom
consisting of 5 red dots on various location as shown in figure
10. The subject was asked to stare at the red dot for 10 second.
The position is collected for 10 times. The average and
standard deviation was computed and tabulated. The result was
showed in table 2. The results demonstrate the compatible
tracking performance when compared with the commercial
one.

(a) (b)
Figure 8 (a) User interface on raspberry Pi monitor (b) Notification on smart
phone
 Reference

[1] Abhilash Kodi, Dixit Kumar, Divya Kodali and I.A.Pasha,“EEG

Controlled Wheelchair for ALS patients,” 2013 International Conference
on Communication Systems and Network Technologies, 2013.
[2] Romil Chauhan, Yash Jain, Harsh Agarwal and Abhijit Patil,“Study of
Implementation of Voice Controlled Wheelchair,” 2016 3rd
International Conference on Advanced Computing and Communication
Figure 10 Tracking tested phantom
Systems (ICACCS-2016), 2016.
[3] Circular Hough Transform based algorithm for finding circles in image.
Retrieved http://www.mathworks.com/images/ref/imfindcircles.html
Eye tracking based circle Eye tracking by [4] https://www.wemos.cc/
Hough transform(CHT) Commercial eye tracker
module
x-axis y-axis x-axis y-axis
Central AVG 20.587 16.357 19.6245 17.698
middle
STD 17.589 16.2145 23.9902 12.7427
Upper AVG 16.547 15.479 15.2581 16.52848
right STD 18.2648 19.5269 22.3691 25.2887
Upper AVG 16.528 19.478 18.4856 18.5988
left
STD 21.5609 17.2348 20.5979 21.6327
Lower AVG 17.587 18.568 16.4367 16.1479
right STD 16.4156 22.9870 22.1560 21.8797
Lower AVG 18.5689 17.5686 15.8791 17.4189
left STD 24.526 22.5970 23.2565 21.5791
Table 2 : precision of cursor position denoted by AVG(average) and STD(
stand deviation)

IV. CONCLUSION AND DISCUSSION

The smart eye tracking system is presented in this
paper. It not only to control wheelchair movement but it also
controls appliances and communicate with caretaker. The eye
movement is used to control cursor that appear on monitor and
eye blink used for entering command but the disadvantage of
this system is a webcam that attached on eyeglass is large size,
it will be bother vision of user However, our system is good
enough to be used for people with disability. In the future work
we would like to improve performance of this system based
cost-effective condition.

ACKNOWLEDGMENT
This paper was supported by the Biomedical
Engineering Program, Faculty of Engineering, King Mongkut’s
Institute of Technology Ladkrabang (KMTIL)

View publication stats