2015 Fifth International Conference on Communication Systems and Network Technologies

Driver Alert System For Accident Avoidance

Nidhi Sinha, Prabhat Kumar, M P Singh and Jyoti Prakash Singh

Computer Science and Engineering,
National Institute of Technology Patna, Bihar, India
Email: [email protected], {prabhat, mps, jps}@nitp.ac.in

Abstract—Road accidents have become the major issue during

these days. Accidents not only bring loss to life and property but
also for our economy. Modern technologies like digital image
processing, ECG (Electrocardiogram), EEG (Electroencephalo-
gram) etc. can help us to prevent or avoid the accidents. Much
remarkable work has been done on the driver alert system. The
paper provides a review of work that has been done in this field.
Keywords- Drowsiness Detection, Eyes Detection, Face Detec-
tion, LBP (Local Binary Pattern), Yawning Detection. TABLE I. ACTIVITIES DURING D RIVING [4]

I. I NTRODUCTION
According to statistics [1] more than 1.2 million people Many of the activities has been performed by the driver while
die each year on the road accidents. 20 - 50 million of people driving shown in the given table.
suffer from non-fatal injuries due to these road accidents across
the world. According to the NHTSA (National Highway Traffic
Safety Administration) in the year 2008, it estimated that 5,870 B. Parameters of Driver Inattention
deaths, 7,45,000 and 350,000 injuries are due to lack of driver
concentration on the road. Damages of billions per year due Several methods has been used to measure driver inat-
to cell phone related crashes occur in U.S [1]. India has the tention. Dinguss [5] measures the glimpse length, number of
highest number of road accidents in the year 2013, according glimpse and frequency of use. The glimpse parameters requires
to the recent world health organization report nearly 1,05,725 a image observing for the drivers face and eyes. Some other
people died in the year 2013 from road accidents. 77.5% approaches has been used for time period and accurateness for
of total road accidents are due to drivers fault. In Europe, the replacement of secondary task such as Peripheral Detection
the estimation shows that 10%-20% of all traffic accidents Task (PDT). These measures are yet not measures in real-
happened due to driver’s fatigue [2]. 25% of all the crashes time for everyday driving. Boer [6] used a driver performance
are due to driver fatigueness and drowsiness. Statistics reveal measure, steering entropy, to measure workload, which unlike
that driver inattention is a major issue for the road accidents eye gaze and some of the secondary tasks is practical for
in the world. Driver inattention may be due to his/her lack of everyday monitoring, and can be calculated in real-time. In this
concentration on secondary tasks like talking on the cell phone, work indicates that it is able to detect driver inattention and
taking a meal, mentally disturbed. Drivers stress behavior driver distraction while engaged in other tasks while driving.
may be the reason for lack of inattention, which causes the
accidents. These crashes only happen between midnight to 6
a.m. [3]. Driver influence of alcohol, it causes severe damage C. Drowsiness detection can be divided into three main classes
of life and property as well as the lost of the economy.
Alcohol and Drugs is the major factor for road accidents. Due (1) Vehicle based evaluation (2) Behavioral based evalua-
to drunk driving driver lost their control and accidents are tion (3) Physiological based evaluation. A comprehensive eval-
occurs. In recent years the number of accidents in vehicle has uation on these parameters will afford insight on the current
been increased manifold due to stress/mobile phone use while systems, concerns related with them and the improvement and
driving, aggressive driving, sleeping disorders, driver’s yawn development that need to be done to make a vigorous system.
frequently, tending to children, mentally depressed, eating a Three different methods for Drowsiness detection has been
meal, grooming etc. as there is a lack of concentration of driver described in Fig 1. An elobrated review on three parameters
while driving. The deaths and financial losses are due to road are discussed in this paper.
accidents.
1) Vehicle based evaluation: Vehicle based measure de-
scribed the divergence from its original path, rotation of
A. Driver Inattention
the steering wheel of the car, pedal movement etc. These
According to the survey, driver inattention is the major parameters are continuously observing and if any modification
issue now a days for the whole world. Driver Inattention is in these occurs that crosses a specified threshold, that means
becoming more and more challenging now a days in whole there is a drastically increment of the probability that the driver
world. The NET survey in 2001 is given below in Table I . is drowsy.

978-1-4799-1797-6/15 $31.00 © 2015 IEEE 1293

DOI 10.1109/CSNT.2015.260
 Fig. 3. Adaptive Crucial Control [16]

Fig. 4. Collision Warning with Brake Support [14]

the purpose of driving pattern and status recognition. They

Fig. 1. Approaches for Drowsiness Detection and Warnings[7] are using to observe the drivers images and monitor the
driver continuously. In [10], Heitmann et al. described that
to monitoring head movement, eye movement for alerting the
driver is their any variation has been done. In [11], ilkwon
park, proposed an approach for measuring the intensity of eye
blinking under various illumination situations (such as day or
night) for drowsiness detection system. Measuring the level of
Fig. 2. Driver Drowsiness System Architecture drowsiness by using eye blinking, it is an way of detection eye
positions and measuring eyelid movements.

2) Behavioral based evaluation: The driver’s performance A. FORD using Accident Avoidance and Driver Assist Tech-
like eye closure period, eye blinking, head rotation, Perclos nologies
etc. is constantly observed through a camera for detecting the
driver’s eyes. If any performance is detected then it shows that FORD using many new technologies for accident avoidance
the driver is drowsy. and driver assist system like in FORD handling and braking
capabilities are used. This technology has been used in de-
3) Physiological based evaluation: Drowsiness is detected
veloped nation. It helps the driver by controlling the vehicle
in physiological measures like continually examining the heart
or alerting the driver while driving and reduced the possible
rate, brain and pulse information. ECG (Electrocardiogram)
accidents. It can support everyday driving tasks by improving
and EEG (Electroencephalogram) can be used in this evalua-
comfort in car[16].
tion for detecting the drowsiness of the driver while driving.
Driver drowsiness system architecture shows in Fig 2. 1) Adaptive Crucial Control: Ford technologies offers
Adaptive crucial control(ACC), it helps the drivers to maintain
II. R ELATED W ORK a distance from a vehicle in front of them. It uses a radar
system which is mounted at the top of the vehicle that measures
Some latest work has been done in the field of development the distance and closing speed of the vehicle as shown in Fig
and technological tools for driver monitoring. Few of them 3. This system automatically preset the distance by which the
are using for the purpose of driver observation, the research speed of the car is adjusted. Adaptive crucial control which
has been used for avoiding driver fatigue and accidents which is radar based is available in FORD Taurus, Flex, Edge sand
may occur in future. They are using numbers of solutions Explorer and the Lincoln MKZ, MKS, MKX and MKT[16].
and technologies for reducing these types of road accidents
in details which are as follows. Image observations is one 2) Collision Warning with Brake Support: The Fusion
of the solution to detect driver drowsiness. Zhu et al.[7], technology offers Ford’s Collision warning with brake support,
proposed an idea for reducing the accident rate. This paper In this technology, which uses the radar for detecting the speed
used two cameras for detecting the image of driver’s face, pupil and the range as shown in Fig 4. It is just like adaptive crucial
movement, head rotation and gesture, facial expressions for control. When a vehicle detects a high risk collision then this
achieving driver drowsiness later this many research has been collision warning with brake support is used for activates a
done in this area. Later on Albu et al.[8], This paper describes visual and audible warning system to alert the driver while
a state in which sleeping while driving is the most serious driving. Faster brake performance is achieved by when the
consequence of fatigueness of driver and also described the brake system is pre-tensioned by servo boost, if required by
physiological state of drowsiness, check out the eye opening the driver. If the system getting suspended sensors become
and closing state. They are using some other measures to blocked by snow, mud or ice, then the driver will get a
identify the fatigue, they monitor the eyes condition to detect message to reduced the functionality. This technology is used
the drowsiness in driving. Lee et al.[9] described driving status by developed nation like North America on the Ford Taurus,
by using two cameras one is using to capture the drivers view Edge, Flex and Explorer and the Lincoln MKS, MKX, MKZ
and the other is used for tracking the path while driving for .

1294
 Fig. 7. Schematic of Sensing System Integration for Driver Drowsiness
Detection and Assistance [13]

Fig. 5. Lane Keeping Assist [17]

Fig. 8. Mouth Model [14]

control from their driving and hit the obstacles by which

an accidents occurs. Drowsiness mainly describes the level
of consciousness. Consciousness is decreased due to lack of
sleep or the driver is in tension. Sleeping disorders like sleep
Fig. 6. Pre-Collision System with Driver Monitor [17] apnoea(a sleeping condition that causes tiredness throughout
the day) is also the main factor in driver drowsiness. Drowsy
driver has a problem for keeping their eyes open.
B. Driver Monitoring System Using in TOYOTA
NHTSA estimates that in U.S.A approximately 37% of the A. ECG And EEG
total fatalities data are due to driver drowsiness[17], So in
developed nation many car companies developed new tech- There are two main physiological signals which are used
nologies in cars for avoiding road accidents based on sensors for detecting the driver drowsiness are ECG (electrocardio-
like in TOYOTO, FORD etc. . TOYOTA has a facility of Lane gram) and EEG (electroencephalogram). The deviation of heart
Keeping Assist technology and which is used to designed for rate changes considerably between the different segments of
alerting the driver, when the vehicle deviates from its original drowsiness, such as alertness, amount of drowsiness detection
trajectory/path. The system can also work in association with using Heart Rate variability (HRV), by which the low fre-
the Radar Cruise Control system to help the driver to control quency rate and high frequency rate varies in the range of
on the steer and keep the vehicle on the right track[17]. 0.04-0.15 Hz and 0.14-0.4 Hz [12], Fig 7, shows physiological
signal sensing system which is used in the vehicle to detect
1) Lane Keeping Assist: The Radar Cruise Control is the driver drowsiness and activate the alarm.
started and the system senses the vehicle conflicting from its
path, by applying counter-steering force, the system helps the
B. Yawning Based Detection
car keep on right track shown in Fig 5. These functions may
not activate depending on factors such as condition of the lane Yawning based detection is used to detect the driver’s
and the speed of the vehicle due to the road environment and drowsiness by detecting and tracking the face, mouth. There
driving conditions. are several steps described below
2) Pre-Collision System with Driver Monitor: Pre-collision 1) Degree of mouth openness: In this paper [18], mouth
system with driver monitor is using in TOYOTA for detecting openness degree of mouth ingenuousness is physically an-
the vehicles and obstacles by the help of camera, it detects the nonated which is totally objective and rotation is not con-
direction where the driver is facing and also check whether it is sidered. Scaling of mouth in different blocks and disparity
night time or day time by using the image processing vision. If between subjects is avoided,aspect ratio represents the degree
the system shows that the driver is not facing forward or if the of mouth openness Fig 8.
driver is sleep or facing in another direction then it activates
the alarm Fig 6. 2) Mouth region detection: Mouth region can be detected
by using the method based on the concentration or color
information. The block which contain the mouth, i.e. mouth
III. D ROWSINESS D ETECTION A ND I TS T ECHNIQUES
window. There are three estimation by which mouth region is
It is a process in which the accident diminishes due detected 1) mouth is in the bottom part of the face block; 2)
to deficiency of sleep and concentration causes the driver lip corners are at some distance from the face region;and 3)
drowsiness. When the driver fall into sleep quietly the fatigue lower lip are at some distance from the chin. The region which
occurs. If a driver is suffering from drowsiness they lost their is estimated is used to searching for mouth region detection,

1295
 Fig. 10. Algorithm for Eye-blink detection and classification system

Fig. 9. Local Binary Pattern [8]

which is used to decreased the searching space and is used for

avoiding the background pixel with similar color.
3) Mouth features detection: Mouth features detection
method is used to detect the mouth features, which is very
efficient and simple way for detecting facial expressions. Sev-
eral projection functions for eye detection such as IPF (Integral
Projection Function), VPF (Variance Projection Function) and
HPF (Hybrid Projection Function) are used for comparing
and a conclusion is drawn. This is very effective method for
detecting the eye features. For analyzing the mouth window,
lip corners are used by projection in vertical direction.

C. LBP(Local Binary Pattern)

Local Binary pattern is the technique by which it calculate
Fig. 11. System Architecture for Driver Alert System
the configuration of image by balancing each pixel with
another pixel in that region. Pleasure, sorrow, enthusiasm is
detected by this technique. Firstly the image is divided into measures of drowsiness. Optical detection is used to measure
four regions then the upper and lower part is detected first[8]. the rate and the duration of drowsiness [19].
In Fig 9, it shows that the working of LBP that it extracts the
image from the video and then the image is divided into four
IV. S YSTEM D ESIGN OF D RIVER A LERT S YSTEM
quadrant and each quadrants produced the histogram.
In this part, we initiate the design of driver alert structure.
D. Eye Blinking Based Detection We firstly initiate the system design of alert system Fig 11.

Eye blinking based detection is use to measure the driver A. System Design Overview
drowsiness. The blinking period of the eye and the closure
period is calculated for detecting the state of the driver. There The driver alert system is mainly focus on the recognition
are mainly two states are described one is normal state and of face and eye. It consists of four major sections: 1) Image
another is abnormal state. Normal state in which the closure acquisition; 2) pupil and face recognition; 3) driver state
period of the eye is for 2 sec and it shows the humble behavior recognition; and 4) activate alarm.
of the driver and abnormal state in which the closure period 1) Image Acquisition System: The purpose of this stage is
of the eye is more than 2 sec and drowsiness is detected by to obtain high quality video images of the driver’s face and
monitoring the eye constantly. If a driver is in sleepy state then eye. It is a major issue to acquire a high class image of eye
their eyes blinking rate are totally changed from the normal and face with adequate details. There are many constraints for
state of the driver. Fig 10 describes the blinking frequency ad implementing such a system. However such a system with the
the time period of closure [20]. This method uses a camera to desired uniqueness up to an extent can be implemented. The
occupy the video and calculate ratio of perceptiveness. In this images used in this study are taken from the record. The image
face movement, eye movement and eyelid measure is the ratio is converted from RGB to gray level for color processing.
of perceptiveness Fig 10 shows the eye blink detection.
2) Face and Eye Detection: In this part, the face and eye
E. Optical Detection detection method are the first level of drowsiness detection
technique. The Open CV library is used in this level for face
Optical Detection is useful for monitoring the driver’s pupil and eye detection. We initiate a very fast and memory efficient
by using optical sensor. Mainly optical sensors uses infrared or face detection algorithm based on Viola-Jones method. This
near-infrared LEDs. In optical detection some of the algorithm detection method proposed by Viola and Jones [21]. It is a
measured the blinking duration and rate for analysing the real time method which is very popular for face detection
drowsiness. Facial expressions, yawning detection are the and implemented in our driver drowsiness detection system

1296
 R EFERENCES
[1] Ji Q.,Zhu Z., and Lan P., ”Real-time Nonintrusive monitoring and pre-
diction of driver fatigue”, IEEE Transactions on Vehicular Technology:
, 2004, vol. 53, pp.1052-1068.
[2] Fasel B.,Luettin J., ”Automatic facial expression analysis:A Survey”,
Pattern Recognition: , 2003, vol. 36, pp.259-275.
[3] Luis M. Bergasa,Miguel A. Sotelo,Rafael Barea ”Real-Time System
for Monitoring Driver Vigilance”, IEEE Transactions on Intelligent
Transportation Systems , March 2006, vol. 7.
[4] Kari Torkkola,Noel Massey,Chip Wood ”Driver Inattention Detection
through Intelligent Analysis of Readily Available Sensors”, IEEE
Intelligent Transportation Systems Conference U.S.A, October 2004,
pp.326-331.
[5] Hankey,J. M.,Dingus,T. A ”The Development of Design Evaluation Tool
and Model of Attention Demand”, Virginia Tech Transportation Institute
Monk C.A(Science Applications International Corporation).
Fig. 12. Face Detection and Eye Detection
[6] Boer,E.R. ”Behavioral Entropy as an Index of Workload”, Proceeding
of the IEA/HFES,2000,pp.125-128.
[7] Arun Sahayadhas,Kenneth Sundaraj ”Detecting Driver Drowsiness
Based On Sensors A Review”, Malaysia 2012,pp.16937-16953,ISSN
1424-8220.
[8] Di Huang”Local Binary Patterns and Its Application to Facial Image
Analysis A Survey”, .http://www.scholarpedia.org/article/Local Binary
Patterns,2011,vol-41,pp.765-781.
[9] A. B. Albu,B. Widsten,T. Wang,J. Lan”A Computer Vision-Based System
for Real Time Detection of Sleep Onset in Fatigued Drivers”, IEEE
Intelligent Vehicles Symposium,June 2008,pp.25-30.
[10] J. Lee,J. Li,L. Liu”A Novel Driving Pattern Recognition
and Status Monitoring System”, In First Pacific Rim
Symposium,PSIVT,2006,pp.504-512.
[11] Ilkwon Park, Jung-Ho Ahn, Hyeran Byun ”Efficient Measurement of
Eye Blinking under Various Illumination Conditions for Drowsiness
Detection Systems”, Proceedings of the 18th International Conference
on Pattern Recognition 2006.
Fig. 13. Blink Detection [12] A.V. Desai,M. A. Haque”Vigilance Monitoring For Operator Safety:A
Simulation Study on Highway Driving”, In Journal of Safety Re-
search,2006,vol. 37,pp.139-147.
Fig 12. The Haar-like facial appearance can be proficiently [13] J. Krajewski,D. Sommer,U. Trutschel,”Steering Wheel Behavior Based
calculated by the demonstration of the image with integral Estimation of Fatigue”, In Proceeding of the 5th International Driv-
ing Symposium on Human Factors in Driver Assessment,Training and
image. The cascaded AdaBoost classifier ensures high face Vehicle Design,2006,pp.118-124.
and eye detection rate and low false alarm rate. The cascade [14] V. D. Lecce,M. Calabrese,”Experimental System to support Real Time
classifiers reduce the time for searching the face area by using Driving Pattern Recognition”, In Advanced Intelligent Comuting The-
Adaboost technique which is an optimal way. Several weak ories and Applications with Aspects of Artificial Intelligents,Annals of
classifier become a strong classifier by the help of boosting Emergency Medicine,2008,vol. 37,pp.1192-1199.
classifier using this cascade algorithm. [15] Dr. Xiong (Bill) Yu, P.E,”Non-Contact Driver Drowsiness Detection
System”, Safety IDEA,2012.s
3) Blink Detection: Blinking of an eye is detected when [16] http://www.daimler.com/dccom/0-5-1210218-1-1210332-1-0-0-
the change of black pixels in the eye area. An open eyeball 1210228-0-0-135-0-0-0-0-0-0-0-0.html.
will have bigger visible pupil than a blinking eye. The pupil [17] http://www.toyota-global.com
is often darker than the color of the rest of the eye. Thus, an [18] V. D. Lecce,M. Calabrese,”Yawning Detection For Determining Driver
open eye should have more black pixels. In the detected eye Drowsiness”, IEEE Int. Workshop VLSI Design and Video Tech.
region, the eye image is converted to a binary image by using Suzhou, China, May 28-30.
image thresholding. Then, the ratio of black pixels in the the [19] Shivangi R.Mishra,Prof. S. B. Somani,”EEG Signal Processing
and Classification of Sensorimoter rhythm-based BCI”,
binary image of the eye is calculated. The ratio of black pixels (IJERT),International Journal of Engineering Research and
is used as the criterion for eye blinks. Fig 13 shows an example Technology,Vol. 1,Issue 4, 2012.
of an open eye and a closed eye, and their binary images after [20] L. Li,M. Xie. H. Dong”A Method of Driving Fatigue Detection
thresholding. Based on Eye Location”, Proceedings of the IEEE 3rd ICCSN
,China,2011,pp.480-484.
V. C ONCLUSION [21] Paul. Viola,Michael. Jones”Rapid Object Detection using a Boosted
Cascade of Simple Features”, CONFERENCE ON COMPUTER VI-
Many modern technologies exist to detect the driver fa- SION AND PATTERN RECOGNITION ,2001,pp.1-9.
tigue. Although the products made by FORD, TOYOTA etc are
quite satisfactory in performance but they are not cost effective.
Since India has a majority of middle class population thus, this
system is not popular among the people. Much work needs to
be done in order to make it cost effective.

1297