© 2021 IJRAR July 2021, Volume 8, Issue 3 www.ijrar.

org (E-ISSN 2348-1269, P- ISSN 2349-5138)

Driver Drowsiness Detection System Using CNN

1
Sushmita Prasad, 2Nihal Shaikh, 3Trushita Jadhav, 4Robin Sen, 5Pushkar Sathe
1
Student, Dept of Electronics and Telecommunication, SIES Graduate School of Technology, Navi Mumbai, 2Student,
Dept of Electronics and Telecommunication, SIES Graduate School of Technology, Navi Mumbai, 3Student, Dept of
Electronics and Telecommunication, SIES Graduate School of Technology, Navi Mumbai, 4Student, Dept of Electronics
and Telecommunication, SIES Graduate School of Technology, Navi Mumbai, 5Professor, Dept of Electronics and
Telecommunication, SIES Graduate School of Technology, Nerul, Navi Mumbai

Abstract : A driver who is drowsy is certainly a lot more dangerous on the road than the one who is driving at higher speeds with a
conscious mind. Automotive scientists and manufacturers are trying to resolve this problem with the help of several technical solutions
that will remove such a crisis. In this project we have detected such drowsiness using neural network-based (CNN) methodologies. Our
previous work in this field involved using machine learning with multi-layer perceptron to detect the same. The CNN based model that
we are going to implement can be used to build a real-time driver drowsiness detection system for embedded systems and Android
devices with good accuracy and ease of use. Driver fatigue is one of the major causes of accidents in our country. Detecting the
drowsiness of driver is one of the ways of measuring driver fatigue. In this project we aim to build a driver drowsiness detection
system/alarm. This system works by monitoring the eyes of the driver and sounding an alarm when his/her eyes are closed for more
than a specific period of time. The system so designed is a real-time monitoring system. The priority is given to improving the safety of
the driver without being obtrusive. In this project the blinking of eyes of the driver is detected. If the driver’s eyes remain closed for
more than a certain period of time, the alarm will beep. For our project, we have used OpenCV using Keras and Tensorflow tools in
python (anaconda 3) for the detection of face and eyes. Collection of data sets of closed and open eyes of either a human being or an
animal like cats and dogs. A lot of high-quality data is required for the classifier to work properly. The utility application is used to build
a vectored output file. Using this file we shall repeat the training process. It extracts the positive samples from images before normalizing
and resizing them to specific and desired width and height of the entire image. In the training process we make the CNN learn the group
of classifiers one at a time. Each classifier in the group is a weak classifier. These weak classifiers usually consist of single variable
decision tree called stumps. In training the decision stump learns its classification decisions from its data and also learns the learned
weights for its vote from its accuracy on the data. Between training each classifier one by one, the data points are reweighted so that
more attention is paid to the data points where errors were made. This process will continue till the total error over the dataset arises due
to the combined weighted sum of the decision trees has fallen below the threshold level value.

Keywords: Accident, prevention, CNN, drowsiness, detect, secure

1. Introduction
A large number of people drive night and day on the highway. Drowsy driving is one of the reasons for majority of accidents. Significant
deaths and disability are caused due to motor vehicle collisions also major financial cost to security and individual because of driver
impairments. In India, there is no monitoring device to measure the drowsiness of driver. Certain systems like real time vision based on
driver monitoring system, driver fatigue monitor, user centre drowsiness driver detection and working system are implemented in abroad.
For prevention of such Majority of system focus either on physiological measures, changes in eye moment or driver performance
measure. Due to illumination variation the traditional system has certain defects. To prevent such type of accidents we have built a
system using Python, Open CV & Keras to alert the driver when he/she feels sleep. Significant deaths and disability are caused due to
motor vehicle collisions also major financial cost to security and individual because of driver impairments. In India, there is no
monitoring device to measure the drowsiness of driver. Certain systems like real time vision based on driver monitoring system, driver
fatigue monitor, user centre drowsiness driver detection and working system are implemented in abroad. For prevention of such Majority
of system focus either on physiological measures, changes in eye moment or driver performance measure. Due to illumination variation
in the traditional system have certain defects. The objective of our proposed system is to tackle the problem of the existing system that
is available.

2. Literature Survey
The main idea of this project [1] is to create a non-disruptive system that can detect driver fatigue and issue a timely warning because a
lot of road accidents happen due to the driver's drowsiness. This program will therefore help prevent many accidents, and as a result
save money and reduce human suffering. This system will monitor the driver's eyes using a camera and by developing algorithms we
can detect signs of driver fatigue in advance to avoid accidents. The project will therefore help to detect driver fatigue in advance and
will provide sound warning and seat vibration in the seat frequency range of between 100 and 300 Hz. In addition the warning will be
disabled by hand rather than automatically. For this purpose, a stop button will be used to disable the warning. Also if the driver feels
drowsy, there is a sudden acceleration or deceleration then we can judge this by plotting a graph on the time domain and where all three
input variables indicate that it is possible to get tired at one particular second or moment then a warning signal in the form of a text or
red circle will be given. Here, a stop button is used for deactivating the warning signal which increases the hardware requirements and
can also distract the driver further from the road.

IJRAR21C1273 International Journal of Research and Analytical Reviews (IJRAR) www.ijrar.org 118
© 2021 IJRAR July 2021, Volume 8, Issue 3 www.ijrar.org (E-ISSN 2348-1269, P- ISSN 2349-5138)
This work [2] detects the characteristic of EEG signal in drowsy driving. A method based on power spectrum analysis and FastICA
algorithm for determining fatigue levels was suggested. In the driving simulation system, the EEG signals of the driver were taken with
the NT-9200 tool in two states, one was awake and sober and the other was drowsy. Channel signals are analyzed with FastICA algorithm
for ocular electric, myoelectric and power frequency interface removal. Power spectral densities were calculated after FFT, and the F
fatigue index was finally obtained. The test results show that the method presented in this paper can be used to determine the drowsiness
of the EEG signal effectively. But, the cost of manufacturing is high and it cannot be deployed in every vehicle.

In this paper [3], an algorithm for driver drowsiness is introduced based on visual sigs that can be extracted from high frame rate video
analysis. A study of visuals that differ from the statistical database is proposed to assess their suitability for drowsiness by data mining.
Subsequently, an algorithm is developed that combines the most closely related blinking features (duration, eye closure percentage,
blinking frequency and amplitude-velocity ratio) using fuzzy logic. This algorithm was tested on a large database representing 60 hours
of driving from 20 different drivers. The main advantage of this algorithm is that it is independent of the driver and does not require any
modifications. In addition, it provides good results with more than 80% positive outcomes. Since CNN helps to perform predict ion,
recognition and classification tasks instead of pattern recognition, we will be implementing our project using CNN.

In this paper [4], a light weight, real time driver's drowsiness detection system is developed and implemented in the Android app. The
system records videos and detects the driver's face in all frames using image processing techniques. The system is able to detect facial
contours and then calculates the Eye Aspect Ratio (EAR) and the Eye Closed Ratio (ECR) to detect driver drowsiness depending on the
adaptive threshold. Mechanical learning algorithms were used to test the effectiveness of the proposed method. The technical results
show that the proposed model is able to achieve 84% accuracy using random forest classification. However, 3 different classification
models are used here which are Naïve bayes, SVM and random forest classifier. We have researched that CNN is a better classification
model and it can be used instead of above 3 classification models.

3. Problem Overview
3.1 Design Components

1. Hardware:
Webcam:
As we know Webcams is use to establish video links, permitting computers to act, other than this it also include security surveillance,
computer vision, video broadcasting. It provides advanced features such as image archiving, motion sensing, custom coding. It can be
used as computer vision.

Fig. 1. Webcam

Webcam specification
• Full HD 720p at 350fps streaming
• Lens: 2P2G
• COMS1/4 Colour chip
• Image range 70cm-90cm
• Recommended resolution: 1280 X 720
• Interface type: USB 2.0 compatible with USB 1.1

2. Software:
Python: Python is a general purpose programming language. Python is a programming language that aims for both new and experienced
programmers to be able to convert ideas into code easily. It is currently the most prevalent, mature, and well-supported among
programming languages in the area of machine learning that is why many developers use Python for CV. We are using Python for
facilitate data analysis and visualization. Because of readable and Maintainability of code Python is used. It will help us in doing Multiple
Programming and it has a Compatible with Major Platforms and Systems. It has a Robust Standard Library it has Many Open Source
Frameworks and Tools and Simplify Complex Software Development. Syntax in Python helps the programmers to do coding in fewer
steps as compared to Java or C++. They usually involve imperative and object-oriented functional programming. It has a dealings and
large standard library that has automatic memory management and dynamic features.
IJRAR21C1273 International Journal of Research and Analytical Reviews (IJRAR) www.ijrar.org 119
© 2021 IJRAR July 2021, Volume 8, Issue 3 www.ijrar.org (E-ISSN 2348-1269, P- ISSN 2349-5138)
OpenCV: OpenCV supports a wide variety of programming languages such as C++, Python, Java, etc., and is available on different
platforms including Windows, Linux, OS X, Android, and iOS. Computer Vision, on the other hand, allows computers to identify objects
through digital images or videos. CV implementation through Python will allow us to automate tasks that involve visualization, most
programming languages support Computer Vision but Python is considered the best in the competition. We use It for face and eyes
detection.

Tensorflow: Tensorflow is a Python-friendly open source library for numerical computation that makes machine learning faster and
easier. It is used as backend by Keras. Because of the low-level library it will provide more flexibility. Tensorflow provides more
network control thus allowing us to understand how operations are implemented across the network. It will provide an accessible and
readable syntax which is essential for making these programming resources easier to use. Complex syntax is need to know given machine
learning’s advanced nature. It will provide high-level operations are essential for carrying out complex parallel computations and for
building advanced neural network models. As it has a low-level library which will provide more flexibility so we can define our own
functionalities or services for our models and it is very important parameter because it allow us to change the model based on changing
user requirements. It also provides more network control and hence allowing us to understand how operations are implemented across
the network so we can keep track of new changes done over time.

Keras: Keras prioritized and is designed for human beings, not machines. Keras is easy to learn and easy to use. Keras is more productive
allowing us to try more ideas and that too faster. It is a neural networks library written in Python that is high-level in nature – which
makes it extremely simple and easily understandable to use. It works as a wrapper to low-level libraries like Tensorflow high-level
neural networks library, written in Python that works as a wrapper to Tensorflow.

Pygame: Pygame is a cross-platform set of Python modules designed for writing video games, which has computer graphics and sound
libraries designed that can be used with the Python programming language. It is used for playing the alarm sound when the eyes of the
driver are closed for more than the specified period. The pygame library is an open-source module for the Python programming language
used for playing the alarm sound when the eyes of the driver are closed for more than the specified period. Pygame can run across many
platforms and operating systems. Pygame is based on Python and Python is great for writing out ideas in one long block of code, so it
is easy and all we need to do is throw thoughts onto the page. Both Python and JavaScript use dynamically typed variables, meaning
that a variable becomes a string, an integer, or float when its assigned; however, making mistakes is easier in JavaScript. Just as typed
variables, both JavaScript and Python have global and local variable scopes. In Python, global variables inside of a function are identified
with the global keyword.

3.2 Methodology

Fig. 2 Flowchart
Explanation:
 As the car Starts the device gets activated.
 As shown in Flow Chart as soon as the Device Stats running It takes Image of Driver as an Input through Web Cam
 After Taking the Image it detects face in the image and create ROI
 After this it Detect eyes on the face image and feed it to CNN classifier
 Here the CNN will start its work and Classifier will detect whether the eyes are open or closed.
 The CNN will process and if eyes are closed for more than a specific period then Alarm is beeped.
 If eyes are open then Alarm won’t beep.

IJRAR21C1273 International Journal of Research and Analytical Reviews (IJRAR) www.ijrar.org 120
© 2021 IJRAR July 2021, Volume 8, Issue 3 www.ijrar.org (E-ISSN 2348-1269, P- ISSN 2349-5138)
Convolutional Neural Network (CNN):

Introduction:
It is the most influential innovations in the field of computer vision and image processing for Image Classification and recognition.
Successful in identifying faces, objects and traffic signs. They’re basically just neural networks that use Convolutional layers which are
based on the mathematical operation of convolution. It consists of a set of filters, which you can think of as just 2D matrices of numbers.

Working:
We can use an input image and a filter to produce an output image by convolving the filter with the input image. This consists of
Overlaying the filter on top of the image at some location. Performing element-wise multiplication between the values in the filter and
their corresponding values in the image. Summing up all the element-wise products. This sum is the output value for the destination
pixel in the output image. Repeating for all locations.

CNN layers:
Input layer: The input layer in CNN must contain image data. Image data is represented by a three-dimensional matrix. We need to
reshape it into one column.
Convolutional layer (with ReLU activation): Primary purpose of Convolution is to extract features from input image and ReLU is
rectified linear unit and it is an activation function.
Pooling layer: It reduce the number of parameters when input is too large. Pooling layers solve this problem. All they do is reduce the
size of the input it’s given by pooling values together in the input. The pooling is usually done by a simple operation like max, min, or
average. It is also called down sampling which reduces dimensionality of each map but retains important information.
Fully connected Layer with Softmax Layer: Softmax turns arbitrary real values into probabilities. What softmax really does is help
us quantify how sure we are of our prediction, which is useful when training and evaluating our CNN.
Output Layer: The output layer contains a label in the form of one-hot encoded.

Model: We have trained the model using 1000 photos; 500 photos of closed eyes and 500 photos of opened eyes. We have fetched the
data from the website: http://mrl.cs.vsb.cz/eyedataset and divided it into 2 parts namely open and close. We have used 15 layers for
training it. In which we have used 5 convolution layers, 5 pooling layers, 2 dropout layers, 2 dense layers and 1 flatten layer. The 5
convolution layers have 32, 64, 128, 128 and 64 neurons respectively. We have incooperated dropout rate of 0.25.
Output:

Fig. 3 Images of open eyes and closed eyes

4. Applications
The system can be used for the following purposes:

 To prevent road accidents by monitoring the driver’s drowsiness

 To monitor security guard/ watchman at night
 To monitor students during online lectures
 To monitor the sleep of baby (opposite logic will be used)
 To monitor patients in the hospital (opposite logic will be used)

5. Conclusion
We have successfully implemented the system using Tensor Flow, Open CV & Keras. The proposed system is most useful in detection
of drowsiness for drivers to prevent accidents and the loss due to such accidents. This project can further be developed and used for
other applications. The advancement of the project can be done by taking various other sensors into picture.

6. Future Work
Decrease in speed: According to the output of CNN, the speed of vehicle can be decreased. So if the driver has slept then the speed will
automatically get decreased for safety reasons.

Seat belt: According to the output of CNN, if the driver’s eyes are closed then the seat belt will vibrate to wake up him/her.

IJRAR21C1273 International Journal of Research and Analytical Reviews (IJRAR) www.ijrar.org 121
© 2021 IJRAR July 2021, Volume 8, Issue 3 www.ijrar.org (E-ISSN 2348-1269, P- ISSN 2349-5138)
References
1. Hardeep Singh, J. S. Bhatia, Jasbir Kaur, “Eye tracking based driver fatigue monitoring and warning system”, India International
Conference on Power Electronics 2010 (IICPE2010)
2. Ming-ai Li, Cheng Zhang, Jin-Fu Yang, “An EEG-based method for detecting drowsy driving state”, 2010 Seventh International
Conference on Fuzzy Systems and Knowledge Discovery
3. Antoine Picot, Sylvie Charbonnier, Alice Caplier, “Drowsiness detection based on visual signs: blinking analysis based on high frame
rate video”, IEEE Instrumentation and Measurement Technology Conference – June 2010
4. Sukrit Mehta, Sharad Dadhich, Sahil Gumber, Arpita Jadhav Bhatt, “Real-Time Driver Drowsiness Detection System Using Eye
Aspect Ratio and Eye Closure Ratio”, Proceedings of International Conference on Sustainable Computing in Science, Technology and
Management (SUSCOM), Amity University Rajasthan, Jaipur - India, February 26-28, 2019
5. W.-B. Horng, C.-Y. Chen, Y. Chang, and C.-H. Fan, “Driver fatigue detection based on eye tracking and dynamk, template matching”
In Proc. of the IEEE Int. Conf. on Networking, Sensing and Control, Taipei, Taiwan, 2004.
6. Saad Albawi, Tareq Abed Mohammed and Saad Al-Zawi, “Understanding of a convolutional neural network”, Published in 2017
International Conference on Engineering and Technology (ICET), August 2017

IJRAR21C1273 International Journal of Research and Analytical Reviews (IJRAR) www.ijrar.org 122