ABES ENGINEERING COLLEGE, GHAZIABAD

AFFILIATED TO
DR. A.P.J. ABDUL KALAM TECHNICAL UNIVERSITY, U.P., LUCKNOW
(Formerly UPTU)

Project Report
on
Driver Fatigue Detection System
Submitted as partial fulfillment for the award of

BACHELOR OF TECHNOLOGY
DEGREE
Session 2021-22
in

Computer Science and Engineering

By
Tanisha Baghel 1900320100175
Tanisha Garg 1900320100176
Tanya Singhal 1900320100178

Under the guidance of

Babli Baliyan
 STUDENT’S DECLARATION
I / We hereby declare that the work being presented in this report entitled
“DRIVER FATIGUE DETECTION SYSTEM” is an authentic record of my /

our own work carried out under the supervision of Dr. /Mr. /Ms. “BABLI

BALIYAN”
The matter embodied in this report has not been submitted by me / us for
the award of any other degree.

Dated: 20/12/2022 Signature of students(s)

(Name(s) Tanisha Baghel
Tanisha Garg
Tanya Singhal)
Department:
Computer Science and
Engineering

This is to certify that the above statement made by the candidates is correct to the
best of my knowledge.
 Signature of Supervisor
(Name )
(Designation)
(Computer Science & Engineering
Department)

CERTIFICATE

This is to certify that Project Report entitled “Driver Fatigue Detection System”
which is submitted by Tanisha Baghel, Tanisha Garg, Tanya Singhal in partial
fulfillment of the requirement for the award of degree B. Tech. in Department of
Computer Science and Engineering of Dr. A.P.J. Abdul Kalam Technical
University, formerly Uttar Pradesh Technical University is a record of the
candidate own work carried out by them under my supervision. The matter
embodied in this thesis is original and has not been submitted for the award of any
other degree.
Supervisor Babli Baliyan

Prof.(Dr.) Divya Mishra

Date 20/12/2022

HOD CSE

ACKNOWLEDGEMENT

It gives us a great sense of pleasure to present the report of the B. Tech Project
undertaken during B. Tech. Final Year. We owe special debt of gratitude to Professor
Babli Baliyan, Department of Computer Science & Engineering, ABESEC Ghaziabad for
her constant support and guidance throughout the course of our work. Her sincerity,
thoroughness and perseverance have been a constant source of inspiration for us. It is
only his cognizant efforts that our endeavors have seen light of the day.

We also take the opportunity to acknowledge the contribution of Professor (Dr.) Divya
Mishra, Head, Department of Computer Science & Engineering, ABESEC Ghaziabadfor
his full support and assistance during the development of the project.

We also do not like to miss the opportunity to acknowledge the contribution of all faculty
members of the department for their kind assistance and cooperation during the
development of our project. Last but not the least, we acknowledge our friends for their
contribution in the completion of the project.

Signature:

Name :Tanya Singhal

Roll No.: 1900320100178

Date : 20/12/2022

Signature:

Name : Tanisha Baghel

Roll No.: 1900320100175

Date : 20/12/2022

Signature:

Name : Tanisha Garg

Roll No.: 1900320100176

Date : 20/12/2022
 ABSTRACT
This paper is a review report on the work done in the field of computer engineering to
create a system for detecting driver drowsiness in order to avoid accidents caused by
fatigued or sleepy drivers. The study offered findings and recommendations for the
project's limited application of the various methodologies. While the project's
implementation provides a practical understanding of how the system functions and the
adjustments that may be made to increase the system's overall utility.

The report also summarises the authors' views in order to support further optimization in
the concerned area and provide the utility with greater effectiveness for a safer road.
 TABLE OF CONTENTS Page

DECLARATION ................................................................................................... ii
CERTIFICATE……....................................................................................................iii
ACKNOWLEDGEMENTS .................................................................................. iv
ABSTRACT ........................................................................................................... v
CHAPTER 1 Introduction
……………………………………………………………………..5
CHAPTER 2 Literature Survey ……………………………………………………..............
6
CHAPTER 3 System Design …………………………………………………………………
3
CHAPTER 4 Implementation and Results ..................................................................40
CHAPTER 5 Conclusion and Future
Scope………………………………………………….
APPENDIX A ......................................................................................................... 45

REFERENCES... ................................................................................................... 49

7
8
 CHAPTER 1
INTRODUCTION
A person who is drowsy has a strong desire to sleep and is in a condition of approaching
sleep. It can refer to both the routine state just before falling asleep and the chronic condition
that refers to remaining in that state without regard to a regular rhythm [16]. When performing
activities that demand continual attention, such as operating a vehicle, sleepiness might be
harmful. A person will feel drowsy while driving if they are sufficiently exhausted, which
increases the risk of a traffic collision.In order for readers to trust the writer, the introduction
must be well written with few errors. In order to keep readers reading, the writer needs to
catch the attention of the reader and write in an interesting way. The unique language
enhancement feature may suggest words to strengthen the writing. Strong writing may hold
readers' attention.

Thousands of tragedies are linked to this magnificent technology every year because we
neglected our responsibilities to promote safer travel. Although it may seem little to most
people, adhering to traffic laws and regulations is crucial. The most powerful object on the
road is an automobile, which may be devastating in the hands of those who lack responsibility
and, occasionally, even endanger the lives of other road users. Not acknowledging when we
are too sleepy to drive is one example of carelessness. Many scholars have submitted
research papers on systems that can detect driver drowsiness in order to keep an eye on the
situation and avoid a catastrophic outcome from such negligence. However, occasionally,
some of the system's observations and points

A significant difficulty in the development of accident avoidance systems is the detection or

prevention of driver drowsiness. Methods for reducing the effects of drowsiness must be
developed due to the danger it poses on the road.

The development of a sleepiness detection system simulation is the goal of this project. The
emphasis will be on creating a system that can precisely track whether the driver's eyes and
mouth are open or closed. It is thought that by keeping a watch on the eyes, a driver's
drowsiness can be identified early enough to prevent an accident. A technique to determine a
driver's level of weariness is yawning detection. When someone is tired, they continue to
yawn to make sure there is enough oxygen for the brain to consume before entering a drowsy
condition. A series of facial picture sequences, along with the timing of an individual's eyes
and mouth opening and closing, are used to detect exhaustion and drowsiness. PERCLOS is
another technique for detecting ocular closure. This detection technique is based on the
proportion of time that the eyes are closed within a given period of time.

Application areas for the analysis of face photos include face recognition, human
identification, and tracking for security systems. This study focuses on the localization of the
eyes and mouth, which entails examining the full image of the face to pinpoint the eyes' and

9
mouths' locations using the techniques now utilised in image-processing algorithms. The
technology is made to detect tiredness and drowsiness after locating the eyes' location and
determining whether they are open or closed.

10
 CHAPTER 2

LITERATURE SURVEY
SYSTEM REVIEW
This survey aims to understand the needs and expectations of the general public, and in
order to achieve that, we searched through several websites and applications for the
necessary information. Based on these facts, we created an audit that gave us fresh
ideas and allowed us to modify our plans for the assignment. We came to the conclusion
that such an application is necessary and that there has been some good advancement in
this area as well.

TECHNOLOGY USED
PYTHON : Python is a general-purpose, high-level, interpreted programming language.
Python's design philosophy places a strong emphasis on code readability and makes
considerable use of whitespace. Its language constructs and object-oriented methodology
are designed to aid programmers in creating clean, comprehensible code for both little
and big projects. Procedural, object-oriented, and functional programming are just a few
of the programming paradigms that Python supports in addition to having dynamic typing.

JUPYTER LAB : A nonprofit organisation called Project Jupyter was established with the
goal of creating open-source software, open standards, and services for interactive
computing across numerous programming languages.

IMAGE PROCESSING: Digital image processing is the application of computer algorithms

to the image processing of digital images in computer science.

MACHINE LEARNING: Using patterns and inference rather than explicit instructions,
machine learning is the scientific study of the algorithms and statistical models that
computer systems employ to carry out a particular task efficiently. It's considered to be a
part of artificial intelligence. In order to generate predictions or judgments without being
explicitly instructed, machine learning algorithms construct a mathematical model from
sample data known as "training data."
11
2.1.1 Interfaces

2.1.3 Hardware Interfaces

I. Laptop with basic hardware.

II. Webcam

2.1.4 Software Interfaces

Python
 Python3

Libraries

 Numpy
 Scipy
 Playsound
 Dlib
 Imutils
 openCV, etc.

Operating System

 Windows or Ubuntu

2.4 Constraints

A system must satisfy the following requirements in order to identify sleepiness:

• A minimally intrusive monitoring system that won't distract the driver

• A reliable system for real-time monitoring that can detect tiredness.

• A system that works both in the day and the dark.

12
 CHAPTER3

SYSTEM DESIGN

13
3.1. Use Case diagram

3.2. Activity diagram

14
3.3. Class diagram

15
CHAPTER 4

16
 IMPLEMENTATION AND RESULTS

4.1 Implementation

One of the best methods for real-time drowsiness detection is deep learning
driver drowsiness detection. In-depth understanding of how to recognise
tiredness in drivers is one of the most effective methods for doing so. This is a
consideration in many of the suggested techniques. Deep learning is used to
develop an OS-independent true-time driver inspection engine. To cover the
most delicate parts of sleeping, we add a face monitoring system that is driven
by a face expression descriptor. For image detection, feature extraction,
picture classification, and alerting, a convolutional neural network is employed.
The CNN recognises facial alignments from a range of faces and establishes
their temporal correlations using the data at hand. In this work, a reliable
technique for identifying intoxicated driving is described. In order to determine
eye-pupil alignment and head posture alignment, information from the facial
region is first extracted.

As in the image above, a device or cell phone with a driver drowsiness

detection app will be placed in front of the driver, on the wheel, and it will
continuously scan the driver's face and watch the driver's eyes; if the eyes start
to draw close, it will sound the alarm.

17
  In our programme, we used Dlib, a pre-trained programme developed to
identify human faces using the pre-defined 68 landmarks. Dlib was
trained using the HELEN dataset.
 The landmarked image is formed as follows;

 We are able to recognise the left eye and right eye aspects of the face
after sending our video stream to the dlib frame by frame.
 We then used OpenCV to draw contours around it.
 We determined the sum of both eyes' aspect ratio, which is equal to the
sum of the two different vertical distances between the eyelids divided
by the horizontal distance, using Scipy's Euclidean function.
 The aspect ratio number is now checked to see if it is less than 0.25
(0.25 was selected as the base case after several tests). An alarm
sounds and the user is warned if it is less.

4.2 Results

18
19
 CHAPTER 5

CONCLUSION

It perfectly satisfies all of the system's goals and needs. The framework has reached an
unchanging state in which all bugs have been eliminated. Clients who are aware of the
framework, understand its goals, and are aware that it addresses the concern of calming
down for those with fatigue-related concerns by informing them about their level of
drowsiness while driving.

Future Scope
 The model can be further enhanced by adding extra variables like blink frequency,
yawning, automobile condition, etc. If each of these criteria is used, it can significantly
increase accuracy.
 We intend to advance the project by including a sensor to monitor the heart rate to
avoid mishaps brought on by an unexpected heart assaults against drivers
 The same paradigm and methodology can be applied to many additional purposes,
including When a user is dozing off, Netflix and other streaming services can tell. then
the video will end appropriately. It may also be applied to situations like inhibits user
from falling asleep

20
 Appendix
Project Code
import cv2
# Numpy for array related functions
import numpy as np
# Dlib for deep learning based Modules and face landmark detection
import dlib
#face_utils for basic operations of conversion
from imutils import face_utils

#Initializing the camera and taking the instance

cap = cv2.VideoCapture(0)

#Initializing the face detector and landmark detector

detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor("shape_predictor_68_face_landmarks.dat")

#status marking for current state

sleep = 0
drowsy = 0
active = 0
status=""
color=(0,0,0)

def compute(ptA,ptB):
dist = np.linalg.norm(ptA - ptB)
return dist

def blinked(a,b,c,d,e,f):
up = compute(b,d) + compute(c,e)
down = compute(a,f)
ratio = up/(2.0*down)

#Checking if it is blinked
if(ratio>0.25):
return 2
elif(ratio>0.21 and ratio<=0.25):
return 1
else:
return 0

while True:
_, frame = cap.read()
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

21
faces = detector(gray)
#detected face in faces array
for face in faces:
x1 = face.left()
y1 = face.top()
x2 = face.right()
y2 = face.bottom()

face_frame = frame.copy()
cv2.rectangle(face_frame, (x1, y1), (x2, y2), (0, 255, 0), 2)

landmarks = predictor(gray, face)

landmarks = face_utils.shape_to_np(landmarks)

#The numbers are actually the landmarks which will show eye
left_blink = blinked(landmarks[36],landmarks[37],
landmarks[38], landmarks[41], landmarks[40], landmarks[39])
right_blink = blinked(landmarks[42],landmarks[43],
landmarks[44], landmarks[47], landmarks[46], landmarks[45])

#Now judge what to do for the eye blinks

if(left_blink==0 or right_blink==0):
sleep+=1
drowsy=0
active=0
if(sleep>6):
status="SLEEPING !!!"
color = (255,0,0)

elif(left_blink==1 or right_blink==1):
sleep=0
active=0
drowsy+=1
if(drowsy>6):
status="Drowsy !"
color = (0,0,255)

else:
drowsy=0
sleep=0
active+=1
if(active>6):
status="Active :)"
color = (0,255,0)

cv2.putText(frame, status, (100,100), cv2.FONT_HERSHEY_SIMPLEX, 1.2, color,3)

for n in range(0, 68):

(x,y) = landmarks[n]

22
 cv2.circle(face_frame, (x, y), 1, (255, 255, 255), -1)

cv2.imshow("Frame", frame)
# cv2.imshow("Result of detector", face_frame)
key = cv2.waitKey(1)
if key == 27:
break

23
 References
[11.1] Z. Zhang and J. Zhang, "A new real-time eye tracking for monitoring driver fatigue based on
nonlinear unscented Kalman filter," Journal of Contr. Theory. Applications, vol. 8, no.

[11.2] 2010, pp. 181–188 Eye detection for a real-time vehicle driver fatigue monitoring system,
R. C. Coetzer and G. P. Hancke, 2011, IEEE Intelligent Vehicles Symposium (IV), Baden-Baden, pp.
66-71

[11.3] Amna Rahman, Department of Software Engineering, Fatima Jinnah Women University
(2015) National Software Engineering Conference (NSEC 2015)

[11.4] Y.C. Tsai, P.W. Lai, P.W. Huang, T.M. Lin, and B.F. Wu, 2020. Real-Time Drowsiness
Detection Using Eye Blink Monitoring Based on vision, an immediate method for determining how
tired a driver is. IEEE Access, vol. 8(6), pp. 6734–6735

[11.5] Happy, S.L., George, Routray, and Dasgupta (2013) a device that monitors drivers' attention
spans while they are driving by using vision. pp. IEEE Transactions on Intelligent Transportation
Systems, 14(4), pp. 1825–1838.

[11.6] "Driver Drowsiness Detection Algorithms Using Electrocardiogram Data Analysis," Mohsen
Babaeian and Mohammed Mozumdar, 9th Annual IEEE Computing and Communication
Workshop and Conference (CCWC), 2019, pages 0001-0006, 2019.

[11.7] The Melbourne, Victoria, Australia, National Road Transportation Commission, Technology
Rep., 2000. G. Krueger, L. Hartley, T. Horberry, and N. Mabbott Examination of fatigue prediction
and detection systems."

[11.8] Lee B. G., Chung W. Y.: a data fusion-based smartphone-based driver safety monitoring
system. Sensors. 12(12):17536-52 (2012)

[11.9] G. Li, W.Y. Chung: Utilizing a support vector machine classifier and wavelet analysis of heart
rate variability, driver fatigue can be identified. Sensors. 13(12):16494-511 (2013) [11.10] Garcia I,
S. Bronte, L. M. Bergasa, J. Almazán, and J. Yebes: Detector of fatigue based on vision for real-
world driving conditions. IEEE's Intelligent Vehicles Symposium IV pp. 618-623). IEEE (2012).

24
25