Foundation of IoT lab

Project: IoT based facial recognition door access control Home Security

System

Guided by: Dr. Anuj Kumar Goel

Prepared by Group 4 of CSE 19A

1. Arihant Jain (20BCS4924)

2. Barkha Rani (20BCS4945)
3. Kumar Utsav (20BCS4946)
4. Sahul Kr. Parida (20BCS4919)

1|Page
 ABSTRACT
Context: The home security system has become very important for each house. Previously,
most doors will be open by victimization ancient ways in which, like keys, security cards,
positive identification or pattern. However, incidents like a key loss has led to a lot of worrying
cases like theft and identity fraud. This has become a major issue. To beat this drawback, face
recognition victimization deep learning technique was introduced and Internet of Things (IoT)
additionally been accustomed perform economical door access system. Raspberry Pi could be
a programmable tiny pc board and used because the main controller for face recognition, youth
system and lockup system. The camera is employed to capture pictures of the person ahead of
the door. IoT system permits the user to manage the door access.

Objective: The main objective of the article is to develop an IoT-based face recognition Model
(which is enforced following the Deep Learning algorithm) for the protection system for
creating choices to lock or unlock the door system. the most aim of the thesis would be to
realize the sting computing construct that brings the unreal Intelligence to the low power web
of Things (IoT) devices with the assistance of containerization construct.

Conclusion: Our article includes the methodology for developing the pack AI model and
deploying the pack application on the Raspberry Pi (IoT device), that consists of the ARM
processor. we tend to conclude that the pack application run with high potency, is transportable
and transferable between multiple platforms, and therefore the pack application is compatible
with multiple processors (ARM, x86, amd64).

Keywords: Deep learning, biometric identification, Home security system, Internet of things
(IoT) and Raspberry pi.

2|Page
 ACKNOWLEDGEMENTS
We would like to thank Chandigarh University(CU) for giving us the opportunity to work on
such an amazing as well as interesting project from where we could also gain a lot of knowledge
and experience in the world of IoT.

We would also like to express our deepest and sincere gratitude to our supervisor Dr. Anuj
Kumar Goel for his valuable guidance, encouragement and continuous support throughout our
thesis. It would not have been possible to accomplish this work without his supervision and
support.

Finally, a huge thanks to our parents and friends for extreme love and support during
this thesis work.

“The Internet of Things is going to be a real game changer for loyalty, just like mobile has

been to date. — Brandon Logsdon, President/CEO, Excentus Corp

3|Page
Table of Contents

1: Introduction ................................................................................................................ 5 – 6

2: Literature Review........................................................................................................ 7 – 9
2.1 Face recognition technology................................................................................................ 7
2.2. Method used for face recognition ....................................................................................... 8
2.3. Deep learning ..................................................................................................................... 8
2.4. Face recognition in raspberry pi .................................................................................... 9
2.5. IoT in face recognition ....................................................................................................... 9

3. METHODOLOGY .................................................................................................... 10 - 13
3.1. Overview ......................................................................................................................... 10
3.2. Face recognition ........................................................................................................ 10 - 12
3.3. Deep learning ................................................................................................................... 13
3.4. Internet of things (IoT) ..................................................................................................... 13

4. RESULT AND ANALYSIS ....................................................................................... 14 - 16

5. CONCLUSION ................................................................................................................. 16

6. REFERENCES .................................................................................................................. 17

4|Page
1. INTRODUCTION
Nowadays, home security system may be a crucial issue. Indeed, this method is to make sure
properties and loves one’s square measure invariably safe and guarded. For past few years, it's
become vital to own a solid security system for home, which might secure within the most ideal
and safe means. several countries square measure step by step deployed home security system.
The vital a part of any home security system is that the person identification to enter and exit
the house. Previously, folks use the normal technique for his or her home security system. the
normal security system depends on the utilization of external things like key, arcanum and ID
card to achieve access. However, because of some limitation, biometric takes place to deliver
such a promising security system. The biometric may be a distinctive and quantitative
parameter for individual recognition. Biometric system needed the used of specialised
hardware like fingerprint scanner, palm print scanner, deoxyribonucleic acid instrument and
etc. moreover, this specific machine needed the target to the touch the hardware to amass
knowledge of human distinctive options. Biometric technology is viewed as a standout among
the foremost secure verification system accessible, by giving an additional elevated quantity of
security than typical technique. Face recognition is that the most illustrious technique in
biometric technology besides fingerprint characteristics. this can be because of additional
stability as face contains additional options. Besides, it's thought of extremely secure as face
can't be taken, borrowed or forge so as to enter the house. Face recognition is probably going
the foremost natural approach to perform biometric verification between people. Face detection
is that the opening of the face recognition system.

“The internet is no longer a web that we connect to. Instead, it’s a computerized, networked,

and interconnected world that we live in. This is the future, and what we’re calling the

Internet of Things.” – Bruce Schneier, American cryptographer, computer security

professional

5|Page
“As the Internet of things advances, the very notion of a clear dividing line between reality

and virtual reality becomes blurred, sometimes in creative ways.” – Geoff Mulgan, Professor

of Collective Intelligence, Public Policy and Social Innovation at University College

London.

Face photos may be caught at a distance with the utilization of an internet camera. The
individual may be recognized while not physical contact on any special hardware to understand
the person's identity. Face recognition mistreatment deep learning technique is employed. Deep
learning may be a piece of the additional intensive cluster of machine learning strategies
supported learning knowledge representations, as critical taskspecific algorithms. Learning
may be managed, semi-directed or unattended. With the deep learning, the system is improved
from time to time. Some pictures of authorizing user square measure used because the info of
system and therefore the system can train the face recognition mechanically. Thus, the accuracy
is enhanced. Home security is AN example of an online of Things (IoT) applications. IoT refers
to the network of associated physical objects which will act and trade info among themselves
while not the necessity of any human intervening.

IoT may be an artistic movement technology wherever devices and net is interconnected. it's
totally different from the web because of net exceed property by permitting any embedded
circuit to speak with one another mistreatment this internet infrastructure. Little question IoT
helps users to regulate one or additional devices and capabilities to change with several daily
chores. By mistreatment IoT, it will facilitate in dominant the door access and conjointly sent
notification throughout the net. during this system, Blynk apps square measure used. Blynk
apps is AN app that allows United States to regulate the door access by planning the graphical
interface within the apps in step with the precise operate to perform. It conjointly able to send
notification to pc, good phone and different smart devices.

6|Page
2. LITERATURE REVIEW

2.1. Face recognition technology

Currently, the quantity of thefts and identity fraud have oftentimes been reported and has
become important problems. ancient ways that for private identification needs external part,
like key, security word, RFID card, and ID card to possess access into a non-public plus or
getting into public house. Several processes like drawing out cash from banks needs word.
different such parking in camera house would conjointly would like ticket. for a few homes,
the house secret's vital. However, all this technique conjointly has many disadvantages like
losing key and forgetting word. once this happens, it is trouble to recover back. This technique
is slowly taken by biometric strategies because it is that the potential thanks to solve those
issues. this system needed to use the special hardware like fingerprint scanner, palm print
scanner, deoxyribonucleic acid analyser to assemble info for the overwhelming majority of the
biometric applications and also the target objects got to bit with the hardware to accumulate
info. As biometric could be a technique that distinctive physical highlights of individuals
consequently it's an intensive kind of utilization in security frameworks and it's viewed together
of the foremost secure strategies. Basically, biometry is classified in 2 classes that ar physical
and behavioural. Recently, the face recognition technology has engaged an awesome variety
of researchers and its step by step displacement different biometric security frameworks. Face
recognition is additionally referred to as image matching. it's an apace growing field wherever
it's heading in an exceedingly direction specified it'll replace the normal technique. Face
recognition is a lot of stable among others identity verification technique because it is
victimisation the face that leads to high accuracy, lowest false recognition rate and it doesn't
modification in people’s life. Thus, this technique is way sensible for heaps of usage, together
with face recognition for the unlocking house door.

7|Page
2.2. Method used for face recognition
In this new era, face recognition plays a very important role in security and observation.
Consequently, there's a demand for an expert and cost-efficient system. Face recognition may
be a technique that's ready to determine and verify peoples. in line with face recognition, outline
as steps to spot, distinguish and processed face is compared with the pictures that hold on within
the info to verify World Health Organization the person is. This face recognition has become
a major technique for user identification. There square measure several techniques that may be
used for face recognition however the Principle Part Analysis (PCA) is one in all the foremost
fashionable techniques used for face recognition. This technique involves a mathematical
procedure to remodel variety of presumably correlate variables into variety of unrelated
variables referred to as principle part. Generally, the PCA technique for face recognition can
utilize the employment of Eigenfaces. it's the effective and economical ways that to represent
footage into Eigenfaces part because it will cut back the scale of the info of the take a look at
image. varied technique is developed and deployed so as to boost the performance of face
recognition technology.

2.3. Deep learning

Deep learning has benefited the human kind for years currently. within the trendy society, a
deep learning technique, particularly convolutional neural network utilized in several
applications like registration code recognition, finger-vein identification, gender recognition,
face recognition, feeling recognition, and different applications. supported deep learning
technique is extremely used for pc vision application. By victimization Convolutional Neural
Network (CNN), it ends up in higher performance for face detection and face recognition.
There square measure several blessings by victimization CNN because it will understand
patterns with high variability and strength to distortions and straightforward geometric
transformations like translation, scaling, rotation, squeezing, stroke breadth and a noise.
Besides, homeward-bound quick and revolved BRIEF(ORB) additionally collectively of the
techniques used for face recognition. it's used for feature extraction, that utilizes a quick binary
descriptor hooked in to temporary and is rotationally invariant. Typically, Deep Learning relies
on supervised learning. The aim of Deep Learning is to form a machine capable to properly
classify pictures. Thus, throughout the supervised learning Raspberry pi square measure shown
a picture to supply associate degree output within the style of vectors scores, with every class
one.

8|Page
2.4. Face recognition in raspberry pi

The first analysis on face recognition goes means back in 1950 within the field of science. The
particular work of automatic machine recognition of faces very started in 1970. From all the
analysis done, there 2 varieties of face recognition methodology that area unit the image-based
face recognition and video-based recognition. Video-based face recognition is that the method
of finding 3D pictures from its 2nd whereas the image primarily based recognition
methodology, is that the method by that human train the machine employing a camera by
showing the camera sets of still pictures. A Face Recognition System could be a framework
that consequently acknowledges and to boot, checks the identity of an individual from digital
pictures or a video define from a video supply. several researchers favour to use embedded
device referred to as Raspberry Pi for coaching and identification purpose. the basic reasons
why they need picked this specific part as a result of its high handling limit, low cost, and its
capability adjusts in varied programming modes. By exploitation Raspberry Pi, it helps to
resolve the limitation of laptop like its weight, size and high power consumption. Raspberry Pi
could be a device which will divide the software system half into 3 elements that area unit
recording pictures, coaching and face recognition. consistent with deep learning and residential
security system as they deployed the used of Raspberry Pi for image capturing system, the
system becomes littler, lighter and has lower power utilization. thus it's additional convenient
compared to PC-based face recognition system.

2.5. IoT in face recognition

IoT has been applied in face recognition in many applications such as unmanned aerial vehicle,
smart classroom, home security system, smart house, smart surveillance and many more
applications. The previous implementation of IoT in face recognition are using conventional
method such local binary pattern, neural network, support vector machine, and k nearest
neighbor. However, in this project deep learning was being used.

9|Page
3. METHODOLOGY

3.1. Overview
This project can style face recognition for period of time use. it's integrated with IoT to perform
sensible home security system. A deep learning technique is employed during this project. So
as to confirm the expected result ar obtained, many major steps got to be conducted like
information collections, implementing, testing, and troubleshooting. These steps ar accustomed
analyze the information and output. With these steps, this project is in a position to be
evaluated.

3.2. Face recognition

The example is constructed by combining a part of face recognition and IoT along. Face
recognition is operated initially place. There square measure 5 steps in face recognition, that
square measure collection pictures, making info, pre-processing pictures, coaching pictures and
testing pictures. Firstly, pictures square measure collected. These pictures square measure
obtained by capturing exploitation camera and used the prevailing pictures. This image is
employed for coaching purpose for the system to be a lot of correct once handling new pictures.
a complete of 5 persons, every with 5 footages is taken from totally different positions. Every
image is roughly 268 x 350 pixels of height and breadth. pictures that square measure collected
square measure hold on within the info as shown in Figure one (a).

10 | P a g e
 Figure 1 (a). Numbered and labelled images

Figure 1 (b). Authorised Facial Database

11 | P a g e
Since face recognition framework a need large number of images, existing images have been
augmented. This is done by using an algorithm. Each picture per person will augment into 100
pictures, resulting 2500 images stored in database. The images vary in brightness, colour,
intensity, and angle. This is to ensure that face recognition system can detect even in different
conditions. Figure 2 (a) shows the transformation from the original image in the database into
the processed photo. The result of each person categorized into each folder. Next, the cropping
process takes place. This process will crop the exact face from the images. This process is
carried out by using an algorithm. The pixel of each picture is reduced to 48x 48 pixels of
height and width. Figure 2(b) shows extract features by separating the face from the
background.

(a) (b)

Figure 2. (a) Data augmentation of Images (b) Exact Face Obtained

12 | P a g e
3.3. Deep learning
Existing design was utilized in the coaching method. pictures were train victimization deep
learning methodology victimization Convolutional Neural Network (CNN) technique. the
present design used is AlexNet that contains eight layers. This design builds with many layer
and activation perform like Convolution, Maxpooling, Flatten, Dense, Activation and Dropout.
the complete neural network approach was enforced in Python language and Keras library. The
coaching involves a hundred epochs initially and perennial with twenty epochs once the testing
part. Figure three can illustrate the coaching method of the dataset.
After coaching method was done image testing is needed to see the accuracy achieved by the
system. during this stage, image that don't seem to be within the information ar used as take a
look at pictures. There ar 10 pictures tested for every tagged that ar recognized and
unrecognized person. every image tested can tagged the image with name or as an unknown.

Figure 3. Training the dataset

3.4. Internet of things (IoT)

Blynk could be a noted app since it's been downloaded over a hundred thousand users. Blynk
could be a platform for iOS and automaton apps that managed to manage Raspberry pi and
plenty of different microcontrollers. it's a digital dashboard that designed for the user to make
their own graphic interface for the project. it's straightforward and straightforward to use
because the user will merely drag and drop the widgets that they have in step with their project
sort. This app is employed in IoT half. Blynk begin on-line because the Raspberry Pi connected
to the net over Wi-Fi. Besides, it's conjointly can get on-line by link to the net through the LAN
or the new ESP8266 chip. For condition wherever face can't be recognized, that person will
press the button and notification area unit sent to smartphone of house owner. Hence, live
streaming video can seem to spot the person attempting to unlock the door.

13 | P a g e
4. RESULT AND ANALYSIS
Face recognition is tested on 2 varieties that square measure by testing image and period of
time to work out the system accuracy. For testing image, there square measure 10 pictures that
don't seem to be within the info square measure tested for every label that square measure
licensed and unknown person. The tested image can have tagged the image with names for
licensed person whereas unknown for unauthorized person. Figure four (a), (b), (c) and (d)
shows the tested image with positive and negative results for licensed and unknown. period of
time face recognition is performed victimisation internet camera. a certified person are often
recognized through the system and the other way around.
The name of the user is going to be shown below their face as shown in Figure five (a) whereas
unauthorized person is shown in Figure five (b).

(a) (b)

(c) (d)

Figure 4. (a) Positive result for authorized (b) Positive result for unknown
(c) Negative result for authorized (d) Negative result for unknown

14 | P a g e
 (a) (b)

Figure 5. (a) Authorized person labelled with name (b) unauthorized

Figure 6. (a) Notification send through Blynk (b) Video streaming in Blynk

Figure 7. An authorized person labelled with the name

15 | P a g e
Face recognition and IoT are integrate and build in prototype. When person face can be
recognized by the system, the door will open automatically as shown in Figure 4.9. If face
cannot recognize by the system, door will remain closed as illustrated in Figure 7 (a) and (b).
Door access can also be controlled through IoT using Blynk app.

Figure 7. (a) Door is unlocking (b) Door is locked

5. CONCLUSION
As a conclusion, security system by using face recognition combined with IoT is successfully
done. The face recognition is able to recognize the face and able to send notification to a user
when an unknown being has been detected through IoT. On the other hand, this project is this
project still has a big room of improvement to be done, especially in the efficiency of the image
processing part. Due to the module used which is Raspberry Pi 3, the processing time of the
coding took a long time so process the image taken and take action. By using another better
module, this project can be improved greatly.

If the person’s face matches, then the python code will pass the output to the Arduino using
serial library and the person will get access of the motorised door.

“If you think that the internet has changed your life, think again. The IoT is
about to change it all over again!” — Brendan O’Brien, Chief Architect & Co-Founder, Aria
Systems.

16 | P a g e
6. REFERENCES

Image-Based Face Detection System-https://www.intechopen.com/books/face-recognition-

semisupervised-classification-subspace-projection-and-evaluation-methods/face-recognition-
issues-methods-and-alternative-applications

Keras: The Python Deep Learning library-https://machinelearningmastery.com/tutorial-first-

neural-network-python-keras/

Home security system based on face recognition-

https://ieeexplore.ieee.org/document/8697695

A brief survey of machine learning methods and their sensor and IoT applications-
https://ieeexplore.ieee.org/document/8316459

What Is Deep Learning AI? A Simple Guide with 8 Practical

Examples.”- https://www.forbes.com/sites/bernardmarr/2018/10/01/what-is-deep-learning-
aia-simple-guide-with-8-practical-examples/

Internet of Things (IoT) for Automated and Smart Applications -

https://www.intechopen.com/books/internet-of-things-iot-forautomated-
and-smart-applications

Face recognition techniques, their advantages, disadvantages and performance

evaluation - https://ieeexplore.ieee.org/abstract/document/7374154

17 | P a g e