A

Project Report
On

Classifying Hand-written Digits Using Neural Network

This project work is submitted in partial fulfillment of the requirement for the
award of Degree of Bachelor of Information Technology under the Faculty of
Science & Technology, RTM Nagpur University (Session 2020 - 2021)

Submitted by

Nihir Yadav (37)

Viranchi Bhure (79)
Abhinav Masane (03)
Sagar Rokde (54)
Ravindra Ghawade (48)
VI Semester B. E. (IT)

Prof S. S Ganorkar
(Designation)

Department of Information Technology

K. D. K. College of Engineering, Nagpur

1
 (2020-2021)

Declaration

This Project work entitled Classifying Hand-written Digits Using Neural Network is our
own carried out under the supervision of Prof S. S Ganorkar at Department of Information
Technology, K D K College of Engineering, Nagpur. It is ensured that proper citation of
references is done.
As far as our knowledge is concern, this work has not been submitted to any other university
for the award of any degree.

Name of the Projectee Signature

1. Nihir Yadav
2. Viranchi Bhure
3. Abhinav Masane
4. Sagar Rokde

5. Ravindra Ghawade

2
 Acknowledgement

We have great pleasure in expressing our most sincere regards and deep sense of gratitude to
our Project Guide Prof S. S Ganorkar for his/her able guidance and valuable suggestion.
We would to like to express our deep sense of gratitude to our respected Dr. Mrs. S. P.
Khandait Professor & Head, Department of Information Technology, K D K College of
Engineering, Nagpur for her encouragement and support.

We feel happy to extend our heartfelt thanks to the Principal, Dr. D P Singh and Vice-
Principal, Dr. A M Badar for being a source of inspiration & motivation.
Last but not least we would like to thank entire Information Technology Department, our
parents and all our friends who have helped us in completing this task successfully.

Projectee

1. Nihir Yadav
2. Viranchi Bhure
3. Abhinav Masane
4. Sagar Rokde

5. Ravindra Ghawade

3
 Department of Information Technology
K. D. K. College of Engineering, Nagpur

Certificate
This is to certify that the project work entitled Server Based Quiz Application Using Google
Firebase is submitted by the following students of VIII Semester B. E. Information
Technology.

Names of Students

1 Nihir Yadav 2 Viranchi Bhure

3 Abhinav Masane 4 Sagar Rokde

5 Ravindra Ghawade

is a bonafied work done under my/our supervision. This project work is submitted in partial
fulfillment of the requirement for the award of Degree of Bachelor of Information Technology
under the Faculty of Science & Technology; RTM Nagpur University, Nagpur during the
Academic Year 2020 – 2021.

Prof S. S Ganorkar
(Designation)

Dr. Mrs. S. P. Khandait Dr. D. P. Singh

Professor & Head, IT, Principal
K. D. K. College of Engineering, Nagpur K. D. K. College of Engineering, Nagpur

4
 Abstract

Identification of handwritten digits automatically by computers or any other equipment is called

handwritten digit recognition. The application of it are increasing day by day, while established
applications are financial statements audit, document processing and postal service. The goal of
this paper is to observe the variation of accuracies of our neural network to classify handwritten
digits using various numbers of hidden layers and epochs and to make the comparison between
the accuracies. The objective is to study the difference between accuracies achieved by our
neural network by fine tuning parameters such as number of hidden layers, epochs, etc. We used
the MNIST (Modified National Institute of Standards and Technology) dataset for evaluating the
performance of our model.

5
 Index
Particulars Page No.

Declaration 2

Acknowledgement 3

Certificate 4

Abstract 5

Chapter Scheme Page No.

Chapter 1: Introduction 7

Chapter 2: Problem Statement 8

Chapter 3: Background Study 9

Chapter 4: Software, Libraries and

Tools Used 10

Chapter 5: Architecture

Chapter 6: Flow chart 12

Chapter 7: Result 13

Chapter 7: Conclusion 16

Chapter 8: References 17

6
 Introduction

Many artificial intelligence, machine learning and deep learning researchers are trying to
replicate the ability of humans to recognize and perceive their surrounding and have machines
or computer do the same. Deep Learning is already being used in a number of fields, which are
already increasing. Domain such as regression analysis, topic categorization, speech recognition,
robotics, spam detection, video analysis, segmentation, pattern recognition, natural language
processing, image classification, visual imagery analyzing, object detection, face recognition
are already making use of neural networks.

Pattern recognition, branch of artificial intelligence extracts, calculates, processes and classifies
samples through computers which can be used in various applications like speech, image,
character and text recognition. The neural networks have been successful in recognizing hand-
written numbers with accuracies reaching close to that of an average human. Hand-written digit
classifier can be used to automate tasks processing of bank documents, envelopes’ postal code
recognition, auditing documents, etc.

The architecture of a neural network consists of an input layer, an output layer. The number of
hidden layers between the first layer i.e. input layer and the output layer is usually very high. The
number of neurons are still high in every layer of neural network even if the model is simple
enough.
The goal of the paper is to observe and study the changes in a model’s accuracy by fine tuning a
number of parameters such as number of hidden layers, number of epochs, which activation
function to use, etc.

7
 Problem Statement

Humans can recognize pattern by differentiating number of things by observing the

characteristics of things. Computers and machines can recognize patterns by learning pixel
patterns.

There are only 10 fixed category of samples in our problem statement i.e. 0 to 9 but the hand-
written digits by number of people will vary and in some cases there will be errors in manual
classification because of bad handwriting and thus automatic recognition can be very difficult
despite having only 10 categories.

We have the training samples having shape 60000, 28, 28. The first number is the quantity of
samples. Then each sample in turn have shape of 28 into 28 pixels. If we try to convert the two
dimensional sample into a single dimension data structure, it will become a 28 into 28 = 784
dimensional array. After changing the dimensions of inputs, then it can be fed to our model.

Neurons also have another function before sending it’s output to the succeeding layer and that is
performing and probability function: an activation function is applied to each neuron’s output.
Activation functions finds non linear patterns in the dataset. To get to our model having a higher
chance of learning correlation and mapping patterns between input and outputs, we add weights
on the input samples and feed it to a activation function and the sum of both is send to the next
layers’ neurons.

8
 Background Study
The handwritten digit database MNIST is used for the testing our model. The dataset contains
sixty thousand training samples and the remaining ten thousand testing samples . Each samples
are in grayscale. As the size of samples are in pixels of 28 * 28, it can transformed into a single
dimensional array having 787 elements. Each element of the array is a integer that too binary.
The binary integer describes the intensity of a particular pixel.

The input layer contains neurons which receive the dataset, in our case the hand-written digits.
The dataset provided to the input layer is in the format of the product of pixels of the images
which was in dataset. Then comes the hidden layers, which takes the output of input layer as
it’s input and apply the activation function to the input.

Lastly, the output layer will classify the outputs of all the previous layer in one of the ten classes
i.e 0 to 9. The data used for training the model is from MNIST dataset . The dataset contains high
amount of 28 * 28 pixel sized images so the input layer will take the input as 784 neurons.
During the model training we also used Gradient descent as the backpropagation algorithm. In
our model, the input were the digits.
9

Software, Libraries and Tools Used

Python
Python is an interpreted high-level general-purpose programming language. Python's design
philosophy emphasizes code readability with its notable use of significant indentation. Python
is undoubtedly the best choice for machine learning. It's easy to understand, which makes data
validation quick and practically error-free. By having access to a widely developed library
ecosystem, developers can perform complex tasks without extensive coding.

Google Colab
Colab is a free Jupyter notebook environment that runs entirely in the cloud. Most
importantly, it does not require a setup and the notebooks that you create can be
simultaneously edited by your team members - just the way you edit documents in Google
Docs. Colab supports many popular machine learning libraries which can be easily loaded in
your notebook.

Keras
Keras is an open-source software library that provides a Python interface for artificial neural
networks. Keras acts as an interface for the TensorFlow library. Keras contains numerous
implementations of commonly used neural-network building blocks such as
layers, objectives, activation functions, optimizers. Keras follows best practices for reducing
 Software, Libraries and Tools Used

Tensorflow
An open source platform and framework for creating, training, evaluating and deploying
machine learning models. It has a comprehensive, flexible ecosystem of tools, libraries and
community resources that lets researchers push the state-of-the-art in ML and developers
easily build and deploy ML powered applications. It can be used across a range of tasks but
has a particular focus on training and inference of deep neural networks. The name
TensorFlow derives from the operations that such neural networks perform on
multidimensional data arrays, which are referred to as tensors.

Matplolib
Matplotlib is a plotting library for the Python programming language and its numerical
mathematics extension NumPy. It provides an object-oriented API for embedding plots into
applications using general-purpose GUI toolkits like Tkinter. Several toolkits are available
which extend Matplotlib functionality. Some are separate downloads, others ship with the
Matplotlib source code but have external dependencies.

MNIST

The MNIST database (Modified National Institute of Standards and Technology database) is a
large database of handwritten digits that is commonly used for training various image
processing systems. The database is also widely used for training and testing in the field
of machine learning. The MNIST database contains 60,000 training images and 10,000 testing
images.
 Architecture

The input layer receives all the features as input and the output y is called the output layer. The
output has only one node. The hidden layer can have a lot of nodes or a very few nodes
depending on how complex the problem may be. Each node gives the output of a linear function
after the linear output passes through an activation function, and takes inputs from each node of
the preceding layer.

The values of weight and bias associated with the functions will have to be learned by the
algorithm as it tries to optimize those values in order to best fit the given data.

In our model, the two hidden layers have 128 neurons, 10 neurons for the output layer as each
neuron will correspond to each digit i.e. from 0 to 9 , and input will be 28 * 28 or 784
dimensional array.

Before the sum is given to its succeeding neuron, it goes through a function called an activation
function. So, each neuron has 2 function. First is the weighted sum Z:
Z = W.X + b
The second function is the activation function output A:
 A = f(Z)
There are various types of activation functions used in Neural Networks. Rectified linear unit of
ReLU function is one of the popular activation function. It has very simple function. It will
perform a mathematical linear function and apply to any positive numbers whereas it simply set
value to 0 for any negative numbers.
One other popular activation function which we are using is softmax. This function calculates
probability values for multiple neurons or classes, in our problem 10 neurons or classes who’s
sum goes to maximum 1. The function returns the probabilities for multiple classes according to
input provided. The neuron with the highest probability will be choose to calculate the
predictions.
Flow chart
 Implementation

Fig: Model Summary

Fig: Model Training

Model’s Predictions
 Conclusion and Future Work

Hand-written digits classifier using neural network has been successfully implemented in this
paper. The model was trained and tested on the MNIST dataset. In this paper, our neural
network model shows that using Deep Learning technology, effective results have been achieved
with a great accuracy. After applying neural network on Keras as Tensorflow its backend, our
model have shown great efficiency and getting an accuracy of 0.9631 or 96.31 in percentage.

As our neural network has less number of neurons it has low memory requirements while
achieving high classification accuracy but need a lot of training data.

Many areas like processing of images has applications for neural network. Online and offline
process of hand-written digit classifiers has already being making use of neural networks. The
extensive the amount of training data sets’ samples, higher will be the classification accuracy.
We have shown even by using simple architecture, having less number of neurons, having low
computational requirements, we can still achieve high enough accuracy that mimic humans
ability to recognize digits. Direction for future work includes We can try implement the LeNet
Architecture of CNN or trying other neural network models like RNN and creating models of
different styled approaches i.e having different number of trainable parameters in our future
works.
 {References}

[1] Fathma Siddique; Shadman Sakib; Md. Abu Bakr Siddique. “Recognition of Handwritten
Digit using Convolutional Neural Network in Python with Tensorflow and Comparison of
Performance for Various Hidden Layers” Proceedings of the 2019 5thInternational
Conference on Advances in Electrical Engineering (ICAEE) (2019)

[2] Tanya Makkar; Yogesh Kumar; Ashwani Kr Dubey; Álvaro Rocha; Ayush Goyal.
“Analogizing time complexity of KNN and CNN in recognizing handwritten digits” Fourth
International Conference on Image Information Processing (ICIIP) (2017)

[3] Shifeng Huang . “Influence of Different Convolutional Neural Network Settings on the
Performance of MNIST Handwritten Digits Recognition” International Conference on
Artificial Intelligence and Education (ICAIE) (2020)

[4] Wenfei Liu, Jingcheng Wei, Qingmin Meng. “ Comparisions on KNN, SVM, BP and the CNN
for Handwritten Digit Recognition”IEEE International Conference on Advances in Electrical
Engineering and Computer Applications (AEECA) (2020)

[5] Mayank Jain; Gagandeep Kaur; Muhammad Parvez Quamar; Harshit Gupta. “Handwritten
Digit Recognition Using CNN” International Conference on Innovative Practices in
Technology and Management (ICIPTM) (2021)

17