TRIBHUVAN UNIVERSITY

INSTITUTE OF ENGINEERING
Khwopa College of Engineering
Libali, Bhaktapur
Department of Computer Engineering

A FINAL REPORT ON
DETECTING COVID-19 INDUCED PNEUMONIA
FROM CHEST X-RAYS

Submitted in partial fulfillment of the requirements for the degree

BACHELOR OF COMPUTER ENGINEERING

Submitted by
Prasanna Adhikari KCE074BCT029
Sangat Rokaya KCE074BCT039
Subin Timilsina KCE074BCT042
Sujan Acharya KCE074BCT043

Under the Supervision of

Er.Dinesh Ghemosu
Department of Computer Engineering

Khwopa College of Engineering

Libali-8, Bhaktapur
22 February 2021
Certificate of Approval
The undersigned certify that the Mid term project entitled “ Detecting COVID-
19 Induced Pneumonia From Chest X-rays ” submitted by Prasanna Adhikari,
Sangat Rokaya, Subin Timilsina and Sujan Acharya to the Department of Computer
Engineering in partial fulfillment of requirement for the degree of Bachelor of
Engineering in Computer Engineering. The project was carried out under special
supervision and within the time frame prescribed by the syllabus.
We found the students to be hardworking, skilled, bona fide and ready to undertake
any commercial and industrial work related to their field of study.

................................
Er. Dinesh Ghemosu
(Project Supervisor)

...........................................................
Prof. Dr. Sashidhar Ram Joshi
(External Examiner)
Central Campus Pulchowk, Institute of Engineering
Lalitpur, Nepal

.........................................
Er. Shiva K. Shrestha
Head of Department
Department of Computer Engineering, KhCE

i
Copyright
The author has agreed that the library, Khwopa College of Engineering and
Management may make this report freely available for inspection. Moreover, the
author has agreed that permission for the extensive copying of this project report
for scholarly purpose may be granted by supervisor who supervised the project
work recorded herein or, in absence the Head of The Department wherein the
project report was done. It is understood that the recognition will be given to
the author of the report and to Department of Computer Engineering, KhCE in
any use of the material of this project report. Copying or publication or other use
of this report for financial gain without approval of the department and author’s
written permission is prohibited. Request for the permission to copy or to make
any other use of material in this report in whole or in part should be addressed
to:

Head of Department
Department of Computer Engineering
Khwopa College of Engineering(KhCE)
Liwali,
Bhaktapur, Nepal.

ii
Acknowledgement
We take this opportunity to express our deepest and sincere gratitude to our
supervisor Er. Dinesh Ghemosu, for his insightful advice, motivating suggestions,
invaluable guidance, help and support in successful completion of this project
and also for his constant encouragement and advice throughout our Bachelor’s
program. We are deeply indebted to our teachers Er. Shiva k. Shrestha, Er.Bindu
Bhandari for boosting our efforts and morale by their valuable advices and suggestion
regarding the project and supporting us in tackling various difficulties. Also,
we would like to thank Er. Ram Sapkota for providing valuable suggestions
and supporting the project. In Addition, we also want to express our gratitude
towards Er. Dinesh Man Gothe for providing the most important advice and
giving realization of the practical scenario of the project.

Prasanna Adhikari KCE074BCT029

Sangat Rokaya KCE074BCT039
Subin Timilsina KCE074BCT042
Sujan Acharya KCE074BCT043

iii
Abstract
Exponential growth of COVID-19 has led to the serious damage across the globe
resulting in rapid use of expensive COVID-19 testing kits which are minimally
available. Those limited kits are not so able to provide the exact diagnosis timely.
So, we intended to the use Chest X-Rays and AI based automatic detection model
as a quick alternative diagnosis to prevent COVID-19 spreading among people
with the minimum cost.This project attempts to identify whether a person is
affected from the COVID-19 or Pneumonia through Chest X-Rays. In this work
we use Deep Learning techniques with deep CNN based models i.e. Dark-Net
model. After models are trained and validated, Grad-CAM is used for the proper
reasoning as it visualizes over the regions of the processed image according to the
weight, which are essentials for making the prediction more precise. Such result
obtained from that model can be accessed and monitored by the expert radiologist
that assist in further diagnosis process of the patient. Using the publicly available
datasets [5, 6], our model gives accuracy of 98.6 percent.
Keywords: Convolution Neural Network,Dark-Net, COVID-19, Neural Network,
Grad-CAM.

iv
Contents
Certificate of Approval . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
Copyright . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi
List of Abbreviation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

1 Introduction 1
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Literature Review 3

3 Requirement Analysis 5
3.1 HARDWARE AND SOFTWARE REQUIREMENT . . . . . . . . 5
3.1.1 Hardware Requirements . . . . . . . . . . . . . . . . . . . . 5
3.1.2 Software Requirements . . . . . . . . . . . . . . . . . . . . . 5
3.2 FUNCTIONAL REQUIREMENT . . . . . . . . . . . . . . . . . . . 5
3.2.1 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.2.2 Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.3 NON-FUNCTIONAL REQUIREMENT . . . . . . . . . . . . . . . 5
3.3.1 Accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.3.2 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.3.3 Portability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.4 FEASIBILITY STUDY . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.4.1 Economic Feasibility . . . . . . . . . . . . . . . . . . . . . . 6
3.4.2 Technical Feasibility . . . . . . . . . . . . . . . . . . . . . . 6
3.4.3 Operational Feasibility . . . . . . . . . . . . . . . . . . . . . 6

4 Project Methodology 7
4.1 System Development Requirement . . . . . . . . . . . . . . . . . . . 7
4.2 Block Diagram of System . . . . . . . . . . . . . . . . . . . . . . . 8
4.3 Description of Working Flow of System . . . . . . . . . . . . . . . 9

5 Implementation Plan 12
5.1 Cost Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

6 Work Completed 13
6.1 Accumulation and Arrangement of Data set . . . . . . . . . . . . . 13
6.2 Creation of Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
6.3 Model Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
6.4 Visualization and Reasoning . . . . . . . . . . . . . . . . . . . . . . 16

v
7 Work in Progress 21
7.1 Visualization and Reasoning . . . . . . . . . . . . . . . . . . . . . . 21
7.2 Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

8 Conclusion 22
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

vi
List of Figures
4.1 Prototype Model for System Development . . . . . . . . . . . . . . 7
4.2 Block Diagram of our system . . . . . . . . . . . . . . . . . . . . . 8
4.3 Architecture of System (Dark-Net CNN) . . . . . . . . . . . . . . . 8
4.4 Block Diagram for generation of Saliency Map . . . . . . . . . . . 9
4.5 Diagram for Feature Extraction form CNN [1] . . . . . . . . . . . . 10

6.1 First part of accuracy traced . . . . . . . . . . . . . . . . . . . . . . 17

6.2 Second part of accuracy traced . . . . . . . . . . . . . . . . . . . . 18
6.3 Final loss traced . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
6.4 F-score calculation . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
6.5 Confusion Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
6.6 Visualization for COVID . . . . . . . . . . . . . . . . . . . . . . . . 20
6.7 Visualization for Pneumonia . . . . . . . . . . . . . . . . . . . . . . 20

vii
List of Abbreviation

CAT Computerized Axial Tomography

CNN Convolution Neural Network
CRX Chest X-Ray
CT Computed Tomography
DCNN Deep Convolution Neural Network
FN False Negative
FP False Positive
FrMEM Fractional Multichannel Exponent Moments
Grad-CAM Gradeint Weight Class Activation Mapping
NIN Network in Network
RNN Recurrent Neural Network
RT-QPCR Reverse Transcription – Quantitative Polymerase Chain Reaction
SVM Support Vector Machine
TN True Negative
TP True Positive
UML Unified Modeling Language
WHO World Health Organization
YOLO You Only Look Once

viii
Chapter 1
Introduction
1.1 Background
The 2019 novel coronavirus (COVID-19) pandemic initiated in Wuhan, China [9]
in December 2019 and has become a serious affair widely. The novel coronavirus
(COVID-19) has become the most pressing issue facing mankind. Like a wildfire
burning through the world, the COVID-19 disease has changed the global landscape
in only three months. A high reproduction rate and a higher chance of complications
has led to border closures, empty streets, rampant stockpiling, mass self-isolation
policies, an economic recession, and a cultural shift towards mistrust. This pandemic
has created a massive threat to all of us as death toll are still rising today in a
great extent worldwide. As per WHO with the increment of each second, a new
death case is reported. As of now (7 February 2021) [9], total cases that has
been reported is approximately 106,375,662, out of which 2,321,294 are dead and
78,021,392 are recuperated. COVID-19 has led to exceptional healthcare crisis
with millions of people are affected across the globe.
The current COVID-19 pandemic has placed a huge strain on healthcare systems,
with hospitals being overloaded, scarcity of testing kits, scarcity of infrastructure
for treatment, as well as lack of skilled manpower to operate them even in the
sophisticated countries. As per detail analysis fever, dry cough and tiredness
were found to be COVID-19 common symptoms but as it worsen severe acute
respiratory syndrome, chest pain, pneumonia, multi-organ failure were seen which
ultimately leads person to death. For the identification of trace of pneumonia,
Chest [9] X-rays are commonly used so that we supposed it would be an intriguing
challenge to determine viral, bacterial and COVID-19 pneumonia infections. Additionally,
it is also stated that CXRs screening for asymptomatic carries of COVID-19 may
aid as a viable substitute for the available RT – QPCR tests. The high infection
rate of the COVID-19 has resulted unprecedented burden on the healthcare and
shortage of the isolation wards. So in the shortage and delay in PCR tests, CXRs
can become one of the fastest and the most affordable ways for the doctors to
the triage (degree of urgency) patients. So, in order to withstand the pandemic
crisis Chest X-rays acts as an non-invasive tool to monitor the progression of the
disease as CXRs is a widespread, relatively cheap, fast, and accessible diagnostic
modality, which can be easily brought to the patient’s bed, even in the emergency
departments. This results in the support of those centers which are facing staff
shortage, overwhelming patient loads. Due to these sorts of problems we were
motivated to develop a system that can assist the health sectors in carrying the
testing process faster and with better efficiency.
With the help of various AI based systems and Deep learning techniques, we
can develop a model that automatically detects COVID-19 by using CXRs on
training a model using various data-sets obtained from the CXRs, using various
network model, to represent a cost beneficial solution for the early diagnosis of

1
infection and timely risk stratification with precision.Also for the addition of the
more transparency and to make the model more explanatory various reasoning
techniques are available among which Grad-CAM visualization techniques is used.
It uses the gradients of the target concept say ’X-Ray’ in classification network
flowing into the final convolution layer to produce a rough localization map contrasting
the affected regions of the processed X- Ray image for making the prediction.

1.2 Problem Statement

As motivated by the growing COVID-19 cases, unconfirmed cases and uncertainty,
we made an effort to assist the health sector to work as efficient as possible within
less time. In this pandemic the health sector has been the busiest working sector
as the numbers of COVID-19 patients are growing rapidly day by day. Different
countries have reported a new phase of COVID-19. This has brought enormous
burden in the health sector as the number of patients are increasing day by day
but the facilities are limited. Public health centers are unable test so many people
for COVID-19 with precision (a number that is increasing day by day). With
the urgency there has been problem in correctly identifying the virus. Similarly
laboratories has been charging between Rs.5500 to Rs.7500 for a screening test
which is impractical for the average people as a result of which death has been
reported in many lower middle class people in our country. The CXR of the
seriously ill person would have easily identified the COVID-19 virus that affected
the patients lungs. In the same way, production of the fake kits and equipment
due to which actual diagnosis has not possible in many places. So we aim to help
everyone to properly mitigate the problem of coronavirus.

1.3 Objective
The main objective of this project is :

• To detect COVID-19 induced Pneumonia from the Chest X-Rays and visualize
the affected regions.

2
Chapter 2
Literature Review
COVID-19 is a world-wide pandemic affecting more than 10 million people. Similarly,
due to its nature and recent outbreak it’s tracing requires time, money and
manpower. So many scientist and engineer were involved in developing a system
that could automatically detect the COVID-19 in those patients whose lungs were
affected. Some of the previously done task and activities are discussed below.
Ozturk Tulin et.al [1] have proposed a deep learning based model to detect
and classify COVID-19 cases from X-ray images. Their model is fully automated
with an end-to-end structure without the need for manual feature extraction. The
developed system is able to perform binary and multi-class tasks with an accuracy
of 98.08% and 87.02%, respectively but this activity was carried in very few input
images.
Elaziz Mohamed Abd et. al [2]the features extracted from the chest x-ray
images using new FrMEMs. A parallel multi-core computational framework utilized
to accelerate the computational process. Then, a modified Manta-Ray Foraging
Optimization based on differential evolution used to select the most significant
features. The proposed method evaluated using two COVID-19 x-ray datasets.
The proposed method achieved accuracy rates of 96.09% and 98.09%.
Asif Sohaib et. al [3]automatically detect COVID-19 pneumonia patients
using digital chest x-ray images while maximizing the accuracy in detection using
DCNN. The dataset consists of 864 COVID-19, 1345 viral pneumonia and 1341
normal chest x-ray images. In this study, DCNN based model with transfer
learning have been proposed for the detection of coronavirus pneumonia infected
patients using chest X-ray radiographs and classification accuracy of more than
98% (training accuracy of 97% and validation accuracy of 93%). The results
demonstrate that transfer learning proved to be effective, showed robust performance
and easily deployable approach for COVID-19 detection.
Encalada Angel et. al [4] pre-trained network to reduce training times and in a
first model obtained an accuracy of 90%. And by adjusting the learning rate they
were able to improve the model to 97%.The pre-trained network that was used
was the Resnet34 with SVM but they used very few images for the training model.
They used radiographs from people from other countries for testing their images
and could perform just enough on the practical cases. We cannot say whether or
not this affects the model because this requires further investigation and is not our
field. We leave this part to the health experts and we clarify that our intention
is to collaborate and combine their experience with this type of tool. To increase
the data-set, they used some transformations but a medical imaging expert might
recommend performing additional transformations and even prefer not to perform
the ones we use.
Mangal Arpan et. al [7] created a COVID-19 AI Detector based on novel
deep neural network for triaging patients for appropriate testing. On the publicly
available COIVD chest XRay data-set the model gave 90.5% accuracy with 100%

3
sensitivity for the COVID-19 infection and there was a significant improvement on
the results of Covid-Net on the same dataset.COVID-net uses a machine-driven
design exploration to learn the architecture design starting from initial design
prototype and requirements. It takes as input a chest X-Ray image and outputs
a prediction among three classes: Normal, Pneumonia and COVID-19.
We used the DarkNet model because the parameters in other models were
very high and difficult to work with. Similarly, the problem of overfitting were
seen much more prevalent in other models but with DarkNet this problem was
minimized to high extent and the number of parameters to work with were less so
the process was handled efficiently.

4
Chapter 3
Requirement Analysis
3.1 HARDWARE AND SOFTWARE REQUIREMENT

The requirements for this project are:

3.1.1 Hardware Requirements

• 8 GB RAM

• GPU with 4+ GB VRAM

3.1.2 Software Requirements

• Libraries Flask, Pandas, Matplotlib

• Fast AI

• Google Colab

3.2 FUNCTIONAL REQUIREMENT

3.2.1 Classification
System should be able to classify three class of COVID-19, Normal and Pneumonia.

3.2.2 Visualization
System should be able to visualize the affected regions.

3.3 NON-FUNCTIONAL REQUIREMENT

3.3.1 Accuracy
Our system is accurate and provide the accuracy upto 98.6 percentage.

3.3.2 Performance
The performance of the system is fast, when image is uploaded then instantly
visualized image of chest X-Ray is obtained with the classification whether it is
Normal, COVID-19 or Pneumonia.

5
3.3.3 Portability
The project is portable as it is implemented on the web.

3.4 FEASIBILITY STUDY

The following points describes the feasibility of the project.

3.4.1 Economic Feasibility

The total expenditure of the project is just computational power. The data-sets [6]
required for the project are easily available and computational power required for
us can be accessed through our local machine as well as some cloud platforms.
Data-sets is available from various sources that is open-sourced and prepared for
studies of various countries and computational power using our own local machine
along-side Google Collab, a cloud computing service so, the project is economically
feasible.

3.4.2 Technical Feasibility

The project needs to be trained with lots of labeled and unlabeled images from
various data-sets. Preparing the data is complex as the images need to be scrutinized
under the algorithm. Creating the project will be challenging but is feasible. In
terms of availability, the data-sets, they can be easily accessed. Training huge
data-sets takes a lot of computation power so a CPU alongside a capable GPU is
required for our model.

3.4.3 Operational Feasibility

The project can be operational after adequate training and validations made using
labeled data using DCNN. After the system for detection of COVID-19 from CRXs
is developed, the system can be operated to be used as an assistant in various
hospitals and health centers with facilities of radiography so it can be used in
assisting for identifying of COVID-19 as well as pneumonia easily and effectively.
Thus, the project is operationally feasible.

6
Chapter 4
Project Methodology
4.1 System Development Requirement
Prototype methodology is a system development model in which a prototype
is built, test and then reworked when needed until an acceptable prototype is
achieved. Prototype is also called as an dummy implementation where, dummy
model is designed with certain parameters and on the basis of the ouput rework
is done.

Figure 4.1: Prototype Model for System Development

For the development of the our model we used the prototype model in which
a prototype is built first with the certain parameters then it is tested, if certain
limit of the accuracy is achieved then it is accepted as the final form otherwise, it
is reworked until acceptable accuracy is achieved. The prototype model is chosen
as our system has the limited functional requirements and also user requirements
are also fixed.

7
4.2 Block Diagram of System
For the development the system, firstly Chest X-rays is fed to the system and
model predicts and classifies whether it is COVID-19 or Pneumonia or Normal
and on the basis of the classification and the visualization algorithm the saliency
map is generated. Saliency maps aim to define where a particular region is different
from its neighbors with respect to image features.
Block Diagram of System is shown Figure 4.2
We used Dark-Net model whose system architecture is shown in Figure 4.3

Figure 4.2: Block Diagram of our system

Various entropy calculations are done for the generation of the sailency map
which is shown in Figure 4.4.

Figure 4.3: Architecture of System (Dark-Net CNN)

8
 Figure 4.4: Block Diagram for generation of Saliency Map

4.3 Description of Working Flow of System

For the project of detection of COVID-19, we firstly gather the data-set required
for us during our model training and validations as data preparation. The main
source of COVID-19 patients X-Ray data-set is from the GitHub repository [5]
of Mr.Joshep Paul Cohen who is postdoctoral fellow at University of Montreal.
The other data of X-Ray images required for us are from the Kaggle website [6]
which includes X-Ray images of normal patients as well as pneumonia patients.
The data-sets are then used to collect the images that are most probably useful
for our models. The images of COVID-19 patient’s X-Ray contains many sort of
X-Ray images ranging form different inter linked diseases to different views of an
X-Ray images like anterio-posterior, sectional view etc. So we select those X-Ray
images that is only linked to COVID-19 as well as the anterio-posterior image
of the lungs and work with them only. So we expect more than 395 images of
COVID-19 patients from this data-set. Then we use the X-Ray images of Normal
and pneumonia patients that we have obtained from the Kaggle data-set where we
might use about 500 images of normal patients with proper shuffling form the given
images and 450 images of pneumonia patients. Then we combine the data-sets into
single header as our data-sets. Then we split those images into Training part and
validation part in the ratio of 80:20 respectively. For the primary purpose we shall
be detecting the COVID-19 positive and negative X-Ray images only as binary
classification. But we shall we extending this model into multi-class classifications
as well by using the pneumonia data sets that we have acquired from the Kaggle.
Then we shall merge this data into a single folder with distinguished header.
After the data collection and preparation task is completed, we move onto
select a correct model that can be used for feature detection to find out whether the
COVID-19 virus or pneumonia has affected the lungs or not. We firstly normalize
our image so that we can have a faster gradient descent in our working model.
The we shall select a Transfer Learning mechanism to treat our images for feature
extraction. The Transfer Learning mechanism is that mechanism where a proven
working model has already been made for detecting feature in the image and can
assist the further working in similar sort of feature extraction process with higher
initial accuracy which further needs to be tuned for other tasks. In this project
we are initially proposing to use DarkNet model which shall we properly tuned to
be in favour of our project. DarkNet model is a typical CNN structure which has
a convolution layer that extracts features from the input with the filter applied
on it, a pooling layer to reduce the size of computational performance, and a fully
connected layer which is combined one or more times. The internal parameters are
used that needs to be properly adjusted to accomplish our task of classification.
DarkNet is a classifier model that forms the basis of real-time object detection
system called YOLO. This system has the state-of-the-art architecture designed
for object detection. Here we initially use small filters to extract tiny features.

9
We are expecting the DarkNet-19 model to be efficient for us that consists of
19 convolution layers and five pooling layers, using Max-pool. This model shall
be focused primarily in detecting COVID-19 positive or negative then shall be
enhanced for multi-class classifications.

The DarkNet architecture is shown as follows where C denote a convolutional

layer, and M denote a Max-pool layer.:

C1-M1-C2-M2-C3-C4-C5-M3-C6-C7-C8-M4-C9-C10-C11-C12-C13-M5-C14-C15-
C16-C17-C18-C19

For input signal X (image) and kernel K, the two-dimensional convolution

operation can be defined as follows.
XX
(X ∗ K)(i, j) = K(m, n)X(i − m, j − n)
m n

where * represents the discrete convolution operation. The K matrix slides

over the input matrix with stride parameter. The Leaky rectified linear unit is
used as an activation function in the DarkNet architecture.

Figure 4.5: Diagram for Feature Extraction form CNN [1]

We are expecting a descent accuracy for the model with this network. As this
network has about 1 - 10 millions training parameters which is decent enough to
train a model without having over-fitting. Since this network focuses on capturing
and learning new small differences rather than going extra deep to find out the
features from the image. But we shall be focusing on improving the models
accuracy to the best value possible with hyper-parameter tuning and many other
fine tuning mechanisms. This can provide the output for us expected from an
X-Ray image.
Now we shall head on for reasoning part for our built model to demonstrate
why the model predicted such output. For this part of project we shall be using
Class Activation Maps or Saliency Maps with the help of Grad-CAM that can
show the part in the X-Ray that has been affected by virus or bacteria. This
can assist a medical professional to easily understand the X-Ray that can help
in accurate and faster surveillance of the patients. He/She can also be assisted
in predicting the severity of the patient’s lungs. The confidence of positive for
the image can help the image to be easily interpreted by the professionals. For

10
this we examine the current X-Ray frame by calculating entropy of every region
by inputting intensity, colour and orientation of the image. Then we map out
the intensity, color and orientation feature in the map showing the area that is
found to be affected.Here Grad-CAM assumes the final score Y c for a particular
class c can be written as a linear combination of its global average pooled last
convolutional layer feature maps Ak .

Where, wkc is the weight for the particular feature map Ak Each spatial location
(i ,j) in the class- specific saliency map Lc is then calculated as:

Here Lcij directly corresponds with the importance of a particular spatial location
(i,j) for a particular class c and functions as a visual explanation of the the class
predicted by the network. Weight for the particular feature map can be calculated
as the:

Here Z is the constant number of the pixel in the activation map and is obtained by
partial derivative of score Y c with respect to the activation map Ak . For obtaining
fine- grained pixel scale representation Guided Grad-CAM is used where class
specific saliency map Lc are unsampled and fused via point wise multiplication
with the visualizations generated by the Back-propagation.
Finally, when our objectives are fulfilled we shall then deploy this model into a
flask server or any other possible for us so that we do not have to train this model
again to use them and also assist in determination of condition of lungs to find
out whether it has been affected by the corona virus or any other bacteria or not.

11
Chapter 5
Implementation Plan
5.1 Cost Estimation
The main requirement of this project is the images of Chest X-Rays for training
and validating our model whose processing gives us the increased accuracy of the
project that we desire and much of the data is readily available on the internet
already so we can carry out the process easily. More the data-sets higher would be
the probability of gaining higher accuracy otherwise it might result in over-fitting
problems during the training process as number of training parameters will be
high in transfer learning process. Similarly, higher computational power is desired
for faster processing and training the huge amount of data. But the processing
ability we currently can access might be sufficient for our desired model building
process by using our local machine as well as some cloud based service for carrying
out the modelling process as well as for proper reasoning of the output provided.
The deployment of the given system might add some cost but we will be try to
make this system to be as cost efficient as possible.
So, the cost in our project is bare minimum but if we are going to increase the
scope of the project then cost can be managed according to the necessity.

12
Chapter 6
Work Completed
In the past duration of time we accomplished the following:

6.1 Accumulation and Arrangement of Data set

For the model training and validation purpose data-set were arranged. We accumulated
data from the GitHub repository of Mr.Joshep Paul Cohen [5] and other data were
collected from the Kaggle website [6]. The GitHub repository contained the images
of COVID-19 patient with images of X-Rays as well as some CT-Scan images. So
using pandas we obtained the CXR images only with COVID findings and the
views of the lungs were anterior-posterior and posterior-anterior. We collected
481 Chest X-Ray images of COVID patient. These were the only views taken as
they were sufficient for the project to detect the traces of COVID. We used the
X-Ray images of normal and pneumonia patients from the Kaggle data-set where
we collected 588 images of normal patients with proper shuffling form the given
images and 536 images of pneumonia patients. We used this data-set for multi-
class classification i.e for showing whether it’s COVID-19 or normal or pneumonia
within the patient. We then spitted this data-set to training part and validation
part i.e. 80% of our data-set for training and remaining 20% for the validation
process.

6.2 Creation of Model

In this project, we used DarkNet model which was properly tuned to be accurate
for our project system. DarkNet is a classifier model that forms the basis of real-
time object detection system called YOLO. DarkNet-19 is a convolutional neural
network that is used as the backbone of YOLO. Similar to the VGG models it
mostly uses filters and doubles the number of channels after every pooling step.
Following the work on NIN it uses global average pooling to make predictions as
well as filters to compress the feature representation between convolutions. Batch
Normalization is used to stabilize training, speed up convergence, and regularize
the model batch.
In short, DarkNet model is a typical CNN structure which has:
• a convolution layer that extracts feature from the input with the filter applied
on it.
• a pooling layer to reduce the size of computational performance.
• a fully connected layer which is combined one or more times.
The YOLO, family of models are a series of end-to-end deep learning models
designed for fast object detection, developed by Joseph Redmon with his team and

13
first described in a paper titled You Only Look Once: Unified, Real-Time Object
Detection [8]. The approach of YOLO involves a single deep convolutional neural
network (originally a version of GoogleNet, later updated and called DarkNet
based on VGG) that splits the input into a grid of cells and each cell directly
predicts a bounding box and object classification. The result is a large number
of candidates bounding boxes that are consolidated into a final prediction by a
post-processing step.
The Biggest advantages of YOLO model are:
• Speed (45 frames per second — better than real-time)

• Network understands generalized object representation (This allowed them

to train the network on real world images and predictions on artwork was
still fairly accurate).

• faster version (with smaller architecture) — 155 frames per sec but is less
accurate.
This system has the state-of-the-art architecture designed for object detection.The
internal parameters are used that needs to be properly adjusted to accomplish our
task of classification.
Here we initially used small filters to extract tiny features. We are expecting
the DarkNet-19 model to be efficient for us as it consists of 19 convolution layers
and five pooling layers, using Max-Pool. This model shall be focused primarily
multi-class classifications.
First, we have trained the DarkNet deep learning model to classify and detect X-
ray images into three categories: COVID-19, Normal, Pneumonia. The performance
of the proposed model is evaluated using the 5-fold cross-validation procedure for
both the binary and triple classification problem. Eighty percent of X-ray images
are used for training and 20 percent for validation. The experiments are repeated
five times. All of the split k pieces are wrapped in folds to use in the validation
stage. We have trained DarkCovidNet for 100 epochs. There is significant increase
in loss values in the beginning of the training which decrease substantially in later
stages of the training which occurs due to the volume of data in the COVID-
19 class, which is much lower than in the other two classes (pneumonia and
undiagnosed data). However, this discrepancy in the model is slowly reduced
when the deep model examines all X-ray images over and over again for each
epoch during the training.
The Multi-class classification performance of the DarkNet model has been
evaluated for each fold, and the average classification performance of the model is
calculated. The overlapped CM is created using the sum of CMs obtained in all
folds. Thus, it is aimed to obtain an idea about the general perforations of the
model. The DarkNet model achieved an average classification accuracy of 99.02
percent to classify:COVID-19, Normal and Pneumonia.
The proposed model has 17 convolution layers. Each DarkNet layer has one
convolutional layer followed by BatchNorm, and LeakyReLU operations, while
each 3 Conv layer has the same setup three times in successive form. The batch
normalization operation is used to standardize the inputs, and this operation has
other benefits, such as reducing training time and increasing stability of the model.
LeakyReLU is a variation of the ReLU operation used to prevent dying neurons.

14
Unlike ReLU or sigmoid activation functions, which have zero value in the negative
part of their derivatives, LeakyReLU has a small epsilon value to overcome the
dying neuron problem. Similar to the Darknet-19 model, the Maxpool method
is used in all the pooling operations. Maxpool downsizes an input by taking the
maximum of a region determined by its filter. When working with three classes,
the proposed model performs the COVID-19 detection task. If three different
classes of images are used in the input, the same model performs the classification
task to determine the labels of the input chest X-ray images as COVID-19 or
Normal or Pneumonia. The developed deep learning model consists of 1,164,434
parameters.

6.3 Model Evaluation

Model was prepared, then in order to evaluate the efficiency of the model it’s
accuracy was checked.
Accuracy tracked when it is executed for the 50 epoch.As we were encountering
the under fitting so that we stopped at the 50 epoch.Here epoch is the that where
entire data-set is passed forward and backward through the neural network.We
apply more than one epoch for the different pattern of the test input but epoch is
used according to the data-set, here we have limited number of data-set so that we
are encountering under fitting problem. Accuracy along with the training loss and
validation loss per epoch are shown in the Figure 6.1 and Figure 6.2 respectively.

In the training we used 42 batches and in the validation we used 9 batches.Here

batch is a hyper-parameter defining the number of samples to work.they are used
for iterating over the more samples and making prediction then, at the end of
batch predictions are compared to expected output variables and an error is
calculated.The trained and validated loss encountered per batches processed shown
in the graph Figure 6.3:

F-score is the measure of the test accuracy which is calculated with the help of
the precision and the recall of the test, where precision is the number of correctly
identified true results divided by total number of all true results including those
not identified true and recall is the number of correctly identified true results
divided by the number of samples that should have been identified as true.
f1 is the harmonic mean of the precision and recall and is calculated as:

f 1 = (2 ∗ precision ∗ recall)/(precision + recall)

Calculation of f1-score for our model is shown in Figure 6.4

Confusion matrix is an matrix used for evaluation of the performance of a
classification model.In this model we have 3 target classes COVID, Normal and
Pneumonia so we have 3 by 3 confusion matrix.It compares the actual target values
with those predicted by the model. In the vertical y axis represents the actual
target values and the horizontal X-axis represents the predicted values.From this
we can evaluate that how well our system is performing and what kind of error it is
making.In this we have four values True positive:The actual value was positive and

15
the model predicted a positive value, True negative:he actual value was negative
and the model predicted a negative value, False positive: he actual value was
negative but the model predicted a positive value and False negative: he actual
value was positive but the model predicted a negative value. In TP and TN
predicted values matches the actual values and in FP and FN predicted values
doesn’t matches the actual values.
We can calculate Accuracy, Precision and Recall from the confusion matrix easily
by using:
Accuracy = (T P + T N )/(T P + F P + T N + F N )
P recision = (T P )/(T P + F P )
Recall = (T P )/(T P + F N )
In the confusion matrix shown in figure 6.5: 85 COVID, 88 Normal and 86
Pneumonia were correctly classified by the model.

6.4 Visualization and Reasoning

For the sake of transparency of the model, the model should be able self explainatory.
In this case, we are using Grad-CAM techniques where the chest X-Rays images is
flowed through the model into the the final convolutional layer in order to produce
a coarse localization map highlighting the affected regions in the chest X-Ray for
predicting the concept.
Grad-CAM algorithm relies on the fact that model has detected some region of
image that brought the classification factor of the Chest X-rays. Grad-CAM
algorithm is based on the forward propagation and the backward propagation.Forward
propagation involves in the calculation and storage of intermediate variables (including
outputs) for a neural network in order from the input layer to the output. And
back-propagation does calculation of the gradient of neural network parameters,
this method traverses the network in reverse order, from the output to the input
layer, according to the chain rule from calculus. The algorithm stores any intermediate
variables (partial derivatives) required while calculating the gradient with respect
to some parameters layer.
So, for the implementation of the Grad-CAM, we used PyTorch hooks for the
visualise activation.Hooks are the function executed when forward and backward
calls are triggered. In PyTorch, the gradient of the leaf node in the computation
are only used so we considered the weight, biases and other parameters at the end
convolutional point. The gradients of the output with respect to the activation’s
are merely intermediate values and are discarded as soon as the gradient propagates
through them on the way back. So we made the maps with the weight basis of the
image at the end node of our model. With our Grad-CAM model, we obtained
visualization for COVID-19 and Pneumonia infected person’s Chest X-Ray as
shown in Figure 6.6 and Figure 6.7 respectively:

16
Figure 6.1: First part of accuracy traced

17
Figure 6.2: Second part of accuracy traced

Figure 6.3: Final loss traced

18
Figure 6.4: F-score calculation

Figure 6.5: Confusion Matrix

19
 Figure 6.6: Visualization for COVID

Figure 6.7: Visualization for Pneumonia

20
Chapter 7
Work in Progress
7.1 Visualization and Reasoning
Although we are known with the concept but we are having difficulties in the
implementation of the Grad-CAM techniques for the localization. We are in touch
with our HOD Er. Shiva k. Shrestha sir and also we are involving in the various
Youtube sessions and also we have been reading various research papers regarding
Grad-CAM implementation.

7.2 Deployment
We also have been working for the deployment of the system into a flask server
so that we don’t have to train model again and again to use them. Also as
per Dr.Jhanak Parajuli sir, we are also facilitated with the Heroku(cloud based
platform for building and executing application). As per the difficulty we shall
choose one among them.

21
Chapter 8
Conclusion
Almost 75 percent of task is completed and our team members have been working
on the various implementation techniques of Grad-CAM and deployment for the
system representation.

Table 8.1: Confusion Matrix of Modified Dark-Net Model

Predicted

Pneumonia
COVID

Normal
COVID 85 0 1
Actual

Normal 0 88 0
Pneumonia 0 3 83

22
Bibliography
[1] Tulin et al. 1. Ozturk. Automated detection of covid-19 cases using deep
neural networks with x-ray images. Computers in biology and medicine vol.
121 (2020): 103792., 2020.

[2] Salah A Darwish MM Lu S Sahlol AT. 2. Elaziz MA, Hosny KM. New
machine learning method for image-based diagnosis of covid-19. In PLoS One.
2020;15(6):e0235187, pages 1–4. PONE, 2020.

[3] Sohaib Asif, Yi Wenhui, Hou Jin, Yi Tao, and Si Jinhai. Classification of
covid-19 from chest x-ray images using deep convolutional neural networks.
medRxiv, 2020.

[4] Andres Torres Christian Tutiven Galvez, Angel Enclada. Covid-19 detector
using xrays. 2020.

[5] Mr.Joshep Paul Cohen. Covid-19 dataset. October 2021.

[6] KAGGLE. Covid-19 dataset. February 2021.

[7] Arpan Mangal, Surya Kalia, Harish Rajgopal, Krithika Rangarajan, Vinay
Namboodiri, Subhashis Banerjee, and Chetan Arora. Covidaid: Covid-19
detection using chest x-ray, 2020.

[8] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only
look once: Unified, real-time object detection, 2016.

[9] WHO. Coronavirus update. December 2020.

23