Machine Learning Basics

Basics of Machine Learning Image Classification Techniques https://iq.opengenus.org/basics-of-machine-learning-image-classification...
OpenGenus IQ: Learn Computer Science
Basics of Image Classification Techniques

in Machine Learning
50+ Linux Commands before joining a Company 💪 [MUST READ]
Machine Learning (ML) image classification
Reading time: 45 minutes
In this post, we will be focusing on different image classification techniques deployed to

make the computer vision as smart as human vision.
After reading this post, you will have an idea about:
What is Image Classification?
The pipeline of an image classification task including data preprocessing

techniques
Performance of different Machine Learning techniques on these tasks like:
Artificial Neural Network
Convolutional Neural Network
K nearest neighbor
Decision tree
Support Vector Machines
This article assumes that you are interested in the technical know-how of machine
learning, image classification in particular!
What is Image Classification?

lassification between objects is a fairly easy task for us, but it has proved to be a complex one for machines and therefore image
1 of 25 10/18/2020, 4:25 PM
assification has been an important task within the field of computer vision.
mage classification refers to the labeling of images into one of a number of predefined classes.
here are potentially n number of classes in which a given image can be classified. Manually checking and classifying images could be
tedious task especially when they are massive in number (say 10,000) and therefore it will be very useful if we could automate this
ntire process using computer vision.
Some examples of image classification include:
Labeling an x-ray as cancer or not (binary classification).
Classifying a handwritten digit (multiclass classification).
Assigning a name to a photograph of a face (multiclass classification).
The advancements in the field of autonomous driving also serve as a great example of the
use of image classification in the real-world. For example, we can build an image
classification model that recognizes various objects, such as other vehicles, pedestrians,
traffic lights, and signposts on the road.
2 of 25 10/18/2020, 4:25 PM
Image Source: Link
Now that we have a fair idea of what image classification comprises of, let’s start analyzing
the image classification pipeline.
Structure of an Image Classification Task

1. Image Preprocessing - The aim of this process is to improve the image
data(features) by suppressing unwanted distortions and enhancement of some
important image features so that our Computer Vision models can benefit from this
improved data to work on.
2. Detection of an object - Detection refers to the localization of an object which

means the segmentation of the image and identifying the position of the object of
interest.
3. Feature extraction and Training- This is a crucial step wherein statistical or deep
learning methods are used to identify the most interesting patterns of the image,
features that might be unique to a particular class and that will, later on, help the
model to differentiate between different classes. This process where the model
learns the features from the dataset is called model training.
4. Classification of the object - This step categorizes detected objects into

predefined classes by using a suitable classification technique that compares the
image patterns with the target patterns.
Let’s discuss the most crucial step which is image preprocessing, in detail!
Image Pre-processing
Pre-processing is a common name for operations with images at the lowest level of
abstraction — both input and output are intensity images.
Need for Image-Preprocessing

Computers are able to perform computations on numbers and is unable to interpret
images in the way that we do. We have to somehow convert the images to numbers for
the computer to understand.
The aim of pre-processing is an improvement of the image data that suppresses unwilling
distortions or enhances some image features important for further processing.
3 of 25 10/18/2020, 4:25 PM
How computers see an '8'

Image Source: Link
Steps for image pre-processing:
Read image
Resize image
Data Augmentation
Gray scaling of image
Reflection
Gaussian Blurring
Histogram Equalization
Rotation
Translation
Step 1
Reading Image
In this step, we simply store the path to our image dataset into a variable and then we
create a function to load folders containing images into arrays so that computers can deal
with it.
Sample code for reading an image dataset with 2 classes:
4 of 25 10/18/2020, 4:25 PM
Step 2.
Resize image
Some images captured by a camera and fed to our AI algorithm vary in size, therefore, we
should establish a base size for all images fed into our AI algorithms by resizing them.
Sample code for resizing images into 229x229 dimensions:
5 of 25 10/18/2020, 4:25 PM

Step 3
Data Augmentation
Data augmentation is a way of creating new 'data' with different orientations. The benefits
of this are two-fold, the first being the ability to generate 'more data' from limited data and
secondly, it prevents overfitting.
Image Source and Credit: Link
Data Augmentation Techniques:
1. Gray Scaling
The image will be converted to gray scale (range of gray shades from white to
black) the computer will assign each pixel a value based on how dark it is. All the
numbers are put into an array and the computer does computations on that array.
Sample code to convert an RGB(3 channels) image into a Gray scale image:
RGB Image
6 of 25 10/18/2020, 4:25 PM
Grayscale Image:
Images Source: Link
2. Reflection/Flip
You can flip images horizontally and vertically. Some frameworks do not provide
function for vertical flips. But, a vertical flip is equivalent to rotating an image by 180
7 of 25 10/18/2020, 4:25 PM
degrees and then performing a horizontal flip.

Sample Code:
Image showing horizontal reflection

Image Source: Link
3. Gaussian Blurring
Gaussian blur (also known as Gaussian smoothing) is the result of blurring an
image by a Gaussian function. It is a widely used effect in graphics software,
typically to reduce image noise.
Sample Code:
8 of 25 10/18/2020, 4:25 PM
Image with blur radius = 5.1

Image Source:Link
4. Histogram Equalization
Histogram equalization is another image processing technique to increase global
contrast of an image using the image intensity histogram. This method needs no
parameter, but it sometimes results in an unnatural looking image.
Sample Code
9 of 25 10/18/2020, 4:25 PM
10 of 25 10/18/2020, 4:25 PM
Image Credit and Source: Link
5. Rotation
This is yet another image augmentation technique. Rotating an image might not
preserve its original dimensions (depending on what angle you choose to rotate it
with )
Sample Code
11 of 25 10/18/2020, 4:25 PM

The images are rotated by 90 degrees clockwise with respect to the previous one, as we
move from left to right.
Image Source and Credit: Link
6. Translation
Translation just involves moving the image along the X or Y direction (or both).
This method of augmentation is very useful as most objects can be located at
almost anywhere in the image. This forces our feature extractor to look everywhere.
Sample Code
Image Source and Credit:Link
Image Classification Techniques

We will start with some statistical machine learning classifiers like Support Vector Machine
and Decision Tree and then move on to deep learning architectures like Convolutional
Neural Networks.
To support their performance analysis, the results from an Image classification task used
to differentiate lymphoblastic leukemia cells from non-lymphoblastic ones have been
provided. The features have been extracted using a convolutional neural network, which
will also be discussed as one of our classifiers. This is because deep learning models
have achieved state of the art results in the feature extraction process.
12 of 25 10/18/2020, 4:25 PM
Different classifiers are then added on top of this feature extractor to classify images.
1. Support Vector Machines

It is a supervised machine learning algorithm used for both regression and classification
problems.
When used for classification purposes, it separates the classes using a linear boundary.
Image Source: Link
It builds a hyper-plane or a set of hyper-planes in a high dimensional space and good

separation between the two classes is achieved by the hyperplane that has the largest
distance to the nearest training data point of any class.
The real power of this algorithm depends on the kernel function being used.
The most commonly used kernels are:
Linear Kernel
Gaussian Kernel
Polynomial Kernel
Code Snippet:
13 of 25 10/18/2020, 4:25 PM
OpenGenus
This isIQ:
theLearn
baseComputer Science
model/feature extractor using Convolutional Neural Network, using Keras
with Tensorflow backend
Fitting of SVM as a classifier
Accuracy score on test data: 85.68
Link to know more about SVM
2. Decision Trees
It is also a supervised machine learning algorithm, which at its core is the tree data
structure only, using a couple of if/else statements on the features selected.
Decision trees are based on a hierarchical rule-based method and permits the acceptance
and rejection of class labels at each intermediary stage/level.
14 of 25 10/18/2020, 4:25 PM
Image Source: Link
This method consists of 3 parts:
Partitioning the nodes
Finding the terminal nodes
Allocation of the class label to terminal node
Code
Feature Extractor
15 of 25 10/18/2020, 4:25 PM

feat_train = model_feat predict X_train
Decision Tree Classifier
Accuracy on test set: 84.61
Link to know more about Decision Trees
3. K Nearest Neighbor
The k-nearest neighbor is by far the most simple machine learning algorithm.
This algorithm simply relies on the distance between feature vectors and classifies
unknown data points by finding the most common class among the k-closest examples.
16 of 25 10/18/2020, 4:25 PM

Image Source: Link
Here we can see there are two categories of images and that each of the data points
within each respective category are grouped relatively close together in an n-dimensional
space.
In order to apply the k-nearest Neighbor classification, we need to define a distance metric
or similarity function. Common choices include the Euclidean distance and Manhattan
distance
Code
Base Model/feature extractor
KNN classifier
Accuracy on test set: 86.32
Link to explore KNN
17 of 25 10/18/2020, 4:25 PM
4. Artificial Neural Networks

Inspired by the properties of biological neural networks, Artificial Neural Networks are
statistical learning algorithms and are used for a variety of tasks, from relatively simple
classification tasks to computer vision and speech recognition.
ANNs are implemented as a system of interconnected processing elements, called nodes,
which are functionally analogous to biological neurons.The connections between different
nodes have numerical values, called weights, and by altering these values in a systematic
way, the network is eventually able to approximate the desired function.
18 of 25 10/18/2020, 4:25 PM
Images Credit and Source: Link
The hidden layers can be thought of as individual feature detectors, recognizing more and
more complex patterns in the data as it is propagated throughout the network. For
example, if the network is given a task to recognize a face, the first hidden layer might act
as a line detector, the second hidden takes these lines as input and puts them together to
form a nose, the third hidden layer takes the nose and matches it with an eye and so on,
until finally the whole face is constructed. This hierarchy enables the network to eventually
recognize very complex objects.
Code
ANN as feature extractor using softmax classifier
19 of 25 10/18/2020, 4:25 PM
Accuracy on test data: 83.1

This result has been recorded for 100 epochs, and the accuracy improves as the epochs
are further increased.
Link to study ANN in detail
5. Convolutional Neural
Networks
Convolutional neural networks (CNN) is a special architecture of artificial neural networks.
CNNs uses some of its features of visual cortex and have therefore achieved state of the
art results in computer vision tasks.
Let’s cover the use of CNN in more detail.
Convolutional neural networks are comprised of two very simple elements, namely
convolutional layers and pooling layers.
Although simple, there are near-infinite ways to arrange these layers for a given computer
vision problem.
20 of 25 10/18/2020, 4:25 PM
The elements of a convolutional neural network, such as convolutional and pooling layers,
are relatively straightforward to understand.
The challenging part of using convolutional neural networks in practice is how to design
model architectures that best use these simple elements.
Image Source: Link
Code
CNN as feature extractor using softmax classifier
21 of 25 10/18/2020, 4:25 PM
Accuracy on test data with 100 epochs: 87.11

Since this model gave the best result amongst all, it was trained longer and it achieved
91% accuracy with 300 epochs.
Link for more on CNN
Performance evaluation
CLASSIFIER ACCURACY PRECISION RECALL ROC
SVM 85.68% 0.86 0.87 0.86
Decision Trees 84.61% 0.85 0.84 0.82
KNN 86.32% 0.86 0.86 0.88
ANN(for 100 epochs) 83.10% 0.88 0.87 0.88
CNN(for 300 epochs) 91.11% 0.93 0.89 0.97
Conclusion
We can conclude from the performance table, that Convolutional Neural networks deliver
the best results in computer vision tasks.
If you liked the content of this post, do share it with others!
Taru Jain Read More

Read more posts by this author.
OpenGenus Foundation
22 of 25 10/18/2020, 4:25 PM

Tags
Machine Learning (ML) image classification
Start Discussion 0 replies
23 of 25 10/18/2020, 4:25 PM
24 of 25 10/18/2020, 4:25 PM
25 of 25 10/18/2020, 4:25 PM

Machine Learning Basics

Uploaded by

Copyright:

Available Formats

Machine Learning Basics

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Machine Learning Basics

Uploaded by

Copyright:

Available Formats

Basics of Machine Learning Image Classification Techniques https://iq.opengenus.org/basics-of-machine-learning-image-classification...

OpenGenus IQ: Learn Computer Science

Basics of Image Classification Techniques

50+ Linux Commands before joining a Company 💪 [MUST READ]

Machine Learning (ML) image classification

Reading time: 45 minutes

In this post, we will be focusing on different image classification techniques deployed to

After reading this post, you will have an idea about:

What is Image Classification?

The pipeline of an image classification task including data preprocessing

Performance of different Machine Learning techniques on these tasks like:

Artificial Neural Network

Convolutional Neural Network

Support Vector Machines

What is Image Classification?

Some examples of image classification include:

Labeling an x-ray as cancer or not (binary classification).

Classifying a handwritten digit (multiclass classification).

Assigning a name to a photograph of a face (multiclass classification).

OpenGenus IQ: Learn Computer Science

Image Source: Link

Structure of an Image Classification Task

2. Detection of an object - Detection refers to the localization of an object which

4. Classification of the object - This step categorizes detected objects into

Need for Image-Preprocessing

OpenGenus IQ: Learn Computer Science

How computers see an '8'

Steps for image pre-processing:

Gray scaling of image

Sample code for reading an image dataset with 2 classes:

OpenGenus IQ: Learn Computer Science

Sample code for resizing images into 229x229 dimensions:

OpenGenus IQ: Learn Computer Science

Image Source and Credit: Link

Data Augmentation Techniques:

OpenGenus IQ: Learn Computer Science

Images Source: Link

degrees and then performing a horizontal flip.

Image showing horizontal reflection

OpenGenus IQ: Learn Computer Science

Image with blur radius = 5.1

OpenGenus IQ: Learn Computer Science

OpenGenus IQ: Learn Computer Science

Image Credit and Source: Link

OpenGenus IQ: Learn Computer Science

Image Source and Credit:Link

Image Classification Techniques

1. Support Vector Machines

Image Source: Link

It builds a hyper-plane or a set of hyper-planes in a high dimensional space and good

Fitting of SVM as a classifier

Accuracy score on test data: 85.68

Link to know more about SVM

OpenGenus IQ: Learn Computer Science

Image Source: Link

This method consists of 3 parts:

Partitioning the nodes

Finding the terminal nodes