1

Convolution operation on image

processing with the help of matrix
theory

Project work by :-

Puja Saha

—1—
2

-: CONTENT :-

Topic :- Page No :-

1. Certificate --------------------------------------------- (4)

2. Abstract ----------------------------------------------- (5)

3. Introduction ------------------------------------------ (6)

4. Background ------------------------------------------ (7)

5. What is Kernel -------------------------------------- (8)

6. What is convolution ------------------------------ (8)

7. Concept of mask -----------------------------------(9)

8. Robinson compass mask ---------------------- (12)

9. Krisch Compass Mask -------------------------- (18)

—2—
3

10. Concept of Blurring ----------------------------- (24)

11. Concept of edge detection ------------------- (26)

12. Sharpening -------------------------------------- (28)

13. High Pass vs Low Pass Filters -------------- (30)

14. Application --------------------------------------(33)

15. Conclusion ---------------------------------------(36)

16. Acknowledgement --------------------------- (37)

17. Reference -------------------------------------- (38)

—3—
4

-:CERTIFICATE:-

This is certify that the project work entitled

“Convolution operation on image processing with
the help of matrix theory” is the investigatory project
work in mathematics, successfully completed by Puja
Saha ,student of B.Sc. Semester:- (VI) (Department of
Mathematics), Abhedananda Mahavidyalaya, under
the University Of Burdwan, bearing Examination Roll
No:--210330100031,Registration No:--202101026516
of 2021-22, under the guidance of Sudipta Senapati,
for the partial fulfilment of requirements for the course
completion in pursuance.

Date :- _______________________
Sudipta Senapati
Department of Mathematics
Abhedananda Mahavidyalaya
Sainthia , Birbhum

—4—
5

-:ABSTRACT:-

Convolution of two functions is an important mathematical

operation that found heavy application in signal
processing. In computer graphics and image processing fields,
we usually work with discrete functions (e.g. an image)
and apply a discrete form of the convolution to remove high
frequency noise, sharpen details, detect edges, or otherwise
modulate the frequency domain of the image. In this project,
we will discuss convolution of image processing. A general 2D
convolution has a high bandwidth requirement as the final
value of a given pixel is determined by several neighboring
pixels. Since memory bandwidth is usually the main limiting
factor of algorithm’s performance, our optimization techniques
will focus on minimizing global memory accesses during the
computations .

In this project at first the concept of convolution and image

processing are given. Also some important concepts and
methods are discussed. At last some useful applications of this
topic (convolution) in image processing and others are given.

—5—
6

Convolution operation on image

processing with the help of matrix theory

Introduction :-

Nowadays, image processing is among rapidly growing

technologies. It forms core research area within engineering
and computer science disciplines too. There are two types of
methods used for image processing namely, analogue and
digital image processing. Analogue image processing can be
used for the hard copies like printouts and photographs. Digital
image processing techniques help in manipulation of the digital
images by using computers.

In this project work we will talk about convolution operation on

image processing with the help of matrix theory. Convolution is
the most important topic in the field of image processing, a
convolution is an operation with which we can merge two
arrays by multiplying them, these arrays could be of different
sizes, the only condition, however, is that the dimensions
should be the same for both arrays. Convolution is the natural
mathematical operation performed by a linear and time-
invariant (LTI) system over its input signal. It doesn’t matter if it
is a 1-D, 2-D, …, N-dimensional system: it only has to be LTI. So
when you use convolution to process an image, it only means
you are processing the image with a LTI filter. There are many
varieties of LTI filters to perform diverse operations on the
images: to blur them, to sharp them, to enhance horizontal
edges, to enhance edges with a 45 degree tilt, etc… There are
also many nonlinear filters commonly applied in images.

—6—
7

perhaps the most popular one is the median filter used for
removing “salt-and-pepper” noise from images.

Convolution provides a way of `multiplying together' two arrays

of numbers, generally of different sizes, but of the same
dimensionality, to produce a third array of numbers of the same
dimensionality. This can be used in image processing to
implement operators whose output pixel values are simple
linear combinations of certain input pixel values.

Background :-

As the earlier section states, image processing is an important

subset of DSP that involves analyzing the characteristics of
image signals or modifying an image in some way to enhance or
remove certain features. Many different approaches exist to
achieve these ends; one approach involves the application of
what is called a filter. In analog, image processing and filtering is
achieved through electrical means. For instance, in CRT displays
(e.g., old TVs and computer monitors), image characteristics
such as brightness are controlled using varying voltage levels,
with a continuous range of values. In contrast, digital image
processing and filtering is done on computers, with numerical
representations of signals to which mathematical operations
can be applied. A key difference between analog and digital
image processing is that digital signals are quantize in both
length and level, that is, the different values a digital signal can
take are a finite, as is the length of the signal. In contrast,
analog signals are continuous with respect to level and length.
Since the mathematics of certain image processing ideas is very
similar, concepts such as convolution will be introduced in both

—7—
8

continuous and discrete form. Digital representations of images

and the concept of convolution kernels will also be presented.

What is Kernel Matrices :-

In image processing, a kernel, convolution matrix, or mask is a

small matrix used for blurring, sharpening, embossing, edge
detection, and more. This is accomplished by doing a
convolution between the kernel and an image. Or more simply,
when each pixel in the output image is a function of the nearby
pixels (including itself) in the input image, the kernel is that
function.

What is Convolution :-

—8—
9

Convolution is the process of adding each element of the image

to its local neighbors, weighted by the kernel. This is related to
a form of mathematical convolution. The matrix operation
being performed—convolution—is not traditional matrix
multiplication, despite being similarly denoted by *.
The general expression of a convolution is

Whare g(x,y) is filtered image, f(x,y) is orginal image and w is

the filter kernel. Every element of the filter kernel is considered
by -a ≤ dx ≤ a and -b ≤ dy ≤ b.

Concept of mask :-

The concept of a mask is also known as spatial filtering. Mask is

a type of filter which performs operation directly on the image.
The filter mask is also known as convolution mask.

To apply a mask on an image, filter mask is moved point to

point on the image. In the original image, at each point(X, Y),
filter is calculated by using a predefined relationship.

There are two types of filters:

Linear filter
Frequency domain filter

1) Linear filter :
A linear filter is the simplest filter. In linear filter, each pixel is
replaced by the average of these pixel values. The entire linear

—9—
10

filter works in the same way except when the weighted average
is formed instead of a simple average.

The formula for a linear filter

Example :

2) Frequency Domain Filter :

In frequency domain filter, an image is represented as the sum
of many sine waves which have different frequencies,
amplitudes and directions. The parameter of sine waves is
referred to as Fourier coefficients.

Reasons for using this approach:

— 10 —
11

For getting extra insight.

Linear filters can also in the frequency domain use Fast Fourier
Transform (FFT)

Filters are used for 2 purposes:-

1) Blurring and noise reduction.

2) Edge detection and sharpness.

1)Blurring and noise reduction :

Filters can be used for blurring as well as noise reduction from
an image. Blurring is used to remove small details from an
image. Noise reduction can also be done with the help of
blurring.

Commonly used masks for blurring are:

Box filter
Weighted average filter

2) Edge Detection and sharpness :

Filters can be used for edge detection and sharpness. To

increase the sharpness of an image, edge detection is used.

— 11 —
12

Robinson compass mask :-

A Robinson compass mask is a type of mask which is used for

edge detection. It has eight orientations. It is also known as the
direction mask. It extracts the edges with respect to its
direction. Following are its eight orientations:

North
North West
West
South West
South
South East
East
North East

It does not have any fixed mask. We have to rotate the mask to
find an edge in any one direction above mention. Masks are
rotated from zero columns on the basis of direction.

— 12 —
13

Following is a mask which is in north direction and the rotated:

North

North West

West

South West

— 13 —
14

South

South East

East

North East

Example of Robinson compass mask applied to an image:

— 14 —
15

North

North West

— 15 —
16

West

South West

South

— 16 —
17

South East

East

North East

— 17 —
18

Krisch Compass Mask :-

Kirsch compass mask is used for finding edges. It is almost the

same as Robinson compass mask. It also has 8 direction
compass. The main difference between Robinson and kirsch
compass masks is that in kirsch we can change the mask
according to the requirements.

Following are its eight orientations:

North
North West
West
South West
South
South East
East
North East

Standard size is taken and then it is rotated in all directions.

— 18 —
19

Following is a mask which is in north direction and the rotated:

North

North West

West

South West

— 19 —
20

South

South East

East

North East

— 20 —
21

Example of Kirsch compass mask applied to an image :

Original image

The maximum gradient in the 8 directions

— 21 —
22

North

North West

West

— 22 —
23

South West

South

South East

— 23 —
24

East

North East

Concept of Blurring :-

In Digital image processing, blurring is used to make an image

smooth in which edges are not observed. Suppose, we have an
image in which there are 508*340=172,720 pixels. If we want to
blur this image, then all the 172,720-pixel values will be
changed by using the blurring technique and convolution as we
know that in an image, there are 8 pixels around it. When these

— 24 —
25

pixels are combined as 8+1=9, a 3x3 matrix is formed. In our

image, there are 172,720 pixels, so there will be 172,720
matrices of 3x3 orders. By applying convolution in all 172,720
matrices with a common matrix called the kernel. The kernel is
a special matrix, it changes the pixels using convolution to make
an image blur. A low pass filter is used for blurring as it allows
the low frequency to allow and stop the high frequency. The
term frequency means to change the value of the pixel.

Blurring vs. Zooming :

When an image is blurred, and it is zoomed, the zooming factor

is increased. This is because when we zoom an image, many
new pixels are added and when an image is blurred the pixels of
the normal image and blurred image remains the same.

Original image

— 25 —
26

Blurred image

Concept of Edge Detection :-

We have discussed briefly about edge detection in our tutorial

of introduction to masks. We will formally discuss edge
detection here.

What are edges ?

We can also say that sudden changes of discontinuities in an

image are called as edges. Significant transitions in an image are
called as edges.

Types of edges :
Generally edges are of three types:

1) Horizontal edges
2) Vertical Edges
3) Diagonal Edges

— 26 —
27

Why detect edges ?

Most of the shape information of an image is enclosed in edges.

So first we detect these edges in an image and by using these
filters and then by enhancing those areas of image which
contains edges, sharpness of the image will increase and image
will become clearer.

Here are some of the masks for edge detection that we will
discuss in the upcoming tutorials.

Prewitt Operator
Sobel Operator
Robinson Compass Masks
Krisch Compass Masks
Laplacian Operator.

Above mentioned all the filters are Linear filters or smoothing

filters.

Prewitt Operator :

Prewitt operator is used for detecting edges horizontally and

vertically.

Sobel Operator :

The sobel operator is very similar to Prewitt operator. It is also a

derivate mask and is used for edge detection. It also calculates
edges in both horizontal and vertical direction.

— 27 —
28

Robinson Compass Masks :

This operator is also known as direction mask. In this operator

we take one mask and rotate it in all the 8 compass major
directions to calculate edges of each direction.

Kirsch Compass Masks :

Kirsch Compass Mask is also a derivative mask which is used for

finding edges. Kirsch mask is also used for calculating edges in
all the directions.

Laplacian Operator :

Laplacian Operator is also a derivative operator which is used to

find edges in an image. Laplacian is a second order derivative
mask. It can be further divided into positive laplacian and
negative laplacian.

All these masks find edges. Some find horizontally and vertically,
some find in one direction only and some find in all the
directions. The next concept that comes after this is sharpening
which can be done once the edges are extracted from the
image.

Sharpening :-

Sharpening is opposite to the blurring. In blurring, we reduce

the edge content and in Sharpening, we increase the edge
content. So in order to increase the edge content in an image,
we have to find edges first.

— 28 —
29

Edges can be find by one of the any method described above by

using any operator. After finding edges, we will add those edges
on an image and thus the image would have more edges, and it
would look sharpen.

This is one way of sharpening an image.

The sharpen image is shown below -

Original image

Sharpen image

— 29 —
30

High Pass vs Low Pass Filters :-

Low pass filter (smoothing) :

A low pass filter is used to pass low-frequency signals. The

strength of the signal is reduced and frequencies which are
passed is higher than the cut-off frequency. The amount of
strength reduced for each frequency depends on the design of
the filter. Smoothing is low pass operation in the frequency
domain.

— 30 —
31

High pass filters (sharpening) :

A highpass filter is used for passing high frequencies but the

strength of the frequency is lower as compared to cut off
frequency. Sharpening is a highpass operation in the frequency
domain. As lowpass filter, it also has standard forms such as
Ideal highpass filter, Butterworth highpass filter, Gaussian
highpass filter.

— 31 —
32

— 32 —
33

Application of convolution :-

Convolution and related operations are found in many

applications in science, engineering and mathematics.

In image processing -

Convolution in Image Processing

In image processing, convolutional filtering can be used to
implement algorithms such as edge detection, image
sharpening, and image blurring.
This is done by selecting the appropriate kernel.

In optics, an out-of-focus photograph is a convolution of the

sharp image with a lens function. The photographic term for
this is bokeh.

In image processing applications such as adding blurring.

In digital data processing -

In analytical chemistry, Savitzky–Golay smoothing filters are

used for the analysis of spectroscopic data. They can improve
signal-to-noise ratio with minimal distortion of the spectra.

In statistics, a weighted moving average is a convolution.

— 33 —
34

In acoustics, rverberation is the convolution of the original

sound with echoes from objects surrounding the sound source.

In digital signal processing, convolution is used to map the

impulse response of a real room on a digital audio signal.

In electronic music convolution is the imposition of a spectral or

rhythmic structure on a sound. Often this envelope or structure
is taken from another sound. The convolution of two signals is
the filtering of one through the other.

In electrical engineering -

The convolution of one function (the input signal) with a second

function (the impulse response) gives the output of a linear
time-invariant system (LTI). At any given moment, the output is
an accumulated effect of all the prior values of the input
function, with the most recent values typically having the most
influence (expressed as a multiplicative factor). The impulse
response function provides that factor as a function of the
elapsed time since each input value occurred.

In physics -

wherever there is a linear system with a "superposition

principle", a convolution operation makes an appearance. For
instance, in spectroscopy line broadening due to the Doppler
effect on its own gives a Gaussian spectral line shape and

— 34 —
35

collision broadening alone gives a Lorentzian line shape. When

both effects are operative, the line shape is a convolution of
Gaussian and Lorentzian, a Voigt function.

In probability theory -

The probability distribution of the sum of two independent

random variables is the convolution of their individual
distributions.

In radiotherapy treatment planning systems -

Most part of all modern codes of calculation applies a

convolution-superposition algorithm.

— 35 —
36

-: CONCLUSION:-

In the field of image processing, convolution and

kernels play a very important role, thus, having a good
knowledge about them helps in several operations
which could be performed over an image or a video.

There are several other techniques and tools under

image processing and machine learning which help in
the transformation of image, these transformations may
also help in restoration of some torned images as well,
with so many use cases, this image processing should be
considered as a great step in the advancement of the
field.

— 36 —
37

-:ACKNOWLEDGEMENT:-

I would like to warmly acknowledge and express

my deep sense of gratitude and indebtedness to
my guide Sudipta Senapati, Department of
Mathematics, Abhedananda Mahavidyalaya,
whose keep guidance, valuable suggestions and
instruction, constant encouragement has served as
the major contribution towards the completion of
this project.

Also, I would like to thank all of my teacher of

department of mathematics, Abhedananda
Mahavidyalaya, for allowing me to work on this
project and their co-operation.

Last but not the least I would like to thank my

parents, brother and friends for their blessings and
inspiration.

....................................
Signature

— 37 —
38

-: REFERENCE:-

Books :
1) Digital Signal Processing: Efficient Convolution and Fourier
Tranform Techniques
-------------------- by - Douglas G. Myers

2) FPGA implementation of Convolution algorithm for Image

Processing
----------------------by - Mohassin Ahmad

Websites :

1) https://en.m.wikipedia.org/wiki/Convolution

2) https://en.wikipedia.org/wiki/Kernel_(image_processing)

3) https://en.wikipedia.org/wiki/Gaussian_blur

4) https://medium.com/@bdhuma/6-basic-things-to-know-
about-convolution-daef5e1bc411

5)
https://www.tutorialspoint.com/dip/concept_of_convolution.h
tm

— 38 —