IOSR Journal of Computer Engineering (IOSR-JCE)

e-ISSN: 2278-0661,p-ISSN: 2278-8727, Volume 17, Issue 4, Ver. III (July – Aug. 2015), PP 01-10
www.iosrjournals.org

A Real Time Approach for Indian Road Analysis using Image

Processing and Computer Vision
T.N.R.Kumar*
*Department of Computer Science M.S.Ramaiah Institute of Technology Bangalore.

Abstract: Road image analysis is an important step towards building automated driver guidance system with
the aid of computer vision. Several road accidents and mishaps are reported every year due to driver negligence
and non ideal road conditions like narrow bridge, potholes, and bumps and so on. Little research is carried out
towards the direction of Indian road image analysis. In this work we propose a unique system for real time
processing of Indian Road images and video stream. We proposed techniques for a) Road boundary and lane
detection b) Pothole detection c) Object detection on the road from video and Road sign classification. In this
we test lane detection and object detection on video streams to justify that the techniques can be used in real
time. Pothole detection is performed on the static images and road sign classification is performed on isolated,
already separated road sign images.
Keywords: Image processing, computer vision, hough transform, morphology, clustering, K Nearest Neighbour
(KNN) classifier, Zernike moments.

I. Introduction
Driving support system is one of the most important aspects of Intelligent Transport System (ITS). An
ITS or driver guidance system is automated software that helps driver during driving by assisting with easy
markings of road lanes, detection of object on the road and potholes, and recognition of traffic signals. A camera
is attached to the vehicle which keeps capturing the frames and identifying objects on the frames. Some systems
marks the objects over the frame overlay, some systems generate alarm sound based on the processing logic
written on the software.
With increasing number of vehicles on the road, increasing physical and mental strain of the drivers
chances of accidents are increasing by every day and sophisticated on board systems are needed for driver
assistance. Using intelligent design of hardware like camera, InfraRed, Ultra Sound, sophisticated system can be
designed that can guide the drives, alert them over possible problems on the road and help minimizing the
accidents. When camera is fitted with the vehicle, it continually captures the frames. Such frames contain many
details including the scene of either side of the road. See figure 1 for understanding the concept of “noisy
elements” in the road images.

Fig.1. Frame Captured by Camera with Road part.

A. What are image processing and computer vision

An image processing is any form of signal processing for which the input is an image, such as a
photograph or video frame; the output of image processing may be either an image or a set of characteristics or
parameters related to the image. Most image-processing techniques involve treating the image as a two-
dimensional signal and applying standard signal-processing techniques to it. Image processing usually refers to
digital image processing, but optical and analogy image processing also are possible.Before going to processing
an image, it is converted into a digital form. Digitization includes sampling of image and quantization of
sampled values. After converting the image into bit information, processing is performed.

DOI: 10.9790/0661-17430110 www.iosrjournals.org 1 | Page

 A Real Time Approach for Indian Road Analysis using Image Processing and Computer Vision

Techniques of image processing are:

1. Image Enhancement
2. Image Restoration
3. Image Compression

Computer vision is a field that includes methods for acquiring, processing, analyzing, and
understanding images and, in general, high-dimensional data from the real world in order to produce numerical
or symbolic information, e.g., in the forms of decisions. Computer vision is concerned with the theory behind
artificial systems that extract information from images. The image data can take many forms, such as video
sequences, views from multiple cameras, or multi-dimensional data from a medical scanner. Some functions
which are found in many computer vision systems such that image acquisition, image acquisition, feature
extraction, detection/segmentation.

II. Related Work

Most of the Indian rural and sub urban roads are not ideal for driving due to faded lanes, irregular
potholes, improper and invisible road signs. This has led to many accidents causing loss of lives and severe
damage to vehicles. Many techniques have been proposed in the past to detect these problems using image
processing methods. But there has been little work specifically carried out for detecting such issues of Indian
roads. To address this acute problem, the study is undertaken with the objectives like, to make a survey of
Indian roads, to suggest the method to detect lanes, potholes and road signs and their classification and to
suggest automated driver guidance mechanism. In this regard, Hough Transformation method is adopted for
Lane detection, where as Colour Segmentation and Shape Modelling with Thin Spline Transformation (TPS) is
used with nearest neighbour classifier for road sign detection and Classification. Further, K-means clustering
based algorithm is adopted for pothole detection. Therefore, the attempt is made to invent an automated driver
guidance mechanism to make the driving safe and easier in Indian roads [1].
They have proposed that an application of computer vision methods to traffic flow monitoring and road
traffic analysis. The application is utilizing image-processing and pattern recognition methods designed and
modified to the needs and constrains of road traffic analysis. These methods combined together gives functional
capabilities of the system to monitor the road, to initiate automated vehicle tracking, to measure the speed, and
to recognize number plates of a car. Software developed was applied in and approved with video monitoring
system, based on standard CCTV cameras connected to wide area network computers. Traffic signal lights are
triggered using an inductive loop. At a traffic light, an automobile will be stopped above an inductive coil and
this will signal a green light. Unfortunately, the device does not work with most motorbikes. Using a passive
system such as a camera along with image processing may prove to be more effective at detecting vehicles than
the current system [2].
They have presented that a method for lane detection in image sequences of a camera mounted behind
the windshield of a vehicle. The main idea is to find the features of the lane in consecutive frames which match
a particular geometric model. The geometric model is a parabola, an approximation of a circular curve. By
mapping this model in image space and calculation of gradient image using Sobel operator, the parameters of
the lane can be calculated using a randomized Hough transform and a genetic algorithm. The proposed method
is tested on different road images taken by a video camera from Ghazvin-Rasht road in Iran [3].
They have proposed that a gray scale based processing of the road images for lanes, zebra crossing
detection. The median filtering approach for detecting detail less image and further extracting the parts of roads
like zebra crossing from contrast differential of the median filtered image with the original image is proposed
here. This article describes our three-step algorithm. First step is to segment markings on image. This process is
a difficult task due to shadows on the road and because of deterioration and dirtiness of markings. Several line
extraction techniques are compared in order to determine which of them can be considered to best filter noise on
road image. This result is extended to extract larger markings [4].
. In this he has presented three new methods for color detection and segmentation of road signs. The
images are taken by a digital camera mounted in a car. The RGB images are converted into IHLS colour space,
and new methods are applied to extract the colours of the road signs under consideration. The methods are tested
on hundreds of outdoor images in different light conditions, and they show high robustness [5].
In this he has proposed that autonomous vehicle system is a demanding application for our daily life.
The vehicle requires on-road vehicle detection algorithms. Given the sequence of images, the algorithms need to
find on-road vehicles in real-time. Basically there are two types of on-road vehicle either travelling in the
opposite direction or travelling in the same direction. Due to the distinct features of two types of vehicles,
different approaches are necessary to detect each direction. Here, I suggest the „optical flow‟ to detect the
coming traffics because the coming traffics represent distinct motion. I use „Haar-like feature detection‟ for the

DOI: 10.9790/0661-17430110 www.iosrjournals.org 2 | Page

 A Real Time Approach for Indian Road Analysis using Image Processing and Computer Vision

traffics in the same direction because the traffics represent relatively stable shape (car rear) and little motion. I
verify the detected region with estimating 3D geometry.
They have proposed that a novel system for the automatic detection and recognition of traffic signs.
The proposed system detects candidate regions as maximally stable external regions (MSERs), which offers
robustness to variations in lighting conditions. Recognition is based on a cascade of support vector machine
(SVM) classifiers that were trained using histogram of oriented gradient (HOG) features. The training data are
generated from synthetic template images that are freely available from an online database; thus, real footage
road signs are not required as training data. The proposed system is accurate at high vehicle speeds, operates
under a range of weather conditions, runs at an average speed of 20 frames per second, and recognizes all
classes of ideogram-based (no text) traffic symbols from an online road sign database. Comprehensive
comparative results to illustrate the performance of the system are presented.
In this Road sign detection is important to a robotic vehicle that automatically drives on roads. In this
road signs are detected by means of rules that restrict colour and shape and require signs to appear only in
limited regions in an image [6]. They are then recognized using a template matching method and tracked
through a sequence of images. The method is fast and can easily be modified to include new classes of signs. .
In this describes a fast method for locating and recognizing road signs in a sequence of images.
Compares the work of which proposes a Hough transform based approach for line intersection
detection with that of where vanishing points are considered to be the statistical properties of the road rather
than a property of line intersection. Further it proposes a conjugate translate transformation for detecting the
vanishing lines in a 3-d plane.

III. Proposed Work

A sample block diagram of the proposed work is represented in figure 2 as shown below.

Fig. 2. Generalized block diagram of the proposed work.

In this we maintain database for Indian road images. In that images and video acquired from sedan
from outside the driver‟s window with a digital still camera from a stationary vehicle. Hence, the images give an
estimated view of the road side as seen by the driver. The proposed work considers stationary images to build
the image processing system and leaves the blur removal filtering for future enhancement in the work.
Acquired images are fed to the image processing system. Generalized Hough transformation is applied
over the image to mark the lanes [7], [8]. As the lanes in some of the roads are found to be not clear, a
morphological based image enhancement is adopted. Many works like Road Traffic Analysis and Traffic Sign

DOI: 10.9790/0661-17430110 www.iosrjournals.org 3 | Page

 A Real Time Approach for Indian Road Analysis using Image Processing and Computer Vision

classification were conducted [9], [10]. Here, Zernike moment is applied over the extracted sign image to get
features for road signs. These features are classified with K-Nearest Neighbor Classifier to classify the road
signs if any, present in the scene. For pothole detection K-Means clustering based segmentation is applied over
the scene. Potholes present distinct change in the texture of the road. Hence areas with pot holes are segmented
as independent unit.
In video proposed a moving segmentation algorithm based on image change detection for the system.
A background registration technique is used to construct reliable background information from the video
sequence. Then, each incoming frame is compared with the background image. If the luminance value of a pixel
differs significantly from the background image, the pixel is marked as moving object; otherwise, the pixel is
regarded as background. Finally, a post-processing step is used to remove noise regions and produce a more
smooth shape boundary.

IV. Methodology
A. Hough Transform(HT)
In automated analysis of digital images, a sub problem often arises of detecting simple shapes, such as
straight lines, circles or ellipses. In many cases an edge detector can be used as a pre-processing stage to obtain
image points or image pixels that are on the desired curve in the image space. Due to imperfections in either the
image data or the edge detector, however, there may be missing points or pixels on the desired curves as well as
spatial deviations between the ideal line/circle/ellipse and the noisy edge points as they are obtained from the
edge detector. For these reasons, it is often non-trivial to group the extracted edge features to an appropriate set
of lines, circles or ellipses. The purpose of the HT is to address this problem by making it possible to perform
groupings of edge points into object candidates by performing an explicit voting procedure over a set of
parameterized image objects.
The simplest case of HT is the linear transform for detecting straight lines. In the image space, the
straight line can be described as y = mx + b and can be graphically plotted for each pair of image points (x, y).
In the HT, a main idea is to consider the characteristics of the straight line not as image points (x1, y1), (x2, y2),
etc., but instead, in terms of its parameters, i.e., the slope parameter m and the intercept parameter b. Based on
that fact, the straight line y = mx + b can be represented as a point (b, m) in the parameter space. However, one
faces the problem that vertical lines give rise to unbounded values of the parameters m and b. For computational
reasons, it is therefore better to use a different pair of parameters, denoted and for the lines in the Hough
transform. These two values, taken in conjunction, define a polar coordinate.

Fig.3. Example of Hough Transform.

The parameter represents the distance between the line and the origin, while the angle of the vector from the
origin to this closest point (see Coordinates). Using this parameterization, the equation of the line can be written
as
y= (-cosθ/sinθ) x+(r/sinθ) (1)

Which can be rearranged to r=x cosθ+y sinθ

It is therefore possible to associate with each line of the image a pair (r, θ) which is unique if θϵ(0,π)
and rϵR, or if θϵ(0,2π) and r≥0. The (r,θ) plane is sometimes referred to as half space for the set of straight lines
in two dimensions. This representation makes the HT conceptually very close to the two-dimensional radon
transform.
For an arbitrary point on the image plane with coordinates, e.g., (x0, y0), the lines that go through it are the
pairs (r, θ) with
r(θ)=x(0)cosθ+y(0)sinθ (2)

Where (the distance between the line and the origin) is determined by θ.

DOI: 10.9790/0661-17430110 www.iosrjournals.org 4 | Page

 A Real Time Approach for Indian Road Analysis using Image Processing and Computer Vision

This corresponds to a sinusoidal curve in the (r, θ) plane, which is unique to that point. If the curves
corresponding to two points are superimposed, the location (in the Hough space) where they cross corresponds
to a line (in the original image space) that passes through both points. More generally, a set of points that form a
straight line will produce sinusoids which cross at the parameters for that line. Thus, the problem of detecting
collinear points can be converted to the problem of finding concurrent curves.
The HT algorithm uses an array, called an accumulator, to detect the existence of a line y = mx + b.
The dimension of the accumulator is equal to the number of unknown parameters of the HT problem. For
example, the linear Hough transform problem has two unknown parameters: the pair (m, b) or the pair (r, θ).
The two dimensions of the accumulator array would correspond to quantized values for (r, θ). For each pixel
and its neighborhood, the HT algorithm determines if there is enough evidence of an edge at that pixel. If so, it
will calculate the parameters of that line, and then look for the accumulator's bin that the parameters fall into,
and increase the value of that bin.

B. Zernike Moments(ZM)
ZM of order n and repetition m is defined as follows:
n+1
Znm= x2+y2≤1
Vnm ρ, θ f x, y dx dy (3)
π

Where:
f(x,y) is the image intensity at (x,y) in Cartesian coordinates,
Vnm(ρ,θ) is a complex of Vnm(ρ, θ)=Rnm(ρ)e−jm θ in polar coordinates (ρ,θ) and j= √(-1)
N≥0 and n- |m| is even positive integer.
The polar coordinate (ρ,θ) in the image domain are related
To the cartesian coordinates (x,y) as x=ρ cos(θ) and y=ρ sin(θ).
Rnm (ρ) is a radial defined as follows:
n +m
2 (−1)s [ n−s !ρn −2n
Rnm (ρ)= s=0 s! n + m −s n − m −s (4)
2 2

The first six orthogonal radial polynomials are:

R00 () = 1 R11 () = ρ
R20 () = 2ρ2 -1 R22 () = ρ2 (5)

R31 () = 3ρ3 – 2 ρ R33 () =ρ3

The discrete approximation of Equation (1) is given as:
4 n+1 N−1 N−1 −jm θkl
Z=
N−1 π 2 k=0 l−1 f k, l R nm ρk, l e
(6)
0≤ρk, l ≤ 1

Where the discrete polar coordinates:

yl
ρk,l == xk2 + yl2 ; θkl = arctan
xk
(7)

Are transformed by
√2 −1 √2 −1
xk = k+ ; yl = l+ (8)
N−1 √2 N−1 √2
For k=0… N -1 and l =0… N-1.

To calculate the ZM of an image f(x, y), the image is first mapped onto the unit disk using polar coordinates,
where the center of the image is the origin of the unit disk. Pixels falling outside the unit disk are not used in the
calculation.
Because Zmn is complex, I use the ZM modules Zmn as the features of shape in the recognition of patterns.

DOI: 10.9790/0661-17430110 www.iosrjournals.org 5 | Page

 A Real Time Approach for Indian Road Analysis using Image Processing and Computer Vision

Fig.4. Example of Zernike Moments.

The magnitude of ZM has rotational invariance property. An image can be better described by a small
set of its Zernike moments than any other type of moments such as geometric moments, Legendre moments, and
complex moments in terms of mean-square error. Zernike moments do not have the properties of translation
invariance and scaling invariance. The way to achieve such invariance is image translation and image
normalization before calculation of ZM.

C. K-Means Clustering
1. K-Means Clustering Overview:
K-Means clustering generates a specific number of disjoint, flat (non-hierarchical) clusters. It is well suited to
generating globular clusters. The K-Means method is numerical, unsupervised, non-deterministic and iterative.

2. K-Means Algorithm Properties

There are always K clusters. There is always at least one item in each cluster. The clusters are non-hierarchical
and they do not overlap. Every member of a cluster is closer to its cluster than any other cluster because
closeness does not always involve the centre of clusters.

3. The K-Means Algorithm Process:

The dataset is partitioned into K clusters and the data points are randomly assigned to the clusters
resulting in clusters that have. Roughly the same number of data points. For each data point calculate the
distance (Mahalanobis or Euclidean) from the data point to each cluster. If the data point is closest to its own
cluster, leave it where it is. If the data point is not closest to its own cluster, move it into the closest cluster.
Repeat the above step until a complete pass through all the data points‟ results in no data point moving from one
cluster to another. At this point the clusters are stable and the clustering process ends. The choice of initial
partition can greatly affect the final clusters that result, in terms of inter-cluster and intra-cluster distances and
cohesion.

D. K-Nearest Neighbour:
It‟s a lazy learning algorithm. Defer the decision to generalize beyond the training example till a new
query is encountered. Whenever we have a new point to classify, we find its K nearest neighbours from the
training data. Distance is calculated using one of the following measures such as Mahalanobis or Euclidean
distance.

Algorithm of KNN: Determine parameter k. Calculate the distance between query instance and the entire
training instance. Sort the distance and determine KNN.
Both Mahalanobis and Euclidean distances are described below clearly.

Mahalanobis Distance: Mahalanobis Distance is a very useful way of determining the”similarity” of a set of
values from an”unknown”: sample to a set of values measured from a collection of”known” samples. Superior
to Euclidean distance because it takes distribution of the points (correlations) into account. Traditionally to
classify observations into different groups. It takes into account not only the average value but also its variance
and the covariance of the variables measured. It compensates for interactions (covariance) between variables It
is dimensionless.

DOI: 10.9790/0661-17430110 www.iosrjournals.org 6 | Page

 A Real Time Approach for Indian Road Analysis using Image Processing and Computer Vision

The formula used to calculate Mahalanobis distance is given below.

Dt(x) = (x – Ci) * Inverse(S) * (x – Ci)
Here X is a data point in the 3-D RGB space,
Ci is the center of a cluster
S is the covariance matrix of the data points in the 3-D RGB space
Inverse(S) is the inverse of covariance matrix S.

Euclidean Distance: The Euclidean distance is the straight-line distance between two pixels.
Euclidean distance = √ ((x1 - x2)² + (y1 - y2)²),
Where (x1, y1) & (x2, y2) are two pixel points or two data points.

V. Results

Fig.6. Sample database for lanes.

Fig.7. Sample database for potholes.

DOI: 10.9790/0661-17430110 www.iosrjournals.org 7 | Page

 A Real Time Approach for Indian Road Analysis using Image Processing and Computer Vision

Fig.8. Sample database for road signs.

Fig.9. Sample result for lane using hough transform in image.

Fig.10. Sample result for lane using morphology in image.

Fig.11. Sample result for lane using both hough transform, morphology in video.

DOI: 10.9790/0661-17430110 www.iosrjournals.org 8 | Page

 A Real Time Approach for Indian Road Analysis using Image Processing and Computer Vision

Fig.12. Sample result for object detection in video.

Fig.13. Sample result for potholes detection in image.

Fig.14.Sample result for trained road signs.

Fig..A.

DOI: 10.9790/0661-17430110 www.iosrjournals.org 9 | Page

 A Real Time Approach for Indian Road Analysis using Image Processing and Computer Vision

Fig.B.
Fig.15. Sample result for detection of road signs as shown in fig.A and fig.B.

VI. Conclusion
Various image processing techniques are proposed over the years for detection and classification of
various road objects like lanes, Zebra Crossing; pot Holes, Bumps, and Vanishing points and so on. Different
techniques use different features for detection of such features. The goal of the work has been to develop a fast
an efficient technique for detecting the Lanes, potholes, and objects of roads in Indian road images and video
frames. Indian rural and sub urban roads profile in a colour model is inconsistent hence making it very
challenging task to extract the road part. The other criteria considered are fast detection of the same. Therefore
in this work a simplistic approach for the problem is proposed which is purely based on image processing in
colour domain and without any significant transformation like Fourier transform to speed up the detection.
Because image is taken from moving vehicle, a certain blurring effect is usual in such image. But due to straight
orientation of the camera such effects is minimized and hence does not require any specific deploring algorithm.
The results show promising efficiency in detection. Combining the results in colour domain image processing
and gray scale processing of the images for Lane and Object detection detects the desired entities with utmost
efficiency.

References
[1]. E. Atkociunas, R. Blake, A. Juozapavicius, M. Kazimianec, “Image Processing in Road Traffic Analysis, Nonlinear Analysis”,
Modelling and Control, 2005
[2]. Jack Greenhalgh and Majid Mirmehdi, Senior Member, IEEE,” Real-Time Detection and Recognition of Road Traffic Signs”
Manuscript received January 13, 2012.
[3]. Ajit Danti, jyoti Y, Kulkarni and P.S. Hiremath “An Image Processing Approach to Detect Lanes, Pot Holes and Recognized Road
Signs in Indian Roads” International Journal of Modeling and Optimizaton, Vol.2 No.6, December 2012
[4]. L. Quan and R. Mohr. “Determining perspective structures using hierarchical Hough transform”, Pattern Recognition Letters, 1989.
[5]. R.T. Collins and R.S. Weiss, “Vanishing point calculation as a statistical inference on the unit sphere”, In Proceedings of the Third
International Conference on Computer Vision, pages 400-403, Osaka, Japan, December 1990.
[6]. Hasan Fleyeh, “color detection and segmentation for road and traffic signs”, proceedings of the 2004 ieee Conference on
cybernetics and intelligent systems singapore, december, 2004.
[7]. F. Samadzadegan a, A. Sarafraz, M. Tabibi, “Automatic Lane Detection in Image Sequences for Vision-Based Navigation Purpose”.
[8]. Jaesik Choi, “Realtime On-Road Vehicle Detection with Optical Flows and Haar-like feature detector”, 2002
[9]. Michael Shneier, “Road Sign Detection and Recognition”, June 2005.
[10]. Gavrilovic Thomas, Ninot Jerˆome and Smadja Laurent, “frequency filtering and connected components characterization for zebra-
crossing and hatched markings detection”, September-1-3, 2010.

DOI: 10.9790/0661-17430110 www.iosrjournals.org 10 | Page