See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/333031633

Intelligent Video Surveillance based on Object Tracking

Conference Paper · November 2018

DOI: 10.1109/ICRAIE.2018.8710421

CITATIONS READS
8 606

3 authors, including:

Devendra Somwanshi Mahesh Bundele

Poornima Group of Colleges Poornima College of Engineering
54 PUBLICATIONS   171 CITATIONS    28 PUBLICATIONS   129 CITATIONS   

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Devendra Somwanshi on 12 May 2019.

The user has requested enhancement of the downloaded file.

 3rd International Conference and Workshops on Recent Advances and Innovations in Engineering, 22-25 November 2018
(IEEE Conference Record # 43534)

Intelligent Video Surveillance based on Object

Tracking
Garima Mathur Devendra Somwanshi Dr. Mahesh M. Bundele
Dept. of EEE, School of Engineering & Dept. of EEE, School of Engineering & Director, Poornima College of
Technology, Technology Engineering
Poornima University, Jaipur, India Poornima University, Jaipur, India Jaipur, India

Abstract –Tracking moving objects as along with their movement II. LITERATURE REVIEW
through video sequences is one of the most basic and most crucial
tasks in computer vision. This serves as the foundation for
numerous higher-level automated applications in numerous
Object Tracking have been studied as the key segment of
domains, including surveillance, augmented reality and motion intelligent video analysis with various applications for
capture in moving object detection. Object tracking is key suspicious activity detection viz loitering[4], unattended
element of an IVS system which can further be modeled for baggage[6], multiple moving object detection[9], intruder
many suspicious activity detection systems. There are many detection [2][7], traffic and transport scenario [3][10] etc.
approaches and proposed algorithms for object tracking, out of Various algorithms have been proposed by researchers for
which mean shift algorithm is the basis of implementation of IVS background subtraction, to detect a tracking object like
in this paper. The work proposed in this paper was defined to Gaussian Mixture Model to detect human fall [5][8], blob
meet following objectives: to create a video sequence for moving based tracking, 3-frame-difference method & timeliness
object tracking; to design an experimental set up for moving background for missing and unattended object [6], Frame
object detection; and, to design and implement moving object Differencing Method, Adaptive parametric statistical
tracking algorithm, The proposed algorithm was implemented on background subtraction method for intrusion detection [7]etc.
a captured video sequence. Object was detected first according to Application of probability density function in identifying the
the image input, and then tracked in subsequent frames. The object to be tracked by deploying mean shift algorithm is
experimental implementation could perform the object tracking proposed which could fetch satisfactory results in object
without missing any frame and could successfully overlay tracking. Out of various approaches, this algorithm stands out
bounding box. It could successfully generate an image sequence robust if we have a distribution points, and peaks in the
after the complete implementation of Mean Shift Algorithm. The
density function could be determined [11] [13]. The major
performance of algorithm was checked by successfully tracking
gaps in object tracking implementation are information loss
the user defined object and performing the overlay function in
the identified object. while conversion of 3D to 2D scene; image resolution
limitation, noise while capturing and processing data,
KeyWords- IVS Intelligent Video Surveillance, Object Tracking complex motion- exhibiting variable velocity, non-rigid
motion, merging with background, less contrast, feature
I. INTRODUCTION changes, variations in scene illumination, partial or full
occlusions etc[14]. It may be necessary to design an
Numerous intelligent techniques have evolved in recent years
application based system to operate under real-time
and extensive research and development to equip artificial
constraints and considerations, obviating the need of
intelligence in such recognition systems have attained novel
effective but slow tracking techniques.
approaches. Neural Network, Support Vector Machine,
Genetic algorithm, Fuzzy technology etc. have all been III. DESIGN METHODOLOGY
researched, experimented and implemented in this emerging
area and have come up with encouraging results. Object
tracking is a threshold step and significant section of computer To design an object tracking system for intelligent video
vision and intelligent video surveillance. Object detection surveillance, a traffic scenario was considered for
involves locating objects in subsequent frames of a video exper9imental set up for its varied applications in evolving
sequence. Object tracking is the process of locating a moving intelligent vehicle systems. To implement Object tracking, a
object or multiple objects over a period of time using a video is captured and converted in .avi format. This file is read
camera. Specifically, tracking involves the estimation of the and information like frame number, data type etc. are
trajectory or path of a moving object in the image plane as it extracted. The frame from video with respect to frame number
moves around a scene. Deducing the suspicious behavior of obtained is read. Image of object to be tracked is obtained
such target objects after detection and tracking is the main from read video sequence. Mean shift Algorithm is used to
objective of setting up an IVS systems. The accessibility and track user defined object by iteratively updating location of
availability of high-powered machines / computers, high object. Region of Image (ROI) to be tracked is extracted to
quality and inexpensive video cameras, and rising need for track the object, and each frame is checked for presence of
automated video analysis has generated a great deal of interest object. A bounding box is laid on the found object, and all
in object tracking algorithms. such frames are compiled to generate image sequence. The

• 978-1-5386-4525-3/18/$31.00 ©2018 IEEE

architectural diagram of proposed work is given in Figure 1 STEP 7: Add Tracking Object information from ROI stored in
below. object tracking session.
STEP 8: Check for each frame if object is found
STEP 9: Draw Bounding Box on found image
STEP 10: Generate Image Sequence

For Loop and Case structure Logics are implemented to

define Object Detection and then tracking. For Loop executed
the code for the no. of frames. In this traffic scenario case, no.
of frames were 115. The For loop executes from 0 till 114. At
first iteration, the program enters Case 0where it adds the
tracking object, and calculates the Region of Image (ROI) of
input frame, in which it has to track the object. From Second
iteration till rest all other iterations, Default Case is operated,
wherein every iteration, object to be tracked is detected using
mean shift method, for each frame, and bounding box is
overlaid on it. Each frame with tracked object is stored in a
Fig. 1: architectural diagram of proposed work shift register. Output is then obtained, after clubbing all the
frames sequentially. When the code is run, Front Panel
IV. EXPERIMENTAL SET UP displays the tracked object with a bounding box, which moves
along with it.The detailed flow chart of the code is as:
Nikon D3100 camera was used for capturing the video Start
sequence from a moving traffic scenario. The captured video
was first converted into .avi file for further processing and Input Video Sequence
data compatibility. For implementation of proposed work,
following data was used as input: Read AVI file [Path / Frame rate]
a. Video Sequence: Captured video of 248 MB in .mov format
was recorded at 480p (640 x 424) at 24 fps in SD Mode. Extract Video Parameters (AVI Refnum & No. of Frames)

b. Processed Video Sequence: Input video was used after

Counter = No. of Frames
processing of original video and was converted to 15.4 MB
Video of 3.8 sec in .avi format, with 30 fps Iteration Count (i) = 0
c. Tracking Image: RGB U 32 image of 5.6 KB, 640 x 240
pixels was input as the image to be tracked. Read Frame Specified by AVI Refnum
Experimental analysis was carried out for following two
scenarios:
Scenario 1: Object Detection; learning object to track; Case-1 Y
Scenario 2: Object Tracking after detection I=0?
Mean Shift algorithm was applied for object detection and
N
tracking, as it is one of the most robust and computationally Draw ROI
efficient algorithm, which has been found adequate for its
gradient based histogram method. Case
Default Add Tracking Object

V. PROCESS FLOW AND IMPLEMENTATION Track Object from image to

image
Input Video is converted in required file format, to be read by Y
Error
the code created in LabVIEW 2017. Video information like About

number of frames, type etc. are extracted and for each of these N
frames, the entire code runs to track the object in each Object
Found? N Increment I by 1
subsequent frame.
The Mean shift Algorithm was implemented for real time Y
object tracking. The algorithm of object tracking implemented Draw Bounding Box
in this process is shown below:
Overlay on Object Tracked

STEP 1: Capture Video and convert in .AVI format

Output Image Sequence
STEP 2: Read AVI file and extract information Store Image in Shift Register
STEP 3: Read AVI frame from video with respect to frame
number Increment I by 1 End of Tracking Session
STEP 4: Read Image from AVI file which is specified from
read video sequence Y N
I<Counter
STEP 5: Apply MEAN SHIFT Algorithm to track user defined
object by iteratively updating location of object
STEP 6: Extract Region of Image (ROI) to be tracked
 VI. RESULTS Case Default finds tracking object in each frame and overlays
A probe was pinned at the Output stage of the Case Default, a bounding box on it
when the object to be tracked was already learned by the VI
code. X, Y coordinates for the object tracked, and the overlaid
bounding box were calculated and tabulated. Liner relation
was observed between the X coordinates of all the positions of
object in every frame, and the X coordinate of bounding box.
However, Y coordinate was the same as the object was
continuously on same track. Minimum Score is the matching
score of the object tracked and the input object, on scale of 0
to 1000, where 1000 means the perfect match. [Table 1:
Tracking Coordinates of Identified Object]
The step by step process and screen captures are shown below:
A. Input Tracking Image
Image sent for Object Tracking for finding specified object
E. Tracking of object and bounding box coordinates

Tracking x,y coordinates of found object and bounding box.

B. Input Video Frame

Video frame for calculation of ROI before object tracking and
detection

F. Displays Final Output

C. ROI Calculation

Case 0 calls in ROI (Region of Image) Calculator for the input

frame and adds tracking object for identifying object to learn

D. Object Detection and Overlaying

 TABLE I.: TRACKING COORDINATES OF IDENTIFIED OBJECT

Location of Bounding Box Location of Bounding Box

Sr. Fram Min. Frame Min
Object coordinate Sr.N Object coordinate
N e No score No score
X Y X Y X Y X Y
1. 2 980 69 131 54 121 23. 66 899 361 134 348 124
2. 4 979 77 131 58 121 24. 70 873 376 132 362 123
3. 6 985 86 131 67 121 25. 72 855 389 131 373 121
4. 8 977 91 131 77 122 26. 74 836 401 131 382 121
5. 10 967 100 133 84 123 27. 78 837 411 130 396 120
6. 12 947 112 133 94 125
28. 80 924 427 129 412 119
7. 14 945 119 133 100 125
29. 82 917 443 129 424 119
8. 16 963 128 136 114 126
30. 86 941 463 129 444 119
9. 20 937 142 136 127 126
31. 88 938 472 129 453 119
10.
22 970 156 137 139 127 32. 90 944 479 129 462 119
11. 28 911 174 137 160 127 33. 92 922 484 129 475 119
12. 30 913 189 137 171 126 34. 94 926 500 129 481 119
13. 32 904 202 136 183 126 35. 96 944 510 129 491 119
14. 36 909 217 135 198 125 36. 98 944 526 129 507 119
15. 38 952 237 135 222 125 37. 100 919 532 129 513 119
16. 42 953 250 134 231 124 38. 102 916 541 129 522 119
17. 50 884 277 134 263 124 39. 104 927 552 129 533 119
18. 54 906 297 134 283 124 40. 106 858 568 128 549 118
19. 56 914 311 134 283 124 41. 108 890 584 128 559 118
20. 60 891 325 134 316 124 42. 110 775 591 128 572 118
21. 62 927 341 134 326 124 43. 113 867 609 127 590 117
22. 64 988 353 134 335 124 44. 115 867 609 125 590 117

VII. OUTPUT IMAGE SEQUENCE

Various Frames with tracked object and moving bounding box along it in each iteration with reference to the frame number. It
displays the upgrading frames with tracked vehicle shown in bounding box.
 CONCLUSION future scope to be improved upon and made more reachable to
common man for benefit of human society.
The proposed algorithm was implemented on a captured video
sequence with 115 frames at rate of 30 fps, which was
REFERENCES
compressed and converted in .avi format. Object was detected
first according to the image input, and then tracked in
[1]. Elhamod, M.; Levine, M.D., "Automated Real-Time Detection of
subsequent frames. Coordinates of the object tracked and the Potentially Suspicious Behavior in Public Transport Areas," in
overlaid bounding box were analyzed and tracked. The Intelligent Transportation Systems, IEEE Transactions on , vol.14, no.2,
experimental implementation could perform the object pp.688-699, June 2013
tracking without missing any frame and could successfully [2]. Freer, J.A.; Beggs, B.J.; Fernandez-Canque, H.L.; Chevrier, F.;
overlay bounding box. It could successfully generate an image Goryashko, A., "Automatic video surveillance with intelligent scene
sequence after the complete implementation of Mean Shift monitoring and intruder detection," in Security Technology, 1996. 30th
Annual 1996 International Carnahan Conference, vol., no., pp.89-94, 2-4
Algorithm. The performance of algorithm was checked by Oct. 1996
successfully tracking the user defined object and performing [3]. Chowdhury, A.; Tripathy, S.S., "Detection of human presence in a
the overlay function in the identified object. surveillance video using fuzzy approach," Signal Processing and
Integrated Networks (SPIN), 2014 International Conference on , vol.,
FUTURE SCOPE no., pp.216,219, 20-21 Feb. 2014
[4]. Bird, N.D.; Masoud, O.; Papanikolopoulos, N.P.; Isaacs, A., "Detection
IVS have tremendous scope for research and development. of loitering individuals in public transportation areas," in Intelligent
Transportation Systems, IEEE Transactions on , vol.6, no.2, pp.167-177,
Intelligent techniques, esp. Neural and SVM are emerging as June 2005
indispensable techniques for behavior analysis. Semantic [5]. Rezaee, K.; Haddadnia, J.; Delbari, A., "Intelligent detection of the falls
analysis for Activity Detection is also taking hold due to its in the elderly using fuzzy inference system and video-based motion
relative simplicity. To make the system fully automatic and estimation method," Machine Vision and Image Processing (MVIP),
robust and to overcome the gaps and imitations like 2013 8th Iranian Conference on , vol., no., pp.284,288, 10-12 Sept. 2013
information loss, occlusion, noise, non-rigid motions etc., [6]. Tai Yu Lai; Jong Yih Kuo; Chien-Hung Liu; Yue Wei Wu; Yong-Yi
multi-view tracking and can be implemented using multiple FanJiang; Shang-Pin Ma, "Intelligent Detection of Missing and
Unattended Objects in Complex Scene of Surveillance Videos," in
cameras. Multi view tracking has the advantage over single Computer, Consumer and Control (IS3C), 2012 International
view tracking because of widerange and multiple view Symposium on , vol., no., pp.662-665, 4-6 June 2012
captures, with different viewing angles for the objects to be [7]. Aramvith, S.; Pumrin, S.; Chalidabhongse, Thanarat; Siddhichai, S.,
tracked. Scope of research lies in improving accuracy of such "Video processing and analysis for surveillance applications," in
systems along with cost optimization, and making them more Intelligent Signal Processing and Communication Systems, 2009.
reachable and applicable for society.This work is the first ISPACS 2009. International Symposium on , vol., no., pp.607-610, 7-9
Jan. 2009
stage of object detection which is the most essential part of an
[8]. Nair, R.; Bing, B., "Intelligent activity detection techniques for
intelligent system, which is an input for high end analysis of advanced HD video surveillance systems," Sarnoff Symposium, 2010
possible suspicious activities. It opens up a broad scope for IEEE , vol., no., pp.1,5, 12-14 April 2010
further implementation of intelligent video systems based on [9]. Ravi kumar A.V , sunil tiwari N, “multiple moving object detection and
object tracking which again has numerous applications and tracking using Harr features with smart video surveillence
system”IJERT, vol 3-issue 6(june 2014),2278-011
[10]. Morris, B.T; Cuong tran; scora, G; Trivedi, M.M;Barth,M.J.,”Real-time [13]. Haridas L. Sanjay., "Object Tracking using Modified Mean Shift
video-based traffic measurement, “Intelligent Transportation system, Algorithm in A Robust Manner," in IJSTE - International Journal of
IEEE Transactions on, vol.13, no.4,pp.1667,.1678,Dec.2012J. Science Technology & Engineering, Volume 2,Issue 1, July 2015,
[11]. Chouhan K. Mahesh, Mishra Rahul, Nitnawwre D.,”Movable object pp.17-24, Jul 2015
tracking by using mean shift method with adjusted background [14]. MathurG. "Research on Intelligent Video Surveillance Techniques for
histogram” volume 2, issue 7, july 2012. Suspicious Activity Detection – Critical Review," in ICRAIE-Second
[12]. D.Comaniciu, Ramesh, P. meer, “Real –time tracking of non rigid IEEE International Conference on Recent Advances and Innovations in
object using mean-shift”, IEEE proc.comput. vis. Pattern recog.,pp. 673- Engineering (ICRAIE-2016), Dec 2016
678,2000.

View publication stats