La Divina Comedia 2

Download as pdf or txt
Download as pdf or txt
You are on page 1of 7

ISSN (Print) : 0974-6846

Indian Journal of Science and Technology, Vol 10(19), DOI: 10.17485/ijst/2017/v10i19/91682, May 2017 ISSN (Online) : 0974-5645

Zigbee Image Transmission for Detecting and


Tracking Multi Color Moving Object
M. Chakravarthi* and D. Aruna Kumari
Department of Electronics and Computer Engineering, KL University, Vaddeswaram, Guntur – 522502,
Andhra Pradesh, India; chakri.39.3@gmail.com, aruna_d@kluniversity.in

Abstract
Objective: To design and implement a Zigbee Imaging system to track and detect the multi colored moving objects using
Additive color model technique. Analysis: Digital image processing methods like Segmentation, background subtraction
and point detectors are very popular and prominently used in the object detecting and tracking from live streaming frame
sequences. These methods require high computing systems to process the data in real time. Methodology: To overcome
such constraints, we implement the low cost IEEE 802.15.4 Zigbee based Image Processing System. This system can process
and transmit the images efficiently even in Dynamic environment conditions. In this paper, we illustrate Zigbee Imaging
system functionality at sensitive places of observation. Finding/Improvements: We implement our proposed system
with a Low cost CMOS color camera and Advanced RISC processor, which are used for supporting the Zigbee imaging
system in detecting and tracking the objects. The system operates at 30 fps with resolution 88x144.This system can detect
and track 8 objects at a time in the camera field view and transmits the tracked information to the host system with
wireless Technology 802.15.4 Zigbee. The Experimental results of Prototype system to image sequences are also figured.

Keywords: Additive Color Model, Multi Color Moving Object, Object Spying, Zigbee Imaging System

1.  Introduction It is always a challenging task to achieve adequate per-


formance in image processing with constrained amount
We have an assortment of utilizations in implementing of computing resources. Generally, such systems require
a basic system based on image processing algorithms for high end processor, huge amounts of Random Access
detecting and tracking the objects1–3. Applications in the Memory (RAM) for storing and processing the images
field of security and surveillance systems monitor the objects and consume enormous amounts of power while pro-
automatically based on their specific features. Object per- cessing the images. Basic image processing systems
ception for tracking the object is done by using some image conventionally comprise of a camera, a frame extractor,
processing algorithms which recognizes and tracks the and a related computer to interface this frame extrac-
object based on the primary features like color and shape. tor to complete the execution of the algorithm. But the
But restrictions like Cost and Complexity to implement such subsequent result is quite difficult and extravagant to the
algorithms on Hardware are needed to be considered. Out developer, where he works on a system limited image
of these algorithms, recognition of the color can be achieved functionalities. Now it is possible to make the neces-
with the assistance of a Digital image processing algorithm, sary changes to scale down the price of those systems
called additive Color Model scheme, which greatly reduces by adapting new technologies. Low Cost, high speed
the complexity and also works in a cost effective manner4. Microcontrollers and Cost effective CMOS color camera

*Author for correspondence


Zigbee Image Transmission for Detecting and Tracking Multi Color Moving Object

modules are two technology improvements which we’ve Each individual module of the system is explained in the
taken into this work. The main advantage of CMOS over following sections.
CCD camera technology is that CMOS can integrate
Analog to Digital converters and related pixel extracting 2.1  System Hardware
circuitry on the same die on the sensor itself. So, we don’t
The basic Hardware of the imaging system comprises
need to interface a separate frame extractor in the system.
of three components. The first two components, RISC
In recent times low cost and high capabilities of controller
integrated Atmega8 and CMOS C3088 Digital cameras
makes it possible to implement simple pixel processing
are vital components of our system and an Xbee Zigbee
as the pixel values of the camera are scanned and driven
module for supporting Zigbee Image Transmission. The
out, making an extra and an unused frame buffer in many
data bus, configuration bus and synchronization pins are
situations5.
directly connected to At mega 8 controller without any
The complex computation methods which were
complex logic for simple and reliable design. For process-
implemented in image processing systems require a large
ing the data in Real time, Atmega8 controller waits for the
number of computing elements. It is difficult to use the
data coming from the camera continuously. Then the con-
huge number of computing elements in real time image
troller transfers the extracted high level data, i.e., image
processing applications. Most of the image processing sys-
to the host system by using asynchronous serial interface
tems transmit the image to the host system using an extra
which is implemented in embedded software. Operating
communication medium such as cables. Subsequently, it
voltage for imaging system is 5 volts and consumes about
will be added to the overall cost of the system.
57 mA of current. The image information to imaging sys-
In this paper, we propose a low cost and reliable
tem is given by a C3088 Digital Camera. It is a 1/4” Digital
Zigbee imaging system implementation and its perfor-
output Color camera module. Omnivison’s CMOS image
mance evaluation in detecting and tracking multi colored
sensor 6620 is integrated into C3088 Camera7. Integrating
moving objects in the ambience. Light illumination levels
this CMOS innovation with an Image processor is a simple
and noise levels in the background are the most common
to utilize computerized interface which makes C3088 cam-
problems which arise during the tracking of these objects.
era integrated system a minimal effort solution for high
These problems can be resolved in this Zigbee imaging
quality video-image application. 8/16 bit-wide image data
system with latest advances in camera technologies where
stream is continuously supplied by digital video port for
it can be adjusted as per the ambient lighting conditions.
tracking the object in real time. All camera functions, such
A Low Cost IEEE 802.15.4 Zigbee is employed as a com-
as exposure, gamma, gain white balance, color matrix, win-
munication medium for facilitating the transmission of
dowing are programmable through I2C interface8. It also
the raw image data to the host system6.
includes a 4.9 mm F2.8 lens for zooming purposes and 17
MHz on-chip Clock crystal for synchronization purpose.
2.  System Design CMOS camera runs independently to transfer 8 bit RGB or
YCrCb color Pixels through an 8 bit or a 16 bit sized data.
Our Zigbee Imaging system is designed to track multi To intimate the start of new frames and horizontal lines,
colored moving object by extracting high-level informa- Synchronization signals, Pixel clocks are being used. The
tion from a camera image. This information is sent to Camera resolution of 352x288 at 60 fps refresh rate is the
a Host System Application for maintaining the tracked maximum attainable image frame size8.
information of the multi color objects. The object color To use video information from the CMOS C3088
which is intended to track is selected from host system Camera, One must legitimately instate the camera and
application, specially designed for imaging system. For afterwards stay synchronized with each of its output sig-
instance, the particular bounded set of RGB (Red Green nals. The camera must be initialized properly to get the
Blue) values is specified in the host system application video data from CMOS C3088 and continue the synchro-
and these values are sent to the imaging system via Zigbee nization signals for real time buffering. The controller,
module that we are implementing as a communication which is used to process the data, is Atmega8. It is RISC
medium. The Zigbee Imaging system employs three dif- processor and instruction execution rate is 1 MIPS per
ferent modules for achieving the efficient performance. MHz It has internal flash EEPROM of size 512 bytes,

2 Vol 10 (19) | May 2017 | www.indjst.org Indian Journal of Science and Technology
M. Chakravarthi and D. Aruna Kumari

programmable memory of size 8 kilo bytes and SRAM of processing during the time gap between pixel extractions
size 1 Kilo bytes. It’s Asynchronous serial port is connected by utilizing the data that streams from the camera. Image
to the Zigbee Xbee module which supports point-to- processing functions drop the trailing G component of
point and multi-point networks. Xbee module is the main the sensor’s RGBG pixel group and skip every other G
module for the Zigbee imaging system which is config- pixel to reduce processing time. This Procedure restricts
ured in Transparent Mode (AT)9. Configuration details of the horizontal resolution to 176 RGB pixels, but sets
Xbee module includes baud rate, Network address (PAN limit on the maximum vertical resolution of 144 pixels.
ID), Destination device short address. After configur- Additional post processing and transmission of data is
ing the device, it starts communicating with destination done in the time gap between the end of each row and at
device. It can establish communication with destination the end of each frame. The imaging system is capable of
device and transmit the data up to a range of 300 feet with
implementing Additive color model scheme for attaining
Line of Sight10. The Real time performance in detecting
these required parameters of the system4.
and tracking can be achieved with our proposed hard-
Coding to for integrating this hardware is written in C
ware design. It is also possible to accommodate multiple
host system with a single imaging system for broadcast- Language and a target hex file is created using the Atmel
ing the tracking information. The hardware of the Zigbee studio compiler. After compiling the firmware, it takes 4K
Imaging system is given in the Figure 1. words of ROM and at some point; it is utilized for stor-
ing image pixels. RAM availability becomes very less after
executing the program. Hence, it is needless to say that
the firmware has to be coded very carefully. We can see
embedded software block diagram in the Figure 2.

Figure 1.  Hardware snapshot of zigbee imaging system. Figure 2.  Embedded software architecture.

2.2  Embedded Software Commands for a typical Zigbee imaging system are
A set of functions in the source code of the system makes formatted in ASCII operation-code (op code). Each com-
it easy to interact with CMOS image sensor at hardware mand is framed with Two ASCII characters followed by
level. Any system with some Imaging processing capabili- an optional parameter and carriage-Return character.
ties can make use of these functions to perform custom Whenever the command is given to imaging system, then
image processing. This embedded software utilizes around it should respond with either a positive Acknowledgment
4K of the 8K on-chip flash program memory,700 bytes of (ACC) indicating the command was successful or a
1K on-chip SRAM, and 48 bytes of the 512 bytes on-chip Negative Acknowledgment (NAC) indicating that the
EEPROM. While developing the software to the system, command is unsuccessful.
the main thing that we have to consider here is the avail- For instance, if the user wants to check the Zigbee
ability of SRAM in Microcontroller. SRAM availability imaging system to ensure that the system is functioning
after programming the software is only 324 bytes. It is properly or not, PN\r (PING) command is sent to system,
beyond the SRAM size to buffer an entire color image and then it responds with a Positive acknowledgment
at a time. To stream the image data from the camera (Acc). The host system confirms that the imaging system
without any buffering, we have to consider the limit in is alive and operating properly, because it has responded
SRAM availability. We were able to perform basic image with a positive acknowledgment (ACC). This system can

Vol 10 (19) | May 2017 | www.indjst.org Indian Journal of Science and Technology 3
Zigbee Image Transmission for Detecting and Tracking Multi Color Moving Object

also give its name with a Command request GN\r (GET 2.4 System Modes and Camera Functional
NAME). The response to this command gives the normal Settings
acknowledgment followed by the Name of the System.
Likewise, the software of the system is coded with simple Zigbee imaging system has three different modes apart
command requests and responses for interacting with the from the normal command response protocol, which
user to ensure its proper functioning. can be initiated by sending commands to the imaging
system, then after it transmits some additional packets
to the host system asynchronously. Each mode takes the
2.3  Color Detection Method
system into different functionalities for distinctive cases.
The Additive color model, also known as RGB color The default mode is idle mode and it is first mode for
model is implemented in embedded software as a the imaging system. In this mode, Host system does not
special function. This function acts as color recovery receive any packets from the imaging system. The Frame
function and maintains a running sum of the inde- dump mode is second mode. The system is entered into
pendent color channel components. Upon completion this mode by sending DF\r (DUMP FRAME) command.
of the frame, the mean color returned by summing of In response to this command request, imaging system
all the pixels is divided by accumulated pixel values. It starts sending Dump packets to the host system until the
additionally gives back an estimation of the total devia- entire image frame has been sent. Once all dump packets
tion from the mean of every color. The color statistics for an entire frame have been sent, the imaging system
can be used as a building block for determining the switches back to idle mode. The color tracking mode is
color of an object or detecting motion in a particular the third mode. The system enters this mode by sending
area in the field of view. Also allows the user to enter a TS\r (TRACKING START) command. Then it starts the
range for either each of the three RGB or YCrCb color tracking and sends this tracking information to the host
channel values depending on camera configurations. system. All these would happen only when requested
After receiving the color boundary values, each pixel command is received properly and positive acknowledg-
is compared with received color range of values. For ment is sent back to the host system. This tracking packet
simplifying this task, Atmega8 controller stores the transmission takes place during the time that system in
global information like Image raw data and bounding the Tracking mode. Tracking information packets are
box coordinates of the Frame. The global data contains not generated when the system is not able to detect the
the upper left x1, y1 coordinates and the lower right object and stays in tracking mode till we send the com-
x2, y2 coordinates which include pixels that satisfy the mand to disable the tracking mode. We should send TP\r
color bounds. The coordinates of the pixels close to the (TRACKING STOP) command to the imaging system.
color bounds are compared against previously stored This command switches the system back to Idle Mode.
coordinates to generate the bounding box. Likewise, The hexadecimal 8-bit values range between 0x00 and
we also count how many pixels come in the bound- 0xFF are transmitted to Host system in the well defined
ary limits of actual color. Once the complete frame has packet format. These tracking packets are not easy to
been processed, further post processing operations are comprehend with a normal serial terminal emulator. We
completed specifically. A scaled ratio between the cal- can see the information in software specially designed for
culated area and the total sum of pixels within the color the Zigbee Imaging system.
boundaries is computed by considering the bounding To detect and track the object at super sensitive places,
box. This value can then be used as reference value it is important to adjust the camera functional settings as
which intimates whether there is only one object or per the lighting conditions. It will have a major effect on
multiple small objects being tracked, which occupies the real capabilities of the system. For instance, reflective
the bounding box. The x and y Coordinate positions of objects appear as high intensity white spots as light fall on
the each detected object is acquired by the system after the reflective objects irrespective of color of the object.
an entire frame has been received. By using these val- We can adjust the camera lens and its parameters as per
ues, system can return the number of detected pixels, lighting conditions. The parameters that are possible to
coordinates of bounding box and reference values of adjust by sending commands to the imaging system from
the bounding box. serial terminal or GUI are auto-adjust mode, Auto-white

4 Vol 10 (19) | May 2017 | www.indjst.org Indian Journal of Science and Technology
M. Chakravarthi and D. Aruna Kumari

balance, and fluorescent light filtering as shown in Figure Name of the system can also be requested from a termi-
3. Then the camera is able to detect the object even in the nal window. To get the image Raw data, we should send
poor lighting conditions. DF\r command. This Raw data ranges from 0x00 to 0xFF
hexadecimal 8-bit values are transmitted to the system in
a well defined packet format as shown in the Figure 4
.

Figure 3.  Camera registers window

3.  Implementation
A user can interact with imaging system through serial
protocol. This imaging system uses a Human under- Figure 4.  Serial terminal window with command responses.
standable ASCII code format. The system is configured
to communicate with the Zigbee imaging system at 115.2 The multi color moving object tracking is enabled
kilo baud rate. This system can also dump the raw image by giving the command TS\r. Then system will send the
data while communicating with host system through tracking packets in hexadecimal format. Each packet
serial port interfaced with Zigbee module. contains tracking information of the color object. We
Frame dump does not occur in real time due to the should send TS\r before changing its mode of operation.
high data rate required. Instead, the imaging system will The Zigbee imaging system is capable of tracking up to
send each row of image data per frame to the host system. eight objects per frame. It may be zero, if the objects are
The maximum frame size of 176x144 is the default resolu- not detected. All the traceable objects information in the
tion and dumping this entire frame takes about 4 seconds. camera field view will be framed as packet and returned
to host system as color-tracking packet. We can also
3.1  Serial Terminal adjust the camera functions like white balance, gamma
and exposure from serial terminal with CR\r (CHANGE
The most straightforward approach to begin with the
CAMERA REGISTER) command and the corresponding
Zigbee imaging system user interface is to utilize a serial
values. We can observe the changes in Tracking informa-
terminal emulator running on a Host System (such as
tion and the raw image data which is sent by the imaging
HyperTerminal, or Real Term). The serial terminal emu-
system.
lator ought to be set up to keep running at a baud rate
of 115.2 kbps,8 data bits, 1 stop bit, and no flow control.
Observe that the terminal is just sending a carriage return/ 3.2 GUI
line-feed when the enter key is pressed. This permits the A Java based Graphical User Interface (GUI) helps in
user to send basic commands to the imaging system and system integration, which makes easy for the user to
see the command responses sent by the system to the host communicate with the imaging system from a Windows
system serial terminal. based PC. This GUI enables you to explore all the func-
For instance, if we give the PN\r command to the tions of the camera from user familiar environment.
imaging system from the terminal then the response The GUI clearly shows real-time information from the
obtained would be as shown in the Figure 4. Likewise, camera in a common way. For instance, the color pixel

Vol 10 (19) | May 2017 | www.indjst.org Indian Journal of Science and Technology 5
Zigbee Image Transmission for Detecting and Tracking Multi Color Moving Object

to be tracked is picked from captured image by clicking to 8 different colors. In this way we can add 8 different
on the enable track button. Then GUI frames user com- colors of our interest.
mand request and sends it to the camera via serial port
connected to it. The camera output data is determined
and viewed in a GUI window that displays the actual
bounding box along with tracking color. The color posi-
tion depends on the reference value that is returned.
Depending on the camera configuration, binary image
of tracking mode and centroid may also be superim-
posed on the bounding box. When user calls the image
recovery function, then returned color image data is
mixed and displayed. The standout component among
the most essential components of the GUI is its “capac-
ity” to show dumped image. Figure 6.  Color map window.
To test the GUI Host system application, we first send
the PN\r command by clicking on the Radio button of We can start the object tracking by clicking on
GUI user interface. Message viewer of the GUI shows “Enable Tracking” button, after setting color to the color
the command request and its response. After getting map. All command bar options are disabled while system
the Positive acknowledgment from the system, we press is in tracking mode. We can see the detected and tracked
the capture button in the command bar to get the entire objects along with the color in the center of the bound-
frame of the camera field view and display it in the frame ing box of tracking window. Based on colors in the color
window as shown in the Figure 5. It takes 4 seconds to palette, as tracking objects move through the imaging sys-
complete the image acquisition. The camera module lens tem field view, then real time tracking windows updates
needs to be adjusted to get the clear image without any to show that the object is being tracked and it is depicted
blur if the image is not clear. in the Figure 7.

Figure 5.  Captured image window.


Figure 7.  Object tracking window
After capturing the image, we can pick the color of
the object that we want to track by moving the mouse The communication between the Zigbee imaging sys-
pointer over the color of interest in the image window. tem and PC can be seen in the Log maintained by the GUI
We can observe the variation in RGB values in the image host system application. It shows the time stamp to know
window whenever the mouse pointer moves over the when the object is identified. This tracking information
image of different colors. To track the color of our inter- file can be in a text file for future reference. Disable track-
est, we add the colors in GUI color map as shown in ing button in command bar needs to be clicked to disable
Figure 6. This color map allows us to efficiently map up the tracking and access entire command bar again.

6 Vol 10 (19) | May 2017 | www.indjst.org Indian Journal of Science and Technology
M. Chakravarthi and D. Aruna Kumari

4.  Performance Evaluation acknowledge the generous inputs and suggestions from
the Embedded Systems and Sensor Networks (ESSN)
The System is designed to track the multi color moving research group of KL University.
object based on Zigbee image transmission and it is work-
ing as per requirements and conditions we’ve defined in
the previous sections. Once, the system receives the color
7.  References
values needs to be tracked then it starts tracking the 1. Baghyasree T, Janakiraman K, Parkavi D. Efficient remote
objects. This System covers the field area up to 35 feet video monitoring for industrial environment based
away in the camera field view.It is also able to receive and on LPC2148 and GSM. Indian Journal of science and
transmit image information from both Serial terminal Technology. 2014 Sep; 7(9):1333–41.
and GUI. Tracking Image transmission takes place at a 2. Krithika N, Seethalakshmi R. Safety scheme for mining
rate of 30 frames per second with a maximum resolution industry using zigbee module. Indian Journal of Science
of 88x144 is attained as per the defined requirements. The and Technology. 2014 Aug; 7(8):1222–7.
System is designed and implemented as per the expecta- 3. Nourbakhsh I, Andre D, Tomasi C, Genesereth M. Mobile
robot obstacle avoidance via depth from focus. Robotics
tions and is successfully tested for the functionality.
and Autonomous Systems. 1997 Nov 20; 22(2):151–8.
Crossref.
5.  Conclusion 4. RGB color model [Internet]. 2017 [cited 2017 Feb 28].
Available from: Crossref.
The objective of this work is to design and implement 5. Sugandi B, Kim HS, Tan JK, Ishikawa S. Real time object
Low cost Zigbee Imaging system for tracking the multi tracking and identification using camera. The 23rd
color moving objects at super sensitive places in dynamic International Technical conference on Circuits/Systems,
environment conditions. We’ve carried out checking the Computers and Communications (ITC–CSCC); 2008. p.
1585–8.
integrity of building a Low cost basic Imaging system
6. Pekhteryev G, Sahinoglu Z, Orlik P, Bhatti G. Image trans-
comprising exclusively of a microcontroller and a CMOS
mission over IEEE 802.15.4 and zigbee networks. Institute
camera chip. It should actualize straightforward algorithms
of Electrical and Electronics Engineers (IEEE) International
at a reasonable frame rate. Tracking multiple objects at var- Symposium on Circuits and Systems. 2005 May 23–26;
ious conditions at 30 fps with Zigbee image transmission is 4:3539–42.
achieved. The proposed system is able to show robust per- 7. Omnivision. OV6620 single chip CMOS CIF color digital
formance even when environment conditions are changed. camera OV6120 single-chip CMOS CIF B and W digital
Still some advancement can be included to the system to camera [Internet]. 1999 [cited 1999 Feb 6]. Available from:
improve the overall performance. The size of the image Crossref.
buffer can be increased and the functionality of the system 8. C3088 1/4 color camera module with digital output data-
can be improved considerably by using various advanced sheet [Internet]. 2010 [cited 2010 Jul 28]. Available from:
processors with higher RAM Sizes. Crossref.
9. Wireless connectivitiy kit [Internet]. 2016 [cited 2016 Nov
30]. Available from: Crossref.
6.  Acknowledgements 10. Microcontrollershop. XBee-PRO ZB S2B zigbee module,
63 mW, RP-SMA connector [Internet]. 2011 [cited 2011
We are grateful to KL University for their constant sup- Oct 17]. Available from: Crossref.
port and encouragement throughout this project. We

Vol 10 (19) | May 2017 | www.indjst.org Indian Journal of Science and Technology 7

You might also like