Implementation Of Vision Based Object Tracking Robot

IEEE/OSA/IAPR Intenational Conference on Informatics, Electronics & Vision Implementation Of Vision Based Object Tracking Robot Kazi Mahmud Hasan, Abdullah-AI-Nahid, Abdullah Al Mamun Electronics and Communication Engineering Discipline Khulna Universiy Khulna, 9208, Bangladesh {shuvro eceku07, nahidku,ronee ece04 }@yahoo.com _ Abstract - This paper introduces a vision based object tracking robot which is driven by wheels and controlled by a computer along with sotware. The objective of this project is to design a robot which is automatically controlled by computer to track and follow a colored object. Emphasis is given on precision vision based robotic applications. Image acquisition by the robot is achieved by using a PC-based webcam, then it is send to image processing software for further processing. The overall paper _ designing simple and effective algorithms which is equivalent to existing huge and far more complex algorithms. Most of the robots made previously are expensive and complex but this robot is very much cost effective, lightweight and straightforward than existing robots like iBOT. II. BLOCK DIAGRAM describes a visual sensor system used in the ield of robotics for identiication and tracking of the object. Indx Terms - Webca, iter, binay image, robotic vision, steering. I. INTRODUCTION The word robot was introduced to the public by the Czech interwar writer Karel Capek in his play R.U.R (Rossum's Universal Robots), published in 1920. The play begins in a factory that creates artiicial people (robots) which is somewhat like modem androids. Robots are electric machines having ability to perform tasks or actions on given electronic programming. As mechanical ways of performing various tasks have been discovered and the development of mechanics and complex mechanisms still going on in full swing, the necessity of human labor has been reduced tremendously. Initially machinery was mainly used for repetitive unctions. Automation is desperately needed to handle task of our day to day life [1]. So with the development of science and technology more complex machines have been developed. Object detection is a fundamental basis of artiicial intelligence and robotic vision system [2]. Object detection methods are used in various ields like science, engineering, medical applications. It is necessary for surveillance applications, guidance of autonomous vehicles, smart tracking of moving objects etc. This paper deals with only object detection in robotics. A camera is used for image acquisition and MATLAB is used to process it. The camera works as the eye of the robot. In order to develop a stable and useful vision based robot proper study and accurate model regarding image processing are very much necessary [3][4]. The entire paper is mainly divided into three parts which includes image processing, algorithm regarding object tracking and steering mechanism of the robot. Designing mechanical model and writing program for robots are dificult as the scale and applications of robots continue to grow. This paper describes our attempt towards 978-1-4673-1154-0112/$31.00 ©2012 IEEE The overall process consists of acquiring image using a webcam, then processing it using MATLAB. Based on the processing, control signals are generated automatically which is fed to the robot driving motors so that the motors can perform necessary movements to follow any particular object. A block diagram is given below to highlight the entire process. Image Acqui s ition (Webcam) Image Processing . (Computer + MATLAB) Control Signal Generation Para lIel Pot Driver Circuit Motion Control Fig. 1 Block diagram of entire system. ICIEV 2012 IEEE/OSA/IAPR Intenational Conference on Informatics, Electronics & Vision III. IMAGE PROCESSNG 1) Image Acquisition: The control of the robot is a dynamic process where images are taken continuously using camera; processing them, ind the required information like position and orientation of an object and inally generate control signals to perform required movement. Image acquisition is the irst step in vision based robotic system. The Image Acquisition toolbox of MATLAB provides support in this regard. To start working in real time environment a unctional USB webcam is connected to the Pc. In order to take images by the webcam, irst video input object has to be created and video stream has to be viewed. Fig. 2 (a) Original RGB image (b) Image ater applying logical conditions containing expected region and nOIse Then a video stream is available and still images are captured rom it. 2) Filtering The Binay Image Containing Detected Object Noise Remove) : The binary image obtained above has quite a lot of noise (unexpected color component scattered throughout the image). For high performance robotic vision any unexpected object must be completely removed. There are several functions available in MATLAB to remove noise in an Image. The most commonly used iltering unctions for binary image are imerodeO, imdialateO, imcloseO, imopenO and imillO [5][6][7][8]. For this robot three basic unctions are used. Images in MATLAB are 3D matrixes. The matrixes are (1) 1(:,:,1) (2) 1(:,:,2) (3) 1(:,:,3) Where'!' is the captured image. 2) Detecting Color Region: The next step in tracking the object is to detect and isolate the color of the object. RGB image comprises of three basic colors red, green and blue. First these three colors are separated rom the original image (3D) matrix and three different matrixes are formed.Suppose the matrixes are: (1) R= RGB(:,:,l) (2) G= RGB(:,:,2) (3) B= RGB(:,:,3) [ Red color component matrix] [ Green color component matrix] [ Blue color component matrix] The unction imcloseO performs morphological closing operation. It ills the gaps between two objects and smoothens the edges. The degree and type of smoothening and joining depend on the structuring element, Structuring element is the basic shape which is used to perform the operation. The structuring element can be a disk, a diamond, a line, etc. The resultant image ater morphological closing operation is given below. Now the corresponding intensities of red, green and blue color components are calculated, which are very much necessary for object region detection. Let the minimum value of red, green and blue color component be 'r1', 'gl' and 'b1'. The maximum values are 'r2','g2' and'b2' respectively. Let the expected object is'X'. So 'X' can be found by simple logical equation as stated below: X=(R>=r1&R<=r2)&(G>=g1&G<=g2)&(B>=b I&B<=b2) Here'X' is a binary image containing'1's and'O's. Where'l' means presence of color (white) and '0' means absence of color (black).The opposite is also true if'X' is complemented. For this robot simple form is used. In the following igure below let red color be the object and next it is detected (white region) containing noise. 860 Fig. 3 Image ater applying morphological closing operation. It is seen that this iltering operation ills the holes within object. This iltering operation can also be done by the unction imillO. The iltered image still contains noise. Many scattered color components may be found throughout the image plane. To remove the scattered color components the function imopenO can be used. ICIEV 2012 IEEE/OSA/IAPR Intenational Conference on Informatics, Electronics & Vision The inal output of the iltering process is a noiseless image containing only the expected object region as follows. Column n . . · . m t . · N -. V\ � � • � ( 1) ( 2) Image Plane Fig. 6 Geometric calculation for center coordinates. Fig. 4 (1) Image ater applying morphological opening operation (2) Complemented form of iltered image. 3) Multiple Object Handling: In real time situation it may happen that the robot racked more than one similar color object. So it is dificult to choose which object the robot should follow. The technique used by this robot is to follow any particular object and reject others. But which objects should it reject? For multiple objects, this robot will selectively keep the largest object and reject objects smaller than the largest, even if the object has only one pixel less than the largest one. The equations of center coordinates are Sum of column Numbers having ' l' Total Number of columns having '1' X = So, X = ------------ L¥ xn N-n+l y = Suppose 5(3) is the iltered binary image. Ater applying multiple object handling technique the resultant image is 5(4) containing only the largest region'C'. •• • • • So , y = Sum of row Numbers having '1' Total Number of rows having '1' ----------. _.. ­ .: LMYm M-m+l A program was written to implement those equations for inding center coordinates. Those equations provide center coordinate which is nearly hundred percent accurate and several times faster than any other process. IV. ALGORITHM (3) ( 4) Fig. 5 (3) Multiple object detection (4) Only the largest object is selected others are rejected. 4) Center Calculation: Finding the appropriate center coordinate (Pixel Coordinate System) is very important for this robot. If the center point is not properly calculated the robot will malunction and it will not be able to track and follow the object. MATLAB provides built-in unction to calculate center point of any region. Alternate way of calculating the center coordinates is to apply simple geometry of inding center of gravity (CQ) which is faster process than using built-in MATLAB unction. That's why this process is used for computing center coordinate of the object. 861 ente (X,Y) The robot takes images of its surroundings repeatedly and tracks the expected object and generates controlling signals based on the position of the object on the image plane [9]. Based on camera principles the image plane is divided into ive segments for ive different commands. The location of the object keeps on changing as the position of the camera changes. This principle leads to develop ive different commands. The commands are forward, backward, let, right and stop. The following igure shows the image plane with corresponding segments along with a tracked object. The algorithm ensures guaranteed tracking of the object if the object is found anywhere on the image plane. ICIEV 2012 IEEE/OSAIIAPR Intenational Conference on Informatics, Electronics & Vision Two motors are used to drive the robot. The processing unit generates control signals which is then send to the parallel port's data pins and fmally fed to the driver circuits of the motors. The process is given in Fig.9. Fig. 9 Computer and hardware interfacing. Fig.7 Diferent segments of image plne. The algorithm is developed using MATLAB to follow the object. First the image is taken and object region is detected. Then the center coordinate it calculated and conditions are applied to fmd the corresponding image segment. If the center of the object is in segment I then the robot goes forward. The robot goes backward for segment 2. It ns right for segment 3 and let for segment 4 and if the center is in segment 5 the robot stops moving. The overall controlling process is given in the block diagram below. Two driving motors moves simultaneously to follow the object. The movement is controlled automatically by the computer. The movements of the robot based on the image processing are listed in the table below. Forward Let Wheel Forward Right wheel Forward Center in segment 2 Backward Backward Backward (3) Center in segment 3 Right Forward Backward (4) Center in segment 4 Let Backward Forward (5) Center in segment 5 Stop Stop Stop No Status Commnd (1) Center in segment 1 (2) Table. 1 Movements of the robot. IV. MorOR DRIVER ND STEERING 1) Motor Driver Circuit: A detailed driver circuit is shown in the following igure. In this circuit the primary elements are 6V relays. Relays are electromechanical switches that can switch between two circuits. Here when voltage is applied, say to input'S', it's concening relay will switch rom ground to Vcc which applies voltage to the let DC motor and inds ground through other relays. So the use of the relay based driver circuit gives a very low resistance path to the motors. Relays help provide much greater current to the motors than semiconductor driver circuits. This circuit provides built in dynamic motor breaking ability which gives extra strength to the steering mechanism. Diodes are used to protect the transistors rom damaging due to reverse magnetic ield. The driver circuit showed better performance than traditional H-bridge motor drivers like L293 or L298. Fig. 8 Algorithm for object tracking. 862 ICIEV 2012 IEEE/OSA/IAPR Intenational Conference on Informatics, Electronics & Vision Vee Vee Vee Ve Fig. 10 Driver circuit of the robot. v. CONCLUSION 2) Driving And Steering Mechanism: Two DC geared motors are connected, one to each of the two drive wheels at the right and at the let of the robot's base. Those two motors are responsible of driving the robot backward and forward as well as steering in any required direction. A ree running wheel is set in ront of the robot. heel Camera Free Running Front Wheel The concept of the robot presented here in this paper makes the use of webcam for acquiring images and instructions rom computer to perform physical movement. Among the important features of this robot is that any of its physical movements can effectively be controlled within millimetre accuracy through the use of sotware. Controlling process has been made much more accurate at the cost of programming complexities. The robot is far more cheaper than any other existing equivalent vision based robot. The programming in sotware made it possible to provide adequate signals with minimum time delay which enabled the robot to manoeuvres luently. Further modiication of this robot includes giving it additional sensors like sonar and inrared [lO]. More powerul program will be written using uzzy logic and neural networks [11][12]. REFERENCES Fig. II Internal arrangement. 863 Fig. 12 Actual photo of the robot. [I] Professor S. G. Tzafestas, Robotics Springer, Intelligent Systems, Control, and Automation: Science And Engineering Volume 43. [2]Vincze,M.; Ayromlou, M. & Kubinger, W (1999). An integrating framework for robust real-time 3D object tracking, Proceedings of the First International Conference on Computer Vision Systems,pp. 135-150, January 1999, LasPalmas, Spain. [3]Buker U. et alii (2001), Vision-based control of an autonomous disassembly station, Robotics and Autonomous Systems, Vo1.35,ppI79I 89robots. Neural Networks II, 1461-1478 (1998) [4] 1. Kramer and M. Scheutz, "Development environments for autonomous mobile robots: A survey," Autonomous Robots, vol. 22. [5] Rafael C. Gonzalez, Richard E. Woods, Steven L. Eddins, 'Digital Image Processing Using MATLAB'. [6] An Introduction to Digital Image Processing with MatlabAlasdair McAndrew School of Computer Science and Mathematics Victoria University of Technology. ICIEV 2012 IEEE/OSAIIAPR Intenational Conference on Informatics, Electronics & Vision [7] Ankur Agrawal ,lIT Kanpur, 'Elementary Introduction to Image Processing Based Robots'. [8] Tim Morris , 'Computer vision and image processing', Basingstoke : Palgrave Macmillan, 2004. [9] James Bruce, Tucker Balch, Mnuela Veloso, 'Fast and Inexpensive Color Image Segmentation for Interactive Robots. [10] M.J.Chantier, 'Detection nd tracking of returns in sector-scan sonar image sequences'. [l1]Floreano, D., Mondada, F.: Evolutionary neurocontrollers for autonomous mobile robots. Neural Networks 11 (1998) 1461-1478 [12]Hagras, H., Pounds-cornish, A., Colley, M., Callaghan, v., Clarke, G.: Evolving Spiking Neural Network Controllers for Autonomous Robots. Proceedings of the 2004 IEEE International Conference on Robotics nd Automation (2004) 4620-4626. 864 ICIEV 2012