Development of a camera-vision guided automatic sprayer

ISSN: 0128-7680 © Universiti Putra Malaysia Press Pertanika J. Sci. & Technol. 16 (2): 97 - 105 (2008) Development of a Camera-Vision Guided Automatic Sprayer Wan Ishak, W.I., Hazlina H. and Awal M.A.1* Faculty of Engineering, Universiti Putra Malaysia, 43400 UPM, Serdang, Selangor, Malaysia 1 Bangladesh Agricultural University, Mymensingh, Bangladesh E-mails: [email protected] or *[email protected] ABSTRACT This study describes the design and development of a camera-vision guided unmanned mover sprayer for the purpose of automatic weed control. The sprayer system was mounted on the mover. Modifications were carried out for both sprayer and mover systems, so that it can be operated remotely. The automated system was developed using the electromechanical system and controllers. It is capable of directing the mover sprayer to the target location given by the user. The electromechanical system was developed to control the ignition, the accelerator and the spraying systems. The controllers consist of an I/O module (ICPCON I-87057) and also a pair of radio modems (SST-2400) for data transmission. The graphical user interface (GUI) software to control the automatic system was developed by using Visual Basic Programming. The GUI has features which enable the user to perform desired tasks using the computer instead of going directly to the sprayer/mover. The combination of the multi controllers and developed control software in the development of the camera-vision-guided unmanned mover sprayer can reduce drudgery and increase safety. Keywords: Automation, sprayer and camera vision INTRODUCTION In the early days of plantation development virtually all work was done manually with the aid of only hand tools. At that early period economics of cost was possibly the greatest incentive because labour was plentiful and relatively inexpensive. However, with labour becoming more expensive and scarce, and also with the gradual introduction of various forms of machines, it is becoming more essential to introduce as many new techniques as possible. The use of machines in agricultural production has been one of the most outstanding developments in Malaysian agriculture in the past decades. The burden and drudgery of farm work has been reduced, and output per worker has greatly increased. Equipping a modern agricultural plantation for field operations involves more than merely shopping for functionally adequate machinery. Mechanization has been suggested as a method to improve productivity and making the task more acceptable to workers in the plantation. The production of plantation crops could be increased with improved agronomy and increased mechanized operations. There is a need to mechanize all activities in spraying operations. Modern and efficient methods of spraying systems need to be introduced in plantations. This could be achieved by integrating all activities in spraying with an integrated mechanized system. Received : 8 January 2007 Accepted : 20 June 2007 * Corresponding Author Wan Ishak, W. I., Hazlina H. and Awal M. A. Automation contributes to the expansion of the industrial and agricultural sector. It reduces the needs for unskilled labour, renders obsolete some existing skills, and demands a high level of technical culture (Thomas, 1969). Many different types of technologies such as machine vision, radio frequency, laser and GPS have been attempted for use in navigation of agricultural vehicles (Noguchi et al., 2002; Zuydam et al., 1994; and Choi et al., 1990). These automated technologies can work with any field operation, including planting, cultivating and harvesting. Since modern agricultural machinery is equipped with many controls, operator fatigue is a serious concern (Noh and Erach, 1993; and Tillet, 1991). Automatic guidance can reduce operator fatigue and improve machinery performance by reducing overlapping or skipping during field operations such as tillage and chemical application (Tillet, 1991; Klassen et al., 1993). In this project, two components were modified namely; the ignition system and accelerator system. The ignition system was modified in such a way that the mover can be remotely started within a certain range (maximum range 5km square) without disabling the manual ignition system. A mechanical relay was used for this purpose. To enable automatic ignition or automatic shut off, a radio modem and ICP I/O module (I87057) was used to energize the relay coil. The 12V voltage from the tractor battery will flow through the Common pole and Normally Open pole to the ignition coil or shut off coil to start or off the mover engine. This connection completes the automatic ignition system (Fig. 1). Fig. 1: Schematic diagram output signal of the ignition system The Accelerator system will determine the speed of the mover, forward and reverse action. Two mechanical relays were used to accelerate the mover autonomously. The first relay was connected to the mover’s forward electric motor and the other was connected to the reverse electric motor (Fig. 2). These two relays will only operate when the ICP modules receive signals either to forward or reverse the mover (Saufi, 2002). The accelerator system of the existing unmanned tractor was connected to the camera vision system. The purpose was to control the speed of the unmanned tractor based on the rate of spraying a weedicide. The rate of spraying depends on the intensity of weeds. Vision system is a new field in the agricultural sector. In agricultural application, especially for fruit handling, we cannot detect fruit quality just by its shape or pattern. One type of fruit with variations in shape and pattern may have similar quality. The 98 Pertanika J. Sci. & Technol. Vol. 16 (2) 2008 Development of a Camera-Vision Guided Automatic Sprayer Fig. 2: Schematic diagram output signal of the accelerator system difficulties encountered with vision for agriculture is to control the environment or the objects to be manipulated. The variations in lighting intensity from time to time and place to place cause difficulty in developing a complete vision system in terms of automatic recognition of an object’s color. The use of cameras has failed to obtain sufficient information about fruits, mainly in unconstrained environments, where many factors affect the scene such as weather conditions, color of leaves and their position in relation to the fruit and light contrast (Kondo and Ting, 1998). Thus a method was devised to capture the initial image of weeds and analyze the weeds for color and saved as the reference color of the weeds to be sprayed. In Malaysia most farm production tasks are not fully automated whereby man power is still needed in certain tasks. Therefore, the autonomous sprayer will be part of advanced agriculture production systems in Malaysia. The main purpose of this study was to design and develop an automatic sprayer, which can be controlled remotely. To achieve this objective a preliminary study was conducted to improve the system by designing the automatic sprayer. Image processing algorithm was developed to enable decision-making concerning the presence of weeds. METHODOLOGY The project activities were divided into three major sections. The first section consisted reconfiguration of a previous unmanned tractor system (Saufi, 2002), the second was to design and develop a camera-vision-guided sprayer, and the third was to attach the camera-vision-guided sprayer to an unmanned tractor. Minor modifications were carried out on the previous unmanned tractor system to enable computer control to replace human control. The development of the camera vision guided sprayer was divided into three main categories: (1) Planning and design of the spraying system, (2) Fabrication of the spraying system and (3) Development of the computer control software by using VB program. In the design stage, the length and the size of the galvanized pipes and hoses were first determined. Hoses and fittings to handle the water at selected operating pressure and quantity were selected. Half inch galvanized pipe was used as the main line for liquid Pertanika J. Sci. & Technol. Vol. 16 (2) 2008 99 Wan Ishak, W. I., Hazlina H. and Awal M. A. flow. Galvanized pipe was chosen because of its high resistance characteristics to high pressure and also because it can hold more weight when other heavy components such as solenoid valves and nozzles were attached to it. The nozzle is the critical part of the sprayer. Nozzles determine the rate of water distribution at particular pressure, forward speed and nozzle spacing. A 5.0 mm narrow nozzle was selected for this purpose. Solenoid valves are popularly used in control and automation application. Generally, solenoid operators and valves were devices which control the flow of liquids, gases, steam and other media. When electrically energized they either open, shut off or direct the flow of media. Two types of solenoid valves were used for this project. One was a Normally Open (NO) solenoid valve and the other one was a Normally Close (NC) solenoid valve. NO solenoid valve was mounted at the bottom of the tank for flow by-pass purposes. It was also used to avoid high pressure in the galvanized pipe and hose when the nozzles were not operated, while the NC solenoid valve was mounted at the nozzles for the purpose of ON/OFF of the nozzles. It will turn ON the nozzle when the camera detects the existence of weeds and turn it OFF when the camera does not detect weeds. This kind of ON/OFF function will help to reduce the amount of volume sprayed and therefore help reduce production cost. Controller Development Modules from ICPCON I-87K and radio modem (SST-2400) were selected for data acquisition and control. SST-2400 radio modem is the ‘heart’ of the PC-based control system. They provide digital input/output and other functions. They handle the signal transfer from the computer to the mover. The radio modem communicates with the computer by using the RS-232 serial port. The radio modem receiver receives the signals and transfers it to the ICP Modules (I-87057) via RS-485 bus. The radio modem needs to be configured before it can be used. In this project, only 7 channels from the ICPCON I-87057 were used. DO0 from the ICPCON I-87057 was connected to the NO solenoid valve for flow by-pass action. When the output signal is activated, the relay will be energized and cause electrical supply to the NO solenoid valve and automatically close the valve. DO1, DO2 and DO3 from the ICPCON I-87057 were connected to the NC solenoid valve for nozzles 1, 2 and 3 respectively. When the output signal is activated, the relay energizes and cause electrical flow to the NC solenoid valve and automatically opens the valve to operate the nozzle. Fig. 1 shows the schematic diagram of the automatic ignition system. DO4 from the ICPCON I-87057 was connected to the ignition relay. When the output signal is activated, the relay will energize and cause electrical supply to flow to the ignition coil and automatically start the mover’s engine. Output signals DO5 and DO6 from the ICPCON I-87057 were used to energize the relay for forward and reverse movement of the mover. Fig. 2 shows the schematic diagram of the accelerator system. Software Development In this project, Microsoft Visual Basic 6.0 was used to develop the GUI to monitor the parameters that control the autonomous sprayer operations. Fig. 3 shows the GUI to control the whole system of the autonomous sprayer. With this program, users can give instructions to the autonomous sprayer to perform a specified task. Fig. 4 shows the screen of the monitor being divided into 3 frames captured by one camera. The three frames were for 3 units of spray nozzles. The spray system consists of 2 cameras located 100 Pertanika J. Sci. & Technol. Vol. 16 (2) 2008 Development of a Camera-Vision Guided Automatic Sprayer on the left and right side of the tractor. The weeds captured by the camera in each frame were analyzed to activate the respective nozzles either to open or close. All features that appear in the GUI contained special codes, which were written in Visual Basic. Fig. 5 shows the flowchart of the program. The camera snaps the image and analyzes the red, green, blue (RGB) value in terms of computer pixel. The sprayer nozzle will turn on or off depending on the percentage or intensity of green color value of weeds. Fig. 3: GUI for controlling the camera vision guided unmanned mover sprayer Fig. 4: Boundaries of image frames in GUI Pertanika J. Sci. & Technol. Vol. 16 (2) 2008 101 Wan Ishak, W. I., Hazlina H. and Awal M. A. Fig. 5: Flowchart of the weed detection algorithm Information in Table 1 is useful in determining the task for each output channel of the ICPCON I-87057. The software packages used for ICPCON I-7000/8000/87K series module were I-7000/8000 Utility and NAP 7000X. Table 1 shows the output signals assigned for the ICPCON I-87057. The source codes were written by referring to the information in Table 1. It also helps in the troubleshooting process when problems occur. 102 Pertanika J. Sci. & Technol. Vol. 16 (2) 2008 Development of a Camera-Vision Guided Automatic Sprayer TABLE 1 Output assigning of the ICPCON I-87057 Module I-87057 Channel DO0 DO1 DO2 DO3 DO4 DO5 DO6 Description Bypass Nozzle1 Nozzle2 Nozzle3 Engine (Ignition) Motor (Forward) Motor (Reverse) RESULTS AND DISCUSSION The autonomous sprayer was successfully designed and fabricated. The electrical power from the tractor’s battery was used as a power source, and ICPCON modules were used to control the operation of the autonomous sprayer. Tests were carried out for the whole system including the ignition, accelerator and the spraying systems. The GUI was tested to make sure that it works according to the autonomous sprayer system working process. The accelerator system of the tractor was operated by using a reversible electrical motor. The user can increase tractor speed by pressing the key for forward (DO5) or reverse (DO6) direction. The Spraying system was tested to make sure that all the components were in good working condition. Nozzles calibration was done through a command button on the GUI. Besides using the command button, it can be done manually to operate the nozzle. Each nozzle (nozzle 1, nozzle 2 and nozzle 3) can be operated individually in their respective area of spray. Images that were captured were divided into three frames. The first nozzle can only operate in frame 1, the second nozzle in frame 2 and the third nozzle in frame 3. The boundaries of each frame were determined by the different values of x-axis as shown in Fig. 4. When the user clicks in the region of frame 1, nozzle 1 will be turned ON while the other nozzles will be turned OFF. The NC solenoid valves mounted at the nozzle carry out these functions. Therefore, the user can select the area to be sprayed from the image. Each application of ON operation will take 2 seconds before it turns OFF again. If there is nothing to be sprayed, the NO solenoid valves mounted at the tank will open to by pass the liquid back to the tank. This was used to avoid high-pressure build up in the main sprayer line. The GUI was designed to have an emergency stop button if the user needs to immediately stop the autonomous sprayer. The visual basic programming languages used the basic API pixel routine. The pixel value will extract RGB colour pixels ranging from 0 to 255. The green colour of the weed was selected and was set at the range of plus and minus 10 from the RGB selected pixel values. The green weeds to be sprayed were calculated based on the percentage of intensity of weeds and percentage of green pixels of the weeds. The sprayer pump and the nozzles will be ON at 21 to 100% intensity of weeds and 0.4 to 100% of pixels percent of green weeds. In this project, the performance of the autonomous sprayer was tested. The characteristics monitored included spraying triggering time, the width of spraying and the overlapping width of spraying. The spraying triggering time was monitored as an Pertanika J. Sci. & Technol. Vol. 16 (2) 2008 103 Wan Ishak, W. I., Hazlina H. and Awal M. A. important aspect in the automatic control system. It determines the relationship between the command given in the GUI and the time taken by the sprayer to operate. The width of spraying was measured in order to determine the coverage area being sprayed in one weed spot per nozzle. Besides that, the overlapping width of spraying was also measured. It represents wastage during spraying. From the results, it was shown that there were different pressure build ups in the distribution system (nozzle). The nearer the nozzle to the pump, the higher was the pressure build up in the nozzle with an increase in the spraying width. The triggering time was instantaneous to the nozzle operations. In this project, spraying duration was set at 2 seconds for one application. The mover speed was set at 7 km/h. The sprayer motor operates once the mover moves. The spraying was measured to cover up to 3m width of weed spot per spraying. Therefore, the coverage area was 2467.8 m2 per hour of spraying. With 7km/hr speed and 3 m spray width, it was found that 26.26% of the coverage area was overlapping spraying. This means that wastage will be higher compared to the wastage which occurs by using only one nozzle at a time. CONCLUSIONS Automation is one of the important components in mechanization. In this project, the automated control system for the camera vision guided unmanned mover sprayer was successfully designed and developed. The control system developed combined the technologies of computerized control system, sensor system, control software, wireless data communication, image processing and web-based control system. The user-friendly object oriented VB software was easy to understand which ensures that the autonomous sprayer can be easily operated. The wireless communication unit, SST-2400 radio modem, was easy to configure for several types of communication modes. The ICP modules were able to control the mover and its implement (sprayer). The modules series can acquire most of the needed data to control the mover. Generally, the automated control system for weeding activities was successfully developed and introduced to optimize the use of weedicide application. This autonomous sprayer has the limitation that it is only suitable for use in plant-free areas. REFERENCES CHOI, C.H., CHOI, D.C., ERBACH and SMITH, R.J. (1990). Navigational tractor guidance system. Trans. ASAE, 33(3), 699–706. KONDO, N. and TING, K.C. (1998). Robotics for bioproduction systems. ASAE, St Joseph Michigan, USA. NOGUCHI, N., KISE, M., ISHII, K. and TERAO, H. (2002). Field automation using robot tractor. ASAE Paper No. 701P0502, Conference Publication (pp. 239–245), July 26–27. NOH, K.M. and ERBACH, D.C. (1993). Self-tuning controller for farm tractor guidance. Trans. ASAE, 36(6), 1583–1594. SAUFI, M. K. (2002). Design and development of automated system for agricultural tractor (A Masters Thesis, Intelligent System and Robotic Laboratory, Institute of Advance Technology, University Putra Malaysia, 2002). 104 Pertanika J. Sci. & Technol. Vol. 16 (2) 2008 Development of a Camera-Vision Guided Automatic Sprayer TILLET, N.D. (1991). Automatic guidance sensors for agricultural field machines: A review. J. Agric. Eng. Res., 50(3), 167–187. ZUYDAM, R., VAN, P. and SONNEVELD, C. (1994). Test of an automatic precision guidance system for cultivation implements. J. Agric. Eng. Res., 59(4), 239–243. Pertanika J. Sci. & Technol. Vol. 16 (2) 2008 105