Manipulator automation for Fresh Fruit Bunch (FFB) harvester

March, 2012 Int J Agric & Biol Eng Vol. 5 No.1 7 Manipulator automation for Fresh Fruit Bunch (FFB) harvester Helena Anusia James Jayaselan, Wan I shak Wan I smail, Desa Ahmad (Department of Biological and Agricultural Engineering, Faculty of Engineering, Universiti Putra Malaysia 43400 Serdang, Selangor Darul Ehsan, Malaysia) Abstract: The need to mechanize major field operations that are labor intensive in oil palm industry of Malaysia has led to the study on agricultural machine automation. In general, study was on machine automation to reduce the number of workers required for harvesting as well as to provide comfortable ergonomic for the operator of oil palm harvester. The objective of the study was to perform interfacing between the oil palm tree and hardware (harvester) as well as to compare the harvesting efficiency between the mechanical and automated manipulator. Kinematic analysis was calculated based on the D-H configuration for the position and orientation of harvester arm using high resolution webcam and ultrasonic sensor to obtain 3D coordinates required by the D-H notations. PIC Circuit Board (PCB) design and fabrication as well as testing and implementation of concept of camera vision operation system for FFB harvester with fully developing a Graphical User Interface (GUI) was conducted to assist the automation of the harvester manipulator. The automation of 5DOF manipulator harvester operation proves to be faster than the manually operated mechanical harvester with an approximation of 60 percent significant decrease in speed of the manipulator with 70 percent of accuracy. Keywords: manipulator, automation, interface, FFB DOI: 10.3965/j.ijabe.20120501.002 Citation: Helena Anusia James Jayaselan, Wan Ishak Wan Ismail, Desa Ahmad. Bunch (FFB) harvester. 1 Manipulator automation for Fresh Fruit Int J Agric & Biol Eng, 2012; 5(1): 7－12. inefficiency of the harvester, automation of the harvester Introduction  was carried out with much faith. The novelty of this The existing mechanized oil palm harvester is research is to transfer the image of FFB (Fresh Fruit claimed to be unsuccessful due to inefficiency in Bunch) to the Denavit & Hartenberg (D-H) model and harvesting fresh fruit bunch. Based on the experiment perform interfacing between the environment with performed before by the MPOB(1990) (Malaysian Palm controllers and hardware for the manipulator automation. Oil Board), the operator of the harvester takes around This study benefits the oil palm industry by increasing the three to five minutes just to adjust the position of the efficiency of the harvesting process by introducing cutter and grabber for one bunch, compared to a labor automation of manipulator of the oil palm harvester. who manages to harvest a tree within a minute. Not general objective is to reduce the number of workers only the operation consumes long time, but also the required for harvesting as well as to provide comfortable operator experiences neck aches and body pain after ergonomic for the operator of oil palm harvester. operating each tree. more specified objective was to perform interfacing So, the ergonomic of the operator was also a major issue here. As a solution to the Received date: 2011-09-13 Accepted date: 2012-02-15 The between the environment and hardware as well as to compare Corresponding author: Helena Anusia James Jayaselan, Department of Biological and Agricultural Engineering, Faculty of Engineering, Universiti Putra Malaysia, 43400 Serdang, Selangor Darul Ehsan, Malaysia. Phone: 6-017-6836550; Email: [email protected]. The the harvesting efficiency mechanical and automated manipulator. between the The study includes design, PIC Circuit Board (PCB) fabrication, testing and implementation of concept of camera vision operation system for FFB harvester with fully developing a Graphical User Interface (GUI) for outdoor distance. agricultural activities. Since a single camera was used, another A robot has human like arm structure, sometimes sensor such as the ultrasonic sensor was used to obtain [1] called anthropomorphic arms . Basically, manipulator the third coordinate to complete 3D information required consists of joints and links where each joints may have by the D-H notations. more than one Degree of Freedom (DOF). In order to The Px, Py and Pz coordinate information were used obtain a large number of position measurements needed for the inverse kinematics calculation to obtain the desired [2] for kinematic calibration, Parker and Gilby proposed angle for the harvester arm movement[4]. The image laser interferometry-based sensing and measuring (LISM) location and calculations were carried out through Matlab technique to perform dynamic measurements of the with the help of the operator to click on the desired robot’s position. Denavit and Hartenberg convention selecting for frames of [3] introduced a reference in robotics application where (D-H) representation has become the standard way of representing robots and modeling their motions. Kinematic analysis was position on the screen[9]. Once the calculations were completed, signal was sent to manipulator to move and harvest the FFB. 2 Methodology calculated based on the D-H configuration for the position The 5DOF mechanical harvester currently located at and orientation of harvester arm, where position of MPOB, Bangi Lama was used for testing with the camera harvester was calculated instantly when all joint variables vision (Figure 1). The machine was developed under were known. Consequently, in order to place the the ‘IRPA’ research grant in collaboration between harvester arm in a desired location, the amount of each Universiti Putra Malaysia (UPM), Malaysian Palm Oil joint movement was calculated through the inverse Board (MPOB) and Universiti Kebangsaan Malaysia [4] kinematic analysis . This was possible with the (UKM). information of the position of the harvester arm with the help of high resolution webcam. A mushroom harvester was automated by Reed et al.[5] to be capable for location, sizing, selection, picking, trimming conveyance and transfer of mushroom using monochromatic camera vision as well as algorithm based on pixel brightness. [6] Giles Similarly, Lee, Slaughter and used automation principle for tomato weed control using computer vision system and selective herbicide application for precise cultivation using solenoid valves. Likewise, the high resolution webcam used to feed the desired position 2D coordinates in the form of pixel which was later converted into meters. Font-Llagunes and Batlle [7] Figure 1 Mechanical harvester located at MPOB, Bangi Lama used a novel technique to estimate a mobile robot pose using odometry and angular The manipulator has the rotations and translation in the discontinuous measurements by laser localization system, order of RTRTR as discussed in Helena[4]. The forward which consists of a rotating laser scanner and a set of kinematics in Equation (2) developed in the section below catadioptric landmarks. [8] Similarly Zhao and Li used using D-H notations on the harvester where the result was laser radar imaging to acquire images in their study, based on the basic notation from Equation (1), represents where a continuous-beam laser was used to send laser the light to the object and collect the returned signals. The transformation between five joints: phase shift in the return signal was used to measure the product of five matrices representing Thus the result of forward kinematics was[4]: the March, 2012 R TH Manipulator automation for Fresh Fruit Bunch (FFB) harvester  nx ox ax Px   ny oy ay Py   = (A1 A2 A3 A4 A5) =   nz oz az Pz    0 0 0 1 (1) 3D coordinates of Px. Py and Pz and was displayed on the GUI screen for the operator’s view[9]. (2) Where C1=cos θ1 and S1=sin θ1. A user friendly Graphical User Interface (GUI) was developed using Matlab for the operator to interact with In this GUI, the video streaming of camera that is placed above the operator was viewed in the first image display while the second image The captured compute the inverse kinematics developed especially for the 5DOF mechanical harvester. At the same time, an ultrasonic sensor was used to measure the distance of the FFB providing the 3rd Inverse kinematics was : A2A3A4A5 = C1nx  S1ny C1ox  S1oy C1ax  S1ay C1Px  S1Py    nz oz az Pz    S1nx  C1ny S1ox  C1oy S1ax  C1ay S1Px  C1Py    0 0 0 1   C 3C 4 C 3S 4 S 3 S 3d 5  C 3a 2  a1  S 3C 4 S 3S 4 C 3 C 3d 5  S 3a 2  =  S4  C4 0 d2   0 0 1  0  θ1= tan-1 [ay / ax] θ1= 0° Thus, C1=1 and S1=0 θ3 = cos-1(-az) dimensional value of ‘z’ to be feed for manipulator computation. The ultrasonic sensor gives its reading in terms of counts which was later converted into centimeters for the inverse kinematics. The operator θ3 = 180° Thus, C3= -1 and S3=0 Then the following is from Equation (9), θ4 = cos-1 (S1ox –C1oy) was required to click on the FFB stalk in the image to obtain the coordinates of the desired point where the cutter is to be placed. Thus to begin with the harvesting process, the operator is required to click tab ‘Run’ to activate the camera and capture the image as in Figure 2. Thus the result of [4] image gives the 2 dimensional values of ‘x’ and ‘y’ in pixels. The 3D coordinates obtained was then passed to the Matlab to C1C3C4 S1S4 C1C3S4 S1C4 C1S3 C1[S3d5C3a2 a1]  S1d2 S1C3C4C1S4 S1C3S4C1C4 S1S3 S1[S3d5C3a2  a1] C1d2     S3C4 S3S4 C3 C3d5 S3a2   0 0 0 1   display shows the still image captured. 9 Then the camera and ultrasonic sensor provides the = the semi-automated system. Vol. 5 No.1 θ4 = 180° Once the angles θ1, θ3 and θ4 were found as 0°, 180° and 180° respectively, they were used to move the harvester rotational joints to the desired position. Matlab was then used for the programming part which will result with the time required to move each joint of the arm to achieve the desired location. The information was then transferred to the Programmable Integrated Controller (PIC) to control the harvester arm motion. Thus, the PIC must always be connected to the laptop or pc to enable information transfer or in other words, as an interface between the pc and the controller. Figure 3 shows the Programmable Circuit Board (PCB) used in this project. It consists of 16F877A programmable Integrated Circuit (PIC), 5 pair of relays, Max 232 (serial port connector microprocessor), RS232 connector, crystal, 0.6 metal film resistors of 10k ohm and 330 ohm, ceramic disc capacitors and power supply circuit. Figure 2 Graphical user interface of oil palm harvester The PIC was suffixed on to a Programmable Circuit Board (PCB) with corresponding relays where two relays were assigned to control one solenoid valve. Figure 3 PCB with MINI40 PIC and relay circuit diagram March, 2012 Manipulator automation for Fresh Fruit Bunch (FFB) harvester The PIC acts as a controller for the harvester arm, deciding on which joint has to move accordingly. This lever since the cutting system was inefficient to be controlled automatically. such a way that it will receive the time (delay in 3 signal the harvester arm to move accordingly. The PIC sends signal to an array of relays, located beside the PIC on a different circuit board, which then sends signal to the respective solenoid valves. Each solenoid valve actuates one double acting cylinder and was controlled by a pair of relays. One relay signals the solenoid valve to extend the cylinder while the other relay 11 Then, the operator will cut the FFB manually using the was preprogrammed into the PIC using C-language, in milliseconds) information from the pc and uses it to Vol. 5 No.1 Results and discussion As mentioned earlier, experiment was conducted to determine the time taken for the end-effectors to move from the home position to the FFB and was repeated three times to examine its consistency as well. The experiment was conducted using digital stopwatch and time was recorded. The results are as shown in Table 1. signals the solenoid valve to retract the same cylinder. Table 1 Operated speed difference between mechanical and Thus a pair of relays was required for the actuation of semi-automated harvester every double acting cylinder. interface between the The array relay acts as the electronic and mechanical components. Thus the time information received from the Matlab was sent to the PIC to actuate the corresponding cylinder to move the respective joint arm. No. Mechanical harvester/min Semi-automated harvester/min 1 3.59 1.25 2 3.27 1.54 3 4.12 1.42 Average 3.66 1.41 This was done for all the joints to enable the arm’s end effectors to move to Table 1 shows that the automated manipulators Thus the cylinders move from their operation of the mechanical harvester proves to be faster home position to the desired position and clamps on the than the manually operated mechanical harvester and was FFB, working in an open loop system as shown in Figure 4. able to move in a rather consistent amount of time. The the desired location. experiment shows an approximation of 60 percent increase in speed of the manipulator which was significant. Then, to ensure accuracy of the joint angles provided by the D-H computation the joint angles were examined manually once the movement was completed. Table 2 shows the percentage of error between the D-H computation joint angle and the joint angle obtained manually upon the movement of the harvester arm to the desired location. The accuracy of the harvester arm was obtained manually using angle measurement apparatus and measurement meter tape. It was determined that there was 70% of accuracy of the oil palm harvester arm coordination using the automation system. It may not be satisfactory, but is a stepping stone for further Figure 4 Flow chart of the semi-automated manipulator harvesting process development of more accurate harvester that will one day completely replace the manual methods of harvesting. Table 2 Difference of percentage error of angle between the actual (manual) and D-H results Experiment 1 No Cylinders 1. Experiment 2 Experiment 3 Ave of error Angle (DH) /(°) Angle (M) /(°) Error /% Angle (D-H) /(°) Angle (M) /(°) Error /% Angle (D-H) /(°) Angle (M) /(°) Error /% C1 182 130 28 176 120 32 170 125 26 2. C2 230 cm 160 cm 20 225 cm 164 cm 27 220 cm 177 cm 19 25 3. C3 105 70 33 103 70 32 102 77 24 30 4. C4 35 cm 25 cm 28 33 cm 27 cm 18 30 cm 28 cm 6 16 5. C5 15 10 33 14 10 28 15 11 26 29 28 Note: D-H – Denavit & Hartenberg, M – manually obtained angle; C1-Cylinder 1, C2-Cylinder 2, C3-Cylinder 3, C4-Cylinder 4, C5-Cylinder 5. 4 [2] Conclusions An investigation of robot arm position measurement using laser tracking techniques system Interface between the environment and the software (pc) was possible through the usage of the high resolution webcam and ultrasonic sensor. Meanwhile interface to measure robot arm performance. machine) was fabricated successfully, known as the PIC Circuit Board (PCB). position the manipulator. Reed J N, Miles S J, Butler J, Baldwin M, Noble R. Res, 2001; 78(1): 15-23. [6] Lee W S, Slaughter D C, Giles D K. system for tomatoes. Robotic weed control Precision Agriculture, 1999; 1: 95- 113. [7] Font-Llagunes J M, Batlle J A. Consistent triangulation for mobile robot localization using discontinuous angular measurements. Robotics and Autonomous Systems, 2009; camera vision operation system for FFB harvester with a agricultural activities was achieved. Kinematics analysis for five DOF Automatic mushroom harvester development. J. Agric Eng fabrication, testing and implementation of the concept of fully develop graphical user interface (GUI) for outdoor Helena A J J, Wan I W I. 3(3): 1-7. [5] A successful automation design involving Denavit & Hartenberg (D-H), PCB Journal of fresh fruit bunch harvester. Int J Agric & Bio Eng, 2010; Therefore, the harvester machine was program, avoiding cramps on the operator’s neck to A kinematic notation for applied mechanics, 1955; pp 215-221. [4] information from the Matlab program and successfully able to move its arm based on instructions from the Jacques D, Hartenberg R S. lower-pair mechanisms based on matrices. Hence the PIC receives passed the information to the respective solenoid valves Sensor Review, 1982; 2(4): 180–184. [3] between the software (pc) to the hardware (harvester on the harvester. Parker G A, Gilby J H. 57(9): 931-942. [8] Zhao D, Li S. A 3D image processing method for manufacturing process automation. Computers in Industry, 2005; 56: 975–985. [References] [1] Manseur R. Robot modeling and kinematics. Da Vinci Engineering Press, Boston Massachusetts. 2006. [9] Helena A J J, Wan I W I, Samsuzana A. Development of sensing system for semi-automated oil palm harvester. Journal of Computers and Electronics in Agriculture, 2011; (Under review).