Vision-based Egg Grade Classifier

Vision-based Egg Grade Classifier Zalhan, Mohd Zin1; Sera Syarmila, Sameon2; Mohd Nazri, Ismail3; Mohd Taha, Ismail4; Universiti Kuala Lumpur Malaysia France Institute (UniKL MFI), Bangi, Malaysia1; College of Information Technology, Universiti Tenaga Nasional, Kajang, Malaysia2; Faculty of Defence Science and Technology, National Defence University of Malaysia (UPNM), Kuala Lumpur, Malaysia3; Universiti Kuala Lumpur Malaysian Institute of Information Technology (UniKL MIIT) Kuala Lumpur, Malaysia4 [email protected];[email protected];[email protected]; [email protected] ABSTRACT Digital image processing techniques (DIP) have been widely used in various types of application recently. A variety of these techniques are now being used in many types of application area such as object classification, intelligent system, robotics, biometrics system, medical visualization, military, law enforcement, image enhancement and restoration, industrial inspection, artistic effect and human computer interfaces. This paper proposes the implementation of digital image processing techniques to classify three different categories of commercial eggs. The proposed system consists of the study on different types and sizes of commercial eggs, real size measurement of these eggs using Coordinate Measure Machine (CMM) and camera, classification algorithm and the development of vision based egg classification system. Image processing techniques such as image filtering and image enhancements have been applied in the system. Results have shown that the proposed system has been able to successfully classify three categories of commercial eggs with accuracy of more than 96%. Keyword Egg, Image processing, image filtering, object classification, vision system. 1. INTRODUCTION In general, visual object recognition can be divided into two related tasks which are classification and identification. Classification is concerned with the general description of an object belong to a natural class of similar objects such as a face or a car. Identification is a more specific level of recognition. It is recognition of a specific individual within a class such as the particular human face or the brand of a particular car. In human vision, classification is a natural task where we can easily classify an object as a person, car, dog, cat, house and other objects based on its appearance. Even a three-year old child can easily classify a large variety of images of many natural classes. Furthermore, the general classification of an object as a member of a general class such as a car, for example, is usually easier than the identification of the specific make of the car [3]. In contrast, current computer vision systems can deal more successfully with the task of identification compared with classification. This may due to the fact that specific identification requires more detail and finer distinctions between objects compared with general classification, and therefore the task appears to be more challenging. Usually, the main difficulty faced by a recognition and classification system is the problem of variability, and the need to generalize across variations in the appearance of objects belonging to the same class. The visual system is constantly faced with views that are different from all other views seen in the past and it is required to generalize correctly from past experience and classify correctly the novel image. In nature, the variability is complex and it is difficult to provide a precise definition for all the allowed variations of a particular object image. The human visual system somehow learns the characteristics of the allowed variability from experience. This makes classification is more difficult for the artificial systems than individual identification. To identify a specific car for example, the system can be trained with a full and exact model of the object of interest and the expected variations can be described with precision. This is the basis for several approaches to identification, for example, methods that use image combinations [1] or interpolation [2] to predict the appearance of a known object under given viewing conditions. In classification, the range of possible variations is wider. In addition to variations in the viewing condition, we must also contend with variations in shape of different objects within the same class. In this paper, the focus was given to the development of a vision-based classification system that should be able to classify different grades of commercial eggs available in Malaysia. It has the purpose to investigate the ability of egg’s grade classification using computer vision system instead of manual grade classification. The main task is to use image processing techniques to manipulate and process the image and later to provide classification decision to the system. 2. RELATED WORK A variety of different approaches have been proposed previously to deal with visual recognition, including the tasks of general classification and the identification of individual objects. Some of the main approaches are reviewed with focus given to methods that are applicable to classification and related to the approach to be developed. A popular framework to classify an object is that the object’s views will be represented as points in a high-dimensional feature space. After this process, these points in feature space will be partitioned into different regions corresponding to the different classes. Typically, a set of n different measurements are applied to the image, and the results constitute an n-dimensional vector representing the image. A variety of different measures have been proposed, including using the raw image as a vector of grey-level values, using global measures such as the overall area of the object’s image, different moments, Fourier coefficients describing the object’s boundary, or the results of applying selected templates to the image. Partitioning of the space is then performed using different techniques. Some of the frequently used techniques include nearest-neighbor classification to class representatives using, for example, vector quantization techniques, nearest-neighbor to a manifold representing a collection of object or class views [4], separating hyper-planes performed for example by using Perceptron-type algorithms and their extensions, or more optimally by support vector machines [5]. Related to the works on object recognition and classification, this project will classify an object such as egg into different categories. In some countries particularly in Africa, the classification of egg’s categories are done manually by human and assisted by grading machinery [6]. to the third moving axis, called Ruby Probe was used to read the value at any particular point that had been set by user. The value was retrieved once the Ruby Probe touches the points at the egg. In this process, three values at three peak points at one particular side of the egg have been taken as shown in Figure 3. The value chosen was the value of the highest peak points. Here, the focus was given to the classification of different size of 3 different types of commercial eggs using image processing technique. The three egg types are Grade A, B and C. The generic classification framework of this project is shown in Figure 1. Input Image Processing Egg Type Classified Classification Process Result Figure 1: Generic Framework of Egg Classification System 3. METHODOLOGY AND SYSTEM DEVELOPMENT The development of the prototype of this system include several tasks such as study and analyze the different types of the existing commercial eggs in Malaysia, study and develop hardware and software components of the system, build the prototype of the system and analyze the performance of the developed system using the real commercial eggs. The main initial task was to develop a system that is able to classify three different types of commercial eggs. Firstly, the existing commercial eggs had to be studied and analyzed in particular on the determination of the category of an egg. Fifteen commercial eggs from each category, Grade A, B and C have been collected and sampled for this purpose. Each of these egg’s diameter have been measured with very detail measurements because it is their diameters that differentiate them into different grades. In order to obtain very accurate measurement, a Coordinate Measuring Machine (CMM) has been used to measure the diameters of the eggs. CMM is a device for measuring the physical geometrical characteristics of an object. This machine may be manually controlled by an operator or it may be computer controlled. Measurements are defined by a probe attached to the third moving axis of this machine. CMM is widely used in industries and are produced by companies such as Mitutuyo, WillRich and Helmel [7]. In our case, CMM has been used to measure diameter of all commercial eggs. Figure 2 show the how measurement of egg’s diameter has been done. Figure 2: Measurement of egg diameter using CMM During measurement process, one CMM component attached 2 Figure 3: Left Figure shows Ruby Probe touch the egg while right Figure shows three peak points selected by user at both side of the egg. Measurement unit had used is mm. The diameters of fifteen commercial eggs from each category of Grade A, B and C have been measured using this technique. With these measures, the three categories of eggs can be divided into three different respective ranges of diameters as shown in Figure 4. In this figure, it can be noticed that the diameters of grade A eggs were always larger than grade B and C eggs. Grade C eggs have the smallest diameters. Diameter of Eggs by Category 50 48 Diameter(mm) Egg Image Captured 46 Egg A 44 Egg B 42 Egg C 40 38 36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 No. of Egg Figure 4: Diameters of Egg for the three different categories of eggs, Grade A, B and C. Measurement was done using CMM and unit used is in mm. Secondly, the diameters of the same commercial egg’s used previously have been re-measured this time using image processing techniques. It involved the same eggs in their respective orders as been used in the first measurement technique using CMM. In this image-based technique, the egg image has been captured first and then processed using image processing tools. The system used a webcam as a device to capture image and the software used was Matlab R2006a. This software provides variety of image processing tools that could be used to manipulate the captured images later. In our system, a dynamic link library file of vcapg2.dll had to be used in order for the camera to communicate with Matlab and retrieves the image captured. In this second measurement process, egg image was captured and saved in RGB combination colour format. In this format, each pixel consists of value of the combination of three basic colours, red, green and blue. Each of them has value from 0 to 255 at each pixel point. The first step was to convert this colour image into a greyscale image. The greyscale image format consist only white and black colours. At each pixel point, the colour is represented in black and white in the range from 0 to 255. The examples of this conversion process are shown in Figure 5. This grayscale image will facilitate the next image processing process that is image filtering. image. Figure 8: Image in black and white format Figure 5: Left Figure shows an original egg image captured in controlled background while the right Figure shows the same image that has been converted into grayscale format. The original image shown in Figure 5 is an image captured in a controlled background. However, there are also several images captured that have some reflection and shadow of the egg. This problem is known as noise of the image and is shown in Figure 6. Figure 9: Representation of 0 and 1 pixel values The sum of all pixels has been calculated. It represents the surface area of the white image. By having the value of the area, the value of the radius can be calculated as simple mathematical equation below: Radius=√ (area /pi); To overcome this problem, we could improve the environment, illumination and the background surrounding the egg or we could also use image filtering technique on image captured. In our case, the later solution has been applied. The spatial domain technique has been used to filter our image. In this technique, direct manipulation of pixel values has been performed. By using the converted greyscales image, we know that the pixel value of totally black is 0 and totally white is 255, and some shadow part of the image could have a pixel value between 0 and 255. Therefore in order to remove the noise, all pixel having values greater than 100 have been changed to 255 and all pixel having values lower than 100 have been changed to 0. By doing this, the shadow in the image has been removed as shown in Figure 7. Diameter of Eggs by Category 390 370 Diameter(pixel) Figure 6: An egg image with noise The diameter was then calculated as twice of radius. However, the value obtained was still in pixel unit and did not represent any standard metric unit. As mentioned earlier, the image-based measurement process has been performed to all commercial eggs that have been used in CMM measurement technique. The results of the measurements are shown in Figure 10. These results have similar pattern with the results in Figure 4 previously. 350 Egg A 330 Egg B 310 Egg C 290 270 250 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 No. of Egg Figure 10: Diameters of Egg for the three different categories of Grade A, B and C, obtained using image-based technique. The measurement is in pixel. Figure 7: Image with noise removed The sequential process was to convert the filtered image into black and white format. This has been done using Matlab im2bw function. In this format, each pixel now had the value either 1 for white or 0 for black. Figure 8 shows image in black and white format. Using this black and white format, Matlab mathematical functions have been applied to determine the diameter of an egg. Figure 9 shows the representation of the value of pixels in the 3 In our case, after performing comparison, analysis and linearization process to both results obtained, we have determined that one pixel was equal to 0.13 mm. The value was taken from the average of all the values obtained as minimum and maximum values in the three categories of Grade A, B and C. It can be written as: � ���� = ∑ ( �=1 � � � + ��� � � � � )⁄ ��� � ∗ � where n is number of grade and � = { , , } which represent grade of A, B and C. In addition to that, from the results obtained in both measurement techniques, the classification of an egg in our proposed system has been defined in pixel unit as the following pseudo-code: if diameter is more or equal to 355 then classify egg as Grade A; if diameter is between 333 and 355 then classify egg as Grade B; if diameter is below than 333 then classify egg as Grade C; The system has been programmed using the above pseudo-code to determine the category of an egg. A Graphical User Interface (GUI) has also been developed to assist user in using this system. The hardware components of the system are shown in Figure 11. Figure 11: Left Figure shows the box that contains lamp, jig and camera to be used to capture an egg image. Right Figure shows the connection of the camera to PC. The Figure shows also an egg that has been put at the jig of the system 4. EXPERIMENTAL WORKS AND RESULTS The experimental works consist of testing and classifying a commercial egg into its appropriate types of Grade A, B and C. The flow process of the system is shown in Figure 12. Figure 12: Flow process of vision based egg classification system The experiments focused on the accuracy of the system to classify an egg into its grade. The eggs were selected from the existing commercial eggs available in Malaysia market. These eggs were different from the eggs used in Section 3. The number of eggs used was in total 75 and the experimental works for this project have been divided into four different experiments. They were as follows: a) Exp_25A – contains 25 eggs samples of Grade A 4 b) Exp_25B – contains 25 eggs samples of Grade B c) Exp_25C – contains 25 eggs samples of Grade C d) Exp_Mix – contains mixed number of eggs samples from Grade A, B and C During the experimental works, each egg has been put at the jig of the system as shown in Figure 10. The process has been repeated and the results of classification were taken. In the first three experiments, Exp_25A, Exp_25B and Exp_25C, the system has been able to classify all eggs correctly according to their grades. The overall diameters of the eggs have been plotted as in the Figure 13. Figure 13: Results of the three experiments show that the system has been able to correctly classify the eggs according to their respective grades. From the figure, we can observe that in Grade A eggs, the smallest egg has a diameter of 364.669 pixels (47.4 mm) while the largest one has 375.785 pixels (48.9 mm). This Grade A diameter’s range is well situated as defined in our pseudo-code. The same observations can be found in the other two egg’s grades with smallest diameter of 338.308 pixels (43.9 mm) for Grade B and 312.343 pixels (40.6 mm) for Grade C while the highest diameter obtained is 351.085 pixels (45.6 mm) for Grade B and 321.328 pixels (41.8 mm). Figure 14: Results of the fourth experiment shows that the system has been able to perform mostly correct classification of the mix eggs according to their respective grades of A, B and C. One misclassification is occurred at egg sample no 17. In the fourth experiment, a mix number of different grade of eggs has been chosen. These eggs were different from the eggs used in Exp_25A, Exp_25B and Exp_25C. There were also 25 total egg samples used. Each egg has been tested for classification using the same technique as in the previous experiments. Figure 14 show all the egg’s diameters measurements obtained and their grade classifications. In this experiment, the system has also been able to classify nearly all the eggs according their respective grades correctly. The accuracy was 96%. There was one grade misclassification that has been occurred at the seventieth egg sample. In fact, this was a grade B egg and it was misclassified as a grade A egg. In this case, this grade B egg’s diameter was measured as 355.153 pixels (46.19 mm). This diameter was slightly larger than the lower limit of grade A that we have specified in our previous pseudo-code which was set to 355. Therefore, any measurement of diameter above 355 pixels is considered as grade A egg even though the egg was a grade B egg. This can be considered as the limitation of the system when dealing with borderline issue. 5. CONCLUSION AND FUTURE WORKS In this paper, visual based technique has been applied to classify commercial eggs into their respective grades. The development of the system involved review of some literature related to object recognition and classification, investigation of the common manual technique that has been used to classify commercial eggs, high accuracy measurement technique used in CMM, study and analysis of different commercial egg grades, build the prototype of visual based system, implementation several image processing technique, integration of hardware and software and also analysis of the results obtained. The proposed system was able to imitate the vision ability of human to classify different class of object. The object used was commercial egg and the task was to classify it according to its grade visually. This visual-based task is often difficult for a human because there is not too much different in term of egg’s size that can be visualized. The difference of egg’s size in mm is very difficult to be seen by human but the developed system was able to calculate it. From the experimental results, the system has been able to mostly classify the samples of commercial eggs correctly according to their respective grades with only one misclassification case. The lowest accuracy of the system obtained in our experimental works was 96%. In this work, the decision module developed as stated in our pseudo-code was reasonably accurate. Image processing techniques were found useful in the development of this system. However, we have observed that one of the main challenges in this kind of work relates to the quality of the image captured. In our case, the background was controlled by using a specific design box as shown in Figure 11. Like a common problem in object recognition and classification, illumination could become primary factor that may affect the quality of image captured in this system. In this case, image processing technique such as the conversion to black and white image and the calculation of egg’s diameter could produce wrong results. This will decrease the performance of the system. Furthermore, the speed of classification was not measured here as our focus was on 5 vision based classification. It can be observed throughout this work that the proposed system took an average of less than one second to classify an egg, but it could not classify more than one egg at a time. On the other hand, this prototype system can be further improved such as the implementation of colour based object recognition or integration with sensors or Peripheral Interface Controller (PIC). Colour based mechanism could be used to differentiate the object based on its colours. In this case, the system could be improved to classify fruit ripeness, fruit categories etc. In our work, even though visual based system is used, the classification is still done manually using our GUI interface and buttons. Sensors or PIC could be used provide an auto-detection and auto-classification features into the system. In addition to that, the proposed system could also be integrated together with the conveyor system such as egg conveyor system. This integration can improve the conveyor system towards an automated egg sorting system. Finally, the decision module of the system could also be improved by applying artificial intelligence techniques such as fuzzy logic or neural networks in order for the system to deal with boundary case scenario. 6. REFERENCES [1] Ullman, S., Basri, R.: Recognition by Linear Combination of Models. IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 13, 10 (1991) 992-1006 [2] Poggio, T., Edelman, S.: A network that learns to recognize three-dimensional objects. Nature Publisher, 343 (1990) 263-266 [3] M. Stojmenovic, Real time detection in image based on Adaboost machine learning, Proceeding of 12th International Workshop on System, Signal and Image Processing, SSIP-SPI, pp 119-124, Greece, September 2005. [4] Murase, H.,Nayar, S.K.: Visual learning and recognition of 3-D objects from appearance. International J. of Com. Vision, 14 (1995) 5-24. [5] Edgar Osuna, Robert Freund and Federico Girosi, Training Support Vector Machines: Application to Face Detection, Proceeding of Computer Vision and Pattern Recognition, June 17-19, Puerto Rico, 1997 [6] Government of Kingdom of Swaziland official website, http://www.gov.sz, accessed on Sept. 2008 [7] Helmel Engineering Official website, http://www.helmel.com/Coordinate-Measuring-Machines.ht m, accessed on Oct 2008