Eggshell defects detection based on color processing

Header for SPIE use Eggshell defects detection based on color processing María C. García-Alegrea, Angela Ribeiroa, Domingo Guineaa, Gabriel Cristóbalb a Instituto de Automática Industrial, Spanish Council for Scientific Research, 28500 Arganda del Rey, Madrid, Spain. b Instituto de Optica, Spanish Council for Scientific Research, 28006 Madrid, Spain. ABSTRACT The automatic classification of defective eggs constitutes a fundamental issue of the poultry industry for both economical and sanitary reasons. The early separation of eggs with spots and cracks is a relevant task as the stains can leak while progressing on the conveyor-belts, degrading all the mechanical parts. Present work is focused on the implementation of an artificial vision system for detecting in real time defective eggs in a poultry farm. The first step of the algorithmic process is devoted to the detection of the egg shape to fix the region of interest. A color processing is then performed only on the eggshell to obtain an image segmentation that allows the discrimination of defective eggs from clean ones in critical time. The results are presented to demonstrate the validity of the proposed visual process on a wide sample of both defective and non-defective eggs. Keywords: Artificial vision system, color processing, feature extraction, industrial applications, eggs, poultry industry. 1. INTRODUCTION The classification of defective eggs from qualified ones constitutes a fundamental issue of the poultry industry for both economical and sanitary reasons. The processing of poultry eggs for human consumption has four steps: collecting, washing, grading and packaging. While the first, second and fourth steps have been mechanized, the eggs grading step, in which eggs are inspected for defects detection such as blood spots, cracks and dirt stains, is still done manually. In Europe, market regulations do not allow any egg washing process, as it happens in USA, and the early separation of defective and cracked eggs is a fundamental issue to be accomplished as stains and leaks degrade all the mechanical parts while progressing on the mechanized conveyor belts either at the farm or at the grader/packer sites. The European poultry industry has to deal with much more defective eggs due to the lack of the washing stage and eggshell defects appear in a great variety of combinations of the more common defects such as: blood spots, dirt stains, cracks, yolk and white of an egg 1. The inspection and sorting process is currently performed by expert graders but suffers from several drawbacks due mainly to human subjectivity, visual stress and tiredness. On the other hand, eggs with cracks and leakshave to be disregarded as soon as possible as they contaminate all the mechanical parts while progressing in the grader/packer machinery. These two reasons have provoked the integration of an artificial vision system during the eggs manufacturing process in the farm to improve quality control and productivity. Thus, the automation of the grading stage at the poultry farm constitutes a promising and innovative field to alleviate both the manual inspection and the early stage rejection of dirt and cracked eggs. Up to now some research has been done with color image analysis at the poultry 2, 3 and forest 4, 5 products industry, obtaining good results only when a unique defect was present but with strong drawbacks when the image displayed a multiplicity of defects. Related to the poultry industry, some other studies have been performed with monochromatic egg images and neural networks to classify a single well-defined type of defect 6, 7. Present work is focused on the design and implementation of an artificial vision system, for the automatic classification of eggs at the farm/grader manufacturing industry. The first objective has addressed the development of well-fitted algorithms for critical time, capable of enhancing and detecting any kind of inhomogeneous pattern on a regular eggshell background under controlled illumination conditions. 2. VISION SYSTEM The first objective in the automation plan was the implementation of an artificial visual system for critical time able to recognize every type of defect appearing on single stationary egg. A commercial vision system is used to account for the detection of all kind of defects that randomly appear on the eggshell surface in multiple combinations of shapes, colors and aggregations. More common surface irregularities correspond to: feather stuck, blood spots, dirt stains, yolk and white of an egg and cracks. The vision system will be incorporated at the actual mechanized platforms already in use in most Spanish poultry farms, Figure 1. Figure 1. Mechanized conveyor belt with six channels, index finger points to the best location to insert a vision system endowed of six aligned color cameras. Several tests have been initially fulfilled using two video cameras to compare both performance and price: 1) SONY DXC-950P model, endowed of 3 CCD per pixel (RGB/Color), image device size 1/3 inch, 752x582 effective pixels with intensity depth 3x8 bits and speed 30 frames per second (fps) . 2) MITSUBISHI 300E model, 1 CCD per pixel (Color), 1/3”, 752x582, depth 1x8 / 2x8 bits, speed 30 fps. The first camera has been selected based on the low noise level of the three video signals, Red, Green and Blue transmitted through separated channels since the main goal of the work was to develop real time algorithms for color processing,. The camera video signal is digitized at a rate of 30 fps, by means of a Matrox Meteor board and digital images are displayed and stored on a computer Pentium 233Mhz, to be processed later. All recorded images have a resolution of 752x582 pixels. An interactive programming environment 8, has been developed in Visual Basic to integrate both the acquisition of the digital images and the filtering and preprocessing tests to be performed at the first stage to fit all the parameters related to camera and environmental conditions, Figure 2. Currently, the experimental work has been developed at the laboratory testing both lighting conditions to get uniform illumination on the curved eggshell surface and short time algorithms to enhance the defects against the homogeneous eggshell pattern. Main difficulties arise from both low differences in intensity and color between defects, background, and the emergence of a bright spot in the center of the eggshells. To overcome this effect, the illuminant must be carefully selected to be as consistent and uniform as possible. Two light sources have been tested to front-illuminate the egg: 1) Indirect fluorescent light, with a set of tubes located at a distance of 1-1.5m from the CCD camera, yielded by a strip of polyester to scatter the light. 2) Halogen light with tungsten bulbs pointing to the ceiling to provide a homogeneous diffuse light. After several tests, the halogen light was established as the best illuminant. The eggshell reflected light is focused on the CCD camera in a top_ bottom, zenith view, to obtain a uniform illumination on the image. Figure 2. Main front of the interactive program that allows for the acquisition and processing of the collected images. 3. IMAGE PROCESSING First of all the image is processed to obtain the region of interest for subsequent calculations to avoid time-consuming computation of the overall image. To determine the egg shape location, the image is scanned to detect the border of the egg. A threshold is fixed and the pixel position presenting intensity value greater than this threshold is recorded as the beginning and the end of the egg at each image row. Five consecutive values fulfilling this condition are required, for the fifth pixel to be considered the starting_point or the endpoint of the egg shape in that row. This restriction is a variable that can be changed depending on both the image noise and image-background intensity levels, thus adding flexibility to the process. The eggshell profile has been derived from the green color plane of the image as it displayed the highest variation between eggshell and image-background intensities. The inner area is considered the region of interest of the image and all processes will run only in this zone. Due to the randomness of the defects in either shape, size, intensity and position, the analysis has been focused to search for an appropriate color combination image able to enhance the defects through the evaluation of the color discontinuities relative to the homogeneous chromatic eggshell background, Figure 3. Several tests have been performed on a variety of images with different defects subjected to different illumination conditions. All tests have confirmed that most defects show little contrast with regular eggshell background. However, differences in color should point to specific reflectivity between defect and background, whenever light source and sensor are kept constant. This last assert prevents the use of absolute color values as the reflected light from the upper eggshell is always brighter than the one coming from the eggshell borders. a) b) c) e) Figure 3. a) Original color image d) [ (R-B) / (R+B) ]image b) Green color plane, e) d) – d) mean value d) f) c) Eggshell image, 83.525 pixels f) e) binary image, 1.179 active pixels (defects) To face this structural problem of spatially uneven egg reflectivity, an image processing based upon differential values is proposed. Being able to acquire low-level noise images of the three-color planes, image processing has been performed on a normalized color differential image, based upon the following transformation: Differential Color Image = [ (Red color image – Blue color image) / (Red color image + Blue color image) ] The transformation holds for the color intensity differences of the defects that are higher at the wavelengths associated with the red and blue planes, due to specific organic components. Departing from this image, all subsequent image processing is accomplished on this differential color image of the eggshell. The next image processing step consists of the subtraction of its mean value to separate the intensity values of the egg-background from those of the defects. Finally, a threshold is selected to obtain a binary representation of the ROI to speed up the counting process on the remaining active pixels, white pixels, that correspond to the shell irregularities, independently of its shape, size or spatial distribution. The surface fraction of defects is determined by measuring both the total egg area (eggshell pixels in Figure 3c) and the defective egg area (active white pixels in Figure 3f). All the algorithmic processes have been designed as simple as possible, keeping in mind that they had to run on a real time application with short upgrading possibilities but leaving the flexibility required to meet possible changes in the market quality control requirements. 4. EXPERIMENTAL RESULTS To demonstrate the performance of the proposed algorithms, a sample of a hundred eggs, 50 defective and 50 regular, were analyzed for sorting purposes and final inference for acceptance or rejection, based on the ratio between defective/active pixels and eggshell pixels. This computer binary classification, accepted or rejected, is confirmed with a human expert at the sorting stage to correctly fit the tunable parameters towards the desired goal. a) b) c) d) Figure 4. Two original: a) and c) and two binary images with bright defects. b) 3930 active pixels; d) 1166 active pixels a) b) c) d) e) f) g) h) Figure 5. Four original color eggshell images: a) and c) with dark defects, e) and f) with dark + bright spots Four binary images: b) 955; d) 507; f) 583; h) 1332 active pixels respectively The study was accomplished on a sample of regular and defective brown and beige eggs, like those produced by the farm companies involved in the project. Best results are achieved with eggs that show dark, light and (dark + light) spots in any spatial configuration, Figures 4, 5, 6. The threshold, for the binarization of the last image, was set at 50 as a good trade-off between both the restrictions imposed by defective eggs rejection and the safety criteria to deliver clean eggs. With this threshold value, an egg is disregarded when 0.1 percent of its total surface is defective, which corresponds to the usual restrictive criteria used by expert graders. This parameter can be easily changed to account for both the market demands or possible variations in quality control standards. From the sample of 50 regular eggs sorted, following the previously described computer based image process, 82% of the eggs were correctly graded but 18% were rejected as defective. This overpull is costly and some improvements have to be conducted at the image processing stages. Better results are obtained from the sample of 50 defective eggs, wherein 92% were correctly sorted and only 8% were misclassified. By carefully investigating the misclassified images, it can be derived concerning defective eggs, that the wrong sorting always correspond to eggshell having both a unique dark defect with limited area or small spots sprayed all over the egg surface. The last case could be corrected by designing algorithms able to account for the spatial distribution of defects in addition to current global active pixels count, Figure 6. The processing time for individual image processing involved in current automated inspection is displayed in Table 1. Image size is always 752x582, but eggshell (ROI) fills only about one third of the image. Full image visualization is about 100 milliseconds (ms). All color planes are simultaneously obtained through the three RGB channels from the camera to the digitizer. Egg Weight (gr.) Large 75 Small 55 Eggshell shape (ms.) 40 30 Differential Color Index 70 50 Mean subtract 20 15 Binary image 10 8 Then global image processing time ranges from 103 to 140 ms, thus always below the critic time of 150ms allocated for the inspection of one of the three images that have to be inspected on a real rotating egg. On the other hand, processing time can be highly reduced with new computers at speeds higher than 500 MHz and performing parallel image processing on several eggshell regions obtained dividing the eggshell in sections a) Figure 6. Original images of two defective eggs wrongly classified as regular. a) spatial distribution of small spots: 44 active pixels b) small spot aggregation: 73 active pixels For this egg size, about 100 active pixels are needed to be sorted as defective. b) 5. CONCLUSIONS The image processing investigated fulfills the requirements of both high success in grading as well as good performance in critical time. Former studies, due mainly to Patel et al., were accomplished on gray level or color images demonstrating success for the detection of only one defect per image and with classification rates of: 80% for dirt eggs7, 86 % for blood defective eggs 7 and 90% 6 for cracked eggs. Their processing time was always over 1.000 milliseconds and the integration on a real time production system has not been reported yet. Present work offers a promising image processing for real time eggshell inspection with a total time processing close to 100 milliseconds and with classification rates ranging from 82 to 92% in the extraction of any type of defect independent of its aggregation, shape, location or color. Some improvements are already on the way, as were commented in former paragraphs, to increase the classification rate of non-defective eggs in order to avoid the cost associated to the actual overpull. At present time a reduced size prototype of the electromechanical conveyor belts for egg collecting and grading has been installed at the Instituto de Automática Industrial by the Zucami Co, to test current visual processing in real time conditions. ACKNOWLEDGMENTS Present work was fully supported by Research Grants: CAM-06G-038-96: “Automation based on an artificial system at the poultry industry”, CICYT 96-1392-C02-01: “Integrated active vision system”, and the cooperation of several medium size companies: AVICU-DAGU, GRANJA SOLEDAD, ZUCAMI, NEW DATA REAL TIME. REFERENCES 1. 2. 3. 4. 5. 6. 7. 8. Moba Newsletters, Omnia 330: the bold advance, Barneveld, Holland, 1996. V.C.Patel, R.W.McClendon, J.W.Goodrum, “Detection of cracks in eggs using color computer vision and artificial neural networks”, ASAE Annual Intern. Meeting, St.Joseph, 1995. M.C.García-Alegre, J.Enciso, A.Ribeiro, D.Guinea, “Towards an automatic visual inspection of eggshell defects”, Proc. Intern.Workshop on Robotics and Automated Machinery for Bio-Productions, pp.51-56, Gandia, Spain, 1997. R.W.Conners, C.W. McMillin, Ch.N. Ng, “The utility of color information in the location and identification of defects in surfaced hardwood lumber”, Proc. First Intern. Conf. On Scanning Tech. In Sawmilling pp.253-258, , Miller Freeman, San Francisco, 1995. R.W.Conners, D.E. Kline, P.A.Araman, “Parts color matching scanner for edge gluing-Research that works”, Proc. Hardwood Symposium, pp.109-116, 1996. V. C.Patel, R.W. McClendon, J.W.Goodrum, “Crack detection in eggs using computer vision and neural networks”, A.I. Applications, 8, 2, pp.21-31, 1994. V.C.Patel, W. McClendon, J.W. Goodrum, “Detection of blood spots and dirt stains in eggs using computer vision and neural network”, Applied Eng. In Agriculture, 12, 2, pp.253-258, 1996. J.Enciso, R.Sucre, D.Guinea, M.C.García-Alegre, “Entorno Interactivo de Proceso de imagen real:FILEPIC”, Tech.Report TR-01/98, Systems Dpt., IAI-CSIC, 1998. http://www.iai.csic.es/gpa/