Palmprint principal lines extraction

Palmprint Principal Lines Extraction Alessandro Bruno, Paolino Carminetti, Vito Gentile, Marco La Cascia, Emanuele Mancino Dipartimento di Ingegneria Chimica, Gestionale, Informatica, Meccanica. University of Palermo. Palermo, Italy [email protected] Abstract— The palmprint recognition has become a focus in biological recognition and image processing fields. In this process, the features extraction (with particular attention to palmprint principal line extraction) is especially important. Although a lot of work has been reported, the representation of palmprint is still an open issue. In this paper we propose a simple, efficient, and accurate palmprint principal lines extraction method. Our approach consists of six simple steps: normalization, median filtering, average filters along four prefixed directions, grayscale bottom-hat filtering, combination of bottom-hat filtering, binarization and post processing. The contribution of our work is a new method for palmprint principal lines detection and a new dataset of hand labeled principal lines images (that we use as ground truth in the experiments). Preliminary experimental results showed good performance in terms of accuracy with respect to three methods of the state of the art. Keywords— Palmprint; Palm line extraction; segmentation; biometric. I. INTRODUCTION Palmprint identification is a biometric technology that in recent years has obtained attention and it has been rapidly developed. Principal lines are salient features to describe the texture of palmprint. Personal physiological characteristics such as fingerprints, facial features, palmprint, retina, iris, sound and dynamic signature are all features that have been widely used in automatic person identification. Palmprint verification has many unique merits, the most notable is that rich texture information and stable line characteristic can be captured from low resolution image. Today the palmprint recognition has become a focus in biological recognition field. In this process, the feature extraction is especially important. Palmprint pricipal lines play an important role, for this reason, many researcher have focused their attention to the extraction of principal line of palmprint. Observing palmprint, we can find some principal lines, wrinkles and ridges. We usually find three principal lines that are most notable and vary little over time. The detection of the principal lines of the palm is important for individual identification, more particularly, the palm-line strength feature is important to describe palmprint when different palmprints have similar line structures. Although great progress has been made, how to represent palmprint for effective classification is still an open problem. Palmprint-based biometric technology generally is composed of two steps: feature extraction and recognition. Image features are extracted and collected as input vector while the remaining step is to recognize or classify that vector into the suitable class to identify people. We focused our attention on first step, more precisely, on principal palm lines extraction. The information of direction and location of the principal lines and wrinkles can be used to uniquely identify a person and they can be obtained from a low resolution image. These methods are composed of two steps: (1) extraction of palm lines in the palmprint image; (2) extraction and matching of palm-line features. The extraction of palm lines is the base and important works in these methods have been developed in last decade. But because online palmprint images captured by a camera or a scanner are low contrast images and palm lines are irregular and have different directions, shape and depth even in the same palm, the extraction of palm lines is very difficult. In this paper a new method for palmprint principal lines detection is proposed. The method consists of six steps: normalization, median filtering, average filters (along 0°, 45°, 90°, 135° directions); grayscale bottom-hat filtering; combinations of bottom-hat operations; binarization and post processing. Our method has been compared with some methods of the state of the art and preliminary results with respect to accuracy statistical metrics are very encouraging. The contributes of our work are: a new, simple and performant method for palmprint principal lines detection, a new dataset of hand labeled palmprint principal lines images (that is our ground truth). The remainder of the paper is organized as follows: In section 2 an overview of the state of the art of palmprint principal line extraction methods is given, in section 3 we describe the proposed method, in section 4 a description of the preliminary results is given, with particular attention to future extensions, section 5 ends the paper with some discussions about future works. II. STATE OF THE ART In this section we give an overview of the state of the art methods for palmprint principal lines detection. In [1] the authors detected palmprint principal lines by priori knowledge of statistical properties about palm lines. First the authors of the method in [1] considered the direction of principal lines and the features of their valley type edges. Only one directional template of 45 degrees based on local minimum gray value is used to segment palmprint to reduce the excessive noise the multi-directional templates may bring. In [2] [3] [4] the palm line detection is perfomed by interest keypoints. Line segments [5] and lines features [6] are also employed to extract parincipal lines. In [7] the authors performed a compact method to detect principle lines of palmprint is proposed with consecutive filtering operations related to gradient and morphological operators. One of the most popular method in palmprint principal line detection (it is also one of the dated approach in state of the art) has been proposed by Smith and Brady [8]: Non-linear filtering is used to define which parts of the image are closely related to each individual pixel; each pixel has associated with it a local image region which is of similar brightness to that pixel. The new feature detectors are based on the minimization of this local image region the comparisons between our method and some state of the art palm principal lined methods. A. Sample images In this work we only considered ROI (region of interest) image, that is the central part of the hand palm. Furthermore, we used images in grayscale. More particularly, we used only the blue channel. This is motivated by experimental observation, which showed best results, compared to other choices (red or green channels only, or the average of RGB channels). As test set we used images of 128 x 128 spatial resolution and 256 gray levels (8 bit). The most of palmprint lines, as visible in fig. 1, are oriented along directions between 15 and 165 degrees with respect to the horizontal direction. Wang and Ruan in [9] used Steerable Filters to extratc palm lines. Steerable filters can be regarded as a set of orientation filters. Wang and Ruan divided their approach in two stages: global filtering and local filtering to detect coarse and fine palm lines. Wu [10] performed palm line extraction by using a set of directional line detectors and, to avoid losing the details of the palm line structure, irregular lines are represented by their chain code. In [11] the authors detected principal palm lines by using Hildtich algorithm, that is an edge tracking approach to get rid of twigs and short lines, to do thinning. Wu and Wang in [12] performed a method composed of two stages: coarse level extraction stage and fine-level extraction stage. In the first stage, morphological operations are used to extract palm-lines in different directions. In the second stage, for each extracted line, a recursive process is employed to further extract and trace the palm-line using the local information of the extracted part. Wang [13] performed a Palm Lines Extraction method by using PCNN (pulse-coupled neuron network) and Image Data Field. Each pixel in enhanced palmprint images is seen as a particle with the mass that produces a data field. To obtain enhanced palmprint image pixel field data are given as input to two PCNNs with different parameters. Liu [14] proposed a palmline detector method to explicitly extract palm lines, but the detector needs to examine the intensity of the nucleus of the mask and count pixels with similar brightness to the nucleus into a USAN (Univalue Segment Aassimilating Nucleus). This two operations are very time-consuming. Our method, consists of six simple steps, as described in section I, it works in real time and the preliminary experimental results achieve good levels of accuracy with respect to [7] [8] and [14]. In the next section we give a more detailed description of the our algorithm. III. PROPOSED METHOD In this section of the paper we describe the steps of the proposed method for palm line extraction. First of all we give some clarifications and some details about the images we used to test our method. Furthermore we give some details about Fig. 1. All the highlighted principal lines are oriented along directions between 15° and 165° B. Palmline extraction algorithm In order, to better explain each step of our method, we represented the algorithm by using a block diagram. The whole algorithm (fig. 2) has highly parallel structure, this is one of the reason of its efficiency. The scheme below (fig. 2) shows that we used four filters Fi. These filters are implemented as four 5 x 5 matrices in order to selectively blur and process the images. In the following section, we describe each step of the method: normalization; median filter; Fi filters; grayscale bottom-hat opeations; combinations of bottom-hat operations; normalization, binarization and post processing. levels of the image is the consequence of the normalization (see fig. 3). After normalization, we clone the image to produce four identical copies, that we name Img0, Img45, Img90 and Img135. Then, we only apply median filter on Img0. (a) (b) Fig. 3. Normalization of gray levels: contrast of input image (a) is stretched in the output one (b) D. Median filter The next operation should remove as much noise as possible. In order to clean up the image, we use a median filter instead of a low pass filter. A median filter has a minor effect on detail smoothing with respect to a low pass one, while still reducing the salt and pepper noise, removing spikes. (a) (b) Fig. 4. Result of median filter: (a) is the input image, while (b) is the output one E. Fi filters In our method we use four average filters along four directions (0°,45°,90°,135°) that we call F0, F45, F90 and F135. The Image is convolved with the filtering masks in fig. 5, that are the average filters weighted along four directions. In this way, the image ImgX is filtered along the direction at X degrees from the horizontal. Below there are the filters used (fig. 5): Fig. 2. Block diagram to sum up the palmline extraction algorithm. C. Normalization First of all, we need to normalize gray levels of each image, in order to minimize differences among the contrast of input images. Let Img(x,y) the generic gray level of the sample image; the normalization is performed with respect to the maxima of the image. The contrast stretching of the gray Fig. 5. Average Filters along 0°, 45°, 90°, 135° Fi filters are able to highlight lines along the directions at 0°, 45°, 90° and 135°, while other details are smoothed by the average filtering. Because these operations reduce details (and also noise), we don’t apply median filter to all images, but only to Img0, that is filtered with F0. In this way, the double noise reduction (median followed by average) is applied only to the image used to highlight horizontal details. As we wrote in previous section, most of palmprint lines are almost never oriented along horizontal direction. Then at that direction (0°), rarely there are significant lines to be highlighted. This is the reason why we can improve noise reduction strength, despite a greater smoothing of details. where ImgBH is the image after the bottom-hat operation, while S1 and S2 are two structural elements used to perform morphological erosion and dilation. Structural elements are binary matrices that represent elementary geometric structures, on the basis of which are performed the above operation. Using grayscale images, the bottom-hat operation can be approximated by redefining erosion and dilation (such as in formula (2) ). Let Img(x,y) the generic pixel of input image, ImgD(x,y) and ImgE(x,y) the same pixel after dilation and erosion respectively. We can define erosion and dilation as follows: The following images show results after applying Fi filters. (2) a) Img0 b) Img45 c) Img90 d) Img135 where Si(j,k) represent the value at (j,k) of the structural element used for each operation. This operation allow to highlight more significant details, and the image is generally transformed in a low key one. The following images show how bottom-hat operation works. (a) (b) Fig. 7. Result of applying bottom-hat operation: the input image (a) is transformed in a low key image (b), in which details are highlighted. e) f) g) h) G. Combining all bottom-hat operations: sum and cleaning Using bottom-hat operation on each image allowed us to highlight details and, in particular, the palmprint lines that we want to detect. Now, we need to combine them and product a single image, that sum up the results included in each image. To this end, we decided to sum all the images. In fig. 8 the results of this operations are shown. Fig. 6. In this figure Img0, Img45, Img90 and Img135 and the corresponding results after applying Fi filters e) f) g) h) are shown. F. Grayscale bottom-hat operation The bottom-hat filtering is typically used for binary images. It consists in subtracting the original image from the closing of it (from a morphological viewpoint). The latter operation is defined as eroding the dilation of the binary image. In a formula, we can write that: (1) the algorithm (i.e. with respect to the maximum value of the image). Then, we adopted a simple threshold method for the binarization of the image. Binarization can be summarized with the following formula (that is a particular case of the previous one), where M’ is the mean value of the image before the binarization: 0° (5) 45° 90° The last step is the post processing, that is used to remove all the white and isolated spikes (and blocks) in the image. In order to achieve this goal, we use a non-linear filter that can remove them: for each pixel ImgBW(x,y), consider a 5x5 matrix centered on it. If the first and last columns and rows contains only black pixels, set all pixels of the matrix to be black. This post processing operations give, as result, an image with principal lines highlited. 135° a) Fig. 8. Combining the outputs of all bottom-hat operations (a). Normalized image Binarized image As visible in last row of the fig. 8, the result of combination of bottom-hat filterings is a low key image (that is very noisy) in which the most relevant details (palmprint lines) are highlighted. In order, to improve the binarization, that is one of the fundamental steps of the whole algorithm, we use the following mapping to reduce the noise: (3) Post Processed Image Fig. 10. Results of the last steps, which produce the final image. Principal lines are correctly highlighted, the number of isolated white block is reduced. where img(x, y) is the result, M is the mean value of the whole image, and k is a scaling coefficient. In our experiment, we observed that a value of 3 leads to good results, but it is possible to dynamically evaluate this value on the basis of the variance of the image. Fig. 9. Results after applying grayscale cleaning H. Normalization, binarization and post processing Finally, we need to stretch again the contrast of the image, with the same normalization method used in the first step of IV. EXPERIMENTAL RESULTS In this section we give a detailed description of preliminary experimental results, in terms of detection accuracy (precision, recall and F-measure) of our method with respect to other methods of the state of the art. As test set we used a dataset that consists of 1000 images but preliminarly we used 50 images randomly chosen. The test images are 128 x 128 grayscale image. The output of our system for palmprint principal lines extraction is a binary image. Even though we tested our method on only 50 images, preliminary results of our method are very encouraging with respect to some state of the art methods [7] [8] [14]. To evaluate our method, we randomly selected 50 images from two popular palmprint datasets [16] [17], then we carefully labeled by hand each of the 50 images to produce our ground truth [15] for palmprint principal lines extraction. True positives, false positives, false negatives are evaluated with respect to our ground truth (binary hand labeled palmprint images). We evaluated the performance of our approache by using statistical indices, such as precision, recall, F-Measure (equation n.6-7 ): 46%. Finally, by analyzing the value of F-Measure, we show that the result obtained with our method amounts to about 73% that is significantly higher than Zhang and Susan (which amounted approximately to 50%), and Thunkpien (only 42%). V. CONCLUSIONS & FUTURE WORKS (6) (7) MD is the binary version of the detected palmprint map, while MR is the binary version of the reference hand labeled principal lines (ground truth). R is the recall, i.e. the ratio between the number of pixels in the intersection between the detected palm lines MD and the reference map MR, and the number of pixels in MR. When it tends to 1, MD covers the whole MR, but we have no information about pixels outside MR. If it tends to 0 detected and reference map have smaller intersection. P is the precision, i.e the ratio of the number of pixels in the intersection between MD and MR, and the number of pixels in MD. F-Measure represents a weighted average of precision and recall, when F-Measure is equal to 1 it represents the best percentage of accuracy. Fig. 11. In this figure, we show the results of our method with respect to three method of the state of the art. precision, recall, F-measure are used as accuracy metrics. In fig. 11 we show that our method provides a higher accuracy rate than the other state of the art methods. For the index of precision, our method achieves about 80%, whereas the other methods show lower precision values: Zhang 70%, Susan 55%, Thunkpien 68%. The average value of Recall obtained with our method is around 70%, which is approximately twice the ones achieved by Zhang 34%, and more than twice obtained by Thunkpien (30%), while Susan method reached In this paper we proposed a new simple, efficient and accurate palmprint principal lines extraction method. The results obtained using the proposed method show a higher degree of accuracy than the other methods of the state of the art [7] [8] [14]. The test showed the results only for 50 images of the dataset, but the results of precision, recall and F-Measure are encouraging enough to try to extend our method to the full dataset and to extend research in this area. In fact, in order to obtain more reliable results, it would be appropriate to test the proposed method on the remaining images of the dataset. Regarding the future developments, we want to extend the size of our ground truth. We also plan to investigate the use of images of the palm of the hand to perform biometric recognition. In fact, after getting the main lines and possibly after scaling and rotating images, we may perform the operations of matching between images of palmprint for the purposes of identification of individuals. REFERENCES [1] Yuan, Weiqi and Lin, Sen and Tong, Haibin and Liu, Shudong. A detection method of palmprint principal lines based on local minimum gray value and line following. Hand-Based Biometrics (ICHB), 2011 International Conference on. pages 1–5 (2011) IEEE. [2] N. Duta, A.K. Jain, and K.V. Mardia. Matching of palmprints. Pattern Recognition Letters, vol. 23, pp. 477-485, 2002. [3] J. You., W. Li, and D. Zhang. Hierarchical palmprint identification via multiple feature extraction. Pattern Recognition, vol. 35, pp. 847-859, 2002. [4] J. You, W.K. Kong, D. Zhang, and K.H. Cheung. On hierarchical palmprint coding with multiple features for personal identification in large databases. IEEE Trans. Circuits and Systems for Video Technology, vol. 14, pp. 234-243, 2004. [5] D. Zhang and W. Shu. Two novel characteristics in palmprint verification: datum point invariance and line feature matching. Pattern Recognition, vol. 32, pp. 691-702, 1999. [6] L. Zhang and D. Zhang. Characterization of palmprints by wavelet signatures via directional context modeling. IEEE Trans. SMC-B, vol. 34, pp. 1335-1347, 2004. [7] Patprapa Tunkpien, Sasipa Panduwadeethorn, Suphakant Phimoltares. Compact extraction of principle lines in palmprint using consecutive filtering operations. Proceedings of the Second International Conference on Knowledge and Smart Technologies 2010 (July, 24-25, 2010) [8] Smith, Stephen M and Brady, J Michael. SUSAN—a new approach to low level image processing. International Journal of Computer Vision, vol 23(1) pages 45—78 (1997) [9] Wang, Yanxia and Ruan, Qiuqi. Palm-line extraction using steerable filters. Signal Processing, 2006 8th International Conference on. Vol(3) 2006. [10] Wu, Xiangqian and Zhang, David and Wang, Kuanquan. Palm line extraction and matching for personal authentication. Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on. Vol(36) n.5 pages 978—987. 2006 [11] Li, Cong, Fu Liu, and Yongzhong Zhang. A principal palm-line extraction method for palmprint images based on diversity and contrast. Image and Signal Processing (CISP), 2010 3rd International Congress on. Vol. 4. IEEE, 2010 [12] Wu, Xiangqian and Wang, Kuanquan and Zhang, David. A novel approach of palm-line extraction. IEEE First Symposium on MultiAgent Security and Survivability, 2004. pages 230—233. (2004) [13] Wang, Yanxia and Zhao, Jianmin and Sun, Guanghua and Wang, Hui and Chen, Xin and Xu, Dewu. Palm lines extraction using PCNN and image data field. Future Information and Communication Technologies [14] [15] [16] [17] for Ubiquitous HealthCare (Ubi-HealthTech), 2013 First International Symposium on IEEE. Pages 1–-5. 2013 Liu, Laura, and David Zhang. "Palm-line detection." Image Processing, 2005. ICIP 2005. IEEE International Conference on. Vol. 3. IEEE, 2005 https://www.dropbox.com/sh/n91cob64rrmaf1j/AADDGmbMKBV0OKmgB-N8Q8La CASIA Dataset: http://biometrics.idealtest.org/dbDetailForUser.do?id=5 COEP Dataset: http://www.coep.org.in/index.php?pid=367