articles by Fernando Alonso-Fernandez
Beautification and augmented reality filters are very popular in applications that use selfie ima... more Beautification and augmented reality filters are very popular in applications that use selfie images. However, they can distort or modify biometric features, severely affecting the ability to recognise the individuals' identity or even detect the face. Accordingly, we address the effect of such filters on the accuracy of automated face detection and recognition. The social media image filters studied modify the image contrast, illumination, or occlude parts of the face. We observe that the effect of some of these filters is harmful to face detection and identity recognition, especially if they obfuscate the eye or (to a lesser extent) the nose. To counteract such effect, we develop a method to reverse the applied manipulation with a modified version of the U-NET segmentation network. This method is observed to contribute to better face detection and recognition accuracy. From a recognition perspective, we employ distance measures and trained machine learning algorithms applied to features extracted using several CNN backbones. We also evaluate if incorporating filtered images into the training set of machine learning approaches is beneficial. Our results show good recognition when filters do not occlude important landmarks, especially the eyes. The combined effect of the proposed approaches also allows mitigating the impact produced by filters that occlude parts of the face.
We address the use of selfie ocular images captured with smartphones to estimate age and gender. ... more We address the use of selfie ocular images captured with smartphones to estimate age and gender. Partial face occlusion has become an issue due to the mandatory use of face masks. Also, the use of mobile devices has exploded, with the pandemic further accelerating the migration to digital services. However, state-of-the-art solutions in related tasks such as identity or expression recognition employ large Convolutional Neural Networks, whose use in mobile devices is infeasible due to hardware limitations and size restrictions of downloadable applications. To counteract this, we adapt two existing lightweight CNNs proposed in the context of the ImageNet Challenge, and two additional architectures proposed for mobile face recognition. Since datasets for soft-biometrics prediction using selfie images are limited, we counteract over-fitting by using networks pre-trained on ImageNet. Furthermore, some networks are further pre-trained for face recognition, for which very large training databases are available. Since both tasks employ similar input data, we hypothesize that such strategy can be beneficial for soft-biometrics estimation. A comprehensive study of the effects of different pre-training over the employed architectures is carried out, showing that, in most cases, a better accuracy is obtained after the networks have been fine-tuned for face recognition.
This article is an open access article distributed under the terms and conditions of the Creative... more This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY
The use of low-resolution images adopting more relaxed acquisition conditions such as mobile phon... more The use of low-resolution images adopting more relaxed acquisition conditions such as mobile phones and surveillance videos is becoming increasingly common in Iris Recognition nowadays. Concurrently, a great variety of single image Super-Resolution techniques are emerging, specially with the use of convolutional neural networks. The main objective of these methods is to try to recover finer texture details generating more photo-realistic images based on the optimization of an objective function depending basically on the CNN architecture and the training approach. In this work, we explore single image Super-Resolution using CNNs for iris recognition. For this, we test different CNN architectures as well as the use of different training databases, validating our approach on a database of 1.872 near infrared iris images and on a mobile phone image database. We also use quality assessment, visual results and recognition experiments to verify if the photo-realism provided by the CNNs which have already proven to be effective for natural images can reflect in a better recognition rate for Iris Recognition. The results show that using deeper architectures trained with texture databases that provide a balance between edge preservation and the smoothness of the method can lead to good results in the iris recognition process. © The Institution of Engineering and Technology 2015
We present a novel face tracking approach where optical flow information is incorporated into a m... more We present a novel face tracking approach where optical flow information is incorporated into a modified version of the Viola-Jones detection algorithm. In the original algorithm, detection is static, as information from previous frames is not considered; in addition, candidate windows have to pass all stages of the classification cascade, otherwise they are discarded as containing no face. In contrast, the proposed tracker preserves information about the number of classification stages passed by each window. Such information is used to build a likelihood map, which represents the probability of having a face located at that position. Tracking capabilities are provided by extrapolating the position of the likelihood map to the next frame by optical flow computation. The proposed algorithm works in real time on a standard laptop. The system is verified on the Boston Head Tracking Database, showing that the proposed algorithm outperforms the standard Viola-Jones detector in terms of detection rate and stability of the output bounding box, as well as including the capability to deal with occlusions. We also evaluate two recently published face detectors based on Convolutional Networks and Deformable Part Models, with our algorithm showing a comparable accuracy at a fraction of the computation time.
Automatically verifying the identity of a person by means of biometrics (e.g., face and fingerpri... more Automatically verifying the identity of a person by means of biometrics (e.g., face and fingerprint) is an important application in our day-to-day activities such as accessing banking services and security control in airports. To increase the system reliability, several biometric devices are often used. Such a combined system is known as a multimodal biometric system. This paper reports a benchmarking study carried out within the framework of the BioSecure DS2 (Access Control) evaluation campaign organized by the University of Surrey, involving face, fingerprint, and iris biometrics for person authentication, targeting the application of physical access control in a medium-size establishment with some 500 persons. While multimodal biometrics is a well-investigated subject in the literature, there exists no benchmark for a fusion algorithm comparison. Working towards this goal, we designed two sets of experiments: quality-dependent and cost-sensitive evaluation. The quality-dependent evaluation aims at assessing how well fusion algorithms can perform under changing quality of raw biometric images principally due to change of devices. The cost-sensitive evaluation, on the other hand, investigates how well a fusion algorithm can perform given restricted computation and in the presence of software and hardware failures, resulting in errors such as failure-to-acquire and failure-to-match. Since multiple capturing devices are available, a fusion algorithm should be able to handle this nonideal but nevertheless realistic scenario. In both evaluations, each fusion algorithm is provided with scores from each biometric comparison subsystem as well as the quality measures of both the template and the query data. The response to the call of the evaluation campaign proved very encouraging, with the submission of 22 fusion systems. To the best of our knowledge, this campaign is the first attempt to benchmark quality-based multimodal fusion algorithms. In the presence of changing image quality which may be due to a change of acquisition devices and/or device capturing configurations, we observe that the top performing fusion algorithms are those that exploit automatically derived quality measurements. Our evaluation also suggests that while using all the available biometric sensors can definitely increase the fusion performance, this comes at the expense of increased cost in terms of acquisition time, computation time, the physical cost of hardware, and its maintenance cost. As demonstrated in our experiments, a promising solution which minimizes the composite cost is sequential fusion, where a fusion algorithm sequentially uses match scores until a desired confidence is reached, or until all the match scores are exhausted, before outputting the final combined score. © 2009 IEEE.
A new multimodal biometric database designed and acquired within the framework of the European Bi... more A new multimodal biometric database designed and acquired within the framework of the European BioSecure Network of Excellence is presented. It is comprised of more than 600 individuals acquired simultaneously in three scenarios: 1) over the Internet, 2) in an office environment with desktop PC, and 3) in indoor/outdoor environments with mobile portable hardware. The three scenarios include a common part of audio/video data. Also, signature and fingerprint data have been acquired both with desktop PC and mobile portable hardware. Additionally, hand and iris data were acquired in the second scenario using desktop PC. Acquisition has been conducted by 11 European institutions. Additional features of the BioSecure Multimodal Database (BMDB) are: two acquisition sessions, several sensors in certain modalities, balanced gender and age distributions, multimodal realistic scenarios with simple and quick tasks per modality, cross-European diversity, availability of demographic data, and compatibility with other multimodal databases. The novel acquisition conditions of the BMDB allow us to perform new challenging research and evaluation of either monomodal or multimodal biometric systems, as in the recent BioSecure Multimodal Evaluation campaign. A description of this campaign including baseline results of individual modalities from the new database is also given. The database is expected to be available for research purposes through the BioSecure Association during 2008. © 2010 IEEE.
One of the challenges in virtual environments is the difficulty users have in interacting with th... more One of the challenges in virtual environments is the difficulty users have in interacting with these increasingly complex systems. Ultimately, endowing machines with the ability to perceive users emotions will enable a more intuitive and reliable interaction. Consequently, using the electroencephalogram as a bio-signal sensor, the affective state of a user can be modelled and subsequently utilised in order to achieve a system that can recognise and react to the userâs emotions. This paper investigates features extracted from electroencephalogram signals for the purpose of affective state modelling based on Russellâs Circumplex Model. Investigations are presented that aim to provide the foundation for future work in modelling user affect to enhance interaction experience in virtual environments. The DEAP dataset was used within this work, along with a Support Vector Machine and Random Forest, which yielded reasonable classification accuracies for Valence and Arousal using feature vectors based on statistical measurements and band power from the and waves and High Order Crossing of the EEG signal. © 2017, The Author(s).
Hand detection and gesture recognition is one of the challenging issues in human-robot interactio... more Hand detection and gesture recognition is one of the challenging issues in human-robot interaction. In this paper we proposed a novel method to detect human hands and recognize gestures from video stream by utilizing a family of symmetric patterns: log-spiral codes. In this case, several log-family spirals mounted on a hand glove were extracted and utilized for positioning the palm and fingers. The proposed method can be applied in real time and even on a low quality camera stream. The experiments are implemented in different conditions to evaluate the illumination, scale, and rotation invariance of the proposed method. The results show that using the proposed technique we can have a precise and reliable detection and tracking of the hand and fingers with accuracy about 98 %. © Springer International Publishing Switzerland 2016.
Hand detection and gesture recognition is one of the challenging issues in human-robot interactio... more Hand detection and gesture recognition is one of the challenging issues in human-robot interaction. In this paper we proposed a novel method to detect human hands and recognize gestures from video stream by utilizing a family of symmetric patterns: log-spiral codes. In this case, several log-family spirals mounted on a hand glove were extracted and utilized for positioning the palm and fingers. The proposed method can be applied in real time and even on a low quality camera stream. The experiments are implemented in different conditions to evaluatethe illumination, scale, and rotation invariance of the proposed method. The results show that using the proposed technique we can have a precise and reliable detection and tracking of the hand and fingers with accuracy about 98 %.
Latent fingerprints are usually processed with Automated Fingerprint Identification Systems (AFIS... more Latent fingerprints are usually processed with Automated Fingerprint Identification Systems (AFIS) by law enforcement agencies to narrow down possible suspects from a criminal database. AFIS do not commonly use all discriminatory features available in fingerprints but typically use only some types of features automatically extracted by a feature extraction algorithm. In this work, we explore ways to improve rank identification accuracies of AFIS when only a partial latent fingerprint is available. Towards solving this challenge, we propose a method that exploits extended fingerprint features (unusual/rare minutiae) not commonly considered in AFIS. This new method can be combined with any existing minutiae-based matcher. We first compute a similarity score based on least squares between latent and tenprint minutiae points, with rare minutiae features as reference points. Then the similarity score of the reference minutiae-based matcher at hand is modified based on a fitting error from the least square similarity stage. We use a realistic forensic fingerprint casework database in our experiments which contains rare minutiae features obtained from Guardia Civil, the Spanish law enforcement agency. Experiments are conducted using three minutiae-based matchers as a reference, namely: NIST-Bozorth3, VeriFinger-SDK and MCC-SDK. We report significant improvements in the rank identification accuracies when these minutiae matchers are augmented with our proposed algorithm based on rare minutiae features. © 2018 Elsevier B.V.
In this study the authors will look at the detection and segmentation of the iris and its influen... more In this study the authors will look at the detection and segmentation of the iris and its influence on the overall performance of the iris-biometric tool chain. The authors will examine whether the segmentation accuracy, based on conformance with a ground truth, can serve as a predictor for the overall performance of the iris-biometric tool chain. That is: If the segmentation accuracy is improved will this always improve the overall performance? Furthermore, the authors will systematically evaluate the influence of segmentation parameters, pupillary and limbic boundary and normalisation centre (based on Daugman's rubbersheet model), on the rest of the iris-biometric tool chain. The authors will investigate if accurately finding these parameters is important and how consistency, that is, extracting the same exact region of the iris during segmenting, influences the overall performance. © The Institution of Engineering and Technology 2016
Imaging using millimeter waves (mmWs) has many advantages including the ability to penetrate obsc... more Imaging using millimeter waves (mmWs) has many advantages including the ability to penetrate obscurants, such as clothes and polymers. After having explored shape information retrieved from mmW images for person recognition, in this paper we aim to gain some insight about the potential of using mmW texture information for the same task, considering not only the mmW face, but also mmW torso and mmW wholebody. We report experimental results using the mmW TNO database consisting of 50 individuals based on both hand-crafted and learned features from Alexnet and VGG-face pretrained convolutional neural networks (CNNs) models. First, we analyze the individual performance of three mmW body parts, concluding that: 1) mmW torso region is more discriminative than mmW face and the whole body; 2) CNN features produce better results compared to hand-crafted features on mmW faces and the entire body; and 3) hand-crafted features slightly outperform CNN features on mmW torso. In the second part of this paper, we analyze different multi-algorithmic and multi-modal techniques, including a novel CNN-based fusion technique, improving verification results to 2% EER and identification rank-1 results up to 99%. Comparative analyses with mmW body shape information and face recognition in the visible and NIR spectral bands are also reported.
The role of soft biometrics to enhance person recognition systems in unconstrained scenarios has ... more The role of soft biometrics to enhance person recognition systems in unconstrained scenarios has not been extensively studied. Here, we explore the utility of the following modalities: gender, ethnicity, age, glasses, beard, and moustache. We consider two assumptions: 1) manual estimation of soft biometrics and 2) automatic estimation from two commercial off-the-shelf systems (COTS). All experiments are reported using the labeled faces in the wild (LFW) database. First, we study the discrimination capabilities of soft biometrics standalone. Then, experiments are carried out fusing soft biometrics with two state-of-the-art face recognition systems based on deep learning. We observe that soft biometrics is a valuable complement to the face modality in unconstrained scenarios, with relative improvements up to 40%/15% in the verification performance when using manual/automatic soft biometrics estimation. Results are reproducible as we make public our manual annotations and COTS outputs of soft biometrics over LFW, as well as the face recognition scores. © 2018 IEEE.
In this paper, we present an integrated research study in On-line Signature Verification undertak... more In this paper, we present an integrated research study in On-line Signature Verification undertaken by several teams that participate in the BioSecure Network of Excellence. This integrated work, started during the First BioSecure Residential Workshop, has as main objective the development of an On-line Signature Verification evaluation platform. As a first step, four On-line Signature Verification Systems based on different approaches are evaluated and compared following the same experimental protocol on MCYT signature database, which is the largest existing on-line western signature database publicly available with 16 500 signatures from 330 clients. A particular focus of work documented in this paper is multi-algorithmic fusion in order to study the complementarity of the approaches involved. To this end, a simple fusion method based on the Mean Rule is used after a normalization phase.
The vulnerabilities of fingerprint-based recognition systems to direct attacks with and without t... more The vulnerabilities of fingerprint-based recognition systems to direct attacks with and without the cooperation of the user are studied. Two different systems, one minutiae-based and one ridge feature-based, are evaluated on a database of real and fake fingerprints. Based on the fingerprint images quality and on the results achieved on different operational scenarios, we obtain a number of statistically significant observations regarding the robustness of the systems. © 2010 Springer Science+Business Media, LLC.
A new software-based liveness detection approach using a novel fingerprint parameterization based... more A new software-based liveness detection approach using a novel fingerprint parameterization based on quality related features is proposed. The system is tested on a highly challenging database comprising over 10,500 real and fake images acquired with five sensors of different technologies and covering a wide range of direct attack scenarios in terms of materials and procedures followed to generate the gummy fingers. The proposed solution proves to be robust to the multi-scenario dataset, and presents an overall rate of 90% correctly classified samples. Furthermore, the liveness detection method presented has the added advantage over previously studied techniques of needing just one image from a finger to decide whether it is real or fake. This last characteristic provides the method with very valuable features as it makes it less intrusive, more user friendly, faster and reduces its implementation costs. © 2010 Elsevier B.V. All rights reserved.
Signal-quality awareness has been found to increase recognition rates and to support decisions in... more Signal-quality awareness has been found to increase recognition rates and to support decisions in multisensor environments significantly. Nevertheless, automatic quality assessment is still an open issue. Here, we study the orientation tensor of fingerprint images to quantify signal impairments, such as noise, lack of structure, blur, with the help of symmetry descriptors. A strongly reduced reference is especially favorable in biometrics, but less information is not sufficient for the approach. This is also supported by numerous experiments involving a simpler quality estimator, a trained method (NFIQ), as well as the human perception of fingerprint quality on several public databases. Furthermore, quality measurements are extensively reused to adapt fusion parameters in a monomodal multialgorithm fingerprint recognition environment. In this study, several trained and nontrained score-level fusion schemes are investigated. A Bayes-based strategy for incorporating experts' past performances and current quality conditions, a novel cascaded scheme for computational efficiency, besides simple fusion rules, is presented. The quantitative results favor quality awareness under all aspects, boosting recognition rates and fusing differently skilled experts efficiently as well as effectively (by training).
Uploads
articles by Fernando Alonso-Fernandez