Multimodal Age and Gender Classification Using Ear and Profile Face Images

Dogucan Yaman* Fevziye Irem Eyiokur* Hazım Kemal Ekenel

Istanbul Technical University, Turkey
{yamand16, eyiokur16, ekenel}@itu.edu.tr

Abstract
arXiv:1907.10081v1 [cs.CV] 23 Jul 2019

In this paper, we present multimodal deep neural net-

work frameworks for age and gender classification, which
take input a profile face image as well as an ear image.
Our main objective is to enhance the accuracy of soft bio-
metric trait extraction from profile face images by addi-
tionally utilizing a promising biometric modality: ear ap-
pearance. For this purpose, we provided end-to-end mul-
timodal deep learning frameworks. We explored different Figure 1. Overview of the multimodal, multitask age and gender
multimodal strategies by employing data, feature, and score classification framework.
level fusion. To increase representation and discrimination
capability of the deep neural networks, we benefited from
ages [3], and combination of them as well [33]. However,
domain adaptation and employed center loss besides soft-
there is only one recent work [30] that performs age classifi-
max loss. We conducted extensive experiments on the UND-
cation from ear images and one that performs age estimation
F, UND-J2, and FERET datasets. Experimental results in-
from profile face images [3].
dicated that profile face images contain a rich source of in-
formation for age and gender classification. We found that In this paper, we presented various end-to-end multi-
the presented multimodal system achieves very high age and modal deep CNN frameworks, which performed multitask
gender classification accuracies. Moreover, we attained su- learning for age and gender classification using ear and pro-
perior results compared to the state-of-the-art profile face file face images as shown in Figure 1. Although profile face
image or ear image-based age and gender classification images, when cropped using a large bounding box, can con-
methods. tain ear appearance, the main motivation to utilize a multi-
modal system instead is to avoid the presence of irrelevant
features. That is, when we expand the bounding box of
the profile face images, hair and background information
1. Introduction are also included in the image, which we found to degrade
Estimating soft biometric traits is a popular research area the performance in our experiments. In the proposed mul-
in biometrics [12, 13, 28, 23]. It has been shown that the soft timodal networks, we explored three different fusion meth-
biometric traits enable to describe subjects better and affect ods. These are: Data fusion —intensity fusion, spatial fu-
the identification performance affirmatively [12, 13, 18]. sion, channel fusion—, feature fusion, and score-level fu-
Age and gender are among the mostly used soft biometric sion. As deep CNN models, we utilized VGG-16 [26] and
traits. ResNet-50 [9]. To increase representation and discrimina-
In recent years, deep convolutional neural network tion capability of the multimodal deep neural networks, we
(CNN) based approaches [17, 22, 30] have been commonly benefited from domain adaptation and employed center loss
used for automatic age and gender classification. Gen- besides softmax loss. In summary, our contributions are
erally, frontal or close to frontal face images have been listed as following:
used in these studies [17, 18, 10, 34, 22]. There have also
been studies about gender classification from ear images • We presented multimodal age and gender classifica-
[7, 14, 16, 30, 11, 1, 19, 24, 21, 5] and profile face im- tion systems that take profile face and ear images as
input. The proposed systems perform end-to-end mul-
* The authors have equally contributed. timodal, multitask learning.
 • We have comprehensively explored various ways of and 92.94% accuracy is obtained. In [3], authors investi-
utilizing multimodal input for age and gender classifi- gated the usability of profile face image in such cases that
cation. We employed three different data fusion meth- frontal face cannot be captured. The experimental results
ods, as well as feature and score level fusion. are conducted on four different datasets. Three types of
ResNet architectures, ResNet-50, ResNet-101, ResNet-152,
• We performed domain adaptation in order to adapt the are employed for feature extraction and sparse partial least-
pretrained CNN models, namely VGG-16 and ResNet- squares regression is used with extracted features. Accord-
50, to the ear domain. For this, we generated the ex- ing to the experiments on FERET dataset, the best result is
tended version of the Multi-PIE ear dataset that was obtained using ResNet-152 features with 5.50 mean abso-
presented in a previous work [6] and named it Multi- lute error (MAE). In [30], the authors present a study on age
PIE extended-ear dataset. Moreover, to learn more dis- and gender classification from ear images. They employed
criminative features, we utilized center loss in combi- both a geometric-based —distances between ear landmarks
nation with the softmax loss. and area information— and an appearance-based represen-
tation —deep CNNs. The authors conducted their exper-
• We conducted extensive experiments on the UND-F, iments on an internal dataset and found that appearance-
UND-J2, and FERET datasets for gender classifica- based representation is more useful [30].
tion, and only on the FERET dataset for age classi- In this work, we improved the previous results on age
fication, since no age information is available in the and gender classification from ear and profile face images
UND-F and UND-J2 datasets. We achieved state-of- by benefiting from domain adaptation, center loss, and mul-
the-art age and gender classification results on these timodal fusion. We have proposed end-to-end multimodal
datasets. deep learning frameworks and further increased the classi-
fication accuracies.
• We presented the first study on age classification using
ear images that is conducted on a publicly available 3. Methodology
dataset, namely, FERET, in contrast to the previous
work [30], in which an internal dataset was used. In ad- In this section, we present employed CNN models and
dition, compared to this previous work, we improved explain proposed multimodal, multitask approach. We also
the age classification accuracy from 52% to 67.59% describe the benefited transfer learning and domain adapta-
for classifying five age groups. tion strategies.
3.1. CNN Models and Loss Functions
2. Related Work
We employed two different well-known deep CNN mod-
In this section, we briefly reviewed the age and gender els, which are VGG-16 [26] and ResNet-50 [9]. In VGG-
classification studies that use ear and profile face images. 16, there are 13 convolutional layers and 3 fully-connected
In [7], authors used ear-hole as a reference point and the layers. To prevent overfitting, dropout method [27] was em-
distance between this point and additional seven points are ployed. Softmax loss was used in order to produce proba-
calculated to extract features. With these features, the best bilities for the classification task. In addition, in this work
result is achieved by a k-nearest neighbor classifier with we utilized combination of softmax loss and center loss. We
90.42% accuracy on an internal dataset, which contains 342 also used weight decay as a regularizer in the training phase
samples. In [33], ear and profile face images are used for the in order to prevent overfitting. Weight decay parameter is
gender classification. Support vector machine (SVM) clas- set to 0.001 in the experiments. The other deep CNN model
sifier is employed with histogram intersection kernel. Score that is used in this work is ResNet-50. In contrast to the
level fusion is utilized to increase the performance. Exper- VGG-16, there are no fully-connected layers except the out-
iments are conducted on the 2D ear images of the UND-F put layer in the ResNet-50. There exists a global pooling
dataset. Multimodal system’s performance is found to be layer between the convolutional part and the output layer.
97.65%, while face-only accuracy is 95.42% and ear-only The input size of both of these networks is 224 × 224.
accuracy is 91.78%. In [14], features are extracted with Ga- We utilized center loss [29] with softmax loss in order to
bor filters and these features are then classified using ma- obtain more discriminative features. The main motivation
jority voting. In total, 128-dimensional features are utilized behind center loss is to provide features that are closer to
and 89.49% classification accuracy is achieved on the UND- corresponding class center. The distance between features
J2 dataset. In [16], gender classification experiments are and related class center is measured and the total center loss
conducted on both 2D and 3D ear images of UND-F and is calculated. Then, center loss part and softmax loss are
UND-J2 datasets. In the experiments, Histogram of Indexed summed to calculate the final loss. The center loss tries to
Shapes features are extracted, SVM is used for classification produce closer features for each class center but it is not
responsible of providing separable features. Thus, this is ear images side-by-side. In the concatenated image, left half
complemented by softmax loss. Besides, for the total loss of the image contains profile face image and right half in-
value, there is a coefficient for center loss that determines cludes ear image, as can be seen from Figure 3 (b). In chan-
the effect over total loss. The formula of the total loss is nel fusion, we have concatenated images along channels.
presented in Equation 1. The first part of the formula is the That is, our input data is [224, 224, 3] dimensional, after
softmax loss and the second part is the center loss. In the channel-based concatenation our input data becomes [224,
center loss formula, cyi represents yi th class center and xi th 224, 6]. While the first three channels belong to profile face
feature. In the experiments, λ coefficient is selected as 0.1 image, the remaining 3 channels contain ear image. Finally,
according to the results obtained on the validation set. in intensity fusion, we average pixel intensity values of the
profile face and ear images as shown in the Figure 3 (c).
m T m
X eWyi xi +byi λX
L=− log Pn T
Wj xi +bj
+ ||xi − cyi ||22 (1) 3.3.2 Feature Fusion
2
i=1 j=1 e i=1
For the feature-based fusion strategy, we have trained two
3.2. Multimodal and Multitask CNN separate CNN models, one of them takes profile face im-
age as input and the other one takes ear image as input.
We investigated the performance of age and gender clas- While the representation part (convolutional part) of these
sification using ear and profile face images both separately, networks have been trained separately, the outputs of the
as unimodal systems, and together as a multimodal, multi- last convolution layers have been concatenated and fed to
task system. As can be seen from Figure 2, we explored the classifier part. For example, for VGG-16, thirteen con-
three end-to-end multimodal architectures. For all experi- volutional layers are the separate part of the networks and
ments, we have employed VGG-16 [26] and ResNet-50 [9] three fully connected layers are the common part of the mul-
models with center loss [29]. For the total loss calculation timodal system. In VGG-16, the size of the output vector of
in multimodal, multitask age and gender classification ex- each representation is 1 × 4096. Two vectors are then con-
periments, we have combined all losses that come from age catenated making a 1×8192 dimensional feature vector and
and gender prediction of the network. The final loss is cal- fed to the fully connected layers. For ResNet-50, we have
culated using the formula presented in Equation 2. combined the outputs of the last layer before global average
pooling layer, then the combined vector is passed through
global average pooling layer and a fully-connected layer,
Ltotal = β ∗ (Ltotalage ) + (1 − β) ∗ (Ltotalgender ) (2)
respectively.
According to the above formula, the calculation of total
age and gender losses are based on summation of the soft- 3.3.3 Score Fusion
max loss and multiplication of the center loss with λ coef-
ficient as explained before. After the measurements of the In this method, we have performed two individual training
total age and gender losses, we have combined these loss with different CNN models. While one CNN model has
values using β coefficient. The main motivation behind in- been trained on profile face images, the other one has been
troducing the β coefficient is that since gender classification trained with ear images. Then, for the score-based fusion,
result is significantly high, we have tried to highlight age we have tested each profile face image and ear image with
classification loss for the final loss. With this coefficient, we related models. After that, for each profile face and ear im-
can basically change the effect of age-specific and gender- age that belong to the same subject, we have acquired prob-
specific losses over total loss. In the experiments, we have ability scores and measured the confidence score of each
tried several different β coefficient and according to the ob- model according to five different confidence score calcula-
tained results on the validation set, we have achieved the tion methods that are presented in Table 1. Later, we have
best performance with β = 0.75 value. selected the prediction of the model that has the maximum
confidence score.
3.3. Fusion Methods
3.4. Transfer Learning and Domain Adaptation
In this section, we present the multimodal fusion meth-
ods that are illustrated in Figure 2. For convolutional neural networks, it is very common
and have been shown to be useful [32, 25, 18] to bene-
fit from previously trained models, more specifically the
3.3.1 Data Fusion
models that have been trained on the large-scale ImageNet
To perform data fusion, we have employed three different dataset [4]. This approach, transfer learning, helps to adapt
methods, namely spatial fusion, intensity fusion, and chan- the successful CNN models to the problems, where only
nel fusion. In spatial fusion, we concatenate profile face and a limited amount of data is available. A way of applying
Figure 2. Multimodal fusion methods. (a) presents employed three different data fusion methods. In the first one, named as intensity fusion,
we have averaged profile face and ear images’ pixel intensity values. In the second approach, spatial fusion, we have combined profile face
and ear images side-by-side. In the last method, channel fusion, we have concatenated profile face and ear images in depth, i.e. along their
channels. (b) presents feature fusion. The representation part of the networks has been trained separately and outputs of these parts have
been concatenated. This concatenated feature vector then fed to the classification part. In (c), score fusion, two different networks have
been trained separately, then score-based concatenation has been performed according to the probability scores of the networks.

transfer learning is to perform fine-tuning. In this approach, Since it has been shown that [18] transferring a pre-
the parameters of the pretrained model are used to initialize trained model from a closer domain leads to better perfor-
CNN models’ weights instead of using random initial val- mance, we have benefited from a large-scale ear dataset,
ues. Then these model weights are further updated using the which is named as Multi-PIE ear dataset [6]. As the name
target dataset. During fine-tuning, depending the similarity implies, this dataset was constructed by running an auto-
between target dataset domain and pretrained dataset do- matic ear detector on the profile and close-to-profile face
main, and the size of the target dataset, some layers’ weights images of the Multi-PIE face dataset [8]. This way, 17183
can be frozen or all layers’ weights can be updated. ear images of 205 subjects were detected [6]. In this work,
Figure 3. Visualization of employed data fusion approaches. (a)
contains original ear and profile face images. (b) represents side-
by-side concatenation, which is spatial fusion and (c) is the aver-
age of the profile face and ear images, that is intensity fusion.

Method Formula
Figure 4. Sample images from FERET dataset [20]. The first col-
Basic c = s[0] umn contains the original images, the second column contains
d2s c = s[0] − s[1] sample detected ear images, and the last column contains detected
d2sr c = 1 − (s[1]/s[0]) profile face images.
PM
avg-diff c = M1−1 i=1 (s[0] − s[i])
PM −1
diff1 c = i=1 ( s[i−1]−s[i]
i )
4. Experimental Results
Table 1. Confidence score calculation methods for score-based fu-
sion. In the formulas, c represents confidence score of the corre-
In this section, we provide information about used
sponding CNN model and s is an array that contains probabilities datasets, experimental setups, implementation details, and
for each class from high to low value. our results, respectively.

4.1. Datasets
we have extended Multi-PIE ear dataset [6] and named it Multi-PIE Extended-ear Dataset contains 39185 ear
as Multi-PIE extended-ear dataset. The coordinates of the images. This dataset was constructed by running an auto-
detected ear images of the Multi-PIE ear dataset [6] have matic ear detector on the profile and close-to-profile face
been shared A . We have used these ear coordinates and ad- images of the Multi-PIE face dataset [8]. As explained in
ditionally, we have executed our improved ear detection al- the Section 3.4, this dataset was used to adapt the pretrained
gorithm on the other images of Multi-PIE dataset that were CNN models to the ear domain.
not listed in the Multi-PIE ear dataset. This way, we have FERET [20] is one of the most well-known face
acquired new ear images in addition to the ones in the Multi- datasets. Sample images from FERET are shown in Fig-
PIE ear dataset [6] and reached 39185 ear images. The co- ure 4. For our experiments, we used both ear and profile
ordinates of this extended-ear dataset will be shared B as face images. We utilized dlib library [15] to detect profile
well. Compared to the Multi-PIE ear dataset, we provided faces. In order to detect the ear regions, we run an off-
a significant increase in the dataset size. For training, we the-shelf ear detector [2]. This way we obtained 1397 ear
have first initialized the CNN models with the parameters images.
of the pretrained CNN models that were trained on the Im-
UND-F [31] contains both 2D and 3D ear images. In this
ageNet dataset. Then, by utilizing the Multi-PIE extended-
work, we only used 2D ear images. There are 942 profile
ear dataset, we have updated the CNN models and adapt
face images that belong to 302 different subjects. We run
them to the ear domain. Afterwards, we have benefited
the aforementioned off-the-shelf ear detector [2] and crop
from adapted pretrained models and further updated them
ear regions from profile face images. We utilized dlib li-
by training on the target ear datasets. Experimental results
brary [15] for detecting profile face images. This dataset
have validated that this learning strategy, that is by perform-
was employed to benchmark gender classification accuracy,
ing domain adaptation via an intermediate fine-tuning step,
since it was also used in the previous work on gender clas-
and by this way using the CNN models that were adapted
sification from ear images [16, 33].
to the ear domain instead of directly using the CNN models
UND-J2 [31] is another dataset that was utilized to
that were trained on a generic image classification dataset,
benchmark gender classification accuracy in previous work
helped to improve the performance of both age and gender
[14, 16]. It includes 2430 profile face images. As in the
classification tasks.
UND-F dataset, we detected the ear regions by applying the
A https://github.com/iremeyiokur/multipie ear dataset off-the-shelf ear detector [2] and we employed dlib [15] to
B https://github.com/iremeyiokur/multipie extended ear dataset detect profile faces.
 Model Data Age Acc. Gender Acc. Model Fusion Age Acc. Gender Acc.
VGG-16 Ear 60.97% 97.56% VGG-16 A-1 61.83% 92.33%
VGG-16 Profile 65.73% 95.81% ResNet-50 A-1 57.49% 91.63%
ResNet-50 Ear 60.97% 98.00% VGG-16 A-2 67.59% 99.11%
ResNet-50 Profile 62.37% 94.05% ResNet-50 A-2 62.71% 99.11%
VGG-16 A-3 62.05% 93.03%
Table 2. Unimodal age and gender classification results on FERET ResNet-50 A-3 58.53% 92.33%
dataset [20]. VGG-16 B 67.28% 98.16%
ResNet-50 B 66.44% 97.56%
4.2. Implementation Details VGG-16 C-1 63.76% 97.90%
ResNet-50 C-1 63.06% 98.00%
In order to have domain adapted pretrained deep models, VGG-16 C-2 63.06% 97.90%
we have performed fine-tuning on Multi-PIE extended ear ResNet-50 C-2 62.02% 98.00%
dataset with VGG-16 [26] and ResNet-50 [9] CNN models. VGG-16 C-3 63.06% 97.90%
For this, we have splitted Multi-PIE extended ear dataset ResNet-50 C-3 62.02% 98.00%
into training, validation, and test sets. 80% of the images VGG-16 C-4 63.76% 97.90%
have been assigned to the training set, and the remaining ResNet-50 C-4 63.06% 98.00%
20% has been assigned evenly to the validation and test sets. VGG-16 C-5 63.76% 97.90%
The same percentage of training, validation, and test splits ResNet-50 C-5 63.06% 98.00%
are also applied in the age and gender classification experi-
ments on FERET, UND-F, and UND-J2 datasets. Table 3. Age and gender classification results of the three differ-
For age classification task, we have five different age ent fusion methods that are explained in Section 3.3. In the fusion
groups based on the following age ranges: 18-28, 29-38, column, A, B, and C correspond to data, feature, and score fu-
39-48, 49-58, 59-68+. These classes have been selected ac- sion methods, respectively. In method A, A-1, A-2, and A-3 are
channel fusion, spatial fusion, and intensity fusion methods, re-
cording to the previous work about age classification from
spectively. In C, C1, C2, C3, C4, and C5 represent different con-
ear images [30]. According to the [30], the changes in the
fidence score calculation methods that are presented in Table 1.
ear can be observable between these age groups. These age C-1 means the first formula of the Table 1 is used, C-2 means the
classes include 419, 435, 316, 169, and 58 ear and profile second formula in the table is employed and etc.
face images. While the number of images for train, valida-
tion, and test is 1110, 144, and 143 respectively. Note that
we have split data into train-validation-test sets with respect According to Table 2, we have achieved the best age clas-
to the data distribution per class. sification result with VGG-16 model using profile face im-
In all experiments, we have set learning rate to 0.0001. ages with 65.73% classification accuracy. While ResNet-50
We have dropped learning rate by 0.1 in every 25 epochs. performance on profile face images is slightly lower than
We have also used L2 regularization with 0.001 coefficient VGG-16 performance, both models have achieved similar
and center loss [29] with 0.1 coefficient as mentioned in accuracy with ear images. The best gender classification
Section 3.1. Moreover, to prevent overfitting, we have result which is 98.00%, is obtained with ear images using
dropped neurons with 75% probability value during train- ResNet-50 CNN model and VGG-16 result is very close to
ing. On the other hand, we have not dropped them in test this result. Although age classification performance is bet-
phase. While we have selected batch size as 32 for the uni- ter with profile face images, ear images are found to be more
modal experiments, we have chosen batch size as 16 during useful than profile face images in gender classification. All
multimodal experiments due to memory constraints. these results indicate that ear and profile face images con-
tain useful cues for age and gender classification. However,
4.3. Unimodal Results age classification needs further investigation because of the
In this section, we present unimodal experiments, which relatively low performance compared to that of gender clas-
are based on profile face and ear images separately. We sification.
have fine-tuned VGG-16 [26] and ResNet-50 [9] CNN mod-
4.4. Multimodal Results
els for age and gender classification tasks with FERET
dataset [20]. The obtained results are presented in Table 2. In this section, we have presented multimodal and multi-
In this table, first column contains used CNN model, second task age and gender classification experiments using VGG-
column contains information about data, which can be ear 16 and ResNet-50 CNN architectures. Table 3 shows age
image or profile face image and, finally, last two columns and gender classification results based on different fusion
show test accuracies of age and gender classification. methods. First column contains name of the employed
 Method/Model Dataset Data Accuracy All these results indicate that the spatial fusion and
Gender Classification feature-based fusion methods are effective multimodal fu-
Distance+KNN [7] Internal Ear 90.42% sion strategies for extracting soft biometric traits from pro-
GoogLeNet [30] Internal Ear 94.00% file face and ear images.
BoF+SVM [33] UND-F Ear 91.78%
HIS+SVM [16] UND-F Ear 92.94%
HIS+SVM [16] UND-J2 Ear 91.92% 4.5. Comparison with State-of-the-art
Gabor+Voting [14] UND-J2 Ear 89.49%
In Table 4, we have compared age and gender classifi-
BoF-SVM [33] UND-F Profile 95.43%
cation performance of our proposed method with previous
BoF-SVM [33] UND-F Multi 97.65%
works. We have achieved the state-of-the-art performance
Ours FERET Multi 99.11%
on both tasks, however, used dataset in this work is not the
Ours UND-F Multi 100%
same as the ones used in previous work [30] for age clas-
Ours UND-J2 Multi 99.79%
sification task. Because of that, we have implemented and
Age Classification used the previous work on age classification [30]. We have
Yaman et al. [30] Internal Ear 52.00% followed the same strategy with them and presented the ob-
Yaman et al.* [30] FERET Ear 58.53% tained result with * symbol in the table. Besides, we have
Ours FERET Multi 67.59% conducted gender classification experiments on the UND-F
and UND-J2 datasets [31], which are benchmark datasets
Table 4. Comparison of the proposed methods with previous
for gender classification from ear images. As a result, we
works. While first part contains gender classification results, sec-
ond part presents age classification results. According to the re- have obtained the state-of-the-art results on both datasets
sults, we have achieved the state-of-the-art results for age and with our spatial fusion method and feature-based fusion
gender classification. We have implemented the proposed method method. We have achieved 100% accuracy on the UND-
in [30] and to have a fair comparison, we have tested it also on F dataset [31] and 99.79% accuracy on the UND-J2 dataset
FERET dataset [20]. The result of this method on FERET dataset [31]. For age classification, we have otained 67.59% accu-
is presented with * symbol. racy, with spatial fusion, that is around 9% better than the
accuracy achieved by the method presented in [30].

CNN model as in the unimodal experiments. Second col-

umn contains fusion methods that are data (A), feature (B), 5. Conclusion
and score (C) fusion methods, respectively. In this column,
A-1 indicates channel fusion, A-2 indicates spatial fusion, In this paper, we presented different end-to-end multi-
and A-3 indicates intensity fusion that are explained in Sec- modal deep learning frameworks for age and gender clas-
tion 3.3.1. The B is the feature fusion method that is also sification, which takes profile face and ear images as input.
explained in Section 3.3.2. Lastly, for C, which is the score- We employed domain adaptation in order to adapt the pre-
level fusion, we have used five different confidence score trained CNN models, namely VGG-16 and ResNet-50, to
calculation methods. These methods are explained in Sec- the ear domain. To learn more discriminative features, we
tion 3.3.3. Next two columns present test accuracies of the combined center loss with softmax loss. We conducted ex-
age and gender classification tasks, respectively. tensive experiments on the UND-F, UND-J2, and FERET
According to the experimental results, the best age clas- datasets for gender classification, and on the FERET dataset
sification performance is achieved with VGG-16 model us- for age classification. We showed that by combining profile
ing A-2 fusion method which is the spatial fusion. However, face and ear image, we can achieve very high accuracies.
the feature-based results, B, are pretty close to the data fu- Spatial fusion and feature fusion methods have led to signif-
sion method and feature fusion with ResNet-50 is around icant performance improvements. We achieved the state-of-
4% better than all the other fusion types with ResNet-50 the-art gender classification accuracies on the UND-F and
model. Generally, except spatial fusion method, other two UND-J2 datasets. In addition, we improved the age classi-
data fusion methods are not as powerful as feature-based fication accuracy. To the best of our knowledge, this is the
and score-based fusion strategies especially for gender clas- first study on age classification using ear images that is con-
sification. ducted on a publicly available dataset, FERET, in contrast
In gender classification experiments, we have achieved to the previous work [30], in which authors used an internal
the best performance again with spatial fusion as in the age dataset. Compared to the reported age classification accu-
classification experiments. Both the VGG-16 and ResNet- racy in this previous work [30], we achieved around 9% im-
50 CNN models achieve 99.11% classification accuracy provement for five class age group classification on FERET
with spatial fusion. dataset.
References [19] A. Pflug and C. Busch. Ear biometrics: A survey of detec-
tion, feature extraction and recognition methods. IET Bio-
[1] A. Abaza, A. Ross, C. Hebert, M. A. F. Harrison, and M. S. metrics, 1(2):114–129, 2012. 1
Nixon. A survey on ear biometrics. ACM Computing Sur- [20] P. J. Phillips, H. Moon, S. A. Rizvi, and P. J. Rauss. The
veys, 45(2):22, 2013. 1 FERET evaluation methodology for face-recognition algo-
[2] G. Bradski and A. Kaehler. OpenCV. Dr. Dobbs Journal of rithms. IEEE Transactions on Pattern Analysis and Machine
Software Tools, 3, 2000. 5 Intelligence, 22(10):1090–1104, 2000. 5, 6, 7
[3] A. M. Bukar and H. Ugail. Automatic age estimation from [21] R. Purkait and P. Singh. Anthropometry of the normal hu-
facial profile view. IET Computer Vision, 11(8):650–655, man auricle: A study of adult Indian men. Aesthetic Plastic
2017. 1, 2 Surgery, 31(4):372–379, 2007. 1
[4] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei- [22] R. Rothe, R. Timofte, and L. Van Gool. Deep expectation
Fei. Imagenet: A large-scale hierarchical image database. In of real and apparent age from a single image without fa-
Computer Vision and Pattern Recognition, pages 248–255. cial landmarks. International Journal of Computer Vision,
IEEE, 2009. 3 126(2-4):144–157, 2018. 1
[5] Ž. Emeršič, V. Štruc, and P. Peer. Ear recognition: More than [23] U. Saeed and M. M. Khan. Combining ear-based traditional
a survey. Neurocomputing, 255:26–39, 2017. 1 and soft biometrics for unconstrained ear recognition. Jour-
[6] F. I. Eyiokur, D. Yaman, and H. K. Ekenel. Domain adapta- nal of Electronic Imaging, 27(5):051220, 2018. 1
tion for ear recognition using deep convolutional neural net- [24] C. Sforza, G. Grandi, M. Binelli, D. G. Tommasi, R. Rosati,
works. IET Biometrics, 7(3):199–206, 2017. 2, 4, 5 and V. F. Ferrario. Age-and sex-related changes in the normal
[7] P. Gnanasivam and S. Muttan. Gender classification using human ear. Forensic Science International, 187(1-3):110–e1,
ear biometrics. In International Conference on Signal and 2009. 1
Image Processing, pages 137–148. Springer, 2013. 1, 2, 7 [25] A. Sharif Razavian, H. Azizpour, J. Sullivan, and S. Carls-
[8] R. Gross, I. Matthews, J. Cohn, T. Kanade, and S. Baker. son. CNN features off-the-shelf: An astounding baseline for
Multi-PIE. Image and Vision Computing, 28(5):807–813, recognition. In Computer Vision and Pattern Recognition
2010. 4, 5 Workshops, pages 806–813, 2014. 3
[9] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learn- [26] K. Simonyan and A. Zisserman. Very deep convolutional
ing for image recognition. In Computer Vision and Pattern networks for large-scale image recognition. arXiv preprint
Recognition, pages 770–778. IEEE, 2016. 1, 2, 3, 6 arXiv:1409.1556, 2014. 1, 2, 3, 6
[10] Y. He, M. Huang, Q. Miao, H. Guo, and J. Wang. Deep em- [27] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and
bedding network for robust age estimation. In International R. Salakhutdinov. Dropout: A simple way to prevent neural
Conference on Image Processing, pages 1092–1096. IEEE, networks from overfitting. The Journal of Machine Learning
2017. 1 Research, 15(1):1929–1958, 2014. 2
[28] D. A. Vaquero, R. S. Feris, D. Tran, L. Brown, A. Hampapur,
[11] A. Iannarelli. Ear identification, forensic identification se-
and M. Turk. Attribute-based people search in surveillance
ries. Paramont Publ Company, 1989. 1
environments. In Workshop on Applications of Computer
[12] A. K. Jain, S. C. Dass, and K. Nandakumar. Soft biometric
Vision, pages 1–8. IEEE, 2009. 1
traits for personal recognition systems. In Biometric Authen-
[29] Y. Wen, K. Zhang, Z. Li, and Y. Qiao. A discrimina-
tication, pages 731–738. Springer, 2004. 1
tive feature learning approach for deep face recognition. In
[13] A. K. Jain and U. Park. Facial marks: Soft biometric for face European Conference on Computer Vision, pages 499–515.
recognition. In International Conference on Image Process- Springer, 2016. 2, 3, 6
ing, pages 37–40. IEEE, 2009. 1 [30] D. Yaman, F. I. Eyiokur, N. Sezgin, and H. K. Ekenel. Age
[14] R. Khorsandi and M. Abdel-Mottaleb. Gender classification and gender classification from ear images. In International
using 2-D ear images and sparse representation. In Workshop Workshop on Biometrics and Forensics. IEEE, 2018. 1, 2, 6,
on Applications of Computer Vision, pages 461–466. IEEE, 7
2013. 1, 2, 5, 7 [31] P. Yan and K. W. Bowyer. Empirical evaluation of advanced
[15] D. E. King. Dlib-ml: A machine learning toolkit. Journal of ear biometrics. In Computer Vision and Pattern Recognition
Machine Learning Research, 10(Jul):1755–1758, 2009. 5 Workshops, page 41. IEEE, 2005. 5, 7
[16] J. Lei, J. Zhou, and M. Abdel-Mottaleb. Gender classifica- [32] J. Yosinski, J. Clune, Y. Bengio, and H. Lipson. How trans-
tion using automatically detected and aligned 3D ear range ferable are features in deep neural networks? In Advances in
data. In International Conference on Biometrics, pages 1–7. Neural Information Processing Systems, pages 3320–3328,
IEEE, 2013. 1, 2, 5, 7 2014. 3
[17] G. Levi and T. Hassner. Age and gender classification us- [33] G. Zhang and Y. Wang. Hierarchical and discriminative bag
ing convolutional neural networks. In Computer Vision and of features for face profile and ear based gender classifica-
Pattern Recognition Workshops, pages 34–42, 2015. 1 tion. In International Joint Conference on Biometrics, pages
[18] G. Ozbulak, Y. Aytar, and H. K. Ekenel. How transferable are 1–8. IEEE, 2011. 1, 2, 5, 7
CNN-based features for age and gender classification? In [34] K. Zhang, N. Liu, X. Yuan, X. Guo, C. Gao, and Z. Zhao.
International Conference of the Biometrics Special Interest Fine-grained age estimation in the wild with attention LSTM
Group, pages 1–6. IEEE, 2016. 1, 3, 4 networks. arXiv preprint arXiv:1805.10445, 2018. 1