3D Rotation Invariant Local Binary Patterns

See
discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/242216671
3D rotation invariant local binary patterns

Conference Paper July 2008
CITATIONS
READS
25
62
2 authors:
Janis Keuper (Fehr)
Hans Burkhardt
Fraunhofer Institute for Industrial Mathematics
University of Freiburg
40 PUBLICATIONS 233 CITATIONS
286 PUBLICATIONS 4,349 CITATIONS
SEE PROFILE
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
Distributed Deep Learning View project
All content following this page was uploaded by Janis Keuper (Fehr) on 22 June 2015.
The user has requested enhancement of the downloaded file. All in-text references underlined in blue are added to the original document
and are linked to publications on ResearchGate, letting you access and read them immediately.
3D Rotation Invariant Local Binary Patterns

J. Fehr and H. Burkhardt
Chair of Pattern Recognition and Image Processing
University of Freiburg, Germany
[email protected]
Abstract
We present a novel method for the fast computation of
rotation invariant local binary patterns (LBP) on 3D
volume data.
Unlike a previous publication on 3D LBP, this new approach is not limited to uniform patterns, providing
a real 3D extension of the standard and rotation invariant LBP. We evaluate our methods in the context of 3D
texture analysis of biological data.
has several drawbacks: Since the number of possible

uniform patterns is drastically increasing in the 3D
case, we had to generate template like, data dependent
uLBP. Also, the approximation method does not
provide complete gray-scale invariance. With our new
method, we are able to cope with these problems.
LBP in 2D. LBP encode the gray-scale invariant pattern of N neighboring pixels with gray values xi , i
{0 . . . N 1} . The neighbors are given as equidistant
points on a circle with radius r around a center pixel
with gray value c.
1 Introduction
LBPrN :=
Local Binary Patterns (LBP) [7] have been established as a standard feature based method for 2D image
analysis. LBP have been successfully applied to a wide
range of different applications from texture analysis [7]
to face recognition [8]. Various extensions to the basic
LBP algorithms were published in recent years, including rotation invariant and computationally efficient
uniform binary patterns (fuLBP) - a comprehensive
overview can be found in [7].
In this paper, we extend the original LBP from 2D
images to 3D volume data. We also generalize the
rotation invariant LBP, implementing full rotation
invariance in 3D.
Related Work. So far, standard LBP have only been
applied to 2D images and 2D time series. There are
several recent publications on volume local binary
patterns (vLBP)[10][9][11], but confusingly these
methods deal with dynamic texture analysis on 2D time
series and not on full 3D volumetric data. Respectively,
vLBP only provide rotation invariance towards rotations around the z-axis.
To the best of our knowledge, there has been only one
publication on full 3D LBP: previously we introduced
a 3D method for the approximate computation of
uniform LBP (uLBP) in [3]. This first approach
978-1-4244-2175-6/08/$25.00 2008 IEEE
N
X
sig(xi c) 2i
(1)
i=0
with sig(x) :=

1
0
for x > 0
otherwise
For more details on 2D LBP refer to [7].

Rotation Invariance in 2D can be achieved via normalization:
rLBPrN
:=
min(ROT(LBPNr , n)
(2)
with n = 0 . . . N 1
Where ROT(LBPNr , n) is a discrete rotation of the
neighbors by n steps. More details on rotation invariant 2D LBP can be found in [7].
LBP in 3D
At a first glimpse, the extension of LBPs to 3D seems

to be straight forward: Simply pick a center voxel
with gray value c, and sample a fixed number of N
equidistant points with gray values x0 . . . xN 1 on the
respective sphere with radius r. Compute sig(xi c)
for all xi and encode the binary pattern as in the usual
LBP algorithm.
This appears to be very simple, but one has to face
several severe problems following this direct approach:

first, equidistant sampling on a sphere is a very hard
task which is known as Fejes Toths problem. In
general, it cannot be solved analytically. Since we need
equidistant sampling in order to achieve full rotation
invariance, we are limited to the few known point sets
where a sampling is known [2] or we have to use rather
expensive numerical approximations. Secondly, rotation invariant LBPs require an ordering of the sampled
points, which is trivial in 2D - but turns out to be a
quite hard problem given three degrees of freedom on
a sphere. And last, computational complexity becomes
an issue with the vast rising number of sampling points
needed on a sphere.
Our approach is based on the pre-computation of socalled 2-patterns. A 2-pattern PNr is the volume representation (3D grid) of a set of N equidistant points
on a sphere with radius r. Each of theses points is
weighted in an arbitrary but fixed order with the gray
values p0 := 20 , . . . , pN 1 = 2N 1 . All other points
in the volume are set to zero.
For each LBP computation, we generate a volume representation X r of the gray values of all points on the
neighborhood sphere with radius r. Given a center
point with gray value c, we then compute the point-wise
threshold of the entire volume grid:
T r : ti T r , xi X r : ti := sig(xi c).
(3)
The resulting LBP is then computed via the dot-product

of the 2-pattern and the threshold:
LBPrN :=< Pnr , T r >
(4)
Hence, given an equidistant sampling, the computation

of 3D LBP is not so difficult - the actual problem is to
obtain rotation invariance in 3D.
2.1 Rotation Invariance
3.1
Fast Implementation
Mathematical Foundations
Let us start with a very brief introduction of the

basic mathematical tools and conventions we need to
construct a fast correlation on a sphere. Please refer
to [5] and [1] for more detailed background on the
methods used.
Spherical Harmonics. Spherical Harmonics (SH) [5]
form an orthonormal base on the 2-sphere. Analogical
to the Fourier Transform, any given real valued signal
f on a sphere with its parameterization over the angles
, (latitude and longitude on the sphere) can be represented by an expansion in its harmonic coefficients:
f (, ) =
m=l
X
X
flm Yml (, )
(6)
l=0 m=l
where l denotes the band of expansion, m the order for

the l-th band and flm the harmonic coefficients. The
harmonic base functions Yml (, ) are computed as follows:
s
2l + 1 (l m)! l
Yml (, ) =
P (cos )eim , (7)
4 (l + m)! m
l
is the associated Legendre polynomial.
where Pm
The harmonic expansion of a signal f will be denoted

by f with corresponding coefficients flm . In our case,
where we are only considering signals on a discrete grid
in R3 , the flm can be computed via point-wise multiplication () of the 3D data grid with pre-computed discrete approximations of the harmonic base functions of
fixed radii:
X
flm =
Yml f
(8)
R3
In the 2D case, where rotations have only one degree

of freedom, invariance can be realized via minimum
search over all cyclic shifts of the circular 2-pattern in
O(N ) (2). Using our fixed 2-pattern on a sphere, we
now encounter three degrees of freedom. This makes
the 3D case a lot more difficult and computationally expensive.
We engage this problem by revising (2): we can reformulate the problem as the computation of the minimum
of the full correlation () over all angles of the fixed
2-pattern PrN with T r :
rLBPrN := min(PNr T r )
(5)
In the next section, we will show how this correlation

can be computed efficiently in the frequency domain.
Rotations in SH. We use the Euler notation in zyzconvention denoted by the angles , , with ,
[0, 2[ and [0, [ to parameterize the rotations
R SO(3) (short hand for R(, , ) SO(3)).
Rotations R(, , ) f in the Euclidean space find
their equivalent representation in the harmonic domain
in terms of the so called Wigner D-Matrices, which
form an irreducible representation of the rotation group
SO(3). For each band l, Dl (, , ) (or short handed
Dl (R)) defines a band-wise rotation in the SH coefficients. Hence, a rotation in the Euclidean space can be
estimated in the harmonic domain (with a maximum expansion b) by
Rf
b
l
l
X
X
X
l=0 m=l n=l
l
Dmn
(R)flm Yml
(9)
3.2
Fast Correlation in SH
We are following the fast correlation method which is

inspired by [6]. The full correlation function corr :
SO(3) R of two signals f and g under the rotation
R SO(3) on a 2-sphere is given as:
Z
corr (f, g, R) := f (R g) ddd
(10)
S2
Using the DFT Convolution Theorem and substituting

f and g with their SH expansions (9, 8) leads to
X
l (R)f g
Dmn
(11)
corr (f, g, R) =
lm ln
lmn
The actual trick to obtain the fast correlation is

now to factorize the original rotation R(, , ) into
R = R1 R2 , choosing R1 (, /2, 0) and R2 (, /2, )
with = /2, = , = /2.
Using the fact that
l
Dmn
(, , ) = eim dlmn ()ein
(12)
where dl is a real valued so called Wigner (small) dmatrix [1], and

l
Dmn
(R1
R2 ) =
l
X
(13)
we can rewrite the rotation as

l
X
orr ),
f g = F F T 1 (C
dlnh (/2)dlhm (/2)ei(n+h+m)
h=l
(14)
Substituting (14) into (11) provides the final formulation of the correlation function regarding the new angles
, and :
X
corr (f, g, , , ) =
dlmh (/2)dlhn (/2)
lmhn
flm gln ei(n+h+m) (15)
where l is running from 0 to the maximum band of expansion, and m, h, n from l, . . . , l. The direct evaluation of this correlation function is of course not possible
- but it is rather straight forward to obtain the Fourier
transform of (15), hence eliminating the missing angle
parameters:
X
corr (f, g, m, h, n) =
dlmh (/2)dlhn (/2)
l
flm gln
(16)
(17)
revealing the correlation values on a sparse grid in a

three dimensional (, , )-space. The minimum correlation min(f g) is found by searching the grid which
has a size of (2b + 1)3 , where b is the maximum band.
Hence, correlation accuracy and computational complexity are directly linked to the maximum spherical
harmonic expansion. Since the grid sizes are still considerably small for any likely b, the full correlation can
be computed in a few milliseconds on a standard PC.
3.3
Final Algorithm
Given our fast correlation, we compute the rLBP in

three steps: first, compute the 2-pattern PNr and T r
using known equidistant samplings [2] with 24 to 124
samples. Then expand both in spherical harmonics and
obtain the coefficients PNr and T r and finally compute
the rLBP via the minimum of the fast correlation of PNr
and T r as stated in (5).
3.4
l
l
Dnh
(R1 )Dhm
(R2 )
h=l
l
(R) =
Dmn
Finally, the full correlation (f g) over all angles

(, , ) can be retrieved via inverse FFT of the entire
orr of all possible corr (m, h, n)
grid C
Further Speedup
The actual bottleneck of our approach is the complexity of the computation of T r which is increasing with
the number of sampling points. For full gray-scale invariance we have to compute T r correctly, but there is
an elegant way to approximate T r while preserving a
r and subtract c
gray-scale robustness: we compute X
in the frequency domain, which only affects the 0th coefficient T0r :
0r c b,
T0r X
ir
Tir X
(18)
Hence, we no longer have a binary but continuous

weighting of the n-pattern. We will refer to this approximation as aLBP.
Experiments
We evaluated the texture analysis performance of the

rLBP and aLBP on 3D volumetric biological data
and compared the results to the fuLBP methods presented in [3] and a Haar-Integration approach in[4]. A
database containing 229 3D volume datasets of 3 different classes of cell-nuclei was given. The cells were
recorded in tissue via confocal laser microscopy using two different anti-body markers, YoPro and Cy3,
perform slightly better. However, the main improvement of the general rLBP is the easy handling compared
to the fuLBP where one has to perform an elaborate precomputation of the uniform patterns. In terms of computational complexity, the cost of the rLBP depends on
the number of samples and can become quite expensive
- a tradeoff between resolution and costs. aLBP provide
an efficient alternative, if gray scale invariance is not
crucial. All LBP methods were outperformed by the
Haar-Intergral based features from [4]. The reason for
this may be that LPB have a larger spacial resolution,
but a weaker gray scale resolution than Haar-features which might be not favorable for the given task.
References
Figure 1. Sample database entry, xyslices of 3D volumetric data. From left to
right: YoPro marker, Cy3 marker, ground
truth labeling of the cell nuclei, binary
mask for the database entry.
type
A
B
C
D
result in [4]
93,3%
84,6%
79,8%
94,1%
fuLBP[3]
88,7%
75,8%
74,2%
90,9%
rLBP
91,3%
79,8%
75,1%
93,4%
aLBP
90,5%
78,2%
74,8%
91,0%
Table 1. Results of the nuclei classification comparing the Haar-Intergral

based features from [4] and fuLBPs with
our new approach. The celltypes are
A:Erythrocyte, B:Endothelial cell, C: Fibroblast and D:Background.
which were recorded in separate channels. For this experiment, we used only the YoPro channel. A sample
database entry is shown in Fig. 1, please refer to [4] for
further details on the database.
We used 12 different features of varying radii, number of samples and expansion bands. After feature
extraction, we performed a voxel-wise classification
via support-vector machine (SVM) following the algorithms in [4]. Results are shown in table 1.
5 Conclusions
We presented a novel method for the computation of rotation invariant 3D LBP for 3D texture analysis on volume data. Our new approach rLBP clearly outperforms
the previous fuLBP, and even the approximative aLBP
View publication stats
[1] D. Brink and G. Satchler. Angular Momentum, Second

Edition. Clarendon Press, Oxford, 1968.
[2] H. Cundy and A. Rollett. Mathematical Models. Oxford
Univ. Press, 2nd ed., 1961.
[3] J. Fehr. Rotational invariant uniform local binary patterns for full 3d volume texture analysis. In Proc. FinSig
2007, 2007.
[4] J. Fehr, H. Kurz, C. Sauer, and H. Ronneberger,
O.and Burkhardt. Identifikation von zellen in intaktem
gewebe - selbst-lernende segmentierung und klassifikation von zellkernen in 3d volumendaten mittels voxelweiser grauwertinvarianten. In Handels H. Ehrhardt
J. Editors, Informatik Aktuell, Bildverarbeitung fr die
Medizin 2006, Hamburg 19. - 21.3.06, pages 368373.
Springer-Verlag, 2006.
[5] H. Groemer. Geometric Applications of Fourier Series
and Spherical Harmonics. Cambridge University Press,
1996.
[6] J. A. Kovacs and W. Wriggers. Fast rotational matching.
Acta Crystallogr, (58):12821286, 2002.
[7] T. Maenpaa and M. Pietikainen. In C. Chen and
P. Wang, editors, Handbook of Pattern Recognition and
Computer Vision, chapter Texture analysis with local binary patterns., pages 197216. World Scientific, 2005.
[8] H. Yang and Y. Wang. A lbp-based face recognition
method with hamming distance constraint. In ICIG 07:
Proceedings of the Fourth International Conference on
Image and Graphics, pages 645649, Washington, DC,
USA, 2007. IEEE Computer Society.
[9] G. Zhao and M. Pietikainen. Dynamic texture recognition using volume local binary patterns. In Proc. ECCV
2006 Workshop on Dynamical Vision, page 12 pp, Graz,
Austria, 2006.
[10] G. Zhao and M. Pietikainen. Dynamic texture recognition using local binary patterns with an application to
facial expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(6, in press.), 2007.
[11] G. Zhao and M. Pietikainen. Improving rotation invariance of the volume local binary pattern operator.
In Proc. IAPR Conference on Machine Vision Applications, pages 327330, Tokyo, Japan, 2007.

3D Rotation Invariant Local Binary Patterns

Uploaded by

Copyright:

Available Formats

3D Rotation Invariant Local Binary Patterns

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

3D Rotation Invariant Local Binary Patterns

Uploaded by

Copyright:

Available Formats

See

3D rotation invariant local binary patterns

Fraunhofer Institute for Industrial Mathematics

40 PUBLICATIONS 233 CITATIONS

286 PUBLICATIONS 4,349 CITATIONS

Distributed Deep Learning View project

3D Rotation Invariant Local Binary Patterns

has several drawbacks: Since the number of possible

978-1-4244-2175-6/08/$25.00 2008 IEEE

For more details on 2D LBP refer to [7].

At a first glimpse, the extension of LBPs to 3D seems

several severe problems following this direct approach:

The resulting LBP is then computed via the dot-product

Hence, given an equidistant sampling, the computation

2.1 Rotation Invariance

Let us start with a very brief introduction of the

where l denotes the band of expansion, m the order for

The harmonic expansion of a signal f will be denoted

In the 2D case, where rotations have only one degree

In the next section, we will show how this correlation

l=0 m=l n=l

We are following the fast correlation method which is

Using the DFT Convolution Theorem and substituting

The actual trick to obtain the fast correlation is

where dl is a real valued so called Wigner (small) dmatrix [1], and

we can rewrite the rotation as

dlnh (/2)dlhm (/2)ei(n+h+m)

flm gln ei(n+h+m) (15)

revealing the correlation values on a sparse grid in a

Given our fast correlation, we compute the rLBP in

Finally, the full correlation (f g) over all angles

Hence, we no longer have a binary but continuous

We evaluated the texture analysis performance of the

Table 1. Results of the nuclei classification comparing the Haar-Intergral

View publication stats

[1] D. Brink and G. Satchler. Angular Momentum, Second

You might also like