Reconstructing 3-D Light-Microscopic Images by Digital Image Processing
Reconstructing 3-D Light-Microscopic Images by Digital Image Processing
Reconstructing 3-D Light-Microscopic Images by Digital Image Processing
image processing
A reconstruction procedure based on linear system theory has been developed for 3-D light-microscopic im-
ages. Inverse filtering with the 3-D optical transfer function was used for imagereconstruction. The proce-
dure allows a significant improvement in spatial resolution in the image planes perpendicular to the optical
axis.
1. Introduction was derived and is used for the inverse filtering proce-
In many applications of light microscopy, high reso- dure of the image.
lution objectives with a low focal depth have to be used, Provided that the integral absorption within the
while, usually, the size parallel to the optical axis of the object is low, the method is object independent and is
object under investigation exceeds the depth of focus. therefore not restricted to any special application. In
Consequently, a light-microscopic image is actually an our laboratory this procedure was applied to various
optical section of the object at the focal plane. It con- histological objects. The results obtained show a sig-
tains only a fraction of the total information content nificant improvement in the image quality, so that fine
present in the object. In addition, superimposed on structures of the object become visible, and the image
such an image are defocused projections from adjacent can be used for more sophisticated measurements.
focal planes. The result is a low spatial resolution
within the image. By 2-D image processing these dis- 11. Theory
tortions cannot be eliminated. The light-microscopic image of an absorption-free
The only appropriate way to attack this problem is 3-D object f(x,y,z) can be described in terms of linear
to generate 3-D images. In contrast to the 2-D case, a system theory: the 3-D imageg(x,y,z) is generated by
3-D light-microscopic image contains the maximum a convolution of the object f(xy,z) with the 3-D point
available information. It can be recorded as a series of spread function psf(xy,z) of the image forming
2-D images by varying the focus plane of the objective. system:
Although each of these image planes is still superim-
g(x,y,z) PSf(x,y,z).
posed with projections from the whole object, they can (1)
be eliminated and the original object can be restored. Let G(u,v,w) and F(u,v,w) be the Fourier transforms
The purpose of this paper is to present a new recon- of g and f, respectively,and OTF(u,v,w) the 3-D optical
struction algorithm for 3-D light-microscopic images. transfer function of the microscope. Then, in the
It is based on the mathematical formulation of the 3-D Fourier domain, the convolution reduces to a simple
imaging properties of the light microscope in the multiplication:
framework of linear system theory. The 3-D optical
G(u,v,w) = F(u,v,w) *OTF(u,v,w).
transfer function of the transmission light microscope (2)
Therefore, in the absence of noise, the operation
F(u,v,w) = G(u,v,w)/OTF(u,v,w) (3)
carried out in the region of definition of the optical'
J. Bille is with University of Heidelberg, Institute of Applied
transfer function and the inverse Fourier transforma-
Physics I, D-6900 Heidelberg, Federal Republic of Germany; the other tion of F generate the reconstructed object.
authors are with German Cancer Research Center, Institute of Ex- Because of the nonlinearity of light absorption, linear
perimental Pathology, D-6900 Heidelberg, Federal Republic of system theory requires absorption-free objects. In
Germany. practice, of course, microscopic objects are never free
Received 29 June 1984. of absorption, but it is sufficient that the absorption
0003-6935/85/020194-07$02.00/0. within the object is small enough to be approximately
© 1985 Optical Society of America. linear. In this case the linear system theory can be
194 APPLIED OPTICS / Vol. 24, No. 2 / 15 January 1985
in the z direction, the direction of the optical axis. This
generates-the 3-D optical transfer function OTF(q,s):
q =Os = 0,
2
2 f(q) [ S 11/2
OTF(q~s)={ 1 - h- _
h[IL)] 0< q < 1, Isl h(q),
7r0zoqoh(q)
elsewhere,
(6)
IV. Reconstruction
V C 0
_
The reconstruction of the focus series is performed
0- off-line on a computer VAX 11/780 with a 2-Mbyte
0.0 0.2 0.4 0.6 0.8 1.0 memory.
q
The values of the 3-D optical transfer function are
Fig. 2. (a) One-dimensional section of the 3-D optical transfer
function on the q axis (s = 0); (b) parallel to the s axis at q = 1/2. (c)
very low near the marginal surface of its bandpass vol-
Values of the 3-D optical transfer function at w = 0 (solid line) com-
ume. In the spectrum of the image, the nonzero values
pared with the 2-D optical transfer function for zero defocus (dashed of the corresponding frequencies are mainly due to
line). (d) The inverse optical transfer function (dashed line) and the noise. Dividing these frequencies by the low values of
inverse effective optical transfer function vs the normalized radial the optical transfer function would cause an overam-
frequency q. All the calculations are based on objective magnification plification of noise, which is certainly not desired.
100X, N.A. 1.3, light wavelength 0.53 Atm. Experiences with various 3-D test objects have shown
that an effective inverse optical transfer function of the
following form yields very good results:
3-D optical transfer function contains the maximum
information about the object which is left after the OTF-l(q,s) if OTF'1(q,s) 100,
image formation process.
OTF-'(q,s) = OTF'l(q,s) cos[p(q,s)] if 100 OTF 1l(q,s) 1000,
III. Data Acquisition 0 elsewhere,
To generate 3-D light-microscopic images we use an
Axiomat microscope (Zeiss) and a plumbicon scanner.
The video signal is digitized into 64 grey levels by an (7)
image analysis system, Quantimet 720 (Cambridge In- wherep(q,s) = log[OTF-'(q,s) - 2r/2.
196 APPLIED OPTICS / Vol. 24, No. 2 / 15 January1985
The maximum of OTF-l(q,O) occurs at -60% of the
incoherent cutoff frequency qo. In Fig. 2(d) the values
of OTF-A(q ,O) are shown in comparison with the values
of the inverse optical transfer function.
The effective inverse optical transfer function still
forms a high pass filter, which causes high positive and
negative values at the edges within the image. In cases
of high edges at the boundaries of the image they can be
suppressed by multiplication of the image with a 3-D
window function.3 For this purpose a Hamming or
Hanning window can be applied. Windowing the image
with a Hamming or a Hanning function before the re-
construction procedure is equivalent to a convolution
of the image spectrum with the Fourier-transformed
window function. In the case of the Hamming or
Hanning window the Fourier transform is very similar
to a 3 function. So the error caused by windowing is
relatively small compared with the error introduced by
overamplification of the boundary edges.
The 3-D preprocessed image is transformed into the
frequency domain with the 3-D Cooley-Tukey algo-
rithm. 4 In the limit of negligible absorption, the fre-
quency spectrum of the image should vanish outside the
region of definition of the optical transfer function. In
fact, however, this part of the spectrum contains low
frequency-independent randomly distributed values
caused by noise. These values are replaced by zero
before inverse filtering with the optical transfer func-
tion.
After multiplication of the frequency spectrum of the
image with OTF-1(q,s) and inverse Fourier transfor-
mation, the reconstruction procedure is completed. In
a postprocessing step the inverse Hamming or Hanning
window is applied, depending on the preprocessing step
chosen. The multiplication with the inverse window
is only carried out within the inner part of the image.
V. Results
Three-dimensional images from Feulgen stained cell
nuclei and various test objects with known 3-D struc-
ture, such as radiolarians and bristles from insect wings,
have been recorded as focus series and reconstructed
according to the procedure described above. Figures Fig. 3. Nucleus of a cell from the mucosa of the large intestine:
3-5 show three examples of the results of the recon- diameter of object, 8 Mm; objective, 100X; N.A. = 1.3, oil; total focal
struction procedure. The reconstructed planes of each shift parallel to the optical axis, 16 lm; number of averaged images,
3-D image (right) are compared with the corresponding 3; factor of oversampling and subsequent reduction, 3; sample spacing
planes of the original series (left). perpendicular to the optical axis, 0.125 pm (after reduction); sample
Figure 3 shows the nucleus of a cell from the mucosa spacing parallel to the optical axis, 0.25 pm; preprocessing steps, none;
of the large intestine. The main histological features postprocessing steps, none. (a) and (b) Planes perpendicular to the
of the nucleus (i.e., nuclear membrane, nucleolus, and optical axis. The original images show an optical dense region at
condensed chromatin granula) are well preserved in the coordinates x = 38, y = 28 which has a clearly visible fine structure,
reconstructed image. The nucleolus can be seen in the as can be seen in the reconstructed image. (c) and (d) Planes parallel
to the optical axis. The optical dense region is located at (c) coordi-
image planes perpendicular to the optical axis [Figs. 3(a)
nates y = 28, z = 18 and at (d) coordinates x = 38, x = 18. The de-
and (b)] of the original focus series as a region of rela- focused projections in the original images are apparent, as well as the
tively high optical density, located close to coordinates enhancement of the spatial resolution perpendicular to the optical
x = 38, y = 28. This region looks similar in both planes. axis. (d) A 1-D section for y = 28, z = 18. The optical density at x
In the reconstructed series, however, the fine structure = 32 - x = 40 in the original image shows no indication of a gap, while
of this region is clearly visible. It is subdivided into two in the reconstructed image two maxima are visible. This shows that
smaller parts lying in close proximity, oriented almost the reconstruction procedure does not merely amplify already existing
parallel to the optical axis. Comparison of the two maxima and minima in grey values but removes defocused projections
planes z = 15 and z = 17 shows that the two regions in the 3-D image.
o7~ .2 -
-e
.0 0 D.
9:.15
HiUEA |;
C0 ,, 0 i
C C90_w
*°^0; =,
s X a;°Planes perpendicular to the optical axis; (c) plane parallel to the op-
X * tical axis. The reconstruction of a test object with known 3-D ge-
E ,.t s F ometry ensures that the structures in the reconstructed images are
> _ 0 not artifacts but fine structures of the object.
~ 0- O
0
2- . o reveal a screwlike structure parallel to the optical axis.
0 I, Fig. 3(c) shows a section parallel to the optical axis at
2 the coordinate x = 38. The improvement of the spatial
V0a:o resolution perpendicular to the optical axis is apparent.
go) The fine structure is clearly visible in the reconstructed
X r as d image, whereas in the original series it is distorted by
:d i, *3 defocused projections from adjacent object planes.
o., . Parallel to the optical axis, only slight improvement of
the spatial resolution can be achieved. Figure 3(d)
CZ 0 shows an image plane parallel to the optical axis at y = 28
-0 > together with a 1-D section at z = 18. As can be seen
AE;0 . from the distribution of grey values in this section, the
.Z ) high absorbing region in the original focus series shows
0 dQ = no indication of a separation at this position. This
confirms that the separation of the two regions in the
. Q; < reconstructed image is not merely an amplification of
bin
> 0if an already existing gap in the distribution of grey values
. 0 M but in fact a separation of two regions.
I a = arctan (r' +