IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-6, NO.

6, NOVEMBER 1984 661

Fractal-Based Description of Natural Scenes

ALEX P. PENTLAND

Abstract-This paper addresses the problems of 1) representing natural

shapes such as mountains, trees, and clouds, and 2) computing their
description from image data. To solve these problems, we must be able
to relate natural surfaces to their images; this requires a good model of
natural surface shapes. Fractal functions are a good choice for modeling
3-D natural surfaces because 1) many physical processes.produce a
fractal surface shape, 2) fractals are widely used as a graphics tool for
generating natural-looking shapes, and 3) a survey of natural imagery
has shown that the 3-D fractal surface model, transformed by the image
formation process, furnishes an accurate description of both textured
and shaded image regions.
The 3-D fractal model provides a characterization of 3-D surfaces and
their images for which the appropriateness of the model is verifiable.
Furthermore, this characterization is stable over transformations of
scale and linear transforms of intensity.
The 3-D fractal model has been successfuly applied to the problems
of 1) texture segmentation and classification, 2) estimation of 3-D
shape information, and 3) distinguishing between perceptually "smooth"
and perceptually "textured" surfaces in the scene.
Index Tenns-Fractals, image segmentation, shading, texture models,
texture perception, 3-D shape estimation, 3-D shape models.
Fig. 1. Fractal-based models of natural shapes (by Manclelbrot and
1. INTRODUCTION Voss [101).
THE world that surrounds us, except for man-made envi- such naturally occurring surfaces has generally restricted image-
ronments, is typically formed of complex, rough, and understanding efforts to a world populated exclusively by
jumbled surfaces (e.g., Fig. 1). If we are to develop machines smooth objects, a sort of "Play-Doh" world [1] that is not
competent to deal with the natural world, therefore, we need a much more general than the blocks world.
representational framework that is able to describe such shapes Shape-from-shading [2], [3] and surface interpolation meth-
succinctly. The problem, then, is how shall we describe the ods [4], for instance, all employ the heuristic of "smoothness"
shape of a crumpled newspaper? A clump of leaves? A jagged to relate neighboring points on a surface. Such heuristics are
mountain? applicable to many man-made surfaces, of course, but are
Current representational schemes employ Plato's notion of demonstrably untrue of most natural surfaces. Texture
ideal forms-e.g., spheres, cylinders, and cubes-to describe descriptors, similarly, have dealt only with patterns assumed
three-dimensional shapes. Such shape-primitive representations to lie on a smooth surface [5], [6], or have discarded 3-D
function well in man-made carpentered environments. When notions entirely and worked only with ad hoc statistical
we attempt to describe the crenulated, crumpled surfaces measures of the image intensity surface. Before we can
typical of natural objects, however, the result is usually employ such techniques in the natural world, we must be able
implausibly complex. Such awkwardness makes these shape- to determine which surfaces are smooth and which are not-or
primitive representations difficult to envisage as the basis for else generalize our techniques to include the rough, crumpled
human-performance-level capabilities. surfaces typically found in nature.
Furthermore, how can we expect to extract 3-D information In either case, we must have recourse to a 3-D model compe-
from the image of a rough or cumpled surface when all of our tent to describe naturally occurring surfaces. A good model of
models refer to smooth surfaces only? We have no models that natural surfaces, together with the physics of image formation,
describe either the shape of such complex surfaces or how they would provide the analytical tools necessary for relating natural
evidence themselves in an image. The lack of a 3-D model for surfaces to their images. A formulation able to relate image to
Manuscript received October 17, 1983; revised February 13, 1984. surface can provide the necessary leverage for usefully repre-
This work was supported by the Defense Advanced Research Projects senting natural surfaces, as well as the computation of such
Agency under Contract MDA 903-83-C-0027 (monitored by the U.S.
Army Engineer Topographic Laboratory) and by National Science descriptions from the image data.'
Foundation Grant DCR-83-12766.
The author is with the Artificial Intelligence Center, SRI International, 1 The word representation will be used to refer to the formal language
Menlo Park, CA 94025. in which descriptions of particular objects are couched.

0162-8828/84/1100-0661$01.00 i) 1984 IEEE

662 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-6, NO. 6, NOVEMBER 1984

Fig. 2. Surfaces of increasing fractal dimension. The fractal dimension corresponds closely to our intuitive notion of
roughness.

This paper, therefore, addresses two related problems: 1) nature [7], [8], [10]. Mandelbrot, for instance, shows that
finding a representation of shape capable of describing suc- fractal surfaces are produced by a number of basic physical
cinctly the surfaces of such natural objects as mountains, trees, processes, ranging from the aggregation of galaxies to the cur-
aid clouds, and 2) detennining how such a description might dling of cheese.
be computed, given only raw image data. The first step towards The defining characteristic of a fractal is that it has a frac-
solving these problems, of course, is to obtain a model of nat- tional dimension, from which we get the word "fractal."
ural surface shapes. Technically, a fractal is defined as a set for which the Haus-
Fractal functions appear to provide such a model, in part, dorff-Besicovich dimension is strictly larger than the topological
because many basic physical processes produce fractal surfaces dimension, i.e., a set for which the only consistent description
(and thus fractals are quite common in nature), but perhaps of its metric properties requires a "dimension" value larger
even more importantly because fractals look like natural than our standard, intuitive definition of the set's "dimension."
surfaces. This natural appearance has spurred recent computer The fractal dimension of a surface corresponds quite closely
graphics research to focus on using fractal processes2 for sim- to our intuitive notion or roughness. Thus, if we were to gen-
ulating natural shapes and textures (as in Fig. 1). Mountains, erate a series of scenes with the same 3-D relief but with in-
clouds, water, plants. trees, and even primitive animals [7] - creasing fractal dimension D, we would obtain a sequences of
[13] are all among the objects that have been realistically surfaces with linearly increasing perceptual roughness, as is
portrayed by use of fractal functions. This is important in- shown in Fig. 2: (a) shows a flat plane (D z 2.0) (b) rolling
formation for workers in computer vision because the natural countryside (D t 2.1), (c) an old, worn mountain range
appearance of fractals is strong evidence that they capture all 2.3), (d) a young, rugged mountain range (D t 2.5), and, fi-
of the perceptually relevant shape structure of natural surfaces. nally (e), a stalagmite-covered plane (D 2.8).
-

Additional support for the fractal model comes from a re- One general characterization of fractals is that they are the
cently conducted survey of natural imagery [14]. This survey end result of physical processes that modify shape through
found that the fractal model of imaged 3-D surfaces furnishes local action. Such processes will, after innumerable repetitions,
an accurate description of most textured and shaded image typically produce a fractal surface shape. Examples are erosion,
regions, thereby further validating this physics-derived model aggregation (e.g., galaxy formation, meteorite accretion, and
for both image texture and shading. snowflake growth), and turbulent flow (e.g., of rivers or lava).
II. FRACTALS AND THE FRACTAL MODEL Experimental Note:
During the last 20 years, B. B. Mandelbrot has developed and Ten naive subjects (natural-language researchers) were
popularized a relatively novel class of mathematical functions shown sets of fifteen I-D curves and 2-D surfaces with
known as fractals [7], [10]. Fractals are found extensively in varying fractal dimension but constant range (e.g., see
'2Most computer graphics techniques actually ernploy a stochastic
Fig. 2), and asked to estimate roughness on a scale of
approximation of true fractal functions [91; however, this distinction is one (smoothest) to ten (roughest). The mean of the
not important for our purposes. subject's estimates of roughness had a nearly perfect
PENTLAND: FRACTAL-BASED DESCRIPTION OF NATURAL SCENES 663

0.98 correlation (p <0.001) with the curve's fractal therefore, will be devoted exclusively to fractal Brownian func-
dimension, i.e., fractal dimension accounted for 96 tions, a mathematical generalization of Brownian motion.
percent of the variance in the roughness estimates. The A random function I(x) is a fractal Brownian function if
fractal measure of perceptual roughness is therefore for all x and Ax
almost twice as accurate as any other reported to date,
e.g., [29]. Pr( I(x + Ax) - I(x) = F(y)
(1)
An Illustration of Fractal Dimension: One familiar example
of naturally occurring fractal curves is coastlines. When we where F(y) is a cumulative distribution function [7]. Note
examine a coastline (as in Fig. 1), we see a familiar scalloped that x and I(x) can be interpreted as vector quantities, thus
curve formed by innumerable bays and peninsulas. If we then providing extension to two or more topological dimensions. If
examine a finer-scale map of the same region, we shall again I(x) is scalar, then the fractal dimension D of the graph de-
see the same type of curve. It turns out that this characteristic scribed by I(x) is
scalloping is present at almost all scales of examination [8], D = 2 -H.
i.e., the statistics of the curve are invariant with respect to (2)
transformations of scale. If H = 1/2 and F(y) comes from a zero-mean Gaussian with
To illustrate the importance of fractal dimension, let us unit variance, then I(x) is the classical Brownian function.
suppose that we wish to measure the area of an island or the The fractal dimension of these functions can be measured
length of its coastline. Metric properties are, in general, either directly from I(x) by use3 of (1), or from I(x)'s Fourier
estimated by taking a measuring instrument of size \, deter- power spectrum P(f), as the spectral density of a fractal
mining that n such instruments will "cover" the curve or area Brownian function is proportional4 to f-2H-1.
to be measured, and applying the formula
A. Fractals and the Imaging Process
M=nXD
Before we can use a fractal model of natural surfaces to help
where M is the metric property to be measured (e.g., length, us understand images we must determine how the imaging
area), and D the topological dimension of the measuring process maps a fractal surface shape into an image intensity
instrument. surface.
The fact that the coastline is scalloped at all scales causes a The first step is to define our terms carefully. Real images
problem when we attempt to measure it because all of the and surfaces can not, of course, be true mathematical fractals,
curve's features that are smaller than the size of the measuring because the latter are defined to exist at all scales. Physical
tool will be missed, whatever the size of the measuring tool surfaces, in contrast, have an overall size that places an upper
selected. When we attempt to estimate the length of such a limit on the range of applicable scales. A lower limit is set by
curve, therefore, the measurement we obtain depends not the size of the surfaces' constituent particles. Fractals, in
only on the coastline but also on the length of the measure- common with all mathematical abstractions, can only approx-
ment tool itself [8] ! imate physical objects over a range of physical parameters.
Mandelbrot pointed out that, in order to obtain a consistent Because it is unreasonable to expect a physical surface to be
measurement of the coastline's length, we must generalize the fractal over all scales, the only physically reasonable definition
notion of dimension to include fractional dimensions. The use of a "fractal surface" is a surface that may be accurately ap-
of a fractional power in our mensuration formula compensates, proximated by a single fractal function over a range of scales.
in effect, for the length or area lost because of details smaller We shall say, therefore, that a surface is fractal if the fractal
thanr X. The unique fractional power that yields consistent dimension is stable over a wide range of scales, the implication
estimates of a set's metric properties in called that set's fractal being that it can be accurately approximated over that range
dimension. Because it provides the correct adjustment factor of scales by a single fractal function.
for all those details smaller than X, it may also be viewed as a These considerations prompt the following two definitions,
measurement of the shape's roughness. the first applicable to a two-dimensional function such as the
One of the more important lessons such examples teach us is image intensity surface, the second applicable to a topologically
the following: standard notions of length and area do NOT two-dimensional surface embedded in three dimensions, such
produce consistent measurements for many natural shapes. as the surface of a mountain.
The basic metric properties of these shapes vary as a function Definition: A fractal Brownian surface is a continuous func-
of their fractal dimension. Fractal dimension, therefore, is a tion that obeys the statistical description given by (1) with x
necessary part of any consistent description of the metric as a two-dimensional vector at all scales (i.e., values of Ax)
properties of such shapes, for any description that lacks it will between some smallest (Axmmn) and largest (Ax max) scales.
not be correct at more than one scale of examination. Definition: A spatially isotropic fractal Brownian surface is
Fractal Brownian Functions: Virtually all fractals encoun- a surface in which the components of the surface normal N =
tered in physical models have two additional properties: 1) (Ny, Ny, N_) are themselves fractal Brownian surfaces of
each segment is statistically similar to all others; 2) they are identical fractal dimension.
statistically invariant over wide transformations of scale. The
path of a particle exhibiting Brownian motion is the canonical 3See the beginning portions of Section III and Section IV-A.
4Discussion of the rather technical proof of this proportionality may
example of this type of fractal; the discussion that follows, be found in [ 1 ].
664 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-6, NO. 6, NOVEMBER 1984

In the next section, I will present evidence showing that many the surface normal dictates the fractal dimension of the image
natural surfaces are spatially isotropic fractals, with Axin\ and intensity surface and, of course, the dimension of the physical
Axmax being the size of the projected pixel and the size of the surface.5 Simulation of the imaging process with a variety of
examined surface patch, respectively. Further, it is interesting imaging geometries and reflectance functions indicates that
to note that practical fractal-generation techniques, such as this proposition will hold quite generally; the "roughness" of
those used in computer graphics, have had to constrain the the surface seems to dictate the "roughness" of the image. If
fractal generating function to produce spatially isotropic we know that the surface is homogeneous,6 therefore, we can
fractal Brownian surfaces in order to obtain realistic imagery estimate the fractal dimension of the surface by measuring the
[9]. Thus, it appears that many real 3-D surfaces are spatially fractal dimension of the image data.
isotropic fractals, at least over a wide range of scales. What we have developed, then, is a method for inferring a
With these definitions in hand, we can now address the prob- basic property of the 3-D surface-its fractal dimension-from
lem of how 3-D fractal surfaces appear in the 2-D image. the image data. That fractal dimension is required to obtain a
Proposition L.A 3-D surface with a spatially isotropic scale-invariant description of a surface's metric properties is an
fractal Brownian shape produces an image whose intensity indication of its usefulness. That fractal dimension has also
surface is fractal Brownian and whose fractal dimension is been shown to correspond closely to our intuitive notion of
identical to that of the components of the surface normal, roughness shows the fundamental importance of the measure-
given a Lambertian surface reflectance function and constant ment: we can now discover from the image data whether the
illumination and albedo. 3-D surface is rough or smooth, isotropic or anisotropic. We
Proof: Under the Lambertian and constancy assumptions, can know, in effect, what kind of cloth the surface was cut
the image intensity I at a point P is a function of the surface from.
normal N at the surface point that projects to P: Experimental Note:
15 naive subjects (mostly language researchers) were
I= pXN L (3) shown digitized images of eight natural textured surfaces
where p is the albedo of the surface, X is the illuminant inten- drawn from Brodatz [15]. These are shown in Fig. 8.
sity, and L = (Ix, ly 1,) is the illuminant direction. Variations They were asked "if you were to draw your finger hori-
in I, therefore, are dependent only upon variations in N. zontally along the surface pictured here, how rough or
The proposition claims that the image intensity I will obey smooth would the surface feel?," i.e., they were asked to
the rule estimate the 3-D roughness/smoothness of the viewed
surfaces. This procedure was then repeated for the
Pr (IX, Y) - Ax +Ax, Y)< ) =F(y). vertical direction, yielding a total of 16 roughness esti-
IA^x I H) () mates for each subject. A scale of one (smoothest) to
To show this, we let N1 be the normal at point (x,y) and N2 ten (roughest) was used to indicate 3-D roughness/
be the normal at point (x + Ax, y). Then we expand using (3), smoothness. The fractal dimension of the 2-D image was
yielding then computed along the horizontal and vertical direc-
tions by the use of (5), as described in the following
(pX(N1 - L) - pX(N2 L) \ section, and the viewed surface's 3-D fractal dimension
X11 -<Y)X- F(y).
-

Pr H
was estimated by the use of Proposition 1. The mean of
the subject's estimates of 3-D roughness had an excellent
Expanding the dot products, we obtain
0.91 correlation (p < 0.001) with roughnesses predicted
by use of the image's 2-D fractal dimension and Proposi-
Pr1 pX(NlxIx + N1 ly + N1 zlz) - pX(N2Xlx + N2 yly + N2 zlz) tion 1, i.e., the 3-D fractal dimension predicted by use of
11|AX|IH the measured 2-D image's fractal dimension accounted
for 83 percent of the variance in the subject's estimates
* <y =F(y). of 3-D roughness. This result, therefore, supports the
general validity of Proposition 1.
As Nx, Ny, and Nz are all fractal Brownian functions, by Properties of Fractal Brownian Functions: Fractal functions
virtue of the surface being assumed a spatially isotropic must be stable over common transformations if they are to be
fractal Brownian function and, as p, X, and L are constant, useful as a descriptive tool. The following propositions prove
then pXNxlx, pXNyly, and pXNzlz are also fractal Brownian that the fractal dimension of a surface is invariant with respect
(see Proposition 2 in the following section); thus to linear transformations of the data and to transformation of
I pX(N L)=
=
pX(NxLx + NyLy + Nzlz) scale. Estimates of fractal dimension, therefore, may be ex-
pected to remain stable over smooth monotonic transforma-
must also be. Note that this proof may be generalized to tions of the image data and over changes of scale.
include all cases in which the reflectance function is an affine
transformation of N. The dimension of I is the same as that of 5The surface normal is a function of the first derivative of depth; thus,
we can construct an integration procedure that converts surface normals
the components of N since multiplication does not affect into surface shape (as a depth map).
fractal dimension (see Proposition 2). o
6Rubin and Richards [28] describe a scheme whereby the homogeneity
This proposition demonstrates that the fractal dimension of of a surface may be determined from its imaged color.
PENTLAND: FRACTAL-BASED DESCRIPTION OF NATURAL SCENES 665

Proposition 2: A linear transformation of a fractal Brownian III. APPLICABILITY OF THE FRACTAL MODEL
function is a fractal Brownian function with the same fractal Proposition 1 proved that a fractal surface implies that the
dimension. image intensity surface is itself fractal. The reverse is also true,
Proof: The proposition claims that if I(x) is a fractal as is proved in the following proposition. This proposition,
Brownian function, i.e., obeys (1) then therefore, gives us a method of evaluating the usefulness of the
fractal surface model for particular image data: to determine
Pr ((AI(x) + B) - (AI(x + Ax) + B)< y) = F(y) whether or not a 3-D surface is fractal, all we need to do is to
IIAXIIH/ determine whether its image is fractal (given that we have first
will be true of AI(x) +B. This second expression may be determined that the surface is homogeneous, perhaps by use
rewritten as of color information [28] ).
Proposition 4: If an image intensity surface is a two-dimen-
pr Pr
((x)-IIAXIIH
I(x + Ax) y\ A~j)=Fy sional fractal Brownian then the imaged 3-D surface must be
spatially-isotropic fractal Brownian, given that the surface is
or Lambertian and the illumination and albedo are constant.
Proof: For a Lambertian surface, the image intensity I is
Pr ( 1(X)- I(x + AX) F(yA)
a linear function of the components of the surface normal Nx,
Ny, and Nz, e.g.,
thus proving the proposition; linear transforms merely scale I = pX(N- L) = pX(Nxlx + Nyly + Nzlz). (4)
the distribution F(y). 0
To prove that a fractal Brownian image
Proposition 3: The fractal dimension of a fractal Brownian necessarily entails a 3-D surface that is intensity surface
function is invariant over transformations of scale. spatially-isotropic
fractal Brownian, it suffices (by definition) to prove that a
Proof: The proposition claims that, if I(x) is a fractal fractal image implies that the components of the surface
Brownian function, i.e., it obeys (1), then normal are fractal.
We first note that Proposition 2 proves that linear transforms
Pr (1(x) - I(x + kAx)
). Fy do not affect the fractal nature of a function nor its dimension.
IlkAxitH y (y)
Thus, (4) shows that the fractal nature (and dimension) of the
will be true of I(x). This is trivially true; we need only set image intensity surface is determined by the sum of N,,
Ax* = kAx, and then the second expression may be rewritten and N,. As the finite sum of nonfractal functions is nonfractal, Nv
as we see that if the image intensity surface is fractal Brownian
then so must be at least one of the components of the surface
Pr IX)IAx*IIH <y) =F(y) normal. If the image is two-dimensionally fractal (e.g., fractal
in both the x and y image directions), then at least two inde-
thus proving the proposition. pendent components of the surface normal must be fractal.
Thus, as the surface normal has only two degrees of freedom
B. Contours and theImagingProcess (by virtue of being constrained to have unit magnitude), a two-
dimensionally fractal Brownian image intensity surface implies
We have described a method whereby the fractal dimension that all of the surface normals' components must be fractal
of the surface can be inferred for homogeneous uniformly lit Brownian and the surface is therefore spatially-isotropic fractal
surfaces. Even if the surface is not homogeneous or uniformly Brownian. n
illuminated, however, we can still hope to infer the fractal To evaluate the applicability of the fractal model for a
dimension of the surface from imaged surface contours and particular surface and its image data, then, we need only verify
bounding contours. the homogeneity of the surface and the fractal nature of the
Contour shape is often primarily a function of surface shape; image intensity surface. It appears that verification of surface
this is especially true for contours that lie mostly within a homogeneity can be done by use of color information [28] ; in
plane intersecting the surface. Common examples of such order to verify the fractalness of the image we first rewrite (1)
approximately-planar contours are bounding contours and to obtain the following description of the manner in which the
contours that are "drawn" on the surface, e.g., cast shadows. second-order statistics of the image change with scale:
The imaged projection of such planar contours is simply a
linear transfonn of the 3-D contour; recalling that linear E(IAIA XI)IIAxll[-H = E(AIAX = 1 1) (5)
transforms do not alter the fractal dimension of a function, where E( AIJAX J) is the expected value of the change in
we see that the fractal dimension of these imaged contours is intensity over distance Ax. Equation (5) is a hypothesized
the same as that of the 3-D contour. relation among the image intensities; a hypothesis that we may
Thus, we may use the fractal dimension of imaged contours test statistically. If we find that (5) is true of the image
to directly infer that of the 3-D surface (the surface's dimen- intensity surface within a homogeneous image region,7 then
sion is simply one plus the contours' dimension). Consequently,
the estimate of fractal dimension obtained from contours can squares7i.e., we calculate the quantities E( IAIAx I) for various Ax, use a least
regression [using the log of (5) ] to estimate H, and examine the
be used to corroborate the one derived from image intensities. residuals.
666

e
b
c

d
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-6, NO. 6, NOVEMBER 1984

f._
0.0

0.O

S..
L.:

1
[email protected]

^ 106 S 735

144.09972

1R4.62744

2 3
:
*__

4
*

*
*

:
*
*
-

_{
_-
*

*
*
w
:
.

*_ w

:r:*
*
_/

_ *
],_
_rv

.*

t_
>

5
S

*
*
:o
.,

o *.
*_;-\
o
_.

0
.
*

*
*

:
*
Z
*
X
*.
\

t
\

\
._
_
\
v *

._
*
;
os

*
*

4_
*
*

6
_
_
.
.
a
_

*
*
:
.
*

:.
*
:
.
-

--

7
**-
*- * . . +
-..i
x

8 9 10
Fig. 3. Results for a typical textured patch.

Proposition 4 tells us that the viewed surface must be a 3-D had become saturated, and thus the poor fit may have been
fractal Brownian surface, and thus the fractal model is appro- artifactual.
priateness for particular image data because it means that we The fact that the vast majority of the regions examined were
can know when (and when not) to use the model. quite well approximated by a fractal Brownian function indi-
To evaluate the suitability of the fractal model for natural cates that the fractal model will often provide a useful descrip-
surfaces, the homogeneous regions from each of six images of tion of natural surfaces and their images. In those cases for
natural scenes were densely sampled. In addition, detailed which the fractal description is appropriate, the only statistical
images of 12 textured surfaces (see Brodatz [15]) were digi- structure that remains unaccounted for by the fractal Brownian
tized and examined. The intensity values within each of these function is zero-mean unit-variance Gaussian noise-indicating
regions were then approximated by a fractal Brownian function that the fractal description effectively exhausts all of the second-
and the approximation error observed. order difference information within the image.
Fig. 3 shows the results for a typical textured patch. The Following initial report of this work [14] a similar investiga-
graphs (a)-(e) show the distribution of intensity differences tion was conducted by Peleg et al. [27]. In their work high-
(i.e., the second-order difference statistics) at one, two, three, resolution images drawn from Brodatz [15] were examined
five, and ten pixel distances; the distributions are approximately over much larger ranges of scale, and their data show that the
Gaussian. Fig. 3(f) shows a plot of the standard deviation of images' fractal dimension was not constant over all scales but
these distributions as a function of scale (i.e., E( IAIJX
function of Ax in pixels). Overlaid on this graph is a least-
|) as a rather only over ranges of scale. These data might be naively
interpreted as indicating that these textures are not fractal,
squares fit of a fractal rule. As can be seen, the fit is quite however, such an interpretation is incorrect.
good-implying that the intensity surface in this region is actu- As observed earlier, physical processes do not typically act at
ally a fractal Brownian function, at least over the 10: 1 range all possible scales but rather only over a range of scales. Thus,
of scales measured. we should expect that a physical surface (and thus its image)
For the majority of the textures examined (77 percent), the will change its fractal characteristics when we pass from a range
fit was as good or better than the example shown. In 15 per- of scales dominated by one formative process to a range of
cent of the cases the region was constant except for random, scales that was shaped in a different manner. It is this realiza-
zero-mean perturbations; consequently, the fractal function tion that real surfaces will be fractal over ranges of scale, rather
correctly approximates the image data, although the estimated than fractal over all scales, that prompted the careful inclusion
fractal dimension is equal to the topological dimension. The of limited ranges of scale in this papers' definition of 2-D and
fit was poor in only 8 percent of the regions examined. In 3-D fractal surfaces. The ranges of constant fractal dimension
some of these cases it appeared that the image digitization observed in the Peleg et al. data, therefore, are consistent with
8In these cases the data's dimensionality is technically not "frac- (and provide independent confirmation of) the fractal surface
tional," but this distinction need not concern us here model.
PENTLAND: FRACTAL-BASED DESCRIPTION OF NATURAL SCENES 667

A. The Relationship Between Fractals and Regular Patterns that we have found a texture edge. Examples of this will be
shown in the following sections.
Fractal Brownian functions do not, of course, describe
regular or large-scale spatial structures such as are seen in the IV. INFERRING SURFACE PROPERTIES
image of a brick wall or a tiled floor. Such structures must be
accounted for by other means. It is important to realize, how- Fractal functions appear to provide a good description of
ever, that while fractal Brownian surfaces are required to have natural surface textures and their images; thus, it is natural to
particular second-order statistics, this does not mean that they use the fractal model for image segmentation, texture classifi-
cannot be regularly patterned. cation, shape-from-texture, and the estimation of 3-D rough-
To understand this, consider that the probability of a random ness from image data. It is also natural to inquire into the
number generator producing the string "1010. . ." is exactly relationship between the fractal surface model and the various
the same as the probability of any other particular string with other models of shape and texture that have previously been
half l's, half 0's. Both strings have the same statistics, and thus reported. This section, consequently, describes the research
the same probability of occurrence, although one is regularly performed in these areas.
patterned and the other is not. Similarly, a surface such as a A. Examples of Image Segmentation
brick wall can be a perfectly good Brownian fractal: the overall
distribution of second-order statistics is correct; it simply Proposition 1 tells us that, within a homogeneous region, the
contains position-dependent patterns. fractal dimension in the image is dependent upon that of the
The fact that fractal Brownian functions can exhibit regu- 3-D surface, thus giving us a technique for inferring a 3-D
larities allows us to smoothly pass from random chaotic sur- property of the viewed surface that closely corresponds to
faces to regular patterned ones within the same conceptual people's concept of roughness/smoothness. This suggests that
framework [16]. Regular surfaces, for instance, can be gener- measurement of the fractal dimension in the image will be
ated by adding constraints (patterning) to the random-number useful in segmenting natural imagery.
generator used in conjunction with computer graphics tech- Fig. 4(a) shows an aerial view of San Francisco Bay. This
niques for recursively generating fractal Brownian functions image was digitized and the fractal dimension computed for
each 8 X 8 block of pixels by means of the Fourier technique,
[9]. i.e., the parameter H was estimated by a least-squares regression
B. Detection ofEdge Points of the Fourier-domain fractal definition onto the power spec-
trum of the block of pixels.9 Orientational information was
It is an important characteristic of the fractal model that we not incorporated into measurement of the local fractal dimen-
can determine its appropriateness for particular image data sion, i.e., differences in dimension among various image direc-
because this allows us to know when, and when not, to use the tions at a point were collapsed into one average measurement.
model. If we discover an image region that does not fit the Fig. 4(b) shows a histogram of the fractal dimensions computed
fractal model, Proposition 4 allows us to infer that we are not over the whole image.10
viewing a homogeneous fractal surface. This histogram of fractal dimension was then broken at the
Boundaries between homogeneous regions are one example "valleys" between the modes of the histogram, and the image
of a physical configuration that does not fit well into the segmented into pixel neighborhoods belonging to one mode or
fractal model. Thus, when we examine points that lie on the another. Fig. 4(c) shows the segmentation obtained by thresh-
boundary between two image regions we find that the fit be- olding at the breakpoint indicated by the arrow under (b);
tween the fractal model and the image data is normally poor. each pixel in (c) corresponds to an 8 X 8 block of pixels in
The fact that boundaries seem to be the most common event the original image. As can be seen, a good segmentation into
giving rise to a nonfractal intensity surface provides a method water and land was achieved-one that cannot be obtained by
of detecting image points that are likely to be edges. thresholding on image intensity.
One simple way to find such points is examination of the Proposition 3 indicates that this segmentation should be
computed fractal dimension. It turns out that when we stable over transformations of scale. To test this prediction,
compute the fractal dimension of a region covering a boundary the image was averaged down, from 512 X 512 pixels to
between two homogeneous areas, by using the regions's Fourier 256 X 256 and 128 X 128 pixel images, and the fractal dimen-
power spectrum,9 we normally calculate a fractal dimension sion recomputed for each of the reduced images. Fig. 4(d) and
that is less than the topological dimension. As this is a physical (e) illustrate the segmentations produced by using the same
impossibility, the implication is that the assumptions of the breakpoint as had been employed in the original full-resolution
fractal model are inappropriate for that specific image data. segmentation. These results, therefore, demonstrate the stabil-
When we observe a measured fractal dimension that is less than ity of the fractal dimension measure across wide (4: 1) varia-
the topological dimension, therefore, we can reasonably expect tions in scale, as predicted by Proposition 3.

9That is, since the power spectrum P(f) is proportional tof-2 I-1, we l°The values to the left of the large spike in (b) have a computed
may use a linear regression on the log of the observed power spectrum fractal dimension that is less than the topological dimension; thus,
as a function of f(e.g., a regression using log (P(f)) = - (2H + 1) log these points are likely caused by patches that cross distinct regional
(f) + k for various values of f) to determine the power H and thus the boundaries; in fact, they all occur along the water-land boundary
fractal dimension. and delineate that boundary.
668 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-6, NO. 6, NOVEMBER 1984

C d 0
Fi& 4. San Francisco Bay.

Fig. 5(a) shows a view of Mount Dawn. This image was digi- is a histogram of the computed fractal dimensions with the
tized into 512 X 512 pixels and the fractal dimension com- breakpoint indicated by an arrow. Fig. 6(c) shows the image
puted as before; (b) shows a histogram of the computed segmented into two classes at the point indicated by arrow 1.
fractal dimension. Arrows at the bottom of (b) show where A good partial segmentation results.'1 Fig. 6(d) illustrates the
the distribution of fractal dimension was broken to produce a points whose computed fractal dimension is less than the
segmentation of the image. Fig. 5(c) shows the image seg- topological dimension [the points to the left of arrow 2 in (b)],
mented into two classes (land; snow-and-sky) at the first as expected, these are edge points.
histogram breakpoint. Fig. 5(d) shows the sky separated from One final example is the desert scene shown in Fig. 7(a).
the land and snow by the second histogram breakpoint. Taken This scene was segmented into three classes based on the his-
together, (c) and (d) demonstrate a good segmentation into togram shown in (b); the segmentations are shown in (c) (road
mountain, snow, and sky. Note that the distinction between and sky versus desert) and (d) (road and desert versus sky). As
snow and sky is very subtle; it is impressive that this fine of a can be seen, there is a good segmentation into desert, road,
separation can be made by use of a simple image-wide histogram and sky.
of roughnesses. Several other images have been segmented in this manner
This imagewas also averaged down to 256 X 256,128 X 128, and, in each case, a good segmentation was achieved. The
and 64 X 64 pixel images, and the fractal dimension recom- computed fractal dimension (and thus the segmentationi) was
puted for each of the reduced images. Fig. 5(e)-60) illustrate always stable over at least 4: 1 variations in scale; most seg-
the segmentations that result from using the same cut points as mentations were stable over a range of 8: 1.
were employed in the original, full-resolution segmentation; it Stability of the fractal description is to be expected because
can be seen that the segmentations in these figures are quite the fractal dimension of the image is directly related to the
similar, again demonstrating the stability of the fractal descrip- fractal dimension of the viewed surface, which is a property of
tion across wide (8: 1) variations in scale. 3-D natural surfaces that is typically stable with respect to
Images of smooth, man-made surfaces can also be usefully transformations of scale [8]. The fact that the fractal descrip-
segmented, as shown in Fig. 6. Fog. 6(a) shows a picture of a
mug and, just behind it, a chairback. This image was digitized 11 Nor can this segmentation be achieved by thresholding on intensity
into 256 X 256 pixels and the fractal dimension computed; (b) values.
PENTLAND: FRACTAL-BASED DESCRIPTION OF NATURAL SCENES 669

~~...........................

_ __ ~...........................

_~~~~~
2_ I .

o g I
a

d
I f h
Fig. 5. Mount Dawn.
J

c d
Fig. 6. A picture of a mug.
670 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-6, NO. 6, NOVEMBER 1984

Fig. 7. A desert scene.

Fig. 8. The Brodatz textures used for comparison.

tion is stable with respect to scale is a critically important[15] was digitized. The mosaic, shown in Fig. 8, was con-
property. After all, let us consider: how can we hope to com- structed by Laws [17], [18] for the purpose of comparing
pute a stable, viewer-independent representation of the world various texture segmentation procedures. The images that
if our information about the world is not stable with respect comprise this data set were chosen to be as visually similar as
to scale? possible; gross statistical differences were removed by mean-
value and histogram-equalization.
B. Comparison to Established Segmentation Techniques Segmentation performance on these data exists for several
To obtain an objective comparison to established segmenta- techniques and, although differences in digitization complicate
tion techniques, a mosaic of eight images of natural surfaces any comparisons we might wish to make, Laws' performance
PENTLAND: FRACTAL-BASED DESCRIPTION OF NATURAL SCENES 671

figures nevertheless serve as a useful yardstick for assessing and a vertical tilt and 470 slant was estimated for the upper
performance on these data. portion of the image in Fig. 7. The necessity of assuming
For this comparison, simple orientational information was isotropy, however, is a serious shortcoming of this technique-
incorporated into the fractal description; the fractal dimension for, when the assumption is wrong, the estimate may be very
was calculated separately along the x and y image directions. much in error.
Fractal dimension was estimated, by using (5), within five An important new result, therefore, is that we may partially
16 X 16 pixel nonoverlapping subregions extracted from each cure this problem by observing the fractal dimensions in the x
of the eight regions. These data were next used to estimate and y directions. If they are unequal we have prima facie
the mean and variance of fractal dimension in the x and y evidence of anisotropy in the surface, because fractal dimen-
directions, and theoretical classification probabilities were sion is largely unaffected by projection.
then computed. Regardless of how a foreshortening-derived estimate of sur-
The two-parameter fractal segmenter yielded a classification face orientation is produced, we may still seek confirmation of
accuracy of 84.4 percent. This performance compares quite it by measuring the perspective gradient; if confirmation is
favorably to other segmentation techniques-despite the found, we may be confident of our estimate. Such a gradient
much larger number of texture features employed by these appears in Fig. 4: the houses dwindle in size with increasing
alternative methods. For example, Laws [17] reports accura- distance from the viewer.
cies of 65 percent for correlation statistics [19], [20], 72 Fig. 9(a) and (b) shows relief plots of the fractal dimension
percent for cooccurrence statistics [21], [22], and a theoret- computed from Fig. 4(a) and 7(a), respectively. In Fig. 9(a)
ical accuracy of 87.4 percent for texture energy statistics.12 and (b) the x and y axes correspond to the horizontal and
The results of this comparison, therefore, indicate that fractal- vertical directions in Fig. 4(a) and 7(a); i.e., they are viewed as
based segmentation will likely prove a general and powerful if from the left-hand side of the original images. The z axis
technique. 13 shows the computed fractal dimension for each 8 X 8 block of
pixels.
C. Shape Estimates In both of these examples there is a gradual rise in the
There are two ways surface shape is reflected in image estimated fractal dimension with increasing distance. We can
patterning: 1) projection foreshortening, a function of the angle track this effect of perspective foreshortening and thus observe
between the viewer and the surface normal, and 2) perspective the perspective gradient.
gradients, which are due to increasing distance between the It is at first somewhat puzzling to observe the fractal dimen-
viewer and the surface. These two phenomena are independent sion changing with increasing distance, for fractal dimension is
in that they have separate causes. Thus, they can serve to stable with respect to changes in scale. What we are in fact
confirm each other, i.e., if projection foreshortening is used to observing in these examples in interaction between our sam-
estimate surface tilt, that estimate is independently confirmed pling rate and the range of scales over which the fractal approx-
if there is a perspective gradient of the proper magnitude and imation is valid (see Section II-A).
same direction [6], [25]. We may be confident our estimate Real surfaces are not fractal at all scales; there are smallest
is correct when such independent confirmation is found. and largest components to their shape (e.g., grain size and
The fractal dimension found in the image, by virtue of its region size). These largest and smallest components define the
independence with respect to scale, appears to be nearly limits between which the surface can be described with a single
independent of the orientation of the surface. Fractal dimen- fractal function. When the projected size of a pixel becomes
sion, therefore, cannot be used to measure projection fore- comparable to either of these limits, the fractal approximation
shortening. Projection foreshortening does, however, affect can break down. If the pixel size is large with respect to the
the variance of the distribution F(y) associated with the fractal largest shape components, we observe the familiar Nyquist
dimension (see Proposition 2) in the same manner in which it sampling behavior: the surface appears to become smoother as
affects the distribution of tangent direction. Thus, to estimate the pixel size is increased. When the pixel size is less than the
surface orientation, we might assume that the surface's struc- smallest shape components, we observe "texture" edges, i.e.,
ture is isotropic and estimate surface orientation on the basis inhomogeneities in the shape structure.
of previously derived shape-from-contour and shape-from By observing the limits within which the fractal approxima-
texture results [5], [6] . tion holds,14 we measure a property intrinsic to the surface:
This estimation technique often works; for instance, a the range of scales over which the surface obeys the fractal
vertical tilt and 45° slant was estimated for the image in Fig. 4, rule. Because such measurement is in terms of pixel size, it
relates the size of the projected pixel relative to the (presum-
'2See [17, p. 1481. ably invariant) size distribution of the surface's shape com-
13On a data set of 12 Brodatz textures, in which the textures were ponents. Measuring the largest scale at which a single fractal
somewhat less similar, the classification accuracy was 87.6 percent.
Following initial report of this work [14], Peleg et al. [271 investigated rule holds, therefore, gives us the ratio between pixel size and
other techniques for calculating fractal dimension and have reported an aspect of the surfaces' intrinsic structure and thus allows
essentially 100 percent accuracy on similar Brodatz textures, by using us to observe the perspective gradient.
separate estimates of fractal dimension at many different scales. This
allowed them to incorporate information about Axmin and Ax max,
the limits at which the textures behave as a single fractal, and thus to 14By use of (5). Also see Peleg et al. [27], which describes an
improve their classification accuracy. alternate fractal-dimension technique for discovery of these scale limits.
672 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-6, NO. 6, NOVEMBER 1984

a b
Fig. 9. Relief plots of the computed fractal dimension for Figs. 4(a) and 7(a).

In Fig. 9 we are measuring the size of the surface's largest variations described by the function F(y) in (1); e.g., they are
shape components by using the fact that the apparent smooth- stationary. Because these surfaces are judged by people to be
ness of the viewed surface increases with increasing size of the ".smooth"15 the fractal model with small values ofHis appro-
projected pixel, given that the pixels' projected size is compa- priate for modeling smooth, shaded regions of the image. In
rable to that of the surface's largest shape components. Mea- contrast, fractals with H > 0 are not perceived as smooth, but
surements of fractal dimension may thus be used to measure rather as rough or textured.
the perspective gradient, thereby providing independent con- The fractal model can therefore encompass both image
firmation of the foreshortening-derived estimates of surface shading and texture, with shading as a limiting case in the
orientation. In imagery of large planar regions it may also be spectrum of texture granularity.16 The fractal model thus
possible to use the magnitude of the perspective gradient to allows us to make a reasonable, rigorous, and perceptually
estimate the surface's orientation. plausible definition of the categories "texture" and "shading"
Note, however, that in Fig. 9(b) the portion of the image in terms that can be measured by using the image data.
closest to the viewer does not exhibit a smooth gradient The abflity to differentiate between "smooth" and "rough"
because the size of the region used to compute the fractal surfaces is critical to the performance of such techniques as
dimension was small relative to the size of the rocks and bushes. shape-from-shading [3], [4], [26], surface interpolation [4],
In this near area the image data used to compute fractal dimen- and shape-from-texture [5], [6] -to mention only the obvious
sion are often boundaries between homogeneous areas, and cases. Thus, use of the fractal model to infer qualitative 3-D
therefore do not fit the fractal model. As described previously, shape, i.e., smoothness/roughness, has the potential to signifi-
the appropriateness of the fractal model may be determined cantly improve the utility of many other machine vision
for the specific image data under consideration. In the case of methods.
the image in Fig. 7(a), the fractal model is inappropriate for
much of the data in the near portion of the image. Thus, most E. Relationship to 2-D Texture Models
of the apparent "perspective gradients" in the near portion of One of the more interesting aspects of the fractal surface
Fig. 9(b) can be identified as artifacts. model is that it relates 2-D texture measures based on co-
These two new results-the ability to obtain evidence of 15The surface may, however, have significant local fluctuations: these
surface anisotropy and the measurement of the perspective are usually seen as "dust" or some other extraneous effect modifying
gradient-represent significant advances in shape estimation the smooth surface. It may also be that beyond some limiting value of
because they offer a way to substantially improve the reliability the variance of F(y) the surface is no longer perceived as smooth.
16 If we assume that incident light is reflected at the angle of incidence
of shape-from-foreshortening [5], [6] techniques. and we make the variance of F(y) small relative to the pixel size, the
surface will be mirrorlike. If, on the other hand, the variance of F(y) is
D. Shading into Texture large relative to the pixel size, the surface will become more isotropically
reflecting. Thus, we can use the fractal model to capture the intuitive
Fractal functions with H 0 do not change their statistics as
-
notion that reflectance functions are due to the structure of the micro-
a function of scale. Such surfaces are planar except for random texture.
PENTLAND: FRACTAL-BASED DESCRIPTION OF NATURAL SCENES 673

occurrence statistics [21], [22], Fourier spectra [23]-[25], wrong, Gagalowicz [3 1 ] has presented evidence that the prob-
Markov processes [13], or autocorrelation [19], [20] to each lems with Julesz's conjecture are obviated by making texture
other and to 3-D surface structure. discrimination dependent upon only the local second-order
We have seen that fractal Brownian functions may be defined statistics. Others, such as Richards and Polit [32], have
in terms of either the way interpixel differences (second-order presented evidence that texture perception is mediated by
statistics) change with distance, or the rate at which the Fourier spatial-frequency tuned channels.
power spectrum falls off with increasing frequency. Similarly, It turns out that both the second-order statistics and the
fractal functions may be characterized by the way the auto- Fourier models fit well with the notion that people use the
correlation function falls off [7], [10], or by Markov processes fractal dimension of the image (and thus of the 3-D surface)
[10], [13]. Because the fractal image of a 3-D fractal surface in preattentive discrimination of unpatterned textures. That
may be described in any of these terms, it follows that for the fractal surface model and these perceptual models com-
fractal images we may relate each of these texture measures to plement each other is not so surprising, for we have already
the other and to the 3-D fractal surface model. The fractal described how to measure the images' fractal dimension either
surface model, therefore, offers the potential of unifying and by use of the local second-order statistics7 or the Fourier
simplifying these various 2-D texture descriptions, as well as power spectrum.9
the possibility of interpreting them in terms of the 3-D struc- Let us look first at Gagalowicz's model of texture perception,
ture of the world. and for the sake of argument let us assume that discrimination
To say that the fractal model can be described in these other between unpatterned texture is based on the perceived rough-
terms in not to say that the fractal model is equivalent to these ness of the corresponding 3-D surface. In this case the fractal
other models: the fractal model is clearly a 3-D model, whereas surface model agrees with Gagalowicz's claim that texture
the texture models are only 2-D. Further, although it is true discrimination will be dependent the local second-order
that a fractal function can be characterized in terms of Fourier statistics-as these determine the images' roughness (i.e., its
spectra, co-occurrence matrices, etc., it is not true that any fractal dimension) and thus the roughness of the 3-D surface.
characterization of an image in these terms captures the 3-D Similarly, because the images' fractal dimension can also be
properties of the viewed surface, as is the case with the fractal measured from its Fourier power spectrum, the image and
description. Characterization of an image in terms of radial surface roughness (fractal dimension) can be determined by
slices of the Fourier domain (for instance) is completely orthog- use of spatial frequency channels. Thus, the suggestion that
onal to the fractal description and, as a result, constrains the spatial frequency channels mediate human texture perception
shape of the 3-D surface hardly at all."7 As a consequence, we also agrees with the fractal surface model and the assumption
cannot expect the image segmentation performance of these that preattentive texture discrimination depends on perceived
texture techniques to be generally indicative of the perfor- roughness.
mance of the fractal surface model. V. SUMMARY
One may still ask, since we can "translate" the fractal sur-
face model into these various texture descriptions, why should Fractal functions seem to provide a good model for describing
one employ the fractal model rather than some other? The the rough, crenulated, and crumpled 3-D surfaces typical of
principal advantage of describing textures in terms of fractal natural scenes. The evidence in support of this assertion is
surfaces, rather than in any of these other vocabularies, is the following:
that it allows us to capture a simple physical relationship that 1) Many basic physical processes produce fractal surfaces.
underlies the texture structure; a relationship that allows us to 2) Fractal surfaces look like natural surfaces, and thus ap-
interpret the 2-D texture measurements in terms of the 3-D pear to capture all of the shape structure relevant to human
world. The fact that this physical interpretation can be lost perception.
with the 2-D characterizations of texture makes it seem advan- 3) We have conducted a survey of natural imagery and found
tageous to characterize texture problems in terms of the 3-D that a fractal model of imaged 3-D surfaces, when transformed
fractal surface model. by the image formation process, furnishes an accurate descrip-
tion of both textured and shaded regions in most natural
F Relationship to Human Texture Perception imagery.
In light of the fact that the fractal surface model has been Fractal functions, therefore, are useful for describing the
shown to predict peoples' perception of 3-D roughness (see complex 3-D surfaces typical of natural objects. By transform-
Section II), it is worth examining the relationship between the ing this 3-D model through the image formation process we
fractal surface model and models ofhuman texture perception. can obtain a useful model of how such surfaces appear in the
The most widely-known model of human texture perception image data. One important aspect of this model is that it is
is due to Julesz [30] who suggested that preattentive texture easy to test its appropriateness for particular image data.
perception is dependent upon the global second-order statistics Characterization of image texture by means of a 3-D fractal
of the texture. Although this suggestion is now known to be surface model has shed considerable light on the physical basis
for several of the 2-D texture techniques currently in use, and
17lllumination effects can account for most variation in such a made it possible to describe image texture in a manner that is
description. In general, a description in these other terms will constrain
the 3-D interpretation only to the extent that the description allows stable over transformations of scale and linear transforms of
recovery of the image's fractal parameters. intensity. These properties of the fractal surface model allow
674 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-6, NO. 6, NOVEMBER 1984

it to serve as the basis for an accurate image segmentation [9] A. Fournier, D. Fussel, and L. Carpenter, "Computer rendering
procedure that is similarly stable over a wide range of scales. of stochastic models," Commun. ACM, vol. 25, no. 6, pp. 371-
384, 1982.
Because fractal dimension is not affected by projection [10] B. B. Mandelbrot, The Fractal Geometry of Nature. San Fran-
distortion, its measurement can significantly enhance our cisco, CA: Freeman, 1982.
ability to estimate shape from foreshortening. Specifically, [11] A. Norton, "Generation and display of geometric fractals in 3-D,"
Comput. Graphics, vol. 16, no. 3, pp. 61-67. 1982.
measurement of fractal dimension can provide 1) evidence of [12] Y. Kawaguchi, "A morphological study of the form of nature,"
surface anisotropy, and 2) an estimate of the perspective Comput. Graphics, vol. 16, no. 3 pp. 223-232,1982.
gradient. Both capabilities are extremely important because [13] L. C. Carpenter, "Vol libre," computer-generated movie, 1980.
they provide a way to obtain independent confirmation of the [14] A. Pentland, "Fractal-based description of natural scenes," in
Proc. IEEE Conf Comput. Vision and Pattern Recognition '83,
assumptions on which previously reported techniques are Arlington, VA, July 1983. See also, A. Pentland, "Fractal-based
based. description," in Proc. Int. Joint Conf: Artificial Intel. '83, Karls-
ruhe, Germany, Aug. 1983.
One further important result is that measurement of the 2-D [151 P. Brodatz, Textures: A Photographic Album for Artistis and
image fractal dimension enables estimation of the 3-D fractal Designers. New York: Dover, 1966.
dimension. Knowledge of the 3-D fractal dimension has been [16] T. Pavlidis, personal communication.
shown to be a nearly perfect predictor of people's perception [17] K. Laws, "Textured image segmentation," USC Image Processing
Institute, Los Angeles, CA, Rep. 940, 1980.
of roughness. Thus, the 3-D fractal model allows us to deter- [18] D. H. Ballard and C. M. Brown, Computer Vision. Englewood
mine which imaged regions are perceived as smooth, and which Cliffs, NJ: Prentice-Hall, 1982.
ones appear textured. This discrimination is of special impor- [19] W. K. Pratt, 0. D. Faugeras, and A. Gagalowicz, "Visual discrimi-
nation of stochastic texture," IEEE Trans. Syst., Man, Cybern.,
tance to shape-from-shading, shape-from-texture, and surface vol SMC-8, pp. 460-473, 1978.
interpolation methods as their perfonnance relies on assump- [20] K. Deguchi and I. Morishita, "Texture characterization and tex-
ture-based image partitioning using two-dimensional linear
tions about the smoothness or roughness of the viewed surface. estimation techniques," IEEE Trans Comput., vol C-27, pp.
The encouraging progress that has already been achieved in 739-745, 1978.
research on these problems augers well for the fractal-based [21] A. Rosenfeld and E. B. Troy, "Visual texture analysis," in IEEE
Conf: Feature Extraction and Analysis, Argonne, IL, pp. 115-
approach. It appears that the 3-D fractal model of surface 124, Oct 1970.
shape will constitute a significant aid in efforts to proceed [22] R. M. Haralick, K. Shanmugam, and J. Dinstein, "Textural fea-
from the image of a natural scene to its description. tures for image classification," IEEE Trans. Syst., Man, Cybern.,
vol. SMC-3, pp. 610-621, 1973.
[23] R. Bajacsy and L. Lieberman, "Computer description of real out-
ACKNOWLEDGMENT door scenes," in Proc. 2nd Int. Joint Conf Pattern Recognition,
The author would like to thank K. Laws for his expertise [24] Copenhagen, Denmark, Aug 1974, pp. 174-179.
H. Maurer, "Texture analysis with Fourier series," in Proc. 9th
and advice, M. Fischler and A. Witkin for their feedback, and Int. Sympo. Remote Sensingof the Environment, Ann Arbor, MI,
A. Hanson for getting me interested in fractals in the first Apr. 1974, pp 1411-1420.
place. [25] R. Bajacsy and L. Lieberman, "Texture gradient as a depth cue,"
Comput. Graphics Image Processing, vol. 5, no. 1, pp 52-67,
1976.
REFERENCES [26] A. P. Pentland, "Local Computation Of Shape," in Proc. Nat.
Conf. Artifical Intell., Pittsburgh, PA 1982.
[1] H. G. Barrow and J. M. Tenenbaum, "Recovering intrinsic scene [271 S. Peleg et al., Univ. Maryland, College Park, MD, Dep. Comput.
characteristics from images," in Computer Vision Systems, A. Sci., TR-1306. July, 1983.
Hanson and E. Riseman, Eds. New York: Academic, 1978. [28] J. M. Rubin and W. A. Richards, "Color vision and image inten-
[2] B. K. P. H. Horn, "Shape from shading: A method for obtaining sities: When Are changes material?," Biol. Cyber., vol, 45, pp.
the shape of a smooth opaque object from one view," Project 215-226, 1982.
MAC, A. I. Tech. Rep. 79, 1970. [29] H. Tamura, S. Mori, and T. Yamawaki, "Textural features corre-
[3] B. K. P. H. Horn and K. Ikeuchi, "Numerical shape from shading sponding to visual perception," IEEE Trans. Syst., Man, Cybern,
and occluding boundaries," Artifical Intelligence, Special Issue vol. SMC-8, pp. 460-473, June 1978.
on Computer Vision, vol. 15, pp. 141-184, 1981. [30] B. Julesz, "Texture and visual perception," Sci. Am., vol. 212,
[4] W. E. L. Grimson, "Computing shape using a theory of human pp.38-48,1965.
stereo vision," Ph.D. dissertation, Dep. Mathematics, Massachus- [31] A. Gagalowicz, "A new method for texture fields synthesis:
setts Inst. Technol., Cambridge, MA, 1980. Some applications to the study of human vision," IEEE Trans.
[51 J. R. Kender, "Shape from texture: An aggregation transform Pattern Anal. Machine Intell., vol. PAMI-3, pp. 520-533, Sept.
that maps a class of textures into surface orientation," Proc. Sixth
Int. Joint Conf. ArtificialIntell., Tokyo, Japan, 1979. 1981.
[6] A. P. Witkin, "Recovering surface shape and orientation from [32] W. Richards and A. Polit, "Texture matching," Kybernetik, vol.
texture," Artificial Intelligence, vol. 17, pp. 17-47, 1981. 16, pp. 155-162, 1974.
[71 B. B. Mandelbrot, Fractals: Form, Chance and Dimension. San
Francisco, CA: Freeman, 1977.
[8] L. F. Richardson, "The problem of contiguity: An appendix of
statistics of deadly quarrels," General Systems Yearbook, vol. 6, Alex P. Pentland, for a photograph and biography, see p. 187 of the
pp. 139-187, 1961. March 1984 issue of this TRANSACTIONS.