Tutorial Biomedical

Image Processing in Biomedical Applications
S. Colantonio, D. Moroni, O. Salvetti
Signals & Images Lab

Institute of Information Science and Technologies
CNR, Pisa
[email protected]
SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 1 / 109

1 Introduction
Biomedical Imaging
Image processing: what for?
2 Digital images
Image acquisition and representation
3 Image enhancement
Spatial Filtering
Intensity transformations
Image histograms and equalization
Convolution filters
Order-statistic filters

4 Image segmentation
Thresholding
Texture segmentation
Edge detection
Active contours
5 Image Registration
Basics
6 Shape analysis
Basics

Imaging modalities

Anatomical vs. Functional Imaging

History: X-ray
Wilhelm Conrad Röntgen
8 November 1895: discovers X-rays
22 November 1895: X-rays
Mrs. Röntgen’s hand
1901: receives first Nobel Prize in
physics

History: Computed Tomography (CT)
The breakthrough
acquiring many projections around the object

enables the reconstruction of the 3D object (or a
cross-sectional 2D slice)
CT pioneers
1917: Johann Radon establishes the

mathematical framework for tomography, now
called the Radon transform
1963: Allan Cormack publishes mathematical
analysis of tomographic image reconstruction,
unaware of Radons work
1972: Godfrey Hounsfield develops first CT
system, unaware of either Radon or Cormacks
work, develops his own reconstruction method
1979: Hounsfield and Cormack receive the
Nobel Prize in Physiology or Medicine

CT: past and present
1975: Siemens SIRETON CT scanner (image size 128 × 128)
Now: Common modern CT scanner (image size 512 × 512)

3D Visualization Capabilities
Extrapolate novel Maximum intensity visualization

views of the structures
Shaded structures visualization

More visualizations

Visualization and Virtual Medicine
Offer a virtual reality environment for
Virtual examination (e.g. virtual colonscopy)
Surgical planning
Medical training

Ultrasound: past and present
1942. Dr. Karl Theodore Dussik: transmission ultrasound investigation of the
brain
1955. Holmes and Howry: Subject submerged in water tank to achieve good
acoustic coupling
1959. Automatic scanner, Glasgow
US scanner 3D Ultrasound
Intra Vascular Doppler

History: MRI
1946: Felix Bloch (Stanford) and Edward Purcell (Harvard)

demonstrate nuclear magnetic resonance (NMR)
1973: Paul Lauterbur (Stony Brook University, Nobel 2003)
published first MRI (Magnetic Resonance Imaging) image in
Nature
Late 1970s: First human MRI images conceived
Early 1980s: First commercial MRI systems available
1993: Functional MRI in humans demonstrated

MRI: basics
MRI measures the effects of magnetic properties of tissue
Effects are tissue-specific
Also specific to blood perfusion/ oxygenization (functional MRI)
MRI is very versatile (but also more expensive than CT)

MRI: basics
Permits the acquisition of several kind of images:
T1, density and T2 weighted MRI

MRI: applications
Functional MRI:
Allows to assess brain activity during certain
tasks
Valuable for brain functional studies
(cognitive sciences)
Also for surgery planning and diagnosis

MRI: recent applications
Cardiac tagged MRI:

Draw magnetic patterns
in the matter
Study how these
patterns are distorted
during heart contraction
Infer information about
heart dynamical
behavior

MRI: very recent applications
Diffusion Tensor Imaging:

Measures the diffusion of water
Allows the tracking of nerve fibers in the brain (white matter)
Visualization challenging!

Molecular imaging
Exotic but trendy

Molecular imaging provides information about specific molecular
processes
Links to genomic and proteonomics
Exploits all portions of the physical spectrum in addition to sound
No one of the previous imaging modality is ideal so combinations
must often be used
Often in vivo

Molecular imaging

Molecular imaging

Image processing: what for?
Image acquisition and reconstruction ⇒ Physics, Maths, Engineer, . . .
studies how to reconstruct efficiently meaningful images from the
raw data
Visualization ⇒ Computer graphics

studies how to visualize the reconstructed images in a way that is
useful for human observers
Image Analysis and Understanding ⇒ Computer Vision

studies how to emulate with a computer perceptual and visual
behaviors similar to the biological ones
studies models, algorithms and techniques to
I obtain objective measurements automatically
I recognize objects, structures and events

Processes in Computer Vision
Acquisition Digitalization
Signal and
Image Processing
Image Processing
Pattern Recognition Image Analysis
Artificial Intelligence Image Understanding

Low-Level Vision (image processing)
Image processing processes: set of operations performed on images

aiming at enhancing their quality and selecting useful information,
which will be processed by humans or other algorithms
Images Image processing Images

Example of low-level processing
Input Image Gaussian blur Contour extraction
Segmentation Contour selection

Mid-Level Vision (image analysis)
Includes extraction of symbolic information from pre-processed

images and analysis techniques of the visual characteristics of the
objects that are in the images
Images
Images Image analysis
Features vector of the seg-

mented objects or of the scene

Example of mid-level processing
Extraction of visual primitives:

I Area
I Circularity
Labeling

High-Level Vision (image understanding)
Aims at obtaining some “comprehension” of the observed scene, as shape recog-
nition or spatial relationship among objects. It includes high-level abstraction pro-
cesses:
Classification
Identification
Localization
Knowledge
base
Images
Image understanding Decision process
Features vector

Example of high-level processing
Clustering
Classification method
♣
Non supervised

Outline
2 Digital images
Image acquisition and representation

Digital Image Acquisition: sensor array
Light source, Object reflection Imaging System: Lenses

CCD sensor Digital Image

Digital Image Acquisition: sensor strip

Sampling & Quantization
Sampling produces continuous values f [x, y]

Digitizing the Pixel Amplitude — Quantization
n bits per pixel — 2n Discrete (monochromatic)

Digital Image Representation
The result of sampling and quantization is a matrix:
 
f (0,0) f (0,1) ··· f (0,N−1)
 f (1,0) f (1,1) · · · f (1,N−1) 
f (x, y ) = 
 
.. .. . .. .. 
 . . . 
f (M−1,0) f (M−1,1) ··· f (M−1,N−1)

Outline
3 Image enhancement
Spatial Filtering
Image histograms and equalization
Convolution filters

Spatial filtering
The simplest way to enhance

an image f
Based on direct manipulation
of the pixels
g(x, y) = T [f (x, y)]
T operator on f
defined over a neighborhood
of the pixel (x, y) (for example
a small subimage centered at
(x, y) )

Spatial filtering with neighborhood size 1 × 1
In this case g depends only on the intensity value of f at (x, y )
It is completely described as a function between intensity values:
s = T (r )

First examples

First examples

Intensity transformations: contrast enhancement
Left plot: Contrast stretching
I Darken gray values below m
I Brighten gray values above
Right plot: Thresholding
I Limit of contrast stretching
I Produces a binary image

Contrast enhancement
a) Intensity transform
b) Low contrast
image
c) Contrast
enhanced image
d) Thresholded
image

Negative
Equivalent of photographic negative

Enhance white or gray details embedded in a black background

Image histograms
Image Histogram
Discrete function
that associates to
the gray level r the
number of pixels Nr
having gray value
equal to r

Image histograms: example

Histogram equalization
An enhancement method based on intensity transformation

Find a function s = T (r ) (strictly increasing) such that the image
g(x, y) = T ◦ f (x, y )
has uniform normalized histogram

Letting p(r ) be the normalized histogram of f
X
T (r ) = p(r 0 )
r 0 ≤r

Histogram equalization: example 1

Histogram equalization: example 2

Convolution filters
Linear spatial filters based on convolution kernels

Convolution filters
Different choices of h lead to very different results:

Smoothing
Sharpening
Edge enhancement

Smoothing filters
Averaging matrix kernel:
1 1 1
1 1 1
1 1 1
(size 3 × 3)

Averaging filters
Original image mask size = 9 × 9 mask size = 35 × 35

Averaging filters
Original image Gaussian Noise added mask size = 3 × 3

Smoothing filters
Gaussian filter:
1 2 1
2 4 2
1 2 1
(size 3 × 3)

Gaussian filters

Sharpening filters
a) Two sharpening
filters
b) Original image
c) Application of I
filter
d) Application of II
filter

Sharpening filters

Edge enhancement: Definition
An edge is a location in the image where there is a steep intensity

variation
Hopefully, these discontinuities correspond to boundaries of object
of interest
How do we enhance edges?
I Determine a measure of intensity change in the pixels
neighbourhood
I First derivative of a two-variate function → Gradient

Edge enhancement: Sobel Operator
Estimation of ∇f in 2 directions:
   
−1 0 1 −1 −2 −1
hhor =  −2 0 2  hvert = 0 0 0 
−1 0 1 1 2 1
It can be thought:
1 First Gaussian blurring
2 Then derivation
Indeed, f 0 ∗ g = (f ∗ g)0 = f ∗ g 0
Sobel operator is separable!

Edge enhancement: Sobel Operator
50
100
150
200
50 100 150 200 250 300 350
Original Horizontal
50 50
100 100
150 150
200 200
50 100 150 200 250 300 350 50 100 150 200 250 300 350
Vertical Gradient norm

Edge enhancement: Laplace operator
∂2
2
2 ∂
∇ =∇ · ∇ = 2
+
∂x ∂y 2
     
0 0 0 0 1 0 0 1 0
∇2 ≈  1 −2 1  +  0 −2 1  =  1 −4 1 
0 0 0 0 1 0 0 1 0
50
100
150
200
50 100 150 200 250 300 350

Sort pixel values inside a m × n neighborhood of pixel (x, y)
9 7 11
6 4 5
1 ≤ 2 ≤ 4 ≤ 5 ≤ 5 ≤ 6 ≤ 7 ≤ 9 ≤ 11
2 5 1
The response of this class of filters depends on the ordering of the

pixel values

Order-statistics filters: Min-filter
9 7 11
6 4 5
1 ≤ 2 ≤ 4 ≤ 5 ≤ 5 ≤ 6 ≤ 7 ≤ 9 ≤ 11
2 5 1
Elimination of salt noise

Order-statistics filters: Max-filter
9 7 11
6 4 5
1 ≤ 2 ≤ 4 ≤ 5 ≤ 5 ≤ 6 ≤ 7 ≤ 9 ≤ 11
2 5 1
Elimination of pepper noise

Order-statistics filters: Median filter
9 7 11
6 4 5
1 ≤ 2 ≤ 4 ≤ 5 ≤ 5 ≤ 6 ≤ 7 ≤ 9 ≤ 11
2 5 1
Elimination of general impulsive noise

Less blurring than linear smoothing filter

Applications of Order-statistics filters: Min filter
Original Salt added Min filter 3 × 3

Applications of Order-statistics filters: Median filter
Original Salt & Pepper added Median filter 3 × 3

Applications of Order-statistics filters: Median filter
Original Gaussian noise added Median filter 3 × 3

Further Example: Anisotropic Diffusion
Original image
Slow Normal Fast Play/Pause Stop

If your pdf viewer does not support this media, click here
Image after evolution

Outline
4 Image segmentation
Thresholding
Texture segmentation
Edge detection
Active contours

Image segmentation: definition
Fundamental step in many applications

Segmentation = Partitioning of the image into homogeneous
regions with respect to some visual feature (e.g. gray level value)
Distinguishing objects from the background
Two approaches:
I Region based methods
I Edge based methods

Thresholding
Selection of an intensity T (called Threshold) capable to divide the

image into two regions, corresponding to higher or lower intensity
value
Given an image f (x, y ) and a threshold T , a binary image g(x, y )
is produced:

1 if f (x, y) ≥ T
g(x, y ) =
0 otherwise
Threshold selection depends on the intensity value of the objects

of interest

Automatic Threshold selection
In case no a priori information is available

Exploiting the statistical properties of the image, e.g. its
histogram
I Histogram valleys as thresholds
I Histogram inflection points as thresholds
I Otsu’s method (1979)
F Optimal threshold selection method
F Minimize the within-group variance
1200
1000
800
600
400
200
0 50 100 150 200 250

Otsu thresholding: Example
Original Otsu (T = 127)
T = 100 T = 160

Example: Improvement via texture features
Thresholding
Texture Features
Range Variance Haralick Entropy

Example: Improvement via texture features
Range Variance Haralick Entropy
Original Texture Smoothing Segmented

Thresholding
Image Operator Filter Image

Further Example: Watershed segmentation
Another popular region based method, like thresholding

Boundaries are local extrema of features
Drawback: over-segmentation

Further Example: Watershed segmentation
Gradient magnitude as feature

Edge based segmentation
This approach identifies the steep

intensity variations in an image,
called edges
Uses edge operators plus
binarization (e.g. thresholding)
Hopefully these correspond to object
boundaries
Edges are extended or deleted so
as to produce closed boundaries
Only good for simple images
Shape can then be used for
recognition

Example:Canny Edge Detector (1986)
Popular but powerful edge detector
1 Gaussian smoothing
2 Gradient computation
3 Search of local maxima in the direction of the gradient, i.e. ridges
in the gradient magnitude image
4 Non-maximal suppression
5 Thresholding of ridge points (actually with hysteresis)

Edge detectors: examples
Original Laplace
Canny Sobel

Further Example: Active contours
Aim: improve boundary detection

Integrate information over distance
Use cues from biological vision (Gestalt cues)
I Smoothness
I Closure
Main ideas:
I Insert an initial contour in the image domain
I Stretch and bend it according to the forces defined by the image
data
I Keep the contour smooth during the evolution with a sort of internal
energy
Active contours techniques use:
I Optimization strategies
I Calculus of variations and PDEs

Example


HEARTFAID project: Example of Activity


5 Image Registration
Basics

Image Registration: definition
Image registration is the process of determining the spatial

transform that maps points from one image to homologous points
on a object in the second image
e.g. align if the images depict the same object, align the points
corresponding to the same material point
or match structures of interest (corners, edges,. . . )
Generally, the criterion is prescribed by an application relying on
the registration task

Registration purposes
Data fusion (e.g. functional data to
anatomical data)
Construction of anatomical atlases
Mapping to anatomical atlases
Comparison of patients
Content based image retrieval
Motion analysis in image sequences
...

Registration bases
Let f , g be images, T a geometric transformation

D(f , g, T ) a criterion asserting the goodness of the matching
Then the registration problem boils down to:
T̄ = arg min D(f , g, T )

T
Thus to develop a registration framework, we may:

I Represent somehow the geometric transformation T
I Design a similarity measure D(f , g, T )
I Devise an optimization algorithm

Representing geometric transformation
Classical (matrix) groups

I rotations, scaling, isometries, affine transformations,...
Splines
General diffeomorphisms
I Discretized as Free Form Deformations (FFD)
I i.e. we assign to every pixel (voxel) x a displacement vector u(x)
s.t.:
T (x) = x + u(x)
I Requires some regularization of the solution (e.g. Tichonov
regularization )

Similarity measures
Landmark based
I If we now that a set of points {p} in the first image f correspond to
the points {q} in the second image g
I Remark: such points should be characteristic points of an object (a
contour, corners or other easily detectable couple of points)
I Then we may choose:
X
D(f , g, T ) = kTp − qk2 (1)
p∈{p}

Similarity measures
Intensity based
I Uses directly the intensity functions of the images
I Remark: No point extraction or segmentation is required
I Then we may choose some functional norm, the simplest ones
being:
Z
2
SSD(f , g) = (f (x) − g(T (x))) = kf − g ◦ T k22
Ω
Z
SAD(f , g) = |f (x) − g(T (x))| = kf − g ◦ T k1
Ω
I Also in use: mutual information, cross correlation,...

I Depending on
F statistical hypothesis (e.g. gaussian noise)
F a priori relation between the intensity functions (identical, linear
dependence, functional dependence,. . . )

Optimization
Generally gradient descent

Non convex functional ⇒ Big dependence on the initialization
Use the pyramid trick:

Examples
T2 weighted and proton density brain MRI

Examples
Moving image, initial and final alignment

6 Shape analysis
Basics

Shape analysis
Shape measurements are physical dimensional measures that

characterize the appearance of an object
The goal is to use the fewest necessary measures to characterize
an object adequately so that it may be unambiguously classified
The shape may not be entirely reconstructable from the
descriptors, but the descriptors for different shapes should be
different enough that the shapes can be discriminated

Area
The area is the number of pixels in a shape

The convex area of an object is the area of the convex hull that
encloses the object

Perimeter
The perimeter (length) is the number of pixels in the boundary of

the object
The convex perimeter of an object is the perimeter of the convex
hull that encloses the object
Perimeter External perimeter Convex perimeter

Axis
The major axis is the longest line

that can be drawn through the object
The minor axis is the longest line
whilst remaining perpendicular with
the major-axis

Aspect Ratio
The major axis is the longest line

The minor axis is the longest line
whilst remaining perpendicular with
the major-axis
The aspect ratio measures the ratio
of the objects height to its width:
Height
AspectRatio =
Width

Compactness
Compactness (also called formfactor) is defined as the ratio of the
area of an object to the area of a circle with the same perimeter:
4π · Area
Compactness =
Perimeter2

Convexity
Convexity is the relative amount that an object differs from a

convex object
A measure of convexity can be obtained by forming the ratio of the
perimeter of an objects convex hull to the perimeter of the object
itself:
Perimeterconvex
Convexity =
Perimeterexternal

Convexity

Solidity
Solidity measures the density of an object

A measure of solidity can be obtained as the ratio of the area of
an object to the area of a convex hull of the object:
Area
Solidility =
Areaconvex

Solidility

Fiber length
Fiber length gives an estimate as to the true length of a threadlike

object
Perimeter + (Perimeter2 − 16 · Area)1/2

FiberLength =
4
The estimate is fairly accurate on threadlike objects with a
formfactor that is less than 0.25 and gets worse as the formfactor
increases

Average Fiber length
The number of skeleton end-points estimates the number N of
fibers (half the number of ends)
TotalFiberLength
AverageFiberLength =
N

Reference Bibliography
Books:
1 Gonzalez RC, Woods RE: Digital Image processing, Prentice Hall, 2001
2 C.M. Bishop: Pattern Recognition and Machine Learning, Springer-Verlag, 2006
3 E.R. Davies: Machine Vision: Theory, Algorithms, Practicalities, Morgan Kaufmann, 2005
4 R. Hartley, A. Zisserman: Multiple View Geometry in Computer Vision, Cambridge University Press, 2003
5 Sonka M, Hlavac V, Boyle R: Image Processing analysis and machine vision, PWS pubs, 2nd ed., 1999
6 Haralick RM, Shapiro LG: Computer and Robot Vision, Vol I-II, Addison-Wesley, Mass., USA, 1992
7 Forsyth DA, Ponce J: Computer Vision: A Modern Approach, Prentice Hall, 2003
8 Yao Wang, Jrn Ostermann, Ya-Qin Zhang: Video Processing and Communications, Prentice Hall, 2005
9 Ballard DH, Brown CM: Computer Vision, Prentice-Hall, 1982
10 Horn BKP: Robot Vision, MIT Press, 1986
11 Pratt WK: Digital Image Processing, John Wiley & Sons, inc., 3rd Edition, 2001
12 Jain AK: Fundamentals of Digital Image Processing, Englewood Cliffs, NJ:Prentice H
Journals:
1 IEEE Transaction on Pattern Analysis and Machine Intelligence
2 IEEE Transaction on Image Processing International
3 Journal of Computer Vision Image and Vision Computing
4 Pattern Recognition
5 Journal of Mathematical Imaging and Vision
6 Pattern Recognition and Image

Sotware tools
Photoshop MATLAB: Image Processing

GIMP Toolbox
ImageJ ITK
Mathematica and Digital VTK
Image Processing Toolbox AVS

Credits
Some materials for this presentation have been draw from public resources available
on the World Wide Web.
In particular, most historical data were taken from Klaus Mueller presentation
(http://www.cs.sunysb.edu/˜mueller/).
Some images and data were taken from:
Gonzalez RC, Woods RE: Digital Image processing
Sam Gahmbir
(http://mips.stanford.edu/public/video_lectures/index.adp)
Guy Gilboa, Nir Sochen and Yehoshua Y. Zeevi (http:
//www.math.ucla.edu/˜gilboa/PDE-based_image_filtering.html)
Chunming Li, Chenyang Xu, Changfeng Gui and Martin D. Fox
(http://www.engr.uconn.edu/˜cmli/code/)
Johns Hopkins University (http://www.jhu.edu/)
Philips, Siemens, GE,...
If you find any omission in the list above, please contact us.

Tutorial Biomedical

Uploaded by

Copyright:

Available Formats

Tutorial Biomedical

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Tutorial Biomedical

Uploaded by

Copyright:

Available Formats

Image Processing in Biomedical Applications

S. Colantonio, D. Moroni, O. Salvetti

Signals & Images Lab

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 1 / 109

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 2 / 109

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 3 / 109

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 4 / 109

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 5 / 109

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 6 / 109

acquiring many projections around the object

1917: Johann Radon establishes the

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 7 / 109

Now: Common modern CT scanner (image size 512 × 512)

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 8 / 109

Extrapolate novel Maximum intensity visualization

Shaded structures visualization

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 9 / 109

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 10 / 109

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 11 / 109

Intra Vascular Doppler

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 12 / 109

1946: Felix Bloch (Stanford) and Edward Purcell (Harvard)

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 13 / 109

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 14 / 109

Permits the acquisition of several kind of images:

T1, density and T2 weighted MRI

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 15 / 109

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 16 / 109

Cardiac tagged MRI:

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 17 / 109

Diffusion Tensor Imaging:

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 18 / 109

Exotic but trendy

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 19 / 109

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 20 / 109

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 21 / 109

Visualization ⇒ Computer graphics

Image Analysis and Understanding ⇒ Computer Vision

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 22 / 109

Pattern Recognition Image Analysis

Artificial Intelligence Image Understanding

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 23 / 109

Image processing processes: set of operations performed on images

Images Image processing Images

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 24 / 109

Input Image Gaussian blur Contour extraction

Segmentation Contour selection

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 25 / 109

Includes extraction of symbolic information from pre-processed

Images Image analysis

Features vector of the seg-

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 26 / 109

Extraction of visual primitives:

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 27 / 109

Image understanding Decision process

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 28 / 109

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 29 / 109

SI-LAB (ISTI-CNR) Image Processing in Biomedical Applications 30 / 109

Light source, Object reflection Imaging System: Lenses