Chapter 19 Numerical Differentiation

Estimate the derivatives (slope, curvature, etc.)

of a function by using the function values at
only a set of discrete points
Ordinary differential equation (ODE)
Partial differential equation (PDE)
Represent the function by Taylor polynomials
or Lagrange interpolation
Evaluate the derivatives of the interpolation
polynomial at selected (unevenly distributed)
nodal points
 y f(x)
dy f ( x x) f ( x)
Differentiation lim
dx x0 ( x x) x
y f ( x x) f ( x) dy
Difference
x ( x x ) x dx

A smaller step x (or h) results in a smaller error

 Evenly distributed points along the x-axis
x h

xi-3 xi-2 xi-1 xi xi+1 xi+2 xi+3

Distance between two neighboring points is the same, i.e. h.

Unevenly distributed points along the x-axis

x1 x2 x3
Numerical Differentiation

Forward difference

Backward difference

Centered difference
Forward difference

h
x
xi1 xi xi+1
Backward difference

h
x
xi1 xi xi+1
Centered difference

2h
x
xi1 xi xi+1
 y
First Derivatives
f ( x )

x
i-2 i-1 i i+1 i+2
f ( x i 1 ) f ( x i ) y i 1 y i
Forward difference f ( x )
x i 1 x i x i 1 x i

Backward difference f ( x ) f ( x i ) f ( x i 1 ) y i y i 1
x i x i 1 x i x i 1
Central difference f ( x i 1 ) f ( x i 1 ) y i 1 y i 1
f ( x )
x i 1 x i 1 x i 1 x i 1
 Truncation Errors
Uniform grid spacing
h2 h3
f ( x i 1 ) f ( x i h ) f ( x i ) hf ( x i ) f ( x i ) f ( x i )
2! 3!
2 3
f ( x ) f ( x h ) f ( x ) hf ( x ) h
f ( x i )
h
f ( x i )

i 1 i i i
2! 3!

f ( xi1 ) f ( xi ) h
forward : f ( x i ) f ( 1 ) O(h)
h 2

f ( xi ) f ( xi 1 ) h
backward : f ( x i ) f ( 2 ) O(h)
h 2
f ( xi 1 ) f ( xi 1 ) h2
central : f ( x i ) f ( 3 ) O(h 2 )
2h 6
 Example: First Derivatives
Use forward and backward difference approximations
to estimate the first derivative of
f ( x ) 0.1 x 4 0.15 x 3 0.5 x 2 0.25 x 1.2
at x = 0.5 with h = 0.5 and 0.25 (exact sol. = -0.9125)
Forward Difference
f ( 1) f (0.5 ) 0.2 0.925
h 0.5 , f ( 0.5 ) 1.45 , t 58.9%
1 0.5 0.5

h 0.25 , f (0.5 ) f (0.75 ) f (0.5 ) 0.63632813 0.925 1.155 , 26.5%
0.75 0.5 0.25
t

Backward Difference
f (0.5 ) f (0 ) 0.925 1.2
h 0.5 , f ( 0.5 ) 0.55 , t 39.7%
0.5 0 0.5

h 0.25 , f (0.5 ) f (0.5 ) f (0.25 ) 0.925 1.10351563 0.714 , 21.7%
0.5 0.25 0.25
t
 Example: First Derivative
Use central difference approximation to estimate the
first derivative of
f ( x ) 0.1 x 4 0.15 x 3 0.5 x 2 0.25 x 1.2
at x = 0.5 with h = 0.5 and 0.25 (exact sol. = -0.9125)
Central Difference
f ( 1 ) f ( 0 ) 0.2 1.2
h 0.5 , f ( 0.5 ) 1.0 , t 9.6%
10 1

f ( 0.75 ) f ( 0.25 )
h 0.25 , f ( 0.5 )
0.75 0.25
0.63632813 1.10351563
0.934 , t 2.4%
0.5
 Second-Derivatives
Taylor-series expansion
Uniform grid spacing
h2 h3 h4
f ( x i 1 ) f ( x i ) hf ( x i ) f ( x i ) f ( x i ) f ( x i )
2! 3! 4!
2 3 4
f ( x ) f ( x ) hf ( x ) h f ( x ) h f ( x ) h f ( x )
i 1 i i
2!
i
3!
i
4!
i

h2 h4
f ( x i 1 ) f ( x i 1 ) 2 f ( x i ) f ( x i ) f ( x i )
2! 4!

Second-order accurate O(h2)

f ( x i 1 ) 2 f ( xi ) f ( xi 1 ) h2
f ( xi ) 2
f ( )
h 4!
Centered Finite-Divided Differences
Forward Finite-divided differences
Backward finite-divided differences
 First Derivatives
f ( x )
Parabolic curve

i-2 i-1 i i+1 i+2

3 -point Forward difference
f ( xi 2 ) 4 f ( xi1 ) 3 f ( xi )
f ( x )
xi 2 xi
3 -point Backward difference
3 f ( xi ) 4 f ( xi 1 ) 3 f ( xi 2 )
f ( x )
xi xi2
 Example: First Derivatives
Use forward and backward difference approximations
of O(h2) to estimate the first derivative of
f ( x ) 0.1 x 4 0.15 x 3 0.5 x 2 0.25 x 1.2
at x = 0.5 with h = 0.25 (exact sol. = -0.9125)
Forward Difference
f ( 1 ) 4 f ( 0.75 ) 3 f ( 0.5 )
f ( 0.5 )
2( 0.25 )
0.2 4( 0.6363281 ) 3( 0.925 )
0.859375 , t 5.82%
0.5
Backward Difference
3 f ( 0.5 ) 4 f ( 0.25 ) f ( 0 )
f ( 0.5 )
2( 0.25 )
3( 0.925 ) 4( 1.035156 ) 1.2
0.878125 , t 3.77%
0.5
 Higher Derivatives
All second-order accurate O(h2)
f ( x i 1 ) f ( x i 1 )
f ( x i )
2h
f ( x i 1 ) 2 f ( x i ) f ( x i 1 )
f ( x i )
h2
f ( x i 2 ) 2 f ( x i 1 ) 2 f ( x i 1 ) f ( x i 2 )
f ( x i )
h3
f ( x i 2 ) 4 f ( x i 1 ) 6 f ( x i ) 4 f ( x i 1 ) f ( x i 2 )

f ( xi )
h4

More nodal points are needed for higher derivatives

Higher order formula may be derived
 19.3 Richardson Extrapolation
D D(h1 ) a2 h1
2
a4 h1
4
........ D(h1 ) E1 (h1 )
2
(1)

D is the true value but unknown and D(h1) is an approximation

based on the step size h1. Reducing the step size to half, h2 =h1/2,
we obtained another approximation D(h2).

D D( h2 ) a2h22 a4 h24 D( h2 ) E2 ( h22 ) (2)

E1( h12 ) 4 D( h2 ) D( h1 )
4 Hence 4*(2) - (1), D +E3 ( h14 )
E2 ( h22 ) 3
By properly combining the two approximations, D(h1) & D(h2),
the error is reduced to O(h4).
 Example of using Richardson Extrapolation
Central Difference Scheme
1 h2
f '( x ) D h [ f ( x h ) f ( x h )] f '''( x ) O ( h 4 ) (1)
2h 6
1 h2
f '( x ) D h / 2 [ f ( x h / 2) f ( x h / 2)] f '''( x ) O( h 4 ) (2)
h 24

4D( h / 2) D( h)
4*(2) - (1), f ' O( h )
4
3

By combining the two approximations, D(h/2) & D(h), the error

of f(h) is reduced to O(h4).
 Ex19.2: Richardson Extrapolation
Use central difference approximation to estimate the
first derivative of
f ( x ) 0.1 x 4 0.15 x 3 0.5 x 2 0.25 x 1.2
at x = 0.5 with h = 0.5 and 0.25 (exact sol. = -0.9125)

f (1) f (0)
D1 ( h 0.5) 1.0, t 9.6%
1 0

f (0.75) f (0.25)
D2 ( h 0.25) 0.934, t 2.4%
0.75 0.25

4 D2 D1
D 0.9125, t 0%
3
 General Three-Point Formula
Lagrange interpolation polynomial for
unequally spaced data

f ( x ) Li 1 ( x ) f ( x i 1 ) Li ( x ) f ( x i ) Li 1 ( x ) f ( x i 1 )
( x x i )( x x i 1 )
f ( xi 1 )
( x i 1 x i )( x i 1 x i 1 )
( x x i 1 )( x x i 1 )
f ( xi )
( x i x i 1 )( x i x i 1 )
( x x i 1 )( x x i )
f ( xi1 )
( x i 1 x i 1 )( x i 1 x i )
 Lagrange Interpolation
1st-order Lagrange polynomial
x x1 x x0
f1 ( x ) L0 f ( x0 ) L1 ( x ) f ( x1 ) f ( x0 ) f ( x1 )
x0 x1 x1 x0

Second-order Lagrange polynomial

( x x1 )( x x 2 )
f2 ( x ) f ( x0 )
( x0 x1 )( x0 x 2 )
( x x0 )( x x 2 )
f ( x1 )
( x1 x0 )( x1 x 2 )
( x x0 )( x x1 )
f ( x2 )
( x 2 x0 )( x 2 x1 )
 Lagrange Interpolation
Third-order Lagrange polynomial
( x x 1 )( x x 2 )( x x 3 )
f3 ( x ) f ( x0 )
( x0 x 1 )( x0 x 2 )( x0 x 3 )
( x x0 )( x x 2 )( x x 3 )
f ( x1 )
( x 1 x0 )( x 1 x 2 )( x 1 x 3 )
( x x0 )( x x 1 )( x x 3 )
f ( x2 )
( x 2 x0 )( x 2 x 1 )( x 2 x 3 )
( x x0 )( x x 1 )( x x 2 )
f ( x3 )
( x 3 x0 )( x 3 x 1 )( x 3 x 2 )
Lagrange Interpolation
L0(x)f(x0) L2(x)f(x2)

x0 x1 x2

L1(x)f(x1)
 General Three-Point Formula
Lagrange interpolation polynomial for
unequally spaced data
f ( x ) Li 1 ( x ) f ( x i 1 ) Li ( x ) f ( x i ) Li 1 ( x ) f ( x i 1 )
( x x i )( x x i 1 ) ( x x i 1 )( x x i 1 )
f ( xi 1 ) f ( xi )
( x i 1 x i )( x i 1 x i 1 ) ( x i x i 1 )( x i x i 1 )
( x x i 1 )( x x i )
f ( xi1 )
( x i 1 x i 1 )( x i 1 x i )

First derivative
2 x xi xi1 2 x xi 1 xi 1
f ( x ) f ( x i 1 ) f ( xi )
( x i 1 x i )( x i 1 x i 1 ) ( x i x i 1 )( x i x i 1 )
2 x x i 1 xi
f ( xi1 )
( x i 1 x i 1 )( x i 1 x i )
 Second Derivative
First Derivative for unequally spaced data

2 x xi xi1 2 x xi 1 xi 1
f ( x ) f ( x i 1 ) f ( xi )
( x i 1 x i )( x i 1 x i 1 ) ( x i x i 1 )( x i x i 1 )
2 x x i 1 xi
f ( xi1 )
( x i 1 x i 1 )( x i 1 x i )

Second Derivative for unequally spaced data

2 2
f ( x ) f ( x i 1 ) f ( xi )
( x i 1 x i )( x i 1 x i 1 ) ( x i x i 1 )( x i x i 1 )
2
f ( xi1 )
( xi1 x i 1 )( x i 1 x i )
Differentiation of Noisy Data
 MATLABs Methods
Derivatives are sensitive to the noise
Use least square fit before taking derivatives

p = polyfit(x, y, n) - coefficients of Pn(x)

polyfit(p, x) - evaluation of Pn(x)
polyder(p) - differentiation
x x( 1 ), x( 2 ), , x( n )
diff ( x ) x( 2 ) x( 1 ), x( 3 ) x( 2 ), , x( n ) x( n 1 )
dy/dx at x(i) forward difference
dy diff(y)./d iff(x)
dy/dx at x(i 1) backward difference