Chapter 3 Wiener Filters
Chapter 3 Wiener Filters
Chapter 3 Wiener Filters
1. Introduction
l
Wiener Filters
Wiener filters are a class of optimum linear filters which involve linear estimation of a desired signal sequence from another related sequence. In the statistical approach to the solution of the linear filtering problem, we assume the availability of certain statistical parameters (e.g. mean and correlation functions) of the useful signal and unwanted additive noise. The problem is to design a linear filter with the noisy data as input and the requirement of minimizing the effect of the noise at the filter output according to some statistical criterion. A useful approach to this filter-optimization problem is to minimize the mean-square value of the error signal that is defined as the difference between some desired response and the actual filter output. For stationary inputs, the resulting solution is commonly known as the Weiner filter. The Weiner filter is inadequate for dealing with situations in which nonstationarity of the signal and/or noise is intrinsic to the problem. In such situations, the optimum filter has to be assumed a time-varying form. A highly successful solution to this more difficult problem is found in the Kalman filter.
Fig. 3.1 shows the block schematic of a linear discrete-time filter W(z) for estimating a desired signal d(n) based on an excitation x(n)
d(n) x(n) y(n) + e(n)
W(z)
Fig. 3.1
l
We assume that both x(n) and d(n) are random processes (discrete-time random signals). The filter output is y(n) and e(n) is the estimation error. To find the optimum filter parameters, the cost function or performance function must be selected. In choosing a performance function the following points have to be considered (1) The performance function must be mathematically tractable. (2) The performance function should preferably have a single minimum so that the optimum set of the filter parameters could be selected unambiguously.
The number of minima points for a performance function is closely related to the filter structure. The recursive (IIR) filters, in general, result in performance function that may have many minima, whereas the non-recursive (FIR) filters are guaranteed to have a single global minimum point if a proper performance function is used. In Weiner filter, the performance function is chosen to be
= E[ e(n) ]
2
3. Wiener Filter the Transversal, Real-Valued Case Fig. 3.2 shows a transversal filter with tap weights w0, w1, , wN-1 . x(n) w0 z-1 w1 x(n-1) z-1 z-1 wN-1
Fig. 3.2
Let
W = [ w0 X ( n) = [ xn
w1 ... wN 1 ]T xn 1 ... x n N +1 ]T
N 1 i =0
....(1)
...(6)
Then we obtain
= E[d 2 ( n)] 2W T P + W T RW ...(7)
Equation (7) is a quadratic function of the tap-weight vector W with a single global minimum. Note that R has to be a positive definite matrix in order to have a unique minimum point in the w-space.
l
Minimization of performance function To obtain the set of tap weights that minimize the performance function, we set or
= 0 , for i = 0,1,..., N 1 wi
... (8)
= 0
...(9)
and
w w
l =0 m=0 l
N 1 N 1 l =0 m=0 l
N 1 N 1
m lm
can be expanded as
N 1 N 1 l =0 m=0 l i mi N 1 2
w w
m lm
...(10)
Then we obtain
N 1 = 2 p i + Wl (rli + ril ) wi l =0
...(11)
W ( r
l =0 l
N 1
li
+ ril ) = 2 pi
...(12)
Note that
rli = E[ x (n l ) x (n i )] = xx (i l )
The symmetry property of autocorrelation function of real-valued signal, we have the relation rli = ril Equation (12) then becomes
r w
l =0 il
N 1
= pi
for i = 0,1,2,..., N 1
where Wop is the optimum tap-weight vector Equation (13) is also known as the Wiener-Hopf equation, which has the solution Wop = R 1 P Assuming that R has inverse. The minimum value of is
min = E [d 2 (n )] Wop P
T
...(14)
...(15)
Assuming that
d (n ) = 2 x( n) + 3 x(n 1) + v (n)
and v(n) represents a stationary white process v2=0.1 , x(n) is also a white process, with unit variancex2=1 Note By this assumption, the actual system function of the plant is 2+3z-1 Design an optimum filter (Weiner filter) to model the plant. Question How to choose the number of taps (N) Once N is determined, the procedure to solve the problem is straightforward. If N=2 is chosen p.55(textbook) then R = 0 1
2 P= 3 1 0
for i = 0,1,2,..., N 1
Then we obtain
= 2 E [e (n) x (n i)] wi for i = 0,1,2,..., N 1
l
...(20)
When the Wiener filter tap, weights are set to their optimal values,
=0. wi
Hence, if e0(n) is the estimation error when wi are set to their optimal values, then equation (20) becomes
E[ e0 ( n ) x( n i )] = 0 for i = 0,1,..., N 1 ...(21)
That is, the estimation error is uncorrelated with the filter tap inputs, x(n-i). This is known as the principles of orthogonality.
l
We can also show that the optimal filter output is also uncorrelated with the estimation error. That is, E[ e0 (n ) y 0 (n )] = 0
....(22)
This result indicates that the optimized Weiner filter output and the estimation error are orthogonal.
If the optimal filter tap weights are expressed by w0,l , l=0,1,2, ,N-1. The estimation error is then given by
e0 (n ) = d (n ) w0,l x (n l )
l =0 N 1
...(23)
and then
d ( n) = e 0 ( n ) + y 0 ( n)
2
...(24)
2
...(26)
and we obtain
min = E[d 2 (n )] E [ y0 (n )]
2
...(27)
?=1 when y(n)=0 ? reaches its minimum value, ?min , when the filter tap-weights are chosen to achieve the minimum mean-squared error. This gives min = 1
E [ y0 (n )] E [d 2 ( n )] ...(29)
6. Wiener Filter Complex-Valued Case In many practical applications, the random signals are complex-valued. For example, the baseband signal of QPSK & QAM in data transmission systems. In the Wiener filter for processing complex-valued random signals, the tap-weights are assumed to be complex variables. The estimation error, e(n), is also complex-valued. We may write
= E [ e(n ) ] = E [e(n )e* (n )]
2
...(30)
The optimum tap-weights of the complex-valued Wiener filter will be obtained from the criterion:
c wi = 0 for i = 0,1,2,..., N 1
That is,
= 0 and =0 wi , R wi ,R
...(33)
...(34)
Noting that
e(n ) = d (n ) wk x (n k )
k =0 c c wi e ( n ) = x( n i ) wi wi * * c * c wi e ( n ) = x ( n i ) wi w i N 1
c wi wi =
wi wi + j = 1 + j ( j) = 0 wi , R wi , I
* and c w wi =
i
wi* wi* + j = 1 + j( j ) = 2 wi ,R wi , I
...(38)
E[ e0 ( n ) x * (n i )] = 0
for i = 0,1,2,..., N 1
...(39)
Equation (39) is the principle of orthogonality for the case of complex-valued signals in wiener filter. The Wiener-Hopf equation can be derived as follows: Define x (n ) [ x (n ) x( n 1) L x (n N + 1)]T and
* w ( n) [w0 T w1* L w* N 1 ]
...(40)
...(41)
...(42)
from equation(39)
E[ x ( n)(d * (n ) x H (n )w0 )] = 0
...(46)
and then
Rw0 = p ...(47)
where R = E[ x (n ) x H (n )] and p = E [ x (n )d * (n )] Equation (47) is the Wiener-Hopf equation for the case of complex-valued signals. The minimum performance function is then expressed as
min = E [d 2 (n )] w0H R w0 ...(48)
Remarks: In the derivation of the above Wiener filter we have made assumption that it is causal and finite impulse response, for both real-valued and complex-valued signals.
W(z)
Fig. 3.1 We assume that the filter W(z) may be non-causal and/or IIR.
l
We consider only the case the signals and the system parameters are real-valued. Moreover, we assume that the complex variable z remains on the unit circle, i.e. z = 1 , thus z*=z-1 , z = 1 .
W ( z ) = W ( z )W * ( z ) and W * ( z ) = W ( z 1 )
2
The performance function given in equation(51) covers IIR and FIR This is a most general form of the Wiener filter performance function. Example 3.2 From general expression of , eq.(51) to demonstrate the special case of real-valued FIR system. Consider the case where the Wiener filter is an N-tap FIR filter with system function W ( z ) = wl z l
l=0 N 1
...(52)
= dd (0) + wl w m
l =0 m=0
N 1 N 1
N 1 1 1 m l 1 z z dz wl xd ( z ) z l 1 dz ( ) 2 xx 2j c 2j c l =0
Note that
p = E[ x (n)d (n)] Thus pl = E[ x( n l )d (n )]
By definition, xy ( k ) = E[ x (n) y * (n k )]
xd (l ) = E[ x (n)d (n + l )] = E[ x( n l )d ( n)]
We assume that all involved signals and system parameters are real-valued. x(n) : input sequence, white process with zero-mean, x2 = 1 . v(n) : additive noise. x(n) & v(n) are uncorrelated. Thus, xx ( z ) = 1 and vx ( z ) = 0 G(z) : system function of the unknown plant.
xd ( z) = G( z 1 ) xx ( z )
We may find that there can be many local minima, and searching for the global minimum of may not be a trivial task.
...(52)
where e0 (n) = d ( n)
0 ,l
x (n l )
...(53)
0 ,l
...(54)
and
W0 (e jw ) = dx (e jw ) xx (e jw ) ...(57)
dx (e jw ) : cross-power spectral density of d(n) and x(n) xx (e jw ) : power spectral density of x(n)
8. Application of Wiener Filters Modelling Consider the modeling problem depicted in Fig. 3.8
d(n) + e(n)
i(n)
x(n)
(n), 0(n), andi(n) are assumed to be stationary, zero-mean and uncorrelated with one another.
Note that
xx ( k ) = E[ x( n) x( n k )] = E [( ( n ) + i (n ))( (n k ) + i (n k ))] = E [ ( n ) ( n k )] + E [ (n )i ( n k )] + E [ i (n ) ( n k )] + E [ i (n ) i (n k )] = (k ) + ii ( k ) ...(61)
where d(n) is the plant output when the additive noise0(n) is excluded from that. Moreover, we have
d ' ( z ) = G( z) ( z )
That is, when i(n) is zero for all values of n. The noise sequencei(n) may be thought of as introduced by a transducer that is used to get samples of the plant input.
l
...(64)
( e jw ) l Define K (e ) (e jw ) + ii ( e jw )
jw
...(65)
We obtain W0(ejw)=K(ejw)G(ejw)
With some mathematic manipulation, we can find the minimum mean-square error, min , expressed by
min = 00 (0) + 1 2
(1 K (e jw )) (e jw ) G( e jw ) dw
...(66)
The best performance that one can expect from the unconstrained Wiener filter is
min = 00 (0)
and this happens when i(n)=0 . The Wiener filter attempts to estimate that part of the target signal d(n) that is correlated with its own input x(n) and leaves the remaining part of d(n) (i.e. 0(n)) unaffected. This is known as the principles of correlation cancellation .
9. Application of Wiener Filters Inverse Modelling Fig. 3.9 depicts a channel equalization scenario.
d(n)=s(n) ?(n) s(n) H(z) y(n) x(n) W(z) + e(n) -
s(n) : data samples H(z) : System function of communication channel ?(n) : additive channel noise which is uncorrelated with s(n) W(z) : filter function used to process the received noisy signal samples to recover the original data samples.
l
When the additive noise at the channel output is absent, the equalizer has the following trivial solution:
W0 ( z ) = 1 H ( z) ...(67)
When the channel noise, ?(n), is non-zero, the solution provided by equation(67) may not be optimal.
x (n ) = hn * s(n ) + (n ) ...(68)
and d ( n) = s( n ) ...(69) where hn is the impulse response of the channel, H(z). From equation(68), we obtain
xx ( z ) = ss ( z ) H ( z) + ( z )
2
...(70)
Also,
dx ( z ) = sx ( z ) = H ( z 1 ) ss ( z ) ...(71)
and then
W0 ( z) = H * ( z) ss ( z ) ss ( z ) H ( z ) + ( z )
2
...(73)
This is the general solution to the equalization problem when there is no constraint on the equalizer length and, also, it may be let to be non-causal.
l
1 H (z)
...(74)
ss (e jw ) H ( e jw ) (e jw )
2
...(75)
where ss ( e jw ) H (e jw )
density and the noise power spectral density, respectively, at the channel output. We obtain
W0 (e jw ) = (e jw ) 1 jw 1 + (e ) H (e jw ) ...(76)
Note that ?(ejw) is a non-negative quantity, since it is the signal-to-noise power spectral density ratio at the equalizer input. Also, 0
( e jw ) 1 1 + (e jw ) ...(77)
Cancellation of ISI and noise enhancement Consider the optimized equalizer with frequency response given by
W0 (e jw ) = (e jw ) 1 jw 1 + (e ) H (e jw )
In the frequency regions where the noise is almost absent, the value of is very large and hence
W0 (e jw ) 1 H (e jw )
The ISI will be eliminated without any significant enhancement of noise. On the other hand, in the frequency regions where the noise level is high, the value of ?(ejw) is not large and hence the equalizer does not approximate the channel inverse well. This is of course, to prevent noise enhancement.
+ -
e(n)
s(n) : signal source ) : noise source s(n) & ?(n) are uncorrelated H(z) & W(z) are two system functions used to mixed the two input signals to form d(n) & x(n). d(n) : primary input x(n) : Reference input x(n)=?(n)+hn*s(n) d(n)=s(n)+gn*?(n)
l
(78) (79)
Since s(n) & ?(n) are uncorrelated with each other, we obtain
dx ( z ) = s dx ( z ) + dx ( z )
...(81)
where
and dx ( z) is dx ( z ) when s(n)=0 for all n . This is because s(n) and ?(n) are uncorrelated with each other, their contribution in dx ( z) can be considered separately. Thus, we obtain
* s dx ( z) = H ( z ) ss ( z )
...(82)
and dx ( z) = G ( z ) ( z )
...(83)
and W0 ( z ) =
H * ( z ) ss ( z ) + G( z ) ( z ) ( z) + ss ( z ) H ( z)
2
...(85)
Equation(85) for noise cancellation may be thought of as a generalization of the results we obtained for the modeling and inverse modeling scenarios, given by equation(63) and (73). To minimize the mean-square value of the output error, we must strike a balance between noise cancellation and signal cancellation at the output of the noise canceller. Cancellation of the noise ?(n) occurs when the Wiener filter W(z) is chosen to be close to G(z), and cancellation of the signal s(n) occurs when the Wiener filter W(z) is close to the inverse of H(z). In the sense, we may note that the noise canceller treats s(n) and ?(n) without making any distinction between them. Define
pri (e jw ) : signal-to-noise PSD at primary input
...(86)
ref ( e jw ) =
...(87)
through two routes one direct and one through the cascade of H(z) and W(z) :
jw jw jw jw s ee (e ) = 1 H (e )W (e ) ss (e ) 2
...(88)
Similarly, ?(n) reaches the output through the routes G(z) and W(z) :
jw jw jw jw ee (e ) = G ( e ) W (e ) ( e ) 2
...(89)
(e jw ) + H ( e jw ) ss (e jw )
2 ee
(e jw )
and (e ) =
jw
jw H ( e jw ) 1 G(e jw ) H (e jw ) 2 ss ( e )
(e jw ) + H (e jw ) ss (e jw )
( e jw )
(e jw ) H ( e jw ) ss ( e jw ) 1 ref ( e jw ) ...(90)
2
This is known as power inversion . ( Widrow et al, 1975) ** This result shows that the noise canceller works better when
ref ( e jw ) is low.
~ x(n) x (n)
w1
s(n) arrives the receiver (antennas) in the direction perpendicular to the line connecting A & B. ?(n) arrives at an angle ?0 with respect to the direction of s(n). a(n) and (n) are narrowband baseband signals. Thus s(n) & ?(n) can be treated as narrowband signals concentrated around w=w0 . since s(n) & ?(n) may be treated as single tones, and thus a filter with two degrees of freedom is sufficient for optimal filtering. Wiener filter coefficients can be determined as follows. s(n) arrives at the same time at both omni-antennas. ?(n) arrives at B first and arrives at A with a delay
0 = l sin 0 c
the delay is normalized by the time step T which corresponds to one increment of time index n. This gives
0 = l sin 0 cT
Thus
d ( n) = (n ) cos nw0 + ( n) cos[(n 0 )w0 ] s (n ) = ( n) cos nw0 + (n ) cos nw0 ~( n) = (n ) sin nw + (n ) sin nw x 0 0 2 ~ E [ x ( n)] E[ x( n ) x (n )] R= ~ ~ 2 ( n )] E[ x E [ x (n ) x ( n )]