Advanced Control

State Regulator

Scope

design of controllers using pole placement and LQ design rules

Keywords

pole placement,
optimal control, LQ regulator, weighting matrices

Prerequisites

state space description

Contact

Roland Bchi, [email protected]

Markus Kottmann, [email protected]

Date

January 17, 2011

Advanced Control, State Regulator

MSE

Preliminary Remarks

The path to be followed in this and the next document is the following:
The first point to be discussed concerns the design of regulators which make
use of the state vector x to control a plant. It is assumed that the plant is
controllable.
The next point concerns the estimation of the state vector x in case that it is
not completely measured. To design such estimators (observers) it is assumed
that the plant is observable.
The third theme concerns the cooperation between the regulator and the observer. Instead of using the true state-vector, the regulator then utilizes the
estimated state-vector. Assumed that the two separate designs of regulator
and observer met their specific requirements are these requirements still
fulfilled if the regulator and the observer are used together?
The assumptions about the plant to be controlled are as follows:
Preferably, the plant is controllable and observable.

If the plant lacks one or both of the above-mentioned properties (see the examples in Fig. 1), then the non-controllable or non-observable parts can be
removed from the model our methods will work for the remaining model
which is controllable and observable.
b)
a)
X1
X1
1
1
U
U
Y
Y
s+1
s+1
++
++
X2
X2
2
1
s+1

c)
U

s2

s2
s+1

X1

1
s2

X2

Figure 1: Examples of systems that are not controllable

If the removed parts are e.g. unstable or badly damped then the control system
will not be very successful. In that case, additional actuators or sensors are
needed.
The plant is assumed to be a low-pass system with a feed-through matrix
D = 0. The case D 6= 0 is not difficult it just produces larger equations
and diagrams.
In the sequel, the plant often is given as a SISO-system. Of course, that is not a
restriction. In principle, it is easier to control a plant with more sensors and more
actuators.
2

Roland Bchi, Markus Kottmann

January 17, 2011

MSE

Advanced Control, State Regulator

Feedback of the Full State Vector

Figure 2: Control structure with feedforward term and full state feedback
In Fig. 2, the control system has two components: the feedback part K, and the
feedforward part Kvf . The state feedback K is used to determine the dynamics of
the closed-loop system, while the prefilter Kvf is used to guarantee the static gain
(y = r )1 .
Note that the error signal e = r y which is used in classical output feedback does
not appear in this control structure.
In the feedback signal uR , the state variables x1 (t), x2 (t) . . . xn (t) are weighted by
the coefficients k1 , k2 , . . . , kn .

x1



x2
uR = k1 x1 + k2 x2 + ... + kn xn = k1 k2 ... kn .. = Kx
.
xn
The control signal u is

u = w uR = Kvf r K x
The matrix equations are
x = A x + B (Kvf r Kx) = A x BKx + BKvf r
x = (A BK) x + BKvf r

and y = C x

The whole system can be described by a new set of matrices Ag , Bg , Cg

Ag = A BK

, Bg = BKvf

, Cg = C

The prefilter, as used here, is just a gain. More sophisticated prefilters might be used to shape
the dynamics of Y (s)/R(s).

January 17, 2011

Roland Bchi, Markus Kottmann

Advanced Control, State Regulator

MSE

The I/O-description becomes

G(s) =

Y (s)
= Cg (sI Ag )1 Bg = C(sI A + BK)1 BKvf
R(s)

For the moment we assume that K is known. Then the static gain can be
adjusted by setting G(0) = 1 or
Kvf = C(A + BK)1 B

1

(1)

This method is sensitive to parameter variations, so in section 4 we will discuss an

alternative which includes integral action.
Sections 2 and 3 deal with the question how to calculate K.

Pole Placement

According to Definition B of controllability, the controllability of (A, B) ensures

that the eigenvalues 1 . . . n of Ag = A BK can be placed arbitrarily by proper
choice of K. The eigenvalues of Ag are given by the roots of det(I A + BK). To
achieve some desired eigenvalues 1 . . . n the following polynomials must coincide:
det(I A + BK) = ( 1 )( 2 ) . . . ( n )

(2)

The question which arises is about the choice of good pole locations. Some
considerations are:
Obviously, the poles must be chosen in the left half plane for stability reasons.
Transients of the signals should decay towards zero faster than some C e1 .

The transients should be well damped, e.g. > 1/ 2.

Forcing transients to decay fast towards zero needs energy. Therefore, an
upper bound 2 > is meaningful.
While the placement of poles is simple for low order systems, it gets somehow
unsatisfying when handling systems of order e.g. 4 or higher.

Optimal Control

An alternative way to determine K makes use of the fundamental idea of optimality.

Changing the value of K results in a different behavior of the control system. If the
behavior of the control system is judged quantitatively, it is convenient to have a
scalar performance index (or cost function) J = f (K). Normally, better performance
corresponds to smaller values of J. The optimal value of K then can be found by
minimizing J, e.g. by varying K systematically. In control theory, systems that
4

Roland Bchi, Markus Kottmann

January 17, 2011

MSE

Advanced Control, State Regulator

= 1/

(2)

Im

Re

= 1/

(2)

Figure 3: Qualitative sketch of region of good pole locations

are adjusted to provide a minimum performance index are called optimal control
systems.
The function f often is defined based on some intermediary variables such as the
state vector x or the input u

J = f (K) =

Ztf

g(x, u)dt

where, of course, x and u depend on K. Since it is useless to look at the values

of x and u at some specific time only, the index J is formulated as an integral
over time. For theoretical considerations, it is convenient to set the final time tf to
infinity. Quadratic terms for the function g are prevalent on one hand these are
meaningful measures, and on the other hand they often yield nice analytical results.

Figure 4: Regulator problem

January 17, 2011

Roland Bchi, Markus Kottmann

Advanced Control, State Regulator

MSE

Consider the following regulator problem, see Fig. 4. The

system is x(0) = x0 . The intention is to take x towards zero as
using state feedback. If the norm of x is taken,

x1
x2



|x|2 = x1 2 + x2 2 + ... + xn 2 = x1 x2 ... xn ..

.
xn
and integrated over time we get the performance index J =

Rtf

initial state of the

fast as possible by

= xT x

xT x dt. In the following

derivation the time horizon is set to infinity, and a more general weighting scheme
for the state variables is used with a symmetric, positive semidefinite matrix Q:
J=

(3)

xT Qx dt

Note that the general form of the performance index incorporates a term containing u to weigh the control energy. This will be discussed in section 3.1.
To obtain the minimum value of J, we follow [1] and postulate the existence of
an exact differential with a constant symmetric matrix P such that
d T
(x P x) = xT Qx
dt
where P is to be determined. Applying the product rule for differentiation,
d T
(x P x) = x T P x + xT |{z}
P x + xT P x
dt
0

defining

H = A BK

(4)

d T
(x P x) = (Hx)T P x + xT P (Hx) = xT H T P x + xT P Hx
dt
= xT (H T P + P H) x = xT Qx
|
{z
}

(5)

and substituting x = Hx, we get

which is the exact differential we are seeking. Substituting that expression in (3)
results in the performance index
J=

Z
0


d T
(x P x)dt = xT P x0 = (0 xT (0)P x(0)) = xT (0)P x(0)
dt
Roland Bchi, Markus Kottmann

January 17, 2011

MSE

Advanced Control, State Regulator

In the evaluation of the limit at t = , we have assumed that the system is stable
and hence x() = 0, as desired. Therefore to minimize the performance index J,
we consider the two equations:
J=

(xT Qx)dt = xT (0)P x(0) and (H T P + P H) = Q

The design steps are then as follows:

1. Determine the matrix P , where H and Q are assumed to be known.
2. Minimize J = xT (0)P x(0) by adjusting one ore more unspecified system parameters.
3.1

Linear Quadratic Regulator (LQR)

In a more general case, we also consider the control energy of u. A method suitable
for computer calculation is stated without proof in the following. Consider the
uncompensated MIMO system
x = Ax + Bu
with feedback
u = Kx.

The performance index is

J=

(xT Qx + uT Ru)dt

(6)

With Q and R both positive definite, it can be shown that (6) is minimized with
K = R1 B T P

(7)

To calculate the symmetric n n matrix P , an algebraic matrix Riccati (ARE)

equation must be solved:
AT P + P A P BR1 B T P = Q.
Since the ARE is a quadratic equation, it has multiple solutions. Equation (6) demands for the (only) solution P which is positive definite. The resulting optimal
controller is called the linear quadratic regulator (MATLAB command: lqr).
The condition about Q can be relaxed: if Q = C T C 0 is positive semidefinite
is observable, then P and K can be calculated in the same way. These
and if (A, C)
In
conditions guarantee that x is observable through the virtual output y = Cx.
case that Q 0 (instead of Q > 0) equation (6) delivers at least a solution P 0.
January 17, 2011

Roland Bchi, Markus Kottmann

Advanced Control, State Regulator

3.2

MSE

Robustness of the Linear Quadratic Regulator

If the control loop is opened at the input u of the plant (see Fig. 4) then the openloop transfer function is given by G0 (s) = K(sI A)1 B. It can be shown, that
G0 (j) has at least a phase margin of 60 and a gain margin of [0.5 . . . [ on each
channel. This makes the LQ-regulator a considerably robust controller.

State Regulator Including Integral Part

In this section we discuss the problem of designing a compensator that provides an

asymptotic tracking of a constant reference input r(t) = r0 with zero steady-state
error. From classical control it is known that this can be achieved if the open-loop
is at least of type 1. Fig. 5 illustrates a possible structure of such a controller
including an open integrator.

Roland Bchi, Markus Kottmann

January 17, 2011

MSE

Advanced Control, State Regulator

Figure 5: Block diagram of a regulator with K1 containing an integral part

If K1 and K2 are regarded as cascaded controllers then their design can be
performed in two steps:
A faster inner loop is designed by determining K2 , e.g. by means of LQR or
pole placement.
K1 , e.g. a PI-controller, is then tuned using classical methods, resulting in a
slower outer loop.
Obviously, in this method the optimality of the inner loop may be destroyed
by K1 . To overcome this disadvantage a monolithic controller design is presented,
which delivers K1 and K2 at once. The idea is formalized here by introducing an
internal model of the reference input in the compensator. In Fig. 5 the tracking
error e is defined as
(note the sign!)

e = y r,
and its time derivative yields
0

z}|{
e = y r = y = C x

Defining two intermediate variables, z and w, as

z = x and w = u,

a new system results:

x

z }| { 
z }| { 
z }| { z }| { u

e
0 C
e
0 z}|{
=
+
w
z
0 A
z
B

(8)

B)
is controllable, then some feedback gain K
which stabilizes (8)
If the system (A,
can be designed e.g. by pole placement or using the LQR method
x.
u = K
January 17, 2011

Roland Bchi, Markus Kottmann

Advanced Control, State Regulator

MSE

Since the error e is a state variable it converges towards zero, i.e. the requirement
of an asymptotic tracking with zero steady-state error is fulfilled. Splitting the
feedback term yields
 
x = [K1 K2 ] e = K1 e K2 z
u = w = K
z
and by integration the final control law
u(t) = K1

Zt

e( )d K2 x(t)

which is illustrated in Fig. 5.

10

Roland Bchi, Markus Kottmann

January 17, 2011

Bibliography
[1] Richard C. Dorf and Robert H. Bishop. Modern Control Systems. Pearson, 2008.
[2] William S. Levine, editor. The Control Handbook. The Electrical Engineering
Handbook Series. CRC Press/IEEE Press, 1996.

January 17, 2011

Roland Bchi, Markus Kottmann

11

Advanced Control, State Regulator

MSE

Solved Exercises
1

Pole Placement

Given is a system in state space description







0 1
0
, C = c1 0
A=
, B=
0 a
Ks

, D=0

a = 4 , Ks = 8 , c1 = 6.4

For each of the following cases, find a controller u = k x + kvf r such that the
closed loop system has a unitary static gain and the poles p1 and p2 :
1. p1 = p2 = 15
2. p1,2 = 15 15j
3. p1 = p2 = 50

12

Roland Bchi, Markus Kottmann

January 17, 2011

MSE

Advanced Control, State Regulator

Optimal Control (1)

Consider the control system shown below. The state vector is x = [x1
open-loop system is unstable, therefore state feedback is introduced.

x2 ]T . The

Figure 6: Block diagram of a LQ regulator

The performance index is
J=

(xT Qx)dt.

Additional conditions are:

Q=I

and k1 = k2 = k

and

|u| 10 at initial conditions

x(0) =

1
0

1. Find the plants matrices A, B, C and D.

2. Find the matrices H and P .
3. Find k by minimizing the performance index J.

January 17, 2011

Roland Bchi, Markus Kottmann

13

Advanced Control, State Regulator

MSE

Optimal Control (2)

Given is a scalar system

x = ax + bu
with a simplified algebraic Riccati equation: 2ap 1r b2 p2 + q = 0
1. Find a controller u = kx by minimizing the performance index J
J=

(qx2 + ru2 )dt , (q > 0, r > 0)

2. Find a controller u = kx, which stabilizes the system using minimal control
energy (q = 0).

14

Roland Bchi, Markus Kottmann

January 17, 2011

MSE

Advanced Control, State Regulator

LQ-Regulator of an Inverted Pendulum

Consider an inverted pendulum, where the input signal u(t) is a force, applied to
the cart, and the output signal y(t) is the position of the cart. The state vector
T . The parameters are given by M = 1kg,
[x1 , x2 , x3 , x4 ]T is assumed to be [y, y,
, ]
2
m = 0.1kg, g = 10m/s , l = 1m.

Figure 7: inverted pendulum

Thus,

0
0
A=
0
0

the following system description can be found:

1
0
0
0
1


0
0 mg
M
, B= M , C= 1 0 0 0
0
0
0
1
(M +m)g
M1 l
0
0
Ml

, D=0

1. Compute some optimal control regulators using Matlab command lqr and
plot step responses of the closed-loop systems at different weights of control
energy r. Find also suitable prefilters Kvf .

2. For the same weights r of the control energy, plot the Bode diagram of the
closed-loop systems and discuss the results.
3. Discuss the robustness of the system.

January 17, 2011

Roland Bchi, Markus Kottmann

15

Advanced Control, State Regulator

MSE

Solutions
1

Solution Pole Placement

From equation (2) we get

det



det (I A + BK) = ( p1 )( p2 )
 

0
0 1
0
0

+
=
0
0 a
K s k 1 Ks k 2



1
det
=
K s k 1 + a + Ks k 2


( + a + Ks k2 ) + Ks k1 =
2 + (a + Ks k2 ) + Ks k1 = 2 (p1 + p2 ) + p1 p2
m
p1 p 2
p 1 p 2 = Ks k 1 k1 =
Ks
p1 + p 2 + a
(p1 + p2 ) = a + Ks k2 k2 =
Ks

The prefilter kvf is calculated according equation (1)

1
kvf = C (A + BK))1 B + D



1
a + Ks k 2 1
 

 Ks k1 0
0

= c1 0
Ks
Ks k1
=
=

c1 Ks
Ks k1

k1
c1

1

1.
(15)(15)
= 28.125
8
15 15 + 4
= 3.25
k2 =
8
28.125
kvf =
= 4.39453125
6.4
k1 =

16

Roland Bchi, Markus Kottmann

January 17, 2011

MSE

Advanced Control, State Regulator

2.
(15 + 15j)(15 15j)
= 56.25
8
15 + 15j 15 15j + 4
k2 =
= 3.25
8
56.25
= 8.7890625
kvf =
6.4
k1 =

3.
(50)(50)
= 312.5
8
50 50 + 4
k2 =
= 12
8
312.5
= 48.828125
kvf =
6.4
k1 =

January 17, 2011

Roland Bchi, Markus Kottmann

17

Advanced Control, State Regulator

MSE

Solution Optimal Control (1)

1. Read out the equations directly from the block diagram or write down the
transfer function G(s) = s12 and use the controllable canonical form.
x 1 = x2
x 2 = u

  
   
x 1
0 1 x1
0
=
+
u
0 0 x2
1
x 2
 
 

 x1
+ 0 u
y= 1 0
x2
2. The solution can be found using equations (4) and (5):
H = A BK

  

0 1
0 
k k
=

0 0
1


0
1
=
k k
H T P + P H = Q
It follows from P = P T that p12 = p21 :




 




p11 p12
p11 p12
0
1
1 0
+
=
0 1
p12 p22
p12 p22 k k
 




kp12 p11 kp12
1 0
kp12
kp22
+
=
0 1
p11 kp12 p12 kp22
kp22 p12 kp22
 


1 0
2kp12
p11 + kp12 + kp22
=
0 1
p11 + kp12 + kp22
2p12 + 2kp22
0 k
1 k

18

1
2k
k+1
=
2k 2
2k + 1
=
2k

2kp12 = 1

p12 =

2p12 + 2kp22 = 1

p22

p11 + kp12 + kp22 = 0

p11

Roland Bchi, Markus Kottmann

January 17, 2011

MSE

Advanced Control, State Regulator

3.
J(k) = xT (0)P x(0)
 


 p11 p12 1
= 1 0
p12 p22 0
2k + 1
1
= p11 =
=1+
2k
2k
Because P has to be positive definite (P > 0), k has to be positive (k > 0).
You can verify this condition by checking the eigenvalues of P which have to
be positive.
J(k) is monotonically decreasing for k > 0. The optimum is achieved for the
highest possible k. Because we have the additional condition |u| 10, k is
bounded by
 

 1
= k 10
u(0) = Kx(0) = k k
0
kopt = 10

January 17, 2011

Roland Bchi, Markus Kottmann

19

Advanced Control, State Regulator

MSE

Solution Optimal Control (2)

1. Solving the Riccati Equation


1 2 2
b p + 2ap + q = 0
r

results in the two solutions

q
r


2a 4a2 + 4 1r b2 q
r
q
p=
= 2 a a2 + b 2
b
r
2 1r b2

We are only interested in the positive solution. Because q 0 and r > 0

r
q
2q
b 0

a a2 + b2
r
r

The positive solution is

r
p= 2
b

a+

a2

b2

q
r

The optimal feedback gain k according equation (7) is

r


1
q
1
2
2
k = r bp =
a+ a +b
b
r
2. If q 0, the Riccati Equation becomes


1 2 2
1 2
b p + 2ap = b p + 2a p = 0
r
r
with the two solutions

2ar
p2 = 0
b2
it follows that the solution p of the Riccati Equation is

0 if a 0
p=
2ar
if a > 0
b2
p1 =

Note that the solution p is positive only if r is positive, too.

The feedback gain k = r1 bp is then

0 if a 0
k=
2a
if a > 0
b

The interpretation of this result is: if the system is stable (a 0), then we
need no control input to stabilize the system (the system operates in open
loop). For an unstable system, the feedback k moves the unstable pole a > 0
to
2a
a bk = a b = a
b
20

Roland Bchi, Markus Kottmann

January 17, 2011

MSE

Advanced Control, State Regulator

Solution LQ-Regulator of an Inverted Pendulum

1.
Listing 1: Calculate LQ-Regulators and plot step responses
g
l
M
m

=
=
=
=

10;
1;
1;
0.1;

A = [0 1 0 0;
0 0 -m * g / M 0;
0 0 0 1;
0 0 ( M + m ) * g /( M * l ) 0] ;
B = [0; 1/ M ; 0; -1/( M * l ) ] ;
C = [1 0 0 0];
Q = eye ( size ( A ) ) ;
R = logspace ( -2 ,2 ,5) ;
L = cell (1 , length ( R ) ) ;
G = cell (1 , length ( R ) ) ;
for i = 1: length ( R )
K = lqr (A ,B ,Q , R ( i ) ) ;
kvf = 1/( C /( - A + B * K ) * B ) ;
G { i } = ss (A - B *K ,B ,C ,[]) * kvf ;
L { i } = ss (A ,B ,K ,[]) ;
end
figure (1) ;
step ( G {:}) ;
legends = cell (1 , length ( R ) ) ;
for i = 1: length ( R )
legends { i } = [ R = num2str ( R ( i ) , % g ) ];
end
legend ( legends , Location , East ) ;

Refer to figure 8 for the step responses.

2.
Listing 2: Create Bode diagrams
figure (2) ;
bode ( G {:}) ;
grid on ;
legend ( legends , Location , East ) ;

Decreasing the weight r for the input u leads to a closed loop system with a
faster response. This can be seen in the step responses in figure 8 and in the
increased bandwidth in the Bode diagrams in figure 9.
January 17, 2011

Roland Bchi, Markus Kottmann

21

Advanced Control, State Regulator

MSE
Step Response

1.2

Amplitude

0.8

0.6

R = 0.01
R = 0.1
R=1
R = 10
R = 100

0.4

0.2

-0.2

10

15

20

25

30

35

Time (sec)

Figure 8: Step responses

3. The robustness of a system can be determined in the nyquist plot of the open
loop system. One can verify that none of the plots in figure 10 enter the unit
circle around 1. This means that all loops have a phase margin m > 60
and a gain margin 12 < Gm < .
Listing 3: Create Nyquist plots
figure (3) ;
phi = (0:100) *2* pi /100;
nyquist ( L {:}) ;
hold on ;
plot ( cos ( phi ) -1 , sin ( phi ) , k ) ;
hold off ;
axis ([ -6 10 -6 6]) ;
axis equal ;
legend ( legends , Location , East ) ;

22

Roland Bchi, Markus Kottmann

January 17, 2011

MSE

Advanced Control, State Regulator

Bode Diagram

Magnitude (dB)

-50

-100

-150
360

Phase (deg)

270
R = 0.01
R = 0.1
R=1
R = 10
R = 100

180

90

0
-2

-1

10

10

10
10
Frequency (rad/sec)

10

10

Figure 9: Bode diagrams

Nyquist Diagram
6

Imaginary Axis

R = 0.01
R = 0.1
R=1
R = 10
R = 100

-2

-4

-6
-6

-4

-2

2
Real Axis

10

Figure 10: Nyquist plots

January 17, 2011

Roland Bchi, Markus Kottmann

23