Chapter 02

Chapter 02
1 Optimization with equality constraints

Optimization with equality constraints refers to the process of …nding the max-
imum or minimum value of a function subject to certain constraints, where
these constraints are expressed as equalities. This type of optimization problem
is commonly encountered in various …elds such as engineering, economics, and
physics.
Mathematically, an optimization problem with equality constraints can be
formulated as follows:
Minimize or maximize:
Optimization with equality constraints refers to the process of …nding the
maximum or minimum value of a function subject to certain constraints, where
these constraints are expressed as equalities.
Mathematically, an optimization problem with equality constraints can be
formulated as follows:
Minimize or maximize: f (x)
Subject to equality constraints: gi (x) = 0; i = 1:::m

Where
f (x) is the objective function or cost function to be optimized.
x is a vector of decision variables.
gi (x) are the equality constraint functions.
m is the number of equality constraints.
- Other way:
8
< min f (x)
s:t
:
= fx 2 Rn gi (x) = 0; i = 1:::mg
Let’s consider a simple examples of optimization with equality constraints:
8
< min f (x; y) = min xy
s:t
: 2
8 = (x; y) 2 R g(x; y) = x2 + y 2 1 = 0;
< min f (x; y) = min x2 + y 2
s:t
: 2
8 = (x; y) 2 R g(x; y) = x + y 1 = 0
< min f (x; y; z) = min x + y + z
s:t
: 3
8 = (x; y; z) 2 R g(x; y) = x2 + y 2 + z 2 1 = 0;
< max f (x; y) = max xy
s:t
:
= (x; y) 2 R2 g(x; y) = x + y 6 = 0;
1
1.1 Constraint set:
Constraint quali…cation (CQ) is a fundamental concept in mathematical opti-
mization, particularly in the context of constrained optimization problems. It
ensures that certain conditions are satis…ed at a feasible solution, which is essen-
tial for the validity of optimality conditions and the convergence of optimization
algorithms.
Example 1 Lets consider the following constraints
= fx 2 Rn gi (x) = 0; i = 1:::mg
The feasible region or the constraints set called Constraint quali…cation with
equality constraints.
1.2 Local Maximum:

A point x 2 is said to be a point of local maximum of f subject to the
constraints g(x) = 0; if there exists an open ball around x ; B" (x ); such that
f (x ) f (x) for all x 2 B" (x ) \ :
1.3 Global Maximum:

A point x 2 is said to be a point of global maximum of f subject to the
constraints g(x) = 0; if f (x ) f (x) for all x 2 :
Remark 2 Local minimum and global minimum can be de…ned similarly by just
reverting the inequalities.
1.4 The Constraint Quali…cation:

The condition
rg(x) 6= 0
is known as the constraint quali…cation.
Notation 3 It is important to check the constraint quali…cation before applying

the theorem of resolution.
1.5 Active and Inactive Constraints

An optimal solution that lies at the intersection point of two constraints causes
both of those constraints to be considered active.
At the stationary point
x = x , some of the constraints gi (x ) = 0. These constraints are called
active constraints.
or
The ith constraint is said to be active (at a solution y) if gi (y) = 0
2
1.6 The method of Lagrange multipliers
In optimization, the method of Lagrange multipliers is a powerful technique
used to solve constrained optimization problems with equality constraints, which
can be represented by functions of the form g(x) = 0:
Consider an optimization problem with an objective function f (x) subject
to equality constraints g(x) = 0:
Minimize f (x)subject to g(x) = 0
The Lagrangian function
L(x; ) is de…ned as:
L(x; ) = f (x) + g(x)

where
is a Lagrange multiplier associated with the constraint
g(x) = 0:
The critical points of
L(x; ) are found by taking partial derivatives with respect to each variable
xi and , and setting them equal to zero:
@L
@xi =0
@L
@ =0
Solving these equations simultaneously gives the critical points (x ; )
These critical points correspond to The solutions of the constrained opti-
mization problem.
However, not all critical points are valid solutions. The critical points must
satisfy the original equality constraint g(x ) = 0 and, in some cases, second-
order conditions may need to be checked for optimality.
In summary, the method of Lagrange multipliers extends optimization to
problems with equality constraints by introducing Lagrange multipliers, which
allow us to …nd critical points that satisfy both the objective function and the
equality constraints.
Problem 4 Minimize f (x; y) = x2 + 2y 2 under the constraint
g(x; y) = x + y 2 1:
Problem 5 Find the shortest distance from the origin to the curve
x6 + 2y 2 = 4:
Problem 6 Which cylindrical soda cans of height h and radius r has minimal
surface for …xed volume?
Problem 7 Find the extrema of f (x; y; z) = z on the sphere
g(x; y; z) = x2 + y 2 + z 2 = 1:
3
Problem 8 Find the dimensions of the box with the largest volume if the total
surface area is 64 cm2
Remark 9 The Lagrange Multiplier Theorem is a fundamental result in math-

ematical optimization that provides necessary conditions for constrained opti-
mization problems. It helps identify critical points where the objective function
is optimized subject to equality constraints.
The method of Lagrange multipliers is a technique in mathematics to …nd

the local maxima or minima of a function f (x) subject to constraints g(x) = 0
.
The theorem can be stated as follows:
Theorem 10 ( Lagrange Multiplier Theorem):

Under the conditions of the establishment of the problem where the con-
straints are quali…ed, if in addition f 2 C 1 ( ),and the function f to have an
extremum relative conditioned at the point ,x , then, there exist m real numbers
such that
rL(x ) = 0:
where,
M
X
L(x) = f (x) + i gi (x)
i=0
( 1 ; :::; m) are called Lagrange multipliers.
Proof. For n = 2, we have f (x; y) 2 C 1 ( ); (x ; y ) extremum subject to

g(x; y) = 0; g(x; y) 2 C 1 and rg(x ; y ) =
6 0; then 9 st
rx;y L(x ; y ) = rf (x ; y ) + rg (x ; y ) = 0
g(x; y) = 0
@g
- rg(x ; y ) 6= 0;assume that @y (x ; y ) 6= 0, also g(x ; y ) = 0 and g 2 C 1
By the implicit function theorem there is a function y = y(x) such that
g(x; y(x)) = 0 and furthermore
gx
y 0 (x) =
gy
- At (x ; y ) the function f (x; y) has a local extremum

) f (x; y(x)) has a local extremum
fx + fy y 0 (x) = 0
y 0 (x) = ggxy
) at (x ; y )
gx
fx fy =0
gy
4
Denote
fy
=
gy
So, we have
fy + gy = 0
fx + gx = 0
Then
rx;y L(x ; y ) = rf (x ; y ) + rg (x ; y ) = 0
Corollary 11 For n = 3 and m = 2; we have f (x; y; z) 2 C 1 ( ); (x ; y ; z )

extremum subject to g(x; y; z) = 0 and h(x; y; z) = 0;
h(x; y; z) 2 C 1 ; g(x; y; z) 2 C 1 and rh(x; y; z) and rg(x; y; z) are linearly
independant at (x ; y ; z ).
Then, 9 and st
8
< rx;y;z L(x ; y ; z ) = rf (x ; y ; z ) + rg (x ; y ; z ) + rh(x ; y ; z ) = 0
g(x; y; z) = 0
:
h(x; y; z) = 0
Example 12 Lets the following problem

8
>
> opt f (x; y; z) = opt x + y + z
<
s:t
2
>
> g(x; y; z) = x + y2 4 = 0
:
h(x; y; z) = x + z 2 = 0
0 1 0 1
2x 1
We have f; g; h 2 C 1 and rg(x; y; z) = @ 2y A ; rh(x; y; z) = @ 0 A
0 1
rh(x; y; z) and rg(x; y; z) are linearly independant unless x = y = 0
but x = y = 0 2= with is a feasible set
Then, 9 and st 8
>
> fx + gx + gx = 0
>
>
< fy + gy + gy = 0
fz + gz + gz = 0
>
>
>
> g(x; y; z) = 0
:
h(x; y; z) = 0
) 8 8
>
> 1+2 x+ =0 >
> x=0
>
> >
>
< 1+2 y =0 < y = 21
1+ =0 ) = 1
>
> 2 2 >
> 2
>
> g(x; y; z) = x + y 4 = 0 >
> x + y2 = 4
: :
h(x; y; z) = x + z 2 = 0 x+z =2
5
Then 8
>
> x=0
>
>
< y = 21
= 1
>
>
>
> y= 2
:
z=2
So, we have two critical points
1
(x1 ; y1 ; z1 ; 1; 1) = (0; 2; 2; ; 1)
4
and
1
(x2 ; y2 ; z2 ; 2; 2) = (0; 2; 2; ; 1)
4
g(x,y,z) in green and h(x,y,z) in red
Finally, f (x1 ; y1 ; z1 ) = 4 and f (x2 ; y2 ; z2 ) = 0

The intersection between g = 0 and h = 0 is the intersection between
cyllinder and plane and the result is closed and bounded.
- The functin f 2 C 1 ( ) such that is a compact set, hence by Weiestrasse
f has a min and max subject to this constraint
max f (x; y; z) = f (x1 ; y1 ; z1 ) = 4

and
min f (x; y; z) = f (x2 ; y2 ; z2 ) = 0

Chapter 02

Uploaded by

Copyright:

Available Formats

Chapter 02

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 02

Uploaded by

Copyright:

Available Formats

Chapter 02

1 Optimization with equality constraints

Subject to equality constraints: gi (x) = 0; i = 1:::m

Example 1 Lets consider the following constraints

1.2 Local Maximum:

1.3 Global Maximum:

1.4 The Constraint Quali…cation:

Notation 3 It is important to check the constraint quali…cation before applying

1.5 Active and Inactive Constraints

L(x; ) = f (x) + g(x)

Problem 4 Minimize f (x; y) = x2 + 2y 2 under the constraint

Problem 7 Find the extrema of f (x; y; z) = z on the sphere

Remark 9 The Lagrange Multiplier Theorem is a fundamental result in math-

The method of Lagrange multipliers is a technique in mathematics to …nd

Theorem 10 ( Lagrange Multiplier Theorem):

( 1 ; :::; m) are called Lagrange multipliers.

Proof. For n = 2, we have f (x; y) 2 C 1 ( ); (x ; y ) extremum subject to

- At (x ; y ) the function f (x; y) has a local extremum

Corollary 11 For n = 3 and m = 2; we have f (x; y; z) 2 C 1 ( ); (x ; y ; z )

Example 12 Lets the following problem

g(x,y,z) in green and h(x,y,z) in red

Finally, f (x1 ; y1 ; z1 ) = 4 and f (x2 ; y2 ; z2 ) = 0

max f (x; y; z) = f (x1 ; y1 ; z1 ) = 4

You might also like