Linear Programming Mathematics Theory and Algorithms

Linear Programming:
Mathematics, Theory and Algorithms

Applied Optimization
Volume 2
The titles published in this series are listed at the end of this volume.
Linear Programming:
Mathematics, Theory
and Algorithms
by
Michael 1. Panik
University ofHartford
KLUWER ACADEMIC PUBLISHERS

DORDRECHT I BOSTON I LONDON
Library of Congress Cataloging-in-Publication Data
Panik. Michael J.
Linear programming mathematics. theory and algorithms / by
Michael J. Panik.
p. cm. -- (Applied optimization; vol. 2)
Inc I udes bib Ii ograph i ca I references and index.
ISBN-13: 978-1-4613-3436-1 e-ISBN-13: 978-4613-3434-7

001: 10.1007/978-4613-3434-7
1. Linear programming. I. Title. II. Series.

T57.74.P345 1996
519.7·2--dc20 95-41545
Published by Kluwer Academic Publishers,

P.O. Box 17,3300 AA Dordrecht, The Netherlands.
Kluwer Academic Publishers incorporates

the publishing programmes of
D. Reidel, Martinus Nijhoff, Dr W. Junk and MTP Press.
Sold and distributed in the U.S.A. and Canada

by Kluwer Academic Publishers,
101 Philip Drive, Norwell. MA 02061, U.S.A.
In all other countries, sold and distributed

by Kluwer Academic Publishers Group,
P.O. Box 322, 3300 AH Dordrecht, The Netherlands.
Printed on acid-free paper
All Rights Reserved

© 1996 Kluwer Academic Publishers
Softcover reprint of the hardcover 1st edition 1996
No part of the material protected by this copyright notice may be reproduced or
utilized in any form or by any means, electronic or mechanical,
including photocopying, recording or by any information storage and
retrieval system, without written permission from the copyright owner.
In Memory of
Alice E. Bourneuf
and
Ann F. Friedlaender
TABLE OF CONTENTS
Chapter 1. INTRODUCTION AND OVERVIEW 1
Chapter 2. PRELIMINARY MATHEMATICS 9

2.1 Vectors in R n 9
2.2 Rank and Linear Transformations 13
2.3 The Solution Set of a System of Simultaneous Linear
Equations 18
2.4 Orthogonal Projections and Least Squares Solutions 23
2.5 Point-Set Theory: Topological Properties of R n 26
2.6 Hyperplanes and Half-Planes (-Spaces) 28
2.7 Convex Sets 31
2.8 Existence of Separating and Supporting Hyperplanes 34
2.9 Convex Cones 39
2.10 Basic Solut.ions to Lineal' Equalities 45
2.11 Faces of Polyhedral Convex Sets: Extreme Points,
Facets, and Edges 53
2.12 Extreme Point Representation for Polyhedral Convex
Sets 56
2.13 Directions for Polyhedral Convex Sets 60
2.14 Combined Extreme Point and Extreme Direction
Representation for Polyhedral Convex Sets 62
2.15 Resolution of Convex Polyhedra 63
2.16 Simplexes 64
2.18 Linear Fundionals 66
Chapter 3. INTRODUCTION TO UNEAR PROGRAMMING 69

3.1 The Canonical Form of a Linear Programming
Problem 69
3.2 A Graphical Solution to the Lineal' Programming
Problem 70
3.3 The Standard Form of a Linear Programming
Problem 72
3.4 Properties of the Feasible Region 75
3.5 Existence and Location of Finite Optimal Solutions 76
3.6 Basic Feasible and Extreme Point Solutions to the
Linear Programming Problem 81
3.7 Solutions and Requirements Spaces 85
viii
Chapter 4. DUALITY THEORY 89

4.1 The Symmetric Dual 89
4.2 Unsymmetric Duals 92
4.3 Duality Theorems 95
Chapter 5. THE THEORY OF LINEAR PROGRAMMING 111

5.1 Finding Primal Basic Feasible Solutions 111
5.2 The Reduced Primal Problem 114
5.3 The Primal Optimality Criterion 115
5.4 Constructing the Dual Solution 116
5.5 The Primal Simplex Method 125
5.6 Degenerate Basic Feasible Solutions 131
5.7 Unbounded Solutions Reexamined 135
5.8 Multiple Optimal Solutions 137
Chapter 6. DUALITY THEORY REVISITED 141

6.1 The Geometry of Duality and Optimality 141
6.2 Lagrangian Saddle Points and Primal Optimality 160
Chapter 7. COMPUTATIONAL ASPECTS OF LINEAR PROGRAMMING 169

7.1 The Primal Simplex Method Reexamined 169
7.2 Improving a Basic Feasible Solution 174
7.3 The Cases of Multiple Optimal, Unbounded, and
Degenerate Solutions 180
7.4 Summary of the Primal Simplex Method 187
7.5 Obtaining the Optimal Dual Solution From the Optimal
Primal Matrix 189
Chapter 8. ONE-PHASE, TWO-PHASE, AND COMPOSITE METHODS OF

LINEAR PROGRAMMING 195
8.1 Artificial Variables 195
8.2 The One-Phase Method 198
8.3 Inconsistency and Redundancy 203
8.4 Unbounded Solutions to the Artificial Problem 211
8.5 The Two-Phase Method 213
8.6 Obtaining the Optimal Primal Solution from the Optimal Dual
Matrix 221
8.7 The Composite Simplex Method 225
IX
Chapter 9. COMPUTATIONAL ASPECTS OF LINEAR PROGRAMMING:

SELECTED TRANSFORMATIONS 233
9.1 Minimizing the Objective Function 233
9.2 Unrestricted Variables 234
9.3 Bounded Variables 236
9.4 Interval Linear Programming 247
9.5 Absolute Value Functionals 249
Chapter 10. THE DUAL SIMPLEX, PRIMAL-DUAL, AND

COMPLEMENTARY PIVOT METHODS 251
10.1 Dual Simplex Method 252
10.2 Computational Aspects of the Dual Simplex Method 256
10.3 Dual Degeneracy 259
10.4 Summary of the Dual Simplex Method 259
10.5 Generating an Initial Primal-Optimal Basic Solution:
The Artificial Constraint Method 261
10.6 Primal-Dual Method 264
10.7 Summary of the Primal-Dual Method 274
10.8 A Robust Primal-Dual Algorithm 276
10.9 The Complementary Pivot Method 278
Chapter 11. POSTOPTIMALITY ANALYSIS I 289

11.1 Sensitivity Analysis 289
11.2 Structural Changes 314
Chapter 12. POSTOPTIMALITY ANALYSIS II 319

12.1 Parametric Analysis 319
12.2 The Primal-Dual Method Revisited 334
Chapter 13. INTERIOR POINT METHODS 341

13.1 Optimization Over a Sphere 341
13.2 An Overview of Karmarkar's Algorithm 343
13.3 The Projective Transformation T(X) 345
13.4 The Transformed Problem 348
13.5 Potential Function Improvement and Computational
Complexity 351
13.6 A Summary of Karmarkar's Algorithm 354
13.7 Transforming a General Linear Program to
Karmarkar Standard Form 355
x
13.8 Extensions and Modifications of Karmarkar's Algorithm 359

13.9 Methods Related to Karmarkar's Routine: Affine Scaling
Scaling Algorithms 368
13.10 Methods Related to Karmarkar's Routine: A Path-Following
Following Algorithm 379
13.11 Methods Related to Karmarkar's Routine: Potential
Reduction Algorithms 398
13.12 Methods Related to Karmarkar's Routine: A Homogeneous
and Self-Dual Interior-Point. Method 424
Chapter 14. INTERIOR POINT ALGORITHMS FOR SOLVING

LINEAR COMPLEMENTARITY PROBLEMS 433
14.1 Introduction 433
14.2 An Interior-Point, Path-Following Algorithm for
LC P( q, M) 436
14.3 An Interior-Point, Potential-Reduction Algorithm for
LC P( q, M) 439
14.4 A Predictor-Corrector Algorithm for Solving
LC P( q, M) 445
14.5 Large-Step Interior-Point Algorithms for Solving
LCP(q, M) 451
14.6 Related Methods for Solving LC P{ q, M) 454
Appendix A: Updating the Basis Inverse 459

Appendix B: Steepest Edge Simplex Mcthods 461
Appendix C: Derivation of the Projection Matrix 467
Refercnces 473
Notation Index 485
Index 489
ACKNOWLEDGEMENT
Anyone familiar with the area of linear programming cannot help but
marvel at its richness, elegance, and diversity. These special qualities of linear
programming are by no means accidental. Many extremely talented and
dedicated individuals helped cultivate these attributes. I am most grateful to
those researchers who pioneered as well as contributed significantly to the
development of linear programming theory and methods and whose writings have
served as a valuable source of reference and inspiration. Specifically, the classic
works of George Dantzig, David Gale, A. J. Goldman, Harold Kuhn, Albert
Tucker, and Philip Wolfe, among others, are particularly noteworthy. More
recently, my interest in the study of interior point techniques is largely due to the
efforts of researchers such as Kurt Anstreicher, Robert Freund, Donald Goldfarb,
Clovis Gonzaga, N. Karmarkar, Masakazu Kojima, Nimrod Megiddo, Shinji
Mizuno, Michael Todd, Yinyu Ye, and Akiko Yoshise. To each of these authors,
as well as to many of their contemporaries, lowe a substantial intellectual debt.
A significant portion of the material presented herein stems from lecture

notes developed for courses such as quantitative decision theory and advanced
quantitative methods which I taught for a number of years in the A.D. Barney
School of Business and Public Administration at the University of Hartford. I
am grateful to the University V.B. Coffin Grant Committee and the Barney
School Dean's Office for providing financial support.
Finally, a special note of gratitude is extended to Marilyn Baleshiski for

her accuracy, dedication, and steadfastness in typing an extremely complicated
and lengthy manuscript.
1 INTRODUCTION AND OVERVIEW
My objective in writing this book is to present to students of management

science, mathematics, engineering, and economics the salient features of the
theory and computational aspects of linear programming. The material is self-
contained in that it offers, in considerable detail, the mathematical tools needed
to study linear programming theory and to implement a set of algorithms
designed to efficiently solve linear programs. The only prerequisites are basic
calculus and some exposure to linear algebra and matrix theory.
An important feature of the presentation is that it fully integrates both the
primal and dual programs throughout the entire text. In fact, duality theory is
introduced early and used extensively as a vehicle for tying together all of the
theoretical and computational results.
While many of the computational routines offered herein are simplex based,
considerable attention has been given to the particulars of the array of interior-
point methods which have come to dominate the algorithmic literature on linear
programming since the early 1980's. This includes Karmarkar's work along with
an assortment of affine scaling, potential reduction, and path-following tech-
mques.
Linear programming is one of the most versatile and powerful mathematical
programming techniques that can be employed for efficiently solving a stylized
1
2 Linear Programming
class of decision problems of the form
max f(X) = C'X s.t. (1.1)

AX~b, X~O,
where C, XfRP,bfRm , A is a matrix of order (mxn), and a prime denotes

transposition. Here f depicts a linear objective function, AX ~ b represents a
simultaneous system of linear structural constraints, and X ~ 0 will be referred to
as the set of nonnegativity conditions. (As will be seen later on, many variations
on this structural theme are possible.) If we let 9G = {X I AX ~ b, X ~ O} represent
the region of admissible solutions (or feasible region for short), then the preceding
problem becomes one of finding a point X in 9G for which f is as large as possible.
The range of applications and the wealth of different types of problems that
can be solved using linear programming algorithms is staggering, e.g., linear
programming has been used with marked success to solve optimization problems
in areas such as economics (including banking and finance), business
administration and management, agriculture and energy, contract bidding,
nutrition (diet) planning, health care, public decision making, facility location,
transportation, strategic planning, and so on. The list is virtually endless.
In particular, the elements within X may represent output levels for a set of
p different products with the elements of C being their associated (constant) unit
profit values. The m components of b can then be thought of as fixed resource
availability levels per production run. The columns of A thus depict a set of p
production activity vectors which indicate the amount of each input going into
the operation of each production activity at the unit level. What we have just
characterized is called an optimum product mix problem. A producing unit seeks
to allocate its inputs to the production activities under a given linear technology
in order to determine the quantities of the various products to produce to
maximize profit.
Introduction and Overview 3
But this is not the whole story. Associated with the above linear program
(which we may call the primal problem) is another such program, obtained
directly from the primal, called its dual, i.e., the problem defined as dual to (1.1)
is
min g(U) = b'U s.t (1.2)
A'U ~C, U~O,
where U f Rm. While a detailed development of the relationship between these

two problems is beyond the scope of this simple introductory discussion, suffice it
to say that the interconnections between the primal and dual problems will be
emphasized frequently throughout the text; indeed, it is this unifying theme of
duality that will permeate most of our theoretical and computational results.
However, we can easily offer, even at this early stage of our exposure to
duality, a modest hint as to the importance of the dual program by considering
the dual to the production problem specified earlier. To this end we may note
that the dual variables or components of U can be thought of as representing the
internal (not market) or imputed unit prices for an organization's inputs. (Such
prices can serve to guide the internal allocation of an organization's resources.)
Hence 9 represents the total internal value of all inputs available to the
organization. The dual structural constraint system A'U ~ C then dictates that
the total internal value of all inputs going into the operation of a production
activity at the unit level must be at least as great as its per unit profit
contribution. The dual problem thus has us find the smallest imputed value of
the stock of inputs which will fully explain total profit given that each production
activity breaks even or is deemed unprofitable and consequently discontinued.
The richness of linear programming lies not only in its diversity of
applications but also in the elegance of the mathematical tools used to
characterize as well as generate solutions. In this volume the development of the
requisite mathematics along with linear programming concepts and methodology
will be emphasized.
Chapter 2 reviews the mathematical tools needed to study linear

programming. Specifically, it succinctly lays out the essentials of: vector and
matrix notation and operations; vector spaces, subspaces, and projections; point-
set theory; simultaneous systems of linear equations and inequalities; linear
transformations; rank and solution{s} of simultaneous linear systems; linear
dependence and independence; affine functionals, hyperplanes and half-planes;
convex sets, combinations and hulls; theorems of the altemative (including Farkas'
theorem); convex cones along with dual and polar cones; conical hulls, extreme
vectors and extreme solutions of homogeneous linear systems; basic solutions to
linear equalities; extreme points and directions for convex and polyhedral convex
sets; the representation of convex polyhedra using extreme points and extreme
directions; and the notion of a simplex. Proofs of many of the theorems
presented in this chapter may be found in Panik (1993).
A formal study of linear programming proper begins with Chapter 3. In
this regard, the remainder of this overview outlines the salient features of linear
programming to be presented in subsequent chapters.
Chapter 3 presents the canonical and standard forms of a linear
programming problem. It discusses the concepts of feasibility and optimality and
illustrates the same with a graphical solution to an example problem. The
properties of the feasible region 9G are discussed along with the existence and
location of finite optimal solutions. To this end an existence theorem involving a
necessary and sufficient condition for a finite optimal solution is presented under
a representation of 9G in terms of extreme points and extreme directions. Next
comes a discussion of the correspondence between basic feasible and extreme point
solutions to a linear program. This is followed by a presentation of the concepts
of solutions space and requirements space.
In Chapter 4 symmetric as well as unsymmetric dual problems are
introduced and their properties explored. This is followed by a battery of duality
theorems and corollaries. In particular, the weak duality theorem and the
sufficient optimality criterion are discussed and then followed by the so-called
fundamental theorems of linear programming, namely the strong duality theorem

(including a discussion of the duality gap), the existence theorem, and both weak
and strong complementary slackness theorems.
Chapter 5 begins with a procedure for finding basic feasible solutions to the
primal linear program. Of particular interest is how the set of basic variables as
well as the objective function can be written in terms of the set of non-basic
variables. This subsequently leads to the specification of the reduced primal
problem. Next comes the development of the primal optimality (dual feasibility)
criterion and the definition of the vector of dual variables. These two notions
then allow us to rationalize the structure of the symmetric dual problem under an
application of the primal Lagrangian function. The dual variables are now
interpreted as marginal imputed values or simplex multipliers or shadow prices
associated with the right-hand sides of primal structural constraints. Optimality
evaluators or relative (reduced) cost coefficients are also defined in terms of dual
variables. Finally, an iterative solution process called the primal simplex method,
with its attendant entry and exit criteria for generating improved solutions to a
linear program, is outlined and illustrated. This is followed by a description of
degenerate basic feasible solutions, unbounded solutions, and multiple optimal
solutions.
Looking to Chapter 6, the concept of a feasible direction along with the set
of feasible directions is introduced and defined within the region of admissible
solutions. The set of feasible directions is then related to the notions of tangent,
dual, and polar support cones. These concepts then set the stage for a discussion
of the geometry underlying the Minkowski-Farkas theorem. All of these items are
then utilized when we next develop the Karush-Kuhn-Tucker necessary
conditions for linear programs. It is further shown that the Lagrange multipliers
for the same are the dual variables in the dual linear programming problem. An
extensive discussion on the existence of dual variables as partial derivatives of the
primal optimal value function follows. Next comes a description of a saddlepoint
of the primal Lagrangian and a set of theorems pertaining to the existence of a
saddle point. This is followed by another look at the strong duality theorem and
the Kuhn-Tucker equivalence theorem which establishes the notion that the
existence of an optimal solution to the primal problem is equivalent to the
existence of a saddle point of its associated Lagrangian. The chapter ends with
another look at the duality and complementary slackness theorems.
Chapter 7 deals with the computational aspects of linear programming. It
reexamines the primal simplex method in terms of simplex matrix calculations.
In particular, we state an optimality criterion and demonstrate how to improve a
basic feasible solution through the use of pivoting; i.e., entry and exit criteria are
used to "swap" one basic feasible solution for another in an attempt to maximize
the primal objective function. We next explore the detection of multiple optimal,
unbounded, and degenerate solutions. The chapter ends with a discussion on
obtaining the optimal dual solution from the optimal primal simplex matrix.
In Chapter 8 we introduce the notion of artificial variables and show how
they may be used to initiate the simplex method. Here the one-phase (or M-
penalty method) and two-phase methods are explored in depth along with how an
inconsistency or redundancy is detected in the constraint system. The last
section offers a discussion on how one may obtain the set of optimal primal
variables from the optimal simplex matrix when the dual minimization problem
is solved using either the one- or two-phase routine.
Chapter 9 presents an assortment of specialized transformations for solving
problems involving departures from the general form: max C'X s. t AX ~ b,
X ~ O. In particular, it considers: minimizing an objective under conversion to a
maximization problem; the cases of unrestricted as well as bounded variables;
interval linear programming; and how to handle absolute value functionals.
Chapter 10 considers three important variants of the standard simplex
method, namely the dual simplex, primal-dual and complementary pivot routines.
The theoretical development of these techniques as well as their computational
features are presented. Dual simplex pivoting and a discussion of dual degeneracy
along with the artificial constraint method (for generating an initial primal-
optimal basic solution) is also offered. In addition to developing the standard

primal-dual routine (involving the extended and restricted primal problem,
restricted basis entry the restricted dual problem, and the primal-dual optimality
theorem), a robust or hybrid primal-dual method is also given. The chapter ends
with a presentation of the complementary pivot method for solving the linear
complementarity problem. Included is a discussion of variable complements,
complementary and almost complementary basic solutions, complementary cones,
and the complementary pivot rule.
The emphasis in chapter 11 is postoptimality analysis. We first consider
sensitivity analysis - which involves an assessment of the extent to which a
problem may be modified without violating the feasibility or optimality of the
original solution. Here we consider the impact on feasibility or optimality of a
discrete change in an objective function coefficient C i' a right-hand side value bj ,
or in a structural constraint coefficient aji' i=l, ... , mj j=l, ... , n. The thrust of
the analysis here is the formulation of lower and upper limits associated with the
said discrete changes. Next comes a discussion of structural changes or the
problem of adding or deleting certain variables or structural constraints within
the optimal simplex matrix and then reoptimizing.
Chapter 12 offers a second round of postoptimality analysis. In particular,
it extends the sensitivity analysis developed in the preceding chapter to what is
called parametric analysis. That is, instead of determining, say, the amount by
which an objective function coefficient may be altered in some particular fashion
before the feasibility or optimality of the current solution is violated, it generates
a sequence of basic solutions which in turn become optimal, one after the other,
as "all" of the components of C vary continuously in some prescribed direction.
In effect, parametric analysis combines sensitivity analysis with simplex pivoting.
The chapter ends with the specification of a parametric method for solving a
linear program. In particular, it includes a modification of the usual primal-dual
method by working with a special parametric version of the primal problem.
The coverage in Chapter 13 involves the presentation of a relatively new

approach to solving linear programming problems, namely interior-point
techniques which have been borrowed from the area of non-linear programming.
These methods (and in particular those found to be computationally tractable)
received considerable attention after 1984 when N. Karmarkar first published his
now famous paper on solving a linear program via a projective rescaling method.
The chapter introduces what is called Kannarkar's standard /onn of a problem
and the concept of a roundness restriction. It then presents the concept of
Karmarkar's potential function and the projective trans/onnation of the feasible
region used to center the solution. Next comes the specification of the
trans/onned or scaled linear program and what is called its restriction. This is
followed by the concept of the transfonned potential function and a discussion of
the complexity of the algorithm. A method for transforming a general linear
program to Karmarkar standard form is also offered along with a discussion of
some important extensions and modifications of Karmarkar's algorithm. All of
this is followed by a specification of methods related to Karmarkar's routine, e.g.,
algorithms that are based on primal and dual affine scaling, projective and affine
potential reduction, and path-following or 6arrier methods.
Chapter 14 extends some of the preceding interior point techniques to the
solution of the linear complementarity pro6lem L CP( q, M). In particular, path-
following, potential reduction, predictor-corrector, and large step algorithms are
considered.
Finally, a set of appendices on updating the 6asis inverse, steepest edge
simplex methods, and deriving orthogonal projection matrices used to determine
feasible step directions for interior point methods serve to round out the text.
2 PRELIMINARY MATHEMATICS
2.1. Vectors in Rn
A vector is an ordered n-tuple of elements expressed as a column (n x 1) or
row (1 x n) matrix
x = [J:] , X' = (xl"'" xnJ

respectively. The n elements of a vector are called its components and a prime
indicates transposition. In all of what follows our discussion will take place in an
n-dimensional real vector space (denoted Rn).
The Cartesian product (or product set) A x <:B of two sets A, <:B in R n is the
set of ordered pairs (a, b), where a€A, b€<:B. Then the product set
Rx Rx ... x R = R n = {X' = (x 1,x2""'x n ) fl,X 2, ... ,x n €R}.
To establish a metric or concept of distance between two points X, Y (: Rn,
let us first state that the scalar product of X, Y is the scalar
n
X· Y = X/Y = XjYj.L (2.1)
i=1
Next, a norm on R n is a function which assigns to each X(: R n some number II X II
such that:
(a) IIxll ~ 0 and Ilxll = 0 if and only if X = 0;
(b) IIX+YII ~ II XII + IIYII (the triangle inequality);
(c) IlcxlI = I clllxlI (homogeneity); and
(d) Ix·YI ~ IIXIIIIYII (the Cauchy-Schwarz inequality).
9
In this regard, the distance between points X, Y t R n induced by the norm "II-If
on R n is
1/2
IIx-YIl = [(X-Y)'(X-Y)]
1/2
= [
EIXi-yd .
n ] (2.2)
1=1
An important special case of this definition occurs when we consider the distance
between any XtRn and the origin 0, i.e., the length (magni-tude) of X or norm
of X is
IIxll = (X'X)I/2 = [tIXd]I/2. (2.3)

2=1
The direction of X ( 'I 0) is a unit vector X/II X II which points in the direction of
X.
Let X, Y t Rn with X, Y 'I O. The cosine of the angle (J between X, Y is
cos() = (X.Y)/IIXIIIIYII, 0 < () ~ 7r.
The Cauchy-Schwarz inequality implies that -1 < cos() < 1. Also, the angle
between X, Y is acute « 7r/2), right (= 7r/2), or obtuse (> 7r/2) according to
whether the scalar product X· Y is positive, zero, or negative. Conversely, the
vectors X, YtRn with X, Y'IO are said to be orthogonal (i.e., mutually
perpendicular) if X· Y = O. Since X· Y = II XIIIIYII cosO, X· Y = 0 if and only if
() = 7r /2 and thus cos() = O.
A vector XtRn is a linear combination of the vectors XjtRn, j=1, ... , m, if
there exist scalars Aj' j=1, ... , m, such that
m
X= E AjXj . (2.4)
j=1
In this light we can state that a set of vectors {X;tRn, j=1, ... , m} is linearly
dependent if there exist scalars Aj' j=1, ... , m, not all zero such that
m
L AjXj = 0, (2.5)
j=1
i.e., the null vector is a linear combination of the vectors Xj. If the only set of
scalars Aj for which (2.5) holds is Aj = 0, j=1, ... , m, then the vectors Xj are
linearly independent, i. e., the trivial combination OX! + ... + OXm is the only
Preliminary Mathematics 11
linear combination of the Xj which equals the null vector.

A set of vectors {XjtRn, j=l, ... , m} is a spanning set for R n if every
vector X t Rn can be written as a linear combination of the vectors
Xj' i.e., if XtR n , then
X
Here the set {Xj t R n , j=l, ... , m} is said to span or generate R n since every
vector in R n is (uniquely) linearly dependent on the spanning set.
A basis for R n is a linearly independent subset of vectors from R n which
spans Rn. Moreover:
(a) every basis for R n contains the same number of basis vectors; and there
are precisely n vectors in every basis for Rnj and
(b) any set of n linearly independent vectors from R n forms a basis for Rn.
The dimension of R n , dim(Rn), is the maximum number of linearly
independent vectors that span the space, i.e., it is the number of vectors in a
basis for Rn. Clearly dim(R n ) = n.
A nonempty subset .Ab of R n is termed a subspace if it is closed under the
operations of vector additional and scalar multiplication, i. e., given vectors
Xl' X2 t.Ab, their linear combination Al Xl +A 2X 2 is also a member of .Ab for
every pair of scalars AI' A2 t R. Addi tionally, if .Ab is a subspace of R n, then .Ab
must contain all linear combinations of its elements. In fact, the set e(.Ab) of all
finite linear combination of points of .Ab is called the span or linear hull of .Ab and
constitutes a subspace of Rn. Here
e(.Ab) ={X I X = t \Xj,Xi E.Ab, \ ER for all i,

i=l and k is an arbitrary positive integer}
is the space spanned or generated by.Ab. Moreover, e(.Ab) is the smallest

subspace containing.Ab. Hence.Ab is a subspace of R n if and only if .Ab = e(.Ab).
By convention, e( ¢» = {o}. A set of vectors :f is a spanning set of .Ab in R n if
.Ab = e(:f).
Let .Ab be a subspace of R". A set 'i of vectors in .Ab is a basis for .Ab
provided:
(1) 'i spans .Ab, i.e., £('i) =.Abj and
(2) 'i is linearly independent.
Given that .Ab is a subspace of R", the dimension of A, dim(.Ab), is ~ n
and equals the maximum number of linearly independent vectors that constitute
a basis for.Ab. So if dim(.Ab) = m < n, a basis for .Ab is a subset of m linearly
independent vectors selected from a set of n linearly independent vectors in R".
If .Ab l , .Ab 2 are subspaces of R", .Ab l n A 2 ={O}, and, taken together, .Ab l
and .Ab2 span R" in the sense that each vector in R" is uniquely expressible as
Z = X+ Y, where Xc Al and Y c .Ab2 ' then R" is expressible as the direct sum of
AI' ~ and written as R" = .Ab l Ef).Ab 2 = {X+YIXc AI' Y c .Ab2 }. Moreover, if
{Xl' ... ' Xr } is a basis for Al and {YI , ... , Y a } is a basis for .Ab 2 ' then the
combined set {Xl' ... ' Xr ' YI! ... , Y a} is a basis for R". In this regard,
dim(R") = n = dim(.Ab 1 Ef) .Ab2 ) = dim(AI) + dim(.Ab 2 ) = r+s. If a subspace
.Ab l is contained in a subspace A 2, then dim(Al) ~ dim(.Ab 2 ).
For .Ab a subspace of R", the orthogonal complement or dual subspace of
A, .Ab 1., is also a subspace of R" and consists of all vectors that are orthogonal
to those in A. Hence.Ab 1. = {YIY'X = 0, Xc.Ab}. Additionally, (.Ab 1. ) 1.
= .Ab.
If X is an arbitrary point in R", the orthogonal projection of X into .Ab is
the point Xc.Ab for which Ilx-xl? = (X-X)'(X-X) assumes a minimum. There
exists at most one such projection of a point X into a subspace.Ab. And if X is
the projection of X into A, then X - X is the projection of X into .Ab 1.. A and
.AI, 1. span R" (any vector Zc R" can be uniquely expressed as Z = X+ Y for
Xc.Ab, Y c.Ab 1.) so that R" = .Ab Ef).Ab 1.. In this regard, X and Yare termed
orthogonal projections of Z onto the subspaces A, .Ab 1. respectively. Clearly
dim(M1.) = n- dim(.Ab).
If the set of vectors {Xl' ... , Xm} is a basis for a subspace .Ab of R", then
.Ab 1. = {YIY'Xj = 0, i=l, ... , m, Xj c.Ab}.
2.2. Rank and Linear Transformations

Given an (m x n) matrix A, the subspace of Rm spanned by the n column
vectors of A will be called the column space of A and denoted Co/(A) while the
subspace of R n spanned by the m row vectors of A is termed the row space of A
and written Row(A). These two subspaces are distinct so that, if m 1= n, then
Co/(A) n Row(A) = ¢ and thus they have no vectors in common.
The nonzero row vectors of the echelon form of A are linearly independent
and thus constitute a basis for Row(A). If there are k nonzero vectors in the
echelon form of A, then k = dim( Row(A»). Similarly, the column vectors of A
corresponding to the pivot columns of the echelon form of A are also linearly
independent and thus form a basis for Co/(A). Here too k = dim( Col(A»).
Hence we may unambiguously define the rank of a matrix A, denoted p(A), as
dim(Row(A» = dim( Co/(A») = k.
Alternatively, we may specify the rank of a matrix A as the order of the
largest non-singular submatrix of A, i.e., it is the order of the largest
nonvanishing determinant in Aj moreover, if p(A) = k, all submatrices of order
k+ 1 are singular. Also, an nth order matrix B is said to be of full rank if
p(B) = nj and this occurs if and only if B is nonsingular (i.e., I B I 1= 0). Hence,
for an nth order matrix B, p(B) < n if and only if B is singular.
A function or transfonnation T from R n to R m is a rule which associates
with each vector X € R n a unique vector Y € R m , where Y = T(X) is called the
image of X under transformation T. Here R n is the domain of T, D(T), and the
collection of those V's which are the image of one or more X under rule T consti-
tutes the range of T, R(T). In particular, a function T: Rn--+Rm is a linear trans-
fonnation if for vectors Xl' X2 € Rn and scalars o:,,a € R, T( o:X I +,aX2 ) =
o:T(XI ) + ,aT(X2 ). Thus a transformation between R n and R m is linear if and
only if it preserves linear combinations of vectors. Also:
(1) T(O) = OJ
(2) T( -X) = -T(X)j
(3) X€R n .- To(X) = 0 (the zero transformation);

(4) X€R n .- TI(X) = X (the identity transformation).
The linear transformation T from R n to R m is termed one-to-one provided
that no two different vectors in R n have the same image in Rm, i.e., T(X I ) =
T(X 2) implies Xl = X2 • T is onto if every vector in R m is the image of at least
one vector in Rn. That is, for each Y€R m there exists an X€ Rnsuch that
Y = T(X). Hence T is onto if R(T) = Rm. To underscore the notions of onto
and one-to-one linear transformations we state
2.1. THEOREM. For T a linear transformation from R n to R m:
(a) if m > n, then T is not onto;
(b) if m < n, then T is not one-to-one; and
(c) if m = n, then T is one-to-one if and only if it is onto.
By virtue of part (c) of this theorem, if T: Rn_Rn is both one-to-one and onto,
then it is invertible, i.e., there exists a linear transformation U: Rn_Rn such that
TU = UT = T I , the identity transformation. HereUisuniqueandisdenoted as T- 1•
The linear transformation T from R n to R m is said to be non-singular if
T(X) = 0 implies X = O. Clearly T is one-to-one if and only if T is non-singular
(since T(X C X 2) = T(X 1 ) - T(X 2) = 0 implies T(X t ) = T(X2) and conversely).
For T: Rn_Rn, the following are equivalent:
(a) T is non-singular;
(b) T is onto;
(c) T is in verti ble;
(d) T is one-to-one.
In what follows we shall typically use an (m x n) matrix A to define a linear
transformation from R n to Rm. Specifically,
2.2. THEOREM. Let T A:Rn_Rm be a linear transformation. Then there
exists a unique (m x n) matrix A such that T A(X) = AX, X € Rn.
Here the aforementioned rule involves (left) multiplication of X by A and is given
by X € R n .- T A(X) = AX = Y € Rm. Clearly T A respects linear combinations of
vectors. Moreover, it is easily shown that a linear transformation is completely
determined under its action on a set of unit column vectors {el , ..., en} or
standard 6asis for Rn. That is, if the /h column of A is the image T A(ej) = Aej
=aj of the /h unit column vector ej, then A= [TA(el), ... ,TA(en)] and thus
TA(X) = Ej=lxjTA(ej) = [TA(el),···,TA(en)]X=AX.
We next consider two fundamental spaces associated with a linear
transformation. First, given T A:Rn-+Rm, the set of all possible values of T A or
the set of vectors Y f R m as X varies over R n is called the range of T A and
denoted R(A) (or R(T A)) = {Y!Y = T A(X) = AX, XfRn}. Here R(A) is the
subspace of R m spanned by the columns of A and thus constitutes the set of
linear combinations of the columns of A or Y = E'J=IXjaj. Clearly R(A) =
Col(A) = e{aj};=l.
If R(A) is finite-dimensional, then the number of vectors in a basis for R(A)
is the dimension of R(A) and is called the rank of T A and denoted p(T A). Here
p(T A) = dim( Col(A)) = dim( R(A)) = p(A), i.e., the rank of a transformation
T A between R n and R m equals the rank of its matrix. For T A:Rn-+Rn, T A is
invertible if and only if A is invertible, i.e., if and only if p(A) = n.
If To: Rn-+Rm represents the zero transformation, then R(T0) is the zero
su6space and p(T0) = O. For the identity transformation T (Rn-+Rm,
R(T/) = R n and thus p(TI) = dimRn = n.
The second fundamental space hinted at above is a subspace of R n called
the null-space (or kerne~ of T A' N(T A) (or simply N(A)), which corresponds to
the set of all XfRn mapped by T A to the null vector, i.e., N(A) =
{X! T A(X) = AX = 0, Xf Rn} is the solution space of the homogeneous linear
equation system AX = o. Given that N(A) is a subspace, aXf N(A) when
Xf N(A), a f R. If X i= 0, then clearly N(A) contains infinitely many non-zero
vectors. The number of vectors in a basis for N(A) or the dimension of N(A)
will be called the nullity of A and denoted n(A). A linear transformation with
nullity zero is said to be of full rank.
The concepts of rank and nullity of a linear transformation T A:Rn-+Rm
enable us to determine whether or not T A is onto or one-to-one. That is:
(a) T A is one-to-one if n(A) = 0;

(b) T A is onto if p(A) = m.
As Figure 2.1 reveals, if N(T A) shrinks to {O}, then T A is one-to-one; and if
R(T A) = R m , then T A is onto.
Rn - - - - - - + 1 Rm
Figure 2.1
We noted earlier that for an (m x n) matrix A, the subspace spanned by the

columns of A is the range of A, R(A) = {YIY = AX, XfRn}. Also, the null-
space of A was defined as N(A)={XIAX=O, XfRn}. Let R(A).l =
{UIU'Y=O, UfRm, YfR(A)} = {UIU'AX=O, UfRm,XfRn} and N(A')=
{Y I A'y = 0, Y f Rm}. Then it can be shown that R(A).l = N(A'). Similarly,
N(A).l = R(A').
For the zero transformation To:Rn-+Rm, N{To) = R n and thus n{To) = n.
If T I: Rn-+Rm depicts the identity transformation, then N{T I) = {O} and
n{TI) =0.
Next,
2.3. THEOREM. Let T A be a linear transformation defined by
the (m x n) matrix A. Then {a1 , .•. , an} is linearly depen-dent if
and only if A has a nontrivial null-space or N{A) =1= {O}.
Clearly this result is consistent with the definition of linear dependence given
above: {a1, ••• , an} is linearly dependent if and only if AX = 0 for some X =1= O.
But from the definition of the linear independence of the vectors {a1 , ... , an}'
AX = 0 has only the trivial solution. Hence the preceding theorem can be
rephrased as: N(A) = {OJ if and only if {a1 , .•. , an} is linearly independent.
(Note that T A is non-singular if N(T A) = {OJ so that only non-singular linear
transformations preserve linear independence.) If A is (m x n) and n > m, then A
has linearly dependent columns and thus N(A) ::f:. {OJ.
The following theorem will enable us to effectively deal with the notions of
the existence and uniqueness of solutions to simultaneous linear equation systems.
Specifically, we have the
2.4. RANK-NULLITY THEOREM. Let T A be a linear transfor-mation
defined by the (m x n) matrix A. Then p(A) + n(A) = n, p(A) ~ m.
Later on in our discussion of linear programming we shall have occasion to
employ two specialized types of transformations, namely projective and affine
transformations. Specifically, given an (n x n) matrix A with vectors d, e€ R n
and a scalar h € R, a projective trans-formation of R n is a function T: Rn-+Rn
such that
X€ Rnl-+ Y = T(X) ~!~, e'X+h::f:. 0,
where the (n+l x n+l) matrix
[..~. ~ ..~.J
e' : h
is assumed nonsingular.
For A of order (n x n) and nonsingular, let us take d = 0, e' = l'(A-I n),
and h = 1. Also, let us restrict X to the simplex u = {X Il'X = 1, X ~ OJ. Then
any point X€u is taken to a point Y€U, where
Y = T(X) = 1'(A-tjX+l = IJ;tX·
This restriction of u to a projective transformation T will be employed in chapter

13 wherein Karmarkar's interior point algorithm for solving a linear program is
presented.
Next, for (n x n) matrices A and B, the function T B: Rn--+Rn is termed an

affine transformation if there exists a linear transformation T A: Rn--+Rn and a
vector C € R n such that
or
Y EX = AX+C.
This expression is essentially a linear transformation followed by a translation,

where C (fixed) specifies the magnitude and direction of the translation.
2.3. The Solution Set of a System of

Simultaneous Linear Equations
A nonhomogeneous system of m linear equations in n unknowns may be
written in matrix form as AX = C, where A is an (m x n) matrix of (constant)
coefficients aij' X is an (n xl) matrix of unknown variables xi' and C is an
(mx 1) matrix of constants ci' i=l, ... , m; j=l, ... , n. Let us now look to the
structure of the solution set for AX = C. Here we shall be interested m the
existence and uniqueness of a solution to this simultaneous linear system. To
facilitate this discussion we note that a system of equations is consistent if it has
at least one solution; it is inconsistent if it does not possess a solution. Moreover,
for Xo a particular solution to AX = C, the set of all solutions X to AX =C is
equivalent to the set of all solutions Xa + h, where h is a member of the set of all
solutions of the homogeneous system Ah = 0 associated with AX = C. In this
regard, the structure of the solution set for AX =C, which involves three
mutually exclusive and collectively exhaustive cases, appears as:
1. No solution exists (AX = C is inconsistent);
2. Exactly one solution exists; }
(AX = C is consistent).
3. An infinity of solutions exists
As we shall now see, issues concerning the existence and unique-ness of
solutions to AX = C, A of order (m x n), are intimately related to the concepts of
the range and null-space of T A' It was mentioned earlier that R(A) consists of
linear combinations of the columns of A, i.e., Cf R(A) = C{aj}7=1 if
C = ~j=lX j3j' Thus the question of the existence of solutions to AX = C boils

down to whether or not C f R(A). In this regard we state first that C f R(A) if
and only if C = AX, XfRn. So if Cf R(A), then AX = C is consistent and has
at least one solution. Moreover, if R(A) = Rm, then AX = C has a solution for
all C f Rm. The following theorem addresses both the existence and uniqueness of
solutions to AX = C.
2.5. THEOREM. Given an (m x n) matrix A, AX = C has either no
solution, exactly one solution, or infinitely many solutions.
An alternative tack for addressing questions concerning the existence and
uniqueness of simultaneous linear systems of the form AX = C is the direct
application of the rank-nullity theorem. In this regard, for A of order (m x n):
(a) if m > n ~ p(A), it follows that R(A)::f Rm. Then for some vector
CfRm, C¢R(A) and thus AX = C has no solution.
(b) if In < n, then n(A) =n - p(A) =n - m > O. In this instance N(A)
contains at least one nonzero vector so that tIle solution to AX = C is not unique,
i.e., since Ah = 0 has n - m linearly independent columns, arbitrary values may
be assigned to n - m of the variables.
(c) if m = n, then p(A) = In if and only if n(A) = O. With N(A) = {O},
AX = C has a unique solution for every CfR = R(A). m
If [A, Cl depicts the augmented matrix associated with AX = C, then we

may also address the consistency issue of this equation system by simply
comparing the ranks of A, [A, Cl. To this end we state
2.6. THEOREM. Given the system AX = C, where A is of order (In X n), if:
1. p[A,Cl > p(A), the system is inconsistent;
2. p[A, Cl = p(A) = number of unknowns n, the system IS
consistent and possesses a unique solution;

3. p[A, Cl = p(A) = k < number of unknowns n, the system is consis-
tent and possesses an infinity of solutions, where arbitrary values
may be assigned to n-k of the variables.
As a particularization of case (2), if A is (n x n) ancl nonsingular, then X = A-I C.
If C = 0, then the resulting homogeneous system can never be inconsistent,

i.e., AX = 0 always has a solution since p(A) = p[A, 0]. Moreover, X = 0 is
always a (trivial) solution. What about the existence of nontrivial solutions?
The answer is provided by a special case of theorem 2.6, namely
2.7. THEOREM. Given the system AX = 0, where A is of order (m x n), if:
1. p(A) = number of unknowns n, the system has a unique (trivial)
solution X = OJ
2. p(A) = k < number of unknowns n, the system has an infinity of
nontrivial solutions, where arbitrary values may be assigned to
n-k of the variables.
If A is (m x n) and m < n, then AX = 0 always possesses an infinity of nontrivial
solutions. In general, for A of order (m x n) and p(A) = k, the solution set 'i of
the homogeneous linear system AX=O, XfRn , is an (n-k)-dimensional
subspace of an, i.e., a basis for 'i consists of n - k linearly independent vectors.
We may relate these observations to some of our earlier concepts by noting
that for the homogeneous system AX = 0, a vector X lies in N(A) if and only if
it is orthogonal to each vector in Row(A), i.e., Row(A), N(A) are orthogonal
complements. Hence X is a solution to AX = 0 if and only if X is orthogonal to
each row vector of A.
It was mentioned earlier that a consistent system AX = C has a unique
solution Xo if and only if Ah = 0 has only the trivial solution h = O. This result
may be strengthened somewhat if A is taken to be an nth order matrix of full
rank n. Then it must always be true that AX = C has a unique solution for any
C if and only if the associated homogeneous system Ah = 0 has the unique
solution h = O. To operationalize this result let us consider the following theorem
of the alternative} for equalities
} A theorem oj the alternative involves two mutually exclusive systems of linear

inequalities and/or equalities denoted simply as (I), (II). It asserts that either system (I) has
a solution or system (II) has a solution, but never both.
2.8. THEOREM. Given the system AX = e, where A is of order (nx n),

either:
(I) AX = e has exactly one solution for each e or
(II) Ab = 0 has a nontrivial solution,
but never both.
So to determine whether or not the nth order system AX = e has a unique
solution for every e, we need only determine whether it has a unique (trivial)
solution for only one particular e, namely e = O.
Let us now examine in greater detail the conditions under which the simul-
taneous linear system AX = e has no solutions. Based upon our earlier observa-
tions it is easily seen that AX = e is inconsistent if p[A, C] = l+p(A).
Another approach to this problem involves an application of
2.9. FARKAS' THEOREM OF THE ALTERNATIVE [Farkas (1902)].
For any (m x n) matrix A and a vector efRm, either
(I) AX = e has a solution X ~ 0, XfRn, or
(II) A'y ~ 0, e'Y < 0 has a solution Y fRm,
but never both.
(A variety of versions of this theorem as well as its geometric interpretation will
be offered in subsequent sections.) In this regard, if the assertion "AX = e is
consistent" is to serve as Farkas' (I), then what should be the appropriate form
of Farkas' (II)? To answer this question and to fully mirror (I), let X = U - V
with U, V ~ O. Then
A(U-V) = e has a solution U, V ~ 0 or
(I') [A, - A] [~] = e has a nonnegative solution
has the form of (I). From (II), the alternative to (I') is
[!"~, ]Y ~ 0, e'Y < 0 or
(II') A'Y = 0, e'Y < 0 has a solution Y.
Actually, the sign of e'Y is irrelevant in the sense that if Y is replaced
by - Y, then (II') becomes
(11") A'y = 0, C'y =f:. 0 has a solution Y.
In sum, either AX =C is consistent or there exists a vector Y which is ortho-

gonal to the columns of A but not orthogonal to C. These observations are
consolidated in the following
2.10. THEOREM. The simultaneous linear system AX = C, where A is of
order (m x n) and X E R n , is inconsistent if and only if the linear
system A'U = 0, C'U = a, 0 =f:. a E R, has a solution U E Rm.
Let us relate the results obtained in theorem 2.7 to the concept of linear
independence (dependence) presented in section 2.1 above. Specifically, if we
redefine the rank of an (m x n) matrix A as the maximum number of linearly
independent vectors Xj (Rm, j=l, ... , n, which span the columns of A (i.e., it is
at most the number of vectors in a basis for R m), then a test for the linear
independence of a set of vectors of the form {Xj(Rm, j=l, ... , n} may be
executed by considering the vector Xj as the lh column of an (m x n) matrix
A = [Xl' ... , Xn]. Then:
1. if m~n and p(A)=n, the set of vectors {X j (R m ,j=l, ... ,n} IS
linearly independent (by part 1 of theorem 2.7); but

2. if m ~ nand p(A) < n, or if m < n, the set of vectors {Xj(Rm,
j=l, ... , n} is linearly dependent (by part 2 of theorem 2.7).
An alternative way of expressing 1 is provided by
2.11. THEOREM. Let A' = [al, ... ,am] be of order (nxm). If the vectors
a l , ... , am are linearly independent in R n, then there exists a vector
Y ERn such that AY = b for any bERm.
Theorem 2.7 dealt with the existence of solutions to homogeneous linear
systems of equalities. However, in certain important areas of linear programming
(e.g., duality theory), the existence of solutions to homogeneous linear systems of
inequalities (that possible contain some strict equalities) is of paramount impor-
tance. In particular, two systems of homogeneous linear inequalities/equalities
are termed dual systems (Tucker (1956» if they have the following structure:
AllXI + Al2 X2 ~ 0 U1 ~O
A21 Xl + A22 X2 = 0 U2 unrestricted
.-lis dual to~
Xl ~O -A~l U1 - A~l U2 ~ 0
X2 unrestricted -AbU I - A~2U2-= 0
The correspondence between these two homogeneous systems possesses the fol-
lowing properties:
(a) each system is dual to the other;
(b) each inequality structural (fully coefficiented) constraint is complementary
to (is in one-to-one correspondence with) a non-negative variable and conversely;
(c) each equality structural constraint is complementary to a variable
unrestricted in sign and conversely;
(d) the coefficient matrices of the lineal' structural relations of one system are
the negative transposes of the coefficient matrices of the linear structural relations
of the other system and conversely.
If Al2 = A21 = A22 = 0 and All = A is skew-symmetric (A'= -A), then as
a special case of the preceding dual systems we obtain, for Xl = X, UI = U, the
self-dual system
A~ ~ ~} .-lis self-dual to~ {
U~O
-A'U ~ O.
Relative to this self-dual system we have the so-called skew-symmetric
2.7.a. THEOREM [Tucker (1956)]. The self-dual system
AX~O
X~O,
where A is skew-symmetric, possesses at least one solution X~ 0

such that
2.4. Orthogonal Projections and Least Squares Solutions

Consider the simultaneous linear equation system AX = C, where A is of
order (m x n) with m > n = p(A) (the columns of A are linearly independent and
form a basis for Col(A». A system such as this is termed overdetermined in

that, since p(A) < m, there are vectors CfRm such that AX = C is inconsistent.
While AX =C has no exact solution, it does admit an approximate solution
which we shall call a pseudosolution or least-squares solution, i.e., a solution X
that comes as close as possible to satisfying AX = C.
Let GJl in R m be a plane through the origin spanned by the linearly
independent columns of A, i.e., GJl = Col(A). (For GJl in R3 and A = [a,.,~], see
Figure 2.2.) Since AX =C is inconsistent, ctGJl. Let C = U + V, where U is
the orthogonal projection of C onto GJl and V is orthogonal to GJl, i.e., V fGJl.l.
Since U f GJl, there exists a vector X f R n such that U = AX. The orthogonal
projection U of the point C onto GJl is the (unique) element of GJl for which
II C - U II is as small as possible.
oP
I
" " " - - - - - -. . .
I
I
I
Figure 2.2
Equivalently, we choose to find an X such that

g(X) = m4n~IC- UII 2 IIC-AxI1 2
E?~l I ci - aiX 12 E~lr~}, (2.6)
where (}:i IS the ith row of A and ri = I ci - (}:iX I is the residual or error
associated with the ith equation in AX = C. Hence we seek a vector X which
renders the smallest sum of squared residuals in the sense that X is as close as
possible to being a solution of an inconsistent system AX = C. Thus among all
possible X, X comes closest to satisfying AX = C in that the residual r = C - AX
has the smallest norm among all possible residual vectors r = C - AX.
To obtain U, the orthogonal projection of C onto c:P, let us consider
2.12. THEOREM. The vector X is a least squares solution to AX = C if
and only if
A'AX = A'C. (2.7)
Here (2.7) is termed the system of nonnal equations which coincides with a least
squares solution to AX = C. To derive this system, let us use (2.6) to
min 9 = min{ C'C - 2X' A'C + X' A' AX}.
X X
Setting agjaX = 0 then renders (2.7).
Since A and the (n x n) matrix A' A have the same null-space, p(A' A)
n - n(A' A) = n - n(A) = p(A) = n (by the rank-nullity theorem). Hence
A' A has an inverse and thus we obtain, from (2.7),
X = (A' A)-1 A'C. (2.7.1)
In sum, if the (m X n) matrix A, where m > n, has maximal rank n, then A' A is
invertible and the unique least squares solution of AX = C is provided by (2.7.1).
Here the matrix At = (A' A) -1 A' is called the generalized or pseudoinverse of A
and serves only as a left-inverse for A since At A = In i- AA t. (If m=n, At =A-1.)
Note that (A,)t = (AA')-1A and that (At), = (A,)t.
Given X from (2.7.1), the orthogonal projection U of C onto c:P = Col(A) is
U = AX = A(A' A)-1 A'C. (2.8)
Clearly the orthogonal projection U of C onto c:P is closer to C than any other
point in c:P.
Alternatively, given U = AX, let Y be an arbitrary vector m Rn. Then
AY = E'J=lY j3j is an element of c:P and thus orthogonal to V c-u

C - AX f c:P .1. Hence
(AY) . V = yl A' (C - AX)
= Y/(A/C-A/AX) = o.
Since this expression holds for any Y f R n, it must be the case that
A/C - A' AX = 0 or (2.7) holds. Then under the preceding rank argument, (2.7)
again has a unique solution and thus (2.8) obtains.
2.5. Point-Set Theory: Topological Properties of Rn

Let 6 be any positive scalar. A 6-neighborhood of a point XofRn (or an
open ball or sphere of radius 6 about Xo) is the set B(Xo, 6) consisting of all
I"
points X such that" X-Xo" < 6, i.e., B(Xo, 6) = {X X-Xo II < 6, 6> O}.
A point X is an interior point of a set ':f in R n if there exists an open ball of
radius 6 about X which contains only points of ':f. The interior of a set ':f in R n,
denoted ':f 0 , is the collection of all its interior points.
A set ':f in an is said to be open if, given any point Xo f ':f, there exists a
positive scalar 6 such that B(Xo' 8) ~ ':f. Hence ':f is open if it contains only
interior points. Moreover:
(a) <p, B(Xo' 6), and R n are all open sets; and
(b) ':f is an open set if and only if':f = ':foe
Let ':f be a set in Rn. Then the complementary set of'1, ':fc, is the collection
of all points of R n exclusive (lying outside) of ':f. A point Xf':f c is an exterior
point of ':f in an if there exists an open ball of radius 8 about X which contains
only points of ':fc. The exterior of a set ':f in an, denoted ':fe, is the collection of
all its exterior points.
A point X is a boundary point of a set ':f in R n if every open ball of radius 6
about X encompasses points in ':f and in ':fc, i.e., it is a point which is neither an
interior nor an exterior point of ':f. The boundary of a set ':f in an, 8(':f), is the
collection of all its boundary points.
A set ':f in an is bounded if there exists a scalar Mfa such that II X II < M for
all Xf'i. Equivalently, ':f in an is bounded if it has a finite diameter d(':f), z.e.,
d(:f) = sup{1I X-YIIIX, Yf:f} < +00. Two additional concepts which utilize the
notion of the distance between points in R n will at times prove useful. First, for
X a vector in R n and :f a subset of R n , the distance between a vector X and a set
'J is d(X,:f)= inf{IIX-YIIIYf:f}. Next, for sets:f, ~ in R n , the distance
between sets 'J,~ is d(:f,~)=inf{IIX-YIlI Xf:f, Yf~}. If :fn~1-¢, then
d(:f,~) = O.
A point X is termed a point of closure or adherent point of a set :f in R n if
every open ball of radius 6 centered on X contains at least one point of:f, i.e.,
B(X, 6) n:f 1- ¢. Hence every open ball of radius 6 about X meets:f. A subset
:f of R n is said to be closed if every point of closure of :f is contained in :f, i.e.,
every point that is arbitrarily close to :f is a member of :f. The closure of a set :f
in R n , denoted f, is the set of points of closure of :f. Clearly a set f in R n is
closed if and only if :f = f.
Closely related to the concept of a point of closure of :f is the notion of a
limit point of a set 'J in R n (also termed a cluster point or point of
accumulation). Specifically,X is a limit point of a set :f if each open ball of
radius 6 about X contains at least one point of :f different from X. Hence B(X, 6)
meets :f - {X}; points of:f different from X pile up at X. In this regard, if X is a
limit point of a set :f in R n , then for every 6> 0, the set :f n B(X, 6) is an infinite
set, i.e., every neighborhood of X contains infinitely many points of :f.
On the basis of the preceding discussion we can alternatively conclude that
a set f in R n is closed if it contains each of its limit points; and the closure of a
set :f in R n is :f itself together with its set of limit points. Furthermore:
(a) ¢, a single vector or point, and R n are all closed sets.
(b) Any finite union of closed sets in R n is closed; any intersection of closed
sets in R n is closed.
(c) A subset :f of R n is closed if and only if its complementary set fC is open.
(d) For :f a set in R n , the boundary of f, 8(f), is the closed set f n fC.
Thus it must be true that f =:f U 8(:f).
(e) A subset :f in R n is closed if and only if:f contains its boundary.
We previously defined the concepts of a closed set and a bounded set. If a

set :1 in R n is both closed and bounded, then it is termed compact. We note
briefly that:
(a) A closed subset of a compact set :1 in Rn is compact.
(b) The union of a finite number of compact sets in R n is compact; the
intersection of any number of compact sets in R n is compact.
(c) Any finite set {Xl' ..., X k } in R n is compact.
A set :1 in R n is said to be locally compact if each of its points has a
neighborhood with compact closure, i.e., for each Xf:1 there is an open sphere of
radius 6 > 0 centered on X, B(X, 6), such that B(X, 6) is compact. It should be
evident that any compact space is locally compact but not conversely, e.g., R n is
locally compact but not compact.
2.6. Hyperplanes and Half-Planes (-Spaces)

A set L in R n is termed a linear variety (manifold) or affine set if for
vectors Xl' X2 f L, their affine combination
"\IX 1 +..\2 X2' ..\1 +..\2 = 1, and ..\1'..\2 fR
is also a member of L. Geometrically, this definition implies that the line
passing through any two distinct points of L lies entirely in L. Alternatively, a
set L in R n is a linear variety or affine set if and only if L = <:P + Xo, where <:P is
a subspace of Rn and Xo IS an arbitrary point of Rn. Here
<:P+Xo= {X+XoIXf<:P eRn, XofRn} is termed the translation ofGJ by Xo. If
Xl' X2 fGJ+Xo, then both X1 -Xo and X2 -Xo are members of <:P. Conversely,
it can be shown that every linear variety is of the form GJ + Xo, i. e., if L is a
linear variety, then there exists a subspace GJ and a point Xo f R n such that
L = GJ+Xo.
The dimension of an affine set L in R n (denoted dim( L)) is the dimension
of the subspace <:P of R n obtained by translating L so that it contains the origin
{a}. If L = <p, then dim(L) = -1. Additionally, affine sets of dimension 0, 1,
and 2 are points, lines, and planes respectively.
As we shall now demonstrate, for a vector C( :f:. 0) f R n and a scalar a f R,

the hyperplane
is an (n-l)-dimensional linear variety or affine subset of Rn. To see this let

Xl f %. Translating by - Xl yields the linear subspace .At:. = % - Xl' What about
the structure of.At:.? Since C'XI = a, it follows that C'X-C'XI = C'(X-XI )
=C'X=a-a=O. Hence X(=X-XI)f.At:. so that.At:. must be the set of all
vectors orthogonal to C. Likewise, C must be (mutually) orthogonal to the
subspace .At:. so that C amounts to the one-dimensional subspace .At:..l = {C} and
is termed the nonnalof %. And since .At:. and {C} taken together span R n , it
follows that dim(.At:.) = dim(%) = n-dim({C}) = n-l.
Any hyperplane % = {XIC'X = a, Xf Rn} divides R n into the two closed
half-planes (-spaces)
[%+] > a, Xf R n },
{XIC'X
[%-] = {XIC'X < a, Xf Rn}.
If the boundary line C'X = a is excluded from the preceding two sets, then we
have the two open half-planes (-spaces)
(%+) = {XIC'X > a, Xf R n },
(%-) = {XIC'X < a, Xf Rn}.
Clearly [%+], [%-] are closed sets while (%+), (%-) are open sets.
The following theorem informs us that every affine subset L of R n corres-
ponds to the solution set of a system of m simultaneous linear equations in n
unknowns. Specifically, we state
2.13. THEOREM [Rockafellar (1970)]. Given an (mxn) matrix B and a
vector bf R m, the solution set L = {XIBX = b, Xf Rn} in R n is affine.
If in theorem 2.13 the affine set L is rewritten as L = {XIPiX =
hi' i=l, ... , m, Xf Rn} = n ~1 %j, where Pj is the ith row of B, hi is the jth
component of b, and %i = {XIPiX = hi' Xf R m }, i = 1, ... , m, is a hyperplane,

then it is evident that
2.14. COROLLARY. Every affine subset in R n is expressible as the
intersection of a finite collection of hyperplanes from Rn.
Next, given any set 'J in R n , the affine hull of'J (or the affine subspace
spanned by 1), denoted aff(,!), is the set of all affine combinations of points from
'! or
aff('!) = {x Ix "t
i=1
\Xj, Xj (; '!,
=
1=1
.f \
= 1, Aj (; R for all i}.
Here aff('!) is unique and can be thought of as the intersection of all affine sets
.A j in R n which contain '!, i.e., aff('!) = n j.Aj' where'! C .Aj for all i. In this
regard, an affine set c:r in R n is the smallest affine set containing '! in R n if and
only if c:r = aff(,!). Then dim(c:r) = dim( aff(,!)).
Let {Xo, Xl' ... ' X k } represent a finite set of k+l vectors in Rn. The affine
hull of {Xo, Xl' ... , X k }, denoted a//{Xo, Xl' ... ' X k }, is the set of all affine
combinations of its members. The set {Xo, Xl' ... ' X k } is affinely independent if
af/{Xo, Xl' ..., X k } = a/f{O, XcXo, ... , Xk-Xo}+Xo = L+Xo has dimension
k. Since L is the smallest subspace spanned by the translated set
{XI-XO' ••• , Xk-Xo}, dim(a/f{Xo, Xl' ... , X k }) = dim(L) = k if the translated
set of vectors is linearly independent. Hence a set of k+ 1 vectors
{Xo, Xl' ... , X k } in R n is said to be affinely independent if and only if the
translated set {XI - Xo, ... , X k - Xo} is linearly independent.
Additional features of affine sets or linear varieties are:
(a) ,p, R n , and the singleton {X} are all affine sets.
(b) The intersection of an arbitrary collection of affine sets is affine.
(c) The dimension of any subset'! of R n equals the dimension of its
affine hull a/./{'!).
(d) The set of vectors {Xo, Xl' ... , X k } in R n is affinely independent if the
unique solution of Ef=o \X i = 0, Ef=o Ai = 0, \ (; R, is Ai = 0 for all i.
(e) Linear independence implies affine independence but not conversely.
(f) The maximum number of affinely independent vectors in R n is n+l
(consisting of n linearly independent vectors and the null vector).
(g) The affine hull of a finite set of points {Xl' ... ' X k } in R n is a subset
of its linear hull, i.e., a//{XI, ... ,Xk } <; l!{XI, ... ,Xk }.
(h) Given the simultaneous linear system AX = C, XERn , let

p(A) = p[A, C] = k. Then the solution set of AX = C is an (n-k) -
dimensional affine set in Rn.
2.7. Convex Sets

A set f in R n is said to be convex if for points Xl' X2 f f, their convex
combination (or internal average)
X = AIX I +A 2 X 2 , Al +A 2 = 1, and 0 :::; AI' A2fR
is also a member of f. Geometrically, this definition implies that the closed line
segment joining Xl and X2 lies entirely in f. (A convex set differs somewhat
from an affine set in that, for the latter, the entire line passing through Xl and
X 2 lies wholly within the set.) A set f in R n containing only a single element is
taken to be (trivially) convex.
An important property of a given collection of convex sets is provided by
2.15. THEOREM. Let {f i } be a family of convex sets in Rn. Then their
intersection n ifi is a convex set.
Also of particular interest is
2.16. THEOREM. Let f be a convex set in Rn. Then the closure f of f is
also a convex set.
Some additional properties of convex sets are:
(a) Trivially, ¢ and R n are each convex sets.
(b) If fin R n is a convex set, then fO (possible empty) is convex.
(c) In general, the union of a collection of convex sets in R n is not convex.
(d) A hyperplane along with both open and closed half-spaces in R n are all
convex sets.
(e) If f I' ... ' f m are each convex sets in Rn , then their linear combination
f = E~l \ f i is also con vex for Ai f R for all i.
(f) Every affine set is convex, but not conversely.
To set the stage for our next concept let us first consider the following
2.17. THEOREM. Given an (mxn) matrix B and a vectorbtRm, the

solution set 'J = {X IBX ~ b, Xt Rn} in R n is convex.
In this regard, for Pi' i=I, ... , m, the ith row of B, let [%iJ
= {X I PiX ~ bi ,
Xt Rn}, i=I, ... , m, where bi is the ith component of b. Clearly [%iJ is a closed
half-space. And since each of these closed half-spaces is convex, it follows from
theorem 2.15 that 'J = n ~1[%iJ is convex. We can now assert that a set 'J in
R n which can be expressed as the intersection of finitely many closed half-spaces
in R n is a polyhedral convex set.
A convex combination of a finite number of points Xi' ;=1, ... , m, in R n is
the point
m m
X =L \Xi, L \ = 1,
0 ~ Ai t R for all i.
;=1 i=1
The following two theorems relate this construct back to the notion of a convex
set presen ted earlier.
2.18. THEOREM. A set 'J in R n is COnvex if and only if every convex
combination of finitely many points of'J is an element of 'J.
The next theorem considers a key property of the set of all convex
combinations of a finite number of points. Specifically,
2.19. THEOREM. Let {Xl"'" Xm} be a finite collection of points in Rn.
Then the set of all convex combinations of the Xi' i = 1, ... , m,
m m
g; = {XIX = L
AiXi' L\
= 1, 0 ~ AitR for all i}
i=1 i=1
is convex.
We note briefly that if 'JI,. .. ,'J m are each convex sets in R n , then their
Convex combination 'J = E~l Ai 'J i , E~l Ai = 1, 0 ~ \ t R is also convex.
The convex hull (or convex closure) co('J) of a set 'J in R n is the collection of
all convex combinations of points from 'J or
co('J) = {X IX = t
2=1
AiXi' Xi t 'J, t \=
2=1 and
1, 0 ~ Ai t R for all i,
m'IS an ar b't
I rary POSI't"Ive
}
III t eger .
2.20. THEOREM. A set <r in R n is the smallest convex set containing an

arbitrary set 'J in R n if and only if GJ = co('J).
A set !f in R n which is the convex hull of finitely many points is said to be

spanned or generated by those points and is called a convex polytope.
How should the word smallest in the preceding theorem be interpreted?
The answer is provided by
2.21. THEOREM. The convex hull of a set !f in R n is the intersection of
all convex sets .A j in R n which contain !f, i.e., co(!f) = n jAj, where
!f C .A j for all i.
We next have
2.22. THEOREM. Let {Xl' ... ' Xm} be a finite collection of points in Rn.
Then the convex hull of the Xj' i = 1, ... , m, is the set of all their
convex combinations, i.e., g; = co{XI , ... , Xm} = {X IX = E~l AiXj,
E~l Ai = 1, 0 ~ Aj {R for all i}.
We note briefly that:
(a) If a set !f in R n is convex, then co(!f) = !f.
(b) For !f a subset of R n , co(!f) is a closed set if!f is a finite set.
(c) If a set !f in R n is compact, then co(!f) is compact.
(d) If !f is an open set in R n , then co(!f) is open. The convex hull of a
closed set in R n need not be closed.
(e) All affine sets are convex (hence convex sets are more general than
affine sets).
(f) For!f a subset of R n , the closed convex hull of !f is the closure of co(!f)
or co(!f).
(g) In general, the convex hull of a closed set need not be closed. If!f is a
bounded subset of R n, then co(!f) = co(1). So if !f is closed and
bounded, then co(!f) is closed and bounded.
(h) For !f(=fo</» an arbitrary set in R n, !f ~ co(!f) ~ aff(!f) ~ e(!f).
(i) For a finite set of points {Xl' ... ' X k } in R n,
co{XI,.··,Xk } ~ aff{XI,···,Xd ~ e{XI,···,Xd·
Earlier in this chapter we encountered the concept of the interior of a set in
Rn. Interestingly enough, for convex sets, interior is subsumed under the notion
of relative interior. Specifically, the relative interior of a convex set 1 in R n ,

denoted ri(1), is the interior which results whenever 1 is considered a subset of its
affine hull a/J(1). To put this definition in its proper perspective, let us admit
another point Y into our discussion. In this regard, the relative interior of ':I
consists of those points X f a/J(1) such that Y f 1 whenever Y is near X and also a
member of a/J(1) or ri(1) = {Xf a/J(1) I Y f B(X, 6) n (a/J(1)) implies Y d}.
A convex set ':I in R n is termed relatively open if ':I = ri(':I). Furthermore, the
relative 6oundaryof a convex subset 1 of Rn is r8(1) = {XIXf1 and XI ri(1)}.
Additional properties or relationships involving the relative interior of a
convex set 1 in R n are:
(a) A non empty convex set 1 in R n always has a nonempty relative interior.
(b) For 1 a convex subset of R n , ri(1) C 1 C 1 c a/J(1).
(c) Any affine set 1 in R n is relatively open (by definition).
(d) For any convex set 1 in Rn and any scalar AfR, ri(A':I) = Ari(1).
Finally, a set 1 is said to be locally convex if each point of 1 has a
neighborhood whose intersection with ':f is convex.
2.8. Existence of Separating and Supporting Hyperplanes

We noted in section 2.6 that for a vector C(f:O)fRn and a scalar cHR, a
hyperplane
% = {XIC'X = a, XfRn}
is an (n-1) - dimensional linear variety or affine set and that any hyperplane %
in R n generates the two closed hal/-spaces
[%+] {X I c'x ~ a, Xf Rn},
[%-] {XIC'X:os; a, XfRn}.
A point in R n lies in [%+], [%-], or in both of these sets.
If Xo is taken to be a fixed reference point in %, then C'Xo = a and, for
any other Xf%, C'X = a so that C'X-C'Xo = C'(X-Xo) = a-a = O. Hence a
hyperplane has the alternative representation
% = {X I C'(X-Xo) = 0, X = Xo fixed, Xf Rn}.
Clearly C is orthogonal to X - Xo for any X (% and is termed the normal of the

hyperplane. Addi tionally, for fixed Xo ( %, we now have an al ternati ve
representation of the preceding closed half-spaces:
[%+] = {XIC'(X-Xo) ~ 0, X = Xo fixed, X(Rn},
[%-] = {X I C'(X-Xo) ~ 0, X = Xo fixed, X( Rn}
In what follows either of these equivalent representations of hyperplanes and half-
planes will be used depending upon which version is most appropriate for the
discussion at hand.
Let :J't,:f2 be nonempty sets in Rn. Then the hyperplane % = {X I C'X = a,
X( Rn} weakly separates :f l,:f 2 if C'X ~ a for X(:f 1 and C'X ~ a for X(:f 2 while
% strictly separates :f l' :f 2 if C'X > a for X ( :f1 and C'X < a for X ( :f 2' Also, %
is said to strongly °
separate :f l' :f 2 if there exists < E ( R such that C'X ~ a + E
for X(:f 1 and C'X~ a for X(:f 2 .
For :f a nonempty set in R n and X a point in the boundary of :f, a hyper-
plane %={XIC'(X-X)=O, X=X fixed, X(Rn} is a supporting (or tangent)
hyperplane of j at X if either j C [%+] (hence C'(X-X) 2: 0 for Xc j) or j C [%-]
(so that C'(X-X) ~ 0 for X(j). Thus % is a supporting hyperplane of j if: (1)
:f lies in one of the two closed half-spaces [%+], [%-] and; (2) % has a point in
common with:f. Equivalently, % is a supporting hyperplane of :f in R n at
XdJ(:f) if either C'X = inf{C'X IX(:f} or C'X = sup{C'X IX( :fl. Moreover, %
properly supports :f in R n at X if :f ct %.
If % is a supporting hyperplane for a convex set f in R n at X(o(f), then
the half-spaces
[%J] = {XIC'X ~ a = C'X, X(Rn},
[%;] = {X IC'X ~ a = C'X, X( Rn}
are supporting closed half-spaces for f.

Based upon these definitions, we begin with the
2.23. STRONG SEPARATION THEOREM. Let:f be a non-empty closed
convex set in R n with Y (Rn a vector not in:f. Then there exists a
vector C( 1= 0) eRn and a scalar a c R such that C'X :5 a < C'Y for all
Xd'.
In essence, this theorem states that if ':I is a closed convex set in R n and if
Y eRn is not a member of ':I, then there exists a strongly separating hyperplane %
= {X! c'x = a, Xc Rn} such that Y is in the open half-space (%+) and ':I is in
the closed half-space [%-] determined by %. Note that by implication the
hyperplane % also strictly separates set ':I and the vector Y.
The preceding theorem was developed under the assumption that ':I is a
nonempty closed convex set. However, if ':I is not closed and Y I 1, then the basic
result of theorem 2.23 still holds with C'y> a = sup{C'X! Xc ':I}. In this
instance theorem 2.23 may be restated in its less restrictive form as
2.23.a. STRONG SEPARATION THEOREM. Let ':I be a non-empty convex
set in R n with Y eRn an exterior point of 1. Then there exists a vec-
torC(I=O)cRnand a scalar acRsuch that C'y> a = sup{C'X!Xd'}.
This result reveals that if ':I is a convex set in R n and Y f R n is not a
member of 1, then there exists a strongly separating hyperplane If, = {X! c'x =
a, Xc Rn} passing through Xc 8(':1) and having ':I on or beneath it with Y an
element of the open half-space (If,+).
We next have the
2.24. PLANE OF SUPPORT THEOREM. Let ':I be a nonempty closed
convex set in R n and let X be a boundary point of ':I. Then there
exists a vector C( 1= 0) eRn and a scalar a c R such that C'X :5 a = C'X
(= sup{C'X!Xd'}) for all Xc':l.
If ':I is not a closed subset of R n, then the basic result of theorem 2.24 may
be framed as
2.24.a. PLANE OF SUPPORT THEOREM. Let ':I be a non empty convex
set with interior points in R n and let X be a boundary point of ':I.
Then there exists a vector C( 1= 0) eRn such that C'X:5 a = C'X
(= sup{C'X! Xc1}) for each X contained within the closure of ':I, 1.
Theorems 2.24, 2.24.a inform us that a convex subset of R n (which mayor
may not be closed) has a supporting hyperplane at each of its boundary points.
So given a boundary point of a convex set :f in R n , there exists a hyperplane
that: (1) contains the boundary point, i.e., %= {XIC'X=a=C'X, XfRn}j
and (2) contains :f in one of its supporting closed half-spaces. One important
implication of these plane of support theorems is that they allow us to represent
a convex set in terms of its associated set of supporting or tangent hyperplanes at
boundary points on:f. In this regard,
2.25. REPRESENTATION THEOREM. A closed convex set :f in R n is the
intersection of all supporting closed half-spaces containing it, z.e.,
:f=ni[%;li (or :f=ni[%;li)'
This result follows from our earlier discussions on properties of convex sets
that the intersection of an arbitrary number of closed and convex sets in R n is
itself a closed convex set. Hence the intersection of all supporting closed half-
spaces containing :f is the smallest closed convex set containing :f so that :f =
co(:f) = n i[%;li (by virtue of theorems 2.20, 2.21).
Note that if ':f in R n is only convex and not also closed, then
-:f = + - -
n i[%8 li (or :f = n i [%81;).
A somewhat stronger version of the preceding theorem is
2.25.a. REPRESENTATION THEOREM. A closed convex set ':f in R n is the
intersection of all the closed half-spaces tangent to it.
As will now be demonstrated, an important application of the notion of a
supporting hyperplane is that it can be used to characterize a closed convex set
with nonempty interior as convex. Specifically,
2.26. THEOREM. Let:f be a closed set in R n with :f0 f:. <p. If:f has a sup-
porting hyperplane at each of its boundary points, then :f is convex.
If we couple theorem 2.26 with theorem 2.24, then a necessary and sufficient
condition for a closed set with nonempty interior to be convex is
2.27. THEOREM. Let ':f be a closed set in R n with :f0 f:. <p. Then:f is
convex if and only if :f has a supporting hyperplane at each of its
boundary points.
The preceding separation theorems involved the existence of a hyperplane

which separates a convex set 'i (possibly closed) in R n and a point exterior to 'i.
Let us now turn to the development of a set of theorems which address the
separation of two convex sets 'iI' 'i2 in Rn.
We state first the
2.28. STRONG SEPARATION THEOREM [Minkowski (1911)]. Let 'iI' 'i2
be two non-empty disjoint convex sets in R n with 'i l compact and 'i2
closed. Then there exists a vector C(#=O)fRn and a scalar cHR such
that the following equivalent conditions hold:
(a) C'XI < C'X2 for Xl f'i l , X2 f'i 2 or
(b) in/{C'X IXf'i 2} > sup{C'X IXf'id or
(c) { C'X < Ct, Xf'i l ;

C'X > Ct, Xf'i 2 •
Next comes the
2.29. WEAK SEPARATION THEOREM [Minkowski (1911)]. Let 'iI' 'i2

be two non-empty disjoint convex sets in Rn. Then there exists a
vector C( #= 0) f R n and a scalar Ct f R such that the following
equivalent conditions hold:
(a) C'X I ~ C'X2 for Xl f 'i I' X2 f 'i 2 or
(b) inJ{C'XIXf'i 2 } ~ sup{C'XIXf'i I } or
(c)
Finally, a set of observations concerning the existence of supporting hyper-

planes for convex sets are the following:
2.30. THEOREM. Let 'i be a nonempty convex set in R n and let GJ be a
nonempty convex subset of'i with ri('i) n GJ = ¢. Then there exists
a proper supporting hyperplane to f containing GJ.
2.31. THEOREM. A l10nempty convex set 'i in R n has a normal
C(#=O)fRn at each of its boundary points, i.e., there exists a C(#=O)
such that C'X = 0 for each Xf8('i).
It is important to note that separability of non empty sets does not imply
disjointness or convexity. But if nonempty sets are convex and disjoint (or at
least have no interior points in common), then they are separable.
If we relax the assumption that :I\,:f 2 are con vex sets in R n, then a
reformulation of theorem 2.28 appears as
2.28.a. STRONG SEPARATION THEOREM. Let :f l ,:f2 be non-empty
compact sets in Rn. Then there exists a vector C( 1=0) € R n and a
scalar 0:' € R such that
C'X < 0:', X€:flj
C'X> 0:', X€:f 2
if and only if co(:f l ) n co(:f 2 ) = cp.
2.9. Convex Cones

A cone e in R n is a set of points such that if X f e, then so is every non-
negative scalar multiple of X, i.e., if Xfe, then AXfe for O::S: AfR, XfRn. If we
consider the set of points g; = {X}, then the cone generated by g; is e = {YIY =
AX, O::S: AfR, Xfg;}. And if O/g; and for each Y(1=0)fe there are unique Xfg;
with A > 0 such that Y = AX, then g; is termed a base of e. Moreover, the point
D(R n is termed the vertex of a cone and is an element of every cone since Y =
OX = Dfe. In general, e is not a subspace of R n since the definition of e holds
only for A ~ 0 and not all A.
In what follows we shall consider cones which are convex. Generally
speaking, a convex cone is one which is a convex set. More specifically, a cone e
in R n is termed a convex cone if and only if it is closed under the operations of
addition and multiplication by a non-negative scalar, i.e., e in Rn is a convex
cone if and only if: (a) for Xfe, AXfe, O::s: AfRj and (b) for xl ,X2 fe,
Xl +X2 fe.
Equivalently, e in R n is a convex cone if any non-negative linear
combination of pointsX l ,X2 0fealso belongs to e, i.e., if (A l X l +A 2X 2 )fe for all
°::s: Ai f R and all Xi f e, i =1,2.
It is important to note that not every cone is convex. However, a hyper-

plane % passing through the origin is a convex cone as are the closed half-spaces
[X,+] , [%-] determined by %. Moreover, the cone e ={Y!Y =AX, A ~ 0, Xc9;}
generated by the set of points 9; = {X} is convex if 9; itself is a convex set.
For a convex cone e = {Y! Y = AX, A ~ 0, XcRn}, the negative cone of e
is the convex cone -e = {-Y ! - Y = A( - X), A ~ 0, Xc e} while the orthogonal
cone of e is the convex cone e.L = {Y I y'y = 0 for all Y c e}. Moreover, C.L is
a subspace of R n since if Y ce.L, then y'y = 0 and AY'y = 0 for all A (not just
A ~ 0) so that AY'y ce.L for any A.
The dimension (or rank) of a convex cone e, dim(e), is the dimension of the
smallest subspace of R n which contains e and thus amounts to the dimension of
e + (-e). Equivalently, dim(e) may be viewed as the maximum number of
linearly independent vectors contained in e.
For e a convex cone in Rn :
(a) The closure eof e is convex.
(b) e is termed solid if eo ::ft ¢.
(c) e is never a strictly bounded set (although it may be bounded either
from above or from below) except if, trivially, 0 is its only element.
A convex cone e in Rn is said to be pointed if it contains no subspace other
than {OJ, i.e., given a vector acRn , e contains no line L = {X I X = Aa for all
AcR, XcRn}. Based upon this definition it follows that if a convex cone has a
vertex, it is unique and must be located at the origin. Hence a convex cone with
vertex 0 is pointed. It is also true that:
(a) A convex cone e in R n is pointed if and only if e n (-e) = {OJ.
(b) If e is a pointed convex cone in R n , there exists a hyperplane % passing
through 0 which separates e, -e and supports both simultaneously.
(c) A convex cone e in R n which is also a linear subspace has no vertex
and consequently is not pointed.
Fundamental operations on convex cones in R n are addition, intersection,
and duality, i.e.,
(a) if el , e2 are convex cones, their sum

el + e2 = {X I X = Xl + X2 ' Xl eel' X2 ee 2 }
IS a convex cone.
(b) if el , e2 are convex cones, their intersection
el ne 2 = {XIXeel and Xce 2 }
IS a convex cone.
(c) if e is a convex cone, the dual cone
e* = {Y I X'y ~ 0 for all Xee}
is a convex cone.
Note that e* consists of all vectors making a non-acute angle ( ~ 7r/2) with all
vectors of e.
Relative to fundamental operation (c) above we may take the negative of
the dual cone e* of e in R n so as to form the polar cone e+ = {Y I X'y ~ 0 for
all Xce} consisting of all vectors making a non-obtuse angle ( ~ 7r/2) with all
vectors of e. Clearly e+ is a convex cone if e is.
A convex cone e in R n is termed a finite cone if it consists of the set of all
non-negative linear combinations of a finite set of vectors, i.e., for points XjeRn ,
j=1, ... ,m,
(2.9)
Here e is said to be generated or spanned by the points Xj. Hence any vector
which can be expressed as a non-negative linear combination of a finite set of
vectors Xi' j= 1, ... , m, lies in the finite cone spanned by those vectors. An
alternative way of structuring e in (2.9) is to state that a convex cone e in R n is
a finite cone if for some (n x m) matrix A' = [al , ..., am], every X e e is a non-
negative linear combination of the (n X 1) column vectors ai' j = 1, ... , m, of A',
z.e.,
(2.9.1)
A cone such as (2.9.1) is sometimes referred to as a closed convex cone. As we

shall see later on, finite cones (those generated by a finite set of vectors) are
synonymous with what will be termed polyhedral convex cones, the latter being
formed as the intersection of a finite number of half-planes, each of whose
associated hyperplanes pass through the origin. Polyhedral convex cones are
typically written as the set {X I AX 5. O} in Rn.
A special case of (2.9), (2.9.1) occurs when there is but a single generator.
That is, a convex cone e in R n generated by a single vector a( =#=0) is termed a ray
or half-line (denoted (a)). So for a{Rn,
(a) = {X I X = Aa, 0 ~ A{R, X{Rn}.
Given (a) = {X I X = Aa, a # 0, A ~ 0, X{ Rn}, the orthogonal cone to (a) is the
hyperplane (a).l = % = {Y I a'y = O} passing through the origin while the dual
0/ (a) is the half-space (a)* = [%-] = {Y Ia'y ~ O}; it consists of all vectors
making a non-acute angle with a. Additionally, the polar of (a) is the half-space
(a)+ = [%+] = {Y I a'y ~ O}; it consists of all vectors making a non-obtuse angle
with a. If we admit to our discussion of cones a finite number of half-lines, then
we may arrive at an alternative specification of a finite cone, namely that a
convex cone e in R n is termed a finite cone if there exists a finite number of half-
lines (aj) in R n, j=I, ... , m, such that e = Ej=l (aj). Here e is to be interpreted
as the sum of the m cones (aj) and may be represented as
e = {X I X = t
J=1
Xj' Xj{(a;), X{Rn}. (2.10)
But since X = E';::l Xj = Ej=l Ajaj = A'A, A ~ 0, equation (2.10) is just e=

{X I X = A'>', >. ~ 0, X{Rn}, thus establishing the equivalence of (2.9), (2.9.1),
and (2.10).'
In sum, a cone e in R n is termed finitely generated if:
(a) each element of e is expressible as a non-negative linear
combination of a finite set of vectors; or equivalently,
(b) e can be written as the sum of a finite number of half-lines.
In defining a finite cone no mention was ma.de of the linear independence of
the spanning set {Xj' j=I, ... , m}. Indeed, the vectors Xj need not be linearly
independent. However, if some subset of the vectors generating a finite cone e is

linearly independent, then we may note that the dimension of e is the maximum
number of linearly independent vectors in e. So if e= {X I X = A' ~,~ ;::: 0,
XfRn}, then dim(e) = p(A/).
It was mentioned earlier that any hyperplane % through the origin is a
convex cone as are [%+], [%-]. In addition, %, [%+], and [%-] are all finite
cones. In this light we note briefly that:
(a) If A' = [a1, ... ,am ] is (nx m), the solution set e of the system of
homogeneous linear inequalities A' ~ ::; 0 is a finite cone. In addition,
the solution set e of A' ~ ;::: 0 is a finite cone.
(b) If A' = [a1, ... ,am] is (nx m), the set of non-negative solutions of the
homogeneous linear system A' ~ =0 forms a finite cone.
For e in R n a finite cone, the dual cone e* is the finite cone consisting of all
vectors making a non-acute angle ( ~ 7f /2) with the vectors of e, i.e., if A'
[a1, ... ,aml is (nxm) and e = {XIX A'~,~;:::O, XfRn}, then e*
{Y lAY::; 0, Y f Rn}. Moreover, e** = e.
We may now observe that a cone e in R n is closed if and only if e** = e,
where e** is the closure e of e. Here e ~ e** ~ e** = e, which implies e** = e
since e** is an intersection of half-spaces and consequently closed.
Next, with e in R n a finite cone, the polar cone e+ is the finite cone
consisting of all vectors making a non-obtuse angle ( ~ 7f /2) with the vectors of
e, z.e., if A'lS (nxm) and e={XIX=A/~, ~;:::O,XfRn}, then
e+ = {Y I AY;::: 0, Y f Rn}. Also, (e+)+ = e. The dual cone e in Rn is
sometimes referred to as the negative polar cone since e* = -e+.
For a set of vectors {al , ... , am} in Rn, their conical (non-negative linear)
combination is represented as the point
m
X= L Ajaj, 0 ~ AjfR for all j.
j=l
And for :f a set in R n, the conical hull of :f is the collection of all conical
combinations of vectors from :f or
coni(1) = {x Ix f,
=
j=l
AjXj' Xjd, 0 ~ Aje R, and m is an }
arbitrary positive integer .
Here the conical hull of any set 1 in R n is a convex cone spanned by 1 with
vertex at the origin; it is the smallest convex cone with vertex 0 containing 1.
Next, the conical hull spanned or generated by a finite collection of points
{~, ... , am} in R n is a finite convex cone and is expressed as the sum of a finite
number of half-lines or
coni{~, ...,am} ={x I x= f, (aj) = f, Ajaj, 0 ~ AjeR}

j=l j=l
= {X I X = A'A, 0 ;;;; AeRm}.
Hence a cone generated by {a1 , ... , am} is the set coni{ aI' ... , am}'
For a cone e in R n , the conical hull of e, coni(e), is the set of all conical or
non-negative linear combinations of points from e or
Moreover, if dim(e) = p, then any vector X(/O) in coni(e) is expressible as the

conical combination of p linearly independent vectors of e.
Next, if e is a finite cone in R n with the vector ae e+, then (a)+ is a
supporting half-space for e if e is contained in (a)+ while (a) .1 , the generator of
(a)+, is called a supporting hyperplane for e.
We noted above that for an (n X m) matrix A' = [~"'" a,J, the solution set
e of the system of homogeneous linear inequalities A/,\::; 0 is a finite cone. It is
also termed a polyhedral convex cone since it is the intersection of a finite collec-
tion of closed half-spaces whose boundary hyperplanes pass through the origin.
We also noted above that a finite cone in R n can be formed as a sum of
half-lines. As we shall now see, a given finite cone can also be generated from a
set of half-spaces. To develop this point in greater detail let {b1 , ••• , b q } be a set
of vectors in Rn. Then a finite cone e can be written as the intersection of a

finite number q of half-spaces (bjt = {X I bjX =:; O}, j = 1, ... , q, as
e =.n
q
(b)o)*.
)=1
Here e consists of all vectors X making non-acute angles (~ 7r /2) with each of
the vectors b I ,. .. , b q • If B' = [bI , ... , bq ] is of order (n x q), then the preceding
cone may be rewritten as e = {X I BX =:; 0, X{ Rn}.
2.10. Basic Solutions to Linear Equalities

Consider the simultaneous linear system AX = b, where A is of order
(m x n) and p(A) = p[A, b] (the system is consistent). If m < n, then we have an
underddennined equation system which, given that it is consistent, possesses an
infinite number of particular solutions. Furthermore, let p(A) = m so that
AX = b is consistent for every b and none of the linear equations is
redundant, i.e., expressible as a linear combination of one or more other equations
in the system.
Let us now select m linearly independent columns from A and express the
variables associated with these columns in terms of the remaining n - m
variables. If arbitrary values are then assigned to the latter, it is evident that we
obtain an infinity of particular solutions. Specifically, let us rearrange the
columns of A so that the first m are linearly independent and constitute the
columns of the mth order basis matrix B = [bI , ... , bnJ The remaining n - m
columns of A then form the (mxn-m) matrix R= [r1, ... ,fn _ m ]. Hence
where X has been partitioned into an (mxI) vector X B , X B= (xm, ... ,xBm)'
containing the m variables corresponding to the m linearly independent columns
of A and an (n-mxI) vector X R , Xn=(xRI, ... ,xRn_m)'
, containing the
n - m remaining variables. Since p(B) = m, B is nonsingular. Hence
(2.11)
and all solutions to AX = b can be generated by assigning arbitrary values to
Xn . In particular, if Xn = 0, X B = B-1b and we obtain what
will be called a unique basic solution to AX = b, namely
(2.11.1)
This type of solution is so named because the columns of B constitute a basis for
R m, i.e., each column of R and the vector b are expressible as a linear
combination of the linearly independent columns of B. Hence the variables in X B
are said to be basic variables while those within Xn are termed nonbasic
variables. Moreover, a basic solution to AX = b is said to be degenerate if one or
more of the basic variables within X B vanishes, i.e., X contains fewer than m
positive variables or, equivalently, X contains more than n - m zeros.
One of the important aspects of the notion of a basic solution to a linear
equation system is its applicability in solving a problem of the form: given an
(mxn) matrix A and vector bfRm, find an XfRn such that AX = b with
X ~ O. A problem such as this can serve as system (I) of the following version of
Farkas' theorem of the alternative: either
(I) AX = b, X ~ 0 has a solution XfRn or
(II) A'y ~ 0, b'y < 0 has a solution YfRm ,
but never both. For system (I), if X B = B- 1 b ~ 0, then
X=l~lJ ~O
=
2We assumed above that for AX b we have m < n. In general, if A is (m X n) and
=
m ~ n, then a solution X of AX b is a basic solution if the columns of A corresponding to
=
nonzero components of X are linearly independent. In this regard, if peA) r and X is a basic
=
solution to AX b, then A has at least one (m X r) submatrix N such that
=
(1) peN) rj and
(2) the components of X corresponding to columns of A not in N are zero.
Conversely, associated with each (m X r) submatrix N of A having rank r there exists a unique
basic solution.
is termed a basic feasible solution to AX = b. Moreover, a basic feasible solution

in nondegenerate if each basic variable is strictly positive.
Looking to Farkas' system (I) of the preceding paragraph, given any feasible
solution X ~ 0 to AX = b, under what circumstances can we find at least one
basic feasible solution? The answer is provided by the following
2.32. EXISTENCE THEOREM FOR BASIC FEASIBLE SOLUTIONS TO
NONHOMOGENEOUS LINEAR EQUALITIES.
For an (m x n) matrix A with p(A) = m <n and a vector bERm, if
AX = b, X ~ 0 has a feasible solution XERn, then it also has a basic
feasible solution.
PROOF. 3 Assume that there exists a feasible solution to AX = b in
k ::; m positive variables. (Without loss of generality, let the
first k components of X be strictly positive.) Then AX = b becomes
k
2: x jaj = b, x j > 0, j =1, ... , k ::; m; (2.12)
j=1
What sort of solution does (2.12) represent? Since the vectors aj, j = 1, ... , k ::; m,
mayor may not be linearly independent, two cases present themselves.
First, let the vectors aj, j = 1, ... , k ::; m, be linearly independent. If k = m,
then associated with this set of vectors is an mth order nonsingular basis matrix
B which provides us with a unique nondegenerate basic feasible solution. If
k < m, we may add to the vectors aj,j = 1, ... , k < m, any m - k additional
vectors selected from those remaining in A to form
k m
2: xjaj + 2: Oaj = b
j=1 j=k+l
and thus a basis for Rm. Since the vectors aj,j=k+l, ... ,m, are admitted to the
basis at a zero level, the resulting basic feasible solution is degenerate.
3The proof of this theorem is constructive in that is spells out a computational

procedure which actually allows us to obtain a basic feasible solution from any feasible
solution to AX = h.
Next, let the vectors aj' j = 1, ... , k ~ m, be linearly dependent. Since our
goal is to ultimately attain a basic feasible solution, we shall introduce a pro-
cedure which allows us to systematically reduce the number of positive variables
step by step until the vectors associated with those that remain are linearly
independent. By hypothesis, there exist coefficients Yj not all zero such that
k
LYjaj = o. (2.13)
j=l
A multiplication of both sides of this equation by a nonzero scalar () yields
(2.13.1)
Upon subtracting this last equation from (2.12) we obtain

k k
L (Xj-(}Yj)aj = L xjaj = h. (2.14)
j=l j=l
If (2.14) is to yield a feasible solution to AX = b, then we require that
Under what conditions will this inequality hold? If () > 0, then xj ~ 0,

j = 1, ... , k, for all Y j ~ O. Hence we need only consider those Y j which are
positive. (If all Yj -I 0 are negative, we may multiply (2.13) by - 1 so as to
insure that there exists at least one Yj > 0.) Rearranging (2.15) yields xlYj ~ ()
so that any () which satisfies
(2.16)
guarantees that a feasible solution is attained. Suppose the minimum is assumed

for (j = xh/Yh. Then if this ratio is substituted into (2.14), the hth coefficient in
this equation turns to zero with the result that ah is dropped from this linear
combination. Hence
(2.14.1)
represents a new feasible solution with no more than k - 1 positive variables. If

the vectors associated with these k - 1 positive variables are linearly independent,
we have actually attained a basic feasible solution; if they are linearly dependent,
we may repeat the above procedure and reduce one of the k - 1 positive variables
to zero. If necessary, this elimination technique may be applied continually until
the remaining vectors are linearly independent and thus provide us with a basic
feasible solution. Q.E.D.
In the next section we shall encounter an extremely important geometric
interpretation of this theorem. Specifically, it will be shown that basic feasible
solutions to the system AX = b are always found at the vertices of the convex
polyhedron '.f = {X IAX = b, X ~ 0 XE Rn}.
Let us again assume that A is of order (m x n) with p(A) = m < nand
bERm. Then a basic solution to AX = b is provided by
X =[n-Ib]
0 ~ O.
Our aim is to develop a procedure for replacing a given vector in the basis by
another so that the new set of vectors is linearly independent. Here too A is
partitioned as A = [B, R]. In this regard, in generating a new basic solution to
AX = b, some vector which currently constitutes a column of the basis matrix B
must be replaced by one of the vectors comprising the columns of R.
Under what conditions can a given nonbasis vector rj, j=l, ..., n-m, within
R replace one of the basis vectors in the set ~ = {bI , ... , b m} so that the new set
of vectors ~ forms a basis for R m? The answer is provided by
2.33. THEOREM. Let the set of vectors {bI , ... , b m} constitute a basis for
Rm. If a vector rE R m is expressible as a linear combination of the
hi' r = E~I \hi' then any b i for which Ai f. 0 can be removed from
the basis and replaced by r with the result that the new collection of
vectors is also a basis for Rm.
PROOF. Without loss generality, let us assume that Am :f. 0 so that
r replaces b m in the basis. If the new set of vectors ~ = {bl ,. '" b m_ l , r}
constitutes a basis for R m, it must be linearly independent and span Rm. Let the
null vector be expressible as a linear combination of the elements in ~, i.e.,
m-l
L f.libi + f.l mr = O.
i=I
A substitution of r = E~l Ajb j into this expression yields
m-l m m-I
L f.ljb j + i=I
i=I
L f.lmA;b j = L (f.li+f.lmA;)b + f.lmAmbm = O.
i=I
j
Since the original set <:B is linearly independent, this last expression equals the
null vector if and only if f.li + f.lmAi = 0, i=I, ... , m - 1, f.l m = O. With f.l m = 0,
f.li = - f.lmA; = 0, i=I, ... , m - 1. Hence f.li = 0, i=I, ... , m, so that ~ is linearly
independent and thus forms a basis for Rm. Note that Am:f. 0 is a sufficient
condition for ~ to be linearly independent; it is also necessary since if Am = 0,
then r = E~11 \bi = 0 and thus ~ would be linearly dependent.
We know that <:B spans Rm. If it can be demonstrated that <:B is expressible
in terms of the new set ~, then the latter must also span Rm. Since Am :f. 0,
m-I
bm = A~ll r - L AiA-:n1b i
i=l
and thus
<:B = {b1,.",bm_ l , A;;/r-Ei:!:lIAiA~lbJ
And since ~ was also found to be linearly independent, it thus represents a basis
for Rm. Q.E.D.
Based upon the preceding theorem, it should be evident that since the
vectors in B form a basis for R m, any vector in R may be written as a linear
combination of the columns of B. If we denote the lh column of R by r j'
j=I, ... , n - m, then for all j,
(2.17)
Let us assume that we have chosen some vector fj to enter the basis and that the
(mx 1) vector Aj in (2.17) associated with fj has at least one positive component
\j. At the present basic solution BXB = h. Upon multiplying both sides of
(2.17) by some number (J and subtracting the result from BX B = h, we obtain
BXB-B((JAj) = b-(Jfj or
XB-(JAj = B-\b-(Jr).
Here (XB - (JAj' 0)' is a basic solution to AX = b and, if
(2.18)
then (XB - (JA j , 0)' represents a basic feasible solution to AX = h.

Under what circumstances will (2.18) be satisfied? If (J > 0, then (2.18)
always holds for all Aij ~ O. Hence we need only concern ourselves with those
components of Aj which are positive. In this regard, we need to find a (J > 0 such
that (2.18) holds for all Aij > o. Rearranging (2.18) yields ~Bi ~ (J so that any (J
IJ
for which
(2.19)
will do, i.e., will ensure that (XB - (JA j , 0)' is a basic feasible solution to AX = b.
In fact, this latter inequality enables us to select the hi to be removed from the
basis and the fj to be entered into the basis. If
{XBi
(J=(J~ = min ~,Aij>O } (2.19.1)
z 'J
is substituted into X B - (JAj' then the component of this vector for which the
minimum is attained reduces to zero. Hence the column of B to be replaced by
fj can always be determined by (2.19.1). In this regard, (2.19.1) is termed an
exit criterion for moving from one basic feasible solution to another.
If ~ = {bl ,. .. , bm } denotes the set of linearly independent vectors obtained
by replacing the rth column b ,. within B by f j' then the new basic feasible
rth column b r within B by ri' then the new basic feasible solution can be
represented as
(2.20)
..... [.....
where B = b t , ..., .....]
b m with
and
.....
() \ . . J . _ .....
() _ :x Br
:XB~I' = :x Bi - "'ii' i .,... rj :x ~ - - -r---. (2.20.1)
Br "'ri
In sum, (2.20.1):
(1) determines the vector b r to be removed from the present basis c:.Bj
(2) determines the value of the variable :x ~ corresponding to the incoming
Br
vector ri for the new basis §j and
(3) ensures that X, the new basic solution to AX = b, is feasible, i.e., X ~ O.
Armed with the concept of a basic feasible solution to AX = b, let us now
address the specific structure of the solution set '1 = {X I AX = b, X ~ 0, Xf Rn}.
As we shall now see, some of the results provided in section 2.3 may be extended
to the case where a simultaneous linear equation system comes under a non-
negativity restriction X ~ 0.
Given '1, the homogeneous system associated with AX = b, X ~ ° (i.e., asso-
ciated with :/') assumes the form AY = 0, Y ~ O. Let c:J' = {Y I AY = 0, Y ~ 0,
Y € Rn}. As indicated in section 2.9, this set of all Y yielding homogeneous
solutions is a finite or polyhedral convex cone. Furthermore, if Xf '1 and Y € c:J',
then X + ()Y €'i, 0 5 () € R. Let us now look to the following
2.34. RESOLUTION THEOREM. Let {Xt, ...,Xd be the set of basic
feasible solutions to AX=b with X€co{Xt, ...,Xk } and Y€c:J'=
{YIAY=O, Y~O, YfRn}.
If X€'i = {XI AX = b,X ~ O,X€R n} 1=,p, then X = X+ Y.
As this theorem indicates, X € '! is expressible as a convex combination of
the set of basic feasible solutions to AX = b plus a homogeneous solution
if Gf admits a unique solution Y = 0, i.e., Gf = {o}. (In the next section a

bounded :J' = {XIAX = b, X ~ 0, X ERn} will be referred to as a convex poly-
tope.)
2.11. Faces of Polyhedral Convex Sets:

Extreme Points, Facets, and Edges
In this section we shall consider a specialized class of convex sets in Rn,
namely those which may be characterized as polyhedral. Any such set is
generated by intersecting a finite number of closed half-spaces (not all of which
pass through the origin) and thus amounts to a closed convex set in Rn. While
any closed convex set in R n is the intersection of all supporting closed half-spaces
containing it (see theorem 2.25), a polyhedral convex set can be represented by
intersecting only a finite number m of closed half-spaces. In this regard, a
nonempty set :J' in R n is a polyhedral convex set (or simply a convex polyhedron)
if:J' = {Xla~X ~ bj, ai # 0', bjER, i=l, ... ,m, XERn}. For ai the ith row of the
(mxn) matrix A and bj ,i=l, ... ,m, the ith component of the (mx 1) vector b,
we may alternatively write :J' = {XIAX ~ b,XERn}. Here:J' serves as a represen-
tation for a convex polyhedron.
If a convex polyhedron in R n is bounded, then it is termed a convex polytope.
Equivalently, a convex polytope can be viewed as the convex hull generated or
spanned by finitely many points in Rn, i.e., for the finite set {Xl, ... ,Xm },
co{X l , ... , XnJ IS a convex polytope consisting of all convex combinations of
{X1,···,Xm }·
Generally speaking, a polyhedral convex set :J' in R n can be represented by a
finite number of inequalities and/or equalities (e.g., the equality
allx l + al2x2 = bl can be replaced by the two inequalities a 11 x l + a l2 x 2 ~ bl ,
- a 11 x l - a l2 x 2 ~ - bl ). Hence we may also express a convex polyhedron as
:J' = {XIAX = b,XERn}. Oftentimes the m inequalities indicated above are
accompanied by a set of nonnegativity conditions Xi ~ 0, i=l, ... , n. In this
instance,:J' = {XIAX ~ b,X ~ O,XERn} or possibly:J' = {XIAX=b, X ~ o,XERn}.
Moreover any unrestricted variable (one which may be positive, negative, or zero)
can be replaced by the difference between two nonnegative variables (e.g., xk
unrestricted in sign is equivalent to xk = x~ - x~, where x~, x~ ~ 0).
A vector X is an extreme point of a (general) convex set j in R n if X cannot be
written as a positive convex combination of two distinct points in j, i.e., X is not
an interior point of any closed line segment in j. When j in R n is a polyhedral
convex set (for concreteness, let j = {XiAX ~ b, X € Rn }), this definition, while still
valid, lends itself to further geometric interpretation. To set the stage for what
follows let us review the notion of a supporting hyperplane for j. Specifically, let j
be a convex polyhedron in R n and, for some vector C( =to) € R n , let % =

{XiC'X = a,a€R, X€R n } be a hyperplane such that: (1) C'X ~ a (or:S a), Xd
(% has j on one of its sides); and (2) there is a point X such that X € j and C'X =
a (i. e., X € j n %). Then % is termed a supporting hyperplane for j at the point X.
The polyhedral convex set j specified above is made up of the m supporting
closed (defining) half-spaces a~X:S bi , each with its associated supporting
hyperplane a~X = bi ,i=I, ... ,m. In this regard, a vector X€:I' is termed an
extreme point (or vertex or corner point) of :I' if and only if X lies on n linearly
independent supporting hyperplanes of :1', i.e., if at X we have aiX = bi , i=l, ... , k,
and aiX < bi ,i=k+l, ... ,m (assume the first k supporting hyperplanes are
satisfied at X or the first k inequalities are tight there), then X is a unique
solution to the system a~X = bi , i=I, ... ,k, if and only if the set {a1, ... ,ak}
contains n linearly independent vectors or the (nxk) matrix [~, ... ,ak] has
(column) rank n. If more than n supporting hyperplanes pass through X, then
this extreme point is said to be degenerate. Hence a nondegenerate extreme point
of j is one which satisfies exactly n supporting hyperplanes of the AX ~ b
system.
If polyhedral :I' = {XIAX = b, X ~ 0, X€ R n }, with A of order (m x n) and
p(A) = m < n, then an extreme point X €:I' occurs at the intersection of the m
linearly independent equations AX = band 71-m additional hyperplane equalities
taken from the nonnegativity conditions X ~ 0. And if :I' = {XiAX ~ b, X ~ 0,
XfRn}, where again A is (mxn) and p(A) = m < n, let D' = [A', In] represent
the coefficient matrix associated with the m+n supporting hyperplane equalities
in:l. If p(D) = m+n(i.e., D is of full row rank), then these m+n supports are
linearly independent and thus any extreme point X f:l must lie on some subset of
n such supports.
A convex polyhedron :I = {XIAX ~ b} in R n is said to be pointed if it has

extreme points or vertices. A necessary and sufficient condition for a convex
polyhedron to be pointed is that it contains no line, i.e., the associated
homogeneous system of equations AU = °admits only the trivial solution U = 0.
An inequality a~X ~ bt from system AX ~ b is deemed redundant (or
superfluous) if it can be omitted from the system without changing the structure
of the polyhedral convex set :I = {XIAX ~ b, XfRn}. Hence any redundant
inequality is implied by the other inequalities of the AX ~ b system.
Let :I be a nonempty polyhedral convex set in Rn. The dimension of '1,
dim(:I), is the dimension of the smallest affine space containing:l. So given a
point X(*O)€!I' and a set of points {XI, ... ,X r } in :1, dim(:1) is the maximum
number r such that the translated set {Xl - X, ... , Xr - X} is linearly indepen-
dent. Looked at in another fashion, dim(:1) = '" if the maximum number of
affinely independent points in ':f is r+1. A convex polyhedron ':f in R n is termed
full-dimensional if dim(:I) = n, and this holds if and only if ':f has an interior
point, i.e., a point X such that a~X < bi for all i.
Let :I = {XI AX ~ b,XfRn} (with AX ~ b nonredundant) be a nonempty
convex polyhedron in R n and let [:lti1 = {Xla~X ~ bi , X f Rn} be a supporting
closed half-space with associated supporting hyperplane :lt j = {Xla~X = bi'
XfRn}. A face "1 of ':f is a subset of points in ':f determined by some nonempty
collection of defining supports of :I. Hence "1 is either ':f itself, the null set, or the
intersection of ':f with :lt j • In this latter instance "1 = {Xla~X = bi , X f ':fl. Here
the inequality a~X ~ bj is said to define or represent "1. In addition, "1 is
termed proper if "1 f:. ¢J and "1 f:.:I. If "1 is a proper face of :I, dim("1) < dim(':f).
In fact, given a proper face "1 for ':f, if r represents the maximum number of
linearly independent support equalities holding at all feasible points of GJ, then
dim(GJ) = n-r. All of the following are types of faces:
(i) facet - a face whose dimension is one less than dim(:I) (since it is the
largest face in :I distinct from :I).
(ii) extreme point - a face of dimension zero (since it occurs at the inter-
section of n linearly independent supporting hyperplanes of :I).
(iii) edge - a face of dimension one (since it is formed by intersecting n-l
linearly independent supporting hyperplanes of :I).
In this regard, a hyperplane defining a facet corresponds to a supporting closed
half-space of :I while an edge is always a line segment joining two extreme points.
Here the two extreme points connected by the edge are said to be adjacent.
Hence an extreme point is located at the intersection of two a more distinct edges
of :I. Furthermore,:I has only finitely many faces, each face being a non empty
convex polyhedron.
Given a polyhedral convex set :I = {X/AX ~ b,XfRn }, if h is replaced by
the null vector, then the resulting set is termed a polyhedral convex cone. In this
regard, a polyhedral convex cone arises as the intersection of a finite number of
closed half-spaces whose defining hyperplanes pass through the origin and appears
as e = {X/AX ~ O,XfRn }. Here the ith row of A,a~, is the normal to the
hyperplane %i = {X/aiX = O,XfRn } generat.ing the ith closed half-space [%i] =
{X / aiX:5 O,XfRn }. More formally,
2.35. THEOREM. A convex polyhedron:l = {XIAX ~ h,XfRn } is a polyhed-
ral convex cone if and only if h = 0, i.e., every polyhedral cone is the
solution set of a homogeneous system of linear inequalities.
So if a polyhedral convex set :f in R n contains the null vector, then the
convex cone generated by :f is polyhedral.
2.12. Extreme Point Representation

for Polyhedral Convex Sets
Let us now consider the polyhedral convex set :f = {X/AX = h,
X~ O,XfRn }, where A is of order (mxn) and p(A) = m. We developed in
section 2.10 the concept of a basic feasible solution to a simultaneous linear

system of the form AX = b given that p(A) = p[A, b] = m. That is, we extracted
m linearly independent columns form A and formed an m th order nonsingular
basis matrix B (hence B- 1 exists) and subsequently obtained a unique basic
feasible solution
where the vector of basic variables X B has as its components the variables in X
corresponding to the m linearly independent columns of A. As we shall now see,
a basic feasible solution is a useful analytical device for locating extreme points of
:l'j it provides us with a necessary and sufficient characterization of an extreme
point. Specifically, we state the
2.36. EXTREME POINT THEOREM. A vector X € R n is an extreme
point of the convex polyhedron :I'= {XIAX = b,X ~ O} (A is (m x n)
and p(A) = m) if and only if X is a basic feasible solution to AX = b.
PROOF. (sufficiency) What we shall demonstrate first is that if
X € R n is a basic feasible solution to AX = b, then X is an extreme point of :1'.
Given that X is a basic feasible solution, if it is also an extreme point of :1', then
it cannot be expressed as a convex combination of any other two distinct vectors
in:l'. Hence there do not exist vectors Xl' X2 €:I', Xl :f:. X2 , different from X such
that
(2.21 )
If X~ = (Y~, 0'), X~ = (Y~, 0'), where Y l , Y 2 are both of order (m xI), then (2.21)
becomes
[~B] = 0[~2] + (1-0) [~1J. 0 0

< <1.
With X B = OY2 + (1-0)Yl = B-lb and AX I = BYI = b, AX2 = BY2 = b, it

follows that
0(Y2 - Y l ) + Y l = 0 (Y2 - Y l ) + B-lb = B-lb
and thus 0(Y2 -Yl ) = 0 or Yl = Y2 • Hence Xl = X2 = X (i.e., the representation
of b in terms of basis vectors is unique), so that there do not exist distinct vectors
X I ,X2 €1 such that (2.21) holds. Hence X is an extreme point of 1.

(necessity) Next, if X is an extreme point of 1, then there are at most m
positive components of X which have associated with them a set of linearly
independent vectors from A that provide a basic feasible solution to AX = h.
Without loss of generality, let the first k:S m components of X' = (Y', 0') be
nonzero, where Y is of order (k xl). Then the vectors associated with these
components correspond to the first k columns of A, namely ~, ... , 3.k' so that
k
AX = AIY = LXj3.j =h,
j=l
where Al is of order (m x k). If the columns of Al yield a basic feasible solution
to AX = h, then they must be linearly independent. Let us assume that the
columns of At are linearly dependent, i.e., there exists at least one coefficient
Aj =f 0 such that
or some scalar 0:' > 0, O:'AI A = 0 so that

AIY + O:'AIA = AI(Y+O:'A) = AIYI = h,
AIY - O:'AIA = AI(Y-O:'A) = At Y2 = h.
Wi th 0:' sufficiently small, both Y t, Y2 ~ ° and th us the vectors X~ = (Y~, 0'),
X~ = (Y~, 0') constitute feasible solutions to AX = h. But X = iXI + iX2 so that
if the vectors a l , ... , ak are linearly dependent, then X is not an extreme point of
1 since it can be written as a convex combination of two distinct vectors
Xl' X2 € 1. Since this contradicts the assumption that X is an extreme point, the
columns of Al must be linearly independent so as to provide a basis for Rk and
thus a basic feasible solution to AX = h.
If k = m, then
m
LXj3.j = BX B = band X B = B-Ib (2.22)
j=1
so that a unique basic feasible solution obtains. If k < m, we may add to Al any
m-k additional columns selected from those remaining in A to form
°
k m
" x ·3 . + '"
'L.J)J L.J 3 J- = b (2.22.1)
j=1 j=k+1
and thus a basis for Rm. But since these vectors are admitted at a zero level, it
is evident that the resulting basic feasible solution to AX = b is not unique, i.e.,
more than one basic feasible solution may correspond to the same extreme point.
So for x j = 0, j = k+ 1, ... , m, it is obvious that such a basic feasible solution is
degenerate. Since a basis for R m cannot contain more than m linearly
independent vectors, there are at most m linearly independent columns in A and
thus at most m positive components in X. Q.E.D.
In sum, every extreme point of j corresponds to a basic feasible solution to
AX = b (although it may be degenerate) and every basic feasible solution to the
same represents an extreme point of j.
It should be intuitively obvious that

2.37. COROLLARY. A convex polyhedron j' = {XIAX = b,X ~ 0, XERn}
has a finite number of extreme points.
In the preceding section we found that by selecting a non basis vector rj to
enter the basis ~ = {bl , ... , b m } and then applying the exit criterion (2.19.1) to
determine the vector b r to be removed from ~, we could easily move from one
basic feasible solution to another. The new basic feasible solution was denoted
and
x Bi -- xB--"O>'--
t tJ' i-l-r·
r , x~Br ="0 = x Br f>. rj·
How can we be sure that this new basic feasible solution is actually an
extreme point of :I = {XI AX = b, X ~ 0, X ERn} ? The answer depends upon
whether the vectors within B are linearly independent. If they are, the answer
must be yes.
In this regard, for polyhedral convex sets in R n , we have
2.38. EXISTENCE THEOREM. The convex polyhedron j = {XIAX = b,
X~O,X€Rn}::/=t/J (A is (mxn) and p(A)=m) has at least one
extreme point.
PROOF. According to theorem 2.32, if AX = b,X ~ °has a feasible
solution, then it also has a basic feasible solution. And since we know that there
exists a one-to-one correspondence between basic feasible and extreme point
solutions (by the preceding theorem), the proof is complete. Q.E.D.
We mentioned earlier that the convex hull of a finite number of points is
termed the convex polytope spanned by these points. Clearly a convex polytope
generated by m points cannot have more than m extreme points (since it is
defined as the set of all convex combinations of these m points). Under what
circumstances can a point within a convex polytope be represented as a convex
combination of its extreme points? To answer this question we have
2.39. REPRESENTATION THEOREM. If j is a convex polytope in R n
and ~ is its finite set of extreme points, then co(~) = :f.
In sum, every convex polytope is the convex hull of its extreme points.
(Conversely, if {XI' ... ,X m} is a finite set of points, then the convex hull of
{Xl"'" Xm} is a convex polytope:f. The set of extreme points ~ is a subset of
{Xl" .. , Xm }·)
2.13. Directions for Polyhedral Convex Sets

If j in R n is the polyhedral convex set {XIAX ~ b, X ~ 0, X€ Rn} ::/= t/J,
where A is of order (m x n), then a vector d( *0) is a recession direction of :f if
and only if
A(X+ Ad) ~ b
X+Ad ~ °
for each 0 ~ A€R and X€j. Since Xd, t.e., AX ~ b and X ~ 0, the first
inequality holds for A arbitrarily large if and only if Ad S O. Similarly,
x + Ad ~ ° for arbitrarily large A if and only if d ~ 0. In sum, d( *0) is a

recession direction for polyhedral j if and only if Ad ~ ° and d ~ 0.
For polyhedral j = {XIAX = b, X ~ 0, X f Rn} f:. </J, d( *0) is a recession
direction for j if and only if Ad = °and d °(this result obtains by applying
~
the preceding argument to the two sets of linear inequalities AX ~ b,

- AX ~ - b implied by AX = b).
Let us now establish the concept of an extreme direction for a polyhedral
convex set j = {XIAX = b,X ~ O,XfRn}, with A of order (m x n) and p(A) = m.
But first we must determine the form of a specialized type of recession direction
for j in Rn. Following the presentation given in section 2.10, we may partition
A as [B, R] (with basis matrix B nonsingular) and d' = (d~, d~), where d 1 is of
order (m x 1) and d 2 is of order (( n-m) x 1). Then the requirement Ad = °
becomes
Hence
If there exists a non basic column rj of R such that B-1rj ~ 0,

- - . ·th
j=l, ... ,n-m, then set d 2 = djej,d j >0 (ej IS the J unit column vector), so
that
(2.23)
is a recession direction for j. To see this let X be an extreme point of j. Then

d/*O) is a recession direction for j if Xfj implies X=X+Adj f1. Since
Adj = 0, d j ~ 0, and AX = b, it follows that AX = b for X = X + Ad j and for all
A. In addition, X is a nonnegative or feasible vector if A ~ 0 and B-1rj ~ 0. As
we shall now demonstrate, recession directions structured as (2.23) serve as
extreme directions of j. Specifically,
2.40. THEOREM. Given the polyhedral convex set
j = {XIAX = b, X ~ O,XfRn}, let A be of order (m x n) with
p( A) = m. A recession direction dj €!I' is also an extreme direction if

and only if dj is a positive scalar multiple of the n-vector
[ -B-llj] ~ 0
e· -'
J
where B-1lj ~ 0 and lj is the jth (nonbasic) column of R.
It is easily demonstrated that
2.41. COROLLARY. A convex polyhedron f = {XIAX = h, X ~ 0, X t Rn}
has a finite number of extreme directions.
We next examine the conditions under which extreme directions exist.
Specifically, we look to the following
2.42. EXISTENCE THEOREM. The convex polyhedron f = {XIAX = h,
X~O,XtRn}, where A is (mxn) and p(A)=m, has at least one
extreme direction if and only if it is unbounded.
In some of the theorems to follow we shall have occasion to invoke the
assumption that certain point sets are bounded. To appropriately characterize
this concept let us now examine a couple of unboundedness criteria. Specifically,
2.43. THEOREM. (a) A convex polyhedron f = {XIAX ~ h, XtR n } in R n
is unbounded if and only if it admits a half-line or (b) if and only if
the polyhedral convex cone e= {XIAX ~ O,XERn } contains a non-
null vector.
2.14. Combined Extreme Point and Extreme

Direction Representation for Polyhedral Convex Sets
In this section we shall see that a point within a convex polyhedron f in R n
can be represented as the sum of a convex combination of a finite number of
extreme points from f and a nonnegative linear combination of a finite number
of extreme directions for f, i.e., any Xtf is representable as the sum of the
convex hull of a finite number of extreme points and the conical hull of a finite
collection of extreme directions, all taken from f. To this end we have
2.44. REPRESENTATION THEOREM [Minkowski (1910)]. Given a non-
empty polyhedral convex set f = {XIAX ~ h, XtR n }, let A be of
order (mxn) with p(A) = n. In addition, let {Xl, ...,Xk }, {dl, ... ,d,}
be finite sets of extreme points and extreme directions for :f
respecti vely. Then
k 1 k
:f = { XIX = I: .\Xj + I: J.ljdj , I: '\ = 1,
i=1 j=1 i=1
o~ AoER 10< all;; 0 ~ PiER 10< all i}

= co{Xl, ... ,Xk } +coni{dl,···,d, }.
We note briefly that the converse of this theorem IS also true, i.e., every
co{Xl , ... , X k } + coni{dl ,. .. , d, } for finite sets {Xl' ... ' X k }, {d l , ... , d,} is a convex
polyhedron.
2.15. Resolution of Convex Polyhedra

We next consider the notion that every polyhedral convex set :f in R n is
decomposable into the sum of a bounded convex polyhedron and a polyhedral
convex cone. To set the stage for this resolution of polyhedra, let us first consider
a supporting definition. Specifically, a pair of finite point sets %1 = {Xl" .. ' X k },
%2 = {Xk +l ,···, Xm} is termed a finite basis for a polyhedron :f in R n if
:f = cO(%l) + coni(%2)' i.e., if X €:f, then X has the representation X =
Ef=l,\Xj + E?:!:k+1 .\Xj, Ef=l '\ = 1, 0 ~ .\ f R, i=I, ... , m.
The resolution theorem for polyhedra to be presented below involves a
statement of the so-called
2.45. FINITE BASIS THEOREM. Every convex polyhedron III R n has a
finite basis.
In view of these considerations we now look to the following
2.46. RESOLUTION THEOREM FOR CONVEX POLYHEDRA [Motzkin
(1936); Goldman (1956)]. A polyhedral convex set
:f = {XIAX ~ b, XfRn} :f ¢ can be expressed as the (vector) sum
co{Xl, ... ,Xk } +coni{Xk+l'···'Xm } =
{XIX= Y+Z, Y €CO{X l , .. .,Xk }, Z€coni{X k+1' ..•'X m }} =

{XIX = Ef=l\Xi+E~k+1AiXi' Ef=lAi = 1,0::; \€R,i=l, ... ,m}
(2.24)
of a polytope (bounded convex polyhedron) co{X l , ... , X k } and a poly-
hedral convex cone coni{X k+l , .. .,X m }. Conversely, any set :f f. </J of
the form co{Xl , ... , Xd + coni{X k+l , .. .,Xm } is a polyhedral convex set.
By virtue of this theorem, a set :f of vectors in R n is a convex polyhedron if
and only if :f =:f l + :f2, where :fl is a convex polytope and :f2 is a polyhedral
convex cone.
It is important to note that in any decomposition such as (2.24),
coni{Xk+l' ...'X m } = {XIAX ~ O,X€Rn}.
Two immediate consequences of theorem 2.46 are:
2.47. COROLLARY [Goldman (1956)]. Let:f = {XIAX~ b,X€Rn} =/:</J be
a polyhedral convex set with p(A) = n. Then:f has a minimal basis
{Xl' .. .,Xk; X k+1' •.. 'X m } which is unique (up to positive multiples of
the Xi for i = k+1, ... ,m). Here {X1, ... ,Xk } is the set of extreme
vectors of :f.
2.48. COROLLARY [Goldman (1956)].
A polyhedral convex set :f = {XIAX ~ b, X € Rn} f. ¢ is bounded if and
only if coni{X k +1' ...'X m } = {X I AX ~ O} = {OJ.
Theorem 2.46 also provides the foundation for the assertion of a finite basis
theorem for polytopes. That is,
2.49. THEOREM. A polyhedral convex set :f = {XIAX ~ b, X € Rn} =/: 0 is a
polytope if and only if :f is bounded.
2.16. Simplexes
We noted above that a set :f ill R n is termed a convex polytope if it
is expressible as the convex hull of finitely many points {Xl' ... ' X k } or
:f = co{X l ,. .. ,Xk }. It was also mentioned that a set of k + 1 vectors
{Xo, Xl'···' X k } in R n is affinely independent if and only if the translated set
{Xl - Xo,···, X k - Xo} is linearly independent or P[XI - Xo,···, Xk - Xol = k.

Upon combining these definitions we may now state that a convex polytope :f in
R n is a k-dimensional simplex (or simply a k-simplex) if and only if it is the
convex hull of a set of k+ 1 affinely independent vectors {Xo, Xl"'" X k },
k :S n+ 1. A k-simplex spanned by the points {Xo, Xl" .. , X k } in R n will be
denoted as
o:S \ ER for all i} . 4 (2.25)
In general, dim((Jk) = number of vertices minus 1 =k since the vertices of (Jk

determine a k-plane or affine subspace of dimension k, i.e., the vertices lie on no
plane of dimension k-l. Clearly the k+l vertices of a k-simplex are extreme
points of the same. Hence we may view a k-simplex as a convex polytope having
exactly k+l extreme points. We note briefly that a subset of (Jk is said to be a
subsimplex of (Jk if it is itself a simplex.
Based upon the preceding definition of a k-simplex we may now state that
if the set of vectors {Xl'" .,X n } in R n is linearly independent, then the convex
hull of {O, Xl' ... , X,J is an n-simplex. In particular, the standard n-simplex in
R n is the convex hull of {O,el, ... ,en}, where ei,i = 1, ... ,n, is the ith unit column
vector. Moreover, an n-dimensional unit simplex in R n +l is formed as the
convex hull of the set of n + 1 unit column vectors {el , ... , en+l} and appears as
(In = {X I~t: Xi = 1, O:S Xi E R for all i}-

The boundary of a k-dimensional simplex contains simplexes of lower
dimension which are generally called simplicial faces or just faces_ The number
of such faces of dimension eis
4Some authors define a k-simplex as the set of all strictly positive (Ai> 0) convex
combinations of a collection of k+l affinely independent points. Clearly any such set must be
open. Equation (2.25) is then used to define the closure of an (open) k-simplex and
subsequently called a closed k-simplex.
( k+l) (k+l)!
€+1 - (€+I)!(k-€)!·
Specifically, an €-simplex u€ is an €-face of a k-simplex uk if all vertices of u€ are

also vertices of uk. Here the vertices of u€ form a subset of the vertices of uk
and the convex hull of this subset is the resulting face. If € = 0, u f = u O is a
vertex of uk; and if f = k - 1, u f is a specialized face which is called a lacet of
uk. In this latter case there is exactly one vertex in uk which is not in u f . If
this vertex is denoted as Xi for some 0 ~ i ~ k, then we may call u f the lace
(Iacet) 01 uk opposite the vertex Xi and refer to it as a (k-l) - simplex. For X =
~~=O,\XiiCO{Xo,Xl, ...,Xk}' let index set 3(X) = {i I Ai > 0, 0 ~ i ~ k}
={io, ... ,i€}. Then X€U f , where the face of u f is termed the camer lace of X
and is formed as co{Xio ' ... ' Xi,}. In fact, X belongs to exactly one face of uk
and amounts to the face of lowest dimension of uk containing X. The union of
all faces of uk is its closure.
2.17. Linear Functionals

A linear functional on R n is a function I: Rn_R which is additive and
homogeneous, i.e., for points X!,X2 €R n and a scalar A€R, I must satisfy:
(a) I(X l +X2 ) = I(X!) + I(X 2 );
(b) I(AX!) = veX!).
The value of a linear functional I at a point X in R n will be represented as the
scalar product I(X) = C'X. (In fact, if I is a linear functional defined on R n,
then it can be shown that there exists a point C€ R n such that I(X) = C'X for
all X€R n.) Hence I: Rn_R maps C 1-+ C'X so that every linear functional on
R n can be represented by a vector C€R n. For I i= 0, we may depict the linear
functional as hyperplane % ={X II(X) = c'x = 0', 0' € R} in R n, where the
vector C( #=0) is a normal to the hyperplane. It can be shown that a linear
functional or hyperplane is continuous and closed.
We noted above that a set :I in R n is compact if it is closed and bounded.
The importance of compact sets for our purposes is that compactness is preserved
by continuous mappings, i.e., if the domain '1 of a continuous real-valued function

f: 'i-+R is compact, then so is its range. In this regard, a continuous real-valued
function is bounded on a compact set. Moreover, the least upper bound and the
greatest lower bound are themselves members of the range of f. That is, there
exist points Xl' X 2 E '1 such that
sup f(X) maxf(X)j

Xd Xf'i
f(X 2 ) = inf f(X) = min f(X).

Xd Xd
These remarks may be summarized by a sufficient condition for the existence of
extreme values of a function, namely
2.50. WEIERSTRASS'S THEOREM. Let the real-valued function
y = f(X) be continuous over a (nonempty) compact set '1 in Rn.
Then f assumes a least and a greatest value at least once over 'i.
A particularization or stronger version of this result is provided by the following
2.51. THEOREM. Let f: 'i-+R be a linear functional defined on a
convex polytope '1 in Rn. Then there exists extreme points X, Y E '1
such that
f(X) = max f(X)j f(Y) = min f(X).
Xf'i Xd
PROOF. Since f is continuous and '1 is compact, Weierstrass's
theorem guarantees the existence of a point XE '1 such that
f(X) = max f(X).
Xd
Let {Xl' ... ' X k } depict the set of extreme points of 'i. Then according to
representation theorem 2.39,
k k
~
X = I: i=I'\X i , I: i=l'\ = 1, 0 ::; '\ E R, i=I, ... , k.
If none of the extreme points satisfy f(X i ) = f(X), then we must have
f(X i ) < f(X) for all i and thus
k k
= I: i=l >.J(Xi ) < I: i=1 \f(X)
~ ~
f(X) f(X).
This contradiction thus implies that some extreme point X E '1 must satisfy
f(X) = f(X). A similar demonstration holds for Y. Q.E.D.
We shall see in the next chapter that the region of admissible solutions for a
linear programming problem is a convex polyhedron % = {X I AX ~ b, X ~ O} in
RR (if % is bounded, it is a convex polytope). In this regard we have
2.52. THEOREM. Let % be a convex polytope (containing no extreme
directions) and let the linear functional f: RR--+R be bounded above
on %. Then there exists an extreme point X € % such that
f(X) = sup f(X) = sup f(X).

Xe% Xe8%
The implication of this theorem is that if the linear programming objective

function f(X) = C'X (a linear functional) is bounded above on %(=1=4», then the
linear program has at least one optimal solution. Furthermore, at least one of
the optimal solutions is an extreme point X of % with f = C'X serving as a
supporting hyperplane of % at X.
INTRODUCTION TO
3 LINEAR PROGRAMMING
3.1. The Ca.nonical Form of a. Linear Progra.mming Problem

We may write the basic structure of a linear programming problem in what
is called canonical form as: maximize a linear functional or hyperplane
(3.1)
subject to (hereafter abbreviated as "s.t.") a set of m structural constraints

(3.2)
and p non-negativity conditions Xj 2: O,j=l, ...,p. If C, XERP, bERm, and A is

of order (m x p), then the linear programming problem may be rewritten in
matrix form as
maxf(X) = C'X s.t. (3.1.1 )
AX~b, X~O, (3.2.1 )
where
Here (3.1) (or (3.1.1» is termed the objective function (the expression to be
maximized) and, taken altogether, the structural constraints and non-negativity
conditions will be referred to as the constraint system. (Note that the constraint
system is made up of a set of m+p half-planes.) Such a system forms the
feasible region % C RP, i.e., the area within which an admissible solution to the
69
problem lies. Thus X is formed as the intersection of the solution sets of the
constraint system or
X ={X IAX ~ b, X ~ O}
=( n?::I{XI~j=laijXj~bi}) n (n~=I{Xlxj~O}).
Clearly 9G is a convex polyhedron since it is the intersection of a finite number of
closed half-spaces. The linear programming problem is termed feasible whenever
Any X which satisfies (3.2) (or (3.2.1)) is called a solution to the linear
programming problem while if such an X also satisfies X ~ 0, i.e., X € X, then it
is termed a feasible solution. Moreover, if Xo is feasible and !(Xo) ~ !(X), where
X is any other feasible solution, then Xo is termed an optimal (i.e., maximal)
solution.
Two additional points are in order. First, the linear programming problem
has been written in terms of maximizing a hyperplane subject to a set of
structural constraints and non-negativity conditions. If we are faced with
minimizing the same, then we may utilize a transformation which converts the
minimization problem into an equivalent maximization problem, z.e.,
min! = - max{ - f} (see Chapter 9 for further details.) Next, in our subsequent
analysis we shall assume that each bi ~ 0, i=I, ... , m. If it happens that bi < 0 for
some value of i, then both sides of the structural constraint to which it
corresponds may be multiplied by -1 so as to fulfill this requirement.
3.2. A Graphical Solution to the Linear Programming Problem

The problem that we wish to solve is:
max! = Xl + 4x2 S.t.
~xI + x 2 ~ 50
Xl - 3x 2 ~ 15
Xl + x 2 ~ 30
2xI - x 2 ~ 10
xI,x2~0.
Introduction to Linear Programming 71
Let us first illustrate the region of feasible solutions %. Since the structural
constraints and non-negativity conditions must all be satisfied simultaneously, it
is obvious that 9G is simply the intersection of the solution sets of the constraint
system (Figure 3.1).
50
30
~~----~--~------------------------~~~XI
5 15 30 100
Figure 3.1
A maximal solution to the linear programming problem will be obtained by

finding the point (or points) in 9G which yields the largest value of the objective
function. For any fixed value of / we obtain the straight line / = xl + 4x2 and,
by choosing different values of /, we generate a series of parallel lines. Such a
series is indicated in Figure 3.1. For / = l any point on the segment ee' yields
a solution which is feasible but not optimal because, by sliding this line upwards
and parallel to itself over 9G, we obtain a larger value of / with feasibility still
maintained. Clearly / = /2 maximizes the objective function since the solution
obtained at point A is feasible and /2 is greater than the value of / at any other
point in 9G. Although / = /3 yields a value of the objective function which is
greater than /2, the requirement that / have at least one point in common with
X is not met. Hence any point on the line corresponding to / = /3 is
inadmissible.
Since the lines 2XI - X 2 = 10 and !x I + x 2 = 50 intersect at point A, the
simultaneous solution of these two equations yields the optimal values of xl and
x 2' namely x~ = 24, xg = 38. Hence the maximum value of / is /0 = 24 + 4 . 38
= 176.
In general, at an optimal feasible solution to the linear programming
problem, the hyperplane / cannot contain any interior point of X and must
contain at least one boundary point of X, i.e., it is a supporting hyperplane of X.
In this light, the optimal solution to a linear program must occur at an extreme
point of X or at some convex combination of extreme points from X.
A formalization of these observations along with a computational technique
for generating maximal extreme point solutions is presented below.
3.3. The Standard Form of a Linear Programming Problem

Instead of working with the "~" structural constraints as a set of
inequalities, we may conveniently transform them to a set of equalities so as to
obtain the augmented structural constraint system. This is accomplished by
introducing m new variables into system (3.2). In this regard, if we add a
nonnegative variable to the left-hand side of each "<" structural constraint,
then we admit m slack variables of the form
(3.3)
In the light of (3.3), the linear programming problem in standard form involves
maximizing the augmented objective function
/(X I ,. .. , x P' x p +l '···, x p +m )

= clx l + ... + cpxp + OXp+I + ... + OX p +m (3.4)
subject to
(3.5)
and x j 2 0, j=I, ... , p, p+ 1, ... , p+m = n. If e, X€ RP+m = R n and Ais of order

(m x n), then the preceding problem is expressible in matrix form as
maxf(X) = e/x s.t. (3.4.1 )

AX= b, X~ 0, (3.5.1)
where
ci xl
xp
=[~J.
Cp
e= X= =[:J
Cp +l xP+1
Cp +m +m
Xp
an alp 1 0 0
a21 a2p 0 1 0
A= [A,lm],
amI ... amp 0 0 1
and Xs is an (m x 1) vector of slack variables xp+i' i=I, ... , m.

Given the augmented problem in standard form as depicted by equations
(3.4.1) and (3.5.1), the augmented feasible region may be represented as
%={XIAX = b, X~ 0 }.
As was the case with %, region % is also a convex polyhedron since it can
equivalently be expressed as
% = {X IAX ~ b, - AX ~ - h, X~ 0 }.
The linear program in standard form is said to be feasible if %f. <p.
What is the connection between % and %? Given any feasible solution to
AX = h, the addition of m non-negative slack variables uniquely determines a
feasible solution to AX = b. Conversely, for any feasible solution to AX = b, its

first p components yield a feasible solution to AX = b. This is because all
vectors in%lie in a p-dimensional hyperplane embedded in R n so that the image
of any vector Xf %projected onto the hyperplane has as its components the first
p components of X. Thus there exists a one-to-one correspondence between
points in % and %.
To formalize this correspondence let us define a function ,: %-+% as
,(X) = X, where Xf% and Xf%. Here ,: (a) is one-to-one; (b) preserves convex
combinations of vectors; and (c) preserves extreme points. That is:
(a) Choose Xl' X2f% with ,(Xl) =,(X2). Then
Hence Xl = X2 and, is one-to-one.

(b) Choose Xl' X2 f % with Xc = 8X I + (1-8)X2' 0 :S 8 :S 1. Then
Xc] [ 8Xl + (1-8)X2 ]
,(Xc) = [b-AXc = 8b+(1-8)b-8AXI -(1-8)AX2
=8 [b_XlxJ+ (1-8) [b_XlxJ = 8,(XI ) + (1- 8h X2).

Thus, preserves convex combinations.
(c) Let {Xl'"'' Xe} depict the set of extreme points of %. Then by
theorem 2.39,
% =co{XI ,· .., Xe}

= {X IX = ~~=IAjXj' ~~=IAj = 1, 0 :S AlR for all j}.
From property (b), , must preserve convex combinations of extreme points or
where Xj is an extreme point of

Thus extreme points in $ are in one-to-one correspondence with extreme points

in 9G.
What is the relationship between the original objective function (3.1) and
the augmented one (3.4)? Since the coefficients on the slack variables in (3.4) are
all zero, the original objective function is unchanged, i.e., I(X) = I(X). In this
regard, ifXb = (~, 0') optimizes IX) subject to (3.5.1), then the first p
components of Xo optimize I(X) subject to (3.2.1) with the result that
I(Xo) = I(Xo)·
By virtue of the aforementioned one-to-one correspondence between points
in $ and 9G and the fact that I(X) = I(X), we will, for the sake of notational
efficiently, adopt the convention of expressing a linear program as either
max I(X) = C'X s.t. } {maXI(X) = C'X s.t.
X€$ = {XIAX~ b,X~ O} C RP or X€$ = {XIAX = b,X~ O} eRn.

In the former case C, X € RP and A is of order (m x p) while in the latter instance
it is to be understood that C, X€R n and A is of order (m x n). The form actually
selected will be determined by the requirements of the problem at hand.
3.4. Properties of the Feasible Region

We noted in Chapter 2 that a closed half-plane is a closed convex set. It
was also indicated there that the intersection of a finite number of closed
(convex) sets is itself closed (convex). Since $ = {X I AX = h, X ~ O} C R n is
the intersection of the 2m + p solution sets of the constraint system, it is obvious
that $ is a convex polyhedron, i.e., $ is a closed convex set whose boundary
consists of, at most, a finite number of edges and thus a finite number of extreme
points.
In addition, since the non-negativity conditions span only the positive
orthant of R n , it is evident that $ is bounded from below. If it is also bounded
from above (i.e., there exists a non-negative vector V € R n with finite components
such that X ~ V for all X€ $), then $ is strictly bounded and thus a convex
polytope, i.e., $ is a strictly-bounded, closed, and convex set with a finite
number of extreme points (e.g., see Figure 3.1). Hence % can be written as the
convex hull of its set of extreme points. In this regard, if the set of extreme
points of % appears as {X1, ...,Xk }, then by virtue of theorem 2.39,
% = co{X1,···,Xk } = {Xl X = E~=l OjX j ,

E~=l OJ = 1,0::; 0j€R,j=I, ...,k}.
That is, each X€% is expressible as a convex combination of the set of extreme
points of %.
However, if % is not strictly bounded, then % is not a convex polytope and
thus, as determined by representation theorem 2.44, % has a finite basis, i.e., %
is expressible as the sum of the convex hull of a finite number of extreme points
and the conical hull of a finite collection of extreme directions, all taken from %.
If the set of extreme points of % again appears as {X1' ... 'Xk } and {dl, ... ,d e}
depicts the set of extreme directions in %, then if A is of order (m x n) with
peA) = n,
% = co{X1,···, X k } + coni{ d l ,···, de}
={xlx= E~=IOjXj+ E~=ljjrdr' E~=IOj= 1,

0::; Oj€R,j=l, ... ,kj 0::; jjr€R, r=I, ... ,e}.
Thus any X€ % can be represented as the sum of a convex combination of a finite
number of extreme points from % and a non-negative linear combination of a
finite number of extreme directions for %.
3.5. Existence and Location of Finite Optimal Solutions

In this section we shall exhibit the significance of the notion that % is a
convex polyhedron along with a collection of important theorems pertaining to
the existence and location of optimal solutions. We first look to a necessary and
sufficient condition for the existence of a finite optimal (maximal) solution. To
this end we state a general
3.1. EXISTENCE THEOREM. For the linear program maxi = C'X

s.t. XdG = {X IAX = h, X~ 0, X ERn} f. <P, let p(A) =m and let
{Xl, ... ,Xk }, {dl, ... ,d e} depict sets of extreme points and extreme
directions for 9G respectively. Then I attains a finite maximum at an
extreme point Xr if and only if C'd j ~ 0, j=I, ..., e. (The angle
between C and each d j must be nonobtuse.) If I assumes its finite
maximum at more than one extreme point, then it takes on the same
value at each convex combination of those extreme points.
PROOF. By representation theorem 2.44, XE 9G if and only if
X = L:~=l 0iXi+ L:!=lJ-ljdj , L:~=l 0i = 1, 0 ~ 0iER,

i=I, ... ,kj 0 ~ J-Ij ER, j=I, ... ,e.
Hence we may rewrite the linear program as
maxi = C'X = L: ~=l 0iC'X i + L: !=lJ-ljC'dj s.t.
L:~=l 0i = 1
0i ~ 0, i=I, ... ,k
J-Ij ~ 0, i=l, ... ,e.
> 0 for some 1 ~ j ~ e, then 1-+ + 00 as J-Ij-+ + 00 so that I is unbounded

If C'd j
from above. Hence I attains a finite maximal solution if and only if C'd j ~ 0,
j=I, ... , e. Let C'd j ~ 0 for all j and set J-Ij = 0 so that the above problem
becomes
,,~ 0;=1,0; > O,i=I, ... ,k.
L.J 1=1' • -
If C'X r = max C'Xi , then replacing each C'Xi by C'X r in the preceding
l:5 i :5 k
1~ i ~ k objective function does not decrease this sum so that
Hence I takes on its finite maximum at an extreme point of 9G.

Next, let I attain its finite maximum at more than one extreme point, e.g.,
,
for extreme points Xl"'" X s' s < k, assume C Xl = ... = C "Xs = C X r . Let X
-
be any convex combination of Xl"",Xs' i.e.,

Then
CX
,-
= L: i=l () i C Xi =C Xr L: i=l ()i =C Xr
B -, , B - ,
since C'Xr = C'Xi , i=l, ... , s, and thus the second part of the theorem is verified.
Q.E.D.
This theorem informs us that if X is a convex polyhedron and C'd j ~ 0 for
each extreme direction d j , j=l, ... , e, for X, then f takes on its finite maximum at
an extreme point of X or at some convex combination of extreme points taken
from !!G. However, if X is a convex polytope (a bounded convex polyhedron),
then f is indeed bounded from above on !!G and thus the existence of a finite
maximum of f at an extreme point of X follows immediately from theorem 2.52 .
As Figure 3.2.a reveals, the maximum of f occurs at a unique extreme point A
while in Figure 3.2.b, f attains the same maximal value at extreme points A, B,
and at every convex combination C of A, B.
mazf=f(A)=f(B)=f(C),
mazf = f(A) C = ()A + (l-()B,
O~()~l
1---o<---L--..6._~.L--'-- _ _---+ zl
a unique optimal solution
. Ie optima
mu Itip . I soIutlOns
. zl
a. b.
Figure 3.2
The above example problem (Figure 3.1 of section 3.2) dealt with the case
where 3G was a convex polytope. However, if 3G is not bounded, then the notion
of whether or not the linear program attains a finite maximum depends upon the
characteristics of the I hyperplane itself. For instance, let us
maxI = - ~X1 + 3x 2 s.t.
- Xl + x 2 :S 1
-~x1 + x 2 :S 3
-~xl + x 2 :S 5
x 1 ,x 2 ~ o.
Here 3G is a convex polyhedron that is unbounded (Figure 3.3). Moreover, 3G has
four extreme points
and two extreme directions that may be constructed as
Hence a point X € 3G can be represented generally as

X = co{X 1 ,X2 ,X3 ,X4 } + coni{d1 ,d2 }.
In particular, X7 € 3G can be expressed as
X7 = 0l X1 + 04 X4 + Jl2 d 2,01 + 04 = 1, 01'04 ~ 0; Jl2 ~ o.
Since C'X 1 = 0, C'X 2 = 3, C'X3 = 9, C'X4 = 9, C'd l = _125, and C'd 2 = -~,
our problem is equivalent to
maxI = 302 + 903 + 9°4 - I;Jl1 - ~Jl2 s.t.
01 + 02 + 03 + °4 = 1
01' ...'04 ~ 0; Jl1,Jl2 ~ o.
With both C'd 1 , C'd2 :S 0, set Jll = Jl2 = O. Then to maximize 1= 302 + 90 3 + 904
subject to the indicated constraints we may, for instance, set 01 = 02 = 03 = 0
and let ° = 1.
4 Hence the optimal extreme point is X4 • Are there other options?
The answer is yes since any point along the entire edge from X3 to X4 serves as a
maximal point with the maximum of I being 9. However, if the objective

I = xl + x 2' it is easily demonstrated that no finite
hyperplane were changed to
maximum exists since the value of I may be increased without bound with
feasibility always maintained.
To see this we first find C'XI = 0, C'X2 = 1, C'X3 = 9, C'X4 = 21, C'dl = 5,
and C'd2 = 4. Hence an equivalent version of the original problem is
maxi = (}2 + 9(}3 + 21(}4 + 5/l l + 4/l2 s.t.
(}l + (}2 + (}3 + ()4 = 1
(}1'''''(}4 ~ 0; /l1,/l2 ~ O.
Since, say, C'd2 > 0, /l2 can be made arbitrarily large without violating the
feasibility of this problem. Hence 1...... 00 as /l2 ...... 00 (with /ll = 0, (}l = 1, and
(}2 = (}3 = (}4 = 0) so that no finite maximum exists.
Figure 3.3
The importance of existence theorem 3.1 is that it drastically limits the size
of the set of points within % which are possible candidates for an optimal
solution. That is to say, instead of examining all points along the boundary of 9G
in our search for a finite maximal solution, we need only look to the set of
extreme points for it is among these points that the optimal solution must be
found. And since there are at most a finite number of such points, we may be
assured that only a finite number of extreme point solutions need be examined in
our search for a maximum of I. An iterative method of search which does in fact
proceed to the maximum of I in a finite number of steps is the simplex method,
which will be discussed in subsequent chapters.
We close this section by noting briefly that:
(a) if 9G = </J, the linear programming problem has no solution;
(b) if 9G is bounded (9G is a convex polytope), the linear program has a
finite optimal solution which occurs at an extreme point or at some
convex combination of extreme points; and
(c) if 9G is unbounded, the linear programming problem has a solution that
may be unbounded.
3.6. Basic Feasible and Extreme Point Solutions

to the Linear Programming Problem
Because of the importance of the connection between extreme points and
basic feasible solutions to a linear program, let us review our earlier discussion of
extreme point solutions as basic solutions to linear equalities which was presented
in Chapter 2. In what follows we shall work with the linear programming
problem in standard form or
maxI = C'X s.t.
AX= b, X;:::: 0,
where X€R n and A is of order (mxn).

As noted in section 2.10, if the (augmented structural constraint) system
AX = b, p(A) = m, has a feasible solution X;:::: 0, then it also possesses a basic
feasible solution. Moreover, it was also mentioned that a vector XfRn is a basic
feasible solution to AX = b if and only if X is an extreme point of a convex
polyhedron (taken herein as 9G). These observations are of paramount impor-

tance since they inform us that we may undertake a search for an optimal
extreme point solution to a linear programming problem by seeking only basic
feasible solutions. In this regard, if we can find a set of m linearly independent
vectors within A and arrive at a basic feasible solution to AX = b, it must yield
an extreme point of 9G, i.e., extreme point solutions are always generated by m
linearly independent columns from A.
At the outset of our search for basic feasible solutions we shall impose the
restriction that p(A) = p[A, b]. If such an assumption is not made, then AX =b
represents an inconsistent system of equations and we are thus precluded from
obtaining a solution to the linear programming problem. Since AX = b is com-
prised of m equations in n unknowns with m < n, it is clear that the augmented
structural constraints represent an underdetermined system of simultaneous linear
equations. For such a system an infinite number of particular solutions exists if
it is consistent. In this regard, if we further assume that p(A) = m, so that none
of the augmented structural constraints is redundant (i. e., a particular structural
constraint cannot be expressed as a linear combination of any others), we may
select m linearly independent columns from A and uniquely express the variables
associated with these columns in terms of the remaining n - m variables. If
arbitrary values are then assigned to the latter, it is evident that an infinity of
solutions ensues. More formally, let us rearrange the columns of A so that the
first m are linearly independent and constitute the columns of the (m x m) matrix
B = [bl , ..., b m ]. The remaining n - m columns of A then form the (m x n-m)
matrix R = [fl , ... ,rn _ m ]. Hence AX = b may be written as
AX = [B, R] [i:]
=BX B + RXR = b,
where X has been partitioned into an (mxl) matrix XB,X B= (xBl"",xBm)'
containing the m variables corresponding to the m linearly independent columns
of A and an (n-m xl) matrix X R' Xn = (x Rl' ... , x R, n-m)' containing the
n - m remaining variables. Since p(A) = m, B is nonsingular. Hence
X B = B-lb- B-lRXR (3.6)
and all solutions to the augmented structural constraint system can be generated
by assigning arbitrary values to X R . In particular, if X R = 0,
and we obtain a basic solution to AX = b, namely X' = (XB' 0'), since the
columns of B constitute a basis for R m , i.e., each column of R and the vector b
are expressible as a linear combination of the linearly independent columns of B.
Hence the variables within X B are said to be basic variables while those within
X R are termed nonbasic variables.
To summarize: given the augmented structural constraint system AX =b
with p(A) = p[A, b] = m, if any m linearly independent columns are chosen from
A and if all the remaining n - m variables not associated with these columns are
set equal to zero, the resulting solution X' = (XB,O') contains no more than m
nonzero variables and is a basic solution to AX = b. Furthermore, if X B ~ 0 so
that X 2: 0, then X is termed a basic feasible solution to AX = b and corresponds
to an extreme point of %. That is, X is an extreme point of % if and only if
X' = (B-lb,O)' ~ 0'. Finally, a basic solution (feasible or otherwise) to AX =b
is said to be degenerate if one or more of the basic variables within X B vanishes,
i.e., X contains less than m positive components.
EXAMPLE 3.1. Find a basic solution to
Xl + 2x2 + x3 + x 4 = 1
2xl - 2x2 - x3 + x 4 = 1.
Here
A - [1 2 1 1] X _
- 2-2-11'
r:~]
-l::' and b=U ].
Since p(A) = p[A, b) = 2, the equations are consistent and nonredundant. Hence
we may extract from A two linearly independent columns so as to form the
nonsingular basis matrix B. Let us try columns two and three. Since
1_; -~ 1= 0,
these columns are obviously not linearly independent so that B- 1 does not exist.
Let us now try columns one and three. Since
I ; -~ I = -3 # 0,
we may now form
B=[~ _~}B-l=[i_n
R=[ _~ : } XB=[::} and XR=[~:l
We noted above that to obtain a basic solution to AX = b we need only solve for
the basic variables in terms of the non basic variables, and then set the nonbasic
variables equal to zero. Proceeding in this fashion yields, from (3.6),
[::] [
2
3"
1
3"
If X R = 0, X B= ~, 1) so that a basic solution to the above system is

X, = (~,l,o,o). Note that since no component of X B is zero and X 2 0, we
actually have obtained a non degenerate basic feasible solution.
3.7. Solutions and Requirements Spaces

We may now examine two types of Euclidian spaces commonly connected
with a linear programming problem. In doing so we shall shed some additional
light on the significance of the nonredundancy assumption that p(A) = m. It was
noted above that the augmented feasible region 9b lies within Rn. That part of
R n which contains 9b will be called the solutions space of the constrained
equation system AX = b, X ~ 0, XfRn. If a vector X satisfies AX = b, then X
lies within the intersection of a set of hyperplanes which comprise the rows of
[A, b]. If p(A) = m, then the said intersection cannot be formed by less than m
hyperplanes, i.e., anyone hyperplane cannot be expressed as a linear combination
of the others.
The second space in question will be referred to as requirements space. The
elements of this space are the columns of [A, b], where b is termed the
requirements vector. Since the columns of [A, b] represent a set of (m xl)
vectors, it is obvious that requirements space is a subset of Rm. If aj denotes the
lh column of A, then
n
AX = L ajx j = b.
j=1
Since a feasible solution to AX = b obtains when b is expressible as a nonnegative
linear combination of the columns of A, b must be an element of the convex
polyhedral cone generated by the columns of A. Hence that part of R m which
contains the convex polyhedral cone
e={YIY=AX= t J=1
ajxj'X~ o}
is requirements space. Now if p(A) = m, the columns of A yield a basis for R m
so that b cannot be expressible as a nonnegative linear combination of fewer than
m of the aj' Thus the number of x j > 0 is never greater than m. In this regard,
if the m linearly independent columns of A constitute the (m x m) matrix B, then
a basic feasible solution to AX = b results when b is expressible as a nonnegative
linear combination of the columns of B so that h is an element of the convex

polyhedral cone
eB~{YIY~ [B,R{~Bl~ BXB~ %1 b;'m,XB~ oJ.

where hi is the ith column of B.
For instance, let AX = [3t, ... ,as]X = b€R n (Figure 3.4.a). Since b€e, i.e.,
b = E~=laizi' system AX = b has a feasible solution in requirements space or
9G "# <p. Moreover, h is expressible as
a.
b.
Figure 3.4
a nonnegative linear combination of ~, 3.;} alone so that eB = {YI BXB = b,

XB~ O}, where B = [~,3.;}), B
X = (xm,xm) = (x 2 ,x3 ). However, if b is redrawn
as in Figure 3.4.b, then 9G = ¢J - now AX =b is inconsistent in requirements
space.
Figure 3.1 depicted the attainment of an optimal solution to a linear
program in solutions space. Let us now illustrate the notion of optimality in
requirements space. Specifically, our problem is to: max f(X) = C'X s.t.
AX = b, X ~ O. If we treat the objective function as a m + 1st equation in the
system
or
'L...J
" i=ln [ ai
ci ] xi = 'L...J
" i=ln aixi
_ = b ]'
[f (3.7)
then we desire to find a solution to (3.7) for which each xi 2: 0, i=I, ... , n, and f
is as large as possible. Clearly the vector [j]. f maximal, must be an element of
the finite cone e spanned by the vectors ai = (ai, Ci)' i=l, ... , n.
For example, let us
maxf(X) s.t.
Xl + x2 + x3 5
xl' X 2 'X 3 2: 0,
where x3 is a slack variable. As indicated above our problem can be rewritten as:
choose xl' X 2 ' X3 2: 0 so that
and f is maximal. From Figure 3.5, it is evident that the maximal solution is a
multiple of the vector a; = (1,2) or
Hence x2 = 5 and f = f O = 2x2 = 10.

[;l
- - - set of points of the form
fa parameter.
o ~~-=--~~----------~
Figure 3.5
4 DUALITY THEORY
4.1. The Symmetric Dual

Associated with every linear programming problem is another such problem
called its dual. Quite generally, duality theory addresses itself to the study of the
connection between· two related linear programming problems, where one of
them, the primal, is a maximization (minimization) problem and the other, the
dual, is a minimization (maximization) problem. Moreover, this association is
such that the existence of an optimal solution to one of these problems
guarantees an optimal solution to the other with the result that their extreme
values are equal.
We now turn to the expression of the primal problem in canonical form and
then look to the construction of its associated dual. Specifically, if the primal
problem appears as
max f = cIx I + ... + cpxp s.t.
allx I + ... + aIpx p ~ bI
a21 x I + ... + a2p x p ~ b2
am1x 1 + ... + ampxp ~ bm

xI,· .. ,x p 2 0,
then its dual is
89
ming = bitt l+ ... + bmtt m s.t.

alltt l + ... + am1ttm 2: C1
a 12 1t l + ... + am2 ttm 2: C2
alpu l + ... + ampU m 2: Cp

Ul ' .. ·, U m 2: O.
Henceforth this form of the dual will be termed the symmetric dual, i.e., in this
instance, the primal and dual problems are said to be mutually symmetric. (A
rationalization of the structure assumed by the symmetric dual program will be
offered in the next chapter wherein we develop a necessary and sufficient
condition for the attainment of primal optimality.) In matrix form, the sym-
metric primal-dual pair of problems appears as
Primal Symmetric Dual
max J(X) = C'X s.t. min g(U) = b'U s.t.

AX:::; b, X ~ 0, X f RP or A'U~C, U~O, UfRm or
- - - -
X€9G p = {XIAX~b,X~O,XfRP} U €9G D = {UIA'U ~ C, U ~ 0, U fRm}
where X and C are of order (p xl), U and b are of order (m xl), A is of order
(m x p), and 9G p ,9G D denote the regions of admissible solutions for the primal
and dual problems respectively. In standard form, the primal and its symmetric
dual may be written as
Primal Symmetric Dual
max J(X,Xs) = C'X+O'Xs s.t. min g(U, Us) = b'U + O'U s s.t.
AX + ImXs = b A'U -IpUs = C
X,Xs ~ 0 U,Us ~ 0
where X s ' Us denote (m x 1) and (p x 1) vectors of primal slack and dual surplus
variables respectively. Here Us has as its lh component the dual surplus
variable
Duality Theory 91
(4.1)
An examination of the structure of the primal and symmetric dual problems

reveals the following salient features:
(a) if the primal problem involves maximization, then the dual problem
involves minimization and conversely;
(b) a new set of variables appears in the dual; in this regard,
(c) if there are p variables and m structural constraints m the primal
problem, there will be m variables and p structural constraints in the dual
problem and con versely. In particular, there exists a one-to-one
correspondence between the lh primal variable and the lh dual structural
constraint and between the ith dual variable and the ith primal structural
constraint;
(d) the primal objective function coefficients become the right-hand sides of
the dual structural constraints and conversely;
(e) the coefficient matrix of the dual structural constraint system is the
transpose of the coefficient matrix of the primal structural constraint
system and conversely. Th us the coefficients on the x j in the primal
problem become the coefficients of the lh dual structural constraint; and
(f) the inequality signs in the structural constraints of the primal problem are
reversed in the dual problem and conversely.
EXAMPLE 4.1. Given that the primal problem appears as
max f = 2xl + 3x 2 + 5x3 s.t.
xl + X 2 +3x 3 < 5
2xl - x2 + x3 < 6
xl + 3x 2 - 4x 3 < 8
:1:2 + x3 < 7
Xl' X2' X3 > 0,
find its dual. In light of the preceding discussion it is obvious that the symmet-
ric dual problem is
ming = 5u 1 + 6u 2 + 8u3 + 7u4 s.t.

Ul + 2u 2 +U3 > 2
u l - u2 + 3u3 + u 4 > 3
3uI + u 2 -4u3 + u4 > 5
u I ' u 2' u'3' u4 ~ o.
4.2. Unsymmetric Duals

In this section we shall develop a variety of duality relationships and forms
which obtain when the primal maximum problem involves structural constraints
ot,her than the "~" variety and variables which are unrestricted in sign. In
what follows our standard operating procedure will be to initially write the given
primal problem in canonical form and then from it obtain the symmetric dual
problem. To this end let us first assume that the primal problem appears as
max f(X) = C'X s.t.
AX=b, X~ o.
In this instance two sets of inequality structural constraints are implied by
AX = b, i.e.,
AX~ b} {AX~ b
AX~- b or -AX~-b
-
must hold simultaneously. Then the above primal problem has the canonical
form
max f(X) = C'X s.t.
The symmetric dual associated with this problem becomes

ming(W, V) = b'W - b'V s.t.
(A"-A')[~] ~ C orA'W-A'V ~ C;W,V~ 0,

where the dual variables W, V are associated with the primal structural
constraints AX ~ b, - AX ~ -b respectively. Upon combining terms this dual
Duality Theory 93
problem appears as
ming(W, V) = b'(W-V) s.t.
A'(W-V) f; Cj w, V f; O.
If U = W - V, then we ultimat.ely obtain
ming(U) = b'U s.t.
A'U f; C, U unrestricted
since if both W, V f; 0, their difference may be greater than or equal to, less than
or equal to, or exactly equal to the null vector.
Next, suppose the primal problem appears as
max f(X) = C'X s.t.
AX f; b, X f; O.
In canonical form we seek to
max f(X) = C'X s.t.
-AX ~ -b, X f; O.
The symmetric dual associated with this problem is
ming(V) = -b'V s.t.
-A'V f; C, V f; O.
If U = -V, then we finally
min g(U) = b'U s. t.
A'U f; C, U ~ O.
Lastly, if the primal problem is of the form
max f(X) = C'X s.t.
AX ~ b, X unrestricted,
then X may by expressed as the difference between two vectors Xl' X2 whose
components are all nonnegative. Hence the primal problem becomes
max f(X I ,X2 ) = C'(XC X2) s.t.
A(X I -X2 ) ~ bj XI ,X2 f; 0
or
max !(XI , X2 ) = (C', -C'{ i~] s.t.
(A, -A{i~] ~ b; X,X ~ O. I 2
The symmetric dual of this latter problem is

ming(U) = b'U s.t.
[-1;] U ~ [ _g] or A'U = C, U ~ O.

To summarize, let us assume that the primal maximum problem
contains a mixture of structural constraints. Then:
(I) If the primal variables are all nonnegative, the dual minimum problem has
the following properties:
(a) the form of the dual structural constraint system is indepen dent of
the form of the primal structural constraint system, i.e., the former
always appears as A'U ~ C.
Moreover, only the dual variables are influenced by the primal structural
constraints. Specifically:
(b) the dual variable Ur corresponding to the 7'th "<" primal structural
constraint is nonnegative;
(c) the dual variable Us corresponding to the sth "2:" primal structural
constraint is nonpositive;
(d) the dual variable ttt corresponding to the tth " = " primal structural
constraint is unrestricted in sign.
(2) If some variable x j in the primal problem is unrestricted in sign,
then the lh dual structural constraint wiII hold as an equality.
EXAMPLE 4.2. The dual of the primal problem
max! = 3x l - x 2 + x3 - x 4 s.t.
Xl + 3: 2 + 2X3 + 3x4 < 5
x3 - x 4 > 4
x l -x 2 -1
x l ,x 2,x4 2: 0, x3 unrestricted
DuaJity Theory 9.5
is
min 9 = 5u I + 4u 2 - u3 s.t.
ul + u3 2: 3
UI
2uI + u2
3uI - u2 2: -1
ul 2: 0, u2 ::; 0, u3 unrestricted.
4.3. Duality Theorems

To further enhance our understanding of the relationship between the
primal and dual linear programming problems, let us consider the following
sequence of duality theorems. First,
4.1. THEOREM. The dual of the dual is the primal problem itself.
PROOF. An equivalent representation of the dual minimum
problem is
max {-g(U) = -b/U} s.t.
- A/U ~ - C, U ~ 0.
The symmetric dual to this problem is
min h(X) = - C/X s.t.
-AX ~ -b, X~ 0
or
max { - h(X) = C/X} s.t.
AX ~ b, X ~ O. Q.E.D.
The implication of this theorem is that it is immaterial which problem is called
the dual.
Our next two theorems address themselves to the relationship between the
objective function values of the primaJ-dual pair of problems. First,
4.2. WEAK DUALITY THEOREM. If Xo is a feasible solution to the
primal maximum problem and Uo is a feasible solution to the dual
minimum problem, then C/Xo ::; b/Uo'
PROOF. Since AXo ~ b, it follows that UbAXo::; Ubb = b/Uo'
Furthermore, with A/Uo ~ C, UbA ~ C' and thus U~AXo ~ C/XO' Combining
the two results yields b/Uo 2: UbAXo 2: C/X O' Q.E.D.
This theorem shows that a feasible solution to either problem yields a bound on
the objective value of the other problem. Next,
4.3. SUFFICIENT OPTIMALITY CRITERION. If XO' Uo are feasible
solu-tions to the primal maximum and dual minimum problems
respectively and C'Xo = b'Uo, then both Xo' Uo are optimal solutions.
PROOF. Let j{ be any other feasible solution to the primal problem.
Then C'j{::s b'Uo = C'Xo and thus Xa is an optimal solution to the primal
problem. Similarly, if U is any other feasible solution to the dual problem, then
b'U ~ C'Xa = b'Uo and thus Uo is an optimal solution to the dual problem.
Q.E.D.
We now turn to the development of some background material which will
serve to set the stage for our discussion of what are commonly called fundamental
theorems of linear programming, namely the duality, existence, and complemen-
tary slackness theorems. Following Goldman and Tucker (1956), let us begin by
considering the m + p + 1 inequalities
-AX+b ~O
A'U -C ~ 0 (4.2)
- b'U + C'X ~ 0,
where the last inequality in this system has been obtained by reversing the sense
of the inequality in b'U ~ C'X (theorem 4.2). In matrix form, system (4.2) may
be written as
KY ~ 0, (4.3)
where
K = [ -b'A'0 -AC'0 -C0b] , Y = [U]

X ,
1
and K is skew-symmetric. If we replace the m + p + 1st component of Y by the

variable t, KY ~ 0 is transformed to the homogeneous system of inequalities
Duality Theory 97
(4.4)
Under what conditions will this modified combined primal-dual linear program-
ming problem possess a nonnegative solution? According to the skew-symmetric
theorem 2.7 .a, there exists a vector Y' = (U', X', 1) ~ 0' such that
KY ~ O}
or
Y+KY>O
-AX+b1 ~O (4.5)
A'U -C1 ~O (4.6)
- b'U +C'X > 0 (4.7)
U-AX+b1 > 0 (4.8)
A'U +x-C1 > 0 (4.9)
- b'U +C'X +1 > o. (4.10)
Then, with 1 :::: 0, there are two cases to consider: 1 > 0 and 1 = O.
First,
4.4. THEOREM [Goldman, Tucker (1956)]. If 1> 0, then there exist
optimal vectors X, U for the primal maximum and dual minimum
problems respectively such that C'X = b'U.
PROOF. With 1 > 0, the homogeneous inequalities (4.5) - (4.10) may
be transformed to
KY;' O} or
Y+KY>O
AX + b ~O (4.5.1)
A'U - C ~O (4.6.1)
-b'U + C'X > 0 (4.7.1)
U-AX+b> 0 (4.8.1)
A'U + X-C > 0 (4.9.1)
- b'U + C'X + 1 > 0, (4.10.1)
where
Then (4.5.1), (4.6.1) indicate that X, U are feasible, (4.7.1) and theorem 4.2
imply that e'x = b'U, and thus, by virtue of theorem 4.3, the optimality of X
and U. Q.E.D.
Next,
4.5. THEOREM [Goldman, Tucker (1956)]. If 1 =0, then one of the
following mutually exclusive cases holds: (a) at least one of the dual
problems has no feasible solution; (b) if the primal maximum problem
has a feasible solution, then its set of feasible solutions is unbounded
and e'x is unbounded from above. Dually, if the minimum problem
has a feasible solution, then its set of feasible solutions is unbounded
and h'U is unbounded from below; and (c) neither the primal nor the
dual problem has an optimal solution.
PROOF. To prove (a), let X represent a feasible solution to the primal
maximum problem. From (4.6) (with 1=0) A'O ~ 0 and thus, with X ~ 0,
X'A'O ~ O. This last inequality, along with (4.10) (1=0) and X ~ 0, yields
e'x> h'O ~ X'A'O ~ 0 (4.11)
(since when b ~ AX, b' ~ X' A' and thus b'O ~ X' A'O). If the dual minimum
problem also has a feasible solution 0, (4.5) (1=0) and 0 ~ 0 imply that
e'x :5 0'AX :5 0 (since if A'O ~ e, 0'A ~ e' and thus 0'AX ~ e'X). This last
set of inequalities clearly contradicts (4.11). In sum, it is not possible for both
the primal and dual problems to have a feasible solution when 1=0.
To prove (b) let us construct the infinite ray X* + AX ~ 0, 0 :5 A € R, where
X* represents a feasible solution to the primal maximum problem and X satisfies
(4.5)-(4.10). From (4.5) (with 1=0) we obtain AX ~o and thus, for A>O,
A(AX) ~ o. With X* feasible, AX* ~ b. Upon combining these last two inequal-
Duality Theory 99
ities it is easily seen that A(X* +AX) ~ AX* ~ b. Thus the entire infinite ray
consists of feasible solutions, indicating that as A increases without bound,
X* + AX remains feasible. Turning now to the latter part of the first assertion in
(b), with e'x > 0 (by (4.11)), it follows from the feasibility of X* that
e'(X*+AX) = e'x* + Ae'X can be made arbitrarily large by choosing an
appropriate value of A. A symmetric argument holds for the dual feasible region
and objective function. So for t = 0, whether the primal or dual problem has a
feasible solution, its objective function is unbounded.
That (c) is true follows as a direct consequence of (b). Q.E.D.
At this point two immediate corollaries to theorems 4.4, 4.5 emerge. First,
4.6. COROLLARY [Goldman, Tucker (1956)]. Either boththe primal
maximum and dual minimum problems have optimal vectors X and U
respectively with e'x = b'U, or neither has an optimal vector.
PROOF. If one of these problems has an optimal vector, then part (c)
of theorem 4.4 implies that t > 0, to wit theorem 4.3 states that both problems
have optimal vectors X, U with e'x = b'U. Q.E.D.
In addition,
4.7. COROLLARY [Goldman, Tucker (1956)]. One (and thus both) of the
primal maximum and dual minimum problems has an optimal solu-
tion if and only if either e'x or b'U is bounded on its associated
nonempty feasible region.
PROOF. (necessity) If X (U) represents an optimal solution to the
primal maximum (dual minimum) problem, then clearly C'X (b'U) cannot be
unbounded over %p::f. ¢ (%D ::f. ¢). (sufficiency) If the primal maximum (dual
minimum) problem is such that C'X (b'U) is bounded over %p::f. ¢ (%D::f. ¢),
then part (a) of theorem 4.5 implies that t> 0 so that by theorem 4.4, both
problems have optimal vectors. Q.E.D.
Having determined the conditions under which (4.4) possesses a nonnegative
solution, let us now look to a discussion of the first two aforementioned
fundamental theorems of linear programming. First,
4.8. STRONG DUALITY THEOREM [Goldman, Tucker (1956)]. A feas-

X to the primal maximum problem is optimal if and only
ible solution
if the dual minimum problem has a feasible solution U with
b'U = e/x. (Likewise, a feasible solution U is optimal if and only if
a feasible X exists for which e/x = b/U.)
PROOF. (necessity) If X is optimal, then by part (c) of theorem 4.5,
t > O. With t > 0, theorem 4.4 insures us that the dual problem also has an
optimal solution U such that b/U = e/x. (suffi ciency) If U is a feasible
solution to the dual problem and b/U = e/x, then theorem 4.3 indicates that U,
X are optimal. The proof of the parenthetical statement parallels that of the
first. Q.E.D.
The importance of this theorem is that it allows us to conclude, for
instance, that:
if X is an optimal solution for the primal problem, then there exists an
optimal solution Ufor the dual problem and r = e/x = b/U = ii.
Conversely,
if U is an optimal solution for the dual problem, then there exists an
optimal solution X for the primal problem and again r = i.
Looked at in another fashion, the sufficient optimality condition 4.3
established the result that if a pair of feasible solutions to the primal and dual
problems render equal objective function values, then these solutions are also
optimal. The strong duality theorem 4.8 posits that the converse of the sufficient
optimality condition is also true, i.e., if either the primal or dual program has a
finite optimal solution, then so does the other and their optimal objective
function values are the same. This latter point is easily illustrated in Figure 4.1.
Here the primal and dual objective functions approach the same optimal value A
but from
Duality Theory 101
j) } ",t of dual objective valu.,;
f}
A
",t of pdmal objective valu"
f
Figure 4.1
different directions (f approaches optimality m a monotonically increasing

fashion while g approaches the same along a monotonically worsening path).
Thus given primal and dual feasible points XE 9G p and UE 9G D respectively with
f, g their corresponding objective values, we know that if - f > 0 by virtue of
the weak duality theorem. If if - f= 0 (f =if=A), then X and U are also
optimal according to the sufficient optimality condition. Conversely, according
to the strong duality theorem, if X yields f =A, then U must give g =A also and
if - f = 0- there is no duality gap.
The above proof of the strong duality theorem 4.8 relied upon so-called
classical results of linear algebra, i. e., it utilized a fundamental result (the skew-
symmetric theorem) characterizing the existence of a nonnegative solution for
certain dual systems of linear inequalities. Other classical proofs of the strong
duality theorem are provided by Shapiro (1979) (Farkas' theorem is applied to a
system of primal-dual nonhomogeneous equations) and Luenberger (1984) (a
strong separation theorem is used to establish the existence of a hyperplane
separating a closed convex set and a point not in the set).
Since the strong duality theorem 4.8 will be of considerable importance in
our subsequent discussions of linear programming, a variety of alternative proofs
will follow as appropriate. For instance, a constructive proof which also defines
the vector of dual variables will be offered in the next chapter wherein we obtain
an optimality criterion for the primal problem based upon the simplex method.
In addition, another proof based upon saddle point optimality conditions will also
be presented.
Up to now in our discussion of the connection between the primal and dual
problems we have primarily addressed situations in which both of these problems
have feasible as well as optimal solutions. However, it is important to specify
exactly what happens to the relationship between the primal-dual pair of
problems if an infeasible or unbounded solution exists in either of these programs.
To this end we state
4.9. EXISTENCE THEOREM [Goldman, Tucker (1956)].
(a) The primal maximum and dual minimum problems possess finite
optimal solutions if and only if both have feasible solutions.
(b) If the primal (dual) objective is unbounded from above (below),
then the dual minimum (primal maximum) problem does not pos-
sess a feasible solution.
(c) If the dual minimum (primal maximum) problem is infeasible but
the primal maximum (dual minimum) problem has a feasible
solution, then the primal (dual) objective is unbounded from above
(below).
(Here (a) can be thought of as a corollary to the strong duality
theorem while (b), (c) can be considered as corollaries to the weak
duality theorem.)
PROOF. (a) (necessity) If the primal problem has an optimal
feasible solution, then the dual problem has a feasible solution by virtue of the
strong duality theorem. (sufficiency) If X, U are feasible solutions to the primal
and dual problems respectively, then both C'X, b'U are finite with b'U ~ C'X
(the weak duality theorem). And since b'U serves as an upper bound on C'X for
any feasible X, the primal problem must possess a finite maximum.
(b) If the primal objective is unbounded, then the primal problem has a feasible
Duality Theory 103
solution. If the dual problem also has a feasible solution, then according to part
(a) above, the primal problem must have a finite optimum - clearly a
contradiction. (A similar proof holds if it is assumed that the dual objective is
unbounded.)
(c) A feasible solution X to the primal problem cannot also be optimal since, if it
were, a feasible solution U to the dual problem would have to exist by virtue of
the strong duality theorem - again a contradiction. But if no feasible solution
to the primal problem can be ma.'Cimal, then clearly the primal objective must be
unbounded from above. (A similar proof can be offered if it is assumed that the
primal problem is infeasible.) Q.E.D.
The contents of the duality and existence theorems may be conveniently
summarized by Table 4.1. As indicated therein, either both the primal
maximum and dual minimum problems have optimal vectors X, U respectively,
Table 4.1
Dual Minitiulin ProbleIIi
Feasible Infeasible
x, U are e'x
Feasible optimal with unbounded
e/x = b/U from above
b/U No solution
Infeasible unbounded exists
from below
with e/x = b/U, or neither has an optimal vector. In addition, one (and thus
both) of the primal maximum and dual minimum problems has an optimal
solution if and only if either the primal or dual objective is bounded on its
associated nonempty feasible region.
EXAIVIPLE 4.3. The optimal solution to the primal problem
max f = 24xI + 25x 2 s.t.
Xl + 5x 2 ~ 1
4xI + x 2 ~ 3
x l ,x2 2: 0
is Xb = O~, 119) and /0 = 19 (Figure 4.2.a) while the same for the dual problem
min g = u 1 + 3u 2 s.t.
u 1 + 4u 2 ~ 24
5u I + u2 ~ 25
u l ' u2 ~ 0
is Ub = (4,5) and gO = 19 (Figure 4.2.b). Here both the primal and dual
problems possess optimal vectors Xa, Uo respectively with /0 = l, as required by
theorem 4.8.
a. b.
Figure 4.2
EXAMPLE 4.4. As Figure 4.3.a reveals, the primal problem

max / = 2xl + 3x 2 s.t.
-2xl +x2 ~ 3
-!xl + x 2 ~ 6
xl,x2~0
has an unbounded objective function. From our preceding discussion we should

anticipate that its dual,
Duality Theory 105
min 9 = 3u I + 6u 2 s. t.
-2u I -~U2 ~ 2
ul + u2 ~ 3
u I ,u2 ~ 0,
does not possess a feasible solution, as Figure 4.3.b clearly indicates.
-----\+--~-----+ ul
a. b.
Figure 4.3
We now turn to the third fundamental theorem of linear programming,

namely the derivation of the complementary slackness conditions. Actually, two
such theorems will be presented - the strong and weak cases. First,
4.10. WEAK COMPLEMENTARY SLACKNESS THEOREM.
The solutions (Xo, (X8)0), (Uo' (U8)0) to the primal maximum and dual
minimum problems respectively are optimal if and only if they satisfy
the complementary slackness conditions
(4.12)
for optimal solutions (Xo, (X8)0), (Uo, (U8)0) to the primal maxImum
and dual minimum problems respec tively:
(a) if a primal slack variable X~+i' i=l, ... , m, is positive, then the ith
dual variable u?,i=l, ... ,m, must equal zero. Conversely, if u?,
i=l, ... , m, is positive, then X~+i' i=l, ... , m, equals zero;
(b) if the lh primal variable x~, j=l, .. ., p, is positive, then the dual
surplus variable U~l+ j' j=l, . .. , p, equals zero. Conversely, if
uom +j' J=
. 1, ... , p, IS
. POSI. t·Ive, th en x o·
j' J= 1, ... , p, must equaI zero.
PROOF. (necessity) The augmented structural con straint systems

of the primal and dual problems were written earlier as
AX + ImX8 = h, A/U - Ip Us =C
respectively. If the first of these equalities is multiplied by U /, we obtain
U/(AX + I m X8 ) = U/b. Subtracting this expression from the primal objective
function yields
C/X - U/AX- U/X 8 (C/-U/A)X- U /X 8

-U~X - U/x., = f(X) - U/b.
Since U/h is a scalar, this last equation may be rewritten as

U~X + U/X., = g(U) - f(X).
Given that both the primal and dual problems have optimal solutions, the
preceding expression becomes
and thus, by virtue of theorem 4.8,

(U8)bXo + Ub(X8)0 = 0 or
uio xp+i
0 = O',Z= 1, ... ,m,
x~ u~+j = O,j=l, ... ,p.
Duality Theory 107
(sufficiency) If both sides of the primal and dual augmented structural constraint
equalities are multiplied by %,~ respectively, with (Us)~Xo = U~(X8)O = 0,
then U~(AXo) = U~b and ~(A'Uo) = X~C. Transposing this latter equality
thus yields U~AXo = C'Xo = b'U o. Hence Xo, Uo represent optimal solutions.
Q.E.D.
How may we interpret (4.12)? Specifically, U?X~+i = 0, i=l, ... ,m, implies
that either X~+i >0 and u? = 0 or u? > 0 and X~+i = 0 or both u?, X~+i = o.
Similarly, x~ u~+ j = 0, j=l, ... , p, implies that either x~ > 0 and u~+ j = 0 or
u~+ j > 0 and x~ =0 or both x~, u~+ j = O. Alternatively, if the constraint
system for the primal-dual pair of problems is written as
--,-+ {au u1 +.. ·+a m1 um :::: c1

Comple-
Primal
}
I~S~C'i? .............. Dual
Constraints
~ a 1]) U1+···+a mpUm >c
- p Constraints
and (4.12) is rewritten as
u9(b.-f a· .xo) = 0' i=l, ... ,m,

1 j=l
1 IJ J
(4.12.1)
x~ C~laijU?-Cj) = 0, j=l, ... ,p,
it is easily seen that if the kth dual variable is positive, then its complementary
primal structural constraint is binding (i. e., holds as an equality). Conversely, if
the ktll primal structural constraint holds with strict inequality, then its
complementary dual variable is zero. Likewise, if the eth primal variable is
positive, then its complementary dual structural constraint is binding.
Conversely, if the eth dual structural constraint holds as a strict inequality, its
p
complementary primal variable is zero. In short, x~ :::: 0 and bi - .~laijX~:::: 0 as
m J-
well as u?:::: 0 and i~l aiju? - Cj :::: 0 are complementary slack since within each
pair of inequalities at least one of them must hold as an equality.
As a final observation, the geometric interpretation of the weak

complementary slackness conditions is that at an optimal solution to the primal-
dual pair of problems, the vector of primal variables Xo is orthogonal to the
vector of dual surplus variables (Us)o while the vector of dual variables Uo is
orthogonal to the vector of primal slack variables (Xs)o'
It is important to note that the weak complementary slackness conditions
(4.12), (4.12.1) must hold for every pair of optimal solutions to the primal and
dual problems. However, it may happen that, say, for some particular value of
j( i), both x~( u?) and u~+ j (X~+i) vanish at the said solutions. In this instance,
can we be sure that there exists at least one pair of optimal solutions to the
primal and dual problems for which this cannot occur? The answer is in the
affirmative, as the following theorem demonstrates.
4.11. STRONG COMPLEMENTARY SLACKNESS THEOREM. For opti-
mal solutions (Xa,(Xs)o), (Uo' (Us)o) to the primal maximum and dual
minimum problems respectively: (a) if the primal slack variable
X~+i' i=l, ... , 111, equals zero, then the ith dual variable u?, i=l, ... , 111, is
positive. Conversely, ifu?, i=l, ... , 111, equals zero, then X~+i' i=l, ... , 111,
is positive; (b) if the /h primal variable x~, j=I, .. ., p, equals zero,
then the dual surplus variable U~l+ j' j=I, ... , p, is positive. Converse-
ly, if u~+j,j=I, ... ,p, equals zero, then x~,j=I, ... ,p, is positive.
Parts (a), (b) taken together may be summarized as:
(4.13)
PROOF. Given that both the primal and dual problems

have optimal solutions, part (c) of theorem 4.5 implies that t > O. Then from
(4.8.1), (4.9.1),
Uo + h-AXa >0
Xo + A'Uo - C > 0
and thus
Duality Theory 109
Uo + (Xs)o > 0,
Xo + (U s)o > 0 or
Uj°+ x ° > O·,l=,1 ...,m,

p+j
x °+ u m°
j + > O·
j ,J= 1, ... , p. Q.E.D.
By way of interpretation: u? + x~+j > 0, i=l, ... , m, implies that either u? = 0

and x~+j > 0 or x~+j = 0 and u? > 0, but not both u?, x~+j = OJ while
X~+U~l+j > 0, j=l, ... ,p, implies that either x~ > 0 and u~+j = 0 or u~+j > 0
and x~ = 0, but not both x~, U~l+ j = O.
Let us now establish the connection between the weak and strong
complementary slackness conditions by first rewriting (4.13) as
U? + (b j -j~1 ajjxj) > 0, i=l, ... , m,

(4.13.1)
xj + C~1 ajju? - Cj) > 0, j=l, ... , p.
Then, for the strong case, if the kth dual variable is zero, then its complementary
primal structural constraint holds as a strict inequality. Conversely, if the
kth primal structural constraint is binding, then its complementary dual variable
is positive. Note that this situation cannot prevail under the weak case since
if u9 = o(bj-.f
J=1
ajJ'xoJ. = 0),it does not follow that bj-.f ajJ'xoJ. > O(u? > 0),
P J=1
0)
I
i.e., p under the weak conditions, u? = O(b j -jEl ajjxj = implies that
bj - jE 1ajjxj ~ O(u? ~ 0). Similarly (again considering the strong case), if the eth
primal variable is zero, it follows that its complementary dual structural
constraint holds with strict inequality. Conversely, if the eth dual structural
constraint is binding, it follows that its complementary primal variable is
positive. Here too a comparison of the strong and weak cases indicates that,
for
m
the latter, if xj = O( 1-1
.~ ajjuj -
Tn
Cj = 0), we may conclude that
j~1 ajjuj - Cj ~ O(xj ~ 0) but not that i~1 ajjuj - Cj > 0 (xj > 0) as in the former.
To summarize:
Weak Complementary Slackness Strong Complementary Slackness

p p
u91 > 0 implies b·1 -:E a· ·x~ = 0
;=1 IJ J
u91 = 0 implies b·1 -:E a· ·x~
;=1 ~ J
>0
and and
p p
b·1 -:E a ..x~
;=1 IJ J
> 0 implies u9I='
0 b·1 -:E a· .x~ = 0 implies u91 > 0,
;=1 IJ J
i = 1, ..., mj while i = 1, ..., mj while

m m
x~
J
:E a·IJ·u91 - cJ. = 0
> 0 implies i=1 x~ = 0 implies
J
:E a· .u9 - cJ. > 0
i=1 IJ 1
and and
m m
:E a· ·u9 - c·J > 0 implies x~J = 0,
i=1 ' J '
:E a·'J.u91 - c·J = 0 implies
;=1
x~
J
> 0,
j = 1, ...,p. j = 1, ... ,p.

THE THEORY OF
5.1. Finding Primal Basic Feasible Solutions

We noted in Chapter 3 that the primal feasible region 9Gp has a finite
number of extreme points. Since each such point has associated with it a basic
feasible solution (unique or otherwise), it follows that there exists a finite number
of basic feasible solutions. Hence an optimal solution to the primal linear
programming problem will be contained within the set of basic feasible solutions
to AX = b. How many elements does this set possess? Since a basic feasible
solution has at most m of n variables different from zero, an upper bound to the
number of basic feasible solutions is
( n) _ n!
m - m!(n-m)!'
i.e., we are interested in the total number of ways in which m basic variables can
be selected (without regard to their order within the vector of primal basic
variables X B ) from a group of n variables. Clearly for large nand m it becomes
an exceedingly tedious task to examine each and every basic feasible solution.
What is needed is a computational scheme which examines, in a selective fashion,
only some small subset of the set of basic feasible solutions. Such a scheme is the
simplex method (Dantzig (1951)). Starting from an initial basic feasible solution
this technique systematically proceeds to alternative basic feasible solutions and,
in a finite number of steps or iterations, ultimately arrives at an optimal
111
basic feasible solution. The path taken to the optimum is one for which the
value of the objective function at any extreme point is at least as great as at an
adjacent extreme point (two extreme points are said to be adjacent if they are
joined by an edge of a convex polyhedron). For instance, if in Figure 5.1 the
extreme point A represents our initial basic feasible solution, the first iteration
slides f upwards parallel to itself over %p until it passes through its adjacent
f
~------------------------------~Xl
Figure 5.1
extreme point B. The next iteration advances f in a similar fashion to its

optimal (maximal) basic feasible solution at C. So with f nondecreasing between
successive basic feasible solutions, this search technique does not examine all
basic feasible solutions but only those which yield a value of f at least as large as
at the preceding basic feasible solution.
To set the stage for the development of the simplex method, let us
The Theory of Linear Programming 113
max / = C'X s.t.

AX=b, X~O,
where X€R n • To insure that the simplex method terminates in a finite number
of steps at an optimal basic feasible solution (provided one exists), let us invoke
the following nondegeneracy assumption: every set of m columns from' A is
linearly independent. Under this restriction, we are guaranteed that:
(1) every basic feasible solution has exactly m nonzero components; and
(2) f increases in value between successive basic feasible solutions.
With p(A) = m, we may select from A a set of m linearly independent vectors
which constitute a basis for R m and thus the columns of the (m x m) nonsingular
basis matrix B. Then
AX = [B,R] [i~
= BXB+RX R b.
Since IB I i= 0, B- 1 exists and we thus obtain
X B = B-lb-B-1RXR . (5.1)
Let us next partition the (n xl) coefficient matrix C into (m xl) and (n-m xl)
matrices CB and CR respectively, where CB contains the objective function
coefficients corresponding to the basic variables in X B and C R contains the coeffi-
cients on the non basic variables in X R . Then
/(X) c'x
[Cn, Cn ] [i:]
= CnX B + CnXR · (5.2)
Substituting X B from above into (5.2) yields
f(X) = CnB-lb + (C n- CnB-1R)X R· (5.2.1)
At a basic feasible solution to the linear programming problem, X R = 0 so that
X = [~B] r~l, ~ 0,
(5.3)
/(X) = cnB-lb.
If b = B-lb ~ 0, then B will be termed a primal feasible basis.
5.2. The Reduced Primal Problem

In what follows we shall find it convenient to express the primal linear
programming problem along with equations (5.1), (5.2.1) in a slightly different
form. In particular, since the vectors in B form a basis for Rm, any vector in R
may be written as a linear combination of the columns of B. If we denote the lh
column of R by r j' j = 1, ... , n-m, then for all j,
rj = I:: 1 yijb i = BYj
so that Yj = B-1rj' j = 1, ... ,n-m, i.e., Yj is the lh column of B- 1 R.
Hence, from (5.1),
(5.1.1)
From (5.2.1), let Cn=Cn-C'nB-1R have as its /h component Cj

CRj - C'nYj = CRj - f j' j = 1, .. .,n-m. Then
f = C'nb+CnXR
= f + L:;,:;n CjXRj' (5.2.2)
On the basis of equations (5.1.1), (5.2.2) the primal linear programming

problem can be expressed alternatively as
f = f + max CnXn s.t.
L: ;,:;n YjXRj + X B = b (5.4)
XB~ 0, XRj 2': 0, j = 1, ... ,n-m,
or
f =f +max I:;~;l CjXRj s.t.
L:;~;n YijXRj + xBi = hi' i = 1, ... ,m
(5.4.1)
xBi 2': 0, i = 1, ... ,m
XRj 2': 0, j = 1, ... ,n-m.
These primal programs are said to be written ill reduced form in nonbasic
variable space since the number of dimensions involved has been reduced to
n - m = p, the number of non basic variables. In this regard the new set of
objective function coefficients Cj = C Rj - f j' j = 1, ..., n-m, are called reduced

benefit (profit) coefficients (or reduced cost coefficients in the case of a
minimization problem). Moreover, the current set of basic variables xBi'
i = 1, ..., m, serves as a set of slack variables for the structural constraints of
(5.4)(or (5.4.1)).
5.3. The Primal Optimality Criterion

At this point in our discussion of primal basic feasible solutions an
important question emerges: how do we know whether or not a particular basic
feasible solution is optimal? The answer is provided by the following
5.1. OPTIMALITY THEOREM. A primal feasible basis B with
b = B- 1 b > °is optimal if and only if Ck = Ck - B
C B- 1 R ~ 0' or
Cj = CRj - f j ::; 0, j = 1, ..., n-m. Moreover, an optimal primal basic
feasible solution is unique if Cj < 0, j = 1, ... , n-m.
PROOF. (sufficiency) From (5.2.2), f(X) = f + CkXR. Since Ck ~ 0'
and XR ~ 0, we have f(X) ::; f. An since f is obtained from the basic feasible
solution X' = (XB,O') = (b', 0'), f = f(X) is the maximum value of the primal
objective function.
(necessity) For some non basic variable XRj let Cj > o. Hence we can
obtain a new basic feasible solution X with an objective function value f larger
than f. That is, starting from X, we move to X in a fashion such that only x Rj
increases from its current value of zero to the level 0> O. Hence
X' = (XB' Xk) = (XB,Ocj). Since X must be feasible, we require that
b-B- 1RXR = b-OB- 1 Re j

- -1 -
b - OB rj = b - OYj ~ 0. (5.5)
Since b ~ 0, 0 can be made arbitrarily small in order for X B to remam

nonnegative as required. Then
f(X) = f + CkXR f + OCkej

= f +OC j > f. (5.6)
Thus X cannot represent a maximal solution and thus B is not an optimal primal
feasible basis.
< 0 for all j implies a unique optimal basic feasible solution for the
That Cj
primal problem follows from the fact that if Cj < 0 for all j :f. k and ck = 0, then,
by setting j = k in (1).5) and (5.6), we can move to an alternative optimal
feasible basis that renders the same value of I, namely f. Q.E.D.
Given that each Cj ::; 0, j = 1, ... , n-m, at an optimal primal basic feasible
solution, these coefficients are often referred to as optimality evaluators.
5.4. Constructing the Dual Solution

We noted in the previous chapter (corollary 4.6) that either both of the
primal-dual pair of problems have optimal solutions, with the values of their
objective functions being the same, or neither has an optimal solution. And as
evidenced by the strong duality theorem, if one of these problems possesses an
optimal solution, then the other does likewise. In this light, let us now undertake
the derivation of the optimal dual solution from the optimal primal solution. In
fact, this exercise can be thought of as an alternative and constructive proof of
the strong duality theorem 4.8.
To this end let us assume that the primal maximum problem
max I = C'X + O'Xs s.t.
AX+lmXs = b
X, Xs f; 0
has an optimal basic feasible solution consisting of
X B = B-lb,
I(X) = CBX B = CBB-lb.
Note that in the last equation the optimal value of the primal objective function
is written in terms of the optimal values of the primal basic variables x Bi'
i = 1, ... , m. As an alternative, let us respecify the optimal value of the primal
objective function in terms of the optimal dual variables by expressing the latter
as
= (.E
.=1
C Bibil"'" f>Bib im )
.=1
= (u l ,···,um ),
where cBi,i = 1, ... ,m, is the ith component of C B and bij,i,j = 1, ... ,m, is the
element in the ith row and /h column of B- 1 • B
Then C B- l = U' and thus
f(X) = U'b. Thus the value of f corresponding to an optimal basic feasible
solution may be expressed in terms of the optimal values of either the primal or
dual variables.
Given that the optimality criterion is satisfied, we must have
C'R - < 0' or -C j --
C'B B- I R = CRj - C'B B- 1r j _< 0,J. -- 1, ... , n-m,
where rj is the /h non basic vector. Moreover, this inequality must hold for basic
vectors as well, i.e., relative to the ith basic variable x Bi' i = 1, ..., m,
ci = cBi-CRei = cBi-cBi = O.
Hence the optimality criterion holds for all columns of [A,lm] (or [B, RD. In this
regard:
(a) for the columns of A,
and thus
U' A ~ C' or A'U ~ C,
i.e., U = (B- l )'CB represents a solution to the dual structural constraint
system;
(b) for the columns in 1m ,
and thus
U'I m ~ 0' or U ~ 0,
i.e., U is a feasible solution to the dual structural constraints. urthermore, since
g(U) = b'U = U'b = CBB-Ib = CBX B = f(X),

U is an optimal solution to the dual minimum problem given that X represents
an optimal basic feasible solution to the primal maximum problem.
An important observation which emerges from this construction is that the
concept of primal optimality is equivalent to the notion of dual feasibility. Hence
if C n- CBB- I R ~ 0', B is termed a dual feasible basis. In this regard, at an
optimal dual feasible solution U, the optimality evaluators Cj' j = 1, ... ,n-m,
within CR correspond to the dual surplus variables in the dual structural
constraint system A'U ~ C, i.e., forming A'U - CR = C, it follows that
-C R = C - A'U ~ 0 under primal optimality and thus CR ~ O. So if all C/s
~ 0, the associated dual solution is dual feasible and hence B is a dual feasible
basis. If any of the C/s determined from the basis B are positive, then the
associated dual solution is infeasible and hence B is dual infeasible.
Having developed a criterion for primal optimality (equivalently, dual
feasibility) and a definition of the vector of dual variables U' = CBB-I, let us see
how we can rationalize the structure of the symmetric dual problem. Again let
the primal problem correspond to
max f = C'X s.t.
AX~b, X~ O.
Let us form the primal Lagrangian function
L(U) = max{C'X + U'(b - AX)}

(5.7)
= U'b+ max (C'- U'A)X
x~o
by weighting or Prlczng out the structural constraints by the vector of dual

variables U ~ O. Two cases can now be addressed.
First, if e' - u'A ~ 0' (note that for U' = CnB- 1 , this inequality is simply
the primal optimality criterion), then the ma.ximum in (5.7) is attained for X = 0
and thus L(U) = U'b. Here L(U) is a finite upper bound on the maximal primal
objective value (by the weak duality theorem 4.2). Second, if C' = U'A ~ 0', say
ck = U'ak > 0, then for Xj = 0, j =1= k,
L(U) = U'b + max (ck - U' ak)xk.
Xk ~o
Clearly L(U)--+oo as xk--+oo, i.e., according to existence theorem 4.9, if the dual
problem is infeasible but the primal problem has a feasible solution, then the
primal objective value is unbounded from above.
To construct the dual problem let us assume that primal optimality holds.
Hence we many determine, via the weak duality result, the least upper bound of
L(U) or
minL(U) s.t.
e' - U'A ~ 0', U ~ 0
or
ming = U'b s.t.
A'U ~ C, U ~ 0,
the symmetric dual problem introduced at the outset of Chapter 4.

Looking to the dual variables themselves, let us note that for any particular
requirements vector b, f(b) depicts the maximum value of f when AX ~ b,
X ~ 0, and b is fixed. Then at optimal feasible solutions X, U to the primal-dual
pair of problems, the strong duality theorem 4.8 informs us that
T = e'x = b'U = E~l bjuj and thus aT jab j = 'iij (generally). (Later on in the
text we shall elaborate on the circumstances under which this strict inequality
might not hold.) That is, when the dual optimal feasible solution U is unique,
the ith dual variable 'iii depicts the rate of change of the optimal primal objective
value with respect to small (per unit) perturbations in the right-hand side value
bj of the ith primal structUl'al constraint, given that all other requirements be'
e=1= i, remain constant. Here 'iii ~ 0 for all i since increasing slightly the size of
the primal feasible region cannot decrease the maximal value of f.
Sometimes Uj is referred to as the marginal imputed value of the ith primal

structural constraint right-hand side value bi. It is also referred to as a simplex
multiplier since it serves as a Lagrange multiplier in (5.7) above. In this regard,
since its role in (5.7) is to price out the ith primal structural constraint, Uj is
often called a shadow (fictitious) price. We shall return to our interpretation of
the dual variables later on when we explore the geometry of the primal-dual pair
of problems.
Having discussed the nature of the dual variables ui' i = 1, ... , m, we are
now equipped to offer two distinct interpretations of the optimality evaluators
Ci' j = 1, ... , n-m. But first, however, let us examine the significance of the
columns within the matrix product B- 1 R in
For j fixed at k, the preceding expression becomes
Thus Yik' i = 1, ... , m, represents the amount by which the ith basic variable
would change in any primal basic solution given that the kth non basic variable
increases from zero (its current nonbasic value) to one, with all other nonbasic
variables remaining at their zero level.
Looking now to the optimality evaluators, again let j = k so that
We may interpret this expression as the net change in f given that x Rk increases
from zero to one. In what sense shall we interpret the notion of net change? As
x Rk increases by a unit, it directly impacts f through its objective function
coefficient c Rk. Next, x Rk affects f indirectly through its impact on the m basic
variables x Bi' which in turn precipitate a change in f via their objective function
coefficients C Bi. Thus the combined indirect effect of a one unit increase in x Rk
on f is fk=E?~lCBiYik. Hence ck=cRk-fk is the net effect on f of
increasing x Rk by a single unit. In sum,
Ck marginal net effect on I of an increase in x Rk

= cRk-h
marginal direct effect on I of an increase in x Rk
marginal indirect effect on I of an increase in x Rk.
So at an optimal feasible solution to the primal linear program we must have

ck $ 0, i.e., the marginal indirect effect on I of increasing xRk cannot fall short
of its direct counterpart.
An alternative approach to interpreting the c/s, j = 1, ..., n-m, is to let
CR= CR- CBB-1R = CR- u'a Then for j = k we have
(Here rk = ak. If rk = ek' then ck = -uk.) While the direct effect on I of a unit
increase in x Rk is the same as before (f changes marginally by C Rk)' the indirect
impact on I when x Rk = 1 is now given by f k = E~l Ujaik. Given that aik
represents the contribution of the ith constraint in supporting x Rk at the unit

level and Uj depicts the imputed contribution to I of relaxing the ith constraint
by one unit (since typically ui = af labi ;::: 0), we see that h amounts to the total
imputed change in f resulting from a per unit increase in x Rk given that at least
one primal structural constraint is binding (i. e., hence at least one Uj can be
positive by virtue of the weak complementary slackness conditions) and the per
unit increase in x Rk is brought about by increasing the right-hand side of at least
one primal structural constraint.
Equivalently, ck is the relative cost of making xRk basic, i.e., it is cRk' the
direct effect on I of increasing xRk by a unit, less the value, in terms of I, of the
real opportunity cost of doing so. This real opportunity cost of making x Rk
basic, in terms of a reduction in I, is U'rk = E~l Ujaik' the shadow value of the
contributions of all constraints in supporting x Rk at the unit level.
EXAMPLE 5.1. To see exactly how (5.3) IS obtained in actual

practice, let us
max f = xl + 3x 2 + 4x3 s.t.
2XI + X2 + 4x3 ::; 100
xl + 3x 2 + x3 < 40
xl' X2 ' X3 ~ O.
Since our starting point is the primal linear programming problem in standard
form, we introduce two nonnegative slack variables x 4 ' Xs and ultimately
max f = + 3x 2 + 4x3 + OX4 + OXs
Xl s.t.
2XI + x 2 + 4x3 + X 4 100
Xl + 3x 2 + X3 40
XI' ..• 'X s ~ o.

In terms or our previous notation,
1
3
A 2 1 4 10]
[ 13101'C 4
o
o
Since the determinant of the (2 X 2) submatrix formed from columns three and
five of A is different from zero, p(A) = 2. With these two columns linearly
independent, we may consequently select x3 and Xs as basic variables with Xl' x 2'
and x 4 nonbasic and proceed to generate our first basic feasible solution. In this
regard,
B [ 41 01 B-
1 '
1
From (5.1),
(5.8)
If X R = 0, we obtain the nondegenerate primal basic feasible solution

X' = (0,0,25,0,15).
We previously denoted the /h column of B- 1 R by Y j . Hence Yij' the ith
component of Y j , represents the amount by which the ith basic variable would
change in any primal basic solution given that the /h nonbasic variable increases
from zero (its current nonbasic value) to one. For instance, in our example, Y13
indicates that if x4 increases from zero to one, x3 decreases by ~ to x3 = 25 - ~(1)
99
4'
From (5.2.1),
f = CnB-lb-(Cn-CBB-lR)XR
100+(-1,2,-1) ~:J
r~R~
(5.9)
Here c1 = -1, c2 = 2, and c3 = -1. Upon setting X R =0 it is easily seen that

the value of the objective function corresponding to this basic feasible solution is
100. We mentioned earlier that Cj depicts the marginal net effect on f of an
increase in x Rj' For example, let us assume that x 2 increases from zero to one
with the remaining non basic variables still equal to zero. The effect of this
change is two-fold. First, there is the direct influence of x 2 on f through its
corresponding coefficient in C R , i.e., since
it is easily seen that when x 2 = 1, I = 3( 1) = 3. Hence I directly increases by

three units. Next, x 2 affects I indirectly through its influence on the basic
variables x3 and xS ' which in turn produce a change in I. Moreover, the said
effect is transmitted through the term CBY 2. That is to say, given
Y"ru = [~ ]",
we see that if x2 = 1, x3 changes by i unit and Xs changes by 141 units since, from
(5.8),
XB = b- Y2xR2 or
But the effect of a change in x3 and Xs upon I is issued through their

corresponding coefficients in C B , i.e.,
For x2 = 1, the indirect effect of x 2 on I is thus 12 = 1 unit. Hence the net

change if I, being the difference between the direct and indirect effects of x 2' is
given by c2 = c R2 - 12 = 3 - 1 = 2 units, which is just the coefficient on x 2
in (5.9).
I = 100 - Xl + 2x2 - x 4 ' 1= 100 when X R = 0
We noted above that since
is definitely not maximal since I can increase in value if x 2 does. Now at the
moment x2 is equal to zero so that if we increase its value from the zero level by
some positive amount, it is obvious that, with p(B) = 2, we must change the
status of some variable which is currently basic to non basic since there are at
most two linearly independent vectors in a basis for R2 and thus at most two
basic variables. Hence we must determine which basic variable, x3 or x s, should
now become non basic (i.e., decrease in value to zero). The answer to this
question is the topic of the next section wherein we consider the generation of a
succession of extreme point solutions to a linear programming problem using the
primal simplex method.
5.5. The Primal Simplex Method

Let us assume that we have obtained a basic feasible solution to a linear
programming problem but that the solution is not optimal, i.e., at least one Cj
= CRj-fj>O,j=I, ... ,n-m, in
(5.10)
How may we obtain a new basic feasible solution with an objective function value
which is at least as great as at the old basic feasible solution? That is to say,
given a basic feasible solution X' = (XB,O') with f(X) = CBX B , we week to
determine an alternative basic feasible solution X' = (X~, 0') € R n such that
f(X) = C~XiJ;::: CBX B , where B denotes the new mth order nonsingular basis
matrix. If more than one Cj > 0, j = 1, ... , n-m, we must first determine which
non basic variable is to become basic (i.e., increase in value from zero). Let us
adopt the convention of picking the nonbasic variable in (5.10) with the largest
positive Cj coefficient since such a choice involves the largest increment in f per
unit change in x Rj' j = 1, ... , n-m. In this regard, if l denotes the index set
consisting of all j for which Cj > 0, i. e., l = {j Ie j > O}, and
(5.11)
then xRk' the nonbasic variable corresponding to the kth column of R, rk'
becomes basic. Next, how do we go about selecting a column of the current basis
matrix B to be replaced by rk? The answer is provided by the following corollary
to optimality theorem 5.1. As we shall now see, the proof of this corollary is
constructive in that it outlines a process called the primal simplex method for
actually obtaining a new basic feasible solution which renders an increase in the
value of f.
5.2. COROLLARY (THE PRIMAL SIMPLEX METHOD). Let B repre-
sent a primal feasible basis with b = B-Ib > 0 (nondegeneracy) and
Cj > 0 for some nonbasic variable x Rj' j = 1, ... , n-m. Then either:
(a) there exists an extreme direction d j with C'dj > 0, in which case
x Rj can be increased indefinitely without violating feasibility so
that the primal objective function value is unbounded; or
(b) we obtain a new primal feasible basis and a new (possibly degener-
ate) basic feasible solution that yields an improved value of f.
PROOF. Let the current basic feasible solution be depicted as X' =
(XB' Xh) = (b',O') with C'X = CBb. Also, let Ch 1:. 0' so that primal
optimality does not hold. In particular, let ck = cRk - CBB-Irk > O. Hence
we may increase x Rk above its current non basic value of zero to the level () > 0 so
as to obtain a larger value of f. To this end, using (5.1.1), XB = h changes to
X B = b - (}Yk while XR = 0 becomes (}ek' Thus our new solution point is
(5.12)
Looking to the primal objective value,
C'X C'X + (}C'd k

cBb + ()( -cBYk + C Rk)
cBh+ (}ck' (5.13)
At this point two cases must now be considered.

1. Y k ~ O. In this instance d~ = (- Y k,ck) ~ 0' serves as an extreme

direction since the requirement Adk =0 renders X feasible, i.e., AX =A(X + Odk)
= AX+OAdk=AX=b. Then C'd k = -CBYk+cRk=ck>O, i.e., we have
found an extreme direction d k along which f can be increased without bound
since, from (5.13), C'X --+00 as 0--+00 when ck > O.
2. Yik > 0 for some i. Then 0 cannot be increased indefinitely without
violating primal feasibility. To see this let us employ (5.4.1) and write
XBi = bi-Yik 02: 0, i = 1, ...,m. (5.14)
In this regard, we need to find a 0> 0 such that (5.14) holds for all
Yik > O. Rearranging (5.14) yields xB;!Yik 2: 0 so that any 0 for which
will do, i.e., will insure that primal feasibility is not violated.
In fact, this latter inequality enables us to select the b i to be removed from
the basis when fk is entered into the basis. If
(5.15)
is substituted into b - BYk' then the component(s) of this vector for which the
minimum (assumed unique) is attained reduces to zero or becomes nonbasic, i.e.,
If the minimum in (5.15.1) is unique (nondegeneracy) then the column of B to be

replaced by fk can always be uniquely determined by the exit criterion. In this
circumstance, if {bl ,. .., bm } denotes the set of linearly independent vectors
obtained by replacing the rth column b r within {bI , ..., bm } by rk' then the new
basic feasible and thus extreme point solution can be represented as
X [~B] [B~Ib l (5.16)
where B [bI ,·· .,bml with

b. I = bi' i =/= rj b r = rk
and
X Bi = bi-BYik' i=/=rj xBr = B = xBr/Yrk' (5.16.1)
Clearly Xmust have exactly m positive components.

In addition, since C Bi = C Bi' i =/= rj C Br = C Rk j and C Br(x Br - BYrk) 0, it
follows that
I I -
CBX B = C B(b- 8Yk)+CRk X Br
cBb + (C Rk - cBY k)8

(5.17)
So with 8> 0, f(X) > CBb if ck > O. Hence we obtain an improved solution to
the primal problem. Q.E.D.
Based upon the preceding discussion it is evident that the exit criterion
(5.15.1): (1) determines the vector b r to be removed from the present primal
feasible basis Bj (2) determines the value of the variable x corresponding to the
Br
A
incoming vector rk for the new primal feasible basis Bj and (3) in the absence of
degeneracy, insures that X has exactly m positive components and thus
constitutes a basic feasible solution to AX = b. And since the columns of Bare
linearly independent, X represents an extreme point of the primal feasible region.
We may summarize the salient features of the preceding construction by
outlining the following stepwise procedure for implementing the simplex method:
5.3. PRIMAL SIMPLEX ALGORITHM.
Step O. Initialize.
Obtain an initial primal feasible basis B. This provides us with
our first basic feasible and thus extreme point solution
x' = (X~, XR) = (h',O') and beginning objective value C'X =

, - --1
CBb, where b = B b.
Step 1. Check the optimality evaluators CR= C R- C~B-l R.
(a)
-, ,
If C R ~ 0 ,stop. We have obtained an optimal basic feas-
ible solution.
(b) If CR~ 0', apply the entry criterion (5.11).
Step 2 If x Rk is chosen to become basic, check Yk'
(a) If Y k ~ 0, stop. The objective value is unbounded along the
infinite ray X + Bd k , where x Rk = B 2: 0, dk = (-Y k,e~J
(b) If Y k ~ 0, apply the exit criterion (5.15.1) to determine the
variable x Br to become nonbasic.
Step 3. Form a new primal feasible basis B by replacing b r within B by
rk and return to step 1.
The entire process involving steps (1) - (3) is repeated until either primal
optimality is attained or we detect that the objective is unbounded. Since the
simplex routine examines (adjacent) extreme points, which are finite in number,
the simplex process must terminate in a finite number of iterations. And as
(5.17) reveals, since ck 2: 0, the objective function will be non decreasing between
successive basic feasible or extreme point solutions. 1
Let us now return to the example set forth at the outset of this chapter and
proceed to generate an optimal basic feasible solution given the initial one. We
previously found, using (5.1.1) and (5.2.2), that
or
[::J
IFor details on an interesting variant of the standard simplex routine see the steepest
edge simplex method presented in Appendix B.
f CBB- 1b + (C n- BC B- 1R)XR
= C'"n-Ib + (CI• ',. '3) ~~

= 100 - xl + 2X2 - X4·
Setting X R = 0, our initial nondegenerate basic feasible solution was found to be

X, = (0,0,25,0,15) with f = 100.
Since c2(=2) > 0, it is obvious that the current basic feasible solution is not
optimal. Hence we must change the status of x 2 from non basic to basic. To this
end we have
Setting this expression equal to the null vector renders the values of X R2 = x2
which respectively reduce x3' x5 to zero, namely x 2 = ~~; = 100, x2 = ~:: = ~~
and thus
~() _ 60 _ m;n {XBl _ 100 xB2 _ 60}
- 11 - • Y12 - 'Y22 - 11 '
i.e., the largest increment in x2 consistent with primal feasibility is ~~. Hence x5
becomes non basic and r2 replaces b2 in the previous basis matrix so that our new
basis matrix is
with
B- 1 [_1: 11
-~1]
11
The Theory of Lineal' Programming 131
(see Appendix A for a method of transforming B to B). In this instance,
and thus an iteration produces

X. =
B
B-1b-B-1:Rx.R or
f C~B-lb+ (c l , c2 ' C3 )Xil
110.91 - ~~xl -18lxS -19lx4 ·
.
Upon settmg Xil = 0, X;0.., = (0,5.46,23.63,0,0) and f = 110.91. And since all the
cis are negative, this second nondegenerate basic feasible solution is optimal.
What are the optimal values of the dual variables associated with this
primal solution? To answer this we need only compute
C/. B- 1
B (4,3) [_1: -~l]
11 11
In addition,
f g b/U (100,40) [ ~] 110.91
are required.
5.6. Degenerate Basic Feasible Solutions

Earlier in our discussion of the simplex method we instituted a
nondegeneracy assumption, the purpose of which was to insure that every basic
feasible solution to the augmented structural constraint system AX = b was
nondegenerate. In this section we shall examine the implications of the existence
of degenerate basic feasible solutions to AX = b. We noted in Chapter 3 that if
peA) = m, we may select m linearly independent columns from A and form an

(m x m) nonsingular matrix B. We then obtain a basic solution to AX = b by
solving BX B = b. Hence such a solution occurs when b is expressible as a linear
combination of the columns of B. Moreover, the said solution is feasible if b can
be represented as a nonnegative linear combination of the same, i.e., b is an
element of the convex polyhedral cone eB = {Y I Y = BXB,XB ~ O} generated
by the columns of B. Let us now get a bit more restrictive. If b is expressible as
a strictly positive linear combination of all the basis vectors in B, then the basic
feasible solution to AX = b nondegenerate. In this instance, none of the basic
variables within X B vanishes. In sum, the point being made is that degeneracy
occurs if b lies within a convex polyhedral cone spanned by only some subset of
vectors of B i.e., b is a positive linear combination of fewer than m basis vectors.
Hence b lies along an edge of eB and not within eB proper so that not all
components of X B are positive.
To relate these observations to our discussion in the previous section
pertaining to the generation of new or improved basic feasible solutions, let us
address ourselves to a couple of remarks concerning the value of 0 which
constitutes the minimum in (5.15). As we shall now see, the presence or absence
of degeneracy is reflected by the behavior of O. What exceptional situation may
arise? Let us assume that 0 is not unique. Hence we may remove from the basis
anyone of the columns of B for which the same minimum value of x Rk (0) is
assumed and obtain a new (call it the eth ) basic feasible solutions. In this
instance, the eth basic feasible solution is degenerate since x~ = 0 for all i
Bi
corresponding to those columns that have the same () = 0 as the eliminated
column. To see this, let us assume that the status of x Ru is to be changed from
~ xB
non basic to basic and that () = - yr, Yru > 0, so that column b r of B is to be
ru
removed from the basis. In addition, for some other column b s in B,
xBr xBs
let - y = - y 'Ysu > o. Then from (5.16.1),
ru su
x iJi = XBi - OYiu

xBr
xm - Y ru Yiu or
xiJj xBi xBr

Yju Yiu - Yru
and, for i = s,
x Bs _ x Br
Y su Yru
= ° or x. -
Bs
0.
So when the variable x Ru turns basic (becoming x iJ) and thus increases from
zero to a level coincident with 0, it reduces to zero not only the variable which it
is displacing, namely x Br' but also any other variable x iJs whose original value
xBs was used in determining O. In this regard, if another iteration is performed
and we obtain the €+ 1st basic feasible solution, the sth component x iJs within
XiJ = fi-1b is zero. It is at this point in our discussion of degeneracy that a
major complication introduced by this notion emerges. For if we now determine
that the v th column of R is to be introduced into the basis, then
o= Y~: = ~v = 0,
X·
y Ysv > 0, i. e., 0 vanishes since the current or €+ 1st basic
feasible solution is degenerate. With 0 = 0, (5.16.1) informs us that if the next
or €+2 nd iteration is executed, the values of the variables common to the €+ 1st
and €+2 nd basic feasible solutions are unchanged while the value of the variable
x· whose status has changed from nonbasic to basic is zero. Hence the €+2 nd
Rv .
basic feasible solution is also degenerate. In this situation, it is clear that no
improvement in the value of the objective function occurs, i.e., although we have
actually obtained a new degenerate basic feasible solution from the old one, we
are still at the same extreme point. On this latter point pertaining to the
absence of a one-to-one correspondence between basic feasible solutions and
extreme points in the presence of degeneracy, see the argument underlying
(2.22.1).
An additional complication precipitated by degeneracy is that we may not
be able to unambiguously interpret the dual variable Uj as the marginal change
in the optimal primal objective value per unit change in the ith requirement bi ,
i.e., ui = of/obi' i = 1, ..., m, may not be uniquely defined in the presence of
degeneracy, a point to which we shall return later on.
It must be mentioned that a new basic feasible solution will not always be
degenerate given that the previous one was. For example, if for some x Br = 0 we
have Yrj < 0, then, since negative Yi/s are not used in computing 0, ~~; = 0 is
not a candidate for the minimum in (5.15) so that x Br = 0 :f. 0 in the next
iteration. Thus the new basic feasible solution will be nondegenerate.
To summarize, when a degenerate basic feasible solution occurs at some
particular round of the simplex method, the value of the objective function is
unchanged when the next iteration is carried out. In fact, it may remain
constant for some large number of subsequent iterations. In addition, there also
exists the possibility that all succeeding basic feasible solutions will be
degenerate. In this latter instance, the simplex process is said to cycle forever in
an endless loop, i.e., a given sequence of bases is repeated again and again so that
convergence to an optimal solution never occurs.2 In the absence of degeneracy,
the value of the objective function increases at each iteration so that we never
return to a previous basis. And as mentioned earlier, since there are a finite
number of bases, we are guaranteed that the simplex method will terminate in a
finite number of iterations.
When degeneracy presents itself, how may it be resolved? The answer is
quite simple. In light of the previously observed fact that the threat of cycling is,
for all practical purposes, virtually nonexistent, no anti degeneracy procedure is
really needed since the effect of degeneracy is simply to slow down the simplex
routine along its path to the optimum rather than to preclude the optimum from
ever being attained. All that one really needs to do is to make an arbitrary
2While cycling is indeed theoretically possible, it is essentially an academic matter since,

in actual practice, no ordinary linear programming problem has ever been known to cycle
(although the sort of degeneracy discussed above is quite common). Artificial problems which
exhibit cycling have been discussed by Beal (1955), Hoffman (1953), Marshall and Suurballe
(1969), Balinski and Tucker (1969), and Yudin and Gol'shtien (1965).
choice from among the tied ratios which constitute the minimum in (5.15). If a
specific anticycling procedure is desired, we may employ:
(1) the perturbation method of Charnes (1952) which perturbs or moves the
vector b a suitably restricted distance and in a fashion such that b
becomes a strictly positive linear combination of the columns of B.
(2) a lexicographic rule for selecting the exiting variable proposed by Dantzig,
Orden, and Wolfe (1954); and
(3) the rule of Bland (1977) for selecting both entering and exiting variables.
Additional considerations regarding anticycling routines are provided by Dantizig
(1951a), Wolfe (1963), Perold (1980), Azpeitia and Dickinson (1964,), and Gass
(1985).
5.7. Unbounded Solutions Reexamined

We noted earlier that any vector r j with Cj > 0 may be introduced into the
basis provided that at least one component of Y j is positive, i.e., there must exist
at least one Yij > 0, i = 1, ... , m. Given that the present basic feasible solution is
nondegenerate, this restriction not only ensures that the new solution will be
basic as well as feasible, it also precludes the existence of feasible solutions which
are non basic (contain more than m nonnegative variables) and for which the
objective function possesses no finite upper bound. To verify this let us amplify
our earlier comments on the existence of an extreme direction d j along which f
can be made to increase without bound (corollary 5.2) and assume that we are to
introduce into the basis some vector r·J for which C·J > 0 and all
Yij ~ 0, i = 1, ... , m. In this circumstance, an examination of the discussion
underlying (5.15) indicates that for the new basic solution to be feasible, r j must
enter the basis at the zero level. But to determine the effect of r j on the
objective function, inserting this vector at a zero level will definitely not do.
Hence the following trade-off will be made. Instead of trying to obtain a basic
solution which is feasible, let us derive a feasible solution which is non basic, z.e.,
which expresses b as a nonnegative linear combination of m+ 1 vectors.
With all Yij ~ 0, i = 1, ... ,m, rj cannot replace any vector currently in the
basis since x Bi - 8Yij > 0, i = 1, ... , m, given that 8> 0, i.e., in this situation, it
is impossible to change the status of any variable from basic to non basic. So if
the current basic feasible solution is nondegenerate, b is expressible as a
nonnegative linear combination of the m + 1 vectors b I , ..., b m , b r and thus m + 1
variables or components of X are different from zero. To this end let us add and
subtract 8rj in the current basic feasible solution to obtain
BXB +8rj-8rj = b.
With Yij ~ O,i = 1, ... ,m, and 8> 0, X B -8Yj ~ 0 so that while this solution to
the augmented structural constraint system AX = b is feasible, it is nonbasic
since b is expressible as a nonnegative linear combination of m + 1 vectors, i.e., X
has m + 1 nonnegative components. The value of the objective function
corresponding to this nonbasic feasible solution is, from (5.17),
I
+ cRi
~ ~
C B (X B -8Y j )
f(X) + cl,
where CBi(XBi-8Yij)i=0, i=I, ... ,m. So with Cj>O, f(X) can be made
arbitrarily large simply by making an appropriate selection of 8. In this instance
f is said to be unbounded since there is, theoretically, no finite upper limit to our
choice of 8 and thus to the value of f. In sum, if all Yij ~ 0, i = 1, ... , m, for
some vector r j in R with Cj > 0, then there exists a feasible solution in the m + 1
variables x B1 , ... , x Bm' x Rj with the result that the values of the objective
function and at least two of these variables may be increased without bounded.
If this case obtains, the simplex process should be terminated.
5.8. Multiple Optimal Solutions

We noted in the previous chapter that, by virtue of the extreme point
theorem, the objective function assumes its optimal value at an extreme point of
the feasible region Xp. Moreover, if the optimum occurs at more than one
extreme point, the objective function takes on that same value at every convex
combination of those points. Now, extreme point solutions correspond to basic
feasible solutions and conversely so that in this latter instance involving multiple
optimal solutions, there exists only a finite number of optimal basic feasible
solutions but an infinite number of optimal nonbasic feasible solutions. In short,
while the optimal value of the objective function is unique, the set of basic
variables which yield the optimum will not be unique in the case of multiple
optimal solutions.
How may we determine whether two or more different (nondegenerate)
basic feasible solutions yield the same value of the objective function? To answer
this question let us assume that we have obtained an optimal basic feasible
solution to a linear programming problem and that Cj = 0 for some vector r j in
R with at least one Yij > 0, i = 1, ... ,m. In this instance it is evident that when
r j replaces one of the vectors currently in the basis, we obtain a new basic
feasible solution X with f(X) = f(X) + c/f = f(X). Hence X must also be
optimal. So when the coefficient Cj on the non basic variable XRj is zero for some
particular optimal basic feasible solution, the set of basic variables which yield
the optimum is not unique, i.e., x Rj can become basic without changing the
maximal value of the objective function so that we obtain an alternative optimal
basic feasible solution. Such a situation is illustrated in Figure 5.2.a. Here the
extreme points A, B yield optimal basic feasible solutions while any non extreme
point C = AA + (I-A)B,O < A < 1, yields an optimal though non basic feasible
solution with f(A) = f(B) = f(C).
C='xA+(1-'x)b
0~'x~1
B
a. b.
Figure 5.2
The above discussion has indicated that if there exists at least two different
optimal basic feasible solutions to a linear programming problem, then there
also exists an infinite number of optimal nonbasic solutions. However, it must be
mentioned that there may occur a situation in which we have an infinite number
of optimal non basic feasible solutions but only a single optimal basic feasible
solution. To verify this let us assume that we have obtained an optimal basic
feasible solution and that Cj = 0 for some vector rj in R with all Yij ~ 0,
i = 1, ..., m. If r j is introduced into the basis, we obtain a new feasible
nonbasic solution
B(X B - O'Yj ) + O'rj = b
in the m + 1 vectors bI , ... , b m , r j. In this instance !(X) =!(X) + Cl = !(X) so

that X is also an optimal solution for any arbitrary 8> o. With rj ::j:. 0, at least
one Yij < 0 so that there exist optimal nonbasic solutions for which at least two
of the m + 1 variables can be made arbitrarily large since the value of xRj can be
increased without bound. This case is illustrated in Figure 5.2.b. Here the
optimal hyperplane is tangent to the feasible region along an edge which extends
to infinity.
How does a situation involving the existence of multiple optimal basic

feasible solutions to a linear programming problem formally compare with one in
which a degenerate basic feasible solution occurs. A definite similarity emerges
since, quite generally, the value of the objective function remains unchanged in
each case as we move from one basic feasible solution to another. Specifically, in
a problem involving multiple optimal solutions, we obtain the same value of the
objective function as at the preceding solution since Cj = 0 for some vector rj not
in the basis with 9 > 0, while a degenerate solution yields, after an additional
iteration, the same value of the objective function because 9 = O. In the first
instance we move from one extreme point and thus basic feasible solution to
another while in the latter we remain at the same extreme point although we
obtain a new degenerate basic feasible solution.
6 DUALITY THEORY REVISITED
6.1. The Geometry of Duality and Optimality

Starting with the primal problem, let us
maxf{X) = c'x s.t.

AX~b, X~O, XERn,
where A is of order (m x n) with rows a I , ... , am and b is an (m x 1) vector with

components bI , ... , bm • Alternatively, if
are respectively of order (m + n x n) and (m + n x 1), then we may
ma.x f{X} = C'X s.t.

b-AX~ 0,
where X E Rn is now unrestricted. In this formulation x j :::: 0, j = 1, ... , n, is

treated as a structural constraint so that b - AX ~ 0 defines a region of feasible
or admissible solutions 9G eRn.
If Xo yields an optimal solution to the primal maximum problem, then no
permissible change in X (i.e., one which does not violate any of the constraints
specifying 9G) can improve the optimal value of the objective function. How may
we characterize such admissible changes? It is evident that, if starting at some
141
feasible X, we can find a vector h such that a small movement along it violates
no constraint, then h specifies a feasible direction at Xo. More formally, let
I5(Xo) be a suitably restricted spherical c5-neighborhood about the point Xo € R n •
Then the (n x 1) vector h is a feasible direction at Xo if there exists a scalar t ~ 0,
o~ t < 15, such that Xo + th is feasible. In this light, for c5(Xa) again a spherical
c5-neighborhood about the feasible point Xo € R n, the set of feasible directions at
Xa,~(Xa), is the collection of all (n x 1) vectors h such that Xa + th is feasible for
t sufficiently small, i.e.,
~(Xo) = {h I if Xa€%, then Xa + th€%, 0 ~ t < c5}.

Here ~(Xo) is the tangent support cone consisting of all feasible directions at Xo
and is generated by the tangent or supporting hyperplanes to % at Xa (Figure
6.1). And since each such hyperplane
-ajh -aj(X-Xa)
hj-ajX=O, i= 1, ... ,m+n,I
arX= hr
."..,., (tangent hyperplane)
."..,.
.".
aa X = ha
" (tangent hyperplane)
'" "
L -_ _ ---""~ _ _ _ _ _ _ _ _ _ ___+ Xl
Figure 6.1
specifies a closed half-space ajX ~ hj' i = 1, ... , m+n, the tangent support cone
~(Xo) represents that portion of the intersection of these closed half-places in the
immediate vicinity of Xa.
. the l·th row

IH ere aj IS 0 f A- W h'l
I e -b j IS
' the l·th component 0 f h- • Specifically:
aI=al' ... ,am=ami lim+j=ej, i=I, ... ,n; bI =hI, ... ,bm=b m ; and bm +j =0,
j= 1, •.. , n.
Duality Theory Revisited 143
To discern the relationship between GJl(Xo)(Xo optimal) and: (a) I(Xo)i (b)
the constraints b - AXo ~ 0, let us consider the following two theorems. First,
6.1. THEOREM. If the primal objective function attains its maximum at
Xo, then any movement from Xo along a feasible direction h cannot
increase the value of I, i.e.,
dl o = C'h = C'(X-Xo):5 0 or C'X:5 C'Xo.
PROOF. If I(Xo) = C'Xo is the maximum of lover %, then for
t>O
C'(Xo+th) - C'X 0 = tC'h :5 0
and thus C'h :5 0 for t > O. Q.E.D.
Hence I must decrease along any feasible direction h. Geometrically, no feasible
direction may form an acute angle « 1r /2) between itself and C.
To set the stage for our next theorem, we note that for any optimal X f %,
it is usually the case that that not all of the inequality constraints are binding or
hold as an equality. To incorporate this observation into our discussion, let us
divide all of the constraints b - Ax ~ 0 into two mutually exclusive classes -
those which are binding at X, hj - ajX = 0, ~nd those which are not, hj - ajX > 0,
i = 1, ..., m + n. So if
3 = {ilhj-ajX = O,XfRn }
depicts the index set of binding constraints, then hj - ajX > 0 for i;' 3. In this
regard,
6.2. THEOREM. The vector hfGJl(Xo) (Xo optimal) if and only if
-ajh ~ 0, if l
PROOF. (necessity) Let XodG with ajXo = hi' if3. For hfGJl(Xo),
ai(Xo + th) :5 hi' 0:5 t < 6, for some i. If if 3, aiXo = hi and thus taih:5 0 or
- aih ~ 0, t > 0, for all if 3. (sufficiency). Let - aih ~ 0 for all if 3. Then for
some i;' 3, aiXo < hi and thus
for all t ~ O. If if 3, aiXo = hi so that

a.(Y+th)
I ~~ = h. + ta·h < h·
I I - I
for each t ~ O. Q.E.D.

To interpret this theorem we note that - a~ may be characterized as an

inward pointing or interior normal to - aih = 0 (or bi - aiX = 0) at Xo. So if h
is a feasible direction, it makes a nonobtuse angle ( ~ 7r/2) with all of the interior
normals to the boundary of % at Xo. Geometrically, these interior normals or
gradients of the binding constraints form a finite cone containing all feasible
directions making non obtuse angles with the supporting hyperplanes to % at Xo.
Such a cone is polar to the tangent support cone GJl(Xo) and will be termed the
polar support cone GJl(Xo)+ (Figure 6.2.a). Thus GJl(Xo)+ is the cone spanned by
the gradients - a~ such that for h € GJl(Xo)' - aih > 0, i d. Looked at from
another perspective, a~ may be considered an outward pointing or exterior normal
to the boundary of % at Xo. In this instance, if h is a feasible direction, it must
now make a nonacute angle ( ~ 7r /2) with all of the outward pointing normals to
the boundary of % at Xo. Again looking to geometry, the exterior normals or
negative gradients of the binding constraints form a finite cone containing all
feasible directions making nonacute angles with the hyperplanes tangent to % at
Xo. This cone is the dual of the tangent support cone GJl(Xo) and is termed the
dual support cone GJl(Xa)* (Figure 6.2.b). Thus GJl(Xo)* is the cone spanned by
the negative gradients a~ such that for all h € GJl(Xa), aih ~ 0, i d.
-- --
3X=b
r r
(tangent hyperplane)
-- -
3SX = bs
" (tangent hyperplane)
"
L-------------------------L-----------~Xl
if hE GJ)(Xo) , then - 3ih ~ 0, if 3
a.
3r X = br
...,. .,..... (tangent hyperplane)
".......
3 SX =
bs
L-----------------------L-------------~Xl
if hE GJ)(Xo) , then 3ih :S 0, if 3
h.
Figure 6.2
At this point let us collect our major results. We found that if f(X) subject
to hj - ajX = 0, i d, attains its maximum at Xo, then
C'h ~ 0 for all h satisfying ajh ~ 0, i d, h fc:D(Xo). (6.1)
How may we interpret this condition? Given any h € c:D(Xo) , (6.1) holds if the
gradient of f or C lies within the finite cone spanned by the exterior normals a~,
i d, i.e., C € c:D(Xo) * (or if - C is contained within the finite cone generated by
the interior normals -a~, id, i.e., -C€c:D(Xo)+). Hence (6.1) requires that the
gradient of f be a nonnegative linear combination of the negative gradients of the
binding constraints at Xo (Figure 6.3). In this regard, there must exist real
numbers u? ~ 0 such that
C " -0 -, . Q
(6.2)
L..,J ui ai' ~€ J.
I
a~ C
arX = hr
C'(X-Xo) = 0
---~~~---- (tangent hyperplane)
-- "
"SaX=h
(tangent
B
hyperplane)
L-----------------------~~----------~Xl
C€ c:D(Xo)*
Figure 6.3
Under what conditions will the numbers u? ~ 0, i d, exist? To answer this

question let us employ the
6.3. MINKOWSKI-F ARKAS THEOREM. A necessary and sufficient con-

dition for the n-component vector V to lie within the finite cone
spanned by the columns of the (m x n) matrix B is that V'Y $ 0 for
all Y satisfying B'y ~ 0, i.e., there exists an n-component vector ~ ~ 0
such that B~ = V if and only if V'Y $ 0 for all Y satisfying B'y ~ o.
PROOF. (sufficiency) if B~ = V, ~ ~ 0, then ~'B' = V' and ~'B'Y =
V'y $ 0 for all Y for which B'y ~ O. (necessity) If V'Y $ 0 for each Y satisfying
B'y ~ 0, then the (primal) linear programming problem
max f(Y) = V'y s.t.
B'y ~ 0, Y unrestricted
has as its optimal solution Y = O. Hence the dual problem
min g(~) = O'~ s.t.
B~=V, ~~ 0
also has an optimal solution by the duality theorem presented above. Hence
there exists at least one ~~ 0 satisfying B~ = V. Q.E.D.
Returning now to our previous question, in terms of the notation used
above, if: C = Vj the vectors ai, i € 3, are taken to be the columns of B (i. e.,
B = - A')j and the u?s, i € 3, are the elements of ~ ~ 0, then a necessary and
sufficient condition for C to lie within the finite cone generated by the vectors ai
is that C'h $ 0 for all h satisfying aih $ 0, i € 3. Hence there exist real numbers
u? ~ 0 such that (6.2) holds.
We may incorporate all of the constraints (active or otherwise) b; - aiX = 0,
i=1, ..., m+n, into our discussion by defining the scalar u? as zero whenever
bi - aiX > 0 or i i 3. Then (6.2) is equivalent to the system
m+n C- A'Uo = 0
C- E uqa~=O
i=1 I I U~(b _ AXo) = 0
u?(bi - aiXo) = 0
or (6.3)
b.-a.X
I I o>
-
0 } ;=l, ...,m+n
Xo unrestricted Xo unrestricted
h
were - '0 = (-0
U -0 )
U 1 ,···, U m +n • Note that u?(1ii-aiX) = 0 for all i values since if
bi-aiX > 0, then u? = 0 while if bi - aiX = 0, then u? ~ O.
When we formed the structural constraint inequality b - AX f; 0 the n
nonnegativity conditions X f; 0 were treated as structural constraints. That is,
Xj~O was converted to bm+j-am+jX=ejX~O, j=l, ... ,n. However, if the
nonnegativity conditions are not written in this fashion but appear explicitly as
X f; 0, then (6.3) may be rewritten in an equivalent form provided by the
6.4. KARUSH-KUHN-TUCKER THEOREM FOR LINEAR PROGRAMS.
The point Xo f Rn solves the primal problem
maxf(X) = C'X s.t.
b - AX f; 0, X f; 0
if and only if
(a.l) C - i=1 u9
I
E
(i::;
1_
0 or
m
(a) C-A'Uo ~ 0 Cj -.~ u?aij ~ 0, j=l, ... , n
n 1-1 m
(b) ~(C-A'Uo) = 0 (b.l) E x~(c ·-E u9a .. ) = 0
;=1) ) i=1 I I)
m -0
(c) U~ (b-AXo) = 0 or (c.l) .E ui(bi-ltiXo) = 0
1=1
(6.4)
(d)b-AXof;O (d.l) bi-ltiXo ~ 0 }.
(e. 1) Ui0 0 z=l, ... ,m
(e) Uo~O ~
(f) Xo ~ 0 (f.l) X~ ~ O,j=l, ... ,n
where Uo is an (m xl) vector with components u~, ... , U~. 2
PROOF. (necessity). To solve the primal problem let us employ the

technique of Lagrange. In this regard, we first convert each of the inequality
constraints to an equality by subtracting from its left-hand side a nonnegative
slack variable. (Actually, since we require that each slack variable be
nonnegative, its square will be subtracted from the left-hand side to insure its
nonnegativity.) That is, bi - ltiX ~ 0 is transformed to bi - ltiX - X!+i = 0,
i=l, ... , m, while x j ~ 0 is converted to x; - x~+m+ j = 0, j=l, ..., n, where X~+i'
2Here (6.4) (as well as (6.3» represents the Karush-Kuhn-Tucker conditions for a
maximum of a function subject to inequality constraints. On all this see Panik (1976).
i=l, ... ,mj z~+m+j' j=l, ...,n, are all squares of slack variables. Hence the
primal Lagrangian appears as
L(ZI"",zn' Zn+l"'" Zn+m' Zn+m+l""'Z2n+m'U1""'Um'V1""'Vn)

n m n
= ?:
3=1
CjZj+ ~ uj(bj-ajX-xn+j) + ?: Vj(Xj-Zn+m+j)'
,=1
2
3=1
2
where U1' ..., Um' VI" .. , Vn are Lagrange multipliers. In matrix terms, the primal
Lagrangian may be written as
L(X,X~I,X~2' U,V) = C'X+U'(b-AX-X~1)+V'(X-X~2)'
where
Here X~1 and U are of order (m xl) while X~2 and V have order (n xl). Then
(a) 'V xL = C - A'U + V = 0
(b) Lx . =2uixn+i = O,i=l, ...,m

n+1
(c) Lx . = 2vjxn+m+j = 0, j=l, ... ,n (6.5)

n+m+3
(d) 'VuL b-AX-X~1 = 0
(e) 'V vL = X- X~2 =0

Let us first transform (6.5.b, c) to U'X~1 = 0, V' X~2 = ° respectively. Then from
U'X~1 = 0 and (6.5.d) we obtain, at Xo and UO' U~(b-AXo) = 0. That is, if the
constraint bj - ajX ~ 0 is not binding at XO' then x~+j > 0, u? = 0 while if it is
binding, z~+j = 0, u? ~ 0, i=l, ... , m. Hence, at least one of each pair
m
(u?,bj-ajXo) vanishes, thus guaranteeing that u?(bj-ajXo) = O. Next, com-
L
j=1
bining V'X~2 = 0 and (6.5) yields X'V = O. From (6.5.a) we have - V = C - A'U.
Then, at Xo and UO' these latter two expressions yield ~(C-A/UO) = °or L x~
n
(Cj- f
,=1
= =
u?ajj) O. In this regard, either z~ 0, allowing Cj to be les/t~an
mo mo ..
?: Uj ajj, or Cj - ~ Uj ajj = 0, ill which case x j may be positive. To see this let
0 ••
3=1 ,=1
us assume, without loss of generality, that the first k < m structural constraints
are binding at Xo. Then bi - aiXo = O(X~+i = 0, u? 2: 0), i=I, ..., k, while
bi - aiXo > 0 (X~+i > 0, u? = 0), i=k+l, ... , m. Now, if the primal problem
possesses a solution at Xo, C'h:S 0 for vectors h which do not violate aih = 0,
i=I, ... , k. If we multiply each equality aih = O. by some
k
constraint u? and form the sum C'h - E u?aih :S 0, we ultimately obtain
i=1
(c- f=
%=1
u? a~)' h :S O. (6.6)
Let us further assume that at least one component of Xo, say x~, is strictly
positive (and thus x~+m+£ > 0, v~ = 0). Then there exists a set of sufficiently
small positive and negative deviations he = b, - b such that, for he = 6 > 0, (6.6)
becomes
k
ce - L u? aie:S 0 (6.7)
i=1
while for he = -6, (6.6) may be written as
k
ce- L u?aie 2: 0 (6.8)
i=1
whence, upon combining (6.7), (6.8),
k
ce-Lu?aie O.
i=1
If x~ =0 (so that xn+m+e = 0, v~ > 0), he can only be positive. Hence
h£ = b > 0 in (6.6) yields
In general then,
m
C- L u?a~:S O.
i=1 .
(sufficiency) For X€%, let us express the Lagrangian of f as

C'X:S C'X+ Ub(b-AX) + VbX = l.(X, Uo,Vo)'
With l.(X, Uo' Vo) concave in X over %, the linear approximation to 1. at Xo is
L(X, UO' Vo) ~ L(X, Uo,Vo) + V X L(X, UO' Vo)'(X-Xo)
C'Xo + (C-A'Uo+Vo)'(X-Xo)
C'Xo .3 (6.9)
Hence C'Xo ~ L(X, UO' Vo) ~ C'X, X € X, and thus Xo solves the primal problem.
Q.E.D.
We note briefly that if the primal problem appears as
min f(X) = C'X s.t.
AX~b, X~O
then (6.4) becomes

(a) C-A'U ~ 0
(b) Xh(C-A'U) o
(c) Uh(b-AXo) o
(6.4.1)
(d) b-AXo~O
(e) Uo~O
(f) Xo ~ 0
It is certainly not coincidental that the Lagrange multipliers appearing in

(6.2), (6.3), and (6.4) (6.4.1») are denoted as "u's," for they are the dual
variables in the dual linear programming problem. To formalize this
observation, let us examine the following corollaries to the preceding theorem.
6.5. COROLLARY. Let (6.4) hold so that the primal maximum problem
possess an optimal solution. Then the dual problem
ming(U) = b'U s.t.
A'U ~ C, U ~ 0
also has an optimal solution Uo with C'Xo = b'Uo
PROOF. From (6.4.a, e) A'Uo ~ C and Uo ~ 0 so that Uo is a feas-
ible solution to the dual problem. With AXo ~ b, XbA' ~ b' or XbA'Uo ~ b'Uo'
3Since L(x, Uo' Vol is linear in X, it is concave over X. Moreover, with L concave, it
lies everywhere on or beneath its tangent hyperplane at Xo as (6.9) indicates.
Since A'UO~ c, XbC = C'Xo ~ b'Uo for Uo feasible. Moreover, from (6.4.b, c),
XbC - XbA'Uo = Ubb - UbAXo or C'Xo = b'Uo since XbA'Uo = UbAXo. Hence
Uo must be an optimal solution to the dual problem. Q.E.D.
It should be evident that corollary 6.5 is a restatement of the strong duality
theorem 4.8. Based upon this result we may now state
6.6. COROLLARY. The optimal value of the ith dual variable u? is
generally a measure of the amount by which the optimal value of the
primal objective function changes given a small (incremental) change
in the amount of the ith requirement bi with all other requirements
bt , t =1= i, i=1, ... , m, remaining constant, i.e.,
(6.10)
PROOF. Let Xo,U o be feasible solutions to the primal maximum and

dual minimum problems respectively with 1° = C'Xo, gO = b'Uo' If (6.4) holds,
then (from corollary 6.~ 1° = gO, i.e., Xo and Uo are optimal solutions and thus
I °= ~
~ biui°so that 81
ab. = ui'° Q.E.D .
•=1 •
So if the right-hand side bi of the ith primal structural constraint were
changed by a sufficiently small amount E i' then the corresponding change in the
maIo
optimal value of I, 1°, would be dlo = u? E i' Moreover, since dlo = l: 8b. db i
m .=1 •
= E u?db i , we may determine the amount by which the optimal value of the
i=1
primal objective function changes given small (marginal) changes in the right-
hand sides of any specific subset of structural constraints. For instance, let us
assume, without loss of generality, that the right-hand sides of the first t < m
primal structural constraints are increased by sufficiently small increments
E 1'"'' E t respectively. Then the optimal value of the primal objective function
t
would be increased by dlo = u? E i' E
i=1
Throughout the preceding discussion the reader may have been wondering
why the qualification generally was included in the statement of corollary 6.6.
The fact of the matter is that there exist values of, say bk , for which ~C is not
defined, i.e., points were ~t is discontinuous so that (6.10) does not hold. To
k
develop this argument 4 let us express f in terms of bk as
f(b k ) = m~x{C'XllriX ~ b?, if:. k, i=l, ... , mj lrkX ~ bk , X ~ O},
where the b?'s depict fixed levels of the requirements bi' if:. k, i=l, ... , m. Clearly
f must be non-decreasing in bk since increasing the value of bk expands the size
of the feasible region.
Moreover, the function f(b k ) is concave. In fact, this assertion is part of a
more general result pertaining to the optimal value function
f(b) = max{C'XIAX::; b, X ~ O}.
(Here f(b) denotes the maximum of the primal objective as a function of the
requirements vector b.) Specifically,
6.7. THEOREM. The optimal value function f(b) is concave.
PROOF. For b 1 , b 2 f R m , let be = ObI + (1-0)b 2, 0 ~ 0 ~ 1. If either
f(b 1 ) or f(b 2) equals -00, then trivially f(b e ) ~ Of(b 1 ) + (l-O)f(b 2). Hence we
shall assume that both f(b 1 ), f(b 2) are greater than -00. If either f(b 1 ) or
f(b 2) equals +00, then the dual problem is infeasible so that f(b) = +00 for all
bfRm •
Next, let both f(b 1 ) = C'X1 , f(b 2) = C'X2 be finite for primal feasible Xl'
X2 • Then for Xc = OX 1 + (1-0)X2' 0 ~ 0 ~ 1, we have AXe ~ be and thus, from
the definition of the optimal value function, f(b e ) ~ C'Xe = Of(b 1 ) + (1-0)f(b 2)
as required. Q.E.D.
If in Figure 6.4 we take area OABCG to be the initial feasible region when
bk = bk (here the line segment CG consists of all feasible points X which satisfy
lrkX = bk), then increasing the eh requirement from bk to bl: changes the optimal
extreme point from C to D. The result of this increase in bk is a concomitant
increase in fa proportionate to the distance E k = bl: - bk, where the constant of
proportionality is uZ. Thus, over this range of bk values, ~{: exists (is
4The discussion which follows has, in part, been adapted from the developments
presented in Balinski, Baumol (1968) and Williams (1963).
b'"
k
o ~----------------+-~+---------------~Xl
Figure 6.4
continuous). If we again augment the level of bk by the same amount as before,

this time shifting the kth structural constraint so that it passes through point E,
(){)bfO becomes d ··
lscontmuous .
smce now t h · Ipomt
e maXlma · moves a Iong a more
k
steeply sloping portion (segment EF) of the boundary of the feasible region so
that any further equal parallel shift in the kth structural constraint would
produce a much larger increment in the optimal value of f than before. Thus
each time the constraint line (tkX = bk passes through an extreme point of the
· · . .m af o
leasl·bl e reglOn,
I: . t here occurs a d
lscontmUlty ab k .
To relate this argument directly to f(b k ) above, let us describe the behavior
of f(b k ) as bk assumes the values bk, b'k, b~, bk' and b~v. In Figure 6.5 the
piecewise continuous curve f(b k ) indicates the effect on f(b k ) precipitated by
changes in bk • (Note that each time the constraint (tkX = bk passes through an
extreme point of the feasible region in Figure 6.4, the f(b k ) curve assumes a kink,
i.e., a point of discontinuity in its derivative. Note also that for a requirements
level of bk > b'k' (say b~V) f cannot increase any further so that beyond F', f(b k )
is horizontal.) As far as the dual objective function g(U) = b'U is concerned, g
m
may be expressed as g(b k ) = U o + bkuk' where U o = E biui = constant is
i#
independent of bk . For bk = b~ (b~ optimal), f(b2) = g(b2). Moreover, since the
dual structural constraints are unaffected by changes in bk , dual feasibility is

preserved for variations in bk away from b~ so that, in general, for any pair of
feasible solutions to the primal and dual problems, f(b k ) ~ g(b k ), i.e., g(b k ) is
tangent to f(b k ) for bk = b~ (point E' in Figure 6.5) while f(bZ + E k) <
g(bZ + E k) for E k( ~ 0) sufficiently small. In this regard, for E k-+0+,
8g(b~)/8bk = uZ > ;f(b~)/8bk while, for E k-+O-, u~ < ifi(b~)/8bk whence
+ 0 0 - 0
8f(b k )/8b k < uk < 8f(b k )/8b k·
f(b~)=g(b~)
I
I I
tJ7(b~) ....... _...! ____ :...-~-I---r
+~ ~ _,: ____ LI __ ~-~-~
I
tJf(bo) I : I I I
__ k ~-'----I--;--r -I~----f
8b k I I I 1
b" bk b2
Figure 6.5
So while both the left- and right-hand derivatives of f(b k ) exist at b~, 8f(b~)/8bk
itself does not exist since ;f(b~)/8bk :f: 8-f(b~)/8bk i.e., 8f /8b k possesses a finite
discontinuity at b~. In fact, 8f /8b k is discontinuous at each of the points A', B',
+
E' and F' in Figure 6.5. For bk = b~, it is evident that 8f(b~)/8bk = u~ =
tJf(b~)/8bk since f(b k ), g(b k ) coincide all along the segment B' E' of Figure 6.5.
In this instance, we many unequivocally write 8f(b~)/8bk = u~. Moreover, for
bk = bLv (see Figures 6.4, 6.5), 8f(bLV )/8b k = uLv = ° since the constraint
akX = bLv is superfluous, i.e., it does not form any part of the boundary of the
+ -
feasible region. In general then, 8f /8b k ~ Uk ~ 8f /8b k.
To formalize the preceding discussion we state
6.S. THEOREM. Let the requirements bi be held fixed at b?, i=1, ..., m.
If f(b k ) = m~x{C'X I aiX ~ b?, if:. k, i=1, ... , mj akX ~ bk, X ~ O}
possesses finite right- and left-hand derivatives at an optimal point
bk = b~, then u~, the optimal value of the corresponding dual variable,
lies between these derivatives, i.e.,
+
8f(b k )
0
8f(b k ) 0
- 0
~ ~ Uk ~ 8b k for all k .
. + 0 -
Moreover, If 8f(b k )/8b k = 8f(b k )/8b k so that
0
o . 8f(b~) 0
8f(b k )/8bk eXists, then ~ = Uk.
k
PROOF.
m~x{ C'X I aiX ~ b?, i f:. k, i = 1, ... , mj akX ~ b~, X ~ O} for b~ optimal (see, for
instance, point E of Figure 6.4). And since f(b~ + E k) < g(b~ + E k)' E k
unrestricted in sign and sufficiently small, it follows that, for E k > 0,
g(b~ + E k) - g(b~)
Ek
and thus
f(b~+ E k) - f(b~)
u~ > lim
E k-+o+ Ek
while for E k < 0,

so that
Hence
theorem holds by definition. Q.E.D.
- +
To summarize: if both the left- and right-hand derivatives of lobi' of/obi
+
exist and their common value is u?, then afO I obi = u?; otherwise of I obi ::;
o - .
ui ::; aflabi' z=l, ... ,m.
An additional source of difficulty in interpreting u~ as of0 lab k emerges
when the optimal basic feasible solution to the primal problem is degenerate. In
this instance the dual possesses multiple optimal basic feasible solutions, with
f(b) nondifferentiable at such points. More specifically, alternative optimal dual
basic feasible solutions all correspond to alternative bases associated with the pri-
mal degenerate basic feasible solution Xo. Hence u~ is not uniquely determined.
If Xo represents a nondegenerate optimal basic feasible solution to the
primal problem, then, according to the (weak) complementary slackness condi-
tions, the dual structural constraints associated with the primal basic variable
columns of A must hold as a set of equalities or U'B = CB- Hence we obtain, as
the unique optimal dual basic feasible solution, U' = CBB- 1 with u~ = of0 lab k
+0 -0
(=af lab k = of lab k )·
While f(b) is nondifferentiable at points where there exist multiple optimal
dual solutions, it does have directional derivatives which exist everywhere. Such
derivatives can be derived from the set of optimal dual solutions and are defined
as: the directional derivative of f at b in the direction h ( i= 0) ERnI, if it exists, is
Dhf(b) = lim f(b+thi - f(b) .
t-+O+
To see this let us assume that 9G D = {U I A'U ~ C, U ~ O} i= ¢. Hence the dual

problem is assumed to posses a feasible solution so that the primal objective is
not unbounded for any b. By the strong duality theorem, f(b) = min {b'U}
UE9G D
(possibly f(b) = -00). By a suitable modification of existence theorem 3.1 to

address the dual problem, f(b) attains a finite minimum if b'dj ~ 0 for all ex-
treme directions dj ,j=I, ... ,e, of %D. Moreover, f(b) = b'Ur for any extreme
points of 9G D such that b'U r = min{b'U s }, where the index s varies over all
extreme points of 9G D.
In this regard, consider a bERm such that f(b) is finite. Then h( =f. O)ERm
is a feasible direction of f at b if f(b+th) is finite for all 0 ~ t ~ o. Since f(b) is
finite and concave, a feasible direction h will always exist at a point b if we can
find at least one other point in R m which yields a finite value of f. Hence
6.9. THEOREM [Shapiro (1979)]. Let f(b) = min {b'U} be finite with
UE%D
h( =f. 0) a feasible direction of f at b. Then the directional derivative
of f at b in the direction h exists and is given by Dhf(b) =
min{h'U, U {~}, where ~ is the set of optimal dual extreme points.
When ~ is the singleton {Uo} ERm (the optimal dual extreme point is unique),
then Dhf(b) = h'Uo for all feasible directions h. In addition, if the set of feasible
directions is all of Rm\O, then f is differentiable and O/o/Obi = u?,i=l, ... ,m
(Shapiro (1979)).
EXAMPLE 6.1. Using the Karush-Kuhn-Tucker conditions specified above,
let us determine an optimal solution to the linear program
max / = Xl + 3x 2 + 4x3 s.t.
2xI + 2xI + x 2 + 4x3 ~ 100
Xl + 3x 2 + x3 ~ 40
x I ,x2,x3 ~ o.
For
2 1 4 100
[.~.-13 ]
1 3 1 40
A -1 0 0 ,i) = 0
0 -1 0 0
0 0 -1 0
we have, from (6.3),

U~(h-Axo)
+ -uoxo
31
+ -uoxo
42
+ -uoxo 0
53= , (6.3.1)
100 - 2x~- xo2 - 4xo

3
-0
u l
40- ° °
xl - 3X 2 - xo
3
-0
U2
h-AXo xo
1 ~ 0, Uo -0
U3 ~ 0,
xo -0
U4
2
xo3 -0
U5
If i = 2,3, 5 d, (6.3.1) may be rewritten as

(a) 1 - ug + ug 0
(b) 3 - 3ug 0
(c) 4 - ug + u~ 0
0(100-2x~-xg-4xg) + ug(O) + ug(O) + O(xg) + u~(O) o (6.3.2)
(d) 100-2x~-xg-4xg>0, u~ = 0
(e) 40 - x~ - 3xg - xg = 0, 2: ug > 0
(f) x~ = 0, ug 2: 0
g x o2 > 0-°0
() ,u4 =
(h) xg =0, u~ 2: O.
Using (6.3.2.a, b, c) we obtain ug = 1, ug = 0 and u~ = -3. Moreover, (6.3.2.e, f,
h) yield xg = ~O while (6.3.2.d, f, h) inform us that xg < 100. Since (6.3.1) is
violated (i.e., u~ = -3 < 0), we are prompted to test additional points for
optimality.
For i = 1,2,3d, (6.3.1) becomes
(a) 1 - 2-U01 + -0
U 2 +-0
U3 = 0
(b) 3 - -0
U 1 - 3-U02 0
(c) 4 - 4-U01 - U-02 0
u~(O) + ug(O) + ug(O) + O(xg) + O(xg) o (6.3.3)
(d) 100-2x~-xg-4xg = 0, u~ > 0
(e) 40 - x~ - 3xg - xg = 0, ug ~ 0
(f) x~ > 0, ug ~ 0
g xo
() 0-00
2 > , u4 =
(h) x3>
0 0, -0
Us =0 .
u~ = 191' ug = 181' A substitution of these values into (6.3.3.a)
Using (6.3.3. b, c)
renders ug = ~f. Additionally, from (6.3.3.d, e, f), xg = ~~, xg = 216 Finally, at
substitution of this latter set of values into f yields fO = 1ii O• By systematically
choosing other combinations of values of i € 3 one can generate additional solu-
tions to the above problem. However, the one exhibited by (6.3.3) maximizes f
subject to the constraints.
6.2. Lagrangian Saddle Points and Primal Optimality

If we structure the primal-dual pair of problems as
PRIMAL PROBLEM DUAL PROBLEM

max f(X) = C'X s.t min g(U) = b'U s.t.
AX~b, X ~O A'U ~ C, U ~ 0
then the Lagrangian associated with the primal problem is L(X, U) =
C'X + U'(b-AX) with X ~ 0, U ~ O. And if we now express the dual problem as
maximize {-b'U} subject to -A'U ~ C, U ~ 0, then the Lagrangian of the dual
problem is .Ab(U, X) = -b'U + X'( -C+A'U), where again U ~ 0, X ~ O. Moreover
, since .Ab(U,X) = -b'U - X'C + X' A'U = -C'X - U'b + U' AX = -L(X, U), we
see that the Lagrangian of the dual problem is actually the negative of the
Lagrangian of the primal problem. Alternatively, if x j ~ 0 is converted to
bm+j - am +jX = ejX ~ 0, j=l, ... , n, then the revised primal problem appears as
max f(X) = C'X s.t.

AX ~ b, X unrestricted
with associated Lagrangian L(X, V) = C'X+ V(b-AX), where V' = (ul, ...,u n +m )
~ 0'. As far as the dual of this revised problem is concerned, we seek to
min 9 (V) = b'V s.t.
A'V = c, V ~ o.
In this instance ..Ab(V, X) = -b'V + X'( -C + A'V) -L(X, V) also.
We next turn to the notion of a saddle point of a Lagrangian. Specifically,
a point (Xo, Uo) £ Rn +m is termed a saddle point of the Lagrangian L(X, U)
C'X + U'(b-AX) if
(6.11)
for all X ~ 0, U ~ O. Alternatively, (Xo, Vo) f. R2n +m is a saddle point of

L(X, V) = c'x + V' (b-AX) if
(6.12)
for all X unrestricted and it ~ o. What these definitions imply is that L(X, U)
(L(X, V») simultaneously attains a maximum with respect to X and a minimum
with respect to U (V). Hence (6.11), (6.12) appear respectively as
L(Xo, Uo) = max{minl.(X, Un = min{maxL(X, un, (6.11.1)

x~o u~o u~o x~o
max{?:!lin L(X, V)} win{max L(X, V)}, X unrestricted.

X u~o u>o X
= (6.12.1)
Under what conditions will L(X, U) (L(X, V») possess a saddle point at
(Xo, Uo) ((Xo, Vo))? To answer these questions we must look to the solution of
what may be called the
SADDLE POINT PROBLEM. To find a point (Xo, Uo) such
that (6.11) holds for all (X, U)f.Rn+m, X ~ 0, U ~ OJ or, to find a point (Xo, Vo)
such that (6.12) holds for all (X, V)f.R2n+m, X unrestricted, V ~ O. In the light
of (6.11.1), (6.12.1) we shall ultimately see that determining a saddle point of a
Lagrangian corresponds to maximinimizing or minimaximizing it. To this end

we look to
6.10. THEOREM (A Necessary and Sufficient Condition). Let L(X, U) be
defined for all (X, U)fR2n +m , where X is unrestricted and U ~ o.
Then L(X, U) has a saddle point at (Xo, Uo) if and only if
(a) L(X, Uo) attains a maximum at Xo,
(b) b-Axo ~ 0 (6.13)
(c) U~(b- AXo) = O.
PROOF. (necessity) The left-hand inequality of (6.12) is equivalent
to (6.13.a) while it is right-hand inequality implies that C'Xo + U'(b-Axo)
~ C'Xo + U~(b- Axo) or (U - uo)' (b - Axo) ~ o. If b - Axo ~ 0, a U may be
chosen sufficiently large so that U - Uo ~ 0 or (U-Uo)'(b-Axo) ~ 0 whence
L(Xo, U) ~ L(Xo, Uo) is violated. Hence (6.13.b) must hold. If U = 0, then
(U-Uo)'(b-AXo) = U~(b-Axo) ~ O. But since Uo ~ 0, b - AXo ~ 0 together
imply U~(b-Axo) ~ 0, we see that (6.13.c) must hold as well. So if L(X, U)
assumes a saddle point at (Xo, Uo), (6.13) holds.
(sufficiency) If Xo maximizes L(X, Uo), it then follows that L(Xo,
Uo) ~ L(X, Uo). If U~(b-Axo) = 0, then L(Xo, Uo) = C'Xo· Hence L(Xo, U)
= C'Xo + U'(b-AXo) = L(Xo, Uo) + U'(b-AXo) or L(Xo, U) ~ L(Xo, Uo)
since U'(b-AXo) ~ 0 if b - AXo ~ o. So if (6.13) is satisfied, L(X, U) has a
saddle point at (Xo, Uo). Q.E.D.
Our discussion in this chapter has centered around the solution of two
important types of problems - the primal linear programming problem and the
saddle point problem. As we shall now see, there exists an important connection
between them. Specifically, what does the attainment of a saddle point of the
Lagrangian of the primal problem imply about the existence of a solution to the
primal problem? To answer this we cite
6.11. THEOREM (A Sufficient Condition). Let L(X, U) be defined for all
(X, U)f R 2n+m, where X is unrestricted and U ~ o. If (Xo, Uo) is a
saddle point for L(X, U), then Xo solves the primal problem.
PROOF. Let (Xo, Uo) be a saddle point for L(X, U). Then by the previ-
ous theorem (6.13) holds and thus L(Xo, Uo) ~ L(X, Uo) or C'Xo + U~(b-AXo) ~
C'X + U~(b-AX). Since U~(b-AXo) = 0 and U~(b-Ax) ~ 0, C'Xo ~ C'X for
all X. Q.E.D.
The importance of the developments in this section is that they set the
stage for an analysis of the Karush-Kuhn-Tucker equivalence theorem. This
theorem (presented below) establishes the notion that the existence of an optimal
solution to the primal problem is equivalent to the existence of a saddle point of
its associated Lagrangian, i.e., solving the primal problem is equivalent to
maximinimizing (minimaximizing) its Lagrangian. In this regard we look to the
6.12. KARUSH-KUHN-TUCKER EQUIVALENCE THEOREM
(A Necessary and Sufficient Condition). A vector Xo is an optimal
solution to the primal linear programming problem
max f(X) = C'X s.t.
b - AX ~ 0, X unrestricted
if and only if there exists a vector Uo ~ 0 such that (Xo, Uo) is a
saddle point of the Lagrangian L(X, U), in which case
C-A'Uo =0
U~(b- AXo) = 0
b-AXo ~ O.
PROOF. (sufficiency) Let L(Xo' Uo) solve the saddle point problem.
Then by theorem (6.10), system (6.13) obtains. In this regard, if Xo maximizes
L(X, Uo) = c'x + U~(b-AX), then
V XL(Xo, Uo) =C - A'Xo = O.
Hence this latter expression replaces (6.13.a) so that (6.13) is equivalent to (6.3),
i.e., if L(X, U) has a saddle point at (Xo, Uo), then (6.3) holds. Thus Xo solves
the primal problem.
(necessity) Let Xo solve the primal problem. Then by corollary 6.5, the
optimality of Xo implies the existence of a Uo ~ 0 such that C'Xo = b'Uo' And
since Uo represents an optimal solution to the dual problem, U~(b-AXo) = o.
Since Xo satisfies b - AXo ~ 0, then for any U ~ 0, it follows that

U/(b-AXa) ~ O. Thus
L(Xa, Uo) = C'Xa + U~(b-AXo) ~ C/Xa + U'(b-Axo)·

Hence the right-hand side of (6.12) is established. To verify its left-hand side we
note that since C - A/UO = 0 is equivalent to (6.13.a), L(Xa, Uo) ~ L(X, Uo) by
the sufficiency portion of theorem 6.10. Q.E.D.
It is instructive to analyze this equivalence from an alternative viewpoint.
We noted above that solving the primal linear programming problem amounts to
maximinimizing (minimaximizing) its associated Lagrangian. In this regard, if
we express the primal problem as
max f(X) = C'X s.t.
b - AX ~ 0, X ~ 0,
then if L(X, Uo) has a maximum in the X-direction at XO' it follows from
(6.4.a, b, f) that
while if L(Xo' U) attains a minimum in the U-direction at Uo, then from

(6.4.c, d, e)
- V' uL 0 ~ 0 or V' u L0 ~ 0
(V' uLo),Uo = 0
Uo~ 0
Thus Xa is an optimal solution to the preceding primal problem if and only if

there exists a vector Uo ~ ° such that (Xo' Uo) is a saddle point of L(X, U), in
which case system (6.4) is satisfied.
As a result of the discussion underlying the Karush-Kuhn-Tucker equiva-
lence theorem we have yet another version of the strong duality theorem, namely
6.13. COROLLARY. Let (6.3) hold so that the primal maximum problem
has an optimal solution Xo. Then the dual minimum problem
ming(U) = h'U s.t.
A'U = c, U~0
also has an optimal solution Uo ~ 0 with C'Xo = h'Uo'
PROOF. If Xo solves the primal problem, then by the preceding
theorem there exists a vector Uo ~ 0 such that (Xo, Uo) is a saddle point of
t(X, U) = -.Ab(U, X), whence .Ab(Uo, X) 2: .Ab(Uo, Xo) 2: .Ab(U, Xo), i.e., .Ab(U, X)
attains a maximum with respect to U and a minimum with respect to X and thus
possesses a saddle point at (Uo'Xo). And since U~(h-AXo) = X~(C - A'Uo) = 0,
-,- - UoAXo
.It follows that Uoh -,- =
"-,-
XoC - XoA Uo = 0 or
,-,-
C Xo = h UO. Q.E.D.
A final point is in order. We may relate the preceding material to the
development of the (weak) complementary slackness theorem by expressing the
primal-dual pair of problems as
PRIMAL PROBLEM DUAL PROBLEM

max f(X) = C'X s.t. min g(U) = h'U s.t.
AX ~ h, X unrestricted. A'U = C, U ~ O.
Then their associated Lagrangians may be written respectively as

t(X, U, Xs) = C'X + (h-AX-Xs)'
.Ab(U, X) = - h'U - X'(C-A'U),
where Xs is an (m + n xl) vector of nonnegative primal slack variables, i.e.,
b.-aX>O,
I 1-
i=1, ... ,m·,
~ 0 with bm+j-am+j X
n+m+l e'- X
J
Xj unrestricted, j=I, ... , n.
In this regard we now look to the

6.14. WEAK COMPLEMENTARY SLACKNESS THEOREM.

The vectors Xo, Vo solve the primal and dual problems
max /(X) = C/X s.t. min g(V) = b'V s.t.
Ax ~ b, X unrestricted A'V = c, V ~ 0
respectively if and only if Xo, Vo satisfy the (weak) complementary
slackness conditions
_, _ a -O-O
{() ui xn+i = 0·1
, z= , ... ,m;
Uo(Xs)o = 0 or -0 -0 . (6.14)
(b) um+i xn+m+i = 0, J=l, ... ,n.
PROOF. (necessity) If Xo solves the primal problem, then there

exists a vector Vo ~ 0 such that (Xo, Va) is a saddle point of L(X, V) =
- A(V, X), whence
C/Xo + V~(b-AXo - (Xs)o)

- b' Va - ~(C-A'VO)
-V~(Xs)o = 0
(sufficiency) IfV~(Xs)o = V~(b-AX) = 0, then L(Xo, Va) = C/Xo, A(Vo' Xo)

= - b'Vo = - L(Xo, Va)' and thus C/Xo = b'Vo· Hence Xo, Va yield optimal
solutions to the primal and dual problems respectively. Q.E.D.
By way of interpretation, (6.14.a) informs us that if the primal slack
variable X~+i' i=l, ..., m, corresponding to the ith primal structural constraint is
positive, then the associated dual variable u?, i=l, ... , m, must equal zero.
Conversely, if u?, i=l, ..., m, is positive, then the ith primal structural constraint
must hold as an equality with X~+i = 0, i=l, ... ,m. Before turning to (6.14.b) let
us examine the form of the lh dual structural constraint. Since A= [£r1, ... ,£rn ),
U 0:' U
1
u m +1 um +1
1'0 [A' : - In] -In C
u m +n n
0:'
um+n
and thus the /h dual structural constraint appears as o:jU - um+i = ci'
j=l, ... ,n. In this light, from (6.14.b), if the /h primal variable x~,j=l, ... ,n, is
different from zero, then the dual surplus variable u~J+ i' j=l, ... , n, associated
with the /h dual structural constraint o:jU -um+i =ci,j=l, ... ,n, equals zero.
Conversely, if u~+i > 0, j=l, ... ,n, then the /h structural constraint is not
binding and thus the /h primal variable x~, j=l, ... , n, must be zero.
It is important to note that the weak complementary slackness conditions
are a consequence of the Karush-Kuhn-Tucker theorem for linear programs. That
is to say,
6.15. COROLLARY. If (6.4) holds so that the primal maximum and dual
minimum problems have optimal solutions (Xo, (Xs)o), (Uo, (Us)o)
respectively, then the (weak) complementary slackness conditions
(U s)~Xo + U~(Xs)o = 0
(a) u? X~+i = 0, i=l, ... , m;

or { (6.15)
(b) x~ u~J+i = 0, j=l, ... ,n.
also hold, i.e., for optimal solutions (Xo, (Xs)o), (Uo, (U 8)0) to the
primal maximum and dual minimum problems respectively:
(a) if a primal slack variable x~+i' i=l" .. ,m, is positive, then the ith
dual variable u?, i=l, ... , m, must equal zero. Conversely, if
o z=
ui' ' 1", " ln, IS
. POSI't'Ive, tl1en xn+i'
0 '1 '" " m, equaIs zero;
z=
(b) if the /h primal variable x~,j=l, .. "n, is positive, then the dual
surplus variable u~+ i' j=l" ", n, equals zero. Conversely, if
u0
m+i'' J=
1 '" " n, IS
' positive,
" t hen xi'
0'1J= ,"', n, must equa I zero,
PROOF. If Xs represents an (m x 1) vector of nonnegative primal

slack variables and Us depicts an (n xl) vector of nonnegative dual surplus
variables, then (6.4.b,c) become Xb(-Us)o=UbXs or (Us)bXo+ Ub(Xs)o = O.
Alternatively, (6.5.a) reveals that Us = -yo so that the (weak) complementary
slackness condition may be rewritten in terms of the primal variables and
Lagrange multipliers as Ub(Xs)o - VbXo = O. Q.E.D.
COMPUTATIONAL ASPECTS
1 OF LINEAR PROGRAMMING
7.1. The Primal Simplex Method Reexamined

In this chapter we shall be concerned with the development of a computa-
tionally efficient version of the simplex procedure for solving a primal linear
program. Let us assume that we are faced with the now familiar problem of
determining the maximum of
I(X) = c'x s.t.
AX=b, X~O.
Given that peA) = m, our previous approach was to first extract m linearly inde-
pendent columns from A so as to form an (m x m) non-singular basis matrix B.
We next determined B- 1 and thus
XB B- 1b-B- 1 RX n = b-B-1RX n ,
I(X) = CBB-1b+(Cn-CBB-IR)Xn
CBb+CnX n ·
Upon setting Xn = 0, we then obtained our first basic feasible solution of the
form
X B = b,
I(X) = cBb.
(If B = 1m , then our initial basic feasible solution is just X B = b, I(X) = CBb.)
As an alternative, let us form the following simultaneous system involving m+ 1
equations and the n+l variables X, I, namely
169
AX (7.1)
-C'X+!
Here the objective function is treated as if it were an additional structural

constraint. In this formulation of the primal linear programming problem, we
desire to obtain a solution to (7.1) wherein ! is as large as possible (and
unrestricted in sign) with X ~ O. To consolidate our notation a bit, let
- = r~C'l
A
A0] ,X- [ ~ land bo = [~ l
where A is of order (m+ 1 x n+ 1), X is of order (n+ 1 xl), and bo is of order
(m+1 x 1). Then (7.1) can be treated more succinctly as
(7.1.1)
Since the columns within A contain m+1 components, a basis for R m+I may be
formed by selecting m+ 1 linearly independent vectors from A. Let the said
collection of vectors constitute the columns of the m+ 1st order nonsingular basis
matrix B = [hI"'" bm+I J. Given that the primal objective function is to be
maximized, the variable ! must be a member of every basic solution. In this
regard, since the last column of A corresponds to the variable /, the unit column
vector e m+I will be a member of every basis matrix. If we adopt the convention
of letting e m+I always appear in the last column of the basis matrix, then B
[bI , .. ·, bm , cm +I ].
How does the task of obtaining a basic solution to AX = bo compare with
our previous problem involving the determination of a basic solution to AX = b?
To answer this question let us see exactly how the vectors bi' i=1, ... , m, are to be
selected. Let the first n columns of A be denoted as
(7.2)
where aj is the /h column of A. If we can find m linearly independent vectors b i

within A, then upon substituting them into (7.2), m of the 8.j can correspon-
dingly be represented as
Computational Aspects of Linear Programming 171
- I.
J
b i = [ b. , z=l, ... ,m,
-cBi
where, as before, B = [bl , ..., bml Hence
i.e., since m linearly independent

B=
3j
[-~B ~ l
uniquely determine m linearly independent
aj' there exists a one-to-one correspondence between the basis matrix B for
AX = b and the basis matrix B for AX = boo If the variables associated with the
m + 1 columns of B are denoted as x Bi ' i=l, ... , m, and I, then a basic feasible
solution to AX = bo will be defined as one for which I is as large as possible and
xll j ~ 0, i=l, ... , m. In order to guarantee that the variant of the primal simplex
process to be presented below converges to an optimal basic feasible solution in a
finite number of iterations, let us again impose the nondegeneracy assumption set
out at the beginning of the previous chapter, namely, that every set of m
columns from A is linearly independent.
To determine an initial basic feasible solution to AX = bo' let us form the
(m+1 x n+2) augmented matrix associated with this system as
[A: bo] = [_~, 0

1
b
0 ] (7.3)
Xl X2 Xp X p +! X p +2 X p +m I
all a 12 alp 1 0 0 0 bl
a 2l a22 a2p 0 1 0 0 b2
= ...................................
amI a m 2 amp 0 0 1 0 bm
-C l -C2 -C p 0 0 0 1 0
(For purposes of exposition, the variables corresponding to the first n+ 1 columns
of (7.3) appear directly above this matrix.) Henceforth this augmented matrix
will be referred to as the primal simplex matrix. 1 For example, if we consider
the problem presented at the outset of the previous chapter, i.e., if we desire to
max f = 2XI + 3x 2 + 4x3 s.t.

2XI + X2 + 4x3 ~ 100
Xl + 3x 2 + x3 ~ 40
Xi ? 0, i=l, 2, 3,
then the simplex matrix becomes
1 4 1 o
3 1 o 1 (7.4)
-3 -4 o o
If the basic variables are taken to be those corresponding to the m+ 1 unit col-
umn vectors in (7.3), then we may express these m+1 variables xp+I' •••'x p +m ' f
in terms of the p remaining or nOll basic variables Xl' .• •,X p as
p
Xp+I = bl - aljXj L
j=l
p
Xp 2 + = b2 - L a 2 jXj
j=l (7.5)
p
x p +m = bm - L amjxj
p j=l
f = L CjXj.
j=l
lin the calculations which follow, our first step in the determination of an optimal basic
feasible solution will be to form the simplex matrix. To establish the connection between the
computational algorithm presented in this chapter and our previous results, we shall
demonstrate that the simplex matrix can also serve as a vehicle with which to derive the
representation of X B and f(X) in terms of the nonbasic variables X R , i.e., if we premultiply
[_~, ~ : ]-[-~il ·~R ~ : ] by the inverse of
-_rtCilB0]
B- 1 ,B
--1 _
-
[B-CBB- I
1 ~
IJ'
o B-Ib ]
the resulting product [
1
CBB-1b has, as its first row,
XWB-IRXR = B-Ib or
Our initial basic feasible solution is then determined by setting the non basic
variables in (7.5) equal to zero, to wit x p +! = bl ,xp +2 = b2, ... ,x p +m = bm, and
f = 0. Have we obtained an optimal (maximal) basic feasible solution? To
answer this question let us examine the last equation in (7.5). It is obvious that
f will increase in value if any non basic variable with a strictly positive coefficient
changes from its current zero level by some positive amount. But if any non basic
variable has a positive coefficient in the m+1 st equation in system (7.5), its
associated coefficient in the m+1 st row of (7.3) must be negative. Thus if any
negative values appear in the last or objective function row of the simplex
matrix, the current basic feasible solution is not optimal. In terms of (7.4), if x 4
and x5 are taken to be basic variables with xl' x 2' and x3 deemed non basic, then
x4 100 - 2xl - x 2 - 4x3
x5 = 40 - Xl - 3x 2 - x3
f 2xl + 3x 2 + 4x3
and thus, for Xl = x 2 = x3 = 0, x4 = 100, x5 =40, and f = O. Clearly this initial

basic feasible solution is not optimal since there are three negative entries in the
last row of (7.4). In general, a particular basic feasible solution to the primal
problem is optimal when all of the coefficients on the non basic variables within
the last row of the primal simplex matrix are nonnegative or -c j ~ 0,
j=l, ... , n-m. Moreover, an optimal primal basic feasible solution is unique if
-cj > 0, j=l, ..., n-m. In this regard, if the coefficient on some particular
non basic variable within the last row of the simplex matrix is zero, then if that
variable increases in value to some positive level, no change in f occurs. Hence
the objective function assumes the same value for more than one basic feasible
solution so that the case of multiple optimal solutions obtains. (As the reader
has probably already recognized, the optimality criterion just stated is simply a
variation of the one provided by optimality theorem 5.1.)
Given that our initial basic feasible solution is not optimal, the next section
sets out an iterative scheme which enables us to generate an optimal basic
feasible solution using (7.3) as the starting point.
7.2. Improving a Basic Feasible Solution

Before proceeding directly to the primal simplex matrix, let us first indicate
some of the specific steps that will be executed in our search for an optimal basic
feasible solution. Given that the simplex matrix has within it a set of m+ 1 unit
column vectors, let us consider a set of operations on this equation system which
transforms it into an equivalent system whose associated simplex matrix also
contains m+l unit column vectors. In particular, the following two types of
elementary row operations will transform (7.3) into a new simplex matrix which
is row-equivalent to the latter:
TYPE 1. Multiply any equation Eh by a nonzero scaler k and replace
Eh by the new equation kE h,h=I, ... ,m. (Note that since hi- m+l,
this type of operation may not be performed on the objective function
row of the simplex matrix.)
TYPE 2. Replace any equation E i by the new equation E i+ kE h'
i=I, ...,m+lj h=I, ... ,m.
In actual practice (7.3) will be transformed into an equivalent system of equa-
tions yielding a new basic feasible solution, with an objective function value
which is at least as great as that obtained from the previous one, by performing a
particular sequence of elementary row operations called a pivot operation. Such
an operation consists of an appropriate sequence of at most m+ 1 elementary row
operations which transform (7.3) into an equivalent system in which a specified
variable has a coefficient of unity in one equation and zeros elsewhere. The
implementation of the pivot process then proceeds as follows:
(1) select a pivot term ars i- 0, r ~ m, s ~ p+m = n, within the simplex
matrix (since r i- m+l,s i- n+l,n+2, the elements within the last
row and last two columns of this matrix may not serve as pivot
elements). Next,
(2) replace E r by a;s E r' Finally,
a·
(3) replace Ei by Ei-a;:E r , i=I, ... ,m+lj ii-r.
In matrix terms, a pivot operation is equivalent to premultiplying the simplex

matrix by the m+ 1st order pivot matrix
-alB
1 0 arB
0
o 1
-a2B
a rs 0
p
o o 1
arB
o
o o 1
Given that at least one -C i < 0 in (7.3), our first step will be to determine which
non basic variable is to become basic. We shall adopt the convention of choosing
the nonbasic variable x j' j=I, ... , p, whose associated coefficient -c j within the
objective function row of the simplex matrix is most negative, i.e., if
ENTRY CRITERION: -Ck = min{

i -c·J' -c·J < O} (7.6)
then xk' the non basic variable corresponding to the kth column of (7.3), becomes
basic. In terms of (7.5), such a choice produces the largest increment in f per
unit change in xi' j=I, ..., p. (Note that this new entry criterion is consistent
with our previous one involving (5.11).) Since only one nonbasic variable will be
introduced into the set of basic variables at a time, x1,,,,,xk_l' xk+l'''''x p are
all held fixed at their current zero level. Hence, from (7.5),
X p +! b1 - alk x k
xp 2 + b2 - a 2k x k
(7.5.1)
Xp +m = bm - amkxk
f = ckxk.
As xk increases in value from zero, the new values of x p +i,i=I, ... ,m,
must still satisfy the non negativity conditions, i.e., we require that
Under what conditions will (7.7) be satisfied? If aik ~ 0, then xp+i > 0 so
that, for the moment, we need only concern ourselves with those aik > O. In this
regard, when at least one of the coefficients aik within the kth column of (7.3) is
:ii ~
positive, there exists an upper bound to an increase in xk which will not violate
(7.7). Rearranging (7.7) yields k xk so that any increase in xk which equals
EXIT CRITERION: (j = min{:.i,

• .k
aik > o} = ;rrk (7.8)
preserves feasibility. So when i=r, in terms of (7.3), column k replaces column

p+r in the basis and thus xk becomes a basic variable while x p + r turns nonbasic.
From (7.7), our new basic feasible solution becomes
br
xk = ark'
xp+i = bi - aik ;;k' i=1, ... , m, i =I r, and

f = C br
k ark·
Turning to the simplex matrix, if xk is to replace x p +r in the set of basic vari-

ables, a pivot operation with ark as the pivot element may be performed so as to
express the new set of basic variables xp+i' i=l, ... , m, i =I r, xk' and f in terms
of the non basic variables x1, ... ,xk_I' xk+l' ...'x p ' and x p + r . To this end let us
first divide Er in (7.3) by ark to obtain
1 0 ... 0 ... 0
a rp
1 ••• ark o o ... _1 •.• 0
ark
am,k-l amk am,k+l··· amp 0 o o 1 0 bm

-ck-l -ck -ck+I ••. -c p 0 o o o 1 0
(7.3.1)
Next, to eliminate xk from the remaining m equations E i ,i=1, ... ,m+1, i=J=r, let
us: (1) multiply Er by -aik and replace Ei by the new equation Ei - aikEr'
i=l, ... , m, i=J=r; and (2) multiply E r by ck and replace E m+I by the new equation
Em+I + ckEr. Hence (7.3.1) is ultimately transformed to
1 ar,k+I ... arp

ark ark
0 0 ... _l_
ark
... 0 0 l!!:.
ark
o am,k+I ... amp 0 0·· .am,p+r
o C~+I C~ 0 0 ... C~+r
(7.3.2)
where
arj , 1 '...I-
= aij - aik ark' z= , ... , m, zrr; j=l, ... ,p+m
/j, , br z=
bi - aik ark' • 1 '...l-
, ... , m, zrr
arj .
C', - c J' + ck a-, J=l, ... , p+m.
J rk
(The reader should ponder the connection between the cj's and the S's defined
earlier.) If the basic variables are again taken to be those corresponding to the
m+l unit column vectors in the simplex matrix, then, from (7.3.2), these
variables may be expressed in terms of the remaining or non basic variables as
(7.9)
f
Upon setting the non basic variables equal to zero, our new primal basic feasible
solution appears as
bi' i=l, ... , m, ii-r, and
(Note that this basic feasible solution may be obtained directly from the simplex
matrix by ignoring all columns within (7.3.2) save those corresponding to the
m+1 unit column vectors and the (last) column of constants.) In addition, this
solution is optimal if each cj, j=l, ... , k-1, k+ 1, ... , p, p+r, within the objective
function row of (7.2.3) is nonnegative. If negative coefficients still appear within
the last row of (7.3.2), then the above pivot process is repeated until an optimal
primal basic feasible solution is attained.
We now return to the maximization problem started earlier. Let us
duplicate the associated simplex matrix (7.4) as
1 I±I 1 o o 100 ]100/4 = 25

3 1 o 1 o 40 40/1 = 40 ((7.4))
-3 -4 o o 1 o
Remembering that the basic variables are those corresponding to the m+1(=3)
unit column vectors in this matrix, our initial basic feasible solution consists of
2: 4 = 100, 2:5 = 40, and f = O. Given that there are some negative coefficients
within the last row of (7.4), the basic feasible solution just obtained is not
optimal. To generate an improved solution let us undertake
ROUND 1. Since -4 is the most negative entry in this row, the nonbasic
variable 2:3 now becomes basic. What is the largest increment in 2:3 which
preserves feasibility? To determine this let us take the positive coefficients on 2:3
in the first two rows of (7.4) (values 4 and 1) and divide them, respectively, into
the first two constants in the last column of this matrix (namely 100 and 40).
Carrying out the indicated division to the right of (7.4) yields 25 = min{25,40}
as the maximum allowable increase in x 3 • Hence the (blocked) pivot element
within simplex matrix (7.4) is 4. In the calculations to follow, column three of
(7.4) will be replaced by a unit column vector with unity in the pivot position
and zeros elsewhere. Such a unit column vector lies in the fourth column of
(7.4). Hence ~4 turns non basic in the sense that it ultimately loses its unit
column e1 to x3. In order to achieve this transformation, the pivot operation on
(7.4) proceeds as follows. If El is replaced by iEl' we obtain
1
4 ill 1
4 0 0
25]
[!2 3
-3
1
-4
0
0
1
0
0
1
:0 .
(7.4.1)
Next, replacing E 2, E3 m (7.4.1) by E2 - E l , E3 +4E1 respectively

produces
1
'2
1
4 1 1
4 0 0 25 25/(1/4) = 100
15/(11/14) = 5.46
~
1 1 1 0 15
'2 0 -4
(7.4.2)
0 -2 0 1 0 1 100
Our second basic feasible solution is thus x3 = 25, Xs = 15, and f = 100. Given
that a negative value appears in the last row of (7.4.2), this basic feasible
solution is still not optimal.
ROUND 2. To obtain a maximal solution, let us change the status of x 2
from nonbasic to basic. Since the largest increment in x 2 which does not violate
the nonnegativity conditions is 5.46 = min{100, 5.46}, the pivot element within
(7.4.2) is 141 • This time column two of (7.4.2) is to be replaced by the unit
column vector appearing in column five of the same so that Xs will be removed
from the basis. To this end we now perform a pivot operation on (7.4.2). First,
let us replace E2 by Ii E2 to obtain
[! 25]
1 1
4 1 4 0 0
[I] I 4
0 -IT IT 0 5.46 . (7.4.3)
-2 0 1 0 1 100
Secondly, upon replacing E I ,E3 in (7.4.3) by E I -t E 2,E +2E2 respectively,

3
we ultimately have
[
5
IT 0 1 3
IT -IT
I
o 23.63]
2
IT 1 0 I
-IT IT
4
o 5.46 . (7.4.4)
4 9 8
IT 0 0 IT IT 1 110.91
This particular basic feasible solution consisting of x2 = 5.46, x3 = 23.63, and

f = 110.91 is optimal since only nonnegative entries appear in the objective
function row of (7.4.4).
7.3. The Cases of Multiple Optimal, Unbounded, and

Degenerate Solutions
Let us begin with
EXAMPLE 7.1. Our objective is to
max f = 2xI + 3x 2 s.t.

Xl < 10
x 2 :S 8
~XI + x 2 :S 12
Xl' X 2 2:: O.
For this problem the starting simplex matrix has the form
1 o 1 o o o 10
o 1 o 1 o o 8
2
:3 1 o o 1 o 12
-2 -3 0 0 0 1 0
and our initial basic feasible solution is .1:3=10, .1: 4 =8, .1:5=12, and /=0. Since
this solution is not optimal, we undertake two rounds of pivoting to obtain
3
0 0 1
00 -2 0 4
0 1 0 1 0 0 8
1 0 0 -23 3 0 6
2
0 0 0 0 3 1 36
Given that all negative entries have been purged from the last row of our simplex
matrix, we have obtained an optimal primal basic feasible solution wherein
.1:1=6, .1:2=8, .1:3=4, and /=36.
Further examination of the objective function row of this last simplex
matrix indicates that since the coefficient on the non basic variable .1: 4 in this row
is zero, we may increase .1:4 by some positive amount without affecting the
optimal value of the objective function, i.e., the status of .1: 4 can change from
nonbasic to basic as we move from one extreme point to another and yet / will
still equal 36. To see this let us introduce .1: 4 into the set of basic variables.
Pivoting on the element ~ yields
2 8
0 0 3 1 -1 0 3
16
0 1 -32 0 1 0 "3
1 0 1 0 0 0 10
0 0 0 0 3 1 36
The second optimal primal basic feasible solution is .1: 1=10, .1: 2 =136, .1: 4 =i, and
/=36. Graphically, our first basic feasible solution corresponds to point A in
Figure 7.1 while the second is located at point B. In addition, the set of all
convex combinations of points A and B gives rise to an infinity of optimal
nonbasic feasible solutions.
f = 36
Figure 7.1
To summarize: the case of multiple optimal primal basic feasible solutions

emerges if, in the objective function row of the optimal simplex matrix, a
non basic variable has a zero coefficient. Is there another way of recognizing that
the objective function will be tangent to the feasible region 9G at more than one
extreme point? A glance at the objective function and the structural constraint
inequalities indicates that the optimal objective function equation is three times
the third structural constraint when the latter is treated as a strict equality. In
this regard, when the objective function can be written as a multiple of some
particular structural constraint, the case of multiple optimal solutions obtains.
EXAMPLE 7.2. We desire to
max f = -xl + 2x2 s.t.
- 2xI +x2 ~ 3
-!xI +x2 ~ 6
x I ,x2 2':O.
Our initial simplex matrix appears as
[~n
1 1 0 0
1 0 1 0
-2 0 0 1
and our initial basic feasible solution is x3 =3,and x 4 =6, and /=0. Since
it is not optimal, we ultimately pivot to
[: :l
1 -3"I 4
3" 0
2 2
0 -3" 3" 0
0 0 2 1 12
The absence of any negative entries in the last row of this final simplex matrix
indicates that this particular basic feasible solution consisting of xl =2, x 2=7 and
/=12 is optimal.
We noted earlier that if the objective function coefficient on a non basic
variable in the optimal simplex matrix is zero, that variable (in this case x 3 ) can
be increased from its current non basic level of zero to yield an alternative
solution which has the same value of /. While this alternative solution is also
optimal, it is, in this particular example, non basic, the reason being that the x3
coefficients in the first two rows of the above matrix are negative. To see this let
us write the basic variables xl' x 2 in terms of x3 as
Xl 2 +~x3
x2 7 +!x3 •
In this instance x 3 ' as well as xl and x 2' can be increased without bound so that
an infinity of alternative optimal non basic feasible solutions obtains. For each of
these alternative solutions we have three positive variables (excluding f) instead
of only two, thus violating our requirement for an optimal basic feasible solution.
As Figure 7.2 demonstrates, there exists but a single optimal basic feasible
solution (point A) and an infinity of optimal nonbasic feasible solutions.
Figure 7.2
EXAMPLE 7.3. Let us

max / = 2xl + 3x 2 s.t.
- 2xl + x2 ~ 3
-~xl +x 2 ~ 6
x 1 ,x2 ~ O.
The associated simplex matrix
:]
1 1 0 0
[ =; -3
-2
1 0
0
1
0
0
1
yields the nonoptimal initial basic feasible solution composed of x3=3, x 4 =6, and
/=0. Two successive pivot steps give us the matrix
[: 0
1 -31 34
0 -32 32
7
-3 "3
16
0
0
1 25
:l
Our third basic feasible solution is thus x 1=2, x 2=7, and /=25.
Computational Aspects of Lineal' Programming 185
X2 7 +lx3
Xl 2 +iX3
I = 25 +~x3'
x3 ' as well as x I ,x2 and I, may be increased without limit (Figure 7.3). Hence
we obtain a feasible though nonbasic solution to this problem since the three
variables Xl' x 2' and x3 each assume positive values. In sum, if the only possible
pivotal column within the simplex matrix corresponding to a non basic variable
X RIc has each of its first m components nonpositive and the objective function
coefficient (the m+l at component) - ck is negative, the value of the objective
function is unbounded.
We may relate this observation to our preceding theoretical results
(corollary 5.2) by noting that, from (nonbasic) column three of the preceding
simplex matrix, we may form Y~ = (-l,-i) ~ 0'. Then
dl = [-~1]~0
serves as an extreme direction along which I increases without bound since
C'd, = -C'BY,+cm = -(3'2)[~]+C3 = ~+o = ~ = ",>0.

However, within the simplex matrix, the objective function coefficient on X RI = x3
IS -c = -3 < .
• - 7 0
I
Figure 7.3
EXAMPLE 7.4.
max 1 = 2XI + 3x 2 S.t.
xl + x2 ::; 60
xl + 2x2 ::; 120
2xI + x 2 ::; 80
x I ,x2 ~ O.
The initial simplex matrix
1 1 1 0 0 0 60
1 2 0 1 0 0 120
2 1 0 0 1 0 80
-2 -3 0 0 0 1 0
yields the nonoptimal basic feasible solution x3=60, x4=120, x5=80, and 1=0.
Execution of the primal simplex entry and exit criteria is illustrated in the
following matrix:
1 1 1 0 0 0 60 60
1 ~ 0 1 0 0 120 60
2 1 0 0 1 0 80 80
-2 -3 0 0 0 1 0
Since there exists a tie for the minimum of the ratios to the right of this last
matrix, we have the choice of replacing either x3 or x 4 by x 2 in the set of basic
variables. Let us choose x 4 • Hence the pivot is the second element in the second
column of the above array. Pivoting produces
rn 0 1 -2"I 0 0 0
I I
2" 1 0 2" 0 0 60
3 I
2" 0 0 -2" 1 0 20
1 3
-2" 0 0 2" 0 1 180
This new basic feasible solution involving x 2=60, x3 =0, x5 =20, and 1=180,
while being nonoptimal, is also degenerate since one of the basic variables,
namely x 3 , vanishes. As anticipated by our previous theoretical discussion, the
next round of the simplex process should also yield a degenerate basic feasible
solution with the same value of 1 as at the previous degenerate basic feasible
solution. Pivoting on ! above generates
1 0 2 -1 0 0 0
0 1 -1 1 0 0 60
0 0 -3 1 1 0 20
0 0 1 1 0 1 180
As readily seen from this last simplex matrix, the indicated degenerate basic
feasible solution is optimal at x 1 =0, x 2 =60, x5 =20, and 1=180.
7.4. Summary of the Primal Simplex Method

Let us now summarize the basic rudiments of the above iterative process. If
we abstract from degeneracy, the principal steps leading to an optimal basic
feasible solution may be stated as follows:
STEP o. Initialize.
Form the simplex matrix [A,bo] (7.3») from system (7.1). To
obtain an initial basic feasible solution to AX = bo' let the m
basic variables be those corresponding to the m+l unit column
vectors in the simplex matrix. The values of these variables along
with the objective function value may be determined directly by
examining the last column of (7.3).
STEP 1. Examine the entries in the last or objective function row of the
simplex matrix:
(a) If they are all positive, the current basic feasible solution is
optimal. Next, determine whether any coefficient correspond-
ing to a non basic variable is zero. If all are positive, the
optimal basic feasible solution is unique. If one happens to be

zero, the case of multiple optimal basic feasible solutions
obtains. Determine the alternative basic feasible solution.
(b) If the coefficient on some non basic variable is negative and all
other components in the column associated with this variable
are nonpositive, there exists a feasible solution in positive
variables which is non basic and for which the value of the
objective function is unbounded. In this instance the simplex
process should be terminated.
(c) If one or more of the coefficients on the nonbasic variables are
negative and for each such variable there is at least one
positive component in its associated column, proceed to the
next step.
STEP 2. Locate the most negative coefficient in the last row of the simplex
matrix in order to determine which nonbasic variable is to become
basic.
STEP 3. If the kth non basic variable is to turn basic, take the positive
components of the column associated with this variable and divide
them into their corresponding components in the last column of
the simplex matrix.
STEP 4. Compute min{:,i ,a 'k > o} so as to determine which basic variable
Let this minimum correspond to 8 = ;r .
, ,k I
is to become nonbasic.
rk
STEP 5. Given that the rth basic variable is to become non basic, ark' the
divisor that yields the smallest quotient in step (6), serves as the
pivot element and a pivot operation is performed.
STEP 6. Return to step (1).
By repeating the pivot process outlined above in steps (1) - (6), each iteration
takes us to a new primal basic feasible solution with the new value of the
objective function being an improvement over the old one. The process is
terminated when there are no negative coefficients remaining in the last row of
the simplex matrix.
7.5. Obtaining the Optimal Dual Solution from the

Optimal Primal Matrix
From a computational viewpoint, how are the optimal values of the dual
variables obtained from the optimal primal solution? The simultaneous augmen-
ted form of the above primal maximum problem is
AX+ImXs b
-C'X + O'Xs + f = 0
with simplex matrix
AIm 0 bJ.
[
-C' 0' 1 0
For this system the (m+ 1 x m+ 1) basis matrix and its inverse appear respectively as
B- 1 -
- [ B-B 0]
1
C B- 1 1 '
where B represents the (m x m) basis matrix associated with the primal

structural constraint system. Then
o
1 :J
o
(7.10)
1
yields a basic feasible solution to the primal maximum problem. Assuming that
the optimality criterion is satisfied, the optimal values of the dual variables
appear as the blocked array of elements in the last row of (7.10), i.e.,
U' = Cl1B-l ~ 0'. Moreover, from the augmented dual structural constraint
system A'U - In U8 = C, we may solve for the dual surplus variables as U~
U' A - C'. But this expression is just the double-blocked array of elements U~
Cl1B-lA-C' ~ 0' in the last row of (7.10). In sum, our rules for determining
the optimal values of the dual variables (both original and surplus) may be
stated as follows:
(a) to find the values of the dual variables U which correspond to an optimal
basic feasible solution of the primal maximum problem, look in the last
row of the final primal simplex matrix under those columns which were, in
the original simplex matrix, the columns of the identity matrix. The ith
dual variable ui' i=I, ... , m, will appear under the column which was
initially the ith column of the identity matrix;
(b) to find the values of the dual surplus variables U8 which correspond to an
optimal basic feasible solution of the primal maximum problem, look in
the last row of the final primal simplex matrix under those columns which
were, in the original simplex matrix, the columns of the matrix A. The
/h dual surplus variable um+j,j=l, ... ,n, will appear under the column
which was initially the /h column of A.
EXAMPLE 7.5. Let us
max f = Xl +X2 s.t.
2Xl + X2 < 20
Xl < 8
x2 < 6
Xl' X2 > O.
Given that x 3 , x 4 , and x5 depict nonnegative slack variables, it is easily

demonstrated that the optimal simplex matrix is of the form
° °° ° 1 1 6
° ° °
1 1
1 2 -2 7
°° °
1 1
-2 1 2 1
°° °
1 1
2 2 1 13
with Xb = (7,6) and /°=13 (see point A of Figure 7.4). In addition,

(X8)~ = (0,1,0) depicts the optimal vector of primal slack variables. Moreover,
according to our previous discussion concerning the construction of the dual
solution, U~ =(!,O,!) and (U8)~ = (0,0), where the latter vector represents the
set of dual surplus variables.
Figure 7.4
Turning now to the complementary slackness conditions we have, for the

weak and strong cases respectively,
Weak Strong
o 0
U 1 X s ,1
1 0
= 2· =0 o +x0s ,1 =2+
U1
1 0 =2>
1 0
U~ X~, 2 = 0 . 1 = 0 u~ + x~, 2 = 0 + 1 = 1 > 0

00 -10-0
U 3 X s,3-2· - uo
3 +x 0s ,3-2+
_1 0- 1
-2> 0
and and
x~ U~, 1 = 7·0 = 0 x~ + U~, 1 = 7 + 0 = 7 > 0

x~ U~, 2 = 6 . 0 = 0 x~ + U~, 2 = 6 + 0 = 6 > 0
Hence at point A both strong and weak complementary slackness prevails.
Let us now recast this problem by changing the objective function to
1 = 2xl + x 2 with the structural constraints and nonnega-tivity conditions
unchanged. For this modified problem the optimal simplex matrix is
0 1 1 -2 0 0 4
1 0 0 1 0 0 8
0 0 -1 2 1 0 2
0 0 1 0 0 1 20
with X~ = (8,4) and Jl=20 (point B of Figure 7.4). (Actually, the case of
multiple optimal solutions emerges, as an examination of the last row of the
preceding simplex matrix reveals. Here I(A) = I(B) = I(C) = 20, where
C = '-\B + (l-'-\)A,O ::;.-\::; 1.) Moreover, (Xs)~ = (0,0,2), U~ = (1,0,0), and
(UB)~ = (0,0). For this problem the complementary slackness conditions appear
as
Weak Strong
U~ x!, 1 = 1 . 0 = 0 U~ + x!. 1 = 1 + 0 = 1 > 0

u~ x!, 2 = 0 . 0 = 0 U~ + x!. 2 = 0 + 0 = 0
U5 X !,3=O.2=0 U5 + X!,3 = 0 + 2 = 2 > 0
and and
x~ U!, 1 = 8 . 0 = 0 x~ + U!, 1 = 8 + 0 = 8 > 0
x~ U!, 2 = 4 . 0 = 0 X~ + U!, 2 = 4 + 0 = 4 > 0
In this instance only the weak complementary slackness conditions hold since an
additional zero-valued dual variable appears in the optimal set of the same, i.e.
now u~ = O. To offset the effect of this extra zero-valued dual variable on the
strong complementary slackness conditions, let us set A = ~ and thus choose point
C = AB + (I-A)A, 0 ~ A~ 1 (Figure 7.4), as our optimal primal solution for
which i c = 2( 1;) + 5 = 20. Then the complementary slackness conditions
become
Weak Strong
u~ z~, 1 = 1 ·0 = 0 u~ + z~, 1 =1 + 0 =1 > 0

U2z~,2 = O· (0.5) = 0 U2 + z~, 2 = 0 + 0.5 = 0.5 >0
u3z~,3=0.1=0 U3+ z~, 3 = 0 + 1 = 1 > 0
and and
z~ u~, 1 = (7.5) ·0 = 0 z~ + u~, 1 = 7.5 + 0 = 7.5 > 0
Z2 u~, 2 = 5·0 = 0 z2 + u~, 2 = 5 + 0 = 5 >0
Notice that now both weak and strong complementary slackness holds.
ONE-PHASE, TWO-PHASE, AND
COMPOSITE METHODS OF
8.1. Artificial Variables

In this section we shall return to the general primal linear programming
problem in standard form set out in Chapter 3, namely
maxI = C'X s. t.
AX = b,X~O
with X {Rn • If the objective function in this formulation is treated as an

additional structural constraint, then as indicated in the preceding chapter, we
seek to determine a solution of
AX
b } or
-G'X+! = 0
where
Here! is to be made as large as possible (and unrestricted in sign) with X ~ O.

Then the (m + 1 X n + 2) simplex matrix associated with this simultaneous
system is
For instance, given that we desire to
195
max! = xl + 4x2 S.t.

Xl - X2 < 6
2xI + 3x2 > 2
xl + x 2 =4
x I ,x2 ~ 0,
the corresponding simplex matrix has the form
n
1 0 0
i
[ 1 -1
0 -1 0
[X, bal = ~ 0 0 0
-1 -4 0 0 1
Since this matrix does not contain within it an m+l = 4th order identity matrix,
an initial basic feasible solution does not exist, i.e., we lack a starting point from
which to initiate the various rounds of the simplex method, the reason being that
not all of the original structural constraints are of the "$" variety. So in
situations where some or all of the original structural constraints are of the " ~ "
or " =" form, specific measures must be taken to obtain an identity matrix and
thus a starting basic feasible solution. To this end our procedure will be to first
transform the augmented st.ructural constrain system AX = b or
Xl + x3
X2 6
2xI + 3x 2 + - x4 2
Xl + x 2 4
into what is termed the artificial augmented structural constraint system
Xl X2 + X3 = 6
2xI + 3x2 - x4 + Xs 2
Xl + X2 + x6 4
by introducing the nonnegative artificial variables Xs and x 6 ' so named because
these variables are meaningless in terms of the formulation of the original
structural constraint system which contains only real or legitimate variables.
Artificial variables are introduced solely to initialize the simplex method, with
One-phase, Two-phase, and Composite Methods 197
the number of such variables corresponding to the number of " ;:::" and "="
structural constraints present. In standard form, the preceding system of
equalities appears as
A*X* b, (8.1)
where
n
xl
~"]
1 0 0 X2
[ 1 -1
x3
A* 2 3 0 -1 1
1 1 0 0 0
X. =[ x4
x5
x6
and Xa represents a vector of artificial variables with elements x 5 ' x 6 • Here the
individual vectors within A* corresponding to artificial variables are termed
artificial vectors and appear as a set of unit column vectors. Note that columns
three, five, and six of A* yield an m = 3rd order identity matrix and thus an
initial basic feasible solution to (8.1), namely x3 = 6, x5 = 2, and x6 = 4 when
Xl = x 2 = x 4 = O.
While we have obtained an initial basic feasible solution to (8.1), it is not a
feasible solution to the original structural constraint system AX = b, the reason
being that Xa 1= O. Hence any basic feasible solution to the latter system must
not admit any artificial variable at a positive level. In this regard, if we are to
obtain a basic feasible solution to the original linear programming problem, we
must utilize the simplex process to remove from the initial basis (i.e., from the
identity matrix) the artificial vectors and substitute in their stead an alternative
set of vectors chosen from those remaining in A*. In doing so we simply obtain
the original structural constraint system AX = b itself. Hence all artificial
variables are to become non basic or zero with the result that a basic feasible
solution to AX = b emerges wherein the basis vectors are chosen from the
coefficient matrix of this system. To summarize: any basic feasible solution to
A*X* = b which is also a basic feasible solution to AX = b has the artificial
vector Xa =O. In the remaining sections of this chapter we shall develop two
important techniques which serve to systematically purge the basis of any

artificial vectors (the one-phase and two-phase methods) and a routine which
economizes on the use of artificial variables (the composite method).
EXAMPLE 8.1. Determine an initial basic feasible solution to A*X* = b
when the initial set of structural constraints appears as
Xl + 2x2 > 1
I
xl + 2'x 2 > 4
I
2'x I + 6x 2 2
x I ,x2 ~ o.
If x 3' x 4 ~ 0 denote surplus variables and xs , x 6' x 7 ~ 0 depict artificial variables,
the preceding system becomes
xl + 2x2 - x3 + x5 = 1
I
xl + '2 x 2 - x4 + x6 4
2'xI + 6x 2
I
+ x7 2
x I " •• ,x7 ~ o.
Since
~
n
1 2 -1 0 1 0
I
A. [ 1 2' 0 -1 0 1
I
'2 6 0 0 0 0
contains within it the columns of a 3rd order identity matrix, the variables
associated with these columns yield an initial basic feasible solution to A*X* = b
once the remaining variables are set equal to zero, i.e., Xs = 1, x6 = 4, and
x 7 = 2.
8.2. The One-Phase Method

The first technique under consideration which serves to remove the artificial
vectors from the basis in but a single phase is the M-penalty method (A.
Charnes, et at. (1953)). Here extremely large unspecified negative coefficients are
assigned to the artificial variables in the objective function in order to preclude
the objective function from attaining a maximum so long as any artificial
variable remains in the set of basic variables at a positive level. In this regard,
for a sufficiently large scalar M > 0, the artificial linear programming problem
may be framed as: maximize the artificial objcctive function
!(X*) = C'X - Ml'X a s.t.
A*X* = b,X* ~ O.
For example, the problem appearing at the outset of this chapter has the
artificial form
Xl + x2 - x3 6
2xl + 3x2 - x4 + x5 2
xl + x2 + x6 4
xl" "'X6 ~ O.
The simplex matrix associated with this problem may now be directly expressed as
1 0 o 0 o
o -1 1 0 o (8.2)
o 0 o 1 o
o 0 MM 1
Before we examme some example problems dealing with the implemen-

tation of this technique, a few summary comments pertaining to the salient
features of the M-penalty method will be advanced. First, once an artificial
vector leaves the basis, it never reenters the basis. Furthermore, once an artificial
variable turns nonbasic, it remains so throughout the remaining rounds of the
simplex process since, if an artificial variable increases in value from zero by a
positive amount, a penalty of -M is incurred and ! concomitantly decreases.
Since the simplex process is one for which! is nondecreasing between succeeding
rounds, this possibility is nonexistent. So as the various rounds of the simplex
process are executed, we may delete from the simplex matrix those columns cor-
responding to artificial vectors that have turned nonbasic. Next, if the maximal
solution to the artificial problem has all artificial variables equal to zero, then its
X component is a maximal solution to the original problem with the values of
the artificial and original objective functions being the same. Finally, if the
maximal solution to the artificial problem contains at least one positive artificial
variable, then the original problem possesses no feasible solution (we shall return
to this last point later on when a detailed analysis pertaining to the questions of
inconsistency and redundancy of the structural constraints is advanced).
EXAMPLE 8.2. Using the M-penalty method, determine the simplex
matrix and an initial basic feasible solution to the problem:
maxI = 4x1 + 5x 2 s.t.
Xl +3x2 < 4
3x 1 x2 > 5
Xl + x2 8
x 1,x2 ~ o.
If x3 depicts a slack variable; x 4 represents a surplus variable; and x 5 ' x6 denote
artificial variables, we obtain
Xl + 3x 2 + x3 = 4
3x 1 - x2 - x4 + x5 5
Xl + x2 + x6 8
-4x1- 5x 2 + MX5+Mx6+ I 0
with associated simplex matrix
1 3 1 0 0 0 0 4
3 -1 0 -1 1 0 0 5
[A*, bol
1 1 0 0 0 1 0 8
-4 -5 0 0 MM 1 0
While this matrix does not yield an obvious initial basic feasible solution, we may
readily transform it into an equivalent representation that does by multiplying
every row containing an artificial variable by -M and adding their sum to the
last row, e.g., when the sum
-3M MOM -M 0 0 5M
- M -M 0 0 0 -M 0 8M
-4M 0 0 M -M -M 0 -13M
is added to the objective function row of the simplex matrix, we obtain
1 3 1 0 0 0 0 4
3 -1 0 -1 1 0 0 5
1 1 0 0 0 1 0 8
-4 -5 0 0 0 0 1 0
-4M 0 0 M 0 0 0 -13M
To facilitate the execution of subsequent rounds of the simplex process, each

entry in the last row has been separated into two parts, one which does not
involve M, and one appearing directly beneath it which does. Hence this double-
row representation actually depicts the single objective function row of the
simplex matrix i.e., the objective row is the sum of the last two rows in the
preceding matrix. It is at this point in our discussion of the one-phase method
that an initial basic feasible solution to the artificial linear programming problem
emerges since the existence of seven unit column vectors within the simplex
matrix is now readily apparent. In this regard, our initial basic feasible solution
is :1:3 = 4, :1:4 = 3, :1: 7 = 6, :1:8 = 5, :1:9 = 8, :1:10 = 3, and f = -13M. Note that
since f is expressed in terms of M, the objective function cannot possibly attain a
maximum with artificial vectors in the basis.
EXAMPLE 8.3. We seek to
ma:l: f = :1:1 +:1: 2 s.t.
:1:1 + 2:1:2 ~ 12
:I:} + > 4
:1:2
-:I:} + :1:2 > 1
:1: 1,:1: 2 ~ o.
If we let :1:3 denote a slack variable; :1:4 ,:1: 5 depict surplus variables; and
:1:6 ,:1:7 represent artificial variables, we obtain the simplex matrix
~ -L~ ~ ~ ~ ~].
000MMI0
To generate an initial basic feasible solution to this artificial linear programming

problem, let us add
-M -M 0 M 0 -M 0 0 -4M
M -M 0 0 M 0 -M 0 -M
0 -2M 0 M M -M -M 0 -5M
to the last row of the simplex matrix to obtain
1 2 1 0 0 0 0 0 12
1 1 0 -1 0 1 0 0 4
-1 1 0 0 -1 0 1 0 1
-1 -1 0 0 0 0 0 1 0
0-2M 0 M M 0 0 o -5M
The first basic feasible solution to the artificial problem thus becomes x3 = 12,
x6 = 4, xl = 1, and f = -5M. Since this solution is clearly not optimal, let us
undertake
ROUND 1.
1 2 1 0 0 0 0 0 12 6
1 1 0 -1 0 1 0 0 4 4
-1 [jJ 0 0 -1 0 1 0 1 1 ---+
-1 -1 0 0 0 0 0 1 0
0-2M 0 M M 0 0 o -5M
3 0 1 0 2 0 -2 0 10
2 0 0 -1 1 1 -1 0 3
-1 1 0 0 -1 0 1 0 1
-2 0 0 0 -1 0 1 1 1
-2M 0 0 M -M 0 2M o -3M
Here the second basic feasible solution to the artificial problem is x2 = 1, x3 = 10,
x6 = 3, and f = I-3M. Since this solution is also not optimal, we proceed to
ROUND 2 (column seven is deleted).
3 0 1 0 2 0 0 10 10/3
[2] 0 0 -1 1 1 0 3 3/2
-1 1 0 0 -1 0 0 1 --+
-2 0 0 0 -1 0 1 1
-2M 0 0 M-M 0 o -3M
3 1 3 11
0 0 1 2 2 -2 0 2"
1 1 3
1 0 0 -21 2 2 0 2
0 1 0 -21 -21 1
2" 0
5
2
0 0 0 -1 0 1 1 4
0 0 0 0 0 M 0 0
The third basic feasible solution to the artificial problem thus appears as xl = i,
x2 =~, x3 = 121, and f = 4. Since all artificial vectors have been removed from
the basis, this particular basic feasible solution to the artificial problem represents
an initial basic feasible solution to the original problem. In this regard, let us
now apply the simplex method to this initial basic feasible solution.
ROUND 3 (column six and the M-row are deleted).
0
1
0 1
0 -21
rn 1
2 0
1
11
2"
3
11/3 0 0 2
3
1
1 1
3 0
2
11
3
10
0 2 0 2 1 0 3 0 3 0 3
--+
0 1 0 -21 -21 0 5
2 0 1 1
3 0 -32 0 13
3
2 1 23
0 0 0 -1 0 1 4 0 0 3 0 3 1 3
Since the optimality criterion is satisfied, the optimal basic feasible solution to
.. I rmear programmmg
t he ongma . pro beml 'IS Xl 10 x = 3'
= 3' 2
11 an d f = 3'
13 x = 3'
4
23
8.3. Inconsistency and Redundancy

We now turn to a discussion of the circumstances under which all artificial
vectors appearing in the initial basis of the artificial linear programming problem
may be removed from the basis in order to proceed to an optimal primal basic
feasible solution to the original problem. In addition, we shall also determine
whether the original augmented structural constraint equalities AX = bare
consistent, and whether any subset of them is redundant. Let us assume that
from an initial basic feasible solution to the artificial problem we obtain a basic
solution for which the optimality criterion is satisfied. Here one of three
mutually exclusive and collectively exhaustive results will hold:
(1) the basis contains no artificial vectors;
(2) the basis contains one or more artificial vectors at the zero level;
(3) the basis contains one or more artificial vectors at a positive level.
Let us discuss the implications of each case in turn.
First, if the basis contains no artificial vectors, then, given that the optimal-
ity criterion is satisfied, it is evident that we have actually obtained an optimal
basic feasible solution to the original primal problem. In this instance the
original structural constraints are consistent and none are redundant, i.e., if there
emerges at least one basic feasible solution to the original structural constraint
system, the simplex process will remove all artificial vectors from the basis and
ultimately reach an optimal solution provided that one exists (a case in point was
example 8.3, above).
We next turn to the case where the basis contains one or more artificial
vectors at the zero level. With all artificial variables equal to zero, we have a
feasible solution to AX = h, i.e., the original structural constraint equations are
consistent. However, there still exists the possibility of redundancy in the
original structural constraint system. Upon addressing ourselves to this latter
point we find that two alternative situations merit our consideration. To set the
stage for this discussion, let Yj denote the /h legitimate nonbasic column of the
optimal simplex matrix. In this regard we first assume that Yjj' the ith com-
ponent of Yj' is different from zero for one or more j and for one or more if.A,
where .A denotes an index set consisting of all i's corresponding to artificial basis
vectors e,·. Now, if for some j = i' we find that y .. , i= 0, it follows that the
'J
associated artificial basis vector ej may be removed from the basis and replaced
by Y.,.
J
Since the artificial vector currently appears in the basis at the zero level,
Y.,J also enters the basis at the zero level, with the result that the optimal value
of the objective function is unchanged in the new basic feasible solution. If this
process is repeated until all artificial vectors have been removed from the basis,
we obtain a degenerate optimal basic feasible solution to the original problem
involving only real or legitimate variables. In this instance none of the original
structural constraints within AX = b is redundant. Next, if this procedure does
not remove all artificial vectors from the basis, we ultimately reach a state where
Yij = 0 for all Yj and all remaining iE.A. Under this circumstance we cannot
replace any of the remaining artificial vectors by some Yj and still maintain a
basis. If we assume that there are k artificial vectors in the basis at the zero
level, every column of the (m X n) matrix A may be written as a linear combina-
tion of the m-k linearly independent columns of A appearing in the basis, i.e.,
the k artificial vectors are not needed to express any column of A in terms of the
basis vectors. Hence p(A) = m-k so that only m-k rows of A are linearly
independent, thus implying that k of the original structural constraints in
AX = b are redundant. [As a practical matter, since inequality structural
constraints can never be redundant (each is converted into an equality by the
introduction of its own slack or surplus variable), our search for redundant
structural constraints must be limited to only some subset of equations appearing
p
in AX = b, namely those expressed as oE at)ox)o = bt , tEer
)=1
= {l, ... ,m}.] To
identify the specific structural constraints within AX = b that are redundant, we
note briefly that if the artificial basis vector ch remains in the basis at the zero
level and Yhj = 0 for all legitimate nonbasis vectors Yj , then the hth structural
constraint of AX = b is redundant. In this regard, if at some stage of the simplex
process we find that the artificial vector ch appears in the basis at the zero level
with Yhj =0 for all Yj , then before executing the next round of the simplex
routine, we may delete the hth row of the simplex matrix containing the zero-
valued artificial variable along with the associated artificial basis vector ch' The
implication here is that any redundant constraint may be omitted from the
original structural constraint system without losing any information contained
within the latter. (See example 8.4 below.)
We finally encounter the case where one or more artificial vectors appear in
the basis at a positive level. In this instance the basic solution generated is not
meaningful, i.e., the original problem possesses no feasible solution smce
otherwise the artificial vectors would all enter the basis at the zero level. Here
the original system either has no solution (the structural constraints are inconsis-
tent) or has solutions that are not feasible. To distinguish between these two
alternatives we first note that since the optimality criterion is satisfied, Yjj ~ 0
for legitimate non basis vectors Vj and for those i EA. If for some j = jf we find
that y . ./ < 0, we can insert
IJ
V./J into the basis and remove the associated artificial
basis vector ej and still maintain a basis. However, with y . ./ < 0, the new basic
IJ
solution will not be feasible since V./J enters the basis at a negative level. If this
process is repeated until all artificial vectors have been driven from the basis, we
obtain a basic though nonfeasible solution to the original problem involving only
legitimate variables (see example 8.5, below). Next, if this procedure does not
remove all artificial vectors from the basis, we ultimately obtain a state where
Yij = 0 for all Vj and for all i remaining in A. Assuming that there are k
artificial vectors in the basis, every column of A may be written as a linear com-
bination of the m-k legitimate basis vectors so that p(A) = m-k. But the fact
that k artificial columns from A appear in the basis at positive levels implies that
b is expressible as a linear combination of more than m-k basis vectors so that
p(A) f. p[A, b]. In this regard, the original structural constraint system AX = b
is inconsistent and thus does not posses a solution (see example 8.6 below).
A consolidation of the results obtained in this section now follows. Given
that the optimality criterion is satisfied: (1) if no artificial vectors appear in the
basis, then the solution is an optimal basic feasible solution to the original primal
linear programming problem. In this instance the original structural constraints
AX = b are consistent and none are redundant; (2) if at least one artificial vector
appears in the basis at a zero level, then AX =b is consistent and we either
obtain a degenerate optimal basic feasible solution to the original primal problem
or at least one of the original structural constraints is redundant; and (3) if at
least one artificial vector appears in the basis at a positive level, the original
primal program has no feasible solution since either AX = b represents an incon-
sistent system or there are solutions but none is feasible.
EXAMPLE 8.4. Let us
maxi = Xl +X 2 +X3 s.t.
Xl + 4X2 + 3x3 6
3x1 + 12x2 + 9x3 18
Xl' X2' X3 ~ o.
Letting x4' x5 represented artificial variables, we obtain the simplex matrix
[A., bal = [ 1
3 1 0
4
3 12 9 0 1
-1 -1 -1 M M
0
0
1
li] ~
1 4 3 1 0 0 6
3 12 9 0 1 0 18
-1 -1 -1 0 0 1 0
-4M -16M -12M 0 0 0 -24M
Our initial basic feasible solution to this artificial problem is x4 = 6, x5 = 18, and
I = -24M. Since the optimality criterion is not satisfied, we proceed to
ROUND 1.
@] 3/2
-
1 3 1 0 0 6
3 12 9 0 1 0 18 3/2
-1 -1 -1 0 0 1 0
-4M -16M -12M 0 0 0 -24M
\\ Y2
1 1 1 3
4 1 4 4 0 0 2
0 0 0 -3 1 0 0
3 1 1 3
-4 0 -4 4 0 1 2
0 0 0 4M 0 0 0
Then our second basic feasible solution to the artificial problem is x2 =~, x5 = 0,
and I =~. Since the optimality criterion is still not satisfied, an additional
round of the simplex routine is warranted. Before moving on to round two, let
us pause for a moment to inspect the preceding simplex matrix. Given that the
artificial vector ~ enters the basis at the zero level (A = {2}) and Y21 = Y22 = 0
for legitimate non basis vectors \\, Y2' it is obvious that the second structural
constraint is redundant so that we may at this point delete row two and column
five. Moreover, since all artificial vectors have been eliminated from the basis,
round two starts with an initial basic feasible solution to the original problem.
We now look to
ROUND 2 (rows two and four along with columns four and five are
deleted).
[~ :]6~ ~ [ :l
3
1 4 0 4 3 0
1 3 2 1
0 -4 1
A glance at this last matrix indicates that we have obtained an optimal basic
feasible solution to the original problem consisting of Xl = 6 and I = 6.
EXAMPLE 8.5. Next, we seek to
maxi = Xl + x 2 s.t.
4xI x2 :::; 1
Xl X2 ~ 3
x I ,x 2 ~ O.
If x3 denotes a slack variable, x4 represents a surplus variable, and x5 depicts an
artificial variable, then we may form the simplex matrix
4 -1 1 0 0 0 1
1 i]~
-1 1 0 0 0 1 -1 0 -1 1 0 3
[L,bo]; [ -1 0 -1 1 0
-1 -1 0 0 0 1 0
-1 -1 0 0 M 1
-M M 0 M 0 o -3M
Since the initial basic feasible solution consisting of x3 = 1, x5 = 3, and I = -3M

is obviously not opt.imal, we proceed to
ROUND 1.
Y1 Y2 Y3
@] -1 1 0 0 0 1 1
4 1 -41 1
4 0 0 0 1
4
1 -1 0 -1 1 0 3 3 0 -43 1
-4 -1 1 0 11
4
--+
1
-1 -1 0 0 0 1 0 0 -45 4 0 0 1 1
4
-M M 0 M 0 o -3M 0 :!M
4
1M
4 M 0 O-liM
4
The second basic feasible solution to this artificial problem thus appears as
- 1 _ 11 d I _ 1 11M Can the artificial basis vector c (.A={2}) be
xl - 4' x5 - 4' an - 4 - T· 2
eliminated from the basis? As we shall now see, the answer is yes. From the
second row of the preceding simplex matrix we find that
x5 = ~1 + !X2 + iX3 + x 4.
Hence, x5 may be driven to zero only if x 2' x 3' or x 4 assumes a negative value
since Y21'Y22'Y23 < O. More formally, since the artificial basis vector ck remains
in the basis at a positive level and Ykj < 0 for all legitimate nonbasis vectors Yj ,
the original problem has no feasible solution in the sense that at least one basic
variable is negative (as Figure 8.1 indicates).
EXAMPLE 8.6. We now desire to
maxi = xl -X2 s.t.
Xl + x2 6
xl +x 2 4
xl'x 2 ? O.
If x3' x 4 represent artificial variables, obtain

1 1 1 0 0 6
]~
1 1 0 0 6 1 1 0 1 0 4
[A.. hoI = [ : 1 0 1 0 4
-1 1 0 0 1 0
-1 1 M M 1 0
-2M-2M 0 0 0-lOM
Figure 8.1
and thus an initial basic feasible solution consisting of x3 = 6, x4 = 4, and

f = -10M. Since the optimality criterion is not satisfied, we look to
ROUND 1.
YI
0 1 1 0 0 6 6 0 0 1 -1 0 2
ill 1 0 1 0 4 4 1 1 0 1 0 4
--+
-1 1 0 0 1 0 0 2 0 1 1 4
-2M-2M 0 0 O-lOM 0 0 o 2M o -2M
Hence the optimal basic feasible solution to the artificial problem is xl =4,
x3 = 2, and f = 4 - 2M. SinceYll = 0, YI cannot replace el (here .A = {I}) in
the basis. In sum, if the artificial basis vector ck remains in the basis at a
positive level and Ykj = 0 for all legitimate nonbasis vectors Yj , then the original
structural constraints AX = b are inconsistent so that no feasible solution to the
original problem exists (Figure 8.2).
~------~~~~----~Xl
4 6
Figure 8.2
8.4. Unbounded Solutions to the Artificial Problem

It may be the case that at some point during the solution of the artificial
problem we encounter a vector Yk for which Yik::; 0, i=l, ... , m, with
-ck = min{-cj,-Cj < O}. Then clearly the artificial objective function value is
3
unbounded. In this circumstance, what can be said about the behavior of the
original problem? Here two subcases present themselves:
1. if -ck = min{-cj,-Cj < O}, Yik::; O,i=l, ... ,m, for some vector Yk ,
3
and all artificial variables are zero, then the original linear program also
has an unbounded objective value.
2. if -ck = mjn{-cj,-Cj < O}, Yik::; 0, i=l, ... ,m, for some vector Yk ,
and at least one artificial variable is positive, then the original problem
is infeasible.
For instance, consider
EXAMPLE 8.7. Let us
max f = Xl +X2 s.t.
1
-Xl + X2 > 1
x 1 ,x2 ,x3 ~ O.
With x 4 ,x5 nonnegative surplus and artificial variables respectively, we form
1 -1 1 0 0 0 1
[A*, hal [-: -1

1
-1 -1
1 0 0 0
0 -1 1 0
0 0 M 1
1
1
0
]~ -1
-1 -1
0 -1
0 0
1
0
0
1
1
0
M-M 0 M 0 0 -M
and thus an initial basic 'feasible solution consisting of x3 = 1, x5 = 1, and

f=-M. Since this current solution is not optimal, we look to
ROUND 1.
YI
1 -1 1 0 0 0 1 0 0 1 -1 1 0 2
-1 [] 0 -1 0 1 1 -1 1 0 -1 1 0 1
--+
-1 -1 0 0 0 1 0 -2 0 0 -1 1 1 1
M-M 0 M 0 0 -M 0 0 0 0 M 0 0
Since -c 1 = -2 = min{ -c i' -Ci < O} and each component of Y1 is non positive, it
J
follows that the artificial problem is unbounded.
ROUND 2 (row four and column five are deleted).
[-~ -2
0
1
0
1 -1
0 -1
0 -1
0
0
1 :}
Given that the artificial variable x5 is zero, a glance at column one of this matrix
(which houses an initial basic feasible solution to the original problem) indicates
that the original program is also unbounded.
What this example problem has revealed is that when the artificial problem
is unbounded and all artificial variables are zero, then there is an admissible
direction d' = (d1,d~) ~ 0' of the feasible set for the artificial problem
{X* I A*X* = b, X* ~ O} such that C'd 1 - Ml'd 2 > O. Since M is large and
positive and d 2 ~ 0, the preceding inequality implies that d 2 = 0 and thus
C'd t > o. Thus d t IS an extreme direction for the original feasible region
{X I AX = b, X ~ O} along which f increases without bound. In fact, from
column one of the preceding simplex matrix, Y~ = (0, -1) and thus
as required.
8.5. The Two-Phase Method

In this section we shall develop an alternative procedure for solving a primal
linear programming problem involving artificial variables. Our approach will be
to frame the simplex method in terms of two successive phases, phase I and phase
II (Dantzig, et ale (1955)). In phase I we seek to drive all artificial variables to
zero by maximizing a surrogate objective function involving only artificial
variables. If the optimal basis for the phase I problem contains no artificial
vectors or contains one or more artificial vectors at the zero level, then phase II is
initiated. However, if the optimal phase I basis contains at lea.gt one artificial
vector at a positive level, the process is terminated. If phase II is warranted, we
proceed by maximizing the original objective function using the optimal phase I
basis as a starting point, i.e., the optimal phase I basis provides an initial though
possibly non basic feasible solution to the original linear programming problem.
PHASE I. In constructing the phase I objective function, the legitimate
variables will be assigned zero coefficients while each artificial variable is given a
coefficient of minus one. Then the phase I surrogate linear programming
problem may be formulated as: maximize the infeasibility form
g(X*) = o'x - l'X a s.t.
A*X* = b, X. ~ O.
For instance, the phase I surrogate version of the problem on page is
max 9 = - x5 - x6 s.t.
Xl - X2 + x3 6
2xI + 3x 2 - x4 + x5 2
Xl + x2 + x6 4
x I , ••• ,x6 ~ 0
with simplex matrix
[j
U
-1 1 0 0 0 0
3 0 -1 1 0 0
[A.,ho] = 1 0 0 0 1 0
(8.3)
0 0 0 1 1 0
Note that with Xa ~ 0, it must be the case that max{g}:S 0 SInce g, which
initially assumes a negative value, can never turn positive. In addition, the
largest possible value that 9 can attain is zero; and this occurs only if the optimal
phase I basis contains no artificial vectors or if those artificial vectors remaining
in the said basis do so at the zero level. So when max{g} < 0, at least one
artificial vector remains in the optimal phase I basis at a positive level.
Before discussing the variety of possible solutions that may emerge during
phase I, let us look to some prominent aspects of the phase I routine. First, as in
the M-penalty method, an artificial vector never reenters the basis once it has
been removed. Here too the columns of the simplex matrix corresponding to
artificial vectors that have turned non basic may be deleted as pivoting is carried
out. Moreover, during phase I, the sequence of vectors which enters and leaves
the basis is the same as in the M-penalty method. Finally, if 9 becomes zero
before the optimality criterion is satisfied, we may terminate phase I and proceed
directly to phase II.
As indicated above, phase I ends when the optimality criterion is satisfied.
In this instance, the optimal basis will exhibit one of three mutually exclusive
and collectively exhaustive characteristics:
(1) max{g} = 0 and no artificial vectors remain in the basis;
(2) max{g} = 0 and one or more artificial vectors remain in the basis at the
zero level;
(3) max{g} < 0 and one or more artificial vectors remaIn In the basis at a
posi ti ve level.
If case one obtains, it is evident that the optimal phase I basis provides a basic
feasible solution to the original primal problem. Here the original structural con-
straints are consistent and none are redundant. Next, if case two prevails, we
obtain a feasible solution to the original problem, i.e., the original structural
constraints are consistent although possible redundant. Finally, for case three,
the surrogate problem has no feasible solution with Xa = 0 and thus the original
problem possesses no feasible solution either. In sum, if either case one or two
emerges at the end of phase I, there exists at least one feasible solution to the
original problem. In this regard, we proceed to
PHASE II. Let us first specify the appropriate phase II objective function.
Since we ultimately desire to maximize f(X), the optimal phase I simplex matrix
is transformed into the initial phase II simplex matrix by respecifying the
objective function coefficients. That is to say, during phase II, the coefficients on
the legitimate variables are the same as those appearing in the original objective
function while the coefficients on the artificial variables are zero, i.e., the phase II
objective function is the original objective function f(X) itself. In this light, the
sequence of operations underlying the two-phase method may be expressed as
max g(X*) O'X - l'X a s.t. [Phase I]
max f(X*) C'X + O'Xa s.t. [Phase II]
A*X* = b, X* ~ O.
We now turn to an analysis concerning the implications for phase II of the
aforementioned characteristics of the optimal phase I simplex matrix.
If case one above occurs, the optimal phase I basis provides an initial basic
feasible solution to the original linear programming problem. Once the modified
objective function is introduced, pivoting proceeds until an optimal basic feasible
solution to the original problem is attained.
Turning now to case ,two, while we have obtained a feasible though
nonbasic solution to the original problem, there still remains the question of
redundancy in the original structural constraints AX = b and the development of
a safeguard against the possibility that one or more artificial vectors appearing in
the initial phase II basis at the zero level will abide in some future basis at a
positive level as the phase II pivot operations are carried out.
Our discussion of redundancy here is similar to that advanced for the M-

penalty method. To this end, let us assume that within the optimal phase I
simplex matrix Yij = 0 for all legitimate non basis vectors Yj and for all i€.A. In
this instance, none of the artificial basis vectors ei can be replaced by the Yj so
that there is redundancy in the original structural constraint equalities. In this
regard, if there are k artificial vectors in the basis at the zero level, then, at the
end of phase I, we may simply delete the k rows of the simplex matrix containing
the zero-valued artificial variables along with the associated artificial basis
vectors ei Then phase II begins with a basis of reduced size and pivoting is car-
ried out until an optimal basic feasible solution to the original problem is attained.
We next examine a procedure which serves to guard against the possibility
that an artificial vector appearing in the initial phase II basis at the zero level
manifests itself in some future basis at a positive level. Let us assume that for
some j = i' the vector Y.,
}
is to enter the basis and y I}. ., -< 0 with y I}
.., < 0 for at
least one i €.A. . ., < 0 will
In this circumstance the artificial vectors for which y I}
not be considered as candidates for removal from the basis. Rather, some
legitimate vector will be removed with the result that the artificial basis vectors
.. , < 0 appear in the new basis at a positive level.
for which y I} To avoid this
difficulty, instead of removing a legitimate vector from the basis, let us adopt the
policy of arbitrarily removing anyone of the artificial basis vectors ej with
. ., < O. The result will be a feasible solution to the original problem wherein
YI}
Y.,
)
enters the new basis at the zero level (since the artificial vector that it
replaced was formerly at the zero leveL) With the value of the incoming variable
equal to zero, the values of the other basic variables for the new solution are the
same as in the preceding solution. Hence, the value of the objective function for
the new solution must be the same as at the previous one (see example 8.9)
below).
EXAMPLE 8.8. Solve the problem appearing in example 8.3 using the two-
phase method. Forming the phase I infeasibility form g = -x6-x7 we easily
obtain the simplex matrix
~]
2 1 0 0 0 0 0
[A., bo]
[-i
1 0 -1 0 1 0 0
1 0 0 -1 0 1 0 1 .
0 0 0 0 1 1 1 0
To generate an initial basic feasible solution to this phase I surrogate

problem, let us add
-1 -1 0 1 0 -1 0 0 -4
1 -1 0 0 1 0 -1 0 -1
0 -2 0 1 1 -1 -1 0 -5
to the last row of the above matrix to obtain
1~
1 0 0 0 0 0
[ 11 21 0 -1 0 1 0 0
]
-1 1 0 0 -1 0 1 0
o -2 0 1 1 0 0 1 -5
The first basic feasible solution to the surrogate problem thus becomes x3 = 12,
x6 =4, x7 = 1, and 9 = -5. Since this solution is obviously not optimal, we
undertake
PHASE I, ROUND 1.
1~
1 0 0 0 0 0 6
[ 11 21 0 -1 0 1 0 0 4
] ---+
-1 [I] 0 0 -1 0 1 0 1
o -2 0 1 1 0 0 1 -5
[-l o 10]
0 1 0 2 0 -2
0 0 -1 1 1 -1 3
1 0 0 -1 0 1 o 1 .
-2 0 0 1 -1 0 2 1 -3
Here the second basic feasible solution to the surrogate problem is xl = 1,

x3 = 10, x6 = 3, and 9 = -3. Since this solution is also not optimal, we turn to
PHASE I, ROUND 2 (column seven is deleted).
3 0 1 0 2 0 0 10 10/3
~ 0 0 -1 1 1 0 3 3/2
---+
-1 1 0 0 -1 0 0 1
-2 0 0 1 -1 0 1 -3
3 1 3 11
0 0 1 2 2 -2 0 "2
1 1 3
1 0 0 -21 2 2 0 2
0 1 0 -21 -21 1
2 0
5
2
0 0 0 0 0 1 1 0
The third basic feasible solution to the surrogate problem thus appears as xl = ~,
x2 =~, x3 = V, and 9 = O. Since it is clearly optimal, phase I ends. Moreover,
since all artificial vectors have been removed from the basis, this particular basic
feasible solution to the surrogate problem represents an initial basic feasible
solution to the original problem. In this regard, phase II begins by deleting
column six of the final phase I simplex matrix and replacing the last row of the
same by the original objective function f = xl + x 2• That is,
3 1 11
0 0 1 2 2 0 2"
1 0 0 -21 -21 0 23
5
0 1 0 -21 -21 0 2
-1 -1 0 0 0 1 0
or, upon adding

1 1 3
1 0 0 -2 2 0 2
1 1 5
0 1 0 -2 -2 0 2
1 1 0 -1 0 1 4
to the last row of the preceding simplex matrix,
3 1 11
0 0 1 2 2 0 2"
1 3
1 0 0 -21 2 0 2
0 1 0 -21 -21 0 5
2
0 0 0 -1 0 1 4
Thus the initial basic feasible solution to the phase II original problem appears as
X _3 X _s X _11 and 1=4. Since the optimality criterion is not
1-2' 2 -2' 3-2'
satisfied, we proceed to
PHASE II, ROUND 1.
0 0 1
ill 1 0 11 11/3 0 0 2
3 1 1
3 0 3
11
--
2 2
1 3 1
1 0 0 1
-2 2 0 2 1 0 3 0 32 0 310
0 1 0 1 1 0 s 0 1 1
0 -31 0 313
-2 -2 2 3
2 1 1 323
0 0 0 -1 0 1 4 0 0 3 0 3
An inspection of the last row of this simplex matrix indicates that we have
obtained an optimal basic feasible solution to the original problem consisting of
- 10 _ 13 _ 11 d I _ 23
Xl - 3' x2 - 3' x 4 - 3' an - 3'
As an exercise the reader should resolve example problems 8.4-8.6 above
using the two-phase method so as to verify that the same conclusions regarding
redundancy, infeasibility, and inconsistency respectively hold.
EXAMPLE 8.9. Let us
maxI = 5x 1 + 4x2 + x3 + x 4 s.t.
xl + +
X2 x3 2
x2 + x3 2
xl + 2x2 + 3x3 + 6x 4 4
x 1,x2,x3 ,x4 ~ O.
Constructing the infeasibility form 9 = -xs - x6 - x 7 we obtain
n--
1 1 1 0 1 0 0 0
0 1 1 0 0 1 0 0
[A" ho] = [ 1 2 3 6 0 0 1 0
0 0 0 0 1 1 1 1
[j !}
1 1 0 1 0 0 0
1 1 0 0 1 0 0
2 6 3 0 0 1 0
-4 -5 -6 0 0 0 1 -8
where x s' x6 and x 7 are artificial variables. Here the initial basic feasible solution
to this surrogate problem is Xs = 2, x6 = 2, x 7 = 4, and 9 = -8. To maximize 9
we turn to
PHASE I, ROUND 1.
1 1 1 0 1 0 0 0 2
0 1 1 0 0 1 0 0 2
1 2 3 [ill 0 0 1 0 4 2/3 --+
-2 -4 -5 -6 0 0 0 1 -8
1 1 1 0 1 0 0 0 2
0 1 1 0 0 1 0 0 2
1 1 1 1 2
:3 2
(; 1 0 0 (; 0 :3
-1 -2 -2 0 0 0 1 1 -4
The second basic feasible solution to the surrogate problem consists of

x4 =~, x5 = 2, x6 = 2, and 9 = -4. Given that the optimality criterion is still
not satisfied, we proceed to
PHASE I, ROUND 2 (column seven is deleted)
1 [] 1 0 1 0 0 2 2
0 1 1 0 0 1 0 2 2
--+
1 1 1 2
:3 2
(; 1 0 0 0 :3 2
-1 -2 -2 0 0 0 1 -4
1 1 1 0 100 2
-1 o o o -1 1 o o
o (;1 1 -~ 0 o o
100020 0
With the optimality criterion satisfied, the optimal basic feasible solution to the
phase I surrogate problem is x2 = 2, x4 = 0, x6 = 0, and 9 = o. Turning now to
phase II, let us delete column five of the final phase I simplex matrix and replace
the last row of the same by the original objective function f = 5x 1 + 4x2
+ x3 + x 4 . To this end we obtain
\\
1 1 1 0 0 0 2 1 1 1 0 0 0 2
-1 0 0 0 1 0 0 -1 0 0 0 1 0 0
-+
1 1 1 1
-6 0 6 1 0 0 0 -6 0 6 1 0 0 0
7 19
-5 -4 -1 -1 0 1 0 -6 0 "6 0 0 1 8
Since the initial phase II basic feasible solution to the original problem consisting
of x2 = 2, x4 = 0, x6 = 0, and f = 8 is not optimal, round one of phase II is
warranted. However, before attempting to generate an improved solution to the
phase II objective function, we note that since Y21 < 0, the artificial vector e2 will
not be considered for removal from the basis. Rather, simplex pivoting dictates
that the vector c l will be deleted from the basis with the result that the current
artificial basis vector e2 will appear in the new basis at a positive level. To
preclude the latter from occurring, instead of removing e l , let us remove the
artificial basis vector e2. To this end we proceed to
PHASE II, ROUND 1.
1110002 o 1 1 0 1 0 2
[;I] 0 0 010 o 1 0 0 0 -1 0 o
-+
o 6
1
1 0 o o o o 1
6" 1 -A 0 o
o 19
"6 o 0 1 8 o o 19
"6 o -~ 1 8
Thus the optimal (though degenerate) basic feasible solution to the phase II
original problem consists of Xl = 0, x 2 = 2, x 4 = 0, and f = 8.
8.6. Obtaining the Optimal Primal Solution

from the Optimal Dual Matrix
We noted earlier that the simplex method can be applied to either the
primal or dual formulation of a linear programming problem, with the choice of
the version actually solved determined by considerations of computational ease.
In this regard, if the dual problem is selected, how may we determine the optimal
values of the primal variables from the optimal dual solution? Given that the
dual problem appears as

ming(U) = b'U s.t.
A'U ~ C, U 2 0
or
max h(U) = -g(U) = -b'U s.t.
A'U ~ C, U ~ 0,
we may utilize the M-penalty method to derive the artificial simultaneous aug-
mented form
A'U InUs + InUa C
b'U + MI'Ua+h 0,
where U a denotes an (m x 1) vector of nonnegative artificial variables and the
associated dual simplex matrix is
[
A' -In In 0
b' 0' MI' 1
C ]
o
I~'-MI'A'
A' o
-+ 1
Relative to this system the (p+l x p+l) basis matrix and its mverse may be
written respectively as
1)-
~ D
bn-MI'D
0
1
l 1)-1 = [ n-'
-bn D- 1 +MI'
where D depicts the (p x p) basis matrix associated with the dual artificial
0
1
l
structural constraint system. Then
]~, -~'I' -M~'C

[ fr' 0 -In In 0
-bn D- 1 +MI' 1 A' MI' 0' 1 ]
D- 1A' _D- 1 0 D- 1 C
(S.4)
II-bnD-1A' + b' II Ib n D- 1 1 1 -b1:lD- 1C
yields a basic feasible solution to the dual minimum problem, where the vectors
associated with the artificial variables, i.e., those corresponding to the third
partitioned column of (8.4), have been deleted since the artificial vectors have
been driven from the basis. If we assume that the optimality criterion is
satisfied, then the optimal values of the primal variables appear as the blocked
array of elements in the objective function row of (8.4), i.e., X' = bVD- 1 ~ 0'.
In addition, from the augmented primal structural constraint system
AX + ImX. = b we may isolate the primal slack variables as X~ = b' - X' A'. But
this expression corresponds to the double-blocked array of elements X~ =
-bDD- 1 A' + b' ~ 0' in the objective function row of (8.4). To summarize, our
rules for isolating the optimal values of the primal variables (original and slack)
may be determined as follows:
(a) to find the values of the primal variables X which correspond to
an optimal basic feasible solution of the dual minimum problem, look in
the last row of the final dual simplex matrix under those columns which
were, in the original simplex matrix, the columns of the negative of the
identity matrix. The lh primal variable xi' j=l, ... , p, will appear under
the column which was initially the lh column of the negative of the
identity matrix;
(b) to find the values of the primal slack variables Xs which correspond to an
optimal basic feasible solution of the dual minimum problem, look in the
last row of the final dual simplex matrix under those columns which were,
in the original simplex matrix, the columns of the matrix A'. The ith
primal slack variable x p+i' i=l, ... , m, will appear under the column which
was initially the ith column of A'.
In this section we have demonstrated how the optimal primal variables may
be obtained from the optimal dual simplex matrix. Earlier in Chapter 6 we
extracted the optimal dual variables from the optimal primal simplex matrix.
Hence no matter which of the primal-dual pair of problems is solved, all of the
pertinent information generated by an optimal basic feasible solution is readily
available, i.e., from either the final primal or dual simplex matrix we can isolate
both the primal and dual original variables as well as the primal slack and dual
surplus variables.
EXAMPLE 8.10. The simultaneous artificial augmented form of the dual
minimum problem presented in example 4.3 is
U1 + 4u 2 - U3 + Us 24
5u1 + u2 - u4 + u6 25
u 1 + 3u2 + Mu s+Mu6 = 0,
where h =-g (here we are employing the transformation min 9 = -max{-g}
given in section 9.1); u3 ' u4 ~ 0 represent surplus variables; and us' u6 ~ 0 depict
artificial variables. In addition, the associated simplex matrix is
4 -1 0 1 0
1 0 -1 0 1
3 0 0 M M
1 4 -1 0 1 o o 24
5 1 0 -1 0 1 o 25
1 3 0 0 0 o 1 0
-6M -5M M M 0 o o -49M
It is easily shown that the optimal simplex matrix associated with this problem is
0 1 s 1
0 5
-19 19
1 4 1 4
1 0 19 -19
14 1 -19
0 0 19 19 0
where the columns associated with the artificial variables and the M-row have
been deleted. Upon examining the objective function row of this matrix we see
that ~ = (14/19, 1/19) and X~ = (0,0) as expected.
To isolate the values of the original primal variables X along with the
primal slack variables Xa when the two-phase method is used to solve the dual
problem, it will simply be noted that if the optimal phase I basis contains no
artificial vectors, then the optimal phase II simplex matrix is of the form
exhibited by (8.4). Hence the same rules (a), (b) above for locating X, Xa apply
for the two-phase method as well.
8.7. The Composite Simplex Method

As was the case for both the M-penalty and two-phase methods, the
composite simplex algorithm is an iterative technique for obtaining an initial
basic feasible solution to a linear programming problem wherein a number of
" >" and/or" =" structural constraints are present. Our previous approach in
this circumstance was to transform AX = b into A*X* = b by inserting into the
former system a sufficient number of artificial variables so that an initial basic
feasible solution to the latter system could be obtained. The M-penalty and two-
phase methods then operated to systematically drive the artificial vectors out of
the basis so that an optimal solution to the original problem could eventually be
determined (if, of course, it existed). And as the results of the preceding sections
indicate, a considerable amount of computational effort is often expended in
implementing the M-penalty and two-phase routines even if only a few artificial
variables are present. To circumvent this difficulty the composite simplex method
(Wolfe (1965)) was designed to curtail the use of artificial variables by allowing
some of the basic variables to initially assume negative values. These variables
are ultimately driven to feasible levels (and any artificial vectors within the
initial basis are routinely forced from the same) so that an optimal solution to
the original problem is finally attained. Let us see exactly how the composite
simplex method operates.
Since a basic variable is not precluded from initially assuming a negative
p
value, let us convert any "~" structural constraints to the form -?: as/I: j
3=1
~ - ba , where - bs is now unrestricted in sign. (Note that if the original
structural constraint system is composed entirely of inequalities, we have an

initial basic - though not necessarily feasible - solution in slack variables.)
Moreover, if equality structural constraints are also present, a non-negative
artificial variable is added to each so that again we have no difficulty in
obtaining an initial basic feasible solution. In view of this approach, it is obvious
that unless the original problem exhibits a substantial number of equality
structural constraints, the number of artificial vectors within the starting basis
will be relatively small.
Given that our starting point is an infeasible basic solution, it is clear that
there are two possible sources of infeasibility, namely: (1) non-negative artificial
variables (negative artificial variables will not be admitted to our discussion
since, if an equality constraint exhibits a negative right-hand side, it may be
multiplied by minus one so that the associated artificial variable enters the set of
basic variables at a positive level); and (2) negative legitimate variables. In this
regard, the ith unit column vector ei of the current basic solution is said to
belong to the index set ~ if the associated basic variable x Bi is non-negative and
artificial while ei belongs to index set GJ if x Bi is negative and legitimate, i. e.,
~ { i I x Bi 2: 0, x Bi artificial},
GJ {i I x Bi < 0, x Bi legitimate}.
To take account of all infeasibilities simultaneously for this particular basic
solution, let us define the total infeasibility of the same as
1= L (8.5)
i(~ u GJ
It is evident that I is always non-negative while I equals zero if and only if the
current solution is a basic feasible solution to the original linear programming
problem. Moreover, I is non-increasing at each iteration i.e., the criteria by
which vectors are chosen to enter and leave the basis guarantee that I decreases
at each round of the composite simplex process until an initial basic feasible
solution to the original problem with I = °is found.
To ascertain the effect on I of introducing a non-basic vector rj into the

basis, let us express (8.5) in terms of the non-basic variables x Rj' j=I, ..., n-m, as
(8.5.1)
n-m
constant + L
d jX Rj'
j=1
where d j = .2: Yij - ~ Yij' Since the constant term in (8.5.1) is non-negative, I
uc:f u~
will decrease if any non-basic variable x Rj for which d j < 0 enters the set of basic
variables. (If dj ~ 0 for all j but the constant term in (8.5.1) is positive, the
original problem has no feasible solution, i.e., I cannot be made smaller for any
nonzero values of the non-basic variables and yet must be positive for any
feasible solution. So if the problem has a feasible solution, until it finally is
found, it must be the case that d j < 0 for at least one j.) In this regard, d j
represents the reduction in feasibility achieved by changing the status of x Rj
from non-basic to basic and will consequently be termed the lh infeasibility
reduction tenn.
To determine which non-basic vector r j enters the basis let us adopt the
convention that
ENTRY CRITERION: d k = m~n{dj,dj < O}, (8.6)

J
i.e., x Rk enters the set of basic variables. Clearly this choice criterion yields the
greatest decrease in I per unit increase in any of the non-basic variables. Having
specified the selection of the pivot column, let us now look to the determination
of the pivot row. Here we seek to select the vector to be deleted from the current
basis. From our previous discussion we know that
x~
Bi
x Bi - OYik' if. r,
(8.7)
x~
xBi _ 0
Br Yrk - .
Now, if the total infeasibility I of the problem is to decrease at each round of the
composite simplex process, then, in order to preclude the creation of any new
infeasibilities, it must be the case that:
(a) no non-negative legitimate or artificial variable changes sign; and
(b) as many negative legitimate variables as possible are to change sign,
where rule (a) has priority over rule (b). To translate these rules into potential
pivot-row-selection criteria, let us relate our discussion to the previous two
guidelines. The following cases then emerge:
(a.1) Given that not all x Bi may be non-negative, no non-negative variable
changes sign so long as the vector to be deleted from the basis is chosen according to
01 = :::~ = min{ ~~: IxBi ~ 0 and legitimate or artificial, Yik > 0}. (8.8)
i.e., the r l th column of the basis matrix B is replaced by fk (here (8.8) is the
standard simplex exit criterion). Moreover, if there is a tie for the minimum,
choose r l so that x Brl is an artificial variable, if at all possible. Next,
(b.1) To eliminate the infeasibility caused by a negative legitimate basic
variable x Bi' let us: (1) remove x Bi from the set of basic variables and, at the
same time; (2) increase the values of all other negative legitimate basic variables
as much as possible by selecting the vector to be deleted from the current basis
according to
O2 = XyBr2
r2k
= m~x{Xy~i Ix Bi < 0 and legitimate, Yik < o}.
I Ik
(8.9)
Here x Br2 is to be replaced by x Rk in the set of basic variables.

What about the relative effect of the choice of a particular Oi' i=1,2, value
on the total infeasibility I of the problem? A moment's reflection indicates that
if 01 ~ O2, x Brl may be deleted from the set of basic variables without creating
any new infeasibilities, while if 01 > O2 , x Br cannot turn non-basic without at
1
least one of the remaining non-negative basic variables becoming negative and
this increasing the value of the total infeasibility I. In the light of these remarks
we shall now consolidate the above two rules into what will be called the
composite
EXIT CRITERION: to determine which current basic variable is to turn
non-basic, choose x Br if 82 < 81 < + 00; otherwise select x Br if 81 :s; 82 ,
2 1 (8.10)
Clearly this choice criterion guarantees that the total infeasibility I decreases at
each iteration and that, if possible, either a negative legitimate variable will be
transformed to zero or a non-negative artificial variable will be removed from the
set of basic variables. We note briefly that this criterion fails when
81 =82 = + 00, a result which emerges only if all positive artificial variables
have Yik:S; 0, and all negative legitimate variables have Yik ~ O. In this
circumstance d k ~ 0, in which case column k would not have been selected as the
pivot column in the first place.
EXAMPLE 8.11. To see how the composite method works, let us
max! = Xl + x2 s.t.
Xl + 2X2 < 12
Xl + x 2 > 4
-Xl + X2 > 1
-13x l + 4x2 -26
x l ,x 2 ~ O.
Upon multiplying the second, third, and fourth structural constraints by -1 with
x 3' x 4 and Xs depicting non-negative slack variables and x6 representing a non-
negative artificial variable, the above system becomes
12
-4
-1
26.
The simplex matrix associated with this problem is then
1 2 1 0 0 0 0 12
-1 -1 0 1 0 0 0 -4
1 -1 0 0 1 0 0 -1
[ll] -4 0 0 0 1 0 26
-1 -1 0 0 0 0 1 0
and our initial basic, though infeasible, solution is x Bl = x3 = 12, x m = x 4 = -4,

xm = Xs = -1, xB4 = x6 = 26, and 1=0 (here x Bi IS the basic variable
associated with the unit column vector ci)' Since ~={4} and GJ={2,3}, the
total infeasibility associated with the CUl'l'Cllt solution is, from (8.5),
I = x B4 - x m - x m = x6 - x 4 - Xs = 31.
In addition, from (8.5.1),
- 1 + 1 - 13 = - 13,
d2 = E Yi2 - E Yi2 = Y22 + Y32 - Y42 = - 1 - 1 + 4 = 2.
ifCiJ if~
Since d l = min{ di' di < O} = - 13, the non-basic variable x Rl = Xl entcrs the
J
set of basic variables. To determine the current basic variable which is to turn
non-basic, let us compare
91 min{ ~~i FBi ~ 0 and legitimate or artificial, Yil > 0}

. {XBl
m,n xB4 = 2 } = 2,
-Y = 12, -Y
i 11 41
92 = mfx{ ~:; FBi < 0 and legitimate, Yil < o} =~:: 4.
Looking to our composite exit criterion (8.10), we see that since 91 = 2 < 92 = 4,
xB4 = x6 will be removed from the set of basic variables. Upon pivoting on the
(boxed) element Y4l = 13 we obtain
0 [ID
13 1 0 0 1
-13 0 10
17 1
0 -13 0 1 0 13 0 -2
9 1
0 -13 0 0 1 -13 0 -3
4 1
1 -13 0 0 0 13 0 2
17 1
0 -13 0 0 0 13 1 2
Hence our second basic (infeasible) solution consists of xm = x3 = 10,

xm = x 4 = - 2, xBJ = x5 = - 3, x B4 = Xl = 2, and f = 2. Since the artificial
variable x6 has been driven from the set of basic variables, its associated column
within the simplex matrix will be deleted in subsequent calculations. Since now
g={¢}, <:f={2,3}, 1= -xm-xBJ= -x 4 -x 5 =5, and d2 =
LYi2 =
id
Y22 + Y32 = - 2, x Rl = x 2 is the only candidate for entry into the set of basic
variables. We next compare
In ;n{X Bl _ 13} _ 13
i• Y12 --3 --3'
. ~ ~ 13
Smce (}1 = (}2 = 3' we select xm = x3 to leave the set of basic variables.
Pivoting on the element Y12 = r~ yields
13 13
0 1 30 0 0 0 "3
17 11
0 0 30 1 0 0 "3
9
0 0 30 0 1 0 0
4 10
1 0 30 0 0 0 "3
17 23
0 0 30 0 0 1 "3
and thus an optimal (though degenerate) basic feasible solution to the original
problem wherein X B1 = x 2 = 133, X B2 = x 4 = 131, x B4 = X5 = 0, x B4 = Xl = 130, and
f -- 23
3·
COMPUTATIONAL ASPECTS OF
LINEAR PROGRAMMING:
9 SELECTED TRANSFORMATIONS
In this chapter our efforts will be directed towards solving a variety of

linear programming problems which do not exactly fit the basic structure of the
general problem considered in earlier chapters, i.e., a problem of the form
max /(X) = C'X s.t.
AX~b, X~o.
For instance, we may want to find the minimum rather than the maximum of a
hyperplane subject to AX ~ b, X ~ 0, or quite possibly the variables may be
unrestricted in sign rather than nonnegative, or the variables may be bounded
from below or above, and so on.
9.1. Minimizing the Objective Function

Let us assume that we desire to
min/(X) = C'X s.t.
AX~b, X~ o.
As we shall now see, to solve this minimization problem, transform it to a maxi-
mization problem, i.e., we shall employ the relationship min / = -max{ - n.
To this end we state
THEOREM 9.1. The minimum of /(X) = c'x occurs at the same point Xo
as the maximum of - /(X) = -C'X or
min/(X) = -max{-/(X)}. (9.1)
233
PROOF. If Xo maximizes - f(X) = - C'X, then - C'Xa 2: - C'X for

all X€9G = {XIAX~b,X~O}. Hence C'Xa$C'X for all X€9G so that Xo
minimizes f(X). Thus the optimal values of f and -fare the negative of each
other so that (9.1) holds. Q.E.D.
EXAMPLE 9.1. To apply this transformation we shall
minI = xl + X 2 s.t.
2xI + 4x2 > 28
5x I + x 2 > 25
x I ,x2 2: o.
From (9.1), let us maximize 9 = - I = - xl - x 2 subject to the above
structural constraints and non-negativity conditions. For x 3 ' x 4 surplus variables
and x S,x6 artificial variables (each non-negative), the initial and optimal simplex
matrices are respectively
2 4 -1 0 1 0 0 28
5 1 0 -1 0 1 0 25
-+
1 1 0 0 0 0 1 0
-7M -5M M M 0 0 0 -53M
[ :J
S 1
0 1 -18 9 0
1 2
1 0 18 -9 0
2 1
0 0 9 9 1 -9
Thus the optimal basic feasible solution consists of xl = 4, x2 = 5, and

maxg = max{-f} = - 9 or minf = - max{-f} = - (-9) = 9.
9.2. Unrestricted Variables
Another variation of the basic linear programming problem is that (one or
more of) the components of X may be unrestricted in sign, i.e" may be positive,
negative, or zero. In this instance we seek to
max I(X) = c'x s.t.
AX ~ b, X unrestricted.
To handle situations of this sort, the variable xk may be expressed as the

difference between two non-negative variables or
(9.2)
So by virtue of this transformation, the above problem may be rewritten, for

X = Xl -X2, as
max !(XI , X2 ) = C'X I - C'X2 s.t.
AX I -AX2 ~ b
Xl' X2 ~ 0,
where X~ (xt, ... ,x1), X~ = (x~, ... ,x!). Then the standard simplex
technique is applicable since all variables are now non-negative. Upon examining
the various subcases which emerge, namely:
()
a 'f
1 > Xk'2 t Ilen
XkI Xk> 0;
2 t Ilen
(b) 1'f XkI < x k' xk < 0; an d
()
C 1'f Xk
I = Xk'2 t hen Xk = 0,
we see that xk' depending upon the relative magnitude of xl and x%, is truly
unrestricted in sign.
One important observation relative to using (9.2) to handle unrestricted
variables is that any basic feasible solution in non-negative variables cannot have
both xl, xi in the set of basic variables since, for the kth column of A, ak'
(xl-xi)ak = xkak + xi( -ak) so that ak' - ak are not linearly independent and
thus cannot both constitute columns of the basis matrix.
EXAMPLE 9.2. Let us
max! = 3x I +x 2 +x 3 s.t.
2xI + 5x 2 - x3 < 10
Xl + 4x2 - x3 < 12
xl' x3 unrestricted, x 2 > O.
Setting
I 2
x1 Xl - Xl'
I 2
X3 - X 3 '
236 Lineal' Programming
we desire to
max f = 3xt - 3x~ + X 2 + X~ - X~ s.t.
2x~ - 2x~ + 5x 2 - X~ + x~ < 10
x~ - x~ + 4x2 + x~ - x~ < 12
I 2 I 2 0
Xl' Xl' X2' X3' X3? •
The standard simplex routine in all non-negative variables now applies, with
xC!J = (xl.)o
J
- (x~)o
J'
)'=1 ,3.
9.3. Bounded Variables

Oftentimes it may be the case that in addition to the usual structural
constraints and non-negativity conditions, a supplemental set of inequality struc-
tural constraints of the form Xj? ej or Xj ~ Uj' j=l, ... ,n, are introduced into a
linear programming problem. Here ej is a finite lower bound on the variable Xj
while Uj is termed a finite upper bound on Xj' (Previously, we held ej = 0 and

Uj = + 00.) In this regard, we may either
[Variables bounded below] [Variables bounded above]

max f(X) = C'X s.t. }
or {
nax f(X) = c'x s,t.
AX= b, X~L~ 0 AX = b, 0 ~ X ~ U,
where both
are of order (n x 1) (and a set of m non-negative slack variables have already

been incorporated into the original structural constraint system). Here both
AX =b, X ~ L ~ 0 and AX =b, 0 ~ X ~ U will be termed capacitated equation
systems. Moreover, it is evident from the presence of these lower or upper
bounds that now a feasible solution to the linear programming problem is one for
which each variable x j assumes a non-negative value not less than its lower
bound ej or not in excess of its upper bound Uj' j=l, ..., n.
Let us assume for the moment that we desire to

max f(X) = C'X s.t.
AX = b, 0 ~ X ~ U.
If this problem is handled in the usual fashion, we would, according to the stan-
dard simplex technique, introduce n additional non-negative slack variables into
the problem - one for each upper bound constraint x j ::; Uj, j=l, ... , n. (Here
each upper bound constraint is treated as a separate structural constraint.) The
resulting problem would then have m+n structural constraints in 2n variables.
In this regard, we would need a basis matrix of order m+n. And if we are to
max f(X) = C'X s.t.
AX = b, X ~ L ~ 0,
we would need a non-negative surplus variable and a non-negative artificial
variable for each lower bound constraint x j 2': Cj' j= 1, ... , n. In this instance the
final problem would involve m+n structural constraints in 3n variables, again
with a basis matrix of order m+n. Fortunately, since these lower and upper
bound constraints possess a very simplified structure, we may exploit this special
structure to our advantage by modifying the standard simplex method so that
only the coefficients on the first m structural constraints AX = b appear
explicitly in the simplex matrix while the lower or upper bound constraints are
handled indirectly. This being the case, we need only work with a basis matrix
of order m instead of m+n.
Let us first consider the case of a finite set of lower bounds on the variables
Xj,j=l, ... ,n. That is, let us
maxf(X) = C'X s.t.
AX = b, X ~ L ~ O.
To solve this problem, we shall utilize a transformation which converts the
original variables Xj with finite lower bounds Cj , j=l, ... ,n, to a new set of
variables with lower bounds of zero. In particular, we may measure each variable
Xj from its lower bound Cj , j=l, ... ,n, i.e., let us transform Xj 2': Cj to
Xj=Xj-C j 2':O, j=l, ... ,n. In matrix terms, X=X-L or X=L+X. Then
the original problem becomes

max f{X) = CI(L+X) s.t.
A(L+X) = b, X ~ 0,
or
ma.x f(X) = CIX s.t.
AX=b, X~ 0, (9.3)
where f{X) = f{X) - elL and b = b - AL. Here AL represents the portion of b
which is necessary to support the minimum allowable level of X, namely L. The
final solution is then obtained as Xo = L + Xo, f(Xo) = e'L + CIXO'
EXAMPLE 9.3. To employ this technique, let us
max f = + x 2 + x3
Xl s.t.
2x1 + x2 + x4 10
2x1 + 3x 2 + x3 + x5 16
Xl 2:2 >0
x 2 2:1 >0
x3 2: 0.5 >0
x 4 2: 1.5 >0
x5 2:2 > 0,
where x 4 ' x5 are slack variables. By virtue of (9.3), we desire to
maxf = 3: 1 +3: 2 +3:3 s.t.
2X1 + 3: 2 + 3: 4 3.5
+ X5 = 6.5
3:j 2: 0, j=1, ... ,5.
Setting up the simplex matrix we have
1 0 1 o o 3.5]
3 1 0 1 o 6.5 (9.4)
-1 -1 0 o 1 0
and an initial basic feasible solution consisting of 3:4 = 3.5, x5 = 6.5, and f=O. If
we introduce x3 into the set of basic variables, (9.4) becomes
1 o 1 o
o 3.5]
3 1o 1 o 6.5 (9.5)
2 o o 1 1 6.5
and thus the optimal basic feasible solution to the modified or transformed
problem is x3 = 6.5, x4 = 3.5 and f(X) = 6.5. Moreover, the optimal basic
feasible solution to the original problem is
2 0 2
1 0 1
Xo L+Xo 0.5 + 6.5 7
1.5 3.5 5
2 0 2
f (Xo) = C L + C Xo = 10.
I I-
We next examine the case where each variable x j possesses a finite upper
bound Uj' j=l, ... , n. That is, let us
max f(X) = C'X s.t..
AX =b, 0 ~ X ~ U.
As we shall now see, this situation is a bit more complicated to handle
computationally than the previous one which admitted lower bound restrictions
on the variables. To develop t.he (upper) bounded variable algorithm, let us start
with the simplex matrix
xl X2 Xp Xl)+! Xp +2 Xp +m f
all al 2 alp 1 0 0 0 bl
a 21 a 22 a 2p 0 1 0 0 b2
(9.6)
o 1
o o 1 o
from which we obtain

(9.7)
and thus an initial basic feasible solution consisting of xp+i = bi' i=I, ... , mj
f = O. Since the variables xl, ... ,x p+m have upper bounds ul, ...,u p+m respec-
tively, it is assumed that none of the basic variables exceed their upper bounds.
Hence the current solution is assumed feasible for both the non-negativity
conditions and the upper bound constraints.
Assume that Xl is selected by the simplex entry criterion to enter the set of
basic variables. Hence (9.7) becomes
X p+ 1 b1 - a ll x 1
x p +2 b2 -a 21 x l
(9.7.1)
Xp +m
f
We noted earlier that, according to the standard simplex exit criterion, the
largest allowable increment in Xl which does not violate the non-negativity
conditions and the upper bound restriction is m#n{bJail,ail > OJ. However, if
I
we explicitly incorporate such a restriction on Xl into our discussion, then the

largest allowable increment in Xl which does not violate the non-negativity
conditions and the upper bound restriction is
min{m/n{bJail,ail > O},u l } (9.8)
It is evident that if the minimum in (9.8) is not attained for u l ' then the
standard simplex pivot process is warranted, with any of the elements ail> 0,
i=I, ..., m, eligible for selection as the pivot. If the minimum in (9.8) is u1 ' then
Xl must equal u l after an iteration is performed. To this end let us transform

the upper bound constraint xl $ u l to
(9.9)
where xi is a non-negative slack variable. (Note that xl = u l for xi = 0, while

xi = u l when xl = 0.) Upon substituting xl = u l - xi into (9.7) we obtain
X p +1 (b l - anu l ) + anxi - a 12 x 2 - ... - alpx p

x p +2 (b 2 - a 2l u l ) + a 2l xi - a 22 x 2 - ••• - a 2p xp
(9.10)
(b m - amlu l ) + amlxi - a m 2X2 - ••• - ampxp
f ClU l - clxi + C2 X 2 + ... + CpXp •
And since xi is non-basic, xi = 0 so that xl = u l . (Here all of the values of the
:ji
basic variables bi - ail Ul are non-negative under this substitution since each of
these differences may be expressed as ail( :/1 - Ui ) ~ 0 for
l > ul ail> 0 and
by virtue of (9.8). Actually, for ail < 0, these values are still non-negative.) In
terms of the simplex matrix, the preceding transformation yields
X*
1 Xp +m f
o o (bCanu l )
o o (b 2 -a21 u l )
(9.11)
o o 1
o o o 1
So to obtain (9.11), in (9.6):

(1) update the first m components of the last column to bi - ailul and the
objective function v&lue to cl u l ;
(2) multiply the first column by -1; and
(3) change the heading of the first column from Xl to xi.
It is conceivable that as the simplex iterations progress xi' may be selected

for entry into the set of basic variables at the level of its upper bound. In this
instance, the preceding transformation (9.9) with xi' = U I - xl is again utilized,
with xl now assuming non-basic status. That is, steps (1) - (3) above are carried
out to obtain the new simplex matrix. In view of these remarks, it is evident that,
in general, either xi or xi may appear in the final or optimal simplex matrix. If
xi appears, then its optimal value x? is located in the usual fashion in the last
column of the optimal simplex matrix. And if xi appears, then (xi)° = ui - x?
so that x? = ui - (xi)°.
One additional contingency remains. It may be the case that, when some
variable changes its status from non-basic to basic and thus increases from zero
to some positive level, one or more of the other basic variables may exceed their
upper bounds. How may we insure that this situation will never occur? For
instance, in (9.7) with x 2 ' ••• ' x p all non-basic we again have
x p +! bl - anx}
x p +2 b2 -a2l x I
f
where the non-basic variable Xl is to enter the set of basic variables. Note first
that the values of the basic variables xp+i increase for negative values of ail'
i=I, ... ,m. Moreover, the upper bounds up+i can be attained if bj-ailxl =
up+j' i=I, ... , m. Thus Xl cannot be increased above the smallest (bj-up+i)/ajl
value, ajl < O. In this regard, the largest allowable increment in the entering
variable Xl which preserves the upper bound restrictions on the basic variables
xp+j, i=I, ... ,m, is
(9.12)
If, for instance, the minimum in (9.12) occurs for i=l, z.e., x p +! reaches its
upper bound up+! after an iteration is performed, then x p+! is set at its upper
bound by substituting x p+! = up+! - x;+l into (9.6) and then pivoting on the
element all. Here (9.9) has again been applied, where x;+! is a non-negative
slack variable. Relative to (9.12), (9.6) is transformed to
xp
alp 1
1 au au o ···0 0
alp
a2p -a 2 1 all 1 ···0 0
(9.13)
o ... 0 1
And since x;+! is now nonbasic, x;+! = 0 so that x p+1 = up+!. So to obtain
(9.13), in (9.6):
(1) update the first component of the last column to bl - up+!j
(2) multiply the (p+lyt column by -1;
(3) pivot on the element au; and
(4) change the heading on the (p+l)st column fmm x p+1 to x;+!.
Upon combining all of the aforementioned exit criteria, we see that the
largest feasible increase in Xl which preserves both the non-negativity conditions
and all of the upper bound constraints is
(9.14)
To summarize, the complete bounded variable exit criterion may be

expressed as follows: given that the simplex entry criterion selects column k as
the pivot column, the pivot row is determined according to
() = min{ (}I' (}2' (}3}'
where
(}2 = m#n{(bi-ui)/ajk,aik < OJ, where Uj is the upper bound of the

I
basic variable associated with row i; and

(}3 = uk·
If row r is chosen as the pivot row, then:
(1) if () = (}l' the standard simplex exit criterion emerges with
the element ark serving as the pivot.
(2) for () = (}2:
(a) if x t is the outgoing basic variable, update the rth com-ponent of the
last column to br - Uti
(b) multiply the outgoing basis vector by -1;
(c) pivot on the element ark;
(d) change the heading of the column associated with the out-going basis
vector from x t to x~.
(3) if () = (}3:
(a) update the first m components of the last column to bj - ajkuk and
add ckuk to the current objective function value.

(b) multiply the kth column by -1; and
(c) change the heading of the ktll column from xk to xl;.
EXAMPLE 9.4. To implement this routine we shall

max f = xl + ~x2 + 2x3 s.t.
Xl 2X2 + 2X3 + x 4 5
-~l 3
where :1:4,:1:5 are slack variables. The associated simplex matrix thus appears as
1 -2 2 1 o 0
[ _1 1 -1 0 1 0 (9.15)
-: -~ -2 0 o 1
with an initial basic but infeasible (in terms of the upper bound restrictions)
solution consisting of:l:4 = 5, :1:5 = 3, and f = O.
Let :1:3 enter the set of basic variables. With ° °= (3-5)/( -1) = =~, 2,
= °= °
1 2
and 03 = 2, let us select 0= min{Ol' 02' 03} 3 2 (since this choice of

involves less computational work than if °= ° 2 were selected). Then (9.15)
lIlt
becomes
:1:*
3
[ 1 ~-2 1 0 0
5 - 2(2)
-1 1 1 0 1 0 lID 3 - (-1)2 (9.16)
-1 -~ 2 0 0 1 I1l 2(2)
This time a basic feasible, although non-optimal, solution emerges where-in

:1:4 = 1, :1:5 = 5, and f = 4.
Next, given that °= °=
1 5, 2 (1-4)/( - 2) = ~,and 03 = 3, it follows that
0-0
-
_3
2-2"· Hence the (boxed) pivot element in (9.16) is -2. In this regard,
(9.16) transforms to
q.
:1:* :1:* :1:*
3 3 4
~ -2
~r'[-~
1
[ 1 -1 0 1 1 2" 0 0
-1 1 1 0 1 o0 5 --+ 16 0 0 1
-2" 1 0 (9.17)
7 3 25
-1 -~ 2 0 0 1 4 -~ 0 2" 4' 0 1 4"
Here our third basic feasible solution is :1:2 =~, :1:5 =~, and f = 2;.
Finally, for °= °=
1 21, 2 (~-3)/( -~) = 3, and 03 = 1, we have
0= 03 = 1. Then (9.17) becomes
x*1 X3* X*4
1 1
2 1 1 2 0 0 ~ ~- (-!)(1)
~
1
-6 0 0 -21 1 0 ~-(!)(1)
+1
~
7 7 3
4' 0 2 4' 0 1 25
4
with the optimal basic feasible solution involving x2 = 2, x5 = I~, and f = 2:.
Moreover, since xi = ui-xi' and xi = x; = x: = 0, it follows that xl = 1, x3 = 2,
and x 4 = 4.
Finally, it may be the case that the variables x j are bounded both from
below and above, i. e., x j is constrained to lie within a range of values or
o::; ej ::; x j ::; U j' j= 1, ... , n. Specifically, the problem at hand is to
max f(X) = C'X s.t.

AX =b, 0 ~ L~X~ U.
To handle this situation, let us measure x j from its lower bound. Hence
€j ::; xj ::; Uj is transformed to 0 ::; xj = xj - €j ::; Uj - €j = Uj' j=I, ... , n. In this
instance we have converted the original finite interval constraint on x j to one
involving only an upper bound restriction on the transformed variables xj'
j=I, ..., n. In matrix notation, the preceding problem becomes, for X = X - L or
X=L+X,
max f(X) = C'X s.t.
AX = h, 0 ~ X ~ U,
where U = u- L, h = b - AL, Xo = L + Xo, and f(Xo) = C'L + C'5to. To

generate a solution to this latter problem, the upper bound algorithm is then
utilized.
EXAMPLE 9.5. To illustrate this transformation, let us
max f = xl + 3x 2 + X3 s.t.
Xl - X2 + X3 + X4 10
2XI + X2 - 4X3 + X5 8
Xl + 3x 2 - 2X3 12
1 ::; xl ::; 3
3 ::; x 2 ::; 4
2 ::; x3 ::; 5
0.5 ::; X 4 ::; 2.5
1 ::; x5 ::; 3
4 ::; x6 ::; 6,
where x 4' x S' and x6 are non-negative slack variables. By virtue of the preceding
discussion, we seek to
max f = xl + 3x 2 + x3 s.t.
Xl - x2 + x3 + x4 9.5
2xI + x2 - 4x3 + X5 10
Xl + 3x 2 - 2X3 + X6 2
o::; Xl ::; 2
o::; x2 ::; 1
o ::; x3 ::; 3
o::; x 4 ::; 2
0::;x5::;2
o::; x6 ::; 2.
The upper bound technique may now be used to generate a solution to this trans-
formed problem.
9.4. Interval Linear Programming

An interval linear programming problem is one for which we seek to
max f(X) = C'X s.t.
hI ~ AX ~ h2' hI ~ h2
X~O,
where b~ = (bL ...,b~), b~ = (b~, ... ,b~J To generate a solution to the same,
let us write the above interval structural constraint system as
AX ~ b 2 1 {AX + Xsl
AX~ bJ or AX -Xs2+Xa
where XBl ' X s2 ' and Xa are (m x 1) vectors of non-negative slack, surplus, and
artificial variables respectively. Then either the M-penalty or two-phase method
may be used to solve the final problem.
Interestingly enough, if A is a pth order non-singular matrix, then we may
obtain an explicit solution to the interval linear program-ming problem in
unrestricted variables according to
THEOREM 9.2. 1 The optimal solution to the problem
max !(X) = C'X s.t.
b1 ~ AX ~ b 2 , b 1 ~ b 2 (9.18)
X unrestricted
with A of order (p x p) and non-singular, is given by
(9.19)
where the lh component of Xo is
(9.20)
and 3j is the lh column of A-I.
PROOF. Substituting X = AX into (9.18) yields the equivalent problem
p
max !(X) = C'A- 1X = ~ (C'3j)Xj
3=1
b 1 ~ X ~ b2 , b1 ~ b2 , X unrestricted, or
s.t. { 1 _ 2 1 2 - . .
bj ~ Xj ~ bj , bj ~ bj , Xj unrestrIcted, )=1, ... ,p,
1Ben-Israel, et al. (1969). A more general fOl'mulation of this theorem which allows A
to be of order (m X p) and thus employs the generalized inverse appears in Ben-Israel, Charnes
(1968).
whose optimal solution is provided by (9.20). This latter expression may then be
used to determine (9.19). Q.E.D.
EXAMPLE 9.6. Let us
max f = xl + 8x 2 s.t.
3 < xl + x2 < 12
4 ~ 2xI + 4x2 < 20
xl' x 2 unrestricted.
Since
this problem is equivalent to

max f = - 6-Xl + 2x2
7- t
s ..
3 ~ xl ~ 12, 4 ~ x2 ~ 20
xl' x2 unrestricted.
With C'a,. = -6 and C'~ =~, (9.20) yields x~ = 3, xg = 20. Then from (9.19),
x~ = -4, xg = 7.
9.5. Absolute Value Functionals
The linear programming problem to which we shall now address ourselves is
of the form
n
max f(X) = E I Xj I s.t.
j=l
AX = h, X unrestricted.
Since Ix j I is essentially non-linear, the objective function may be linearized by
utilizing the transformation Xj = x} - x;, with x}, x; ~ O. Then for X = Xl - X2 ,
where again X~ = (xL ... ,x~) and X~ = (x?, ... ,x~), the preceding problem may
be rewritten as
As pointed out earlier, x} and x~ cannot both appear in any basic solution since
their associated columns aj' - aj are not linearly indenpendent.
And if the initial problem appears as
n
max f(X) = ?: I I
Cj Xj s.t.
J=l
AX = b, X unrestricted,
then the above transformation X = Xl - X2 , with Xl' X2 ~ 0, allows us to

rewrite this problem as
max f(X I ,X2 ) = C'X I - C'X2 s.t.
AX I -AX2 = b
X I ,X2 ~ o.
Interestingly enough, this latter problem possesses a standard simplex solution
only if each Cj term is non-positive (for a minimization problem, the c/s must all
be non-negative) (Shanno, Wei! (1971); Rao (1972)).
THE DUAL SIMPLEX, PRIMAL-DUAL,
AND COMPLEMENTARY
10 PIVOT METHODS
In this chapter we shall first consider two important computational variants

of the standard simplex method, namely the dual simplex and primal-dual
routines. These techniques are essentially modifications of the simplex method
because their implementation involves a sequence of pivot operations applied to
the primal problem, but based upon alternative pivot selection or entry and exit
criteria. These methods were developed to: (1) reduce the number of iterations
needed to achieve optimality when the number of variables (n) is large; and (2)
exploit situations involving the availability of an initial basic feasible solution to
the dual problem
Both of these algorithms start with an infeasible basic solution to the
primal problem and, through a sequence of pivot operations, strive for feasibility
in a fashion such that, once attained, the resulting basic solution is also optimal.
In this regard, there is no need for the usual-Phase I calculation, i.e., since the
initial basic feasible solution to the original primal problem found at the end of
Phase I may be far from optimal (Phase I ignores primal optimality and seeks
only primal feasibility by driving the artificial variables to zero), these alternative
methods are constructed so that they simultaneously work towards primal
feasibility and optimality. The starting point for each is a dual feasible but
primal infeasible solution and, at each iteration, dual feasibility as well as
complementary slackness is maintained. When dual as well as primal feasibility
is achieved, the algorithms are terminated.
251
A third simplex variant, called the complementary pivot method, actually

utilizes a formal complementarity requirement (constraint) to facilitate pivoting.
A linear program is first converted to an equivalent linear complementarity
problem. Then under a complementary pivot rule, we move from what is called
an almost complementary basic solution to another until we attain a complemen-
tary basic solution (containing exactly one basic variable from each
complementary pair of variables). At this point we have an optimal solution to
the original linear pl'Ogram and the algorithm is terminated.
10.1. Dual Simplex Method

In attempting to generate a optimal basic feasible solution to a linear
programming problem, our procedure, as outlined in chapters five and seven, was
to employ the primal simplex method. To briefly review this process, we start
from an initial basic feasible solution for which the optimality criterion is not
satisfied, i.e., not all Cj :::; 0, j=l, ... , n - m (the dual problem is infeasible). We
then make changes in the basis, one vector at a time, maintaining nonnegativity
of the basic variables at each iteration, until we obtain a basic feasible solution
for which all Cj ~ 0, j=l, ... , n - m (so that dual feasibility now holds). Hence
the primal simplex method is a systematic technique that preserves primal
feasibility while striving for primal optimality, thereby reducing, at each round,
dual infeasibility, until primal optimality, and thus dual feasibility, is ultimately
achieved. This last observation precipitates a very interesting question.
Specifically, can we solve a linear programming problem by a pl'Ocedure which
works in a fashion opposite to that of the primal simplex algorithm? That is to
say, by a method which rela.xes primal feasibility while maintaining primal
optimality. As we shall now see, the answer is yes. In this regard, we now look
to an alternative simplex routine called the dual simplex method (Lemke (1954»,
so named because it involves an application of the simplex method to the dual
problem, yet is constructed so as to have its pivot operations carried out within
the standard primal simplex matrix. Here the starting point is an initial basic
Dual Simplex, Primal-Dual, and Complementary Pivot Methods 253
but nonfeasible solution to the primal problem for which optimality (and thus
dual feasibility) holds. As is the usual case, we move from this solution to an
optimal basic feasible solution through a sequence of pivot operations which
involve changing the status of a single basis vector at a time. Hence the dual
simplex method preserves primal optimality (dual feasibility) while reducing, at
each iteration, primal infeasibility, so that an optimal basic feasible solution to
both the primal and dual problems is simultaneously achieved.
Let us now determine exactly how the dual simplex method works. Given
the primal maximum problem in standard form
max f(X) = C'X s.t.
AX = b, X~O,
the dual minimum problem is of the form
min g(U) = b'U s.t.
A'U ~ C, U unrestricted.
Moreover, let X B = B- l b = b represent a basic solution to the primal maximum
problem for which not all x Bi ~ 0, i=I, ... , m. Since X B is not feasible, the solu-
tion vector U to the dual minimum problem is not optimal. In this regard, we
should be able to find a new vector U such that A'U ~ C and g(U) < g(U). Now,
where {Ji' i=l, ... , m, denotes the ith row of B- 1 • Hence x Bi = {Jib, i=l, ... , m.
Moreover, since
it follows that
6· .
IJ
_{I, i=jj
...J..
0, iri,
where i, j=l, ... , m. For some i=r let x Br = (Jrb < O. Since we desire to
ultimately obtain an optimal basic feasible solution to both the primal and dual
problems simultaneously, let us: (1) eliminate x Br from the current set of primal
basic variables (and thus b r from the current primal basis); and (2) use U to
obtain a new solution vector U for which g(U) < g(U) and A'U ~ C. If we form
= U-O{Jr'
~,
U
then
g(U) = U'b = U'b - O{Jrb g(U) - OXBr (10.1)
and thus, for 0 < 0, g(U) < g(U) since x Br < O.

Let us pause momentarily to consider two important aspects of the dual
simplex algorithm. First, since g(U) < g(U), it is evident that 9 starts out at a
superoptimal level and monotonically decreases to its optimal feasible value.
Second, to obtain the largest possible decrease in 9 at each iteration, it follows
that the difference g(U) - g(U) = Ox Bi' x Bi < 0, should be as large as possible.
In this regard, to select the variable to leave the set of basic variables, let us
adopt the convention
EXIT CRITERION: xBk = m#n{xBi,xBi < OJ, (10.2)

1
in which case the kth column of the primal basis matrix B, b k , is removed from
the basis. So from this point on, we shall assume that the variable xBr in (10.1)
satisfies (10.2).
Under what circumstances will U satisfy the dual structural constraint
system A'U ~ C? Since U' = C' BB-l, u'n = C' B or U'bj = C Bi' i=l, ... , m.
Then for those vectol'S b i within the primal basis,
~ {C Br - 0 ~ CBr' 0 ~ 0, i=r ;
U'b.1 = U'b. -
1
O{J r b·I=IcB'I Oc·· =
- } •...J..
cBi' '.,-r,
where i=l, ... , m. And for all vectors j not in the primal basis,
,
f
~,
U fj = U fj - O{Jrfj'
Since fj = BYj , Y j = B-1fj has as its rth component Yrj ={Jrfj' Then
~, ,
U fj = U fj-OYrj'
Here two cases emerge. First, if Yrj ~ 0 for all j, then for any () ::; 0,
~, .
U rj = CRj - (}Yrj ~ CRj' 3=1, .. .,n - m.
In this instance () can be made an arbitrarily large negative number while still
maintaining the feasibility of U so that g(U) = g(U) - (}x Br becomes arbitrarily
small, i.e., the dual minimum problem has an unbounded feasible solution. But
if this is the case, our previous results indicate that the primal maximum
problem has no feasible solution. Next, let us assume that there exists at least
one non basis vector rj for which Yrj < O. For all such Yrj < 0 to satisfy
~,
U r·J
,
= U r J. - (}y rJ. > cR .
- J
it must be true that
In this instance () cannot be made an arbitrarily large negative number without

violating the feasibility of U. From the previous expression
U'r ,-cR'
() > J J
- Yrj
and thus, to preserve dual feasibility, let us choose a (), call it 0, according to
- {U\-CRj
() = m~x y . ' Yrj < 0} .
J rJ
Since U'rj - C Rj = C~B-Irj - C Rj = - Cj ~ 0, the preceding expression may be

alternatively written as
ENTRY CRITERION: {j = -Y ck = mq,x {-Y c.j, Yrj <
rk J rJ
o}. (10.3)
We now have a solution to the dual problem for which

~
U'b· =
{c Br - 0 ~ C Br' 0 ::; 0, i=r ;
cm, i*r, i=1, ... ,m;
I
U\ ~ CRi' j*k, j=1, ... , n - m,

~,
U rk = U,rk -
- (}Yrk = C Ilk'
-
() ::; 0;
g(U) = g(U) - Ox Br'
With 0 ::; 0, g(U) < g(U) if x Br < O. And with Yrk f 0, rk replaces b r in the basis
to yield a new basis matrix
B= [bl ,. .., b r _ l , rk' b r +l , .. ·, b m l·
10.2. Computational Aspects of the Dual Simplex Method

Having discussed the theoretical underpinnings of the dual simplex method,
let us now examine the type of problems to which it is applicable along with the
basic rudiments of their solution. Specifically, some of the more common
instances in which a dual feasible solution is readily obtainable are:
(1) the primal problem is such that a surplus variable IS added to
each" 2::" structural constraint (as we shall see below, the dual simplex
method enables us to circumvent the procedure of adding artificial
variables to the primal structural constraints); and
(2) an additional structural constraint is added to the optimal simplex matrix.
For instance, the dual simplex method is readily applicable if the primal problem
appears as:
min f(X) = C'X s.t.
AX ~ b with C, X ~ 0
or
max h(X) = - f(X) = - C'X s.t.
- AX ~ - b with C, X ~ O.
To see this let us consider
EXAMPLE 10.1. Using the dual simplex method,
min = 2xI + 4x2
f s.t.
Xl + 3X2 > 20
Xl + X2 > 15
2xI + x 2 > 25
x 1 ,x 2 2:: 0.
Upon converting this problem to a maximization problem and subtracting non-
negative surplus variables x3' x 4, and Xs from the left-hand sides of the
structural constraints, we obtain the simplex matrix
1 3 -1 0 0 0 20 -1 -3 o 0 0 -20
1 1 0 -1 0 0 15 -1 -1 o o 0 -15
--+
2 1 0 0 -1 0 25 [;1] -1 o 0 1 0 -25
2 4 0 0 0 0 2 4 o 0 0 1 o
and an initial primal optimal (dual feasible) basic solution x3 = -20, x 4 = -15,
x5 = -25, and h = 0, where h = -f. Since the current solution is not primal
feasible, let us undertake the first round of the dual simplex method by
determining
-25 = mjn{xBi' XBi < O} = min{-20,
I
-15, -25},
-CO }
(j = -1 = mJx{ _Y:j' -Yrj < 0 = max{2/(-2) = -1, 4/(-1) = -4}.
Here xl is to replace x5 in the set of primal basic variable (alternatively, r 1
replaces h3 in the primal basis) so that the (blocked) pivot element is -2. An
appropriate pivot operation yields
0
8J
1
1 0 -21 0 -2
15
0 -2 0 1 -21 0 5
-2
1
1 2 0 0 -21 0 25
2
0 3 0 0 1 1 -25
· lb·
an d a second prima aslC soIutlOn
· conslstmg
. . 0 f Xl =2'
25 x3 = -2'
15 x = -2'
4 5 an d
h = -25. Again primal feasibility does not hold so that we next compute:
15 = mIn XBi' XBi < O}
-2 0 {
= mm 0 { 15 -2
-2' 5}'
If we now undertake an additional pivot operation with -25 as the pivot we

obtain
1
0 1 -s2 0 S 0 3
0 0 -s1 11-~1 0 -1
1
1 0 S 0 -s3 0 11
6 2
0 0 S 0 S 1 -34
and thus a third primal basic solution wherein Xl = 11, X2 = 3, x 4 = -1, and
h = -34. Since primal infeasibility is still evident, we again look to our
respective exit and entry criteria
-1 = n#n{xBi' xBi < O} = min{-I},
- {-Cj }= {(6/5)
J
(2/5) }
O=-I=mjx _Yrj'-Yrj<O max (_1/5)=-6'(_2/5)=-1
to obtain a pivot of -~. A final iteration yields
0 1 -2I I
2 0 0 5
2
I 5 5
0 0 2 -2 1 0 2
I 3 25
1 0 2 -2 0 0 T
o 0 1 1 0 1 -35
and an optimal primal basic feasible solution with Xl = 225 , X 2 =~, X5 =~, and
f = 35.
It was pointed out at the beginning of this section that the dual simplex
method is particularly useful in instances where "~" structural constraints are
present. In this regard, the dual simplex method can be conveniently used to
solve the dual problem
min g(U) = b/U s.t.} max h(U) = -g(U) = -b/U s.t.
or {
A/U ~ C, with b, U ~ 0 - A/U ~ -C, with b, U ~ 0
for both the primal as well as the dual variables. The simultaneous augmented
form of this problem appears as
-A/U + In -C
h'U +h o
so that the associated simplex matrix is
[
-A' In
b' 0'
0
1
-c].
0
Relative to this system the (n+ 1 x n+ 1) basis matrix and its lllverse may be
expressed respectively as
_ [-E
E =
b'E
o ],
1
--1
E = [_E-
E
b E- 1
1
0]1 ,
where E is an (n x n) basis matrix for the dual structural constraint system. Then
E- 1A' o
-bEE- 1A' +b' 1
Notice that this latter matrix is of the same basic form as (S.4) save for the
fact that E- 1 appears instead of 0- 1 . Hence the primal vectors X, Xs are readily
obtainable as before.
10.3. Dual Degeneracy

We now turn to some additional remarks concerning the value of 8 which
constitutes the maximum of (10.3). Let us assume that at some point in the
dual simplex routine it turns out that (j is not unique. Hence we may introduce
into the primal basis anyone of the columns of R for which the same maximum
value of 8 is assumed and obtain a new (call it the eh ) primal basic solution. In
this instance Cj = 0 for all j corresponding to those columns that have the same
8 = (j as the incoming basis vector and thus the k+1 st iteration is executed with
(j = 0, i.e., the case of dual degeneracy emerges. Moreover, for the k+ 1st primal
basic solution, the primal objective value is unchanged. (Note that the dual
objective value must also remain invariant since, from (10.1), g(U) = g(U)). In
fact the primal objective value may remain unchanged for several successive
iterations. However, in the absence of cycling, the dual simplex algorithm will
eventually converge to an optimal basic feasible solution to both the primal and
dual problems simultaneously.
IDA. Summary of the Dual Simplex Method

Let us now consolidate our discussion of the theory and major computa-
ional aspects of the dual simplex method. If we abstract from the possibility of
degeneracy, then a summary of the principal steps leading to an optimal basic

feasible solution to both the primal and dual problems simultaneously is provided
by the:
10.1. DUAL SIMPLEX METHOD.
STEP O. Initialize.
Obtain an initial dual feasible (primal optimal) solution with
('j~O, j=l, ... ,n-m.
STEP 1. If primal feasibility also holds, then the current solution is
an optimal basic feasible solution to both the primal and
dual problems. If not all x Bi ~ 0, i=l, ... , m, (primal
feasibility is relaxed), then proceed with the next step.
STEP 2. Find xBI' = min{xBi,xBi
, < O}. The vector b r corresponding
to the minimum is to be removed from the basis.
STEP 3. Determine whether -Yrj ~ 0, j=l, ... , n - m. If the answer is
yes, the process is terminated since, in this instance, the pri-
mal problem is infeasible and thus the dual objective function
is unbounded. If the answer is no, step 4 is warranted.
STEP 4. Compute e= mjx{ ~::j '-Y"j < 0}. If ~:r~ =e, then rk
replaces b" in the basis.

At this point let us contrast the pivot operation involved in the dual simplex
method with the OJle encountered previously in the primal simplex method.
First, only negative elements can serve as pivot elements in the dual simplex
method whereas only positive pivot clements are admitted in the primal simplex
routine. Second, the dual simplex method first decides which vector is to leave
the basis and next determines the vector to enter the basis. However, just the
opposite order of operations for determining those vectors entering and leaving
the basis holds for the primal simplex method. Next:
STEP 5. Determine the new dual feasible (primal optimal) solution by
pivoting on -Yrk' Return to step 1.
By continuing this process of replacing a single vector in the basis at a time, we

move to a new dual feasible solution with the value of the dual objective function
diminished and primal infeasibility reduced. The process is terminated when we
attain our first primal feasible solution, in which case we attain an optimal basic
feasible solution to both the primal and dual problems. 1
10.5. Generating an Initial Primal-Optimal Basic Solution:

The Artificial Constraint Method
One of the major difficulties associated with the implementation of the dual
simplex method is that an initial primal-optimal (dual-feasible) basic solution
may not be readily available. In this regard, steps must be taken to construct
such a solution and to this end we now proceed. For instance, let us assume that
the primal problem appears as
p
?:aijxj ~ bi' i=l, ... ,m, with Cj, Xj ~ 0, j=l, ... ,p.
3=1
Upon introducing non-negative surplus variables x p+i' i=l, ... , m, we can readily
obtain an initial primal-basic but not primal-optimal (and thus dual infeasible)
solution x p+i = -bi' i=l, ... , m, with f = O. At this point let us add to the
original problem the artificial structural constraint
(artificial in the sense that it is meaningless in terms of the formulation of the

original structural constraint system which contains only legitimate constraints),
where the index set .A. = {j Ie j < O} and M is an arbitrarily large positive
number. For x M a non-negative artificial slack variable, the previous inequality
becomes
I For the details on the steepest edge modification of the dual simplex method see
appendix B.
Since our immediate goal is primal optimality, let us employ the criterion
(10.4)
to express the eth legitimate variable as
This expression for x, is then substituted into the original objective function and
structural constraint, system and also included in the latter as a supplementary
structural constraint. Once this is accomplished, primal optimality is attained.
Moreover, if one or more of the primal basic variables xp+i = -bi+ailM,
i=l, ... , m, is negative, then the dual simplex method is directly applicable.
As far as the salient features of the artificial constraint method are
concerned, we briefly note that: if at some point in t.he dual simplex routine the
vector corresponding to x M enters the primal basis, then a basic solution to the
original problem is ohtained by ignoring both the row and column to which it
corresponds; moreover, the row and column associated with xM may be deleted
from the simplex mat.rix and the dual simplex iterations continued until primal
feasibility is attained.
EXAMPLE 10.2. To view the artificial-constraint method in action,
max f = xl + 3x 2 s.t.
Xl + X2 < 15
xl +4X2 > 20
5x1 + x2 > 25
xI,x2~0.
With x3 a slack variable and x 4 • x5 surplus variables (each non-negative) we

form
Xl + X2 + X3 = 15
- xl - 4x2 + X4 = -20
- 5x 1 - X2 + X5 = -25
- X1 - 3x 2 + / 0
Clearly an initial basic solution derived from the system is both primal and dual
infeasible. To achieve dual feasibility, let us introduce the artificial structural
constraint
Xl +X2+ XM = M.
Since Ic21 = rrax{ ICj I, Cj < OJ, we substitute x 2 = M - XM - xl into the
Jf.A
above simultaneous form to obtain
X3 - xM = 15-M
3x 1 + x4 + 4x M = -20+4M
-4x1 + x5 + xM = -25+M
xl + X2 + XM =M
2X1 +3 XM +/ =3M
with simplex matrix
0 0 1 0 0 8l 0 15-M
3 0 0 1 0 4 0 -20+4M
-4 0 0 0 1 1 0 -25+M
1 1 0 0 0 1 0 M
2 0 0 0 0 3 1 3M
and initial primal-optimal (dual-feasible) basic solution x 2 = M, x3 = 15-M,

x4 = -20+4M, x5 = -25+M, and / = 3M. Since the current solution is not
primal feasible, we look to
15 - M = m#n{xBj, xBi
I
< OJ,
(j = -3 = mr{.=-:;j' - Yrj < O},

which yields a pivot of -1, to wit X M replaces x3 in the set of primal basic
variables. Pivoting on -1 results in
0 0 -1 0 0 1 0 -15+M
3 0 4 1 0 0 0 40
g] 0 1 0 1 0 0 -10
1 1 1 0 0 0 0 15
2 0 3 0 0 0 1 45
Thus our second primal basic solution becomes x 2 = 15, x4 = 40, x5 = -10,
xM = -15+M, and f = 45. Since x M is feasible, row one and column six may
be deleted from this latter simplex matrix. And with
-10 = mjn{xBi' xBi<O},
my {-c. }
I
1
"0 = -2 = -Y:j'
X - Yrj <0 ,
we pivot on the element -4 to obtain
19 3 65
0 0 "4 1 4 0 2
5
1 0 -41 0 -41 0 2"
5 1 25
0 1 4 0 4 0 2
7 1
0 0 2" 0 2 1 40
and an optimal primal basic feasible solution consisting of x 1 -- 2'

5 x - 25
2-2'
=
x 4 625 and f =
40.
10.6. Primal-Dual Method

The primal-dual algorithm (Dantzig, Ford, Fulkerson (1956)) requires that
we solve an adjusted Phase I problem. The legitimate vectors selected to replace
the artificial vectors in the basis are chosen in a way that is quite different from
the usual Phase I entry criterion. In this regard, we begin with a feasible solution
to the dual problem from which we attempt to reduce primal infeasibility by
inserting only certain selected legitimate primal vectors into the primal basis,
i.e., we utilize what is called restricted basis entry when solving the pdmal
problem. We next employ the standard simplex method as a subalgorithm until
optimality is achieved for those vectors marked for entry into the primal basis.
An improved solution to the dual (in the sense that the dual objective value is
made to decrease) is thus obtained which in turn targets a new set of legitimate
primal vectors as candidates for (restricted) entry into the primal basis. This
process is repeated wit.h complementary slackness maintained at each step so that
the first primal feasible solution obtained is also optimal. Hence the primal-dual
routine, so named because we attempt to increase the value of the Phase I
infeasibility form and, at the same time, decrease the dual objective function
value, works towards primal feasibility and optimality at the same time.
Let the primal-dual pair of problems under consideration appear as
PRIMAL DUAL
max f(X) = C/X s.t. min g(U)= b/U s.t.
AX= b, X~O A/U ~ C, U unrestricted. (10.5)
If C ~ 0, then it is easily seen that U = ° is a feasible solution to the above dual

problem. If ci 1: 0 for all j, then a feasible solution to the same is not readily
available. However, to generate an initial dual feasible solution, let us modify
(10.5) slightly by int.roducing the artificial constraint Xo + I/X = bo into the
primal problem (Beale (1954». Here Xo ~ 0 and bo is taken to be an unspecified
large positive constant. In addition. the primal objective function coefficient
associated with Xo (call it co) is zero. Clearly the introduction of this constraint
does not affect the values of the primal variables. In this regard, the modifica-
tion of (10.5) appears as
MODIFIED PRIMAL MODIFIED DUAL

max f(X) = c/x s.t. min 9 = b/U + bouo s.t.
(10.6)
Xo ~ 0, X ~ 0, Co = 0 U o ~ 0, U unrestricted,
where ai' j=O, 1, ... , n, denotes the lh column of the modified primal coefficient
matl'lx,
•
z.e.,
.
aa = cm +1 • ai = ai' 1),and aj
- -I (' .
IS th e J·th co I umn 0 fA· 1 ... , n.
,J=,
An obvious feasible solution to the modified dual is thus Uo ~ 0, U = 0, and
lu o ~ C or U o = m~x·{cj}' j=I, ... ,n.
J
If we now introduce nonnegative dual surplus variables uso' Us into the

modified dual program, the latter becomes
min 9 = b'U + bouo s.t.
[f, i ][~]-[~d =[ ~] (10.7)
U o ~ 0, U unrestricted, uso ~ 0, Us ~ O.
Note that if U o = 0, any feasible solution to (10.7) is also a feasible solution to

the original dual problem appearing in (10.5).
Let us now return to the modified primal problem and introduce the
artificial variables xaO ~ 0, Xa ~ 0 as we would for the usual Phase I calculation.
This yields what is called the extended primal problem
max f* = -I'Xa - xaO s.t.
(10.8)
In what follows we shall use a variation of this problem to carry out the primal-
dual pivot operations.
To see exactly how the primal-dual method works, we note first that if the
vectors X, U represent feasible solutions to the primal-dual pair of problems in
(10.5), then they are also optimal solutions to the same if and only if the
following weak complementary slackness conditions hold:
(a) xou so + X'(C-A'U) = xouso + X'Us 0;

(10.9)
(b) uo(bo-xo-l'X) + U'(b-AX) = o.
(Since the structural constraints of the primal are strict equalities, (10.9.b) is
automatically satisfied when a feasible primal solution x O' X is obtained.)
Relative to the current feasible solution to the original dual in (10.5), some of the
structural constraints of this problem are binding and some are not. For those
which are satisfied as equalities (usj=O), the corresponding x j can be any positive
value and (lO.9.a) holds. If one or more of these constraints hold as a strict
inequality, then the corresponding x j = 0 so that again (10.9.a) obtains. How
can we find a vector X that satisfies (l0.9.a) (complementary slackness is
preserved) and the constraints of the primal problem in (10.5)? We noted above
that an initial feasible solution to the modified dual (and thus to (lO.7») is
U = 0, Uo = m«;lx{cj},j=I, ...,n. Then from the partitioned rows of (10.7) we
J
have
Uso = Uo = m~x{cj}, Us = luo - C = 1 m«;lx{cj} - C, j=I, ... ,n.

J J
(10.10)
Clearly at least one component of (uso, U~) must vanish by virtue of the
definition of uo. Moreover, for this solution, 9 = b'U + bouo = bouo. For the
current feasible solution to the dual, let' = {j I Usj = O}, i.e., , represents the set
of subscripts j corresponding to the dual binding structural constraints. The
importance of index set , is that it provides us with a device for using
information about the dual with which to select legitimate vectors aj to enter the
primal basis. (Initially the primal basis for (10.8) contains only artificial
vectors.) That is, given that the complementary slackness conditions must be
maintained throughout all of our computations, if j f " then any x j ~ 0 will
satisfy (10.9.a) and thus aj can be admitted to the primal basis of (10.8); and if
U, so that Usj > 0, then (10.9.a) is satisfied only if x j = 0 so that aj remains
nonbasic. (Since U o = m«;lx{ C j}, j=I, ..., n, , :f:. tP initially so that at least one aj
J
is eligible for restricted basis entry in (10.8).) Hence we allow only those aj' j d,
to enter the primal basis at each iteration. These observations on restricted basis
entry thus lead us to construct what is called the restricted primal problem (a
f',
modification of the standard Phase I calculation): we must find a nonnegative
linear combination of the vectors aj' j which equals (b', bo) by solving
max f; = -1'Xa - xaO s.t.
2: ajXj +lXaJ = [bb]0

• CI. xaO
= b (10.11)
Jfd
Xj ~ 0, j=O, 1, .. .,n, xaO ~ 0, Xa ~ O.

To generate a solution to (10.11) we can apply the primal simplex method in the
usual fashion; terminating the calculations when Cj ~ 0 for all aj' j e', entering
the primal basis.
If f; = 0, then xai = 0, i=l, ... , m, and the resulting set of x /s for which
j d along with those x /s for U, yield a feasible solution to the primal in (10.6)
(since the constraints in (10.11) with xai = 0 are the same as those in (10.6) with
the columns aj' j e', deleted). Moreover, this solution is also optimal since the
x /s satisfy the complementary slackness conditions. Hence f = g for the pair of
problems in (10.6). How does this discussion relate to the solution of the original
pair of problems in (10.5)? Specifically, if x O' X is a feasible solution to the
modified primal and (10.9.a) holds, then x O' X is also an optimal solution to this
problem and thus uO' U is an optimal solution to the modified dual. And if
Uo = 0, then f =g =9 with X'U s = O. Thus X is an optimal solution to the
original primal in (10.5) and U is an optimal solution to its dual. (However, if
there exists a solution to the modified dual for which uo::fi 0 and (10.9.a) is in
effect, then with f = g = btU + bouo, bo and thus f can be made arbitrarily large
so that the modified and thus the original primal objective is unbounded.) These
comments may be summarized by the
10.2. PRIMAL-DUAL OPTIMALITY THEOREM. Solutions (U', uo), (xo, X')
to the modified dual and restricted primal problems respectively with
1*=0, uo=O, and x j=O, j ¢" are optimal for the original primal-dual
pair of problems.
PROOF 2 • With 1* = 0 we have Xa = 0, xa = O. And with U o = 0,
the vectors X, U satisfy the original primal and dual structural constraints, i.e., if
we premultiply both sides of the structural constraint system in (10.11) by
(U t, uo) = (U',O) we obtain, for Xa = 0, xao = 0,
~ U'3jXj = U'b.
e'
)e'
Since for j we have U'3j = Cj' the preceding expression becomes
2This proof is a slight variation of the one provided by Dantzig, et al. (1956).
Thus X, U are optimal solutions to the original primal and dual problems
respectively. Q.E.D.
If f; < 0 and V is an optimal solution to the dual of (10.11), z.e., to the
restricted dual problem
min g; = b'y s.t.
ajY ~ 0, jd (10.12)
Y~ 1,
where Y' = (U', uo)' then we can demonstrate that there exists some () > 0 such
that Y* = Y + ()V is a new feasible solution to the dual in (10.6) and that y*
produces a decrease in the dual objective, i.e., b'y* > b'Y, where Y is a feasible
solution to the said dual. To this end let us consider the following two possible
ways in which this can occur.
First, it may be the case that aj V ~ 0, for j f- j. In this case the primal in
(10.6) has no feasible solution. To see this we may note that if V is an optimal
solution to (10.12), then ai V ~ 0, j E~; and since by assumption ai V ~ 0, N~, it
follows that aj V ~ 0 for all j = 0,1, ... , n. If Y is a feasible solution to the dual
problem in (10.6), then so is Y* = Y + ()V, () > 0, i.e.,
aly*
J
= a'V
J
+ ()a'·V
J
> c J.
-
since aj Y ~ Cj and aj V ~ 0, j=O, 1, ... , n. Furthermore, the dual objective in
(10.6) can be made arbitrarily small, i.e., since b'y* = b'Y + ()b'V and
b'V = g; = f; < 0, it follows that as () becomes arbitrarily large, b'Y* decreases
without bound. Thus from theorem 4.9, the modified primal in (10.6) has no
feasible solution.
Second, let aj V < 0 for at least one j I j. Since V is an optimal solution to
(10.12), we know that aj V ~ 0, j {~. And for a suitably restricted value of () > 0,
y* = Y + ()V is feasible for the dual in (10.6). To see this let us increase the
value of () to the point where one of the inequalities ai Y* > C j' j I~, j=O, 1, ... , n,
becomes an equality, i.e., the largest allowable increase in () which does not
violate dual feasibility in (10.6) is
(10.13)
Then for 0::5 0,
a'·y*
3
= a'·y
3
+ Oa'·V
3 -
> a'·y
3
+ Oa'V
3
> a'·y + (e 3
-3
·-a'·y)
3
= e3· for a'V
3-'
< o·
a'·y*
3
= a'·y
3
+ Oa'·V > a'·y > c· for
3-3-3
a'·V
3
> O.
In addition, Y* yields an improved solution to the dual in (10.6) relative to the
original feasible vector y, i.e., g* = h'Y* = h'Y + Oh'V = 9 + Og; = 9 + Of; ::5 9
for 0 ~ 0, f; = g; < O. Clearly 0 =0 yields the largest possible decrease in g;.
If B,. depicts the current basis matrix for the restricted primal in (10.12)
when the optimality criterion is satisfied for all aj' j { 4, and C B has as its
r
components the objective function coefficients (being either 0 or -1) associated
with the basic variables, then V' = CB1.B;:-1. Thus the optimality evaluators may
be expressed as -*
Cj = ellj - C'Br B- 1
r -aj = -aj
-'Y- d WIt
an, . I1 Usj = aj'V- - Cj' .
It
follows that (10.13) can be rewritten as
-
0::5 0 = Usk
-::;j;'" = .
m~n {U S.3 J· f 4, - -*j < 0 }.
-::;j;"', C (10.13.1)
ek 3 Cj
Based upon the new solution vector V' = (fi', uo) we can obtain a new set of dual
surplus variables U;j = aj Y* - Cj = Usj + O( -cj), j=O, 1, ... , n, which enable us
to redefine the index set 4 and subsequently carry out an additional set of
iterations until the optimal solutions to the problems in (10.6) are obtained or it
is revealed that the original primal in (10.5) has no feasible solution. It is
important to note that the new index set contains at least one subscript j which
was not included in the old one. In fact, the revised 4 contains the subscript k
appearing in (10.13) (or (10.13.1» along with any other index corresponding to a
positive value of Xj emerging from the solution to (10.11) because of the
complementary slackness requirement. For any such j we have aj V = 0 and thus
aj y* = aj Y + Oaj V = aj Y = C j so that Usj = O. Hence at least one new vector 'iij
which was previously not a candidate for restricted basis entry will be released for
entry into the primal basis. Once the new index set , has been determined, we
solve the new restricted primal problem, the outcome of which allows us to
obtain another solution to the dual problem in (10.6) with a smaller objective
value than before. This new dual solution again updates " i.e., chooses one or
more vectors for restricted basis entry, and the process is repeated. Assuming the
absence of degeneracy, no basis of the primal in (10.6) is ever repeated and thus
the primal-dual method converges to an optimal basic feasible solution in a finite
number of steps. When f* = 0 we have an optimal basic feasible solution to the
modified primal and, for U o = 0, f = 9 = g. If uo:f. 0, the modified primal has
an unbounded solution. And if at some iteration we have f* < 0 and cj :$ 0 for
all j, then the primal in (10.6) has no feasible solution.
EXAMPLE 10.3. Using the primal-dual technique,
max f = 2xl + 10x2 s.t.

2xl + 4x2 < 20
4xl + x2 > 20
xl,x2~0.
Since C 1:. 0, we must solve the extend primal problem (10.8) or
max f* = -xs - xaO s.t.

2:1:1 + 4.tl2 + .tl3 = 20
4xl + x 2 x4 + Xs = 20 (10.14)
x O+x1 + x 2 + x3 + x4 + xaO bo
x O,xl' •.. 'xs,x ao ~ 0,
where x3 is a slack variable, x 4 is a surplus variable, and xS ' xaO are artificial
variables. As indicated earlier, a starting feasible solution to the modified dual
problem (10.6) is
U = 0, tto = m~x{cj} = 10 = uso'
J
Us! 10-2 8
us2 10-10 0
us = 1uo -C , and
us3 10-0 10
us4 10-0 10
9 = bouo + b'U = lObo·

Given that complementary slackness must be preserved at each iteration, it
is evident that with u s2 = 0, x 2 can become positive and thus, = {2}. From
(10.14), the initial simplex matrix appears as
0 2 4 1 0 0 0 0 20
0 4 1 0 -1 1 0 0 20
--.
1 1 1 1 1 0 1 0 bo
0 0 0 0 0 1 1 1 0
x2
0 2 [11 1 0 0 0 0 20
0 4 1 0 -1 1 0 0 20
(10.15)
1 1 1 1 1 0 1 0 bo
-1 -5 -2 -1 0 0 0 1 -20-bo
wherein only the artificial variables are basic, i.e., Xs = 20, xaO = boo Under
restricted basis entry, the (blocked) pivot in (10.15) must be 4 (since bo is
arbitrarily large). Then (10.15) is transformed to
Xl X2
I 1
0 2 1 4 0 0 0 0 5
0
rn 0 1
-4 -1 1 0 0 15
(10.16)
1 3 bo-5
1 2 0 4 1 0 1 0
1
-1 -4 0 -2 0 0 0 1 -10-bo
with x 2 = 5, x5 = 15, x ao = bo - 5, and f; = -10 - bo depicting the optimal basic

feasible solution to the restricted primal.
To find the values of the new dual surplus variables we can use
the relationship U;j = Usj + 8( -cj), where
With
U so 10 u sl u s4 10
-:=;j;"""
Co T = 10, -:=;j;"""
cl 48 -- 2, and -:=;j;"""
c4 1/2
20,
it follows that 8 = 2 and thus

*
u so u so + 8( -co) 10+2(-1) = 8
*
u sl u sl + 8( -ci) 8+2(-4) 0
*
u s2 u s2 + 8( -( 2) 0+2(0) 0
*
u s3 u s3 + 8( -(3) 10 + 2( -~) 9
*
u s4 u s4 + 8( -c:i) 10+2(0) 10.
Since u;l = u;2 = 0, it is evident, by virtue of the complementary slackness

requirement, that both Xl J = {I, 2}.
and x 2 can assume positive values and thus
Moreover, the new modified dual objective value is g* = 9 + Of;
lObo + 2( -10-bo ) = - 20 + 8b o'
Again employing restricted basis entry we can easily determine that the
pivot in (10.16) is ~ and thus the new optimal basic feasible solution to the
restricted primal appears as
Xo Xl X2
2 1 20
0 0 1 '7 '7 0 0 7
1 2 30
0 1 0 -14 -'7 0 0 7
(10.17)
11
[I] 0 0 14
8
"7 1 0 bO_5~
11 8
-1 0 0 -14 -'7 0 1 5~_bo
where Xl = 37°, X 2 = 27°, X ao = b° - 50

7' an d f*r = 750 - b0' (Note that the column
corresponding to the outgoing artificial basis vector associated with x5 has been
deleted from the current simplex matrix.) Proceeding as above, since

uso _ 8 us3 _ 126 and us4 _ 35
Co - , c; - 11 ' c4 - 4'
it follows that 0= 8 and thus the new set of dual surplus variables appears as
u;o 8+8(-1) = 0
0+8(0) o
U;2 0+8(0) 0
u;3 9+8(-11/14) = 19/7
u;4 10 + 8( -8/7) = 6/7
with g* = 9 + Of; = 2~0. Given tlla t * = u s1

uso * = u s2
* = 0, 3= {0,1,2}.
Upon pivoting in (10.17) we ultimately obtain
2 1 20
0 0 1 7 7 0 T
1 2 30
0 1 0 -14 -7 0 T
1 0 0 11
14
8
7 0 o-
b 570
0 0 0 0 0 1 0
where the column associated with the outgoing artificial basic variable xaO has
been deleted. Here Xl = 3~, x 2 = 2~, f; = 0, and g* = 9 + 0(0) = 2~0 = f.
10.7. Summary of the Primal-Dual Method

At this point we can review the salient computational features of the
primal-dual technique. A summary of the main steps which underlie a process
which strives for primal feasibility and optimality simultaneously is incorporated
in the:
10.3. PRIMAL-DUAL METHOD.
STEP O. Initialize.
Obtain an initial dual feasible (primal optimal) solution U.
This determines a set of dual surplus variables Us and thus
the elements of the index set l= {j I Usj = O}. The
associated restricted primal problem can then be constructed.
STEP 1. Solve the restricted primal problem. If f; = 0 (Phase I

terminates), the corresponding solution is also optimal for
the original primal problem.
STEP 2. f; < 0 and cj ~ 0 for all j, then the primal problem has no
feasible solution and thus the process is terminated. If
f; < 0 and -cj < 0 for at least one j ¢" then step 3 IS
warranted.
STEP 3. Determine a new set of dual surplus variables according to
where
- c~J < o} '
along with an updated objective value g* = 9 +Bf; (or

f; if an artificial constraint is not included.)
y* = 9 + ()
STEP 4. Use the u;/s to specify a new index set ,. Return to step 1.
So to find an optimal X given a starting feasible U, we solve, under weak
complementarity, the restricted primal problem, obtain information from its
dual, and then use this information to update the initial dual solution. The
entire cycle is repeated as required or terminated. (The following schematic
illustrates the main steps of this iterative process.)
DUAL
. - - - - -...,\ U, Us feasible; 1-------;
specify,
RESTRICTED DUAL RESTRICTED PRIMAL

V is feasible; , main tain com plemen tary
get V* and slackness; solve via the
update U, Us simplex method
An appealing aspect of the primal-dual routine is that it can be initiated

using any dual feasible solution (not necessarily basic). This then allows us to
immediately specify , and in turn the corresponding restricted primal problem.
By solving the latter we obtain an improved solution to the dual. From this we
can form a new restricted primal program, the solution of which leads to a
further improvement in the dual objective. After a finite number of such
improvements, an optimal basic feasible solution is generated for the original
primal problem. Thus the primal-dual process may be viewed as a way of
starting with an infeasible basic solution to the primal problem and using an
available dual feasible solution to decrease the infeasibility form of the extended
primal problem in a prescribed manner such that, when a primal basic feasible
solution is obtained, it will also be optimal.
10.8. A Robust Primal-Dual Algorithm

In the preceding sections of this chapter we have utilized either the dual
simplex or primal-dual algorithms exclusively to solve linear programming
problems. We shall now examine a solution method which exploits certain
features of both the primal simplex and dual simplex techniques (thus accounting
for its robustness), with the actual routine selected to carry out a particular
iteration dependent upon whether a primal simplex or dual simplex pivot
operation is feasible (or desirable). In this regard, the robust or mixed method
relaxes both primal and dual feasibility simultaneously (here not all - Cj ~ 0 and
not all xBi ~ 0 at some round in the simplex process) and works towards primal
optimality and feasibility.
Ignoring the possibility of degeneracy, we may now summarize the salient
features of the
lOA. ROBUST PRIMAL-DUAL METHOD.
STEP O. Initialize.
Transform all structural constraints to the "~" variety.
Thus no artificial variables are needed since: (a) any "~"
structural constraint can be converted to "~" form by

admitting primal infeasibility; and (b) if any equality
constraints
p
'L, aijxj = bi,i=l, ... ,r, (10.18)
j=l
are present, the latter may be replaced by the equivalent
system of inequalities
p
(a) ?::: aijx j ::; bi , i=l, ... , r,
J=l - ( 10.19)
(b) -f-( ~
L.J L.J a.IJ-x J-) -
i=l j=l
<- f-
L.J b-I·
i=l
The substitution of (10.19) for (10.18) in the linear program

again allows us to economize on the use of artificial variables at
the modest cost of including (10.19.b) as an additional
structural constraint.
STEP 1. Evaluate the effect of both primal simplex and dual simplex
pivots on the primal objective function, i.e.,
(a) if a primal simplex pivot is possible, denote its effect on f as
~ ~
illp=c/J,()=m/n Yi;'Yij>O;
{XB- }
(b) if a dual simplex pivot is possible, denote its effect on I as
ill D = x BiO, 0 = myx{ ~::i' Yij < 0}-

If a pivot is not available for the smallest -c j value (in the
case of primal simplex pivoting) or the smallest - x Bi term
(in the case of dual simplex pivoting), choose the next
smallest - Cj or - x Bi value for pivot selection.
STEP 2. Perform either a primal or dual simplex pivot according to
which has the greatest impact of I, i.e., the largest increase in
I is determined as max{ill p, ill D}. If neither a primal nor
dual simplex pivot is possible, even though the current solution
is primal infeasible and/or nonoptimal, terminate the process
- the original problem has no feasible solution. Otherwise
return to step 1.
10.9. The Complementary Pivot Method

One of the salient features of the primal-dual method is that at each round
of calculations the (weak) com plemen tary slackness conditions are satisfied. In a
similar vein, a computational routine which explicitly requires that a comple-
mentarity condition holds at each iteration, namely the complementary pivot
method (Lemke (1965)), has been used to generate a solution to the so-called
linear complementarity problem (hereafter abbreviated as LC P): given a vector
q € R n and an nth order matrix M, find a vector Z € Rn such that
(a) q + MZ 2: 0
(b) Z2:0 (10.20)
(c) Z'(q+MZ) = 0
Following convention, the LC P will be denoted by the pair (q, M).

If we let W = q + MZ, then the LC P( q, M) may be written IJ1 the alter-
native fashion: find vectors W, Z € Rn such that
(a) W -MZ = q
(b) W~O,Z~O (10.21)
(c) W'Z = o.
Here W'Z = E ~=1 WjZj = 0 or WjZj = 0, i=l, ... , n, is called the complementarity
condition, where the variables Wj and Zj, called a complementary pair, are said to
be complements of each other, i.e., at least one variable in each pair Wi' Zi must
equal zero.
A nonnegative solution (W, Z) to the linear system (10.21.a) is called a
feasible solution to the LCP. Furthermore, a feasible solution (W,Z) to the
LC P which also satisfies the complementarity condition W'Z = 0 is called a
complementary solution. Interestingly enough, problems (10.20), (10.21) contain
no objective function; and there is but a single structural constraint in the
variables vectors Wand Z, namely the complementarity restriction W'Z = O.
If q ~ 0, then there exists an obvious (trivial) complementary basic solution
given by W = q, Z = O. Hence the LC P has a nontrivial solution only when
q?;; O. In this circumstance, the initial basic solution given by W = q, Z = 0 is

infeasible even though the complementarity condition W'Z = 0 holds.
We now demonstrate that any symmetric primal-dual pair of problems
PRIMAL DUAL
max !(X) = C'X s.t. min g(U) = h'U s.t.
«10.5))
AX~h, X~ 0 A'U ~C, U ~ 0
where X€RP, U€R m, and A is of order (mxp), can always be written as a

LCP(q,M). To this end let Xs€Rm and Us€RP be vectors of primal slack and
dual surplus variables respectively. Then our aim is to find vectors X, U, X S '
and Us such that
Xs = -AX+h
Us = A'U-C
(10.22)
X, U, X s ' Us ~ 0
X'U s + U'X s = o.
Z and M [A'0-A]0 '

then (10.22) can be easily rewritten as
[~:] - [1,-~][~] [-~]

[~:]~o,[~]~o (10.22.1)
(X~, U~) [ ~] = 0
or (10.21). Hence we may solve a linear program by converting it into an

equivalent LCP, i.e., it follows from the weak complementary slackness theorem
that solving the symmetric primal-dual pair of problems above is equivalent to
finding vectors W, Z such that (10.22.1) holds.
Looking to the geometry of the LC P( q, M) let us consider the (n X 2n)
matrix
associated with the equation system In W - MZ = q. Since q is expressable as a

nonnegative linear combination of the columns of [ In' - M ],
(10.23)
and wizi = 0, i=l, ..., n, it is evident that if either wi or zi is positive, its

complement must be zero. Hence ei' - mi' i=l, ... , n, cannot both be used in
(10.23). Here ei' - mi' 1 =::; i =::; n, will be called the ith complementary pair of
vectors. In general, to form (10.23), for each i=l, ..., n, select either the column
ei of In or the ith column - mj of M and denote the column actually chosen by
b(i). Then the convex cone formed from the complementary set of vectors b(j),
i=l, ... , n, is called the complementary cone of M and denoted as
coni{b(l), ... ,b(n)} = {VIV = E;=l (Jjb(i), O=::;(JjfR}.
Since b(i) equals ej or -mi for each 1 =::; i =::; n, it is evident that there exists 2n
possible complementary cones of an (n x n) matrix M. Let e(M) denote the class
of complementary cones of M. Then solving the LCP(q, M) (10.22) (or (10.22.1»)
amounts to finding a complementary cone in e(M) that contains the vector q,
i. e., we desire to find a complementary set of vectors { b(l), ..., b( n) } such that:
(1) b(i) = ei or -mi' 1 =::; i =::; nj and
(2) qfConi{b(l), ... ,b(n)}Ce(M).
EXAMPLE lOA. For n-2 let
- 'M = [ -~ -: l
l
To find e(M), let us form
["to e,. -mI' -"'>] = [~ ~ ~2 ~

From this matrix we may obtain 22 = 4 complementary sets of vectors and thus
four complementary cones (Figure 10.1):
coni{el'~}, coni{ -ml , -m2 },

coni{el , -m2}, and coni{~, -md.
Figure 10.1
EXAMPLE 10.5. For n=2, let
M [3 -1], -1 2
q [ -1 ].
-2
Then LC P( q, M) appears as
WI 3z1 + z2 -1
w2 + zl - 2z2 -2
wI,w 2,zl,z2 ~ 0
w1z 1 = 0, w 2z 2 = O.
If we set WI = w 2 = 0, then the preceding system becomes
3z 1 + z2 = -1
zl - 2z2 = -2
zl,z2 ~ 0
with solution zl =~, z2 = f. Hence q lies within the complementary cone
coni{ -mI' - m 2} (Figure 10.2) and the complete solution to the LCP(q, M) is
w , -- 0' Zl - (1 1)
, - 5' 5 •
Figure 10.2
In order to solve the LC P( q, M) associated with a linear program, let us

examine the workings of the complementary pivot method. This algorithm starts
with an initial basic solution to (10.21) of the form W = q, Z = 0, where q 1::. o.
To avoid having at least one Wj < 0, a nonnegative artificial variable Zo is
introduced at a sufficiently positive level (take Zo = - mjn{qj, i=I, ... , n}) into
I
the left-hand side of each equation in W - MZ = q. In this regard, we now desire

to find vectors W, Z and a variable Zo such that
(a) W - MZ - zol = q
(b) W, Z~o, zo~O (10.24)
(c) W'Z = 0 or WjZj o
for at least n-l of the i values given Zo ~ O.

With each new right-hand side value qj + Zo nonnegative in (10.24), a basic
solution to the same amounts to W = q + zol ~ 0, Z = O. But while this basic
solution is nonnegative and satisfies all of the relationships in (10.24) (including
complementarity), it is nevertheless infeasible for the original LCP (10.21) since
Zo > O. A solution such as this will be termed an almost complementary basic
solution. More specifically, while a complementary basic solution to (10.21) is

one which contains exactly one basic variable from each complementary pair of
variables (wi,zi)' i=l, ... ,n, an almost complementary basic solution of (10.21) is
a basic solution for (10.24) in which Zo is a basic variable and there is exactly one
basic variable from each of only n-1 complementary pairs of variables.
In this light, the complementary pivot method has us move from one almost
complementary basic solution of (10.21) to another until we reach a
complementary basic solution to (10.21), in which case we have wizi = 0 for all
i=l, ... , n. At this point the algorithm is terminated. To
see this let us rewrite (10.24) as

wI W2 Wr Wn Zl Zr zn Zo
1 0 0 0 -mIl -mlr -1n ln -1 ql
0 1 0 0 -m21 -m2r -m2n -1 q2
o o 1 o -m r I ~
... -m rr ... -m rn L.::.l.J qr
o o o
(10.25)
To find the initial almost complementary basic solution the artificial

variable Zo is made basic by having it replace the current basic variable with the
most negative qi value. Assume qr = min {qi,i=l, ... ,n}. Hence Zo replaces wr
I
in the set of basic variables. To accomplish this we pivot on the (boxed) element
-1 in (10.25). This renders
1 o -1 mIn 0 qi
o 1 -1 m2n 0 q2
o o -1
o o -1
(10.26)
where
qr = - qr; qj = qj - qr' i=!=,', i=l" .. , n,
mrj mrj' j=1, ... ,n,
mjj = mrj - mjj' i,/r, j=l, ... , n.
As (10.26) reveals, an almost complementary basic solution to (10.21) is wi = qi'

i,/r, Zo = qr' and zi = 0, i=1, ... ,n.
The complementary pivot algorithm generates a sequence of almost
complementary basic solutions until Zo becomes nonbasic or zero. Moreover,
pivoting must be done in a fashion such that: (1) complementarity between the
variables wi' zi is maintained at each iteration; and (2) each successive basic
solution is nonnegative.
At the moment both w r ' zr are non basic (wrz r = 0 is satisfied). Since wr
turned nonbasic in (10.26), the appropriate variable to choose for entry into the
set of basic variables is its complement zr' In fact, this selection criterion for
determining the non basic variable to become basic is referred to as the
COMPLEMENTARY PIVOT RULE: choose as the incoming basic variable
the one complementary to the basic variable which just tUl'l1ed non basic.
Once the entering variable is selected, the outgoing variable can readily be
determined from the standard simplex exit criterion, i.e., wk is replaced by zk in
the set of basic variables if
Jk = min{ mar
8 = mkr J~ , fi/'ir > 0,
I
i=l, ... , n} . (10.27)
A pivot operation is then performed using mkr as the pivot element. Once wk
turns nonbasic, the complementary pivot rule next selects zk for entry into the
set of basic variables. Complementary pivoting continues until one of two
possible outcomes obtains, at which point the complementary pivot algorithm
terminates:
(1) the simplex exit criterion selects row r as the pivot row and Zo turns non-
basic. The resulting solution is a complementary basic solution to (10.21); or
(2) no mir > 0 in (10.27). In this instance the complementary pivot algorithm
fails to yield a complementary basic solution and ray termination occurs,
i.e., an extreme ray (Y.] +~d, ~<R, O:S d an extreme direction, of
almost complementary aslC solutions is determined.

A few additional comments pertaining to the complementary pivot
algorithm are in order. First, in a linear programming setting, M is a positive
semidefinite matrix. 3 In this instance the said algorithm always terminates with
a complementary basic solution if the original linear programming problem has a
solution. Next, if ray termination obtains, the linear program either has no
feasible solution or, if a primal (dual) feasible solution exists, the primal (dual)
objective function is unbounded.
EXAMPLE 10.6. For the primal-dual pair of problems
PRIMAL DUAL
max f = Xl + 3x 2 + 4x3 s.t. min g = 100uI + 40u2 s.t.
2xI + x2 + 4x3 ~ 100 2u I + u 2 > 1
Xl + 3x 2 + x3 ~ 40 ul + 3u2 > 3
xI,x2,x3~0 4u I + u 2 > 4
u I ,u2 ~ 0
we may form
3 A sufficient condition for an (n X n) matrix A to be positive semidefinite is that: (a)

all principal diagonal elements are nonnegative; and (b) all principal minors are nonnegative.
(A principal minor of order k is a determinant of a principal submatrix of order k, where the
latter is a (k X k) submatrix obtained from A by deleting any n-k variables and their
corresponding columns.)
X4 100 ul
x5 40 u2
W = [~:] = u3 , q =
u4
[-~] = -1
-3
, Z = [~] = xl , and
x2
u5 -4 x3
0 0 -2 -1 -4
0 0 -1 -3 -1
M = 2 1 0 0 0
1 3 0 0 0
4 1 0 0 0
Then (10.24) becomes

x4 + 2xI + x 2 + 4x 3 -
= 100 Zo
x5 + xl + 3x 2 + x3 - Zo = 40
u3 - 2u l - u2 - Zo = -1
u4 - u l - 3u 2 - Zo = -3
u 5 - 4u l - u2 - Zo = -4
x I , ••• ,x 5 ~ 0; u I , ••• ,u5 ~ 0
x 4u I = x 5u 2 = u3x I = u4 x 2 = u5x3 = 0
The simplex matrix associated with this problem is
x 4 x5 u3 u 4 u 5 u l u2 xl X2 x3 Zo
1 0 0 0 0 0 0 2 1 4 -1 100
0 1 0 0 0 0 0 1 3 1 -1 40
0 0 1 0 0 -2 -1 0 0 0 -1 -1 ---+
0 0 0 1 0 -1 -3 0 0 0 -1 -3
0 0 0 0 1 -4 -1 0 0 0 [;JJ -4
u5 x3
1 0 0 0 -1 4 1 2 1 [1] 0 104
0 1 0 0 -1 4 1 1 3 1 0 44
0 0 1 0 -1 2 0 0 0 0 0 3
0 0 0 1 -1 3 -2 0 0 0 0 1
0 0 0 0 -1 4 1 0 0 0 1 4
Here Zo has been made basic by pivoting on the (boxed) element -1. Thus the
resulting almost complementary basic solution is x 4 = 104, Xs =44, u3 = 3,
u4 = 1, Zo =4, and Us = u l =u 2 = xl = x 2 = x3 = O. Since Us turned non basic,
its complementary variable x3 will be chosen for entry into the set of basic
variables.
In this regard, pivoting on the (boxed) element 4 yields
x4 UI
1 1 1 1
4 0 0 0 -41 1 4 2 4 1 0 26
1 3 1 11
-4 1 0 0 -43 3 4 2 "4 0 0 18
0 0 1 0 -1 2 0 0 0 0 0 3
0 0 0 1 -1 lID -2 0 0 0 0 1
0 0 0 0 -1 4 1 0 0 0 1 4
Our second almost complementary solution is Xs = 18, u3 = 3, ul =4, x3 = 26,

Zo = 4, and x 4 = Us = u l =u2 = xl = x 2 = o. Given that x 4 turned non basic, its
complementary variable u l will now become basic. To this end we now pivot on
the (boxed) element 3. The resulting matrix is
U4 X2
1 11 77
4 0 0 -31 1
12 0 12
1
2
1
4 1 0 "3
1
-4 1 0 -1 1
4
11 1
0 "4 2 [¥J 0 0 17
4 7
0 0 1 -32 -31 0 3 0 0 0 0 3
1 1 1
0 0 0 3 -3" 1 -3"2 0 0 0 0 3
4 1 11 8
0 0 0 -3" 3 0 "3 0 0 0 1 3"
As this matrix reveals, the third almost complementary basic solution amounts
to Xs = 17, U _7 u- I and
3 - 3' 1 - 3'
U2 = xl = X 2 = O. Since u4 tumed non basic, its complementary variable x 2
becomes basic, i.e., pivoting on 141 above gives
X5 U2
3 1 8 2 1 5 1 796
IT -11 0 -33 33 0 6" 11 0 0 33
1 4 4 1 2 68
-11 IT 0 -11 IT 0 1 IT 1 0 0 IT
2 1 4 7
0 0 1 -3 -3 0 3 0 0 0 0 3
0 1 1 1 2 0 0 1
0 0 3 -3 -3 0 0 3
0 0 0 4
-3
1
3 0
00 0 0 0 1 8
3
Hence the current almost complementary basic solution amounts to U - 7

3-3'
U - 1 x - 68 x - 796 Z - 8
1 - 3' 2 - 11' 3 - 33' 0 - 3'
and x - x - U - U - U2
4 - 5 - 4 - 5- = Xl = o. With x5
now nonbasic, its complementary variable tt2 is selected for entry into the set of
basic variables. Pivoting on Vabove yields
Zo
3 1 2 1 5 0 1 1
23.630
IT -IT 0 -IT 22 0 0 IT -33
1 4 2 1 3
-11 11 0 0 0 0 0 IT 0 -IT 5.464
0 1 2 5 0 0 0 0 4 1.360
0 -IT
0 -IT -IT
1 3 2
0 0 0 IT -IT 1 0 0 0 0 IT 0.818
4 1 3 0.727
0 0 0 -IT IT 0 1 0 0 0 11
Since Zo has turned non basic, we now have a complementary basic and thus
optimal solution consisting of u3 = 1.36, u1 = 0.818, tt2 = 0.727, x2 = 5.464,
x3 = 23.63, and x 4 = x5 = u4 = u 5 = Xl = o.
11 POSTOPTIMALITY ANALYSIS I
Our point of departure in this chapter involves the assumption that we have
already obtained an optimal basic feasible solution to a linear programming
problem in standard form, i.e., to a problem of the form
maxf(X) = C'X s.t.
AX = b, X~O.
We then seek to employ the information contained within the optimal simplex
matrix to gain some insight into the solution of a variety of linear programming
problems which are essentially slight modifications of the original one. Specifical-
ly, two broad classes of variations in the original problem may be identified. First,
there may exist instances in which a sensitivity analysis is appropriate, e.g., there
may occur discrete changes in any of the components of the matrices C, b, or A,
in which case the values of cj,b i , or aij' i=l, ... ,m; j=l, ... ,n-m, respectively are
altered (increased or decreased) in order to determine the extent to which the
original problem may be modified without violating the feasibility or optimality
of the original solution. Second, we may address ourselves to the problem of
adding or deleting certain variables or structural constraints, in which case this
latter class of problems will simply be 3termed the analysis of structural changes.
11.1. Sensitivity Analysis

Turning again to our linear programming problem in standard form, let us
maxf(X) = CBXB+CnX n s.t.
BX B + RXn = b, with X B ' Xn ~ O.
289
Then as previously determined, with

X B = B-1b- B-1RXR ,
f(X) = CBB-1b+ (Cn-CBB-1R)X R,
it follows that if X R = 0, the resulting solution is basic (primal) feasible if
X B = B-1b ~ 0 and optimal (dual feasible) if C n- CBB-1R ~ 0'.
(1) CHANGING AN OBJECTIVE FUNCTION COEFFICIENT. If we now
change C' = (C B, Cn ) to C' + LlC' = (CB+LlCB,Cn+LlC R), then
f(X) = (CB+LlCB)'XB + (CR+LlCR)'X R
= (CB+LlCB)'(B-Ib-B-IRXR) + (CR+LlCR)'X R
= CBB-1b+ LlCBB-1b+ (Cn-CBB-IR+LlCn-LlCBB-IR)XR'
Since X B = B-1b is independent of C, this new basic solution remains primal
feasible while if (CR-CBB-1b) + (LlCR-LlCBB-1R) ~ 0', dual feasibility or pri-
mal optimality holds. Since our discussion from this point on will be framed in
terms of examining selected components within the optimal simplex matrix, it is evi-
dent from the last row of the same (footnote 1, Chapter 7) that primal optimal-
ity or dual feasibility is preserved if C~B-IR- C n+ (LlC~B-IR-LlCn) ~ 0'.
Let us now consider the various subcases which may obtain.
If LlC B =0 (so that an objective function coefficient associated with a
non-basic variable changes), it follows that optimality is preserved if
CBB-1R- Cn - LlCn ~ 0', i.e., if
- Cj - LlCRj ~ 0, j=I, ..., n - m. (11.1)
How may changes in CRj affect (ILl)? First, if CRj decreases in value so that
LlCRj < 0, then (11.1) is still satisfied. In fact, CRj may decrease without bound
and still not violate (11.1), i.e., this lower limit may be expressed as
limLlc Rj = - 00. (11.2.a)
Next, if CRj increases so that LlCRj > 0, then (11.1) still holds so long as CRj
increases by no more than -C j since (11.1) requires that -C j ~ Llc Rj'
j=l, ..., n-m. In this instance the upper limit to LlCRj is
(11.2.b)
Postoptimality Analysis I 291
Hence the only circumstance in which x Rj becomes basic is if its objective

function coefficient CRj increases by more than -Cj. We may represent the
upper limit depicted by (11.2.b) in terms of the new level of CRj , denoted as
CRj' since for Lic Rj = CRj - CRj'
(11.2.c)
Hence this equation provides the upper limit to which C Rj can increase without
violating primal optimality.
If LicR = 0 (so that an objective function coefficient associated with a basic
variable changes), it follows that the optimality criterion is not violated if
CBB-IR-CR+LiCBB-IR~ 0', i.e., if
m
- Cj + LyijLic Bi ~ 0, j=1, ... , n-m. (11.3)
i=1
Let us assume that cBk changes. Hence (11.3) becomes
- Cj + ykjLic Bk ~ 0, j=1, ..., n-m. (11.3.1)
We now consider the circumstances under which changes in cBk preserve the
sense of this inequality. First, if Ykj = 0, then (11.3.1) holds regardless of the
direction or magnitude of the change in cBk. Second, if Ykj > 0, then (11.3.1) is
satisfied if LicBk ~ 0, i.e., LicBk may increase without bound so that its upper
limit becomes
IimLicBk = + 00, Ykj > 0. (11.4.a)
Hence (11.3.1) is affected only when cBk decreases of LicBk < 0. Since (11.3.1)
implies that ClYkj ~ Lic Bk' j=I, ..., n-m, it is evident that the lower limit to
which Lic Bk can decrease without violating optimality is
(11.4.b)
Hence x Rj remains non-basic so long as its objective function coefficient does not
decrease by more than clYkj. Now, it may be the case that Ykj > ° for two or
more non-basic variables. To insure that (11.3.1) is satisfied for all such j, the
lower limit to which Llc Bk can decrease without violating the optimality of the
current solution is
limLlcBk = mq.x{c/Ykj'Ykj > O}. (l1.4.c)
J
Finally, if Ykj < 0, (11.3.1) holds if LlcBk ~ O. Hence LlcBk may decrease
without bound so that its lower limit is
limLleBk = - 00, Ykj < O. (11.5.a)
Thus the optimality criterion is affected by changes in eBk only when Llc Bk > O.
In this instance (11.3.1) may be rewritten as c/Ykj ~ LlcBk,j=I, •.. ,n-m,so
that the upper limit to any change in C Bk is
(11.5.b)
Hence x Rj maintains its non-basic status provided that the increase in e Bk does
not exceed c/Ykj. If Ykj < 0 for two or more non-basic variables, then (11.3.1)
holds for all relevant j if the upper limit to which Lle Bk can increase is specified
as
(11.5.c)
Actually, the optimality criterion (11.3.1) will be maintained if both the

lower and upper limits (l1.4.c), (11.5.c) respectively are subsumed under the
inequality
limLlcBk
(11.6)
Hence (11.6) provides the bounds within which LlcBk may decrease or increase
without affecting the optimality of the current solution. As an alternative to the
approach pursued in defining (11.4.c) and (11.5.c), we may express these limits in
terms of the new level of eBk' call it CBk' by noting that for Lle Bk = CBk - C Bk'
(11.4.c) becomes
(11.4.c.l)
while (11.5.c) may be rewritten as

(11.5.c.1 )
Hence (11.4.c.1) (11.5.c.1)} specifies the lower (upper) limit to which cBk can
decrease (increase) without violating the optimality criterion (11.3.1). In view of
this modification, (11.6) appears as
(11.6.1)
One final point is in order. If cBk changes in a fashion such that (11.6) and
(11.6.1) are satisfied, then the current basic feasible solution remains optimal but
the value of the objective function changes to f = C~B-Ib + L1cBkzBk.
EXAMPLE 11.1. We found in chapter 7 that the optimal simplex matrix
associated with the problem
maz f = 2z1 + 3z 2 + 4x3 s.t.
2z1 + z2 + 4z3 ~ 100
zl + 3x 2 + x3 ~ 40
X I ,Z2,Z3 ~ 0
appears as
23.63]
[
5
0 1 3 I 0
IT IT -IT
2 1 0 I 4 0 5.46 .
IT -IT IT
4 0 0 9 8
1 110.91
IT IT IT
A glance at the objective function row of this matrix reveals that -c I = t1'
-c2 = 191' and -c3 = 181. Hence (11.2.b) indicates that xRl = xI' xR2 = z4' and
X R3 = x5 should each maintain their non-basic status provided that their
objective function coefficients do not increase by more than L1cRl = 141'
L1cR2 = 191' and L1cR3 = 181 units respectively. Additionally, (11.2.c) may be
employed to depict the upper limits to which cRI = c I ' cR2 = c 4' and cR3 = c 5
can individually increase without violating primal optimality (Table 11.1).
TABLE 11.1.
-
26
ZR1 = Zl CR1 = C1 = 2 -00 CR1 -C 1= 11
-00
- 9
ZR2 =Z4 CR2=C4 =0 CR2- C2 = 11
- 8
Zm =Z5 Cm=C 5 =0 -00 Cm - C3 = IT
Suppose that CR1 = c1 is increased to 4. Since limeR1 = ~~ = CR1 - c1

4 - c1' it follows that - c1 = - ~~, which in turn implies that the current
solution is no longer optimal. In terms of the preceding simplex matrix, 1~ is
replaced by - ~~ in the last row of the same. An additional round of calculations
thus yields a new optimal (though not unique) basic feasible solution consisting
of zl = 30, z3 = 10, and f = 160 as evidenced by the following simplex matrix
[
o
1
1
o
-1
2
o
o 10
30 1 .
o o 4 1 160
Next, from (11.6) we see that the limits within which LlcBl = Llc3, LlcB2 = Llc 2
can vary without affecting the optimality of the current solution are
• { C1 -4/11 4 C2 -9/11 }
hmLlcBl = maz -y
- 11
= -5/ 12 = -3 /11- = -3
11 = --5' -y
4 -.- . -8/11 }{C3
= -5 ~ LlcBl ~ lzmLlcBl = mm Yl3 = -1/11 = 8 = 8;
• { C1 -4/11 C3 -8/11 }
ltmLlcB2 = maz Y21 = 2/11 = -2, Y23 = 4/11 =-2
= -2 ~ Llcm ~ limLlcB2 = min{ ::2 = =~j:: = 9} = 9.

Moreover, we may utilize (11.6.1) to specify the limits within which C Bl = c3 '
cm = c2 can individually decrease or increase while preserving optimality (Table
11.2).
TABLE 11.2.
C3
cBI +-y
13
= 12
C2
CB2+-y = 12
22
Suppose that CBl = c3 increases to 14. Since changing CBl affects the
objective function coefficients on the non-basic variables as well as the optimal
value of I itself, it follows that I increases by LlI = Llc BI X BI = 10
= 236.3 to I =1+ LlI = 110.91 + 236.3 = 347.21 while,
~
(23.63) by virtue of
(11.3.1), -c1 = I~' -c2= 191' and -c3 = 1~ change respectively to
= -ci +Y11 Llc Bl I~ + 151 10

54
~
-C I
IT'
+ Yl2 Llc BI = I; + I; 10 39
~
-C 2 - C2 IT'
~
-C3 - C3 + YI3 Llc BI = 181 - A10 -IT·

2
A substitution of these values into the last row of the optimal simplex matrix
yields, after an additional iteration,
[';.!..
11
54
IT
0
1
0
1
0
0
3
11
I
-IT
39
IT
I
-11
4
11
-11
2
0
0
23.63
5.46
1 347.21
J -+
[ 25]
I I I 0
2 4 1 4 0
I 11 1 1 0 15 ,
2 '4 0 -4
I 7
1 350
5 2 0 2 0
and thus a new optimal basic feasible solution consisting of x3 = 25, x5 = 15, and
f = 350.
(2) CHANGING A COMPONENT OF THE REQUIREMENTS VECTOR. If
we next change b to b + Llb, the new vector of basic variables becomes
XB = B- 1 (b+Llb)
(11. 7)
= B- l b+ B- I Llb.
If this new basic solution is primal feasible (XB ~ 0), then, since X B is
independent of C, primal optimality or dual feasibility is also preserved. For
Llb' = (Llb l , ...,Llbm ), (11.7) becomes
m
E f3. ·Llb ) .
j=1 I)
or (11.8)
m
xBi xBi+.E f3ijLlbj' i=l, ... ,m, (11.8.1)
)=1
where Pi = (f3il, .•. ,f3im ), i=l, ... ,m, denotes the ith row of B- 1 • Let us assume
that bk changes. In order for this change to insure the feasibility
of the resulting basic solution, we require, by virtue of (11.8.1), that
(11.8.2)
Under what conditions do changes in bk preserve the sense of this inequality?

First, if f3ik = 0, then (11.8.2) holds for any Llb k. Second, for f3 i k > 0, (11.8.2) is
satisfied if Llb k ~ 0, i.e., Llb k may increase without bound so that its upper limit
is
(11.9.a)
Thus the feasibility of the new solution is affected only when Llb k < O. Since
(11.8.2) implies that -xBdf3 ik ~ Llb k , i=l, ... ,m, it follows that the lower limit
to which Llb k can decrease without violating primal feasibility is
(11.9.b)
Hence x Bi remains in the set of basic variables so long as bk does not decrease by
more than - x BJ f3 ik' If f3 ik > 0 for two or more basic variables x Bi' then to
insure that (11.8.2) is satisfied for all such i, the lower limit to which Llb k can
decrease without violating the feasibility of the new primal solution is
(11.9.c)
Finally, if f3 ik < 0, (11.8.2) holds if Llb k ~ O. Hence Llb k can decrease without
bound so that its lower limit is
(11.10.a)
Thus the feasibility of the new basic solution is affected by changes in bk only
when Llb k > O. In this instance (11.8.2) may be rewritten as -xB;/f3ik 2': Llb k ,
i=l, ..., m, so that the upper limit to Llb k is
limLlb k = - xBJf3 ik , f3 ik < O. (11.10.b)
Hence x Bi maintains its basic status provided that bk does not increase by more
than - x Bd f3 ik' And if f3 ik < 0 for more than one basic variable x Bi' then
(11.8.2) holds for all relevant i if the upper limit to which Llb k can increase is
specified as
limLlb k = mint -x Bd f3 ik' f3 ik < O}.
I
(11.10.c)
If we choose to combine the lower and upper limits (11.9.c),

(11.10.c) respectively into a single expression, then (11.8.2) will be maintained if
(11.11)
mint -x BJ f3 ik'
I
f3 ik < O}.
Hence (11.11) provides the bounds within which Llb k may decrease or increase
without affecting the feasibility of XB . As an alternative to (11.11), we may
express the limits contained therein in terms of the new level of bk , call it bk , by
noting that since Llb k = bk - bk'
(11.11.1)
What about the effect of a change in a component of the requirements

vector on the optimal value of the objective function? By virtue of the preceding
discussion, if (11.11) or (11.11.1) is satisfied, those variables which are currently
basic remain so. However, a change in any of the requirements bi ,i=l, ... ,m,
alters the optimal values of these variables, which in turn affect the optimal
value of I. To determine the magnitude of this change in I, let us form
f(XB) = CnxB = Cn(B- 1b+B- 1Llb)

n
= I(X B ) + C B- 1Llb
= f(X B ) + 1=1
.ECBi(ff3ijLlbj).
3=1
If again bk alone changes, then
and thus
(11.12)
By virtue of our earlier specification of the dual variables and (11.12),
(11.12.1)
i.e., as noted in chapter 5, Llf / Llb k represents a measure of the amount by which
the optimal value of the primal objective function changes per unit change in the
kth requirement bk • Hence the total change in f when the kth requirement
changes by Llb k units is

(11.12.2)
EXAMPLE 11.2. Let us again look to the optimal simplex matrix provided
at the outset of the previous example problem. In the light of (7.10), if we
examine the columns of this final primal simplex matrix which were, in the
original simplex matrix, the columns of the identity matrix, we find that the ith
column of B- 1 appears under the column which was initially the ith column of
12 , Hence the inverse of the optimal basis matrix is
B-
1
=[ -X -X l
By utilizing the components of this inverse we may determine the limits within
which .1b1, .1b2 can vary without affecting the feasibility of the new vector of
basic variables according to
max{ -xBl
fJ l l
= -23.63
3/11
= - 86.64} = -86.64 ~ .1b1
<
-
lim.1b 1 = min{-XfJ 21B2 = -1/11
-5.46 = 60.06} 60.06;
max{
-x B2
fJ 22 = -5.46
4/11 = - 15.02 = -15.02 ~ .1b 2
}
~ lim.1b 2 = min{-fJXBl
12
= -23/.63
-1 11
= 259.93} = 259.93.
Addi tionally, we may utilize (11.11.1) to specify the limi ts wi thin which b1 , b2
can individually decrease or increase without violating the feasibility of XB
(Table 11.3).
TABLE 11.3.
xBl = x3 b1 = 100 b1 -rxBl = 13.36

11
b1- rxB2 = 160.06
21
b2 -r = 24.98 rxBl =299.93

xB2
xB2 =x2 b2 = 40
22
b2-
12
If we increase the first requirement b1 by 10 units so that .1b' = (10,0)

then, from (11.7),
= [23.63]+[
5.46
_1:1 -~1 ][10]0 =[26.36].
11 11 4.55
(Note that the components of this vector may alternatively be obtained from
(11.8.2).) Hence this new basic solution is primal feasible as well as primal
optimal. Moreover, from (11.12.2), the optimal value of 1 increases by
LlI = 1; 10 = 8.18 units.
Let us now increase the second requirement b2 by 300 units. In this
instance ..1b' = (0,300). Again employing (11.7) we obtain
[
23.63] [ 3
11 -A] r 1 r 1
0 -3. 64
5.46 + -A Ii l300J = ~14.55J
(Here too (11.8.2) may be utilized to determine the components of XB .) In
addition, from (11.12.2), 1 increases to 1+..11 = 110.91+181 300 = 329. Upon
substituting these values into the last column of the optimal simplex matrix we
obtain, using the dual simplex routine,
[~ 11
0
1
1
0
3
IT
1
-IT
1
-IT
4
IT
o
o
-3.64
1
114.55 -+
[ -5 o -11 -3 1
2 1 4 1 0
o
o 40
100 ,
1
4
11 0 0 9
IT
8
IT 1 329 4 o 8 3 0 1 300
Hence the new optimal basic feasible solution which emerges is x 2 = 100, Xs = 40,
and 1= 300.
(3) CHANGING A COMPONENT OF THE COEFFICIENT MATRIX. We
now look to the effect on the optimal solution of a change in a component of the
coefficient matrix A associated with the structural constraint system AX = b.
Since A was previously partitioned as A = [B, R], it may be the case that: some
component r ii' i=l, ..., m; j=l, ... , n-m, of a non-basic vector ri changes; or a
component bjk , i, k=l, ..., m, of a basis vector b j changes.
Let us begin by assuming that the eth component of the nonbasic vector
r~=(rlk, ... ,rek, ... ,rmk) changes to rek=rek+..1rek so that rk is replaced by
rk = rk + ..1r tket in R = [rl , ..., rk' ... ' r n - m ]. Hence the matrix of non-basic
vectors may now be expressed as R= [rl, ...,rk, ...,rn_nJ. With
XB = B-1b- B-1iixjz independent of ..1rtk at a basic feasible solution (since
Xjz = 0), it follows that this non-basic solution is primal feasible, while if
- C'jz + C~B-IR = - C k+ U'R ~ 0', dual feasibility or primal optimality is also
preserved. Since
and - Cj ~ 0, j=I, ... , k, ..., n-m, it is evident that primal optimality is

maintained if
(11.13)
Under what circumstances will changes in r ek preserve the sense of this

inequality? First, if ue = 0, then (11.13) holds regardless of the direction or
magnitude of the change in rek. Second, for Ut > 0, (11.13) is satisfied if
..1rek ~ 0, i.e., ..1rtk may increase without bound so that its upper limit becomes
lim..1rek = + 00, ue > 0. (11.14.a)
Hence (11.13) is affected only when rtk decreases or ..1rek < 0. Since (11.13)
implies that ck/Ut::; ..1rek' it IS evident that the lower limit to
which ..1r ek can decrease without violating optimality is
(11.14.b)
Thus rk enters the basis (or xjzk enters the set of basic variables) only if the
decrease in rek is greater than ck/ue. If we express this lower limit in terms of
the new level of rtk' rek' then for ..1rek = rek - rek' (11.14.b) becomes
(11.14.b.l)
Next, let us assume that the lth component of the basis vector
b~ = (blk, ...,btk, ...,bmk) changes to bek = bek + ..1b ek so that b k is replaced in
the basis matrix B = [bl , ..., b k , . •• , bnJ by
Now, for the current optimal feasible solution the inverse of the basis matrix B is
known. Hence our problem becomes one of replacing the kth column of B by bk
so as to obtain the inverse of the new basis matrix B= [bl , ..., bk' ..•, bnJ. Since
bk is expressible as a linear combination of the vectors within the current basis
matrix, i.e.,
m
bk L: Yikbi BYk,
i=I
it follows that
PI PI bk+PleLlbek Pl eLl ek
1+ f3 keLlb ek
-1~
Y k = B bk = Pk bk = Pk b k+f3 keLlb ek
Pm P b k+f3 meLlb ek f3mkLlbek

where Pie' i=l, ... , m, is the ern component of Pi' the ith ro of B- 1 •
Then, by virtue of (A.I) of appendix A,
-P l eLlb ek
I o
1+ PkeLlb ek
o I o
o 1
Hence from (A.2), B- 1 = I m B- l and thus

f3 me Llbek
xBm -1+f3 Llbe xBk
ke k
For XB primal feasible, we require that
If 1 + f3keLlbek > 0, then, upon solving the second inequality for Llb ek ,
(11.15)
Since these inequalities must hold for all if:.k when 1 + f3 keLlb ek > 0, the lower
and upper limits to which Llb ek can decrease or increase without violating primal
feasibility may be expressed, from (l1.15.a,b) respectively, as
(11.16)
If we express these limits in terms of the new level of bek' bek' then for Llb ek
bek - bek , (11.16) becomes
(11.16.1)
What is the effect of a change in bek upon the primal optimality (dual
feasibility) requirement ChB-IR-C~~ O'? Now the lh compo nent of
C'.TmB-1R-
Jr C'R = C'B .1. . m [Y1'· .. , Yn-m ] - c'R => 0'
is expressible as
(11.17)
If we add
to the left-hand side of (11.17) we obtain
(11.17.1)
If 1 + f3 keLlb ek > 0, then, upon solving for Llb ek ,
(11.18)
Since these inequalities must hold for all j=l, ... , n-m when 1 + /3 keLlb ek > 0, the
lower and upper limits to which Llb ek can decrease or increase without violating
primal optimality may be expressed, from (I1.1S.a, b) respectively, as
(11.19)
Alternatively, we may express these limits in terms of the new level of bek ,
namely bek , since, if Llb ek = bek -b ek , (11.19) becomes
limb ek = mC:ZX{b ek + ,-:j'/3

J YkJue C J ke
IYkjUe+Cj/3ke<O}~Llbek
(11.19.1)
We next determine the effect of a change in the element bek upon the
optimal value of the objective function. To this end let
(11.20)
Adding
(11.20.1)
EXAMPLE 11.3. It was indicated at the beginning of example 11.1 that

the optimal simplex matrix corresponding to the problem
max f = 2x1 + 3x 2 + 4x3 s.t.
2x1 + x2 + 4x3 < 100
xl + 3x 2 + X3 < 40
is
5 3 1
IT 0 1 IT -IT 0 23.63
2 1 4
n 1 0 -IT IT 0 5.46
4 9 8
11 0 0 11 11
1 110.91
Moreover, since x2' x3 are basic variables,
-A]
4
IT
'
From the objective function row of the optimal simplex matrix we find, for
instance, that CI = - ;i. Hence (11.14. b) indicates that x R1 =x 1 should
maintain its non-basic status provided that rw r 21 do not decrease by more than
Llrn = -~, Llr21 = -! respectively. In addition, (11.14.b.1) may be utilized to
depict the lower limits to which rw r 21 can individually decrease without
violating primal optimality (Table 11.4).
Table 11.4.
c1
rn =2 r 11 +u 1=914 +00
c1 1
r 21 = 1 r 21 +u2 =2 +00
Suppose that rn is decreased to l. Then lim rn = 194 = rn + c1 /u 1

= ~ + Jjc1 or - c1 = -~~, which implies that the current solution is no longer
optimal. In terms of the preceding simplex matrix, Ii is replaced by - ~~ in the
last row of the same. Furthermore, an additional adjustment in the optimal simplex
matrix is in order. Since XB=B-1b-B-1RXR=B-1b-[Y1'''''Yn_m]XR'
where Y j = B-1 rj' j=l, ... ,n-m, it follows that if rk is replaced in R by rk
rk + L1r ek ee, then Y k is replaced in B- 1 R by
Since the ith component of Yk is Yik = Yik + L1r eki3 ie' Y11 = 151 and Y21 = 121
are replaced by Yll = Yll + .1r ll i3 11 = l2 and Y21 = Y21 + .1r ll i321 = 272
respectively in the optimal simplex matrix. Hence an additional iteration yields
the new optimal simplex matrix
1 3 1
22 0 1 IT -IT 0 23.63
7 1 4
22
1 0 -IT IT 0 5.46 ---+
19 9 8
-22 0 0 IT IT 1 110.91
1 22 11
0 -"1 1 77 -77 0 22.85
22 2 8
1 7" 0 -"1 "1 0 17.16
19 44 132
0 7" 0 7" n 1 125.73
and thus a new optimal basic feasible solution consisting of Xl = 17.16,

x3 = 22.85, and f = 125.73.
Next, from (11.16) we see that the limits within which the components of
the basis vectors b~ = (611'6 21 ), ~ = (6 12,6 22 ) can vary without violating
primal feasibility are:
5.46 }
(1~J23.63) -( -A)(5.46)
= 0.60;
23.63 }
(131)(5.46) -( -A
)(23.63)
6.51;
Moreover, we may employ (11.16.1) to specify the limits to which 611' 621 , 612 and
622 can individually decrease or increase without affecting the feasibility of the
current solution (Table 11.5).
TABLE 11.5.
611 =4 611 +{J

zB2
{J
21 Zm- 11 z B2
= 2.50 +00
621 =1 -00 621 +{J Z {J Z

zB2
22 m- 12 B2
= 1.60
612 =1 -00 612 +{J

n ZB2-{J 21 z m = 7.51
zB1
Zm
622 =3 622 +{J {J
12z B2- 22 z m
=0.40 +00
We now turn to the specification of the limits within which the components
of b I • ~ can vary without affecting primal optimality. From (11.19):
1·•
2',
max {Y22 ,-::,P22 - (-trX;)~\*H)}

U
- 2.25 ~ L1b 22 ~ limL1b 22 = + 00.

Additionally, from (11.19.1), we may determine the lower and upper limits to
which b11 , b21 , b12 , and b22 can individually decrease or increase while still
preserving primal optimality (Table 11.6).
TABLE 11.6.
-00
-00
+00
Since changes in the components of b1 , b2 may affect both primal feasibility and
optimality, we may combine Tables 11.5, 6 to obtain the lower and upper limits
to which b11,b21,bI2' and b 22 can individually decrease or increase without
affecting either primal feasibility or optimality (Table 11.7).
TABLE 11.7.
2.50 5.33
-00 1.60
-00 2
0.75 +00
Suppose now that b11 = 4 is decreased to 2. As evidenced from Tables 11.5,

7, such a change violates primal feasibility. Before initiating this change, let us
drive hI out of the basis. This may be accomplished by using the standard
primal simplex exit criterion to find a pivot that will remove x3 from the set of
basic variables so as to yield a primal feasible though non-optimal basic solution.
If x 4 is chosen to become basic, we ultimately have a problem in which only the
component r12 = 4 of the non-basic vector r~ = (4,1) need change. But this is
similar to the one just solved. In this regard, pivoting on the element 131 yields
5 3 1
IT 0 1 IT -IT 0 23.63
2 1 4 5.46
IT 1 0 -IT IT 0 -+
4 9 8
IT 0 0 11 11 1 110.91
5 11 1
3 0 "3 1 -3 0 86.66
1 1 1
3 1 3 0 3 0 13.33
-1 0 -3 0 1 1 40
Moreover, with
= [: ; : J
and
[n+(-2)[ ~] [: J
it follows that y 12 -3"
_11 IS replaced by ~ 5
Y12= 3 III the previous simplex
rnat rIX
. WI'th Y22
~
Y22 =
1 Then
3' =
5 5 1
3 0 3 1 -3 0 86.66
1 1 1
3 1 3 0 3 0 13.33 -+
-1 0 -3 0 1 1 40
o -5 o 1 -2 0 20
1 3 1 o 1 0 40
2 9 o o 4 1 160
Hence the new optimal basic feasible solution consists of x3 = 40, x4 = 20, and
f = 160.
Finally, let b22 =3 decrease to!. In this instance Tables 11.6, 7 indicate
that this change violates primal optimality. Proceeding as in the previous case,
let us remove b 2 from the basis (or x 2 from the set of basic variables) by pivoting
on the element 121 (Xl is chosen to become basic) to obtain
5 3 1
11 0 1 IT -IT 0 23.63
2 1 4
IT 1 0 -IT IT 0 5.46 --+
4 9 8
IT 0 0 IT 11
1 110.91
5 1 -1
0 -'2 1 '2 0 10
11 1
1 2' 0 -'2 2 0 30
0 -2 0 1 0 1 100
Here the component 7'21 = 3 of the non-basic vector r~ = (1,3) is to be changed.

Since
R ~ [::: ::: ::: ] ~ [~ ~ : J and
VI ~ YI+4r'I[:::] ~ [~n+(-2.5)[-~] [:}

it follows that Y11 =-~, Y21 =121 are replaced by Yn =0, Y21 =! respectively in
the previous simplex matrix. Hence
0 0 1 ! -1 0 10 0 0 2 1 -2 0 20
1 1 2 2 1 2 0 2 0 80
1 '2 0 -'2 0 30 --+
0 -2 0 1 0 1 100 4 0 2 0 6 1 240
so that the new optimal basic feasible solution is :1:2 = 80, :1:4 = 20, and
f = 240.
11.2. Structural Changes

(1) ADDITION OF A NEW VARIABLE. We now look to the effect on the
optimal solution of a given linear programming problem of the introduction of a
new variable x nH ' with c nH and a nH representing its associated objective
function coefficient and vector of structural constraint coefficients respectively.
In particular, we are now cofronted with solving a problem of the form
max/eX, x nH ) = C'X+c n +1x nH s.t.
[A, a nH ] ~:J = b, X~ 0, xnH 2: o.

If X B = B- 1b represents an optimal basic feasible solution to the problem
max leX) = c'x s.t.
AX=b, X~O,
then we may also view it as a basic feasible solution to the former problem with
x nH deemed non-basic or zero. Moreover, this solution will be optimal if
cnH - m B
= cR,nH-m -C B- 1r n H_m = cnH - U'anH ::; o.
Obviously then, if cnH - m > 0, the current solution is no longer optimal so that
a n +1 must be brought into the basis, in which case we must insert into the
optimal simplex matrix an additional column of the form
where -cnH - m < O.

(2) ADDITION OF A NEW STRUCTURAL CONSTRAINT. Oftentimes one
is faced with the problem of introducing an additional structural constraint into a
linear programming problem for which an optimal basic feasible solution has
already been obtained. Let us write this new (m+ 1Bt) structural constraint as
n
LamH,jxj::;' 2:, = bmH or aX::;, 2:, = bmH ,
j=1
where a=(a mH ,l, ... ,a mH ,n) is a (1xn) vector of coefficients on the variables
Xi' j=1, ... ,n.
First, for aX:::; bm +1, if xn+l represents a non-negative slack variable, then
we seek a solution to a problem of the form
max!(X,x n+1 ) = C'X+xn+l s.t.
AX+xn+lO =b
aX + x n +1 = bm +1
X~ 0, x n +1 ~ O.
The simplex matrix associated with this problem may be written as
[ ~ :: ':+I: b:+],
-Cn-Cn 0 1 0
where a= (aB' an)· Here aB is a (1 xm) vector containing those components
am+1,j which correspond to the columns of n and an is a (1 x n-m) vector
whose components a m +1, j correspond to the columns of R. If this simplex
matrix is premultiplied by the inverse of the m+ 1th order basis matrix
the resulting product
[
1m n-1R 0 0
n-'b ]
0' an-aBB-IR 1 0 bm+1-aBB-Ib
0' CBB-IR-Cn 0 1 C'BB-Ib
has, with Xn = 0, the partitioned rows

X B = B-1b,
x n +1 = bm +1 - aBB-1b
! = CBB-1b = CBX B
respectively, That is
Now, if x B, m+1 ~ 0, this basic solution to the enlarged problem is feasible, 2. e.,
the optimal basic feasible solution to the original problem also satisfies the new
structural constraint or aX B ~ bm+1 • Moreover, if the original basic solution was
optimal, then an examination of the last row of the preceding simplex matrix
reveals that the solution to the extended problem is also optimal (since the
optimality criterion for each is the same) with equality holding between the
optimal values of the objective functions for the original and enlarged problems.
If x B, m+l < °
(in which case aX B > bm +1) so that primal feasibility is violated,
then, since the optimality criterion is satisfied, the dual simplex method may be
employed to obtain a basic feasible solution to the modified problem. In this
regard, it is obvious that the optimal objective value for the original problem
provides us with an upper bound on the same for the enlarged problem.
Next if aX ~ bm +l' then xn+1 represents a nonnegative surplus variable. In
this circumstance let us add aX - xn+1 = bm +1 or aX + xn+1 = bm +1 to the
original problem, where a = -a and bm +1 = - bm +1 • The treatment then
proceeds exactly as above with a and bm +1 replacing a and bm +1 respectively.
Finally, if aX = bm +1' then x n+l will represent a nonnegative artificial
variable. If in the solution to the enlarged problem we find that x B, m+1 =
xn+1 = bm+1 - aBXB = 0, then it is evident that the optimal basic feasible
solution for the original problem exactly satisfies the new constraint, in which
case the original solution is also the optimal solution for the enlarged problem.
The implication of this degenerate solution to the enlarged problem is that the
added structural constraint is redundant. If x B, m+1 < 0, then the artificial
variable xn+1 may be treated as if it were a slack variable, in which case the dual
simplex method applies. And if x B, m+1 ~ 0, then the artificial variable x n +1
may be handled as if it represented a surplus variable. Again the dual simplex

method may be applied.
(3) DELETION OF A VARIABLE. The deletion of a positive variable is
tantamount to driving it to zero so that its contribution to optimality is
withdrawn. This being the case, it is obvious that the deletion of a non-basic
variable is a trivial process. To delete a positive basic variable, we may simply
pivot its associated basis vector out of the basis, where the pivotal column is
determined by the dual simplex entry criterion. This procedure will result in a
basic solution which is primal optimal but not necessarily primal feasible, at
which point the column corresponding to the outgoing basis vector may be
deleted from the simplex matrix. If warranted, reoptimization of the primal
objection function may then be carried out using the dual simplex routine.
(4) DELETION OF A STRUCTURAL CONSTRAINT. If a structural con-
straint is to be deleted from the optimal simplex matrix, consideration must first
be given to whether or not that constraint is binding at the optimal basic feasible
solution. In this regard, if the original structural constraint was of the " :S" or
" ?" variety and its associated slack or surplus variable appears in the optimal
set of basic variables at a positive level, thus indicating that the constraint is not
binding at the optimal extreme point, then clearly the current solution will also
be an optimal basic feasible solution to the modified problem.
If, on the one hand, the structural constraint to be deleted is of the "~"
type and its corresponding slack variable is zero at the optimal basic feasible
solution, thus indicating that the constraint is binding there, then the constraint
may be deleted from the optimal simplex matrix by subtracting a non-negative
surplus variable from its left-hand side. To justify this operation let us assume
that the rth structural constraint is binding at an optimal basic feasible solution
so that a rl Xl + ... + arnx n = br' Given that this constraint is to be deleted from
the optimal simplex matrix, the introduction of the surplus variable x n +l allows
for the possibility that a,'l Xl + ... + a,'nxn > br once this constraint is deleted,
i.e., arlx l + ... + arnx n - br = x n+l ? O. A glance back at the section involving
the addition of a new variable to a given linear programming problem reveals

that in this instance we seek to generate a solution to a problem of the form
max f(X, x n+1) = c'x + OX n +1 s.t.
S·IDce -c n+ l - m = cn,n+l-m - C'B B- 1r n+ l - m = U'e r = u r > 0,the current

solution is no longer optimal so that we must insert into the optimal simplex
matrix an additional column of the form
The primal simplex routine then yields an optimal basic feasible solution to the
modified problem. On the other hand, if the structural constraint to be deleted is
of the "~" variety and its associated surplus variable is zero at the optimal
basic feasible solution so that the constraint is binding there, then the constraint
may be deleted from the optimal simplex matrix by adding to its left-hand side a
non-negative slack variable. Again looking to the rth (binding) structural
constraint, the introduction of the slack variable xn+1 provides for the likelihood
that arl Xl + ... + arnx n < br when the constraint is deleted, i.e., xn+1 =
br - ar1x 1 - •.• - arnx n ~ O. Proceeding as above, we wish to determine a
solution to
max f(X, x n+1) = C'X + OX n +1 s.t.
[A,er]r XJ =
fn+l
b,X~O,xn+l~O.
And if the (rth) structural constraint to be deleted was initially a strict equality,
then we may simultaneously add the subtract non-negative slack and surplus
variables x n +1 ,xn +2 respectively on its left-hand side, i.e., we must now solve
max f(X, x n +1' x n +2) = C'X + OX n +1 + OX n+2 s.t.
[A,er, -erl rx~J = b, X ~ 0, x n +1 ~ 0, x n +2 ~ O.

~n+~J
12 POSTOPTIMALITY ANALYSIS II
In the preceding chapter we considered, in part, an assortment of

postoptimality problems involving discrete changes in only selected components
of the matrices C, h, or A. Therein emphasis was placed upon the extent to
which a given problem may be modified without breaching its feasibility or
optimality. We now wish to extend this sensitivity analysis a bit further to what
is called parametric analysis. That is, instead of just determining the amount by
which a few individual components of the aforementioned matrices may be
altered in some particular way before the feasibility or optimality of the current
solution is violated, let us generate a sequence of basic solutions which in turn
become optimal, one after the other, as all of the components of C, b, or a
column of A vary continuously in some prescribed direction. In this regard, the
following parametric analysis will involve a marriage between sensitivity analysis
and simplex pivoting.
12.1. Parametric Analysis

If we seek to
max f{X) = C'X s.t.
AX = h, X~O,
then an optimal basic feasible solution emerges if X B B-lb ~ 0 and

C n- CBB- 1R ~ 0' (or, 111 terms of the optimal simplex matrix,
n
CBB-lR- C ~ 0') with f{X) = CBX B = CBB-lb. Given this result as our
starting point, let us examine the process of
319
(I) PARAMETRIZING THE OBJECTIVE FUNCTION. Given the above

optimal basic feasible solution, let us replace C by C* = C + OS, where 0 is a non-
negative scalar parameter and S is a specified, albeit arbitrary, (n xl) vector
which determines a given direction of change in the coefficients C i' j=l, ... , n. In
this regard, thec;, j=l, ... , n, are specified as linear functions of the parameter O.
Currently - Cn + CBB-1R ~ 0' or - Cj = - CRj + CBY j 2 0, j=l, ... , n-m.
If C* is partitioned as
where SB(SR) contains the components of S corresponding to the components of C

within C B{C R)' then when C* replaces C, the revised optimality condition becomes
- (Cn+OS n ) + (CB+OS B )B-1R

(12.1)
(-Cn+CBB-1R) + O( -Sn +SBB-1R) ~ 0'
or, in terms of the individual components of (12.1),
(12.1.1)
(Note that the parametrization of f affects only primal optimality and not
primal feasibility since X B is independent of C). Let us now determine the
largest value of 0 (known as its critical value, 0c) for which (12.1.1) holds. Upon
examining this expression it is evident that the critical value of 0 is that for
which any increase in 0 beyond 0c makes at least one of the -c; values negative,
thus violating optimality.
How large of an increase 111 0 preserves optimality? First, if
-Sn + SBB- 1 R ~ 0' or -s Rj + SBY j ~ 0, then 0 can be increased without bound
while still maintaining the revised optimality criterion since, in this instance,
(12.1.1) reveals that -c;~-ci20, j=l, ... ,n-m. Next, if = -SRj+SB
Y i < 0 for some particular value of j, t~en
c·
-c; ~ 0 for
o< J , = 0c.
- SRj + SB Y j
Postoptimality Analysis II 321
Hence the revised optimality criterion is violated when () > () c for some non-basic
variable XRj' Moreover, if -SRj+SB Y j < 0 for two or more non-basic
variables, then
() c = min{
j - S Rj
Cj
+ SB Y j I-SRj+SB Yj<O}. ( 12.2)
So of () increases and the minimum in (12.2) in attained for j=k, it follows that
() c = Ck/ ( -s Rk +SB Y k) or -ck = O. With -ck = 0, the case of multiple
optimal basic feasible solutions obtains,i.e. the current basic feasible solution (call
it X~») remains optimal and the alternative optimal basic feasible solution
(denoted X~») emerges when rk is pivoted into the basis (or xRk enters the set of
basic variables). And as () increases slightly beyond () c' -ck < 0 so that X~)
becomes the unique optimal basic feasible solution.
What about the choice of the direction vector S? As indicated above, S
represents the direction in which the objective function coefficients are varied. So
while the components of S are quite arbitrary (e.g., some of the Cj values may be
increased while others are decreased), it is often the case that only a single
objective function coefficient Cj is to be changed, i.e., S = ej (or -ej)' Address-
ing ourselves to this latter case, if C* = C + (}ek and xk is the non-basic variable
xRk' then (12.1.1) reduces to -cj = -Cj 2: 0 for ji=k, while for j=k, -ck =
-ck+(}(-sRk) = -ck-(} 2: O. A comparison of this expression with (11.1)
reveals that here () == Lk Rk (generally, Lk Rk == (}s Rk)' If () increases so that
() c = - ck' or - ck = 0, then the preceding discussion dictates that the column
within the simplex matrix associated with xk is to enter the basis, with no corres-
ponding adjustment in the current value of f warranted. Next, if xk is the basic
variable xBk' (12.1.1) becomes -cj = -Cj+(}sBkYkj = -cj+(}Ykj 2: 0,
j=I, ... ,n-m. Upon comparing this expression with (11.3.1) we find that here
() == .::lcBk (in general, .::lcBk == (}sBk)' In this instance (12.2) may be rewritten as
(12.2.1)
which is thus equivalent to (11.5.c). If the minimum in (12.2.1) is attained for

j=r, then 0c =yCkrr or - c; =O. Then rr may be pivoted into the basis so that
again the case of multiple optimal basic feasible solutions holds. Furthermore,
when x Rr is pivoted into the set of basic variables, the current value of 1 must
be adjusted by an amount LlI = LlcBkxk =0cxBk as determined in the preceding
chapter.
Let us now examine the geometric interpretation underlying the process of
parametrizing the objective function along with the subsequent generation of a
sequence of optimal basic feasible solutions as ° increases through a succession of
critical values. In Figure 12.1, let A represent the unique optimal extreme point
associated with a given maximization problem, where 1 is the corresponding
hyperplane with normal C. Given S, if 0IS, 0 < 01 < 0c = 0~1), is added to C to
yield a new normal Ci = C + 01 S (be virtue of the parallelogram law for vector
addition) we obtain a new hyperplane 1(01) which also represents a maximal
Figure 12.1
hyperplane since A continues to serve as the optimal extreme point. Thus A

yields an optimal basic feasible solution to both the original and adjusted
problems
max f(X) = C'X s.t.} max f(X) = (C+B1S)'X s.t.

and {
AX=b, X~O AX=b, X~O
respectively. If B is next increased to Be = (}~l), the optimal hyperplane f({}~l))

with normal C*(l) = C + B~I)S is tangent to % along the edge from A to B so
Be
that the adjusted problem
max f(X) = (C+{}~I)S)'X s.t.
AX = b, X~ 0
has two optimal extreme point solutions: (1) point A; and (2) point B, with the
latter emerging when the non-basic vector associated with the minimum in (12.2)
is pivoted into the basis. (Note that the effect of parametrizing the objective
function is to change its slope, i.e., as {} increases from zero to (}~l), the maximal
hyperplane is rotated about extreme point A.) And if B increases a bit beyond
(}~1) to, say, {}2' with (}~l) < (}2 < (}~2), B replaces A as the unique optimal extreme
point solution. In this instance B represents an optimal basic feasible solution
to the problem
max f(X) = (C+{}2S)'X s.t.
AX = h, X~O.
How much further can B be increased beyond (}2 without violating the optimality
of extreme point B? Actually, the current basic feasible solution at B remains
optimal for the problem
max f(X) = [C + ({}~I)+{})Sl'X s.t.
AX = h, X~O
as long as B~l) ::; {} ::; B~2), where B~2) = (}~1) + Be' with Be determined from (12.2)
in conjunction with the appropriate information contained within the latest
optimal simplex matrix. Moreover, if {} increases to (}~2), the optimal hyperplane
f({}~2)) with normal C*(2) = C + {}~2)S is tangent to % along the edge from B to
Be
C, with the adjusted problem

max f(X) = (C + (9~2)S)'X s.t.
AX = b,X~O
again exhibiting two optimal extreme point solutions: (1) point Bj and (2) point
C, with only the latter surviving when the non-basic vector corresponding to the
minimum in (12.2) enters the basis. And by increasing 9 slightly above 9~2), C
replaces B as the unique optimal extreme point solution. Finally, with f
bounded over $, a basic feasible solution will ultimately be found which preserves
optimality as 9 approaches infinity.
Two final observations concerning the geometry of parametrizing the objec-
tive function are in order. First, the maximum of f (written max f) is a piece-
wise continuous linear function of 9. If the objective function associated with the
hypothetical problem of Figure 12.1 appears as f = clxl + c 2x 2 and S = el , the
parametrized objective function is f = (c l +9)xl + c2x2. Then for 0 ::; 9 < 9~1),
max f = (c l +9)xf + c2x: is a linear function of 9 with slope d max f / d9 = xt
(Figure 12.2.a). For O~l):5 0 :5 9~2), max f = (c l + (O~l)+O) )xf + c2xf, with
d max f / dO = xf. And for 9~2):5 9 < + 00, max f = (c l +O~2»x~ + c2x~, with
d max f / d9 = x~. Second, both xl' x 2 are discontinuous functions of 9 since the
values of these variables change abruptly between basic feasible solutions (Figures
12.2.b,c).
max!
0
0(1) 0(2)
c c
a.
x2
x 2A
xB
2
xC
2
0 0
0(1) 0(2) 0(1) 0(2)
c c c c
b. c.
Figure 12.2
(2) PARAMETRIZING THE REQUIREMENTS VECTOR. Given that our

starting point is again an optimal basic feasible solution to a linear programming
problem, let us replace b by b* = b + ¢t, where ¢ is a non-negative scalar
parameter and t is a given (arbitrary) (m x 1) vector which specifies the direction
of change in the requirements bi , i=l, ... , m. Hence the bi, i=l, ..., m, vary
linearly with the parameter ¢. Currently primal feasibility holds with
X B = B- 1b ~ 0 or x Bi ~ 0, i=I, ..., m. With b* replacing b, the revised primal
feasibility requirement becomes
x*B n-1 (b+<pt)

= X B +<p B- 1t
(12.3)
where Pi denotes the ith row of B- 1 • In terms of the individual components of

(12.3),
X}1i = x Bi + <pPit ~ 0, i=I, ..., m. (12.3.1)
(Since the optimality criterion is independent of b, the parametrization of the

latter affects only primal feasibility and not primal optimality.) What is the
largest value of <P for which (12.3.1) holds? Clearly the implied critical value of
<P'<Pc' is that for which any increase in <P beyond <Pc violates feasibility, i.e., makes
at least one of the x}1i values negative.
To determine <Pc' let us first assume that Pit ~ 0, i=I, ... , m. Then <P can be
increased without bound while still maintaining the revised feasibility criterion
since, in this instance, x}1i ~ x Bi ~ 0, i=I, ..., m. Next if Pit < 0 for some i,
x}1i ~ 0 for
Hence the revised feasibility criterion is violated when <p> <Pc for some basic
variable xBi. Moreover, if Pit < 0 for two or more basic variables, then
(12.4)
Now, if <P increases and the minimum in (12.4) is attained for i=k, it follows that
<Pc = - x Bk/ Pkt or xBk =o. And as <P increases beyond <Pc' xBk < 0 so that b k
must be removed from the basis to preserve primal feasibility, with its
replacement vector determined by the dual simplex entry criterion.
Although the specification of the direction vector t is arbitrary, it is
frequently the case that only a single requirement bi is to be changed, so that
t = ei (or -eJ With respect to this particular selection of t, if b* = b + <Pek' then
(12.3.1) simplifies to :l:Bk=:l:Bi+¢f3ik~O, i=1, ... ,m. A comparison of this

expression with (11.8.2) indicates that here ¢ == .1bk . For this choice of t then,
(12.4) may be rewritten as
n1 -:l:
¢c = mi'l. PikB O' f3ik < 0} ,
I (12.4.1)
which is thus equivalent to (11.10.c). If the minimum in (12.4.1) is attained for

i= e, ¢c =
-x e
f3 B or :I:~e = O. If ¢ then increases above this critical value, :I:~e
ek
turns negative so that be is pivoted out of the basis to maintain primal feasibility.
Moreover, as bk changes, / must be modified by an amount .1/ = uk.1bk = uk¢c'
Let us now examine the geometry of parametrizing the requirements vector.
m
For the current optimal basic feasible solution (with ¢ = 0), BX B = l: bi:l: Bi =
i=1
b, i.e., b is a member of the convex polyhedral cone e B = {Y! Y = BX B ,
XB ~ O} spanned or generated by the columns of the basis matrix B. (For
instance, in Figure 12.3, let us assume that B = [a1'~] so that :l: 1a 1 + :l:2~ = b,
with both :1: 1,:1:2>0 since b€eBproper). Given t€e={Y!Y=AX, X~O}, if
¢It, 0 < ¢I < ¢c < ¢~I), is added to b to yield a new vector b: (where b: is
1 I
-aIj~--------------------~~----------------~aIj
o
Figure 12.3
obtained via the parallelogram law for vector addition) and b;I d~B' then there
exists an optimal basic feasible solution to the modified problem
max f(X) = C'X s.t.
AX = b!, X ~O,
'1'1 -
which may be written as

m m
BX~ = L bix~i = L b j (XBi+¢t!Jjt) (12.5)
i=1 i=1
If A.
'I'
is next increased to A. -
'l'c-'I'c
",(1) - - x Bk
- fJk t ' it follows that x*Bk =0 so
that (12.5) becomes
(12.6)
where bk is to be pivoted out of the optimal basis. Moreover, once the dual
simplex entry criterion chooses the vector to replace bk in the basis, the incoming
vector enters the basis at the zero level. If ¢ then increases beyond ¢~1), the
incoming vector also appears in the basis at the zero level. In terms of Figure
12.3, if ¢ = ¢I' b: I feB so that B yields an optimal basic feasible solution to the
adjusted problem obtained by replacing b by b~. And when ¢c ¢~I),
1
0 = xi =
so that Oal + x;~ = b* (1).
In this instance b* (1) lies along an edge of e B so that
¢c cP c
the associated optimal basic feasible solution is degenerate as expected. Hence ~
may be deleted from the current basis and replaced by 3.;3, where the new basis
matrix is B* = [3.;3, a 2] and e B* is the convex polyhedral cone spanned by ~, 3.;3.
Since b*(I) also lies along an edge of eB *, 03.;3 + x2~ = b*(I) so that 3.;3 enters
cP c cP c (1)
the basis at the zero level. And if () increases a bit above () c ' to, say, ¢2'
b~2feB* proper with X;3.;3+x~~=b:2 and x;,x~>O. (Note that the
process of parametrizing b involves rotating b about the origin.)
In general, as ¢ increases without bound, we obtain a series of critical values
and a corresponding sequence of optimal basic feasible solutions. And with t fe,
b; points in the direction of t as ¢ approaches infinity so that for the final basis,
t itself serves as the requirements vector (see Figure 12.3). For t; e, ¢ cannot
become arbitrarily large without violating primal feasibility since, as Figure 12.4
indicates, ¢ is the largest value of ¢ which restricts b* to e (again the
parallelogram law for vector addition applies). So for if; > ¢, b!; e, i.e., once ¢
increases beyond ¢ and (12.4) determines the vector to leave the optimal basis,
the dual simplex entry criterion cannot isolate a pivot element since in this case
all elements within the pivot row of the simplex matrix are non-negative.
Further increases in ¢ subsequently violate primal feasibility.
-alj~--------------~~---------------------------+alj
Figure 12.4
One final point is in order. In the preceding discussion we parametrized b

directly. However, if we consider the associated dual problem, then
parametrizing b can be handled in the same fashion as parametrizing the
objective function.
(3) PARAMETRIZING AN ACTIVITY VECTOR. Given that we have
obtained an optimal basic feasible solution to the problem
max f(X) = C'X s.t.
AX = b, X~O,
let us undertake the parametrization of a column of the activity matrix A. With
A previously partitioned as A = [B, R), it may be the case that some non-basic
vector r j is replaced by rj = r j + TV, j=I, ..., n-mj or a basic vector hi is

replaced by h7 = hi + TV, i=I, ... , m, where T represents a non-negative scalar
parameter and V is an arbitrary (m xl) vector which determines the direction of
change in the components of either r j or b i . In each individual case, the said
components are expressed as linear functions of T.
Let us initially assume that rk is replaced by rt = rk + TV in
R = [rl, ...,rk, ...,rn- m] so that the new matrix of non-basic vectors appears as
R* = [rl, ...,rt, ...,rn_ml. Since the elements within the basis matrix Bare
independent of the parametrization of rk' primal feasibility is unaffected (we still
have X B = B-1h ~ 0) but primal optimality may be compromised. In this
regard, for T = 0, CBB-1R- Cn~ 0' or - Cj = CBYj - CRj ~ 0, j=I, ... ,n-m.
Upon parametrizing rk' the revised optimality criterion becomes
C'B B-IR* - C'R -- (-c

-1 ,.··, -*
-ck"··' -
-c n _)
m > 0' ,
=
where
CBB-1(rk+TV) - cRk (12.7)

-ck+TCBB-1V= -ck+TU'V~O
and U' = CBB- 1 is a (1 x m) vector of non-negative dual variables. Since

-Cj ~ 0, j=I, ...,k, ... ,n-m, primal optimality will be preserved if (12.7) holds.
Now as T increases, primal optimality must be maintained. Given this restric-
tion, what is the largest value of T (its critical value, Tc) for which (12.7) holds?
To find T c' we shall first assume that U'V ~ O. Then T can be increased
indefinitely while still maintaining the revised optimality criterion. Next, if
U'V < 0, then -ct ~ 0 for
Ck
T ~ U'V = T c· (12.8)
In this regard, as T increases to T c' it follows that - ck = 0 so that the case of

multiple optimal solutions emerges. That is, the current basic feasible solution
(call it X~» remains optimal and an alternative optimal basic feasible solution
X~) obtains when rk is pivoted into the basis, with the primal simplex exit
criterion determining the vector to be removed from the current basis. And as T
increases a bit beyond T c' it follows that - eZ < 0 so that X~) becomes the
unique optimal basic feasible solution.
As indicated above, the choice of the direction vector V is completely
arbitrary. However, if only a single component rjk of the activity vector rk is to
change, then V = ej (or -ej). In this regard, if rZ = rk + Tee, then (12.7)
becomes - eZ = - ek + TUe ~ O. A comparison of this inequality with (11.13)
indicates that here T == Llrek' So for this choice of V, with ue ~ O,T may be
increased without bound while still preserving primal optimality. Next, if rZ =
rk - Tee' (12.7) simplifies to - eZ = - ek - TUe ~ O. In this instance (12.8)
may be rewritten as
(12.8.1)
If T then increases above this critical value, - eZ turns negative so that rk is

pivoted into the basis to maintain primal optimality.
When T is increased slightly above T = T~l), it may be the case that: T may
be increased without bound while still preserving primal optimality; or there
exists a second critical value of T beyond which the new basis ceases to yield an
optimal feasible solution. Relative to this latter instance, since rk has entered the
optimal basis, we now have to determine a procedure for parametrizing a basis
vector.
To parametrize a vector within the optimal basis, let us replace the kth
column bk of the basis matrix B = [b1 , ... , bk , • •• , bml by bZ = b k + TV
m
so as to obtain B* [b1, ••• , bZ, ... , bml. Since bZ =.E Yjkbj = BYZ, it follows
1=1
that
y*k
From (A.l) of appendix A,
1 o
1*m o 1
o
o 1
And from (A.2), (B*)-l = I~JB-I and thus

X'B = (B*)-l = I~B-Ib
r{JIV
xBl -l+r{Jk V xBk
Since X'B must be primal feasible, we require that
If 1 + T{Jk V> 0, then, upon solving the second inequality for T and remembering
that T must be strictly positive,
(12.9)
Since (12.9) must hold for all if:. k when 1 + T{Jk V> 0, it follows that the upper
limit or critical value to which T can increase without violating primal feasibility
is
(12.10)
To determine the effect of parametrizing the basis vector upon primal

optimality, let us express the revised optimality criterion as
C'B*(B*)-lR-C'R C'B* I*n-IR-C' R
n~ 0'
m
C~*I~[Y l' ... , Yn-ml- C

or, in terms of its components,
(12.11)
- C Rj ~ 0, j=I, ... , n-m.
If we now add and subtract the term

TCB*Jh V
C B*kYkj + 1 + TfJk V Ykj
on the left-hand side of (12.11), the latter simplifies to
TYkj Y;,c B*ifJi V

~ cB*iYij- 1 ~TfJkV -CRj
(12.11.1)
TykjU'V
1 + TfJkV Cj ~ 0, j=I, ... ,n-m.
If 1 + TfJk V> 0, and again remembering that T must be strictly positive, we may
solve (12.11.1) for T as
(12.12)
Since (12.12) is required to hold for all j=I, ... , n-m when 1 + TfJk V> 0, we may
write the critical value to which T can increase while still preserving primal
optimality as
T"
c (12.13)
As we have just seen, the parametrization of a basis vector may affect both
the feasibility and optimality of the current solution. This being the case, the
critical value of T which preserves both primal feasibility and optimality
simultaneously is simply
Tc = . {'
mzn T c' T If}
c • (12.14)
Finally, to ascertain the effect on the optimal value of f of parametrizing
b k , let us find
(12.15)
If the term
TCB*kPk V
cB*kxBk + 1 + TP k V xBk
is added and subtracted on the left-hand side of (12.15), we obtain
(12.15.1)
As far as the specification of the arbitrary direction vector V is concerned, it

is sometimes the case that only a single component of b i is to change, i.e., V = ei
(or -ei). Then if bt = bk + Tee' (12.9) reduces to (11.15.b) (here T == ..:1b ek ) so
that (12.10) is equivalent to the right-hand inequality of (11.19). Finally,
(12.15.1), (11.20.1) are one and the same for V = ee.
12.2. The Primal-Dual Method Revisited

In this section we shall consider what may be termed a parametric method
for solving a linear programming problem. While the technique to be presented
shortly is not a post-optimality routine proper, it utilizes the basic approach of
parametrizing the objective function discussed at the beginning of this chapter.
As indicated in chapter eight, when artificial variables appear in a linear

programming problem, possibly the one-phase (M-penalty) or two-phase method
may be employed to drive the artificial variables to zero so as to obtain an initial
basic feasible solution to the original problem. Once such a solution emerges, the
standard primal simplex pivot process can be used to generate a solution which is
also optimal. However, at the end of phase I (or when all artificial vectors have
turned non-basic in the M-penalty technique), the resulting solution may require
a considerable number of additional iterations to achieve optimality, i.e., both
the one- and two-phase methods ignore optimality in their search for an initial
basic feasible solution to the original problem.
It is this last observation then that has prompted the development of a
method that simultaneously works towards both feasibility and optimality of the
original problem. In this regard, when all artificial variables have been driven to
zero, the resulting solution is both primal feasible and optimal, e.g., phase I
terminates with an optimal basic feasible solution to the given problem.
While the original formulation of the primal-dual method (chapter 10) uses
alternatively the primal and dual problems, a modification of the standard
primal-dual routine (van de Panne, Whinston(1968); van de Panne (1975)) will
now be presented wherein a parametrized version of only the primal problem is
utilized. As demonstrated in the preceding two references, the original version of
the primal-dual method and its parametric analog are equivalent in the sense
that each gives rise to the same sequence of basic feasible solutions.
Let us assume that the primal problem appears as
min f(X) = C'X s.t.

AX = b, with C, X ~ o.
Then if we introduce into the structural constraint system an appropriate number
of non-negative artificial variables (here we shall assume that m such variables
are required), we obtain the artificial problem
min/(X, Xa) = C'X+ O'X a s.t.

AX + ImXa = b, with C, X, Xa ~ 0 or
max{h(X,Xa) -/(X,Xa) = -C'X-O'Xa} s.t.
b, with C, X, Xa ~ 0,
where Xa is an (m x 1) vector of non-negative artificial variables. To consolidate

our notation a bit, let us
max h(X) = C'X s.t.
AX = b, with C ~ 0, X ~ 0,
where
are respectively of order (n+m x 1), (n+m x 1), and (m X n+m). If we replace C
[-c]
by
-*
C = C- + AS = [-C]0 + A [-10] -AI '
where A is a non-negative scalar parameter and S is an (n+m xl) vector which
determines the direction of change in the coefficients cj, j=1, ... , n - m, then we
ultimately seek to solve the extended artificial problem
max{ h(X) = (C*)'X = - C'X - A1'Xa} s.t.

AX + ImXa = b, with C, X, Xa ~ O.
The simplex matrix associated with this problem is
[
AIm
C' AI' 1
0
o ].
b ( 12.16)
If we multiply the first m rows of this matrix by - A and add their sum to the
last row of (12.16), we obtain
o
(12.16.1)
1
Since an (m+l x m+l) identity matrix appears in this latter simplex matrix, the
m+l basic variables are the m artificial variables and f. With A = 0, X = 0, it
follows that Xa = b, °
f = represents an initial optimal basic feasible solutions to
the extended artificial problem since C f; O. The motivation underlying the
primal-dual method then is to increase A parametrically above its current zero
level (i.e., optimality is preserved from one basic feasible solution to the next) so
that for each successive iteration an artificial vector is replaced by a legitimate
one in the optimal basis. And as the parametrization process increases A through
a succession of critical values, the sequence of optimal basic feasible solutions to
the extended artificial problem converges to an optimal basic feasible solution to
the original problem.
In actual practice, the implementation of the primal-dual method mirrors
the procedure employed above in parametrizing the objective function. To see
this let us partition C* as
With B = 1m = B-1 , R = A, (12.16.1) may be rewritten as
-A~'b]
[ B-'R 1m 0 (12.16.2)
-CR+AS~B-1R 0' 1
Y2 Yn
l-~l'b}
1m 0
[ Y.
-CR+AS~Y1 -cR2+AS~Y2 -cRn +AS~Yn 0' 1
where a typical term taken from the first n of those appearing within the
last row of this matrix is
(12.17)
A careful examination of (12.17) reveals that it is equivalent to (12.1.1).

How large of an increase in ,\ above zero preserves primal optimality (dual
feasibility)? First, if SBYj ~ 0 for all j, then ,\ can be increased without bound
while still maintaining the optimality of the extended artificial problem. Next, if
SBYj < 0 for at least one value of j, then the critical value of,\ is
(12.18)
An examination of this expression indicates that it is equivalent to (12.2) So if ,\

increases and the minimum in (12.18) is attained for j = k, the kth non-basic
column replaces an artificial vector in the basis, with the latter determined by the
standard primal simplex exit criterion.
EXAMPLE 12.1. Using the parametric primal-dual technique, find
min f = xl + x 2 + x3 + 2X4 s.t.
Xl + 2X2 + 3x4 15
X2 + X3 + 2x4 10
xl, ... ,x4 ~ o.
If x 5' x6 represent artificial variables, the above problem may be rewritten in
extended artificial form as
maxh(x l , ... ,x6 ) = -xl-x2-x3-2x4-'\x5-'\x6 s.t.
xl + 2X2 + 3x4 + x5 15
X2 + x3 + 2x4
xl, ... ,x6 ~ 0,
1 203 1 o o 15
1 2 o3 1 0
o 15] o 1 120 1 o 10
[ o1 120 1 o 10 - - 1 1 120 o 1 0
1 1 1 2 ,\ ,\ 1 0
-,\ -3'\ -,\ -5'\ 0 o o -25'\
To facilitate the execution of subsequent rounds of the primal-dual method, each

entry in the objective function row has been separated into two parts, one which
does not involve A, and one appearing directly beneath it which does. Hence this
"double-row" representation actually depicts the "single" objective function row
of the simplex matrix.
For A = 0, we obtain an initial basic feasible solution to the extended
artificial problem consisting of x5 = 15, x6 = 10, and h =O. Let A now increase
in value above its current zero level. From (12.18),
1 _
dl) _
I\c - 3 - mJn. {-- 1l' - 1 - 1 - 2}
- 3' - 1 ' - 5
so that column two represents the pivot column. Upon performing a pivot
operation we obtain
1 3 15
2 1 0 2 0 0 2"
1 1 5
-2 0 1 2 1 0 2
1 1 15
2" 0 1 2" 0 1 -2
1A
2 o -A -p 0 0 -~A
where the column associated with the artificial variable x5 has been deleted since
x5 has turned non-basic. In this instance the optimal basic feasible solution to
the extended artificial problem is x2 = ~5, x6 =~, and h = - 1; - ~A. If we
increase A a bit further we may again employ (12.18) to determine
Since a tie for the minimum occurs, let us arbitrarily select column three of the
preceding simplex matrix as the pivot column. Upon pivoting we have
o
: ':],
3
1 2
o 1 21
o o o 1 -10
where this time the column associated with the artificial variable %6 has been
deleted along with the bottom ">' l'OW" since the latter contains only zero
elements. Here %2 = 125, %3 =~, and h = - f = -10 or f = 10 represents an
optimal basic feasible solution to the original problem. Moreover, the optimal
simplex matrix indicates that there exists multiple optimal basic feasible
solutions to the original problem, as the reader may easily verify.
13 INTERIOR POINT METHODS
13.1. Optimization Over a Sphere

It was mentioned earlier that the standard simplex method searches for an
optimum to a linear program by moving along the surface of a convex
polyhedron from one extreme point to an adjacent extreme point in a fashion
such that the objective value is nondecreasing between successive basic feasible
solutions. The simplex process then either terminates, in a finite number of
steps, at a finite optimal solution or reveals that no such solution exists. In this
chapter we shall examine a set of interior point methods for solving a linear
program (e.g., projective and affine potential reduction, primal and dual affine
scaling, and path following), so named because they converge to an optimal point
(if one exists) by passing through the interior of a convex polytope in a fashion
such that, at each iteration, a weighted least squares problem is solved.
The first serious challenge to the computational efficiency of the simplex
method came in 1984 when N. Karmarkar (1984) introduced his projective
scaling algorithm. Karmarkar's method is used to solve problems in what can be
termed K armarkar standard form:
min C'X s.t.
AX =0, I'X = 1, X ~ 0, (13.1.a)
where C, XfR", A is (m x n), with p(A) = m and n ~ 2, and both C, A are

integer matrices. (Section 13.6 demonstrates that any linear program can be put
341
in this form.) It is further assumed that:

(a) Al = 0, i.e., Xo = (~)I is feasible; and
(b) the optimal objective value for (13.1.a) is zero.
Based upon these considerations, it is evident that the problem is feasible, boun-
ded, and thus has an optimum. Moreover, the dual of (13.1.a) can be written
, -
max em +1 U s.t.
[A', I] ij ~ C
ij unrestricted.
If we let ij' = (U', z), where U € R m and Z€ R, then the preceding problem
appears as
max z s.t.
A'U + z I ~ C (13.1.b)
U, z unrestricted.
This dual is also feasible. That is, for any U € R m , one can choose
z= min(C-A'U)j so that the point (U, z) is feasible in (13.1.b).
3
A glance at the constraints in (13.1.a) reveals that the feasible region % is
formed as the intersection of the (n-m )-dimensional linear subspace
:I = {X I AX = 0, X€R n } and the (n-l)-dimensional unit simplex u=
{Xll'X=I, X~O, X€R n }, i.e., the polytope %=:lnu. Furthermore, the
m+l structural constraints in (13.1.a) are taken to be linearly independent,
Xo € %, and C'X ~ 0 for each X € %. Hence we seek to
min C'X s.t. X€%. (13.1.1)
In contrast to the simplex method, the projective transformation routine

generates a sequence of points inside % whose associated objective values (costs as
they are usually called in a minimization problem) approach zero. At
termination, it seeks out a vertex of no greater cost, which is then deemed
optimal.
To simplify the solution to (13.1.1), let us replace the constraint X€u by
the roundness restriction X€B(Xo,r), where B(Xo,r) = {XIIIX-XoII<r} is
Interior Point Methods 343
the largest sphere with center Xo and radius r that is contained in (1. Under this
restriction, the new feasible region (corresponding to the intersection of the
subspace 1 and a sphere centered in it) is a lower-dimensional sphere B(Xo,r) of
the same radius inside :1', i.e., B(Xo, 1') = :I' n B(Xo, r), where AXo = O. If C p
denotes the orthogonal projection of C onto 1, then we ultimately desire to
min C~X s.t. (13.1.2)
To solve this problem, we need only start at Xo and take a step of length r in the
direction of steepest descent -Cp ' thereby reaching a point Z on the surface of
B(Xo, r). The point Z so obtained minimizes the objective in (13.1.2). If
B(Xo, r) isn't much smaller than %, then C~Z should give a good approximation
to the minimum of the original objective function in (13.1.a).
13.2. An Overview of Karmarkar's Algorithm

In view of (13.1.2), Karmarkar's algorithm creates a sequence of feasible
points Xo,X1"",X k whose corresponding costs approach zero (under assumption
(b) above) by undertaking the following sequence of steps:
STEP O. Initialize.
Xo = (k) 1 = center of (1.
STEP 1. Compute the next point in the sequence

X k +1 = (X k ), k=O, 1, ... ,
To determine the role of the function (, let us assume that the current solution
point is X k with C'X k > O. To obtain the next point, apply the projective
transformation T(X) of simplex (1 that maps the input point X k to the center
Xo, i.e., X k is replaced by Xo under this centering transformation. (Each
iteration always starts at the center of (1 since the improvement in the objective
value obtained by moving from the center of the polytope in the direction of
steepest descent is typically much greater than the improvement resulting by
moving in the same direction from a point near the boundary of the polytope.)
The feasible region % of the new problem is formed, as was %, by intersecting the
simplex (1 and a linear space. (1 is then rounded about Xo by replacing it by the

largest sphere B(Xo, r) inscribed within 9G. By minimizing a transformed
objective function over B(Xo, r), we obtain a new point Y k+1 m transformed
space whose cost in the transformed problem is smaller than that of Xo. If the
inverse of the transformation T is applied to Y k +1, we obtain the output or new
solution point X k +1 =T- 1(Yk +1) in our original or true space.
Because the cost of Y k +1 in the transformed problem is less than the cost of
Xo, the cost of Xk+l in the original problem will be less than the cost of X k.
Repeated application of this procedure renders a sequence of points whose costs
approach zero.
STEP 2. Check for infeasibility.
Since the transformed objective function IS inherently nonlinear, Karmarkar
introduces a potential function
f(X) = ~ In (C~~) ~ C'X ( 13.2)
in true space to keep track of the progress of the objective function towards
optimality. (This potential function is a surrogate for the original linear
objective function since it tends to be much better behaved under the projective
transformation.) An improvement 6 in the potential function is expected at each
step of the algorithm, i.e., !(Xk+l) ::; !(X k ) - 6. If the expected improvement in
f(X) does not materialize, then we terminate the process and conclude that the
minimum value of the objective function is strictly positive. Hence the original
problem does not possess a finite optimum, i.e., the original problem is either
infeasible or unbounded.
STEP 3. check for optimality.
This involves going from the current point in true space to the nearest extreme
point in (1 without increasing the value of the objective function and testing the
extreme point for optimality.
Let us now look to the particulars of Karmarkar's algorithm. Specifically,
to examine the preceding steps in detail, we need to further scrutinize: the
projective transformation itself; the transformed problem; the notion of

optimization over a sphere; requirements for potential function improvement; the
test for optimality; and algorithm complexity.
13.3. The Projective Transformation T(X)

We noted above that to solve (3.1.2) we should move from Xo = (*)1 to
the boundary of the inscribed sphere along the radius vector of most rapid
objective function improvement or descent. After a single such step, the solution
is no longer at the center of (T. To produce an algorithm that always steps from
the center of the simplex, use a projective transformation so that, at each
iteration, calculations are performed in a transformed space, where the current
solution is the center of (T (and thus of the inscribed sphere), and then mapped
back to true space. How shall we form this recentering transformation?
Karmarkar's projective transformation is a one-to-one and onto vector-
valued function T: (T~(T. Specifically, for each interior point of (T = {X 11'X = 1,
X ~ O} there is a unique projective transformation T of l'X = 1 fixing all vectors
of (T and sending X to the center Xo = (*)1. Hence we have a transformation
that converts the current problem into a new one in which the current feasible
solution X k > 0 is replaced by Xo.
Since the image of every X E (T must also lie in (T, the sum of the coordinates
of T(X) must be one. Thus a transformation which maps X k to Xo is to map
the point (x 1, ... ,x n ) to (xt/xk1"",xn/xkn) and then divide by the sum of the
coordinates, i.e.,
T:X~Y, where Yj = Tj(X) ( 13.3)
In matrix terms, if
-1
xk1 0 0
-1
0 xk2 0
Ck
-1
0 0 xkn
then (13.3) becomes

(13.3.1)
Alternatively, we may, at times, find it convenient to rewrite this expression as

follows. If
c-k l o
o 0 ... xkn
then (13.3.1) is replaced by

O-IX
Y = T(X) = k 1 . (13.3.2)
1'0-
k X
Under this transformation any point X€ 0" is transformed into Y € 0" since I'Y =
I'T(X) = 1, Y ~ O. Thus the current point X k is transformed into Yo = Xo =
(~)1.
Since T is one-to-one and onto, it has a unique inverse given by
(13.4)
or
(13.4.1)
Looking to the properties of the projective transformation T we have:

(a) T(X k ) = (~)1.
(b) T is a one-to-one mapping from 0" onto 0" with T- 1 given by (13.4).
(c) For X€O", let Y = T(X). Then:f = {X I AX=O} = {T-I(y) I AOkY=O}.
(d) If f = {Y I AOkY=O}, then T(:f) = f. Hence if Xk€:f then Xo = Yo =
(~)ld.
I prom (13.3.2), X = (Dk Y)(I'D~ly). Since I'X = I, the preceding expression for X
becomes 1 = (I'D kY)(I'Dk" I X) or I'Dk" X = 1/I'DkY. Hence
X = (DkY)(I'Dk-:-IX) = Dk Y
l'DkY'
(e) Since T(u) = 0' and T(:f) = 1, it follows that

T(9G) = T(un:f) = un1 = %.
For instance, to verify (a), it is easily seen, from (13.3), that
T .(Xk) = n(xk;lxk j ) = ft.

J Ee=l(x ke/ x ke)
Hence T(X k ) (It)1. That (c) holds follows from the notion that AX = 0 if
and only if
Geometrically, Figure 13.1 illustrates the projective transformation of a

point X k to Yo.
{YIADk Y=O, l'Y=1, Y ~ O}
Yl
a. b.
Projective Transformation of the
Feasible Region: Xk f {X I AX=O, l'X=1, X ~ O}
is transformed under T to
Figure 13.1
13.4. The Transformed Problem

Under (13.4.1), the original problem (13.1.a) can be transformed into a
linear fractional program in Y over a subset of {Y Il'Y=I, Y ~ O} as
. {C'D IADkY -
mm l'D kkY
Y - 0, '
1Y- > o} ,
- 1, Y = (13.5 )
z.e., the projective transformation T carries u into itself, X k to Yo =(~)l, the

feasible region {X I AX=O, l'X=I, X ~ O} into {Y I ADkY=O, l'Y=I, Y ~ OJ, and
C'X into the fractional objective C'DkY/l'DkY. By virtue of assumption (b)
above, we need only minimize the numerator in (13.5) since the denominator is
different from zero for all Y for which l'Y=I, Y;::: O. Hence the preceding
problem becomes the scaled linear program
min e'Y s.t.

(13.5.1)
PY=Po'Y~O,
where
As discussed earlier, instead of solving (13.5.1) directly, we shall solve its

restriction to a sphere inscribed within u. If B(Yo' r) is the largest sphere with
center Yo = (~)l and radius r contained in u, then the said restriction of (13.5.1) is
min e'Y s.t.

(13.5.2)
Y € {Y I PY=Po} n B(Yo' r), Y ~ 0,
Here the feasible region is formed as the intersection of the (n-m-l )-dimensional
affine space {Y I PY=Po} and the sphere B(YO,7') and amounts to a (n-m-l)-
dimensional sphere centered at Yo with radius r. If C p denotes the orthogonal
projection of e onto the null-space of P or onto {Y I PY=O} (so that C p lies on
the constraint surface of PY=Po), then we ultimately desire to
min C~Y s.t.

(13.5.3)
Y € {Y I PY=Po} n B(Yo' r), Y ~ 0,
where
(13.6)
This can be accomplished by moving from Yo in the direction of steepest des-

cent - C p to the boundary of the sphere.
Let the direction of Cp =P 0 be Cp = C p / I C p II. Then using a step
parameter a, 0 < a < 1, and taking a step of length ar in the projected steepest
descent direction - C p renders the new point
(13.7)
in transformed space. To generate the corresponding point in true space we need

only employ (13.4.1) to obtain Xk+l =r- 1(yk +1 ) or
D k Y k +1
Xk +1 = I'D Y . (13.8)
k k+l
Given that Y k+I > 0 (i t lies in the interior of B(Y0' ar»), it follows that Xk+I > O.
Figure 13.2
2See Appendix C for details regarding the derivation of this projection vector.
Looking to the geometry of (step 3 of) Karmarkar's algorithm, Figure 13.2

depicts the intersection of a two-dimensional simplex u = {Y 11'Y=1, Y ~ O} and
a three-dimensional sphere with center Yo and radius r chosen so that the said
intersection is a two-dimensional sphere or circle inscribed within the simplex.
Starting at Yo' the algorithm moves us a distance ar along the constraint surface
{Y I PY=Po} in the direction of - Cp to the boundary of the sphere B(Yo,ar).
We noted earlier that the approximation to the true cost objective C'X in
true space was given by the potential function (13.2) as
I(X) = t In(~~)
3=1 3
= nln (C'X) - t In(zj)'
3=1
(13.2.1)
By virtue of (13.4) and (13.4.1), C'X/z j = C'DkY/ZkjYj = C'Y/zkjYj so that

the approximation to transformed cost in transformed space is provided by the
translonned potential function
g(Y)
(13.9)
_ n n
= nln(C'Y)- L In(Yj)- L In(zkj)'
j=1 j=1
Direct substitution easily establishes that I(X) = 9 (T(X»), i. e., I measures the
same progress towards optimality in true space that 9 measures in transformed
space.
In sum, Karmarkar's algorithm is a projected gradient method designed to
minimize the potential function I, where each iteration is performed in
transformed space and the transformation moves the current point X k f u to the
center of the simplex while keeping the form of I invariant. Under repeated
projective transformations, each followed by optimization over a sphere inscribed
within the feasible region, the algorithm generates a sequence of interior points
converging to the optimal solution.
13.5. Potential Function Improvement

and Computational Complexity
We previously denoted the radius of the largest sphere with center
Yo = (~)l that can be inscribed within the simplex {Y II /Y=l, Y ~ O} as r. If
we view r as the distance from the center of this sim plex Y~ = (~ , ... , ~) to one of
its facets, where a facet is actually a simplex of one fewer dimension, then
choosing, say, the facet at which the component Yl = 0, it is easily verified that r
is the distance from (~, ... ,~) to ( 0, n:l'''·' n:l) or r = ';n(~-l)·

For computational purposes, instead of optimizing over B(Yo' r), the
algorithm is designed to optimize over the smaller sphere B(Yo' ar), 0 < a < 1.
This enables the optimum of the transformed potential function to be
approximated very closely by the optimum of the linear objective in (13.5.3).
Karmarkar's analysis of how large a step ar to take is treated in terms of
potential functions. Specifically, Karmarkar (1984, pp. 379-381) addresses the
problem of minimizing g(Y) for Y E {Y I PY=Po} n B(Yo' ar) = %. He
establishes that there exists a point that achieves a constant reduction in this
transformed potential function, i.e., there exists a point YE % such that
g(Y) ~ g(Yo) - 8. And since ming(Y) can be approximated by minC/Y (i.e., the
potential function is almost invariant under T(X) or g( T(X») - f(X) is
independent of X), Karmarkar next proves that if Y minimizes C/Y over %, then
g(Y) ~ g(Yo) - 8. How can we find the Y that minimizes C/Y? As Karmarkar
easily demonstrates, the point Y generated by the projective scaling algorithm
minimizes e'y over %.
Based upon the preceding discussion we may keep track of any potential
function improvement by stating the following
13.1. LEMMA. For Y k +1 given by (13.3.2), either
-,
(a) C Yk+I = 0 or
(b) g(Yk+I) ~ g(Yo) - 8.
(Since f(X) = g( T(X)), it follows that if g(Yk+I) - g(Y)o) - 8, then
g(T(Xk+1)) ~ g( T(X k )) - 8 or f(X k+1) ~ f(X k ) - 8.) If case (a) obtains, then
the inverse transformation (13.4.1) is used to return X k +1 and we stop. If case

(b) holds, we again apply the inverse of T to get X k +1 • The almost invariance of
1 implies that I(X k +1) ~ I(X k ) - 6 (provided that the minimum cost is zero).
The procedure is repeated, with each iteration decreasing 1 by 6.
By virtue of the preceding discussion we should expect an improvement 6
(which depends upon the a chosen) in the potential function at each step. If the
expected improvement does not materialize, i.e., if I(X k +1) > I(X k ) - 6, then we
stop and conclude that the minimum value of the objective function must be
strictly positive, in which case the original problem (13.1.a) has no solution.
The notion of the complexity 01 an algorithm addresses the issue of the
computational resources expended in attaining a solution to a particular instance
of a problem. More specifically, complexity theory categories problems in terms
of the forms of the mathematical functions which express the amount of
computation as a function of instance size. Moreover, computational complexity
assessments are based on the worst-case computational performance of the
algorithm. For instance, if an algorithm requires a number of computations that
can be expressed as an 7· th order polynomial of the problem size n, then the
algorithm is said to be 01 order nf" and written O(nr).
In general, computations can grow polynomially or exponentially. For
instance, the standard simplex algorithm is termed computationally inefficient
because of its worst-case behavior, i.e., although it usually requires a low-order
polynomial number of iterations to reach an optional solution (provided one
exists), it exhibits an exponential growth in computational effort on certain types
of problems as problem size increases (Klee, Minty (1972) construct a class of
problems in m=n constraints and 2n nonnegative variables for which the simplex
method requires 2n - 1 iterations). However, Karmarkar's algorithm is
considered computationally efficient since it exhibits a growth in solution effort
that is polynomial in the size of the problem. To see this let us consider an
instance of Karmarkar's problem in standard form. The size of this instance is
given by m, n, and L, where L, termed the input length of an instance of the
problem, is the number of binary bits required to record all of the data of the
problem. Given integer data matrices A and C, the length of input can be
expressed as
m n
L = E E rlog( I aij I + 1)l + 1,
i=O j=1
where rx 1denotes the smallest integer? x and aOj == Cj (Goldfarb, Todd, (1989),
p. 142).
In view of the preceding discussion, Karmarkar's algorithm generates a
sequence of strictly feasible solutions, starting from Xo = (~)1, that satisfy, for
a = ~ (a value that seems to work well), g(Yk +1) - g(Yo):::; So when -l.3
l
a ~~, g, and thus I, falls by about 8 ~ after every iteration. Hence over k
iterations, I{X k +1) - I(X k ):::; -~. From (13.2.1),
I(X k ) - I(Xo) = n In[Cc:Xk] -

Xo
t
)=1
In(nxkj)'
Using this expression it can be shown (Bazaraa, et al., op. cit.) that
C'Xk]
nln [ - - < --k or
C'Xo - 5
C'X k :::; e- k / 5n C'Xo, k=O, 1,2, ....
For k = 10nL, this performance guarantee renders C'X k < 2- L , i.e.,
Hence Karmarkar's algorithm produces a solution with objective value less than
2-£ within O{nL) iterations, and with an overall effort (in terms of arithmetic
operations) of polynomial complexity O(n3 .5 L) in the worst case (Karmarkar,
(1984)).
3Using (13.9) it can be shown that
-2
g(Yk +1) - g(Yo):::; - a+ a_ 2 = -6,
2( I-a)
where a = na/(n-l), O<a<1 (Bazaraa, et al. (1990), pp. 394-398). A simplified version of
this argument sets 6 = a - a 2/(I-a), 0<a<I/2.
13.6. A Summary of Karmarkar's Algorithm

We may summarize the sequence of steps involved in implementing
Karmarkar's projective scaling algorithm as follows.
13.2. KARMARKAR'S ALGORITHM. Quite generally, the algorithm
creates a sequence of points Xo, Xl' ..., X k from the steps:
STEP O. Initialize.
Choose Xo = (A)l as the center of the simplex 0".
STEP 1. Compute the next point in the sequence.
a. Perform the projective transformation T(X) of the simplex 0"
that maps the input point X€O" to the center Yo€O".

b. Optimize (approximately) the transformed objective function
over an inscribed sphere to find a point Y k+I.
c. Apply the inverse of T to Y HI to obtain the output point
Xk+I·
STEP 2. Check for infeasibility.
If C'XHI = 0, stop; if !(Xk+I) > !(X k ) - 8, conclude the optimal
cost value is positive so that the problem has no feasible solution.
STEP 3. Check for optimality.
Find a vertex X of cost no greater that C'Xk+I. Then take
C'X = O.
More specifically, for problem (13.1.a) with A, C integer matrices and
p(A) = m, let Xo = (A)l be feasible and let the optimal objective value be zero.
Execute:
STEP O. Initialization.
Choose Xo = (A)l, r = 1/ In(n-l) , (l = (n-l)/3n and fix the
tolerance L.
Set k = O.
STEP 1. Stopping.
If C'X k < 2- L , stop. X k may be rounded to an optimal solution.
Otherwise proceed to
STEP 2. Projection.
b. Let the projection of C onto P's null-space be

Cp = [In - P'(pp')-lp] C.
c. Let Cp = Cp /IICpll. If Cp = 0, let Cp = O.
STEP 3. New Point Update in Transformed Space.
Take a step of length or in the direction of - Cp by forming
Y k +1 = Yo-orC p , 0 < 0 < 1.
STEP 4. New Point Update in True Space.
D k k+1 Y
Xk+1 = T -1 (Y k+1) = I'D Y
k k+1
.
STEP 5. Advance k to k + 1.
Return to step 1.
In step I immediately above it was indicated that if stopping is in order,
then X k may be rounded to an optimal solution. If n linearly independent
structural constraints are binding at X k , then clearly X k constitutes an optimal
basic feasible solution to the linear program. Otherwise, there exists a feasible
direction d f. 0, IIdll = 1, satisfying 1'X =I and the binding constraints within
AX = O. If C'd < 0, we use the algorithm to move along the direction d (or
along - d if C'd> 0) until we reach some constraint boundary. Hence at least
one additional linearly independent structural constraint is binding at the new
solution point, where the new objective value is no greater than C'X k • This
purification process is continued until a basic feasible solution X to (13.1.a)
emerges with C'X < 2- L . Thus X is optimal with C'X = o.
13.7. Transforming a General Linear Program

to Karmarkar Standard Form
There are a few ways in which a given linear programming problem can be
converted to the form depicted by (13.l.a). First, let us
minC'X s.t.
(13.10)
AX=b, X~O,
where A is (m x n) and peA) = m. Next, we introduce the compac1ifying

(bounding) constraint I'X:5 nQ, where Q is chosen large enough so that any
feasible solution X satisfies I'X < nQ unless (13.10) is unbounded. Taking x n +1
as a nonnegative slack variable, (13.10) becomes
minC'X s.t.
AX=b
(13.10.1)
I'X + x n +1 = nQ
X ~ 0, x n +1 2: O.
If we set
-
X = (nQ) -1 X,
xn +1 = (nQ)-l x n +1 ,
- -1
b = (nQ) h,
then we may rewrite (13.10.1) as

mine'X s.t.
I
,-X+x n +1 = 1
( 13.10.2)
X ~ 0, xn +1 2: O.
Subtracting bI'X + bX n +1 = b from AX = b leaves us with the problem
min C'X s.t.
(A - bI')X - bX n +1 = 0
(13.10.3)
Let us now introduce an artificial variable xn +2 2: 0 into (13.10.3) in order

to render (x 1 , ... ,x n , xn +1 ' xn +2) = (1/n+2, ... ,1/n+2) an initial feasible
solution. To this end let the objective function coefficient on xn +2 be M and,
within the structural constraints, let the coefficients on each of the resulting
homogeneous equations sum to zero and let the coefficient on xn +2 within the
final constraint equation be one. Hence (13.10.3) becomes
minC'X+ MXn+2 s.t.

(A - bl')X - bX n +1 - [(A-bl') 1 - b]x n +2 = 0
(13.10.4)
I'X + xn+1 + x n +2 = 1
X ~ 0, xn+1 ~ 0, xn+2 ~ O.
Program (13.10) has been transformed to Karmarkar standard form (13.1.a) in

n +2 variables with (n~2)1 a feasible solution (assumption (a) above). If the
optimal objective value for (13.10.4) is zero (assumption (b)), then Karmarkar's
algorithm applies. (If in solving (13.10.4) we find that xn +2 > 0, then (13.10) is
infeasible. And if xn +1 = 0, then (13.10) is unbounded.) If a problem is in
Karmarkar standard form (13.1.a) and has a known objective function value of
v*, then to construct a new objective function with a minimum of zero as
required by Karmarkar's algorithm, simply form 1= / - v* = C'X - v*I'X =
(C-v*I)'X. Hence Karmarkar's algorithm can be applied using I instead of J.
Next, to obtain a transformed problem which satisfies both assumptions (a),
(b) stated earlier, let us apply the preceding procedure to the combined primal
and dual problems (Karmarkar (1984); Goldfarb, Mehrotra (1988)). For
example, from the primal-dual pair of problems
Primal Dual
minC'X s.t. max b'U s.t.
AX~b, X~O A'U ~C, U ~O
we may form the combined primal-dual problem
AX~b
A'U:::;C
(13.11)
C'X- b'U o
X~O, U ~O,
where A is (m x n) and p(A) = m. Clearly this combined problem is feasible if

and only if the original primal problem has a finite optimal solution.
Upon inserting vectors of surplus and slack variables X s ' Us respectively

into the structural constraint inequalities of (13.11) and introducing a
compactifying constraint on the sum of the variables, (13.11) becomes
AlX l =b
A 2 Ul = C
(C', O')Xl - (b', O')U l = 0 (13.11.1)
l'Xl + l'U l + x n +l = R
Xl' Ul ~ 0; X n +l ~ 0,
where
Xn+1 represents a slack variable, and R depicts an integer upper bound.

Next, setting
- -1 - 1
Xl = R Xl' Ul = R- Ul ;
b = R-lb, C = R-lC;
. -,- ,- - - - - -,-
andsubtractmgb(1 Xl+1 U l +X n +1) = b from AlX l = band C(I Xl+1 U l +x n +1)
,- -
C from A2 fi l = C enables us to transform (13.11.1) to
(Al-bl')Xl - bl'fi l - bX n +l = 0
- CI'X I + (A2 -CI')fi l - CX n +l o
(C', O')X l - (b', O')fil = 0 (13.11.2)
,- ,- -
I Xl + I Ul + x n +1 = 1
Xl' fi l ~ 0; Xn+1 ~ O.
Finally, let us modify (13.11.2) slightly by introducing xn+2 ~ 0 as an

artificial variable whose value must be driven to zero and whose structural
constraint coefficients are chosen so that within each homogeneous equation the
sum of the coefficients is zero and within the last equality constraint the x n +2
coefficient is unity. We thus obtain
minx n +2 s.t.
(AC bl')X1 - bl'U1 - bX n +1 -[(Al -2bl')I- bfn+2 = 0
- Cl'X1 + (A 2 -C'l)U 1 - CX n+1 - [(A2 -2Cl')1- cfn+2 o

(C', O')X1 - (6', O')U1 - (C'l-b'l)x n+2 = 0 ( 13.11.3)
Here (13.11.3) is of the requisite form (13.1) and also satisfies assumptions (a),
(b) above. Solution by Karmarkar's algorithm then yields an optimal pair of
primal-dual solutions( provided they exist).
13.8. Extensions and Modifications of Karmarkar's Algorithm

Discussions pertaining to the largest decrease in the potential function f via
selecting the best value of the step parameter C\' at each iteration have been
offered by Blair (1986), Pad berg (1985), Shanno (1988), and Todd and Burrell
(1986). While specific values of C\' have been suggested as optimal, Todd and
Burrell (1986) note that it is worthwhile to conduct a line search on the potential
in the direction - C p from X k rather than choosing a fixed C\' (since along any
such line f has a unique minimum).
We noted at the outset of this chapter that the optimal value of the
objective function in (13.1.a) is zero (assumption (b)). Attempts at relaxing this
restriction have been made by Anstreicher (1986), Gay (1987), Todd and Burrell
(1986), and Ye and Kojima (1987). For instance, Todd and Burrell (1986)
demonstrate that, in transformed space, the objective value moves toward a
lower bound given by duality, rather than toward zero. Thus their technique,
which is essentially a primal projective algorithm using dual variables, applies
even when the optimal value of the objective is not zero or when it is actually
unknown. Moreover, it applies even in degenerate cases and generates both
primal and dual interior solutions with lower bound objective values which
converge (from below) to the common optimal primal and dual solutions. (Note
that the Todd and Burrell extension of Karmarkar's algorithm provides us with
an optimal dual solution - a feature absent from Karmarkar's original
formulation. )
To see how the Todd-Burrell method works let us first drop the assumption
that the optimal objective value for (13.1.a) is zero and rewrite its dual (13.1.b))
as
max Z s.t.
(13.12)
A'U +zl ~ C.
Let the optimal solution for the primal objective (13.1.a) be z* (a part of the
optimal solution of (13.12)). Since z* is unknown, we may use lower bounds zk
on z*. To this end let us replace C by C(zk) = C-zkl at the kth iteration. Todd
and Burrell (1986) then demonstrate that Karmarkar's algorithm can be modified
to generate sequences {Xd, {( Uk' zk) } satisfying:
(a) Xke{X I AX = 0, (ft)l'X = 1, X> OJ,
(b) Uk = (ADlA')-lADlCeRm , (13.13)
(c) zk = min(C-A'Uk)j
3
such that
(d) f(Xo; C(zk)) - f(X k; C(Zk)) ~ In for all k.
Here Uk in (l3.13.b) results as the solution of the system of normal equations
(ADZA')U = AD~C for the weighted least squares problem nunll Dk(A'U-C) IF
(Appendix C). (For k = 0 choose Xo = (ft)l, Uti = (AA,)-l AC, and the corres-
ponding zOo Then (13.13.a-d) hold trivially.)
Based upon these considerations we now state the Todd-Burrell
13.3. ALGORITHM.
(1) At iteration k we have X k , Uk' Zk. Compute, for Ak = ADk ,
W = P AkDkC and V = P AkXk'
where
P Ak = In - Ak(AkA~)-l A k·
is the projection onto the null-space of A k .
(2) If
min(W-zkV)j::; 0
J
(we have not improved our previous lower bound zk on z*),
then set Uk+I = Uk' Zk+l = zk' Otherwise find zk+I > zk with
mjn[Dk(C(Zk+l) - A'Uk+ 1 )] = mjn(W-zk+I V)j = 0
(we apply a minimum ratio test to obtain zk+I) and set
Uk+1 = (A~)tXkC(Zk+l) = (AkA~) -lAkDkC(Zk+I)'
(3) Compute, for P = In -ll'jn (the matrix for a projection onto
{Y Il'Y=O} € R n ),
dk = -P(W-zk+l V), d k = Dkdk,
and
(4) Set
On the basis of this routine we are thus able to conclude that

13.4. THEOREM [Todd, Burrell (1986)]. The preceding algorithm gener-
ates a sequence {Xd of primal feasible and a sequence {(Uk,Zk)} of
dual feasible solutions, with C'X k and zk approaching z*. Indeed,
(C'Xk-z*) ::; exp( -kj5n)(C'Xo-z*).
As was the case with Karmal'kar's basic projection algorithm, the Todd-Burrell
mod-ification also converges in O(nL) iterations and O(n 3 .5 L) arithmetic opera-
tions.
Variants of Karmarkar's algorithm which apply to the standard form of a
linear program
minC'X s.t.
(13.14)
AX=b, X~O
rather than to Karmarkar's standard form (l3.1.a) and which do not impose any
a priori restriction on the optimal value of the objective function have been
introduced by Barnes (1986), Anstreicher (1986), deGhellinck and Vial (1986),
Gay (1987), Ye and Kojima (1987), Todd (1988a), and Gonzaga (1989a). Such
methods are referred to as standard tonn projective algorithms. Common to all

of these techniques is a procedure for maintaining lower bounds on the optimal
objective value (typically the one advanced by Todd and Burrell is used although
an alternative procedure based on geometry has been proposed by Anstreicher
(1986)) and the reduction of a potential function which drives the gap between
the objective value and the lower bound to zero. For instance, we may rewrite
(13.14) as
min (C', 0) [ ~] s.t.
(13.15)
1, X ~ 0, 1] ~ 0
or
~,~
minC X s.t.
-- ,- -
AX = 0, d X = 1, X ~ 0,
(13.15.1)
where
C [~ l X= [~l A = (A, -b), d = [~l

Clearly optimal primal and dual solutions to (13.14) may be obtained directly
from the same for (13.15.1) and conversely.
As indicated by Todd (1988a), if X* is a strictly feasible solution
to (13.15.1), then a scalar multiple of X* is strictly feasible for
min (C-zd)'X s.t.

(13.16)
AX = 0, I'X = 1, X~ 0.
Moreover, if z* is the optimal objective value for (13.15.1), then a scalar multiple
of z* is optimal in (13.16). Based upon these considerations, if at iteration k we
have a strictly positive feasible solution X k for (13.16) and zk depicts a lower
bound on the optimal objective value for the same, then zk is also a lower bound
on the optimal objective value for (13.15.1) under dual feasibility. If the feasible
region for (13.15.1) is bounded, then we may apply the preceding algorithm
advanced by Todd and Burrell and given immediately above to (13.16) so as to

generate a solution to (13.15.1) and thus to (13.14). The above theorem
accompanying their algorithm applies here also.
We noted in the preceding section that Karmarkar's projective algorithm
can be applied to a combined primal-dual linear program since this formulation
conveniently has a feasible interior solution on the simplex. However, applying
the projective algorithm in primal-only form requires that an initial feasible
solution be at hand.
While the problem of obtaining an initial interior solution has typically
been addressed by simply assuming that one exists or by introducing an artificial
variable and its attendant M-penalty into the problem objective, Anstreicher
(1986, 1989) has approached this problem by including an explicit constraint that
restricts an artificial variable to zero at optimality, thus avoiding any M-penalty
term in the standard form linear program. That is, Anstreicher deals with the
artificial variable T ~ 0 directly by imposing a constraint (satisfied in the limit)
that it be zero, thus leading to the problem
m;n( 6',0,0 {~] ,.t.
(K,-b, b-Kl)m 0
(w'l'O{~] 1 (13.15.2)
(W,O,I)m ~0
m~o.
The procedure for updating lower bounds on the primal objective value z* also
renders dual feasible solutions and his variant of Karmarkar's projective
algorithm generates a sequence of strictly positive points satisfying all constraints
save T = O. Starting from 1, if (13.15.2) has a nonempty bounded set of optimal
solutions,4 the method approaches feasibility and optimality simultaneously.
Once a lower bound is found, the next iteration always begins at the initial
solution Xo = 1. Moreover, the algorithm either detects an infeasibility in a
finite number of iterations or produces a sequence of lower bounds for z* which
diverges to - 00.
A modification of Anstreicher's standard form projective algorithm has been

offered by Todd (1992). Starting from a problem of the form
minC'X s.t.
AX=O
d'X= 1 (13.15.3)
eX=O
X~O
and given that a strictly positive solution satisfying all but one constraint (the
exception is e'X = 0) is known, Todd develops improved search directions by
solving a two-constraint linear program (essentially a relaxation of a scaled
version of (13.15.3)), the optimum value of which, if finite, serves to provide a
lower bound on the objective value z* and signals the existence of good search
directions. These modified directions are generated by a norm-minimizing
subproblem, where the solution of the relaxed problem provides a feasible
solution of the direction-finding subproblem. The solution of this subproblem is
a search direction that assures progress towards feasibility, optimality, and, if
possible, monotonicity. In Todd's approach, when a lower bound is found, the
next iterate is derived from the current one and not from Xo = 1 and Zo = zk as
in Anstreicher's algorithm.
4If we set C ,=(C,O,O),

-, - - - -
A=(A,-b,b-Al), d I =(0,1,0
')
and X, = (~'
X ,'1,r) and if
(13.15.2) is feasible with {X I C'X:5 0, AX = O,d'X = 0, X ~ O} = {O}, then (13.15.2) has a
bounded set of optimal solutions.
Additional standard form primal projective algorithms which are also

combined phase I-phase II methods (i.e., they do not require an initial feasible
solution) are those of deGhellinck and Vial (1986), Fraley (1990), Shaw and
Goldfarb (1991), and Anstreicher (1993). DeGhellinck and Vial develop an
interior point method that produces a sequence of not necessarily feasible iterates
that, like those in the algorithms of Anstreicher (1986, 1989) and Todd (1992),
approach feasibility and optimality simultaneously. However, unlike Ans-
treicher's and Todd's routines, their method does not explicitly incorporate an
artificial variable in the constraint system.
The lower-bound updating procedure of de Ghellinck and Vial (which was
modified by Fraley) is based on solving n quadratic equations and their algori-
thm has its foundation in their 1987 feasibility method for solving a linear system
of homogeneous equations. A study by Todd and Wang (1993) which compares
search directions and lower-bound updates in the algorithms of Anstreicher (1986,
1989), Todd (1992), and deGhellinck and Vial (1986) concludes that Todd's
direction (obtained from his direction-finding subproblem) and Fraley's lower-
bound update have modest computational advantages over those of the others.
The version of the standard form projective algorithm advanced by Shaw
and Goldfarb (1991), using a constant, fixed steplength, achieves O( y'nL )-step
complexity rather than the O(nL)-step complexity originally demonstrated for
the projective algorithm. A modification of their approach which utilizes a line
search of the potential function on each iteration to accelerate convergence is
offered by Anstreicher (1993). In particular, Anstreicher further weakens the
weakened ball lower bound update of Shaw and Goldfarb so as to control the size
of the lower bound adjustment when the bound is updated and adds a single
simple constraint to the ball minimization problem on which the lower bounding
and direction finding logic of the algorithm is based. In this way Anstreicher
derives anew, strictly monotone (in the presence of the simple constraint) version
of the standard form projective algorithm which retains the O( y'nL)-step
complexity property.
An algorithm which is an analog of Karmarkar's algorithm and which

works directly in the space of the linear structural constraint inequalities AX ~ b
and not in the space of feasible solutions to AX = 0, l'X = 1, X ~ 0 has been
proposed by Freund (1988). The algorithm performs a simple projective transfor-
mation of the feasible region that maps the current iterate X to the analytic
center of the transformed feasible region. Freund's algorithm employs a set of
assumptions similar to those used by Karmarkar and also utilizes a potential
function to chart the progress of the algorithm.
Finally, given the primal-dual pair of problems (13.14) and (13.17.1)
(infra), Ye (1990c) presents, in the tradition of Karmarkar (1984), a class of
projective transformations under which potential functions (such as the primal-
dual potential function of Todd and Ye (1990)
4>(X,S) = p InX'S- 'E;=lln(xjsj),
with n::; p < + 00) are invariant and which leads to a new projective algorithm
which converges in O( In L) iterations. The algorithm is not central path
following and assumes only that: (a) the interiors of the feasible regions of the
primal and dual problems are nonemptYi and (b) the coefficient matrix A for the
primal structural constraints has full row rank m.
For r a positive integer and Xo€9G = {X€RnIAX=b, X~O} an initial
feasible solution, Ye defines a one-to-one projective transformation from 9G in Rn
onto 9(; in Rn +r as
T:X --+
~YI.nJ
Y = Yn:+I,
Yn +r
where
(n+r)XO IX
r+ l'X01X
contains the first n components of Y € R n +r ,
y. n+r
, l' J. = n+ 1, ... , n+r,
3 r+lXo- X
Xo = diag(Xo), and the inverse transformation is given by

X = T-I(y) = )(0 Y I : n
n+r .
I: j=n+I (Y/I')
Since l'y = n+l', %is contained in the (n+I')-simplex (J= {YERn+rl l'y=
n+r, Y ~ a}. In particular, T(Xo) = 1 (the center of (J).
If we write the primal objective function and structural constraint system as
C'X - z 2:: 0 (where z = z* is the minimal objective value for the primal problem)
and AX - b = 0 respectively, then under T above, the primal problem
corresponds to the program
min C(z)'Y s.t.
YE% = {YER n +r I AY = 0, l'y = n+l', Y ~ O},
where
,- z z
~,
C(z) = (CX o, -1', ... ,-1')'
A = (AXo,
- b
- 1""" b)
-1' .
(Note that C(z)'l = C'Xo - z.)

Setting Zo = b'Uo and So = C - A'U o > 0 we may form the ball optimiza-
tion problem
min C(z)'Y s.t.
AY 0
l'y n + l'
for which the minimal solution can be expressed as

p(z)
Y( z) = 1 - ,B p( z) I II'
where p(z) = PC(z) and P is the projection matrix onto the null-space of AY = 0,
l'y = O.
Let us rewrite p(z) as
p(z) C(z)-A'U(z)- C(z)'ll

n + I' ,
where U(z) = (A.A')-lAC(z). Then in terms of the coefficients and matrices of

the original primal and dual problems (13.14) and (13.17.1) respectively, p(z) can
be expressed as
Xo(C-A'U(z))
p(z)
(b'U(z) - z)/r C'Xu - z
n+r 1.
(b'U(z) - z)/r
On the basis of a potential reduction theorem, Ye (1990c, p. 462) establishes

the fact that the primal-dual potential function 4; can be reduced by a constant
by solving the above ball optimization problem on the interior of the primal and
dual feasible regions for arbitrary Xo' Uo' In addition, in the formal statement of
the algorithm (Ye (1990c, p. 463)), a line search is undertaken to minimize 4;,
i.e., at the kth iteration,
Y(Zk) = 1- ~P(zk)' ~ = arg min 4;(I-,8p(zk)' Sk)'

,B~o
This primal projective algorithm can start from any interior point Xu and a
strictly lower bound Zo < z* and its performance IS such that, if
4;(Xo' So) :::; O( Vii L), then it terminates in O( Vii L) iterations, with each
iteration requiring 0(n3) arithmetic operations.
13.9. Methods Related to Karmarkar's Routine:

Affine Scaling Algorithms
The following collection of solution techniques rely upon an affine rather
than a projective transformation at the start of each iteration. Moreover, they
operate directly on the problem in standard form and do not impose any a priori
restriction on the optimal value of the objective function.
Let us represent the standard form of a linear program as (13.14); its
corresponding dual problem may be written as
max b'U s.t.

(13.17)
Interior Point lVlethods 369
Given these primal and dual structures, we first consider what is called the
primal affine scaling algorithm. (Arbel (1993), Asic, et al. (1990), Barnes (1986),
Barnes, et al. (1988), Vanderbei, et al. (1986), and Vanderbei and Lagarias
(1990)). This variant of Karmarkar's algorithm: (1) is formulated within the
positive orthant rather than within the unit simplex; (2) assumes both primal
and dual nonelegeneracy;5 (3) assumes that (13.14) is bounded and feasible; (4)
employs a recentereel projected graelient search for the optimum; and (5) assumes
the existence of an interior feasible point (X k ), e.g., a point X k > 0 which
satisfies AX = h.
As we shall soon see, the primal affine scaling algorithm starts with an
interior feasible point (say Xo) in the original or true space, centers it using a scal-
ing operation, takes a step from the centereel point Yo in the direction of steepest
descent in the scaled space, and obtains a new point (call it Y 1 ) which is then
transformed back to a point Xl in the original space via a rescaling operation.
To this end let X k be an interior feasible point for (13.14). If Dk represents
an nth order diagonal matrix containing the components of X k, then these
components are scaled under the affine transformation T: X ---" Y, where
Y = T(X) = Dk"lX.
(Note that T- 1 : Y ---"X yields X = DkY.) Clearly Y k = Dk"lXk = 1. Under
this change of units the seal eel vector Y k is placed equidistant from the walls of
the polytope AX = b, X ~ O. (Note that Dk" 1 exists since X k > 0.) Hence the
scaled linear program appears as
min e'y s.t.
(13.18)
AY = b, Y~O,
where
e = DkC, A = ADk.
Under assumption (2) above, A has full row rank for every feasible X.
In order to obtain a steepest descent direction vector dk for the scaled
problem (13.18), let us first determine the orthogonal projection of e onto the
5The seminal work by Dikin (1967) which first explored an affine scaling procedure
assumed only primal nondegenracy.
null-space of A or
Cp = P,.C-- = [In - A'(AA,)-1 A]O
DI.IC - A'(AD%A,)-1 AD%C], 6 (13.19)
where the inverse of AA' exists since A has full row rank. If we let
Uk = (AD 2k A,)-1 AD2k C ( 13.20)
represent the m-vector of dual variables, then (13.19) can be rewritten as
(13.19.1)
where lk is the vector of reduced costs corresponding to X k • (Note that if C p ~ 0

for some feasible interior point of (13.14), then this problem is unbounded. Also,
if for some feasible interior point of the same we have C p = 0, then every feasible
point is optimal.) Then the direction of steepest descent in scaled space is
(13.21)
i.e., the direction opposite to that of the projected gradient C p , where the latter
is the projection of the scaled gradient 0 onto the null-space of the scaled
coefficient matrix A. If dk ~ 0, then Uk is dual feasible so that h'U k yields a
valid lowel' bound on the primal objective value for the transformed problem.
Given dk in scaled space, its counterpart in true or unscaled space is the
steepest descent direction
(13.22)
Taking a step in the d k direction yields the next iterate of X, namely
( 13.23)
where 0 < G:k € R is the steplength. To ensure that X k +1 ~ 0, G:k must be selected
so t hat Xjk+1 = Xj+G:k
k d jk ~ 0,J=
. 1, ... ,n, or
G:k = mJn. { -Xjk / . 1, ... ,n,

dkj' dkj< 0,J= } (13.24)
6See Appendix C to this chapter regarding the derivation of this expression.

where d~ is the /h component of the n-vector d k .

While (13.24) guarantees primal feasibility, it also forces at least one
component of X k +1 to vanish at the boundary of the constraint polytope. When
this happens X k +1 is not an interior point of the said polytope. So to guarantee
that X k +1 represents an interior point of the constraint polytope, we need only
take some fraction r (typically r = 0.95) of the steplength Gk' z.e., (13.23) is
modified as
(13.23.1)
At this point two comments concerning the steepest descent direction d k are
= AXk + rGkAdk = b or
in order. First, d k is in the null space of A since AXk+1
Adk = O. Second, d k indeed takes us in a descent direction since C'd k = C'Dkd k
- C'PrC = - C'P~C = -P>rCII2 ~ O.
What about the determination of the dual variables Uk given by (13.20)?
. It can be shown that they are the solution of the normal equations
(AD~A')Uk = AD~C
for the weighted or scaled least squares problem
min" Dk(C-A'U k) 112.

Uk
For this problem the vector of residuals is the reduced cost vector rk given in
(13.19.1).
Let us next consider the limiting behavior of the primal affine scaling
algorithm. The limit points of the sequence X k , k=O, 1, ... generated by (13.23.1)
belong to the set'! = {X I X ~ 0, AX = b, Dkrk = O}. Under the assumption
that A has full row rank and rk has at most m zero components, it follows that '!
is the set of all vertices of the feasible region. And since Dkrk = 0, '! consists of
those feasible points for which complementary slackness holds, i.e., x~r~ = 0,
j=l, ... , n. Based upon these considerations, it can be shown (Vanderbei, et al.
(1986, pp. 399-402)) that
lim Dx r(Xk )
k-+oo k
o.
Moreover, for c:r = {X I X ~ 0, AX = b, rk ~ O} the set of points for which dual

feasibility holds, it follows that if X € 1 n c:r, then X is optimal. Finally, under
this characterization of optimality, it can also be shown that the sequence {Xk }
is convergent with Xoo = lim X k optimal.
k-+oo
We may summarize the sequence of steps involved in executing the primal
affine scaling algorithm as follows:
13.5 PRIMAL AFFINE SCALING ALGORITHM.
The algorithm generates a sequence of points Xo, Xl"'" X k from the steps:
STEP O. Initialize.
Choose Xo > 0 as a feasible interior solution and set X k = Xo.
STEP 1. Find the vector of dual variables.
The dual vector Uk is the solution to the system of normal
equations
(AD~A')Uk = AD~C,
where Dk diag(xt, ... , x~).
STEP 2. Find the vector of reduced costs.
The reduced cost vector is rk = C - A'U k'
If rk ~ 0, then Uk is dual feasible.
STEP 3. Find the direction of steepest descent in true space.
The vector of steepest descent is
d k = -D~rk'
If d k > 0, stop; the original primal problem is unbounded.
STEP 4. Compute the next point in the sequence.
The new point update in true space is
X k+1 = X k + rUkdk' 0 < r < 1,
where
Uk = mjn{ - x~ / d~, d~ < 0, j=l, ... ,n}.
(Alternatively, set rUk = r / II d k II. This choice corresponds to a
descent step that moves a proportion r of the way to the
boundary of a sphere inscribed within the non-negative orthant
and centered at Yk = 1 in scaled space.)

The sufficient optimality criterion informs us that C'X k = h'U k
holds only at optimality while the weak duality theorem states
that the dual objective value provides us with a lower bound on
the primal objective value (since we are minimizing in the primal
problem), with C'Xk - h'Uk serving as the duality gap. So for a
small prespecified tolerance level 0 < E E R, the iterations stop
when the relative duality gap
Otherwise, proceed to
Return to step 1.
We end this presentation of primal affine scaling with a few comments

pertaining to the practical implementation of the primal affine scaling algorithm
along with a discussion on how it compares with the standard simplex method.
First, any unrestricted primal variable is obviously excluded from the ratio
test given in (13.24). If any such variable turns negative on any iteration of the
primal affine scaling routine, simply negate its current value to make it positive.
This will ensure that Dk has a strictly positive diagonal. In addition we also
negate its associated column in A and component in C. The algorithm is then
continued.
The second point deals with starting the primal affine scaling algorithm.
Given an interior point Xo > 0 which does not satisfy AX = h, let us form the
residual vector h - AXo and include it as an n + 1st column within the structural
constraint matrix so as to obtain
AX = [A: b - AXo] rlXn+l

X J = h,
where x n +1 is a nonnegative artificial variable corresponding to the last column

of A. If if, = (Xb, 1), then

AXa = [A: b-AXo][ 7]= b.
Hence Xo serves as a new initial feasible solution vector to AX = b given any
Xo> O. Under these changes the modified primal problem is
min(C, M )'-
X s.t.
AX=b, X~O,
where M is a large positive constant (penalty). Since M can be infinitely large,
we are assured that xn+1 vanishes at the optimum given that the original primal
problem is feasible. If xn+1 does not vanish at optimality, then the original
primal problem is infeasible.
Recently Ishihara and Kojima (1993) presented a method for updating M in
an affine scaling algorithm which is essentially a variant of the technique
proposed by Kojima, Mizuno, and Yoshise (1993) for implementing primal-dual
interior-point algorithms. The method:
(i) detects whether M should be increased;
(ii) allows M to be increased (if warranted) during an iteration of the
affine scaling algorithm; and
(iii) provides information on the possible infeasibility of the primal and
dual programs as M increases in value.
An interesting set of parallels with, and differences between, the primal
affine scaling method and the simplex method has been highlighted by Vander-
bei, et al. (1986). Specifically:
1. The simplex algorithm starts from an initial feasible vertex X' = (XB, 0)
while the affine scaling algorithm starts from a strictly feasible interior
solution X> O.
2. Both algorithms calculate the vector of reduced costs as r = C - A'U
= C- A'(AD 2 A,)-l AD 2 C. Furthermore, for the simplex algorithm,
3. The simplex algorithm stopping criterion for detecting an optimal X is

S ~ 0; for the affine scaling algorithm no exact stopping rule exists.
4. Both algorithms use r to find a direction d within the null-space of A
that will decrease the objective function. For the simplex algorithm the
decision rule is to choose i such that Sj < 0 and find
for the affine scaling algorithm we choose the steepest descent direction
d = -D2r.
5. Both algorithms move from X to X= X + (-&) d, where:
(i) for the simplex algorithm l' = 1, a = mqxt
{sdXj,Xj > O}, and X is
a new feasible vertex; and;
(ii) for the affine scaling algorithm, 0 < r < 1, a = mqx{sdxj}, and
z
X is a new feasible interior point.
An important mode of implementation of the primal affine scaling
technique is to apply it to the dual of (13.14) (see (13.17)) or to
max h/U s.t.

(13.17.1)
A/U + S = C, U unrestricted, S ~ 0,
where S is an (n xl) vector of dual slack variables (the reduced cost vector) and
A is of full row rank m. The resulting computational routine has become known
as the dual affine scaling algoritlLm. (See, for instance, Arbel (1993), Marsten, et
al. (1989), and Monma and Morton (1987).)
Let D, S represent an initial feasible pair of solution vectors to (13.17.1),
i.e., S 2:: 0, A'D + S = C. For S> 0 we may apply an affine scaling transforma-
tion to Sk so as to obtain tk = Dk1S k , where Dk = diag(sl' ... ' sn). Hence
(13.17.1) becomes
maxb/U k s.t.
(13.17.2)
Dkl A/U k + tk = Dk1C, Uk unrestricted, tk 2:: O.
Clearly U, t are feasible for this program given that U, S are feasible in (13.17.1).
If we multiply both sides of the constraint equality in (13.17.2)
by AD k l , then we may solve for
(13.25)
where Rk = (ADk 2A')-lADk l . Upon substituting (13.25) into (13.17.2) we may

eliminate Uk so that the dual problem can be rewritten solely in terms of tk as
max b'U k min(-b'U k )

min(b'Rktk - b'RkDklC) s.t. (13.17.3)
Rktk RkDklC, tk ~ 0,
where Rk = In - DklA'Rk. Since (13.17.3) is of the form (13.14), we may

apply the primal affine scaling algorithm to the former (with the constant term
in the objective function suppressed).
From (13.19), the orthogonal projection of b'Rk onto the null-space of Rk is
[III - Rk(RkR~J-l Rk] Rkb

(13.26)
Rkb = Dkl A'(AD k 2A,)-lb
. - ,
smce RkRk = O.
Let the new iterates for the dual problem be
Uk+! = Uk + Qkd~, Uk unrestricted
tk+! = tk + Qkd~, tk ~ 0, k=l, ... ,
where d~ € R m , d~ € R n represent direction vectors for the U, t variables
respectively and Qk depicts the maximum allowable step size. To maintain dual
feasibility, these iterates must satisfy the equality constraints in (13.17.2) or
Dkl A'(Uk+Qkd~) + tk + Qkd~ = DklC
and thus
Dkl A'd~ + d~ = O. ( 13.27)
Multiplying both sides of (13.27) by ADkl and solving for d~ yields
d~ = - (ADk2A')-lADkld~. (13.28)
Although (13.28) is valid for any direction vector d~, let us use the steepest
descent direction dt = -C p (( 13.26)) so that the preceding expression becomes
(13.29)
Given (13.29), the direction vector d~ can be obtained from (13.27) as
d tk -
-
_ D-1A'd
k
u
k' (13.30)
Applying the reverse affine transformation to dt yields a direction vector for the
original slack variables
d~ = DkdL = - A'df. (13.31)
Hence from (13.29), (13.31) we have a feasible direction vector d k
Our final set of iterates can thus be written as
Uk+ 1 = Uk + Q' kdf, Uk unrestricted
Sk+l = Sk + Q'kd~, Sk ~ 0, k=O, 1, ....
To preserve Sk ~ 0, let us choose Q'k in a fashion such that s~+l =
sZ + Q'kdkj 2 0, j=1, ... , n, z.e., according to the ratio test
( 13.32)
where d kj is the /h component of the n-vector df Moreover, by virtue of

(13.32) and an argument similar to that underlying (13.23.1), we may form the
modified S iterate as
Sk+l = Sk + rakd~, 0 < I" < l.
Finally, one can easily obtain an estimate of the primal vector X from the
current U, S iterates according to an appropriate modification of (13.22), i.e.,
Xk+l = - Dk2d~ = - Dk 2 ( -A'df) = Dk 2 A'(ADk 2 A,)-lb.
Clearly AXk +1 = b. If, in addition, X k +1 ~ 0, then we have a feasible solution
to the primal problem. Moreover C'X k+ 1 then provides us with a valid lower
bound on the optimal objective value for the dual.
Let us now consolidate our results on the dual affine scaling routine by
formally stating the:
13.6. DUAL AFFINE SCALING ALGORITHM.

The algorithm generates a sequence of pairs of points (Uo,So)' (U 1 ,SI)' ... '
(Uk' Sk) from the steps:
STEP O. Initialize.
Choose Uo, So > 0 as a feasible interior solution and set Uk = Uo,
Sk = So·
STEP 1. Find the direction vectors for the dual variables.
a. The direction vector d~ is the solution to the system
of normal equations
where Dk = diag(s~, ... , s~).

b. The direction vector d~ is determined from
dks -- - A'dU
k·
If d~ > 0, stop; the dual problem is unbounded and thus the

primal problem has no feasible solution.
STEP 2. Compute the next pair of points in the sequence.
The new point updates are
Uk+1 = Uk + rUkd~,
Sk+1 = Sk + rUkd~,
where
uk = mjn{ -s1 / dkj,d kj < 0, j=1, ... ,n}, 0 < r < 1.
STEP 3. Determine the primal solution.
The current primal variable estimates corresponding to the latest
dual variable values are found using
X k +1 = - Dk" 2dr
If d~ ~ 0, then X k +1 is feasible for the primal problem and thus
C'Xk +1 serves as an upper bound on the optimal dual objective
value.
For 0 ~ E € R a small prespecified tolerance level, stop when the
relative improvement in the dual objective
Return to step 1.
The preceding dual affine scaling algorithm has been extended by Marsten,
et al. (1989) to the case where the variables are free as well as subject to simple
upper bound constraints. In addition, these authors provide a discussion on ways
of reducing problem size and on the effects of scaling on the performance of the
algorithm.
We assumed at the outset of our discussion of the primal affine scaling
algorithm that both primal and dual nondegeneracy must hold. However, new
convergence results for affine scaling offered by Tseng and Luo (1992) and
Tsuchiya (1990) do not require any nondegeneracy assumption. Specifically, it is
demonstrated that affine scaling methods for linear programs are globally conver-
gent without either primal or dual nondegeneracy assumptions if some restric-
tions are imposed on stepsize, e.g., Tseng and Luo assume a very small stepsize
of order 2- 0 (£) while Tsuchiya requires that the stepsize be about one-
eighth of the maximum allowable distance.

A Path-Following Algorithm
In this section we examine a primal-dual interior-point algorithm which, at
each iteration, preserves both primal and dual feasibility while following the so-
called central path toward optimality. Along the said path the algorithm moves
us in a set of directions which successively reduce the duality gap so that, in the
limit, as the duality gap is closed and optimality is approached, complementary
slackness holds. The process converges in O( ylnL) iterations and O( n3 L)
arithmetic operations.
As will be demonstrated below, the path-following algorithm uses a
projected Newton method to trace out the central path of minimizers for the
logarithmic barrier family of subproblems. Here the central path amounts to the
classical barrier path under the restriction of primal-dual complementarity (Arbel
(1993), Gill, et al. (1986), McShane, et al. (1989), Megiddo (1986, 1989b),
Kojima, et al. (1989b), and Monteiro and Adler (1987)).
Our starting point is the set of primal and dual problems given by (13.14)
and (13.17.1) respectively, where, as before, it is assumed that A has full row
rank m. We may reformulate these problems by replacing the nonnegativity
conditions within each by an appropriate barrier term so as to obtain:
Primal Barrier Problem
min C'X - J.L E ;=11n(x j) s.t.
(13.33)
AX= b, J.L >0.
Dual Barrier Problem

max b'U + J.L E ;=11n(s j) s.t. (13.34)
A'U + S = C, J.L > 0,
where J.L(usually small and positive) is the barrier parameter. Here the objectives
in (13.33), (13.34) depict logarithmic barrier functions while the barrier term, say
E ;=l ln(x j)' keeps the solution centered within the feasible polytope and away
from the boundary of the same, where a singularity occurs when x j = 0,
j=l, ... ,n, i.e., as some :1:;--+0, In (:l:j)-oo so that an infinite penalty is imposed
for departing from the central path of interior solutions. At each iteration of the
path-following algorithm (13.33), (13.34) represent individual subproblems that
must be solved and where an updated J.L is specified for each new subproblem.
For a given J.L and for feasible interior vectors X > 0, U, and
reduced costs S > 0, the Lagrangians for (13.33), (13.34) are respectively
(13.35)
(13.36)
Looking to the Karush-Kuhn-Tucker first-order conditions for an optimum

we have, from (13.35),

(a) aJfjax C - j.tX- I l- A'U 0,
(13.37)
(b) alfjau b-AX = 0;
and from (13.36),
(a) alPjaX = C-A'U-S = 0,
(b) alP jau b-AX = ° (13.38)
(c) alP jas = j.tS-Il- X = 0,

where X, S are the nth order diagonal matrices
(Note that X Xl, S = Sl and that X-I and S-I exist for interior X and S
respectively. ) Since the objective function in (13.33)(13.34)) is
convex (concave), subproblem (13.33)((13.34)) has a unique global minimum
(maximum) which is completely determined by (13.37)( (13.38)).
Given the same j.t in both (13.37) and (13.38), a substitution of C from
(l3.38.a) into (13.37.a) yields XSI = j.tl or (13.38.c). In this regard, the system
of equations characterizing the optimality for the primal-dual pair of problems
with the same j.t is
(a) XSI = j.tl or XjSj = j.t, j=l, ... ,n,

(b) AX = b, (13.39)
(c) A'U + S = C.
Here (13.39.a) depicts the JJ-complementarity relation; (13.39.a, b) assure primal

feasibility; (13.39.a,c) assure dual feasibility; and, as j.t -+ 0, (13.39) itself
guarantees optimality for the original primal and dual problems. (In fact, given
that X> 0, U, and S> ° are feasible interior solutions to (13.33), (l3.34) above,
(13.39) provides a necessary and sufficient condition for each of these sub-
problems to attain an optimal solution.) Moreover, if either the primal or dual

feasible region is bounded and has a nonempty interior, then (13.39) has a unique
solution for X ~ 0, S ~ O.
Based upon the preceding discussion we can now offer a more precise
definition of the central path or trajectory. For a fixed J.L, we can solve (13.39)
for X, U, and S in terms of J.L. In fact, for each such J.L , we generate an entire
family of solutions depending on J.L or X(J.L), U(J.L), and S(J.L). Let
:f = {X I AX = h, X> O} € Rn,
~ = {(U, S) I A'U + S = C, S > O} € R m x Rn , and
Then the central path r(J.L) is defined as the set of vectors X(J.L), U(J.L), and S(J.L)
satisfying (13.39) or
(13.40)
As J.L -7 0, r leads to a pair of optimal solutions X*, (U*, S*) to the primal and
dual problems respectively which satisfy, in the limit, the complementary
slackness conditions xjsj = 0, j=1, ... , n. Thus optimal solutions to the primal
and dual problems can be approximated by tracing the trajectory r(J.L) until the
parameter J.L becomes sufficiently small.
The following interior-point primal-dual algorithm traces out r(J.L) on its
route to optimality and thus constitutes a path-following routine. Furthermore, if
W(J.L) € r(J.L), then the gap between the primal objective value at X(J.L) € :f(J.L) and
the dual objective value at (U(J.L), S(J.L)) € ~(J.L) is
C'X(J.L) - X(J.L)' A'U(J.L)

X(J.L)'S(J.L) = l'X(J.L)'S(J.L)l
= l'(J.Ll) = 7lJ.L. (13.41)
To initiate the (central) path-following algorithm we need a set of feasible

interior solutions (X,U,S)€:fx~ (i.e., (13.39.b,c) are satisfied but (13.39.a) is
not). Following Kojima, et al. (1989b) and McShane, et al. (1989), let us derive
the (projected) Newton direction of (13.39) at (X, U,S).7 That is, for a given J.L,
we seek to determine a set of step directions (corrections to the current
estimates) d X , d U , and dS which, for small movements along them, preserve
feasibility while reducing the errors in (13.39.a). The resulting directions will
yield a new solution point which satisfies (13.39).
From (13.42), with V(J.L) = J.Ll- XSl, we obtain
[!o ~ !][:~] [V~)j !~~ +Xds V(J.L)
= or o (13.43)
A' In dS 0 A'dU + d S O.
Since the solution to (13.43) depends linearly upon J.L, we may express the same as
[S-l - S-lXA'(AS- 1XA')AS- 1j V(J.L).

- (AS- 1XA,)-1 AS- 1 V(J.L), (13.44)
A'(AS- 1XA,)-1 AS- 1 V(J.L)
or
- (AS- 1XA,)-1 AS- 1 V(J.L),
- A'dU(J.L), (13.44.1)
S-lV(J.L) - S-lXds(J.L).
7To employ Newton's method, let us rewrite (13.39) as the vector-valued

function
T~t ].
In addition, let (X k , Uk' Sk) be a given point. By Taylor's expansion,
F(X, U, S) ::::: F(Xk' Uk' Sk) + J ~Xk' Uk' Sk) dk ,
wh're J F J~obi= =~:' ali ~2{i;r

i, th'
Hence the original equations may be approximated by
o. (13.42)
Now, let us consider the diagonal matrix
To rationalize the role of D, let us note that we may view X as the rescaling
matrix for primal space while 8-1 has the same function in dual space. For the
combined primal-dual algorithm, D is the common rescaling matrix (the
geometric mean of X and 8-1) that is applied simultaneously to the primal and
dual spaces (Kojima, et al. (1989)). In addition, let t(J.L) = (X8)1/21_ J.L(X8)1/2.
Finally, let P = DA'(AD2A,)-IAD€R n be the orthogonal projection matrix onto
the linear subspace .Ab = {YIDA'U = Y} while In - P depicts the projection
matrix operator orthogonal to.Ab. Then the projected Newton step directions
can be expressed as
dX(J.L) D(In-P) t(J.L),
dU(J.L) - (AD2 A,)-1 ADt(J.L) , (13.45)
dS(J.L) D- 1 Pt(J.L).
Given this set of direction vectors, the new iterates can be determined, for a
given p, as
u U + o:DdU, U unrestricted, ( 13.46)

S= S + o:DdS, S ~ 0,
where o:P,o:D must be selected so that X;:::- 0 and S;:::

-
0 or
(a) o:P = mjn{ -xj/dxj' dXj < O,j=I, ... ,n}.

(13.47)
(b) o:D = min{ -sj/dsj,ds j < O,j=I, ... ,n},
J
where dXj,ds j are the /h components of the n-vectors dX,d S respectively. And
if we again employ an argument similar to that underlying (13.23.1), then we
may form the modified X, S iterates as
P X
X = X + ro: d ,0 < r < 1,
~
D S
S = S + ro: d , 0 < r < 1.
~
We may summarize the sequence of steps involved in executing the primal-

dual path-following algorithm as follows:
13.7. PRIMAL-DUAL PATH-FOLLOWING ALGORITHM.
The algorithm generates a sequence of primal-dual triples (Xo, Uo, So),
(Xl' U1 , Sl)' ... , (X k, Uk' Sk) from the steps:
STEP O. Initialize.
Choose Xo > 0, Uo, So > 0 as a feasible interior solution and set
X k = Xo, Uk = Uo, Sk = So·
STEP 1. Find the primal-dual scaling matrix.
Define
- = dzag
Xk k k) ,
. (x1""'Xn
S-l = diag( 1/ st, ... , 1/ s~)
so that
STEP 2. Evaluate the barrier parameter JJ and the vector V(JJ). Let
C'X k - b'U k
Jl = n
V(Jl) = Jll - XS1.
STEP 3. Find the direction vectors for the primal-dual set of variables.
a. The direction vector df is the solution to the system of
normal equations
(AD~A') df = - AS- 1 V(Jl)
for the weighted least squares problem
minll Dk(A'df +X- 1 V(Jl)) f

df
b. The direction vector d~ is determined from
d~ = - A'df.
c. The direction vector dJ is determined as
STEP 4. Compute the next primal-dual triple in the sequence.

The new primal-dual variable updates are
X k +! X k + ro:fd{,
Uk+! Uk + rofdf,
Sk+! = Sk+rafd~,
where
af = mr{ -x~/dx~, dx~ < O,j=l, ...,n},
af mjn{-s~/ds~, ds~ < O,j=l, ...,n},
O<r<1.
For 0 < E € R a small prespecified tolerance level, stop when the
relative duality gap
Return to Step 1.
A study which offers a comprehensive implementation of this primal-dual
algorithm has been provided by Lustig, Marsten, and Shanno (1991). Their
approach allows for the easy handling of simple bounds on the primal variables
and incorporates free variables into the calculations. These authors discuss in
detail a variety of computational issues (e.g., the role of barrier parameter, effects
of preprocessing, scaling, removing variables which appear to be approaching zero
as their optimal value, and reducing infeasibility by altering slack variables
whenever possible) and provides an extensive set of numerical results for a variety
of test problems.
At step 0 of the above algorithm we must find an (X, U, S) € 'W so that the
algorithm can be initialized. To this end let (Xo, Uo, So) be an initial point
satisfying Xo > 0 and So> O. In addition, let x n +1 ,x n +2 ~ 0 represent primal
artificial variables while Um+I' Sn+I' Sn+2 ~ 0 are to serve as dual artificial
variables. Then corresponding to the primal and dual problems (13.14), (13.17.1)
respectively let us form the following artificial primal and dual programs
(Kojima, et al. (1989b); McShane, et al. (1989)):
AX + (b-AXo)xn+l b
(A'Uo+So-C)'X (13.48)
where cn+I,bm+I are taken to be sufficiently large positive numbers; and
A'U + (A'Uo+So-C)um+I + S C
(b-AXo)'U + sn+I (13.49)
U m+1 + sn+2 0
U,S ~ 0; u m+I,sn+I,sn+2 ~ O.
If cn+l! bm+I are chosen so that
(a) bm+I > (A'Uo+So-C)'Xo, (13.50)

(b) cn+I > (b-AXo)'Uo
(here X = Xo, U = Uo), then Xo' x~+I' and X~+2 along with Uo' u~+I' So' s~+I
and s~+2 are feasible solutions to (13.48), (13.49) respectively, where
X~+l 1
x~+2 bm +1 - (A'Uo+So-C)'Xo,
u~+I = -1,
s~+I cn+I - (b-AXo)'Uo,
s~+2 1.
The preceding algorithm can then be applied to the artificial problems (13.48),
(13.49) using these initial feasible solutions.
Let X* and (U*, S*) represent optimal solutions to the original primal and
dual problems (13.14), (13.17.1) respectively. The following theorem provides us
with a sufficient condition on the values chosen for bm +1 , cn +1 in order for the
algorithm applied to (13.48), (13.49) to render approximate solutions to (13.14),
(13.17.1). Specifically,
13.8. THEOREM [Kojima, et al. (1989b)]. If (13.50.a, b) are satisfied and,
in addition,
bm +1 > (A'Uo+So-C)'X*,
c n +1 > (b-AXoYU*,
then
(i) a feasible solution (X, xn +1 ,xn +2) to (13.48) is minimal if and
only if X is a minimal solution to (13.14) with xn +1 = OJ
(ii) a feasible solution (U,u m+1 ,s,Sn+1,Sn+2) to (13.49) is maximal
if and only if (U,S) is a maximal solution to (13.17.1) with
um +1 = O.
In practice, McShane, et al. (1989) choose
c n +1 = cn2maxlCjl,
bm +1 = bn2maxlbd,
where c, b = 3 or 30. Both can be shown to satisfy the conditions for equivalence
of the original and artificial primal and dual problems.
We next turn to an additional path-following interior-point algorithm,
namely the large step predictor-corrector algorithm of Mizuno, et al. (1993b) (see
also Sonnevend, Stoer, and Zhao (1989) and Ye, et al. (1993a)). Starting with a
primal-dual pair of problems of the form (13.14), (13.17.1) respectively, let
%++ = {(X, S) 10< (X, S) E R 2n , AX = b, A'U + S = C for some U E Rm} =p <p
with the central path specified as r(fL) = {(X,S) I (X,S)E%++, XS = fLI} and the
duality gap for feasible (X, S) given by X'S = C'X - b'U, where X = diag(X).
The authors pose a path-following algorithm that generates a sequence of iter-
ates in a neighborhood 8(a) = {(X,S) IlIxs- fL111 ~ afL, fL = X'S/n, (X,S)E%++}
of the central path. The algorithm takes a single corrector step to the central
path after each predictor step in order to decrease J-L. In this regard, for a
constant 0 < {3 ~ t, the algorithm produces nearly centered pairs (X, S) € 8(2{3)
(sets of intermediate iterates) after a predictor step and very nearly centered
pairs (X, S) € 8({3) after a corrector step.
Looking to the motivation behind this predictor-corrector routine, suppose
we have a pair (X, S) € 8({3), 0 < (3 ~ t, where r(J-L) C 8({3) C 9G++, J-L = X'S/n,
and 'Y€[O,l]. Then we can generate search directions d X , dS, and d U via the
preceding primal-dual scaling method by solving the linear system
Sd X +Xds = 'YJ-Ll-XS
Ad x 0 (13.43.1 )
A'du +ds = 0,
with 'Y = O. Given a solution (dX,d s ) to (13.43.1),8 to undertake the kth

predictor step in the direction (d:,d~) from the current iterate (X,S) = (Xk,Sk)
to a point near the central path r(J-L), set
X( 0) = X k + Odt
S(O) = Sk + Bd~
and J-L(O) = X(O)'S(O)/n for some steplength B > O.
How should 0 be chosen? Following Mizuno, et al., we have
13.9. LEMMA rYe, et al. (1993)]. Let a€ (0,1). If there exists a positive Ok
such that
II X(O) S(O) - J-L(O)lll ~ aJ-L(O) for all 0 ~ 0 ~ Ok'
then (X(Ok)' S(Ok))€8(a).
Since the feasibility of (X(Ok)) is guaranteed by this lemma, let us choose the
Slf D =X1 / 2S- 1 / 2 and N(AD), R(DA') depict the null-space of AD and the range of
DA' respectively, then the directions d X , dS obtained from (13.43.1) with 1'=0 can be
expressed as
d x -- - DPN(AD)r
d S = - D- 1p R(AD,)r,
where r = ZI/21 and Z = diag(XS) (Ye, et al. (1993a)).

largest steplength Ok::; 1 such that IIX(Ok) S(Ok) - Jl(Ok)ll1::; exJl(Ok) is satisfied
for ex = 2{3, and let X k = X(Ok)' Sk = S(Ok)'
We now solve (13.43.1) with (X,S) = (Xk'sk) €6(2,8), Jl = XkSk/n, and
r = 1. This enables us to form
X kH = Xk + ~fX
SkH = Sk+ ds
with (XkH , SkH) € 6({3). This completes the corrector step.
It is evident that the large step predictor-corrector algorithm requires that
(13.43.1) be solved twice at each iteration. In addition, Ye, et ai. (1993a)
demonstrate that a lower bound for Ok can be obtained as follows: if Ok is the
largest Ok satisfying the conditions of lemma 13.9 with ex = 2,8, then
2
Ok>
-VI + 411 ,
c5 k ll/,8 +1
where 6k = dfdVJ.Lk and df = diag(df). Using this lower bound Ye, et ai. prove
that the large step predictor-corrector algorithm maintains D( JnL) iteration
complexity and also gives quadratic convergence of the duality gap sequence to
zero without assuming nondegeneracy or the convergence of the iteration sequence
{(Xk' SkH. In this regard, according to Ye, et al.: if {(Xk' SkH is the sequence
generated by the large step predictor-corrector algorithm with 0 < ,8 ::; ~ and
a = 2,8, then the algorithm has iteration complexity D(JnL) and XkSk~O Q-
quadratically, i.e., the duality gap sequence {XkS k } converges Q-quadratically to
zero if
We may now formally state the

13.10. LARGE STEP PREDICTOR-CORRECTOR ALGORITHM
STEP O. Initialize.
Given (Xo' So) € 6(~) with ~So::; 2L (here we assume integer
data), set k = O.
STEP 1. Predictor step.

Set (X,S) to (Xk,Sk)' 1=0, and compute d X , d S from (13.43.1).
Form
X(O) = X k + Od{
S(O) Sk + Od~
1'(0) X(O)'S(O)/n.
Compute the largest Ok so that (X(O), S(O» f o(!) for

0> 2 0<f.l<1
k - ';1 + 411c5k ll! f3 + 1 ' ,v - 4'
= d-sk dkS /I'k and -

where 15k d k = dzag(d k ).
S· S
Set Xk = X(Ok)' Sk = S(Ok)'

STEP 2. Corrector step.
. ~ - 1 -,-
Solve (13.43.1) with (X,S) = (X k,Sk)fO(2)' I' = XkSdn, and
1=1. Form ~ ~X
X k +1 = Xk+d
Sk+1 = Sk+ dS
with (Xk+1' Sk+1) f 6(~).
If X~Sk ~ T L, stop;
otherwise proceed to
Return to step 1.
A generic primal-dual interior-point algorithm which provides a general
framework for many existing primal-dual interior-point methods has been
developed by Kojima, Megiddo, and Mizuno (1993b). Their generic routine also
generates a sequence {(Xk' Uk,Sk)} C r, i.e., for each k the point (Xk' Uk,Sk) lies
on the central trajectory.
For the kth iterate (Xk' Uk,Sk)' let fk = XkSk/n. If (Xk' Uk,Sk) lies on
the central path r, then x~s~ = 1', i=I, ...,n, for some 1'. Hence fk = 1'. Since
nfk = XkS k coincides with the duality gap C'X k - b'Uk' it follows that
fk = XkSk/n = (C'Xk-b'Uk)/n.
We next consider the Newton direction (d X , dU , d S ) at (Xk' Uk' Sk) f r for

approximating a point (X(JL), U(JL), S(JL») with JL = f3fk on r(JL), where the
parameter f3 f [0,1]. In the Newton system (13.43) above we defined V(JL) =
JLl- XSl. If this expression is replaced by V(3(JL) = f3 fk l - XSl in (13.43), then
the direction (d X , dU , dS ) emerges as a unique solution of the system
V (3(JL)
o (13.43.2)
Finally, step lengths at, oP are chosen to generate a new iterate (Xk+I' Uk+I'
Sk+l) f 'W such that
Xk+I = X k + a P df,
Uk+I = Uk + aDd~, (13.46.1)
Sk+l = Sk + aDdf
It is evident that this generic primal-dual interior-point algorithm depends
upon three parameters: a search direction parameter f3 and primal and dual
steplength parameters a P , aD which determine steplengths in the primal and dual
spaces respectively. Given an initial solution (-'Co, UO, So) € 'W, if appropriate
values are assigned to these parameters at each successive iteration of the generic
method, we obtain a particular primal-dual interior-point algorithm.
The search direction parameter f3 fixes a target point (X(f3 fk), U(f3 fk),
S(f3 fk») f r which is to be approximated by the new iterate (Xk+I' Uk+I'
Sk+I) f'W. When f3 = 1, the target point (X(fk), U(fk), S(fk»)f r minimizes
the distance II XS - XkS k II from the current iterate (Xk' Uk' Sk) f'W to points
(X,U,S)€r. Hence the resulting Newton direction (dX,dU,d S ) = (d;,d~,d~)
may be regarded as a centering direction. At the other extreme, for f3 = 0, we
obtain the optimality condition (13.39) with JL = f3fk = 0. In this instance the
Newton direction (dX,dU,d S ) = (d;,d~, d~)from the current point (Xk' Uk,Sk) f'W
aims at optimal solutions X, (U, S) of the primal and dual problems respectively.
The direction (d;;,d~,d~) will be termed an affine scaling direction since it does
not involve any centering direction. In general, each direction (d X , dU , dS) with
f3 € [0, 1] can be expressed as a convex combination of the centering direction and
the affine scaling direction or
If the same step length a P = al( = a is chosen Jl1 the primal and dual
spaces, then the new duality gap can be written as
C'Xk+I - b'Uk+l
C'(Xk+ad X ) - b'(Uk+ad U )
(1- a(1-f3)) (C'Xk-b'U k)
(Kojima, Mizuno, and Yoshise (1989b)). Hence it is evident from this expression
that:
(i) we must have f3 :S 1 if the duality gap is not to increase; and
(ii) the smaller f3 and the larger a, the greater the reduction in the duality gap.
In this regard, while any f3 € [0,1] will satisfy (i) and (ii) above, since it is
required that (X k + 1 , Uk+l' Sk+l) € 'W, we must choose the step lengths a P ,
smaller than
aP = max{ a I X + ad x ~ o},
k
aD = max{ a I Sk + adS ~ o},

respectively. If we select a P = oP = a, then this common step length is bounded
above by min{~P ~D}
a ,a
max{ a I X + ad x ~ 0, Sk + ads ~ 0}-
k
It is known that the bounds aP, aD, and a depend upon the location of the
current iterate (X k , Uk' Sk') and the value of f3. In fact, computational
experience indicates that small values of f3 tend to increase the efficiency of the
generic primal-dual method. Moreover, a will tend to be larger when the current
point (X k , Uk' Sk) € 'W is near the central path r.
After surveying the assortment of values of the search direction and

steplength parameters actually selected by various researchers for the theoretical
development and practical implementation of primal-dual interior-point
algorithms, Kojima, Megiddo, and Mizuno offer two sets of values for controlling
p, OI P , and OlD in the generic primal-dual method. Their first rule (Rule G)
ensures global convergence while the second (Rule P) ensures polynomial-time
complexity. Both rules depend only on the steplengths r;P,r;D from the current
iterates to the boundaries of the primal and dual feasible regions respectively, and
allow taking large steps without performing any line search. (For full details see
Kojima, Megiddo, and Mizuno (1993b), pp. 6-8, along with their theorems 3.2
and 4.1, pp. 10-17.)
Let us now examine some important theoretical/computational consider-
ations regarding the preceding primal-dual interior-point methods. One such
issue involving the implementation of an interior point algorithm is the
preparation of an initial interior point. The usual approach (mentioned above) is
to embed the original problem to be solved into an artificial problem having a
known initial solution from which a primal-dual interior-point algorithm starts.
A drawback of this strategy is that the M-penalty constants must be sufficiently
large in order to ensure the equivalence of the original and artificial problems.
To avoid numerical instabilities and computational inefficiencies if the M's
are too large, and to circumvent the problem of no solution if they are too small,
Kojima, Mizuno, and Yoshise (199~) offer a theorem which makes it possible to
design flexible methods for updating M as the iterations of interior-point
algorithms progress. Their technique is framed in terms of an artificial self-dual
linear program with a single M-penalty that can be applied using practically
efficient primal-dual interior-point algorithms which have been developed to
handle a pair of primal-dual linear programs in standard form.
A strategy which avoids altogether having to construct an artificial primal-
dual set of programs is to admit an infeasibility, i.e., a primal-dual infeasible-
interior-point algorithm chooses a new point along the Newton direction
(d X , dU , d S ) towards a point on the central trajectory, but does not confine the
iterates to lie within the feasible region. While an infeasible-interior-point
algorithm exhibits the same basic structure as the primal-dual interior-point
algorithm (see (13.46), (13.47)), the generated sequence of iterates {(Xk' Uk' Sk)}
is not restricted to the interior of the feasible region. Thus an interior
(Xk' Uk' Sk) is not required to satisfy the equality constraints of the primal and
dual problems (AX = h, A'U + S = C); only the positivity restrictions X> 0,
S> 0 apply. Hence we can approach optimal solutions by moving throughout
the interior of the feasible region for the primal-dual pair of problems or along a
track outside of the same. In fact, this is a key advantage that an infeasible-
interior-point algorithm has over a primal-dual interior-point algorithm when
solving the primal-dual pair of problems (13.14), (13.17.1).
To exploit this advantage, Kojima, Megiddo, and Mizuno (1993a), using an
infeasible-interior-point algorithm, propose a steplength rule under which the
algorithm takes large distinct steplengths in the primal and dual spaces to ensure
its global convergence. Their algorithm finds approximate optimal solutions to
the primal-dual pair if both problems have interior points, and they detect
infeasibility when the sequence of iterates diverges. These authors also claim that
computation of the Newton direction at each iteration of the primal-dual interior-
point algorithm as applied to the artificial self-dual linear program utilized by
Kojima, Mizuno, and Yoshise (1993) is still a bit more expensive than that of the
infeasible-in terior-poin t routine.
Mizuno (1994) has proposed a G(nL) - iteration modification of the Kojima,
Megiddo, and Mizuno (1993a) infeasible-interior-point algorithm based on the
G( y'nL) - iteration interior-point predictor-corrector algorithm of Mizuno, Todd,
and Ye (1993b). The primal-dual pair of problems is solved without assuming
the existence of feasible or optimal points using a single steplength in primal and
dual spaces that is chosen to minimize the duality gap. Mizuno also demon-
strates that the Kojima, Megiddo, and Mizuno infeasible-interior-point method
terminates in G(n2 L) iterations by imposing certain conditions on the iterates.
In a similar vein, Potra (1994) (see also Mizuno (1992) and Potra (1992))
develop an alternative predictor-corrector method for solving linear programs
from infeasible starting points. (If a feasible starting point is used, the algorithm
reduces to the Mizuno, et al. (1993b) predictor-corrector method considered
above.) The new algorithm is well-defined in that at each iteration it generates a
positive (a-centered) primal and dual pair of solutions with feasibility and
optimality improved at the same rate. It is globally and quadratically
convergent when started from any positive primal-dual pair that is approxi-
mately centered given that an optimal primal-dual set of solutions exists. For
starting points large enough the algorithm has O(nL)-iteration complexity and,
under certain conditions, displays O( JnL) - iteration complexity.
Next, we may readily observe that many interior point methods for solving
linear programs solve a simultaneous linear equation system with coefficient
matrix A0 2A', where 0 is a diagonal matrix with strictly positive elements and
A is the coefficient matrix for the primal structural constraint system. To avoid
the computational difficulties arising when A is dense, Vandel'bei and Carpenter
(1993) propose solving instead the related indefinite system
l~2 ~'][ ~~] = [;].

In the context of a primal-dual path-following algorithm, Vanderbei and
Carpenter set
02 XS- I ,
r C - A'U - /lX-II,
P b-AX
and subsequently demonstrate that the strat,egy based on this system is actually
a generalization of the preceding methods involving AD2 N. At each iteration
only 0- 2 , r, and p change. Moreover, 0- 2 need not be diagonal; it must simply
be positive definite and symmetric.
Finally, Choi and Goldfarb (1993) address the issue of special structure of a
linear program within the context of polynomial-time path-following algorithms.
Interior Point l'vlethods 397
Specifically, Choi and Goldfarb consider a primal-dual path-following algorithm

that applies directly to a linear programming problem of the form
min{C'X I AX = h, HX ~ U, X ~ 0, X€ Rn}. Their algorithm handles, in a uni-
fied fashion, upper bounds generalized upper bounds, variable upper bounds, and
block diagonal structure.
It was mentioned in section 13.8 that Shaw and Goldfarb (1991) proposed a
standard form projective algorithm which achieves O( jiiL) step complexity.
However, as indicated in Anstreicher (1993), since the primal-dual pair of
solutions generated by Shaw and Goldfarb on each iteration satisfy an
approximate centering criterion, their algorithm can be strictly interpreted as a
path-following technique that uses a projective transformation on each iteration
rather than a simple affine rescaling of variables as utilized by the algorithms of
this section.
In this same tradition Shaw and Goldfarb (1994) present a new projective
interior point routine which actually follows a central trajectory to an optimal
solution of a linear program of the form
min C'X s.t.
AX = 0, xn = 1, X ~ 0, X€R n
in at most O( jiiL) iterations. The algorithm is identical to Anstreicher's (1986)

lower bound variant of Karmarkar's (1984) projective algorithm except that it
employs a weakened (larger) ball to update the lower bound on the optimal
objective function value and starts at a point near the central path. Although
the projective method developed by Shaw and Goldfarb is not derived using a
path-following framework (e.g., it does not first define the path of centers in the
usual path-following format as the path of local minimizers of a logarithmic
barrier function or potential function constrained by a set of linear equalities), it
operates, for the most part, in a fashion similar to traditional path-following
methods. In fact, the Shaw and Goldfarb results underscore the striking
similarity that exists between projective and path-following methods.

Potential Reduction Algorithms
Since the introduction of Karmarkar's projective algorithm and its
attendant potential function, several affine potential reduction algorithms have
been developed. These algorithms seek reductions in a potential function, like
Karmarkar's algorithm, but do so without resorting to the use of the projective
transformation. (For the primal case we have Gonzaga (1990); primal-dual
variants are provided by, for instance, Anstreicher and Bosch (1992), Freund
(1991a), Gonzaga (1991b), Gonzaga and Todd (1992), Kojima, et al. (1991,
1992), and Ye (1990a).)
Let us begin with the primal affine potential reduction algorithm introduced
by Gonzaga (1988, 1990). For a primal problem of the form
min C'X s.t. AX = b, X ~ 0, (13.14) )
let the feasible region % = {X I AX = b, X ~ O} in R n be bounded with %0:1 ¢.

Assume an initial feasible point Xo € %0 and a lower bound Zo on the optimal
value of the primal objective z* or Zo ~ z*.
To solve this primal problem using an interior point method Gonzaga uses,
for a given z ~ z*, a potential function f: R++ -R of the form
(13.51)
where q > n is fixed. An X € %0 with C'X - z ~ TL is taken to be an optimal

solution of the primal. Moreover, if for every iteration we find X k € %0 and
zk ~ z* such that
(13.52)
for a fixed 6> 0, then the primal problem is solved in O(qjn)L) iterations if
fzo(Xo) = O(nL). In this regard, we must specify a rule for determining Xk+1'
zk+1 such that (13.52) holds for an appropriate 6.
To this end let each iteration employ the affine scaling Y = (i:)-IX, where
X = diag(X). Then % is mapped to %x = {Y I AXY = b, Y ~ O} in R n , X k is
mapped to Yk = 1, and the scaled potential function 9 z: R++ - R has the form
gz(Y) = q log (C'XY-z) - I:i=l log Yi. ( 13.53)
For X k, X k+1 € 9(;0 the difference f z(Xk+1) - f z(X k) equals the difference
gz(Yk+1) - gz(Yk )· Moreover, for a fixed z,
'V gz(Y) = q XC - (y)-11

C'XY -z '
where Y = diag (Y). For Y = 1,
q -
'V gz(l) C'XI _ z XC - 1. (13.54)
For h the search direction and A the steplength, any interior point Y € 9(;~
X
can be expressed as> 0 and must satisfy AXh = 0 (since
Y = 1 + Ah
AX(I+Ah) = b). Also, h must be chosen so that 'V gz(l)'h < 0, i.e., the scaled
potential 9 z must decrease locally for movements in the h direction.
Given a fixed search direction h, we may choose the optimal steplength
A = ~ according to the following
13.11. THEOREM [Gonzaga (1990)]. Let h be a search direction
satisfying 'V gz(l)'h < o. Then
(13.55)
for any z ::; z*, where
~ ={- 'V gz(l)'h / 411hlf if - 'V gz(I)'hllhlloo/llhIl2::; 2;

1/211 h 1100 if- 'V gz(I)'hll hlloo/llhlf >2.
( Here II h 1100 = max {I hI I, .. ·,1 h I})
n
What about our choice of h? By virtue of the preceding theorem, we need

an h that satisfies 'V 9 Al)'h < 0 and which makes ('V 9 z(I)'h / II h II Ysuffi-
ciently large. To accomplish this let us select h = - PAX 'V 9z(I), where
PAX 'V gAl) is the projection of 'V g(l) onto the null space of AX and
PAX = In - X' A'(AX 2A,)-l AX is the projection matrix (Gonzaga (1990)). For
'V gz(I)'h = -lIhlf < 0, we obtain, from (13.55),
for any z:5 z*. Based upon this discussion we see that - PAX 'V g%(l) is the
appropriate search direction when I PAX 'V g%(l) II is sufficiently large; for
" PAX 'V g%(l) II too small, we need an improved lower bound z.
For z a lower bound satisfying z < z :5 z*, it can be shown that
,-
q log C Xl - z
-
<0 (13.56)
C'X1-z .
To exploit this relationship we use

13.12. LEMMA [Gonzaga (1990)]. For z:5 z* and 0 < E :5 1 let
I PAX 'V g%(l) II < E. Then
Z = C'ltl_~(l+~PAX~,(l)II)(C'ltl_Z) (13.57)
is a lower bound with

C'X1-z
C'Xl- z
Based upon this result, (13.56) becomes
where E has replaced II PAX 'V g%(1) I in (13.57) (Gonzaga (1990». So for
q > n + E.;n, g%(l) - gAl) < 0 and thus the scaled potential 9 decreases when z
(13.57») is used to update z.
We are now in a position to state Gonzaga's (1990, 1992)
13.13. PRIMAL POTENTIAL REDUCTION ALGORITHM.
STEP o. Initialize.
Choose Xo€xo, zO:5 z*, 0 < E :5 1, q > n + E .;n, and set k = O.
STEP 1. Determine the search direction.
a. Find the projection matrix
-, , -2 ,-1 -
PAXk = In - XkA (AXkA) AXk ,
where Xk = diag(X k ).
b. Find the projection of "gzk(1) onto the null- space ofAXk or

hk = - P AXk "gzk(l)
= - P AXk( c'xki - zk XkC-I)-

STEP 2. Line search.
Find
X k +1 = Xk(I+A*h k)
= X k + A*Xkh k
with
A* = argmln{gz/I+Ahk) P > 0, 1 + Ahk > O}-
STEP 3. Update the lower bound z.
a. If II hk II < E, compute
,- n(
zk+1 = C Xkl- q 1 + Ilhkll)
yin (C,-Xkl-Zk)i
b. Otherwise

If C'X k - zk ~ 2- L , stop;
Return to step 1.
Under the preceding solution scheme it can be demonstrated that for E =1
1
and q = 2n, the scaled (and the original) potential function decreases by 6 = at
each iteration and the algorithm converges to a minimal solution of the above
primal problem in O( 71L) iterations.
One of the shortcomings of Gonzaga's algorithm is that it requires prior
knowledge of the optimal primal objective value z*. An extension of Gonzaga's
algorithm which dispenses with this requirement has been provided by Freund
(1991a). Freund's approach is based only on primal affine scaling and the
computation of the projected gradient of the potential function
F(X, S) = q In(X'S) - E;=l ln xi' q = 71 + ,fii ,
where S is a vector of dual slack variables associated with the dual problem:
max h'U s.t. A'U + S = C, S ~ O. The algorithm computes dual variables and
takes either a primal step or recomputes feasible dual variables at each iteration.
Moreover, Freund demonstrates that primal iterates decrease the potential
function by at least 0.18 whereas the dual iterates decrease the potential function
by at least 0.2y1n, thus leading to an overall bound of O(nL) iterations.
Let the primal and dual problems again appear as
min C'X s.t. AX = h, X~O (13.14) )

and
maxh'U s.t. A'U +S = c, S~0 (13.17.1»)
respectively. It is assumed that the primal problem enjoys a strictly positive

feasible solution X > 0 and that the set of optimal solutions for the primal is
nonempty and bounded (and thus there exists a solution (U, S) to the dual with
S> 0).
For X > 0, let X = diag(X) so that Y = X-I X ~ O. Then the rescaled
primal and dual problems appear respectively as
minC'XY s.t. AXY h, Y~O, (13.58)
max b'U s.t. XA'U + t = XC, t = XS ~ O. (13.59)
Then X is feasible for (13.14) if and only if Y is feasible for (13.58) while (U, S) is
feasible for (13.17.1) if and only if (U, t) = (U, XS) is feasible for (13.59). And for
sets of primal and dual variables X, (U, S) respectively, the duality gap is simply
C'X = h'U = X'S.
Let us assume that X, (U, S) are primal and dual feasible solutions
respectively with X > O. Since scaling does not affect the potential function, take
X = 1. Then the current duality gap is l'S. In addition, the projection of
V X F(l,S) = (l?S'-l
onto the null space of A is
d(l'S) = [In - A'(AA,)-IA] [C~S) S -1].

Given the projected gradient d(I'S) = d(Ll), where Ll = l'S is an unknown

positive scalar, Freund demonstrates that, for q = n + In:
a. if II d( l'S) II ~ 0.8, we can decrease F(X, S) by 0.18 by taking a step in
the primal direction - d(I'S);
b. if Ild(l'S) II < 0.8, we can decrease F(X,S) by 0.2Jn by replacing S by
S = S(Li ), where Li € (0, I'S), and II d(Li ) II = 0.8.
These remarks thus lead us to Freund's (1991a)
13.14. EXTENDED PRIMAL POTENTIAL REDUCTION ALGORITHM
STEP o. Initialize.
Choose Xo > 0, (U o' So), q = n + In = 0.8, an optimality toler-
ance level j.l, and set k = o.
STEP 1. Test for Optimality.
If X~Sk :S j.l, stop.
STEP 2. Rescale.
Set Ak = AXk, <\ = xke, tk = XkS, where Xk = diag(X k ).
STEP 3. Compute the Projected Gradient Direction
Function.
d(Ll) = [InAk(AkAk)-l AkJ K~) tk - I}
Iflld(I'tk) II ~" go to step 5; otherwise go to step 4.
STEP 4. Recompute Dual Variables.

Solve for Li € (0, I't k) such that Li is the smallest Ll for which
IId(Li) II = ,. Set
and
Set Uk = '"
U k and Sk = X
--1'" d
k t, an go to step 1.
STEP 5. Take a step in Primal Variable Space.

Set d k = d(I't k) and Y k+1 = 1- 0.38d k llidd.
Set X k +1 = Xk Y k +1 , advance k to k+l, and return to step 1.
In terms of the implementation of this algorithm, Freund notes that:

1. After the algorithm leaves step 4, the next projected gradient direction
function d(L1) will be the same as the current direction function. Thus
these is no need to resolve the least squares problem in step 3 at the
next iteration.
2. In step 5 we could replace Y k+1 = 1-0.38k dk/II d k II by Yk+1 = I - O'd k ,
where 0' is determined by a line search that (inexactly) minimizes the
potential function F(I-O'd k , t k ) over the range 0' E (0.38/11 d k 11,00).
3. The choice of 2i in step 4 could be accomplished by a line search of L1
with the potential function F(I, (L1/q)[I+d(L1)]), where L1 E(0, I' tk).
4. The algorithm need not be started with a dual feasible solution. It need
only be started with a lower bound z on f. Hence one replaces d(L1) in
step 3 by
d(L1) = [In-Ak(AkAD-IAk][(~)Ck-ll
Also, I'tk replaced by C'X k - z in steps 3, 4, and 5. The algorithm
eventually computes feasible dual variables unless z is the optimal value
of the primal problem (13.14).
We next explore the interior-point algorithm offered by Ye (1990a) which
achieves O( Vii L)- iteration and O( n3 L )-ad thmetic-operation con vergence like the
preceding path-following algorithm 13.7 but avoids actually having to follow the
central path by simply relying upon potential reduction alone. Specifically, Ye
employs the primal-dual potential function described by Todd and Ye (1990) to
measure the solution's progress and develops an interior-point scaled-gradient
projection algorithm for a linear program in standard form which directly
minimizes the potential function using a line search. Moreover, no step
size restriction applies during the iteration process and the potential function is
reduced by a constant from any interior point.
Let us depict the primal-dual pair of problems (13.14), (13.17.1) as
a. b.
Primal Dual
minC'X s.t. max b'U s.t.
AX=b, X~O S = C-A'Y~O
where A is of order (m x n), S is an (n x 1) vector of slack variables, and it is

assumed that:
a. the relative interiors of the feasible regions for the above primal and dual
problems are nonempty and we have an interior feasible Xa, Uo for the
primal and dual problems respectively such that AXa = b, Xa > 0 and
So = C-A'Uo > O.
b. p(A) = mj
c. the primal and dual objective functions are not constant on their respec-
tive feasible regions.
Consider the following set of parametrized potential functions:
a. PRIMAL: ¢(x,~) = p In(C'X-~) - E'J=lln x j' where ~ = b'U is a para-
metric lower bound;
b. DUAL: ~(U, z) = p In(z-b'U) - E;=11nsj' (13.60)
where z = C'X is parametric upper boundj and
c. PRIMAL-DUAL:
¢(X, S) = pin (X'S) - Ej=lln (x jS j) = (p-n) In (X'S) - Ej=11n (x jS lX'S),
where!:5 z* :5 z(z* optimal), n :5 p < + 00, and p can be thought of as a search
direction parameter. Since X'S = X' (C-A'U) = c'x - b'U, (13.60.a, b) are related
to (13.60.c) as
a. ¢(X,!) ¢(X, S) + Ej=l In S jj (13.60.1)
b. ~(U, z) ¢(X, S) + Ej=1 In x j
with
V¢ = V x¢ = [p/(C'X - ~)] C- X- 11j
V~ = V s¢ = [p/(z-b'U)]b-AS- 1 1,
where X = diag(X), S = diag(S).

Based upon the inequality relating the geometric and the arithmetic mean,
it can be shown that if ¢ is reduced below - (p-n) L for a pair of positively
feasible X and S, then C'X - b'U = X'S S 2- L .
To determine the primal form of the affine potential reduction algorithm,
let ~o = b'Uo for some So =C =A'Uo > O. Then following Ye (1990a), we seek
to minimize the linearized primal potential function subject to the ellipsoid
constraint corresponding to the second-order term in a quadratic approximation
to the primal problem (13.14), i.e.,
min V ¢(Xo,~o)'(X-Xo) s.t.
A(X-Xo) = 0 (13.61)
I/eXo)-\X-Xo)I/s (3 < 1,
where X = diag (Xo).

Forming the Lagrangian function
J., = V ¢'(X-Xo) + ,,~( A(X-Xo)) + >'2((3 - (X-Xo)'(XO)-2(X-XO))
and eliminating "I' >'2 from the Karush-Kuhn-Tucker first-order conditions for a
minimum renders
(13.62)
where the constrained minimum for (13.61) occurs at Xl and P AXo =

In - XoA'(AX5A,)-1 AXo is the projection matrix. If we set p(~o)
PAXoXo V¢(Xo'~o)' then (13.62) can be rewritten as

- p(~o)
I
Xl = Xo - (3Xo p(!o) II' (13.62.1)
For computational purposes, p(~o) can be expressed as
( 13.63)
where
(13.64)
and
(13.65)
with
Ul (A~A')-l A~C,
U2 (A~A')-lb.
Based upon the preceding developments we have the following

13.15. POTENTIAL REDUCTION THEOREM [Ye (1990a)].
Let Xo, Uo be any interior feasible solutions for the primal and dual
pair of problems (13.14), (13.17.1) respectively and let p = n+...;n,
~o = b'Uo, Xl be given by (13.62.1), and Sl = S(~o) and Ul = U(~)
determined by (13.64), (13.65) respectively. Then either
¢(X1,So) :S ¢(Xo, So) - 6 or
¢(XO,Sl) :S ¢(Xo, So) - 6,
where 6 > 0.05.
This theorem thus establishes the notion that ¢ can be reduced by a constant if
(13.61) is solved on the interior of (13.14), (13.17.1) no matter how Xo, Uo are
selected.
We are now in a position to state Ye's (1990a)
13.16. PRIMAL AFFINE POTENTIAL REDUCTION ALGORITHM.
STEP O. Initialize.
Choose Xo > 0, So = C - A'Uo > 0 as a feasible interior solution
(AXo=b); let ~o = b'Uo and set k=O.
a. Find S(~k) = C - A'U(~k)' where
U(~d = Ul _ C'X~ - ~k U2
and
U1 = (AXiA,)-l AXiC,
U 2 = (AX~A')-lb.
b. Find
with
(3* arg min ¢(X k - (3X k P(zk)' Sk);
,I3~0 -
Sk+l = Sk and ;'k+l =;'k
b. Otherwise Sk+l = S(;.*) with
;.* = arg min ¢(X k , S(;.));

~ ~~k
X k +l = X k and ;'k+l = b'U(;.*).

If C'X k - b'U k < 2- L , stop;
STEP 4. Advance k to k+1.
Return to step 1.
A modification of this algorithm has us update the projection matrix as
follows. If we replace Xo in (13.62) by a positive diagonal matrix D such that
1/1.1::; dj/x~::; 1.1, j=l, ... ,n, then
p(;'o)
Xl = Xo - (3D II p(;'o) II '
where
P - -1
C'X -;'0 DS(;.o) - D(Xo) 1
D(XO)-lp(;.O)
and
(13.65.1)
Based upon these developments we now have Ye's (1990a)

13.17. MODIFIED PRIMAL AFFINE POTENTIAL REDUCTION

ALGORITHM.
STEP O. Initialize.
Choose Xo > 0, So = C - A'Uo as a feasible interior solution
(AXa=b); let ~o = b'Uo, D = Xu, and set a = 0.43, (3 = 0.25, and
k= o.
a. If d/xj ¢ [1/Ll,1.1], j=I, ... ,n, then dj=xj. Find
S(~k) = C - A'U(~k)' where now
C'X -z
U~k) = (AD 2 A,)-lAD(DC- ~ -k D (X k )-l1.
b. Find
Sk+l = Sk and ~k+l = ~k·

b. See step 2.b of algorithm 13.16.
STEP 3. See step 3 of algorithm 13.16.
STEP 4. See step 4 of algorithm 13.16.
The complexity performance of this modified algorithm results from the following
13.18. THEOREM [Ye(1990a)]. Let p = n+y'n and ¢(Xa, So)::; O(y'nL).
Then the modified primal potential reduction algorithm terminates in
O(y'nL) iterations and uses a total of O(n 3 L) arithmetic operations.
Let us now consider the dual variant of the affine potential reduction algo-
rithm (Ye (1990a)). For Xa > 0 (AXo=b) a feasible interior solution to (13.14)
and C'Xa = zo, let us minimize the linearized dual potential function subject to
the ellipsoid constraint corresponding to the second-order term in a quadratic

approximation to the dual problem (13.17.1), i.e.,
(13.66)
If UI denotes the minimal solution to the preceding problem, then
( 13.67)
where the direction vector do is defined as
(13.68)
and the vector of slack variables is given by
( 13.69)
Let
(13.70)
where
(13.71)
with
(SO)-2 A'(A(So)-2 A')-Ib,
- 1 - 1
(So)- P A(SO)-l(SO)- C,
and P - 1
A(So)-
Based upon this discussion we can now state Ye's (1990a)

13.19. DUAL AFFINE POTENTIAL REDUCTION ALGORITHM.
STEP O. Initialize.
Choose Xo > 0, So - C - A'U o > 0 as a feasible interior solution
(AXo=b); let zo = C'Uo and set k=O.

a. Find
where
b. Find
a. If II P(zk) II ~ min( Ja n';a2 ' 1 - a). then

Sk+l = Sk + /3*SkP(zk)
with
/3*
b. Otherwise X k +1 = X(z*) with
z* = arg min ¢(X(z), Sk)i

z ~zk
Sk+1 = Sk and zk+1 = C'X(z*).

If C'X k - b'U k < TL, stop;
STEP 4. Advance k to k+ 1.
Return to step 1.
The performance of this dual algorithm IS identical to that of the primal

potential reduction algorithm 13.16.
A primal algorithm similar to Ye's (1990a) potential reduction algorithm
13.16 has been proposed by Gonzaga (1992). According to Gonzaga, Ye's
method is really a type of path-following algorithm since it only updates the
lower bounds or dual variables at nearly central points. Anstreicher and Bosch
(1992), however, extend Ye's results by constructing an algorithm whose steps are
not restricted to remain approximately centered, i.e., long primal steps can be
taken. (These authors use a safeguard line search or sufficient decrease criterion
which controls the partial updating of the diagonal scaling matrix at each
iteration,i.e., prevents updating that is excessive relative to the potential
reduction obtained.) Moreover, Ye's dual algorithm 13.19 is actually equivalent
to his first algorithm 13.16, i.e., the dual update keeps no memory of Sk and thus
the dual algorithm should be viewed as a primal algorithm (Gonzaga (1992)).
A discussion on the near boundary behavior of the primal potential
reduction algorithm has been provided by Ye, et al. (1993b). These authors find
that for any iteration p can be chosen within an extremely wide interval
([ n +.J7i, +00») without compromising the convergence rate of O(.J7iL) itera-
tions. So as the iterate drifts away from the central path towards the boundary
of the positive orthant, p need not be set close to n +.J7i. This result is
consistent with the recent findings of Zhang, et al. (1992) concerning the conver-
gence rate of the algorithm. They conclude that it is desirable to use both a
large step size and a large p to attain a better local convergence rate.
In addition to providing an extension of Gonzaga's potential reduction
algorithm (13.13), Freund (1991a) also introduces an alternative form of Ye's
O(.J7iL) primal algorithm (13.16). This involves an extension of algorithm
13.14 by using in place of F(X, S) the primal-dual potential function
G(X, S) = q In (X'S) - l.: ;=1 In x j - l.: ;=1 In S j
with q = n +.J7i, where Sj is the lh component of the vector of dual slack

variables S.
To see exactly how Freund's (1991a) algorithm works, let X and (U, S) be
primal and dual feasible solution to (13.14), (13.17.1) respectively with X> O. If
(13.14) is rescaled so that X = 1, then the current duality gap is l'S. As before,
let us determine the projected gradient direction function d(Ll) as
and find del'S), the projected gradient of G(X, S) with respect to X at (X, S) =
(1, S). Then of II d(1'S) II ~ 0.22, we can achieve a constant decrease in the
potential function G(X, S) by taking a step in the direction - del'S) from the
current primal feasible point 1. That is, according to Freund, if q = n+Jn,
then:
a. If Ild(I'S) II ~ 0.22, we can decrease G(X,S) by 0.02 by taking a step in
the primal variables in the direction - del'S);
b. if II del'S) II < 0.22, we can decrease G(X, S) by 0.02 by replacing S by S
= Sea'), where ..1 € (0, I'S) satisfies II deLl) II = 0.22.
These considerations thus lead us to the following alternative form of Ye's
(1990a) algorithm for solving (13.14). To this end we state Freund's (1991a)
13.20. ALTERNATIVE FORM OF YE'S ALGORITHM (13.14).
STEP O. Initialize.
Choose Xo > 0, (Uo' So), q = n + In, 'Y = 0.22, an optimality
tolerance level JI., and set k = O.
If XkS k ~ Jl, stop.
STEP 2. Rescale.
Set Ak = AXk , Ck
STEP 3. Compute the Projected Gradient Direction Function.
d(L1) = [In-Ak(AkA~.)-lAk][(~)tk-ll
If II d(l'tk ) II ~ 'Y, go to step 5; otherwise go to step 4.
STEP 4. Recompute Dual Variables.

Solve for J € (0,1' t k ) such that J is the smallest L1 for which
IId(J)II='Y. Set
t =( f) (1 + d( J) ) and
U = (AkAk)-l A{Ck - ( f )1].
STEP 5. Take a step in Primal Variable Space.

1 dk
Set d k = d(l' t k ), Q' = 1- ~' and Y k +1 = 1- Q' Ildkij.
Set X k +1 = XkYk+1 , advance k to k+l, and return to step 1.
A glance back at Ye's algorithm (13.16) reveals that, except for the
specification for when to take primal versus dual steps at each iteration, the two
algorithms are identical. Here too Freund's modification at Ye's algorithm does
not explicitly require a condition that successive iterates lie near the central path.
In addition, both algorithms compute the same primal and/or dual projected
gradient directions at each iteration and both employ the same logic for when to
take a primal versus dual step at each iteration: either the projected gradient of
the potential function is sufficiently large, or the current iterate is sufficiently
centered. Thus the choice between either a primal or dual step depends only on
Ye's method has precipitated the development of other primal-dual

potential reduction routines with low complexity and which are not wedded to
following the central path, among which are the algorithms of Kojima, et at.
(1991) and Gonzaga and Todd (1992). Kojima, et at. offer an algorithm which
achieves the O(..;n L) step complexity of Ye's potential reduction method but
takes simultaneous primal and dual steps using a primal-dual scaling
D = (XS- 1)1/2, where X = diag(X k ), S = diag(Sk)' and Sk is a vector of current
dual slack variables. Gonzaga and Todd, however, develop a version of Ye's
algorithm which takes either a primal step based on a primal scaling D = X, or a
dual step based on a dual scaling D = S-l.
Looking to the Kojima, et al. (1991) primal-dual potential reduction
algorithm, let the primal and dual problems be given by (13.14), (13.17.1)
respectively or
Primal Dual
min C'X s.t. max b'U s.t.
AX=b, X~O A'U + S = C, S ~ O.
As Kojima, et al. demonstrate, the said algorithm is designed to reduce the

potential function by at least 0.2 in each iteration requiring O(n 3 ) arithmetic
operations. It starts from an initial feasible solution with the potential function
bounded by O( .,fii L) and generates, in at most O(.,fii L) iterations, an
approximate solution from which an exact solution can be computed in O(n3 )
arithmetic operations.
Let X++ = {(X,S) 10< (X,S) f R2n , AX = h, A'U + S = C for some
U ERm} 'f:. <p. To chart the progress of the primal and dual problems, Kojima, et
al. use the primal-dual potential function
!(X,S) = (n+.,fii) In (X'S) - l:;=lln(XjSj)'
Starting from a current point (Xk' Sk) f X++, an iteration of the algorithm deter-
mines a direction (df, d~), a steplength () > 0, and a new iterate (Xk+1' Sk+l) f X++
with (Xk+1' Sk+1) = (Xk' Sk) - (}(df,d~). In general, the direction vector
(dX,dU,d S ) is found by solving the lineal' system
o
o (13.72)
where X = diag(X), S = diag(S), D = (XS- 1)1/2, V' = (..,;;;rs;, ..., ~),

V = diag(V), and V = [( n + .,fii)/II V If]v - V-II. The unique solution to the
preceding system is given by
V
dX D(In-PAD) Ilvll
dU _ (AD2 A,)-1 AD V (13.73)
!lvll
dS D-1p V
ADllvll'
where PAD = DA'(AD2 A,)-l AD is the orthogonal projection matrix onto the
range of (AD)' or the subspace {(AD)'W I W f am} in an. (Note the similarity
between the directions given in (13.73) and those provided by (13.45).)
To ensure the feasibility of a new point (X k +1 , Sk+l)f%++, Kojima, et al.
impose on the direction (dX,dS ) and step parameter () the requirement that
(i) AdX = 0, A'dU +dS = 0; (13.74)

(ii) ()IIX-1dX!L ~ ()IIS-ldS!L ~
T, T,
where T f (0,1 ),() > 0, and II U 1100 = rnax{ I Uj I ,i=1, ... , n}. They then go on to
demonstrate that, for n;:::: 2, T = 0.4, and for directions (d X , dS) given by
(17.73), there always exists a () for which their algorithm generates a sequence
{(Xk' Sk) f %++} such that !(Xk+I' Sk+I) ~ f(Xk' Sk) - 0.2, [( = 1,2, ..., and
that one obtains an approximate solution (Xp' Sp) of the primal-dual pair of
problems for some p = O(ynL) such that X~Sp ~ 2- 2L , from which an exact
solution is reached in O( n 3 ) arithmetic operations.
Specifically, we have the Kojima, et al. (1991)
13.21. PRIMAL-DUAL POTENTIAL REDUCTION ALGORITHM.
STEP O. Initialize.
Choose (Xo,So) > °as a feasible initial solution.
Set k = 0 and let 6 = 0.2.
Compute the descent direction (dX,dS ) of (13.73).
Find
with
"8 = arg min !(X-()dX , S - ()d S ).
STEP 3. Check for optimality. ()
At each iteration
()(dx ,dS)
! ( (Xk,Sk) - - ) ~ f(Xk,Sk) - 6.
If X~Sk ~ 2- 2L for some k = O(.jii L), stop and compute an exact
solution; otherwise proceed to
STEP 4. Advance k to k+ 1.
Return to step 1.
As mentioned earlier, this algorithm does not invoke any proximity criterion in
that it does not have to follow the central path. Moreover, it performs at each
iteration a primal-dual affine scaling that does not take the present iterate to 1.
Kojima, et al. offer an interesting interpretation of the descent direction
- (d X , dS) in their algorithm. For a fixed (X, S) € [( ++, the real-valued function
h(dX,dS) = II D-1dX+Dd s II defines a norm in the linear subspace Tspanned by all
the feasible directions at (X, S), i.e, T = {(d X ,dS)€R2n I AdX=O, A'du+dS=O}.
Then the direction - (dX,d S) determined by (13.72) may be regarded as the
steepest descent feasible direction of the potential function f on [( ++ with
respect to the metric induced by h, i.e., - (dX,d S ) coincides with the solution
(;IX, dS) of the problem
min{ V xf(X,S)d x + V sf(X,S)ds } s.t.
Ad x = 0, A'du + d S = 0, II D-1d x + Dds 112 ~ 1
and may be regarded as a affinely scaled steepest descent direction of the
potential function f.
Recently Bosch and Anstreicher (1993) introduced a partial updating
routine for the Kojima, et al. (1991) primal-dual potential reduction algorithm
(since convergence of the same is tied to the reduction of a potential function)
which employs a safeguard line search condition for controlling the number of
updates incurred on primal-dual iterations. Their upgrading is based directly on
error in the diagonal scaling matrix D and allows for unequal step lengths in the
primal and dual variables.
Bosch and Anstreicher demonstrate that their modified algorithm, like the
original algorithm of Kojima, et al. (1991), requires 0(.Jn L) iterations and offer
a condition which is sufficient to exert control over the total number of updates,
thus leading to an overall complexity of 0(n 3 L) iterations.
Gonzaga and Todd (1992) present a primal-dual interior point algorithm for
the standard form primal linear programming problem
min e'x s.t.

AX = b, X~O.
The algorithm reduces a primal-dual potential function at each iteration by
applying separate scalings to the primal and dual problems and takes a scaled
projected steepest-descent step in either primal or dual space. The authors
demonstrate that a constant reduction in the potential function can always be
achieved in either primal or dual variables, thus leading to a bound of O( In L)
iterations given that the matrices A, e, and b contain all integer elements.
While the conventional dual of the preceding primal problem is
max b'U s.t.
A'U + S = e, S ~ 0, U unrestricted,
Gonzaga and Todd work with the dual problem also in standard form involving
only the dual slack variables S ~ O. To this end, let F be a matrix whose rows
span the null-space of A (N(A)={XIAX=O, X€R n }) or FA'=O and let
Fe = g. Then A'U + S = e for some U if and only if FS = g. For d any vector
satisfying Ad = b, b'U =d' A'U =d'e - d'S. Hence the conventional dual can be
written in terms of S alone as
min d'S s.t. (13.75)
FS = g, S~ O.
Here weak duality assumes the form e'x ~ d'e - d'S for all feasible X, Sand
strong duality states that equality holds if and only if X, S are optimal.
Moreover, the duality gap e'x - b'U equals e'x + d'S - d'e (and also X'S).
Let 9Gp, 9Gb denote the set of strictly positive feasible solutions to the
primal and dual problems respectively. For (X, S) € tU + = 9Gp x 9Gb let the
primal-dual potential function be
¢p(X,S) = p InX'S- '2:;=1 lnxjsj' P > n. (13.76)
Also, let the primal and dual penalized functions appear as
(a) f~(X) = ae'X-'2:;=l lnx j,a>O;

(13.77)
(b) f:?(S) = ad'S - '2: ;=1 illS j' a > 0,
respecti vely .
To ascertain the relationship between the potential function <p p and the
penalized functions f~(X) and f;;(S), let P A denote the projection onto the null
space of A. Then for any Sf9bb' PAS = PAC (the rows of P A span N(A».
Also, it can easily be shown that
(a) P A V X <pp(X,S) = PA V f~(X), a = ;,S;

(13.78)
(b) P F V S <pp(X,S) = P F V f;;(S), a = ;,S.
Let the minimizer of f~ over %p be called the a-center of X~ and denoted

X(a). At any such point P A V f~(X) = O. Likewise, the minimizer of f;; over
9Gb is termed the a-center of Xb and denoted S( a). Here too P A V f;;(S) = 0 at
S(a). Thus the equations relating the a-centers ':lbP,/(b are
X(a)S(a)1 = a-II, X(a)'S(a) = ~,
where X and S are respectively diagonal matrices containing the components of X
and S. The path {X(a), S(a)} is thus the central trajectory (see the discussion
underlying (13.40».
In order for their algorithm to effectively minimize the primal-dual
potential function <p p' Gonzaga and Todd note that the projected gradient of <p p
must be large so that <p p decreases substantially at each iteration. And this will
occur if the algorithm takes us far from the central path where the projected
gradients of the primal and dual penalized functions tend to be small. If fact, the
authors verify for potential function (13.76) that the primal and dual projected
gradients can never be small. That is, they prove that, given (X, S) € CU +, we
cannot simultaneously have X close to X(a) and S close to S(a) for a = ;,S.
Hence constant reduction in <p p can be achieved at each iteration by taking a step
in either primal or dual space using separate steplengths in each.
Two important elements of the Gonzaga and Todd algorithm are scaling
and measuring centrality. First, for (X, S) € CU +, let X = diag (X) so that X is
transformed into 1. Hence X is used to scale X when considering changes in X
and S-I scales S when contemplating changes in S. (Hence scaling allows us to
assume that the current X (01' S) iterate is 1.)
Next, we can effectively measure the distance from X€ %p to the a-center of

$p by considering the norm of the (scaled) projected gradient of f~.
Specifically, the measure of centrality of X€ %p to the a-center
X(a) is
and the measure of centrality of S€%"jj to the a-center S(a) is
It was mentioned above that, for a suitable choice of a, we cannot simul-

taneously have X close to X(a) and S close to S(a). In fact, according to
Gonzaga and Todd (1992) (their corollary 2.1, p. 354): let (X, S) € 'U + and let
a=p/X'S, where p = n + v..;n, v > 0, and let .d > 0 be such that .d ~! and
.d <~. Then we cannot have both c5 P (X,a) ~.d and c5 D (S, a) ~.d. This result
implies that, when p = n+v..;n with v ~ 2, we cannot have both P AXX 'V xtPp(X, S)
and PFSS 'V stPp(X,S) with norms at most l Thus the potential function tP p can
be decreased substantially by taking a step in either primal space or dual space.
This observation consequently leads to the Gonzaga and Todd (1992)
13.22. ALGORITHM.
STEP O. Initialize.
Choose (Xa, So) € 'U + and termination parameter 1] > 0 (to obtain
the promised polynomial bound, (Xa, So) must be chosen with
tPp(Xa, So) = O(..;n L) and 1] should be 2- 2L ). Let p = n + v..;n
for v ~ 2 and set k = O.
STEP 1. Set (X, S) to (Xk' Sk) and select either a primal or dual step. A
primal step can be chosen only if
" PAXX 'V xtPp(X, S) II ~ ! j
a dual step only if
Choose either:
(a) Primal Step

X= X - ,8XPAXX "V x<pp(X,S) ,
s= S; or
(b) Dual Step
X=X
S=S -,8SPFSS"V S<Pp (X,S),
where ,8 > 0 is chosen as

( 611 P AXX "V x¢J p(X, S) 11)-1 or
(611 PFSS"V s¢Jp(X,S) Ilr l
according to whether a primal or dual step is taken.
If XicSk :S TI, stop; otherwise proceed to
STEP 3. Advance K. to J( +1
Set (X, S) to (Xk+l' Sk+1)'
Return to step 1.
As a practical matter, since the conditions for taking a primal or a dual
step can always be met (by virtue of the aforementioned corollary 2.1 of Gonzaga
and Todd (1992)), one can choose a primal or dual step corresponding to the
larger projected gradient in step 1. In addition, if ,8 is selected according to the
criterion stated in step 1, then ¢J p(X, S) :S ¢J p(X, S) - 410'
Finally, Gonzaga and Todd note that their algorithm is more symmetric
than that of Ye (1990a) (his O(n3L) potential reduction algorithm) which either
takes a primal step or updates the dual, but it is less so relative to the primal
dual potential reduction algorithm of Kojima, et al. (1991).
An affine potential reduction algorithm for solving linear programs that
simultaneously seeks feasibility and optimality is provided by Todd (1993).
Based upon the affine potential reduction methods of Anstreicher (1989),
Gonzaga (1990), Ye (1992), and Freund (1991a), Todd's routine utilizes two
potential functions - one driving an artificial variable to zero in order to attain

feasibility and the other moving the objective value toward a lower bound -
which are decreased simultaneously. Specifically, Todd uses a two-variable linear
program at each iteration to obtain a better current lower bound on the objective
value. Moreover, his method does not require that an initial lower bound be
given since the algorithm actually generates one. In fact, progress is made in the
phase-II potential function even before such a lower bound is attained. By using
a subproblem to find a search direction that treats the phase-I and phase-II
objectives more symmetrically than similar algorithms, the phase-I objective
decreases faster than its phase-II counterpart, so that feasibility can be attained
(phase-I is terminated) exactly before optimality (Todd(1993)).
A rather novel algorithm that solves a standard form primal linear program
(13.14») directly from an infeasible warm start, i.e., directly from an infeasible
solution X satisfying AX = b but not X ~ 0, has been developed by Freund
(1991b). By starting from an initial infeasible solution, Freund's algorithm
dispenses with the need for a phase-I procedure involving the introduction of
artificial variables and generates a sequence of primal solutions that become
increasingly less infeasible and increasingly more optimal, and that coverage to
an optimal feasible solution. Derived as an altered version of Ye's algorithm
(13.14) (Ye (1990a), Freund (1991a)), Freund offers a potential reduction routine
that utilizes affine scaling and employs a potential function of the form
F(X, B) = (n + yin) In (C'X - B) - Ej=l In( x j + hj( c'x- B) ).
where B is a lower bound on the optimal value of the primal objective function
and h j is the lh component of a strictly positive shift vector hf R n which, with
the aid of the duality gap C'X - B, is used to shift the non-negativity conditions
from X ~ ° to X+ E h ~ 0, 0 < E fR. Under suitable conditions (e.g., A has
full row rank; sets of optimal primal and dual solutions are nonempty and
bounded; we have an initial X for which AX = 0, X 1:: 0; we know a lower bound
B on z*, the optimal primal objective value; and the dual feasible region is
bounded and that a lower bound on the size of the dual feasible region is known
in advance), Freund's algorithm achieves a constant decrease in the duality gap

(and in infeasibility) in O(n) iterations and is designed to solve the potential
function minimization problem
~Sl F(X,B) = (n+yIn) In(C'X-B)-Ej=l ln(Xj+h j (C'X-B)) s.t.

AX = b
X + b(C'X-B) > 0 (13)9)
B ~ z*.
(The last inequality ill (13.79) is equivalent to the condition that B ~ b'U for
some dual feasible solution (U, S).)
The algorithm computes a projected gradient d that is used to take a step
in the negative of the primal direction, where the steplength 0:' can be determined
analytically or by a line search of the potential function. As in Ye's O( n 3 L)
potent.ial reduction algorithm (Ye (1990a)), if d is large (e.g., Ildll ~ / (=0.5)),
the algorithm takes a primal step; otherwise, if II d II < /, the algorithm updates
the lower bound 13 by taking a dual step and thus computing new dual variables.
Whenever the algorithm takes its first dual step it generates a dual feasible
solution. The algorithm is designed to reduce the potential function by at least
112 at each iteration.
A modification of (13.79) that includes a barrier term for the dual slack
variables is
min H(X,S,B) = (n+yIn) In(C'X-B)
X,U,S,B
-Ej=lln(xj+h/C'X-B))-Ej=llnsj s.t.
AX =b
X+b(C'X-B) > 0 (13.80)
A'U +S = C
S>O
b'U = B.
To solve this revised potential function minimization problem Freund introduces
a modified algorithm requiring an initial dual feasible solution CU, S) and an
initial primal lower bound B = b'U. Under a set of assumptions more restrictive
than those given above to solve (13.79) (i.e., we have a known initial interior
dual feasible solution (U, S) for which fi, S are feasible in (13.80) so that S> 0,
X+ h(X'S) > OJ and a bound on the set of all dual feasible slack vectors S is
known), the iterates of Freund's refined algorithm, corresponding to improved
values of both primal and dual variables, decrease H by at least 0.4 at each
iteration, thus leading to a complexity of O( In) iterations to achieve a constant
decrease in the duality gap C'X - B.
We close this section by mentioning that Mizuno and Nagasawa (1993) offer
a potential-reduction algorithm which employs the primal-dual affine scaling
direction as a search direction. Using the potential function of Todd and Ye
(1990), the authors choose a steplength at each iteration of the algorithm such
that the potential function does not increase, thus admitting a step size longer
than that which minimizes the potential function. Computational complexity is
based on the reduction of the duality gap at each iteration and their algorithm is
proved to be polynomial-time bounded. Mizuno and Nagasawa also propose a
low-complexity routine in which a centering direction is used whenever an
iteration is far from the central path.

A Homogeneous and Self-Dual Interior Point Method
Given a pair of primal and dual problems of the (standard) form
PRIMAL: min C'X s.t. AX = b, X ~ OJ «13.14»
DUAL: max b'U s.t. A'U ~ C, U unrestricted «13.17.1»
respectively, virtually all of the interior point algorithms examined above solve,
say, the primal problem under the regularity assumption 9G++ =
{(X,S) 10< (X,S) eR2n , AX = b, A'U + S =C for some U eRm } I ¢ or the
optimal solution set for both the primal and dual problems is bounded. How-
ever, slllce no a priori information on the status of %++ is available, the usual
procedure is to explicitly bound the feasible region of the linear program. (With
integer data, a worst-case bound of 2L is chosen in practice.) This issue, along
with the parallel problem of finding ways to accomplish the difficult task of
obtaining a strictly feasible initial point for starting an interior-point algorithm,
has been addressed by Ye, Todd, and Mizuno (1994).
Beginning with the idealized notion that a complete linear programming
algorithm should be versatile enough to: (1) detect whether a linear program is
infeasible or unbounded; and (2) generate a finite optimal solution if one exists,
these authors present a homogeneous and self-dual linear programming algorithm
having O(ynL) iteration complexity. Specifically, the algorithm: (a) solves a
linear program without regularity assumptions concerning the existence of opti-
mal, feasible, or interior feasible solutions; (b) can start at any positive primal-
dual pair, feasible or infeasible, near the central ray of the positive orthant, and
does not use any M -penalty parameter or lower bound; (c) generates a sequence
of iterates that approaches feasibility and optimality simultaneously (provided a
solution exists) and if the problem is infeasible or unbounded, correctly detects
infeasibility for at least one of the primal and dual problems; and (d) at each
iteration solves a linear equation system whose dimension is virtually the same as
those solved by the standard primal-dual iterior-point routines.
In particular, for a self-dual linear program, its dual is equivalent to its
primal. That is, if A IS a (p x p) skew-symmetric matrix (i.e.,
A = -A') and b = -C<:RP, then it is easily shown that the program
minC'U s.t.
AU~b, U~O
is equivalent to its dual. In this regard, if U is feasible in the preceding problem,

then it is also feasible in the dual problem, with the two objective values
summing to zero. Hence this program has an optimal solution value of zero.
Clearly a distinct advantage of self-duality is that we can apply a primal-dual
interior point algorithm to solve a self-dual linear program without doubling the
dimension of the linear equation system solved at each iteration.

Ye, et al. (1994) consider a homogeneous and self-dual artificial linear
program relating the above primal and dual problems. Given any Xo > 0,
So > 0, and Uo' they formulate the problem
min(Xb So + 1) () s.t.
(a) AX-bT+bo = 0
(b) -A'U +CT-C() ~ 0 (13.81 )
(c) b'U -C'X + z() > 0
(d) -b'U + C'X - ZT -XbSo- 1
U unrestricted, X ~ 0, T ~ 0, () unrestricted,
where
b = b - AXo, C= C - A'Uo - So' Z = C'Xo + 1 - b'Uo.
Here b, C, and Z represent, respectively, the infeasibility of the initial primal
point, dual point, and duality gap. It is easily shown that the constraints of
(13.81) constitute a skew-symmetric system.
Expressions (13.81.a-c) with T = 1, () =0, and X ~ 0 depict primal and dual
feasibility and reversed weak duality and, taken together, define primal and dual
optimality. Making T a variable, thereby homogenizing (13.81.a, b), adds the
requisite variable complementary to (13.81.c). To achieve feasibility for X = Xo
and (U, S) = (Uo' So), Ye, et al. introduce the artificial variable () with infeasi-
bility coefficients h, -C, and z. Finally, (13.81.d) is a normalizing constraint
added to achieve self-duality. So while (13.81) is characterized as homogeneous,
it nevertheless admits (13.81.d) as a single inhomogeneous constraint.
Let S depict the (n x 1) vector of slack variables for (l3.8lob) and K repre-
sent the slack scalar for (l3.8loc). In addition, let X h denote the set offeasible
points (U,X,T,(),S,K) for (13.81) with X~ = {(X,T,S,K) 10< (X,T,S,K)tX h } the
set of strictly feasible points of Xli.
Looking to the dual of (13.81), let Uand X be the vectors of dual variables
complementary to constraints (13.81.a, b) respectively and let f and (j be the
dual variables for constraints (13.81.c,d) respectively. By definition, the dual of

(13.81) must simply be (13.81) itself expressed in terms of iI, X, f, and 0.
Trivially, any feasible point of (13.81) must be a feasible point of its dual and
vice versa.
To establish the connection between (13.81) and (13.14) (respectively
(13.17.1)), Ye, et al. (1994) point out that if Xo, (Uo,So) is feasible in (13.14)
«13.17.1)), then b (C) is zero, and thus every feasible solution to (13.81) with
T °
> has X/T feasible in (13.14) «U,S)/T is feasible in (13.17.1)). Conversely, if
Z < 0, then every feasible solution to (13.81) with 0> and T > °has °
°
C'X - b'U ~ zO < so that either X/T or (U, S)/T must be infeasible. Moreover,
Ye, et al. also verify (essentially their theorem 2, pp. 56-57) that:
a. (13.81) has a strictly feasible point U = Uo' X = Xo > 0, T = 1, 0 = 1,
S = So > 0, and /'i, = 1.
b. (13.81) has an optimal solution and its optimal solution set is bounded.
c. The optimal value of (13.81) is zero, and for any feasible point
(U,X, T,O,S,/'i,) €%h, (~So+l)O = X'S + TIl:.
d. There is an optimal solution (U*,X*,T*,O* = O,S*,/'i,*)€ %h that is
strictly self-complementary, i.e.,
( X*+S*)
T* +/'i,*
> °
(any such solution for (13.81) must also be one for its dual and vice
versa).
Based upon these considerations, Ye, et al. choose Uo=O, Xo=l, and So=!'
to wit (13.81) becomes
min (n+1)O s.t.
(a) AX- bT+bo
°
(b) -A'U + CT - CO-S
° (13.82)
=°
(c) b'U - C'X + zO -/'i,
(d) -b'U+C'X-ZT -(n+1)
U unrestricted, X ~ 0, T ~ 0, 0 unrestricted, S ~ 0, /'i, ~ 0,

where
h = b - AI, C = C - I, and z = C'I + 1.
We may combine the constraints of (13.82) and conveniently rewrite (13.82.d) as
I'X + I'S + T + K, - (n+I)O = n + 1.
We can now relate optimal solutions for (13.82) to those of (13.14),
(13.17.1). Specifically, we have
13.23. THEOREM [Ye, et al. (1994)]. Let (U*,X*,T*,O* = 0, S*,II:*) be a
strictly self-complementary solution for (13.81). Then:
(i) (13.14) has a feasible and bounded solution if and only if T * > 0
so that X*/T * is an optimal solution for (13.14) and
(U*/T*,S*/T*) is an optimal solution for (13.17.1).
(ii) If T * = 0, then K,* > 0 and thus C'X* - b'U* < 0, i.e., at least
one of C'X and - b'U * is strictly less than zero. If C'X* < 0,
then (13.17.1) is infeasible; if - b'U* < 0, then (13.14) is
infeasible; and if both C'X*, - b'U* < 0, then both (13.14) and
(13.17.1) are infeasible.
Moreover,
13.24. COROLLARY lYe, et al. (1994)]. Let (U, X, r, 0 = 0, S, K) be any
optimal solution for (13.82). Then if K > 0, either (13.14) or
(13.17.1) is infeasible.
If the primal and dual pair of problems is written in symmetric form as
PRIMAL: min C'X s.t. AX - t = b, X ~ 0, t ~ 0;
DUAL: maxb'U s.t. A'U+S=C, U~O, S~O,
where t, S denote (m xl) and (n xl) vectors of primal surplus and dual slack
variables respectively, then, given any Xo > 0, to> 0, So> 0, and Uo > 0, we may
write the homogeneous and self-dual program as
min (Xf>So + ~Uo+1)0 s.t.
(a) AX - bT + b9 ~ 0
(b) -A'U + CT - CO ~ 0 (13.83)
(c) b'U - C'X + zO ~ 0
(d) - h'U + C'X - zT = - Xf>So - t~Uo-1
U ~ 0, X ~ 0, T ~ 0, 0 unrestricted,
where b = b - AXo + to' C = C - A Uo - So' and Zo = C Xo + 1 - b Uo.

- - I - I I
The preceding collection of homogeneous and self-dual programs do not re-

quire any particular or stylized algorithm for their solution. However, Ye, et al.
(1994) indicate that interior-point algorithms that generate a strictly self-comple-
mentary solution (see Giiler and Ye (1993» can be utilized. For instance, one
can conveniently adapt the preceding central path routine (see section 13.10) to,
say, (13.82) by employing
13.25. THEOREM lYe, et al. (1994)]. (i) For any JJ > 0, there is a unique
point (U,X,T,O,S,K)€9G~ such that
(~~) > JJl,
where X = diag(X). (ii) Let M be the coefficient matrix of (13.82.a-d)

and let N(M) denote its null-space. If the direction vector
d= (dU,dX,dT,dO, dS,dK)€N(M), i.e., Md= 0, then
(dX)/d S + dT dK = o.
The importance of this theorem is that it defines the self-central path for
(13.82) or
r h = {(U,X'T,O,S'K)€9G~I(~~) = X'~t[Kl}
If Xo So = 1, Xo = diag(Xo), then the initial interior feasible point U = Uo'
x = Xo > 0, S = So > 0, T = 0 = K = 1 is on r h with JJ = 1. Our earlier selection
of Uo = 0, Xo = So = 1 meets this requirement. A neighborhood of the self-

central path can be defined, for some ,13 € (0,1), as
6(,13) = {(U, X, T, 0, S, K) € 9G~ III( ~~) - JJlll ~ ,l3JJ, JJ -_ X/S+TK}

n+1 .
Armed with the results of the preceding theorem, Ye, et al. (1994) apply the
predictor-corrector algorithm (Mizuno, et at. (1993» to (13.82) along with the
termination technique introduced by Ye (1992) (see also Mehrotra and Ye (1991»
and a least squares projection that produces a strictly self-complementary
optimal solution. Specifically, given an interior feasible point (Uk' X k , T k'
Ok' Sk' Kk) € 9G~ and the linear system:
(13.84)
where Xk = diag(X k ) and Sk = diag(Sk)' let us consider the

PREDICTOR STEP. For any even number k let (Uk,Xk,Tk, Bk ,Sk,Kk)Ec5(f3)
with f3 =i. Solve (13.84) with "( =0 and let

U(a) = Uk + ad u B( 0')
X(a) = X k + ad X Sea)
T(a) = Tk + ad T
Determine the steplength adaptively from
a = max{a I (U(a),X(a),T(a),B(a),S(a),K(a»Ec5(2f3)} (13.85)
and compute the next points from
Uk = U(a) 'Ok B(a)

Xk = X(a) Sk Sea)
Tk = T(a) Kk K(a ).
Next comes the
CORRECTOR STEP. For any odd number k solve (13.84) with
(Uk,Xk,7\,Bk,SkJ~k)E6(2f3) and "( = 1. Then form
Uk+ 1 = Uk + dU Bk+ 1 = '0 k + dO
Xk+1 = Xk+dx Sk+l = Sk+ ds
~ d-T - -K
Tk+l = Tk+ Kk+l = Kk+d
with (U k+1 ,Xk+1 ,Tk+l,Bk+l,Sk+l,Kk+l)E6(f3).
TERMINATION. Let index set jk=Ulxj2:sj,j=l, ... ,n} and let B
contain the columns aj of A with j E jk and let R contain the remaining columns
of A. Then to obtain a solution that is strictly self-complementary and optimal
for (13.82), consider:
Case 1. If Tk 2: Kk, solve for U, X B, and T from

minll Uk - U/ 2 +//XBk - XBIF + (Tk-T)2 s.t.
BX B - hT = 0
-B/U +CBT = 0
h/U - CBX B = 0;
Case 2. If Tk < Kk' solve for U, X B, and K from
minllUk - UI12+IIXBk - XBI12 + (Kk- K)2 s.t.
BXB=O
-B/U = 0
h/U - CBX B - K O.
This projection guarantees that solutions XL and S~ (where S~ = CRT* - R/U*

for case 1 and S~ = - R/U* in case 2) are positive, and T* is positive in case 1
and K* is positive for case 2, as long as Xi.Sk + T kKk is sufficiently small.
If (13.14) and (13.17.1) have integer data with bit length L, then the
process just described renders a strictly self-complementary solution for (13.82) in
OC/nL) iterations. But this implies that, within O( Vn L) iterations, this
solution scheme also generates either optimal solutions to (13.14) and (13.17.1) or
indicates that one or the other of these problems is infeasible. Problem (13.14)
has an optimal solution if Tk remains bounded away from zero for all k; it has no
optimal solution if both Tk and (}k converge to zero at the same rate. In fact, if
both (13.14) and (13.17.1) are feasible with nonempty interior, then T* > 0 at
any self-complementary solution (U*,X*,T*,()* = 0, S*,K*) for (13.82) (Ye, et al.
(1994)).
INTERIOR POINT ALGORITHMS
FOR SOLVING LINEAR
14 COMPLEMENTARITY PROBLEMS
14.1. Introduction
In chapter ten we employed a variant of the standard simplex routine,
called the complementary pivot method, to generate a solution to the linear
complementarity problem LCP(q,M), which we shall now express as: find an
(X, Y) € R2n satisfying
(a) Y = MX+q
(b) (X, Y) ~ (0, 0) (14.1)
(c) xiYi = 0, i=l, ... ,n,
where M is of order (n x n) and q € Rn. Here the feasible region % associated with
(14.1), its relative interior %0, and the set of all solutions to (14.1) will be
denoted as:
% = {(X,Y) I (X,Y) ~ (0,0), Y = MX+q},
%0 = %nR~+ = {(X,Y)€R~+ IY = MX+q},
and
%LCP = {(X,Y)€%lxiYi=O,i=l, ...,n}
respectively.
Our objective m this chapter is to explore several recent interior point
algorithms for solving (14.1) or its equivalent. These algorithms are based upon
the techniques developed in the preceding chapter and are designed to generate an
approximate solution from which an exact solution can readily be determined. An
overview of the principal approaches to solving LC P( q, M) will be given,
433
followed by the particulars of the same. As we shall now see, virtually all of the
theoretical and algorithmic machinery developed in the preceding chapter nicely
carries over to the material presented herein.
One of the earliest adaptations of an interior point method to solving linear
complementarity problems was introduced by Kojima, Mizuno, and Yoshise
(1989a). Starting from a known initial point in a neighborhood of the central
trajectory, the algorithm generates a sequence {(Xk' Y k)} near the same until we
attain a sufficiently small value for a parameter JL. For some positive constant 0,
each iteration decreases the error X~ Yk at least linearly by (1-0/ yin), so that
the algorithm terminates within O( ylnL) iterations when an approximate
solution satisfying (X, Y) € X, and X'V < 2- 2L is attained, where the size L of
LC P( q, M), M positive semidefinite, is
L = l~ ;=1 ~ ;=1 log( I aij 1+1) + log(n 2)J+ 1,
where LzJ is the largest integer not greater than z and aij is the i, /h element of
the (n x n+l) matrix A = [M,q). Each iteration computes a new point near the
central trajectory by using a Newton direction obtained as the solution of a linear
equation system. This output requires O(n3) arithmetic operations. Hence the
total number of arithmetic operations taken over O( ylnL) iterations is O( n 3 .5 L).
An algorithm closely related to the preceding central path-following algori-
thm is the O( ylnL) potential reduction algorithm given by Kojima, Mizuno, and
Yoshise (1991b). Again considering a positive semidefinite linear complemen-
tarity problem, this algorithm reduces a potential function by a constant amount
at each iteration requiring O(n 3 ) arithmetic operations. Starting from an interior
feasible solution with the potential function value bounded by O( ylnL), the
algorithm generates, in at most O( ylnL) iterations, an approximate solution with
potential function value equal to - O( ylnL). From this an exact solution can be
found in O(n3) additional arithmetic operations. An important feature of this
algorithm relative to the preceding path-following one is that the former can be
initiated from an arbitrary interior feasible point, and then readily implemented
using a line search along a descent direction of the potential function.
Interior Point Algorithms for Linear Complementarity Problems 435
In a similar vein, Kojima, Megiddo, and Ye (1992) also offer an interior

point potential reduction algorithm for solving LC P( q, M) with M positive
semidefinite. Starting with an equivalent quadratic program, these authors
employ a linear scaling transformation to map a point (X, Y) from original space
to (1,1), take a step in transformed space in the direction of steepest descent of
the potential function, and then transform back to original space. This process
yields a polynomial time algorithm which converges in O( n 2 L) iterations.
A third class of algorithms for solving linear complementarity problems,
termed predictor-corrector routines, have been offered by Ye and Anstreicher
(1993) and Mehrotra and Stubbs (1994). Ye and Anstreicher observe that the
predictor-corrector algorithm for solving linear programs exhibits asymptotic
quadratic convergence of the duality gap X~Y k to zero without any assumptions
concerning nondegeneracy (a requirement for a unique solution) or the
convergenc~ of the iteration sequence {(Xk' Y k)}' They then proceed to prove a
similar result for the linear complementarity problem assuming only that:
9(;0 :f: <p; M is positive semidefinite; and that a strictly complementary solution
exists (this last condition appears to be necessary for achieving superlinear
convergence of the algorithm). By extending the predictor-corrector method of
Mizuno, et al. (1993b) to the linear complementarity problem (as suggested by
Ji, et al. (1991», Ye and Anstreicher demonstrate that their algorithm has
iteration complexity O(..jiiL) and X~Yk -+0 Q-quadratically.1
IFor a sequence of points {(Xk' Y k)} such that (Xk' Y k) (%0 and Xk Y k ....... 0. the duality
gap sequence {XkYk } converges Q-superlinearly to zeroIf
. Xk+IYk+1
lam I =0;
k ....... oo XkYk
and Q-quadratically to zero if
. Xk +IYk +I
lam sup 2 < + 00.
k-+oo k
(X Y k)
Neither of these notions of convergence of {(X~ Y k)} requires the convergence of the iteration
sequence {(Xk' Y k)}'
Several new predictor-corrector methods for solving LC P{ q, M) have

recently been developed by Mehrotra and Stubbs (1994). These methods replace
the Newton predictor step with directions computed from certain specialized ball
optimization and trust region problems. Although these new methods retain the
basic predictor-corrector strategy, they compute alternative predictor directions
at each iteration by solving subproblems defined over ellipsoidal approximations
of the feasible region and solve the linear complementary problem in O{ y'nL)
iterations. As the authors note, since the effectiveness of a predictor-corrector
method is essentially determined by the progress made in the predictor step,
using alternative predictor directions to secure better approximations can osten-
sibly lead to improved interior point algorithms for solving linear
complementarity problems.
We noted in the preceding chapter that Kojima, Megiddo, and Mizuno
(1993b) demonstrate theoretical convergence of primal-dual interior point
algorithms for linear programs by considering improved steplength rules (their
rules G and P) which admit large steplengths independent of any line search and
which rely only on the lengths of steps taken from the current iterates in primal
and dual spaces to the boundaries of the primal and dual feasible regions
respectively. Kojima, Kurita, and Mizuno (1993) modify the aforementioned
rules and extend the analysis of Kojima, et al. (1993b) and Megiddo (1989b) to
the construction of interior point algorithms for solving LCP(q, M), M positive
semidefinite, on the central trajectory by moving in the Newton direction towards
said path at each iteration. Their new steplength rules, Rules a' and pi, ensure
global convergence and O(nL) iteration polynomial-time complexity respectively.
14.2. An Interior-Point, Path-Following Algorithm

for LCp(q,M)
To set the stage for the development of the Kojima, Mizuno, and Yoshise
(1989a) path-following algorithm, let us represent the path of centers Xcen as
%cen = {(X, Y) € $1 :CjYi = J.L, i=I, ... , n, for some J.L > 0 },
o$ J.L € R a parameter. In addition, it is assumed that:
(i) n>- 2',
(ii) all elements of M, q are integers;
(iii) M is positive semidefinite (i.e., X'MX ~ 0 for every X € Rn); and
(iv) each row of M has at least one nonzero element.
Consider the mapping H: R~+2n ...... R2n defined by
H(J.L,X,Y) = (XV1-J.L1, Y-MX-q)
for every (J.L,X, Y) € R~+2n, where X = diag(X),V = diag(Y). Then
H(J.L,X,Y) = 0, (X,Y)€R~n, J.L ~ 0, (14.2)
represents a family of systems of equations with parameter J.L and thus

(14.1) is equivalent to the system of equations
H(O,X,Y) = 0, (X,Y)€R~n, (14.2.1)
i.e., (X, Y) € R~n is a solution of LC P( q, M) if and only if it is a solution of

(14.2.1). Moreover, a point (X, Y) satisfying (14.2) for some J.L ~ 0 will be termed
a center of $, and the path of centers $cen can now be reexpressed as
%cen = {(X,Y)€R~n 1H(J.L,X,Y) = 0 for some J.L > O}
= {(X,Y)€$oIXVl = J.L1forsomeJ.L>0}.
When %0 -.:j:. ¢ ,and M is positive semidefinite, (14.1) has a unique solution for
each J.L > 0 and %cen is a smooth curve running through %0 to a point on the
boundary of %.
Let us next define an ex-center neighborhood of the path of centers as
where II XY1-(X'Y/n)111 serves as a measure of the deviation of each point

(X, Y) €%o from the central trajectory. Clearly (X, Y) €$o lies on %cen if and
only if II XVl- (X'Yln) 1 II = O. Hence we seek to refine our approximation to
%cen in a fashion such that I XV1- (X'Yln) 1 I ...... 0 as X'y ...... O.
Given an initial (known in advance) point (X, Y) t 9G cen ( a) with X'Y ~

2°(£) and a parameter j.l > 0, let us form a local linearization of (14.2) as
oHlo(X,Y) [~1 = H(j.l,X,Y)

with Jacobian matrix oH lo(X, Y). Then the Newton direction at (X, Y) is the
solution (dX,d Y ) of the system
YdX+Xd Y = XY1-j.l1
- Md X +d Y = 0 (14.3)
or
(M +X-I Y) -1 (YI-J-lX- 1 1)
Md X . (14.4)
(Since M is positive semidefinite, the directions d X , yY form an acute

angle, i.e., (dX)'Md X 2: 0.) The new point is then given by
~
(X,Y)
~
= (X,Y)-(dX ,dY ) (14.5)
with Y MX + q > O. Moreover, if (X, Y)

for any (X, Y) t 9G and any j.l
t 9Gcen ( a) and we choose a suitable J-l > 0, then (X, Y) remaills III 9G cen ( a).
Formally, we have
14.1. THEOREM [Kojima, et al. (1989a)]. Let a ~ 0.2 and 6 = al(l-a).
Suppose that (X,Y)t9G cen (a) and J-l = (1-6Iyfn)X'Yln. Then the
point (X, Y) given by (14.4), (14.5) satisfies
(X, Y) t 9Gcen ( a),
X'Y ~ (1-612yfn)X'Y.
Armed with these considerations, we may now state the Kojima, et al.
(1989a) interior point
14.2. PATH-FOLLOWING ALGORITHM FOR LCP(q,M).
STEP O. Initialize.
Choose an initial point (Xo' Yo) € 9G cen (a) with ~Yo ~
2°(£). Choose 0 < a ~ 0.2and6 = al(l-a) and set k = O.
STEP 1. Let J.L = (1-6/Jn)X kY k /n and set (X,Y) = (Xk,Y k).

STEP 2. Determine the Search Direction.
Compute the Newton direction (dX,d Y ) as the solution to
( 14.4).
STEP 3. Take a step in the Newton Direction.
Determine the new point
(Xk+1Yk+l)
If Xk+l Y k+l < 2- 2L , stop. Otherwise set k = k+ 1 and
return to step 1.
As required, the sequence {(Xk' Y k)} generated by the algorithm lies in the (t-
center neighborhood and the error Xl. Y k decreases at least linearly along the cen-
tral trajectory. Once an approximate solution eX, f) € % satisfying .K'f < T2L
has been found, the authors provide a mechanism for determining an exact solu-
tion of LCP(q, M) in O(n 3 ) additional arithmetic operations (their appendix B).
While algorithm 14.2 may be viewed as an O( n 3 .5 L) algorithm, Kojima, et
al. (1989a) seek to improve its computational complexity to O(n 3 L) by
modifying the Newton direction using approximations to the diagonal matrices
X, Y. Their modification requires an average of O(n 2 . 5 ) arithmetic operations
per iteration and preserves the linear convergence rate of 1 - 8/ In for some
8> O. (On all this see Kojima, et al. (1989a, pp. 7-10).)
Finally, Kojima, et al. (1989a, pp. 17-20) offer a technique for preparing the
initial point (Xo, Yo) from which their algorithm(s) start. Then demonstrate how
to construct an artificial linear complementarity problem which satisfies
assumptions (i)-(iii) above and has a trivial initial point.
14.3. An Interior-Point, Potential-Reduction Algorithm for

LCp(q,M)
Given (14.1), let M be positive semidefinite with %0 i= ¢J and let
the potential function be of the form
leX, V) = ..jn log X'V - E: 1 10g(nxiyJX/Y)
for every (X, Y) dGo. (Clearly leX, V) ~ ..jn log X'V for every (X, Y) dGo and
thus X'V ~ 2- 0 (L) if leX, Y) = - O(..jn L).)
Let us assume that a point (X, V) f %0 satisfying leX, Y) ~ O(..jn L) is
known in advance. (If such a point is not readily available, then we can again
invoke the procedure outlined in Kojima, et at. (1989a).) Given this point, let us
determine a feasible descent direction (dX,d Y ) and a step parameter () such that
!«X, Y) - ()(dx,d Y »~ leX, Y) - o.
(To accomplish this, Kojima, Mizuno, and Voshise (1991b) suggest undertaking a
line search along the direction - (dx,d Y ) to find either a new solution
(X, Y) - ()*(dx,d Y ), where ()* = sup{O I (X, V) - ()(dx,d Y ) f%}, or an approxi-
mation to the optimal step parameter 0f (0, ()*) such that
leX, V) - O(dx,dY ) ~ lex, Y) - O(dX,dY )
foreverY()f(O,()*).) Let the new point rendered be eX, Y) = (X, V) - ()(dX,dY)€%O.
To ensure the feasibility of (X, V), Kojima, et at. (1991b) impose on the direction
(dx,d Y ) and steplength () the requirement that
d Y = MdX and
~I X- 1dx lbo ~ r, ~I y- 1dY lbo ~ r (14.6)
for some r f (0,1) and X = diag(X), Y = diag(Y). (Remember that for V f R n ,

II V 1100 = max{ I vi I ,i=l, ... ,n}.)
How should the descent direction (dX,dY ) be chosen? Kojima, et al.
(1991b) start by demonstrating that the difference between the potential function
values at the new and old points is expressible as
,...,... XY 2 XY
!(X,Y)-!(X,Y)~()gl(d ,d )+()g2(d ,d),
- n ~/.f (Y'dx+X/d Y ) + l'(X- 1dX +y- 1d Y ),
= n+..jn (dX)/dY )1 X- 1d X 1 2+lly-1dY If

X'Y 2(1-r)·
Let us restrict our attention to the term Ogl(d x ,dY ) which is linear in
the step parameter 0. Then gl(d X ,dY ) can be rewritten as
gl(d X ,dY ) = (y- 1 1_ n +.jii Yl)' dU

IIVlf '
where
V, = « X 1Yl )1/2 , ••• , (XnY n )1/2) ,

Y = diag(V) = (Xy)I/2, and
dU = y-l(Yd X + XdY ).
If dU is taken as the steepest descent direction for gl(d X ,dY ), i.e.,

dU = y- 1 1_ «n+.jii)/IIVII2 )V
II y- 1 1_ « n+.jii)/II V 112) V II
(here d U is the negative of the y- 1 1_ «n+.jii)/II V 112) V) , then we can
choose the direction (dX,d Y ) as the unique solution of the linear system
(14.7)
On the basis of these developments, Kojima, et al. (1991b) proceed to

establish
14.3. THEOREM [Kojima, et al. (1991b)]. Let vmin = min{v1 , ... ,vn }
and let 0 = vminT for some TE(O,I). If (dx,d Y ) satisfies (14.7),
then (14.6) holds and
Ogl~ -h/3T
o2g2 ~ max{n+Vn 1 } 2
---rn-' 2(I-T) T.
In the light of these results it is easily shown that if n ~ 2 and T = 0.4,
!(X, Y) - !(X, Y) ~ - 0.2.
We can now summarize the Kojima, et al. (1991b)
14.4. POTENTIAL-REDUCTION ALGORITHM FORLCP(q,M).
STEP O. Initialize.
Choose an initial point (Xo' Yo) d(;o with !(Xo, Yo)
~ O( .jii L). Choose 6 = 0.2 and set k = O.
STEP 1. Determine the Search Direction.

Compute (dX,d Y) as the solution to (14.7).
STEP 2. Determine the Search Parameter.
Compute a step parameter -0 via linesearch along
- (dX,d Y ) or use an approximation of the optimal
step length (see above).
STEP 3. Take a step in the (dX,d Y) Direction.
Determine the new point(Xk +1' Y k +1) - O(dX,dY)€Xo such
that
- X Y
f«Xk+l'Y k+1)-B(d ,d ))~f(Xk'Yk)-8.
If XkY k ~ 2- 2L , stop. Otherwise set k = k+1 and return to
step 1.
The algorithm generates a sequence {(Xk' Y k)} € XO such that
f( X k +1' Y k +1 ) ~ f(X k , Y k ) - 8, k =0,1, ... ,
and ultimately an approximate solution (Xp, Y p) of LC P( q, M) for some
p = O(ynL) such that X~Yp ~ Tn. An exact solution is then obtainable in
O(n 3 ) arithmetic operations.
To establish the connection between algorithms 14.2 and 14.4, let
(X,Y)€Xo. If we take Jl = (1-yn/(n+yn»(X'Y/n), then the solution of
system (14.7) gives the same direction (dX,d Y ) as the solution of the Newton
system (14.3) at (X, Y).
An alternative interior-point potential-reduction algorithm for solving a
linear complementarity problem of the form (14.1) has been offered by Kojima,
Megiddo, and Ye (1992). Working with its equivalent quadratic programming
formulation which seeks to minimize the total complementarity over X,
minX'Y s.t.
Y = MX+q, (X,Y) ~ (0,0) (14.8)
(note that if minX'y= 0, (14.1) has a solution), the authors define an €-
complementary solution as a pair (X, Y) ~ (0,0) such that Y = MX + q and

X'V < €. In this regard, Kojima, et al. (1992) are interested in finding a feasible
point (X, V) with x'v < € for a given € > O. Their goal is to develop a poly-
nomial time algorithm for finding an €-complementary solution to (14.8). Once
such a solution is determined, an exact solution can readily by obtained from it
by following the O(n 3 ) arithmetic operation procedure of Kojima, et al. (1989a).
For (X, V) € 9(;0, let the potential function assume the form
!(X,V) = plnX'V - L:1lnxjYj, p = p(n) > n.
In addition, let p(L) depict any polynomial in input length L. If the potential
function value at some initial point is p(L) and if the potential value decreases
during each iteration by at least n- k for some k, then the algorithm takes a
polynomial number of steps to reach an €-complementary solution. In fact, if M
is positive semidefinite and p = 2n + ~, a polynomial time algorithm obtains,
i.e., if M is positive semidefinite, then starting at any point (X, V) € 9(;0 where the
potential value is O(nL), the algorithm converges in O(n 2 L) iterations.
Kojima, et al. (1992) respecify problem (14.8) under a scaling of the
variables. For (Xo, Vo)€ 9(; 0, X = diag(X), and V = diag(Y), a linear transforma-
tion of the original space is defined as
If we let
and, for W € Rn , W = XV, then the transformed problem is
minX'WY s.t.
(14.8.1)
(Here if is positive semidefinite if M is.) Under this transformation, feasible

solutions of (14.8) are mapped into feasible solutions of (14.8.1), i.e.,
y = V-IV = V-1(MX+q) = MX+q, and the point (Xo, Yo) is mapped into (1,1).
In addition, the transformed potential function appears as
....... -- -- - , - ....... n ................

I(X,Y) = plnXWY- Ei=llnxjwjYj' (14.9)
Since I = I, any reduction in one of these functions precipitates the same

reduction in the other.
To guarantee that their algorithm takes a polynomial number of steps,
Kojima, et al. (1992) suggest starting the algorithm at an interior feasible point
where the value of the potential function is bounded by a polynomial p(L) in size
L. To obtain an initial feasible interior point, let us consider the modified linear
complementarity problem
Y = MX+(I-Ml-q)xo +q
Yo = Xo
(X, Y) ~ (0,0), (xo' Yo) ~ (0,0) (14.10)
XjYj = 0, i=O, 1, ... , n.
Setting X = Y = 1 and Xo = Yo = 1 thus renders an interior feasible solution with

potential function value pIn (n+ 1). Then an €-complementary solution of (14.10)
is also an approximate solution to (14.8) so that following the procedure outlined
by Kojima, et al. (1989a) will yield an exact initial solution.
To execute an iteration in original space, we must: (1) apply the preceding
linear scaling transformation in order to map the current point (Xk' Y k ) to (1,1)
and I to I; (2) take a step in transformed space in the direction of the gradient
of I, where the steplength () >0 is either prespecified by the algorithm or
determined by a line search which minimizes the value of I; (3) determine the
new point in transformed space according to (X k ,\\) = (l,l)-(}d X ,d Y); and
(4) map the new point in transformed space back to original space to obtain
(Xk+l' Yk+l)'
How may we specify the direction vector (dX,d Y )? Let us assume that the
current point is (1,1) and the potential function is of the form given by (14.9).
Given that it is to be understood that we are working in the transformed space,
the circumflex "~,, above X, Y, M, and I will be omitted for convenience. Then
from (14.9),
P - --1
Vxf(X,Y) X'WX WY -X 1,
Vyf(X,Y) - L wy-y- 1 1.
X'WX
Let
and, for (X, Y) = (1,1), it can readily be shown that
V Xf(I, 1) = Vyf(I,I) = I'~I WI-I = g.
Hence we may view (dX,d Y) as the orthogonal projection of V f(I, I)' = (g',g')
on the linear space d Y = Md X, i.e., (dX,d Y) is determined as the solution of
the constrained optimization problem
From the Lagrangian

i., = (dX_g)'(dX_g) + (d Y -g)'(dY -g) + ,\'(dY -MdX )
we can obtain the first-order optimality conditions
d X - g - M''\ = 0,
dY-g+'\ = 0,
d Y -Md X = 0.
Upon eliminating ,\ we have
dX (I n-M'M)-1(I n+M')g,
dY Mdx
(where the M's in this system are actually M's since we are operating in trans-
formed space).
14.4. A Predictor-Corrector Algorithm for Solving LCp(q,M)

The predictor-corrector algorithm of Ye and Anstreicher (1993a) generates a
solution to the linear complementarity problem (14.8) under the assumptions
that:
(i) the feasible region for (14.8) has a nonempty relative interior or %0 ¥= <p;
(ii) M is positive semidefinite; and
(iii) (14.8) has a strictly complementary solution,
i.e., while (X*, Y*)€ % is optimal (or complementary) if and only if xryr = 0,
i=I, ... , n, a strictly complementary solution is an optimal solution satisfying the
condI·t·Ion Xi* +Yi* > o·
,z=1,...,n.
Define a neighborhood 6(a) = {(X, Y) €% III XY/J.l-III $ a,J.l = X'Y/n,
0$ a $ I} and take 0 < fJ $ t (typically fJ = t). In addition, the search direction
(dX,dY ) is determined as the solution of the linear system
XdY+Yd X = 1'J.lI-XY
d Y -Md x = 0, (14.11)
0< l' < 1 (Kojima, et al. (1989a». As we shall now see, (14.11) must be solved
twice at each iteration of the predictor-corrector routine - once to find the
Newton direction for the predictor step and again to determine a centering
direction for the corrector step.
To undertake the kth predictor step, given (Xk' Y k) € 6(fJ), solve (14.11)
with (X, y) = (Xk' Y k) and l' = o. Denote the resulting direction by (dX,d Y )
and, for some steplength () ~ 0, let
X( ()) = X k + ()df
Y«()) Yk + ()dk
and p«()) = X«())'Y«())/n. A guide to the choice of () is provided by
14.5. LEMMA [Ye and Anstreicher (1993a)]. If for some 0 < ()k < 1 we
have
" X«())Y«())/J.l«()) - 111 $ a < 1 for all 0 $ () $ ()k' (14.12)
then (X( () k)' Y( () k» € 6(a )%.

Since the feasibility of (X«()k)' Y«()k)) is guaranteed if (14.12) is satisfied, let us
choose the largest steplength ()k $ 1 such that II X«()k)Y«()k)/J.l«()k) -111 $ a$ 1
holds for a = fJ + T, 0 < T $ fJ and let Xk = X«()k)' Yk = Y«()k).
To execute the kth corrector step, solve (14.11) with (X, Y) = (Xk' Y k ) € 6(f3+T),
p = XkYk/n and r = 1. If the resulting solution is denoted as (~fX,dY), then we
may form
Xk+I = Xk+d X
Yk+I = Yk+d Y
long as 0 < 13 ~ 1 and 0 < T ~ (3. This process,
involving a single corrector or centering step to the central trajectory after each
predictor step in order to decrease p, generates a feasible sequence of points
{(Vk' Yk)} satisfying ptkYk/ Pk - IiI ~ (3 and, for 0 < (3 ~ 1 and ex = 2(3, has
O(.jTiL) iteration complexity with XkYk-+O Q-quadratically.
Ye and Anstreicher point out that there is no need to choose the largest
steplength ()k in the predictor step - a lower bound for ()k will suffice, i.e., if ()k
is that largest ()k ~ 1 satisfying the preceding lemma with ex = (3 + T and

0< T ~ 13, then ()k ~ 2T/«T2+4~lc5k/1)1/2+T), where 15k = atdk/Pk and at =
diag(dt)·
Based upon the preceding considerations, Ye and Anstreicher formulate the
14.6. PREDICTOR-CORRECTOR ALGORITHM FOR LC P( q, M).
STEP O. Initialize.
Given (Xo, Yo) € 6(1), set k = O.
STEP 1. Predictor step.
Set (X, Y) to (Xk' Y k), r = 0, and compute dt,dk from
(14.11). Form
X( ()) Xk+ ()dt
Y«()) = Y k + ()dk
p«()) = X«())'Y«())/n.
Compute the largest () k so that (X( ()), Y( ())) € 6(!) for
() 2T 0 1
k>(
- (T2+4~16k 11)1/2 + T
)' <T<4'
where 15k= -x
d k d k / Pk and d k = dzag( d k ).
Y -X· X
Set X k = X«()k)' Y k =Y«()k)'

STEP 2. Corrector Step.

Solve (14.11) with (X, Y)
and 'Y = 1. Form
~ ~x
X k+1 = Xk+d
~ ~Y
Y k+1 = Yk+d
with (Xk+1'Yk+1)€(~)'
STEP 3. Check for Optimality.
If XkY k ::; 2- 2L , stop. Otherwise proceed to
STEP 4. Advance k to k+1.
Return to step 1.
The preceding predictor-corrector methodology has been modified somewhat
by Mehrotra and Stubbs (1994). As mentioned earlier, these authors modify the
predictor step so as to determine alternative predictor directions as solutions to
subproblems defined over ellipsoidal approximations of 9G. Two broad categories
of subproblems are considered: (a) methods using primal-dual scaling ellipsoids;
and (b) methods employing affine scaling ellipsoids. Within these, both ball
optimization and trust region approximations are considered.
Relative to primal-dual scaling, the ball optimization sub-problem may be
derived by first letting D = X1 / 2y-l/2. Then the predictor direction in step 1 of
algorithm 14.6 emerges as the solution to
Ball Optimization Subproblem I
max~Y'dX + X'd Y } s.t.
Md'x -d Y = 0
(14.13)
II ~~~dX Ir ::; X'Y.
Here we seek to maximize a linear function over a lower-dimensional ball. If we

let Y = X1 / 2yl/2 1,g = D-1dX, and gY = Dd Y , then (14.13) is equivalent to
max{Y1 X + Y'gY} s.t.
MDg - D-1gY = 0
(14.13.1 )
II~r ~ X'y.'
Based upon these considerations, the Mehrotra-Stubbs modification of predictor-

corrector algorithm 14.6 is to replace the NewtOI1 direction (d X , d Y) determined
in step 1 by the direction obtained by solving (14.13). Under this substitution,
Mehrotra and Stubbs then verify that if we start at an initial point
(Xo, Yo) € 6(,8), 0:::;,8:::;~, the iterates (Xk' Yk) generated by their modified
algorithm also satisfy (Xk' Y k) € 6(,8) so that an O( yin L) iteration algorithm
emerges. Move formally, they prove
14.7. THEOREM [Mehrotra and Stubbs (1994)]. Let (X k ,Yk )€6(,8),
0:::;,8:::;~, and (dX,d Y ) be the optimal solution of (14.13) for predic-
=
tor step 1 of algorithm 14.6. Let () 0.073/ yin, and Xk X k - ()d x ,=
Y
Y k = Y k - ()d. Then under corrector step 2 of algorithm 14.6,
~
.
wlth X k+1 = X k - d 'Y k+1
~ ~X
=Y k -
~ ,
d ,we have X k+1Y k+1 X kY k
~Y / '
:::; 1 - (0.013)/ yin) and (XHl' Y HI) € 6(A)·

Alternatively, the predictor step required at each iteration of algorithm 14.6
can be undertaken by solving a trust region subproblem that minimizes a convex
quadratic objective over an ellipsoidal region. In this regard, we now determine
the predictor direction by solving
Trust Region Subproblem I
min(X-d X )' (Y -d Y ) s.t.

Md X -d Y = 0
(14.14)
I ~~~dX Ir : :; 2
a X'Y
or, since d Y = Md X ,
2For W' = (V', V'), the solution direction (gX,gY) for (14.13.1) can be obtained by
projecting Worthogonally onto the null-space of 8 = [MD,-D- 1] or
(::) = pPBW,
where P B = In - 8'(88,)-18 and p = (X'y)I/2/11 PBWII. PBW::f:. o.

min{X'Y + (dX)'MdX - (Y+M'X)'dX} s.t.

(14.14.1)
(dX)'[M'02M+O- 2 ]dx $ a 2X'Y.
Here too Mehrotra and Stubbs verify that if (14.14) is solved in the predictor step
of algorithm 14.6, then (Xk' Y k) € 6(f3), 0 $ f3 $ l, and convergence of the
predictor-corrector algorithm in O.j1i iterations results. That is,
14.8. THEOREM [Mehrotra and Stubbs (1994)]. Let (Xk' Y k ) € 6(f3),
OS f3 $ 1, with (dX,dY ) the optimal solution of (14.14) in the pre-
dictor step 1 of algorithm 14.6. Let Xk = X k - dX, \\ =Y k - dY
,...,...X """"Y
and, at corrector step 2, let X k +1 = X k - d ,Yk+l = Yk - d .
Then for a = 0.0731.j1i, Xk+1Yk+l/XkYk $ 1- (0.013.j1i) and
(Xk+1' Yk+l) € 6(l)·
Next, subproblems that employ ellipsoids obtained by an affine scaling of
the feasible region have also been utilized by Mehrotra and Stubbs to construct
on O( .j1i L) iteration predictor-corrector algorithm for solving the linear
complementarity problem. For instance, we may determine the predictor direc-
tion by solving either
Ball Optimization Subproblem II
max{Y'd X + X'd Y} s.t.
Md x -d Y = 0
(14.15)
I ~=~:~ Ir $ ~ f3 1
or
Trust Region Subproblem II
min(X-d X)' (Y -dY ) s.t.
MdX -d Y = 0
(14.16)
I ;~::: Ir $ t~nf3'
Then theorems analogous to theorems 14.7, 14.8 given in support of ball

optimization subproblem I and trust region subproblem I respectively can now be
offered for this second wave of subproblems. Specifically, relative to ball
optimization subproblem II,
14.9. THEOREM [Mehrotra and Stubbs (1994)]. Let (Xk' Y k) € 6(f3),
o~ f3 ~~, and (dX,d Y) be the optimal solution of (14.15) for pre-
X
dictor step 1 of algorithm 14.6.
~
Let X k X k - (Jd , =
~ Y
Y k = Y k - (Jd. Then for corrector step 2 of algorithm 14.6, let
'-"""'x -"--y
X k+1 = X k - d , Yk+l = Y k - d. Then for (J = 0.06185/ vn,
X~+lYk+d X~Yk~ 1- (0.0015/vn) and (X k + 1'Y k +1)€6(f3).
And for trust region subproblem II,
14.10. THEOREM [Mehrotra and Stubbs (1994)]. Let (Xk' Y k) € 6(f3),
o ~ f3 ~~, and (dX,d Y ) be the optimal solution of (14.16) in the pre-
dictorstep 1 of algorithm 14.6. Let Xk = X k - d X, \\ = Yk - d Y and, in
. .... -x -'-y
corrector step 2 ofalgonthm 14.6,letXk+1 = Xk-d 'Yk +1 =Yk-d .
Then for B = O.06185/vn, Xi.:+lYk+1/Xi.:Yk ~ 1- (O.0015/vn) and
(Xk+l' Y k+1) € 6(f3).
14.5. Large-Step Interior-Point Algorithms for Solving LCp( q,M)

Kojima, Kurita, and Mizuno (1993) offer a set of improved steplength rules
in a path-following setting for finding a solution to an equivalent representation
of LCP(q, M) (14.1) or to the problem: find a point (X, Y) € R 2n such that
Y = XM+q
(X, Y) ~ (0,0) (14.1.1)
XY = 0,
where X = diag(X). It is assumed that: (a) an initial point (Xv, Yo) € %0 is
known in advance and; (b) M is positive semidefinite. In addition, for parameter
J1. > 0, let the central path for (14.1.1) be defined as
r LCP(J.L) = ((X(J.L), Y(J.L» IY = MX + q, XY = 1'1, (X, Y) ~ (O,O)}.

Then as 1'-0, rLCp(J.L) leads to a solution of (14.1.1) under assumptions (a), (b)
above.
Given the current iterate (Xk' Y k)' let f k = X~Y kin with 13 f [0,1] the
search parameter. To find a point (X(J.L),Y(J.L»frLCp(J.L), J.L=f3h, let us
determine the Newton search direction (dX,d Y ) at (Xk' Y k ) by solving the linear
system
(14.17)
Then we may take a single steplength ak over 9G and obtain a new

iterate
(14.18)
where ak = max{a I (Xk+ad X , Yk+ad Y ) ~ (O,O)}. (Note that ak is always

finite under assumptions (a), (b).)
To obtain their modified Rules a' and p', Kojima, et al. (1993) let
0$ 13 = 13k < 1 and set a = ak in (14.18) so as to express total complementarity
at the new iterate Xk+1Yk+I as a quadratic function of a or
Xk+IYk+I XkYk+a(X~dY +Y~dX)+a2(dx)'dY
= XkY k - a(l-f3k)X~Yk + a 2(d X )'dY .
Since -a(l-f3k)X~Yk < 0 and a 2(d x )'dY > 0 (by virtue of the fact that M is
positive semidefinite), it is evident that the total complementarity for the linear
complementarity problem is not a monotonic decreasing function of a if
(dX)'d Y > O. Minimizing X~+IYk+I with respect to a consequently renders
akin
. = {(I- x f3 k )XkY k if (dX)'d Y > o·
2(d )'dY ,
+ 00 otherwise.
Hence we must choose a steplength a = ak satisfying 0 < a < ak and a $ kin.

Based upon these results Kojima, et al. (1993) state their Rules G', P' for
controlling the parameters a, (3 in an interior-point, path-following algorithm for
solving LCP(q,M). That is,
Rule G': Let 0 ~ p < 1, 0 < 0 < 1, and 0 < 0'* be fixed. At each
iteration choose a search direction parameter {3 = {3 k € [0, P]
and a steplength 0'= ak = min{a',ak'in}, where
a'€[Oa*,ok] ifok~a*;
a'€[0(ok)2/ a*, (ok)2/a*] ifok<a*.
Rule p': Let 0 < {3* ~ p =~, 0 = ~ ~ ()* < 1 and 0 < 0'* ~ 1 be fixed. At
each iteration choose a search direction parameter {3 = {3 k € [{3*, P]
and a steplength parameter a = ak = min{a',ak'in}, where
a' € [00'* , ()*o k ] if ° > 0'*'
k - ,
a' € [0(ok)2/ 0'*, ()*(ok)2 /0'*] if ok < 0'*.
(Here Rule P' is a special case of Rule G'.) Using either of these rules in the fol-
lowing interior-point, path-following algorithm generates a sequence {(Xk' Yk)} € 9(;0
such that X~+1Yk+l < X~Yk' As determined by Kojima, et al. (1993), the algo-
rithm using Rule G' ensures global convergence, i.e., it generates a bounded
sequence {(Xk' Y k)} such that X kYk ->0 as k->oo while Rule P' ensures O( nL)
iteration polynomial-time convergence.
We may now state the Kojima, et al. (1993) modified interior-point
14.11. PATH-FOLLOWING ALGORITHM FOR LPC(q,M)
STEP O. Initialize.
Let (Xo, Yo) € 9(;0 be a known initial feasible point with
XbYo ~ 20 (L).
STEP 1. Determine the Search Parameter, Steplength, and Search
Direction.
Choose either Rule G' or Rule P' with (dX,d Y ) determined
from (14.17).
STEP 2. Take a Step in the Newton Direction.

Compute a new iterate (Xk+I' Yk+I) according to (14.18)
with steplength a = ak satisfying 0 < min{Oa*,O(O:k)2/
a*,ak in } :s a < O:k and a:S akin regardless of whether Rule
G' or Rule pI is chosen.
If X~+l Yk+I < 2- 2L , stop. Otherwise set k = k+1 and
return to step 1.
As the iterations progress, the quantity
0< 15 k = min{xfyf / h, i=l, .. .,n}:s 1
measures the extent of the deviation of the current iterate (Xk' Yk) € 9(;0 from the
central trajectory (Kojima, et ale (1991b)j Kojima, et ale (1993». Here too we
can start the algorithm from a known interior feasible solution of an equivalent
artificial linear complementarity problem. This then produces, in O( nL) itera-
tions, an approximate solution to the artificial problem satisfying the optimality
criterion in step 3, from which an exact solution can be refined (provided one
exists) in O(n3) arithmetic operations.
As a final point, Kojima, et ale (1993) indicate that the predictor-corrector
algorithm for solving LCP(q,M) offered by Ye and Anstreicher (1993) is a special
case of their algorithm 14.11. In particular, they point out that under certain
restrictions on ak and {3k and if the linear complementarity problem has a unique
strictly complementary solution, then under their computational scheme the
complementarity sequence {X~Y k} also converges to zero superlinearly as k-+oo.
14.6. Related Methods for Solving LCp( q, M)

Other approaches to solving LCP(q,M) have been advanced in the
literature. For instance, McShane (1994) presents an interior-point adaptive
path-following algorithm for solving (14.8) (with M positive semidefinite) which
requires at most O(.;n L) iterations to determine feasible vectors X, Y such that
X/Y:s 2- 2L and from which an optimal solution can be derived in O(n3) arithme-
tic operations. The algorithm is based on an adaptive path-following scheme in

that the relative weights on the search directions are not fixed from iteration to
iteration. For (14.8) nondegenerate, if there exists a unique strictly complemen-
tary solution (X*, Y *), then the iterates (X k , Y k) converge to (X*, Y *), and the
duality gap and each component of the sequence {Xk Ykl} converges to zero Q-
quadratically. Under an approximately centered condition, the iterates are forced
to stay close to the central path, thus allowing for a sufficient decrease in the
objective function between iterations and guaranteeing strict complementarity.
An approach for finding a solution to a linear complementarity problem
which departs significantly from the preceding collection of algorithms has been
proposed by Zhang (1994). Observing that finding an initial interior feasible
solution to LCP(q,M) can, at times, involve as much work as solving the
original problem itself, Zhang suggests an approach which involves neither
problem reformulation (e.g., one need not introduce artificial variables and
embed 9G into a new feasible region of higher dimension) nor a feasible starting
point. The method can start from any positive but otherwise infeasible point
(i.e., (Xo, Yo) > (0, 0) but Yo 1= MXo + q) and seeks to achieve feasibility and
optimality simultaneously.
The infeasible interior-point algorithm developed by Zhang is essentially a
centered and damped Newton method with positive starting point and is applied
to a general model called the horizontal linear complementarity problem that
includes linear and quadratic programs as well as problems (14.1) and (14.8).
The structure of the horizontal linear complementarity problem is depicted by
the following nonlinear system:
(MX 1~r -h) = 0, (X, Y) ~ (0,0),

where X, Y,he Rnj M, N are both of order (n x n)j and X = diag(X), Y = diag(Y).
(Here the term horizontal is used to convey the notion that the matrix [M, N] has
fewer rows than columns.) At each iteration a steplength is chosen to ensure that
the next iterate remains positive and a centering step is added to the Newton step
in order to prevent the iterates from approaching the boundary of the positive
orthant too early. As the iterates approach a solution, the centering restriction
becomes progressively weaker. Given an appropriate starting point, Zhang
establishes an O(n2ln (1/f» - iteration convergence bound for the algorithm to
attain €-feasibility and €-complementarity. Moreover, under a set of fairly
unrestrictive assumptions and properly controlled steps, the algorithm converges
at a global Q-linear rate.
Recently, Wright (1994) has proposed a modification of Zhang's algorithm
to obtain an O(n2L) infeasible-interior-point algorithm for solving (14.1) that
exhibits an asymptotic Q-subquadratic convergence rate under the assumptions
that (14.1) has a strictly feasible point and exhibits a strictly complementary
solution. The iterates follow the Newton direction and, as in Zhang's procedure,
are subject to a centering condition that keeps them from approaching the
boundary of the positive orthant too soon. However, unlike Zhang's algorithm,
Wright also imposes a condition that gives a higher priority to feasibility rather
than complementarity. The desirability of each point is described by a merit
function ¢(X, Y) = X'V + II Y - MX - q" whose first term measures uncomple-
mentarity and whose second term depicts infeasibility. Hence a point (X*, Y *) is
a solution of (14.1) if and only if (X*,Y*) ~ (0,0) and ¢(X*,Y*) = O. As the
iterations progress and the merit function exceeds a certain threshold level,
Wright's algorithm is identical to Zhang's algorithm. At each iteration a linear
system of equations is solved to yield a so-called safe step. If the value of the
merit function falls below the threshold, a fast step that may yield rapid
convergence is determined by the algorithm. If the fast step does not render a
significant reduction in the merit function, it is discarded in favor of a safe step.
Additional algorithms designed to solve linear complementarity problems
that are slightly more general than (14.1) have been proposed by Mizuno (1992);
Kojima, Mizuno, and Yoshise (1991b); and Kojima, Megiddo, Noma, and Yoshise
(1991a). Indeed, unified models involving various path-following and potential
reduction models as special cases have been developed.
For instance, Mizuno (1992) presents a new polynomial-time algorithm for

solving a linear complementarity problem with M positive semidefinite and which
follows a sequence of points to a solution of a linear complementarity problem
more general than, say, (14.1.1). That is, if the sequence falls on a path, it
constitutes a path-following method; and if the sequence is based on a potential
function, it amounts to a potential-reduction method. Given an initial feasible
point whose components are bounded between 2- 0 (£), the algorithm solves
(14.1.1) in O(JnL) iterations.
In this regard, for each V to R++ and Q' ~ 0, let us define a neighborhood
6(V,Q') = {U IIV-U II:::; Q'vmin} in R n , where vmin = min{vi,i=I, ... ,n}. Then
for each to ~ 0, V* to R++, and (Xo, Yo) to R~n+, Mizuno considers the problem:
Find an (X, Y) to S ++ such that

- to 8(V*,to), where
XY
s = {(X, Y) to R2n I Y - Yo M(X-Xon (14.19)
S++ = S n R~n+.
Mizuno's technique solves (14.19) for each to> 0 and, if (Xo, Yo) is a feasible
solution of (14.1.1), then any solution of (14.19) for to = 1 and V* = 2- 2 (£+1)1 is
a solution of the problem:
Find an (X, Y) to R++ such that

(14.20)
Y = MX + q, XY
-
:::; 2-2£ 1.
Moreover, Kojima, et al. (1989a) demonstrate that the solution of (14.1.1) can be
obtained from the solution of (14.20) in O(n3) arithmetic operations. (For
details on implementing Mizuno's method for finding a solution to (14.19) see
corollaries 4, 5 in Mizuno (1992, pp. 36-37).) Mizuno then goes on to relate his
technique to more traditional methods for solving linear complementarity
problems by constructing O( In L) path-following and potential-reduction
methods for solving (14.20) that are based on his more general method (see
Mizuno (1992, pp. 39-42)).
Next, we note briefly that Kojima, Mizuno, and Yoshise (1991b) offer a
unified model for both the path-following and potential reduction algorithms
(algorithms 14.2 and 14.4 respectively) presented earlier. To obtain their unified
model, the authors generalize the potential function f given above by replacing
yin by a parameter v ~ 0, i.e., now
f(X, Y) = {n+v)logX'Y - L ~=1 log xiYi - n log n.
Then for each (X, Y) f%++, consider the Newton equations
(14.21)
obtained from the system H{p" X, Y) = (XY - Ill, Y - MX - q) 0, p, =

{1- vj(n+v)}(X'Y/n). Given that the Newton direction - (dX,d Y ) obtained
from (14.21) is a continuous function in (X, Y) f%++, each resulting trajectory in
%++ converges to a solution of LCP(q, M) when v> 0, or to a point on the
central path %cen when v = 0, with the value of f decreasing monotonically
along each trajectory. Kojima, Mizuno, and Yoshise (1991b) note that the path-
following algorithm given earlier by Kojima, Mizuno, and Yoshise (1989a) may
be regarded as a special case of the continuous model given above, i.e.,
v = O(yIn) with v/(n+v) = 6/yIn, 6> 0, was used in the path-following
algorithm of Kojima, et al. (1989a).
Finally, Kojima, Megiddo, Noma, and Yoshise (1991a) extend the unified
approach of Kojima, Mizuno,and Yoshise (1991b) to a large class of interior point
potential-reduction algorithms that: (a) move in a Newton direction toward the
central trajectory; and (b) choose a new point along the said direction in a
certain neighborhood of the central trajectory such that a significant reduction in
the potential function occurs. Their chapter two provides an excellent summary
of the unified interior point method along with global and polynomial-time
convergence results for some special cases of the same such as potential-reduction
and path-following algorithms and a damped Newton method for solving linear
complementarity problems starting from an interior feasible solution. On all this
see Kojima, Megiddo, Noma, and Yoshise (1991a, pp. 7-23).
APPENDIX A:
UPDATING THE BASIS INVERSE
It was noted in section 5.5 above that to obtain a new basic feasible
solution to a linear programming problem via the simplex method, we need
change only a single vector in the basic matrix at each iteration. Hence we are
faced with the problem of finding the inverse of a matrix B which has only one
column different from that of a matrix B whose inverse is known. In this regard,
given B = [hI' ... ' b"J with B- 1 known, we wish to replace the rth column b r of B
by rj so as obtain the inverse of B=[bl, ... ,br_l,rj,br+I, ... ,bm]. Since the
columns of B form a basis for Rm, rj = BYj . If the columns of B are also to
yield a basis for Rm, then, by theorem 2.33, it must be true that Yrj #- o. So if
m
r·) = i=l
Ey··b.
I) 1
= Ylj hI + ... + Yr-l,j br _ 1 + Yrj br + Yr+l,j br+1 + ... + ymjbm

with Yrj #- 0, then
Ylj Yr-I,j 1 Yr+I,j Ymj
br = -Yrjb1 - ... ---y;;-br - 1 +Yr/j-~br+I - ... - Yrj b m
"'. = (_ Ylj
where Y _ Yr-l,j _1__ Yr+I,j _ Ym j ) H b
) Yrj'···' Yrj 'Yrj' Yrj , ••. , Yrj· ence r
459
has been removed from the basis with r j entered in its stead. If we next replace
the rth column of the mth order identity matrix 1m by Yj , we obtain
1m = [el,· ..,er_l' Yj,er+I, ... ,em), (A.1)
where, as previously defined, ej is the lh unit column vector. Now it is easily

demonstrated that B = HIm so that
H-I = 1mB- I . (A.2)
APPENDIX B:
STEEPEST EDGE SIMPLEX METHODS
We noted in section 5.1 that, in the absence of degeneracy, the standard

(primal) simplex method starts from an initial basic feasible solution and
proceeds along a path to the optimum for which the value of the objective
function at any extreme point is greater than the objective value at an adjacent
extreme point. If the primal problem is a minimization program of the form
mini = C'X s.t.
AX b, X~O
(an assumption consistent with the development of this body of theory), where A
is of order (m x n) with p(A) = m < n, then the simplex method traverses a path
to the minimum from an initial extreme point to an adjacent extreme point
along edges of the polytope that render the objective function strictly decreasing.
Hence under minimization, the direction taken is simply downhill. However, in
the steepest ~dge simplex routine (see Kuhn and Quandt (1963), Harris (1973),
Goldfarb (1976), Goldfarb and Reid (1977), and Forrest and Goldfarb (1992»,
the direction selected at each iteration is the one that is most downhill. That is,
the direction edge chosen is the one which makes the most obtuse angle with C,
i.e., the edge direction 'II for which the directional derivative of I in the 'II
direction or C'('II/II'11I1) is smallest.
461
Following Forrest and Goldfarb, let us assume that at the start of a single
step of the simplex method the first m components of X (and thus the first m
columns of A) are basic. Then X, A may be partitioned as X' = (Xl" X R),
A = [B, R] respectively so that a simplex step has the form
-
X = X + OTlq = [B-1b]
0 + OTlq, (B. I)
where X is the basic solution at the start of the step, () is the steplength, and Tlq
is one of the edge directions
Tlj = [ -B-1R]
Ip
.
ej_m' J = m+l, ... ,n = p+m, (B.2)
emanating from vertex X. If -B-IR = [- YI , ... , - Y p ]' then, from (B.2), the
collection of edge directions appears as
TIm+! = [ -e~Il Tlm+2 = [ -e~2l···'Tlm+p = [ -e: p} (B.2.1)
Hence the current basic solution X is a vertex of the polytope % = {X I AX = b,

X ~ 0, X € Rn} and the p vectors Tl m+!, ••• , Tlm+p point down the p edges of %
that emanate from X. In the light of (B.2.!), (B.l) becomes
(B.1.l)
In the standard primal simplex routine, if basic x Bt is to be replaced by

non basic x /lq' then 0 is a scalar (determined by a ratio test) which makes
XBt = O. Moreover, since the angle between Tlq and C must be obtuse, the
reduced cost
must be negative, i.e., Tlq points downhill. By virtue of the usual primal entry
criterion,
Cq = C'Tlq = rnin{Cj,Cj<O}.
J>m
In the steepest-edge algorithm, the direction edge Tlq chosen (or the non basic
variable x Rq selected to become basic) is the edge that is most downhill relative
to Cor
Appendix B 463
(B.3)
So in the standard simplex method, we minimize the reduced cost Cj (the change
in the objective function per unit change in x Rj) while the steepest-edge method
has us minimize normalized reduced cost C''1/11 '1j II. The latter entry criterion
corresponds to choosing the edge of steepest descent of f.
To expedite pivoting, Goldfarb and Reid (1977) (see also Forrest and
Goldfarb (1992)) derive recursive formulas for updating after each pivot the
norms II '1j I or the quantities 'Y j = 11'1j IF = '1j'lj for all nonbasic subscripts j > m.
In this regard, if the nonbasic variable x Rq replaces x Bt in the set of basic
variables, then the edge direction recurrence formulas appear as
(a) TJt -'1q/O: q,
(BA)
(b) TJj = '1j-'1qO:j,j> m,j =I q,
where O:j = c~B-laj (the components of row t of B-1aj)' O:j = O:/O:q (the
components of row t of Ii-1aj' where Ii is the new basis matrix after pivoting);
and, from (BA), we can obtain the recurrence formulas
(a) "it = 'Yq/O:~,
(b) "i j = 'Y j - 20: jajV + O:~'Y q' j > m, j =I q,
where BW = a q and B'V = W (note that O:q = w t ).
We can also update the reduced costs by obtaining, from (B.4), a recurrence
of the form
(a) C'TJt = -C''1q/O: q = -cq/O:q,

(B.5)
(b) C'TJj = C'TJj-C''1qO:j = Cj-CqO:j,j > m,j =I q.
Based upon the preceding discussion let us now formally offer a single step
of the Goldfarb and Reid (1977)
B.l. PRIMAL STEEPEST EDGE ALGORITHM.
STEP 1. Check optimality.
If Cj > 0, j > m, the current solution is optimal and the
algorithm terminates. Otherwise proceed to
STEP 2. Determine the pivot column.

Find cq / II fJ q II from (B.3) or equivalently find
rnax{c~hj ICj < O} = c~hq
3> m
(B.3.1)
and compute W = B- 13 q •
STEP 3. Check for an unbounded solution.
If wi :::; 0, i=I, ... , m, then the problem is unbounded and the
algorithm terminates. Otherwise proceed to
STEP 4. Determine the pivot row.
Find
m:in
1 $1 $ m
{XBi/Wilwi>O} = XBtlWt
and store w t separately.
STEP 5. Update the basic solution.
x Bt = X Bt/Wtl
xBi = xBi-wiXBt' i=I, ...,mj i:;: t.

STEP 6. Update c, 'Y, and B-1: compute cq , 'Yq afresh as
cq = C/eq-C~ W,
'Yq = 1 + W'W.
Let V = (B'r 1 W.
Update B- 1 to B- 1 and compute
-
aj = e'B--
t
13j ( -
oraj=e'B-
t
1 /
3j w t ) '
Cj = Cj-alq,
rj = max( 'Y j - 2aj3jV + a~'Yq' 1 + a~}
cq = -c/wt ,
-
'Yq = 'Yq / 2
Wt '
j>m,j:;:q.
Let us next consider a steepest-edge modification of the dual simplex
method (Forrest and Goldfarb (1992)). To apply the dual simplex routine to the
preceding primal problem (min! =C'X s.t. AX = b, X ~ 0), let us consider the
associated dual program
Appendix B 465
max 9 = b'U s.t.

Then the dual solution U corresponding to the primal basic feasible solution X B
B- 1 b, X R = 0 must satisfy, under weak complementary slackness,
(a) B'U = C B ,
(B.6)
(b) R'U ~ CR'
In this regard, on a dual simplex step, selecting x Bt to leave the current set of
basic variables amounts to relaxing the tth equation in (B.6.a), i.e., the tth con-
straint a~U ::;: ct can now hold as a strict inequality, where at is the tth column of A.
For a dual solution of the form
U = U + OPt = (B')-lC B + OPt,
where B' p~ = - ct ' steplength 0 ~ 0, a~U = ci,l::;: i ::;: m, i =f. t, and a~U ::;: c t '
the rows Pi' i=l, ... , m, of - B- 1 are taken as the edge directions emanating from
the vertex U of the dual polyhedron {U I A'U ~ C, U € Rm}. So in the dual sim-
plex method the edge direction Pt selected (or the basic variable x Bt chosen to
become nonbasic) must make an acute angle with the gradient b of the dual
objective function or Ptb = - c~B-1b = - x Bt must be positive (or x Bt < 0).
Moreover, if Pt is a steepest edge, then it must be the one that points most
downhill. Hence the direction edge chosen is the one which makes the smallest
acute angle with b, i. e., the P direction or (pili P lI)b is smallest. Thus the
direction edge Pt selected is the edge that is most downhill relative to b or
Pt b b
IIPtl1 = l$t~. {Pi }
I Pill' pjb>O .
As was the case with the primal steepest edge algorithm given above, a key
element underlying steepest-edge pivoting in the dual simplex method is updating
after each iteration the norms II Pi II or the quantities f3 i = PiP~ = I Pi 112 ,
i=l, ... , m. In this regard, if variable x R.q replaces variable x Bt in the set of basic
variables, then the updated basis matrix inverse is
W-c
B- 1 B -1 - ----w;-t Ct'B- 1 '
where W B- 1a q • Then
(a) Pt (lIWt)pt,
(B.7)
(b) Pi Pi - (wdWt)pt, i=l, ... , m, i =f. t,
and
- 2
(a) P t = (l/w t ) Pt ,
(B.8)
(b) 13i = Pi -2(wJWt )Ti + (wJw t )2Pt , i=l, ... ,m, i:f;t,
where B' 0' = et , BT = 0', and Pt = 0"0'.
Forrest and Goldfarb extend their steepest edge simplex algorithm to the
case where upper bounds on the pl'imal variable are present, i.e., to solving a
problem of the form
min! = C'X s.t.
AX~ b, O~X~ U.
By working with the dual variant of this program or with
max 9 = b'y - U'V s.t.
A'Y-V+S = C
Y unrestricted, V ~ 0, S ~ 0,
where S is a vector of dual slack variables, Forrest and Goldfarb apply the (dual)
steepest edge simplex routine and demonstrate that the preceding recursive
formulas (B.8) with T appropl'iately defined (computed) still apply. (For details
see Forrest and Goldfarb (1992, pp. 348-350).)
In addition to the basic primal and dual steepest edge simplex algorithms,
Forrest and Goldfarb offer additional algorithmic embellishments involving a
discussion of approximate steepest-edge algorithms (variants of steepest edge
algorithms in which the edge direction norms used in pivoting are replaced by
approximate weights) as well as projected steepest edge algorithms (modifications
of steepest edge algorithms that are based on, say, the exact norms of subvectors
'fjj comprising only those components of the 'IJ/s in a certain reference framework
which is initially and periodically identified with the current set of nonbasic
variables. )
APPENDIX C:
DERIVATION OF
THE PROJECTION MATRIXl
We mentioned in section 13.3 above that in order to solve problem

(13.5.2) we had to move from Yo = (ft)l in the direction of steepest descent - Cp
to the boundary of the sphere B(Yo' ll'r). Here Cp depicts the orthogonal
projection of the transformed cost vector C onto the null-space of P (or onto the
constraint surface of PY=Po) , where C, P are specified in (13.5.1). Thus at a
feasible point Yo we seek a feasible (descent) direction d satisfying C/d < 0 given
that d must also satisfy Pd = 0 with pep) = m+1, i.e., d is an element of the
tangent subspace .At, = {d I Pd = 0, dfRn} of dimension n - m -1 in R n spanned
by the gradients (the rows of P) of the transformed constraint equations.
Consistent with this requirement, the direction vector selected must be the
orthogonal projection of the gradient C onto the subspace ..Ab.
To obtain this projection let us represent the (m-1 )-dimension subspace
.L
orthogonal to .Ab as A = {X I X = p/~, ~fRm+I} in Rn. Since any vector in
R n can be expressed as the sum of vectors from each of these two complementary
_ I .L
subspaces, we may write C = d + p/~, where d f A, P ~ f ..Ab From this
representation of C we may solve for ~ as follows. Since d = C- p/~,
Pd = PC-pp/~ = 0
lThe reader is encouraged to become familiar with the concepts developed in sections
2.2-2.4 before attempting to read this material.
467
and thus
«pp,)-l exists since the rows of P are linearly independent). Then
d = Cp = C-p'~
= C - P'(pp,)-lpC
= [In - P'(pp,)-lp]C PrC' (C.I)
Here
(C.2)
is termed the projection operator or projection matrix associated with subspace

.Ab, i. e., the operation of P r on any vector in Rn yields the projection of that
vector onto.Ab. Note that Pr = p~, Pr = p~, and PP r =O. If Cp = PrC:f:. 0,
then PCp = 0 and - C p serves as a descent direction on the constraint surface
{Y I PY = Po}· That is, movement along - C p improves the objective value
sinceC'(-d) = C'(-Cp) = (Cp+V)'(-Cp) = -C~Cp<O.
Let us examine the geometry involved in the derivation of equation (C.I).
Again .Ab = {Z I PZ = O,Z€R n } depicts the null-space of P while its
l.
complementary subspace.Ab = {X I X = p'~, ~€ Rm+1}. Let
C = Cp + V = Cp+(C-Cp)f.Ab,
where Cp is the orthogonal projection of Con .Ab and C - Cp is orthogonal to all
_ l.
vectors in .Ab or (C-Cp) €.Ab (Figure (C. I). Hence
o
Figure C.l
Appendix C 469
C - C p must be a linear combination of the columns of P' or C - Cp = P'l.

Since Cp is orthogonal to the columns of P', it follows that
P(C-P'l) = o.
Given that pp' has rank m+1 «pp,)-I exists), the preceding expression yields
l = (pp')-IpC.
1 - 1.
Since P'l = P'(PP')- PC is a vector in .At, , we define
M = P'(pp,)-Ip
1.
to be the orthogonal projection operator or projection matrix onto .At, , i. e., onto the
linear subspace spanned by the columns of P'. Then In - M is the projection oper-
1. _ _
ator orthogonal to.At, or onto.At,. Hence Cp = (In-M)C = PrC or (C.1) above.
Let us consider two alternative approaches to the derivation of (C.1) which
involve the direct solution of a constrained optimization problem. First, consider
minC'h s.t. Ph = 0, h'h = 1, (C.3)
where P is of order (m+l x n) and of full rank m+l :5 n with h a (unit) direction
vector. From the (primal) Lagrangian function
L = C'h + lH -Pl) + l2(1-h'h)
we may write the first-order conditions for a minimum as
(a) Lb = C-P'll -2A2h = 0,

(b) L Al = - Ph = ° (C.4)
(c) L],.2 = 1-d'd = O.
From (CA. a) we have
1 (-
h = 2..\ C-P,II ). (C.5)
2
Substituting this expression into (C.4.b) yields
(C.6)
Both (C.5), (C.6) together allow us to write, using (CA.C), 2A2 = C'h = - 1
or l2 = -! (since C+h,h are orthogonal). Combining this result with (C.5),
(C.6) yields
i.e. h is a unit vector in the direction of - Cpo In sum, the optimal solution to
problem (C.3) is h = - Cp /11 Cp II ,where - Cp is the projection of - C onto the
null-space of P.
Next, an alternative track to the derivation of (C.2) has us solve the least
squares problem
minllC-CplF s.t. PCp = o. (C.S)
Here we seek to make the distance between C and Cp as small as possible given
that Cp is an element of the null-space of P. From the Lagrangian
t = (C-Cp)'(C-Cp) +.\( -PCp)
we find
(a) tcp = 2(C-Cp ) - p'.\' = 0,

(C.9)
(b) t>. = - PCp = o.
From (C.9.a),
(C.lO)
Upon substituting this expression into (C.9.b) we have
(C.Il)
Then inserting (C.Il) into (C.lO) yields

Cp = [In - P'(pp')-lp]C.
An alternative least squares approach to finding the (negative) projected
gradient is offered by Todd (l988a). Specifically, for
ADkl = 0 (the current solution is feasible) implies that

Appendix C 471
t
pp' = AD2A'
k
0] •
0' n
Hence it is sufficient to find a vector U € R m satisfying

(AD~A')U = AD~C,
which are the normal equations for the weighted least squares problem
Having obtained U, let zk = C'Dkl/n. Then the negative projected gradient is

provided by
473
REFERENCES
Adler, I., N. Karmarkar, M.G.C. Resende, G. Veiga. 1989. "An Implementation

of Karmarkar's Algorithm for Linear Programming." Mathematical Pro-
gramming, 44, 297-335.
Anstreicher, K.M. 1986. "A Monotonic Projective Algorithm for Fractional Line-
ar Programming." Algorithmica, 1, 483-494.
Anstreicher, K.M. 1989. "A Combined Phase I - Phase II Projective Algorithm for
Linear Programming." Mathematical Programming, 43, 209-223.
Anstreicher, K.M. 1991. "On the performance of Karmarkar's Algorithm Over a

Sequence of Iterations." SIAM Journal on Optimization, 1, 22-29.
Anstreicher, K.M. 1993. "Strict Monotonicity and Improved Complexity in the

Standard Form Projective Algorithm for Linear Programming." Mathemat-
ical Programming, 62, 517-535.
Anstreicher, K., R. Bosch. 1992. "Long Steps in an O(n3 L) Algorithm for Linear
Programming." Mathematical Programming, 54, 251-265.
Arbel, A. 1993. Exploring Interior-Point Linear Programming. Cambridge, MA:

The MIT Press.
Asic, M.D., V.V. Kovacevic-Vujcic, M.D. Radosavljevic-Nikolic. 1990.

"Asymptotic Behavior of Karmarkar's Method for Linear Programming."
Mathematical Programming, 46, 173-190.
Azpeitia, A.G., D.J. Dickinson. 1964. "A Decision Rule in the Simplex Method
that Avoids Cycling." Numerische Mathematik, 6, 324-331.
Balinski, M.L., W.J. Baumol. 1968. "The Dual in Nonlinear Programming and
its Economic Interpretation." Review of Economic Studies, 35, 237-256.
Balinski, M.L., A.W. Tucker. 1969. "Duality Theory of Linear Programs: A

Constructive Approach with Applications." SIAM Review, 11, 347-377.
Barnes, E.R. 1986. "A Variation of Karmarkar's Algorithm for Solving Linear
Programming Problems. " Mathematical Programming, 36, 174-182.
Barnes, E., S. Chopra, D. Jensen. 1988. "A Polynomial Time Version of the
Affine-Scaling Algorithm." Manuscript, IBM T.J. Watson Research Center,
Yorktown Heights, N.Y.
474
Bayer, D., J.C. Lagarias. 1991. "Karmarkar's Linear Programming Algorithm

and Newton's Method." Mathematical Programming, 50, 291-330.
Bazaraa, M., J.Jarvis, H. Sherali. 1990. Linear Programming and Network Flows,
2nd ed. New York: John Wiley & Sons, Inc.
Beale, E.M.L. 1955. "Cycling in the Dual Simplex Algorithm." Naval Research
Logistics Quarterly, 2, 269-275.
Ben-Israel, A., A. Charnes. 1968. "An Explicit Solution of a Special Class of

Linear Programming Problems." Operations Research, 16, 1166-1175.
Blair, C.E. 1986. "The Iterative Step in the Linear Programming Algorithm of N.
Karmarkar." Algorithmica, 1,537-539.
Bland, R.G. 1977. "New Finite Pivot Rules for the Simplex Method."
Mathematics of Operations Research, 2, 103-107.
Bosch, R.A., K.M. Anstreicher. 1993. "On Partial Updating in a Potential Re-
duction Linear Programming Algorithm of Kojima, Mizuno, and Yoshise."
Algorithmica, 9, 184-197.
Charnes, A. 1952. "Optimality and Degeneracy in Linear Programming."

Econometrica, 20, 160-170.
Charnes, A., W.W. Cooper, A. Henderson. 1953. An Introduction to Linear

Programming. New York: John Wiley & Sons.
Choi, I.C., D. Goldfarb, 1993. "Exploiting Special Structure in a Primal-Dual

Path-Following Algorithm." Mathematical Programming, 58, 33-52.
Choi, I.C., C.L. Monma, D.F. Shanno. 1990. "Further Development of a Primal-
Dual Interior Point Method." ORSA Journal on Computing, 2, 304-311.
Dantzig, G.B. 1951a. "Maximization of a Linear Function of Variables Subject to

Linear Inequalities." In Koopmans, T.C. (ed). Activity Analysis of Produc-
tion and Allocation, Cowles Commission Monograph 13. New York: John
Wiley & Sons, Inc.
Dantzig, G.B. 1951b. "Application of the Simplex Method to a Transportation

Problem." In Koopmans, T.C. (ed). Activity Analysis of Production and
Allocation, Cowles Commission Monograph 13. New York: John Wiley &
Sons, Inc.
475
Dantzig, G.B. 1963. Linear Programming and Extensions. Princeton: Princeton

University Press.
Dantzig, G.B., A. Orden, and P. Wolfe. 1954. Generalized Simplex Method for
Minimizing a Linear Form Under Linear Inequality Restraints, RAND Re-
port RM1264. Santa Monica, CA: The RAND Corp.
Dantzig, G.B., A. Orden, and P. Wolfe. 1955. "The Generalized Simplex Method
for Minimizing a Linear Form Under Linear Inequality Restraints." Pacific
Journal of Mathematics, 5, 183-195.
Dikin, 1.1. 1967. "Iterative Solution of Problems of Linear and Quadratic

Programming." Soviet Mathematics Doklady, 8, 674-675.
Ding, J., T.Y. Li. 1991. "A Polynomial-Time Predictor-Corrector Algorithm for a
Class of Linear Complementarity Problems." SIAM Journal of Optimiza-
tion, 1, 83-92.
Forrest, J., D.Goldfarb, 1992. "Steepest Edge Simplex Algorithms for Linear
Programming." Mathematical Programming, 59, 341-374.
Fraley, C. 1990. "Linear Updates for a Single-Phase Projective Method."

Operations Research Leiters, 9, 169-174.
Freund, R. 1988. "Projective Transformations for Interior Point Methods, Part I:

Basic Theory an Linear Programming." MIT Operations Research Center
Working Paper, OR, 179-188.
Freund, R.M. 1991a. "Polynomial-Time Algorithm for Linear Programming

Based Only on Primal Affine Scaling and Projected Gradients of a Potential
Function." Mathematical Programming, 51, 203-222.
Freund, R.M. 1991b. "A Potential-Function Reduction Algorithm for Solving a

Linear Program Directly From an Infeasible 'Warm Start'." Mathematical
Programming, 52, 441-466.
Freund, R.M. 1993. "Projective Transformations for Interior-Point Algorithms,

and a Superlinearly Convergent Algorithm for the W-Center Problem."
Gale, D. 1960. The Theory of Linear Economic Models. New York: McGrawHill
Book Co., Inc.
Gay, D. 1987. "A Variant of Karmarkar's Linear Programming Algorithm for

Problems in Standard Form." Mathematical Programming, 37, 81-90.
476
de Ghellinck, G., J.P. Vial. 1986. "A Polynomial Newton Method for Linear
Programming." Algorithmica, 1,425-453.
Gass, S.1. 1985. Linear Programming, 5th ed. New York: McGrawHill Book Co.
Gill, P.E., W. Murray, M.A. Saunders, J.A. Tomlin, M.H. Wright. 1986. "On
Projected Barrier Methods for Linear Programming and an Equivalence to
Karmarkar's Projective Method." Mathematical Programming, 36, 183-209.
Goldfarb, D. 1976. "Using the Steepest Edge Simplex Algorithm to Solve Sparse
Linear Programs." In Bunch, J., D. Rose (eds.). Sparse Matrix
Computations. New York: Academic Press.
Goldfarb, D., J.K. Reid. 1977. "A Practicable Steepest Edge Simplex Algorithm."
Goldfarb, D., S. Mehrotra. 1988a. "Relaxed Variants of Karmarkar's Algorithm

for Linear Programming with Unknown Optimal Objective Value." Mathe-
matical Programming, 40, 183-196.
Goldfarb, D., S. Mehrotra. 1988b. "A Relaxed Version of Karmarkar's Method."

Goldfarb, D., M.J. Todd. 1989. "Linear Programming." In Nemhauser, G.L.,

A.H.G. Rinnooy Kan (eds). Optimization, Handbook in Operations Re-
search and Management Science, Vol.1. Amsterdam: Elsevier/North-
Holland Publishing Co.
Goldman, A.J., A.W. Tucker. 1956. "Theory of Linear Programming." In Kuhn,

H.W., A.W. Tucker (eds). Linear Inequalities and Relaxed Systems, Annals
of Mathematical Studies 38. New Jersey: Princeton University Press.
Gonzaga, C. 1988. "Polynomial Affine Algorithms for Linear Programming."

Dept. of Systems Engineering and Computer Science, COPPE-Federal
University of Rio de Janeiro, ES-139/88. Rio de Janeiro, RJ, Brazil.
Gonzaga, C. 1989a. "A Conical Projection Algorithm for Linear Programming."

Gonzaga, C. 1989b. "An Algorithm for Solving Linear Programming Problems in

O(n3L) Operations." In Megiddo, N. (ed). Progress in Mathematical Pro-
gramming: Interior Point and Related Methods. New York: Springer-
Verlag.
477
Gonzaga, C. 1990. "Polynomial Affine Algorithms for Linear Programming."

Gonzaga, C. 1991a. "Large-Step Path-Following Methods for Linear Program-

ming, Part 1: Barrier Function Method." SIAM Journal on Opti-mization,
1,268-279.
Gonzaga, C. 1991b. "Large-Step Path-Following Methods for Linear Program-

ming, Part 2: Potential Reduction Method." SIAM Journal on Optimiza-
tion, 1, 280-292.
Gonzaga, C. 1991c. "Search Directions for Interior Linear Programming Meth-

ods." Algorithmica, 6, 153-181.
Gonzaga, C. 1992. "Path-Following Methods for Linear Programming." SIAM

Review, 34, 167-224.
Gonzaga, C., M.J. Todd. 1992. "An O(nL) Iteration Large-Step Primal-Dual
Affine Algorithm for Linear Programming." SIAM Journal on Optimiza-
tion, 2, 349-359.
Harris, P.M.J. 1973. "Pivot Selection Methods of the Devex LP Code." Mathe-
matical Programming, 5, 1-28.
den Hertog, D., C. Roos, T. Terlaky. 1992. "A Complexity Reduction for the
Long-Step Path-Following Algorithm for Linear Programming." SIAM
Journal on Optimization, 2, 71-87.
Hoffman, A. 1953. Cycling in the Simplex Algorithm. Washington, D.C.:

National Bureau of Standards.
Ishihara, T., M. Kojima. 1993. "On the Big M in the Affine Scaling Algorithm."
Jan, G.M., S.C. Fang. 1991. "A Variant of Primal Affine Scaling Algorithm for
Linear Programs." Optimization, 22, 681-715.
Ji, J., F.Potra, S. Huang. 1991. "A Predictor-Corrector Method for Linear Com-
plementarity Problems with Polynomial Complexity and Super linear
Convergence." Report No. 18, University of Iowa, Iowa City, Iowa.
Karmarkar, N. 1984. "A New Polynomial Time Algorithm for Linear Program-
ming." Combinatorica, 4, 373-395.
478
Karush, W. 1939. "Minima of Functions of Several Variables with Inequalities as

Side Conditions." M.S. Thesis, Department of Mathematics, Univeristy of
Chicago.
Klee, V., G.L. Minty. 1992. "How Good is the Simplex Algorithm?" In Shisha,
O. (ed). Inequalities III. New York: Academic Press.
Kojima, M. 1986. "Determining Basic Variables of Optional Solutions in

Karmarkar's New Algorithm." Algorithmica, 1, 499-515.
Kojima, M., N. Megiddo, T. Noma, A. Yoshise. 1991a. A Unified Approach to

Interior Point Algorithms for Linear Complementarity Problems. New
York: Springer-Verlag.
Kojima, M., N. Megiddo, Y.Ye. 1992. "An Interior Point Potential Reduction
Algorithm for the Linear Complementarity Problem." Mathematical Pro-
gramming, 54, 267-279.
Kojima, M., S. Mizuno, A. Yoshise. 1989a. "A Polynomial-Time Algorithm for a

Class of Linear Complementarity Problems." Mathematical Programming,
44, 1-26.
Kojima, M., S. Mizuno, A. Yoshise. 1989b. "A Primal-Dual Interior Point

Method for Linear Programming." In Megiddo, N. (ed). Progress in Mathe-
matical Programming: Interior Point and Related Methods. New York:
Springer-Verlag.
Kojima, M., Y. Kurita, S. Mizuno. 1993. "Large-Step Interior Point Algorithms

for Linear Complementarity Problems." SIAM Journal on Optimization, 3,
398-412.
Kojima, M., N. Megiddo, S. Mizuno. 1993a. "A Primal-Dual Infeasible Interior

Point Algorithm for Linear Programming." Mathematical Programming,
61, 263-280.
Kojima, M., N. Megiddo, S. Mizuno. 1993b. "Theoretical Convergence of Large-

Step Primal-Dual Interior Point Algorithms for Linear Programming."
Kojima, M., S. Mizuno, A. Yoshise. 1991b. "An O(nL) Iteration Potential

Reduction Algorithm for Linear Complementarity Problems." Mathemat-
ical Programming, 50, 331-342.
Kojima, M., S. Mizuno, A. Yoshise. 1993b. "A Little Theorem of the Big M in
Interior Point Algorithms." Mathematical Programming, 59, 361-375.
479
Kortanek, K.O., J. Zhu. 1988. "New Purification Algorithms for Linear Program-
ming." Naval Research Logistics, 35, 571-583.
Kuhn, H., R. Quandt. 1963. "An Experimental Study of the Simplex Method."
Proceedings of the Symposia in Applied Mathematics, Vol. XV, American
Mathematical Society.
Lemke, C.E. 1954. "The Dual Method for Solving Linear Programming Prob-
lem." Naval Research Logistics Quarterly, 1, 48-54.
Lemke, C.E. 1965. "Bimatrix Equilibrium Points and Mathematical Program-

ming." Management Science, 11, 681-689.
Luenberger, D.G. 1984. Linear and Nonlinear Programming, 2nd ed. Reading
MA: Addison-Wesley Publishing Co., Inc.
Lustig, I.J., R.E. Marsten, D.F. Shanno. 1991. "Computational Experience with
a Primal-Dual Interior Point Method for Linear Programming." Linear
Algebra and Its Applications, 152, 191-222.
Marshall, K.T., J.W. Suurballe. 1969. "A Note on Cycling in the Simplex
Method." Naval Research Logistics Quarterly, 16, 121-137.
Marsten, R., M. Saltzman, M. Shanno, D. Pierce, J. Ballintijn. 1989. "Imple-

mentation of a Dual Affine Interior Point Algorithm for Linear
Programming." ORSA Journalon Computing, 1, 287-297.
McShane, K. 1994. "Superlinearly Convergent O(nL) Iteration Interior-Point

Algorithms for Linear Programming and the Monotone Linear Complemen-
tarity Problem." SIAM Journal on Optimization, 4, 247-261.
McShane, K.A., C.L. Monma, D. Shanno. 1989. "An Implementation of a

Primal-Dual Interior Point Method for Linear Programming." ORSA
Journal on Computing, 1, 70-83.
Megiddo, N. 1986. "Introduction: New Approaches to Linear Programming."

Algorithmica, 1, 387-394.
Megiddo, N. 1989a. Progress in Mathematical Programming: Interior-Point and

Related Methods. New York: Springer-Verlag.
Megiddo, N. 1989b. "Pathways to the Optimal Set in Linear Programming." In

Megiddo, N. (ed). Progress in Mathematical Programming: Interior-Point
and Related Methods. New York: Springer-Verlag.
480
Megiddo, N., M. Shub. 1989. "Boundary Behavior of Interior Point Algorithms in

Linear Programming." Mathematics of Operations Research, 14, 97-146.
Mehrotra, S. 1991. "On Finding a Vertex Solution Using Interior Point

Methods." Linear Algebra and its Applications, 152, 106-111.
Mehrotra, S., R. Stubbs. 1994. "Predictor-Corrector Methods for a Class of

Linear Complementarity Problems." SIAM Journal on Optimization, 4,
441-453.
Mizuno, S. 1992. "A New Polynomial Time Method for a Linear Complemen-
tarity Problem." Mathematical Programming, 56, 31-43.
Mizuno, S., A. Nagasawa. 1993a. "A Primal-Dual Affine-Scaling Potential-

Reduction Algorithm for Linear Programming." Mathematical Program-
ming, 62, 119-131.
Mizuno, S., M.J. Todd, Y. Yeo 1993b. "On Adaptive-Step Primal-Dual Interior
Point Algorithms for Linear Programming." Mathematics of Operations
Research, 18, 964-981.
Mizuno, S. 1994. "Polynomiality of Infeasible Interior-Point Algorithms for

Monma, C.L., A.J. Morton. 1987. "Computational Experience with a Dual Affine
Variant of Karmarkar's Method for Linear Programming." Operations
Research Letters, 6, 261-267.
Monteiro, R.C. 1991. "Convergence and Boundary Behavior of the Projective

Scaling Trajectories for Linear Programming." Mathematics of Operations
Research, 16, 842-858.
Monteiro, R.C., I. Adler. 1989. "Interior Path Following Primal-Dual

Algorithms. Part 1: Linear Programming." Mathematical Programming, 44,
27-42.
Monteiro, R.D.C., I. Adler, M.G.C. Resende. 1990. "A Polynomial-Time Primal-

Dual Affine Scaling Algorithm for Linear and Convex Quadratic
Programming and Its Power Series Extension." Mathematics of Operations
Research, 15, 191-214.
Padberg, M.W. 1986. "A Different Convergence Proof of the Projective Method
for Linear Programming." Operations Research Letters, 4, 253-257.
481
Panik, M.J. 1976. Classical Optimization: Foundations and Extensions. The

Netherlands: North-Holland Publishing Co.
Panik, M.J. 1993. Fundamentals of Convex Analysis. Boston MA: Kluwer Aca-
demic Publishers.
Perold, A.F. 1980. "A Degeneracy Exploiting LU Factorization for the Simplex
Method." Mathematical Programming, 19, 239-254.
Potra, F. 1992. "On a Predictor-Corrector Method for Solving Linear Programs

From Infeasible Starting Points." Reports on Computational Mathematics
No. 34, Dept. of Mathematics, University of Iowa, Iowa City, Iowa.
Potra, F. 1994. "A Quadratically Convergent Predictor-Corrector Method for

Solving Linear Programs from Infeasible Starting Points." Mathematical
Renegar, J. 1988. "A Polynomial-Time Algorithm Based on Newton's Method for

Rockafellar, R.T. 1970. Convex Analysis. Princeton, NJ: Princeton University

Press.
Roos, C. 1989. "A New Trajectory Following Polynomial-Time Algorithm for

Linear Programming Problem." Journal of Optimization Theory and Appli-
cations, 63, 433-458.
Shanno, D.F. 1988. "Computing Karmarkar Projection Quickly." Mathematical

Shanno, D.F., R.L. Weil. 1971. "Linear Programming with Absolute Value Func-
tionals." Management Science, 16, 408.
Shapiro, J. 1979. Mathematical Programming: Structures and Algorithms. New

York: John Wiley & Sons, Inc.
Shaw, D., D. Goldfarb. 1991. "A Primal Projective Interior Point Method for
Shaw, D., D. Goldfarb. 1994. "A Path-Following Projective Interior Point

Method for Linear Programming." SIAM Journal on Optimization, 4, 65-85.
Shub, M. 1987. "On the Asymptotic Behavior of the Projective Rescaling

Algorithm for Linear Programming." Journal of Complexity, 3, 258-269.
482
Sonnevend, G., J. Stoer, G. Zhao. 1989. "On the Complexity of Following the
Central Path of Linear Programs by Linear Extrapolation." Methods of
Operations Research, 63, 19-31.
Tapia, R.A., Y. Zhang. 1990. "Cubically Convergent Method for Locating a

Nearby Vertex in Linear Programming." Journal of Optimization Theory
and Applications, 67, 217-225.
Tardos, E. 1986. "A Strongly Polynomial Algorithm to Solve Combinatorial

Linear Programs." Operations Research, 34, 250-256.
Todd, M.J. 1986a. "Large Scale Linear Programming-Geometry, Working Bases,

and Factorization." Mathematical Programming, 26, 1-20.
Todd, M.J. 1986b. "Polynomial Expected Behavior of a Pivoting Algorithm for

Linear Complementarity and Linear Programming Problems." Mathemati-
cal Programming, 35, 173-192.
Todd, M.J. 1998a. "Improved Bounds and Containing Ellipsoids in Karmarkar's

Linear Programming Algorithm." Mathematics of Operations Research, 13,
650-659.
Todd, M.J. 1988b. "Exploring Special Structure in Karmarkar's Linear Program-

ming Algorithm." Mathematical Programming, 41, 97-113.
Todd, M.J. 1992. "On Anstreicher's Combined Phase I - Phase II Projec-tive

Algorithm for Linear Programming." Mathematical Programming, 55, 1-16.
Todd, M.J. 1993. "Combining Phase I and Phase II in a Potential Reduction

Algorithm for Linear Programming." Mathematical Programming, 59, 133-
150.
Todd, M.J., B.P. Burrell. 1986. "An Extension to Karmarkar's Algorithm for
Linear Programming Using Dual Variables." Algorithmica, 1, 409-424.
Todd, M.J., Y. Wang. 1993. "On Combined Phase 1 - Phase 2 Projective Meth-
ods for Linear Programming." Algorithmica, 9, 64-83.
Todd, M.J., Y. Yeo 1990. "A Centered Projective Algorithm for Linear Pro-
gramming." Mathematics of Operations Research, 15, 508-529.
Tomlin, M.J. 1987. "An Experimental Approach to Karmarkar's Projective

Method for Linear Programming." Mathematical Programming Studies, 31,
175-191.
483
Tseng, P., Z-Q. Luo. 1992. "On the Convergence of the Affine-Scaling Algori-
thm." Mathematical Programming, 56, 301-319.
Tsuchiya, T. 1991. "Global Convergence of the Affine Scaling Methods for

Degenerate Linear Programming Problems." Mathematical Programming,
52,377-404.
Vanderbei, R.J. 1989. "Karmarkar's Algorithm and Problems With Free

Variables." Mathematical Programming, 43, 31-44.
Vanderbei, R.J., T. Carpenter. 1993. "Symmetric Indefinite Systems for Interior

Point Methods." Mathematical Programming, 58, 1-32.
Vanderbei, R.J., J.C. Lagarias. 1990. "1.1. Dikin's Convergence Result for the
Affine-Saling Algorithm." Contemporary Mathematics, 114, 109-119.
Vanderbei, R.J., M.S. Meketon, B.A. Freedman. 1986. "A Modification of

Karmarkar's Linear Programming Algorithm." Algorithmica, 1, 395-407.
Vial, JP. 1989. "A Unified Approach to Projective Algorithms for Linear
Programming." Optimization Fifth French-German Conference, Castel
Novel 1988, Lecture Notes in Mathematics 1405. New York: Springer-
Verlag, 191-220.
Williams, A.C. 1963. "Marginal Values in Linear Programming." SIAM Journal

of Applied Mathematics, 11, 82-94.
Wolfe, P. 1963. "A Technique for Resolving Degeneracy in Linear

Programming." SIAM Journal, 11, 205-211.
Wolfe, P. 1965. "The Composite Simplex Algorithm." SIAM Review, 7,42-54.
Wright, S. 1994. "An Infeasible-Interior-Point Algorithm for Linear

Complementarity Problems." Mathematical Programming, 67, 29-51.
Ye, Y. 1987. "Karmarkar's Algorithm and the Ellipsoid Method." Operations

Research Letters, 4, 177-182.
Ye, Y. 1988. "A Class of Potential Functions for Linear Programming."

Manuscript, Department of Management Sciences, The University of Iowa,
Iowa City, Iowa.
Ye, Y. 1990a. "An O(n 3 L) Potential Reduction Algorithm for Linear

Programming." Contemporary Mathematics, 114, 91-107.
484
Ye, Y. 1990b. "Recovering Optimal Basic Variables in Karmarkar's Polynomial

Algorithm for Linear Programming." Mathematics of Operations Research,
15, 564-572.
Ye, Y. 1990c. "A Class of Projective Transformations for Linear Programming."
SIAM Journal on Computing, 19, 457-466.
Ye, Y. 1992. "A Potential Reduction Algorithm Allowing Column Generations."
SIAM Journal on Optimization, 2, 7-20.
Ye, Y., K. Anstreicher. 1993a. "On Quadratic and O(n3 L) Convergence of a
Predictor-Corrector Algorithm for LCP." Mathematical Programming, 62,
537-551.
Ye, Y., O. GiBer, R.A. Tapia, Y. Zhang. 1993b. "A Quadratically Convergent
O(nL) Iteration Algorithm for Linear Programming." Mathematical Pro-
gramming, 59, 151-162.
Ye, Y., M. Kojima. 1987. "Recovering Optimal Dual Solutions in Karmarkar's
Polynomial Algorithm for Linear Programming." Mathematical Program-
ming, 39, 305-317.
Ye, Y., P. Pardalos. 1989. "A Class of Linear Complementarity Problems
Solvable in Polynomial Time." Manuscript, Department of Management
Sciences, The University of Iowa, Iowa City, Iowa.
Ye, Y., R.A. Tapia, Y. Zhang. 1991. "A Superlinearly Convergent O(nL)
Iteration Algorithm for Linear Programming." Technical Report TR9122,
Department of Mathematical Sciences, Rice University, Houston, TX.
Ye, Y., M.J. Todd. 1990. "Containing and Shrinking Ellipsoids in the Path-
Following Algorithm." Mathematical Programming, 42, 1-9.
Ye, Y., M.J. Todd, S. Mizuno. 1994. "An O(nL) Iteration Homogeneous and Self-
Dual Linear Programming Algorithm." Mathematics of Operations Re-
search, 19, 53-68.
Yudin, D.B., E.G. Gol'shtein. 1965. Linear Programming. Jerusalem: Israel
Program for Scientific Translations Ltd.
Zhang, Y. 1994. "On the Convergence of a Class of Infeasible Interior Point
Methods for the Horizontal Linear Complementarity Problem." SIAM
Journal on Optimization, 4, 208-227.
Zhang, Y., R.A. Tapia, J.E. Dennis. 1992. "On the Superlinear and Quadratic
Convergence of Primal-Dual Interior-Point Linear Programming Algori-
thms." SIAM Journal on Optimization, 2, 304-324.
485
NOTATION INDEX
R set of real numbers

tP null or empty set
X = (Zi) the vector whose ith component is zi
X~O Zi ~ 0 for all i
X>O zi > 0 for all i
1 the sum vector containing all 1's
o the null vector/matrix containing all O's
identity matrix of order n
e·I ith unit column vector
X, transpose of a vector X
X-Y scalar product of vectors X, Y
II - II Euclidean norm
X/IIXII direction of vector X
IIX-yll distance between points X, Y
A an (m x n) matrix A
p(A) rank of a matrix A
IAI determinant of an nth order matrix A
diag(X) a diagonal matrix whose elements are the components of X
Col(A) column space of A
Row(A) row space of A
N(A) null-space of A
n(A) nullity of A
At pseudo-inverse of A
A-I inverse of A
Xc:::r X is an element of a set :r
X/:r X is not an element of a set :r
.A.~~ set .A. is a subset of set ~ (possibly .A. = ~)
.A.C~ set .A. is a proper subset of set ~

486
Rn n-space or the set of all real n-component vectors

R++ set of all real n-component vectors with strictly positive
components
A subspace of Rn
dim(A) dimension of subspace A
€(A) linear hull of subspace A
Al $ A2 direct sum of subspaces AI' A2
Ai orthogonal complement or dual subspace of A
smallest integer ~ x
largest integer not greater than x
rth order polynomial of problem size n
arg min minimization with respect to a particular argument

d(:f) diameter of a set :f
d(X, :f) distance between vector X and set :f
d(:f, ~) distance between sets :f, ~
B(Xo,8) 8-neighborhood of a point Xa € R n (or an open ball or
sphere of radius 8 about Xo)
e cone
e+ polar cone
e* dual cone
(a) ray or halfline
(a) i orthogonal cone to (a)
(a)+ polar of (a)
(a)* dual of (a)
:f0 in terior of set :f
exterior of set :f
complementary set of :f
"1 closure of set :f
8(:f) boundary of set :f
coni(:f) conical hull of set :f
487
alJ('.!) affine hull of set '.!

co('.!) convex hull of set '.!
ri('.!) relative interior of convex set '.!
ro('.!) relative boundary of set '.!
sup supremum or least upper bound
inl infimum or greatest lower bound
lim sup limit superior (or upper limit)
.Ax~ Cartesian product of sets .A, ~
s.t. subject to
V gradient (del operator)
{X k } seqeunce of points
% hyperplane
[%+], [%-] closed half-planes
(%+),(%-) open half-planes
(Jk k-dimensional simplex
I: Rn-+R function or mapping from R n to R
1-1 single-valued inverse mapping
T:Rn-+Rm transformation from R n to R m
T- 1 inverse transformation
Y= T(X) image of X under transformation T
D(T) domain of T
R(T) range of T
~(Xo) set of feasible directions at X
~(Xo)+ polar support cone
~(Xo)* dual support cone
%p (or ~) primal feasible region
%D dual feasible region
LCP(q,M) linear complementarity problem
lim upper limit
lim lower limit
489
INDEX
Absolute value functional 249 degenerate 46, 83, 131, 186, 187
Adler, I. 380 multiple 180-184
Affine combination 28 non-basic variables 46, 83
Affine potential reduction non-degenerate 47
algorithm 448, 407 optimal 115, 118
dual 410 unbounded 184, 185
modified primal 409 Basis
Affine scaling algorithm dual feasible 118, 221
dual 375, 378 inverse 46, 83, 117, 459
primal 369, 372 primal feasible 113
Affine set 28 see vector space (subspace)
dimension of 28 Baumol, W.J. 153
a-center neighborhood 419, 437 Bayer, D. 474
Anstreicher, K. 359, 361-365, 397, Bazaraa, M. 353
398,411,417, 421, 435, 445, 446, Beal, E.M. 134
454 Ben-Israel, A. 248
Arbel, A. 369, 375, 380 Blair, C.E. 359
Artificial constraint method 261 Bland, R.G.D. 135
Artificial problem 199 Bounded variables 236
unbounded solution to 211 bounded above 236
Artificial objective function 199 bounded below 236
Artificial slack variable 261 Bosch, R. 398, 411, 417
Artificial variable 196 Burrell, B.P. 359-361
Artificial vector 197
Asic, M.D. 369 Capacitated equation system 236
Azpeita, A.G. 135 Carpenter, T. 396
Centering transformation
Balinski, M.L. 134, 153 affine 18, 369
Ball optimization problem 367 projective 17,343,345,366
subproblem 448, 450 Central path (trajectory) 379,382,
Ballintijn, J. 479 388, 436, 451
Barnes, E.R. 361, 369 Charnes, A. 135, 198, 248
Barrier problem Choi, I.C. 396, 397
barrier parameter 380 Chopra, S. 473
barrier term 380, 423 Combined primal-dual problem 357
dual 380 Complementary slackness
logarithmic barrier function 380 theorem
primal 380 strong 108
Basic solution 46, 81, 83 weak 105, 166
basic feasible 47, 83, 84, 169 Complexity of an algorithm 352
basic variable 46, 83 exponential 352
490
input length 352 Convex polytope 33, 53, 64

order of an algorithm 352 Convex set 31
polynomial 352 extreme point of 54
Composite simplex method 225 locally convex 34
entry criterion 227 relative boundary of 34
exit criterion 229 relative interior of 34
infeasibility reduction term 227 relatively open 34
total infeasibility 226 Cooper, W.W. 474
Cone(s) 39 Critical value 320, 326, 330
base of 39 see parametric analysis
closed 41, 43 Cycling 134
closure of 43
conical hull 43 Dantzig, G.W. 111, 135, 213, 264,
convex 39 268
dimension of 40, 43 Degenerate solution 46, 83, 186
dual 41,43 de Ghellnick, G. 361, 365
dual support 144 den Hertog, D. 477
finite 41 Dennis, J .E. 484
finitely generated 41, 42 Dickinson, D.J. 135
generated 39, 41 Dikin, I.I. 369
intersection of 41 Ding, J. 475
negative 40 Directional derivative 157
negative polar 43 Direct sum 12
orthogonal 40 Dual degeneracy 259
pointed 40 Dual simplex method 252, 260
polar 41,43 entry criterion 255
polar support 144 exit criterion 254
polyhedral convex 42, 44, 56 Dual linear program 3
solid 40 Lagrangian of 160
sum of 41 see linear program
supporting half-space for 44 symmetric 90
supporting hyperplane for 44 unsymmetric 92
tangent support 142 Dual variable 90, 151, 190
vertex 39 marginal imputed value 120,
Conical combination 43 152, 156
Constraints 2, 69 shadow price 120
artificial 196, 261, 265 simplex multiplier 120
compactifying 356 Duality gap 101
inconsistent 203-207, 215 relative 373, 386
redundant 55, 203-207, 216 Duality theorem
superfluous 55, 156 strong 100, 165
Convex combination 32 weak 95, 119
Convex polyhedron
see polyhedral convex set Entry criterion 125
491
Exit criterion 51, 127 Half-plane 29

Extreme point(s) 54, 56, 60 closed 29, 34
adjacent 56, 112 open 29
degenerate 54 supporting 35
Extreme point theorem 57 Harris, P.M.J. 461
Henderson, A. 474
Farkas, J. 21 Hoffman, A. 134
Fang, S.C. 477 Homogeneous and self-dual
Farkas' theorem 21, 47, 101 program 424
Feasible direction(s) 142 corrector step 430
set of 142 predictor step 430
Feasible region 2, 69, 73, 141 self-central path 429
augmented 73 strictly self-complementary
center of 437 solution 427
dual 90 Huang, S. 477
primal 90 Hyperplane 29, 34
Feasible solution 70, 253 normal of 29, 35
Finite basis theorem 63, 64 separating 35
Ford, L. 264 strictly 35
Forrest, J. 461 strongly 35
Fraley, C. 365 weakly 35
Freedman, B.A. 483 supporting (tangent) 35, 54
Freund, R. 366, 398, 401, 403, properly 35
412, 413, 421, 422
Fulkerson, D. 264 Infeasibility form 213
Interior feasible point 369, 415,
Gale, D. 475 418, 424, 437
Gass, S.1. 135 Interval linear programming 247
Gay, D. 359, 361 Ishihara, T. 374
Generic primal-dual algorithm 391
affine scaling direction 393 Jan, G.M. 477
centering direction 392 Jarvis, J. 474
primal and dual steplength Jensen, D. 473
parameter 392 Ji, J. 435
search direction parameter 392
Gill, P.E. 380 Karmarkar, N. 17,341,353,357,
Goldfarb, D. 353, 357, 365, 396, 366, 397
397, 461 algorithm 343, 354
Goldman, A.J. 63,64,97-100, 102 Karmarkar standard form 341
Gol'shtein, E.G. 134 Karush, W. 478
Gonzaga, C.G. 361, 398-400, 411, Karush- Kuhn-Tucker theorem
412,414,417,420,421 148, 167
GiBer, O. 429 conditions 148, 151, 163, 164, 380
equivalence theorem 163
492
Klee, V. 352 Linear equality constraints

Kojima, M. 359, 361, 374, 380, 382 basic solution 46
384, 387, 388, 391, 393-396, 398 basic variables 46
414-417,421,434-444,446,451-458 degenerate 46
Kortanek, K.O. 479 feasible 47
Kovacevic-Vujcic, V.V. 473 non basic variables 46
Kuhn, H. 148, 151, 163, 164,380,461 nondegenerate 47
Kurita, Y. 436, 451 homogeneous system
associated with 52
Lagarias, J .C. 369 Linear (inequality) system
Least squares problem 25, 470 consistent 18
residual vector 25, 373 dual 22
weighted 360, 371, 385, 471 homogeneous 15
Legitimate variable 196 inconsistent 18
Lemke, C.E. 252, 277 least-squares solution 24, 25
Li, T.Y. 475 nonhomogeneous 18
Line 40 normal equations 25
Linear combination 10 overdetermined 24
Linear complementarity problem 277 redundant 45, 82
almost complementary basic residual 25
solution 282 self-dual 23
complementarity condition 278 underdetermined 45, 82
complementary basic solution 278 Linear functional 66
complementary cone 280 Linear program 2, 69, 72
complementary pair of variables augmented structural constraints
278 72
complementary pair of vectors 280 canonical form 69
complementary pivot rule 284 dual form 3, 90
complementary set of vectors 280 non-negativity conditions 2, 69
complementary solution 278, 446 objective function 2, 69, 72
complements 278 minimization of 70, 233
feasible solution 278 primal form 3, 89
horizontal 455 standard form 72, 81
ray termination 285 structural constraints 2, 69
Linear complementarity problem Luenberger, D.G. 101
and interior point methods 433 Luo, Z-Q. 379
central path 451 Lustig, I.J. 386
€-complementary solution 442
equivalent quadratic program Marshall, K.T. 134
435,442 Marsten, R.E. 375, 379, 386
large step 451 Matrix
path following 436, 438, 453 augmented 19
potential reduction 439, 441, 442 column space of 13
predictor-corrector 445-448 nullity of 15
493
positive semidefinite 285 Optimality theorem 115

pseudoinverse of 25 Orden, A. 135
rank of 13, 15, 22 Orthogonal projection 12, 24, 376,
row space of 13 445, 449, 467
skew-symmetric 23, 96, 425
McShane, K. 380, 382, 387, 388, 454 Padberg, M.W. 359
Megiddo, N. 380, 391, 394, 395, 435, Panik, M. 4, 148
436, 442, 456, 458 Parametric analysis of
Mehrotra, S. 357,429,435,448-451 activity vector 329
Meketon, M.S. 483 objective function 320
Merit function 456 requirements vector 325
Metric 9 Parametric lower bound 405
Minkowski, H. 38, 62 Parametric potential function 405
Minkowski-Farkas theorem 147 Parametric primal-dual method 334
see Farkas' theorem artificial problem 335
Minty, G.L. 352 extended artificial problem 336
Mizuno, S. 374,388,389,391, 393- Parametric upper bound 405
396, 424, 425, 429, 434-436, 440, Pardalos, P. 484
451, 456-458 Perold, A.F. 135
Monma, C.L. 375 Pierce, D. 479
Monteiro, R.C. 380 Pivot matrix 175
Morton, A.J. 375 Pivot operation 174
Motzkin, J.J. 63 entry criterion 175
M-penalty method 198 exit criterion 176
It-complementarity relation 381 pivot term 174
Multiple optimal solutions 137, Plane of support theorem 36
180-184 Polyhedral convex set 32, 53
Murray, W. 476 convex polytype 33, 53
dimension of 55
Nagasawa, A. 424 extreme direction of 61, 127
Neighborhood (open sphere) 26 extreme point (vertex) of 54
Newton's method 383 adjacent 56
Noma, T. 456,458 degenerate 54
Non-degeneracyassumption 113 face of 55
Norm 9 edge 56
Cauchy-Schwarz inequality 9 facet 56
homogeneity 9 proper 55
triangle inequality 9 finite basis for 63
pointed 55
Optimal solution 70 recession direction for 60
existence theorem for 77 representation for 53
Optimal value function 153 resolution theorem for 63
Optimality evaluators 116 supporting hyperplane for 54
relative cost 121 unbounded 62
494
Potential function 344, 366, 398,401, Projection matrix (operator) 367,

405,412,415,418,422,440,443,458 384, 399, 406, 408, 410, 416, 468
tranformed 350, 399, 444 orthogonal 384, 467-469
Potential reduction theorem 407 Projected Newton step directions
Potra, F. 396 383, 384
Predictor-corrector algorithm 388,390 Purification process 355
affine scaling direction 393
centering direction 392 Q-quadratic convergence 390, 435, 436
corrector step 390 Q-superlinear convergence 436
predictor step 389 Quandt, R. 461
search direction parameter 392
steplength parameter 392 Radosavljevic-Nikolic, M.D. 473
Primal-dual interior point algorithm Rank-nullity theorem 17
379,420 Rao, M. 250
generic 391 Ray (half-line) 42
path following 385 dual of 42
Primal-dual method 264, 274 orthogonal cone to 42
extended primal 266 polar of 42
modified dual 265 Reduced cost (benefit)
modified primal 265 coefficient 115
optimality theorem 268 see optimality evaluator
restricted basis entry 264, 267 Reid, J.K. 461
restricted dual 268 Renegar, J. 481
restricted primal 267 Representation theorem 37, 60, 62
robust 276 Requirements space 85
Primal-dual potential reduction Requirements vector 85
algorithm 416 Resende, M.G.C. 473
Primal linear program Resolution theorem 52, 63
see linear program Rockafellar, R. T. 29
Lagrangian of 118, 149, 160, Roos, C. 477
469,470 Roundness restriction 342
reduced form 114
Primal potential reduction Saddle point 160
algorithm 400 Lagrangian 160
extended 403 primal optimality 162
Primal simplex method 125, 126, 169 problem 161
221 Safeguard line search 412
algorithm 128, 187 Saltzman, M. 479
entry criterion 125, 175 Saunders, M.A. 476
exit criterion 127, 176 Scaled linear program 348, 369
Projected gradient 348, 369, 370, 399 restriction of 348
401-403, 412, 413, 419, 423, 444, Sensitivity analysis of
445, 467-471 coefficient matrix 300
objective function 290
495
requirements vector 296 k-plane 65

Separation theorem n-simplex 65
strong 35, 36, 38, 39 standard 65
weak 38 sUbsimplex 65
Set(s) unit 65
affine 28 Simplex matrix 171
affine hull of 30 Simplex method 111
boundary of 26 bounded variables 243
boundary point of 26 composite 225
bounded 26 dual 252, 260
Cartesian product 9 primal 125, 126, 187
closed 27 primal-dual 264, 274
closure of 27 Slack variable 72
compact 28 Solutions space 85
complementary 26 Sonnevend, G. 388
convex hull of 32 Standard form projective
closed 33 algorithm 362
convex set 32 Steepest edge simplex algorithm
diameter 26 461
distance between sets 27 approximate 466
distance from a vector 27 dual 464
exterior 26 edge direction 461, 462
exterior point of 26 primal 463
interior 26 projected 466
interior point of 26, 55 reduced costs 462
limit point of 27 Step parameter 349
linear variety (affine set) 28 Stoer, J. 388
locally compact 28 Structural changes
open 26 adding a constraint 314
point of closure 27 adding a variable 314
translated 28, 30 deleting a constraint 317
Shanno, D.F. 250, 359, 386 deleting a variable 317
Shapiro, J. 101, 158 Stubbs, R. 435, 448-451
Shaw, D. 365,397 Subspace 11
Sherali, H. 474 affine 30
Shift vector 422 basis 12
Shub, M. 480 dimension 12
Simplex 64 orthogonal complement 12
boundary of 65 orthogonal projections 12
closure of 66 span or linear hull 11
face of 65, 66 spanning set 11
carrier 66 translation of 28
facet of 66, 351 Sufficient optimality criterion 96
k-dimensional 65 Surplus variable 90
496
Surrogate problem 213 Vector space (Rn) 9

Suurballe, J.W. 134 basis 11
dimension 11
Tapia, R.A. 482, 484 spanning set 11
Tardos, E. 482 see subspace
Terlaky, T. 477 standard basis 15
Theorem of the alternative 20 Vector(s) 9
Todd, M.J. 353, 359-362, 364, 366, affinely independent 30
395,398,404,414,417,420-422,425 components 9
Tomlin, M.J. 476 cosine of angle between 10
Tseng, P. 379 direction 10
Tsuchiya, T. 379 distance between 10
Transformation (function) 13 length 10
affine 18, 369, 375 linearly dependent 10
domain 13 linearly independent 10
identity 14 orthogonal 10
image 13 orthogonal projection 12
invertible 14 scalar product 9
linear 13 Veiga, G. 473
see linear functional Vial, J.P. 361,365
nonsingular 14
null-space 15 Wang, Y. 365
one-to-one 14 Weierstrass's theorem 67
onto 14 Weil, R. 250
projective 17,345,366 Whinston, A. 335
range 13, 15 Williams, A.C. 153
rank 15 Wolfe, P. 135, 225
zero 14 Wright, M.H. 476
zero subspace 15 Wright, S. 456
Trust region subproblem 449, 450
Tucker, A.W. 22,23,96-100, 102, 134 Ye, Y. 359, 361, 366, 368, 388-390,
Two-phase method 213 395, 398, 404, 406-413, 421-431,
phase I 213 435, 442, 445-447, 454
phase II 215 Yoshise, A. 374, 393-395, 434, 436
440, 456, 458
Unbounded objective function 136, Yudin, D.B. 134
184, 185
Unrestricted variables 234 Zhang, Y. 412, 455
Zhao, G. 388
van de Panne 335 Zhu, J. 479
Vanderbei, R.J. 369,371,374,396
Applied Optimization
1. D.-Z. Du and D.F. Hsu (eds.): Combinatorial Network Theory. 1996

ISBN 0-7923-3777-8
2. M.J. Panik: Linear Programming: Mathematics, Theory and Algorithms. 1996
ISBN 0-7923-3782-4
3. RB. Kearfott and V. Kreinovich (eds.): Applications of Interval Computations.
1996 ISBN 0-7923-3847-2
KLUWER ACADEMIC PUBLISHERS - OORDRECHT / BOSTON / LONDON

Linear Programming Mathematics Theory and Algorithms

Uploaded by

Copyright:

Available Formats

Linear Programming Mathematics Theory and Algorithms

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Linear Programming Mathematics Theory and Algorithms

Uploaded by

Copyright:

Available Formats

Linear Programming:

Mathematics, Theory and Algorithms

KLUWER ACADEMIC PUBLISHERS

ISBN-13: 978-1-4613-3436-1 e-ISBN-13: 978-4613-3434-7

1. Linear programming. I. Title. II. Series.

Published by Kluwer Academic Publishers,

Kluwer Academic Publishers incorporates

Sold and distributed in the U.S.A. and Canada

In all other countries, sold and distributed

Printed on acid-free paper

All Rights Reserved

Chapter 1. INTRODUCTION AND OVERVIEW 1

Chapter 2. PRELIMINARY MATHEMATICS 9

Chapter 3. INTRODUCTION TO UNEAR PROGRAMMING 69

Chapter 4. DUALITY THEORY 89

Chapter 5. THE THEORY OF LINEAR PROGRAMMING 111

Chapter 6. DUALITY THEORY REVISITED 141

Chapter 7. COMPUTATIONAL ASPECTS OF LINEAR PROGRAMMING 169

Chapter 8. ONE-PHASE, TWO-PHASE, AND COMPOSITE METHODS OF

Chapter 9. COMPUTATIONAL ASPECTS OF LINEAR PROGRAMMING:

Chapter 10. THE DUAL SIMPLEX, PRIMAL-DUAL, AND

Chapter 11. POSTOPTIMALITY ANALYSIS I 289

Chapter 12. POSTOPTIMALITY ANALYSIS II 319

Chapter 13. INTERIOR POINT METHODS 341

13.8 Extensions and Modifications of Karmarkar's Algorithm 359

Chapter 14. INTERIOR POINT ALGORITHMS FOR SOLVING

Appendix A: Updating the Basis Inverse 459

A significant portion of the material presented herein stems from lecture

Finally, a special note of gratitude is extended to Marilyn Baleshiski for

My objective in writing this book is to present to students of management

class of decision problems of the form

max f(X) = C'X s.t. (1.1)

where C, XfRP,bfRm , A is a matrix of order (mxn), and a prime denotes

where U f Rm. While a detailed development of the relationship between these

Chapter 2 reviews the mathematical tools needed to study linear

fundamental theorems of linear programming, namely the strong duality theorem

optimal basic solution) is also offered. In addition to developing the standard

The coverage in Chapter 13 involves the presentation of a relatively new

x = [J:] , X' = (xl"'" xnJ

IIxll = (X'X)I/2 = [tIXd]I/2. (2.3)

linear combination of the Xj which equals the null vector.

e(.Ab) ={X I X = t \Xj,Xi E.Ab, \ ER for all i,

is the space spanned or generated by.Ab. Moreover, e(.Ab) is the smallest

2.2. Rank and Linear Transformations

(3) X€R n .- To(X) = 0 (the zero transformation);

(a) T A is one-to-one if n(A) = 0;

We noted earlier that for an (m x n) matrix A, the subspace spanned by the

Y = T(X) = 1'(A-tjX+l = IJ;tX·

This restriction of u to a projective transformation T will be employed in chapter

Next, for (n x n) matrices A and B, the function T B: Rn--+Rn is termed an

This expression is essentially a linear transformation followed by a translation,

2.3. The Solution Set of a System of

C = ~j=lX j3j' Thus the question of the existence of solutions to AX = C boils

If [A, Cl depicts the augmented matrix associated with AX = C, then we

consistent and possesses a unique solution;

If C = 0, then the resulting homogeneous system can never be inconsistent,

} A theorem oj the alternative involves two mutually exclusive systems of linear

2.8. THEOREM. Given the system AX = e, where A is of order (nx n),

(11") A'y = 0, C'y =f:. 0 has a solution Y.

In sum, either AX =C is consistent or there exists a vector Y which is ortho-