Applied Math Derivations

Derivations of Applied Mathematics
Thaddeus H. Black
Revised 7 January 2009

ii
Thaddeus H. Black, 1967–.

Derivations of Applied Mathematics.
7 January 2009.
U.S. Library of Congress class QA401.
c 1983–2009 by Thaddeus H. Black [email protected].

Copyright
Published by the Debian Project [15].
This book is free software. You can redistribute and/or modify it under the
terms of the GNU General Public License [22], version 2.
Contents
Preface xvii
1 Introduction 1
1.1 Applied mathematics . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Rigor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2.1 Axiom and definition . . . . . . . . . . . . . . . . . . 2
1.2.2 Mathematical extension . . . . . . . . . . . . . . . . . 4
1.3 Complex numbers and complex variables . . . . . . . . . . . 5
1.4 On the text . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
I The calculus of a single variable 7
2 Classical algebra and geometry 9

2.1 Basic arithmetic relationships . . . . . . . . . . . . . . . . . . 9
2.1.1 Commutivity, associativity, distributivity . . . . . . . 9
2.1.2 Negative numbers . . . . . . . . . . . . . . . . . . . . 11
2.1.3 Inequality . . . . . . . . . . . . . . . . . . . . . . . . 12
2.1.4 The change of variable . . . . . . . . . . . . . . . . . 12
2.2 Quadratics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.3 Integer and series notation . . . . . . . . . . . . . . . . . . . 15
2.4 The arithmetic series . . . . . . . . . . . . . . . . . . . . . . 17
2.5 Powers and roots . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.5.1 Notation and integral powers . . . . . . . . . . . . . . 18
2.5.2 Roots . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.5.3 Powers of products and powers of powers . . . . . . . 21
2.5.4 Sums of powers . . . . . . . . . . . . . . . . . . . . . 22
2.5.5 Summary and remarks . . . . . . . . . . . . . . . . . 23
2.6 Multiplying and dividing power series . . . . . . . . . . . . . 23
iii
iv CONTENTS
2.6.1 Multiplying power series . . . . . . . . . . . . . . . . 24

2.6.2 Dividing power series . . . . . . . . . . . . . . . . . . 24
2.6.3 Dividing power series by matching coefficients . . . . 28
2.6.4 Common quotients and the geometric series . . . . . 31
2.6.5 Variations on the geometric series . . . . . . . . . . . 32
2.7 Constants and variables . . . . . . . . . . . . . . . . . . . . . 32
2.8 Exponentials and logarithms . . . . . . . . . . . . . . . . . . 34
2.8.1 The logarithm . . . . . . . . . . . . . . . . . . . . . . 34
2.8.2 Properties of the logarithm . . . . . . . . . . . . . . . 35
2.9 Triangles and other polygons: simple facts . . . . . . . . . . 36
2.9.1 Triangle area . . . . . . . . . . . . . . . . . . . . . . . 36
2.9.2 The triangle inequalities . . . . . . . . . . . . . . . . 36
2.9.3 The sum of interior angles . . . . . . . . . . . . . . . 37
2.10 The Pythagorean theorem . . . . . . . . . . . . . . . . . . . . 38
2.11 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.12 Complex numbers (introduction) . . . . . . . . . . . . . . . . 41
2.12.1 Rectangular complex multiplication . . . . . . . . . . 43
2.12.2 Complex conjugation . . . . . . . . . . . . . . . . . . 44
2.12.3 Power series and analytic functions (preview) . . . . . 46
3 Trigonometry 47
3.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.2 Simple properties . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.3 Scalars, vectors, and vector notation . . . . . . . . . . . . . . 49
3.4 Rotation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.5 Trigonometric sums and differences . . . . . . . . . . . . . . 55
3.5.1 Variations on the sums and differences . . . . . . . . 56
3.5.2 Trigonometric functions of double and half angles . . 57
3.6 Trigonometrics of the hour angles . . . . . . . . . . . . . . . 57
3.7 The laws of sines and cosines . . . . . . . . . . . . . . . . . . 61
3.8 Summary of properties . . . . . . . . . . . . . . . . . . . . . 62
3.9 Cylindrical and spherical coordinates . . . . . . . . . . . . . 64
3.10 The complex triangle inequalities . . . . . . . . . . . . . . . . 67
3.11 De Moivre’s theorem . . . . . . . . . . . . . . . . . . . . . . . 67
4 The derivative 69
4.1 Infinitesimals and limits . . . . . . . . . . . . . . . . . . . . . 69
4.1.1 The infinitesimal . . . . . . . . . . . . . . . . . . . . . 70
4.1.2 Limits . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.2 Combinatorics . . . . . . . . . . . . . . . . . . . . . . . . . . 72
CONTENTS v
4.2.1 Combinations and permutations . . . . . . . . . . . . 72

4.2.2 Pascal’s triangle . . . . . . . . . . . . . . . . . . . . . 74
4.3 The binomial theorem . . . . . . . . . . . . . . . . . . . . . . 74
4.3.1 Expanding the binomial . . . . . . . . . . . . . . . . . 74
4.3.2 Powers of numbers near unity . . . . . . . . . . . . . 75
4.3.3 Complex powers of numbers near unity . . . . . . . . 76
4.4 The derivative . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.4.1 The derivative of the power series . . . . . . . . . . . 77
4.4.2 The Leibnitz notation . . . . . . . . . . . . . . . . . . 78
4.4.3 The derivative of a function of a complex variable . . 80
4.4.4 The derivative of z a . . . . . . . . . . . . . . . . . . . 81
4.4.5 The logarithmic derivative . . . . . . . . . . . . . . . 82
4.5 Basic manipulation of the derivative . . . . . . . . . . . . . . 82
4.5.1 The derivative chain rule . . . . . . . . . . . . . . . . 82
4.5.2 The derivative product rule . . . . . . . . . . . . . . . 83
4.5.3 A derivative product pattern . . . . . . . . . . . . . . 84
4.6 Extrema and higher derivatives . . . . . . . . . . . . . . . . . 85
4.7 L’Hôpital’s rule . . . . . . . . . . . . . . . . . . . . . . . . . . 87
4.8 The Newton-Raphson iteration . . . . . . . . . . . . . . . . . 88
5 The complex exponential 93

5.1 The real exponential . . . . . . . . . . . . . . . . . . . . . . . 93
5.2 The natural logarithm . . . . . . . . . . . . . . . . . . . . . . 96
5.3 Fast and slow functions . . . . . . . . . . . . . . . . . . . . . 98
5.4 Euler’s formula . . . . . . . . . . . . . . . . . . . . . . . . . . 100
5.5 Complex exponentials and de Moivre . . . . . . . . . . . . . 103
5.6 Complex trigonometrics . . . . . . . . . . . . . . . . . . . . . 103
5.7 Summary of properties . . . . . . . . . . . . . . . . . . . . . 105
5.8 Derivatives of complex exponentials . . . . . . . . . . . . . . 105
5.8.1 Derivatives of sine and cosine . . . . . . . . . . . . . . 105
5.8.2 Derivatives of the trigonometrics . . . . . . . . . . . . 108
5.8.3 Derivatives of the inverse trigonometrics . . . . . . . 108
5.9 The actuality of complex quantities . . . . . . . . . . . . . . 110
6 Primes, roots and averages 113

6.1 Prime numbers . . . . . . . . . . . . . . . . . . . . . . . . . . 113
6.1.1 The infinite supply of primes . . . . . . . . . . . . . . 113
6.1.2 Compositional uniqueness . . . . . . . . . . . . . . . . 114
6.1.3 Rational and irrational numbers . . . . . . . . . . . . 117
6.2 The existence and number of roots . . . . . . . . . . . . . . . 118
vi CONTENTS
6.2.1 Polynomial roots . . . . . . . . . . . . . . . . . . . . . 118

6.2.2 The fundamental theorem of algebra . . . . . . . . . 119
6.3 Addition and averages . . . . . . . . . . . . . . . . . . . . . . 120
6.3.1 Serial and parallel addition . . . . . . . . . . . . . . . 120
6.3.2 Averages . . . . . . . . . . . . . . . . . . . . . . . . . 123
7 The integral 127

7.1 The concept of the integral . . . . . . . . . . . . . . . . . . . 127
7.1.1 An introductory example . . . . . . . . . . . . . . . . 128
7.1.2 Generalizing the introductory example . . . . . . . . 131
7.1.3 The balanced definition and the trapezoid rule . . . . 131
7.2 The antiderivative . . . . . . . . . . . . . . . . . . . . . . . . 133
7.3 Operators, linearity and multiple integrals . . . . . . . . . . . 135
7.3.1 Operators . . . . . . . . . . . . . . . . . . . . . . . . 135
7.3.2 A formalism . . . . . . . . . . . . . . . . . . . . . . . 135
7.3.3 Linearity . . . . . . . . . . . . . . . . . . . . . . . . . 136
7.3.4 Summational and integrodifferential commutivity . . 137
7.3.5 Multiple integrals . . . . . . . . . . . . . . . . . . . . 139
7.4 Areas and volumes . . . . . . . . . . . . . . . . . . . . . . . . 141
7.4.1 The area of a circle . . . . . . . . . . . . . . . . . . . 141
7.4.2 The volume of a cone . . . . . . . . . . . . . . . . . . 141
7.4.3 The surface area and volume of a sphere . . . . . . . 142
7.5 Checking an integration . . . . . . . . . . . . . . . . . . . . . 145
7.6 Contour integration . . . . . . . . . . . . . . . . . . . . . . . 146
7.7 Discontinuities . . . . . . . . . . . . . . . . . . . . . . . . . . 147
7.8 Remarks (and exercises) . . . . . . . . . . . . . . . . . . . . . 150
8 The Taylor series 153

8.1 The power-series expansion of 1/(1 − z)n+1 . . . . . . . . . . 154
8.1.1 The formula . . . . . . . . . . . . . . . . . . . . . . . 154
8.1.2 The proof by induction . . . . . . . . . . . . . . . . . 156
8.1.3 Convergence . . . . . . . . . . . . . . . . . . . . . . . 157
8.1.4 General remarks on mathematical induction . . . . . 158
8.2 Shifting a power series’ expansion point . . . . . . . . . . . . 159
8.3 Expanding functions in Taylor series . . . . . . . . . . . . . . 161
8.4 Analytic continuation . . . . . . . . . . . . . . . . . . . . . . 163
8.5 Branch points . . . . . . . . . . . . . . . . . . . . . . . . . . 165
8.6 Entire and meromorphic functions . . . . . . . . . . . . . . . 167
8.7 Extrema over a complex domain . . . . . . . . . . . . . . . . 168
8.8 Cauchy’s integral formula . . . . . . . . . . . . . . . . . . . . 169
CONTENTS vii
8.8.1 The meaning of the symbol dz . . . . . . . . . . . . . 170

8.8.2 Integrating along the contour . . . . . . . . . . . . . . 170
8.8.3 The formula . . . . . . . . . . . . . . . . . . . . . . . 174
8.8.4 Enclosing a multiple pole . . . . . . . . . . . . . . . . 175
8.9 Taylor series for specific functions . . . . . . . . . . . . . . . 176
8.10 Error bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
8.10.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . 179
8.10.2 Majorization . . . . . . . . . . . . . . . . . . . . . . . 180
8.10.3 Geometric majorization . . . . . . . . . . . . . . . . . 182
8.10.4 Calculation outside the fast convergence domain . . . 184
8.10.5 Nonconvergent series . . . . . . . . . . . . . . . . . . 186
8.10.6 Remarks . . . . . . . . . . . . . . . . . . . . . . . . . 187
8.11 Calculating 2π . . . . . . . . . . . . . . . . . . . . . . . . . . 188
8.12 Odd and even functions . . . . . . . . . . . . . . . . . . . . . 188
8.13 Trigonometric poles . . . . . . . . . . . . . . . . . . . . . . . 189
8.14 The Laurent series . . . . . . . . . . . . . . . . . . . . . . . . 190
8.15 Taylor series in 1/z . . . . . . . . . . . . . . . . . . . . . . . 193
8.16 The multidimensional Taylor series . . . . . . . . . . . . . . . 194
9 Integration techniques 197

9.1 Integration by antiderivative . . . . . . . . . . . . . . . . . . 197
9.2 Integration by substitution . . . . . . . . . . . . . . . . . . . 198
9.3 Integration by parts . . . . . . . . . . . . . . . . . . . . . . . 199
9.4 Integration by unknown coefficients . . . . . . . . . . . . . . 201
9.5 Integration by closed contour . . . . . . . . . . . . . . . . . . 204
9.6 Integration by partial-fraction expansion . . . . . . . . . . . 209
9.6.1 Partial-fraction expansion . . . . . . . . . . . . . . . . 209
9.6.2 Repeated poles . . . . . . . . . . . . . . . . . . . . . . 210
9.6.3 Integrating a rational function . . . . . . . . . . . . . 213
9.6.4 The derivatives of a rational function . . . . . . . . . 215
9.6.5 Repeated poles (the conventional technique) . . . . . 215
9.6.6 The existence and uniqueness of solutions . . . . . . . 217
9.7 Frullani’s integral . . . . . . . . . . . . . . . . . . . . . . . . 218
9.8 Products of exponentials, powers and logs . . . . . . . . . . . 219
9.9 Integration by Taylor series . . . . . . . . . . . . . . . . . . . 221
10 Cubics and quartics 223

10.1 Vieta’s transform . . . . . . . . . . . . . . . . . . . . . . . . . 224
10.2 Cubics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
10.3 Superfluous roots . . . . . . . . . . . . . . . . . . . . . . . . . 227
viii CONTENTS
10.4 Edge cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

10.5 Quartics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
10.6 Guessing the roots . . . . . . . . . . . . . . . . . . . . . . . . 233
II Matrices and vectors 237
11 The matrix 239

11.1 Provenance and basic use . . . . . . . . . . . . . . . . . . . . 242
11.1.1 The linear transformation . . . . . . . . . . . . . . . . 242
11.1.2 Matrix multiplication (and addition) . . . . . . . . . 243
11.1.3 Row and column operators . . . . . . . . . . . . . . . 245
11.1.4 The transpose and the adjoint . . . . . . . . . . . . . 246
11.2 The Kronecker delta . . . . . . . . . . . . . . . . . . . . . . . 247
11.3 Dimensionality and matrix forms . . . . . . . . . . . . . . . . 248
11.3.1 The null and dimension-limited matrices . . . . . . . 250
11.3.2 The identity, scalar and extended matrices . . . . . . 251
11.3.3 The active region . . . . . . . . . . . . . . . . . . . . 253
11.3.4 Other matrix forms . . . . . . . . . . . . . . . . . . . 253
11.3.5 The rank-r identity matrix . . . . . . . . . . . . . . . 254
11.3.6 The truncation operator . . . . . . . . . . . . . . . . 255
11.3.7 The elementary vector and lone-element matrix . . . 255
11.3.8 Off-diagonal entries . . . . . . . . . . . . . . . . . . . 256
11.4 The elementary operator . . . . . . . . . . . . . . . . . . . . 256
11.4.1 Properties . . . . . . . . . . . . . . . . . . . . . . . . 258
11.4.2 Commutation and sorting . . . . . . . . . . . . . . . . 259
11.5 Inversion and similarity (introduction) . . . . . . . . . . . . . 260
11.6 Parity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
11.7 The quasielementary operator . . . . . . . . . . . . . . . . . 266
11.7.1 The general interchange operator . . . . . . . . . . . 267
11.7.2 The general scaling operator . . . . . . . . . . . . . . 268
11.7.3 Addition quasielementaries . . . . . . . . . . . . . . . 269
11.8 The unit triangular matrix . . . . . . . . . . . . . . . . . . . 271
11.8.1 Construction . . . . . . . . . . . . . . . . . . . . . . . 273
11.8.2 The product of like unit triangular matrices . . . . . 274
11.8.3 Inversion . . . . . . . . . . . . . . . . . . . . . . . . . 274
11.8.4 The parallel unit triangular matrix . . . . . . . . . . 275
11.8.5 The partial unit triangular matrix . . . . . . . . . . . 280
11.9 The shift operator . . . . . . . . . . . . . . . . . . . . . . . . 282
11.10 The Jacobian derivative . . . . . . . . . . . . . . . . . . . . . 282
CONTENTS ix
12 Rank and the Gauss-Jordan 285

12.1 Linear independence . . . . . . . . . . . . . . . . . . . . . . . 286
12.2 The elementary similarity transformation . . . . . . . . . . . 288
12.3 The Gauss-Jordan decomposition . . . . . . . . . . . . . . . . 288
12.3.1 Motive . . . . . . . . . . . . . . . . . . . . . . . . . . 290
12.3.2 Method . . . . . . . . . . . . . . . . . . . . . . . . . . 293
12.3.3 The algorithm . . . . . . . . . . . . . . . . . . . . . . 294
12.3.4 Rank and independent rows . . . . . . . . . . . . . . 302
12.3.5 Inverting the factors . . . . . . . . . . . . . . . . . . . 303
12.3.6 Truncating the factors . . . . . . . . . . . . . . . . . . 303
12.3.7 Properties of the factors . . . . . . . . . . . . . . . . 305
12.3.8 Marginalizing the factor In . . . . . . . . . . . . . . . 305
12.3.9 Decomposing an extended operator . . . . . . . . . . 306
12.4 Vector replacement . . . . . . . . . . . . . . . . . . . . . . . . 307
12.5 Rank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
12.5.1 A logical maneuver . . . . . . . . . . . . . . . . . . . 311
12.5.2 The impossibility of identity-matrix promotion . . . . 312
12.5.3 General matrix rank and its uniqueness . . . . . . . . 315
12.5.4 The full-rank matrix . . . . . . . . . . . . . . . . . . 317
12.5.5 Under- and overdetermined systems (introduction) . . 318
12.5.6 The full-rank factorization . . . . . . . . . . . . . . . 319
12.5.7 Full column rank and the Gauss-Jordan’s K and S . 320
12.5.8 The significance of rank uniqueness . . . . . . . . . . 321
13 Inversion and orthonormalization 323

13.1 Inverting the square matrix . . . . . . . . . . . . . . . . . . . 324
13.2 The exactly determined linear system . . . . . . . . . . . . . 328
13.3 The kernel . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
13.3.1 The Gauss-Jordan kernel formula . . . . . . . . . . . 330
13.3.2 Converting between kernel matrices . . . . . . . . . . 334
13.3.3 The degree of freedom . . . . . . . . . . . . . . . . . . 334
13.4 The nonoverdetermined linear system . . . . . . . . . . . . . 336
13.4.1 Particular and homogeneous solutions . . . . . . . . . 337
13.4.2 A particular solution . . . . . . . . . . . . . . . . . . 337
13.4.3 The general solution . . . . . . . . . . . . . . . . . . . 338
13.5 The residual . . . . . . . . . . . . . . . . . . . . . . . . . . . 338
13.6 The pseudoinverse and least squares . . . . . . . . . . . . . . 339
13.6.1 Least squares in the real domain . . . . . . . . . . . . 341
13.6.2 The invertibility of A∗ A . . . . . . . . . . . . . . . . . 343
13.6.3 Positive definiteness . . . . . . . . . . . . . . . . . . . 344
x CONTENTS
13.6.4 The Moore-Penrose pseudoinverse . . . . . . . . . . . 344

13.7 The multivariate Newton-Raphson iteration . . . . . . . . . . 348
13.8 The dot product . . . . . . . . . . . . . . . . . . . . . . . . . 349
13.9 The complex vector triangle inequalities . . . . . . . . . . . . 351
13.10 The orthogonal complement . . . . . . . . . . . . . . . . . . . 353
13.11 Gram-Schmidt orthonormalization . . . . . . . . . . . . . . . 353
13.11.1 Efficient implementation . . . . . . . . . . . . . . . . 355
13.11.2 The Gram-Schmidt decomposition . . . . . . . . . . . 356
13.11.3 The Gram-Schmidt kernel formula . . . . . . . . . . . 360
13.12 The unitary matrix . . . . . . . . . . . . . . . . . . . . . . . 361
14 The eigenvalue 365

14.1 The determinant . . . . . . . . . . . . . . . . . . . . . . . . . 365
14.1.1 Basic properties . . . . . . . . . . . . . . . . . . . . . 367
14.1.2 The determinant and the elementary operator . . . . 370
14.1.3 The determinant of a singular matrix . . . . . . . . . 371
14.1.4 The determinant of a matrix product . . . . . . . . . 372
14.1.5 Determinants of inverse and unitary matrices . . . . . 372
14.1.6 Inverting the square matrix by determinant . . . . . . 373
14.2 Coincident properties . . . . . . . . . . . . . . . . . . . . . . 374
14.3 The eigenvalue itself . . . . . . . . . . . . . . . . . . . . . . . 376
14.4 The eigenvector . . . . . . . . . . . . . . . . . . . . . . . . . 377
14.5 Eigensolution facts . . . . . . . . . . . . . . . . . . . . . . . . 378
14.6 Diagonalization . . . . . . . . . . . . . . . . . . . . . . . . . . 379
14.7 Remarks on the eigenvalue . . . . . . . . . . . . . . . . . . . 381
14.8 Matrix condition . . . . . . . . . . . . . . . . . . . . . . . . . 382
14.9 The similarity transformation . . . . . . . . . . . . . . . . . . 383
14.10 The Schur decomposition . . . . . . . . . . . . . . . . . . . . 385
14.10.1 Derivation . . . . . . . . . . . . . . . . . . . . . . . . 385
14.10.2 The nondiagonalizable matrix . . . . . . . . . . . . . 390
14.11 The Hermitian matrix . . . . . . . . . . . . . . . . . . . . . . 392
14.12 The singular-value decomposition . . . . . . . . . . . . . . . 396
14.13 General remarks on the matrix . . . . . . . . . . . . . . . . . 398
15 Vector analysis 401

15.1 Reorientation . . . . . . . . . . . . . . . . . . . . . . . . . . . 404
15.1.1 The Tait-Bryan rotations . . . . . . . . . . . . . . . . 404
15.1.2 The Euler rotations . . . . . . . . . . . . . . . . . . . 406
15.2 Multiplication . . . . . . . . . . . . . . . . . . . . . . . . . . 406
15.2.1 The dot product . . . . . . . . . . . . . . . . . . . . . 407
CONTENTS xi
15.2.2 The cross product . . . . . . . . . . . . . . . . . . . . 407

15.3 Orthogonal bases . . . . . . . . . . . . . . . . . . . . . . . . . 410
15.4 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416
15.4.1 Components by subscript . . . . . . . . . . . . . . . . 416
15.4.2 Einstein’s summation convention . . . . . . . . . . . . 418
15.4.3 The Kronecker delta and the Levi-Civita epsilon . . . 419
15.5 Algebraic identities . . . . . . . . . . . . . . . . . . . . . . . 424
15.6 Isotropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426
15.7 Parabolic coordinates . . . . . . . . . . . . . . . . . . . . . . 427
15.7.1 The parabola . . . . . . . . . . . . . . . . . . . . . . . 428
15.7.2 Parabolic coordinates in two dimensions . . . . . . . 430
15.7.3 Properties . . . . . . . . . . . . . . . . . . . . . . . . 432
15.7.4 The parabolic cylindrical coordinate system . . . . . 434
15.7.5 The circular paraboloidal coordinate system . . . . . 435
16 Vector calculus 437

16.1 Fields and their derivatives . . . . . . . . . . . . . . . . . . . 437
16.1.1 The ∇ operator . . . . . . . . . . . . . . . . . . . . . 439
16.1.2 Operator notation . . . . . . . . . . . . . . . . . . . . 441
16.1.3 The directional derivative and the gradient . . . . . . 442
16.1.4 Divergence . . . . . . . . . . . . . . . . . . . . . . . . 444
16.1.5 Curl . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446
16.1.6 Cross-directional derivatives . . . . . . . . . . . . . . 448
16.2 Integral forms . . . . . . . . . . . . . . . . . . . . . . . . . . 449
16.2.1 The divergence theorem . . . . . . . . . . . . . . . . . 449
16.2.2 Stokes’ theorem . . . . . . . . . . . . . . . . . . . . . 451
16.3 Summary of definitions and identities . . . . . . . . . . . . . 451
16.4 The Laplacian, et al. . . . . . . . . . . . . . . . . . . . . . . . 454
16.5 Contour derivative product rules . . . . . . . . . . . . . . . . 456
16.6 Metric coefficients . . . . . . . . . . . . . . . . . . . . . . . . 457
16.6.1 Displacements, areas and volumes . . . . . . . . . . . 458
16.6.2 The vector field and its scalar components . . . . . . 459
16.7 Nonrectangular notation . . . . . . . . . . . . . . . . . . . . . 460
16.8 Derivatives of the basis vectors . . . . . . . . . . . . . . . . . 461
16.9 Derivatives in the nonrectangular systems . . . . . . . . . . . 461
16.9.1 Derivatives in cylindrical coordinates . . . . . . . . . 461
16.9.2 Derivatives in spherical coordinates . . . . . . . . . . 466
16.9.3 Finding the derivatives geometrically . . . . . . . . . 468
16.10 Vector infinitesimals . . . . . . . . . . . . . . . . . . . . . . . 474
xii CONTENTS
III Transforms, special functions and other topics 477
17 The Fourier series 479

17.1 Parseval’s principle . . . . . . . . . . . . . . . . . . . . . . . . 480
17.2 Time, space and frequency . . . . . . . . . . . . . . . . . . . 483
17.3 The square and triangular pulses . . . . . . . . . . . . . . . . 485
17.4 Expanding waveforms in Fourier series . . . . . . . . . . . . . 486
17.4.1 Derivation of the Fourier-coefficient formula . . . . . 487
17.4.2 The square wave . . . . . . . . . . . . . . . . . . . . . 488
17.4.3 The rectangular pulse train . . . . . . . . . . . . . . . 488
17.4.4 Linearity and sufficiency . . . . . . . . . . . . . . . . 491
17.4.5 The trigonometric form . . . . . . . . . . . . . . . . . 493
17.5 The sine-argument function . . . . . . . . . . . . . . . . . . . 494
17.5.1 Derivative and integral . . . . . . . . . . . . . . . . . 495
17.5.2 Properties of the sine-argument function . . . . . . . 496
17.5.3 Properties of the sine integral . . . . . . . . . . . . . 497
17.5.4 The sine integral’s limit by complex contour . . . . . 499
17.6 Gibbs’ phenomenon . . . . . . . . . . . . . . . . . . . . . . . 502
18 The Fourier and Laplace transforms 507
Appendices 511
A Hexadecimal notation, et al. 513

A.1 Hexadecimal numerals . . . . . . . . . . . . . . . . . . . . . . 514
A.2 Avoiding notational clutter . . . . . . . . . . . . . . . . . . . 515
B The Greek alphabet 517
C A sketch of pure complex theory 521
D Manuscript history 525

List of Tables
2.1 Basic properties of arithmetic. . . . . . . . . . . . . . . . . . . 10

2.2 Power properties and definitions. . . . . . . . . . . . . . . . . 18
2.3 Dividing power series through successively smaller powers. . . 27
2.4 Dividing power series through successively larger powers. . . 29
2.5 General properties of the logarithm. . . . . . . . . . . . . . . 36
3.1 Simple properties of the trigonometric functions. . . . . . . . 50

3.2 Trigonometric functions of the hour angles. . . . . . . . . . . 59
3.3 Further properties of the trigonometric functions. . . . . . . . 63
3.4 Rectangular, cylindrical and spherical coordinate relations. . 65
5.1 Complex exponential properties. . . . . . . . . . . . . . . . . 106

5.2 Derivatives of the trigonometrics. . . . . . . . . . . . . . . . . 109
5.3 Derivatives of the inverse trigonometrics. . . . . . . . . . . . . 111
6.1 Parallel and serial addition identities. . . . . . . . . . . . . . . 122
7.1 Basic derivatives for the antiderivative. . . . . . . . . . . . . . 134
8.1 Taylor series. . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
9.1 Antiderivatives of products of exps, powers and logs. . . . . . 221
10.1 The method to extract the three roots of the general cubic. . 227
10.2 The method to extract the four roots of the general quartic. . 234
11.1 Elementary operators: interchange. . . . . . . . . . . . . . . . 261

11.2 Elementary operators: scaling. . . . . . . . . . . . . . . . . . 262
11.3 Elementary operators: addition. . . . . . . . . . . . . . . . . . 262
11.4 Matrix inversion properties. . . . . . . . . . . . . . . . . . . . 264
11.5 Properties of the parallel unit triangular matrix. . . . . . . . 279
xiii
xiv LIST OF TABLES
12.1 Some elementary similarity transformations. . . . . . . . . . . 289

12.2 A few properties of the Gauss-Jordan factors. . . . . . . . . . 305
12.3 The symmetrical equations of § 12.4. . . . . . . . . . . . . . . 309
15.1 Properties of the Kronecker delta and Levi-Civita epsilon. . . 421

15.2 Algebraic vector identities. . . . . . . . . . . . . . . . . . . . . 425
15.3 Parabolic coordinate properties. . . . . . . . . . . . . . . . . . 434
15.4 Circular paraboloidal coordinate properties. . . . . . . . . . . 435
16.1 Definitions and identities of vector calculus. . . . . . . . . . . 453

16.2 Metric coefficients. . . . . . . . . . . . . . . . . . . . . . . . . 458
16.3 Derivatives of the basis vectors. . . . . . . . . . . . . . . . . . 462
16.4 Vector derivatives in cylindrical coordinates. . . . . . . . . . . 466
16.5 Vector derivatives in spherical coordinates. . . . . . . . . . . . 469
B.1 The Roman and Greek alphabets. . . . . . . . . . . . . . . . . 518

List of Figures
1.1 Two triangles. . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1 Multiplicative commutivity. . . . . . . . . . . . . . . . . . . . 10

2.2 The sum of a triangle’s inner angles: turning at the corner. . 37
2.3 A right triangle. . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.4 The Pythagorean theorem. . . . . . . . . . . . . . . . . . . . 39
2.5 The complex (or Argand) plane. . . . . . . . . . . . . . . . . 42
3.1 The sine and the cosine. . . . . . . . . . . . . . . . . . . . . . 48

3.2 The sine function. . . . . . . . . . . . . . . . . . . . . . . . . 49
3.3 A two-dimensional vector u = x̂x + ŷy. . . . . . . . . . . . . 51
3.4 A three-dimensional vector v = x̂x + ŷy + ẑz. . . . . . . . . . 51
3.5 Vector basis rotation. . . . . . . . . . . . . . . . . . . . . . . . 54
3.6 The 0x18 hours in a circle. . . . . . . . . . . . . . . . . . . . . 58
3.7 Calculating the hour trigonometrics. . . . . . . . . . . . . . . 60
3.8 The laws of sines and cosines. . . . . . . . . . . . . . . . . . . 61
3.9 A point on a sphere. . . . . . . . . . . . . . . . . . . . . . . . 65
4.1 The plan for Pascal’s triangle. . . . . . . . . . . . . . . . . . . 74

4.2 Pascal’s triangle. . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.3 A local extremum. . . . . . . . . . . . . . . . . . . . . . . . . 85
4.4 A level inflection. . . . . . . . . . . . . . . . . . . . . . . . . . 86
4.5 The Newton-Raphson iteration. . . . . . . . . . . . . . . . . . 89
5.1 The natural exponential. . . . . . . . . . . . . . . . . . . . . . 96

5.2 The natural logarithm. . . . . . . . . . . . . . . . . . . . . . . 97
5.3 The complex exponential and Euler’s formula. . . . . . . . . . 101
5.4 The derivatives of the sine and cosine functions. . . . . . . . . 107
7.1 Areas representing discrete sums. . . . . . . . . . . . . . . . . 128
xv
xvi LIST OF FIGURES
7.2 An area representing an infinite sum of infinitesimals. . . . . 130

7.3 Integration by the trapezoid rule. . . . . . . . . . . . . . . . . 132
7.4 The area of a circle. . . . . . . . . . . . . . . . . . . . . . . . 141
7.5 The volume of a cone. . . . . . . . . . . . . . . . . . . . . . . 142
7.6 A sphere. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
7.7 An element of a sphere’s surface. . . . . . . . . . . . . . . . . 143
7.8 A contour of integration. . . . . . . . . . . . . . . . . . . . . . 147
7.9 The Heaviside unit step u(t). . . . . . . . . . . . . . . . . . . 148
7.10 The Dirac delta δ(t). . . . . . . . . . . . . . . . . . . . . . . . 148
8.1 A complex contour of integration in two segments. . . . . . . 171

8.2 A Cauchy contour integral. . . . . . . . . . . . . . . . . . . . 175
8.3 Majorization. . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
9.1 Integration by closed contour. . . . . . . . . . . . . . . . . . . 205
10.1 Vieta’s transform, plotted logarithmically. . . . . . . . . . . . 225
13.1 Fitting a line to measured data. . . . . . . . . . . . . . . . . . 340
15.1 A point on a sphere. . . . . . . . . . . . . . . . . . . . . . . . 402

15.2 The dot product. . . . . . . . . . . . . . . . . . . . . . . . . . 408
15.3 The cross product. . . . . . . . . . . . . . . . . . . . . . . . . 410
15.4 The cylindrical basis. . . . . . . . . . . . . . . . . . . . . . . . 413
15.5 The spherical basis. . . . . . . . . . . . . . . . . . . . . . . . . 414
15.6 A vector projected onto a plane. . . . . . . . . . . . . . . . . 425
15.7 The parabola. . . . . . . . . . . . . . . . . . . . . . . . . . . . 429
15.8 Locating a point by parabolic construction. . . . . . . . . . . 431
15.9 The parabolic coordinate grid in two dimensions. . . . . . . . 432
17.1 A square wave. . . . . . . . . . . . . . . . . . . . . . . . . . . 480

17.2 Superpositions of sinusoids. . . . . . . . . . . . . . . . . . . . 481
17.3 The square and triangular pulses. . . . . . . . . . . . . . . . . 486
17.4 A rectangular pulse train. . . . . . . . . . . . . . . . . . . . . 489
17.5 A Dirac delta pulse train. . . . . . . . . . . . . . . . . . . . . 490
17.6 The sine-argument function. . . . . . . . . . . . . . . . . . . . 495
17.7 The sine integral. . . . . . . . . . . . . . . . . . . . . . . . . . 496
17.8 The points at which t intersects tan t. . . . . . . . . . . . . . 498
17.9 A complex contour about which to integrate eiz /i2z. . . . . . 500
17.10Gibbs’ phenomenon. . . . . . . . . . . . . . . . . . . . . . . . 504
Preface
I never meant to write this book. It emerged unheralded, unexpectedly.

The book began in 1983 when a high-school classmate challenged me to
prove the Pythagorean theorem on the spot. I lost the dare, but looking the
proof up later I recorded it on loose leaves, adding to it the derivations of
a few other theorems of interest to me. From such a kernel the notes grew
over time, until family and friends suggested that the notes might make the
material for a book.
The book, a work yet in progress but a complete, entire book as it
stands, first frames coherently the simplest, most basic derivations of ap-
plied mathematics, treating quadratics, trigonometrics, exponentials, deriv-
atives, integrals, series, complex variables and, of course, the aforementioned
Pythagorean theorem. These and others establish the book’s foundation in
Chapters 2 through 9. Later chapters build upon the foundation, deriving
results less general or more advanced.
The book is neither a tutorial on the one hand nor a bald reference on the
other. It is a study reference, in the tradition of, for instance, Kernighan’s
and Ritchie’s The C Programming Language [37]. In the book, you can look
up some particular result directly, or you can begin on page one and read—
with toil and commensurate profit—straight through to the end of the last
chapter.
The book generally follows established applied mathematical convention
but where worthwhile occasionally departs therefrom. One particular re-
spect in which the book departs requires some defense here, I think: the
book employs hexadecimal numerals.
There is nothing wrong with decimal numerals as such. Decimal numer-
als are fine in history and anthropology (man has ten fingers), finance and
accounting (dollars, cents, pounds, shillings, pence: the base hardly mat-
ters), law and engineering (the physical units are arbitrary anyway); but
they are merely serviceable in mathematical theory, never aesthetic. There
unfortunately really is no gradual way to bridge the gap to hexadecimal
xvii
xviii PREFACE
(shifting to base eleven, thence to twelve, etc., is no use). If one wishes to

reach hexadecimal ground then one must leap. Twenty years of keeping my
own private notes in hex have persuaded me that the leap justifies the risk.
In other matters, by contrast, the book leaps seldom. The book in general
walks a tolerably conventional applied mathematical line.
The book belongs to the emerging tradition of open-source software,
where at the time of this writing it fills a void. Nevertheless it is a book, not a
program. Lore among open-source developers holds that open development
inherently leads to superior work. Well, maybe. Often it does in fact.
Personally with regard to my own work, I should rather not make too many
claims. It would be vain to deny that professional editing and formal peer
review, neither of which the book enjoys, had substantial value. On the other
hand, it does not do to despise the amateur (literally, one who does for the
love of it: not such a bad motive, after all1 ) on principle, either—unless
one would on the same principle despise a Washington or an Einstein, or a
Debian Developer [15]. Open source has a spirit to it which leads readers to
be far more generous with their feedback than ever could be the case with
a traditional, proprietary book. Such readers, among whom a surprising
concentration of talent and expertise are found, enrich the work freely. This
has value, too.
The book’s manner of publication guarantees that it cannot go out of
print. Its certain availability2 is a significant feature, lending other authors
the unusual assurance that, when they cite this book, their readers can
easily follow the citation. If such a feature saves such authors scarce pages
in appendices they would rather omit then it will have achieved its purpose.
As to a grand goal, underlying purpose or hidden plan, the book has
none, other than to derive as many useful mathematical results as possible,
recording the derivations together in an orderly manner in a single volume.
What constitutes “useful” or “orderly” is a matter of perspective and judg-
ment, of course. My own peculiar heterogeneous background in military
service, building construction, electrical engineering, electromagnetic anal-
ysis and Debian development, my nativity, residence and citizenship in the
United States, undoubtedly bias the selection and presentation to some de-
gree. How other authors go about writing their books, I do not know, but I
1
The expression is derived from an observation I seem to recall George F. Will making.
2
A publisher (or anyone else for that matter) can archive this book on and distribute
it from the publisher’s own Web site without further permission from me. Professional
courtesy naturally recommends informing me of such archival and distribution but even
so little is technically not required. Such is the nature of the book’s standard open-source
license. [22]
xix
suppose that what is true for me is true for many of them also: we begin by
organizing notes for our own use, then observe that the same notes might
prove useful to others, and then undertake to revise the notes and to bring
them into a form which actually is useful to others. Whether this book
succeeds in the last point is for the reader to judge.
Features and acknowledgements

A few of the book’s editional features call for brief prefatory mention, begin-
ning with the book’s footnotes. The book’s peculiar mission and program
bring footnotes in uncommon quantity. Many of these footnotes are discur-
sive in nature, offering nonessential material which, though edifying, coheres
insufficiently well to join the book’s main narrative. The footnote is an im-
perfect messenger, of course. Catching the reader’s eye, it can break the
flow of otherwise good prose. Modern publishing offers various alternatives
to the footnote—numbered examples, sidebars, special fonts, colored inks,
etc. Some of these are merely trendy. Others, like numbered examples, re-
ally do help the right kind of book; but for this book the humble footnote,
long sanctioned by an earlier era of publishing, extensively employed by such
sages as Gibbon [24] and Shirer [56], seems the most able messenger. In this
book it shall have many messages to bear.
In typical science/engineering style, the book numbers its sections, ta-
bles, figures and formulas, but not its theorems, the last of which it generally
sets in italic type. Within the book, a theorem is referenced by the number
of the section that states it.
The book subjoins an alphabetical index as a standard convenience.
Even so, the canny reader will avoid using the index (of this and most other
books), which alone of the book’s pages is not to be regarded as a proper
part of the book. Such a reader will tend rather to consult the book’s table
of contents which is a proper part.
The book includes a bibliography listing works I have referred to while
writing. This is as it should be. The book however is a book of applied
mathematics, whereas mathematics by its very nature promotes queer bib-
liographies because mathematics’ methods and truths are established by
derivation rather than authority. Much of the book consists of common
mathematical knowledge or of proofs I have worked out with my own pen-
cil from various ideas gleaned who knows where over the years. The latter
proofs are perhaps original or semi-original from my personal point of view,
but it is unlikely that many if any of them are truly new. To the initiated,
the mathematics itself often tends to suggest the form of the proof: if to me,
xx PREFACE
then surely also to others who came before; and even where a proof is new
the idea proven probably is not.
Among the bibliography’s entries stands a reference [10] to my doctoral
adviser G.S. Brown, though the book’s narrative seldom explicitly invokes
the reference. Prof. Brown had nothing directly to do with the book’s devel-
opment, for a substantial bulk of the manuscript, or of the notes underlying
it, had been drafted years before I had ever heard of Prof. Brown or he of
me, and my work under him did not regard the book in any case. However,
the ways in which a good doctoral adviser influences his student are both
complex and deep. Prof. Brown’s style and insight touch the book in many
places and in many ways, usually too implicitly coherently to cite.
Steady encouragement from my wife and children contribute to the book
in ways only an author can properly appreciate.
More and much earlier than to Prof. Brown or to my wife and children,
the book owes a debt to my mother and, separately, to my father, without
either of whom the book would never have come to be. Admittedly, any
author of any book might say as much in a certain respect, but it is no
office of a mathematical book’s preface to burden readers with an author’s
expressions of filial piety. No, it is in entirely another respect that I lay the
matter here. My mother taught me at her own hand most of the mathe-
matics I ever learned as a child, patiently building a foundation that cannot
but be said to undergird the whole book today. More recently, my mother
has edited tracts of the book’s manuscript. My father generously financed
much of my formal education but—more than this—one would have had
to grow up with my brother and me in my father’s home to appreciate the
grand sweep of the man’s curiosity, the general depth of his knowledge, the
profound wisdom of his practicality and the enduring example of his love of
excellence.
May the book deserve such a heritage.
THB
Chapter 1
Introduction
This is a book of applied mathematical proofs. If you have seen a mathe-

matical result, if you want to know why the result is so, you can look for
the proof here.
The book’s purpose is to convey the essential ideas underlying the deriva-
tions of a large number of mathematical results useful in the modeling
of physical systems. To this end, the book emphasizes main threads of
mathematical argument and the motivation underlying the main threads,
deëmphasizing formal mathematical rigor. It derives mathematical results
from the purely applied perspective of the scientist and the engineer.
The book’s chapters are topical. This first chapter treats a few intro-
ductory matters of general interest.
1.1 Applied mathematics

What is applied mathematics?
Applied mathematics is a branch of mathematics that concerns
itself with the application of mathematical knowledge to other
domains. . . . The question of what is applied mathematics does
not answer to logical classification so much as to the sociology
of professionals who use mathematics. [41]
That is about right, on both counts. In this book we shall define ap-
plied mathematics to be correct mathematics useful to scientists, engineers
and the like; proceeding not from reduced, well defined sets of axioms but
rather directly from a nebulous mass of natural arithmetical, geometrical
and classical-algebraic idealizations of physical systems; demonstrable but
generally lacking the detailed rigor of the professional mathematician.
1
2 CHAPTER 1. INTRODUCTION
1.2 Rigor
It is impossible to write such a book as this without some discussion of math-
ematical rigor. Applied and professional mathematics differ principally and
essentially in the layer of abstract definitions the latter subimposes beneath
the physical ideas the former seeks to model. Notions of mathematical rigor
fit far more comfortably in the abstract realm of the professional mathe-
matician; they do not always translate so gracefully to the applied realm.
The applied mathematical reader and practitioner needs to be aware of this
difference.
1.2.1 Axiom and definition

Ideally, a professional mathematician knows or precisely specifies in advance
the set of fundamental axioms he means to use to derive a result. A prime
aesthetic here is irreducibility: no axiom in the set should overlap the others
or be specifiable in terms of the others. Geometrical argument—proof by
sketch—is distrusted. The professional mathematical literature discourages
undue pedantry indeed, but its readers do implicitly demand a convincing
assurance that its writers could derive results in pedantic detail if called
upon to do so. Precise definition here is critically important, which is why
the professional mathematician tends not to accept blithe statements such
as that
1
= ∞,
0
without first inquiring as to exactly what is meant by symbols like 0 and ∞.
The applied mathematician begins from a different base. His ideal lies
not in precise definition or irreducible axiom, but rather in the elegant mod-
eling of the essential features of some physical system. Here, mathematical
definitions tend to be made up ad hoc along the way, based on previous
experience solving similar problems, adapted implicitly to suit the model at
hand. If you ask the applied mathematician exactly what his axioms are,
which symbolic algebra he is using, he usually doesn’t know; what he knows
is that the bridge has its footings in certain soils with specified tolerances,
suffers such-and-such a wind load, etc. To avoid error, the applied mathe-
matician relies not on abstract formalism but rather on a thorough mental
grasp of the essential physical features of the phenomenon he is trying to
model. An equation like
1
=∞
0
1.2. RIGOR 3
may make perfect sense without further explanation to an applied mathe-

matical readership, depending on the physical context in which the equation
is introduced. Geometrical argument—proof by sketch—is not only trusted
but treasured. Abstract definitions are wanted only insofar as they smooth
the analysis of the particular physical problem at hand; such definitions are
seldom promoted for their own sakes.
The irascible Oliver Heaviside, responsible for the important applied
mathematical technique of phasor analysis, once said,
It is shocking that young people should be addling their brains

over mere logical subtleties, trying to understand the proof of
one obvious fact in terms of something equally . . . obvious. [46]
Exaggeration, perhaps, but from the applied mathematical perspective

Heaviside nevertheless had a point. The professional mathematicians
Richard Courant and David Hilbert put it more soberly in 1924 when they
wrote,
Since the seventeenth century, physical intuition has served as

a vital source for mathematical problems and methods. Recent
trends and fashions have, however, weakened the connection be-
tween mathematics and physics; mathematicians, turning away
from the roots of mathematics in intuition, have concentrated on
refinement and emphasized the postulational side of mathemat-
ics, and at times have overlooked the unity of their science with
physics and other fields. In many cases, physicists have ceased
to appreciate the attitudes of mathematicians. [14, Preface]
Although the present book treats “the attitudes of mathematicians” with

greater deference than some of the unnamed 1924 physicists might have
done, still, Courant and Hilbert could have been speaking for the engineers
and other applied mathematicians of our own day as well as for the physicists
of theirs. To the applied mathematician, the mathematics is not principally
meant to be developed and appreciated for its own sake; it is meant to be
used. This book adopts the Courant-Hilbert perspective.
The introduction you are now reading is not the right venue for an essay
on why both kinds of mathematics—applied and professional (or pure)—
are needed. Each kind has its place; and although it is a stylistic error
to mix the two indiscriminately, clearly the two have much to do with one
another. However this may be, this book is a book of derivations of applied
mathematics. The derivations here proceed by a purely applied approach.
Figure 1.1: Two triangles.
h h
b1 b2 −b2
b b
1.2.2 Mathematical extension

Profound results in mathematics are occasionally achieved simply by ex-
tending results already known. For example, negative integers and their
properties can be discovered by counting backward—3, 2, 1, 0—then asking
what follows (precedes?) 0 in the countdown and what properties this new,
negative integer must have to interact smoothly with the already known
positives. The astonishing Euler’s formula (§ 5.4) is discovered by a similar
but more sophisticated mathematical extension.
More often, however, the results achieved by extension are unsurprising
and not very interesting in themselves. Such extended results are the faithful
servants of mathematical rigor. Consider for example the triangle on the left
of Fig. 1.1. This triangle is evidently composed of two right triangles of areas
b1 h
A1 = ,
2
b2 h
A2 =
2
(each right triangle is exactly half a rectangle). Hence the main triangle’s
area is
(b1 + b2 )h bh
A = A1 + A2 = = .
2 2
Very well. What about the triangle on the right? Its b1 is not shown on the
figure, and what is that −b2 , anyway? Answer: the triangle is composed of
the difference of two right triangles, with b1 the base of the larger, overall
one: b1 = b + (−b2 ). The b2 is negative because the sense of the small right
triangle’s area in the proof is negative: the small area is subtracted from
1.3. COMPLEX NUMBERS AND COMPLEX VARIABLES 5
the large rather than added. By extension on this basis, the main triangle’s
area is again seen to be A = bh/2. The proof is exactly the same. In fact,
once the central idea of adding two right triangles is grasped, the extension
is really rather obvious—too obvious to be allowed to burden such a book
as this.
Excepting the uncommon cases where extension reveals something in-
teresting or new, this book generally leaves the mere extension of proofs—
including the validation of edge cases and over-the-edge cases—as an exercise
to the interested reader.
1.3 Complex numbers and complex variables

More than a mastery of mere logical details, it is an holistic view of the
mathematics and of its use in the modeling of physical systems which is the
mark of the applied mathematician. A feel for the math is the great thing.
Formal definitions, axioms, symbolic algebras and the like, though often
useful, are felt to be secondary. The book’s rapidly staged development of
complex numbers and complex variables is planned on this sensibility.
Sections 2.12, 3.10, 3.11, 4.3.3, 4.4, 6.2, 9.5 and 9.6.5, plus all of Chs. 5
and 8, constitute the book’s principal stages of complex development. In
these sections and throughout the book, the reader comes to appreciate that
most mathematical properties which apply for real numbers apply equally
for complex, that few properties concern real numbers alone.
Pure mathematics develops an abstract theory of the complex variable.1
The abstract theory is quite beautiful. However, its arc takes off too late
and flies too far from applications for such a book as this. Less beautiful but
more practical paths to the topic exist;2 this book leads the reader along
one of these.
For supplemental reference, a bare sketch of the abstract theory of the
complex variable is found in Appendix C.
1.4 On the text

The book gives numerals in hexadecimal. It denotes variables in Greek
letters as well as Roman. Readers unfamiliar with the hexadecimal notation
will find a brief orientation thereto in Appendix A. Readers unfamiliar with
the Greek alphabet will find it in Appendix B.
1
[3][20][57][31]
2
See Ch. 8’s footnote 8.
Licensed to the public under the GNU General Public Licence [22], ver-
sion 2, this book meets the Debian Free Software Guidelines [16].
If you cite an equation, section, chapter, figure or other item from this
book, it is recommended that you include in your citation the book’s precise
draft date as given on the title page. The reason is that equation numbers,
chapter numbers and the like are numbered automatically by the LATEX
typesetting software: such numbers can change arbitrarily from draft to
draft. If an exemplary citation helps, see [8] in the bibliography.
Part I
The calculus of a single

variable
7
Chapter 2
Classical algebra and

geometry
Arithmetic and the simplest elements of classical algebra and geometry, we

learn as children. Few readers will want this book to begin with a treatment
of 1 + 1 = 2; or of how to solve 3x − 2 = 7. However, there are some basic
points which do seem worth touching. The book starts with these.
2.1 Basic arithmetic relationships

This section states some arithmetical rules.
2.1.1 Commutivity, associativity, distributivity, identity and

inversion
Table 2.1 lists several arithmetical rules, each of which applies not only to
real numbers but equally to the complex numbers of § 2.12. Most of the
rules are appreciated at once if the meaning of the symbols is understood. In
the case of multiplicative commutivity, one imagines a rectangle with sides
of lengths a and b, then the same rectangle turned on its side, as in Fig. 2.1:
since the area of the rectangle is the same in either case, and since the area
is the length times the width in either case (the area is more or less a matter
of counting the little squares), evidently multiplicative commutivity holds.
A similar argument validates multiplicative associativity, except that here
we compute the volume of a three-dimensional rectangular box, which box
we turn various ways.1
1
[57, Ch. 1]
9
10 CHAPTER 2. CLASSICAL ALGEBRA AND GEOMETRY
Table 2.1: Basic properties of arithmetic.
a+b = b+a Additive commutivity

a + (b + c) = (a + b) + c Additive associativity
a+0=0+a = a Additive identity
a + (−a) = 0 Additive inversion
ab = ba Multiplicative commutivity
(a)(bc) = (ab)(c) Multiplicative associativity
(a)(1) = (1)(a) = a Multiplicative identity
(a)(1/a) = 1 Multiplicative inversion
(a)(b + c) = ab + ac Distributivity
Figure 2.1: Multiplicative commutivity.
b a
a
b
2.1. BASIC ARITHMETIC RELATIONSHIPS 11
Multiplicative inversion lacks an obvious interpretation when a = 0.

Loosely,
1
= ∞.
0
But since 3/0 = ∞ also, surely either the zero or the infinity, or both,
somehow differ in the latter case.
Looking ahead in the book, we note that the multiplicative properties
do not always hold for more general linear transformations. For example,
matrix multiplication is not commutative and vector cross-multiplication is
not associative. Where associativity does not hold and parentheses do not
otherwise group, right-to-left association is notationally implicit:2,3
A × B × C = A × (B × C).
The sense of it is that the thing on the left (A × ) operates on the thing on
the right (B × C). (In the rare case in which the question arises, you may
want to use parentheses anyway.)
2.1.2 Negative numbers
Consider that
(+a)(+b) = +ab,
(+a)(−b) = −ab,
(−a)(+b) = −ab,
(−a)(−b) = +ab.
The first three of the four equations are unsurprising, but the last is inter-
esting. Why would a negative count −a of a negative quantity −b come to
2
The fine C and C++ programming languages are unfortunately stuck with the reverse
order of association, along with division inharmoniously on the same level of syntactic
precedence as multiplication. Standard mathematical notation is more elegant:
(a)(bc)
abc/uvw = .
(u)(vw)
3
The nonassociative cross product B × C is introduced in § 15.2.2.
a positive product +ab? To see why, consider the progression
..
.
(+3)(−b) = −3b,
(+2)(−b) = −2b,
(+1)(−b) = −1b,
(0)(−b) = 0b,
(−1)(−b) = +1b,
(−2)(−b) = +2b,
(−3)(−b) = +3b,
..
.
The logic of arithmetic demands that the product of two negative numbers
be positive for this reason.
2.1.3 Inequality
If4
a < b,
then necessarily
a + x < b + x.
However, the relationship between ua and ub depends on the sign of u:
ua < ub if u > 0;
ua > ub if u < 0.
Also,
1 1
> .
a b
2.1.4 The change of variable

The applied mathematician very often finds it convenient to change vari-
ables, introducing new symbols to stand in place of old. For this we have
4
Few readers attempting this book will need to be reminded that < means “is less
than,” that > means “is greater than,” or that ≤ and ≥ respectively mean “is less than
or equal to” and “is greater than or equal to.”
2.2. QUADRATICS 13
the change of variable or assignment notation5
Q ← P.
This means, “in place of P , put Q”; or, “let Q now equal P .” For example,
if a2 + b2 = c2 , then the change of variable 2µ ← a yields the new form
(2µ)2 + b2 = c2 .
Similar to the change of variable notation is the definition notation
Q ≡ P.
This means, “let the new symbol Q represent P .”6

The two notations logically mean about the same thing. Subjectively,
Q ≡ P identifies a quantity P sufficiently interesting to be given a permanent
name Q, whereas Q ← P implies nothing especially interesting about P or Q;
it just introduces a (perhaps temporary) new symbol Q to ease the algebra.
The concepts grow clearer as examples of the usage arise in the book.
2.2 Quadratics
Differences and sums of squares are conveniently factored as
a2 − b2 = (a + b)(a − b),
a2 + b2 = (a + ib)(a − ib),
(2.1)
a2 − 2ab + b2 = (a − b)2 ,
a2 + 2ab + b2 = (a + b)2
(where i is the imaginary unit, a number defined such that i2 = −1, in-
troduced in more detail in § 2.12 below). Useful as these four forms are,
5
There appears to exist no broadly established standard mathematical notation for
the change of variable, other than the = equal sign, which regrettably does not fill the
role well. One can indeed use the equal sign, but then what does the change of variable
k = k + 1 mean? It looks like a claim that k and k + 1 are the same, which is impossible.
The notation k ← k + 1 by contrast is unambiguous; it means to increment k by one.
However, the latter notation admittedly has seen only scattered use in the literature.
The C and C++ programming languages use == for equality and = for assignment
(change of variable), as the reader may be aware.
6
One would never write k ≡ k + 1. Even k ← k + 1 can confuse readers inasmuch as
it appears to imply two different values for the same symbol k, but the latter notation is
sometimes used anyway when new symbols are unwanted or because more precise alter-
natives (like kn = kn−1 + 1) seem overwrought. Still, usually it is better to introduce a
new symbol, as in j ← k + 1.
In some books, ≡ is printed as ,.
however, none of them can directly factor the more general quadratic7 ex-
pression
z 2 − 2βz + γ 2 .
To factor this, we complete the square, writing
z 2 − 2βz + γ 2 = z 2 − 2βz + γ 2 + (β 2 − γ 2 ) − (β 2 − γ 2 )
= z 2 − 2βz + β 2 − (β 2 − γ 2 )
= (z − β)2 − (β 2 − γ 2 ).
The expression evidently has roots8 where
(z − β)2 = (β 2 − γ 2 ),
or in other words where9
p
z=β± β2 − γ2. (2.2)
This suggests the factoring10
z 2 − 2βz + γ 2 = (z − z1 )(z − z2 ), (2.3)
where z1 and z2 are the two values of z given by (2.2).
It follows that the two solutions of the quadratic equation
z 2 = 2βz − γ 2 (2.4)
are those given by (2.2), which is called the quadratic formula.11 (Cubic and
quartic formulas also exist respectively to extract the roots of polynomials
of third and fourth order, but they are much harder. See Ch. 10 and its
Tables 10.1 and 10.2.)
7
The adjective quadratic refers to the algebra of expressions in which no term has
greater than second order. Examples of quadratic expressions include x2 , 2x2 − 7x + 3 and
x2 +2xy +y 2 . By contrast, the expressions x3 −1 and 5x2 y are cubic not quadratic because
they contain third-order terms. First-order expressions like x + 1 are linear; zeroth-order
expressions like 3 are constant. Expressions of fourth and fifth order are quartic and
quintic, respectively. (If not already clear from the context, order basically refers to the
number of variables multiplied together in a term. The term 5x2 y = 5[x][x][y] is of third
order, for instance.)
8
A root of f (z) is a value of z for which f (z) = 0. See § 2.11.
9
The symbol ± means “+ or −.” In conjunction with this symbol, the alternate
symbol ∓ occasionally also appears, meaning “− or +”—which is the same thing except
that, where the two symbols appear together, (±z) + (∓z) = 0.
10
It suggests it because the expressions on the left and right sides of (2.3) are both
quadratic (the highest power is z 2 ) and have the same roots. Substituting into the equation
the values of z1 and z2 and simplifying proves the suggestion correct.
11
The form of the quadratic formula which usually appears in print is
√
−b ± b2 − 4ac
x= ,
2a
2.3. INTEGER AND SERIES NOTATION 15
2.3 Integer and series notation

Sums and products of series arise so frequently in mathematical work that
one finds it convenient to define terse notations to express them. The sum-
mation notation
Xb
f (k)
k=a
means to let k equal each of the integers a, a+ 1, a+ 2, . . . , b in turn, evaluat-

ing the function f (k) at each k, then adding the several f (k). For example,12
6
X
k2 = 32 + 42 + 52 + 62 = 0x56.
k=3
The similar multiplication notation
b
Y
f (j)
j=a
P
meansQ to multiply the several f (j) rather than to add them. The symbols
and come respectively from the Greek letters for S and P, and may be
regarded as standing for “Sum” and “Product.” The j or k is a dummy
variable, index of summation or loop counter —a variable with no indepen-
dent existence, used only to facilitate the addition
Q or multiplication of the
13
Q
series. (Nothing prevents one from writing k rather than j , inciden-
tally. For a dummy variable,
P oneQcan use any letter one likes. However, the
general habit of writing k and j proves convenient at least in § 4.5.2 and
Ch. 8, so we start now.)
which solves the quadratic ax2 + bx + c = 0. However, this writer finds the form (2.2)
easier to remember. For example, by (2.2) in light of (2.4), the quadratic
z 2 = 3z − 2
has the solutions s„ «

2
3 3
z= ± − 2 = 1 or 2.
2 2
12
The hexadecimal numeral 0x56 represents the same number the decimal numeral 86
represents. The book’s preface explains why the book represents such numbers in hex-
adecimal. Appendix A tells how to read the numerals.
13
Section 7.3 speaks further of the dummy variable.
The product shorthand

n
Y
n! ≡ j,
j=1
n
Y
n!/m! ≡ j,
j=m+1
is very frequently used. The notation n! is pronounced “n factorial.” Re-

garding the notation n!/m!, this can of course be regarded correctly as n!
divided by m! , but it usually proves more amenable to regard the notation
as a single unit.14
Because multiplication in its more general sense as linear transformation
(§ 11.1.1) is not always commutative, we specify that
b
Y
f (j) = [f (b)][f (b − 1)][f (b − 2)] · · · [f (a + 2)][f (a + 1)][f (a)]
j=a
rather than the reverse order of multiplication.15 Multiplication proceeds

from right to left. In the event that the reverse order of multiplication is
needed, we shall use the notation
b
a
f (j) = [f (a)][f (a + 1)][f (a + 2)] · · · [f (b − 2)][f (b − 1)][f (b)].
j=a
Note that for the sake of definitional consistency,

N
X N
X
f (k) = 0 + f (k) = 0,
k=N +1 k=N +1
N
Y N
Y
f (j) = (1) f (j) = 1.
j=N +1 j=N +1
This means among other things that
0! = 1. (2.5)
14
One reason among others for this is that factorials rapidly multiply to extremely large
sizes, overflowing computer registers during numerical computation. If you can avoid
unnecessary multiplication by regarding n!/m! as a single unit, this is a win.
15
The extant mathematical Q literature lacks an established standard on the order of
multiplication implied by the “ ” symbol, but this is the order we shall use in this book.
2.4. THE ARITHMETIC SERIES 17
Context tends to make the notation

N, j, k ∈ Z
unnecessary, but if used (as here and in § 2.5) it states explicitly that N , j
and k are integers. (The symbol Z represents16 the set of all integers: Z ≡
{. . . , −5, −4, −3, −2, −1, 0, 1, 2, 3, 4, 5, . . .}. The symbol ∈ means “belongs
to” or “is a member of.” Integers conventionally get the letters17 i, j, k,
m, n, M and N when available—though i is sometimes avoided because the
same letter represents the imaginary unit of § 2.12. Where additional letters
are needed ℓ, p and q, plus the capitals of these and the earlier listed letters,
can be pressed into service, occasionally joined even by r and s. Greek
letters are avoided, as—ironically in light of the symbol Z—are the Roman
letters x, y and z. Refer to Appendix
P QB.)
On first encounter, the and notation seems a bit overwrought,
whether or not the ∈ Z notation also is used. Admittedly it is easier for the
beginner to read “f (1)+ f (2)+ · · · + f (N )” than “ N
P
k=1 f (k).” However, ex-
perience shows the latter notation to be extremely useful in expressing more
sophisticated mathematical ideas. We shall use such notation extensively in
this book.
2.4 The arithmetic series

A simple yet useful application of the series sum of § 2.3 is the arithmetic
series
Xb
k = a + (a + 1) + (a + 2) + · · · + b.
k=a
Pairing a with b, then a+ 1 with b− 1, then a+ 2 with b− 2, etc., the average
of each pair is [a+b]/2; thus the average of the entire series is [a+b]/2. (The
pairing may or may not leave an unpaired element at the series midpoint
k = [a + b]/2, but this changes nothing.) The series has b − a + 1 terms.
Hence,
b
X a+b
k = (b − a + 1) . (2.6)
2
k=a
16
The letter Z recalls the transitive and intransitive German verb zählen, “to count.”
17
Though Fortran is perhaps less widely used a computer programming language than it
once was, it dominated applied-mathematical computer programming for decades, during
which the standard way to declare an integer variable to the Fortran compiler was simply
to let its name begin with I, J, K, L, M or N; so, this alphabetical convention is fairly well
cemented in practice.
Table 2.2: Power properties and definitions.
n
Y
n
z ≡ z, n ≥ 0
j=1
z = (z 1/n )n = (z n )1/n
√
z ≡ z 1/2
(uv)a = ua v a
z p/q = (z 1/q )p = (z p )1/q
z ab = (z a )b = (z b )a
z a+b = z a z b
za
z a−b =
zb
1
z −b =
zb
j, n, p, q ∈ Z
Success with
P∞this karithmetic series leads one to wonder about the geo-
metric series k=0 z . Section 2.6.4 addresses that point.
2.5 Powers and roots

This necessarily tedious section discusses powers and roots. It offers no
surprises. Table 2.2 summarizes its definitions and results. Readers seeking
more rewarding reading may prefer just to glance at the table then to skip
directly to the start of the next section.
In this section, the exponents
j, k, m, n, p, q, r, s ∈ Z
are integers, but the exponents a and b are arbitrary real numbers.
2.5.1 Notation and integral powers

The power notation
zn
2.5. POWERS AND ROOTS 19
indicates the number z, multiplied by itself n times. More formally, when

the exponent n is a nonnegative integer,18
n
Y
n
z ≡ z. (2.7)
j=1
For example,19
z 3 = (z)(z)(z),
z 2 = (z)(z),
z 1 = z,
z 0 = 1.
Notice that in general,

zn
z n−1 =.
z
This leads us to extend the definition to negative integral powers with
1
z −n = . (2.8)
zn
From the foregoing it is plain that
z m+n = z m z n ,
zm (2.9)
z m−n = n ,
z
for any integral m and n. For similar reasons,
z mn = (z m )n = (z n )m . (2.10)
On the other hand from multiplicative associativity and commutivity,
(uv)n = un v n . (2.11)
18
The symbol “≡” means “=”, but it further usually indicates that the expression on
its right serves to define the expression on its left. Refer to § 2.1.4.
19
The case 00 is interesting because it lacks an obvious interpretation. The specific
interpretation depends on the nature and meaning of the two zeros. For interest, if E ≡
1/ǫ, then
„ «1/E
ǫ 1
lim ǫ = lim = lim E −1/E = lim e−(ln E)/E = e0 = 1.
ǫ→0+ E→∞ E E→∞ E→∞
2.5.2 Roots
Fractional powers are not something we have defined yet, so for consistency
with (2.10) we let
(u1/n )n = u.
This has u1/n as the number which, raised to the nth power, yields u. Setting
v = u1/n ,
it follows by successive steps that
v n = u,
(v n )1/n = u1/n ,
(v n )1/n = v.
Taking the u and v formulas together, then,
(z 1/n )n = z = (z n )1/n (2.12)
for any z and integral n.

The number z 1/n is called the nth root of z—or in the very common case
n = 2, the square root of z, often written
√
z.
When z is real and nonnegative, the last notation is usually implicitly taken
to mean the real, nonnegative square root. In any case, the power and root
operations mutually invert one another.
What about powers expressible neither as n nor as 1/n, such as the 3/2
power? If z and w are numbers related by
wq = z,
then
wpq = z p .
Taking the qth root,
wp = (z p )1/q .
But w = z 1/q , so this is
(z 1/q )p = (z p )1/q ,
2.5. POWERS AND ROOTS 21
which says that it does not matter whether one applies the power or the
root first; the result is the same. Extending (2.10) therefore, we define z p/q
such that
(z 1/q )p = z p/q = (z p )1/q . (2.13)
Since any real number can be approximated arbitrarily closely by a ratio of
integers, (2.13) implies a power definition for all real exponents.
Equation (2.13) is this subsection’s main result. However, § 2.5.3 will
find it useful if we can also show here that
(z 1/q )1/s = z 1/qs = (z 1/s )1/q . (2.14)
The proof is straightforward. If
w ≡ z 1/qs ,
then raising to the qs power yields
(ws )q = z.
Successively taking the qth and sth roots gives
w = (z 1/q )1/s .
By identical reasoning,
w = (z 1/s )1/q .
But since w ≡ z 1/qs , the last two equations imply (2.14), as we have sought.
2.5.3 Powers of products and powers of powers

Per (2.11),
(uv)p = up v p .
Raising this equation to the 1/q power, we have that
(uv)p/q = [up v p ]1/q

h i1/q
= (up )q/q (v p )q/q
h i1/q
= (up/q )q (v p/q )q
h iq/q
= (up/q )(v p/q )
= up/q v p/q .
In other words
(uv)a = ua v a (2.15)
for any real a.
On the other hand, per (2.10),
z pr = (z p )r .
Raising this equation to the 1/qs power and applying (2.10), (2.13) and
(2.14) to reorder the powers, we have that
z (p/q)(r/s) = (z p/q )r/s .
By identical reasoning,
z (p/q)(r/s) = (z r/s )p/q .
Since p/q and r/s can approximate any real numbers with arbitrary preci-
sion, this implies that
(z a )b = z ab = (z b )a (2.16)
for any real a and b.
2.5.4 Sums of powers

With (2.9), (2.15) and (2.16), one can reason that
z (p/q)+(r/s) = (z ps+rq )1/qs = (z ps z rq )1/qs = z p/q z r/s ,
or in other words that

z a+b = z a z b . (2.17)
In the case that a = −b,
1 = z −b+b = z −b z b ,
which implies that

1
z −b =
. (2.18)
zb
But then replacing −b ← b in (2.17) leads to
z a−b = z a z −b ,
which according to (2.18) is

za
z a−b = . (2.19)
zb
2.6. MULTIPLYING AND DIVIDING POWER SERIES 23
2.5.5 Summary and remarks

Table 2.2 on page 18 summarizes the section’s definitions and results.
Looking ahead to § 2.12, § 3.11 and Ch. 5, we observe that nothing in
the foregoing analysis requires the base variables z, w, u and v to be real
numbers; if complex (§ 2.12), the formulas remain valid. Still, the analysis
does imply that the various exponents m, n, p/q, a, b and so on are real
numbers. This restriction, we shall remove later, purposely defining the
action of a complex exponent to comport with the results found here. With
such a definition the results apply not only for all bases but also for all
exponents, real or complex.
2.6 Multiplying and dividing power series

A power series 20 is a weighted sum of integral powers:
∞
X
A(z) = ak z k , (2.20)
k=−∞
where the several ak are arbitrary constants. This section discusses the
multiplication and division of power series.
20
Another name for the power series is polynomial. The word “polynomial” usually
connotes a power series with a finite number of terms, but the two names in fact refer to
essentially the same thing.
Professional mathematicians use the terms more precisely. Equation (2.20), they call a
“power series” only if ak = 0 for all k < 0—in other words, technically, not if it includes
negative powers of z. They call it a “polynomial” only if it is a “power series” with a
finite number of terms. They call (2.20) in general a Laurent series.
The name “Laurent series” is a name we shall meet again in § 8.14. In the meantime
however we admit that the professionals have vaguely daunted us by adding to the name
some pretty sophisticated connotations, to the point that we applied mathematicians (at
least in the author’s country) seem to feel somehow unlicensed actually to use the name.
We tend to call (2.20) a “power series with negative powers,” or just “a power series.”
This book follows the last usage. You however can call (2.20) a Laurent series if you
prefer (and if you pronounce it right: “lor-ON”). That is after all exactly what it is.
Nevertheless if you do use the name “Laurent series,” be prepared for people subjectively—
for no particular reason—to expect you to establish complex radii of convergence, to
sketch some annulus in the Argand plane, and/or to engage in other maybe unnecessary
formalities. If that is not what you seek, then you may find it better just to call the thing
by the less lofty name of “power series”—or better, if it has a finite number of terms, by
the even humbler name of “polynomial.”
Semantics. All these names mean about the same thing, but one is expected most
2.6.1 Multiplying power series

Given two power series
∞
X
A(z) = ak z k ,
k=−∞
∞ (2.21)
X
k
B(z) = bk z ,
k=−∞
the product of the two series is evidently

∞
X ∞
X
P (z) ≡ A(z)B(z) = aj bk−j z k . (2.22)
k=−∞ j=−∞
2.6.2 Dividing power series

The quotient Q(z) = B(z)/A(z) of two power series is a little harder to
calculate, and there are at least two ways to do it. Section 2.6.3 below will
do it by matching coefficients, but this subsection does it by long division.
For example,
2z 2 − 3z + 3 2z 2 − 4z z + 3 z+3
= + = 2z +
z−2 z−2 z−2 z−2
z−2 5 5
= 2z + + = 2z + 1 + .
z−2 z−2 z−2
The strategy is to take the dividend21 B(z) piece by piece, purposely choos-
ing pieces easily divided by A(z).
carefully always to give the right name in the right place. What a bother! (Someone
once told the writer that the Japanese language can give different names to the same
object, depending on whether the speaker is male or female. The power-series terminology
seems to share a spirit of that kin.) If you seek just one word for the thing, the writer
recommends that you call it a “power series” and then not worry too much about it
until someone objects. When someone does object, you can snow him with the big word
“Laurent series,” instead.
The experienced scientist or engineer may notice that the above vocabulary omits the
name “Taylor series.” The vocabulary omits the name because that name fortunately
remains unconfused in usage—it means quite specifically a power series without negative
powers and tends to connote a representation of some particular function of interest—as
we shall see in Ch. 8.
21
If Q(z) is a quotient and R(z) a remainder, then B(z) is a dividend (or numerator )
and A(z) a divisor (or denominator ). Such are the Latin-derived names of the parts of a
long division.
If you feel that you understand the example, then that is really all there
is to it, and you can skip the rest of the subsection if you like. One sometimes
wants to express the long division of power series more formally, however.
That is what the rest of the subsection is about. (Be advised however that
the cleverer technique of § 2.6.3, though less direct, is often easier and faster.)
Formally, we prepare the long division B(z)/A(z) by writing
B(z) = A(z)Qn (z) + Rn (z), (2.23)
where Rn (z) is a remainder (being the part of B[z] remaining to be divided);

and
K
X
A(z) = ak z k , aK 6= 0,
k=−∞
N
X
B(z) = bk z k ,
k=−∞
RN (z) = B(z),
(2.24)
QN (z) = 0,
X n
Rn (z) = rnk z k ,
k=−∞
NX
−K
Qn (z) = qk z k ,
k=n−K+1
where K and N identify the greatest orders k of z k present in A(z) and

B(z), respectively.
Well, that is a lot of symbology. What does it mean? The key to
understanding it lies in understanding (2.23), which is not one but several
equations—one equation for each value of n, where n = N, N − 1, N − 2, . . . .
The dividend B(z) and the divisor A(z) stay the same from one n to the
next, but the quotient Qn (z) and the remainder Rn (z) change. At start,
QN (z) = 0 while RN (z) = B(z), but the thrust of the long division process
is to build Qn (z) up by wearing Rn (z) down. The goal is to grind Rn (z)
away to nothing, to make it disappear as n → −∞.
As in the example, we pursue the goal by choosing from Rn (z) an easily
divisible piece containing the whole high-order term of Rn (z). The piece
we choose is (rnn /aK )z n−K A(z), which we add and subtract from (2.23) to
obtain

rnn n−K rnn n−K
B(z) = A(z) Qn (z) + z + Rn (z) − z A(z) .
aK aK
Matching this equation against the desired iterate
B(z) = A(z)Qn−1 (z) + Rn−1 (z)
and observing from the definition of Qn (z) that Qn−1 (z) = Qn (z) +
qn−K z n−K , we find that
rnn
qn−K = ,
aK (2.25)
Rn−1 (z) = Rn (z) − qn−K z n−K A(z),
where no term remains in Rn−1 (z) higher than a z n−1 term.
To begin the actual long division, we initialize
RN (z) = B(z),
for which (2.23) is trivially true. Then we iterate per (2.25) as many times
as desired. If an infinite number of times, then so long as Rn (z) tends to
vanish as n → −∞, it follows from (2.23) that
B(z)
= Q−∞ (z). (2.26)
A(z)
Iterating only a finite number of times leaves a remainder,
B(z) Rn (z)
= Qn (z) + , (2.27)
A(z) A(z)
except that it may happen that Rn (z) = 0 for sufficiently small n.
Table 2.3 summarizes the long-division procedure.22 In its qn−K equa-
tion, the table includes also the result of § 2.6.3 below.
It should be observed in light of Table 2.3 that if23
K
X
A(z) = ak z k ,
k=Ko
N
X
B(z) = bk z k ,
k=No
22
[59, § 3.2]
23
The notations Ko , ak and z k are usually pronounced, respectively, as “K naught,” “a
sub k” and “z to the k” (or, more fully, “z to the kth power”)—at least in the author’s
country.
Table 2.3: Dividing power series through successively smaller powers.
B(z) = A(z)Qn (z) + Rn (z)

K
X
A(z) = ak z k , aK 6= 0
k=−∞
N
X
B(z) = bk z k
k=−∞
RN (z) = B(z)
QN (z) = 0
Xn
Rn (z) = rnk z k
k=−∞
NX
−K
Qn (z) = qk z k
k=n−K+1
NX
−K
!
rnn 1
qn−K = = bn − an−k qk
aK aK
k=n−K+1
n−K
Rn−1 (z) = Rn (z) − qn−K z A(z)
B(z)
= Q−∞ (z)
A(z)
then
n
X
Rn (z) = rnk z k for all n < No + (K − Ko ). (2.28)
k=n−(K−Ko )+1
That is, the remainder has order one less than the divisor has. The reason
for this, of course, is that we have strategically planned the long-division
iteration precisely to cause the leading term of the divisor to cancel the
leading term of the remainder at each step.24
The long-division procedure of Table 2.3 extends the quotient Qn (z)
through successively smaller powers of z. Often, however, one prefers to
extend the quotient through successively larger powers of z, where a z K
term is A(z)’s term of least order. In this case, the long division goes by the
complementary rules of Table 2.4.
2.6.3 Dividing power series by matching coefficients

There is another, sometimes quicker way to divide power series than by the
long division of § 2.6.2. One can divide them by matching coefficients.25 If
B(z)
Q∞ (z) = , (2.29)
A(z)
where
∞
X
A(z) = ak z k , aK 6= 0,
k=K
X∞
B(z) = bk z k
k=N
24
If a more formal demonstration of (2.28) is wanted, then consider per (2.25) that
rmm m−K
Rm−1 (z) = Rm (z) − z A(z).
aK
If the least-order term of Rm (z) is a z No term (as clearly is the case at least for the
initial remainder RN [z] = B[z]), then according to the equation so also must the least-
order term of Rm−1 (z) be a z No term, unless an even lower-order term be contributed
by the product z m−K A(z). But that very product’s term of least order is a z m−(K−Ko )
term. Under these conditions, evidently the least-order term of Rm−1 (z) is a z m−(K−Ko )
term when m − (K − Ko ) ≤ No ; otherwise a z No term. This is better stated after the
change of variable n + 1 ← m: the least-order term of Rn (z) is a z n−(K−Ko )+1 term when
n < No + (K − Ko ); otherwise a z No term.
The greatest-order term of Rn (z) is by definition a z n term. So, in summary, when n <
No + (K − Ko ), the terms of Rn (z) run from z n−(K−Ko )+1 through z n , which is exactly
the claim (2.28) makes.
25
[40][20, § 2.5]
Table 2.4: Dividing power series through successively larger powers.
B(z) = A(z)Qn (z) + Rn (z)

X∞
A(z) = ak z k , aK 6= 0
k=K
X∞
B(z) = bk z k
k=N
RN (z) = B(z)
QN (z) = 0
X∞
Rn (z) = rnk z k
k=n
n−K−1
X
Qn (z) = qk z k
k=N −K
n−K−1
!
rnn 1 X
qn−K = = bn − an−k qk
aK aK
k=N −K
n−K
Rn+1 (z) = Rn (z) − qn−K z A(z)
B(z)
= Q∞ (z)
A(z)
are known and

∞
X
Q∞ (z) = qk z k
k=N −K
is to be calculated, then one can multiply (2.29) through by A(z) to obtain

the form
A(z)Q∞ (z) = B(z).
Expanding the left side according to (2.22) and changing the index n ← k
on the right side,
∞
X n−K
X ∞
X
an−k qk z n = bn z n .
n=N k=N −K n=N
But for this to hold for all z, the coefficients must match for each n:
n−K
X
an−k qk = bn , n ≥ N.
k=N −K
Transferring all terms but aK qn−K to the equation’s right side and dividing
by aK , we have that
n−K−1
!
1 X
qn−K = bn − an−k qk , n ≥ N. (2.30)
aK
k=N −K
Equation (2.30) computes the coefficients of Q(z), each coefficient depending

on the coefficients earlier computed.
The coefficient-matching technique of this subsection is easily adapted
to the division of series in decreasing, rather than increasing, powers of z
if needed or desired. The adaptation is left as an exercise to the interested
reader, but Tables 2.3 and 2.4 incorporate the technique both ways.
Admittedly, the fact that (2.30) yields a sequence of coefficients does not
necessarily mean that the resulting power series Q∞ (z) converges to some
definite value over a given domain. Consider for instance (2.34), which
diverges when26 |z| > 1, even though all its coefficients are known. At
least (2.30) is correct when Q∞ (z) does converge. Even when Q∞ (z) as
such does not converge, however, often what interest us are only the series’
first several terms
n−K−1
X
Qn (z) = qk z k .
k=N −K
26
See footnote 27.
In this case,
B(z) Rn (z)
Q∞ (z) = = Qn (z) + (2.31)
A(z) A(z)
and convergence is not an issue. Solving (2.31) for Rn (z),
Rn (z) = B(z) − A(z)Qn (z). (2.32)
2.6.4 Common power-series quotients and the geometric se-

ries
Frequently encountered power-series quotients, calculated by the long di-
vision of § 2.6.2, computed by the coefficient matching of § 2.6.3, and/or
verified by multiplying, include27
∞
X
 (∓)k z k , |z| < 1;



1 
k=0
= −1 (2.33)
1±z  X
k k
− (∓) z , |z| > 1.



k=−∞
Equation (2.33) almost incidentally answers a question which has arisen

in § 2.4 and whichP oftenk arises in practice: to what total does the infinite
geometric series ∞ k=0 z , |z| < 1, sum? Answer: it sums exactly to 1/(1 −
z). However, there is a simpler, more aesthetic way to demonstrate the same
thing, as follows. Let
X∞
S≡ z k , |z| < 1.
k=0
Multiplying by z yields
∞
X
zS ≡ zk .
k=1
Subtracting the latter equation from the former leaves
(1 − z)S = 1,
which, after dividing by 1 − z, implies that
∞
X 1
S≡ zk = , |z| < 1, (2.34)
1−z
k=0
as was to be demonstrated.
27
The notation |z| represents the magnitude of z. For example, |5| = 5, but also
|−5| = 5.
2.6.5 Variations on the geometric series

Besides being more aesthetic than the long division of § 2.6.2, the difference
technique of § 2.6.4 permits one to extend the basic geometric series in
several ways. For instance, the sum
∞
X
S1 ≡ kz k , |z| < 1
k=0
(which arises in, among others, Planck’s quantum blackbody radiation cal-
culation28 ), we can compute as follows. We multiply the unknown S1 by z,
producing
∞
X X∞
zS1 = kz k+1 = (k − 1)z k .
k=0 k=1
We then subtract zS1 from S1 , leaving

∞ ∞ ∞ ∞
X
k
X
k
X
k
X z
(1 − z)S1 = kz − (k − 1)z = z =z zk = ,
1−z
k=0 k=1 k=1 k=0
where we have used (2.34) to collapse the last sum. Dividing by 1 − z, we

arrive at
∞
X z
S1 ≡ kz k = , |z| < 1, (2.35)
(1 − z)2
k=0
which was to be found.

Further series of the kind, such as k k2 z k , k (k + 1)(k)z k , k k3 z k ,
P P P
etc., can be calculated in like manner as the need for them arises.
2.7 Indeterminate constants, independent vari-

ables and dependent variables
Mathematical models use indeterminate constants, independent variables
and dependent variables. The three are best illustrated by example. Con-
sider the time t a sound needs to travel from its source to a distant listener:
∆r
t= ,
vsound
28
[44]
2.7. CONSTANTS AND VARIABLES 33
where ∆r is the distance from source to listener and vsound is the speed of
sound. Here, vsound is an indeterminate constant (given particular atmo-
spheric conditions, it doesn’t vary), ∆r is an independent variable, and t
is a dependent variable. The model gives t as a function of ∆r; so, if you
tell the model how far the listener sits from the sound source, the model
returns the time needed for the sound to propagate from one to the other.
Note that the abstract validity of the model does not necessarily depend on
whether we actually know the right figure for vsound (if I tell you that sound
goes at 500 m/s, but later you find out that the real figure is 331 m/s, it
probably doesn’t ruin the theoretical part of your analysis; you just have
to recalculate numerically). Knowing the figure is not the point. The point
is that conceptually there preëxists some right figure for the indeterminate
constant; that sound goes at some constant speed—whatever it is—and that
we can calculate the delay in terms of this.
Although there exists a definite philosophical distinction between the
three kinds of quantity, nevertheless it cannot be denied that which par-
ticular quantity is an indeterminate constant, an independent variable or
a dependent variable often depends upon one’s immediate point of view.
The same model in the example would remain valid if atmospheric condi-
tions were changing (vsound would then be an independent variable) or if the
model were used in designing a musical concert hall29 to suffer a maximum
acceptable sound time lag from the stage to the hall’s back row (t would
then be an independent variable; ∆r, dependent). Occasionally we go so far
as deliberately to change our point of view in mid-analysis, now regarding
as an independent variable what a moment ago we had regarded as an inde-
terminate constant, for instance (a typical case of this arises in the solution
of differential equations by the method of unknown coefficients, § 9.4). Such
a shift of viewpoint is fine, so long as we remember that there is a difference
29
Math books are funny about examples like this. Such examples remind one of the kind
of calculation one encounters in a childhood arithmetic textbook, as of the quantity of air
contained in an astronaut’s round helmet. One could calculate the quantity of water in
a kitchen mixing bowl just as well, but astronauts’ helmets are so much more interesting
than bowls, you see.
The chance that the typical reader will ever specify the dimensions of a real musical
concert hall is of course vanishingly small. However, it is the idea of the example that mat-
ters here, because the chance that the typical reader will ever specify something technical
is quite large. Although sophisticated models with many factors and terms do indeed play
a major role in engineering, the great majority of practical engineering calculations—for
quick, day-to-day decisions where small sums of money and negligible risk to life are at
stake—are done with models hardly more sophisticated than the one shown here. So,
maybe the concert-hall example is not so unreasonable, after all.
between the three kinds of quantity and we keep track of which quantity is
which kind to us at the moment.
The main reason it matters which symbol represents which of the three
kinds of quantity is that in calculus, one analyzes how change in indepen-
dent variables affects dependent variables as indeterminate constants remain
fixed.
(Section 2.3 has introduced the dummy variable, which the present sec-
tion’s threefold taxonomy seems to exclude. However, in fact, most dummy
variables are just independent variables—a few are dependent variables—
whose scope is restricted to a particular expression. Such a dummy variable
does not seem very “independent,” of course; but its dependence is on the
operator controlling the expression, not on some other variable within the
expression. Within the expression, the dummy variable fills the role of an
independent variable; without, it fills no role because logically it does not
exist there. Refer to §§ 2.3 and 7.3.)
2.8 Exponentials and logarithms

In § 2.5 we have considered the power operation z a , where (in § 2.7’s lan-
guage) the independent variable z is the base and the indeterminate con-
stant a is the exponent. There is another way to view the power operation,
however. One can view it as the exponential operation
az ,
where the variable z is the exponent and the constant a is the base.
2.8.1 The logarithm

The exponential operation follows the same laws the power operation follows,
but because the variable of interest is now the exponent rather than the base,
the inverse operation is not the root but rather the logarithm:
loga (az ) = z. (2.36)
The logarithm loga w answers the question, “What power must I raise a to,
to get w?”
Raising a to the power of the last equation, we have that
z)
aloga (a = az .
2.8. EXPONENTIALS AND LOGARITHMS 35
With the change of variable w ← az , this is

aloga w = w. (2.37)
Hence, the exponential and logarithmic operations mutually invert one an-
other.
2.8.2 Properties of the logarithm

The basic properties of the logarithm include that
loga uv = loga u + loga v, (2.38)
u
loga = loga u − loga v, (2.39)
v
loga (wz ) = z loga w, (2.40)
z z loga w
w = a , (2.41)
loga w
logb w = . (2.42)
loga b
Of these, (2.38) follows from the steps
(uv) = (u)(v),
loga uv
(a ) = (aloga u )(aloga v ),
aloga uv = aloga u+loga v ;
and (2.39) follows by similar reasoning. Equations (2.40) and (2.41) follow
from the steps
wz = (wz ) = (w)z ,
z)
wz = aloga (w = (aloga w )z ,
(w z )
wz = aloga = az loga w .
Equation (2.42) follows from the steps
w = blogb w ,
loga w = loga (blogb w ),
loga w = logb w loga b.
Among other purposes, (2.38) through (2.42) serve respectively to trans-
form products to sums, quotients to differences, powers to products, expo-
nentials to differently based exponentials, and logarithms to differently based
logarithms. Table 2.5 repeats the equations along with (2.36) and (2.37)
(which also emerge as restricted forms of eqns. 2.40 and 2.41), thus summa-
rizing the general properties of the logarithm.
Table 2.5: General properties of the logarithm.
loga uv = loga u + loga v

u
loga = loga u − loga v
v
loga (wz ) = z loga w
wz = az loga w
loga w
logb w =
loga b
loga (az ) = z
w = aloga w
2.9 Triangles and other polygons: simple facts

This section gives simple facts about triangles and other polygons.
2.9.1 Triangle area

The area of a right triangle30 is half the area of the corresponding rectangle.
This is seen by splitting a rectangle down its diagonal into a pair of right
triangles of equal size. The fact that any triangle’s area is half its base length
times its height is seen by dropping a perpendicular from one point of the
triangle to the opposite side (see Fig. 1.1 on page 4), dividing the triangle
into two right triangles, for each of which the fact is true. In algebraic
symbols,
bh
A= , (2.43)
2
where A stands for area, b for base length, and h for perpendicular height.
2.9.2 The triangle inequalities

Any two sides of a triangle together are longer than the third alone, which
itself is longer than the difference between the two. In symbols,
|a − b| < c < a + b, (2.44)
where a, b and c are the lengths of a triangle’s three sides. These are the
triangle inequalities. The truth of the sum inequality c < a + b, is seen by
30
A right triangle is a triangle, one of whose three angles is perfectly square.
2.9. TRIANGLES AND OTHER POLYGONS: SIMPLE FACTS 37
Figure 2.2: The sum of a triangle’s inner angles: turning at the corner.
φ
ψ
sketching some triangle on a sheet of paper and asking: if c is the direct

route between two points and a + b is an indirect route, then how can a + b
not be longer? Of course the sum inequality is equally good on any of the
triangle’s three sides, so one can write a < c + b and b < c + a just as well
as c < a + b. Rearranging the a and b inequalities, we have that a − b < c
and b − a < c, which together say that |a − b| < c. The last is the difference
inequality, completing (2.44)’s proof.31
2.9.3 The sum of interior angles
A triangle’s three interior angles32 sum to 2π/2. One way to see the truth
of this fact is to imagine a small car rolling along one of the triangle’s sides.
Reaching the corner, the car turns to travel along the next side, and so on
round all three corners to complete a circuit, returning to the start. Since
the car again faces the original direction, we reason that it has turned a
total of 2π: a full revolution. But the angle φ the car turns at a corner
and the triangle’s inner angle ψ there together form the straight angle 2π/2
(the sharper the inner angle, the more the car turns: see Fig. 2.2). In
31
Section 13.9 proves the triangle inequalities more generally, though regrettably with-
out recourse to this subsection’s properly picturesque geometrical argument.
32
Many or most readers will already know the notation 2π and its meaning as the angle
of full revolution. For those who do not, the notation is introduced more properly in
§§ 3.1, 3.6 and 8.11 below. Briefly, however, the symbol 2π represents a complete turn,
a full circle, a spin to face the same direction as before. Hence 2π/4 represents a square
turn or right angle.
You may be used to the notation 360◦ in place of 2π, but for the reasons explained in
Appendix A and in footnote 16 of Ch. 3, this book tends to avoid the former notation.
mathematical notation,
φ1 + φ2 + φ3 = 2π,
2π
φk + ψk = , k = 1, 2, 3,
2
where ψk and φk are respectively the triangle’s inner angles and the an-
gles through which the car turns. Solving the latter equations for φk and
substituting into the former yields
2π
ψ1 + ψ2 + ψ3 = , (2.45)
2
which was to be demonstrated.
Extending the same technique to the case of an n-sided polygon, we have
that
n
X
φk = 2π,
k=1
2π
φk + ψk = .
2
Solving the latter equations for φk and substituting into the former yields
n
X 2π
− ψk = 2π,
2
k=1
or in other words n
X 2π
ψk = (n − 2) . (2.46)
2
k=1
Equation (2.45) is then seen to be a special case of (2.46) with n = 3.
2.10 The Pythagorean theorem

Along with Euler’s formula (5.11), the fundamental theorem of calculus (7.2)
and Cauchy’s integral formula (8.29), the Pythagorean theorem is one of the
most famous results in all of mathematics. The theorem holds that
a2 + b2 = c2 , (2.47)
where a, b and c are the lengths of the legs and diagonal of a right triangle,
as in Fig. 2.3. Many proofs of the theorem are known.
2.10. THE PYTHAGOREAN THEOREM 39
Figure 2.3: A right triangle.
c
b
Figure 2.4: The Pythagorean theorem.
c
b
b a
One such proof posits a square of side length a + b with a tilted square of
side length c inscribed as in Fig. 2.4. The area of each of the four triangles
in the figure is evidently ab/2. The area of the tilted inner square is c2 .
The area of the large outer square is (a + b)2 . But the large outer square is
comprised of the tilted inner square plus the four triangles, hence the area
of the large outer square equals the area of the tilted inner square plus the
areas of the four triangles. In mathematical symbols, this is

2 2 ab
(a + b) = c + 4 ,
2
which simplifies directly to (2.47).33

The Pythagorean theorem is readily extended to three dimensions as
a2 + b2 + h2 = r 2 , (2.48)
where h is an altitude perpendicular to both a and b, thus also to c; and

where r is the corresponding three-dimensional diagonal: the diagonal of
the right triangle whose legs are c and h. Inasmuch as (2.47) applies to any
right triangle, it follows that c2 + h2 = r 2 , which equation expands directly
to yield (2.48).
2.11 Functions
This book is not the place for a gentle introduction to the concept of the
function. Briefly, however, a function is a mapping from one number (or
vector of several numbers) to another. For example, f (x) = x2 − 1 is a
function which maps 1 to 0 and −3 to 8, among others.
One often speaks of domains and ranges when discussing functions. The
domain of a function is the set of numbers one can put into it. The range
of a function is the corresponding set of numbers one can get out of it. In
the example, if the domain is restricted to real x such that |x| ≤ 3, then the
corresponding range is −1 ≤ f (x) ≤ 8.
33
This elegant proof is far simpler than the one famously given by the ancient geometer
Euclid, yet more appealing than alternate proofs often found in print. Whether Euclid was
acquainted with the simple proof given here this writer does not know, but it is possible
[67, “Pythagorean theorem,” 02:32, 31 March 2006] that Euclid chose his proof because
it comported better with the restricted set of geometrical elements he permitted himself
to work with. Be that as it may, the present writer encountered the proof this section
gives somewhere years ago and has never seen it in print since, so can claim no credit for
originating it. Unfortunately the citation is now long lost. A current source for the proof
is [67] as cited earlier in this footnote.
2.12. COMPLEX NUMBERS (INTRODUCTION) 41
Other terms which arise when discussing functions are root (or zero),
singularity and pole. A root (or zero) of a function is a domain point at
which the function evaluates to zero (the example has roots at x = ±1). A
singularity of a function is a domain point at which the function’s output
diverges; that is, where the function’s output is infinite.34 A pole is a sin-
√
gularity that behaves locally like 1/x (rather than, for example, like 1/ x).
A singularity that behaves as 1/xN is a multiple pole, which (§ 9.6.2) can be
thought of as N poles. The example’s function f (x) has no singularities for
finite x; however, the function h(x) = 1/(x2 − 1) has poles at x = ±1.
(Besides the root, the singularity and the pole, there is also the trouble-
some branch point, an infamous example of which is z = 0 in the function
√
g[z] = z. Branch points are important, but the book must lay a more
extensive foundation before introducing them properly in § 8.5.35 )
2.12 Complex numbers (introduction)

Section 2.5.2 has introduced square
√roots. What it has not done is to tell us
how to regard a quantity such as −1. Since there exists no real number i
such that
i2 = −1 (2.49)
and since the quantity i thus defined is found to be critically important
across broad domains of higher mathematics, we accept (2.49) as the defi-
nition of a fundamentally new kind of quantity: the imaginary number.36
34
Here is one example of the book’s deliberate lack of formal mathematical rigor. A
more precise formalism to say that “the function’s output is infinite” might be
lim |f (x)| = ∞.
x→xo
The applied mathematician tends to avoid such formalism where there seems no immediate
use for it.
35
There is further the essential singularity, an example of which is z = 0 in p(z) =
exp(1/z), but the best way to handle such unreasonable singularities is almost always to
change a variable, as w ← 1/z, or otherwise to frame the problem such that one need
not approach the singularity. This book will have little to say of such singularities. (Such
singularities are however sometimes the thing one implicitly uses an asymptotic series to
route around, particularly in work with special functions—but special functions are an
advanced topic the book won’t even begin to treat until [chapters not yet written]; and,
even then, the book will not speak of the essential singularity explicitly.)
36
The English word imaginary is evocative, but perhaps not of quite the right concept
in this usage. Imaginary numbers are not to mathematics as, say, imaginary elfs are to the
physical world. In the physical world, imaginary elfs are (presumably) not substantial ob-
jects. However, in the mathematical realm, imaginary numbers are substantial. The word
Figure 2.5: The complex (or Argand) plane, and a complex number z
therein.
iℑ(z)
i2
i z
ρ
φ
ℜ(z)
−2 −1 1 2
−i
−i2
Imaginary numbers are given their own number line, plotted at right
angles to the familiar real number line as in Fig. 2.5. The sum of a real
number x and an imaginary number iy is the complex number
z = x + iy.
The conjugate z ∗ of this complex number is defined to be37
z ∗ = x − iy.
imaginary in the mathematical sense is thus more of a technical term than a descriptive
adjective.
The number i is just a concept, of course, but then so is the number 1 (though you and
I have often met one of something—one apple, one chair, one summer afternoon, etc.—
neither of us has ever met just 1). The reason imaginary numbers are called “imaginary”
probably has to do with the fact that they emerge from mathematical operations only,
never directly from counting things. Notice, however, that the number 1/2 never emerges
directly from counting things, either. If for some reason the iyear were offered as a unit
of time, then the period separating your fourteenth and twenty-first birthdays could have
been measured as −i7 iyears. Madness? No, let us not call it that; let us call it a useful
formalism, rather.
The unpersuaded reader is asked to suspend judgment a while. He will soon see the
use.
37
For some inscrutable reason, in the author’s country at least, professional mathe-
maticians seem universally to write z instead of z ∗ , whereas rising engineers take the
mathematicians’ classes and then, having passed them, promptly start writing z ∗ for the
rest of their lives. The writer has his preference between the two notations and this book
The magnitude (or modulus, or absolute value) |z| of the complex number is
defined to be the length ρ in Fig. 2.5, which per the Pythagorean theorem
(§ 2.10) is such that
|z|2 = x2 + y 2 . (2.50)
The phase arg z of the complex number is defined to be the angle φ in the
figure, which in terms of the trigonometric functions of § 3.138 is such that
y
tan(arg z) = . (2.51)
x
Specifically to extract the real and imaginary parts of a complex number,
the notations
ℜ(z) = x,
(2.52)
ℑ(z) = y,
are conventionally recognized (although often the symbols ℜ[·] and ℑ[·] are
written Re[·] and Im[·], particularly when printed by hand).
2.12.1 Multiplication and division of complex numbers in

rectangular form
Several elementary properties of complex numbers are readily seen if the
fact that i2 = −1 is kept in mind, including that
z1 z2 = (x1 x2 − y1 y2 ) + i(y1 x2 + x1 y2 ), (2.53)

z1 x1 + iy1 x2 − iy2 x1 + iy1
= =
z2 x2 + iy2 x2 − iy2 x2 + iy2
(x1 x2 + y1 y2 ) + i(y1 x2 − x1 y2 )
= . (2.54)
x22 + y22
It is a curious fact that

1
= −i. (2.55)
i
It is a useful fact that
z ∗ z = x2 + y 2 = |z|2 (2.56)
(the curious fact, eqn. 2.55, is useful, too).
reflects it, but the curiously absolute character of the notational split is interesting as a
social phenomenon.
38
This is a forward reference. If the equation doesn’t make sense to you yet for this
reason, skip it for now. The important point is that arg z is the angle φ in the figure.
2.12.2 Complex conjugation

An important property of complex numbers descends subtly from the fact
that
i2 = −1 = (−i)2 .
If one defined some number j ≡ −i, claiming that j not i were the true
imaginary unit,39 then one would find that
(−j)2 = −1 = j 2 ,
and thus that all the basic properties of complex numbers in the j system
held just as well as they did in the i system. The units i and j would differ
indeed, but would perfectly mirror one another in every respect.
That is the basic idea. To establish it symbolically needs a page or so of
slightly abstract algebra as follows, the goal of which will be to show that
[f (z)]∗ = f (z ∗ ) for some unspecified function f (z) with specified properties.
To begin with, if
z = x + iy,
then
z ∗ = x − iy
by definition. Proposing that (z k−1 )∗ = (z ∗ )k−1 (which may or may not be
true but for the moment we assume it), we can write
z k−1 = sk−1 + itk−1 ,

(z ∗ )k−1 = sk−1 − itk−1 ,
where sk−1 and tk−1 are symbols introduced to represent the real and imag-
inary parts of z k−1 . Multiplying the former equation by z = x + iy and the
latter by z ∗ = x − iy, we have that
z k = (xsk−1 − ytk−1 ) + i(ysk−1 + xtk−1 ),

(z ∗ )k = (xsk−1 − ytk−1 ) − i(ysk−1 + xtk−1 ).
With the definitions sk ≡ xsk−1 − ytk−1 and tk ≡ ysk−1 + xtk−1 , this is

written more succinctly
z k = sk + itk ,
(z ∗ )k = sk − itk .
39
[19, § I:22-5]
In other words, if (z k−1 )∗ = (z ∗ )k−1 , then it necessarily follows that (z k )∗ =

(z ∗ )k . Solving the definitions of sk and tk for sk−1 and tk−1 yields the reverse
definitions sk−1 = (xsk + ytk )/(x2 + y 2 ) and tk−1 = (−ysk + xtk )/(x2 + y 2 ).
Therefore, except when z = x + iy happens to be null or infinite, the impli-
cation is reversible by reverse reasoning, so by mathematical induction40 we
have that
(z k )∗ = (z ∗ )k (2.57)
for all integral k. We have also from (2.53) that
(z1 z2 )∗ = z1∗ z2∗ (2.58)
for any complex z1 and z2 .

Consequences of (2.57) and (2.58) include that if
∞
X
f (z) ≡ (ak + ibk )(z − zo )k , (2.59)
k=−∞
X∞
f ∗ (z) ≡ (ak − ibk )(z − zo∗ )k , (2.60)
k=−∞
where ak and bk are real and imaginary parts of the coefficients peculiar to
the function f (·), then
[f (z)]∗ = f ∗ (z ∗ ). (2.61)
In the common case where all bk = 0 and zo = xo is a real number, then
f (·) and f ∗ (·) are the same function, so (2.61) reduces to the desired form
[f (z)]∗ = f (z ∗ ), (2.62)
which says that the effect of conjugating the function’s input is merely to
conjugate its output.
Equation (2.62) expresses a significant, general rule of complex numbers
and complex variables which is better explained in words than in mathemat-
ical symbols. The rule is this: for most equations and systems of equations
used to model physical systems, one can produce an equally valid alter-
nate model simply by simultaneously conjugating all the complex quantities
present.41
40
Mathematical induction is an elegant old technique for the construction of mathemat-
ical proofs. Section 8.1 elaborates on the technique and offers a more extensive example.
Beyond the present book, a very good introduction to mathematical induction is found
in [27].
41
[27][57]
2.12.3 Power series and analytic functions (preview)

Equation (2.59) is a general power series42 in z − zo . Such power series have
broad application.43 It happens in practice that most functions of interest
in modeling physical phenomena can conveniently be constructed as power
series (or sums of power series)44 with suitable choices of ak , bk and zo .
The property (2.61) applies to all such functions, with (2.62) also apply-
ing to those for which bk = 0 and zo = xo . The property the two equations
represent is called the conjugation property. Basically, it says that if one
replaces all the i in some mathematical model with −i, then the resulting
conjugate model is equally as valid as the original.45
Such functions, whether bk = 0 and zo = xo or not, are analytic functions
(§ 8.4). In the formal mathematical definition, a function is analytic which
is infinitely differentiable (Ch. 4) in the immediate domain neighborhood of
interest. However, for applications a fair working definition of the analytic
function might be “a function expressible as a power series.” Chapter 8
elaborates. All power series are infinitely differentiable except at their poles.
There nevertheless exist one common group of functions which cannot be
constructed as power series. These all have to do with the parts of complex
numbers and have been introduced in this very section: the magnitude |·|;
the phase arg(·); the conjugate (·)∗ ; and the real and imaginary parts ℜ(·)
and ℑ(·). These functions are not analytic and do not in general obey the
conjugation property. Also not analytic are the Heaviside unit step u(t) and
the Dirac delta δ(t) (§ 7.7), used to model discontinuities explicitly.
We shall have more to say about analytic functions in Ch. 8. We shall
have more to say about complex numbers in §§ 3.11, 4.3.3, and 4.4, and
much more yet in Ch. 5.
42
[31, § 10.8]
43
That is a pretty impressive-sounding statement: “Such power series have broad appli-
cation.” However, air, words and molecules also have “broad application”; merely stating
the fact does not tell us much. In fact the general power series is a sort of one-size-fits-all
mathematical latex glove, which can be stretched to fit around almost any function. The
interesting part is not so much in the general form (2.59) of the series as it is in the specific
choice of ak and bk , which this section does not discuss.
Observe that the Taylor series (which this section also does not discuss; see § 8.3) is a
power series with ak = bk = 0 for k < 0.
44
The careful reader might observe that this statement neglects Gibbs’ phenomenon,
but that curious matter will be dealt with in § 17.6.
45
To illustrate, from the fact that (1 + i2)(2 + i3) + (1 − i) = −3 + i6 the conjugation
property infers immediately that (1 − i2)(2 − i3) + (1 + i) = −3 − i6. Observe however
that no such property holds for the real parts: (−1 + i2)(−2 + i3) + (−1 − i) 6= 3 + i6.
Chapter 3
Trigonometry
Trigonometry is the branch of mathematics which relates angles to lengths.

This chapter introduces the trigonometric functions and derives their several
properties.
3.1 Definitions
Consider the circle-inscribed right triangle of Fig. 3.1.
In considering the circle, we shall find some terminology useful: the
angle φ in the diagram is measured in radians, where a radian is the an-
gle which, when centered in a unit circle, describes an arc of unit length.1
Measured in radians, an angle φ intercepts an arc of curved length ρφ on
a circle of radius ρ (that is, of distance ρ from the circle’s center to its
perimeter). An angle in radians is a dimensionless number, so one need not
write “φ = 2π/4 radians”; it suffices to write “φ = 2π/4.” In mathematical
theory, we express angles in radians.
The angle of full revolution is given the symbol 2π—which thus is the
circumference of a unit circle.2 A quarter revolution, 2π/4, is then the right
angle, or square angle.
The trigonometric functions sin φ and cos φ (the “sine” and “cosine” of φ)
relate the angle φ to the lengths shown in Fig. 3.1. The tangent function is
then defined as
sin φ
tan φ ≡ , (3.1)
cos φ
1
The word “unit” means “one” in this context. A unit length is a length of 1 (not one
centimeter or one mile, just an abstract 1). A unit circle is a circle of radius 1.
2
Section 8.11 computes the numerical value of 2π.
47
48 CHAPTER 3. TRIGONOMETRY
Figure 3.1: The sine and the cosine (shown on a circle-inscribed right trian-
gle, with the circle centered at the triangle’s point).
ρ sin φ
φ ρ cos φ
b x
which is the “rise” per unit “run,” or slope, of the triangle’s diagonal. In-
verses of the three trigonometric functions can also be defined:
arcsin (sin φ) = φ,
arccos (cos φ) = φ,
arctan (tan φ) = φ.
When the last of these is written in the form
y
arctan ,
x
it is normally implied that x and y are to be interpreted as rectangular
coordinates3,4 and that the arctan function is to return φ in the correct
quadrant −π < φ ≤ π (for example, arctan[1/(−1)] = [+3/8][2π], whereas
arctan[(−1)/1] = [−1/8][2π]). This is similarly the usual interpretation
when an equation like
y
tan φ =
x
3
Rectangular coordinates are pairs of numbers (x, y) which uniquely specify points in
a plane. Conventionally, the x coordinate indicates distance eastward; the y coordinate,
northward. For instance, the coordinates (3, −4) mean the point three units eastward and
four units southward (that is, −4 units northward) from the origin (0, 0). A third rectan-
gular coordinate can also be added—(x, y, z)—where the z indicates distance upward.
4
Because the “oo” of “coordinates” is not the monophthongal “oo” of “boot” and
“door,” the old publishing convention this book generally follows should style the word
as “coördinates.” The book uses the word however as a technical term. For better or for
worse, every English-language technical publisher the author knows of styles the technical
term as “coordinates.” The author neither has nor desires a mandate to reform technical
publishing practice, so “coordinates” the word shall be.
3.2. SIMPLE PROPERTIES 49
Figure 3.2: The sine function.
sin(t)
1
t
2π 2π
−
2 2
is written.
By the Pythagorean theorem (§ 2.10), it is seen generally that5
cos2 φ + sin2 φ = 1. (3.2)
Fig. 3.2 plots the sine function. The shape in the plot is called a sinusoid.
3.2 Simple properties

Inspecting Fig. 3.1 and observing (3.1) and (3.2), one readily discovers the
several simple trigonometric properties of Table 3.1.
3.3 Scalars, vectors, and vector notation

In applied mathematics, a vector is an amplitude of some kind coupled with
a direction.6 For example, “55 miles per hour northwestward” is a vector,
as is the entity u depicted in Fig. 3.3. The entity v depicted in Fig. 3.4 is
also a vector, in this case a three-dimensional one.
5
The notation cos2 φ means (cos φ)2 .
6
The same word vector is also used to indicate an ordered set of N scalars (§ 8.16) or
an N × 1 matrix (Ch. 11), but those are not the uses of the word meant here. See also
the introduction to Ch. 15.
Table 3.1: Simple properties of the trigonometric functions.
sin(−φ) = − sin φ cos(−φ) = + cos φ

sin(2π/4 − φ) = + cos φ cos(2π/4 − φ) = + sin φ
sin(2π/2 − φ) = + sin φ cos(2π/2 − φ) = − cos φ
sin(φ ± 2π/4) = ± cos φ cos(φ ± 2π/4) = ∓ sin φ
sin(φ ± 2π/2) = − sin φ cos(φ ± 2π/2) = − cos φ
sin(φ + n2π) = sin φ cos(φ + n2π) = cos φ
tan(−φ) = − tan φ
tan(2π/4 − φ) = +1/ tan φ
tan(2π/2 − φ) = − tan φ
tan(φ ± 2π/4) = −1/ tan φ
tan(φ ± 2π/2) = + tan φ
tan(φ + n2π) = tan φ
sin φ
= tan φ
cos φ
cos2 φ + sin2 φ = 1
1
1 + tan2 φ =
cos2 φ
1 1
1+ 2 =
tan φ sin2 φ
3.3. SCALARS, VECTORS, AND VECTOR NOTATION 51
Figure 3.3: A two-dimensional vector u = x̂x + ŷy, shown with its rectan-
gular components.
ŷy u
x̂x
x
Figure 3.4: A three-dimensional vector v = x̂x + ŷy + ẑz.
v
x
z
Many readers will already find the basic vector concept familiar, but for
those who do not, a brief review: Vectors such as the
u = x̂x + ŷy,
v = x̂x + ŷy + ẑz
of the figures are composed of multiples of the unit basis vectors x̂, ŷ and ẑ,
which themselves are vectors of unit length pointing in the cardinal direc-
tions their respective symbols suggest.7 Any vector a can be factored into
an amplitude a and a unit vector â, as
a = âa,
where the â represents direction only and has unit magnitude by definition,
and where the a represents amplitude only and carries the physical units
if any.8 For example, a = 55 miles per hour, â = northwestward. The
unit vector â itself can be expressed in terms of the unit basis
√ vectors:√for
√ then â =√−x̂(1/ 2)+ ŷ(1/ 2),
example, if x̂ points east and ŷ points north,
where per the Pythagorean theorem (−1/ 2)2 + (1/ 2)2 = 12 .
A single number which is not a vector or a matrix (Ch. 11) is called
a scalar. In the example, a = 55 miles per hour is a scalar. Though the
scalar a in the example happens to be real, scalars can be complex, too—
which might surprise one, since scalars by definition lack direction and the
Argand phase φ of Fig. 2.5 so strongly resembles a direction. However,
phase is not an actual direction in the vector sense (the real number line
7
Printing by hand, one customarily writes a general vector like u as “ ~u ” or just “ u ”,
and a unit vector like x̂ as “ x̂ ”.
8
The word “unit” here is unfortunately overloaded. As an adjective in mathematics,
or in its nounal form “unity,” it refers to the number one (1)—not one mile per hour,
one kilogram, one Japanese yen or anything like that; just an abstract 1. The word
“unit” itself as a noun however usually signifies a physical or financial reference quantity
of measure, like a mile per hour, a kilogram or even a Japanese yen. There is no inherent
mathematical unity to 1 mile per hour (otherwise known as 0.447 meters per second,
among other names). By contrast, a “unitless 1”—a 1 with no physical unit attached—
does represent mathematical unity.
Consider the ratio r = h1 /ho of your height h1 to my height ho . Maybe you are taller
than I am and so r = 1.05 (not 1.05 cm or 1.05 feet, just 1.05). Now consider the ratio
h1 /h1 of your height to your own height. That ratio is of course unity, exactly 1.
There is nothing ephemeral in the concept of mathematical unity, nor in the concept of
unitless quantities in general. The concept is quite straightforward and entirely practical.
That r > 1 means neither more nor less than that you are taller than I am. In applications,
one often puts physical quantities in ratio precisely to strip the physical units from them,
comparing the ratio to unity without regard to physical units.
3.4. ROTATION 53
in the Argand plane cannot be said to run west-to-east, or anything like

that). The x, y and z of Fig. 3.4 are each (possibly complex) scalars; v =
x̂x + ŷy + ẑz is a vector. If x, y and z are complex, then9
|v|2 = |x|2 + |y|2 + |z|2 = x∗ x + y ∗ y + z ∗ z
= [ℜ(x)]2 + [ℑ(x)]2 + [ℜ(y)]2 + [ℑ(y)]2
+ [ℜ(z)]2 + [ℑ(z)]2 . (3.3)
A point is sometimes identified by the vector expressing its distance
and direction from the origin of the coordinate system. That is, the point
(x, y) can be identified with the vector x̂x + ŷy. However, in the general
case vectors are not associated with any particular origin; they represent
distances and directions, not fixed positions.
Notice the relative orientation of the axes in Fig. 3.4. The axes are
oriented such that if you point your flat right hand in the x direction, then
bend your fingers in the y direction and extend your thumb, the thumb then
points in the z direction. This is orientation by the right-hand rule. A left-
handed orientation is equally possible, of course, but as neither orientation
has a natural advantage over the other, we arbitrarily but conventionally
accept the right-handed one as standard.10
Sections 3.4 and 3.9 and Chs. 15 and 16 speak further of the vector.
3.4 Rotation
A fundamental problem in trigonometry arises when a vector
u = x̂x + ŷy (3.4)
must be expressed in terms of alternate unit vectors x̂′ and ŷ′ , where x̂′
and ŷ′ stand at right angles to one another and lie in the plane11 of x̂ and ŷ,
9
Some books print |v| as kvk or even kvk2 to emphasize that it represents the real,
scalar magnitude of a complex vector. The reason the last notation subscripts a numeral 2
is obscure, having to do with the professional mathematician’s generalized definition of a
thing he calls the “norm.” This book just renders it |v|.
10
The writer does not know the etymology for certain, but verbal lore in American
engineering has it that the name “right-handed” comes from experience with a standard
right-handed wood screw or machine screw. If you hold the screwdriver in your right hand
and turn the screw in the natural manner clockwise, turning the screw slot from the x
orientation toward the y, the screw advances away from you in the z direction into the
wood or hole. If somehow you came across a left-handed screw, you’d probably find it
easier to drive that screw with the screwdriver in your left hand.
11
A plane, as the reader on this tier undoubtedly knows, is a flat (but not necessarily
level) surface, infinite in extent unless otherwise specified. Space is three-dimensional. A
Figure 3.5: Vector basis rotation.
u
ŷ ′ ŷ
φ x̂′
φ
x
x̂
but are rotated from the latter by an angle φ as depicted in Fig. 3.5.12 In
terms of the trigonometric functions of § 3.1, evidently
x̂′ = +x̂ cos φ + ŷ sin φ,

(3.5)
ŷ′ = −x̂ sin φ + ŷ cos φ;
and by appeal to symmetry it stands to reason that
x̂ = +x̂′ cos φ − ŷ′ sin φ,

(3.6)
ŷ = +x̂′ sin φ + ŷ′ cos φ.
Substituting (3.6) into (3.4) yields
u = x̂′ (x cos φ + y sin φ) + ŷ′ (−x sin φ + y cos φ), (3.7)
which was to be derived.

Equation (3.7) finds general application where rotations in rectangular
coordinates are involved. If the question is asked, “what happens if I rotate
plane is two-dimensional. A line is one-dimensional. A point is zero-dimensional. The
plane belongs to this geometrical hierarchy.
12
The “ ′ ” mark is pronounced “prime” or “primed” (for no especially good reason of
which the author is aware, but anyway, that’s how it’s pronounced). Mathematical writing
employs the mark for a variety of purposes. Here, the mark merely distinguishes the new
unit vector x̂′ from the old x̂.
3.5. TRIGONOMETRIC SUMS AND DIFFERENCES 55
not the unit basis vectors but rather the vector u instead?” the answer is
that it amounts to the same thing, except that the sense of the rotation is
reversed:
u′ = x̂(x cos φ − y sin φ) + ŷ(x sin φ + y cos φ). (3.8)
Whether it is the basis or the vector which rotates thus depends on your
point of view.13
Much later in the book, § 15.1 will extend rotation in two dimensions to
reorientation in three dimensions.
3.5 Trigonometric functions of sums and differ-

ences of angles
With the results of § 3.4 in hand, we now stand in a position to consider
trigonometric functions of sums and differences of angles. Let
â ≡ x̂ cos α + ŷ sin α,
b̂ ≡ x̂ cos β + ŷ sin β,
be vectors of unit length in the xy plane, respectively at angles α and β
from the x axis. If we wanted b̂ to coincide with â, we would have to rotate
it by φ = α − β. According to (3.8) and the definition of b̂, if we did this
we would obtain
b̂′ = x̂[cos β cos(α − β) − sin β sin(α − β)]
+ ŷ[cos β sin(α − β) + sin β cos(α − β)].
Since we have deliberately chosen the angle of rotation such that b̂′ = â, we
can separately equate the x̂ and ŷ terms in the expressions for â and b̂′ to
obtain the pair of equations
cos α = cos β cos(α − β) − sin β sin(α − β),
sin α = cos β sin(α − β) + sin β cos(α − β).
Solving the last pair simultaneously14 for sin(α − β) and cos(α − β) and
observing that sin2 (·) + cos2 (·) = 1 yields
sin(α − β) = sin α cos β − cos α sin β,
(3.9)
cos(α − β) = cos α cos β + sin α sin β.
13
This is only true, of course, with respect to the vectors themselves. When one actually
rotates a physical body, the body experiences forces during rotation which might or might
not change the body internally in some relevant way.
14
The easy way to do this is
• to subtract sin β times the first equation from cos β times the second, then to solve
With the change of variable β ← −β and the observations from Table 3.1
that sin(−φ) = − sin φ and cos(−φ) = + cos(φ), eqns. (3.9) become
sin(α + β) = sin α cos β + cos α sin β,

(3.10)
cos(α + β) = cos α cos β − sin α sin β.
Equations (3.9) and (3.10) are the basic formulas for trigonometric functions
of sums and differences of angles.
3.5.1 Variations on the sums and differences

Several useful variations on (3.9) and (3.10) are achieved by combining the
equations in various straightforward ways.15 These include
cos(α − β) − cos(α + β)
sin α sin β = ,
2
sin(α − β) + sin(α + β)
sin α cos β = , (3.11)
2
cos(α − β) + cos(α + β)
cos α cos β = .
2
With the change of variables δ ← α − β and γ ← α + β, (3.9) and (3.10)

become

γ+δ γ−δ γ+δ γ−δ
sin δ = sin cos − cos sin ,
2 2 2 2

γ +δ γ−δ γ+δ γ−δ
cos δ = cos cos + sin sin ,
2 2 2 2

γ+δ γ−δ γ+δ γ−δ
sin γ = sin cos + cos sin ,
2 2 2 2

γ +δ γ−δ γ+δ γ−δ
cos γ = cos cos − sin sin .
2 2 2 2
the result for sin(α − β);

• to add cos β times the first equation to sin β times the second, then to solve the
result for cos(α − β).
This shortcut technique for solving a pair of equations simultaneously for a pair of variables
is well worth mastering. In this book alone, it proves useful many times.
15
Refer to footnote 14 above for the technique.
3.6. TRIGONOMETRICS OF THE HOUR ANGLES 57
Combining these in various ways, we have that

γ+δ γ−δ
sin γ + sin δ = 2 sin cos ,
2 2

γ+δ γ−δ
sin γ − sin δ = 2 cos sin ,
2 2
(3.12)
γ+δ γ−δ
cos δ + cos γ = 2 cos cos ,
2 2

γ+δ γ−δ
cos δ − cos γ = 2 sin sin .
2 2
3.5.2 Trigonometric functions of double and half angles

If α = β, then eqns. (3.10) become the double-angle formulas
sin 2α = 2 sin α cos α,

(3.13)
cos 2α = 2 cos2 α − 1 = cos2 α − sin2 α = 1 − 2 sin2 α.
Solving (3.13) for sin2 α and cos2 α yields the half-angle formulas
1 − cos 2α
sin2 α = ,
2 (3.14)
1 + cos 2α
cos2 α = .
2
3.6 Trigonometric functions of the hour angles

In general one uses the Taylor series of Ch. 8 to calculate trigonometric
functions of specific angles. However, for angles which happen to be integral
multiples of an hour —there are twenty-four or 0x18 hours in a circle, just
as there are twenty-four or 0x18 hours in a day16 —for such angles simpler
expressions exist. Figure 3.6 shows the angles. Since such angles arise very
frequently in practice, it seems worth our while to study them specially.
Table 3.2 tabulates the trigonometric functions of these hour angles. To
see how the values in the table are calculated, look at the square and the
16
Hence an hour is 15◦ , but you weren’t going to write your angles in such inelegant
conventional notation as “15◦ ,” were you? Well, if you were, you’re in good company.
The author is fully aware of the barrier the unfamiliar notation poses for most first-time
readers of the book. The barrier is erected neither lightly nor disrespectfully. Consider:
• There are 0x18 hours in a circle.
Figure 3.6: The 0x18 hours in a circle.

3.6. TRIGONOMETRICS OF THE HOUR ANGLES 59
Table 3.2: Trigonometric functions of the hour angles.
ANGLE φ
[radians] [hours] sin φ tan φ cos φ
0 0 0 0 1
√ √ √
2π 3−1 3−1 3+1
1 √ √ √
0x18 2 2 3+1 2 2
√
2π 1 1 3
2 √
0xC 2 3 2
2π 1 1
3 √ 1 √
8 2 2
√
2π 3 √ 1
4 3
6 2 2
√ √ √
(5)(2π) 3+1 3+1 3−1
5 √ √ √
0x18 2 2 3−1 2 2
2π
6 1 ∞ 0
4
Figure 3.7: A square and an equilateral triangle for calculating trigonometric

functions of the hour angles.
√
2
1 1 1
√
3
2
1 1/2 1/2
equilateral triangle17 of Fig. 3.7. Each of the square’s four angles naturally
measures six hours; and since a triangle’s angles always total twelve hours
(§ 2.9.3), by symmetry each of the angles of the equilateral triangle in the
figure measures four. Also by symmetry, the perpendicular splits the trian-
gle’s top angle into equal halves of two hours each and its bottom leg into
equal segments of length 1/2 each; and the diagonal splits the square’s cor-
ner into equal halves of three hours each. The Pythagorean theorem (§ 2.10)
then supplies the various other lengths in the figure, after which we observe
• There are 360 degrees in a circle.

Both sentences say the same thing, don’t they? But even though the “0x” hex prefix is
a bit clumsy, the first sentence nevertheless says the thing rather better. The reader is
urged to invest the attention and effort to master the notation.
There is a psychological trap regarding the hour. The familiar, standard clock face
shows only twelve hours not twenty-four, so the angle between eleven o’clock and twelve
on the clock face is not an hour of arc! That angle is two hours of arc. This is so because
the clock face’s geometry is artificial. If you have ever been to the Old Royal Observatory
at Greenwich, England, you may have seen the big clock face there with all twenty-four
hours on it. It’d be a bit hard to read the time from such a crowded clock face were it not
so big, but anyway, the angle between hours on the Greenwich clock is indeed an honest
hour of arc. [7]
The hex and hour notations are recommended mostly only for theoretical math work.
It is not claimed that they offer much benefit in most technical work of the less theoret-
ical kinds. If you wrote an engineering memo describing a survey angle as 0x1.80 hours
instead of 22.5 degrees, for example, you’d probably not like the reception the memo got.
Nonetheless, the improved notation fits a book of this kind so well that the author hazards
it. It is hoped that after trying the notation a while, the reader will approve the choice.
17
An equilateral triangle is, as the name and the figure suggest, a triangle whose three
sides all have the same length.
3.7. THE LAWS OF SINES AND COSINES 61
Figure 3.8: The laws of sines and cosines.
b α c
y
h
γ β
x
a
from Fig. 3.1 that
• the sine of a non-right angle in a right triangle is the opposite leg’s

length divided by the diagonal’s,
• the tangent is the opposite leg’s length divided by the adjacent leg’s,
and
• the cosine is the adjacent leg’s length divided by the diagonal’s.
With this observation and the lengths in the figure, one can calculate the
sine, tangent and cosine of angles of two, three and four hours.
The values for one and five hours are found by applying (3.9) and (3.10)
against the values for two and three hours just calculated. The values for
zero and six hours are, of course, seen by inspection.18
3.7 The laws of sines and cosines

Refer to the triangle of Fig. 3.8. By the definition of the sine function, one
can write that
c sin β = h = b sin γ,

sin β sin γ
= .
b c
18
The creative reader may notice that he can extend the table to any angle by repeated
application of the various sum, difference and half-angle formulas from the preceding
sections to the values already in the table. However, the Taylor series (§ 8.9) offers a
cleaner, quicker way to calculate trigonometrics of non-hour angles.
But there is nothing special about β and γ; what is true for them must be
true for α, too.19 Hence,
sin α sin β sin γ

= = . (3.15)
a b c
This equation is known as the law of sines.
On the other hand, if one expresses a and b as vectors emanating from
the point γ,20
a = x̂a,
b = x̂b cos γ + ŷb sin γ,
then
c2 = |b − a|2
= (b cos γ − a)2 + (b sin γ)2
= a2 + (b2 )(cos2 γ + sin2 γ) − 2ab cos γ.
Since cos2 (·) + sin2 (·) = 1, this is
c2 = a2 + b2 − 2ab cos γ, (3.16)
known as the law of cosines.
3.8 Summary of properties

Table 3.2 on page 59 has listed the values of trigonometric functions of the
hour angles. Table 3.1 on page 50 has summarized simple properties of the
trigonometric functions. Table 3.3 summarizes further properties, gathering
them from §§ 3.4, 3.5 and 3.7.
19
“But,” it is objected, “there is something special about α. The perpendicular h drops
from it.”
True. However, the h is just a utility variable to help us to manipulate the equation
into the desired form; we’re not interested in h itself. Nothing prevents us from dropping
additional perpendiculars hβ and hγ from the other two corners and using those as utility
variables, too, if we like. We can use any utility variables we want.
20
Here is another example of the book’s judicious relaxation of formal rigor. Of course
there is no “point γ”; γ is an angle not a point. However, the writer suspects in light of
Fig. 3.8 that few readers will be confused as to which point is meant. The skillful applied
mathematician does not multiply labels without need.
3.8. SUMMARY OF PROPERTIES 63
Table 3.3: Further properties of the trigonometric functions.
u = x̂′ (x cos φ + y sin φ) + ŷ′ (−x sin φ + y cos φ)

sin(α ± β) = sin α cos β ± cos α sin β
cos(α ± β) = cos α cos β ∓ sin α sin β
cos(α − β) − cos(α + β)
sin α sin β =
2
sin(α − β) + sin(α + β)
sin α cos β =
2
cos(α − β) + cos(α + β)
cos α cos β =
2
γ +δ γ−δ
sin γ + sin δ = 2 sin cos
2 2

γ+δ γ−δ
sin γ − sin δ = 2 cos sin
2 2

γ+δ γ−δ
cos δ + cos γ = 2 cos cos
2 2

γ +δ γ−δ
cos δ − cos γ = 2 sin sin
2 2
sin 2α = 2 sin α cos α
cos 2α = 2 cos2 α − 1 = cos2 α − sin2 α = 1 − 2 sin2 α
1 − cos 2α
sin2 α =
2
1 + cos 2α
cos2 α =
2
sin γ sin α sin β
= =
c a b
c2 = a2 + b2 − 2ab cos γ
3.9 Cylindrical and spherical coordinates

Section 3.3 has introduced the concept of the vector
v = x̂x + ŷy + ẑz.
The coefficients (x, y, z) on the equation’s right side are coordinates—specif-

ically, rectangular coordinates—which given a specific orthonormal21 set of
unit basis vectors [x̂ ŷ ẑ] uniquely identify a point (see Fig. 3.4 on page 51;
also, much later in the book, refer to § 15.3). Such rectangular coordinates
are simple and general, and are convenient for many purposes. However,
there are at least two broad classes of conceptually simple problems for which
rectangular coordinates tend to be inconvenient: problems in which an axis
or a point dominates. Consider for example an electric wire’s magnetic
field, whose intensity varies with distance from the wire (an axis); or the
illumination a lamp sheds on a printed page of this book, which depends on
the book’s distance from the lamp (a point).
To attack a problem dominated by an axis, the cylindrical coordinates
(ρ; φ, z) can be used instead of the rectangular coordinates (x, y, z). To
attack a problem dominated by a point, the spherical coordinates (r; θ; φ)
can be used.22 Refer to Fig. 3.9. Such coordinates are related to one another
and to the rectangular coordinates by the formulas of Table 3.4.
Cylindrical and spherical coordinates can greatly simplify the analyses of
the kinds of problems they respectively fit, but they come at a price. There
are no constant unit basis vectors to match them. That is,
v = x̂x + ŷy + ẑz 6= ρ̂ρ + φ̂φ + ẑz 6= r̂r + θ̂θ + φ̂φ.
It doesn’t work that way. Nevertheless, variable unit basis vectors are de-
fined:
ρ̂ ≡ +x̂ cos φ + ŷ sin φ,
φ̂ ≡ −x̂ sin φ + ŷ cos φ,
(3.17)
r̂ ≡ +ẑ cos θ + ρ̂ sin θ,
θ̂ ≡ −ẑ sin θ + ρ̂ cos θ;
21
Orthonormal in this context means “of unit length and at right angles to the other
vectors in the set.” [67, “Orthonormality,” 14:19, 7 May 2006]
22
Notice that the φ is conventionally written second in cylindrical (ρ; φ, z) but third in
spherical (r; θ; φ) coordinates. This odd-seeming convention is to maintain proper right-
handed coordinate rotation. (The explanation will seem clearer once Chs. 15 and 16 are
read.)
3.9. CYLINDRICAL AND SPHERICAL COORDINATES 65
Figure 3.9: A point on a sphere, in spherical (r; θ; φ) and cylindrical (ρ; φ, z)

coordinates. (The axis labels bear circumflexes in this figure only to disam-
biguate the ẑ axis from the cylindrical coordinate z.)
ẑ
r
θ z
ŷ
φ
x̂ ρ
Table 3.4: Relations among the rectangular, cylindrical and spherical coor-
dinates.
ρ2 = x2 + y 2
r 2 = ρ2 + z 2 = x2 + y 2 + z 2
ρ
tan θ =
z
y
tan φ =
x
z = r cos θ
ρ = r sin θ
x = ρ cos φ = r sin θ cos φ
y = ρ sin φ = r sin θ sin φ
or, substituting identities from the table,
x̂x + ŷy
ρ̂ = ,
ρ
−x̂y + ŷx
φ̂ = ,
ρ (3.18)
ẑz + ρ̂ρ x̂x + ŷy + ẑz
r̂ = = ,
r r
−ẑρ + ρ̂z
θ̂ = .
r
Such variable unit basis vectors point locally in the directions in which their
respective coordinates advance.
Combining pairs of (3.17)’s equations appropriately, we have also that
x̂ = +ρ̂ cos φ − φ̂ sin φ,

ŷ = +ρ̂ sin φ + φ̂ cos φ,
(3.19)
ẑ = +r̂ cos θ − θ̂ sin θ,
ρ̂ = +r̂ sin θ + θ̂ cos θ.
Convention usually orients ẑ in the direction of a problem’s axis. Occa-

sionally however a problem arises in which it is more convenient to orient x̂
or ŷ in the direction of the problem’s axis (usually because ẑ has already
been established in the direction of some other pertinent axis). Changing
the meanings of known symbols like ρ, θ and φ is usually not a good idea,
but you can use symbols like
(ρx )2 = y 2 + z 2 , (ρy )2 = z 2 + x2 ,
ρx ρy
tan θ x = , tan θ y = , (3.20)
x y
z x
tan φx = , tan φy = ,
y z
instead if needed.23
23
Symbols like ρx are logical but, as far as this writer is aware, not standard. The writer
is not aware of any conventionally established symbols for quantities like these, but § 15.6
at least will use the ρx -style symbology.
3.10. THE COMPLEX TRIANGLE INEQUALITIES 67
3.10 The complex triangle inequalities

If the real, two-dimensional vectors a, b and c represent the three sides of
a triangle such that a + b + c = 0, then per (2.44)
|a| − |b| ≤ |a + b| ≤ |a| + |b| .
These are just the triangle inequalities of § 2.9.2 in vector notation.24 But
if the triangle inequalities hold for real vectors in a plane, then why not
equally for complex scalars? Consider the geometric interpretation of the
Argand plane of Fig. 2.5 on page 42. Evidently,
|z1 | − |z2 | ≤ |z1 + z2 | ≤ |z1 | + |z2 | (3.21)
for any two complex numbers z1 and z2 . Extending the sum inequality, we
have that
X X
zk ≤ |zk | . (3.22)

k k
Naturally, (3.21) and (3.22) hold equally well for real numbers as for com-
plex; one may find the latter formula useful for sums of real numbers, for
example, when some of the numbers summed are positive and others nega-
tive.25 P
P An important consequence of (3.22) is that if |zk | converges, then
zk also converges. Such a consequence is importantP because mathematical
derivations sometimes need the convergence P of z k established, which can
be hard to do P directly. Convergence of |zk |, which per (3.22) implies
convergence of zk , is often easier to establish.
See also (9.15). Equation (3.22) will find use among other places in
§ 8.10.3.
3.11 De Moivre’s theorem

Compare the Argand-plotted complex number of Fig. 2.5 (page 42) against
the vector of Fig. 3.3 (page 51). Although complex numbers are scalars not
vectors, the figures do suggest an analogy between complex phase and vector
direction. With reference to Fig. 2.5 we can write
z = (ρ)(cos φ + i sin φ) = ρ cis φ, (3.23)

24
Reading closely, one might note that § 2.9.2 uses the “<” sign rather than the “≤,”
but that’s all right. See § 1.2.2.
25
Section 13.9 proves the triangle inequalities more generally.
where
cis φ ≡ cos φ + i sin φ. (3.24)
If z = x + iy, then evidently
x = ρ cos φ,
(3.25)
y = ρ sin φ.
Per (2.53),
z1 z2 = (x1 x2 − y1 y2 ) + i(y1 x2 + x1 y2 ).
Applying (3.25) to the equation yields
z1 z2
= (cos φ1 cos φ2 − sin φ1 sin φ2 ) + i(sin φ1 cos φ2 + cos φ1 sin φ2 ).
ρ1 ρ2
But according to (3.10), this is just
z1 z2
= cos(φ1 + φ2 ) + i sin(φ1 + φ2 ),
ρ1 ρ2
or in other words
z1 z2 = ρ1 ρ2 cis(φ1 + φ2 ). (3.26)
Equation (3.26) is an important result. It says that if you want to multiply
complex numbers, it suffices
• to multiply their magnitudes and
• to add their phases.
It follows by parallel reasoning (or by extension) that
z1 ρ1
= cis(φ1 − φ2 ) (3.27)
z2 ρ2
and by extension that
z a = ρa cis aφ. (3.28)
Equations (3.26), (3.27) and (3.28) are known as de Moivre’s theorem.26 ,27
We have not shown yet, but shall in § 5.4, that
cis φ = exp iφ = eiφ ,
where exp(·) is the natural exponential function and e is the natural loga-
rithmic base, both defined in Ch. 5. De Moivre’s theorem is most useful in
this light.
26
Also called de Moivre’s formula. Some authors apply the name of de Moivre directly
only to (3.28), or to some variation thereof; but since the three equations express essentially
the same idea, if you refer to any of them as de Moivre’s theorem then you are unlikely
to be misunderstood.
27
[57][67]
Chapter 4
The derivative
The mathematics of calculus concerns a complementary pair of questions:1
• Given some function f (t), what is the function’s instantaneous rate of

change, or derivative, f ′ (t)?
• Interpreting some function f ′ (t) as an instantaneous rate of change,

what is the corresponding accretion, or integral, f (t)?
This chapter builds toward a basic understanding of the first question.
4.1 Infinitesimals and limits

Calculus systematically treats numbers so large and so small, they lie beyond
the reach of our mundane number system.
1
Although once grasped the concept is relatively simple, to understand this pair of
questions, so briefly stated, is no trivial thing. They are the pair which eluded or con-
founded the most brilliant mathematical minds of the ancient world.
The greatest conceptual hurdle—the stroke of brilliance—probably lies simply in stating
the pair of questions clearly. Sir Isaac Newton and G.W. Leibnitz cleared this hurdle for
us in the seventeenth century, so now at least we know the right pair of questions to ask.
With the pair in hand, the calculus beginner’s first task is quantitatively to understand
the pair’s interrelationship, generality and significance. Such an understanding constitutes
the basic calculus concept.
It cannot be the role of a book like this one to lead the beginner gently toward an
apprehension of the basic calculus concept. Once grasped, the concept is simple and
briefly stated. In this book we necessarily state the concept briefly, then move along.
Many instructional textbooks—[27] is a worthy example—have been written to lead the
beginner gently. Although a sufficiently talented, dedicated beginner could perhaps obtain
the basic calculus concept directly here, he would probably find it quicker and more
pleasant to begin with a book like the one referenced.
69
70 CHAPTER 4. THE DERIVATIVE
4.1.1 The infinitesimal

A number ǫ is an infinitesimal if it is so small that
0 < |ǫ| < a
for all possible mundane positive numbers a.

This is somewhat a difficult concept, so if it is not immediately clear
then let us approach the matter colloquially. Let me propose to you that I
have an infinitesimal.
“How big is your infinitesimal?” you ask.
“Very, very small,” I reply.
“How small?”
“Very small.”
“Smaller than 0x0.01?”
“Smaller than what?”
“Than 2−8 . You said that we should use hexadecimal notation in this
book, remember?”
“Sorry. Yes, right, smaller than 0x0.01.”
“What about 0x0.0001? Is it smaller than that?”
“Much smaller.”
“Smaller than 0x0.0000 0000 0000 0001?”
“Smaller.”
“Smaller than 2−0x1 0000 0000 0000 0000 ?”
“Now that is an impressively small number. Nevertheless, my infinitesi-
mal is smaller still.”
“Zero, then.”
“Oh, no. Bigger than that. My infinitesimal is definitely bigger than
zero.”
This is the idea of the infinitesimal. It is a definite number of a certain
nonzero magnitude, but its smallness conceptually lies beyond the reach of
our mundane number system.
If ǫ is an infinitesimal, then 1/ǫ can be regarded as an infinity: a very
large number much larger than any mundane number one can name.
The principal advantage of using symbols like ǫ rather than 0 for in-
finitesimals is in that it permits us conveniently to compare one infinitesimal
against another, to add them together, to divide them, etc. For instance,
if δ = 3ǫ is another infinitesimal, then the quotient δ/ǫ is not some unfath-
omable 0/0; rather it is δ/ǫ = 3. In physical applications, the infinitesimals
are often not true mathematical infinitesimals but rather relatively very
small quantities such as the mass of a wood screw compared to the mass
4.1. INFINITESIMALS AND LIMITS 71
of a wooden house frame, or the audio power of your voice compared to

that of a jet engine. The additional cost of inviting one more guest to the
wedding may or may not be infinitesimal, depending on your point of view.
The key point is that the infinitesimal quantity be negligible by comparison,
whatever “negligible” means in the context.2
The second-order infinitesimal ǫ2 is so small on the scale of the common,
first-order infinitesimal ǫ that the even latter cannot measure it. The ǫ2 is
an infinitesimal to the infinitesimals. Third- and higher-order infinitesimals
are likewise possible.
The notation u ≪ v, or v ≫ u, indicates that u is much less than v,
typically such that one can regard the quantity u/v to be an infinitesimal.
In fact, one common way to specify that ǫ be infinitesimal is to write that
ǫ ≪ 1.
4.1.2 Limits
The notation limz→zo indicates that z draws as near to zo as it possibly
can. When written limz→zo+ , the implication is that z draws toward zo from
the positive side such that z > zo . Similarly, when written limz→zo− , the
implication is that z draws toward zo from the negative side.
The reason for the notation is to provide a way to handle expressions
like
3z
2z
as z vanishes:
3z 3
lim = .
z→0 2z 2
The symbol “limQ ” is short for “in the limit as Q.”
Notice that lim is not a function like log or sin. It is just a reminder
that a quantity approaches some value, used when saying that the quantity
2
Among scientists and engineers who study wave phenomena, there is an old rule
of thumb that sinusoidal waveforms be discretized not less finely than ten points per
wavelength. In keeping with this book’s adecimal theme (Appendix A) and the concept of
the hour of arc (§ 3.6), we should probably render the rule as twelve points per wavelength
here. In any case, even very roughly speaking, a quantity greater then 1/0xC of the
principal to which it compares probably cannot rightly be regarded as infinitesimal. On the
other hand, a quantity less than 1/0x10000 of the principal is indeed infinitesimal for most
practical purposes (but not all: for example, positions of spacecraft and concentrations of
chemical impurities must sometimes be accounted more precisely). For quantities between
1/0xC and 1/0x10000, it depends on the accuracy one seeks.
equaled the value would be confusing. Consider that to say
lim (z + 2) = 4
z→2−
is just a fancy way of saying that 2 + 2 = 4. The lim notation is convenient

to use sometimes, but it is not magical. Don’t let it confuse you.
4.2 Combinatorics
In its general form, the problem of selecting k specific items out of a set of n
available items belongs to probability theory ([chapter not yet written]). In
its basic form, however, the same problem also applies to the handling of
polynomials or power series. This section treats the problem in its basic
form.3
4.2.1 Combinations and permutations

Consider the following scenario. I have several small wooden blocks of var-
ious shapes and sizes, painted different colors so that you can clearly tell
each block from the others. If I offer you the blocks and you are free to take
all, some or none of them at your option, if you can take whichever blocks
you want, then how many distinct choices of blocks do you have? Answer:
you have 2n choices, because you can accept or reject the first block, then
accept or reject the second, then the third, and so on.
Now, suppose that what you want is exactly k blocks, neither more nor
fewer. Desiring exactly k blocks, you select your favorite block first: there
are n options for this. Then you select your second favorite: for this, there
are n − 1 options (why not n options? because you have already taken one
block from me; I have only n − 1 blocks left). Then you select your third
favorite—for this there are n − 2 options—and so on until you have k blocks.
There are evidently
n
P ≡ n!/(n − k)! (4.1)
k
ordered ways, or permutations, available for you to select exactly k blocks.
However, some of these distinct permutations put exactly the same
combination of blocks in your hand; for instance, the permutations red-
green-blue and green-red-blue constitute the same combination, whereas
red-white-blue is a different combination entirely. For a single combination
3
[27]
4.2. COMBINATORICS 73
of k blocks (red, green, blue), evidently k! permutations are possible (red-

green-blue, red-blue-green, green-red-blue, green-blue-red, blue-red-green,
blue-green-red). Hence dividing the number of permutations (4.1) by k!
yields the number of combinations

n n!/(n − k)!
≡ . (4.2)
k k!
„ «
n
Properties of the number of combinations include that
k

n n
= , (4.3)
n−k k
n
X n
= 2n , (4.4)
k
k=0

n−1 n−1 n
+ = , (4.5)
k−1 k k

n n−k+1 n
= (4.6)
k k k−1

k+1 n
= (4.7)
n−k k+1

n n−1
= (4.8)
k k−1

n n−1
= . (4.9)
n−k k
Equation (4.3) results from changing the variable k ← n − k in (4.2). Equa-
tion (4.4) comes directly from the observation (made at the head of this
section) that 2n total combinations are possible if any k is allowed. Equa-
tion (4.5) is seen when an nth block—let us say that it is a black block—is
added to an existing set of n − 1 blocks; to choose k blocks then, you can
either choose k from the original set, or the black block plus k − 1 from the
original set. Equations (4.6) through (4.9) come directly from the defini-
tion (4.2); they relate combinatoric coefficients to their neighbors in Pascal’s
triangle (§ 4.2.2).
Because one can choose neither fewer than zero nor more than n from n
blocks,
n
= 0 unless 0 ≤ k ≤ n. (4.10)
k
„ «
n
For k
when n < 0, there is no obvious definition.
Figure 4.1: The plan for Pascal’s triangle.
„ «
0
„ 0
«„ «
1 1
„ 0
«„ 1
«„ «
2 2 2
„ 0
«„ 1
«„ 2
«„ «
3 3 3 3
„ 0
«„ 1
«„ 2
«„ 3
«„ «
4 4 4 4 4
0 1 2 3 4
..
.
4.2.2 Pascal’s triangle

„ «
Consider the triangular layout in Fig. 4.1 of the various possible nk .
Evaluated, this yields Fig. 4.2, Pascal’s triangle. Notice how each entry
in the triangle is the sum of the two entries immediately above, as (4.5)
predicts. (In fact this is the easy way to fill Pascal’s triangle out: for each
entry, just add the two entries above.)
4.3 The binomial theorem

This section presents the binomial theorem and one of its significant conse-
quences.
4.3.1 Expanding the binomial

The binomial theorem holds that4
n
X
n n
(a + b) = an−k bk . (4.11)
k
k=0
4
The author is given to understand that, by an heroic derivational effort, (4.11) can be
extended to nonintegral n. However, since applied mathematics does not usually concern
itself with hard theorems of little known practical use, the extension is not covered in this
book. The Taylor series for (1 + z)a−1 for complex z and complex a is indeed covered
notwithstanding, in Table 8.1, and this amounts to much the same thing.
4.3. THE BINOMIAL THEOREM 75
Figure 4.2: Pascal’s triangle.
1
1 1
1 2 1
1 3 3 1
1 4 6 4 1
1 5 A A 5 1
1 6 F 14 F 6 1
1 7 15 23 23 15 7 1
..
.
In the common case that a = 1, b = ǫ, |ǫ| ≪ 1, this is

n
n
X n
(1 + ǫ) = ǫk (4.12)
k
k=0
(actually this holds for any ǫ, small or large; but the typical case of interest
has |ǫ| ≪ 1). In either form, the binomial theorem is a direct consequence
of the combinatorics of § 4.2. Since
(a + b)n = (a + b)(a + b) · · · (a + b)(a + b),
each (a+b) factor corresponds to one of the “wooden blocks,” where a means
rejecting the block and b, accepting it.
4.3.2 Powers of numbers near unity

„ « „ «
n n
Since 0
= 1 and 1
= n, it follows from (4.12) for
(m, n) ∈ Z, m > 0, n ≥ 0, |δ| ≪ 1, |ǫ| ≪ 1, |ǫo | ≪ 1,
that5
1 + mǫo ≈ (1 + ǫo )m
5
The symbol ≈ means “approximately equals.”
to excellent precision. Furthermore, raising the equation to the 1/m power

then changing δ ← mǫo , we have that
δ
(1 + δ)1/m ≈ 1 + .
m
Changing 1 + δ ← (1 + ǫ)n and observing from the (1 + ǫo )m equation above
that this implies that δ ≈ nǫ, we have that
n
(1 + ǫ)n/m ≈ 1 + ǫ.
m
Inverting this equation yields
1 [1 − (n/m)ǫ] n
(1 + ǫ)−n/m ≈ = ≈ 1 − ǫ.
1 + (n/m)ǫ [1 − (n/m)ǫ][1 + (n/m)ǫ] m
Taken together, the last two equations imply that
(1 + ǫ)x ≈ 1 + xǫ (4.13)
for any real x.

The writer knows of no conventional name6 for (4.13), but named or
unnamed it is an important equation. The equation offers a simple, accurate
way of approximating any real power of numbers in the near neighborhood
of 1.
4.3.3 Complex powers of numbers near unity

Equation (4.13) is fine as far as it goes, but its very form suggests the
question: what if ǫ or x, or both, are complex? Changing the symbol z ← x
and observing that the infinitesimal ǫ may also be complex, one wants to
know whether
(1 + ǫ)z ≈ 1 + zǫ (4.14)
still holds. No work we have yet done in the book answers the question,
because although a complex infinitesimal ǫ poses no particular problem, the
action of a complex power z remains undefined. Still, for consistency’s sake,
one would like (4.14) to hold. In fact nothing prevents us from defining the
action of a complex power such that (4.14) does hold, which we now do,
logically extending the known result (4.13) into the new domain.
6
Actually, “the first-order Taylor expansion” is a conventional name for it, but so
unwieldy a name does not fit the present context. Ch. 8 will introduce the Taylor expansion
as such.
4.4. THE DERIVATIVE 77
Section 5.4 will investigate the extremely interesting effects which arise
when ℜ(ǫ) = 0 and the power z in (4.14) grows large, but for the moment
we shall use the equation in a more ordinary manner to develop the concept
and basic application of the derivative, as follows.
4.4 The derivative

Having laid down (4.14), we now stand in a position properly to introduce
the chapter’s subject, the derivative. What is the derivative? The derivative
is the instantaneous rate or slope of a function. In mathematical symbols
and for the moment using real numbers,
f (t + ǫ/2) − f (t − ǫ/2)
f ′ (t) ≡ lim . (4.15)
ǫ→0+ ǫ
Alternately, one can define the same derivative in the unbalanced form
f (t + ǫ) − f (t)
f ′ (t) = lim ,
ǫ→0+ ǫ
but this book generally prefers the more elegant balanced form (4.15), which
we will now use in developing the derivative’s several properties through the
rest of the chapter.7
4.4.1 The derivative of the power series

In the very common case that f (t) is the power series
∞
X
f (t) = ck tk , (4.16)
k=−∞
where the ck are in general complex coefficients, (4.15) says that

∞
X (ck )(t + ǫ/2)k − (ck )(t − ǫ/2)k
f ′ (t) = lim
ǫ→0+ ǫ
k=−∞
∞
X (1 + ǫ/2t)k − (1 − ǫ/2t)k
= lim ck tk .
ǫ→0+ ǫ
k=−∞
7
From this section through § 4.7, the mathematical notation grows a little thick. There
is no helping this. The reader is advised to tread through these sections line by stubborn
line, in the good trust that the math thus gained will prove both interesting and useful.
Applying (4.14), this is

∞
′
X (1 + kǫ/2t) − (1 − kǫ/2t)
f (t) = lim ck tk ,
ǫ→0+ ǫ
k=−∞
which simplifies to
∞
X
f ′ (t) = ck ktk−1 . (4.17)
k=−∞
Equation (4.17) gives the general derivative of the power series.8
4.4.2 The Leibnitz notation

The f ′ (t) notation used above for the derivative is due to Sir Isaac New-
ton, and is easier to start with. Usually better on the whole, however, is
G.W. Leibnitz’s notation9
dt = ǫ,
df = f (t + dt/2) − f (t − dt/2),
such that per (4.15),

df
f ′ (t) =
. (4.18)
dt
Here dt is the infinitesimal, and df is a dependent infinitesimal whose size
relative to dt depends on the independent variable t. For the independent
infinitesimal dt, conceptually, one can choose any infinitesimal size ǫ. Usu-
ally the exact choice of size does not matter, but occasionally when there
are two independent variables it helps the analysis to adjust the size of one
of the independent infinitesimals with respect to the other.
The meaning of the symbol d unfortunately depends on the context.
In (4.18), the meaning is clear enough: d(·) signifies how much (·) changes
8
Equation (4.17) admittedly has not explicitly considered what happens when the real t
becomes the complex z, but § 4.4.3 will remedy the oversight.
9
This subsection is likely to confuse many readers the first time they read it. The reason
is that Leibnitz elements like dt and ∂f usually tend to appear in practice in certain specific
relations to one another, like ∂f /∂z. As a result, many users of applied mathematics
have never developed a clear understanding as to precisely what the individual symbols
mean. Often they have developed positive misunderstandings. Because there is significant
practical benefit in learning how to handle the Leibnitz notation correctly—particularly in
applied complex variable theory—this subsection seeks to present each Leibnitz element
in its correct light.
when the independent variable t increments by dt.10 Notice, however, that

the notation dt itself has two distinct meanings:11
• the independent infinitesimal dt = ǫ; and
10
If you do not fully understand this sentence, reread it carefully with reference to (4.15)
and (4.18) until you do; it’s important.
11
This is difficult, yet the author can think of no clearer, more concise way to state it.
The quantities dt and df represent coordinated infinitesimal changes in t and f respectively,
so there is usually no trouble with treating dt and df as though they were the same kind
of thing. However, at the fundamental level they really aren’t.
If t is an independent variable, then dt is just an infinitesimal of some kind, whose
specific size could be a function of t but more likely is just a constant. If a constant,
then dt does not fundamentally have anything to do with t as such. In fact, if s and t
are both independent variables, then we can (and in complex analysis sometimes do) say
that ds = dt = ǫ, after which nothing prevents us from using the symbols ds and dt
interchangeably. Maybe it would be clearer in some cases to write ǫ instead of dt, but the
latter is how it is conventionally written.
By contrast, if f is a dependent variable, then df or d(f ) is the amount by which f
changes as t changes by dt. The df is infinitesimal but not constant; it is a function of t.
Maybe it would be clearer in some cases to write dt f instead of df , but for most cases the
former notation is unnecessarily cluttered; the latter is how it is conventionally written.
Now, most of the time, P what we are interested
R in is not dt or df as such, but rather the
ratio df /dt or the sum k f (k dt) dt = f (t) dt. For this reason, we do not usually worry
about which of df and dt is the independent infinitesimal, nor do we usually worry about
the precise value of dt. This leads one to forget that dt does indeed have a precise value.
What confuses is when one changes perspective in mid-analysis, now regarding f as the
independent variable. Changing perspective is allowed and perfectly proper, but one must
take care: the dt and df after the change are not the same as the dt and df before the
change. However, the ratio df /dt remains the same in any case.
Sometimes when writing a differential equation like the potential-kinetic energy equation
ma dx = mv dv, we do not necessarily have either v or x in mind as the independent
variable. This is fine. The important point is that dv and dx be coordinated so that the
ratio dv/dx has a definite value no matter which of the two be regarded as independent,
or whether the independent be some third variable (like t) not in the equation.
One can avoid the confusion simply by keeping the dv/dx or df /dt always in ratio, never
treating the infinitesimals individually. Many applied mathematicians do precisely that.
That is okay as far as it goes, but it really denies the entire point of the Leibnitz notation.
One might as well just stay with the Newton notation in that case. Instead, this writer
recommends that you learn the Leibnitz notation properly, developing the ability to treat
the infinitesimals individually.
Because the book is a book of applied mathematics, this footnote does not attempt to
say everything there is to say about infinitesimals. For instance, it has not yet pointed
out (but does so now) that even if s and t are equally independent variables, one can have
dt = ǫ(t), ds = δ(s, t), such that dt has prior independence to ds. The point is not to
fathom all the possible implications from the start; you can do that as the need arises.
The point is to develop a clear picture in your mind of what a Leibnitz infinitesimal really
is. Once you have the picture, you can go from there.
• d(t), which is how much (t) changes as t increments by dt.
At first glance, the distinction between dt and d(t) seems a distinction with-
out a difference; and for most practical cases of interest, so indeed it is.
However, when switching perspective in mid-analysis as to which variables
are dependent and which are independent, or when changing multiple inde-
pendent complex variables simultaneously, the math can get a little tricky.
In such cases, it may be wise to use the symbol dt to mean d(t) only, in-
troducing some unambiguous symbol like ǫ to represent the independent
infinitesimal. In any case you should appreciate the conceptual difference
between dt = ǫ and d(t).
Where two or more independent variables are at work in the same equa-
tion, it is conventional to use the symbol ∂ instead of d, as a reminder that
the reader needs to pay attention to which ∂ tracks which independent vari-
able.12 A derivative ∂f /∂t or ∂f /∂s in this case is sometimes called by the
slightly misleading name of partial derivative. (If needed or desired, one
can write ∂t (·) when tracking t, ∂s (·) when tracking s, etc. Use discretion,
though. Such notation appears only rarely in the literature, so your audi-
ence might not understand it when you write it.) Conventional shorthand
for d(df ) is d2 f ; for (dt)2 , dt2 ; so
d(df /dt) d2 f
= 2
dt dt
is a derivative of a derivative, or second derivative. By extension, the nota-
tion
dk f
dtk
represents the kth derivative.
4.4.3 The derivative of a function of a complex variable

For (4.15) to be robust, written here in the slightly more general form
df f (z + ǫ/2) − f (z − ǫ/2)
= lim , (4.19)
dz ǫ→0 ǫ
one should like it to evaluate the same in the limit regardless of the complex
phase of ǫ. That is, if δ is a positive real infinitesimal, then it should be
equally valid to let ǫ = δ, ǫ = −δ, ǫ = iδ, ǫ = −iδ, ǫ = (4 − i3)δ or
12
The writer confesses that he remains unsure why this minor distinction merits the
separate symbol ∂, but he accepts the notation as conventional nevertheless.
any other infinitesimal value, so long as 0 < |ǫ| ≪ 1. One should like the
derivative (4.19) to come out the same regardless of the Argand direction
from which ǫ approaches 0 (see Fig. 2.5). In fact for the sake of robustness,
one normally demands that derivatives do come out the same regardless
of the Argand direction; and (4.19) rather than (4.15) is the definition we
normally use for the derivative for this reason. Where the limit (4.19) is
sensitive to the Argand direction or complex phase of ǫ, there we normally
say that the derivative does not exist.
Where the derivative (4.19) does exist—where the derivative is finite
and insensitive to Argand direction—there we say that the function f (z) is
differentiable.
Excepting the nonanalytic parts of complex numbers (|·|, arg[·], [·]∗ , ℜ[·]
and ℑ[·]; see § 2.12.3), plus the Heaviside unit step u(t) and the Dirac
delta δ(t) (§ 7.7), most functions encountered in applications do meet the
criterion (4.19) except at isolated nonanalytic points (like z = 0 in h[z] = 1/z
√
or g[z] = z). Meeting the criterion, such functions are fully differentiable
except at their poles (where the derivative goes infinite in any case) and
other nonanalytic points. Particularly, the key formula (4.14), written here
as
(1 + ǫ)w ≈ 1 + wǫ,
works without modification when ǫ is complex; so the derivative (4.17) of
the general power series,
∞ ∞
d X k
X
ck z = ck kz k−1 (4.20)
dz
k=−∞ k=−∞
holds equally well for complex z as for real.
4.4.4 The derivative of z a

Inspection of § 4.4.1’s logic in light of (4.14) reveals that nothing prevents us
from replacing the real t, real ǫ and integral k of that section with arbitrary
complex z, ǫ and a. That is,
d(z a ) (z + ǫ/2)a − (z − ǫ/2)a

= lim
dz ǫ→0 ǫ
(1 + ǫ/2z) a − (1 − ǫ/2z)a
= lim z a
ǫ→0 ǫ
a (1 + aǫ/2z) − (1 − aǫ/2z)
= lim z ,
ǫ→0 ǫ
which simplifies to
d(z a )
= az a−1 (4.21)
dz
for any complex z and a.
How exactly to evaluate z a or z a−1 when a is complex is another matter,
treated in § 5.4 and its (5.12); but in any case you can use (4.21) for real a
right now.
4.4.5 The logarithmic derivative

Sometimes one is more interested in knowing the rate of f (t) relative to
the value of f (t) than in knowing the absolute rate itself. For example, if
you inform me that you earn $ 1000 a year on a bond you hold, then I may
commend you vaguely for your thrift but otherwise the information does not
tell me much. However, if you inform me instead that you earn 10 percent
a year on the same bond, then I might want to invest. The latter figure is
a relative rate, or logarithmic derivative,
df /dt d
= ln f (t). (4.22)
f (t) dt
The investment principal grows at the absolute rate df /dt, but the bond’s
interest rate is (df /dt)/f (t).
The natural logarithmic notation ln f (t) may not mean much to you yet,
as we’ll not introduce it formally until § 5.2, so you can ignore the right side
of (4.22) for the moment; but the equation’s left side at least should make
sense to you. It expresses the significant concept of a relative rate, like 10
percent annual interest on a bond.
4.5 Basic manipulation of the derivative

This section introduces the derivative chain and product rules.
4.5.1 The derivative chain rule

If f is a function of w, which itself is a function of z, then13

df df dw
= . (4.23)
dz dw dz
13
For example, one can rewrite
p
f (z) = 3z 2 − 1
4.5. BASIC MANIPULATION OF THE DERIVATIVE 83
Equation (4.23) is the derivative chain rule.14
4.5.2 The derivative product rule

In general per (4.19),
 
Y Y dz
Y
dz

d  fj (z) =
 fj z + − fj z − .
2 2
j j j
But to first order,

dz dfj dz dfj
fj z ± ≈ fj (z) ± = fj (z) ± ;
2 dz 2 2
so, in the limit,

 
Y Y df j
Y
df j

d  fj (z) = fj (z) + − fj (z) − .
2 2
j j j
Since the product of two or more dfj is negligible compared to the first-order
infinitesimals to which they are added here, this simplifies to
   " #  " #
Y Y X dfk Y X −dfk
d  fj (z) =  fj (z) −  fj (z) ,
2fk (z) 2fk (z)
j j k j k
in the form
f (w) = w1/2 ,
w(z) = 3z 2 − 1.
Then
df 1 1
= = √ ,
dw 2w1/2 2 3z 2 − 1
dw
= 6z,
dz
so by (4.23), „ «„ «
df df dw 6z 3z
= = √ = √ .
dz dw dz 2 3z 2 − 1 3z 2 − 1
14
It bears emphasizing to readers who may inadvertently have picked up unhelpful ideas
about the Leibnitz notation in the past: the dw factor in the denominator cancels the dw
factor in the numerator; a thing divided by itself is 1. That’s it. There is nothing more
to the proof of the derivative chain rule than that.
or in other words  " #

Y Y X dfk
d fj =  fj  . (4.24)
fk
j j k
In the common case of only two fj , this comes to

d(f1 f2 ) = f2 df1 + f1 df2 . (4.25)
On the other hand, if f1 (z) = f (z) and f2 (z) = 1/g(z), then by the derivative
chain rule (4.23), df2 = −dg/g2 ; so,

f g df − f dg
d = . (4.26)
g g2
Equation (4.24) is the derivative product rule.
After studying the complex exponential in Ch. 5, we shall stand in a
position to write (4.24) in the slightly specialized but often useful form15
 
Y aj Y Y
d  gj ebj hj ln cj pj 
j j j
 
a
Y Y Y
=  gj j ebj hj ln cj pj 
j j j
" #
X dgk X X dpk
× ak + bk dhk + . (4.27)
gk pk ln ck pk
k k k
where the ak , bk and ck are arbitrary complex coefficients and the gk , hk

and pk are arbitrary functions.16
4.5.3 A derivative product pattern

According to (4.25) and (4.21), the derivative of the product z a f (z) with
respect to its independent variable z is
d a df
[z f (z)] = z a + az a−1 f (z).
dz dz
15
This paragraph is extra. You can skip it for now if you prefer.
16
The subsection is sufficiently abstract that it is a little hard to understand unless one
already knows what it means. An example may help:
» 2 3 – » 2 3 –» –
u v −5t u v −5t du dv dz ds
d e ln 7s = e ln 7s 2 +3 − − 5 dt + .
z z u v z s ln 7s
4.6. EXTREMA AND HIGHER DERIVATIVES 85
Figure 4.3: A local extremum.
f (xo ) b
f (x)
x
xo
Swapping the equation’s left and right sides then dividing through by z a
yields
df f d(z a f )
+a = a , (4.28)
dz z z dz
a pattern worth committing to memory, emerging among other places in
§ 16.9.
4.6 Extrema and higher derivatives

One problem which arises very frequently in applied mathematics is the
problem of finding a local extremum—that is, a local minimum or max-
imum—of a real-valued function f (x). Refer to Fig. 4.3. The almost dis-
tinctive characteristic of the extremum f (xo ) is that17

df
= 0. (4.29)
dx x=xo
At the extremum, the slope is zero. The curve momentarily runs level there.
One solves (4.29) to find the extremum.
Whether the extremum be a minimum or a maximum depends on wheth-
er the curve turn from a downward slope to an upward, or from an upward
17
The notation P |Q means “P when Q,” “P , given Q,” or “P evaluated at Q.” Some-
times it is alternately written P |Q or [P ]Q .
Figure 4.4: A level inflection.
f (xo ) b
f (x)
x
xo
slope to a downward, respectively. If from downward to upward, then the

derivative of the slope is evidently positive; if from upward to downward,
then negative. But the derivative of the slope is just the derivative of the
derivative, or second derivative. Hence if df /dx = 0 at x = xo , then
d2 f

> 0 implies a local minimum at xo ;
dx2 x=xo
d2 f

< 0 implies a local maximum at xo .
dx2 x=xo
Regarding the case

d2 f

= 0,
dx2 x=xo
this might be either a minimum or a maximum but more probably is neither,
being rather a level inflection point as depicted in Fig. 4.4.18 (In general
the term inflection point signifies a point at which the second derivative is
zero. The inflection point of Fig. 4.4 is level because its first derivative is
zero, too.)
18
Of course if the first and second derivatives are zero not just at x = xo but everywhere,
then f (x) = yo is just a level straight line, but you knew that already. Whether one chooses
to call some random point on a level straight line an inflection point or an extremum, or
both or neither, would be a matter of definition, best established not by prescription but
rather by the needs of the model at hand.
4.7. L’HÔPITAL’S RULE 87
4.7 L’Hôpital’s rule

If z = zo is a root of both f (z) and g(z), or alternately if z = zo is a pole of
both functions—that is, if both functions go to zero or infinity together at
z = zo —then l’Hôpital’s rule holds that

f (z) df /dz
lim = . (4.30)
z→zo g(z) dg/dz z=zo
In the case where z = zo is a root, l’Hôpital’s rule is proved by reasoning19
f (z) f (z) − 0
lim = lim
z→zo g(z) z→z o g(z) − 0
f (z) − f (zo ) df df /dz

= lim = lim = lim .
z→zo g(z) − g(zo ) z→zo dg z→zo dg/dz
In the case where z = zo is a pole, new functions F (z) ≡ 1/f (z) and
G(z) ≡ 1/g(z) of which z = zo is a root are defined, with which
f (z) G(z) dG −dg/g2

lim = lim = lim = lim ,
z→zo g(z) z→zo F (z) z→zo dF z→zo −df /f 2
where we have used the fact from (4.21) that d(1/u) = −du/u2 for any u.
Canceling the minus signs and multiplying by g2 /f 2 , we have that
g(z) dg
lim = lim .
z→zo f (z) z→zo df
Inverting,
f (z) df df /dz
lim = lim = lim .
z→zo g(z) z→zo dg z→zo dg/dz
And if zo itself is infinite? Then, whether it represents a root or a pole, we
define the new variable Z = 1/z and the new functions Φ(Z) = f (1/Z) =
f (z) and Γ(Z) = g(1/Z) = g(z), with which we apply l’Hôpital’s rule for
Z → 0 to obtain
f (z) Φ(Z) dΦ/dZ df /dZ
lim = lim = lim = lim
z→∞ g(z) Z→0 Γ(Z) Z→0 dΓ/dZ Z→0 dg/dZ
(df /dz)(dz/dZ) (df /dz)(−z 2 ) df /dz
= lim = lim 2
= lim .
z→∞, (dg/dz)(dz/dZ) z→∞ (dg/dz)(−z ) z→∞ dg/dz
Z→0
19
Partly with reference to [67, “L’Hopital’s rule,” 03:40, 5 April 2006].
Nothing in the derivation requires that z or zo be real. Nothing prevents

one from applying l’Hôpital’s rule recursively, should the occasion arise.20
L’Hôpital’s rule is used in evaluating indeterminate forms of the kinds
0/0 and ∞/∞, plus related forms like (0)(∞) which can be recast into either
of the two main forms. Good examples of the use require math from Ch. 5
and later, but if we may borrow from (5.7) the natural logarithmic function
and its derivative,21
d 1
ln x = ,
dx x
then a typical l’Hôpital example is 22
ln x 1/x 2
lim √ = lim √ = lim √ = 0.
x→∞ x x→∞ 1/2 x x→∞ x
The example incidentally shows that natural logarithms grow slower than
square roots, an instance of a more general principle we shall meet in § 5.3.
Section 5.3 will put l’Hôpital’s rule to work.
4.8 The Newton-Raphson iteration

The Newton-Raphson iteration is a powerful, fast converging, broadly appli-
cable method for finding roots numerically. Given a function f (z) of which
the root is desired, the Newton-Raphson iteration is

f (z)
zk+1 = z − d . (4.31)
dz f (z)

z=zk
One begins the iteration by guessing the root and calling the guess z0 .
Then z1 , z2 , z3 , etc., calculated in turn by the iteration (4.31), give suc-
cessively better estimates of the true root z∞ .
20
Consider for example the ratio limx→0 (x3 + x)2 /x2 , which is 0/0. The easier way to
resolve this particular ratio would naturally be to cancel a factor of x2 from it; but just to
make the point let us apply l’Hôpital’s rule instead, reducing the ratio to limx→0 2(x3 +
x)(3x2 + 1)/2x, which is still 0/0. Applying l’Hôpital’s rule again to the result yields
limx→0 2[(3x2 +1)2 +(x3 +x)(6x)]/2 = 2/2 = 1. Where expressions involving trigonometric
or special functions (Chs. 3, 5 and [not yet written]) appear in ratio, a recursive application
of l’Hôpital’s rule can be just the thing one needs.
Observe that one must stop applying l’Hôpital’s rule once the ratio is no longer 0/0 or
∞/∞. In the example, applying the rule a third time would have ruined the result.
21
This paragraph is optional reading for the moment. You can read Ch. 5 first, then
come back here and read the paragraph if you prefer.
22
[55, § 10-2]
4.8. THE NEWTON-RAPHSON ITERATION 89
Figure 4.5: The Newton-Raphson iteration.
x
xk xk+1
f (x)
To understand the Newton-Raphson iteration, consider the function y =

f (x) of Fig 4.5. The iteration approximates the curve f (x) by its tangent
line23 (shown as the dashed line in the figure):

˜ d
fk (x) = f (xk ) + f (x) (x − xk ).
dx x=xk
It then approximates the root xk+1 as the point at which f˜k (xk+1 ) = 0:

˜ d
fk (xk+1 ) = 0 = f (xk ) + f (x) (xk+1 − xk ).
dx x=xk
Solving for xk+1 , we have that

f (x)
xk+1 = x− d ,
dx f (x)

x=xk
which is (4.31) with x ← z.

23
A tangent line, also just called a tangent, is the line which most nearly approximates
a curve at a given point. The tangent touches the curve at the point, and in the neighbor-
hood of the point it goes in the same direction the curve goes. The dashed line in Fig. 4.5
is a good example of a tangent line.
The relationship between the tangent line and the trigonometric tangent function of
Ch. 3 is slightly obscure, maybe more of linguistic interest than of mathematical. The
trigonometric tangent function is named from a variation on Fig. 3.1 in which the triangle’s
bottom leg is extended to unit length, leaving the rightward leg tangent to the circle.
Although the illustration uses real numbers, nothing forbids complex z

and f (z). The Newton-Raphson iteration works just as well for these.
The principal limitation of the Newton-Raphson arises when the function
has more than one root, as most interesting functions do. The iteration often
converges on the root nearest the initial guess zo but does not always, and in
any case there is no guarantee that the root it finds is the one you wanted.
The most straightforward way to beat this problem is to find all the roots:
first you find some root α, then you remove that root (without affecting any
of the other roots) by dividing f (z)/(z − α), then you find the next root by
iterating on the new function f (z)/(z − α), and so on until you have found
all the roots. If this procedure is not practical (perhaps because the function
has a large or infinite number of roots), then you should probably take care
to make a sufficiently accurate initial guess if you can.
A second limitation of the Newton-Raphson is that, if you happen to
guess z0 especially unfortunately, then the iteration might√ never converge at
all. For example, the roots of f (z) = z 2 + 2 are z = ±i 2, but if you guess
that z0 = 1 then the iteration has no way to leave √ the real number line, so
24
it never converges (and if you guess that z0 = 2—well, try it with your
pencil and see what z2 comes out to be). You can fix the problem with a
different, possibly complex initial guess.
A third limitation arises where there is a multiple root. In this case,
the Newton-Raphson normally still converges, but relatively slowly. For
instance, the Newton-Raphson converges relatively slowly on the triple root
of f (z) = z 3 . However, even the relatively slow convergence is still pretty
fast and is usually adequate, even for calculations by hand.
Usually in practice, the Newton-Raphson iteration works very well. For
most functions, once the Newton-Raphson finds the root’s neighborhood, it
converges on the actual root remarkably quickly. Figure 4.5 shows why: in
the neighborhood, the curve hardly departs from the straight line.
The Newton-Raphson iteration is a champion square root calculator,
incidentally. Consider
f (x) = x2 − p,
whose roots are
√
x = ± p.
Per (4.31), the Newton-Raphson iteration for this is

1 p
xk+1 = xk + . (4.32)
2 xk
24
It is entertaining to try this on a computer. Then try again with z0 = 1 + i2−0x10 .
4.8. THE NEWTON-RAPHSON ITERATION 91
If you start by guessing

x0 = 1
√
and iterate several times, the iteration (4.32) converges on x∞ = p fast.
To calculate the nth root x = p1/n , let
f (x) = xn − p
and iterate25,26 " #

1 p
xk+1 = (n − 1)xk + n−1 . (4.33)
n xk
Section 13.7 generalizes the Newton-Raphson iteration to handle vector-
valued functions.
This concludes the chapter. Chapter 8, treating the Taylor series, will
continue the general discussion of the derivative.
25
Equations (4.32) and (4.33) work not only for real p but also usually for complex.
Given x0 = 1, however, they converge reliably and orderly only for real, nonnegative p.
(To see why, sketch f [x] in the fashion of Fig. 4.5.)
If reliable, orderly convergence is needed for complex p = u + iv = σ cis ψ, σ ≥ 0, you
can decompose p1/n per de Moivre’s theorem (3.28) as p1/n = σ 1/n cis(ψ/n), in which
cis(ψ/n) = cos(ψ/n) + i sin(ψ/n) is calculated by the Taylor series of Table 8.1. Then σ
is real and nonnegative, upon which (4.33) reliably, orderly computes σ 1/n .
The Newton-Raphson iteration however excels as a practical root-finding technique, so
it often pays to be a little less theoretically rigid in applying it. If so, then don’t bother to
decompose; seek p1/n directly, using complex zk in place of the real xk . In the uncommon
event that the direct iteration does not seem to converge, start over again with some
randomly chosen complex z0 . This saves effort and usually works.
26
[55, § 4-9][45, § 6.1.1][66]
Chapter 5
The complex exponential
In higher mathematics, the complex natural exponential is almost impossible

to avoid. It seems to appear just about everywhere. This chapter introduces
the concept of the natural exponential and of its inverse, the natural log-
arithm; and shows how the two operate on complex arguments. It derives
the functions’ basic properties and explains their close relationship to the
trigonometrics. It works out the functions’ derivatives and the derivatives
of the basic members of the trigonometric and inverse trigonometric families
to which they respectively belong.
5.1 The real exponential

Consider the factor
(1 + ǫ)N .
This is the overall factor by which a quantity grows after N iterative rounds
of multiplication by (1 + ǫ). What happens when ǫ is very small but N is
very large? The really interesting question is, what happens in the limit, as
ǫ → 0 and N → ∞, while x = ǫN remains a finite number? The answer is
that the factor becomes
exp x ≡ lim (1 + ǫ)x/ǫ . (5.1)
ǫ→0
Equation (5.1) defines the natural exponential function—commonly, more

briefly named the exponential function. Another way to write the same
definition is
exp x = ex , (5.2)
e ≡ lim (1 + ǫ)1/ǫ . (5.3)
ǫ→0
93
94 CHAPTER 5. THE COMPLEX EXPONENTIAL
Whichever form we write it in, the question remains as to whether the

limit actually exists; that is, whether 0 < e < ∞; whether in fact we can
put some concrete bound on e. To show that we can,1 we observe per (4.19)
that the derivative of the exponential function is
d exp(x + δ/2) − exp(x − δ/2)

exp x = lim
dx δ→0 δ
(1 + ǫ)(x+δ/2)/ǫ − (1 + ǫ)(x−δ/2)/ǫ
= lim
δ,ǫ→0 δ
(1 + ǫ)+δ/2ǫ − (1 + ǫ)−δ/2ǫ
= lim (1 + ǫ)x/ǫ
δ,ǫ→0 δ
(1 + δ/2) − (1 − δ/2)
= lim (1 + ǫ)x/ǫ
δ,ǫ→0 δ
= lim (1 + ǫ)x/ǫ ,
ǫ→0
which is to say that

d
exp x = exp x. (5.4)
dx
This is a curious, important result: the derivative of the exponential function
is the exponential function itself; the slope and height of the exponential
function are everywhere equal. For the moment, however, what interests us
is that
d
exp 0 = exp 0 = lim (1 + ǫ)0 = 1,
dx ǫ→0
which says that the slope and height of the exponential function are both
unity at x = 0, implying that the straight line which best approximates
the exponential function in that neighborhood—the tangent line, which just
grazes the curve—is
y(x) = 1 + x.
With the tangent line y(x) found, the next step toward putting a concrete
bound on e is to show that y(x) ≤ exp x for all real x, that the curve runs
nowhere below the line. To show this, we observe per (5.1) that the essential
action of the exponential function is to multiply repeatedly by 1 + ǫ as x
increases, to divide repeatedly by 1 + ǫ as x decreases. Since 1 + ǫ > 1, this
1
Excepting (5.4), the author would prefer to omit much of the rest of this section, but
even at the applied level cannot think of a logically permissible way to do it. It seems
nonobvious that the limit limǫ→0 (1 + ǫ)1/ǫ actually does exist. The rest of this section
shows why it does.
5.1. THE REAL EXPONENTIAL 95
action means for real x that
exp x1 ≤ exp x2 if x1 ≤ x2 .
However, a positive number remains positive no matter how many times one
multiplies or divides it by 1 + ǫ, so the same action also means that
0 ≤ exp x
for all real x. In light of (5.4), the last two equations imply further that
d d
exp x1 ≤ exp x2 if x1 ≤ x2 ,
dx dx
d
0 ≤ exp x.
dx
But we have purposely defined the tangent line y(x) = 1 + x such that
exp 0 = y(0) = 1,
d d
exp 0 = y(0) = 1;
dx dx
that is, such that the line just grazes the curve of exp x at x = 0. Rightward,
at x > 0, evidently the curve’s slope only increases, bending upward away
from the line. Leftward, at x < 0, evidently the curve’s slope only decreases,
again bending upward away from the line. Either way, the curve never
crosses below the line for real x. In symbols,
y(x) ≤ exp x.
Figure 5.1 depicts.

Evaluating the last inequality at x = −1/2 and x = 1, we have that

1 1
≤ exp − ,
2 2
2 ≤ exp (1) .
But per (5.2), exp x = ex , so
1
≤ e−1/2 ,
2
2 ≤ e1 ,
Figure 5.1: The natural exponential.
exp x
x
−1
or in other words,
2 ≤ e ≤ 4, (5.5)
which in consideration of (5.2) puts the desired bound on the exponential
function. The limit does exist.
By the Taylor series of Table 8.1, the value
e ≈ 0x2.B7E1
can readily be calculated, but the derivation of that series does not come
until Ch. 8.
5.2 The natural logarithm

In the general exponential expression bx , one can choose any base b; for
example, b = 2 is an interesting choice. As we shall see in § 5.4, however, it
turns out that b = e, where e is the constant introduced in (5.3), is the most
interesting choice of all. For this reason among others, the base-e logarithm
is similarly interesting, such that we define for it the special notation
ln(·) = loge (·),
and call it the natural logarithm. Just as for any other base b, so also for
base b = e; thus the natural logarithm inverts the natural exponential and
vice versa:
ln exp x = ln ex = x,
(5.6)
exp ln x = eln x = x.
5.2. THE NATURAL LOGARITHM 97
Figure 5.2: The natural logarithm.
ln x
1
x
−1
Figure 5.2 plots the natural logarithm.

If
y = ln x,
then
x = exp y,
and per (5.4),
dx
= exp y.
dy
But this means that
dx
= x,
dy
the inverse of which is
dy 1
= .
dx x
In other words,
d 1
ln x = . (5.7)
dx x
Like many of the equations in these early chapters, here is another rather
significant result.2
2
Besides the result itself, the technique which leads to the result is also interesting and
is worth mastering. We shall use the technique more than once in this book.
One can specialize Table 2.5’s logarithmic base-conversion identity to

read
ln w
logb w = . (5.8)
ln b
This equation converts any logarithm to a natural logarithm. Base b = 2
logarithms are interesting, so we note here that
1
ln 2 = − ln ≈ 0x0.B172,
2
which Ch. 8 and its Table 8.1 will show how to calculate.
5.3 Fast and slow functions

The exponential exp x is a fast function. The logarithm ln x is a slow func-
tion. These functions grow, diverge or decay respectively faster and slower
than xa .
Such claims are proved by l’Hôpital’s rule (4.30). Applying the rule, we
have that
(
ln x −1 0 if a > 0,
lim a = lim a
=
x→∞ x x→∞ ax +∞ if a ≤ 0,
( (5.9)
ln x −1 −∞ if a ≥ 0,
lim = lim a =
x→0 xa x→0 ax 0 if a < 0,
which reveals the logarithm to be a slow function. Since the exp(·) and ln(·)
functions are mutual inverses, we can leverage (5.9) to show also that

exp(±x) exp(±x)
lim = lim exp ln
x→∞ xa x→∞ xa
= lim exp [±x − a ln x]
x→∞

ln x
= lim exp (x) ±1 − a
x→∞ x
= lim exp [(x) (±1 − 0)]
x→∞
= lim exp [±x] .
x→∞
That is,
exp(+x)
lim = ∞,
x→∞ xa (5.10)
exp(−x)
lim = 0,
x→∞ xa
5.3. FAST AND SLOW FUNCTIONS 99
which reveals the exponential to be a fast function. Exponentials grow or

decay faster than powers; logarithms diverge slower.
Such conclusions are extended to bases other than the natural base e
simply by observing that logb x = ln x/ ln b and that bx = exp(x ln b). Thus
exponentials generally are fast and logarithms generally are slow, regardless
of the base.3
It is interesting and worthwhile to contrast the sequence
3! 2! 1! 0! x0 x1 x2 x3 x4
...,− , , − , , , , , , ,...
x4 x3 x2 x1 0! 1! 2! 3! 4!
against the sequence
3! 2! 1! 0! x1 x2 x3 x4
...,− , , − , , ln x, , , , ,...
x4 x3 x2 x1 1! 2! 3! 4!
As x → +∞, each sequence increases in magnitude going rightward. Also,
each term in each sequence is the derivative with respect to x of the term to
its right—except left of the middle element in the first sequence and right of
the middle element in the second. The exception is peculiar. What is going
on here?
The answer is that x0 (which is just a constant) and ln x both are of
zeroth order in x. This seems strange at first because ln x diverges as x → ∞
whereas x0 does not, but the divergence of the former is extremely slow—
so slow, in fact, that per (5.9) limx→∞ (ln x)/xǫ = 0 for any positive ǫ no
matter how small.4 Figure 5.2 has plotted ln x only for x ∼ 1, but beyond the
figure’s window the curve (whose slope is 1/x) flattens rapidly rightward, to
the extent that it locally resembles the plot of a constant value; and indeed
one can write
ln(x + u)
x0 = lim ,
u→∞ ln u
which casts x0 as a logarithm shifted and scaled. Admittedly, one ought
not strain such logic too far, because ln x is not in fact a constant, but the
point nevertheless remains that x0 and ln x often play analogous roles in
mathematics. The logarithm can in some situations profitably be thought
of as a “diverging constant” of sorts.
3
There are of course some degenerate edge cases like b = 0 and b = 1. The reader can
detail these as the need arises.
4
One does not grasp how truly slow the divergence is until one calculates a few concrete
values. Consider for instance how far out x must run to make ln x = 0x100. It’s a long,
long way. The natural logarithm does indeed eventually diverge to infinity, in the literal
sense that there is no height it does not eventually reach, but it certainly does not hurry.
As we have seen, it takes practically forever just to reach 0x100.
Less strange-seeming perhaps is the consequence of (5.10) that exp x is

of infinite order in x, that x∞ and exp x play analogous roles.
It befits an applied mathematician subjectively to internalize (5.9) and
(5.10), to remember that ln x resembles x0 and that exp x resembles x∞ . A
qualitative sense that logarithms are slow and exponentials, fast, helps one
to grasp mentally the essential features of many mathematical models one
encounters in practice.
Now leaving aside fast and slow functions for the moment, we turn our
attention in the next section to the highly important matter of the expo-
nential of a complex argument.
5.4 Euler’s formula

The result of § 5.1 leads to one of the central questions in all of mathematics.
How can one evaluate
exp iθ = lim (1 + ǫ)iθ/ǫ ,

ǫ→0
where i2 = −1 is the imaginary unit introduced in § 2.12?

To begin, one can take advantage of (4.14) to write the last equation in
the form
exp iθ = lim (1 + iǫ)θ/ǫ ,
ǫ→0
but from here it is not obvious where to go. The book’s development up
to the present point gives no obvious direction. In fact it appears that the
interpretation of exp iθ remains for us to define, if we can find a way to define
it which fits sensibly with our existing notions of the real exponential. So,
if we don’t quite know where to go with this yet, what do we know?
One thing we know is that if θ = ǫ, then
exp(iǫ) = (1 + iǫ)ǫ/ǫ = 1 + iǫ.
But per § 5.1, the essential operation of the exponential function is to multi-
ply repeatedly by some factor, where the factor is not quite exactly unity—in
this case, by 1 + iǫ. So let us multiply a complex number z = x + iy by
1 + iǫ, obtaining
(1 + iǫ)(x + iy) = (x − ǫy) + i(y + ǫx).
The resulting change in z is
∆z = (1 + iǫ)(x + iy) − (x + iy) = (ǫ)(−y + ix),

5.4. EULER’S FORMULA 101
Figure 5.3: The complex exponential and Euler’s formula.
iℑ(z)
i2
∆z
i z
ρ
φ
ℜ(z)
−2 −1 1 2
−i
−i2
in which it is seen that

p
|∆z| = (ǫ) y 2 + x2 = (ǫ) |z| ,
x 2π
arg(∆z) = arctan = arg z + .
−y 4
In other words, the change is proportional to the magnitude of z, but at a

right angle to z’s arm in the complex plane. Refer to Fig. 5.3.
Since the change is directed neither inward nor outward from the dashed
circle in the figure, evidently its effect is to move z a distance (ǫ) |z| coun-
terclockwise about the circle. In other words, referring to the figure, the
change ∆z leads to
∆ρ = 0,
∆φ = ǫ.
These two equations—implying an unvarying, steady counterclockwise pro-

gression about the circle in the diagram—are somewhat unexpected. Given
the steadiness of the progression, it follows from the equations that
θ
|exp iθ| = lim (1 + iǫ)θ/ǫ = |exp 0| + lim ∆ρ = 1,

ǫ→0 ǫ→0 ǫ
h i θ
arg [exp iθ] = lim arg (1 + iǫ)θ/ǫ = arg [exp 0] + lim ∆φ = θ.
ǫ→0 ǫ→0 ǫ
That is, exp iθ is the complex number which lies on the Argand unit circle at
phase angle θ. Had we known that θ was an Argand phase angle, naturally
we should have represented it by the symbol φ from the start. Changing
φ ← θ now, we have for real φ that
|exp iφ| = 1,
arg [exp iφ] = φ,
which says neither more nor less than that
exp iφ = cos φ + i sin φ = cis φ. (5.11)
where cis(·) is as defined in § 3.11.

Along with the Pythagorean theorem (2.47), the fundamental theorem of
calculus (7.2) and Cauchy’s integral formula (8.29), eqn. (5.11) is one of the
most famous results in all of mathematics. It is called Euler’s formula,5,6
and it opens the exponential domain fully to complex numbers, not just for
the natural base e but for any base. How? Consider in light of Fig. 5.3
and (5.11) that one can express any complex number in the form
z = x + iy = ρ exp iφ.
If a complex base w is similarly expressed in the form
w = u + iv = σ exp iψ,
then it follows that
wz = exp[ln wz ]
= exp[z ln w]
= exp[(x + iy)(iψ + ln σ)]
= exp[(x ln σ − ψy) + i(y ln σ + ψx)].
Since exp(α + β) = eα+β = exp α exp β, the last equation is
wz = exp(x ln σ − ψy) exp i(y ln σ + ψx), (5.12)

5
For native English speakers who do not speak German, Leonhard Euler’s name is
pronounced as “oiler.”
6
An alternate derivation of Euler’s formula (5.11)—less intuitive and requiring slightly
more advanced mathematics, but briefer—constructs from Table 8.1 the Taylor series
for exp iφ, cos φ and i sin φ, then adds the latter two to show them equal to the first
of the three. Such an alternate derivation lends little insight, perhaps, but at least it
builds confidence that we actually knew what we were doing when we came up with the
incredible (5.11).
5.5. COMPLEX EXPONENTIALS AND DE MOIVRE 103
where
x = ρ cos φ,
y = ρ sin φ,
p
σ = u2 + v 2 ,
v
tan ψ = .
u
Equation (5.12) serves to raise any complex number to a complex power.
Curious consequences of Euler’s formula (5.11) include that
e±i2π/4 = ±i,
e±i2π/2 = −1, (5.13)
ein2π = 1.
For the natural logarithm of a complex number in light of Euler’s formula,
we have that
ln w = ln σeiψ = ln σ + iψ. (5.14)
5.5 Complex exponentials and de Moivre’s theo-

rem
Euler’s formula (5.11) implies that complex numbers z1 and z2 can be written
z1 = ρ1 eiφ1 ,
(5.15)
z2 = ρ2 eiφ2 .
By the basic power properties of Table 2.2, then,
z1 z2 = ρ1 ρ2 ei(φ1 +φ2 ) = ρ1 ρ2 exp[i(φ1 + φ2 )],

z1 ρ1 i(φ1 −φ2 ) ρ1
= e = exp[i(φ1 − φ2 )], (5.16)
z2 ρ2 ρ2
z a = ρa eiaφ = ρa exp[iaφ].
This is de Moivre’s theorem, introduced in § 3.11.
5.6 Complex trigonometrics

Applying Euler’s formula (5.11) to +φ then to −φ, we have that
exp(+iφ) = cos φ + i sin φ,
exp(−iφ) = cos φ − i sin φ.
Adding the two equations and solving for cos φ yields

exp(+iφ) + exp(−iφ)
cos φ = . (5.17)
2
Subtracting the second equation from the first and solving for sin φ yields
exp(+iφ) − exp(−iφ)
sin φ = . (5.18)
i2
Thus are the trigonometrics expressed in terms of complex exponentials.
The forms (5.17) and (5.18) suggest the definition of new functions
exp(+φ) + exp(−φ)
cosh φ ≡ , (5.19)
2
exp(+φ) − exp(−φ)
sinh φ ≡ , (5.20)
2
sinh φ
tanh φ ≡ . (5.21)
cosh φ
These are called the hyperbolic functions. Their inverses arccosh, etc., are de-
fined in the obvious way. The Pythagorean theorem for trigonometrics (3.2)
is that cos2 φ + sin2 φ = 1; and from (5.19) and (5.20) one can derive the
hyperbolic analog:
cos2 φ + sin2 φ = 1,
(5.22)
cosh2 φ − sinh2 φ = 1.
Both lines of (5.22) hold for complex φ as well as for real.7

The notation exp i(·) or ei(·) is sometimes felt to be too bulky. Although
less commonly seen than the other two, the notation
cis(·) ≡ exp i(·) = cos(·) + i sin(·)

7
Chapter 15 teaches that the “dot product” of a unit vector and its own conjugate is
unity—v̂∗ · v̂ = 1, in the notation of that chapter—which tempts one incorrectly to suppose
by analogy that cos∗ φ cos φ + sin∗ φ sin φ = 1 and that cosh∗ φ cosh φ − sinh∗ φ sinh φ =
1 when the angle φ is complex. However, (5.17) through (5.20) can generally be true
only if (5.22) holds exactly as written for complex φ as well as for real. Hence in fact
cos∗ φ cos φ + sin∗ φ sin φ 6= 1 and cosh∗ φ cosh φ − sinh∗ φ sinh φ 6= 1.
Such confusion probably tempts few readers unfamiliar with the material of Ch. 15, so
you can ignore this footnote for now. However, if later you return after reading Ch. 15
and if the confusion then arises, then consider that the angle φ of Fig. 3.1 is a real angle,
whereas we originally derived (5.22)’s first line from that figure. The figure is quite handy
for real φ, but what if anything the figure means when φ is complex is not obvious. If the
confusion descends directly or indirectly from the figure, then such thoughts may serve to
clarify the matter.
5.7. SUMMARY OF PROPERTIES 105
is also conventionally recognized, as earlier seen in § 3.11. Also conven-

tionally recognized are sin−1 (·) and occasionally asin(·) for arcsin(·), and
likewise for the several other trigs.
Replacing z ← φ in this section’s several equations implies a coherent
definition for trigonometric functions of a complex variable. Then, compar-
ing (5.17) and (5.18) respectively to (5.19) and (5.20), we have that
cosh z = cos iz,

i sinh z = sin iz, (5.23)
i tanh z = tan iz,
by which one can immediately adapt the many trigonometric properties of

Tables 3.1 and 3.3 to hyperbolic use.
At this point in the development one begins to notice that the sin, cos,
exp, cis, cosh and sinh functions are each really just different facets of the
same mathematical phenomenon. Likewise their respective inverses: arcsin,
arccos, ln, −i ln, arccosh and arcsinh. Conventional names for these two
mutually inverse families of functions are unknown to the author, but one
might call them the natural exponential and natural logarithmic families.
Or, if the various tangent functions were included, then one might call them
the trigonometric and inverse trigonometric families.
5.7 Summary of properties

Table 5.1 gathers properties of the complex exponential from this chapter
and from §§ 2.12, 3.11 and 4.4.
5.8 Derivatives of complex exponentials

This section computes the derivatives of the various trigonometric and in-
verse trigonometric functions.
5.8.1 Derivatives of sine and cosine

One can compute derivatives of the sine and cosine functions from (5.17)
and (5.18), but to do it in that way doesn’t seem sporting. Better applied
style is to find the derivatives by observing directly the circle from which
the sine and cosine functions come.
Table 5.1: Complex exponential properties.
i2 = −1 = (−i)2
1
= −i
i
eiφ = cos φ + i sin φ
eiz = cos z + i sin z
z1 z2 = ρ1 ρ2 ei(φ1 +φ2 ) = (x1 x2 − y1 y2 ) + i(y1 x2 + x1 y2 )
z1 ρ1 i(φ1 −φ2 ) (x1 x2 + y1 y2 ) + i(y1 x2 − x1 y2 )
= e =
z2 ρ2 x22 + y22
z a = ρa eiaφ
wz = ex ln σ−ψy ei(y ln σ+ψx)
ln w = ln σ + iψ
eiz − e−iz ez − e−z

sin z = sin iz = i sinh z sinh z =
i2 2
eiz + e−iz ez + e−z
cos z = cos iz = cosh z cosh z =
2 2
sin z sinh z
tan z = tan iz = i tanh z tanh z =
cos z cosh z
cos2 z + sin2 z = 1 = cosh2 z − sinh2 z
z ≡ x + iy = ρeiφ
w ≡ u + iv = σeiψ
exp z ≡ ez
cis z ≡ cos z + i sin z = eiz
d
exp z = exp z
dz
d 1
ln w =
dw w
df /dz d
= ln f (z)
f (z) dz
ln w
logb w =
ln b
5.8. DERIVATIVES OF COMPLEX EXPONENTIALS 107
Figure 5.4: The derivatives of the sine and cosine functions.
ℑ(z)
dz
dt
ρ z
ωt + φo
ℜ(z)
Refer to Fig. 5.4. Suppose that the point z in the figure is not fixed but
travels steadily about the circle such that
z(t) = (ρ) [cos(ωt + φo ) + i sin(ωt + φo )] . (5.24)
How fast then is the rate dz/dt, and in what Argand direction?

dz d d
= (ρ) cos(ωt + φo ) + i sin(ωt + φo ) . (5.25)
dt dt dt
Evidently,
• the speed is |dz/dt| = (ρ)(dφ/dt) = ρω;
• the direction is at right angles to the arm of ρ, which is to say that

arg(dz/dt) = φ + 2π/4.
With these observations we can write that

dz 2π 2π
= (ρω) cos ωt + φo + + i sin ωt + φo +
dt 4 4
= (ρω) [− sin(ωt + φo ) + i cos(ωt + φo )] . (5.26)
Matching the real and imaginary parts of (5.25) against those of (5.26), we
have that
d
cos(ωt + φo ) = −ω sin(ωt + φo ),
dt (5.27)
d
sin(ωt + φo ) = +ω cos(ωt + φo ).
dt
If ω = 1 and φo = 0, these are
d
cos t = − sin t,
dt (5.28)
d
sin t = + cos t.
dt
5.8.2 Derivatives of the trigonometrics

Equations (5.4) and (5.28) give the derivatives of exp(·), sin(·) and cos(·).
From these, with the help of (5.22) and the derivative chain and product
rules (§ 4.5), we can calculate the several derivatives of Table 5.2.8
5.8.3 Derivatives of the inverse trigonometrics

Observe the pair
d
exp z = exp z,
dz
d 1
ln w = .
dw w
The natural exponential exp z belongs to the trigonometric family of func-

tions, as does its derivative. The natural logarithm ln w, by contrast, belongs
to the inverse trigonometric family of functions; but its derivative is simpler,
not a trigonometric or inverse trigonometric function at all. In Table 5.2,
one notices that all the trigonometrics have trigonometric derivatives. By
analogy with the natural logarithm, do all the inverse trigonometrics have
simpler derivatives?
It turns out that they do. Refer to the account of the natural logarithm’s
derivative in § 5.2. Following a similar procedure, we have by successive steps
8
[55, back endpaper]
5.8. DERIVATIVES OF COMPLEX EXPONENTIALS 109
Table 5.2: Derivatives of the trigonometrics.
d d 1 1
exp z = + exp z = −
dz dz exp z exp z
d d 1 1
sin z = + cos z = −
dz dz sin z tan z sin z
d d 1 tan z
cos z = − sin z = +
dz dz cos z cos z
d 1
tan z = + 1 + tan2 z

= + 2
dz cos z
d 1 1 1
= − 1+ 2 = − 2
dz tan z tan z sin z
d d 1 1
sinh z = + cosh z = −
dz dz sinh z tanh z sinh z
d d 1 tanh z
cosh z = + sinh z = −
dz dz cosh z cosh z
d 1
tanh z = 1 − tanh2 z = +
dz cosh2 z
d 1 1 1
= 1− = −
dz tanh z tanh2 z sinh2 z
that
arcsin w = z,
w = sin z,
dw
= cos z,
dz
dw p
= ± 1 − sin2 z,
dz
dw p
= ± 1 − w2 ,
dz
dz ±1
= √ ,
dw 1 − w2
d ±1
arcsin w = √ . (5.29)
dw 1 − w2
Similarly,
arctan w = z,
w = tan z,
dw
= 1 + tan2 z,
dz
dw
= 1 + w2 ,
dz
dz 1
= ,
dw 1 + w2
d 1
arctan w = . (5.30)
dw 1 + w2
Derivatives of the other inverse trigonometrics are found in the same way.
Table 5.3 summarizes.
5.9 The actuality of complex quantities

Doing all this neat complex math, the applied mathematician can lose sight
of some questions he probably ought to keep in mind: Is there really such a
thing as a complex quantity in nature? If not, then hadn’t we better avoid
these complex quantities, leaving them to the professional mathematical
theorists?
As developed by Oliver Heaviside in 1887,9 the answer depends on your
point of view. If I have 300 g of grapes and 100 g of grapes, then I have 400 g
9
[46]
5.9. THE ACTUALITY OF COMPLEX QUANTITIES 111
Table 5.3: Derivatives of the inverse trigonometrics.
d 1
ln w =
dw w
d ±1
arcsin w = √
dw 1 − w2
d ∓1
arccos w = √
dw 1 − w2
d 1
arctan w =
dw 1 + w2
d ±1
arcsinh w = √
dw w2 + 1
d ±1
arccosh w = √
dw w2 − 1
d 1
arctanh w =
dw 1 − w2
altogether. Alternately, if I have 500 g of grapes and −100 g of grapes,

again I have 400 g altogether. (What does it mean to have −100 g of
grapes? Maybe that I ate some!) But what if I have 200 + i100 g of grapes
and 200 − i100 g of grapes? Answer: again, 400 g.
Probably you would not choose to think of 200 + i100 g of grapes
and 200 − i100 g of grapes, but because of (5.17) and (5.18), one often
describes wave phenomena as linear superpositions (sums) of countervailing
complex exponentials. Consider for instance the propagating wave
A A
A cos[ωt − kz] = exp[+i(ωt − kz)] + exp[−i(ωt − kz)].
2 2
The benefit of splitting the real cosine into two complex parts is that while
the magnitude of the cosine changes with time t, the magnitude of either
exponential alone remains steady (see the circle in Fig. 5.3). It turns out to
be much easier to analyze two complex wave quantities of constant magni-
tude than to analyze one real wave quantity of varying magnitude. Better
yet, since each complex wave quantity is the complex conjugate of the other,
the analyses thereof are mutually conjugate, too; so you normally needn’t
actually analyze the second. The one analysis suffices for both.10 (It’s like
10
If the point is not immediately clear, an example: Suppose that by the Newton-
reflecting your sister’s handwriting. To read her handwriting backward, you

needn’t ask her to try writing reverse with the wrong hand; you can just
hold her regular script up to a mirror. Of course, this ignores the question of
why one would want to reflect someone’s handwriting in the first place; but
anyway, reflecting—which is to say, conjugating—complex quantities often
is useful.)
Some authors have gently denigrated the use of imaginary parts in phys-
ical applications as a mere mathematical trick, as though the parts were not
actually there. Well, that is one way to treat the matter, but it is not the
way this book recommends. Nothing in the mathematics requires you to
regard the imaginary parts as physically nonexistent. You need not abuse
Occam’s razor! (Occam’s razor, “Do not multiply objects without neces-
sity,”11 is a fine philosophical tool as far as it goes, but is overused in some
circles. More often than one likes to believe, the necessity remains hidden
until one has ventured to multiply the objects, nor reveals itself to the one
who wields the razor, whose hand humility should stay.) It is true by Eu-
ler’s formula (5.11) that a complex exponential exp iφ can be decomposed
into a sum of trigonometrics. However, it is equally true by the complex
trigonometric formulas (5.17) and (5.18) that a trigonometric can be decom-
posed into a sum of complex exponentials. So, if each can be decomposed
into the other, then which of the two is the real decomposition? Answer:
it depends on your point of view. Experience seems to recommend view-
ing the complex exponential as the basic element—as the element of which
the trigonometrics are composed—rather than the other way around. From
this point of view, it is (5.17) and (5.18) which are the real decomposition.
Euler’s formula itself is secondary.
The complex exponential method of offsetting imaginary parts offers an
elegant yet practical mathematical way to model physical wave phenomena.
So go ahead: regard the imaginary parts as actual. It doesn’t hurt anything,
and it helps with the math.
Raphson iteration (§ 4.8) you have found a root of the polynomial x3 + 2x2 + 3x + 4 at
x ≈ −0x0.2D + i0x1.8C. Where is there another root? Answer: at the complex conjugate,
x ≈ −0x0.2D − i0x1.8C. One need not actually run the Newton-Raphson again to find
the conjugate root.
11
[61, Ch. 12]
Chapter 6
Primes, roots and averages
This chapter gathers a few significant topics, each of whose treatment seems
too brief for a chapter of its own.
6.1 Prime numbers

A prime number —or simply, a prime—is an integer greater than one, divis-
ible only by one and itself. A composite number is an integer greater than
one and not prime. A composite number can be composed as a product of
two or more prime numbers. All positive integers greater than one are either
composite or prime.
The mathematical study of prime numbers and their incidents consti-
tutes number theory, and it is a deep area of mathematics. The deeper
results of number theory seldom arise in applications,1 however, so we shall
confine our study of number theory in this book to one or two of its simplest,
most broadly interesting results.
6.1.1 The infinite supply of primes

The first primes are evidently 2, 3, 5, 7, 0xB, . . . . Is there a last prime? To
show that there is not, suppose that there were. More precisely, suppose
that there existed exactly N primes, with N finite, letting p1 , p2 , . . . , pN
represent these primes from least to greatest. Now consider the product of
1
The deeper results of number theory do arise in cryptography, or so the author has
been led to understand. Although cryptography is literally an application of mathematics,
its spirit is that of pure mathematics rather than of applied. If you seek cryptographic
derivations, this book is probably not the one you want.
113
114 CHAPTER 6. PRIMES, ROOTS AND AVERAGES
all the primes,

N
Y
C= pj .
j=1
What of C + 1? Since p1 = 2 divides C, it cannot divide C + 1. Similarly,

since p2 = 3 divides C, it also cannot divide C + 1. The same goes for
p3 = 5, p4 = 7, p5 = 0xB, etc. Apparently none of the primes in the pj
series divides C + 1, which implies either that C + 1 itself is prime, or that
C + 1 is composed of primes not in the series. But the latter is assumed
impossible on the ground that the pj series includes all primes; and the
former is assumed impossible on the ground that C + 1 > C > pN , with pN
the greatest prime. The contradiction proves false the assumption which
gave rise to it. The false assumption: that there were a last prime.
Thus there is no last prime. No matter how great a prime number one
finds, a greater can always be found. The supply of primes is infinite.2
Attributed to the ancient geometer Euclid, the foregoing proof is a clas-
sic example of mathematical reductio ad absurdum, or as usually styled in
English, proof by contradiction.3
6.1.2 Compositional uniqueness
Occasionally in mathematics, plausible assumptions can hide subtle logical

flaws. One such plausible assumption is the assumption that every positive
integer has a unique prime factorization. It is readily seen that the first
several positive integers—1 = (), 2 = (21 ), 3 = (31 ), 4 = (22 ), 5 = (51 ),
6 = (21 )(31 ), 7 = (71 ), 8 = (23 ), . . . —each have unique prime factorizations,
but is this necessarily true of all positive integers?
To show that it is true, suppose that it were not.4 More precisely, sup-
pose that there did exist positive integers factorable each in two or more
distinct ways, with the symbol C representing the least such integer. Noting
that C must be composite (prime numbers by definition are each factorable
2
[58]
3
[52, Appendix 1][67, “Reductio ad absurdum,” 02:36, 28 April 2006]
4
Unfortunately the author knows no more elegant proof than this, yet cannot even cite
this one properly. The author encountered the proof in some book over a decade ago. The
identity of that book is now long forgotten.
6.1. PRIME NUMBERS 115
only one way, like 5 = [51 ]), let

Np
Y
Cp ≡ pj ,
j=1
Nq
Y
Cq ≡ qk ,
k=1
Cp = Cq = C,
pj ≤ pj+1 ,
qk ≤ qk+1 ,
p1 ≤ q 1 ,
Np > 1,
Nq > 1,
where Cp and Cq represent two distinct prime factorizations of the same
number C and where the pj and qk are the respective primes ordered from
least to greatest. We see that
pj 6= qk
for any j and k—that is, that the same prime cannot appear in both
factorizations—because if the same prime r did appear in both then C/r
either would be prime (in which case both factorizations would be [r][C/r],
defying our assumption that the two differed) or would constitute an am-
biguously factorable composite integer less than C when we had already
defined C to represent the least such. Among other effects, the fact that
pj 6= qk strengthens the definition p1 ≤ q1 to read
p1 < q 1 .
Let us now rewrite the two factorizations in the form
Cp = p1 Ap ,
Cq = q1 Aq ,
Cp = Cq = C,
Np
Y
Ap ≡ pj ,
j=2
Nq
Y
Aq ≡ qk ,
k=2
where p1 and q1 are the least primes in their respective factorizations.

Since C is composite and since p1 < q1 , we have that
√
1 < p1 < q1 ≤ C ≤ Aq < Ap < C,
which implies that

p1 q1 < C.
The last inequality lets us compose the new positive integer
B = C − p1 q 1 ,
which might be prime or composite (or unity), but which either way enjoys
a unique prime factorization because B < C, with C the least positive
integer factorable two ways. Observing that some integer s which divides C
necessarily also divides C ± ns, we note that each of p1 and q1 necessarily
divides B. This means that B’s unique factorization includes both p1 and q1 ,
which further means that the product p1 q1 divides B. But if p1 q1 divides B,
then it divides B + p1 q1 = C, also.
Let E represent the positive integer which results from dividing C
by p1 q1 :
C
E≡ .
p1 q 1
Then,
C
Eq1 = = Ap ,
p1
C
Ep1 = = Aq .
q1
That Eq1 = Ap says that q1 divides Ap . But Ap < C, so Ap ’s prime

factorization is unique—and we see above that Ap ’s factorization does not
include any qk , not even q1 . The contradiction proves false the assumption
which gave rise to it. The false assumption: that there existed a least
composite number C prime-factorable in two distinct ways.
Thus no positive integer is ambiguously factorable. Prime factorizations
are always unique.
We have observed at the start of this subsection that plausible assump-
tions can hide subtle logical flaws. Indeed this is so. Interestingly however,
the plausible assumption of the present subsection has turned out absolutely
correct; we have just had to do some extra work to prove it. Such effects
6.1. PRIME NUMBERS 117
are typical on the shadowed frontier where applied shades into pure math-
ematics: with sufficient experience and with a firm grasp of the model at
hand, if you think that it’s true, then it probably is. Judging when to delve
into the mathematics anyway, seeking a more rigorous demonstration of a
proposition one feels pretty sure is correct, is a matter of applied mathe-
matical style. It depends on how sure one feels, and more importantly on
whether the unsureness felt is true uncertainty or is just an unaccountable
desire for more precise mathematical definition (if the latter, then unlike
the author you may have the right temperament to become a professional
mathematician). The author does judge the present subsection’s proof to
be worth the applied effort; but nevertheless, when one lets logical minutiae
distract him to too great a degree, he admittedly begins to drift out of the
applied mathematical realm that is the subject of this book.
6.1.3 Rational and irrational numbers

A rational number is a finite real number expressible as a ratio of integers5
p
x= , (p, q) ∈ Z, q 6= 0.
q
The ratio is fully reduced if p and q have no prime factors in common. For
instance, 4/6 is not fully reduced, whereas 2/3 is.
An irrational
√ number is a finite real number which is not rational. For
√
example, 2 is irrational. In fact any x = n is irrational unless integral;
√
there is no such thing as a n which is not an integer but is rational.
To prove6 the last point, suppose that there did exist a fully reduced
p √
x= = n, (n, p, q) ∈ Z, n > 0, p > 0, q > 1.
q
Squaring the equation, we have that
p2
= n,
q2
which form is evidently also fully reduced. But if q > 1, then the fully
reduced n = p2 /q 2 is not an integer as we had assumed that it was. The
contradiction proves false the assumption which gave rise to it. Hence there
√
exists no rational, nonintegral n, as was to be demonstrated. The proof is
readily extended to show that any x = nj/k is irrational if nonintegral, the
5
Section 2.3 explains the ∈ Z notation.
6
A proof somewhat like the one presented here is found in [52, Appendix 1].
extension by writing pk /q k = nj then following similar steps as those this

paragraph outlines.
That’s all the number theory the book treats; but in applied math, so
little will take you pretty far. Now onward we go to other topics.
6.2 The existence and number of polynomial roots

This section shows that an N th-order polynomial must have exactly N roots.
6.2.1 Polynomial roots

Consider the quotient B(z)/A(z), where
A(z) = z − α,
N
X
B(z) = bk z k , N > 0, bN 6= 0,
k=0
B(α) = 0.
In the long-division symbology of Table 2.3,
B(z) = A(z)Q0 (z) + R0 (z),

where Q0 (z) is the quotient and R0 (z), a remainder. In this case the divisor
A(z) = z − α has first order, and as § 2.6.2 has observed, first-order divisors
leave zeroth-order, constant remainders R0 (z) = ρ. Thus substituting yields
B(z) = (z − α)Q0 (z) + ρ.
When z = α, this reduces to
B(α) = ρ.
But B(α) = 0 by assumption, so

ρ = 0.
Evidently the division leaves no remainder ρ, which is to say that z − α

exactly divides every polynomial B(z) of which z = α is a root.
Note that if the polynomial B(z) has order N , then the quotient Q(z) =
B(z)/(z − α) has exactly order N − 1. That is, the leading, z N −1 term of
the quotient is never null. The reason is that if the leading term were null, if
Q(z) had order less than N − 1, then B(z) = (z − α)Q(z) could not possibly
have order N as we have assumed.
6.2. THE EXISTENCE AND NUMBER OF ROOTS 119
6.2.2 The fundamental theorem of algebra

The fundamental theorem of algebra holds that any polynomial B(z) of or-
der N can be factored
N
X N
Y
k
B(z) = bk z = bN (z − αj ), bN 6= 0, (6.1)
k=0 j=1
where the αk are the N roots of the polynomial.7

To prove the theorem, it suffices to show that all polynomials of order
N > 0 have at least one root; for if a polynomial of order N has a root αN ,
then according to § 6.2.1 one can divide the polynomial by z − αN to obtain
a new polynomial of order N − 1. To the new polynomial the same logic
applies: if it has at least one root αN −1 , then one can divide it by z−αN −1 to
obtain yet another polynomial of order N − 2; and so on, one root extracted
at each
QN step, factoring the polynomial step by step into the desired form
bN j=1 (z − αj ).
It remains however to show that there exists no polynomial B(z) of order
N > 0 lacking roots altogether. To show that there is no such polynomial,
consider the locus8 of all B(ρeiφ ) in the Argand range plane (Fig. 2.5), where
z = ρeiφ , ρ is held constant, and φ is variable. Because ei(φ+n2π) = eiφ and
no fractional powers of z appear in (6.1), this locus forms a closed loop. At
very large ρ, the bN z N term dominates B(z), so the locus there evidently
has the general character of bN ρN eiN φ . As such, the locus is nearly but not
quite a circle at radius bN ρN from the Argand origin B(z) = 0, revolving N
times at that great distance before exactly repeating. On the other hand,
when ρ = 0 the entire locus collapses on the single point B(0) = b0 .
Now consider the locus at very large ρ again, but this time let ρ slowly
shrink. Watch the locus as ρ shrinks. The locus is like a great string or
rubber band, joined at the ends and looped in N great loops. As ρ shrinks
smoothly, the string’s shape changes smoothly. Eventually ρ disappears and
the entire string collapses on the point B(0) = b0 . Since the string originally
has looped N times at great distance about the Argand origin, but at the
end has collapsed on a single point, then at some time between it must have
swept through the origin and every other point within the original loops.
7
Professional mathematicians typically state the theorem in a slightly different form.
They also prove it in rather a different way. [31, Ch. 10, Prob. 74]
8
A locus is the geometric collection of points which satisfy a given criterion. For
example, the locus of all points in a plane at distance ρ from a point O is a circle; the
locus of all points in three-dimensional space equidistant from two points P and Q is a
plane; etc.
After all, B(z) is everywhere differentiable, so the string can only sweep
as ρ decreases; it can never skip. The Argand origin lies inside the loops at
the start but outside at the end. If so, then the values of ρ and φ precisely
where the string has swept through the origin by definition constitute a
root B(ρeiφ ) = 0. Thus as we were required to show, B(z) does have at
least one root, which observation completes the applied demonstration of
the fundamental theorem of algebra.
The fact that the roots exist is one thing. Actually finding the roots nu-
merically is another matter. For a quadratic (second order) polynomial, (2.2)
gives the roots. For cubic (third order) and quartic (fourth order) polyno-
mials, formulas for the roots are known (see Ch. 10) though seemingly not
so for quintic (fifth order) and higher-order polynomials;9 but the Newton-
Raphson iteration (§ 4.8) can be used to locate a root numerically in any
case. The Newton-Raphson is used to extract one root (any root) at each
step as described above, reducing the polynomial step by step until all the
roots are found.
The reverse problem, findingQ the polynomial given the roots, is much
easier: one just multiplies out j (z − αj ), as in (6.1).
6.3 Addition and averages

This section discusses the two basic ways to add numbers and the three
basic ways to calculate averages of them.
6.3.1 Serial and parallel addition

Consider the following problem. There are three masons. The strongest and
most experienced of the three, Adam, lays 120 bricks per hour.10 Next is
Brian who lays 90. Charles is new; he lays only 60. Given eight hours, how
many bricks can the three men lay? Answer:
(8 hours)(120 + 90 + 60 bricks per hour) = 2160 bricks.
Now suppose that we are told that Adam can lay a brick every 30 seconds;
Brian, every 40 seconds; Charles, every 60 seconds. How much time do the
9
In a celebrated theorem of pure mathematics [65, “Abel’s impossibility theorem”], it is
said to be shown that no such formula even exists, given that the formula be constructed
according to certain rules. Undoubtedly the theorem is interesting to the professional
mathematician, but to the applied mathematician it probably suffices to observe merely
that no such formula is known.
10
The figures in the example are in decimal notation.
6.3. ADDITION AND AVERAGES 121
three men need to lay 2160 bricks? Answer:

2160 bricks 1 hour
1 1 1 = 28,800 seconds
30 + 40 + 60 bricks per second 3600 seconds
= 8 hours.
The two problems are precisely equivalent. Neither is stated in simpler terms
than the other. The notation used to solve the second is less elegant, but
fortunately there exists a better notation:
(2160 bricks)(30 k 40 k 60 seconds per brick) = 8 hours,
where
1 1 1 1
= + + .
30 k 40 k 60 30 40 60
The operator k is called the parallel addition operator. It works according
to the law
1 1 1
= + , (6.2)
akb a b
where the familiar operator + is verbally distinguished from the k when
necessary by calling the + the serial addition or series addition operator.
With (6.2) and a bit of arithmetic, the several parallel-addition identities of
Table 6.1 are soon derived.
The writer knows of no conventional notation for parallel sums of series,
but suggests that the notation which appears in the table,
b
X
k f (k) ≡ f (a) k f (a + 1) k f (a + 2) k · · · k f (b),
k=a
might serve if needed.

Assuming that none of the values involved is negative, one can readily
show that11
a k x ≤ b k x iff a ≤ b. (6.3)
This is intuitive. Counterintuitive, perhaps, is that
a k x ≤ a. (6.4)
Because we have all learned as children to count in the sensible man-

ner 1, 2, 3, 4, 5, . . .—rather than as 1, 21 , 13 , 14 , 51 , . . .—serial addition (+) seems
11
The word iff means, “if and only if.”
Table 6.1: Parallel and serial addition identities.
1 1 1 1 1 1
= + = k
akb a b a+b a b
ab ab
akb = a+b =
a+b akb
1 a 1 a
ak = a+ =
b 1 + ab b 1 k ab
akb = bka a+b = b+a
a k (b k c) = (a k b) k c a + (b + c) = (a + b) + c
ak∞ = ∞ka = a a+0 =0+a = a
a k (−a) = ∞ a + (−a) = 0
(a)(b k c) = ab k ac (a)(b + c) = ab + ac
1 X 1 1 X 1
P = P = k
k k ak k
ak k ak
k
ak
more natural than parallel addition (k) does. The psychological barrier is
hard to breach, yet for many purposes parallel addition is in fact no less
fundamental. Its rules are inherently neither more nor less complicated, as
Table 6.1 illustrates; yet outside the electrical engineering literature the par-
allel addition notation is seldom seen.12 Now that you have seen it, you can
use it. There is profit in learning to think both ways. (Exercise: counting
from zero serially goes 0, 1, 2, 3, 4, 5, . . .; how does the parallel analog go?)13
Convention brings no special symbol for parallel subtraction, inciden-
tally. One merely writes
a k (−b),
which means exactly what it appears to mean.
6.3.2 Averages
Let us return to the problem of the preceding section. Among the three
masons, what is their average productivity? The answer depends on how
you look at it. On the one hand,
120 + 90 + 60 bricks per hour

= 90 bricks per hour.
3
On the other hand,
30 + 40 + 60 seconds per brick

= 43 13 seconds per brick.
3
These two figures are not the same. That is, 1/(43 31 seconds per brick) 6=
90 bricks per hour. Yet both figures are valid. Which figure you choose
depends on what you want to calculate. A common mathematical error
among businesspeople is not to realize that both averages are possible and
that they yield different numbers (if the businessperson quotes in bricks per
hour, the productivities average one way; if in seconds per brick, the other
way; yet some businesspeople will never clearly consider the difference).
Realizing this, the clever businessperson might negotiate a contract so that
the average used worked to his own advantage.14
12
In electric circuits, loads are connected in parallel as often as, in fact probably more
often than, they are connected in series. Parallel addition gives the electrical engineer a
neat way of adding the impedances of parallel-connected loads.
13
[54, eqn. 1.27]
14
“And what does the author know about business?” comes the rejoinder.
The rejoinder is fair enough. If the author wanted to demonstrate his business acumen
(or lack thereof) he’d do so elsewhere not here! There are a lot of good business books
When it is unclear which of the two averages is more appropriate, a third

average is available, the geometric mean
[(120)(90)(60)] 1/3 bricks per hour.
The geometric mean does not have the problem either of the two averages
discussed above has. The inverse geometric mean
[(30)(40)(60)] 1/3 seconds per brick
implies the same average productivity. The mathematically savvy sometimes

prefer the geometric mean over either of the others for this reason.
Generally, the arithmetic, geometric and harmonic means are defined
P ! !
w x
k k k
X 1 X
µ ≡ P = k wk xk , (6.5)
k wk k
wk
k
 1/ Pk wk  Pk k 1/wk
Y wj Y wj
µ Π ≡  xj  =  xj  , (6.6)
j j
P ! !
k x /w xk
Pk k k =
X X
µk ≡ wk k , (6.7)
k k 1/wk k k
wk
where the xk are the several samples and the wk are weights. For two
samples weighted equally, these are
a+b
µ = , (6.8)
√2
µΠ = ab, (6.9)
µk = 2(a k b). (6.10)
out there and this is not one of them.
The fact remains nevertheless that businesspeople sometimes use mathematics in pecu-
liar ways, making relatively easy problems harder and more mysterious than the problems
need to be. If you have ever encountered the little monstrosity of an approximation banks
(at least in the author’s country) actually use in place of (9.12) to accrue interest and
amortize loans, then you have met the difficulty.
Trying to convince businesspeople that their math is wrong, incidentally, is in the au-
thor’s experience usually a waste of time. Some businesspeople are mathematically rather
sharp—as you presumably are if you are in business and are reading these words—but
as for most: when real mathematical ability is needed, that’s what they hire engineers,
architects and the like for. The author is not sure, but somehow he doubts that many
boards of directors would be willing to bet the company on a financial formula containing
some mysterious-looking ex . Business demands other talents.
If a ≥ 0 and b ≥ 0, then by successive steps,15
0 ≤ (a − b)2 ,
0 ≤ a2 − 2ab + b2 ,
4ab ≤ a2 + 2ab + b2 ,
√
2 ab ≤ a + b,
√
2 ab a+b
≤ 1 ≤ √ ,
a+b 2 ab
2ab √ a+b
≤ ab ≤ ,
a+b 2
√ a+b
2(a k b) ≤ ab ≤ .
2
That is,
µk ≤ µΠ ≤ µ. (6.11)
The arithmetic mean is greatest and the harmonic mean, least; with the
geometric mean falling between.
Does (6.11) hold when there are several nonnegative samples of various
nonnegative weights? To show that it does, consider the case of N = 2m
nonnegative samples of equal weight. Nothing prevents one from dividing
such a set of samples in half, considering each subset separately, for if (6.11)
holds for each subset individually then surely it holds for the whole set (this
is so because the average of the whole set is itself the average of the two sub-
set averages, where the word “average” signifies the arithmetic, geometric
or harmonic mean as appropriate). But each subset can further be divided
in half, then each subsubset can be divided in half again, and so on until
each smallest group has two members only—in which case we already know
that (6.11) obtains. Starting there and recursing back, we have that (6.11)
15
The steps are logical enough, but the motivation behind them remains inscrutable
until the reader realizes that the writer originally worked the steps out backward with his
pencil, from the last step to the first. Only then did he reverse the order and write the
steps formally here. The writer had no idea that he was supposed to start from 0 ≤ (a−b)2
until his pencil working backward showed him. “Begin with the end in mind,” the saying
goes. In this case the saying is right.
The same reading strategy often clarifies inscrutable math. When you can follow the
logic but cannot understand what could possibly have inspired the writer to conceive the
logic in the first place, try reading backward.
obtains for the entire set. Now consider that a sample of any weight can
be approximated arbitrarily closely by several samples of weight 1/2m , pro-
vided that m is sufficiently large. By this reasoning, (6.11) holds for any
nonnegative weights of nonnegative samples, which was to be demonstrated.
Chapter 7
The integral
Chapter 4 has observed that the mathematics of calculus concerns a com-

plementary pair of questions:
• Given some function f (t), what is the function’s instantaneous rate of
change, or derivative, f ′ (t)?
• Interpreting some function f ′ (t) as an instantaneous rate of change,
what is the corresponding accretion, or integral, f (t)?
Chapter 4 has built toward a basic understanding of the first question. This
chapter builds toward a basic understanding of the second. The understand-
ing of the second question constitutes the concept of the integral, one of the
profoundest ideas in all of mathematics.
This chapter, which introduces the integral, is undeniably a hard chapter.
Experience knows no reliable way to teach the integral adequately to
the uninitiated except through dozens or hundreds of pages of suitable ex-
amples and exercises, yet the book you are reading cannot be that kind of
book. The sections of the present chapter concisely treat matters which
elsewhere rightly command chapters or whole books of their own. Concision
can be a virtue—and by design, nothing essential is omitted here—but the
bold novice who wishes to learn the integral from these pages alone faces a
daunting challenge. It can be done. However, for less intrepid readers who
quite reasonably prefer a gentler initiation, [27] is warmly recommended.
7.1 The concept of the integral

An integral is a finite accretion or sum of an infinite number of infinitesimal
elements. This section introduces the concept.
127
128 CHAPTER 7. THE INTEGRAL
Figure 7.1: Areas representing discrete sums.
f1 (τ ) f2 (τ )
0x10 0x10
∆τ = 1 ∆τ = 21
S1 S2
τ τ
0x10 0x10
7.1.1 An introductory example

Consider the sums
0x10−1
X
S1 = k,
k=0
0x20−1
1 X k
S2 = ,
2 2
k=0
0x40−1
1 X k
S4 = ,
4 4
k=0
0x80−1
1 X k
S8 = ,
8 8
k=0
..
.
(0x10)n−1
1 X k
Sn = .
n n
k=0
What do these sums represent? One way to think of them is in terms of the
shaded areas of Fig. 7.1. In the figure, S1 is composed of several tall, thin
rectangles of width 1 and height k; S2 , of rectangles of width 1/2 and height
7.1. THE CONCEPT OF THE INTEGRAL 129
k/2.1 As n grows, the shaded region in the figure looks more and more like
a triangle of base length b = 0x10 and height h = 0x10. In fact it appears
that
bh
lim Sn = = 0x80,
n→∞ 2
or more tersely
S∞ = 0x80,
is the area the increasingly fine stairsteps approach.
Notice how we have evaluated S∞ , the sum of an infinite number of
infinitely narrow rectangles, without actually adding anything up. We have
taken a shortcut directly to the total.
In the equation
(0x10)n−1
1 X k
Sn = ,
n n
k=0
let us now change the variables
k
τ← ,
n
1
∆τ ← ,
n
to obtain the representation
(0x10)n−1
X
Sn = ∆τ τ;
k=0
or more properly,
(k|τ =0x10 )−1
X
Sn = τ ∆τ,
k=0
where the notation k|τ =0x10 indicates the value of k when τ = 0x10. Then
(k|τ =0x10 )−1

X
S∞ = lim τ ∆τ,
∆τ →0+
k=0
1
If the reader does not fully understand this paragraph’s illustration, if the relation
of the sum to the area seems unclear, the reader is urged to pause and consider the
illustration carefully until he does understand it. If it still seems unclear, then the reader
should probably suspend reading here and go study a good basic calculus text like [27].
The concept is important.
Figure 7.2: An area representing an infinite sum of infinitesimals. (Observe

that the infinitesimal dτ is now too narrow to show on this scale. Compare
against ∆τ in Fig. 7.1.)
f (τ )
0x10
S∞
τ
0x10
in which it is conventional as ∆τ vanishes to change the symbol dτ ← ∆τ ,

where dτ is the infinitesimal of Ch. 4:
(k|τ =0x10 )−1

X
S∞ = lim τ dτ.
dτ →0+
k=0
P(k| )−1
The symbol limdτ →0+ k=0τ =0x10 is cumbersome, so we replace it with the
0x10
new symbol2 0
R
to obtain the form
Z 0x10
S∞ = τ dτ.
0
This means, “stepping in infinitesimal intervals of dτ , the sum of all τ dτ

from τ = 0 to τ = 0x10.” Graphically, it is the shaded area of Fig. 7.2.
2 P
Like the
R Greek S, , denoting discrete summation, the seventeenth century-styled
Roman S, , stands for Latin “summa,” English “sum.” See [67, “Long s,” 14:54, 7 April
2006].
7.1. THE CONCEPT OF THE INTEGRAL 131
7.1.2 Generalizing the introductory example

Now consider a generalization of the example of § 7.1.1:
bn−1
1 X k
Sn = f .
n n
k=an
(In the example of § 7.1.1, f [τ ] was the simple f [τ ] = τ , but in general it

could be any function.) With the change of variables
k
τ← ,
n
1
∆τ ← ,
n
whereby
k|τ =a = an,
k|τ =b = bn,
(k, n) ∈ Z, n 6= 0,
(but a and b need not be integers), this is

(k|τ =b )−1
X
Sn = f (τ ) ∆τ.
k=(k|τ =a )
In the limit,
(k|τ =b )−1 Z b
X
S∞ = lim f (τ ) dτ = f (τ ) dτ.
dτ →0+ a
k=(k|τ =a )
This is the integral of f (τ ) in the interval a < τ < b. It represents the area
under the curve of f (τ ) in that interval.
7.1.3 The balanced definition and the trapezoid rule

Actually, just as we have defined the derivative in the balanced form (4.15),
we do well to define the integral in balanced form, too:
 
Z b  f (a) dτ (k|τ =b )−1
X f (b) dτ 
f (τ ) dτ ≡ lim + f (τ ) dτ + . (7.1)
a dτ →0+  2 2 
k=(k|τ =a )+1
Figure 7.3: Integration by the trapezoid rule (7.1). Notice that the shaded
and dashed areas total the same.
f (τ )
b
b
b
b
τ
a dτ b
Here, the first and last integration samples are each balanced “on the edge,”
half within the integration domain and half without.
Equation (7.1) is known as the trapezoid rule. Figure 7.3 depicts it. The
name “trapezoid” comes of the shapes of the shaded integration elements in
the figure. Observe however that it makes no difference whether one regards
the shaded trapezoids or the dashed rectangles as the actual integration
elements; the total integration area is the same either way.3 The important
point to understand is that the integral is conceptually just a sum. It is a
sum of an infinite number of infinitesimal elements as dτ tends to vanish,
but a sum nevertheless; nothing more.
Nothing actually requires the integration element width dτ to remain
constant from element to element, incidentally. Constant widths are usually
easiest to handle but variable widths find use in some cases. The only
requirement is that dτ remain infinitesimal. (For further discussion of the
point, refer to the treatment of the Leibnitz notation in § 4.4.2.)
3
The trapezoid rule (7.1) is perhaps the most straightforward, general, robust way to
define the integral, but other schemes are possible, too. For example, one can give the
integration elements quadratically curved tops which more nearly track the actual curve.
That scheme is called Simpson’s rule. [A section on Simpson’s rule might be added to the
book at some later date.]
7.2. THE ANTIDERIVATIVE 133
7.2 The antiderivative and the fundamental theo-

rem of calculus
If Z x
S(x) ≡ g(τ ) dτ,
a
then what is the derivative dS/dx? After some reflection, one sees that the
derivative must be
dS
= g(x).
dx
This is so because the action of the integral is to compile or accrete the
area under a curve. The integral accretes area at a rate proportional to the
curve’s height f (τ ): the higher the curve, the faster the accretion. In this
way one sees that the integral and the derivative are inverse operators; the
one inverts the other. The integral is the antiderivative.
More precisely,
Z b
df
dτ = f (τ )|ba , (7.2)
a dτ
where the notation f (τ )|ba or [f (τ )]ba means f (b) − f (a).
The importance of (7.2), fittingly named the fundamental theorem of
calculus,4 can hardly be overstated. As the formula which ties together the
complementary pair of questions asked at the chapter’s start, (7.2) is of
utmost importance in the practice of mathematics. The idea behind the
formula is indeed simple once grasped, but to grasp the idea firmly in the
first place is not entirely trivial.5 The idea is simple but big. The reader
4
[27, § 11.6][55, § 5-4][67, “Fundamental theorem of calculus,” 06:29, 23 May 2006]
5
Having read from several calculus books and, like millions of others perhaps including
the reader, having sat years ago in various renditions of the introductory calculus lectures
in school, the author has never yet met a more convincing demonstration of (7.2) than the
formula itself. Somehow the underlying idea is too simple, too profound to explain. It’s like
trying to explain how to drink water, or how to count or to add. Elaborate explanations
and their attendant constructs and formalities are indeed possible to contrive, but the idea
itself is so simple that somehow such contrivances seem to obscure the idea more than to
reveal it.
One ponders the formula (7.2) a while, then the idea dawns on him.
If you want some help pondering, try this: Sketch some arbitrary function f (τ ) on a
set of axes at the bottom of a piece of paper—some squiggle of a curve like
f (τ )
τ
a b
will do nicely—then on a separate set of axes directly above the first, sketch the cor-
Table 7.1: Basic derivatives for the antiderivative.

b
df
Z
dτ = f (τ )|ba
a dτ
d τa

τ a−1 = , a 6= 0
dτ a
1 d
= ln τ, ln 1 = 0
τ dτ
d
exp τ = exp τ, exp 0 = 1
dτ
d
cos τ = sin τ, sin 0 = 0
dτ
d
sin τ = (− cos τ ) , cos 0 = 1
dτ
is urged to pause now and ponder the formula thoroughly until he feels
reasonably confident that indeed he does grasp it and the important idea
it represents. One is unlikely to do much higher mathematics without this
formula.
As an example of the formula’s use, consider that because
(d/dτ )(τ 3 /6) = τ 2 /2, it follows that
x x
x
τ 2 dτ τ3 τ 3 x3 − 8

d
Z Z
= dτ = = .
2 2 2 dτ 6 6 2 6
Gathering elements from (4.21) and from Tables 5.2 and 5.3, Table 7.1
lists a handful of the simplest, most useful derivatives for antiderivative use.
Section 9.1 speaks further of the antiderivative.
responding slope function df /dτ . Mark two points a and b on the common horizontal
axis; then on the upper, df /dτ plot, shade the integration area under the curve. Now
consider (7.2) in light of your sketch.
There. Does the idea not dawn?
Another way to see the truth of the formula begins by canceling its (1/dτ ) dτ to obtain
Rb
the form τ =a df = f (τ )|ba . If this way works better for you, fine; but make sure that you
understand it the other way, too.
7.3. OPERATORS, LINEARITY AND MULTIPLE INTEGRALS 135
7.3 Operators, linearity and multiple integrals

This section presents the operator concept, discusses linearity and its conse-
quences, treats the commutivity of the summational and integrodifferential
operators, and introduces the multiple integral.
7.3.1 Operators
An operator is a mathematical agent that combines several values of a func-
tion.
Such a definition, unfortunately, is extraordinarily unilluminating to
those who do not already know what it means. A better way to introduce
the operator
P isQbyRgiving examples. Operators include +, −, multiplication,
division, , , and ∂. The essential action of an operator is to take
Q
several values of a function and combine them in some way. For example,
is an operator in
5
Y
(2j − 1) = (1)(3)(5)(7)(9) = 0x3B1.
j=1
Notice that the operator has acted to remove the variable j from the
expression 2j − 1. The j appears on the equation’s left side but not on its
right. The operator has used the variable up. Such a variable, used up by
an operator, is a dummy variable, as encountered earlier in § 2.3.
7.3.2 A formalism
But then how are + and − operators? They don’t use any dummy variables
up, do they? Well, it depends on how you look at it. Consider the sum
S = 3 + 5. One can write this as
1
X
S= f (k),
k=0
where 
3
 if k = 0,
f (k) ≡ 5 if k = 1,

undefined otherwise.

Then,
1
X
S= f (k) = f (0) + f (1) = 3 + 5 = 8.
k=0
By such admittedly excessive formalism, the + operator can indeed be said

to use a dummy variable up. The point is that + is in fact an operator just
like the others.
Another example of the kind:
D = g(z) − h(z) + p(z) + q(z)
= g(z) − h(z) + p(z) − 0 + q(z)
= Φ(0, z) − Φ(1, z) + Φ(2, z) − Φ(3, z) + Φ(4, z)
X4
= (−)k Φ(k, z),
k=0
where



 g(z) if k = 0,




 h(z) if k = 1,

p(z) if k = 2,
Φ(k, z) ≡


 0 if k = 3,




 q(z) if k = 4,

undefined otherwise.
Such unedifying formalism is essentially useless in applications, except
as a vehicle
P for Rdefinition. Once you understand why + and − are operators
just as and are, you can forget the formalism. It doesn’t help much.
7.3.3 Linearity
A function f (z) is linear iff (if and only if) it has the properties
f (z1 + z2 ) = f (z1 ) + f (z2 ),
f (αz) = αf (z),
f (0) = 0.
The functions f (z) = 3z, f (u, v) = 2u − v and f (z) = 0 are examples of
√
linear functions. Nonlinear functions include6 f (z) = z 2 , f (u, v) = uv,
f (t) = cos ωt, f (z) = 3z + 1 and even f (z) = 1.
6
If 3z + 1 is a linear expression, then how is not f (z) = 3z + 1 a linear function?
Answer: it is partly a matter of purposeful definition, partly of semantics. The equation
y = 3x + 1 plots a line, so the expression 3z + 1 is literally “linear” in this sense; but the
definition has more purpose to it than merely this. When you see the linear expression
3z + 1, think 3z + 1 = 0, then g(z) = 3z = −1. The g(z) = 3z is linear; the −1 is the
constant value it targets. That’s the sense of it.
An operator L is linear iff it has the properties
L(f1 + f2 ) = Lf1 + Lf2 ,

L(αf ) = αLf,
L(0) = 0.
P R
The operators , , +, − and ∂ are examples of linear operators. For
instance,7
d df1 df2
[f1 (z) + f2 (z)] = + .
dz dz dz
Nonlinear operators include multiplication, division and the various trigono-
metric functions, among others.
Section 16.1.2 will have more to say about operators and their notation.
7.3.4 Summational and integrodifferential commutivity

Consider the sum  
b q k
X X x
S1 =  .
j!
k=a j=p
This is a sum of the several values of the expression xk /j!, evaluated at every
possible pair (j, k) in the indicated domain. Now consider the sum
q
" b #
X X xk
S2 = .
j!
j=p k=a
This is evidently a sum of the same values, only added in a different order.
Apparently S1 = S2 . Reflection along these lines must soon lead the reader
to the conclusion that, in general,
XX XX
f (j, k) = f (j, k).
k j j k
Now consider that an integral is just a sum of many elements, and that
a derivative is just a difference of two elements. Integrals and derivatives
must then have the same commutative property discrete sums have. For
7
You don’t see d in the list of linear operators? But d in this context is really just
another way of writing ∂, so, yes, d is linear, too. See § 4.4.2.
example,
Z ∞ Z b Z b Z ∞
f (u, v) du dv = f (u, v) dv du;
v=−∞ u=a u=a v=−∞
Z X XZ
fk (v) dv = fk (v) dv;
k k
∂ ∂f
Z Z
f du = du.
∂v ∂v
In general,
Lv Lu f (u, v) = Lu Lv f (u, v), (7.3)
P R
where L is any of the linear operators , or ∂.
Some convergent summations, like
∞ X
1
X (−)j
,
2k + j + 1
k=0 j=0
diverge once reordered, as

∞
1 X
X (−)j
.
2k + j + 1
j=0 k=0
One cannot blithely swap operators here. This is not because swapping is
wrong, but rather because the inner sum after the swap diverges, hence the
outer sum after the swap has no concrete summand on which to work. (Why
does the inner sum after the swap diverge? Answer: 1 + 1/3 + 1/5 + · · · =
[1] + [1/3 + 1/5] + [1/7 + 1/9 + 1/0xB + 1/0xD] + · · · > 1[1/4] + 2[1/8] +
4[1/0x10] + · · · = 1/4 + 1/4 + 1/4 + · · · . See also § 8.10.5.) For a more
twisted example of the same phenomenon, consider8

1 1 1 1 1 1 1 1
1 − + − + ··· = 1 − − + − − + ··· ,
2 3 4 2 4 3 6 8
which associates two negative terms with each positive, but still seems to
omit no term. Paradoxically, then,

1 1 1 1 1 1 1
1 − + − + ··· = − + − + ···
2 3 4 2 4 6 8
1 1 1 1
= − + − + ···
2 4 6 8
1 1 1 1
= 1 − + − + ··· ,
2 2 3 4
8
[1, § 1.2.3]
or so it would seem, but cannot be, for it claims falsely that the sum is half
itself. A better way to have handled the example might have been to write
the series as

1 1 1 1 1
lim 1 − + − + · · · + −
n→∞ 2 3 4 2n − 1 2n
in the first place, thus explicitly specifying equal numbers of positive and
negative terms.9 So specifying would have prevented the error.
The conditional convergence 10 of the last paragraph, which can occur in
integrals as well as in sums, seldom poses much of a dilemma in practice.
One can normally swap summational and integrodifferential operators with
little worry. The reader however should at least be aware that conditional
convergence troubles can arise where a summand or integrand varies in sign
or phase.
7.3.5 Multiple integrals

Consider the function
u2
.
f (u, v) =
v
Such a function would not be plotted as a curved line in a plane, but rather
as a curved surface in a three-dimensional space. Integrating the function
seeks not the area under the curve but rather the volume under the surface:
Z u2 Z v2 2
u
V = dv du.
u1 v1 v
This is a double integral. Inasmuch as it can be written in the form
Z u2
V = g(u) du,
u1
v2
u2
Z
g(u) ≡ dv,
v1 v
9
Some students of professional mathematics would assert that the false conclusion was
reached through lack of rigor. Well, maybe. This writer however does not feel sure that
rigor is quite the right word for what was lacking here. Professional mathematics does
bring an elegant notation and a set of formalisms which serve ably to spotlight certain lim-
ited kinds of blunders, but these are blunders no less by the applied approach. The stalwart
Leonhard Euler—arguably the greatest series-smith in mathematical history—wielded his
heavy analytical hammer in thunderous strokes before professional mathematics had con-
ceived the notation or the formalisms. If the great Euler did without, then you and I
might not always be forbidden to follow his robust example.
On the other hand, the professional approach is worth study if you have the time.
Recommended introductions include [39], preceded if necessary by [27] and/or [1, Ch. 1].
10
[39, § 16]
its effect is to cut the area under the surface into flat, upright slices, then
the slices crosswise into tall, thin towers. The towers are integrated over v
to constitute the slice, then the slices over u to constitute the volume.
In light of § 7.3.4, evidently nothing prevents us from swapping the
integrations: u first, then v. Hence
Z v2 Z u2 2
u
V = du dv.
v1 u1 v
And indeed this makes sense, doesn’t it? What difference does it make
whether we add the towers by rows first then by columns, or by columns
first then by rows? The total volume is the same in any case.
Double integrations arise very frequently in applications. Triple inte-
grations arise about as often. For instance, if µ(r) = µ(x, y, z) represents
the variable mass density of some soil,11 then the total soil mass in some
rectangular volume is
Z x 2 Z y2 Z z 2
M= µ(x, y, z) dz dy dx.
x1 y1 z1
As a concise notational convenience, the last is often written

Z
M= µ(r) dr,
V
where the V stands for “volume” and is understood to imply a triple inte-
gration. Similarly for the double integral,
Z
V = f (ρ) dρ,
S
where the S stands for “surface” and is understood to imply a double inte-
gration.
Even more than three nested integrations are possible. If we integrated
over time as well as space, the integration would be fourfold. A spatial
Fourier transform ([section not yet written]) implies a triple integration; and
its inverse, another triple: a sixfold integration altogether. Manifold nesting
of integrals is thus not just a theoretical mathematical topic; it arises in
sophisticated real-world engineering models. The topic concerns us here for
this reason.
11
Conventionally the Greek letter ρ not µ is used for density, but it happens that we
need the letter ρ for a different purpose later in the paragraph.
7.4. AREAS AND VOLUMES 141
Figure 7.4: The area of a circle.
dφ
ρ
7.4 Areas and volumes

By composing and solving appropriate integrals, one can calculate the peri-
meters, areas and volumes of interesting common shapes and solids.
7.4.1 The area of a circle

Figure 7.4 depicts an element of a circle’s area. The element has wedge
shape, but inasmuch as the wedge is infinitesimally narrow, the wedge is
indistinguishable from a triangle of base length ρ dφ and height ρ. The area
of such a triangle is Atriangle = ρ2 dφ/2. Integrating the many triangles, we
find the circle’s area to be
Z π Z π 2
ρ dφ 2πρ2
Acircle = Atriangle = = . (7.4)
φ=−π −π 2 2
(The numerical value of 2π—the circumference or perimeter of the unit

circle—we have not calculated yet. We shall calculate it in § 8.11.)
7.4.2 The volume of a cone

One can calculate the volume of any cone (or pyramid) if one knows its base
area B and its altitude h measured normal12 to the base. Refer to Fig. 7.5.
12
Normal here means “at right angles.”
Figure 7.5: The volume of a cone.
A cross-section of a cone, cut parallel to the cone’s base, has the same shape
the base has but a different scale. If coordinates are chosen such that the
altitude h runs in the ẑ direction with z = 0 at the cone’s vertex, then
the cross-sectional area is evidently13 (B)(z/h)2 . For this reason, the cone’s
volume is
Z h
B h 2 B h3
z 2
Bh
Z
Vcone = (B) dz = 2 z dz = 2 = . (7.5)
0 h h 0 h 3 3
7.4.3 The surface area and volume of a sphere

Of a sphere, Fig. 7.6, one wants to calculate both the surface area and
the volume. For the surface area, the sphere’s surface is sliced vertically
down the z axis into narrow constant-φ tapered strips (each strip broadest
at the sphere’s equator, tapering to points at the sphere’s ±z poles) and
horizontally across the z axis into narrow constant-θ rings, as in Fig. 7.7. A
surface element so produced (seen as shaded in the latter figure) evidently
has the area
dS = (r dθ)(ρ dφ) = r 2 sin θ dθ dφ.
13
The fact may admittedly not be evident to the reader at first glance. If it is not yet
evident to you, then ponder Fig. 7.5 a moment. Consider what it means to cut parallel
to a cone’s base a cross-section of the cone, and how cross-sections cut nearer a cone’s
vertex are smaller though the same shape. What if the base were square? Would the
cross-sectional area not be (B)(z/h)2 in that case? What if the base were a right triangle
with equal legs—in other words, half a square? What if the base were some other strange
shape like the base depicted in Fig. 7.5? Could such a strange shape not also be regarded
as a definite, well characterized part of a square? (With a pair of scissors one can cut any
shape from a square piece of paper, after all.) Thinking along such lines must soon lead
one to the insight that the parallel-cut cross-sectional area of a cone can be nothing other
than (B)(z/h)2 , regardless of the base’s shape.
7.4. AREAS AND VOLUMES 143
Figure 7.6: A sphere.
ẑ
r
θ z
ŷ
φ
x̂ ρ
Figure 7.7: An element of the sphere’s surface (see Fig. 7.6).
ẑ
r dθ
ŷ
ρ dφ
The sphere’s total surface area then is the sum of all such elements over the
sphere’s entire surface:
Z π Z π
Ssphere = dS
φ=−π θ=0
Z π Z π
= r 2 sin θ dθ dφ
φ=−π θ=0
Z π
2
= r [− cos θ]π0 dφ
φ=−π
Z π
= r2 [2] dφ
φ=−π
2
= 4πr , (7.6)
where we have used the fact from Table 7.1 that sin τ = (d/dτ )(− cos τ ).
Having computed the sphere’s surface area, one can find its volume just
as § 7.4.1 has found a circle’s area—except that instead of dividing the circle
into many narrow triangles, one divides the sphere into many narrow cones,
each cone with base area dS and altitude r, with the vertices of all the cones
meeting at the sphere’s center. Per (7.5), the volume of one such cone is
Vcone = r dS/3. Hence,
r dS r r
I I I
Vsphere = Vcone = = dS = Ssphere ,
S S 3 3 S 3
where the useful symbol

I
indicates integration over a closed surface. In light of (7.6), the total volume
is
4πr 3
Vsphere = . (7.7)
3
(One can compute the same spherical volume more prosaically, without R ref-
erence to cones, by writing dV = r 2 sin θ dr dθ dφ then integrating V dV .
The derivation given above, however, is preferred because it lends the addi-
tional insight that a sphere can sometimes be viewed as a great cone rolled
up about its own vertex. The circular area derivation of § 7.4.1 lends an
analogous insight: that a circle can sometimes be viewed as a great triangle
rolled up about its own vertex.)
7.5. CHECKING AN INTEGRATION 145
7.5 Checking an integration

Dividing 0x46B/0xD = 0x57 with a pencil, how does one check the result?14
Answer: by multiplying (0x57)(0xD) = 0x46B. Multiplication inverts divi-
sion. Easier than division, multiplication provides a quick, reliable check.
Likewise, integrating
b
τ2 b3 − a3
Z
dτ =
a 2 6
with a pencil, how does one check the result? Answer: by differentiating
3
∂ b − a3 τ2

= .
∂b 6 b=τ 2
Differentiation inverts integration. Easier than integration, differentiation

like multiplication provides a quick, reliable check.
More formally, according to (7.2),
b
df
Z
S≡ dτ = f (b) − f (a). (7.8)
a dτ
Differentiating (7.8) with respect to b and a,

∂S df
= ,
∂b b=τ
dτ
(7.9)
∂S df
=− .
∂a a=τ dτ
Either line of (7.9) can be used to check an integration. Evaluating (7.8) at

b = a yields
S|b=a = 0, (7.10)
which can be used to check further.15
As useful as (7.9) and (7.10) are, they nevertheless serve only inte-
grals with variable limits. They are of little use to check definite integrals
14
Admittedly, few readers will ever have done much such multidigit hexadecimal arith-
metic with a pencil, but, hey, go with it. In decimal, it’s 1131/13 = 87.
Actually, hexadecimal is just proxy for binary (see Appendix A), and long division in
straight binary is kind of fun. If you have never tried it, you might. It is simpler than
decimal or hexadecimal division, and it’s how computers divide. The insight gained is
worth the trial.
15
Using (7.10) to check the example, (b3 − a3 )/6|b=a = 0.
like (9.14) below, which lack variable limits to differentiate. However, many
or most integrals one meets in practice have or can be given variable limits.
Equations (7.9) and (7.10) do serve such indefinite integrals.
It is a rare irony of mathematics that, although numerically differenti-
ation is indeed harder than integration, analytically precisely the opposite
is true. Analytically, differentiation is the easier. So far the book has in-
troduced only easy integrals, but Ch. 9 will bring much harder ones. Even
experienced mathematicians are apt to err in analyzing these. Reversing an
integration by taking an easy derivative is thus an excellent way to check a
hard-earned integration result.
7.6 Contour integration

To this point we have considered only integrations in which the variable
of integration
R b advances in a straight line from one point to another: for
instance, a f (τ ) dτ , in which the function f (τ ) is evaluated at τ = a, a +
dτ, a + 2dτ, . . . , b. The integration variable is a real-valued scalar which can
do nothing but make a straight line from a to b.
Such is not the case when the integration variable is a vector. Consider
the integral
Z ŷρ
S= (x2 + y 2 ) dℓ,
r=x̂ρ
where dℓ is the infinitesimal length of a step along the path of integration.

What does this integral mean? Does it mean to integrate from r = x̂ρ to
r = 0, then from there to r = ŷρ? Or does it mean to integrate along the
arc of Fig. 7.8? The two paths of integration begin and end at the same
points, but they differ in between, and the integral certainly does not come
out the same both ways. Yet many other paths of integration from x̂ρ to ŷρ
are possible, not just these two.
Because multiple paths are possible, we must be more specific:
Z
S = (x2 + y 2 ) dℓ,
C
where C stands for “contour” and means in this example the specific contour
of Fig. 7.8. In the example, x2 + y 2 = ρ2 (by the Pythagorean theorem) and
dℓ = ρ dφ, so
Z 2π/4
2π 3
Z
2
S= ρ dℓ = ρ3 dφ = ρ .
C 0 4
7.7. DISCONTINUITIES 147
Figure 7.8: A contour of integration.
ρ
φ
x
In the example the contour is open, but closed contours which begin and
end at the same point are also possible, indeed common. The useful symbol
I
indicates integration over a closed contour. It means that the contour ends
where it began: the loop is closed. The contour of Fig. 7.8 would be closed,
for instance, if it continued to r = 0 and then back to r = x̂ρ.
Besides applying where the variable of integration is a vector, contour
integration applies equally where the variable of integration is a complex
scalar. In the latter case some interesting mathematics emerge, as we shall
see in §§ 8.8 and 9.5.
7.7 Discontinuities
The polynomials and trigonometrics studied to this point in the book of-
fer flexible means to model many physical phenomena of interest, but one
thing they do not model gracefully is the simple discontinuity. Consider a
mechanical valve opened at time t = to . The flow x(t) past the valve is
(
0, t < to ;
x(t) =
xo , t > t o .
Figure 7.9: The Heaviside unit step u(t).
u(t)
Figure 7.10: The Dirac delta δ(t).
δ(t)
One can write this more concisely in the form
x(t) = u(t − to )xo ,
where u(t) is the Heaviside unit step,

(
0, t < 0;
u(t) ≡ (7.11)
1, t > 0;
plotted in Fig. 7.9.

The derivative of the Heaviside unit step is the curious Dirac delta
d
δ(t) ≡ u(t), (7.12)
dt
plotted in Fig. 7.10. This function is zero everywhere except at t = 0, where
it is infinite, with the property that
Z ∞
δ(t) dt = 1, (7.13)
−∞
7.7. DISCONTINUITIES 149
and the interesting consequence that

Z ∞
δ(t − to )f (t) dt = f (to ) (7.14)
−∞
for any function f (t). (Equation 7.14 is the sifting property of the Dirac
delta.)16
The Dirac delta is defined for vectors, too, such that
Z
δ(r) dr = 1. (7.15)
V
16
It seems inadvisable for the narrative to digress at this point to explore u(z) and δ(z),
the unit step and delta of a complex argument, although by means of Fourier analysis
(Ch. 18) or by conceiving the Dirac delta as an infinitely narrow Gaussian pulse ([chapter
or section not yet written]) it could perhaps do so. The book has more pressing topics to
treat. For the book’s present purpose the interesting action of the two functions is with
respect to the real argument t.
In the author’s country at least, a sort of debate seems to have run for decades between
professional and applied mathematicians over the Dirac delta δ(t). Some professional
mathematicians seem to have objected that δ(t) is not a function, inasmuch as it lacks
certain properties common to functions as they define them [48, § 2.4][17]. From the
applied point of view the objection is admittedly a little hard to understand, until one
realizes that it is more a dispute over methods and definitions than over facts. What the
professionals seem to be saying is that δ(t) does not fit as neatly as they would like into
the abstract mathematical framework they had established for functions in general before
Paul Dirac came along in 1930 [67, “Paul Dirac,” 05:48, 25 May 2006] and slapped his
disruptive δ(t) down on the table. The objection is not so much that δ(t) is not allowed
as it is that professional mathematics for years after 1930 lacked a fully coherent theory
for it.
It’s a little like the six-fingered man in Goldman’s The Princess Bride [26]. If I had
established a definition of “nobleman” which subsumed “human,” whose relevant traits
in my definition included five fingers on each hand, when the six-fingered Count Rugen
appeared on the scene, then you would expect me to adapt my definition, wouldn’t you?
By my preëxisting definition, strictly speaking, the six-fingered count is “not a nobleman”;
but such exclusion really tells one more about flaws in the definition than it does about
the count.
Whether the professional mathematician’s definition of the function is flawed, of course,
is not for this writer to judge. Even if not, however, the fact of the Dirac delta dispute,
coupled with the difficulty we applied mathematicians experience in trying to understand
the reason the dispute even exists, has unfortunately surrounded the Dirac delta with a
kind of mysterious aura, an elusive sense that δ(t) hides subtle mysteries—when what it
really hides is an internal discussion of words and means among the professionals. The
professionals who had established the theoretical framework before 1930 justifiably felt
reluctant to throw the whole framework away because some scientists and engineers like
us came along one day with a useful new function which didn’t quite fit, but that was
the professionals’ problem not ours. To us the Dirac delta δ(t) is just a function. The
internal discussion of words and means, we leave to the professionals, who know whereof
they speak.
7.8 Remarks (and exercises)

The concept of the integral is relatively simple once grasped, but its im-
plications are broad, deep and hard. This chapter is short. One reason
introductory calculus texts run so long is that they include many, many
pages of integration examples and exercises. The reader who desires a gen-
tler introduction to the integral might consult among others the textbook
the chapter’s introduction has recommended.
Even if this book is not an instructional textbook, it seems not meet
that it should include no exercises at all here. Here are a few. Some of them
do need material from later chapters, so you should not expect to be able to
complete them all now. The harder ones are marked with ∗ asterisks. Work
the exercises if you like.
Rx Rx
1. Evaluate (a) 0 τ dτ ; (b) 0 τ 2 dτ . (Answer: x2 /2; x3 /3.)
Rx Rx Rx Rx
2. Evaluate (a) 1 (1/τ 2 )Rdτ ; (b) a 3τ −2 dτ ; (c) a Cτ n dτ ; (d) 0
x
(a2 τ 2 + a1 τ ) dτ ; ∗ (e) 1 (1/τ ) dτ .
R x P∞ k P∞ R x k R x P∞
3. ∗ Evaluate (a) 0 k=0 τ dτ ; (b) k=0 0 τ dτ ; (c) 0
k
k=0 (τ /k!)
dτ .
Rx
4. Evaluate 0 exp ατ dτ .
R5 R −2
5. Evaluate (a) −2 (3τ 2 − 2τ 3 ) dτ ; (b) 5 (3τ 2 − 2τ 3 ) dτ . Work the
exercise by hand in hexadecimal and give the answer in hexadecimal.
R∞
6. Evaluate 1 (3/τ 2 ) dτ .
7. ∗ Evaluate the integral of the example of § 7.6 along the alternate con-
tour suggested there, from x̂ρ to 0 to ŷρ.
Rx Rx Rx
8. Evaluate (a) 0 cos ωτ dτ ; (b) 0 sin ωτ dτ ; ∗ (c)17 0 τ sin ωτ dτ .
Rx√ Ra √ √
9. ∗ Evaluate18 (a) 1 1 + 2τ dτ ; (b) x [(cos τ )/ τ ] dτ.
Rx Rx
10. ∗ Evaluate19 (a) 0 [1/(1 + τ 2 )] dτ (answer: arctan x); (b) 0 [(4 +
√
i3)/ 2 − 3τ 2 ] dτ (hint: the answer involves another inverse trigono-
metric).
Rx R∞
11. ∗∗ Evaluate (a) −∞ exp[−τ 2 /2] dτ ; (b) −∞ exp[−τ 2 /2] dτ .
17
[55, § 8-2]
18
[55, § 5-6]
19
[55, back endpaper]
7.8. REMARKS (AND EXERCISES) 151
The last exercise in particular requires some experience to answer. Moreover,

it requires a developed sense of applied mathematical style to put the answer
in a pleasing form (the right form for part b is very different from that for
part a). Some of the easier exercises, of course, you should be able to work
right now.
The point of the exercises is to illustrate how hard integrals can be to
solve, and in fact how easy it is to come up with an integral which no one
really knows how to solve very well. Some solutions to the same integral
are better than others (easier to manipulate, faster to numerically calculate,
etc.) yet not even the masters can solve them all in practical ways. On the
other hand, integrals which arise in practice often can be solved very well
with sufficient cleverness—and the more cleverness you develop, the more
such integrals you can solve. The ways to solve them are myriad. The
mathematical art of solving diverse integrals is well worth cultivating.
Chapter 9 introduces some of the basic, most broadly useful integral-
solving techniques. Before addressing techniques of integration, however, as
promised earlier we turn our attention in Chapter 8 back to the derivative,
applied in the form of the Taylor series.
Chapter 8
The Taylor series
The Taylor series is a power series which fits a function in a limited domain
neighborhood. Fitting a function in such a way brings two advantages:
• it lets us take derivatives and integrals in the same straightforward

way (4.20) we take them with any power series; and
• it implies a simple procedure to calculate the function numerically.
This chapter introduces the Taylor series and some of its incidents. It also
derives Cauchy’s integral formula. The chapter’s early sections prepare the
ground for the treatment of the Taylor series proper in § 8.3.1
1
Because even at the applied level the proper derivation of the Taylor series involves
mathematical induction, analytic continuation and the matter of convergence domains,
no balance of rigor the chapter might strike seems wholly satisfactory. The chapter errs
maybe toward too much rigor; for, with a little less, most of §§ 8.1, 8.2, 8.4 and 8.6 would
cease to be necessary. For the impatient, to read only the following sections might not
be an unreasonable way to shorten the chapter: §§ 8.3, 8.5, 8.8, 8.9 and 8.11, plus the
introduction of § 8.1.
From another point of view, the chapter errs maybe toward too little rigor. Some pretty
constructs of pure mathematics serve the Taylor series and Cauchy’s integral formula.
However, such constructs drive the applied mathematician on too long a detour. The
chapter as written represents the most nearly satisfactory compromise the writer has been
able to attain.
153
154 CHAPTER 8. THE TAYLOR SERIES
8.1 The power-series expansion of 1/(1 − z)n+1

Before approaching the Taylor series proper in § 8.3, we shall find it both
interesting and useful to demonstrate that
∞
1 X n+k
= z k , n ≥ 0. (8.1)
(1 − z)n+1 n
k=0
The demonstration comes in three stages. Of the three, it is the second

stage (§ 8.1.2) which actually proves (8.1). The first stage (§ 8.1.1) comes
up with the formula for the second stage to prove. The third stage (§ 8.1.3)
establishes the sum’s convergence. In all the section,
i, j, k, m, n, K ∈ Z.
8.1.1 The formula

In § 2.6.4 we found that
∞
1 X
= zk = 1 + z + z2 + z3 + · · ·
1−z
k=0
for |z| < 1. What about 1/(1 − z)2 , 1/(1 − z)3 , 1/(1 − z)4 , and so on? By the
long-division procedure of Table 2.4, one can calculate the first few terms of
1/(1 − z)2 to be
1 1
2
= = 1 + 2z + 3z 2 + 4z 3 + · · ·
(1 − z) 1 − 2z + z 2
whose coefficients 1, 2, 3, 4, . . . happen to be the numbers down the first

diagonal of Pascal’s triangle (Fig. 4.2 on page 75; see also Fig. 4.1). Dividing
1/(1 − z)3 seems to produce the coefficients 1, 3, 6, 0xA, . . . down the second
diagonal; dividing 1/(1 − z)4 , the coefficients down the third. A curious
pattern seems to emerge, worth investigating more closely. The pattern
recommends the conjecture (8.1).
To motivate the conjecture a bit more formally (though without actually
proving it yet), suppose that 1/(1− z)n+1 , n ≥ 0, is expandable in the power
series
∞
1 X
= ank z k , (8.2)
(1 − z)n+1
k=0
8.1. THE POWER-SERIES EXPANSION OF 1/(1 − Z)N +1 155
where the ank are coefficients to be determined. Multiplying by 1 − z, we

have that
∞
1 X
= [ank − an(k−1) ]z k .
(1 − z)n
k=0
This is to say that
a(n−1)k = ank − an(k−1) ,
an(k−1) + a(n−1)k = ank . (8.3)
Thinking of Pascal’s triangle, (8.3) reminds one of (4.5), transcribed here in

the symbols
m−1 m−1 m
+ = , (8.4)
j−1 j j
except that (8.3) is not a(m−1)(j−1) + a(m−1)j = amj .
Various changes of variable are possible to make (8.4) better match (8.3).
We might try at first a few false ones, but eventually the change
n + k ← m,
k ← j,
recommends itself. Thus changing in (8.4) gives

n+k−1 n+k−1 n+k
+ = .
k−1 k k
Transforming according to the rule (4.3), this is

n + [k − 1] [n − 1] + k n+k
+ = , (8.5)
n n−1 n
which fits (8.3) perfectly. Hence we conjecture that

n+k
ank = , (8.6)
n
which coefficients, applied to (8.2), yield (8.1).

Equation (8.1) is thus suggestive. It works at least for the important
case of n = 0; this much is easy to test. In light of (8.3), it seems to imply
a relationship between the 1/(1 − z)n+1 series and the 1/(1 − z)n series for
any n. But to seem is not to be. At this point, all we can say is that (8.1)
seems right. We shall establish that it is right in the next subsection.
8.1.2 The proof by induction

Equation (8.1) is proved by induction as follows. Consider the sum
∞
X n+k
Sn ≡ zk . (8.7)
n
k=0
Multiplying by 1 − z yields
∞
X n+k n + [k − 1]
(1 − z)Sn = − zk .
n n
k=0
Per (8.5), this is

∞
X [n − 1] + k
(1 − z)Sn = zk . (8.8)
n−1
k=0
Now suppose that (8.1) is true for n = i − 1 (where i denotes an integer

rather than the imaginary unit):
∞
1 X [i − 1] + k
= zk . (8.9)
(1 − z)i i−1
k=0
In light of (8.8), this means that

1
= (1 − z)Si .
(1 − z)i
Dividing by 1 − z,
1
= Si .
(1 − z)i+1
Applying (8.7),
∞
1 X i+k
= zk . (8.10)
(1 − z)i+1 i
k=0
Evidently (8.9) implies (8.10). In other words, if (8.1) is true for n = i − 1,
then it is also true for n = i. Thus by induction, if it is true for any one n,
then it is also true for all greater n.
The “if” in the last sentence is important. Like all inductions, this one
needs at least one start case to be valid (many inductions actually need a
consecutive pair of start cases). The n = 0 supplies the start case
∞ ∞
1 X k k
X
= z = zk ,
(1 − z)0+1 0
k=0 k=0
which per (2.34) we know to be true.

8.1. THE POWER-SERIES EXPANSION OF 1/(1 − Z)N +1 157
8.1.3 Convergence
The question remains as to the domain over which the sum (8.1) converges.2
To answer the question, consider that per (4.9),

m m m−1
= , m > 0.
j m−j j
With the substitution n + k ← m, n ← j, this means that

n+k n+k n + [k − 1]
= ,
n k n
or more tersely,
n+k
ank = an(k−1) ,
k
where
n+k
ank ≡
n
are the coefficients of the power series (8.1). Rearranging factors,
ank n+k n
= =1+ . (8.11)
an(k−1) k k
2
The meaning of the verb to converge may seem clear enough from the context and
from earlier references, but if explanation here helps: a series converges if and only if it
approaches a specific, finite value after many terms. A more rigorous way of saying the
same thing is as follows: the series
X∞
S= τk
k=0
converges iff (if and only if), for all possible positive constants ǫ, there exists a finite
K ≥ −1 such that ˛ n ˛
˛ X ˛
τk ˛ < ǫ,
˛ ˛
˛
˛ ˛
k=K+1
for all n ≥ K (of course it is also required that the τk be finite, but you knew that already).
The professional mathematical literature calls such convergence “uniform convergence,”
distinguishing it through a test devised by Weierstrass from the weaker “pointwise con-
vergence” [1, § 1.5]. The applied mathematician can profit substantially by learning the
professional view in the matter, but the effect of trying to teach the professional view in a
book like this would not be pleasing. Here, we avoid error by keeping a clear view of the
physical phenomena the mathematics is meant to model.
It is interesting nevertheless to consider an example of an integral for which convergence
is not so simple, such as Frullani’s integral of § 9.7.
Multiplying (8.11) by z k /z k−1 gives the ratio
ank z k n
= 1 + z,
an(k−1) z k−1 k
which is to say that the kth term of (8.1) is (1 + n/k)z times the (k − 1)th
term. So long as the criterion3
n
1+ z ≤ 1 − δ

k
is satisfied for all sufficiently large k > K—where 0 < δ ≪ 1 is a small posi-
tive constant—then the series evidently converges (see § 2.6.4 and eqn. 3.22).
But we can bind 1+n/k as close to unity as desired by making K sufficiently
large, so to meet the criterion it suffices that
|z| < 1. (8.12)
The bound (8.12) thus establishes a sure convergence domain for (8.1).
8.1.4 General remarks on mathematical induction

We have proven (8.1) by means of a mathematical induction. The virtue
of induction as practiced in § 8.1.2 is that it makes a logically clean, air-
tight case for a formula. Its vice is that it conceals the subjective process
which has led the mathematician to consider the formula in the first place.
Once you obtain a formula somehow, maybe you can prove it by induction;
but the induction probably does not help you to obtain the formula! A
good inductive proof usually begins by motivating the formula proven, as in
§ 8.1.1.
Richard W. Hamming once said of mathematical induction,
The theoretical difficulty the student has with mathematical in-

duction arises from the reluctance to ask seriously, “How could
I prove a formula for an infinite number of cases when I know
that testing a finite number of cases is not enough?” Once you
3
Although one need not ask the question to understand the proof, the reader may
nevertheless wonder why the simpler |(1 P+ n/k)z| < 1 is not given as a criterion. The
surprising answer is that
P not all series τk with |τk /τk−1 | < 1 converge! ForPexample,
the extremely simple 1/k does not converge. As we see however, all series τk with
|τk /τk−1 | < 1 − δ do converge. The distinction is P
subtle but rather important.
The really
R x curious reader may now ask why 1/k does not converge. Answer: it
majorizes 1 (1/τ ) dτ = ln x. See (5.7) and § 8.10.
8.2. SHIFTING A POWER SERIES’ EXPANSION POINT 159
really face this question, you will understand the ideas behind
mathematical induction. It is only when you grasp the problem
clearly that the method becomes clear. [27, § 2.3]
Hamming also wrote,
The function of rigor is mainly critical and is seldom construc-
tive. Rigor is the hygiene of mathematics, which is needed to
protect us against careless thinking. [27, § 1.6]
The applied mathematician may tend to avoid rigor for which he finds no
immediate use, but he does not disdain mathematical rigor on principle.
The style lies in exercising rigor at the right level for the problem at hand.
Hamming, a professional mathematician who sympathized with the applied
mathematician’s needs, wrote further,
Ideally, when teaching a topic the degree of rigor should follow
the student’s perceived need for it. . . . It is necessary to require
a gradually rising level of rigor so that when faced with a real
need for it you are not left helpless. As a result, [one cannot
teach] a uniform level of rigor, but rather a gradually rising level.
Logically, this is indefensible, but psychologically there is little
else that can be done. [27, § 1.6]
Applied mathematics holds that the practice is defensible, on the ground
that the math serves the model; but Hamming nevertheless makes a perti-
nent point.
Mathematical induction is a broadly applicable technique for construct-
ing mathematical proofs. We shall not always write inductions out as ex-
plicitly in this book as we have done in the present section—often we shall
leave the induction as an implicit exercise for the interested reader—but this
section’s example at least lays out the general pattern of the technique.
8.2 Shifting a power series’ expansion point

One more question we should treat before approaching the Taylor series
proper in § 8.3 concerns the shifting of a power series’ expansion point.
How can the expansion point of the power series
∞
X
f (z) = (ak )(z − zo )k , (8.13)
k=K
(k, K) ∈ Z, K ≤ 0,
which may have terms of negative order, be shifted from z = zo to z = z1 ?

The first step in answering the question is straightforward: one rewrites
(8.13) in the form
∞
X
f (z) = (ak )([z − z1 ] − [zo − z1 ])k ,
k=K
then changes the variables

z − z1
w← ,
zo − z1 (8.14)
ck ← [−(zo − z1 )]k ak ,
to obtain
∞
X
f (z) = (ck )(1 − w)k . (8.15)
k=K
Splitting the k < 0 terms from the k ≥ 0 terms in (8.15), we have that
f (z) = f− (z) + f+ (z), (8.16)

−(K+1)
X c[−(k+1)]
f− (z) ≡ ,
(1 − w)k+1
k=0
∞
X
f+ (z) ≡ (ck )(1 − w)k .
k=0
Of the two subseries, the f− (z) is expanded term by term using (8.1), after
which combining like powers of w yields the form
∞
X
f− (z) = qk w k ,
k=0
−(K+1) (8.17)
X n+k
qk ≡ (c[−(n+1)] ) .
n
n=0
The f+ (z) is even simpler to expand: one need only multiply the series out
term by term per (4.12), combining like powers of w to reach the form
∞
X
f+ (z) = pk w k ,
k=0
∞ (8.18)
X n
pk ≡ (cn ) .
k
n=k
8.3. EXPANDING FUNCTIONS IN TAYLOR SERIES 161
Equations (8.13) through (8.18) serve to shift a power series’ expansion

point, calculating the coefficients of a power series for f (z) about z = z1 ,
given those of a power series about z = zo . Notice that—unlike the original,
z = zo power series—the new, z = z1 power series has terms (z − z1 )k only
for k ≥ 0; it has no terms of negative order. At the price per (8.12) of
restricting the convergence domain to |w| < 1, shifting the expansion point
away from the pole at z = zo has resolved the k < 0 terms.
The method fails if z = z1 happens to be a pole or other nonanalytic
point of f (z). The convergence domain vanishes as z1 approaches such
a forbidden point. (Examples of such forbidden points include z = 0 in
√
h[z] = 1/z and in g[z] = z. See §§ 8.4 through 8.8.) Furthermore, even
if z1 does represent a fully analytic point of f (z), it also must lie within
the convergence domain of the original, z = zo series for the shift to be
trustworthy as derived.
The attentive reader might observe that we have formally established
the convergence neither of f− (z) in (8.17) nor of f+ (z) in (8.18). Regarding
the former convergence, that of f− (z), we have strategically framed the
problem so that one needn’t worry about it, running the sum in (8.13)
from the finite k = K ≤ 0 rather than from the infinite k = −∞; and
since according to (8.12) each term of the original f− (z) of (8.16) converges
for |w| < 1, the reconstituted f− (z) of (8.17) safely converges in the same
domain. The latter convergence, that of f+ (z), is harder to establish in the
abstract because that subseries has an infinite number of terms. As we shall
see by pursuing a different line of argument in § 8.3, however, the f+ (z)
of (8.18) can be nothing other than the Taylor series about z = z1 of the
function f+ (z) in any event, enjoying the same convergence domain any such
Taylor series enjoys.4
8.3 Expanding functions in Taylor series

Having prepared the ground, we now stand in position to treat the Taylor
series proper. The treatment begins with a question: if you had to express
4
A rigorous argument can be constructed without appeal to § 8.3 if desired, from the
ratio n/(n − k) of (4.9) and its brethren, which ratio approaches unity with increasing n.
A more elegant rigorous argument can be made indirectly by way of a complex contour
integral. In applied mathematics, however, one does not normally try to shift the ex-
pansion point of an unspecified function f (z), anyway. Rather, one shifts the expansion
point of some concrete function like sin z or ln(1 − z). The imagined difficulty (if any)
vanishes in the concrete case. Appealing to § 8.3, the important point is the one made in
the narrative: f+ (z) can be nothing other than the Taylor series in any event.
some function f (z) by a power series

∞
X
f (z) = (ak )(z − zo )k ,
k=0
with terms of nonnegative order k ≥ 0 only, how would you do it? The
procedure of § 8.1 worked well enough in the case of f (z) = 1/(1−z)n+1 , but
it is not immediately obvious that the same procedure works more generally.
What if f (z) = sin z, for example?5
Fortunately a different way to attack the power-series expansion problem
is known. It works by asking the question: what power series, having terms
of nonnegative order only, most resembles f (z) in the immediate neighbor-
hood of z = zo ? To resemble f (z), the desired power series should have
a0 = f (zo ); otherwise it would not have the right value at z = zo . Then it
should have a1 = f ′ (zo ) for the right slope. Then, a2 = f ′′ (zo )/2 for the
right second derivative, and so on. With this procedure,
∞
!
dk f (z − zo )k
X
f (z) = . (8.19)
dz k z=zo k!
k=0
Equation (8.19) is the Taylor series. Where it converges, it has all the same
derivatives f (z) has, so if f (z) is infinitely differentiable then the Taylor
series is an exact representation of the function.6
5
The actual Taylor series for sin z is given in § 8.9.
6
Further proof details may be too tiresome to inflict on applied mathematical readers.
However, for readers who want a little more rigor nevertheless, the argument goes briefly
as follows. Consider an infinitely differentiable function F (z) and its Taylor series f (z)
about zo . Let ∆F (z) ≡ F (z) − f (z) be the part of F (z) not representable as a Taylor
series about zo .
If ∆F (z) is the part of F (z) not representable as a Taylor series, then ∆F (zo ) and
all its derivatives at zo must be identically zero (otherwise by the Taylor series formula
of eqn. 8.19, one could construct a nonzero Taylor series for ∆F [zo ] from the nonzero
derivatives). However, if F (z) is infinitely differentiable and if all the derivatives of ∆F (z)
are zero at z = zo , then by the unbalanced definition of the derivative from § 4.4, all the
derivatives must also be zero at z = zo ± ǫ, hence also at z = zo ± 2ǫ, and so on. This
means that ∆F (z) = 0. In other words, there is no part of F (z) not representable as a
Taylor series.
A more formal way to make the same argument would be to suppose that
dn ∆F/dz n |z=zo +ǫ = h for some integer n ≥ 0; whereas that this would mean that
dn+1 ∆F/dz n+1 ˛z=z = h/ǫ; but that, inasmuch as the latter is one of the derivatives of
˛
o
∆F (z) at z = zo , it follows that h = 0.
The interested reader can fill the details in, but basically that is how the more rigorous
proof goes. (A more elegant rigorous proof, preferred by the professional mathemati-
8.4. ANALYTIC CONTINUATION 163
The Taylor series is not guaranteed to converge outside some neighbor-

hood near z = zo , but where it does converge it is precise.
When zo = 0, the series is also called the Maclaurin series. By either
name, the series is a construct of great importance and tremendous practical
value, as we shall soon see.
8.4 Analytic continuation

As earlier mentioned in § 2.12.3, an analytic function is a function which is
infinitely differentiable in the domain neighborhood of interest—or, maybe
more appropriately for our applied purpose, a function expressible as a Tay-
lor series in that neighborhood. As we have seen, only one Taylor series
about zo is possible for a given function f (z):
∞
X
f (z) = (ak )(z − zo )k .
k=0
However, nothing prevents one from transposing the series to a different

expansion point z = z1 by the method of § 8.2, except that the transposed
series may there enjoy a different convergence domain. As it happens, this
section’s purpose finds it convenient to swap symbols zo ↔ z1 , transposing
rather from expansion about z = z1 to expansion about z = zo . In the
swapped notation, so long as the expansion point z = zo lies fully within
(neither outside nor right on the edge of) the z = z1 series’ convergence
domain, the two series evidently describe the selfsame underlying analytic
function.
cians [40][3][20] but needing significant theoretical preparation, involves integrating over a
complex contour about the expansion point. Appendix C sketches that proof.) The reason
the rigorous proof is confined to a footnote is not a deprecation of rigor as such. It is a
deprecation of rigor which serves little purpose in applications. Applied mathematicians
normally regard mathematical functions to be imprecise analogs of physical quantities of
interest. Since the functions are imprecise analogs in any case, the applied mathematician
is logically free implicitly to define the functions he uses as Taylor series in the first place;
that is, to restrict the set of infinitely differentiable functions used in the model to the
subset of such functions representable as Taylor series. With such an implicit definition,
whether there actually exist any infinitely differentiable functions not representable as
Taylor series is more or less beside the point.
In applied mathematics, the definitions serve the model, not the other way around.
(It is entertaining to consider [66, “Extremum”] the Taylor series of the function
sin[1/x]—although in practice this particular function is readily expanded after the obvi-
ous change of variable u ← 1/x.)
Since an analytic function f (z) is infinitely differentiable and enjoys a

unique Taylor expansion fo (z −zo ) = f (z) about each point zo in its domain,
it follows that if two Taylor series f1 (z − z1 ) and f2 (z − z2 ) find even a small
neighborhood |z − zo | < ǫ which lies in the domain of both, then the two can
both be transposed to the common z = zo expansion point. If the two are
found to have the same Taylor series there, then f1 and f2 both represent
the same function. Moreover, if a series f3 is found whose domain overlaps
that of f2 , then a series f4 whose domain overlaps that of f3 , and so on,
and if each pair in the chain matches at least in a small neighborhood in
its region of overlap, then the whole chain of overlapping series necessarily
represents the same underlying analytic function f (z). The series f1 and
the series fn represent the same analytic function even if their domains do
not directly overlap at all.
This is a manifestation of the principle of analytic continuation. The
principle holds that if two analytic functions are the same within some do-
main neighborhood |z − zo | < ǫ, then they are the same everywhere.7 Ob-
serve however that the principle fails at poles and other nonanalytic points,
because the function is not differentiable there.
The result of § 8.2, which shows general power series to be expressible as
Taylor series except at their poles and other nonanalytic points, extends the
analytic continuation principle to cover power series in general, including
power series with terms of negative order.
Now, observe: though all convergent power series are indeed analytic,
one need not actually expand every analytic function in a power series.
Sums, products and ratios of analytic functions are no less differentiable
than the functions themselves—as also, by the derivative chain rule, is an
analytic function of analytic functions. For example, where g(z) and h(z) are
analytic, there also is f (z) ≡ g(z)/h(z) analytic (except perhaps at isolated
points where h[z] = 0). Besides, given Taylor series for g(z) and h(z) one
can make a power series for f (z) by long division if desired, so that is all
right. Section 8.15 speaks further on the point.
The subject of analyticity is rightly a matter of deep concern to the
professional mathematician. It is also a long wedge which drives pure and
applied mathematics apart. When the professional mathematician speaks
7
The writer hesitates to mention that he is given to understand [57] that the domain
neighborhood can technically be reduced to a domain contour of nonzero length but zero
width. Having never met a significant application of this extension of the principle, the
writer has neither researched the extension’s proof nor asserted its truth. He does not
especially recommend that the reader worry over the point. The domain neighborhood
|z − zo | < ǫ suffices.
8.5. BRANCH POINTS 165
generally of a “function,” he means any function at all. One can construct

some pretty unreasonable functions if one wants to, such as
f ([2k + 1]2m ) ≡ (−)m , (k, m) ∈ Z;

f (z) ≡ 0 otherwise.
However, neither functions like this f (z) nor more subtly unreasonable func-
tions normally arise in the modeling of physical phenomena. When such
functions do arise, one transforms, approximates, reduces, replaces and/or
avoids them. The full theory which classifies and encompasses—or explicitly
excludes—such functions is thus of limited interest to the applied mathe-
matician, and this book does not cover it.8
This does not mean that the scientist or engineer never encounters non-
analytic functions. On the contrary, he encounters several, but they are not
subtle: |z|; arg z; z ∗ ; ℜ(z); ℑ(z); u(t); δ(t). Refer to §§ 2.12 and 7.7. Such
functions are nonanalytic either because they lack proper derivatives in the
Argand plane according to (4.19) or because one has defined them only over
a real domain.
8.5 Branch points

√
The function g(z) = z is an interesting, troublesome function. Its deriv-
√
ative is dg/dz = 1/2 z, so even though the function is finite at z = 0,
its derivative is not finite there. Evidently g(z) has a nonanalytic point at
z = 0, yet the point is not a pole. What is it?
We call it a branch point. The defining characteristic of the branch point
is that, given a function f (z) with such a point at z = zo , if one encircles9 the
point once alone (that is, without also encircling some other branch point) by
a closed contour in the Argand domain plane, while simultaneously tracking
f (z) in the Argand range plane—and if one demands that z and f (z) move
8
Many books do cover it in varying degrees, including [20][57][31] and numerous others.
The foundations of the pure theory of a complex variable, though abstract, are beautiful,
and though they do not comfortably fit a book like this even an applied mathematician can
profit substantially by studying them. The few pages of Appendix C trace only the pure
theory’s main thread. However that may be, the pure theory is probably best appreciated
after one already understands its chief conclusions. Though not for the explicit purpose
of serving the pure theory, the present chapter does develop just such an understanding.
9
For readers whose native language is not English, “to encircle” means “to surround”
or “to enclose.” The verb does not require the boundary to have the shape of an ac-
tual, geometrical circle; any closed shape suffices. However, the circle is a typical shape,
probably the most fitting shape to imagine when thinking of the concept abstractly.
smoothly, that neither suddenly skip from one spot to another—then one
finds that f (z) ends in a different place than it began, even though z itself
has returned precisely to its own starting point. The range contour remains
open even though the domain contour is closed.
In complex analysis, a branch point may be thought of informally

as a point zo at which a “multiple-valued function” changes val-
ues when one winds once around zo . [67, “Branch point,” 18:10,
16 May 2006]
√
An analytic function like g(z) = z having a branch point evidently
is not single-valued. It is multiple-valued. For a single z more than one
distinct g(z) is possible.
An analytic function like h(z) = 1/z, by contrast, is single-valued even
though it has a pole. This function does not suffer the syndrome described.
When a domain contour encircles a pole, the corresponding range contour
is properly closed. Poles do not cause their functions to be multiple-valued
and thus are not branch points.
Evidently f (z) ≡ (z − zo )a has a branch point at z = zo if and only if a
is not an integer. If f (z) does have a branch point—if a is not an integer—
then the mathematician must draw a distinction between z1 = zo + ρeiφ
and z2 = zo + ρei(φ+2π) , even though the two are exactly the same number.
Indeed z1 = z2 , but paradoxically f (z1 ) 6= f (z2 ).
This is difficult. It is confusing, too, until one realizes that the fact
of a branch point says nothing whatsoever about the argument z. As far
as z is concerned, there really is no distinction between z1 = zo + ρeiφ and
z2 = zo + ρei(φ+2π) —none at all. What draws the distinction is the multiple-
valued function f (z) which uses the argument.
It is as though I had a mad colleague who called me Thaddeus Black,
until one day I happened to walk past behind his desk (rather than in front
as I usually did), whereupon for some reason he began calling me Gorbag
Pfufnik. I had not changed at all, but now the colleague calls me by a
different name. The change isn’t really in me, is it? It’s in my colleague,
who seems to suffer a branch point. If it is important to me to be sure that
my colleague really is addressing me when he cries, “Pfufnik!” then I had
better keep a running count of how many times I have turned about his
desk, hadn’t I, even though the number of turns is personally of no import
to me.
The usual analysis strategy when one encounters a branch point is simply
to avoid the point. Where an integral follows a closed contour as in § 8.8,
8.6. ENTIRE AND MEROMORPHIC FUNCTIONS 167
the strategy is to compose the contour to exclude the branch point, to shut
it out. Such a strategy of avoidance usually prospers.10
8.6 Entire and meromorphic functions

Though an applied mathematician is unwise to let abstract definitions en-
thrall his thinking, pure mathematics nevertheless brings some technical
definitions the applied mathematician can use. Two such are the definitions
of entire and meromorphic functions.11
A function f (z) which is analytic for all finite z is an entire function.
Examples include f (z) = z 2 and f (z) = exp z, but not f (z) = 1/z which
has a pole at z = 0.
A function f (z) which is analytic for all finite z except at isolated poles
(which can be n-fold poles if n is a finite, positive integer), which has no
branch points, of which no circle of finite radius in the Argand domain
plane encompasses an infinity of poles, is a meromorphic function. Examples
include f (z) = 1/z, f (z) = 1/(z + 2) + 1/(z − 1)3 + 2z 2 and f (z) = tan z—
the last of which has an infinite number of poles, but of which the poles
nowhere cluster in infinite numbers. The function f (z) = tan(1/z) is not
meromorphic since it has an infinite number of poles within the Argand unit
circle. Even the function f (z) = exp(1/z) is not meromorphic: it has only
the one, isolated nonanalytic point at z = 0, and that point is no branch
point; but the point is an essential singularity, having the character of an
infinitifold (∞-fold) pole.12
If it seems unclear that the singularities of tan z are actual poles, inci-
dentally, then consider that
sin z cos w
tan z = =− ,
cos z sin w
wherein we have changed the variable
2π
w ← z − (2n + 1) , n ∈ Z.
4
10
Traditionally associated with branch points in complex variable theory are the notions
of branch cuts and Riemann sheets. These ideas are interesting, but are not central to the
analysis as developed in this book and are not covered here. The interested reader might
consult a book on complex variables or advanced calculus like [31], among many others.
11
[66]
12
[40]
Section 8.9 and its Table 8.1, below, give Taylor series for cos z and sin z,
with which
−1 + w2 /2 − w4 /0x18 − · · ·
tan z = .
w − w3 /6 + w5 /0x78 − · · ·
By long division,
1 w/3 − w3 /0x1E + · · ·
tan z = − + .
w 1 − w2 /6 + w4 /0x78 − · · ·
(On the other hand, if it is unclear that z = [2n + 1][2π/4] are the only
singularities tan z has—that it has no singularities of which ℑ[z] 6= 0—then
consider that the singularities of tan z occur where cos z = 0, which by
Euler’s formula, eqn. 5.17, occurs where exp[+iz] = exp[−iz]. This in turn
is possible only if |exp[+iz]| = |exp[−iz]|, which happens only for real z.)
Sections 8.14, 8.15 and 9.6 speak further of the matter.
8.7 Extrema over a complex domain

If a function f (z) is expanded by (8.19) or by other means about an analytic
expansion point z = zo such that
∞
X
f (z) = f (zo ) + (ak )(z − zo )k ;
k=1
and if
ak = 0 for k < K, but

aK 6= 0,
(k, K) ∈ Z, 0 < K < ∞,
such that aK is the series’ first nonzero coefficient; then, in the immediate
neighborhood of the expansion point,
f (z) ≈ f (zo ) + (aK )(z − zo )K , |z − zo | ≪ 1.

′
Changing ρ′ eiφ ← z − zo , this is
′
f (z) ≈ f (zo ) + aK ρ′K eiKφ , 0 ≤ ρ′ ≪ 1. (8.20)
Evidently one can shift the output of an analytic function f (z) slightly in
any desired Argand direction by shifting slightly the function’s input z.
8.8. CAUCHY’S INTEGRAL FORMULA 169
Specifically according to (8.20), to shift f (z) by ∆f ≈ ǫeiψ , one can shift z

by ∆z ≈ (ǫ/aK )1/K ei(ψ+n2π)/K , n ∈ Z. Except at a nonanalytic point of
f (z) or in the trivial case that f (z) were everywhere constant, this always
works—even where [df /dz]z=zo = 0.
That one can shift an analytic function’s output smoothly in any Ar-
gand direction whatsoever has the significant consequence that neither the
real nor the imaginary part of the function—nor for that matter any lin-
ear combination ℜ[e−iω f (z)] of the real and imaginary parts—can have an
extremum within the interior of a domain over which the function is fully an-
alytic. That is, a function’s extrema over a bounded analytic domain never
lie within the domain’s interior but always on its boundary.13,14
8.8 Cauchy’s integral formula
In § 7.6 we considered the problem of vector contour integration, in which the

sum value of an integration depends not only on the integration’s endpoints
but also on the path, or contour, over which the integration is done, as in
Fig. 7.8. Because real scalars are confined to a single line, no alternate choice
of path is possible where the variable of integration is a real scalar, so the
contour problem does not arise in that case. It does however arise where
the variable of integration is a complex scalar, because there again different
paths are possible. Refer to the Argand plane of Fig. 2.5.
Consider the integral
Z z2
Sn = z n−1 dz, n ∈ Z. (8.21)
z1
If z were always a real number, then by the antiderivative (§ 7.2) this inte-
gral would evaluate to (z2n − z1n )/n; or, in the case of n = 0, to ln(z2 /z1 ).
Inasmuch as z is complex, however, the correct evaluation is less obvious.
To evaluate the integral sensibly in the latter case, one must consider some
specific path of integration in the Argand plane. One must also consider the
meaning of the symbol dz.
13
Professional mathematicians tend to define the domain and its boundary more care-
fully.
14
[62][40]
8.8.1 The meaning of the symbol dz
The symbol dz represents an infinitesimal step in some direction in the

Argand plane:
dz = [z + dz] − [z]
h i h i
= (ρ + dρ)ei(φ+dφ) − ρeiφ
h i h i
= (ρ + dρ)ei dφ eiφ − ρeiφ
h i h i
= (ρ + dρ)(1 + i dφ)eiφ − ρeiφ .
Since the product of two infinitesimals is negligible even on infinitesimal

scale, we can drop the dρ dφ term.15 After canceling finite terms, we are left
with the peculiar but fine formula
dz = (dρ + iρ dφ)eiφ . (8.22)
8.8.2 Integrating along the contour
Now consider the integration (8.21) along the contour of Fig. 8.1. Integrat-
15
The dropping of second-order infinitesimals like dρ dφ, added to first order infinites-
imals like dρ, is a standard calculus technique. One cannot always drop them, however.
Occasionally one encounters a sum in which not only do the finite terms cancel, but also
the first-order infinitesimals. In such a case, the second-order infinitesimals dominate and
cannot be dropped. An example of the type is
(1 − ǫ)3 + 3(1 + ǫ) − 4 (1 − 3ǫ + 3ǫ2 ) + (3 + 3ǫ) − 4

lim 2
= lim = 3.
ǫ→0 ǫ ǫ→0 ǫ2
One typically notices that such a case has arisen when the dropping of second-order
infinitesimals has left an ambiguous 0/0. To fix the problem, you simply go back to the
step where you dropped the infinitesimal and you restore it, then you proceed from there.
Otherwise there isn’t much point in carrying second-order infinitesimals around. In the
relatively uncommon event that you need them, you’ll know it. The math itself will tell
you.
Figure 8.1: A contour of integration in the Argand plane, in two segments:

constant-ρ (za to zb ); and constant-φ (zb to zc ).
y = ℑ(z)
zc
zb
ρ
φ
za
x = ℜ(z)
ing along the constant-φ segment,
Z zc Z ρc
n−1
z dz = (ρeiφ )n−1 (dρ + iρ dφ)eiφ
zb ρ
Z bρc
= (ρeiφ )n−1 (dρ)eiφ
ρb
Z ρc
inφ
= e ρn−1 dρ
ρb
einφ n
= (ρc − ρnb )
n
zcn − zbn
= .
n
Integrating along the constant-ρ arc,

Z zb Z φb
n−1
z dz = (ρeiφ )n−1 (dρ + iρ dφ)eiφ
za φa
Z φb
= (ρeiφ )n−1 (iρ dφ)eiφ
φa
Z φb
n
= iρ einφ dφ
φa
iρn inφb
= e − einφa
in
zbn − zan
= .
n
Adding the two, we have that
Z zc
zcn − zan
z n−1 dz = ,
za n
surprisingly the same as for real z. Since any path of integration between
any two complex numbers z1 and z2 is approximated arbitrarily closely by a
succession of short constant-ρ and constant-φ segments, it follows generally
that Z z2
z n − z1n
z n−1 dz = 2 , n ∈ Z, n 6= 0. (8.23)
z1 n
The applied mathematician might reasonably ask, “Was (8.23) really
worth the trouble? We knew that already. It’s the same as for real numbers.”
Well, we really didn’t know it before deriving it, but the point is well
taken nevertheless. However, notice the exemption of n = 0. Equation (8.23)
does not hold in that case. Consider the n = 0 integral
Z z2
dz
S0 = .
z1 z
Following the same steps as before and using (5.7) and (2.39), we find that
Z ρ2 Z ρ2 Z ρ2
dz (dρ + iρ dφ)eiφ dρ ρ2
= iφ
= = ln . (8.24)
ρ1 z ρ1 ρe ρ1 ρ ρ1
This is always real-valued, but otherwise it brings no surprise. However,

Z φ2 Z φ2 Z φ2
dz (dρ + iρ dφ)eiφ
= =i dφ = i(φ2 − φ1 ). (8.25)
φ1 z φ1 ρeiφ φ1
The odd thing about this is in what happens when the contour closes a
complete loop in the Argand plane about the z = 0 pole. In this case,
φ2 = φ1 + 2π, thus
S0 = i2π,
even though the integration ends where it begins.
Generalizing, we have that
I
(z − zo )n−1 dz = 0, n ∈ Z, n 6= 0;
(8.26)
dz
I
= i2π;
z − zo
H
where as in § 7.6 the symbol represents integration about a closed contour
that ends where it begins, and where it is implied that the contour loops
positively (counterclockwise, in the direction of increasing φ) exactly once
about the z = zo pole.
Notice that the formula’s i2π does not depend on the precise path of
integration, but only on the fact that the path loops once positively about
the pole. Notice also that nothing in the derivation of (8.23) actually requires
that n be an integer, so one can write
Z z2
z a − z1a
z a−1 dz = 2 , a 6= 0. (8.27)
z1 a
However, (8.26) does not hold in the latter case; its integral comes to zero
for nonintegral a only if the contour does not enclose the branch point at
z = zo .
For a closedHcontour which encloses no pole or other nonanalytic point,
(8.27) has that z a−1 dz = 0, or with the change of variable z − zo ← z,
I
(z − zo )a−1 dz = 0.
But because any analytic function can be expanded in the form f (z) =
ak −1 (which is just a Taylor series if the a happen to be
P
k (ck )(z − zo ) k
positive integers), this means that
I
f (z) dz = 0 (8.28)
if f (z) is everywhere analytic within the contour.16

16
The careful reader will observe that (8.28)’s derivation does not explicitly handle
8.8.3 The formula

The combination of (8.26) and (8.28) is powerful. Consider the closed con-
tour integral
f (z)
I
dz,
z − zo
where the contour encloses no nonanalytic point of f (z) itself but does en-
close the pole of f (z)/(z − zo ) at z = zo . If the contour were a tiny circle
of infinitesimal radius about the pole, then the integrand would reduce to
f (zo )/(z − zo ); and then per (8.26),
f (z)
I
dz = i2πf (zo ). (8.29)
z − zo
But if the contour were not an infinitesimal circle but rather the larger
contour of Fig. 8.2? In this case, if the dashed detour which excludes the
pole is taken, then according to (8.28) the resulting integral totals zero;
but the two straight integral segments evidently cancel; and similarly as
we have just reasoned, the reverse-directed integral about the tiny detour
circle is −i2πf (zo ); so to bring the total integral to zero the integral about
the main contour must be i2πf (zo ). Thus, (8.29) holds for any positively-
directed contour which once encloses a pole and no other nonanalytic point,
whether the contour be small or large. Equation (8.29) is Cauchy’s integral
formula.
If the contour encloses multiple poles (§§ 2.11 and 9.6.2), then by the
principle of linear superposition (§ 7.3.3),
I " X fk (z)
#
X
fo (z) + dz = i2π fk (zk ), (8.30)
z − zk
k k
where the fo (z) is a regular part;17 and again, where neither fo (z) nor any
of the several fk (z) has a pole or other nonanalytic point within (or on) the
an f (z) represented by a Taylor series with an infinite number of terms and a finite
convergence domain (for example, f [z] = ln[1 − z]). However, by § 8.2 one can transpose
such a series from zo to an overlapping convergence domain about z1 . Let the contour’s
interior be divided into several cells, each of which is small enough to enjoy a single
convergence domain. Integrate about each cell. Because the cells share boundaries within
the contour’s interior, each interior boundary is integrated twice, once in each direction,
canceling. The original contour—each piece of which is an exterior boundary of some
cell—is integrated once piecewise. This is the basis on which a more rigorous proof is
constructed.
17
[43, § 1.1]
Figure 8.2: A Cauchy contour integral.
ℑ(z)
zo
ℜ(z)
contour. The values fk (zk ), which represent the strengths of the poles, are
called residues. In words, (8.30) says that an integral about a closed contour
in the Argand plane comes to i2π times the sum of the residues of the poles
(if any) thus enclosed. (Note however that eqn. 8.30 does not handle branch
points. If there is a branch point, the contour must exclude it or the formula
will not work.)
As we shall see in § 9.5, whether in the form of (8.29) or of (8.30) Cauchy’s
integral formula is an extremely useful result.18
8.8.4 Enclosing a multiple pole

When a complex contour of integration encloses a double, triple or other
n-fold pole, the integration can be written
f (z)
I
S= dz, m ∈ Z, m ≥ 0,
(z − zo )m+1
where m + 1 = n. Expanding f (z) in a Taylor series (8.19) about z = zo ,

∞
!
dk f

dz
I X
S= .
dz z=zo (k!)(z − zo )m−k+1
k
k=0
18
[31, § 10.6][57][67, “Cauchy’s integral formula,” 14:13, 20 April 2006]
But according to (8.26), only the k = m term contributes, so
!
dm f

dz
I
S = m
dz z=zo (m!)(z − zo )

!I
dm f

1 dz
= m
m! dz z=zo (z − zo )
!
i2π dm f

= ,
m! dz m z=zo
where the integral is evaluated in the last step according to (8.29). Alto-
gether,
!
dm f

f (z) i2π
I
dz = , m ∈ Z, m ≥ 0. (8.31)
(z − zo )m+1 m! dz m z=zo
Equation (8.31) evaluates a contour integral about an n-fold pole as (8.29)

does about a single pole. (When m = 0, the two equations are the same.)19
8.9 Taylor series for specific functions
With the general Taylor series formula (8.19), the derivatives of Tables 5.2
and 5.3, and the observation from (4.21) that
d(z a )
= az a−1 ,
dz
19
[40][57]
8.9. TAYLOR SERIES FOR SPECIFIC FUNCTIONS 177
one can calculate Taylor series for many functions. For instance, expanding
about z = 1,
ln z|z=1 = ln z|z=1 = 0,

d 1
ln z = = 1,
dz z z=1
z=1
d2

−1
ln z = = −1,
dz 2 z 2 z=1
z=1
d3

2
ln z = = 2,
dz 3 z=1 z3 z=1
..
.
dk −(−)k (k − 1)!

= −(−)k (k − 1)!, k > 0.

ln z
=
dz k z=1 z k
z=1
With these derivatives, the Taylor series about z = 1 is

∞ h i (z − 1)k ∞
X
k
X (1 − z)k
ln z = −(−) (k − 1)! =− ,
k! k
k=1 k=1
evidently convergent for |1 − z| < 1. (And if z lies outside the convergence

domain? Several strategies are then possible. One can expand the Taylor
series about a different point; but cleverer and easier is to take advantage
of some convenient relationship like ln w = − ln[1/w]. Section 8.10.4 elab-
orates.) Using such Taylor series, one can relatively efficiently calculate
actual numerical values for ln z and many other functions.
Table 8.1 lists Taylor series for a few functions of interest. All the se-
ries converge for |z| < 1. The exp z, sin z and cos z series converge for all
complex z. Among the several series, the series for arctan z is computed
indirectly20 by way of Table 5.3 and (2.33):
Z z
1
arctan z = 2
dw
0 1+w
Z zX ∞
= (−)k w2k dw
0 k=0
∞
X (−)k z 2k+1
= .
2k + 1
k=0
20
[55, § 11-7]
Table 8.1: Taylor series.
∞
! k
dk f

X Y z − zo
f (z) = k
dz z=zo
j
k=0 j=1
∞ Y
k
X a
(1 + z)a−1 = −1 z
j
k=0 j=1
∞ Y k ∞
X z X zk
exp z = =
j k!
k=0 j=1 k=0
 
∞ k 2
X Y −z
sin z = z 
(2j)(2j + 1)
k=0 j=1
k
∞ Y
X −z 2
cos z =
(2j − 1)(2j)
k=0 j=1
 
∞ k
X Y z2
sinh z = z 
(2j)(2j + 1)
k=0 j=1
∞ Y
k
X z2
cosh z =
(2j − 1)(2j)
k=0 j=1
∞ k ∞
X 1Y X zk
− ln(1 − z) = z=
k k
k=1 j=1 k=1
 
∞ k ∞
X 1  Y X (−)k z 2k+1
arctan z = z (−z 2 ) =
2k + 1 2k + 1
k=0 j=1 k=0
8.10. ERROR BOUNDS 179
It is interesting to observe from Table 8.1 the useful first-order approxi-

mations that
lim exp z = 1 + z,
z→0
(8.32)
lim sin z = z,
z→0
among others.
8.10 Error bounds

One naturally cannot actually sum a Taylor series to an infinite number of
terms. One must add some finite number of terms, then quit—which raises
the question: how many terms are enough? How can one know that one has
added adequately many terms; that the remaining terms, which constitute
the tail of the series, are sufficiently insignificant? How can one set error
bounds on the truncated sum?
8.10.1 Examples
Some series alternate sign. For these it is easy if the numbers involved
happen to be real. For example, from Table 8.1,

3 1 1 1 1 1
ln = ln 1 + = − + − + ···
2 2 (1)(2 ) (2)(2 ) (3)(2 ) (4)(24 )
1 2 3
Each term is smaller in magnitude than the last, so the true value of ln(3/2)
necessarily lies between the sum of the series to n terms and the sum to
n + 1 terms. The last and next partial sums bound the result. Up to but
not including the fourth-order term, for instance,
1 3
S4 − < ln < S4 ,
(4)(24 ) 2
1 1 1
S4 = − + .
(1)(2 ) (2)(2 ) (3)(23 )
1 2
Other series however do not alternate sign. For example,

1 1
ln 2 = − ln = − ln 1 − = S 5 + R5 ,
2 2
1 1 1 1
S5 = + + + ,
(1)(2 ) (2)(2 ) (3)(2 ) (4)(24 )
1 2 3
1 1
R5 = + + ···
(5)(2 ) (6)(26 )
5
The basic technique in such a case is to find a replacement series (or inte-
gral) Rn′ which one can collapse analytically, each of whose terms equals or
exceeds in magnitude the corresponding term of Rn . For the example, one
might choose
∞
′ 1X 1 2
R5 = k
= ,
5 2 (5)(25 )
k=5
wherein (2.34) had been used to collapse the summation. Then,
S5 < ln 2 < S5 + R5′ .
For real 0 ≤ x < 1 generally,
Sn < − ln(1 − x) < Sn + Rn′ ,
n−1
X xk
Sn ≡ ,
k
k=1
∞
X xk xn
Rn′ ≡ = .
n (n)(1 − x)
k=n
Many variations and refinements are possible, some of which we shall

meet in the rest of the section, but that is the basic technique: to add several
terms of the series to establish a lower bound, then to overestimate the
remainder of the series to establish an upper bound. The overestimate Rn′
majorizes the series’ true remainder Rn . Notice that the Rn′ in the example
is a fairly small number, and that it would have been a lot smaller yet had we
included a few more terms in Sn (for instance, n = 0x40 would have bound
ln 2 tighter than the limit of a computer’s typical double-type floating-point
accuracy). The technique usually works well in practice for this reason.
8.10.2 Majorization
To majorize in mathematics is to be, or to replace by virtue of being, ev-
erywhere at least as great as. This is best explained by example. Consider
the summation
∞
X 1 1 1 1
S= = 1 + 2 + 2 + 2 + ···
k2 2 3 4
k=1
The exact value this summation totals to is unknown to us, but the sum-
mation does rather resemble the integral (refer to Table 7.1)
1 ∞
Z ∞
dx
I= = − = 1.
1 x2 x 1
Figure 8.3: Majorization. The area I between the dashed curve and the x
axis majorizes the area S − 1 between the stairstep curve and the x axis,
because the height of the dashed curve is everywhere at least as great as
that of the stairstep curve.
1/22 y = 1/x2
1/32
1/42
x
1 2 3
Figure 8.3 plots S and I together as areas—or more precisely, plots S − 1

and I together as areas (the summation’s first term is omitted). As the plot
shows, the unknown area S − 1 cannot possibly be as great as the known
area I. In symbols, S − 1 < I = 1; or,
S < 2.
The integral I majorizes the summation S − 1, thus guaranteeing the ab-
solute upper limit on S. (Of course S < 2 is a very loose limit, but that
isn’t the point of the example. In practical calculation, one would let a com-
puter add many terms of the series first numerically, and only then majorize
the remainder. Even so, cleverer ways to majorize the remainder of this
particular series will occur to the reader, such as in representing the terms
graphically—not as flat-topped rectangles—but as slant-topped trapezoids,
shifted in the figure a half unit rightward.)
Majorization serves surely to bound an unknown quantity by a larger,
known quantity. Reflecting, minorization 21 serves surely to bound an un-
known quantity by a smaller, known quantity. The quantities in question
21
The author does not remember ever encountering the word minorization heretofore
in print, but as a reflection of majorization the word seems logical. This book at least will
use the word where needed. You can use it too if you like.
are often integrals and/or series summations, the two of which are akin as
Fig. 8.3 illustrates. The choice of whether to majorize a particular unknown
quantity by an integral or by a series summation depends on the convenience
of the problem at hand.
The series S of this subsection is interesting, incidentally. It is a har-
monic series rather than a power series, because although its terms do
decrease in magnitude it has no z k factor (or seen from another point of
view, it does have a z k factor, but z = 1), and the ratio of adjacent terms’
magnitudes approaches unity as k grows. Harmonic series can be hard to
sum accurately, but clever majorization can help (and if majorization does
not help enough, the series transformations of Ch. [not yet written] can help
even more).
8.10.3 Geometric majorization
Harmonic series can be hard to sum as § 8.10.2 has observed, but more
common than harmonic series are true power series, easier to sum in that
they include a z k factor in each term. There is no one, ideal bound that
works equally well for all power series. However, the point of establishing
a bound is not to sum a power series exactly but rather to fence the sum
within some sufficiently (rather than optimally) small neighborhood. A
simple, general bound which works quite adequately for most power series
encountered in practice, including among many others all the Taylor series
of Table 8.1, is the geometric majorization
|τn |
|ǫn | < . (8.33)
1 − |ρn |
Here, τn represents the power series’ nth-order term (in Table 8.1’s series for
exp z, for example, τn = z n /[n!]). The |ρn | is a positive real number chosen,
preferably as small as possible, such that

τk+1
τ ≤ |ρn | for all k ≥ n, (8.34)

k

τk+1
τk < |ρn | for at least one k ≥ n,

0 < |ρn | < 1; (8.35)

which is to say, more or less, such that each term in the series’ tail is smaller
than the last by at least a factor of |ρn |. Given these definitions, if22
n−1
X
Sn ≡ τk ,
k=0
(8.36)
ǫn ≡ S ∞ − S n ,
where S∞ represents the true, exact (but uncalculatable, unknown) infinite

series sum, then (2.34) and (3.22) imply the geometric majorization (8.33).
If the last paragraph seems abstract, a pair of concrete examples should
serve to clarify. First, if the Taylor series
∞
X zk
− ln(1 − z) =
k
k=1
of Table 8.1 is truncated before the nth-order term, then

n−1
X zk
− ln(1 − z) ≈ ,
k
k=1
|z n | /n
|ǫn | < ,
1 − |z|
where ǫn is the error in the truncated sum.23 Here, |τk+1 /τk | = [k/(k +
1)] |z| < |z| for all k ≥ n > 0, so we have chosen |ρn | = |z|.
Second, if the Taylor series
∞ Y
k ∞
X z X zk
exp z = =
j k!
k=0 j=1 k=0
also of Table 8.1 is truncated before the nth-order term, and if we choose to
stipulate that
n + 1 > |z| ,
22
Some scientists and engineers—as, for example, the authors of [47] and even this
writer in earlier years—prefer to define ǫn ≡ Sn − S∞ , oppositely as we define it here.
This choice is a matter of taste. Professional mathematicians—as, for example, the author
of [63]—seem to tend toward the ǫn ≡ S∞ − Sn of (8.36).
23
This particular error bound fails for n = 0, but that is no flaw. There is no reason
to use the error bound for n = 0 when, merely by taking one or two more terms into the
truncated sum, one can quite conveniently let n = 1 or n = 2.
then
n−1 k n−1
XY z X zk
exp z ≈ = ,
j k!
k=0 j=1 k=0
n
|z | /n!
|ǫn | < .
1 − |z| /(n + 1)
Here, |τk+1 /τk | = |z| /(k + 1), whose maximum value for all k ≥ n occurs
when k = n, so we have chosen |ρn | = |z| /(n + 1).
8.10.4 Calculation outside the fast convergence domain

Used directly, the Taylor series of Table 8.1 tend to converge slowly for some
values of z and not at all for others. The series for − ln(1 − z) and (1 + z)a−1
for instance each converge for |z| < 1 (though slowly for |z| ≈ 1); whereas
each series diverges when asked to compute a quantity like − ln 3 or 3a−1
directly. To shift the series’ expansion points per § 8.2 is one way to seek
convergence, but for nonentire functions (§ 8.6) like these a more probably
profitable strategy is to find and exploit some property of the functions to
transform their arguments, such as
1
− ln γ = ln ,
γ
1
γ a−1 = ,
(1/γ)a−1
which leave the respective Taylor series to compute quantities like − ln(1/3)
and (1/3)a−1 they can handle.
Let f (1 + ζ) be a function whose Taylor series about ζ = 0 converges for
|ζ| < 1 and which obeys properties of the forms24

1
f (γ) = g f ,
γ (8.37)
f (αγ) = h [f (α), f (γ)] ,
where g[·] and h[·, ·] are functions we know how to compute like g[·] = −[·]
24
This paragraph’s notation is necessarily abstract. To make it seem more concrete,
consider that the function f (1 + ζ) = − ln(1 − z) has ζ = −z, f (γ) = g[f (1/γ)] = −f (1/γ)
and f (αγ) = h[f (α), f (γ)] = f (α) + f (γ); and that the function f (1 + ζ) = (1 + z)a−1 has
ζ = z, f (γ) = g[f (1/γ)] = 1/f (1/γ) and f (αγ) = h[f (α), f (γ)] = f (α)f (γ).
or g[·] = 1/[·]; and like h[·, ·] = [·] + [·] or h[·, ·] = [·][·]. Identifying
1
= 1 + ζ,
γ
1
γ= , (8.38)
1+ζ
1−γ
= ζ,
γ
we have that
1−γ
f (γ) = g f 1 + , (8.39)
γ
whose convergence domain |ζ| < 1 is |1 − γ| / |γ| < 1, which is |γ − 1| < |γ|
or in other words
1
ℜ(γ) > .
2
Although the transformation from ζ to γ has not lifted the convergence
limit altogether, we see that it has apparently opened the limit to a broader
domain.
Though this writer knows no way to lift the convergence limit altogether
that does not cause more problems than it solves, one can take advantage of
the h[·, ·] property of (8.37) to sidestep the limit, computing f (ω) indirectly
for any ω 6= 0 by any of several tactics. One nonoptimal but entirely effective
tactic is represented by the equations
ω ≡ in 2m γ,
|ℑ(γ)| ≤ ℜ(γ),
(8.40)
1 ≤ ℜ(γ) < 2,
m, n ∈ Z,
whereupon the formula
f (ω) = h[f (in 2m ), f (γ)] (8.41)
calculates f (ω) fast for any ω 6= 0—provided only that we have other means
to compute f (in 2m ), which not infrequently we do.25
25
Equation (8.41) admittedly leaves open the question of how to compute f (in 2m ),
but at least for the functions this subsection has used as examples this is not hard.
For the logarithm, − ln(in 2m ) = m ln(1/2) − in(2π/4). For the power, (in 2m )a−1 =
cis[(n2π/4)(a − 1)]/[(1/2)a−1 ]m . The sine and cosine in the cis function are each calcu-
lated directly by Taylor series, as are the numbers ln(1/2) and (1/2)a−1 . The number 2π,
we have not calculated yet, but shall in § 8.11.
Notice how (8.40) fences γ within a comfortable zone, keeping γ moder-

ately small in magnitude but never too near the ℜ(γ) = 1/2 frontier in the
Argand plane. In theory all finite γ rightward of the frontier let the Taylor
series converge, but extreme γ of any kind let the series converge only slowly
(and due to compound floating-point rounding error inaccurately) inasmuch
as they imply that |ζ| ≈ 1. Besides allowing all ω 6= 0, the tactic (8.40) also
thus significantly speeds series convergence.
The method and tactic of (8.37) through (8.41) are useful in themselves
and also illustrative generally. Of course, most nonentire functions lack
properties of the specific kinds that (8.37) demands, but such functions may
have other properties one can analogously exploit.26
8.10.5 Nonconvergent series

Variants of this section’s techniques can be used to prove that a series does
not converge at all. For example,
∞
X 1
k
k=1
does not converge because
k+1
1 dτ
Z
> ;
k k τ
hence,
∞ ∞ Z k+1 ∞
1 dτ dτ
X X Z
> = = ln ∞.
k τ τ
k=1 k=1 k 1
26
To draw another example from Table 8.1, consider that
arctan ω = α + arctan ζ,
ω cos α − sin α
ζ≡ ,
ω sin α + cos α
where arctan ω is interpreted as the geometrical angle the vector x̂ + ŷω makes with x̂.
Axes are rotated per (3.7) through some angle α to reduce the tangent from ω to ζ, thus
causing the Taylor series to converge faster or indeed to converge at all.
Any number of further examples and tactics of the kind will occur to the creative reader,
shrinking a function’s argument by some convenient means before feeding the argument
to the function’s Taylor series.
8.10.6 Remarks
The study of error bounds is not a matter of rules and formulas so much as of
ideas, suggestions and tactics. There is normally no such thing as an optimal
error bound—with sufficient cleverness, some tighter bound can usually be
discovered—but often easier and more effective than cleverness is simply
to add a few extra terms into the series before truncating it (that is, to
increase n a little). To eliminate the error entirely usually demands adding
an infinite number of terms, which is impossible; but since eliminating the
error entirely also requires recording the sum to infinite precision, which
is impossible anyway, eliminating the error entirely is not normally a goal
one seeks. To eliminate the error to the 0x34-bit (sixteen-decimal place)
precision of a computer’s double-type floating-point representation typically
requires something like 0x34 terms—if the series be wisely composed and if
care be taken to keep z moderately small and reasonably distant from the
edge of the series’ convergence domain. Besides, few engineering applications
really use much more than 0x10 bits (five decimal places) in any case. Perfect
precision is impossible, but adequate precision is usually not hard to achieve.
Occasionally nonetheless a series arises for which even adequate precision
is quite hard to achieve. An infamous example is
∞
X (−)k 1 1 1
S=− √ = 1 − √ + √ − √ + ··· ,
k=1
k 2 3 4
which obviously converges, but sum it if you can! It is not easy to do.
Before closing the section, we ought to arrest one potential agent of
terminological confusion. The “error” in a series summation’s error bounds
is unrelated to the error of probability theory. The English word “error” is
thus overloaded here. A series sum converges to a definite value, and to the
same value every time the series is summed; no chance is involved. It is just
that we do not necessarily know exactly what that value is. What we can
do, by this section’s techniques or perhaps by other methods, is to establish
a definite neighborhood in which the unknown value is sure to lie; and we
can make that neighborhood as tight as we want, merely by including a
sufficient number of terms in the sum.
The topic of series error bounds is what G.S. Brown refers to as “trick-
based.”27 There is no general answer to the error-bound problem, but there
are several techniques which help, some of which this section has introduced.
Other techniques, we shall meet later in the book as the need for them arises.
27
[10]
8.11 Calculating 2π
The Taylor series for arctan z in Table 8.1 implies a neat way of calculating
the constant 2π. We already know that tan 2π/8 = 1, or in other words that
2π
arctan 1 = .
8
Applying the Taylor series, we have that
∞
X (−)k
2π = 8 . (8.42)
2k + 1
k=0
The series (8.42) is simple but converges extremely slowly. Much faster con-
vergence is given by angles smaller than 2π/8. For example, from Table 3.2,
√
3−1 2π
arctan √ = .
3+1 0x18
Applying the Taylor series at this angle, we have that28

∞ √ !2k+1
X (−)k 3−1
2π = 0x18 √ ≈ 0x6.487F. (8.43)
2k + 1 3 + 1
k=0
8.12 Odd and even functions

An odd function is one for which f (−z) = −f (z). Any function whose
Taylor series about zo = 0 includes only odd-order terms is an odd function.
Examples of odd functions include z 3 and sin z.
An even function is one for which f (−z) = f (z). Any function whose
Taylor series about zo = 0 includes only even-order terms is an even function.
Examples of even functions include z 2 and cos z.
Odd and even functions are interesting because of the symmetry they
bring—the plot of a real-valued odd function being symmetric about a point,
the plot of a real-valued even function being symmetric about a line. Many
28
The writer is given to understand that clever mathematicians have invented subtle,
still much faster-converging iterative schemes toward 2π. However, there is fast and there is
fast. The relatively straightforward series this section gives converges to the best accuracy
of your computer’s floating-point register within a paltry 0x40 iterations or so—and, after
all, you only need to compute the numerical value of 2π once.
Admittedly, the writer supposes that useful lessons lurk in the clever mathematics
underlying the subtle schemes, but such schemes are not covered here.
8.13. TRIGONOMETRIC POLES 189
functions are neither odd nor even, of course, but one can always split an
analytic function into two components—one odd, the other even—by the
simple expedient of sorting the odd-order terms from the even-order terms
in the function’s Taylor series. For example, exp z = sinh z + cosh z.
8.13 Trigonometric poles

The singularities of the trigonometric functions are single poles of residue ±1
or ±i. For the circular trigonometrics, all the poles lie along the real number
line; for the hyperbolic trigonometrics, along the imaginary. Specifically, of
the eight trigonometric functions
1 1 1
, , , tan z,
sin z cos z tan z
1 1 1
, , , tanh z,
sinh z cosh z tanh z
the poles and their respective residues are

z − kπ
= (−)k ,
sin z z = kπ

z − (k − 1/2)π
= (−)k ,
cos z
z = (k − 1/2)π

z − kπ
= 1,
tan z z = kπ
[z − (k − 1/2)π] tan z|z = (k − 1/2)π = −1,

z − ikπ (8.44)
= (−)k ,
sinh z z = ikπ

z − i(k − 1/2)π
= i(−)k ,
cosh z
z = i(k − 1/2)π

z − ikπ
= 1,
tanh z z = ikπ
[z − i(k − 1/2)π] tanh z|z = i(k − 1/2)π = 1,
k ∈ Z.
To support (8.44)’s claims, we shall marshal the identities of Tables 5.1
and 5.2 plus l’Hôpital’s rule (4.30). Before calculating residues and such,
however, we should like to verify that the poles (8.44) lists are in fact the
only poles that there are; that we have forgotten no poles. Consider for
instance the function 1/ sin z = i2/(eiz − e−iz ). This function evidently goes
infinite only when eiz = e−iz , which is possible only for real z; but for real z,
the sine function’s very definition establishes the poles z = kπ (refer to
Fig. 3.1). With the observations from Table 5.1 that i sinh z = sin iz and
cosh z = cos iz, similar reasoning for each of the eight trigonometrics forbids
poles other than those (8.44) lists. Satisfied that we have forgotten no poles,
therefore, we finally apply l’Hôpital’s rule to each of the ratios
z − kπ z − (k − 1/2)π z − kπ z − (k − 1/2)π
, , , ,
sin z cos z tan z 1/ tan z
z − ikπ z − i(k − 1/2)π z − ikπ z − i(k − 1/2)π
, , ,
sinh z cosh z tanh z 1/ tanh z
to reach (8.44).
Trigonometric poles evidently are special only in that a trigonometric
function has an infinite number of them. The poles are ordinary, single
poles, with residues, subject to Cauchy’s integral formula and so on. The
trigonometrics are meromorphic functions (§ 8.6) for this reason.29
The six simpler trigonometrics sin z, cos z, sinh z, cosh z, exp z and
cis z—conspicuously excluded from this section’s gang of eight—have no
poles for finite z, because ez , eiz , ez ± e−z and eiz ± e−iz then likewise are
finite. These simpler trigonometrics are not only meromorphic but also en-
tire. Observe however that the inverse trigonometrics are multiple-valued
and have branch points, and thus are not meromorphic at all.
8.14 The Laurent series
Any analytic function can be expanded in a Taylor series, but never about
a pole or branch point of the function. Sometimes one nevertheless wants
to expand at least about a pole. Consider for example expanding
e−z
f (z) = (8.45)
1 − cos z
29
[40]
8.14. THE LAURENT SERIES 191
about the function’s pole at z = 0. Expanding dividend and divisor sepa-

rately,
1 − z + z 2 /2 − z 3 /6 + · · ·
f (z) =
z 2 /2 − z 4 /0x18 + · · ·
P∞ j j

j=0 (−) z /j!
=
− ∞ (−)k z 2k /(2k)!
P
P∞ k=1 2k 2k+1 /(2k + 1)!

k=0 −z /(2k)! + z
= P∞ k 2k
.
k=1 (−) z /(2k)!
By long division,
( ∞
2 2 2 2 X (−)k z 2k
f (z) = − + − 2+
z2 z z z (2k)!
k=1
∞ ) ,X ∞
X z 2k z 2k+1 (−)k z 2k
+ − +
(2k)! (2k + 1)! (2k)!
k=0 k=1
(∞
(−)k 2z 2k−2 (−)k 2z 2k−1

2 2 X
= − + − +
z2 z (2k)! (2k)!
k=1
∞ ) ,X ∞
X z 2k z 2k+1 (−)k z 2k
+ − +
(2k)! (2k + 1)! (2k)!
k=0 k=1
(∞
2 2 X (−)k 2z 2k (−)k 2z 2k+1

= 2
− + −
z z (2k + 2)! (2k + 2)!
k=0
∞ ) ,X ∞
X z 2k z 2k+1 (−)k z 2k
+ − +
(2k)! (2k + 1)! (2k)!
k=0 k=1
∞
2 2 X −(2k + 1)(2k + 2) + (−)k 2 2k
= 2
− + z
z z (2k + 2)!
k=0
,X ∞
(2k + 2) − (−)k 2 2k+1 (−)k z 2k
+ z .
(2k + 2)! (2k)!
k=1
The remainder’s k = 0 terms now disappear as intended; so, factoring z 2 /z 2

from the division leaves
∞
2 2 X (2k + 3)(2k + 4) + (−)k 2 2k
f (z) = − + z
z2 z (2k + 4)!
k=0
,X∞
(2k + 4) + (−)k 2 2k+1 (−)k z 2k
− z .
(2k + 4)! (2k + 2)!
k=0
(8.46)
One can continue dividing to extract further terms if desired, and if all the
terms
2 2 7 z
f (z) = 2 − + − + · · ·
z z 6 2
are extracted the result is the Laurent series proper,
∞
X
f (z) = (ak )(z − zo )k , (k, K) ∈ Z, K ≤ 0. (8.47)
k=K
However for many purposes (as in eqn. 8.46) the partial Laurent series
−1 P∞
X
k (bk )(z − zo )k
f (z) = (ak )(z − zo ) + Pk=0
∞ k
, (8.48)
k=K k=0 (ck )(z − zo )
(k, K) ∈ Z, K ≤ 0, c0 6= 0,
suffices and may even be preferable. In either form,

−1
X
f (z) = (ak )(z − zo )k + fo (z − zo ), (k, K) ∈ Z, K ≤ 0, (8.49)
k=K
where, unlike f (z), fo (z − zo ) is analytic at z = zo . The fo (z − zo ) of (8.49)

is f (z)’s regular part at z = zo .
The ordinary Taylor series diverges at a function’s pole. Handling the
pole separately, the Laurent series remedies this defect.30,31
Sections 9.5 and 9.6 tell more about poles generally, including multiple
poles like the one in the example here.
30
The professional mathematician’s treatment of the Laurent series usually begins by
defining an annular convergence domain (a convergence domain bounded without by a
large circle and within by a small) in the Argand plane. From an applied point of view
however what interests us is the basic technique to remove the poles from an otherwise
analytic function. Further rigor is left to the professionals.
31
[31, § 10.8][20, § 2.5]
8.15. TAYLOR SERIES IN 1/Z 193
8.15 Taylor series in 1/z

A little imagination helps the Taylor series a lot. The Laurent series of
§ 8.14 represents one way to extend the Taylor series. Several other ways
are possible. The typical trouble one has with the Taylor series is that
a function’s poles and branch points limit the series’ convergence domain.
Thinking flexibly, however, one can often evade the trouble.
sin(1/z)
f (z) = .
cos z
This function has a nonanalytic point of a most peculiar nature at z = 0.
The point is an essential singularity, and one cannot expand the function
directly about it. One could expand the function directly about some other
point like z = 1, but calculating the Taylor coefficients would take a lot
of effort and, even then, the resulting series would suffer a straitly limited
convergence domain. All that however tries too hard. Depending on the
application, it may suffice to write
sin w 1
f (z) = , w≡ .
cos z z
This is
z −1 − z −3 /3! + z −5 /5! − · · ·
f (z) = ,
1 − z 2 /2! + z 4 /4! − · · ·
which is all one needs to calculate f (z) numerically—and may be all one
needs for analysis, too.
As an example of a different kind, consider
1
g(z) = .
(z − 2)2
Most often, one needs no Taylor series to handle such a function (one sim-
ply does the indicated arithmetic). Suppose however that a Taylor series
specifically about z = 0 were indeed needed for some reason. Then by (8.1)
and (4.2),
∞ k X ∞
1/4 1X 1+k z k+1 k
g(z) = 2
= = z ,
(1 − z/2) 4 1 2 2k+2
k=0 k=0
That expansion is good only for |z| < 2, but for |z| > 2 we also have that
∞ k X ∞
1/z 2 1 X 1+k 2 2k−2 (k − 1)
g(z) = = = ,
(1 − 2/z)2 z2 1 z zk
k=0 k=2
which expands in negative rather than positive powers of z. Note that

we have computed the two series for g(z) without ever actually taking a
derivative.
Neither of the section’s examples is especially interesting in itself, but
their point is that it often pays to think flexibly in extending and applying
the Taylor series. One is not required immediately to take the Taylor series
of a function as it presents itself; one can first change variables or otherwise
rewrite the function in some convenient way, then take the Taylor series
either of the whole function at once or of pieces of it separately. One can
expand in negative powers of z equally validly as in positive powers. And,
though taking derivatives per (8.19) may be the canonical way to determine
Taylor coefficients, any effective means to find the coefficients suffices.
8.16 The multidimensional Taylor series

Equation (8.19) has given the Taylor series for functions of a single variable.
The idea of the Taylor series does not differ where there are two or more
independent variables, only the details are a little more complicated. For
example, consider the function f (z1 , z2 ) = z12 +z1 z2 +2z2 , which has terms z12
and 2z2 —these we understand—but also has the cross-term z1 z2 for which
the relevant derivative is the cross-derivative ∂ 2 f /∂z1 ∂z2 . Where two or
more independent variables are involved, one must account for the cross-
derivatives, too.
With this idea in mind, the multidimensional Taylor series is
!
X ∂kf (z − zo )k

f (z) = . (8.50)
∂zk z=zo k!
k
Well, that’s neat. What does it mean?
• The z is a vector 32 incorporating the several independent variables

z1 , z2 , . . . , zN .
• The k is a nonnegative integer vector of N counters—k1 , k2 , . . . , kN —

one for each of the independent variables. Each of the kn runs indepen-
dently from 0 to ∞, and every permutation is possible. For example,
32
In this generalized sense of the word, a vector is an ordered set of N elements. The
geometrical vector v = x̂x + ŷy + ẑz of § 3.3, then, is a vector with N = 3, v1 = x, v2 = y
and v3 = z. (Generalized vectors of arbitrary N will figure prominently in the book from
Ch. 11 onward.)
8.16. THE MULTIDIMENSIONAL TAYLOR SERIES 195
if N = 2 then
k = (k1 , k2 )
= (0, 0), (0, 1), (0, 2), (0, 3), . . . ;
(1, 0), (1, 1), (1, 2), (1, 3), . . . ;
(2, 0), (2, 1), (2, 2), (2, 3), . . . ;
(3, 0), (3, 1), (3, 2), (3, 3), . . . ;
...
• The ∂ k f /∂zk represents the kth cross-derivative of f (z), meaning that

N
!
∂kf Y ∂ kn
≡ f.
∂zk (∂zn )kn
n=1
• The (z − zo )k represents
N
Y
k
(z − zo ) ≡ (zn − zon )kn .
n=1
• The k! represents
N
Y
k! ≡ kn !.
n=1
With these definitions, the multidimensional Taylor series (8.50) yields all
the right derivatives and cross-derivatives at the expansion point z = zo .
Thus within some convergence domain about z = zo , the multidimensional
Taylor series (8.50) represents a function f (z) as accurately as the simple
Taylor series (8.19) represents a function f (z), and for the same reason.
Chapter 9
Integration techniques
Equation (4.19) implies a general technique for calculating a derivative sym-

bolically. Its counterpart (7.1), unfortunately, implies a general technique
only for calculating an integral numerically—and even for this purpose it is
imperfect; for, when it comes to adding an infinite number of infinitesimal
elements, how is one actually to do the sum?
It turns out that there is no one general answer to this question. Some
functions are best integrated by one technique, some by another. It is hard
to guess in advance which technique might work best.
This chapter surveys several weapons of the intrepid mathematician’s
arsenal against the integral.
9.1 Integration by antiderivative

The simplest way to solve an integral is just to look at it, recognizing its
integrand to be the derivative of something already known:1
z
df
Z
dτ = f (τ )|za . (9.1)
a dτ
For instance,
x
1
Z
dτ = ln τ |x1 = ln x.
1 τ
One merely looks at the integrand 1/τ , recognizing it to be the derivative
of ln τ , then directly writes down the solution ln τ |x1 . Refer to § 7.2.
1
The notation f (τ )|za or [f (τ )]za means f (z) − f (a).
197
198 CHAPTER 9. INTEGRATION TECHNIQUES
The technique by itself is pretty limited. However, the frequent object of

other integration techniques is to transform an integral into a form to which
this basic technique can be applied.
Besides the essential
τa

a−1 d
τ = , (9.2)
dτ a
Tables 7.1, 5.2, 5.3 and 9.1 provide several further good derivatives this
antiderivative technique can use.
One particular, nonobvious, useful variation on the antiderivative tech-
nique seems worth calling out specially here. If z = ρeiφ , then (8.24)
and (8.25) have that
z2
dz ρ2
Z
= ln + i(φ2 − φ1 ). (9.3)
z1 z ρ1
This helps, for example, when z1 and z2 are real but negative numbers.
9.2 Integration by substitution
Consider the integral

x2
x dx
Z
S= .
x1 1 + x2
This integral is not in a form one immediately recognizes. However, with

the change of variable
u ← 1 + x2 ,
whose differential is (by successive steps)
d(u) = d(1 + x2 ),
du = 2x dx,
9.3. INTEGRATION BY PARTS 199
the integral is
x2
x dx
Z
S =
u
Zx=x
x2
1
2x dx
=
x=x1 2u
1+x22
du
Z
=
u=1+x21 2u
1+x2
1 2
= ln u
2 u=1+x2 1
1 1 + x22
= ln .
2 1 + x21
To check the result, we can take the derivative per § 7.5 of the final expression
with respect to x2 :
∂ 1 1 + x22

1 ∂ 2
2

ln = ln 1 + x 2 − ln 1 + x 1
∂x2 2 1 + x21 x2 =x 2 ∂x2 x2 =x
x
= ,
1 + x2
which indeed has the form of the integrand we started with.
The technique is integration by substitution. It does not solve all inte-
grals but it does solve many, whether alone or in combination with other
techniques.
9.3 Integration by parts

Integration by parts is a curious but very broadly applicable technique which
begins with the derivative product rule (4.25),
d(uv) = u dv + v du,
where u(τ ) and v(τ ) are functions of an independent variable τ . Reordering

terms,
u dv = d(uv) − v du.
Integrating,
Z b Z b
u dv = uv|bτ =a − v du. (9.4)
τ =a τ =a
Equation (9.4) is the rule of integration by parts.

For an example of the rule’s operation, consider the integral
Z x
S(x) = τ cos ατ dτ.
0
Unsure how to integrate this, we can begin by integrating part of it. We can
begin by integrating the cos ατ dτ part. Letting
u ← τ,
dv ← cos ατ dτ,
we find that2
du = dτ,
sin ατ
v= .
α
According to (9.4), then,
τ sin ατ x
Z x
sin ατ x
S(x) = − dτ = sin αx + cos αx − 1.
α
0 0 α α
Integration by parts is a powerful technique, but one should understand

clearly what it does and does not do. It does not just integrate each part
of an integral separately. It isn’t that simple. What it does is to integrate
one part of an integral separately—whichever part one has chosen to iden-
tify as dv—while contrarily differentiating the other part u, R upon which it
rewards the mathematician only with a whole new integral v du. The R new
integral may or may not be easier to integrate than was the original u dv.
The virtue of the technique
R lies in that one often can find a part dv which
does yield an easier v du. The technique is powerful for this reason.
For another kind of example of the rule’s operation, consider the definite
integral3 Z ∞
Γ(z) ≡ e−τ τ z−1 dτ, ℜ(z) > 0. (9.5)
0
2
The careful reader will observe that v = (sin ατ )/α + C matches the chosen dv for
any value of C, not just for C = 0. This is true. However, nothing in the integration by
parts technique requires us to consider all possible v. Any convenient v suffices. In this
case, we choose v = (sin ατ )/α.
3
[43]
9.4. INTEGRATION BY UNKNOWN COEFFICIENTS 201
Letting
u ← e−τ ,
dv ← τ z−1 dτ,
we evidently have that
du = −e−τ dτ,
τz
v= .
z
Substituting these according to (9.4) into (9.5) yields
z ∞ Z ∞ z

−τ τ τ
−e−τ dτ

Γ(z) = e −
z τ =0 z
Z ∞ z0
τ −τ
= [0 − 0] + e dτ
0 z
Γ(z + 1)
= .
z
When written
Γ(z + 1) = zΓ(z), (9.6)
this is an interesting result. Since per (9.5)
Z ∞
∞
e−τ dτ = −e−τ 0 = 1,

Γ(1) =
0
it follows by induction on (9.6) that
(n − 1)! = Γ(n). (9.7)
Thus (9.5), called the gamma function, can be taken as an extended defi-
nition of the factorial (z − 1)! for all z, ℜ(z) > 0. Integration by parts has
made this finding possible.
9.4 Integration by unknown coefficients

One of the more powerful integration techniques is relatively inelegant, yet
it easily cracks some integrals that give other techniques trouble. The tech-
nique is the method of unknown coefficients, and it is based on the antideriva-
tive (9.1) plus intelligent guessing. It is best illustrated by example.
Consider the integral (which arises in probability theory)

Z x
2
S(x) = e−(ρ/σ) /2 ρ dρ. (9.8)
0
If one does not know how to solve the integral in a more elegant way, one
can guess a likely-seeming antiderivative form, such as
2 /2 d −(ρ/σ)2 /2
e−(ρ/σ) ρ= ae ,
dρ
where the a is an unknown coefficient. Having guessed, one has no guarantee
that the guess is right, but see: if the guess were right, then the antiderivative
would have the form
2 /2 d −(ρ/σ)2 /2
e−(ρ/σ) ρ = ae
dρ
aρ 2
= − 2 e−(ρ/σ) /2 ,
σ
implying that
a = −σ 2
(evidently the guess is right, after all). Using this value for a, one can write
the specific antiderivative
2 /2 d h 2 −(ρ/σ)2 /2 i
e−(ρ/σ) ρ= −σ e ,
dρ
with which one can solve the integral, concluding that
h 2
ix h 2
i
S(x) = −σ 2 e−(ρ/σ) /2 = σ 2 1 − e−(x/σ) /2 . (9.9)
0
The same technique solves differential equations, too. Consider for ex-
ample the differential equation
dx = (Ix − P ) dt, x|t=0 = xo , x|t=T = 0, (9.10)
which conceptually represents4 the changing balance x of a bank loan ac-

count over time t, where I is the loan’s interest rate and P is the borrower’s
payment rate. If it is desired to find the correct payment rate P which pays
4
Real banks (in the author’s country, at least) by law or custom actually use a needlessly
more complicated formula—and not only more complicated, but mathematically slightly
incorrect, too.
9.4. INTEGRATION BY UNKNOWN COEFFICIENTS 203
the loan off in the time T , then (perhaps after some bad guesses) we guess
the form
x(t) = Aeαt + B,
where α, A and B are unknown coefficients. The guess’ derivative is
dx = αAeαt dt.
Substituting the last two equations into (9.10) and dividing by dt yields
αAeαt = IAeαt + IB − P,
which at least is satisfied if both of the equations
αAeαt = IAeαt ,
0 = IB − P,
are satisfied. Evidently good choices for α and B, then, are
α = I,
P
B= .
I
Substituting these coefficients into the x(t) equation above yields the general
solution
P
x(t) = AeIt + (9.11)
I
to (9.10). The constants A and P , we establish by applying the given bound-
ary conditions x|t=0 = xo and x|t=T = 0. For the former condition, (9.11)
is
P P
xo = Ae(I)(0) + =A+ ;
I I
and for the latter condition,
P
0 = AeIT + .
I
Solving the last two equations simultaneously, we have that
−e−IT xo
A= ,
1 − e−IT (9.12)
Ixo
P = .
1 − e−IT
Applying these to the general solution (9.11) yields the specific solution
xo h
(I)(t−T )
i
x(t) = 1 − e (9.13)
1 − e−IT
to (9.10) meeting the boundary conditions, with the payment rate P required
of the borrower given by (9.12).
The virtue of the method of unknown coefficients lies in that it permits
one to try an entire family of candidate solutions at once, with the family
members distinguished by the values of the coefficients. If a solution exists
anywhere in the family, the method usually finds it.
The method of unknown coefficients is an elephant. Slightly inelegant the
method may be, but it is pretty powerful, too—and it has surprise value (for
some reason people seem not to expect it). Such are the kinds of problems
the method can solve.
9.5 Integration by closed contour

We pass now from the elephant to the falcon, from the inelegant to the
sublime. Consider the definite integral5
Z ∞
τa
S= dτ, −1 < a < 0.
0 τ +1
This is a hard integral. No obvious substitution, no evident factoring into

parts, seems to solve the integral; but there is a way. The integrand has
a pole at τ = −1. Observing that τ is only a dummy integration variable,
if one writes the same integral using the complex variable z in place of the
real variable τ , then Cauchy’s integral formula (8.29) has that integrating
once counterclockwise about a closed complex contour, with the contour
enclosing the pole at z = −1 but shutting out the branch point at z = 0,
yields
za
I a
I= dz = i2πz a |z=−1 = i2π ei2π/2 = i2πei2πa/2 .
z+1
The trouble, of course, is that the integral S does not go about a closed
complex contour. One can however construct a closed complex contour I of
which S is a part, as in Fig 9.1. If the outer circle in the figure is of infinite
5
[43, § 1.2]
9.5. INTEGRATION BY CLOSED CONTOUR 205
Figure 9.1: Integration by closed contour.
ℑ(z)
I2
z = −1
I1
ℜ(z)
I4 I3
radius and the inner, of infinitesimal, then the closed contour I is composed
of the four parts
I = I1 + I2 + I3 + I4
= (I1 + I3 ) + I2 + I4 .
The figure tempts one to make the mistake of writing that I1 = S = −I3 ,
but besides being incorrect this defeats the purpose of the closed contour
technique. More subtlety is needed. One must take care to interpret the
four parts correctly. The integrand z a /(z + 1) is multiple-valued; so, in
fact, the two parts I1 + I3 6= 0 do not cancel. The integrand has a branch
point at z = 0, which, in passing from I3 through I4 to I1 , the contour has
circled. Even though z itself takes on the same values along I3 as along I1 ,
the multiple-valued integrand z a /(z + 1) does not. Indeed,
∞ Z ∞
(ρei0 )a ρa
Z
I1 = dρ = dρ = S,
0 (ρei0 ) + 1 0 ρ+1
∞ Z ∞
(ρei2π )a ρa
Z
−I3 = dρ = ei2πa dρ = ei2πa S.
0 (ρei2π ) + 1 0 ρ+1
Therefore,
I = I1 + I2 + I3 + I4
= (I1 + I3 ) + I2 + I4
2π Z 2π
za za
Z
i2πa
= (1 − e )S + lim dz − lim dz
ρ→∞ φ=0 z + 1 ρ→0 φ=0 z + 1
Z 2π Z 2π
i2πa a−1
= (1 − e )S + lim z dz − lim z a dz
ρ→∞ φ=0 ρ→0 φ=0
a 2π a+1 2π

z z
= (1 − ei2πa )S + lim

− lim .
ρ→∞ a φ=0 ρ→0 a + 1 φ=0
Since a < 0, the first limit vanishes; and because a > −1, the second
limit vanishes, too, leaving
I = (1 − ei2πa )S.
But by Cauchy’s integral formula we have already found an expression for I.

Substituting this expression into the last equation yields, by successive steps,
i2πei2πa/2 = (1 − ei2πa )S,

i2πei2πa/2
S = ,
1 − ei2πa
i2π
S = −i2πa/2 ,
e − ei2πa/2
2π/2
S = − .
sin(2πa/2)
That is,
∞
τa 2π/2
Z
dτ = − , −1 < a < 0, (9.14)
0 τ +1 sin(2πa/2)
an astonishing result.6
Another example7 is
Z 2π
dθ
T = , ℑ(a) = 0, |ℜ(a)| < 1.
0 1 + a cos θ
6
So astonishing is the result, that one is unlikely to believe it at first encounter. How-
ever, straightforward (though computationally highly inefficient) numerical integration
per (7.1) confirms the result, as the interested reader and his computer can check. Such
results vindicate the effort we have spent in deriving Cauchy’s integral formula (8.29).
7
[40]
9.5. INTEGRATION BY CLOSED CONTOUR 207
As in the previous example, here again the contour is not closed. The
previous example closed the contour by extending it, excluding the branch
point. In this example there is no branch point to exclude, nor need one
extend the contour. Rather, one changes the variable
z ← eiθ
and takes advantage of the fact that z, unlike θ, begins and ends the inte-
gration at the same point. One thus obtains the equivalent integral
dz/iz i2 dz
I I
T = =− 2
1 + (a/2)(z + 1/z) a z + 2z/a + 1
i2 dz
I
= − h √ ih √ i,
a z − −1 + 1 − a2 /a z − −1 − 1 − a2 /a
whose contour is the unit circle in the Argand plane. The integrand evidently
has poles at
√
−1 ± 1 − a2
z= ,
a
whose magnitudes are such that
√
2 2 − a2 ∓ 2 1 − a2
|z| = .
a2
One of the two magnitudes is less than unity and one is greater, meaning
that one of the two poles lies within the contour and one lies without, as is
seen by the successive steps8
a2 < 1,
0 < 1 − a2 ,
(−a2 )(0) > (−a2 )(1 − a2 ),
0 > −a2 + a4 ,
1 − a2 > 1 − 2a2 + a4 ,
2
1 − a2 > 1 − a2 ,
√
1 − a2 > 1 − a2 ,
√ √
− 1 − a2 < −(1 − a2 ) < 1 − a2 ,
√ √
1 − 1 − a2 < a2 < 1 + 1 − a2 ,
√ √
2 − 2 1 − a2 < 2a2 < 2 + 2 1 − a2 ,
√ √
2 − a2 − 2 1 − a2 < a2 < 2 − a2 + 2 1 − a2 ,
2
√ √
2−a −2 1−a 2 2 − a2 + 2 1 − a2
< 1 < .
a2 a2
Per Cauchy’s integral formula (8.29), integrating about the pole within the
contour yields

−i2/a 2π
T = i2π √ =√ .
z − −1 − 1 − a2 /a √ 1 − a2
z=(−1+ 1−a2 )/a
Observe that by means of a complex variable of integration, each example

has indirectly evaluated an integral whose integrand is purely real. If it seems
unreasonable to the reader to expect so flamboyant a technique actually
to work, this seems equally unreasonable to the writer—but work it does,
nevertheless. It is a great technique.
The technique, integration by closed contour, is found in practice to solve
many integrals other techniques find almost impossible to crack. The key
to making the technique work lies in closing a contour one knows how to
treat. The robustness of the technique lies in that any contour of any shape
will work, so long as the contour encloses appropriate poles in the Argand
domain plane while shutting branch points out.
8
These steps are perhaps best read from bottom to top. See Ch. 6’s footnote 15.
9.6. INTEGRATION BY PARTIAL-FRACTION EXPANSION 209
The extension Z z2
Z z2

f (z) dz ≤ |f (z) dz| (9.15)
z1 z1
of the complex triangle sum inequality (3.22) from the discrete to the con-
tinuous case sometimes proves useful in evaluating integrals by this section’s
technique, as in § 17.5.4.
9.6 Integration by partial-fraction expansion

This section treats integration by partial-fraction expansion. It introduces
the expansion itself first.9 Throughout the section,
j, j ′ , k, ℓ, m, n, p, p(·) , M, N ∈ Z.
9.6.1 Partial-fraction expansion

−4 5
f (z) = + .
z−1 z−2
Combining the two fractions over a common denominator10 yields
z+3
f (z) = .
(z − 1)(z − 2)
Of the two forms, the former is probably the more amenable to analysis.
For example, using (9.3),
Z 0 Z 0 Z 0
−4 5
f (τ ) dτ = dτ + dτ
−1 −1 τ − 1 −1 τ − 2
= [−4 ln(1 − τ ) + 5 ln(2 − τ )]0−1 .
The trouble is that one is not always given the function in the amenable
form.
Given a rational function
PN −1
bk z k
f (z) = QNk=0 (9.16)
j=1 (z − αj )
9
[48, Appendix F][31, §§ 2.7 and 10.12]
10
Terminology (you probably knew this already): A fraction is the ratio of two numbers
or expressions B/A. In the fraction, B is the numerator and A is the denominator. The
quotient is Q = B/A.
in which no two of the several poles αj are the same, the partial-fraction
expansion has the form
N
X Ak
f (z) = , (9.17)
z − αk
k=1
where multiplying each fraction of (9.17) by

hQ i
N
j=1 (z − αj ) /(z − αk )
hQ i
N
j=1 (z − α j ) /(z − αk )
puts the several fractions over a common denominator, yielding (9.16). Di-
viding (9.16) by (9.17) gives the ratio
PN −1 , N
k
k=0 bk z
X Ak
1 = QN .
j=1 (z − αj )
z − αk
k=1
In the immediate neighborhood of z = αm , the mth term Am /(z − αm )

dominates the summation of (9.17). Hence,
PN −1 ,
b z k
k Am
1 = lim QN k=0 .
z→αm
j=1 (z − αj )
z − αm
Rearranging factors, we have that

PN −1 k

k=0 b k z
Am = hQ i = lim [(z − αm )f (z)] , (9.18)
N z→αm
j=1 (z − αj ) /(z − αm )

z=αm
where Am , the value of f (z) with the pole canceled, is called the residue
of f (z) at the pole z = αm . Equations (9.17) and (9.18) together give the
partial-fraction expansion of (9.16)’s rational function f (z).
9.6.2 Repeated poles

The weakness of the partial-fraction expansion of § 9.6.1 is that it cannot
directly handle repeated poles. That is, if αn = αj , n 6= j, then the residue
formula (9.18) finds an uncanceled pole remaining in its denominator and
thus fails for An = Aj (it still works for the other Am ). The conventional
way to expand a fraction with repeated poles is presented in § 9.6.5 below;
but because at least to this writer that way does not lend much applied
insight, the present subsection treats the matter in a different way. Here,
we separate the poles.
N −1
X Cei2πk/N
g(z) = , N > 1, 0 < ǫ ≪ 1, (9.19)
k=0
z − ǫei2πk/N
where C is a real-valued constant. This function evidently has a small circle

of poles in the Argand plane at αk = ǫei2πk/N . Factoring,
N −1
C X ei2πk/N
g(z) = i2πk/N )/z
.
z 1 − (ǫe
k=0
Using (2.34) to expand the fraction,

 !j 
N −1 ∞
C X
ei2πk/N
X ǫei2πk/N
g(z) = 
z z
k=0 j=0
N −1 X
∞ j−1 i2πjk/N
X ǫ e
= C
zj
k=0 j=1
∞ j−1 N −1
X ǫ X
i2πj/N
k
= C e .
zj
j=1 k=0
But11
N −1
(
X k N if j = mN,
ei2πj/N =
k=0
0 otherwise,
so
∞
X ǫmN −1
g(z) = N C .
m=1
z mN
For |z| ≫ ǫ—that is, except in the immediate neighborhood of the small
circle of poles—the first term of the summation dominates. Hence,
ǫN −1
g(z) ≈ N C , |z| ≫ ǫ.
zN
11
If you don’t see why, then for N = 8 and j = 3 plot the several (ei2πj/N )k in the
Argand plane. Do the same for j = 2 then j = 8. Only in the j = 8 case do the terms
add coherently; in the other cases they cancel.
This effect—reinforcing when j = nN , canceling otherwise—is a classic manifestation
of Parseval’s principle, which § 17.1 will formally introduce later in the book.
Having achieved this approximation, if we strategically choose
1
C= ,
N ǫN −1
then
1
g(z) ≈ , |z| ≫ ǫ.
zN
But given the chosen value of C, (9.19) is
N −1
1 X ei2πk/N
g(z) = , N > 1, 0 < ǫ ≪ 1.
N ǫN −1 z − ǫei2πk/N
k=0
Joining the last two equations together, changing z − zo ← z, and writing

more formally, we have that
N −1
1 1 X ei2πk/N
= lim , N > 1. (9.20)
(z − zo )N ǫ→0 N ǫN −1

k=0
z − zo + ǫei2πk/N
The significance of (9.20) is that it lets one replace an N -fold pole with
a small circle of ordinary poles, which per § 9.6.1 we already know how to
handle. Notice incidentally that 1/N ǫN −1 is a large number not a small.
The poles are close together but very strong.
An example to illustrate the technique, separating a double pole:
z2 − z + 6
f (z) =
(z − 1)2 (z + 2)
z2 − z + 6
= lim
ǫ→0 (z − [1 + ǫei2π(0)/2 ])(z − [1 + ǫei2π(1)/2 ])(z + 2)
z2 − z + 6
= lim
ǫ→0 (z − [1 + ǫ])(z − [1 − ǫ])(z + 2)
z2 − z + 6

1
= lim
ǫ→0 z − [1 + ǫ] (z − [1 − ǫ])(z + 2) z=1+ǫ
z2 − z + 6

1
+
z − [1 − ǫ] (z − [1 + ǫ])(z + 2) z=1−ǫ
z2 − z + 6

1
+
z+2 (z − [1 + ǫ])(z − [1 − ǫ]) z=−2

1 6+ǫ
= lim
ǫ→0 z − [1 + ǫ] 6ǫ + 2ǫ2

1 6−ǫ
+
z − [1 − ǫ] −6ǫ + 2ǫ2

1 0xC
+
z+2 9

1/ǫ − 1/6 −1/ǫ − 1/6 4/3
= lim + +
ǫ→0 z − [1 + ǫ] z − [1 − ǫ] z+2

1/ǫ −1/ǫ −1/3 4/3
= lim + + + .
ǫ→0 z − [1 + ǫ] z − [1 − ǫ] z − 1 z + 2
Notice how the calculation has discovered an additional, single pole at z = 1,

the pole hiding under dominant, double pole there.
9.6.3 Integrating a rational function
If one can find the poles of a rational function of the form (9.16), then one
can use (9.17) and (9.18)—and, if needed, (9.20)—to expand the function
into a sum of partial fractions, each of which one can integrate individually.
Continuing the example of § 9.6.2, for 0 ≤ x < 1,
x x
τ2 − τ + 6
Z Z
f (τ ) dτ = 2
dτ
0 0 (τ − 1) (τ + 2)
Z x
1/ǫ −1/ǫ −1/3 4/3
= lim + + + dτ
ǫ→0 0 τ − [1 + ǫ] τ − [1 − ǫ] τ − 1 τ + 2

1 1
= lim ln([1 + ǫ] − τ ) − ln([1 − ǫ] − τ )
ǫ→0 ǫ ǫ
x
1 4
− ln(1 − τ ) + ln(τ + 2)
3 3 0
x
1 [1 + ǫ] − τ 1 4
= lim ln − ln(1 − τ ) + ln(τ + 2)
ǫ→0 ǫ [1 − ǫ] − τ 3 3
0x
1 [1 − τ ] + ǫ 1 4
= lim ln − ln(1 − τ ) + ln(τ + 2)
ǫ→0 ǫ [1 − τ ] − ǫ 3 3
x 0
1 2ǫ 1 4
= lim ln 1 + − ln(1 − τ ) + ln(τ + 2)
ǫ→0 ǫ 1−τ 3 3 0
x
1 2ǫ 1 4
= lim − ln(1 − τ ) + ln(τ + 2)
ǫ→0 ǫ 1−τ 3 3 0
x
2 1 4
= lim − ln(1 − τ ) + ln(τ + 2)
ǫ→0 1 − τ 3 3
0
2 1 4 x+2
= − 2 − ln(1 − x) + ln .
1−x 3 3 2
To check (§ 7.5) that the result is correct, we can take the derivative of the
final expression:

d 2 1 4 x+2
− 2 − ln(1 − x) + ln
dx 1−x 3 3 2 x=τ
2 −1/3 4/3
= + +
(τ − 1)2 τ − 1 τ + 2
τ2 − τ + 6
= ,
(τ − 1)2 (τ + 2)
which indeed has the form of the integrand we started with, confirming the
result. (Notice incidentally how much easier it is symbolically to differentiate
than to integrate!)
9.6.4 The derivatives of a rational function

Not only the integral of a rational function interests us; its derivatives in-
terest us, too. One needs no special technique to compute such derivatives,
of course, but the derivatives do bring some noteworthy properties.
First of interest is the property that a function in the general rational
form
wp h0 (w)
Φ(w) = , g(0) 6= 0, (9.21)
g(w)
enjoys derivatives in the general rational form
dk Φ wp−k hk (w)
= , 0 ≤ k ≤ p, (9.22)
dwk [g(w)]k+1
where g and hk are polynomials in nonnegative powers of w. The property

is proved by induction. When k = 0, (9.22) is (9.21), so (9.22) is good at
least for this case. Then, if (9.22) holds for k = n − 1,
dn Φ d dn−1 Φ d wp−n+1 hn−1 (w) wp−n hn (w)

= = n = ,
dwn dw dwn−1 dw [g(w)] [g(w)]n+1
dhn−1 dg
hn (w) ≡ wg − nwhn−1 + (p − n + 1)ghn−1 , 0 < n ≤ p,
dw dw
which makes hn (like hn−1 ) a polynomial in nonnegative powers of w. By
induction on this basis, (9.22) holds for all 0 ≤ k ≤ p, as was to be demon-
strated.
A related property is that
dk Φ

= 0 for 0 ≤ k < p. (9.23)
dwk w=0
That is, the function and its first p − 1 derivatives are all zero at w = 0. The
reason is that (9.22)’s denominator is [g(w)]k+1 6= 0, whereas its numerator
has a wp−k = 0 factor, when 0 ≤ k < p and w = 0.
9.6.5 Repeated poles (the conventional technique)

Though the technique of §§ 9.6.2 and 9.6.3 affords extra insight, it is not
the conventional technique to expand in partial fractions a rational function
having a repeated pole. The conventional technique is worth learning not
only because it is conventional but also because it is usually quicker to apply
in practice. This subsection derives it.
A rational function with repeated poles,

PN −1
bk z k
f (z) = QM k=0 , (9.24)
pj
j=1 (z − αj )
M
X
N ≡ pj ,
j=1
pj ≥ 0,
αj ′ 6= αj if j ′ 6= j,
where j, k, M , N and the several pj are integers, cannot be expanded solely

in the first-order fractions of § 9.6.1, but can indeed be expanded if higher-
order fractions are allowed:
M pj −1
X X Ajℓ
f (z) = . (9.25)
j=1 ℓ=0
(z − αj )pj −ℓ
What the partial-fraction expansion (9.25) lacks are the values of its several
coefficients Ajℓ .
One can determine the coefficients with respect to one (possibly re-
peated) pole at a time. To determine them with respect to the pm -fold
pole at z = αm , 1 ≤ m ≤ M , one multiplies (9.25) by (z − αm )pm to obtain
the form
M pj −1
X X (Ajℓ )(z − αm )pm pXm −1
pm
(z − αm ) f (z) = pj −ℓ
+ (Amℓ )(z − αm )ℓ .
j=1, ℓ=0
(z − αj ) ℓ=0
j6=m
But (9.23) with w = z − αm reveals the double summation and its first
pm − 1 derivatives all to be null at z = αm ; that is,

M pj −1
dk X X (Ajℓ )(z − αm )pm

= 0, 0 ≤ k < pm ;
dz k (z − αj )pj −ℓ
j=1, ℓ=0
j6=m

z=αm
so, the (z − αm )pm f (z) equation’s kth derivative reduces at that point to
pX
m −1
dk h pm
i dk h i
ℓ
(z − αm ) f (z) = (Amℓ )(z − αm )
dz k z=αm dz k z=αm
ℓ=0
= k!Amk , 0 ≤ k < pm .
Changing j ← m and ℓ ← k and solving for Ajℓ then produces the coeffi-
cients
ℓ h i
1 d pj
Ajℓ = (z − αj ) f (z) , 0 ≤ ℓ < p, (9.26)
ℓ! dz ℓ z=αj
to weight the expansion (9.25)’s partial fractions. In case of a repeated pole,

these coefficients evidently depend not only on the residual function itself
but also on its several derivatives, one derivative per repetition of the pole.
9.6.6 The existence and uniqueness of solutions

Equation (9.26) has solved (9.24) and (9.25). A professional mathematician
might object however that it has done so without first proving that a unique
solution actually exists.
Comes from us the reply, “Why should we prove that a solution exists,
once we have actually found it?”
Ah, but the professional’s point is that we have found the solution only
if in fact it does exist, and uniquely; otherwise what we have found is a
phantom. A careful review of § 9.6.5’s logic discovers no guarantee that
all of (9.26)’s coefficients actually come from the same expansion. Maybe
there exist two distinct expansions, and some of the coefficients come from
the one, some from the other. On the other hand, maybe there exists no
expansion at all, in which event it is not even clear what (9.26) means.
“But these are quibbles, cavils and nitpicks!” we are inclined to grumble.
“The present book is a book of applied mathematics.”
Well, yes, but on this occasion let us nonetheless follow the professional’s
line of reasoning, if only a short way.
Uniqueness is proved by positing two solutions
M pj −1 M pj −1
X X Ajℓ X X Bjℓ
f (z) = p −ℓ
=
j=1 ℓ=0
(z − αj ) j
j=1 ℓ=0
and computing the difference

M pj −1
X X Bjℓ − Ajℓ
j=1 ℓ=0
between them. Logically this difference must be zero for all z if the two
solutions are actually to represent the same function f (z). This however
is seen to be possible only if Bjℓ = Ajℓ for each (j, ℓ). Therefore, the two
solutions are one and the same.
Existence comes of combining the several fractions of (9.25) over a com-
mon denominator and comparing the resulting numerator against the numer-
ator of (9.24). Each coefficient bk is seen thereby to be a linear combination
of the several Ajℓ , where the combination’s weights depend solely on the
locations αj and multiplicities pj of f (z)’s several poles. From the N coeffi-
cients bk and the N coefficients Ajℓ , an N × N system of N linear equations
in N unknowns results—which might for example (if, say, N = 3) look like
b0 = −2A00 + A01 + 3A10 ,

b1 = A00 + A01 + A10 ,
b2 = 2A01 − 5A10 .
We shall show in Chs. 11 through 14 that when such a system has no so-
lution, there always exist an alternate set of bk for which the same system
has multiple solutions. But uniqueness, which we have already established,
forbids such multiple solutions in all cases. Therefore it is not possible for
the system to have no solution—which is to say, the solution necessarily
exists.
We shall not often in this book prove existence and uniqueness explicitly,
but such proofs when desired tend to fit the pattern outlined here.
9.7 Frullani’s integral

One occasionally meets an integral of the form
∞
f (bτ ) − f (aτ )
Z
S= dτ,
0 τ
where a and b are real, positive coefficients and f (τ ) is an arbitrary

R complex
Rexpression in τ . One wants to split such an integral in two as [f (bτ )/τ ] dτ −
[f (aτ )/τ ] dτ ; but if f (0+ ) 6= f (+∞), one cannot, because each half-integral
alone diverges. Nonetheless, splitting the integral in two is the right idea,
provided that one first relaxes the limits of integration as
(Z )
1/ǫ 1/ǫ
f (bτ ) f (aτ )
Z
S = lim dτ − dτ .
ǫ→0+ ǫ τ ǫ τ
9.8. PRODUCTS OF EXPONENTIALS, POWERS AND LOGS 219
Changing σ ← bτ in the left integral and σ ← aτ in the right yields

(Z )
b/ǫ Z a/ǫ
f (σ) f (σ)
S = lim dσ − dσ
ǫ→0+ bǫ σ aǫ σ
(Z )
bǫ Z a/ǫ Z b/ǫ
−f (σ) f (σ) − f (σ) f (σ)
= lim dσ + dσ + dσ
ǫ→0+ aǫ σ bǫ σ a/ǫ σ
(Z )
b/ǫ Z bǫ
f (σ) f (σ)
= lim dσ − dσ
ǫ→0+ a/ǫ σ aǫ σ
(here on the face of it, we have split the integration as though a ≤ b, but in
fact it does not matter which of a and b is the greater, as is easy to verify).
So long as each of f (ǫ) and f (1/ǫ) approaches a constant value as ǫ vanishes,
this is
( Z b/ǫ Z bǫ )
dσ dσ
S = lim f (+∞) − f (0+ )
ǫ→0+ a/ǫ σ aǫ σ

b/ǫ + bǫ
= lim f (+∞) ln − f (0 ) ln
ǫ→0+ a/ǫ aǫ
b
= [f (τ )]∞0 ln .
a
Thus we have Frullani’s integral,
Z ∞
f (bτ ) − f (aτ ) b
dτ = [f (τ )]∞
0 ln , (9.27)
0 τ a
which, if a and b are both real and positive, works for any f (τ ) which has
definite f (0+ ) and f (+∞).12
9.8 Integrating products of exponentials, powers

and logarithms
The products exp(ατ )τ n (where n ∈ Z) and τ a−1 ln τ tend to arise13 among
other places in integrands related to special functions ([chapter not yet writ-
ten]). The two occur often enough to merit investigation here.
12
[43, § 1.3][1, § 2.5.1][66, “Frullani’s integral”]
13
One could write the latter product more generally as τ a−1 ln βτ . According to Ta-
ble 2.5, however, ln βτ = ln β + ln τ ; wherein ln β is just a constant.
Concerning exp(ατ )τ n , by § 9.4’s method of unknown coefficients we

guess its antiderivative to fit the form
n
d X
exp(ατ )τ n = ak exp(ατ )τ k
dτ
k=0
n n
X X ak
= αak exp(ατ )τ k + exp(ατ )τ k−1
k
k=0 k=1
n−1
X
n ak+1
= αan exp(ατ )τ + αak + exp(ατ )τ k .
k+1
k=0
If so, then evidently

1
an = ;
α
ak+1
ak = − , 0 ≤ k < n.
(k + 1)(α)
That is,
1 (−)n−k
ak = Qn = , 0 ≤ k ≤ n.
α j=k+1 (−jα) (n!/k!)αn−k+1
Therefore,14
n
d X (−)n−k
exp(ατ )τ n = exp(ατ )τ k , n ∈ Z, n ≥ 0, α 6= 0.
dτ (n!/k!)αn−k+1
k=0
(9.28)
The right form to guess for the antiderivative of τ a−1 ln τ is less obvious.
Remembering however § 5.3’s observation that ln τ is of zeroth order in τ ,
after maybe some false tries we eventually do strike the right form
d a
τ a−1 ln τ = τ [B ln τ + C]
dτ
= τ a−1 [aB ln τ + (B + aC)],
which demands that B = 1/a and that C = −1/a2 . Therefore,15
d τa

a−1 1
τ ln τ = ln τ − , a 6= 0. (9.29)
dτ a a
14
[55, Appendix 2, eqn. 73]
15
[55, Appendix 2, eqn. 74]
9.9. INTEGRATION BY TAYLOR SERIES 221
Table 9.1: Antiderivatives of products of exponentials, powers and loga-

rithms.
n
d X (−)n−k
exp(ατ )τ n = exp(ατ )τ k , n ∈ Z, n ≥ 0, α 6= 0
dτ (n!/k!)αn−k+1
k=0
d τa

1
τ a−1 ln τ = ln τ − , a 6= 0
dτ a a
ln τ d (ln τ )2
=
τ dτ 2
Antiderivatives of terms like τ a−1 (ln τ )2 , exp(ατ )τ n ln τ and so on can be

computed in like manner as the need arises.
Equation (9.29) fails when a = 0, but in this case with a little imagination
the antiderivative is not hard to guess:
ln τ d (ln τ )2
= . (9.30)
τ dτ 2
If (9.30) seemed hard to guess nevertheless, then l’Hôpital’s rule (4.30),

applied to (9.29) as a → 0, with the observation from (2.41) that
τ a = exp(a ln τ ), (9.31)
would yield the same (9.30).

9.9 Integration by Taylor series
With sufficient cleverness the techniques of the foregoing sections solve many,
many integrals. But not all. When all else fails, as sometimes it does, the
Taylor series of Ch. 8 and the antiderivative of § 9.1 together offer a concise,
practical way to integrate some functions, at the price of losing the functions’
known closed analytic forms. For example,

x ∞
xX
τ2 (−τ 2 /2)k
Z Z
exp − dτ = dτ
0 2 0 k=0 k!
xX∞
(−)k τ 2k
Z
= dτ
0 k=0 2k k!
"∞ #x
X (−)k τ 2k+1
=
(2k + 1)2k k!
k=0 0
∞ ∞ k
X (−)k x2k+1 X 1 Y −x2
= = (x) .
(2k + 1)2k k! 2k + 1 2j
k=0 k=0 j=1
The result is no function one recognizes; it is just a series. This is not

necessarily bad, however. After all, when a Taylor series from Table 8.1
is used to calculate sin z, then sin z is just a series, too. The series above
converges just as accurately and just as fast.
Sometimes it helps to give the series a name like
∞ ∞ k
X (−)k z 2k+1 X 1 Y −z 2
myf z ≡ = (z) .
(2k + 1)2k k! 2k + 1 2j
k=0 k=0 j=1
Then,
x
τ2
Z
exp − dτ = myf x.
0 2
The myf z is no less a function than sin z is; it’s just a function you hadn’t
heard of before. You can plot the function, or take its derivative
2
d τ
myf τ = exp − ,
dτ 2
or calculate its value, or do with it whatever else one does with functions.
It works just the same.
Chapter 10
Cubics and quartics
Under the heat of noonday, between the hard work of the morning and the
heavy lifting of the afternoon, one likes to lay down one’s burden and rest a
spell in the shade. Chapters 2 through 9 have established the applied math-
ematical foundations upon which coming chapters will build; and Ch. 11,
hefting the weighty topic of the matrix, will indeed begin to build on those
foundations. But in this short chapter which rests between, we shall refresh
ourselves with an interesting but lighter mathematical topic: the topic of
cubics and quartics.
The expression
z + a0
is a linear polynomial, the lone root z = −a0 of which is plain to see. The
quadratic polynomial
z 2 + a1 z + a0
has of course two roots, which though not plain to see the quadratic for-
mula (2.2) extracts with little effort. So much algebra has been known since
antiquity. The roots of higher-order polynomials, the Newton-Raphson iter-
ation (4.31) locates swiftly, but that is an approximate iteration rather than
an exact formula like (2.2), and as we have seen in § 4.8 it can occasionally
fail to converge. One would prefer an actual formula to extract the roots.
No general formula to extract the roots of the nth-order polynomial
seems to be known.1 However, to extract the roots of the cubic and quartic
polynomials
z 3 + a2 z 2 + a1 z + a0 ,
z 4 + a3 z 3 + a2 z 2 + a1 z + a0 ,
1
Refer to Ch. 6’s footnote 9.
223
224 CHAPTER 10. CUBICS AND QUARTICS
though the ancients never discovered how, formulas do exist. The 16th-
century algebraists Ferrari, Vieta, Tartaglia and Cardano have given us the
clever technique. This chapter explains.2
10.1 Vieta’s transform

There is a sense to numbers by which 1/2 resembles 2, 1/3 resembles 3,
1/4 resembles 4, and so forth. To capture this sense, one can transform a
function f (z) into a function f (w) by the change of variable3
1
w+ ← z,
w
or, more generally,
wo2
w+ ← z. (10.1)
w
Equation (10.1) is Vieta’s transform.4
For |w| ≫ |wo |, we have that z ≈ w; but as |w| approaches |wo | this
ceases to be true. For |w| ≪ |wo |, z ≈ wo2 /w. The constant wo is the corner
value, in the neighborhood of which w transitions from the one domain to
the other. Figure 10.1 plots Vieta’s transform for real w in the case wo = 1.
An interesting alternative to Vieta’s transform is

wo2
wk ← z, (10.2)
w
which in light of § 6.3 might be named Vieta’s parallel transform.
Section 10.2 shows how Vieta’s transform can be used.
10.2 Cubics
The general cubic polynomial is too hard to extract the roots of directly, so
one begins by changing the variable
x+h←z (10.3)
2
[66, “Cubic equation”][66, “Quartic equation”][67, “Quartic equation,” 00:26, 9 Nov.
2006][67, “François Viète,” 05:17, 1 Nov. 2006][67, “Gerolamo Cardano,” 22:35, 31 Oct.
2006][59, § 1.5]
3
This change of variable broadly recalls the sum-of-exponentials form (5.19) of the
cosh(·) function, inasmuch as exp[−φ] = 1/ exp φ.
4
Also called “Vieta’s substitution.” [66, “Vieta’s substitution”]
10.2. CUBICS 225
Figure 10.1: Vieta’s transform (10.1) for wo = 1, plotted logarithmically.
ln z
ln w
to obtain the polynomial
x3 + (a2 + 3h)x2 + (a1 + 2ha2 + 3h2 )x + (a0 + ha1 + h2 a2 + h3 ).
The choice
a2
h≡− (10.4)
3
casts the polynomial into the improved form
a22
a 3
3 a1 a2 2
x + a1 − x + a0 − +2 ,
3 3 3
or better yet
x3 − px − q,
where
a22
p ≡ −a1 + ,
3 (10.5)
a1 a2 a 3
2
q ≡ −a0 + −2 .
3 3
The solutions to the equation
x3 = px + q, (10.6)
then, are the cubic polynomial’s three roots.

So we have struck the a2 z 2 term. That was the easy part; what to do
next is not so obvious. If one could strike the px term as well, then the
roots would follow immediately, but no very simple substitution like (10.3)
achieves this—or rather, such a substitution does achieve it, but at the
price of reintroducing an unwanted x2 or z 2 term. That way is no good.
Lacking guidance, one might try many, various substitutions, none of which
seems to help much; but after weeks or months of such frustration one might
eventually discover Vieta’s transform (10.1), with the idea of balancing the
equation between offsetting w and 1/w terms. This works.
Vieta-transforming (10.6) by the change of variable
wo2
w+ ←x (10.7)
w
we get the new equation
wo2 wo6
w3 + (3wo2 − p)w + (3wo2 − p) + 3 = q, (10.8)
w w
which invites the choice
p
wo2 ≡ , (10.9)
3
reducing (10.8) to read
(p/3)3
w3 + = q.
w3
Multiplying by w3 and rearranging terms, we have the quadratic equation
q p 3
(w3 )2 = 2 w3 − , (10.10)
2 3
which by (2.2) we know how to solve.
Vieta’s transform has reduced the original cubic to a quadratic.
The careful reader will observe that (10.10) seems to imply six roots,
double the three the fundamental theorem of algebra (§ 6.2.2) allows a cubic
polynomial to have. We shall return to this point in § 10.3. For the moment,
however, we should like to improve the notation by defining5
p
P ←− ,
3 (10.11)
q
Q←+ ,
2
5
Why did we not define P and Q so to begin with? Well, before unveiling (10.10),
we lacked motivation to do so. To define inscrutable coefficients unnecessarily before the
need for them is apparent seems poor applied mathematical style.
10.3. SUPERFLUOUS ROOTS 227
Table 10.1: The method to extract the three roots of the general cubic
polynomial. (In the definition of w3 , one can choose either sign.)
0 = z 3 + a2 z 2 + a1 z + a0
a1 a2 2
P ≡ −
3 3
1 a a a 3
1 2 2
Q ≡ −a0 + 3 −2
2 3 3 3

2Q p if P = 0,
w3 ≡
Q ± Q2 + P 3 otherwise.

0 if P = 0 and Q = 0,
x ≡
w − P/w otherwise.
a2
z = x−
3
with which (10.6) and (10.10) are written
x3 = 2Q − 3P x, (10.12)
3 2 3 3
(w ) = 2Qw + P . (10.13)
Table 10.1 summarizes the complete cubic polynomial root extraction meth-
od in the revised notation—including a few fine points regarding superfluous
roots and edge cases, treated in §§ 10.3 and 10.4 below.
10.3 Superfluous roots

As § 10.2 has observed, the equations of Table 10.1 seem to imply six roots,
double the three the fundamental theorem of algebra (§ 6.2.2) allows a cubic
polynomial to have. However, what the equations really imply is not six
distinct roots but six distinct w. The definition x ≡ w − P/w maps two w
to any one x, so in fact the equations imply only three x and thus three
roots z. The question then is: of the six w, which three do we really need
and which three can we ignore as superfluous?
The six w naturally come in two groups of three: one group of three from
the one w3 and a second from the other. For this reason, we shall guess—and
logically it is only a guess—that a single w3 generates three distinct x and
thus (because z differs from x only by a constant offset) all three roots z. If
the guess is right, then the second w3 cannot but yield the same three roots,
which means that the second w3 is superfluous and can safely be overlooked.
But is the guess right? Does a single w3 in fact generate three distinct x?
To prove that it does, let us suppose that it did not. Let us suppose
that a single w3 did generate two w which led to the same x. Letting the
symbol w1 represent the third w, then (since all three w come from the
same w3 ) the two w are e+i2π/3 w1 and e−i2π/3 w1 . Because x ≡ w − P/w,
by successive steps,
P P
e+i2π/3 w1 − = e−i2π/3 w1 − ,
e+i2π/3 w1 e−i2π/3 w1
P P
e+i2π/3 w1 + −i2π/3 = e−i2π/3 w1 + +i2π/3 ,
e w1 e w
1
P P
e+i2π/3 w1 + = e−i2π/3 w1 + ,
w1 w1
which can only be true if

w12 = −P.
Cubing6 the last equation,
w16 = −P 3 ;
but squaring the table’s w3 definition for w = w1 ,
p
w16 = 2Q2 + P 3 ± 2Q Q2 + P 3 .
Combining the last two on w16 ,

p
−P 3 = 2Q2 + P 3 ± 2Q Q2 + P 3 ,
or, rearranging terms and halving,

p
Q2 + P 3 = ∓Q Q2 + P 3 .
Squaring,
Q4 + 2Q2 P 3 + P 6 = Q4 + Q2 P 3 ,
then canceling offsetting terms and factoring,
(P 3 )(Q2 + P 3 ) = 0.
6
The verb to cube in this context means “to raise to the third power,” as to change y
to y 3 , just as the verb to square means “to raise to the second power.”
10.4. EDGE CASES 229
The last equation demands rigidly that either P = 0 or P 3 = −Q2 . Some

cubic polynomials do meet the demand—§ 10.4 will treat these and the
reader is asked to set them aside for the moment—but most cubic polyno-
mials do not meet it. For most cubic polynomials, then, the contradiction
proves false the assumption which gave rise to it. The assumption: that the
three x descending from a single w3 were not distinct. Therefore, provided
that P 6= 0 and P 3 6= −Q2 , the three x descending from a single w3 are
indeed distinct, as was to be demonstrated.
The conclusion: p either, not both, of the two signs in the table’s quadratic
3
solution w ≡ Q ± Q2 + P 3 demands to be considered. One can choose
either sign; it matters not which.7 The one sign alone yields all three roots
of the general cubic polynomial.
In calculating the three w from w3 , one can apply the Newton-Raphson
iteration (4.33), the Taylor series of Table 8.1, or any other convenient root-
finding technique to find a single root w1 such that w13 = w3 . Then√the other
two roots come easier. They are e±i2π/3 w1 ; but e±i2π/3 = (−1 ± i 3)/2, so
√
−1 ± i 3
w = w1 , w1 . (10.14)
2
We should observe, incidentally, that nothing prevents two actual roots
of a cubic polynomial from having the same value. This certainly is possible,
and it does not mean that one of the two roots is superfluous or that the
polynomial has fewer than three roots. For example, the cubic polynomial
(z − 1)(z − 1)(z − 2) = z 3 − 4z 2 + 5z − 2 has roots at 1, 1 and 2, with a
single root at z = 2 and a double root—that is, two roots—at z = 1. When
this happens, the method of Table 10.1 properly yields the single root once
and the double root twice, just as it ought to do.
10.4 Edge cases

Section 10.3 excepts the edge cases P = 0 and P 3 = −Q2 . Mostly the book
does not worry much about edge cases, but the effects of these cubic edge
cases seem sufficiently nonobvious that the book might include here a few
words about them, if for no other reason than to offer the reader a model of
how to think about edge cases on his own. Table 10.1 gives the quadratic
solution p
w 3 ≡ Q ± Q2 + P 3 ,
7
Numerically, it can matter. As a simple rule, because w appears in the denominator
of x’s definition, when the two w3 differ in magnitude one might choose the larger.
in which § 10.3 generally finds it sufficient to consider either of the two signs.
In the edge case P = 0,
w3 = 2Q or 0.
In the edge case P 3 = −Q2 ,

w3 = Q.
Both edge cases are interesting. In this section, we shall consider first the
edge cases themselves, then their effect on the proof of § 10.3.
The edge case P = 0, like the general non-edge case, gives two distinct
quadratic solutions w3 . One of the two however is w3 = Q − Q = 0, which
is awkward in light of Table 10.1’s definition that x ≡ w − P/w. For this
reason, in applying the table’s method when P = 0, one chooses the other
quadratic solution, w3 = Q + Q = 2Q.
The edge case P 3 = −Q2 gives only the one quadratic solution w3 = Q;
or more precisely, it gives two quadratic solutions which happen to have the
same value. This is fine. One merely accepts that w3 = Q, and does not
worry about choosing one w3 over the other.
The double edge case, or corner case, arises where the two edges meet—
where P = 0 and P 3 = −Q2 , or equivalently where P = 0 and Q = 0. At
the corner, the trouble is that w3 = 0 and that no alternate w3 is available.
However, according to (10.12), x3 = 2Q − 3P x, which in this case means
that x3 = 0 and thus that x = 0 absolutely, no other x being possible. This
implies the triple root z = −a2 /3.
Section 10.3 has excluded the edge cases from its proof of the sufficiency
of a single w3 . Let us now add the edge cases to the proof. In the edge case
P 3 = −Q2 , both w3 are the same, so the one w3 suffices by default because
the other w3 brings nothing different. The edge case P = 0 however does
give two distinct w3 , one of which is w3 = 0, which puts an awkward 0/0 in
the table’s definition of x. We address this edge in the spirit of l’Hôpital’s
rule, by sidestepping it, changing P infinitesimally from P = 0 to P = ǫ.
Then, choosing the − sign in the definition of w3 ,
ǫ3 ǫ3
p
3
w = Q− Q2 + ǫ3 = Q − (Q) 1 + =− ,
2Q2 2Q
ǫ
w = − ,
(2Q)1/3
ǫ ǫ
x = w− =− + (2Q)1/3 = (2Q)1/3 .
w (2Q)1/3
10.5. QUARTICS 231
But choosing the + sign,

p
w3 = Q + Q2 + ǫ3 = 2Q,
w = (2Q)1/3 ,
ǫ ǫ
x = w − = (2Q)1/3 − = (2Q)1/3 .
w (2Q)1/3
Evidently the roots come out the same, either way. This completes the
proof.
10.5 Quartics
Having successfully extracted the roots of the general cubic polynomial, we
now turn our attention to the general quartic. The kernel of the cubic tech-
nique lay in reducing the cubic to a quadratic. The kernel of the quartic
technique lies likewise in reducing the quartic to a cubic. The details dif-
fer, though; and, strangely enough, in some ways the quartic reduction is
actually the simpler.8
As with the cubic, one begins solving the quartic by changing the variable
x+h←z (10.15)
to obtain the equation
x4 = sx2 + px + q, (10.16)
where
a3
h≡− ,
4
a 2
3
s ≡ −a2 + 6 ,
4
a a 3 (10.17)
3 3
p ≡ −a1 + 2a2 −8 ,
4 4
a a 2 a 4
3 3 3
q ≡ −a0 + a1 − a2 +3 .
4 4 4
8
Even stranger, historically Ferrari discovered it earlier [66, “Quartic equation”]. Ap-
parently Ferrari discovered the quartic’s resolvent cubic (10.22), which he could not solve
until Tartaglia applied Vieta’s transform to it. What motivated Ferrari to chase the quar-
tic solution while the cubic solution remained still unknown, this writer does not know,
but one supposes that it might make an interesting story.
The reason the quartic is simpler√to reduce is probably related to the fact that (1)1/4 =
1/3 1/4
±1, ±i, whereas (1) = 1, (−1±i 3)/2. The (1) brings a much neater result, the roots
lying nicely along the Argand axes. This may also be why the quintic is intractable—but
here we trespass the professional mathematician’s territory and stray from the scope of
this book. See Ch. 6’s footnote 9.
To reduce (10.16) further, one must be cleverer. Ferrari9 supplies the clev-
erness. The clever idea is to transfer some but not all of the sx2 term to the
equation’s left side by
x4 + 2ux2 = (2u + s)x2 + px + q,
where u remains to be chosen; then to complete the square on the equation’s

left side as in § 2.2, but with respect to x2 rather than x, as
2
x2 + u = k2 x2 + px + j 2 , (10.18)
where
k2 ≡ 2u + s,
(10.19)
j 2 ≡ u2 + q.
Now, one must regard (10.18) and (10.19) properly. In these equations, s,
p and q have definite values fixed by (10.17), but not so u, j or k. The
variable u is completely free; we have introduced it ourselves and can assign
it any value we like. And though j 2 and k2 depend on u, still, even after
specifying u we remain free at least to choose signs for j and k. As for u,
though no choice would truly be wrong, one supposes that a wise choice
might at least render (10.18) easier to simplify.
So, what choice for u would be wise? Well, look at (10.18). The left
side of that equation is a perfect square. The right side would be, too, if
it were that p = ±2jk; so, arbitrarily choosing the + sign, we propose the
constraint that
p = 2jk, (10.20)
or, better expressed,
p
j= . (10.21)
2k
Squaring (10.20) and substituting for j 2 and k2 from (10.19), we have that
p2 = 4(2u + s)(u2 + q);
or, after distributing factors, rearranging terms and scaling, that
s 4sq − p2
0 = u3 + u2 + qu + . (10.22)
2 8
9
[66, “Quartic equation”]
10.6. GUESSING THE ROOTS 233
Equation (10.22) is the resolvent cubic, which we know by Table 10.1 how
to solve for u, and which we now specify as a second constraint. If the
constraints (10.21) and (10.22) are both honored, then we can safely substi-
tute (10.20) into (10.18) to reach the form
2
x2 + u = k2 x2 + 2jkx + j 2 ,
which is 2 2
x2 + u = kx + j . (10.23)
The resolvent cubic (10.22) of course yields three u not one, but the
resolvent cubic is a voluntary constraint, so we can just pick one u and
ignore the other two. Equation (10.19) then gives k (again, we can just
pick one of the two signs), and (10.21) then gives j. With u, j and k
established, (10.23) implies the quadratic
x2 = ±(kx + j) − u, (10.24)
which (2.2) solves as

s
k k 2
x = ± ±o ± j − u, (10.25)
2 2
wherein the two ± signs are tied together but the third, ±o sign is indepen-
dent of the two. Equation (10.25), with the other equations and definitions
of this section, reveals the four roots of the general quartic polynomial.
In view of (10.25), the change of variables
k
K← ,
2 (10.26)
J ← j,
improves the notation. Using the improved notation, Table 10.2 summarizes
the complete quartic polynomial root extraction method.
10.6 Guessing the roots

It is entertaining to put pencil to paper and use Table 10.1’s method to
extract the roots of the cubic polynomial
0 = [z − 1][z − i][z + i] = z 3 − z 2 + z − 1.
Table 10.2: The method to extract the four roots of the general quartic
polynomial. (In the table, the resolvent cubic is solved for u by the method of
Table 10.1, where any one of the three resulting u serves. Either of the two K
similarly serves. Of the three ± signs in x’s definition, the ±o is independent
but the other two are tied together, the four resulting combinations giving
the four roots of the general quartic.)
0 = z 4 + a3 z 3 + a2 z 2 + a1 z + a0
a 2
3
s ≡ −a2 + 6
4
a a 3
3 3
p ≡ −a1 + 2a2 −8
4 4
a a 2 a 4
3 3 3
q ≡ −a0 + a1 − a2 +3
4 4 4
s 4sq − p2
0 = u3 + u2 + qu +
√ 2 8
2u + s
K ≡ ±
( p2
± u2 + q if K = 0,
J ≡
p/4K otherwise.
p
x ≡ ±K ±o K 2 ± J − u
a3
z = x−
4
10.6. GUESSING THE ROOTS 235
One finds that

1 2
z = w+ − 2 ,
3 √3 w
2 5+ 3 3
w3 ≡ ,
33
which says indeed that z = 1, ±i, but just you try to simplify it! A more
baroque, more impenetrable way to write the number 1 is not easy to con-
ceive. One has found the number 1 but cannot recognize it. Figuring the
square and cube roots in the expression numerically, the root of the poly-
nomial comes mysteriously to 1.0000, but why? The root’s symbolic form
gives little clue.
In general no better way is known;10 we are stuck with the cubic baro-
quity. However, to the extent that a cubic, a quartic, a quintic or any other
polynomial has real, rational roots, a trick is known to sidestep Tables 10.1
and 10.2 and guess the roots directly. Consider for example the quintic
polynomial
7 1
z 5 − z 4 + 4z 3 + z 2 − 5z + 3.
2 2
Doubling to make the coefficients all integers produces the polynomial
2z 5 − 7z 4 + 8z 3 + 1z 2 − 0xAz + 6,
which naturally has the same roots. If the roots are complex or irrational,
they are hard to guess; but if any of the roots happens to be real and rational,
it must belong to the set

1 2 3 6
±1, ±2, ±3, ±6, ± , ± , ± , ± .
2 2 2 2
No other real, rational root is possible. Trying the several candidates on the
polynomial, one finds that 1, −1 and 3/2 are indeed roots. Dividing these
out leaves a quadratic which is easy to solve for the remaining roots.
The real, rational candidates are the factors of the polynomial’s trailing
coefficient (in the example, 6, whose factors are ±1, ±2, ±3 and ±6) divided
by the factors of the polynomial’s leading coefficient (in the example, 2,
whose factors are ±1 and ±2). The reason no other real, rational root is
10
At least, no better way is known to this author. If any reader can straightforwardly
simplify the expression without solving a cubic polynomial of some kind, the author would
like to hear of it.
possible is seen11 by writing z = p/q—where p, q ∈ Z are integers and the

fraction p/q is fully reduced—then multiplying the nth-order polynomial
by q n to reach the form
an pn + an−1 pn−1 q + · · · + a1 pq n−1 + a0 q n = 0,
where all the coefficients ak are integers. Moving the q n term to the equa-
tion’s right side, we have that
an pn−1 + an−1 pn−2 q + · · · + a1 q n−1 p = −a0 q n ,

which implies that a0 q n is a multiple of p. But by demanding that the

fraction p/q be fully reduced, we have defined p and q to be relatively prime
to one another—that is, we have defined them to have no factors but ±1
in common—so, not only a0 q n but a0 itself is a multiple of p. By similar
reasoning, an is a multiple of q. But if a0 is a multiple of p, and an , a multiple
of q, then p and q are factors of a0 and an respectively. We conclude for this
reason, as was to be demonstrated, that no real, rational root is possible
except a factor of a0 divided by a factor of an .12
Such root-guessing is little more than an algebraic trick, of course, but it
can be a pretty useful trick if it saves us the embarrassment of inadvertently
expressing simple rational numbers in ridiculous ways.
One could write much more about higher-order algebra, but now that
the reader has tasted the topic he may feel inclined to agree that, though the
general methods this chapter has presented to solve cubics and quartics are
interesting, further effort were nevertheless probably better spent elsewhere.
The next several chapters turn to the topic of the matrix, harder but much
more profitable, toward which we mean to put substantial effort.
11
The presentation here is quite informal. We do not want to spend many pages on
this.
12
[59, § 3.2]
Part II
Matrices and vectors
237
Chapter 11
The matrix
Chapters 2 through 9 have laid solidly the basic foundations of applied

mathematics. This chapter begins to build on those foundations, demanding
some heavier mathematical lifting.
Taken by themselves, most of the foundational methods of the earlier
chapters have handled only one or at most a few numbers (or functions) at
a time. However, in practical applications the need to handle large arrays
of numbers at once arises often. Some nonobvious effects emerge then, as,
for example, the eigenvalue of Ch. 14.
Regarding the eigenvalue: the eigenvalue was always there, but prior to
this point in the book it was usually trivial—the eigenvalue of 5 is just 5,
for instance—so we didn’t bother much to talk about it. It is when numbers
are laid out in orderly grids like
2 3
6 4 0
C= 4 3 0 1 5
3 1 0
that nontrivial eigenvalues arise (though you√cannot tell just by looking, the
eigenvalues of C happen to be −1 and [7 ± 0x49]/2). But, just what is an
eigenvalue? Answer: an eigenvalue is the value by which an object like C
scales an eigenvector without altering the eigenvector’s direction. Of course,
we have not yet said what an eigenvector is, either, or how C might scale
something, but it is to answer precisely such questions that this chapter and
the three which follow it are written.
So, we are getting ahead of ourselves. Let’s back up.
An object like C is called a matrix. It serves as a generalized coefficient or
multiplier. Where we have used single numbers as coefficients or multipliers
239
240 CHAPTER 11. THE MATRIX
heretofore, one can with sufficient care often use matrices instead. The
matrix interests us for this reason among others.
The technical name for the “single number” is the scalar. Such a number,
as for instance 5 or −4 + i3, is called a scalar because its action alone
during multiplication is simply to scale the thing it multiplies. Besides acting
alone, however, scalars can also act in concert—in orderly formations—thus
constituting any of three basic kinds of arithmetical object:
• the scalar itself, a single number like α = 5 or β = −4 + i3;
• the vector, a column of m scalars like
» –
5
u= ,
−4 + i3
which can be written in-line with the notation u = [5 −4 + i3]T (here

there are two scalar elements, 5 and −4+i3, so in this example m = 2);
• the matrix, an m × n grid of scalars, or equivalently a row of n vectors,
like » –
0 6 2
A = 1 1 −1 ,
which can be written in-line with the notation A = [0 6 2; 1 1 −1] or

the notation A = [0 1; 6 1; 2 −1]T (here there are two rows and three
columns of scalar elements, so in this example m = 2 and n = 3).
Several general points are immediately to be observed about these various
objects. First, despite the geometrical Argand interpretation of the com-
plex number, a complex number is not a two-element vector but a scalar;
therefore any or all of a vector’s or matrix’s scalar elements can be complex.
Second, an m-element vector does not differ for most purposes from an m×1
matrix; generally the two can be regarded as the same thing. Third, the
three-element (that is, three-dimensional) geometrical vector of § 3.3 is just
an m-element vector with m = 3. Fourth, m and n can be any nonnegative
integers, even one, even zero, even infinity.1
Where one needs visually to distinguish a symbol like A representing
a matrix, one can write it [A], in square brackets.2 Normally however a
simple A suffices.
1
Fifth, though the progression scalar, vector, matrix suggests next a “matrix stack” or
stack of p matrices, such objects in fact are seldom used. As we shall see in § 11.1, the chief
advantage of the standard matrix is that it neatly represents the linear transformation of
one vector into another. “Matrix stacks” bring no such advantage. This book does not
treat them.
2
Alternate notations seen in print include A and A.
241
The matrix is a notoriously hard topic to motivate. The idea of the

matrix is deceptively simple. The mechanics of matrix arithmetic are de-
ceptively intricate. The most basic body of matrix theory, without which
little or no useful matrix work can be done, is deceptively extensive. The
matrix neatly encapsulates a substantial knot of arithmetical tedium and
clutter, but to understand the matrix one must first understand the tedium
and clutter the matrix encapsulates. As far as the author is aware, no one
has ever devised a way to introduce the matrix which does not seem shallow,
tiresome, irksome, even interminable at first encounter; yet the matrix is too
important to ignore. Applied mathematics brings nothing else quite like it.3
Chapters 11 through 14 treat the matrix and its algebra. This chapter,
3
In most of its chapters, the book seeks a balance between terseness the determined
beginner cannot penetrate and prolixity the seasoned veteran will not abide. The matrix
upsets this balance.
Part of the trouble with the matrix is that its arithmetic is just that, an arithmetic,
no more likely to be mastered by mere theoretical study than was the classical arithmetic
of childhood. To master matrix arithmetic, one must drill it; yet the book you hold is
fundamentally one of theory not drill.
The reader who has previously drilled matrix arithmetic will meet here the essential
applied theory of the matrix. That reader will find this chapter and the next three te-
dious enough. The reader who has not previously drilled matrix arithmetic, however, is
likely to find these chapters positively hostile. Only the doggedly determined beginner
will learn the matrix here alone; others will find it more amenable to drill matrix arith-
metic first in the early chapters of an introductory linear algebra textbook, dull though
such chapters be (see [42] or better yet the fine, surprisingly less dull [30] for instance,
though the early chapters of almost any such book give the needed arithmetical drill.)
Returning here thereafter, the beginner can expect to find these chapters still tedious but
no longer impenetrable. The reward is worth the effort. That is the approach the author
recommends.
To the mathematical rebel, the young warrior with face painted and sword agleam,
still determined to learn the matrix here alone, the author salutes his honorable defiance.
Would the rebel consider alternate counsel? If so, then the rebel might compose a dozen
matrices of various sizes and shapes, broad, square and tall, decomposing each carefully
by pencil per the Gauss-Jordan method of § 12.3, checking results (again by pencil; using
a machine defeats the point of the exercise, and using a sword, well, it won’t work) by
multiplying factors to restore the original matrices. Several hours of such drill should build
the young warrior the practical arithmetical foundation to master—with commensurate
effort—the theory these chapters bring. The way of the warrior is hard, but conquest is
not impossible.
To the matrix veteran, the author presents these four chapters with grim enthusiasm.
Substantial, logical, necessary the chapters may be, but exciting they are not. At least, the
earlier parts are not very exciting (later parts are better). As a reasonable compromise, the
veteran seeking more interesting reading might skip directly to Chs. 13 and 14, referring
back to Chs. 11 and 12 as need arises.
Ch. 11, introduces the rudiments of the matrix itself.4
11.1 Provenance and basic use

It is in the study of linear transformations that the concept of the matrix
first arises. We begin there.
11.1.1 The linear transformation

Section 7.3.3 has introduced the idea of linearity. The linear transformation 5
is the operation of an m × n matrix A, as in
Ax = b, (11.1)
to transform an n-element vector x into an m-element vector b, while re-

specting the rules of linearity
A(x1 + x2 ) = Ax1 + Ax2 = b1 + b2 ,

A(αx) = αAx = αb, (11.2)
A(0) = 0.
For example, » –
0 6 2
A= 1 1 −1
is the 2 × 3 matrix which transforms a three-element vector x into a two-

element vector b such that
» –
0x1 + 6x2 + 2x3
Ax = 1x1 + 1x2 − 1x3
= b,
where 2 3
x1 » –
b1
x= 4 x2 5, b= b2
.
x3
4
[5][21][30][42]
5
Professional mathematicians conventionally are careful to begin by drawing a clear
distinction between the ideas of the linear transformation, the basis set and the simul-
taneous system of linear equations—proving from suitable axioms that the three amount
more or less to the same thing, rather than implicitly assuming the fact. The professional
approach [5, Chs. 1 and 2][42, Chs. 1, 2 and 5] has much to recommend it, but it is not
the approach we will follow here.
11.1. PROVENANCE AND BASIC USE 243
In general, the operation of a matrix A is that6,7

n
X
bi = aij xj , (11.3)
j=1
where xj is the jth element of x, bi is the ith element of b, and
aij ≡ [A]ij
is the element at the ith row and jth column of A, counting from top left
(in the example for instance, a12 = 6).
Besides representing linear transformations as such, matrices can also
represent simultaneous systems of linear equations. For example, the system
0x1 + 6x2 + 2x3 = 2,

1x1 + 1x2 − 1x3 = 4,
is compactly represented as
Ax = b,
with A as given above and b = [2 4]T . Seen from this point of view, a
simultaneous system of linear equations is itself neither more nor less than
a linear transformation.
11.1.2 Matrix multiplication (and addition)

Nothing prevents one from lining several vectors xk up in a row, industrial
mass production-style, transforming them at once into the corresponding
6
As observed in Appendix B, there are unfortunately not enough distinct Roman and
Greek letters available to serve the needs of higher mathematics. In matrix work, the
Roman letters ijk conventionally serve as indices, but the same letter i also serves as the
imaginary unit, which is not an index and has nothing P to do with indices. Fortunately,
the meaning is usually clear from the context: i in i or aij is an index; i in −4 + i3
or eiφ is the imaginary unit. Should a case arise in which the meaning is not clear, one
can use ℓjk or some other convenient letters for the indices.
7
Whether to let the index j run from 0 to n−1 or from 1 to n is an awkward question of
applied mathematical style. In computers, the index normally runs from 0 to n − 1, and in
many ways this really is the more sensible way to do it. In mathematical theory, however,
a 0 index normally implies something special or basic about the object it identifies. The
book you are reading tends to let the index run from 1 to n, following mathematical
convention in the matter for this reason.
Conceived more generally, an m × n matrix can be considered an ∞ × ∞ matrix with
zeros in the unused cells. Here, both indices i and j run from −∞ to +∞ anyway, so the
computer’s indexing convention poses no dilemma in this case. See § 11.3.
vectors bk by the same matrix A. In this case,
X ≡ [ x1 x2 · · · xp ],
B ≡ [ b1 b2 · · · bp ],
AX = B, (11.4)
n
X
bik = aij xjk .
j=1
Equation (11.4) implies a definition for matrix multiplication. Such matrix

multiplication is associative since
n
X
[(A)(XY )]ik = aij [XY ]jk
j=1
n
" p #
X X
= aij xjℓ yℓk
j=1 ℓ=1
p
XX n
= aij xjℓ yℓk
ℓ=1 j=1
= [(AX)(Y )]ik . (11.5)
Matrix multiplication is not generally commutative, however;
AX 6= XA, (11.6)
as one can show by a suitable counterexample like A = [0 1; 0 0], X =

[1 0; 0 0]. To multiply a matrix by a scalar, one multiplies each of the
matrix’s elements individually by the scalar:
[αA]ij = αaij . (11.7)
Evidently multiplication by a scalar is commutative: αAx = Aαx.

Matrix addition works in the way one would expect, element by element;
and as one can see from (11.4), under multiplication, matrix addition is
indeed distributive:
[X + Y ]ij = xij + yij ;

(A)(X + Y ) = AX + AY ; (11.8)
(A + C)(X) = AX + CX.
11.1. PROVENANCE AND BASIC USE 245
11.1.3 Row and column operators

The matrix equation Ax = b represents the linear transformation of x
into b, as we have seen. Viewed from another perspective, however, the
same matrix equation represents something else; it represents a weighted
sum of the columns of A, with the elements of x as the weights. In this
view, one writes (11.3) as
n
X
b= [A]∗j xj , (11.9)
j=1
where [A]∗j is the jth column of A. Here x is not only a vector; it is also an
operator. It operates on A’s columns. By virtue of multiplying A from the
right, the vector x is a column operator acting on A.
If several vectors xk line up in a row to form a matrix X, such that
AX = B, then the matrix X is likewise a column operator:
n
X
[B]∗k = [A]∗j xjk . (11.10)
j=1
The kth column of X weights the several columns of A to yield the kth
column of B.
If a matrix multiplying from the right is a column operator, is a matrix
multiplying from the left a row operator? Indeed it is. Another way to write
AX = B, besides (11.10), is
n
X
[B]i∗ = aij [X]j∗ . (11.11)
j=1
The ith row of A weights the several rows of X to yield the ith row of B.
The matrix A is a row operator. (Observe the notation. The ∗ here means
“any” or “all.” Hence [X]j∗ means “jth row, all columns of X”—that is,
the jth row of X. Similarly, [A]∗j means “all rows, jth column of A”—that
is, the jth column of A.)
Column operators attack from the right; row operators, from the left.
This rule is worth memorizing; the concept is important. In AX = B, the
matrix X operates on A’s columns; the matrix A operates on X’s rows.
Since matrix multiplication produces the same result whether one views
it as a linear transformation (11.4), a column operation (11.10) or a row
operation (11.11), one might wonder what purpose lies in defining matrix
multiplication three separate ways. However, it is not so much for the sake
of the mathematics that we define it three ways as it is for the sake of

the mathematician. We do it for ourselves. Mathematically, the latter
two do indeed expand to yield (11.4), but as written the three represent
three different perspectives on the matrix. A tedious, nonintuitive matrix
theorem from one perspective can appear suddenly obvious from another
(see for example eqn. 11.63). Results hard to visualize one way are easy
to visualize another. It is worth developing the mental agility to view and
handle matrices all three ways for this reason.
11.1.4 The transpose and the adjoint

One function peculiar to matrix algebra is the transpose
C = AT ,
(11.12)
cij = aji ,
which mirrors an m × n matrix into an n × m matrix. For example,

2 3
0 1
AT = 4 6 1 5.
2 −1
Similar and even more useful is the conjugate transpose or adjoint 8
C = A∗ ,
(11.13)
cij = a∗ji ,
which again mirrors an m × n matrix into an n × m matrix, but conjugates

each element as it goes.
The transpose is convenient notationally to write vectors and matrices in-
line and to express certain matrix-arithmetical mechanics; but algebraically
the transpose is artificial. It is the adjoint rather which mirrors a matrix
properly. (If the transpose and adjoint functions applied to words as to ma-
trices, then the transpose of “derivations” would be “snoitavired,” whereas
the adjoint would be “snoitavired.” See the difference?) On real-valued
matrices like the A in the example, of course, the transpose and the adjoint
amount to the same thing.
8
Alternate notations sometimes seen in print for the adjoint include A† (a notation
which in this book means something unrelated) and AH (a notation which recalls the
name of the mathematician Charles Hermite). However, the book you are reading writes
the adjoint only as A∗ , a notation which better captures the sense of the thing in the
author’s view.
11.2. THE KRONECKER DELTA 247
If one needed to conjugate the elements of a matrix without transposing

the matrix itself, one could contrive notation like A∗T . Such a need seldom
arises, however.
Observe that
(A2 A1 )T = AT1 AT2 ,
(11.14)
(A2 A1 )∗ = A∗1 A∗2 ,
and more generally that9

!T
Y a
Ak = ATk ,
k k
!∗ (11.15)
Y a
Ak = A∗k .
k k
11.2 The Kronecker delta

Section 7.7 has introduced the Dirac delta. The discrete analog of the Dirac
delta is the Kronecker delta 10
(
1 if i = 0,
δi ≡ (11.16)
0 otherwise;
or (
1 if i = j,
δij ≡ (11.17)
0 otherwise.
The Kronecker delta enjoys the Dirac-like properties that
∞
X ∞
X ∞
X
δi = δij = δij = 1 (11.18)
i=−∞ i=−∞ j=−∞
and that
∞
X
δij ajk = aik , (11.19)
j=−∞
the latter of which is the Kronecker sifting property. The Kronecker equa-
tions (11.18) and (11.19) parallel the Dirac equations (7.13) and (7.14).
Chs. 11 and 14 will find frequent use for the Kronecker delta. Later,
§ 15.4.3 will revisit the Kronecker delta in another light.
9
Q ‘
Recall from § 2.3 that k Ak = · · · A3 A2 A1 , whereas k Ak = A1 A2 A3 · · · .
10
[67, “Kronecker delta,” 15:59, 31 May 2006]
11.3 Dimensionality and matrix forms

An m × n matrix like 2 3
−4 0
X= 4 1 2 5
2 −1
can be viewed as the ∞ × ∞ matrix
2 3
.. .. .. .. .. .. .. ..
6 . . . . . . . . 7
6
6 ··· 0 0 0 0 0 0 0 ··· 7
7
6
6 ··· 0 0 0 0 0 0 0 ··· 7
7
··· 0 0 −4 0 0 0 0 ···
6 7
6 7
X= 6
6 ··· 0 0 1 2 0 0 0 ··· 7,
7
··· 0 0 2 −1 0 0 0 ···
6 7
6 7
··· 0 0 0 0 0 0 0 ···
6 7
6 7
··· 0 0 0 0 0 0 0 ···
6 7
6 7
.. .. .. .. .. .. .. ..
4 5
. . . . . . . .
with zeros in the unused cells. As before, x11 = −4 and x32 = −1, but
now xij exists for all integral i and j; for instance, x(−1)(−1) = 0. For
such a matrix, indeed for all matrices, the matrix multiplication rule (11.4)
generalizes to
B = AX,
∞
X
bik = aij xjk . (11.20)
j=−∞
For square matrices whose purpose is to manipulate other matrices or

vectors in place, merely padding with zeros often does not suit. Consider
for example the square matrix
2 3
1 0 0
A3 = 4 5 1 0 5.
0 0 1
This A3 is indeed a matrix, but when it acts A3 X as a row operator on

some 3 × p matrix X, its effect is to add to X’s second row, 5 times the first.
Further consider 2 3
1 0 0 0
6 5 1 0 0 7
A4 = 6
4 0 0 1
7,
0 5
0 0 0 1
which does the same to a 4 × p matrix X. We can also define A5 , A6 , A7 , . . .,
if we want; but, really, all these express the same operation: “to add to the
second row, 5 times the first.”
11.3. DIMENSIONALITY AND MATRIX FORMS 249
The ∞ × ∞ matrix
2 3
.. .. .. .. .. .. .. ..
6 . . . . . . . . 7
6
6 ··· 1 0 0 0 0 0 0 ··· 7
7
6
6 ··· 0 1 0 0 0 0 0 ··· 7
7
··· 0 0 1 0 0 0 0 ···
6 7
6 7
A= 6
6 ··· 0 0 5 1 0 0 0 ···
7
7
··· 0 0 0 0 1 0 0 ···
6 7
6 7
··· 0 0 0 0 0 1 0 ···
6 7
6 7
··· 0 0 0 0 0 0 1 ···
6 7
6 7
.. .. .. .. .. .. .. ..
4 5
. . . . . . . .
expresses the operation generally. As before, a11 = 1 and a21 = 5, but now
also a(−1)(−1) = 1 and a09 = 0, among others. By running ones infinitely
both ways out the main diagonal, we guarantee by (11.20) that when A
acts AX on a matrix X of any dimensionality whatsoever, A adds to the
second row of X, 5 times the first—and affects no other row. (But what if X
is a 1 × p matrix, and has no second row? Then the operation AX creates
a new second row, 5 times the first—or rather so fills in X’s previously null
second row.)
In the infinite-dimensional view, the matrices A and X differ essen-
tially.11 This section explains, developing some nonstandard formalisms
the derivations of later sections and chapters can use.12
11
This particular section happens to use the symbols A and X to represent certain
specific matrix forms because such usage flows naturally from the usage Ax = b of § 11.1.
Such usage admittedly proves awkward in other contexts. Traditionally in matrix work
and elsewhere in the book, the letter A does not necessarily represent an extended operator
as it does here, but rather an arbitrary matrix of no particular form.
12
The idea of infinite dimensionality is sure to discomfit some readers, who have studied
matrices before and are used to thinking of a matrix as having some definite size. There is
nothing wrong with thinking of a matrix as having some definite size, only that that view
does not suit the present book’s development. And really, the idea of an ∞ × 1 vector or
an ∞ × ∞ matrix should not seem so strange. After all, consider the vector u such that
uℓ = sin ℓǫ,
where 0 < ǫ ≪ 1 and ℓ is an integer, which holds all values of the function sin θ of a real
argument θ. Of course one does not actually write down or store all the elements of an
infinite-dimensional vector or matrix, any more than one actually writes down or stores
all the bits (or digits) of 2π. Writing them down or storing them is not the point. The
point is that infinite dimensionality is all right; that the idea thereof does not threaten
to overturn the reader’s preëxisting matrix knowledge; that, though the construct seem
unfamiliar, no fundamental conceptual barrier rises against it.
Different ways of looking at the same mathematics can be extremely useful to the applied
mathematician. The applied mathematical reader who has never heretofore considered
11.3.1 The null and dimension-limited matrices

The null matrix is just what its name implies:
2 3
.. .. .. .. .. ..
6 . . . . . . 7
6
6 ··· 0 0 0 0 0 ··· 7
7
··· 0 0 0 0 0 ···
6 7
6 7
0= 6
6 ··· 0 0 0 0 0 ··· 7;
7
··· 0 0 0 0 0 ···
6 7
6 7
··· 0 0 0 0 0 ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
or more compactly,
[0]ij = 0.
Special symbols like 0, 0 or O are possible for the null matrix, but usually a
simple 0 suffices. There are no surprises here; the null matrix brings all the
expected properties of a zero, like
0 + A = A,
[0][X] = 0.
The same symbol 0 used for the null scalar (zero) and the null matrix
is used for the null vector, too. Whether the scalar 0, the vector 0 and the
matrix 0 actually represent different things is a matter of semantics, but the
three are interchangeable for most practical purposes in any case. Basically,
a zero is a zero is a zero; there’s not much else to it.13
Now a formality: the ordinary m × n matrix X can be viewed, infinite-
dimensionally, as a variation on the null matrix, inasmuch as X differs from
the null matrix only in the mn elements xij , 1 ≤ i ≤ m, 1 ≤ j ≤ n. Though
the theoretical dimensionality of X be ∞ × ∞, one need record only the mn
elements, plus the values of m and n, to retain complete information about
such a matrix. So the semantics are these: when we call a matrix X an
m × n matrix, or more precisely a dimension-limited matrix with an m × n
infinite dimensionality in vectors and matrices would be well served to take the opportunity
to do so here. As we shall discover in Ch. 12, dimensionality is a poor measure of a matrix’s
size in any case. What really counts is not a matrix’s m × n dimensionality but rather its
rank.
13
Well, of course, there’s a lot else to it, when it comes to dividing by zero as in Ch. 4,
or to summing an infinity of zeros as in Ch. 7, but those aren’t what we were speaking of
here.
active region, we shall mean formally that X is an ∞ × ∞ matrix whose

elements are all zero outside the m × n rectangle:
xij = 0 except where 1 ≤ i ≤ m and 1 ≤ j ≤ n. (11.21)
By these semantics, every 3 × 2 matrix (for example) is also a formally a

4 × 4 matrix; but a 4 × 4 matrix is not in general a 3 × 2 matrix.
11.3.2 The identity and scalar matrices and the extended

operator
The general identity matrix —or simply, the identity matrix —is
2 3
.. .. .. .. .. ..
6 . . . . . . 7
6
6 ··· 1 0 0 0 0 ··· 7
7
··· 0 1 0 0 0 ···
6 7
6 7
I= 6
6 ··· 0 0 1 0 0 ··· 7,
7
··· 0 0 0 1 0 ···
6 7
6 7
··· 0 0 0 0 1 ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
or more compactly,
[I]ij = δij , (11.22)
where δij is the Kronecker delta of § 11.2. The identity matrix I is a matrix 1,
as it were,14 bringing the essential property one expects of a 1:
IX = X = XI. (11.23)
The scalar matrix is

2 3
.. .. .. .. .. ..
6 . . . . . . 7
6
6 ··· λ 0 0 0 0 ··· 7
7
··· 0 λ 0 0 0 ···
6 7
6 7
λI = 6
6 ··· 0 0 λ 0 0 ··· 7,
7
··· 0 0 0 λ 0 ···
6 7
6 7
··· 0 0 0 0 λ ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
or more compactly,
[λI]ij = λδij , (11.24)
14
In fact you can write it as 1 if you like. That is essentially what it is. The I can be
regarded as standing for “identity” or as the Roman numeral I.
If the identity matrix I is a matrix 1, then the scalar matrix λI is a matrix λ,

such that
[λI]X = λX = X[λI]. (11.25)
The identity matrix is (to state the obvious) just the scalar matrix with
λ = 1.
The extended operator A is a variation on the scalar matrix λI, λ 6= 0,
inasmuch as A differs from λI only in p specific elements, with p a finite
number. Symbolically,
(
(λ)(δij + αk ) if (i, j) = (ik , jk ), 1 ≤ k ≤ p,
aij = (11.26)
λδij otherwise;
λ 6= 0.
The several αk control how the extended operator A differs from λI. One
need record only the several αk along with their respective addresses (ik , jk ),
plus the scale λ, to retain complete information about such a matrix. For
example, for an extended operator fitting the pattern
2 3
.. .. .. .. .. ..
6 . . . . . . 7
6
6 ··· λ 0 0 0 0 ··· 7
7
··· λα1 λ 0 0 0 ···
6 7
6 7
A= 6
6 ··· 0 0 λ λα2 0 ··· 7,
7
··· 0 0 0 λ(1 + α3 ) 0 ···
6 7
6 7
··· 0 0 0 0 λ ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
one need record only the values of α1 , α2 and α3 , the respective addresses
(2, 1), (3, 4) and (4, 4), and the value of the scale λ; this information alone
implies the entire ∞ × ∞ matrix A.
When we call a matrix A an extended n × n operator, or an extended
operator with an n × n active region, we shall mean formally that A is an
∞ × ∞ matrix and is further an extended operator for which
1 ≤ ik ≤ n and 1 ≤ jk ≤ n for all 1 ≤ k ≤ p. (11.27)
That is, an extended n × n operator is one whose several αk all lie within
the n × n square. The A in the example is an extended 4 × 4 operator (and
also a 5 × 5, a 6 × 6, etc., but not a 3 × 3).
(Often in practice for smaller operators—especially in the typical case
that λ = 1—one finds it easier just to record all the n × n elements of
the active region. This is fine. Large matrix operators however tend to be
sparse, meaning that they depart from λI in only a very few of their many
elements. It would waste a lot of computer memory explicitly to store all
those zeros, so one normally stores just the few elements, instead.)
Implicit in the definition of the extended operator is that the identity
matrix I and the scalar matrix λI, λ 6= 0, are extended operators with 0 × 0
active regions (and also 1 × 1, 2 × 2, etc.). If λ = 0, however, the scalar
matrix λI is just the null matrix, which is no extended operator but rather
by definition a 0 × 0 dimension-limited matrix.
11.3.3 The active region

Though maybe obvious, it bears stating explicitly that a product of dimen-
sion-limited and/or extended-operational matrices with n × n active regions
itself has an n × n active region.15 (Remember that a matrix with an m′ × n′
active region also by definition has an n × n active region if m′ ≤ n and
n′ ≤ n.) If any of the factors has dimension-limited form then so does the
product; otherwise the product is an extended operator.16
11.3.4 Other matrix forms

Besides the dimension-limited form of § 11.3.1 and the extended-operational
form of § 11.3.2, other infinite-dimensional matrix forms are certainly pos-
sible. One could for example advantageously define a “null sparse” form,
recording only nonzero elements and their addresses in an otherwise null
matrix; or a “tridiagonal extended” form, bearing repeated entries not only
along the main diagonal but also along the diagonals just above and just
below. Section 11.9 introduces one worthwhile matrix which fits neither the
dimension-limited nor the extended-operational form. Still, the dimension-
15
The section’s earlier subsections formally define the term active region with respect
to each of the two matrix forms.
16
If symbolic proof of the subsection’s claims is wanted, here it is in outline:
aij = λa δij unless 1 ≤ (i, j) ≤ n,

bij = λb δij unless 1 ≤ (i, j) ≤ n;
X
[AB]ij = aik bkj
k
(P
(λa δik )bkj = λa bij unless 1 ≤ i ≤ n
= Pk
k aik (λb δkj ) = λb aij unless 1 ≤ j ≤ n
= λa λb δij unless 1 ≤ (i, j) ≤ n.
It’s probably easier just to sketch the matrices and look at them, though.
limited and extended-operational forms are normally the most useful, and
they are the ones we shall principally be handling in this book.
One reason to have defined specific infinite-dimensional matrix forms is
to show how straightforwardly one can fully represent a practical matrix of
an infinity of elements by a modest, finite quantity of information. Further
reasons to have defined such forms will soon occur.
11.3.5 The rank-r identity matrix

The rank-r identity matrix Ir is the dimension-limited matrix for which
(
δij if 1 ≤ i ≤ r and/or 1 ≤ j ≤ r,
[Ir ]ij = (11.28)
0 otherwise,
where either the “and” or the “or” can be regarded (it makes no difference).
The effect of Ir is that
Im X = X = XIn ,
(11.29)
Im x = x,
where X is an m × n matrix and x, an m × 1 vector. Examples of Ir include
2 3
1 0 0
I3 = 4 0 1 0 5.
0 0 1
(Remember that in the infinite-dimensional view, I3 , though a 3 × 3 matrix,

is formally an ∞ × ∞ matrix with zeros in the unused cells. It has only the
three ones and fits the 3 × 3 dimension-limited form of § 11.3.1. The areas
of I3 not shown are all zero, even along the main diagonal.)
The rank r can be any nonnegative integer, even zero (though the rank-
zero identity matrix I0 is in fact the null matrix, normally just written 0).
If alternate indexing limits are needed (for instance for a computer-indexed
identity matrix whose indices run from 0 to r − 1), the notation Iab , where
(
δij if a ≤ i ≤ b and/or a ≤ j ≤ b,
[Iab ]ij ≡ (11.30)
0 otherwise,
can be used; the rank in this case is r = b − a + 1, which is just the count
of ones along the matrix’s main diagonal.
The name “rank-r” implies that Ir has a “rank” of r, and indeed it does.
For the moment, however, we shall discern the attribute of rank only in the
rank-r identity matrix itself. Section 12.5 defines rank for matrices more
generally.
11.3.6 The truncation operator

The rank-r identity matrix Ir is also the truncation operator. Attacking
from the left, as in Ir A, it retains the first through rth rows of A but
cancels other rows. Attacking from the right, as in AIr , it retains the first
through rth columns. Such truncation is useful symbolically to reduce an
extended operator to dimension-limited form.
Whether a matrix C has dimension-limited or extended-operational form
(though not necessarily if it has some other form), if it has an m × n active
region17 and
m ≤ r,
n ≤ r,
then
Ir C = Ir CIr = CIr . (11.31)
For such a matrix, (11.31) says at least two things:
• It is superfluous to truncate both rows and columns; it suffices to
truncate one or the other.
• The rank-r identity matrix Ir commutes freely past C.
Evidently big identity matrices commute freely where small ones cannot
(and the general identity matrix I = I∞ commutes freely past everything).
11.3.7 The elementary vector and the lone-element matrix

The lone-element matrix Emn is the matrix with a one in the mnth cell and
zeros elsewhere:
(
1 if i = m and j = n,
[Emn ]ij ≡ δim δjn = (11.32)
0 otherwise.
P
By this definition, C = i,j cij Eij for any matrix C. The vector analog of
the lone-element matrix is the elementary vector em , which has a one as the
mth element: (
1 if i = m,
[em ]i ≡ δim = (11.33)
0 otherwise.
By this definition, [I]∗j = ej and [I]i∗ = eTi .
17
Refer to the definitions of active region in §§ 11.3.1 and 11.3.2. That a matrix has
an m × n active region does not necessarily mean that it is all zero outside the m × n
rectangle. (After all, if it were always all zero outside, then there would be little point in
applying a truncation operator. There would be nothing there to truncate.)
11.3.8 Off-diagonal entries

It is interesting to observe and useful to note that if
[C1 ]i∗ = [C2 ]i∗ = eTi ,
then also
[C1 C2 ]i∗ = eTi ; (11.34)
and likewise that if
[C1 ]∗j = [C2 ]∗j = ej ,
then also
[C1 C2 ]∗j = ej . (11.35)
The product of matrices has off-diagonal entries in a row or column only
if at least one of the factors itself has off-diagonal entries in that row or
column. Or, less readably but more precisely, the ith row or jth column of
the product of matrices can depart from eTi or ej , respectively, only if the
corresponding row or column of at least one of the factors so departs. The
reason is that in (11.34), C1 acts as a row operator on C2 ; that if C1 ’s ith
row is eTi , then its action is merely to duplicate C2 ’s ith row, which itself is
just eTi . Parallel logic naturally applies to (11.35).
11.4 The elementary operator

Section 11.1.3 has introduced the general row or column operator. Con-
ventionally denoted T , the elementary operator is a simple extended row
or column operator from sequences of which more complicated extended
operators can be built. The elementary operator T comes in three kinds.18
• The first is the interchange elementary
T[i↔j] = I − (Eii + Ejj ) + (Eij + Eji ), (11.36)
which by operating T[i↔j]A or AT[i↔j] respectively interchanges A’s

ith row or column with its jth.19
18
In § 11.3, the symbol A specifically represented an extended operator, but here and
generally the symbol represents any matrix.
19
As a matter of definition, some authors [42] forbid T[i↔i] as an elementary operator,
where j = i, since after all T[i↔i] = I; which is to say that the operator doesn’t actually
do anything. There exist legitimate tactical reasons to forbid (as in § 11.6), but normally
this book permits.
11.4. THE ELEMENTARY OPERATOR 257
• The second is the scaling elementary

Tα[i] = I + (α − 1)Eii , α 6= 0, (11.37)
which by operating Tα[i] A or ATα[i] scales (multiplies) A’s ith row or
column, respectively, by the factor α.
• The third and last is the addition elementary
Tα[ij] = I + αEij , i 6= j, (11.38)
which by operating Tα[ij] A adds to the ith row of A, α times the jth
row; or which by operating ATα[ij] adds to the jth column of A, α
times the ith column.
Examples of the elementary operators include
2 3
.. .. .. .. .. ..
6 . . . . . . 7
6
6 ··· 0 1 0 0 0 ··· 7
7
··· 1 0 0 0 0 ···
6 7
6 7
T[1↔2] = 6
6 ··· 0 0 1 0 0 ··· 7,
7
··· 0 0 0 1 0 ···
6 7
6 7
··· 0 0 0 0 1 ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
2 3
.. .. .. .. .. ..
6 . . . . . . 7
··· 1 0 0 0 0 ···
6 7
6 7
··· 0 1 0 0 0 ···
6 7
6 7
T5[4] = 6
6 ··· 0 0 1 0 0 ··· 7,
7
··· 0 0 0 5 0 ···
6 7
6 7
··· 0 0 0 0 1 ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
2 3
.. .. .. .. .. ..
6 . . . . . . 7
6
6 ··· 1 0 0 0 0 ··· 7
7
··· 5 1 0 0 0 ···
6 7
6 7
T5[21] = 6
6 ··· 0 0 1 0 0 ··· 7.
7
··· 0 0 0 1 0 ···
6 7
6 7
··· 0 0 0 0 1 ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
It is good to define a concept aesthetically. One should usually do so when one can; and
indeed in this case one might reasonably promote either definition on aesthetic grounds.
However, an applied mathematician ought not to let a mere definition entangle him. What
matters is the underlying concept. Where the definition does not serve the concept well,
the applied mathematician considers whether it were not worth the effort to adapt the
definition accordingly.
Note that none of these, and in fact no elementary operator of any kind,
differs from I in more than four elements.
11.4.1 Properties
Significantly, elementary operators as defined above are always invertible
(which is to say, reversible in effect), with
−1
T[i↔j] = T[j↔i] = T[i↔j],
−1
Tα[i] = T(1/α)[i] , (11.39)
−1
Tα[ij] = T−α[ij] ,
being themselves elementary operators such that
T −1 T = I = T T −1 (11.40)
in each case.20 This means that any sequence
Q
of elementaries k Tk can
safely be undone by the reverse sequence k Tk−1 :
`
a Y Y a
Tk−1 Tk = I = Tk Tk−1 . (11.41)
k k k k
The rank-r identity matrix Ir is no elementary operator,21 nor is the

lone-element matrix Emn ; but the general identity matrix I is indeed an
elementary operator. The last can be considered a distinct, fourth kind of
elementary operator if desired; but it is probably easier just to regard it as
an elementary of any of the first three kinds, since I = T[i↔i] = T1[i] = T0[ij] .
From (11.31), we have that
Ir T = Ir T Ir = T Ir if 1 ≤ i ≤ r and 1 ≤ j ≤ r (11.42)
for any elementary operator T which operates within the given bounds.
Equation (11.42) lets an identity matrix with sufficiently high rank pass
through a sequence of elementaries as needed.
In general, the transpose of an elementary row operator is the corre-
sponding elementary column operator. Curiously, the interchange elemen-
tary is its own transpose and adjoint:
∗ T
T[i↔j] = T[i↔j] = T[i↔j] . (11.43)
20
The addition elementary Tα[ii] and the scaling elementary T0[i] are forbidden precisely
because they are not generally invertible.
21
If the statement seems to contradict statements of some other books, it is only a
matter of definition. This book finds it convenient to define the elementary operator in
infinite-dimensional, extended-operational form. The other books are not wrong; their
underlying definitions just differ slightly.
11.4. THE ELEMENTARY OPERATOR 259
11.4.2 Commutation and sorting

Elementary operators often occur in long chains like
A = T−4[32] T[2↔3] T(1/5)[3] T(1/2)[31] T5[21] T[1↔3] ,
with several elementaries of all kinds intermixed. Some applications demand

that the elementaries be sorted and grouped by kind, as

A = T[2↔3] T[1↔3] T−4[21] T(1/0xA)[13] T5[23] T(1/5)[1]
or as
A = T−4[32] T(1/0xA)[21] T5[31] T(1/5)[2] T[2↔3] T[1↔3] ,
among other possible orderings. Though you probably cannot tell just by
looking, the three products above are different orderings of the same ele-
mentary chain; they yield the same A and thus represent exactly the same
matrix operation. Interesting is that the act of reordering the elementaries
has altered some of them into other elementaries of the same kind, but has
changed the kind of none of them.
One sorts a chain of elementary operators by repeatedly exchanging
adjacent pairs. This of course supposes that one can exchange adjacent
pairs, which seems impossible since matrix multiplication is not commuta-
tive: A1 A2 6= A2 A1 . However, at the moment we are dealing in elementary
operators only; and for most pairs T1 and T2 of elementary operators, though
indeed T1 T2 6= T2 T1 , it so happens that there exists either a T1′ such that
T1 T2 = T2 T1′ or a T2′ such that T1 T2 = T2′ T1 , where T1′ and T2′ are elemen-
taries of the same kinds respectively as T1 and T2 . The attempt sometimes
fails when both T1 and T2 are addition elementaries, but all other pairs
commute in this way. Significantly, elementaries of different kinds always
commute. And, though commutation can alter one (never both) of the two
elementaries, it changes the kind of neither.
Many qualitatively distinct pairs of elementaries exist; we shall list these
exhaustively in a moment. First, however, we should like to observe a natural
hierarchy among the three kinds of elementary: (i) interchange; (ii) scaling;
(iii) addition.
• The interchange elementary is the strongest. Itself subject to alter-

ation only by another interchange elementary, it can alter any elemen-
tary by commuting past. When an interchange elementary commutes
past another elementary of any kind, what it alters are the other el-
ementary’s indices i and/or j (or m and/or n, or whatever symbols
happen to represent the indices in question). When two interchange

elementaries commute past one another, only one of the two is al-
tered. (Which one? Either. The mathematician chooses.) Refer to
Table 11.1.
• Next in strength is the scaling elementary. Only an interchange ele-

mentary can alter it, and it in turn can alter only an addition elemen-
tary. Scaling elementaries do not alter one another during commuta-
tion. When a scaling elementary commutes past an addition elemen-
tary, what it alters is the latter’s scale α (or β, or whatever symbol
happens to represent the scale in question). Refer to Table 11.2.
• The addition elementary, last and weakest, is subject to alteration by

either of the other two, itself having no power to alter any elementary
during commutation. A pair of addition elementaries are the only pair
that can altogether fail to commute—they fail when the row index of
one equals the column index of the other—but when they do commute,
neither alters the other. Refer to Table 11.3.
Tables 11.1, 11.2 and 11.3 list all possible pairs of elementary operators, as
the reader can check. The only pairs that fail to commute are the last three
of Table 11.3.
11.5 Inversion and similarity (introduction)

If Tables 11.1, 11.2 and 11.3 exhaustively describe the commutation of one
elementary past another elementary, then what can one write of the com-
mutation of an elementary past the general matrix A? With some matrix
algebra,
T A = (T A)(I) = (T A)(T −1 T ),
AT = (I)(AT ) = (T T −1 )(AT ),
one can write that
T A = [T AT −1 ]T,
(11.44)
AT = T [T −1 AT ],
where T −1 is given by (11.39). An elementary commuting rightward changes

A to T AT −1 ; commuting leftward, to T −1 AT .
11.5. INVERSION AND SIMILARITY (INTRODUCTION) 261
Table 11.1: Inverting, commuting, combining and expanding elementary

operators: interchange. In the table, i 6= j 6= m 6= n; no two indices are
the same. Notice that the effect an interchange elementary T[m↔n] has in
passing any other elementary, even another interchange elementary, is simply
to replace m by n and n by m among the indices of the other elementary.
T[m↔n] = T[n↔m]
T[m↔m] = I
IT[m↔n] = T[m↔n] I
T[m↔n] T[m↔n] = T[m↔n] T[n↔m] = T[n↔m] T[m↔n] = I
T[m↔n] T[i↔n] = T[i↔n] T[m↔i] = T[i↔m] T[m↔n]
2
= T[i↔n] T[m↔n]
T[m↔n] T[i↔j] = T[i↔j] T[m↔n]
T[m↔n] Tα[m] = Tα[n] T[m↔n]
T[m↔n] Tα[i] = Tα[i] T[m↔n]
T[m↔n] Tα[ij] = Tα[ij] T[m↔n]
T[m↔n] Tα[in] = Tα[im] T[m↔n]
T[m↔n] Tα[mj] = Tα[nj] T[m↔n]
T[m↔n] Tα[mn] = Tα[nm] T[m↔n]

operators: scaling. In the table, i =
6 j 6= m 6= n; no two indices are the
same.
T1[m] = I
ITβ[m] = Tβ[m] I
T(1/β)[m] Tβ[m] = I
Tβ[m] Tα[m] = Tα[m] Tβ[m] = Tαβ[m]
Tβ[m] Tα[i] = Tα[i] Tβ[m]
Tβ[m] Tα[ij] = Tα[ij] Tβ[m]
Tβ[m] Tαβ[im] = Tα[im] Tβ[m]
Tβ[m] Tα[mj] = Tαβ[mj] Tβ[m]

operators: addition. In the table, i 6= j 6= m 6= n; no two indices are the
same. The last three lines give pairs of addition elementaries that do not
commute.
T0[ij] = I
ITα[ij] = Tα[ij] I
T−α[ij] Tα[ij] = I
Tβ[ij] Tα[ij] = Tα[ij] Tβ[ij] = T(α+β)[ij]
Tβ[mj] Tα[ij] = Tα[ij] Tβ[mj]
Tβ[in] Tα[ij] = Tα[ij] Tβ[in]
Tβ[mn] Tα[ij] = Tα[ij] Tβ[mn]
Tβ[mi] Tα[ij] = Tα[ij] Tαβ[mj] Tβ[mi]
Tβ[jn] Tα[ij] = Tα[ij] T−αβ[in] Tβ[jn]
Tβ[ji] Tα[ij] 6= Tα[ij] Tβ[ji]
11.5. INVERSION AND SIMILARITY (INTRODUCTION) 263
First encountered in § 11.4, the notation T −1 means the inverse of the

elementary operator T , such that
T −1 T = I = T T −1 .
Matrix inversion is not for elementary operators only, though. Many more
general matrices C also have inverses such that
C −1 C = I = CC −1 . (11.45)
(Do all matrices have such inverses? No. For example, the null matrix has
no such inverse.) The broad question of how to invert a general matrix C,
we leave for Chs. 12 and 13 to address. For the moment however we should
like to observe three simple rules involving matrix inversion.
First, nothing in the logic leading to (11.44) actually requires the ma-
trix T there to be an elementary operator. Any matrix C for which C −1 is
known can fill the role. Hence,
CA = [CAC −1 ]C,
(11.46)
AC = C[C −1 AC].
The transformation CAC −1 or C −1 AC is called a similarity transformation.

Sections 12.2 and 14.9 speak further of this.
Second,
−1 T
CT = C −T = C −1 ,
−1 ∗ (11.47)
C∗ = C −∗ = C −1 ,
where C −∗ is condensed notation for conjugate transposition and inversion

in either order and C −T is of like style. Equation (11.47) is a consequence
of (11.14), since for conjugate transposition
∗ ∗ ∗ ∗
C −1 C ∗ = CC −1 = [I]∗ = I = [I]∗ = C −1 C = C ∗ C −1

and similarly for nonconjugate transposition.

Third,
!−1
Y a
Ck = Ck−1 . (11.48)
k k
This rule emerges upon repeated application of (11.45), which yields
a Y Y a
Ck−1 Ck = I = Ck Ck−1 .
k k k k
Table 11.4: Matrix inversion properties. (The properties work equally for
C −1(r) as for C −1 if A honors an r ×r active region. The full notation C −1(r)
for the rank-r inverse incidentally is not standard, usually is not needed, and
normally is not used.)
C −1 C = I = CC −1
C −1(r) C = Ir = CC −1(r)
−1 T
CT = C −T = C −1
−1 ∗
C∗ = C −∗ = C −1

CA = [CAC −1 ]C
AC = C[C −1 AC]
!−1
Y a
Ck = Ck−1
k k
A more limited form of the inverse exists than the infinite-dimensional

form of (11.45). This is the rank-r inverse, a matrix C −1(r) such that
C −1(r) C = Ir = CC −1(r) . (11.49)
The full notation C −1(r) is not standard and usually is not needed, since
the context usually implies the rank. When so, one can abbreviate the
notation to C −1 . In either notation, (11.47) and (11.48) apply equally for
the rank-r inverse as for the infinite-dimensional inverse. Because of (11.31),
eqn. (11.46) too applies for the rank-r inverse if A’s active region is limited
to r × r. (Section 13.2 uses the rank-r inverse to solve an exactly determined
linear system. This is a famous way to use the inverse, with which many or
most readers will already be familiar; but before using it so in Ch. 13, we
shall first learn how to compute it reliably in Ch. 12.)
11.6 Parity
Consider the sequence of integers or other objects 1, 2, 3, . . . , n. By succes-
sively interchanging pairs of the objects (any pairs, not just adjacent pairs),
one can achieve any desired permutation (§ 4.2.1). For example, beginning
11.6. PARITY 265
with 1, 2, 3, 4, 5, one can achieve the permutation 3, 5, 1, 4, 2 by interchanging

first the 1 and 3, then the 2 and 5.
Now contemplate all possible pairs:
(1, 2) (1, 3) (1, 4) · · · (1, n);
(2, 3) (2, 4) · · · (2, n);
(3, 4) · · · (3, n);
.. ..
. .
(n − 1, n).
In a given permutation (like 3, 5, 1, 4, 2), some pairs will appear in correct
order with respect to one another, while others will appear in incorrect order.
(In 3, 5, 1, 4, 2, the pair [1, 2] appears in correct order in that the larger 2
stands to the right of the smaller 1; but the pair [1, 3] appears in incorrect
order in that the larger 3 stands to the left of the smaller 1.) If p is the
number of pairs which appear in incorrect order (in the example, p = 6), and
if p is even, then we say that the permutation has even or positive parity; if
odd, then odd or negative parity.22
Now consider: every interchange of adjacent elements must either incre-
ment or decrement p by one, reversing parity. Why? Well, think about it.
If two elements are adjacent and their order is correct, then interchanging
falsifies the order, but only of that pair (no other element interposes, thus
the interchange affects the ordering of no other pair). Complementarily, if
the order is incorrect, then interchanging rectifies the order. Either way, an
adjacent interchange alters p by exactly ±1, thus reversing parity.
What about nonadjacent elements? Does interchanging a pair of these
reverse parity, too? To answer the question, let u and v represent the two el-
ements interchanged, with a1 , a2 , . . . , am the elements lying between. Before
the interchange:
. . . , u, a1 , a2 , . . . , am−1 , am , v, . . .
After the interchange:
. . . , v, a1 , a2 , . . . , am−1 , am , u, . . .
The interchange reverses with respect to one another just the pairs
(u, a1 ) (u, a2 ) · · · (u, am−1 ) (u, am )
(a1 , v) (a2 , v) · · · (am−1 , v) (am , v)
(u, v)
22
For readers who learned arithmetic in another language than English, the even integers
are . . . , −4, −2, 0, 2, 4, 6, . . .; the odd integers are . . . , −3, −1, 1, 3, 5, 7, . . . .
The number of pairs reversed is odd. Since each reversal alters p by ±1, the
net change in p apparently also is odd, reversing parity. It seems that re-
gardless of how distant the pair, interchanging any pair of elements reverses
the permutation’s parity.
The sole exception arises when an element is interchanged with itself.
This does not change parity, but it does not change anything else, either, so
in parity calculations we ignore it.23 All other interchanges reverse parity.
We discuss parity in this, a chapter on matrices, because parity concerns
the elementary interchange operator of § 11.4. The rows or columns of a
matrix can be considered elements in a sequence. If so, then the interchange
operator T[i↔j] , i 6= j, acts precisely in the manner described, interchanging
rows or columns and thus reversing parity. It follows that Q
if ik 6= jk and q is
odd, then qk=1 T[ik ↔jk ] 6= I. However, it is possible that qk=1 T[ik ↔jk ] = I
Q
if q is even. In any event, even q implies even p, which means even (positive)
parity; odd q implies odd p, which means odd (negative) parity.
We shall have more to say about parity in §§ 11.7.1 and 14.1.
11.7 The quasielementary operator
Multiplying sequences of the elementary operators of § 11.4, one can form

much more complicated operators, which per (11.41) are always invertible.
Such complicated operators are not trivial to analyze, however, so one finds
it convenient to define an intermediate class of operators, called in this book
the quasielementary operators, more complicated than elementary operators
but less so than arbitrary matrices.
A quasielementary operator is composed of elementaries only of a single
kind. There are thus three kinds of quasielementary—interchange, scaling
and addition—to match the three kinds of elementary. With respect to
interchange and scaling, any sequences of elementaries of the respective kinds
are allowed. With respect to addition, there are some extra rules, explained
in § 11.7.3.
The three subsections which follow respectively introduce the three kinds
of quasielementary operator.
23
This is why some authors forbid self-interchanges, as explained in footnote 19.
11.7. THE QUASIELEMENTARY OPERATOR 267
11.7.1 The interchange quasielementary or general inter-

change operator
Any product P of zero or more interchange elementaries,
Y
P = T[ik ↔jk ] , (11.50)
k
constitutes an interchange quasielementary, permutation matrix, permutor

or general interchange operator.24 An example is
2 3
.. .. .. .. .. ..
6 . . . . . . 7
6
6 ··· 0 0 1 0 0 ··· 7
7
··· 0 0 0 0 1 ···
6 7
6 7
P = T[2↔5] T[1↔3] = 6
6 ··· 1 0 0 0 0 ··· 7.
7
··· 0 0 0 1 0 ···
6 7
6 7
··· 0 1 0 0 0 ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
This operator resembles I in that it has a single one in each row and in each
column, but the ones here do not necessarily run along the main diagonal.
The effect of the operator is to shuffle the rows or columns of the matrix it
operates on, without altering any of the rows or columns it shuffles.
By (11.41), (11.39), (11.43) and (11.15), the inverse of the general inter-
change operator is
!−1
Y a
P −1
= T[ik ↔jk ] = T[i−1
k ↔jk ]
k k
a
= T[ik ↔jk ]
k
!∗
a Y
= T[i∗k ↔jk ] = T[ik ↔jk ]
k k
= P∗ = PT (11.51)
(where P ∗ = P T because P has only real elements). The inverse, transpose

and adjoint of the general interchange operator are thus the same:
P T P = P ∗P = I = P P ∗ = P P T . (11.52)
24
The letter P here recalls the verb “to permute.”
A significant attribute of the general interchange operator P is its par-

ity: positive or even parity if the number of interchange elementaries T[ik ↔jk ]
which compose it is even; negative or odd parity if the number is odd. This
works precisely as described in § 11.6. For the purpose of parity determina-
tion, only interchange elementaries T[ik ↔jk ] for which ik 6= jk are counted;
any T[i↔i] = I noninterchanges are ignored. Thus the example’s P above
has even parity (two interchanges), as does I itself (zero interchanges), but
T[i↔j] alone (one interchange) has odd parity if i 6= j. As we shall see in
§ 14.1, the positive (even) and negative (odd) parities sometimes lend actual
positive and negative senses to the matrices they describe. The parity of
the general interchange operator P concerns us for this reason.
Parity, incidentally, is a property of the matrix P itself, not just of
the operation P represents. No interchange quasielementary P has positive
parity as a row operator but negative as a column operator. The reason
is that, regardless of whether one ultimately means to use P as a row or
column operator, the matrix is nonetheless composable as a definite sequence
of interchange elementaries. It is the number of interchanges, not the use,
which determines P ’s parity.
11.7.2 The scaling quasielementary or general scaling oper-

ator
Like the interchange quasielementary P of § 11.7.1, the scaling quasielemen-

tary, diagonal matrix or general scaling operator D consists of a product of
zero or more elementary operators, in this case elementary scaling opera-
tors:25
2 3
.. .. .. .. .. ..
6 . . . . . . 7
6
6 ··· ∗ 0 0 0 0 ··· 7
7
∞
Y ∞
a ∞
X 6
6 ··· 0 ∗ 0 0 0 ···
7
7
D= Tαi [i] = Tαi [i] = αi Eii = 6
6 ··· 0 0 ∗ 0 0 ···
7
7
··· 0 0 0 ∗ 0 ···
6 7
i=−∞ i=−∞ i=−∞ 6 7
··· 0 0 0 0 ∗ ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
(11.53)
(of course it might be that αi = 1, hence that Tαi [i] = I, for some, most
or even all i; however, αi = 0 is forbidden by the definition of the scaling
25
The letter D here recalls the adjective “diagonal.”
11.7. THE QUASIELEMENTARY OPERATOR 269
elementary). An example is
2 3
.. .. .. .. .. ..
6 . . . . . . 7
··· 7 0 0 0 0 ···
6 7
6 7
··· 0 4 0 0 0 ···
6 7
6 7
D = T−5[4] T4[2] T7[1] = 6
6 ··· 0 0 1 0 0 ··· 7.
7
··· 0 0 0 −5 0 ···
6 7
6 7
··· 0 0 0 0 1 ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
This operator resembles I in that all its entries run down the main diagonal;
but these entries, though never zeros, are not necessarily ones, either. They
are nonzero scaling factors. The effect of the operator is to scale the rows
or columns of the matrix it operates on.
The general scaling operator is a particularly simple matrix. Its inverse
is evidently
∞ ∞ ∞
−1
a Y X Eii
D = T(1/αi )[i] = T(1/αi )[i] = , (11.54)
αi
i=−∞ i=−∞ i=−∞
where each element down the main diagonal is individually inverted.

A superset of the general scaling operator is the diagonal matrix, defined
less restrictively that [A]ij = 0 for i 6= j, where zeros along the main diagonal
are allowed. The conventional notation
[diag{x}]ij ≡ δij xi = δij xj , (11.55)

2 3
.. .. .. .. .. ..
6 . . . . . . 7
··· x1 0 0 0 0 ···
6 7
6 7
··· 0 x2 0 0 0 ···
6 7
6 7
diag{x} = 6
6 ··· 0 0 x3 0 0 ··· 7,
7
··· 0 0 0 x4 0 ···
6 7
6 7
··· 0 0 0 0 x5 ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
converts a vector x into a diagonal matrix. The diagonal matrix in general

is not invertible and is no quasielementary operator, but is sometimes useful
nevertheless.
11.7.3 Addition quasielementaries

Any product of interchange elementaries (§ 11.7.1), any product of scaling
elementaries (§ 11.7.2), qualifies as a quasielementary operator. Not so, any
product of addition elementaries. To qualify as a quasielementary, a product

of elementary addition operators must meet some additional restrictions.
Four types of addition quasielementary are defined:26
• the downward multitarget row addition operator,27
∞
Y ∞
a
L[j] = Tαij [ij] = Tαij [ij] (11.56)
i=j+1 i=j+1
∞
X
= I+ αij Eij
i=j+1
2 3
.. .. .. .. .. ..
6 . . . . . . 7
6
6 ··· 1 0 0 0 0 ··· 7
7
··· 0 1 0 0 0 ···
6 7
6 7
= 6
6 ··· 0 0 1 0 0 ··· 7,
7
··· 0 0 ∗ 1 0 ···
6 7
6 7
··· 0 0 ∗ 0 1 ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
whose inverse is
∞
a ∞
Y
L−1
[j] = T−αij [ij] = T−αij [ij] (11.57)
i=j+1 i=j+1
∞
X
= I− αij Eij = 2I − L[j];
i=j+1
26
In this subsection the explanations are briefer than in the last two, but the pattern is
similar. The reader can fill in the details.
27
The letter L here recalls the adjective “lower.”
11.8. THE UNIT TRIANGULAR MATRIX 271
• the upward multitarget row addition operator,28
j−1
a j−1
Y
U[j] = Tαij [ij] = Tαij [ij] (11.58)
i=−∞ i=−∞
j−1
X
= I+ αij Eij
i=−∞
2 3
.. .. .. .. .. ..
6 . . . . . . 7
6
6 ··· 1 0 ∗ 0 0 ··· 7
7
··· 0 1 ∗ 0 0 ···
6 7
6 7
= 6
6 ··· 0 0 1 0 0 ··· 7,
7
··· 0 0 0 1 0 ···
6 7
6 7
··· 0 0 0 0 1 ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
whose inverse is
j−1
Y j−1
a
−1
U[j] = T−αij [ij] = T−αij [ij] (11.59)
i=−∞ i=−∞
j−1
X
= I− αij Eij = 2I − U[j] ;
i=−∞
• the rightward multitarget column addition operator, which is the trans-

pose LT[j] of the downward operator; and
• the leftward multitarget column addition operator, which is the trans-

T of the upward operator.
pose U[j]
11.8 The unit triangular matrix
Yet more complicated than the quasielementary of § 11.7 is the unit trian-
gular matrix, with which we draw this necessary but tedious chapter toward
28
The letter U here recalls the adjective “upper.”
a long close:
∞
X i−1
X ∞
X ∞
X
L = I+ αij Eij = I + αij Eij (11.60)
i=−∞ j=−∞ j=−∞ i=j+1
2 3
.. .. .. .. .. ..
6 . . . . . . 7
6
6 ··· 1 0 0 0 0 ··· 7
7
··· ∗ 1 0 0 0 ···
6 7
6 7
= 6
6 ··· ∗ ∗ 1 0 0 ··· 7;
7
··· ∗ ∗ ∗ 1 0 ···
6 7
6 7
··· ∗ ∗ ∗ ∗ 1 ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
∞
X ∞
X ∞
X j−1
X
U = I+ αij Eij = I + αij Eij (11.61)
i=−∞ j=i+1 j=−∞ i=−∞
2 3
.. .. .. .. .. ..
6 . . . . . . 7
6
6 ··· 1 ∗ ∗ ∗ ∗ ··· 7
7
··· 0 1 ∗ ∗ ∗ ···
6 7
6 7
= 6
6 ··· 0 0 1 ∗ ∗ ··· 7.
7
··· 0 0 0 1 ∗ ···
6 7
6 7
··· 0 0 0 0 1 ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
The former is a unit lower triangular matrix; the latter, a unit upper tri-
angular matrix. The unit triangular matrix is a generalized addition quasi-
elementary, which adds not only to multiple targets but also from multi-
ple sources—but in one direction only: downward or leftward for L or U T
(or U ∗ ); upward or rightward for U or LT (or L∗ ).
The general triangular matrix LS or US , which by definition can have
any values along its main diagonal, is sometimes of interest, as in the Schur
decomposition of § 14.10.29 The strictly triangular matrix L − I or U − I is
likewise sometimes of interest, as in Table 11.5.30 However, such matrices
cannot in general be expressed as products of elementary operators and this
section does not treat them.
This section presents and derives the basic properties of the unit trian-
gular matrix.
29
The subscript S here stands for Schur. Other books typically use the symbols L and U
for the general triangular matrix of Schur, but this book distinguishes by the subscript.
30
[67, “Schur decomposition,” 00:32, 30 Aug. 2007]
11.8.1 Construction
To make a unit triangular matrix is straightforward:
∞
a
L= L[j] ;
j=−∞
∞ (11.62)
Y
U= U[j] .
j=−∞
So long as the multiplication is done in the order indicated,31 then conve-

niently,
h i h i
L = L[j] ,
ij ij
h i h i (11.63)
U = U[j] ,
ij ij
which is to say that the entries of L and U are respectively nothing more than
the relevant entries of the several L[j] and U[j]. Equation (11.63) enables one
to use (11.62) immediately and directly, without calculation, to build any
unit triangular matrix desired.
The correctness of (11.63) is most easily seen if the several L[j] and U[j]
are regarded as column operators acting sequentially on I:
 
∞
a
L = (I)  L[j]  ;
j=−∞
 
∞
Y
U = (I)  U[j]  .
j=−∞
The reader can construct an inductive proof symbolically on this basis with-
out too much difficulty if desired, but just thinking about how L[j] adds
columns leftward and U[j] , rightward, then considering the order in which
the several L[j] and U[j] act, (11.63) follows at once.
31
Q ‘
Recall again from
Q § 2.3 that k Ak = · · · A3 A2 A1 , whereas k Ak = A1 A2 A3 · · · .
This means that ( ‘k Ak )(C) applies first A1 , then A2 , A3 and so on, as row operators
to C; whereas (C)( Qk Ak ) ‘
applies first A1 , then A2 , A3 and so on, as column operators
to C. The symbols and as this book uses them can thus be thought of respectively
as row and column sequencers.
11.8.2 The product of like unit triangular matrices

The product of like unit triangular matrices,
L1 L2 = L,
(11.64)
U1 U2 = U,
is another unit triangular matrix of the same type. The proof for unit lower
and unit upper triangular matrices is the same. In the unit lower triangular
case, one starts from a form of the definition of a unit lower triangular
matrix: (
0 if i < j,
[L1 ]ij or [L2 ]ij =
1 if i = j.
Then,
∞
X
[L1 L2 ]ij = [L1 ]im [L2 ]mj .
m=−∞
But as we have just observed, [L1 ]im is null when i < m, and [L2 ]mj is null
when m < j. Therefore,
(
0 if i < j,
[L1 L2 ]ij = Pi
m=j [L1 ]im [L2 ]mj if i ≥ j.
Inasmuch as this is true, nothing prevents us from weakening the statement

to read (
0 if i < j,
[L1 L2 ]ij = Pi
m=j [L1 ]im [L2 ]mj if i = j.
But this is just

(
0 if i < j,
[L1 L2 ]ij =
[L1 ]ij [L2 ]ij = [L1 ]ii [L2 ]ii = (1)(1) = 1 if i = j,
which again is the very definition of a unit lower triangular matrix. Hence
(11.64).
11.8.3 Inversion
Inasmuch as any unit triangular matrix can be constructed from addition
quasielementaries by (11.62), inasmuch as (11.63) supplies the specific quasi-
elementaries, and inasmuch as (11.57) or (11.59) gives the inverse of each
such quasielementary, one can always invert a unit triangular matrix easily
by
∞
Y
L −1
= L−1
[j] ,
j=−∞
∞ (11.65)
a
−1 −1
U = U[j] .
j=−∞
In view of (11.64), therefore, the inverse of a unit lower triangular matrix

is another unit lower triangular matrix; and the inverse of a unit upper
triangular matrix, another unit upper triangular matrix.
It is plain to see but still interesting to note that—unlike the inverse—
the adjoint or transpose of a unit lower triangular matrix is a unit upper
triangular matrix; and that the adjoint or transpose of a unit upper triangu-
lar matrix is a unit lower triangular matrix. The adjoint reverses the sense
of the triangle.
11.8.4 The parallel unit triangular matrix
If a unit triangular matrix fits the special, restricted form
k ∞
{k}
X X
Lk = I+ αij Eij (11.66)
j=−∞ i=k+1
2 3
.. .. .. .. .. ..
6 . . . . . . 7
6
6 ··· 1 0 0 0 0 ··· 7
7
··· 0 1 0 0 0 ···
6 7
6 7
= 6
6 ··· 0 0 1 0 0 ···
7
7
··· ∗ ∗ ∗ 1 0 ···
6 7
6 7
··· ∗ ∗ ∗ 0 1 ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
or
∞ X
k−1
{k}
X
Uk = I+ αij Eij (11.67)
j=k i=−∞
2 3
.. .. .. .. .. ..
6 . . . . . . 7
6
6 ··· 1 0 ∗ ∗ ∗ ··· 7
7
··· 0 1 ∗ ∗ ∗ ···
6 7
6 7
= 6
6 ··· 0 0 1 0 0 ··· 7,
7
··· 0 0 0 1 0 ···
6 7
6 7
··· 0 0 0 0 1 ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
confining its nonzero elements to a rectangle within the triangle as shown,

then it is a parallel unit triangular matrix and has some special properties
the general unit triangular matrix lacks.
The general unit lower triangular matrix L acting LA on a matrix A
adds the rows of A downward. The parallel unit lower triangular matrix
{k} {k}
Lk acting Lk A also adds rows downward, but with the useful restriction
that it makes no row of A both source and target. The addition is from A’s
rows through the kth, to A’s (k + 1)th row onward. A horizontal frontier
separates source from target, which thus march in A as separate squads.
Similar observations naturally apply with respect to the parallel unit
{k} {k}
upper triangular matrix Uk , which acting Uk A adds rows upward, and
{k}T {k}T {k}T {k}T
also with respect to Lk and Uk , which acting ALk and AUk
{k}T
add columns respectively rightward and leftward (remembering that Lk
{k}T
is no unit lower but a unit upper triangular matrix; that Uk is the lower).
Each separates source from target in the matrix A it operates on.
The reason we care about the separation of source from target is that,
in matrix arithmetic generally, where source and target are not separate
but remain intermixed, the sequence matters in which rows or columns are
added. That is, in general,
Tα1 [i1 j1 ] Tα2 [i2 j2 ] 6= I + α1 Ei1 j1 + α2 Ei2 j2 6= Tα2 [i2 j2 ] Tα1 [i1 j1 ] .
It makes a difference whether the one addition comes before, during or after
the other—but only because the target of the one addition might be the
source of the other. The danger is that i1 = j2 or i2 = j1 . Remove this
danger, and the sequence ceases to matter (refer to Table 11.3).
That is exactly what the parallel unit triangular matrix does: it separates
source from target and thus removes the danger. It is for this reason that
the parallel unit triangular matrix brings the useful property that
k ∞
{k}
X X
Lk =I+ αij Eij
j=−∞ i=k+1
k
a ∞
Y k
a ∞
a
= Tαij [ij] = Tαij [ij]
j=−∞ i=k+1 j=−∞ i=k+1
k
Y ∞
Y k
Y ∞
a
= Tαij [ij] = Tαij [ij]
j=−∞ i=k+1 j=−∞ i=k+1
∞
Y k
a ∞
a k
a
= Tαij [ij] = Tαij [ij] (11.68)
i=k+1 j=−∞ i=k+1 j=−∞
∞
Y k
Y ∞
a k
Y
= Tαij [ij] = Tαij [ij] ,
i=k+1 j=−∞ i=k+1 j=−∞
∞ X
k−1
{k}
X
Uk =I+ αij Eij
j=k i=−∞
∞ k−1
Y a
= Tαij [ij] = · · · ,
j=k i=−∞
which says that one can build a parallel unit triangular matrix equally well
in any sequence—in contrast to the case of the general unit triangular ma-
trix, whose construction per (11.62) one must sequence carefully. (Though
eqn. 11.68 does not show them, even more sequences are possible. You can
scramble the factors’ ordering any random way you like. The multiplication
is fully commutative.) Under such conditions, the inverse of the parallel unit
triangular matrix is particularly simple:32
k ∞
{k} −1 {k}
X X
Lk =I− αij Eij = 2I − Lk
j=−∞ i=k+1
k
Y ∞
a
= T−αij [ij] = · · · ,
j=−∞ i=k+1
(11.69)
∞ X
k−1
{k} −1 {k}
X
Uk =I− αij Eij = 2I − Uk
j=k i=−∞
∞
a k−1
Y
= T−αij [ij] = · · · ,
j=k i=−∞
where again the elementaries can be multiplied in any order. Pictorially,

2 3
.. .. .. .. .. ..
6 . . . . . . 7
··· 1 0 0 0 0 ···
6 7
6 7
··· 0 1 0 0 0 ···
6 7
{k} −1
6 7
Lk = 6
6 ··· 0 0 1 0 0 ··· 7,
7
··· −∗ −∗ −∗ 1 0 ···
6 7
6 7
··· −∗ −∗ −∗ 0 1 ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
2 3
.. .. .. .. .. ..
6 . . . . . . 7
6
6 ··· 1 0 −∗ −∗ −∗ ··· 7
7
··· 0 1 −∗ −∗ −∗ ···
6 7
{k} −1
6 7
Uk = 6
6 ··· 0 0 1 0 0 ··· 7.
7
··· 0 0 0 1 0 ···
6 7
6 7
··· 0 0 0 0 1 ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
The inverse of a parallel unit triangular matrix is just the matrix itself,
only with each element off the main diagonal negated. Table 11.5 records a
few properties that come immediately of the last observation and from the
parallel unit triangular matrix’s basic layout.
32
There is some odd parochiality at play in applied mathematics when one calls such
collections of symbols as (11.69) “particularly simple.” Nevertheless, in the present context
the idea (11.69) represents is indeed simple: that one can multiply constituent elementaries
in any order and still reach the same parallel unit triangular matrix; that the elementaries
in this case do not interfere.
Table 11.5: Properties of the parallel unit triangular matrix. (In the table,
the notation Iab represents the generalized dimension-limited indentity ma-
{k} −1 {k}′
trix or truncator of eqn. 11.30. Note that the inverses Lk = Lk and
{k} −1 {k}′
Uk = Uk are parallel unit triangular matrices themselves, such that
the table’s properties hold for them, too.)
{k} {k} −1 {k} {k} −1
Lk + Lk Uk + Uk
=I =
2 2
∞ k{k} {k} ∞ {k} k
Ik+1 Lk I−∞ = Lk −I = Ik+1 (Lk − I)I−∞
k−1 {k} {k} k−1 {k}
I−∞ Uk Ik∞ = Uk − I = I−∞ (Uk − I)Ik∞
{k}
If Lk honors an n × n active region, then
{k} {k} {k}
(In − Ik )Lk Ik = Lk − I = (In − Ik )(Lk − I)Ik
{k} {k}
and (I − In )(Lk − I) = 0 = (Lk − I)(I − In ).
{k}
If Uk honors an n × n active region, then
{k} {k} {k}
Ik−1 Uk (In − Ik−1 ) = Uk − I = Ik−1 (Uk − I)(In − Ik−1 )
{k} {k}
and (I − In )(Uk − I) = 0 = (Uk − I)(I − In ).
11.8.5 The partial unit triangular matrix
Besides the notation L and U for the general unit lower and unit upper
{k} {k}
triangular matrices and the notation Lk and Uk for the parallel unit
lower and unit upper triangular matrices, we shall find it useful to introduce
the additional notation
∞ X
X ∞
[k]
L = I+ αij Eij (11.70)
j=k i=j+1
2 3
.. .. .. .. .. ..
6 . . . . . . 7
··· 1 0 0 0 0 ···
6 7
6 7
··· 0 1 0 0 0 ···
6 7
6 7
= 6
6 ··· 0 0 1 0 0 ··· 7,
7
··· 0 0 ∗ 1 0 ···
6 7
6 7
··· 0 0 ∗ ∗ 1 ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
k
X j−1
X
[k]
U = I+ αij Eij (11.71)
j=−∞ i=−∞
2 3
.. .. .. .. .. ..
6 . . . . . . 7
6
6 ··· 1 ∗ ∗ 0 0 ··· 7
7
··· 0 1 ∗ 0 0 ···
6 7
6 7
= 6
6 ··· 0 0 1 0 0 ···
7
7
··· 0 0 0 1 0 ···
6 7
6 7
··· 0 0 0 0 1 ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
for unit triangular matrices whose off-diagonal content is confined to a nar-

row wedge and
k
X ∞
X
L{k} = I + αij Eij (11.72)
j=−∞ i=j+1
2 3
.. .. .. .. .. ..
6 . . . . . . 7
6
6 ··· 1 0 0 0 0 ··· 7
7
··· ∗ 1 0 0 0 ···
6 7
6 7
= 6
6 ··· ∗ ∗ 1 0 0 ··· 7,
7
··· ∗ ∗ ∗ 1 0 ···
6 7
6 7
··· ∗ ∗ ∗ 0 1 ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
j−1
∞ X
X
{k}
U = I+ αij Eij (11.73)
j=k i=−∞
2 3
.. .. .. .. .. ..
6 . . . . . . 7
··· 1 0 ∗ ∗ ∗ ···
6 7
6 7
··· 0 1 ∗ ∗ ∗ ···
6 7
6 7
= 6
6 ··· 0 0 1 ∗ ∗ ···
7
7
··· 0 0 0 1 ∗ ···
6 7
6 7
··· 0 0 0 0 1 ···
6 7
6 7
4 .. .. .. .. .. ..
5
. . . . . .
for the supplementary forms.33 Such notation is not standard in the liter-
ature, but it serves a purpose in this book and is introduced here for this
reason. If names are needed for L[k], U [k] , L{k} and U {k} , the former pair can
be called minor partial unit triangular matrices, and the latter pair, major
partial unit triangular matrices. Whether minor or major, the partial unit
triangular matrix is a matrix which leftward or rightward of the kth column
resembles I. Of course partial unit triangular matrices which resemble I
above or below the kth row are equally possible, and can be denoted L[k]T ,
U [k]T , L{k}T and U {k}T .
{k} {k}
Observe that the parallel unit triangular matrices Lk and Uk of
§ 11.8.4 are in fact also major partial unit triangular matrices, as the nota-
tion suggests.
33 {k}
The notation is arguably imperfect in that LP + L[k] − P
I 6= L but rather that
{k} [k+1]
− I = L. The conventional notation k=a f (k) + ck=b f (k) 6= ck=a f (k)
b P
L +L
suffers the same arguable imperfection.
11.9 The shift operator

Not all useful matrices fit the dimension-limited and extended-operational
forms of § 11.3. An exception is the shift operator Hk , defined that
[Hk ]ij = δi(j+k) . (11.74)
For example, 2 3
.. .. .. .. .. .. ..
6 . . . . . . . 7
6
6 ··· 0 0 0 0 0 0 0 ··· 7
7
··· 0 0 0 0 0 0 0 ···
6 7
6 7
··· 1 0 0 0 0 0 0 ···
6 7
6 7
H2 = 6
6 ··· 0 1 0 0 0 0 0 ··· 7.
7
··· 0 0 1 0 0 0 0 ···
6 7
6 7
··· 0 0 0 1 0 0 0 ···
6 7
6 7
··· 0 0 0 0 1 0 0 ···
6 7
6 7
.. .. .. .. .. .. ..
4 5
. . . . . . .
Operating Hk A, Hk shifts A’s rows downward k steps. Operating AHk , Hk
shifts A’s columns leftward k steps. Inasmuch as the shift operator shifts
all rows or columns of the matrix it operates on, its active region is ∞ × ∞
in extent. Obviously, the shift operator’s inverse, transpose and adjoint are
the same:
HkT Hk = Hk∗ Hk = I = Hk Hk∗ = Hk HkT ,
(11.75)
Hk−1 = HkT = Hk∗ = H−k .
Further obvious but useful identities include that
(Iℓ − Ik )Hk = Hk Iℓ−k ,
(11.76)
H−k (Iℓ − Ik ) = Iℓ−k H−k .
11.10 The Jacobian derivative

Chapter 4 has introduced the derivative of a function with respect to a scalar
variable. One can also take the derivative of a function with respect to a
vector variable, and the function itself can be vector-valued. The derivative
is
df ∂fi
= . (11.77)
dx ij ∂xj
For instance, if x has three elements and f has two, then
2 3
∂f1 ∂f1 ∂f1
df 6 ∂x ∂x2 ∂x3 7
= 6 1 7.
dx 4 ∂f
2 ∂f2 ∂f2 5
∂x1 ∂x2 ∂x3
11.10. THE JACOBIAN DERIVATIVE 283
This is called the Jacobian derivative, the Jacobian matrix, or just the Ja-
cobian.34 Each of its columns is the derivative with respect to one element
of x.
The Jacobian derivative of a vector with respect to itself is
dx
= I. (11.78)
dx
The derivative is not In as one might think, because, even if x has only n
elements, still, one could vary xn+1 in principle, and ∂xn+1 /∂xn+1 6= 0.
The Jacobian derivative obeys the derivative product rule (4.25) in the
form35
" # " T #T
d T df dg
g Af = gTA + Af ,
dx dx dx
" # " ∗ #T (11.79)
d ∗ df dg
g Af = g∗ A

+ Af ,
dx dx dx
valid for any constant matrix A—as is seen by applying the definition (4.19)
of the derivative, which here is
∂ (g∗ Af ) (g + ∂g/2)∗ A(f + ∂f /2) − (g − ∂g/2)∗ A(f − ∂f /2)

= lim ,
∂xj ∂xj →0 ∂xj
and simplifying.
The shift operator of § 11.9 and the Jacobian derivative of this section
complete the family of matrix rudiments we shall need to begin to do in-
creasingly interesting things with matrices in Chs. 13 and 14. Before doing
interesting things, however, we must treat two more foundational matrix
matters. The two are the Gauss-Jordan decomposition and the matter of
matrix rank, which will be the subjects of Ch. 12, next.
34
[67, “Jacobian,” 00:50, 15 Sept. 2007]
35
Notice that the last term on (11.79)’s second line is transposed, not adjointed.
Chapter 12
Matrix rank and the

Gauss-Jordan decomposition
Chapter 11 has brought the matrix and its rudiments, the latter including
• lone-element matrix E (§ 11.3.7),
• the null matrix 0 (§ 11.3.1),
• the rank-r identity matrix Ir (§ 11.3.5),
• the general identity matrix I and the scalar matrix λI (§ 11.3.2),
• the elementary operator T (§ 11.4),
• the quasielementary operator P , D, L[k] or U[k] (§ 11.7), and
• the unit triangular matrix L or U (§ 11.8).
Such rudimentary forms have useful properties, as we have seen. The general
matrix A does not necessarily have any of these properties, but it turns out
that one can factor any matrix whatsoever into a product of rudiments which
do have the properties, and that several orderly procedures are known to
do so. The simplest of these, and indeed one of the more useful, is the
Gauss-Jordan decomposition. This chapter introduces it.
Section 11.3 has deëmphasized the concept of matrix dimensionality m×
n, supplying in its place the new concept of matrix rank. However, that
section has actually defined rank only for the rank-r identity matrix Ir . In
fact all matrices have rank. This chapter explains.
285
286 CHAPTER 12. RANK AND THE GAUSS-JORDAN
Before treating the Gauss-Jordan decomposition and the matter of ma-

trix rank as such, however, we shall find it helpful to prepare two preliminar-
ies thereto: (i) the matter of the linear independence of vectors; and (ii) the
elementary similarity transformation. The chapter begins with these.
Except in § 12.2, the chapter demands more rigor than one likes in such
a book as this. However, it is hard to see how to avoid the rigor here, and
logically the chapter cannot be omitted. We shall drive through the chapter
in as few pages as can be managed, and then onward to the more interesting
matrix topics of Chs. 13 and 14.
12.1 Linear independence

Linear independence is a significant possible property of a set of vectors—
whether the set be the several columns of a matrix, the several rows, or
some other vectors—the property being defined as follows. A vector is lin-
early independent if its role cannot be served by the other vectors in the
set. More formally, the n vectors of the set {a1 , a2 , a3 , a4 , a5 , . . . , an } are
linearly independent if and only if none of them can be expressed as a linear
combination—a weighted sum—of the others. That is, the several ak are
linearly independent iff
α1 a1 + α2 a2 + α3 a3 + · · · + αn an 6= 0 (12.1)
for all nontrivial αk , where “nontrivial αk ” means the several αk , at least

one of which is nonzero (trivial αk , by contrast, would be α1 = α2 = α3 =
· · · = αn = 0). Vectors which can combine nontrivially to reach the null
vector are by definition linearly dependent.
Linear independence is a property of vectors. Technically the property
applies to scalars, too, inasmuch as a scalar resembles a one-element vector—
so, any nonzero scalar alone is linearly independent—but there is no such
thing as a linearly independent pair of scalars, because one of the pair can
always be expressed as a complex multiple of the other. Significantly but
less obviously, there is also no such thing as a linearly independent set which
includes the null vector; (12.1) forbids it. Paradoxically, even the single-
member, n = 1 set consisting only of a1 = 0 is, strictly speaking, not
linearly independent.
For consistency of definition, we regard the empty, n = 0 set as linearly
independent, on the technical ground that the only possible linear combina-
tion of the empty set is trivial.1
1
This is the kind of thinking which typically governs mathematical edge cases. One
12.1. LINEAR INDEPENDENCE 287
If a linear combination of several independent vectors ak forms a vec-

tor b, then one might ask: can there exist a different linear combination of
the same vectors ak which also forms b? That is, if
β1 a1 + β2 a2 + β3 a3 + · · · + βn an = b,
where the several ak satisfy (12.1), then is
β1′ a1 + β2′ a2 + β3′ a3 + · · · + βn′ an = b
possible? To answer the question, suppose that it were possible. The differ-
ence of the two equations then would be
(β1′ − β1 )a1 + (β2′ − β2 )a2 + (β3′ − β3 )a3 + · · · + (βn′ − βn )an = 0.
According to (12.1), this could only be so if the coefficients in the last

equation where trivial—that is, only if β1′ − β1 = 0, β2′ − β2 = 0, β3′ −
β3 = 0, . . . , βn′ − βn = 0. But this says no less than that the two linear
combinations, which we had supposed to differ, were in fact one and the
same. One concludes therefore that, if a vector b can be expressed as a
linear combination of several linearly independent vectors ak , then it cannot
be expressed as any other combination of the same vectors. The combination
is unique.
Linear independence can apply in any dimensionality, but it helps to
visualize the concept geometrically in three dimensions, using the three-
dimensional geometrical vectors of § 3.3. Two such vectors are independent
so long as they do not lie along the same line. A third such vector is
independent of the first two so long as it does not lie in their common plane.
A fourth such vector (unless it points off into some unvisualizable fourth
dimension) cannot possibly then be independent of the three.
We discuss the linear independence of vectors in this, a chapter on ma-
trices, because (§ 11.1) a matrix is essentially a sequence of vectors—either
of column vectors or of row vectors, depending on one’s point of view. As
we shall see in § 12.5, the important property of matrix rank depends on
the number of linearly independent columns or rows a matrix has.
could define the empty set to be linearly dependent if one really wanted to, but what
then of the observation that adding a vector to a linearly dependent set never renders
the set independent? Surely in this light it is preferable justP
to define the empty set as
independent in the first place. Similar thinking makes 0! = 1, −1 k
k=0 ak z = 0, and 2 not 1
the least prime, among other examples.
12.2 The elementary similarity transformation

Section 11.5 and its (11.46) have introduced the similarity transformation
CAC −1 or C −1 AC, which arises when an operator C commutes respectively
rightward or leftward past a matrix A. The similarity transformation has
several interesting properties, some of which we are now prepared to discuss,
particularly in the case in which the operator happens to be an elementary,
C = T . In this case, the several rules of Table 12.1 obtain.
Most of the table’s rules are fairly obvious if the meaning of the symbols
is understood, though to grasp some of the rules it helps to sketch the
relevant matrices on a sheet of paper. Of course rigorous symbolic proofs
can be constructed after the pattern of § 11.8.2, but they reveal little or
nothing sketching the matrices does not. The symbols P , D, L and U of
course represent the quasielementaries and unit triangular matrices of §§ 11.7
and 11.8. The symbols P ′ , D ′ , L′ and U ′ also represent quasielementaries
and unit triangular matrices, only not necessarily the same ones P , D, L
and U do.
The rules permit one to commute some but not all elementaries past a
quasielementary or unit triangular matrix, without fundamentally altering
the character of the quasielementary or unit triangular matrix, and some-
times without altering the matrix at all. The rules find use among other
places in the Gauss-Jordan decomposition of § 12.3.
12.3 The Gauss-Jordan decomposition

The Gauss-Jordan decomposition of an arbitrary, dimension-limited, m × n
matrix A is2
A = G> Ir G< = P DLU Ir KS,

G< ≡ KS, (12.2)
G> ≡ P DLU,
where
• P and S are general interchange operators (§ 11.7.1);

2
Most introductory linear algebra texts this writer has met call the Gauss-Jordan
decomposition instead the “LU decomposition” and include fewer factors in it, typically
merging D into L and omitting K and S. They also omit Ir , since their matrices have
pre-defined dimensionality. Perhaps the reader will agree that the decomposition is cleaner
as presented here.
12.3. THE GAUSS-JORDAN DECOMPOSITION 289
Table 12.1: Some elementary similarity transformations.
T[i↔j] IT[i↔j] = I
T[i↔j]P T[i↔j] = P ′
T[i↔j]DT[i↔j] = D ′ = D + ([D]jj − [D]ii ) Eii + ([D]ii − [D]jj ) Ejj
T[i↔j]DT[i↔j] = D if [D]ii = [D]jj
[k]
T[i↔j]L T[i↔j] = L[k] if i < k and j < k
T[i↔j]U [k] T[i↔j] = U [k] if i > k and j > k
T[i↔j] L{k} T[i↔j] = L {k} ′ if i > k and j > k
T[i↔j] U {k}
T[i↔j] = U {k} ′ if i < k and j < k
{k} {k} ′
T[i↔j] Lk T[i↔j] = Lk if i > k and j > k
{k} {k} ′
T[i↔j] Uk T[i↔j] = Uk if i < k and j < k
Tα[i] IT(1/α)[i] = I
Tα[i] DT(1/α)[i] = D
Tα[i] AT(1/α)[i] = A′ where A is any of
{k} {k}
L, U, L[k] , U [k] , L{k} , U {k} , Lk , Uk
Tα[ij] IT−α[ij] = I
Tα[ij] DT−α[ij] = D + ([D]jj − [D]ii ) αEij 6= D ′
Tα[ij] DT−α[ij] = D if [D]ii = [D]jj
Tα[ij] LT−α[ij] = L′ if i > j
Tα[ij] U T−α[ij] = U′ if i < j
• D is a general scaling operator (§ 11.7.2);
• L and U are respectively unit lower and unit upper triangular matrices
(§ 11.8);
{r}T
• K = Lk is the transpose of a parallel unit lower triangular matrix,
being thus a parallel unit upper triangular matrix (§ 11.8.4);
• G> and G< are composites3 as defined by (12.2); and
• r is an unspecified rank.
The Gauss-Jordan decomposition is also called the Gauss-Jordan factoriza-

tion.
Whether all possible matrices A have a Gauss-Jordan decomposition
(they do, in fact) is a matter this section addresses. However—at least for
matrices which do have one—because G> and G< are composed of invertible
factors, one can left-multiply the equation A = G> Ir G< by G−1
> and right-
−1
multiply it by G< to obtain
U −1 L−1 D−1 P −1 AS −1 K −1 = G−1 −1

> AG< = Ir ,
S −1 K −1 = G−1
< , (12.3)
−1 −1 −1 −1
U L D P = G−1
> ,
the Gauss-Jordan’s complementary form.
12.3.1 Motive
Equation (12.2) seems inscrutable. The equation itself is easy enough to
read, but just as there are many ways to factor a scalar (0xC = [4][3] =
[2]2 [3] = [2][6], for example), there are likewise many ways to factor a matrix.
Why choose this particular way?
There are indeed many ways. We shall meet some of the others in
§§ 13.11, 14.6, 14.10 and 14.12. The Gauss-Jordan decomposition we meet
here however has both significant theoretical properties and useful practical
applications, and in any case needs less advanced preparation to appreciate
than the others, and (at least as developed in this book) precedes the oth-
ers logically. It emerges naturally when one posits a pair of square, n × n
3
One can pronounce G> and G< respectively as “G acting rightward” and “G acting
leftward.” The letter G itself can be regarded as standing for “Gauss-Jordan,” but ad-
mittedly it is chosen as much because otherwise we were running out of available Roman
capitals!
matrices, A and A−1 , for which A−1 A = In , where A is known and A−1 is
to be determined. (The A−1 here is the A−1(n) of eqn. 11.49. However, it is
only supposed here that A−1 A = In ; it is not yet claimed that AA−1 = In .)
To determine A−1 is not an entirely trivial problem. The matrix A−1
such that A−1 A = In may or may not exist (usually it does exist if A is
square, but even then it may not, as we shall soon see), and even if it does
exist, how to determine it is not immediately obvious. And still, if one
can determine A−1 , that is only for square A; what if A is not square? In
the present subsection however we are not trying to prove anything, only
to motivate, so for the moment let us suppose a square A for which Q A−1
−1
does exist, and let us seek A by left-multiplying A by a sequence T
of elementary row operators, each of which makes the matrix more nearly
resemble In . When In is finally achieved, then we shall have that
Y
T (A) = In ,
or, left-multiplying by In and observing that In2 = In ,

Y
(In ) T (A) = In ,
which implies that Y

A−1 = (In ) T .
The product of elementaries which transforms A to In , truncated (§ 11.3.6)
to n × n dimensionality, itself constitutes A−1 . This observation is what
motivates the Gauss-Jordan decomposition.
By successive steps,4 a concrete example:
» –
2 −4
A = 3 −1
,
1
» – » –
0 1 −2
2
0 1
A = 3 −1
,
1
» –» – » –
1 0 0 1 −2
−3 1
2
0 1
A = 0 5
,
1
» –» –» – » –
1 0 1 0 0 1 −2
0 1
−3 1
2
0 1
A = 0 1
,
5
1
» –» –» –» – » –
1 2 1 0 1 0 0 1 0
1
2 A = ,
0 1 0 5
−3 1 0 1 0 1
1
» –» –» –» –» – » –
1 0 1 2 1 0 1 0 0 1 0
0 1 0 1 0 1
−3 1
2
0 1
A = 0 1
.
5
4
Theoretically, all elementary operators including the ones here have extended-
operational form (§ 11.3.2), but all those · · · ellipses clutter the page too much. Only
the 2 × 2 active regions are shown here.
Hence,
" #" #" #" #" # " #
1 1 2
−1 1 0 1 2 1 0 1 0 0 −A
A = 1
2
= 3
5
1
.
0 1 0 1 0 5
−3 1 0 1 −A 5
Using the elementary commutation identity that Tβ[m] Tα[mj] = Tαβ[mj] Tβ[m] ,
from Table 11.2, to group like operators, we have that
" #" #" #" #" # " #
1 1 2
−1 1 0 1 2 1 0 1 0 0 −A
A = 2
= 5
;
0 1 0 1 − 35 1 0 1
5
0 1 3
−A 1
5
or, multiplying the two scaling elementaries to merge them into a single
general scaling operator (§ 11.7.2),
" #" #" #" # " #
1 1 2
1 0 1 2 1 0 0 −A
A−1 = 2
= 5
.
0 1 0 1 − 35 1 0 1
5
3
−A 1
5
The last equation is written symbolically as
A−1 = I2 U −1 L−1 D−1 ,
from which
» –» –» –» – » –
2 0 1 0 1 −2 1 0 2 −4
A = DLU I2 = 0 5 3
1 0 1 0 1
= 3 −1
.
5
Now, admittedly, the equation A = DLU I2 is not (12.2)—or rather, it

is (12.2), but only in the special case that r = 2 and P = S = K = I—which
begs the question: why do we need the factors P , S and K in the first place?
The answer regarding P and S is that these factors respectively gather row
and column interchange elementaries, of which the example given has used
none but which other examples sometimes need or want, particularly to
avoid dividing by zero when they encounter a zero in an inconvenient cell of
the matrix (the reader might try reducing A = [0 1; 1 0] to I2 , for instance; a
row or column interchange is needed here). Regarding K, this factor comes
into play when A has broad rectangular rather than square shape, and also
sometimes when one of the rows of A happens to be a linear combination
of the others. The last point, we are not quite ready to detail yet, but
at present we are only motivating not proving, so if the reader will accept
the other factors and suspend judgment on K until the actual need for it
emerges in § 12.3.3, step 12, then we shall proceed on this basis.
12.3.2 Method
The Gauss-Jordan decomposition of a matrix A is not discovered at one
stroke but rather is gradually built up, elementary by elementary. It begins
with the equation
A = IIIIAII,
where the six I hold the places of the six Gauss-Jordan factors P , D, L, U ,
K and S of (12.2). By successive elementary operations, the A on the right
is gradually transformed into Ir , while the six I are gradually transformed
into the six Gauss-Jordan factors. The decomposition thus ends with the
equation
A = P DLU Ir KS,
which is (12.2). In between, while the several matrices are gradually being
transformed, the equation is represented as
A = P̃ D̃ L̃Ũ I˜K̃ S̃, (12.4)
where the initial value of I˜ is A and the initial values of P̃ , D̃, etc., are all I.
Each step of the transformation goes as follows. The matrix I˜ is left- or
right-multiplied by an elementary operator T . To compensate, one of the
six factors is right- or left-multiplied by T −1 . Intervening factors are mul-
tiplied by both T and T −1 , which multiplication constitutes an elementary
similarity transformation as described in § 12.2. For example,

A = P̃ D̃T(1/α)[i] Tα[i] L̃T(1/α)[i] Tα[i] Ũ T(1/α)[i] Tα[i] I˜ K̃ S̃,
which is just (12.4), inasmuch as the adjacent elementaries cancel one an-
other; then,
I˜ ← Tα[i] I,
˜
Ũ ← Tα[i] Ũ T(1/α)[i] ,
L̃ ← Tα[i] L̃T(1/α)[i] ,
D̃ ← D̃T(1/α)[i] ,
thus associating the operation with the appropriate factor—in this case, D̃.
Such elementary row and column operations are repeated until I˜ = Ir , at
which point (12.4) has become the Gauss-Jordan decomposition (12.2).
12.3.3 The algorithm

Having motivated the Gauss-Jordan decomposition in § 12.3.1 and having
proposed a basic method to pursue it in § 12.3.2, we shall now establish a
definite, orderly, failproof algorithm to achieve it. Broadly, the algorithm
• copies A into the variable working matrix I˜ (step 1 below),
• reduces I˜ by suitable row (and maybe column) operations to unit

upper triangular form (steps 2 through 7),
• establishes a rank r (step 8), and
• reduces the now unit triangular I˜ further to the rank-r identity ma-
trix Ir (steps 9 through 13).
Specifically, the algorithm decrees the following steps. (The steps as written
include many parenthetical remarks—so many that some steps seem to con-
sist more of parenthetical remarks than of actual algorithm. The remarks
are unnecessary to execute the algorithm’s steps as such. They are however
necessary to explain and to justify the algorithm’s steps to the reader.)
1. Begin by initializing
P̃ ← I, D̃ ← I, L̃ ← I, Ũ ← I, K̃ ← I, S̃ ← I,
I˜ ← A,
i ← 1,
where I˜ holds the part of A remaining to be decomposed, where i is

a row index, and where the others are the variable working matrices
of (12.4). (The eventual goal will be to factor all of I˜ away, leaving
I˜ = Ir , though the precise value of r will not be known until step 8.
Since A by definition is a dimension-limited m×n matrix, one naturally
need not store A beyond the m × n active region. What is less clear
until one has read the whole algorithm, but nevertheless true, is that
one also need not store the dimension-limited I˜ beyond the m×n active
region. The other six variable working matrices each have extended-
operational form, but they also confine their activity to well defined
regions: m × m for P̃ , D̃, L̃ and Ũ ; n × n for K̃ and S̃. One need store
none of the matrices beyond these bounds.)
2. (Besides arriving at this point from step 1 above, the algorithm also
reënters here from step 7 below. From step 1, I˜ = A and L̃ = I, so
this step 2 though logical seems unneeded. The need grows clear once
one has read through step 7.) Observe that neither the ith row of I˜
nor any row below it has an entry left of the ith column, that I˜ is
all-zero below-leftward of and directly leftward of (though not directly
below) the pivot element ı̃ ii .5 Observe also that above the ith row, the
matrix has proper unit upper triangular form (§ 11.8). Regarding the
other factors, notice that L̃ enjoys the major partial unit triangular
form L{i−1} (§ 11.8.5) and that d˜kk = 1 for all k ≥ i. Pictorially,
2 3
.. .. .. .. .. .. .. ..
6 . . . . . . . . 7
6
6 ··· ∗ 0 0 0 0 0 0 ··· 7
7
6
6 ··· 0 ∗ 0 0 0 0 0 ··· 7
7
··· 0 0 ∗ 0 0 0 0 ···
6 7
6 7
D̃ = 6
6 ··· 0 0 0 1 0 0 0 ··· 7,
7
··· 0 0 0 0 1 0 0 ···
6 7
6 7
··· 0 0 0 0 0 1 0 ···
6 7
6 7
··· 0 0 0 0 0 0 1 ···
6 7
6 7
.. .. .. .. .. .. .. ..
4 5
. . . . . . . .
2 3
.. .. .. .. .. .. .. ..
6 . . . . . . . . 7
6
6 ··· 1 0 0 0 0 0 0 ··· 7
7
6
6 ··· ∗ 1 0 0 0 0 0 ··· 7
7
··· ∗ ∗ 1 0 0 0 0 ···
6 7
6 7
L̃ = L{i−1} = 6
6 ··· ∗ ∗ ∗ 1 0 0 0 ··· 7,
7
··· ∗ ∗ ∗ 0 1 0 0 ···
6 7
6 7
··· ∗ ∗ ∗ 0 0 1 0 ···
6 7
6 7
··· ∗ ∗ ∗ 0 0 0 1 ···
6 7
6 7
.. .. .. .. .. .. .. ..
4 5
. . . . . . . .
2 3
.. .. .. .. .. .. .. ..
6 . . . . . . . . 7
6
6 ··· 1 ∗ ∗ ∗ ∗ ∗ ∗ ··· 7
7
6
6 ··· 0 1 ∗ ∗ ∗ ∗ ∗ ··· 7
7
··· 0 0 1 ∗ ∗ ∗ ∗ ···
6 7
I˜ =
6 7
··· 0 0 0 ∗ ∗ ∗ ∗ ··· 7,
6 7
6
··· 0 0 0 ∗ ∗ ∗ ∗ ···
6 7
6 7
··· 0 0 0 ∗ ∗ ∗ ∗ ···
6 7
6 7
··· 0 0 0 ∗ ∗ ∗ ∗ ···
6 7
6 7
.. .. .. .. .. .. .. ..
4 5
. . . . . . . .
where the ith row and ith column are depicted at center.
5
The notation ı̃ ii looks interesting, but this is accidental. The ı̃ relates not to the
doubled, subscribed index ii but to I.˜ The notation ı̃ ii thus means [I]
˜ ii —in other words,
it means the current iith element of the variable working matrix I. ˜
3. Choose a nonzero element ı̃ pq 6= 0 on or below the pivot row, where

p ≥ i and q ≥ i. (The easiest choice may simply be ı̃ ii , where p =
q = i, if ı̃ ii 6= 0; but any nonzero element from the ith row downward
can in general be chosen. Beginning students of the Gauss-Jordan
or LU decomposition are conventionally taught to choose first the least
possible q then the least possible p. When one has no reason to choose
otherwise, that is as good a choice as any. There is however no actual
need to choose so. In fact alternate choices can sometimes improve
practical numerical accuracy.6,7 Theoretically nonetheless, when doing
exact arithmetic, the choice is quite arbitrary, so long as ı̃ pq 6= 0.) If
no nonzero element is available—if all remaining rows p ≥ i are now
null—then skip directly to step 8.
4. Observing that (12.4) can be expanded to read

A = P̃ T[p↔i] T[p↔i] D̃T[p↔i] T[p↔i] L̃T[p↔i] T[p↔i] Ũ T[p↔i]

× T[p↔i]IT˜ [i↔q] T[i↔q] K̃T[i↔q] T[i↔q] S̃

= P̃ T[p↔i] D̃ T[p↔i] L̃T[p↔i] Ũ

× T[p↔i]IT˜ [i↔q] K̃ T[i↔q] S̃ ,
6
A typical Intel or AMD x86-class computer processor represents a C/C++ double-
type floating-point number, x = 2p b, in 0x40 bits of computer memory. Of the 0x40
bits, 0x34 are for the number’s mantissa 2.0 ≤ b < 4.0 (not 1.0 ≤ b < 2.0 as one might
expect), 0xB are for the number’s exponent −0x3FF ≤ p ≤ 0x3FE, and one is for the
number’s ± sign. (The mantissa’s high-order bit, which is always 1, is implied not stored,
thus is one neither of the 0x34 nor of the 0x40 bits.) The out-of-bounds exponents p =
−0x400 and p = 0x3FF serve specially respectively to encode 0 and ∞. All this is standard
computing practice. Such a floating-point representation is easily accurate enough for most
practical purposes, but of course it is not generally exact. [33, § 1-4.2.2]
7
The Gauss-Jordan’s floating-point errors come mainly from dividing by small pivots.
Such errors are naturally avoided by avoiding small pivots, at least until as late in the
algorithm as possible. Smallness however is relative: a small pivot in a row and a column
each populated by even smaller elements is unlikely to cause as much error as is a large
pivot in a row and a column each populated by even larger elements.
To choose a pivot, any of several heuristics are reasonable. The following heuristic if
programmed intelligently might not be too computationally expensive: Define the pivot-
smallness metric
2 2ı̃ ∗pqı̃ pq
η̃pq ≡ Pm ∗ Pn ∗ .
p′ =i ı̃ p′ qı̃ p′ q + q ′ =i ı̃ pq ′ ı̃ pq ′
2
Choose the p and q of least η̃pq . If two are equally least, then choose first the lesser column
index q, then if necessary the lesser row index p.
let
P̃ ← P̃ T[p↔i],
L̃ ← T[p↔i]L̃T[p↔i] ,
I˜ ← T[p↔i]IT
˜ [i↔q] ,
S̃ ← T[i↔q] S̃,
thus interchanging the pth with the ith row and the qth with the
ith column, to bring the chosen element to the pivot position. (Re-
fer to Table 12.1 for the similarity transformations. The Ũ and K̃
transformations disappear because at this stage of the algorithm, still
Ũ = K̃ = I. The D̃ transformation disappears because p ≥ i and
because d˜kk = 1 for all k ≥ i. Regarding the L̃ transformation, it does
not disappear, but L̃ has major partial unit triangular form L{i−1} ,
which form according to Table 12.1 it retains since i − 1 < i ≤ p.)

A = P̃ D̃Tı̃ ii [i] T(1/ı̃ ii )[i] L̃Tı̃ ii [i] T(1/ı̃ ii )[i] Ũ Tı̃ ii [i]

× T(1/ı̃ ii )[i] I˜ K̃ S̃

= P̃ D̃Tı̃ ii [i] T(1/ı̃ ii )[i] L̃Tı̃ ii [i] Ũ T(1/ı̃ ii )[i] I˜ K̃ S̃,
normalize the new ı̃ ii pivot by letting
D̃ ← D̃Tı̃ ii [i] ,
L̃ ← T(1/ı̃ ii )[i] L̃Tı̃ ii [i] ,
I˜ ← T(1/ı̃ )[i] I.
ii
˜
This forces ı̃ ii = 1. It also changes the value of d˜ii . Pictorially after

this step,
2 3
.. .. .. .. .. .. .. ..
6 . . . . . . . . 7
6
6 ··· ∗ 0 0 0 0 0 0 ··· 7
7
6
6 ··· 0 ∗ 0 0 0 0 0 ··· 7
7
··· 0 0 ∗ 0 0 0 0 ···
6 7
6 7
D̃ = 6
6 ··· 0 0 0 ∗ 0 0 0 ··· 7,
7
··· 0 0 0 0 1 0 0 ···
6 7
6 7
··· 0 0 0 0 0 1 0 ···
6 7
6 7
··· 0 0 0 0 0 0 1 ···
6 7
6 7
.. .. .. .. .. .. .. ..
4 5
. . . . . . . .
2 3
.. .. .. .. .. .. .. ..
6 . . . . . . . . 7
6
6 ··· 1 ∗ ∗ ∗ ∗ ∗ ∗ ··· 7
7
6
6 ··· 0 1 ∗ ∗ ∗ ∗ ∗ ··· 7
7
··· 0 0 1 ∗ ∗ ∗ ∗ ···
6 7
I˜ =
6 7
··· 0 0 0 1 ∗ ∗ ∗ ··· 7.
6 7
6
··· 0 0 0 ∗ ∗ ∗ ∗ ···
6 7
6 7
··· 0 0 0 ∗ ∗ ∗ ∗ ···
6 7
6 7
··· 0 0 0 ∗ ∗ ∗ ∗ ···
6 7
6 7
.. .. .. .. .. .. .. ..
4 5
. . . . . . . .
(Though the step changes L̃, too, again it leaves L̃ in the major partial
unit triangular form L{i−1} , because i − 1 < i. Refer to Table 12.1.)

A = P̃ D̃ L̃Tı̃ pi [pi] T−ı̃ pi [pi] Ũ Tı̃ pi [pi] T−ı̃ pi [pi] I˜ K̃ S̃

˜
= P̃ D̃ L̃Tı̃ pi [pi] Ũ T−ı̃ pi [pi] I K̃ S̃,
˜ ith column below the pivot by letting

clear I’s
 
m
a
L̃ ← L̃  Tı̃ pi [pi]  ,
p=i+1
 
m
Y
I˜ ←  T−ı̃ pi [pi]  I˜ .
p=i+1
This forces ı̃ ip = 0 for all p > i. It also fills in L̃’s ith column below the
pivot, advancing that matrix from the L{i−1} form to the L{i} form.
Pictorially,
2 3
.. .. .. .. .. .. .. ..
6 . . . . . . . . 7
6
6 ··· 1 0 0 0 0 0 0 ··· 7
7
6
6 ··· ∗ 1 0 0 0 0 0 ··· 7
7
··· ∗ ∗ 1 0 0 0 0 ···
6 7
6 7
L̃ = L{i} = 6
6 ··· ∗ ∗ ∗ 1 0 0 0 ··· 7,
7
··· ∗ ∗ ∗ ∗ 1 0 0 ···
6 7
6 7
··· ∗ ∗ ∗ ∗ 0 1 0 ···
6 7
6 7
··· ∗ ∗ ∗ ∗ 0 0 1 ···
6 7
6 7
.. .. .. .. .. .. .. ..
4 5
. . . . . . . .
2 3
.. .. .. .. .. .. .. ..
6 . . . . . . . . 7
6
6 ··· 1 ∗ ∗ ∗ ∗ ∗ ∗ ··· 7
7
6
6 ··· 0 1 ∗ ∗ ∗ ∗ ∗ ··· 7
7
··· 0 0 1 ∗ ∗ ∗ ∗ ···
6 7
I˜ =
6 7
··· 0 0 0 1 ∗ ∗ ∗ ··· 7.
6 7
6
··· 0 0 0 0 ∗ ∗ ∗ ···
6 7
6 7
··· 0 0 0 0 ∗ ∗ ∗ ···
6 7
6 7
··· 0 0 0 0 ∗ ∗ ∗ ···
6 7
6 7
.. .. .. .. .. .. .. ..
4 5
. . . . . . . .
(Note that it is not necessary actually to apply the addition elemen-

taries here one by one. Together they easily form an addition quasiel-
ementary L[i] , thus can be applied all at once. See § 11.7.3.)
7. Increment
i ← i + 1.
Go back to step 2.
8. Decrement
i←i−1
to undo the last instance of step 7 (even if there never was an instance
of step 7), thus letting i point to the matrix’s last nonzero row. After
decrementing, let the rank
r ≡ i.
Notice that, certainly, r ≤ m and r ≤ n.
reënters here from step 11 below.) If i = 0, then skip directly to
step 12.

A = P̃ D̃L̃ Ũ Tı̃ pi [pi] T−ı̃ pi [pi] I˜ K̃ S̃,
˜ ith column above the pivot by letting

clear I’s
 
Y i−1
Ũ ← Ũ  Tı̃ pi [pi]  ,
p=1
 
i−1
a
I˜ ←  T−ı̃ pi [pi]  I˜ .
p=1
This forces ı̃ ip = 0 for all p 6= i. It also fills in Ũ ’s ith column above the
pivot, advancing that matrix from the U {i+1} form to the U {i} form.
Pictorially,
2 3
.. .. .. .. .. .. .. ..
6 . . . . . . . . 7
6
6 ··· 1 0 0 ∗ ∗ ∗ ∗ ··· 7
7
6
6 ··· 0 1 0 ∗ ∗ ∗ ∗ ··· 7
7
··· 0 0 1 ∗ ∗ ∗ ∗ ···
6 7
6 7
Ũ = U {i} = 6
6 ··· 0 0 0 1 ∗ ∗ ∗ ··· 7,
7
··· 0 0 0 0 1 ∗ ∗ ···
6 7
6 7
··· 0 0 0 0 0 1 ∗ ···
6 7
6 7
··· 0 0 0 0 0 0 1 ···
6 7
6 7
.. .. .. .. .. .. .. ..
4 5
. . . . . . . .
2 3
.. .. .. .. .. .. .. ..
6 . . . . . . . . 7
6
6 ··· 1 ∗ ∗ 0 0 0 0 ··· 7
7
6
6 ··· 0 1 ∗ 0 0 0 0 ··· 7
7
··· 0 0 1 0 0 0 0 ···
6 7
I˜ =
6 7
··· 0 0 0 1 0 0 0 ··· 7.
6 7
6
··· 0 0 0 0 1 0 0 ···
6 7
6 7
··· 0 0 0 0 0 1 0 ···
6 7
6 7
··· 0 0 0 0 0 0 1 ···
6 7
6 7
.. .. .. .. .. .. .. ..
4 5
. . . . . . . .
(As in step 6, here again it is not necessary actually to apply the ad-
dition elementaries one by one. Together they easily form an addition
quasielementary U[i] . See § 11.7.3.)
11. Decrement i ← i − 1. Go back to step 9.

12. Notice that I˜ now has the form of a rank-r identity matrix, except
with n − r extra columns dressing its right edge (often r = n however;
then there are no extra columns). Pictorially,
2 3
.. .. .. .. .. .. .. ..
6 . . . . . . . . 7
6
6 ··· 1 0 0 0 ∗ ∗ ∗ ··· 7
7
··· 0 1 0 0 ∗ ∗ ∗ ···
6 7
6 7
··· 0 0 1 0 ∗ ∗ ∗ ···
6 7
I˜ =
6 7
··· 0 0 0 1 ∗ ∗ ∗ ··· 7.
6 7
6
··· 0 0 0 0 0 0 0 ···
6 7
6 7
··· 0 0 0 0 0 0 0 ···
6 7
6 7
··· 0 0 0 0 0 0 0 ···
6 7
6 7
.. .. .. .. .. .. .. ..
4 5
. . . . . . . .
Observing that (12.4) can be expanded to read

˜ −ı̃ [pq] Tı̃ [pq] K̃ S̃,
A = P̃ D̃ L̃Ũ IT pq pq
use the now conveniently elementarized columns of I’s ˜ main body to

suppress the extra columns on its right edge by
 
n r
I˜ ← I˜ 
Y a
T−ı̃ pq [pq]  ,
q=r+1 p=1
 
n
a r
Y
K̃ ←  Tı̃ pq [pq] K̃ .
q=r+1 p=1
(Actually, entering this step, it was that K̃ = I, so in fact K̃ becomes

just the product above. As in steps 6 and 10, here again it is not neces-
sary actually to apply the addition elementaries one by one. Together
they easily form a parallel unit upper—not lower—triangular matrix
{r}T
Lk . See § 11.8.4.)
13. Notice now that I˜ = Ir . Let
P ≡ P̃ , D ≡ D̃, L ≡ L̃, U ≡ Ũ , K ≡ K̃, S ≡ S̃.
End.
Never stalling, the algorithm cannot fail to achieve I˜ = Ir and thus a com-
plete Gauss-Jordan decomposition of the form (12.2), though what value
the rank r might turn out to have is not normally known to us in advance.
(We have not yet proven, but shall in § 12.5, that the algorithm always pro-
duces the same Ir , the same rank r ≥ 0, regardless of which pivots ı̃ pq 6= 0
one happens to choose in step 3 along the way. We can safely ignore this
unproven fact however for the immediate moment.)
12.3.4 Rank and independent rows

Observe that the Gauss-Jordan algorithm of § 12.3.3 operates always within
the bounds of the original m × n matrix A. Therefore, necessarily,
r ≤ m,
(12.5)
r ≤ n.
The rank r exceeds the number neither of the matrix’s rows nor of its
columns. This is unsurprising. Indeed the narrative of the algorithm’s
step 8 has already noticed the fact.
Observe also however that the rank always fully reaches r = m if the
rows of the original matrix A are linearly independent. The reason for this
observation is that the rank can fall short, r < m, only if step 3 finds a null
row i ≤ m; but step 3 can find such a null row only if step 6 has created one
(or if there were a null row in the original matrix A; but according to § 12.1,
such null rows never were linearly independent in the first place). How do
we know that step 6 can never create a null row? We know this because the
action of step 6 is to add multiples only of current and earlier pivot rows
to rows in I˜ which have not yet been on pivot.8 According to (12.1), such
action has no power to cancel the independent rows it targets.
8
If the truth of the sentence’s assertion regarding the action of step 6 seems nonobvious,
one can drown the assertion rigorously in symbols to prove it, but before going to that
extreme consider: The action of steps 3 and 4 is to choose a pivot row p ≥ i and to
shift it upward to the ith position. The action of step 6 then is to add multiples of the
chosen pivot row downward only—that is, only to rows which have not yet been on pivot.
This being so, steps 3 and 4 in the second iteration find no unmixed rows available to
choose as second pivot, but find only rows which already include multiples of the first
pivot row. Step 6 in the second iteration therefore adds multiples of the second pivot row
downward, which already include multiples of the first pivot row. Step 6 in the ith iteration
adds multiples of the ith pivot row downward, which already include multiples of the first
through (i − 1)th. So it comes to pass that multiples only of current and earlier pivot rows
are added to rows which have not yet been on pivot. To no row is ever added, directly
or indirectly, a multiple of itself—until step 10, which does not belong to the algorithm’s
main loop and has nothing to do with the availability of nonzero rows to step 3.
12.3.5 Inverting the factors

Inverting the six Gauss-Jordan factors is easy. Sections 11.7 and 11.8 have
shown how. One need not however go even to that much trouble. Each Q
of the six factors—P , D, L, U , K and S—is composed of a sequence T
of elementary operators. Each of the six inverse factors—P −1 , D−1` , L−1 ,
U −1 , K −1 and S −1 —is therefore composed of the reverse sequence T −1
of inverse elementary operators. Refer to (11.41). If one merely records the
sequence of elementaries used to build each of the six factors—if one reverses
each sequence, inverts each elementary, and multiplies—then the six inverse
factors result.
And, in fact, it isn’t even that hard. One actually need not record the
individual elementaries; one can invert, multiply and forget them in stream.
This means starting the algorithm from step 1 with six extra variable work-
ing matrices (besides the seven already there):
P̃ −1 ← I; D̃−1 ← I; L̃−1 ← I; Ũ −1 ← I; K̃ −1 ← I; S̃ −1 ← I.
(There is no I˜−1 , not because it would not be useful, but because its initial
value would be9 A−1(r) , unknown at algorithm’s start.) Then, for each
operation on any of P̃ , D̃, L̃, Ũ , K̃ or S̃, one operates inversely on the
corresponding inverse matrix. For example, in step 5,
D̃ ← D̃Tı̃ ii [i] , D̃ −1 ← T(1/ı̃ ii )[i] D̃ −1 ,

L̃ ← T(1/ı̃ ii )[i] L̃Tı̃ ii [i] , L̃−1 ← T(1/ı̃ ii )[i] L̃−1 Tı̃ ii [i] ,
I˜ ← T(1/ı̃ ii )[i] I.
˜
With this simple extension, the algorithm yields all the factors not only
of the Gauss-Jordan decomposition (12.2) but simultaneously also of the
Gauss-Jordan’s complementary form (12.3).
12.3.6 Truncating the factors

None of the six factors of (12.2) actually needs to retain its entire extended-
operational form (§ 11.3.2). The four factors on the left, row operators, act
wholly by their m × m squares; the two on the right, column operators, by
their n × n. Indeed, neither Ir nor A has anything but zeros outside the
m × n rectangle, anyway, so there is nothing for the six operators to act
upon beyond those bounds in any event. We can truncate all six operators
to dimension-limited forms (§ 11.3.1) for this reason if we want.
9
Section 11.5 explains the notation.
To truncate the six operators formally, we left-multiply (12.2) by Im and

right-multiply it by In , obtaining
Im AIn = Im P DLU Ir KSIn .
According to § 11.3.6, the Im and In respectively truncate rows and columns,

actions which have no effect on A since it is already a dimension-limited m×n
matrix. By successive steps, then,
A = Im P DLU Ir KSIn
7
= Im P DLU Ir2 KSIn3 ;
and finally, by using (11.31) or (11.42) repeatedly,
A = (Im P Im )(Im DIm )(Im LIm )(Im U Ir )(Ir KIn )(In SIn ), (12.6)
where the dimensionalities of the six factors on the equation’s right side are
respectively m × m, m × m, m × m, m × r, r × n and n × n. Equation (12.6)
expresses any dimension-limited rectangular matrix A as the product of six
particularly simple, dimension-limited rectangular factors.
By similar reasoning from (12.2),
A = (Im G> Ir )(Ir G< In ), (12.7)
where the dimensionalities of the two factors are m × r and r × n.

The book will seldom point it out again explicitly, but one can straight-
forwardly truncate not only the Gauss-Jordan factors but most other factors
and operators, too, by the method of this subsection.10
10
Comes the objection, “Black, why do you make it more complicated than it needs to
be? For what reason must all your matrices have infinite dimensionality, anyway? They
don’t do it that way in my other linear algebra book.”
It is a fair question. The answer is that this book is a book of applied mathematical the-
ory; and theoretically in the author’s view, infinite-dimensional matrices are significantly
neater to handle. To append a null row or a null column to a dimension-limited matrix
is to alter the matrix in no essential way, nor is there any real difference between T5[21]
when it row-operates on a 3 × p matrix and the same elementary when it row-operates on
a 4 × p. The relevant theoretical constructs ought to reflect such insights. Hence infinite
dimensionality.
Anyway, a matrix displaying an infinite field of zeros resembles a shipment delivering
an infinite supply of nothing; one need not be too impressed with either. The two matrix
forms of § 11.3 manifest the sense that a matrix can represent a linear transformation,
whose rank matters; or a reversible row or column operation, whose rank does not. The
extended-operational form, having infinite rank, serves the latter case. In either case,
however, the dimensionality m × n of the matrix is a distraction. It is the rank r, if any,
that counts.
Table 12.2: A few properties of the Gauss-Jordan factors.
P ∗ = P −1 = P T
S ∗ = S −1 = S T
P −∗ = P = P −T
S −∗ = S = S −T
K + K −1
=I
2
Ir K(In − Ir ) = K −I = Ir (K − I)(In − Ir )
Ir K −1 (In − Ir ) = K −1 − I = Ir (K −1 − I)(In − Ir )
(I − In )(K − I) = 0 = (K − I)(I − In )
(I − In )(K −1 − I) = 0 = (K −1 − I)(I − In )
12.3.7 Properties of the factors

One would expect such neatly formed operators as the factors of the Gauss-
Jordan to enjoy some useful special properties. Indeed they do. Table 12.2
lists a few. The table’s properties formally come from (11.52) and Table 11.5;
but, if one firmly grasps the matrix forms involved and comprehends the
notation (neither of which is trivial to do), if one understands that the
operator (In − Ir ) is a truncator that selects columns r + 1 through n of the
matrix it operates leftward upon, and if one sketches the relevant factors
schematically with a pencil, then the properties are plainly seen without
reference to Ch. 11 as such.
The table’s properties regarding P and S express a general advantage
all permutors share. The table’s properties regarding K are admittedly less
significant, included mostly only because § 13.3 will need them. Still, even
the K properties are always true. They might find other uses.
Further properties of the several Gauss-Jordan factors can be gleaned
from the respectively relevant subsections of §§ 11.7 and 11.8.
12.3.8 Marginalizing the factor In

If A happens to be a square, n × n matrix and if it develops that the rank
r = n, then one can take advantage of (11.31) to rewrite the Gauss-Jordan
decomposition (12.2) in the form
P DLU KSIn = A = In P DLU KS, (12.8)
thus marginalizing the factor In . This is to express the Gauss-Jordan solely

in row operations or solely in column operations. It does not change the
algorithm and it does not alter the factors; it merely reorders the factors
after the algorithm has determined them. It fails however if A is rectangular
or r < n.
12.3.9 Decomposing an extended operator

Once one has derived the Gauss-Jordan decomposition, to extend it to de-
compose a reversible, n × n extended operator A (where per § 11.3.2 A
outside the n × n active region resembles the infinite-dimensional identity
matrix I) is trivial. One merely writes
A = P DLU KS,
wherein the Ir has become an I. Or, equivalently, one decomposes the n × n

dimension-limited matrix In A = In AIn = AIn as
AIn = P DLU In KS = P DLU KSIn ,
from which, inasmuch as all the factors present but In are n × n extended
operators, the preceding equation results.
One can decompose only reversible extended operators so. The Gauss-
Jordan fails on irreversible extended operators, for which the rank of the
truncated form AIn is r < n. See § 12.5.
This subsection’s equations remain unnumbered because they say little
new. Their only point, really, is that what an operator does outside some
appropriately delimited active region is seldom interesting, because the vec-
tor on which the operator ultimately acts is probably null there in any event.
In such a context it may not matter whether one truncates the operator.
Indeed this was also the point of § 12.3.6 and, if you like, of (11.31), too.11
11
If “it may not matter,” as the narrative says, then one might just put all matrices
in dimension-limited form. Many books do. To put them all in dimension-limited form
however brings at least three effects the book you are reading prefers to avoid. First, it
leaves shift-and-truncate operations hard to express cleanly (refer to §§ 11.3.6 and 11.9
and, as a typical example of the usage, eqn. 13.7). Second, it confuses the otherwise natural
extension of discrete vectors into continuous functions. Third, it leaves one to consider
the ranks of reversible operators like T[1↔2] that naturally should have no rank. The last
12.4. VECTOR REPLACEMENT 307
Regarding the present section as a whole, the Gauss-Jordan decomposi-

tion is a significant achievement. It is not the only matrix decomposition—
further interesting decompositions include the Gram-Schmidt of § 13.11, the
diagonal of § 14.6, the Schur of § 14.10 and the singular-value of § 14.12,
among others—but the Gauss-Jordan nonetheless reliably factors an arbi-
trary m × n matrix A, which we had not known how to handle very well,
into a product of unit triangular matrices and quasielementaries, which we
do. We shall put the Gauss-Jordan to good use in Ch. 13. However, before
closing the present chapter we should like finally, squarely to define and to
establish the concept of matrix rank, not only for Ir but for all matrices. To
do that, we shall first need one more preliminary: the technique of vector
replacement.
12.4 Vector replacement

Consider a set of m + 1 (not necessarily independent) vectors
{u, a1 , a2 , . . . , am }.
As a definition, the space these vectors address consists of all linear combina-
tions of the set’s several vectors. That is, the space consists of all vectors b
formable as
βo u + β1 a1 + β2 a2 + · · · + βm am = b. (12.9)
Now consider a specific vector v in the space,
ψo u + ψ1 a1 + ψ2 a2 + · · · + ψm am = v, (12.10)
for which
ψo 6= 0.
Solving (12.10) for u, we find that
1 ψ1 ψ2 ψm
v− a1 − a2 − · · · − am = u.
ψo ψo ψo ψo
of the three is arguably most significant: matrix rank is such an important attribute that
one prefers to impute it only to those operators about which it actually says something
interesting.
Nevertheless, the extended-operational matrix form is hardly more than a formality. All
it says is that the extended operator unobtrusively leaves untouched anything it happens
to find outside its operational domain, whereas a dimension-limited operator would have
truncated whatever it found there. Since what is found outside the operational domain is
often uninteresting, this may be a distinction without a difference, which one can safely
ignore.
With the change of variables
1
φo ← ,
ψo
ψ1
φ1 ← − ,
ψo
ψ2
φ2 ← − ,
ψo
..
.
ψm
φm ← − ,
ψo
for which, quite symmetrically, it happens that
1
ψo = ,
φo
φ1
ψ1 = − ,
φo
φ2
ψ2 = − ,
φo
..
.
φm
ψm = − ,
φo
the solution is
φo v + φ1 a1 + φ2 a2 + · · · + φm am = u. (12.11)
Equation (12.11) has identical form to (12.10), only with the symbols u ↔ v
and ψ ↔ φ swapped. Since φo = 1/ψo , assuming that ψo is finite it even
appears that
φo 6= 0;
so, the symmetry is complete. Table 12.3 summarizes.
Now further consider an arbitrary vector b which lies in the space ad-
dressed by the vectors
{u, a1 , a2 , . . . , am }.
Does the same b also lie in the space addressed by the vectors
{v, a1 , a2 , . . . , am }?
12.4. VECTOR REPLACEMENT 309
Table 12.3: The symmetrical equations of § 12.4.
ψo u + ψ1 a1 + ψ2 a2 φo v + φ1 a1 + φ2 a2
+ · · · + ψm am = v + · · · + φm am = u
1 1
0 6= = φo 0 6= = ψo
ψo φo
ψ1 φ1
− = φ1 − = ψ1
ψo φo
ψ2 φ2
− = φ2 − = ψ2
ψo φo
.. ..
. .
ψm φm
− = φm − = ψm
ψo φo
To show that it does, we substitute into (12.9) the expression for u from
(12.11), obtaining the form
(βo )(φo v + φ1 a1 + φ2 a2 + · · · + φm am ) + β1 a1 + β2 a2 + · · · + βm am = b.
Collecting terms, this is
βo φo v + (βo φ1 + β1 )a1 + (βo φ2 + β2 )a2 + · · · + (βo φm + βm )am = b,
in which we see that, yes, b does indeed also lie in the latter space. Nat-
urally the problem’s u ↔ v symmetry then guarantees the converse, that
an arbitrary vector b which lies in the latter space also lies in the former.
Therefore, a vector b must lie in both spaces or neither, never just in one
or the other. The two spaces are, in fact, one and the same.
This leads to the following useful conclusion. Given a set of vectors
{u, a1 , a2 , . . . , am },
one can safely replace the u by a new vector v, obtaining the new set
{v, a1 , a2 , . . . , am },
provided that the replacement vector v includes at least a little of the re-
placed vector u (ψo 6= 0 in eqn. 12.10) and that v is otherwise an honest
linear combination of the several vectors of the original set, untainted by

foreign contribution. Such vector replacement does not in any way alter the
space addressed. The new space is exactly the same as the old.
As a corollary, if the vectors of the original set happen to be linearly in-
dependent (§ 12.1), then the vectors of the new set are linearly independent,
too; for, if it were that
γo v + γ1 a1 + γ2 a2 + · · · + γm am = 0
for nontrivial γo and γk , then either γo = 0—impossible since that would

make the several ak themselves linearly dependent—or γo 6= 0, in which
case v would be a linear combination of the several ak alone. But if v were
a linear combination of the several ak alone, then (12.10) would still also
explicitly make v a linear combination of the same ak plus a nonzero multiple
of u. Yet both combinations cannot be, because according to § 12.1, two
distinct combinations of independent vectors can never target the same v.
The contradiction proves false the assumption which gave rise to it: that
the vectors of the new set were linearly dependent. Hence the vectors of the
new set are equally as independent as the vectors of the old.
12.5 Rank
Sections 11.3.5 and 11.3.6 have introduced the rank-r identity matrix Ir ,
where the integer r is the number of ones the matrix has along its main
diagonal. Other matrices have rank, too. Commonly, an n × n matrix has
rank r = n, but consider the matrix
2 3
5 1 6
4 3 6 9 5.
2 4 6
The third column of this matrix is the sum of the first and second columns.
Also, the third row is just two-thirds the second. Either way, by columns
or by rows, the matrix has only two independent vectors. The rank of this
3 × 3 matrix is not r = 3 but r = 2.
This section establishes properly the important concept of matrix rank.
The section demonstrates that every matrix has a definite, unambiguous
rank, and shows how this rank can be calculated.
To forestall potential confusion in the matter, we should immediately
observe that—like the rest of this chapter but unlike some other parts of the
book—this section explicitly trades in exact numbers. If a matrix element
12.5. RANK 311
here is 5, then it is exactly 5; if 0, then exactly 0. Many real-world matrices,

of course—especially matrices populated by measured data—can never truly
be exact, but that is not the point here. Here, the numbers are exact.12
12.5.1 A logical maneuver
In § 12.5.2 we shall execute a pretty logical maneuver, which one might

name, “the end justifies the means.”13 When embedded within a larger
logical construct as in § 12.5.2, the maneuver if unexpected can confuse, so
this subsection is to prepare the reader to expect the maneuver.
The logical maneuver follows this basic pattern.
If P1 then Q. If P2 then Q. If P3 then Q. Which of P1 , P2

and P3 are true is not known, but what is known is that at least
one of the three is true: P1 or P2 or P3 . Therefore, although
one can draw no valid conclusion regarding any one of the three
predicates P1 , P2 or P3 , one can still conclude that their common
object Q is true.
One valid way to prove Q, then, would be to suppose P1 and show that it
led to Q, then alternately to suppose P2 and show that it separately led
to Q, then again to suppose P3 and show that it also led to Q. The final
step would be to show somehow that P1 , P2 and P3 could not possibly all be
false at once. Herein, the means is to assert several individually suspicious
claims, none of which one actually means to prove. The end which justifies
the means is the conclusion Q, which thereby one can and does prove.
It is a subtle maneuver. Once the reader feels that he grasps its logic,
he can proceed to the next subsection where the maneuver is put to use.
12
It is false to suppose that because applied mathematics permits imprecise quantities,
like 3.0 ± 0.1 inches for the length of your thumb, it also requires them. On the contrary,
the length of your thumb may indeed be 3.0±0.1 inches, but surely no triangle has 3.0±0.1
sides! A triangle has exactly three sides. The ratio of a circle’s circumference to its radius
is exactly 2π. The author has exactly one brother. A construction contract might require
the builder to finish within exactly 180 days (though the actual construction time might
be an inexact t = 172.6 ± 0.2 days), and so on. Exact quantities are every bit as valid
in applied mathematics as imprecise ones are. Where the distinction matters, it is the
applied mathematician’s responsibility to distinguish between the two kinds of quantity.
13
The maneuver’s name rings a bit sinister, does it not? The author recommends no
such maneuver in social or family life! Logically here however it helps the math.
12.5.2 The impossibility of identity-matrix promotion

Consider the matrix equation
AIr B = Is . (12.12)
If r ≥ s, then it is trivial to find matrices A and B for which (12.12) holds:

A = Is = B. If
r < s,
however, it is not so easy. In fact it is impossible. This subsection proves
the impossibility. It shows that one cannot by any row and column opera-
tions, reversible or otherwise, ever transform an identity matrix into another
identity matrix of greater rank (§ 11.3.5).
Equation (12.12) can be written in the form
(AIr )B = Is , (12.13)
where, because Ir attacking from the right is the column truncation oper-
ator (§ 11.3.6), the product AIr is a matrix with an unspecified number of
rows but only r columns—or, more precisely, with no more than r nonzero
columns. Viewed this way, per § 11.1.3, B operates on the r columns of AIr
to produce the s columns of Is .
The r columns of AIr are nothing more than the first through rth
columns of A. Let the symbols a1 , a2 , a3 , a4 , a5 , . . . , ar denote these columns.
The s columns of Is , then, are nothing more than the elementary vectors
e1 , e2 , e3 , e4 , e5 , . . . , es (§ 11.3.7). The claim (12.13) makes is thus that
the several vectors ak together address each of the several elementary vec-
tors ej —that is, that a linear combination14
b1j a1 + b2j a2 + b3j a3 + · · · + brj ar = ej (12.14)
exists for each ej , 1 ≤ j ≤ s.

The claim (12.14) will turn out to be false because there are too many ej ,
but to prove this, we shall assume for the moment that the claim were true.
The proof then is by contradiction,15 and it runs as follows.
14
Observe that unlike as in § 12.1, here we have not necessarily assumed that the
several ak are linearly independent.
15
As the reader will have observed by this point in the book, the technique—also called
reductio ad absurdum—is the usual mathematical technique to prove impossibility. One
assumes the falsehood to be true, then reasons toward a contradiction which proves the
assumption false. Section 6.1.1 among others has already illustrated the technique, but
the technique’s use here is more sophisticated.
12.5. RANK 313
Consider the elementary vector e1 . For j = 1, (12.14) is
b11 a1 + b21 a2 + b31 a3 + · · · + br1 ar = e1 ,
which says that the elementary vector e1 is a linear combination of the

several vectors
{a1 , a2 , a3 , a4 , a5 , . . . , ar }.
Because e1 is a linear combination, according to § 12.4 one can safely replace

any of the vectors in the set by e1 without altering the space addressed. For
example, replacing a1 by e1 ,
{e1 , a2 , a3 , a4 , a5 , . . . , ar }.
The only restriction per § 12.4 is that e1 contain at least a little of the
vector ak it replaces—that bk1 6= 0. Of course there is no guarantee specif-
ically that b11 6= 0, so for e1 to replace a1 might not be allowed. However,
inasmuch as e1 is nonzero, then according to (12.14) at least one of the sev-
eral bk1 also is nonzero; and if bk1 is nonzero then e1 can replace ak . Some
of the ak might indeed be forbidden, but never all; there is always at least
one ak which e1 can replace. (For example, if a1 were forbidden because
b11 = 0, then a3 might be available instead because b31 6= 0. In this case the
new set would be {a1 , a2 , e1 , a4 , a5 , . . . , ar }.)
Here is where the logical maneuver of § 12.5.1 comes in. The book to this
point has established no general method to tell which of the several ak the
elementary vector e1 actually contains (§ 13.2 gives the method, but that
section depends logically on this one, so we cannot licitly appeal to it here).
According to (12.14), the vector e1 might contain some of the several ak
or all of them, but surely it contains at least one of them. Therefore, even
though it is illegal to replace an ak by an e1 which contains none of it, even
though we have no idea which of the several ak the vector e1 contains, even
though replacing the wrong ak logically invalidates any conclusion which
flows from the replacement, still we can proceed with the proof, provided
that the false choice and the true choice lead ultimately alike to the same
identical end. If they do, then all the logic requires is an assurance that
there does exist at least one true choice, even if we remain ignorant as to
which choice that is.
The claim (12.14) guarantees at least one true choice. Whether as the
maneuver also demands, all the choices, true and false, lead ultimately alike
to the same identical end remains to be determined.
Now consider the elementary vector e2 . According to (12.14), e2 lies in

the space addressed by the original set
{a1 , a2 , a3 , a4 , a5 , . . . , ar }.
Therefore as we have seen, e2 also lies in the space addressed by the new set
{e1 , a2 , a3 , a4 , a5 , . . . , ar }
(or {a1 , a2 , e1 , a4 , a5 , . . . , ar }, or whatever the new set happens to be). That

is, not only do coefficients bk2 exist such that
b12 a1 + b22 a2 + b32 a3 + · · · + br2 ar = e2 ,
but also coefficients βk2 exist such that
β12 e1 + β22 a2 + β32 a3 + · · · + βr2 ar = e2 .
Again it is impossible for all the coefficients βk2 to be zero, but moreover,
it is impossible for β12 to be the sole nonzero coefficient, for (as should
seem plain to the reader who grasps the concept of the elementary vector,
§ 11.3.7) no elementary vector can ever be a linear combination of other
elementary vectors alone! The linear combination which forms e2 evidently
includes a nonzero multiple of at least one of the remaining ak . At least
one of the βk2 attached to an ak (not β12 , which is attached to e1 ) must be
nonzero. Therefore by the same reasoning as before, we now choose an ak
with a nonzero coefficient βk2 6= 0 and replace it by e2 , obtaining an even
newer set of vectors like
{e1 , a2 , a3 , e2 , a5 , . . . , ar }.
This newer set addresses precisely the same space as the previous set, thus
also as the original set.
And so it goes, replacing one ak by an ej at a time, until all the ak are
gone and our set has become
{e1 , e2 , e3 , e4 , e5 , . . . , er },
which, as we have reasoned, addresses precisely the same space as did the
original set
{a1 , a2 , a3 , a4 , a5 , . . . , ar }.
And this is the one identical end the maneuver of § 12.5.1 has demanded. All
intermediate choices, true and false, ultimately lead to the single conclusion
of this paragraph, which thereby is properly established.
12.5. RANK 315
The conclusion leaves us with a problem, however. There are more ej ,

1 ≤ j ≤ s, than there are ak , 1 ≤ k ≤ r, because, as we have stipulated,
r < s. Some elementary vectors ej , r < j ≤ s, are evidently left over. Back
at the beginning of the section, the claim (12.14) made was that
{a1 , a2 , a3 , a4 , a5 , . . . , ar }
together addressed each of the several elementary vectors ej . But as we

have seen, this amounts to a claim that
{e1 , e2 , e3 , e4 , e5 , . . . , er }
together addressed each of the several elementary vectors ej . Plainly this

is impossible with respect to the left-over ej , r < j ≤ s. The contradiction
proves false the claim which gave rise to it. The false claim: that the
several ak , 1 ≤ k ≤ r, addressed all the ej , 1 ≤ j ≤ s, even when r < s.
Equation (12.13), which is just (12.12) written differently, asserts that B
is a column operator which does precisely what we have just shown impossi-
ble: to combine the r columns of AIr to yield the s columns of Is , the latter
of which are just the elementary vectors e1 , e2 , e3 , e4 , e5 , . . . , es . Hence fi-
nally we conclude that no matrices A and B exist which satisfy (12.12) when
r < s. In other words, we conclude that although row and column operations
can demote identity matrices in rank, they can never promote them. The
promotion of identity matrices is impossible.
12.5.3 General matrix rank and its uniqueness

Step 8 of the Gauss-Jordan algorithm (§ 12.3.3) discovers a rank r for any
matrix A. One should like to think that this rank r were a definite property
of the matrix itself rather than some unreliable artifact of the algorithm,
but until now we have lacked the background theory to prove it. Now we
have the theory. Here is the proof.
The proof begins with a formal definition of the quantity whose unique-
ness we are trying to prove.
• The rank r of an identity matrix Ir is the number of ones along its

main diagonal. (This is from § 11.3.5.)
• The rank r of a general matrix A is the rank of an identity matrix Ir

to which A can be reduced by reversible row and column operations.
A matrix A has rank r if and only if matrices B> and B< exist such that
B> AB< = Ir ,
−1 −1
A = B> Ir B< ,
−1 −1
(12.15)
B> B> = I = B> B> ,
−1 −1
B< B< = I = B< B< .
The question is whether in (12.15) only a single rank r is possible.

To answer the question, we suppose that another rank were possible,
that A had not only rank r but also rank s. Then,
−1 −1
A = B> Ir B< ,
A = G−1 −1
> Is G< .
Combining these equations,

−1 −1
B> Ir B< = G−1 −1
> Is G< .
Solving first for Ir , then for Is ,
(B> G−1 −1
> )Is (G< B< ) = Ir ,
−1 −1
(G> B> )Ir (B< G< ) = Is .
Were it that r 6= s, then one of these two equations would constitute the
demotion of an identity matrix and the other, a promotion. But according
to § 12.5.2 and its (12.12), promotion is impossible. Therefore r 6= s is also
impossible, and
r=s
is guaranteed. No matrix has two different ranks. Matrix rank is unique.
This finding has two immediate implications:
• Reversible row and/or column operations exist to change any matrix

of rank r to any other matrix of the same rank. The reason is that, ac-
cording to (12.15), reversible operations exist to change both matrices
to Ir and back.
• No reversible operation can change a matrix’s rank.
The discovery that every matrix has a single, unambiguous rank and the
establishment of a failproof algorithm—the Gauss-Jordan—to ascertain that
rank have not been easy to achieve, but they are important achievements
12.5. RANK 317
nonetheless, worth the effort thereto. The reason these achievements matter
is that the mere dimensionality of a matrix is a chimerical measure of the
matrix’s true size—as for instance for the 3 × 3 example matrix at the head
of the section. Matrix rank by contrast is an entirely solid, dependable
measure. We shall rely on it often.
Section 12.5.8 comments further.
12.5.4 The full-rank matrix

According to (12.5), the rank r of a matrix can exceed the number neither
of the matrix’s rows nor of its columns. The greatest rank possible for an
m × n matrix is the lesser of m and n. A full-rank matrix, then, is defined
to be an m × n matrix with maximum rank r = m or r = n—or, if m = n,
both. A matrix of less than full rank is a degenerate matrix.
Consider a tall m × n matrix C, m ≥ n, one of whose n columns is a
linear combination (§ 12.1) of the others. One could by definition target the
dependent column with addition elementaries, using multiples of the other
columns to wipe the dependent column out. Having zeroed the dependent
column, one could then interchange it over to the matrix’s extreme right,
effectively throwing the column away, shrinking the matrix to m × (n − 1)
dimensionality. Shrinking the matrix necessarily also shrinks the bound on
the matrix’s rank to r ≤ n − 1—which is to say, to r < n. But the shrink,
done by reversible column operations, is itself reversible, by which § 12.5.3
binds the rank of the original, m × n matrix C likewise to r < n. The
matrix C, one of whose columns is a linear combination of the others, is
necessarily degenerate for this reason.
Now consider a tall matrix A with the same m × n dimensionality, but
with a full n independent columns. The transpose AT of such a matrix has
a full n independent rows. One of the conclusions of § 12.3.4 was that a
matrix of independent rows always has rank equal to the number of rows.
Since AT is such a matrix, its rank is a full r = n. But formally, what this
says is that there exist operators B< T and B T such that I = B T AT B T , the
> n < >
transpose of which equation is B> AB< = In —which in turn says that not
only AT , but also A itself, has full rank r = n.
Parallel reasoning rules the rows of broad matrices, m ≤ n, of course.
To square matrices, m = n, both lines of reasoning apply.
Gathering findings, we have that
• a tall m × n matrix, m ≥ n, has full rank if and only if its columns are
linearly independent;
• a broad m × n matrix, m ≤ n, has full rank if and only if its rows are
linearly independent;
• a square n × n matrix, m = n, has full rank if and only if its columns
and/or its rows are linearly independent; and
• a square matrix has both independent columns and independent rows,
or neither; never just one or the other.
To say that a matrix has full column rank is to say that it is tall or
square and has full rank r = n ≤ m. To say that a matrix has full row
rank is to say that it is broad or square and has full rank r = m ≤ n. Only
a square matrix can have full column rank and full row rank at the same
time, because a tall or broad matrix cannot but include, respectively, more
columns or more rows than Ir .
12.5.5 Underdetermined and overdetermined linear systems

(introduction)
The last paragraph of § 12.5.4 provokes yet further terminology. A lin-
ear system Ax = b is underdetermined if A lacks full column rank—that
is, if r < n—because inasmuch as some of A’s columns then depend lin-
early on the others such a system maps multiple n-element vectors x to the
same m-element vector b, meaning that knowledge of b does not suffice to
determine x uniquely. Complementarily, a linear system Ax = b is overde-
termined if A lacks full row rank—that is, if r < m. If A lacks both, then
the system is paradoxically both underdetermined and overdetermined and
is thereby degenerate. If A happily has both, then the system is exactly
determined.
Section 13.2 solves the exactly determined linear system. Section 13.4
solves the nonoverdetermined linear system. Section 13.6 analyzes the un-
solvable overdetermined linear system among others. Further generalities
await Ch. 13; but, regarding the overdetermined system specifically, the
present subsection would observe at least the few following facts.
An overdetermined linear system Ax = b cannot have a solution for
every possible m-element driving vector b. The truth of this claim can be
seen by decomposing the system’s matrix A by Gauss-Jordan and then left-
multiplying the decomposed system by G−1 > to reach the form
Ir G< x = G−1
> b.
If the m-element vector c ≡ G−1

> b, then Ir G< x = c, which is impossible
unless the last m − r elements of c happen to be zero. But since G> is
12.5. RANK 319
invertible, each b corresponds to a unique c and vice versa; so, if b is an

unrestricted m-element vector then so also is c, which verifies the claim.
Complementarily, a nonoverdetermined linear system Ax = b does have
a solution for every possible m-element driving vector b. This is so because
in this case the last m − r elements of c do happen to be zero; or, better
stated, because c in this case has no nonzeros among its last m − r elements,
because it has no last m − r elements, for the trivial reason that r = m.
It is an analytical error, and an easy one innocently to commit, to require
that
Ax = b
for unrestricted b when A lacks full row rank. The error is easy to commit
because the equation looks right, because such an equation is indeed valid
over a broad domain of b and might very well have been written correctly in
that context, only not in the context of unrestricted b. Analysis including
such an error can lead to subtly absurd conclusions. It is never such an
analytical error however to require that
Ax = 0
because, whatever other solutions such a system might have, it has at least
the solution x = 0.
12.5.6 The full-rank factorization

One sometimes finds dimension-limited matrices of less than full rank in-
convenient to handle. However, every dimension-limited, m × n matrix of
rank r can be expressed as the product of two full-rank matrices, one m × r
and the other r × n, both also of rank r:
A = BC. (12.16)
The truncated Gauss-Jordan (12.7) constitutes one such full-rank factoriza-

tion: B = Im G> Ir , C = Ir G< In , good for any matrix. Other full-rank
factorizations are possible, however, including among others the truncated
Gram-Schmidt (13.56). The full-rank factorization is not unique.16
Of course, if an m × n matrix already has full rank r = m or r = n, then
the full-rank factorization is trivial: A = Im A or A = AIn .
Section 13.6.4 uses the full-rank factorization.
16
[5, § 3.3][50, “Moore-Penrose generalized inverse”]
12.5.7 Full column rank and the Gauss-Jordan factors K

and S
The Gauss-Jordan decomposition (12.2),
A = P DLU Ir KS,
of a tall or square m × n matrix A of full column rank r = n ≤ m always

finds the factor K = I, regardless of the pivots one chooses during the
Gauss-Jordan algorithm’s step 3. If one happens always to choose q = i as
pivot column then not only K = I but S = I, too.
That K = I is seen by the algorithm’s step 12, which creates K. Step 12
nulls the spare columns q > r that dress I’s ˜ right, but in this case I˜ has
only r columns and therefore has no spare columns to null. Hence step 12
does nothing and K = I.
That S = I comes immediately of choosing q = i for pivot column dur-
ing each iterative instance of the algorithm’s step 3. But, one must ask, can
one choose so? What if column q = i were unusable? That is, what if the
only nonzero elements remaining in I’s ˜ ith column stood above the main
diagonal, unavailable for step 4 to bring to pivot? Well, were it so, then one
would indeed have to choose q 6= i to swap the unusable column away right-
ward, but see: nothing in the algorithm later fills such a column’s zeros with
anything else—they remain zeros—so swapping the column away rightward
could only delay the crisis. The column would remain unusable. Eventually
the column would reappear on pivot when no usable column rightward re-
mained available to swap it with, which contrary to our assumption would
mean precisely that r < n. Such contradiction can only imply that if r = n
then no unusable column can ever appear. One need not swap. We con-
clude that though one might voluntarily choose q 6= i during the algorithm’s
step 3, the algorithm cannot force one to do so if r = n. Yet if one always
does choose q = i, as the full-column-rank matrix A evidently leaves one
free to do, then indeed S = I.
Theoretically, the Gauss-Jordan decomposition (12.2) includes the fac-
tors K and S precisely to handle matrices with more columns than rank.
Matrices of full column rank r = n, common in applications, by definition
have no such problem. Therefore, the Gauss-Jordan decomposition theoret-
ically needs no K or S for such matrices, which fact lets us abbreviate the
decomposition for such matrices to read
A = P DLU In . (12.17)
12.5. RANK 321
Observe however that just because one theoretically can set S = I does
not mean that one actually should. The column permutor S exists to be
used, after all—especially numerically to avoid small pivots during early
invocations of the algorithm’s step 5. Equation (12.17) is not mandatory
but optional for a matrix A of full column rank (though still r = n and thus
K = I for such a matrix, even when the unabbreviated eqn. 12.2 is used).
There are however times when it is nice to know that one theoretically could,
if doing exact arithmetic, set S = I if one wanted to.
Since P DLU acts as a row operator, (12.17) implies that each row of
the full-rank matrix A lies in the space the rows of In address. This is
obvious and boring, but interesting is the converse implication of (12.17)’s
complementary form,
U −1 L−1 D −1 P −1 A = In ,
that each row of In lies in the space the rows of A address. The rows of In
and the rows of A evidently address the same space. One can moreover say
the same of A’s columns, since B = AT has full rank just as A does. In the
whole, if a matrix A is square and has full rank r = n, then A’s columns
together, A’s rows together, In ’s columns together and In ’s rows together
each address the same, complete n-dimensional space.
12.5.8 The significance of rank uniqueness

The result of § 12.5.3, that matrix rank is unique, is an extremely important
matrix theorem. It constitutes the chapter’s chief result, which we have
spent so many pages to attain. Without this theorem, the very concept
of matrix rank must remain in doubt, along with all that attends to the
concept. The theorem is the rock upon which the general theory of the
matrix is built.
The concept underlying the theorem promotes the useful sensibility that
a matrix’s rank, much more than its mere dimensionality or the extent of its
active region, represents the matrix’s true size. Dimensionality can deceive,
after all. For example, the honest 2 × 2 matrix
» –
5 1
3 6
has two independent rows or, alternately, two independent columns, and,
hence, rank r = 2. One can easily construct a phony 3 × 3 matrix from
the honest 2 × 2, however, simply by applying some 3 × 3 row and column
elementaries:
2 3
» – 5 1 6
5 1
T(2/3)[32] T1[13] T1[23] = 4 3 6 9 5.
3 6
2 4 6
The 3 × 3 matrix on the equation’s right is the one we met at the head
of the section. It looks like a rank-three matrix, but really has only two
independent columns and two independent rows. Its true rank is r = 2. We
have here caught a matrix impostor pretending to be bigger than it really
is.17
Now, admittedly, adjectives like “honest” and “phony,” terms like “im-
poster,” are a bit hyperbolic. The last paragraph has used them to convey
the subjective sense of the matter, but of course there is nothing mathemat-
ically improper or illegal about a matrix of less than full rank, so long as the
true rank is correctly recognized. When one models a physical phenomenon
by a set of equations, one sometimes is dismayed to discover that one of the
equations, thought to be independent, is really just a useless combination
of the others. This can happen in matrix work, too. The rank of a matrix
helps one to recognize how many truly independent vectors, dimensions or
equations one actually has available to work with, rather than how many
seem available at first glance. That is the sense of matrix rank.
17
An applied mathematician with some matrix experience actually probably recognizes
this particular 3 × 3 matrix as a fraud on sight, but it is a very simple example. No one
can just look at some arbitrary matrix and instantly perceive its true rank. Consider for
instance the 5 × 5 matrix (in hexadecimal notation)
2 3
12 9 3 1 0
6 3 F 15
2 12 7
6 2 2 2
E
7
7.
6 D 9 −19 − −6 7
6 3
4 −2 0 6 1 5 5
1 −4 4 1 −8
As the reader can verify by the Gauss-Jordan algorithm, the matrix’s rank is not r = 5
but r = 4.
Chapter 13
Inversion and
orthonormalization
The undeniably tedious Chs. 11 and 12 have piled the matrix theory deep
while affording scant practical reward. Building upon the two tedious chap-
ters, this chapter brings the first rewarding matrix work.
One might be forgiven for forgetting after so many pages of abstract the-
ory that the matrix afforded any reward or had any use at all. Uses however
it has. Sections 11.1.1 and 12.5.5 have already broached1 the matrix’s most
basic use, the primary subject of this chapter, to represent a system of m
linear scalar equations in n unknowns neatly as
Ax = b
and to solve the whole system at once by inverting the matrix A that char-
acterizes it.
Now, before we go on, we want to confess that such a use alone, on
the surface of it—though interesting—might not have justified the whole
uncomfortable bulk of Chs. 11 and 12. We already knew how to solve a
simultaneous system of linear scalar equations in principle without recourse
to the formality of a matrix, after all, as in the last step to derive (3.9) as
far back as Ch. 3. Why should we have suffered two bulky chapters, if only
to prepare to do here something we already knew how to do?
The question is a fair one, but admits at least four answers. First,
the matrix neatly solves a linear system not only for a particular driving
vector b but for all possible driving vectors b at one stroke, as this chapter
1
The reader who has skipped Ch. 12 might at least review § 12.5.5.
323
324 CHAPTER 13. INVERSION AND ORTHONORMALIZATION
explains. Second and yet more impressively, the matrix allows § 13.6 to
introduce the pseudoinverse to approximate the solution to an unsolvable
linear system and, moreover, to do so both optimally and efficiently, whereas
such overdetermined systems arise commonly in applications. Third, to solve
the linear system neatly is only the primary and most straightforward use
of the matrix, not its only use: the even more interesting eigenvalue and
its incidents await Ch. 14. Fourth, specific applications aside, one should
never underestimate the blunt practical benefit of reducing an arbitrarily
large grid of scalars to a single symbol A, which one can then manipulate
by known algebraic rules. Most students first learning the matrix have
wondered at this stage whether it were worth all the tedium; so, if the
reader now wonders, then he stands in good company. The matrix finally
begins to show its worth here.
The chapter opens in § 13.1 by inverting the square matrix to solve the
exactly determined, n × n linear system in § 13.2. It continues in § 13.3 by
computing the rectangular matrix’s kernel to solve the nonoverdetermined,
m × n linear system in § 13.4. In § 13.6, it brings forth the aforementioned
pseudoinverse, which rightly approximates the solution to the unsolvable
overdetermined linear system. After briefly revisiting the Newton-Raphson
iteration in § 13.7, it concludes by introducing the concept and practice of
vector orthonormalization in §§ 13.8 through 13.12.
13.1 Inverting the square matrix
Consider an n×n square matrix A of full rank r = n. Suppose that extended

operators G> , G< , G−1 −1
> and G< can be found, each with an n × n active
13.1. INVERTING THE SQUARE MATRIX 325
region (§ 11.3.2), such that2
G−1 −1
> G> = I = G> G> ,
G−1 −1
< G< = I = G< G< , (13.1)
A = G> In G< .
Observing from (11.31) that
In A = A = AIn ,
−1 −1
In G< G> = G−1 −1
< In G> = G−1 −1
< G> In ,
we find by successive steps that
A = G> In G< ,
In A = G> G< In ,
−1 −1
G< G> In A = In ,
(G< In G−1
−1
> )(A) = In ;
2
The symbology and associated terminology might disorient a reader who had skipped
Chs. 11 and 12. In this book, the symbol I theoretically represents an ∞ × ∞ identity
matrix. Outside the m × m or n × n square, the operators G> and G< each resemble
the ∞ × ∞ identity matrix I, which means that the operators affect respectively only the
first m rows or n columns of the thing they operate on. (In the present section it happens
that m = n because the matrix A of interest is square, but this footnote uses both symbols
because generally m 6= n.)
The symbol Ir contrarily represents an identity matrix of only r ones, though it too
can be viewed as an ∞ × ∞ matrix with zeros in the unused regions. If interpreted as
an ∞ × ∞ matrix, the matrix A of the m × n system Ax = b has nonzero content only
within the m × n rectangle.
None of this is complicated, really. Its purpose is merely to separate the essential features
of a reversible operation like G> or G< from the dimensionality of the vector or matrix on
which the operation happens to operate. The definitions do however necessarily, slightly
diverge from definitions the reader may have been used to seeing in other books. In this
book, one can legally multiply any two matrices, because all matrices are theoretically
∞ × ∞, anyway (though whether it makes any sense in a given circumstance to multiply
mismatched matrices is another question; sometimes it does make sense, as in eqns. 13.24
and 14.49, but more often it does not—which naturally is why the other books tend to
forbid such multiplication).
To the extent that the definitions confuse, the reader might briefly review the earlier
chapters, especially § 11.3.
or alternately that
A = G> In G< ,
AIn = In G> G< ,
−1 −1
AIn G< G> = In ,
(A)(G−1 −1
< In G> ) = In .
Either way, we have that

A−1 A = In = AA−1 ,
(13.2)
A−1 ≡ G−1 −1
< In G> .
Of course, for this to work, G> , G< , G−1 −1

> and G< must exist, be known
and honor n × n active regions, which might seem a practical hurdle. How-
ever, (12.2), (12.3) and the body of § 12.3 have shown exactly how to find
just such a G> , G< , G−1 −1
> and G< for any square matrix A of full rank,
without exception; so, there is no trouble here. The factors do exist, and
indeed we know how to find them.
Equation (13.2) features the important matrix A−1 , the rank-n inverse
of A. We have not yet much studied the rank-n inverse, but we have defined
it in (11.49), where we gave it the fuller, nonstandard notation A−1(n) . When
naming the rank-n inverse in words one usually says simply, “the inverse,”
because the rank is implied by the size of the square active region of the
matrix inverted; but the rank-n inverse from (11.49) is not quite the infinite-
dimensional inverse from (11.45), which is what G−1 −1
> and G< are. According
to (13.2), the product of A−1 and A—or, written more fully, the product of
A−1(n) and A—is, not I, but In .
Properties that emerge from (13.2) include the following.
• Like A, the rank-n inverse A−1 (more fully written A−1(n) ) too is an
n × n square matrix of full rank r = n.
• Since A is square and has full rank (§ 12.5.4), its rows and, separately,
its columns are linearly independent, so it has only the one, unique
inverse A−1 . No other rank-n inverse of A exists.
• On the other hand, inasmuch as A is square and has full rank, it does
per (13.2) indeed have an inverse A−1 . The rank-n inverse exists.
• If B = A−1 then B −1 = A. That is, A is itself the rank-n inverse

of A−1 . The matrices A and A−1 thus form an exclusive, reciprocal
pair.
13.1. INVERTING THE SQUARE MATRIX 327
• If B is an n × n square matrix and either BA = In or AB = In , then

both equalities in fact hold; thus, B = A−1 . One can have neither
equality without the other.
• Only a square, n × n matrix of full rank r = n has a rank-n inverse.

A matrix A′ which is not square, or whose rank falls short of a full
r = n, is not invertible in the rank-n sense of (13.2).
That A−1 is an n × n square matrix of full rank and that A is itself the
inverse of A−1 proceed from the definition (13.2) of A−1 plus § 12.5.3’s find-
ing that reversible operations like G−1 −1
> and G< cannot change In ’s rank.
That the inverse exists is plain, inasmuch as the Gauss-Jordan decompo-
sition plus (13.2) reliably calculate it. That the inverse is unique begins
from § 12.5.4’s observation that the columns (like the rows) of A are lin-
early independent because A is square and has full rank. From this begin-
ning and the fact that In = AA−1 , it follows that [A−1 ]∗1 represents3 the
one and only possible combination of A’s columns which achieves e1 , that
[A−1 ]∗2 represents the one and only possible combination of A’s columns
which achieves e2 , and so on through en . One could observe likewise re-
specting the independent rows of A. Either way, A−1 is unique. Moreover,
no other n × n matrix B 6= A−1 satisfies either requirement of (13.2)—that
BA = In or that AB = In —much less both.
It is not claimed that the matrix factors G> and G< themselves are
unique, incidentally. On the contrary, many different pairs of matrix fac-
tors G> and G< can yield A = G> In G< , no less than that many different
pairs of scalar factors γ> and γ< can yield α = γ> 1γ< . Though the Gauss-
Jordan decomposition is a convenient means to G> and G< , it is hardly the
only means, and any proper G> and G< found by any means will serve so
long as they satisfy (13.1). What are unique are not the factors but the A
and A−1 they produce.
What of the degenerate n × n square matrix A′ , of rank r < n? Rank
promotion is impossible as §§ 12.5.2 and 12.5.3 have shown, so in the sense
of (13.2) such a matrix has no inverse; for, if it had, then A′−1 would by defi-
nition represent a row or column operation which impossibly promoted A′ to
the full rank r = n of In . Indeed, in that it has no inverse such a degenerate
matrix closely resembles the scalar 0, which has no reciprocal. Mathemati-
cal convention owns a special name for a square matrix which is degenerate
and thus has no inverse; it calls it a singular matrix.
3
The notation [A−1 ]∗j means “the jth column of A−1 .” Refer to § 11.1.3.
And what of a rectangular matrix? Is it degenerate? Well, no, not

exactly, not necessarily. The definitions of the present particular section
are meant for square matrices; they do not neatly apply to nonsquare ones.
Refer to §§ 12.5.3 and 12.5.4. However, appending the right number of
null rows or columns to a nonsquare matrix does turn it into a degenerate
square, in which case the preceding argument applies. See also §§ 12.5.5,
13.4 and 13.6.
13.2 The exactly determined linear system

Section 11.1.1 has shown how the single matrix equation
Ax = b (13.3)
concisely represents an entire simultaneous system of linear scalar equa-

tions. If the system has n scalar equations and n scalar unknowns, then
the matrix A has square, n × n dimensionality. Furthermore, if the n scalar
equations are independent of one another, then the rows of A are similarly
independent, which gives A full rank and makes it invertible. Under these
conditions, one can solve (13.3) and the corresponding system of linear scalar
equations by left-multiplying (13.3) by the A−1 of (13.2) and (13.1) to reach
the famous formula
x = A−1 b. (13.4)
Inverting the square matrix A of scalar coefficients, (13.4) concisely solves
a simultaneous system of n linear scalar equations in n scalar unknowns. It
is the classic motivational result of matrix theory.
It has taken the book two long chapters to reach (13.4). If one omits
first to prepare the theoretical ground sufficiently to support more advanced
matrix work, then one can indeed reach (13.4) with rather less effort than
the book has done.4 As the chapter’s introduction has observed, however, we
4
For motivational reasons, introductory, tutorial linear algebra textbooks like [30]
and [42] rightly yet invariably invert the general square matrix of full rank much ear-
lier, reaching (13.4) with less effort. The deferred price the student pays for the simpler-
seeming approach of the tutorials is twofold. First, the student fails to develop the Gauss-
Jordan decomposition properly, instead learning the less elegant but easier to grasp “row
echelon form” of “Gaussian elimination” [30, Ch. 1][42, § 1.2]—which makes good matrix-
arithmetic drill but leaves the student imperfectly prepared when the time comes to study
kernels and eigensolutions or to read and write matrix-handling computer code. Second,
in the long run the tutorials save no effort, because the student still must at some point
develop the theory underlying matrix rank and supporting each of the several coincident
properties of § 14.2. What the tutorials do is pedagogically necessary—it is how the
13.3. THE KERNEL 329
shall soon meet additional interesting applications of the matrix which in any
case require the theoretical ground to have been prepared. Equation (13.4)
is only the first fruit of the effort.
Where the inverse does not exist, where the square matrix A is singu-
lar, the rows of the matrix are linearly dependent, meaning that the cor-
responding system actually contains fewer than n useful scalar equations.
Depending on the value of the driving vector b, the superfluous equations
either merely reproduce or flatly contradict information the other equations
already supply. Either way, no unique solution to a linear system described
by a singular square matrix is possible—though a good approximate solu-
tion is given by the pseudoinverse of § 13.6. In the language of § 12.5.5, the
singular square matrix characterizes a system that is both underdetermined
and overdetermined, thus degenerate.
13.3 The kernel

If a matrix A has full column rank (§ 12.5.4), then the columns of A are
linearly independent and
Ax = 0 (13.5)
is impossible if In x 6= 0. If the matrix however lacks full column rank
then (13.5) is possible even if In x 6= 0. In either case, any n-element x
(including x = 0) that satisfies (13.5) belongs to the kernel of A.
Let A be an m × n matrix of rank r. A second matrix,5 AK , minimally
represents the kernel of A if and only if
• AK has n × (n − r) dimensionality (which gives AK tall rectangular
form unless r = 0),
writer first learned the matrix and probably how the reader first learned it, too—but it is
appropriate to a tutorial, not to a study reference like this book.
In this book, where derivations prevail, the proper place to invert the general square
matrix of full rank is here. Indeed, the inversion here goes smoothly, because Chs. 11
and 12 have laid under it a firm foundation upon which—and supplied it the right tools
with which—to work.
5
The conventional mathematical notation for the kernel of A is ker{A}, null{A} or
something nearly resembling one of the two—the notation seems to vary from editor
to editor—which technically represent the kernel space itself, as opposed to the nota-
tion AK which represents a matrix whose columns address the kernel space. This book
deëmphasizes the distinction and prefers the kernel matrix notation AK .
If we were really precise, we might write not AK but AK(n) to match the A−1(r)
of (11.49). The abbreviated notation AK is probably clear enough for most practical
purposes, though, and surely more comprehensible to those who do not happen to have
read this particular book.
• AK has full rank n − r (that is, the columns of AK are linearly inde-
pendent, which gives AK full column rank), and
• AK satisfies the equation
AAK = 0. (13.6)
The n−r independent columns of the kernel matrix AK address the complete
space x = AK a of vectors in the kernel, where the (n − r)-element vector a
can have any value. In symbols,
Ax = A(AK a) = (AAK )a = 0.
The definition does not pretend that the kernel matrix AK is unique.
Except when A has full column rank the kernel matrix is not unique; there
are infinitely many kernel matrices AK to choose from for a given matrix A.
What is unique is not the kernel matrix but rather the space its columns
address, and it is the latter space rather than AK as such that is technically
the kernel (if you forget and call AK “a kernel,” though, you’ll be all right).
The Gauss-Jordan kernel formula 6
AK = S −1 K −1 Hr In−r = G−1
< Hr In−r (13.7)
gives a complete kernel AK of A, where S −1 , K −1 and G−1 < are the factors
their respective symbols indicate of the Gauss-Jordan decomposition’s com-
plementary form (12.3) and Hr is the shift operator of § 11.9. Section 13.3.1
derives the formula, next.
13.3.1 The Gauss-Jordan kernel formula

To derive (13.7) is not easy. It begins from the statement of the linear
system
Ax = b, where b = 0 or r = m, or both; (13.8)
and where b and x are respectively m- and n-element vectors and A is an
m × n matrix of rank r. This statement is broader than (13.5) requires but
it serves § 13.4, too; so, for the moment, for generality’s sake, we leave b un-
specified but by the given proviso. Gauss-Jordan factoring A, by successive
steps,
G> Ir KSx = b,
Ir KSx = G−1
> b,
Ir (K − I)Sx + Ir Sx = G−1
> b.
6
The name Gauss-Jordan kernel formula is not standard as far as the writer is aware,
but we would like a name for (13.7). This name seems as fitting as any.
Applying an identity from Table 12.2 on page 305,

Ir K(In − Ir )Sx + Ir Sx = G−1
> b.
Rearranging terms,
Ir Sx = G−1
> b − Ir K(In − Ir )Sx. (13.9)
Equation (13.9) is interesting. It has Sx on both sides, where Sx is
the vector x with elements reordered in some particular way. The equation
has however on the left only Ir Sx, which is the first r elements of Sx; and
on the right only (In − Ir )Sx, which is the remaining n − r elements.7 No
element of Sx appears on both sides. Naturally this is no accident; we have
(probably after some trial and error not recorded here) planned the steps
leading to (13.9) to achieve precisely this effect. Equation (13.9) implies that
one can choose the last n − r elements of Sx freely, but that the choice then
determines the first r elements.
The implication is significant. To express the implication more clearly
we can rewrite (13.9) in the improved form
f = G−1
> b − Ir KHr a,

f
Sx = = f + Hr a,
a (13.10)
f ≡ Ir Sx,
a ≡ H−r (In − Ir )Sx,
where a represents the n − r free elements of Sx and f represents the r
dependent elements. This makes f and thereby also x functions of the free
parameter a and the driving vector b:
f (a, b) = G−1
> b − Ir KHr a,
(13.11)

f (a, b)
Sx(a, b) = = f (a, b) + Hr a.
a
If b = 0 as (13.5) requires, then
f (a, 0) = −Ir KHr a,

f (a, 0)
Sx(a, 0) = = f (a, 0) + Hr a.
a
7
Notice how we now associate the factor (In − Ir ) rightward as a row truncator, though
it had first entered acting leftward as a column truncator. The flexibility to reassociate
operators in such a way is one of many good reasons Chs. 11 and 12 have gone to such
considerable trouble to develop the basic theory of the matrix.
Substituting the first line into the second,
Sx(a, 0) = (I − Ir K)Hr a. (13.12)
In the event that a = ej , where 1 ≤ j ≤ n − r,
Sx(ej , 0) = (I − Ir K)Hr ej .
For all the ej at once,
Sx(In−r , 0) = (I − Ir K)Hr In−r .
But if all the ej at once, the columns of In−r , exactly address the domain
of a, then the columns of x(In−r , 0) likewise exactly address the range of
x(a, 0). Equation (13.6) has already named this range AK , by which8
SAK = (I − Ir K)Hr In−r . (13.13)

Left-multiplying by S −1 = S ∗ = S T produces the alternate kernel formula
AK = S −1 (I − Ir K)Hr In−r . (13.14)
The alternate kernel formula (13.14) is correct but not as simple as it

could be. By the identity (11.76), eqn. (13.13) is
SAK = (I − Ir K)(In − Ir )Hr

= [(In − Ir ) − Ir K(In − Ir )]Hr
= [(In − Ir ) − (K − I)]Hr , (13.15)
8
These are difficult steps. How does one justify replacing a by ej , then ej by In−r ,
then x by AK ? One justifies them in that the columns of In−r are the several ej , of which
any (n − r)-element vector a can be constructed as the linear combination
n−r
X
a = In−r a = [ e1 e2 e3 ··· en−r ]a = aj ej
j=1
weighted by the elements of a. Seen from one perspective, this seems trivial; from another
perspective, baffling; until one grasps what is really going on here.
The idea is that if we can solve the problem for each elementary vector ej —that is,
in aggregate, if we can solve the problem for the identity matrix In−r —then we shall
implicitly have solved it for every a because a is a weighted combination of the ej and the
whole problem is linear. The solution
x = AK a
for a given choice of a becomes a weighted combination of the solutions for each ej , with
the elements of a again as the weights. And what are the solutions for each ej ? Answer:
the corresponding columns of AK , which by definition are the independent values of x
that cause b = 0.
where we have used Table 12.2 again in the last step. How to proceed
symbolically from (13.15) is not obvious, but if one sketches the matrices
of (13.15) schematically with a pencil, and if one remembers that K −1 is
just K with elements off the main diagonal negated, then it appears that
SAK = K −1 Hr In−r . (13.16)
The appearance is not entirely convincing,9 but (13.16) though unproven still
helps because it posits a hypothesis toward which to target the analysis.
Two variations on the identities of Table 12.2 also help. First, from the
identity that
K + K −1
= I,
2
we have that
K − I = I − K −1 . (13.17)
Second, right-multiplying by Ir the identity that
Ir K −1 (In − Ir ) = K −1 − I
and canceling terms, we have that
K −1 Ir = Ir (13.18)
(which actually is pretty obvious if you think about it, since all of K’s
interesting content lies by construction right of its rth column). Now we
have enough to go on with. Substituting (13.17) and (13.18) into (13.15)
yields
SAK = [(In − K −1 Ir ) − (I − K −1 )]Hr .
Adding 0 = K −1 In Hr − K −1 In Hr and rearranging terms,
SAK = K −1 (In − Ir )Hr + [K −1 − K −1 In − I + In ]Hr .
Factoring,
SAK = K −1 (In − Ir )Hr + [(K −1 − I)(I − In )]Hr .
According to Table 12.2, the quantity in square brackets is zero, so
SAK = K −1 (In − Ir )Hr ,

9
Well, no, actually, the appearance pretty much is entirely convincing, but let us finish
the proof symbolically nonetheless.
which, considering that the identity (11.76) has that (In − Ir )Hr = Hr In−r ,
proves (13.16). The final step is to left-multiply (13.16) by S −1 = S ∗ = S T ,
reaching (13.7) that was to be derived.
One would like to feel sure that the columns of (13.7)’s AK actually
addressed the whole kernel space of A rather than only part. One would
further like to feel sure that AK had no redundant columns; that is, that it
had full rank. Moreover, the definition of AK in the section’s introduction
demands both of these features. In general such features would be hard
to establish, but here the factors conveniently are Gauss-Jordan factors.
Regarding the whole kernel space, AK addresses it because AK comes from
all a. Regarding redundancy, AK lacks it because SAK lacks it, and SAK
lacks it because according to (13.13) the last rows of SAK are Hr In−r . So,
in fact, (13.7) has both features and does fit the definition.
13.3.2 Converting between kernel matrices

If C is a reversible (n−r)×(n−r) operator by which we right-multiply (13.6),
then the matrix
A′K = AK C (13.19)
like AK evidently represents the kernel of A:
AA′K = A(AK C) = (AAK )C = 0.
Indeed this makes sense: because the columns of AK C address the same
space the columns of AK address, the two matrices necessarily represent the
same underlying kernel. Moreover, some C exists to convert AK into every
alternate kernel matrix A′K of A. We know this because § 12.4 lets one
replace the columns of AK with those of A′K , reversibly, one column at a
time, without altering the space addressed. (It might not let one replace the
columns in sequence, but if out of sequence then a reversible permutation
at the end corrects the order. Refer to §§ 12.5.1 and 12.5.2 for the pattern
by which this is done.)
The orthonormalizing column operator R−1 of (13.54) below incidentally
tends to make a good choice for C.
13.3.3 The degree of freedom

A slightly vague but extraordinarily useful concept has emerged in this sec-
tion, worth pausing briefly to appreciate. The concept is the concept of the
degree of freedom.
A degree of freedom is a parameter one remains free to determine within

some continuous domain. For example, Napoleon’s artillerist10 might have
enjoyed as many as six degrees of freedom in firing a cannonball: two in
where he chose to set up his cannon (one degree in north-south position, one
in east-west); two in aim (azimuth and elevation); one in muzzle velocity (as
governed by the quantity of gunpowder used to propel the ball); and one
in time. A seventh potential degree of freedom, the height from which the
artillerist fires, is of course restricted by the lay of the land: the artillerist
can fire from a high place only if the place he has chosen to fire from happens
to be up on a hill, for Napoleon had no flying cannon. Yet even among the
six remaining degrees of freedom, the artillerist might find some impractical
to exercise. The artillerist probably preloads the cannon always with a
standard charge of gunpowder because, when he finds his target in the field,
he cannot spare the time to unload the cannon and alter the charge: this
costs one degree of freedom. Likewise, the artillerist must limber up the
cannon and hitch it to a horse to shift it to better ground; for this too
he cannot spare time in the heat of battle: this costs two degrees. And
Napoleon might yell, “Fire!” canceling the time degree as well. Two degrees
of freedom remain to the artillerist; but, since exactly two degrees are needed
to hit some particular target on the battlefield, the two are enough.
Now consider what happens if the artillerist loses one of his last two
remaining degrees of freedom. Maybe the cannon’s carriage wheel is broken
and the artillerist can no longer turn the cannon; that is, he can still choose
firing elevation but no longer azimuth. In such a strait to hit some particular
target on the battlefield, the artillerist needs somehow to recover another
degree of freedom, for he needs two but has only one. If he disregards
Napoleon’s order, “Fire!” (maybe not a wise thing to do, but, anyway, . . . )
and waits for the target to traverse the cannon’s fixed line of fire, then he
can still hope to hit even with the broken carriage wheel; for could he choose
neither azimuth nor the moment to fire, then he would almost surely miss.
Some apparent degrees of freedom are not real. For example, muzzle
velocity gives the artillerist little control firing elevation does not also give.
Other degrees of freedom are nonlinear in effect: a certain firing elevation
gives maximum range; nearer targets can be hit by firing either higher or
lower at the artillerist’s discretion. On the other hand, too much gunpowder
might break the cannon.
10
The author, who has never fired an artillery piece (unless an arrow from a Boy Scout
bow counts), invites any real artillerist among the readership to write in to improve the
example.
All of this is hard to generalize in unambiguous mathematical terms,

but the count of the degrees of freedom in a system is of high conceptual
importance to the engineer nonetheless. Basically, the count captures the
idea that to control n output variables of some system takes at least n in-
dependent input variables. The n may possibly for various reasons still not
suffice—it might be wise in some cases to allow n + 1 or n + 2—but in no
event will fewer than n do. Engineers of all kinds think in this way: an
aeronautical engineer knows in advance that an airplane needs at least n
ailerons, rudders and other control surfaces for the pilot adequately to con-
trol the airplane; an electrical engineer knows in advance that a circuit needs
at least n potentiometers for the technician adequately to tune the circuit;
and so on.
In geometry, a line brings a single degree of freedom. A plane brings
two. A point brings none. If the line bends and turns like a mountain
road, it still brings a single degree of freedom. And if the road reaches
an intersection? Answer: still one degree. A degree of freedom has some
continuous nature, not merely a discrete choice to turn left or right. On
the other hand, a swimmer in a swimming pool enjoys three degrees of
freedom (up-down, north-south, east-west) even though his domain in any
of the three is limited to the small volume of the pool. The driver on the
mountain road cannot claim a second degree of freedom at the mountain
intersection (he can indeed claim a choice, but the choice being discrete
lacks the proper character of a degree of freedom), but he might plausibly
claim a second degree of freedom upon reaching the city, where the web or
grid of streets is dense enough to approximate access to any point on the
city’s surface. Just how many streets it takes to turn the driver’s “line”
experience into a “plane” experience is a matter for the mathematician’s
discretion.
Reviewing (13.11), we find n − r degrees of freedom in the general under-
determined linear system, represented by the n − r free elements of a. If the
underdetermined system is not also overdetermined, if it is nondegenerate
such that r = m, then it is guaranteed to have a family of solutions x. This
family is the topic of the next section.
13.4 The nonoverdetermined linear system

The exactly determined linear system of § 13.2 is common, but also common
is the more general, nonoverdetermined linear system
Ax = b, (13.20)
13.4. THE NONOVERDETERMINED LINEAR SYSTEM 337
in which b is a known, m-element vector; x is an unknown, n-element vector;

and A is a square or broad, m × n matrix of full row rank (§ 12.5.4)
r = m ≤ n. (13.21)
Except in the exactly determined edge case r = m = n of § 13.2, the
nonoverdetermined linear system has no unique solution but rather a family
of solutions. This section delineates the family.
13.4.1 Particular and homogeneous solutions

The nonoverdetermined linear system (13.20) by definition admits more than
one solution x for a given driving vector b. Such a system is hard to solve
all at once, though, so we prefer to split the system as
Ax1 = b,
A(AK a) = 0, (13.22)
K
x = x1 + A a,
which, when the second line is added to the first and the third is substi-
tuted, makes the whole form (13.20). Splitting the system does not change
it, but it does let us treat the system’s first and second lines in (13.22) sepa-
rately. In the split form, the symbol x1 represents any one n-element vector
that happens to satisfy the form’s first line—many are possible; the mathe-
matician just picks one—and is called a particular solution of (13.20). The
(n − r)-element vector a remains unspecified, whereupon AK a represents
the complete family of n-element vectors that satisfy the form’s second line.
The family of vectors expressible as AK a is called the homogeneous solution
of (13.20).
Notice the italicized articles a and the.
The Gauss-Jordan kernel formula (13.7) has given us AK and thereby
the homogeneous solution, which renders the analysis of (13.20) already half
done. To complete the analysis, it remains in § 13.4.2 to find a particular
solution.
13.4.2 A particular solution

Any particular solution will do. Equation (13.11) has that
f (a, b) = G−1
> b − Ir KHr a,

K
f (a, b)
(S) x1 (a, b) + A a = = f (a, b) + Hr a,
a
where we have substituted the last line of (13.22) for x. This holds for any a
and b. We are not free to choose the driving vector b, but since we need
only one particular solution, a can be anything we want. Why not
a = 0?
Then
f (0, b) = G−1
> b,

f (0, b)
Sx1 (0, b) = = f (0, b).
0
That is,
x1 = S −1 G−1
> b. (13.23)
13.4.3 The general solution

Assembling (13.7), (13.22) and (13.23) yields the general solution
x = S −1 (G−1
> b+K
−1
Hr In−r a) (13.24)
to the nonoverdetermined linear system (13.20).

In exact arithmetic (13.24) solves the nonoverdetermined linear system
in theory exactly. Of course, practical calculations are usually done in lim-
ited precision, in which compounded rounding error in the last bit eventually
disrupts (13.24) for matrices larger than some moderately large size. Avoid-
ing unduly small pivots early in the Gauss-Jordan extends (13.24)’s reach
to larger matrices, and for yet larger matrices a bewildering variety of more
sophisticated techniques exists to mitigate the problem, which can be vex-
ing because the problem arises even when the matrix A is exactly known.
Equation (13.24) is useful and correct, but one should at least be aware
that it can in practice lose floating-point accuracy when the matrix it at-
tacks grows too large. (It can also lose accuracy when the matrix’s rows
are almost dependent, but that is more the fault of the matrix than of the
formula. See § 14.8, which addresses a related problem.)
13.5 The residual

Equations (13.2) and (13.4) solve the exactly determined linear system
Ax = b. Equation (13.24) broadens the solution to include the nonoverde-
termined linear system. None of those equations however can handle the
13.6. THE PSEUDOINVERSE AND LEAST SQUARES 339
overdetermined linear system, because for general b the overdetermined lin-

ear system
Ax ≈ b (13.25)
has no exact solution. (See § 12.5.5 for the definitions of underdetermined,
overdetermined, etc.)
One is tempted to declare the overdetermined system uninteresting be-
cause it has no solution and to leave the matter there, but this would be a
serious mistake. In fact the overdetermined system is especially interesting,
and the more so because it arises so frequently in applications. One seldom
trusts a minimal set of data for important measurements, yet extra data
imply an overdetermined system. We need to develop the mathematics to
handle the overdetermined system properly.
The quantity11,12
r(x) ≡ b − Ax (13.26)
measures how nearly some candidate solution x solves the system (13.25).
We call this quantity the residual, and the smaller, the better. More pre-
cisely, the smaller the nonnegative real scalar
|ri (x)|2
X
[r(x)]∗ [r(x)] = (13.27)
i
is, called the squared residual norm, the more favorably we regard the can-
didate solution x.
13.6 The Moore-Penrose pseudoinverse and the

least-squares problem
A typical problem is to fit a straight line to some data. For example, suppose
that we are building-construction contractors with a unionized work force,
whose labor union can supply additional, fully trained labor on demand.
Suppose further that we are contracted to build a long freeway and have
been adding workers to the job in recent weeks to speed construction. On
Saturday morning at the end of the second week, we gather and plot the
production data on the left of Fig. 13.1. If ui and bi respectively represent
the number of workers and the length of freeway completed during week i,
11
Alas, the alphabet has only so many letters (see Appendix B). The r here is unrelated
to matrix rank r.
12
This is as [63] defines it. Some authors [47] however prefer to define r(x) ≡ Ax − b,
instead.
Figure 13.1: Fitting a line to measured data.
during the week

during the week b
bb bbb
Length newly
Length newly
completed
completed
Number of workers Number of workers
then we can fit a straight line b = σu + γ to the measured production data

such that » –» – » –
u1 1 σ b1
= ,
u2 1 γ b2
inverting the matrix per §§ 13.1 and 13.2 to solve for x ≡ [σ γ]T , in the hope
that the resulting line will predict future production accurately.
That is all mathematically irreproachable. By the fifth Saturday however
we shall have gathered more production data, plotted on the figure’s right,
to which we should like to fit a better line to predict production more accu-
rately. The added data present a problem. Statistically, the added data are
welcome, but geometrically we need only two points to specify a line; what
are we to do with the other three? The five points together overdetermine
the linear system 2 3 2 3
u1 1 b1
6 u2 1 7» – 6 b2 7
7 σ
= 7.
6 6 7
6 u3 1 7 6 b3
6 7 γ 6 7
4 u4 1 5 4 b4 5
u5 1 b5
There is no way to draw a single straight line b = σu + γ exactly through
all five, for in placing the line we enjoy only two degrees of freedom.13
The proper approach is to draw among the data points a single straight
line that misses the points as narrowly as possible. More precisely, the proper
13
Section 13.3.3 characterized a line as enjoying only one degree of freedom. Why now
two? The answer is that § 13.3.3 discussed travel along a line rather than placement of a
line as here. Though both involve lines, they differ as driving an automobile differs from
washing one. Do not let this confuse you.
approach chooses parameters σ and γ to minimize the squared residual norm

[r(x)]∗ [r(x)] of § 13.5, given that
u1 1 b1
2 3 2 3
6
6 u2 1 7
7
6
6 b2 7
7
6 u3 1 7 »
σ
– 6 b3 7
A= 6
6 u4 1 7,
7
x= γ
, b= 6
6 b4 7.
7
6 7 6 7
6 u5 1 7 6 b5 7
.. ..
4 5 4 5
. .
Such parameters constitute a least-squares solution.

The matrix A in the example has two columns, data marching on the
left, all ones on the right. This is a typical structure for A, but in general any
matrix A with any number of columns of any content might arise (because
there were more than two relevant variables or because some data merited
heavier weight than others, among many further reasons). Whatever ma-
trix A might arise from whatever source, this section attacks the difficult
but important problem of approximating optimally a solution to the general,
possibly unsolvable linear system (13.25), Ax ≈ b.
13.6.1 Least squares in the real domain

The least-squares problem is simplest when the matrix A enjoys full column
rank and no complex numbers are involved. In this case, we seek to minimize
the squared residual norm
[r(x)]T [r(x)] = (b − Ax)T (b − Ax)

= xT AT Ax + bT b − xT AT b + bT Ax

= xT AT Ax + bT b − 2xT AT b
= xT AT (Ax − 2b) + bT b,
in which the transpose is used interchangeably for the adjoint because all
the numbers involved happen to be real. The norm is minimized where
d
rT r = 0

dx
(in which d/dx is the Jacobian operator of § 11.10). A requirement that
d T T
x A (Ax − 2b) + bT b = 0

dx
comes of combining the last two equations. Differentiating by the Jacobian

product rule (11.79) yields the equation
T
xT AT A + AT (Ax − 2b) = 0;

or, after transposing the equation, rearranging terms and dividing by 2, the
simplified equation
AT Ax = AT b.
Assuming (as warranted by § 13.6.2, next) that the n×n square matrix AT A
is invertible, the simplified equation implies the approximate but optimal
least-squares solution
−1 T
x = AT A A b (13.28)
to the unsolvable linear system (13.25) in the restricted but quite typical
case that A and b are real and A has full column rank.
Equation (13.28) plots the line on Fig. 13.1’s right. As the reader can
see, the line does not pass through all the points, for no line can; but it does
pass pretty convincingly nearly among them. In fact it passes optimally
nearly among them. No line can pass more nearly, in the squared-residual
norm sense of (13.27).14
14
Here is a nice example of the use of the mathematical adjective optimal in its ad-
verbial form. “Optimal” means “best.” Many problems in applied mathematics involve
discovering the best of something. What constitutes the best however can be a matter
of judgment, even of dispute. We shall leave to the philosopher and the theologian the
important question of what constitutes objective good, for applied mathematics is a poor
guide to such mysteries. The role of applied mathematics is to construct suitable models
to calculate quantities needed to achieve some definite good; its role is not, usually, to
identify the good as good in the first place.
One generally establishes mathematical optimality by some suitable, nonnegative, real
cost function or metric, and the less, the better. Strictly speaking, the mathematics
cannot tell us which metric to use, but where no other consideration prevails the applied
mathematician tends to choose the metric that best simplifies the mathematics at hand—
and, really, that is about as good a way to choose a metric as any. The metric (13.27) is
so chosen.
“But,” comes the objection, “what if some more complicated metric is better?”
Well, if the other metric really, objectively is better, then one should probably use it. In
general however the mathematical question is: what does one mean by “better?” Better by
which metric? Each metric is better according to itself. This is where the mathematician’s
experience, taste and judgment come in.
In the present section’s example, too much labor on the freeway job might actually slow
construction rather than speed it. One could therefore seek to fit not a line but some
downward-turning curve to the data. Mathematics offers many downward-turning curves.
A circle, maybe? Not likely. An experienced mathematician would probably reject the
circle on the aesthetic yet practical ground that the parabola b = αu2 + σu + γ lends
13.6.2 The invertibility of A∗ A

Section 13.6.1 has assumed correctly but unwarrantedly that the prod-
uct AT A were invertible for real A of full column rank. For real A, it
happens that AT = A∗ , so it only broadens the same assumption to suppose
that the product A∗ A were invertible for complex A of full column rank.15
This subsection warrants the latter assumption, thereby incidentally also
warranting the former.
Let A be a complex, m × n matrix of full column rank r = n ≤ m.
Suppose falsely that A∗ A were not invertible but singular. Since the prod-
uct A∗ A is a square, n × n matrix, this is to suppose (§ 13.1) that the prod-
uct’s rank r ′ < n were less than full, implying (§ 12.5.4) that its columns
(as its rows) depended on one another. This would mean that there existed
a nonzero, n-element vector u for which
A∗ Au = 0, In u 6= 0.
Left-multiplying by u∗ would give that
u∗ A∗ Au = 0, In u 6= 0,

n
|[Au]i |2 = 0, In u 6= 0.
X
∗
(Au) (Au) =
i=1
But this could only be so if
Au = 0, In u 6= 0,
impossible when the columns of A are independent. The contradiction

proves false the assumption which gave rise to it. The false assumption:
that A∗ A were singular.
Thus, the n × n product A∗ A is invertible for any tall or square, m × n
matrix A of full column rank r = n ≤ m.
itself to easier analysis. Yet even fitting a mere straight line offers choices. One might fit
the line to the points (bi , ui ) or (ln ui , ln bi ) rather than to the points (ui , bi ). The three
resulting lines differ subtly. They predict production differently. The adjective “optimal”
alone evidently does not always tell us all we need to know.
Section 6.3 offers a choice between averages that resembles in spirit this footnote’s choice
between metrics.
15
Notice that if A is tall, then A∗ A is a compact, n × n square, whereas AA∗ is a big,
m × m square. It is the compact square that concerns this section. The big square is not
very interesting and in any case is not invertible.
13.6.3 Positive definiteness

An n × n matrix C is positive definite if and only if
ℑ(u∗ Cu) = 0 and ℜ(u∗ Cu) > 0 for all In u 6= 0. (13.29)
As in § 13.6.2, here also when a matrix A has full column rank r = n ≤ m

the product u∗ A∗ Au = (Au)∗ (Au) is real and positive for all nonzero, n-
element vectors u. Thus per (13.29) the product A∗ A is positive definite for
any matrix A of full column rank.
An n × n matrix C is nonnegative definite if and only if
ℑ(u∗ Cu) = 0 and ℜ(u∗ Cu) ≥ 0 for all u. (13.30)
By reasoning like the last paragraph’s, the product A∗ A is nonnegative def-

inite for any matrix A whatsoever.
Such definitions might seem opaque, but their sense is that a positive
definite operator never reverses the thing it operates on, that the product Au
points more in the direction of u than of −u. Section 13.8 explains further.
A positive definite operator resembles a positive scalar in this sense.
13.6.4 The Moore-Penrose pseudoinverse

Not every m × n matrix A enjoys full rank. According to (12.16), however,
every m × n matrix A of rank r can be factored into a product16
A = BC
of an m × r tall or square matrix B and an r × n broad or square matrix C,

both of which factors themselves enjoy full rank r. (If A happens to have
full row or column rank, then one can just choose B = Im or C = In ;
but even if A lacks full rank, the Gauss-Jordan decomposition of eqn. 12.2
finds at least the full-rank factorization B = G> Ir , C = Ir G< .) This being
so, a conjecture seems warranted. Suppose that, inspired by (13.28), we
manipulated (13.25) by the successive steps
Ax ≈ b,
BCx ≈ b,
∗ −1
(B B) B BCx ≈ (B ∗ B)−1 B ∗ b,
∗
Cx ≈ (B ∗ B)−1 B ∗ b.
16
This subsection uses the symbols B and b for unrelated purposes, which is unfortunate
but conventional. See footnote 11.
Then suppose that we changed
C ∗ u ← x,
thus restricting x to the space addressed by the independent columns of C ∗ .

Continuing,
CC ∗ u ≈ (B ∗ B)−1 B ∗ b,
u ≈ (CC ∗ )−1 (B ∗ B)−1 B ∗ b.
Changing the variable back and (because we are conjecturing and can do as
we like), altering the “≈” sign to “=,”
x = C ∗ (CC ∗ )−1 (B ∗ B)−1 B ∗ b. (13.31)
Equation (13.31) has a pleasingly symmetrical form, and we know from

§ 13.6.2 at least that the two matrices it tries to invert are invertible. So
here is our conjecture:
• no x enjoys a smaller squared residual norm r∗ r than the x of (13.31)

does; and
• among all x that enjoy the same, minimal squared residual norm, the x
of (13.31) is strictly least in magnitude.
The conjecture is bold, but if you think about it in the right way it is not un-
warranted under the circumstance. After all, (13.31) does resemble (13.28),
the latter of which admittedly requires real A of full column rank but does
minimize the residual when its requirements are met; and, even if there were
more than one x which minimized the residual, one of them might be smaller
than the others: why not the x of (13.31)? One can but investigate.
The first point of the conjecture is symbolized
r∗ (x)r(x) ≤ r∗ (x + ∆x)r(x + ∆x),
where ∆x represents the deviation, whether small, moderate or large, of

some alternate x from the x of (13.31). According to (13.26), this is
[b − Ax]∗ [b − Ax] ≤ [b − (A)(x + ∆x)]∗ [b − (A)(x + ∆x)].
Reorganizing,
[b − Ax]∗ [b − Ax] ≤ [(b − Ax) − A ∆x]∗ [(b − Ax) − A ∆x].

Distributing factors and canceling like terms,
0 ≤ −∆x∗ A∗ (b − Ax) − (b − Ax)∗ A ∆x + ∆x∗ A∗ A ∆x.
But according to (13.31) and the full-rank factorization A = BC,
A∗ (b − Ax) = A∗ b − A∗ Ax
= [C ∗ B ∗ ][b] − [C ∗ B ∗ ][BC][C ∗ (CC ∗ )−1 (B ∗ B)−1 B ∗ b]
= C ∗ B ∗ b − C ∗ (B ∗ B)(CC ∗ )(CC ∗ )−1 (B ∗ B)−1 B ∗ b
= C ∗ B ∗ b − C ∗ B ∗ b = 0,
which reveals two of the inequality’s remaining three terms to be zero, leav-
ing an assertion that
0 ≤ ∆x∗ A∗ A ∆x.
Each step in the present paragraph is reversible,17 so the assertion in the
last form is logically equivalent to the conjecture’s first point, with which
the paragraph began. Moreover, the assertion in the last form is correct
because the product of any matrix and its adjoint according to § 13.6.3 is a
nonnegative definite operator, thus establishing the conjecture’s first point.
The conjecture’s first point, now established, has it that no x+∆x enjoys
a smaller squared residual norm than the x of (13.31) does. It does not claim
that no x + ∆x enjoys the same, minimal squared residual norm. The latter
case is symbolized
r∗ (x)r(x) = r∗ (x + ∆x)r(x + ∆x),
or equivalently by the last paragraph’s logic,
0 = ∆x∗ A∗ A ∆x;
or in other words,
A ∆x = 0.
But A = BC, so this is to claim that
B(C ∆x) = 0,
which since B has full column rank is possible only if
C ∆x = 0.
17
The paragraph might inscrutably but logically instead have ordered the steps in reverse
as in §§ 6.3.2 and 9.5. See Ch. 6’s footnote 15.
Considering the product ∆x∗ x in light of (13.31) and the last equation, we
observe that
∆x∗ x = ∆x∗ [C ∗ (CC ∗ )−1 (B ∗ B)−1 B ∗ b]

= [C ∆x]∗ [(CC ∗ )−1 (B ∗ B)−1 B ∗ b],
which is to observe that

∆x∗ x = 0
for any ∆x for which x + ∆x achieves minimal squared residual norm.
Returning attention to the conjecture, its second point is symbolized
x∗ x < (x + ∆x)∗ (x + ∆x)
for any
∆x 6= 0
for which x+∆x achieves minimal squared residual norm (note that it’s “<”
this time, not “≤” as in the conjecture’s first point). Distributing factors
and canceling like terms,
0 < x∗ ∆x + ∆x∗ x + ∆x∗ ∆x.
But the last paragraph has found that ∆x∗ x = 0 for precisely such ∆x as
we are considering here, so the last inequality reduces to read
0 < ∆x∗ ∆x,
which naturally for ∆x 6= 0 is true. Since each step in the paragraph is

reversible, reverse logic establishes the conjecture’s second point.
With both its points established, the conjecture is true.
If A = BC is a full-rank factorization, then the matrix18
A† ≡ C ∗ (CC ∗ )−1 (B ∗ B)−1 B ∗ (13.32)
of (13.31) is called the Moore-Penrose pseudoinverse of A, more briefly the

pseudoinverse of A. Whether underdetermined, exactly determined, overde-
termined or even degenerate, every matrix has a Moore-Penrose pseudoin-
verse. Yielding the optimal approximation
x = A† b, (13.33)
18
Some books print A† as A+ .
the Moore-Penrose solves the linear system (13.25) as well as the system
can be solved—exactly if possible, with minimal squared residual norm if
impossible. If A is square and invertible, then the Moore-Penrose A† = A−1
is just the inverse, and then of course (13.33) solves the system uniquely and
exactly. Nothing can solve the system uniquely if A has broad shape but the
Moore-Penrose still solves the system exactly in that case as long as A has
full row rank, moreover minimizing the solution’s squared magnitude x∗ x
(which the solution of eqn. 13.23 fails to do). If A lacks full row rank,
then the Moore-Penrose solves the system as nearly as the system can be
solved (as in Fig. 13.1) and as a side-benefit also minimizes x∗ x. The Moore-
Penrose is thus a general-purpose solver and approximator for linear systems.
It is a significant discovery.19
13.7 The multivariate Newton-Raphson iteration

When we first met the Newton-Raphson iteration in § 4.8 we lacked the
matrix notation and algebra to express and handle vector-valued functions
adeptly. Now that we have the notation and algebra we can write down the
multivariate Newton-Raphson iteration almost at once.
The iteration approximates the nonlinear vector function f (x) by its
tangent
d
f̃k (x) = f (xk ) + f (x) (x − xk ),
dx x=xk
where df /dx is the Jacobian derivative of § 11.10. It then approximates the

root xk+1 as the point at which f̃k (xk+1 ) = 0:

d
f̃k (xk+1 ) = 0 = f (xk ) + f (x) (xk+1 − xk ).
dx x=xk
Solving for xk+1 (approximately if necessary), we have that

†
d
xk+1 = x − f (x) f (x) , (13.34)

dx
x=xk
where [·]† is the Moore-Penrose pseudoinverse of § 13.6—which is just the

ordinary inverse [·]−1 of § 13.1 if f and x happen each to have the same
number of elements. Refer to § 4.8 and Fig. 4.5.20
19
[5, § 3.3][50, “Moore-Penrose generalized inverse”]
20
[53]
13.8. THE DOT PRODUCT 349
Despite the Moore-Penrose notation of (13.34), the Newton-Raphson

iteration is not normally meant to be applied at a value of x for which the
Jacobian is degenerate. The iteration intends rather in light of (13.32) that
−1
[df /dx]∗ ([df /dx] [df /dx]∗ )

†  if r = m ≤ n,
d 
f (x) = [df /dx]−1 if r = m = n, (13.35)
dx −1
([df /dx]∗ [df /dx]) [df /dx]∗

if r = n ≤ m,

where B = Im in the first case and C = In in the last. It does not intend
to use the full (13.32). If both r < m and r < n—which is to say, if the
Jacobian is degenerate—then (13.35) fails, as though the curve of Fig. 4.5
ran horizontally at the test point—when one quits, restarting the iteration
from another point.
13.8 The dot product

The dot product of two vectors, also called the inner product,21 is the product
of the two vectors to the extent to which they run in the same direction. It
is written
a · b.
In general,
a · b = (a1 e1 + a2 e2 + · · · + an en ) · (b1 e1 + b2 e2 + · · · + bn en ).
But if the dot product is to mean anything, it must be that
ei · ej = δij . (13.36)
Therefore,
a · b = a1 b1 + a2 b2 + · · · + an bn ;
or, more concisely,
∞
X
a · b ≡ aT b = aj bj . (13.37)
j=−∞
21
The term inner product is often used to indicate a broader class of products than
the one defined here, especially in some of the older literature. Where used, the notation
usually resembles ha, bi or (b, a), both of which mean a∗ · b (or, more broadly, some
similar product), except that which of a and b is conjugated depends on the author. Most
recently, at least in the author’s country, the usage ha, bi ≡ a∗ · b seems to be emerging
as standard where the dot is not used [5, § 3.1][21, Ch. 4]. This book prefers the dot.
The dot notation does not worry whether its arguments are column or row
vectors, incidentally:
a · b = a · bT = aT · b = aT · bT = aT b.
That is, if either vector is wrongly oriented, the notation implicitly reorients
it before using it. (The more orderly notation aT b by contrast assumes that
both are proper column vectors.)
Where vectors may have complex elements, usually one is not interested
in a · b so much as in
∞
X
∗ ∗
a ·b≡a b= a∗j bj . (13.38)
j=−∞
The reason is that
ℜ(a∗ · b) = ℜ(a) · ℜ(b) + ℑ(a) · ℑ(b),
with the product of the imaginary parts added not subtracted, thus honoring
the right Argand sense of “the product of the two vectors to the extent to
which they run in the same direction.”
By the Pythagorean theorem, the dot product
|a|2 = a∗ · a (13.39)
gives the square of a vector’s magnitude, always real, never negative. The
unit vector in a’s direction then is
a a
â ≡ =√ , (13.40)
|a| a∗ · a
from which
|â|2 = â∗ · â = 1. (13.41)
When two vectors do not run in the same direction at all, such that
a∗ · b = 0, (13.42)
the two vectors are said to lie orthogonal to one another. Geometrically this
puts them at right angles. For other angles θ between two vectors,
â∗ · b̂ = cos θ, (13.43)
which formally defines the angle θ even when a and b have more than three
elements each.
13.9. THE COMPLEX VECTOR TRIANGLE INEQUALITIES 351
13.9 The complex vector triangle inequalities

The triangle inequalities (2.44) and (3.21) lead one to hypothesize generally
that
|a| − |b| ≤ |a + b| ≤ |a| + |b| (13.44)
for any complex, n-dimensional vectors a and b.
The proof of the sum hypothesis that |a + b| ≤ |a| + |b| is by contradic-
tion. We suppose falsely that
|a + b| > |a| + |b| .
Squaring and using (13.39),
(a + b)∗ · (a + b) > a∗ · a + 2 |a| |b| + b∗ · b.
Distributing factors and canceling like terms,
a∗ · b + b∗ · a > 2 |a| |b| .
Splitting a and b each into real and imaginary parts on the inequality’s left
side and then halving both sides,
ℜ(a) · ℜ(b) + ℑ(a) · ℑ(b) > |a| |b| .
Defining the new, 2n-dimensional real vectors

ℜ(a1 ) ℜ(b1 )
2 3 2 3
6
6 ℑ(a1 ) 7
7
6
6 ℑ(b1 ) 7
7
6
6 ℜ(a2 ) 7
7
6
6 ℜ(b2 ) 7
7
f≡ 6 ℑ(a2 ) 7,
7
g≡ 6 ℑ(b2 ) 7,
7
.. ..
6 6
6 7 6 7
6
6 . 7
7
6
6 . 7
7
4 ℜ(an ) 5 4 ℜ(bn ) 5
ℑ(an ) ℑ(bn )
we make the inequality to be
f · g > |f | |g| ,
in which we observe that the left side must be positive because the right side
is nonnegative. (This naturally is impossible for any case in which f = 0 or
g = 0, among others, but wishing to establish impossibility for all cases we
pretend not to notice and continue reasoning as follows.) Squaring again,
(f · g)2 > (f · f )(g · g);

or, in other words, X X

fi gi fj gj > fi2 gj2 .
i,j i,j
Reordering factors,
X X
[(fi gj )(gi fj )] > (fi gj )2 .
i,j i,j
2
P
Subtracting i (fi gi ) from each side,
X X
[(fi gj )(gi fj )] > (fi gj )2 ,
i6=j i6=j
which we can cleverly rewrite in the form

X X
[2(fi gj )(gi fj )] > [(fi gj )2 + (gi fj )2 ],
i<j i<j
P P2n−1 P2n
where i<j = i=1 j=i+1 . Transferring all terms to the inequality’s
right side, X
0> [(fi gj )2 + 2(fi gj )(gi fj ) + (gi fj )2 ].
i<j
This is X
0> [fi gj + gi fj ]2 ,
i<j
which, since we have constructed the vectors f and g to have real elements
only, is impossible in all cases. The contradiction proves false the assumption
that gave rise to it, thus establishing the sum hypothesis of (13.44).
The difference hypothesis that |a| − |b| ≤ |a + b| is established by defin-
ing a vector c such that
a + b + c = 0,
whereupon according to the sum hypothesis (which we have already estab-
lished),
|a + c| ≤ |a| + |c| ,
|b + c| ≤ |b| + |c| .
That is,
|−b| ≤ |a| + |−a − b| ,

|−a| ≤ |b| + |−a − b| ,
13.10. THE ORTHOGONAL COMPLEMENT 353
which is the difference hypothesis in disguise. This completes the proof

of (13.44).
As in § 3.10, here too we can extend the sum inequality to the even more
general form
X X
ak ≤ |ak | . (13.45)

k k
13.10 The orthogonal complement

The m × (m − r) kernel (§ 13.3)22
A⊥ ≡ A∗K (13.46)
is an interesting matrix. By definition of the kernel, the columns of A∗K

are the independent vectors uj for which A∗ uj = 0, which—inasmuch as
the rows of A∗ are the adjoints of the columns of A—is possible only when
each uj lies orthogonal to every column of A. This says that the columns of
A⊥ ≡ A∗K address the complete space of vectors that lie orthogonal to A’s
columns, such that
A⊥∗ A = 0 = A∗ A⊥ . (13.47)
The matrix A⊥ is called the orthogonal complement 23 or perpendicular ma-
trix to A.
Among other uses, the orthogonal complement A⊥ supplies the columns
A lacks to reach full row rank. Properties include that
A∗K = A⊥ ,
(13.48)
A∗⊥ = AK .
13.11 Gram-Schmidt orthonormalization

If a vector x = AK a belongs to a kernel space AK (§ 13.3), then so equally
does any αx. If the vectors x1 = AK a1 and x2 = AK a2 both belong, then
so does α1 x1 + α2 x2 . If I claim AK = [3 4 5; −1 1 0]T to represent a kernel,
then you are not mistaken arbitrarily to rescale each column of my AK by
a separate nonzero factor, instead for instance representing the same kernel
22
The symbol A⊥ [30][5][42] can be pronounced “A perp,” short for “A perpendicular,”
since by (13.47) A⊥ is in some sense perpendicular to A.
If we were really precise, we might write not A⊥ but A⊥(m) . Refer to footnote 5.
23
[30, § 3.VI.3]
as AK = [6 8 0xA; 17 − 71 0]T . Kernel vectors have no inherent scale. Style

generally asks the applied mathematician to remove the false appearance of
scale by using (13.40) to normalize the columns of a kernel matrix to unit
magnitude before reporting them. The same goes for the eigenvectors of
Ch. 14 to come.
Where a kernel matrix AK has two or more columns (or a repeated
eigenvalue has two or more eigenvectors), style generally asks the applied
mathematician not only to normalize but also to orthogonalize the columns
before reporting them. One orthogonalizes a vector b with respect to a
vector a by subtracting from b a multiple of a such that
a∗ · b⊥ = 0,
b⊥ ≡ b − βa,
where the symbol b⊥ represents the orthogonalized vector. Substituting the

second of these equations into the first and solving for β yields
a∗ · b
β= .
a∗ · a
Hence,
a∗ · b⊥ = 0,
a∗ · b (13.49)
b⊥ ≡ b − a.
a∗ · a
√
But according to (13.40), a = â a∗ · a; and according to (13.41), â∗ · â = 1;
so,
b⊥ = b − â(â∗ · b); (13.50)
or, in matrix notation,
b⊥ = b − â(â∗ )(b).
This is arguably better written
b⊥ = [I − (â)(â∗ )] b (13.51)
(observe that it’s [â][â∗ ], a matrix, rather than the scalar [â∗ ][â]).
One orthonormalizes a set of vectors by orthogonalizing them with re-
spect to one another, then by normalizing each of them to unit magnitude.
The procedure to orthonormalize several vectors
{x1 , x2 , x3 , . . . , xn }
13.11. GRAM-SCHMIDT ORTHONORMALIZATION 355
therefore is as follows. First, normalize x1 by (13.40); call the result x̂1⊥ .

Second, orthogonalize x2 with respect to x̂1⊥ by (13.50) or (13.51), then
normalize it; call the result x̂2⊥ . Third, orthogonalize x3 with respect to x̂1⊥
then to x̂2⊥ , then normalize it; call the result x̂3⊥ . Proceed in this manner
through the several xj . Symbolically,
xj⊥
x̂j⊥ = q ,
x∗j⊥ xj⊥
"j−1 # (13.52)
Y
xj⊥ ≡ (I − x̂i⊥ x̂∗i⊥ ) xj .
i=1
By the vector replacement principle of § 12.4 in light of (13.49), the resulting

orthonormal set of vectors
{x̂1⊥ , x̂2⊥ , x̂3⊥ , . . . , x̂n⊥ }
addresses the same space as did the original set.

Orthonormalization naturally works equally for any linearly independent
set of vectors, not only for kernel vectors or eigenvectors. By the technique,
one can conveniently replace a set of independent vectors by an equivalent,
neater, orthonormal set which addresses precisely the same space.
13.11.1 Efficient implementation

To turn an equation like the latter line of (13.52) into an efficient numerical
algorithm sometimes demands some extra thought, in perspective of what-
ever it happens to be that one is trying to accomplish. If all one wants is
some vectors orthonormalized, then the equation as written is neat but is
overkill because the product x̂i⊥ x̂∗i⊥ is a matrix, whereas the product x̂∗i⊥ xj
implied by (13.50) is just a scalar. Fortunately, one need not apply the
latter line of (13.52) exactly as written. Q One can instead introduce inter-
mediate vectors xji , representing the multiplication in the admittedly
messier form
xj1 ≡ xj ,
xj(i+1) ≡ xji − (x̂∗i⊥ · xji ) x̂i⊥ , (13.53)
xj⊥ = xjj .
Besides obviating the matrix I − x̂i⊥ x̂∗i⊥ and the associated matrix multipli-
cation, the messier form (13.53) has the significant additional practical virtue
that it lets one forget each intermediate vector xji immediately after using
it. (A well written orthonormalizing computer program reserves memory for
one intermediate vector only, which memory it repeatedly overwrites—and,
actually, probably does not even reserve that much, working rather in the
memory space it has already reserved for x̂j⊥ .)24
Other equations one algorithmizes can likewise benefit from thoughtful
rendering.
13.11.2 The Gram-Schmidt decomposition

The orthonormalization technique this section has developed is named the
Gram-Schmidt process. One can turn it into the Gram-Schmidt decomposi-
tion
A = QR = QU DS,
(13.54)
R ≡ U DS,
also called the orthonormalizing or QR decomposition, by an algorithm

that somewhat resembles the Gauss-Jordan algorithm of § 12.3.3; except
that (12.4) here becomes
A = Q̃Ũ D̃ S̃ (13.55)
and initially Q̃ ← A. By elementary column operations based on (13.52)

and (13.53), the algorithm gradually transforms Q̃ into a dimension-limited,
m×r matrix Q of orthonormal columns, distributing the inverse elementaries
to Ũ , D̃ and S̃ according to Table 12.1—where the latter three working
matrices ultimately become the extended-operational factors U , D and S
of (13.54).
Borrowing the language of computer science we observe that the indices i
and j of (13.52) and (13.53) imply a two-level nested loop, one level looping
over j and the other over i. The equations suggest j-major nesting, with
the loop over j at the outer level and the loop over i at the inner, such that
the several (i, j) index pairs occur in the sequence (reading left to right then
top to bottom)
(1, 2)
(1, 3) (2, 3)
(1, 4) (2, 4) (3, 4)
..
··· ··· ··· .
24
[67, “Gram-Schmidt process,” 04:48, 11 Aug. 2007]
In reality, however, (13.53)’s middle line requires only that no x̂i⊥ be used
before it is fully calculated; otherwise that line does not care which (i, j)
pair follows which. The i-major nesting
(1, 2) (1, 3) (1, 4) · · ·

(2, 3) (2, 4) · · ·
(3, 4) · · ·
..
.
bringing the very same index pairs in a different sequence, is just as valid.
We choose i-major nesting on the subtle ground that it affords better infor-
mation to the choice of column index p during the algorithm’s step 3.
The algorithm, in detail:
1. Begin by initializing
Ũ ← I, D̃ ← I, S̃ ← I,
Q̃ ← A,
i ← 1.
reënters here from step 9 below.) Observe that Ũ enjoys the major
partial unit triangular form L{i−1}T (§ 11.8.5), that D̃ is a general
scaling operator (§ 11.7.2) with d˜jj = 1 for all j ≥ i, that S̃ is permutor
(§ 11.7.1), and that the first through (i − 1)th columns of Q̃ consist of
mutually orthonormal unit vectors.
3. Choose a column p ≥ i of Q̃ containing at least one nonzero element.

(The simplest choice is perhaps p = i as long as the ith column does
not happen to be null, but one might instead prefer to choose the
column of greatest magnitude, or to choose randomly, among other
heuristics.) If Q̃ is null in and rightward of its ith column such that no
column p ≥ i remains available to choose, then skip directly to step 10.

A = Q̃T[i↔p] T[i↔p] Ũ T[i↔p] T[i↔p] D̃T[i↔p] T[i↔p]S̃

= Q̃T[i↔p] T[i↔p] Ũ T[i↔p] D̃ T[i↔p]S̃ ,
where the latter line has applied a rule from Table 12.1, interchange
the chosen pth column to the ith position by
Q̃ ← Q̃T[i↔p],
Ũ ← T[i↔p]Ũ T[i↔p] ,
S̃ ← T[i↔p]S̃.

A = Q̃T(1/α)[i] Tα[i] Ũ T(1/α)[i] Tα[i] D̃ S̃,
normalize the ith column of Q̃ by
Q̃ ← Q̃T(1/α)[i] ,
Ũ ← Tα[i] Ũ T(1/α)[i] ,
D̃ ← Tα[i] D̃,
where rh i h i
∗
α= Q̃ · Q̃ .
∗i ∗i
6. Initialize
j ← i + 1.
reënters here from step 8 below.) If j > n then skip directly to step 9.
Otherwise, observing that (13.55) can be expanded to read

A = Q̃T−β[ij] Tβ[ij]Ũ D̃S̃,
orthogonalize the jth column of Q̃ per (13.53) with respect to the ith
column by
Q̃ ← Q̃T−β[ij] ,
Ũ ← Tβ[ij] Ũ ,
where h i∗ h i
β = Q̃ · Q̃ .
∗i ∗j
8. Increment
j ←j+1
and return to step 7.
9. Increment
i←i+1
and return to step 2.
10. Let
Q ≡ Q̃, U ≡ Ũ , D ≡ D̃, S ≡ S̃,
r = i − 1.
End.
Though the Gram-Schmidt algorithm broadly resembles the Gauss-
Jordan, at least two significant differences stand out: (i) the Gram-Schmidt
is one-sided because it operates only on the columns of Q̃, never on the
rows; (ii) since Q is itself dimension-limited, the Gram-Schmidt decomposi-
tion (13.54) needs and has no explicit factor Ir .
As in § 12.5.7, here also one sometimes prefers that S = I. The algorithm
optionally supports this preference if the m × n matrix A has full column
rank r = n, when null columns cannot arise, if one always chooses p = i
during the algorithm’s step 3. Such optional discipline maintains S = I
when desired.
Whether S = I or not, the matrix Q = QIr has only r columns, so one
can write (13.54) as
A = (QIr )(R).
Reassociating factors, this is
A = (Q)(Ir R), (13.56)
which per (12.16) is a proper full-rank factorization with which one can
compute the pseudoinverse A† of A (see eqn. 13.32, above; but see also
eqn. 13.65, below).
If the Gram-Schmidt decomposition (13.54) looks useful, it is even more
useful than it looks. The most interesting of its several factors is the m × r
orthonormalized matrix Q, whose orthonormal columns address the same
space the columns of A themselves address. If Q reaches the maximum pos-
sible rank r = m, achieving square, m × m shape, then it becomes a unitary
matrix —the subject of § 13.12. Before treating the unitary matrix, however,
let us pause to extract a kernel from the Gram-Schmidt decomposition in
§ 13.11.3, next.
13.11.3 The Gram-Schmidt kernel formula

Like the Gauss-Jordan decomposition in (13.7), the Gram-Schmidt decom-
position too brings a kernel formula. To develop and apply it, one decom-
poses an m × n matrix
A = QR (13.57)
per the Gram-Schmidt (13.54) and its algorithm in § 13.11.2. Observing
that the r independent columns of the m × r matrix Q address the same
space the columns of A address, one then constructs the m × (r + m) matrix
A′ ≡ Q + Im H−r = Q Im

(13.58)
and decomposes it too,

A′ = Q′ R′ , (13.59)
again by Gram-Schmidt—with the differences that, this time, one chooses
p = 1, 2, 3, . . . , r during the first r instances of the algorithm’s step 3, and
that one skips the unnecessary step 7 for all j ≤ r; on the ground that the
earlier Gram-Schmidt application of (13.57) has already orthonormalized
first r columns of A′ , which columns, after all, are just Q. The resulting
m × m, full-rank square matrix
Q′ = Q + A⊥ H−r = Q A⊥

(13.60)
consists of
• r columns on the left that address the same space the columns of A
address and
• m−r columns on the right that give a complete orthogonal complement

(§ 13.10) A⊥ of A.
Each column has unit magnitude and conveniently lies orthogonal to every
other column, left and right.
Equation (13.60) is probably the more useful form, but the Gram-
Schmidt kernel formula as such,
A∗K = A⊥ = Q′ Hr Im−r , (13.61)
extracts the rightward columns that express the kernel, not of A, but of A∗ .
To compute the kernel of a matrix B by Gram-Schmidt one sets A = B ∗
and applies (13.57) through (13.61). Refer to (13.48).
13.12. THE UNITARY MATRIX 361
In either the form (13.60) or the form (13.61), the Gram-Schmidt kernel
formula does everything the Gauss-Jordan kernel formula (13.7) does and
in at least one sense does it better; for, if one wants a Gauss-Jordan kernel
orthonormalized, then one must orthonormalize it as an extra step, whereas
the Gram-Schmidt kernel comes already orthonormalized.
Being square, the m × m matrix Q′ is a unitary matrix, as the last
paragraph of § 13.11.2 has alluded. The unitary matrix is the subject of
§ 13.12 that follows.
13.12 The unitary matrix

When the orthonormalized matrix Q of the Gram-Schmidt decomposition
(13.54) is square, having the maximum possible rank r = m, it brings one
property so interesting that the property merits a section of its own. The
property is that
Q∗ Q = Im = QQ∗ . (13.62)
The reason that Q∗ Q = Im is that Q’s columns are orthonormal, and that
the very definition of orthonormality demands that the dot product [Q]∗∗i ·
[Q]∗j of orthonormal columns be zero unless i = j, when the dot product of
a unit vector with itself is unity. That Im = QQ∗ is unexpected, however,
until one realizes25 that the equation Q∗ Q = Im characterizes Q∗ to be the
rank-m inverse of Q, and that § 13.1 lets any rank-m inverse (orthonormal
or otherwise) attack just as well from the right as from the left. Thus,
Q−1 = Q∗ , (13.63)
a very useful property. A matrix Q that satisfies (13.62), whether derived
from the Gram-Schmidt or from elsewhere, is called a unitary matrix. (Note
that the permutor of § 11.7.1 enjoys the property of eqn. 13.63 precisely
because it is unitary.)
One immediate consequence of (13.62) is that a square matrix with either
orthonormal columns or orthonormal rows is unitary and has both.
The product of two or more unitary matrices is itself unitary if the ma-
trices are of the same dimensionality. To prove it, consider the product
Q = Qa Qb (13.64)
of m × m unitary matrices Qa and Qb . Let the symbols qj , qaj and qbj
respectively represent the jth columns of Q, Qa and Qb and let the sym-
bol qbij represent the ith element of qbj . By the columnwise interpretation
25
[21, § 4.4]
(§ 11.1.3) of matrix multiplication,

X
qj = qbij qai .
i
The adjoint dot product of any two of Q’s columns then is

X
q∗j ′ · qj = ∗
qbi ∗
′ j ′ qbij qai′ · qai .
i,i′
But q∗ai′ · qai = δi′ i because Qa is unitary,26 so

X
q∗j ′ · qj = ∗
qbij ∗
′ qbij = qbj ′ · qbj = δj ′ j ,
which says neither more nor less than that the columns of Q are orthonormal,
which is to say that Q is unitary, as was to be demonstrated.
Unitary operations preserve length. That is, operating on an m-element
vector by an m × m unitary matrix does not alter the vector’s magnitude.
To prove it, consider the system
Qx = b.
Multiplying the system by its own adjoint yields
x∗ Q∗ Qx = b∗ b.
But according to (13.62), Q∗ Q = Im ; so,
x∗ x = b∗ b,
as was to be demonstrated.
Equation (13.63) lets one use the Gram-Schmidt decomposition (13.54)
to invert a square matrix as
A−1 = R−1 Q∗ = S ∗ D −1 U −1 Q∗ . (13.65)
Unitary extended operators are certainly possible, for if Q is an m × m

dimension-limited matrix, then the extended operator
Q∞ = Q + (I − Im ),
26
This is true only for 1 ≤ i ≤ m, but you knew that already.
13.12. THE UNITARY MATRIX 363
which is just Q with ones running out the main diagonal from its active
region, itself meets the unitary criterion (13.62) for m = ∞.
Unitary matrices are so easy to handle that they can sometimes justify
significant effort to convert a model to work in terms of them if possible.
We shall meet the unitary matrix again in §§ 14.10 and 14.12.
The chapter as a whole has demonstrated at least in theory (and usu-
ally in practice) techniques to solve any linear system characterized by a
matrix of finite dimensionality, whatever the matrix’s rank or shape. It has
explained how to orthonormalize a set of vectors and has derived from the
explanation the useful Gram-Schmidt decomposition. As the chapter’s in-
troduction had promised, the matrix has shown its worth here; for without
the matrix’s notation, arithmetic and algebra most of the chapter’s findings
would have lain beyond practical reach. And even so, the single most inter-
esting agent of matrix arithmetic remains yet to be treated. This last is the
eigenvalue, and it is the subject of Ch. 14, next.
Chapter 14
The eigenvalue
The eigenvalue is a scalar by which a square matrix scales a vector without

otherwise changing it, such that
Av = λv.
This chapter analyzes the eigenvalue and the associated eigenvector it scales.
Before treating the eigenvalue proper, the chapter gathers from across
Chs. 11 through 14 several properties all invertible square matrices share, as-
sembling them in § 14.2 for reference. One of these regards the determinant,
which opens the chapter.
14.1 The determinant

Through Chs. 11, 12 and 13 the theory of the matrix has developed slowly
but pretty straightforwardly. Here comes the first unexpected turn.
It begins with an arbitrary-seeming definition. The determinant of an
n × n square matrix A is the sum of n! terms, each term the product of n
elements, no two elements from the same row or column, terms of positive
parity adding to and terms of negative parity subtracting from the sum—a
term’s parity (§ 11.6) being the parity of the permutor (§ 11.7.1) marking
the positions of the term’s elements.
Unless you already know about determinants, the definition alone might
seem hard to parse, so try this. The inverse of the general 2 × 2 square
matrix
a11 a12
A2 = ,
a21 a22
365
366 CHAPTER 14. THE EIGENVALUE
by the Gauss-Jordan method or any other convenient technique, is found to

be
a22 −a12
−a21 a11
A−1
2 = .
a11 a22 − a12 a21
The quantity1
det A2 = a11 a22 − a12 a21
in the denominator is defined to be the determinant of A2 . Each of the de-
terminant’s terms includes one element from each column of the matrix and
one from each row, with parity giving the term its ± sign. The determinant
of the general 3 × 3 square matrix by the same rule is
det A3 = (a11 a22 a33 + a12 a23 a31 + a13 a21 a32 )
− (a13 a22 a31 + a12 a21 a33 + a11 a23 a32 );
and indeed if we tediously invert such a matrix symbolically, we do find that

quantity in the denominator there.
The parity rule merits a more careful description. The parity of a term
like a12 a23 a31 is positive because the parity of the permutor, or interchange
quasielementary (§ 11.7.1),
2 3
0 1 0
P = 4 0 0 1 5
1 0 0
marking the positions of the term’s elements is positive. The parity of

a term like a13 a22 a31 is negative for the same reason. The determinant
comprehends all possible such terms, n! in number, half of positive parity
and half of negative. (How do we know that exactly half are of positive
and half, negative? Answer: by pairing the terms. For every term like
a12 a23 a31 whose marking permutor is P , there is a corresponding a13 a22 a31
whose marking permutor is T[1↔2] P , necessarily of opposite parity. The sole
exception to the rule is the 1 × 1 square matrix, which has no second term
to pair.)
1
The determinant det A used to be written |A|, an appropriately terse notation for
which the author confesses some nostalgia. The older notation |A| however unluckily
suggests “the magnitude of A,” which though not quite the wrong idea is not quite the right
idea, either. The magnitude |z| of a scalar or |u| of a vector is a real-valued, nonnegative,
nonanalytic function of the elements of the quantity in question, whereas the determinant
det A is a complex-valued, analytic function. The book follows convention by denoting
the determinant as det A for this reason among others.
14.1. THE DETERMINANT 367
Normally the context implies a determinant’s rank n, but the nonstan-

dard notation
det(n) A
is available especially to call the rank out, stating explicitly that the deter-
minant has exactly n! terms. (See also §§ 11.3.5 and 11.5 and eqn. 11.49.2 )
It is admitted3 that we have not, as yet, actually shown the determinant
to be a generally useful quantity; we have merely motivated and defined it.
Historically the determinant probably emerged not from abstract consider-
ations but for the mundane reason that the quantity it represents occurred
frequently in practice (as in the A−12 example above). Nothing however log-
ically prevents one from simply defining some quantity which, at first, one
merely suspects will later prove useful. So we do here.4
14.1.1 Basic properties

The determinant det A enjoys several useful basic properties.
• If 
ai′′ ∗
 when i = i′ ,
ci∗ = ai′ ∗ when i = i′′ ,

ai∗ otherwise,

or if 
a∗j ′′
 when j = j ′ ,
c∗j = a∗j ′ when j = j ′′ ,

a∗j otherwise,

where i′′ 6= i′ and j ′′ 6= j ′ , then
det C = − det A. (14.1)
Interchanging rows or columns negates the determinant.
• If (
αai∗ when i = i′ ,
ci∗ =
ai∗ otherwise,
2
And further Ch. 13’s footnotes 5 and 22.
3
[21, § 1.2]
4
[21, Ch. 1]
or if (
αa∗j when j = j ′ ,
c∗j =
a∗j otherwise,
then
det C = α det A. (14.2)
Scaling a single row or column of a matrix scales the matrix’s deter-
minant by the same factor. (Equation 14.2 tracks the linear scaling
property of § 7.3.3 and of eqn. 11.2.)
• If (
ai∗ + bi∗ when i = i′ ,
ci∗ =
ai∗ = bi∗ otherwise,
or if (
a∗j + b∗j when j = j ′ ,
c∗j =
a∗j = b∗j otherwise,
then
det C = det A + det B. (14.3)
If one row or column of a matrix C is the sum of the corresponding rows
or columns of two other matrices A and B, while the three matrices
remain otherwise identical, then the determinant of the one matrix is
the sum of the determinants of the other two. (Equation 14.3 tracks
the linear superposition property of § 7.3.3 and of eqn. 11.2.)
• If
ci′ ∗ = 0,
or if
c∗j ′ = 0,
then
det C = 0. (14.4)
A matrix with a null row or column also has a null determinant.
• If
ci′′ ∗ = γci′ ∗ ,
or if
c∗j ′′ = γc∗j ′ ,
det C = 0. (14.5)
The determinant is zero if one row or column of the matrix is a multiple

of another.
• The determinant of the adjoint is just the determinant’s conjugate,

and the determinant of the transpose is just the determinant itself:
det C ∗ = (det C)∗ ;
(14.6)
det C T = det C.
These basic properties are all fairly easy to see if the definition of the de-
terminant is clearly understood. Equations (14.2), (14.3) and (14.4) come
because each of the n! terms in the determinant’s expansion has exactly
one element from row i′ or column j ′ . Equation (14.1) comes because a
row or column interchange reverses parity. Equation (14.6) comes because
according to § 11.7.1, the permutors P and P ∗ always have the same par-
ity, and because the adjoint operation individually conjugates each element
of C. Finally, (14.5) comes because, in this case, every term in the deter-
minant’s expansion finds an equal term of opposite parity to offset it. Or,
more formally, (14.5) comes because the following procedure does not al-
ter the matrix: (i) scale row i′′ or column j ′′ by 1/γ; (ii) scale row i′ or
column j ′ by γ; (iii) interchange rows i′ ↔ i′′ or columns j ′ ↔ j ′′ . Not
altering the matrix, the procedure does not alter the determinant either;
and indeed according to (14.2), step (ii)’s effect on the determinant cancels
that of step (i). However, according to (14.1), step (iii) negates the determi-
nant. Hence the net effect of the procedure is to negate the determinant—to
negate the very determinant the procedure is not permitted to alter. The
apparent contradiction can be reconciled only if the determinant is zero to
begin with.
From the foregoing properties the following further property can be de-
duced.
• If (
ai∗ + αai′ ∗ when i = i′′ ,
ci∗ =
ai∗ otherwise,
or if (
a∗j + αa∗j ′ when j = j ′′ ,
c∗j =
a∗j otherwise,

det C = det A. (14.7)
Adding to a row or column of a matrix a multiple of another row or
column does not change the matrix’s determinant.
To derive (14.7) for rows (the column proof is similar), one defines a matrix B
such that (
αai′ ∗ when i = i′′ ,
bi∗ ≡
ai∗ otherwise.
From this definition, bi′′ ∗ = αai′ ∗ whereas bi′ ∗ = ai′ ∗ , so
bi′′ ∗ = αbi′ ∗ ,
which by (14.5) guarantees that
det B = 0.
On the other hand, the three matrices A, B and C differ only in the (i′′ )th
row, where [C]i′′ ∗ = [A]i′′ ∗ + [B]i′′ ∗ ; so, according to (14.3),
det C = det A + det B.
Equation (14.7) results from combining the last two equations.
14.1.2 The determinant and the elementary operator

Section 14.1.1 has it that interchanging, scaling or adding rows or columns
of a matrix respectively negates, scales or does not alter the matrix’s deter-
minant. But the three operations named are precisely the operations of the
three elementaries of § 11.4. Therefore,
det T[i↔j]A = − det A = det AT[i↔j],

det Tα[i] A = α det A = det ATα[j] ,
(14.8)
det Tα[ij] A = det A = det ATα[ij] ,
1 ≤ (i, j) ≤ n, i 6= j,
for any n × n square matrix A. Obviously also,
det IA = det A = det AI,

det In A = det A = det AIn , (14.9)
det I = 1 = det In .
If A is taken to represent an arbitrary product of identity matrices (In

and/or I) and elementary operators, then a significant consequence of (14.8)
and (14.9), applied recursively, is that the determinant of a product is the
product of the determinants, at least where identity matrices and elementary
operators are concerned. In symbols,5
!
Y Y
det Mk = det Mk , (14.10)
k k

Mk ∈ In , I, T[i↔j] , Tα[i] , Tα[ij] ,
1 ≤ (i, j) ≤ n.
This matters because, as the Gauss-Jordan decomposition of § 12.3 has

shown, one can build up any square matrix of full rank by applying ele-
mentary operators to In . Section 14.1.4 will put the rule (14.10) to good
use.
14.1.3 The determinant of a singular matrix

Equation (14.8) gives elementary operators the power to alter a matrix’s
determinant almost arbitrarily—almost arbitrarily, but not quite. What an
n × n elementary operator6 cannot do is to change an n × n matrix’s deter-
minant to or from zero. Once zero, a determinant remains zero under the
action of elementary operators. Once nonzero, always nonzero. Elementary
operators being reversible have no power to breach this barrier.
Another thing n × n elementaries cannot do according to § 12.5.3 is to
change an n × n matrix’s rank. Nevertheless, such elementaries can reduce
any n × n matrix reversibly to Ir , where r ≤ n is the matrix’s rank, by
the Gauss-Jordan algorithm of § 12.3. Equation (14.4) has that the n × n
determinant of Ir is zero if r < n, so it follows that the n × n determinant
of every rank-r matrix is similarly zero if r < n; and complementarily that
the n × n determinant of a rank-n matrix is never zero. Singular matrices
always have zero determinants; full-rank square matrices never do. One
can evidently tell the singularity or invertibility of a square matrix from its
determinant alone.
5
Notation like “∈”, first met in § 2.3, can be too fancy for applied mathematics, but it
does help here. The notation Mk ∈ {. . .} restricts Mk to be any of the things between the
braces. As it happens though, in this case, (14.11) below is going to erase the restriction.
6
That is, an elementary operator which honors an n × n active region. See § 11.3.2.
14.1.4 The determinant of a matrix product

Sections 14.1.2 and 14.1.3 suggest the useful rule that
det AB = det A det B. (14.11)
To prove the rule, we consider three distinct cases.

The first case is that A is singular. In this case, B acts as a column
operator on A, whereas according to § 12.5.2 no operator has the power to
promote A in rank. Hence the product AB is no higher in rank than A,
which says that AB is no less singular than A, which implies that AB like A
has a null determinant. Evidently (14.11) holds in the first case.
The second case is that B is singular. The proof here resembles that of
the first case.
The third case is that neither matrix is singular. Here, we use Gauss-
Jordan to decompose both matrices into sequences of elementary operators
and rank-n identity matrices, for which
det AB = det {[A] [B]}

nhY a i hY a io
= det T In T T In T
Y a Y a
= det T det In det T det T det In det T
hY a i hY a i
= det T In T det T In T
= det A det B,
which is a schematic way of pointing out in light of (14.10) merely that

since A and B are products of identity matrices and elementaries, the de-
terminant of the product is the product of the determinants.
So it is that (14.11) holds in all three cases, as was to be demonstrated.
The determinant of a matrix product is the product of the matrix determi-
nants.
14.1.5 Determinants of inverse and unitary matrices

From (14.11) it follows that
1
det A−1 = (14.12)
det A
because A−1 A = In and det In = 1.

From (14.6) it follows that if Q is a unitary matrix (§ 13.12), then
det Q∗ det Q = 1,
(14.13)
|det Q| = 1.
This reason is that |det Q|2 = (det Q)∗ (det Q) = det Q∗ det Q = det Q∗ Q =
det Q−1 Q = det In = 1.
14.1.6 Inverting the square matrix by determinant

The Gauss-Jordan algorithm comfortably inverts concrete matrices of mod-
erate size, but swamps one in nearly interminable algebra when symbolically
inverting general matrices larger than the A2 at the section’s head. Slogging
through the algebra to invert A3 symbolically nevertheless (the reader need
not actually do this unless he desires a long exercise), one quite incidentally
discovers a clever way to factor the determinant:
C T A = (det A)In = AC T ;
cij ≡ det Rij ;

1
 if i′ = i and j ′ = j, (14.14)
[Rij ]i′ j ′ ≡ 0 if i′ = i or j ′ = j but not both,

ai′ j ′ otherwise.

Pictorially, 2 3
.. .. .. .. ..
6 . . . . . 7
··· ∗ ∗ 0 ∗ ∗ ···
6 7
6 7
··· ∗ ∗ 0 ∗ ∗ ···
6 7
6 7
Rij = 6
6 ··· 0 0 1 0 0 ··· 7,
7
··· ∗ ∗ 0 ∗ ∗ ···
6 7
6 7
··· ∗ ∗ 0 ∗ ∗ ···
6 7
6 7
4 .. .. .. .. .. 5
. . . . .
same as A except in the ith row and jth column. The matrix C, called the
cofactor of A, then consists of the determinants of the various Rij .
Another way to write (14.14) is
[C T A]ij = (det A)δij = [AC T ]ij , (14.15)
which comprises two cases. In the case that i = j,

X X
[AC T ]ij = [AC T ]ii = aiℓ ciℓ = aiℓ det Riℓ = det A = (det A)δij ,
ℓ ℓ
P
wherein the equation ℓ aiℓ det Riℓ = det A states that det A, being a deter-
minant, consists of several terms, each term including one factor from each
row of A, where aiℓ provides the ith row and Riℓ provides the other rows.7
In the case that i 6= j,
X X
[AC T ]ij = aiℓ cjℓ = aiℓ det Rjℓ = 0 = (det A)(0) = (det A)δij ,
ℓ ℓ
P
wherein ℓ aiℓ det Rjℓ is the determinant, not of A itself, but rather of A
with the jth row replaced by a copy of the ith, which according to (14.5)
evaluates to zero. Similar equations can be written for [C T A]ij in both cases.
The two cases together prove (14.15), hence also (14.14).
Dividing (14.14) by det A, we have that8
A−1 A = In = AA−1 ,
CT
A−1 = . (14.16)
det A
Equation (14.16) inverts a matrix by determinant. In practice, it inverts
small matrices nicely, through about 4 × 4 dimensionality (the A−1 2 equation
at the head of the section is just eqn. 14.16 for n = 2). It inverts 5 × 5 and
even 6 × 6 matrices reasonably, too—especially with the help of a computer
to do the arithmetic. Though (14.16) still holds in theory for yet larger
matrices, and though symbolically it expresses the inverse of an abstract,
n × n matrix concisely whose entries remain unspecified, for concrete matri-
ces much bigger than 4 × 4 to 6 × 6 or so its several determinants begin to
grow too great and too many for practical calculation. The Gauss-Jordan
technique (or even the Gram-Schmidt technique) is preferred to invert con-
crete matrices above a certain size for this reason.9
14.2 Coincident properties

Chs. 11, 12 and 13, plus this chapter up to the present point, have discovered
several coincident properties of the invertible n × n square matrix. One does
7
This is a bit subtle, but if you actually write out A3 and its cofactor C3 symbolically,
trying (14.15) on them, then you will soon see what is meant.
8
Cramer’s rule [21, § 1.6], of which the reader may have heard, results from apply-
ing (14.16) to (13.4). However, Cramer’s rule is really nothing more than (14.16) in a less
pleasing form, so this book does not treat Cramer’s rule as such.
9
For very large matrices, even the Gauss-Jordan grows impractical, due to compound
floating-point rounding error and the maybe large but nonetheless limited quantity of
available computer memory. Iterative techniques ([chapter not yet written]) serve to invert
such matrices approximately.
14.2. COINCIDENT PROPERTIES 375
not feel the full impact of the coincidence when these properties are left
scattered across the long chapters; so, let us gather and summarize the
properties here. A square, n × n matrix evidently has either all of the
following properties or none of them, never some but not others.
• The matrix is invertible (§ 13.1).
• Its rows are linearly independent (§§ 12.1 and 12.3.4).
• Its columns are linearly independent (§ 12.5.4).
• Its columns address the same space the columns of In address, and its
rows address the same space the rows of In address (§ 12.5.7).
• The Gauss-Jordan algorithm reduces it to In (§ 12.3.3). (In this, per

§ 12.5.3, the choice of pivots does not matter.)
• Decomposing it, the Gram-Schmidt algorithm achieves a fully square,

unitary, n × n factor Q (§ 13.11.2).
• It has full rank r = n (§ 12.5.4).
• The linear system Ax = b it represents has a unique n-element solu-

tion x, given any specific n-element driving vector b (§ 13.2).
• The determinant det A 6= 0 (§ 14.1.3).
• None of its eigenvalues is zero (§ 14.3, below).
The square matrix which has one of these properties, has all of them. The
square matrix which lacks one, lacks all. Assuming exact arithmetic, a
square matrix is either invertible, with all that that implies, or singular;
never both. The distinction between invertible and singular matrices is
theoretically as absolute as (and indeed is analogous to) the distinction
between nonzero and zero scalars.
Whether the distinction is always useful is another matter. Usually the
distinction is indeed useful, but a matrix can be almost singular just as a
scalar can be almost zero. Such a matrix is known, among other ways, by
its unexpectedly small determinant. Now it is true: in exact arithmetic, a
nonzero determinant, no matter how small, implies a theoretically invert-
ible matrix. Practical matrices however often have entries whose values are
imprecisely known; and even when they don’t, the computers which invert
them tend to do arithmetic imprecisely in floating-point. Matrices which live
on the hazy frontier between invertibility and singularity resemble the in-
finitesimals of § 4.1.1. They are called ill conditioned matrices. Section 14.8
develops the topic.
14.3 The eigenvalue itself

We stand ready at last to approach the final major agent of matrix arith-
metic, the eigenvalue. Suppose a square, n×n matrix A, a nonzero n-element
vector
v = In v 6= 0, (14.17)
and a scalar λ, together such that
Av = λv, (14.18)
or in other words such that Av = λIn v. If so, then
[A − λIn ]v = 0. (14.19)
Since In v is nonzero, the last equation is true if and only if the matrix
[A − λIn ] is singular—which in light of § 14.1.3 is to demand that
det(A − λIn ) = 0. (14.20)
The left side of (14.20) is an nth-order polynomial in λ, the characteristic

polynomial, whose n roots are the eigenvalues 10 of the matrix A.
What is an eigenvalue, really? An eigenvalue is a scalar a matrix resem-
bles under certain conditions. When a matrix happens to operate on the
right eigenvector v, it is all the same whether one applies the entire matrix
or just the eigenvalue to the vector. The matrix scales the eigenvector by
the eigenvalue without otherwise altering the vector, changing the vector’s
10
An example:
» –
2 0
A = ,
3 −1
» –
2−λ 0
det(A − λIn ) = det
3 −1 − λ
= (2 − λ)(−1 − λ) − (0)(3)
= λ2 − λ − 2 = 0,
λ = −1 or 2.
14.4. THE EIGENVECTOR 377
magnitude but not its direction. The eigenvalue alone takes the place of the
whole, hulking matrix. This is what (14.18) means. Of course it works only
when v happens to be the right eigenvector, which § 14.4 discusses.
When λ = 0, (14.20) makes det A = 0, which as we have said is the
sign of a singular matrix. Zero eigenvalues and singular matrices always
travel together. Singular matrices each have at least one zero eigenvalue;
nonsingular matrices never do.
The eigenvalues of a matrix’s inverse are the inverses of the matrix’s
eigenvalues. That is,
λ′j λj = 1 for all 1 ≤ j ≤ n if A′ A = In = AA′ . (14.21)
The reason behind (14.21) comes from answering the question: if Avj
scales vj by the factor λj , then what does A′ Avj = Ivj do to vj ?
Naturally one must solve (14.20)’s nth-order polynomial to locate the
actual eigenvalues. One solves it by the same techniques by which one solves
any polynomial: the quadratic formula (2.2); the cubic and quartic methods
of Ch. 10; the Newton-Raphson iteration (4.31). On the other hand, the
determinant (14.20) can be impractical to expand for a large matrix; here
iterative techniques help: see [chapter not yet written].11
14.4 The eigenvector

It is an odd fact that (14.19) and (14.20) reveal the eigenvalues λ of a
square matrix A while obscuring the associated eigenvectors v. Once one
has calculated an eigenvalue, though, one can feed it back to calculate the
associated eigenvector. According to (14.19), the eigenvectors are the n-
element vectors for which
[A − λIn ]v = 0,
which is to say that the eigenvectors are the vectors of the kernel space of the
degenerate matrix [A − λIn ]—which one can calculate (among other ways)
by the Gauss-Jordan kernel formula (13.7) or the Gram-Schmidt kernel for-
mula (13.61).
An eigenvalue and its associated eigenvector, taken together, are some-
times called an eigensolution.
11
The inexpensive [21] also covers the topic competently and readably.
14.5 Eigensolution facts

Many useful or interesting mathematical facts concern the eigensolution,
among them the following.
• If the eigensolutions of A are (λj , vj ), then the eigensolutions of A +

αIn are (λj + α, vj ). The eigenvalues move over by αIn while the
eigenvectors remain fixed. This is seen by adding αvj to both sides of
the definition Avj = λvj .
• A matrix and its inverse share the same eigenvectors with inverted
eigenvalues. Refer to (14.21) and its explanation in § 14.3.
• Eigenvectors corresponding to distinct eigenvalues are always linearly

independent of one another. To prove this fact, consider several inde-
pendent eigenvectors v1 , v2 , . . . , vk−1 respectively with distinct eigen-
values λ1 , λ2 , . . . , λk−1 , and further consider another eigenvector vk
which might or might not be independent but which too has a distinct
eigenvalue λk . Were vk dependent, which is to say, did nontrivial
coefficients cj exist such that
k−1
X
vk = cj vj ,
j=1
then left-multiplying the equation by A − λk would yield

k−1
X
0= (λj − λk )cj vj ,
j=1
impossible since the k − 1 eigenvectors are independent. Thus vk too

is independent, whereupon by induction from a start case of k = 1
we conclude that there exists no dependent eigenvector with a distinct
eigenvalue.
• If an n × n square matrix A has n independent eigenvectors (which is

always so if the matrix has n distinct eigenvalues and often so even
otherwise), then any n-element vector can be expressed as a unique
linear combination of the eigenvectors. This is a simple consequence
of the fact that the n × n matrix V whose columns are the several
eigenvectors vj has full rank r = n. Unfortunately some matrices with
repeated eigenvalues also have repeated eigenvectors—as for example,
14.6. DIAGONALIZATION 379
curiously,12 [1 0; 1 1]T , whose double eigenvalue λ = 1 has the single

eigenvector [1 0]T . Section 14.10.2 speaks of matrices of the last kind.
• An n × n square matrix whose eigenvectors are linearly independent of

one another cannot share all eigensolutions with any other n×n square
matrix. This fact proceeds from the last point, that every n-element
vector x is a unique linear combination of independent eigenvectors.
Neither of the two proposed matrices A1 and A2 could scale any of the
eigenvector components of x differently than the other matrix did, so
A1 x − A2 x = (A1 − A2 )x = 0 for all x, which in turn is possible only
if A1 = A2 .
• A positive definite matrix has only real, positive eigenvalues. A non-

negative definite matrix has only real, nonnegative eigenvalues. Were
it not so, then v∗ Av = λv∗ v (in which v∗ v naturally is a positive real
scalar) would violate the criterion for positive or nonnegative definite-
ness. See § 13.6.3.
• Every n × n square matrix has at least one eigensolution if n > 0,

because according to the fundamental theorem of algebra (6.1) the
matrix’s characteristic polynomial (14.20) has at least one root, an
eigenvalue, which by definition would be no eigenvalue if it had no
eigenvector to scale, and for which (14.19) necessarily admits at least
one nonzero solution v because its matrix A − λIn is degenerate.
14.6 Diagonalization
Any n × n matrix with n independent eigenvectors (which class per § 14.5
includes, but is not limited to, every n×n matrix with n distinct eigenvalues)
can be diagonalized as
A = V ΛV −1 , (14.22)
where
λ1 0 ··· 0 0
2 3
6 0 λ2 ··· 0 0 7
.. .. .. ..
6 7
Λ= 6 .. 7
6
6 . . . . . 7
7
4 0 0 ··· λn−1 0 5
0 0 ··· 0 λn
12
[29]
is an otherwise empty n × n matrix with the eigenvalues of A set along its

main diagonal and

V = v1 v2 · · · vn−1 vn
is an n × n matrix whose columns are the eigenvectors of A. This is so

because the identity Avj = vj λj holds for all 1 ≤ j ≤ n; or, expressed more
concisely, because the identity
AV = V Λ
holds.13 The matrix V is invertible because its columns the eigenvectors

are independent, from which (14.22) follows. Equation (14.22) is called the
eigenvalue decomposition, the diagonal decomposition or the diagonalization
of the square matrix A.
One might object that we had shown only how to compose some matrix
V ΛV −1 with the correct eigenvalues and independent eigenvectors, but had
failed to show that the matrix was actually A. However, we need not show
this, because § 14.5 has already demonstrated that two matrices with the
same eigenvalues and independent eigenvectors are in fact the same matrix,
whereby the product V ΛV −1 can be nothing other than A.
An n × n matrix with n independent eigenvectors (which class, again,
includes every n × n matrix with n distinct eigenvalues and also includes
many matrices with fewer) is called a diagonalizable matrix. Besides factor-
ing a diagonalizable matrix by (14.22), one can apply the same formula to
compose a diagonalizable matrix with desired eigensolutions.
The diagonal matrix diag{x} of (11.55) is trivially diagonalizable as
diag{x} = In diag{x}In .
It is a curious and useful fact that
A2 = (V ΛV −1 )(V ΛV −1 ) = V Λ2 V −1
and by extension that

Ak = V Λk V −1 (14.23)
for any diagonalizable matrix A. The diagonal matrix Λk is nothing more
than the diagonal matrix Λ with each element individually raised to the kth
power, such that h i
Λk = δij λkj .
ij
13
If this seems confusing, then consider that the jth column of the product AV is Avj ,
whereas the jth column of Λ having just the one element acts to scale V ’s jth column
only.
14.7. REMARKS ON THE EIGENVALUE 381
Changing z ← k implies the generalization14
Az = V Λz V −1 ,
h i (14.24)
Λz = δij λzj ,
ij
good for any diagonalizable A and complex z.

Nondiagonalizable matrices are troublesome and interesting. The non-
diagonalizable matrix vaguely resembles the singular matrix in that both
represent edge cases and can be hard to handle numerically; but the resem-
blance ends there, and a matrix can be either without being the other. The
n × n null matrix for example is singular but still diagonalizable. What a
nondiagonalizable matrix is in essence is a matrix with a repeated eigenso-
lution: the same eigenvalue with the same eigenvector, twice or more. More
formally, a nondiagonalizable matrix is a matrix with an n-fold eigenvalue
whose corresponding eigenvector space fewer than n eigenvectors fully char-
acterize. Section 14.10.2 will have more to say about the nondiagonalizable
matrix.
14.7 Remarks on the eigenvalue

Eigenvalues and their associated eigenvectors stand among the principal
reasons one goes to the considerable trouble to develop matrix theory as we
have done in recent chapters. The idea that a matrix resembles a humble
scalar in the right circumstance is powerful. Among the reasons for this
is that a matrix can represent an iterative process, operating repeatedly
on a vector v to change it first to Av, then to A2 v, A3 v and so on. The
dominant eigenvalue of A, largest in magnitude, tends then to transform v
into the associated eigenvector, gradually but relatively eliminating all other
components of v. Should the dominant eigenvalue have greater than unit
magnitude, it destabilizes the iteration; thus one can sometimes judge the
stability of a physical process indirectly by examining the eigenvalues of the
matrix which describes it. Then there is the edge case of the nondiagonaliz-
able matrix, which matrix surprisingly covers only part of its domain with
eigenvectors. All this is fairly deep mathematics. It brings an appreciation
of the matrix for reasons which were anything but apparent from the outset
of Ch. 11.
Remarks continue in §§ 14.10.2 and 14.13.
14
It may not be clear however according to (5.12) which branch of λzj one should choose
at each index j, especially if A has negative or complex eigenvalues.
14.8 Matrix condition

The largest in magnitude of the several eigenvalues of a diagonalizable opera-
tor A, denoted here λmax , tends to dominate the iteration Ak x. Section 14.7
has named λmax the dominant eigenvalue for this reason.
One sometimes finds it convenient to normalize a dominant eigenvalue
by defining a new operator A′ ≡ A/ |λmax |, whose own dominant eigenvalue
λmax / |λmax | has unit magnitude. In terms of the new operator, the iteration
becomes Ak x = |λmax |k A′k x, leaving one free to carry the magnifying effect
|λmax |k separately if one prefers to do so. However, the scale factor 1/ |λmax |
scales all eigenvalues equally; thus, if A’s eigenvalue of smallest magnitude
is denoted λmin , then the corresponding eigenvalue of A′ is λmin / |λmax |. If
zero, then both matrices according to § 14.3 are singular; if nearly zero, then
both matrices are ill conditioned.
Such considerations lead us to define the condition of a diagonalizable
matrix quantitatively as15
λmax
κ≡ , (14.25)
λmin
by which
κ≥1 (14.26)
is always a real number of no less than unit magnitude. For best invertibility,
κ = 1 would be ideal (it would mean that all eigenvalues had the same
magnitude), though in practice quite a broad range of κ is usually acceptable.
Could we always work in exact arithmetic, the value of κ might not interest
us much as long as it stayed finite; but in computer floating point, or where
the elements of A are known only within some tolerance, infinite κ tends
to emerge imprecisely rather as large κ ≫ 1. An ill conditioned matrix by
definition16 is a matrix of large κ ≫ 1. The applied mathematician handles
such a matrix with due skepticism.
Matrix condition so defined turns out to have another useful application.
Suppose that a diagonalizable matrix A is precisely known but that the
corresponding driving vector b is not. If
A(x + δx) = b + δb,
15
[62]
16
There is of course no definite boundary, no particular edge value of κ, less than which
a matrix is well conditioned, at and beyond which it turns ill conditioned; but you knew
that already. If I tried to claim that a matrix with a fine κ = 3 were ill conditioned, for
instance, or that one with a wretched κ = 20x18 were well conditioned, then you might
not credit me—but the mathematics nevertheless can only give the number; it remains to
the mathematician to interpret it.
14.9. THE SIMILARITY TRANSFORMATION 383
where δb is the error in b and δx is the resultant error in x, then one should
like to bound the ratio |δx| / |x| to ascertain the reliability of x as a solution.
Transferring A to the equation’s right side,
x + δx = A−1 (b + δb).
Subtracting x = A−1 b and taking the magnitude,
|δx| = A−1 δb .

Dividing by |x| = A−1 b,

−1
|δx| A δb
= .
|x| |A−1 b|
The quantity A−1 δb cannot exceed λ−1 δb. The quantity A−1 b cannot

min
fall short of λ−1

max b . Thus,

−1
|δx| λ δb λmax |δb|
min
≤ −1 = .
|x| λmax b λmin |b|
That is,
|δx| |δb|
≤κ . (14.27)
|x| |b|
Condition, incidentally, might technically be said to apply to scalars as
well as to matrices, but ill condition remains a property of matrices alone.
According to (14.25), the condition of every nonzero scalar is happily κ = 1.
14.9 The similarity transformation

Any collection of vectors assembled into a matrix can serve as a basis by
which other vectors can be expressed. For example, if the columns of
2 3
1 −1
B= 4 0 2 5
0 1
are regarded as a basis, then the vector

2 3 2 3 2 3
» – 1 −1 4
5
B 1
= 54 0 5 + 14 2 5 = 4 2 5
0 1 1
is (5, 1) in the basis B: five times the first basis vector plus once the second.
The basis provides the units from which other vectors can be built.
Particularly interesting is the n×n, invertible complete basis B, in which
the n basis vectors are independent and address the same full space the
columns of In address. If
x = Bu
then u represents x in the basis B. Left-multiplication by B evidently
converts out of the basis. Left-multiplication by B −1 ,
u = B −1 x,
then does the reverse, converting into the basis. One can therefore convert
any operator A to work within a complete basis B by the successive steps
Ax = b,
ABu = b,
−1
[B AB]u = B −1 b,
by which the operator B −1 AB is seen to be the operator A, only transformed
to work within the basis17,18 B.
The conversion from A into B −1 AB is called a similarity transformation.
If B happens to be unitary (§ 13.12), then the conversion is also called a
unitary transformation. The matrix B −1 AB the transformation produces
is said to be similar (or, if B is unitary, unitarily similar ) to the matrix A.
We have already met the similarity transformation in §§ 11.5 and 12.2. Now
we have the theory to appreciate it properly.
Probably the most important property of the similarity transformation
is that it alters no eigenvalues. That is, if
Ax = λx,
then, by successive steps,
B −1 A(BB −1 )x = λB −1 x,
[B −1 AB]u = λu. (14.28)
17
The reader may need to ponder the basis concept a while to grasp it, but the concept
is simple once grasped and little purpose would be served by dwelling on it here. Basically,
the idea is that one can build the same vector from alternate building blocks, not only
from the standard building blocks e1 , e2 , e3 , etc.—except that the right word for the
relevant “building block” is basis vector. The books [30] and [42] introduce the basis more
gently; one might consult one of those if needed.
18
The professional matrix literature sometimes distinguishes by typeface between the
matrix B and the basis B its columns represent. Such semantical distinctions seem a little
too fine for applied use, though. This book just uses B.
14.10. THE SCHUR DECOMPOSITION 385
The eigenvalues of A and the similar B −1 AB are the same for any square,
n × n matrix A and any invertible, square, n × n matrix B.
14.10 The Schur decomposition

The Schur decomposition of an arbitrary, n × n square matrix A is
A = QUS Q∗ , (14.29)
where Q is an n × n unitary matrix whose inverse, as for any unitary matrix

(§ 13.12), is Q−1 = Q∗ ; and where US is a general upper triangular matrix
which can have any values (even zeros) along its main diagonal. The Schur
decomposition is slightly obscure, is somewhat tedious to derive and is of
limited use in itself, but serves a theoretical purpose.19 We derive it here
for this reason.
14.10.1 Derivation
Suppose that20 (for some reason, which will shortly grow clear) we have a
matrix B of the form
2 3
.. .. .. .. .. .. .. ..
6 . . . . . . . . 7
6
6 ··· ∗ ∗ ∗ ∗ ∗ ∗ ∗ ··· 7
7
··· 0 ∗ ∗ ∗ ∗ ∗ ∗ ···
6 7
6 7
··· 0 0 ∗ ∗ ∗ ∗ ∗ ···
6 7
6 7
B= 6
6 ··· 0 0 0 ∗ ∗ ∗ ∗ ··· 7,
7
(14.30)
··· 0 0 0 0 ∗ ∗ ∗ ···
6 7
6 7
··· 0 0 0 0 ∗ ∗ ∗ ···
6 7
6 7
··· 0 0 0 0 ∗ ∗ ∗ ···
6 7
6 7
.. .. .. .. .. .. .. ..
4 5
. . . . . . . .
19
The alternative is to develop the interesting but difficult Jordan canonical form, which
for brevity’s sake this chapter prefers to omit.
20
This subsection assigns various capital Roman letters to represent the several matrices
and submatrices it manipulates. Its choice of letters except in (14.29) is not standard and
carries no meaning elsewhere. The writer had to choose some letters and these are ones
he chose.
This footnote mentions the fact because good mathematical style avoid assigning letters
that already bear a conventional meaning in a related context (for example, this book
avoids writing Ax = b as T e = i, not because the latter is wrong but because it would be
extremely confusing). The Roman alphabet provides only twenty-six capitals, though, of
which this subsection uses too many to be allowed to reserve any. See Appendix B.
where the ith row and ith column are depicted at center. Suppose further
that we wish to transform B not only similarly but unitarily into
2 3
.. .. .. .. .. .. .. ..
6 . . . . . . . . 7
6
6 ··· ∗ ∗ ∗ ∗ ∗ ∗ ∗ ··· 7
7
6
6 ··· 0 ∗ ∗ ∗ ∗ ∗ ∗ ··· 7
7
··· 0 0 ∗ ∗ ∗ ∗ ∗ ···
6 7
6 7
C ≡ W ∗ BW = 6
6 ··· 0 0 0 ∗ ∗ ∗ ∗ ··· 7,
7
(14.31)
··· 0 0 0 0 ∗ ∗ ∗ ···
6 7
6 7
··· 0 0 0 0 0 ∗ ∗ ···
6 7
6 7
··· 0 0 0 0 0 ∗ ∗ ···
6 7
6 7
.. .. .. .. .. .. .. ..
4 5
. . . . . . . .
where W is an n × n unitary matrix, and where we do not mind if any or all

of the ∗ elements change in going from B to C but we require zeros in the
indicated spots. Let Bo and Co represent the (n − i) × (n − i) submatrices
in the lower right corners respectively of B and C, such that
Bo ≡ In−i H−i BHi In−i ,
(14.32)
Co ≡ In−i H−i CHi In−i ,
where Hk is the shift operator of § 11.9. Pictorially,
∗ ∗ ∗ ··· ∗ ∗ ∗ ···
2 3 2 3
6 ∗ ∗ ∗ ··· 7 6 0 ∗ ∗ ··· 7
Bo = 6
6 ∗ ∗ ∗ ··· 7,
7
Co = 6
6 0 ∗ ∗ ··· 7.
7
.. .. .. .. .. ..
4 5 4 5
.. ..
. . . . . . . .
Equation (14.31) seeks an n × n unitary matrix W to transform the ma-

trix B into a new matrix C ≡ W ∗ BW such that C fits the form (14.31)
stipulates. The question remains as to whether a unitary W exists that sat-
isfies the form and whether for general B we can discover a way to calculate
it. To narrow the search, because we need not find every W that satisfies the
form but only one such W , let us look first for a W that fits the restricted
template
2 3
.. .. .. .. .. .. .. ..
6 . . . . . . . . 7
6
6 ··· 1 0 0 0 0 0 0 ··· 7
7
··· 0 1 0 0 0 0 0 ···
6 7
6 7
··· 0 0 1 0 0 0 0 ···
6 7
6 7
W = Ii + Hi Wo H−i = 6
6 ··· 0 0 0 1 0 0 0 ··· 7,
7
(14.33)
··· 0 0 0 0 ∗ ∗ ∗ ···
6 7
6 7
··· 0 0 0 0 ∗ ∗ ∗ ···
6 7
6 7
··· 0 0 0 0 ∗ ∗ ∗ ···
6 7
6 7
.. .. .. .. .. .. .. ..
4 5
. . . . . . . .
which contains a smaller, (n − i) × (n − i) unitary submatrix Wo in its lower

right corner and resembles In elsewhere. Beginning from (14.31), we have
by successive, reversible steps that
C = W ∗ BW
= (Ii + Hi Wo∗ H−i )(B)(Ii + Hi Wo H−i )
= Ii BIi + Ii BHi Wo H−i + Hi Wo∗ H−i BIi
+ Hi Wo∗ H−i BHi Wo H−i .
The unitary submatrix Wo has only n−i columns and n−i rows, so In−i Wo =
Wo = Wo In−i . Thus,
C = Ii BIi + Ii BHi Wo In−i H−i + Hi In−i Wo∗ H−i BIi

+ Hi In−i Wo∗ In−i H−i BHi In−i Wo In−i H−i
= Ii [B]Ii + Ii [BHi Wo H−i ](In − Ii ) + (In − Ii )[Hi Wo∗ H−i B]Ii
+ (In − Ii )[Hi Wo∗ Bo Wo H−i ](In − Ii ),
where the last step has used (14.32) and the identity (11.76). The four terms
on the equation’s right, each term with rows and columns neatly truncated,
represent the four quarters of C ≡ W ∗ BW —upper left, upper right, lower
left and lower right, respectively. The lower left term is null because
(In − Ii )[Hi Wo∗ H−i B]Ii = (In − Ii )[Hi Wo∗ In−i H−i BIi ]Ii
= (In − Ii )[Hi Wo∗ H−i ][(In − Ii )BIi ]Ii
= (In − Ii )[Hi Wo∗ H−i ][0]Ii = 0,
leaving
C = Ii [B]Ii + Ii [BHi Wo H−i ](In − Ii )

+ (In − Ii )[Hi Wo∗ Bo Wo H−i ](In − Ii ).
But the upper left term makes the upper left areas of B and C the same,
and the upper right term does not bother us because we have not restricted
the content of C’s upper right area. Apparently any (n − i) × (n − i) unitary
submatrix Wo whatsoever obeys (14.31) in the lower left, upper left and
upper right.
That leaves the lower right. Left- and right-multiplying (14.31) by the
truncator (In −Ii ) to focus solely on the lower right area, we have the reduced
requirement that
(In − Ii )C(In − Ii ) = (In − Ii )W ∗ BW (In − Ii ). (14.34)

Further left-multiplying by H−i , right-multiplying by Hi , and applying the

identity (11.76) yields
In−i H−i CHi In−i = In−i H−i W ∗ BW HiIn−i ;
or, substituting from (14.32),
Co = In−i H−i W ∗ BW HiIn−i .
Expanding W per (14.33),
Co = In−i H−i (Ii + Hi Wo∗ H−i )B(Ii + Hi Wo H−i )Hi In−i ;
or, since In−i H−i Ii = 0 = Ii Hi In−i ,
Co = In−i H−i (Hi Wo∗ H−i )B(Hi Wo H−i )Hi In−i

= In−i Wo∗ H−i BHi Wo In−i
= Wo∗ In−i H−i BHi In−i Wo .
Per (14.32), this is

Co = Wo∗ Bo Wo . (14.35)
The steps from (14.34) to (14.35) are reversible, so the latter is as good a
way to state the reduced requirement as the former is. To achieve a unitary
transformation of the form (14.31), therefore, it suffices to satisfy (14.35).
The increasingly well-stocked armory of matrix theory we now have to
draw from makes satisfying (14.35) possible as follows. Observe per § 14.5
that every square matrix has at least one eigensolution. Let (λo , vo ) repre-
sent an eigensolution of Bo —any eigensolution of Bo —with vo normalized
to unit magnitude. Form the broad, (n − i) × (n − i + 1) matrix

F ≡ vo e1 e2 e3 · · · en−i .
Decompose F by the Gram-Schmidt technique of § 13.11.2, choosing p = 1

during the first instance of the algorithm’s step 3 (though choosing any
permissible p thereafter), to obtain
F = Q F RF .
Noting that the Gram-Schmidt algorithm orthogonalizes only rightward,

observe that the first column of the (n − i) × (n − i) unitary matrix QF
remains simply the first column of F , which is the unit eigenvector vo :
[QF ]∗1 = QF e1 = vo .
Transform Bo unitarily by QF to define the new matrix
G ≡ Q∗F Bo QF ,
then transfer factors to reach the equation
QF GQ∗F = Bo .
Right-multiplying by QF e1 = vo and noting that Bo vo = λo vo , observe that
QF Ge1 = λo vo .
Left-multiplying by Q∗F ,
Ge1 = λo Q∗F vo .
Noting that the Gram-Schmidt process has rendered orthogonal to vo all
columns of QF but the first, which is vo , observe that
λo
2 3
6 0 7
Ge1 = λo Q∗F vo = λo e1 = 6
6 0 7,
7
..
4 5
.
which means that

λo ∗ ∗ ···
2 3
6 0 ∗ ∗ ··· 7
G=6
6 0 ∗ ∗ ··· 7,
7
.. .. ..
4 5
..
. . . .
which fits the very form (14.32) the submatrix Co is required to have. Con-
clude therefore that
W o = QF ,
(14.36)
Co = G,
where QF and G are as this paragraph develops, together constitute a valid

choice for Wo and Co , satisfying the reduced requirement (14.35) and thus
also the original requirement (14.31).
Equation (14.36) completes a failsafe technique to transform unitarily
any square matrix B of the form (14.30) into a square matrix C of the
form (14.31). Naturally the technique can be applied recursively as
B|i=i′ = C|i=i′ −1 , 1 ≤ i′ ≤ n, (14.37)

because the form (14.30) of B at i = i′ is nothing other than the form (14.31)
of C at i = i′ − 1. Therefore, if we let
B|i=0 = A, (14.38)
then it follows by induction that
B|i=n = US , (14.39)
where per (14.30) the matrix US has the general upper triangular form the
Schur decomposition (14.29) requires. Moreover, because the product of
unitary matrices according to (13.64) is itself a unitary matrix, we have
that
n−1
a
Q= (W |i=i′ ) , (14.40)
i′ =0
which along with (14.39) accomplishes the Schur decomposition.
14.10.2 The nondiagonalizable matrix

The characteristic equation (14.20) of the general upper triangular ma-
trix US is
det(US − λIn ) = 0.
Unlike most determinants, this determinant brings only the one term
n
Y
det(US − λIn ) = (uSii − λ) = 0
i=1
whose factors run straight down the main diagonal, where the determinant’s
n! − 1 other terms are all zero because each of them includes at least one
zero factor from below the main diagonal.21 Hence no element above the
main diagonal of US even influences the eigenvalues, which apparently are
λi = uSii , (14.41)
21
The determinant’s definition in § 14.1 makes the following two propositions equivalent:
(i) that a determinant’s term which includes one or more factors above the main diagonal
also includes one or more factors below; (ii) that the only permutor that marks no position
below the main diagonal is the one which also marks no position above. In either form,
the proposition’s truth might seem less than obvious until viewed from the proper angle.
Consider a permutor P . If P marked no position below the main diagonal, then it would
necessarily have pnn = 1, else the permutor’s bottom row would be empty which is not
allowed. In the next-to-bottom row, p(n−1)(n−1) = 1, because the nth column is already
occupied. In the next row up, p(n−2)(n−2) = 1; and so on, thus affirming the proposition.
the main-diagonal elements.

According to (14.28), similarity transformations preserve eigenvalues.
The Schur decomposition (14.29) is in fact a similarity transformation; and,
as we have seen, every matrix A has a Schur decomposition. If therefore
A = QUS Q∗ ,
then the eigenvalues of A are just the values along the main diagonal of US .22
One might think that the Schur decomposition offered an easy way to cal-
culate eigenvalues, but it is less easy than it first appears because one must
calculate eigenvalues to reach the Schur decomposition in the first place.
Whatever practical merit the Schur decomposition might have or lack, how-
ever, it brings at least the theoretical benefit of (14.41): every square matrix
without exception has a Schur decomposition, whose triangular factor US
openly lists all eigenvalues along its main diagonal.
This theoretical benefit pays when some of the n eigenvalues of an n × n
square matrix A repeat. By the Schur decomposition, one can construct
a second square matrix A′ , as near as desired to A but having n distinct
eigenvalues, simply by perturbing the main diagonal of US to23
US′ ≡ US + ǫ diag{u}, (14.42)

ui′ 6= ui if λi′ = λi ,
where |ǫ| ≪ 1 and where u is an arbitrary vector that meets the criterion
given. Though infinitesimally near A, the modified matrix A′ = QUS′ Q∗
unlike A has n (maybe infinitesimally) distinct eigenvalues. With sufficient
toil, one can analyze such perturbed eigenvalues and their associated eigen-
vectors similarly as § 9.6.2 has analyzed perturbed poles.
22
An unusually careful reader might worry that A and US had the same eigenvalues
with different multiplicities. It would be surprising if it actually were so; but, still, one
would like to give a sounder reason than the participle “surprising.” Consider however
that
A − λIn = QUS Q∗ − λIn = Q[US − Q∗ (λIn )Q]Q∗

= Q[US − λ(Q∗ In Q)]Q∗ = Q[US − λIn ]Q∗ .
According to (14.11) and (14.13), this equation’s determinant is
det[A − λIn ] = det{Q[US − λIn ]Q∗ } = det Q det[US − λIn ] det Q∗ = det[US − λIn ],
which says that A and US have not only the same eigenvalues but also the same charac-
teristic polynomials, thus further the same eigenvalue multiplicities.
23
Equation (11.55) defines the diag{·} notation.
Equation (14.42) brings us to the nondiagonalizable matrix of the sub-

section’s title. Section 14.6 and its diagonalization formula (14.22) diago-
nalize any matrix with distinct eigenvalues and even any matrix with re-
peated eigenvalues but distinct eigenvectors, but fail where eigenvectors re-
peat. Equation (14.42) separates eigenvalues, thus also eigenvectors—for
according to § 14.5 eigenvectors of distinct eigenvalues never depend on one
another—permitting a nonunique but still sometimes usable form of diag-
onalization in the limit ǫ → 0 even when the matrix in question is strictly
nondiagonalizable.
The finding that every matrix is arbitrarily nearly diagonalizable illu-
minates a question the chapter has evaded up to the present point. The
question: does a p-fold root in the characteristic polynomial (14.20) neces-
sarily imply a p-fold eigenvalue in the corresponding matrix? The existence
of the nondiagonalizable matrix casts a shadow of doubt until one realizes
that every nondiagonalizable matrix is arbitrarily nearly diagonalizable—
and, better, is arbitrarily nearly diagonalizable with distinct eigenvalues. If
you claim that a matrix has a triple eigenvalue and someone disputes the
claim, then you can show him a nearly identical matrix with three infinites-
imally distinct eigenvalues. That is the essence of the idea. We shall leave
the answer in that form.
Generalizing the nondiagonalizability concept leads one eventually to the
ideas of the generalized eigenvector 24 (which solves the higher-order linear
system [A − λI]k v = 0) and the Jordan canonical form,25 which together
roughly track the sophisticated conventional pole-separation technique of
§ 9.6.5. Then there is a kind of sloppy Schur form called a Hessenberg form
which allows content in US along one or more subdiagonals just beneath
the main diagonal. One could profitably propose and prove any number of
useful theorems concerning the nondiagonalizable matrix and its generalized
eigenvectors, or concerning the eigenvalue problem26 more broadly, in more
and less rigorous ways, but for the time being we shall let the matter rest
there.
14.11 The Hermitian matrix

An m × m square matrix A that is its own adjoint,
A∗ = A, (14.43)
24
[23, Ch. 7]
25
[21, Ch. 5]
26
[69]
14.11. THE HERMITIAN MATRIX 393
is called a Hermitian or self-adjoint matrix. Properties of the Hermitian

matrix include that
• its eigenvalues are real,
• its eigenvectors corresponding to distinct eigenvalues lie orthogonal to

one another, and
• it is unitarily diagonalizable (§§ 13.12 and 14.6) such that
A = V ΛV ∗ . (14.44)
That the eigenvalues are real is proved by letting (λ, v) represent an

eigensolution of A and constructing the product v∗ Av, for which
λ∗ v∗ v = (Av)∗ v = v∗ Av = v∗ (Av) = λv∗ v.
That is,
λ∗ = λ,
which naturally is possible only if λ is real.
That eigenvectors corresponding to distinct eigenvalues lie orthogonal to
one another is proved27 by letting (λ1 , v1 ) and (λ2 , v2 ) represent eigensolu-
tions of A and constructing the product v2∗ Av1 , for which
λ∗2 v2∗ v1 = (Av2 )∗ v1 = v2∗ Av1 = v2∗ (Av1 ) = λ1 v2∗ v1 .
That is,
λ∗2 = λ1 or v2∗ v1 = 0.
But according to the last paragraph all eigenvalues are real; the eigenval-
ues λ1 and λ2 are no exceptions. Hence,
λ2 = λ1 or v2∗ v1 = 0.
To prove the last hypothesis of the three needs first some definitions as
follows. Given an m × m matrix A, let the s columns of the m × s matrix Vo
represent the s independent eigenvectors of A such that (i) each column
has unit magnitude and (ii) columns whose eigenvectors share the same
eigenvalue lie orthogonal to one another. Let the s × s diagonal matrix Λo
carry the eigenvalues on its main diagonal such that
AVo = Vo Λo ,
27
[42, § 8.1]
where the distinction between the matrix Λo and the full eigenvalue matrix Λ
of (14.22) is that the latter always includes a p-fold eigenvalue p times,
whereas the former includes a p-fold eigenvalue only as many times as the
eigenvalue enjoys independent eigenvectors. Let the m−s columns of the m×
(m − s) matrix Vo⊥ represent the complete orthogonal complement (§ 13.10)
to Vo —perpendicular to all eigenvectors, each column of unit magnitude—
such that
Vo⊥∗ Vo = 0 and Vo⊥∗ Vo⊥ = Im−s .
Recall from § 14.5 that s 6= 0 but 0 < s ≤ m because every square matrix
has at least one eigensolution. Recall from § 14.6 that s = m if and only
if A is diagonalizable.28
With these definitions in hand, we can now prove by contradiction that
all Hermitian matrices are diagonalizable, falsely supposing a nondiagonal-
izable Hermitian matrix A, whose Vo⊥ (since A is supposed to be nondiag-
onalizable, implying that s < m) would have at least one column. For such
a matrix A, s × (m − s) and (m − s) × (m − s) auxiliary matrices F and G
necessarily would exist such that
AVo⊥ = Vo F + Vo⊥ G,
not due to any unusual property of the product AVo⊥ but for the mundane
reason that the columns of Vo and Vo⊥ together by definition addressed
28
A concrete example: the invertible but nondiagonalizable matrix
2 3
−1 0 0 0
6 −6 5 25 − 52 7
A=6 4 0
7
0 5 0 5
0 0 0 5
has a single eigenvalue at λ = −1 and a triple eigenvalue at λ = 5, the latter of whose

eigenvector space is fully characterized by two eigenvectors rather than three such that
2 1 3
√
2
0 0 2 3
2
0
3
6 √1 1 0 7 7 −1 0 0 0 7
⊥
6
Vo = 6 2 7, Λo = 4 0 5 0 5, Vo = 6 4 √12 5.
6
0 √12 5
7
4 0
0 0 5
0 0 √12 − √12
The orthogonal complement Vo⊥ supplies the missing vector, not an eigenvector but per-
pendicular to them all.
In the example, m = 4 and s = 3.
All vectors in the example are reported with unit magnitude. The two λ = 5 eigenvectors
are reported in mutually orthogonal form, but notice that eigenvectors corresponding to
distinct eigenvalues need not be orthogonal when A is not Hermitian.
14.11. THE HERMITIAN MATRIX 395
the space of all m-element vectors—including the columns of AVo⊥ . Left-

multiplying by Vo∗ , we would have by successive steps that
Vo∗ AVo⊥ = Vo∗ Vo F + Vo∗ Vo⊥ G,

(AVo )∗ Vo⊥ = Is F + Vo∗ Vo⊥ G,
(Vo Λo )∗ Vo⊥ = F + Vo∗ Vo⊥ G,
Λ∗o Vo∗ Vo⊥ = F + Vo∗ Vo⊥ G,
Λ∗o (0) = F + (0)G,
0 = F,
where we had relied on the assumption that A were Hermitian and thus
that, as proved above, its distinctly eigenvalued eigenvectors lay orthogonal
to one another; in consequence of which A∗ = A and Vo∗ Vo = Is .
The finding that F = 0 reduces the AVo⊥ equation above to read
AVo⊥ = Vo⊥ G.
In the reduced equation the matrix G would have at least one eigensolution,
not due to any unusual property of G but because according to § 14.5 every
square matrix, 1 × 1 or larger, has at least one eigensolution. Let (µ, w)
represent an eigensolution of G. Right-multiplying by the (m − s)-element
vector w 6= 0, we would have by successive steps that
AVo⊥ w = Vo⊥ Gw,

A(Vo⊥ w) = µ(Vo⊥ w).
The last equation claims that (µ, Vo⊥ w) were an eigensolution of A, when
we had supposed that all of A’s eigenvectors lay in the space addressed by
the columns of Vo , thus by construction did not lie in the space addressed
by the columns of Vo⊥ . The contradiction proves false the assumption that
gave rise to it. The assumption: that a nondiagonalizable Hermitian A
existed. We conclude that all Hermitian matrices are diagonalizable—and
conclude further that they are unitarily diagonalizable on the ground that
their eigenvectors lie orthogonal to one another—as was to be demonstrated.
Having proven that all Hermitian matrices are diagonalizable and have
real eigenvalues and orthogonal eigenvectors, one wonders whether the con-
verse holds: are all diagonalizable matrices with real eigenvalues and or-
thogonal eigenvectors Hermitian? To show that they are, one can construct
the matrix described by the diagonalization formula (14.22),
A = V ΛV ∗ ,
where V −1 = V ∗ because this V is unitary (§ 13.12). The equation’s adjoint

is
A∗ = V Λ∗ V ∗ .
But all the eigenvalues here are real, which means that Λ∗ = Λ and the
right sides of the two equations are the same. That is, A∗ = A as was
to be demonstrated. All diagonalizable matrices with real eigenvalues and
orthogonal eigenvectors are Hermitian.
This section brings properties that greatly simplify many kinds of matrix
analysis. The properties demand a Hermitian matrix, which might seem a
severe and unfortunate restriction—except that one can left-multiply any
exactly determined linear system Cx = d by C ∗ to get the equivalent Her-
mitian system
[C ∗ C]x = [C ∗ d], (14.45)
in which A = C ∗ C and b = C ∗ d, for which the properties obtain.29
14.12 The singular-value decomposition

Occasionally an elegant idea awaits discovery, overlooked, almost in plain
sight. If the unlikely thought occurred to you to take the square root of a
matrix, then the following idea is one you might discover.30
Consider the n × n product A∗ A of a tall or square, m × n matrix A of
full column rank
r=n≤m
and its adjoint A∗ . The product A∗ A is invertible according to § 13.6.2; is
positive definite according to § 13.6.3; and, since (A∗ A)∗ = A∗ A, is clearly
Hermitian according to § 14.11; thus is unitarily diagonalizable according
to (14.44) as
A∗ A = V ΛV ∗ . (14.46)
Here, the n × n matrices Λ and V represent respectively the eigenvalues and
eigenvectors not of A but of the product A∗ A. Though nothing requires the
product’s eigenvectors to be real, because the product is positive definite
§ 14.5 does require all of its eigenvalues to be real and moreover positive—
which means among other things that the eigenvalue matrix Λ has full rank.
That the eigenvalues, the diagonal elements of Λ, are real and positive is
29
The device (14.45) worsens a matrix’s condition and may be undesirable for this
reason, but it works in theory at least.
30
[67, “Singular value decomposition,” 14:29, 18 Oct. 2007]
14.12. THE SINGULAR-VALUE DECOMPOSITION 397
a useful fact; for just as a real, positive scalar has a real, positive square
root, so equally has √Λ a real, positive square root under these conditions.
Let the symbol Σ = Λ represent the n × n real, positive square root of the
eigenvalue matrix Λ such that
Λ = Σ∗ Σ, (14.47)
√
+ λ1 0 ··· 0 0
2 3
√
6 0 + λ2 ··· 0 0 7
.. .. .. ..
6 7
∗ ..
Σ =Σ = 6
. . . 7,
7
√. .
6
6 7
4 0 0 ··· + λn−1 0
√
5
0 0 ··· 0 + λn
where the singular values of A populate Σ’s diagonal. Applying (14.47)

to (14.46) then yields
A∗ A = V Σ∗ ΣV ∗ ,
(14.48)
V ∗ A∗ AV = Σ∗ Σ.
Now consider the m × m matrix U such that
AV Σ−1 = U In ,
AV = U Σ, (14.49)
∗
A = U ΣV .
Substituting (14.49)’s second line into (14.48)’s second line gives the equa-
tion
Σ∗ U ∗ U Σ = Σ∗ Σ;
but ΣΣ−1 = In , so left- and right-multiplying respectively by Σ−∗ and Σ−1
leaves that
In U ∗ U In = In ,
which says neither more nor less than that the first n columns of U are
orthonormal. Equation (14.49) does not constrain the last m − n columns
of U , leaving us free to make them anything we want. Why not use Gram-
Schmidt to make them orthonormal, too, thus making U a unitary matrix?
If we do this, then the surprisingly simple (14.49) constitutes the singular-
value decomposition of A.
If A happens to have broad shape then we can decompose A∗ , instead,
so this case poses no special trouble. Apparently every full-rank matrix has
a singular-value decomposition.
But what of the matrix of less than full rank r < n? In this case the
product A∗ A is singular and has only s < n nonzero eigenvalues (it may be
that s = r, but this is irrelevant to the proof at hand). However, if the s

nonzero eigenvalues are arranged first in Λ, then (14.49) becomes
AV Σ−1 = U Is ,
AV = U Σ, (14.50)
∗
A = U ΣV .
The product A∗ A is nonnegative definite in this case and ΣΣ−1 = Is , but

the reasoning is otherwise the same as before. Apparently every matrix of
less than full rank has a singular-value decomposition, too.
If A happens to be an invertible square matrix, then the singular-value
decomposition evidently inverts it as
A−1 = V Σ−1 U ∗ . (14.51)
14.13 General remarks on the matrix

Chapters 11 through 14 have derived the uncomfortably bulky but—incred-
ibly—approximately minimal knot of theory one needs to grasp the matrix
properly and to use it with moderate versatility. As far as the writer knows,
no one has yet discovered a satisfactory way to untangle the knot. The choice
to learn the basic theory of the matrix is almost an all-or-nothing choice;
and how many scientists and engineers would rightly choose the “nothing”
if the matrix did not serve so very many applications as it does? Since it
does serve so very many, the “all” it must be.31 Applied mathematics brings
nothing else quite like it.
These several matrix chapters have not covered every topic they might.
The topics they omit fall roughly into two classes. One is the class of more
advanced and more specialized matrix theory, about which we shall have
more to say in a moment. The other is the class of basic matrix theory these
chapters do not happen to use. The essential agents of matrix analysis—
multiplicative associativity, rank, inversion, pseudoinversion, the kernel, the
orthogonal complement, orthonormalization, the eigenvalue, diagonalization
and so on—are the same in practically all books on the subject, but the way
the agents are developed differs. This book has chosen a way that needs some
tools like truncators other books omit, but does not need other tools like
31
Of course, one might avoid true understanding and instead work by memorized rules.
That is not always a bad plan, really; but if that were your plan then it seems spectacularly
unlikely that you would be reading a footnote buried beneath the further regions of the
hinterland of Chapter 14 in such a book as this.
14.13. GENERAL REMARKS ON THE MATRIX 399
projectors other books32 include. What has given these chapters their hefty
bulk is not so much the immediate development of the essential agents as the
preparatory development of theoretical tools used to construct the essential
agents, yet most of the tools are of limited interest in themselves; it is the
agents that matter. Tools like the projector not used here tend to be omitted
here or deferred to later chapters, not because they are altogether useless but
because they are not used here and because the present chapters are already
too long. The reader who understands the Moore-Penrose pseudoinverse
and/or the Gram-Schmidt process reasonably well can after all pretty easily
figure out how to construct a projector without explicit instructions thereto,
should the need arise.33
Paradoxically and thankfully, more advanced and more specialized ma-
trix theory though often harder tends to come in smaller, more manageable
increments: the Cholesky decomposition, for instance; or the conjugate-
gradient algorithm. The theory develops endlessly. From the present pause
one could proceed directly to such topics. However, since this is the first
proper pause these several matrix chapters have afforded, since the book
is Derivations of Applied Mathematics rather than Derivations of Applied
Matrices, maybe we ought to take advantage to change the subject.
32
Such as [30, § 3.VI.3], a lengthy but well-knit tutorial this writer recommends.
33
Well, since we have brought it up (though only as an example of tools these chapters
have avoided bringing up), briefly: a projector is a matrix that flattens an arbitrary
vector b into its nearest shadow b̃ within some restricted subspace. If the columns of A
represent the subspace, then x represents b̃ in the subspace basis iff Ax = b̃, which is to
say that Ax ≈ b, whereupon x = A† b. That is, per (13.32),
b̃ = Ax = AA† b = [BC][C ∗ (CC ∗ )−1 (B ∗ B)−1 B ∗ ]b = B(B ∗ B)−1 B ∗ b,
in which the matrix B(B ∗ B)−1 B ∗ is the projector. Thence it is readily shown that the
deviation b − b̃ lies orthogonal to the shadow b̃. More broadly defined, any matrix M for
which M 2 = M is a projector. One can approach the projector in other ways, but there
are two ways at least.
Chapter 15
Vector analysis
Leaving the matrix, this chapter and the next turn to a curiously underap-
preciated agent of applied mathematics, the three-dimensional geometrical
vector, first met in §§ 3.3, 3.4 and 3.9. Seen from one perspective, the three-
dimensional geometrical vector is the n = 3 special case of the general,
n-dimensional vector of Chs. 11 through 14. Because its three elements
represent the three dimensions of the physical world, however, the three-
dimensional geometrical vector merits closer attention and special treat-
ment.1
It also merits a shorter name. Where the geometrical context is clear—as
it is in this chapter and the next—we shall call the three-dimensional geo-
metrical vector just a vector. A name like “matrix vector” or “n-dimensional
vector” can disambiguate the vector of Chs. 11 through 14 where necessary
but, since the three-dimensional geometrical vector is in fact a vector, it
usually is not necessary to disambiguate. The lone word vector serves.
In the present chapter’s context and according to § 3.3, a vector con-
sists of an amplitude of some kind plus a direction. Per § 3.9, three scalars
called coordinates suffice together to specify the amplitude and direction
and thus the vector, the three being (x, y, x) in the rectangular coordinate
system, (ρ; φ, z) in the cylindrical coordinate system, or (r; θ; φ) in the spher-
ical spherical coordinate system—as Fig. 15.1 illustrates and Table 3.4 on
page 65 interrelates—among other, more exotic possibilities (§ 15.7).
The vector brings an elegant notation. This chapter and Ch. 16 detail
1
[12, Ch. 2]
401
402 CHAPTER 15. VECTOR ANALYSIS
Figure 15.1: A point on a sphere, in spherical (r; θ; φ) and cylindrical (ρ; φ, z)

coordinates. (The axis labels bear circumflexes in this figure only to disam-
biguate the ẑ axis from the cylindrical coordinate z. See also Fig. 15.5.)
ẑ
r
θ z
ŷ
φ
x̂ ρ
403
it. Without the notation, one would write an expression like
(z − z ′ ) − [∂z ′ /∂x]x=x′ ,y=y′ (x − x′ ) − [∂z ′ /∂y]x=x′ ,y=y′ (y − y ′ )

q
[1 + (∂z ′ /∂x)2 + (∂z ′ /∂y)2 ]x=x′ ,y=y′ [(x − x′ )2 + (y − y ′ )2 + (z − z ′ )2 ]
for the aspect coefficient relative to a local surface normal (and if the sen-
tence’s words do not make sense to you yet, don’t worry; just look the
symbols over and appreciate the expression’s bulk). The same coefficient in
standard vector notation is
n̂ · ∆r̂.
Besides being more evocative (once one has learned to read it) and much
more compact, the standard vector notation brings the major advantage
of freeing a model’s geometry from reliance on any particular coordinate
system. Reorienting axes (§ 15.1) for example knots the former expression
like spaghetti but does not disturb the latter expression at all.
Two-dimensional geometrical vectors arise in practical modeling about
as often as three-dimensional geometrical vectors do. Fortunately, the two-
dimensional case needs little special treatment, for it is just the three-
dimensional with z = 0 or θ = 2π/4 (see however § 15.6).
Here at the outset, a word on complex numbers seems in order. Unlike
most of the rest of the book this chapter and the next will work chiefly
in real numbers, or at least in real coordinates. Notwithstanding, complex
coordinates are possible. Indeed, in the rectangular coordinate system com-
plex coordinates are perfectly appropriate and are straightforward enough to
handle. The cylindrical and spherical systems however, which these chapters
also treat, were not conceived with complex coordinates in mind; and, al-
though it might with some theoretical subtlety be possible to treat complex
radii, azimuths and elevations consistently as three-dimensional coordinates,
these chapters will not try to do so.2 (This is not to say that you cannot
have a complex vector like, say, ρ̂[3 + j2] − φ̂[1/4] in a nonrectangular basis.
You can have such a vector, it is fine, and these chapters will not avoid it.
What these chapters will avoid are complex nonrectangular coordinates like
[3 + j2; −1/4, 0].)
Vector addition will already be familiar to the reader from Ch. 3 or (quite
likely) from earlier work outside this book. This chapter therefore begins
with the reorientation of axes in § 15.1 and vector multiplication in § 15.2.
2
The author would be interested to learn if there existed an uncontrived scientific or
engineering application that actually used complex, nonrectangular coordinates.
15.1 Reorientation
Matrix notation expresses the rotation of axes (3.5) as
x̂′
2 3 2 32 3
cos φ sin φ 0 x̂
4 ŷ′ 5 = 4 − sin φ cos φ 0 54 ŷ 5.
ẑ′ 0 0 1 ẑ
In three dimensions however one can do more than just to rotate the x and y
axes about the z. One can reorient the three axes generally as follows.
15.1.1 The Tait-Bryan rotations

With a yaw and a pitch to point the x axis in the desired direction plus a
roll to position the y and z axes as desired about the new x axis,3 one can
reorient the three axes generally:
x̂′
2 3 2 32 32 32 3
1 0 0 cos θ 0 − sin θ cos φ sin φ 0 x̂
4 ŷ′ 5 = 4 0 cos ψ sin ψ 54 0 1 0 54 − sin φ cos φ 0 54 ŷ 5;
ẑ′ 0 − sin ψ cos ψ sin θ 0 cos θ 0 0 1 ẑ
(15.1)
or, inverting per (3.6),
2 3 2 32 32 32 ′ 3
x̂ cos φ − sin φ 0 cos θ 0 sin θ 1 0 0 x̂
4 ŷ 5 = 4 sin φ cos φ 0 54 0 1 0 54 0 cos ψ − sin ψ 54 ŷ′ 5.
ẑ 0 0 1 − sin θ 0 cos θ 0 sin ψ cos ψ ẑ′
(15.2)
These are called the Tait-Bryan rotations, or alternately the Cardano rota-
tions.4,5
3
The English maritime verbs to yaw, to pitch and to roll describe the rotational motion
of a vessel at sea. For a vessel to yaw is for her to rotate about her vertical axis, so her
bow (her forwardmost part) yaws from side to side. For a vessel to pitch is for her to
rotate about her “beam axis,” so her bow pitches up and down. For a vessel to roll is for
her to rotate about her “fore-aft axis” such that she rocks or lists (leans) without changing
the direction she points [67, “Glossary of nautical terms,” 23:00, 20 May 2008]. In the
Tait-Bryan rotations as explained in this book, to yaw is to rotate about the z axis, to
pitch about the y, and to roll about the x [36]. In the Euler rotations as explained in this
book later in the present section, however, the axes are assigned to the vessel differently
such that to yaw is to rotate about the x axis, to pitch about the y, and to roll about
the z. This implies that the Tait-Bryan vessel points x-ward whereas the Euler vessel
points z-ward. The reason to shift perspective so is to maintain the semantics of the
symbols θ and φ (though not ψ) according to Fig. 15.1.
If this footnote seems confusing, then read (15.1) and (15.7) which are correct.
4
The literature seems to agree on no standard order among the three Tait-Bryan rota-
tions; and, though the rotational angles are usually named φ, θ and ψ, which angle gets
which name admittedly depends on the author.
5
[11]
15.1. REORIENTATION 405
Notice in (15.1) and (15.2) that the transpose (though curiously not the
adjoint) of each 3 × 3 Tait-Bryan factor is also its inverse.
In concept, the Tait-Bryan equations (15.1) and (15.2) say nearly all
one needs to say about reorienting axes in three dimensions; but, still, the
equations can confuse the uninitiated. Consider a vector
v = x̂x + ŷy + ẑz. (15.3)
It is not the vector one reorients but rather the axes used to describe the
vector. Envisioning the axes as in Fig. 15.1 with the z axis upward, one first
yaws the x axis through an angle φ toward the y then pitches it downward
through an angle θ away from the z. Finally, one rolls the y and z axes
through an angle ψ about the new x, all the while maintaining the three
axes rigidly at right angles to one another. These three Tait-Bryan rotations
can orient axes any way. Yet, even once one has clearly visualized the Tait-
Bryan sequence, the prospect of applying (15.2) (which inversely represents
the sequence) to (15.3) can still seem daunting until one rewrites the latter
equation in the form
 
x
v = x̂ ŷ ẑ  y  , (15.4)
z
after which the application is straightforward. There results
v′ = x̂′ x′ + ŷ′ y ′ + ẑ′ z ′ ,
where
x′
2 3 2 32 32 32 3
1 0 0 cos θ 0 − sin θ cos φ sin φ 0 x
4 y′ 5 ≡ 4 0 cos ψ sin ψ 54 0 1 0 54 − sin φ cos φ 0 54 y 5,
z′ 0 − sin ψ cos ψ sin θ 0 cos θ 0 0 1 z
(15.5)
and where Table 3.4 converts to cylindrical or spherical coordinates if and
as desired. Since (15.5) resembles (15.1), it comes as no surprise that its
inverse,
2 3 2 32 32 32 ′ 3
x cos φ − sin φ 0 cos θ 0 sin θ 1 0 0 x
4 y 5 = 4 sin φ cos φ 0 54 0 1 0 54 0 cos ψ − sin ψ 54 y ′ 5,
z 0 0 1 − sin θ 0 cos θ 0 sin ψ cos ψ z′
(15.6)
resembles (15.2).
15.1.2 The Euler rotations

A useful alternative to the Tait-Bryan rotations are the Euler rotations,
which view the problem of reorientation from the perspective of the z axis
rather than of the x. The Euler rotations consist of a roll and a pitch
followed by another roll, without any explicit yaw:6
x̂′
2 3 2 32 32 32 3
cos ψ sin ψ 0 cos θ 0 − sin θ cos φ sin φ 0 x̂
4 ŷ′ 5 = 4 − sin ψ cos ψ 0 54 0 1 0 54 − sin φ cos φ 0 54 ŷ 5;
ẑ′ 0 0 1 sin θ 0 cos θ 0 0 1 ẑ
(15.7)
and inversely
2 3 2 32 32 32 ′ 3
x̂ cos φ − sin φ 0 cos θ 0 sin θ cos ψ − sin ψ 0 x̂
4 ŷ 5 = 4 sin φ cos φ 0 54 0 1 0 54 sin ψ cos ψ 0 54 ŷ′ 5.
ẑ 0 0 1 − sin θ 0 cos θ 0 0 1 ẑ′
(15.8)
Whereas the Tait-Bryan point the x axis first, the Euler tactic is to point
first the z.
So, that’s it. One can reorient three axes arbitrarily by rotating them in
pairs about the z, y and x or the z, y and z axes in sequence—or, general-
izing, in pairs about any of the three axes so long as the axis of the middle
rotation differs from the axes (Tait-Bryan) or axis (Euler) of the first and
last. A firmer grasp of the reorientation of axes in three dimensions comes
with practice, but those are the essentials of it.
15.2 Multiplication
One can multiply a vector in any of three ways. The first, scalar multipli-
cation, is trivial: if a vector v is as defined by (15.3), then
ψv = x̂ψx + ŷψy + ẑψz. (15.9)
Such scalar multiplication evidently scales a vector’s length without divert-

ing its direction. The other two forms of vector multiplication involve multi-
plying a vector by another vector and are the subjects of the two subsections
that follow.
6
As for the Tait-Bryan, for the Euler also the literature agrees on no standard sequence.
What one author calls a pitch, another might call a yaw, and some prefer to roll twice
about the x axis rather than the z. What makes a reorientation an Euler rather than a
Tait-Bryan is that the Euler rolls twice.
15.2. MULTIPLICATION 407
15.2.1 The dot product

We first met the dot product in § 13.8. It works similarly for the geometrical
vectors of this chapter as for the matrix vectors of Ch. 13:
v1 · v2 = x1 x2 + y1 y2 + z1 z2 , (15.10)
which, if the vectors v1 and v2 are real, is the product of the two vectors
to the extent to which they run in the same direction. It is the product to
the extent to which the vectors run in the same direction because one can
reorient axes to point x̂′ in the direction of v1 , whereupon v1 · v2 = x′1 x′2
since y1′ and z1′ have vanished.
Naturally, to be valid, the dot product must not vary under a reorienta-
tion of axes; and indeed if we write (15.10) in matrix notation,
 
x2
v1 · v2 = x1 y1 z1  y2  , (15.11)
z2
and then expand each of the two factors on the right according to (15.6),
we see that the dot product does not in fact vary. As in (13.43) of § 13.8,
here too the relationship
v1∗ · v2 = v1∗ v2 cos θ,

(15.12)
v̂1∗ · v̂2 = cos θ,
gives the angle θ between two vectors according Fig. 3.1’s cosine if the vectors
are real, by definition hereby if complex. Consequently, the two vectors are
mutually orthogonal—that is, the vectors run at right angles θ = 2π/4 to
one another—if and only if
v1∗ · v2 = 0.
That the dot product is commutative,
v2 · v1 = v1 · v2 , (15.13)
is obvious from (15.10). Fig. 15.2 illustrates the dot product.
15.2.2 The cross product

The dot product of two vectors according to § 15.2.1 is a scalar. One can
also multiply two vectors to obtain a vector, however, and it is often useful
to do so. As the dot product is the product of two vectors to the extent to
Figure 15.2: The dot product.
b
a · b = ab cos θ
θ
a
b cos θ
which they run in the same direction, the cross product is the product of two
vectors to the extent to which they run in different directions. Unlike the
dot product the cross product is a vector, defined in rectangular coordinates
as

x̂ ŷ ẑ

v1 × v2 = x1 y1 z1 (15.14)
x2 y2 z2
≡ x̂(y1 z2 − z1 y2 ) + ŷ(z1 x2 − x1 z2 ) + ẑ(x1 y2 − y1 x2 ),
where the |·| notation is a mnemonic (actually a pleasant old determinant

notation § 14.1 could have but did not happen to use) whose semantics are
as shown.
As the dot product, the cross product too is invariant under reorienta-
tion. One could demonstrate this fact by multiplying out (15.2) and (15.6)
then substituting the results into (15.14): a lengthy, unpleasant exercise.
Fortunately, it is also an unnecessary exercise; for, inasmuch as a reorienta-
tion consists of three rotations in sequence, it suffices merely that rotation
about one axis not alter the dot product. One proves the proposition in the
latter form by setting any two of φ, θ and ψ to zero before multiplying out
and substituting.
Several facets of the cross product draw attention to themselves.
• The cyclic progression
··· → x → y → z → x → y → z → x → y → ··· (15.15)
of (15.14) arises again and again in vector analysis. Where the pro-
gression is honored, as in ẑx1 y2 , the associated term bears a + sign,
otherwise a − sign, due to § 11.6’s parity principle and the right-hand
rule.
15.2. MULTIPLICATION 409
• The cross product is not commutative. In fact,
v2 × v1 = −v1 × v2 , (15.16)
which is a direct consequence of the previous point regarding parity, or

which can be seen more prosaically in (15.14) by swapping the places
of v1 and v2 .
• The cross product is not associative. That is,
(v1 × v2 ) × v3 6= v1 × (v2 × v3 ),
as is proved by a suitable counterexample like v1 = v2 = x̂, v3 = ŷ.
• The cross product runs perpendicularly to each of its two factors if the
vectors involved are real. That is,
v1 · (v1 × v2 ) = 0 = v2 · (v1 × v2 ), (15.17)
as is seen by substituting (15.14) into (15.10) with an appropriate

change of variables and simplifying.
• Unlike the dot product, the cross product is closely tied to three-
dimensional space. Two-dimensional space (a plane) can have a cross
product so long as one does not mind that the product points off
into the third dimension, but to speak of a cross product in four-
dimensional space would require arcane definitions and would oth-
erwise make little sense. Fortunately, the physical world is three-
dimensional (or, at least, the space in which we model all but a few,
exotic physical phenomena is three-dimensional), so the cross prod-
uct’s limitation as defined here to three dimensions will seldom if ever
disturb us.
• Section 15.2.1 has related the cosine of the angle between vectors to
the dot product. One can similarly relate the angle’s sine to the cross
product if the vectors involved are real, as
|v1 × v2 | = v1 v2 sin θ,
(15.18)
|v̂1 × v̂2 | = sin θ,
demonstrated by reorienting axes such that v̂1 = x̂′ , that v̂2 has no
component in the ẑ′ direction, and that v̂2 has only a nonnegative com-
ponent in the ŷ′ direction; by remembering that reorientation cannot
Figure 15.3: The cross product.
c = a×b
= ĉab sin θ
alter a cross product; and finally by applying (15.14) and comparing

the result against Fig. 3.1’s sine. (If the vectors involved are com-
plex then nothing prevents the operation |v1∗ × v2 | by analogy with
eqn. 15.12—in fact the operation v1∗ × v2 without the magnitude sign
is used routinely to calculate electromagnetic power flow7 —but each of
the cross product’s three rectangular components has its own complex
phase which the magnitude operation flattens, so the result’s relation-
ship to the sine of an angle is not immediately clear.)
Fig. 15.3 illustrates the cross product.
15.3 Orthogonal bases

A vector exists independently of the components by which one expresses
it, for, whether q = x̂x + ŷy + ẑz or q = x̂′ x′ + ŷ′ y ′ + ẑ′ z ′ , it remains
the same vector q. However, where a model involves a circle, a cylinder
or a sphere, where a model involves a contour or a curved surface of some
kind, to choose x̂′ , ŷ′ and ẑ′ wisely can immensely simplify the model’s
analysis. Normally one requires that x̂′ , ŷ′ and ẑ′ each retain unit length,
run perpendiclarly to one another, and obey the right-hand rule (§ 3.3),
but otherwise any x̂′ , ŷ′ and ẑ′ can serve. Moreover, a model can specify
various x̂′ , ŷ′ and ẑ′ under various conditons, for nothing requires the three
to be constant.
7
[28, eqn. 1-51]
15.3. ORTHOGONAL BASES 411
Recalling the constants and variables of § 2.7, such a concept is flexible

enough to confuse the uninitiated severely and soon. As in § 2.7, here too an
example affords perspective. Imagine driving your automobile down a wind-
ing road, where q represented your speed8 and ℓ̂ represented the direction
the road ran, not generally, but just at the spot along the road at which your
automobile momentarily happened to be. That your velocity were ℓ̂q meant
that you kept skilfully to your lane; on the other hand, that your velocity
were (ℓ̂ cos ψ + v̂ sin ψ)q—where v̂, at right angles to ℓ̂, represented the di-
rection right-to-left across the road—would have you drifting out of your
lane at an angle ψ. A headwind had velocity −ℓ̂qwind ; a crosswind, ±v̂qwind .
A car a mile ahead of you had velocity ℓ̂2 q2 = (ℓ̂ cos β + v̂ sin β)q2 , where β
represented the difference (assuming that the other driver kept skilfully to
his own lane) between the road’s direction a mile ahead and its direction at
your spot. For all these purposes the unit vector ℓ̂ would remain constant.
However, fifteen seconds later, after you had rounded a bend in the road,
the symbols ℓ̂ and v̂ would by definition represent different vectors than
before, with respect to which one would express your new velocity as ℓ̂q but
would no longer express the headwind’s velocity as −ℓ̂qwind because, since
the road had turned while the wind had not, the wind would no longer be
a headwind. And this is where confusion can arise: your own velocity had
changed while the expression representing it had not; whereas the wind’s
velocity had not changed while the expression representing it had. This is
not because ℓ̂ differs from place to place at a given moment, for like any
other vector the vector ℓ̂ (as defined in this particular example) is the same
vector everywhere. Rather, it is because ℓ̂ is defined relative to the road at
your automobile’s location, which location changes as you drive.
If a third unit vector ŵ were defined, perpendicular both to ℓ̂ and to v̂
such that [ℓ̂ v̂ ŵ] obeyed the right-hand rule, then the three together would
constitute an orthogonal basis. Any three real,9 right-handedly mutually
perpendicular unit vectors [x̂′ ŷ′ ẑ′ ] in three dimensions, whether constant
8
Conventionally one would prefer the letter v to represent speed, with velocity as v
which in the present example would happen to be v = ℓ̂v. However, this section will
require the letter v for an unrelated purpose.
9
A complex orthogonal basis is also theoretically possible but is normally unnecessary
in geometrical applications and involves subtleties in the cross product. This chapter,
which specifically concerns three-dimensional geometrical vectors rather than the general,
n-dimensional vectors of Ch. 11, is content to consider real bases only. Note that one can
express a complex vector in a real basis.
or variable, for which
ŷ′ · ẑ′ = 0, ŷ′ × ẑ′ = x̂′ , ℑ(x̂′ ) = 0,

ẑ′ · x̂′ = 0, ẑ′ × x̂′ = ŷ′ , ℑ(ŷ′ ) = 0, (15.19)
x̂′ · ŷ′ = 0, x̂′ × ŷ′ = ẑ′ , ℑ(ẑ′ ) = 0,
constitutes such an orthogonal basis, from which other vectors can be built.
The geometries of some models suggest no particular basis, when one usually
just uses a constant [x̂ ŷ ẑ]. The geometries of other models however do
suggest a particular basis, often a variable one.
• Where the model features a contour like the example’s winding road,
an [ℓ̂ v̂ ŵ] basis (or a [û v̂ ℓ̂] basis or even a [û ℓ̂ ŵ] basis) can be
used, where ℓ̂ locally follows the contour. The variable unit vectors v̂
and ŵ (or û and v̂, etc.) can be defined in any convenient way so
long as they remain perpendicular to one another and to ℓ̂—such that
(ẑ × ℓ̂) · ŵ = 0 for instance (that is, such that ŵ lay in the plane of ẑ
and ℓ̂)—but if the geometry suggests a particular v̂ or ŵ (or û), like
the direction right-to-left across the example’s road, then that v̂ or ŵ
should probably be used. The letter ℓ here stands for “longitudinal.”10
• Where the model features a curved surface like the surface of a wavy
sea,11 a [û v̂ n̂] basis (or a [û n̂ ŵ] basis, etc.) can be used, where n̂
points locally perpendicularly to the surface. The letter n here stands
for “normal,” a synonym for “perpendicular.” Observe, incidentally
but significantly, that such a unit normal n̂ tells one everything one
needs to know about its surface’s local orientation.
• Combining the last two, where the model features a contour along a
curved surface, an [ℓ̂ v̂ n̂] basis can be used. One need not worry about
choosing a direction for v̂ in this case since necessarily v̂ = n̂ × ℓ̂.
• Where the model features a circle or cylinder, a [ρ̂ φ̂ ẑ] basis can
be used, where ẑ is constant and runs along the cylinder’s axis (or
perpendicularly through the circle’s center), ρ̂ is variable and points
locally away from the axis, and φ̂ is variable and runs locally along
the circle’s perimeter in the direction of increasing azimuth φ. Refer
to § 3.9 and Fig. 15.4.
10
The assertion wants a citation, which the author lacks.
11
[49]
Figure 15.4: The cylindrical basis. (The conventional symbols b and

respectively represent vectors pointing out of the page toward the reader and
into the page away from the reader. Thus, this figure shows the constant
basis vector ẑ pointing out of the page toward the reader. The dot in the
middle of the b is supposed to look like the tip of an arrowhead.)
φ̂
ρ̂
b
ρ
ẑ
φ
• Where the model features a sphere, an [r̂ θ̂ φ̂] basis can be used,
where r̂ is variable and points locally away from the sphere’s cen-
ter, θ̂ is variable and runs locally tangentially to the sphere’s surface
in the direction of increasing elevation θ (that is, though not usually
in the −ẑ direction itself, as nearly as possible to the −ẑ direction
without departing from the sphere’s surface), and φ̂ is variable and
runs locally tangentially to the sphere’s surface in the direction of in-
creasing azimuth φ (that is, along the sphere’s surface perpendicularly
to ẑ). Standing on the earth’s surface, with the earth as the sphere, r̂
would be up, θ̂ south, and φ̂ east. Refer to § 3.9 and Fig. 15.5.
• Occasionally a model arises with two circles that share a center but
whose axes stand perpendicular to one another. In such a model one
conventionally establishes ẑ as the direction of the principal circle’s
axis but then is left with x̂ or ŷ as the direction of the secondary
x x x y y y
circle’s axis, upon which an [x̂ ρ̂x φ̂ ], [φ̂ r̂ θ̂ ], [φ̂ ŷ ρ̂y ] or [θ̂ φ̂ r̂]
basis can be used locally as appropriate. Refer to § 3.9.
Many other orthogonal bases are possible (as in § 15.7, for instance), but the
foregoing are the most common. Whether listed here or not, each orthogonal
basis orders its three unit vectors by the right-hand rule (15.19).
Figure 15.5: The spherical basis (see also Fig. 15.1).
r̂
b
φ̂
θ̂
Quiz: what does the vector expression ρ̂3 − φ̂(1/4) + ẑ2 mean? Wrong
answer: it meant the cylindrical coordinates (3; −1/4, 2); or, it meant the
position vector x̂3 cos(−1/4) + ŷ3 sin(−1/4) + ẑ2 associated with those co-
ordinates. Right answer: the expression means nothing certain in itself
but acquires a definite meaning only when an azimuthal coordinate φ is
also supplied, after which the expression indicates the ordinary rectangular
vector x̂′ 3 − ŷ′ (1/4) + ẑ′ 2, where x̂′ = ρ̂ = x̂ cos φ + ŷ sin φ, ŷ′ = φ̂ =
−x̂ sin φ + ŷ cos φ, and ẑ′ = ẑ. But, if this is so—if the cylindrical basis
[ρ̂ φ̂ ẑ] is used solely to express rectangular vectors—then why should we
name this basis “cylindrical”? Answer: only because cylindrical coordinates
(supplied somewhere) determine the actual directions of its basis vectors.
Once directions are determined such a basis is used purely rectangularly,
like any other orthogonal basis.
This can seem confusing until one has grasped what the so-called non-
rectangular bases are for. Consider the problem of air flow in a jet engine.
It probably suits such a problem that instantaneous local air velocity within
the engine cylinder be expressed in cylindrical coordinates, with the z axis
oriented along the engine’s axle; but this does not mean that the air flow
within the engine cylinder were everywhere ẑ-directed. On the contrary, a
local air velocity of q = [−ρ̂5.0 + φ̂30.0 − ẑ250.0] m/s would have air moving
through the point in question at 250.0 m/s aftward along the axle, 5.0 m/s
inward toward the axle and 30.0 m/s circulating about the engine cylinder.
In this model, it is true that the basis vectors ρ̂ and φ̂ indicate differ-
ent directions at different positions within the cylinder, but at a particular
position the basis vectors are still used rectangularly to express q, the in-
stantaneous local air velocity at that position. It’s just that the “rectangle”
is rotated locally to line up with the axle.
Naturally, you cannot make full sense of an air-velocity vector q unless
you have also the coordinates (ρ; φ, z) of the position within the engine
cylinder at which the air has the velocity the vector specifies—yet this is
when confusion can arise, for besides the air-velocity vector there is also,
separately, a position vector r = x̂ρ cos φ + ŷρ sin φ + ẑz. One may denote
the air-velocity vector as12 q(r), a function of position; yet, though the
position vector is as much a vector as the velocity vector is, one nonetheless
handles it differently. One will not normally express the position vector r in
the cylindrical basis.
It would make little sense to try to express the position vector r in the
cylindrical basis because the position vector is the very thing that determines
the cylindrical basis. In the cylindrical basis, after all, the position vector
is necessarily r = ρ̂ρ + ẑz (and consider: in the spherical basis it is the
even more cryptic r = r̂r), and how useful is that, really? Well, maybe it
is useful in some situations, but for the most part to express the position
vector in the cylindrical basis would be as to say, “My house is zero miles
away from home.” Or, “The time is presently now.” Such statements may
be tautologically true, perhaps, but they are confusing because they only
seem to give information. The position vector r determines the basis, after
which one expresses things other than position, like instantaneous local air
velocity q, in that basis. In fact, the only basis normally suitable to express
a position vector is a fixed rectangular basis like [x̂ ŷ ẑ]. Otherwise, one
uses cylindrical coordinates (ρ; φ, z), but not a cylindrical basis [ρ̂ φ̂ ẑ], to
express a position r in a cylindrical geometry.
Maybe the nonrectangular bases were more precisely called “rectangular
bases of the nonrectangular coordinate systems,” but those are too many
words and, anyway, that is not how the usage has evolved. Chapter 16 will
elaborate the story by considering spatial derivatives of quantities like air
velocity, when one must take the variation in ρ̂ and φ̂ from point to point
12
Conventionally, one is much more likely to denote a velocity vector as u(r) or v(r),
except that the present chapter is (as footnote 8 has observed) already using the letters u
and v for an unrelated purpose. To denote position as r however is entirely standard.
into account, but the foregoing is the basic story nevertheless.
15.4 Notation
The vector notation of §§ 15.1 and 15.2 is correct, familiar and often expe-
dient but sometimes inconveniently prolix. This admittedly difficult section
augments the notation to render it much more concise.
15.4.1 Components by subscript

The notation
ax ≡ x̂ · a, aρ ≡ ρ̂ · a,
ay ≡ ŷ · a, ar ≡ r̂ · a,
az ≡ ẑ · a, aθ ≡ θ̂ · a,
an ≡ n̂ · a, aφ ≡ φ̂ · a,
and so forth abbreviates the indicated dot product. That is to say, the
notation represents the component of a vector a in the indicated direction.
Generically,
aα ≡ α̂ · a. (15.20)
Applied mathematicians use subscripts for several unrelated or vaguely re-
lated purposes, so the full dot-product notation α̂ · a is often clearer in print
than the abbreviation aα is, but the abbreviation especially helps when sev-
eral such dot products occur together in the same expression.
Since13
â = x̂ax + ŷay + ẑaz ,

b̂ = x̂bx + ŷby + ẑbz ,
the abbreviation lends a more amenable notation to the dot and cross prod-
ucts of (15.10) and (15.14):
a · b = ax bx + ay by + az bz ; (15.21)

x̂ ŷ ẑ

a × b = ax ay az . (15.22)
bx by bz
13
“Wait!” comes the objection. “I thought that you said that ax meant x̂ · a. Now you
claim that it means the x component of a?”
But there is no difference between x̂ · a and the x component of a. The two are one and
the same.
15.4. NOTATION 417
In fact—because, as we have seen, reorientation of axes cannot alter the dot

and cross products—any orthogonal basis [x̂′ ŷ′ ẑ′ ] (§ 15.3) can serve here,
so one can write more generally that
a · b = ax′ bx′ + ay′ by′ + az ′ bz ′ ; (15.23)

′
ŷ′ ẑ′

x̂

a × b = ax′ ay′ az ′ . (15.24)
bx ′ by′ bz ′
Because all those prime marks burden the notation and for professional
mathematical reasons, the general forms (15.23) and (15.24) are sometimes
rendered
a · b = a1 b1 + a2 b2 + a3 b3 ,

ê1 ê2 ê3

a × b = a1 a2 a3 ,
b1 b2 b3
but you have to be careful about that in applied usage because people are
not always sure whether a symbol like a3 means “the third component of
the vector a” (as it does here) or “the third vector’s component in the â
direction” (as it would in eqn. 15.10). Typically, applied mathematicians will
write in the manner of (15.21) and (15.22) with the implied understanding
that they really mean (15.23) and (15.24) but prefer not to burden the
notation with extra little strokes—that is, with the implied understanding
that x, y and z could just as well be ρ, φ and z or the coordinates of any
other orthogonal, right-handed, three-dimensional basis.
Some pretty powerful confusion can afflict the student regarding the
roles of the cylindrical symbols ρ, φ and z; or, worse, of the spherical sym-
bols r, θ and φ. Such confusion reflects a pardonable but remediable lack
of understanding of the relationship between coordinates like ρ, φ and z
and their corresponding unit vectors ρ̂, φ̂ and ẑ. Section 15.3 has already
written of the matter; but, further to dispel the confusion, one can now
ask the student what the cylindrical coordinates of the vectors ρ̂, φ̂ and ẑ
are. The correct answer: (1; φ, 0), (1; φ + 2π/4, 0) and (0; 0, 1), respectively.
Then, to reinforce, one can ask the student which cylindrical coordinates
the variable vectors ρ̂ and φ̂ are functions of. The correct answer: both are
functions of the coordinate φ only (ẑ, a constant vector, is not a function
of anything). What the student needs to understand is that, among the
cylindrical coordinates, φ is a different kind of thing than z and ρ are:
• z and ρ are lengths whereas φ is an angle;
• but ρ̂, φ̂ and ẑ are all the same kind of thing, unit vectors;
• and, separately, aρ , aφ and az are all the same kind of thing, lengths.
Now to ask the student a harder question: in the cylindrical basis, what is
the vector representation of (ρ1 ; φ1 , z1 )? The correct answer: ρ̂ρ1 cos(φ1 −
φ) + φ̂ρ1 sin(φ1 − φ) + ẑz1 . The student that gives this answer probably
grasps the cylindrical symbols.
If the reader feels that the notation begins to confuse more than it de-
scribes, the writer empathizes but regrets to inform the reader that the rest
of the section, far from granting the reader a comfortable respite to absorb
the elaborated notation as it stands, shall not delay to elaborate the no-
tation yet further! The confusion however is subjective. The trouble with
vector work is that one has to learn to abbreviate or the expressions in-
volved grow repetitive and unreadably long. For vectors, the abbreviated
notation really is the proper notation. Eventually one accepts the need and
takes the trouble to master the conventional vector abbreviation this section
presents; and, indeed, the abbreviation is rather elegant once one becomes
used to it. So, study closely and take heart! The notation is not actually as
impenetrable as it at first will seem.
15.4.2 Einstein’s summation convention

Einstein’s summation convention is this: that repeated indices are implicitly
summed over.14 For instance, where the convention is in force, the equa-
tion15
a · b = ai bi (15.25)
means that X
a·b = ai bi
i
or more fully that

X
a·b = ai bi = ax′ bx′ + ay′ by′ + az ′ bz ′ ,
i=x′ ,y ′ ,z ′
14
[32]
15
Some professional mathematicians now write a superscript ai in certain cases in place
of a subscript ai , where the superscript bears some additional semantics [67, “Einstein
notation,” 05:36, 10 February 2008]. Scientists and engineers however tend to prefer
Einstein’s original, subscript-only notation.
15.4. NOTATION 419
which is (15.23), except that Einstein’s form (15.25) expresses it more suc-
cinctly. Likewise,
a × b = ı̂(ai+1 bi−1 − bi+1 ai−1 ) (15.26)
is (15.24)—although an experienced applied mathematician would probably
apply the Levi-Civita epsilon of § 15.4.3, below, to further abbreviate this
last equation to the form of (15.27) before presenting it.
Einstein’s summation convention is also called the Einstein notation, a
term sometimes taken loosely to include also the Kronecker delta and Levi-
Civita epsilon of § 15.4.3.
What is important to understand about Einstein’s summation conven-
tion is that, in and of itself, it brings no new mathematics. It is rather a
notational convenience.16 It asks a reader to regard a repeated index P like
the i in “ai bi ” as a dummy index (§ 2.3) and thus to read “ai bi ” as “ i ai bi .”
It does not magically create a summation where none existed; it just hides
the summation sign to keep it from cluttering the page. It is the kind of
notational trick an accountant
P might appreciate. Under the convention, the
summational operator i is implied not written, but the operator is still
there. Admittedly confusing on first encounter, the convention’s utility and
charm are felt after only a little practice.
Incidentally, nothing requires you to invoke Einstein’s summation con-
vention everywhere and for all purposes. You can waive the convention,
writing the summation symbol out explicitly whenever you like.17 In con-
texts outside vector analysis, to invoke the convention at all may make little
sense. Nevertheless, you should indeed learn the convention—if only because
you must learn it to understand the rest of this chapter—but once having
learned it you should naturally use it only where it actually serves to clarify.
Fortunately, in vector work, it often does just that.
Quiz:18 if δij is the Kronecker delta of § 11.2, then what does the sym-
bol δii represent where Einstein’s summation convention is in force?
15.4.3 The Kronecker delta and the Levi-Civita epsilon

Einstein’s summation convention expresses the dot product (15.25) neatly
but, as we have seen in (15.26), does not by itself wholly avoid unseemly
repetition in the cross product. The Levi-Civita epsilon 19 ǫijk mends this,
16
[65, “Einstein summation”]
17
[51]
18
[32]
19
Also called the Levi-Civita symbol, tensor, or permutor. For native English speakers
who do not speak Italian, the “ci” in Levi-Civita’s name is pronounced as the “chi” in
rendering the cross-product as

a × b = ǫijk ı̂aj bk , (15.27)
where20

+1
 if (i, j, k) = (x′ , y ′ , z ′ ), (y ′ , z ′ , x′ ) or (z ′ , x′ , y ′ );
ǫijk ≡ −1 if (i, j, k) = (x′ , z ′ , y ′ ), (y ′ , x′ , z ′ ) or (z ′ , y ′ , x′ ); (15.28)

0 otherwise [for instance if (i, j, k) = (x′ , x′ , y ′ )].

In the language of § 11.6, the Levi-Civita epsilon quantifies parity. (Chap-

ters 11 and 14 did not use it, but the Levi-Civita notation applies in any
number of dimensions, not only three as in the present chapter. In this
more general sense the Levi-Civita is the determinant of the permutor whose
ones hold the indicated positions—which is a formal way of saying that it’s
a + sign for even parity and a − sign for odd. For instance, in the four-
dimensional, 4 × 4 case ǫ1234 = 1 whereas ǫ1243 = −1: refer to §§ 11.6,
11.7.1 and 14.1. Table 15.1, however, as the rest of this section and chapter,
concerns the three-dimensional case only.)
Technically, the Levi-Civita epsilon and Einstein’s summation conven-
tion are two separate, independent things, but a canny reader takes the
Levi-Civita’s appearance as a hint that Einstein’s convention is probably in
force, as in (15.27). The two tend to go together.21
The Levi-Civita epsilon ǫijk relates to the Kronecker delta δij of § 11.2
approximately as the cross product relates to the dot product. Both delta
and epsilon find use in vector work. For example, one can write (15.25)
alternately in the form
a · b = δij ai bj .
Table 15.1 lists several relevant properties,22 each as with Einstein’s
summation convention in force.23 Of the table’s several properties, the
“children.”
20
[50, “Levi-Civita permutation symbol”]
21
The writer has heard the apocryphal belief expressed that the letter ǫ, a Greek e,
stood in this context for “Einstein.” As far as the writer knows, ǫ is merely the letter
after δ, which represents the name of Paul Dirac—though the writer does not claim his
indirected story to be any less apocryphal than the other one (the capital letter ∆ has
a point on top that suggests the pointy nature of the Dirac delta of Fig. 7.10, which
makes for yet another plausible story). In any event, one sometimes hears Einstein’s
summation convention, the Kronecker delta and the Levi-Civita epsilon together referred
to as “the Einstein notation,” which though maybe not quite terminologically correct is
hardly incorrect enough to argue over and is clear enough in practice.
22
[51]
23
The table incidentally answers § 15.4.2’s quiz.
15.4. NOTATION 421
Table 15.1: Properties of the Kronecker delta and the Levi-Civita epsilon,
with Einstein’s summation convention in force.
δjk = δkj
δij δjk = δik
δii = 3
δjk ǫijk = 0
δnk ǫijk = ǫijn
ǫijk = ǫjki = ǫkij = −ǫikj = −ǫjik = −ǫkji
ǫijk ǫijk = 6
ǫijn ǫijk = 2δnk
ǫimn ǫijk = δmj δnk − δmk δnj
property that ǫimn ǫijk = δmj δnk − δmk δnj is proved by observing that, in
the case that i = x′ , either (j, k) = (y ′ , z ′ ) or (j, k) = (z ′ , y ′ ), and also
either (m, n) = (y ′ , z ′ ) or (m, n) = (z ′ , y ′ ); and similarly in the cases that
i = y ′ and i = z ′ (more precisely, in each case the several indices can take
any values, but combinations other than the ones listed drive ǫimn or ǫijk ,
or both, to zero, thus contributing nothing to the sum). This implies that
either (j, k) = (m, n) or (j, k) = (n, m)—which, when one takes parity into
account, is exactly what the property in question asserts. The property that
ǫijn ǫijk = 2δnk is proved by observing that, in any given term of the Einstein
sum, i is either x′ or y ′ or z ′ and that j is one of the remaining two, which
leaves the third to be shared by both k and n. The factor 2 appears because,
for k = n = x′ , an (i, j) = (y ′ , z ′ ) term and an (i, j) = (z ′ , y ′ ) term both
contribute positively to the sum; and similarly for k = n = y ′ and again for
k = n = z′ .
Unfortunately, the last paragraph likely makes sense to few who do not
already know what it means. A concrete example helps. Consider the
compound product c × (a × b). In this section’s notation and with the use
of (15.27), the compound product is
c × (a × b) = c × (ǫijk ı̂aj bk )
= ǫmni m̂cn (ǫijk ı̂aj bk )i
= ǫmni ǫijk m̂cn aj bk
= ǫimn ǫijk m̂cn aj bk
= (δmj δnk − δmk δnj )m̂cn aj bk
= δmj δnk m̂cn aj bk − δmk δnj m̂cn aj bk
= ̂ck aj bk − k̂cj aj bk
= (̂aj )(ck bk ) − (k̂bk )(cj aj ).
That is, in light of (15.25),
c × (a × b) = a(c · b) − b(c · a), (15.29)
a useful vector identity. Written without the benefit of Einstein’s summation

convention, the example’s central step would have been
X
c × (a × b) = ǫimn ǫijk m̂cn aj bk
i,j,k,m,n
X
= (δmj δnk − δmk δnj )m̂cn aj bk ,
j,k,m,n
15.4. NOTATION 423
which makes sense if you think about it hard enough,24 and justifies the
24
If thinking about it hard enough does not work, then here it is in interminable detail:
X
ǫimn ǫijk f (j, k, m, n)
i,j,k,m,n
= ǫx′ y ′ z ′ ǫx′ y ′ z ′ f (y ′ , z ′ , y ′ , z ′ ) + ǫx′ y ′ z ′ ǫx′ z ′ y ′ f (y ′ , z ′ , z ′ , y ′ )

+ ǫx′ z ′ y ′ ǫx′ y ′ z ′ f (z ′ , y ′ , y ′ , z ′ ) + ǫx′ z ′ y ′ ǫx′ z ′ y ′ f (z ′ , y ′ , z ′ , y ′ )
+ ǫy ′ z ′ x′ ǫy ′ z ′ x′ f (z ′ , x′ , z ′ , x′ ) + ǫy ′ z ′ x′ ǫy ′ x′ z ′ f (z ′ , x′ , x′ , z ′ )
+ ǫy ′ x′ z ′ ǫy ′ z ′ x′ f (x′ , z ′ , z ′ , x′ ) + ǫy ′ x′ z ′ ǫy ′ x′ z ′ f (x′ , z ′ , x′ , z ′ )
+ ǫz ′ x′ y ′ ǫz ′ x′ y ′ f (x′ , y ′ , x′ , y ′ ) + ǫz ′ x′ y ′ ǫz ′ y ′ x′ f (x′ , y ′ , y ′ , x′ )
+ ǫz ′ y ′ x′ ǫz ′ x′ y ′ f (y ′ , x′ , x′ , y ′ ) + ǫz ′ y ′ x′ ǫz ′ y ′ x′ f (y ′ , x′ , y ′ , x′ )
= f (y ′ , z ′ , y ′ , z ′ ) − f (y ′ , z ′ , z ′ , y ′ ) − f (z ′ , y ′ , y ′ , z ′ ) + f (z ′ , y ′ , z ′ , y ′ )
+ f (z ′ , x′ , z ′ , x′ ) − f (z ′ , x′ , x′ , z ′ ) − f (x′ , z ′ , z ′ , x′ ) + f (x′ , z ′ , x′ , z ′ )
+ f (x′ , y ′ , x′ , y ′ ) − f (x′ , y ′ , y ′ , x′ ) − f (y ′ , x′ , x′ , y ′ ) + f (y ′ , x′ , y ′ , x′ )
f (y ′ , z ′ , y ′ , z ′ ) + f (z ′ , x′ , z ′ , x′ ) + f (x′ , y ′ , x′ , y ′ )
ˆ
=
+ f (z ′ , y ′ , z ′ , y ′ ) + f (x′ , z ′ , x′ , z ′ ) + f (y ′ , x′ , y ′ , x′ )
˜
− f (y ′ , z ′ , z ′ , y ′ ) + f (z ′ , x′ , x′ , z ′ ) + f (x′ , y ′ , y ′ , x′ )
ˆ
+ f (z ′ , y ′ , y ′ , z ′ ) + f (x′ , z ′ , z ′ , x′ ) + f (y ′ , x′ , x′ , y ′ )
˜
f (y ′ , z ′ , y ′ , z ′ ) + f (z ′ , x′ , z ′ , x′ ) + f (x′ , y ′ , x′ , y ′ )
ˆ
=
+ f (z ′ , y ′ , z ′ , y ′ ) + f (x′ , z ′ , x′ , z ′ ) + f (y ′ , x′ , y ′ , x′ )
+ f (x′ , x′ , x′ , x′ ) + f (y ′ , y ′ , y ′ , y ′ ) + f (z ′ , z ′ , z ′ , z ′ )
˜
− f (y ′ , z ′ , z ′ , y ′ ) + f (z ′ , x′ , x′ , z ′ ) + f (x′ , y ′ , y ′ , x′ )
ˆ
+ f (z ′ , y ′ , y ′ , z ′ ) + f (x′ , z ′ , z ′ , x′ ) + f (y ′ , x′ , x′ , y ′ )
+ f (x′ , x′ , x′ , x′ ) + f (y ′ , y ′ , y ′ , y ′ ) + f (z ′ , z ′ , z ′ , z ′ )
˜
X
= (δmj δnk − δmk δnj )f (j, k, m, n).
j,k,m,n
That is for the property that ǫimn ǫijk = δmj δnk −δmk δnj . For the property that ǫijn ǫijk =
2δnk , the corresponding calculation is
X
ǫijn ǫijk f (k, n)
i,j,k,n
= ǫy ′ z ′ x′ ǫy ′ z ′ x′ f (x′ , x′ ) + ǫz ′ y ′ x′ ǫz ′ y ′ x′ f (x′ , x′ )
+ ǫz ′ x′ y ′ ǫz ′ x′ y ′ f (y ′ , y ′ ) + ǫx′ z ′ y ′ ǫx′ z ′ y ′ f (y ′ , y ′ )
+ ǫx′ y ′ z ′ ǫx′ y ′ z ′ f (z ′ , z ′ ) + ǫy ′ x′ z ′ ǫy ′ x′ z ′ f (z ′ , z ′ )
= f (x′ , x′ ) + f (x′ , x′ ) + f (y ′ , y ′ ) + f (y ′ , y ′ ) + f (z ′ , z ′ ) + f (z ′ , z ′ )
2 f (x′ , x′ ) + f (y ′ , y ′ ) + f (z ′ , z ′ )
ˆ ˜
=
X
= 2 δnk f (k, n).
k,n
For the property that ǫijk ǫijk = 6,

X
ǫijk ǫijk = ǫ2x′ y ′ z ′ + ǫ2y ′ z ′ x′ + ǫ2z ′ x′ y ′ + ǫ2x′ z ′ y ′ + ǫ2y ′ x′ z ′ + ǫ2z ′ y ′ x′ = 6.
i,j,k
table’s claim that ǫimn ǫijk = δmj δnk − δmk δnj . (Notice that the compound
Kronecker operator δmj δnk includes nonzero terms for the case that j = k =
m = n = x′ , for the case that j = k = m = n = y ′ and for the case that
j = k = m = n = z ′ , whereas the compound Levi-Civita operator ǫimn ǫijk
does not. However, the compound Kronecker operator −δmk δnj includes
canceling terms for these same three cases. This is why the table’s claim is
valid as written.)
To belabor the topic further here would serve little purpose. The reader
who does not feel entirely sure that he understands what is going on might
work out the table’s several properties with his own pencil, in something like
the style of the example, until he is satisfied that he adequately understands
the several properties and their correct use.
Section 16.7 will refine the notation for use when derivatives with respect
to angles come into play but, before leaving the present section, we might
pause for a moment to appreciate (15.29) in the special case that b = c = n̂:
−n̂ × (n̂ × a) = a − n̂(n̂ · a). (15.30)
The difference a − n̂(n̂·a) evidently projects a vector a onto the plane whose
unit normal is n̂. Equation (15.30) reveals that the double cross product
−n̂ × (n̂ × a) projects the same vector onto the same plane. Figure 15.6
illustrates.
15.5 Algebraic identities

Vector algebra is not in principle very much harder than scalar algebra is, but
with three distinct types of product it has more rules controlling the way its
products and sums are combined. Table 15.2 lists several of these.25,26 Most
of the table’s identities are plain by the formulas (15.9), (15.21) and (15.22)
respectively for the scalar, dot and cross products, and two were proved
as (15.29) and (15.30). The remaining identity is proved in the notation of
§ 15.4 as
ǫijk ci aj bk = ǫijk ci aj bk = ǫkij ck ai bj = ǫjki cj ak bi
= ǫijk ci aj bk = ǫijk ai bj ck = ǫijk bi cj ak
= c · (ǫijk ı̂aj bk ) = a · (ǫijk ı̂bj ck ) = b · (ǫijk ı̂cj ak ).
It is precisely to encapsulate such interminable detail that we use the Kronecker delta,
the Levi-Civita epsilon and the properties of Table 15.1.
25
[60, Appendix II][28, Appendix A]
26
Nothing in any of the table’s identities requires the vectors involved to be real. The
table is equally as valid when vectors are complex.
15.5. ALGEBRAIC IDENTITIES 425
Figure 15.6: A vector projected onto a plane.
n̂(n̂ · a)
n̂
−n̂ × (n̂ × a)
= a − n̂(n̂ · a)
Table 15.2: Algebraic vector identities.
ψa = ı̂ψai a · b ≡ ai bi a × b ≡ ǫijk ı̂aj bk

a∗ · a = |a|2 (ψ)(a + b) = ψa + ψb
b·a = a·b b × a = −a × b
c · (a + b) = c·a+c·b c × (a + b) = c × a + c × b
a · (ψb) = (ψ)(a · b) a × (ψb) = (ψ)(a × b)
c · (a × b) = a · (b × c) = b · (c × a)
c × (a × b) = a(c · b) − b(c · a)
−n̂ × (n̂ × a) = a − n̂(n̂ · a)
That is,
c · (a × b) = a · (b × c) = b · (c × a). (15.31)
Besides the several vector identities, the table also includes the three vector
products in Einstein notation.27
Each definition and identity of Table 15.2 is invariant under reorientation
of axes.
15.6 Isotropy
A real,28 three-dimensional coordinate system29 (α; β; γ) is isotropic at a
point r = r1 if and only if
β̂(r1 ) · γ̂(r1 ) = 0,
γ̂(r1 ) · α̂(r1 ) = 0, (15.32)
α̂(r1 ) · β̂(r1 ) = 0,
and
∂r ∂r ∂r

∂α =
= . (15.33)
r=r1 ∂β r=r1 ∂γ r=r1
That is, a three-dimensional system is isotropic if its three coordinates ad-

vance locally at right angles to one another but at the same rate.
Of the three basic three-dimensional coordinate systems—indeed, of all
the three-dimensional coordinate systems this book treats—only the rect-
angular is isotropic according to (15.32) and (15.33).30 Isotropy admittedly
would not be a very interesting property if that were all there were to it.
However, there is also two-dimensional isotropy, more interesting because it
arises oftener.
27
If the reader’s native language is English, then he is likely to have heard of the
unfortunate “back cab rule,” which actually is not a rule but an unhelpful mnemonic for
one of Table 15.2’s identities. The mnemonic is mildly orthographically clever but, when
learned, significantly impedes real understanding of the vector. The writer recommends
that the reader forget the rule if he has heard of it for, in mathematics, spelling-based
mnemonics are seldom if ever a good idea.
28
The reader is reminded that one can licitly express a complex vector in a real basis.
29
This chapter’s footnote 31 and Ch. 16’s footnote 21 explain the usage of semicolons
as coordinate delimiters.
30
Whether it is even possible to construct an isotropic, nonrectangular coordinate sys-
tem in three dimensions is a question we shall leave to the professional mathematician.
The author has not encountered such a system.
15.7. PARABOLIC COORDINATES 427
A real, two-dimensional coordinate system (α; β) is isotropic at a point

ργ = ργ1 if and only if
α̂(ργ1 ) · β̂(ργ1 ) = 0 (15.34)
and γ γ
∂ρ ∂ρ
∂α γ γ = ∂β γ γ , (15.35)

ρ =ρ 1 ρ =ρ 1
where ργ = α̂α + β̂β represents position in the α-β plane. (If the α-β plane
happens to be the x-y plane, as is often the case, then ργ = ρz = ρ and per
eqn. 3.20 one can omit the superscript.) The two-dimensional rectangular
system (x, y) naturally is isotropic. Because |∂ρ/∂φ| = (ρ) |∂ρ/∂ρ| the
standard two-dimensional cylindrical system (ρ; φ) as such is nonisotropic,
but the change of coordinate
ρ
λ ≡ ln , (15.36)
ρo
where ρo is some arbitrarily chosen reference radius, converts the system

straightforwardly into the logarithmic cylindrical system (λ; φ) which is iso-
tropic everywhere in the plane except at the origin ρ = 0. Further two-
dimensionally isotropic coordinate systems include the parabolic system of
§ 15.7.2, to follow.
15.7 Parabolic coordinates

Scientists and engineers find most spatial-geometrical problems they en-
counter in practice to fall into either of two categories. The first category
comprises problems of simple geometry conforming to any one of the three
basic coordinate systems—rectangular, cylindrical or spherical. The second
category comprises problems of complicated geometry, analyzed in the rect-
angular system not because the problems’ geometries fit that system but
rather because they fit no system and thus give one little reason to depart
from the rectangular. One however occasionally encounters problems of a
third category, whose geometries are simple but, though simple, neverthe-
less fit none of the three basic coordinate systems. Then it may fall to the
scientist or engineer to devise a special coordinate system congenial to the
problem.
This section will treat the parabolic coordinate systems which, besides
being arguably the most useful of the various special systems, serve as good
examples of the kind of special system a scientist or engineer might be
called upon to devise. The two three-dimensional parabolic systems are the
parabolic cylindrical system (σ, τ, z) of § 15.7.4 and the circular paraboloidal
system31 (η; φ, ξ) of § 15.7.5, where the angle φ and the length z are familiar
to us but σ, τ , η and ξ—neither angles nor lengths but root-lengths (that
is, coordinates having dimensions of [length]1/2 )—are new.32 Both three-
dimensional parabolic systems derive from the two-dimensional parabolic
system (σ, τ ) of § 15.7.2.33
However, before handling any parabolic system we ought formally to
introduce the parabola itself, next.
15.7.1 The parabola

Parabolic coordinates are based on a useful geometrical curve called the
parabola, which many or most readers will have met long before opening
this book’s covers. The parabola, simple but less obvious than the circle,
may however not be equally well known to all readers, and even readers al-
ready acquainted with it might appreciate a reëxamination. This subsection
reviews the parabola.
Given a point, called the focus, and a line, called the directrix,34 plus the
plane in which the focus and the directrix both lie, the associated parabola
is that curve which lies in the plane everywhere equidistant from both focus
and directrix.35 See Fig. 15.7.
Referring to the figure, if rectangular coordinates are established such
that x̂ and ŷ lie in the plane, that the parabola’s focus lies at (x, y) = (0, k),
and that the equation y = k − σ 2 describes the parabola’s directrix, then
the equation
x2 + (y − k)2 = (y − k + σ 2 )2
31
The reader probably will think nothing of it now, but later may wonder why the
circular paraboloidal coordinates are (η; φ, ξ) rather than (ξ; φ, η) or (η, ξ; φ). The peculiar
ordering is to honor the right-hand rule (§ 3.3 and eqn. 15.19), since η̂ × ξ̂ = −φ̂ rather
than +φ̂. See § 15.7.5. (Regarding the semicolon “;” delimiter, it doesn’t mean much.
This book arbitrarily uses a semicolon when the following coordinate happens to be an
angle, which helps to distinguish rectangular coordinates from cylindrical from spherical.
Admittedly, such a notational convention ceases to help much when parabolic coordinates
arrive, but we shall continue to use it for inertia’s sake. See also Ch. 16’s footnote 21.)
32
The letters σ, τ , η and ξ are available letters this section happens to use, not neces-
sarily standard parabolic symbols. See Appendix B.
33
[67, “Parabolic coordinates,” 09:59, 19 July 2008]
34
Whether the parabola’s definition ought to forbid the directrix to pass through the
focus is a stylistic question this book will leave unanswered.
35
[55, § 12-1]
Figure 15.7: The parabola.
b
a
a
σ2
evidently expresses the equidistance rule of the parabola’s definition. Solving

for y − k and then, from that solution, for y, we have that
x2 σ2

y= 2 + k− . (15.37)
2σ 2
With the definitions that
1
µ≡ ,
2σ 2
(15.38)
σ2
κ≡k− ,
2
given which
1
σ2 = ,
2µ
(15.39)
1
k =κ+ ,
4µ
eqn. (15.37) becomes
y = µx2 + κ. (15.40)
Equations fitting the general form (15.40) arise extremely commonly in ap-
plications. To choose a particularly famous example, the equation that
describes a projectile’s flight in the absence of air resistance fits the form.
Any equation that fits the form can be plotted as a parabola, which for
example is why projectiles fly in parabolic arcs.
Observe that the parabola’s definition does not actually require the di-
rectrix to be ŷ-oriented: the directrix can be x̂-oriented or, indeed, oriented
any way (though naturally in that case eqns. 15.37 and 15.40 would have
to be modified). Observe also the geometrical fact that the parabola’s track
necessarily bisects the angle between the two line segments labeled “a” in
Fig. 15.7. One of the consequences of this geometrical fact—a fact it seems
better to let the reader visualize and ponder than to try to justify in so many
words36 —is that a parabolic mirror reflects precisely37 toward its focus all
light rays that arrive perpendicularly to its directrix (which for instance is
why satellite dish antennas have parabolic cross-sections).
15.7.2 Parabolic coordinates in two dimensions

Parabolic coordinates are most easily first explained in the two-dimensional
case that z = 0. In two dimensions, the parabolic coordinates (σ, τ ) repre-
sent the point in the x-y plane that lies equidistant
• from the line y = −σ 2 ,
• from the line y = +τ 2 , and
• from the point ρ = 0,
where the parameter k of § 15.7.1 has been set to k = 0. Figure 15.8

depicts the construction described. In the figure are two dotted curves, one
of which represents the point’s parabolic track if σ were varied while τ were
held constant and the other of which represents the point’s parabolic track
if τ were varied while σ were held constant. Observe according to § 15.7.1’s
bisection finding that each parabola necessarily bisects the angle between
two of the three line segments labeled a in the figure. Observe further
that the two angles’ sum is the straight angle 2π/2, from which one can
36
If it helps nevertheless, some words: Consider that the two line segments labeled a
in the figure run in the directions of increasing distance respectively from the focus and
from the directrix. If you want to draw away from the directrix at the same rate as you
draw away from the focus, thus maintaining equal distances, then your track cannot but
exactly bisect the angle between the two segments.
Once you grasp the idea, the bisection is obvious, though to grasp the idea can take
some thought.
To bisect a thing, incidentally—if the context has not already made the meaning plain—
is to divide the thing at its middle into two equal parts.
37
Well, actually, physically, the ray model of light implied here is valid only insofar as
λ ≪ σ 2 , where λ represents the light’s characteristic wavelength. Also, regardless of λ,
the ray model breaks down in the immediate neighborhood of the mirror’s focus. Such
wave-mechanical considerations are confined to a footnote not because they were untrue
but rather because they do not concern the present geometrical discussion. Insofar as rays
are concerned, the focusing is precise.
Figure 15.8: Locating a point in two dimensions by parabolic construction.
a
τ2
ρ=0 b
a
a
σ2
conclude, significantly, that the two parabolas cross precisely at right angles
to one another.
Figure 15.9 lays out the parabolic coordinate grid. Notice in the figure
that one of the grid’s several cells is subdivided at its quarter-marks for
illustration’s sake, to show how one can subgrid at need to locate points like,
for example, (σ, τ ) = ( 72 , − 94 ) visually. (That the subgrid’s cells approach
square shape implies that the parabolic system is isotropic, a significant fact
§ 15.7.3 will demonstrate formally.)
Using the Pythagorean theorem, one can symbolically express the equi-
distant construction rule above as
a = σ 2 + y = τ 2 − y,
(15.41)
a2 = ρ2 = x2 + y 2 .
From the first line of (15.41),
τ 2 − σ2
y= . (15.42)
2
On the other hand, combining the two lines of (15.41),
(σ 2 + y)2 = x2 + y 2 = (τ 2 − y)2 ,
or, subtracting y 2 ,
σ 4 + 2σ 2 y = x2 = τ 4 − 2τ 2 y.
Substituting (15.42)’s expression for y,
x2 = (στ )2 .
Figure 15.9: The parabolic coordinate grid in two dimensions.
σ=0 ±1
±2
±3
±3
±2
τ =0 ±1
That either x = +στ or x = −στ would satisfy this equation. Arbitrarily

choosing the + sign gives us that
x = στ. (15.43)
Also, since ρ2 = x2 + y 2 , (15.42) and (15.43) together imply that
τ 2 + σ2
ρ= . (15.44)
2
Combining (15.42) and (15.44) to isolate σ 2 and τ 2 yields
σ 2 = ρ − y,
(15.45)
τ 2 = ρ + y.
15.7.3 Properties
The derivatives of (15.43), (15.42) and (15.44) are
dx = σ dτ + τ dσ,
dy = τ dτ − σ dσ, (15.46)
dρ = τ dτ + σ dσ.
Solving the first two lines of (15.46) simultaneously for dσ and dτ and then
collapsing the resultant subexpression τ 2 + σ 2 per (15.44) yields
τ dx − σ dy
dσ = ,
2ρ
(15.47)
σ dx + τ dy
dτ = ,
2ρ
from which it is apparent that
x̂τ − ŷσ
σ̂ = √ ,
τ 2 + σ2
x̂σ + ŷτ
τ̂ = √ ;
τ 2 + σ2
or, collapsing again per (15.44), that
x̂τ − ŷσ
σ̂ = √ ,
2ρ
(15.48)
x̂σ + ŷτ
τ̂ = √ ,
2ρ
of which the dot product
σ̂ · τ̂ = 0 if ρ 6= 0 (15.49)
is null, confirming our earlier finding that the various grid parabolas cross
always at right angles to one another. Solving (15.48) simultaneously for x̂
and ŷ then produces
τ̂ σ + σ̂τ
x̂ = √ ,
2ρ
(15.50)
τ̂ τ − σ̂σ
ŷ = √ .
2ρ
One can express an infinitesimal change in position in the plane as
dρ = x̂ dx + ŷ dy
= x̂(σ dτ + τ dσ) + ŷ(τ dτ − σ dσ)
= (x̂τ − ŷσ) dσ + (x̂σ + ŷτ ) dτ,
in which (15.46) has expanded the differentials and from which
∂ρ
= x̂τ − ŷσ,
∂σ
∂ρ
= x̂σ + ŷτ,
∂τ
Table 15.3: Parabolic coordinate properties.
x = στ τ̂ σ + σ̂τ
x̂ = √
τ 2 − σ2 2ρ
y = τ̂ τ − σ̂σ
2 ŷ = √
τ + σ2
2 2ρ
ρ = x̂τ − ŷσ
2 σ̂ = √
ρ = x + y2
2 2 2ρ
σ2 = ρ − y x̂σ + ŷτ
τ̂ = √
τ2 = ρ + y 2ρ
σ̂ × τ̂ = ẑ
· τ̂ = 0
σ̂
∂ρ
= ∂ρ

∂σ ∂τ
and thus

∂ρ ∂ρ
= . (15.51)
∂σ ∂τ
Equations (15.49) and (15.51) respectively meet the requirements (15.34)

and (15.35), implying that the two-dimensional parabolic coordinate system
is isotropic except at ρ = 0.
Table 15.3 summarizes, gathering parabolic coordinate properties from
this subsection and § 15.7.2.
15.7.4 The parabolic cylindrical coordinate system

Two-dimensional parabolic coordinates are trivially extended to three di-
mensions by adding a z coordinate, thus constituting the parabolic cylindri-
cal coordinate system (σ, τ, z). The surfaces of constant σ and of constant τ
in this system are parabolic cylinders (and the surfaces of constant z natu-
rally are planes). All the properties of Table 15.3 apply. Observe however
that the system is isotropic only in two dimensions not three.
The orthogonal parabolic cylindrical basis is [σ̂ τ̂ ẑ].
Table 15.4: Circular paraboloidal coordinate properties.
ρ = ηξ ξ̂η + η̂ξ
ρ̂ = √
ξ 2 − η2 2r
z =
2 ξ̂ξ − η̂η
ẑ = √
ξ 2 + η2 2r
r =
2 ρ̂ξ − ẑη
η̂ = √
r 2 = ρ2 + z 2 = x2 + y 2 + z 2 2r
η2 = r − z ρ̂η + ẑξ
ξ̂ = √
ξ2 = r + z 2r
η̂ × ξ̂ = −φ̂
η̂ · ξ̂ = 0

∂r
= ∂r

∂η ∂ξ
15.7.5 The circular paraboloidal coordinate system
Sometimes one would like to extend the parabolic system to three dimensions
by adding an azimuth φ rather than a height z. This is possible, but then
one tends to prefer the parabolas, foci and directrices of Figs. 15.8 and 15.9
to run in the ρ-z plane rather than in the x-y. Therefore, one defines the
coordinates η and ξ to represent in the ρ-z plane what the letters σ and τ
have represented in the x-y. The properties of Table 15.4 result, which are
just the properties of Table 15.3 with coordinates changed. The system is
the circular paraboloidal system (η; φ, ξ).
The surfaces of constant η and of constant ξ in the circular paraboloidal
system are paraboloids, parabolas rotated about the z axis (and the surfaces
of constant φ are planes, or half planes if you like, just as in the cylindrical
system). Like the parabolic cylindrical system, the circular paraboloidal
system too is isotropic in two dimensions.
Notice that, given the usual definition of the φ̂ unit basis vector, η̂ × ξ̂ =
−φ̂ rather than +φ̂ as one might first guess. The correct, right-handed
sequence of the orthogonal circular paraboloidal basis therefore would be
[η̂ φ̂ ξ̂].38
This concludes the present chapter on the algebra of vector analysis.
Chapter 16, next, will venture hence into the larger and even more interest-
ing realm of vector calculus.
38
See footnote 31.
Chapter 16
Vector calculus
Chapter 15 has introduced the algebra of the three-dimensional geometrical

vector. Like the scalar, the vector is a continuous quantity and as such has
not only an algebra but also a calculus. This chapter develops the calculus
of the vector.
16.1 Fields and their derivatives

A scalar quantity σ(t) or vector quantity f (t) whose value varies over time is
“a function of time t.” We can likewise call a scalar quantity1 ψ(r) or vector
quantity a(r) whose value varies over space “a function of position r,” but
there is a special, alternate name for such a quantity. We call it a field.
A field is a quantity distributed over space or, if you prefer, a function
in which spatial position serves as independent variable. Air pressure p(r) is
an example of a scalar field, whose value at a given location r has amplitude
but no direction. Wind velocity2 q(r) is an example of a vector field, whose
value at a given location r has both amplitude and direction. These are
typical examples. Tactically, a vector field can be thought of as composed
of three scalar fields
q(r) = x̂qx (r) + ŷqy (r) + ẑqz (r);

1
This ψ(r) is unrelated to the Tait-Bryan and Euler roll angles ψ of § 15.1, an unfortu-
nate but tolerable overloading of the Greek letter ψ in the conventional notation of vector
analysis. In the unlikely event of confusion, you can use an alternate letter like η for the
roll angle. See Appendix B.
2
As § 15.3, this section also uses the letter q for velocity in place of the conventional v
[6, § 18.4], which it needs for another purpose.
437
438 CHAPTER 16. VECTOR CALCULUS
but, since
q(r) = x̂′ qx′ (r) + ŷ′ qy′ (r) + ẑ′ qz ′ (r)
for any orthogonal basis [x′ y′ z′ ] as well, the specific scalar fields qx (r),
qy (r) and qz (r) are no more essential to the vector field q(r) than the specific
scalars bx , by and bz are to a vector b. As we said, the three components
come tactically; typically, such components are uninteresting in themselves.
The field q(r) as a whole is the interesting thing.
Scalar and vector fields are of utmost use in the modeling of physical
phenomena.
As one can take the derivative dσ/dt or df /dt with respect to time t
of a function σ(t) or f (t), one can likewise take the derivative with respect
to position r of a field ψ(r) or a(r). However, derivatives with respect to
position create a notational problem, for it is not obvious what symbology
like dψ/dr or da/dr would actually mean. The notation dσ/dt means “the
rate of σ as time t advances,” but if the notation dψ/dr likewise meant “the
rate of ψ as position r advances” then it would necessarily prompt one to
ask, “advances in which direction?” The notation offers no hint. In fact
dψ/dr and da/dr mean nothing very distinct in most contexts and we shall
avoid such notation. If we will speak of a field’s derivative with respect to
position r then we shall be more precise.
Section 15.2 has given the vector three distinct kinds of product. This
section gives the field no fewer than four distinct kinds of derivative: the
directional derivative; the gradient; the divergence; and the curl.3
So many derivatives bring the student a conceptual difficulty one could
call “the caveman problem.” Imagine a caveman. Suppose that you tried
to describe to the caveman a house or building of more than one floor. He
might not understand. You and I who already grasp the concept of upstairs
and downstairs do not find a building of two floors, or three or even thirty,
especially hard to envision, but our caveman is used to thinking of the ground
and the floor as more or less the same thing. To try to think of upstairs
and downstairs might confuse him with partly false images of sitting in a
tree or of clambering onto (and breaking) the roof of his hut. “There are
many trees and antelopes but only one sky and floor. How can one speak of
many skies or many floors?” The student’s principal conceptual difficulty
with the several vector derivatives is of this kind.
3
Vector veterans may notice that the Laplacian is not listed. This is not because the
Laplacian were uninteresting but rather because the Laplacian is actually a second-order
derivative—a derivative of a derivative. We shall address the Laplacian in § 16.4.
16.1. FIELDS AND THEIR DERIVATIVES 439
16.1.1 The ∇ operator

Consider a vector
a = x̂ax + ŷay + ẑaz .
Then consider a “vector”
c = x̂[Tuesday] + ŷ[Wednesday] + ẑ[Thursday].
If you think that the latter does not look very much like a vector, then the
writer thinks as you do, but consider:
c · a = [Tuesday]ax + [Wednesday]ay + [Thursday]az .
The writer does not know how to interpret a nonsensical term like
“[Tuesday]ax ” any more than the reader does, but the point is that c behaves
as though it were a vector insofar as vector operations like the dot product
are concerned. What matters in this context is not that c have amplitude
and direction (it has neither) but rather that it have the three orthonormal
components it needs to participate formally in relevant vector operations.
It has these. That the components’ amplitudes seem nonsensical is beside
the point. Maybe there exists a model in which “[Tuesday]” knows how to
operate on a scalar like ax . (Operate on? Yes. Nothing in the dot product’s
definition requires the component amplitudes of c to multiply those of a.
Multiplication is what the component amplitudes of true vectors do, but c
is not a true vector, so “[Tuesday]” might do something to ax other than
to multiply it. Section 16.1.2 elaborates the point.) If there did exist such
a model, then the dot product c · a could be licit in that model. As if this
were not enough, the cross product c × a too could be licit in that model,
composed according to the usual rule for cross products. The model might
allow it. The dot and cross products in and of themselves do not forbid it.
Now consider a “vector”
∂ ∂ ∂
∇ = x̂ + ŷ + ẑ . (16.1)
∂x ∂y ∂z
This ∇ is not a true vector any more than c is, maybe, but if we treat it as
one then we have that
∂ax ∂ay ∂az
∇·a= + + .
∂x ∂y ∂z
Such a dot product might or might not prove useful; but, unlike the terms
in the earlier dot product, at least we know what this one’s terms mean.
Well, days of the week, partial derivatives, ersatz vectors—it all seems
rather abstract. What’s the point? The answer is that there wouldn’t be
any point if the only nonvector “vectors” in question were of c’s nonsensical
kind. The operator ∇ however shares more in common with a true vector
than merely having x, y and z components; for, like a true vector, the
operator ∇ is amenable to having its axes reoriented by (15.1), (15.2), (15.7)
and (15.8). This is easier to see at first with respect the true vector a, as
follows. Consider rotating the x and y axes through an angle φ about the z
axis. There ensues
a = x̂ax + ŷay + ẑaz

= (x̂′ cos φ − ŷ′ sin φ)(ax′ cos φ − ay′ sin φ)
+ (x̂′ sin φ + ŷ′ cos φ)(ax′ sin φ + ay′ cos φ) + ẑ′ az ′
= x̂′ [ax′ cos2 φ − ay′ cos φ sin φ + ax′ sin2 φ + ay′ cos φ sin φ]
+ ŷ′ [−ax′ cos φ sin φ + ay′ sin2 φ + ax′ cos φ sin φ + ay′ cos2 φ]
+ ẑ′ az ′
= x̂′ ax′ + ŷ′ ay′ + ẑ′ az ′ ,
where the final expression has different axes than the original but, relative
to those axes, exactly the same form. Further rotation about other axes
would further reorient but naturally also would not alter the form. Now
consider ∇. The partial differential operators ∂/∂x, ∂/∂y and ∂/∂z change
no differently under reorientation than the component amplitudes ax , ay
and az do. Hence,
∂ ∂ ∂ ∂
∇ = ı̂ = x̂′ ′ + ŷ′ ′ + ẑ′ ′ , (16.2)
∂i ∂x ∂y ∂z
evidently the same operator regardless of the choice of basis [x̂′ ŷ′ ẑ′ ]. It is
this invariance under reorientation that makes the ∇ operator useful.
If ∇ takes the place of the ambiguous d/dr, then what takes the place
of the ambiguous d/dr′ , d/dro , d/dr̃, d/dr† and so on? Answer: ∇′ , ∇o , ∇, ˜
†
∇ and so on. Whatever mark distinguishes the special r, the same mark
distinguishes the corresponding special ∇. For example, where ro = ı̂io ,
there ∇o = ı̂ ∂/∂io . That is the convention.4
Introduced by Oliver Heaviside, informally pronounced “del” (in the
author’s country at least), the vector differential operator ∇ finds extensive
use in the modeling of physical phenomena. After a brief digression to
4
A few readers not fully conversant with the material of Ch. 15, to whom this chap-
ter had been making sense until the last two sentences, may suddenly find the notation
discuss operator notation, the subsections that follow will use the operator
to develop and present the four basic kinds of vector derivative.
16.1.2 Operator notation

Section 16.1.1 has introduced operator notation without explaining what it
is or even what it concerns. This subsection digresses to explain.
Operator notation concerns the representation of unary operators and
the operations they specify. Section 7.3 has already broadly introduced the
notion of the operator. A unary operator is a mathematical agent that
transforms a single discrete quantity, a single distributed quantity, a single
field, a single function or another
R t single mathematical object in some def-
inite way. For example, J ≡ 0 dt is a unary operator, more fully written
Rt
J ≡ 0 · dt where the “·” holds the place of the thing operated upon,5 whose
effect is such that, for instance, Jt = t2 /2 and J cos ωt = (sin ωt)/ω. Any
letter might serve as well as the example’s J; but what distinguishes oper-
ator notation is that, like the matrix row operator A in matrix notation’s
product Ax (§ 11.1.1), the operator J in operator notation’s operation Jt
attacks from the left. Thus, generally, Jt 6= tJ if J is a unary operator,
though the notation Jt usually formally resembles multiplication in other
respects as we shall see.
The matrix actually is a type of unary operator and matrix notation
is a specialization of operator notation, so we have met operator notation
before. And, in fact, we have met operator notation much earlier than that.
The product 5t can if you like be regarded as the unary operator “5 times,”
operating on t. Perhaps you did not know that 5 was an operator—and,
indeed, the scalar 5 itself is no operator but just a number—but where no
other operation is defined operator notation implies scalar multiplication
by default. Seen in this way, 5t and t5 actually mean two different things;
though naturally in the specific case of scalar multiplication, which happens
to be commutative, it is true that 5t = t5.
The a · in the dot product a · b and the a × in the cross product a × b
can profitably be regarded as unary operators.
Whether operator notation can licitly represent any unary operation
incomprehensible. The notation is Einstein’s. It means
X
ı̂io = ı̂io = x̂′ x′o + ŷ′ yo′ + ẑ′ zo′ ,
i=x′ ,y ′ ,z ′
in the leftmost form of which the summation sign is implied not written. Refer to § 15.4.
5
[10]
whatsoever is a definitional question we shall leave for the professional math-

ematician to answer, but in normal usage operator notation represents only
linear unary operations, unary operations that honor § 7.3.3’s rule of linear-
ity. The operators J and A above are examples of linear unary operators;
the operator K ≡ · + 3 is not linear and almost certainly should never be
represented in operator notation as here, lest an expression like Kt mislead
an understandably unsuspecting audience. Linear unary operators often do
not commute, so J1 J2 6= J2 J1 generally; but otherwise linear unary opera-
tors follow familiar rules of multiplication like (J2 + J3 )J1 = J2 J1 + J3 J1 .
Linear unary operators obey a definite algebra, the same algebra matrices
obey. It is precisely this algebra that makes operator notation so useful.
Operators associate from right to left (§ 2.1.1) so that, in operator nota-
tion, Jωt = J(ωt), not (Jω)t. Observe however that the perceived need for
parentheses comes only of the purely notational ambiguity as to whether ωt
bears the semantics of “the product of ω and t” or those of “the unary
operator ‘ω times,’ operating on t.” The perceived need and any associ-
ated confusion would vanish if Ω ≡ (ω)(·) were unambiguously an opera-
tor, in which case the product JΩ would itself be an operator, whereupon
(JΩ)t = JΩt = J(Ωt) = J(ωt). Indeed, one can compare the distinction in
§ 11.3.2 between λ and λI against the distinction between ω and Ω here, for
a linear unary operator enjoys the same associativity (11.5) a matrix enjoys,
and for the same reason. Still, rather than go to the trouble of defining
extra symbols like Ω, it is usually easier just to write the parentheses, which
take little space on the page and are universally understood; or, better, to
rely on the right-to-left convention that Jωt = J(ωt). Modern conventional
applied mathematical notation though generally excellent remains imper-
fect; so, notationally, when it matters, operators associate from right to left
except where parentheses group otherwise.
One can speak of a unary operator like J, A or Ω without giving it any-
thing in particular to operate upon. One can leave an operation R t unresolved.
For example, tJ is itself a unary operator—it is the operator t 0 dt—though
one can assign no particular value to it until it actually operates on some-
thing. The operator ∇ of (16.2) is an unresolved unary operator of the same
kind.
16.1.3 The directional derivative and the gradient

In the calculus of vector fields, the derivative notation d/dr is ambiguous
because, as the section’s introduction has observed, the notation gives r
no specific direction in which to advance. In operator notation, however,
given (16.2) and accorded a reference vector b to supply a direction and a

scale, one can compose the directional derivative operator
∂
(b · ∇) = bi (16.3)
∂i
to express the derivative unambiguously. This operator applies equally to
the scalar field,
∂ψ
(b · ∇)ψ(r) = bi ,
∂i
as to the vector field,
∂a ∂aj
(b · ∇)a(r) = bi = ̂bi . (16.4)
∂i ∂i
For the scalar field the parentheses are unnecessary and conventionally are
omitted, as
∂ψ
b · ∇ψ(r) = bi . (16.5)
∂i
In the case (16.4) of the vector field, however, ∇a(r) itself means nothing co-
herent6 so the parentheses usually are retained. Equations (16.4) and (16.5)
define the directional derivative.
Note that the directional derivative is the derivative not of the reference
vector b but only of the field ψ(r) or a(r). The vector b just directs and
scales the derivative; it is not the object of it. Nothing requires b to be
constant, though. It can be a vector field b(r) that varies from place to
place; the directional derivative does not care.
Within (16.5), the quantity
∂ψ
∇ψ(r) = ı̂ (16.6)
∂i
is called the gradient of the scalar field ψ(r). Though both scalar and
vector fields have directional derivatives, only scalar fields have gradients.
The gradient represents the amplitude and direction of a scalar field’s locally
steepest rate.
Formally a dot product, the directional derivative operator b · ∇ is in-
variant under reorientation of axes, whereupon the directional derivative is
invariant, too. The result of a ∇ operation, the gradient ∇ψ(r) is likewise
invariant.
6
Well, it does mean something coherent in dyadic analysis [9, Appendix B], but this
book won’t treat that.
16.1.4 Divergence
There exist other vector derivatives than the directional derivative and gra-
dient of § 16.1.3. One of these is divergence. It is not easy to motivate
divergence directly, however, so we shall approach it indirectly, through the
concept of flux as follows.
The flux of a vector field a(r) outward from a region in space is
I
Φ≡ a(r) · ds, (16.7)
S
where
ds ≡ n̂ · ds (16.8)
is a vector infinitesimal of amplitude ds, directed normally outward from

the closed surface bounding the region—ds being the area of an infinitesi-
mal element of the surface, the area of a tiny patch. Flux is flow through
a surface: in this case, net flow outward from the region in question. (The
paragraph says much in relatively few words. If it seems opaque then try to
visualize eqn. 16.7’s dot product a[r] · ds, in which the vector ds represents
the area and orientation of a patch of the region’s enclosing surface. When
something like air flows through any surface—not necessarily a physical bar-
rier but an imaginary surface like the goal line’s vertical plane in a football
game7 —what matters is not the surface’s area as such but rather the area the
surface presents to the flow. The surface presents its full area to a perpen-
dicular flow, but otherwise the flow sees a foreshortened surface, as though
the surface were projected onto a plane perpendicular to the flow. Refer to
Fig. 15.2. Now realize that eqn. 16.7 actually describes flux not through an
open surface but through a closed—it could be the imaginary rectangular
box enclosing the region of football play to goal-post height; where wind
blowing through the region, entering and leaving, would constitute zero net
flux; but where a positive net flux would have barometric pressure falling
and air leaving the region maybe because a storm is coming—and you’ve
got the idea.)
A region of positive flux isR a source; of negative flux, a sink. One can
contemplate the flux Φopen = S a(r) · ds through an open surface as well as
through a closed, but it is the outward flux (16.7) through a closed surface
that will concern us here.
7
The author has American football in mind but other football games have goal lines
and goal posts, too. Pick your favorite brand of football.
The outward flux Φ of a vector field a(r) through a closed surface bound-
ing some definite region in space is evidently
ZZ ZZ ZZ
Φ= ∆ax (y, z) dy dz + ∆ay (z, x) dz dx + ∆az (x, y) dx dy,
where
xmax (y,z)
∂ax
Z
∆ax (y, z) = dx,
xmin (y,z) ∂x
ymax (z,x)
∂ay
Z
∆ay (z, x) = dy,
ymin (z,x) ∂y
zmax (x,y)
∂az
Z
∆az (x, y) = dz
zmin (x,y) ∂z
represent the increase across the region respectively of ax , ay or az along

an x̂-, ŷ- or ẑ-directed line.8 If the field has constant derivatives ∂a/∂i, or
equivalently if the region in question is small enough that the derivatives
are practically constant through it, then these increases are simply
∂ax
∆ax (y, z) = ∆x(y, z),
∂x
∂ay
∆ay (z, x) = ∆y(z, x),
∂y
∂az
∆az (x, y) = ∆z(x, y),
∂z
upon which
∂ax ∂ay
ZZ ZZ
Φ = ∆x(y, z) dy dz + ∆y(z, x) dz dx
∂x ∂y
∂az
ZZ
+ ∆z(x, y) dx dy.
∂z
But each of the last equation’s three integrals represents the region’s vol-
ume V , so
∂ax ∂ay ∂az
Φ = (V ) + + ;
∂x ∂y ∂z
8
Naturally, if the region’s boundary happens to be concave, then some lines might enter
and exit the region more than once, but this merely elaborates the limits of integration
along those lines. It changes the problem in no essential way.
or, dividing through by the volume,

Φ ∂ax ∂ay ∂az ∂ai
= + + = = ∇ · a(r). (16.9)
V ∂x ∂y ∂z ∂i
We give this ratio of outward flux to volume,
∂ai
∇ · a(r) = , (16.10)
∂i
the name divergence, representing the intensity of a source or sink.
Formally a dot product, divergence is invariant under reorientation of
axes.
16.1.5 Curl
Curl is to divergence as the cross product is to the dot product. Curl is a
little trickier to visualize, though. It needs first the concept of circulation
as follows.
The circulation of a vector field a(r) about a closed contour in space is
I
Γ ≡ a(r) · dℓ, (16.11)
H
where, unlikeH the S of (16.7) which represented a double integration over a
surface, the here represents only a single integration. One can in general
contemplate circulation about any closed contour, but it suits our purpose
here to consider specifically a closed contour that happens not to depart
from a single, flat plane in space.
Let [û v̂ n̂] be an orthogonal basis with n̂ normal to the contour’s plane
such that travel positively along the contour tends from û toward v̂ rather
than the reverse. The circulation Γ of a vector field a(r) about this contour
is evidently Z Z
Γ= ∆av (v) dv − ∆au (u) du,
where
umax (v)
∂av
Z
∆av (v) = du,
umin (v) ∂u
vmax (u)
∂au
Z
∆au (u) = dv
vmin (u) ∂v
represent the increase across the contour’s interior respectively of av or au
along a û- or v̂-directed line. If the field has constant derivatives ∂a/∂i, or
equivalently if the contour in question is short enough that the derivatives

are practically constant over it, then these increases are simply
∂av
∆av (v) = ∆u(v),
∂u
∂au
∆au (u) = ∆v(u),
∂v
upon which
∂av ∂au
Z Z
Γ= ∆u(v) dv − ∆v(u) du.
∂u ∂v
But each of the last equation’s two integrals represents the area A within
the contour, so
∂av ∂au
Γ = (A) − ;
∂u ∂v
or, dividing through by the area,
Γ ∂av ∂au
= −
A ∂u ∂v
û v̂ n̂
∂ak
= n̂ · ǫijk ı̂ = n̂ · ∂/∂u ∂/∂v ∂/∂n
∂j au av an
= n̂ · [∇ × a(r)]. (16.12)
We give this ratio of circulation to area,

∂ak ∂av ∂au
n̂ · [∇ × a(r)] = n̂ · ǫijk ı̂ = − , (16.13)
∂j ∂u ∂v
the name directional curl, representing the intensity of circulation, the degree
of twist so to speak, about a specified axis. The cross product in (16.13),
∂ak
∇ × a(r) = ǫijk ı̂ , (16.14)
∂j
we call curl.
Curl (16.14) is an interesting quantity. Although it emerges from di-
rectional curl (16.13) and although we have developed directional curl with
respect to a contour in some specified plane, curl (16.14) itself turns out
to be altogether independent of any particular plane. We might have cho-
sen another plane and though n̂ would then be different the same (16.14)
would necessarily result. Directional curl, a scalar, is a property of the field
and the plane. Curl, a vector, unexpectedly is a property of the field only.
Directional curl evidently cannot exceed curl in magnitude, but will equal
it when n̂ points in its direction, so it may be said that curl is the locally
greatest directional curl, oriented normally to the locally greatest directional
curl’s plane.
We have needed n̂ and (16.13) to motivate and develop the concept
(16.14) of curl. Once developed, however, the concept of curl stands on its
own, whereupon one can return to define directional curl more generally
than (16.13) has defined it. As in (16.4) here too any reference vector b or
vector field b(r) can serve to direct the curl, not only n̂. Hence,

∂ak ∂ak
b · [∇ × a(r)] = b · ǫijk ı̂ = ǫijk bi . (16.15)
∂j ∂j
This would be the actual definition of directional curl. Note however that
directional curl so defined is not a distinct kind of derivative but rather is
just curl, dot-multiplied by a reference vector.
Formally a cross product, curl is invariant under reorientation of axes.
An ordinary dot product, directional curl is likewise invariant.
16.1.6 Cross-directional derivatives
The several directional derivatives of the b · ∇ class, including the scalar

(16.5) and vector (16.4) directional derivatives themselves and also including
directional curl (16.15), compute rates with reference to some direction b.
Another class of directional derivatives however is possible, that of the cross-
directional derivatives.9 These compute rates perpendicularly to b. Unlike
the vector directional derivative (16.4), the cross-directional derivatives are
not actually new derivatives but are cross products of b with derivatives
already familiar to us. The cross-directional derivatives are
∂ψ
b × ∇ψ = ǫijk ı̂bj ,
∂k (16.16)
∂ai ∂aj
b × ∇ × a = ̂bi − .
∂j ∂i
9
The author is unaware of a conventional name for these derivatives. The name cross-
directional seems as apt as any.
16.2. INTEGRAL FORMS 449
We call these respectively the cross-directional derivative (itself) and cross-

directional curl, the latter derived as

∂ak
b × ∇ × a = b × ǫijk ı̂
∂j

∂ak ∂ak
= ǫmni m̂bn ǫijk ı̂ = ǫmni ǫijk m̂bn
∂j i ∂j
∂ak
= (δmj δnk − δmk δnj )m̂bn
∂j
∂ak ∂ak ∂ai ∂aj
= ̂bk − k̂bj = ̂bi − ̂bi
∂j ∂j ∂j ∂i
where the Levi-Civita identity that ǫmni ǫijk = ǫimn ǫijk = δmj δnk − δmk δnj
comes from Table 15.1.
16.2 Integral forms

The vector field’s distinctive maneuver is the shift between integral forms,
which we are now prepared to treat. This shift comes in two kinds. The two
subsections that follow explain.
16.2.1 The divergence theorem

Section 16.1.4 has contemplated the flux of a vector field a(r) from a volume
small enough that the divergence ∇ ·a were practically constant through the
volume. One would like to treat the flux from larger, more general volumes
as well. According to the definition (16.7), the flux from any volume is
I
Φ= a · ds.
S
If one subdivides a large volume into infinitesimal volume elements dv, then
the flux from a single volume element is
I
Φelement = a · ds.
Selement
Even a single volume element however can have two distinct kinds of surface
area: inner surface area shared with another element; and outer surface area
shared with no other element because it belongs to the surface of the larger,
overall volume. Interior elements naturally have only the former kind but
boundary elements have both kinds of surface area, so one can elaborate the
last equation to read
Z Z
Φelement = a · ds + a · ds
Sinner Souter
H R R
for a single element, where Selement = Sinner + Souter . Adding all the ele-
ments together, we have that
X X Z X Z
Φelement = a · ds + a · ds;
elements elements Sinner elements Souter
but the inner sum is null because it includes each interior surface twice,
because each interior surface is shared by two elements such that ds2 = −ds1
(in other words, such that the one volume element’s ds on the surface the
two elements share points oppositely to the other volume element’s ds on
the same surface), so
X X Z I
Φelement = a · ds = a · ds.
elements elements Souter S
In this equation, the last integration is over the surface of the larger, overall
volume, which surface after all consists of nothing other than the several
boundary elements’ outer surface patches. Applying (16.9) to the equation’s
left side to express the flux Φelement from a single volume element yields
X I
∇ · a dv = a · ds.
elements S
That is, Z I
∇ · a dv = a · ds. (16.17)
V S
Equation (16.17) is the divergence theorem.10 The divergence theorem,

the vector’s version of the fundamental theorem of calculus (7.2), neatly
relates the divergence within a volume to the flux from it. It is an important
result. The integral on the equation’s left and the one on its right each arise
in vector analysis more often than one might expect. When they do, (16.17)
swaps the one integral for the other, often a profitable maneuver.11
10
[60, eqn. 1.2.8]
11
Where a wave propagates through a material interface, the associated field can be
discontinuous and, consequently, the field’s divergence can be infinite, which would seem
16.3. SUMMARY OF DEFINITIONS AND IDENTITIES 451
16.2.2 Stokes’ theorem

Corresponding to the divergence theorem of § 16.2.1 is a second, related the-
orem for directional curl, developed as follows. If an open surface, whether
the surface be confined to a plane or be warped in three dimensions (as for ex-
ample in bowl shape), is subdivided into infinitesimal surface elements ds—
each element small enough not only to experience essentially constant curl
but also to be regarded as planar—then according to (16.11) the circulation
about the entire surface is I
Γ = a · dℓ
and the circulation about any one surface element is

I
Γelement = a · dℓ.
element
From this equation, reasoning parallel to that of § 16.2.1—only using (16.12)

in place of (16.9)—concludes that
Z I
(∇ × a) · ds = a · dℓ. (16.18)
S
Equation (16.18) is Stokes’ theorem,12 ,13 neatly relating the directional curl
over a (possibly nonplanar) surface to the circulation about it. Like the di-
vergence theorem (16.17), Stokes’ theorem (16.18) serves to swap one vector
integral for another where such a maneuver is needed.
16.3 Summary of definitions and identities of vec-

tor calculus
Table 16.1 lists useful definitions and identities of vector calculus,14 the
first several of which it gathers from §§ 16.1 and 16.2, the last several of
to call assumptions underlying (16.17) into question. However, the infinite divergence
at a material interface is normally integrable in the same way the Dirac delta of § 7.7,
though infinite, is integrable. One can integrate finitely through either infinity. If one can
conceive of an interface not as a sharp layer of zero thickness but rather as a thin layer of
thickness ǫ, through which the associated field varies steeply but continuously, then the
divergence theorem necessarily remains valid in the limit ǫ → 0.
12
[60, eqn. 1.4.20]
13
If (16.17) is “the divergence theorem,” then should (16.18) not be “the curl theorem”?
Answer: maybe it should be, but no one calls it that. Sir George Gabriel Stokes evidently
is not to be denied his fame!
14
[4, Appendix II.3][60, Appendix II][28, Appendix A]
which (exhibiting heretofore unfamiliar symbols like ∇2 ) it gathers from

§ 16.4 to follow. Of the identities in the middle of the table, a few are
statements of the ∇ operator’s distributivity over summation. The rest are
vector derivative product rules (§ 4.5.2).
The product rules resemble the triple products of Table 15.2, only with
the ∇ operator in place of the vector c. However, since ∇ is a differential
operator for which, for instance, b · ∇ 6= ∇ · b, its action differs from a
vector’s in some cases, and there are more distinct ways in which it can act.
Among the several product rules the easiest to prove is that
∂(ψω) ∂ψ ∂ω
∇(ψω) = ı̂ = ωı̂ + ψı̂ = ω∇ψ + ψ∇ω.
∂i ∂i ∂i
The hardest to prove is that
∂(ai bi ) ∂ai ∂bi

∇(a · b) = ∇(ai bi ) = ̂ = ̂bi + ̂ai
∂j ∂j ∂j

∂aj ∂ai ∂aj ∂bj ∂bi ∂bj
= ̂bi + ̂bi − + ̂ai + ̂ai −
∂i ∂j ∂i ∂i ∂j ∂i
= (b · ∇)a + b × ∇ × a + (a · ∇)b + a × ∇ × b
= (b · ∇ + b × ∇ × )a + (a · ∇ + a × ∇ × )b,
because to prove it one must recognize in it the cross-directional curl of

(16.16). Also nontrivial to prove is that
∇ × (a × b) = ∇ × (ǫijk ı̂aj bk )
∂(ǫijk ı̂aj bk )i ∂(aj bk )
= ǫmni m̂ = ǫmni ǫijk m̂
∂n ∂n
∂(aj bk )
= (δmj δnk − δmk δnj )m̂
∂n
∂(aj bk ) ∂(aj bk ) ∂(aj bi ) ∂(ai bj )
= ̂ − k̂ = ̂ − ̂
∂k ∂j ∂i ∂i

∂aj ∂bi ∂bj ∂ai
= ̂bi + ̂aj − ̂ai + ̂bj
∂i ∂i ∂i ∂i
= (b · ∇ + ∇ · b)a − (a · ∇ + ∇ · a)b.
16.3. SUMMARY OF DEFINITIONS AND IDENTITIES 453
Table 16.1: Definitions and identities of vector calculus (see also Table 15.2
on page 425).
∂ ∂
∇ ≡ ı̂ b · ∇ = bi
∂i ∂i
∂ψ ∂ψ
∇ψ = ı̂ b · ∇ψ = bi
∂i ∂i
∂ai ∂a ∂aj
∇·a = (b · ∇)a = bi = ̂bi
∂i ∂i ∂i
∂ak ∂ak
∇ × a = ǫijk ı̂ b·∇×a = ǫijk bi
∂j ∂j
∂ψ ∂ai ∂aj
b × ∇ψ = ǫijk ı̂bj b×∇×a = ̂bi −
∂k ∂j ∂i
Z Z I
Φ ≡ a · ds ∇ · a dv = a · ds
Z S Z V I S
Γ ≡ a · dℓ (∇ × a) · ds = a · dℓ
C S
∇ · (a + b) = ∇ · a + ∇ · b
∇ × (a + b) = ∇ × a + ∇ × b
∇(ψ + ω) = ∇ψ + ∇ω
∇(ψω) = ω∇ψ + ψ∇ω
∇ · (ψa) = a · ∇ψ + ψ∇ · a
∇ × (ψa) = ψ∇ × a − a × ∇ψ
∇(a · b) = (b · ∇ + b × ∇ × )a + (a · ∇ + a × ∇ × )b
∇ · (a × b) = b · ∇ × a − a · ∇ × b
∇ × (a × b) = (b · ∇ + ∇ · b)a − (a · ∇ + ∇ · a)b
∂2 ∂ 2 ai
∇2 ≡ ∇∇ · a = ̂
∂i2 ∂j ∂i
∂ 2ψ ∂ 2a ∂ 2 aj
∇2 ψ = ∇ · ∇ψ = 2 ∇2 a = = ̂ = ̂∇2 (̂ · a)
∂i ∂i2 ∂i2
∇ × ∇ψ = 0 ∇·∇×a = 0

∂ ∂ai ∂aj
∇ × ∇ × a = ̂ −
∂i ∂j ∂i
∇∇ · a = ∇2 a + ∇ × ∇ × a
The others are less hard:15

∂(ψai ) ∂ψ ∂ai
∇ · (ψa) = = ai +ψ = a · ∇ψ + ψ∇ · a;
∂i ∂i ∂i
∂(ψak ) ∂ak ∂ψ
∇ × (ψa) = ǫijk ı̂ = ǫijk ı̂ψ + ǫijk ı̂ak
∂j ∂j ∂j
= ψ∇ × a − a × ∇ψ;
∂(ǫijk aj bk ) ∂aj ∂bk
∇ · (a × b) = = ǫijk bk + ǫijk aj
∂i ∂i ∂i
= b · ∇ × a − a · ∇ × b.
Inasmuch as none of the derivatives or products within the table’s several
product rules vary under rotation of axes, the product rules are themselves
invariant. That the definitions and identities at the top of the table are
invariant, we have already seen; and § 16.4, next, will give invariance to
the definitions and identities at the bottom. The whole table therefore is
invariant under rotation of axes.
16.4 The Laplacian and other second-order deriv-

atives
Table 16.1 ends with second-order vector derivatives. Like vector products
and first-order vector derivatives, second-order vector derivatives too come
in several kinds, the simplest of which is the Laplacian 16
∂2
∇2 ≡ ,
∂i2
∂2ψ
∇2 ψ = ∇ · ∇ψ = , (16.19)
∂i2
∂2a ∂ 2 aj
∇2 a = 2 = ̂ 2 = ̂∇2 (̂ · a).
∂i ∂i
Other second-order vector derivatives include
∂ 2 ai
∇∇ · a = ̂ ,
∂j ∂i
(16.20)
∂ ∂ai ∂aj
∇ × ∇ × a = ̂ − ,
∂i ∂j ∂i
15
And probably should have been left as exercises, except that this book is not actually
an instructional textbook. The reader who wants exercises might hide the page from sight
and work the three identities out with his own pencil.
16
Though seldom seen in applied usage in the author’s country, the alternate symbol ∆
replaces ∇2 in some books.
16.4. THE LAPLACIAN, ET AL. 455
the latter of which is derived as

∂ak
∇ × ∇ × a = ∇ × ǫijk ı̂
∂j
∂ 2 ak

∂ ∂ak
= ǫmni m̂ ǫijk ı̂ = ǫmni ǫijk m̂
∂n ∂j i ∂n ∂j
∂ 2 ak
= (δmj δnk − δmk δnj )m̂
∂n ∂j
∂ 2 ak ∂ 2 ak ∂ 2 ai ∂ 2 aj
= ̂ − k̂ 2 = ̂ − ̂ 2 .
∂k ∂j ∂j ∂i ∂j ∂i
Combining the various second-order vector derivatives yields the useful iden-
tity that
∇∇ · a = ∇2 a + ∇ × ∇ × a. (16.21)
The table includes two curious null identities,
∇ × ∇ψ = 0,
(16.22)
∇ · ∇ × a = 0.
In words, (16.22) states that gradients do not curl and curl does not diverge.
This is unexpected but is a direct consequence of the definitions of the
gradient, curl and divergence:
∂2ψ

∂ψ
∇ × ∇ψ = ∇ × ı̂ = ǫmni m̂ = 0;
∂i ∂n ∂i
∂ 2 ak

∂ak
∇ · ∇ × a = ∇ · ǫijk ı̂ = ǫijk = 0.
∂j ∂i ∂j
A field like ∇ψ that does not curl is called an irrotational field. A field like
∇×a that does not diverge is called a solenoidal, source-free or (prosaically)
divergenceless field.17
17
In the writer’s country, the United States, there has been a mistaken belief afoot
that, if two fields b1 (r) and b2 (r) had everywhere the same divergence and curl, then the
two fields could differ only by an additive constant. Even at least one widely distributed
textbook expresses this belief, naming it Helmholtz’s theorem; but it is not just the one
textbook, for the writer has heard it verbally from at least two engineers, unacquainted
with one other, who had earned Ph.D.s in different eras in different regions of the country.
So the belief must be correct, mustn’t it?
Well, maybe it is, but the writer remains unconvinced. Consider the admittedly con-
trived counterexample of b1 = x̂y + ŷx, b2 = 0.
Each of this section’s second-order vector derivatives—including the vec-

tor Laplacian ∇2 a, according to (16.21)—is or can be composed of first-order
vector derivatives already familiar to us from § 16.1. Therefore, inasmuch as
each of those first-order vector derivatives is invariant under reorientation
of axes, each second-order vector derivative is likewise invariant.
16.5 Contour derivative product rules

Equation (4.25) gives the derivative product rule for functions of a scalar
variable. Fields—that is, functions of a vector variable—obey product rules,
too, several of which Table 16.1 lists. The table’s product rules however
are general product rules that regard full spatial derivatives. What about
derivatives along an arbitrary contour? Do they obey product rules, too?
That is, one supposes that18
∂ ∂ψ ∂ω
(ψω) = ω +ψ ,
∂ℓ ∂ℓ ∂ℓ
∂ ∂ψ ∂a
(ψa) = a +ψ ,
∂ℓ ∂ℓ ∂ℓ (16.23)
∂ ∂a ∂b
(a · b) = b · +a· ,
∂ℓ ∂ℓ ∂ℓ
∂ ∂a ∂b
(a × b) = −b × +a× .
∂ℓ ∂ℓ ∂ℓ
where ℓ is the distance along some arbitrary contour in space. As a hypoth-
esis, (16.23) is attractive. But is it true?
That the first line of (16.23) is true is clear, if you think about it in
the right way, because, in the restricted case (16.23) represents, one can
treat the scalar fields ψ(r) and ω(r) as ordinary scalar functions ψ(ℓ) and
On an applied level, the writer knows of literally no other false theorem so widely
believed to be true, which leads the writer to suspect that he himself had somehow erred
in judging the theorem false. What the writer really believes however is that Hermann
von Helmholtz probably originally had put some appropriate restrictions on b1 and b2
which, if obeyed, made his theorem true but which at some time after his death got lost in
transcription. That a transcription error would go undetected so many years would tend
to suggest that Helmholtz’s theorem, though interesting, were not actually very necessary
in practical applications. (Some believe the theorem necessary to establish a “gauge” in
a wave equation, but if they examine the use of their gauges closely then they will likely
discover that one does not logically actually need to invoke the theorem to use the gauges.)
Corrections by readers are invited.
18
The − sign in (16.23)’s last line is an artifact of ordering the line’s factors in the style
of Table 16.1. Before proving the line, the narrative will reverse the order to kill the sign.
See below.
16.6. METRIC COEFFICIENTS 457
ω(ℓ) of the scalar distance ℓ along the contour, whereupon (4.25) applies—
for (16.23) never evaluates ψ(r) or ω(r) but along the contour. The same
naturally goes for the vector fields a(r) and b(r), which one can treat as
vector functions a(ℓ) and b(ℓ) of the scalar distance ℓ; so the second and
third lines of (16.23) are true, too, since one can write the second line in the
form
∂ ∂ψ ∂ai
ı̂ (ψai ) = ı̂ ai +ψ
∂ℓ ∂ℓ ∂ℓ
and the third line in the form
∂ ∂ai ∂bi
(ai bi ) = bi + ai ,
∂ℓ ∂ℓ ∂ℓ
each of which, according to the first line, is true separately for i = x, for
i = y and for i = z.
The truth of (16.23)’s last line is slightly less obvious. Nevertheless, one
can reorder factors to write the line as
∂ ∂a ∂b
(a × b) = ×b+a× ,
∂ℓ ∂ℓ ∂ℓ
the Levi-Civita form (§ 15.4.3) of which is

∂ ∂aj ∂bk
ǫijk ı̂ (aj bk ) = ǫijk ı̂ bk + aj .
∂ℓ ∂ℓ ∂ℓ
The Levi-Civita form is true separately for (i, j, k) = (x, y, z), for (i, j, k) =
(x, z, y), and so forth, so (16.23)’s last line as a whole is true, too, which
completes the proof of (16.23).
16.6 Metric coefficients

A scalar field ψ(r) is the same field whether expressed as a function ψ(x, y, z)
of rectangular coordinates, ψ(ρ; φ, z) of cylindrical coordinates or ψ(r; θ; φ)
of spherical coordinates,19 or indeed of coordinates in any three-dimensional
system. However, cylindrical and spherical geometries normally recommend
cylindrical and spherical coordinate systems, systems which make some of
the identities of Table 16.1 hard to use.
The reason a cylindrical or spherical system makes some of the table’s
identities hard to use is that some of the table’s identities involve derivatives
19
For example, the field ψ = x2 + y 2 in rectangular coordinates is ψ = ρ2 in cylindrical
coordinates. Refer to Table 3.4.
Table 16.2: The metric coefficients of the rectangular, cylindrical and spher-
ical coordinate systems.
RECT. CYL. SPHER.

hx = 1 hρ = 1 hr = 1
hy = 1 hφ = ρ hθ = r
hz = 1 hz = 1 hφ = r sin θ
d/di, notation which per § 15.4.2 stands for d/dx′ , d/dy ′ or d/dz ′ where the
coordinates x′ , y ′ and z ′ represent lengths. But among the cylindrical and
spherical coordinates are θ and φ, angles rather than lengths. Because one
cannot use an angle as though it were a length, the notation d/di cannot
stand for d/dθ or d/dφ and, thus, one cannot use the table in cylindrical or
spherical coordinates as the table stands.
We therefore want factors to convert the angles in question to lengths (or,
more generally, when special coordinate systems like the parabolic systems of
§ 15.7 come into play, to convert coordinates other than lengths to lengths).
Such factors are called metric coefficients and Table 16.2 lists them.20 The
use of the table is this: that for any metric coefficient hα a change dα in
its coordinate α sweeps out a length hα dα. For example, in cylindrical
coordinates hφ = ρ according to table, so a change dφ in the azimuthal
coordinate φ sweeps out a length ρ dφ—a fact we have already informally
observed as far back as § 7.4.1, which the table now formalizes.
In the table, incidentally, the metric coefficient hφ seems to have two
different values, one value in cylindrical coordinates and another in spherical.
The two are really the same value, though, since ρ = r sin θ per Table 3.4.
16.6.1 Displacements, areas and volumes

In any orthogonal, right-handed, three-dimensional coordinate system
(α; β; γ)—whether the symbols (α; β; γ) stand for (x, y, z), (y, z, x), (z, x, y),
(x′ , y ′ , z ′ ), (ρ; φ, z), (r; θ; φ), (φx , r; θ x ), etc.,21 or even something exotic like
20
[12, § 2-4]
21
The book’s admittedly clumsy usage of semicolons “;” and commas “,” to delimit
coordinate triplets, whereby a semicolon precedes an angle (or, in this section’s case,
precedes a generic coordinate like α that could stand for an angle), serves well enough
to distinguish the three principal coordinate systems (x, y, z), (ρ; φ, z) and (r; θ; φ) visu-
ally from one another but ceases to help much when further coordinate systems such as
(φx , r; θx ) come into play. Logically, maybe, it would make more sense to write in the
16.6. METRIC COEFFICIENTS 459
the parabolic (σ, τ, z) of § 15.7—the product
ds = α̂hβ hγ dβ dγ (16.24)
represents an area infinitesimal normal to α̂. For example, the area infinites-
imal on a spherical surface of radius r is ds = r̂hθ hφ dθ dφ = r̂r 2 sin θ dθ dφ.
Again in any orthogonal, right-handed, three-dimensional coordinate
system (α; β; γ), the product
dv = hα hβ hγ dα dβ dγ (16.25)
represents a volume infinitesimal. For example, the volume infinitesimal in

a spherical geometry is dv = hr hθ hφ dr dθ dφ = r 2 sin θ dr dθ dφ.
Notice that § 7.4 has already calculated several integrals involving area
and volume infinitesimals of these kinds.
A volume infinitesimal (16.25) cannot meaningfully in three dimensions
be expressed as a vector as an area infinitesimal (16.24) can, since in three
dimensions a volume has no orientation. Naturally however, a length or
displacement infinitesimal can indeed be expressed as a vector, as
dℓ = α̂hα dα. (16.26)
Section 16.10 will have more to say about vector infinitesimals in non-
rectangular coordinates.
16.6.2 The vector field and its scalar components

Like a scalar field ψ(r), a vector field a(r) too is the same field whether
expressed as a function a(x, y, z) of rectangular coordinates, a(ρ; φ, z) of
cylindrical coordinates or a(r; θ; φ) of spherical coordinates, or indeed of co-
ordinates in any three-dimensional system. A vector field however is the sum
of three scalar fields, each scaling an appropriate unit vector. In rectangular
coordinates,
a(r) = x̂ax (r) + ŷay (r) + ẑaz (r);
in cylindrical coordinates,
a(r) = ρ̂aρ (r) + φ̂aφ (r) + ẑaz (r);

manner of (, x, y, z), but to do so seems overwrought and fortunately no one the author
knows of does it in that way. The delimiters just are not that important.
The book adheres to the semicolon convention not for any deep reason but only for lack
of a better convention. See also Ch. 15’s footnote 31.
and in spherical coordinates,
a(r) = r̂ar (r) + θ̂aθ (r) + φ̂aφ (r).
The scalar fields aρ (r), ar (r), aθ (r) and aφ (r) in and of themselves do not
differ in nature from ax (r), ay (r), az (r), ψ(r) or any other scalar field. One
does tend to use them differently, though, because constant unit vectors x̂,
ŷ and ẑ exist to combine the scalar fields ax (r), ay (r), az (r) to compose the
vector field a(r) whereas no such constant unit vectors exist to combine the
scalar fields aρ (r), ar (r), aθ (r) and aφ (r). Of course there are the variable
unit vectors ρ̂(r), r̂(r), θ̂(r) and φ̂(r), but the practical and philosophical
differences between these and the constant unit vectors is greater than it
might seem. For instance, it is true that ρ̂ · φ̂ = 0, so long as what is meant
by this is that ρ̂(r) · φ̂(r) = 0. However, ρ̂(r1 ) · φ̂(r2 ) 6= 0, an algebraic error
fairly easy to commit. On the other hand, that x̂ · ŷ = 0 is always true.
(One might ask why such a subsection as this would appear in a section
on metric coefficients. The subsection is here because no obviously better
spot for it presents itself, but moreover because we shall need the under-
standing the subsection conveys to apply metric coefficients consistently and
correctly in § 16.9 to come.)
16.7 Nonrectangular notation

Section 15.4 has introduced Einstein’s summation convention, the Kronecker
delta δij and the Levi-Civita epsilon ǫijk together as notation for use in the
definition of vector operations and in the derivation of vector identities.
The notation relies on symbols like i, j and k to stand for unspecified co-
ordinates, and Tables 15.2 and 16.1 use it extensively. Unfortunately, the
notation fails in the nonrectangular coordinate systems when derivatives
come into play, as they do in Table 16.1, because ∂/∂i is taken to represent
a derivative specifically with respect to a length whereas nonrectangular co-
ordinates like θ and φ are not lengths. Fortunately, this failure is not hard
to redress.
Whereas the standard Einstein symbols i, j and k can stand only for
lengths, the modified Einstein symbols ı̃, ̃ and k̃, which this section now
introduces, can stand for any coordinates, even for coordinates like θ and φ
that are not lengths. The tilde “˜” atop the symbolı̃ warns readers that the
coordinate it represents is not necessarily a length and that, if one wants a
length, one must multiply ı̃ by an appropriate metric coefficient hı̃ (§ 16.6).
The products hı̃ ı̃, h̃ ̃ and hk̃k̃ always represent lengths.
16.8. DERIVATIVES OF THE BASIS VECTORS 461
The symbols ı̂, ̂ and k̂ need no modification even when modified symbols
like ı̃, ̃ and k̃ are in use, since ı̂, ̂ and k̂ are taken to represent unit vectors
and [ı̂ ̂ k̂], a proper orthogonal basis irrespective of the coordinate system—
so long, naturally, as the coordinate system is an orthogonal, right-handed
coordinate system as are all the coordinate systems in this book.
The modified notation will find use in § 16.9.3.
16.8 Derivatives of the basis vectors

The derivatives of the various unit basis vectors with respect to the several
coordinates of their respective coordinate systems are not hard to compute.
In fact, looking at Fig. 15.1 on page 402, Fig. 15.4 on page 413, and Fig. 15.5
on page 414, one can just write them down. Table 16.3 records them.
Naturally, one can compute the table’s derivatives symbolically, instead,
as for example
∂ ρ̂ ∂
= (x̂ cos φ + ŷ sin φ) = −x̂ sin φ + ŷ cos φ = +φ̂.
∂φ ∂φ
Such an approach prospers in special coordinate systems like the parabolic

systems of Tables 15.3 and 15.4, but in cylindrical and spherical coordinates
it is probably easier just to look at the figures.
16.9 Derivatives in the nonrectangular systems

This section develops vector derivatives in cylindrical and spherical coordi-
nates.
16.9.1 Derivatives in cylindrical coordinates

According to Table 16.1,
∂ψ
∇ψ = ı̂ ,
∂i
but as § 16.6 has observed Einstein’s symbol i must stand for a length not an
angle, whereas one of the three cylindrical coordinates—the azimuth φ—is
an angle. The cylindrical metric coefficients of Table 16.2 make the necessary
conversion, the result of which is
∂ψ ∂ψ ∂ψ
∇ψ = ρ̂ + φ̂ + ẑ . (16.27)
∂ρ ρ ∂φ ∂z
Table 16.3: Derivatives of the basis vectors.
RECTANGULAR
∂ x̂ ∂ x̂ ∂ x̂
=0 =0 =0
∂x ∂y ∂z
∂ ŷ ∂ ŷ ∂ ŷ
=0 =0 =0
∂x ∂y ∂z
∂ẑ ∂ẑ ∂ẑ
=0 =0 =0
∂x ∂y ∂z
CYLINDRICAL
∂ ρ̂ ∂ ρ̂ ∂ ρ̂
=0 = +φ̂ =0
∂ρ ∂φ ∂z
∂ φ̂ ∂ φ̂ ∂ φ̂
=0 = −ρ̂ =0
∂ρ ∂φ ∂z
∂ẑ ∂ẑ ∂ẑ
=0 =0 =0
∂ρ ∂φ ∂z
SPHERICAL
∂r̂ ∂r̂ ∂r̂
=0 = +θ̂ = +φ̂ sin θ
∂r ∂θ ∂φ
∂ θ̂ ∂ θ̂ ∂ θ̂
=0 = −r̂ = +φ̂ cos θ
∂r ∂θ ∂φ
∂ φ̂ ∂ φ̂ ∂ φ̂
=0 =0 = −ρ̂ = −r̂ sin θ − θ̂ cos θ
∂r ∂θ ∂φ
16.9. DERIVATIVES IN THE NONRECTANGULAR SYSTEMS 463
Again according to Table 16.1,
∂a
(b · ∇)a = bi .
∂i
Applying the cylindrical metric coefficients, we have that
∂a ∂a ∂a
(b · ∇)a = bρ + bφ + bz . (16.28)
∂ρ ρ ∂φ ∂z
Expanding the vector field a in the cylindrical basis,

∂ ∂ ∂
(b · ∇)a = bρ + bφ + bz ρ̂aρ + φ̂aφ + ẑaz .
∂ρ ρ ∂φ ∂z
Here are three derivatives of three terms, each term of two factors. Evaluat-
ing the derivatives according to the contour derivative product rule (16.23)
yields (3)(3)(2) = 0x12 (eighteen) terms in the result. Half the 0x12 terms
involve derivatives of the basis vectors, which Table 16.3 computes. Some
of the 0x12 terms turn out to be null. The result is that

∂aρ ∂aφ ∂az
(b · ∇)a = bρ ρ̂ + φ̂ + ẑ
∂ρ ∂ρ ∂ρ

bφ ∂aρ ∂aφ ∂az
+ ρ̂ − aφ + φ̂ + aρ + ẑ
ρ ∂φ ∂φ ∂φ

∂aρ ∂aφ ∂az
+ bz ρ̂ + φ̂ + ẑ . (16.29)
∂z ∂z ∂z
To evaluate divergence and curl wants more care. It also wants a constant
basis to work in, whereas [x̂ ŷ ẑ] is awkward in a cylindrical geometry and
[ρ̂ φ̂ ẑ] is not constant. Fortunately, nothing prevents us from defining a
constant basis [ρ̂o φ̂o ẑ] such that [ρ̂ φ̂ ẑ] = [ρ̂o φ̂o ẑ] at the point r = ro at
which the derivative is evaluated. If this is done, then the basis [ρ̂o φ̂o ẑ] is
constant like [x̂ ŷ ẑ] but not awkward like it.
According to Table 16.1,
∂ai
∇·a=
∂i
In cylindrical coordinates and the [ρ̂o φ̂o ẑ] basis, this is22
∂(ρ̂o · a) ∂(φ̂o · a) ∂(ẑ · a)
∇·a= + + .
∂ρ ρ ∂φ ∂z
Applying the contour derivative product rule (16.23),
∂a ∂ ρ̂o ∂a ∂ φ̂o ∂a ∂ẑ
∇ · a = ρ̂o · + · a + φ̂o · + · a + ẑ · + · a.
∂ρ ∂ρ ρ ∂φ ρ ∂φ ∂z ∂z
But [ρ̂o φ̂o z] are constant unit vectors, so
∂a ∂a ∂a
∇ · a = ρ̂o · + φ̂o · + ẑ · .
∂ρ ρ ∂φ ∂z
That is,
∂a ∂a ∂a
∇ · a = ρ̂ ·
+ φ̂ · + ẑ · .
∂ρ ρ ∂φ ∂z
Expanding the field in the cylindrical basis,

∂ ∂ ∂
∇ · a = ρ̂ · + φ̂ · + ẑ · ρ̂aρ + φ̂aφ + ẑaz .
∂ρ ρ ∂φ ∂z
As above, here again the expansion yields 0x12 terms. Fortunately, this time
most of the terms turn out to be null. The result is that
∂aρ aρ ∂aφ ∂az
∇·a= + + + ,
∂ρ ρ ρ ∂φ ∂z
or, expressed more cleverly in light of (4.28), that
∂(ρaρ ) ∂aφ ∂az
∇·a= + + . (16.30)
ρ ∂ρ ρ ∂φ ∂z
∂ak
∇ × a = ǫijk ı̂
∂j
" # " #
∂(ẑ · a) ∂(φ̂o · a) ∂(ρ̂o · a) ∂(ẑ · a)
= ρ̂o − + φ̂o −
ρ ∂φ ∂z ∂z ∂ρ
" #
∂(φ̂o · a) ∂(ρ̂o · a)
+ ẑ − .
∂ρ ρ ∂φ
22
Mistakenly to write here that
∂aρ ∂aφ ∂az
∇·a = + + ,
∂ρ ρ ∂φ ∂z
which is not true, would be a ghastly error, leading to any number of hard-to-detect false
conclusions. Refer to § 16.6.2.
That is,

∂a ∂a ∂a ∂a
∇ × a = ρ̂ ẑ · − φ̂ · + φ̂ ρ̂ · − ẑ ·
ρ ∂φ ∂z ∂z ∂ρ

∂a ∂a
+ ẑ φ̂ · − ρ̂ · .
∂ρ ρ ∂φ
Expanding the field in the cylindrical basis,

∂ ∂ ∂ ∂
∇×a = ρ̂ ẑ · − φ̂ · + φ̂ ρ̂ · − ẑ ·
ρ ∂φ ∂z ∂z ∂ρ

∂ ∂
+ ẑ φ̂ · − ρ̂ · ρ̂aρ + φ̂aφ + ẑaz .
∂ρ ρ ∂φ
Here the expansion yields 0x24 terms, but fortunately as last time this time
most of the terms again turn out to be null. The result is that

∂az ∂aφ ∂aρ ∂az ∂aφ aφ ∂aρ
∇ × a = ρ̂ − + φ̂ − + ẑ + − ,
ρ ∂φ ∂z ∂z ∂ρ ∂ρ ρ ρ ∂φ
or, expressed more cleverly, that

∂az ∂aφ ∂aρ ∂az ẑ ∂(ρaφ ) ∂aρ
∇ × a = ρ̂ − + φ̂ − + − . (16.31)
ρ ∂φ ∂z ∂z ∂ρ ρ ∂ρ ∂φ

One can compute a second-order vector derivative in cylindrical coor-
dinates as a sequence of two first-order cylindrical vector derivatives. For
example, because Table 16.1 gives the scalar Laplacian as ∇2 ψ = ∇ · ∇ψ,
one can calculate ∇2 ψ in cylindrical coordinates by taking the divergence
of ψ’s gradient.23 To calculate the vector Laplacian ∇2 a in cylindrical co-
ordinates is tedious but nonetheless can with care be done accurately by
means of Table 16.1’s identity that ∇2 a = ∇∇ · a − ∇ × ∇ × a. (This means
that to calculate the vector Laplacian ∇2 a in cylindrical coordinates takes
not just two but actually four first-order cylindrical vector derivatives, for
the author regrettably knows of no valid shortcut—the clumsy alternative,
23
A concrete example: if ψ(r) = eiφ /ρ, then ∇ψ = (−ρ̂ + iφ̂)eiφ /ρ2 per Table 16.4,
whereupon
»“ ” eiφ – “ ” „ iφ «
e eiφ “ ”
∇2 ψ = ∇ · −ρ̂ + iφ̂ 2
= −ρ̂ + iφ̂ · ∇ 2
+ 2 ∇ · −ρ̂ + iφ̂ .
ρ ρ ρ
To finish the example is left as an exercise.
Table 16.4: Vector derivatives in cylindrical coordinates.
∂ψ ∂ψ ∂ψ
∇ψ = ρ̂ + φ̂ + ẑ
∂ρ ρ ∂φ ∂z
∂a ∂a ∂a
(b · ∇)a = bρ + bφ + bz
∂ρ ρ ∂φ ∂z

∂aρ ∂aφ ∂az
= bρ ρ̂ + φ̂ + ẑ
∂ρ ∂ρ ∂ρ

bφ ∂aρ ∂aφ ∂az
+ ρ̂ − aφ + φ̂ + aρ + ẑ
ρ ∂φ ∂φ ∂φ

∂aρ ∂aφ ∂az
+ bz ρ̂ + φ̂ + ẑ
∂z ∂z ∂z
∂(ρaρ ) ∂aφ ∂az
∇·a = + +
ρ ∂ρ ρ ∂φ ∂z

∂az ∂aφ ∂aρ ∂az ẑ ∂(ρaφ ) ∂aρ
∇×a = ρ̂ − + φ̂ − + −
ρ ∂φ ∂z ∂z ∂ρ ρ ∂ρ ∂φ
less proper, less insightful, even more tedious and not recommended, be-
ing to take the Laplacian in rectangular coordinates and then to convert
back to the cylindrical domain; for to work cylindrical problems directly in
cylindrical coordinates is almost always advisable.)
16.9.2 Derivatives in spherical coordinates

One can compute vector derivatives in spherical coordinates as in cylindrical
coordinates (§ 16.9.1), only the spherical details though not essentially more
complicated are messier. According to Table 16.1,
∂ψ
∇ψ = ı̂ .
∂i
Applying the spherical metric coefficients of Table 16.2, we have that
∂ψ ∂ψ ∂ψ
∇ψ = r̂ + θ̂ + φ̂ . (16.32)
∂r r ∂θ (r sin θ) ∂φ
∂a
(b · ∇)a = bi .
∂i
Applying the cylindrical metric coefficients, we have that

∂a ∂a ∂a
(b · ∇)a = br + bθ + bφ . (16.33)
Expanding the vector field a in the spherical basis,

∂ ∂ ∂
(b · ∇)a = br + bθ + bφ r̂ar + θ̂aθ + φ̂aφ .
Evaluating the derivatives,

∂ar ∂aθ ∂aφ
(b · ∇)a = br r̂ + θ̂ + φ̂
∂r ∂r ∂r

bθ ∂ar ∂aθ ∂aφ
+ r̂ − aθ + θ̂ + ar + φ̂
r ∂θ ∂θ ∂θ

bφ ∂ar ∂aθ
+ r̂ − aφ sin θ + θ̂ − aφ cos θ
r sin θ ∂φ ∂φ

∂aφ
+ φ̂ + ar sin θ + aθ cos θ . (16.34)
∂φ
According to Table 16.1, reasoning as in § 16.9.1,
∂ai ∂a ∂a ∂a
∇·a= = r̂ · + θ̂ · + φ̂ · .
∂i ∂r r ∂θ (r sin θ) ∂φ
Expanding the field in the spherical basis,

∂ ∂ ∂
∇ · a = r̂ · + θ̂ · + φ̂ · r̂ar + θ̂aθ + φ̂aφ .
Evaluating the derivatives, the result is that
∂ar 2ar ∂aθ aθ ∂aφ
∇·a= + + + + ,
∂r r r ∂θ r tan θ (r sin θ) ∂φ
1 ∂(r 2 ar ) ∂(aθ sin θ)

∂aφ
∇·a= + + . (16.35)
r r ∂r (sin θ) ∂θ (sin θ) ∂φ
Again according to Table 16.1, reasoning as in § 16.9.1,
∂ak
∇ × a = ǫijk ı̂
∂j

∂a ∂a ∂a ∂a
= r̂ φ̂ · − θ̂ · + θ̂ r̂ · − φ̂ ·
r ∂θ (r sin θ) ∂φ (r sin θ) ∂φ ∂r

∂a ∂a
+ φ̂ θ̂ · − r̂ · .
∂r r ∂θ
Expanding the field in the spherical basis,

∂ ∂ ∂ ∂
∇×a = r̂ φ̂ · − θ̂ · + θ̂ r̂ · − φ̂ ·
r ∂θ (r sin θ) ∂φ (r sin θ) ∂φ ∂r

∂ ∂
+ φ̂ θ̂ · − r̂ · r̂ar + θ̂aθ + φ̂aφ .
∂r r ∂θ
Evaluating the derivatives, the result is that

∂aφ aφ ∂aθ ∂ar ∂aφ aφ
∇ × a = r̂ + − + θ̂ − −
r ∂θ r tan θ (r sin θ) ∂φ (r sin θ) ∂φ ∂r r

∂aθ aθ ∂ar
+ φ̂ + − ,
∂r r r ∂θ

r̂ ∂(aφ sin θ) ∂aθ θ̂ ∂ar ∂(raφ )
∇×a = − + −
r sin θ ∂θ ∂φ r (sin θ) ∂φ ∂r

φ̂ ∂(raθ ) ∂ar
+ − . (16.36)
r ∂r ∂θ

One can compute a second-order vector derivative in spherical coordi-
nates as in cylindrical coordinates, as a sequence of two first-order vector
derivatives. Refer to § 16.9.1.
16.9.3 Finding the derivatives geometrically

The method of §§ 16.9.1 and 16.9.2 is general, reliable and correct, but there
exists an alternate, arguably neater method to derive nonrectangular formu-
las for most vector derivatives. Adapting the notation to this subsection’s
purpose we can write (16.9) as
Φ
∇ · a(r) ≡ lim , (16.37)
∆V →0 ∆V
thus defining a vector’s divergence fundamentally as in § 16.1.4, geometri-

cally, as the ratio of flux Φ from
H a vanishing test volume ∆V to the volume
itself; where per (16.7) Φ = S a(r ) · ds, where r′ is a position on the test
′
volume’s surface, and where ds = ds(r′ ) is the corresponding surface patch.

Table 16.5: Vector derivatives in spherical coordinates.
∂ψ ∂ψ ∂ψ
∇ψ = r̂ + θ̂ + φ̂
∂a ∂a ∂a
(b · ∇)a = br + bθ + bφ

∂ar ∂aθ ∂aφ
= br r̂ + θ̂ + φ̂
∂r ∂r ∂r

bθ ∂ar ∂aθ ∂aφ
+ r̂ − aθ + θ̂ + ar + φ̂
r ∂θ ∂θ ∂θ

bφ ∂ar ∂aθ
+ r̂ − aφ sin θ + θ̂ − aφ cos θ
r sin θ ∂φ ∂φ

∂aφ
+ φ̂ + ar sin θ + aθ cos θ
∂φ
1 ∂(r 2 ar ) ∂(aθ sin θ)

∂aφ
∇·a = + +
r r ∂r (sin θ) ∂θ (sin θ) ∂φ

r̂ ∂(aφ sin θ) ∂aθ θ̂ ∂ar ∂(raφ )
∇×a = − + −
r sin θ ∂θ ∂φ r (sin θ) ∂φ ∂r

φ̂ ∂(raθ ) ∂ar
+ −
r ∂r ∂θ
So long as the test volume ∆V includes the point r and is otherwise in-
finitesimal in extent, we remain free to shape the volume as we like,24 so let
us give it six sides and shape it as an almost rectangular box that conforms
precisely to the coordinate system (α; β; γ) in use:
∆α ∆α
α− ≤ α′ ≤ α + ;
2 2
∆β ∆β
β− ≤ β′ ≤ β + ;
2 2
∆γ ∆γ
γ− ≤ γ′ ≤ γ + .
2 2
The fluxes outward through the box’s +α- and −α-ward sides will then be25
Φ+α = (+aα )(hβ hγ ∆β ∆γ)|r′ =r(α+∆α/2;β;γ)

= +aα hβ hγ |r′ =r(α+∆α/2;β;γ) ∆β ∆γ,
Φ−α = (−aα )(hβ hγ ∆β ∆γ)|r′ =r(α−∆α/2;β;γ)
= −aα hβ hγ |r′ =r(α−∆α/2;β;γ) ∆β ∆γ,
products of the outward-directed field components and the areas (16.24) of

the sides through which the fields pass. Thence by successive steps, the net
24
A professional mathematician would probably enjoin the volume’s shape to obey cer-
tain technical restrictions, such as that it remain wholly enclosed within a sphere of
vanishing radius, but we shall not try for such a level of rigor here.
25
More rigorously, one might digress from this point to expand the field in a three-
dimensional Taylor series (§ 8.16) to account for the field’s variation over a single side of
the test volume. So lengthy a digression however would only formalize what we already
knew; namely, that one can approximate to first order the integral of a well behaved
quantity over an infinitesimal domain by the quantity’s value at the domain’s midpoint.
R τ +∆τ /2
If you will believe that lim∆τ →0 τ −∆τ /2 f (τ ′ ) dτ ′ = f (τ ) ∆τ for any τ in the neighborhood
of which f (τ ) is well behaved, then you will probably also believe its three-dimensional
analog in the narrative. (If the vagueness in this context of the adjective “well behaved”
deeply troubles any reader then that reader may possess the worthy temperament of a
professional mathematician; he might review Ch. 8 and then seek further illumination in
the professional mathematical literature. Other readers, of more practical temperament,
are advised to visualize test volumes in rectangular, cylindrical and spherical coordinates
and to ponder the matter a while. Consider: if the field grows in strength across a single
side of the test volume and if the test volume is small enough that second-order effects
can be ignored, then what single value ought one to choose to represent the field over
the whole side but its value at the side’s midpoint? Such visualization should soon clear
up any confusion and is what the writer recommends. Incidentally, the contrast between
the two modes of thought this footnote reveals is exactly the sort of thing Courant and
Hilbert were talking about in § 1.2.1.)
flux outward through the pair of opposing sides will be
Φα = Φ+α + Φ−α
h i
= aα hβ hγ |r′ =r(α+∆α/2;β;γ) − aα hβ hγ |r′ =r(α−∆α/2;β;γ) ∆β ∆γ

∂(aα hβ hγ ) ∂(aα hβ hγ )
= ∆α ∆β ∆γ = ∆α ∆β ∆γ
∂α ∂α
∆V ∂(aα hβ hγ )
= .
hα hβ hγ ∂α
Naturally, the same goes for the other two pairs of sides:
∆V ∂(aβ hγ hα )
Φβ = ;
hα hβ hγ ∂β
∆V ∂(aγ hα hβ )
Φγ = .
hα hβ hγ ∂γ
The three equations are better written
∆V ∂ h3 aα

Φα = ,
h3 ∂α hα
∆V ∂ h3 aβ

Φβ = 3 ,
h ∂β hβ
∆V ∂ h3 aγ

Φγ = 3 ,
h ∂γ hγ
where
h3 ≡ hα hβ hγ . (16.38)
The total flux from the test volume then is
Φ = Φα + Φβ + Φγ
∂ h3 aβ ∂ h3 aγ
3
∆V ∂ h aα
= + + ;
h3 ∂α hα ∂β hβ ∂γ hγ
or, invoking Einstein’s summation convention in § 16.7’s modified style,
∆V ∂ h3 aı̃

Φ= 3 .
h ∂ı̃ hı̃
Finally, substituting the last equation into (16.37),
3
∂ h aı̃
∇·a= 3 . (16.39)
h ∂ı̃ hı̃
An analogous formula for curl is not much harder to derive but is harder
to approach directly, so we shall approach it by deriving first the formula
for γ̂-directed directional curl. Equation (16.12) has it that26
Γ
γ̂ · ∇ × a(r) ≡ lim , (16.40)
∆A→0 ∆A
where per (16.11) Γ = γ a(r′ ) · dℓ and the notation γ reminds us that the
H H
contour of integration lies in the α-β plane, perpendicular to γ̂. In this case
the contour of integration bounds not a test volume but a test surface, which
we give four edges and an almost rectangular shape that conforms precisely
to the coordinate system (α; β; γ) in use:
∆α ∆α
α− ≤ α′ ≤ α + ;
2 2
∆β ∆β
β− ≤ β′ ≤ β + ;
2 2
γ ′ = γ.
The circulations along the +α- and −α-ward edges will be
Γ+α = +hβ aβ |r′ =r(α+∆α/2;β;γ) ∆β,

Γ−α = −hβ aβ |r′ =r(α−∆α/2;β;γ) ∆β,
and likewise the circulations along the −β- and +β-ward edges will be
Γ−β = +hα aα |r′ =r(α;β−∆β/2;γ) ∆α,

Γ+β = −hα aα |r′ =r(α;β+∆β/2;γ) ∆α,
whence the total circulation about the contour is

∂(hβ aβ ) ∂(hα aα )
Γ = ∆α ∆β − ∆β ∆α
∂α ∂β

hγ ∆A ∂(hβ aβ ) ∂(hα aα )
= − .
h3 ∂α ∂β
Substituting the last equation into (16.40), we have that

hγ ∂(hβ aβ ) ∂(hα aα )
γ̂ · ∇ × a = 3 − .
h ∂α ∂β
26
The appearance of both a and A in (16.40) is unfortunate but coincidental, as is the
appearance of both γ̂ and Γ. The capital and minuscule symbols here represent unrelated
quantities.
Likewise,

hα ∂(hγ aγ ) ∂(hβ aβ )
α̂ · ∇ × a = 3 − ,
h ∂β ∂γ

hβ ∂(hα aα ) ∂(hγ aγ )
β̂ · ∇ × a = 3 − .
h ∂γ ∂α
But one can split any vector v into locally rectangular components as v =
α̂(α̂ · v) + β̂(β̂ · v) + γ̂(γ̂ · v), so
∇ × a = α̂(α̂ · ∇ × a) + β̂(β̂ · ∇ × a) + γ̂(γ̂ · ∇ × a)

α̂hα ∂(hγ aγ ) ∂(hβ aβ ) β̂hβ ∂(hα aα ) ∂(hγ aγ )
= − + −
h3 ∂β ∂γ h3 ∂γ ∂α

γ̂hγ ∂(hβ aβ ) ∂(hα aα )
+ 3 −
h ∂α ∂β

α̂hα β̂hβ γ̂hγ
1
= ∂/∂α ∂/∂β ∂/∂γ ;
h3
hα aα hβ aβ hγ aγ
or, in Einstein notation,27

ǫı̃̃k̃ ı̂hı̃ ∂(hk̃ ak̃ )
∇×a= . (16.41)
h3 ∂̃
Compared to the formulas (16.39) and (16.41) for divergence and curl,
the corresponding gradient formula is trivial. It is
ı̂ ∂ψ
∇ψ = . (16.42)
hı̃ ∂ı̃
One can generate most of the vector-derivative formulas of Tables 16.4
and 16.5 by means of this subsection’s (16.39), (16.41) and (16.42). One can
generate additional vector-derivative formulas for special coordinate systems
like the parabolic systems of § 15.7 by means of the same equations.
27
What a marvel mathematical notation is! If you can read (16.41) and understand
the message it conveys, then let us pause a moment to appreciate a few of the many
concepts the notation implicitly encapsulates. There are the vector, the unit vector, the
field, the derivative, the integral, circulation, parity, rotational invariance, nonrectangular
coordinates, three-dimensional geometry, the dummy variable and so on—each of which
concepts itself yet encapsulates several further ideas—not to mention multiplication and
division which themselves are not trivial. It is doubtful that one could explain it even
tersely to the uninitiated in fewer than fifty pages, and yet to the initiated one can express
it all in half a line.
16.10 Vector infinitesimals

To integrate a field over a contour or surface is a typical maneuver of vector
calculus. One might integrate in any of the forms
Z Z Z Z
ψ dℓ a dℓ ψ ds a ds
ZC ZC ZS ZS
ψ dℓ a · dℓ ψ ds a · ds
C C S S
Z Z
a × dℓ a × ds
C S
among others. Where the integration is over a contour, a pair of functions

α(γ) and β(γ) typically can serve to specify the contour. Where over a sur-
face, a single function γ(α; β) can serve. Given such functions and a field
integral to compute, one wants an expression for the integrand’s infinitesi-
mal dℓ or ds in terms respectively of the contour functions α(γ) and β(γ)
or of the surface function γ(α; β).
The contour infinitesimal is evidently

hα dα hβ dβ
dℓ = γ̂hγ + α̂ + β̂ dγ, (16.43)
dγ dγ
consisting of a step in the γ̂ direction plus the corresponding steps in the or-
thogonal α̂ and β̂ directions. This is easy once you see how to do it. Harder
is the surface infinitesimal, but one can nevertheless correctly construct it
as the cross product

hγ ∂γ hγ ∂γ
ds = α̂hα + γ̂ dα × β̂hβ + γ̂ dβ
∂α ∂β

1 ∂γ ∂γ
= γ̂ − α̂ − β̂ h3 dα dβ (16.44)
hγ hα ∂α hβ ∂β
of two vectors that lie on the surface, one vector normal to β̂ and the other
to α̂, edges not of a rectangular patch of the surface but of a patch whose
projection onto the α-β plane is an (hα dα)-by-(hβ dβ) rectangle.
So, that’s it. Those are the essentials of the three-dimensional geomet-
rical vector—of its analysis and of its calculus. The geometrical vector of
Chs. 15 and 16 and the matrix of Chs. 11 through 14 have in common
that they represent well developed ways of marshaling several quantities
together to a common purpose: three quantities in the specialized case of
16.10. VECTOR INFINITESIMALS 475
the geometrical vector; n quantities in the generalized case of the matrix.

Matrices and vectors have admittedly not been easy for us to treat but after
a slow start, it must be said, they have proven unexpectedly interesting. In
applications, they are exceedingly significant. Matrices and vectors vastly
expand the domain of physical phenomena a scientist or engineer can model.
Mathematically, one cannot well manage without them.
The time nevertheless has come to change the subject. Turning the
page, we shall begin from the start of the next chapter to introduce a series
of advanced topics that pick up where Ch. 9 has left off, entering first upon
the broad topic of the Fourier transform.
Part III
Transforms, special functions

and other topics
477
Chapter 17
The Fourier series
The book starts on its most advanced, most interesting mathematics from
here.
It might fairly be said that, among advanced mathematical techniques,
none is so useful, and few so appealing, as the one Lord Kelvin has acclaimed
“a great mathematical poem,”1 the Fourier transform, which this chapter
and the next will develop. This first of the two chapters will develop the
Fourier transform in its primitive guise as the Fourier series.
The Fourier series is an analog of the Taylor series of Ch. 8 but meant
for repeating waveforms, functions f (t) of which
f (t) = f (t + nT1 ), ℑ(T1 ) = 0, T1 6= 0, for all n ∈ Z, (17.1)
where T1 is the waveform’s characteristic period. Examples include the

square wave of Fig. 17.1. A Fourier series expands such a repeating waveform
as a superposition of complex exponentials or, alternately, if the waveform
is real, of sinusoids.
Suppose that you wanted to approximate the square wave of Fig. 17.1 by
a single sinusoid. You might try the sinusoid at the top of Fig. 17.2—which
is not very convincing, maybe, but if you added to the sinusoid another,
suitably scaled sinusoid of thrice the frequency then you would obtain the
somewhat better fitting curve in the figure’s middle. The curve at the fig-
ure’s bottom would yet result after you had added in four more sinusoids
respectively of five, seven, nine and eleven times the primary frequency.
1
[35, Ch. 17]
479
480 CHAPTER 17. THE FOURIER SERIES
Figure 17.1: A square wave.
f (t)
A
t
T1
Algebraically,

8A (2π)t 1 3(2π)t
f (t) = cos − cos
2π T1 3 T1

1 5(2π)t 1 7(2π)t
+ cos − cos + ··· . (17.2)
5 T1 7 T1
How faithfully (17.2) really represents the repeating waveform and why its
coefficients happen to be 1, − 31 , 15 , − 17 , . . . are among the questions this chap-
ter will try to answer; but, visually at least, it looks as though superimposing
sinusoids worked.
The chapter begins in preliminaries, starting with a discussion of Parse-
val’s principle.
17.1 Parseval’s principle

Parseval’s principle is that a step in every direction is no step at all. In the
Argand plane (Fig. 2.5), stipulated that
∆ω T1 = 2π,
ℑ(∆ω) = 0,
ℑ(to ) = 0, (17.3)
ℑ(T1 ) = 0,
T1 6= 0,
17.1. PARSEVAL’S PRINCIPLE 481
Figure 17.2: Superpositions of one, two and six sinusoids to approximate

the square wave of Fig. 17.1.
f (t)
f (t)
f (t)
t
and also that2

j, n, N ∈ Z,
n 6= 0,
(17.4)
|n| < N,
2 ≤ N,
the principle is expressed algebraically as that3

Z to +T1 /2
ein ∆ω τ dτ = 0 (17.5)
to −T1 /2
or alternately in discrete form as that

N
X −1
ei2πnj/N = 0. (17.6)
j=0
Because the product ∆ω T1 = 2π relates ∆ω to T1 , the symbols ∆ω

and T1 together represent in (17.3) and (17.5) not two but only one in-
dependent parameter. If T1 bears physical units then these typically will
be units of time (seconds, for instance), whereupon ∆ω will bear the cor-
responding units of angular frequency (such as radians per second). The
frame offset to and the dummy variable τ naturally must have the same
dimensions4 T1 has and normally will bear the same units. This matter is
discussed further in § 17.2.
To prove (17.5) symbolically is easy: one merely carries out the indi-
cated integration. To prove (17.6) symbolically is not much harder: one
replaces the complex exponential ei2πnj/N by limǫ→0+ e(i−ǫ)2πnj/N and then
uses (2.34) to evaluate the summation. Notwithstanding, we can do better,
for an alternate, more edifying, physically more insightful explanation of the
two equations is possible as follows. Because n is a nonzero integer, (17.5)
and (17.6) represent sums of steps in every direction—that is, steps in every
phase—in the Argand plane (more precisely, eqn. 17.6 represents a sum over
2
That 2 ≤ N is a redundant requirement, since (17.4)’s other lines imply it, but it
doesn’t hurt to state it anyway.
3
An expression like to ± T1 /2 means to ± (T1 /2), here and elsewhere in the book.
4
The term dimension in this context refers to the kind of physical unit. For example,
a quantity like T1 measurable in seconds or years (but not, say, in kilograms or dollars)
has dimensions of time. An automobile’s speed having dimensions of length divided by
time can be expressed in miles per hour as well as in meters per second but not directly,
say, in volts per centimeter; and so on.
17.2. TIME, SPACE AND FREQUENCY 483
a discrete but balanced, uniformly spaced selection of phases). An appeal to

symmetry forbids such sums from favoring any one phase n ∆ω τ or 2πnj/N
over any other. This being the case, how could the sums of (17.5) and (17.6)
ever come to any totals other than zero? The plain answer is that they can
come to no other totals. A step in every direction is indeed no step at all.
This is why (17.5) and (17.6) are so.5
We have actually already met Parseval’s principle, informally, in § 9.6.2.
One can translate Parseval’s principle from the Argand realm to the
analogous realm of geometrical vectors, if needed, in the obvious way.
17.2 Time, space and frequency

A frequency is the inverse of an associated period of time, expressing the
useful concept of the rate at which a cycle repeats. For example, an internal-
combustion engine whose crankshaft revolves once every 20 milliseconds—
which is to say, once every 1/3000 of a minute—runs thereby at a frequency
of 3000 RPM (revolutions per minute). Frequency however comes in two
styles: cyclic frequency (as in the engine’s example), conventionally repre-
sented by letters like ν and f ; and angular frequency, by letters like ω and k.
If T , ν and ω are letters taken to stand respectively for a period of time,
the associated cyclic frequency and the associated angular frequency, then
by definition
νT = 1,
ωT = 2π, (17.7)
ω = 2πν.
The period T will have dimensions of time like seconds. The cyclic fre-
quency ν will have dimensions of inverse time like hertz (cycles per second).6
5
The writer unfortunately knows of no conventionally established name for Parseval’s
principle. The name Parseval’s principle seems as apt as any and that is the name this
book will use.
A pedagogical knot seems to tangle Marc-Antoine Parseval’s various namesakes. Be-
cause Parseval’s principle can be extracted as a special case from Parseval’s theorem
(eqn. [not yet written] in the next chapter), the literature sometimes indiscriminately ap-
plies the name “Parseval’s theorem” to both. This is fine as far as it goes, but the knot
arrives when one needs Parseval’s principle to derive the Fourier series, which one needs
to derive the Fourier transform, which one needs in turn to derive Parseval’s theorem, at
least as this book develops them. The way to untie the knot is to give Parseval’s principle
its own name and to let it stand as an independent result.
6
Notice incidentally, contrary to the improper verbal usage one sometimes hears, that
there is no such thing as a “hert.” Rather, “Hertz” is somebody’s name. The uncapitalized
The angular frequency ω will have dimensions of inverse time like radians
per second.
The applied mathematician should make himself aware, and thereafter
keep in mind, that the cycle per second and the radian per second do not dif-
fer dimensionally from one another. Both are technically units of [second]−1 ,
whereas the words “cycle” and “radian” in the contexts of the phrases “cy-
cle per second” and “radian per second” are verbal cues that, in and of
themselves, play no actual part in the mathematics. This is not because the
cycle and the radian were ephemeral but rather because the second is un-
fundamental. The second is an arbitrary unit of measure. The cycle and the
radian are definite, discrete, inherently countable things; and, where things
are counted, it is ultimately up to the mathematician to interpret the count
(consider for instance that nine baseball hats may imply nine baseball play-
ers and one baseball team, but that there is nothing in the number nine itself
to tell us so). To distinguish angular frequencies from cyclic frequencies, it
remains to the mathematician to lend factors of 2π where needed.
The word “frequency” without a qualifying adjective is usually taken to
mean cyclic frequency unless the surrounding context implies otherwise.
Frequencies exist in space as well as in time:
kλ = 2π. (17.8)
Here, λ is a wavelength measured in meters or other units of length. The

wavenumber 7 k is an angular spatial frequency measured in units like radians
per meter. (Oddly, no conventional symbol for cyclic spatial frequency seems
to be current. The literature just uses k/2π which, in light of the potential
for confusion between ν and ω in the temporal domain, is probably for the
best.)
Where a wave propagates the propagation speed
λ ω
v= = (17.9)
T k
relates periods and frequencies in space and time.
Now, we must admit that we fibbed when we said that T had to have
dimensions of time. Physically, that is the usual interpretation, but math-
ematically T (and T1 , t, to , τ , etc.) can bear any units and indeed are not
required to bear units at all, as § 17.1 has observed. The only mathematical
form “hertz” thus is singular as well as plural.
7
The wavenumber k is no integer, notwithstanding that the letter k tends to represent
integers in other contexts.
17.3. THE SQUARE AND TRIANGULAR PULSES 485
requirement is that the product ωT = 2π (or ∆ω T1 = 2π or the like, as

appropriate) be dimensionless. However, when T has dimensions of length
rather than time it is conventional—indeed, it is practically mandatory if
one wishes to be understood—to change λ ← T and k ← ω as this section
has done, though the essential Fourier mathematics is the same regardless
of T ’s dimensions (if any) or of whether alternate symbols like λ and k are
used.
17.3 The square and triangular pulses

The Dirac delta of § 7.7 and of Fig. 7.10 is useful for the unit area it covers
among other reasons, but for some purposes its curve is too sharp. One oc-
casionally finds it expedient to substitute either the square or the triangular
pulse of Fig. 17.3,
(
1 if |t| ≤ 1/2,
Π(t) ≡
0 otherwise;
( (17.10)
1 − |t| if |t| ≤ 1,
Λ(t) ≡
0 otherwise;
for the Dirac delta, both of which pulses evidently share Dirac’s property
that
Z ∞
1 τ − to
δ dτ = 1,
−∞ T T
Z ∞
1 τ − to
Π dτ = 1, (17.11)
−∞ T T
Z ∞
1 τ − to
Λ dτ = 1,
−∞ T T
for any real T > 0 and real to . In the limit,

1 t − to
lim Π = δ(t − to ),
T →0+ T T
(17.12)
1 t − to
lim Λ = δ(t − to ),
T →0+ T T
constituting at least two possible implementations of the Dirac delta in case

such an implementation were needed.
Figure 17.3: The square and triangular pulses.
Π(t)
t
− 21 1
2
Λ(t)
t
−1 1
17.4 Expanding repeating waveforms in Fourier

series
The Fourier series represents a repeating waveform (17.1) as a superposition

of sinusoids. More precisely, inasmuch as Euler’s formula (5.17) makes a
sinusoid a sum of two complex exponentials, the Fourier series supposes
that a repeating waveform were a superposition
∞
X
f (t) = aj eij ∆ω t (17.13)
j=−∞
of complex exponentials, in which (17.3) is obeyed yet in which neither

the several Fourier coefficients aj nor the waveform f (t) itself need be real.
Whether one can properly represent every repeating waveform as a super-
position (17.13) of complex exponentials is a question §§ 17.4.4 and 17.6 will
address later; but, at least to the extent to which one can properly represent
such a waveform, we shall now assert that one can recover any or all of the
waveform’s Fourier coefficients aj by choosing an arbitrary frame offset to
17.4. EXPANDING WAVEFORMS IN FOURIER SERIES 487
(to = 0 is a typical choice) and then integrating
to +T1 /2
1
Z
aj = e−ij ∆ω τ f (τ ) dτ. (17.14)
T1 to −T1 /2
17.4.1 Derivation of the Fourier-coefficient formula
Why does (17.14) work? How does it recover a Fourier coefficient aj ? The
answer is that it recovers a Fourier coefficient aj by isolating it, and that it
isolates it by shifting frequencies and integrating. It shifts the jth component
aj eij ∆ω t of (17.13)’s waveform—whose angular frequency is ω = j ∆ω—
down to a frequency of zero, incidentally shifting the waveform’s several
other components to various nonzero frequencies as well. Significantly, it
leaves each shifted frequency to be a whole multiple of the waveform’s funda-
mental frequency ∆ω. By Parseval’s principle (17.5), the integral of (17.14)
then kills all the thus frequency-shifted components except the zero-shifted
one by integrating the components over complete cycles, passing only the
zero-shifted component which, once shifted, has no cycle. Changing dummy
variables τ ← t and ℓ ← j in (17.13) and then substituting into (17.14)’s
right side the resulting expression for f (τ ), we have by successive steps that
to +T1 /2
1
Z
e−ij ∆ω τ f (τ ) dτ
T1 to −T1 /2
to +T1 /2 ∞
1
Z X
= e−ij ∆ω τ aℓ eiℓ ∆ω τ dτ
T1 to −T1 /2 ℓ=−∞
∞ Z to +T1 /2
1 X
= aℓ ei(ℓ−j) ∆ω τ dτ
T1 to −T1 /2
ℓ=−∞
Z to +T1 /2
aj
= ei(j−j) ∆ω τ dτ
T1 to −T1 /2
aj to +T1 /2
Z
= dτ = aj ,
T1 to −T1 /2
in which Parseval’s principle (17.5) has killed all but the ℓ = j term in the
summation. Thus is (17.14) formally proven.
17.4.2 The square wave

According to (17.14), the Fourier coefficients of Fig. 17.1’s square wave are,
if to = T1 /4 is chosen and by successive steps,
3T1 /4
1
Z
aj = e−ij ∆ω τ f (τ ) dτ
T1 −T1 /4
"Z #
T1 /4 3T1 /4
A
Z
= − e−ij ∆ω τ dτ
T1 −T1 /4 T1 /4
" 3T1 /4 #
iA −ij ∆ω τ T1 /4
= e − .
2πj
−T1 /4

T1 /4
But
e−ij ∆ω τ τ =−T1 /4 = e−ij ∆ω τ τ =3T1 /4 = ij ,

e−ij ∆ω τ = (−i)j ,

τ =T1 /4
so
" 3T1 /4 #
−ij ∆ω τ
T1 /4
e
−
−T1 /4 T1 /4
= [(−i) − i ] − [ij − (−i)j ] = 2[(−i)j − ij ]

j j
= . . . , −i4, 0, i4, 0, −i4, 0, i4, . . . for j = . . . , −3, −2, −1, 0, 1, 2, 3, . . .
Therefore,
i2A
aj = (−i)j − ij

2πj
(
(j−1)/2 (17.15)
(−) 4A/2πj for odd j,
=
0 for even j
are the square wave’s Fourier coefficients which, when the coefficients are
applied to (17.13) and when (5.17) is invoked, indeed yield the specific series
of sinusoids (17.2) and Fig. 17.2 have proposed.
17.4.3 The rectangular pulse train

The square wave of § 17.4.2 is an important, canonical case and (17.2) is
arguably worth memorizing. After the square wave however the variety of
Figure 17.4: A rectangular pulse train.
f (t)
A
t
T1 ηT1
possible repeating waveforms has no end. Whenever an unfamiliar repeating

waveform arises, one can calculate its Fourier coefficients (17.14) on the spot
by the straightforward routine of § 17.4.2. There seems little point therefore
in trying to tabulate waveforms here.
One variant on the square wave nonetheless is interesting enough to
attract special attention. This variant is the pulse train of Fig. 17.4,
∞
X t − jT1
f (t) = A Π ; (17.16)
ηT1
j=−∞
where Π(·) is the square pulse of (17.10); the symbol A represents the pulse’s
full height rather than the half-height of Fig. 17.1; and the dimensionless
factor 0 ≤ η ≤ 1 is the train’s duty cycle, the fraction of each cycle its pulse
is as it were on duty. By the routine of § 17.4.2,
T1 /2
1
Z
aj = e−ij ∆ω τ f (τ ) dτ
T1 −T1 /2
ηT1 /2
A
Z
= e−ij ∆ω τ dτ
T1 −ηT1 /2
iA −ij ∆ω τ ηT1 /2

2A 2πηj
= e = sin
2πj
−ηT1 /2 2πj 2
for j 6= 0. On the other hand,
T1 /2 ηT1 /2
1 A
Z Z
a0 = f (τ ) dτ = dτ = ηA,
T1 −T1 /2 T1 −ηT1 /2
Figure 17.5: A Dirac delta pulse train.
f (t)
t
T1
the waveform’s mean value. Altogether for the pulse train,
 2A sin 2πηj if j 6= 0,

aj = 2πj 2 (17.17)
ηA if j = 0.

Equation (17.25) will improve the notation of (17.17) later.

An especially interesting special case occurs when the duty cycle grows
very short. Since limη→0+ sin(2πηj/2) = 2πηj/2 according to (8.32), it
follows from (17.17) that
lim aj = ηA, (17.18)
η→0+
the same for every index j. As the duty cycle η tends to vanish the pulse
tends to disappear and the Fourier coefficients along with it; but we can com-
pensate for vanishing duty if we wish by increasing the pulse’s amplitude A
proportionally, maintaining the product
ηT1 A = 1 (17.19)
of the pulse’s width ηT1 and its height A, thus preserving unit area8 under
the pulse. In the limit η → 0+ , the pulse then by definition becomes the
Dirac delta of Fig. 7.10, and the pulse train by construction becomes the
Dirac delta pulse train of Fig. 17.5. Enforcing (17.19) on (17.18) yields the
8
In light of the discussion of time, space and frequency in § 17.2, we should clarify
that we do not here mean a physical area measurable in square meters or the like. We
merely mean the dimensionless product of the width (probably measured in units of time
like seconds) and the height (correspondingly probably measured in units of frequency like
inverse seconds) of the rectangle a single pulse encloses in Fig. 17.4. The term area is thus
overloaded.
Dirac delta pulse train’s Fourier coefficients
1
aj = . (17.20)
T1
17.4.4 Linearity and sufficiency

The Fourier series is evidently linear according to the rules of § 7.3.3. That
is, if the Fourier coefficients of f1 (t) are aj1 and the Fourier coefficients
of f2 (t) are aj2 , and if the two waveforms f1 (t) and f2 (t) share the same
fundamental period T1 , then the Fourier coefficients of f (t) = f1 (t) + f2 (t)
are aj = aj1 + aj2 . Likewise, the Fourier coefficients of αf (t) are αaj and
the Fourier coefficients of the null waveform fnull (t) ≡ 0 are themselves null,
thus satisfying the conditions of linearity.
All this however supposes that the Fourier series actually works.9
Though Fig. 17.2 is suggestive, the figure alone hardly serves to demon-
strate that every repeating waveform were representable as a Fourier series.
To try to consider every repeating waveform at once would be too much
to try at first in any case, so let us start from a more limited question:
does there exist any continuous, repeating waveform10 f (t) 6= 0 of period T1
whose Fourier coefficients aj = 0 are identically zero?
If the waveform f (t) in question is continuous then nothing prevents us
from discretizing (17.14) as
M
1 X (−ij ∆ω)(to +ℓ ∆τM )
aj = lim e f (to + ℓ ∆τM ) ∆τM ,
M →∞ T1
ℓ=−M
T1
∆τM ≡ ,
2M + 1
and further discretizing the waveform itself as

∞
X t − (to + p ∆τM )
f (t) = lim f (to + p ∆τM ) Π ,
M →∞
p=−∞
∆τM
in which Π[·] is the square pulse of (17.10). Substituting the discretized

9
The remainder of this dense subsection can be regarded as optional reading.
10
As elsewhere in the book, the notation f (t) 6= 0 here forbids only the all-zero waveform.
It does not forbid waveforms like f (t) = A sin ωt that happen to take a zero value at certain
values of t.
waveform into the discretized formula for aj , we have that
M ∞
∆τM X X
aj = lim e(−ij ∆ω)(to +ℓ ∆τM ) f (to + p ∆τM )Π(ℓ − p)
M →∞ T1
ℓ=−M p=−∞
M
∆τM X (−ij ∆ω)(to +ℓ ∆τM )
= lim e f (to + ℓ ∆τM ).
M →∞ T1
ℓ=−M
If we define the (2M + 1)-element vectors and (2M + 1) × (2M + 1) matrix
[fM ]ℓ ≡ f (to + ℓ ∆τM ),

[aM ]j ≡ aj ,
∆τM (−ij ∆ω)(to +ℓ ∆τM )
[CM ]jℓ ≡ e ,
T1
−M ≤ (j, ℓ) ≤ M,
then matrix notation renders the last equation as
lim aM = lim CM fM ,
M →∞ M →∞
whereby
−1
lim fM = lim CM aM ,
M →∞ M →∞
assuming that CM is invertible.

But is CM invertible? This seems a hard question to answer until we
realize that the rows of CM consist of sampled complex exponentials which
repeat over the interval T1 and thus stand subject to Parseval’s princi-
ple (17.6). Realizing this, we can do better than merely to state that CM is
invertible: we can write down its actual inverse,
−1 T1
[CM ]ℓj = e(+ij ∆ω)(to +ℓ ∆τM ) ,
(2M + 1) ∆τM
−1 M and thus per (13.2) also that C −1 C
such that11 CM CM = I−M M
M M = I−M .
So, the answer to our question is that, yes, CM is invertible.
Because CM is invertible, § 14.2 has it that neither fM nor aM can be
null unless both are. In the limit M → ∞, this implies that no continuous,
11 M
Equation (11.30) has defined the notation I−M , representing a (2M + 1)-dimensional
identity matrix whose string of ones extends along its main diagonal from j = ℓ = −M
through j = ℓ = M .
repeating waveform f (t) 6= 0 exists whose Fourier coefficients aj = 0 are

identically zero.
Now consider a continuous, repeating waveform F (t) and its Fourier
series f (t). Let ∆F (t) ≡ F (t)−f (t) be the part of F (t) unrepresentable as a
Fourier series, continuous because both F (t) and f (t) are continuous. Being
continuous and unrepresentable as a Fourier series, ∆F (t) has null Fourier
coefficients; but as the last paragraph has concluded this can only be so if
∆F (t) = 0. Hence, ∆F (t) = 0 indeed, which implies12 that f (t) = F (t).
In other words, every continuous, repeating waveform is representable as a
Fourier series.
And what of discontinuous waveforms? Well, the square wave of Figs.
17.1 and 17.2 this chapter has posed as its principal example is a repeat-
ing waveform but, of course, not a continuous one. A truly discontinuous
waveform would admittedly invalidate the discretization above of f (t), but
see: nothing prevents us from approximating the square wave’s discontinuity
by an arbitrarily steep slope, whereupon this subsection’s conclusion again
applies.13
The better, more subtle, more complete answer to the question though
is that a discontinuity incurs Gibbs’ phenomenon, which § 17.6 will derive.
17.4.5 The trigonometric form
It is usually best, or at least neatest and cleanest, and moreover more evoca-
tive, to calculate Fourier coefficients and express Fourier series in terms of
complex exponentials as (17.13) and (17.14) do. Occasionally, though, when
the repeating waveform f (t) is real, one prefers to work in sines and cosines
rather than in complex exponentials. One writes (17.13) by Euler’s for-
mula (5.11) as
∞
X
f (t) = a0 + [(aj + a−j ) cos j ∆ω t + i(aj − a−j ) sin j ∆ω t] .
j=1
12
Chapter 8’s footnote 6 has argued in a similar style, earlier in the book.
13
Where this subsection’s conclusion cannot be made to apply is where unreasonable
waveforms like A sin[B/ sin ωt] come into play. We shall leave to the professional mathe-
matician the classification of such unreasonable waveforms, the investigation of the wave-
forms’ Fourier series and the provision of greater rigor generally.
Then, superimposing coefficients in (17.14),

to +T1 /2
1
Z
a0 = f (τ ) dτ,
T1 to −T1 /2
Z to +T1 /2
2
bj ≡ (aj + a−j ) = cos(j ∆ω τ )f (τ ) dτ, (17.21)
T1 to −T1 /2
to +T1 /2
2
Z
cj ≡ i(aj − a−j ) = sin(j ∆ω τ )f (τ ) dτ,
T1 to −T1 /2
which give the Fourier series the trigonometric form

∞
X
f (t) = a0 + (bj cos j ∆ω t + cj sin j ∆ω t) . (17.22)
j=1
The complex conjugate of (17.14) is

to +T1 /2
1
Z
a∗j = e+ij ∆ω τ f ∗ (τ ) dτ.
T1 to −T1 /2
If the waveform happens to be real then f ∗ (t) = f (t), which in light of the
last equation and (17.14) implies that
a−j = a∗j if ℑ[f (t)] = 0. (17.23)
Combining (17.21) and (17.23), we have that

)
bj = 2ℜ(aj )
if ℑ[f (t)] = 0. (17.24)
cj = −2ℑ(aj )
17.5 The sine-argument function

Equation (17.17) gives the pulse train of Fig. 17.4 its Fourier coefficients,
but a better notation for (17.17) is
2πηj
aj = ηA Sa , (17.25)
2
where
sin z
Sa z ≡ (17.26)
z
17.5. THE SINE-ARGUMENT FUNCTION 495
Figure 17.6: The sine-argument function.
sin t
Sa t ≡
t
1
−2π − 2π
2
2π
2 2π
t
is the sine-argument function,14 plotted in Fig. 17.6. The function’s Taylor

series is " j
∞
#
X Y −z 2
Sa z = , (17.27)
(2m)(2m + 1)
j=0 m=1
the Taylor series of sin z from Table 8.1, divided by z.

This section introduces the sine-argument function and some of its prop-
erties, plus also the related sine integral.
17.5.1 Derivative and integral

The sine-argument function’s derivative is computed from the definition
(17.26) and the derivative product rule (4.26) to be
d cos z − Sa z
Sa z = . (17.28)
dz z
The function’s integral is expressed as a Taylor series after integrating the
function’s own Taylor series (17.27) term by term to obtain the form
∞ j
" #
z
z −z 2
Z X Y
Si z ≡ Sa τ dτ = , (17.29)
0 2j + 1 m=1 (2m)(2m + 1)
j=0
14
Many (including the author himself in other contexts) call it the sinc function, denot-
ing it sinc(·) and pronouncing it as “sink.” Unfortunately, some [48, § 4.3][13, § 2.2][18]
use the sinc(·) notation for another function,
2πz sin(2πz/2)
sincalternate z ≡ Sa = .
2 2πz/2
The unambiguous Sa(·) suits this particular book better, anyway, so this is the notation
we will use.
Figure 17.7: The sine integral.
Rt
Si t ≡ 0
Sa τ dτ
2π
4
− 2π 1
−2π 2
t
2π 2π
−1 2
− 2π
4
plotted in Fig. 17.7. Convention gives this integrated function its own name
and notation: it calls it the sine integral 15 ,16 and denotes it by Si(·).
17.5.2 Properties of the sine-argument function

Sine-argument properties include the following.
• The sine-argument function is real over the real domain. That is, if
ℑ(t) = 0 then ℑ(Sa t) = 0.
• The zeros of Sa z occur at z = nπ, n 6= 0, n ∈ Z.
• It is that |Sa t| < 1 over the real domain ℑ(t) = 0 except at the global
maximum t = 0, where
Sa 0 = 1. (17.30)
• Over the real domain ℑ(t) = 0, the function Sa t alternates between

distinct, positive and negative lobes. That is, (−)n Sa(±t) > 0 over
nπ < t < (n + 1)π for each n ≥ 0, n ∈ Z.
• Each of the sine-argument’s lobes has but a single peak. That is, over
the real domain ℑ(t) = 0, the derivative (d/dt) Sa t = 0 is zero at only
a single value of t on each lobe.
15
[43, § 3.3]
16
The name “sine-argument” incidentally seems to have been back-constructed from the
name “sine integral.”
• The sine-argument function and its derivative converge toward
lim Sa t = 0,
t→±∞
d (17.31)
lim Sa t = 0.
t→±∞ dt
Some of these properties are obvious in light of the sine-argument function’s

definition (17.26). Among the less obvious properties, that |Sa t| < 1 says
merely that |sin t| < |t| for nonzero t—which must be true since t, interpreted
as an angle or a curved distance about a unit circle, can hardly be shorter
than sin t, interpreted as the corresponding direct shortcut to the axis (see
Fig. 3.1). For t = 0, (8.32) obtains—or, if you prefer, (17.27).
That each of the sine-argument function’s lobes should have but a single
peak seems right in view of Fig. 17.6 but is nontrivial to prove. To assert
that each lobe has but a single peak is to assert that (d/dt) Sa t = 0 exactly
once in each lobe; or, equivalently—after setting (17.28)’s left side to zero,
multiplying by z 2 / cos z and changing t ← z—it is to assert that
tan t = t
exactly once in each interval
nπ ≤ t < (n + 1)π, n ≥ 0,
for t ≥ 0; and similarly for t ≤ 0. But according to Table 5.2

d 1
tan t = ≥ 1,
dt cos2 t
whereas dt/dt = 1, implying that tan t is everywhere at least as steep as t
is—and, well, the most concise way to finish the argument is to draw a
picture of it, as in Fig. 17.8, where the curves evidently cannot but intersect
exactly once in each interval.
17.5.3 Properties of the sine integral

Properties of the sine integral Si t of (17.29) include the following.
• Over the real domain, ℑ(t) = 0, the sine integral Si t is positive for
positive t, negative for negative t and, of course, zero for t = 0.
• The local extrema of Si t over the real domain ℑ(t) = 0 occur at the
zeros of Sa t.
Figure 17.8: The points at which t intersects tan t.
f (t)
− 2π
2
t
2π
2
t
tan t
• The global maximum and minimum of Si t over the real domain ℑ(t) =
0 occur respectively at the first positive and negative zeros of Sa t,
which are t = ±π.
• The sine integral converges toward

2π
lim Si t = ± . (17.32)
t→±∞ 4
That the sine integral should reach its local extrema at the sine-argument’s
zeros ought to be obvious to the extent to which the concept of integration is
understood. To explain the other properties it helps first to have expressed
the sine integral in the form
Z t
Si t = Sn + Sa τ dτ,
nπ
n−1
X
Sn ≡ Uj ,
j=0
Z (j+1)π
Uj ≡ Sa τ dτ,
jπ
nπ ≤ t < (n + 1)π,
0 ≤ n, (j, n) ∈ Z,
where each partial integral Uj integrates over a single lobe of the sine-
argument. The several Uj alternate in sign but, because each lobe majorizes
the next (§ 8.10.2)—that is, because,17 in the integrand, |Sa τ | ≥ |Sa τ + π|
for all τ ≥ 0—the magnitude of the area under each lobe exceeds that under
the next, such that
Z t
j
0 ≤ (−) Sa τ dτ < (−)j Uj < (−)j−1 Uj−1 ,
jπ
jπ ≤ t < (j + 1)π,
0 ≤ j, j ∈ Z
(except that the Uj−1 term of the inequality does not apply when j = 0,
since there is no U−1 ) and thus that
0 = S0 < S2m < S2m+2 < S∞ < S2m+3 < S2m+1 < S1
for all m > 0, m ∈ Z.
The foregoing applies only when t ≥ 0 but naturally one can reason similarly
for t ≤ 0, concluding that the integral’s global maximum and minimum
over the real domain occur respectively at the sine-argument function’s first
positive and negative zeros, t = ±π; and further concluding that the integral
is positive for all positive t and negative for all negative t.
Equation (17.32) wants some cleverness to calculate and will be the sub-
ject of the next subsection.
17.5.4 The sine integral’s limit by complex contour

Equation (17.32) has proposed that the sine integral converges toward a
value of 2π/4, but why? The integral’s Taylor series (17.29) is impractical
to compute for large t and is useless for t → ∞, so it cannot answer the
question. To evaluate the integral in the infinite limit, we shall have to
think of something cleverer.
Noticing per (5.18) that
e+iz − e−iz
Sa z = ,
i2z
rather than trying to integrate the sine-argument function all at once let us
first try to integrate just one of its two complex terms, leaving the other
17
More rigorously, to give the reason perfectly unambiguously, one could fuss here for a
third of a page or so over signs, edges and the like. To do so is left as an exercise to those
that aspire to the pure mathematics profession.
Figure 17.9: A complex contour about which to integrate eiz /i2z.
ℑ(z)
I3
I4 I2
I6
I5 I1
ℜ(z)
term aside to handle later, for the moment computing only

Z ∞ iz
e dz
I1 ≡ .
0 i2z
To compute the integral I1 , we shall apply the closed-contour technique of
§ 9.5, choosing a contour in the Argand plane that incorporates I1 but shuts
out the integrand’s pole at z = 0.
Many contours are possible and one is unlikely to find an amenable
contour on the first attempt, but perhaps after several false tries we discover
and choose the contour of Fig. 17.9. The integral about the inner semicircle
of this contour is
Z 0 Z 0
eiz dz eiz (iρeiφ dφ) ei0 dφ 2π
Z
I6 = = lim iφ
= =− .
C6 i2z ρ→0+ 2π/2 i2(ρe ) 2π/2 2 4
The integral across the contour’s top segment is

Z −a i(x+ia) Z a
eiz dz e dx −eix e−a dx
Z
I3 = = lim = lim ,
C3 i2z a→∞ a i2z a→∞ −a i2z
from which, according to the continuous triangle inequality (9.15),
Z a ix −a Z a −a
−e e dx e dx
|I3 | ≤ lim
= a→∞
lim ;
a→∞ −a i2z −a 2 |z|
which, since 0 < a ≤ |z| across the segment, we can weaken to

Z a −a
e dx
|I3 | ≤ lim = lim e−a = 0,
a→∞ −a 2a a→∞
only possible if
I3 = 0.
The integral up the contour’s right segment is
Z a i(a+iy) Z a ia −y
eiz dz e dy e e dy
Z
I2 = = lim = lim ,
C2 i2z a→∞ 0 2z a→∞ 0 2z
from which, according to the continuous triangle inequality,

Z a ia −y Z a −y
e e dy e dy
|I2 | ≤ lim = lim ;
a→∞ 0 2z a→∞ 0 2 |z|
which, since 0 < a ≤ |z| across the segment, we can weaken to

Z a −y
e dy 1
|I2 | ≤ lim = lim = 0,
a→∞ 0 2a a→∞ 2a
only possible if
I2 = 0.
The integral down the contour’s left segment is
I4 = 0
for like reason. Because the contour encloses no pole,

I iz
e dz
= I1 + I2 + I3 + I4 + I5 + I6 = 0,
i2z
which in light of the foregoing calculations implies that
2π
I1 + I5 = .
4
Now,
∞
eiz dz eix dx
Z Z
I1 = =
C1 i2z 0 i2x
is the integral we wanted to compute in the first place, but what is that I5 ?
Answer: Z 0 ix
eiz dz e dx
Z
I5 = = ;
C5 i2z −∞ i2x
or, changing −x ← x,
∞
−e−ix dx
Z
I5 = ,
0 i2x
which fortuitously happens to integrate the heretofore neglected term of the

sine-argument function we started with. Thus,
∞ ∞
e+ix − e−ix 2π
Z Z
lim Si t = Sa x dx = dx = I1 + I5 = ,
t→∞ 0 0 i2x 4
which was to be computed.18
17.6 Gibbs’ phenomenon

Section 17.4.4 has shown how the Fourier series suffices to represent a con-
tinuous, repeating waveform. Paradoxically, the chapter’s examples have
been of discontinuous waveforms like the square wave. At least in Fig. 17.2
the Fourier series seems to work for such discontinuous waveforms, though
we have never exactly demonstrated that it should work for them, or how.
So, what does all this imply?
In one sense, it does not imply much of anything. One can represent a
discontinuity by a relatively sharp continuity—as for instance one can rep-
resent the Dirac delta of Fig. 7.10 by the triangular pulse of Fig. 17.3, with
its sloped edges, if T in (17.12) is sufficiently small—and, considered in this
light, the Fourier series works. Mathematically however one is more likely
to approximate a Fourier series by truncating it after some finite number N
of terms; and, indeed, so-called19 “low-pass” physical systems that naturally
suppress high frequencies20 are common, in which case to truncate the se-
ries is more or less the right thing to do. Yet, a significant thing happens
18
Integration by closed contour is a subtle technique, is it not? What a finesse this
subsection’s calculation has been! The author rather strongly sympathizes with the reader
who still somehow cannot quite believe that contour integration actually works, but in
the case of the sine integral another, quite independent method to evaluate the integral
is known and it finds the same number 2π/4. The interested reader can extract this
other method from Gibbs’ calculation in § 17.6, which refers a sine integral to the known
amplitude of a square wave.
We said that it was fortuitous that I5 , which we did not know how to eliminate, turned
out to be something we needed anyway; but is it really merely fortuitous, once one has
grasped the technique? An integration of −e−iz /i2z is precisely the sort of thing an
experienced applied mathematician would expect to fall out as a byproduct of the contour
integration of eiz /i2z. The trick is to discover the contour from which it actually does fall
out, the discovery being a process of informed trial and error.
19
So called because they pass low frequencies while suppressing high ones, though sys-
tems encountered in practice admittedly usually suffer a middle frequency domain through
which frequencies are only partly suppressed.
20
[35, § 15.2]
17.6. GIBBS’ PHENOMENON 503
when one truncates the Fourier series. At a discontinuity, the Fourier series
oscillates and overshoots.21
Henry Wilbraham investigated this phenomenon as early as 1848.
J. Willard Gibbs explored its engineering implications in 1899.22 Let us
along with them refer to the square wave of Fig. 17.2 on page 481. As
further Fourier components are added the Fourier waveform better approx-
imates the square wave, but, as we said, it oscillates and overshoots—it
“rings,” in the electrical engineer’s vernacular—about the square wave’s
discontinuities. This oscillation and overshot turn out to be irreducible, and
moreover they can have significant physical effects.
Changing t − T1 /4 ← t in (17.2) to delay the square wave by a quarter
cycle yields
∞
8A X 1 (2j + 1)(2π)t
f (t) = sin ,
2π 2j + 1 T1
j=0
which we can, if we like, write as

N −1
8A X 1 (2j + 1)(2π)t
f (t) = lim sin .
N →∞ 2π 2j + 1 T1
j=0
Again changing
2(2π)t
∆v ←
T1
makes this
N −1
T1 4A X 1
f ∆v = lim Sa j + ∆v ∆v.
2(2π) N →∞ 2π 2
j=0
Stipulating that ∆v be infinitesimal,
0 < ∆v ≪ 1,
such that dv ≡ ∆v and, therefore, that the summation become an integra-

tion; and further defining
u ≡ N ∆v;
we have that
4A u

T1 4A
Z
lim f u = Sa v dv = Si u. (17.33)
N →∞ 2(2π)N 2π 0 2π
21
[38]
22
[68][25]
Figure 17.10: Gibbs’ phenomenon.
f (t)
A
t
Now, (17.32) gives us that limu→∞ Si u = 2π/4, so (17.33) as it should has

it that f (t) ≈ A when23 t ≩ 0. When t ≈ 0 however it gives the waveform
locally the sine integral’s shape of Fig. 17.7.
Though unexpected the effect can and does actually arise in physical
systems. When it does, the maximum value of f (t) is of interest to me-
chanical and electrical engineers among others because, if an element in an
engineered system will overshoot its designed position, the engineer wants
to allow safely for the overshot. According to § 17.5.3, the sine integral Si u
reaches its maximum at
2π
u= ,
2
where according to (17.29)
∞ j
" #
4A 2π 4A X 2π/2 Y −(2π/2)2
fmax = Si = ≈ (0x1.2DD2)A.
2π 2 2π 2j + 1 (2m)(2m + 1)
j=0 m=1
This overshot, peaking momentarily at (0x1.2DD2)A, and the associated

sine-integral ringing constitute Gibbs’ phenomenon, as Fig. 17.10 depicts.
We have said that Gibbs’ phenomenon is irreducible, and indeed strictly
this is so: a true discontinuity, if it is to obey Fourier, must overshoot
according to Gibbs. Admittedly as earlier alluded, one can sometimes sub-
stantially evade Gibbs by softening a discontinuity’s edge, giving it a steep
but not vertical slope and maybe rounding its corners a little;24 or, alter-
23
Here is an exotic symbol: ≩. It means what it appears to mean, that t > 0 and t 6≈ 0.
24
If the applied mathematician is especially exacting he might represent a discontinuity
by the probability integral of [not yet written], and indeed there are times at which he
might do so. However, such extra-fine mathematical craftsmanship is unnecessary to this
section’s purpose.
17.6. GIBBS’ PHENOMENON 505
nately, by rolling the Fourier series off gradually rather than truncating it
exactly at N terms. Engineers may do one or the other, or both, explicitly or
implicitly, which is why the full Gibbs is not always observed in engineered
systems. Nature may do likewise. Neither however is the point. The point
is that sharp discontinuities do not behave in the manner one might naı̈vely
have expected, yet that one can still analyze them profitably, adapting this
section’s subtle technique as the circumstance might demand. A good en-
gineer or other applied mathematician will make himself aware of Gibbs’
phenomenon and of the mathematics behind it for this reason.
Chapter 18
The Fourier and Laplace

transforms
The Fourier series of Ch. 17 though useful applies solely to waveforms that
repeat. An effort to generalize the Fourier series to the broader domain of
nonrepeating waveforms leads to the Fourier transform, this chapter’s chief
subject.
[Chapter to be written.]
507
508 CHAPTER 18. THE FOURIER AND LAPLACE TRANSFORMS
Plan
The following chapters are tentatively planned to complete the book.1
19. Probability2
20. Transformations to speed series convergence3
21. Basic special functions4,5
22. The wave equation6
23. Cylinder functions

1
The end of Ch. 18 is tentatively to introduce spatial Fourier transforms.
2
Chapter 19 is tentatively to treat at least the Gaussian, Rayleigh and Poisson dis-
tributions plus the basic concept of sample variance. Unless the mathematics grow too
twisted, demanding deferral to Ch. 21, Ch. 19 also hopes to develop the properties of the
probability integral—even if it requires some forward references to Ch. 21 to do so. It
would be nice if the chapter could treat ideal-gas mechanics.
3
Chapter 20 is tentatively to treat at least the Poisson sum formula, Mosig’s
summation-by-parts technique and, the author believes, the Watson transformation; plus
maybe some others as seems appropriate. This might also be a good chapter in which
to develop the infinite-product forms of the sine and the cosine and thence Euler’s and
Andrews’ clever closed-form series summations from [1, § 1.7 and exercises] and maybe
from other, similar sources.
4
Chapter 21 is tentatively to treat at least the gamma function, the exponential in-
tegral, the sine and cosine integrals, and the Fresnel integral. It hopes to try to avoid
developing the asymptotic large-argument form specifically of the gamma function, which
needs a messy derivation, but it might at least develop Stirling’s formula without error
bounds. The contents of this chapter must be driven in part by the needs of later chapters.
5
The author does not yet feel sure of just how many special functions and associated
properties the book ought to develop, an interesting but potentially endless effort. The
book certainly does not mean to cover anything like the whole of [1][43][64], else the book
would reach four-digit page numbers which are not wanted!
6
Chapter 22 might begin with Poisson’s equation and the corresponding static case.
After treating the wave equation proper, it might end with the parabolic wave equation.
509
510 CHAPTER 18. THE FOURIER AND LAPLACE TRANSFORMS
24. Orthogonal polynomials7,8
25. Numerical integration
26. The conjugate-gradient algorithm
27. Remarks
Chapters are likely yet to be inserted, removed, divided, combined and shuf-
fled, but that’s the planned outline at the moment.
The book means to stop short of hypergeometric functions, parabolic
cylinder functions, selective-dimensional (Weyl and Sommerfeld) Fourier
transforms, wavelets, and iterative techniques more advanced than the con-
jugate-gradient (the advanced iterative techniques being too active an area
of research for such a book as this yet to treat). The book may however add
a new appendix “Additional derivations” before the existing Appendix D to
prove a few obscure results.9
7
Chapter 24 would be pretty useless if it did not treat Legendre polynomials, so pre-
sumably it will do at least this.
8
The author has not yet decided how to apportion the treatment of the wave equation
in spherical geometries between Chs. 22, 23 and 24.
9
Additional derivations in the new appendix might include those of distributed resis-
tance, of resistance to ground and of the central angle in a methane molecule or other
tetrahedron.
Appendices
511
Appendix A
Hexadecimal and other

notational matters
The importance of conventional mathematical notation is hard to overstate.

Such notation serves two distinct purposes: it conveys mathematical ideas
from writer to reader; and it concisely summarizes complex ideas on paper
to the writer himself. Without the notation, one would find it difficult even
to think clearly about the math; to discuss it with others, nearly impossible.
The right notation is not always found at hand, of course. New mathe-
matical ideas occasionally find no adequate preëstablished notation, when it
falls to the discoverer and his colleagues to establish new notation to meet
the need. A more difficult problem arises when old notation exists but is
inelegant in modern use.
Convention is a hard hill to climb, and rightly so. Nevertheless, slavish
devotion to convention does not serve the literature well; for how else can
notation improve over time, if writers will not incrementally improve it?
Consider the notation of the algebraist Girolamo Cardano in his 1539 letter
to Tartaglia:
[T]he cube of one-third of the coefficient of the unknown is
greater in value than the square of one-half of the number. [46]
If Cardano lived today, surely he would express the same thought in the
form a 3 x 2
> .
3 2
Good notation matters.
Although this book has no brief to overhaul applied mathematical nota-
tion generally, it does seek to aid the honorable cause of notational evolution
513
514 APPENDIX A. HEXADECIMAL NOTATION, ET AL.
in a few specifics. For example, the book sometimes treats 2π implicitly as

a single symbol, so that (for instance) the quarter revolution or right angle
is expressed as 2π/4 rather than as the less evocative π/2.
As a single symbol, of course, 2π remains a bit awkward. One wants to
introduce some new symbol ξ = 2π thereto. However, it is neither necessary
nor practical nor desirable to leap straight to notational Utopia in one great
bound. It suffices in print to improve the notation incrementally. If this
book treats 2π sometimes as a single symbol—if such treatment meets the
approval of slowly evolving convention—then further steps, the introduction
of new symbols ξ and such, can safely be left incrementally to future writers.
A.1 Hexadecimal numerals

Treating 2π as a single symbol is a small step, unlikely to trouble readers
much. A bolder step is to adopt from the computer science literature the
important notational improvement of the hexadecimal numeral. No incre-
mental step is possible here; either we leap the ditch or we remain on the
wrong side. In this book, we choose to leap.
Traditional decimal notation is unobjectionable for measured quantities
like 63.7 miles, $ 1.32 million or 9.81 m/s2 , but its iterative tenfold structure
meets little or no aesthetic support in mathematical theory. Consider for
instance the decimal numeral 127, whose number suggests a significant idea
to the computer scientist, but whose decimal notation does nothing to con-
vey the notion of the largest signed integer storable in a byte. Much better
is the base-sixteen hexadecimal notation 0x7F, which clearly expresses the
idea of 27 − 1. To the reader who is not a computer scientist, the aesthetic
advantage may not seem immediately clear from the one example, but con-
sider the decimal number 2,147,483,647, which is the largest signed integer
storable in a standard thirty-two bit word. In hexadecimal notation, this is
0x7FFF FFFF, or in other words 20x1F − 1. The question is: which notation
more clearly captures the idea?
To readers unfamiliar with the hexadecimal notation, to explain very
briefly: hexadecimal represents numbers not in tens but rather in sixteens.
The rightmost place in a hexadecimal numeral represents ones; the next
place leftward, sixteens; the next place leftward, sixteens squared; the next,
sixteens cubed, and so on. For instance, the hexadecimal numeral 0x1357
means “seven, plus five times sixteen, plus thrice sixteen times sixteen, plus
once sixteen times sixteen times sixteen.” In hexadecimal, the sixteen sym-
bols 0123456789ABCDEF respectively represent the numbers zero through
A.2. AVOIDING NOTATIONAL CLUTTER 515
fifteen, with sixteen being written 0x10.

All this raises the sensible question: why sixteen?1 The answer is that
sixteen is 24 , so hexadecimal (base sixteen) is found to offer a convenient
shorthand for binary (base two, the fundamental, smallest possible base).
Each of the sixteen hexadecimal digits represents a unique sequence of ex-
actly four bits (binary digits). Binary is inherently theoretically interesting,
but direct binary notation is unwieldy (the hexadecimal number 0x1357 is
binary 0001 0011 0101 0111), so hexadecimal is written in proxy.
The conventional hexadecimal notation is admittedly a bit bulky and
unfortunately overloads the letters A through F, letters which when set in
italics usually represent coefficients not digits. However, the real problem
with the hexadecimal notation is not in the notation itself but rather in the
unfamiliarity with it. The reason it is unfamiliar is that it is not often en-
countered outside the computer science literature, but it is not encountered
because it is not used, and it is not used because it is not familiar, and so
on in a cycle. It seems to this writer, on aesthetic grounds, that this partic-
ular cycle is worth breaking, so this book uses the hexadecimal for integers
larger than 9. If you have never yet used the hexadecimal system, it is worth
your while to learn it. For the sake of elegance, at the risk of challenging
entrenched convention, this book employs hexadecimal throughout.
Observe that in some cases, such as where hexadecimal numbers are
arrayed in matrices, this book may omit the cumbersome hexadecimal pre-
fix “0x.” Specific numbers with physical units attached appear seldom in
this book, but where they do naturally decimal not hexadecimal is used:
vsound = 331 m/s rather than the silly-looking vsound = 0x14B m/s.
Combining the hexadecimal and 2π ideas, we note here for interest’s sake
that
2π ≈ 0x6.487F.
A.2 Avoiding notational clutter

Good applied mathematical notation is not cluttered. Good notation does
not necessarily include every possible limit, qualification, superscript and
1
An alternative advocated by some eighteenth-century writers was twelve. In base
twelve, one quarter, one third and one half are respectively written 0.3, 0.4 and 0.6. Also,
the hour angles (§ 3.6) come in neat increments of (0.06)(2π) in base twelve, so there
are some real advantages to that base. Hexadecimal, however, besides having momentum
from the computer science literature, is preferred for its straightforward proxy of binary.
516 APPENDIX A. HEXADECIMAL NOTATION, ET AL.
subscript. For example, the sum

M X
X N
S= a2ij
i=1 j=1
might be written less thoroughly but more readably as

X
S= a2ij
i,j
if the meaning of the latter were clear from the context.

When to omit subscripts and such is naturally a matter of style and
subjective judgment, but in practice such judgment is often not hard to
render. The balance is between showing few enough symbols that the inter-
esting parts of an equation are not obscured visually in a tangle and a haze
of redundant little letters, strokes and squiggles, on the one hand; and on
the other hand showing enough detail that the reader who opens the book
directly to the page has a fair chance to understand what is written there
without studying the whole book carefully up to that point. Where appro-
priate, this book often condenses notation and omits redundant symbols.
Appendix B
The Greek alphabet
Mathematical experience finds the Roman alphabet to lack sufficient sym-

bols to write higher mathematics clearly. Although not completely solving
the problem, the addition of the Greek alphabet helps. See Table B.1.
When first seen in mathematical writing, the Greek letters take on a
wise, mysterious aura. Well, the aura is fine—the Greek letters are pretty—
but don’t let the Greek letters throw you. They’re just letters. We use
them not because we want to be wise and mysterious1 but rather because
1
Well, you can use them to be wise and mysterious if you want to. It’s kind of fun,
actually, when you’re dealing with someone who doesn’t understand math—if what you
want is for him to go away and leave you alone. Otherwise, we tend to use Roman
and Greek letters in various conventional ways: Greek minuscules (lower-case letters)
for angles; Roman capitals for matrices; e for the natural logarithmic base; f and g for
unspecified functions; i, j, k, m, n, M and N for integers; P and Q for metasyntactic
elements (the mathematical equivalents of foo and bar); t, T and τ for time; d, δ and ∆ for
change; A, B and C for unknown coefficients; J, Y and H for Bessel functions; etc. Even
with the Greek, there still are not enough letters, so each letter serves multiple conventional
roles: for example, i as an integer, an a-c electric current, or—most commonly—the
imaginary unit, depending on the context. Cases even arise in which a quantity falls back
to an alternate traditional letter because its primary traditional letter is already in use:
for example, the imaginary unit falls back from i to j where the former represents an a-c
electric current.
This is not to say that any letter goes. If someone wrote
e2 + π 2 = O 2
for some reason instead of the traditional
a2 + b2 = c2
for the Pythagorean theorem, you would not find that person’s version so easy to read,
would you? Mathematically, maybe it doesn’t matter, but the choice of letters is not a
matter of arbitrary utility only but also of convention, tradition and style: one of the early
517
518 APPENDIX B. THE GREEK ALPHABET
Table B.1: The Roman and Greek alphabets.
ROMAN
Aa Aa Gg Gg Mm Mm Tt Tt
Bb Bb Hh Hh Nn Nn Uu Uu
Cc Cc Ii Ii Oo Oo Vv Vv
Dd Dd Jj Jj Pp Pp Ww Ww
Ee Ee Kk Kk Qq Qq Xx Xx
Ff Ff Lℓ Ll Rr Rr Yy Yy
Ss Ss Zz Zz
GREEK
Aα alpha Hη eta Nν nu Tτ tau
Bβ beta Θθ theta Ξξ xi Υυ upsilon
Γγ gamma Iι iota Oo omicron Φφ phi
∆δ delta Kκ kappa Ππ pi Xχ chi
Eǫ epsilon Λλ lambda Pρ rho Ψψ psi
Zζ zeta Mµ mu Σσ sigma Ωω omega
we simply do not have enough Roman letters. An equation like
α2 + β 2 = γ 2
says no more than does an equation like
a2 + b2 = c2 ,
after all. The letters are just different.

Applied as well as professional mathematicians tend to use Roman and
Greek letters in certain long-established conventional sets: abcd; f gh; ijkℓ;
mn (sometimes followed by pqrst as necessary); pqr; st; uvw; xyzw. For
writers in a field has chosen some letter—who knows why?—then the rest of us follow.
This is how it usually works.
When writing notes for your own personal use, of course, you can use whichever letter
you want. Probably you will find yourself using the letter you are used to seeing in print,
but a letter is a letter; any letter will serve. Excepting the rule, for some obscure reason
almost never broken, that i not follow h, the mathematical letter convention consists of
vague stylistic guidelines rather than clear, hard rules. You can bend the convention at
need. When unsure of which letter to use, just pick one; you can always go back and
change the letter in your notes later if you need to.
519
the Greek: αβγ; δǫ; κλµνξ; ρστ ; φψωθχ (the letters ζηθξ can also form a
set, but these oftener serve individually). Greek letters are frequently paired
with their Roman congeners as appropriate: aα; bβ; cgγ; dδ; eǫ; f φ; kκ; ℓλ;
mµ; nν; pπ; rρ; sσ; tτ ; xχ; zζ.2
Some applications (even in this book) group the letters slightly differ-
ently, but most tend to group them approximately as shown. Even in West-
ern languages other than English, mathematical convention seems to group
the letters in about the same way.
Naturally, you can use whatever symbols you like in your own private
papers, whether the symbols are letters or not; but other people will find
your work easier to read if you respect the convention when you can. It is a
special stylistic error to let the Roman letters ijkℓmn, which typically rep-
resent integers, follow the Roman letters abcdef gh directly when identifying
mathematical quantities. The Roman letter i follows the Roman letter h
only in the alphabet, almost never in mathematics. If necessary, p can fol-
low h (curiously, p can alternatively follow n, but never did convention claim
to be logical).
Mathematicians usually avoid letters like the Greek capital H (eta),
which looks just like the Roman capital H, even though H (eta) is an entirely
proper member of the Greek alphabet. The Greek minuscule υ (upsilon) is
avoided for like reason, for mathematical symbols are useful only insofar as
we can visually tell them apart.3
2
The capital pair Y Υ is occasionally seen but is awkward both because the Greek
minuscule υ is visually almost indistinguishable from the unrelated (or distantly related)
Roman minuscule v; and because the ancient Romans regarded the letter Y not as a
congener but as the Greek letter itself, seldom used but to spell Greek words in the
Roman alphabet. To use Y and Υ as separate symbols is to display an indifference to,
easily misinterpreted as an ignorance of, the Graeco-Roman sense of the thing—which is
silly, really, if you think about it, since no one objects when you differentiate j from i, or u
and w from v—but, anyway, one is probably the wiser to tend to limit the mathematical
use of the symbol Υ to the very few instances in which established convention decrees it.
(In English particularly, there is also an old typographical ambiguity between Y and a
Germanic, non-Roman letter named “thorn” that has practically vanished from English
today, to the point that the typeface in which you are reading these words lacks a glyph
for it—but which sufficiently literate writers are still expected to recognize on sight. This
is one more reason to tend to avoid Υ when you can, a Greek letter that makes you look
ignorant when you use it wrong and pretentious when you use it right. You can’t win.)
The history of the alphabets is extremely interesting. Unfortunately, a footnote in an
appendix to a book on derivations of applied mathematics is probably not the right place
for an essay on the topic, so we’ll let the matter rest there.
3
No citation supports this appendix, whose contents (besides the Roman and Greek
alphabets themselves, which are what they are) are inferred from the author’s subjective
observation of seemingly consistent practice in English-language applied mathematical
520 APPENDIX B. THE GREEK ALPHABET
publishing, plus some German and a little French, dating back to the 1930s. From the
thousands of readers of drafts of the book, the author has yet to receive a single seri-
ous suggestion that the appendix were wrong—a lack that constitutes a sort of negative
(though admittedly unverifiable) citation if you like. If a mathematical style guide exists
that formally studies the letter convention’s origins, the author is unaware of it (and if a
graduate student in history or the languages who, for some reason, happens to be reading
these words seeks an esoteric, maybe untouched topic on which to write a master’s thesis,
why, there is one).
Appendix C
A bare sketch of the pure

theory of the complex
variable
At least three of the various disciplines of pure mathematics stand out for
their pedagogical intricacy and the theoretical depth of their core results.
The first of the three is number theory which, except for the simple results
of § 6.1, scientists and engineers tend to get by largely without. The second
is matrix theory (Chs. 11 through 14), a bruiser of a discipline the applied
mathematician of the computer age—try though he might—can hardly es-
cape. The third is the pure theory of the complex variable.
The introduction’s § 1.3 admires the beauty of the pure theory of the
complex variable even while admitting that “its arc takes off too late and
flies too far from applications for such a book as this.” To develop the
pure theory properly is a worthy book-length endeavor of its own requiring
relatively advanced preparation on its reader’s part which, however, the
reader who has reached the end of the present book’s Ch. 9 possesses. If the
writer doubts the strictly applied necessity of the pure theory, still, he does
not doubt its health to one’s overall mathematical formation. It provides
another way to think about complex numbers. Scientists and engineers
with advanced credentials occasionally expect one to be acquainted with it
for technical-social reasons, regardless of its practical use. Besides, the pure
theory is interesting. This alone recommends some attention to it.
The pivotal result of pure complex-variable theory is the Taylor series
by Cauchy’s impressed residue theorem. If we will let these few pages of
appendix replace an entire book on the pure theory, then Cauchy’s and
521
522 APPENDIX C. A SKETCH OF PURE COMPLEX THEORY
Taylor’s are the results we shall sketch. (The bibliography lists presentations
far more complete; this presentation, as advertised, is just a sketch. The
reader who has reached the end of the Ch. 9 will understand already why the
presentation is strictly optional, interesting maybe but deemed unnecessary
to the book’s applied mathematical development.)
Cauchy’s impressed residue theorem 1 is that
1 f (w)
I
f (z) = dw (C.1)
i2π w−z
if z lies within the closed complex contour about which the integral is taken
and if f (z) is everywhere analytic (§ 8.4) within and along the contour. More
than one proof of the theorem is known, depending on the assumptions from
which the mathematician prefers to start, but this writer is partial to an
instructively clever proof he has learned from D.N. Arnold2 which goes as
follows. Consider the function
1 f [z + (t)(w − z)]
I
g(z, t) ≡ dw,
i2π w−z
whose derivative with respect to the parameter t is3

∂g 1
I
= f ′ [z + (t)(w − z)] dw.
∂t i2π
We notice that this is

∂g 1 ∂ f [z + (t)(w − z)]
I
= dw
∂t i2π ∂w t
f [z + (t)(w − z)] b

1
= ,
i2π t a
1
This is not a standard name. Though they name various associated results after
Cauchy in one way or another, neither [31] nor [3] seems to name this particular result,
though both do feature it. Since (C.1) impresses a pole and thus also a residue on a
function f (z) which in the domain of interest lacks them, the name Cauchy’s impressed
residue theorem ought to serve this appendix’s purpose ably enough.
2
[3, § III]
3
The book does not often employ Newton’s notation f ′ (·) ≡ [(d/dζ)f (ζ)]ζ=(·) of § 4.4
but the notation is handy here because it evades the awkward circumlocution of changing
ζ ← z in (C.1) and then writing
[(d/dζ)f (ζ)]ζ=z+(t)(w−z)
I
∂g 1
= dw.
∂t i2π w−z
523
where a and b respectively represent the contour integration’s beginning and

ending points. But this integration ends where it begins, so a = b and the
factor {·}ba in braces vanishes, whereupon
∂g
= 0,
∂t
meaning that g(z, t) does not vary with t. Observing per (8.26) that
1 dw
I
= 1,
i2π w−z
we have that
f (z) dw 1 f (w)
I I
f (z) = = g(z, 0) = g(z, 1) = dw
i2π w−z i2π w−z
as was to be proved. (There remains a basic question as to whether the

paragraph’s integration is even valid. Logically, it ought to be valid, since
f [z] being analytic is infinitely differentiable,4 but when the integration is
used as the sole theoretical support for the entire calculus of the complex
variable, well, it seems an awfully slender reed to carry so heavy a load.
Admittedly, maybe this is only a psychological problem, but a professional
mathematician will devote many pages to preparatory theoretical constructs
before even attempting the integral, the result of which lofty effort is not
in the earthier spirit of applied mathematics. On the other hand, now that
the reader has followed the book along its low road and the high integration
is given only in reserve, now that the integration reaches a conclusion al-
ready believed and, once there, is asked to carry the far lighter load of this
appendix only, the applied reader may feel easier about trusting it.)
One could follow Arnold hence toward the proof of the theorem of one
Goursat and further toward various other interesting results, a path of study
the writer recommends to sufficiently interested readers: see [3]. Being in
a tremendous hurry ourselves, however, we shall leave Arnold and follow
F.B. Hildebrand5 directly toward the Taylor series. Positing some expansion
point zo and then expanding (C.1) geometrically per (2.34) about it, we have
4
The professionals minimalistically actually require only that the function be once
differentiable under certain conditions, from which they prove infinite differentiability,
but this is a fine point which will not concern us here.
5
[31, § 10.7]
524 APPENDIX C. A SKETCH OF PURE COMPLEX THEORY
that
1 f (w)
I
f (z) = dw
i2π (w − zo ) − (z − zo )
1 f (w)
I
= dw
i2π (w − zo )[1 − (z − zo )/(w − zo )]
∞
f (w) X z − zo k

1
I
= dw
i2π w − zo w − zo
k=0
∞
1 f (w)
X I
k
= dw (z − zo ) ,
i2π (w − zo )k+1
k=0
which, being the power series

∞
X
f (z) = (ak )(z − zo )k ,
k=0 (C.2)
1 f (w)
I
ak ≡ dw,
i2π (w − zo )k+1
by definition constitutes the Taylor series (8.19) for f (z) about z = zo ,

assuming naturally that |z − zo | < |w − zo | for all w along the contour so
that the geometric expansion above will converge.
The important theoretical implication of (C.2) is that every function
has a Taylor series about any point across whose immediate neighborhood
the function is analytic. There evidently is no such thing as an analytic
function without a Taylor series—which we already knew if we have read
and believed Ch. 8, but some readers may find it more convincing this way.
Comparing (C.2) against (8.19), incidentally, we have also that
dk f

k! f (w)
I
k
= dw, (C.3)
dz z=zo
i2π (w − zo )k+1
which is an alternate way to write (8.31).

Appendix D
Manuscript history
The book in its present form is based on various unpublished drafts and notes
of mine, plus some of my wife Kristie’s (née Hancock), going back to 1983
when I was fifteen years of age. What prompted the contest I can no longer
remember, but the notes began one day when I challenged a high-school
classmate to prove the quadratic formula. The classmate responded that
he didn’t need to prove the quadratic formula because the proof was in the
class math textbook, then counterchallenged me to prove the Pythagorean
theorem. Admittedly obnoxious (I was fifteen, after all) but not to be out-
done, I whipped out a pencil and paper on the spot and started working.
But I found that I could not prove the theorem that day.
The next day I did find a proof in the school library,1 writing it down,
adding to it the proof of the quadratic formula plus a rather inefficient proof
of my own invention to the law of cosines. Soon thereafter the school’s chem-
istry instructor happened to mention that the angle between the tetrahe-
drally arranged four carbon-hydrogen bonds in a methane molecule was 109◦ ,
so from a symmetry argument I proved that result to myself, too, adding it
to my little collection of proofs. That is how it started.2
The book actually has earlier roots than these. In 1979, when I was
twelve years old, my father bought our family’s first eight-bit computer.
The computer’s built-in BASIC programming-language interpreter exposed
1
A better proof is found in § 2.10.
2
Fellow gear-heads who lived through that era at about the same age might want to
date me against the disappearance of the slide rule. Answer: in my country, or at least
at my high school, I was three years too young to use a slide rule. The kids born in 1964
learned the slide rule; those born in 1965 did not. I wasn’t born till 1967, so for better
or for worse I always had a pocket calculator in high school. My family had an eight-bit
computer at home, too, as we shall see.
525
526 APPENDIX D. MANUSCRIPT HISTORY
functions for calculating sines and cosines of angles. The interpreter’s man-
ual included a diagram much like Fig. 3.1 showing what sines and cosines
were, but it never explained how the computer went about calculating such
quantities. This bothered me at the time. Many hours with a pencil I spent
trying to figure it out, yet the computer’s trigonometric functions remained
mysterious to me. When later in high school I learned of the use of the Tay-
lor series to calculate trigonometrics, into my growing collection of proofs
the series went.
Five years after the Pythagorean incident I was serving the U.S. Army as
an enlisted troop in the former West Germany. Although those were the last
days of the Cold War, there was no shooting war at the time, so the duty
was peacetime duty. My duty was in military signal intelligence, frequently
in the middle of the German night when there often wasn’t much to do.
The platoon sergeant wisely condoned neither novels nor cards on duty, but
he did let the troops read the newspaper after midnight when things were
quiet enough. Sometimes I used the time to study my German—the platoon
sergeant allowed this, too—but I owned a copy of Richard P. Feynman’s
Lectures on Physics [19] which I would sometimes read instead.
Late one night the battalion commander, a lieutenant colonel and West
Point graduate, inspected my platoon’s duty post by surprise. A lieutenant
colonel was a highly uncommon apparition at that hour of a quiet night, so
when that old man appeared suddenly with the sergeant major, the company
commander and the first sergeant in tow—the last two just routed from
their sleep, perhaps—surprise indeed it was. The colonel may possibly have
caught some of my unlucky fellows playing cards that night—I am not sure—
but me, he caught with my boots unpolished, reading the Lectures.
I snapped to attention. The colonel took a long look at my boots without
saying anything, as stormclouds gathered on the first sergeant’s brow at his
left shoulder, then asked me what I had been reading.
“Feynman’s Lectures on Physics, sir.”
“Why?”
“I am going to attend the university when my three-year enlistment is
up, sir.”
“I see.” Maybe the old man was thinking that I would do better as a
scientist than as a soldier? Maybe he was remembering when he had had
to read some of the Lectures himself at West Point. Or maybe it was just
the singularity of the sight in the man’s eyes, as though he were a medieval
knight at bivouac who had caught one of the peasant levies, thought to be
illiterate, reading Cicero in the original Latin. The truth of this, we shall
never know. What the old man actually said was, “Good work, son. Keep
527
it up.”
The stormclouds dissipated from the first sergeant’s face. No one ever
said anything to me about my boots (in fact as far as I remember, the first
sergeant—who saw me seldom in any case—never spoke to me again). The
platoon sergeant thereafter explicitly permitted me to read the Lectures on
duty after midnight on nights when there was nothing else to do, so in the
last several months of my military service I did read a number of them. It
is fair to say that I also kept my boots better polished.
In Volume I, Chapter 6, of the Lectures there is a lovely introduction to
probability theory. It discusses the classic problem of the “random walk” in
some detail, then states without proof that the generalization of the random
walk leads to the Gaussian distribution
exp(−x2 /2σ 2 )
p(x) = √ .
σ 2π
For the derivation of this remarkable theorem, I scanned the book in vain.
One had no Internet access in those days, but besides a well equipped gym
the Army post also had a tiny library, and in one yellowed volume in the
library—who√ knows how such a book got there?—I did find a derivation
of the 1/σ 2π factor.3 The exponential factor, the volume did not derive.
Several days later, I chanced to find myself in Munich with an hour or two
to spare, which I spent in the university library seeking the missing part
of the proof, but lack of time and unfamiliarity with such a German site
defeated me. Back at the Army post, I had to sweat the proof out on my
own over the ensuing weeks. Nevertheless, eventually I did obtain a proof
which made sense to me. Writing the proof down carefully, I pulled the old
high-school math notes out of my military footlocker (for some reason I had
kept the notes and even brought them to Germany), dusted them off, and
added to them the new Gaussian proof.
That is how it has gone. To the old notes, I have added new proofs
from time to time, and although somehow I have misplaced the original
high-school leaves I took to Germany with me the notes have nevertheless
grown with the passing years. These years have brought me the good things
years can bring: marriage, family and career; a good life gratefully lived,
details of which interest me and mine but are mostly unremarkable as seen
from the outside. A life however can take strange turns, reprising earlier
themes. I had become an industrial building construction engineer for a
living (and, appropriately enough, had most lately added to the notes a
3
The citation is now unfortunately long lost.
mathematical justification of the standard industrial building construction

technique to measure the resistance-to-ground of a new building’s electrical
grounding system), when at a juncture between construction projects an
unexpected opportunity arose to pursue a Ph.D. in engineering at Virginia
Tech, courtesy (indirectly, as it developed) of a research program not of the
United States Army as last time but this time of the United States Navy.
The Navy’s research problem turned out to be in the highly mathematical
fields of theoretical and computational electromagnetics. Such work natu-
rally brought a blizzard of new formulas, whose proofs I sought or worked
out and, either way, added to the notes—whence the manuscript and, in due
time, this book.
The book follows in the honorable tradition of Courant’s and Hilbert’s
1924 classic Methods of Mathematical Physics [14]—a tradition subsequently
developed by, among others, Jeffreys and Jeffreys [34], Arfken and Weber [2],
and Weisstein4 [66]. The present book’s chief intended contribution to the
tradition lies in its applied-level derivations of the many results it presents.
Its author always wanted to know why the Pythagorean theorem was so.
The book is presented in this spirit.
A book can follow convention or depart from it; yet, though occasional
departure might render a book original, frequent departure seldom renders
a book good. Whether this particular book is original or good, neither or
both, is for the reader to tell, but in any case the book does both follow and
depart. Convention is a peculiar thing: at its best, it evolves or accumulates
only gradually, patiently storing up the long, hidden wisdom of generations
past; yet herein arises the ancient dilemma. Convention, in all its richness,
in all its profundity, can, sometimes, stagnate at a local maximum, a hillock
whence higher ground is achievable not by gradual ascent but only by descent
first—or by a leap. Descent risks a bog. A leap risks a fall. One ought not
4
Weisstein lists results encyclopedically, alphabetically by name. I organize results
more traditionally by topic, leaving alphabetization to the book’s index, that readers who
wish to do so can coherently read the book from front to back.
There is an ironic personal story in this. As children in the 1970s, my brother and
I had a 1959 World Book encyclopedia in our bedroom, about twenty volumes. It was
then a bit outdated (in fact the world had changed tremendously in the fifteen or twenty
years following 1959, so the encyclopedia was more than a bit outdated) but the two of
us still used it sometimes. Only years later did I learn that my father, who in 1959 was
fourteen years old, had bought the encyclopedia with money he had earned delivering
newspapers daily before dawn, and then had read the entire encyclopedia, front to back.
My father played linebacker on the football team and worked a job after school, too, so
where he found the time or the inclination to read an entire encyclopedia, I’ll never know.
Nonetheless, it does prove that even an encyclopedia can be read from front to back.
529
run such risks without cause, even in such an inherently unconservative

discipline as mathematics.
Well, the book does risk. It risks one leap at least: it employs hexadeci-
mal numerals.
This book is bound to lose at least a few readers for its unorthodox use
of hexadecimal notation (“The first primes are 2, 3, 5, 7, 0xB, . . .”). Perhaps
it will gain a few readers for the same reason; time will tell. I started keeping
my own theoretical math notes in hex a long time ago; at first to prove to
myself that I could do hexadecimal arithmetic routinely and accurately with
a pencil, later from aesthetic conviction that it was the right thing to do.
Like other applied mathematicians, I’ve several own private notations, and
in general these are not permitted to burden the published text. The hex
notation is not my own, though. It existed before I arrived on the scene, and
since I know of no math book better positioned to risk its use, I have with
hesitation and no little trepidation resolved to let this book use it. Some
readers will approve; some will tolerate; undoubtedly some will do neither.
The views of the last group must be respected, but in the meantime the
book has a mission; and crass popularity can be only one consideration, to
be balanced against other factors. The book might gain even more readers,
after all, had it no formulas, and painted landscapes in place of geometric
diagrams! I like landscapes, too, but anyway you can see where that line of
logic leads.
More substantively: despite the book’s title, adverse criticism from some
quarters for lack of rigor is probably inevitable; nor is such criticism neces-
sarily improper from my point of view. Still, serious books by professional
mathematicians tend to be for professional mathematicians, which is under-
standable but does not always help the scientist or engineer who wants to
use the math to model something. The ideal author of such a book as this
would probably hold two doctorates: one in mathematics and the other in
engineering or the like. The ideal author lacking, I have written the book.
So here you have my old high-school notes, extended over twenty-five
years and through the course of two-and-a-half university degrees, now
partly typed and revised for the first time as a LATEX manuscript. Where this
manuscript will go in the future is hard to guess. Perhaps the revision you
are reading is the last. Who can say? The manuscript met an uncommonly
enthusiastic reception at Debconf 6 [15] May 2006 at Oaxtepec, Mexico; and
in August of the same year it warmly welcomed Karl Sarnow and Xplora
Knoppix [70] aboard as the second official distributor of the book. Such de-
velopments augur well for the book’s future at least. But in the meantime,
if anyone should challenge you to prove the Pythagorean theorem on the
spot, why, whip this book out and turn to § 2.10. That should confound
’em.
THB
Bibliography
[1] Larry C. Andrews. Special Functions of Mathematics for Engineers.

Macmillan, New York, 1985.
[2] George B. Arfken and Hans J. Weber. Mathematical Methods for Physi-
cists. Academic Press, Burlington, Mass., 6th edition, 2005.
[3] Douglas N. Arnold. Complex analysis. Dept. of Mathematics, Penn

State Univ., 1997. Lecture notes.
[4] Constantine A. Balanis. Advanced Engineering Electromagnetics. John

Wiley & Sons, New York, 1989.
[5] Christopher Beattie, John Rossi, and Robert C. Rogers. Notes on Ma-
trix Theory. Unpublished, Department of Mathematics, Virginia Poly-
technic Institute and State University, Blacksburg, Va., 6 Dec. 2001.
[6] R. Byron Bird, Warren E. Stewart, and Edwin N. Lightfoot. Transport

Phenomena. John Wiley & Sons, New York, 1960.
[7] Kristie H. Black. Private conversation, 1996.
[8] Thaddeus H. Black. Derivations of Applied Mathematics. The Debian

Project, http://www.debian.org/, 7 January 2009.
[9] J. van Bladel. Singular Electromagnetic Fields and Sources. Number 28

in Engineering Science Series. Clarendon Press, Oxford, 1991.
[10] Gary S. Brown. Private conversation, 2004–08.
[11] CERN (European Organization for Nuclear Research, author un-

known). The delta transformation. http://aliceinfo.cern.ch/
Offline/Activities/Alignment/deltatr.html. As retrieved 24 May
2008.
531
532 BIBLIOGRAPHY
[12] David K. Cheng. Field and Wave Electromagnetics. Series in Electrical

Engineering. Addison-Wesley, Reading, Mass., 2nd edition, 1989.
[13] Leon W. Couch II. Modern Communication Systems: Principles and

Applications. Prentice Hall, Upper Saddle River, N.J., 1995.
[14] Richard Courant and David Hilbert. Methods of Mathematical Physics.

Interscience (Wiley), New York, first English edition, 1953.
[15] The Debian Project. http://www.debian.org/.
[16] The Debian Project. Debian Free Software Guidelines, version 1.1.
http://www.debian.org/social_contract#guidelines.
[17] G. Doetsch. Guide to the Applications of the Laplace and z-Transforms.

Van Nostrand Reinhold, London, 1971. Referenced indirectly by way
of [48].
[18] John W. Eaton. GNU Octave. http://www.octave.org/. Software

version 2.1.73.
[19] Richard P. Feynman, Robert B. Leighton, and Matthew Sands. The

Feynman Lectures on Physics. Addison-Wesley, Reading, Mass., 1963–
65. Three volumes.
[20] Stephen D. Fisher. Complex Variables. Books on Mathematics. Dover,

Mineola, N.Y., 2nd edition, 1990.
[21] Joel N. Franklin. Matrix Theory. Books on Mathematics. Dover, Mi-

neola, N.Y., 1968.
[22] The Free Software Foundation. GNU General Public License, version 2.
/usr/share/common-licenses/GPL-2 on a Debian system. The De-
bian Project: http://www.debian.org/. The Free Software Founda-
tion: 51 Franklin St., Fifth Floor, Boston, Mass. 02110-1301, USA.
[23] Stephen H. Friedberg, Arnold J. Insel, and Lawrence E. Spence. Linear

Algebra. Pearson Education/Prentice-Hall, Upper Saddle River, N.J.,
4th edition, 2003.
[24] Edward Gibbon. The History of the Decline and Fall of the Roman
Empire. 1788.
BIBLIOGRAPHY 533
[25] J.W. Gibbs. Fourier series. Nature, 59:606, 1899. Referenced indirectly
by way of [67, “Gibbs Phenomenon,” 06:12, 13 Dec. 2008], this letter
of Gibbs completes the idea of the same author’s paper on p. 200 of the
same volume.
[26] William Goldman. The Princess Bride. Ballantine, New York, 1973.
[27] Richard W. Hamming. Methods of Mathematics Applied to Calculus,

Probability, and Statistics. Books on Mathematics. Dover, Mineola,
N.Y., 1985.
[28] Roger F. Harrington. Time-harmonic Electromagnetic Fields. Texts in

Electrical Engineering. McGraw-Hill, New York, 1961.
[29] Harvard University (author unknown). Math 21B review. http://www.

math.harvard.edu/archive/21b_fall_03/final/21breview.pdf,
13 Jan. 2004.
[30] Jim Hefferon. Linear Algebra. Mathematics, St. Michael’s College,

Colchester, Vt., 20 May 2006. (The book is free software, and be-
sides is the best book of the kind the author of the book you hold
has encountered. As of 3 Nov. 2007 at least, one can download it from
http://joshua.smcvt.edu/linearalgebra/.).
[31] Francis B. Hildebrand. Advanced Calculus for Applications. Prentice-

Hall, Englewood Cliffs, N.J., 2nd edition, 1976.
[32] Theo Hopman. Introduction to indicial notation. http://www.

uoguelph.ca/~thopman/246/indicial.pdf, 28 Aug. 2002.
[33] Intel Corporation. IA-32 Intel Architecture Software Developer’s Man-

ual, 19th edition, March 2006.
[34] H. Jeffreys and B.S. Jeffreys. Methods of Mathematical Physics. Cam-

bridge University Press, Cambridge, 3rd edition, 1988.
[35] David E. Johnson, Johnny R. Johnson, and John L. Hilburn. Electric

Circuit Analysis. Prentice Hall, Englewood Cliffs, N.J., 1989.
[36] Eric M. Jones and Paul Fjeld. Gimbal angles, gimbal lock, and
a fourth gimbal for Christmas. http://www.hq.nasa.gov/alsj/
gimbals.html. As retrieved 23 May 2008.
534 BIBLIOGRAPHY
[37] Brian W. Kernighan and Dennis M. Ritchie. The C Programming Lan-

guage. Software Series. Prentice Hall PTR, Englewood Cliffs, N.J., 2nd
edition, 1988.
[38] M.A. Khamsi. Gibbs’ phenomenon. http://www.sosmath.com/

fourier/fourier3/gibbs.html. As retrieved 30 Oct. 2008.
[39] Konrad Knopp. Theory and Application of Infinite Series. Hafner,

New York, 2nd English ed., revised in accordance with the 4th German
edition, 1947.
[40] Werner E. Kohler. Lecture, Virginia Polytechnic Institute and State

University, Blacksburg, Va., 2007.
[41] The Labor Law Talk. http://encyclopedia.laborlawtalk.com/

Applied_mathematics. As retrieved 1 Sept. 2005.
[42] David C. Lay. Linear Algebra and Its Applications. Addison-Wesley,

Reading, Mass., 1994.
[43] N.N. Lebedev. Special Functions and Their Applications. Books on

Mathematics. Dover, Mineola, N.Y., revised English edition, 1965.
[44] David McMahon. Quantum Mechanics Demystified. Demystified Series.

McGraw-Hill, New York, 2006.
[45] Ali H. Nayfeh and Balakumar Balachandran. Applied Nonlinear Dy-

namics: Analytical, Computational and Experimental Methods. Series
in Nonlinear Science. Wiley, New York, 1995.
[46] John J. O’Connor and Edmund F. Robertson. The MacTu-

tor History of Mathematics. School of Mathematics and Statis-
tics, University of St. Andrews, Scotland, http://www-history.mcs.
st-andrews.ac.uk/. As retrieved 12 Oct. 2005 through 2 Nov. 2005.
[47] Andrew F. Peterson and Raj Mittra. Convergence of the conjugate

gradient method when applied to matrix equations representing elec-
tromagnetic scattering problems. IEEE Transactions on Antennas and
Propagation, AP-34(12):1447–54, Dec. 1986.
[48] Charles L. Phillips and John M. Parr. Signals, Systems and Transforms.
Prentice-Hall, Englewood Cliffs, N.J., 1995.
BIBLIOGRAPHY 535
[49] W.J. Pierson, Jr., and L. Moskowitz. A proposed spectral form for fully
developed wind seas based on the similarity theory of S.A. Kitaigorod-
skii. J. Geophys. Res., 69:5181–90, 1964.
[50] PlanetMath.org. Planet Math. http://www.planetmath.org/. As

retrieved 21 Sept. 2007 through 20 Feb. 2008.
[51] Reed College (author unknown). Levi-Civita symbol, lecture 1,

Physics 321, electrodynamics. http://academic.reed.edu/physics/
courses/Physics321/page1/files/LCHandout.pdf, Portland, Ore.,
27 Aug. 2007.
[52] Carl Sagan. Cosmos. Random House, New York, 1980.
[53] Wayne A. Scales. Lecture, Virginia Polytechnic Institute and State

University, Blacksburg, Va., 2004.
[54] Adel S. Sedra and Kenneth C. Smith. Microelectronic Circuits. Series in

Electrical Engineering. Oxford University Press, New York, 3rd edition,
1991.
[55] Al Shenk. Calculus and Analytic Geometry. Scott, Foresman & Co.,
Glenview, Ill., 3rd edition, 1984.
[56] William L. Shirer. The Rise and Fall of the Third Reich. Simon &
Schuster, New York, 1960.
[57] Murray R. Spiegel. Complex Variables: with an Introduction to Confor-

mal Mapping and Its Applications. Schaum’s Outline Series. McGraw-
Hill, New York, 1964.
[58] Susan Stepney. Euclid’s proof that there are an infinite num-
ber of primes. http://www-users.cs.york.ac.uk/susan/cyc/p/
primeprf.htm. As retrieved 28 April 2006.
[59] James Stewart, Lothar Redlin, and Saleem Watson. Precalculus: Math-
ematics for Calculus. Brooks/Cole, Pacific Grove, Calif., 3rd edition,
1993.
[60] Julius Adams Stratton. Electromagnetic Theory. International Series

in Pure and Applied Physics. McGraw-Hill, New York, 1941.
[61] Bjarne Stroustrup. The C++ Programming Language. Addison-Wesley,

Boston, “special” (third-and-a-half?) edition, 2000.
536 BIBLIOGRAPHY
[62] Eric de Sturler. Lecture, Virginia Polytechnic Institute and State Uni-
versity, Blacksburg, Va., 2007.
[63] Henk A. van der Vorst. Iterative Krylov Methods for Large Linear
Systems. Number 13 in Monographs on Applied and Computational
Mathematics. Cambridge University Press, Cambridge, 2003.
[64] G.N. Watson. A Treatise on the Theory of Bessel Functions. Macmillan,

New York, 2nd edition, 1944.
[65] Eric W. Weisstein. Mathworld. http://mathworld.wolfram.com/. As

retrieved 29 May 2006 through 20 Feb. 2008.
[66] Eric W. Weisstein. CRC Concise Encyclopedia of Mathematics. Chap-

man & Hall/CRC, Boca Raton, Fla., 2nd edition, 2003.
[67] The Wikimedia Foundation. Wikipedia. http://en.wikipedia.org/.
[68] Henry Wilbraham. On a certain periodic function. Cambridge and

Dublin Mathematical Journal, 3:198–201, 1848. Referenced indirectly
by way of [67, “Henry Wilbraham,” 04:06, 6 Dec. 2008].
[69] J.H. Wilkinson. The Algebraic Eigenvalue Problem. Monographs on

Numerical Analysis. Clarendon Press, Oxford, 1965.
[70] Xplora. Xplora Knoppix . http://www.xplora.org/downloads/

Knoppix/.
Index
′
, 53, 416 accountant, 419
0 (zero), 9 accretion, 133
dividing by, 70 active region, 250, 253
matrix, 250 addition
vector, 250, 286 of matrices, 243
1 (one), 9, 47, 251 of rows or columns, 369
2π, 37, 47, 513 of vectors, 403
calculating, 188 parallel, 120, 224
LU decomposition, 288 serial, 120
QR decomposition, 356 series, 120
inverting a matrix by, 362 addition operator
≈, 75 downward multitarget, 270
δ, 70, 148, 247, 419 elementary, 257
ǫ, 70, 419 leftward multitarget, 271
≡, 13 multitarget, 269
∈ Z, 17 rightward multitarget, 271
←, 12 upward multitarget, 270
≪ and ≫, 71 addition quasielementary, 269
0x, 514 row, 270
∇ (del), 439 addressing a vector space, 307
π, 513 adjoint, 246, 369, 392
ρ, 458 of a matrix inverse, 263
dℓ, 146 aeronautical engineer, 335
dz, 170 aileron, 335
e, 93 air, 437, 444
i, 41 algebra
n-dimensional vector, 240, 401 classical, 9
nth root fundamental theorem of, 119
calculation of by Newton-Raphson, higher-order, 223
90 linear, 239
nth-order expression, 223 of the vector, 401
reductio ad absurdum, 114, 312 algorithm
16th century, 223 Gauss-Jordan, 294
implementation of from an equation,
absolute value, 43 355
abstraction, 287 alternating signs, 179, 496
537
538 INDEX
altitude, 40 arithmetic, 9
AMD, 295 exact, 295, 310, 320, 338, 374, 382
amortization, 202 of matrices, 243
amplitude, 49, 401 arithmetic mean, 124
analytic continuation, 163 arithmetic series, 17
analytic function, 46, 163 arm, 100
angle, 37, 47, 57, 407, 409, 457 articles “a” and “the”, 337
double, 57 artillerist, 334
half, 57 assignment, 12
hour, 57 associativity, 9
interior, 37 nonassociativity of the cross prod-
of a polygon, 38 uct, 409
of a triangle, 37 of matrix multiplication, 243
of rotation, 55 of unary linear operators, 442
right, 47 automobile, 410
square, 47 average, 123
sum of, 37 axes, 53
angular frequency, 483 changing, 53
antelope, 438 invariance under the reorientation
antenna of, 407, 408, 439, 454, 455
parabolic, 429 reorientation of, 404
satellite dish, 429 rotation of, 53, 401
antiderivative, 133, 197 axiom, 2
and the natural logarithm, 198 axis, 64
guessing, 201 of a cylinder or circle, 412
of a product of exponentials, powers axle, 414
and logarithms, 221 azimuth, 334, 412, 413
antiquity, 223
applied mathematics, 1, 148, 158 barometric pressure, 444
foundations of, 239 baroquity, 233
approximation to first order, 177 baseball, 484
arc, 47 basis, 383
arccosine, 47 complete, 384
derivative of, 111 constant, 411, 463
arcsine, 47 converting to and from, 384
derivative of, 111 cylindrical, 412
arctangent, 47 secondary cylindrical, 413
derivative of, 111 spherical, 413
area, 9, 38, 141, 458, 490 variable, 411
enclosed by a contour, 446 basis, orthogonal, 410
surface, 141 battle, 334
arg, 43 battlefield, 334
Argand domain and range planes, 164 belonging, 17
Argand plane, 42 binomial theorem, 74
Argand, Jean-Robert (1768–1822), 42 bisection, 429
INDEX 539
bit, 249, 295 unit, 47

Black, Thaddeus H. (1967–), 525 circuit, 335
blackbody radiation, 32 circular paraboloidal coordinates, 435
block, wooden, 72 circulation, 446, 451
bond, 82 cis, 104
borrower, 202 citation, 6
bound on a power series, 179 city street, 336
boundary condition, 202 classical algebra, 9
boundary element, 449, 451 cleverness, 151, 231, 351, 463, 522
box, 9 clock, 57
branch point, 41, 165 closed analytic form, 221
strategy to avoid, 166 closed contour
Bryan, George Hartley (1864–1928), 404 about a multiple pole, 175
building construction, 339 closed contour integration, 146
businessperson, 123 closed form, 221
closed surface integration, 144
C and C++, 11, 12 clutter, notational, 416, 515
calculus, 69, 127 coefficient, 239
fundamental theorem of, 133, 450 Fourier, 486
of the vector, 437 inscrutable, 226
the two complementary questions of, matching of, 28
69, 127, 133 metric, 457, 458
vector, definitions and identities of, nontrivial, 286, 378
453 unknown, 201
cannon, 334 coincident properties of the square ma-
Cardano rotations, 404 trix, 374
Cardano, Girolamo (also known as Car- column, 240, 296, 331, 357
danus or Cardan, 1501–1576), addition of, 369
223, 404 null, 368
carriage wheel, 335 null, appending a, 327
Cauchy’s integral formula, 169, 204 orthonormal, 361
Cauchy, Augustin Louis (1789–1857), 169, scaled and repeated, 368
204 spare, 301, 320
caveman, 438 column operator, 245
chain rule, derivative, 82 column rank, 343
change of variable, 12 full, 318, 320
change, rate of, 69 column vector, 312, 349
characteristic polynomial, 376, 390, 392 combination, 72
checking an integration, 145 properties of, 73
checking division, 145 combinatorics, 72
choice of wooden blocks, 72 commutation
Cholesky, André-Louis (1875–1918), 399 hierarchy of, 259
circle, 47, 57, 100, 412 of elementary operators, 259
area of, 141 of matrices, 260
secondary, 413 of the identity matrix, 255
540 INDEX
commutivity, 9 of a scalar, 383

noncommutivity of matrix multipli- conditional convergence, 138
cation, 243 cone
noncommutivity of the cross prod- volume of, 141
uct, 409 conjecture, 344
noncommutivity of unary linear op- conjugate, 41, 44, 111, 246
erators, 441 conjugate transpose, 246
of the dot product, 407 of a matrix inverse, 263
summational and integrodifferential, conjugation, 44
137 constant, 32
complete basis, 384 constant expression, 13
completing the square, 14 constant, indeterminate, 32
complex conjugation, 44 constraint, 232
complex contour contour, 147, 165, 412
about a multiple pole, 175 closed, 451
complex coordinate, 403 closed planar, 446
complex exponent, 102 complex, 170, 175, 205, 500
complex exponential, 93, 486 complex, about a multiple pole, 175
and de Moivre’s theorem, 103 derivative product rule along, 456
derivative of, 105 contour infinitesimal, 459, 474
inverse, derivative of, 105 contour integration, 146, 474
properties of, 106 closed, 146
complex number, 5, 41, 67 closed complex, 169, 204
actuality of, 110 complex, 170
being a scalar not a vector, 240 of a vector quantity, 147, 474
conjugating, 44 contract, 123
imaginary part of, 43 contractor, 339
magnitude of, 43 contradiction, proof by, 114, 312
multiplication and division, 43, 67 control, 335
phase of, 43 control surface, aeronautical, 335
real part of, 43 convention, 513
complex plane, 42 convergence, 67, 137, 157
complex power, 76 conditional, 138
complex trigonometrics, 103 domain of, 184
complex variable, 5, 80, 521 slow, 184
component coordinate, 401
of a vector field, 437 complex, 403
components of a vector by subscript, 416 primed and unprimed, 416
composite number, 113 real, 403
compositional uniqueness of, 114 coordinate grid
computer memory, 355 parabolic, 432
computer processor, 295 coordinate rotation, 64
concert hall, 33 coordinates, 64, 457
concision, 416 circular paraboloidal, 435
condition, 375, 382 cyclic progression of, 408
INDEX 541
cylindrical, 64, 457 in spherical coordinates, 467

isotropic, 426 of a curl, 454
logarithmic cylindrical, 426 of a gradient, 455
parabolic, 427, 473 cycle, 484
parabolic cylindrical, 434 integration over a complete, 487
parabolic, in two dimensions, 430 cyclic frequency, 483
parabolic, isotropy of, 432 cyclic progression of coordinates, 408
parabolic, properties of, 432 cylinder, 412
rectangular, 47, 64 parabolic, 434
relations among, 65 cylindrical basis, 412
special, 427 cylindrical coordinates, 64, 401, 457
spherical, 64, 457 parabolic, 434
corner case, 230
corner value, 224 datum, 339
cosine, 47, 407 day, 57
derivative of, 105, 109 days of the week, 439
in complex exponential form, 104 de Moivre’s theorem, 67
law of, 62 and the complex exponential, 103
cost function, 342 de Moivre, Abraham (1667–1754), 67
countability, 484 Debian, 5
counterexample, 243 Debian Free Software Guidelines, 5
Courant, Richard (1888–1972), 3 decomposition
Cramer’s rule, 374 LU , 288
Cramer, Gabriel (1704–1752), 374 QR, 356
cross product, 407 diagonal, 379
nonassociativity of, 409 differences between the Gram-
noncommutivity of, 409 Schmidt and Gauss-Jordan, 359
perpendicularity of, 409 eigenvalue, 379
cross-derivative, 194 Gauss-Jordan, 288
cross-directional curl, 448 Gram-Schmidt, 356
cross-directional derivative, 448 Gram-Schmidt, inverting a matrix
cross-section by, 362
parabolic, 429 orthonormalizing, 356
cross-term, 194 Schur, 385
crosswind, 410 singular-value, 396
cryptography, 113 definite integral, 145
cubic expression, 13, 120, 223, 224 definition, 2, 148
roots of, 227 of vector calculus, 453
cubic formula, 227 definition notation, 13
cubing, 228 degenerate linear system, 318
curl, 446 degenerate matrix, 317, 327
cross-directional, 448 degree of freedom, 334
directional, 446, 451 del (∇), 439
in cylindrical coordinates, 463 delta function, Dirac, 148
in nonrectangular coordinates, 472 implementation of, 485
542 INDEX
sifting property of, 148 of sine, cosine and tangent, 109

delta, Kronecker, 247, 419 of the natural exponential, 94, 109
properties of, 421 of the natural logarithm, 97, 111
sifting property of, 247 of the sine-argument function, 495
denominator, 24, 209 of z a , 81
density, 140 partial, 80, 439
dependent element, 331 product rule for, 83, 84, 199, 283,
dependent variable, 78 452, 456
derivation, 1 second, 85
derivative, 69 unbalanced form, 77
balanced form, 77 with respect to position, 438
chain rule for, 82 derivative pattern, 84
constant, 444 derivative product
cross-, 194 a pattern of, 84
cross-directional, 448 determinant, 365
definition of, 77 and the elementary operator, 370
directional, 442 definition of, 366
higher, 85 inversion by, 373
Jacobian, 282, 341, 348 of a matrix inverse, 372
Leibnitz notation for, 78 of a product, 372
logarithmic, 82 of a unitary matrix, 372
manipulation of, 82 product of two, 372
Newton notation for, 77 properties of, 367
of z a /a, 198 rank-n, 366
of a complex exponential, 105 zero, 371
of a field, 437 deviation, 345
of a field in cylindrical coordinates, DFSG (Debian Free Software Guidelines),
461 5
of a field in cylindrical coordinates, diag notation, the, 269
second-order, 465 diagonal, 38, 47
of a field in spherical coordinates, main, 248, 390
466 three-dimensional, 40
of a field in spherical coordinates, diagonal decomposition, 379
second-order, 468 diagonal matrix, 268, 269
of a field, nonrectangular, 461 diagonalizability, 380
of a field, second-order, 454 diagonalizable matrix, 393
of a function of a complex variable, diagonalization, 379
80 differentiability, 81
of a rational function, 215 differential equation, 202
of a trigonometric, 109 solving by unknown coefficients, 202
of a unit basis vector, 461 differentiation
of an inverse trigonometric, 111 analytical versus numeric, 146
of arcsine, arccosine and arctangent, dimension, 240, 482, 483
111 dimension-limited matrix, 250
of sine and cosine, 105 dimensionality, 53, 248, 321
INDEX 543
dimensionlessness, 47, 484 double root, 229

Dirac delta function, 148 down, 47, 336
implementation of, 485 downstairs, 438
sifting property of, 148 downward multitarget addition operator,
Dirac delta pulse train, 490 270
Dirac, Paul (1902–1984), 148 driving vector, 329, 331, 337
direction, 49, 401 dummy variable, 15, 135, 204
directional curl, 446, 451 duty cycle, 489
directional derivative, 442
in cylindrical coordinates, 461 east, 47, 334, 412
in spherical coordinates, 466 edge
directrix, 428 inner and outer, 451
discontinuity, 147 edge case, 5, 229, 286
discontinuous waveform, 502 efficient implementation, 355
dish antenna, 429 eigensolution, 377, 378
displacement, 458 count of, 379
displacement infinitesimal, 459, 474 impossibility to share, given inde-
distributivity, 9 pendent eigenvectors, 379
of unary linear operators, 441 of a matrix inverse, 378
divergence, 444, 449 repeated, 381, 391
in cylindrical coordinates, 463 eigenvalue, 365, 376
in nonrectangular coordinates, 468 distinct, 378, 380
in spherical coordinates, 467 dominant, 381, 382
of a curl, 455 large, 382
of a gradient, 454 magnitude of, 382
divergence theorem, 449 of a matrix inverse, 377
divergence to infinity, 40 perturbed, 391
divergenceless field, 455 real, 393
dividend, 24 repeated, 378, 391
dividing by zero, 70 shifting of, 378
division, 43, 67 small, 382
by matching coefficients, 28 zero, 377, 382
checking, 145 eigenvalue decomposition, 379
divisor, 24 eigenvalue matrix, 379
domain, 40, 184, 331 eigenvector, 376, 377
sidestepping a, 185 generalized, 392
domain contour, 165 independent, 380
domain neighborhood, 163 linearly independent, 378
dominant eigenvalue, 381, 382 of a matrix inverse, 378
dot product, 349, 407 orthogonal, 393
abbrevated notation for, 416 repeated, 381, 391
commutivity of, 407 Einstein notation, 418, 460
double angle, 57 Einstein’s summation convention, 418, 460
double integral, 140 Einstein, Albert (1879–1955), 418
double pole, 210, 215 electric circuit, 335
544 INDEX
electrical engineer, 335, 504 Euler’s formula, 100

electromagnetic power, 409 curious consequences of, 103
element, 240 Euler, Leonhard (1707–1783), 100, 102,
free and dependent, 331 138, 406
interior and boundary, 449, 451 evaluation, 85
elementary operator, 256 even function, 188
addition, 257, 262 evenness, 264
and the determinant, 370 exact arithmetic, 295, 310, 320, 338, 374,
combination of, 260 382
commutation of, 259 exact matrix, 310
expansion of, 260 exact number, 310
interchange, 256, 261 exact quantity, 310
inverse of, 256, 260 exactly determined linear system, 328
invertibility of, 258 exercises, 150
scaling, 257, 262 existence, 217
sorting of, 259 expansion of 1/(1 − z)n+1 , 154
two, of different kinds, 259 expansion point
elementary similarity transformation, 288 shifting of, 159
elementary vector, 255, 312, 349 exponent, 18, 34
left-over, 314 complex, 102
elevation, 334, 413 exponent, floating-point, 295
elf, 41 exponential, 34, 102
empty set, 286 approximation of to first order, 177
end, justifying the means, 311 complex, 486
engine, 483 general, 96
engineer, 335, 504 integrating a product of a power and,
entire function, 167, 184, 190 219
epsilon, Levi-Civita, 419 natural, error of, 183
properties of, 421 resemblance of to x∞ , 99
equation exponential, complex, 93
simultaneous linear system of, 243, and de Moivre’s theorem, 103
328 exponential, natural, 93
solving a set of simultaneously, 55, compared to xa , 98
243, 328 derivative of, 94, 109
superfluous, 329 existence of, 93
equator, 142 exponential, real, 93
equidistance, 428 extended operator, 249, 251, 362
error decomposing, 306
due to rounding, 338 extension, 4
in the solution to a linear system, extra column, 301, 320
382 extremum, 85, 168
error bound, 179 global, 496
essential singularity, 41, 167, 193 local, 497
Euclid (c. 300 B.C.), 38, 114 of the sine integral, 498
Euler rotations, 406 of the sine-argument function, 496
INDEX 545
factor in trigonometric form, 493

truncating, 303 linearity of, 491
factorial, 15, 200 sufficiency of, 491
factorization, 13 Fourier transform, 507
QR, 356 spatial, 140
full-rank, 319, 344, 359 Fourier, Jean Baptiste Joseph
Gauss-Jordan, 288 (1768–1830), 479, 507
Gram-Schmidt, 356 fraction, 209
orthonormalizing, 356 free element, 331
prime, 114 freedom, degree of, 334
false try, 155, 500 freeway, 339
family of solutions, 336 frequency, 483
fast function, 98 angular, 483
Ferrari, Lodovico (1522–1565), 223, 231 cyclic, 483
field, 437 primary, 479
curl of, 446 shifting of, 487
derivative of, 437 spatial, 484
derivative of, in cylindrical coordi- frontier, 276
nates, 461 Frullani’s integral, 218
derivative of, in spherical coordinates, Frullani, Giuliano (1795–1834), 218
466 full column rank, 318, 320
derivative of, nonrectangular, 461 full rank, 317
derivative product rule for, 452, 456 full row rank, 318, 353
directional derivative of, 442 full-rank factorization, 319, 344, 359
divergence of, 444 function, 40, 148
divergenceless, 455 analytic, 46, 163
gradient of, 442 entire, 167, 184, 190
irrotational, 455 extremum of, 85
second-order derivative of, 454 fast, 98
solenoidal, 455 fitting of, 153
source-free, 455 linear, 136
first-order approximation, 177 meromorphic, 167, 190
flaw, logical, 116 nonanalytic, 46
floating-point number, 295, 382 nonlinear, 136
floor, 438 odd or even, 188
flux, 444, 449 of a complex variable, 80
focus, 428 of position, 437
football, 444 rational, 209
forbidden point, 161, 164 rational, derivatives of, 215
formalism, 136 rational, integral of, 213
Fortran, 17 single- and multiple-valued, 164, 166
fourfold integral, 140 slow, 98
Fourier coefficient, 486 fundamental theorem of algebra, 119
recovery of, 487 fundamental theorem of calculus, 133,
Fourier series, 479, 486 450
546 INDEX
gamma function, 200 Gram, Jørgen Pedersen (1850–1916), 353

Gauss, Carl Friedrich (1777–1855), 288, Gram-Schmidt decomposition, 356
330 differences of against the Gauss-
Gauss-Jordan algorithm, 294 Jordan, 359
Gauss-Jordan decomposition, 288 factor Q of, 359
and full column rank, 320 factor S of, 359
differences of against the Gram- inverting a matrix by, 362
Schmidt, 359 Gram-Schmidt kernel formula, 360
factors K and S of, 320 Gram-Schmidt process, 353
inverting the factors of, 303 grapes, 111
Gauss-Jordan factorization, 288 Greek alphabet, 517
Gauss-Jordan factors Greenwich, 57
properties of, 305 grid, parabolic coordinate, 432
Gauss-Jordan kernel formula, 330 guessing roots, 233
general exponential, 96 guessing the form of a solution, 201
general identity matrix, 251 gunpowder, 334
general interchange operator, 267
General Public License, GNU, 5 half angle, 57
general scaling operator, 268 Hamming, Richard W. (1915–1998), 127,
general solution, 338 158
general triangular matrix, 272, 390 handwriting, reflected, 111
generality, 330 harmonic mean, 124
generalized eigenvector, 392 harmonic series, 182
geometric majorization, 182 headwind, 410
geometric mean, 124 Heaviside unit step function, 147
geometric series, 31 Heaviside, Oliver (1850–1925), 3, 110,
majorization by, 182 147, 440
variations on, 32 Helmholtz, Hermann Ludwig Ferdinand
geometrical argument, 2, 468 von (1821–1894), 455
geometrical vector, 401 Hermite, Charles (1822–1901), 246, 392
geometrical visualization, 287, 468 Hermitian matrix, 392
geometry, 36 hertz, 483
Gibbs phenomenon, 502 Hessenberg matrix, 392
Gibbs, Josiah Willard (1839–1903), 502 Hessenberg, Gerhard (1874–1925), 392
GNU General Public License, 5 hexadecimal, 514
GNU GPL, 5 higher-order algebra, 223
goal post and goal line, 444 Hilbert, David (1862–1943), 3
Goldman, William (1931–), 148 homogeneous solution, 337
Goursat, Edouard (1858–1936), 523 horse, 334
GPL, 5 hour, 57
gradient, 442 hour angle, 57
in cylindrical coordinates, 461 hut, 438
in nonrectangular coordinates, 473 hyperbolic functions, 104
in spherical coordinates, 466 properties of, 105
of a divergence, 454 hypotenuse, 38
INDEX 547
hypothesis, 332 inner edge, 451

inner product, 349
identity inner surface, 449
differential, of the vector, 451 insight, 482
of vector calculus, 453 integer, 15, 18
vector, algebraic, 424 composite, 113
identity matrix, 251, 254 compositional uniqueness of, 114
r-dimensional, 254 prime, 113
commutation of, 255 integrability, 450
impossibility to promote of, 312 integral, 127
rank-r, 254 and series summation, 181
identity, arithmetic, 9 as accretion or area, 128
iff, 121, 136, 157 as antiderivative, 133
ill conditioned matrix, 375, 382 as shortcut to a sum, 129
imaginary number, 41 balanced form, 131
imaginary part, 43 closed complex contour, 169, 204
imaginary unit, 41 closed contour, 146
implementation closed surface, 144
efficient, 355 complex contour, 170
imprecise quantity, 310, 382 concept of, 127
indefinite integral, 145 contour, 146, 474
independence, 286, 317 definite, 145
independent infinitesimal double, 140
variable, 132 fourfold, 140
independent variable, 78 indefinite, 145
indeterminate form, 88 magnitude of, 208
index, 15, 242 multiple, 139
of multiplication, 15 of a rational function, 213
of summation, 15 of the sine-argument function, 495
induction, 44, 156 sixfold, 140
inequality, 12, 345 surface, 140, 142, 474
infinite differentiability, 163 swapping one vector form for an-
infinite dimensionality, 248 other in, 450, 451
infinitesimal, 70 triple, 140
and the Leibnitz notation, 78 vector contour, 147, 474
displacement, 459, 474 volume, 140
dropping of when negligible, 170 integral forms of vector calculus, 449
independent, variable, 132 integral swapping, 140
practical size of, 70 integrand, 197
second- and higher-order, 71 magnitude of, 208
surface, 458, 474 integration
vector, 474 analytical versus numeric, 146
volume, 459 as summation, 503
infinity, 70 by antiderivative, 197
inflection, 85 by closed contour, 204
548 INDEX
by partial-fraction expansion, 209 of parabolic coordinates, 432

by parts, 199 iteration, 88, 382
by substitution, 198
by Taylor series, 221 Jacobi, Carl Gustav Jacob (1804–1851),
by unknown coefficients, 201 282
checking of, 145 Jacobian derivative, 282, 341, 348
of a product of exponentials, powers jet engine, 414
and logarithms, 219 Jordan, Wilhelm (1842–1899), 288, 330
over a complete cycle, 487
integration techniques, 197 Kelvin, Lord (1824–1907), 479, 507
Intel, 295 kernel, 329
interchange, 357, 367 alternate formula for, 332
refusing an, 320, 356 Gauss-Jordan formula for, 330
interchange operator Gram-Schmidt formula for, 360
elementary, 256 kernel matrix, 329
general, 267 converting between two of, 334
interchange quasielementary, 267 kernel space, 330, 334
interest, 82, 202 Kronecker delta, 247, 419
interior element, 449, 451 properties of, 421
internal-combustion engine, 483 sifting property of, 247
invariance under the reorientation of axes, Kronecker, Leopold (1823–1891), 247, 419
407, 408, 439, 454, 455
inverse l’Hôpital’s rule, 87, 189
determinant of, 372 l’Hôpital, Guillaume de (1661–1704), 87
existence of, 326 labor union, 339
mutual, 326 Laplace transform, 507
of a matrix product, 263 Laplace, Pierre-Simon (1749–1827), 454,
of a matrix transpose or adjoint, 263 507
rank-r, 324 Laplacian, 454
uniqueness of, 326 Laurent series, 23, 190
inverse complex exponential Laurent, Pierre Alphonse (1813–1854),
derivative of, 105 23, 190
inverse trigonometric family of functions, law of cosines, 62
105 law of sines, 61
inversion, 260, 323, 324 least squares, 339
by determinant, 373 least-squares solution, 341
multiplicative, 9 leftward multitarget addition operator,
symbolic, 365, 373 271
inversion, arithmetic, 9 leg, 38
invertibility, 343 Leibnitz notation, 78, 132
of the elementary operator, 258 Leibnitz, Gottfried Wilhelm (1646–1716),
irrational number, 117 69, 78
irreducibility, 2 length, 457
irrotational field, 455 curved, 47
isotropy, 426 preservation of, 362
INDEX 549
length infinitesimal, 459, 474 logarithmic cylindrical coordinates, 426

Levi-Civita epsilon, 419 logarithmic derivative, 82
properties of, 421 logic, 311
Levi-Civita, Tullio (1873–1941), 419 reverse, 345
light ray, 429 logical flaw, 116
limit, 71 lone-element matrix, 255
line, 53, 336 long division, 24, 190
fitting a, 339 by z − α, 118
linear algebra, 239 procedure for, 27, 29
linear combination, 136, 286, 317 loop, 356
linear dependence, 286, 317 loop counter, 15
linear expression, 13, 136, 223 lower triangular matrix, 271
linear independence, 286, 317
linear operator, 136 Maclaurin series, 162
unary, 441 Maclaurin, Colin (1698–1746), 162
linear superposition, 174 magnification, 382
linear system magnitude, 43, 67
classification of, 318 of a vector, 350
degenerate, 318 of an eigenvalue, 382
exactly determined, 328 of an integral, 208
nonoverdetermined, 336 preservation of, 362
nonoverdetermined, general solution unit, 382
of, 338 main diagonal, 248, 390
nonoverdetermined, particular solu- majority, 356
tion of, 337 majorization, 157, 179, 180
overdetermined, 318, 339, 340 geometric, 182
taxonomy of, 318 maneuver, logical, 311
underdetermined, 318 mantissa, 295
linear transformation, 242 mapping, 40
linearity, 136 marking
of a function, 136 permutor, 366
of an operator, 136, 441 marking quasielementary, 366
of the Fourier series, 491 mason, 120
loan, 202 mass density, 140
locus, 119 matching coefficients, 28
logarithm, 34 mathematician
integrating a product of a power and, applied, 1
219 professional, 2, 116, 217
properties of, 35 mathematics
resemblance of to x0 , 99 applied, 1, 148, 158
logarithm, natural, 96 applied, foundations of, 239
and the antiderivative, 198 professional or pure, 2, 116, 148, 521
compared to xa , 98 matrix, 239, 240
derivative of, 97, 111 addition of, 243
error of, 183 arithmetic of, 243
550 INDEX
associativity of the multiplication of, rank-r inverse of, 264

243 real, 341
basic operations of, 242 rectangular, 327
basic use of, 242 row of, 296
broad, 317, 336 scalar, 251
column of, 296, 331, 357 self-adjoint, 392
commutation of, 260 singular, 327, 382
condition of, 382 singular-value, 396
degenerate, 317, 327 sparse, 252
diagonalizable, 393 square, 248, 317, 320, 324
dimension-limited, 250 square, coincident properties of, 374
dimensionality of, 321 tall, 317, 320
eigenvalue, 379 triangular, construction of, 273
exact, 310 truncating, 255
form of, 248 unit lower triangular, 271
full-rank, 317 unit parallel triangular, 275
general identity, 251 unit triangular, 271
Hermitian, 392 unit triangular, partial, 280
identity, 251, 254 unit upper triangular, 271
identity, impossibility to promote of, unitary, 361
312 matrix operator, 248, 441
ill conditioned, 375, 382 matrix rudiments, 239, 285
inversion of, 260, 323, 324, 373 matrix vector, 240, 401
inversion properties of, 264 maximum, 85
large, 338, 374 mean, 123
lone-element, 255 arithmetic, 124
main diagonal of, 248, 390 geometric, 124
motivation for, 239, 242 harmonic, 124
multiplication of, 243, 248 means, justified by the end, 311
multiplication of by a scalar, 243 mechanical engineer, 504
noncommutivity of the multiplica- membership, 17
tion of, 243 memory, computer, 355
nondiagonalizable, 390 meromorphic function, 167, 190
nondiagonalizable versus singular, 381 metric, 342
null, 250 metric coefficient, 457, 458
null column of, 357 minimum, 85
orthogonally complementary, 353 minorization, 181
padding of with zeros, 248 mirror, 111
parallel unit triangular, properties parabolic, 429
of, 279 mnemonic, 407
perpendicular, 353 model, 2, 116, 411, 457
projection, 398 modulus, 43
provenance of, 242 Moivre, Abraham de (1667–1754), 67
raised to a complex power, 380 Moore, E.H. (1862–1932), 339
rank of, 315 Moore-Penrose pseudoinverse, 339, 359
INDEX 551
motivation, 328 nonanalytic function, 46

mountain road, 336 nonanalytic point, 164, 173
multiple pole, 40, 210, 215 nonassociativity
enclosing, 175 of the cross product, 409
multiple-valued function, 164, 166 noncommutivity
multiplication, 15, 43, 67 of matrix multiplication, 243
index of, 15 of the cross product, 409
of a vector, 406 nondiagonalizable matrix, 390
of a vector by a scalar, 406 versus a singular matrix, 381
of matrices, 243, 248 noninvertibility, 326
multiplicative inversion, 9 nonlinearity, 335
multiplier, 239 nonnegative definiteness, 344, 379
multitarget addition operator, 269 nonoverdetermined linear system, 336
downward, 270 general solution of, 338
leftward, 271 particular solution of, 337
rightward, 271 nonrectangular notation, 460
upward, 270 normal unit vector, 412
multivariate Newton-Raphson iteration, normal vector or line, 141
348 normalization, 353, 358, 382
north, 47, 334
Napoleon, 334 notation
natural exponential, 93 for the vector, concise, 416
compared to xa , 98 for the vector, nonrectangular, 460
complex, 93 of the operator, 441
derivative of, 94, 109 of the vector, 401
error of, 183 null column, 357
existence of, 93 null matrix, 250
real, 93 null vector, 286
natural exponential family of functions, null vector identity, 455
105 number, 41
natural logarithm, 96 complex, 5, 41, 67
and the antiderivative, 198 complex, actuality of, 110
compared to xa , 98 exact, 310
derivative of, 97, 111 imaginary, 41
error of, 183 irrational, 117
of a complex number, 103 rational, 117
natural logarithmic family of functions, real, 41, 403
105 very large or very small, 70
neighborhood, 163 number theory, 113
nesting, 356 numerator, 24, 209
Newton, Sir Isaac (1642–1727), 69, 77,
88, 348 Observatory, Old Royal, 57
Newton-Raphson iteration, 88, 223 Occam’s razor
multivariate, 348 abusing, 112
nobleman, 148 odd function, 188
552 INDEX
oddness, 264 padding a matrix with zeros, 248

off-diagonal entries, 256 parabola, 428
Old Royal Observatory, 57 parabolic antenna, 429
one, 9, 47, 251 parabolic arc, 428
operator, 135, 439, 441 parabolic coordinate grid, 432
+ and − as, 135 parabolic coordinates, 427, 473
downward multitarget addition, 270 in two dimensions, 430
elementary, 256 isotropy of, 432
general interchange, 267 properties of, 432
general scaling, 268 parabolic cross-section, 429
leftward multitarget addition, 271 parabolic cylinder, 434
linear, 136, 441 parabolic cylindrical coordinates, 434
multitarget addition, 269 parabolic mirror, 429
nonlinear, 136 parabolic track, 430
quasielementary, 266 paraboloid, 435
rightward multitarget addition, 271 paraboloidal coordinates, 435
truncation, 255 parallel addition, 120, 224
unary, 441 parallel subtraction, 123
parallel unit triangular matrix, 275
unary linear, 441
properties of, 279
unresolved, 442
parameter, 482
upward multitarget addition, 270
parity, 264, 365, 408
using a variable up, 135
and the Levi-Civita epsilon, 419
operator notation, 441
Parseval’s principle, 211, 480
optimality, 342
Parseval, Marc-Antoine (1755–1836), 211,
order, 13
480
orientation, 53 partial derivative, 80, 439
origin, 47, 53 partial sum, 179
orthogonal basis, 410 partial unit triangular matrix, 280
constant, 463 partial-fraction expansion, 209
variable, 411 particular solution, 337
orthogonal complement, 353 Pascal’s triangle, 74
orthogonal vector, 350, 407 neighbors in, 73
orthogonalization, 354, 358 Pascal, Blaise (1623–1662), 74
orthonormal rows and columns, 361 patch, 474
orthonormal vectors, 64 path integration, 146
orthonormalization, 323, 353 pattern
orthonormalizing decomposition, 356 derivative product, 84
inverting a matrix by, 362 payment rate, 202
oscillation, 502 pencil, 305, 332
outer edge, 451 Penrose, Roger (1931–), 339
outer surface, 449 period, 479
overdetermined linear system, 318, 339, permutation, 72
340 permutation matrix, 267
overshot, 502 permutor, 267, 365
INDEX 553
marking, 366 real, 20

perpendicular matrix, 353 sum of, 22
perpendicular unit vector, 412 power series, 23, 46
perturbation, 210, 391 bounds on, 179
Pfufnik, Gorbag, 166 common quotients of, 31
phase, 43 derivative of, 77
physical insight, 482 division of, 24
physical unit, 482 division of by matching coefficients,
physical world, 401 28
physicist, 3 extending the technique of, 32
pilot, 335 multiplying, 24
pitch, 404 shifting the expansion point of, 159
pivot, 294 with negative powers, 23
small, 338 pressure, 437
Planck, Max (1858–1947), 32 primary frequency, 479
plane, 53, 336 prime factorization, 114
projection onto, 424 prime mark (′ ), 53, 416
plausible assumption, 114 prime number, 113
point, 53, 64, 336 infinite supply of, 113
in vector notation, 53 relative, 235
pole, 40, 87, 164, 166, 173, 190 primed coordinate, 416
circle of, 211 processor, computer, 295
double, 210, 215 product, 15
multiple, 40, 210, 215 determinant of, 372
multiple, enclosing a, 175 dot or inner, 349
of a trigonometric function, 189 of a vector and a scalar, 406
repeated, 210, 215 of determinants, 372
separation of, 210 of vectors, 406
polygon, 36 product rule, derivative, 83, 199, 283
polynomial, 23, 118 a pattern of, 84
characteristic, 376, 390, 392 of the contour, 456
having at least one root, 119 of the vector, 452
of order N having N roots, 119 productivity, 123
position vector, 414 professional mathematician, 116, 217
positive definiteness, 344, 379 professional mathematics, 2, 116, 148,
potentiometer, 335 521
power, 18 progression of coordinates, cyclic, 408
complex, 76 projectile, 428
electromagnetic, 409 projection
fractional, 20 onto a plane, 424
integral, 18 projector, 398
notation for, 18 prolixity, 416
of a power, 21 proof, 1
of a product, 21 by contradiction, 114, 312
properties of, 18 by induction, 44, 156
554 INDEX
by sketch, 2, 36, 461, 497 rank, 310, 315

propagation speed, 484 and independent rows, 302
proving backward, 125 column, 318, 320, 329, 343
proviso, 330 full, 317
pseudoinverse, 339, 359 impossibility to promote of, 312
pulse, 485 maximum, 302
square, 485 row, 318, 353
triangular, 485 uniqueness of, 315
pulse train, 488 rank-n determinant, 366
rectangular, 488 rank-r inverse, 264, 324
pure mathematics, 2, 116, 148, 521 Raphson, Joseph (1648–1715), 88, 348
pyramid rate, 69
volume of, 141 relative, 82
Pythagoras (c. 580–c. 500 B.C.), 38 ratio, 20, 209
Pythagorean theorem, 38 fully reduced, 117
and the hyperbolic functions, 104 rational function, 209
and the sine and cosine functions, derivatives of, 215
49 integral of, 213
in three dimensions, 40 rational number, 117
rational root, 235
quadrant, 47 ray, 429
quadratic expression, 13, 120, 223, 226 real coordinate, 403
quadratic formula, 14 real exponential, 93
quadratics, 13 real number, 41, 403
quantity approximation of as a ratio of inte-
exact, 310 gers, 20
imprecise, 310 real part, 43
quartic expression, 13, 120, 223, 231 reciprocal, 9, 327
resolvent cubic of, 232 reciprocal pair, 326
roots of, 234 rectangle, 9
quartic formula, 234 splitting of down the diagonal, 36
quasielementary operator, 266 rectangular coordinates, 47, 64, 401
addition, 269 rectangular matrix, 327
interchange, 267 rectangular pulse train, 488
marking, 366 reference vector, 442
row addition, 270 regular part, 174, 190
scaling, 268 relative primeness, 235
quintic expression, 13, 120, 235 relative rate, 82
quiz, 413, 419 remainder, 24
quotient, 24, 28, 209 after division by z − α, 118
zero, 118
radian, 47, 57, 484 reorientation, 404
radius, 47 invariance under, 407, 408, 439, 454,
range, 40, 331 455
range contour, 165 repeated eigensolution, 391
INDEX 555
repeated eigenvalue, 378, 381, 391 null, appending a, 327

repeated pole, 210, 215 orthonormal, 361
repeating waveform, 479 scaled and repeated, 368
repetition, unseemly, 419 row addition quasielementary, 270
residual, 182, 338 row operator, 245
minimizing the, 344 row rank
squared norm of, 339 full, 318
residue, 174, 209 row vector, 349
resolvent cubic, 232 Royal Observatory, Old, 57
reverse logic, 345 RPM, 483
reversibility, 288, 315 rudder, 335
revolution, 47 rudiments, 239, 285
right triangle, 36, 47 run, 47
right-hand rule, 53, 408, 412, 435
rightward multitarget addition operator, satellite dish antenna, 429
271 Saturday, 339
rigor, 2, 138, 158 scalar, 49, 240
ringing, 503 complex, 52
rise, 47 condition of, 383
road scalar field, 437, 457
mountain, 336 directional derivative of, 442
winding, 410 gradient of, 442
roll, 404 scalar matrix, 251
Roman alphabet, 517 scalar multiplication
roof, 438 of a vector, 406
root, 13, 20, 40, 87, 118, 223 scaling, 367
double, 229 scaling operator
finding of numerically, 88, 348 elementary, 257
guessing of, 233 general, 268
rational, 235 scaling quasielementary, 268
superfluous, 227 schematic, 305, 332
triple, 230 Schmidt, Erhard (1876–1959), 353
root extraction Schur decomposition, 385
from a cubic polynomial, 227 Schur, Issai (1875–1941), 272, 385
from a quadratic polynomial, 14 screw, 53
from a quartic polynomial, 234 sea
root-length, 427 wavy surface of, 412
rotation, 53, 401 second, 484
angle of, 55 second derivative, 85
Euler, 406 secondary circle, 413
Tait-Bryan or Cardano, 404 secondary cylindrical basis, 413
rounding error, 338 selection from among wooden blocks, 72
row, 240, 296 self-adjoint matrix, 392
addition of, 369 separation of poles, 210
null, 368 serial addition, 120
556 INDEX
series, 15 determinant of, 371

arithmetic, 17 versus a nondiagonalizable matrix,
convergence of, 67 381
Fourier, 479, 486 singular value, 396
geometric, 31, 182 singular-value decomposition, 396
geometric, variations on, 32 singular-value matrix, 396
harmonic, 182 singularity, 40
multiplication order of, 16 essential, 41, 167, 193
notation for, 15 sink, 444
product of, 15 sinusoid, 49, 486
sum of, 15 superposition of, 479
Taylor, 153, 161 sixfold integral, 140
series addition, 120 skepticism, 382
set, 17 sketch, proof by, 2, 36, 461, 497
shape sky, 438
area of, 141 slope, 47, 85
shift operator, 282, 330 slow convergence, 184
shifting an eigenvalue, 378 slow function, 98
shifting an expansion point, 159 solenoidal field, 455
sifting property, 148, 247 solid
sign surface area of, 142
alternating, 179, 496 volume of, 141
similarity, 384 solution, 323
similarity transformation, 260, 288, 383 error in, 382
Simpson’s rule, 132 family of, 336
simultaneous system of linear equations, general, 338
243, 328 guessing the form of, 201
sinc function, 494 of least-squares, 341
sine, 47, 409 particular, 337
approximation of to first order, 177 particular and homogeneous, 337
derivative of, 105, 109 sound, 32
in complex exponential form, 104 source, 276, 444
law of, 61 source-free field, 455
sine integral, 495 south, 47, 334, 412
evaluation of by complex contour, space, 53, 307, 330, 483
499 address of, 321
properties of, 497 three-dimensional, 409
Taylor series for, 495 two-dimensional, 409
sine-argument function, 494 space and time, 140
derivative of, 495 spare column, 301, 320
integral of, 495 sparsity, 252
properties of, 496 spatial frequency, 484
Taylor series for, 494 speed, 410
single-valued function, 164, 166 of propagation, 484
singular matrix, 327, 382 sphere, 65, 402, 413
INDEX 557
surface area of, 142 superposition, 111, 174, 368

volume of, 144 of sinusoids, 479
spherical basis, 413 surface, 139
spherical coordinates, 64, 401, 457 closed, 444, 449
spherical surface, 458 inner and outer, 449
split form, 337 orientation of, 412
square, 57 spherical, 458
tilted, 38 surface area, 142
square matrix, 248, 320, 324 surface element, 451, 474
coincident properties of, 374 surface infinitesimal, 458, 474
degenerate, 327 surface integration, 140, 142, 474
square pulse, 485 closed, 144
square root, 20, 41 surface normal, 412
calculation of by Newton-Raphson, sweeping out a length, 458
90 swimming pool, 336
square wave, 479, 488 symmetry, 188, 309
square, completing the, 14 appeal to, 53, 482
squared residual norm, 339
squares, least, 339 Tait, Peter Guthrie (1831–1901), 404
squares, sum or difference of, 13 Tait-Bryan rotations, 404
squaring, 228 tall matrix, 320
steepest rate, 443 tangent, 47
step in every direction, 480 compared against its argument, 498
Stokes’ theorem, 451 derivative of, 109
Stokes, Sir George Gabriel (1819–1903), in complex exponential form, 103
451 tangent line, 88, 94
strictly triangular matrix, 272 tapered strip, 142
strip, tapered, 142 target, 276
style, 3, 116, 150 Tartaglia, Niccolò Fontana (1499–1557),
subscript 223
indicating the components of a vec- tautology, 415
tor by, 416 taxonomy
subtraction of the linear system, 318
parallel, 123 Taylor expansion, first-order, 76, 177
sum, 15 Taylor series, 153, 161
partial, 179 converting a power series to, 159
weighted, 286 for specific functions, 176
summation, 15 for the sine integral, 495
as integration, 503 for the sine-argument function, 494
compared to integration, 181 in 1/z, 193
convergence of, 67 integration by, 221
index of, 15 multidimensional, 194
summation convention, Einstein’s, 418, transposition of to a different ex-
460 pansion point, 163
superfluous root, 227 Taylor, Brook (1685–1731), 153, 161
558 INDEX
technician, 335 triviality, 286

term truncation, 303
cross-, 194 truncation operator, 255
finite number of, 179 Tuesday, 439
theory, 328 tuning, 335
three-dimensional geometrical vector, 401 two-dimensional geometrical vector, 403
three-dimensional space, 409, 434 two-dimensional space, 409, 430
Thursday, 439
time, 483 unary linear operator, 441
time and space, 140 unary operator, 441
transform, 507 unresolved, 442
transformation, linear, 242 underdetermined linear system, 318
transpose, 246, 369 uniqueness, 217, 327
conjugate, 246 of matrix rank, 315
of a matrix inverse, 263 unit, 41, 47, 49
trapezoid rule, 132 imaginary, 41
tree, 438 physical, 482
triangle, 36, 61 real, 41
area of, 36 unit basis vector, 49
equilateral, 57 cylindrical, 64, 460
right, 36, 47 derivative of, 461
triangle inequalities, 36 spherical, 64, 460
complex, 67, 208 variable, 64, 460
complex vector, 351 unit circle, 47
vector, 67, 351 unit lower triangular matrix, 271
triangular matrix, 272, 390 unit magnitude, 382
construction of, 273 unit normal, 412
parallel, properties of, 279 unit step function, Heaviside, 147
partial, 280 unit triangular matrix, 271
unit parallel, 275 construction of, 273
triangular pulse, 485 parallel, properties of, 279
trigonometric family of functions, 105 partial, 280
trigonometric function, 47 unit upper triangular matrix, 271
derivative of, 109 unit vector, 49, 350, 460
inverse, 47 normal or perpendicular, 412
inverse, derivative of, 111 unitary matrix, 361
of a double or half angle, 57 determinant of, 372
of a sum or difference of angles, 55 unitary similarity, 384
of an hour angle, 57 unitary transformation, 384
poles of, 189 unity, 9, 47, 49
trigonometrics, complex, 103 unknown coefficient, 201
trigonometry, 47 unresolved operator, 442
properties of, 50, 63 unsureness, logical, 116
triple integral, 140 up, 47, 336, 412
triple root, 230 upper triangular matrix, 271
INDEX 559
upstairs, 438 notation for, 49

upward multitarget addition operator, 270 orientation of, 349
utility variable, 61 orthogonal, 350, 407
orthogonalization of, 354
variable, 32 orthonormal, 64
assignment, 12 orthonormalization of, 353
change of, 12 point, 53
complex, 5, 80, 521 position, 414
definition notation for, 13 projection of onto a plane, 424
dependent, 32, 78 reference, 442
independent, 32, 78 replacement of, 307, 334
utility, 61 rotation of, 53
variable independent infinitesimal, 132 row, 349
variable dτ , 132 row of, 240
vector, 49, 194, 240, 401 scalar multiplication of, 406
n-dimensional, 240, 401 second-order derivative of, 454
addition of, 403 three-dimensional, 51
algebraic identities of, 424 three-dimensional geometrical, 401
angle between two, 350, 407 two-dimensional, 51
arbitrary, 286, 307 two-dimensional geometrical, 403
building of from basis vectors, 383 unit, 49, 350, 460
column, 312, 349 unit basis, 49
concise notation for, 416 unit basis, cylindrical, 64, 460
derivative of, 437 unit basis, derivative of, 461
derivative of, in cylindrical coordi- unit basis, spherical, 64, 460
nates, 461 unit basis, variable, 64, 460
derivative of, in spherical coordinates, zero or null, 286
466 vector algebra, 401
derivative of, nonrectangular, 461 vector analysis, 401
derivative product rule for, 452, 456 vector calculus, 437
differential identities of, 451 definitions and identities of, 453
dot or inner product of two, 349, 407 integral forms of, 449
driving, 329, 331 vector field, 437
elementary, 255, 312, 349 components of, 460
elementary, left-over, 314 curl of, 446
ersatz, 439 decomposition of, 459
generalized, 194, 240 directional derivative of, 442
integer, 194 divergence of, 444
local, 414 vector infinitesimal, 474
magnitude of, 350 vector notation, 401
matrix, 240, 401 vector space, 307, 330, 331
multiplication of, 406 address of, 321, 334
nonnegative integer, 194 vector, driving, 337
nonrectangular notation for, 460 velocity, 410, 437
normalization of, 353 local, 414
560 INDEX
vertex, 141 vector, 250

Vieta’s parallel transform, 224 zero matrix, 250
Vieta’s substitution, 224 zero vector, 250, 286
Vieta’s transform, 224, 225
Vieta, Franciscus (François Viète, 1540–
1603), 223, 224
visualization, geometrical, 287
volume, 9, 139, 141, 458
enclosed by a surface, 444, 449
in a spherical geometry, 459
volume element, 449
volume infinitesimal, 459
volume integration, 140
wave
complex, 111
propagating, 111
square, 479, 488
waveform
approximation of, 479
discontinuous, 502
real, 494
repeating, 479
wavy sea, 412
Wednesday, 439
week
days of, 439
Weierstrass, Karl Wilhelm Theodor (1815–
1897), 157
weighted sum, 286
west, 47, 334
Wilbraham, Henry (1825–1883), 502
wind, 410, 437
winding road, 410
wooden block, 72
worker, 339
world, physical, 401
x86-class computer processor, 295
yaw, 404
zero, 9, 40
dividing by, 70
matrix, 250
padding a matrix with, 248

Applied Math Derivations

Uploaded by

Copyright:

Available Formats

Applied Math Derivations

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Applied Math Derivations

Uploaded by

Copyright:

Available Formats

Derivations of Applied Mathematics

Revised 7 January 2009

Thaddeus H. Black, 1967–.

c 1983–2009 by Thaddeus H. Black [email protected].

Published by the Debian Project [15].

I The calculus of a single variable 7

2 Classical algebra and geometry 9

2.6.1 Multiplying power series . . . . . . . . . . . . . . . . 24

4.2.1 Combinations and permutations . . . . . . . . . . . . 72

5 The complex exponential 93

6 Primes, roots and averages 113

6.2.1 Polynomial roots . . . . . . . . . . . . . . . . . . . . . 118

7 The integral 127

8 The Taylor series 153

8.8.1 The meaning of the symbol dz . . . . . . . . . . . . . 170

9 Integration techniques 197

10 Cubics and quartics 223

10.4 Edge cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

II Matrices and vectors 237

11 The matrix 239

12 Rank and the Gauss-Jordan 285

13 Inversion and orthonormalization 323

13.6.4 The Moore-Penrose pseudoinverse . . . . . . . . . . . 344

14 The eigenvalue 365

15 Vector analysis 401

15.2.2 The cross product . . . . . . . . . . . . . . . . . . . . 407

16 Vector calculus 437

III Transforms, special functions and other topics 477

17 The Fourier series 479

18 The Fourier and Laplace transforms 507

A Hexadecimal notation, et al. 513

B The Greek alphabet 517

C A sketch of pure complex theory 521

D Manuscript history 525

2.1 Basic properties of arithmetic. . . . . . . . . . . . . . . . . . . 10

3.1 Simple properties of the trigonometric functions. . . . . . . . 50

5.1 Complex exponential properties. . . . . . . . . . . . . . . . . 106

6.1 Parallel and serial addition identities. . . . . . . . . . . . . . . 122

7.1 Basic derivatives for the antiderivative. . . . . . . . . . . . . . 134

8.1 Taylor series. . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

9.1 Antiderivatives of products of exps, powers and logs. . . . . . 221

11.1 Elementary operators: interchange. . . . . . . . . . . . . . . . 261

12.1 Some elementary similarity transformations. . . . . . . . . . . 289

15.1 Properties of the Kronecker delta and Levi-Civita epsilon. . . 421

16.1 Definitions and identities of vector calculus. . . . . . . . . . . 453

B.1 The Roman and Greek alphabets. . . . . . . . . . . . . . . . . 518

1.1 Two triangles. . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.1 Multiplicative commutivity. . . . . . . . . . . . . . . . . . . . 10

3.1 The sine and the cosine. . . . . . . . . . . . . . . . . . . . . . 48

4.1 The plan for Pascal’s triangle. . . . . . . . . . . . . . . . . . . 74

5.1 The natural exponential. . . . . . . . . . . . . . . . . . . . . . 96

7.1 Areas representing discrete sums. . . . . . . . . . . . . . . . . 128

7.2 An area representing an infinite sum of infinitesimals. . . . . 130

8.1 A complex contour of integration in two segments. . . . . . . 171

9.1 Integration by closed contour. . . . . . . . . . . . . . . . . . . 205

10.1 Vieta’s transform, plotted logarithmically. . . . . . . . . . . . 225

13.1 Fitting a line to measured data. . . . . . . . . . . . . . . . . . 340

15.1 A point on a sphere. . . . . . . . . . . . . . . . . . . . . . . . 402

17.1 A square wave. . . . . . . . . . . . . . . . . . . . . . . . . . . 480

I never meant to write this book. It emerged unheralded, unexpectedly.