Notes 5 – Matrix addition and multiplication

L5.1 Matrices and their entries. A matrix is a rectangular array of numbers. Examples:
 
  0 0 0 0 1
1 1
  0 0 0 0  
1 2 3 0 
0
 1 2
  1 0 0 0
0 6 9 0 
0
 1 4
0 1 0 0
0
0 0 0 1 0

The individual numbers are called the entries, elements, or components of the matrix. If the
matrix has m rows and n columns, we say that it has size ‘m by n’ or m×n. The above
examples have respective sizes 2×3, 4×1, 5×5, 2×2. If m = n (as in the last two cases) the
matrix is obviously square.
The set of matrices of size m × n whose entries are real numbers is denoted by Rm,n ; the
first superscript is always the number of rows. Sometimes we use symbols to represent
unspecified numbers, so the statement
 
a b
∈ R2,2
c d

is tantamount to saying that a, b, c, d are real numbers.

For matrices with more than about 4 entries, it is convenient to use subscripts to label the
entries. Given a matrix A, we typically denote by aij the entry in the ith row and j th
column (lower case to emphasize that the entry is a number).
Example. In this notation, the generic 3×4 matrix is
 
a11 a12 a13 a14
 a21 a22 a23 a24  .
a31 a32 a33 a34

Mathematicians like to deal in generalities and will even write a matrix as A = (aij ) without
specifying its size.

Definition. The transpose of a matrix A is the matrix, indicated tA or A> , is obtained by

interchanging its rows and columns.

For example
 T  
  0 0 0 0 1 0 1 0 0 0
1 1 0 0 0 0 0 0 1 0 0
(1, 2, −7)> =  2  ,
   
 =
0 1 0 0 0 0 0 0 1 0
  
−7 0 0 1 0 0 0 0 0 0 1
0 0 0 1 0 1 0 0 0 0

If A ∈ Rm,n then A> ∈ Rn,m . In subscript notation, we have

(A> )ij = aji .

Notice that (A> )> = A, so the operation of taking the transpose is self-inverse.
L5.2 Vectors. Of special importance are matrices that have only one row or column; they are
called row and column vectors. In writing a row vector with digits, it is useful to use commas
to separate the entries. For example, both the matrices
 
1
A = (1, 2, −7) ∈ R1,3 , B =  2  ∈ R3,1
−7

can be used to represent the point in space with Cartesian coordinates x = 1, y = 2, z = −7.
(Sometimes commas are used to distinguish between matrices and row vectors, but it is
simpler to regard them as the same object.)
One can switch between row and column vectors by observing that A = B> or B = A> . For
this reason, the distinction between a row vector and a column vector is often unimportant,
and the sets R1,n and Rn,1 can be written more simply Rn , and we can refer to both A ∈ R3
and B ∈ R3 as ‘vectors’ of length 3. We shall use such vectors to study analytic geometry
later in the course.
Whenever we write ‘Rn ’ the reader is free to use row or column vectors as he or she prefers;
if such a choice is not possible, we shall use the other notation to specify either rows or
columns. Actually, vectors tend to be given lower-case names, and a vector of unspecified
length n is more likely to be written
  
v1 x1
 ·   · 
u = (u1 , . . . , un ) or v=
 · 
 or  .
 · 
vn xn

Row and column vectors are not merely special cases of matrices. Any matrix can be re-
garded as an ordered list of both row vectors and column vectors. Given a matrix A ∈ Rm,n ,
we shall denote its rows (thought of as matrices in their own right) by

r1 , . . . , rm ∈ R1,n

and its columns

c1 , . . . , cn ∈ Rm,1
More informally (ignoring parentheses in a way that would be spell disaster in a computer
program), we may write
   
← r1 → ↑ · · ↑
 ···  = A =  c1 · · cn  .
← rm → ↓ · · ↓

Much of the study of matrices is ultimately be based on one or other of these two descrip-
tions.

L5.3 Addition of matrices. A matrix is much more than an array of data. It is an algebraic
object that is subject to operations generalizing the more familar ones applicable to numbers
and vectors.
Definition. To form the sum of two matrices A, B , they must have the same size. The
entries are then added component-wise.

For example
     
1 2 3 0 −2 4 1 0 7
A= , B= ⇒ A+B =
0 6 9 2 −6 1 2 0 10

Of course, the result is still a matrix of size 2×3.

In particular, if we add a matrix to itself, we merely double every entry and it is reasonable
to call the result 2A:  
2 4 6
A+A= = 2A
0 12 18

Definition. If c ∈ R and A ∈ Rm,n then cA is the matrix formed by multiplying every entry
of A by c.

If c is zero, we get a null matrix

 
0 0 0
0A = = 0.
0 0 0

A null matrix is denoted by 0 or 0 (or even 0 like the number), provided the context makes
clear its size. Of course, these definitions apply equally to vectors, so for example

2(x, y, z) = (2x, 2y, 2z).

We denote (−1)B by −B , so that matrices can be subtracted in the obvious way:

 
1 4 −1
A−B = = A + (−B).
−2 12 8

Exercise. Explain why A + B = B + A and (A+B)> = A> + B> .

L5.4 Matrix multiplication. First we define a numerical product between two vectors u, v
of the same length. For this it does not really matter whether they are row or column vectors,
but for egalitarian purposes we shall suppose that the first is a row vector and the second a
column vector. Thus, we consider
 
v1
 · 
u = (u1 , . . . , un ) ∈ R1,n , n,1
 ·  ∈R .
v= 

vn

Definition. The dot or scalar product of u and v , written u · v is the number

n
X
ui vi = u1 v1 + · · · + un vn .
i=1

(We shall not use the summation symbol much in this course, but students should be famil-
iar with its use.) The dot product provides the basis for multiplying matrices:
Definition. The product of two matrices A, B is only defined if the number of columns of A
equals the number of rows of B . If A ∈ Rm,n has rows r1 , . . . rm and B ∈ Rn,p has columns
c1 , . . . , cp then AB is the matrix with entries ri · cj and has size m×p.

More explicitly,
    
← r1 → ↑ · · ↑ r1 · c1 · · r1 · cp
AB =  ···   c1 · · cp  =  · · · ··· .
← rm → ↓ · · ↓ rm · c1 · · rm · cp

One should imagine taking each row of A, rotating it and placing it on top of each column
of B in turn so as to perform the dot product.
Example. A very special case is the product r1 c1 = (r1 · c1 ) of a single row and a column.
Strictly speaking, this is a 1×1 matrix, but (again ignoring parentheses) we shall regard it as a
number, i.e. the dot product. With this convention, if v = (x, y, z) then
 
x
vv> = (x, y, z)  y  = x2 + y 2 + z 2 .
z

Later, we shall refer to the square root of this quantity as the norm of the vector v (it is the
distance from the corresponding point to the origin). By contrast, note that
 2 
x xy xz
>
v v =  yx y 2 yz 
zx zy z 2

is a 3×3 matrix.

An intermediate case of the matrix product is that in which the second factor is a single
column v = (x1 , . . . , xn )> ∈ Rn,1 , so that
 
r1 · v
Av =  · .
rm · v

In general we can say that  

↑ · · ↑
AB =  Ac1 · · Acp  .
↓ · · ↓
This shows clearly that each column of the product is obtained by premultiplying the corre-
sponding column of B .
The rule for manipulating the sizes can be remembered by the scheme

m×n n×p m × p,
| {z }

and matrix multiplication defines a mapping

Rm,n × Rn,p −→ Rm,p .

Even if AB is defined, it will often be the case that BA is not. The situation is much more
symmetrical if m = n, and we investigate this next.