Dr. W. A. Abd Al-wahid M.Sc.

stage Lecture 1

A matrix is a rectangular array of elements (scalars) from a field. The size of a matrix
is specified by the number of rows (m) and the number of columns (n) and the element
in the ith row and jth column is often denoted by aij

a11 a12 ……. a1n

a21 a22 ……. a2n
a31 a32 ……. a3n
……. ……. ……. …….
am1 am2 ……. amn

The matrix is square if the number of rows and columns are equal (i.e. m = n) and the
elements aij of a square matrix are called the main diagonal
1 0 0
Unit Matrix (Identity Matrix)= [0 1 0]
0 0 1
Elementary operations with matrices
1- Equality:- Two (m X n) matrices and A and B are said to be equal if: aij = bij
𝑥 − 2𝑦 0 3 0
[ ]=[ ]
−2 6 −2 𝑥 + 𝑦
x-2y=3 …….(1)
x+y=6 …….(2) *(-1)
x-2y=3
-x-y=-6 →-3y=-3 →y=1→ sub in any of above equations
x-2(1) =3 → x=5

1
Engineering Technical College/Najaf
Dr. W. A. Abd Al-wahid M.Sc. stage Lecture 1

2- Addition: - The sum of two matrices of like dimensions is the matrix of the sum of the
corresponding elements. Thus:
1) A+B = B+A
2) A+ (B+C) = (A+B) +C
3) A-(B-C) = A-B+C
Ex: Find (A+B) and (A-B) if
2 1 3 1 −2 2
A=[ ] B=[ ]
1 0 −2 2 3 −1
2 + 1 1 + (−2) 3+2 3 −1 5
Sol: A+B=[ ]=[ ]
1+2 0+3 −2 + (−1) 3 3 −3
2 − 1 1 − (−2) 3− 2 1 3 1
A-B=[ ]=[ ]
1−2 0− 3 −2 − (−1) −1 −3 −1
3. Multiplication by a scalar: - The matrix A is multiplied by the scalar C by multiplying
each element of A by C
1 −2
Ex: Assume A=[2 2 ] find 3A
3 −3
3∗1 3 ∗ (−2) 3 −6
Sol: 3*A =[3 ∗ 2 3 ∗ 2 ] =[6 6 ]
3∗3 3 ∗ (−3) 9 −9
4. Matrix multiplication: - For the matrix product AB to be defined, it is necessary that
the number of columns of A be equal to the number of rows of B.
6 5 4
1 2 3
Ex: Assume A=[ ] , B= [−1 1 −1] Find AB and BA
−1 0 1
0 2 0
Sol:
AB=
(1 ∗ 6) + (2 ∗ (−1)) + (3 ∗ 0) (1 ∗ 5) + (2 ∗ 1) + (3 ∗ 2) (1 ∗ 4) + (2 ∗ (−1) ) + (3 ∗ 0)
[ ]
(−1 ∗ 6) + (0 ∗ (−1)) + (1 ∗ 0) (−1 ∗ 5) + (0 ∗ 1) + (1 ∗ 2) (−1 ∗ 4) + (0 ∗ (−1)) + (1 ∗ 0)

2
Engineering Technical College/Najaf
Dr. W. A. Abd Al-wahid M.Sc. stage Lecture 1

4 13 2
AB=[ ]
−6 −3 −4
Note that we can’t achieve BA because the number of element in each row in B is more
than that in each column in A
Note that AB≠BA
1 2 3 −1
Ex: Assume A=[ ] , B=[ ]
0 3 2 1
Sol:

1 2 3 −1 (1 ∗ 3 ) + (2 ∗ 2 ) 1 ∗ (−1) + 2 ∗ 1 7 1
AB=[ ]∗[ ]= [ ] =[ ]
0 3 2 1 (0 ∗ 3 ) + (3 ∗ 2 ) 0 ∗ (−1) + 3 ∗ 1 6 3

3 −1 1 2 (3 ∗ 1) + (−1 ∗ 0) (3 ∗ 2) + (−1) ∗ 3 3 3
BA=[ ]∗[ ]= [ ]=[ ]
2 1 0 3 (2 ∗ 1 ) + (1 ∗ 0 ) 2∗2+1∗ 3 2 7
1 2
Ex: Assume A=[ ], find A2
4 3
1 2 1 2 9 8
Sol:A2=[ ][ ]=[ ]
4 3 4 3 16 17

5. Transpose of matrix:- Let A is any (mXn) matrix, the transpose of A is (nXm)

matrix(Aꞌ ) formed by interchanging the role of rows and columns.

2 −3 1
Ex: Assume A=[5 −2 −1] find (Aꞌ)
0 −4 −3
2 5 0
Sol: Aꞌ =[−3 −2 −4]
1 −1 −3
5 2
Ex: Assume A=[−2] , B=[−1] find AꞌB, ABꞌ, AB and AꞌBꞌ
1 3
Sol: Aꞌ=[5 −2 1] and Bꞌ=[2 −1 3]
3
Engineering Technical College/Najaf
Dr. W. A. Abd Al-wahid M.Sc. stage Lecture 1

2
AꞌB=[5 −2 1] [−1] = (5*2) + (-2*(-1)) + (1*3) =15
3
5 5∗2 5 ∗ (−1) 5∗3 10 −5 15
ABꞌ=[−2] [2 −1 3]=[−2 ∗ 2 −2 ∗ (−1) −2 ∗ 3]=[−4 2 −6]
1 1∗2 1 ∗ (−1) 1∗3 2 −1 3
Note that we can’t achieve BA and AꞌBꞌ
Determinants
The minor of the element aij in a matrix A is the determinant of the matrix that remains
when the row and column containing aij are deleted. For example, let:-
𝑎11 𝑎12 𝑎13
𝑎12 𝑎13
A=[𝑎21 𝑎22 𝑎23]then the minor of a21 is =[ ]
𝑎32 𝑎33
𝑎31 𝑎32 𝑎33
𝑎11 𝑎12 𝑎13 𝑎14
𝑎11 𝑎13 𝑎14
𝑎21 𝑎22 𝑎23 𝑎24
B=[ ]then the minor of a32 is =[𝑎21 𝑎23 𝑎24]
𝑎31 𝑎32 𝑎33 𝑎34
𝑎41 𝑎43 𝑎44
𝑎41 𝑎42 𝑎43 𝑎44
The cofactor of aij is the determinant Aij that is (−1)𝑖+𝑗 times the minor of aij. Thus:-
𝟐+𝟏 𝑎12 𝑎13 𝑎12 𝑎13
For matrix (3X3) →A21= (-1) =| |= -| |
𝑎32 𝑎33 𝑎32 𝑎33
𝟐+𝟑
𝑎11 𝑎13 𝑎14 𝑎11 𝑎13 𝑎14
For matrix (4X4) →A32= (-1) =[𝑎21 𝑎23 𝑎24]= -|𝑎21 𝑎23 𝑎24|
𝑎41 𝑎43 𝑎44 𝑎41 𝑎43 𝑎44
With each square matrix A we associate a number det A or |𝑨| called the determinant
of A, calculated from the entries of A in the following way:-
I. for n = 1, A= [a] → |𝐴|=a
𝑎11 𝑎12
II. For n = 2, A=[ ] → |𝐴|= (a11*a22)-(a12*a21)
𝑎21 𝑎22
2 2 −2 0 4 −2
Ex: Find the determinant of the matrices: A=[ ], B=[ ], C=[ ]
−1 3 1 1 2 −1
2 2
Sol: A =| |= (2*3)-(-1*2) =6-(-2) =8
−1 3
4
Engineering Technical College/Najaf
Dr. W. A. Abd Al-wahid M.Sc. stage Lecture 1

−2 0
B =| |= (-2*1)-(0*1) = -2
1 1
4 −2
C =| |= (-1*4)-(-2*2) =-4-(-4) =-4+4=0
2 −1

Solving system of linear simultaneous equations by matrices

1. Gaussian Elimination:

5
Engineering Technical College/Najaf
Dr. W. A. Abd Al-wahid M.Sc. stage Lecture 1

6
Engineering Technical College/Najaf
Dr. W. A. Abd Al-wahid M.Sc. stage Lecture 1

2. Gauss-Jordan elimination method:

7
Engineering Technical College/Najaf
Dr. W. A. Abd Al-wahid M.Sc. stage Lecture 1

8
Engineering Technical College/Najaf
Dr. W. A. Abd Al-wahid M.Sc. stage Lecture 1

9
Engineering Technical College/Najaf
Dr. W. A. Abd Al-wahid M.Sc. stage Lecture 1

10
Engineering Technical College/Najaf
Dr. W. A. Abd Al-wahid M.Sc. stage Lecture 1

3. Solution by the Gauss and Gauss-Seidel Iteration Methods:

11
Engineering Technical College/Najaf
Dr. W. A. Abd Al-wahid M.Sc. stage Lecture 1

12
Engineering Technical College/Najaf
Dr. W. A. Abd Al-wahid M.Sc. stage Lecture 1

H.W.:

LU decomposition
Suppose we have the system of equations
AX = B.
The motivation for an LU decomposition based on the observation that systems of
equations involving triangular coefficient matrices are easier to deal with. Indeed, the
whole point of Gaussian elimination is to replace the coefficient matrix with one that is
triangular. The LU decomposition is another approach designed to exploit triangular
systems.
We suppose that we can write
A = LU
Where L is a lower triangular matrix and U is an upper triangular matrix. Our aim is to
find L and U and once we have done. Therefore, we found an LU decomposition of A.
It turns out that we need only consider lower triangular matrices L that have 1st down
the diagonal.
LU Decomposition of 2x2 matrix
Find an LU decomposition of
3 1 𝐿 0 𝑈11 𝑈12
A=[ ]= LU=[ 11 ][ ]
−6 −4 𝐿 21 𝐿 22 0 𝑈22

13
Engineering Technical College/Najaf
Dr. W. A. Abd Al-wahid M.Sc. stage Lecture 1

Suppose that L 11=L22 =1

𝑈 𝑈12 3 1
[ 11 ]=[ ]
𝐿 21𝑈11 𝐿 21 𝑈12 + 𝑈22 −6 −4
Then, comparing the left and right hand sides row by row implies that
U11 = 3, U12 = 1, L21U11 = −6 ∴ L21= -2 , L21U12 + U22 = −4 ∴ U22=-2
3 1 1 0 3 1
Hence LU decomposition of [ ]=[ ][ ]
−6 −4 −2 1 0 −2
LU Decomposition for 3x3 matrix
Here is an example. Let
1 2 4 𝐿11 0 0 𝑈11 𝑈12 𝑈13
A= [3 8 14]=LU, Where L=[𝐿 21 𝐿 22 0 ] and U=[ 0 𝑈22 𝑈23 ]
2 6 13 𝐿 31 𝐿 32 𝐿 33 0 0 𝑈33
Suppose that L 11=L22=L33=1
Multiplying out LU and setting the answer equal to A gives
𝑈11 𝑈12 𝑈13 1 2 4
[𝐿 21 𝑈11 𝐿 21 𝑈12 + 𝑈22 𝐿 21 𝑈13 + 𝑈23 ]=[3 8 14]
𝐿 31 𝑈11 𝐿 31 𝑈12 + 𝐿 32 𝑈22 𝐿 31𝑈13 + 𝐿 32𝑈23 + 𝑈33 2 6 13
Now, we use this to find the entries in L and U. Fortunately this is not nearly as hard as
it might at first seem. We begin by running along the top row to see that
U11=1 , U12=2 , U13=4
Now consider the second row
L21 U11=3 → L21*1=3 ∴ L21=3
L21 U12+ U22=8 →3*2+ U22=8 ∴ U22=2
L21U13 + U23 = 14 → 3 × 4 + U23 = 14 ∴ U23=2
Notice how, at each step, the equation being considered has only one unknown in it,
and other quantities that we have already found. This pattern continues on the last row
L31U11 = 2 → L31 × 1 = 2 ∴ L31=2
L31U12 + L32U22 = 6 → 2 × 2 + L32 × 2 = 6 ∴ L32=1
L31U13 + L32U23 + U33 = 13 → (2 × 4) + (1 × 2) + U33 = 13 ∴ U33=3
14
Engineering Technical College/Najaf
Dr. W. A. Abd Al-wahid M.Sc. stage Lecture 1

We have shown that

1 2 4 1 0 0 1 2 4
A= [3 8 14]= [3 1 0] [0 2 2]
2 6 13 2 1 1 0 0 3
In addition, this is an LU decomposition of A.

H.W
3 1 6
Find an LU decomposition of [−6 0 −16]
0 8 −17
Answer
3 1 6 1 0 0 3 1 6
[−6 0 −16]=[−2 1 0] [0 2 −4]
0 8 −17 0 4 1 0 0 −1
Using LU decomposition to solve systems of equations
Once a matrix A has been decomposed into lower and upper triangular parts it is possible
to obtain the solution to AX = B in a direct way. The procedure canbe summarized as
follows
1- Given A, find L and U so that A = LU. Hence LUX = B.
2- Let Y = UX so that LY = B. Solve this triangular system for Y.
3- Finally solve the triangular system UX = Y for X.
The benefit of this approach is that we only ever need to solve triangular systems. The
cost is that we have to solve two of them.
Example
𝑥1 1 2 4 𝑥1 3
𝑥 𝑥
Find the solution of X=[ 2 ]of the system [3 8 14][ 2 ]=[13]
𝑥3 2 6 13 𝑥3 4
Sol:
The first step is to calculate the LU decomposition of the coefficient matrix on the left-
hand side. In this case, that job has already been done.

15
Engineering Technical College/Najaf
Dr. W. A. Abd Al-wahid M.Sc. stage Lecture 1

since this is the matrix we considered earlier. We found that

1 0 0 1 2 4
L=[3 1 0], U=[0 2 2]
2 1 1 0 0 3
𝑦1
The next step is to solve LY = B for the vector Y =[𝑦2 ].That is we consider
𝑦3
1 0 0 𝑦1 3
LY= [3 1 0][𝑦2 ] = [13] = 𝐵
2 1 1 𝑦3 4
Which solved by forward substitution. From the top equation we can see that y1 =
3. The middle equation states that 3y1 + y2 = 13 and hence y2= 4. Finally the bottom
line says that 2y1 + y2 + y3 = 4 from which we see that y3 = −6.
Finally, that we have found Y we finish the procedure by solving UX = Y for X.
That is we solve
1 2 4 𝑥1 3
𝑥
UX=[0 2 2] [ 2 ] = [ 4 ]=Y
0 0 3 𝑥3 −6
By using back substitution. Starting with the bottom equation we see that 3x3 = −6 so
clearly x3 = −2. The middle equation implies that 2x2 + 2x 3 = 4 and it follows that x2 =
4.The top equation states that x1 + 2x 2 + 4x 3 = 3 and consequently x1 = 3
Therefore we have found that the solution to the system of simultaneous equations.
1 2 4 𝑥1 3 3
[3 8 14] [𝑥2 ]=[13]is X=[ 4 ]
2 6 13 𝑥3 4 −2
H.W
𝑥1 3 1 6 𝑥1 0
Find the solution of X=[𝑥2 ]of the system [−6 0 −16][𝑥2 ] = [ 4 ]
𝑥3 0 8 −17 𝑥3 17
Answer
0 2
Y = [4] , x = [ 0 ]
1 −1

16
Engineering Technical College/Najaf
Dr. W. A. Abd Al-wahid M.Sc. stage Lecture 1

Do matrices always have an LU decomposition?

No, sometimes it is impossible to write a matrix in the form “LU decomposition”
Why not?
An invertible matrix A has an LU decomposition provided that all its leading
submatrices have non-zero determinants. The kth leading submatrix of A is denoted Ak
and is the k ×k matrix found by looking only at the top k rows and leftmost k columns.
For example if
1 2 4
A=[3 8 14]
2 6 13
Then the leading submatrices are
1 2 4
1 2
A1=1 , A2=[ ] , A3=[3 8 14]
3 8
2 6 13
A1=1
A2=2
A3=6
(Where the 3 × 3 determinant wasfound by expanding along the top row)
Example
1 2 3
Show that [2 4 5] does not have an LU decomposition.
1 3 4
Sol: The second leading submatrix has determinant equal to
1 2
A2=[ ]= (1*4)-(2*2)=0
2 4
Which means that an LU decomposition is not possible in this case.
Can we get around this problem?
Yes. It is always possible to re-order the rows of an invertible matrix so that all of the
submatrices have non-zero determinants
Example

17
Engineering Technical College/Najaf
Dr. W. A. Abd Al-wahid M.Sc. stage Lecture 1

1 2 3
Reorder the rows of A = [2 4 5]so that the reordered matrix has an LU
1 3 4
Sol:
Swapping the first and second rows does not help us since the second leading
submatrix will still have a zero determinant. Let us swap the second and third rows and
consider
1 2 3 𝟏 𝟐 𝟑
𝟏 𝟐
B =[1 3 4], B1=1, B2=[ ] = 𝟏, B3=[𝟏 𝟑 𝟒 ] = −𝟏
𝟏 𝟑
2 4 5 𝟐 𝟒 𝟓
All three of these determinants are non-zero and we conclude that B does have an LU
decomposition.
Example
1 −3 7
Reorder the rows of A =[−2 6 1 ], so that the reordered matrix has an LU
0 3 −2
decomposition.
Sol:
Let us swap the second and third rows and consider
1 −3 7
B =[ 0 3 −2]
−2 6 1
Which have determinants B1=1, B2=3 and B3=45 respectively. All of these are non-
zero and we conclude that B does indeed have an LU decomposition.
H.W
Solve the following, use LU method
x1+6x2+2x3=9
2x1+12x2+5x3=-4
-x1-3x2-x3=17

18
Engineering Technical College/Najaf