Module 6

POLYTECHNIC UNIVERSITY OF THE PHILIPPINES
COLLEGE OF ENGINEERING
DEPARTMENT OF INDUSTRIAL ENGINEERING
MODULE 6
MULTIPLE LINEAR REGRESSION
In Industrial Engineering, many applications of regression

analysis involve situations wherein there is more than one
regressor. A regression model that contains more than one
regressor is called a multiple regression model.
I. The Multiple Linear Regression Model
Suppose that the effective life of a cutting tool (Y ) depends on

the cutting speed ( x1 ) and the tool angle ( x 2 ) . A multiple
regression model that might describe this relationship is:
Y = 0 + 1 x1 + 2 x 2 +
Specifically, the model stated above is a multiple linear

regression model with two regressors because it is a linear
function of the unknown parameters 0 , 1 , 2 .
Models that are more complex in structure may often still be

analyzed by multiple linear regression models techniques.
Consider the polynomial model in one regressor variable:
Y = 0 + 1 x + 2 x 2 + 3 x 3 +
If we let x1 = x, x 2 = x , x3 = x :
2 3
Y = 0 + 1 x + 2 x2 + 3 x3 +
Models that include interaction effects may also be analyzed by

multiple linear regression methods.
Y = 0 + 1 x + 2 x2 + 12 x1 x2 +
If we let x3 = x1 x 2 and 3 = 12 , we will have:
Y = 0 + 1 x + 2 x2 + 3 x3 +
II. Least Squares Estimation of the Parameters
Similar to the simple linear regression model, the least squares

estimation methodology may be used to estimate the regression
parameters.
Using the same derivation method in simple linear regression

and letting x ij denote the ith observation of variable x j , we
have the following least square normal equations below:
n n n n
n0 + 1 xi1 + 2 xi 2 + + k xik = yi
i =1 i =1 i =1 i =1
n n n n n
0 xi1 + 1 xi1 xi1 + 2 xi1 xi 2 + + k xi1 xik = xi1 y i
i =1 i =1 i =1 i =1 i =1
n n n n n
0 xik + 1 xik xi1 + 2 xik xi 2 + + k xik xik = xik y i
i =1 i =1 i =1 i =1 i =1
EXAMPLE:
You are given the wire bond data in the succeeding table. Fit a
multiple linear regression model for the given data.
Observation Pull Strength Wire Length Die Height

Number (y) (x1) (x2)
1 9.95 2 50
2 24.45 8 110
3 31.75 11 120
4 35.00 10 550
5 25.02 8 295
6 16.86 4 200
7 14.38 2 375
8 9.60 2 52
9 24.35 9 100
10 27.50 8 300
11 17.08 4 412
12 37.00 11 400
13 41.95 12 500
14 11.66 2 360
15 21.65 4 205
16 17.89 4 400
17 69.00 20 600
18 10.30 1 585
19 34.93 10 540
20 46.59 15 250
21 44.88 15 290
22 54.12 16 510
23 56.63 17 590
24 22.13 6 100
25 21.15 5 400
Given the data above, we arrive at the following values:
25 25 25
n = 25 yi = 725.82 xi1 = 206 xi 2 = 8,294
i =1 i =1 i =1
25 25 25
xi1 xi1 = 2,396 xi 2 xi 2 = 3,531,848 xi1 xi 2 = 77,177
i =1 i =1 i =1
25 25
xi1 yi = 8,008.37 xi 2 yi = 274,811.31
i =1 i =1
For this particular case, the normal equations are:

25 25 25
n0 + 1 xi1 + 2 xi 2 = yi
i =1 i =1 i =1
25 25 25 25
0 xi1 + 1 xi1 xi1 + 2 xi1 xi 2 = xi1 yi
i =1 i =1 i =1 i =1
25 25 25 25
0 xik + 1 xik xi1 + 2 xik xi 2 = xi 2 yi
i =1 i =1 i =1 i =1
Inserting the computed values into the normal equation, we

have:
250 + 2061 + 8,2942 = 725.82

206 + 2,396 + 77,177 = 8,008.37
0 1 2
8,2940 + 77,177 1 + 3,531,8482 = 274,811.31
We have three unknowns 0 , 1 , 2 but we have three equations.

Using the system of equations methodology, the values of the
three unknowns are as follows:
0 = 2.264, 1 = 2.744, 2 = 0.013
Therefore, the fitted (multiple linear) regression equation is:
y = 2.264 + 2.744 x1 + 0.013x2
The equation above can now be used to predict pull strength for
pairs of values of the regressor variables wire length and die
height.
III. Matrix Approach to Multiple Linear Regression
In fitting a multiple regression model, it is much more

convenient to express the mathematical operations using a
matrix notation. Suppose that there are k regressor variables and
n observations, and that the model relating the regressors to the
response variable y is:
y i = 0 + 1 xi1 + 2 xi 2 + + k xik + i i = 1, 2, ,n
This model is a system of n equations that can be expressed in

matrix notation as
y = X +
where
y1 1 x11 x12 x1k 0 1

y2 1 x 21 x 22 x2k 1 2
y= X= = =
yn 1 x n1 xn2 x nk k n
To solve for the values of , we use the matrix equation:
= ( X ' X )1 X ' y
EXAMPLE # 1: Let us use the matrix approach in coming up

with the fitted regression model using the data in the previous
example.
1 2 50 9.95
1 8 110 24.45
1 11 120 31.75
1 10 550 35.00
1 8 295 25.02
1 4 200 16.86
1 2 375 14.38
1 2 52 9.60
1 9 100 24.35
1 8 300 27.50
1 4 412 17.08
1 11 400 37.00
X = 1 12 500 y = 41.95
1 2 360 11.66
1 4 205 21.65
1 4 400 17.89
1 20 600 69.00
1 1 585 10.30
1 10 540 34.93
1 15 250 46.59
1 15 290 44.88
1 16 510 54.12
1 17 590 56.63
1 6 100 22.13
1 5 400 21.15
The XX matrix is (please check!):

25 206 8,294
X ' X = 206 2,396 77,177
8,294 77,177 3,531,848
and the Xy vector (please check!):
725.82
X'y = 8,008.37
274,811.31
The least squares estimates are found by:
= ( X ' X )1 X ' y
0 2.26379143
1 = 2.74426964
2 0.01252781
Therefore, the fitted regression model is:
y = 2.264 + 2.744 x1 + 0.013x2
which is the same as what we got previously.
Estimating
2
Just as in simple linear regression, it is important to estimate

2
in a multiple regression model. In simple linear regression, the

equation was
SS E
2 =
n2
Remember that the reason why we subtracted 2 from n is
because we estimated two parameters in our regression model,
0 and 1 . However, in multiple linear regression, we estimate
more than two parameters in our regression model. We adjust
the previous equation to take this fact into consideration. Thus, a
more general equation for estimating the variance of the error
term is
n
ei2
SS E
2 = i =1
=
n p n p
where p is the number of parameters, i.e. regressors, including

the intercept, estimated.
EXAMPLE # 2: The electric power consumed each month by a

chemical plant is thought to be related to the average ambient
temperature (x1 ) , the number of days in the month (x2 ) , the
average product purity (x3 ) , and the tons of product produced
(x4 ) . The past years historical data are available and are
presented in the Excel file. Use the Data Analysis function in
Excel to come up with a regression analysis of the problem.
IV. Hypothesis Tests in Multiple Linear Regression
In multiple linear regression problems, there are certain tests of

hypotheses about the model parameters that are useful in
measuring model adequacy. Similar to the simple linear
regression model, a major assumption for the hypothesis tests is
that the error terms are NID(0, 2).

Module 6

Uploaded by

Copyright:

Available Formats

Module 6

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Module 6

Uploaded by

Copyright:

Available Formats

POLYTECHNIC UNIVERSITY OF THE PHILIPPINES

In Industrial Engineering, many applications of regression

I. The Multiple Linear Regression Model

Suppose that the effective life of a cutting tool (Y ) depends on

Specifically, the model stated above is a multiple linear

Models that are more complex in structure may often still be

Models that include interaction effects may also be analyzed by

If we let x3 = x1 x 2 and 3 = 12 , we will have:

II. Least Squares Estimation of the Parameters

Similar to the simple linear regression model, the least squares

Using the same derivation method in simple linear regression

Observation Pull Strength Wire Length Die Height

Given the data above, we arrive at the following values:

For this particular case, the normal equations are:

Inserting the computed values into the normal equation, we

250 + 2061 + 8,2942 = 725.82

8,2940 + 77,177 1 + 3,531,8482 = 274,811.31

We have three unknowns 0 , 1 , 2 but we have three equations.

0 = 2.264, 1 = 2.744, 2 = 0.013

Therefore, the fitted (multiple linear) regression equation is:

y = 2.264 + 2.744 x1 + 0.013x2

III. Matrix Approach to Multiple Linear Regression

In fitting a multiple regression model, it is much more

This model is a system of n equations that can be expressed in

y1 1 x11 x12 x1k 0 1

To solve for the values of , we use the matrix equation:

EXAMPLE # 1: Let us use the matrix approach in coming up

The XX matrix is (please check!):

and the Xy vector (please check!):

The least squares estimates are found by:

Therefore, the fitted regression model is:

y = 2.264 + 2.744 x1 + 0.013x2

which is the same as what we got previously.

Just as in simple linear regression, it is important to estimate

in a multiple regression model. In simple linear regression, the

where p is the number of parameters, i.e. regressors, including

EXAMPLE # 2: The electric power consumed each month by a

IV. Hypothesis Tests in Multiple Linear Regression

In multiple linear regression problems, there are certain tests of

You might also like