Type author name/s here

Dougherty

Introduction to Econometrics,
5th edition
Chapter heading
Chapter 2: Properties of the
Regression Coefficients and
Hypothesis Testing

© Christopher Dougherty, 2016. All rights reserved.

 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

TYPES OF DATA

Cross-sectional: Observations on individuals, households, enterprises,

countries, etc at one moment in time (Chapters 1–10,
Models A and B).

Time series: Observations on income, consumption, interest rates,

etc over a number of time periods (years, quarters,
months, …) (Chapters 11–13, Model C).

Panel data: Observations on the same cross-section of individuals,

households, etc over a number of time periods (Chapter
14, Model B).

During this course we will work with the three types of data described above.

1
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

TYPES OF MODEL

Model A: Cross-sectional data with nonstochastic regressors. Their

values in the observations in a sample do not have stochastic
(random) components.

Model B: Cross-sectional data with stochastic regressors. The values

of the regressors are drawn randomly and independently from
defined populations.

Model C: Time series data. The values of the regressors may exhibit
persistence over time. Regressions with time series data
potentially involve complex technical issues that are best
avoided initially.

Different regression models are appropriate for different types of data. We will consider
three types of regression model, as shown above.

2
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

TYPES OF MODEL

Model A: Cross-sectional data with nonstochastic regressors. Their

values in the observations in a sample do not have stochastic
(random) components.

Model B: Cross-sectional data with stochastic regressors. The values

of the regressors are drawn randomly and independently from
defined populations.

Model C: Time series data. The values of the regressors may exhibit
persistence over time. Regressions with time series data
potentially involve complex technical issues that are best
avoided initially.

We will start with Model A. We will do this purely for analytical convenience. It enables us
to conduct the discussion of regression analysis within the relatively straightforward
framework of what is known as the Classical Linear Regression Model.
3
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

TYPES OF MODEL

Model A: Cross-sectional data with nonstochastic regressors. Their

values in the observations in a sample do not have stochastic
(random) components.

Model B: Cross-sectional data with stochastic regressors. The values

of the regressors are drawn randomly and independently from
defined populations.

Model C: Time series data. The values of the regressors may exhibit
persistence over time. Regressions with time series data
potentially involve complex technical issues that are best
avoided initially.

We will replace it in Chapter 8 by the weaker and more realistic assumption, appropriate for
regressions with cross-sectional data, that the observations on the regressors are
randomly drawn from defined populations.
4
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.1 The model is linear in parameters and correctly specified.

For example:
Y  1   2 X  u

Examples of models that are not linear in parameters:

Y   1 X 2  u
Y  1   2 X 2   3 X 3   2 3 X 4  u

‘Linear in parameters’ means that each term on the right side includes a b as a simple
factor and there is no built-in relationship among the b parameters. We will defer a
discussion of issues relating to linearity and nonlinearity to Chapter 4.
5
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.2 There is some variation in the regressor in the sample.

There must be some variation in the regressor in the sample. Otherwise it cannot account
for any of the variation in Y.

6
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.2 There is some variation in the regressor in the sample.

ˆ2   X  X   Y Y 
i i

 X  X 
2
i

If Xi  X for all i,

ˆ 0
2 
0

If we tried to regress Y on X, when X is constant, we would find that we would not be able to
compute the regression coefficients. Both the numerator and the denominator of the
expression for ̂ 2 would be equal to zero. We would not be able to obtain ̂ 1 either.
7
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.3 The disturbance term has zero expectation

E  ui   0 for all i

We assume that the expected value of the disturbance term in any observation should be
zero. Sometimes the disturbance term will be positive, sometimes negative, but it should
not have a systematic tendency in either direction.
8
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.3 The disturbance term has zero expectation

E  ui   0 for all i

Actually, if an intercept is included in the regression equation, it is usually reasonable to

assume that this condition is satisfied automatically. The role of the intercept is to pick up
any systematic but constant tendency in Y not accounted for by the regressor(s).
9
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.3 The disturbance term has zero expectation

E  ui   0 for all i

Suppose Y  1   2 X  u E  ui    u  0

Suppose that the disturbance term had a nonzero population mean.

10
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.3 The disturbance term has zero expectation

E  ui   0 for all i

Suppose Y  1   2 X  u E  ui    u  0

Define v i  ui   u

Define a new random variable vi = ui – mu.

11
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.3 The disturbance term has zero expectation

E  ui   0 for all i

Suppose Y  1   2 X  u E  ui    u  0

Define v i  ui   u

Yi   1   2 X i  v i   u
  1*   2 X i  v i where  1   1   u
*

Then we can rewrite the model as shown. vi becomes the new disturbance term and the
intercept has absorbed the constant mu.

12
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS
ASSUMPTIONS FOR
FOR MODEL
MODEL A
A

A.3 The disturbance term has zero expectation

E  ui   0 for all i

Suppose Y  1   2 X  u E  ui    u  0

Define v i  ui   u

Yi   1   2 X i  v i   u
  1*   2 X i  v i where  1   1   u
*

Then E  v i   E  ui   u   E  ui   E   u    u   u  0

The disturbance term in the revised model now satisfies Assumption A.3.

13
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.3 The disturbance term has zero expectation

E  ui   0 for all i

Suppose Y  1   2 X  u E  ui    u  0

Define v i  ui   u

Yi   1   2 X i  v i   u
  1*   2 X i  v i where  1   1   u
*

Then E  v i   E  ui   u   E  ui   E   u    u   u  0

The price that we pay is that the interpretation of the intercept has changed. It has
absorbed the nonzero component of the disturbance term in addition to whatever had
previously been responsible for it.
14
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.3 The disturbance term has zero expectation

E  ui   0 for all i

Suppose Y  1   2 X  u E  ui    u  0

Define v i  ui   u

Yi   1   2 X i  v i   u
  1*   2 X i  v i where  1   1   u
*

Then E  v i   E  ui   u   E  ui   E   u    u   u  0

This is usually acceptable because the role of the constant is usually to pick up any
systematic tendency in Y not accounted for by the regressor(s).

15
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.4 The disturbance term is homoscedastic

 u2i   u2 for all i

We assume that the disturbance term is homoscedastic, meaning that its value in each
observation is drawn from a distribution with constant population variance.

16
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.4 The disturbance term is homoscedastic

 u2i   u2 for all i

In the language of the section on sampling and estimators in the Review chapter, this is a
‘beforehand’ concept, where we are thinking about the potential behavior of the disturbance
term before the sample is actually generated.
17
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.4 The disturbance term is homoscedastic

 u2i   u2 for all i

Once we have generated the sample, the disturbance term will turn out to be greater in
some observations, and smaller in others, but there should not be any reason for it to be
more erratic in some observations than in others.
18
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.4 The disturbance term is homoscedastic

 u2i   u2 for all i


 u2i  E  ui   u 
2
  E  ui2 

E  ui2    u2 for all i

Since E(ui) = 0, by Assumption A.3, the population variance of ui is equal to E(ui2), so the
condition can also be written as shown.

19
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.4 The disturbance term is homoscedastic

 u2i   u2 for all i


 u2i  E  ui   u 
2
  E  ui2 

E  ui2    u2 for all i

If Assumption A.4 is not satisfied, the OLS regression coefficients will be inefficient, and
you should be able to obtain more reliable results by using a modification of the regression
technique. This will be discussed in Chapter 7.
20
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.5 The values of the disturbance term have independent distributions

ui is distributed independently of uj for all j ≠ i

We assume that the disturbance term is not subject to autocorrelation, meaning that there
should be no systematic association between its values in any two observations.

21
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.5 The values of the disturbance term have independent distributions

ui is distributed independently of uj for all j ≠ i

For example, just because the disturbance term is large and positive in one observation,
there should be no tendency for it to be large and positive in the next (or large and
negative, for that matter, or small and positive, or small and negative).
22
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.5 The values of the disturbance term have independent distributions

ui is distributed independently of uj for all j ≠ i

 ui u j  E   ui   u   u j   u    E  ui u j 
 E  ui  E  u j   0

The assumption implies that the population covariance between ui and uj is zero. Note that
the population means of ui and uj are both zero, by virtue of Assumption A.3, and that E(uiuj)
can be decomposed as E(ui)E(uj) if ui and uj are generated independently – see the Review.
23
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.5 The values of the disturbance term have independent distributions

ui is distributed independently of uj for all j ≠ i

 ui u j  E   ui   u   u j   u    E  ui u j 
 E  ui  E  u j   0

If this assumption is not satisfied, OLS will again give inefficient estimates. Chapter 12
discusses the problems that arise and ways of getting around them. Violations of this
assumption are in any case rare with cross-sectional data.
24
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.6 The disturbance term has a normal distribution

We usually assume that the disturbance term has a normal distribution. The justification for
the assumption depends on the Lindeberg‒Feller Central Limit theorem.

25
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.6 The disturbance term has a normal distribution

In essence, this CLT states that, if a random variable is the composite result of the effects
of a large number of other random variables, it will have an approximately normal
distribution even if its components do not, provided that none of them is dominant.
26
 TYPES OF REGRESSION MODEL AND ASSUMPTIONS FOR MODEL A

ASSUMPTIONS FOR MODEL A

A.6 The disturbance term has a normal distribution

The disturbance term u is composed of a number of factors not appearing explicitly in the
regression equation so, even if we know nothing about the distribution of these factors, we
are usually entitled to assume that the disturbance term is normally distributed.
27
Copyright Christopher Dougherty 2016.

These slideshows may be downloaded by anyone, anywhere for personal use.

Subject to respect for copyright and, where appropriate, attribution, they may be
used as a resource for teaching an econometrics course. There is no need to
refer to the author.

The content of this slideshow comes from Section 2.2 of C. Dougherty,

Introduction to Econometrics, fifth edition 2016, Oxford University Press.
Additional (free) resources for both students and instructors may be
downloaded from the OUP Online Resource Centre
http://www.oxfordtextbooks.co.uk/orc/dougherty5e/

Individuals studying econometrics on their own who feel that they might benefit
from participation in a formal course should consider the London School of
Economics summer school course
EC212 Introduction to Econometrics
http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx
or the University of London International Programmes distance learning course
EC2020 Elements of Econometrics
www.londoninternational.ac.uk/lse.

2016.04.18