Simple Regression: OLS,

Interpretation, ANOVA, and

R-squared

Lecture 5

Reading: Chapter 7, 19.4

Why Are Indian Children So Short?

• “One in four children under age five, worldwide, is so
short as to be stunted (UNICEF 2014). Child
stunting—a key marker of child malnutrition—[can
mean adults that] are less healthy, have lower
cognitive ability, and earn less.”
• “[India’s] child stunting rate is over 40 percent, an
outlier even among poor countries (IIPS 2010).”
• “[We use] survey data on over 168,000 children from
India and 25 African countries.”
“Why Are Indian Children So Short? The Role of Birth Order and Son Preference,”
Jayachandran and Pande (2017), https://doi.org/10.1257/aer.20151282, pp. 2600-01.
2

Lecture 5 Slides, ECO220Y1Y, 1

 Child Height versus National GDP
• “Figure 1 graphs average
child height-for-age for
sub-Saharan African
countries [blue] and
Indian states [red] against
income.”
• “Both regions exhibit a
positive correlation between income and child height, but the
curve for India [red] is lower; at a given level of income, Indian
children are shorter.”
• “Given that India performs better than African countries on
most health and development indicators, this contrast is
striking and is the focus of this paper.”
3

Figure 1. Child Height versus National GDP

Notes: The averages are over all children less than 60 months old. National
GDP data are from the Penn World Table 9.0 (Feenstra, Inklaar, and Timmer
2015). The lines are the best linear fit for each sample. 4

Lecture 5 Slides, ECO220Y1Y, 2

 HFA 𝑧-scores [Excerpts, pp. 2604-05]
“[We] create the child’s height-for-age (HFA) 𝑧-score based on
the World Health Organization (WHO) universally applicable
growth standard for children aged zero to five years.10
A z-score of 0 represents the median of the gender- and age-
specific reference population, and a 𝑧-score of -2 indicates that
the child is two standard deviations below that reference-
population median, which is the cutoff for being stunted.
Our primary outcome of interest is the HFA 𝑧-score because it is
the child health measure that has been most often linked to
later-life outcomes and is viewed as the best cumulative
measure of child malnutrition.”

10 The WHO standard describes how children should grow if they receive proper nutrition and health
care. It is premised on the fact that the height distribution among children under age five who receive
adequate nutrition and health care has been shown to be similar in most ethnic groups (de Onis et al.
2006; WHO Multicentre Growth Reference Study Group 2006a).
5

“The Family as a Social Institution”

Jayachandran and Pande (2017) use son preference to help explain the
striking differences in stunting between India and Sub-Saharan Africa. India
outperforms Sub-Saharan Africa on most health and development indicators,
but at a given level of income, Indians are much shorter than Sub-Saharan
Africans. Consistent with the authors’ argument that a strong preference for
oldest sons in Hindu culture also explains a large portion of the average
height/stunting gap between Indian and African children, the height/stunting
gradients over birth order are substantially steeper in India than the nearby
Muslim countries of Pakistan and Bangladesh, than in Sub-Saharan Africa, and
than in traditionally matrilineal areas of India. Strikingly, pre-natal
investments are higher for second-born female children with no older brother
before gender is observed, but post-natal investments after gender is
observed are lower.
Bau and Fernandez (2021) discussing the research explored in previous slides
(https://www.nber.org/system/files/working_papers/w28918/w28918.pdf).
6

Lecture 5 Slides, ECO220Y1Y, 3

 Regression Analysis:
Least Squares Method
• Ordinary Least Squares (OLS): Quantitative
method to fit a line through a scatter diagram
• Formula for a geometric line: 𝑦 = 𝑎 + 𝑏𝑥
• 𝑎 is the y-intercept and 𝑏 is the slope
• 𝑥 variable, independent variable, RHS variable
• 𝑦 variable, dependent variable, LHS variable
• But careful, interpreting an OLS line is not the same as
for a geometric line
• Goal: Find line that “best” fits data (𝑎 and 𝑏)
7

Results of a Double-Blind Drug Trial

Dosage Sleep
𝑖
(mg) 𝑥 (hrs) 𝑦
3 4 5 6 7 8
Hours of Sleep

1 5.9 4.6
2 3.5 5.8
3 7.2 6.9
4 3.6 5.8
… … … 2 4 6 8
Dosage (mg)
25 8.2 7.6
X-bar = 4.61 Y-bar = 5.66 Next, discuss how to
sx = 1.51 sy = 1.18 find the red OLS line
sxy = 1.09
8

Lecture 5 Slides, ECO220Y1Y, 4

 OLS: Returns the 𝑎 (intercept) and
𝑏 (slope) that minimize SSE
• Predicted value of y (y-

3 4 5 6 7 8
Hours of Sleep
hat): 𝑦 = 𝑎 + 𝑏𝑥
• Residual: 𝑒 = 𝑦 − 𝑦
• OLS minimizes the SSE:
• 𝑆𝑆𝐸 = ∑ 𝑒
2 4 6 8
• 𝑆𝑆𝐸 = ∑ 𝑦 −𝑦 Dosage (mg)
• 𝑆𝑆𝐸 = ∑ 𝑦 − 𝑎 − 𝑏𝑥 .
𝑏= = = 0.48
.

𝑎 = 𝑦 − 𝑏𝑥̅ = 5.66 − 0.48 ∗ 4.61

= 3.44
𝑏= =𝑟 𝑎 = 𝑦 − 𝑏𝑥̅
𝑦 = 3.44 + 0.48𝑥
9

Interpreting an OLS line

• Must recognize that data scattered around the
line, be context-specific, specify units, be clear on
causality, and assess magnitude (big or small)
– For 𝑦 = 3.44 + 0.48𝑥 , how to interpret 0.48?
• We estimate that an extra mg of drug causes [yields/results
in] about an extra half hour (29 mins) of sleep on average,
which is a considerable boost in sleep.
– For 𝑦 = 3.44 + 0.48𝑥 , how to interpret 3.44?
• We don’t: 0 mg is outside the range of the data and we
should not extrapolate in this case because 0 mg is distinct.
These interpretations are 1 precise sentence. Sometimes need more.
10

Lecture 5 Slides, ECO220Y1Y, 5

 Causality and Observational Data?
“Glassdoor [used] records of 293
companies across 13 industries
between 2008 and 2018. It studied
the link between employee
satisfaction, based on its own
ratings, and the American Customer
Satisfaction Index, a benchmark
gauge of shoppers’ sentiment.”
As an interpretation of the dashed
OLS line, what’s wrong with this?
“A one-point improvement in Glassdoor’s rating (on a five-point scale)
translated into a statistically significant 1.3-point increase in customer
satisfaction (rated from zero to 100).” What does it get right?
The Economist, 2019, “How to keep your customers happy: One way is to ensure your workers are,
too” https://www.economist.com/business/2019/08/22/how-to-keep-your-customers-happy 11

Descriptive Interpretations
OLS line: 𝑦 = 62.83 + 4.73𝑥
(solid line) Interpretation in
2 – 3 sentences?
For U.S. retail firms from 2008 to
2018 tracked by Glassdoor, those
with employee satisfaction that is
1 point higher on a five-point
scale have customer satisfaction
that is 4.7 points higher on a 100
point index on average.
This is a modest difference because a 1 point higher employee
satisfaction is huge (e.g. going from 10th to 90th percentile!).
We cannot infer causality: we simply describe the pattern. 12

Lecture 5 Slides, ECO220Y1Y, 6

 Prices & Production: Manitoba Corn Farms

Annual Data: 1990 - 2011 (n=22) Demand Shifters:

Q-hat = -46.34 + 2.20P, R2 = 0.24
Quantity (1000's tonnes)

500
400
300
Quantity
200 Price
Supplied
100
0
100 150 200 250 Supply Shifters:
Price ($/tonne)

Which kind of data are these?

Source: Manitoba gov’t:

http://www.gov.mb.ca/agriculture/statistics/pdf/crop_grain_corn_sector.pdf 13

Supply Estimation: Price Endogenous

P
S
Demand shifters NOT lurking/omitted/
unobserved/confounding variables 1

Demand Shifters D’
D
1
Price 2 Quantity 2 Q
P
(P) Supplied (QS) S
S’
3 4
Supply Shifters 3

Supply shifters ARE lurking/omitted/ D

unobserved/confounding variables 4 Q
Even if price unchanged,
shift directly affects QS 14

Lecture 5 Slides, ECO220Y1Y, 7

 Interpretation of Correlation, Revisited
y-hat = 3.44 + 0.48x y-hat = 0 + 0.61x

Standardized
8 2
7 1
Sleep,

Sleep,
hours
6 0
5 -1
4
3 -2
2 4 6 8 -2 -1 0 1 2
Dosage, Dosage,
mg Standardized
Increasing the dosage by one
𝑏=𝑟 standard deviation results in
patients sleeping about an
extra 0.6 standard deviation
See pp. 177 – 179 more on average.
of the textbook. 15

Least Squares (OLS) = Regression

Analysis
• Sweet peas: Compare size
of initial seed to seeds
produced by plant once
mature
• Intelligence: Compare
children to their parents
• Heights: Compare children
to their parents

Sir Francis Galton 1822-1911 16

Lecture 5 Slides, ECO220Y1Y, 8

 Regression Towards the Mean
Galton's Height Data, n = 1078
For fathers: mean is
Slope 0.51, Intercept 33.89
80 67.7 inches and s.d.
is 2.7 inches.
75
Son, inches

70 For sons: mean is

68.7 inches and s.d.
65
is 2.8 inches.
60
60 65 70 75
Father, inches

17

Suppose Standardized Heights

Slope 0.50, Intercept 0.00
Because 𝑥 and 𝑦
4 are each
Son's height (standardized)

standardized,
slope (𝑏) equals
2
coefficient of
correlation (𝑟)
0 and −1 ≤ 𝑟 ≤ 1.
Hence, a 1 s.d.
-2 increase in 𝑥
cannot be
-4 associated with
-4 -2 0 2 4 more than 1 s.d.
Father's height (standardized)
change in 𝑦.

18

Lecture 5 Slides, ECO220Y1Y, 9

 Variance and Scatter:
Not about 𝑛
∑
• Recall 𝑠 =
– Both numerator (SST) &
denominator rise with 𝑛:
there is no net effect
– How much 𝑦 varies in
the population is the key
• Scatter is about our
trouble predicting 𝑦: it
depends on how useful
𝑥 is, not on 𝑛
19

Residuals (error): 𝑒 = 𝑦 − 𝑦
• Constant term  mean 𝑦 = 3.44 + 0.48𝑥
of residuals is 0
𝑥 𝑦 𝑦 𝑒
• Estimated s.d. of 𝑖
(mg) (hrs) (hrs) (hrs)
residuals; Root MSE 1 5.9 4.6 6.3 -1.7
(Mean Square Error):
2 3.5 5.8 5.1 0.7
∑ 𝑒 −0 3 7.2 6.9 6.9 0.0
𝑠 =
𝑛−2 4 3.6 5.8 5.2 0.6
“standard
error of 𝑆𝑆𝐸 … … …
estimate” 𝑠 = 𝑛−2 25 8.2 7.6 7.4 0.2

Used up 2 degrees of freedom: ei = yi – a – bxi 20

Lecture 5 Slides, ECO220Y1Y, 10

 y-hat = 3.44 + 0.48x mean = 0, sd = 0.93
8

.2 .4 .6
Sleep, hours
7

Density
6
5
4
3

0
2 4 6 8 -2 -1 0 1 2
Dosage, mg Residual (e)

2
Residual (hours)

Homoscedasticity: The
1 variance of the residual is
0 constant
-1 Homoscedasticity means that
-2 it makes sense to talk about
4 5 6 7 8 the standard deviation of the
Predicted Sleep (hours) residual
21

Homoscedasticity Heteroscedasticity
3
-1 0 1 2 3

2
Y1

Y2
1
0

0 2 4 6 8 10 0 2 4 6 8 10
X1 X2

Homoscedasticity Heteroscedasticity
-1-.5 0 .5 1 1.5

-1.5-1-.5 0 .5 1
Residual

Residual

0 .5 1 1.5 2 2.5 0 .5 1 1.5 2

Y1-hat Y2-hat
22

Lecture 5 Slides, ECO220Y1Y, 11

 Analysis of Variance (ANOVA): Fit
• ANOVA: How total SST = 33.5 SSR = 12.6 SSE = 20.9
variability of y relates to x

4 5 6 7 8
Hours of Sleep
versus everything else
• Total sum of squares:
𝑆𝑆𝑇 = ∑ 𝑦 −𝑦 ∑
𝑠 =
• Regression sum of squares:
𝑆𝑆𝑅 = ∑ 𝑦 −𝑦

3
2 4 6 8
• Sum of squared errors: Dosage (mg)

𝑆𝑆𝐸 = ∑ 𝑦 −𝑦 Units of measurement?

• 𝑆𝑆𝑇 = 𝑆𝑆𝑅 + 𝑆𝑆𝐸

23

Understanding the SST, SSR, & SSE

𝑥 somewhat explains 𝑦 What if 𝑛 decreases?
y1, y1-hat

y2, y2-hat

𝑆𝑆𝑇 =
∑ 𝑦 −𝑦

𝑆𝑆𝑅 =
If 𝑥 unrelated with 𝑦? If 𝑥 explains 𝑦 perfectly? ∑ 𝑦 −𝑦

𝑆𝑆𝐸 =
y4, y4-hat
y3, y3-hat

∑ 𝑦 −𝑦

24

Lecture 5 Slides, ECO220Y1Y, 12

 𝑅2 (Coefficient of Determination):
A Measure of Fit
• 𝑅 = = =1−
– Fraction of total variation in dependent variable
(𝑦) explained by indep. variable (𝑥)
• 1 − 𝑅 : unexplained variation in 𝑦
– For simple regression: 𝑅 = 𝑟
.
• For example, Galton’s data: 𝑅 = = 0.25
.
Twenty five percent of the variation in height across
sons is explained by variation in the heights of their
fathers

25

Y-hat = 9.12 + 0.86X Y-hat = 9.16 + 0.86X

R-squared = 0.90 R-squared = 0.86
Females, 2012

Females, 2012

80 80
70 Turkey 70
60 60
50 50
40 40
30 30
20 40 60 80 20 40 60 80
Females, 2006 Females, 2006

Y-hat = -12.53 + 1.00X Y-hat = -13.97 + 1.05X

R-squared = 0.37 R-squared = 0.60
Females, 2012

Females, 2012

80 80
70 70
60
50 60
40 Turkey 50
30 Mexico
40
60 65 70 75 80 85 60 65 70 75 80 85
Males, 2012 Males, 2012

OECD data, again (n = 34 countries) 26

Lecture 5 Slides, ECO220Y1Y, 13

 Summary Values
• If the available data contain summaries of a
larger data set (e.g. means, totals, etc.) this
affects the regression analysis
• For example, suppose an analyst knows that
the Canadian and U.S. economies are closely
linked and regresses Canadian GDP growth (%)
on U.S. GDP growth
– Does it matter if it is annual or quarterly data?
See Section 19.4 of your textbook – “Working with Summary Values” – which
is required readings for this week.
27

1971 – 2012 Data (OECD)

Which is the quarterly data? Annual data?

28

Lecture 5 Slides, ECO220Y1Y, 14

 Top 10 reasons why interpreting an
OLS line is NOT like a geometric line
1. Most/all data do NOT lie on an OLS line: say “on average” to
recognize scatter. A geometric line maps from 𝑥 to 𝑦 exactly.
2. Even for a sample size of two, where the OLS and geometric
lines would overlap, when interpreting the OLS line you must
remember the huge sampling error for such a small sample.
3. Consider how well an OLS line fits data with statistics like 𝑅 .
4. The OLS slope is the coefficient of correlation if the data are
standardized. This makes no sense for a geometric line.
5. A geometric line implies causality but an OLS line can only be
interpreted causally with experimental data; for
observational data, slope likely suffers an endogeneity bias.

29

Top 10 reasons why interpreting an

OLS line is NOT like a geometric line
6. For a geometric line, can write 𝑦 in terms of 𝑥 or 𝑥 in terms
of 𝑦: 𝑦 = 𝑎 + 𝑏𝑥 equivalent to 𝑥 = − + 𝑦. For an OLS line
if you switch 𝑥 and 𝑦 variables you get a different line, not
just a re-written one.
7. While an intercept has a simple interpretation for a
geometric line (value of 𝑦 when 𝑥 is zero), for an OLS line the
intercept often has no meaning or should not be interpreted
because zero is well beyond the range of the data.
8. An OLS line is not robust to outliers: report results with and
without the outlier(s). In contrast, there is only one
geometric line and all points lie on it.

30

Lecture 5 Slides, ECO220Y1Y, 15

 Top 10 reasons why interpreting an
OLS line is NOT like a geometric line
9. An OLS line may describe variables where one or both have
been non-linearly transformed and the “slope” is not
interpreted as a slope. This is the point of the required
reading “Logarithms in Regression Analysis with Asiaphoria.”
10. OLS lines require substantial expertise to interpret properly.
But, this doesn’t mean nothing translates. Changes in units of
measurement change a line in perfectly predictable ways:
If y-hat = 107.07 – 4.20*x when x measured in hours,
then y-hat = 107.07 – 0.07*x when x is in minutes
If y-hat = 20.48 + 0.61*x when y measured in $100’s,
then y-hat = 2.048 + 0.061*x when y is in $1,000’s
For more practice with unit changes and a line, see the Prerequisite Review.
31

“Initial Coin Offerings: Financing Growth

with Cryptocurrency Token Sales”
ABSTRACT: Initial coin offerings (ICOs) are sales of blockchain-
based digital tokens associated with specific platforms or assets.
Since 2014 ICOs have emerged as a new financing instrument,
with some parallels to IPOs, venture capital, and pre-sale
crowdfunding. We examine the relationship between issuer
characteristics and measures of success, with a focus on liquidity,
using 453 ICOs that collectively raise $5.7 billion. We also employ
propriety transaction data in a case study of Filecoin, one of the
most successful ICOs. We find that liquidity and trading volume
are higher when issuers offer voluntary disclosure, credibly
commit to the project, and signal quality.

Howell et. al. (2018) http://www.nber.org/papers/w24774

32

Lecture 5 Slides, ECO220Y1Y, 16

 Figure 9: Cumulative Returns at 5 Months
[𝑛 = 444 ICOs, Fitted line in red]

Is “first date traded”

140 day returns [percent]

as a quantitative x
variable in OLS valid?

Note: Cumulative returns between the start of trading and 140 days (5 months)
subsequently. Right panel excludes observations w/ returns > 30 [outliers]. 33

Review Formulas: Lectures 2 & 4

Bank of Canada data
• 𝑦 = 1.433 ∑ 𝑦 n = 101 months
𝑦= 2.5
𝑛
2
CPIX

• 𝑠 = 0.497 1.5
1
∑ 𝑦 −𝑦 .5
𝑠 = 0 1 2 3 4 5
𝑛−1 Lagged Target Rate

• 𝑥̅ = 2.453
• 𝑟 = 0.309
• 𝑠 = 1.395
𝑠 ∑ 𝑧 𝑧
• 𝑠 = 0.215 𝑟= =
𝑠 𝑠 𝑛−1
∑ 𝑥 − 𝑥̅ 𝑦 − 𝑦
𝑠 =
𝑛−1
34

Lecture 5 Slides, ECO220Y1Y, 17

 Review Formulas: Lecture 5
Bank of Canada data
• 𝑦 = 1.16 + 0.11𝑥 n = 101 months
𝑠 𝑠 2.5
𝑏= =𝑟 𝑎 = 𝑦 − 𝑏𝑥̅ 2

CPIX
𝑠 𝑠 1.5
1
.5
• Residuals: 𝑒 =𝑦 −𝑦 0 1 2 3 4 5
– E.g. Jan. 2013: 𝑥 = 1, 𝑦 Lagged Target Rate
= 0.5, 𝑦 = 1.27,
𝑒 = −0.77 (= 0.5 − 1.27) • 𝑆𝑆𝑇 = ∑ 𝑦 −𝑦 = 24.7
• s.d. of resids: 𝑠 = 0.4748 • 𝑆𝑆𝑅 = ∑ 𝑦 −𝑦 = 2.4
∑ 𝑒 • 𝑆𝑆𝐸 = ∑ 𝑦 −𝑦 = 22.3
𝑠 = • 𝑅 = 𝑆𝑆𝑅/𝑆𝑆𝑇 = 0.10
𝑛−2
If standardized both variables, slope of regression line? 35

Recap
• Simple regression is an important tool, even in
cutting-edge empirical research
• Finding an OLS line, easy, interpreting it, hard
• Spent a lot of time on scatter and residuals
– The 𝑠 , 𝑟, or 𝑅 quantify scatter about an OLS line
(fit is opposite of scatter): all derive from ANOVA
– Addressed outliers and summary values
• An OLS line is not an ordinary line

36

Lecture 5 Slides, ECO220Y1Y, 18