AE 248: AI and Data Science: Prabhu Ramachandran 2024-03-01
AE 248: AI and Data Science: Prabhu Ramachandran 2024-03-01
AE 248: AI and Data Science: Prabhu Ramachandran 2024-03-01
Parameter Estimation
Prabhu Ramachandran
2024-03-01
Parameter Estimation
Estimates
Estimates/Estimators
• Point estimates
• Interval estimates
• Confidence
• Estimator: statistic to estimate unknown parameter 𝜃
1
. . .
Solution
𝑃 {𝑋𝑖 = 𝑥} = 𝑝𝑥 (1 − 𝑝)(1−𝑥) , 𝑥 = 0, 1
Answer
𝑛
∑ 𝑥𝑖
𝑝̂ = 𝑖=1
𝑛
Solution
𝑒−𝑛𝜆 𝜆∑ 𝑥𝑖
𝑓(𝑥1 , … , 𝑥𝑛 |𝜆) =
𝑥1 ! … 𝑥𝑛 !
maximize {log 𝑓(𝑥1 , … , 𝑥𝑛 |𝜆)}
2
Answer
𝑛
∑𝑖=1 𝑥𝑖
𝜆̂ =
𝑛
Problem
The number of traffic accidents in Mumbai, in 10 randomly chosen non-rainy days in 2003 is
as follows:
4, 0, 6, 5, 2, 1, 2, 0, 4, 3
Use this to estimate the proportion of non rainy days that had 2 or fewer accidents that year.
• Self-study
• Same idea and approach
• Two parameters, so maximize w.r.t. each
• If 𝑥 ∈ (0, 𝜃)
• 𝜃 should be small
• But large enough for largest 𝑋𝑖
Interval estimates
. . .
3
Interval estimates
√ ̄
• 𝑛 𝑋−𝜇𝜎 is a standard normal
• So, for example:
√ 𝑋̄ − 𝜇
𝑃 {−1.96 < 𝑛 < 1.96} = 0.95
𝜎
Interval estimates
𝜎 𝜎
𝑃 {𝑋̄ − 1.96 √ < 𝜇 < 𝑋̄ + 1.96 √ } = 0.95
𝑛 𝑛
Example
Interpretation
• If we estimated the interval a 100 times (with 100 samples), then we expect the number
of intervals that contain the true mean 𝜇 to tend to 90.
• There is a 90% probability that if we calculate the CI in a future experiment it will
encompass the true mean.
Suppose that when a signal having value 𝜇 is transmitted from location A the value received
at location B is normally distributed with mean 𝜇 and variance 4. That is, if 𝜇 is sent, then
the value received is 𝜇 + 𝑁 where N , representing noise, is normal with mean 0 and variance
4. To reduce error, suppose the same value is sent 9 times. If the successive values received
are 5, 8.5, 12, 15, 7, 9, 7.5, 6.5, 10.5, let us construct a 95 percent confidence interval for 𝜇.
4
Two-sided vs one-sided
√ 𝑋̄ − 𝜇
𝑃{ 𝑛 < 1.645} = 0.95
𝜎
𝜎
𝑃 {𝑋̄ − 1.645 √ < 𝜇} = 0.95
𝑛
One-sided intervals
• Recall 𝑃 {𝑍 > 𝑧𝛼 } = 𝛼
• 𝑃 {−𝑧𝛼/2 < 𝑍 < 𝑧𝛼/2 } = 1 − 𝛼
𝜎 𝜎
𝑃 {𝑋̄ − 𝑧𝛼/2 √ < 𝜇 < 𝑋̄ + 𝑧𝛼/2 √ } = 1 − 𝛼
𝑛 𝑛
Finding suitable n
𝑠 𝑠
𝑃 {𝑋̄ − 𝑡𝛼/2,𝑛−1 √ < 𝜇 < 𝑋̄ + 𝑡𝛼/2,𝑛−1 √ } = 1 − 𝛼
𝑛 𝑛
5
Non-normal populations
Example
The weights of 5 students was found to be 61, 65, 68, 58, and 70 Kgs. Determine a 95%
confidence interval for their mean. Also determine a 95% lower confidence interval for this
mean.
Difference in means
Difference in means
• MLE of 𝜇1 − 𝜇2 is 𝑋̄ − 𝑌 ̄
𝑋̄ − 𝑌 ̄ − (𝜇1 − 𝜇2 )
2
∼ 𝒩(0, 1)
√ 𝜎1 + 𝜎22
𝑛 𝑚
• If 𝜎1 ≠ 𝜎2 we have a problem
• If they are the same the same approach as before can be used
𝑋̄ − 𝑌 ̄ − (𝜇1 − 𝜇2 )
∼ 𝒩(0, 1)
2 𝜎2
√ 𝜎𝑛 + 𝑚
6
Variances unknown
(𝑛 − 1)𝑆12 + (𝑚 − 1)𝑆22
𝑆𝑝2 =
𝑛+𝑚−2
Variances unknown
𝑋̄ − 𝑌 ̄ − (𝜇1 − 𝜇2 )
∼ 𝑡𝑛+𝑚−2
√𝑆𝑝2 ( 𝑛1 + 1
𝑚)
Unbiased estimators
7
Bayes estimator
𝑓(𝜃, 𝑥1 , … , 𝑥𝑛 )
𝑓(𝜃|𝑥1 , … , 𝑥𝑛 ) =
𝑓(𝑥1 , … , 𝑥𝑛 )
𝑝(𝜃)𝑓(𝑥1 , … , 𝑥𝑛 |𝜃)
=
∫ 𝑓(𝑥1 , … , 𝑥𝑛 |𝜃)𝑝(𝜃)𝑑𝜃
Bayes estimator