Statistical Theory 098414 Summary Notes Statistical Models

❙t❛t✐st✐❝❛❧ ❚❤❡♦r② ✵✾✽✹✶✹ ✕ ❙✉♠♠❛r② ◆♦t❡s ❏✉❧② ✺✱ ✷✵✶✾ ❙t❛t✐st✐❝❛❧ ▼♦❞❡❧s ❘❡❣✉❧❛r✐t② ❈♦♥❞✐t✐♦♥s {Pθ }θ∈Θ ✐s ❝❛❧❧❡❞ ❆ st❛t✐st✐❝❛❧ ♠♦❞❡❧ ✐s ❛ ❢❛♠✐❧② ♦❢ ❞✐str✐❜✉t✐♦♥s✱ {Pθ }θ∈Θ ✳ ✶✳ Pθ ✐s ❝♦♥t✐♥✉♦✉s ❢♦r ❡✈❡r② θ ∈ Θ • X ∼ Pθ ❛r❡ t❤❡ r❛♥❞♦♠ ♦❜s❡r✈❛t✐♦♥s ❛ss♦❝✐❛t❡❞ ✇✐t❤ t❤❡ ♠♦❞❡❧✳ • LX (θ) = Pθ (X) ✐s ❝❛❧❧❡❞ t❤❡ • • r❡❣✉❧❛r ✐❢ ❡✐t❤❡r ♦❢ t❤❡ ❢♦❧❧♦✇✐♥❣ ❤♦❧❞s✿ ∞ ✷✳ Pθ ✐s ❞✐s❝r❡t❡ P∞ ❢♦r ❡✈❡r② θ ∈ Θ ❛♥❞ t❤❡r❡ ❡①✐sts ❛ ❝♦✉♥t❛❜❧❡ s❡t X = {ξi }i=1 s✉❝❤ t❤❛t i=1 Pθ (X = ξi ) = 1✱ ∀θ ∈ Θ✳ ❧✐❦❡❧✐❤♦♦❞ ❢✉♥❝t✐♦♥✳ P❛r❛♠❡tr✐❝ ♠♦❞❡❧✿ ✇❤❡♥ Θ ✐s ✜♥✐t❡ ❞✐♠❡♥s✐♦♥❛❧ ✭❡✳❣✳✱ R3 ✮✳ ❋❛❝t♦r✐③❛t✐♦♥ ❚❤❡♦r❡♠ ✭❋✐s❤❡r✲◆❡②♠❛♥✮ ◆♦♥✲♣❛r❛♠❡tr✐❝ ♠♦❞❡❧✿ ✇❤❡♥ Θ ❤❛s ✐♥✜♥✐t❡ ❞✐♠❡♥s✐♦♥✳ ■❢ {Pθ }θ∈Θ ✐s r❡❣✉❧❛r✱ t❤❡♥ T (X) ✐s ❛ s✉✣❝✐❡♥t st❛t✐st✐❝ ✐❢✲❛♥❞✲♦♥❧②✲✐❢ t❤❡r❡ ❡①✐st • ◆✉✐s❛♥❝❡ ♣❛r❛♠❡t❡rs✿ t❤❡ ❝♦♠♣♦♥❡♥ts ♦❢ θ ✇❤✐❝❤ ❛r❡ ♥♦t ♦❢ ✐♥t❡r❡st ✭❡✳❣✳✱ ✇❤❡♥
❢✉♥❝t✐♦♥s g ❛♥❞ h s✉❝❤ t❤❛t θ = µ, σ 2 ❛♥❞ ♦♥❡ ✐s ✐♥t❡r❡st❡❞ ♦♥❧② ✐♥ µ✱ t❤❡♥ σ 2 ✐s ❛ ♥✉✐s❛♥❝❡ ♣❛r❛♠❡t❡r✮✳ Pθ (x) = g (T (x) , θ) · h (x) ■❞❡♥t✐✜❛❜✐❧✐t② ▼✐♥✐♠❛❧ ❙✉✣❝✐❡♥t ❙t❛t✐st✐❝ ❆ ♠♦❞❡❧ Pθ ✐s ✐❞❡♥t✐✜❛❜❧❡ ✐❢ ❢♦r ❡✈❡r② θ1 , θ2 ∈ Θ✱ θ1 6= θ2 ⇒ S (X) ✐s ❝❛❧❧❡❞ ♠✐♥✐♠❛❧ s✉✣❝✐❡♥t st❛t✐st✐❝ ✐❢ ❢♦r ❡✈❡r② s✉✣❝✐❡♥t st❛t✐st✐❝ T (X) t❤❡r❡ ❡①✐st ❛ ❢✉♥❝t✐♦♥ f s✉❝❤ t❤❛t S (x) = f (T (x)) ✭♥❛♠❡❧② S ✐s ❝♦❛rs❡r t❤❛♥ T ✮✳ Pθ1 6= Pθ2 • ❚❤❡ ✐♥✈❡rs❡ ✐s tr✐✈✐❛❧❧② tr✉❡✱ t❤❛t ✐s✿ Pθ1 6= Pθ2 ⇒ θ1 6= θ2 ✳ • ■❞❡♥t✐✜❛❜✐❧✐t② ♠❡❛♥s ❛ ♦♥❡✲t♦✲♦♥❡ ❝♦rr❡s♣♦♥❞❡♥❝❡ ❜❡t✇❡❡♥ Θ ❛♥❞ {Pθ }θ∈Θ ❀ ■t ❚❤❡♦r❡♠✿ ✐s ❛ ♥❡❝❡ss❛r② ❝♦♥❞✐t✐♦♥ ❢♦r t❤❡ ❛❜✐❧✐t② t♦ ❡st✐♠❛t❡ t❤❡ ♣❛r❛♠❡t❡r θ✳ ❆ s✉✣❝✐❡♥t st❛t✐st✐❝ S ✐s ♠✐♥✐♠❛❧ ✐❢ ❢♦r ❡✈❡r② x, y t❤❡ ❢♦❧❧♦✇✐♥❣ ❝♦♥❞✐t✐♦♥ ❤♦❧❞s✿ ❙✉✣❝✐❡♥t ❙t❛t✐st✐❝ ❆ss✉♠❡ X ∼ Pθ ✳ • ❆ ❢✉♥❝t✐♦♥ T (X) ✐s ❝❛❧❧❡❞ ❛ • T (X) ✐s ❝❛❧❧❡❞ S (x) = S (y) ❞✐str✐❜✉t✐♦♥ ♦❢ X| T = t ❞♦❡s ♥♦t ❞❡♣❡♥❞ ♦♥ θ✳ Pθ (x) ❞♦❡s ♥♦t ❞❡♣❡♥❞ ♦♥ θ Pθ (y) ❈♦♠♣❧❡t❡ ❙t❛t✐st✐❝ st❛t✐st✐❝ ✐❢ ✐t ❞♦❡s ♥♦t ❞❡♣❡♥❞ ♦♥ θ✳ s✉✣❝✐❡♥t st❛t✐st✐❝ ✐❢ ❢♦r ❡✈❡r② t ✐♥ t❤❡ r❛♥❣❡ ♦❢ ⇐⇒ ❆ s✉✣❝✐❡♥t st❛t✐st✐❝ T (X) ✐s ❝❛❧❧❡❞ ❝♦♠♣❧❡t❡ ✐❢ ∀θ ∈ Θ ✐t ❤♦❧❞s t❤❛t T (·)✱ t❤❡ Eθ [g (T (X))] = 0 ✶ =⇒ g (T (X)) = 0 ❛✳s✳ ❊①♣♦♥❡♥t✐❛❧ ❋❛♠✐❧✐❡s ❇❛②❡s✐❛♥ ▼♦❞❡❧s ❚❤❡ ♠♦❞❡❧ X ∼ Pθ ❜❡❧♦♥❣s t♦ ❛ ♦♥❡✲❞✐♠❡♥s✐♦♥❛❧ ❡①♣♦♥❡♥t✐❛❧ ❢❛♠✐❧② ✐❢ • ■♥ ❇❛②❡s✐❛♥ ♠♦❞❡❧s✱ θ ✐s ❝♦♥s✐❞❡r❡❞ ❛s ❛ r❛♥❞♦♠ ✈❛r✐❛❜❧❡✳ Pθ (x) = exp [c (θ) T (x) + d (θ) + S (x)] · IA (x) • (X, θ) ❤❛s ❛ ❥♦✐♥t ❞✐str✐❜✉t✐♦♥✱ ✇❤❡r❡ ( X| θ = θ0 ) ∼ Pθ0 ❛♥❞ θ ∼ π ✳ ✇❤❡r❡ IA (x) = I (x ∈ A) ❛♥❞ t❤❡ s✉♣♣♦rt A ❞♦❡s ♥♦t ❞❡♣❡♥❞ ♦♥ θ✳ • Pθ0 ✐s t❤❡ ♠♦❞❡❧ ❛♥❞ π ✐s ❝❛❧❧❡❞ t❤❡ ♣r✐♦r ✭✇✐t❤ s✉♣♣♦rt Θ✮✳ ❘❡♠❛r❦s✿ • ▼♦❞❡❧ ✰ ♣r✐♦r ❞❡t❡r♠✐♥❡ t❤❡ ❞✐str✐❜✉t✐♦♥ ♦❢ ( θ| X = x)✱ ✇❤✐❝❤ ✐s t❤❡ ♣♦st❡✲ r✐♦r✳ • T (X) ✐s ❛ s✉✣❝✐❡♥t st❛t✐st✐❝ ✭❜❛s❡❞ ♦♥ t❤❡ ❢❛❝t♦r✐③❛t✐♦♥ t❤❡♦r❡♠✮✳ • ❈♦♠♠♦♥ ❡①❛♠♣❧❡s✿ ❇✐♥♦♠✐❛❧✱ ●❛♠♠❛✱ ◆♦r♠❛❧✱ ❇❡t❛✱ ●❡♦♠❡tr✐❝✳ • ❚❤❡ ❧✐❦❡❧✐❤♦♦❞ ❢✉♥❝t✐♦♥ ✐♥ t❤✐s ❝❛s❡ ✐s LX (θ0 ) = p (X |θ = θ0 ) = Pθ0 (X)✳ • ❈♦♠♠♦♥ ❡①❛♠♣❧❡s ♦❢ ♥♦♥✲❡①♣♦♥❡♥t✐❛❧ ❢❛♠✐❧✐❡s✿ ❯♥✐❢♦r♠✱ ❍②♣❡r✲●❡♦♠❡tr✐❝✱ ❈♦♥❥✉❣❛t❡ ❉✐str✐❜✉t✐♦♥s ❈❛✉❝❤②✱ ●❛✉ss✐❛♥ ▼✐①t✉r❡✱ t✲❞✐str✐❜✉t✐♦♥✱ ❋✲❞✐str✐❜✉t✐♦♥✳ • ❚❤❡ ❦✲❞✐♠❡♥s✐♦♥❛❧ ❝❛s❡ ♦❢ ❡①♣♦♥❡♥t✐❛❧ ❢❛♠✐❧② ✐s✿   k X cj (θ) Tj (x) + d (θ) + S (x) · IA (x) Pθ (x) = exp  • ■❢ t❤❡ ♣r✐♦r ❛♥❞ ♣♦st❡r✐♦r ❛r❡ ♦❢ t❤❡ s❛♠❡ ❢❛♠✐❧② ♦❢ ❞✐str✐❜✉t✐♦♥s✱ t❤❡② ❛r❡ ❝❛❧❧❡❞ ❝♦♥❥✉❣❛t❡ ❞✐str✐❜✉t✐♦♥s✳ • ■♥ t❤✐s ❝❛s❡✱ t❤❡ ♣r✐♦r ✐s ❝❛❧❧❡❞ ❛ ❝♦♥❥✉❣❛t❡ ♣r✐♦r ❢♦r t❤❡ ❧✐❦❡❧✐❤♦♦❞ ❢✉♥❝t✐♦♥✳ j=1 • ❊①❛♠♣❧❡s✿ ◆❛t✉r❛❧ ❋♦r♠✿ ✕ ❇❡t❛ ❞✐str✐❜✉t✐♦♥ ✐s ❛ ❝♦♥❥✉❣❛t❡ ♣r✐♦r ✇✐t❤ r❡s♣❡❝t t♦ ❛ ❇✐♥♦♠✐❛❧ ♠♦❞❡❧✳ ❙✉♣♣♦s❡ t❤❛t c (θ) = η ✐s ❛ ♦♥❡✲t♦✲♦♥❡ ❢✉♥❝t✐♦♥ ❛♥❞ ❞❡✜♥❡ d0 (η) = d c−1 (η) ✱ ✇❤❡r❡ η ∈ H ✳ ❚❤❡♥ t❤❡ ♥❛t✉r❛❧ ❢♦r♠ ♦❢ t❤❡ ❡①♣♦♥❡♥t✐❛❧ ❢❛♠✐❧② ✐s t❤❡ ❢♦❧❧♦✇✐♥❣ r❡✲♣❛r❛♠❡t❡r✐③❛t✐♦♥ ♦❢ t❤❡ ♠♦❞❡❧✿
✕ ❋♦r t❤❡ ❡①♣♦♥❡♥t✐❛❧ ❢❛♠✐❧②✱ t❤❡ ❝♦♥❥✉❣❛t❡ ♣r✐♦r ✐s πt1 ,t2 (θ0 ) = exp (t1 c (θ0 ) + t2 d (θ0 ) + w (t1 , t2 )) I (θ0 ∈ Θ) P̃η (x) = exp [η · T (x) + d0 (η) + S (x)] · IA (x) ✇❤❡r❡ w (t1 , t2 ) = − log Θ (t1 c (θ) + t2 d (θ)) dθ ✐s ❛ ♥♦r♠❛❧✐③✐♥❣ ❝♦♥st❛♥t ❛♥❞ t❤❡ ♣❛r❛♠❡t❡r s♣❛❝❡ ✐s Ω = {(t1 , t2 ) : w (t1 , t2 ) < ∞}✳ R ❚❤❡♦r❡♠s • ■❢ {Pθ }θ∈Θ ✐s ❛ ♦♥❡✲❞✐♠❡♥s✐♦♥❛❧ ❞✐s❝r❡t❡ ❡①♣♦♥❡♥t✐❛❧ ❢❛♠✐❧②✱ t❤❡♥ T ∼ Qθ ✇❤❡r❡ {Qθ }θ∈Θ ✐s ❛❧s♦ ❛ ♦♥❡✲❞✐♠❡♥s✐♦♥❛❧ ❡①♣♦♥❡♥t✐❛❧ ❢❛♠✐❧②✳ P❛r❛♠❡t❡r ❊st✐♠❛t✐♦♥ • ▲❡t X = (X1 , . . . , Xn ) ❜❡ ❛ s❛♠♣❧❡ ♦❢ s✐③❡ n ❢r♦♠ Pθ ✱ ✇✐t❤ {Pθ }θ∈Θ ❜❡✐♥❣ ❛ ♦♥❡✲❞✐♠❡♥s✐♦♥❛❧ ❡①♣♦♥❡♥t✐❛❧ ❢❛♠✐❧②✳ ❚❤❡♥ t❤❡ ❞✐str✐❜✉t✐♦♥ P ♦❢ X ✐s ❛❣❛✐♥ ❛ ♦♥❡✲❞✐♠❡♥s✐♦♥❛❧ ❡①♣♦♥❡♥t✐❛❧ ❢❛♠✐❧② ✇✐t❤ s✉✣❝✐❡♥t st❛t✐st✐❝ ni=1 T (Xi )✳ • ❚❤❡ ✉♥❦♥♦✇♥ ♣❛r❛♠❡t❡r ✐s θ ∈ Θ✳ • ❚❤❡ r❛♥❞♦♠ ♦❜s❡r✈❛t✐♦♥s ❛r❡ ❛ss✉♠❡❞ t♦ ❜❡ X1 , . . . , Xn ∼ Pθ ✳ ✐✳✐✳❞ • ❈♦♥s✐❞❡r t❤❡ ♥❛t✉r❛❧ ❢♦r♠ ♦❢ t❤❡ ❡①♣♦♥❡♥t✐❛❧ ❢❛♠✐❧② ❛♥❞ ❛ss✉♠❡ t❤❛t η ✐s ❛♥ ✐♥♥❡r ♣♦✐♥t ✐♥ H ✱ t❤❡♥ ✕ ❚❤❡r❡❢♦r❡✱ t❤❡ ❞✐str✐❜✉t✐♦♥ ♦❢ X = (X1 , . . . , Xn ) ✐s ✶✳ MT (t) = exp (d0 (η) − d0 (η + t))✱ t❤❡ ♠♦♠❡♥t✲❣❡♥❡r❛t✐♥❣ ❢✉♥❝t✐♦♥ ♦❢ T ✳ ✷✳ E (T ) = −d′0 (η) ❛♥❞ Var (T ) = −d′′0 (η)✳ Qn i=1 Pθ (xi )✳ • ❆♥ ❡st✐♠❛t♦r ♦r ❞❡❝✐s✐♦♥ r✉❧❡ θ̂ = δ̃ (X) ✐s s♦♠❡ ❢✉♥❝t✐♦♥ ♦❢ X✳ ✷ ▲♦ss ❛♥❞ ❘✐s❦ • ❆  ❧♦ss ❢✉♥❝t✐♦♥ L θ̂, θ ✈❛❧✉❡ ✐s ✕ θ❀  • ♠❡❛s✉r❡s ✏❤♦✇ ❜❛❞✑ ❛♥ ❡st✐♠❛t❡ ✕ ❙q✉❛r❡❞ ❧♦ss ♦r r✐s❦ ❢✉♥❝t✐♦♥ ✕ ✕ • ❚❤❡  ■t ✐s ❝♦♠♠♦♥ t♦ ❞❡♥♦t❡ ✐t ❛s θ̂ = δ̃ (X) 2   R θ̂, θ ❚❤❡ r✐s❦ ✐s ❛ ❢✉♥❝t✐♦♥ ♦❢ ❆❞♠✐ss✐❜✐❧✐t② ❆♥ ❡st✐♠❛t♦r θ̂ ▲❡t • ❚❤❡ ♠♦♠❡♥t ❡st✐♠❛t♦r ♦❢ • θ̂✮✳ ✐s ❝❛❧❧❡❞ X✳ • h i  • ∃θ ∈ Θ ✇✐t❤ str✐❝t ✐♥❡q✉❛❧✐t②✳ ❆♥ ❡st✐♠❛t♦r θ̂ ✐s ❝❛❧❧❡❞ • • ▲❡t ❈♦♥s✐❞❡r t❤❡ L2 ✲❧♦ss ❢✉♥❝t✐♦♥✱ s♦  R θ̂, θ = Pn i=1 Eθ i=1 Xik ✳ q (θ) = g (m1 (θ) , . . . , mr (θ)) ✐s ❞❡✜♥❡❞ ❛s g ✐s ♥♦t ✉♥✐q✉❡✳ ( n Y Pθ (Xi ) i=1 ) LX (θ)✿ ) ( n X log Pθ (Xi ) = arg max θ∈Θ i=1 ■❢ t❤❡ tr✉❡ ♣❛r❛♠❡t❡r ✈❛❧✉❡ ✐s θ0 ✱ t❤❡♥ ❜② t❤❡ ✇❡❛❦ ❧❛✇ ♦❢ ❧❛r❣❡ ♥✉♠❜❡rs✱ n 1X P log Pθ (Xi ) −→ Eθ0 [log Pθ (X1 )] n i=1 ❖♥ t❤❡ ♦t❤❡r ❤❛♥❞✱ ✐t ❤♦❧❞s t❤❛t ❢♦r ❛❧❧ θ ∈ Θ✱ Eθ0 [log Pθ (X1 )] ≤ Eθ0 [log Pθ0 (X1 )] ❜❡ ✐♥❞❡♣❡♥❞❡♥t r❛♥❞♦♠ ✈❛r✐❛❜❧❡s✱  ✕ ✕ ✭✐t ❢♦❧❧♦✇s ❢r♦♠ ❏❡♥s❡♥ ✐♥❡q✉❛❧✐t② ❛♥❞ t❤❡ ❝♦♥✈❡①✐t② ♦❢ ❛❞♠✐ss✐❜❧❡ ✐❢ t❤❡r❡ ❡①✐st ♥♦ s✉❝❤ ❡st✐♠❛t♦r θ̃✳ Xi ∼ N (θi , 1) , i = 1, . . . , n Pn ❲❤② ▼▲ ❡st✐♠❛t✐♦♥ ✐s ❛ ✏❣♦♦❞ ✐❞❡❛✑❄ s✉❝❤ t❤❛t  • ❙t❡✐♥✬s P❛r❛❞♦①✿ • 1 n ❚❤❡ ▼▲❊ ✐s t❤❡ ♠❛①✐♠✐③❡r ♦❢ t❤❡ ❧✐❦❡❧✐❤♦♦❞ ❢✉♥❝t✐♦♥ θ∈Θ R θ̃, θ ≤ R θ̂, θ , ∀θ ∈ Θ ❛♥❞ ❢♦r ❛♥②  m̂k = ❛♥❞ θ̂M L (X) = arg max   h  i2 = Varθ θ̂ + biasθ θ̂  2 kXk ▼❛①✐♠✉♠ ▲✐❦❡❧✐❤♦♦❞ ❊st✐♠❛t✐♦♥ ✭▼▲❊✮ ✐♥❛❞♠✐ss✐❜❧❡ ✐❢ t❤❡r❡ ❡①✐st ❛♥ ❡st✐♠❛t♦r θ̃  θ̂✱ ❚❤❡ ♠♦♠❡♥t ❡st✐♠❛t♦r ✐s ♥♦t ✉♥✐q✉❡✱ s✐♥❝❡ ❢✉♥❝t✐♦♥✱ t❤❡ r✐s❦ ✐s t❤❡ ▼❡❛♥✲❙q✉❛r❡❞✲❊rr♦r✿ θ̂ − θ n−2 q̂ = g (m̂1 , . . . , m̂r ) ✐t ✐s ♥♦t ❛ r❛♥❞♦♠ ✈❛r✐❛❜❧❡✳ 2    mk (θ) = Eθ X1k • t♦ ✐♥❞✐❝❛t❡ t❤❡ ❞❡♣❡♥❞❡♥❝❡ ♦♥ t❤❡    1−   ❜✐❛s ♦❢ ❛♥ ❡st✐♠❛t♦r ✐s ❞❡✜♥❡❞ ❛s biasθ θ̂ = E θ̂ − θ✳ L2 ✲❧♦ss n ≥ 3❀ ❜✉t ✐♥❛❞♠✐ss✐❜❧❡ ❢♦r ! n ≥ 3✿  R θ̂ST , θ < n = R θ̂, θ , ∀θ ∈ Θ ❞♦♠✐♥❛t❡s t❤❡ ✏♥❛t✉r❛❧✑ ❡st✐♠❛t♦r ✭❜✉t ♥♦t ♦♥ t❤❡ r❛♥❞♦♠ ✈❛❧✉❡ ♦❢ θ❀ n = 1, 2✱ ▼♦♠❡♥t ❊st✐♠❛t✐♦♥ ❚❤❡ ❡①♣❡❝t❛t✐♦♥ ✐s ♦✈❡r t❤❡ ❞✐str✐❜✉t✐♦♥ ♦❢ MSE = Eθ •  i h  R θ; δ̃ = Eθ L δ̃ (X) , θ  ❋♦r t❤❡ ✐s ❛❞♠✐ss✐❜❧❡ ❢♦r θ̂ST = X · ✐s t❤❡ ❡①♣❡❝t❛t✐♦♥ ♦❢ t❤❡ ❧♦ss✿ ❞❡❝✐s✐♦♥ r✉❧❡ ♦❢ •   L2 ✲❧♦ss✱ L θ̂, θ = θ̂ − θ  ✕ L1 ✲❧♦ss✱ L θ̂, θ = θ̂ − θ     ✕ ❩❡r♦✲♦♥❡✲❧♦ss✱ L θ̂, θ = I θ̂ 6= θ ❚❤❡ ✐s ✇❤❡♥ t❤❡ tr✉❡ θ̂ = X ❙t❡✐♥✬s ❡st✐♠❛t♦r✱ t❤❡r❡❢♦r❡✱ t❤❡ ❧♦ss ✐s ❛ r❛♥❞♦♠ ✈❛r✐❛❜❧❡✳ ❈♦♠♠♦♥ ❧♦ss ❢✉♥❝t✐♦♥s✿  • θ̂ ❚❤❡ ❡st✐♠❛t♦r  θ̂i − θ 2  Θ = Rn ✳ • ✳ ❲❛r♥✐♥❣s✿ ✕ ✕ ❚❤❡r❡ ❛r❡ ❝❛s❡s ✇❤❡r❡ ▼▲❊ ❞♦❡s ♥♦t ❡①✐st ✭✐✳❡✳✱ t❤❡r❡ ✐s ♥♦ ♠❛①✐♠✐③❡r✮✳ ❚❤❡r❡ ❛r❡ ♦t❤❡r ❝❛s❡s ✇❤❡r❡ ▼▲❊ ✐s ♥♦t ✉♥✐q✉❡✳ ■♥ r❡❣✉❧❛r ♠♦❞❡❧s✱ t❤❡ ▼▲❊ ✐s ❢✉♥❝t✐♦♥ ♦❢ ❛ s✉✣❝✐❡♥t st❛t✐st✐❝ LX (θ) = g (T (X) , θ) · h (X) ✸ f (z) = − log (z)✮✳ T (X)✱ s✐♥❝❡ ❇❛②❡s✐❛♥ ❊st✐♠❛t✐♦♥ ❋✐s❤❡r✬s ■♥❢♦r♠❛t✐♦♥ • ❇❛②❡s r✐s❦ ✭♥✉♠❜❡r✱ ♥♦t ❛ ❢✉♥❝t✐♦♥ ♦❢ θ✮ ❘❡❣✉❧❛r✐t② ❈♦♥❞✐t✐♦♥s ✭❢♦r ❝♦♥t✐♥✉♦✉s ❝❛s❡✮ • Θ ⊆ R ✐s ❛♥ ♦♣❡♥ s❡t✳ • ❇❛②❡s ❡st✐♠❛t♦rs ❛r❡ ❛❞♠✐ss✐❜❧❡ • ❚❤❡ s✉♣♣♦rt ♦❢ Pθ ❞♦❡s ♥♦t ❞❡♣❡♥❞ ♦♥ θ✳ ❊♠♣✐r✐❝❛❧ ❇❛②❡s • ❋♦r ❡✈❡r② x ❛♥❞ θ ∈ Θ✱ ∂ ∂θ Pθ (x) ❡①✐sts ❛♥❞ ✜♥✐t❡✳ • ❋♦r ❡✈❡r② st❛t✐st✐❝ T ✇✐t❤ ✜♥✐t❡ ❡①♣❡❝t❛t✐♦♥✱ Z Z ∂ ∂ T (x) Pθ (x) dx = T (x) Pθ (x) dx ∂θ ∂θ ❚❇❉ ▼✐♥✐♠❛① ❊st✐♠❛t♦r ❉❡✜♥✐t✐♦♥ ✫ Pr♦♣❡rt✐❡s ❚❇❉ • ❚❤❡ ❋✐s❤❡r✬s ■♥❢♦r♠❛t✐♦♥✱ I (θ)✱ ✐s ❛ ❢✉♥❝t✐♦♥ ♦❢ θ t❤❛t ✐s ❞❡✜♥❡❞ ✇✐t❤ r❡s♣❡❝t t♦ ❛ st❛t✐st✐❝❛❧ ♠♦❞❡❧ X ∼ Pθ ❜② " 2 # ∂ I (θ) = Eθ log Pθ (X) ∂θ ❊q✉❛❧✐③❡r ✴ ❊q✉❛❧✐③✐♥❣ ❘✉❧❡ ❚❇❉ • ■♥ ❢❛❝t✱ ✐t ❤♦❧❞s t❤❛t ❯▼❱❯ ❊st✐♠❛t♦r Eθ ❆♥ ✉♥❜✐❛s❡❞ ❡st✐♠❛t♦r T ⋆ ✐s ❝❛❧❧❡❞ ❯▼❱❯❊ ✭❯♥✐❢♦r♠ ▼✐♥✐♠❛❧ ❱❛r✐❛♥❝❡ ❯♥❜✐❛s❡❞ ❊st✐♠❛t♦r✮ ✐❢ Varθ (T ⋆ ) ≤ Varθ (T )✱ ∀θ ∈ Θ ❢♦r ❡✈❡r② ✉♥❜✐❛s❡❞ ❡st✐♠❛t♦r T ✳  ∂ log Pθ (X) = 0 ∂θ s♦ t❤❡ ✐♥❢♦r♠❛t✐♦♥ ❝❛♥ ❛❧s♦ ❜❡ ✇r✐tt❡♥ ❛s I (θ) = Var ❘❛♦✲❇❧❛❝❦✇❡❧❧ ❚❤❡♦r❡♠  ∂ log Pθ (X) ∂θ  • ■♥ ❛❞❞✐t✐♦♥ ✐t ❝❛♥ ❜❡ ❝♦♠♣✉t❡❞ ✉s✐♥❣ t❤❡ s❡❝♦♥❞ ❞❡r✐✈❛t✐✈❡✱ ❜②   2 ∂ log Pθ (X) I (θ) = −Eθ ∂θ2 ❙✉♣♣♦s❡ t❤❛t T (X) ✐s ❛ s✉✣❝✐❡♥t st❛t✐st✐❝ ❛♥❞ Eθ |S (X)| < ∞ ❢♦r ❛❧❧ θ ∈ Θ✳ ❉❡✜♥❡ T ⋆ (X) = E [S (X) |T (X) ]✱ t❤❡♥ h i h i 2 2 Eθ (T ⋆ (X) − θ) ≤ Eθ (S (X) − θ) ,  ∀θ ∈ Θ • ❲❤❡♥ θ ∈ Rd ✱ t❤❡ ✐♥❢♦r♠❛t✐♦♥ ✭♠❛tr✐①✮ ✐s ❞❡✜♥❡❞ ❜② h i T I (θ) = Eθ ∇θ log Pθ (X) · (∇θ log Pθ (X)) ∈ Rd×d ✇✐t❤ str✐❝t ✐♥❡q✉❛❧✐t② ✇❤❡♥ Varθ [S (X)] < ∞ ❛♥❞ T (X) 6= S (X) ❛✳s✳ ⋆ ▲❡❤♠❛♥♥✲❙❝❤❡✛❡ ❚❤❡♦r❡♠ • ❲❤❡♥ X1 , . . . , Xn ∼ Pθ ✱ t❤❡ ✐♥❢♦r♠❛t✐♦♥ ♦❢ X = (X1 , . . . , Xn ) ✐s ✐✳✐✳❞ ■❢ T (X) ✐s ❛ ❝♦♠♣❧❡t❡ s✉✣❝✐❡♥t st❛t✐st✐❝ ❛♥❞ S (X) ✐s ❛♥ ✉♥❜✐❛s❡❞ ❡st✐♠❛t❡ ♦❢ θ ✇✐t❤ ✜♥✐t❡ ✈❛r✐❛♥❝❡✱ t❤❡♥ T ⋆ (X) = E [S (X) |T (X) ] ✐s ❯▼❱❯ ❡st✐♠❛t♦r✳ In (θ) = n · I (θ) ✹ ❈r❛♠❡r✲❘❛♦ ▲♦✇❡r ❇♦✉♥❞ ✭❈❘▲❇✮ ❍♦❞❣❡s ❙✉♣❡r ❊✣❝✐❡♥❝② ❚❇❉ • ❋♦r ❛ st❛t✐st✐❝ T (X) ❞❡✜♥❡ ψ (θ) = Eθ [T (X)] • ❉❡♥♦t❡ t❤❡ ❋✐s❤❡r✬s ■♥❢♦r♠❛t✐♦♥ ♦❢ X ❜② I (θ) ❍②♣♦t❤❡s✐s ❚❡st✐♥❣ ❛♥❞ ❈♦♥✜❞❡♥❝❡ ■♥t❡r✈❛❧s • ■❢ Eθ |T (X)| < ∞, ∀θ✱ t❤❡♥ 2 Varθ [T (X)] ≥ (ψ ′ (θ)) ≡ ❈❘▲❇ I (θ) Pr❡❧✐♠✐♥❛r② ◆♦t❡s✿ ▲❡t X ∼ Pθ ✱ ✇✐t❤ θ ∈ Θ✳ • ❲❤❡♥ T (X) ✐s ❛♥ ✉♥❜✐❛s❡❞ ❡st✐♠❛t♦r ♦❢ θ✱ t❤❡♥ ψ ′ (θ) = 1✳ • ❚❤❡ r❛♥❞♦♠ ✐♥t❡r✈❛❧ (L (X) , U (X)) ✐s ❝❛❧❧❡❞ ❛ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧ (1 − α) ✐❢ Pθ (L (X) ≤ θ ≤ U (X)) ≥ 1 − α • ■❢ s✉❝❤ ❛♥ ❡st✐♠❛t♦r ❛❝❤✐❡✈❡s Varθ [T (X)] = 1 /I (θ) t❤❡♥ ✐t ✐s ❯▼❱❯❊✳ • ●❡♥❡r❛❧ ❢♦r♠ ♦❢ • ❲❤❡♥ T = Tn ❞❡♣❡♥❞s ♦♥ X1 , . . . , Xn ∼ Pθ ❛♥❞ I (θ) ✐s t❤❡ ✐♥❢♦r♠❛t✐♦♥ ♦❢ ❛ s✐♥❣❧❡ Xi ✱ t❤❡♥ t❤❡ ❞❡♥♦♠✐♥❛t♦r ♦❢ t❤❡ ❈❘▲❇ ✐s n · I (θ)✳ ✐✳✐✳❞ ✇❤❡r❡ Θ0 , Θ1 ❛r❡ ❞✐s❥♦✐♥t s✉❜s❡ts ♦❢ Θ✳ • ❍②♣♦t❤❡s✐s H0 ✐s ❝❛❧❧❡❞ ❈♦♥s✐st❡♥❝② n o∞ ✐s ❝❛❧❧❡❞ ❝♦♥s✐st❡♥t ✐❢ ✐t s❛t✐s✜❡s θ̂n −→ θ✳ n o∞ ✐s ❝❛❧❧❡❞ ❡✣❝✐❡♥t ✐❢ ✐t s❛t✐s✜❡s n=1 n=1 s✐♠♣❧❡ ✇❤❡♥ Θ0 = {θ0 } ✭♦t❤❡r✇✐s❡ ✐t ✐s ❝♦♠♣♦s✐t❡✮✳ • ❚❡st ✴ ❉❡❝✐s✐♦♥ ❘✉❧❡✿ δ (X) ∈ {0, 1}✱ ✇❤❡r❡ δ = 1 ♠❡❛♥s H0 ✐s r❡❥❡❝t❡❞✳ P • ❚②♣❡✲■ ❡rr♦r✿ ✇❤❡♥ H0 ✐s ✇r♦♥❣❧② r❡❥❡❝t❡❞✳ • ❚②♣❡✲■■ ❡rr♦r✿ ✇❤❡♥ H0 ✐s ✇r♦♥❣❧② ❛❝❝❡♣t❡❞✳ ❊✣❝✐❡♥❝② ❆ s❡q✉❡♥❝❡ ♦❢ ❡st✐♠❛t♦rs θ̂n ❤②♣♦t❤❡s✐s t❡st✐♥❣ ✐s H0 : θ ∈ Θ0 ✭t❤❡ ♥✉❧❧✮ ✈❡rs✉s H1 : θ ∈ Θ1 ✭t❤❡ ❛❧t❡r♥❛t✐✈❡✮ ❆s②♠♣t♦t✐❝❛❧ Pr♦♣❡rt✐❡s ❆ s❡q✉❡♥❝❡ ♦❢ ❡st✐♠❛t♦rs θ̂n ✐♥t❡r✈❛❧ ❢♦r θ ❛t ❙✐❣♥✐✜❝❛♥❝❡ ❛♥❞ P♦✇❡r ♦❢ ❚❡sts • ❆ t❡st δ (X) ✐s ♦❢  √  D n θ̂n − θ −→ N (0, 1 /I (θ) ) s✐❣♥✐✜❝❛♥❝❡ ❧❡✈❡❧ α ✐❢ sup Eθ [δ (X)] ≤ α θ∈Θ0 ✇❤❡r❡ I (θ) ✐s t❤❡ ✐♥❢♦r♠❛t✐♦♥ ♦❢ ❛ s✐♥❣❧❡ ♦❜s❡r✈❛t✐♦♥ X ∼ Pθ ✳ • ❚❤❡ • ❙✉❝❤ ❛♥ ❡st✐♠❛t♦r ✐s ❛s②♠♣t♦t✐❝❛❧❧② ✉♥❜✐❛s❡❞ ❛♥❞ ❛❝❤✐❡✈❡s t❤❡ ❈❘▲❇ ✭♥♦t✐❝❡ t❤❡ ✈❛r✐❛♥❝❡ ♦❢ θ̂n ✐s 1 /nI (θ) ✮✳ ♣♦✇❡r ❢✉♥❝t✐♦♥ ♦❢ δ (X) ✐s ❞❡✜♥❡❞ ❢♦r ❡✈❡r② θ ∈ Θ ❛s β (θ) = Eθ [δ (X)] • ❲❤❡♥ θ ∈ Θ1 ✱ t❤❡♥ β (θ) ✐s ❝❛❧❧❡❞ t❤❡ ♣♦✇❡r ♦❢ δ (X)✿ t❤❡ ♣r♦❜❛❜✐❧✐t② ♦❢ ❝♦rr❡❝t❧② ❛❝❝❡♣t✐♥❣ H1 ✱ t❤❛t ✐s 1 − Pr {t②♣❡✲■■ ❡rr♦r}✳ ▼▲❊ ♣r♦♣❡rt✐❡s • ❲❤❡♥ θ ∈ Θ0 ✱ t❤❡♥ β (θ) ✐s t❤❡ ♣r♦❜❛❜✐❧✐t② ♦❢ t②♣❡✲■ ❡rr♦r✳ ❚❇❉ ✺ ❈♦♥✜❞❡♥❝❡ ■♥t❡r✈❛❧ ❢♦r ❇✐♥♦♠✐❛❧ ❚r✐❛❧s ✕ ❚❇❉ ●❡♥❡r❛❧✐③❡❞ ▲✐❦❡❧✐❤♦♦❞ ❘❛t✐♦ ❚❡sts ✭●▲❘❚✮ • ❯s✐♥❣ ❡st✐♠❛t❡❞ ✈❛r✐❛♥❝❡ ❚❇❉ • ❯s✐♥❣ t❤❡ ❉❡❧t❛ ▼❡t❤♦❞ ❲✐❧❦✬s ❚❤❡♦r❡♠ • ❇② s♦❧✈✐♥❣ q✉❛❞r❛t✐❝ ❡q✉❛t✐♦♥ ◆❡②♠❛♥✲P❡❛rs♦♥ ❚❇❉ ❆♣♣r♦❛❝❤✿ ❝♦♥tr♦❧ t❤❡ t②♣❡✲■ ❡rr♦r ✇❤✐❧❡ ♠✐♥✐♠✐③✐♥❣ t❤❡ t②♣❡✲■■ ❡rr♦r✳ ●♦♦❞♥❡ss ♦❢ ❋✐t ▲✐❦❡❧✐❤♦♦❞ ❘❛t✐♦ ❋♦r s✐♠♣❧❡ ❤②♣♦t❤❡s✐s t❡st✐♥❣✱ H0 : θ = θ0 ✈s✳ H1 : θ = θ1 ✇✐t❤ θ1 > θ0 ✿ • ❚❤❡ ❧✐❦❡❧✐❤♦♦❞ r❛t✐♦ T (X) = • ❚❤❡ ❧✐❦❡❧✐❤♦♦❞ ❚❇❉ ✐s ❞❡✜♥❡❞ ❛s t❤❡ st❛t✐st✐❝ r❛t✐♦ t❡st Pθ1 (X) Pθ0 (X) ❙♦♠❡ ❯s❡❢✉❧ Pr♦♣❡rt✐❡s ✕ ❚❇❉ ✐s ❞❡✜♥❡❞ ❜② δC (X) = I (T (X) > C)✱ ✇✐t❤ C > 0✳ • ❈❤✐✲❙q✉❛r❡❞ ◆❡②♠❛♥✲P❡❛rs♦♥✬s ▲❡♠♠❛ • t✲❞✐str✐❜✉t✐♦♥ ❚❤❡ ❧✐❦❡❧✐❤♦♦❞ r❛t✐♦ t❡st✱ δC ✱ ✐s ♦♣t✐♠❛❧ ✐♥ t❤❡ s❡♥s❡ t❤❛t ❡✈❡r② t❡st δ ✇✐t❤ t❤❡ s❛♠❡ s✐❣♥✐✜❝❛♥❝❡ ❧❡✈❡❧ ❤❛s ❛ s♠❛❧❧❡r ♣♦✇❡r✱ ✐✳❡✿ ∀δ : • ❋✲❞✐str✐❜✉t✐♦♥ Eθ0 [δ (X)] ≤ Eθ0 [δC (X)] ⇒ Eθ1 [δ (X)] ≤ Eθ1 [δC (X)] ❉✐str✐❜✉t✐♦♥s ❯♥✐❢♦r♠❧② ▼♦st P♦✇❡r❢✉❧ ✭❯▼P✮ ❚❡sts ✕ ❚❇❉ • ❉❡✜♥✐t✐♦♥ • ❙✐♥❣❧❡✲s✐❞❡❞ ❤②♣♦t❤❡s✐s t❡st✐♥❣ ❇❡r♥♦✉❧❧✐ • ❯♥✐q✉❡♥❡ss X ∼ Ber (θ) ✐s t❤❡ r❡s✉❧t ♦❢ ❛ ❇❡r♥♦✉❧❧✐ tr✐❛❧✿ s✉❝❝❡ss (X = 1) ✇✐t❤ ♣r♦❜❛❜✐❧✐t② θ ♦r ❢❛✐❧✉r❡ (X = 0) ✇✐t❤ ♣r♦❜❛❜✐❧✐t② 1 − θ✱ ✇❤❡r❡ θ ∈ [0, 1]✳ • ❑❛r❧✐♥✲❘✉❜✐♥ ❚❤❡♦r❡♠ ❉✉❛❧✐t② Pθ (x) = θx (1 − θ) E (X) = θ ❚❇❉ 1−x · I (x ∈ {0, 1}) Var (X) = θ (1 − θ) ♣✲❱❛❧✉❡ i.i.d • ■❢ X1 , . . . , Xn ∼ Ber (θ) ❛♥❞ T = ❚❇❉ ✻ Pn i=1 Xi t❤❡♥ T ∼ Bin (n, θ)✳ ❇✐♥♦♠✐❛❧  s✐③❡ N ✇✐t❤ ❡①❛❝t❧② K = N θ ❞❡❢❡❝t✐✈❡ ✐t❡♠s✱ ✇❤❡r❡ θ ∈ 0, N1 , N2 , . . . , 1 ✳ X ∼ Bin (n, θ) ✐s t❤❡ ♥✉♠❜❡r ♦❢ s✉❝❝❡ss❡s ♦✉t ♦❢ n ✐♥❞❡♣❡♥❞❡♥t ❇❡r♥♦✉❧❧✐ tr✐❛❧s ✇✐t❤ s✉❝❝❡ss ♣r♦❜❛❜✐❧✐t② θ✳   n n−x · θx (1 − θ) · I (x ∈ {0, 1, . . . , n}) Pθ (x) = x Pθ (x) = Nθ x E (X) = nθ
· N (1−θ)
n−x N n Var (X) = nθ (1 − θ) E (X) = nθ Var (X) = nθ (1 − θ)
· I (max {0, n − N (1 − θ)} ≤ x ≤ min {n, N θ}) N −n N −1 P♦✐ss♦♥ ▼✉❧t✐♥♦♠✐❛❧ X ∼ P oisson (θ)✳ ❲❤❡♥ r❛♥❞♦♠ ❡✈❡♥ts ♦❝❝✉r ✐♥❞❡♣❡♥❞❡♥t❧② ❛♥❞ ❛t ❛ ❝♦♥st❛♥t ♠❡❛♥ r❛t❡ θ✱ t❤❡♥ X ✐s t❤❡ ♥✉♠❜❡r ♦❢ ❡✈❡♥ts ♣❡r ✉♥✐t ♦❢ t✐♠❡✳ ●❡♥❡r❛❧✐③❛t✐♦♥ ♦❢ ❇✐♥♦♠✐❛❧✿ θx e−θ · I (x ∈ {0, 1, 2, . . .}) x! E (X) = θ • ❊❛❝❤ tr✐❛❧ ❝❛♥ ❤❛✈❡ K ♣♦ss✐❜❧❡ ♦✉t❝♦♠❡s (v1 , . . . , vK ) ✇✐t❤ ♣r♦❜❛❜✐❧✐t✐❡s PK (θ1 , . . . , θK ) ≡ θ✱ ✇❤❡r❡ j=1 θj = 1 ❛♥❞ θj ∈ [0, 1] , j = 1, . . . , K ✳ Pθ (x) = • ❋♦r n ✐♥❞❡♣❡♥❞❡♥t tr✐❛❧s✱ Nj ❝♦✉♥ts t❤❡ ♥✉♠❜❡r ♦❢ vj ✲♦✉t❝♦♠❡s✳   K K X Y n! nj nj = n θ ·I Pθ (n1 , . . . , nK ) = n1 ! · . . . · nK ! j=1 j j=1 Var (X) = θ ◆♦r♠❛❧ ✭●❛✉ss✐❛♥✮
X ∼ N µ, σ 2 ❤❛s ❛ ♣r♦❜❛❜✐❧✐t② ❞❡♥s✐t② ❢✉♥❝t✐♦♥ E (Nj ) = nθj Var (Nj ) = nθj (1 − θj ) (x − µ) exp − f (x) = √ 2 2σ 2 2πσ 1 Cov (Ni , Nj ) = −nθi θj , i 6= j ●❡♦♠❡tr✐❝ 2 ! • ❋♦r ●❛✉ss✐❛♥ r❛♥❞♦♠ ✈❡❝t♦r✱ X ∼ N (µ, Σ)✱ ✇✐t❤ ❞✐♠❡♥s✐♦♥ K ✿   T exp − 21 (x − µ) Σ−1 (x − µ) q f (x) = K (2π) det (Σ) X ∼ G (θ) ✐s t❤❡ ♥✉♠❜❡r ♦❢ ❢❛✐❧✉r❡s ❜❡❢♦r❡ t❤❡ ✜rst s✉❝❝❡ss✱ ✐♥ ❛ s❡q✉❡♥❝❡ ♦❢ ✐♥❞❡♣❡♥❞❡♥t ❇❡r♥♦✉❧❧✐ tr✐❛❧s ✇✐t❤ s✉❝❝❡ss ♣r♦❜❛❜✐❧✐t② θ✱ x Pθ (x) = (1 − θ) θ · I (x ∈ {0, 1, 2, . . .}) 1−θ E (X) = θ 1−θ Var (X) = θ2
• ❊rr♦r ❢✉♥❝t✐♦♥ ✕ ❞❡✜♥❡❞ ❢♦r Y ∼ N 0, 21 ✿ 2 erf (y) = Pr {−y ≤ Y ≤ y} = √ π ❍②♣❡r✲●❡♦♠❡tr✐❝ Z y 0
exp −t2 dt • ❆♣♣r♦①✐♠❛t❡ ♣r♦❜❛❜✐❧✐t✐❡s ❢♦r X ∼ N µ, σ ✿ X ∼ HG (N, N θ, n) ✐s t❤❡ ♥✉♠❜❡r ♦❢ ❞❡❢❡❝t✐✈❡ ✐t❡♠s ✐♥ r❛♥❞♦♠ s❛♠♣❧❡ ♦❢ s✐③❡ n✱ t❤❛t ✇❛s ❞r❛✇♥ ✇✐t❤♦✉t r❡♣❧❛❝❡♠❡♥t ✭✐✳❡✳✱ ♥♦t ✐♥❞❡♣❡♥❞❡t❧②✮ ❢r♦♠ ❛ ♣♦♣✉❧❛t✐♦♥ ♦❢ Pr {−a ≤ X − µ ≤ a} ✼ a=σ 68.27%
2 a = 2σ 95.45% a = 3σ 99.73% ❙t❛♥❞❛r❞ ◆♦r♠❛❧✿ ❯♥✐❢♦r♠ Z ∼ N (0, 1) X ∼ N µ, σ
2 • ❈❛♥ ❜❡ ♦❜t❛✐♥❡❞ ❢r♦♠ • ❈✉♠✉❧❛t✐✈❡ ❞✐str✐❜✉t✐♦♥ ❢✉♥❝t✐♦♥ ✭❈❉❋✮✿ 1 Φ (z) = FZ (z) = √ 2π • ◗✉❛♥t✐❧❡ ❢✉♥❝t✐♦♥✱ Z ❜② Z = (X − µ)/ σ ❈♦♥t✐♥✉♦✉s ✉♥✐❢♦r♠ ❞✐str✐❜✉t✐♦♥✱ ❉✐s❝r❡t❡ ✉♥✐❢♦r♠ ❞✐str✐❜✉t✐♦♥✱ p ∈ (0, 1)✿ α = 0.05✱ ✉s❡❢✉❧ q✉❛♥t✐❧❡s ❛r❡✿ z1− α2 ≃ 1.96 ❛♥❞ z1−α ≃ 1.65✳ X ∼ ▲♦❣✲◆♦r♠❛❧ µ, σ 2
t❤❡♥ Y = log X ∼ N µ, σ 2 2
(log x − µ) f (x) = √ exp − 2σ 2 x 2πσ 2   σ2 E (X) = exp µ + 2


2 Var (X) = exp σ − 1 exp 2µ + σ 2 1 ❛♥❞ ! ❇❡t❛ ✐s ❛ t✇♦✲♣❛r❛♠❡t❡r ❢❛♠✐❧② ♦❢ ❞✐str✐❜✉t✐♦♥s ♦✈❡r X = exp (Y )✳ Beta (a, b)✱ X ∼ Gamma (a, b)✱ ✇✐t❤ a>0 ✭s❤❛♣❡✮ ❛♥❞ b>0 ❛♥❞ b > 0✱ x ∈ [0, 1]✳ ❲❤❡♥ X ∼ 1 b−1 xa−1 (1 − x) · I (0 < x < 1) β (a, b) a E (X) = a+b ab Var (X) = 2 (a + b) (a + b + 1) · I (x > 0) ❊①♣♦♥❡♥t✐❛❧ X ∼ Exp (θ)✱ ✇✐t❤ a>0 f (x) = ●❛♠♠❛ ■❢ θ ∈ N+ ✿ ❇❡t❛ ▲♦❣✲◆♦r♠❛❧ ■❢ ✇✐t❤ 1 · I (x ∈ {1, . . . , θ}) θ θ−1 E (X) = 2 θ2 − 1 Var (X) = 12 Pr {|Z| ≤ zp } = 2p − 1 ❋♦r t❤❡ ❝♦♠♠♦♥ X ∼ U {1, θ}✱ Pθ (x) = Pr {Z ≤ zp } = p • θ ∈ R++ ✿ 1 1 · I (0 ≤ x ≤ θ) = · I (x ≥ 0) · I (x ≤ θ) θ θ 1 E (X) = θ 2 1 2 θ Var (X) = 12   
z 1 1 + erf √ exp −t2 /2 dt = 2 2 −∞ ❢♦r ✇✐t❤ fθ (x) = z Φ−1 (p) ≡ zp ✱ X ∼ U (0, θ)✱ ❛ s♣❡❝✐❛❧ ❝❛s❡ ♦❢ t❤❡ Gamma (a, b) ❞✐str✐❜✉t✐♦♥ ✭a =1 ❛♥❞ b = θ✮✱ fθ (x) = θe−θx · I (x > 0) 1 E (X) = θ 1 Var (X) = 2 θ ✭r❛t❡✮✱ ba a−1 −bx x e · I (x > 0) Γ (a) a E (X) = b a Var (X) = 2 b f (x) = • ✽ ❲❤❡♥ r❛♥❞♦♠ ❡✈❡♥ts ♦❝❝✉r ✐♥❞❡♣❡♥❞❡♥t❧② ❛♥❞ ❛t ❛ ❝♦♥st❛♥t ♠❡❛♥ r❛t❡ θ ✱ t❤❡♥ X ✐s t❤❡ t✐♠❡ ❡❧❛♣s❡❞ ❜❡t✇❡❡♥ t✇♦ ❡✈❡♥ts✳ ❈❤✐✲❙q✉❛r❡❞ D ✷✳ Xn −→ X ✭❝♦♥✈❡r❣❡♥❝❡ ✐♥ ❞✐str✐❜✉t✐♦♥✮ ✐❢ i.i.d ■❢ Z1 , . . . , Zk ∼ N (0, 1) t❤❡♥ X = ❞❡❣r❡❡s✲♦❢✲❢r❡❡❞♦♠✱ X ∼ χ2k ✳ Pk i=1 Zk2 ❤❛s ❛ ❝❤✐✲sq✉❛r❡❞ ❞✐str✐❜✉t✐♦♥ ✇✐t❤ k lim Pr {Xn ≤ t} = Pr {X ≤ t} n→∞ ❢♦r ❛❧❧ ❝♦♥t✐♥✉♦✉s ♣♦✐♥ts t ♦❢ FX (t) = Pr {X ≤ t} ✭♥♦t❡ t❤❛t FX ✐s ❞❡✜♥❡❞ ♦✈❡r R ❡✈❡♥ ✇❤❡♥ X ✐s ❛ ❞✐s❝r❡t❡ r❛♥❞♦♠ ✈❛r✐❛❜❧❡✮✳ 1 x(k/2)−1 e−x/2 · I (x > 0) 2k/2 · Γ (k/2) E (X) = k f (x) = ✸✳ Xn −→ X ✭♥❛♠❡❧②✱ Xn ❝♦♥✈❡r❣❡s ❛❧♠♦st✲s✉r❡❧② t♦ X ✮ ✐❢ ❛✳s✳ Var (X) = 2k Pr • χ2k ✐s ❛ s♣❡❝✐❛❧ ❝❛s❡ ♦❢ t❤❡ Gamma (a, b) ❞✐str✐❜✉t✐♦♥ ✇✐t❤ a = 21 k ❛♥❞ b = 21 ✳
2
Pk i.i.d • ■❢ Y1 , . . . , Yk ∼ N µ, σ 2 t❤❡♥ X = σ12 i=1 Yi − Ȳk ∼ χ2k−1 ✳ ❚❤❡♦r❡♠s ✫ Pr♦♣❡rt✐❡s ❛✳s✳ • Xn −→ X t✲❉✐str✐❜✉t✐♦♥ i.i.d • ■❢ Y1 , . . . , Yk ∼ N µ, σ
2 Z ❛♥❞ σ̂ 2 = X= √ 1 k−1 Pk i=1 ⇒ o =1 D Xn −→ X P • ❲❤❡♥ E (X) ✐s ✜♥✐t❡✱ Xn −→ X ❞♦❡s ◆❖❚ ✐♠♣❧② E (Xn ) ✐s ✜♥✐t❡✳ ❛✳s✳ Yi − Ȳk
n→∞ D ∼ tk W/k P Xn −→ X lim Xn = X • ■❢ Xn −→ C ❢♦r ❛ ❝♦♥st❛♥t C ✱ t❤❡♥ Xn −→ C ✳ ❲❤❡♥ Z ∼ N (0, 1) ❛♥❞ W ∼ χ2k ❛r❡ ✐♥❞❡♣❡♥❞❡♥t✱ t❤❡♥ X=p ⇒ n k Ȳk − µ ∼ tk−1 σ̂
2 ▲❛✇ ♦❢ ▲❛r❣❡ ◆✉♠❜❡rs t❤❡♥ ▲❡t X1 , . . . , Xn ❜❡ ✐✳✐✳❞ ✇✐t❤ ✜♥✐t❡ ♠❡❛♥ µ ❛♥❞ ❧❡t X̄n = P • X̄n −→ µ ✭❲❡❛❦ ▲❛✇ ♦❢ ▲❛r❣❡ ◆✉♠❜❡rs✮ 1 n Pn i=1 Xi ✱ t❤❡♥ • X̄n −→ µ ✭❙tr♦♥❣ ▲❛✇ ♦❢ ▲❛r❣❡ ◆✉♠❜❡rs✮ ❛✳s✳ ❋✲❉✐str✐❜✉t✐♦♥ ▼♦r❡ ❣❡♥❡r❛❧❧②✱ ✐❢ t❤❡ kth ✲♠♦♠❡♥t ♦❢ X ∼ Pθ ❡①✐sts✱ t❤❡♥ ■❢ W ∼ χ2k ❛♥❞ V ∼ χ2m ❛r❡ ✐♥❞❡♣❡♥❞❡♥t✱ t❤❡♥ X= W/k ∼ Fk,m V /m Pr♦❜❛❜✐❧✐t② • 1 n • 1 n Pn i=1 Pn i=1
P Xik −→ Eθ X k ✭❲❡❛❦ ▲❛✇ ♦❢ ▲❛r❣❡ ◆✉♠❜❡rs✮
a.s. Xik −→ Eθ X k ✭❙tr♦♥❣ ▲❛✇ ♦❢ ▲❛r❣❡ ◆✉♠❜❡rs✮ ❈❡♥tr❛❧ ▲✐♠✐t ❚❤❡♦r❡♠ ✭❈▲❚✮ ▲❡t X1 , . . . , Xn ❜❡ ✐✳✐✳❞ ✇✐t❤ ♠❡❛♥ µ ❛♥❞ ✈❛r✐❛♥❝❡ σ 2 s✉❝❤ t❤❛t 0 < σ 2 < ∞✱ t❤❡♥ ❆s②♠♣t♦t✐❝s √ ❈♦♥✈❡r❣❡♥❝❡ ❉❡✜♥✐t✐♦♥s P ✶✳ Xn −→ X ✭❝♦♥✈❡r❣❡♥❝❡ ✐♥ ♣r♦❜❛❜✐❧✐t②✮ ✐❢ lim Pr {|Xn − X| < ε} = 1, n→∞
D
n X̄n − µ −→ N 0, σ 2 ❈♦♥t✐♥✉♦✉s ▼❛♣♣✐♥❣ ❚❤❡♦r❡♠ ✭❈▼❚✮ P P ■❢ Xn −→ X ❛♥❞ g ✐s ❛ ❝♦♥t✐♥✉♦✉s ❢✉♥❝t✐♦♥✱ t❤❡♥ g (Xn ) −→ g (X)✳ ∀ε > 0 ✾ ❙❧✉ts❦②✬s ▲❡♠♠❛ D ❏❡♥s❡♥✬s ■♥❡q✉❛❧✐t② ■❢ ϕ ✐s ❛ ❝♦♥✈❡① ❢✉♥❝t✐♦♥ t❤❡♥ ϕ (E [X]) ≤ E [ϕ (X)]✳ P ■❢ Xn −→ X ❛♥❞ Yn −→ C ❢♦r ❛ ❝♦♥st❛♥t C ✱ t❤❡♥ ❇✐♥♦♠✐❛❧ ❈♦❡✣❝✐❡♥ts D Xn + Yn −→ X + C D ❚❤❡ ❉❡❧t❛ ▼❡t❤♦❞ ■❢ √   n! n = k! (n − k)! k Xn · Yn −→ X · C ●❛♠♠❛ ❋✉♥❝t✐♦♥ D n (Xn − C) −→ X ❛♥❞ g ❤❛s ❝♦♥t✐♥✉♦✉s ❞❡r✐✈❛t✐✈❡✱ t❤❡♥ √ D n (g (Xn ) − g (C)) −→ X · g ′ (C) ❋♦r ❛ ♥✉♠❜❡r z ✇✐t❤ ♣♦s✐t✐✈❡ r❡❛❧ ♣❛rt✱ t❤❡ ●❛♠♠❛ ❋✉♥❝t✐♦♥ ✐s ❞❡✜♥❡❞ ❜② Γ (z) = ❊❧❡♠❡♥t❛r② Pr♦♣❡rt✐❡s PX,Y (x, y) = PX|Y (x |y ) · PY (y) = PY |X (y |x ) · PX (x) i ❲❤❡♥ y > 0 ✐s ❛❧s♦ ❛♥ ✐♥t❡❣❡r✱ Γ (y) = (y − 1)!✳ Pr (A) = X i ❱❛r✐❛♥❝❡✿ Var (X) = E (X − EX) 2 i ❇❡t❛ ❋✉♥❝t✐♦♥ Pr (A |Bi ) · Pr (Bi ) ❋♦r r❡❛❧ ♥✉♠❜❡rs a > 0 ❛♥❞ b > 0✱ t❤❡ ❜❡t❛ ❢✉♥❝t✐♦♥ ✐s ❞❡✜♥❡❞ ❛s =E X 2
− (EX) 2 β (a, b) = ■❢ X ❛♥❞ Y ❛r❡ ✐♥❞❡♣❡♥❞❡♥t ❛♥❞ c ✐s ❛ ❝♦♥st❛♥t✱ t❤❡♥ 2 Var (cX) = c Var (X) 2 2 2 hu, vi ≤ hu, ui · hv, vi = kuk · kvk ❈❤❡❜②s❤❡✈✬s ■♥❡q✉❛❧✐t② • ❊q✉❛❧✐t② ❤♦❧❞s ✐❢✲❛♥❞✲♦♥❧②✲✐❢ ∃α ∈ R s✳t✳ u = αv ✳ ❲❤❡♥ E (X) = µ ✐s ✜♥✐t❡ ❛♥❞ Var (X) = σ 2 > 0 ✐s ✜♥✐t❡✱ t❤❡♥ ❢♦r ❛♥② ε > 0 n Γ (a) Γ (b) Γ (a + b) ❈❛✉❝❤②✲❙❝❤✇❛r③ ■♥❡q✉❛❧✐t② Var (X + Y ) = Var (X) + Var (Y ) 2 o Pr {|X − µ| ≥ ε} = Pr (X − µ) ≥ ε2 ≤ ▼❛r❦♦✈✬s ■♥❡q✉❛❧✐t② 0 Γ (y + 1) = y · Γ (y) Pr {Bi } = 1✿ h xz−1 e−x dx Γ (1) = 1 ▲❛✇ ♦❢ ❚♦t❛❧ Pr♦❜❛❜✐❧✐t② P ∞ ❛♥❞ ❤❛s t❤❡ ❢♦❧❧♦✇✐♥❣ r❡❝✉rr❡♥❝❡ r❡❧❛t✐♦♥ ❢♦r ♣♦s✐t✐✈❡ r❡❛❧ ♥✉♠❜❡rs y ❇❛②❡s ❘✉❧❡ ❆ss✉♠✐♥❣ Z • ❚r✐❛♥❣❧❡ ✐♥❡q✉❛❧✐t② r❡s✉❧ts ❢r♦♠ ✐t✿ ku + vk ≤ kuk + kvk✳ 2 σ ε2 ■❢ X ✐s ❛ ♥♦♥✲♥❡❣❛t✐✈❡ r❛♥❞♦♠ ✈❛r✐❛❜❧❡ ❛♥❞ a > 0✱ t❤❡♥ Pr {X ≥ a} ≤ E (X) ✶✵