Mathematical Statistics (MA212M) : Lecture Slides

Mathematical Statistics (MA212M)
Lecture Slides
Lecture 25
Sufficient Statistics
Def: A statistic T = T (X ) is called a sufficient statistic for unknown
parameter θ is the conditional distribution of X given T = t does
not include θ for all t in the support of T .
i.i.d.
Pn 117: X1 , X2 , . . . , Xn ∼ Bernoulli(p), p ∈ (0, 1). Take
Example
T = i=1 Xi . We know that T ∼ Bin(n, p). Now for
t = 0, 1, . . . , n,
P (X1 = x1 , . . . , Xn = xn , T = t)
P (X1 = x1 , . . . , Xn = xn |T = t) =
 Qn xT = t1−x
 i=1 p i (1−p) i if Pn x = t
(
P(X1 =x1 , ..., Xn =xn ) P n
T =t if i=1 xi = t (nt)pt (1−p)n−t i=1 i
= =
0 otherwise 0 otherwise
( 1 Pn
n if i=1 xi = t
= (t )
0 otherwise,
Pn
which does not include p. Hence T = i=1 Xi is a sufficient statistics for p.
Nayman-Fisher Factorization Theorem
Theorem: (Without proof) Let X1 , . . . , Xn be RS with JPMF/JPDF

fX (x, θ), θ ∈ Θ. Then T = T (X1 , . . . , Xn ) is sufficient for θ if and
only if
fX (x, θ) = h(x)gθ (T (x)) ,
where h(x) does not involve θ, gθ (·) depends on θ and x only

through T (x).
Examples
i.i.d.
Example 118: Let X1 , X2 , . . . , Xn ∼ P(λ), λ > 0. Here the
likelihood function is
e −nλ λnx
L(λ, x) = Qn = h(x)gλ (T (x)) ,
i=1 (xi !)
−1
where h(x) = [ ni=1 (xi !)] , gλ (t) = e −nλ λnt , and T (x) = x. This
Q
shows that T = X is a sufficient statistic for λ.
i.i.d. 2
Example 119: Let X1 , X2 , . . . , Xn ∼ N(µ, σP ), µ ∈ R and σ > 0.
A sufficient statistic for (µ, σ ) is ( i=1 Xi , ni=1 Xi2 ).
2 n
P
Examples
i.i.d.
Example 120: Let X1 , X2 , . . . , Xn ∼ U(0, θ), θ > 0. Here the
likelihood function is
1
L(θ, x) = I(0, ∞) (x(1) )I(0, θ) (x(n) )
θn
= h(x)gθ (T (x)) ,
where h(x) = I(0, ∞) (x(1) ), gθ (t) = θ1n I(0, θ) (t) and T (x) = x(n) . Hence
T = X(n) is sufficient statistic for θ.
i.i.d.
Example 121: Let X1 , X2 , . . . , Xn ∼ U(θ − 1/2, θ + 1/2), θ ∈ R.
Here the likelihood function is
L(θ, x) = h(x)gθ (T (x)),
where h(x) = 1, gθ (t) = I(θ−1/2, θ+1/2) (x(1) )I(θ−1/2, θ+1/2) (x(n) ) and
T (x) = (x(1) , x(n) ). Hence T = X(1) , X(n) is sufficient for θ.
Remarks
Note that we will be able to use the definition of sufficient

statistic if we can guess one. However the theorem gives
necessary and sufficient conditions, which can be used to find a
sufficient statistic.
Note that the RS is always sufficient for unknown parameters.
However, most of the cases we will not talk about this trivial
sufficient statistic, as it does not provide any dimension
reduction.
Sometimes the RS is only sufficient statistic. For example
consider a Cauchy (0, θ) distribution. (Proof is not need, it is
only for information.)
Remarks
If T is sufficient for θ, then for any one-to-one function of T is

also sufficient for θ. (Can be proved
easily using Factorization
theorem.) For example X , S 2 is sufficient for parameters of
1
Pn 2
N(µ, σ 2 ), where S 2 = n−1 i=1 Xi − X .
Any function of sufficient statistic is not sufficient. (If so, then
any statistic will be sufficient.)
One-dimensional parameter may have multidimensional sufficient
statistic. (Consider the last example.)
T and θ are of same dimension and T is sufficient for θ do not
imply that the jth component of T is sufficient for the jth
component of θ. It only tells that T is jointly sufficient for θ.
Relation between MLE and Sufficient Statistics
Theorem: Let T be a sufficient statistics for θ. If a unique MLE exist

for θ, it is a function of T . If MLE of θ exist but is not unique, then
one can find a MLE that is a function of T .
Proof: Easy using factorization theorem.
i.i.d.
Example 122: Let X1 , X2 , . . . , Xn ∼ U(0, θ), θ > 0. We know that
the MLE is unique and X(n) , which is also sufficient.
i.i.d.
Example 123: Let X1 , X2 , . . . , Xn ∼ U(θ − 1/2, θ + 1/2), θ ∈ R.
Here a sufficient statistic is T = (X(1) , X(n) ). Also
MLE is not unique
1 1
and any point in the interval X(n) − 2 , X(1) + 2 is a MLE of θ.
Hence 21 X(1) + X(n) is a MLE and it is also a function of T . On
the other hand Q = sin X1 X(n) − 12 + (1 − sin X1 ) X(1) − 21 is a
MLE but not a function of T only.

Mathematical Statistics (MA212M) : Lecture Slides

Uploaded by

Copyright:

Available Formats

Mathematical Statistics (MA212M) : Lecture Slides

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Mathematical Statistics (MA212M) : Lecture Slides

Uploaded by

Copyright:

Available Formats

Mathematical Statistics (MA212M)

Theorem: (Without proof) Let X1 , . . . , Xn be RS with JPMF/JPDF

fX (x, θ) = h(x)gθ (T (x)) ,

where h(x) does not involve θ, gθ (·) depends on θ and x only

L(θ, x) = h(x)gθ (T (x)),

Note that we will be able to use the definition of sufficient

If T is sufficient for θ, then for any one-to-one function of T is

Theorem: Let T be a sufficient statistics for θ. If a unique MLE exist

You might also like