Lectures Prepared By: Elchanan Mossel Yelena Shvets

Introduction to probability
Stat 134
FAll 2005
Berkeley
Lectures prepared by: Elchanan Mossel Yelena Shvets

Follows Jim Pitmans book: Probability Section 4.5
Cumulative Distribution Function

Definition: For a random variable X, the function F(x) = P(X x), is called the cumulative distribution function (cdf). A distribution is called continuous when the cdf is continuous.
Properties of the CDF.

Claim: The cdf determines the probability of lying inside intervals: P(a < X b) = F(b) F(a).
P{(a , b]} = F(b) F(a). P{(a , b)} = limx ! b- F(x) F(a). P{(a,b) (c,d) } = P{(c,b)}, for a<c<b<d.
P{(a,b) [ (c,d) } = P{(a,d)}, for a<c<b<d.

P{b} = P{(a,b]} P{(a,b)} = F(b) - limx ! b- F(x).
If F is continuous, P(a,b) = F(b) F(a) and F(x) = 0 for all x.
Cumulative Distribution Function
Discrete Distributions
Claim: If X is a discrete variable, then F(x) = y xP(X=y). Example: Let X be the numbers on a fair die.
1 5/6
0.2 0.15 0.1 0.05 0 0 1 2 3 4 5 6 7
2/3 1/2 1/3 1/6 0 0 1 2 3 4 5 6 7
P{x} = F(x) F(x -) :
P{X=6} = 1 5/6 = 1/6.
Example: Let X be the number of rolls before the first six appears when rolling a fair die. X Geom(1/6), P(n)=(5/6)n-1 1/6 (for integer n)
F(x) = 1 (5/6) b x c
1
0.2
Discrete Distributions
0.15
0.1
0.05
0 0
For integer x:
20
40
60
20
40
60
P{n} = F(n) F(n-) = 1 - (5/6)n (1 - (5/6)n-1) = (5/6)n-1 1/6.
Continuous Distributions with a Density

Claim: If X is a continuous variable with a density fX(x) then F(x) =s-1x fX(y)dy.
Claim: if X has a continuous distribution given by a c.d.f F(x) and F is differentiable in all but finitely many points, Then the distribution has a density fX(x) = F(x).
Pf: We have F(x) = s-1x fX(t) dt So: F(x) = fX(x) at the points where F is differentiable.
Distributions with a Density

Example: U Uniform(0,1).
2 1
-1
Question: Show that X = 2|U | is a Uniform variable as well.
0 -1 0 1 2
Solution: P(X x) = P(2|U | x) = P( - x/2 U + x/2) =F(1/2 + x/2) F(1/2 x/2) = 1/2 + x/2- 1/2 + x/2 =x = F(x). So X has the same cdf as U therefore the same distribution. Question: What is the distribution of 4| |U-1/2| - 1/4| ?
Distribution of Min and Max.

Let X1, X2, , Xn be independent random variables.
Xmax = max{X1, X2, , Xn} & Xmin = min{X1, X2, , Xn} .

Its easy to find the distribution of Xmax and Xmin by using the cdfs by using the following observations: For any x Xmax x iff Xi x for all i. Xmin > x iff Xi > x for all i.
Distribution of Min and Max.

F (x)= x)F = (x) P(X... x, X2 x, Xn x) 1 F max Fmax (x) P(X = max F1(x) (x). 2 n = P(X1 x)P( X2 x) P(Xn x) , by indep. Fmin (x) = 1 (1 F1(x)) (1 F2(x)) (1 Fn(x)). = F1(x) F2(x) ... Fn(x). Fmin (x)= P(Xmin x) = 1 P(Xmin > x) Example: Suppose exponential = 1 - P(X1X >is x,all X2have > x, Xn > x) distributions with rates li. = 1 (1 F1(x)) (1 F2(x)) (1 Fn(x)).
Question: Find the distribution of Xmin.
Solution: For x <0, Fi(x) =0 and for x 0, Fi = 1-e-l x. So for x < 0, Fmin = 0 and for x 0 ,Fmin = 1- e-l1 + l2 + +ln. Xmin Exp( l1 + l2 + ln}.
Order Statistics
Let X1, X2, Xn be random variables, We can use the CDF to find the CDF of the kth largest of these X. We call this X(k), or the kth order statistic. The density for the min and max are special cases for k = 1 and k = n. If all the Xi are identical and independent, we can find the density of the order statistics
Order Statistics
P(X(k) ~ (x, x+dx)) =
P(one of the Xs in dx)* P(k-1 of the Xs below x)* P(n-k of the Xs above x)=
P(one of the Xs in dx)
P(n-k of the Xs above x) (P(X > x) =1- F(x)) P(k-1 of the Xs below x) (P(X < x) = F(x))
Number of ways to order Xs
When Xi are Unif(0,1) we obtain:
Order Statistics for Uniform R.V
f(k)(x) =
Def: For r,s > 0 the beta(r,s) distribution is the distribution on (0,1) with density xr-1(1-x)s-1/B(r,s) where B(r,s) = s01 xr-1(1-x)s-1 dx Claim: If (Xi)i=1n are i.i.d Unif(0,1) then X(k) has beta(k,n-k+1) distribution.
Claim: If r=k and s=n-k+1 are integers then:
Beta and Gamma distributions
B(r,s) =
where (r) = (r-1)! For integer rs. Problem: Calculate E(X) and Var(X) for X with beta(r,s) distribution for integer r and s. Problem: For the order statistics of n i.i.d. unif(0,1) random variables, calculate E(X) and Var(X).
Percentiles and Inverse CDF.

Definition: The pth quantile of the distribution of a random variable X is given by the number x such that P(X x) = p.
Simulating a RV with a given distribution Example:

A pseudo random number generator generates a sequence U1,U2,Un each between 0 and 1, which are i.i.d. Uniform(0,1) random variables.
Question: You are working for a company that is developing an on-line casino. How can you use this sequence to simulate n rolls of a fair die? Solution: You need a new sequence of variables each with a discrete uniform distribution on 1,2,3,4,5,6. Simple idea -- break up the unit interval into 6 intervals Ak = ((k-1)/6, k/6], each of length 1/6 & let g(u) = k if u 2 Ak.
Simulating a RV with a given distribution

1 5/6 2/3 1/2 1/3 1/6 0 0 1 2 3 4 5 6 7
Then Xi = g(Ui) each has the desired distribution: P(Xi = k) = P(g(Ui)=k) = P(Ak) = 1/6.

1 5/6 2/3 1/2 1/3 1/6 0 0 1 2 3 4 5 6 7
Note that for i=1,2,3,4,5,6 it holds that: g(F(i)) = g(i/6)=i.

For a general discrete random variable with P(k) = pk, k=1,2,3 we can use the same idea, except each interval Ak is now of length pk :
Ak = (p1+p2++pk-1 , p1+p2++pk-1+pk]. The function g(u) defined in this way is a kind of an inverse of the cdf:
g(F(k)) = k for all k=1,2,3,
This observation can be used to simulate continuous random variables as well.

Define F-1(x) = miny F(y) = x.
Inverse cdf Applied to Standard Uniform

Define F-1(x) = min { y : F(y) = x} Theorem: For any cumulative distribution function F, with inverse function F-1, if U has a Uniform (0,1) distribution, then a random variable G = F-1(U) has F as a cdf.
Proof (imagine first that F is strictly increasing):

P(G x) = P{UF(x)) = F(x).

A pseudo random number generator gives a sequence U1,U2,Un each between 0 and 1, which behave like a sequence of Uniform(0,1) random variables.
Question: You are working for an engineering company that wants to model failure of circuits. You may assume that a circuit has n components whose life-times have Exp(li) distribution. How can you generate n variables that would have the desired distributions?
Solution:
So we let
The cdfs are Fi(x) = 1 e-li x .

Gi = G(Ui) where G(u) = -log(1-u)/li.

Lectures Prepared By: Elchanan Mossel Yelena Shvets

Uploaded by

Copyright:

Available Formats

Lectures Prepared By: Elchanan Mossel Yelena Shvets

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lectures Prepared By: Elchanan Mossel Yelena Shvets

Uploaded by

Copyright:

Available Formats

Introduction to probability

Lectures prepared by: Elchanan Mossel Yelena Shvets

Cumulative Distribution Function

Properties of the CDF.

P{(a,b) [ (c,d) } = P{(a,d)}, for a<c<b<d.

If F is continuous, P(a,b) = F(b) F(a) and F(x) = 0 for all x.

Cumulative Distribution Function

2/3 1/2 1/3 1/6 0 0 1 2 3 4 5 6 7

P{x} = F(x) F(x -) :

P{X=6} = 1 5/6 = 1/6.

P{n} = F(n) F(n-) = 1 - (5/6)n (1 - (5/6)n-1) = (5/6)n-1 1/6.

Continuous Distributions with a Density

Distributions with a Density

Question: Show that X = 2|U | is a Uniform variable as well.

Distribution of Min and Max.

Xmax = max{X1, X2, , Xn} & Xmin = min{X1, X2, , Xn} .

Distribution of Min and Max.

P(one of the Xs in dx)

Number of ways to order Xs

When Xi are Unif(0,1) we obtain:

Order Statistics for Uniform R.V

Claim: If r=k and s=n-k+1 are integers then:

Beta and Gamma distributions

Percentiles and Inverse CDF.

Simulating a RV with a given distribution Example:

Simulating a RV with a given distribution

Simulating a RV with a given distribution

Note that for i=1,2,3,4,5,6 it holds that: g(F(i)) = g(i/6)=i.

Simulating a RV with a given distribution

This observation can be used to simulate continuous random variables as well.

Inverse cdf Applied to Standard Uniform

Proof (imagine first that F is strictly increasing):

Simulating a RV with a given distribution

The cdfs are Fi(x) = 1 e-li x .

You might also like