Chap 5

STAT355 - Probability & Statistics Chapter 5: Joint Probability Distributions and Random Samples
Fall 2011
STAT355 () - Probability & Statistics
Chapter Fall 2011 5: Joint 1 Probab / 34
Chap 5 - Joint Probability Distributions and Random Samples
5.1 Jointly Distributed Random Variables 5.2 Expected Values, Covariance, and Correlation 5.3 Statistics and Their Distributions 5.4 The Distribution of the Sample Mean 5.5 The Distribution of a Linear Combination
Two Discrete Random Variables
The probability mass function (pmf) of a single discrete rv X species how much probability mass is placed on each possible X value. The joint pmf of two discrete rvs X and Y describes how much probability mass is placed on each possible pair of values (x , y ).
Two Discrete Random Variables

Denition
Let X and Y be two discrete rvs dened on the sample space S of an experiment. The joint probability mass function p (x , y ) is dened for each pair of numbers (x , y ) by p (x , y ) = P (X = x and Y = y ) The marginal probability mass function of X , denoted by pX (x ), is given by pX (x ) = p (x , y ) for each possible value x
y
Similarly, the marginal probability mass function of Y is pY (y ) =

x
p (x , y ) for each possible value y .
Two Discrete Random Variables - Remarks
It must be the case that p (x , y ) 0 and
p (x , y ) = 1.
Let A be any set consisting of pairs of (x , y ) values (e.g., A = (x , y ) : x + y = 5 or (x , y ) : max(x , y ) 3). Then the probability P [(X , Y ) A] is obtained by summing the joint pmf over pairs in A: P [(X , Y ) A] =
(x ,y ) A
p (x , y )
Two Discrete Random Variables - Examples

A large insurance agency services a number of customers who have purchased both a homeowners policy and an automobile policy from the agency. For each type of policy, a deductible amount must be specied. For an automobile policy, the choices are $100 and $250, whereas for a homeowners policy, the choices are 0, $100, and $200. Suppose an individual with both types of policy is selected at random from the agencys les. Let X = the deductible amount on the auto policy, and Y = the deductible amount on the homeowners policy. Possible (X , Y ) pairs are then (100, 0), (100, 100), (100, 200), (250, 0), (250, 100), and (250, 200); the joint pmf species the probability associated with each one of these pairs, with any other pair having probability zero.

Suppose the joint pmf is given in the accompanying joint probability table: y p(x,y) x 100 250 Then p (100, 100) = P (X = 100andY = 100) = P ($100 deductible on both policies) = .10. The probability P (Y 100) is computed by summing probabilities of all (x , y ) pairs for which y 100: P (Y 100) = p (100, 100)+ p (250, 100)+ p (100, 200)+ p (250, 200) = 0.75
0 0.2 0.05
100 0.10 0.15
200 0.20 0.30

The possible X values are x = 100 and x = 250, so computing row totals in the joint probability table yields pX (100) = p (100, 0) + p (100, 100) + p (100, 200) = .50 and pX (250) = p (250, 0) + p (250, 100) + p (250, 200) = .50 The marginal pmf of X is then pX (x ) = And the marginal for Y is 0.25 0.50 pY (y ) = 0 if y = 0, 100 if y = 200 otherwise . 0.5 0 if x = 100, 200 otherwise .
Two Continuous Random Variables
The probability that the observed value of a continuous rv X lies in a one-dimensional set A (such as an interval) is obtained by integrating the pdf f (x ) over the set A. Similarly, the probability that the pair (X , Y ) of continuous rvs falls in a two-dimensional set A (such as a rectangle) is obtained by integrating a function called the joint density function.

Denition
Let X and Y be continuous rvs. A joint probability density function f (x , y ) for these two variables is a function satisfying f (x , y ) 0 and f (x , y )dxdy = 1. Then for any two-dimensional set A P [(X , Y ) A] =
A
f (x , y )dxdy
In particular, if A is the two-dimensional rectangle {(x , y ) : a x b , c y d }, then

b d
P [(X , Y ) A] = P (a X b , c Y d ) =
a c
f (x , y )dxdy
Denition
The marginal probability density functions of X and Y , denoted by fX (x ) and fY (y ), respectively, are given by
fX ( x ) =

f (x , y )dy for x f (x , y )dx for y
(1) (2)
fY (y ) =
Two Continuous Random Variables - Examples

A bank operates both a drive-up facility and a walk-up window. On a randomly selected day, let X = the proportion of time that the drive-up facility is in use (at least one customer is being served or waiting to be served) and Y = the proportion of time that the walk-up window is in use. Then the set of possible values for (X , Y ) is the rectangle D = {(x , y ) : 0 x 1, 0 y 1}. Suppose the joint pdf of (X , Y ) is given by f (x , y ) =
6 5 (x
+ y 2) 0
if 0 x 1, 0 y 1 otherwise .

Suppose the joint pdf of (X , Y ) is given by f (x , y ) =
6 5 (x
+ y 2) 0
if 0 x 1, 0 y 1 otherwise .
Verify that this is a legitimate pdf 1 f (x , y ) 0 2 f (x , y )dxdy = 1
Independent Random Variables
Denition
Two random variables X and Y are said to be independent if for every pair of x and y values p (x , y ) = pX (x )pY (y ) when X and Y are discrete or f (x , y ) = fX (x )fY (y ) when X and Y are continuous If (3) is not satised for all (x , y ), then X and Y are said to be dependent. (3)

In the insurance situation p (100, 100) = .10 = (.5)(.25) = pX (100)pY (100) so X and Y are not independent. Independence of two random variables is most useful when the description of the experiment under study suggests that X and Y have no eect on one another. Then once the marginal pmfs or pdfs have been specied, the joint pmf or pdf is simply the product of the two marginal functions. It follows that P (a X b , c Y d ) = P (a X b )P (c Y d )
Conditional Distributions
Denition
Let X and Y be two continuous rvs with joint pdf f (x , y ) and marginal X pdf fX (x ). Then for any X value x for which fX (x ) > 0, the conditional probability density function of Y given that X = x is fY |X (y |x ) = f (x , y ) fX (x ) <y <
Exercise (5.1) 13
You have two lightbulbs for a particular lamp. Let X = the lifetime of the rst bulb and Y = the lifetime of the second bulb (both in 1000s of hours). Suppose that X and Y are independent and that each has an exponential distribution with parameter = 1. 1 What is the joint pdf of X and Y ? 2 What is the probability that each bulb lasts at most 1000 hours (i.e. X 1 and Y 1)? 3 What is the probability that the total lifetime of the two bulbs is at most 2? [Hint: Draw a picture of the region A = {(x , y ) : x 0, y 0, x + y 2} before integrating.] 4 What is the probability that the total lifetime is between 1 and 2?
Expected Values
Proposition
Let X and Y be jointly distributed rvs with pmf p (x , y ) or pdf f (x , y ) according to whether the variables are discrete or continuous. Then the expected value of a function h(X , Y ), denoted by E [h(X , Y )] or h(X ,Y ) , is given by E [h(X , Y )] =
x y h (x , y )p (x , y ) h(x , y )f (x , y )dxdy
if X and Y are discrete if X and Y are continuous
Expected Values - Example

The joint pdf of the amount X of almonds and amount Y of cashews in a 1-lb can of nuts was f (x , y ) = 24xy 0 0 x 1, 0 y 1, x + y 1 otherwise
If 1 lb of almonds cost the company $100, 1 lb of cashews costs $1.50, and 1 lb of peanuts costs $0.50, then the cost of the contents of a can is h(X , Y ) = (1)X + (1.5)Y + (0.5)(1 X Y ) = 0.5 + 0.5X + Y The expected total cost is E [h(X , Y )] =
1
h(x , y )f (x , y )dxdy
1x
=
0 0
(0.5 + 0.5x + y )24xydxdy
Covariance
When two random variables X and Y are not independent, it is frequently of interest to assess how strongly they are related to one another.
Denition
The covariance between two rvs X and Y is Cov (X , Y ) = E [(X X )(Y Y )] x y (x X )(y Y )p (x , y ) = (x X )(y Y )f (x , y )dxdy
X , Y discrete X , Y cont.
The following shortcut formula for Cov (X , Y ) simplies the computations.
Proposition
Cov (X , Y ) = E (XY ) X Y
Covariance
Since X X and Y Y are the deviations of the two variables from their respective mean values, the covariance is the expected product of deviations. Remarks:
1
Cov (X , X ) = E [(X X )2 ] = V (X ). If X and Y have a strong positive relationship to one another then Cov (X , Y ) should be quite positive. For a strong negative relationship, Cov (X , Y ) should be quite negative. If X and Y are not strongly related, Cov (X , Y ) is near 0.
Correlation
Denition
The correlation coecient of X and Y , denoted by Corr (X , Y ), X ,Y , or just , is dened by Cov (X , Y ) X ,Y = X Y where X and sigmaY are the standard deviations of X and Y .
Proposition
If a and c are either both positive or both negative, Corr (aX + b , cY + d ) = Corr (X , Y ) For any two rvs X and Y , 1 Corr (X , Y ) 1.
Correlation
Proposition
1
If X and Y are independent, then X ,Y = 0, but = 0 does not imply independence. = 1 or 1 i Y = aX + b for some numbers a and b with a = 0.
This proposition says that is a measure of the degree of linear relationship between X and Y , and only when the two variables are perfectly related in a linear manner will be as positive or negative as it can be. A less than 1 in absolute value indicates only that the relationship is not completely linear, but there may still be a very strong nonlinear relation.
Exercise (5.2) 27
Annie and Alvie have agreed to meet for lunch between noon (0:00pm) and 1:00pm. Denote Annies arrival time by X , Alvies by Y , and suppose X and Y are independent with pdfs fX ( x ) = fY (y ) = 3x 2 0 x 1 0 otherwise
2y 0 y 1 0 otherwise What are the expected amount of time that the one who arrives rst must wait for the other person? [Hint: h(X , Y ) = |X Y |]
Exercise (5.2) 35
1
Use the rules of expected value to show that Cov (aX + b , cY + d ) = ac Cov (X , Y ). Use part 1. along with the rules of variance and standard deviation to show that Corr (aX + b , cY + d ) = Corr (X , Y ) when a and c have the same sign. What happens if a and c have opposite sign.
Random Samples
Denition
A statistic is any quantity whose value can be calculated from sample data. A statistic is a random variable and will be denoted by an uppercase letter; a lowercase letter is used to represent the calculated or observed value of the statistic.
Denition
The rvs X1 , X2 , ..., Xn are said to form a (simple) random sample of size n if
1 2
The Xi s are independent rvs. Every Xi has the same probability distribution.
A random sample Xi , i = 1, ..., n is sometimes referred to as iid (independent and identically distributed).
Exercise (5.3) 39
It is known that 80% of all brand A zip drives work in a satisfactory manner throughout the warranty period (are successes). Suppose that n = 10 drives are randomly selected. Let X = the number of successes in the sample. The statistic X /n is the sample proportion (fraction) of successes. Obtain the sampling distribution of this statistic. [Hint: One possible value of X /n is 0.3. What is the probability of this value (what kind of random variable is X )?
The Distribution of the Sample Mean

Notation: Let X1 , ..., Xn be an iid rvs. The sample mean is denoted by
n
= X
i =1
Xi
Proposition
Let X1 , X2 , ..., Xn be a random sample from a distribution with mean value and standard deviation . Then ) = = 1 E (X X 2 /n and = /n ) = 2 2 V (X = X X In addition, with T0 = X1 + ... + Xn , E (T0 ) = n.
The Distribution of the Sample Mean
The Central Limit Theorem (CLT)
Theorem
Let X1 , X2 , ..., Xn be a random sample from a distribution with mean has approximately a and variance 2 . Then if n is suciently large, X 2 = 2 /n and T also normal distribution with mean X and variance X 0 has approximately a normal distribution with mean T0 = n and variance 2 = n 2 . T 0 Remark: The larger the value of n, the better the approximation. Rule of Thumb: If n > 30, the Central Limit Theorem can be used.
CLT - Example
The CLT can be used to justify the normal approximation to the binomial distribution discussed earlier. We know that a binomial variable X is the number of successes in a binomial experiment consisting of n independent success/failure trials with p = P (S ) for any particular trial. Dene a new rv X1 by X1 = 1 0 if the rst trial results in a success if the rst trial results in a failure
and dene X2 , X3 , ..., Xn analogously for the other n1 trials. Each Xi indicates whether or not there is a success on the corresponding trial. Because the trials are independent and P (S ) is constant from trial to trial, the Xi s are iid (a random sample from a Bernoulli distribution).The CLT then implies that if n is suciently large, both the sum and the average of the Xi s have approximately normal distributions.
Exercise (5.4) 55
The number of parking tickets issued in a certain city on any given weekday has a Poisson distribution with parameter = 50. What is the approximate probability that 1 between 35 and 70 tickets are given out on a particular day? [Hint: When is large, a Poisson rv has approximately a normal distribution.] 2 The total number of tickets given out during a 5-day week is between 225 and 175?
The Distribution of a Linear Combination
Denition
Given a collection of n random variables X1 , ..., Xn and n numerical constants a1 , ..., an , the rv Y = a1 X1 + ... + an Xn is called a linear combination of the Xi s.
The Distribution of a Linear Combination

Proposition
Let X1 , X2 , ..., Xn have mean values 1 , ..., n , respectively, and variances 2 , ..., 2 respectively. 1 n
1
Whether or not the Xi s are independent, E (a1 X1 + ... + an Xn ) = a1 E (X1 ) + ... + an E (Xn ) = a1 1 + ... + an n
If X1 , ..., Xn are independent,

2 2 2 2 2 2 V (a1 X1 + ... + an Xn ) = a1 V (X1 ) + ... + an V (Xn ) = a1 1 + ... + an n
For any X1 , ..., Xn ,

n n
V (a1 X1 + ... + an Xn ) =
i =1 j =1
ai aj Cov (Xi , Xj )
Exercise (5.5) 73
Suppose the expected tensile strength of type-A steel is 105 ksi and the standard deviation of tensile strength is 8 ksi. For type-B steel, suppose the expected tensile strength and standard deviation of tensile strength are = the sample average tensile 100 ksi and 6 ksi, respectively. Let X = the strength of a random sample of 40 type-A specimens, and let Y sample average tensile strength of a random sample of 35 type-B specimens. ?, Of Y ? 1 What is the approximate distribution of X
2 3 4
Y ? Justify your answer. What is the approximate distribution of X Y 1) Calculate (approximately) P (1 X Y 10). If you actually observed X Y 10, Calculate P (X would you doubt that 1 2 = 5?

Chap 5

Uploaded by

Copyright:

Available Formats

Chap 5

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chap 5

Uploaded by

Copyright:

Available Formats

STAT355 - Probability & Statistics Chapter 5: Joint Probability Distributions and Random Samples

STAT355 () - Probability & Statistics

Chapter Fall 2011 5: Joint 1 Probab / 34

Chap 5 - Joint Probability Distributions and Random Samples

STAT355 () - Probability & Statistics

Chapter Fall 2011 5: Joint 2 Probab / 34

Two Discrete Random Variables

STAT355 () - Probability & Statistics

Chapter Fall 2011 5: Joint 3 Probab / 34

Two Discrete Random Variables

Similarly, the marginal probability mass function of Y is pY (y ) =

p (x , y ) for each possible value y .

STAT355 () - Probability & Statistics

Chapter Fall 2011 5: Joint 4 Probab / 34

Two Discrete Random Variables - Remarks

It must be the case that p (x , y ) 0 and

STAT355 () - Probability & Statistics

Chapter Fall 2011 5: Joint 5 Probab / 34

Two Discrete Random Variables - Examples

Chapter Fall 2011 5: Joint 6 Probab / 34

Two Discrete Random Variables - Examples

100 0.10 0.15

200 0.20 0.30

Chapter Fall 2011 5: Joint 7 Probab / 34

Two Discrete Random Variables - Examples

STAT355 () - Probability & Statistics

Chapter Fall 2011 5: Joint 8 Probab / 34

Two Continuous Random Variables

STAT355 () - Probability & Statistics

Chapter Fall 2011 5: Joint 9 Probab / 34

Two Continuous Random Variables

In particular, if A is the two-dimensional rectangle {(x , y ) : a x b , c y d }, then

STAT355 () - Probability & Statistics

Chapter Fall 2011 5: Joint 10 Probab / 34

Two Continuous Random Variables

f (x , y )dy for x f (x , y )dx for y

STAT355 () - Probability & Statistics

Chapter Fall 2011 5: Joint 11 Probab / 34

Two Continuous Random Variables - Examples

STAT355 () - Probability & Statistics

Chapter Fall 2011 5: Joint 12 Probab / 34

Two Continuous Random Variables - Examples

Verify that this is a legitimate pdf 1 f (x , y ) 0 2 f (x , y )dxdy = 1

STAT355 () - Probability & Statistics

Chapter Fall 2011 5: Joint 13 Probab / 34

Independent Random Variables

STAT355 () - Probability & Statistics

Chapter Fall 2011 5: Joint 14 Probab / 34

Two Continuous Random Variables - Examples

STAT355 () - Probability & Statistics

Chapter Fall 2011 5: Joint 15 Probab / 34

STAT355 () - Probability & Statistics

Chapter Fall 2011 5: Joint 16 Probab / 34

STAT355 () - Probability & Statistics

Chapter Fall 2011 5: Joint 17 Probab / 34

if X and Y are discrete if X and Y are continuous

STAT355 () - Probability & Statistics

Chapter Fall 2011 5: Joint 18 Probab / 34