Ali Karimnezhad: Version September 30, 2015

MAT2377
Ali Karimnezhad
Version September 30, 2015
Ali Karimnezhad
MAT2377 Probability and Statistics for Engineers Chapter 3
Comments
• These slides cover material from Chapter 3.
• In class, I may use a blackboard. I recommend reading these slides before

you come to the class.
• I am planning to spend 4 lectures on this chapter.
• I am not re-writing the textbook. The reference book contains many

interesting and practical examples.
• There may be some typos. The final version of the slides will be posted
after the chapter is finished.
Ali Karimnezhad 1
Bernoulli Distribution
A Bernoulli trial is a random experiment with two possible outcomes, success
and failure.
One obvious application deals with the testing of items as they come off an
assembly line. Each trial may indicate a defective or a nondefective item.
• Denote p as the probability of a success.
• Let X be a sucess indicator in a Bernoulli trial.
• Then, X ∼ Ber(p) with the probability function

1 − p for x = 0
P (X = x) =
p for x = 1
= px (1 − p)1−x, x = 0, 1.
Ali Karimnezhad 2
Binomial Distribution
A binomial experiment consists of n repeated independent Bernoulli trials,

each with the same probability of succes.
• Denote p as the probability of a success.
• Let X be number of sucesses in a Binomial trial.
• Then, X ∼ B(n, p) with the probability function

n x
b(x; n, p) = P (X = x) = p (1 − p)n−x, x = 0, 1, . . . , n.
x
• If n = 1, X ∼ B(1, p) or equivalently X ∼ Ber(p).
Ali Karimnezhad 3
Examples:
• The probability that a patient recovers from a rare blood disease is

0.4. If 5 people are known to have contracted this disease, what is the
probability that exactly 2 survive?
• A large chain retailer purchases a certain kind of electronic device from a

manufacturer. The manufacturer indicates that the defective rate of the
device is 3%. The inspector randomly picks 20 items from a shipment.
What is the probability that there will be at most one defective item
among these 20?
Ali Karimnezhad 4
Expectation and Variance for Binomial Distribution

• From math basis, we know

n 0 n n 1 n−1 n n 0
(p + q)n = p q + p q + ... + p q .
0 1 n
• Expecation;
n
X n x
E(X) = x p (1 − p)n−x = · · · = np.
x=0
x
• Variance
n
X n x
Var(X) = E (X − np)2 = (x−np)2 p (1−p)n−x = np(1−p) .

x=0
x
Ali Karimnezhad 5
Example:
Suppose that each sample of water has 10% of being polluted. If 12 samples
are selected independently, then it is reasonable to model the number X
polluted sample as in the sample as B(12, 0.1). Find
(a) E(X) and Var(X).
(b) P (X = 3).
(c) P (X ≤ 3) (tables can be used).
Ali Karimnezhad 6
Solution:
(a) If X ∼ B(n, p) then E(X) = np and Var(X) = np(1 − p) so
E(X) = 12 × 0.1 = 1.2 ; Var(X) = 12 × 0.1 × 0.9 = 1.08 .
12

(b) P (X = 3) = 3 (0.1)3(0.9)9 ≈ 0.0852.
(c) P (X ≤ 3) = P (X = 0) + P (X = 1) + P (X = 2) + P (X = 3) = · · · .
However, P (X ≤ 3) for X ∼ B(12, 0.1) is tabulated in Appendix A.1 of
the Reference Book: ≈ 0.9744.
• Also, since P (X ≤ 2) = P (X = 0) + P (X = 1) + P (X = 2) ≈ 0.8891

from tables we see that P (X = 3) = P (X ≤ 3) − P (X ≤ 2) ≈
0.9744 − 0.8891 = 0.0853 (note rounding error).
Ali Karimnezhad 7
Ali Karimnezhad 8
Example:
According to a chemical engineering progress study, approximately 30% of
all pipework failures in chemical plants are caused by operator error.
(a) What is the probability that out of the next 13 pipework failures at least
10 are due to operator error?
(b) What is the probability that no more than 4 out of 13 such failures are
due to operator error?
Ali Karimnezhad 9
Multinomial Distribution
The Binomial experiment becomes a Multinomial experiment if we let each

trial have more than two possible outcomes.
If a given trial can result in the k outcomes E1, E2, . . . , Ek with probabilities
p1, p2, . . . pk , then the probability distribution of the random variables
X1, X2, . . . , Xk , representing the number of occurrences for E1, E2, . . . , Ek
in n independent trials, is

n x
P (X1 = x1, X2 = x2, . . . , Xk = xk ) = px1 1 px2 2 . . . pkk ,
x1, x2, . . . , xk
where x1 + x2 + . . . + xk = n and p1 + p2 + . . . + pk = 1.
Ali Karimnezhad 10
Example:
In a certain town, 40% of the eligible voters prefer candidate A, 10% prefer
candidate B, and the remaining 50% have no preference. You randomly
sample 10 eligible voters. What is the probability that 4 will prefer candidate
A, 1 will prefer candidate B, and the remaining 5 will have no preference?
Ali Karimnezhad 11
Hypergeometric Distribution
Example-Motivation:
• Suppose that an urn contains 8 red balls and 4 green balls. We draw 2
balls from the urn without replacement. If we assume that at each draw
each ball in the urn is equally likely to be chosen, what is the probability
that both balls are Red?
Define R1 = {the first ball is Red} and R2 = {the second ball is Red}:
8

2 8×7
P (R1 ∩ R2) = 12 = . . . = .
2
12 × 11
Let X be number of Red balls in the drawing:

8 4

2 0 8×7
P (X = 2) = 12
= ... = .
2
12 × 11
Ali Karimnezhad 12
Hypergeometric Distribution
A Hypergeometric experiment possesses the following two properties:
1. A random sample of size n is selected without replacement from N

items.
2. Of the N items, k interesting items have a same property in common,

and N − k have another same property in common.
If X is number of type-k items in a random sample of size n, then

X ∼ HG(N, n, k) with the probability function
k N −k

x n−x
h(x; N, n, k) = P (X = x) = N
, max{0, n − (N − k)} ≤ x ≤ min{n, k}.
n
Ali Karimnezhad 13
Example:
Lots of 40 components each are deemed unacceptable if they contain 3 or
more defectives. The procedure for sampling a lot is to select 5 components
at random and to reject the lot if a defective is found.
What is the probability that exactly 1 defective is found in the sample if
there are 3 defectives in the entire lot?
Define X as number of defective items. Then, using the Hypergeometric

distribution with n = 5, N = 40 and k = 3, X ∼ HG(40, 5, 3). Thus,
3 40−3

1 5−1
P (X = 1) = 40
= 0.3011.
5
Ali Karimnezhad 14
Expectation and Variance for Hypergeometric

Distribution
If X ∼ HG(N, n, k),
• Expecation
n k n−k

X x n−x nk
E(X) = x N
= .
x=0 n
N
• Variance

nk 2 N − n nk k
Var(X) = E (X − ) = (1 − ).
N N −1 N N
For a proof, see Appendix A.12 of the reference book.
Ali Karimnezhad 15
Geometric Distribution
Example-Motivation:
• Flip a coin repeatedly until you get a Head. Let the probability that the
coin lands Heads up be p and the probability that the coin lands Tails up
be q = 1 − p. Suppose at each flip a Head or a Tail is equally likely to
be observed.
What is probability of observing a Head (success) in the 4th flip?
Define X as number of required trails to observe a Head (success). Thus,
P (X = 4) = P ({T T T H}) = p q 3.
Ali Karimnezhad 16
Geometric Distribution
A Geometric experiment possesses the following two properties:
1. A Bernoulli trail with probability p of success and probability of failure

q = 1 − p at each step is repeated.
2. The experiment is stopped when only one success occurs.
If X is the number of steps required until the 1st success occurs, then
X ∼ Ge(p) with the probability function
g(x; p) = P (X = x) = p q x−1, x = 1, 2, . . . .
Ali Karimnezhad 17
Example:
For a certain manufacturing process, it is known that, on the average, 1 in
every 100 items is defective.
What is the probability that the 5th item inspected is the first defective
item found?
Define X as number of required trails until the 1st defective item is found.
1
Then, using the Geometric distribution with p = 100 , X ∼ Ge(0.01). Thus,
P (X = 5) = (0.01) (0.99)4 = 0.0096.
Ali Karimnezhad 18
Expectation and Variance for Geometric Distribution
If X ∼ Ge(p),
• Expectation
∞
X
x−1 1
E(X) = x pq = .
x=1
p
• Variance
1 q
Var(X) = E (X − )2 = 2 .
p p
Ali Karimnezhad 19
Negative Binomial Distribution
Example-Motivation:
• In a game, we flip a coin repeatedly until we get two Heads (the game is
stopped when the second Head is observed). Let the probability that the
coin lands Heads up be p and the probability that the coin lands Tails up
be q = 1 − p. Suppose at each flip a Head or a Tail is equally likely to
be observed.
What is probability that the game is stopped in the 4th flip?
Define X as number of required trails to observe the second Head
(success). Thus,

4 − 1 2 4−2
P (X = 4) = P ({T T HH, HT T H, T HT H}) = 3p2 q 2 = p q .
2−1
Ali Karimnezhad 20
Negative Binomial Distribution
A Negative Binomial experiment possesses the following two properties:
1. A Bernoulli trail with probability p of success and probability of failure

q = 1 − p at each step is repeated.
2. The experiment is stopped when a kth success occurs.
If X is the number of steps required until the kth success occurs, then
X ∼ N B(k, p) with the probability function

x − 1 k x−k
b∗(x; k, p) = P (X = x) = p q , x = k, k + 1, . . . .
k−1
Ali Karimnezhad 21
Expectation and Variance for Negative Binomial

Distribution
If X ∼ N B(k, p),
• Expecation
∞
X x − 1 k x−k k
E(X) = x p q = .
k−1 p
x=k
• Variance
k k
Var(X) = E (X − )2 = 2 .
p p
Ali Karimnezhad 22
Poisson Distribution-Concept
Experiments yielding numerical values of a random variable X, the number
of outcomes occurring during a given time interval or in a specified region,
are called Poisson experiments.
• The given time interval may be of any length, such as a minute, a day, a
week, a month, or even a year. For example, a Poisson experiment can
generate observations for the random variable X representing
– the number of telephone calls received per hour by an office,
– the number of games postponed due to rain during a baseball season.
• The specified region could be a line segment, an area, a volume, or

perhaps a piece of material. In such instances, X might represent
– the number of typing errors per page,
– the number of accidents in an intersection.
Ali Karimnezhad 23
Poisson Process
A Poisson experiment is derived from the Poisson process and possesses the
following properties.
1. The number of outcomes occurring in one time interval or specified

region of space is independent of the number that occur in any other
disjoint time interval or region. In this sense, we say that the Poisson
process has no memory.
2. The probability that a single outcome will occur during a very short time
interval or in a small region is proportional to the length of the time
interval or the size of the region and does not depend on the number of
outcomes occurring outside this time interval or region.
3. The probability that more than one outcome will occur in such a short
time interval or fall in such a small region is negligible.
Ali Karimnezhad 24
Poisson Distribution
• The number X of outcomes occurring during a Poisson experiment is
called a Poisson random variable, and its probability distribution is called
the Poisson distribution.
• The mean number of outcomes is computed from µ = λt, where t is the

specific “time”, “distance”, “area” or “volume” of interest.
• The probability function of the Poisson random variable X, representing

the number of outcomes occurring in a given time interval or specified
region denoted by t, is
e−λt(λt)x
p(x; λt) = P (X = x) = , x = 0, 1, . . .
x!
where λ is the average number of outcomes per unit time, distance, area,
or volume and e = 2.71828. We simply write X ∼ P (λt).
Ali Karimnezhad 25
Example:
Births in a hospital occur randomly at an average rate of 2 births per hour.
(a) What is the probability of observing 4 births in a given hour at the
hospital?
Let X be number of births in a given hour. Then, X ∼ P (λt) with
λ = 2 and t = 1:
e−2 24
p(4; 2) = P (X = 4) = .
4!
(b) What would be the probability of observing 4 births in a given day at the
hospital?
Let Y be number of births in a given day. Then, Y ∼ P (λt) with λ = 2
and t = 24:
e−2×24 (2 × 24)4
p(4; 2 × 24) = P (Y = 4) = .
4!
Ali Karimnezhad 26
(c) What is the probability of observing at least 4 births in a given hour at

the hospital? X3
e−2 2x
P (X ≥ 4) = 1 − P (X ≤ 3) = 1 − .
x=0
x!
Ali Karimnezhad 27
Expectation and Variance for Poisson Distribution
If X ∼ P (λt),
• Expecation
∞
X e−λt(λt)x
E(X) = x = λt.
x=0
x!
• Variance
2

Var(X) = E (X − λt) = λt.
For a proof, see Appendix A.13 of the reference book.
Ali Karimnezhad 28
Continuous Uniform Distribution

• One of the simplest continuous distributions in all of statistics is the
continuous uniform distribution.
• The distribution is characterized by a density function that is “flat”, and

thus the probability is uniform in a closed interval, say [a, b].
• The density function of the continuous uniform random variable X on

the interval [a, b] is
1
f (x) = b−a for a ≤ x ≤ b
0 for elsewhere.
• The mean and variance of the uniform distribution are

a+b (b − a)2
E(X) = , V ar(X) = .
2 2
Ali Karimnezhad 29
Example:
Suppose that a large conference room at a certain company can be reserved
for no more than 4 hours. Both long and short conferences occur quite
often. In fact, it can be assumed that the length X of a conference has a
uniform distribution on the interval [0, 4].
(a) What is the probability density function?
(b) What is the probability that any given conference lasts at least 3 hours?
Ali Karimnezhad 30
Normal Distribution
• The most important continuous probability distribution in the entire field
of statistics is the normal distribution.
• Its graph, called the normal curve, is a bell-shaped curve, which

approximately describes many phenomena that occur in nature, industry,
and research.
Ali Karimnezhad 31
Normal Distribution
• The density of the normal random variable X, with mean µ and variance
σ 2, is
1 − 1 2 (x−µ)2
n(x; µ, σ) = f (x) = √ e 2σ , µ ∈ <, σ > 0, x ∈ <,
2πσ 2
where π = 3.14159... and e = 2.71828....
• The normal distribution is often referred to as the Gaussian distribution
• We simply write X ∼ N (µ, σ 2).
Ali Karimnezhad 32
Properties of the Normal Curve
• The mode, which is the point on the horizontal axis where the curve is
a maximum, occurs at x = µ.
• The curve is symmetric about a vertical axis through the mean µ.
• The curve has its points of inflection at x = µ − σ and x = µ + σ;

it is concave downward if µ − σ < x < µ + σ and is concave upward
otherwise.
• The normal curve approaches the horizontal axis asymptotically as we

proceed in either direction away from the mean.
• The total area under the curve and above the horizontal axis is equal to
1.
Ali Karimnezhad 33
Expectation and Variance for Normal Distribution
If X ∼ N (µ, σ 2),
• Expecation Z +∞
E(X) = xf (x)dx = µ.
−∞
• Variance
Var(X) = E (X − µ) = σ 2.
2

Ali Karimnezhad 34
Areas under the Normal Curve
Z x2 Z x2
1 − 1 2 (x−µ)2
P (x1 < X < x2) = f (x)dx = √ e 2σ dx
x1 x1 2πσ 2
Ali Karimnezhad 35
Standard Normal Distribution
X−µ
• If X ∼ N (µ, σ 2), then Z = σ has a standard Normal distribution, or
briefly Z ∼ N (0, 1).
• The density of the standard normal random variable Z is
1 − 1 z2
n(z; 0, 1) = f (z) = φ(z) = √ e 2 , z ∈ <,
2π
where π = 3.14159... and e = 2.71828....
Ali Karimnezhad 36
Standard Normal Distribution

x1 − µ x2 − µ
P (x1 < X < x2) = P <Z<
σ σ
= P (z1 < Z < z2)
= Φ(z2) − Φ(z1)
Ali Karimnezhad 37
Example:
Given a standard normal distribution, find the area under the curve that lies
to the right of z = 0.84.
Ali Karimnezhad 38
Example:
Given a standard normal distribution, find the value of k such that
P (Z > k) = 0.3015.
Ali Karimnezhad 39
Example:
An electrical firm manufactures light bulbs that have a life, before burn-out,
that is normally distributed with mean equal to 300 hours and a standard
deviation of 50 hours. Find the probability that a bulb burns at least 342
hours.
Ali Karimnezhad 40
Normal Approximation to the Binomial
• If X ∼ B(n, p), a Binomial random variable with mean µ = np and

variance σ 2 = npq, then the limiting form of the distribution of
X − np
Z= √ ,
npq
as n → ∞, is the standard normal distribution N (0, 1).
Ali Karimnezhad 41
Example: X ∼ B(15, 0.4):
• Direct computation:
P (X = 4) = 0.1268
• Approximate computation:
P (3.5 < X < 4.5) = P (−1.32 < Z < −0.79)

= P (Z < −0.79) − P (Z ≤ −1.32)
Ali Karimnezhad 42
Example:
The probability that a patient recovers from a rare blood disease is 0.4. If
100 people are known to have contracted this disease, what is the probability
that fewer than 30 survive?
!
29.5 − 100(0.4)
P (X < 30) = P (X ≤ 29) = P Z<p
100(0.4)(0.6)
Ali Karimnezhad 43
Gamma Function
• Gamma distribution is based on the Gamma function

Z ∞
Γ(α) = xα−1e−xdx, α>0
0
• The Gamma function has the following properties:

– Γ(n) = (n − 1)Γ(n − 1), for a positive integer n;
– Γ(n) = (n − 1)!, for a positive integer n;
– Γ(α) = (α − 1)Γ(α − 1), for α > 1;
– Γ(1) = 1;
√
1
– Γ( 2 ) = π.
Ali Karimnezhad 44
Gamma and Exponential Distributions
• The continuous random variable X has a Gamma Distribution, with

parameters α and β, if its density function is given by
1 x
α−1 − β
f (x; α, β) = α x e , α > 0, β > 0, x > 0.
β Γ(α)
• We simply write X ∼ G(α, β).
• Special Case; Exponential Distribution

1 − βx
f (x; β) = e , β > 0, x > 0.
β
• We simply write X ∼ Exp(β).
Ali Karimnezhad 45
• The exponential and gamma distributions play an important role in both

queuing theory and reliability problems.
Ali Karimnezhad 46
Expectation and Variance for Gamma and Exponential

Distributions
If X ∼ G(α, β),
Z +∞
• Expecation E(X) = xf (x)dx = αβ.
0
2
Var(X) = E (X − αβ) = αβ 2.

• Variance
If X ∼ Exp(β),
Z +∞
• Expecation E(X) = xf (x)dx = β.
0
• Variance Var(X) = E (X − β) = β 2.
2

Ali Karimnezhad 47
Example:
Suppose that a system contains a certain type of component whose time,
in years, to failure is given by T . The random variable T is modeled nicely
by the exponential distribution with mean time to failure 5.
(a) What is the probability that a given components is still functioning at
the end of 8 years?
(b) If 5 of these components are installed in different systems, what is the

probability that at least 2 are still functioning at the end of 8 years?
Ali Karimnezhad 48
The Memoryless Property and Its Effect on the

Exponential Distribution
• If X ∼ Exp(β),
P (X ≥ t + t0|X ≥ t0) = P (X ≥ t).
Example:
Suppose that the amount of time one spends in a bank is exponentially
distributed with mean 10 minutes.
• What is the probability that a customer will spend more than 15 minutes
in the bank?
• What is the probability that a customer will spend more than 15 minutes
in the bank given that he is still in the bank after 10 minutes?
Ali Karimnezhad 49
Chi-Squared Distribution
• Another very important special case of the Gamma distribution is

obtained by letting α = ν2 and β = 2, where ν is a positive integer. The
result is called the Chi-Squared Distribution.
• ν called the degrees of freedom.
• The density function is given by

1 ν x
f (x; ν) = ν ν x 2 −1e− 2 , ν > 0, x > 0.
2 2 Γ( 2 )
• We simply write X ∼ χ2(ν).
• E(X) = ν and V ar(X) = 2ν.
Ali Karimnezhad 50

Ali Karimnezhad: Version September 30, 2015

Uploaded by

Copyright:

Available Formats

Ali Karimnezhad: Version September 30, 2015

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Ali Karimnezhad: Version September 30, 2015

Uploaded by

Copyright:

Available Formats

MAT2377

Version September 30, 2015

• These slides cover material from Chapter 3.

• In class, I may use a blackboard. I recommend reading these slides before

• I am planning to spend 4 lectures on this chapter.

• I am not re-writing the textbook. The reference book contains many

• Denote p as the probability of a success.

• Let X be a sucess indicator in a Bernoulli trial.

• Then, X ∼ Ber(p) with the probability function

A binomial experiment consists of n repeated independent Bernoulli trials,

• Denote p as the probability of a success.

• Let X be number of sucesses in a Binomial trial.

• Then, X ∼ B(n, p) with the probability function

• If n = 1, X ∼ B(1, p) or equivalently X ∼ Ber(p).

• The probability that a patient recovers from a rare blood disease is

• A large chain retailer purchases a certain kind of electronic device from a

Expectation and Variance for Binomial Distribution

(a) E(X) and Var(X).

(c) P (X ≤ 3) (tables can be used).

(a) If X ∼ B(n, p) then E(X) = np and Var(X) = np(1 − p) so

E(X) = 12 × 0.1 = 1.2 ; Var(X) = 12 × 0.1 × 0.9 = 1.08 .

• Also, since P (X ≤ 2) = P (X = 0) + P (X = 1) + P (X = 2) ≈ 0.8891

The Binomial experiment becomes a Multinomial experiment if we let each

Let X be number of Red balls in the drawing:

1. A random sample of size n is selected without replacement from N

2. Of the N items, k interesting items have a same property in common,

If X is number of type-k items in a random sample of size n, then

Define X as number of defective items. Then, using the Hypergeometric

Expectation and Variance for Hypergeometric

For a proof, see Appendix A.12 of the reference book.

A Geometric experiment possesses the following two properties:

1. A Bernoulli trail with probability p of success and probability of failure

2. The experiment is stopped when only one success occurs.

P (X = 5) = (0.01) (0.99)4 = 0.0096.

Expectation and Variance for Geometric Distribution

Negative Binomial Distribution

Negative Binomial Distribution

A Negative Binomial experiment possesses the following two properties:

1. A Bernoulli trail with probability p of success and probability of failure

2. The experiment is stopped when a kth success occurs.

Expectation and Variance for Negative Binomial

• The specified region could be a line segment, an area, a volume, or

1. The number of outcomes occurring in one time interval or specified

• The mean number of outcomes is computed from µ = λt, where t is the

• The probability function of the Poisson random variable X, representing

(c) What is the probability of observing at least 4 births in a given hour at

Expectation and Variance for Poisson Distribution

For a proof, see Appendix A.13 of the reference book.

Continuous Uniform Distribution

• The distribution is characterized by a density function that is “flat”, and

• The density function of the continuous uniform random variable X on

• The mean and variance of the uniform distribution are

• Its graph, called the normal curve, is a bell-shaped curve, which

where π = 3.14159... and e = 2.71828....

• The normal distribution is often referred to as the Gaussian distribution

• We simply write X ∼ N (µ, σ 2).

Properties of the Normal Curve

• The curve is symmetric about a vertical axis through the mean µ.

• The curve has its points of inflection at x = µ − σ and x = µ + σ;