Probability and Statistics

Lec-4,5,6
Elements of Probability

Dr. Farkhanda Afzal

Dept. of Humanities and Basics Sciences
MCS, NUST
[email protected]
Contents
 Basic probability concepts
– Sample spaces and events, simple probability, joint
probability

 Conditional probability
– Statistical independence, marginal probability

 Bayes’s theorem
Sample Space

 Probability: quantify the variability in the outcome of

any experiment whose exact outcome cannot be
predicted with certainty.

 The Space of outcome!!

 Sample space: a set of all possible outcomes of an

experiment. Usually denoted by S.

3
Sample Spaces

 Collection of all possible outcomes

– e.g.: All six faces of a die:

– e.g.: All 52 cards in a deck:

Example 2.1
Consider the experiment of tossing a die. If we are interested
in the number that shows on the top face, the sample space
would be

S1 

S1 

If we are interested only in whether the number is even or odd,

the sample space is simply

S2 
5
Example 2.2
An experiment consists of flipping a coin and then flipping it a
second time if a head occurs. If a tail occurs on the first flip,
then a die is tossed once

T1

T4

6 T6

S={HH, HT, T1, T2, T3, T4, T5, T6}

6
Example 2.3
Suppose that three items are selected at random from a
manufacturing process. Each item is inspected and classified
defective, D, or non-defective, N.

7
Sample Space by Rule
Can we write
No. of elements in a sample sample space
space is very large as a set?

1. For example, if the possible outcomes of an experiment are the set of

cities in the world with a. population over 1 million

Read as “such that”

2. all points (x,y) on the boundary or the interior of a circle of radius 2

with center at the origin


S  ( x, y) |x  y  4
2 2
 2

8
Events

An event is a subset
of a sample space

9
Example 2.4
Given the sample space S = {t | t > 0}, where t is the life in years of a certain
electronic component, then the event A that the component fails before the
end of the fifth year is the subset,

What is event A?

A = {t | 0<t <5}

10
Complement of Event

The complement of an event A with

respect to S is the subset of all elements
of S that are not in A. We denote the
complement of A by the symbol A'.

11
Example 2.6

 Consider the sample space

S = {book, catalyst, cigarette, ash, engineer, rivet}

 Let A = {catalyst, rivet, book, cigarette}.

 Then the complement of A is

A' = {ash, engineer}

12
Simple and Joint Event

 Simple event
– Outcome from a sample space with one
characteristic
– e.g.: A red card from a deck of cards
 Joint event
– Involves two characteristics/outcomes
simultaneously
– e.g.: An ace that is also red from a deck of
cards
Simple Events

The Event of a Triangle ?

There are 5 triangles in this collection of 18 objects

Joint Events

The event of a triangle AND blue in color ?

2 triangles that are blue

Special Events
 Impossible event Null Event
e.g.: Club & diamond on one card
draw 
 Mutually exclusive events
– Two events cannot occur together
– e.g. A: queen of diamonds; B: queen of
clubs
 Events A and B are mutually
exclusive
Special Events
 Collectively exhaustive events (continued)

– One of the events must occur

– The set of events covers the whole sample space
– e.g. A: all the aces; B: all the black cards; C: all the
diamonds; D: all the hearts
 Events A, B, C and D are collectively
exhaustive
 Events B, C and D are also collectively
exhaustive
Compound Events
 Define an event set of more than two possible
equally likely elementary events.

 Compound event: An event that consists of a

set of elementary events.

 The compound event occurs if any of the

elementary events occurs.
e.g. if a die is tossed

S

E1= {2,4,6} and E2= {1,3,5}

18
Compound Events
Compound Events

Subsets containing more

than one possible
event/outcome

Review Set
Theory
Requires knowledge of Set
Theory

19
Set Theory: Terminologies
 Set: A Set is any well defined collection of
“objects.”
 Elements: The elements of a set are the objects in a
set.
 Notations: Usually we denote sets with upper-case
letters, elements with lower-case letters. The
following notation is used to show set membership
 xA means that x is a member of the set A
 xA means that x is not a member of the set A.

20
Set Representation
 List of Items
A= 1,2,3,4,5,6

 Give a verbal description

“A is the set of all integers from 1 to 6, inclusive”

 Give a mathematical inclusion rule

A= Integers x 1  x  6

21
Some Special Sets

 The Null Set or Empty Set: This is a set with

no elements, often symbolized by


 The Universal Set: This is the set of all
elements currently under consideration, and is
often symbolized by


Membership Relationships

 A Subset:

A  B “A is a subset of B”

We say “A is a subset of B” if x  A  x  B ,
i.e., all the members of A are also members
of B. The notation for subset is very similar to
the notation for “less than or equal to,” and
means, in terms of the sets, “included in or
equal to.”
Membership Relationships

 Proper Subset:

A B “A is a proper subset of B”
We say “A is a proper subset of B” if all the
members of A are also members of B, but in
addition there exists at least one element c such
that c  B but c  A . The notation for subset
is very similar to the notation for “less than,” and
means, in terms of the sets, “included in but not
equal to.”
Combining Sets – Set Union

A B
 “A union B” is the set of all elements that are in A, or B,
or both.

 This is similar to the logical “or” operator.

Combining Sets – Set Intersection

A B
 “A intersect B” is the set of all elements that are in both A
and B.
 This is similar to the logical “and”
Set Complement

A
 “A complement,” or “not A” is the set of all elements not
in A.
 The complement operator is similar to the logical not,
and is reflexive, that is,

A A
Set Difference

A B
 The set difference “A minus B” is the set of elements that
are in A, with those that are in B subtracted out. Another
way of putting it is, it is the set of elements that are in A,
and not in B, so

A B  A B

{1,2,3,4,5,6}  {2,3,5,7,9,11} =
___________
{1,4,6} The complement of B with respect to A
Disjointedness
Help, I’ve
 Two sets A, B are called been
disjointed!
disjoint (i.e., unjoined)
iff their intersection is
empty. (AB=)

 Example: the set of even

integers is disjoint with
the set of odd integers.
Examples

  {1,2,3,4,5,6}

A  {1,2,3} B  {3,4,5,6}

A  B  {3} A  B  {1,2,3,4,5,6}

B  A  {4,5,6} B  {1,2}
Venn Diagrams

 Venn Diagrams use topological areas to stand for sets.

A B

AB
Venn Diagrams

 Try this one!

A B

AB
Venn Diagrams

 Here is another one

A B

AB
Mutually Exclusive and Exhaustive Sets

 Exhaustive Sets: We say that a group of sets is

exhaustive of another set if their union is equal to that
set. For example, if A  B  C
we say that A and B are exhaustive with respect to C.

 Mutually Exclusive Sets: We say that two sets A and B

are mutually exclusive if A  B   , that is, the sets
have no elements in common.
Set Partition
 Definition: We say that a group of sets
partitions another set if they are mutually
exclusive and exhaustive with respect to that
set. When we “partition a set,” we break it
down into mutually exclusive and exhaustive
regions, i.e., regions with no overlap. The
Venn diagram next should help you get the
picture. In this diagram, the set A (the
rectangle) is partitioned into sets W,X, and Y.
Set Partition

W
X Y
Some Test Questions

A   ?
Some Test Questions

A  A=?
Some Test Questions

A   ?
Some Test Questions

A  A=?
Some Test Questions

AA ?
Some Test Questions

A  ?
Some Test Questions

  ?
Some Test Questions

If A  B then
AB ?
Some Test Questions

If A  B then
AB ?
Laws

 Identity: A=A AU=A

 Domination: AU=U A=

 Idempotent: AA = A = AA

 Double complement: A A
 Commutative: AB=BA AB=BA

 Associative: A(BC)=(AB)C
A(BC)=(AB)C
DeMorgan’s Law

A B  A  B
A B  A  B
Counting Problem

 Most of time, probability problem is solved by counting

the number of points in the sample space without
actually listing each element

Counting
Sample Points

48
Fundamental Counting Principal
If an operation is performed in ‘n1’
ways, and if for each of these ways a
Multiplication
Rule
second operation can be performed in
‘n2’ ways, the two operations can be
performed in ‘n1×n2’ ways

Example
How many sample points are there in the sample space when a
pair of dice is thrown once?

Sol:
n1=6
n2=6
N=n1 × n2 = 6 ×6=36

49
 Fundamental Counting Principal

A student is to roll a die and flip a coin. How

Example
many possible outcomes will there be?

1H 2H 3H 4H 5H 6H
1T 2T 3T 4T 5T 6T

12 outcomes
2 outcomes

6*2 = 12 outcomes

6 outcomes
50
Example 2.14

51
Counting Arrangements

 we are interested in a sample space that contains

elements as all possible orders or arrangements of a
group of objects

Permutation

52
Permutations

Order
Matters !
A Permutation is an arrangement
of items in a particular order.

To find the number of Permutations of n items,

we can use the Fundamental Counting
Principle or factorial notation.

53
Permutations
Example

The number of ways to arrange the letters ABC

____ ____ ____

Number of choices for first blank? 3 ____ ____

Number of choices for second blank? 3 2 ___
Number of choices for third blank? 3 2 1

3*2*1 = 6 3! = 3*2*1 = 6
ABC ACB BAC BCA CAB CBA
Permutations
To find the number of Permutations of n
items chosen r at a time, you can use the
formula

n!
P  where 0  r  n .
n r (n  r )!
e.g. if ‘3’ items are chosen from ‘5’ items
5! 5!
5 P3    5 * 4 * 3  60
(5  3)! 2!
Permutations
Example

From a club of 24 members, a President, Vice

President, Secretary, Treasurer and Historian are to be
elected. In how many ways can the offices be filled?

Try Now
Permutations
Example

From a club of 24 members, a President, Vice

President, Secretary, Treasurer and Historian are to be
elected. In how many ways can the offices be filled?

24! 24!
24 p5   
( 24  5)! 19!
24 * 23 * 22 * 21* 20  5,100,480
Combinations

Order Does
not Matters !
A Combination is an arrangement of
items in which order does not matter

Since the order does not matter in combinations,

there are fewer combinations than
permutations. The combinations are a "subset" of
the permutations.
Combinations

To find the number of Combinations of

n items chosen r at a time, you can use
the formula

n!
C  where 0  r  n .
n r r! ( n  r )!
Combinations
Example

A student must answer 3 out of 5 essay

questions on a test. In how many different
ways can the student select the questions?

Answer Now
Combinations
Example

A student must answer 3 out of 5 essay

questions on a test. In how many different
ways can the student select the questions?

5! 5! 5 * 4
5 C3     10
3! (5  3)! 3!2! 2 * 1
Useful Results
N N
0! = 1,   = 1,   = 1
0 n
N N  N  N(N- 1) N
  = 1,   = N,   = ,  =1
0  1 2 2 N
N  N   24   24  24 23 22
 =  , e.g.,   =   =
 r   N- r   21   3  321

The one with the smaller of r or N- r will be easier to compute.

62
Probability
 Probability is the numerical
measure of the likelihood
that an event will occur

 Lies Between 0 &1

Probability

 An “event” E will occur or not occur

 P(E) is a number that equals the probability that

E will occur.

 By convention, 0 < P(E) < 1.

 E' = the event that E does not occur

 P(E') = the probability that E does not occur.

64
Sample Space and Probability

65
 Computing Probabilities

 The probability of an event E:

number of event outcomes

P( E ) 
total number of possible outcomes in the sample space
X

T e.g. P( ) = 2/36
(There are 2 ways to get one 6 and the other 4)

 Each of the outcomes in the sample space is equally

likely to occur
Computing Joint Probability:
Pick a Card

 Event A = Diamond: P(Diamond) = 13/52

2♦ 3♦ 4♦ 5♦ 6♦ 7♦ 8♦ 9♦ 10♦ J♦ Q♦ K♦ A♦

 Event B = Ace: P(Ace) = 4/52

A♦ A♥ A♣ A♠

 Event A or B = Diamond or Ace

P(Diamond or Ace)
= P(Diamond) + P(Ace) – P(Diamond Ace)
= 13/52 + 4/52 – 1/52 = 16/52

67
Rules (Axioms) of Probability

 Axiom 1. 0  P( A)  1 for each event A in S.

 Axiom 2. P(S) = 1

 Axiom 3. If A and B are mutually exclusive events in S,

then
P( A  B)  P( A)  P( B)

68
Propositions of Probability
Proposition 1

P( A)  1  P( A)
Proposition 2

If A  B , then P( A)  P( B)

Proposition 3 If A and B are any events in S, then

P( A  B)  P( A)  P( B)  P( A  B)
69
Propositions of Probability
Proposition 4

If A and B are independent

P( A  B)  P( A).P( B)

70
Example

71
Example (Cont.)

72
Summary

 If P(E) = 0, then E cannot (will not) occur

 If P(E) = 1, then E must (will) occur

 E and E' are exhaustive – either E or E' will occur.

 Something will occur, P(E) + P(E') = 1

 Only one thing can occur. If E occurs, then E' will

not occur – E and E' are exclusive.

73