MS13 CH 16

Chapter 16
Markov Processes
 Market Share Analysis

 Transition Probabilities
 Steady-State Probabilities
 Absorbing States
 Transition Matrix with Submatrices
 Fundamental Matrix
© 2011 Cengage Learning. All Rights Reserved. May not be scanned, copied
Slide 1
or duplicated, or posted to a publicly accessible website, in whole or in part.
Markov Processes
 Markov process models are useful in studying the

evolution of systems over repeated trials or sequential
time periods or stages.
• the promotion of managers to various positions
within an organization
• the migration of people into and out of various
regions of the country
• the progression of students through the years of
college, including eventually dropping out or
graduating
Slide 2
Markov Processes
 Markov processes have been used to describe the

probability that:
• a machine that is functioning in one period will
function or break down in the next period.
• a consumer purchasing brand A in one period will
purchase brand B in the next period.
Slide 3
Example: Market Share Analysis
Suppose we are interested in analyzing the market

share and customer loyalty for Murphy’s Foodliner and
Ashley’s Supermarket, the only two grocery stores in a
small town. We focus on the sequence of shopping trips
of one customer and assume that the customer makes
one shopping trip each week to either Murphy’s
Foodliner or Ashley’s Supermarket, but not both.
Slide 4
We refer to the weekly periods or shopping trips

as the trials of the process. Thus, at each trial, the
customer will shop at either Murphy’s Foodliner or
Ashley’s Supermarket. The particular store selected in
a given week is referred to as the state of the system in
that period. Because the customer has two shopping
alternatives at each trial, we say the system has two
states.
State 1. The customer shops at Murphy’s Foodliner.
State 2. The customer shops at Ashley’s Supermarket.
Slide 5
Suppose that, as part of a market research study, we

collect data from 100 shoppers over a 10-week period.
In reviewing the data, suppose that we find that of all
customers who shopped at Murphy’s in a given week,
90% shopped at Murphy’s the following week while
10% switched to Ashley’s.
Suppose that similar data for the customers who
shopped at Ashley’s in a given week show that 80%
shopped at Ashley’s the following week while 20%
switched to Murphy’s.
Slide 6
Transition Probabilities
 Transition probabilities govern the manner in which

the state of the system changes from one stage to the
next. These are often represented in a transition
matrix.
Slide 7
Transition Probabilities
 A system has a finite Markov chain with stationary

transition probabilities if:
• there are a finite number of states,
• the transition probabilities remain constant from
stage to stage, and
• the probability of the process being in a particular
state at stage n+1 is completely determined by the
state of the process at stage n (and not the state at
stage n-1). This is referred to as the memory-less
property.
Slide 8
 Transition Probabilities
pij  probability of making a transition from state i

in a given period to state j in the next period
p11 p12 0.9 0.1

P= =
p21 p22 0.2 0.8
Slide 9
 State Probabilities
Murphy’s
.9 P = .9(.9) = .81
Murphy’s
Ashley’s
.9
Murphy’s .1 P = .9(.1) = .09
Murphy’s P = .1(.2) = .02
Ashley’s .2
.1
Ashley’s P = .1(.8) = .08
.8
Slide 10
 State Probabilities for Future Periods

Beginning Initially with a Murphy’s Customer
 State Probabilities for Future Periods

Beginning Initially with an Ashley’s Customer
Slide 11
Steady-State Probabilities
 The state probabilities at any stage of the process can

be recursively calculated by multiplying the initial
state probabilities by the state of the process at stage
n.
 The probability of the system being in a particular
state after a large number of stages is called a steady-
state probability.
Slide 12
Steady-State Probabilities
 Steady state probabilities can be found by solving the

system of equations P =  together with the
condition for probabilities that i = 1.
• Matrix P is the transition probability matrix
• Vector  is the vector of steady state probabilities.
Slide 13
Let 1 = long run proportion of Murphy’s visits

2 = long run proportion of Ashley’s visits
Then,
.9 .1
[ ] = [ ]
.2 .8
continued . . .
Slide 14
 +  =  (1)

 +  =  (2)
 +  = 1 (3)
Substitute  = 1 -  into (1) to give:

1 =  + .2(1 - 1) = 2/3 = .667
Substituting back into (3) gives:
 = 1/3 = .333.
Slide 15
Thus, if we have 1000 customers in the system, the

Markov process model tells us that in the long run,
with steady-state probabilities 1  .667 and 2  .333,
21
3
667 customers will be Murphy’s and 333 customers
will be Ashley’s.
Slide 16
Suppose Ashley’s Supermarket is contemplating an

advertising campaign to attract more of Murphy’s
customers to its store. Let us suppose further that
Ashley’s believes this promotional strategy will increase
the probability of a Murphy’s customer switching to
Ashley’s from 0.10 to 0.15.
Slide 17
 Revised Transition Probabilities
Slide 18
 Revised Steady-State Probabilities
 +  =  (1)

 +  =  (2)
 +  = 1 (3)
Substitute  = 1 -  into (1) to give:

1 =  + .20(1 - 1) = .57
Substituting back into (3) gives:
 = .43.
Slide 19
Suppose that the total market consists of 6000

customers per week. The new promotional strategy will
increase the number of customers doing their weekly
shopping at Ashley’s from 2000 to 2580.
If the average weekly profit per customer is $10,
the proposed promotional strategy can be expected to
increase Ashley’s profits by $5800 per week. If the cost
of the promotional campaign is less than $5800 per
week, Ashley should consider implementing the
strategy.
Slide 20
Example: North’s Hardware
Henry, a persistent salesman, calls North's

Hardware Store once a week hoping to speak with
the store's buying agent, Shirley. If Shirley does not
accept Henry's call this week, the probability she
will do the same next week is .35. On the other hand,
if she accepts Henry's call this week, the probability
she will not do so next week is .20.
Slide 21
 Transition Matrix
Next Week's Call

Refuses Accepts
This Refuses .35 .65
Week's
Call Accepts .20 .80
Slide 22
Question
How many times per year can Henry expect to
talk to Shirley?
Answer
To find the expected number of accepted calls
per year, find the long-run proportion
(probability) of a call being accepted and
multiply it by 52 weeks.
continued . . .
Slide 23
Answer (continued)
Let 1 = long run proportion of refused calls
2 = long run proportion of accepted calls
Then,
.35 .65
[ ] = [ ]
.20 .80
continued . . .
Slide 24
Answer (continued)
 +  =  (1)

 +  =  (2)
 +  = 1 (3)
Solve for  and 
continued . . .
Slide 25
Answer (continued)
Solving using equations (2) and (3). (Equation 1 is
redundant.) Substitute  = 1 -  into (2) to give:
.65(1 - 2) +  = 2
This gives  = .76471. Substituting back into
equation (3) gives  = .23529.
Thus the expected number of accepted calls per
year is:
(.76471)(52) = 39.76 or about 40
Slide 26
 State Probability
Question
What is the probability Shirley will accept
Henry's next two calls if she does not accept his
call this week?
Slide 27
Answer Refuses
.35 P = .35(.35) = .1225
Refuses
Accepts
.35
Refuses .65 P = .35(.65) = .2275
Refuses P = .65(.20) = .1300
Accepts .20
.65
Accepts P = .65(.80) = .5200
.80
Slide 28
Question
What is the probability of Shirley accepting
exactly one of Henry's next two calls if she accepts
his call this week?
Slide 29
Answer
The probability of exactly one of the next two calls
being accepted if this week's call is accepted can be
found by adding the probabilities of (accept next week
and refuse the following week) and (refuse next week
and accept the following week) =
.13 + .16 = .29
Slide 30
Absorbing States
 An absorbing state is one in which the probability that

the process remains in that state once it enters the
state is 1.
 If there is more than one absorbing state, then a
steady-state condition independent of initial state
conditions does not exist.
Slide 31
Transition Matrix with Submatrices
 If a Markov chain has both absorbing and

nonabsorbing states, the states may be rearranged so
that the transition matrix can be written as the
following composition of four submatrices: I, 0, R,
and Q:
I 0
R Q
Slide 32
Transition Matrix with Submatrices
I = an identity matrix indicating one always

remains in an absorbing state once it is reached
0 = a zero matrix representing 0 probability of
transitioning from the absorbing states to the
nonabsorbing states
R = the transition probabilities from the
nonabsorbing states to the absorbing states
Q = the transition probabilities between the
nonabsorbing states
Slide 33
Example: Jetair Aerospace
The vice president of personnel at Jetair Aerospace

has noticed that yearly shifts in personnel can be
modeled by a Markov process. The transition matrix is:
Next Year
Same Pos. Promotion Retire Quit Fired
Current Year
Same Position .55 .10 .05 .20 .10
Promotion .70 .20 0 .10 0
Retire 0 0 1 0 0
Quit 0 0 0 1 0
Fired 0 0 0 0 1
Slide 34
 Transition Matrix
Next Year
Retire Quit Fired Same Promotion
Current Year
Retire 1 0 0 0 0
Quit 0 1 0 0 0
Fired 0 0 1 0 0
Same .05 .20 .10 .55 .10

Promotion 0 .10 0 .70 .20
Slide 35
Fundamental Matrix
 The fundamental matrix, N, is the inverse of the

difference between the identity matrix and the Q
matrix.
N = (I - Q )-1
 (I - Q )-1 = I + Q + Q2 + Q3 + …
Slide 36
Slide 37
 Fundamental Matrix
The determinant, d = aa - aa
= (.45)(.80) - (-.70)(-.10) = .29
Thus,
.80/.29 .10/.29 2.76 .34

N= =
.70/.29 .45/.29 2.41 1.55
Slide 38
NR Matrix
 The NR matrix is the product of the fundamental (N)

matrix and the R matrix.
N = (I - Q )-1 = I + Q + Q2 + Q3 + …
NR = R + QR + Q2R + Q3R + …
 It gives the probabilities of eventually moving from
each nonabsorbing state to each absorbing state.
 Multiplying any vector of initial nonabsorbing state
probabilities by NR gives the vector of probabilities
for the process eventually reaching each of the
absorbing states. Such computations enable
economic analyses of systems and policies.
Slide 39
 NR Matrix
The probabilities of eventually moving to the
absorbing states from the nonabsorbing states are
given by:
2.76 .34 .05 .20 .10

NR = x
2.41 1.55 0 .10 0
Slide 40
 NR Matrix (continued)
Retire Quit Fired
Same .14 .59 .28

NR =
Promotion .12 .64 .24
Slide 41
 Absorbing States
Question
What is the probability of someone who was just
promoted eventually retiring? . . . quitting? . . .
being fired?
Slide 42
 Absorbing States (continued)

Answer
The answers are given by the bottom row of the
NR matrix. The answers are therefore:
Eventually Retiring = .12

Eventually Quitting = .64
Eventually Being Fired = .24
Slide 43

MS13 CH 16

Uploaded by

Copyright:

Available Formats

MS13 CH 16

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

MS13 CH 16

Uploaded by

Copyright:

Available Formats

Chapter 16

 Market Share Analysis

 Markov process models are useful in studying the

 Markov processes have been used to describe the

Suppose we are interested in analyzing the market

We refer to the weekly periods or shopping trips

Suppose that, as part of a market research study, we

 Transition probabilities govern the manner in which

 A system has a finite Markov chain with stationary

pij  probability of making a transition from state i

p11 p12 0.9 0.1

 State Probabilities for Future Periods

 State Probabilities for Future Periods

 The state probabilities at any stage of the process can

 Steady state probabilities can be found by solving the

Let 1 = long run proportion of Murphy’s visits

 +  =  (1)

Substitute  = 1 -  into (1) to give:

Thus, if we have 1000 customers in the system, the

Suppose Ashley’s Supermarket is contemplating an

 Revised Transition Probabilities

 Revised Steady-State Probabilities

 +  =  (1)

Substitute  = 1 -  into (1) to give:

Suppose that the total market consists of 6000

Henry, a persistent salesman, calls North's

Next Week's Call

 +  =  (1)

Solve for  and 

.13 + .16 = .29

 An absorbing state is one in which the probability that

 If a Markov chain has both absorbing and

I = an identity matrix indicating one always

The vice president of personnel at Jetair Aerospace

Same .05 .20 .10 .55 .10

 The fundamental matrix, N, is the inverse of the

.80/.29 .10/.29 2.76 .34

 The NR matrix is the product of the fundamental (N)

2.76 .34 .05 .20 .10

Retire Quit Fired

Same .14 .59 .28

 Absorbing States (continued)

Eventually Retiring = .12

You might also like