Markov Analysis

Markov Analysis
Introduction
Markov Analysis
A
technique dealing with probabilities of

future occurrences with currently known
probabilities
Numerous
applications in
Business (e.g., market share analysis),

Bad debt prediction
University enrollment predictions
Machine breakdown prediction
Markov Analysis
Matrix of Transition Probabilities
Shows the likelihood that the system will

change from one time period to the next
This is the Markov Process.

It enables the prediction of future states or
conditions.
States and State

Probabilities
States are
used to identify all possible conditions of a process or

system
A
system can exist in only one state at a time.

Examples include:
Working and broken states of a machine
Three shops in town, with a customer able to patronize
one at a time
Courses in a student schedule with the student able to
occupy only one class at a time
Assumptions of Markov
Analysis
1.
A finite number of possible states
2.
Probability of changing states remains the
same over time

3. Future state predictable from previous state
and the transition matrix
4. Size and states of the system remain the same
during analysis
5. States are collectively exhaustive
All possible states have been identified
6. States are mutually exclusive

Only one state at a time is possible
States and State Probabilities continued

1.
Identify all states.
2.
Determine the probability that the system

is in this state.
This information is placed into a vector of state

probabilities.
(i) = vector of state probabilities for period i
= (n)
where
n = number of states
n = P (being in state 1, 2, , state n)
Most of the time, problems deal with more than one
item!

Three Departmental Stores example:
100,000 customers monthly for the 3 grocery stores
State 1 = store 1 = 40,000/100,000 = 40%
State 2 = store 2 = 30,000/100,000 = 30%
State 3 = store 3 = 30,000/100,000 = 30%
vector of state probabilities:

(1) = (0.4,0.3,0.3)
where
(1) = vector of state probabilities in period 1
= 0.4 = P (of a person being in store 1)

Three Departmental Stores example, continued:
Probabilities in the vector of states for the stores

represent market share for the first period.
In period 1, the market shares are
Store 1: 40%
Store 2: 30%
Store 3: 30%
But, every month, customers who frequent one store

have a likelihood of visiting another store.
Customers from each store have different probabilities

for visiting other stores.

Three Grocery Store example, continued:
Store-specific customer probabilities for visiting a
store in the next month:
Store 1:
Store 2:
Return to Store 1 = 80% Visit Store 1 = 10%

Visit Store 2 = 10%
Return to Store 2 = 70%
Visit Store 3 = 10%
Visit Store 3 = 20%
Store 3:
Visit Store 1 = 20%
Visit Store 2 = 20%
Return to Store 3 = 60%

Three Grocery Store example, continued:
Combining the starting market share with the
customer probabilities for visiting a store next period
yields the market shares in the next period:
Initial
Share P(1) P(2) P(3)
Store 1:
40% 80% 10% 10%
Next Period: 32%

Store 2:
4%
30% 10% 70% 20%
Next Period:
Store 3:
4%
3% 21%
6%
30% 20% 20% 60%
Next Period:
6%
6% 18%
New Shares: 41% 31% 28% = 100%
Matrix of Transition
Probabilities
To calculate periodic changes, it is much more
convenient to use
a matrix of transition probabilities.
a matrix of conditional probabilities of being in a
future state given a current state.
Let Pij = conditional probability of being in state j in

the future given the current state of i, P (state j at
time = 1 | state i at time = 0)
For example, P12 is the probability of being in state
2 in the future given the event was in state 1 in
the prior period
Matrix of Transition Probabilities continued

Let P = matrix of transition probabilities
Row
Sum
P11 P12 P13

P=
P1n
P21 P22 P23 ***** P2n
*
*
*
*
Pm1
*****
******
Pmn
*
*
*
*
Important:
Each row must sum to 1.
But, the columns do NOT necessarily sum to 1.
Matrix of Transition Probabilities continued

Three Grocery Stores, revisited
The previously identified transitional probabilities
for each of the stores can now be put into a
matrix:
0.8
P=
0.1
0.1
0.1
0.7
0.2
0.2
0.2
0.6
Row 1 interpretation:
0.8 = P11 = P (in state 1 after being in state 1)
Predicting Future Market Shares

Grocery Store example
A purpose of Markov analysis is to predict the
future
Given the
1. vector of state probabilities and
2. matrix of transitional probabilities.
It is easy to find the state probabilities in the

future.
This type of analysis allows the computation of the

probability that a person will be at one of the
grocery stores in the future.
Since this probability is equal to market share, it is
Predicting Future Market Shares continued
Grocery Store example

When the current period is 0, finding the state
probabilities for the next period (1) can be found
using:
(1) = (0)P
Generally, in any period n, the state probabilities
for
period n+1 can be computed as:
(n+1) = (n)P
Predicting Future States continued

= state probabilities
(0) .4 .3 .3
.8
P .1
.2
.1
.7
.2
.1
.2
.6
(1) (0) P
(1) = .4 .3 .3
.8
.1
.2
.1
.7
.2
.1
.2
.6
( 1 ) [ (.4*.8 + .3*.1 + .3*.2),

(.4*.1 + .3*.7 + .3*.2),
]
(.4*.1 + .3*.2 + .3*.6)
(1) = [ 0.41 0.31 0.28]
Predicting Future Market Shares continued
In general,
(n) = (0)Pn
Therefore, the state probabilities n periods
in the future can be obtained from the
current state probabilities and the
matrix of transition probabilities.
Another Example of Markov

Analysis: The Machine Operations
States and State Probabilities

For example:
If dealing with only 1 machine, given the fact

that it is currently functioning correctly.
The vector of states can then be shown.

(1) = (1,0)
where
(1) = vector of states for the machine in period 1
= 1 = P (being in state 1) = P (machine working)
= 0 = P (being in state 2) = P (machine broken)
Markov Analysis of Machine Operations
P=
0.8 0.2
0.1 0.9
where
P11 = 0.8 = probability of machine working this period if
working last period
P12 = 0.2 = probability of machine not working this period
if working last
P21 = 0.1 = probability of machine working
this period if not working last
P22 = 0.9 = probability machine not working this period if
not working last
Markov Analysis of Machine Operations continued
What is the probability the machine will be working

next month?
(1) = (0)P
= (1,0) 0.8 0.2

0.1 0.9
= [(1)(0.8)+(0)(0.1), (1)(0.2)+(0)(0.9)]
= (0.8, 0.2)
Thus, if the machine works this month, then
there is
an 80% chance that it will be working
next month and
a 20% chance it will be broken.
Markov Analysis of Machine Operations continued
What is the probability the machine will be working

in two months?
(2) = (1)P
0.8 0.2
= (0.8,
0.1 0.9
0.2)
= [(0.8)(0.8)+(0.2)(0.1), (0.8)(0.2)+(0.2)(0.9)]
= (0.66,
Thus,
0.34) if the machine works next month, then
in two months there is
a 66% chance that it will be working
and
a 34% chance it will be broken.
Equilibrium State and

Absorbing State (Only
Understanding of Concepts
Required/No Maths Required)
Equilibrium Conditions
Equilibrium
state probabilities are the long-run

average probabilities for being in each state.
Equilibrium
conditions exist if state

probabilities do not change after a large
number of periods.
At
equilibrium, state probabilities for the next

period equal the state probabilities for current
period.
Equilibrium Conditions continued

One
way to compute the equilibrium share of the

market is to use Markov analysis for a large
number of periods and see if the future amounts
approach stable values.
On
the next slide, the Markov analysis is repeated

for 15 periods for the machine example.
By
the 15th period, the share of time the machine

spends working and broken is around 66% and
34%, respectively.
Machine Example: Periods to Reach

Equilibrium
Period
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
State 1
State 2
1.0
.8
.66
.562
.4934
.44538
.411766
.388236
.371765
.360235
.352165
.346515
.342560
.339792
.337854
0.0
.2
.34
.438
.5066
.55462
.588234
.611763
.628234
.639754
.647834
.653484
.657439
.660207
.662145
The Markov Process

Equilibrium Conditions
(n)
Current
State
P
Matrix of
Transition
(n+1)
New
State
Equilibrium Equations
(i 1) (i ) P
Assume:
(i) 1
p
2 , P 11
p21
p12
p22
Then:
1 2 1P11 2 P21 1P12 2 P22
or :
1 1P11 2 P21,
2 1P12 2 P22
Therefore:
2 p 21
1 p12
1
2
and
1 p11
1 p22
continued
It is always true that
(next period) = (this period) P
or
(n+1) = (n) P
At Equilibrium:
* (n+1) = (n) = (n) P
* (n) = (n) P
Dropping the n term:
=P
Equilibrium Equations continued

Machine Breakdown example
At Equilibrium:
=P
0.8 0.2
(1, 2) = (1,
0.1 0.9
2)
Applying matrix multiplication:
(1, 2) = [(1)(0.8) + (2)(0.1), (1)(0.2) + (2)(0.9)]
Multiplying through yields:

1 = 0.8 1 + 0.1 2
2 = 0.2 1 + 0.9 2
continued
Machine Breakdown example
The state probabilities must sum to 1,
therefore:
s = 1
In this example, then:
1 + 2 = 1
In a Markov analysis, there are always n state

equilibrium equations and 1 equation of state
probabilities summing to 1.
continued
Machine Breakdown Example
Summarizing the equilibrium equations:
1 = 0.8 1 + 0.1 2
2 = 0.2 1 + 0.9 2
1 + 2 = 1
Solving by simultaneous equations:
1 = 0.333333
2 = 0.666667
Therefore, in the long-run, the machine
will be functioning 33.33% of the time
and broken down 66.67% of the time.
Absorbing States
Any
state that does not have a probability

of moving to another state is called an
absorbing state.
If an entity is in an absorbing state now,
the probability of being in an absorbing
state in the future is 100%.
An
example of such a process is accounts

receivable.
Bills are either paid, delinquent, or written off as
bad debt.
Once paid or written off, the debt stays paid or
Absorbing States
Accounts Receivable example
The possible states are
Paid
Bad debt
Less than 1 month old debt
1 to 3 months old debt
transition matrix for this would look similar to:

Paid
Paid
Bad
<1
1-3
1
0
0.6
0.4
Bad
0
1
0
0.1
<1
0
0
0.2
0.3
1-3
0
0
0.2
0.2
Markov Process
Fundamental Matrix
0
P=
1
0
0.6
0.4 A
0
1
0
0
0
0
0.2
0.3 B
0
0
0.2
0.2
Partition the probability matrix into 4 quadrants to

make 4 new sub-matrices:
I, 0, A, and B
Markov Process
Fundamental Matrix continued
Let P
A
Where I = Identity matrix,

and 0 = Null matrix
Then
F I B
Once F is found, multiply by the A matrix: FA

The FA indicates the probability that an amount in one of the
non-absorbing states will end up in one of the absorbing
states.
Markov Process
Fundamental Matrix continued
Once the FA matrix is found, multiply by the
M vector, which is the starting values for the
non-absorbing states, MFA,
where
M = (M1, M2, M3, Mn)
The resulting vector will indicate how many
observations end up in the first nonabsorbing state and the second nonabsorbing state, respectively.

Markov Analysis

Uploaded by

Copyright:

Available Formats

Markov Analysis

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Markov Analysis

Uploaded by

Copyright:

Available Formats

Markov Analysis

technique dealing with probabilities of

Business (e.g., market share analysis),

Shows the likelihood that the system will

This is the Markov Process.

States and State

used to identify all possible conditions of a process or

system can exist in only one state at a time.

A finite number of possible states

Probability of changing states remains the

same over time

6. States are mutually exclusive

States and State Probabilities continued

Identify all states.

Determine the probability that the system

This information is placed into a vector of state

States and State Probabilities continued

State 1 = store 1 = 40,000/100,000 = 40%

State 2 = store 2 = 30,000/100,000 = 30%

State 3 = store 3 = 30,000/100,000 = 30%

vector of state probabilities:

States and State Probabilities continued

Probabilities in the vector of states for the stores

In period 1, the market shares are

But, every month, customers who frequent one store

Customers from each store have different probabilities

States and State Probabilities continued

Return to Store 1 = 80% Visit Store 1 = 10%

Return to Store 2 = 70%

Visit Store 3 = 10%

Visit Store 3 = 20%

States and State Probabilities continued

40% 80% 10% 10%

Next Period: 32%

30% 10% 70% 20%

30% 20% 20% 60%

New Shares: 41% 31% 28% = 100%

Let Pij = conditional probability of being in state j in

Matrix of Transition Probabilities continued

P11 P12 P13

P21 P22 P23 ***** P2n

Matrix of Transition Probabilities continued

Predicting Future Market Shares

It is easy to find the state probabilities in the

This type of analysis allows the computation of the

Since this probability is equal to market share, it is

Predicting Future Market Shares continued

Grocery Store example

Predicting Future States continued

( 1 ) [ (.4*.8 + .3*.1 + .3*.2),

Predicting Future Market Shares continued

current state probabilities and the

matrix of transition probabilities.

Another Example of Markov

States and State Probabilities

If dealing with only 1 machine, given the fact

The vector of states can then be shown.

Markov Analysis of Machine Operations

Markov Analysis of Machine Operations continued

What is the probability the machine will be working

( 1 ) [ (.4.8 + .3.1 + .3*.2),