Review

Review
1. (a) State the definitions of concepts related to Markov chain: stationary distribution, irreducibility,
recurrent state, transient state, ergodic theorem, convergence of transition probability after in-
finitive steps.
(b) State the definitions of Exponential distribution, Poisson distribution and Poisson processes.
(c) State the definitions of conditional expectation, filtration, stopping times, (sub/super)-martingales.
State the optional sampling theorem.
2. (a) Let X and Y are identicaly distributed and independent random variables.
x x1 x2
P[X = x] p 1−p
Suppose 0 < x1 < x2 and p ∈ (0, 1). Let Z = XY . Compute E[Z|X = x1 ], and E[X|Z].
(b) Let X and Y be two random variables with joint density function
(
6e−2x−3y if x, y > 0,
fX,Y (x, y) =
0 otherwise.
Find the distribution of X and Y . Compute E[Z|Y ] and E[Y |Z] where Z = X + Y .
3. Let (Xn )n≥1 be a sequence of independent random variables.
b
P[Xn = −a] = 1 − P[Xn = b] = ,
a+b
for some positive constants a and b. Let Fn = σ{X1 , . . . , Xn }, S0 = 0, and Sn = X1 + . . . + Xn for n ≥ 1.
(a) Show that (Sn )n≥0 is a Markov chain. Find its transition probability.
(b) Show that (Sn , Fn ) is a martingale and (Sn2 , Fn ) is a sub-martingale
(c) Find the Doob decomposition of Sn2 .
4. Let N (t) be a Poisson process with rate λ > 0. Let Tn denote the time of the nth arrival.
(a) Compute the conditional probabilities:

P(N (m) = m + n | N (n) = m) and P(N (n) = n | N (m) = m) for some integers m > n > 0
(b) Compute E[Tn ] and E[Tm+n |N (n) = m] for some positive integers m and n.
5. A sociology professor postulates that in each decade 8% of women in the work force leave it and 20%
of the women not in it begin to work. Compare the predictions of his model with the following data on
the percentage of women working: 43.3% in 1970, 51.5% in 1980, 57.5% in 1990, and 59.8% in 2000. In
the long run what fraction of women will be working?
6. A rapid transit system has just started operating. In the first month of operation, it was found that 25%
of commuters are using the system while 75% are travelling by automobile. Suppose that each month
10% of transit users go back to using their cars, while 30% of automobile users switch to the transit
system.
(a) Compute the three step transition probability P 3 .
(b) What will be the fractions using rapid transit in the fourth month? In the long run?
7. A regional health study indicates that from one year to the next, 75% percent of smokers will continue
to smoke while 25% will quit. 8% of those who stopped smoking will resume smoking while 92% will
not. If 70% of the population were smokers in 1995, what fraction will be smokers in 1998? in 2005? in
the long run?
8. (a) Suppose brands A and B have consumer loyalties of 0.5 and 0.6, meaning that a customer who buys
A one week will with probability 0.6 buy it again the next week, or try the other brand with 0.4. What is
the limiting market share for each of these products?
(b) Suppose now there is a third brand with loyalty .6, and that a consumer who changes brands picks
one of the other two at random. What is the new limiting market share for these three products?
9. A sociologist studying living patterns in a certain region determines that the pattern of movement be-
tween urban (U), suburban (S), and rural areas (R) is given by the following transition matrix.
U S R
U 0.86 0.08 0.06
S 0.05 0.88 0.07
R 0.03 0.05 0.92
In the long run what fraction of the population will live in the three areas?
10. Consider the Markov chain with transition matrix:
1 2 3 4
1 0 0 0.1 0.9
2 0 0 0.6 0.4
3 0.8 0.2 0 0
4 0.4 0.6 0 0
(a) Compute p2 .
(b) Find the stationary distributions of p and all of the stationary distributions of p2 .
(c) Find the limit of p2n (x; x) as n → ∞.
11. At a nationwide travel agency, newly hired employees are classified as beginners ( B ). Every six months
the performance of each agent is reviewed. Past records indicate that transitions through the ranks to
intermediate (I) and qualified (Q) are according to the following Markov chain, where (F) indicates
workers that were fired:
B I Q F
B .5 .4 0 .1
I 0 .6 .1 .3
Q 0 0 1 0
F 0 0 0 1
(a) What fraction eventually become qualified?

(b) What is the expected time until a beginner is fired or becomes qualified?

Review

Uploaded by

Copyright:

Available Formats

Review

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Review

Uploaded by

Copyright:

Available Formats

Review

3. Let (Xn )n≥1 be a sequence of independent random variables.

(a) Compute the conditional probabilities:

10. Consider the Markov chain with transition matrix:

(a) What fraction eventually become qualified?

You might also like