A new generation of applied probability textbooks

Operations Research Letters 33 (2005) 544 – 550 Operations Research Letters www.elsevier.com/locate/orl A new generation of applied probability textbooks Bert Zwart Department of Mathematics and Computer Science, Eindhoven University of Technology, P.O. Box 513, 5600 MB Eindhoven, and CWI, P.O. Box 94079, 1090 GB Amsterdam, The Netherlands Received 21 October 2004; accepted 21 October 2004 Available online 19 December 2004 Abstract This article reviews the following books: • S. Asmussen, Applied Probability and Queues, second ed., Springer, Berlin, 2003, ISBN 0-387-00211-1, xii+438pp., EUR 85.55. • H. Chen, D. Yao, Fundamentals of Queueing Networks, Springer, Berlin, 2003, ISBN 0-387-95166-0, xviii+405pp., EUR 74,95. • W. Whitt, Stochastic-Process Limits, Springer, Berlin, 2002, ISBN 0-387-95358-2, xxiv+602pp., EUR 106,95. © 2004 Elsevier B.V. All rights reserved. 1. Introduction Recently, an unusually large number of new textbooks in the area of applied probability/stochastic operations research have appeared. This article aims to give a comparative review of three of these books. The first edition of the classic Applied Probability and Queues (henceforth abbreviated APQ) was published in 1987, and went out of print in 1999. The book under review is the second edition. The author, SZren Asmussen, is currently professor of applied probability at Aarhus University, Denmark. Like the first edition, the scope of the book is quite broad, and covers all the classical problems in applied probability such as random walks, Markov chains, regenerative processes, and related topics. Special emphasis is put E-mail address: [email protected] (B. Zwart). 0167-6377/$ - see front matter © 2004 Elsevier B.V. All rights reserved. doi:10.1016/j.orl.2004.10.002 on queueing theory; in particular product form networks, phase-type distributions and Matrix-analytic methods, fluid flow models, and classical single- and multi-server queues. Other problems that are treated are Lévy processes, storage models, and risk processes. In total, the second edition contains over 100 pages of new material, and other parts have been significantly revised. In particular, the reader can find an extended treatment of queueing networks, and completely new topics including insensitivity, Palm theory, rare events and extreme values, Skorokhod problems, Siegmund duality, light traffic, and heavy tails. The focus of Fundamentals of Queueing Networks (FQN) is on queueing networks. Both authors, Hong Chen (Faculty of Commerce, UBC, Vancouver) and David Yao (IEOR Department, Columbia University, New York) are leading researchers in this field. The subtitle of the book, Performance, Asymptotics and B. Zwart / Operations Research Letters 33 (2005) 544 – 550 Optimization, reveals much more about the real scope of this book: Apart from giving a treatment of the classical theory, like birth and death queues, Jackson Networks and Kelly/Whittle Networks, a large part of the book is concerned with timely problems such as stochastic ordering, stability and fluid approximations, diffusion approximations, and various optimization issues. The book Stochastic-Process Limits (SPL) is completely focused on stochastic process limits for queues. The author, Ward Whitt, has made many important contributions to this area. Whitt is currently on the faculty of the IEOR Department of Columbia University but wrote his book when he was at the Shannon Laboratories, AT&T. A major theme of the book is the occurrence of “nonstandard” (i.e. non-Brownian) limit processes, which arise in communication networks exhibiting features as heavy tails and long-range dependence. The book contains a long informal introduction to stochastic process limits by means of a large number of simulation experiments. Whitt aims to emphasize both the applied significance as well as the underlying mathematics. The first five chapters of the book give both an informal and formal introduction. The next five chapters present a large number of heavytraffic limits for queues. The more technical problems, in particular various types of topologies and related continuity topics are treated in the second part of the book. This part, in particular the last chapter, contains new results that did not appear before. So far I gave a small sample of what can be found in the books. The next three sections go into more detail, and indicate how the textbooks under review fit in the research area of (applied) probability and stochastic operations research. Section 5 compares the books, focusing on organization and presentation. Concluding remarks are given in Section 6. 2. Queueing as a branch of probability theory Queueing research in the early (pre-1970, say) years was focused on obtaining exact expressions for performance measures in queueing models. Methods were often analytical, and emphasis was put on the analytic part of the solution procedure. Solutions were often given in terms of Laplace transforms; see e.g. the impressive monograph on the single-server queue 545 by Cohen [7], which is still a very relevant source of information. The often lamented Laplacian curtain, and the rise of computers made simulation become a more and more important alternative. An algorithmic/computational approach, based on phase-type distributions, was proposed by Neuts [15]. Neuts and his co-workers put more emphasis on computational issues, and popularized matrix-analytic methods. The notion of the value of Laplace transform as a complete solution became properly recognized after the work of Abate and Whitt [1], who showed that Laplace transforms are objects which are computationally tractable. Any transform formula in Cohen’s book can be numerically inverted. Meanwhile, probability theory had undergone several spectacular developments on the theoretical side: Ito calculus, diffusions and general Markov processes, stochastic differential equations, semi-martingales, and la théorie générale des processus are important keywords in this respect. Martingales became a (if not the) central notion in probability. Queues, at that time usually being intrinsically discrete time processes, did not seem to require these sophisticated tools. Occasionally, some papers linking queues and martingales appeared, cf. [2,17], but this did not seem to generate much interest. A paper worthy of special mentioning is Kella and Whitt [10]. Using elementary results from general stochastic integration theory, they introduce a martingale, which is perfectly suited for the type of problems studied in queueing. In particular, Ref. [10] contains a very elegant derivation of the Pollaczek–Khinchine formula. Subsequent generalizations and more applications, especially to fluid flow models, followed. It is quite pleasing to see that Asmussen has now incorporated these important developments into his textbook APQ. Chapter IX of APQ is titled Lévy processes, reflection and duality. Section 3 of that chapter particularly focuses on the Kella–Whitt martingale. A martingale proof of the Pollaczek–Khinchine formula is given in Corollary IX.3.4. The treatment of the Kella–Whitt martingale in APQ is highly representative, in the sense that APQ places all classical queueing results in a modern probabilistic perspective, as was already the case in the first edition. Incorporating topics like these in the current edition further, strengthens the role APQ is play- 546 B. Zwart / Operations Research Letters 33 (2005) 544 – 550 ing as a bridge between stochastic operations research and probability theory. Other purely probabilistic topics that can be found in APQ are Palm theory, coupling, extreme value theory, Lévy processes, Siegmund duality, Markov additive processes, and exponential change of measure; this list is far from exhaustive. What I also find appealing is Asmussen’s treatment of matrix-analytic methods, stressing the relation with the classical Wiener–Hopf method. Asmussen treats all methods; not just one. Naturally, Asmussen emphasizes the probabilistic issues rather than the analytic ones. Basic results as Foster’s criteria for Markov processes and renewal theorems are proven by means of the convergence theorem for supermartingales and coupling arguments. Another example is his treatment of the timedependent properties of the M/M/1 queue in Section III.8. There, he shows how Bessel functions naturally arise when considering the difference of two Poisson processes. When he examines the speed of convergence towards steady state, he prefers to quote a higher-order expansion for the CLT, rather than quoting an asymptotic estimate for Bessel functions (cf. Lemma 3.11). Another example where such a (in my view also analytical) result is applied is in the proof of Theorem XIII 2.1 (Bahadur–Rao’s theorem). But that does not mean that the analytic approach is neglected. For example, Asmussen gives both an analytic and probabilistic proof of Kingman’s heavy traffic limit theorem in Section X.7. 3. Structural properties of stochastic networks Initially, queueing theorists focused on obtaining exact results for performance measures of single queues. Research on queueing networks started in the 1950s and 1960s by Burke and Jackson. A classic textbook on stochastic networks is Kelly [11]. The main messages are that, for a small but relevant class of queueing networks, the joint queue length distribution at the nodes is of product form and that, moreover, the distribution is insensitive to the service-time distribution, apart from their mean. Although these results are very powerful and continue to find their way into practice, the underlying assumptions are sometimes restrictive and often different insights in the models at hand are required. Sometimes, one is mainly interested in certain qualitative properties of the system. An important example concerns stability. A classical result of Loynes [14] states that the G/G/1 queue is stable if and only if
< 1. Here,
is the average amount of work offered to the system per unit of time, and “stability” means that the waiting time in steady state is a.s. finite. Intuitively speaking, the single server queue is stable when the amount of work offered is less than the service capacity. This is not surprising, and consequently the stability issue is simply ignored in the performance analysis of various systems. Similar intuitive conditions for single-class networks hold. However, it came as a major shock that the usual
< 1 condition may not be sufficient for multiclass networks, see, e.g. [12,18]. The service discipline in these networks contains priorities. Multiclass FIFO examples were found later by Bramson [6] and Seidman [19]. An important concept to identify additional necessary conditions for stability is that of a virtual station. Rybko and Stolyar obtained their conditions for (in)stability by the analysis of a corresponding fluid limit. This approach was later generalized by Dai [8]. The above-mentioned phenomena are not pathological: important examples where these issues arise are networks representing large semiconductor manufacturing systems. To the best of the reviewer’s knowledge, FQN is the first textbook giving a treatment of these important developments. Chapter 8 of FQN gives a very nice and clear overview by using a two-station multiclass queueing network. By means of completely worked out examples, simulation experiments, and extensive discussions they make various notions intuitively clear. Related timely problems that are given a textbook treatment are fluid and diffusion limits, and strong approximation results. Chen and Yao provide an introduction to such notions in Chapter 5, which makes the book self-contained. What I found appealing is that they treat the single server queue in great detail in Chapter 6: the subject is already difficult enough without having to think about complicated network structures and associated notation. That chapter also give a very clear treatment of the one-dimensional reflection mapping, and provides an analysis of reflected Brownian motion, using Ito’s lemma. Sections 6.3 and 6.4, respectively, derive a fluid and a diffusion limit. B. Zwart / Operations Research Letters 33 (2005) 544 – 550 Section 6.5 explains how to apply such limits to approximate the workload and queue length processes. More advanced topics, such as a functional law of the iterated logarithm, a strong approximation, and exponential speed of convergence are derived after that. Other models discussed in the book are generalized Jackson networks (Chapter 7) and multiclass feedforward networks (Chapter 9). The setup of these chapters is similar. A whole chapter on approximation of queueing networks by Brownian models is Chapter 10. That chapter also contains illustrative numerical results. Thus, Chapters 5–10 give a complete picture of the use of fluid and diffusion limits in queueing network theory. This makes FQN an extremely useful book. Apart from the above-mentioned topics, FQN also treats the classical product-form theory in the first four chapters. A significant amount of attention is paid to stochastic ordering results, which is not surprising, since it is one of the research interests of the second co-author. Both the first and second part of the book are applied in the third and final part, which deals with optimization and control. Chapter 11 is a very readable account of (applications of) conservation laws. Here, concepts from combinatorial optimization, such as polymatroids, are introduced and applied. Chapter 12 is concerned with scheduling of queueing networks, using the associated fluid approximations derived before. 4. Non-standard topologies in queueing theory At the 1965 symposium on congestion theory, Kingman [13] presented a heavy traffic approximation for the waiting time of the GI/GI/1 queue. Apart from Loynes’ criterium for ergodicity of the single-server queue, this result was one of the first qualitative results in queueing theory. Other pioneers in this area are Borovkov, Iglehart, and Whitt. Iglehart and Whitt emphasized the role of weak convergence methods in obtaining heavy traffic limits for queueing systems. Such methods became accessible to a wide audience after Billingsley [5] wrote his masterpiece Convergence of Probability Measures. An important role in weak convergence of stochastic processes is provided by the continuous mapping approach. If a sequence of stochastic processes (Xn )n
1 547 converges to X as n → ∞, and  is a continuous mapping, then ((Xn ))n
1 converges to (X). An important functional in queueing theory is the reflection mapping, which converts input processes into workload processes. Unfortunately, this mapping is not continuous in the topology induced by convergence of the finite dimensional distributions. Instead, it is necessary to use one of the Skorokhod topologies. The main technical point Whitt makes in his book is that the usual Skorokhod J1 topology does not always provide the right setting. A simple example is a sequence of fluid queues, which all have continuous sample paths, which, after normalization, converge to a reflected stable Lévy process, having discontinuous sample paths. The J1 topology is too strong to allow such a type of convergence. The M1 topology, which is weaker than J1 but still sufficiently strong, successfully deals with this issue. By providing illuminating pictures, Whitt masterfully reveals the essence and differences between the various Skorokhod topologies. As Whitt writes on page viii in his preface: “. . . Thus, while the J1 topology sometimes cannot be used, the M1 topology can almost always be used. Moreover, the extra strength of the J1 topology is rarely exploited. Thus, we would be so bold so as to suggest that, if only one topology on the function space D is to be considered, then it should be the M1 topology.” Throughout the whole book, Whitt supports this statement by providing many examples in which the M1 rather than the J1 topology should be used; moreover, Whitt proves M1 -continuity of many useful functions, which makes the M1 topology useful for applications. This makes the book also interesting for general probabilists. The above discussion makes it appear that SPL is a book on topologies and weak convergence of stochastic processes. But the book is much more than that. Whitt shows when, where, and how his limit theorems can be applied. The topological issues mentioned above are directly motivated by the recent finding that traffic in communication networks exhibits many non-standard features like heavy (infinite variance) tails and long-range dependence. These features let the classical assumptions leading to Brownian approximations break down, and directly lead to the issues discussed above. Moreover, at many places in the book, Whitt pauses to focus on the engineering relevance of the obtained results. The first two chapters of the book provide a 548 B. Zwart / Operations Research Letters 33 (2005) 544 – 550 remarkable introduction: Whitt gives a very original pictorial presentation of limit theorems. Chapter 2 gives examples of random walk in practice. The last section of that chapter is a well-written account on the engineering significance of limit theorems. Whitt returns to this topic at various parts in his book. For example, Whitt explains what can be learned from the scaling itself, rather than just considering the limit process, and explains how limit theorems can be useful in planning simulations: an important problem in simulation is how long a simulation run should take, and Whitt shows how to use limit theory to deal with this issue. Computational issues are not ignored either. Whitt shows how to apply the limit theorems combined with, e.g. transform inversion to produce accurate numerical results. The in-depth and integrated treatment of both theoretical and applied issues makes SPL a unique book. It is not only a research monograph, but also aims to serve as an introduction to stochastic process limits and their applications. I believe that both goals have been achieved. As Whitt writes in his preface, he does not give a comprehensive overview of stochastic process limits. His focus is on the treatment of non-standard topologies, and on the continuous-mapping approach. A short treatment of the compactness approach is provided in Chapter 11. A treatment of the semi-martingale method is not provided. But Whitt gives excellent pointers to the literature at the right time and the right place, so as to make this feature not a problem. What I also did not find in the book was a discussion of multi-class networks. Here, diffusion approximations often exhibit state-space collapse. It is a pity that the reader interested in engineering applications is kept from these important developments, but on the other hand, it is not surprising: general multi-class networks seem difficult to analyze with the continuous mapping approach. 5. Discussion The previous sections show that the books under review have several things in common. All books are of high quality, and give first textbook treatments of several topics that are still subject of active research. All books use more advanced machinery from proba- bility theory than older textbooks, such as martingale and weak convergence methods. This raises the question to which extent these books are suitable for operations researchers, especially students. I think this is not a problem, since most graduate curricula nowadays also contain courses on financial mathematics, which are not less demanding when it comes to the use of probability theory. Of course, there are also differences between the books. Asmussen’s writing style is concise. But this seems an advantage given the large number of topics that are dealt with. Whitt is on the other end of the spectrum. He often likes to make side remarks; and the book does not stop after 600 pages: a supplement of 300 extra pages, containing some of the more technical topics, is provided on his webpage. Fortunately, Whitt’s proofs are in the same style as the rest of the book. He offers a great amount of detail, which makes the book very useful for self-study. Asmussen’s proofs are usually somewhat compact. The writing style of Chen and Yao seems to be somewhere in the middle. Their first four chapters on the classical theory contain altogether less than 100 pages, but they manage to explain all the essential results in a sufficient amount of detail. Some of the most technical issues in later chapters are only discussed on a heuristic level. The organization of the three books is also entirely different. Asmussen’s book consists of 14 chapters, which are roughly divided into three parts. Each chapter contains an incredible amount of information. For example, Chapter 7, titled Further Topics in Renewal Theory and Regenerative Processes, contains sections on Spread-Out Distributions, The Coupling Method, Markov-Processes: Regeneration and Harris Recurrence, Markov Renewal Theory, Semi-Regenerative Processes, and Palm Theory, Rate Conservation and PASTA. Asmussen succeeds to discuss the essentials of all these topics in 44 pages, and still manages to find room for a set of exercises at the end of each section. This is representative for the whole book. One of the features of APQ I like the most is that each section contains various useful notes and pointers to the literature. The bibliography is more than impressive. Asmussen seems to know all the relevant papers on each topic. This makes APQ the major reference in applied probability. If one reads the book from cover to cover, one occasionally notes that some theorems are proven with the aid of results from later B. Zwart / Operations Research Letters 33 (2005) 544 – 550 chapters. Some technical details can be found in the eight appendices. The set up of FQN is a bit different from that of Asmussen. All notes are given at the end of each chapter, and there is no single bibliography, but one for every chapter. FQN contains no appendices, but there is no need since most of the technicalities are presented in Chapter 5, or just introduced when they are used (like polymatroids in Chapter 11). FQN gives more examples, simulations, and numerical results than APQ, which makes the book more suitable for self-study. This does not mean that APQ cannot be used as a textbook; I think this is well possible since additional details can be given in class. The assumed knowledge of probability is rather advanced, which the reader is warned for in Asmussen’s preface. An additional appendix on martingale theory would not have been a waste of space. FQN introduces martingales in Chapter 5. Whitt’s book SPL consist of four parts, which are grouped in an increasing level of difficulty. The technical foundations are provided in Chapters 11–14, which are necessary in Chapters 6–10, where a large number of different queueing network models are analyzed. An informal introduction to the topic is provided in the first five chapters. Consequently, Whitt refers quite often to future chapters. The book is completely self-contained, and like FQN, spends a considerable amount of space to intuition. The list of references is very impressive, which makes the book a very useful reference for researchers. The book also points towards a number of open problems. Information on the web is available for APQ [3] and SPL [21]. Both sites contain a list of typos and additional comments, and [21] also contains an extensive internet supplement. 6. Conclusion Without any hesitation, I can recommend all three books to the reader. In particular, I can also recommend APQ to people who already own the first edition of that book, since the second edition contains many new topics and many updates (including bibliographical ones). APQ is definitely indispensable for researchers in applied probability. It will be demanding, even for graduate students, although I believe it 549 is possible to use it as a textbook. FQN and SPL are very useful to both researchers and students as well. Although both books treat stochastic process limits, there is surprisingly little overlap. As written in its preface, FQN is designed to be a textbook, and I think the authors have been successful on that. Among the three books under review, this book seems to be the most accessible one for students. SPL is primarily a research monograph, but is also useful to serve as an introduction to stochastic process limits. The above-review mentions several recent developments in applied probability, and showed how these developments became textbook material. An excellent recent overview of research in stochastic OR is provided by Stidham [20]. Other very recent textbooks in the area are Baccelli and Brémaud [4], Robert [16], and Ganesh et al. [9]. References [1] J. Abate, W. Whitt, The Fourier series method for inverting Laplace transforms of probability distributions, Queueing Syst. 10 (1992) 5–87. [2] F. Baccelli, A. Makowski, Dynamic, transient and stationary behaviour of the M/G/1 queue via martingales, Ann. Probab. 17 (1989) 1691–1699. [3] http://home.imf.au.dk/asmus/books/books.html. [4] F. Baccelli, P. Brémaud, Elements of Queueing Theory, third ed., Springer, New York, 2003. [5] P. Billingsley, Convergence of Probability Measures, second ed., Wiley, New York, 1999. [6] M. Bramson, Instability of FIFO queueing networks, Ann. Appl. Probab. 4 (1994) 414–431. [7] J.W. Cohen, The Single Server Queue, second ed., NorthHolland, Amsterdam, 1982. [8] J.G. Dai, On positive Harris recurrence of multiclass queueing networks: a unified approach via fluid limit models, Ann. Appl. Probab. 5 (1995) 49–77. [9] A. Ganesh, N. O’Connell, D. Wischik, Big Queues, Lecture Notes in Mathematics, Springer, New York, 2004. [10] O. Kella, W. Whitt, Useful martingales for stochastic storage processes with Lévy input, J. Appl. Probab. 29 (1992) 396– 403. [11] F.P. Kelly, Reversibility and Stochastic Networks, Wiley, New York, 1979. [12] P.R. Kumar, T.I. Seidman, Dynamic instabilities and stabilization methods in distributed real-time scheduling of manufacturing systems, IEEE Trans. Automat. Control 35 (1990) 289–298. [13] J.F.C. Kingman, The heavy traffic approximation in the theory of queues, in: W.L. Smith, W.E. Wilkinson (Eds.), Proceedings of the Symposium on Congestion Theory, The University of North Carolina Press, Chapel Hill, 1965, pp. 137–159. 550 B. Zwart / Operations Research Letters 33 (2005) 544 – 550 [14] R.M. Loynes, The stability of a queue with non-independent interarrival and service times, Proceedings of the Cambridge Philosophical Society 58 (1962) 497–520. [15] M.F. Neuts, Matrix-Geometric Solutions in Stochastic Models, Johns Hopkins University Press, Baltimore, MD, 1981. [16] Ph. Robert, Stochastic Networks and Queues, Springer, New York, 2003. [17] W. Rosenkrantz, Calculation of the Laplace transform of the length of the busy period for the M/G/1 queue via martingales, Ann. Probab. 11 (1983) 817–818. [18] A.N. Rybko, A.L. Stolyar, On the ergodicity of random processes that describe the functioning of open queueing networks, Problems Inform. Transmission 28 (1992) 3–26. [19] T.I. Seidman, First come first served can be unstable, IEEE Trans. Automat. Control 39 (1993) 2166–2171. [20] S. Stidham, Analysis, design, and control of queueing systems, Oper. Res. 50 (2002) 197–216. [21] http://www.columbia.edu/∼ww2040/supplement.html.