Optimal Stopping

In order to adapt to the time-varying nature of wireless channels, various channel-adaptive schemes have been proposed to exploit inherent spatial diversity in wireless ad hoc networks where there are usually alternate forwarding nodes available at a given forwarding node. However, existing schemes along this line are designed based on heuristics, implying room for performance enhancement. Thereby, to seek a theoretical foundation for improving spatial diversity gain, we formulate the selection of the next-hop relay as a sequential decision problem and derive a general "Optimal Stopping Relaying (OSR)" framework for designing such spatial-diversity schemes. As a particular example, assuming Rayleigh fading channels, we implement an OSR strategy to optimize information efficiency (IE) in a protocol stack consisting of Greedy Perimeter Stateless Routing (GPSR) and IEEE 802.11 MAC protocols. We present an analysis of the algorithm for a single node. In addition, we perform extensive simulations (using QualNet) to evaluate the end-to-end performance of the proposed forwarding strategy. The results demonstrate the superiority of OSR over other existing schemes.

This paper considers the American put option valuation in a jump-diffusion model and relates this optimal-stopping problem to a parabolic integro-differential free-boundary problem, with special attention to the behavior of the optimal-stopping boundary. We study the regularity of the American option value and obtain in particular a decomposition of the American put option price as the sum of its counterpart European price and the early exercise premium. Compared with the Black-Scholes (BS) [5] model, this premium has an additional term due to the presence of jumps. We prove the continuity of the free boundary and also give one estimate near maturity, generalizing a recent result of Barleset al. [3] for the BS model. Finally, we study the effect of the market price of jump risk and the intensity of jumps on the American put option price and its critical stock price.

We theorize on the performance implications of the timing at which entrepreneurs stop exploring their business opportunities and start exploiting them. Using an optimal-stopping approach, we characterize the time when exploitation should begin based primarily on when an entrepreneur's ignorance has been sufficiently reduced through knowledge accumulation. This "ignorance threshold" captures a tradeoff between the time needed to increase legitimacy and the necessity to act now to minimize competition. Changes in legitimacy and competition are based on how entrepreneurs manage their knowledge (tacit versus explicit) under differing degrees of novelty for the business opportunity. These changes, in turn, impact the performance and timing of opportunity exploitation.

In this article we study a decoupled forward backward stochastic differential equation (FBSDE) and the associated system of partial integro-differential obstacle problems, in a flexible Markovian set-up made of a jump-diffusion with regimes. These equations are motivated by numerous applications in financial modeling, whence the title of the paper. This financial motivation is developed in the first part of the paper, which provides a synthetic view of the theory of pricing and hedging financial derivatives, using backward stochastic differential equations (BSDEs) as main tool. In the second part of the paper, we establish the well-posedness of reflected BSDEs with jumps coming out of the pricing and hedging problems exposed in the first part. We first provide a construction of a Markovian model made of a jump-diffusion – like component X interacting with a continuous-time Markov chain – like component N. The jump process N defines the so-called regime of the coefficients of X, when...

In the sequential decision making task known as the best choice problem, n items are presented in a random order one at a time. After each item, the decision maker (DM) can determine only their relative ranks. The DMÕs goal is to select the best of all n items without the possibility of recalling previously observed items. The purpose of this paper is to compare the optimal policy to three classes of heuristic decision rules that were identified and studied in previous research. We also investigate the limiting case as n ! 1.

We present the solution of a portfolio optimization problem for an economic agent endowed with a stochastic insurable stream, under a liquidity constraint over the time interval [0, T ]. Generally, the existence of labor income complicates the agent's decisions. Moreover, in the real world the economic agents are restricted in their ability to borrow against their future labor income. We deal with this kind of liquidity constraint following the lines of American option valuation which allows us to give a precise characterization of the optimal consumption as well as the terminal wealth. In a Markovian case, with infinite horizon and HARA utility, we obtain a closed form solution.

Purpose – This paper aims to focus on the problem of the optimal relocation policy for a firm that faces two types of uncertainty: one about the moments in which new (and more efficient) sites will become available; and the other regarding the degree of efficiency improvement inherent ...

Consider the American put and Russian option with the stock price modeled as an exponential Lévy process. We find an explicit expression for the price in the dense class of Lévy processes with phase-type jumps in both directions. The solution rests on the reduction to the first passage time problem for (reflected) Lévy processes and on an explicit solution of the latter in the phase-type case via martingale stopping and Wiener-Hopf factorisation. The same type of approach is also applied to the more general class of regime switching Lévy processes with phase-type jumps.

We study a two-sided game-theoretic version of this optimal stopping problem, where men search for a woman to marry at the same time as women search for a man to marry. We find that in the unique subgame perfect equilibrium, the expected rank grows as the square root of N and that, surprisingly, the leading coefficient is exactly 1. We also discuss some possible variations.

This paper describes the selection and automation of a method for estimating how many replications should be run to achieve a required accuracy in the output. The motivation is to provide an easy to use method that can be incorporated into existing simulation software that enables practitioners to obtain results of a specified accuracy. The processes and decisions involved in selecting and setting up a method for automation are explained. The extensive test results are outlined, including results from applying the algorithm to a collection of artificial and real models.

In this paper, we develop a theoretical framework for the common business practice of rolling horizon decision making. The main idea of our approach is that the usefulness of rolling horizon methods is, to a great extent, implied by the fact that forecasting the future is a costly activity. We, therefore, consider a general, discrete-time, stochastic dynamic optimization problem in which the decision maker has the possibility to obtain information on the uncertain future at given cost. For this non-standard optimization problem with optimal stopping decisions, we develop a dynamic programming formulation. We treat both finite and infinite horizon cases. We also provide a careful interpretation of the dynamic programming equations and illustrate our results by a simple numerical example. Various generalizations are shown to be captured by straightforward modifications of our model.

Slot machine games used to be very simple, often limited to just one spin of the reels, but have evolved to accommodate a variety of features that provide added excitement and the potential for greater wins. They also engage the players more by, quite often, requiring them to make choices. This paper presents one of these games, Bitz & Pizzas. This case study introduces a novel application of Operational Research (OR) and serves as evidence that as slot games, and their corresponding models, are becoming more sophisticated, the role of a well-trained OR professional is becoming more important and necessary in this industry. It also shows that OR practitioners are in touch with the needs of the gambling industry and understand current practices and legislation.

This paper shows that penalized backward stochastic differential equation (BSDE), which is often used to approximate and solve the corresponding reflected BSDE, admits both optimal stopping representation and optimal control representation. The new feature of the optimal stopping representation is that the player is allowed to stop at exogenous Poisson arrival times. The convergence rate of the penalized BSDE then follows from the optimal stopping representation. The paper then applies to two classes of equations, namely multidimensional reflected BSDE and reflected BSDE with a constraint on the hedging part, and gives stochastic control representations for their corresponding penalized equations.

Among a comprehensive scope of mitigation measures for climate change, CO2 capture and sequestration (CCS) plays a potentially significant role in industrialised countries. In this paper, we develop an analytical real options model that values the choice between two emissions-reduction technologies available to a coal-fired power plant. Specifically, the plant owner may decide to invest in either full CCS (FCCS) or partial CCS (PCCS) retrofits given uncertain electricity, CO2, and coal prices. We first assess the opportunity to upgrade to each technology independently by determining the option value of installing a CCS unit as a function of CO2 and fuel prices. Next, we value the option of investing in either FCCS or PCCS technology. If the volatilities of the prices are low enough, then the investment region is dichotomous, which implies that for a given fuel price, retrofitting to the FCCS (PCCS) technology is optimal if the CO2 price increases (decreases) sufficiently. The numerical examples provided in this paper using current market data suggest that neither retrofit is optimal immediately. Finally, we observe that the optimal stopping boundaries are highly sensitive to CO2 price volatility.

This study relaxes the assumption of perfect capital markets in the classical Faustmann forest rotation model. A constraint on borrowing changes all the basic properties of optimal forest harvesting. Instead of a constant rotation period, an in"nite chain of rotations is determined by a nonlinear di!erence equation. The rotation period evolves over time and approaches a stationary rotation in "nite or in"nite time. The length of optimal rotation, timber supply and the value of forest land depend on forest-ownerspeci"c factors that are normally nonexistent in rotation problems. The model explains several empirical "ndings that cause problems for the classical versions of the Faustmann model. ." (O. Tahvonen).

We study a problem when a solution to optimal stopping problem for one-dimensional diffusion will generate by threshold strategy. Namely, we give necessary and sufficient conditions under which an optimal stopping time can be specified as the first time when the process exceeds some level (threshold), and a continuation set is a semi-interval. We give also second-order conditions, which allow to discard such solutions to free-boundary problem that are not the solutions to optimal stopping problem.

We use a model in which media of exchange are essential to examine the role of liquidity and monetary policy on production and investment decisions in which time is an important element. Specifically, we consider the effects of monetary policy on the length of production time and entry and exit decisions for firms. We show that higher rates of inflation cause households to substitute away from money balances and increase the allocation of bonds in their portfolio thereby causing a decline in the real interest rate. The decline in the real interest rate causes the period of production to increase and the productivity thresholds for entry and exit to decline. This implies that when the real interest rate declines, prospective firms are more likely to enter the market and existing firms are more likely to stay in the market. Finally, we present reduced form empirical evidence consistent with the predictions of the model.

In this survey, we show that various stochastic optimization problems arising in option theory, in dynamical allocation problems, and in the microeconomic theory of intertemporal consumption choice can all be reduced to the same problem of representing a given stochastic process in terms of running maxima of another process. We describe recent results of Bank and El Karoui (2002) on the general stochastic representation problem, derive results in closed form for Lévy processes and diffusions, present an algorithm for explicit computations, and discuss some applications.

I introduce and study new derivative securities which I call game options (or Israeli options to put them in line with American, European, Asian, Russian etc. ones). These are contracts which enable both their buyer and seller to stop them at any time and then the buyer can exercise the right to buy (call option) or to sell (put option) a specified security for certain agreed price. If the contract is terminated by the seller he must pay certain penalty to the buyer. A more general case of game contingent claims is considered. The analysis is based on the theory of optimal stopping games (Dynkin's games). Game options can be sold cheaper than usual American options and their introduction could diversify financial markets.

The problem of option hedging in the presence of proportional transaction costs can be formulated as a singular stochastic control problem. Hodges and Neuberger [1989. Optimal replication of contingent claims under transactions costs. Review of Futures Markets 8, 222-239] introduced an approach that is based on maximization of the expected utility of terminal wealth. We develop a new algorithm to solve the corresponding singular stochastic control problem and introduce a new approach to option hedging which is closer in spirit to the pathwise replication of Black and Scholes [1973. The pricing of options and corporate liabilities. Journal of Political Economy 81, 637-654]. This new approach is based on minimization of a Black-Scholes-type measure of pathwise risk, defined in terms of a market delta, subject to an upper bound on the hedging cost. We provide an efficient backward induction algorithm for the problem of cost-constrained risk minimization, whose associated singular stochastic control problem is shown to be equivalent to an optimal stopping problem. This algorithm is then modified to solve the singular stochastic control problem associated with utility maximization, which cannot be reduced to an optimal stopping problem. We propose to choose an optimal parameter (risk-aversion coefficient or Lagrange multiplier) in either approach by minimizing the mean squared hedging error and demonstrate that with this ''best'' choice of the parameter, both approaches have similar performance. We also discuss the different notions of risk in both approaches and propose a volatility adjustment for the risk-minimization approach, which is analogous to that introduced by Zakamouline [2006. European option pricing and hedging with both fixed and proportional transaction costs. Journal of Economic Dynamics and Control 30, 1-25] for the utility maximization approach, thereby providing a unified treatment of both approaches.

In this paper we consider a general optimal consumption-portfolio selection problem of an infinitely-lived agent whose consumption rate process is subject to subsistence constraints before retirement. That is, her consumption rate should be greater than or equal to some positive constant before retirement. We integrate three optimal decisions which are the optimal consumption, the optimal investment choice and the optimal stopping problem in which the agent chooses her retirement time in one model. We obtain the explicit forms of optimal policies using a martingale method and a variational inequality arising from the dual function of the optimal stopping problem. We treat the optimal retirement time as the first hitting time when her wealth exceeds a certain wealth level which will be determined by a free boundary value problem and duality approaches. We also derive closed forms of the optimal wealth processes before and after retirement. Some numerical examples are presented for the case of constant relative risk aversion (CRRA) utility class.

We extend the Longstaff-Schwartz algorithm for approximately solving optimal stopping problems on high-dimensional state spaces. We reformulate the optimal stopping problem for Markov processes in discrete time as a generalized statistical learning problem. Within this setup we apply deviation inequalities for suprema of empirical processes to derive consistency criteria, and to estimate the convergence rate and sample complexity. Our results strengthen and extend earlier results.

I introduce and study new derivative securities which I call game options (or Israeli options to put them in line with American, European, Asian, Russian etc. ones). These are contracts which enable both their buyer and seller to stop them at any time and then the buyer can exercise the right to buy (call option) or to sell (put option) a specified security for certain agreed price. If the contract is terminated by the seller he must pay certain penalty to the buyer. A more general case of game contingent claims is considered. The analysis is based on the theory of optimal stopping games (Dynkin's games). Game options can be sold cheaper than usual American options and their introduction could diversify financial markets.

We study an infinite horizon optimal stopping problem which arises naturally in the optimal timing of a firm/project sale or in the valuation of natural resources: the functional to be maximised is a sum of a discounted running reward and a discounted final reward. The running and final rewards as well as the instantaneous interest rate (used for calculating discount factors) depend on a Feller-Markov process modelling the underlying randomness in the world, such as prices of stocks, prices of natural resources (gas, oil, etc.), or factors influencing the interest rate. For years, it had seemed sensible to assume that interest rates were uniformly separated from 0, which is needed for the existing theory to work. However, recent developments in Japan and in Europe showed that interest rates can get arbitrarily close to 0. In this paper we establish the feasibility of the stopping problem, prove the existence of optimal stopping times and a variational characterisation (in the viscosity sense) of the value function when interest rates are not uniformly separated from 0. Our results rely on certain ergodic properties of the underlying (non-uniformly) ergodic Markov process. We provide several criteria for diffusions and jump-diffusions.

A given number of n applicants are to be interviewed for a position of secretary. They present themselves one-by-one in random order, all n! permutations being equally likely. Two players I and II jointly interview the i-th applicant and observe that his (or her) relative rank is y for I and z for II, relative to i − 1 applicants that have already seen (rank 1 is for the best). Each player chooses one of the two choices Accept or Reject. If choice-pair is R-R, then the i-th is rejected, and the players face the next i + 1-th applicant. If A-A is chosen, then the game ends with payoff to I (II), the expected absolute rank under the condition that the i-th has the relative rank y (z). If players choose different choices, then arbitration comes in, and forces players to take the same choices as I's (II's) with probability p (p = 1 − p). Arbitration is fair if p = 1/2. If all applicants except the last have been rejected, then A-A should be chosen for the last. Each player aims to minimize the expected payoff he can get. Explicit solution is derived to this n stage game, and numerical results are given for some n and p. The possibility of an interactive approach in this selection problem is analyzed.

The American put is one of the oldest problems in mathematical finance. We review the development of the relevant literature over the last 40 years. Today the mainstream computational problems have been solved satisfactorily and the target of research is shifting towards the development of further insights into the value of timing investment decisions.

This paper investigates value function approximation in the context of zero-sum Markov games, which can be viewed as a generalization of the Markov decision process (MDP) framework to the two-agent case. We generalize error bounds from MDPs to Markov games and describe generalizations of reinforcement learning algorithms to Markov games. We present a generalization of the optimal stopping problem to a two-player simultaneous move Markov game. For this special problem, we provide stronger bounds and can guarantee convergence for LSTD and temporal difference learning with linear value function approximation. We demonstrate the viability of value function approximation for Markov games by using the Least squares policy iteration (LSPI) algorithm to learn good policies for a soccer domain and a flow control problem.

Suppose that there are finitely many simple hypotheses about the unknown arrival rate and mark distribution of a compound Poisson process, and that exactly one of them is correct. The objective is to determine the correct hypothesis with minimal error probability and as soon as possible after the observation of the process starts. This problem is formulated in a Bayesian framework, and its solution is presented. Provably convergent numerical methods and practical near-optimal strategies are described and illustrated on various examples. P π {Θ = i} = π i for some π ∈ E {(π 1 , . . . , π M ) ∈ [0, 1] M : M k=1 π k = 1}, and we start observing the process X. The objective is to determine the correct hypothesis as quickly as possible with minimal probability of making a wrong decision. We are allowed to observe the process X before the final decision as long as we want. Postponing the final decision improves the odds of its correctness but increases the sampling and lost opportunity costs.

A new PV design, called ''one axis three position sun tracking PV module'', with low concentration ratio reflector was proposed in the present study. Every PV module is designed with a low concentration ratio reflector and is mounted on an individual sun tracking frame. The one axis tracking mechanism adjusts the PV position only at three fixed angles (three position tracking): morning, noon and afternoon. This ''one axis three position sun tracking PV module'' can be designed in a simple structure with low cost. A design analysis was performed in the present study. The analytical results show that the optimal stopping angle b in the morning or afternoon is about 50°f rom the solar noon position and the optimal switching angle that controls the best time for changing the attitude of the PV module is half of the stopping angle, i.e. h H = b/2, and both are independent of the latitude. The power generation increases by approximately 24.5% as compared to a fixed PV module for latitude / < 50°. The analysis also shows that the effect of installation misalignment away from the true south direction is negligible (<2%) if the alignment error is less than 15°. An experiment performed in the present study indicates that the PV power generation can increase by about 23% using low concentration (2X) reflectors. Hence, combining with the power output increase of 24.5%, by using one axis three position tracking, the total increase in power generation is about 56%. The economic analysis shows that the price reduction is between 20% and 30% for the various market prices of flat plate PV modules.

In this article it is shown that one is able to evaluate the price of perpetual calls, puts, Russian and integral options directly as the Laplace transform of a stopping time of an appropriate di usion using standard uctuation theory. This approach is o ered in contrast to the approach of optimal stopping through free boundary problems see volume 39,1 of Theory of Probability and its Applications]. Following ideas in 5], we discuss the Canadization of these options as a method of approximation to their nite time counterparts. Fluctuation theory is again used in this case.

As contemporary software-intensive systems reach increasingly large scale, it is imperative that failure detection schemes be developed to help prevent costly system downtimes. A promising direction towards the construction of such schemes is the exploitation of easily available measurements of system performance characteristics such as average number of processed requests and queue size per unit of time. In this work, we investigate a holistic methodology for detection of abrupt changes in time series data in the presence of quasi-seasonal trends and long-range dependence with a focus on failure detection in computer systems. We propose a trend estimation method enjoying optimality properties in the presence of long-range dependent noise to estimate what is considered " normal " system behaviour. To detect change-points and anomalies, we develop an approach based on the ensembles of " weak " detectors. We demonstrate the performance of the proposed change-point detection scheme using an artificial dataset, the publicly available Abilene dataset as well as the proprietary geoinformation system dataset.

Disinvestment, in the sense of project termination and liquidation of assets, is an important realm of entrepreneurial decision-making which has still not been entirely investigated. This study presents the results of an experimental investigation modelling the choice to disinvest as a dynamic problem of optimal stopping in which the value of flexibility is manipulated and the patterns of decisions of entrepreneurs and non-entrepreneurs are compared. The experimental evidence is then confronted with the benchmark predictions of traditional and new investment theory, as represented respectively by the Net Present Value and Real Options Approach. The experimental results reject the Net Present Value approach and reveal a significant correlation between the behavior observed experimentally and the theoretical predictions of the Real Options Approach, but also provide evidence for psychological inertia, which can be related to the status-quo phenomena. The study provides evidence for entrepreneurs being slightly more prone than non-entrepreneurs to holding on to a project for too long.

A singular stochastic control problem with state constraints in twodimensions is studied. We show that the value function is C 1 and its directional derivatives are the value functions of certain optimal stopping problems. Guided by the optimal stopping problem, we then introduce the associated no-action region and the free boundary and show that, under appropriate conditions, an optimally controlled process is a Brownian motion in the noaction region with reflection at the free boundary. This proves a conjecture of Martins, Shreve and Soner [SIAM J. Control Optim. 34 (1996) 2133-2171] on the form of an optimal control for this class of singular control problems. An important issue in our analysis is that the running cost is Lipschitz but not C 1 . This lack of smoothness is one of the key obstacles in establishing regularity of the free boundary and of the value function. We show that the free boundary is Lipschitz and that the value function is C 2 in the interior of the no-action region. We then use a verification argument applied to a suitable C 2 approximation of the value function to establish optimality of the conjectured control.

We present an approximation method for discrete time nonlinear filtering in view of solving dynamic optimization problems under partial information. The method is based on quantization of the Markov pair process filter-observation (Π, Y ) and is such that, at each time step k and for a given size N k of the quantization grid in period k, this grid is chosen to minimize a suitable quantization error. The algorithm is based on a stochastic gradient descent combined with Monte-Carlo simulations of (Π, Y ). Convergence results are given and applications to optimal stopping under partial observation are discussed. Numerical results are presented for a particular stopping problem : American option pricing with unobservable volatility.

In this paper stochastic dynamic programming is used to investigate land conversion decisions taken by a multitude of landholders under uncertainty about the value of environmental services and irreversible development. We study land conversion under competition on the market for agricultural products when voluntary and mandatory measures are combined by the Government to induce adequate participation in a conservation plan. We study the impact of uncertainty on the optimal conversion policy and discuss conversion dynamics under different policy scenarios on the basis of the relative long-run expected rate of deforestation. Interestingly, we show that uncertainty, even if it induces conversion postponement in the short-run, increases the average rate of deforestation and reduces expected time for total conversion in the long run. Finally, we illustrate our findings through some numerical simulations.

We analyze the robustness properties of the Snell envelope backward evolution equation for the discrete time optimal stopping problem. We consider a series of approximation schemes, including cut-off type approximations, Euler discretization schemes, interpolation models, quantization tree models, and the Stochastic Mesh method of Broadie-Glasserman. In each situation, we provide non asymptotic convergence estimates, including Lp-mean error bounds and exponential concentration inequalities. We deduce these estimates from a single and general robustness property of Snell envelope semigroups. In particular, this analysis allows us to recover existing convergence results for the quantization tree method and to improve significantly the rates of convergence obtained for the Stochastic Mesh estimator of Broadie-Glasserman. In the second part of the article, we propose a new approach using a genealogical tree approximation of the reference Markov process in terms of a neutral type genetic...

We explore properties of the value function and existence of optimal stopping times for functionals with discontinuities related to the boundary of an open (possibly unbounded) set O ⊂ E. The stopping horizon is either random, equal to the first exit from the set O, or fixed: finite or infinite. The payoff function is continuous with a possible jump at the boundary of O. Using a generalization of the penalty method we derive a numerical algorithm for approximation of the value function for general Feller-Markov processes and show existence of optimal or ε-optimal stopping times.