Papers by Enrique Lemus Rodríguez
summary:This paper deals with a certain class of unbounded optimization problems. The optimizatio... more summary:This paper deals with a certain class of unbounded optimization problems. The optimization problems taken into account depend on a parameter. Firstly, there are established conditions which permit to guarantee the continuity with respect to the parameter of the minimum of the optimization problems under consideration, and the upper semicontinuity of the multifunction which applies each parameter into its set of minimizers. Besides, with the additional condition of uniqueness of the minimizer, its continuity is given. Some examples of nonconvex optimization problems that satisfy the conditions of the article are supplied. Secondly, the theory developed is applied to discounted Markov decision processes with unbounded cost functions and with possibly noncompact actions sets in order to obtain continuous optimal policies. This part of the paper is illustrated with two examples of the controlled Lindley's random walk. One of these examples has nonconstant action sets
It is well-known that in Markov Decision Processes, with a total discounted reward, for instance,... more It is well-known that in Markov Decision Processes, with a total discounted reward, for instance, it is not always possible to explicitly find the optimal stationary policy f*. But using the Value Iteration, a stationary policy fN such that the optimal discounted rewards of f* and fN are close, for the N-th iteration of the procedure, a question arises: are the actions f*(x) and fN(x) necessarily close for each state x? To our knowledge this question is still largely open. In this paper it is studied when it is possible to stop the value iteration algorithm so that the corresponding maximizer stationary policy fN approximates an optimal policy both in the total discounted reward and in the action space (uniformly over the state space). This kind of results will shed light on important computability issues of great practical interest. In this article the action space is assumed to be a compact set and the reward function bounded. An ergodicity condition on the transition probability ...
Communications in Computer and Information Science, 2017
In this paper, we study a Markov Decision Process version of the classical Ramsey’s Growth model ... more In this paper, we study a Markov Decision Process version of the classical Ramsey’s Growth model where the evolution of the labor component is assumed to be stochastic. As this is a discrete-time model, it is much easier to study the corresponding long-run behavior of the optimal strategies, as it would be for a continuous version. A set of natural conditions in the Euler Equation context are presented that guarantee a stable long-term behavior of the optimal process.
Proceedings of 5th the International Conference on Operations Research and Enterprise Systems, 2016
In this paper we study a version of Ramsey's discrete time Growth Model where the evolution of La... more In this paper we study a version of Ramsey's discrete time Growth Model where the evolution of Labor through time is stochastic. Taking advantage of recent theoretical results in the field of Markov Decision Processes, a first set of conditions on the model are established that guarantee a long-term stable behavior of the underlying Markov chain.
Kybernetika, 2016
Institute of Mathematics of the Czech Academy of Sciences provides access to digitized documents ... more Institute of Mathematics of the Czech Academy of Sciences provides access to digitized documents strictly for personal use. Each copy of any part of this document must contain these Terms of use.
Mathematical Methods of Operations Research, 2007
We find inequalities to estimate the stability (robustness) of a discounted cost optimization pro... more We find inequalities to estimate the stability (robustness) of a discounted cost optimization problem for discrete-time Markov control processes on a Borel state space. The one stage cost is allowed to be unbounded. Unlike the known results in this area we consider a perturbation of transition probabilities measured by the Kantorovich metric, closely related to the weak convergence. The results obtained make possible to estimate the vanishing rate of the stability index when approximation is made through empirical measures. Keywords Discrete-time Markov control process • Total discounted cost • Stability inequalities • Kantorovich metric • Empirical measure 1 Motivation and problem setting We consider a standard control problem for discrete-time Markov processes (see, for instance Dynkin and Yushkevich 1979; Hernández-Lerma and Lasserre 1999). The total discounted cost is used as the optimization criterion.
Mathematical Methods of Operations Research, 2008
We study perturbations of a discrete-time Markov control process on a general state space. The am... more We study perturbations of a discrete-time Markov control process on a general state space. The amount of perturbation is measured by means of the Kantorovich distance. We assume that an average (per unit of time on the infinite horizon) optimal control policy can be found for the perturbed (supposedly known) process, and that it is used to control the original (unperturbed) process. The onestage cost is not assumed to be bounded. Under Lyapunov-like conditions we find upper bounds for the average cost excess when such an approximation is used in place of the optimal (unknown) control policy. As an application of the found inequalities we consider the approximation by relevant empirical distributions. We illustrate our results by estimating the stability of a simple autoregressive control process. Also examples of unstable processes are provided.
Journal of Applied Mathematics, 2013
From the classical point of view, it is important to determine if in a Markov decision process (M... more From the classical point of view, it is important to determine if in a Markov decision process (MDP), besides their existence, the uniqueness of the optimal policies is guaranteed. It is well known that uniqueness does not always hold in optimization problems (for instance, in linear programming). On the other hand, in such problems it is possible for a slight perturbation of the functional cost to restore the uniqueness. In this paper, it is proved that the value functions of an MDP and its cost perturbed version stay close, under adequate conditions, which in some sense is a priority. We are interested in the stability of Markov decision processes with respect to the perturbations of the cost-as-you-go function.
Kybernetika (Prague), 2009
Raúl Montes-de-Oca; Enrique Lemus-Rodríguez; Daniel Cruz-Suárez A stopping rule for discounted Ma... more Raúl Montes-de-Oca; Enrique Lemus-Rodríguez; Daniel Cruz-Suárez A stopping rule for discounted Markov decision processes with finite action sets ... Persistent URL: http://dml.cz/dmlcz/140043 ... © Institute of Information Theory and Automation AS CR, 2009
Stochastic Analysis and Applications, 1999
Uploads
Papers by Enrique Lemus Rodríguez