Pricing Optimization for Selling Reusable Resources

Pricing Optimization for Selling Reusable Resources (Extended Abstract) 1 Jiang Rong1 , Tao Qin2 , Bo An3 , Tie-Yan Liu2 The Key Laboratory of Intelligent Information Processing, ICT, CAS & University of Chinese Academy of Sciences [email protected] 2 Microsoft Research {taoqin, tyliu}@microsoft.com 3 School of Computer Science and Engineering, Nanyang Technological University [email protected] ABSTRACT ket competition. Xu and Hopp [22] assume that customers’ arrival rates follow the geometric Brownian motion and the perfect Bayesian equilibrium is used to model providers’ behaviors. Levin et al. [11] consider strategic users and propose the subgame-perfect equilibrium. However, they focus on the one-shot inventory replenishment problem with dynamic pricing, which cannot describe the market with reusable products. In this paper, we adopt the Poisson process and formulate the dynamic and competitive market as continuous-time Markov chains [8, 13, 16]. Since each provider aims to maximize his/her expected revenue, the optimal policy is supposed to be a Nash Equilibrium (NE). Our second contribution lies in that we show it is difficult to compute the NE because a provider’s revenue cannot be explicitly represented as a function of his/her pricing policy and then introduce the Approximate Equilibrium (AE) solution concept [12, 18]. By utilizing the principles of uniformization theory [9, 17] and Bellman equation [1, 4], we propose an algorithm based on the best-response principle to efficiently compute the AE, which we demonstrate is more practical than the NE in the real market. We conduct extensive experiments to evaluate our algorithm which shows good convergence performance. The results indicate that our pricing policy outperforms existing strategies and the incentive to deviate from the AE is tiny. The market for selling reusable products is growing rapidly. Existing works for policy optimization often ignore the dynamic property of demand and the competition among providers. This paper studies service providers’ dynamic pricing in consideration of market competition and dynamics, which makes two key contributions. First, we propose a comprehensive model that takes into account the dynamic demand under market competition and formulate the optimal pricing policy as an equilibrium. Second, as it is difficult to compute the Nash equilibrium due to incomplete information and implicit revenue function, we develop an efficient algorithm to calculate an approximate equilibrium, which is more practical in the real world. The experiments show that the proposed policy outperforms existing strategies and the incentive to deviate the approximate equilibrium is small. 1. INTRODUCTION In many real-world applications, the service providers’ resources are reusable. Dynamic pricing policy plays an important role in making profits from price-sensitive users, which has shown great success in industries, e.g., the car rentals [7], hotel reservations [2, 19], network services [14], and the cloud computing [10, 20], and has attracted lots of research attention [5, 11, 15]. There are two important properties for the market: 1) users’ demand is stochastic over time, which leads to dynamic inventories; and 2) providers that offer similar services need to compete against each other. However, existing works have partially neglected or treated these characteristics in an inadequate way. Against this background, this paper investigates dynamic pricing to match demand with inventory in order to maximize providers’ long-term revenues in the competitive market, which gives solid theoretical and experimental analyses and makes two key contributions. First, we propose a comprehensive model to describe the real-world applications with multiple providers and stochastic user demand, where a product can be reused, e.g., resources in a cloud platform. Existing works ignore either the competition or the dynamic feature. Demand forecast is studied in [7, 19] and the most widely-used model to describe users’ dynamic demand is the Poisson process [5, 6, 14, 20]. However, those works do not consider the mar- 2. MODELING COMPETITIVE MARKET WITH STOCHASTIC DEMAND We use K to represent the set of service providers in the market. Following the common practice in the literature [5, 14, 21, 22], we assume that users’ demand for the service of provider k ∈ K is determined by two independent Poisson processes, namely the arrival process that models the coming of new demand and the departure process that corresponds to the leaving of existing requests. Specifically, we use λk (·) to represent the Poisson arrival rate (number of new demand instances per unit time) for provider k, which satisfies the following properties [3] λk (p) ≥ 0, ∂λk (p)/∂pk < 0 and ∂λk (p)/∂pk′ 6=k > 0, where p = (p1 , p2 , . . . , p|K| ). Similarly, the Poisson departure process is modeled by µk (·), which satisfies that µk (p) ≥ 0, ∂µk (p)/∂pk > 0 and ∂µk (p)/∂pk′ 6=k < 0. We use the notation (pk , p−k ) = p. Let Nk be the maximal capacity of provider k and [Nk ] denote the set {0, 1, . . . , Nk }. Since both the arrival and departure of demand are random process, the number of instances used by customers can be formulated as a continuous-time Markov process and the pricing policy of provider k is represented as Pk = (pk,0 , pk,1 , . . . , pk,Nk ), Appears in: Proc. of the 16th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2017), S. Das, E. Durfee, K. Larson, M. Winikoff (eds.), May 8–12, 2017, São Paulo, Brazil. Copyright c 2017, International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved. 1719 provider k can observe others’ P−k and π−k and then optibk (P−k |π−k ), which will make the mize his/her policy with B policy to converge to the Pb∗ eventually. where pk,n is the price for the state n. Then the trank sition rate matrix for provider k is Qk (P ) = (qi,j (P ))i,j , i, j ∈ [Nk ]:  π (P ) −k  Ep−k ∈P−k {λk (pk,i , p−k )}, if j = i + 1;    π−k (P ) Ep−k ∈P−k {µk (pk,i , p−k )}, if j = i − 1; k qi,j (P ) = (1) − P q k (P ), if j = i;  i,l  l6 = i   0, otherwise, 4. EXPERIMENTAL EVALUATION We use the following arrival and departure rate functions P in our experiments: λk (p) = lk (1 − p2k ) uk p2k P 2 i6=k pi |K|−1 and µk (p) = 2 i6=k (1−pi ) , where lk and uk are parameters. To evaluate the benefits of the proposed Pb∗ , we compare it with the existing optimal dynamic pricing [5, 14, 20], which maxP Nk imizes k=1 πk (Pk )npk,n for each provider k without consideration of others’ strategy profile P−k . The results are shown in Figure 1, which indicate that the noncompetitive strategy will lead to about 10% drop of revenue as compared with Pb∗ . The evaluation for the tightness of ǫ is depicted in Table 1. We see that the benefit of deviating from Pbk∗ is very limited. Thus, it is reasonable to assume providers to use Pbk∗ – a more realistic equilibrium strategy that can be computed under both full and partial information assumptions. where πk (P ) is called the stationary (or steady-state) probaP bility satisfying n∈[Nk ] πk,n (P ) = 1 and πk (P )·Qk (P ) = 0. When n instances are being used by customers, provider k can receive n · pk,n revenue per unit time. Thus the average long-term expected revenue rate for provider k is X Nk πk,n (P ) · n · pk,n . (2) Jk (Pk , P−k ) = n=0 3. OPTIMAL DYNAMIC PRICING We first introduce the notation of NE. Definition 1 (Nash equilibrium). A Nash equilibrium is a pricing policy profile P ∗ = ×k∈K Pk∗ , such that ∗ ∗ ∀k ∈ K, Jk (Pk∗ , P−k ) ≥ Jk (Pk , P−k ) for all Pk . |K|−1 That is, no one can gain higher revenue rate by unilateral changing his/her equilibrium policy. Motivated by this observation, the NE can be computed by a best response procedure, which optimizes each provider j’s pricing Pj while keeping others’ P−j fixed in each iteration, until no one wants to change his/her pricing policy. However, Jk (Pk , P−k ) is not an explicit function with respect to P = (Pk , P−k ). To address this challenge, when we optimize provider k’s policy in the best-response procedure, we view the steadystate probabilities π−k of others as fixed (i.e., they do not change with P ). Provider k’s stationary probability under this assumption, P π bk (P |π−k ), can be calculated based on the bk,n (P |π−k ) = 1 and π bk (P |π−k ) · linear equations n∈[Nk ] π Qk (P |π−k ) = 0, where Qk (P |π−k ) is the same with Qk (P ) except that π−k (P ) in Eq.(1) is replaced with π−k . The corresponding revenue rate with fixed π−k is X Nk π bk,n (P |π−k ) · n · pk,n . (3) Jbk (Pk , P−k |π−k ) = 5 Approximate equilibrium Noncompetitive Maximal Rvenue Rate Maximal Revenue Rate 5 4.5 4 3.5 3 Provider 1 Provider 2 4 3.5 3 Provider 3 Approximate equilibrium Noncompetitive 4.5 Provider 1 (a) S1 Provider 2 Provider 3 (b) S2 Figure 1: Strategy Comparison n=0 Setting k lk uk Nk Jbk∗ (·) ǫk ǫk /Jbk∗ (·) S1 1 2 3 2 1.6 1.2 1 1 1 6 6 6 4.6390 4.5194 4.3506 .0620 .0553 .0463 1.33% 1.22% 1.06% S2 1 2 3 1.6 1.6 1.6 0.8 1 1.2 6 6 6 4.6753 4.5603 4.4589 .0626 .0572 .0396 1.34% 1.25% 0.89% Table 1: Tightness of ǫ The optimal (best-response) policy that maximizes the above bk (P−k |π−k ) = arg maxP ∈∆ revenue rate is defined as B k k Jbk (Pk , P−k |π−k ), which can be computed with Bellman Equation. When the random best response algorithm termi∗ bk (Pb−k nates, it follows that, ∀k ∈ K, Pbk∗ = B |π−k (Pb∗ )) and ∗ ∗ ∗ ∗ hence Jbk (Pbk , Pb−k |π−k (Pb )) ≥ Jbk (Pk , Pb−k |π−k (Pb∗ )) for all Pk ∈ ∆k . The policy Pb∗ is not a NE according to Definition 1, which is an AE, as defined below. 5. CONCLUSION AND FUTURE WORK Definition 2 (Approximate equilibrium). An ǫ- approximate equilibrium is a pricing policy profile Pb∗ = ×k∈K Pbk∗ with a vector ǫ = (ǫ1 , ǫ2 , . . . , ǫ|K| ), such that ∀k ∈ K, Jk (Pbk∗ , ∗ ∗ ) for all Pk . ) + ǫk ≥ Jk (Pk , Pb−k Pb−k The ǫk can be viewed as the additional revenue provider k can gain by unilaterally deviating from Pb∗ , which is shown to be very small in the experiments. If ǫk = 0 for all k ∈ K, then the AE is equal to the NE. The policy Pb∗ is more practical than P ∗ in the real world with incomplete information because providers cannot calculate P ∗ , however, each We studied the dynamic pricing optimization problem for the service providers selling reusable products and made two main contributions. First, we proposed a comprehensive model that captures the dynamic and competitive features of the market. Second, we formulated providers’ optimal pricing policies as an approximate equilibrium and developed an efficient algorithm to solve it. Our experimental results showed that the policy we computed outperforms existing methods in the literature. In future work, we will investigate the structural properties of the AE policy, e.g., its monotonicity with respect to the capacity utilization. Besides, we will propose more efficient algorithms to compute the best-response strategy and the AE. Acknowledgments This paper is supported by 2015 Microsoft Research Asia Collaborative Research Program. 1720 REFERENCES [12] N. Nisan, T. Roughgarden, E. Tardos, and V. V. Vazirani. Algorithmic game theory, volume 1. Cambridge University Press, 2007. [13] J. R. Norris. Markov chains. Cambridge University Press, 1998. [14] I. C. Paschalidis and J. N. Tsitsiklis. Congestion-dependent pricing of network services. IEEE/ACM Transactions on Networking, 8(2):171–184, 2000. [15] A. Şen. A comparison of fixed and dynamic pricing policies in revenue management. Omega, 41(3):586–597, 2013. [16] R. G. Simmons and H. L. Younes. Solving generalized semi-Markov decision processes using continuous phase-type distributions. In Proceedings of the Nineteenth National Conference of Artificial Intelligence, pages 742–747, 2004. [17] W. J. Stewart. Probability, Markov chains, queues, and simulation: The mathematical basis of performance modeling. Princeton University Press, 2009. [18] H. Tsaknakis and P. G. Spirakis. An optimization approach for approximate Nash equilibria. Internet Mathematics, 5(4):365–382, 2008. [19] L. R. Weatherford and S. E. Kimes. A comparison of forecasting methods for hotel revenue management. International Journal of Forecasting, 19(3):401–415, 2003. [20] H. Xu and B. Li. Maximizing revenue with dynamic cloud pricing: The infinite horizon case. In 2012 IEEE International Conference on Communications, pages 2929–2933, 2012. [21] H. Xu and B. Li. Dynamic cloud pricing for revenue maximization. IEEE Transactions on Cloud Computing, 1(2):158–171, 2013. [22] X. Xu and W. J. Hopp. A monopolistic and oligopolistic stochastic flow revenue management model. Operations Research, 54(6):1098–1109, 2006. [1] R. E. Bellman and S. E. Dreyfus. Applied dynamic programming. Princeton University Press, 2015. [2] S. Choi and A. S. Mattila. Hotel revenue management and its impact on customers’ perceptions of fairness. Journal of Revenue and Pricing Management, 2(4):303–314, 2004. [3] E. Dockner and S. Jørgensen. Optimal pricing strategies for new products in dynamic oligopolies. Marketing Science, 7(4):315–334, 1988. [4] S. Dreyfus. Richard bellman on the birth of dynamic programming. Operations Research, 50(1):48–51, 2002. [5] W. Elmaghraby and P. Keskinocak. Dynamic pricing in the presence of inventory considerations: Research overview, current practices, and future directions. Management Science, 49(10):1287–1309, 2003. [6] G. Gallego and G. Van Ryzin. Optimal dynamic pricing of inventories with stochastic demand over finite horizons. Management Science, 40(8):999–1020, 1994. [7] M. K. Geraghty and E. Johnson. Revenue management saves national car rental. Interfaces, 27(1):107–127, 1997. [8] K. Gopalratnam, H. Kautz, and D. S. Weld. Extending continuous time Bayesian networks. In Proceedings of the National Conference on Artificial Intelligence, volume 20, page 981, 2005. [9] D. Gross and D. R. Miller. The randomization technique as a modeling tool and solution procedure for transient Markov processes. Operations Research, 32(2):343–361, 1984. [10] V. Kantere, D. Dash, G. Francois, S. Kyriakopoulou, and A. Ailamaki. Optimal service pricing for a cloud cache. IEEE Transactions on Knowledge and Data Engineering, 23(9):1345–1358, 2011. [11] Y. Levin, J. McGill, and M. Nediak. Dynamic pricing in the presence of strategic consumers and oligopolistic competition. Management Science, 55(1):32–46, 2009. 1721