Competing for consumer's attention

Automatica 44 (2008) 361 – 370 www.elsevier.com/locate/automatica Competing for consumer’s attention夡 Guiomar Martín-Herrán a , Olivier Rubel b , Georges Zaccour c,∗ a Dpto. Economía Aplicada (Matemáticas), Universidad de Valladolid, Avda. Valle Esgueva, 6 47011 Valladolid, Spain b GERAD and Marketing Department, HEC Montréal, 3000 Côte-Sainte-Catherine, Montréal, Canada H3T 2A7 c Chair in Game Theory & Management, GERAD, HEC Montréal, 3000 Côte-Sainte-Catherine, Montréal, Canada H3T 2A7 Received 12 September 2006; received in revised form 26 April 2007; accepted 1 June 2007 Available online 12 September 2007 Abstract We consider an infinite-horizon differential game played by two direct marketers. Each player controls the number of emails sent to potential customers at each moment in time. There is a cost associated to the messages sent, as well as a potential reward. The latter is assumed to depend on the state variable defined as the level of the representative consumer’s attention. Two features are included in the model, namely, marginal decreasing returns and bounded rationality. By the latter, we mean that the representative consumer has a limited capacity for processing the information received. The evolution of this capacity depends on its level, as well as on the emails sent by both players. This provides environmental flavour where, usually, one player’s pollution emissions (here emails) also affect the payoff of the other player by damaging the common environment (here, the stock of consumer attention). We characterize competitive equilibria for different scenarios based on each player’s type, i.e., whether the player is a spammer or not. We define a spammer as a myopic player, i.e., a player who cares only about short-term payoff and ignores the impact of her action on the state dynamics. In all scenarios, the game turns out to be of the linear-quadratic variety. Feedback Nash equilibria for the different scenarios are characterized and the equilibrium strategies and outcomes are compared. Finally, we analyze the game in normal form, where each player has the option of choosing between being a spammer or not, and we characterize Nash equilibria. 䉷 2007 Elsevier Ltd. All rights reserved. Keywords: Electronic business dynamics; Electronic mail; Direct marketing; Differential games; Spam 1. Introduction Assume your competitor is a spammer; how would you react to this in terms of emailing strategy to your potential customers? Would it drive you to behave like her, or would you remain a good citizen (non-spammer)? Basically, these are the research questions we wish to tackle in this paper. More specifically, we are interested in analyzing the strategic behavior of competing Internet marketers. Direct marketers that are non-spammers are suffering their spamming colleagues for various reasons. First, spammers are fierce, low-cost competitors. Apparently, it is only necessary 夡 This paper was not presented at any IFAC meeting. This paper was recommended for publication in revised form by Editor Berc Rüstem. ∗ Corresponding author. E-mail addresses: [email protected] (G. Martín-Herrán), [email protected] (O. Rubel), [email protected] (G. Zaccour). to complete one transaction per one million emails sent to be profitable.1 Second, they are not only tapping into the pocketbook of the same consumer as the non-spammers, but also into the same consumer’s attention stock. This implies a reduction in the efficiency of the non-spammers’ marketing campaigns. Third, in their battle against spam, by e.g., changing their email addresses and installing filters, consumers are (intentionally or not, the result is the same) lowering the value of email databases, which are an important asset, for firms doing business on the Internet.2 Actually, spam is an issue for all Internet stakeholders (Sipior, Ward, & Bonner, 2004). Spam is creating congestion, which is not cost free, on the networks of all Internet service providers. Firms have to deal with the 1 According to an article in the Spanish newspaper, El País, in its issue dated February 19, 2006. 2 According to a study (Industry Canada, 2005), 16% of email address changes are due to spam. 0005-1098/$ - see front matter 䉷 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.automatica.2007.06.009 Electronic copy available at: http://ssrn.com/abstract=1474404 362 G. Martín-Herrán et al. / Automatica 44 (2008) 361 – 370 management and safety of their computer systems and internal networks, and also with loss of productivity due to the time employees spend processing junk emails.3 Individuals are irritated by the overwhelming volume of email they receive and are concerned about their privacy. Finally, governments and international bodies (OECD, EU, etc.) are interested in finding legal and technological solutions for reducing spam, in order to respond to the concerns of their constituencies.4 Many disciplines are interested in the spam phenomenon. For instance, computer scientists are devoting considerable effort to designing anti-spam protection systems and intelligent agents for managing email. Researchers in law and public decisionmakers are looking for legal frameworks to reduce the burden of spam. Economists have been researching, among other things, the costs and the welfare implications of this form of pollution, as well as the problems caused by consumer information overload. This paper is mainly related to this last topic and to the strategic behavior of direct marketers. We consider an email to be a communication emanating from a firm to a potential customer. The latter needs to process the email to obtain information from it, e.g., on the product, the price, the delivery conditions, etc. If there are many senders (or even a few sending many emails), this potential customer may quickly become overloaded with information. Some papers, e.g., Jacoby, Speller, and Berning (1974), Malhorta (1982), Keller and Staelin (1987), Lurie (2004), and Lee and Lee (2004), have looked into the impact of information overload, in terms of either quantity or quality and structure of information, on the evaluation of alternatives by customers, or on the optimality of consumer decisions. Actually, the problem of information overload would not exist if the consumer’s cognitive capacity were unlimited. Therefore, if the consumer’s bounded rationality is acknowledged, then too much information necessarily leads to a decrease in the customer’s attention (Simon, 1997) and in the number of alternatives evaluated. Then, a lower response rate to direct-marketing offers is expected to follow. Research devoted to information overload started well before the Internet (and spam) era. However, the low cost of designing electronic direct-marketing campaigns and the absence of entry barriers have emphasized, more than ever, the common (public-good) nature of the consumer’s attention. MacFadden (2001) argues that the management of the digital commons is perhaps the most critical issue of market design that our society faces. Van Zandt (2004) considers the competition for attention when senders interact strategically and face information-overloaded receivers, and proposes an attention allocation mechanism. Anderson and de Palma (2005) assess the tragedy of the commons associated with customer attention and evaluate the welfare implications of sending too much information. Shiman (1996, 1997) also deals with the welfare implications of the decrease in the cost of technolo3 The cost for US corporations has been estimated at $8.9 billion (Industry Canada, 2005). Ferris Research, a consultancy, evaluates the loss to the European economy, due to spam, at $2.5 billion (cited in El País in its issue dated February 19, 2006). 4 Half of the states in the USA already have laws forbidding spamming. gies for getting information about customers and for sending them messages. This burgeoning literature has lacked a definition of what a spammer is (which may have been seemed too obvious) and has not taken into account the heterogeneity in sender type: spammers or non-spammers. Clearly, information overload could occur even when all firms are good citizens. In this paper, we wish to test whether or not heterogeneity itself affects all senders’ behavior. To attempt to answer our research questions, we build a parsimonious model where two direct marketers compete for consumer attention. Our simple model includes the following features: • Dynamics: The representative consumer is endowed with an attention’s capacity, that evolves overtime. The adoption of a dynamic rather than static framework allows us to make the distinction between flows (emails) and stock (consumer capacity). Also, it also allows us to take into account the carryover effects of emails and spam on this capacity. This means that the consumer remembers these events, which seems reasonable, simply by relying on her own experience. • Strategic competition: All firms are drawing from the same pool of consumer attention (and the same pocketbooks); hence, it is mandatory that we incorporate competition in the model. A response model can ignore competition, accommodate for it in a passive way, i.e., by fixing the values of the decision variables for the competitor and assuming no reaction to a firm’s actions, or by considering strategic interactions.5 Our research questions point quite naturally towards a model where players can react to each other. • Different types of players: On addition to considering that each player influences the other’s strategy, we wish to assess the impact of having heterogeneous players. Thus, we assume that each player can choose between being a spammer or not. This requires us to define precisely what is a spammer. The spammer is a player who optimizes her short-term (or current) payoff. This means, synonymously, that this player disregards the evolution of the system (the consumer’s attention), or is myopic, i.e., having an infinite discount rate. Our main results can be summarized as follows: • The steady state of the consumer’s attention decreases with the number of spammers. • A non-spammer sends more emails when facing a spammer than when her competitor is a non-spammer firm. • The Nash equilibrium of the game where each player chooses her type depends on the initial stock of attention. 5 Note that there is an important direct-marketing literature interested in developing tools to describe customer response rates to direct offers (thus including emails) in terms of, e.g., the frequency and monetary value of the purchases. For instance, econometric models are designed to segment consumers and forecast their response rates (see, e.g., Bult & Wansbeek, 1995). Mathematical programming approaches, on the other hand, seek to optimize the frequency of mailing campaigns (see, e.g., Bitran & Mondschein, 1996; Gönül & Ze Shi, 1998; Piersma & Jonker, 2004). Strategic interaction between multiple senders on customers’ response rates has been largely neglected in these approaches. Electronic copy available at: http://ssrn.com/abstract=1474404 363 G. Martín-Herrán et al. / Automatica 44 (2008) 361 – 370 The rest of the paper is organized as follows. In Section 2, we introduce the differential game model of competition for consumer attention. In Section 3, we characterize Nash equilibria in the different scenarios and compare the resulting strategies and steady states. In Section 4, we analyze the game in normal form, where each player chooses her type. In Section 5, we briefly conclude. To help interpret the function G(x), and as a benchmark, we state the following lemma, which characterizes the steady state of the resource if no emails were sent. The lemma also allows us to determine the value of the upper bound on the attention stock xmax . Lemma 1. If no email is sent, then the attention stock converges to L/. 2. Model To focus on strategic behavior, and following a long tradition in prisoner’s dilemma literature, we shall assume away all sources of asymmetries. Put differently, we suppose that the players are symmetric in all of the problem’s data except type, which is either spammer or non-spammer. Clearly, this symmetry assumption does not correspond to the above statement that a spammer is a low-cost competitor to a non-spammer. However, we prefer to keep a better control on the experiment we are conducting with this model. Let time t be continuous. Denote by x(t), t ∈ [0, ∞), the capacity of the representative consumer to process the information inflow (emails received). We assume that this capacity, called consumer attention,6 is non-negative and bounded, i.e., 0 x(t) xmax , ∀t ∈ [0, ∞), where xmax is the upper bound. The rate of variation of this stock depends on two factors, namely, the rate of depletion (or use) and the rate of regeneration. We assume that depletion results from the entering emails. To keep things simple, we suppose that the set of senders (players) is made up of two firms sending emails at rate ni (t), t ∈ [0, ∞). This leads to a loss of attention measured by H (n1 (t), n2 (t)), a non-negative and increasing function in both arguments. On the other hand, the regeneration rate is given by a non-negative function that we denote G(x). As a result, the evolution of the consumer’s attention is captured by the following differential equation: ẋ(t) = dx(t) = G(x(t)) − H (n1 (t), n2 (t)), dt x(0) = x0 , where x0 denotes the initial stock of attention. Since not much insight can be obtained from the above general dynamics, we shall assume that both G(x(t)) and H (n1 (t), n2 (t)) can be well approximated by the following linear functions: G(x(t)) = L − x(t), L,  > 0, H (n1 (t), n2 (t)) = (n1 (t) + n2 (t)),  > 0. With these specifications, the evolution of the stock becomes ẋ(t) = L − x(t) − (n1 (t) + n2 (t)), x(0) = x0 . (1) The constant L is the regeneration rate when the stock converges to zero and  is the natural decay rate of attention. The parameter  is a scaling factor transforming emails (sent and received) into loss of attention or depletion of the resource. 6 This concept of consumer attention could be related to the notion of email acceptance in Chen and Sudhir (2004). Proof. If no email is sent, the attention trajectory over time is given by the solution of ẋ(t) = L − x(t), x(0) = x0 , where x0 denotes the initial attention level. The time trajectory of the customer attention stock is x(t) = L (1 − e−t  ) + x0 e−t  ,  which converges to L/ when t goes to infinity.  The above lemma shows that, even if no emails were sent, i.e., the capacity were not used, the stock is still bounded, thanks to the natural decay rate. Note that if the initial attention level, x0 , is lower than L/, then the attention stock converges to the upper bound L/ and ∀t, x0 x(t) L/. However, if x0 is greater than L/, then the attention stock converges to the lower bound L/ and ∀t, L/x(t)x0 . From now on, we focus on the first scenario7 (x0 x(t)L/ = xmax ) and therefore, the natural growth of the attention is always positive on the interval [0, L/], but the growth is decreasing with respect to x. Note that  represents the speed of convergence to the (here upper) bound (L/), i.e., the higher the value of , the faster the consumer “recovers” from past received messages. The “production” of ni (t) emails by firm i, i = 1, 2, implies a cost, which is independent of the consumer’s attention and denoted Ci (ni (t)), and a revenue, assumed to depend on both the consumer’s attention and the number of emails sent, Ri (ni (t), x(t)). We shall hereunder skip the time argument when no ambiguity may arise. The (total) cost can be schematically decomposed into two components: the sending cost8 and the preparation cost. The latter includes, e.g., the design, the message content, the targeting, the updating of the database, etc. For instance, the firm has to frequently change the design of its message to attract the consumer’s attention. Also, given the frequency with which consumers change email addresses, the firm has to continuously invest in updating its database, a main asset for direct marketers. Further, the firm has to invest in competitive and technological intelligence to follow developments in new viruses, filters, etc. We believe that all these items can be captured by an increasing convex cost function that we take, for simplicity, 7 A similar analysis could be done under the hypothesis x  L/. 0 8 Martin, Van Durme, Raulas, and Merisavo (2003) evaluate the sending cost to be in the range of $5–$7 per 1000 messages. Note that this cost is between $500 and $700 for traditional direct-marketing media. 364 G. Martín-Herrán et al. / Automatica 44 (2008) 361 – 370 to be quadratic: Ci (ni ) = n2i . 2 (2) Note that multiplying this cost by a positive constant, different from one, would not qualitatively change the results. On the revenue side, we require the reward (i) to be zero if the attention’s stock is (momentarily) exhausted, or if no mail is sent, (ii) to be increasing with the stock of attention and with the production (i.e., the number of emails sent), and (iii) to exhibit a positive interaction between the control ni and the state x. This last item implies that, for a given attention level, the higher the number of emails sent, the higher is the revenue. Similarly, for a given number of emails sent, the higher the attention level, the higher the revenue. Although many functional forms could easily satisfy these requirements, we adopt for its simplicity and interpretability, the following multiplicatively separable function: Ri (ni , x) = (x)g(ni ), with (x) = x xmax = x , L g(ni ) = ri ni , ri > 0. Clearly, Ri (ni , x) has the following properties: Ri (ni , 0) = Ri (0, x) = Ri (0, 0) = 0, jRi (ni , x) 0, jx jRi (ni , x) 0, jni j2 Ri (ni , x) > 0. jni jx Furthermore, note that (x) satisfies 0 (x) 1, ′ the number of emails to be sent in order to maximize her objective function (3). However, a non-spammer firm also cares about the long-term payoff and takes into account the dynamics of consumer attention, given by (1) when maximizing her objective function (3). We characterize the competitive equilibria for three different scenarios: first, neither player is a spammer; second, only one player behaves as a spammer; third, both players are spammers. The resulting equilibrium payoffs will form the entries in the matrix of the game in normal form, where each player chooses to be a spammer or not. 3. Equilibria In the previous section, we defined by (1) and (3) an infinite-horizon linear-quadratic differential game between two emailers.9 We focus on stationary affine symmetric strategies. The stationary assumption, is standard in infinite-horizon differential games, and means that the players’ value functions, as well as their optimal strategies, do not depend explicitly on time. The linear and symmetric assumptions are also usually adopted in most of the applications of differential games involving a symmetric linear-quadratic structure and aiming at analytical results. (Note, however, that the game may admit non-linear and/or non-symmetric strategies, see Engwerda, 2005 for a full coverage of linear-quadratic differential games.) In the following propositions we characterize symmetric stationary feedback Nash equilibrium strategies for the different scenarios. Proposition 1. The symmetric stationary feedback Nash equilibrium emailing strategy is n∗ (NS, NS; x) = rx − (A1 x + A2 ) if x   (x) 0, (0) = 0, which confers to this function a propensity interpretation, which is appealing. Indeed, one expects the consumer to respond to an offer imbedded in an email with a certain “probability.” Without rendering the model stochastic, and hence more complex, the idea of a “probabilistic” response is captured to some extent by (x). Further, given our assumption of strictly increasing convex costs, the linear specification of g(ni ), instead of having a more classical concave one, is less severe. Without any loss of generality, we shall normalize the maximum attention stock to be equal to one, hence taking  = L. Assuming a profit-optimization behavior, the objective functional for player i then reads as follows:    ∞ n2i −t dt, (3) max Ji = e r i ni x − ni  0 2 0 where  denotes the constant discount rate. Our objective is the study of the competition for consumer attention under different scenarios, depending on whether or not the firm is a spammer. As stated earlier, a spammer is assumed to behave as a myopic player, in the sense that she does not take into account the effect of her action on the dynamics of the attention stock. A spammer decides the optimal path of A2  , r − A1  (4) and zero otherwise, where NS, NS denotes that both players are non-spammers. The symmetric firm’s value function V (NS, NS; x) is given by V (NS, NS; x) = 1 2 A1 x 2 + A2 x + A3 , (5) where A1 = 2L + 4r +  −  (2L + 4r + )2 − 12r 2 2 62 A1 L > 0, L + 2r − 3A1 2 +  A2 (2L + 3A2 2 ) A3 = > 0. 2 > 0, (6) A2 = Proof. See Appendix B. (7)  The optimal emailing strategy is a trade-off between the marginal reward and the marginal cost of sending an email. Indeed, the marginal revenue is given by rx. The marginal total 9 In Appendix A the differential game (1)–(3) is rewritten in the standard linear-quadratic format. We are indebted to one reviewer for suggesting this reformulation. 365 G. Martín-Herrán et al. / Automatica 44 (2008) 361 – 370 loss is the sum of the marginal cost (ni ) and the loss in terms of the consumer’s attention. The latter is given by the product of the marginal impact of sending an email on the evolution of the stock of attention, i.e., jẋ/jni = −, valued at the shadow price of the stock, i.e., V ′ (x)=A1 x +A2 . An important, albeit obvious, observation for the sequel is that, in her strategy, a farsighted player takes into account both the direct (or immediate) impact and the indirect (or future/dynamic) effects of emailing. Actually, Mahajan and Venkatesh (2000) remark that accounting for both effects is a welcome move in e-business models. Corollary 1. The non-zero emailing strategy is strictly increasing with the attention stock. Proof. It is easy to prove that r − A1 > 0 (see the expression of A1 in Appendix A), and hence the result.  An implication of this result is that the higher the attention, the higher the number of emails sent by the firm, i.e., (n∗ (NS, NS; x))′ = r − A1 > 0. The value of the attention at the steady state, xss (NS, NS), is obtained after replacing the equilibrium strategies in (1), and solving for x when ẋ(t) = 0: xss (NS, NS) = L + 2A2 2 . L + 2r − 2A1 2 (8) The steady state is always non-negative and lower than one. Thus, excluding the uninteresting case where neither player sends any email, the consumer’s attention is, as expected, not at its maximal value in the steady state. If we define information overload (IO) as the difference in the consumer’s attention stock between its maximal value and the steady state, then 2(r − (A1 + A2 )) IO = 1 − xss (NS, NS) = . L + 2r − 2A1 2 Clearly, IO ∈ [0, 1], and its actual level in steady state depends on the parameters’ values. The next proposition provides some static comparative results. regeneration of the stock. Finally, the more impatient are the players (higher ), the more they use the resource and the lower is the steady-state value of the consumer’s attention.10 Assume now that at least one of the two firms is a spammer. Recall that a spammer is a player who disregards the state dynamics. Therefore, the optimal spamming strategy is derived by solving, at each moment in time, a static optimization problem, i.e., maximizing the instantaneous profit. We shall suppose that player 1 is the non-spammer firm and player 2, the spammer. The latter optimization problem is thus    ∞ 1 −t max J2 = e rx − n2 n2 dt. (9) n2  0 2 0 The next proposition characterizes the non-spammer and spammer optimal strategies and value functions. Proposition 3. When player 1 is a non-spammer and player 2 is a spammer, the feedback Nash equilibrium strategies are given by n∗1 (NS, S; x) = rx − (B1 x + B2 ) Proof. Replace in (8) the expressions of the parameters A1 and A2 given in (6) and (7). Take the partial derivatives with respect to the different parameters. After straightforward but tedious computations, the results are achieved.  The results are quite intuitive. Indeed, increasing L shifts the stock upward, all else being equal. When sending an email becomes more attractive, i.e., when the value of r is higher, the players increase their sending activities which in turn reduces the stock. The result of varying  can be explained as follows. Increasing the marginal damage cost  leads the players to decrease their sending activities but at a lower pace than the B2  0, r − B1  (10) and zero otherwise; n∗2 (NS, S; x) = rx. (11) The players’ value functions are as follows: V1 (NS, S; x) = 21 B1 x 2 + B2 x + B3 , V2 (NS, S; x) = 1 (rx)2 , 2 (12) (13) where arguments (NS, S) denote that the first player is a nonspammer and the second player is a spammer, and  2L + 4r +  − (2L + 4r + )2 − 4r 2 2 B1 = > 0, 22 B2 = Proposition 2. (i) Increasing L increases the steady state of the consumer’s attention. (ii) Increasing , r or  decreases the steady state of the consumer’s attention. if x  B1 L > 0, L + 2r − B1 2 +  Proof. See Appendix B. B3 = B2 (2L + B2 2 ) > 0. 2  Later, we shall compare the results (strategies and steadystate values) obtained under the different behavioral assumptions. For the moment, we observe that the spammer’s strategy is independent of what the other player is doing (which is observable in reality). This is due to the fact that each firm’s objective function depends only on her decision variable. Hence, by not seeing the interaction between the players’ strategies in the state dynamics, the spammer is also ignoring the competitor when optimizing her payoff. 10 The results in this proposition hold true for L,  and r in the other scenarios and will not be repeated. The result for  applies in the scenario with one spammer. In the case of two spammers, the steady state is independent of . 366 G. Martín-Herrán et al. / Automatica 44 (2008) 361 – 370 The steady-state level of the attention stock in this context is given by xss (NS, S) = L + B2 2 . L + 2r − B1 2 (14) It is easy to prove that 0 xss (NS, S)1. The last scenario involves two myopic players. Given that a myopic player implements the strategy in (11) irrespective of what the other competitor is doing, the equilibrium strategies in this scenario are given by n∗i (S, S; x) = rx, i = 1, 2. (15) The value function of any myopic player is Vi (S, S; x) = 1 (rx)2 , 2 Proposition 5. The steady-state attention stocks satisfy the following inequalities: i = 1, 2, (16) 1 xss (NS, NS) xss (NS, S) xss (S, S)0. where arguments (S, S) denote that both players are spammers. Inserting the two optimal spamming strategies given by (15) in (1), and solving for x when ẋ(t) = 0, we get the steady-state attention level xss (S, S) = presence of a spammer has a dual effect on the number of emails sent: one (tautological) direct effect, i.e., the spammer sends more emails than if she were not a spammer; and an indirect one, the non-spammer also sends more emails. Therefore, the good citizen is pushed into an escalation strategy. This points towards the result that the two emailing policies are strategic complements.11 Therefore, if we extrapolate to the case where the spammer faces a lower cost than does the non-spammer, then we can conjecture that everything would be worsened, i.e., even more emails sent and a lower steady-state value for the consumer’s attention. The following proposition is somehow a direct consequence of the previous one. L . L + 2r Proof. It suffices to compare the expressions of the steady states for the different scenarios, given by (8), (14) and (17), taking into account the Riccati equations that define the coefficients of the value functions.  (17) 4. To spam or not to spam: a strategic choice Note that 0 xss (S, S) 1. 3.1. Comparison The next propositions compare the equilibrium emailing strategies and the consumer’s attention levels, obtained under the different scenarios. Proposition 4. The equilibrium emailing strategies compare as follows: n∗ (NS, NS; x) n∗1 (NS, S; x) n∗2 (NS, S; x) = n∗ (S, S; x) ∀x ∈ [0, 1], where n∗ (NS, NS; x), n∗1 (NS, S; x), n∗2 (NS, S; x), n∗ (S, S; x) are given by (4), (10) and (11), respectively. Proof. From (10) and (11), inequality n∗1 (NS, S; x) n∗2 (NS, S; x) ∀x ∈ [0, 1] follows immediately since B1 , B2 > 0. From (4) and (10), the following equivalence is deduced: n∗ (NS, NS; x) n∗1 (NS, S; x) ⇔ (B2 −A2 )+(B1 −A1 )x 0. We suppose now that the two players have the opportunity to choose their type (spammer or non-spammer) and that this choice is based only on a comparison of the payoffs. The latter are given by the value functions evaluated at the initial attention stock x0 , under the different scenarios (SC), i.e., Vi (SC, SC; x0 ), i = 1, 2, SC ∈ {NS, S}. Table 1 shows the normal form of the game, where the decisions of player 1 are displayed in rows, and those of player 2, in columns. More specifically, the quantities in this matrix are as follows: • V (NS, NS; x0 ) is the value function of a non-spammer firm competing against another non-spammer firm, given in (5). • V1 (NS, S; x0 ) is the value function of a non-spammer firm competing against a spammer, given in (12). • V2 (NS, S; x0 ) is the value function of a spammer firm competing against a non-spammer, given in (13). • Finally, since the value function of a spammer firm competing against another spammer is the same as that of a spammer competing against a non-spammer, the payoffs in cell (2, 2) are both equal to V2 (NS, S; x0 ). Some easy computations allow us to establish that B1 − A1 < 0, B2 − A2 < 0. Therefore, the above inequality is satisfied for any positive value of the attention stock.  Table 1 Normal form of the game (1;2) NS S This proposition shows that, independently of the level of the attention stock, the non-spammer firm sends always a fewer number of emails than the competing spammer firm: (n∗1 (NS, S; x) n∗2 (NS, S; x) = n∗ (S, S; x)). Moreover, a nonspammer firm sends more emails when it is competing against a spammer firm than it does when its competitor is also a nonspammer: (n∗ (NS, NS; x) n∗1 (NS, S; x)). This means that the NS S (V (NS, NS; x0 );V (NS, NS; x0 )) (V2 (NS, S; x0 );V1 (NS, S; x0 )) (V1 (NS, S; x0 );V2 (NS, S; x0 )) (V2 (NS, S; x0 );V2 (NS, S; x0 )) 11 Strategic complementarity means that if one player increases the value of her strategic variable, the other player will do the same. Strategic substitutability works the other way around. 367 G. Martín-Herrán et al. / Automatica 44 (2008) 361 – 370 To analyze this game, we have to compare two second-order polynomials, i.e., P1 (x0 ) = V (NS, NS; x0 ) − V2 (NS, S; x0 ) and P2 (x0 ) = V1 (NS, S; x0 ) − V2 (NS, S; x0 ). By the definition of a Nash equilibrium, we have • (NS, NS) is a Nash equilibrium if and only if P1 (x0 ) 0. • (S, S) is a Nash equilibrium if and only if P2 (x0 ) 0. • (NS, S) and (S, NS) are Nash equilibria if and only if P1 (x0 ) 0 and P2 (x0 )0. The expressions of the polynomials P1 (x0 ) and P2 (x0 ) involve the coefficients of the value functions and are very complicated to allow for clear-cut results regarding their signs. However, we are still able to prove the following proposition. Proposition 6. 1. (NS, NS) is the unique Nash equilibrium if and only if x0 < x̃0 . 2. (S, S) is the unique Nash equilibrium if and only if x0 > x̂0 . 3. (NS, NS) and (S, S) are Nash equilibria if and only if x0 ∈ [x̃0 , x̂0 ]. The expressions of x̂0 and x̃0 are given in Appendix C. Proof. See Appendix C.  The proposition characterizes Nash equilibria in terms of the initial value of the attention stock, x0 . The pair of strategies (NS, NS) is the only Nash equilibrium if the initial attention stock is “low.” Conversely, if the initial attention stock is sufficiently high, then the pair (S, S) is the unique Nash equilibrium of the game. If the initial attention stock is of “intermediate” value, then the pairs (NS, NS) and (S, S) are Nash equilibria. The somewhat surprising result is that, regardless of the parameters’ values, are, the case where the two firms are of different types can never be an equilibrium. Clearly, this is not what we observe in reality. This apparent contradiction can be explained as a modelling issue, i.e., the symmetry assumption is not valid, or it reflects the fact that the direct-marketing industry has not yet reached an equilibrium with only one surviving type. As mentioned earlier, one way of leaving out symmetry would be to assume that the spammer firm faces an almost zero cost and a very low rate of return. The most likely impact is that the spammer will send even more emails, without however, having any impact on the characterization of the equilibrium. An alternative interpretation of the result is that, like in a prisoner’s dilemma situation, an off-diagonal equilibrium, i.e., an equilibrium involving different strategies by the two players, never occurs because of its very heavy cost to the player behaving “cooperatively”. that if entry into the direct-marketing industry remains wide open, with almost no operation costs, then we can expect a long-term deterioration in the attention level. The corollary is a low response rate to offers made by firms via the Internet. This is one more reason to find a solution to the spam phenomenon. Finally, an important conclusion is that the scenario involving different types of firms is not part of a Nash equilibrium. At the modelling level, one contribution of this paper is its clear definition of a spammer. This goes further than the previous characterizations of spammers based on email content. Unlike the economic literature dealing with information overload, our approach allows to explicitly assess the impact of spammers on consumer attention. Further, our model endogenizes the strategic choice regarding type made by a firm. Our model suffers from several limitations. First, the response functions are linear in the state variable mainly to preserve mathematical tractability. Considering non-linear response functions, at the cost, however, of having to fully rely on numerical methods to obtain certain results, may lead to different insights. Second, we have assumed that the only players are the firms sending emails. One could introduce a third party that regulates business conduct and examine its impact on the emailing strategies. Third, we have assumed that only the number of messages matters, without considering content or quality, which could be used by non-spammers to differentiate themselves from spammers. Finally, the number of players is fixed. It would be interesting to extend the model by considering this number as endogenously determined by the gains and losses made by each category of player, using an evolutionary game theory approach. The model we proposed is a first step towards understanding the competition for attention in an information-rich environment such as the Internet. It is also a first attempt to assess the impacts of the spam phenomenon from a dynamic perspective. The above improvements are only some of the many interesting research questions still open for investigation. Acknowledgments We wish to thank the two anonymous Reviewers for their constructive comments. Research supported by NSERC Canada. The second author’s research was partly supported by MEC under project SEJ2005-03858/ECON, con-financed by FEDER funds. Research initiated when the third author was a visiting professor at Universidad de Valladolid, under Grant SAB2004-0162. 5. Concluding remarks We analyzed a dynamic game between emailers who seek to capture consumer attention. Although the expressions of the strategies and outcomes are tedious, we are still able to obtain some qualitative insight from the results. Namely, we showed that the strategies used by a spammer and a nonspammer are very different, and that the steady-state attention stock decreases with the number of spammers. This suggests Appendix A. Reformulation of the differential game (1)–(3) Let us introduce the new variables: y1 (t) = e−1/2t x(t), vi (t) = e−1/2t ni (t), y2 (t) = e−1/2t , 368 G. Martín-Herrán et al. / Automatica 44 (2008) 361 – 370 and denote y(t) = [y1 (t) y2 (t)]T . The differential game (1)–(3) can be rewritten as  ∞ 1 − 21 r − min 2y T (t) vi (t) + (vi (t))2 dt 0 vi  0 2 0 −L − 21  L y(t) s.t. ẏ(t) = 0 − 21  − − + v1 (t) + v2 (t), 0 0 T y(0) = [x0 1] . Let ui = vi + 2[− 21 r 0]y, then the differential game can be rewritten into the standard linear-quadratic form:  ∞ − min {y T (t)Qi y(t) + uTi (t)Ri ui (t)} dt vi  0 s.t. 0 ẏ(t) = Ay(t) + B1 u1 (t) + B2 u2 (t), y(0) = [x0 1]T , where 1 2 −L − 2r − 0 1 2 1 0 Qi = − r , 0 0 2 A= L , − 21  1 Ri = . 2 Bi = − , 0 Assuming that the players choose their actions from the set of stabilizing feedback matrices, the equilibrium actions are obtained by finding all symmetric stabilizing solutions of the next set of coupled algebraic Riccati equations (see, for example, Engwerda, 2005): (A − S1 K1 − S2 K2 )T K1 + K1 (A − S1 K1 − S2 K2 ) + K1 S1 K1 + Q1 = 0, (A − S1 K1 − S2 K2 )T K2 + K2 (A − S1 K1 − S2 K2 ) + K2 S2 K2 + Q2 = 0, where Si = Bi Ri−1 BiT . Since our differential game is symmetric we focus on symmetric strategies, i.e., we assume that the equilibrium strategies for both players are the same. This assumption implies that K1 =K2 =K. Furthermore, since Q1 =Q2 =Q and S1 =S2 =S, the solution we are looking for is obtained by finding the stabilizing solution of the following equation: AT K + KA − 3KSK + Q = 0. This yields then the results in Propositions 1 and 3. solutions of the HJB equations. This equation for player i is given by   1 rx − ni ni + (Vi (NS, NS; x))′ Vi (NS, NS; x) = max ni  0 2 ×(L(1 − x) − n1 − n2 ) , (18) where Vi (NS, NS; x) denotes player i’s value function in the scenario where neither player is a spammer. The first-order optimality condition reads n∗i (NS, NS; x) = rx − (Vi (NS, NS; x))′ . We are assuming that both players are symmetric; therefore, we omit the subscript denoting each player. The symmetric emailing strategy is n∗ (NS, NS; x) = rx − (A1 x + A2 ) if x  A2  , r − A1  (19) and zero otherwise. Inserting (19) in (18), and assuming that the value function is quadratic due to the linear-quadratic structure of the model, and given by V (NS, NS; x) = 21 A1 x 2 + A2 x + A3 , (20) the coefficients A1 , A2 , A3 are determined by identification, as follows:  2L + 4r +  ± (2L + 4r + )2 − 12r 2 2 A1 = > 0, 62 A1 L > 0, A2 = L + 2r − 3A1 2 +  A2 (2L + 3A2 2 ) > 0. (21) A3 = 2 The value inside the square root in the expression of A1 is always positive and it is easy to prove that both roots in (21) are positive real numbers. Let A′1 be the root with the positive sign affecting the square root, and A′′1 the root with the negative sign. A sufficient condition guaranteeing that the expressions in (20) and (19) are firms’ value functions and emailing strategies is given by lim e−t V (NS, NS; x(NS, NS; t)) = 0, t→∞ (22) where x(NS, NS; t) is the solution of the closed-loop dynamics obtained after substitution of the optimal emailing strategies (19) into the attention stock dynamics given by (1). This solution can be written as Appendix B x(NS, NS; t) = (x0 − xss (NS, NS))e1 t + xss (NS, NS), B.1. Proof of Proposition 1 where xss (Ns, NS) refers to the steady state of the attention variable given by (8), and 1 is The sufficient condition for a stationary feedback Nash equilibrium requires us to find bounded and continuously differentiable functions denoted by Vi (NS, NS; x), i = 1, 2, which satisfy, for all x(t) 0, the Hamilton–Jacobi–Bellman (HJB) equations for player i = 1, 2. We first concentrate on finding (23) 1 = −L − 2r + 22 A1 . The quadratic functional specification in (20) allows (22) to be satisfied when the attention stock is bounded. This condition is guaranteed if the steady state is globally asymptotically stable. G. Martín-Herrán et al. / Automatica 44 (2008) 361 – 370 The attention dynamics, once the optimal strategy (19) has been replaced, is ẋ(t) = L(1 − x) − 2 (rx −  (A1 x + A2 )) . Collecting with respect to x, we get ẋ(t) = L + 2A2 2 + x(−L − 2r + 22 A1 ). The steady state, xss (Ns, NS), is globally asymptotically stable if and only if 1 = −L − 2r + 22 A1 < 0. After some manipulations, it can be proved that if A′1 is chosen, the value of 1 is always positive, leading to an unbounded attention stock. However, if A′′1 is selected, we have 2  + L + 2r − 3(r)2 , 2 1 =  − L − 2r − 2 which can easily be proved negative. This choice leads to a globally stable steady state, implying that, for any initial value of the attention stock, x0 , the optimal time path of the attention stock x(t) converges to the steady state xss (NS, NS). From (19) the optimal symmetric strategy is positive if and only if rx − (A1 x + A2 )0. Collecting this expression with respect to x, we get −A2  + x(r − A1 ). 369 we identify the following coefficients:  2L + 4r +  ± (2L + 4r + )2 − 4r 2 2 > 0, B1 = 22 B1 L > 0, B2 = L + 2r − B1 2 +  B3 = B2 (2L + B2 2 ) > 0. 2 (27) The value inside the square root in the expression of B1 is always positive and it is easy to prove that both roots in (27) are positive real numbers. Let B1′ be the root with the positive sign affecting the square root, and B1′′ the root with the negative sign. Along the same lines as in the proof of Proposition 1, we look for a globally asymptotically stable steady state, which implies bounded attention stock along its optimal time path given by x(NS, S; t) = (x0 − xss (NS, S))e2 t + xss (NS, S), (28) where xss (NS, S) refers to the steady state of the attention variable given by (14), and 2 is 2 = −L + B1 2 − 2r. The steady state, xss (NS, S), is globally asymptotically stable if and only if 2 < 0. It can be easily proved that this last condition can only be ensured if coefficient B1′′ is selected. The non-spammer’s optimal emailing strategy is positive if and only if rx − (B1 x + B2 ) 0. Since A2 0 and some straightforward manipulations show that r − A′′1  > 0, therefore the optimal strategy is non-negative as long as Collecting the terms in x A2  . x r − A′′1  Note that B2 0 and after some manipulations, it can be proved that B.2. Proof of Proposition 3 −B1′′ + r 0. Here we follow the same steps as in the proof of Proposition 1 to derive the equilibrium strategies under the assumption that firm 2 behaves as a spammer, while firm 1 is a non-spammer. The HJB equation for player 1 is   1 V1 (NS, S; x) = max rx − n1 n1 + (V1 (NS, S; x))′ n1  0 2 ×(L(1 − x) − n1 − n2 ) . (24) x(−B1  + r) − B2  0. Therefore, n∗1 (NS, S; x) is positive if and only if x B2  . r − B1′′  The optimal spamming strategy is derived straightforwardly from the first-order optimality condition. To get the expression of the spammer’s value function it suffices to replace the optimal spamming strategy given by (11) in the HJB equation associated with the spammer’s optimization problem. Appendix C The first-order optimality condition for player 1 is n∗1 (NS, S; x) = rx − (V1 (NS, S; x))′ . (25) Inserting (25) and (11) into (24), and assuming a quadratic value function such as V1 (NS, S; x) = 2 1 2 B1 x + B2 x + B3 , (26) C.1. Proof of Proposition 6 The polynomials P1 (x0 ) and P2 (x0 ) are given by P1 (x0 ) = 21 (A1 − C1 )x02 + A2 x0 + A3 , P2 (x0 ) = 21 (B1 − C1 )x02 + B2 x0 + B3 , 370 G. Martín-Herrán et al. / Automatica 44 (2008) 361 – 370 where r2 > 0.  C1 = Some tedious but easy computations allow us to establish that A1 − C1 < 0, C1 − B1 > 0. Therefore, from the study of the second-order polynomials P1 (x0 ) and P2 (x0 ), the following results are derived: P1 (x0 )0 ⇔ x0  x̂0 , (29) where x̂0 = −A2 −  A22 − 2(A1 − C1 )A3 A1 − C1 > 0, P2 (x0 ) 0 ⇔ x0 > x̃0 , (30) where x̃0 = B2 +  B22 + 2(C1 − B1 )B3 C1 − B 1 > 0. It can be proved that the coefficients of the value functions compare as follows: A1 − B1 > 0, A2 − B2 > 0, A3 − B3 > 0. These inequalities allow us to establish that x̂0 > x̃0 . From the above conditions, together with (29) and (30) and the definition of a Nash equilibrium, the different results in Proposition 6 follow. References Anderson, S., & de Palma, A. (2005). A theory of information overload. Mimeo. Bitran, G., & Mondschein, S. (1996). Mailing decisions in the catalog sales industry. Management Science, 42(9), 1364–1381. Bult, J. R., & Wansbeek, T. (1995). Optimal selection for direct marketing. Marketing Science, 14(4), 378–394. Chen, Y., & Sudhir, K. (2004). When shopbots meet Emails: Implications for price competition on the Internet. Quantitative Marketing and Economics, 2(3), 233–255. Engwerda, J. C. (2005). LQ optimization and differential games. New York: Wiley. Gönül, F., & Ze Shi, M. (1998). Optimal mailing of catalogs: A new methodology using estimable structural dynamic programming models. Management Science, 44(9), 1249–1262. Industry Canada (2005). http://e-com.ic.gc.ca/epic/internet/inecic-ceac.nsf/ en/h_gv00170e.htmlv , consulted in April 2005. Jacoby, J., Speller, D. E., & Berning, C. K. (1974). Brand choice behavior as a function of information load-replication and extension. Journal of Consumer Research, 1, 33–41. Keller, K. L., & Staelin, R. (1987). Effects of quality and quantity of information on decision effectiveness. Journal of Consumer Research, 14, 200–213. Lee, B. K., & Lee, W. N. (2004). The effect of information overload on consumer choice quality in an on-line environment. Psychology and Marketing, 21(3), 159–162. Lurie, N. (2004). Decision making in information-rich environments: The role of information structure. Journal of Consumer Research, 30(4), 473–486. View publication stats MacFadden, D. (2001). The tragedy of the commons. Forbes ASAP, 168, 61–62. Mahajan, V., & Venkatesh, R. (2000). Marketing modeling of e-business. International Journal of Research in Marketing, 17, 215–225. Malhorta, N. K. (1982). Information load and consumer decision making. Journal of Consumer Research, 8, 419–430. Martin, B., Van Durme, J., Raulas, M., & Merisavo, M. (2003). Email advertising: Exploratory insights from Finland. Journal of Advertising Research, 43(3), 293–306. Piersma, N., & Jonker, J. (2004). Determining the optimal direct mailing frequency. European Journal of Operational Research, 158, 173–182. Shiman, D. (1996). When Emails becomes junk mail: The welfare implications of the advancement of communications technology. Review of Industrial Organization, 11(1), 35–48. Shiman, D. (1997). The impact of firms’ increased information about consumers on the volume and targeting of direct marketing. Available at SSRN: http://ssrn.com/abstract = 555646 . Simon, H. (1997). Designing organizations for an information-rich world: Models of bounded rationality. Cambridge, MA: MIT Press. Sipior, J., Ward, B., & Bonner, G. (2004). Should spam be on the menu?. Communications of the ACM, 47(6), 59–63. Van Zandt, T. (2004). Information overload in a network of targeted communication. Rand Journal of Economics, 35(3), 542–560. Guiomar Martín-Herrán received an M.S. degree in Mathematics and a Ph.D. degree in Economics, both from the University of Valladolid, Spain. She is an associate professor in the Applied Economics Department of the University of Valladolid. Her main research areas include applied optimal control and differential games, in which she has published about 35 papers in scientific journals, refereed conference proceedings and book chapters. Her research is regularly funded by the Ministry of Education and Science of Spain. Since 2002 she is a frequently visiting professor at GERAD, HEC Montréal. She is currently a member of the executive committee of the International Society of Dynamic Games. Olivier Rubel is finishing his Ph.D. in marketing at HEC Montréal. He holds a D.E.A. in management science from Université ParisDauphine, a bachelor in economics and management from École Normale Supérieure de Cachan and an agrégation in economics and management from the French Ministry of Education. His research interests include differential games, optimal control and operations research applied to marketing and economics. He has published in Annals of Dynamic Games and has presented in numerous international conferences in marketing science and operations research. Georges Zaccour holds a Ph.D. in management science, an M.Sc. in international business from HEC Montréal and a licence in mathematics and economics from Université Paris-Dauphine. He is a full professor of Marketing at HEC Montréal. He served as a director of GERAD, an interuniversity research center and director of marketing department and Ph.D. program at HEC Montréal. His research areas include differential games, optimal control and operations research applied to marketing, energy sector and environmental management, in which he has published more than 85 papers and co-edited 12 volumes. He coauthors the book Differential Games in Marketing. His research is regularly funded by the Natural Sciences and Engineering Research Council of Canada. He is an associate editor of the International Game Theory Review, Environmental Modeling and Assessment and Computational Management Science. He is a fellow of The Royal Society of Canada and was currently the president of the International Society of Dynamic Games (2002–2006).