Arbitrage-based recovery

Journal of Financial Economics 163 (2025) 103969
Contents lists available at ScienceDirect
Journal of Financial Economics

journal homepage: www.elsevier.com/locate/finec
Arbitrage-based recovery✩
Ferenc Horvath ∗
University of Liverpool, United Kingdom
ARTICLE INFO ABSTRACT
Dataset link: MATLAB code (Original data) We develop a novel recovery theorem based on no-arbitrage principles. To implement our Arbitrage-Based
Recovery Theorem empirically, one needs to observe the Arrow–Debreu prices only for one single maturity.
JEL classification:
We perform several different density tests and mean prediction tests using more than 26 years of S&P 500
G1
G12
options data, and we find evidence that our method can correctly recover the probability distribution of the
G13 S&P 500 index return on a monthly horizon, despite the presence of a non-trivial permanent SDF component.
G17
Keywords:
Recovery theorem
Physical probabilities
Stochastic discount factor
No-arbitrage
1. Introduction that asset prices, per se, depend only on risk-neutral probabilities,
but not on physical probabilities. Therefore, extracting information
Extracting information about the physical probabilities of future about physical probabilities from asset pricing data is not possible
events from market data has been the focus of finance research for without imposing further assumptions. In a recent seminal paper, a
several decades. A huge variety of applications in financial economics set of such assumptions was stated and the corresponding physical
relies on knowledge of physical probabilities, such as portfolio choice, probability recovery theory developed by Ross (2015). In a finite-state
risk management, and asset pricing, just to name a few. Market practice framework Ross demonstrates that if we are able to observe the entire
usually hinges upon estimating physical probabilities from histori- Arrow–Debreu price matrix and the stochastic discount factor (SDF)
cal data. Nevertheless, estimates are only as reliable as good of a is transition independent,2 then the physical probability measure can
representative the historical data are.1 Therefore, extracting physical
be recovered. Since in reality we can observe only one row of the
probabilities from real-time market data in a forward-looking manner
Arrow–Debreu price matrix (and not the entire matrix), Ross proposes
(as opposed to estimating them from backward-looking historical data)
an approach where observing the transition state prices for as many
would be of paramount importance.
maturities as the number of possible states, the entire Arrow–Debreu
However, since the seminal works of Black and Scholes (1973)
and Merton (1973) (and even earlier, of Bachelier, 1900), we know price matrix can be reconstructed.3
✩ Nikolai Roussanov was the editor for this article. We thank Zsolt Bihary, Jens C. Jackwerth, Christian S. Jensen, Yongjin Kim, Péter Kondor, Yaxuan Qi,
Alexander Szimayer, Jianfeng Xu, and Steven Utke for useful comments.
∗ Correspondence to: University of Liverpool, Management School, Chatham Street, Liverpool, L69 7ZH, United Kingdom.
E-mail address: [email protected].
1
Already Markowitz (1952) warns about basing one’s investment decision on expected returns and (co)variances which have been estimated from historical
data: ‘‘... (W)e must have procedures for finding reasonable 𝜇𝑖 and 𝜎𝑖𝑗 . These procedures (...) should combine statistical techniques and the judgment of practical men.
(...) (T)he statistical computations should be used to arrive at a tentative set of 𝜇𝑖 and 𝜎𝑖𝑗 . Judgment should then be used in increasing or decreasing some of these
𝜇𝑖 and 𝜎𝑖𝑗 on the basis of factors or nuances not taken into account by the formal computations.’’
2
The stochastic discount factor is transition independent if it can be written as the product of a constant and a fraction, the numerator of which is a positive
scalar-valued function of the state variable evaluated at the arrival state, and the denominator of which is the same function evaluated at the initial state. I.e., a
( ) ( )
transition-independent SDF has the functional form 𝑚𝑖,𝑗 = 𝛿 × ℎ 𝜃𝑗 ∕ℎ 𝜃𝑖 , where 𝑖 is the initial state, 𝑗 is the arrival state, 𝛿 ∈ R>0 , 𝜃 is the state variable, and
ℎ is a positive scalar-valued function.
3
As Jensen et al. (2019) point out, the recovered Arrow–Debreu matrix is almost surely unique. I.e., the set of parameters for which there exists a continuum
of recovered Arrow–Debreu matrices (instead of a unique recovered Arrow–Debreu matrix) has a measure of zero.
https://doi.org/10.1016/j.jfineco.2024.103969
Received 16 July 2022; Received in revised form 25 October 2024; Accepted 28 October 2024
Available online 8 November 2024
0304-405X/© 2024 The Author. Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
F. Horvath Journal of Financial Economics 163 (2025) 103969
Ross’s recovery theorem has initiated a lively academic debate function (with a fixed number of parameters) of the state variable.
both in the theoretical and in the empirical finance literature. Sev- Recovery then means recovering the parameter values. Then, after
eral years before the emergence of the recovery theorem, Alvarez calculating the SDF values, recovering the physical probabilities is
and Jermann (2005) and Hansen and Scheinkman (2009) showed straightforward. In this structured version of the Generalized Recovery
that the stochastic discount factor process can be decomposed into framework, one needs to observe the transition state prices for at least
the product of a transition-independent factor (the transitory compo- as many maturities as the number of unknown parameters.
nent ) and a martingale (the permanent component ). Building on this An unattractive requirement of each of the currently available
factorization, Borovička et al. (2016) point out that Ross’s approach recovery theorems is that one needs to observe the transition state
recovers the true physical probabilities if and only if the martingale prices for several different maturities in order to recover the physical
component of the SDF process is constant. This implicit assumption of a probabilities for one single maturity. Even in the simplest case when the
constant martingale component is, however, incompatible with several only state variable is the asset price, one needs to observe the transition
mainstream asset pricing models, for example, with the long-run risk state prices for at least as many different maturities as many possible
model of Bansal and Yaron (2004). Using options on the 30-year values the asset price can take. For example, if we are interested in the
Treasury bond futures, Bakshi et al. (2018) confirm empirically that physical probabilities over a one-month horizon and we can observe
the martingale component is unlikely to be constant. Qin et al. (2018) the monthly transition state prices for up to one year of maturity,
arrive at the same conclusion, using U.S. Treasury data. Dillschneider then (assuming non-overlapping observation periods) the asset price
and Maurer (2019), after extending the recovery theorem to contin- is allowed to take only twelve different values. As Ross (2015) notes,
uous state spaces, use S&P 500 index options to argue that Ross’s such a coarse grid leads to poorly discretized transition state prices
recovery seems to be misspecified. Tran and Xia (2018) highlight the and eventually poorly discretized recovered probabilities. Audrino et al.
importance of the Arrow–Debreu price matrix dimension when imple- (2019) use overlapping observation periods and transition state prices
menting Ross’s recovery theorem. Martin and Ross (2019) apply the of 1∕10 of a month, but without additional restrictions, the recovered
idea of the recovery theorem to study the properties of the spot yield one-month transition state prices still exhibit counterintuitive features,
curve. Schneider and Trojani (2019) recover the conditional minimum as it is demonstrated by Jackwerth and Menner (2020). An adapted
variance projection of the pricing kernel on tradeable realized moments version of the Generalized Recovery approach of Jensen et al. (2019)
of market returns. Pazarbasi et al. (2021) amend the recovery frame- suggests a way out: if the SDF can be written as a known function (with
work with investor heterogeneity. In an early paper, Carr and Yu (2012) a small number of unknown parameters) of the state variables, then
extend Ross’s recovery to a bounded diffusion context, and Walden
it is enough to observe the transition state prices for only as many
(2017) derives necessary and sufficient conditions of when recovery
different maturities as the number of parameters (which is, ideally,
is possible with unbounded diffusion processes. A (non-exhaustive)
much lower than the number of possible values of the state variable).
list of further studies on recovery includes Dubynskiy and Goldstein
There is, however, a trade-off: the modeler has to exogenously provide
(2013), Huang and Shaliastovich (2014), Liu (2014), Massacci et al.
the functional form of the SDF. The validity of the recovered physical
(2016), Park (2016), Qin and Linetsky (2016), Ghosh and Roussellet
probabilities will then also depend on the validity of the assumed SDF
(2020), Jensen (2021), and Heston (2021).
functional form. Furthermore, one still needs to observe the transition
Jackwerth and Menner (2020) perform a thorough empirical anal-
state prices for several different maturities, if the SDF has more than
ysis to assess whether the physical probabilities recovered by Ross’s
one parameter.
Recovery Theorem and other approaches are indeed equal to the
When recovery theorems are empirically implemented in the lit-
true physical probabilities. Using monthly observations of more than
erature, the price of a traded asset is usually assumed to constitute
30 years of S&P 500 European-style options, they recover the one-
the state variable. The goal is then to extract a ‘‘flexible’’ form of
month physical probability distributions of the S&P 500 index level.
the SDF in the sense that the SDF is restricted only by the transition
Then, based on several different density tests and mean- and variance
independence assumption. In this paper, we show that such a flexibility
prediction tests, Jackwerth and Menner reject the hypothesis that
is actually spurious, and the recovered SDF cannot be anything else
realized S&P 500 index values are drawn from the recovered physical
probability distributions. but the reciprocal of the gross asset return. This result follows from an
The empirical analysis of Jackwerth and Menner (2020) also points additional no-arbitrage restriction ignored by the existing literature.
out that applying Ross’s recovery theorem to reconstruct the entire First, we develop our Arbitrage-Based Recovery approach when
Arrow–Debreu price matrix can lead to a recovered transition state the state space (spanned by the price of a traded asset) is discrete.
price matrix with highly counterintuitive features. E.g., even though In such a framework, the pricing operator is represented by a posi-
one would expect high recovered state prices on the main diagonal tive square matrix (the Arrow–Debreu price matrix), and the existence
of the Arrow–Debreu price matrix, the recovered Arrow–Debreu price and uniqueness of our recovered probability measure relies on the
matrix of Jackwerth and Menner tends to have high state prices in Perron–Frobenius theorem. Then, we show that the implications of
states which are far off-diagonal. Furthermore, the model-implied risk- Arbitrage-Based Recovery also hold when the state space is continuous
free rates are often negative or their magnitude is as high as several and the state variable (the price) can take any real value in a closed
hundred percent. and bounded interval. Since in such a framework the pricing operator
Besides assuming a constant SDF martingale component, empirical is an integral operator (instead of a square matrix) and one cannot rely
implementations of Ross’s recovery theorem also assume that transition on the Perron–Frobenius theorem, we prove existence and uniqueness
state prices are time-homogeneous. Jensen et al. (2019) develop the of our recovered probability measure directly. Finally, we show that
Generalized Recovery Theorem, where they relax this assumption. They our results still hold and Arbitrage-Based Recovery can also be applied
show that as long as we are able to observe the transition state prices when the state variable is the price–dividend ratio instead of the price
for at least as many maturities as the number of possible states, physical itself.
probabilities (corresponding to the assumed SDF) can almost surely be Besides contributing to the literature on recovery theorems, our
recovered, without assuming time-homogeneity. paper also contributes to the emerging literature on the factorization
Jensen et al. (2019) also demonstrate when (and how) Generalized of the SDF into a transitory and a permanent component. Namely, we
Recovery can be implemented even if the number of possible states show that as long as the state variable is the price of a traded asset
grows over time. Standard examples for such a case include the models and the state space is bounded and closed, under a new numeraire the
of Mehra and Prescott (1985) and of Cox et al. (1979). Basically, one transitory SDF component is equal to the inverse of the realized gross
needs to make use of the fact that the SDF can be expressed as a return on that asset. Then, we show that the same result holds when
2
the state variable is the price–dividend ratio (instead of the price itself) paid cash flow in future states, because options are almost always
under another new numeraire. written on post-cash-flow prices, e.g., stock options are written on
Our results on the SDF factorization lead us directly to the next post-dividend stock prices.
contribution of our paper. Namely, we add to the recent literature As an empirical contribution, we implement our Arbitrage-Based
on estimating the expected return on the market in a forward-looking Recovery Theorem using more than 26 years of S&P 500 stock market
manner, based on options data. Martin (2017) shows that as long as index options data. We test the hypothesis that the realized monthly
the ‘‘negative correlation condition’’ is satisfied, a lower bound for the returns of the S&P 500 market index are indeed drawn from the
expected return on the market can be derived in real time, based on probability distributions recovered by our Arbitrage-Based Recovery
option prices. Then, Martin demonstrates empirically that the lower Theorem. We find that this hypothesis cannot be rejected when using
our recovery approach, while it is confidently rejected when Jackwerth
bound appears to be tight in the sense that the correlation in question
and Menner (2020) test it using other recovery approaches. We also
seems to be close to zero. Our paper contributes to these results in two
demonstrate that this result does not contradict the findings of the
aspects. First, we show that Martin’s negative correlation condition is
empirical literature on the importance of the permanent SDF compo-
actually a condition on the martingale component of the SDF under
nent. We show that the restrictive assumptions imposed by empirical
a new numeraire when a bounded and closed price process spans implementations of recovery theorems (e.g., that a closed and bounded
the state space. Namely, it imposes that the martingale component of price process spans the space) imply an SDF which is very unlikely to
the SDF under the so-called dividend account numeraire is negatively be equivalent to the true transitory SDF component. Hence, the fact
correlated with the total market return under the dollar numeraire.4 that the permanent SDF component is very volatile empirically (as we
Second, we show that the probabilities recovered by the Arbitrage- confirm in Section 8.1) does not contradict the empirical success of our
Based Recovery Theorem imply the same expected return on the market Arbitrage-Based Recovery Theorem.
as Martin’s lower bound. The paper is organized as follows. In Section 2, we develop our
Ross (2015) and other recovery theorems do not specify the state Arbitrage-Based Recovery Theorem in a discrete-state-space frame-
variable. The state space is allowed to be spanned by any variable, as work. In Section 3, we discuss several alternative interpretations of
long as options are traded on that state variable. The vast majority of the probabilities recovered by Arbitrage-Based Recovery. In Section 4,
the empirical literature on recovery, however, focuses on state spaces we connect Arbitrage-Based Recovery to the long-term factorization
spanned by prices. The reason is pragmatic: most (liquid) options are of Alvarez and Jermann (2005) and Hansen and Scheinkman (2009).
written on prices (e.g., on the S&P 500 index level).5 The fact that the In Section 5, we extend our model into a continuous-state-space frame-
state variable is a price itself, however, can be used to our advantage, work, while in Section 6 we show how our theory can be implemented
and this aspect of the empirical implementation of recovery theorems if the state variable is the price–dividend ratio (instead of the price
has been so far largely overlooked by the literature. In this paper, itself). In Section 7, we implement our Arbitrage-Based Recovery The-
orem empirically using more than 26 years of S&P 500 index options
we demonstrate that – after an appropriate change of numeraire –
data. We test the hypothesis that the realized monthly returns of the
the transitory component of the SDF can be directly inferred from
S&P 500 market index are indeed drawn from the probability distri-
a theoretical recursive no-arbitrage restriction which the asset price
butions recovered by our Arbitrage-Based Recovery Theorem, and we
spanning the underlying state must satisfy. In a recent paper, Zhu
also perform several robustness checks. In Section 8, we demonstrate
(2020) formulates a critique of Ross’s Recovery Theorem based on its how the empirical success of our recovery approach can be reconciled
ignoring the Fundamental Theorem of Asset Pricing when developing with the literature’s findings that the permanent SDF component is
the Recovery Theorem itself. However, when the state variable is not a empirically very volatile. Section 9 concludes. In Appendix A, we
price, the only no-arbitrage requirement is that the Arrow–Debreu price provide methodological details on how we attach left and right tails
matrix be non-negative – which in itself is assumed by Ross’s Recovery. to the risk-neutral probability density functions when we implement
The Fundamental Theorem of Asset Pricing does not necessitate impos- our Arbitrage-Based Recovery empirically. In the accompanying Online
ing any additional assumptions in general. The additional connection Appendix, we give a brief overview of the currently available recovery
between Ross’s Recovery Theorem and the Fundamental Theorem of theorems, focusing on how Arbitrage-Based Recovery offers itself as the
Asset Pricing proposed by Zhu (2020) exists only in the special case next logical step in the evolutionary pathway of recovery theorems; we
when the state variable is the price of a traded asset and the asset does provide a detailed treatment of specific dividend payment structures;
not pay any intermediate cash flow. In that case, the Perron–Frobenius and we also provide there the proofs which are not included in the
eigenvector is indeed equal to the vector of possible prices and the paper.
corresponding eigenvalue is equal to one, as argued by Zhu (2020).
2. Arbitrage-based recovery
Still, Ross’s Recovery Theorem remains valid and it is not inconsistent
with the Fundamental Theorem of Asset Pricing; rather, the latter
Since the original Recovery Theorem was developed by Ross (2015),
pins down the unique solution of Ross’s recovery problem. However,
a sequence of alternative recovery approaches has emerged. In the
when the underlying asset potentially pays any intermediate cash flow Online Appendix, we provide a detailed comparison of the different
(e.g., dividend), as it is the case for stocks or the S&P 500 stock recovery approaches and the exact assumptions they pose.
market index, this implication does not hold any more under the dollar In this section, we develop a novel recovery theorem which does
numeraire. The numeraire has to be changed in order for it to reflect not require the observation of Arrow–Debreu prices for several different
the effects of the intermediate cash flow. Importantly, the recovered maturities in order to recover the probabilities over a single period. The
stochastic discount factor will be transition independent only under key element of our approach is a change of numeraire, which enables us
the new numeraire (and not the dollar), and the recovery exercise can to link the solution of the recovery (eigenvalue–eigenvector) problem
be interpreted as the long-term factorization of the SDF only under to an additional no-arbitrage criterion. Under the new numeraire, we
the new numeraire. A pragmatic approach to handle intermediate cash retain Ross’s assumption of a transition-independent stochastic discount
flows might be to use the cash-flow-reinvestment account value as the factor. By applying our version of the recovery theorem, we need to
state variable, but this still requires making an assumption about the observe the Arrow–Debreu prices only for the current state as initial
state and only for one single maturity, and from those observations we
can recover the physical transition probabilities corresponding to that
4
We provide the relevant definitions and we discuss these results in details maturity. Our model can be applied to recover the physical probabil-
in Section 3.4. ities of the price movement of any asset, as long as the state space is
5
Examples of options the underlying variable of which is not a price spanned by the price of that asset. A schematic demonstration of our
include, among others, interest rate options and weather options. recovery theory can be found in Fig. 1.
3
Fig. 1. Arbitrage-based recovery. One needs to observe the Arrow–Debreu prices (where the current state is the initial state) for only one maturity. If the SDF under the dividend
( ) ( )
account numeraire takes the form 𝛿̂ × ℎ̂ 𝑆𝑗 ∕ℎ̂ 𝑆𝑖 , where 𝛿̂ is a positive constant, 𝑆 is the asset price and the only state variable, and ℎ̂ is a positive function of 𝑆, then in the
absence of arbitrage opportunities the physical transition probabilities can be recovered.
Fig. 2. Timing in our model. At time 𝑡, we observe the stock price in the current state 𝑖. Then, at time 𝑡 + 1 in state 𝑗 the dividend 𝐷𝑖,𝑗 is paid. Right afterwards, we observe the
stock price 𝑆𝑗 .
D
2.1. Framework one period in state 𝑗 (and nothing in other states), 𝐴𝑖,𝑗 is the current
price in dollars (in state 𝑖) of receiving 1 × 𝐹𝑖,𝑗 dollars in one period
Consider a discrete-time complete financial market. For the sake of in state 𝑗 (and nothing in other states). Alternatively, we can interpret
D
concreteness, we consider here the market for a stock, but our approach 𝐴𝑖,𝑗 as the current price in units of 𝐷 (in state 𝑖) of receiving one unit
can readily be applied for other long-term financial assets as well. The D
only state variable in our model is the stock price, which can take 𝑛 of 𝐷 in one period in state 𝑗 (and nothing in other states).7 Hence, 𝑨
possible values, collected in the 𝑛 × 1 column vector 𝑺 ∈ R𝑛>0 . The 𝑛 × 𝑛 is the Arrow–Debreu price matrix under the 𝐷-numeraire.
matrix of one-period Arrow–Debreu prices is 𝑨$ ∈ R𝑛×𝑛 . The dollar The stochastic discount factor (SDF) under the dollar numeraire,
>0
sign in the superscript emphasizes that the Arrow–Debreu price matrix 𝒎$ ∈ R𝑛×𝑛
>0
, is the (unique) 𝑛 × 𝑛 matrix such that
corresponds to the dollar numeraire. The stock pays dividend in one 𝑨$ = 𝒎$ ⊙ 𝝅 (2)
period, which is allowed to be stochastic. Denoting by 𝐷𝑖,𝑗 ∈ R≥0 the
dollar amount of dividend paid in one period if we go from state 𝑖 to holds, where 𝝅 ∈ R𝑛×𝑛
is the physical transition probability matrix.
>0
state 𝑗, we introduce the 𝑛 × 𝑛 gross dividend yield matrix 𝑭 ∈ R𝑛×𝑛 ≥1
Similarly, the SDF under the 𝐷-numeraire is the unique 𝑛 × 𝑛 matrix
with elements D
𝒎 ∈ R𝑛×𝑛
>0
such that
𝐷𝑖,𝑗 + 𝑆𝑗 D D
𝐹𝑖,𝑗 = , (1) 𝑨 =𝒎 ⊙𝝅 (3)
𝑆𝑗
D
where 𝑆𝑗 is the stock price in state 𝑗, observed just after the dividend holds.8 Substituting 𝑨 = 𝑨$ ⊙ 𝑭 in (3) and then dividing both sides
has been paid. The timeline in Fig. 2 demonstrates the timing in our (element-wise) by 𝝅, we obtain that the relationship between the dollar
model. In our theoretical discussion, we do not restrict the form of SDF and the 𝐷-SDF is
the dividend yield matrix 𝑭 apart from requiring dividends to be non- D
𝒎 = 𝒎$ ⊙ 𝑭 . (4)
negative (and therefore the elements of 𝑭 to be greater than or equal
to one).6
Now, we introduce the concept of a dividend account. A dividend Following Ross (2015), we now introduce the definition of a
account has an initial (time-zero) value of $1. Then, after one period, transition-independent stochastic discount factor.
the account is increased by multiplying its value by the realized gross
dividend yield. So, its value after one period will be $1 × 𝐹𝑖,𝑗 if we Definition 1. Consider a discrete-time finite-state complete-market
go from state 𝑖 to state 𝑗. Using the value of the dividend account as model where the only state variable is the post-dividend stock price 𝑆.
numeraire, we can define a ‘‘currency’’ (denoted by 𝐷) whose value is The stochastic discount factor 𝒎$ is said to be transition independent
always equal to the dividend account value. if it can be expressed as
D ( )
Next, consider the Hadamard (element-wise) product 𝑨 ≜ 𝑨$ ⊙ 𝑭 . ℎ 𝑆𝑗
$
Since 𝐴𝑖,𝑗 is the current price in dollars (in state 𝑖) of receiving $1 in 𝑚$𝑖,𝑗 = 𝛿 × ( ) , (5)
ℎ 𝑆𝑖
7
6
In the Online Appendix, we discuss several specific forms of the gross To see this, recall that at time zero 𝐷1 = $1.
8
dividend yield matrix 𝑭 in details. In our empirical analysis in Section 7, we Note that the physical transition probability matrix 𝝅 in (3) is the same
assume that the dividend is known one period ahead. as in (2).
4
where 𝑖 is the initial state, 𝑗 is the arrival state, 𝛿 is a positive scalar, Hence, interestingly, we find that the no-arbitrage condition (9)
and ℎ ∶ R>0 → R>0 is a positive scalar-valued function. is actually a condition on the Perron–Frobenius eigenvector and the
D D
Similarly, the stochastic discount factor 𝒎 is said to be transition Perron–Frobenius eigenvalue of 𝑨 , i.e., the Arrow–Debreu price ma-
independent if it can be expressed as trix under the 𝐷-numeraire. We formalize this in the following lemma.
( )
D ℎ̂ 𝑆𝑗
̂
𝑚𝑖,𝑗 = 𝛿 × ( ) , (6)
ℎ̂ 𝑆𝑖 Lemma 1. Consider a positive square Arrow–Debreu price matrix under
D
the D-numeraire, 𝑨 ∈ R𝑛×𝑛 >0
, in a discrete-time finite-state complete-market
where 𝛿̂ is a positive scalar, and ℎ̂ ∶ R>0 → R>0 is a positive scalar- model. The only state variable is the post-dividend stock price in dollars,
valued function. the possible values of which are collected in the vector 𝑺 ∈ R𝑛>0 . If there is
D
no arbitrage opportunity, then the Perron–Frobenius eigenvector of 𝑨 is 𝑺
2.2. No-arbitrage and recovered physical probabilities
and its Perron–Frobenius eigenvalue is 1.
The purpose of our recovery theory – just as it is the purpose of
D
any other recovery theory – is to decompose the observed Arrow– Proof. The fact that the Perron–Frobenius eigenvector of 𝑨 is 𝑺 and
Debreu prices into the SDF and the physical probabilities. The first its Perron–Frobenius eigenvalue is 1 readily follows from Eq. (9) and
building block of our approach, which we formalize in the following the Perron–Frobenius theorem. ■
theorem, establishes the uniqueness of the recovered physical transition Now, we are ready to state our main theorem, in which we provide
probabilities. Since the theorem holds under any numeraire, we do not our recovered physical transition probabilities.
specify the numeraire of the Arrow–Debreu price matrix.
Theorem 2 (Arbitrage-Based Recovery Theorem). Consider a discrete-time
Theorem 1. Consider a discrete-time finite-state complete-market model finite-state complete-market model with a positive square Arrow–Debreu
with a positive square Arrow–Debreu price matrix 𝑨 ∈ R𝑛×𝑛 >0
. There exists price matrix under the dollar numeraire, 𝑨$ ∈ R𝑛×𝑛 , and a gross dividend
>0
a unique decomposition of the matrix 𝑨 such that 𝑨 = 𝜙𝒁 𝜫 𝒁 −1 , where yield matrix 𝑭 ∈ R𝑛×𝑛 . The state variable is the post-dividend stock price,
≥1
𝒁 ∈ R𝑛×𝑛≥0
is a diagonal matrix with positive diagonal elements, 𝜫 ∈ R𝑛×𝑛>0 the possible values of which are collected in the vector 𝑺 ∈ R𝑛>0 . The
is a stochastic matrix, 𝜙 ∈ R>0 is a positive scalar, and the uniqueness of stochastic discount factor under the D-numeraire is assumed to be transition
𝒁 is to be understood as ‘‘unique up to a positive scale factor’’. independent. If there is no arbitrage opportunity, then the elements of the
physical transition probability matrix 𝜫 ∈ R𝑛×𝑛 >0
are
$
𝑆𝑗 × 𝐹𝑖,𝑗
Proof. The proof follows the logic of the proof of Result 1 in Martin and 𝛱𝑖,𝑗 = 𝐴𝑖,𝑗 × (10)
Ross (2019). First, we prove the existence of 𝒁, 𝜫, and 𝜙. According 𝑆𝑖
to the Perron–Frobenius theorem, there exist a positive vector 𝒛 ∈ R𝑛>0 for all 𝑖 ∈ {1, … , 𝑛} and 𝑗 ∈ {1, … , 𝑛}.
and a positive scalar 𝜙 ∈ R>0 such that 𝑨𝒛 = 𝜙𝒛. Define 𝜫 ≜ 𝜙1 𝒁 −1 𝑨𝒁,
where 𝒁 is a diagonal matrix with the elements of 𝒛 in its diagonal. Proof. According to Theorem 1, there exist a unique positive scalar
Since 𝜫 is positive and its elements in each of its rows sum to one, it 𝜙, a unique diagonal matrix 𝒁 with positive diagonal elements, and
is a stochastic matrix. As one can check, 𝑨 = 𝜙𝒁 𝜫 𝒁 −1 holds. Thus, D
a unique stochastic matrix 𝜫 such that 𝑨 = 𝜙𝒁 𝜫 𝒁 −1 . Rearranging
there indeed exist 𝜙, 𝒁, and 𝜫 satisfying the appropriate claims in the 1 −1 D
this, we obtain 𝜫 = 𝜙 𝒁 𝑨 𝒁. According to Lemma 1, 𝜙 = 1 and
theorem.
𝑍𝑖,𝑖 = 𝑆𝑖 . Substituting these values into the rearranged equation and
Now, we prove the uniqueness of 𝒁, 𝜫, and 𝜙. Multiplying both D
using the identity 𝑨 = 𝑨$ ⊙ 𝑭 , we find 𝛱𝑖,𝑗 = 𝐴$𝑖,𝑗 × 𝐹𝑖,𝑗 × 𝑆𝑗 ∕𝑆𝑖 . This
sides of 𝑨 = 𝜙𝒁 𝜫 𝒁 −1 by 𝒁 𝜾 from the right (where 𝜾 is the 𝑛 × 1
completes the proof. ■
column vector of ones), we obtain 𝑨𝒛 = 𝜙𝒛. According to the Perron–
Frobenius theorem, 𝜙 is unique and 𝒛 is unique up to a positive scale The result of Theorem 2 offers itself as the next logical step in the
factor. Hence, 𝒁 is also unique up to a positive scale factor. And if we evolutionary path of recovery theorems. From a practical perspective,
rearrange 𝑨 = 𝜙𝒁 𝜫 𝒁 −1 as 𝜫 = 𝜙1 𝒁 −1 𝑨𝒁, we find that therefore 𝜫 is according to the theorem, if there is no arbitrage opportunity and the
also unique. ■ SDF under the 𝐷-numeraire is transition independent, we can uniquely
recover the physical transition probabilities in the state space spanned
Now, we impose our additional no-arbitrage condition. If there is no
by the (post-dividend) asset price.
arbitrage opportunity, then the time-𝑡 stock price must be equal to the
sum of each Arrow–Debreu prices multiplied by the time-(𝑡 + 1) cash
2.3. Inferred stochastic discount factor
flows in the respective states. Since the stock cash flow at time 𝑡 + 1 is
the sum of the dividend and the post-dividend stock price, this means
Similarly to recovering the physical transition probabilities, we
that
can also infer the unique transition-independent stochastic discount
∑𝑛
( ) D
𝑆𝑖 = 𝐴$𝑖,𝑗 × 𝐹𝑖,𝑗 × 𝑆𝑗 ∀𝑖 ∈ {1, 2, … , 𝑛} (7) factor under the 𝐷-numeraire, 𝒎 . We formalize this in the following
𝑗=1 corollary.
must hold, if there is no arbitrage opportunity. Note that 𝑆 is always
the stock price which we can observe in the stock market after dividend Corollary 1. Consider a discrete-time finite-state complete-market model
payment : on the left-hand side 𝑆𝑖 is the stock price which we observe at with a positive square Arrow–Debreu price matrix under the dollar nu-
time 𝑡, and 𝑆𝑗 on the right-hand side is the stock price which we observe meraire, 𝑨$ ∈ R𝑛×𝑛 >0
, and a gross dividend yield matrix 𝑭 ∈ R𝑛×𝑛 ≥1
. The
at time 𝑡 + 1, right after we have received the dividend. Expressing (7) state variable is the post-dividend stock price, the possible values of which
in matrix notation (and switching the left-hand side and the right-hand are collected in the vector 𝑺 ∈ R𝑛>0 . The stochastic discount factor under the
side), we find that D-numeraire is assumed to be transition independent. If there is no arbitrage
( ) opportunity, then the stochastic discount factor under the D-numeraire is
𝑨$ ⊙ 𝑭 𝑺 = 𝑺 (8) D 𝑆
𝑚𝑖,𝑗 = 𝑖 . (11)
D 𝑆𝑗
must hold, if there is no arbitrage opportunity. And since 𝑨 = 𝑨$ ⊙ 𝑭
by definition, this no-arbitrage condition can be equivalently written Furthermore, the stochastic discount factor under the dollar numeraire is
as 𝑆𝑖
𝑚$𝑖,𝑗 = . (12)
D 𝐹𝑖,𝑗 × 𝑆𝑗
𝑨 𝑺 = 𝑺. (9)
5
Proof. The inferred SDFs follow readily from the definition of the variable is the price of a traded asset (e.g., the S&P 500 index level)
D
Arrow–Debreu price, the relationship 𝑨 = 𝑨$ ⊙ 𝑭 , and our recovered and the dividend (or other similar cash flow) is for sure non-negative,
probabilities in (10). ■ then the Perron–Frobenius eigenvalue of the Arrow–Debreu (dollar)
price matrix is for sure lower than or equal to one. Furthermore, if
3. Alternative interpretations and further discussion the dividend is strictly positive in at least one initial-state-arrival-state
combination, then the Perron–Frobenius eigenvalue (and therefore the
Although the focus of the current paper is the canonical problem subjective discount factor of the pseudo-representative agent) is strictly
of recovering the physical probabilities from observable asset prices, lower than one.10 We formalize this in the following theorem.
the theory of Arbitrage-Based Recovery is also related to several other
branches of the literature. In this section, we discuss these connec- Theorem 3. Consider a discrete-time finite-state complete-market model
tions and highlight several alternative interpretations of our recovered with a positive square Arrow–Debreu price matrix under the dollar nu-
probabilities. meraire, 𝑨$ ∈ R𝑛×𝑛 >0
, and a gross dividend yield matrix 𝑭 ∈ R𝑛×𝑛 ≥1
. The
state variable is the post-dividend stock price, the possible values of which
3.1. Log-utility agent are collected in the vector 𝑺 ∈ R𝑛>0 . If there is no arbitrage opportunity,
then the Perron–Frobenius eigenvalue of the matrix 𝑨$ is less than or equal
The form of our inferred dollar SDF in (12) lends itself for an al- to one. Furthermore, if at least one element of the gross dividend yield
ternative economic interpretation. Namely, it can be interpreted as the matrix 𝑭 is strictly greater than one, then the Perron–Frobenius eigenvalue
stochastic discount factor implied by a pseudo-representative investor of the matrix 𝑨$ is strictly less than one.
who has a subjective discount rate of zero and log-utility over the
realized total cash flow, and who invests her wealth fully in the stock.9
Proof. First, consider the case when the underlying asset does not
Consequently, this pseudo-representative agent makes her investment
pay any dividend (i.e., when each element of the gross dividend yield
decision based on our recovered physical probabilities, (10). The next D
matrix 𝑭 is equal to one). Then, by definition, 𝑨 = 𝑨$ . Hence, due to
corollary formalizes this.
Lemma 1, the Perron–Frobenius eigenvalue of the matrix 𝑨$ is equal
to one.
Corollary 2. Consider a log-utility agent within a single-period finite-state
Now, consider the case when at least one element of 𝑭 is strictly
complete-market arbitrage-free model, where the states are indexed by the D D
greater than one. Then 𝑨$ ≤ 𝑨 and 𝑨 ≠ 𝑨$ . Due to Corollary 6.16
possible values of the stock, expressed in dollars. She allocates her wealth at
in Zhan (2013), this implies that the Perron–Frobenius eigenvalue of
𝑡 = 0 so that her expected utility from the dollar value of her wealth at 𝑡 = 1 D
𝑨$ is strictly less than the Perron–Frobenius eigenvalue of 𝑨 . Since
is maximized. If she invests her wealth fully in the stock, then the probability D
according to Lemma 1 the Perron–Frobenius eigenvalue of 𝑨 is equal
measure used by this agent is the same as the probability measure recovered
to one, the Perron–Frobenius eigenvalue of 𝑨$ must then be strictly less
by the Arbitrage-Based Recovery Theorem.
than one. This completes the proof. ■
Proof. We provide the proof in the Online Appendix. ■
3.4. Negative correlation condition of Martin (2017)
3.2. Discounting by the return on the SÐ -asset
In a model-free framework, Martin (2017) shows that the condi-
Another alternative interpretation of our recovered probabilities is tional expected excess return on the market can be decomposed into
that they are used by an agent who discounts everything by the return two terms. Concretely,
on an asset whose next-period payoff (in state 𝑗) will be 𝑆𝑗 units of 𝐷. ( ) ( ) ( )
1
We formalize this in the next corollary. E𝑡 𝑅$𝑡+1 − 𝑅$𝑓 ,𝑡+1 = 𝑣𝑎𝑟∗𝑡 𝑅$𝑡+1 − 𝑐 𝑜𝑣𝑡 𝑀𝑡+1
$
𝑅$𝑡+1 , 𝑅$𝑡+1 , (13)
$
𝑅𝑓 ,𝑡+1
Corollary 3. Consider an asset (the SD -asset) whose next-period payoff where 𝑅$𝑡+1 and 𝑅$𝑓 ,𝑡+1 are the market return and the risk-free rate over
in state 𝑗 is 𝑆𝑗 units of D. The probabilities recovered by the Arbitrage- the period from 𝑡 to 𝑡+ 1, the SDF corresponding to the same time period
Based Recovery Theorem correspond to the probabilities used by an agent $
is 𝑀𝑡+1 , the asterisk means that the variance is calculated under the
who discounts the next-period cash flow by the return on the SD -asset. risk-neutral probability measure, the 𝑡 subscript denotes conditioning
𝐷 on the information available at time 𝑡, and the dollar sign in the super-
Proof. The time-𝑡 price in state 𝑖 of the 𝑆 -asset is 𝑆𝑖 units of 𝐷, due script emphasizes that the numeraire is the dollar (and not the dividend
to the no-arbitrage condition (9). Thus, the one-period gross return on account). The strength of the decomposition in (13) – besides its model-
𝐷
the 𝑆 -asset under the 𝐷-numeraire is 𝑆𝑗 ∕𝑆𝑖 . Comparing this to the free nature – lies in the fact that the first term on the right-hand side can
recovered probabilities in (10), the result follows immediately. ■ be observed directly in the market in real time (via option prices), while
the sign of the second term can be ‘‘controlled’’ by economic reasoning.
3.3. Positivity of Ross’s subjective discount rate
As Martin (2017) argues, in all mainstream asset pricing models the
sign of the covariance in (13) is negative. As a consequence, the risk-
In Ross’s original recovery theorem, the Perron–Frobenius eigen-
neutral variance of the market return scaled by the risk-free rate serves
value is often interpreted as the subjective discount factor of a pseudo-
as a lower bound for the expected excess market return. Then, Martin
representative agent. The Perron–Frobenius theorem assures that this
(2017) shows that empirically the magnitude of the covariance term
subjective discount factor is positive. Economic intuition further sug-
is small, thus the lower bound is a good approximation for the true
gests that its value should be lower than one, in accordance with the
expected excess market return.
concept of time value of money. Based on the logic of our Arbitrage-
Our Arbitrage-Based Recovery helps understand the essence of Mar-
Based Recovery Theorem, now we prove rigorously that if the state
tin’s negative correlation condition. Concretely, we can show that the
negative correlation condition is actually a condition on the correlation
9
But, importantly, we do not assume any form of preference or even between the martingale component of the 𝐷-SDF (in a framework
D
the existence of a representative agent. We only assume that the SDF 𝒎
is transition independent. The particular functional form of our inferred SDF
(both under the dollar and the 𝐷-numeraire) is then a direct consequence of 10
This is a special case of a more general, recent result by Borovička and
the no-arbitrage condition. Stachurski (2021). We thank the referee for pointing this out.
6
where the state is spanned by a closed, bounded price process) and call this asset the 𝑇 -period zero-coupon D-bond. For example, if the initial
the market return. To see this, note that the true SDF under the state at 𝑡 = 0 is 𝑖 and in 𝑇 periods we arrive at state 𝑚 by taking the path
𝐷-numeraire can be written as (𝑖 → 𝑗 → 𝑘 → ... → 𝑙 → 𝑚), then the 𝑇 -period zero-coupon 𝐷-bond will
𝐷 𝑆𝑡 𝐷
pay 𝐹𝑖,𝑗 × 𝐹𝑗 ,𝑘 × ⋯ × 𝐹𝑙,𝑚 dollars, which is equivalent to one unit of the
𝑀𝑡+1 = × 𝐻𝑡+1 , (14)
𝑆𝑡+1 dividend account then. Now, we show that the recovered probabilities
𝐷 in (10) indeed correspond to the long-forward probabilities under the
where 𝐻𝑡+1 is a positive random variable with expected value of one,
𝐷-numeraire.
also known as the martingale component of the 𝐷-SDF. Dividing both
sides by the gross dividend yield 𝐹𝑡+1 , we obtain
𝐷 Corollary 4. The probabilities recovered by the Arbitrage-Based Re-
𝑀𝑡+1 𝑆𝑡 𝐷
covery Theorem correspond to the long-forward probabilities under the
= × 𝐻𝑡+1 , (15)
𝐹𝑡+1 𝑆𝑡+1 × 𝐹𝑡+1 D-numeraire.
which can be equivalently written as
𝐷
$
𝑀𝑡+1 × 𝑅$𝑡+1 = 𝐻𝑡+1 . (16) Proof. If we go from state 𝑖 to state 𝑗, the one-period gross return on
the 𝐷 long bond is
Hence, Martin’s covariance term can be written as [( ) ]
( ) ( ) ∑𝑛 D 𝑇 −1
𝐷 𝐴
$
𝑐 𝑜𝑣𝑡 𝑀𝑡+1 𝑅$𝑡+1 , 𝑅$𝑡+1 = 𝑐 𝑜𝑣𝑡 𝐻𝑡+1 , 𝑅$𝑡+1 . (17) 𝐷
𝑘=1
𝑗 ,𝑘
𝑅∞,𝑖,𝑗 = lim [( ) ] , (18)
𝑇 →∞ ∑𝑛 D 𝑇
The negative correlation condition is thus, in essence, equivalent to 𝑘=1 𝐴
the condition that the martingale component of the SDF (under the 𝑖,𝑘
dividend account numeraire) is negatively correlated with the total where we raised the one-period Arrow–Debreu price matrix 𝑨 to the
D
market return (under the dollar numeraire).
power of 𝑇 and 𝑇 − 1, to obtain the 𝑇 - and the (𝑇 − 1)-period Arrow–
Since Arbitrage-Based Recovery assumes that the permanent compo-
Debreu price matrices. According to the Perron–Frobenius theorem, we
nent of the 𝐷-SDF is constant, its implied probabilities correspond to
have
an expected return which is exactly equal to the lower bound of Martin ( D )𝑇
(2017). Hence, Arbitrage-Based Recovery can also be considered as lim 𝑨 = 𝑺 𝒘′ , (19)
𝑇 →∞
an extension of Martin’s approach: while Martin (2017) recovers the
D
first moment of the stock market return from option prices, Arbitrage- where 𝒘 is the left Perron–Frobenius eigenvector of 𝑨 . Hence, the
Based Recovery extracts a consistent probability distribution of the return in (18) can be written as
∑
stock market return from the same option prices. And just as Martin’s 𝐷 𝑆𝑗 𝑛𝑘=1 𝑤𝑘 𝑆𝑗
recovered expected returns are hard to reject empirically, in Section 7 𝑅∞,𝑖,𝑗 = ∑𝑛 = . (20)
𝑆𝑖 𝑘=1 𝑤𝑘 𝑆𝑖
we similarly find that the probability distributions implied by our
D
Arbitrage-Based Recovery Theorem are difficult to reject empirically as Using the identity 𝑨 = 𝑨$ ⊙ 𝑭 , we can express the recovered
well. probabilities in (10) as
D 𝑆𝑗
4. Long-term factorization 𝛱𝑖,𝑗 = 𝐴𝑖,𝑗 × . (21)
𝑆𝑖
𝐷
Ross’s Recovery Theorem is closely related to the branch of the Substituting 𝑅∞,𝑖,𝑗 in place of 𝑆𝑗 ∕𝑆𝑖 in (21) and then dividing both
literature initiated by Alvarez and Jermann (2005) which concerns 𝐷
the long-term factorization of the stochastic discount factor and the sides by 𝑅∞,𝑖,𝑗 , we readily see that the recovered probabilities indeed
corresponding long-forward probability measure. As Alvarez and Jer- correspond to the long-forward probabilities under the 𝐷-numeraire.
mann (2005) and Hansen and Scheinkman (2009) show, the stochastic This completes the proof. ■
discount factor can be decomposed into the product of a transitory
component and a permanent component. The transitory SDF component,
in turn, is equal to the reciprocal of the return on the long bond, i.e., on a 5. Arbitrage-based recovery in continuous space
zero-coupon bond the maturity of which is in the infinity.11 Since Ross’s
Recovery Theorem actually recovers the transitory SDF component (as So far, we have assumed that the only state variable in our model
shown by Borovička et al., 2016), the SDF recovered by Ross is also
is the stock price, and that it can take only finitely many values.
equal to the reciprocal of the return on the long bond. In this section,
Consequently, our pricing operator has been represented by a square
we show that this interpretation carries over to our Arbitrage-Based
matrix of positive Arrow–Debreu prices, and our recovery approach
Recovery approach, but under the dividend numeraire 𝐷 (instead of
has relied on the uniqueness of the positive eigenvector (and the
the dollar numeraire). In other words, our inferred SDF is equal to the
corresponding positive eigenvalue) of this matrix.
reciprocal of the return on a zero-coupon bond denominated in units
of 𝐷 and with maturity in the infinity. In the remainder of this section, Although one might argue that stock prices are quoted using dis-
we demonstrate this in more details. crete price increments (‘‘ticks’’), there is no economic rationale behind
If we take the dividend account as the numeraire, the equivalent assuming that prices can take only certain (discrete) values. The good
of a zero-coupon bond with maturity in 𝑇 periods is a security which news is that our framework can readily be extended to accommodate
will pay one unit of the dividend account in each state in 𝑇 periods. We a continuous state space represented by the price of a traded asset, as
we now show in this section.
We still consider a discrete-time financial market where the only
11
The essence of this result predates the emergence of the literature on state variable is the stock price, but this price can now take any real
[ ]
recovery theorems and on the SDF decomposition and was first pointed out value in the interval 𝑆, 𝑆 where 0 < 𝑆 < 𝑆. There exists a linear
by Kazemi (1992). Importantly, the result itself does not rely on either Marko-
vianity of the model or the Perron–Frobenius Theorem. Further discussion can positive operator (the pricing operator)
[ A]$ mapping the set of bounded
be found in Alvarez and Jermann (2005) and Martin and Ross (2019). Qin and real-valued functions with domain 𝑆, 𝑆 into the same set. The pricing
Linetsky (2017) generalize the result to semimartingale settings. operator can be represented as
7
[ ] [ ]
( ) 𝑆 where 𝐴$ (𝑥, 𝑦) ∶ 𝑋, 𝑋 × 𝑋, 𝑋 → R>0 is the dollar state-price
A $ 𝑓 (𝑥) = 𝐴$ (𝑥, 𝑦) 𝑓 (𝑦) 𝑑 𝑦, (22) density if we go from
∫𝑆 [ state
] 𝑥[to state
] 𝑦. There exists a dividend growth
[ ] [ ] function ℎ (𝑥, 𝑦) ∶ 𝑋, 𝑋 × 𝑋, 𝑋 → R>0 representing the dividend
where 𝐴$ (𝑥, 𝑦) ∶ 𝑆, 𝑆 × 𝑆, 𝑆 → R>0 is the dollar state-price density growth 𝐷𝑡+1 ∕𝐷𝑡 if we go from state 𝑥 (at time 𝑡) to state 𝑦 (at time
going from state 𝑆𝑡 = 𝑥 to 𝑆𝑡+1 = 𝑦. Similarly to the gross dividend 𝑡 + 1).
yield matrix 𝑭 defined in (1) in the discrete-state
[ ]case,
[ now
] there Similarly to previous sections, also now we propose a change of nu-
exists a gross dividend yield function 𝐹 (𝑥, 𝑦) ∶ 𝑆, 𝑆 × 𝑆, 𝑆 → R≥1 meraire. Since our state variable is the post-dividend stock price scaled
representing the gross dividend yield if we go from state 𝑆𝑡 = 𝑥 to by the dividend itself (i.e., 𝑋𝑡 = 𝑆𝑡 ∕𝐷𝑡 ), we can define a corresponding
𝑆𝑡+1 = 𝑦. The state-price density under the 𝐷-numeraire is defined as state price density 𝐴$ (𝑥, 𝑦) × ℎ (𝑥, 𝑦) which also uses the dividend as
𝐷
𝐴 (𝑥, 𝑦) = 𝐴$ (𝑥, 𝑦) × 𝐹 (𝑥, 𝑦), which, in turn, defines the pricing oper- numeraire. While 𝐴$ (𝑥, 𝑦) expresses the rate of exchange between one
𝐷 dollar today and one dollar in a future state, 𝐴$ (𝑥, 𝑦) × ℎ (𝑥, 𝑦) is the
ator under the 𝐷-numeraire, A , accordingly. If there is no arbitrage
rate of exchange between one unit of dividend today and one unit
opportunity in the market, then
of dividend in a future state. We can also go one step further and
𝐷 𝐺
A 𝑓 (𝑥) = 𝑓 (𝑥) (23) define the state price density 𝐴 (𝑥, 𝑦) ≜ 𝐴$ (𝑥, 𝑦) × ℎ (𝑥, 𝑦) × 𝐹 (𝑦), where
must hold for 𝑓 (𝑥) = 𝑥. I.e., absence of arbitrage implies that 𝑓 (𝑥) = 𝐹 (𝑦) = 1 + 1∕𝑦 corresponds to the 𝑭 matrix in our discrete-state-space
𝐷 framework in Section 2 and to the 𝐹 function in our continuous-state-
𝑥 is an eigenfunction of the operator A with the corresponding
eigenvalue being equal to one. space framework in Section 5, evaluated at the arrival state 𝑦. Since the
To show that our results developed in a discrete state space also hold state variable now is the price–dividend ratio itself (as opposed to the
when the state space is continuous (i.e., when the stock price can take earlier sections, where the state variable was the price), 𝐹 is a function
[ ]
of the arrival state only, and it is independent of the departure state.
any real value in the interval 𝑆, 𝑆 ), it suffices to show that 𝑓 (𝑥) = 𝑥 𝐺
The state price density 𝐴 (𝑥, 𝑦) expresses the rate of exchange between
is the unique (up to a multiplicative constant) positive eigenfunction of
𝐷 one unit of dividend today and 𝐹 units of dividend in a future state. The
the operator A . We formalize this in the following theorem. 𝐺
pricing operator under the 𝐺-numeraire, A , is defined accordingly as
𝐷 [ 𝐺 ] 𝑋
Theorem 4. The pricing operator A has a unique (up to a multiplicative A 𝑓 (𝑥) =
𝐺
𝐴 (𝑥, 𝑦) 𝑓 (𝑦) d𝑦. (25)
constant) positive eigenfunction, namely, 𝑓 (𝑥) = 𝑥. Furthermore, the ∫𝑋
eigenvalue corresponding to this eigenfunction is equal to one.
Recovery under the 𝐺-numeraire entails solving the eigenvalue–
eigenfunction problem
Proof. We provide the proof in the Online Appendix. ■ [ 𝐺 ]
[ ] A 𝑔 (𝑥) = 𝛾 𝑔 (𝑥) (26)
When the stock price is not restricted to the interval 𝑆, 𝑆 , but
it can take any positive real value, the recovery problem can still for 𝑔 (𝑥) > 0 and 𝛾 > 0. Note that if there is no arbitrage opportunity,
𝐷
be formulated. Then, however, the pricing operator A might have then
several different positive eigenfunctions, in which case unique recovery 𝑋
is not possible. Whether there exists a unique (up to a multiplicative 𝑥= 𝐴$ (𝑥, 𝑦) ℎ (𝑥, 𝑦) 𝐹 (𝑦) 𝑦 d𝑦 (27)
∫𝑋
constant) positive eigenfunction, depends on the exact properties of the
𝐷
pricing operator A , besides its linearity and positivity. Importantly, must hold, which can equivalently be expressed as
even if the recovery problem has several distinct solutions, the solution 𝑋
𝐺
proposed by our Arbitrage-Based Recovery approach will always be 𝑥= 𝐴 (𝑥, 𝑦) 𝑦 d𝑦. (28)
∫𝑋
among the solutions. Furthermore, if the recovery problem has a unique
solution, then our approach will provide this unique solution. Therefore, absence of arbitrage implies that 𝑔 (𝑥) = 𝑥 and 𝛾 = 1 is
a solution of the eigenfunction–eigenvalue problem (26). Using the
6. Price–dividend ratio as the state variable same reasoning as in Theorem 4, it can be shown that this is the
only solution. The corresponding conditional probability density of the
In our frameworks so far, we have assumed that the only state price–dividend ratio is
variable is the price of a traded asset, and the state space is bounded. 𝐺 𝑦
𝜋 (𝑥, 𝑦) = 𝐴 (𝑥, 𝑦) × , (29)
The assumption that prices can take values only in a given interval is, 𝑥
however, not supported empirically. Assuming instead that the price– where 𝑥 is the price–dividend ratio at the departure state and 𝑦 is the
dividend ratio can take values in an interval is more plausible. The good price–dividend ratio at the arrival state. This density, however, can
news is that the principles of our Arbitrage-Based Recovery approach equivalently be expressed as
can also be applied in an environment where the state variable is the
price–dividend ratio (instead of the price itself). Furthermore, our re- 𝜋 (𝑥, 𝑦) = 𝐴$ (𝑥, 𝑦) × 𝑅 (𝑥, 𝑦) , (30)
( ) ( )
covered probability distribution of the price–dividend ratio is consistent where 𝑅 𝑋𝑡 , 𝑋𝑡+1 = 𝐷𝑡+1 + 𝑃𝑡+1 ∕𝑃𝑡 is the gross total return on the
with our previously recovered probability distribution of the price. In security itself, if we go from state 𝑋𝑡 to state 𝑋𝑡+1 . This recovered
this section, we show how Arbitrage-Based Recovery can be adapted probability measure is therefore consistent with the implications of our
to a framework with the price–dividend ratio as the state variable. We Theorem 2 (and, of course, our results in Section 5).
only show the continuous-state-space case; the discrete-state-space case If we relax the assumption [ that] the price–dividend ratio can take
can be treated analogously.
values only in the interval 𝑋, 𝑋 and allow it to take any positive
Consider a discrete-time economy where the only state variable is
real value, the recovery problem does not necessarily have a unique
the price–dividend
[ ]ratio, 𝑋 ∈ R>0 , which can take any real value in solution. Whether a unique solution exists depends on the exact prop-
the interval 𝑋, 𝑋 where 0 < 𝑋 < 𝑋. There exists a linear positive 𝐺
erties of the pricing operator A , besides its linearity and positivity.
operator (the pricing operator)
[ ]mapping the set of bounded real-valued However, if the solution is unique, then it will be equal to our solution,
functions with domain 𝑋, 𝑋 into the same set. This operator can be otherwise there would be an arbitrage opportunity in the market.
represented as When multiple solutions exist and we assume that the price–dividend
( ) 𝑋 ratio is stationary and ergodic under the true physical probability
A $ 𝑓 (𝑥) = 𝐴$ (𝑥, 𝑦) 𝑓 (𝑦) d𝑦, (24) measure, there is at most one solution under which the price–dividend
∫𝑋
8
ratio remains stationary and ergodic, as shown by Borovička et al. Yahoo Finance for each observation date and for each option exercise
(2016). Whether there is such a solution (which retains stationarity and date.
ergodicity) and whether it corresponds to our solution depends, again, Altogether, we have 317 observation dates, one in each month of
𝐺
on the exact properties of the operator A . our sample period. On each observation date, we determine the 28-day
continuously-compounded risk-free interest rate by linearly interpolat-
7. Empirical tests of arbitrage-based recovery ing between the two closest available maturities.15 Furthermore, using
the S&P 500 price index and the S&P 500 total return index data, we
From a theoretical perspective, our Arbitrage-Based Recovery Theo- calculate the dividend received during each 28-day period.
rem has several appealing properties. Not only does it require observing
only one row of the Arrow–Debreu price matrix for only one single
7.2. Risk-neutral probabilities
maturity, but it also explicitly excludes arbitrage opportunities.
Nonetheless, the proof of the pudding is in the eating, and the
defense of a theory resides in its empirical validity. Jackwerth and On each observation date, we first obtain the risk-neutral proba-
Menner (2020) systematically test to what extent the different ver- bility density function (pdf) characterizing the risk-neutral probability
sions of recovery theorems are supported empirically. They test three distribution of the S&P 500 index value 28 days later. To this end, we
different recovery theorem versions (Ross Original, Ross Basic, Ross first transform each option price into its implied volatility, using the
Stable/Generalized Recovery), with and without imposing economic Black–Scholes option pricing formula.16 As standard in the literature,
restrictions on the Arrow–Debreu price matrix. Based on 32 years we use only out-of-money (OTM) options. To handle the ‘‘jump’’ of
of S&P 500 options data (from April 1986 to December 2017) and the implied volatility function at the moneyness level of one, we
using several different density tests (the Berkowitz test, two forms adapt an approach similar to Figlewski (2010) to smooth the implied
of the Knüppel test, and the Kolmogorov–Smirnov test), they find volatilities around the current S&P 500 index level. Then, applying a
strong evidence that the one-month S&P 500 returns do not follow slightly modified version of the Fast and Stable Method of Jackwerth
the distribution predicted by the recovery theorems. These findings (2004),17 we fit a smooth curve on the ‘‘observed’’ implied volatilities
are also supported by mean-prediction and variance-prediction tests. for each observation day, using a fine grid of 𝛥 = $0.10 along the
We test our Arbitrage-Based Recovery Theorem following an approach strike price dimension between the lowest and the highest traded strike
similar to Jackwerth and Menner (2020), using more than 26 years of prices. For demonstration purposes, in Fig. 3, we depict our implied
S&P 500 options data. Our results are summarized in Table 5. volatility curves (together with the ‘‘observed’’ implied volatilities) for
Before describing our empirical exercise in details, we would like three observation dates: August 20, 2004; November 21, 2008; and
to point out a duality regarding how our empirical results can be April 16, 2020.
interpreted. Having access to a time series of the one-month-ahead Transforming back each implied volatility along the grid into a
S&P 500 index level state price density, we apply our methodology to European call option price and then taking the (numeric) second deriva-
recover the probability distribution of the index level in one month. As tive of the option price with respect to the strike price, we obtain the
our theoretical results in Sections 5 and 6 show, this empirical exercise state-price density for each observation day. For details on why the
can be motivated and approached from two different perspectives. second derivative of a European call option price with respect to the
Concretely, either we assume that the state variable is the index level strike price is equal to the state-price density, we refer to Ross (1976)
itself or we assume that the state variable is the price–dividend ratio. and Breeden and Litzenberger (1978). Dividing the state-price densities
Since the dividend is assumed to be known one period in advance by the risk-free discount factors, we obtain the risk-neutral densities.
(as we will explain later), the recovered probability densities will be
Finally, to add tails to the risk-neutral density functions below the
exactly the same, only the random variable (i.e., the index level or
lowest and above the highest traded strike prices, we follow Figlewski
the price–dividend ratio) is scaled by a constant. Similarly, the results
(2010) and choose appropriately parameterized generalized extreme
of our empirical tests will be exactly the same, regardless of whether
value (GEV) distributions. We provide more details on our methodology
the index level or the price–dividend ratio is the state variable. For
of adding tails to the risk-neutral density curves in Appendix A. In
brevity, in this section we will refer to our exercise as one attempting
Fig. 4, we plot the risk-neutral probability density functions (as dotted
to recover the probability distribution of the index level itself, keeping
curves) for the same three dates as we used in Fig. 3 for the implied
in mind that the exercise and its results can also be interpreted as one
volatilities. The square markers indicate the lowest and the highest
attempting to recover the price–dividend ratio probability distribution.
traded strike prices for each date.
7.1. Data
7.3. Recovering the physical probabilities
We collect prices of European-style options written on the S&P 500
index from January 19, 1996 to May 19, 2022. The options expire at The Arrow–Debreu price is the product of the physical probability
the market opening of the third Friday12 of each month, and we observe and the stochastic discount factor, and our ultimate goal is to separately
their bid and ask prices 29 days before their expiry.13 Our source of
option price data from January 19, 1996 to December 23, 2021 is
OptionMetrics, and for 2022 observations it is Bloomberg. We discard 15
For observations between 1996 and 2021, our spot yield curve data is
all options with a bid price lower than $0.50, and then we calculate from OptionMetrics. For 2022, we use the 4-week Treasury Bill secondary
the mid option prices. We also collect the S&P 500 price index values market rate, provided by the St. Louis Fed.
from OptionMetrics14 and the S&P 500 total return index values from 16
Importantly, we do not assume that the assumptions of the Black–Scholes
model hold. We use the Black–Scholes formula only as a tool which provides
a one-to-one mapping between option prices and implied volatilities.
12 17
Until February 15, 2015, our observed options expire at the market When calculating the numerical derivatives of the fitted volatility
opening of the Saturday immediately following the third Friday of the month. curve, Jackwerth (2004) assumes that the volatility corresponding to an
13 exercise price just one grid step outside the boundary is equal to the implied
Due to the options expiring at the market opening, this corresponds to a
horizon of 28 days. volatility on the boundary. To allow for larger flexibility, our boundary
14
For 2022, our S&P 500 price index value data is obtained from Yahoo condition instead assumes that the second derivatives on the boundaries are
Finance. zero.
9
Fig. 3. Implied volatility curves on three dates, calculated from European-style S&P 500 index options with 28 days to maturity. The ‘‘*’’ markers indicate implied volatilities
corresponding to mid option prices, while the ‘‘⋅’’ markers denote implied volatilities corresponding to bid and ask option prices. Implied volatilities are calculated by the Black–
Scholes option pricing formula. The 28-day risk-free interest rates are obtained by linearly interpolating between the two risk-free rates with the closest maturities (e.g., 25 days
and 31 days). The dividend yield parameter of the Black–Scholes formula is calculated under the assumption that the dividend is received just before the option exercise date, and
the dollar amount of this dividend is known on the observation date. To interpolate between the observation points on the implied volatility curves, we use a slightly modified
version of the Fast and Stable Method of Jackwerth (2004).
identify these two components, based on our Arbitrage-Based Recovery level slightly higher than one, and it features a slight left skew. The
Theorem. In the previous step, we have determined the Arrow–Debreu two other dates correspond to crisis periods, with much higher implied
prices.18 But to apply our theorem, we still need to determine the volatilities. By 21 November, 2008, the severity of the global financial
gross dividend yield 𝐹𝑖,𝑗 . Since we want to recover the S&P 500 index crisis became evident, and the strong negative skewness of the density
level over a one-month horizon, we can reasonably assume that the function indicates that the market was prepared for further serious
dividend to be received in one month is deterministic and known at declines in the S&P 500 level. A high (more than 10% per month)
the time of observation.19 In Fig. 4, we show the physical probability increase in the index level had also much higher probability than on the
density functions recovered by our model on the three dates used in ‘‘relaxed’’ date of 20 August, 2004, which reflects the huge uncertainty
the previous subsection. For comparison purposes, we also show the perceived by the market. The situation was similar on 16 April, 2020,
risk-neutral densities as dotted curves. The first date, August 20, 2004, by when the seriousness of the COVID-19 pandemic on a global level
reflects relaxed market conditions, with implied volatilities between became apparent. Comparing the two crises dates, the market during
12 and 26 percent for the observed options. The recovered physical the pandemic attributed a much lower probability to a large positive
probability density function looks ‘‘standard’’: it is centered around a monthly jump in the S&P 500 index level, which is reflected in the
difference between the right tails of the two ‘‘crisis’’ distributions. The
effect of risk adjustment is much more apparent on the two crisis dates,
18
We can readily calculate the Arrow–Debreu prices from the state price
reflected in the observable differences between the risk-neutral and
densities by integrating the state price density function between any two strike the physical density functions. Risk adjustment has a more substantial
prices (or moneyness levels). effect in ‘‘very bad’’ and ‘‘very good’’ states (corresponding to low and
19
As Chetty et al. (2005) note, firms usually announce dividend payments high moneyness levels, respectively), and its effect is more negligible
about four to six weeks before the actual payment takes place. around a moneyness level of one. This is intuitive from an economic
10
Fig. 4. Risk-neutral and physical probability density functions on three dates, calculated from European-style S&P 500 index options with 28 days to maturity. The risk-neutral
probability density functions are shown as dotted curves, while the dash-dotted, the solid, and the dashed curves correspond to the physical probability density functions. The square
markers indicate the lowest and the highest traded strike prices for each date. Between these two strike prices, the risk-neutral densities are obtained as the second derivatives of
the call option price with respect to the strike price, divided by the risk-free discount factor. Call price curves are obtained as transforms of the implied volatility curves in Fig. 3,
and the 28-day risk-free discount factors are obtained by linearly interpolating between the two risk-free interest rates with the closest maturities (e.g., 25 days and 31 days).
Then, following Figlewski (2010), we add tails to the risk-neutral probability density functions by choosing appropriately parameterized generalized extreme value distributions.
Afterwards, we apply the Arbitrage-Based Recovery Theorem to transform these risk-neutral probability density functions into physical probability density functions. During this
transformation, we assume that the dividend is received just before the option exercise date, and the dollar amount of this dividend is known on the observation date.
Fig. 5. Recovered physical probability distributions of the 1-month non-annualized total returns on the S&P 500 market index, calculated from European-stye S&P 500 A.M.-settled
index options with 29 days to maturity. The dash-dotted lines denote returns lying between the 10th and the 90th percentiles, the boxes denote returns between the 25th and the
75th percentiles, and the solid lines inside the boxes correspond to returns between the 45th and the 55th percentiles. First, we obtain a smooth implied volatility curve for each
observation date by applying a slightly modified version of the Fast and Stable Method of Jackwerth (2004), which we then transform back into risk-neutral probability density
functions. We attach tails to these risk-neutral density functions by choosing appropriately parameterized generalized extreme value distributions, following Figlewski (2010). Then,
we transform the risk-neutral probability distributions into physical probability distributions by applying the Arbitrage-Based Recovery theorem. Throughout, we assume that the
dividend is received just before the option exercise date, and the dollar amount of the dividend is known on the observation date. The 28-day risk-free interest rates are obtained
by linearly interpolating between the two risk-free rates with the closest maturities (e.g., 25 days and 31 days) (until 2021). For 2022, we use the 4-week Treasury Bill secondary
market rate as the risk-free rate.
11
Fig. 6. Recovered physical moments of the 1-month non-annualized total return on the S&P 500 market index. We obtain the physical probability distributions by applying the
Arbitrage-Based Recovery Theorem, as shown in Fig. 5. Then, for each observation date, we calculate the expected value (mean), the standard deviation (volatility), the skewness,
and the kurtosis of the return. The recovered physical moments suggest that throughout our observation period, the probability distribution of the market return is left-skewed
and it features fat tails. Both its mean and its volatility are highly volatile. Furthermore, both the magnitude and the volatility of its skewness and kurtosis increased substantially
after the 2008 financial crisis. The mean and the volatility of the return are highly correlated, which is in accord with economic intuition that higher risk (volatility) should be
compensated with higher reward (mean).
Table 1 Table 2
Summary statistics of recovered physical moments of the 1-month non-annualized total Cross-correlations of recovered physical moments of the 1-month non-annualized total
return on the S&P 500 market index. From January 1996 to May 2022, we obtain the return on the S&P 500 market index. From January 1996 to May 2022, we obtain the
physical probability distributions with monthly frequency by applying the arbitrage- physical probability distributions with monthly frequency by applying the Arbitrage-
based recovery theorem, as shown in Fig. 5. Then, for each observation date, we Based Recovery Theorem, as shown in Fig. 5. Then, for each observation date, we
calculate the expected value (mean), the standard deviation (volatility), the skewness, calculate the expected value (mean), the standard deviation (volatility), the skewness,
and the kurtosis of the return. and the kurtosis of the return.
Median Expected Standard Minimum Maximum Mean Volatility Skewness Kurtosis
value deviation
Mean 1 – – –
Mean 0.42% 0.52% 0.43% 0.10% 4.20% Volatility 0.87 1 – –
Volatility 4.64% 5.01% 2.12% 2.38% 17.90% Skewness 0.45 0.45 1 –
Skewness −1.42 −1.53 0.69 −4.27 −0.34 Kurtosis −0.42 −0.44 −0.94 1
Kurtosis 8.03 10.14 6.29 3.37 40.78
The expected (total) return on the market index is on average 0.52%

perspective: in ‘‘very bad’’ states (corresponding to low moneyness per month (about 6% per year), and it is extremely volatile, ranging
levels) market participants’ marginal utility is high, hence they value from 0.10% per month during relaxed market conditions to 4.20% per
cash flows in those states highly, which corresponds to attributing month in October 2008. Furthermore, in untabulated results, we con-
much higher risk-neutral probabilities to those states than physical firm the finding of Martin (2017) that the distribution of the expected
probabilities. The opposite is true for ‘‘very good’’ states with high return itself is right skewed (its skewness is 4.21). The volatility of the
moneyness levels. These effects of the risk adjustment can be seen by (monthly) market return is also highly volatile, ranging from 2.38%
comparing the risk-neutral density functions (dotted curves) and the to 17.90%, with an average level of 5.01%. We find a strong positive
recovered physical density functions (dash-dotted, solid, and dashed correlation (Table 2) between recovered physical expected returns and
curves) in Fig. 4. volatilities, which is in line with economic intuition: a higher risk
In Fig. 5, we show the recovered physical probability distributions (higher volatility) must be compensated by a higher expected return.
of the 1-month non-annualized total returns on the S&P 500 market The recovered monthly return distributions exhibit negative skew-
index for each month between January 1996 and May 2022. The ness and positive kurtosis throughout our observation period, which
day of recovery is always 29 days before the Saturday immediately is in accordance with the empirical literature. The skewness and the
following the third Friday of the month (until February 2015) or 29 kurtosis are strongly negatively correlated, which suggests that fat
days before the third Friday of the month (after February 2015). Due tails tend to coincide with left skews. Interestingly, return distributions
to all options being AM-settled, this corresponds to a 28-day recovery which are more left skewed and which exhibit fatter tails tend to occur
period. The dash-dotted lines denote returns between the 10th and the during low-volatility and low-expected-return periods. This is in accord
90th percentiles, the boxes denote returns between the 25th and the with the recent findings of Gormsen and Jensen (2020). Furthermore,
75th percentiles, and the solid lines inside the boxes correspond to as Fig. 6 suggests, both the magnitude and the volatility of the return
returns between the 45th and the 55th percentiles. Return distributions skewness and kurtosis increased substantially after the 2008 financial
are in general left skewed with a median slightly above 0%, and they crisis.
clearly exhibit time-varying volatility. This is further confirmed in Fig. 7 plots the Sharpe ratio of the monthly non-annualized total
Fig. 6, where we plot the time series of the recovered physical moments; return on the S&P 500 index, calculated on each observation date under
and in Table 1, where we provide summary statistics of the recovered the physical probabilities implied by the Arbitrage-Based Recovery
physical moments. Theorem. First, for each observation date, we calculate the expected
12
Fig. 7. Sharpe ratio of the 1-month non-annualized total return on the S&P 500 market index. We obtain the physical probability distributions by applying the Arbitrage-Based
Recovery Theorem, as shown in Fig. 5. Then, for each observation date, we calculate the expected value and the standard deviation of the market return. The Sharpe ratio is then
obtained by subtracting the 1-month risk-free rate from the expected market return and dividing the difference by the volatility of the market return. The Sharpe ratio is volatile,
and its value is especially high at the beginning of crisis periods (October and November 2008, and March 2020).
Fig. 8. Moment risk premia of the 1-month non-annualized total return on the S&P 500 market index. We obtain the physical probability distributions by applying the Arbitrage-
Based Recovery Theorem, as shown in Fig. 5. Then, for each observation date, we calculate the expected value, the variance, the skewness, and the kurtosis of the return under
both the risk-neutral and the recovered physical probability measures. The moment premia are then obtained as the difference between the physical and the risk-neutral moments.
In accord with economic intuition, the equity premium is positive and the variance premium is negative throughout our observation period, and the skewness premium is almost
always positive while the kurtosis premium is almost always negative. The equity premium and the variance premium are strongly negatively correlated, just as the skewness
premium and the kurtosis premium. Similarly to the Sharpe ratio, the magnitudes of the equity premium and the variance premium are especially high at the beginning of crisis
periods (October and November 2008, and March 2020). Furthermore, similarly to the physical skewness and kurtosis, both the magnitude and the volatility of the skewness
premium and the kurtosis premium increased substantially after the 2008 financial crisis.
value and the standard deviation of the non-annualized market return S&P 500 market index. Fig. 8 shows the time series of the moment risk
under the recovered probability measure. The Sharpe ratio is then premia, while Tables 3 and 4 show their summary statistics and their
obtained by subtracting the 1-month non-annualized risk-free rate from cross-correlations, respectively. The signs of the moment risk premia
the expected market return and dividing the difference by the volatility are in accordance with economic intuition (see, e.g., Ebert, 2013): odd
of the market return. The Sharpe ratio is highly volatile, and its value moments have positive risk premia, and even moments have negative
is especially high at the beginning of crisis periods (October and risk premia.20 Each of the four moment risk premia features strong
time variation, their standard deviations being at least as high as the
November 2008, and March 2020).
We also calculate the risk premia of the first four moments (eq-
uity premium, variance premium, skewness premium, and kurtosis 20
On every single observation date, the equity premium is positive and the
premium), defined as the difference between the physical and the risk- variance premium is negative. The skewness premium is positive on more than
neutral moments of the 1-month non-annualized total returns on the 98% of our observation dates, while the kurtosis premium is negative on more
13
Table 3 (buy-and-hold) and the third (market-timing) strategies more directly

Summary statistics of moment risk premia of the 1-month non-annualized total return
comparable, we choose the constant of proportionality in the third
on the S&P 500 market index. From January 1996 to May 2022, we obtain the
physical probability distributions with monthly frequency by applying the arbitrage-
strategy so that the average (over the observation period) portfolio
based recovery theorem, as shown in Fig. 5. Then, for each observation date, we weight of the market index is equal to one. We re-evaluate the port-
calculate the expected value, the variance, the skewness, and the kurtosis of the folio weights monthly (on each observation date), and re-balance our
return under both the risk-neutral and the recovered physical probability measures. portfolio accordingly. The difference between the wealth paths of the
The moment premia are then obtained as the difference between the physical and the
market-timing strategy and the other two strategies can be attributed
risk-neutral moments.
to the effect of market timing based on the Arbitrage-Based Recovery
Median Expected Standard Minimum Maximum
value deviation Theorem. As we see in Fig. 9, the market-timing strategy outperforms
not only the rolled-over (locally) risk-free portfolio, but also the total
Equity premium 0.24% 0.34% 0.40% 0.07% 3.96%
Variance premium −0.02% −0.04% 0.07% −0.77% −0.002% return market index.21
Skewness premium 0.11 0.18 0.21 −0.05 2.90
Kurtosis premium −0.61 −1.55 3.60 −56.68 0.35 7.4. Density tests
Table 4 Now, we test whether realized S&P 500 index levels on the option
Cross-correlations of moment risk premia of the 1-month non-annualized total return exercise dates are indeed drawn from our model-implied distributions.
on the S&P 500 market index. From January 1996 to May 2022, we obtain the Following Jackwerth and Menner (2020), we perform several density
physical probability distributions with monthly frequency by applying the Arbitrage-
tests and mean-prediction tests. Our null hypothesis is that the S&P 500
Based Recovery Theorem, as shown in Fig. 5. Then, for each observation date, we
calculate the expected value, the variance, the skewness, and the kurtosis of the index levels observed on the option exercise dates are realizations of a
return under both the risk-neutral and the recovered physical probability measures. random variable with our model-implied probability density function.
The moment risk premia are then obtained as the difference between the physical and To perform our density tests, first we observe the S&P 500 index
the risk-neutral moments. level on each option exercise day.22 Then, we transform our model-
Equity Variance Skewness Kurtosis implied physical probability density functions into cumulative distri-
premium premium premium premium
bution functions (CDFs). For each option, we determine the CDF value
Equity premium 1 – – –
corresponding to the realized S&P 500 level. Under our null hypothesis,
Variance premium −0.96 1 – –
Skewness premium −0.10 0.00 1 –
these CDF values are drawn from a uniform distribution with support
Kurtosis premium 0.11 −0.02 −0.96 1 [0, 1], and they are independent of each other. To test this hypothesis,
we perform the Berkowitz test, two versions of the Knüppel test, and
the Kolmogorov–Smirnov test.
absolute value of their expected values. Similarly to the Sharpe ratio,

7.4.1. Berkowitz test
the magnitudes of the equity premium and the variance premium are
To perform the test of Berkowitz (2001), we first transform our
especially high at the beginning of crisis periods (October and Novem-
observed (‘‘realized’’) CDF values using the inverse of the cumulative
ber 2008, and March 2020). Furthermore, similarly to the physical
distribution function of the standard normal distribution. I.e., denoting
skewness and kurtosis, both the magnitude and the volatility of the
our CDF value observations by 𝑥𝑡 , we perform the transformation
skewness premium and the kurtosis premium increased substantially ( )
after the 2008 financial crisis. The equity premium and the variance 𝑧𝑡 = 𝛷−1 𝑥𝑡 , (31)
premium are strongly negatively correlated, as it is also confirmed
where 𝛷 denotes the CDF of a standard normally distributed random
empirically by Bollerslev et al. (2009). The skewness premium and
variable. Then, under the null hypothesis, our observed 𝑧𝑡 values follow
the kurtosis premium are also strongly negatively correlated, which
standard normal distributions and they are independent of each other.
is in agreement with economic intuition: when the market requires a
This implies that in the AR(1) model
high compensation for a stronger left skew, it tends to require a high ( )
compensation for fatter tails as well. 𝑧𝑡 − 𝜇 = 𝜌 𝑧𝑡−1 − 𝜇 + 𝜀𝑡 , (32)
To demonstrate a possible practical application of our Arbitrage- ( )
under the null hypothesis we have 𝜇 = 0, 𝜌 = 0, and 𝑉 𝑎𝑟 𝜀𝑡 = 1.
Based Recovery Theorem, we also perform a simple market-timing
Using our 𝑧𝑡 observations, we obtain the maximum likelihood estimates
exercise. Starting with $1 on January 19, 1996, we invest our wealth ( ) ( )
of 𝜇, 𝜌, and 𝑉 𝑎𝑟 𝜀𝑡 (denoted by 𝜇, ̂ and 𝑉̂
̂ 𝜌, 𝑎𝑟 𝜀𝑡 ), and perform a
according to three different investment strategies with monthly rebal-
likelihood ratio test where we compare the AR(1) model (32) with
ancing, and we plot the evolution of our wealth (on a logarithmic ( )
parameters 𝜇, ̂ and 𝑉̂
̂ 𝜌, 𝑎𝑟 𝜀𝑡 to the same model with parameters 𝜇 = 0,
scale) in Fig. 9. According to the first investment strategy (shown as ( )
𝜌 = 0, and 𝑉 𝑎𝑟 𝜀𝑡 = 1. We find that the 𝑝-value of this test is 0.340.
the blue dashed curve), the total wealth is invested in the one-month
risk-free asset and upon maturity it is always reinvested at the actual
one-month risk-free interest rate. The second strategy (shown as the 21
Our market-timing strategy invests in the stock market index heavily
red dotted curve) corresponds to a buy-and-hold investment policy, (taking highly leveraged positions) when the Sharpe ratio (implied by the
where the total wealth is invested in the S&P 500 total return index on Arbitrage-Based Recovery Theorem) is high, and less heavily (characterized by
January 19, 1996, and it is kept there indefinitely. The third investment a balanced mix of the stock market index and the risk-free asset) during periods
policy (shown as the black solid curve) is based on a market-timing with a low Sharpe ratio. Consequently, during low-Sharpe-ratio periods, our
strategy using the Arbitrage-Based Recovery Theorem. Concretely, we wealth path closely tracks the S&P 500 total return index. On the other hand,
invest in the one-month risk-free asset and in the S&P 500 total return during high-Sharpe-ratio periods, our wealth path exhibits higher volatility
index so that the portfolio weight of the S&P 500 total return index than the S&P 500 total return index, and these are the periods when the
is proportional to the Sharpe ratio of its one-month return, calcu- gap between the two wealth paths suddenly increases or decreases. This can
be seen by comparing Figs. 7 and 9: the gap between the two wealth paths
lated by the Arbitrage-Based Recovery Theorem. To make the second
increases around 2002, 2012, and 2020, and it decreases around 2008, all
of which periods are characterized by a high Sharpe ratio. Effectively, our
featured market-timing strategy attempts to identify and reap the benefits of
than 96% of our observation dates. In the few instances when the skewness high-Sharpe-ratio periods.
22
premium is negative and/or the kurtosis premium is positive, their magnitude More precisely, since the options are AM-settled, we observe the closing
is always very small. level of the S&P 500 index on the previous trading day.
14
Fig. 9. Wealth evolution under three different investment strategies, starting with $1 on January 19, 1996. On the vertical axis, we show the wealth on a logarithmic scale.
According to the first investment strategy (blue dashed curve), the total wealth is invested in the one-month risk-free asset and upon maturity it is always reinvested in the actual
one-month risk-free asset. The second strategy (red dotted curve) corresponds to a buy-and-hold investment policy, where the total wealth is invested in the S&P 500 total return
index on January 19, 1996, and it is kept there indefinitely. The third investment policy (black solid curve) is based on a market-timing strategy using Arbitrage-Based Recovery.
Concretely, we invest in the one-month risk-free asset and in the S&P 500 total return index so that the portfolio weight of the S&P 500 total return index is proportional to the
recovered Sharpe ratio of its one-month return. We choose the constant of proportionality such that the average (over the observation period) portfolio weight of the market index
is equal to one. We re-evaluate the portfolio weights monthly (on each observation date), and re-balance our portfolio accordingly. The difference between the wealth paths of
the market-timing strategy and the other two strategies can be attributed to the effect of market timing based on the Arbitrage-Based Recovery Theorem. (For interpretation of
the references to colour in this figure legend, the reader is referred to the web version of this article.)
Hence, at the standard significance levels, we cannot reject the null the Kolmogorov distribution. Performing the test, we find that the 𝑝-
hypothesis that the Arbitrage-Based Recovery Theorem recovers the value is 0.179. This, again, suggests that at the standard significance
true probability distributions. levels we cannot reject the null hypothesis that the Arbitrage-Based
Recovery Theorem recovers the true probability distributions.
7.4.2. Knüppel test
Next, we perform two versions of the test of Knüppel (2015). To this 7.5. Mean prediction tests
end, we transform our ‘‘realized’’ CDF values (denoted by 𝑥𝑡 ) according
to Besides carrying out density tests, we also test whether our model-
√ ( ) implied expected returns indeed predict the realized returns. Con-
𝑦𝑡 = 12 × 𝑥𝑡 − 0.5 . (33) cretely, we run the regression
Under[ the
√ null
√ ]hypothesis, 𝑦𝑡 follows a uniform distribution with sup- 𝑅𝑡 = 𝑎 + 𝑏𝜇𝑡 + 𝜖𝑡 , (35)
port − 3, 3 . We calculate the first three empirical raw moments
where 𝑅𝑡 is the realized return at time 𝑡 (not accounting for dividends),
of our observed 𝑦𝑡 values, and form the 3 × 1 column vector 𝑫 3 of
and 𝜇𝑡 is our model-implied expected return (again, without dividends).
differences between the empirical and theoretical moments. We also
Under the null hypothesis, the intercept is 𝑎 = 0 and the slope is
estimate the covariance matrix 𝜴3 of 𝑦𝑡 −𝑚1 , 𝑦2𝑡 −𝑚2 , and 𝑦3𝑡 −𝑚3 , where
𝑏 = 1. Assuming 𝑏 = 1 and testing whether the intercept is statistically
𝑚𝑖 denotes the 𝑖th theoretical raw moment. Under the null hypothesis,
significantly different from zero, we find a 𝑝-value of 0.2920. Assuming
the test statistic
a zero intercept and testing whether the slope is statistically signifi-
𝛼3 = 𝑇 × 𝑫 ′3 𝜴−1
3
𝑫3 (34) cantly different from one, our 𝑝-value is 0.2915. Finally, without any
restriction on our regression model, testing the joint hypothesis that
asymptotically follows a 𝜒2
distribution with three degrees of freedom. 𝑎 = 0 and 𝑏 = 1, we find a 𝑝-value of 0.5593. Thus, neither of the
Performing the test, we find that the 𝑝-value is 0.284. Hence, at the above three mean prediction tests rejects the null hypothesis that the
usual significance levels the Knüppel test with three moments does not expected returns implied by the Arbitrage-Based Recovery Theorem are
reject the null hypothesis that our recovery approach indeed recovers indeed the true expected returns.
the true probability distributions. Repeating the Knüppel test with the Our results confirm the findings of Martin (2017), who argues
first four moments (instead of the first three), we find that the 𝑝-value empirically that his negative correlation condition is tight, i.e., that it
is 0.080, which is just below the ten percent significance threshold. is close to zero. Our recovered probabilities correspond to the case
when Martin’s correlation is exactly zero. And just like Martin, we
7.4.3. Kolmogorov–Smirnov test cannot reject the hypothesis that the expected returns corresponding
To perform the Kolmogorov–Smirnov test, we determine the maxi- to Martin’s zero correlation coincide with the true expected returns.
mum difference between the empirical cumulative distribution function
implied by our ‘‘realized’’ physical CDF values and the theoretical cu- 7.6. Empirical tests: summary and robustness checks
mulative distribution function of the uniform distribution with support
[0, 1]. Under the null hypothesis, this maximum difference multiplied by We summarize the results of our empirical tests in Table 5. For
the square root of the number of observations asymptotically follows comparison purposes, we also report the results of Jackwerth and
15
Table 5
Empirical test 𝑝-values. From January 19, 1996 to May 19, 2022, we obtain the physical probability distributions of the 1-month return on the S&P 500 market index with
monthly frequency by applying the arbitrage-based recovery theorem, as shown in Fig. 5. We also collect the realized returns on the S&P 500 market index one month after each
observation date. We test the null hypothesis that the realized returns on the S&P 500 index are realizations of random variables with probability density functions implied by
the arbitrage-based recovery theorem. We perform four different density tests (Berkowitz test, Knüppel test with three and four moments, Kolmogorov–Smirnov test) and three
different mean prediction tests on the regression equation (35). For comparison purposes, we also report the results of Jackwerth and Menner (2020), who analyze alternative
recovery theorems performing the same tests and using very similar data to ours (concretely, their sample period is from April 1, 1986 to December 31, 2017). Furthermore, we
also perform the tests on the risk-neutral probabilities.
Ross basic Ross bounded Ross unimodal Ross stable Arbitrage-based recovery Risk-neutral probabilities
Jackwerth and Menner (2020) This paper
Berkowitz test 0.001 0.000 0.000 0.002 0.340 0.070
Knüppel test (3 moments) 0.000 0.012 0.000 0.028 0.284 0.070
Kolmogorov–Smirnov test 0.020 0.049 0.044 0.045 0.179 0.022
Intercept test 0.652 0.001 0.001 0.002 0.292 0.022
Slope test 0.000 0.055 0.022 0.219 0.292 0.023
Joint intercept and slope test 0.000 0.000 0.000 0.000 0.559 0.011
Menner (2020).23 Furthermore, we also perform the tests on the risk- 7.7. Effects of problem regularization
neutral probabilities. Apart from the Knüppel test with four moments
(which has a 𝑝-value of 0.080), at the usual significance levels none of We find (Table 5) that the probabilities implied by Arbitrage-Based
our empirical tests reject the hypothesis that realized returns are drawn Recovery seem to be substantially closer to the true probabilities than
from the probability distributions implied by the Arbitrage-Based Re- the probabilities implied by alternative recovery approaches. Since
covery Theorem. And although failing to reject the null hypothesis does each of the listed recovery approaches attempt to extract the transitory
not translate into accepting the alternative hypothesis, comparing the 𝑝- SDF component in a framework with a closed and bounded price pro-
cess as the only state variable, each approach should recover the same
values of our recovery methodology to those of other recovery methods
SDF as long as the model environment is specified correctly. Under the
(and also to the 𝑝-values of the risk-neutral probabilities), our results
maintained assumptions, this recovered SDF should be the reciprocal of
are promising regarding the empirical plausibility and usefulness of the
the gross return. It is not possible to extract a ‘‘flexible’’ form of the SDF
Arbitrage-Based Recovery Theorem.
(which is constrained only by the transition independence assumption),
Since our recovery approach infers the Perron–Frobenius eigenvalue and any attempt to do so should be futile.
and eigenvector of the Arrow–Debreu price matrix from a theoretical Confronting this consideration with the results in Table 5, it is
recursive no-arbitrage restriction instead of reconstructing the entire very likely that the empirical implementation of the listed alternative
matrix and then solving an eigenvalue–eigenvector problem, most of recovery approaches suffers from some form of model misspecification.
the robustness concerns of other recovery approaches do not apply This is further confirmed by Jackwerth and Menner (2020), who in
to Arbitrage-Based Recovery. For example, when implementing the their Fig. 5 show the SDFs on February 17, 2010 implied by the
original form of Ross’s recovery theorem, one might be concerned about different recovery approaches they study. While in absence of model
how the state space is formed: should moneyness or log-moneyness be misspecification these implied SDFs should each be equivalent to the
used, should the grid be equidistant or not, how many states should reciprocal of the gross stock market return, in reality this is far from the
there be, what should be the ‘‘span’’ of the grid (i.e., what should case. The SDF implied by the Ross basic approach features an inverted
the boundary points of the grid be), etc. By contrast, Arbitrage-Based U-shape, the SDF of the Ross bounded approach has a U-shape and
then an inverted U-shape at high moneyness levels, the Ross unimodal
Recovery is unaffected by how the discretized state space is formed
approach features alternating U-shapes and inverted U-shapes along the
or what its boundaries are. Therefore, our robustness check focuses
moneyness level in the implied SDF, while the SDF of the Ross stable
on whether potentially relevant information is lost by excluding very
approach is mostly flat.
cheap options from our sample (as it is usually done in the literature),
In the case of the Ross basic, Ross bounded, and Ross unimodal
and on whether the fineness of our grid has any significant effect on the approaches, the entire Arrow–Debreu price matrix needs to be con-
smooth state price density curve which might carry over to our results. structed and, in absence of arbitrage opportunities, they should satisfy
𝐷
During our data cleaning procedure, we exclude options with lower the 𝑨 𝑺 = 𝑺 identity. Again, the shapes of the recovered SDFs
than $0.50 bid price. To check that this practice does not lead to in Jackwerth and Menner (2020) strongly suggest that this identity
losing significant information (especially about the tail probabilities, is violated. Possible reasons of this violation can be the effects of
since we use only out-of-the-money options), we repeat our analysis by discretization (i.e., the price is allowed to take only discrete values),
excluding only those options the bid price of which is lower than $0.10 truncation (the price process is assumed to be bounded), or failure of
(instead of $0.50). As we see in Table 6, this does not change our results the Markov property (i.e., the price process in reality is unlikely to
significantly. be Markovian). In the case of the Ross stable approach, although the
In our main analysis, we use a fine grid of 𝛥 = $0.1. To ensure that entire Arrow–Debreu price matrix does not need to be constructed per
se, one row of the matrix needs to be constructed for several different
the fineness of our grid does not have a significant effect on our state
maturities and as long as these rows are consistent with the assumed
price density curve (and therefore on our empirical tests), we repeat
model environment (e.g., time homogeneity and a Markovian price pro-
our analysis using an even finer grid of 𝛥 = $0.033 and a much coarser
cess), the implied (one-period) SDF should be the reciprocal of the gross
grid of 𝛥 = $5. As we show in Table 7, our 𝑝-values do not change
stock market return. Besides Jackwerth and Menner (2020), further
significantly. discussion on the detrimental effects of the enforced regularization of
the problem (e.g., discretization and truncation) on the recovery results
can be found in Walden (2017) and Tran and Xia (2018).
23
Jackwerth and Menner (2020) perform the same tests as we do, using very Arbitrage-Based Recovery, unlike the other listed recovery
similar data. Using data from April 1, 1986 to December 31, 2017, they have approaches, does not require constructing the entire Arrow–Debreu
380 observations, while our data is from January 19, 1996 to May 19, 2022 price matrix (or one row of the matrix for different maturities) and
and we have 317 observations. it also circumvents many of the regularization requirements of the
16
Table 6
Empirical test 𝑝-values, excluding options with a bid price lower than $0.10 or excluding options with a bid price lower than $0.50. From
January 19, 1996 to May 19, 2022, we obtain the physical probability distributions of the 1-month return on the S&P 500 market index with
monthly frequency by applying the arbitrage-based recovery theorem, as explained in Section 7. We also collect the realized returns on the
S&P 500 market index one month after each observation date. We test the null hypothesis that the realized returns on the S&P 500 index are
realizations of random variables with probability density functions implied by the arbitrage-based recovery theorem. We perform four different
density tests (Berkowitz test, Knüppel test with three and four moments, Kolmogorov–Smirnov test) and three different mean prediction tests
on the regression equation (35). We also perform the tests on the risk-neutral probabilities.
Arbitrage-based recovery Risk-neutral probabilities
Excl. if Excl. if Excl. if Excl. if
Pb < $0.10 Pb < $0.50 Pb < $0.10 Pb < $0.50
Berkowitz test 0.330 0.340 0.064 0.070
Knüppel test (3 moments) 0.249 0.284 0.060 0.070
Knüppel test (4 moments) 0.062 0.080 0.016 0.020
Kolmogorov–Smirnov test 0.117 0.179 0.018 0.022
Intercept test 0.283 0.292 0.022 0.023
Slope test 0.282 0.292 0.023 0.023
Joint intercept and slope test 0.543 0.559 0.011 0.011
Table 7
Empirical test 𝑝-values, using grids with different levels of coarseness. From January 19, 1996 to May 19, 2022, we obtain the physical
probability distributions of the 1-month return on the S&P 500 market index with monthly frequency by applying the arbitrage-based recovery
theorem, as explained in Section 7. We also collect the realized returns on the S&P 500 market index one month after each observation date. We
test the null hypothesis that the realized returns on the S&P 500 index are realizations of random variables with probability density functions
implied by the arbitrage-based recovery theorem. We perform four different density tests (Berkowitz test, Knüppel test with three and four
moments, Kolmogorov–Smirnov test) and three different mean prediction tests on the regression equation (35). We also perform the tests on
the risk-neutral probabilities.
Arbitrage-based recovery Risk-neutral probabilities
𝛥 = $0.033 𝜟 = $0.1 𝛥 = $5 𝛥 = $0.033 𝛥 = $0.1 𝛥 = $5
Berkowitz test 0.339 0.340 0.321 0.070 0.070 0.066
Kolmogorov–Smirnov test 0.177 0.179 0.178 0.022 0.022 0.022
Intercept test 0.292 0.292 0.293 0.022 0.022 0.022
Slope test 0.292 0.292 0.293 0.023 0.023 0.023
Joint intercept and slope test 0.559 0.559 0.561 0.011 0.011 0.011
other approaches. Instead, it directly infers the SDF from a theoretical gross returns (including dividends) on a value-weighted portfolio of all
recursive no-arbitrage restriction which the asset price must satisfy, stocks listed on the NYSE, AMEX, or NASDAQ from 31 January, 1947
and it requires only the one-period Arrow–Debreu prices (or state price to 29 December, 2023, with monthly frequency. We also observe the
densities) as an input, which can be obtained from observed one- one-month risk-free rates and the monthly realized returns on the long
period option prices readily along the lines of Breeden and Litzenberger bond.24 We find that the lower bound is 0.7352, which is very close
(1978). to the values of 0.7673 and 0.7755 obtained by Alvarez and Jermann
(2005) using data from December 1946 to December 1999. Hence, the
8. SDF decomposition and arbitrage-based recovery permanent SDF component is very volatile and it plays a significant role
in pricing assets.
The empirical literature (Alvarez and Jermann, 2005 and Bakshi
et al., 2018, among others) documents that the permanent component 8.2. True transitory SDF component vs. recovered SDF
of the stochastic discount factor is very volatile and it is considerably
more important (from an asset pricing perspective) than the transitory Arbitrage-Based Recovery – along with the empirical implemen-
component. At first sight, our results in Section 7 might seem to tation of other recovery approaches – assumes a framework with a
contradict these findings. In this section, we show that there is no stationary, closed and bounded price process. The inferred SDF is equiv-
contradiction and our empirical results can be reconciled with the alent to the transitory SDF component of this featured financial market.
well-documented importance of the permanent SDF component. However, importantly, if the assumed model is not correctly specified
8.1. Empirical significance of the permanent SDF component

24
For the stock market returns, the sources of our data are the CRSP Stock
As Alvarez and Jermann (2005) show, the unconditional volatil- File Indexes. Our risk-free rates are from the CRSP Treasuries Monthly Riskfree
ity of the permanent SDF component relative to the unconditional Series. For the returns on the long bond from January 1947 to December 1985,
volatility of the SDF itself is at least as large as we use the yield curve data in McCulloch and Kwon (1993). To obtain the
[ ( )] monthly long bond returns, at the end of each month we calculate the price
𝑅
E log 𝑅 𝑡+1 of a zero-coupon bond with maturity in 𝜏 years, then one month later we
𝑡+1,∞
[ ( )] ( ), (36) calculate the price of a zero-coupon bond with maturity in 𝜏 − 1 years and 11
𝑅
E log 𝑅 𝑡+1 +𝐿 𝑅1 months (by linearly interpolating between the 𝜏-year and the 𝜏-minus-one-year
𝑡+1,1 𝑡+1,1
yields). For each monthly return, we use the longest possible maturity (i.e., 𝜏)
where( the)unconditional
[ ( )]volatility
[ (of a random
)] variable 𝑥𝑡+1 is defined such that the necessary yield curve data is available to calculate the return via
as 𝐿 𝑥𝑡+1 ≜ log E 𝑥𝑡+1 − E log 𝑥𝑡+1 , 𝑅𝑡+1 is the gross one-period the above procedure. From January 1986 to December 2023, we use the yield
return on any traded asset, 𝑅𝑡+1,∞ is the gross one-period return on curve data provided by the Board of Governors of the Federal Reserve System
the long bond (as defined in Section 4), and 𝑅𝑡+1,1 is the gross one- obtained using the methodology of Gurkaynak et al. (2007). We calculate the
period risk-free rate corresponding to the period from 𝑡 to 𝑡 + 1. Now, monthly returns on the long bond using the same procedure as above, now
we estimate this lower bound using our data. We observe the monthly using a maturity of 𝜏 = 30 years.
17
Fig. 10. Realized values of the inferred SDF and realized values of the transitory component of the true SDF. The SDFs are projected on the space spanned by the monthly
non-annualized gross stock market index return. The inferred SDF is implied by Arbitrage-Based Recovery, and its realized values are equal to the reciprocals of the monthly
non-annualized gross stock market index returns. The realized values of the transitory component of the true SDF are equal to the reciprocals of the monthly non-annualized gross
returns on the long bond. The observation period is from January 1947 to December 2023, and the observation frequency is monthly.
(e.g., if the price process is not stationary in reality), then the inferred
The standard assumption of recovery approaches that the price
SDF might not be equivalent to the true transitory SDF component.
process is bounded, closed, and stationary, is not supported empirically.
The inferred SDF may partially absorb both the true transitory SDF
To show that this indeed causes the inferred SDF to be substantially
component and the true permanent SDF component. Consequently,
different from the transitory component of the true SDF, we plot the
the inferred SDF might be very similar to the true SDF (and hence
realized values of the inferred SDF against the realized values of the
it might be hard to reject statistically using market data) even if the
transitory component of the true SDF (Fig. 10). The SDFs are projected
true SDF has a significant permanent component. This observation can
on the space spanned by the monthly non-annualized gross stock
help reconcile the empirical success (in the sense of Section 7) of our
market index return. The inferred SDF is implied by Arbitrage-Based
recovery approach with the empirical significance of the permanent
Recovery, and its realized values are equal to the reciprocals of the
SDF component (as we found in Section 8.1).
monthly non-annualized gross stock market index returns. The realized
values of the transitory component of the true SDF are equal to the
Example 1. For demonstration purposes, consider an economy with
reciprocals of the monthly non-annualized gross returns on the long
only two assets: a stock (paying no dividend) and a locally risk-free
bond. The observation period is from January 1947 to December 2023,
security. The stock price follows a geometric Brownian motion. The
and the observation frequency is monthly. Further details on our data
risk-free rate and the market price of risk are constant. In this economy,
[ and on how we construct the long bond returns can be found in
the SDF corresponding to a holding period 𝜏 is exp −𝑟𝑓 𝜏 − 𝜆2 𝜏∕2
( P P
)] Footnote 24.
−𝜆 𝑊𝑡+𝜏 − 𝑊𝑡 , where 𝑟𝑓 is the continuously-compounded risk-free If the inferred SDF were equivalent to the transitory component
rate, 𝜆 is the market price of risk, and 𝑊𝑡P is a Wiener process under of the true SDF, the observations in Fig. 10 would all be on a line
the physical probability measure P. The transitory SDF component is
( ) with unit slope. We find that in reality the case is vastly different.
equivalent to the risk-free discount factor (i.e., to exp −𝑟𝑓 𝜏 ), and this We also calculate the correlation between the inferred SDF and the
is also equivalent to the reciprocal of the return on the long bond transitory component of the true SDF, and we find that their correlation
(since the risk-free rate is constant). The permanent SDF component
[ ( P )] is only 0.0595. Based on the sample autocorrelation function, we find
is equivalent to exp −𝜆2 𝜏∕2 − 𝜆 𝑊𝑡+𝜏 − 𝑊𝑡P . The lower bound in no evidence of autocorrelation in the differences between the realized
(36) is equal to one, and the unconditional volatility of the SDF is reciprocals of the stock market index return and the long bond return,
equal to the unconditional volatility of the permanent SDF component. although we find some evidence of weak positive autocorrelation (with
However, if we apply our Arbitrage-Based Recovery approach in this a coefficient around 0.2) in the absolute differences.
economy, our inferred SDF is the reciprocal of the gross stock return,
[ ( P )] Since we find empirically that our inferred SDF (pricing the S&P 500
i.e., exp −𝜇 𝜏 + 𝜎 2 𝜏∕2 − 𝜎 𝑊𝑡+𝜏 − 𝑊𝑡P , where 𝜇 and 𝜎 are the drift stock market index return) is very different from the transitory com-
and the volatility parameters of the stock return process, respectively. ponent of the true SDF, the fact that the permanent SDF component
Clearly, our inferred SDF absorbs a part of the true permanent SDF is very relevant for pricing assets (as we show in Section 8.1) does
component. For example, using reasonable parameter values of 𝜇 = 0.1, not contradict the empirical success of our Arbitrage-Based Recovery
𝜎 = 0.2, and 𝜆 = 0.4, the SDF implied by Arbitrage-Based Recovery approach.
has the same geometric mean as the permanent component of the true The fact that Arbitrage-Based Recovery implies an SDF which is not
SDF, while the former’s geometric volatility is half of the latter’s. Hence, generally equivalent to the true transitory SDF component can also be
in this economy, our inferred SDF can indeed be deemed to absorb a argued at a more fundamental level. Consider the (true) SDF projected
substantial part of the permanent component of the true SDF. on the space spanned by any arbitrary return (i.e., not necessarily the
18
stock market index return). Then, the transitory component of this SDF Declaration of competing interest
is the same, regardless of which return spans the space. To see this, note
that the transitory SDF component is always equal to the reciprocal of The authors declare that they have no known competing finan-
the gross return on the long bond. And just like there is one unique cial interests or personal relationships that could have appeared to
one-period risk-free rate in the economy, there is one unique 𝑘-period influence the work reported in this paper.
risk-free rate in the economy for any arbitrary positive 𝑘 integer as well.
Hence, regardless of which return spans the space, the SDF processes Appendix A
corresponding to those spaces will imply the same 𝑘-period risk-free
rate for any arbitrary positive 𝑘 integer. This holds both at time 𝑡
In this appendix, we describe our procedure of attaching tails to the
and at time 𝑡 + 1. Thus, observing a time series of realized transitory
risk-neutral probability density functions in Section 7.
SDF components, we observe the same realizations, regardless of which
return spans the space. And since the time series of realized returns
on different stocks will obviously be different (and Arbitrage-Based Attaching left tails
Recovery always identifies the inferred SDF with the reciprocal of the
spanning stock return) but the true transitory SDF components are the To attach left tails to the risk-neutral probability density functions
same, the inferred SDFs clearly cannot all be equivalent to the true below the lowest traded strike prices, we proceed as follows. Consider
transitory SDF component. an observation date. First of all, note that between the lowest and the
At this point it is worth referring back to Corollary 2, according to highest traded strike prices we have already obtained the risk-neutral
which the conditional probabilities implied by Arbitrage-Based Recov- pdf by calculating the second derivative of the call price with respect
ery are equivalent to the probabilities used by a pseudo-representative to the strike price, and then dividing this second derivative by risk-free
investor with log-utility over the terminal wealth, who fully invests her discount factor. In a similar vein, we also obtain the risk-neutral cdf
wealth in the market. Actually, the conditional probabilities implied between the lowest and the highest traded strike prices by calculating
by Arbitrage-Based Recovery could also be obtained in a standard the first derivative of the call price with respect to the strike price,
representative agent framework, where the representative investor has dividing this first derivative by the risk-free discount factor, and adding
logarithmic utility over terminal wealth, and the market constitutes all one.
wealth. In such a model, the SDF is the reciprocal of the gross return Let us denote the lowest traded strike price by 𝑆𝑙 , and the strike
on the stock market. The derivation of this SDF does not hinge upon price one grid point to the right from it by 𝑆𝑙+1 . We calculate the risk-
the stock price being the only state variable, it does not assume a neutral pdf and cdf values at 𝑆𝑙+1 . Then, we attach the ‘‘reflected’’ (in
stationary environment, and it does not rely on our additional arbitrage the sense explained below) right tail of a generalized extreme value
condition. Importantly, it does not assume or imply that the permanent (GEV) distribution pdf to the risk-neutral pdf at 𝑆𝑙+1 , as the left tail
SDF component is constant, the framework is silent on this. Since of the risk-neutral pdf itself. Since the GEV distribution is uniquely
the implied conditional probabilities of such a representative investor
characterized by three parameters, we choose the GEV distribution
framework are the same as what our Arbitrage-Based Recovery implies,
which satisfies the following three conditions:
our results in Section 7 can also be interpreted as empirical tests of such
a representative investor framework. This log-utility representative • the GEV distribution pdf value at −𝑆𝑙+1 is equal to the risk-neutral
investor model is also consistent with the results of Martin (2017) in pdf value at 𝑆𝑙+1 ;
the sense that the implied expected stock market return is exactly the
• the GEV distribution cdf value at −𝑆𝑙+1 is equal to one minus the
same as the lower bound of Martin (2017). Interpreting our results from
risk-neutral cdf value at 𝑆𝑙+1 ; and
the perspective of such a representative investor framework, there is
• the GEV distribution cdf value at 𝑆 = 0 is equal to one.
no (seeming) tension between our empirical results and the literature’s
findings on the empirical relevance of the permanent SDF component. Then, we reflect the obtained GEV distribution pdf on a vertical line
at −𝑆𝑙+1 , and attach the reflected GEV pdf section below −𝑆𝑙+1 to the
9. Conclusion previously obtained (‘‘tailless’’) risk-neutral pdf at the strike price 𝑆𝑙+1 .
We develop a novel recovery theorem based on no-arbitrage princi-

Attaching right tails
ples. Our Arbitrage-Based Recovery Theorem requires the observation
of option prices only for one single maturity and only for the current
state of the world. To test how well our theorem works empirically, After adding left tails to the risk-neutral pdfs, we now similarly
we use more than 26 years of S&P 500 options data with monthly attach right tails to them. Let us denote the highest traded strike price
observation frequency, and perform several density tests and mean on a particular observation date by 𝑆𝑟 , the strike price one grid point to
prediction tests. Using the same tests and very similar data to ours, its left by 𝑆𝑟−1 , and the strike price one grid point to its right by 𝑆𝑟+1 .
the literature tends to reject the empirical validity of other, competing We calculate the risk-neutral pdf and cdf values at 𝑆𝑟−1 . Then, we attach
recovery theorems. In contrast with this, none of the commonly used the right tail of a GEV distribution pdf to the risk-neutral pdf at 𝑆𝑟−1 ,
density tests and mean prediction tests rejects the empirical validity as the right tail of the risk-neutral pdf itself. Concretely, we choose the
of the Arbitrage-Based Recovery Theorem at the standard five percent GEV distribution which satisfies the following three conditions:
significance level, and the vast majority of the tests cannot even reject
it at a very conservative 15% significance level. After conducting our • the GEV distribution pdf value at 𝑆𝑟−1 is equal to the risk-neutral
empirical exercise, we demonstrate that our results do not contradict pdf value at 𝑆𝑟−1 ;
the findings of Alvarez and Jermann (2005), among others, that the • the GEV distribution cdf value at 𝑆𝑟−1 is equal to the risk-neutral
permanent component of the stochastic discount factor is empirically cdf value at 𝑆𝑟−1 ; and
very volatile. • the GEV distribution cdf value at 2 × 𝑆𝑟+1 is equal to one.
CRediT authorship contribution statement

Modifying the tails to enforce no-arbitrage
Ferenc Horvath: Writing – review & editing, Writing – origi-
nal draft, Visualization, Validation, Supervision, Software, Resources, After adding left and right tails to the risk-neutral density functions,
Project administration, Methodology, Investigation, Funding acquisi- we still need to make sure that these density functions are arbitrage-
tion, Formal analysis, Data curation, Conceptualization. free. In other words, we must check whether the price (implied by the
19
risk-neutral density function) of the future stock cash flow (i.e., divi- Cox, J.C., Ross, S.A., Rubinstein, M., 1979. Option pricing: A simplified approach. J.
dend plus post-dividend stock price) is equal to the current stock price. Financ. Econ. 7 (3), 229–263.
Dillschneider, Y., Maurer, R., 2019. Functional Ross recovery: Theoretical results and
Since we did not enforce this no-arbitrage condition when attaching
empirical tests. J. Econom. Dynam. Control 108.
left and right tails, it is not necessarily the case that our risk-neutral Dubynskiy, S., Goldstein, R.S., 2013. Recovering Drifts and Preference Parameters from
pdf at this stage is free of arbitrage. Therefore, we slightly modify the Financial Derivatives. Working Paper.
attached left and right tails as follows. Ebert, S., 2013. Moment characterization of higher-order risk preferences. Theory and
First, we divide both tails into two equal sections. Then, on each Decision 74, 267–284.
Figlewski, S., 2010. In: Bollerslev, T., Russell, J., Watson, M. (Eds.), Volatility and Time
of the four sections (two on the left tail and two on the right tail) we
Series Econometrics: Essays in Honor of Robert Engle. Oxford Scholarship Online,
define a modifier function 𝑔(𝑆) ∶ R>0 → R>0 as
( ) chap. Estimating the Implied Risk Neutral Density for the U.S. Market Portfolio.
𝑆 − 𝑆𝑎 Ghosh, A., Roussellet, G., 2020. Identifying Beliefs from Asset Prices. Working Paper.
𝑔 (𝑆) = 𝛼 × sin × 𝜋 + 1, (37) Gormsen, N.J., Jensen, C.S., 2020. Higher-Moment Risk. Working Paper.
𝑆𝑏 − 𝑆𝑎
Gurkaynak, R.S., Sack, B., Wright, J.H., 2007. The U.S. Treasury yield curve: 1961 to
where 𝛼 ∈ R is a scalar greater than minus one, and 𝑆𝑎 ∈ R>0 and the present. J. Monetary Econ. 54 (8), 2291–2304.
𝑆𝑏 ∈ R>0 are the lower and the upper end points of the section, Hansen, L.P., Scheinkman, J.A., 2009. Long-term risk: an operator approach.
respectively. The modifier parameters (denoted by 𝛼 in (37)) of the Econometrica 77 (1), 177–234.
Heston, S.L., 2021. Recovering the Variance Premium. Working Paper.
four sections can be different. To obtain the new left and right tails
Huang, D., Shaliastovich, I., 2014. Risk Adjustment and the Temporal Resolution of
of the risk-neutral density function, we multiply the original pdf values Uncertainty: Evidence from Options Markets. Working Paper.
on the tails by the modifier function of the respective section. Finally, Jackwerth, J.C., 2004. Option-Implied Risk-Neutral Distributions and Risk Aversion.
we choose the four modifier parameters so that neither the area under The Research Foundation of AIMR.
the left tail nor the area under the right tail changes, but the expected Jackwerth, J.C., Menner, M., 2020. Does the Ross recovery theorem work
empirically? J. Financ. Econ. 137 (3), 723–739.
value implied by the new risk-neutral density function, multiplied by Jensen, C.S., 2021. The Ex Ante Physical Distributions of Individual Stock Returns.
the risk-free discount factor, is exactly equal to the current stock price Working Paper.
less the price of the dividend.25 Throughout our sample, the magnitudes Jensen, C.S., Lando, D., Pedersen, L.H., 2019. Generalized recovery. J. Financ. Econ.
of the modifier parameters are small (close to zero), which suggests that 133 (1), 154–174.
Kazemi, H.B., 1992. An Intemporal Model of Asset Prices in a Markov Economy with
the new (‘‘modified’’ and arbitrage-free) risk-neutral density functions
a Limiting Stationary Distribution. Rev. Financ. Stud. 5 (1), 85–104.
are very similar to their ‘‘non-modified’’ counterparts. Knüppel, M., 2015. Evaluating the calibration of multi-step-ahead density forecasts
using raw moments. J. Bus. Econom. Statist. 33 (2), 270–281.
Appendix B. Supplementary data Liu, F., 2014. Recovering Conditional Return Distributions by Regression: Estimation
and Applications. Working Paper.
Markowitz, H., 1952. Portfolio selection. J. Finance 7 (1), 77–91.
Supplementary material related to this article can be found online
Martin, I., 2017. What is the expected return on the market? Q. J. Econ. 132 (1),
at https://doi.org/10.1016/j.jfineco.2024.103969. 367–433.
Martin, I.W.R., Ross, S., 2019. Notes on the yield curve. J. Financ. Econ. 134 (3),
Data availability 689–702.
Massacci, F., Williams, J.M., Zhang, Y., 2016. Hansen-Scheinkman Factorization and
Ross Recovery from Option Panels. Working Paper.
MATLAB code (Original data) (Mendeley Data) McCulloch, J., Kwon, H., 1993. U.S. Term Structure Data, 1947–1991. Ohio State
University Working Paper 93-6.
Mehra, R., Prescott, E.C., 1985. The equity premium: A puzzle. J. Monetary Econ. 15
(2), 145–161.
References Merton, R.C., 1973. Theory of rational option pricing. Bell J. Econ. Manag. Sci. 4 (1),
141–183.
Park, H., 2016. Ross recovery with recurrent and transient processes. Quant. Finance
Alvarez, F., Jermann, U.J., 2005. Using asset prices to measure the persistence of the
16 (5), 667–676.
marginal utility of wealth. Econometrica 73 (6), 1977–2016.
Pazarbasi, A., Schneider, P., Vilkov, G., 2021. Dispersion of Beliefs Bounds: Sentimental
Audrino, F., Huitema, R., Ludwig, M., 2019. An empirical implementation of the Ross
Recovery. Swiss Finance Institute Research Paper Series 19–57.
recovery theorem as a prediction device. J. Financ. Econom. (nbz002).
Qin, L., Linetsky, V., 2016. Positive Eigenfunctions of Markovian Pricing Operators:
Bachelier, L., 1900. Théorie de la spéculation. Ann. Sci. Ec. Norm. Supér. 17, 21–86.
Hansen-Scheinkman Factorization, Ross Recovery, and Long-Term Pricing. Oper.
Bakshi, G., Chabi-Yo, F., Gao, X., 2018. A recovery that we can trust? Deducing and
Res. 64 (1), 99–117.
testing the restrictions of the recovery theorem. Rev. Financ. Stud. 31 (2), 532–555.
Qin, L., Linetsky, V., 2017. Long term risk: A martingale approach. Econometrica 85
Bansal, R., Yaron, A., 2004. Risks for the long run: A potential resolution of asset
(1), 299–312.
pricing puzzles. J. Finance 59 (4), 1481–1509.
Qin, L., Linetsky, V., Nie, Y., 2018. Long forward probabilities, recovery, and the term
Berkowitz, J., 2001. Testing density forecasts, with applications to risk management.
structure of bond risk premiums. Rev. Financ. Stud. 31 (12), 4863–4883.
J. Bus. Econom. Statist. 19 (4), 465–474.
Ross, S.A., 1976. Options and efficiency. Q. J. Econ. 90 (1), 75–89.
Black, F., Scholes, M., 1973. The pricing of options and corporate liabilities. J. Polit.
Ross, S., 2015. The recovery theorem. J. Finance 70 (2), 615–648.
Econ. 81 (3), 637–654.
Schneider, P., Trojani, F., 2019. (Almost) model-free recovery. J. Finance 74 (1),
Bollerslev, T., Tauchen, G., Zhou, H., 2009. Expected stock returns and variance risk
323–370.
premia. Rev. Financ. Stud. 22 (11), 4463–4492.
Borovička, J., Hansen, L.P., Scheinkman, J.A., 2016. Misspecified recovery. J. Finance Tran, N.-K., Xia, S., 2018. Specified Recovery. Working Paper.
71 (6), 2493–2544. Walden, J., 2017. Recovery with unbounded diffusion processes. Rev. Finance 21 (4),
Borovička, J., Stachurski, J., 2021. Stability of equilibrium asset pricing models: A 1403–1444.
necessary and sufficient condition. J. Econom. Theory. Zhan, X., 2013. Matrix Theory. American Mathematical Society.
Breeden, D.T., Litzenberger, R.H., 1978. Prices of state-contingent claims implicit in Zhu, S., 2020. Why Does the Recovery Theorem Not Work Empirically? A
option prices. J. Bus. 51 (4), 621–651. Martingale-Based Explanation. Working Paper.
Carr, P., Yu, J., 2012. Risk, return, and Ross recovery. J. Deriv. 20 (1), 38–59.
Chetty, R., Rosenberg, J., Saez, E., 2005. The Effects of Taxes on Market Responses
to Dividend Announcements and Payments: What Can We Learn from The 2003
Dividend Tax Cut?. NBER Working Paper.
25
In our empirical analysis in Section 7, we assume that the dividend to be
received until the maturity of the option (i.e., in the next 28 days) is known
at the time of observation. As noted by Chetty et al. (2005), firms usually
announce their dividends four to six weeks before the dividend payment itself,
so this assumption on our side is reasonable.
20

Arbitrage-based recovery

Uploaded by

Copyright:

Available Formats

Arbitrage-based recovery

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Arbitrage-based recovery

Uploaded by

Copyright:

Available Formats

Journal of Financial Economics 163 (2025) 103969

Contents lists available at ScienceDirect

Journal of Financial Economics

ARTICLE INFO ABSTRACT

The expected (total) return on the market index is on average 0.52%

Table 3 (buy-and-hold) and the third (market-timing) strategies more directly

absolute value of their expected values. Similarly to the Sharpe ratio,

8.1. Empirical significance of the permanent SDF component

We develop a novel recovery theorem based on no-arbitrage princi-

CRediT authorship contribution statement

You might also like