The Stationary Bootstrap Author(s): Dimitris N. Politis and Joseph P.

Romano Reviewed work(s): Source: Journal of the American Statistical Association, Vol. 89, No. 428 (Dec., 1994), pp. 13031313 Published by: American Statistical Association Stable URL: http://www.jstor.org/stable/2290993 . Accessed: 23/01/2012 00:46
Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at . http://www.jstor.org/page/info/about/policies/terms.jsp JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected].

American Statistical Association is collaborating with JSTOR to digitize, preserve and extend access to Journal of the American Statistical Association.

http://www.jstor.org

The Stationary Bootstrap

N. Dimitris POLITis and Joseph P. ROMANO*
This article introduces resampling a procedure calledthestationary as bootstrap a meansofcalculating standard errors estimators of and constructing confidence regionsforparameters based on weaklydependentstationary observations. a Previously, technique based on resampling blocksofconsecutive observations introduced construct was to for confidence intervals a parameter the mof dimensional joint distribution m consecutive of wherem is fixed. This procedure been generalized constructing observations, has by a "blocksof blocks"resampling schemethatyieldsasymptotically valid procedures even fora multivariate of parameter thewhole (i.e., infinite-dimensional) distribution the stationary joint of sequence of observations. These methodssharethe construction of resampling blocksof observations forma pseudo-timeseries,so thatthe statistic interest to of may be recalculated based on the resampled data set.But in thecontext applying methodto stationary of this to data,it is natural require resampled the pseudo-time seriesto be stationary (conditionalon the original the data) as well. Although aforementioned lack thisproperty, procedures the stationary procedure and developedhereis indeedstationary possessesother desirable The properties. stationary is procedure based on resampling blocksof randomlength, wherethe length each block has a geometric of In distribution. thisarticle, fundamental and consistency weak convergence of properties thestationary schemeare developed. resampling KEY WORDS: Approximate confidence limit;Time Series.

In Section2 the actualconstruction the stationary of is and bootstrap presented comparisons madewith are the Thebootstrap Efron of (1979)hasproven be a powerful blockresampling to method Kunsch(1989) and Liu and of nonparametric for tool approximating sampling the distri- Singh (1992).Sometheoretical propertiesthemethod of are bution and variance complicated of statistics basedon iid investigatedSection inthecaseofthemean.In Section in 3 observations. Recently, Kiinsch (1989) and Liu and Singh 4 it is shown howthetheory be extended may beyond the (1992) haveindependently introduced nonparametric caseofthe vermean construct to valid asymptotically confidence sionsofthebootstrap jackknife are applicable regions general and that to for parameters. weakly dependent stationary observations. resampling Their technique amounts resampling deleting to or one-by-one 2. THESTATIONARY BOOTSTRAP whole blocks observations, of toobtain consistent procedures RESAMPLING SCHEME for parameter them-dimensional a of distribution marginal Supposethat{Xn, n E Z } is a strictly and stationary ofthe stationary Their series. resampling procedure been has weakly dependent series, time where X, arefor the nowasgeneralized Politis Romano(1992a,1992b)and by and by real-valued. that of whole Suppose ,tisa parameterthe Politis, Romano, Lai (1992)byresampling and "blocks of sumed (i.e.,infinite-dimensional)distributionthesequence joint of blocks" observations obtain of to asymptotically provalid cedures evenfor multivariate parameters thewhole of (i.e., { X, n E Z }. For example,,t mightbe the mean of the or distribution function. Given dataX1, infinite-dimensional) joint distribution stationary ofthe time process thespectral
... XN, the goal is to make inferences about ,t based on some estimator = TN(X1,. .. XN). In particular, are we TN In thisarticle introduce newresampling we a method, in interested constructing a confidence region ,t. Typifor calledthestationary that bootstrap, is also generally appli-

1. INTRODUCTION

series.

an of of distribution TNis recableforstationary time weakly dependent series. Similar cally, estimate thesampling and quired, thestationary method here bootstrap proposed totheblock resampling the techniques, stationary bootstrap In is developed this for purpose. general, areledto conwe involves resampling original toform pseudo-time the data a a or pivot series from which statistic quantity interest be sidering"root" an approximate RN = RN(X1, * *, the or of may which justsomefunctional is XN;,u), on depending thedata this recalculated; resampling procedure repeated build is to on be and possibly ,t as well. For example,RN might ofthe to up an approximationthesampling distribution staofthe formRN = TN- ,t or possiblya studentized version.The In tistic. contrast theaforementioned resampling to block of distribution RN were the methods, pseudo-time series generated thestationaryidea is thatif thetruesampling by then known, probability statements aboutRNcouldbe inmethod actually stationary series. is a time bootstrap That to confidence statements ,u. stationary about The is, conditional the original on data X1, . .. , XN,a pseudo- verted yield is that to bootstrap a method can be applied approximate time seriesX *, ..., XN is generated an appropriate by thedistributionRN.To describe algorithm, of the let resampling scheme thatis actually stationary. Hencethis procedure attempts mimic original to the model retaining by * * * Xi+b-1 } (1) Bi,b = {Xi, Xi+l, thestationarity in of property theoriginal series theresamfrom of starting Xi. pledpseudo-time series. willbe seen,thepseudo-time be theblockconsisting b observations As In thecase > N, Xj is defined beXi, where = j(modN) i to j series generated resampling is blocksof random by size, in and number [0, 1]. Independent where length eachblock a geometric the of has distribution. Xo = XN.Letp be a fixed
ofX1, . .. , XNlet L1, L2, . . . be a sequence of iid random
? 1994 American Statistical Association Journal of the American Statistical Association December 1994, Vol. 89, No. 428, Theory and Methods * Dimitris Politis Assistant N. is Purdue Professor, of Department Statistics, WestLafayette, 47907. Joseph Romano is AssociateProIN P. University, of Stanford CA fessor, Department Statistics, University, Stanford, 94305. 1303

1304

Journal of the American Statistical Association, December

1994

on tribution { 1, ...,

series thesequence b valuesin are of sothat prob- in thepseudo-time distribution, the variables having geometric the series are in b abilityof the event { Li = m} is (1 - p)m`p form = 1, B*, thenext observations thepseudo-time N in of .... Independent theXi and theLi, letII, I2, ... be a theb values B', andso on. In thecase, is notdivisible 2, integer satisfying> N. Resample bk smallest uniform dis- byb,letkbethe sequence iidvariables havethediscrete of that
X to N}. Now, a pseudo-timeseriesX *1, kblocksas previously generate
1,... ,

X the j in following Sample sequence delete observationsJ'for > N. a .. ., X Nisgeneratedthe way. the and between staSomeofthesimilarities differences of blocksof randomlength the prescription by B1,L1, blocksbootstrap and algotionary bootstrap the moving in The first observations thepseudo-time L1 BI2,L2I .... the series should apparent. begin, pseudo-time To rithms be block X are determinedby the first series X *, ..., is blocksmethod notstationary. generated themoving by L2 XI +Ll-1, and thenext XI,, BII,L, of observations In of Both involve resampling blocks observations. methods in series theobservations are observations thepseudo-time of the blocks themoving technique, number observations in the second sampledblock BI2,L2, namelyXI2, .... bootstrap ineachblock a fixed number Inthestationary b. is thisprocess stopped is onceN obser- method, number observationseachblock random XI2+L2-1 Of course, in is the of serieshave been generated andhasa geometric vationsin the pseudo-time in also distribution. methods differ The allowsfor howthey method it (though is clearthattheresampling For there end because dealwith effects. example, Once X *, is no data after themoving of to timeseries arbitrary length be generated). does not blocksmethod XN, X has been generated, compute TN(X N*, *., XN) . b at define blockoflength beginning XN(ifb > 1). To a series. The achieve or RN(X 1*, the ..., XN; TN) for pseudo-time timeseries, stathe for stationarity theresampled of conditional distribution RN(X *,. . ., X N; TN) given X1, tionary method "wraps"the data aroundin a bootstrap ... , XNisthestationary to approximation thetrue "circle," that "follows" bootstrap X1 so XNof sampling distribution RN(Xl, . . , X,, ui). By simulating basedon resampling bootstrap Variants thestationary on B in a large series thesamemanner, blocks random number ofpseudo-time are Instead assuming of of length possible. the truedistribution RN(Xl, . . , XN; u) can be approx- that Li havea geometric of one the distribution, can consider of imatedby the empirical distribution the B numbers other for Alternative distributions theIi can distributions. 1X XAN, 1TNJR\fAX In blocks be viewed beusedas well. this themoving way may An alternative perhaps simpler description there- as a special of and a discase.The choiceofLi having geometric follows. X *be picked random tribution Ii as the Let at sampling algorithm discrete uniform distribution made was and so fromthe original observations, thatX = XI,. With so that resampled N Of other the series stationary. course, is the at from original resampling probabilityletX 2 be picked random p, for sestationary theresamples schemes achieve 1 N observations; probability- p, letA 2 = XI1+1 with SO ries.For example,one could take the seriesX V,..., X N and series in constructed add an independent thatXA wouldbe the"next"observation theoriginal as previously given thatX * is de- Z , .. ., Z Nto it, as a "smoothing"device. For the sake timeseries following In general, X,,. will on particular attention focus the scheme termined theJth original series, ofconcreteness, time by observation inthe XJ 1with probability p andpicked that initially we letX *+ be equaltoXJ+1 proposed. the N aboutthedifference between from original observations probabil- Another to think with at random the way is blocksmethod thestationary and bootstrap as p. ity moving a For blocksize b, one can compute follows. each fixed error 1. distributionan estimate standard ofan or of Proposition Conditionalon X1, . .. , XN, A 1, A 2, bootstrap A , X N is stationary. here The method estimator. stationary bootstrap proposed Muchmore actually if of is true. example, theoriginal is essentially weighted For a blocks average thesemoving orestimates standard where observations ... , XNareall distinct, thenewseries bootstrap then distributions of X1, error, AT ... AT is, conditionalon X1, ..., XN, a stationary theweights determined a geometric Itis are distribution. by N 1, Markov to that aspect chain.If,on theother hand,twoof theoriginal important keepinmind a difficult inapplying is blocks observations identical theremaining distinct,thesemethods how to chooseb in the moving are and are Inscheme. then the new seriesX .*, . ., A N is a stationary and howto choose in thestationary second- scheme p An a order Markov generalization, depending deed,theissuebecomes "smoothing" chain. obvious problem. on thenumber identical of of subsequences observations, 3. THEMEAN if canbe made.In fact, misthelargest suchthat, some b for 1 i distinct j (and bothi andj between and N), Bi,b from In thissection, specialcase of thesamplemeanis the andBj,bareidentical m = 0 ifall observations dis- considered a first tojustify validity the are (and of stationary as the step Mar- bootstrap then X . tinct), theseries ,, ., X Nis a (m + 1)-order scheme.Let y = E(X1) and setTN(X1, resampling kovchain. under stationarity, *. *, XN) = XN = N- 1 ziN 1Xi. Notethat Thestationary resampling scheme proposed here if a2 is defined be thevariance N112XN, bootstrap of then to is distinct from by (1989) and Liu thatproposed Kiinsch and Singh(1992). Their"moving blocks"method deis with scribed follows. as SupposethatN = kb. Resample replacement theblocks from B1,b, ... ., BN-b+?,b to get rek b sampled blocks, B*, . .., Bk*.The first observationsUnder assumption E]-i Icov(X1 X1)I < so, which say that , the
'

X bk.Now simply

Politisand Romano: The Stationary Bootstrap

1305

is implied typical by assumptions weakdependence, hence of it doesnotaccount thebiasofTN as an estimator for of follows i -2__ a as N -s oc, where that TN ,t (unless haszerobias). if Remark2. In fact, weconsider more the general (posscheme sibly nonstationary) resampling where Li's are the iidwith common a (possibly nongeometric) distribution, but Moreover, typically we have thatRN(X1, ..., XN; /i) theIi's areiiduniform { 1,. .. , N}, then conditional on the - N1/2(XN - t) tends distributionthenormal in to distri- meanofX N isXN. In particular, closecousinofthemoving a a mean andvariance 2 . A primary ofthis blocks 0 bution with goal scheme yields correct that the bootstrap (conditional) the section toestablish validity thestationary is of bootstrap mean the for corresponding bootstrap distribution isobtained defined the conditional distribution byletting bethedistribution of approximation by mass assigning onetoa fixed Li X N; XN) giventhedata. RN(X *1, ..., b (seePolitis Romano1992c). and As a first toward end,and ofinterest itsown in step this We now consider stationary the bootstrap estimate of right, first we consider meanand variance N112X varianceofN112XN defined the of N by p8 var(N1/2X I X1,.... N X *. Be- XN). In Lemma 1, a formulaforUN,p iS obtained,so that X on (conditional thedata),where N = N-1 cause E(X 1 IX1, ... , XN) = XN, a trivialconsequenceof UN,p maybe calculated without In resampling. thelemma, UN,p is given terms the in of circular autocovariances, defined , XN) = XN. Because thetrue is E(X N X1, stationarity .I. distribution N112(XN- Ht) has mean 0, it follows of thatthe by to distribution of bootstrap approximation the sampling lN
a2
=

var(X ) + 2 z cov(Xl,Xl+i).
i=l

(3)

N1/2(XN

has thissame mean.

CN(i)

NJ=1 Remark Forthemoving 1. blocks it scheme, is notthe casethat E(X N*I X13, . .. , XN) = XN.It is easyto seethat andtheusualcovariance estimates, IX1, E(X N* .. *

[(XJ -

XN) (Xj+i

XN)-

XN)
b1 i(Xi

+ XN-i+l)+ b (N - b+ l)b
E(XNIX,...,

j=b

RN(i) =

(4)

Lemma 1.
UyN,p = CN(O)

NJ=1

[(iXj-XN)(Xj+i

XN)]-

+ Op(b/N). To see why, calculate meanand the simply of variance E(X NIX1, ... , XN) -XN with aid of(4) or Alternatively, the seetheproof (iii)in theorem ofLiu and Singh of 6 (1992). In summary, moving to the blocks bootstrap approximation ]N,p = thesampling distributionN12 (XN- /i) hasa meanthat of of byLiu andSingh (1992),toachieve consistencythemov- where ingblocks bootstrap estimate variance N112XN, it is of of
is Op(b/N1/2) as N
-#

Thusif b/N 0as

N-oo,

XN)=XN

-NO -P)CN(M) N

(5)

RN(O)

N-1

z
i=l1

bN(i)RN(

i)

(6)

oo and b/N -# 0. As demonstrated . Moreover, Kunsch( 1989)

(7) P) ( l p)N* N is proved thechoice ocN1/3 optimal minimize that b to the N ~ meansquared of error themoving blocks bootstrap estimate estimate Evidently, Lemma1 tellsus that bootstrap the of variance. sucha choice, moving For the blocksbootto ofvariance Np, givenby(6), is closelyrelated a lag winis at strapdistribution centered a location, Op(b/N1/2) estimate density * wheref() isthespecoff(O), = Op(N-1/6),which tends zeroquiteslowly. to Thusone dowspectral of * tral density theoriginal process. Assuming thatf( ) exists cannot the blocks to expect moving bootstrap possess any it of (which does undersummability covariances), is f(O) second-order at not coroptimality properties,least without a2 where is given (3). Henceit is clear simplya /22r, by for the recting thebias by recentering bootstrap distribua2 1 for accounting thefactor /27r, estimating or aN the of tion.One possibility to approximate distribution that, is in in to [given (2)] is equivalent estimatingf(O)a first-order N112(XN -t) bythe(conditional) distributionN1 2[X N of of We a sense. nowprove consistency asymptotic property -E(X N IX1,... , XN) ] (seeLahiri 1992).Suchan approach UN. Although authors theorems on havedeveloped many in the but maybe satisfactory caseofthemean, itweakens theconsistency of properties spectral estimates, including theclaim that bootstrap supposed be a general is to the pur- Priestley (1981), Zurbenko (1986), and Brillinger (1981), would nonefits this pose"automatic" technique. Moreover, approach in In easily ourframework.Theorem K4(s, r, v) 1, notwork welloutside case ofthemean.Thatis, in is thefourth cumulant thedistribution(xj, as the of of joint Xj+r, thegeneral context estimatingparameter by some Xjj+s, of a ,t Theassumptions thetheorem similar of are AXJ+s+r+v). estimator = TN(X1, ... , XN), consider approximation those the TN usedbyBrillinger to (1981)and Rosenblatt (1984). to thesampling distributionN112(TN - ,) bythe(conof Let X2,.. bea strictly X1, . stationary process ditional) distribution N1/2[ -E( TN IX1, ... *,X)] of Theorem]1. TN
bN(i) +(1

that necessary b -# oo as N -so

whereTN

that mating bootstrap distribution necessarily mean0 and <soc. Assume p has

TN(X

1', . ..,

X N). In thiscase the approxi-

with covariance function satisfying + ErIrR ( r) I R (.*) R (0) = PN ? , NPN -00o, and

1306

Journal of the American Statistical Association, December

1994

u,v,w

IK4(U,V,W)I=K<oo.

(8)

with distribution B has where (independently) a geometric

mean PN , yielding
CN(O)2+2 MN
00

to &N,PN tends a2 of estimate variance Thenthebootstrap in probability. it moreeffort, can be shown In fact, with onlyslightly of 1, the that, under sameconditions Theorem a^,PN tends to a2 inthesenseE(&,PN -2 actually )-2 O. Theproof In [see shows muchmore. particular (19)],
00

b- I
1-b (I -PN)b

E NB)=

CN(O)+ 2
0 00 i=1 b=+

1 __
b

(1-P)PNN(i)

E(& k,PN)

(2-

2PN

,
i=l1

iR(i) + o(PN)

(9)

CN(O)+ 2
i=

bN(i)Cl(i) b
1

and var( & choosep =

,pN)

of then of estimator aN is minimized, theorder thesquared

PNSO

if Consequently, thegoal is to of thatthemean squarederror &2,PN as an

0(1

where
bN(i) =

/NPN).

(1

PN) +

iPN(-

PN) X 109
(J))

bias,PN. shouldbe thesame orderas thevariance, (NpN) -. toward also The Thisoccurs ocN-1/3. calculation points ifPN

)Pj

bN(i), where pNlog(pN) -O 0 as PN O 0, bN(i) For inchoosing optimally. ifthe goalremains Because p the difficulty estimate bootstrap in of PN error NP, then should bN( i) is given (7). Hencethestationary minimizing meansquared the avas approximatelya weighted maybe viewed _# on the c N1/3PN c, where constantdepends intricateofvariance satisfy basedon resampling of such process, as :iiR(i). Estimationerageoverb ofestimates variance of original propertiesthe of that suggests thechoice p in of b; length this blocks fixed fundamental difficult. c Fortunately, ofthis constantappears thanthechoiceofb in is scheme lesscrucial are of bootstrap unaffected not thestationary by consistency propertiesthe simblocks scheme. Moreover, an argument by to to It choosing optimally. is important havep tending 0 themoving p 1, mN ,bN)-- 0 ifb = bN = 1/ but second-order rate properties, ilarto Theorem var(UN,PN at theproper to achieve The of 1 in seemsto enter third-order andtheconditions Theorem aresatisfied. same PN, c the getting constant right by claimcan be madeifMN,b is replaced theexactmoving properties. of estimate variance. blocks bootstrap the Remark We nowcompare stationary 3. substantiate blocks the bootstrap Two SimulatedExamples. To empirically of estimate variance, with moving ^N,P, wereconsidered, some numerical examples theseclaims, for that estimate variance. of Suppose, simplicity, N = kb. were X1, observations .. . , X200 First, of is bootstrap estimate variance kl basedon simulation. Thenthemoving blocks
-

accordingto the model X, = Zt + Zt-1 + Z-2 generated normal. Because + Zt-3 + Zt-4, where theZt areiid standard X b* is a block fixed ) Bib, from b at length chosen random of withN = 200, variance N1/2XN, of Zi, E 1O?0 Xi 5 E 12= the ... BN-b,b.Except end the blocks bootstrap for effects, moving 02 EXoXk> 25. the nearly Notethat autocovariances to of is estimate variance equivalent mN,b- bi var(SI,b IX1, is very blocks stationary and 1, any in Bi,b for "lag"k. InFigure themoving XN), whereSi,b is thesumoftheobservations . as of are estimates variance N1/2XN plotted of { in at from 1,... , N}. bootstrap defined (1) andI is chosen random
N var(Xl +
+f +X

IXI ...

XN)

.where

(X

,..

to similar Lemma1, Byan argument

mNb
=

CN(O)

+2

(1

--A)iN(i). b

(10)

that size functions block b and I /p.Notice thestationary of that is less estimate variance much variable; is, of bootstrap blocks the of to itis lesssensitive thechoice p than moving of is bootstrap to thechoice b.
+ Z4-2-

the b. that provided p-i is approximately Intuitively, staof blocks random length scheme samples bootstrap tionary the 1/p,so thetwoapproaches roughly sameiftheexare in blockis number observations eachresampled of pected To substantiate claim the thesamefor both methods. further . ........... that 2 expansion Kc' Kuinsch's /b,notethat MN,b 2P ifp = 1 for biasofthe the estimate variance of exactly blocks moving coincides with(9). In fact, (10) showsthatthe moving blocks-andhence alsothestationary bootstrap-variance to estimates both are equivalent a lagwindow approximately kernel Priestley 1981 estimate Bartlett's (see using spectral 200 150 100 50 0 the way interesting to view But more for details). a perhaps size block One can compute is estimates as follows. two variance overa each b and thenaverage by mN,b defined (10) for Line) (Dotted Blocks(SolidLine)andStationary 1. Figure TheMoving
U,, C\j 75

Comparing MN,b with ', in (5), thetwoare quite close,in 1 I - ip, (1 view of the approximation - iN-1) (I - p)

fromthe model X, = Zt-Zt-l Next, 200 observations whereagaintheZt are Z-3 + Zt-4 weregenerated,

compute ofb In distribution values. particular, E(mAN,B),

Estimates. Bootstrap

Politisand Romano: The Stationary Bootstrap

LO

1307

normal.In thiscase theautocovariances EXoXk, iidstandard in fork = 1, 2, . . ., alternate signuntiltheybecome 0 for than 4. In Figure2, the movingblocks and co lags k greater again of of estimates variance N112XNare bootstrap stationary it of as plotted functions block size b and I /p. As before, is of estimate variance bootstrap thatthe stationary observed to is much less sensitive the choice of p than the moving blocksis to the choice of b. In thissecond model,the true varianceof the sample mean is near 1, and (standardized) estimateis nearerto 1 fora wide bootstrap the stationary 200 150 100 50 0 quitegenhas rangeofp values;thisbehavior been observed block size in examples.Note thatbothFigure1 and Figure erally other bootstrap Figure2. The MovingBlocks (Solid Line) and Stationary our 2 confirm previousclaim thatthe stationary (DottedLine) as estimateof variance may be viewed approximately a BootstrapEstimates. estimates blocksbootstrap overb ofmoving average weighted ofvariance. or the qN( of We nowtakeup theproblem estimating distribution Due to possiblediscreteness uniquenessproblems, 1 to confidence - a) shouldbe defined be the 1 - a quantileofthe(conwiththe goal of constructing of N12 (XNt), let on ofX assumption the original ditional)distribution XN- XN; in general, the 1 - a mixing for intervals ,u.A strong G distribution be inf{q: G(q) ? 1 processwillbe in force.That is, it is assumedthatdata X1, quantileof an arbitrary interval that follows thebootstrap sequence { X,, n - aI}. Then itimmediately XN are observedfroman infinite Z }. Let ax(k) = SUPA,BI P(AB) - P(A)P(B) 1,whereA [XN- q^N(1 - a /2), XN - qN(a /2)] has asymptotic coverage E by generated { X,, n 1- a. Indeed, the theorem implies qN(l - a) -#,, and B varyovereventsin the a fields - a) in probability. < O} and {X, n ? k}. may be distribution Other bootstrapconfidenceintervalssimilarly to approximation thesampling The bootstrap valid in the sense of havingthe - ) is thedistribution N12(X N- XN), con- shownto be asymptotically of ofN112(XN a for coverage; example, simplepercentile correct asymptotic on ditional X1, ... ., XNt. methodor thebootstrap process In practice, is inevitable stationary X2, 2. Theorem LetX1, ... bea strictly thata data-basedchoice forp it R R function ( *) satisfying(0) + r IrR( r) I would be made. For example,as previously covariance with if mentioned, p < oo. Assume(8) in Theorem 1. Assume,forsome d > 0, is chosento minimizethemean squarederror -2 p,then of < oo. Then, p should satisfyN3PN - C. The constant C will depend on that EIX Id+2 < oo and >Lk [ax(k)]dl(2+d) if > Moreover, 0o O,then a2 givenin (3) is finite. say consistently, and can be estimated density the spectral by some sequence CN. One could then choose PN ( 1) 0. N2(X,_)?'x} - ?(x/a)I supX IP{N1 Theorem2 withsome additionaleffort, N 3113CN. In fact, to can be generalized considera data-basedchoice forp. function. where 1( *) is the standardnormal distribution Assume thatPN O- 0 and NPN-* oo. Then the bootstrap Subsequentworkwillfocuson a properchoice ofp. At this p in distribution the stage,it is clear that as long as p satisfies -* 0 and Np is distribution close to thetruesampling properties, -* oo, thechoiceof willnotenter intofirst-order p sense procedure. bootstrap of error, thestationary suchas coverage ? xl Xi, ... , XN} rate forp to tend to 0 will undoubtedly the right Getting N-XN) supl P{ N/1(X2 T "optimal" but properties, getting enterinto second-order 0 (12) x}I -P{N112(XN-u) Such consideration. will be a third-order correct constants is thoughof vitalimportance, beyondthe an investigation, in probability. secwork.A steptowardunderstanding scope ofthepresent was by properties presented Lahiri(1992) in the 1 and 2, the condition(8) is ond-order Remark4. In Theorems impliedbyEl Xi16e < oo and >Lk k2[a(k)][e/(6+e)I < oo. To case of movingblocksbootstrap. I see why, (A. 1) ofKiinsch( 1989). Hence theconappreciate 4. EXTENSIONS solelyin termsof ditionsforTheorem2 may be expressed without referring In thissectionwe extendtheresults Section3 to more condition, and condition moment a mixing in assume for some E > 0 that to cumulants.In summary, A of generalparameters interest. basic themeis thatresults by aboutthesamplemean readily are conditions implied El Xi16+e < oo. Thenthemixing for implyresults muchmore = O(k-r) forsome r condition ax(k) the single mixing statistics. complicated also implies>Lr IrR(r) I < oo. > 3(6 + E)/E. This condition The immediate applicationofTheorem2 lies in thecon- 4.1 Multivariate Mean for intervals ,u.For example,letqN(1 of struction confidence jth SupposethattheXi takevaluesin Rd, with component by distribution thebootstrap - ae) be obtainedfrom denoted by X1,1.Interestfocuses on the mean vector,,u . N( 1 -a)} =1-at. P{X N-XN-C jth component = E(X1,1) The definition ~,u = E(Xi ), having
C\N

1308

Journal of the American Statistical Association, December

1994

of ax( * ) readily appliesto the multivariate case. As before, the stationary is resampling algorithm the same, yielding a pseudo-multivariate time seriesX 1*,..., X N withmean X vector N*

propriately smooth function fromRP to Rq, and interest now focuseson the parameter,t = f(0). Assume that f = (f, . *. ,fq), wheref(y1, ... , yp)is a real-valued function from havinga nonzerodifferential (yi, . .. , yP) = (OI RP at .. ., Qp).Let D be thep X q matrix with(i, j) entry df(y1, Theorem3. Suppose, forsome e > 0, that El xi 6+e ... , yp))/dyj evaluatedat (Or,..p., Q). Then the following < oo. Assumethataxx(k)= O(k-r) forsome r> 3(6 + E)/ is true. E. ThenN12(XN -t) tends distribution themultivariate in to Gaussian distribution withmean 0 and covariancematrix 4. Suppose thatf satisfies aforementioned Theorem the X = (ai,j), where smoothnessassumptions.Assume that for some E > 0, 00 E[hj(X ) 6+-, <o0 and thatforsome r> 3(6 + e)/E,ax(k) = O(k-r). Then,ifPN -O 0 and NPN -0 oo, (14) and (15) aij = cov(X1,i, X1,j) + 2 cov(X1,J, X1+k,J). k= 1 hold. Moreover,
z

Then ifPN-* 0 and NPN-*

SUPS I P* {
IIXcN*

oo,

d(P{N1/2[f(ON) - f()] P{ IIXN - All - SI I

c x},
/[N(04) -f(ON)]

C X-NII SI}

O
(13) in probability and
SUp,I P { Lf(MN) -f(O)

P* {N1

< X})

in probability, where K is any normon Rd and P* refers to a probability conditional theoriginal on series. The immediateapplicationof the theoremis the construction joint confidence of for regions pt= (t1, ... *, d) Variouschoicesforthenormyielddifferent-shaped regions. Noticehoweasilythebootstrap handlestheproblem conof structing simultaneous confidence regions.An asymptotic approachwouldinvolvefinding distribution thenorm the of of a multivariate Gaussian randomvariablehavinga complicated(unknown)covariancestructure. The resampling approachavoids such a calculationand handlesall norms withequal facility.
*

1?

s 5}
P*{f(N)f( P

N)

?0s}I

-0O

in probability. As an immediateapplication,considerthe problemof uniformconfidencebands for (R(1), .... constructing R(q)), whereR(i) = cov(XI, XI+j). (To applytheprevious let theorem, Wi = (Xi, . . ., Xi+q), for1 < i N' = N- q.) Although even asymptoticdistribution theoryfor even Gaussian data seems formidable, stationary the bootstrap resampling approachhandlesthe problemeasily.The only caveatis to notethatq is fixed N -* oo. as 4.3 Differentiable Functionals

4.2

Smooth Function of Means

,

Again, suppose thatthe Xi take values in Rd. Suppose For simplicity, assume thatthe Xi are real-valuedwith that0 = (01, ... .,p), whereOj= E[ hj( Xi ) ]. Interest focuses common continuous distribution function Suppose that F. on 0 or somefunction 0. Let 0N = (ON,1,1 * * , fof where the parameter interest is some functional of F. A of T ,t of the sensibleestimate F is T(FN), whereEN is the empirical ON,j = i=1 hj(Xi )/N. Assume momentconditions of on conditions theXi. Then,bythemultivariate distribution X1, ..., XN. Assume that T is Frechetdifhj and mixing of case, the bootstrapapproximation the distribution to of ferentiable; is, supposethat that N12(ON - 0) is appropriately close in the sense hFd(G - F) + o( |G - F||), T( G) = T(F) + N) X) X-} 0 bN- 0) < x}, P* {N d(P { N' N(O-

forsome (influence) function centered thatf hF dF so hF, = 0. For concreteness, suppose that 11 11 the supremum d is in probability, whered is any metricmetrizing weak connorm,but thiscan be generalized. Then in vergence RP. Moreover, (14) d(P{N'12( N- 0) '

x}, P{Z ' x}) O-0,

(15)

N1/2[T(FN)

T(F)]

Z where is multivariate Gaussianwith mean0 and covariance matrix having( i, j) component X,

cov(Zi, Zj) = cov[hi (Xl ), hj (X)]
00
z

N-12

z
i=l1

hF(Xi) + o(N112'FN-F|).

(16)

+ 2

k= 1

cov[hi(X1), hi(Xl+k)].

Ifforsomed ? 0, E[hF(Xl )]2d < oo and lk [ax(k)] d(2?d), thenN`1/2 1i hF(Xi) is asymptotically normalwithmean 0 and variance E[h2(Xi)] + 2
00

cov[hF(Xl), hF(Xl+k)]. (17) To see why,define to be the vectorin RP with comjth Yi k=l1 ponent h1 (Xe). Then the Yi are weaklydependentif the original are weaklydependent; fact,agy(k)? aex(k). To handletheremainder in Xi term ( 16), Deo ( 1973) has shown in Hence,witha moment assumption thehi, we are exactly thatif zk k2[ax(k)]l/>-r < oo forsome 0 < T < 2, then on back in the multivariate case. Now supposethatfis an ap- N1 /2[FN() )F(.*) ], regarded a randomelementofthe as
z
-

Politisand Romano: The Stationary Bootstrap

1309

norm, schemeby Politisand Romano (1992a, 1992b). To apprewith supremum the endowed spaceofcadlagfunctions the consider problem of to weakly Z( *), whereZ( *) is a Gaussian process ciatetheapplicability thisapproach, converges of estimatingthe spectral densityf(w). Suppose that paths,mean 0, and havingcontinuous is theperiodogram at evaluated w basedon data TL,M,L(W) cov[Z(t), Z(s)] in fact,TN(w) is approximately equal to BartSi,M,L. Then, can lett'skernelestimateoff(w). Otherkernelestimators = E[gs(X1)gt(X1)] + I E[gs(Xl)gt(Xl+k)] of tapering the obtainedby appropriate be (approximately) k= I A estimates. greatadvantageofthe individualperiodogram approach is that it easily yields simultaneous resampling + I E[gs(X1+k)9t(X1)b oversome finite density for regions the spectral confidence k= 1 in gridof w values. Otherexamplesfalling thisframework whereasympmeasureand cross-spectrum, wheregt(x) = I[o,t](x)- F(t). Hence Deo's resultimplies are thespectral are distributions particuto normal with toticapproximations sampling that N1/2[T(FN) - T(F)] is asymptotically larlyintractable. mean 0 and variancegivenby (17). The bootstrap approximationto the distributionof 4.5 Future Work on conditional Xi, N1/2[T(FN) - T(F)] is thedistribution, problems. important workwillfocuson three Subsequent - T(FN)], where FN is the emXN, of N1/2 [ T(FN1) conuniform to results construct theoretical establish First, by ofX1,... , X Nobtained thestationary distribution pirical fidencebands forthe spectralmeasure.The discussionin If termsin the differential procedure. the error resampling confidence Section 4.4 will readilyallow one to construct thenit is clearthat of approximation T(FNJ) are negligible, gridofw values, measureovera finite bands forthespectral because Theorem 2 is will behave correctly, the bootstrap uniform By unsatisfying. constructing butthisis theoretically of negligibility essentially applicable.The keyto justifying confidence bands over the whole continuousrangeof w,a * 0 is terms to showp(N12[F error N(*)], Z(. Seccan procedures be established. basis forgoodness-of-fit weak con- ond, higher-order in probability, wherep is any metricmetrizing are especiallyto asymptotics necessary, space. By Theorem3, it compareprocedures, as in theiid case. Finally, pracin vergence the assumedfunction the just distributionsof ticalimplementation, is clear that the finite-dimensional the especially choiceofp, and thefinite of tightness is Z(.*). The onlytechnicaldifficulty showing In fact,by an argument the bootstrapempiricalprocess. can similar Deo's, tightness be shownif NpN -* oo. The to technical detailswillappear elsewhere. appliesif actually argument In fact, foregoing sketchy the example, For differentiable. T is only assumed compactly follows. for validity quantilefunctionals asymptotic 4.4 Linear Statistics Defined on Subseries
*N1/2[FN(.

F(*)] will appropriatelyconverge to those of

will based on simulations be addressed. samplevalidity 5. NUMERICALEXAMPLES FUR-THER

AssumethatXi E Rd. In thissectionwe discusshow the Xt- 1.352Xt-I + 1.338Xt-2 - .662Xt-3 + .24OXt-4 maybe appliedto yieldvalidinferences bootstrap stationary - .2Zt_j+ .04Zt-2, fora parameter E RD thatmay depend on the whole in,t = of finite-dimensional distribution theprocess. normal N(0, 1) random withthe Zt's being independent Consider subseries the 1, Si,M,L = (X(i-l )L?+ *...* X(i-)L+M) is of variables (and c = 0). A realization theYtseries exhibited the {Xi } by a "winThese subseries be obtainedfrom can M dow" ofwidth "moving"at lag L. Suppose thatTi,M,L is an estimateof,t based on the subseriesSi,M,L, so Ti,M,L R OM from dM to R D. Let = kM( Si,M,L) , forsome function + 1; here {[(N-M)/L]} TN= I T,M,L/Q,whereQ= t to 0 To function. applyresampling integer [ ] is the greatest of ( the approximate distribution TN,just regard T1,M,L, ...* Note thatM, L, TQ,M,L) as a time seriesin its own right. of and Q may depend on N. Weak dependenceproperties 12 90 18 80 14 080 into weak dependence the originalseriesreadilytranslate 0~~~~~~~~~ya back of properties thisnew series.Hence we are essentially yxTapns nulNmero Fiue3(h0ieSeiso is A into the sample mean setting. technicalcomplication so arrayof variables, thatwe are dealingwitha triangular By thatTheorem2 mustbe generalized. takingthisviewand consistency weak convergence point,one can establish Indeed,thisapproach bootstrap. of properties thestationary blocksresampling in has been appliedfruitfully themoving

In Figure3, thewell-known Canadian lynxdata are disin the played,representing numberof lynxtrappings the in theyears1821 to 1934; a histogram of MackenzieRiver Politis, and so notnormal. Leger, thedata reveals is skewed it and Romano (1992) analyzedthe Canadian lynxdata, towiththeartificial seriesYt,t = 1, . . . , 200, whereY, gether = X, IXII + c and theX, seriesfollows ARMA model the

1310
0 c\J

Journal of the American Statistical Association, December

1994

c'J O

0 O

0
0

10 Lag

20

30

50

100

150

200

TimeSeries Y. Figure4. TheArtificial

Figure6. Sample Autocovariance Y series. of

in Figure4; a histogram revealsthisdata is also not normal due to heavytails. In L'eger al. (I1992)constructed et confidence intervals for themeanoftheLynxseries wereconstructed usingthemoving blocks technique.They also discussedthe choice of b (and hencep = I /b forthe stationary bootstrap).The stationar bootstrap "hybrid" (i.e.,based on theapproximation P* { 7n( T*- T) ' x} - P{ Vn(t T- ) < x} ) 95% confidenceintervalfor the mean ,u of the Lynx data was withp [1,233.816, 1,832.7191,based on 500 replications = .05. (The sample mean of the Lynx data is 1,538.018.) This is remarkably close to the Moving Blocks 95% conficoJ dence interval [ 1,233.37,1,826.071 of et presented L'eger by al. (1992), whichwas again based on 500 replications with b = 25. Note thatin the stationary bootstrap simulation, p waschosen suchthatI lp - b, where choiceofb - 25 the 802 was explainedby Le'ger al. (1992). et But we mightalso considerthe median m of the Lynx data as the parameter interest. of The obvious estimator is thesamplemedian,whichwas equal to 771. But we need to attach standard a error confidence or interval thisestimate. to The stationary inbootstrap (i.e., "hybrid" 95% confidence ) due~~~~~~~~~~~ topheavyitails tervalforthemedian m of the Lynxdata was [242.5, 957], based on 1,000replications with = .05. p Turningto the artificial series,it was mentioned that Y, (andhenep. T he stationary Theanc sta-ate o bootstrap ) the distribution Y,, forsome fixedt, is non-Gaussian. of

trimmed meanswouldbe moreefficient thesample than meanforestimation thelocation of parameter because c, thetwo-sided distribution 1 degree freedom X2 with of can be thought as being"close" to thedoubleexponential of distribution, closeas theX2 distribution 1 degree (as with of freedom totheX2 distribution 2 degrees freedom). is with of Forthesimulation, constant wassetto 0, and sixdifthe c ferent estimatorsc were of considered: sample the mean, the and means(i.e.,themeanofthe median, thea-trimmed observations throwing the[na] largest remaining after away andthe[na] smallest a ones),with = .1, .2, .3,and .4. the First, problem choosing for stationary of p the bootTo strap. do this, lookat thesample meancase,for which a simple expression thevariance of exists: >Y, var 1:y vr200 i= i
=-200 1 200

\200~~~~~_ /0
(var(Yi) 200 + 2
200~00

cov(Yl, Yl+)

The stationary bootstrap estimates thevariance the of of meanfor sample different of choices p E (0, .8) arepictured in Figure andthesample 5, autocovariance sequence the of is in 6. Y,series pictured Figure It is seenthat autocothe variances lags for greater 6 arenotsignificantly than different x2 it with Indeed, is a two-sided distribution I degree of from Thiswould 0. leadtoan empirically acceptable choice centered symmetric and aroundtheconstant By of b for moving freedom, c. the blockmethod theorder 10 (see of of analogyto the iid case, it is expectedthatthe median and Leger, Politis Romano1992).Bytheapproximate and correspondencethe of moving blocks method the and stationary with bootstrap p = 1/b,thechoice p = .1 is suggested. of Having decided usep = .1,letus proceed comparing to in thesixproposed estimators c. Basedon 500 stationary of bootstrap replications, Table 1 reports stationary the bootstrap estimate variance thecorresponding of of estimator,
at/
a

Table 1. Trimmed Mean Confidence Intervals

astat.bootstrap

95% confidence interval

.1 .2 .3 .4 .5

.0386 .0159 .0094 .0050 .0028

.1034

[-.089, .662] [-.092, .413] [-.030,,345] [-.080,.202] [-.082, .105]

[-.282, .984]

Politisand Romano: The Stationary Bootstrap

0 0 (.0 0 0

1311

C14
o)

III
0.0

Elm..
0.1 0.2 0.3 0.4 0.5

-1.0

-0.5

0.0

0.5

1.0

of Figure7. BootstrapDistribution the Y Series Sample Mean.

of Figure10. BootstrapDistribution the Y Series Sample Median.

be can means,thebootstrap further shownto be a for interval thepa- trimmed confidence as wellas the95%bootstrap in estimators the notation, meanandthemedian viable methodof choosingamong competing c. rameter Forcompact a with equalto 0 and .5. an adaptivemanner(see Legerand Romano 1990a,b). means, as trimmed were denoted the suggesting the that intuition the from table Itis obvious the Indeed, seemsto be correct. APPENDIX: PROOFS as median mostefficient bootby (estimated thestationary hasthesmallest median of interval Proof Lemma 1. confidence the and variance, yields shortest strap) to taken equal 0 in thissimulation. In theproof, expectations covariances conditional c that was for recall c; on are and all be should preferred.XI, . . ., XN.Recall L1 in theconstruction thestationary the reasoning, median to According this resamof histo- plingscheme.Then bootstrap 7 10, In Figures through thestationary of distribution thesamplemean,thea .1 of grams the E(X 1X *+ I L1 > i)P(LI > i) E(X 1X *j)= of and thesamplemedian the Y, means, and .3 trimmed and replications bootstrap basedon 1000 are series pictured, + E(X *X+1 L1 ? i)P(LI ? i) = .1. The bootstrap median is of distributionthesample p N theory Basedon theasymptotic the clearly leastdisperse. + fC2[I _(I -p)i]. =N-1 z Jjl_p)i to approximations each of the the justifying bootstrap J=l
Hence cov(X *, X *+) = CN(i)(l X * series,
0
Lfl -

p)i. So, by (2) applied to the

UN,p =

CN(O) + 2

(1

N)(1 N
=

-P)1CN(i), CN(O),

RN(O) that (5). yielding To get(6), note

0

1,thenCN(z)

= RN(i)

+ RN(N

by i). Therefore, (5),

andif1 ? i ? N

00

|. lll mEm IIIiiEEE

-0.2 0.0 0.2 0.4 0.6 0.8 1.0

the = Letting] NN-i in thelastsumyields result.

Proof of Theorem 1.

Mean. of Figure8. BootstrapDistribution the Y Series .1 Trimmed

that we of Forpurposes theproof, mayassume E(Xi )

0. Let
(A.1)

cnJ

00

-0.1

. IIIIIIwmml ...|1iIiII
N-I

+ S,pN = RN,O(O) SN=

2 1 bN(i)RN0P(i)i
i=lI N-1

By where RN,O(i) = oJfh XjXr+o/N. (5),

aN

SNPN

RN0P-XN-

2X bN

bN( i).

0.0

0.1

0.2

0.3

0.4

XN = , (N"2). Also, {%hjl bN(i) ? 2/ Under theassumptions, Oj to X PN,which implies Nz N+l bN( i) = Op( 1). Hence itsuffices show To inprobability. ac(A.1) satisfies /2S e SN thestimatorin o

Mean. ofthe Distribution Y Series.3 Trimmed 9. Figure Bootstrap

of the this, complish we show biasand variance SNtendto 0. By that itfollows (7) andE[RN,o(i)I = (N -i/N)R(i),

1312

Journal of the American Statistical Association, December

1994

E(SN)

R(O) + 2

1 2'j( 1+ 2

pN)R(i)

(C1) and NPN

var(SI,R)

0 as well.

o, N-1E[var(SI,R)IRI

0, yieldingN1
mN

Step 2.

-k)k(1

-PN)R(i).

NpN/mN

-*

Show thatforany fixedsequence m = 1, thedistribution of

N 1/2(
EN M

satisfying (A.3)

tendsto the normaldistribution withmean 0 and variance .2. First,note that E(SI,,L,) = XNPN. For 1 < i < MN, let YNj /T/2 N-I N-1 =M1/2 TheE /ndAI\ = . Then (A.3) is mN [ YmN ,L /N1/2 ME(YmN)I, and YmN 2 , R(i) - 2PN , iR(i) + o(PN). (A.2) = I= YN I/MN is the averageof iid variables.But, as in step 1, i=1 i=1 var(YN,j)is the same as the varianceof MN/N timesthe variance Hence E(SN) = a2 + O(PN). To calculate the varianceof SN, ofSI,R, whereI is uniform { 1, . . . , N} and R is geometric on with by the result(5.3.21) of Priestley (1981) originally due to Bart- meanPN. Again,applytherelationship

The absolute value of the last term is bounded above by 2 IiR(i) I/N = O (N-'). To handle the first summation, use the 1 - ipN to getthistermis approximation - pN) (1

NXN

lVkJ~NmN

NPN/

lett(1946), Cov[RN,O(i), RN,O(I)] 2M=-co IR(m)I +K. Now,

N-1 N-1

S/N,

where S = 2R(O)

var(SI,R) = E[var(SI,RI R)

+ var[E(SI,R

I R)

(A.4)

var(SN,p)

b bN(i)bN(j)cov[AN,o(i) s-N-1 i=-(N-1) N1S j=-(N-1)

RN,o(j)]

The second termon the right side of (A.4) is var(RXN) = XN(1 R = r) is in factgiven -PN)/PN -* 0 by (C1). Also, r'-var(SI,RI in by MN,r defined (10). Thus
mN

<_ N
ifNPN
->

bN(i) bN(W <

- '

N
PN(l PI N N -PN)

var(S1,R)

= -E(Rm2,R)

+ ?(1) (1-PN)CN(i)+O(1).

oc andPN

->

0. Thustheresult proved. is

Proof Theorem2 of

NPN

CN(O) +

NPN i=l

loss Without ofgenerality, assume = 0. Theresult11)follows ,u ( 2. To completestep2, by Katz's (1963) Berry-Esseen bound, immediately corollary ofHallandHeyde from 5.1 (1980).To prove it suffices showthat to ( 12), fornow assumethefollowing three hold for the convergences EI m /2 YNi - E(YN,i)I2+1 _>0 (A.5) sequence X2....* Xi,
(C2)

Bytheassumption thatvar(YN,j) NpN/mN -* 1 and (C2), itfollows

(C 1)

NX N/ (NpN) -_0. CN(O) + 2 z-= 1 (I -PN)

PN o
__

C()
2+6

as mN -- oo. But the leftside of (A.5) is (by conditioning R) on equal to

MN

(C3) N'+12

Si,r -

(1

- PN)

PN

. 0-

N'+6/2

EN

SI,R

12+5

N 25,r -2P)PN
r

MN iN nN1

S j

r - )r-1PNX

In (C3), Si,b is defined be thesumofobservationsBi,b to in defined in(1).

Xi,

whichtendsto 0 by (C3).

-pN

showthattheconvergence (C1), (C2), and (C3) holdin probability and J = LM + J,. Then EN,M-X is just N1 for original the sequenceX1,X2 .... First, (C1) holdsin probability times sumoftheobservations BIM,LM, deleting first the in after the because N"12XNis order 1 in probability NPN -> oo. Second, and N - J,ofthem. R1bethe Let exact number observations of required (C2) holds in probability an argument by very so from blockBIM,LM that observations theM blocks N from have the convergence similar Theorem1.Finally, showthat to to (C3) holdsin probability, beensampled; is,R1 = N - J,.Also, R = LM - R1. Note that let the thatR, conditional (R1, J,), has a geometric on with write termin questionas distribution
=

Theproof this of claim be given five will in In steps. theproof, This essentially followsby an extensionof Theorem 7.3.2 (to a all calculations to referringthis will bootstrap distribution be as- triangular of arraysetting) Chung (1974). In our case, M = NPN sumedconditional XA,..., XN. Set EN,m = (SI1,L1 + * * - + Op(N'1l2p 1/2) on + 2N + SI,Lm)/N, where II, I2, ... areiiduniform {1, .. ., N} the on Step 5. Combine steps 1 and 4 to provetheclaim. andtheL1, L2, . . . areiidgeometric mean1/PN. LetM be with Now to deduce (12), by a subsequenceargument suffices it to thesmallest m L1 integer suchthat + *** + Lm> N. Also, J, let
L1 +
.. . LM-1

(C2), and (C3).

Claim. The distribution N'"2(X -XN), conditional of on Step 4. The distribution N"12(EN,M - XN) tendsto normal of .. ., XN, tends weakly thenormal to distribution mean0 withmean 0 and variance U2 . To see why,ifM is any random with U2 and variance , for every sequence X2,... satisfying (C1), variable (sequence) satisfying Xi, then M/NpN -* 1 in probability,
N' 2(EN,M - XN) tends to normal with mean 0 and variance a.

of Step 3. The distribution N"12(EN,MN - XN) tendsto normal withmean 0 and variance U2 . This follows step2 and (C1). by

It suffices showthat(A.6) raisedto thepower(2 + 5)'- tendsto to 0 in (conditional) 0 in probability, which Minkowski's by inequality boundedabove is probability. theforegoing By observation, enough show itis to that by themeanand variance N`'/2SI,R tends 0. But E[SI,R I RI of to Pn 1/(2+6) = RXN, SO that N-12E(SI,R) = N1/2XN/(NPN)-> 0. Now, ) [El SI R - RXN I1]' ( (N51/2 Step 1. Show that N'/2(ENM
- XN

meanI/PN. Thisfollows from "memoryless" the property the of distribution. geometric HenceEN,M - X N is equalin distribution toN-'SI,R, where is uniform { 1,. . . N}. I on
)
-*

N6/2

PN E|SI,R SIR

PN

(A.6)

N-'var(SI,R)

N1E[var(SI,RI
=

R]

But var[E(SI,RIR)I

var(RXN) =

+ N-'var[E(SI,RI R)]. Thus,by XN(1 - PN)/PN.

+(

P~N5)

XNE[I R - PN 12+511/(2+5).

(A.7)

Politisand Romano: The Stationary Bootstrap

1313

A DZD'; see theorem of Serfling (1980, in The secondterm (A.7) is oforder XNN'12[NPN] -(1+5)/(2+5), which 0 and covariancematrix p. 122). It to tendsto 0 in probability. now suffices show

El N612

PNp'

[ReceivedApril1992. RevisedApril1993.]
S,R RXNI

- 0

REFERENCES
Propof Specification Sampling M. Bartlett, S. (1946), "On theTheoretical of Time Series," Journal theRoyalStatistical ertiesof Autocorrelated 8, Supplement,27-41. Society San and DataAnalysis Theory, Francisco: Series: D. Brilhinger, (1981), Time Holden-Day. (2nd in Theory ed.), New York: Chung,K. (1974), A Course Probability AcademicPress. SeDeo, C. (1973), "A Note on EmpiricalProcessesof Strong-Mixing 1, of quences,"Annals Probability,870-875. Efron,B. (1979), "BootstrapMethods:AnotherLook at the Jackknife," 7, of TheAnnals Statistics, 1-26. and Application, Limit Theory its Hall, P., and Heyde,C. (1980), Martingale New York: AcademicPress. of Theorem,"Annals MathKatz, M. (1963), "Note on the Berry-Esseen Statistics, 1107-1108. 34, ematical for and Kiinsch,H. R. (1989), "The Jackknife the Bootstrap GeneralSta17, of The Observations," Annals Statistics, 1217-1241. tionary for by Correction Moving Block Bootstrap Lahiri,S. (1992), "Edgeworth the of Data," in Exploring Limits Bootstrap, and Stationary Nonstationary New York: JohnWiley. eds. R. LePage and L. Billard, Technology and D., Leger,C., Politis, and Romano, J. (1992), "Bootstrap 34, Technometrics,378-398. Applications," Choice of TuningParamLeger,C., and Romano, J. (1990a), "Bootstrap 4, Mathematics,709-735. of Annal the of Institute Statistical eters," Mean ExThe Estimation: Trimmed Adaptive (1990b), "Bootstrap 18, Journal Statistics, 297-314. of ample," Canadian and K. Liu, R. Y., and Singh, (1992), "MovingBlocksJackknife Bootstrap eds. the of CaptureWeak Dependence,"in Exploring Limits Bootstrap, New York: JohnWiley. R. LePage and L. Billard, Politis,D., and Romano, J. (1992a), "A GeneralResamplingScheme for to of Arrays a-MixingRandom VariablesWithApplication Triangular TheAnnals Statistics, of the Problemof SpectralDensityEstimation," 20, 1985-2007. for Procedure Multivariate Resampling (1992b), "A Nonparametric and in Science Confidence Regionsin Time SeriesAnalysis," Computing eds. on of Proceedings the 22nd Symposium theInterface, C. Statistics, pp. Page and R. LePage, New York: Springer-Verlag, 98-103. (1992c), "A CircularBlock ResamplingProcedureforStationary eds. of the Data," in Exploring Limits Bootstrap, R. LePage and L. Billard, New York: JohnWiley,pp. 263-270. Bands Confidence Politis,D., Romano, J.,and Lai, T. (1992), "Bootstrap on IEEE TransactionsSignal Processing, and Cross-Spectra," for Spectra 40, 1206-1215. New York: Acand Series, Analysis Time M. Priestley, B. (1981), Spectral ademic Press. and Spectral Mixing, Strong Normality, M. Rosenblatt, (1984), "Asymptotic 12, Annals Probability, 1167-1180. of DensityEstimates," Boston: Birkand Fields, Sequences Random (1985),Stationary hauser. Statistics, Theorems Mathematical of R. Serfling, (1980), Approximation New York: JohnWiley. MixingSequences," Yokoyama,R. (1980), "MomentBoundsforStationary

tendsto 0; thatis, or in probability, thatitsexpectation

z PNl+62: To bound El Si,r

E[ I Si,r- rXNI2+1,

rXNI

](1 -PN)
<

PN

0.

(A.8)
<

then Yokoyama's (1980) moment inequalityapplies, yielding K El Si,rI2+5 <Kr'+(512- , wherethe constant dependsonlyon the mixingsequence { a(k) }. Thus, by Minkowski'sinequalityand we thenYokoyama's inequality, have
El
Si,r -

note that if 1

<

i + r- 1

N,

rXNI2+6 ?< [K1(2+6)r1+(/2)][1/(2+?)]

I2+6 + (E IrXNJ )2+)1/(2+b)

< [K'/(2+6rY'+(b/2)][1/(2+b)] + -rKN +(/2)][1/(2+5)112+6 N
< (2K)2+br( 1+6)/2.

In the case + (Xi + *

i + r-

to koyama'sinequality getEl Sir I < 22+'Kr' 652). Then, arguing 2+6 El we as earlier, find IS,r - rXNI < (3K)2+r'1+(6/2). In thegeneral case, suppose that r + N(j - 1) + r, where 1 < r < N. Then Si,r
=

+ XN) 1 > Nbut r < N, write Si,r = (Xi + * + XL+r-I -N). Apply Minkowski's inequality and Yo-

(j-

1)NXN

+ Si,Pr SO

E I Si,r-

rXNI

El

S,,F

rXN

I,

applies. Hence (A.8) is and the general bound (3K)2+r'1+(6/2) boundedabove by

PN
00

(3K)25r' ya z N52r=1

+/( Nr-1p PN /12(1 - PN)

(PN st/2 kN

PN

1+

1
2)

o(

Proofof Theorem 3 linearcombinaby immediately considering The prooffollows Theorem2, whichis appliand applying tionsof thecomponents mapping by cable by Remark4. Then (13) follows thecontinuous continuouswith theorem(because a norm is almost everywhere to respect a Gaussian measure). Proofof Theorem 4

52, Gebiete, 45undVerwandte ZeitschriftWahrscheinlichkeitstheorie fur Theorem from as follows (14) and (15) are immediate The proof 57. on assumptions f implythat N'12[f( N) Zurbenko, G. (1986), TheSpectral 3, and the smoothness Amsterdam: Series, of I. Analysis Time with North-Holland. Gaussiandistribution mean multivariate -f() ] hasa limiting