Empirical Rank Processes for Detecting Dependence

Les Cahiers du GERAD ISSN: 0711–2440 Empirical Rank Processes for Detecting Dependence Christian Genest Bruno Rémillard G–2002–03 January 2002 Les textes publiés dans la série des rapports de recherche HEC n’engagent que la responsabilité de leurs auteurs. La publication de ces rapports de recherche bénéficie d’une subvention du Fonds F.C.A.R. Empirical Rank Processes for Detecting Dependence Christian Genest Département de mathématiques et de statistique Université Laval Québec (Québec) Canada G1K 7P4 [email protected] Bruno Rémillard Service de l’enseignement des méthodes quantitatives de gestion and GERAD École des Hautes Études Commerciales 3000, chemin de la Côte-Sainte-Catherine Montréal (Québec) Canada H3T 2A7 [email protected] January, 2002 Les Cahiers du GERAD G–2002–03 c 2002 GERAD Copyright ° Abstract Exploiting an overlooked observation of Blum, Kiefer & Rosenblatt (1961), Dugué (1975) and Deheuvels (1981a) described a decomposition of empirical distribution processes into a finite sum of asymptotically mutually independent terms whose limiting distribution is simple under the hypothesis that a multivariate distribution is equal to the product of its marginals. This paper revisits this idea and proposes to test independence using a combination of Cramér-von Mises statistics arising from the decomposition of the empirical copula process, which involves only the ranks of the observations. Asymptotic and finite-sample tables of critical values are provided for carrying out the test, based on Fisher’s method of combining P -values. While the new statistic is inferior to the standard likelihood ratio test for multivariate normal data, Monte Carlo simulations show that it can be much more powerful than the latter when the marginal distributions of the data or their underlying dependence structure are non-normal. Using the canonical decomposition of Dugué and Deheuvels, a graphical device called a “dependogram” is also proposed which helps identify the dependence structure when the null hypothesis is rejected. The mathematical exposition, which is based on recent work of Ghoudi, Kulperger & Rémillard (2001), allows for a simultaneous treatment of the serial and non-serial case. It is shown, among other things, that the asymptotic distribution of rank statistics based on the empirical copula process is the same in both cases, thereby shedding new light on the theory of nonparametric tests of serial dependence initiated by Hallin, Ingenbleek & Puri (1985). Keywords: Copula, Cramér-von Mises statistic, empirical process, Möbius inversion formula, pseudo-observations, semi-parametric models, serial dependence, tests of independence. Acknowledgments: Partial funding in support of this work was provided by the Natural Sciences and Engineering Research Council of Canada, the Fonds pour la formation de chercheurs et l’aide à la recherche du Gouvernement du Québec, the MITACS Network of Centres of Excellence and by the Institut de finance mathématique de Montréal. Résumé Exploitant une importante remarque de Blum, Kiefer & Rosenblatt (1961) qui a été négligée par la suite, Dugué (1975) and Deheuvels (1981a) décrivent une décomposition d’un processus empirique en une somme finie de termes qui sont asymptotiquement indépendants et dont la limite s’exprime simplement, sous l’hypothèse que la fonction de répartition multivariée est le produit de ses marginales. Cet article jette un nouveau regard sur cette idée et propose de tester l’hypothèse d’indépendance en utilisant une combinaison de statistiques du type Cramé-von Mises provenant de la décomposition de la copule empirique, qui s’écrit en fonction des rangs des observations. Des tableaux pour les valeurs critiques d’un test basé sur la méthode de Fisher pour la combinaison de probabilités critiques sont donnés, dans le cas de petits échantillons ainsi que dans le cas limite. Comme on pouvait s’y attendre, le nouveau test proposé est inférieur au test du rapport de vraisemblance dans le cas multivarié gaussien. Des simulations Monte Carlo montrent qu’il peut être beaucoup plus puissant lorsque les lois marginales ou la copule ne sont pas gaussiennes. Utilisant la décomposition canonique proposée par Dugué et Deheuvels, un outil graphique appelé “dépendogramme” est proposé afin de permettre l’identification de structures de dépendance lorsque l’hypothèse nulle est rejetée. La méthodologie mathématique, basée sur le récent article de Ghoudi, Kulperger & Rémillard (2001), permet le traitement simultané des cas sériels et non sériels. En outre, il est montré que la loi asymptotique de statistiques de rangs basées sur la copule empirique est la même dans les deux cas, expliquant ainsi les résultats de la théorie des tests non paramétriques dans le cas de séries chronologiques initiés par Hallin, Ingenbleek & Puri (1985). Les Cahiers du GERAD 1 G–2002–03 1 Introduction In a seminal paper concerned with testing the null hypothesis of independence between the p ≥ 2 components of a multivariate vector with continuous distribution H and marginals F1 , . . . , Fp , Blum, Kiefer & Rosenblatt (1961) investigated the use of a Cramér-von Mises statistic derived from the process   p  Y √  Fnj (tj ) , t = (t1 , . . . , tp ) ∈ IRp Hn (t) = n Hn (t) −   j=1 that measures the difference between the empirical distribution function (EDF) H n of H and the product of the marginal EDFs Fjn associated with the p components of the random vector. As Hoeffding (1948) had already noted, the asymptotic distribution of this test is generally not tractable, and hence tables of critical values are required for its use. Such tables were provided by Blum et al. (1961) themselves in the case p = 2, and were later expanded to p ≥ 3 by Cotterill & Csörgő (1982, 1985), based on strong approximations of Hn . See also Jing & Zhu (1996) for a bootstrap approach. Despite this work and the anticipation that the Cramér-von Mises statistic Z H2n dHn should be powerful, most subsequent research focussed on the case p = 2, where alternative tests (typically based on moment characterizations of independence) were proposed by Feuerverger (1993), Shih & Louis (1996), Gieser & Randles (1997) and Kallenberg & Ledwina (1999), among others. Curiously, the literature seems to have largely ignored a suggestion of Blum et al. (1961) to circumvent the inconvenience caused by the complex nature of the limiting distribution of Hn . To be specific, let X1 = (X11 , . . ., X1p ), . . ., Xn = (Xn1 , . . . , Xnp ) be a random sample from distribution H, and for arbitrary A ⊂ Sp = {1, . . . , p} with |A| > 1, consider the empirical process n 1 XY {I (Xij ≤ tj ) − Fnj (tj )}. GA,n (t) = √ n i=1 j∈A Using Möbius’ inversion formula (cf., e.g., Spitzer 1974, p. 127), Blum, Kiefer and Rosenblatt showed that Hn may be conveniently expressed as Y X Fn,j (tj ). GA,n (t) Hn (t) = A⊂Sp , |A|>1 j∈Sp \A Although their paper only discussed the case p = 3, these authors claimed (and this was later confirmed by Dugué 1975) that under the hypothesis of independence, G A,n converges weakly to a continuous centered Gaussian process with covariance function Y covA (s, t) = [min{Fj (sj ), Fj (tj )} − Fj (sj )Fj (tj )] j∈A Les Cahiers du GERAD G–2002–03 2 whose eigenvalues, given by π 2|A| (i 1 , 2 1 · · · i|A| ) i1 , . . . , i|A| ∈ N = {1, 2, . . .} may be deduced from the Karhunen-Loève decomposition of the Brownian bridge. More importantly still, Blum et al. (1961) and Dugué (1975) pointed out that the processes G A,n and GA′ ,n are mutually independent asymptotically whenever A 6= A′ , so that Cramér-von Mises statistics based on the individual GA,n ’s could be combined to construct suitable statistics for testing against independence. An obvious limitation of tests based on this approach, however, is the dependence of the asymptotic null distribution of the GA,n ’s on the marginals of H. To alleviate this problem, Deheuvels (1981a) suggested that the original observations X1 , . . . , Xn be replaced by their associated rank vectors R1 = (R11 , . . . , R1p ), . . . , Rn = (Rn1 , . . . , Rnp ). He then went on to characterize the asymptotic behavior of a Möbius decomposition of the copula process   p  Y √  Cn (t) = n Cn (t) − tj , (1)   j=1 where n Cn (t1 , . . . , tp ) = p 1 XY I(Rij ≤ ntj ) n i=1 j=1 is an estimation of the unique copula C defined implicitly by C{F1 (t1 ), . . . Fp (tp )} = H(t1 , . . . , tp ). The latter reduces to C(t1 , . . . , tp ) = t1 · · · tp under the null hypothesis of independence. Although Cramér-von Mises and Kolmogorov-Smirnov statistics based on C n ’s decomposition were then proposed by Deheuvels (1982a,b,c), he did not compute the quantiles for the asymptotic or finite-sample distribution of these statistics; nor did he mention how the 2p − p − 1 statistics derived from the rank analogues of the GA,n ’s could be combined to obtain a global statistic for testing independence. The first objective of this paper is to fill this gap. In Section 2, unbiased variants of Deheuvels’ rank-based processes GA,n are introduced, and their asymptotic theory is briefly recapped. Two strategies for testing the null hypothesis of independence are then described in Section 3. Since the Cramér-von Mises functional of the GA,n has a simple limiting distribution that depends only on the size of A, one possibility is to fix a global level for the test and to check for lack of independence in each subset A of variables at appropriate subsidiary levels. A table of 5% critical values is provided for this procedure, which allows for visual determination of specific violations of the independence assumption. A second option consists of combining the p-values of the 2p − p − 1 tests into a single statistic that is asymptotically distributed as a chi-square random variable with that many degrees of freedom. As shown in Section 6, this statistic turns out to be quite powerful in Les Cahiers du GERAD G–2002–03 3 a variety of settings, using as a benchmark the classical likelihood test statistic based on the multivariate Gaussian model. The second goal of the paper, pursued in Section 4, is to show how these developments can be adapted easily to test against randomness in a time series context. Indeed, if vectors Xi = (Yi , . . . , Yi+p−1 ) , 1≤i≤n−p+1 (2) are constructed from p consecutive values of a stationary univariate time series Y 1 , Y2 , . . ., recent results of Ghoudi, Kulperger & Rémillard (2001) imply that under the null hypothesis of white noise, the serial and non-serial versions of the empirical rank processes G A,n have the same asymptotic behavior when restricted to subsets A with 1 ∈ A and |A| > 1. In other words, the fact that two successive values of Xi have p − 1 components in common (shifted by one) has no impact, in the limit. Given that many of the linear rank statistics traditionally used for testing independence are simple continuous functionals of the G A,n ’s, it is not surprising, therefore, that serialized and non-serialized versions of Spearman’s rho and other rank statistics (Laplace, van der Waerden, Wilcoxon, etc.) share the same limiting distribution. This point, which is made in Section 5, sheds new light on earlier results of Hallin, Ingenbleek & Puri (1985) obtained in the case |A| = 2. For a general survey of the literature on rank tests of independence, see Hallin & Puri (1992) or Hallin & Werker (1999). For specific tests of serial dependence based on the Blum-Kiefer-Rosenblatt approach, refer to Skaug & Tjøstheim (1993) or Tjøstheim (1996). Other recent contributions are those of Kulperger & Lockhart (1998), Hong (1998, 2000), and Genest, Quessy & Rémillard (2001). 2 Set-indexed decomposition of the copula process Let X1 , . . . , Xn be independent random vectors in IRp with continuous distribution function H and marginals Fj , 1 ≤ j ≤ p. Let also R1 , . . . , Rn be the associated ranks vectors. A simple application of Möbius’ formula implies that the empirical copula process Cn may be expressed in the form X Y Cn (t) = GA,n (t) tj A⊂Sp , |A|>1 j∈Sp \A in terms of set-indexed processes 1 GA,n (t) = √ n  p n Y X i=1  j=1  p  Y I (Rij ≤ ntj ) − tj  j=1 defined for arbitrary A ⊂ Sp with |A| > 1. Following Blum et al. (1961) and Dugué (1975), Deheuvels (1981a,b,c) proved that under the null hypothesis of independence, G A,n converges weakly to a continuous centered Gaussian process GA with covariance function Y ΓA (s, t) = {min(sj , tj ) − sj tj } . (3) j∈A Les Cahiers du GERAD G–2002–03 4 Furthermore, he showed that GA,n and GA′ ,n are asymptotically independent whenever A 6= A′ , so that Cramér-von Mises or Kolmogorov-Smirnov statistics based on the G A,n ’s could be combined into a global test of independence. It is worth pointing out in passing that if ½ tj if j ∈ B, B t = 1 if j 6∈ B, the asymptotic processes may actually be expressed as X ¡ ¢ Y GA (t) = (−1)|A\B| C tB tj B⊂A j∈A\B in terms of the limit C in D([0, 1]p ) of the continuous functional Hn {F1−1 (t1 ), . . . , Fp−1 (tp )} of the Blum-Kiefer-Rosenblatt process that would be obtained by applying the probability integral transformation to the marginal distributions of the Xi ’s, if they were known. Alternatively, C in the above expression can be replaced by the limit in D([0, 1]p ) of (1), viz. p X Y tk (4) C̃(t) = C(t) − βj (tj ) j=1 k6=j which Deheuvels (1979, 1980), Stute (1984) and Gänssler & Stute (1987) showed to be a continuous centered Gaussian process involving p independent Brownian bridges β 1 , . . . , βp connected to C through the relations β1 (u) = C(u, 1, . . . , 1), . . . , βp (u) = C(1, . . . , 1, u). The occurrence of this linear combination of Brownian bridges in the limiting process (4) is a consequence of the fact that these marginals are not known. Section 3 describes how it is possible to exploit the asymptotic independence of the GA,n ’s to construct diagnostics and powerful tests of independence. In order to ensure the quality of the asymptotic approximation in small samples, however, it is actually preferable to work with zero-mean versions of the GA,n ’s defined by n G⋆A,n (t) 1 XY [I {Rij ≤ (n + 1)tj } − Un (tj )] , =√ n i=1 j∈A where Un is the distribution function of a discrete random variable U uniformly distributed on the set {1/(n + 1), 2/(n + 1), . . . , n/(n + 1)}, i.e., ½ ¾ [(n + 1)u] Un (u) = min ,1 (5) n Les Cahiers du GERAD 5 G–2002–03 with [(n + 1)u] standing for the integer part of (n + 1)u. The limiting behavior of the processes G⋆A,n is given in the following extension of Theorem 2 in Deheuvels (1981a). Proposition 1. Under the null hypothesis of independence, the processes G ⋆A,n defined for arbitrary A ⊂ Sp with |A| > 1 converge weakly to the mutually independent continuous centered Gaussian processes GA with covariance function (3). 3 Tests statistics for independence As is well known, p events E1 , . . . , Ep are mutually independent if and only if for all A ⊂ Sp = {1, . . . , p}, à ! \ Y P P (Ei ). Ei = i∈A i∈A Likewise, a distribution H is equal to the product of its marginals if and only if its associated copula C verifies the condition X Y (−1)|A\B| C(tB ) tk = 0, (6) B⊂A k∈A\B for all t ∈ [0, 1]p and A ⊂ Sp , as proved in Ghoudi et al. (2001, Proposition 2.1). Condition (6) is referred to as A-independence by Deheuvels (1981a). Now since it is generally true that à ! Y X Y Y xi (xi + yi ) = yj , i∈A B⊂A i∈B j∈A\B it follows at once from the ergodic theorem that for all t ∈ [0, 1]p ,   ¾ ½ ⋆ Y GA,n (t) √ = E [I {R1j ≤ (n + 1)tj } − Un (tj )] E n j∈A   X Y Y = (−1)|A\B| E  I {R1j ≤ (n + 1)tj } Un (tk ) B⊂A j∈A\B k∈A\B tends to the left-hand term of (6). It seems reasonable, therefore, to reject the hypothesis of independence whenever the process G⋆A,n is significantly different from zero for some A ⊂ Sp . This suggests basing a test of independence on a combination of statistics involving all the G⋆A,n ’s. For a fixed A, two obvious possibilities are the Cramér-von Mises statistic Z {G⋆A,n (t)}2 dt, (7) TA,n = [0,1]p Les Cahiers du GERAD 6 G–2002–03 and the Kolmogorov-Smirnov statistic SA,n = sup |G⋆A,n (t)|. (8) t∈[0,1]p Although the null distribution of these rank-based statistics can easily be tabulated for any sample size, this paper focuses on the use of the TA,n ’s because of their convenient asymptotic distribution, given below. See Deheuvels (1981a, Section 3) for a proof of this result. Proposition 2. Under the null hypothesis of independence, the limiting distribution of TA,n is the same as that of ξ|A| , where ξk = X (i1 ,...,ik )∈Nk π 2k (i 1 Zi21 ,...,ik , 2 1 · · · ik ) and the Zi1 ,...,ik ’s are independent N (0, 1) random variables. In the light of the above discussion, a reasonable testing procedure consists of rejecting independence whenever the observed value tA,n of at least one of the TA,n ’s is too large. There are two natural ways to go about this: either critical values could be chosen for each of the statistics so as to achieve a predetermined global level, or the p-values of these statistics could be computed and then combined. The first option, considered in Section 3.1, leads naturally to a diagnostic test that makes it possible to identify failures of the independence assumption. As will be seen, however, the alternative test described in Section 3.2 turns out to be more powerful. 3.1 Fixing a global level One option consists of rejecting the null hypothesis of independence at approximate level α if TA,n = tA,n > cA for some A ⊂ Sp , where cA = c|A| is chosen so that  P [ A⊂Sp , |A|>1  {TA,n > cA } ≈ 1 − = 1− Y A⊂Sp , |A|>1 p Y k=2 ¡ ¢ P ξ|A| ≤ c|A| p {P (ξk ≤ ck )}(k) = α, where the first identity is justified by the fact that the limiting distribution ξ|A| of TA,n depends only on the size of A and TA,n ≡ 0 for |A| ≤ 1. Les Cahiers du GERAD G–2002–03 7 Table 1. Critical values c2 , . . . , cp for the global test of independence at level α = 5% for a p-variate distribution, p = 2, . . . , 5, based on a random sample of size n = 20, 50, 100, 250 and n → ∞. n = 20 p=2 p=3 p=4 p=5 c2 0.052693 0.071797 0.085028 0.096548 c3 0.007985 0.009174 0.010018 c4 0.001184 0.001288 c5 0.000189 n = 50 c2 0.056104 0.078200 0.094521 0.109683 c3 0.009339 0.010883 0.012303 c4 0.001382 0.001516 c5 0.000205 n = 100 c2 0.057534 0.080452 0.097565 0.112749 c3 0.009768 0.011405 0.012908 c4 0.001450 0.001611 c5 0.000216 n = 250 c2 0.058256 0.081890 0.100115 0.115406 c3 0.010071 0.012099 0.013757 c4 0.001500 0.001668 c5 0.000221 n=∞ c2 0.059279 0.084364 0.102630 0.117481 c3 0.010358 0.012056 0.013358 c4 0.001511 0.001628 c5 0.000214 It is especially convenient to choose the ck ’s in such a way that P(ξk ≤ ck ) = β for all 2 ≤ k ≤ p, whence Pp p α = 1 − β k=2 (k) , p that is, β = (1 − α)1/(2 −p−1) . Table 1 gives the values of c2 , . . . , cp for p = 2, . . . , 5 and α = 1, 5, 10%. These values were obtained from the Cornish-Fisher asymptotic expansion of the distribution of ξk using its first six cumulants given by κm = 2m−1 (m − 1)! ζ(2m)k , π 2km 1 ≤ m ≤ 6, and ζ(·) denotes Riemann’s zeta function. Besides the fact that it gives the same weight to all the TA,n ’s, choosing P(ξk ≤ ck ) = β for all 2 ≤ k ≤ p allows for graphical representation of the test in the same spirit as the Les Cahiers du GERAD 8 G–2002–03 0.10 0.0 0.05 Tan 0.15 0.20 classical correlogram. On the x-axis of what can be called a “dependogram,” the subsets A are ordered lexicographically by size, beginning with |A| = 2. The corresponding values of TA,n are represented by vertical bars, and horizontal lines placed at height c |A| make it easy to identify subsets leading of the rejection of the independence hypothesis. 0 5 10 15 20 25 Subsets Figure 1. Dependogram of asymptotic level α = 5% constructed for a random sample of size n = 50 from a vector (X1 , . . . , X5 ) with standard normal marginals, pairwise but not joint independence between X1 , X2 , X3 , and corr(X4 , X5 ) = 1/2. See end of Section 3.1 for details of construction. The construction of the dependogram is illustrated in Figure 1 for a random sample of size n = 50 from a vector (X1 , . . . , X5 ) with standard normal √ marginals with X1 = |Z1 |sign(Z2 Z3 ), X2 = Z2 , X3 = Z3 , X4 = Z4 and X5 = Z4 /2 + 3Z5 /2, where the Zi ’s are mutually independent standard normal random variates. In this case, X1 , X2 , X3 are pairwise but not jointly independent (Romano & Siegel 1985, p. 33); they are, however, independent from the pair (X4 , X5 ). As can be seen on Figure 1, the subset A = {4, 5}, which is tenth on the list, exhibits a clear dependence at approximate level α = 5% between the two corresponding variables. In addition, the eleventh subset, A = {1, 2, 3}, makes apparent the joint dependence between variables X1 , X2 , X3 which the pairwise statistics TA,n with A = {1, 2}, {1, 3} and {2, 3} could not possibly have detected. 3.2 Combining P -values Under the null hypothesis of independence, the statistic TA,n has distribution function F|A|,n and the corresponding P -value Les Cahiers du GERAD 9 G–2002–03 pA,n = 1 − F|A|,n (TA,n ) is asymptotically uniform on the interval (0, 1). Since the pA,n ’s are also mutually independent in the limit, the combined statistic X TF,n = −2 log (pA,n ) , A⊂Sp , |A|>1 originally proposed by Fisher (1950, pp. 99–101), converges in law to a chi-square variable with 2(2p − p − 1) degrees of freedom. As shown by Littell & Folks (1973), the statistic TF,n is asymptotically Bahadur optimal within a large class of reasonable combination procedures. Its good properties are confirmed in simulations reported in Section 6. Table 2 below provides the α = 5% critical values of TF,n when n = 20, 50, 100. Note the fairly quick convergence of the critical values to those of the chi-square with 2(2 p − p − 1) degrees of freedom, entered under n = ∞ in the table. Although it would be computationally more convenient to replace F|A|,n by the distribution function of ξ|A| identified in Proposition 2, the convergence of the statistic would then be much slower, as intensive simulations confirm. Table 2. Critical points for testing independence at the α = 5% level using Fisher’s combined P -value test for a p-variate distribution, p = 2, . . . 5, from a random sample of size n = 20, 50, 100 and n → ∞. p=2 p=3 p=4 p=5 p 2(2 − p − 1) 2 8 22 52 n = 20 6.00 15.08 33.79 74.23 n = 50 6.00 15.43 34.02 72.14 n = 100 5.85 15.30 33.6 70.30 n = ∞ 5.99 15.51 33.92 69.83 4 The serial case This section describes how the previous results must be adapted to test the white noise hypothesis for a stationary univariate time series Y1 , Y2 , . . . with continuous marginal distribution F . Tests of this hypothesis are commonly based on vectors Xi = (Yi , . . . , Yi+p−1 ) of successive data points constructed as in (2) for each 1 ≤ i ≤ n + 1 − p. As noted by Delgado (1996), the empirical process   p n−p+1 p Y X Y © ª © ª 1  Hsn F −1 (t1 ), . . . , F −1 (tp ) = √ I Yi+j−1 ≤ F −1 (tj ) − tj  n i=1 j=1 j=1 and its rank-based analogue 1 Csn (t) = √ n  p n−p+1 X Y i=1  j=1 I (Ri+j−1  p  Y ≤ ntj ) − tj  j=1 Les Cahiers du GERAD 10 G–2002–03 have unwieldy asymptotic limits in D ([0, 1]p ). The latter are given respectively by Cs and C̃s , which are shown in Appendix A to be connected by the relation s s C̃ (t) = C (t) − p X β(tj ) j=1 Y tk (9) k6=j with β a Brownian bridge satisfying β(u) = Cs (u, 1, . . . , 1). Applying the Möbius transformation to Csn allows one to write it as a weighted linear combination of subprocesses X Y GsA,n (t) = (−1)|A\B| Csn (tB ) tj B⊂A j∈A\B which converge weakly to X Y X Y tj , tj = (−1)|A\B| Cs (tB ) (−1)|A\B| C̃s (tB ) GsA (t) = B⊂A B⊂A j∈A\B j∈A\B where the second equality is justified by the chain of identities X X X X (−1)|A\B| (−1)|A\B| aj = aj B⊂A j∈B j∈A = X j∈A B⊂A,B∋j aj (1 − 1)|A|−1 = 0 (10) Q with aj = β(tj ) k6=j tk . Following Theorem 2.2 in Ghoudi et al. (2001), the GsA have the same joint distribution as the GA ’s for all sets A satisfying 1 ∈ A and |A| > 1. Let   n−p+1 p p X Y Y 1  Cs⋆ I {Ri+j−1 ≤ (n + 1)tj } − Un (tj ) . n (t1 , . . . , tp ) = √ n i=1 j=1 j=1 Since Gs⋆ A,n (t) = X B (−1)|A\B| Cs⋆ n (t ) B⊂A = Y Un (tj ) j∈A\B n−p+1 1 X Y √ [I {Ri+j−1 ≤ (n + 1)tj } − Un (tj )] n i=1 j∈A obviously has the same limit as GsA,n , the serial analogue of Proposition 1 reads as follows. Proposition 3. Under the null hypothesis of independence, the processes G s⋆ A,n defined for arbitrary A ⊂ Sp such that 1 ∈ A and |A| > 1 converge weakly to the mutually independent continuous centered Gaussian processes GA with covariance function (3). Les Cahiers du GERAD 11 G–2002–03 Table 3. Critical values cs2 , . . . , csp for the global test of white noise at level α = 5% against dependence of order p = 2, . . . , 6, based on a stationary univariate time series of length n = 20, 50, 100, 250 and n → ∞. n = 20 c2 c3 c4 c5 c6 n = 50 c2 c3 c4 c5 c6 n = 100 c2 c3 c4 c5 c6 n = 250 c2 c3 c4 c5 c6 n=∞ c2 c3 c4 c5 c6 p=2 0.049223 p=3 0.059356 0.006363 p=4 0.065056 0.006876 0.001009 p=5 0.070118 0.007412 0.001087 0.000168 p=6 0.072402 0.007814 0.001172 0.000180 0.000031 0.054372 0.069768 0.008237 0.081284 0.009278 0.001221 0.091476 0.010056 0.001321 0.000202 0.100569 0.010866 0.001429 0.000216 0.000037 0.056516 0.073182 0.008938 0.085884 0.010237 0.001327 0.097123 0.011310 0.001451 0.000208 0.108527 0.012528 0.001586 0.000221 0.000036 0.057427 0.076014 0.009314 0.090718 0.010697 0.001402 0.103241 0.012089 0.001526 0.000210 0.116332 0.013313 0.001625 0.000225 0.000034 0.059279 0.079103 0.009855 0.094541 0.011315 0.001441 0.108080 0.012543 0.001555 0.000207 0.120404 0.013603 0.001650 0.000216 0.000030 Arguing as in Section 3.1, a serial version of the dependogram can be based on Proposip−1 tion 3 by selecting critical values csk so that for each k, P(ξk ≤ csk ) = β = (1 − α)1/(2 −1) . The global level of the test is then Les Cahiers du GERAD  [ P 12 G–2002–03 A⊂Sp , |A|>1, 1∈A  s {TA,n > csA } ≈ 1 − = 1− ³ ´ P ξ|A| ≤ cs|A| Y A⊂Sp , |A|>1, 1∈A p Y p−1 {P (ξk ≤ csk )}(k−1) k=2 = α. 0.0 0.05 0.10 Tan 0.15 0.20 0.25 Table 3 gives the values of cs2 , . . . , csp for p = 2, . . . , 6 and α = 1, 5, 10%. These critical points were again obtained from the Cornish-Fisher asymptotic expansion of the distribution of ξk using its first few cumulants. 0 5 10 15 20 25 30 Subsets Figure 2. Serial dependogram of asymptotic level α = 5% constructed for two intertwined AR(1) series, each of length n = 50 and having autocorrelation coefficient of 1/2. Figure 2 displays a serial dependogram with p = 6. This diagram allows both for a visual inspection and a formal test of the white noise hypothesis at an asymptotic nominal level α = 5%. The series analyzed here is composed of two intertwined AR(1) models with an autocorrelation coefficient of 1/2 each, to ensure stationarity. The statistic T A,n corresponding to the set A = {1, 3} is seen to be highly significant, as expected from the fact that observations xt and xt+2 in the series are successive observations from the same AR(1) process. The fact that no other statistic exceeds its threshold is a combined effect of the quickly fading dependence within each series and the fact that the global level of the test is 5%. As a second example, consider the deterministic “tent map” series defined by Xi+1 = f (Xi ), where Les Cahiers du GERAD 13 G–2002–03 f (x) = ½ 2x if 0 ≤ x ≤ 1/2, 2(1 − x) if 1/2 < x ≤ 1, and X0 is uniformly distributed on [0, 1]. As is well known (cf., e.g., Chatterjee & Yilmaz 1992), this series has the same autocorrelation function as white noise. In this situation, which is typical of chaotic time series, the dependence is usually stronger for small lags but slowly spreads to higher lags as the length of the series increases. This is clearly reflected in Figure 3, which depicts the serial dependogram with p = 6 and α = 5% corresponding to the series with initial value X0 = 0.1789. Despite its interest as a graphical diagnostic tool, the test based on the serial dependogram turns out to be somewhat less efficient than Fisher’s combined P -value statistic X © ª s TF,n = −2 log 1 − F|A|,n (TA,n ) , A⊂Sp , |A|>1, 1∈A 0.15 0.0 0.05 0.10 Tan 0.20 0.25 which is asymptotically chi-square with 2(2p−1 −1) degrees of freedom asymptotically. This convergence, which is an immediate consequence of Proposition 3, is illustrated in Table 4, s where α = 5% critical values of TA,n are given for testing against dependence of order p = 2, . . . , 6 from a univariate time series of length n = 20, 50, 100. Observe the somewhat slow convergence of these critical points to the limiting values, especially when p = 6. This is due to the relatively small number of (5-dependent!) Xi vectors; for instance, there are only n + 1 − p = 15 data points to work with when n = 20 and p = 6! 0 5 10 15 20 25 30 Subsets Figure 3. Serial dependogram of asymptotic level α = 5% constructed from the deterministic tentmap series of length n = 100 with initial value X0 = 0.1789. Les Cahiers du GERAD 14 G–2002–03 Table 4. Critical points for Fisher’s combined P -value test of white noise at the α = 5% level against dependence of order p = 2, . . . , 6 from a univariate time series of length n = 20, 50, 100 and n → ∞. n = 20 n = 50 n = 100 n=∞ 5 p=2 8.09 6.50 6.28 5.99 p=3 16.96 13.55 13.05 12.59 p=4 35.5 25.9 24.9 23.69 p=5 73.5 50.7 47.3 43.77 p=6 145.4 104.0 95.6 81.38 Relations with standard linear rank statistics The purpose of this section is to highlight the implications of results presented in Sections 2 and 4 for the asymptotic theory of linear rank statistics traditionally used to test independence, both in the serial and non-serial case. By a way of introduction, restrict initially to the non-serial case and for given A = {j, k} of size 2, consider the statistic ρA,n ¶µ ¶ n µ Rik 1 1 1 X Rij , − − = n n+1 2 n+1 2 i=1 which is proportional to Spearman’s rho statistic between two components X j and Xk of a random vector (X1 , . . . , Xp ). A simple calculation shows that under the null hypothesis of independence, Z √ G⋆A,n (t) dt. nρA,n = √ [0,1]p In view of Proposition 1, therefore, the nρA,n ’s are asymptotically normally distributed with mean zero and variance 1/144 = 1/12|A| . In addition, ρA,n and ρA′ ,n are asymptotically independent whenever A 6= A′ . As already observed by Deheuvels (1981a, Theorem 6), a distinct advantage of the above representation is that it generalizes readily to arbitrary subsets A of S p = {1, . . . , p}, namely µ ¶ Z n Rij 1 1 XY |A| G⋆A,n (t) dt. − = (−1) ρA,n = n n+1 2 [0,1]p i=1 j∈A Note in passing that the asymptotic distribution of this statistic is only equivalent to that of Kendall’s tau when |A| = 2; cf., e.g., Barbe et al. (1996). A similar extension is also possible in the serial case, viz. ρsA,n µ ¶ Z n−p+1 1 X Y Ri+j−1 1 = − = (−1)|A| Gs⋆ A,n (t) dt. n n+1 2 p [0,1] i=1 j∈A It follows from Propositions 1 and 3 that under the null hypothesis of independence (or randomness), ρA,n and ρsA,n have the same asymptotic normal distribution with zero mean Les Cahiers du GERAD G–2002–03 15 and variance 1/12|A| . Furthermore, stochastic independence between ρA,n and ρA′ ,n continues to hold asymptotically whenever A 6= A′ , the same being true in the serial case (with the restriction that the element 1 must belong to the subsets considered). The following proposition generalizes these findings to a much larger class of linear rank statistics which also includes as special cases the Laplace, van der Waerden and Wilcoxon rank statistics. This sheds new light on earlier results of Hallin, Ingenbleek & Puri (1985, 1987) for the case |A| = 2. Proposition 4. Let K = (K1 , . . . , Kp ) be a vector whose jth component Kj is a distribution function with left-continuous inverses Lj = Kj−1 and variance σj2 < ∞. For arbitrary A ⊂ Sp with |A| > 1, define    O O IRA =  IR  [0, 1] j∈Ac j∈A and n JA,n 1 XY =√ n i=1 j∈A ½ µ ¾ ¶ Z Rij |A| Lj G⋆A,n {K(x)} dx, − L̄j = (−1) n+1 IRA where n 1X Lj L̄j = n i=1 When 1 ∈ A, let also s JA,n µ i n+1 ¶ . ¾ ¶ ½ µ Z n−p+1 Ri+j−1 1 X Y |A| =√ Gs⋆ − L̄j = (−1) Lj A,n {K(x)} dx. A n+1 n IR i=1 j∈A s Under the null hypothesis of independence, the statistics JA,n and JA,n converge weakly to mutually independent centered Gaussian variables Z |A| (−1) GA {K(x)} dx IRA 2 = whose variances are equal to σA 2 j∈A σj . Q Taking Lj (u) = u for j = 1, . . . , p leads to the statistics ρA,n discussed by Deheuvels (1981a). Similarly, the van der Waerden statistic obtains when L1 (u) = · · · = Lp (u) = Φ−1 (u), the quantile function of the standard normal distribution. In this case, it is easy to show that σA = 1 for all admissible subsets A ⊂ Sp . When p = 2, a third example is provided by the Wilcoxon statistic, which corresponds to the choices L 1 (u) = u and L2 (u) = log{u/(1 − u)}. Finally, taking L1 (u) = {1 + sign(u)}/2 and L2 (u) = log{2 min(u, 1 − u)} yields the Laplace statistic also referred to as the median test-score statistic. Les Cahiers du GERAD 16 G–2002–03 Once again, the asymptotic independence allows for construction of Spearman-ogram, van der Waerden-ogram, etc. Remark. Proceeding as in the proof of Proposition 4, it can readily be seen that if J˜A,n s and J˜A,n are defined as above in terms of distribution functions K̃1 , . . . , K̃p ), then for arbitrary A, A′ ∈ Sp , ³ ´ ³ ´ s lim cov JA,n , J˜A′ ,n = lim cov JA,n , J˜s ′ n→∞ n→∞ A ,n The latter equals o YZ n Kj (xj ) ∧ K̃j (yj ) − Kj (xj )K̃j (yj ) dxj dyj j∈A if A = A′ and zero otherwise. 6 Simulation study In order to illustrate the above results and investigate the power of the two tests of independence proposed in this paper, extensive simulations were run, both in serial and non-serial contexts, and for a variety of alternatives. As the test based on Fisher’s combination of P -values turned out to be generally the more powerful of the two for the models considered, only those results are presented here. Since the conclusions were substantivally the same for time series and multivariate data, this section concentrates mostly on the latter. Given a random sample from some p-variate distribution, the most common procedure used for checking independence is the likelihood ratio test (LRT), derived under the assumption of multivariate normality. The statistic is defined as ¶ µ 2p + 5 log{det(Rp,n )}, LRT = − n − 1 − 6 in terms of the empirical p × p correlation matrix Rp,n . Because X ¡√ ¢2 1 nrij,n + oP (1/n), det(Rp,n ) = 1 − n 1≤i<j≤p it is well known that, under the null hypothesis of independence, the test statistic converges in law to X 2 Zij , 1≤i<j≤p where the Zij are independent standard Gaussian random variables. Note that this remains true whether the underlying population is multivariate normal or not. Therefore, the LRT statistic is asymptotically chi-square with p(p − 1)/2 degrees of freedom, so long as the second moments exist. The following pages show comparative graphs of the power function of the LRT and Fisher’s statistic TF,n for Les Cahiers du GERAD 17 G–2002–03 - two dimensions, viz. p = 2 and 5; - three sample sizes, viz. n = 20, 50 and 100; - three dependence structures, viz., the equicorrelated normal and two copula models, i.e., those of Clayton (1978) and Gumbel (1960); - four marginal distributions, viz., standard normal, exponential, and Cauchy, as well as the Pareto distribution with distribution function 1 − x−4 for x ≥ 1. Here, the normal model refers to a p-variate Gaussian vector with compo√ √ equicorrelated nents rZ0 + 1 − rZi , 1 ≤ i ≤ p, where Z0 , . . . , Zp form a random sample from the standard univariate normal distribution. The p-variate model of Clayton (1978), with arbitrary marginals F1 , . . ., Fp , is defined as Cθ {F1 (x1 ), . . . , Fp (xp )} (11) in terms of the copula Cθ (u1 , . . . , up ) = à p X i=1 u−θ i +p−1 !−1/θ , θ>0 with C0 (u1 , . . . , up ) = limθ→0 Cθ (u1 , . . . , up ) = u1 × · · · × up for all 0 ≤ u1 , . . . , up ≤ 1. This Archimedean copula model is quite popular in survival analysis, where it provides a natural multivariate extension of Cox’s proportional hazards model; cf., e.g., Oakes (2001, Section 7.3). The p-variate copula model of Gumbel (1960), which arises in extreme value theory, is also of the form (11), but with  ( )θ  p X Cθ (u1 , . . . , up ) = exp − | log(ui )|1/θ  , 0 < θ ≤ 1 i=1 with C1 being the independence copula. Figures 4–6 pertain to the case n = 20, Figures 7–9 to the case n = 50, and Figures 10–12 to the case n = 100. In each figure, eight graphs are displayed in two columns, corresponding to dimension p = 2 on the left, and p = 5 on the right. Each graph compares the power curve of Fisher’s test TF,n (solid line) to that of the LRT (dotted line). In order to facilitate comparisons between dependence structures, all curves are plotted as a function of the population value of Kendall’s tau for the various bivariate alternatives. Thus 2 τ = arcsin(r) π for the equicorrelated normal model, while τ= θ θ+2 and τ = 1 − θ Les Cahiers du GERAD 18 G–2002–03 for Clayton’s and Gumbel’s model, respectively; cf., e.g., Genest & MacKay (1986) or Nelsen (1999, Chapter 5). All graphs were smoothed from 10,000 replicates obtained for each value of τ = j/20 for 0 ≤ j ≤ 19, with j = 0 corresponding to the null hypothesis of stochastic independence. Detailed inspection of these 72 graphs leads to the following general observations: a) both the dependence structure and the choice of marginals influence the relative performance of the tests; b) while the nonparametric test always maintains its nominal level, such is not the case for the LRT, particularly when the marginals are Cauchy; c) Fisher’s test is far superior to the LRT for a Clayton type of dependence, except when the marginals are exponential; d) for normal or Gumbel type dependences, the results are more mitigated when n = 20 or 50, but the new test outperforms the classical one, except in the case of multivariate normality (for which the LRT is optimal) and either when Pareto marginals are coupled with an Gumbel extreme value dependence structure, or when exponential marginals are mixed with Clayton’s copula; e) as a general rule, the LRT looses ground to the test based on TF,n when sample size increases, so that TF,n gradually catches up with the LRT or leaves it further behind as n → ∞; f) differences between the two tests are more noticeable for small to moderate degrees of dependence (i.e., τ < 1/2) and far less perceivable with p = 5 than with p = 2. As a numerical illustration of the final point, consider the ratio Q = Π(TF,n )/ Π(LRT) comparing the power of the two tests for a random sample of size n = 100 from either one of the nine p-variate families obtained by crossing the three dependence structures and either normal, exponential or Pareto marginals. (The data for the Cauchy were ignored, given the inappropriate level of the LRT). Looking at the range of Q reported in Table 5 for the case p = 2, one can see that except in the cases along the main diagonal, one has nothing to loose — and potentially much to gain — by preferring the Fisher test T F,n to the standard likelihood ratio test statistic. Judging from Table 6, however, larger sample sizes are required to reach the same conclusion in a multivariate population of dimension 5. Table 5. Range of the ratio Π(TF,n )/ Π(LRT) measuring the power of Fisher’s test TF,n relative to the LRT for random samples of size n = 100 from the specified bivariate models (p = 2). Copula/Marginals Normal Clayton Gumbel Normal 0.86 – 1.00 1.00 – 1.46 1.07 – 2.36 Exponential 0.98 – 1.10 0.78 – 1.00 1.00 – 3.79 Pareto 1.00 – 1.45 1.00 – 3.67 0.48 – 1.04 Les Cahiers du GERAD 19 G–2002–03 Table 6. Range of the ratio Π(TF,n )/ Π(LRT) measuring the power of Fisher’s test TF,n relative to the LRT for random samples of size n = 100 from the specified p-variate models with p = 5. Copula/Marginals Normal Clayton Gumbel Normal 0.60 – 1.00 0.60 – 1.09 0.71 – 1.49 Exponential 0.67 – 1.00 0.67 – 1.00 1.00 – 2.94 Pareto 0.62 – 1.07 1.00 – 2.90 0.94 – 1.14 Much similar conclusions were obtained in simulations involving time series. In the serial case, comparisons were made with the Ljung-Box statistic (n − p) 2p−1 X−1 2 rj,n , j=1 where rj,n is the empirical autocorrelation coefficient of order j. Under the null hypothesis of randomness, the latter is known to be asymptotically distributed as a chi-square with 2p−1 − 1 degrees of freedom, as long as the second moment exists. Figure 13 offers an extreme illustration of the gains in power associated with the serial s of Fisher’s test, based on 10,000 independent series of length n = 50 from the version TF,n deterministic tentmap series of Section 4, with initial value X0 drawn from the uniform distribution on the unit interval. While the Ljung-Box test is incapable of detecting this s picks it up rather strong dependence because of the vanishing autocorrelation function, T F,n easily, though much less quickly as p increases from 2 to 6. In this case, the poor performance of the Ljung-Box statistic is easy to explain. Indeed, a simple application of the central limit theorem for reverse martingales (cf., e.g., Genest et al. 2001) shows that when the data arise from the tentmap series, the Ljung-Box statistic converges in law to 2p−1 X−1 Zj2 , j=1 where the Zj ’s are jointly centered Gaussian variables with variance 1 + 42−k /5 and cov(Zj , Zk ) = 3/4k−1 for 1 ≤ j < k. Les Cahiers du GERAD Power 0.4 0.4 0.8 Gaussian marginals (n=20, p = 5) 0.8 Gaussian marginals (n=20, p = 2) Power 20 G–2002–03 0.0 TF LRT 0.0 TF LRT 0.2 0.4 0.6 0.8 1.0 0.0 0.4 0.6 Exponential marginals (n=20, p = 2) Exponential marginals (n=20, p = 5) 0.8 Power 0.4 0.0 TF LRT 0.0 TF LRT 0.4 0.6 0.8 1.0 0.0 0.4 0.6 tau Pareto marginals (n=20, p = 2) Pareto marginals (n=20, p = 5) 0.8 1.0 Power 0.4 0.4 Power 0.2 tau 0.8 0.2 0.8 0.0 0.0 TF LRT 0.0 TF LRT 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 tau tau Cauchy marginals (n=20, p = 2) Cauchy marginals (n=20, p = 5) 0.8 1.0 Power 0.4 0.4 0.8 0.2 0.8 0.0 Power 1.0 0.8 tau 0.4 Power 0.2 tau 0.8 0.0 0.0 TF LRT 0.0 TF LRT 0.0 0.2 0.4 0.6 tau 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 tau Figure 4. Power curves of the LRT and Fisher’s TF,n test at the 5% level, based on 10,000 independent pseudo-random samples of size n = 20 from a p-variate distribution, p = 2, 5, with normal copula and marginal distributions that are either Gaussian, exponential, Pareto or Cauchy. Les Cahiers du GERAD Power 0.4 0.4 0.8 Gaussian marginals (n=20, p = 5) 0.8 Gaussian marginals (n=20, p = 2) Power 21 G–2002–03 0.0 TF LRT 0.0 TF LRT 0.2 0.4 0.6 0.8 1.0 0.0 0.4 0.6 Exponential marginals (n=20, p = 2) Exponential marginals (n=20, p = 5) 0.8 Power 0.4 0.0 TF LRT 0.0 TF LRT 0.4 0.6 0.8 1.0 0.0 0.4 0.6 tau Pareto marginals (n=20, p = 2) Pareto marginals (n=20, p = 5) 0.8 1.0 Power 0.4 0.4 Power 0.2 tau 0.8 0.2 0.8 0.0 0.0 TF LRT 0.0 TF LRT 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 tau tau Cauchy marginals (n=20, p = 2) Cauchy marginals (n=20, p = 5) 0.8 1.0 Power 0.4 0.4 0.8 0.2 0.8 0.0 Power 1.0 0.8 tau 0.4 Power 0.2 tau 0.8 0.0 0.0 TF LRT 0.0 TF LRT 0.0 0.2 0.4 0.6 tau 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 tau Figure 5. Power curves of the LRT and Fisher’s TF,n test at the 5% level, based on 10,000 independent pseudo-random samples of size n = 20 from a p-variate distribution, p = 2, 5, with Clayton copula and marginal distributions that are either Gaussian, exponential, Pareto or Cauchy. Les Cahiers du GERAD Power 0.4 0.4 0.8 Gaussian marginals (n=20, p = 5) 0.8 Gaussian marginals (n=20, p = 2) Power 22 G–2002–03 0.0 TF LRT 0.0 TF LRT 0.2 0.4 0.6 0.8 1.0 0.0 0.4 0.6 Exponential marginals (n=20, p = 2) Exponential marginals (n=20, p = 5) 0.8 Power 0.4 0.0 TF LRT 0.0 TF LRT 0.4 0.6 0.8 1.0 0.0 0.4 0.6 tau Pareto marginals (n=20, p = 2) Pareto marginals (n=20, p = 5) 0.8 1.0 Power 0.4 0.4 Power 0.2 tau 0.8 0.2 0.8 0.0 0.0 TF LRT 0.0 TF LRT 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 tau tau Cauchy marginals (n=20, p = 2) Cauchy marginals (n=20, p = 5) 0.8 1.0 Power 0.4 0.4 0.8 0.2 0.8 0.0 Power 1.0 0.8 tau 0.4 Power 0.2 tau 0.8 0.0 0.0 TF LRT 0.0 TF LRT 0.0 0.2 0.4 0.6 tau 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 tau Figure 6. Power curves of the LRT and Fisher’s TF,n test at the 5% level, based on 10,000 independent pseudo-random samples of size n = 20 from a p-variate distribution, p = 2, 5, with Gumbel copula and marginal distributions that are either Gaussian, exponential, Pareto or Cauchy. Les Cahiers du GERAD Power 0.4 0.4 0.8 Gaussian marginals (n=50, p = 5) 0.8 Gaussian marginals (n=50, p = 2) Power 23 G–2002–03 0.0 TF LRT 0.0 TF LRT 0.2 0.4 0.6 0.8 1.0 0.0 0.4 0.6 Exponential marginals (n=50, p = 2) Exponential marginals (n=50, p = 5) 0.8 Power 0.4 0.0 TF LRT 0.0 TF LRT 0.4 0.6 0.8 1.0 0.0 0.4 0.6 tau Pareto marginals (n=50, p = 2) Pareto marginals (n=50, p = 5) 0.8 1.0 Power 0.4 0.4 Power 0.2 tau 0.8 0.2 0.8 0.0 0.0 TF LRT 0.0 TF LRT 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 tau tau Cauchy marginals (n=50, p = 2) Cauchy marginals (n=50, p = 5) 0.8 1.0 Power 0.4 0.4 0.8 0.2 0.8 0.0 Power 1.0 0.8 tau 0.4 Power 0.2 tau 0.8 0.0 0.0 TF LRT 0.0 TF LRT 0.0 0.2 0.4 0.6 tau 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 tau Figure 7. Power curves of the LRT and Fisher’s TF,n test at the 5% level, based on 10,000 independent pseudo-random samples of size n = 50 from a p-variate distribution, p = 2, 5, with normal copula and marginal distributions that are either Gaussian, exponential, Pareto or Cauchy. Les Cahiers du GERAD Power 0.4 0.4 0.8 Gaussian marginals (n=50, p = 5) 0.8 Gaussian marginals (n=50, p = 2) Power 24 G–2002–03 0.0 TF LRT 0.0 TF LRT 0.2 0.4 0.6 0.8 1.0 0.0 0.4 0.6 Exponential marginals (n=50, p = 2) Exponential marginals (n=50, p = 5) 0.8 Power 0.4 0.0 TF LRT 0.0 TF LRT 0.4 0.6 0.8 1.0 0.0 0.4 0.6 tau Pareto marginals (n=50, p = 2) Pareto marginals (n=50, p = 5) 0.8 1.0 Power 0.4 0.4 Power 0.2 tau 0.8 0.2 0.8 0.0 0.0 TF LRT 0.0 TF LRT 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 tau tau Cauchy marginals (n=50, p = 2) Cauchy marginals (n=50, p = 5) 0.8 1.0 Power 0.4 0.4 0.8 0.2 0.8 0.0 Power 1.0 0.8 tau 0.4 Power 0.2 tau 0.8 0.0 0.0 TF LRT 0.0 TF LRT 0.0 0.2 0.4 0.6 tau 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 tau Figure 8. Power curves of the LRT and Fisher’s TF,n test at the 5% level, based on 10,000 independent pseudo-random samples of size n = 50 from a p-variate distribution, p = 2, 5, with Clayton copula and marginal distributions that are either Gaussian, exponential, Pareto or Cauchy. Les Cahiers du GERAD Power 0.4 0.4 0.8 Gaussian marginals (n=50, p = 5) 0.8 Gaussian marginals (n=50, p = 2) Power 25 G–2002–03 0.0 TF LRT 0.0 TF LRT 0.2 0.4 0.6 0.8 1.0 0.0 0.4 0.6 Exponential marginals (n=50, p = 2) Exponential marginals (n=50, p = 5) 0.8 Power 0.4 0.0 TF LRT 0.0 TF LRT 0.4 0.6 0.8 1.0 0.0 0.4 0.6 tau Pareto marginals (n=50, p = 2) Pareto marginals (n=50, p = 5) 0.8 1.0 Power 0.4 0.4 Power 0.2 tau 0.8 0.2 0.8 0.0 0.0 TF LRT 0.0 TF LRT 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 tau tau Cauchy marginals (n=50, p = 2) Cauchy marginals (n=50, p = 5) 0.8 1.0 Power 0.4 0.4 0.8 0.2 0.8 0.0 Power 1.0 0.8 tau 0.4 Power 0.2 tau 0.8 0.0 0.0 TF LRT 0.0 TF LRT 0.0 0.2 0.4 0.6 tau 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 tau Figure 9. Power curves of the LRT and Fisher’s TF,n test at the 5% level, based on 10,000 independent pseudo-random samples of size n = 50 from a p-variate distribution, p = 2, 5, with Gumbel copula and marginal distributions that are either Gaussian, exponential, Pareto or Cauchy. Les Cahiers du GERAD Power 0.4 0.4 0.8 Gaussian marginals (n=100, p = 5) 0.8 Gaussian marginals (n=100, p = 2) Power 26 G–2002–03 0.0 TF LRT 0.0 TF LRT 0.2 0.4 0.6 0.8 1.0 0.0 0.4 0.6 0.8 Exponential marginals (n=100, p = 2) Exponential marginals (n=100, p = 5) Power 0.4 0.0 TF LRT 0.0 TF LRT 0.4 0.6 0.8 1.0 0.0 0.4 0.6 tau Pareto marginals (n=100, p = 2) Pareto marginals (n=100, p = 5) 0.8 1.0 Power 0.4 0.4 Power 0.2 tau 0.8 0.2 0.8 0.0 0.0 TF LRT 0.0 TF LRT 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 tau tau Cauchy marginals (n=100, p = 2) Cauchy marginals (n=100, p = 5) 0.8 1.0 Power 0.4 0.4 0.8 0.2 0.8 0.0 Power 1.0 0.8 tau 0.4 Power 0.2 tau 0.8 0.0 0.0 TF LRT 0.0 TF LRT 0.0 0.2 0.4 0.6 tau 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 tau Figure 10. Power curves of the LRT and Fisher’s TF,n test at the 5% level, based on 10,000 independent pseudo-random samples of size n = 100 from a p-variate distribution, p = 2, 5, with normal copula and marginal distributions that are either Gaussian, exponential, Pareto or Cauchy. Les Cahiers du GERAD Power 0.4 0.4 0.8 Gaussian marginals (n=100, p = 5) 0.8 Gaussian marginals (n=100, p = 2) Power 27 G–2002–03 0.0 TF LRT 0.0 TF LRT 0.2 0.4 0.6 0.8 1.0 0.0 0.4 0.6 0.8 Exponential marginals (n=100, p = 2) Exponential marginals (n=100, p = 5) Power 0.4 0.0 TF LRT 0.0 TF LRT 0.4 0.6 0.8 1.0 0.0 0.4 0.6 tau Pareto marginals (n=100, p = 2) Pareto marginals (n=100, p = 5) 0.8 1.0 Power 0.4 0.4 Power 0.2 tau 0.8 0.2 0.8 0.0 0.0 TF LRT 0.0 TF LRT 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 tau tau Cauchy marginals (n=100, p = 2) Cauchy marginals (n=100, p = 5) 0.8 1.0 Power 0.4 0.4 0.8 0.2 0.8 0.0 Power 1.0 0.8 tau 0.4 Power 0.2 tau 0.8 0.0 0.0 TF LRT 0.0 TF LRT 0.0 0.2 0.4 0.6 tau 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 tau Figure 11. Power curves of the LRT and Fisher’s TF,n test at the 5% level, based on 10,000 independent pseudo-random samples of size n = 100 from a p-variate distribution, p = 2, 5, with Clayton copula and marginal distributions that are either Gaussian, exponential, Pareto or Cauchy. Les Cahiers du GERAD Power 0.4 0.4 0.8 Gaussian marginals (n=100, p = 5) 0.8 Gaussian marginals (n=100, p = 2) Power 28 G–2002–03 0.0 TF LRT 0.0 TF LRT 0.2 0.4 0.6 0.8 1.0 0.0 0.4 0.6 0.8 Exponential marginals (n=100, p = 2) Exponential marginals (n=100, p = 5) Power 0.4 0.0 TF LRT 0.0 TF LRT 0.4 0.6 0.8 1.0 0.0 0.4 0.6 tau Pareto marginals (n=100, p = 2) Pareto marginals (n=100, p = 5) 0.8 1.0 Power 0.4 0.4 Power 0.2 tau 0.8 0.2 0.8 0.0 0.0 TF LRT 0.0 TF LRT 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 tau tau Cauchy marginals (n=100, p = 2) Cauchy marginals (n=100, p = 5) 0.8 1.0 Power 0.4 0.4 0.8 0.2 0.8 0.0 Power 1.0 0.8 tau 0.4 Power 0.2 tau 0.8 0.0 0.0 TF LRT 0.0 TF LRT 0.0 0.2 0.4 0.6 tau 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 tau Figure 12. Power curves of the LRT and Fisher’s TF,n test at the 5% level, based on 10,000 independent pseudo-random samples of size n = 100 from a p-variate distribution, p = 2, 5, with Gumbel copula and marginal distributions that are either Gaussian, exponential, Pareto or Cauchy. Les Cahiers du GERAD 29 G–2002–03 Power 0.4 0.8 n=100, p = 2 0.0 TF LRT 0.0 0.05 0.10 0.15 0.20 0.25 rho Power 0.4 0.8 n=100, p = 3 0.0 TF LRT 0.0 0.05 0.10 0.15 0.20 0.25 rho Power 0.4 0.8 n=100, p = 4 0.0 TF LRT 0.0 0.05 0.10 0.15 0.20 0.25 rho Power 0.4 0.8 n=100, p = 5 0.0 TF LRT 0.0 0.05 0.10 0.15 0.20 0.25 0.15 0.20 0.25 rho Power 0.0 0.4 0.8 n=100, p = 6 0.0 0.05 0.10 rho s of Fisher’s test at the 5% Figure 13. Power curves of the LRT and the serial version TF,n level, based on 10,000 independent tent map series of length n = 50. Les Cahiers du GERAD 7 30 G–2002–03 Conclusion This paper has shown how it is possible to exploit the asymptotic independence of a Möbius decomposition of the empirical copula process to devise powerful tests of independence based on Cramér-von Mises statistics. Following the work of Deheuvels (1979, 1980, 1981a,b,c), two different strategies were proposed. Fixing the overall level of the test first led to a graphical diagnostic tool for identifying failures of the independence hypothesis in subsets of variables. A second approach, based on Fisher’s methods for the combination of P -values, led to an even better test that compares quite favorably to the standard likelihood ratio test statistic in a variety of settings. Both tests apply either to a multivariate or serial context, and the treatment provided herein allows for a unified study of the asymptotic behavior of linear rank statistics derived from copulas. In future work, it would be of interest to calculate Pitman’s asymptotic relative efficiency of the newly proposed tests with respect to the likelihood ratio statistic, and to extend numerical comparisons with alternative, more specialized, parametric tests. A greater challenge would be to try to characterize structures of serial and non-serial dependence for which Fisher’s test is most likely to be powerful in small samples and for large dimensions. Appendix A: Proof of Proposition 2 The derivation of representation (9) is based on the technique of pseudo-observations developed by Ghoudi & Rémillard (1998, 2000). To this end write Csn in the alternative form   n−p+1 p p X Y Y 1  Csn (t) = √ I {Fn (Yi+j−1 ) ≤ tj } − tj  n i=1 j=1 j=1 in terms of pseudo-observations ei,n = (Fn (Yi ), . . . , Fn (Yi+p−1 )), where Fn is the empirical distribution function of the sample Y1 , . . . , Yn . √ Now, as is well known, Fn (y) = n{Fn (y) − F (y)} converges in law to a continuous Gaussian process F = β ◦ F , where β is a Brownian bridge having representation β(u) = Cs (u, 1, . . . , 1). Since conditions R1–R3 of Theorem 2.3 of Ghoudi et al. (2001) are easily verified, one can conclude that the limit of Csn in D ([0, 1]p ) can be written as C̃s (t) = Cs (t) − p X µj (t, Hj ), j=1 where Hj = F. Here, the functional µj is defined by µj (t, φ ◦ F ) = φ(tj ) Y k6=j tk Les Cahiers du GERAD 31 G–2002–03 for any continuous function φ vanishing at both ends of the interval [0, 1]. Therefore, Y µj (t, Hj ) = µj (t, β ◦ F ) = β(tj ) tk , k6=j as claimed. Appendix B: Proof of Proposition 4 As the proof is similar in the serial and non-serial cases, only the latter is sketched here. Given M > 1, it is clear that the mapping Z f ◦ K(x) dx, f 7→ IRA ∩[−M,M ]p defined for all continuous functions f vanishing outside (0, 1)p , is continuous. To prove the proposition, therefore, it suffices to show that the variance of Z G⋆A,n {K(x)} dx IRA ∩(IRp \[−M,M ]p ) is arbitrarily small, when M is large enough. Since    Y Y X  IEj   IEjc  1 − IE1 ×···×Ep = j∈B c B⊂Sp , B6=∅ j∈B for arbitrary events E1 , . . . , Ep , then in particular, when E1 = · · · = Ep = [−M, M ], "Z # "Z # X var1/2 G⋆A,n {K(x)} dx ≤ var1/2 G⋆A,n {K(x)} dx IRA ∩(IRp \[−M,M ]p ) B⊂Sp , B6=∅ by Minkowski’s inequality, where the event    \ O O EB,M = IRA  Ej   Ejc  j∈B c EB,M j∈B is either empty or has at least one component of the form [−M, M ]c . The latter occurs Les Cahiers du GERAD 32 G–2002–03 only when B ⊂ A, in which case "Z # ( ⋆ var GA,n {K(x)} dx = 1+ EB,M × (−1)|A| (n − 1)|A|−1 ( Y Z  ≤ 2 [−M,M ]c ×[−M,M ]c j∈B × Y j∈A\B Y j∈A\B × ) Y j∈B (Z Vn (xj , yj ) dxj dyj Vn (xj , yj ) dxj dyj [−M,M ]2 ) )  σj2  (Z [−M,M ]c ×[−M,M ]c Vn (xj , yj ) dxj dyj ) , where Vn (xj , yj ) = Un {Kj (xj ∧ yj )} − Un {Kj (xj )}Un {Kj (yj )} ≥ 0. Since B 6= ∅, it only remains to show that for any 1 ≤ j ≤ p, Z Vn (xj , yj ) dxj dyj → 0 lim sup n→∞ [−M,M ]c ×[−M,M ]c as M tends to infinity. The left-hand side is the sum of three positive terms, each of which involves V (xj , yj ) = Kj (xj ∧yj )−Kj (xj )Kj (yj ) ≥ 0 as an integrand. They are respectively Z −M Z −M Z −M yj Kj (yj )dyj , V (xj , yj ) dxj dyj ≤ −2 2 and Z −M −∞ Z −∞ −∞ −∞ ∞ V (xj , yj ) dxj dyj = M Z ∞Z ∞ M M Z −M Kj (yj )dyj −∞ V (xj , yj ) dxj dyj ≤ 2 Z Z ∞ M {1 − Kj (xj )} dxj , ∞ M yj {1 − Kj (yj )} dyj . In all cases, the upper bound approaches 0 as M → ∞ because σj2 is finite. 2 of J Finally, the limiting variance σA A,n is given by ¸ ·Z Z 2 GA {K(x)}GA {K(y)} dx dy σA = E IRA IRA ¸ Y ·Z Z {Kj (xj ∧ yj ) − Kj (xj )Kj (yj )} dxj dyj = j∈A = Y j∈A IR σj2 , IR Les Cahiers du GERAD G–2002–03 33 by Hoeffding’s identity. For the non-serial case, ¾ ¸ Z ∞· ½ Rij ≤ Kj (xj ) − Un ◦ Kj (xj ) dxj Iij,n = I n+1 −∞ ½ µ ¶ ¶¾ ½ ¾ µ n−1 X ℓ+1 ℓ ℓ −1 −1 = Kj − Kj I(ℓ ≥ Rij ) − n+1 n+1 n ℓ=1 ¶¾ µ n−1 X ℓ ½ µℓ+1¶ ℓ = − Lj − Lj n n+1 n+1 ℓ=1 µ ¶¾ n−1 X ½ µℓ+1¶ ℓ − Lj Lj + n+1 n+1 ℓ=Rij µ µ ¶ ¶ n Rij 1X ℓ Lj + = −Lj n+1 n n+1 l=1 ¶ µ Rij + L̄j . = −Lj n+1 Therefore Z n 1 XY GA,n {K(x)} dx = √ Iij,n = (−1)|A| JA,n . n IRA i=1 j∈A The serial case is similar. References Barbe, P., Genest, C., Ghoudi, K. & Rémillard, B. (1996). On Kendall’s process. Journal of Multivariate Analysis, 58, 197–229. Blum, J. R., Kiefer, J. & Rosenblatt, M. (1961). Distribution free tests of independence based on the sample distribution function. The Annals of Mathematical Statistics, 32, 485–498. Chatterjee, S. & Yilmaz, M. R. (1992). Chaos, fractals and statistics (with discussion). Statistical Science, 7, 49–121. Clayton, D. G. (1978). A model for association in bivariate life tables and its applications in epidemiological studies of familial tendency in chronic disease incidence. Biometrika, 65, 141–151. Cotterill, D. S. & Csörgő, M. (1982). On the limiting distribution of and critical values for the multivariate Cramér-von Mises statistic. The Annals of Statistics, 10, 233–244. Cotterill, D. S. & Csörgő, M. (1985). On the limiting distribution of and critical values for the Hoeffding, Blum, Kiefer, Rosenblatt independence criterion. Statistics and Decisions, 3, 1–48. Les Cahiers du GERAD G–2002–03 34 Csörgő, M. (1979). Strong approximations of the Hoeffding, Blum, Kiefer, Rosenblatt multivariate empirical process. Journal of Multivariate Analysis, 9, 84–100. Deheuvels, P. (1979). La fonction de dépendance empirique et ses propriétés. Un test non paramétrique d’indépendance. Académie royale de Belgique, Bulletion de la classe des sciences 5ième série, 65 274–292. Deheuvels, P. (1980). Nonparametric test of independence. In Nonparametric Asymptotic Statistics (Proc. Conf., Rouen, 1979), pp. 95–107. Lecture Notes in Mathematics No 821, Springer, Berlin. Deheuvels, P. (1981a). An asymptotic decomposition for multivariate distribution-free tests of independence. Journal of Multivariate Analysis, 11, 102–113. Deheuvels, P. (1981b). A non parametric test for independence. Publications de l’Institut de statistique de l’Université de Paris, 26, 29–50. Deheuvels, P. (1981c). A Kolmogorov-Smirnov type test for independence and multivariate samples. Revue roumaine de mathématiques pures et appliquées, 26, 213–226. Delgado, M. A. (1996). Testing serial independence using the sample distribution function. Journal of Time Series Analysis, 11, 271–285. Dugué, D. (1975). Sur des tests d’indépendance “indépendants de la loi.” Comptes rendus de l’Académie des sciences de Paris, série A, 281, 1103–1104. Feuerverger, A. (1993). A consistent test for bivariate dependence. International Statistical Review, 61, 419–433. Fisher, R. A. (1950). Statistical Methods for Research Workers, 11th Edition. Oliver and Boyd: London. Gänssler, P. & Stute, W. (1987). Seminar on Empirical Processes. DMV Seminar 9, Birkhäuser Verlag, Basel. Genest, C. & MacKay, R. J. (1986). Copules archimédiennes et familles de lois bidimensionnelles dont les marges sont données. The Canadian Journal of Statistics, 14, 145–159. Genest, C., Quessy, J.-F. & Rémillard, B. (2000). Testing for randomness against serial dependence using Kendall’s process. Submitted. Ghoudi, K., Kulperger, R. J. & Rémillard, B. (2001). A nonparametric test of serial dependence for time series and residuals. Journal of Multivariate Analysis, 79, 191– 218. Ghoudi, K. & Rémillard, B. (1998). Empirical processes based on pseudo-observations. In Asymptotic Methods in Probability and Statistics: A Volume in Honour of Miklos Csörgő, pp. 171–197 (B. Szyskowitz, ed.) Elsevier, Amsterdam. Ghoudi, K. & Rémillard, B. (2000). Empirical processes based on pseudo-observations II: the multivatiate case. Submitted. Gieser, P. W. & Randles, R. H. (1997). A nonparametric test of independence between two vectors. Journal of the American Statistical Association, 92, 561–567. Gumbel, E. J. (1960). Distributions des valeurs extrêmes en plusieurs dimensions. Bulletin de l’Institut de statistique de l’Université de Paris, 9, 171–173. Les Cahiers du GERAD G–2002–03 35 Hallin, M., Ingenbleek, J.-F. & Puri, M. L. (1985). Linear serial rank tests for randomness against ARMA alternatives. The Annals of Statistics, 13, 1156–1181. Hallin, M., Ingenbleek, J.-F. & Puri, M. L. (1987). Linear and quadratic serial rank tests for randomness against serial dependence. Journal of Time Series Analysis, 8, 409–424. Hallin, M. & M. L. Puri, M. L. (1992). Rank tests for time series analysis: A survey. New Directions in Time Series Analysis, pp. 112–153. (D. R. Brillinger, E. Parzen & M. Rosenblatt, eds.), Springer, New York. Hallin, M. & Werker, B. J. M. (1999). Optimal testing for semi-parametric AR models — from Gaussian Lagrange multipliers to autoregression rank scores and adaptive tests. Asymptotics, Nonparametrics, and Time Series, pp. 295–350 (S. Ghosh, ed.), Marcel Dekker, New York. Hoeffding, W. (1948). A non-parametric test of independence. The Annals of Mathematical Statistics, 19, 546–557. Hong, Y. (1998). Testing for pairwise serial independence via the empirical distribution function. Journal of the Royal Statistical Society, Series B, 60, 429–453. Hong, Y. (2000). Generalized spectral tests for serial dependence. Journal of the Royal Statistical Society, Series B, 62, 557–574. Jing, P. & Zhu, L.-X. (1996). Some Blum-Kiefer-Rosenblatt type tests for the joint independence of variables. Communications in Statistics — Theory and Methods, 25, 2127–2139. Kallenberg, W. C. M. & Ledwina, T. (1999). Data-driven rank tests for independence. Journal of the American Statistical Association, 94, 285–301. Kulperger, R. J. & Lockhart, R. A. (1998). Tests of independence in time series. Journal of Time Series Analysis, 19, 165–185. Littell, R. C. & Folks, J. L. (1973). Asymptotic optimality of Fisher’s method of combining independent tests II. Journal of the American Statistical Association, 68, 193–194. Nelsen, R. B. (1999). An Introduction to Copulas. Lecture Notes in Statistics No 139. Springer, Berlin. Oakes, D. (2001). Biometrika centenary: Survival analysis. Biometrika, 88, 99–142. Quessy, J.-F. (2000). Processus de Kendall sériel. Master’s Thesis No 18223, Université Laval. Romano, J. P. & Siegel, A. F. Counterexamples in Probability and Statistics. Wadsworth, London. Shih, J. H. & Louis, T. A. (1996). Tests of independence for bivariate survival data. Biometrics, 52, 1440–1449. Skaug, H. J. & Tjøstheim, D. (1993). A nonparametric test of serial independence based on the empirical distribution function. Biometrika, 80, 591–602. Spitzer, F. L. (1974). Introduction aux processus de Markov à paramètre dans Z ν . In École d’été de probabilités de Saint-Flour III-1973, pp. 115–189. Lecture Notes in Mathematics No 390 (A. Badrikian & P.-L. Hennequin, eds), Springer, New York. Les Cahiers du GERAD G–2002–03 36 Stute, W. (1984). The oscillation behavior of empirical processes: The multivariate case. The Annals of Probability, 12, 361–379. Tjøstheim, D. (1996). Measures of dependence and tests of independence. Statistics, 28, 249–284.