Extendibility of spherical matrix distributions

JOURNAL OF MULTIVARIATE ANALYSIS Extendibility 8, 559466 (1978) of Spherical A. P. Matrix Distributions DAWID University College London, London, England* Communicated by D. A. S. Fraser We investigate the structure of distributions for matrices which can embedded in arbitrarily large matrices whose distributions have properties invariance under orthogonal rotations. be of 1. INTRODUCTION Let Y be a random (n x p) matrix. We recall the following definitions of Dawid [3]. We call Y or its distribution left-spherical,if, for every fixed orthogonal U (n x n), UY w Y, that is, UY and Y have the same distribution. We call Y right-spherical if its transpose Y’ is left-spherical, and sphericalif Y is simultaneouslyleft- and right-spherical. Now let %Sbe the set of symmetric (p x p) matrices, and yD C %7D the set of non-negative definite membersof VP . Let Y be a random element of ‘S?= and U a fixed (p x p) orthogonal matrix. Then UYU’ E %‘,,. We call Y or its distribution rotatabZeif UYU’ w Y for all U. This property is most commonly applied to caseswhere YE yp with probability one, when it will be termed 9’-rotatability. We note YE Ya G- UYU’ E 97 . A rotatable matrix is Y-rotatable if and only if its diagonalelementsare, with probability one, nonnegative. In this paper an attempt is made to characterize those left-spherical, rightspherical, spherical, and Y-rotatable distributions of matrices which can be consideredas submatricesof arbitrarily large arrays sharing the sameproperty. For ‘the first two cases,a simple representation in terms of mixed multivariate normal distributions is available. Characterizationsin the last two casesare shown to be equivalent, and a conjectured characterization is given, but a full theory is still lacking. Received October 13, 1977. AMS 1970 subject classification: Primary 62ElO. Key words and phrases: Spherical; rotatable; extendibility; matrix; normal distribution; exchangeability. * Now at The City University, London, England. presentability; random 559 0047-259X/78/0084-0559$02.00/0 All Copyright Q 1978 rights of reproduction by Academic Press, Inc. in any form reserved. 560 A. p. DAWID 2. SPHERICITE. AND NORMALITY LetZ=(+:l <i<n,l <j<p)b e a random matrix whose entries are independent standard normal variables. If Y = AZB, then the entries of Y are multivariate normal, with mean0, and cov( yij , yrJ = yi,.ui, , where r = AA’, Z = B’B. We shall write 2 N N(0; 1, , I,), Y - N(0; r, Z). Clearly UYV N(0; UIYJ’, VW). M oreover, the rows of N(0; 1, , Z) are independentN(0; 2) vectors, and similarly for the columnsof N(0; r, I,). Suppose Y - N(0; In , Z), and U (n x n) is orthogonal. Then UY N(0; UU’, Z) = N(0; 1, , Z). It follows that Y is left-spherical. Now suppose Y (n x p) (n > 2) is left-spherical, with independent rows. Then, for someZ E yV , Y - N(0; I,, , 2). This follows directly from Maxwell’s Theorem, the corresponding result for p = I [7, Section TII.41, applied to x (n x 1) = Yc, where c (p x 1) is arbitrary. Similarly, Y (n x p) (p 2 2) is right-spherical with independent columns if and only if Y - N(0; r, ID) for some r E 9, . In particular, if Y is spherical with independent elements, and (n, p) # (1, l), then Y - N(0; ~“1~ , I,) for someG. If Y EV, is rotatable, and the entries ( yii: i < j) are independent, then, for -A+, 269 and yu -JV(p, us) (i <j) [15]. In particular, no SOme I4 s,Yii Y-rotatable distribution can have this independenceproperty. 3. PFUZSENTABILITY Taking mixtures preservesleft- or right-sphericity. It follows that, if Z hasany distribution Ii’ over yP , and, given Z, Y - N(0; 1, , Z), then the marginal distribution of Y is left-spherical. If the distribution of Y can arisein this way, we call Y row-presentable.We define column-presentabilityof a right-spherical distribution in parallel fashion. If Y = XA, where X - N(0; 1, , I,) independently of A, then, given A, Y - N(0; 1, , Z) (Z = A’A), whence it follows that Y - N(0; 1, , Z) given only Z, so that Y is row-presentable. Conversely, suppose, given 2, Y - iV(0; 1, , 2). Then clearly, Y w XA, where X - N(0; 1, ,I,) independently of A, and A has a distribution such that A’A M z1; for instance we can take A to be the symmetric squareroot of Z. We remark that if Y = XA, then Y’Y = A’X’XA, and XX hasthe Wishart distribution IV(n; I,). So, given 2, Y’Y - IV(n; Z). If 7t > p and Y(n x p) is row-presentable,then the distribution 17 of C may be uniquely recovered from that of Y. This is a consequenceof [l, Corollary 31. Now the joint characteristic function of the rows ( y, ,..., y,J of Y is SPHERICAL MATRIX = E{exp - tr(ZS)} 561 DISTRIBUTIONS with S = i u&. k=l For n > p S ranges over gP, and so as a by-product we find that a distribution over yP is uniquely determined by its “Laplace transform” [IO] I+% 9?? -+ [w given by #(S) = E{exp - tr(ZS)}. For n < p the distribution 17 is not generally determined by that of Y. However, let B be (Q x p) (Q < n) and consider @ = BZB’. Then @ E yQ, and, for SE 9*, E{exp - tr(@S)} = E{exp - tr(ET)} with T = B’SB. But T E 9’P and rank T < 8. It follows that T may be expressed as CF==, u& , and so the distribution of Y determines the Laplace transform, and hence the distribution, of @. In particular, the distribution of a principal (n x n) submatrix of .Z is determined. As an example of nonuniqueness, take n = 1, p > 1. Then the distribution of Y = $ only depends on the marginal distributions of the quadratic forms u’Zu. The two different distributions (i) .Z = ~1, , 7 N xy2 and (ii) Z N W,(V; 1,) both yield U’ZU N (u’u)xy2, and so give identical distributions for yr . Mitra [14] displays yet another distribution for Z with the same property. A similar result holds for the inverse x2 and inverse Wishart distributions for Z, both of which produce a multivariate t distribution fory, [5, Section 13.61. Sphericity and presentability. A concept parallel to presentability for the spherical case is none too obvious. Instead, we may demand both row- and column-presentability simultaneously. However, the following analysis shows that, for the spherical case, these two concepts are effectively equivalent. Let Y (n x p) be spherical and row-presentable, so Y M XA, with X N N(0; I,, I,) independently of A. We suppose n < p. Now consider Y* = X*AlJ, with X*, A, U all independent, X* N N(0; ID, ID,), and U having the uniform distribution B,9 given by the Haar measure over the set of Y (p x p) orthogonal matrices [3]. Then Y has the same distribution as the first n rows of X*A, and hence, by right-sphericity, as the first n rows of Y*. Moreover, Y* (p x p) is clearly spherical, so that, by [3, Theorem I], Y*’ M Y*. Since Y* is row-presentable, it is therefore also column-presentable, and with the same covariance distribution n as in its row-representation. Restricting attention to the first n rows of Y* yields the following result. 562 A. P. DAWID THEOREM 1. Let n < p, and let Y (n x p) be spherical Then Y is column-presentable. and row-presentable. Remark. Suppose the row-presentation of Y is: Given Z, Y N N(0; I, , Z), with.EwII over Sp, . It is seen from the above discussion that II may be taken to be rotatable (2 = U’A’AU), although even then, if n < p, the distribution 17 need not be determined uniquely by that of Y. Now, taking l7 to be rotatable, denote by n, the distribution of a principal (n x n) submatrix of Z when 22 N n. We see that the column-presentation of Y is: Given r, Y N N(0; I’, lP) with r N l7, . Note that I& is uniquely determined, and is rotatable. 4. EXTENDIBILITY If any of the properties of (left-) (right-) sphericity or (y-) rotatability holds for a matrix Y, it also holds for any submatrix of Y, or any principal submatrix in the case of (9-) rotatability. This hereditary property allows us to extend the definitions to infinite arrays. If ?V is a (N x P) array, where either N or P, or both, may be infinite, denote the leading (n x p) subarray of g by Y,,, . Then g is said to be left-spherical, right-spherical, or spherical, according as the same property holds for every finite subarray Y,,, . Let V, be the set of doubly infinite symmetric arrays. For Z E V, , we denote by ZD the leading (p x p) principal submatrix of 22. We define 9a = {Z E g,,,: ZD E & for all p}. Then a random matrix Z over Um(9m) is said to be rotatable (Y-rotatable) if the same property holds for every 2$ . Note that the distribution of an infinite array g, if considered as defined over the u-field generated by the cylinder-sets, is fully determined by those of its finite submatrices. Moreover, if we are given a family {A,} of distributions for the various finite submatrices {YJ; and if these are consistent in the sense that, when Y, is a submatrix of Yn , then the marginal distribution of Y,, , calculated fundamental theorem [13, Secwhen Y,, N A, , is A,; then, by KolmogorofI’s tion III.41 we can indeed construct a distribution for g for which YA N A, , all A. Clearly, it is enough to consider the distributions of all leading submatrices, or, if I is (co x co), all leading principal submatrices. If Y (n x p) is left-spherical, and there exists a left-spherical g (co x p) such that Y m Y,,, , we call Y row-extendible, and similarly define columnextendibility of a a right-spherical distribution. For spherical and (Y-) rotatable distributions, extendibility means the existence of a doubly infinite array, with the same property, having a subarray with the given distribution. If Y (n x p) is row-presentable, we can generate Y as follows: First generate a random matrix .Z from an appropriate distribution over Sp, , and then generate the rows of Y independently from N(0, 22). This construction may be extended to an infinite number of rows, so that Y is row-extendible. SPHERICAL MATRIX DISTRIBUTIONS 563 The converse result, row-extendibility * row-presentability, has been shown in the casep = 1 by Freedman [8], Kelker [ll], and Kingman [12]. Kingman’s proof is easily adapted for general p. THEOREM 2. Let Y (n x p) be left-spherical and row-extendible. Then Y is row-presentable. Proof. Consider a left-spherical $ (co x p) which extends Y. If $ has rows y1I , yZI ,..., these rows are exchangeable,and hence, by de Finetti’s theorem [4], there exists a u-field $ such that the { yi} are independent and identically distributed, given $. Let xi = c’yi for some fixed c (p x 1). Then 9 = $c = (Xl , x2 Y> ( co x 1) is left-spherical and the {xi} are, given 2, independent and identically distributed. As in Kingman [12], it follows that, given $, the {xi) are, with probability one, normally distributed with meanzero and commonvariance. Since this holds for all rational c, the result follows. 5. SPHERICITY AND EXTENDIBILITY Theorem 2 provides a complete characterization of left-spherical distributions over an (co x p) array $ (p < co): Take an arbitrary distribution n over 9P , let Z N l7 and, given Z, let the rows of $ be independent N(0, Z). This extends simply to the casep = co, by considering subarrayswith p < co. The analog for right-spherical distributions is obvious. In particular there is a one-one correspondence between all distributions over 9*, and all left-spherical distributions for $ (co x p). For sphericaland Y-rotatable (co x co) arrays, the characterization problems are much more subtle. However, the two problems may be shown to collapse into one as follows. Let Z (co x co) be randomly distributed accordingto a Y-rotatable distribution 17. Equivalently, we have a consistentsequence(17,: Y = 1,2,...) of rotatable distributions for (Zr: T = 1,2,...), with ,Zr E 9r. For each (n,p), define a distribution A,,, for Y (n x p) by: Y N N(0; I,, , Z9) given ZP and ZP N 17,. Then the distributions (A,,J are sphericaland consistentand sodefine a spherical distribution A for $ (co x co). Furthermore, from Theorem 1 and the subsequent remark, we seethat the consistent set of distributions for the Y’s may equally be expressedin the column-presentableform: Y N N(0; r, ,I,) given l-‘,,,withr,,~i&,. Now start with a sphericaldistribution A for $(a x co), or equivalently the consistent spherical set (A,,$ By Theorem 2 each Y is row-presentable, and, taking n > p, we get a uniquely defined distribution II, over 9V yielding the row-presentation. Then the distributions (fl,) are rotatable and consistent, and so determine a rotatable distribution lI over Sp, . Working instead with the column-presentability of $, we get the samedistribution 17. Thus, while the 564 A. P. DAWID need to consider either row- or column-presentability of Ug appears to introduce an undesirable asymmetry between rows and columns, the symmetry is nevertheless restored. The problem of characterizing infinite spherical or Y-rotatable distributions remains unsolved. One approach might be to note that a spherical (rotatable) matrix distribution is fully determined by that of the singular values (eigenvalues) of the matrix, and to investigate the behavior of these distributions as larger and larger matrices are considered. Much valuable work in this area has been done by Wachter [17, 181. An alternative approach is as follows. We first note the following result. THEOREM 3. If Y(p x p) is rotatable, then the distribution of Y is uniquely determinedby that of its diagonal. Proof. The characteristic function of Y is4(S) = E[exp{i tr( YS)}], where we may take S E g9. Now write S = PAP’, with P orthogonal and A diagonal. Then tr( YS) = tr( YPAP’) = tr(P’YPA) = tr(Yfl), since Y is rotatable, =xL A, YKk. So 4(S) = 4(h), where # is the characteristic function of (Yn 3 Y22 9-*-Y ykk) and h contains the eigenvaluesof S. The result follows. A similar result holds for spherical Y, as follows on using the singular value decompositionfor arbitrary S. Now let 17 be an infinite rotatable distribution for Z, with distributions I7, for (n x n) principal submatricesZ,, , and let P, denote the distribution of the diagonal of Z,, . Then II, may be recovered from P,, . Clearly the {P,> are consistent and hence define a joint distribution P for the infinite sequence (u111 >(522 > 033 ,...). Moreover, this distribution is clearly exchangeable.It therefor follows [4] that P can be expressed as a mixture of distributions in each of which the (uii) are independent and identically distributed; indeed the (Q) may be taken to have this property conditional on the tail u-field of the diagonal.Since this tail u-field is invariant under the finite rotations that leave the distribution of Z invariant, the conditional distributions of Z are likewise rotatable, and determined by those of the diagonal. Hence we only need to characterize those infinite (9-) rotatable distributions for which the diagonal elementsare independent and identically distributed. The general casefollows on taking mixtures of these. It might be thought that, for Y-rotatability in this restricted class, any distribution for uii over the nonnegative line was allowable: but this is not so. Let uii N xv2 independently, and consider the problem of finding a (p x p) Y-rotatable matrix .ZDwith diagonal(on ,..., u,,). The characteristic function of (f-311 ,.**, u,,) is I#) = JJlsl(l - 2ihx)-y/2, so that, following the proof of Theorem 3, that of .Z’-would haire to be E[exp{i tr(.Z,,S)}] = 1I, - 2iS I--V/~ (S E wp). But, for p > 1, this cannot be a characteristic function for arbitrarily small v > 0, for this would imply the infinite divisibility of the Wishart’ distribu- SPHERICAL MATRIX DISTRIBUTIONS 565 tion W( 1; I,), in which v = 1; and this is contradicted by the results of Shanbhag [16]. If v is an integer or v > p - 1, we can produce a distribution, namely the Wishart distribution W(v; I,), having characteristic function 11, - 2iS J-V/~[6, Chap. 81. Eaton conjectures that this is not possible if v < p - 1 is not an integer, and the above argument lends some support to this conjecture. If it is true, then, sincep is arbitrary, it would follow that any infinite Y-rotatable distribution with independent xy2 distributions for the diagonal entries would have to have v an integer. Such a distribution having (p x p) submatrices ZD - W(v; ID) does exist, and may be termed the (infinite-dimensional) Wishart distribution W(v; Im). However large p may be, rank (Z D) ,< V, so that this is essentially a singular finite-dimensional distribution. We can regard W(v; Im) as an independent sum Es=1 W, , with W, N W( 1;I,). The infinite spherical distribution corresponding to Z - W(l; Im) may be represented as having entries yu = x,z, , in which the x’s and a’s are all indepenJent standard Normal. Note that, given the z’s, Y,,, - N(0; 1, , Z,), where 2$ = (Ziaj), so that indeed ZV - W( 1; 1,). Likewise, the infinite spherical di&ibution corresponding to Z - W(v; 1,) can be represented as having entries yij = C’,=i xilcajk , where again the x’s and x’s are all independent standard Normal. We can construct more infinite y-rotatable distributions with independent diagonal elements by taking infinite independent sums of the form with W, w W(l;I,), and the h’s constant, A, > 0, A, > A, > ... > 0, ET=‘=, hk < co. Then uii -A, + z,“=, A,+, , where tiy - xl2 independently. If an infinite number of A’s are positive, Za will (with probability one) have full rank for all p. The corresponding infinite spherical distribution may be represtandard Normal sented by yij = &Sij + Xc”=, &+zjn: , with independent 5’s, x’s, and 2s. I conjecture that the above form yields all infinite 9’-rotatable distributions with independent diagonal entries, and their spherical counterparts. If so, then the general form follows on allowing the X’s to have a joint distribution, independent of the other variables. One construction that may be shown to yield Y-rotatable distributions of the above form is formation of a compoundWishart distribution [2]. Let Y (03 x co) have a Y-rotatable distribution, and, for fixed integral p, define a distribution for Z by: Given Y, & - W(p; ‘y,). These distributions are consistent and y-rotatable. They may be considered derived as follows. Take the spherical infinite array ?V corresponding to Y. We can consider Y,,, - I?(O; ul, , I,) given Y, and so Z,, w Y,,,Yh,, . But we can also consider Y,,, m X,&l, where X,,, N N(0; I, , I,) inde683/8/4-6 566 A. P. DAWID pendently of A (p x p), and A’A m lu, . Thus Z* e X,,,AA’Xi,, . Now let AA’ = PAP’ be the eigenvalue decomposition of AA’, where we can take P, A independent of X,,, . Then, given (P, A), Xn,pAA’XL,p = X,,,PAP’X~,, w is right-spherical. Thus, given A, ZS M C& hkWk , -G,,f.J-K,, , since X,,, where W, = xkx; , xk being the kth column of X,,, . Thus W, - W(l; I,) independently, and we have exhibited the appropriate representation. The distribution of cbr hRWk for fixed A’s has been considered by Hayakawa [9]. Here the h’s are random, and in fact their joint distribution is just that of the eigenvalues of YP . REFERENCES [I] [Z] [3] [4] [S] [6] [7’j [8] [9] [lo] [1 l] [12] [13] [14] 1151 [16] [17] [IS] BARNDORFF-NIELSEN, 0. (1965). Idenffiability of mixtures of exponential families. J. Math. Anal. A#. 12 115-121. BECKER, P. J. AND Roux, J. J. J. (1976). Compound distributions on the Wishart matrix. South African Statist. 1. 10 63-68. DAWID, A. P. (1977). Spherical matrix distributions and a multivariate modei. J. Roy. Statist. Sot. Ser. B 39 254-261. DE FINETTI, B. (1937). Foresight: its logical laws, its subjective sources (in Frenkch). English translation in Studies in Subjective Probability (H. E. Kyburg and H. E. Smokler, Eds.), pp. 93-158. Wiley, New York, 1964. DBILIPSTER, A. P. (1969). Elements of Continuous Multiwariute Analysis. AddisonWesley, Reading, Mass. EATON, M. L. (1972). Multivariate Statistical Analysis. Institute of Mathematical Statistics, University of Copenhagen. FRLLER, W. (1966). An Introduction to Probability Theory and its Applications, Vol. II. Wiley, New York. FREEDMAN, D. A. (1963). Invariants under mixing which generalize de Finetti’s theorem: continuous time parameter. Ann. Math. Statist. 34 1194-1216. HAYAKAWA, T. (1966). On the distribution of a quadratic form in a multivariate normal sample. Ann. Inst. Statist. Math. 18 191-201. I-Ixxz, C. S. (1955). Bessel functions of matrix argument. Ann. of Math. 61 474-523. KELKER, D. (1970). Distribution theory of spherical distributions and a location-scale parameter generalization. Sankkyn Ser. A 32 419-430. KINGMAN, J. F. C. (1972). 0 n random sequences with spherical symmetry. Biometrika 59 492-494. KOLOMOGOROFF, A. (1933). Gruna%egr@Je der Wahrscheinlichkeitsrechmmg. SpringerVerlag, Berlin. MITRA, S. K. (1969). Some characteristic and non-characteristic properties of the Wishart distribution. Sankhya Ser. A 31 19-22. OLSON, W. H. AND UPPULURI, V. R. R. (1970). Characterization of the distribution of a random matrix by rotational invariance. Sank&y2 Ser. A 32 325-328. SHANBHAG, D. N. (1976). On the structure of the W&art distribution. J. Multivariate Anal. 6, 347-355. WACHTER, K. W. (1974). Exchangeability and asymptotic random matrix spectra. In Progress in Statistics (J. Gani, K. Sarkadi, and I. Vince, Eds.), pp. 895-908. North-Holland, Amsterdam. WACHTW, K. W. (1978). The strong limits of random matrix spectra for sample matrices of independent elements. Ann. Probability 6 1-18.