Why orthogonal rotations might be not so orthogonal as you think

Johan Westerhuis

Why orthogonal rotations might be not so orthogonal as you think

Johan Westerhuis

2017, Journal of Chemometrics

visibility

…

description

4 pages

link

1 file

Received: 31 March 2017 Revised: 16 May 2017 Accepted: 16 May 2017 DOI: 10.1002/cem.2920 GUEST COLUMN Why orthogonal rotations might be not so orthogonal as you think One of the properties of orthogonal rotation is that angles are preserved. That means that 2 orthogonal vectors stay orthogonal after rotation. To investigate this let us look at 2 vectors: 0 1 1 0 −3 1 B C B C h1 ¼ @ 12 Aand h2 ¼ @ 0 A: 3 1 These 2 vectors are perpendicular; this can be seen by inspecting their inner product: 1 −3 þ 12 0 þ 3 1 ¼ 0: An orthogonal rotation matrix in 2 dimensional space can be written as Q ¼ cosθ sinθ ,where θ is the angle of rota- − sinθ cosθ tion. Define the matrix H as a 3 × 2 matrix with the first column equal to h1 and the second column equal to h2. Multiplying matrix H with the rotation matrix Q (for θ = π/6) results in: 2 1 6 6 Hr ¼ HQ ¼ 4 12 3 2 1pﬃﬃﬃ 3 32 3 −3 1pﬃﬃﬃ 1 62 3 þ 2 6 7 62 3 pﬃﬃﬃ 2 7¼6 4 0 7 5 6 6 3 5 1 1pﬃﬃﬃ 6 4 3pﬃﬃﬃ 1 3 − 1 2 2 3− 2 2 3 3pﬃﬃﬃ 1 − 3þ 7 2 27 7 7: 6 7 p ﬃﬃ ﬃ 1 3 5 3þ 2 2 The length of each of the rows has not changed because of the rotation (eg, the second row before and after rotation has length √144) = 12. The angles between the rows have also not changed. Each row is rotated in the same way. For example, pﬃﬃ the angle between the second row before and after rotation equals θ ¼ cos−1 12126123 ¼ π=6 . Thus, the angles between the rows are not affected because of the rotation. The angle between the columns has changed. Taking the inner product of the 2 columns after rotation of this matrix yields pﬃﬃﬃ 36 3, which obviously is not equal to zero. The 2 column vectors are not perpendicular anymore after applying an orthogonal rotation! So what is going on here? As our colleague Richard Brereton has explained in one of his columns, a matrix has a row space and a column space.1 The rotation of matrix H rotates each row of H in the row space of H with an angle defined by θ. The properties of angle conservation hold for the row space of H. However, as we showed above, the orthogonality between the columns of H is destroyed. Thus, orthogonal rotations may not be so orthogonal as expected. This fact has implications for the rotation of, eg, PCA solutions and these will be discussed below. Journal of Chemometrics. 2017;e2920. https://doi.org/10.1002/cem.2920 wileyonlinelibrary.com/journal/cem Copyright © 2017 John Wiley & Sons, Ltd. 1 of 4 GUEST COLUMN 2 of 4 Some proofs: Consider a matrix A with 2 columns and I rows, (i, 1…I). 2 a11 a12 6 6 ⋮ 6 6 A ¼ 6 ai1 6 6 ⋮ 4 aI1 3 7 ⋮ 7 7 ai2 7 7 7 ⋮ 7 5 aI2 The rotation of this matrix can be written as 2 a11 6 6 ⋮ 6 6 Ar ¼ A Q ¼ 6 ai1 6 6 ⋮ 4 aI1 a12 2 3 a11 cosθ−a12 sinθ 6 ⋮ 6 6 6 ai1 cosθ−ai2 sinθ ¼6 6 cosθ 6 ⋮ 4 aI1 cosθ−aI2 sinθ 7 ⋮ 7 7 ai2 7 cosθ 7 7 − sinθ ⋮ 7 5 aI2 sinθ a11 sinθ þ a12 cosθ 3 7 ⋮ 7 7 ai1 sinθ þ ai2 cosθ 7: 7 7 7 ⋮ 5 aI1 sinsθ þ aI2 cosθ The inner product of the 1st column of Ar and the 2nd column of Ar equals 20 a11 cosθ−a12 sinθ 6B ⋮ 6B 6B 6B ai1 cosθ−ai2 sinθ 6B 6B 6B ⋮ 4@ aI1 cosθ−aI2 sinθ 1T 0 C C C C C C C A a11 sinθ þ a12 cosθ B ⋮ B B B ai1 sinθ þ ai2 cosθ B B B ⋮ @ aI1 sinθ þ aI2 cosθ 13 C7 C7 C7 I C7 C7 ¼ ∑ a2i1 cosθ sinθ−a2i2 cosθ sinθ þ ai1 ai2 cos2 θ−sin2 θ C7 i¼1 C7 A5 I I I ¼ ∑ a2i1 cosθ sinθ− ∑ a2i2 cosθ sinθ þ ∑ ðai1 ai2 Þ cos2 θ−sin2 θ : i¼1 i¼1 i¼1 This inner product is 0 if I I I i¼1 i¼1 i¼1 ∑ a2i1 ¼ ∑ a2i2 ; and ∑ ðai1 ai2 Þ ¼ 0 or in special cases of θ: • The inner product is also 0 if sinθ or cosθ = 0, and ∑Ii¼1 ðai1 ai2 Þ ¼ 0, • The inner product is also 0 if (cos2θ − sin2θ) = 0, which is at π/4, 3π/4, 5π/4, etc and I I i¼1 i¼1 ∑ a2i1 ¼ ∑ a2i2 : Concluding, only if A has orthogonal columns of equal length, then every rotation of A maintains the orthogonality between its columns. Rotations with angles of π/4 (or multiples if it) are special cases where only one of the conditions of orthogonality (π/2 or multiples of it), or equal length (at π/4, 3π/4, 5π/4) has to hold. IMPL ICATIO NS F OR “ORTHOGONAL” ROTAT I O N S O F P CA S C O R E S AN D LOADINGS In some occasions, orthogonal rotations of the PCA scores and loadings may provide better interpretable subspaces. The rotation is usually applied to the PCA loadings such that after rotation, many loading values have become small. In chemometrics, PCA is usually applied in such a way that the loadings are scaled to length 1 and the scores carry the variance of the component. GUEST COLUMN 3 of 4 Let us suppose, a PCA of matrix X(I × J) provided 2 scores T(I × 2) and loadings P(J × 2) with TTT = D and PTP = I, where D is a diagonal matrix and I the identity matrix. A rotation of the loadings requires also a rotation of the scores: X ¼ TPT þ E ¼ TQQT PT þ E: The rotated loadings (P*Q) maintain their orthogonality because the loadings have equal length, but the rotated scores (T*Q) lose their orthogonality because the lengths of the 2 score vectors were unequal (as they explained a different amount of variation). V I S UA L I Z I NG TH E E X A M P L E S The example matrix H is rotated orthogonally with the rotation matrix Q. The columns of H are orthogonal, but their length is unequal. Figure 1 visualizes the row space and the column space representation of this matrix before and after rotation. In the row space representation, we see that each row of the matrix (solid lines) is rotated over a constant angle toward the rotated version (dotted lines). The 2 columns of H and of Hr lie in a 2D subspace of the 3D column space. The columns of H are used here as the base vectors of that subspace with their length equal to the length in the 3D. The rotated columns are indicated FIGURE 1 Rotation of a matrix with orthogonal columns but unequal length. In the left figure, the row space representation shows the rows of matrix H before (solid) and after (dotted) rotation. The right figure shows a 2D subspace in the 3D column space that contains all columns of H and the rotated version of H FIGURE 2 Rotation of an orthogonal matrix with each column having length one. In the left figure, the row space representation shows the rows of matrix Hscaled before (solid) and after (dotted) rotation. The right figure shows a 2D subspace in the 3D column space that contains all columns of Hscaled and the rotated version of Hscaled GUEST COLUMN 4 of 4 by the dotted lines. It can be observed that the angle between the dotted lines is smaller than 90°, and thus, the rotated columns are not orthogonal. As the 2nd example, we take the same example matrix H, but now scale each column of H to length 1. 2 6 H scaled ¼ 6 4 1 pﬃﬃﬃﬃﬃﬃ 154 12 pﬃﬃﬃﬃﬃﬃ 154 −3 pﬃﬃﬃﬃ 10 0 pﬃﬃﬃﬃ 10 3 pﬃﬃﬃﬃﬃﬃ 154 1 pﬃﬃﬃﬃ 10 3 7 7 5 As shown in Figure 2, rotation of Hscaled matrix maintains the orthogonality between the columns. Now, in the right figure, we can see that the rotation did not change the angle between the 2 columns of Hscaled. CONCLUSION As self‐appointed fact checkers in chemometrics, we would qualify the use of the term orthogonal rotation matrix used in transforming PCA solution as factually correct but misleading. Huub C.J. Hoefsloot Frans M. van der Kloet Johan A. Westerhuis Biosystems Data Analysis, Swammerdam Institute for Life Sciences, University of Amsterdam, Amsterdam, Netherlands Correspondence Johan A. Westerhuis, Biosystems Data Analysis, Swammerdam Institute for Life Sciences, University of Amsterdam, Amsterdam, Netherlands. Email: [email protected] R E F E RENC E 1. Brereton RG. Visualizing matrices. Journal of Chemometrics. 2017;31(3):e2834. https://doi.org/10.1002/cem.2834 How to cite this article: Hoefsloot HCJ, van der Kloet FM, Westerhuis JA. Why orthogonal rotations might be not so orthogonal as you think. Journal of Chemometrics. 2017;e2920. https://doi.org/10.1002/cem.2920

Log In

Why orthogonal rotations might be not so orthogonal as you think

Related papers

Related papers

Related topics