The Statistician (1997)

46, No. 1, pp. 71–79

Local influence in elliptical linear regression models

By MANUEL GALEA,
Universidad de Valparaı́so, Chile

GILBERTO A. PAULA{ and HELENO BOLFARINE

Universidade de São Paulo, Brazil

[Received May 1996. Revised October 1996]

SUMMARY
Influence diagnostic methods are extended in this paper to elliptical linear models. These include several symmetric
multivariate distributions such as the normal, Student t-, Cauchy and logistic distributions, among others. For a
particular perturbation scheme and for the likelihood displacement the diagnostics agree with those developed for the
normal linear regression model by Cook when the coefficients and the scale parameter are treated separately. This
result shows the invariance of the diagnostics with respect to the induced model in the elliptical linear family.
However, if the coefficients and the scale parameter are treated jointly we have a different diagnostic for each induced
model, which makes this approach helpful for selecting the less sensitive model in the elliptical linear family. An
example on the salinity of water is given for illustration.
Keywords: Diagnostic; Influence; Likelihood displacement; Multivariate symmetric distributions

1. Introduction
Diagnostic techniques for normal linear regression models have been extensively studied in the
statistical literature. See, for example, Belsley et al. (1980), Cook and Weisberg (1982) and
Chatterjee and Hadi (1988). Several of the diagnostic techniques evaluate the effect of deleting
observations on parameter estimates. An alternative approach that assesses the influence of small
(local) perturbations from the assumed model on key results is considered in Cook (1986).
Additional results on local influence and applications can be found in Beckman et al. (1987),
Lawrance (1988), Thomas and Cook (1990), Tsai and Wu (1992), Paula (1993) and Kim (1995).
The method of local influence was proposed by Cook (1986, 1987) as a general tool for
assessing the effect of local departures from model assumptions. In this paper, the local influ-
ence approach is applied to elliptical linear regression models, i.e. when the error vector has
an elliptical distribution. The perturbation scheme considered here is the scheme in which the
scale parameter is modified to allow convenient perturbations in the model.
In Section 2, along with the notation, the elliptical linear models are defined. The local
influence method is reviewed in Section 3. Section 4 deals with the derivation of the
diagnostic procedures for the elliptical linear models. An illustrative example is given in the
last section.

2. Elliptical linear models

The class of elliptical distributions has been of considerable interest in the recent statistical

{ Address for correspondence: Instituto de Matemática e Estatı́stica, Universidade de São Paulo, Caixa Postal 66281
(Agência Cidade de São Paulo), 05315-970 São Paulo, Brazil.
E-mail: [email protected]

& 1997 Royal Statistical Society 0039–0526/97/46071

72 GALEA, PAULA AND BOLFARINE

literature. See, for example, Fang et al. (1990), Fang and Zhang (1990) and Fang and
Anderson (1990). An n 3 1 random vector Y has an elliptical distribution with location vector
µ and scale positive definite matrix Λ, if its density takes the form
f Y ( y) ˆ jΛjÿ1 2 g f( y ÿ µ)T Λÿ1 ( y ÿ µ)g,
=
(1)
y 2 R , where the function g: R ! [0, 1) is such that
n

…1
u nÿ1 g(u 2 ) du , 1:
0

The function g is typically known as the density generator. For a vector Y distributed according
to density (1), we use the notation Y  El n (µ, Λ, g) or, simply, Y  El n (µ, Λ). When µ ˆ 0 and
Λ ˆ I, we obtain the spherical family of densities. This class of symmetric distributions
includes the normal, Student t-, contaminated normal and logistic (both, univariate and
multivariate) distributions, among others, as considered, for example, by Fang et al. (1990).
Table 1, taken from Fang et al. (1990), reports examples of distributions in the elliptical family.
The notation c1 , c2 , c3 and c4 is used to denote normalizing constants.
Consider now the linear regression model
Y ˆ X β ‡ E, (2)
where Y is an n 3 1 vector of responses, X is a known n 3 p matrix of rank p, β is a p-
dimensional vector of parameters and E is a p-dimensional error vector with distribution
El n (0, φI), where φ is the scale parameter. Thus, it follows that Y  El n (X β, φI). This is
typically called the elliptical linear regression model. If g is a continuous and decreasing
function then the maximum likelihood estimators of β and φ are given by (see Fang and
Anderson (1990))
^
⠈ (X T X )ÿ1 X T Y ,
^ ˆ Q(β)
^
(3)
φ = ug ,

where Q(β) ˆ (Y ÿ X β) (Y ÿ X β) and u

T
g maximizes the function
h(u) ˆ u n=2
g(u), u > 0: (4)
Typically, if g in equation (4) is continuous and decreasing then its maximum ug exists and is
finite and positive. Moreover, if g is continuous and differentiable then ug is the solution to
(Fang and Anderson, 1990)
g9(u) ‡ g(u) ˆ 0
n
2u
or, equivalently, the solution of the equation
n
2u
‡ Wg (u) ˆ 0, (5)

TABLE 1
Multivariate elliptical distributions

Distribution Notation Generating function

Normal N n (µ, Λ) g(u) ˆ c1 exp (ÿ u=2), u > 0
Student t tn (µ, Λ, ν) g(u) ˆ c2 (1 ‡ u=ν)ÿ(ν‡ n) 2
=

Contaminated normal CN n (µ, Λ, δ, τ) g(u) ˆ c1 f(1 ÿ δ) exp (ÿ u=2) ‡ δτÿ n 2 exp (ÿ u=2τ)g
=

Cauchy Cn (µ, Λ) g(u) ˆ c3 (1 ‡ u)ÿ( n‡1) 2

=

Logistic Ln (µ, Λ) g(u) ˆ c4 exp (ÿ u)=f1 ‡ exp (ÿ u)g2

 LOCAL INFLUENCE 73

where
dflog g(u)g g9(u)
W g (u) ˆ ˆ g(u) :
du
It is easy to see that, for the normal and t-distributions, ug ˆ n. However, for the contaminated
normal and logistic distributions, ug must be obtained numerically. For the logistic distribution,
for example, equation (5) becomes

n
2u
ˆ tanh
u
2
,

where tanh (:) denotes the hyperbolic tangent.

3. Local influence on likelihood displacement

Let L(θ) denote the log-likelihood function from the postulated model (here θ ˆ (βT , φ)T ),
and let ω be a q 3 1 vector of perturbations restricted to some open subset ω 2 R q . The
perturbations are made on the likelihood function, such that it takes the form L(θjω). Denoting
the vector of no perturbation by ω0, we assume that L(θjω0 ) ˆ L(θ). To assess the influence of
the perturbations on the maximum likelihood estimate θ, ^ we may consider the likelihood dis-

placement
LD(ω) ˆ 2f L(θ)
^ ÿ L(θ
^ ) g,
ω

ω denotes the maximum likelihood estimate under the model L(θjω).

where θ^

Small perturbations to the model may be important, especially when assessing whether the
sample is robust with respect to the induced model. To assess this kind of robustness, Cook
(1986) suggested studying the local influence around ω0 . The idea consists of studying the
normal curvature of the surface α(ω) ˆ (ωT , LD(ω))T and then taking the direction around ω0
corresponding to the largest normal curvature.
Cook (1986) showed that the normal curvature in the direction l takes the form
Cl (θ) ˆ 2j lT ∆T ( L)ÿ1 ∆lj, (6)
where i li ˆ 1, ÿ L is the observed information matrix for the postulated model (ω ˆ ω0 ) and ∆
is the ( p ‡ 1) 3 q matrix with elements
L(θjω)
ˆ
2
@
∆ ij ,
@ θi @ ω i
evaluated at θ ˆ θ^ and ω ˆ ω , i ˆ 1, . . ., p ‡ 1 and j ˆ 1, . . ., q.
0
Therefore, the maximization of equation (6) is equivalent to finding the largest eigenvalue
Cmax of the matrix B ˆ ∆T ( L)∆, and the largest direction around ω0, denoted by lmax, is the
corresponding eigenvector. If Cmax is much greater than the remaining eigenvalues of B, the
index plot for lmax may be helpful in assessing the influence of small perturbations on LD(ω).
Otherwise, it should be more informative to perform the index plot for the eigenvectors
corresponding to the largest eigenvalues.

4. Curvature derivation for elliptical linear models

Firstly, we consider the unperturbed model. The log-likelihood function under the induced
model is given by
74 GALEA, PAULA AND BOLFARINE

L(θ) ˆ log [jφI n jÿ1 2 gf( y ÿ X β)T (φI n )ÿ1 ( y ÿ X β)g]

=

ˆ ÿ 2n log φ ‡ log g(u),

where u ˆ Q(β)=φ and θ ˆ (βT , φ)T . Then, the first derivatives of L(θ) with respect to β and φ
are
@ L(θ)
@β
ˆ ÿ φ2 Wg (u)X T ( y ÿ X β) (7)

and
@ L(θ)
@φ
ˆ ÿ 2φ
n
ÿ φ12 Wg (u) Q(β): (8)

From equations (7) and (8) it follows, after some algebraic manipulation, that
 
ˆ φ2 W g (u)(X T X ) ‡ W 9g (u)X T ( y ÿ X β)( y ÿ X β)T X ,
2
@ L(θ) 2
@ β@ β φ
T

ˆ φ2 fW (u) ‡ W 9 (u) Q(β)g( y ÿ X β)

2
@ L(θ) T
g g X,
φ@ βT 3
 
@

ˆ 2φn ‡ Q(β) 2W g (u) ‡ W 9g (u)

2
@ L(θ) Q(β)
,
@φ φ φ
2 2 3

where

W 9g (u) ˆ
dW g (u)
:
du
Evaluating these derivatives at θ ˆ θ,
^ given in equations (3), and by noting that ( y ÿ X β)
^ T
X
ˆ 0 and Q(β)^ ^ ˆ u , we have
=φ
02 1
g

B W g ( u)(X T X )^ 0 C
LˆB C
φ ^

@ 0
n
‡ ug
f 2W g ( u) ‡ ug W 9g ( u)g
^
A, ^
^
2φ 2 2 ^
φ
where ^u ˆ Q(β)
^ ^ ˆ u .
=φ g
Table 2 shows W g (u) and W 9g (u) for the distributions given in Table 1, where
f i (u) ˆ (1 ÿ δ) exp (ÿ u=2) ‡ δτÿ( n 2)ÿ i exp (ÿ u=2τ), =
i ˆ 0, 1, 2:
Note that for the normal case (ug ˆ n, W (u) ˆ ÿ 1
and W 9g (u) ˆ 0) the matrix L reduces to
 
g 2

Lˆ
ÿ(1 φ)(X X ) =^
T
0
ÿ n=2φ^ 2
 :
0
Consider now model (2), with the assumption that E  El n f0, φ Dÿ1 (ω)g, where D(ω) ˆ
diag(ω1 , . . ., ω n ) and Dÿ1 (ω) denotes the inverse of D(ω). Here q ˆ n and ω i is the weight
corresponding to the ith case, i ˆ 1, . . ., n. When ω ˆ ω0 ˆ 1, the perturbed model reduces to
the postulated model. Under the perturbed model we shall denote Y  El n f X β, φ Dÿ1 (ω)g.
Thus, the log-likelihood function is given by

L(θjω) ˆ ÿ log φ ‡ 12 log j D(ω)j ‡ log g(uω ),

n
2
 LOCAL INFLUENCE 75
TABLE 2
Wg (u) and W9g (u) for some elliptical distributions

Distribution W g (u) W 9g (u)

Normal ÿ 1
2
0
Student t ÿf(ν ‡ n) 2νg(1 ‡ u ν)ÿ
= =
1
f(n ‡ ν) 2ν g(1 ‡ u ν)ÿ
=
2
=
2

Cauchy ÿf(1 ‡ n) 2g(1 ‡ u)ÿ

=
1
f(1 ‡ n) 2g(1 ‡ u)ÿ
=
2

" f 2 #
Contaminated normal ÿ 12 ff 1 (u) 1 f 2 (u)
ÿ (u)
1

0 (u) 4 f 0 (u) f 0 (u)

Logistic ÿtanh (u =2) ÿ2 fexp (u
= = 2) ‡ exp (ÿ u=2)g2

where uω ˆ Qω (β)=φ and Qω ˆ ( y ÿ X β)T D(ω)( y ÿ X β).

Following the same procedure as for the unperturbed case we have
L(θjω)
@

@β
ˆ ÿ 2
φ
W g (uω )f X T D(ω) y ÿ X T D(ω)X βg,

L(θjω)
@

@φ
ˆ ÿ 2φ
n
ÿ φ1 W (u 2 g ω ) Qω (β):

From these equations we obtain

L(θjω)
 
ˆ ÿ φ2 Wg (uω )X T D(E) ‡ φ1 W 9g (uω)X T D(ω)ET D(E) ,
2
@

@ β@ ω
T

L(θjω)
 
ˆ ÿ φ1 W g (uω )ET D(E) ‡
2
@ 1
W 9g (uω )Qω (β)ET D(E) ,
@ φ@ ω φ
T 2

since @ f X T D(ω)Eg=@ ωT ˆ X T D(ω) and @ uω =@ ωT ˆ (1 = φ)ET D(E), where E ˆ y ÿ X β. Evaluat-

ing the matrix ∆ at (θ, ω) ˆ (θ,
^ ω ) we find
0
0 1
B ÿ 2 W (u)X g ^
T
D(e) C
∆ˆB C
^
φ
@ A,
ÿ ^ 2 fWg (^u) ‡ ug W 9g (^u)ge T D(e)
1
φ
where e ˆ y ÿ X β.
^
For the normal linear case the matrix ∆ reduces to
 
∆ˆ
X T D(e)=φ
^
,
e D(e)=2φ2
T ^

which is in agreement with the expression obtained by Cook (1986). Therefore, we may write
B ˆ ∆T Lÿ1 ∆ ˆ B1 ‡ B2 ,
where

B1 ˆ 2 W (u) D(e)P D(e)

^
g ^
φ
76 GALEA, PAULA AND BOLFARINE

and

ˆ 1 fWg (^u) ‡ ug W 9g (^u)g2 D(e)ee T D(e),

^ 2 [(n=2) ‡ ug f2 W g ( ^
u) ‡ ug W 9g (^u)g]
B2
φ
with P ˆ X (X T X )ÿ1 X T . Also, for the normal linear case, the matrix B reduces to the matrix
obtained by Cook (1986), equation (31). Then, the normal curvature in the direction l takes the
form
Cl (θ) ˆ 2j l T (B1 ‡ B2 )lj:
In particular, if we are interested in the vector β, the normal curvature in the direction l yields
Cl (β) ˆ 2j l T B1 lj

^
ˆ 4 jW (u)l
g ^
T
D(e)P D(e)lj
φ

ˆ 4^ jWg (^u)j j l T D(e)P D(e)lj:

φ
Thus, the index plot for lmax of the matrix D(e)P D(e) may show how to perturb the scale
parameter to obtain larger changes in the regression coefficients.
For a particular coefficient, namely β1, rearranging the columns of X ˆ (X 1 , X 2 ), such that
X 1 and X 2 are matrices with dimensions n 3 1 and n 3 ( p ÿ 1) respectively, it follows from
expression (33) of Cook (1986) that
4j W g (^u)j j l T D(e)rr T D(e)lj
Cl (β1 ) ˆ ,
i ri 2 φ
^

where
r ˆ (I ÿ P )X , 2 1

P2 ˆ X (X X )ÿ X
2
T
2 2
1 T
2

and iai denotes the norm of the vector a. Thus, the maximum curvature occurs in the direction
lmax / D(e)r :

Accordingly, the cases with j ri ei j large are locally most influential on the estimate β
^
1.
Similarly, the normal curvature for the scale parameter φ in the direction l is given by
Cl (φ) ˆ 2j l T B2 lj

ˆ 2 jC j j l
^2
ω
T
D(e)ee T D(e)lj,
φ
where

ˆ n 2 ‡fWu f(u2)W‡ (uu)W‡9u(u)Wg 9 (u)g

2
g ^ g g ^
Cω :
= g g ^ g g ^

Here, for the largest curvature,

lmax / D(e)e,
which means that the observations with large values for e2i are most influential on φ.
^

Therefore, at least for the perturbation scheme defined in Section 4 and for the likelihood
displacement, we may conclude that the diagnostics for the elliptical linear models are
equivalent to those deduced by Cook (1986) for the normal linear model when β and φ are
treated separately, i.e. the index plots do not change with the induced model in the elliptical
 LOCAL INFLUENCE 77

linear family. However, if β and φ are treated jointly, the lmax -vector may change from one
model to another, which suggests a helpful way of discovering those observations that are most
locally influential under each model.

5. Water salinity
To illustrate the methodology described in this paper we consider the data set reported by
Ruppert and Carroll (1980) on the salinity of water during the spring in Pamlico Sound, North
Carolina. The response Y is biweekly salinity, and the explanatory variables are salinity lagged
2 weeks, x1 , a dummy variable x2 for the time period and river discharge, x3 . The value of x1i
may differ from yiÿ1 , once the data are not a contiguous sequence. This data set has been
analysed, for instance, by Atkinson (1985), Carroll and Ruppert (1985) and Davison and Tsai
(1992). Atkinson (1985) assumed a normal distribution for the response whereas Davison and
Tsai (1992) considered a Student t-distribution with 3 degrees of freedom to allow for the
possibility that the data have tails that are longer than for the normal distribution. In both
cases, the linear model
Yi ˆβ ‡β x ‡β x ‡β x ‡E
0 1 1i 2 2i 3 3i i

is assumed, where E i follows either a normal or a Student t-distribution with 3 degrees of

freedom.
Atkinson (1985) and Davison and Tsai (1992) used deletion diagnostic methods to assess
^
the individual influence of the observations on ⠈ (β^0 , β^1 , β^2 , β^3 )T and φ.
^ Atkinson, assuming

normally distributed errors, found cases 16 and 5 to be the most influential. Case 5 was shown

Fig. 1. Index plots of j lmax j for (a) β

^ ^
and (b) φ
78 GALEA, PAULA AND BOLFARINE

Fig. 2. Index plots of j lmax j for θ

^ for (a) the normal, (b) the Cauchy, (c) the Student t- (with 3 degrees of freedom)

and (d) the logistic elliptical linear models

to be influential after a correction was made for case 16. In Davison and Tsai’s analysis, where
a Student t-distribution with 3 degrees of freedom was used, cases 16, 5 and 3 appear the most
influential. Fig. 1 presents the index plot of j lmax j for β
^ ^ separately. We see in Fig. 1(a)
and φ
outstanding local influence for case 16, whereas in Fig. 1(b) it follows that cases 9, 15, 16
and 17 present the highest local influences. In contrast, when we use the global log-likelihood
L(θ) in Cook’s approach instead of the profiles L(βjφ) or L(φjβ), the local influence of the
observations on θ ^ is no longer invariant in the elliptical linear family. Fig. 2 illustrates this

behaviour. Moreover, Fig. 2 shows that case 16 is the most locally influential for the normal,
Cauchy and Student t-models. However, for the logistic model, cases 9, 15 and 17 also appear
with a high local influence. Therefore, we may conclude from this example that the index plot
of j lmax j for θ
^ may be helpful in selecting the less sensitive model with respect to local
^ ^
perturbations in the elliptical linear family, especially when we are interested in both β and φ.

Acknowledgements
The authors acknowledge partial financial support from the Conselho Nacional de Desen-
volvimento Cientı́fico e Technológico Brasil.

References
Atkinson, A. C. (1985) Plots, Transformations, and Regression. Oxford: Clarendon.
Beckman, R. J., Nachtsheim, C. J. and Cook, R. D. (1987) Diagnostics for mixed-model analysis of variance.
Technometrics, 29, 413–426.
Belsley, D. A., Kuh, E. and Welsch, R. E. (1980) Regression Diagnostics: Identifying Influential Data and Sources of
Collinearity. New York: Wiley.
 LOCAL INFLUENCE 79

Carroll, R. J. and Ruppert, D. (1985) Transformations in regression: a robust analysis. Technometrics, 27, 1–12.
Chatterjee, S. and Hadi, A. S. (1988) Sensitivity Analysis in Linear Regression. New York: Wiley.
Cook, R. D. (1986) Assessment of local influence (with discussion) J. R. Statist. Soc. B, 48, 133–169.
— (1987) Influence assessment. J. Appl. Statist., 14, 117–131.
Cook, R. D. and Weisberg, S. (1982) Residuals and Influence in Regression. London: Chapman and Hall.
Davison, A. C. and Tsai, C.-L. (1992) Regression model diagnostics. Int. Statist. Rev., 60, 337–353.
Fang, K. T. and Anderson, T. W. (1990) Statistical Inference in Elliptical Contoured and Related Distributions. New
York: Allerton.
Fang, K. T., Kotz, S. and Ng, K. W. (1990) Symmetric Multivariate and Related Distributions. London: Chapman and
Hall.
Fang, K. T. and Zhang, Y. T. (1990) Generalized Multivariate Analysis. London: Springer.
Kim, M. G. (1995) Local influence in multivariate regression. Communs Statist. Theory Meth., 24, 1271–1278.
Lawrance, A. J. (1988) Regression transformation diagnostics using local influence. J. Am. Statist. Ass., 84, 125–141.
Paula, G. A. (1993) Assessing local influence in restricted regression models. Comput. Statist. Data Anal., 16, 63–79.
Ruppert, D. and Carroll, R. J. (1980) Trimmed least squares estimation in the linear model. J. Am. Statist. Ass., 75,
828–838.
Thomas, W. and Cook, R. D. (1990) Assessing influence on predictions from generalized linear models. Technometrics,
32, 59–65.
Tsai, C.-H. and Wu, X. (1992) Assessing local influence in linear regression models with first-order autoregressive or
heteroscedastic error structure. Statist. Probab. Lett., 14, 247–252.