Adm Policy Ment Health

DOI 10.1007/s10488-015-0672-7

ORIGINAL ARTICLE

Discrepancies in Leader and Follower Ratings

of Transformational Leadership: Relationship
with Organizational Culture in Mental Health
Gregory A. Aarons1,3 • Mark G. Ehrhart2 • Lauren R. Farahnak1,3 •

Marisa Sklar1,3 • Jonathan Horowitz1,3

Ó Springer Science+Business Media New York 2015

Abstract The role of leadership in the management and Keywords Leadership
Discrepancy
Organizations

delivery of health and allied health services is often dis- Management
Public sector
Mental health
cussed but lacks empirical research. Discrepancies are
often found between leaders’ self-ratings and followers’
ratings of the leader. To our knowledge no research has Introduction
examined leader–follower discrepancies and their associa-
tion with organizational culture in mental health clinics. Multi-source feedback about leadership is common in
The current study examines congruence, discrepancy, and organizational research and applied practice (London and
directionality of discrepancy in relation to organizational Smither 1995; Tornow 1993) and is thought to overcome
culture in 38 mental health teams (N = 276). Supervisors biases and limitations inherent in using only self-ratings
and providers completed surveys including ratings of the (Ashford 1993; Hanser and Muchinsky 1978; Podsakoff and
supervisor transformational leadership and organizational Organ 1986). Although self-ratings can provide important
culture. Polynomial regression and response surface anal- insight into how leaders perceive their own abilities and
ysis models were computed examining the associations of behaviors, follower ratings are particularly useful in that they
leadership discrepancy and defensive organizational cul- provide a different perspective on leader behavior. Follower
ture and its subscales. Discrepancies between supervisor ratings may be more reliable (greater number of data points)
and provider reports of transformational leadership were and may offer additional information that is not available to
associated with a more negative organizational culture. supervisors’ own supervisors (Riggio and Cole 1992). For
Culture suffered more where supervisors rated themselves example, follower reports may be less subject to self-per-
more positively than providers, in contrast to supervisors ceptual biases that may influence leader self-report (Mar-
rating themselves lower than the provider ratings of the tinko and Gardner 1987; Yammarino and Atwater 1993).
supervisor. Leadership and leader discrepancy should be a Interpreting multi-source feedback can become com-
consideration in improving organizational culture and for plicated when discrepant ratings emerge. The authors’
strategic initiatives such as quality of care and the imple- anecdotal experience and other published studies suggest
mentation and sustainment of evidence-based practice. that discrepancies between self- and other-ratings of lead-
ers are not uncommon (Ostroff et al. 2004) and can lead to
strong emotional responses from those receiving discrepant
& Gregory A. Aarons feedback. Leaders tend to overrate their own leadership
[email protected]
abilities relative to ratings provided by followers (Atwater
1
Department of Psychiatry, University of California, San and Yammarino 1992; Fleenor et al. 1996). Where dis-
Diego, 9500 Gilman Drive (0812), La Jolla, CA 92093-0812, crepancies are present, leaders may express surprise, con-
USA sternation, disbelief, or emotional distress upon learning of
2
San Diego State University, San Diego, USA such discrepancies. In the present study, we use quantita-
3
Child & Adolescent Services Research Center, San Diego, tive data to examine the degree to which such leader–
USA follower discrepancies exist, the directionality of

123
 Adm Policy Ment Health

discrepancies (i.e., leader more positive ratings compared may not have received from followers. Further, they are
to followers or leader less positive ratings relative to fol- likely to discount negative feedback (Taylor and Brown
lowers), and the degree to which discrepancies are related 1988). For example, leaders may discount or reinterpret
to organizational culture. negative feedback in a way that is less threatening to their
Also consistent with our anecdotal experience, a number self-esteem. Consistent with self-verification theory, lead-
of previous investigations have found poor agreement ers are more likely to attend to feedback that is consistent
between self- and other-ratings (e.g., Atwater and Yam- with their self-opinion, whether that opinion is positive or
marino 1992; Harris and Schaubroeck 1988; Mabe and negative (Swann and Read 1981). Leaders may acknowl-
West 1992; Ostroff et al. 2004). In a review of 22 studies, edge the validity of their followers’ feedback, but may
Thornton (1980) found that leader self-ratings did not rationalize it as the inevitable result of a difference in
correlate strongly with ratings from other sources, and that perspective.
self-ratings tended to be more positive and less variable. As Discrepancies in ratings may provide insight into both
another example, Harris and Schaubroeck (1988) con- individual performance and organizational context. Several
ducted a meta-analysis of discrepancy studies and found investigators have found that degree of discrepancy is
lower agreement on self-supervisor and self-peer rating related to job performance (Atwater et al. 1995; Bass and
comparisons than on peer-supervisor comparisons. Simi- Yammarino 1991; Church 1997; Fleenor et al. 1996). One
larly, Mount (1984) found that agreement was higher for reason for this may be that accurate self-perceptions are
supervisor-follower (other–other) ratings than for self-other important to social adjustment and functioning (Jourard
(self-supervisor or self-follower) ratings. and Landsman 1980; Schulz 1977). Self-awareness mod-
Several factors may give rise to self-other discrepancies. erates the relationship between transformational leadership
The formation of accurate self-perceptions is a delicate behavior and managerial performance (Sosik and Megerian
process, particularly for individuals in management or 1999). Metacognitive skill (Kruger and Dunning 1999), the
supervisory roles which are often ambiguous and fluid ability to recognize one’s own errors, may be a precursor to
(London 2002). Bias in self-ratings may arise in several the development of a skill as complex as leadership.
ways. First, leaders may fail to adequately receive or seek Individuals who are unable to identify their own weak-
out feedback from followers thus limiting confirmation or nesses or incompetencies are unable to make the behavioral
disconfirmation. Second, some leaders may judge super- changes necessary to improve (Atwater et al. 2005).
visee feedback to be relatively unimportant, particularly if Leadership is an important predictor of organizational
their own performance evaluations do not depend on their culture and climate (Aarons and Sawitzky 2006a, b; Ash-
followers’ perceptions. Third, although leaders may value kanasy et al. 2000; Ehrhart 2004; Ehrhart et al. 2014;
follower feedback, they may fail to seek it out for fear that Litwin and Stringer 1968; Tsui et al. 2006; Zohar and
this would communicate uncertainty (Ashford 1986; Ash- Tenne-Gazit 2008). Although individual characteristics are
ford and Tsui 1991). Fourth, leaders may arrive at inac- likely to explain some of the variance in discrepancies, it is
curate self-perceptions by gathering or attending to worth considering whether discrepancies are related to
information from a biased sample. For example, because organizational characteristics. There is a small literature
receiving negative feedback may be a threat to one’s self- examining the relationship between culture and discrep-
perception, leaders may seek more feedback from follow- ancies in leadership ratings. For example, Atwater et al
ers they believe to hold a favorable opinion of them. (2009) examined self- and follower-ratings of leadership in
Even if leaders are able to obtain adequate and accurate a sample of 964 managers from 21 countries. They found
information regarding their followers’ opinions on an that the relationship between self- and follower-ratings was
ongoing basis, they may fail to effectively integrate this more strongly positive in countries that are marked by
information and make the necessary behavioral changes higher levels of assertiveness. In another example, Atwater
where discrepancies exist. More general biases may also et al. (2005) examined the relationship between self-other
exist. For example, social psychological studies have found discrepancies and managerial performance in the U.S. and
that when evaluating themselves on general abilities, five European countries and found that the impact of self-
individuals tend to rank themselves above the median, other discrepancies varied across countries. In the U.S.,
indicating that most people hold unrealistically positive consideration of both self- and other-ratings were impor-
views of themselves (Buunk and Van Yperen 1991; Taylor tant for the prediction of performance, whereas in Europe,
and Brown 1988). Consistent with research on self-en- only follower ratings were predictive of performance.
hancement (Sedikides and Gregg 2008), leaders are likely In this paper, we examine the relationship between
to have overly positive self-ratings at baseline (Podsakoff leader–follower discrepancies in transformational leader-
and Organ 1986), independent of feedback they may or ship and culture. Summarizing Bass’s (1985) theory of

123
Adm Policy Ment Health

transformational leadership, Antonakis and House (2002) Methods

defined the concept as follows: ‘‘Transformational leaders
act as agents of change by arousing and transforming fol- Participants
lowers’ attitudes, beliefs, and motives from a lower to a
higher level of arousal. They provide vision, develop Participants were recruited from public sector mental
emotional relationships with followers and make them health clinics in San Diego County, California. Based on
aware of, and believe in, superordinate goals that go administrative data, 99 teams within mental health pro-
beyond self-interest’’ (p. 8). Leaders enact transformational grams were initially identified. Of the 99 teams, 26 were
leadership through behaviors that demonstrate inspirational considered ineligible because they provided residential
motivation, individualized consideration, intellectual treatment and/or information was not provided to identify
stimulation, and idealized influence (Bass and Avolio their supervisor and thus could not be nested within teams
1995). Organizational culture refers to the set of expecta- for analysis (i.e., a single supervisor for the team was not
tions and norms that govern behavior within an organiza- identified). In addition, one team was excluded due to
tion (Ehrhart et al. 2014; Glisson et al. 2008). Although nonresponse despite repeated contact attempts, resulting in
there is some research on the role of national culture on a total of 72 eligible and responsive teams. Seven of the
discrepancies in leadership ratings (Atwater et al. 2005, remaining 72 teams declined participation, resulting in 68
2009), we are aware of no prior research that has examined participating teams (89 % response rate). Some remaining
how leader–follower discrepancies in leadership ratings, teams were excluded due to having no direct supervisor
and particularly transformational leadership ratings, are data, only supervisor data, only one provider, or incomplete
related to the culture that emerges in the unit. measures. This resulted in a final analytic sample of 276
Alignment between leader and follower perspectives on direct service providers (n = 236) and their supervisors
managerial effectiveness is indicative of better communica- (n = 40), which comprised 38 teams with a mean of 7.3
tion between leaders and their followers (Keyton 2010), members in each team (range 3–15; SD = 3.0). Providers
which is likely to lead to more positive/less negative orga- were clinicians or case-managers providing direct services
nizational cultures. As noted previously, there are many to clients. Supervisors were program managers or team
possible explanations for differences between leaders and leaders that directly supervised providers. In the context of
their followers with regard to their opinions on managerial this study we identify providers as ‘‘followers’’ and
effectiveness, including lack of leader feedback seeking, lack supervisors as ‘‘leaders’’. Table 1 shows participant
of interest in follower opinions, or leaders avoiding threats to demographics.
self-perceptions and only seeking favorable opinions. In all of
these cases, the implications for the norms that develop in the Procedure
unit are likely dysfunctional, resulting in a more negative
organizational culture. Furthermore, the negative implica- Research staff conducted meetings with each of the par-
tions are likely more stark when the leaders’ ratings are higher ticipating teams in order to administer the surveys in paper
than follower ratings. Follower ratings that are higher than format. This study was approved by the IRB of Children’s
leader ratings may indicate other dysfunction within the team, Hospital San Diego. Consent forms were signed by each
such as lack of leader self-confidence, but the implications are participant and returned to research assistants prior to
likely less negative than the opposite. Thus, for the present survey administration. Each meeting included the team’s
study we make the following hypotheses: entire clinical staff, unless team members were out of the
office, refused to participate, or were on-call and needed to
Hypothesis 1 There will be discrepancies in transfor-
leave to address client issues. The surveys took an average
mational leadership, such that leaders will rate their
of 60 min (range 45–180 min) to complete. Participants
transformational leadership more positively than their
returned completed surveys to research assistants who then
followers.
checked for completeness.
Hypothesis 2 Leadership discrepancies will predict rat-
ings of organizational culture.
Measures
Hypothesis 3 When discrepancies are present between
leader and follower ratings of leadership styles, there will The Multifactor Leadership Questionnaire (Bass and
be less favorable ratings of organizational culture when Avolio 1995) 45-item form 59 was used to assess provi-
follower ratings of leadership styles are lower than leader ders’ perceptions of the supervisors’ transformational
ratings than when follower ratings of leadership styles are leadership behaviors. Transformational leadership is com-
greater than leader ratings. prised of the following four subscales: idealized influence

123
 Adm Policy Ment Health

Table 1 Demographic characteristics of clinicians and supervisors Data Analyses

Clinicians Supervisors
(n = 236) (n = 40) The data analytic approach involved aggregating provider
ratings to the team level (K = 38) in order to make team-
Age 35.85 41.26 level inferences about relationships among variables.
Years of experience 7.68 13.95 Supervisor ratings of leadership were already at the team
Years in agency 2.80 5.80 level. To justify aggregation of the -provider data to the
Gender team level, intraclass correlation coefficients (ICC(1)s) and
Male 20.1 % 32.5 % within group agreement (rWG(j)) statistics were calculated;
Female 79.9 % 67.5 % these are presented in Table 2 and support the aggregation
Ethnicity of team ratings.
Caucasian 51.1 % 70.0 % Prior to conducting polynomial regression with response
Hispanic 24.5 % 15.0 % surface analysis for examining discrepancies, researchers
African American 8.6 % 0.0 % have recommended inspecting the data to ensure that dis-
Asian American 4.3 % 2.5 % crepancies indeed exist in the data being analyzed (Sha-
Other 11.6 % 0.0 % nock et al. 2010). Consistent with Fleenor et al. (1996),
Educational level transformational leadership scores were standardized and
High school 1.3 % 0.0 % supervisor and provider ratings of transformational lead-
Some college 6.0 % 0.0 % ership that were discrepant by half a standard deviation or
Bachelor’s degree 14.1 % 0.0 % less were considered to be ‘‘in agreement.’’ This also
Some graduate work 7.7 % 0.0 % provided information relevant to our first hypothesis
Master’s degree 62.8 % 87.5 % regarding the presence and direction of discrepancies in
Doctoral degree 6.4 % 12.5 % supervisor-provider ratings. We then tested the relationship
Other 1.7 % 0.0 % between discrepancies in leadership ratings and culture
Major of highest degree utilizing polynomial regression and response surface
Marriage/family therapy 46.0 % 50.0 % methodology (Edwards 2002; Shanock et al. 2013, 2010).
Social work 22.6 % 37.5 % First, the omnibus tests from four polynomial regression
Psychology 14.9 % 10.0 % analyses were assessed to determine whether variance in
Child development 3.8 % 0.0 % organizational culture was predicted by the polynomial
Human relations 2.1 % 0.0 % trends in and the interaction between supervisor self-rat-
Other 10.6 % 2.5 % ings and provider ratings of supervisor transformational
leadership. Next, we tested the slope and curvature along
N = 276
the x = y and x = -y axes of the response surface because
they correspond directly to the substantive research ques-
tions of interest. The x = y axis is the axis along which
(8 items, a = 0.87), inspirational motivation (four items, provider and supervisor ratings are congruent, whereas the
a = 0.91), intellectual stimulation (four items, a = 0.90), x = -y axis is the axis along which provider and super-
and individual consideration, (four items, a = 0.90). Our visor ratings are incongruent. The relationship between
analyses focused on overall transformational leadership (20 organizational culture and either congruence or incongru-
items, a = 0.95). ence was then explored.
Organizational Culture: The Children’s Services Survey
(Glisson and James 2002) was used to assess organizational
culture. The CSS has established excellent psychometric Results
characteristics, and subscales are associated with work
attitudes, staff turnover, and service quality in human ser- Means, standard deviations, and correlations among the
vice organizations (Glisson and James 2002). The organi- study variables included in the discrepancy analyses are
zational culture subscales were adapted for use in mental presented in Table 3.
health services (Cooke and Rousseau 1988; Farmer et al.
2002; Glisson and James 2002). The subscales used in the Supervisor-Provider Discrepancy in Leadership
current study assessed defensive culture including sub- Ratings
scales of consensus (nine items, a = 0.85), conformity
(seven items, a = 0.82) and subservience (four items, Table 4 shows the means, proportions, and directionality of
a = 0.82). discrepancy between supervisor (self) and provider ratings

123
Adm Policy Ment Health

Table 2 Subscale intraclass

Subscale # Items ICC 95 % CI for ICC Mean rWG(j)
correlation coefficients (ICC)
and within group agreement Multifactor leadership questionnaire: leader self-report
Transformational leadership 20 0.342 0.187, 0.496
Multifactor leadership questionnaire: clinician report
Transformational leadership 20 0.342 0.187, 0.496 0.914
Organizational culture: clinician report
Defensive cultures 20 0.12 0, 0.224 0.824
Consensus 9 0.049 0, 0.146 0.795
Conformity 7 0.092 0, 0.203 0.676
Subservience 4 0.148 0.024, 0.272 0.598

Table 3 Means, standard

Mean sd 1 2 3 4 5
deviations and correlations of
study variables Provider TL ratings 2.40 0.58 –
Supervisor TL 2.50 0.48 0.36* –
ratings
OCM—consensus 1.66 0.33 -0.43** -0.08 –
OCM—conformity 1.56 0.40 0.60** -0.29 0.83** –
OCM—subservience 1.67 0.48 -0.62** -0.26 0.81** 0.93** –
* p \ 0.05; ** p \ 0.01

Table 4 Supervisor and

Agreement groups N Percentage Mean supervisors Mean providers
provider discrepancy on the
multifactor leadership Transformational leadership
questionnaire
Supervisors [ providers 17 44.74 2.65 2.01
Supervisors = providers 10 26.32 2.69 2.68
Supervisors \ providers 11 28.95 2.09 2.75

of transformational leadership. Over 70 % of our sample Discrepancy in Leadership Ratings

had supervisor and provider ratings of overall transforma- and Organizational Culture
tional leadership that were different from each other in one
direction or the other. Specifically, 44.7 % of teams con- We then tested our second hypothesis, that leadership
sisted of supervisors who rated their transformational discrepancies would predict ratings of organizational cul-
leadership higher than that of providers, and 28.9 % of ture. Results of omnibus tests suggested that supervisor
supervisors rated their transformational leadership behav- self-ratings and provider ratings of supervisor transforma-
iors lower than that of providers. Because of the level of tional leadership predicted variance in consensus culture
discrepancies present and because both directions of dis- (R2 = 0.306, p = 0.032), conformity culture (R2 = 0.413,
crepancy were well-represented in the data, we could move p = 0.003), subservience culture (R2 = 0.454, p = 0.001),
forward with the polynomial regression analyses. This and overall defensive culture (R2 = 0.0413, p = 0.003).
analysis also allowed provided information relevant for Response surface analyses were examined next to further
testing Hypothesis 1. Although more teams showed a pat- test our second hypothesis that discrepancies between
tern in which supervisors rated themselves higher than supervisor self-ratings and provider ratings of supervisor
providers than vice versa, the mean of the supervisor self- transformational leadership predict ratings of organiza-
ratings (M = 2.495) was not significantly greater than that tional culture. Surface tests were also examined to test our
of provider ratings (M = 2.398) of the supervisor’s trans- third hypothesis, that ratings of organizational culture
formational leadership (Mdiff = 0.097, SE = 0.098, t(37) = would be more negative when provider ratings of trans-
0.994, p = 0.327). Thus, Hypothesis 1 was not supported. formational leadership are lower than supervisor ratings

123
 Adm Policy Ment Health

than when provider ratings of transformational leadership than when providers’ ratings of transformational leadership
are greater than supervisor ratings. Table 5 shows results were high combined with low supervisor self-ratings of
from all polynomial and response surface analyses. transformational leadership.
Graphical representations of response surfaces are pre- The pattern of results assessing the relationship between
sented in Figs. 1, 2, 3, 4. congruence/incongruence between supervisor self-ratings
Surface tests assessing the relationship between con- and provider ratings of supervisor transformational lead-
gruence/incongruence between supervisor self-ratings and ership and subservience culture followed a similar pattern
provider ratings of supervisor transformational leadership to that of conformity culture. Results demonstrated that
and consensus culture were examined first. Results sug- agreement between supervisors and providers’ ratings of
gested that agreement between supervisors and providers’ transformational leadership significantly influenced sub-
ratings of transformational leadership did not influence servience (a1 = -0.46, t = -2.906, p = 0.006). Specifi-
consensus (a1 = -0.13, t = -1.071, p = 0.291). How- cally, the greatest level of subservience was when
ever, incongruence between supervisors and providers’ supervisor and providers’ similarly rated transformational
ratings of transformational leadership had a significant leadership low. Subservience decreased as both supervisor
curvilinear relationship with consensus (a4 = 1.18, and providers’ ratings of transformational leadership
t = 3.281, p = 0.002), such that as ratings between increased. Incongruence between supervisors and provi-
supervisors and providers became increasingly discrepant, ders’ ratings of transformational leadership had a signifi-
ratings of a consensus culture increased. Further analysis cant curvilinear relationship with subservience (a4 = 1.45,
revealed that the direction of discrepancy between super- t = 3.142, p = 0.003), such that as ratings between
visor and providers’ ratings of transformational leadership supervisors and providers became increasingly discrepant,
significantly influenced consensus culture (a3 = 0.51, ratings of a subservient culture increased. Further analysis
t = 2.741, p = 0.010). Consistent with our third hypothe- revealed that the direction of discrepancy between super-
sis, consensus culture was greater when providers’ ratings visor and providers’ ratings of transformational leadership
of transformational leadership were low and supervisor significantly influenced subservience culture (a3 = 0.62,
self-ratings of transformational leadership were high than t = 2.668, p = 0.011). Consistent with our third hypothe-
when providers’ ratings of transformational leadership sis, subservience culture was greater when providers’ rat-
were high combined with low supervisor self-ratings of ings of transformational leadership were low and
transformational leadership. supervisor self-ratings of transformational leadership were
A mostly similar pattern emerged when assessing the high than when providers’ ratings of transformational
relationship between congruence/incongruence between leadership were high combined with low supervisor self-
supervisor self-ratings and provider ratings of supervisor ratings of transformational leadership.
transformational leadership and conformity culture. Unlike This same pattern of results emerged when assessing the
the results regarding a consensus culture, results suggested relationship between congruence/incongruence between
that agreement between supervisors and clinicians’ ratings supervisor self-ratings and provider ratings of supervisor
of transformational leadership significantly influenced con- transformational leadership and overall defensive culture.
formity (a1 = -0.42, t = -3.021, p = 0.005). Specifically, Results suggested that agreement between supervisors and
the greatest level of conformity was when supervisor and providers’ ratings of transformational leadership signifi-
providers’ similarly rated transformational leadership low. cantly influenced overall defensive culture (a1 = -0.34,
Conformity decreased as both supervisor and clinicians’ t = -2.620, p = 0.013). Specifically, the greatest level of
ratings of transformational leadership increased. Similar to defensive culture was when supervisor and providers’
the results with consensus culture, incongruence between similarly rated transformational leadership low. Defensive
supervisors and providers’ ratings of transformational lead- culture decreased as both supervisor and providers’ ratings
ership had a significant curvilinear relationship with con- of transformational leadership increased. Incongruence
formity (a4 = 0.96, t = 2.381, p = 0.023), such that as between supervisors and providers’ ratings of transforma-
ratings between supervisors and providers became increas- tional leadership had a significant curvilinear relationship
ingly discrepant, ratings of a conformity culture increased. with overall defensive culture (a4 = 1.20, t = 3.120,
Further analysis revealed that the direction of discrepancy p = 0.004), such that as ratings between supervisors and
between supervisor and providers’ ratings of transforma- providers became increasingly discrepant, ratings of over-
tional leadership significantly influenced conformity culture all defensive culture increased. Further analysis revealed
(a3 = 0.42, t = 2.032, p = 0.050). Consistent with our third that the direction of discrepancy between supervisor and
hypothesis, conformity culture was greater when provides’ providers’ ratings of transformational leadership influenced
ratings of transformational leadership were low and super- defensive culture (a3 = 0.52, t = 2.605, p = 0.013).
visor self-ratings of transformational leadership were high Consistent with our third hypothesis, overall defensive

123
Adm Policy Ment Health

Table 5 Polynomial regression and response surface analyses

Consensus culture and overall transformational leadership
Variable regressed onto consensus culture b (se)

Polynomial regression analysis Constant 1.592**

Transformational leadership: supervisor 0.191 (0.124)
Transformational leadership: team -0.323 (0.099)*
Transformational leadership: sup squared 0.233 (0.211)
Transformational leadership: sup 9 team -0.681 (0.289)*
Transformational leadership: team squared 0.262 (0.206)
R2 0.306*
Response surface tests a1 -0.13 (0.12)
a2 -0.19 (0.28)
a3 0.51 (0.19)*
a4 1.18 (0.36)**

Conformity culture and overall transformational leadership

Variable regressed onto conformity culture b (se)

Polynomial regression analysis Constant 1.418**

Transformational leadership: supervisor 0.002 (0.138)
Transformational leadership: team -0.420 (0.110)**
Transformational leadership: sup squared 0.300 (0.236)
Transformational leadership: sup 9 team -0.327 (0.323)
Transformational leadership: team squared 0.334 (0.230)
R2 0.413**
Response surface tests a1 -0.42 (0.14)**
a2 0.31 (0.31)
a3 0.42 (0.21)*
a4 0.96 (0.40)*

Subservience culture and overall transformational leadership

Variable regressed onto subservience culture b (se)

Polynomial regression analysis Constant 1.494**

Transformational Leadership: supervisor 0.089 (0.158)
Transformational leadership: team -0.547 (0.126)**
Transformational leadership: sup squared 0.371 (0.270)
Transformational leadership: sup 9 team -0.618 (0.370)
Transformational leadership: team squared 0.461 (0.263)
R2 0.454**
Response surface tests a1 -0.46 (0.16)**
a2 0.21 (0.35)
a3 0.64 (0.24)*
a4 1.45 (0.46)**

Overall defensive culture and overall transformational leadership

Variable regressed onto defensive culture b (se)

Polynomial regression analysis Constant 1.501**

Transformational leadership: supervisor 0.094 (0.132)
Transformational leadership: team -0.430 (0.105)**
Transformational leadership: sup squared 0.301 (0.226)
Transformational leadership: sup 9 team -0.542 (0.309)
Transformational leadership: team squared 0.353 (0.220)
R2 0.413**

123
 Adm Policy Ment Health

Table 5 continued
Overall defensive culture and overall transformational leadership
Variable regressed onto defensive culture b (se)

Response surface tests a1 -0.34 (0.13)*

a2 0.11 (0.29)
a3 0.52 (0.20)*
a4 1.20 (0.38)**

N = 38; * p \ .05; ** p \ .01

4.0 4.0
3.5 3.5
3.0 3.0

Conformity
Consensus

2.5 2.5
2.0 2.0
1.5 2 1.5 2
1.0 1.0

Ratin rvisor
1 1
Rati e r v is o r

Sup
Sup

0.5 0 0.5 0
ngs o

gs o f
0.0

e
0.0
2 -1 2 -1
1 1
f TL

TL
0 0
-1 -2 -1 -2
-2 -2
Provider Ratings of TL Provider Ratings of TL

Fig. 1 Consensus predicted by discrepancy between supervisor and Fig. 2 Conformity predicted by discrepancy between supervisor and
provider ratings of Transformational Leadership (TL). a1: The line of provider ratings of Transformational Leadership (TL). a1: The line of
perfect agreement (solid black line on floor) as related to consensus perfect agreement (solid black line on floor) as related to conformity
(CNS) has a non-significant slope. Thus, agreement between super- (CON) has a negative slope. Thus, agreement between supervisors
visors and provider’s TL ratings are not related to CNS. a4: Moving and provider’s TL ratings matters such that the greatest level of CON
along the line of incongruence (dashed line on floor) away from the is at the front corner of the graph where supervisor and provider TL
center of the graph to either the left or right shows how the degree of are both low, and decreases toward the back of the graph where both
discrepancy between supervisors and provider’s TL has a significant supervisors and provider’s TL are both in agreement and high. a4:
curvilinear relationship with CNS. The graph shows that toward the Moving along the line of incongruence (dashed line on floor) away
left and right of the graph, where ratings become more and more from the center of the graph to either the left or right shows how the
discrepant, CNS increases. a3: The direction of discrepancy matters. degree of discrepancy between supervisors and provider’s TL has a
At the bottom right corner of the graph (where provider ratings of TL significant curvilinear relationship with CON. The graph shows that
is low combined with high supervisor ratings of TL), CNS is very toward the left and right of the graph, where ratings become more and
high. At the bottom left corner of the graph (where provider ratings of more discrepant, CON increases. a3: The direction of discrepancy
TL is high combined with low supervisor ratings of TL), CNS is high, matters. At the bottom right corner of the graph (where provider
though not as high as when supervisor rating of TL are high and ratings of TL is low combined with high supervisor ratings of TL),
provider ratings are low CON is very high. At the bottom left corner of the graph (where
provider ratings of TL is high combined with low supervisor ratings
of TL), CON is high, though not as high as when supervisor rating of
culture was greater when providers’ ratings of transfor- TL are high and provider ratings are low
mational leadership were low and supervisor self-ratings of
transformational leadership were high than when provi- We also found that the magnitude and direction of this
ders’ ratings of transformational leadership were high discrepancy were associated with decrements in organiza-
combined with low supervisor self-ratings of transforma- tional culture. That is, greater discrepancy between super-
tional leadership. visors and providers was associated with a more defensive
(i.e., more negative) organizational culture. This effect was
more pronounced where supervisors rated themselves more
Discussion positively than did providers as compared to supervisors
rating themselves less positively than the providers they
Our primary findings were that supervisor and provider supervised. These findings are consistent with the literature
ratings of supervisors in the organizational context of and with our anecdotal experience in working with public
public mental health service organizations do indeed differ. sector mental health and social service organizations.

123
Adm Policy Ment Health

4.0 4.0

Passive Defensive Culture

3.5 3.5
3.0 3.0
Subservience

2.5 2.5
2.0 2.0
1.5 2 1.5 2
1.0 1 1.0 1

Rati ervisor
Rati ervisor

Sup
0.5 0.5

S up
0 0

ngs
ngs o
0.0 0.0
2 -1 2 -1

of T
1 1

f TL
0 0

L
-1 -2 -1 -2
-2 -2
Provider Ratings of TL Provider Ratings of TL

Fig. 3 Subservience predicted from discrepancy between supervisor Fig. 4 Defensive culture predicted from discrepancy between super-
and provider ratings of Transformational Leadership (TL). a1: The visor and provider ratings of Transformational Leadership (TL). a1:
line of perfect agreement (solid black line on floor) as related to The line of perfect agreement (solid black line on floor) as related to
subservience (S) has a negative slope. Thus, agreement between overall Defensive Culture (PDC) has a negative slope. Thus,
supervisors and provider’s TL ratings matters such that the greatest agreement between supervisors and provider’s TL ratings matters
level of S is at the front corner of the graph where supervisor and such that the greatest level of PDC is at the front corner of the graph
provider TL are both low, and decreases toward the back of the graph where supervisor and provider TL are both low, and decreases toward
where both supervisors and provider’s TL are both in agreement and the back of the graph where both supervisors and provider’s TL are
high. a4: Moving along the line of incongruence (dashed line on floor) both in agreement and high. a4: Moving along the line of incongru-
away from the center of the graph to either the left or right shows how ence (dashed line on floor) away from the center of the graph to either
the degree of discrepancy between supervisors and provider’s TL has the left or right shows how the degree of discrepancy between
a significant curvilinear relationship with S. The graph shows that supervisors and provider’s TL had a significant curvilinear relation-
toward the left and right of the graph, where ratings become more and ship with PDC. The graph shows that toward the left and right of the
more discrepant, S increases. a3: The direction of discrepancy matters. graph, where ratings become more and more discrepant, PDC
At the bottom right corner of the graph (where provider ratings of TL increases. a3: The direction of discrepancy matters. At the bottom
is low combined with high supervisor ratings of TL), S is very high. right corner of the graph (where provider ratings of TL is low
At the bottom left corner of the graph (where provider ratings of TL is combined with high supervisor ratings of TL), PDC is very high. At
high combined with low supervisor ratings of TL), S is high, though the bottom left corner of the graph (where provider ratings of TL is
not as high as when supervisor rating of TL are high and provider high combined with low supervisor ratings of TL), PDC is high,
ratings are low though not as high as when supervisor rating of TL are high and
provider ratings are low
We found that discrepancy in either direction (i.e.,
supervisor rating themselves higher than did providers, or additional questions. First, if supervisors have difficulty
supervisors rating themselves lower than did providers) accurately assessing their own behaviors, do they also have
was associated with more negative organizational culture. biases in perceptions of their providers or of organizational
Thus, helping leaders develop self-perceptions that are social context? For example, some supervisors may avoid
more consistent with followers perceptions of the leader, accurate introspection, and may also avoid or be unable to
may help to mitigate negative effects of such discrepant make accurate assessments or interpretations of the orga-
perceptions on organizational culture. Consistent with our nizational culture and climate in which they operate
anecdotal experience and reports from mental health pro- (Ashkanasy et al. 2000). Furthermore, their own behavior
gram supervisors, supervisors often ascend from providing (based on bias or misperception) may contribute to less
clinical services to supervising services and other providers than optimal organizational context. Common-source bias
while receiving very little training on supervision, man- could also account for this finding; however, a number of
agement and leadership (Aarons et al. 2015). Indeed, when factors decrease the likelihood of this explanation and we
management training does occur, it is often focused on discuss this issue in the limitations section.
day-to-day issues such as time management, billing The very act of seeking feedback from providers may
requirements, and completing appropriate documentation lead to improved perceptions of a supervisor’s competence.
for service contracts or insurance reimbursements. There is Ashford and Tsui (1991) found that leaders willing to seek
little to no training on effective leadership or how to negative feedback have a better understanding of the cri-
appraise and develop one’s own leadership knowledge, teria by which followers judge their work. Leaders who
skills, and abilities. sought negative feedback tended to be more positively
The finding that discrepancies between supervisor and rated by followers, whereas leaders seeking positive feed-
provider ratings of transformational leadership were related back were rated more negatively. Thus, organizations
to provider ratings of organizational culture raises some might use feedback mechanisms to seek out and provide

123
 Adm Policy Ment Health

both positive and negative feedback in order to facilitate However, the GLOBE project focused on top management
leader development and promote a more positive organi- leadership. Future research should continue the examina-
zational culture. tion of first level leadership (Petkevičiūt_e and Giedraitis
2013; Priestland and Hanig 2005) across contexts and
Clinical and Organizational Practice Implications culture.
Although the GLOBE project looked at top management
It is important to note that supervisors/leaders often respond leadership, current work in first-level strategic leadership for
emotionally and with distress when their self-ratings of implementation of evidence-based health and allied health is
leadership are highly discrepant, particularly when their a new area of study. For example, the newly developed
own ratings are more positive than their provider/supervisee Implementation Leadership Scale assess four dimensions of
ratings. This raises the issue of how supervisors respond to leader characteristics including the degree to which the
negatively discrepant feedback. Attributional biases may leader is knowledgeable, proactive, supportive, and perse-
impact the degree to which supervisors experience distress verant in regard to a strategic initiative (Aarons et al. 2014).
or emotional stress related to negatively discrepant feed- Future studies should examine the impact of both transfor-
back. Anecdotally, supervisors generally appear to wonder mational and implementation leadership discrepancy on
about the causes of positively discrepant feedback, but do organizational and implementation climate, and evidence-
not dwell on it or experience negative emotionality or affect based practice implementation and sustainment.
as a result. There may even be a sense of relief that their
own moderate or negative self-evaluation is disconfirmed
by relatively objective observers. Conclusions

Limitations The role of discrepancy in organizational functioning, orga-

nizational change, leader–follower relationships, effective
Consistent with recommendations by Podsakoff et al. provider supervision, and effective implementation are criti-
(2003), items that may potentially exhibit common source cal areas not just for research but also for practice. Organi-
bias have proximal and methodological separation in that zations should consider how discrepancy might support or
they are measured in different ways and in different sub- derail change initiatives and effective functioning in general.
stantive sections of the survey. In this study, one set of Use of multi-source feedback can be used as a developmental
provider questions related to ratings of observable super- tool to help support leaders and supervisors in understanding
visor behavior, measured on a Likert-like type scale, and where they stand, and what biases or misperceptions they may
another set of questions assessed perceptions of organiza- hold. The proximal goals of this approach were to develop and
tional culture. Each set of questions were embedded in a support leaders. The more distal goals included improving
series of questions with a different substantive focus (i.e., organization relationships and functioning, and improving the
leadership, culture). We considered the use of analytical bottom-line, whether that be quality improvement, organiza-
models to assess common source bias (i.e., latent variables tional viability, or clinical care outcomes. Although leader-
constructed of all potentially biased items), but the ship is often discussed, it is not often well-addressed in public
approach proved untenable given the small sample size and sector service settings. The work presented here should con-
differing variable types (i.e., categorical, count, continu- tribute to an understanding of how to identify leadership
ous). Finally, to promote accurate and unbiased responses discrepancy and its impact as a step toward effective organi-
and minimize any social pressures or expectations, the zational intervention and support for mental health and other
survey was conducted voluntarily, confidentially, and allied health organizations.
online (Podsakoff et al. 2003).
Discrepancy may be an issue in other settings and it may Acknowledgments This work was supported by National Institute of
Mental Health Grants R01MH072961; R21MH082731; R21MH098124;
also be that discrepancy in self-other ratings varies in dif- K01MH001695. The authors thank the community stakeholders, provi-
ferent countries and cultures. For example Cullen et al. ders, and supervisors who participated in this study.
(2015) found that self-enhancement and self-diminishment
were dependent on culture. The Global Leadership and
Organizational Behavior Effectiveness (GLOBE) project
has documented issues in leadership from 62 societies and References
27 European societies (House et al. 2013). Studies in cross-
Aarons, G. A., Ehrhart, M. G., & Farahnak, L. R. (2014). The
cultural comparative management have documented mul- Implementation Leadership Scale (ILS): Development of a brief
tiple universal desirable and undesirable leadership char- measure of unit level implementation leadership. Implementation
acteristics, but also 35 that are contingent on culture. Science, 9(1), 45.

123
Adm Policy Ment Health

Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Hurlburt, M. S. leader derailment in individualistic and collectivistic cultures.
(2015). Leadership and organizational change for implementa- Applied Psychology, 64(1), 161–207.
tion (LOCI): a randomized mixed method pilot study of a Edwards, J. R. (2002). Alternatives to difference scores: Polynomial
leadership and organization development intervention for evi- regression and response surface methodology. In F. Drasgow &
dence-based practice implementation. Implementation Science, N. W. Schmitt (Eds.), Advances in measurement and data
10(1), 11. analysis (pp. 350–400). San Francisco: Jossey-Bass.
Aarons, G. A., & Sawitzky, A. C. (2006a). Organizational climate Ehrhart, M. G. (2004). Leadership and procedural justice climate as
partially mediates the effect of culture on work attitudes and staff antecedents of unit-level organizational citizenship behavior.
turnover in mental health services. Administration and Policy in Personnel Psychology, 57(1), 61–94.
Mental Health and Mental Health Services Research, 33(3), Ehrhart, M. G., Schneider, B., & Macey, W. H. (2014). Organiza-
289–301. tional climate and culture: An introduction to theory, research,
Aarons, G. A., & Sawitzky, A. C. (2006b). Organizational culture and and practice. New York: Routledge.
climate and mental health provider attitudes toward evidence- Farmer, E. M. Z., Compton, S. N., Burns, J. B., & Robertson, E. B.
based practice. Psychological Services, 3(1), 61–72. (2002). Review of the evidence base for treatment of childhood
Antonakis, J., & House, R. J. (2002). An analysis of the full-range psychopathology: Externalizing disorders. Journal of Consulting
leadership theory: The way forward. In B. J. Avolio & F. and Clinical Psychology, 70(6), 1267–1302.
J. Yammarino (Eds.), Transformational and charismatic leader- Fleenor, J. W., McCauley, C. D., & Brutus, S. (1996). Self-other
ship: The road ahead (pp. 3–34). Amsterdam: JAI. rating agreement and leader effectiveness. Leadership Quarterly,
Ashford, S. J. (1986). Feedback seeking in individual adaptation: A 7(4), 487–506.
resource perspective. Academy of Management Journal, 29(3), Glisson, C., & James, L. (2002). The cross-level effects of culture and
465–487. climate in human service teams. Journal of Organizational
Ashford, S. J. (1993). The feedback environment: An exploratory Behavior, 23, 767–794.
study of cue use. Journal of Organizational Behavior, 14, Glisson, C., Landsverk, J., Schoenwald, S., Kelleher, K., Hoagwood,
201–224. K., Mayberg, S., & Green, P. (2008). Assessing the Organiza-
Ashford, S. J., & Tsui, A. S. (1991). Self-regulation for managerial tional Social Context (OSC) of mental health services: Implica-
effectiveness: The role of active feedback seeking. Academy of tions for research and practice. Administration and Policy in
Management Journal, 34(2), 251–280. Mental Health and Mental Health Services Research. Special
Ashkanasy, N. M., Wilderom, C. P. M., & Peterson, M. F. (2000). Issue: Improving mental health services, 35(1–2), 98–113.
Handbook of organizational culture and climate. Thousand Hanser, L. M., & Muchinsky, P. M. (1978). Work as an information
Oaks: Sage. environment. Organizational Behavior and Human Perfor-
Atwater, L. E., Roush, P., & Fischthal, A. (1995). The influence of mance, 21, 43–62.
upward feedback on self and follower ratings of leadership. Harris, M. M., & Schaubroeck, J. (1988). A meta-analysis of self-
Personnel Psychology, 48(1), 35–59. supervisor, self-peer, and peer-supervisor ratings. Personnel
Atwater, L. E., Waldman, D. A., Ostroff, C., Robie, C., & Johnson, K. Psychology, 41(1), 43–62.
M. (2005). Self-other agreement: Comparing its relationship House, R. J., Dorfman, P. W., Javidan, M., Hanges, P. J., & de Luque,
with performance in the U.S. and Europe. International Journal M. F. S. (2013). Strategic leadership across cultures: GLOBE
of Selection and Assessment, 13(1), 25–40. study of CEO leadership behavior and effectiveness in 24
Atwater, L. E., Wang, M., Smither, J. W., & Fleenor, J. W. (2009). countries. Thousand Oaks: Sage Publications.
Are cultural characteristics associated with the relationship Jourard, S. M., & Landsman, T. (1980). Healthy personality: An
between self and others’ ratings of leadership. Journal of Applied approach from the viewpoint of humanistic psychology (4th ed.).
Psychology, 94(4), 876–886. New York: Macmillan.
Atwater, L. E., & Yammarino, F. J. (1992). Does self-other agreement Keyton, J. (2010). Communication and organizational culture: A key
on leadership perceptions moderate the validity of leadership and to understanding work experiences. Thousand Oaks: Sage
performance predictions. Personnel Psychology, 45(1), 141–164. Publications.
Bass, B. M. (1985). Leadership and performance beyond expecta- Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How
tions. New York: Free Press. difficulties in recognizing one’s own incompetence lead to
Bass, B. M., & Avolio, B. J. (1995). MLQ: Multifactor leadership inflated self-assessments. Journal of Personality and Social
questionnaire (Technical Report). Binghamton University, NY: Psychology, 77(6), 1121–1134.
Center for Leadership Studies. Litwin, G., & Stringer, R. (1968). Motivation and organizational
Bass, B. M., & Yammarino, F. J. (1991). Congruence of self and climate. Cambridge: Harvard University Press.
others’ leadership ratings of Naval officers for understanding London, M. (2002). Leadership development: Paths to self-insight
successful performance. Applied Psychology: An International and professional growth. Mahwah: Lawrence Erlbaum Associ-
Review, 40(4), 437–454. ates Inc.
Buunk, B. P., & Van Yperen, N. W. (1991). Referential comparison, London, M., & Smither, J. W. (1995). Can multi-source feedback
relational comparisons, and exchange orientation: Their relation change perceptions of goal accomplishment, self-evaluations, and
to marital satisfaction. Personality and Social Psychology performance-related outcomes? Theory-based applications and
Bulletin, 17(6), 709–717. directions for research. Personnel Psychology, 48(4), 803–839.
Church, A. H. (1997). Managerial self-awareness in high-performing Mabe, P. A., & West, S. G. (1992). Validity of self-evaluation of
individuals in organizations. Journal of Applied Psychology, ability: A review and meta-analysis. Journal of Applied
82(2), 281–292. Psychology, 67(3), 280–296.
Cooke, R. A., & Rousseau, D. M. (1988). Behavioral norms and Martinko, M. J., & Gardner, W. L. (1987). The leader/member
expectations: A quantitative approach to the assessment of attribution process. Academy of Management Review, 12(2),
organizational culture. Group Organization Management, 13(3), 235–249.
245–273. Mount, M. K. (1984). Psychometric properties of subordinate ratings
Cullen, K. L., Gentry, W. A., & Yammarino, F. J. (2015). Biased self- of managerial performance. Personnel Psychology, 37(4),
perception tendencies: self-enhancement/self-diminishment and 687–702.

123
 Adm Policy Ment Health

Ostroff, C., Atwater, L. E., & Feinberg, B. J. (2004). Understanding Shanock, L. R., Baran, B. E., Gentry, W. A., Pattison, S. C., &
self-other agreement: A look at rater and rate characteristics, Heggestad, E. D. (2010). Polynomial regression with response
context, and outcomes. Personnel Psychology, 57(2), 333–375. surface analysis: A powerful approach for examining moderation
Petkevičiūt_e, N., & Giedraitis, A. (2013). Leadership skills formation and overcoming limitations of difference scores. Journal of
in workgroup of the first level managers in manufacturing Business and Psychology, 25(4), 543–554.
companies. Organizaciju˛ Vadyba: Sisteminiai Tyrimai, 67, Sosik, J. J., & Megerian, L. E. (1999). Understanding leader
69–82. emotional intelligence and performance: The role of self-other
Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2003). agreement on transformational leadership perceptions. Group
Common method biases in behavioral research: A critical review and Organization Management, 24(3), 367–390.
of the literature and recommended remedies. Journal of Applied Swann, W. B., & Read, S. J. (1981). Self-verification processes: How
Psychology, 88(5), 879–903. we sustain our self-conceptions. Journal of Experimental Social
Podsakoff, P. M., & Organ, D. W. (1986). Self-reports in organiza- Psychology, 17(4), 351–372.
tional research: Problems and prospects. Journal of Manage- Taylor, S. E., & Brown, J. D. (1988). Illusion and well-being: A social
ment, 12(4), 531–544. psychological perspective on mental health. Psychological
Priestland, A., & Hanig, R. (2005). Developing first-level leaders. Bulletin, 103(2), 193–210.
Harvard Business Review, 83(6), 112–120. Thornton, G. C. (1980). Psychometric properties of self-appraisals of
Riggio, R. E., & Cole, E. J. (1992). Agreement between subordinate job performance. Personnel Psychology, 33(2), 263–271.
and superior ratings of supervisory performance and effects on Tornow, W. W. (1993). Editor’s note: Introduction to special issues
self and subordinate job satisfaction. Journal of Occupational on 360-degree feedback. Human Resource Management,
and Organizational Psychology, 65(2), 151–158. 32(2–3), 211–219.
Schulz, D. (1977). Growth psychology: Models of the healthy Tsui, A. S., Zhang, Z.-X., Wang, H., Xin, K. R., & Wu, J. B. (2006).
personality. New York: Van Nostrand Reinhold. Unpacking the relationship between CEO leadership behavior
Sedikides, C., & Gregg, A. P. (2008). Self-enhancement: Food for and organizational culture. Leadership Quarterly, 17, 113–137.
thought. Perspectives on Psychological Science, 3(2), 102–116. Yammarino, F. J., & Atwater, L. E. (1993). Understanding self-
Shanock, L. R., Allen, J. A., Dunn, A. M., Baran, B. E., Scott, C. W., perception accuracy: Implications for human resource manage-
& Rogelberg, S. G. (2013). Less acting, more doing: How ment. Human Resource Management Review, 32(2–3), 231–247.
surface acting relates to perceived meeting effectiveness and Zohar, D., & Tenne-Gazit, O. (2008). Transformational leadership
other employee outcomes. Journal of Occupational and Orga- and group interaction as climate antecedents: A social network
nizational Psychology, 86(4), 457–476. analysis. Journal of Applied Psychology, 93, 744–757.

123