Using 360 Feedback

CENTER FOR CREATIVE LEADERSHIP
John W. Fleenor
Jeffrey Michael Prince
USING 360-DEGREE
FEEDBACK
IN ORGANIZATIONS
AN ANNOTATED BIBLIOGRAPHY
USING 360-DEGREE
FEEDBACK
IN ORGANIZATIONS
USING 360-DEGREE
FEEDBACK
IN ORGANIZATIONS
John W. Fleenor
Jeffrey Michael Prince
Center for Creative Leadership
Greensboro, North Carolina
The Center for Creative Leadership is an international, nonprofit educational institution
founded in 1970 to advance the understanding, practice, and development of leadership
for the benefit of society worldwide. As a part of this mission, it publishes books and
reports that aim to contribute to a general process of inquiry and understanding in which
ideas related to leadership are raised, exchanged, and evaluated. The ideas presented in its
publications are those of the author or authors.
The Center thanks you for supporting its work through the purchase of this volume. If
you have comments, suggestions, or questions about any CCL Press publication, please
contact the Director of Publications at the address given below.
Center for Creative Leadership
Post Office Box 26300
Greensboro, North Carolina 27438-6300
336-288-7210 www.ccl.org
1997 Center for Creative Leadership
All rights reserved. No part of this publication may be reproduced, stored in a retrieval
system, or transmitted, in any form or by any means, electronic, mechanical, photocopy-
ing, recording, or otherwise, without the prior written permission of the publisher. Printed
in the United States of America.
CCL No. 338
Library of Congress Cataloging-in-Publication Data
Fleenor, John W.
Using 360-degree feedback in organizations : an annotated bibliography /
John W. Fleenor and Jeffrey Michael Prince.
p. cm.
Includes bibliographical references and indexes.
ISBN 1-882197-33-X
1. EmployeesRating of. 2. 360-degree feedback (Rating of employees).
I. Prince, Jeffrey Michael. II. Title. III. Using three hundred sixty-degree feedback in
organizations.
HF5549.5.R3F553 1997
658.3'125dc21 97-38197
CIP
v
Table of Contents
List of Annotations .................................................................................................. vii
Acknowledgments ..................................................................................................... xi
Introduction ............................................................................................................... 1
Annotated Bibliography............................................................................................ 3
Frequently Asked Questions about 360-degree Feedback ................................... 51
How and When Did 360-degree Feedback Develop?......................................... 51
What Are the Benefits of 360-degree Feedback? ............................................... 52
How Should 360-degree Feedback Be Used? .................................................... 54
Feedback for development only................................................................... 55
Feedback for performance appraisal ............................................................ 55
Feedback for development and appraisal .................................................... 55
How Do You Integrate 360-degree Feedback into Organizational
Development Systems?....................................................................................... 56
Prepare the organization and employees ..................................................... 56
Administer the assessment ........................................................................... 57
Conduct the follow-up ................................................................................. 58
What Are Future Trends and Research Needs for 360-degree Feedback? ......... 59
Appendix A: Related Sources ................................................................................. 63
Appendix B: Selection Criteria .............................................................................. 69
Glossary of Terms.................................................................................................... 71
Author Index ............................................................................................................ 73
Title Index ................................................................................................................ 75
vii
List of Annotations
David Antonioni
Designing an Effective 360-degree Appraisal Feedback Process ............................... 3
Leanne E. Atwater and Francis J. Yammarino
Self-Other Rating Agreement: A Review and Model ................................................. 4
H. John Bernardin, Sue A. Dahmus, and Gregory Redmon
Attitudes of First-line Supervisors toward Subordinate Appraisals ............................ 4
David W. Bracken
Straight Talk about Multirater Feedback..................................................................... 5
Stphane Brutus, John Fleenor, and Manuel London
Elements of Effective 360-degree Feedback ............................................................... 6
Matthew Budman and Berkeley Rice
The Rating Game ......................................................................................................... 7
Robert Carey
Coming Around to 360-degree Feedback.................................................................... 8
Dennis E. Coates
Multi-source Feedback: Seven Recommendations ..................................................... 8
Barbara Crystal
The 360 Assessment .................................................................................................. 9
Maxine Dalton
Multi-rater Feedback and Conditions for Change ..................................................... 10
Wayne E. Denton
Developing Employee Skills to Match Company Needs .......................................... 11
Marvin M. Dunnette
My Hammer or Your Hammer? ................................................................................ 11
Colin C. Edwards
360 Degree Feedback ................................................................................................ 12
Mark R. Edwards and Ann J. Ewen
360 Feedback: The Powerful New Model for Employee Assessment and
Performance Improvement ........................................................................................ 13
viii
Using 360-degree Feedback in Organizations
Susan Gebelein
Employee Development: Multi-rater Feedback Goes Strategic ................................ 13
Kathleen A. Guinn
Assessment Techniques for Top Executives ............................................................. 14
Eric L. Harvey
Turning Performance Appraisals Upside Down........................................................ 15
Joy Hazucha, Sarah Hezlett, and Robert J. Schneider
The Impact of 360-degree Feedback on Management Skills Development .............. 15
William J. Heisler
360-degree Feedback: An Integrated Perspective ..................................................... 17
Marcie S. Hirsch
360 Degrees of Evaluation ........................................................................................ 17
Robert Hoffman
Ten Reasons You Should Be Using 360-degree Feedback ....................................... 18
Katherine Holt, Raymond A. Noe, and Marcie Cavanaugh
Managers Developmental Responses to 360-degree Feedback ............................... 19
John E. Jones and William Bearley
360 Feedback: Strategies, Tactics, and Techniques for Developing Leaders.......... 20
Robert E. Kaplan and Charles J. Palus
Enhancing 360-degree Feedback for Senior Executives ........................................... 21
Manuel London and Richard W. Beatty
360-degree Feedback as a Competitive Advantage................................................... 23
Manuel London and James W. Smither
Can Multi-source Feedback Change Self-evaluation, Skill Development,
and Performance? ...................................................................................................... 24
Joann S. Lublin
Turning the Tables: Underlings Evaluate Bosses ...................................................... 25
Kate Ludeman
To Fill the Feedback Void ......................................................................................... 25
Marc Marchese and John J. McGowan
The Power of 360-degree Feedback .......................................................................... 26
ix
Michelle Martinez
Multi-source Better Describes Peer Feedback .......................................................... 27
Cynthia D. McCauley and Russell S. Moxley
Developmental 360: How Feedback Can Make Managers More Effective .............. 27
Robert McGarvey and Scott Smith
When Workers Rate the Boss .................................................................................... 28
Kathleen Melymuka
Im OK, Youre OK .................................................................................................. 29
John F. Milliman, Robert A. Zawacki, Carol Norman, Lynda Powell,
and Jay Kirksey
Companies Evaluate Employees from All Perspectives ........................................... 30
Allan M. Mohrman, Jr., Susan M. Resnick-West, and Edward E. Lawler, III
Designing Performance Appraisal Systems .............................................................. 31
Joel Moses, George P. Hollenbeck, and Melvin Sorcher
Other Peoples Expectations...................................................................................... 32
George P. Nicholas
Upward Trend Continues in Appraisal Process ......................................................... 32
Dianne Nilsen and David P. Campbell
Self-Observer Rating Discrepancies: Once an Overrater, Always an Overrater? ..... 33
Kenneth M. Nowack
Self-assessment and Rater Assessment as a Dimension of Management
Development .............................................................................................................. 34
Kenneth M. Nowack
360-degree Feedback: The Whole Story ................................................................... 35
Brian OReilly
360-degree Feedback Can Change Your Life ........................................................... 35
Shirley Parker-Gore
Perception Is Reality: Using 360-degree Appraisal against Behavioral
Competencies to Effect Organizational Change and Improve Management
Performance ............................................................................................................... 36
Helen Peters
Peer Coaching for Executives ................................................................................... 37
List of Annotations
x
Using 360-degree Feedback in Organizations
Catherine Romano
Fear of Feedback ....................................................................................................... 38
Warren Shaver, Jr.
How to Build and Use a 360-degree Feedback System ............................................ 38
James W. Smither, Manuel London, Nicholas L. Vasilopoulos,
Richard R. Reilly, Roger E. Millsap, and Nat Salvemini
An Examination of the Effects of an Upward Feedback Program Over Time .......... 39
Sandra Sweet
Human Resource Management: Compensation Reporter ......................................... 40
TEAMS, Inc.
Multi-source, 360 Feedback Survey Results ........................................................... 41
Walter W. Tornow
Perceptions or Reality: Is Multi-perspective Measurement a Means or an End?...... 42
Ellen Van Velsor and Jean B. Leslie
Feedback to Managers, Volume II: A Review and Comparison of Sixteen
Multi-rater Feedback Instruments ............................................................................. 42
Ellen Van Velsor, Jean B. Leslie, and John W. Fleenor
Choosing 360: A Guide to Evaluating Multi-rater Feedback Instruments
for Management Development .................................................................................. 43
Mary N. Vinson
The Pros and Cons of 360-degree Feedback: Making It Work ................................. 44
Stephen J. Wall
Feedback on Feedback .............................................................................................. 45
Peter Ward
A 360-degree Turn for the Better .............................................................................. 46
Francis J. Yammarino and Leanne E. Atwater
Self-perception Accuracy: Implications for Human Resource Management ............ 47
Gary Yukl and Richard Lepsinger
How to Get the Most Out of 360-degree Feedback................................................... 48
xi
Acknowledgments
We would like to express our appreciation to several people at the
Center for Creative Leadership (CCL) for their roles in the completion of this
bibliography. First, we thank Russ Moxley, then director of Leadership
Resources at CCL and currently the director of CCLs nonprofit initiative, for
recognizing the need for an introductory review of the literature on the
current state of 360-degree feedback practice. Additionally, we are indebted
to several people who helped in the search for this information: Peggy
Cartner, Maxine Dalton, Bill Howland, Tom Kealey, and Carol Keck. We
also wish to thank Nancy Burton, Craig Chappelow, Kris Downing, Karen
Hornfeck, Jean Leslie, Don Prince, and Walter Tornow for their continued
interest in and support of this project.
We are grateful to our reviewers, Gina Hernez-Broome, Roger Conway,
Betsy Jacobson, and Barbara Neisendorf, whose suggestions were most
helpful in revising this report. And, finally, thanks to the members of the
Publication Review CommitteeMarcia Horowitz, Kelly Lombardino,
Dianne Nilsen, Joan Tavares, Jill Wachholz, and Martin Wilcoxfor their
work on this publication.
1
Introduction
One of the more important ways that employees can develop is to
receive ratings of their performance from their co-workersbosses, peers,
subordinates, and others. Known as 360-degree feedback, this activity is now
widely used by human resources professionals and in leadership development
programs. As its popularity has increased, however, so has the number of
publications written on its use. Anyone looking for information about 360-
degree feedback faces a dilemma: where to begin?
Thus, the purpose of this bibliography is to provide an introduction to
the available literature on the use of 360-degree feedback in organizations,
offering a foundation on which more detailed knowledge of it can be built.
The fifty-six works annotated in the first section were selected from books
and articles that convey the current state of practice, and they focus on its
use in organizations for such purposes as assessment for development and
performance appraisal. (Additional unannotated references can be found in
Appendix A.)
In selecting these works, we found the following areas mentioned
repeatedly: (1) history, (2) benefits, (3) tactics, (4) strategies, and (5) future
trends. To complement the information in the annotations, each of these areas
is discussed in the second section of this report within the framework of these
frequently asked questions: How and when did 360-degree feedback develop?
What are its benefits? How should it be used? How can it be integrated into
development programs in organizations? What does the future hold for 360-
degree feedback?
This bibliography does not attempt to answer these questions defini-
tively for the reader. It does, however, attempt to reflect the available knowl-
edge in these areas. Further, it provides a foundation on which judgments
about the use of 360-degree feedback in organizations can be based.
The literature was located in part by searching two relevant databases
Psych-Lit (January 1987 to June 1994) and ABI-Inform (January 1987 to June
1994)with additional sources up to 1997 also included. Only current
(published since 1990), relevant sources were used. Other information was
gathered from various contacts in the field and from on-line research.
(Complete details on the criteria used to select the literature is located in
Appendix B.)
2 Using 360-degree Feedback in Organizations
3
Annotated Bibliography
This section contains annotations of fifty-six works. They were ob-
tained from a search of the literature by using key terms such as feedback,
assessment, appraisal, evaluation, multi-rater, multisource, and 360 degree.
Annotations are listed alphabetically by the last name of the first author. (The
indexes at the end of this report provide complete listings of authors and
titles.)
David Antonioni. Designing an effective 360-degree appraisal feedback
process. Organizational Dynamics, April 1996, pp. 24-38.
Antonioni discusses twelve factors important to the design of a success-
ful (that is, effective) 360-degree-feedback process: (1) Raters prefer the use
of 360-degree feedback for developmental purposes as opposed to using
feedback for compensation purposes. Such use of 360-degree feedback could
produce negative reactions from participants. (2) Written and descriptive 360-
degree feedback may be more helpful than rating-scale information.
(3) Managers prefer accountability for ratings by the rater. Raters, however,
prefer their ratings to be anonymous. (4) Direct report raters (subordinates)
who were held accountable for upward feedback (their names were associated
with the feedback) gave their managers higher ratings. Anonymous raters
gave lower ratings. (5) Managers indicated the following about the nature of
the feedback they received: 25% was expected positive feedback, 30% was
unexpected positive feedback, 20 to 30% was expected negative feedback,
and 15 to 20% was unexpected negative feedback. (6) Raters estimated that
19% of managers would be surprised by low ratings in the feedback. (7) Only
half of the raters reported that managers had shared summary results of the
upward appraisals. (8) Managers who score high on achievement motivation
(that is, want to succeed) and who value feedback are more likely to discuss
results with their raters. (9) Managers are not likely to develop specific goals
or action plans based on 360-degree feedback. (10) Improvement of low
performance ratings is left to the ratee. (11) Seventy-two percent of ratees
reported that supervisors did not follow up on action plans related to 360-
degree feedback. (12) Eighty-seven percent of ratees felt raters had not
recognized their improvement efforts.

Leanne E. Atwater and Francis J. Yammarino. Self-other rating agreement: A
review and model. Research in Personnel and Human Resources
Management 15, 1997, pp. 141-164.
According to the authors, self-other rating agreement is the degree to
which self-perceptions are congruent with the perceptions of others. These
perceptions are usually captured with 360-degree-feedback instruments.
Atwater and Yammarino present a model of self-other rating agreement based
on a review and integration of the existing literature in this area. The authors
offer forty propositions that link demographic (for example, gender and age)
and personality characteristics, cognitive processes, job experience, and
contextual factors to differences between self- and others ratings. They also
propose a scheme for categorizing self-raters based on the amount of agree-
ment with others ratings. Twenty-four propositions related to relationships
between self-other rating agreement and individual and organizational out-
comes are presented. For example, in general, very positive individual and
organizational outcomes are expected for individuals whose self-ratings are in
agreement with others that their performance is good.

H. John Bernardin, Sue A. Dahmus, and Gregory Redmon. Attitudes of first-
line supervisors toward subordinate appraisals. Human Resource
Management 32:2&3, Summer/Fall 1993, pp. 315-324.
This article describes a study of three groups of supervisors in a large
federal agency on the subordinate feedback aspects of performance appraisal.
The participants were first-line supervisors in an organization that was
interested in finding alternatives to traditional, top-down appraisals. Ques-
tionnaires were completed that would provide feedback to supervisors and
evaluate their attitudes toward appraisal. Group 1 received feedback from
both managers and subordinates. Group 2 received feedback only from their
managers. Group 3 received feedback from subordinates only.
The focus on attitudes toward upward appraisal, along with previous
research, led to four hypotheses: (1) Group 1 will find the feedback more
useful for improving performance than the groups who receive feedback from
only one source. (2) Attitudes of Group 1 will be more positive (regarding the
appraisal process) than attitudes of those receiving only their managers
feedback. (3) Group 3 will be more concerned about the use and impact of
5
subordinate feedback than those receiving feedback from both subordinates
and managers. (4) Attitudes of those receiving subordinate appraisal will be
more positive than supervisors who do not receive subordinate appraisal.
Overall, the results supported the incorporation of subordinate appraisal
into the appraisal process. The data also support a more comprehensive, 360-
degree approach, but authors assert that cautionary actions should be taken to
ensure the positive aspects of this method of assessment. A comprehensive,
360-degree approach provides the opportunity to identify critical behaviors
and the opportunity to determine qualified sources of feedback who can
provide valid and useful data.

David W. Bracken. Straight talk about multirater feedback. Training and
Development, September 1994, pp. 44-51.
Bracken discusses factors that contribute to the success or failure of
multi-rater feedback systems. Along with the increasing use of multi-rater
feedback for development, there is an increased use of feedback for appraisal
and decision making for pay and promotion. The author offers a definition of
a successful multi-rater process: (1) It is reliable and provides consistent
ratings. (2) It is valid because it provides job-related feedback. (3) It is easy to
use, understandable, and relevant. (4) It creates positive change on an indi-
vidual and organizational level. Success or failure in implementing a system
can occur at any stage of the process.
Brackens system for a successful multi-rater process includes: process
design and planning, instrument development, instrument design, administra-
tion, feedback processing and reporting, and action-planning. He discusses in
detail each of these stages and addresses key issues.
Bracken concludes that if multi-rater feedback is to be used for person-
nel decision making, it is imperative that the data be used consistently and
accurately. A possible approach is to use multi-rater feedback to create a
developmental action plan and record the plan in the appraisal process. The
manager could then be evaluated on progress toward the development plan.
Bracken advises careful consideration of relevant questions and issues prior
to the implementation of a multi-rater feedback system.

Stphane Brutus, John Fleenor, and Manuel London. Elements of effective
360-degree feedback. In W. Tornow & M. London (Eds.), Maximizing
the Value of 360-degree Feedback: A Process for Successful Individual
and Organizational Development. San Francisco: Jossey-Bass, in press.
According to the authors, developmental activities using 360-degree
feedback have become one of the most popular methods for developing
managers in organizations. More and more organizations are using this
process to help improve the effectiveness of their managers. Many Fortune
500 companies are either using 360-degree feedback or are considering its use
in the near future.
In this chapter, the contribution of 360-degree feedback to individual
development is described. Because the practice of 360-degree feedback is
generally ahead of research, there are not a lot of studies that focus specifi-
cally on this process. This chapter, therefore, attempts to connect research
findings and theoretical arguments from different areas of the behavioral
sciences to build a case for the value of 360-degree feedback for individual
development.
The authors state that 360-degree feedback possesses the same basic
four elements as any control system: (1) a set of standards or goals, (2) an
information processor, (3) the collection of information relative to the stan-
dards, and (4) the use of this information to meet the standards. They argue
that the outcomes of 360-degree feedback are contingent on the quality of the
entire system. The feedback process must be based on appropriate perfor-
mance standards. Feedback, once provided, will enable individuals to develop
along these standards. Performance standards not only are essential for
individual development, but they represent the link to organizational strategy.
This chapter also highlights the importance of measurement in the 360-
degree-feedback process. Each rater possesses a unique and valid perspective
from which the performance of the focal manager can be assessed. When
valid performance ratings from multiple perspectives are linked to develop-
mental planning, goal-setting, and organizational support, the 360-degree
feedback process can lead to positive outcomes for the focal manager as well
as the organization as a whole.

7
Matthew Budman and Berkeley Rice. The rating game. Across the Board
31:2, February 1994, pp. 35-38.
This article describes 360-degree feedback as an increasingly used
option for performance appraisal and discusses its use in various organiza-
tions. The trend toward using 360-degree feedback (as opposed to exclusively
using top-down, supervisory ratings) allows employees to have control over
evaluations. For organizations shifting to the use of 360-degree performance
appraisals, a key factor is how timing affects employee trust in the process.
Although managers may desire a new approach to traditional appraisals, they
are often wary and insecure about receiving feedback from peers and subordi-
nates. Although training can allay some fears, trust occurs only with time and
organizational support for using 360-degree appraisal. The authors discuss
various issues related to this method of appraisal: training, impact, managers
sensitivity to receiving upward feedback, and format of the feedback
assessment.
The use of 360-degree feedback in a team setting is examined by using
two examples of organizations. First, in a Quaker Oats plant in Topeka,
Kansas, it was found that when the feedback-and-appraisal system was not a
priority and when it was not reviewed, the ratings lost their impact both for
decision making and for development. Second, at W.L. Gore in Newark,
Delaware, team members recognized the importance of confidentiality of
ratings. Managers from both companies asserted that the appraisal system and
the feedback assessment must fit within the organizational culture.
The authors also discuss fairness issues involved with using 360-degree
feedback for decision making. Many companies use it for developmental
purposes only, because of questions about the fairness of peer and subordi-
nate ratings linked to pay and promotion. Additionally, users of 360-degree
feedback discussed the necessity of including customers perspectives in the
feedback assessment.
Whatever the specifics of the assessment, as organizations become less
hierarchical in structure, top-down performance appraisal will need to be
reconsidered. Patience and planning will allow the use of 360-degree feed-
back to be a key process in this reform.

Robert Carey. Coming around to 360-degree feedback. Performance, March
1995, pp. 56-60.
Carey presents various approaches to the use of 360-degree feedback, as
well as generalizations regarding the successful implementation of 360-
degree systems. He asserts that 360-degree feedback can create productive
relationships between managers and employees. To minimize risks involved
with its use, Carey makes these suggestions: (1) All parties should understand
what the feedback can really accomplish; it is the beginning of a larger
change process. (2) Consider enlisting the help of a third party, such as a
management consulting or a human resources development firm. (3) Do not
make 360-degree feedback the primary determinant of salary, promotions, or
termination. (4) Guarantee that feedback will be anonymous. (5) Leaders of
the organization must support the process.
Carey also describes three organizational approaches to 360-degree
feedback at AT&T, Trompeter Electronics, and Xerox. Finally, the issue of
how feedback is used is addressed. There are two main perspectives: Some
believe that 360-degree feedback should only be used for developmental
purposes, whereas some feel it can be effectively used for performance
appraisal and decision making. Ray Calvin, of Trompeter Electronics, de-
scribes his companys use of feedback for development and as part of ap-
praisals. All conclude that implementation of 360-degree-feedback systems
requires ample planning and precaution.

Dennis E. Coates. Multi-source feedback: Seven recommendations. Career
Development International 1:3, 1996, pp. 32-36.
The author discusses his seven suggestions for preventing 360-feedback
assessments from losing impact and effectiveness: (1) Learn the technology
before investing in it. (2) Make sure the organization is prepared for the 360-
degree process. (3) Use well-researched and well-constructed survey items.
(4) Protect the confidentiality of raters. (5) Use skilled facilitators to imple-
ment the process. (6) Follow up with developmental activities. (7) Separate
developmental feedback from personnel and compensation decisions.
Coates bases these suggestions on seven years of research and develop-
ment of 360-degree-feedback instruments. Each suggestion is followed by
information that will allow users to resolve issues, evaluate options, prepare
well, and increase the probability of success. The author asserts that experi-
9
ence has shown that 360-degree feedback can be a powerful tool, but it must
be used wisely. He predicts that 360-degree-feedback technology will con-
tinue to increase its versatility and accessibility. Careful preparation will
allow users to implement a 360-degree system that will meet organizational
needs.

Barbara Crystal. The 360 assessment. Healthcare Executive 9:6, November/
December 1994, pp. 18-21.
This article discusses 360-degree assessments and their impact on
healthcare employees and organizations. The growth of 360-degree assess-
ment can be attributed to various marketplace realities (such as downsizing
and mergers) and organizational transformations that require a new focus on
leadership and effectiveness. In the healthcare field, 360-degree assessments
can assist career planning, leadership development, cooperation and commu-
nication between individuals and departments, as well as foster preferred
leadership styles. It also provides executives with an understanding of the
difference between performance and expectations.
Crystal describes current methods of 360-degree assessment and offers
four guidelines for choosing an assessment process. (1) The process should
have been developed by experts trained in leadership assessment. (2) It
should assess leadership effectiveness and not be used for pay or promotion
decisions. (3) It should include one-on-one counseling and prepare a person-
alized action plan for continual leadership development. (4) It should provide
comprehensive evaluation of performance. This feedback should include
interpersonal skills, communication skills, administrative skills, and other
aspects of leadership effectiveness. Essentially, the 360-degree process
should elicit confidential data to be interpreted by facilitators, who can then
relate the results to the ratee. The process is designed to help executives
understand the steps needed to develop leadership effectiveness.
Crystal includes quotes from experts in healthcare and in leadership develop-
ment fields.

Maxine Dalton. Multi-rater feedback and conditions for change. Consulting
Psychology Journal: Practice and Research 48:1, 1996, pp. 12-16.
This paper discusses the use of multi-rater feedback for developmental
purposes versus appraisal purposes. One major distinction between these uses
is ownership of information. When multi-rater feedback is used for develop-
ment, the data are private or owned by the ratee. When used for appraisal, the
feedback data are not private. Dalton argues that using multi-rater feedback
for appraisal violates basic counseling principles about learning, growth, and
change. The first step in eliciting individual change is to set conditions so that
individuals will understand that their behavior is not being perceived as they
intend it to be. Hearing personal information that is different from ones own
perception can be difficult and painful. Using feedback for development
ensures that the data are private, providing the recipient with the psychologi-
cal safety necessary for acceptance. Dalton also asserts that if data are not
private (that is, used for appraisal) then the ratings themselves are often
affected. Evidence shows that raters tend to be lenient when their ratings will
affect a co-workers employment status.
Dalton identifies two motives for organizations wanting to use multi-
rater feedback for appraisal: It opens new markets for vendors, and HR
professionals assume that the information on the employee will be more
fair. She also gives three main reasons why this second assumption might
not be realistic: (1) The work of Herb Meyer and others at General Electric in
1965 [H. H. Meyer, E. Kay, and J. P. R. French, Jr. Split roles in performance
appraisal. Harvard Business Review 43:1, 1965, pp. 123-129] showed that
public and quantitative appraisals rarely produce change, regardless of who
appraises. (2) Public use of feedback data leads to rater leniency and, there-
fore, it is not valid or useful. (3) Subordinates may fear reprisal, thus a lack of
confidentiality can hinder the process. Dalton concludes that multi-rater
feedback used for appraisal will not produce change any more than any other
quantitative appraisal, and it may damage the credibility of the process. More
research is required on how ratings are affected in developmental (private)
versus appraisal (public) feedback conditions.

11
Wayne E. Denton. Developing employee skills to match company needs.
Credit World, May/June 1994, pp. 19-20.
Denton, a human resources vice-president at Limited Credit Services
(LCS) in Columbus, Ohio, describes 360-degree-feedback instruments, their
benefits, and the development of a 360-degree system at LCS. The author
states that the current rate of change in the world makes individual develop-
ment necessary to maintain an edge. Gathering valid and honest informa-
tion, especially at the executive and managerial levels, is difficult. Thus, it is
necessary to have leadership skills assessed in order to find skills and behav-
iors that need improvement. Denton states that 360-degree-feedback instru-
ments are the most effective type of leadership assessment available by
providing a complete picture of performance and effectiveness. The benefits
are especially effective in organizations that are trying to focus developmen-
tal efforts for managers. The instruments decrease bias and provide candid
data to channel managers into appropriate training programs or interventions.
Some instruments also provide participants with insight on the values and
skills that are necessary for a particular position or organization.
At LCS, an evaluation was needed that rated individuals against strate-
gic issues that the organization valued in its leaders. A 360-degree-feedback
instrument was used, showing managers their strengths and weaknesses. The
feedback was followed up with the creation of a development plan, involving
the manager, his boss, and the assessment facilitator. Re-administering the
instrument every eighteen to twenty-four months lets LCS managers
benchmark progress, and allows for continuing development of leadership
effectiveness.

Marvin M. Dunnette. My hammer or your hammer? Human Resource Man-
agement 32:2&3, Summer/Fall 1993, pp. 373-384.
This article reviews several articles found in the 1993 Human Resource
Management special issue on 360-degree feedback. Dunnette summarizes the
main points of these articles and addresses the lack of cohesion that exists in
this area of research. He also elaborates on the impact of 360-degree feedback
as an emerging human-resources management tool. In addition to its rel-
evance to organizational practices such as appraisal and development initia-
tives, 360-degree feedback has many psychological aspects: person percep-
tion, impression formation, individuals conceptions of self, impression
management, and behavioral change.
The articles discussed represent themes, issues, and trends that focus on
accuracy, content, methodological hazards, and organizational or individual
change. Dunnette provides a brief description of five 360-degree-feedback
instruments that were investigated by the authors in this special issue. These
instruments include Benchmarks
, Campbell Leadership Index
, Executive
Success Profile, Management Skills Profile, and Insight-Out. These instru-
ments are designed to provide feedback on managerial strengths and
weaknesses.

Colin C. Edwards. 360 degree feedback. Management Services 39:6, June
1995, p. 24.
Edwards discusses the emergence of 360-degree feedback as a tool for
facilitating personal development. He cautions potential users about the
composition of the questions on an assessment survey. Questions should not
prompt answers and should focus on removing preconceptions regarding the
person being evaluated. He also discusses the feedback received from a 360-
degree assessment, which can be surprising and insightful. It can focus on
talents, ethics, leadership, personality, habits, and both strengths and weak-
nesses. The most hurtful criticisms, however, tend to be those regarding
interpersonal skills. The information that is uncovered from co-workers can
illustrate how limited a persons self-understanding can be.
Edwards also asserts that the impact of feedback is affected by the
persons attitude prior to receiving feedback. A person who possesses a
positive outlook on development and the feedback process will maximize
360-degree feedbacks benefits. Users should also be aware of the ability of
the process to harm those that are unable to accept the feedback; restoring
such an employees self-esteem can be a difficult task. In addition, Edwards
suggests that 360-degree feedback should only be used for developmental
purposes, rather than for pay and performance decision making. He asserts
that the use of 360-degree feedback for these purposes could produce
invalid results.

13
Mark R. Edwards and Ann J. Ewen. 360 Feedback: The Powerful New
Model for Employee Assessment and Performance Improvement. New
York: American Management Association, 1996, 247 pages.
This book deals with the complexities and sensitive issues surrounding
the use of 360-degree feedback for performance appraisal rather than strictly
for development. Chapter 1 defines 360-degree feedback and outlines a brief
history of its benefits in relation to the changing nature of organizations.
Although accepted approaches to 360-degree feedback are varied, the authors
describe their technique for implementing and evaluating the process. Chap-
ters 2 and 3 provide background, various approaches, and applications for
360-degree systems.
The next section (Chapters 4, 5, and 6) deals with implementation of a
360-degree system. Chapter 7 discusses common mistakes in the feedback
process and how to avoid them. Issues such as technology, confidentiality,
safeguards, training, and norm comparisons are addressed. Chapter 8 dis-
cusses common criticisms of 360-degree feedback. According to Edwards
and Ewen, anticipation of and reaction to criticism is necessary to maximize
the possible benefits of 360-degree feedback.
Chapter 9 discusses the impact of 360-degree feedback. It has shown
some promise in terms of positive, measurable impact. The authors list the
following outcomes: improved employee satisfaction with the work environ-
ment, significant behavior changes aligned with the organizations objectives,
and better individual and team performance that extends to external custom-
ers. The authors describe various ways of measuring impact on performance
and development, as well as reaction to feedback. Chapter 10 closes with an
outlook for the future of 360-degree feedback.

Susan Gebelein. Employee development: Multi-rater feedback goes strategic.
HRFocus, January 1996, pp. 1, 4-6.
This article discusses how to build a multi-rater feedback process that
will support organizational goals by facilitating employee development that is
aligned with corporate strategy. Many 360-degree systems fall short of their
potential because they are used merely for individual development and do not
support organizational change. There are several phases to implementing
multi-rater assessment that will support organizational goals. First, the organi-
zation must decide what the assessment is to accomplish by clarifying the
companys goals and strategies; defining the scope of the process; and plan-
ning the process in terms of who will participate, how the instrument will be
designed, and how often it will be used. Second, the system must be linked to
a competency model that defines the skills and behaviors needed by employ-
ees in order to achieve the companys goals. Third, planning administration of
the instrument involves educating those involved with the process and then
determining the strategy for interpreting results. The steps of implementation
from this point are sequential: (1) Hold kickoff meetings. (2) Roll out the
process. (3) Focus on using the feedback for development, rather than for
making personnel decisions. (4) Evaluate the process to determine its impact
and effectiveness.

Kathleen A. Guinn. Assessment techniques for top executives. Career Devel-
opment International 1:3, 1996, pp. 9-14.
This article discusses the purpose of feedback for executives, barriers to
its implementation, and three techniques for providing useful feedback.
Individuals within an executives circle of influence are often reluctant to
give feedback openly for fear of reproach. As a result, many top executives
do not receive the feedback necessary to change and improve their perfor-
mance. Also, many executives, having enjoyed success in the past, do not
perceive a need to change and may feel a fear of change. The benefits of
feedback and development opportunities for executives affect both individu-
als and organizations, making the assessment a worthwhile pursuit. Three
techniques for assessment are offered here: psychological testing, multisource
assessment, and individual competence assessments.
Guinn briefly discusses what multisource assessment involves and how
its use has grown. Despite the great impact of feedback from multiple
sources, some are concerned about the subjectivity of the process. She con-
tends that subjectivity can be decreased by the questionnaire design and
certain precautions can ensure the collection of reliable and valid data.
Although multisource assessment can be time consuming, the payoff is rich
and likely to be accepted by executives.
Regardless of how it is collected, feedback should provide information
about appropriate behaviors that can be recognized and changed. This can
involve the use of a competence model, which describes the appropriate skills
for the executive.
15
Guinn includes four case studies of assessment systems as they relate to
evaluation and feedback issues. Executive assessments provide valuable
feedback that can be applied to various initiatives within an organization and
can affect an organization and its employees by facilitating development.

Eric L. Harvey. Turning performance appraisals upside down. Human Re-
source Professional 7:2, March/April 1994, pp. 30-32.
This article discusses the use of 360-degree feedback for performance-
appraisal purposes. Performance appraisals tend to be regarded negatively in
the business environment. Harvey attributes this to four common aspects of
performance appraisal: (1) Needs cannot be entirely met by one performance
evaluation. (2) Appraisals from a single, supervisory source are inherently
problematic. (3) The effect of the appraisal and feedback on employee devel-
opment can be lost or overlooked. (4) Many managers are not capable of
providing appropriate evaluations.
Harvey then asserts that there are ways to improve the effectiveness of
employee evaluation and performance assessment. He offers three ideas:
(1) Supervisory and single-source assessments should be replaced with multi-
rater, or 360-degree, feedback collected from various sources. (2) Pay and
promotion decisions should be separate from feedback collected for develop-
ment only. (3) The improvements that result from performance appraisal
should be received as positive changes within the organization.

Joy Hazucha, Sarah Hezlett, and Robert J. Schneider. The impact of 360-
degree feedback on management skills development. Human Resource
Management 32:2&3, Summer/Fall 1993, pp. 325-351.
The purpose of this study is to better understand 360-degree feedback
as a management-development intervention. It addresses some key questions,
including: How do managers skills develop after feedback? How do initial
feedback, development efforts, and environmental support relate to skills
development? Which developmental activities are most effective? What
affects the level of effort managers put into their development?
At a large midwestern utility company, 198 managers completed the
Management Skills Profile (MSP), a 360-degree-feedback instrument. Two
years later, the same managers were given the chance to retake the MSP and,
in addition, the Management Development Questionnaire (MDQ). The MDQ
is designed to measure effort, change, developmental activities, and support
for development. Of the 198 managers, 48 took the MSP twice and also
received the follow-up MDQ.
Based on previous research, the authors formulated the following
hypotheses:
1a: As a group, skills will improve from the initial assessment (Time
1) to the follow-up (Time 2), as measured by self- and others ratings.
1b: The agreement between self- and others ratings will increase
from Time 1 to Time 2.
2: Managers whose skills are rated more highly (at Time 1) will
receive larger pay increases and will be more likely to receive promo-
tions between Times 1 and 2.
3a: Managers who put more effort into skill development will be
more likely to improve management skills.
3b: Activities that require follow-through will be more effective than
those that focus on understanding results and formulating intentions.
3c: Managers who receive more support from superiors will be more
likely to improve their skills.
3d: Managers who receive more organizational support will be more
likely to improve their skills.
4a: Managers receiving more favorable ratings at Time 1 will be
more likely to complete developmental activities.
4b: Managers whose feedback from others is closer to self-ratings
will be more likely to complete developmental activities.
The major finding is that receiving 360-degree feedback was generally
helpful, but follow-through on development was the most critical factor in
improving skills. These results have implications for (1) how managers can
develop their own skills, (2) how supervisors can support others develop-
ment, and (3) how the organization can facilitate development. The absence
of a comparison group in this study, however, makes determination of the
actual effects of 360-degree feedback on skill improvement difficult.

17
William J. Heisler. 360-degree feedback: An integrated perspective. Career
Development International 1:3, 1996, pp. 20-23.
This article is a case study of a 360-degree assessment implemented in
one of the operating divisions of Tenneco. The company developed a 360-
degree assessment to measure nine leadership competencies thought to be
necessary to the organizations culture. Heisler uses Tennecos approach to
support the belief that the 360-degree-feedback process must be implemented
slowly; it must be seen as more than a data collection system. Heisler sug-
gests that 360-degree assessments be designed to measure behaviors and
characteristics that relate to an organizations strategy. The feedback process
must also be supported with tools for the analysis and interpretation of data.
Finally, he says that the process should be followed by developmental plan-
ning for the participants. The purpose of 360-degree systems is not only
assessment but also to provide feedback to stimulate improvement and to
promote an organizations strategic business objectives.

Marcie S. Hirsch. 360 degrees of evaluation. Working Woman Magazine.
August 1994, pp. 20-21.
Hirsch provides a description of various aspects of 360-degree-feedback
programs, planning, and impact, using information gathered from profession-
als in the field. She states that dissatisfaction with traditional performance
appraisal has increased; in 1992, 68% of companies cited better performance
management as a priority.
George Hollenbeck, an organizational psychologist, gives two reasons
for the need for better evaluation. First, smaller organizations need to maxi-
mize individual effectiveness. Second, employees in downsized organizations
need better feedback to alleviate insecurity and stress. Organizations are
increasingly using 360-degree feedback to provide more accurate evaluation
and effective communication. After 360-degree evaluations are completed,
many organizations, with the help of either a human resources professional or
an outside consultant, encourage development initiatives. Stride Rite, for
example, assists managers with an individual development plan.
Hirsch also discusses preparation for the implementation of a 360-
degree-feedback program. Cost varies with program size, degree of
customization, and amount of outside consulting involved. Comparison of
costs between standardized instruments (for example, Benchmarks
from the
Center for Creative Leadership) and customized programs show customi-
zation to be fairly costly.
Whether using a customized program or a standardized instrument,
veteran users stress the need for planning. The following five steps are
recommended: (1) Establish the programs goals. Linda Kane, of Stride Rite,
discusses the importance of disconnecting 360-degree evaluation from pay
decisions. She suggests that 360-degree feedback be used for management
development and coaching initiatives, while other techniques be used for
performance appraisal. (2) Decide who will be evaluated. Options range from
comprehensive implementation for all managers to a targeted implementation.
Targeting high-potential or low-performing employees can be effective.
Implementing feedback assessment across the company can reinforce pre-
ferred behaviors. (3) Decide on a standardized program or a customized one.
Statistical reliability and validity are strong arguments for standardized
instruments, whereas customized instruments are more specific to the needs
of the organization. (4) Decide how many evaluations to collect from peers,
subordinates, supervisors, and customers. The suggested range is from three
to four. The average instrument uses five to ten. (5) Communicate the rules
regarding anonymity and use of the feedback. Anonymity and developmental
use of feedback will ensure honest evaluations, according to organizational
psychologist and consultant Lynn Summers.

Robert Hoffman. Ten reasons you should be using 360-degree feedback. HR
Magazine, April 1995, pp. 82-85.
Hoffman describes the increased use of 360-degree feedback in organi-
zations. The technique can be used for developmental purposes only, but it is
increasingly being used for performance appraisal. It enables companies to
align employee performance with the organizations needs and overall
strategy [which] leads to the development of a more involved, less hierarchi-
cal work force. Hoffman offers these ten observations about what 360-
degree feedback can do: (1) define corporate competencies; (2) increase the
focus on customer service; (3) support team initiatives; (4) create a high-
involvement workplace; (5) decrease hierarchies and promote streamlining;
(6) detect barriers to success; (7) assess developmental needs; (8) lessen
discrimination and bias; (9) identify performance thresholds; and (10) be easy
to implement.
19
Hoffman also considers issues that must be addressed to maximize the
benefits of 360-degree feedback: (1) How will the process be communicated
to the organization? (2) Will the process be combined with other appraisal
systems? (3) What training will be provided to the participants? (4) Should
outside consultants be used? (5) Should an off-the-shelf program or a custom-
ized program be implemented? (6) Is a computer-based evaluation or a paper-
and-pencil form best for the organization? Full consideration of these issues
should allow for smooth implementation of 360-degree feedback.

Katherine Holt, Raymond A. Noe, and Marcie Cavanaugh. Managers Devel-
opmental Responses to 360-degree Feedback. Unpublished manuscript,
1996, 47 pages.
This study investigates the impact of individual differences, feedback
characteristics, instrument characteristics, and organizational support on
managers developmental responses to 360-degree feedback. The authors
define these individual difference factors: (1) feedback-seeking behavior: a
managers propensity to actively seek feedback related to organizational and
personal goals, (2) learning orientation: an individuals belief regarding
whether behavior can be learned and modified, and (3) self-confidence: the
belief in ones own competence to handle a variety of situations.
Three hypotheses are formed relating to the impact of these individual
differences: (1) There is a positive relationship between feedback-seeking
orientation and developmental response to 360-degree feedback. (2) There is
a positive relationship between learning orientation and developmental
response. (3) Self-confidence is positively related to developmental response.
Two hypotheses are formed relating to feedback characteristics such as
confidentiality, sign (Is the feedback positive or negative?), accuracy, timing,
quantity, and specificity: (4) There will be a positive relationship between the
number of people who see the feedback and developmental response.
(5) There will be a negative relationship between perceived feedback sign and
developmental response (that is, negative feedback will elicit more response
than positive). The next two hypotheses related to the characteristics of the
feedback system: (6) Managers required to participate in development plan-
ning discussion will have more response than those not required to plan
development. (7) There will be a positive relationship between the complete-
ness of the managers development plan and developmental response. The
final hypotheses relate to the organizational conditions for development:
(8) Organizational support for development will positively relate to manag-
ers response to feedback. (9) Total Quality Management (TQM) will be
positively related to managers developmental response.
A sample of 236 managers from 25 organizations completed the Man-
agement Skills Development Profile (a 360-degree instrument from Personnel
Decisions, Inc.) two different times. The results were as follows: Characteris-
tics of the feedback and feedback system as well as TQM philosophy were
more strongly related to developmental responses than individual differences
(self-confidence, learning orientation, feedback-seeking orientation). In fact,
individual differences did not affect the impact of the 360-degree feedback
(contrary to hypotheses 1 to 3). Contrary to hypothesis 4, characteristics of
feedback itself (that is, publicness) was not positively related to response.
The sign of the feedback (hypothesis 5) was not related to developmental
response. Hypotheses 6 and 7 were partially supported. Hypotheses 8 and 9
received only limited support; organizational support did not affect develop-
mental impact of the feedback.
The authors suggest that organizations should encourage managers
to take initiative in self-development and consider mandating extensive
development-planning sessions for 360-degree-feedback programs. The study
showed these two factors to have a great deal of impact on the developmental
response to the feedback.

John E. Jones and William Bearley. 360 Feedback: Strategies, Tactics, and
Techniques for Developing Leaders. Amherst, MA: HRD Press &
Lakewood Publications, 1996, 184 pages.
Jones and Bearley begin by discussing the three major shifts in organi-
zations that make the use of multi-rater feedback necessary. First, focus has
shifted from management skills to leadership skills. Second, there has been a
shift from dependency to self-responsibility in career planning. Third, there
has been a shift from traditional hierarchy and structure to organizational
culture. Organizations are best served when they provide employees with
information necessary for their own leadership development, and this can best
be achieved with multi-rater feedback.
This book is also a how-to manual that describes steps to take and
pitfalls to avoid in implementing multi-rater feedback. The authors explore
the several common uses for multi-rater feedback: (1) system intervention,
(2) team building, (3) components of training or development courses,
21
(4) assessment of training needs, (5) embedding upward feedback into em-
ployee surveys, (6) assessment of training outcomes, (7) return-on-investment
analyses regarding development schemes, and (8) linking feedback to perfor-
mance appraisal.
The authors conclude with chapters that outline the creation and imple-
mentation of multi-rater feedback programs, as well as ways of dealing with
reliability of programs in general. They assert that no instrument is inherently
valid or invalid: Validity is always situation-specific; it does not reside in the
questionnaire or its items. They also suggest that results of the assessment be
compared to normative data (descriptive statistics on a known group).
According to Jones and Bearley, the most important goal of multi-rater
feedback is to inform and motivate feedback recipients to engage in self-
directed action-planning for improvement. It is the feedback process, not the
measurement process, that generates the real payoffs. They describe various
developmental follow-up strategies that will maximize the payoffs of multi-
rater feedback. It is best used to facilitate employee development; however,
many organizations are experimenting with multi-rater feedback for perfor-
mance appraisals.
These two uses differ in one important respect: Performance appraisal is
used in retrospect to evaluate what employees have done, whereas multi-rater
feedback used for development evaluates how the employees do their jobs.
The authors contend that the integration of 360-degree feedback and perfor-
mance appraisal will require experimentation. Finally, they say that the ideas
behind such feedback are not a fad, and that 360-degree feedback is necessary
in the shifting climate of organizational paradigms and structure.

Robert E. Kaplan and Charles J. Palus. Enhancing 360-degree Feedback for
Senior Executives. Greensboro, NC: Center for Creative Leadership,
1994, 19 pages.
This report discusses the use of what the authors refer to as enhanced
360-degree feedback. This differs from traditional 360-degree feedback in
that it is used for senior-level executives and generally provides them with a
richer feedback experience. The authors give two main reasons why senior
executives need this enhanced feedback experience. First, they face a broader
range of challenges than other managers. Second, enhanced feedback is more
appropriate for the psychological makeup of senior executives, rather than
focusing on behavior alone.
Enhanced feedback goes beyond standard 360-degree feedback in both
amount and type of data collected. Several types of information are collected
with enhanced 360-degree feedback in addition to the data collected by
standard 360-degree processes. These are: (1) numerical ratings plus verbatim
comments, (2) data from the workplace plus data from personal life,
(3) behavioral information plus information on motivation, and (4) data on
the present plus information on early history.
Not all of these additional sources need to be used, and sources should
be chosen carefully. The more data that are collected, the greater the impact,
both positive and negative. This amount of feedback can be overwhelming to
its recipients. The risk involved with enhanced feedback can be great, and it is
necessary to evaluate all aspects of its use.
The authors lay out a plan for its effective implementation. First, the
right feedback-provider staff must be selected. The staff that provides the
assessment and feedback should be: competent in management development,
proficient in personal development, effective with senior managers, and
mature. Second, the right participants must be selected. The authors discuss
three ways of selecting participants: (1) the organization nominates a candi-
date, (2) the candidate volunteers, and (3) the service provider selects the
candidate. It is important to note that participation must be voluntary, regard-
less of who chooses potential participants. One human resources executive
observed, Allowing them to say no is important.
Finally, to safely use enhanced 360-degree feedback, both the service
provider and the organization must assist the executive throughout the entire
process. This means focusing on these suggestions: (1) The facilitators must
prepare the participant. (2) The facilitators should stress the positive aspects
of the feedback. (3) The service provider should never attack or coerce the
participant. (4) The service provider should stay in touch after the feedback
session. (5) The service provider should help the executive get closure.
(6) The facilitators should challenge the executive to use the feedback to
make real progress. (7) The service provider should build a strong relation-
ship with the client.
According to the authors, The key to minimizing the risk of using
enhanced feedback is to reduce uncertainty. If enhanced feedback is used
safely, then the executive gains a unique awareness of his or her impact.
Enhanced 360-degree feedback stimulates natural development processes by
focusing attention on issues that might later come to a head on their own.

23
Manuel London and Richard W. Beatty. 360-degree feedback as a competi-
tive advantage. Human Resource Management 32:2&3, Summer/Fall
1993, pp. 353-372.
This article focuses on the use and design of multi-rater programs to
gain a competitive advantage. Competitive advantage is defined as providing
a product or service perceived by its customers as adding value and doing so
in a way that is unique and difficult for a competitor to readily duplicate.
The authors begin by addressing various considerations for 360-degree
practice. They assert that the process of developing effective multi-rater
feedback involves five steps: (1) establishing and communicating the purpose
of the effort, (2) developing the content and format of the assessment,
(3) establishing a report format and procedures for distributing results,
(4) facilitating the use of the results by managers, and (5) following up to
assess progress and improvement. It may also involve training raters to use
the process, and training leaders or facilitators to interpret and utilize results
for development.
The authors then compare the use of multi-rater instruments for devel-
opment and for performance appraisal. The primary difference is that perfor-
mance appraisal requires the measurement of results: what is done. Develop-
mental applications require measurement of behaviors and competencies: how
the job is done. Performance appraisal is performed primarily for evaluation
and organizational consequences, such as pay treatment or promotion; it is
rarely geared toward individual or organizational development. The remain-
der of this section focuses mainly on similarities and differences between
developmental and evaluative use of multi-rater feedback.
The risks and rewards of using multi-rater feedback are also addressed.
The rewards can include the improvement of two-way communication,
increased formal and informal communication, building effective work
relationships, increased employee involvement, and the demonstration of
respect for employee opinions. The authors then discuss the process of
designing multi-rater feedback systems that reduce risk and maximize reward,
thus augmenting competitive advantage. This process focuses on (1) content,
(2) involvement of employees in program design, (3) item types and format,
(4) item relevance, (5) implementation, (6) frequency, (7) uses for the feed-
back, (8) self-assessment, and (9) report format.
The article concludes with the authors call for continued research and
development of multi-rater feedback and assessment. The areas in need of
research include the types of multi-rater feedback processes, the differential
outcomes of use for development or evaluation, criteria of feedback success,
and, most importantly, the use of multi-rater feedback and assessment to gain
competitive advantage.

Manuel London and James W. Smither. Can multi-source feedback change
self-evaluation, skill development, and performance? Personnel Psy-
chology 48, 1995, pp. 375-390.
This paper explores how 360-degree feedback can go beyond traditional
performance appraisal by providing information that can be used for behavior
change and employee development. The authors present a model and seven
associated propositions specifying how differences in perceptions of perfor-
mance between the focal individual and his or her co-workers can affect
factors such as self-image, goal-setting, behavior, and performance. Compo-
nents in their model include personal variables, such as feedback-seeking
behavior, self-monitoring, and impression management. Situational factors in
the model include how the feedback process is implemented and organiza-
tional performance standards. According to the authors, the literature on
multi-rater feedback has not devoted much attention to these issues.
The authors also discuss two basic assumptions often made about
multisource feedback: (1) multi-rater feedback helps the focal individual
develop a more accurate self-concept, and (2) multisource feedback provides
information that is valuable for development and performance improvement.
The first assumption suggests that others ratings are always accurate com-
pared to self-ratings; however, research has shown that these others often do
not agree among themselves. Additionally, there is little direct evidence
concerning the effects of multi-rater feedback on performance. There is some
evidence that respondents modify their ratings when the purpose of the
assessment is performance appraisal, which may result in inaccurate ratings
from others.
London and Smither argue that ratings from various sources (for ex-
ample, bosses, peers, and direct reports) should not be combined into a
composite other rating. These sources often have different perspectives of
the focal individuals behavior, and may not agree on that persons level of
performance. So, a managers self-ratings may be in agreement with the
ratings of peers but not agree with the ratings of direct reports. When these
ratings are combined for feedback, the focal individual cannot determine if
his or her self-ratings agree or disagree with the ratings from the various
25
sources. This can have a negative impact on the usefulness of these ratings for
developmental purposes.
The authors conclude by summarizing research issues and methods that
can be used to improve the understanding and effectiveness of multisource
feedback.

Joann S. Lublin. Turning the tables: Underlings evaluate bosses. The Wall
Street Journal, October 4, 1994, p. B1.
This article describes the impact of 360-degree assessment in view of
its recent increased use, particularly the impact of upward feedback (that is,
from subordinates upward). The author states that AT&T recently imple-
mented upward feedback (alone or as a 360-degree program) with over
68,000 managers in a year. Paul Jones, a performance-improvement manager
at Dow Chemical Company, asserts that ensuring proper leadership is essen-
tial for organizations to keep a competitive edge. Proponents of 360-degree
assessments contend that feedback from multiple levels is effective because it
provides valuable insights to managerial performance.
One of the criticisms of upward feedback is the problem with collecting
accurate ratings. It is also difficult for some managers to accept feedback,
especially from their subordinates. Both of these factors can diminish the
impact and effectiveness of 360-degree feedback.
Many companies support the use of 360-degree feedback for develop-
ment and training but avoid its use for pay decisions. Many professionals feel
that using upward feedback for pay decisions can affect the acceptance and
impact of the feedback. The article ends with an anecdote from Allied Signal
Inc. that illustrates how fear of retribution can affect the accuracy of upward
feedback in 360-degree systems.

Kate Ludeman. To fill the feedback void. Training and Development, August
1995, pp. 38-41.
This article describes the necessity for and process of gathering feed-
back for senior-level executives who, Ludeman contends, do not receive
adequate feedback from peers and subordinates. Thus, they work in what she
calls a feedback void, which results in unexpected consequences resulting
from unknown performance deficiencies. The feedback void is created when
managers move up in organizations, become less effective, and no longer
recognize their impact on the organization.
Ludeman presents a three-part process for obtaining feedback, which
includes: (1) a 360-degree-assessment instrument, customized to measure
specific problem areas, (2) the Myers-Briggs Type Indicator
(MBTI
), and
(3) the 16 Personality Factor Questionnaire (a personality measure designed
to be administered by psychologists). This process leads to employee open-
ness, problem confrontation, and upward communication. Enriched perfor-
mance by top executives will have a positive effect on an organization that
relates directly to the bottom line. Honest and effective feedback can prevent
surprises and costly misunderstandings on the executive level.

Marc Marchese and John J. McGowan. The power of 360-degree feedback.
Pennsylvania CPA Journal, December 1995, pp. 19, 47.
The authors describe 360-degree feedback, how it works, and some
drawbacks to its increasing use. Traditional appraisals often set performance
goals and encourage employees to develop on their own. However, 360-
degree feedback offers a broader perspective of performance and assists
managers who, because of management downsizing, now supervise more
subordinates. It utilizes evaluations from supervisors, peers, subordinates, and
self, which are collected through anonymous questionnaires. Responses are
then interpreted by the facilitators of the assessment.
The authors state that 360-degree feedback can be very influential, and
the process offers these additional advantages: (1) It reinforces the belief in a
customer-driven approach to success, (2) it empowers the workforce by
making every opinion count, and (3) it provides a more accurate view of
performance.
The drawbacks of 360-degree assessment discussed here include issues
that affect the impact of the feedback, such as not all participants will accu-
rately fill out questionnaires in a timely manner. Some may consider the
assessment to be a distraction. Also, in order to elicit change, an employee
must be open to the feedback and be seeking development. More time is
required for 360-degree systems than for traditional appraisals, and the design
of the system can affect its developmental impact. Finally, using 360-degree
assessments for pay and promotion decisions often adds bias to the responses.
27
In addition, the quality of the questionnaire, the purpose of the system, and
organizational culture and support can affect the impact of the feedback.
Despite its drawbacks, 360-degree feedback is becoming an increas-
ingly popular method of assessment and is used by companies such as UPS,
AT&T, Amoco, General Mills, and Procter & Gamble.

Michelle Martinez. Multi-source better describes peer feedback. HR Maga-
zine 40:6, June 1995, p. 16.
The article begins with a quote from Karen Stoneman of Watson Wyatt
Companys organizational effectiveness division. She says that strictly
upward feedback is being replaced by 360-degree feedback, and that most
companies aspire to use the 360-degree-feedback process. Most companies,
however, are not truly using 360-degree feedback, but are using 180-degree
feedback instead.
David Gartenburg, a staff consultant at Rohm and Haas, a chemical
specialty company, is in the process of developing a guide for effective multi-
rater feedback programs. In his research, he concluded that performance
management was a top concern with employees, followed by concern for
rewards and recognition. The time consumed by the feedback process was
also a concern. In response, Gartenburg asserts that if people took the time to
talk about performance right after an event occurred, these programs wouldnt
be necessary.

Cynthia D. McCauley and Russell S. Moxley. Developmental 360: How
feedback can make managers more effective. Career Development
International 1:3, 1996, pp. 15-19.
This paper examines the impact of 360-degree feedback as part of a
larger development process. The authors summarize three conclusions about
self-awareness and development based on their research on management
development and on their work with managers: (1) People can learn, grow,
and change to become better leaders and managers, (2) self-awareness is the
cornerstone of development, and (3) development is an ongoing process
intricately related to work.
McCauley and Moxley assert that the role of 360-degree feedback is to
allow managers to compare their self-views with the views of others. It
encourages the manager to align self-views with others views and demon-
strates others recognition of a managers weaknesses. This enhancement of
self-awareness can lead to an unfreezing process, whereby a manager is
motivated to rethink his or her behavior and its impact on others.
Receiving feedback is only one step of the development process. Its
impact serves as one factor in unfreezing the self-view and subsequent efforts
to improve or change behavior. The development process involves:
(1) unfreezing the self-view by providing a manager with good data from
multiple perspectives and encouraging openness to the feedback, (2) maxi-
mizing subsequent efforts to improve by creating a development plan that is
acted on, and (3) ensuring organizational support for development.
The authors also discuss the issues involved with 360-degree feedback
(validity, rater confidentiality, design of the feedback process, goals and
development planning) as each affects the stages of development. Unlike
feedback assessments for decision-making purposes, 360-degree feedback
used for development is a part of an ongoing process. It should not replace the
seeking of informal feedback in the workplace, nor should it be a one-time
event. The impact of 360-degree feedback will be diminished if the feedback
is not handled in a way that is accepted by the manager.

Robert McGarvey and Scott Smith. When workers rate the boss. Training,
March 1993, pp. 31-34.
The authors discuss the growth in popularity of upward feedback and
discuss various aspects of its implementation in an organization. They state
that upward feedback is powerful because it provides a unique perspective for
managers; subordinates are often in the best position to evaluate a managers
performance. To truly experience the benefits of upward feedback, encourag-
ing informal feedback is not enough. The process must be instrumented and
facilitated to maximize benefits with the least amount of pain. Fear of
reprisal for negative ratings is a concern of many subordinates. Whether using
an off-the-shelf or custom-designed instrument, confidentiality is a necessity.
Even when confidentiality is ensured, McGarvey and Smith contend,
the fear of reprisal can remain a problem until organizations become more
experienced with using upward feedback. Additionally, the managers being
rated often have fears about subordinate feedback for the following reasons:
29
(1) Employees may not be sensitive to the pressures of managerial positions
and may have biases, and (2) employees may be influenced by recent
eventsa problem known as recencyand fail to consider the entire period
of time being evaluated. To alleviate fears from both raters and ratees, the
authors suggest training. H. John Bernardin, professor of management at
Florida Atlantic University, states that, with training, it is possible for em-
ployees to give an unbiased rating. Also with 360-degree feedback, individual
biases may be balanced out by other raters in the group.
There are many different options in the implementation of upward
feedback. The authors suggest that this feedback be used for developmental
purposes only to help managers recognize their weaknesses. They also
address instruments that compare feedback results to norms or to other
managers results, discussing the Center for Creative Leaderships
SkillScope
and Benchmarks
, as well as Personnel Decisions Profilor
instruments. These instruments use upward feedback as part of a multi-rater

assessment.
Help with interpreting the feedback can be a major factor in a
managers reaction to and acceptance of what can be surprising results. The
use of outside consultants is compared to the use of in-house human-
resources professionals to interpret data.
The authors also discuss how to choose a 360-degree-feedback instru-
ment. For example, there are four main considerations in choosing an off-the-
shelf product: (1) content, (2) length, (3) research, and (4) cost. They con-
clude with ideas on the frequency and timing of using 360-degree feedback.

Kathleen Melymuka. Im OK, youre OK. ComputerWorld, November 7,
1994, pp. 89-96.
Melymuka describes 360-degree feedback, why it is effective, and
discusses various issues involved with implementing 360-degree assessments.
One purpose of implementing a 360-degree system, she states, is to evaluate
self-managed teams and to support the use of team feedback. Depending on
corporate culture and commitment, 360-degree feedback can provide effec-
tive and honest feedback that affects participants. Two impediments to
success of a 360-degree system exist, one emotional and one administrative.
Emotionally, feedback that is honest can be unsettling. Administratively,
organizations must decide on the details of implementation.
There are certain issues that must be resolved before a 360-degree
system can be successfully implemented: (1) confidentiality of raters,
(2) purpose of the feedback (pay and job security or development only), and
(3) time and cost involved with the process. Melymuka also gives testimony
from various professionals and users of 360-degree systems on its overall
effectiveness.

John F. Milliman, Robert A. Zawacki, Carol Norman, Lynda Powell, and Jay
Kirksey. Companies evaluate employees from all perspectives. Person-
nel Journal, November 1994, pp. 99-103.
The authors provide a brief summary of the use of 360-degree assess-
ment and feedback for performance appraisal. Rater groups are divided into
two categories: internal and external customers. Internal customers are those
who work with the ratee in their organization. External customers can include
clients, suppliers, consultants, and community officials. The authors conclude
that, in general, 360-degree appraisals provide a broader view of perfor-
mance, and they give specific reasons that this is beneficial.
The authors also mention issues that an organization must resolve in
order to use 360-degree appraisal effectively. These include the number of
raters and their relationship with the ratee, criteria by which employees
should be judged, and the number of items on the appraisal form. Other
considerations are confidentiality of raters and summarizing of data.
There are also disadvantages inherent in the use of 360-degree feed-
back. These disadvantages are identified and solutions and implications are
offered. Even when used for appraisal, the authors conclude that 360-degree
assessments provide only a means to an end, and fail to provide a final
judgment. It is necessary for organizations to develop a plan and change
process to implement 360-degree appraisals. Employees should also receive
training related to the evaluation process and instruments. The discussion is
supported with examples of appraisal processes used at Johnson & Johnson,
Advanced Behavioral Technology, Digital Equipment Corporation, and
Hamilton Standard Commercial Aircraft.

31
Allan M. Mohrman, Jr., Susan M. Resnick-West, and Edward E. Lawler, III.
Designing Performance Appraisal Systems. San Francisco: Jossey-Bass,
1990, pp. 90-106, 160-184.
The two chapters in this book that address multi-rater appraisals discuss
how to determine who appraises performance (Chapter 4) and legal consider-
ations of appraisals (Chapters 7 and 8).
In Chapter 4, the authors state that choosing the right people to appraise
an employees performance will help the accuracy of an appraisal. In choos-
ing appraisers, one factor to consider is the context of the appraisal. There are
both organizational-design issues (functional design, team design, or matrix
design) and management-style issues (controlling or empowering). The
authors also consider each possible source of appraisal (supervisor, peer,
subordinate, customer, and self) and describe the particular perspective that
each provides. Each appraiser tends to focus on particular performance
aspects, which is useful for specific appraisal purposes, and the authors
discuss the effectiveness of each possible appraiser in various organizational
settings. This discussion evaluates the conditions and purposes for which
certain appraisers are more relevant than others.
Chapter 7 deals primarily with legal considerations in the use of ap-
praisal systems for pay, promotion, and rewards. There are two main consid-
erations in designing and implementing performance appraisals: (1) what is
being measured, and (2) the process by which measurements are made. To
eliminate discrimination, appraisals must measure appropriate and work-
oriented performance. The process must be monitored and audited for any
content that is not related to performance. Data collected by the process
should be valid, job-oriented, and objective. Choosing appropriate appraisers
and training will help to obtain valid data. Also, the appraisal system should
be an open process that allows employees to react to the data and to appeal, if
necessary.
Chapter 8 discusses application considerations for the use of perfor-
mance feedback for pay and promotion. The decision to tie pay and promo-
tion to performance appraisals is complicated. The potential negative effects
are as great as the potential benefits (that is, fairness and organizational
impact), depending on the development and implementation of the process.
Essentially, the authors assert that the use of feedback for pay decisions can
potentially affect the data collected by the process. The overall validity of the
data is key to system effectiveness, and it affects the organizational and
developmental impact of the feedback.

Joel Moses, George P. Hollenbeck, and Melvin Sorcher. Other peoples
expectations. Human Resource Management 32:2&3, Summer/Fall
1993, pp. 283-297.
This article discusses possible limitations of 360-degree-feedback
instruments. The authors assert that these limitations are the result of:
(1) reliance on generalized traits, (2) limited or nonexistent frame of reference
for ratings, (3) a memory-based, often incomplete description of past perfor-
mance, (4) the inability of observers to interpret behavior, and (5) the reliance
on the instruments designer for interpretation of data.
The key to the limitations of these instruments is their lack of situation-
based context. Most instruments rely on Other Peoples Observations (OPOs)
as the basis for the assessment. Using observation-based judgments leaves the
rater free to establish the context for the rating. Observers, to preserve accu-
racy, need a situational context on which to base their judgments. The au-
thors, therefore, differentiate the measurement of OPOs from the measure-
ment of Other Peoples Expectations (OPEs). Using OPEs as the basis for
judgment leads to the measurement of how a person will behave in a specific
situation. The authors assert that OPEs, because they are situation specific,
will provide the raters and ratee with a more accurate frame of reference.
They discuss the development of their instrument (Insight-Out), which was
designed to measure OPEs.
Many people are not aware of the expectations others hold for them.
Awareness of OPEs allows for more constructive suggestions for change. The
authors believe that OPE feedback information allows individuals to manage
the expectations of others. The quality and usefulness of 360-degree feedback
will improve with increased attention to situation specificity.

George P. Nicholas. Upward trend continues in appraisal process. HR Focus,
September 1992, p. 17.
Nicholas discusses two reasons for the increasing popularity of upward
feedback. First, more managers are seeking upward appraisal as a way of
obtaining candid, usable information. Even if the information is not used for
pay or promotion decisions, it can still motivate managers to change. Also,
subordinate comments may reveal managerial talents and strengths that might
otherwise be unrecognized. Upward appraisal is also gaining attention be-
cause it supports the shifting emphasis on the managers role as a coach.
33
Janina Latack, associate professor of management at The Ohio State Univer-
sity, asserts that a coach must have an accurate assessment of how he or she is
perceived by the people who are being coached.
Nicholas discusses a coaching/mentoring process called Helping Others
Succeed, developed by Blessing-White, a New Jersey-based human-resources
consulting firm. In this process, managers first complete a self-assessment,
then they participate in a workshop where ways to improve their coaching
skills are discussed, using the feedback. This initial discussion is followed by
subsequent progress evaluations, which feature face-to-face subordinate
feedback addressing specific coaching skills.
Latack suggests that face-to-face feedback be incorporated under two
conditions. First, the employees must trust the organization in general. Sec-
ond, the employees must feel that the managers they are appraising are
credible and interested in their development.

Dianne Nilsen and David P. Campbell. Self-observer rating discrepancies:
Once an overrater, always an overrater? Human Resource Management
32:2&3, Summer/Fall 1993, pp. 265-281.
This article discusses the nature of self-observer rating differences as
they occur in 360-degree feedback. Research has generally focused on situ-
ational factors of self-assessment accuracy. Although situations do affect
ones motivation to accurately report self-perceptions, the authors focus on
whether some individuals self-ratings are more accurate than others regard-
less of the context.
The authors conducted a test-retest reliability study to answer two
questions: (1) Is self-observer rating discrepancy a stable individual differ-
ence or merely a fleeting phenomenon? (2) Is the tendency to overrate,
underrate, or accurately rate ones self stable over time and does it generalize
across dimensions?
The first part of the study determined the stability of self-observer
rating discrepancies. The Campbell Leadership Index
, a multi-rater assess-
ment instrument, was completed by thirty-one female managers. One month
later, the managers completed the instrument again, as did the same raters
from the initial assessment. The results were that the discrepancy between
self- and observer ratings appeared to be stable.
The second part of the study looked at whether self-observer discrepan-
cies generalize across dimensions. High-level executives and low- to mid-
level managers were rated by the Executive Skills Profile and the Manage-
ment Skills Profile, respectively. In general, this study showed that individu-
als who over-, under-, or accurately rate themselves in one area are likely to
do so in another area. Also, the study suggested that such discrepancies in
personality assessments are related to discrepancies in skill assessments.
The authors concluded that: (1) The stability of self-observer rating
discrepancies suggests that they can provide useful information. (2) Accuracy
in self-perception is a stable individual difference. (3) Knowledgeable observ-
ers ratings are more valid and accurate than self-ratings. (4) Accurate self-
raters are better able to improve performance than inaccurate self-raters.
(5) Personality assessments might be more valid using observer ratings rather
than self-ratings.

Kenneth M. Nowack. Self-assessment and rater assessment as a dimension of
management development. Human Resource Development Quarterly,
Summer 1992, pp. 141-155.
In this study, the congruence between self-assessments and the assess-
ments of others (supervisors, peers, direct reports) was investigated. Ratings
for 335 managers were used to compare self-ratings to others ratings on
several management dimensions. The results indicated that the amount of
agreement between self and others was relatively low, with self-ratings being
significantly higher than the ratings of others. According to Nowack, many
managers tend to be more aware of their strengths and less cognizant of their
developmental needs. This inflated self-view may actually interfere with their
development as managers because they may not recognize the need for
improvement.
Nowack argues that managerial behavior change is a function of:
(1) awareness of individual strengths and developmental needs, (2) the
motivation to change, and (3) the capability to change specific behaviors. One
of the strengths of 360-degree feedback is that it exposes differences between
the managers self-perceptions and the perceptions of others, which may
motivate them to pursue developmental activities. When differences exist
between self- and others ratings, the differences should be explored, possibly
leading to positive behavior and performance improvement.

35
Kenneth M. Nowack. 360-degree feedback: The whole story. Training and
Development, January 1993, pp. 69-72.
Nowack discusses key issues related to the selection and development
of a 360-degree-feedback instrument. He gives five reasons for the increased
use of 360-degree systems for feedback: (1) the need for a cost-effective
alternative to assessment centers, (2) increasing availability of assessment
software, (3) the need for continuous measurement in continuous-
improvement efforts, (4) the need for feedback for career-plateaued employ-
ees, and (5) the need to maximize employees potential.
Before deciding to purchase a feedback instrument or to develop one,
an organization must first decide what is to be measured. Off-the-shelf
instruments are designed on five basic models: (1) job analysis: measures
knowledge, skills, and abilities based on a traditional analysis of the position
of interest (for example, managerial); (2) competency-based: measures
behaviors as compared to high performers and low performers; (3) strategic
planning: measures knowledge, skills, and abilities based on an
organizations strategic plans; (4) developmental theory: measures knowl-
edge, skills, and abilities based on theoretical and conceptual models of
employee growth and development; and (5) personality theory: measures
knowledge, skills, and abilities associated with personality factors.
An organization must also decide on issues such as scoring, confidenti-
ality and anonymity, ownership of results, and instrument validity. Nowack
discusses each of these issues and provides examples of various applications.

Brian OReilly. 360-degree feedback can change your life. Fortune, October
17, 1994, pp. 93-100.
This article provides an overview of current practices and thinking
related to 360-degree feedback, as well as examples of its impact. The author
first asserts that 360-degree feedback, as it is now applied, should be used
only for developmental purposes and is not effective as an appraisal tool.
When used for performance appraisal, rating biases can influence the out-
come. The author addresses the fact that, even when feedback is not used for
appraisal, it can still be surprising and sometimes difficult for managers to
accept. He concludes that expected feedback creates the most potential for
learning.
OReilly gives various examples of managers experiences that support
this conclusion. This powerful feedback is also being used for cultural change
associated with the shift to teamwork and employee empowerment. Often, to
elicit change, managers need massive doses of feedback, such as those
offered in week-long programs provided by the Center for Creative
Leadership.
The author also notes the different ways people rate themselves and
describes three types of raters: high self-raters, low self-raters, and accurate
self-raters. Finally, he summarizes conditions that are necessary for feedback
to be effective. First, the participant has to want to change. Next, recipients of
feedback should discuss results with everyone who provided ratings. Finally,
the focus of development should be on only a few shortcomings, and concrete
remedies should be chosen. The author closes with two examples of feedback
programs designed by executives to promote their companys vision to
their employees.

Shirley Parker-Gore. Perception is reality: Using 360-degree appraisal against
behavioral competencies to effect organizational change and improve
management performance. Career Development International 1:3,
1996, pp. 24-27.
Parker-Gore is the head of employee development at Mercury One 2
One Corporation, and presents a case study of their process for identifying
managerial skills, attitudes, and behaviors that are necessary for effective
performance. The first step involved focus groups of managers to determine
what skills were necessary in various roles and how behaviors were critical to
the company goals. This led to defining a behavioral model for managers
called Personal Success Factors. The next step was to implement a 360-
degree appraisal designed to measure these factors. Essentially, the organiza-
tion assessed managerial competencies and used development plans to ad-
dress the shortfalls in relation to business goals and targets. In exploring the
results of this process, managers have become aware of competence models
and how specific behaviors and skills relate to business success, as well as the
positive use of 360-degree feedback.
The author asserts that competencies can be used for both organiza-
tional development and individual performance management. One 2 One uses
data collected from 360-degree assessments to review performance and to
facilitate development. Ratings are sought from appropriate sources, and the
37
feedback is compared to a success profile (competence model), which is
thought to be critical for success and development in a changing business
environment. Finally, for an effective tool to reach its potential, it must be a
process owned by the participants. This will facilitate the shift toward an
interactive and interdependent culture that maximizes team performance and
produces a flexible, learning organization.

Helen Peters. Peer coaching for executives. Training and Development 50:3,
March 1996, pp. 39-41.
In this article, Peters describes a program utilized at National Semicon-
ductor, a microchip manufacturing company, to enhance management and
leadership skills. The program was designed as a two-phase plan to facilitate
corporate transformation. Phase one consisted of making the company viable,
and employed downsizing, manufacturing consolidation, and product-pruning
strategies. Phase two began with executives and managers establishing a
leadership model, against which their own competencies would be assessed.
The assessment of these leadership and management skills involved
extensive use of multi-rater feedback. The feedback was used to help develop
employees being paired into peer-coaching partnerships. These partnerships
used the multi-rater feedback as a basis for identifying development goals.
The author then describes a five-step process for using multi-rater
feedback with peer-coaching partnerships: (1) The Contract: Individuals
commit to the partnership and coaching roles are assigned. Individuals are
responsible for performance and must provide their own motivation to
change. (2) The Discussion: Partners analyze the multi-rater feedback they
have received and extract key themes. (3) The Plan: Partners distill the
themes from the feedback and determine specific behavioral and performance
goals. (4) Ongoing Sessions: Partners agree on times, generally once a month,
to meet formally. The purpose of these meetings is to review progress and
create necessary action plans. (5) The Follow-up/Reassessment: Partners
formally review all of the progress made toward established goals, six to eight
months after initial goals are set.

Catherine Romano. Fear of feedback. Management Review, December 1993,
pp. 38-41.
Romano discusses some of the following issues regarding 360-degree
feedback and then provides four case studies. She asserts that as the tradi-
tional, top-down performance appraisal technique is becoming less effective,
the use of upward feedback is gaining recognition. The nature of supervisor-
subordinate relationships is changing as organizations flatten and emphasize
teamwork. Training and development is the most common use of 360-degree
feedback; however, as the value of such feedback is recognized and the
process becomes familiar, managers increasingly want to link 360-degree
feedback to appraisal for pay and promotions.
Also, many managers are threatened by upward feedback if it is used
for decision making, thus the information can be ineffective. Brian Davis, of
Personnel Decisions, Inc., suggests using 360-degree feedback for develop-
ment until the process becomes familiar. He also recommends an Olympic
rating system, where the highest and lowest scores are disregarded.
Romano concludes that the use of 360-degree programs will help
managers eliminate the fear of feedback. She includes four case studies of
360-degree systems used at AT&T, MassMutual Insurance, Amoco Corpora-
tion, and Deloitte & Touche.

Warren Shaver, Jr. How to build and use a 360-degree feedback system. Info-
Line. American Society for Training and Development, August 1995,
16 pages.
The author discusses various pitfalls and benefits of using a multi-rater
system. First he talks about the four ways to implement multi-rater feedback
systems: Buy an off-the-shelf instrument, hire outside consultants, build an
internal system, or use a combination of any of these. Regardless of the type
of feedback system used, it must be reliable and consistent, valid, easy to use,
and should create positive change.
Second, Shaver says that because 360-degree feedback is a relatively
new and subtle technique (that can be potentially harmful), it is important to
understand the process of setting up a system for using it. He provides these
five steps: (1) Design and plan the process. Key factors are deciding who will
rate and be rated, ensuring fairness, utilizing timing, and ensuring confidenti-
ality. (2) Design and develop the instrument. Focusing the instrument on the
39
organizations vision can ensure that it is relevant to the future of the organi-
zation. (3) Administer the instrument. The format of the feedback system (for
example, questionnaire design) affects the overall success of the assessment.
(4) Process and report the feedback. Shaver cautions against problems of
inaccurate transcription, editing, and slow processing. (5) Plan responses to
the feedback. Receiving the data is only the first step in a managers develop-
ment process. Creating and adhering to an action plan is where the improve-
ment actually occurs. In this regard, there are three sources of help for manag-
ers using feedback for development: one-on-one facilitators, group discus-
sions or workshops, and workbooks.
The use of 360-degree feedback invariably raises some concerns.
Shaver asserts that relatively little is known about the impact of 360-degree
feedback. Also, many professionals are inexperienced with 360-degree
feedback; therefore, training is essential to implementing a system. Finally,
the author discusses the use of external customers as sources of feedback and
the use of feedback for performance appraisal and pay.

James W. Smither, Manuel London, Nicholas L. Vasilopoulos, Richard R.
Reilly, Roger E. Millsap, and Nat Salvemini. An examination of the
effects of an upward feedback program over time. Personnel Psychol-
ogy 48, 1995, pp. 1-34.
This article describes a study of the effects of upward feedback on
changes in managerial performance over time. The effects of (1) receiving
feedback, (2) completing self-ratings, (3) the managers initial level of
performance, and (4) the subordinates familiarity with the manager were
investigated by using goal-setting, control, and social-cognition theory.
There are two main hypotheses: (1) Managers whose initial level of
performance is low will be motivated to improve their performance, and
(2) managers whose initial performance level is low will improve when their
self-evaluations are higher than subordinate ratings. However, managers
whose initial self and subordinate ratings are low will not be motivated to
change (that is, have low aspiration level). The study also examined factors
affecting the observation of managerial behavior.
In the study, an upward-feedback program was administered to 238
managers. The program was then used six months later, and the results were
compared to measure performance change. Respondents rated performance in
areas such as commitment to quality, communications, support of subordi-
nates, and fairness. The results were that managers whose initial performance
was low or moderate improved after the feedback. However, managers with
high initial performance did not improve. Finally, opportunities to observe
behavior apparently increased after the initial assessment. Further implica-
tions and questions raised by the study are discussed.

Sandra Sweet (Ed.). Human Resource Management: Compensation Reporter.
Riverwoods, IL: CCH Incorporated, September 1995, pp. 3741.1-
3741.29.
This report discusses the uses and implementation of 360-degree
feedback and examines three sample instruments. Sweet asserts that 360-
degree assessment is an important method in the search for reliable, fair, and
valid evaluation. According to the 1994 Performance Management Survey
performed by Hewitt Associates, only 6% of 437 companies surveyed used
formal 360-degree programs. It is primarily used to assess developmental
needs, but it is being used more frequently for performance appraisal to
determine promotions and pay.
Multi-rater feedback is also being used for team-building purposes
how a company uses the feedback will depend on its objectives. The report
gives various experts opinions on implementation. They primarily advise
that 360-degree feedback be used for individual development, rather than for
performance decisions. Additionally, 360-degree feedback offers the organi-
zation an opportunity to reinforce business goals and values.
There are generally ten steps to arranging and implementing 360-degree
feedback: (1) Set objectives. (2) Decide if participation is mandatory or
voluntary. (3) Decide who will be assessed. (4) Select the competencies,
behaviors, and values to be assessed. (5) Write the questions. (6) Test the
questions. (7) Decide the response format. (8) Determine the method for
administration. (9) Set a schedule. (10) Arrange training or developmental
activities. Training is important for preparing both raters and those who
will be assessed. Confidentiality should be emphasized and ensured for
participants. If the data are not private, then the ratings can be biased and
ineffective.
The report gives examples of two types of 360-degree-feedback instru-
ments, Benchmarks
and SkillScope
, and TEAMS, Inc., showing how their

impact is the creation of workable action or developmental plans. Various
approaches are presented for the interpretation of the feedback and the
41
design of the development plan, as well as pitfalls to avoid. The report sug-
gests using various safeguards that have been implemented at DuPont. The
final section discusses a case study of the 360-degree-feedback program used
at AT&T.

TEAMS, Inc. Multi-source, 360 Feedback Survey Results. Tempe, AZ:
Author, October 1995, 11 pages.
A survey about multisource feedback was distributed to 6,600 human
resources professionals. These professionals were attendees at various na-
tional conferences, including those of the American Society for Training and
Development, the Society of Human Resource Managers, and the American
Compensation Association. Surveys were also sent to people on the TEAMS,
Inc., mailing list. The survey was designed to determine who was using 360-
degree feedback and how it was being used, and to determine the keys to
successful implementation. Information was also gathered regarding various
organizations 360-degree project guidelines and practices. Survey results are
summarized from the 97 surveys that were returned.
The survey first determined how many organizations were using
360-degree feedback for developmental purposes and for performance-
management purposes. Assessment-for-development-only was the most
common form of 360-degree feedback (24%). Only 8.7% of companies used
360-degree feedback for performance management.
The survey then asked for the following information about multisource
feedback: (1) the reasons for moving to multisource assessment, (2) how
multisource information is used, (3) targeted groups for multisource assess-
ment, (4) group size, (5) who initiated the multisource assessment process,
(6) use of standardized versus customized instruments, (7) how raters are
chosen, (8) features of multisource systems, (9) how often assessment pro-
cesses are used, (10) number of feedback recipients, (11) number of different
instruments used, (12) time needed to complete each survey, (13) number of
points on the rating scale, (14) number of items on the instrument, (15) who
initiates the process, (16) data capture medium (paper, disk, phone, etc.),
(17) timing of the feedback, (18) feedback given at a focal point (all at one
time) or distributed over time, (19) sharing results with supervisors (encour-
aged or discouraged), (20) measures taken for process effect or impact, and
(21) organizational demographics.

Walter W. Tornow. Perceptions or reality: Is multi-perspective measurement
a means or an end? Human Resource Management 32:2&3, Summer/
Fall 1993, pp. 221-229.
The author examines whether multi-rater assessment is an end in itself
(the scientific perspective) or a means to an end (the practical perspective).
The goal of the scientist, who sees measurement as an end in itself, is to
enhance the accuracy of measurement by reducing error and rater variation.
The practitioner, on the other hand, values the differing input of raters be-
cause this input represents significant and meaningful sources of variation
from which much can be learned. Tornow equates the practitioner in this
scenario to a manager in an organization. The manager uses multi-rater
feedback as an insightful and valid means to an end.
The author also discusses research on the agreement between self and
other ratings. There are three core assumptions that relate to the discrep-
ancy between self and other ratings, which are found in many management
and leadership development programs: (1) Awareness of how ones skills and
behaviors are seen by others is a key to managerial effectiveness, (2) manag-
ers need to improve in the area of self-objectivity, and (3) self-other agree-
ment can be enhanced with feedback (self-other differences motivate manag-
ers to alter their self-views and/or improve their performance).
Tornow believes that when multi-rater programs use rater discrepancies
as feedback, they become powerful tools for development. Multiple-rater
assessments, in order to be effective development tools, must focus on the
multiple perceptions of the raters, rather than on measurement as an end in
itself.

Ellen Van Velsor and Jean B. Leslie. Feedback to Managers, Volume II: A
Review and Comparison of Sixteen Multi-rater Feedback Instruments.
Greensboro, NC: Center for Creative Leadership, 1991, 276 pages.
This volume is a comparison of sixteen widely used multi-rater assess-
ment instruments. It focuses on those instruments that relate self-view to the
views of others on multiple management or leadership domains. First,
information was collected by contacting the authors and publishers of the
twenty-four instruments reviewed in the original Feedback to Managers
(1978). Then, an on-line computer literature search was conducted using
43
behavioral science and business indexes. Finally, remaining reference manu-
als were examined and other authors and vendors were contacted.
The information received for each of the sixteen instruments was
divided into three categories: descriptive information, research information,
and training information. Descriptive information includes the author(s),
vendor, copyright date(s), statement of purpose, target audience, cost, scoring
process, certification process, instrument length, item-response format, and
type(s) of raters. Research information includes the item origins, feedback
scales, scale definitions and sample items, and written reports or data from
studies of the instruments reliability and validity. The training information
for each instrument includes a copy of the instrument, a sample feedback
report, and any support materials (instructional or developmental) provided
for trainers or participants.
A comprehensive summary of each instrument is added at the end of the
report. Although each of the instruments met acceptable standards of instru-
ment development, they are only a sample of what is available. Their inclu-
sion in this report does not indicate that they would necessarily be suitable for
the needs of all organizations. The sixteen instruments included are:
ACUMEN
Group Feedback, Benchmarks
, Campbell Leadership Index
,
COMPASS: The Managerial Practices Survey, Executive Success Profile,
Leader Behavior Analysis II
, The Visionary Leader: Leader Behavior

Questionnaire, Leadership Practices Inventory, Life Styles Inventory
,
Management Effectiveness Profile System, Management Skills Profile,
Multifactor Leadership Questionnaire, PRAXIS
, Survey of Leadership
Practices, The Survey of Management Practices, and SYMLOG
.
Note: An updated edition of this volume with all new instruments
will be published by the Center for Creative Leadership in the first quarter
of 1998.

Ellen Van Velsor, Jean B. Leslie, and John W. Fleenor. Choosing 360: A
Guide to Evaluating Multi-rater Feedback Instruments for Management
Development. Greensboro, NC: Center for Creative Leadership, 1997,
56 pages.
In this volume, the authors address the expanded use of multi-rater
assessment instruments and affirm the need for an up-to-date reference
manual describing an evaluation process for these instruments. This update to
Feedback to Managers, Volume I: A Guide to Evaluating Multi-rater Feed-
back Instruments (Center for Creative Leadership, 1991) is designed to assist
human resources professionals in the assessment of 360-degree-feedback
instruments.
This report presents a simplified and comprehensive model for analyz-
ing multi-rater feedback instruments. A step-by-step process is presented by
which any multi-rater feedback instrument can be evaluated. Evaluation
focuses on the use of these instruments for developmental purposes, and does
not address the use or design of products for performance appraisal.
The volume divides the evaluation into sixteen sequential steps:
(1) Find out what is available. (2) Collect a complete set of materials.
(3) Compare your intended use to instrument characteristics. (4) Examine
feedback scales. (5) Familiarize yourself with the instrument-development
process. (6) Learn how items and feedback scales were developed. (7) Find
out how consistent scores tend to be. (8) Assess basic aspects of validity:
Does the instrument measure what it claims to measure? (9) Think about face
validity. (10) Examine the response scale. (11) Evaluate the feedback display.
(12) Understand how breakout of rater responses is handled. (13) Learn what
strategies are used to facilitate interpretation of scores. (14) Look for develop-
ment and support materials. (15) Compare cost and value for the price.
(16) Consider length a minor issue.
Applications for each step are discussed. Although the steps are not
equal in complexity and importance, the authors suggest that critical decisions
should be made early in the process to eliminate instruments that are either
not valid or do not meet the organizations needs. And, although no instru-
ment is perfect in structure, content, and applicability, careful evaluation will
allow users to make trade-offs in order to select the most appropriate instru-
ment for their needs.

Mary N. Vinson. The pros and cons of 360-degree feedback: Making it work.
Training and Development, April 1996, pp. 11-12.
Vinson begins by discussing a 1993 survey that indicated that most
employees were dissatisfied with their organizations performance appraisal
system. Other studies, however, show that employees value knowing where
they stand and what they must do to improve. Vinson offers 360-degree, or
multi-rater, feedback as an alternative to traditional feedback. She begins by
discussing the benefits of using this type of feedback. She asserts that 360-
degree feedback can be effective when it is accompanied by a follow-up
45
session. Many companies have used 360-degree feedback as part of total
quality management efforts. As such, it may be useful for promoting team
cohesiveness. It also tends to reduce rating bias because the responsibility
for feedback involves more people than in traditional, single-source, down-
ward appraisal.
The author also discusses the downsides of 360-degree feedback. First,
it can be painful. Many feedback experts believe that the respondent provid-
ing the harshest criticism is often considered untrustworthy by the recipient.
Second, there are often conflicting opinions, and there is no way to determine
whose feedback is right. Also, raters often suffer from survey fatigue,
which results from having to participate in too many feedback assessments.
These concerns raise the question of the reliability and accuracy of 360-
degree feedback. Feedback that is not accurate will not be of any use to its
recipient.
The final question asked about 360-degree-feedback systems is whether
or not they improve performance. The author asserts that performance im-
provement can be less than satisfactory with any type of feedback. Often
intentions to change are high but employees lose their commitment. Vinson
offers several suggestions to facilitate the use of 360-degree feedback to
produce change: (1) The feedback must be confidential and anonymous.
(2) One must consider the length of time an employee has been in the posi-
tion. (3) A feedback expert should interpret the feedback. (4) Follow-up and
the development of specific action plans are essential. (5) Feedback should
not be used to determine salary or promotions. (6) Feedback should include
written descriptions as well as numerical ratings. (7) One must ensure that
instruments are statistically valid and reliable. (8) Feedback should not be
utilized with too many employees at one time.

Stephen J. Wall. Feedback on feedback. Training and Development 49:1,
January 1995, pp. 2-4.
Wall, of Manus Associates in New York City, discusses two conclu-
sions that he and George Yukl of the State University of New York in Albany
have arrived at from working together extensively on the development and
implementation of 360-degree-feedback systems.
The author first discusses the work of David Bracken (1994; see anno-
tation on page 5) about the design and use of 360-degree-feedback assess-
ments. Bracken offers prescriptions for making sure that 360-degree feedback
is experienced as a powerful tool for individual and organizational develop-
ment. Wall agrees with Bracken that 360-degree feedback is essential to the
development process for employees. His experience, however, has led him to
make the following observations.
The first involves the use of response scales. Wall suggests that re-
sponse scales based on frequency are preferable to response scales based on
degrees of satisfaction or agreement. He asserts that frequency-based re-
sponses measure observable and specific behaviors and minimize subjectiv-
ity. Second, Wall compares two response-scale formats (how many possible
choices are available to a rater using the scale). He believes that a six-point
response scale adds no appreciable value over a four-point scale. In addition,
a six-point scale may tend to confuse and burden respondents. One recom-
mendation is to use a four-point scale (that is, give the respondent four
choices) but supplement it with the constant option of Dont Know or Not
Applicable. Wall states that 360-degree feedback is an excellent way to
support various initiatives, but it is only powerful to the extent that it is
reliable and accurate. To guarantee accuracy, questions must be clear, under-
standable, and based on observable interactions.

Peter Ward. A 360-degree turn for the better. People Management 1:3,
February 9, 1995, pp. 20-22.
This article presents various methods and considerations involved with
360-degree feedback that will help ensure maximum benefits from the pro-
cess. The author asserts that 360-degree feedback, by collecting a broad
picture of an individuals performance or skills, provides valuable informa-
tion needed for the changing roles of managers. He presents three fundamen-
tal questions about feedback: (1) Why have feedback at all? (2) Who can be
assessed by 360-degree feedback? (3) Should feedback assessments try to
describe behavior or judge behavior? Ward addresses various issues related to
these questions, along with their implications.
He then discusses various issues that concern implementing 360-degree
feedback. Descriptions are provided regarding the perspectives of various
rater groups: bosses, direct reports, peers/co-workers, customers, and self.
Wards concern about implementation is that, in order to achieve the maxi-
mum developmental impact, 360-degree methods must be properly applied.
There are five pitfalls that should be avoided: (1) 360-degree assess-
ments should not focus only on the persons weaknesses or needs. (2) Feed-
47
back should be collected with an emphasis on proper implementation and on
follow-up action. (3) Confidentiality is necessary for honest ratings. (4) The
purpose of the feedback should be communicated to all employees involved.
(5) The feedback and its use for developmental action-planning should be
supported within the organization. The final step, after collecting and using
360-degree feedback, should be an evaluation of the organizations training
and development needs.

Francis J. Yammarino and Leanne E. Atwater. Self-perception accuracy:
Implications for human resource management. Human Resource Man-
agement 32:2&3, Summer 1993, pp. 231-245.
The authors argue that, according to previous research, self-ratings are
generally problematic. Self-ratings tend to be inaccurate when compared to
more objective ratersin this case, subordinates, peers, supervisors, and
customers or clients. Although research is fairly limited on this subject, the
authors use available studies to develop a model for self-perception accuracy.
They then discuss the implications of self-rating accuracy for human re-
sources professionals.
Self-perception is measured (via self-ratings) on a particular dimension
of interest. Others perceptions are measured on the same dimension of
interest (leadership abilities, for example). Self-other agreement is determined
by comparing the two perceptions. The degree to which they agree or dis-
agree determines self-rating accuracy. The model shows that this comparison
process results in three categories of individuals: accurate estimators (high
degree of self-other agreement), over-estimators (inflated self-ratings com-
pared to others), and under-estimators (deflated self-ratings compared to
others). The implications for human resources professionals are that over-
estimators proceed to have diminished organizational or personal outcomes
from the feedback, accurate estimators have enhanced organizational/
individual outcomes, and under-estimators have mixed outcomes.
The authors then discuss the elements of their model of self-perception
accuracy. First, they look at the determinants of self-ratings and other-ratings
and the process that determines the two stages of perception: initial percep-
tion and reviewed perception. Next, they discuss the process through which
the self-other agreement is measured by categorizing the degrees of agree-
ment. Finally, implications of self-other agreement are discussed. The authors
address the need for further research, stating that it is necessary to better
understand self-perception accuracy and its implications.

Gary Yukl and Richard Lepsinger. How to get the most out of 360-degree
feedback. Training 32:12, December 1995, pp. 45-50.
Yukl and Lepsinger give specific suggestions for implementing 360-
degree feedback and discuss factors that affect its impact on development.
They assert that 360-degree feedback can prompt measurable changes in
managerial behavior. This behavioral impact is the result of providing manag-
ers with honest and specific feedback about how their behavior affects others.
This impact can be diminished, however, if certain issues are not examined.
There are two categories of pitfalls: (1) design and administration of the
assessment, and (2) design and facilitation of the follow-up activities.
In designing a questionnaire or selecting one from an external source,
certain qualities are desirable. First, it should be well researched to assure that
the items and scales can be linked directly to managerial effectiveness.
Second, the items should measure specific and observable behaviors. Third,
the behaviors should be described in positive terms, and each item should be
worded so that it pertains directly to the ratee. Fourth and last, the question-
naire should specify the relationship between the rater and the ratee. Separate
questionnaires should be considered for each group: self, supervisors, peers,
subordinates, and customers.
There are a number of suggestions for administration of the assessment.
Respondents should be chosen who are important to the managers effective-
ness and are influenced by it. The number of respondents should be large
enough to form a comprehensive sample and ensure confidentiality of ratings.
Training should familiarize raters with the purpose of the feedback, ensure
confidentiality, and advise raters to avoid bias. For example, halo error
occurs when a well-liked manager receives high ratings on all scales, regard-
less of his or her actual level of performance.
The feedback report should present the data from each rater group
separately. Self-ratings should be compared to both others ratings and to
norms, or a managerial effectiveness model. Feedback should also be dis-
played by item as well as by scale. The overall purpose of the feedback is for
use to develop an action plan. One way to facilitate follow-up is to have a
feedback workshop involving a group of managers and a facilitator. In this
workshop, managers interpret data and develop specific goals and plans.
49
Follow-up on plans can include skills-training, support and coaching, and
assessments of impact and effectiveness of the assessment process itself.
The impact of 360-degree feedback requires further research, but experience
has demonstrated the potential benefits of using 360-degree feedback for
development.
51
Frequently Asked Questions about 360-degree Feedback
Users of 360-degree feedback, such as program participants, research-
ers, human resources professionals, and others, often ask about its back-
ground, the theories associated with it, and its practical uses. What follows
are some of their commonly asked questions with responses that are generally
based on the information contained in the annotations.
How and When Did 360-degree Feedback Develop?
The ideas behind 360-degree feedback are not new. Assessment centers
developed by the German military during World War II recognized the value
of gaining performance insights from multiple perspectives. In organizations,
early methods of 360-degree feedback were by surveys used to gather em-
ployee opinions. The fundamental premise was that information gathered
from multiple perspectives was thought to be more comprehensive and
objective than the information obtained from only one source. Over the
decades, these opinion surveys were used to gather information about various
aspects of the organization, including compensation, benefits, leadership, and
so forth. But the use of 360-degree instruments for individual assessment was
rare prior to the 1980s. Because of the traditional hierarchical structure of
most organizations during that time, performance appraisals typically were
top-down, single-source assessments.
But things have changed. Information regarding an employees perfor-
mance is now sought for more than just the annual performance evaluation
for some of the following reasons (Hirsch, 1994; Holt, Noe, & Cavanaugh,
1996; Jones & Bearley, 1996). Because of changes in the business environ-
ment, employees who once felt satisfied and secure in their jobs now find
themselves responsible for their own careers (Nowack, 1993; OReilly, 1994;
Romano, 1993). Also, empowerment has made leadership a necessary compe-
tency for all levels of employees (Crystal, 1994) and downsizing has made it
important for even skilled managers to be aware of their own strengths and
weaknesses to remain competitive (Hirsch, 1994; Hoffman, 1995). In addi-
tion, teams and partnering initiatives have made communication and manage-
rial effectiveness vital to individual and organizational success (Melymuka,
1994; Peters, 1996).
Partly because of these trends, a new concept was born: the utilization
of organizational resources for individual development (Jones & Bearley,
1996; Peters, 1996). People desiring to become leaders began to study and to
develop ways of becoming better employees. Individuals began to seek new
sources of information that would allow them to see both their strengths and
their shortcomingsstrengths and developmental needs that might not be
exposed in traditional performance evaluations (Budman & Rice, 1994;
Dunnette, 1993; Hirsch, 1994). Managers realized that the first step in devel-
opment is discovering what specifically needs to be developed. They under-
stood that in order to gain more useful knowledge of themselves, they needed
better tools to gather this information. The tools being used for these purposes
are more and more often 360-degree-feedback assessments (Hirsch, 1994;
Jones & Bearley, 1996; Lublin, 1994; McGarvey & Smith, 1993). With the
success of this technique for developmental purposes, the idea emerged that
360-degree feedback could also be useful for performance appraisal and
decision-making purposes (Edwards & Ewen, 1996).
What Are the Benefits of 360-degree Feedback?
It is becoming widely recognized that 360-degree feedback offers
several advantages over single-source assessment (Budman & Rice, 1994;
Dunnette, 1993; Hoffman, 1995; Jones & Bearley, 1996; Kaplan & Palus,
1994; Lublin, 1994; McGarvey & Smith, 1993; Nicholas, 1992; Nowack,
1993; Vinson, 1996). Its benefits can be grouped into four categories:
(1) 360-degree assessments offer new perspectives by which an
individuals skills, behaviors, abilities, or performance can be judged
(Denton, 1994; McCauley & Moxley, 1996; McGarvey & Smith, 1993;
Milliman, Zawacki, Norman, Powell, & Kirksey, 1994; Mohrman, Resnick-
West, & Lawler, 1990; Nicholas, 1992).
Often, particular aspects of an individuals performance or behavior are
not observable by or do not affect all employees, depending on their relation-
ships with the individual being evaluated. Because 360-degree assessments
obtain input from new and varied perspectives, the feedback creates a more
complete picture of an individuals performance and skills. Managers are able
to see how their behavior affects those around them and how others perceive
their ability.
(2) 360-degree assessments alleviate some recognized deficiencies of
top-down, single-source assessments (Bracken, 1994; Budman & Rice, 1994;
Harvey, 1994; Hirsch, 1994; Jones & Bearley, 1996; Marchese & McGowan,
1995; Nicholas, 1992; Vinson, 1996; Ward, 1995).
Single-source assessments are subject to the biases and subjectivity of a
single individual. Reliance on a single source (for example, the supervisor)
for retrospective evaluations, which often cover a year of performance infor-
mation, may not provide fair and valid feedback.
53
There are, however, some rating biases that may affect the validity of
all performance ratings, including those collected with 360-degree-feedback
instruments. One such error is known as halo error, where an individual is
rated as good or poor based on his or her reputation rather than on the actual
level of performance (Yukl & Lepsinger, 1995). Another source of error,
recency effects, results from raters giving too much weight to performance
that occurred a few months before the assessment. For example, an employee
with poor performance over most of the year is still rated highly because of a
project successfully completed a few weeks before the appraisal. Recency
effects often occur in organizations that do not emphasize or prioritize perfor-
mance or developmental assessments (McGarvey & Smith, 1993).
Although it is true that all raters may be affected by these biases, the
use of more than one perspective permits the ratings to be averaged across a
number of respondents, which may provide a truer evaluation of the focal
managers performance (Denton, 1994).
(3) 360-degree assessments provide the unique opportunity for indi-
viduals to rate themselves (Dunnette, 1993; Hazucha, Hezlett, & Schneider,
1993; London & Beatty, 1993; McCauley & Moxley, 1996; Nilson &
Campbell, 1993; Smither, London, Vasilopoulos, Reilly, Millsap, &
Salvemini, 1995; Tornow, 1993; Van Velsor & Leslie, 1991; Ward, 1995;
Yammarino & Atwater, 1993).
Self-evaluations have been used for purposes other than 360-degree
feedback; however, these evaluations traditionally focused on job satisfaction
or organizational effectiveness, rather than on individual performance. With
360-degree feedback, self-evaluation adds yet another perspective from which
performance and behaviors can be observed. The act of evaluating ones own
effectiveness can enhance the assessment process itself. For example, self-
ratings can more fully involve the focal individual in the rating process
(Crystal, 1994). This involvement can positively affect both the implementa-
tion and administration of the process, as well as the impact of the feedback.
Employees tend to place more trust in a process in which they themselves are
a factor (Edwards, 1995), which can lead to better acceptance of the feedback
(Budman & Rice, 1994; Vinson, 1996).
In addition to generating buy-in, self-ratings offer unique insight into
how self-perception affects managerial performance. In many 360-degree-
feedback instruments, self-ratings are compared to ratings from other groups.
Some assessment systems compare self-ratings to normative data (Jones &
Bearley, 1996). There have been a few studies, and more are emerging, on the
comparison of self- and others ratings. For the purposes of this review, those
studies are not presented in depth. There are, however, some practical impli-
cations that relate to the degree of agreement between self- and others
ratings. The primary finding is that discrepancies between self- and others
ratings can provide important insights about managerial and leadership
effectiveness (Nilsen & Campbell, 1993). Although there is little research on
the causes of these differences, many users of 360-degree assessments feel
that understanding the differences between ones self-view and the views of
others is an essential step in leadership development (Dalton, 1996;
McCauley & Moxley, 1996). Adding self-awareness to the assessment,
therefore, can lead to greater impact of the feedback. This type of impact can
only be obtained when the assessment provides feedback from all perspec-
tives relevant to a managers effectiveness, including self-ratings.
(4) 360-degree assessments can be used to reinforce organizational
values and vision (Crystal, 1994; Heisler, 1996; London & Smither, 1995;
OReilly, 1994; Parker-Gore, 1996; Shaver, 1995; Sweet, 1995).
According to Parker-Gore (1996), organizational cultures must be
reinforced and developed. In this capacity, 360-degree feedback can reiterate
the value of specific abilities, behaviors, or actions to overall organizational
values (Heisler, 1996). For example, a manager who must be innovative in
order to be successful can receive specific feedback on his or her ability to be
creative and open to new ideas. When 360-degree assessments are aligned
with organizational visions and values, individual feedback may be more
valid for use in that organization. Gebelein (1996) argues that the full impact
of 360-degree feedback will not be realized if it only supports individual
change and does not integrate that change into the organizational strategy.
How Should 360-degree Feedback Be Used?
Feedback received from 360-degree assessments can be used for almost
any initiative that requires extensive information about an employees
effectiveness (Crystal, 1994; Wall, 1995). These initiatives fall into two
general categories: feedback for development and feedback for performance
appraisal.
Ratings are gathered similarly for both uses of 360-degree feedback.
The key difference is who owns the data that are collected (Dalton, 1996).
When 360-degree feedback is used exclusively for development, the feedback
is available only to the focal individual (and sometimes to the facilitator who
provides the feedback). However, if the feedback is used for performance
appraisal, the data must be available to the person conducting the appraisal,
typically the employees supervisor. Deciding how the 360-degree feedback
55
is to be used raises some important issues that can affect the whole process
(Melymuka, 1994).
Feedback for development only. Many feel that 360-degree feedback
should not be used for appraisal purposes (for example, Crystal, 1994;
Dalton, 1996; Edwards, 1995). The main contention is that using 360-degree
feedback for performance appraisal affects how the raters evaluate the focal
individual (Antonioni, 1996; Marchese & McGowan, 1995). Research has
shown that raters are affected by the knowledge that their ratings can influ-
ence anothers career (Lublin, 1994). Additionally, knowing that ones boss
will see the assessment results can affect how an employee rates him- or
herself. A commonly reported effect of using feedback for appraisal is that
both self-ratings and the ratings of co-workers tend to be inflated when
collected for decision-making purposes (Antonioni, 1996).
Another issue involves the content of the items that are rated in devel-
opmental assessments versus performance appraisals. Assessments that are
used for development only tend to concentrate on skills or behaviors and
often involve predictive evaluations. Performance appraisals differ in that
they measure past performance and are specific to the employees particular
job. Because performance appraisals are retrospective, they can be more
easily affected by bias and may not provide the information an individual
needs to improve performance (Jones & Bearley, 1996).
Feedback for performance appraisal. Even when feedback is gath-
ered from multiple sources for performance appraisal, the final decision about
pay or promotion is typically made by a single person or management com-
mittee. It is the responsibility of the decision-makers to interpret the results of
a 360-degree assessment and to use the data to make appropriate decisions.
Although some feel that using 360-degree assessment in performance ap-
praisal makes the evaluation more fair for the employee (Bernardin, Dahmus,
& Redmon, 1993), these evaluations are subject to bias and, as with any
human process, there will be errors in judgment. A possible advantage of
using 360-degree feedback for appraisal purposes is that multiple sources of
feedback may cancel out those biases and provide a more well-rounded
picture of the employees past performance (McGarvey & Smith, 1993).
Some also feel that using 360-degree assessment for appraisal will promote
the acceptance of decisions because the information on which the decisions
are based is more complete.
Feedback for development and appraisal. Some organizations are
now attempting to incorporate 360-degree feedback into their appraisal and
developmental processes (Dunnette, 1993; London & Beatty, 1993). This
trend is evolving as companies attempt to maximize the potential of this
method. The literature suggests, however, that 360-degree assessments first
be used for only one purpose in order to acclimate employees to the system
(Jones & Bearley, 1996). These assessments can then slowly be administered
for other organizational purposes. This will ensure that the feedback will be
accepted and effective for both developmental and decision-making purposes
(Romano, 1993). It also may be necessary to develop separate assessments for
each purpose, because feedback gathered for development is not necessarily
applicable to pay or promotion decisions (Harvey, 1994; Yukl & Lepsinger,
1995). If 360-degree feedback is to be used in more than one way, it should
be collected and interpreted according to its intended use.
How Do You Integrate 360-degree Feedback into Organizational
Development Systems?
Whatever the use of the 360-degree feedback, there are three main
stages to the implementation process: preparation, administration, and follow-
up. There are many points to consider at each stage that can affect the success
of the process. Again, the literature provides various recommendations
relating to each issue.
Prepare the organization and employees. Preparation is the first step
in the process of implementing 360-degree feedback (Bracken, 1994; Coates,
1996; Shaver, 1995). This is necessary regardless of the purpose of the
feedback (Carey, 1995; Hirsch, 1994). For example, an organization may lack
effective leaders and wish to develop managerial skills and improve manage-
rial behavior. Or, some organizations may be dissatisfied with their current
appraisal process and decide they need a process that is more fair.
After determining what organizational needs can be met by using 360-
degree feedback, the logistics of implementing the process must be consid-
ered. One of the first decisions is what type of instrument to use. There are
three types of 360-degree instruments, each with positive and negative
aspectsthe literature suggests that instrument design can affect the results
of the assessment (Guinn, 1996; Marchese & McGowan, 1995). First, there
are off the shelf feedback instruments that measure generic skills or compe-
tencies. Second, there are instruments that are developed by an outside group
(such as a consulting firm) that focus on specific organizations and their
assessment needs (that is, custom-designed). Third, there are 360-degree-
assessment systems that are developed internally and relate to specific posi-
tions or teams within the organization (Melymuka, 1994).
57
In choosing which of these to use, cost becomes a primary factor
(Hirsch, 1994; Nowack, 1993; Van Velsor & Leslie, 1991). Off-the-shelf
instruments are generally the least expensive, followed by customized instru-
ments. Internally developed assessment systems generally involve more
indirect costs, such as the time and expertise required for development and
design. Reliability and validity are important factors to consider in choosing
which assessment instrument to use (Van Velsor, Leslie, & Fleenor, 1997).
These qualities usually vary according to the expertise and experience of the
instrument developer.
Preparing individual participants for a 360-degree process involves
educating them about the assessment and the feedback (Shaver, 1995). First,
employees should be informed of how their feedback will be used (Ward,
1995). Then, participants in the feedback assessment should be trained to
avoid common rating errors. Training can also reinforce the role of the
feedback in reinforcing company values and strategy (Gebelein, 1996).
Participants should also be informed as to whether they will be held account-
able for their ratings. This issue of rater confidentiality is key to preparing
participants, and the literature presents a variety of opinions on the subject
(Crystal, 1994; Dalton, 1996; Holt et al., 1996; McCauley & Moxley, 1996;
McGarvey & Smith, 1993; Milliman et al., 1994; Nowack, 1993; Shaver,
1995; Sweet, 1995; Yukl & Lepsinger, 1995).
Training generally should be extended not only to the raters but also to
the person being rated (Budman & Rice, 1994). If the feedback is being used
for development, the employee being rated should be prepared to receive and
to act on the feedback. If it is used for performance appraisal, the ratee should
be aware of how the assessment results will affect decisions such as promo-
tion and compensation. In both cases, feedback obtained from multiple
sources can often be surprising and difficult to deal with for the recipient
(Ludeman, 1995). This is especially true when an employee learns that self-
perceived strengths are seen as developmental needs by others. Preparing
managers to receive feedback from their subordinates is a complex issue
one that is unique to upward feedback.
Administer the assessment. After deciding what 360-degree instru-
ment to use, how to use its feedback, and preparing the participants for the
process, the assessment instrument can be administered (Bracken, 1994).
According to Shaver (1995), design and administration of the assessment can
affect the outcome of the process. Most 360-degree feedback instruments are
questionnaires that use scaled items to assess employees (Shaver, 1995; Yukl
& Lepsinger, 1995), but some include sections for open-ended responses. It is
necessary to consider the medium for the data collection in advance. An
important factor to consider is who will be evaluated (Hirsch, 1994; Sweet,
1995; Ward, 1995). Feedback for development has been gathered both for
fast-track, high-performing employees and for employees who lack the
necessary skills to be effective (Hirsch, 1994; Jones & Bearley, 1996).
However, 360-degree systems are now being used throughout organizations
for every level of employee. Once the employees to be assessed are decided
upon, it is necessary to determine who will provide the ratings (Mohrman et
al., 1990).
Typically, to be effective, the raters should be in positions within the
employees circle of influence and should be able to observe the employees
behavior and performance. Sources to consider include peers, subordinates,
direct reports, and supervisors, as well as internal and external customers.
Another decision to make is how many raters to include for each ratee
(Hirsch, 1994; Milliman et al., 1994). Issues that affect the number of raters
to use include confidentiality and rating bias. Also, in the feedback, the
ratings from each source should be presented individually, rather than com-
bined into a single others rating (London & Smither, 1995).
The next step in implementing the 360-degree assessment involves
providing feedback to the focal individual. Interpreting the feedback is
another aspect that is treated in various ways (London & Beatty, 1993;
McGarvey & Smith, 1993; Moses, Hollenbeck, & Sorcher, 1993; Sweet,
1995; Van Velsor & Leslie, 1991; Vinson, 1996; Yukl & Lepsinger, 1995).
Some instruments are accompanied by instructions and guidelines for inter-
preting the data. Other instruments use an outside consultant or facilitator to
interpret the feedback for the employee. Whatever process is used, the goal of
360-degree feedback is to provide usable and relevant information, either for
employee development or for appraisal by the organization. The feedback
should be presented in a way that exposes weaknesses or shortcomings in
performance or behavior, and should identify strengths as well. The compari-
son of self-ratings to the ratings of others can also be an important aspect of
360-degree feedback (Jones & Bearley, 1996; McCauley & Moxley, 1996;
Nowack, 1993; Yammarino & Atwater, 1993).
Conduct the follow-up. Once the feedback is interpreted, the question
arises: What next? The follow-up, or how the feedback is used by the ratee, is
an important part of the 360-degree process (Shaver, 1995; Sweet, 1995).
This is especially true if the feedback is to be used for developmental pur-
poses. In these cases, receiving feedback from multiple sources is only the
first step in a larger development process (Dalton, 1996; McCauley &
59
Moxley, 1996; Ward, 1995). Vinson (1996) indicates that once strengths and
developmental needs are identified, the employee must decide how to use the
information to facilitate improvement. In general, this stage involves creating
some sort of development plan by using the feedback to set specific goals
(Yukl & Lepsinger, 1995). The employee then must work to achieve those
goals using this plan (Peters, 1996).
Using feedback from multiple sources is vital to the effectiveness of
development plans for two reasons. First, it provides potentially accurate and
diverse information on where employees stand in relation to where they
should be in terms of skills and behaviors. Second, 360-degree feedback may
increase self-awareness and provide insight into how employees behavior
and performance affects those around them. Both of these aspects are thought
to be important contributions of 360-degree feedback.
To use multi-rater feedback for development, certain factors should be
considered relating to both the organization and the individual. On the organi-
zational level, development should be encouraged and supported (Denton,
1994). After an individuals developmental needs are identified, it is neces-
sary for the organization to play an active role in the development process
(Bracken, 1994). Organizational support can take various forms, from a
simple show of support to extensive and specific training (Yukl & Lepsinger,
1995). Organizational support for individual development may require a shift
in the culture of the organization, and it may take time for development based
on 360-degree feedback to reach its maximum potential.
Using 360-degree feedback for performance appraisal also involves
follow-up. If the feedback is linked to pay and promotion decisions, then an
appeal process may be necessary (Mohrman et al., 1990). There should be an
opportunity for the employee involved to react to the feedback and the
associated decisions. This is especially true in the early stages of implement-
ing a 360-degree system for performance appraisal. Because performance
appraisal focuses on past performance and behavior, it should be followed by
plans or suggestions to improve future performance (Hirsch, 1994). Essen-
tially, performance appraisals evaluate what an employee has done, whereas
feedback for development addresses how the employee gets things done
(London & Beatty, 1993).
What Are Future Trends and Research Needs for 360-degree Feedback?
One trend in 360-degree feedback is the growing popularity of elec-
tronic methods, such as personal computers (PCs), for administration and
feedback reporting. Software programs for administering instruments and
producing feedback reports can be licensed or purchased from several ven-
dors of 360-degree-feedback products. Instruments that are sold for use with
PCs usually have scoring programs built into the software. Although com-
puter PC administration may increase the price for the consumer, the method
has many benefits, especially in the presentation of the feedback to the
recipient. Instead of sifting through mounds of data to find, for example, the
highest rated items, a manager can get the software program to sort and
arrange the information in a fraction of the time (Van Velsor et al., 1997).
Instrument customization is another trend related to the electronic
administration of 360-degree feedback. Advances in technology allow ven-
dors to use what is called an open architecture approach in developing
instruments. This approach permits clients to request that specific items and
scales, which are relevant to their organizations, be added to an instrument.
These items are usually selected from a collection of items in the vendors
database known as an item bank. Although this trend began in order to meet
customer demand, there is currently some disagreement among experts as to
the degree to which this practice affects the validity of the instrument (Van
Velsor et al., 1997).
A future trend to watch for is the administration of 360-degree feedback
over the Internet. Several vendors are developing 360-degree-feedback
processes that will be delivered over the World Wide Web (WWW). The
method shows some promise because it eliminates the need for passing out
and collecting paper forms or diskettes containing an instrument. A downside
to this trend is that all employees do not yet have access to PCs or to the
Internet.
Many of the publications annotated here call for further research on the
360-degree-feedback process. Not only is more research needed related to
business initiatives and impact, there is a need for additional research on the
psychological implications of giving and receiving 360-degree feedback
(London & Beatty, 1993; Yammarino & Atwater, 1993). This research should
be targeted toward issues such as rater psychology, self-awareness, and
feedback receptivity; it should also focus on the overall impact and design of
360-degree feedback, including the possible effects of electronic administra-
tion and customization of 360-degree-feedback instruments.
Every indication is that the use of 360-degree feedback will continue to
grow (Bracken, 1994; Budman & Rice, 1994; Hirsch, 1994; Hoffman, 1995;
McGarvey & Smith, 1993; Romano, 1993). Further experience with this
technique will lead to new ideas and methods but also may identify pitfalls to
avoid. There is still little agreement in the literature on the several issues
61
surrounding 360-degree feedback. This lack of consensus, however, has not
seemed to slow its growing popularity. To fairly evaluate 360-degree feed-
back, one must be aware of its benefits but also be cautious of its complexity.
Because it can be such a powerful tool, the potential for error and misuse is
increased. Adding to such potential drawbacks is the fact that 360-degree
feedback is a relatively new method. Although it has been around in various
forms for some time, only with its recent surge in popularity has it been
closely examined. As 360-degree feedback is used more for performance
appraisal, the repercussions of errors in the process become greater (Jones &
Bearley, 1996).
63
Appendix A: Related Sources
These sources, located in the literature search, are relevant to 360-
degree feedback but did not meet the criteria for including works in this
report. However, many of them address technical details relating to 360-
degree feedback and assessment in general and are included here as supple-
mentary information.
Dennis J. Adsit, Steven Crom, Dana Jones, and Manuel London. Management
performance from different perspectives: Do supervisors and subordi-
nates agree? Journal of Managerial Psychology 9:2, 1994, pp. 22-29.
Richard D. Arvey, David A. Cole, Joy F. Hazucha, and Frans M. Hartanto.
Statistical power of training evaluation designs. Personnel Psychology
38, 1985, pp. 493-508.
Susan Ashford. The feedback environment: An exploratory study of cue use.
Journal of Organizational Behavior 14, 1993, pp. 201-224.
Leanne Atwater, Paul Roush, and Allison Fischthal. The influence of upward
feedback on self and follower ratings of leadership. Personnel Psychol-
ogy 48, 1995, pp. 35-59.
Leanne Atwater and Francis Yammarino. Does self-other agreement on
leadership perceptions moderate the validity of leadership and perfor-
mance predictions? Personnel Psychology 45, 1992, pp. 141-164.
Enrico Auteri. Upward feedback leads to culture change. HR Magazine 39:6,
June 1994, pp. 78-84.
Timothy T. Baldwin and J. Kevin Ford. Transfer of training: A review and
directions for future research. Personnel Psychology 41, 1988, pp.
63-105.
Julie H. Barclay and Lynn K. Harland. Peer performance appraisals. Group
and Organizational Management 20:1, 1995, pp. 39-60.
Galen L. Baril, Roya Ayman, and David J. Palmiter. Measuring leader
behavior: Moderators of discrepant self and subordinate descriptions.
Journal of Applied Social Psychology 24:1, 1994, pp. 82-94.
Thomas E. Becker and Richard Klimoski. A field study of the relationship
between the organizational feedback environment and performance.
Personnel Psychology 42, 1989, pp. 343-358.
H. John Bernardin and Richard W. Beatty. Can subordinate appraisals en-
hance managerial productivity? Sloan Management Review 28:4, 1987,
pp. 63-73.
Walter C. Borman, Leonard A. White, and David W. Dorsey. Effects of ratee
task performance and interpersonal factors on supervisor and peer
performance ratings. Journal of Applied Psychology 80:1, 1995, pp.
168-177.
Mary L. Broad and John W. Newstrom. Transfer of Training. Reading, MA:
Addison-Wesley, 1992.
Stphane Brutus, John Fleenor, and Manuel London. Does 360-degree
feedback work in different industries? A between-industry comparison
of the reliability and validity of multi-source ratings. Journal of Man-
agement Development, in press.
Stphane Brutus, John Fleenor, and Cynthia McCauley. Self-other rating
discrepancy in 360-degree feedback: An investigation of demographic
and personality predictors. Paper presented at the meeting of the
Society for Industrial and Organizational Psychology, San Diego, CA,
April 1996.
Stphane Brutus, John Fleenor, and Sylvester Taylor. Impact of different
congruence methodologies on 360-degree feedback research. Paper
presented at the meeting of the Society for Industrial and Organizational
Psychology, San Diego, CA, April 1996.
Aubry Daniels. Performance Management. Tucker, GA: Performance Man-
agement Publications, 1989.
Mary Beth DeGregorio. An investigation of the influence of self-other feed-
back congruence on feedback acceptance. Paper presented at the
Conference of the Society for Industrial and Organizational Psychol-
ogy, St. Louis, MO, April 1991.
David L. DeVries, Ann M. Morrison, Sandra L. Shullman, and Michael L.
Gerlach. Performance Appraisal on the Line. New York: John Wiley &
Sons, 1981, 160 pages.
65
John Fleenor, Stphane Brutus, and Cynthia McCauley. Does self-other
rating agreement moderate the relationship between personality and
leader effectiveness? Paper presented at the meeting of the American
Psychological Association, Toronto, Canada, August 1996.
John Fleenor, Cynthia McCauley, and Stphane Brutus. Self-other rating
agreement and leader effectiveness. Leadership Quarterly 7, 1996, pp.
487-506.
Clive Fletcher. Appraisal: Routes to Improved Performance. Great Britain:
Short Run Press, 1993, 197 pages.
Harry Gaines. Upward performance appraisals: Employees chance to rate the
boss. Human Resources Professional 4:4, Summer 1992, pp. 53-55.
Richard D. Goffin and Douglas N. Jackson. Analysis of multitrait-multirater
performance appraisal data: Composite direct product method versus
confirmatory factor analysis. Multivariate Behavioral Research 27:3,
1992, pp. 363-385.
Michael M. Harris and John Schaubroeck. A meta-analysis of self-supervisor,
self-peer, and peer-supervisor ratings. Personnel Psychology 41, 1988,
pp. 43-59.
Joy F. Hazucha, C. Szymanski, and S. Birkeland. Will my boss see my rat-
ings? Paper presented at American Psychological Association, August
1992, Washington, DC.
William J. Heisler (Ed.). 360-degree feedback: Payoffs, practices, pitfalls, and
prescriptions. Career Development International 1:3, 1996, 36 pages.
Lowell W. Hellervick, Joy F. Hazucha, and Robert J. Schneider. Behavior
change: Models, methods, and a review of evidence. In M. D. Dunnette
& L. Hough (Eds.), Handbook of Industrial and Organizational Psy-
chology (2nd ed., Vol. 3.). Palo Alto, CA: Consulting Psychologists
Press, 1992.
Ann Howard, William Byham, and Patrick Hauenstein. Multirater assessment
and feedback: Applications, implementation, and implications.
Pittsburgh: DDI, 1994.
Michael J. Kavanagh (Ed.). Special issue on 360-degree feedback systems.
Group & Organization Management 22:2&3, June 1997, pp. 147-309.
Donald L. Kirkpatrick. Evaluating Training Programs: The Four Levels. San
Francisco: Berrett-Koehler, 1994.
Jack J. Kramer and Jane C. Conoley (Eds.). The Eleventh Mental Measure-
ments Yearbook. The Buros Institute of Mental Measurements. Lincoln:
The University of Nebraska, 1992.
Chris Lee. Talking back to the boss. Training, April 1990, pp. 29-34.
Delbert C. Miller. Handbook of Research Design and Social Measurement.
Newbury Park, CA: Sage, 1991.
Milan Moravec, Herman Gyr, and Lisa Friedman. A 21st century communi-
cation tool. HR Magazine 38:7, July 1993, pp. 77-81.
Raymond A. Noe and Steffannie Wilk. Investigation of the factors that
influence employees participation in development activities. Journal of
Applied Psychology 78, 1993, pp. 291-302.
Jean-Francois Pinsonnault. Grading the bosss performance. CMA Magazine
66:5, June 1992, pp. 10-12.
Tom Redman and Ed Snape. Upward and onward: Can staff appraise their
managers? Personnel Review 21:7, 1992, pp. 32-46.
Catherine Romano. Conquering the fear of feedback. Personnel 71:3, March
1994, pp. 9-10.
Barry R. Schlenker. Impression Management. Belmont, CA: Wadsworth,
1980.
Will Schutz. The Human Element: Productivity, Self-esteem, and the Bottom
Line. San Francisco: Jossey-Bass, 1994.
Heinz Shuler, James L. Farr, and Mike Smith (Eds.). Personnel Selection and
Assessment. Hillsdale, NJ: Lawrence Erlbaum Associates, 1993, 375
pages.
James W. Smither, Arthur J. Wohlers, and Manuel London. A field study of
reactions to normative versus individualized upward feedback. Group
and Organizational Management 20:1, 1995, pp. 61-89.
67
Lynn Summers. A logical approach to individual development planning.
Training and Development, November 1994, pp. 22-30.
Richard Sweetland and Daniel J. Keyser (Eds.). Tests: A Comprehensive
Reference for Assessment in Psychology, Education and Business (3rd
edition). Austin, TX: Pro-ed, 1990.
Scott I. Tannenbaum and Gary Yukl. Training and development in organiza-
tions. Annual Review of Psychology 43, Palo Alto, CA: Annual Re-
views, 1992, pp. 399-441.
Carol W. Timmreck. Upward feedback in the trenches: Challenges and
realities. Paper presented at the conference of the Society for Industrial
and Organizational Psychology, May 1995, Orlando, FL.
Walter W. Tornow (Ed.). Special issue on 360-degree feedback. Human
Resource Management 32:2&3, Summer/Fall 1993, pp. 209-407.
Robert Tucker and Milan Moravec. How BPX fights bureaucracy with
upward feedback. Human Resources Professional 4:3, Spring 1992, pp.
30-32.
Ellen Van Velsor, Sylvester Taylor, and Jean Leslie. An examination of the
relationships among self-perception accuracy, self-awareness, gender,
and leader effectiveness. Human Resource Management 32, 1993, pp.
249-264.
Arthur Wohlers and Manuel London. Ratings of managerial characteristics:
Evaluation difficulty, co-worker agreement, and self-awareness. Per-
sonnel Psychology 42, 1989, pp. 235-261.
Minda Zetlin. Evaluate a salesperson by asking peers and clients. Sales &
Marketing Management 146:14, December 1994, p. 85.
69
Appendix B: Selection Criteria
In searching the literature related to 360-degree feedback and assess-
ment, we noticed the emergence of three trends. First, it appears that more
literature has been published on this subject in the past five years than in the
five decades before that. Second, in each subsequent year, more articles,
papers, books, and studies on 360-degree feedback have appeared. Third, the
literature on 360-degree feedback can sometimes be difficult to recognize and
to locate.
This difficulty is attributable to the following factors. The first obstacle
to finding published literature is the lack of a consistent terminology for 360-
degree feedback. Many terms are used (multisource, multi-rater, multiple
level, 360 degree, and so forth) that are essentially synonymous. Second,
usage of these terms only recently became widespread, and most articles
published before 1990 or so did not use these labels. Third, many relevant
papers and articles are yet to be published, although some of the more rel-
evant unpublished sources are included in this review.
In addition, the search for information on 360-degree feedback involved
the prevalence of literature on related topics. These topics included: feed-
back, assessment, appraisal, ratings, surveys, questionnaires, performance,
and so on, including combinations of these terms. Searches for these broader
and more prevalent terms revealed hundreds of sources. Many of these
sources, however, were unrelated to 360-degree feedback and, thus, were
eliminated from consideration.
After narrowing it down, the literature on 360-degree feedback fell into
three categories. The first category included the literature whose subject was
wholly 360-degree feedback or assessment. Again, the term 360-degree is
used to include the literature on feedback from multiple sources. The second
category of literature dealt primarily with rater issues. These sources dis-
cussed issues surrounding the collection of performance ratings. Topics
included psychological factors in rating others, the psychometrics of self-
other rating discrepancies, and the effects of confidentiality on the ratings.
The third category included articles related specifically to one rating source,
such as upward or peer feedback, but did not focus on the full 360-degree-
feedback process. For example, some articles briefly mentioned that upward
feedback was part of 360-degree feedback, but the primary subject matter
concerned only upward feedback.
Because the purpose of this report is to provide an introduction and
overview of 360-degree feedback and assessment, the first category of litera-
ture is emphasized. There are a few articles in this bibliography that fall into
the second category. They are included because of their relevance to the
general topic of 360-degree feedback. The primary focus of this report is on
organizational practices, such as leadership development and performance
management; therefore, articles that deal primarily with psychometric issues
were not included.
In sum, literature was chosen for inclusion that was relevant, concise,
accessible, and focused on general issues related to 360-degree feedback. The
books and articles were selected using four main criteria, the first being
relevance. The second criterion was timeliness. This report is intended to be
an up-to-date resource on this subject; therefore, any literature published
before 1990 is not included. This criterion was fairly simple to apply, because
only a few significant references were published before 1990. The third
criterion dealt with the applicability to organizations. The articles included in
this review contain usable and understandable information. Exclusions were
made to increase the accessibility of this information for a general audience.
We hope that the information presented here will be useful to an audience
with varying levels of knowledge about 360-degree feedback. The final
criterion used to select the literature was their diversity and uniqueness in
relation to other sources. To produce an objective and fair-minded report,
we attempted to select works that span the spectrum of thought on 360-
degree feedback.
71
Glossary of Terms
There are a variety of terms used in the literature for 360-degree feed-
back: multisource assessment, multi-rater feedback, 360-degree assessment,
and full-circle evaluation are some. Below, we first define 360-degree assess-
ment, no matter what term is used, and then define some technical language
found in the annotations:
360-degree assessment: An activity of gathering data about an em-
ployee from multiple sources in that persons circle of influence. The
raters are usually superiors, peers, and subordinates and sometimes
family, friends, and acquaintances. Most 360-degree-feedback pro-
cesses use a survey or questionnaire to gather information. These
surveys usually consist of several items (questions), which are
grouped together to form the scales of the instrument. These scales
often represent some type of competency, such as decision making,
delegation, communication, and so forth.
Response scales: Most 360-degree assessments collect scaled re-
sponses to the questionnaire items (for example, a scale of 1 to 5, with
5 meaning all the time and 1 meaning never with regard to how
often something is done). These response scales are also sometimes
called rating scales.
Raters: The respondents to the questionnaire who assess the focal (or
target) individual (the ratee) on behaviors, skills, competencies,
accomplishments, styles, abilities, or performance. The results of
these ratings are collectively the feedback that the person being rated
will receive.
Competencies: Skills or abilities that are considered necessary to
perform the tasks of a job effectively.
73
Author Index
Adsit, Dennis J., 63
Antonioni, David, 3
Arvey, Richard D., 63
Ashford, Susan, 63
Atwater, Leanne E., 4, 47, 63
Auteri, Enrico, 63
Ayman, Roya, 63
Baldwin, Timothy T., 63
Barclay, Julie H., 63
Baril, Galen L., 63
Bearley, William, 20
Beatty, Richard W., 23, 64
Becker, Thomas E., 64
Bernardin, H. John, 4, 64
Birkeland, S., 65
Borman, Walter C., 64
Bracken, David W., 5
Broad, Mary L., 64
Brutus, Stphane, 6, 64, 65
Budman, Matthew, 7
Byham, William, 65
Campbell, David P., 33
Carey, Robert, 8
Cavanaugh, Marcie, 19
Coates, Dennis E., 8
Cole, David A., 63
Conoley, Jane C., 66
Crom, Steven, 63
Crystal, Barbara, 9
Dahmus, Sue A., 4
Dalton, Maxine, 10
Daniels, Aubry, 64
DeGregorio, Mary Beth, 64
Denton, Wayne E., 11
DeVries, David L., 64
Dorsey, David W., 64
Dunnette, Marvin M., 11
Edwards, Colin C., 12
Edwards, Mark R., 13
Ewen, Ann J., 13
Farr, James L., 66
Fischthal, Allison, 63
Fleenor, John W., 6, 43, 64, 65
Fletcher, Clive, 65
Ford, J. Kevin, 63
Friedman, Lisa, 66
Gaines, Harry, 65
Gebelein, Susan, 13
Gerlach, Michael L., 64
Goffin, Richard D., 65
Guinn, Kathleen A., 14
Gyr, Herman, 66
Harland, Lynn K., 63
Harris, Michael M., 65
Hartanto, Frans M., 63
Harvey, Eric L., 15
Hauenstein, Patrick, 65
Hazucha, Joy F., 15, 63, 65
Heisler, William J., 17, 65
Hellervick, Lowell W., 65
Hezlett, Sarah, 15
Hirsch, Marcie S., 17
Hoffman, Robert, 18
Hollenbeck, George P., 32
Holt, Katherine, 19
Howard, Ann, 65
Jackson, Douglas N., 65
Jones, Dana, 63
Jones, John E., 20
Kaplan, Robert E., 21
Kavanagh, Michael J., 65
Keyser, Daniel J., 67
Kirkpatrick, Donald L., 66
Kirksey, Jay, 30
Klimoski, Richard, 64
Kramer, Jack J., 66
Lawler, Edward E., III, 31
Lee, Chris, 66
Lepsinger, Richard, 48
Leslie, Jean B., 42, 43, 67
London, Manuel, 6, 23, 24, 39, 63, 64,
66, 67
Lublin, Joann S., 25
Ludeman, Kate, 25
Marchese, Marc, 26
Martinez, Michelle, 27
McCauley, Cynthia D., 27, 64, 65
McGarvey, Robert, 28
McGowan, John J., 26
Melymuka, Kathleen, 29
Miller, Delbert C., 66
Milliman, John F., 30
Millsap, Roger E., 39
Mohrman, Allan M., Jr., 31
Moravec, Milan, 66, 67
Morrison, Ann M., 64
Moses, Joel, 32
Moxley, Russell S., 27
Newstrom, John W., 64
Nicholas, George P., 32
Nilsen, Dianne, 33
Noe, Raymond A., 19, 66
Norman, Carol, 30
Nowack, Kenneth M., 34, 35
OReilly, Brian, 35
Palmiter, David J., 63
Palus, Charles J., 21
Parker-Gore, Shirley, 36
Peters, Helen, 37
Pinsonnault, Jean-Francois, 66
Powell, Lynda, 30
Redman, Tom, 66
Redmon, Gregory, 4
Reilly, Richard R., 39
Resnick-West, Susan M., 31
Rice, Berkeley, 7
Romano, Catherine, 38, 66
Roush, Paul, 63
Salvemini, Nat, 39
Schaubroeck, John, 65
Schlenker, Barry R., 66
Schneider, Robert J., 15, 65
Schutz, Will, 66
Shaver, Warren, Jr., 38
Shuler, Heinz, 66
Shullman, Sandra L., 64
Smith, Mike, 66
Smith, Scott, 28
Smither, James W., 24, 39, 66
Snape, Ed, 66
Sorcher, Melvin, 32
Summers, Lynn, 67
Sweet, Sandra, 40
Sweetland, Richard, 67
Szymanski, C., 65
Tannenbaum, Scott I., 67
Taylor, Sylvester, 64, 67
TEAMS, Inc., 41
Timmreck, Carol W., 67
Tornow, Walter W., 42, 67
Tucker, Robert, 67
Van Velsor, Ellen, 42, 43, 67
Vasilopoulos, Nicholas L., 39
Vinson, Mary N., 44
Wall, Stephen J., 45
Ward, Peter, 46
White, Leonard A., 64
Wilk, Steffannie, 66
Wohlers, Arthur J., 66, 67
Yammarino, Francis J., 4, 47, 63
Yukl, Gary, 48, 67
Zawacki, Robert A., 30
Zetlin, Minda, 67
75
Title Index
360 Degree Feedback, 12
360 Degrees of Evaluation, 17
360-degree Feedback: An Integrated
Perspective, 17
360-degree Feedback as a Competitive
Advantage, 23
360-degree Feedback Can Change Your
Life, 35
360-degree Feedback: Payoffs, Practices,
Pitfalls, and Prescriptions, 65
360-degree Feedback: The Whole Story,
35
360 Feedback: Strategies, Tactics, and
Techniques for Developing Leaders, 20
360 Feedback: The Powerful New Model
for Employee Assessment and Perfor-
mance Improvement, 13
A 21st Century Communication Tool, 66
A 360-degree Turn for the Better, 46
A Field Study of Reactions to Normative
Versus Individualized Upward Feed-
back, 66
A Field Study of the Relationship between
the Organizational Feedback Environ-
ment and Performance, 64
A Logical Approach to Individual
Development Planning, 67
A Meta-Analysis of Self-Supervisor, Self-
Peer, and Peer-Supervisor Ratings, 65
Analysis of Multitrait-Multirater Perfor-
mance Appraisal Data: Composite
Direct Product Method Versus Confir-
matory Factor Analysis, 65
An Examination of the Effects of an
Upward Feedback Program Over Time,
39
An Examination of the Relationships
among Self-perception Accuracy, Self-
awareness, Gender, and Leader Effec-
tiveness, 67
An Investigation of the Influence of Self-
Other Feedback Congruence on
Feedback Acceptance, 64
Appraisal: Routes to Improved Perfor-
mance, 65
Assessment Techniques for Top Execu-
tives, 14
Attitudes of First-line Supervisors toward
Subordinate Appraisals, 4
Behavior Change: Models, Methods, and
a Review of Evidence, 65
Can Multi-source Feedback Change Self-
evaluation, Skill Development, and
Performance?, 24
Can Subordinate Appraisals Enhance
Managerial Productivity?, 64
Choosing 360: A Guide to Evaluating
Multi-rater Feedback Instruments for
Management Development, 43
Coming Around to 360-degree Feedback,
8
Companies Evaluate Employees from All
Perspectives, 30
Conquering the Fear of Feedback, 66
Designing an Effective 360-degree
Appraisal Feedback Process, 3
Designing Performance Appraisal
Systems, 31
Developing Employee Skills to Match
Company Needs, 11
Developmental 360: How Feedback Can
Make Managers More Effective, 27
Does 360-degree Feedback Work in
Different Industries?, 64
Does Self-Other Agreement on Leader-
ship Perceptions Moderate the Validity
of Leadership and Performance Predic-
tions?, 63
Does Self-Other Rating Agreement
Moderate the Relationship between
Personality and Leader Effectiveness?,
65
Effects of Ratee Task Performance and
Interpersonal Factors on Supervisor and
Peer Performance Ratings, 64
Elements of Effective 360-degree Feed-
back, 6
Employee Development: Multi-rater
Feedback Goes Strategic, 13
Enhancing 360-degree Feedback for
Senior Executives, 21
Evaluate a Salesperson by Asking Peers
and Clients, 67
Evaluating Training Programs: The Four
Levels, 66
Fear of Feedback, 38
Feedback on Feedback, 45
Feedback to Managers, Volume II: A
Review and Comparison of Sixteen
Multi-rater Feedback Instruments, 42
Grading the Bosss Performance, 66
Handbook of Research Design and Social
Measurement, 66
How BPX Fights Bureaucracy with
Upward Feedback, 67
How to Build and Use a 360-degree
Feedback System, 38
How to Get the Most Out of 360-degree
Feedback, 48
Human Resource Management: Compen-
sation Reporter, 40
Im OK, Youre OK, 29
Impact of Different Congruence Method-
ologies on 360-degree Feedback
Research, 64
Impression Management, 66
Investigation of the Factors That Influence
Employees Participation in Develop-
ment Activities, 66
Management Performance from Different
Perspectives: Do Supervisors and
Subordinates Agree?, 63
Managers Developmental Responses to
360-degree Feedback, 19
Measuring Leader Behavior: Moderators
of Discrepant Self and Subordinate
Descriptions, 63
Multi-rater Assessment and Feedback:
Applications, Implementation, and
Implications, 65
Multi-rater Feedback and Conditions for
Change, 10
Multi-source, 360 Feedback Survey
Results, 41
Multi-source Better Describes Peer
Feedback, 27
Multi-source Feedback: Seven Recom-
mendations, 8
My Hammer or Your Hammer?, 11
Other Peoples Expectations, 32
Peer Coaching for Executives, 37
Peer Performance Appraisals, 63
Perception Is Reality: Using 360-degree
Appraisal Against Behavioral Compe-
tencies to Effect Organizational Change
and Improve Management Performance,
36
Perceptions or Reality: Is Multi-
perspective Measurement a Means or an
End?, 42
Performance Appraisal on the Line, 64
Performance Management, 64
Personnel Selection and Assessment, 66
Ratings of Managerial Characteristics:
Evaluation Difficulty, Co-worker
Agreement, and Self-awareness, 67
Self-assessment and Rater Assessment as
a Dimension of Management Develop-
ment, 34
Self-Observer Rating Discrepancies: Once
an Overrater, Always an Overrater?, 33
Self-Other Rating Agreement and Leader
Effectiveness, 65
Self-Other Rating Agreement: A Review
and Model, 4
Self-Other Rating Discrepancy in 360-
degree Feedback: An Investigation of
Demographic and Personality Predic-
tors, 64
Self-perception Accuracy: Implications
for Human Resource Management, 47
Special Issue on 360-degree Feedback, 67
Special Issue on 360-degree Feedback
Systems, 65
77
Statistical Power of Training Evaluation
Designs, 63
Straight Talk about Multirater Feedback,
5
Talking Back to the Boss, 66
Ten Reasons You Should Be Using 360-
degree Feedback, 18
Tests: A Comprehensive Reference for
Assessment in Psychology, Education
and Business, 67
The 360 Assessment, 9
The Eleventh Mental Measurements
Yearbook, 66
The Feedback Environment: An Explor-
atory Study of Cue Use, 63
The Human Element: Productivity, Self-
esteem, and the Bottom Line, 66
The Impact of 360-degree Feedback on
Management Skills Development, 15
The Influence of Upward Feedback on
Self and Follower Ratings of Leader-
ship, 63
The Power of 360-degree Feedback, 26
The Pros and Cons of 360-degree Feed-
back: Making It Work, 44
The Rating Game, 7
To Fill the Feedback Void, 25
Training and Development in Organiza-
tions, 67
Transfer of Training, 64
Transfer of Training: A Review and
Directions for Future Research, 63
Turning Performance Appraisals Upside
Down, 15
Turning the Tables: Underlings Evaluate
Bosses, 25
Upward and Onward: Can Staff Appraise
Their Managers?, 66
Upward Feedback in the Trenches:
Challenges and Realities, 67
Upward Feedback Leads to Culture
Change, 63
Upward Performance Appraisals: Em-
ployees Chance to Rate the Boss, 65
Upward Trend Continues in Appraisal
Process, 32
When Workers Rate the Boss, 28
Will My Boss See My Ratings?, 65
Title Index
Ordering Information
For more information, to order additional CCL Press publications,
or to find out about bulk-order discounts, please contact us by
phone at 336-545-2810 or visit our online bookstore at
www.ccl.org/publications.
USING 360-DEGREE FEEDBACK
IN ORGANIZATIONS
The use of multi-rater assessment tools in organizations has increased
dramatically in recent years and so has the number of books and articles
about their use. This publication provides an introduction to this growing
practical literature. In addition to the fifty-six annotations, the report looks
at the literature from several perspectives, including When did 360
develop? What are its benefits? and What are its future trends? Also
included are a list of some important related publications and author and
title indexes.
The Authors
John W. Fleenor is an adjunct researcher at the Center for Creative
Leadership and a research scientist at Mediappraise Corp., a provider of
360-degree-feedback services via the Internet. His research focuses on the
development, use, and validation of 360-degree-feedback instruments.
With a Ph.D. in industrial and organizational psychology from North
Carolina State University, he is a co-author of Choosing 360: A Guide to
Evaluating Multi-rater Feedback Instruments for Management Develop-
ment (CCL, 1997).
Jeffrey Michael Prince has held internships with the Leadership Re-
sources and Marketing Groups at the Center for Creative Leadership in
various research and writing capacities. His work has focused on current
trends in leadership and executive education, as well as the use of assess-
ment for development. He is a senior at the University of Texas at Austin.

Using 360 Feedback

Uploaded by

Copyright:

Available Formats

Using 360 Feedback

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Using 360 Feedback

Uploaded by

Copyright:

Available Formats

CENTER FOR CREATIVE LEADERSHIP

, Campbell Leadership Index

, as well as Personnel Decisions Profilor

instruments. These instruments use upward feedback as part of a multi-rater

, and TEAMS, Inc., showing how their

Group Feedback, Benchmarks

, Campbell Leadership Index

, The Visionary Leader: Leader Behavior

You might also like