Measuring Intercultural Sensitivity - Bennet's Article

ARTICLE IN PRESS International Journal of Intercultural Relations 27 (2003) 421–443 Measuring intercultural sensitivity: The intercultural development inventory Mitchell R. Hammera,*, Milton J. Bennettb, Richard Wisemanc a International Peace and Conflict Resolution Program, The School of International Service, American University, 4400 Massachusetts Ave. NW, Washington, DC 20016, USA b The Intercultural Communication Institute, 8835 SW Canyon Lane, Suite 238, Portland, OR 97225, USA c Department of Speech Communication, California State University at Fullerton, Fullerton, CA 92834, USA Abstract Today, the importance of intercultural competence in both global and domestic contexts is well recognized. Bennett (1986, 1993b) posited a framework for conceptualizing dimensions of intercultural competence in his developmental model of intercultural sensitivity (DMIS). The DMIS constitutes a progression of worldview ‘‘orientations toward cultural difference’’ that comprise the potential for increasingly more sophisticated intercultural experiences. Three ethnocentric orientations, where one’s culture is experienced as central to reality (Denial, Defense, Minimization), and three ethnorelative orientations, where one’s culture is experienced in the context of other cultures (Acceptance, Adaptation, Integration), are identified in the DMIS. Based on this theoretical framework, the Intercultural Development Inventory (IDI) was constructed to measure the orientations toward cultural differences described in the DMIS. The result of this work is a 50-item (with 10 additional demographic items), paper-and-pencil measure of intercultural competence. Confirmatory factor analyses, reliability analyses, and construct validity tests validated five main dimensions of the DMIS, which were measured with the following scales: (1) DD (Denial/Defense) scale (13 items, alpha=0.85); (2) R (Reversal) scale (9 items, alpha=0.80); (3) M (Minimization) scale (9 items, alpha=0.83), (4) AA (Acceptance/Adaptation) scale (14 items, alpha=0.84; and (5) an EM (Encapsulated Marginality) scale (5 items, alpha=0.80). While no systematic gender differences were found, significant differences by gender were found on one of the five scales (DD scale). No significant differences on the scale *Corresponding author. 267 Kentlands, Blvd, PMB #705, North Potomac, MD 20878, USA. Tel.: +1202-885-1653; fax: +1-301-926-7450. E-mail addresses: [email protected] (M.R. Hammer), [email protected] (M.J. Bennett), [email protected] (R. Wiseman). 0147-1767/03/$ - see front matter r 2003 Elsevier Ltd. All rights reserved. doi:10.1016/S0147-1767(03)00032-4 ARTICLE IN PRESS 422 M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 scores were found for age, education, or social desirability, suggesting the measured concepts are fairly stable. r 2003 Elsevier Ltd. All rights reserved. Keywords: Intercultural sensitivity; Intercultural competence; Development model of intercultural sensitivity; Intercultural development inventory; Cross-cultural effectiveness; Adaptation 1. Introduction As we begin the next millennium, the importance of effective intercultural relations in both global and domestic contexts is well recognized (Brislin, Cushner, Cherie, & Yong, 1986; Hammer, 1989, 1999a; Kealey, 1989). As Bhawuk and Brislin (1992) suggested, ‘‘To be effective in another culture, people must be interested in other cultures, be sensitive enough to notice cultural differences, and then also be willing to modify their behavior as an indication of respect for the people of other cultures’’ (p. 416). We will use the term ‘‘intercultural sensitivity’’ to refer to the ability to discriminate and experience relevant cultural differences, and we will use the term ‘‘intercultural competence’’ to mean the ability to think and act in interculturally appropriate ways. We argue that greater intercultural sensitivity is associated with greater potential for exercising intercultural competence. Research studies in such diverse areas as overseas effectiveness (e.g., Brislin, 1981; Cleveland, Mangone, & Adams, 1960; Kealey & Ruben, 1983; Landis & Brislin, 1983a-c; Landis & Bhaget, 1996), international management (e.g., Adler, 1991; Black, 1990; Black, Gregersen, & Mendenhall, 1992; Black & Mendenhall, 1990), international study abroad (e.g., Klineberg & Hull, 1979), and international transfer of technology and information (e.g., Hawes & Kealey, 1979, 1981; Kealey, 1996) have identified intercultural competence as central in increasing understanding and improving relations across cultures (Bennett, 1993a, b; Hammer, 1999b). Additional research on domestic intercultural relations (contact across forms of ethnicity, gender, age, sexual orientation, etc.) has found a similar key role for intercultural competence (e.g., Gardenswartz & Rowe, 1993). While cross-cultural research has posited the importance of intercultural competence in both global and domestic contexts, work by Bennett (1986, 1993b) has additionally suggested an underlying theoretical framework, which he calls the Developmental Model of Intercultural Sensitivity (DMIS), for conceptualizing intercultural sensitivity and competence. This theoretical framework provided conceptual guidance as we undertook the construction of the Intercultural Development Inventory (IDI) to measure the orientations toward cultural differences described in the DMIS. The result of this work is a 50-item, paperand-pencil instrument (with 10 additional demographic items). It is the purpose of this article first to review briefly the DMIS; second, to summarize the initial (Phase 1) and then subsequent (Phase 2) final development of the IDI; and third, to conclude with some comments on the potential for applying the concept of intercultural competence and the IDI. ARTICLE IN PRESS M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 423 2. Developmental Model of Intercultural Sensitivity The Developmental Model of Intercultural Sensitivity (DMIS) was created by Bennett (1986, 1993b) as an explanation of how people construe cultural difference. Using a grounded theory approach (e.g., Glaser & Strauss, 1967; Strauss & Corbin, 1990), Bennett applied concepts from cybernetic constructivism (cf. Von Foerster, 1984; Brown, 1972; Maturana & Varela, 1987) to his observations of intercultural adaptation and identified six orientations that people seem to move through in their acquisition of intercultural competence. The underlying assumption of the model is that as one’s experience of cultural difference becomes more complex and sophisticated, one’s potential competence in intercultural relations increases. According to this constructivist view, experience does not occur simply by being in the vicinity of events when they occur. Rather, experience is a function of how one construes the events (Kelly, 1963). The more perceptual and conceptual discriminations that can be brought to bear on the event, the more complex will be the construction of the event, and thus the richer will be the experience. In the case of intercultural relations, the ‘‘event’’ is that of cultural difference. The extent to which the event of cultural difference will be experienced is a function of how complexly it can be construed. The set of distinctions that is appropriate to a particular culture is referred to as a cultural worldview. Individuals who have received largely monocultural socialization normally have access only to their own cultural worldview, so they are unable to construe (and thus are unable to experience) the difference between their own perception and that of people who are culturally different. The crux of the development of intercultural sensitivity is attaining the ability to construe (and thus to experience) cultural difference in more complex ways. The DMIS assumes that construing cultural difference can become an active part of one’s worldview, eventuating in an expanded understanding of one’s own and other cultures and an increased competence in intercultural relations. Each orientation of the DMIS is indicative of a particular worldview structure, with certain kinds of attitudes and behavior vis-a" -vis cultural difference typically associated with each configuration. Thus, the DMIS is not a descriptive model of changes in attitudes and behavior. Rather, it is a model of changes in worldview structure, where the observable behavior and self-reported attitudes at each stage are indicative of the state of the underlying worldview. Each change in worldview structure generates new and more sophisticated issues to be resolved in intercultural encounters. The resolution of the relevant issues activates the emergence of the next orientation. Since issues may not be totally resolved, movement may be incomplete and one’s experience of difference diffused across more than one worldview. However, movement through the orientations is posited to be unidirectional, with only occasional ‘‘retreats.’’ In other words, people do not generally regress from more complex to less complex experiences of cultural difference. As illustrated in Fig. 1, the first three DMIS orientations are conceptualized as more ethnocentric, meaning that one’s own culture is experienced as central to reality ARTICLE IN PRESS 424 M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 Denial Defense Reversal Minimization ETHNOCENTRISM Acceptance Adaptation Integration ETHNORELATIVISM Fig. 1. in some way. Denial of cultural difference is the state in which one’s own culture is experienced as the only real one. Other cultures are either not discriminated at all, or they are construed in rather vague ways. As a result, cultural difference is either not experienced at all, or it is experienced as associated with a kind of undifferentiated other such as ‘‘foreigner’’ or ‘‘immigrant.’’ People with a Denial worldview generally are disinterested in cultural difference when it is brought to their attention, although they may act aggressively to eliminate a difference if it impinges on them. In a more extreme form of Denial, the people of one’s own culture may be perceived to be the only real ‘‘humans’’ and other people are viewed as simpler forms in the environment to be tolerated, exploited, or eliminated as necessary. Bennett suggests that Denial of cultural difference is the default condition of typical, monocultural primary socialization. Defense against cultural difference is the state in which one’s own culture is experienced as the only viable one. People at Defense have become adept at discriminating difference, so they experience cultural differences as more ‘‘real’’ than do people at Denial. But the Defense worldview structure is not sufficiently complex to generate an equally ‘‘human’’ experience of the other. The cultural differences experienced by people with a Defense perspective are still stereotypical, however they seem real by comparison to the Denial condition. Consequently, people at Defense are more openly threatened by cultural differences than are people in a state of Denial. The world is organized into ‘‘us’’ and ‘‘them,’’ where one’s own culture is superior and other cultures are inferior. People of dominant cultures are likely to experience Defense as an attack on their values (often perceived by others as privileges). People of non-dominant cultures are more likely to experience Defense as discovering and solidifying a separate cultural identity in contrast to the dominant group (cf., Banks, 1988; Parham, 1989). A variation on Defense is Reversal, where an adopted culture is experienced as superior to the culture of one’s primary socialization (‘‘going native,’’ or ‘‘passing’’). Reversal is like Defense in that it maintains a polarized, ‘‘us’’ and ‘‘them’’ worldview. It is unlike Defense in that it does not maintain the other culture as a threat. Minimization of cultural difference is the state in which elements of one’s own cultural worldview are experienced as universal. The threat associated with cultural differences experienced in Defense is neutralized by subsuming the differences into familiar categories. For instance, cultural differences may be subordinated to the overwhelming similarity of people’s biological nature (physical universalism). The experience of similarity of natural physical processes may then be generalized to other assumedly natural phenomena such as needs and motivations. The experience of similarity might also be experienced in the assumed cross-cultural applicability of ARTICLE IN PRESS M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 425 certain religious, economic, or philosophical concepts (transcendent universalism). Because these ‘‘universal absolutes’’ obscure deep cultural differences, other cultures may be trivialized or romanticized. People at Minimization expect similarities, and they may become insistent about correcting others’ behavior to match their expectations. Particularly for people of dominant cultures, Minimization tends to mask recognition of their own culture (ethnicity) and the institutional privilege it affords its members. The second three DMIS orientations are defined as more ethnorelative, meaning that one’s own culture is experienced in the context of other cultures. Acceptance of cultural difference is the state in which one’s own culture is experienced as just one of a number of equally complex worldviews. By discriminating differences among cultures (including one’s own), and by constructing a metalevel consciousness, people with this worldview are able to experience others as different from themselves, but equally human. People at Acceptance can construct culture-general categories that allow them to generate a range of relevant cultural contrasts among many cultures. Thus, they are not just experts in one or more cultures (although they might also be that); rather, they are adept at identifying how cultural differences in general operate in a wide range of human interactions. Acceptance does not mean agreement—some cultural difference may be judged negatively—but the judgment is not ethnocentric in the sense of withholding equal humanity. The major issue to be resolved in this perspective regards ‘‘value relativity.’’ To accept the relativity of values to cultural context (and thus to attain the potential to experience the world as organized by different values), people need to figure out how to maintain ethical commitment in the face of such relativity (cf., Perry, 1970). Adaptation to cultural difference is the state in which the experience of another culture yields perception and behavior appropriate to that culture. One’s worldview is expanded to include relevant constructs from other cultural worldviews. People at Adaptation can engage in empathy—the ability to take perspective or shift frame of reference vis-a" -vis other cultures. As noted earlier, this shift is not merely cognitive; it is a change in the organization of lived experience, which necessarily includes affect and behavior. Thus, people at Adaptation are able to express their alternative cultural experience in culturally appropriate feelings and behavior. If the process of frame shifting is deepened and habitualized, it becomes the basis of biculturality or multiculturality. Integration of cultural difference is the state in which one’s experience of self is expanded to include the movement in and out of different cultural worldviews. Here, people are dealing with issues related to their own ‘‘cultural marginality’’; they construe their identities at the margins of two or more cultures and central to none. As suggested by Bennett (1993a), cultural marginality may have two forms: an encapsulated form, where the separation from culture is experienced as alienation; and a constructive form, in which movements in and out of cultures are a necessary and positive part of one’s identity. Integration is not necessarily better than Adaptation in situations demanding intercultural competence, but it is descriptive of a growing number of people, including many members of non-dominant cultures, long-term expatriates, and ‘‘global nomads.’’ ARTICLE IN PRESS 426 M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 In general, the more ethnocentric orientations can be seen as ways of avoiding cultural difference, either by denying its existence, by raising defenses against it, or by minimizing its importance. The more ethnorelative worldviews are ways of seeking cultural difference, either by accepting its importance, by adapting perspective to take it into account, or by integrating the whole concept into a definition of identity. 3. Phase 1: developing an initial (60-item) version of the IDI Some empirical research has been undertaken focused on developing preliminary measures of DMIS concepts (Pederson, 1998; Tower, 1990). However, these instruments were not subjected to psychometric testing. Therefore, we undertook an effort to develop a measure of the identified DMIS orientations following scale construction guidelines (e.g., DeVellis, 1991). This effort consisted of two phases. In the first phase, a preliminary, 60-item version of the IDI was developed. Subsequent testing of this version by Paige, Jacobs-Cassuto, Yershova and DeJaeghere (1999) suggested specific directions in further development of the IDI. In the second phase, we completed further analysis that resulted in a revised, 50-item IDI that is presented in this paper. We were initially concerned that the empirical observations upon which the DMIS was based could be re-created in systematic ways. This concern was addressed by examining discourse of people from a variety of cultures in order to determine if observers could reliably categorize the discourse in ways identified in the DMIS theoretical framework. A qualitative interview guide was designed to elicit perceptions of a group of respondents concerning their experience with cultural differences. This interview guide included questions that focused generally on how people experience cultural differences.1 A research team was assembled, trained in cross-cultural interviewing techniques, and introduced to the DMIS. Following 1 The interview guide first asked respondents to introduce themselves by stating their name and describing their background. The interviewer then stated (1) the purpose of the interview was to discuss the respondents’ experiences with interacting with people from different cultural backgrounds, (2) the length of time to complete the interview (i.e., 45 min.), and (3) a statement about the interview format (i.e., openended, conversational). Following this, the interviewers discussed confidentiality issues and informed the respondents that they may end the interview any time they wish. Finally, the respondents were asked to sign an ‘‘informed consent’’ form that formally indicated their willingness to participate in the interview. A set of six general discussion questions comprised the interview guide. Each question was designed to stimulate further discussion concerning how the respondents experience cultural differences. The questions were: (1) Do you think there is much cultural differences around here (Denial); (2) What kinds of difficulties or problems associated with having cultural differences around here exist (Defense); (3) When it comes down to the bottom-line, is it more important to pay attention to cultural differences or similarities among us. If respondent emphasizes the importance to pay attention to similarities, follow-up with, What do you think the similarities are (Minimization); (4) Do you make any specific efforts to find out more about the cultures around you (Acceptance); (5) Do you try to adapt your communication to people from other cultures. Does it mean anything to you to look at the world through the eyes of a person from another culture? Do you feel you have two or more cultures (Adaptation); and (6) has your adjustment to other cultures led you to question your identity? Do you feel apart from those cultures that you are involved in (Integration). ARTICLE IN PRESS M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 427 training, the interviewers pilot-tested the interview guide by interviewing international student volunteers from a private university in the United States. While the pilot interviews were conducted with individual students from a variety of cultures, it was decided that the actual sample of interviewees would consist of people of varied cultural backgrounds who also extended beyond the university community. Therefore, the interview sample was selected from residents from such places as the International House in Washington, DC (where professionals from many different countries reside) as well as various places of employment in and around the Washington, DC area. A total of 40 men and women representative of a wide range of ages, experiences, and cultures were interviewed, their responses audio-taped, and a verbatim transcript subsequently prepared. Respondents from this sample were from the following countries: USA, European American (n=12), USA, South Asian American (n ¼ 3), England (n ¼ 1), Japan (n ¼ 3), Switzerland (n ¼ 2), China (n ¼ 1), Korea (n ¼ 2), Ireland (n ¼ 2), Denmark (n ¼ 1), Spain (n ¼ 1), France (n ¼ 3), Germany (n ¼ 1), Russia (n ¼ 2), Estonia (n ¼ 1), India (n ¼ 1), Turkey (n ¼ 1), Ecuador (n ¼ 1), Guyana (n ¼ 1), and Ivory Coast (n ¼ 1). For the international participants, their average length of stay in the United States was as follows: less than 1 year (n ¼ 9), 1–2 years (n ¼ 2), 3–5 years (n ¼ 7), and over 7 years (n ¼ 7). Overall, the sample of 40 individuals represented a culturally diverse group with varying international experiences. Four members of the research team then independently reviewed 25 randomly selected transcripts from among the 40 and rated the DMIS orientations the interviewees’ most consistently expressed during the interview. Ratings were obtained across all six orientations (Denial, Defense/Reversal, Minimization, Acceptance, Adaptation, Integration) identified in the DMIS. Inter-rater reliabilities (Cohen’s kappa) were then calculated across these 25 interviews for ratings given by the four members of the research team. Cohen’s kappa for the ratings among the four raters was good, ranging from 0.66 to 0.86 for the stage ratings. According to Fleiss (1981) and Brennan and Prediger (1981), a kappa of 0.40–0.60 is fair, 0.60– 0.75 is good, and 0.75 or greater is excellent. Identification of a pool of items that reflect the latent variables underlying them (i.e., the worldview orientations to cultural difference) was then undertaken. One important consideration is the cross-cultural generalizability of the pool of items. In addressing this issue, we reviewed each transcript obtained from the 40 interviews and, because the transcripts were verbatim, we selected the actual statements used by the culturally diverse respondents when talking about cultural differences. Our previous analysis suggested that these statements clustered within the identified DMIS orientations. Therefore, we listed each statement made by the respondents that was indicative of their identified worldview. This produced a large (200+) list of statements. Once this base pool of items was identified, we reviewed the items for clarity, sentence structure, and ambiguous meanings. This resulted in the elimination of some of the items from the pool. Following this, we generated additional items for some of the dimensions based on our assessment of the number and ARTICLE IN PRESS 428 M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 comprehensiveness of the items identified in this initial pool (DeVellis, 1991). These additional items comprised less than 20% of the final list of 239 IDI sample items. Finally, the following response options were incorporated: 1=strongly disagree, 2=disagree, 3=slightly disagree, 4=neutral, 5=slightly agree, 6=agree, 7=strongly agree. A pilot version of the IDI was then constructed using the 239 sample items. Two pilot test administrations of the IDI with culturally diverse groups of people were completed in order to identify difficulties respondents may have had with such issues as clarity of instructions, item clarity, response option applicability, and overall amount of time taken to complete the instrument. Based upon feedback from respondents, the IDI was further revised in these format areas. 4. Panel review A panel of experts then reviewed the item pool. This aided in further establishing the relevancy of the items to the construct of intercultural competence as well as providing initial reliability and validity estimates (DeVellis, 1991). The panel of experts was selected based on their demonstrated expertise within the intercultural field and familiarity with the DMIS in particular.2 A list of the 239 randomly ordered IDI items was sent to each expert to review. These individuals were asked to independently categorize the DMIS orientations or to check a response option of ‘‘unable to identify’’ if they felt the item could not be appropriately categorized. Inter-rater reliability among the expert ratings was then determined for each item. The criteria for selecting items from this analysis included the following: (1) a minimum of 5 of the 7 experts were able to categorize the item (i.e., if more than 2 of the 7 experts felt the item was too difficult to categorize, the item was eliminated from further consideration) and (2) inter-rater agreement for placing the item in the same category among the experts had to be 0.60 or above. These agreement criteria reflected exact agreement among 3, 4, or 5 of 5 raters; 4, 5, or 6 of 6 raters; and 5, 6, or 7 of 7 raters. Those items that could not either be categorized or reliably categorized by achieving an agreement rating of 0.60 or higher were eliminated.3 Finally, each expert provided comments concerning each 2 A demographic information survey was administered to prospective expert raters. This survey assessed each potential expert rater in terms of: (1) degrees received, (2) current position, (3) number of years worked in the field of intercultural communication, (4) countries and cultures with which rater had significant interaction, (5) description of how the DMIS is used by the respondent, (6) areas of publication and research, and (7) other information. Based on this information, a group of seven experts in the field of intercultural communication who were intimately familiar with the DMIS were identified and asked to participate in the study. These experts all had advanced degrees (M.A., Ph.D.) with a concentration in intercultural communication. Four of the seven experts were completing their Ph.D. degree in intercultural communication at two major communication programs in the United States. The remaining three experts possessed Ph.D. degrees with specialization in intercultural communication. All expert raters had lived 2 years or more in other cultures. 3 It should be noted that by far the majority of categorizations made by the panel of experts coincided with the initial raters’ categorization of items from the written transcripts. However, where discrepancies arose, the items were reclassified into the stages and forms identified by the panel of experts. ARTICLE IN PRESS M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 429 item’s clarity and conciseness. This produced a further refinement of the IDI, resulting in a smaller pool of 145 items. 5. Sample testing This 145-item version of the IDI was administered to a sample of 226 subjects along with selected demographic items. The sample size approached the sample requirement recommended by Nunnally (1978) of 300 respondents for scale testing. The sample of respondents came from all walks of life and was not primarily drawn from a college student population. Of the 226 respondents, 43% were men (n ¼ 97) and 57% were women (n ¼ 127). The ages ranged from the low teens to over 60 years of age. The majority of respondents were between the ages of 22–30 (45%), with 10% under 21 years of age, 18% between 31 and 40, 16% between 41 and 50, 5% between 51 and 60 and 0.5% over 60 years of age. Thirty respondents were high school graduates (13%), 90 were college graduates (40%), 70 had M.A. or equivalent graduate degrees (31%), and 13 had Ph.D. or equivalent degrees (5%). Twenty percent (n ¼ 46) of the respondents never lived in another culture, 12% (n ¼ 27) lived overseas less than 6 months. Seventeen respondents (7%) lived in another culture 3–6 months, 15 (6%) lived 7–12 months, 39 (17%) lived 1–2 years, 33 (14%) lived 3–5 years, 20 (9%) lived 6–10 years, and 26 (12%) lived over 10 years in another culture. Seventy percent of the respondents (n ¼ 177) were from the United States and 30% (n ¼ 49) came from 28 different countries, from all continents. Of the respondents who indicated they were from the United States, 112 indicated their cultural identity as ‘‘American,’’ eight indicated ‘‘African American,’’ two ‘‘Asian American,’’ four ‘‘Hispanic American,’’ and nine ‘‘Jewish American.’’ The 145 items in the IDI were statistically examined to determine their suitability for inclusion in the IDI questionnaire. The first test undertaken concerned the degree to which the identified items comprised unidimensional scales. Because the extensive time and effort expended with experts in categorizing items directly addressed content validity concerns, it was decided that a more targeted (rather than exploratory) factor analysis would be conducted on all the items categorized within each of the DMIS orientations. Therefore, six separate factor analyses were run on: (1) the 21 items which comprised the ‘‘Denial’’ stage, (2) the 37 items that comprised the ‘‘Defense/Reversal’’ stage, (3) the 22 items identified with the ‘‘Minimization’’ stage, (4) the 18 items that comprise ‘‘Acceptance,’’ (5) the 26 items identified with ‘‘Adaptation,’’ and (6) the 21 items associated with the ‘‘Integration’’ stage. The most interpretable factor structures for each of the factor analyses of the IDI dimensions of Denial, Minimization, Acceptance, Adaptation, and Integration emerged with a forced, two-factor principal components analysis with VARIMAX rotation. The most interpretable factor structure for the IDI dimension of Defense/ Reversal was obtained with a three-factor principal components analysis with VARIMAX rotation. A minimum primary loading of 0.50 and a secondary loading ARTICLE IN PRESS 430 M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 of 0.20 or less than the primary loading was used to isolate factors (Neuliep & McCroskey, 1997). In addition, eigenvalues greater than 2.0 for factors were also used as criteria for determining the number of factors to be further considered (i.e., factors with eigenvalues of less than 2.0 were not considered for further interpretation). In selecting specific scale items from interpretable factors, loadings of approximately 0.55 and higher in one factor with a minimum of 0.20 loading difference in all other factors were used. Results from the targeted factor analysis and the reliability analyses of the 145 items identified six scales: (1) Denial scale (10 items, alpha=0.87), (2) Defense scale (10 items, alpha=0.91), (3) Minimization scale (10 items, alpha=0.87), (4) Acceptance scale (10 items, alpha=0.80), (5) Cognitive Adaptation (10 items, alpha=0.85), and (6) Behavioral Adaptation (10 items, alpha=0.80).4 This approach did not produce reliable scales that assessed the Reversal orientation nor the Integration orientation identified in the DMIS. Further, this analysis produced two separate scales for Adaptation: One scale focused more on ‘‘cognitive frame shifting’’ while the other scale focused more on ‘‘behavioral code shifting’’ forms of cultural adaptation. These scales were named Cognitive Adaptation and Behavioral Adaptation, respectively. 6. Independent research on the 60-item IDI Research by Paige et al. (1999) examined the empirical structure of this 60-item IDI with a sample of 330 respondents. They conducted one general factor analysis across all 60 items of the IDI, identifying six factors. Factor 1 consisted of a combination of Denial and Defense items. Factor 2 consisted of eight Cognitive Adaptation and four Behavioral Adaptation items that were worded in a more cognitive manner. Factor 3 consisted of five items that referred to the physical universalism form of Minimization, and Factor 4 consisted of three items that referred to the transcendent universalism form of Minimization. Factor 5 contained three Behavioral Adaptation items, and, in a factor not consistent with the DMIS, Factor 6 contained two Acceptance and two Denial items. In addition, the authors examined a two-factor solution to ‘‘see if more global categories of ethnocentric and ethnorelative stages manifest themselves’’ (p. 15). Results indicated that the ethnorelative scales of Acceptance, Cognitive Adaptation, and Behavioral Adaptation comprised one factor, Denial and Defense items clustered on the second factor ‘‘with Minimization in the middle between the two’’ (p. 16). The results from the Paige et al. (1999) study suggested that the factors identified in the 60-item IDI might not be as stable as desired. Further, their findings suggested the possibility of three more fundamental dimensions: a factor composed of Denial and Defense items, a Minimization factor, and a factor that largely consists of Acceptance and Adaptation. Overall, these findings indicated that additional 4 For additional information on the development of this initial (60-item) version of the IDI, see Hammer and Bennett (1998). ARTICLE IN PRESS M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 431 analysis should be done in establishing a more stable set of factors and subsequent measurement scales for the orientations theoretically proposed in the DMIS. Also, we felt additional work should attempt to measure the Reversal and Integration orientations proposed in the DMIS that were not obtained as scales in the development of the 60-item version of the IDI. Therefore, we embarked on Phase 2 of the development of the IDI, building on the earlier development of the 60-item inventory and the insights obtained from Paige et al. (1999). 7. Phase 2: developing the final 50-item IDI We reviewed the 145 original items (from which the initial IDI version was constructed) in the light of our experience using the 60-item IDI with a variety of culturally diverse groups. By minor editing and selecting valid alternative items, we finalized a set of 122 items for subsequent administration. In this final set of items were some additional items that we generated to assess Reversal and Integration orientations. We also decided that a five-point response scale was more appropriate than the original seven-point response scale used in the development of the 60-item IDI. The response options for the 122 items were: 1=disagree, 2=disagree somewhat more than agree, 3=disagree some and agree some, 4=agree somewhat more than disagree, and 5=agree. Finally, as was the case in the initial version, we obtained demographic information on gender, age, amount of previous experience living in another culture, educational level, national and ethnic background, and the world region in which the respondent lived during his/her formative years. The 122 revised items were combined with items from three additional scales—the Worldmindedness scale and the Intercultural Anxiety scale that were subsequently used for establishing construct validity, and a scale to check for social desirability effects. Then the questionnaire was administered to a new sample of culturally diverse respondents. 8. Sample The sample of 591 respondents was not primarily drawn from a college student population, so it represented a relatively wide range of age and activity. The sample size exceeds the sample requirement recommended by Nunnally (1978) of 300 respondents for scale testing. Of the 591 respondents, 35% were men (n ¼ 204) and 65% were women (n ¼ 376). Their ages ranged from the high teens to over 60 years of age. The respondents were evenly split among the age categories, with the largest number of subjects between the ages of 22–30 (25%; n ¼ 144), with 12% under 21 years of age (n ¼ 69), 15% between 31 and 40 (n ¼ 94), 15% between 41 and 50 (n ¼ 94), 14% were 51–60 years of age (n ¼ 81), and 17% were over 60 years of age (n ¼ 99). ARTICLE IN PRESS 432 M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 Twenty-five percent of the respondents attended or graduated from high school (n ¼ 149), 36% were college graduates (n ¼ 216), 23% had M.A. or equivalent graduate degrees (n ¼ 134), and 7% had Ph.D. or equivalent degrees (n ¼ 40). Thirty-two percent (n ¼ 192) of the respondents never lived in another culture, 14% (n ¼ 81) lived overseas less than 3 months, 10% (n ¼ 57) lived in another culture 3–6 months, 6% (n ¼ 34) lived 7–12 months, 10% (n ¼ 58) lived 1–2 years, 10% (n ¼ 60) lived 3–5 years, 5% (n ¼ 33) lived 6–10 years, and 11% (n ¼ 62) lived over 10 years in another culture. Eighty-three percent of the respondents (n ¼ 476) indicated they primarily lived during their formative years to age 18 in North America (United States, Canada, Mexico) while the remaining 17% lived in other parts of the world. In response to the question that asked respondents to indicate their national and/ or ethnic/cultural background, 87% indicated the United States (n ¼ 478) while 13% indicated an international background from 37 different countries. Ethnic background of respondents from the United States was difficult to assess, with 92 respondents ambiguously indicating US American (15%), 178 indicating US White American (30%), 32 indicating US African American (6%), 6 indicating US Latino (1%), 8 indicating US Asian American (2%), and 162 respondents (27%) indicating a number of other cultural, religious, and other affiliations. 9. Confirmatory factor analysis and scale reliability results 5 Confirmatory factor analysis was employed to test whether the proposed DMIS model is consistent with (‘‘fit’’) the data collected from the IDI. A number of statistical procedures were employed to test the adequacy of the fit. First, the ratio of chi-square to degrees of freedom (w2 =df ) was used to assess the discrepancy between the proposed model and the data. Byrne (1989) suggested that the w2 =df should be . . and Sorbom’s (1984) goodnessless than two for an adequate fit. Second, Joreskog of-fit index (GFI) is a generalized estimation criterion, which ranges from zero (no fit) to one (perfect fit). Third, the root mean-square residual (RMR) is an estimate obtained by comparing the values of variances and covariances predicted by the model with the actual variances and covariances ascertained from the data. The larger the RMR, the greater the discrepancy between the model and the data, with zero representing a perfect fit. Finally, the RMSEA provides a fit of the data taking into consideration the complexity of the model. Since there are many items involved 5 Due to the large number of significance tests run in this study coupled with the rather large number of subjects, the 0.01 level of significance was employed for all results presented. In fact, however, all but two significant tests presented as ‘‘not significant’’ at the 0.01 level were also not significant at the 0.05 level, with only two exceptions: (1) the correlations reported between the Worldmindedness scale and the Minimization scale (r ¼ 0:09; p ¼ 0:04; n ¼ 534), and (2) the correlations between the Social Desirability scale and the Minimization scale (r ¼ 0:09; p ¼ 0:03; n ¼ 531). While these two correlations are not significant at the 0.01 level employed in this study, even using the more liberal 0.05 significance level criteria, correlations of 0.09 suggest that very little variance is actually accounted for. Therefore, the 0.01 significance level appears appropriate in this study. ARTICLE IN PRESS M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 433 in the IDI and multiple dimensions of the DMIS, it was felt that RMSEA should be employed, especially when models of different complexity are being compared. Browne and Cudeck (1989) recommend that a criterion of 0.80 or less for the RMSEA in terms of providing a good fit of the data. Using the above criteria, three models were tested via confirmatory factor analysis. The first model consisted of seven dimensions originally posited by Bennett (1986, 1993b). They are Denial, Defense (and its alternative, Reversal), Minimization, Acceptance, Adaptation, and Integration (Encapsulated Marginality form only). Based on the findings from Paige et al. (1999), a second model was comprised of the dimensions of Minimization, Reversal, and Integration, as well as the two merged dimensions of Denial/Defense and Acceptance/Adaptation. The third model was based on the theoretical supposition originally proposed by Bennett that ethnocentrism and ethnorelativism are two underlying dimensions of an individual’s orientation toward cultural differences. This two-dimensional model consisted of an ethnocentric orientation (the merging of Denial, Defense, Minimization, and Reversal) and an ethnorelative orientation (the merging of Acceptance, Adaptation, and Integration). The confirmatory factor analysis of the seven-dimensional model indicated only a modest fit with the data (w2 =df ¼ 1:60; GFI=0.66, RMR=0.09, and RMSEA=0.05). An examination of the modification indices revealed that distinctions between Denial and Defense were problematic. The confirmatory factor analysis of the five-dimensional DMIS was a better fit of the data (w2 =df ¼ 1:63; GFI=0.85, RMR=0.07, and RMSEA=0.03). Further evidence of the better fit for the five-dimensional model was found by comparing the chi-square decrease vs. the decrease in degrees of freedom for the two models. For the seven-dimensional model, chi-square was 6964.5 with 4361 degrees of freedom, while for the five-dimensional model, the chi-square was 2973.3 with 1810 degrees of freedom. The resultant differences in chi-square and degrees of freedom indicate a significantly better fit for the five-dimensional solution (w2diff ¼ 3991:2 and dfdiff ¼ 2551; po0:0001). These results provide support for the five-dimensional model over the seven-dimensional model. It was also decided to test the most parsimonious version of the DMIS model, namely, a two-dimensional representation (ethnocentrism and ethnorelativism). The fit for this simpler model was marginally adequate, but certainly not as good as the five-dimensional model (for the two-dimensional model, w2 =df ¼ 2:20; GFI=0.77, RMR=0.08, and RMSEA=0.05). Further evidence for the better fit of the five-dimensional model over the two-dimensional one was obtained by comparing the differences in chi-squares (2973.3 and 4706.0, respectively) and degrees of freedom (1810 and 2135, respectively) for the two models. The w2diff ¼ 1732:7 and dfdiff ¼ 325 for the two-model comparison; this is highly significant (po0:0001), suggesting that the five-dimensional model is a much better fit of the IDI data. It was decided to opt for the five-dimensional solution and examine the nature of the five dimensions. The confirmatory factor analysis narrowed the final set of items to 52, distributed across the five factors thusly: (1) DD (Denial/Defense) factor ARTICLE IN PRESS 434 M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 (14 items), (2) R (Reversal) factor (9 items), (3) M (Minimization) factor (10 items), (4) AA (Acceptance/Adaptation) factor (14 items), and (5) EM (Encapsulated Marginality) factor (5 items). Scale reliabilities (coefficient alpha) were computed for DD, R, M, AA, and EM items. For individual diagnostic purposes, it was decided that the scale’s reliability should be 0.70 (Nunnally, 1978) or higher (DeVellis, 1991). The reliability results are: DD scale (14 items, alpha=0.85), R scale (9 items, alpha=0.80), M scale (10 items, alpha=0.85), AA scale (14 items, alpha=0.84), and EM scale (5 items, alpha=0.80). Example items for the DD scale are: (1) It is appropriate that people do not care what happens outside their country, (2) People should avoid individuals from other cultures who behave differently, and (3) Our culture’s way of life should be a model for the rest of the world. Sample items from the R scale are: (1) People from our culture are less tolerant compared to people from other cultures, (2) People from our culture are lazier than people from other cultures, and (3) Family values are stronger in other cultures than in our culture. Some items from the M scale include: (1) Our common humanity deserves more attention than culture difference, (2) Cultural differences are less important than the fact that people have the same needs, interests and goals in life, and (3) Human behavior worldwide should be governed by natural and universal ideas of right and wrong. Selected items from the AA scale are: (1) I have observed many instances of misunderstanding due to cultural differences in gesturing or eye contact, (2) I evaluate situations in my own culture based on my experiences and knowledge of other cultures, and (3) when I come in contact with people from a different culture, I find I change my behavior to adapt to theirs. Finally, example items from the EM scale are: (1) I feel rootless because I do not think I have a cultural identification, (2) I do not identify with any culture, but with what I have inside, and (3) I do not feel I am a member of any one culture or combination of cultures. Table 1 presents a summary of the items that comprise the five scales of the 52-item IDI.6 6 All statistical analyses conducted in this study were completed on the 52 items identified through the confirmatory factor analysis. After all analyses were completed, we conducted a final review of the clarity of these 52 items. At that point, we decided to drop two of the items from our final version of the IDI. Specifically, we dropped item # 29: ‘‘Some cultures are just better’’ from the DD scale because the content of the item does not clearly reflect a focus on the superiority of one’s own primary culture. Therefore, the final version of the IDI has 13 items that comprise the DD scale (rather than the 14 discussed in this paper). The scale reliability for the DD scale remains the same, 0.85. The second item we dropped was # 39: ‘‘Fundamentally, people are the same the world over’’ because of the generality of the item and the overall redundancy of the item with other items in the Minimization scale. Therefore, the final Minimization scale consists of 9 items rather than the 10 items discussed in this paper. The overall reliability of this 9-item Minimization scale is 0.83. Dropping these two items does not create any change in the results on any of the statistical analyses conducted in this study. In short, after validating a 52-item IDI, we simply eliminated two of the items due to further considerations regarding item content clarity. ARTICLE IN PRESS M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 435 Table 1 Descriptive information on 52 items of the final IDI (est. n ¼ 556) Item Mean Std. dev. DD: # 9. DD: # 70. DD: # 92. DD: # 54. DD: # 98. DD: # 106. DD: # 118. DD: # 29. DD: # 51. DD: # 65. DD: # 88. DD: # 99. DD: # 100. DD: # 105. Reversal: # 24. Reversal: # 56. Reversal: # 67. Reversal: # 73. Reversal: # 74. Reversal: # 79. Reversal: # 85. Reversal: # 97. Reversal: # 110. Minimization: # 30. Minimization: # 39. Minimization: # 75. Minimization: # 96. Minimization: # 104. Minimization: # 112. Minimization: # 31. Minimization: # 52. Minimization: # 114. Minimization: # 119. AA: # 13. AA: # 23. AA: # 63. AA: # 66. AA: # 115. AA: # 33. AA: # 38. AA: # 48. AA: # 44. AA: # 58. AA: # 37. AA: # 84. AA: # 120. Cultural Marginality: Cultural Marginality: Cultural Marginality: Cultural Marginality: Cultural Marginality: 1.33 1.76 2.19 1.41 1.34 1.18 1.95 1.67 1.53 1.42 1.40 1.55 1.51 1.76 1.91 1.49 2.11 1.49 1.73 1.87 2.09 2.62 2.05 2.89 3.14 4.12 3.47 3.27 3.55 2.90 3.00 3.31 3.00 3.72 3.88 3.76 3.64 3.49 3.17 3.18 3.16 2.68 3.62 3.18 2.83 2.94 1.61 1.56 1.90 2.62 2.09 0.79 1.15 1.23 0.85 0.78 0.53 1.11 1.07 0.98 0.85 0.81 0.98 0.98 1.04 1.07 0.93 1.20 0.85 1.03 1.11 1.21 1.37 1.19 1.38 1.35 1.11 1.25 1.31 1.26 1.18 1.34 1.13 1.24 1.23 1.19 1.19 1.15 1.22 1.23 1.17 1.31 1.14 0.99 1.08 1.32 1.25 1.00 1.00 1.27 1.43 1.31 # # # # # 5. 36. 60. 68. 69. ARTICLE IN PRESS 436 M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 10. Testing the validity of the IDI Validity is concerned with whether a measurement procedure measures what it claims to measure (Emmert & Barker, 1989). Both content and construct validity of the IDI was addressed. Content validity focuses on the systematic sampling of the universe of items related to the construct being measured (Emmert & Barker, 1989). The content validity of the IDI was addressed in Phase 1 through the in-depth interviews conducted with people from a variety of cultures. Because these interviews were transcribed, we were able to identify a wide range of statements or items from this sample related to the various orientations toward cultural differences identified in the DMIS. Content validity was also addressed by the use of raters and the panel of experts who, in Phase 1, rated each of the generated items in terms of the developmental orientations as described in the DMIS. Because a substantial number of these original items were employed in the Phase 2 research effort, we believe that the interrater reliabilities calculated for these item evaluations continue to provide evidence for the content validity of the items vis-a" -vis the DMIS theory. To establish construct validity for the IDI, we compared the scales to two other related constructs. As Emmert and Barker (1989) state: ‘‘If a researcher has developed a measure of a construct, the scores resulting from that measure should relate in a theoretically meaningful manner to other variables with which the construct is supposed to be connected’’ (p. 114). In order to test the construct validity of the IDI, we examined the relationship of the respondents’ scores for the DD, R, M, AA, and EM scales to two theoretically related variables: Worldmindedness and Intercultural Anxiety. The first measure, an assessment of international attitudes, was a 6-item version of the Worldmindedness scale (Sampson & Smith, 1957) employed by Wiseman, Hammer, and Nishida (1989).7 The Worldmindedness scale has been found to possess strong reliability (0.93 split-half reliability and 0.93 for test–retest reliability) and validity (0.71 correlation with an 11-item Ethnocentrism scale (Shaw & Wright, 1967)). The second measure was a modified version of the Social Anxiety scale developed by Stephen and Stephen (1985). This modified (Intercultural Anxiety) version focuses on the degree of anxiety respondents experience when interacting with people from cultures different than their own. Respondents indicate how they feel overall when interacting with people from other cultures (on a 1–7 scale, less or more anxious, comfortable, accepted, irritated, awkward, impatient, defensive, suspicious, 7 Specifically, the following six items comprised this scale: (1) Our country is probably no better than many others, (2) It would be better to be a citizen of the world than of any particular nation, (3) Our responsibility to people of other races ought to be as great as our responsibility to people of our own area, (4) Any healthy individual, regardless of race or religion, should be allowed to live wherever she/he wants to in the world, (5) Our schools should teach history of the world rather than our own nation, and (6) Our country should permit the immigration of foreign peoples even if it lowers our standard of living. Response options ranged from strongly disagree to strongly agree (1=strongly disagree; 2=disagree; 3=slightly disagree; 4=slightly agree; 5=agree; 6=strongly agree). The higher the score, the more worldminded the response. ARTICLE IN PRESS M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 437 self-conscious, careful, and nervous). A number of studies have found this measure to maintain satisfactory reliabilities across cultural contexts (e.g., Gao & Gudykunst, 1990; Gudykunst, 1989; Hammer, Wiseman, Rasmussen, & Bruschke, 1998). Based on the initial findings provided by Paige et al. (1999), AA scores (which empirically relate to the Acceptance, Cognitive Adaptation, and Behavioral Adaptation scales on the earlier 60-item IDI and theoretically relate to the Acceptance and Adaptation orientations identified in the DMIS) should positively correlate with higher scores on the Worldmindedness measure and negatively correlate with scores on the Intercultural Anxiety instrument. In contrast, DD scores (which conceptually relate to the Denial and Defense scales on the earlier 60-item IDI employed by Paige et al. (1999) and to the same theoretical dimensions identified in the DMIS) should negatively correlate with the Worldmindedness measure and positively correlate with the Intercultural Anxiety scale. Further, because Paige et al. found that Minimization empirically existed between these two dimensions, M scores should not substantially correlate with either the Worldmindedness or the Intercultural Anxiety measures. Because it was not included in the original 60-item IDI, Reversal was not studied by Paige et al. (1999). Nevertheless, we can make a theoretical prediction about how the R scale scores will correlate with Worldmindedness and Intercultural Anxiety. Reversal typically involves an initial encounter with a different cultural system where the usual tendency to evaluate the other culture negatively is reversed, yielding a judgment of superiority of the other culture compared to the inferiority of one’s own culture. In this case, one’s ability to relate ethnorelatively to the new cultural system is compromised because of the essentially evaluative nature of the comparison that is made between one’s home culture and the host culture. Because the other culture is evaluated positively, contact with it does not elicit anxiety. However, because people with Reversal worldviews do not really shift perspective or generate adaptive behavior, their lack of anxiety is not accompanied by a higher level of intercultural competence. This should mean that the R scale will not be significantly positively or negatively related to either Worldmindedness or to Intercultural Anxiety. In contrast, Encapsulated Marginality (EM) should positively correlate with both the Worldmindedness scale and the Intercultural Anxiety measure. This should occur because respondents who are experiencing this aspect of Integration are more ethnorelative in their cognitive and behavioral capabilities for adapting to cultural differences, yet they are also experiencing affective difficulties vis-a" -vis their own sense of cultural identity. Before validity testing of the IDI scales with the Worldmindedness scale and the Intercultural Anxiety scale was undertaken, internal consistency (coefficient alpha) was computed for the 6-item Worldmindedness scale and the 11-item Intercultural Anxiety scale. Reliability (coefficient alpha) for the Worldmindedness scale was 0.67 (n ¼ 561) and 0.86 for the Intercultural Anxiety scale (n ¼ 571). Correlations run on the Worldmindedness scale and the Intercultural Anxiety scale revealed an expected small, negative correlation between these two measures (r ¼ 0:14; p ¼ 011; n ¼ 317). This correlation reveals that the measures, while slightly related, do ARTICLE IN PRESS 438 M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 Table 2 Correlation coefficients between five IDI scales and the worldmindedness and intercultural anxiety scales Correlation coefficients estimated (n ¼ 553; p ¼ 0:01) IDI scale DD scale Reversal Minimization scale AA scale Cultural marginality scale Worldmindedness scale 0.29 Ns Ns 0.29 0.12 Intercultural anxiety scale 0.16 Ns Ns 0.13 0.14 nevertheless, measure substantively different domains related to the intercultural arena. Table 2 presents the correlations comparing the Worldmindedness and Intercultural Anxiety scale scores to the following IDI scales: DD scale, R scale, M scale, AA scale, and the EM scale. Significant (p ¼ 0:01) negative correlations were found between Worldmindedness and the DD scale (r ¼ 0:29; n ¼ 537) and significant positive correlations (r ¼ 0:29; n ¼ 523) with the AA scale. As predicted, R scores were not significantly related to Worldmindedness scale responses. M scores were also not significantly related to Worldmindedness while EM scale scores were significantly positively related (r ¼ 0:12; n ¼ 544) to Worldmindedness scores. Also as predicted, a significant positive correlation was observed between Intercultural Anxiety and the DD scale (r ¼ 0:16; n ¼ 543). No significant correlations were found between Intercultural Anxiety and R or M scale scores. A significant, negative correlation was found between Intercultural Anxiety and the AA scale (r ¼ 0:13; n ¼ 527) and a significant positive correlation was observed between Intercultural Anxiety scores and EM (r ¼ 0:14; n ¼ 555). Overall, these results confirm the theoretically postulated relationships among the IDI scales and the two validation measures. Specifically, higher scores on the DD scale (which indicate a stronger Denial and Defense orientation) are related to less Worldmindedness and greater Intercultural Anxiety. Greater intercultural competence as reflected in higher AA scores (which indicate a stronger Acceptance and Adaptation orientation) is significantly related to more Worldmindedness and decreased Intercultural Anxiety (i.e., greater comfort interacting with people from different cultures). Higher scores on the EM scale (which reflect a greater sense of cultural rootlessness) are related to significantly higher levels of Worldmindedness and also increased Intercultural Anxiety. Overall then, the DD scale, the AA, and the EM scales were significantly correlated in the direction hypothesized with both the Worldmindedness and Intercultural Anxiety measures. The M scale (which measures the transition between DD and AA) was not related, as predicted, to either Worldmindedness or Intercultural Anxiety. And, as predicted for different reasons, the R scale was also not correlated with either the Worldmindedness or the Intercultural Anxiety measure. ARTICLE IN PRESS M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 439 11. Testing for gender, age, education, and social desirability Additional tests were run on the IDI, examining the effects of gender, age, education, and social desirability on IDI scale scores. T-tests were run on each of the IDI scales by gender. No significant differences were found on four of the five measures. Significant differences were found on the DD scale (t ¼ 4:84; df ¼ 553; po0:01) with the Male mean (1.73) significantly higher than the female mean score (1.46). However, because no such effect was systematically observed across the other four scales, it would appear that the IDI is not systematically influenced by gender differences. In order to test for significant differences on the IDI scale scores by age and education level, analyses of variance (ANOVAs) were computed. No significant differences (p ¼ 0:01) were found on any of the IDI scales. In order to examine possible effects of social desirability, the short form (10-item) Marlowe–Crown social desirability scale (Strahan & Gerbasi, 1972) was included in the questionnaire completed by the 591 respondents in the sample. Correlations between social desirability and all five IDI scales revealed no significant differences, meaning IDI scale scores did not appear to be influenced by any general tendency for respondents to provide socially desirable responses. This finding is particularly notable, since tests of intercultural competence are often casually criticized as being ‘‘transparent.’’ The IDI has addressed this concern in terms of social desirability. 12. Conclusion Overall, the results of the confirmatory factor analysis completed as part of the development of a revised IDI instrument indicate that a five-factor solution (DD, R, M, AA, and EM scales) provides a good fit to the data. Further, in a direct comparison of the five-factor solution with both the original, seven-dimensional model of intercultural sensitivity proposed by Bennett (1986, 1993b) and a twodimensional, more global model (of ethnocentrism and ethnorelativism), the fivefactor solution was found to be superior. Additional testing done on these five scales on gender, age, education level, and social desirability reveals no significant effects by age, education level, or social desirability and no significant effects on four of the five scales by gender. The better fit of the five-factor solution is consistent with the initial findings of Paige et al. (1999) regarding the original IDI. By factor analyzing all 60 of the original items together, they had observed that Denial and Defense did not emerge as distinct factors, but appeared to constitute only one factor. Their testing also showed Acceptance and Adaptation constituting a single factor. In the revised IDI, these combinations are reflected in the DD and AA scales, respectively. Analysis of the revised IDI data also supports the initial finding of Paige et al. (1999) that Minimization occupies a position between Denial/Defense and Acceptance/Adaptation. An interpretation of this is that the DMIS Minimization worldview represents a resolution of issues generated by a Denial/Defense ARTICLE IN PRESS 440 M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 orientation but does not incorporate the recognition of fundamental shift in cultural frame of reference necessary to the Acceptance/Adaptation worldview perspective. A new finding from the analysis of the revised IDI responses is that, contrary to DMIS theoretical expectation, Reversal empirically comprises a separate orientation toward cultural differences. This probably means that the positive evaluation of another culture within a polarized worldview is experienced differently than is the negative evaluation of another culture within a similarly polarized worldview. Further research on the R scale of the revised IDI may illuminate how this different experience is theoretically related to ethnocentrism in general. DMIS expectations were supported by the confirmatory factor analysis and subsequent analyses that identified Encapsulated Marginality as a fifth empirically distinct factor. EM appears to be, as expected, a form of ethnorelativism more advanced than Acceptance or Adaptation (AA). Also as expected, the encapsulated form of Integration is associated with anxiety. We hope that the next version of the IDI will be able to add the constructive form of Integration to the scales, allowing us to measure more advanced ethnorelativism that is not associated with intercultural anxiety. In order to facilitate additional analyses of the IDI by other researchers, an effort was undertaken (after establishing the validity and reliability of this 50-item IDI), based on an additional data set of 766 respondents who completed the 50-item IDI, to develop a ‘‘total IDI score’’ grounded in the DMIS theory such that lower scores reflect more ethnocentric orientations and higher scores reflect more ethnorelative worldviews (i.e., Acceptance and Adaptation). Specifically, the DD, R, M and AA scale scores were incorporated into a formula that produced a standardized (z-score) ‘‘total IDI score’’ with a mean of 100 and a standard deviation of 15 (because the EM scale is theoretically viewed as an incomplete measure of the ‘‘Integration’’ stage in the DMIS theory, it was not incorporated into the calculation of the total IDI score). In this study, our goal was to develop valid and reliable measures of intercultural sensitivity guided by the Developmental Model of Intercultural Sensitivity. This research produced a set of measures that both comprehensively assesses developmental constructs and also provides empirical evidence that could support construct revision to the developmental theory. The purpose of this study was not to test the DMIS theory proposed by Bennett. However, we believe further work undertaken to more formally test the DMIS is aided by the research methods used in this study and the instrument that was developed from it. For example, additional work should be undertaken examining the accuracy of the developmental sequence (i.e., from a more Denial/Defense orientation to a Minimization perspective to an Acceptance/Adaptation worldview). As part of this effort, research should also examine the developmental function and ‘‘place’’ in this sequence of Reversal and Encapsulated Marginality perspectives. Other research can also investigate the predictive validity of the IDI insofar as the DMIS theory postulates that individuals with more complex cultural categories are better able to navigate through cultural differences. For example, the IDI scales developed in this study can be employed to empirically examine whether higher ARTICLE IN PRESS M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 441 ‘‘total IDI scores’’ (i.e., more ethnorelativism) predict: (1) less cultural stress among sojourners, (2) more satisfaction with living/working in a foreign culture, (3) greater job accomplishment in culturally different environments, (4) lower levels of prejudice and discrimination against culturally different others, (5) less resistance to diversity initiatives in organizations, and (5) decreased conflict and/or violence toward people from different cultures? In conclusion, we believe the DMIS model is largely supported by testing associated with the development of the IDI. Thus we feel that the final, 50-item IDI can be used with confidence as a measurement of the five dimensions of the DMIS identified in this research. This measurement should be useful for purposes of assessing training needs, guiding interventions for individual and group development of intercultural competence, contributing to personnel selection, and evaluating programs. Acknowledgements We would like to acknowledge and thank the following people who provided valuable support during the development of the IDI. Michael Paige has been instrumental in helping refine our measurement methodology, and has been supportive of our efforts since the very beginning. Janet Bennett and the Intercultural Communication Institute have sponsored the educational effort associated with the IDI. Barbara Kappler gave considerable time in assisting in the expert ratings portion of the research study. We would also like to thank Judith Martin, Shelley Smith, Laura Paulson, Dr. Ray O’Connor has provided valuable statistical assistance in developing the ‘‘total IDI score’’. Rhonda Petri, John Baldwin, and Linda Larkey for their help as ‘‘expert raters.’’ We would also like to express our gratitude to the following graduate students from American University who conducted initial interviews for us: Sabina Behague, Jessica Gilbert, Karen Rosenthal, Jessica Vesper, Tracy Weber, Neity Bhavnani, Susan Finnegan, and Kavita Munjal. References Adler, N. (1991). International dimensions of organizational behavior (2nd ed.). Boston: PWS-Kent. Banks, J. A. (1988). Multiethnic education: Theory and practice (2nd ed.). Boston: Allyn and Bacon. Bennett, J. M. (1993a). Cultural marginality: Identity issues in intercultural training. In R. M. Paige (Ed.), Education for the intercultural experience (pp. 109–136). Yarmouth, ME: Intercultural Press. Bennett, M. J. (1986). Towards ethnorelativism: A developmental model of intercultural sensitivity. In R. M. Paige (Ed.), Cross-cultural orientation: New conceptualizations and applications (pp. 27–70). New York: University Press of America. Bennett, M. J. (1993b). Towards ethnorelativism: A developmental model of intercultural sensitivity. In R. M. Paige (Ed.), Education for the intercultural experience (pp. 21–71). Yarmouth, ME: Intercultural Press. Bhawuk, D. P. S., & Brislin, R. (1992). The measurement of intercultural sensitivity using the concepts of individualism and collectivism. International Journal of Intercultural Relations, 16(4), 413–436. ARTICLE IN PRESS 442 M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 Black, J. S. (1990). The relationship of personal characteristics with the adjustment of Japanese expatriate managers. Management International Review, 30, 119–134. Black, J. S., & Mendenhall, M. (1990). Cross-cultural training effectiveness: A review and theoretical framework for future research. Academy of Management Review, 15, 113–136. Black, J. S., Gregersen, H. B., & Mendenhall, M. E. (1992). Global assignments. San Francisco, CA: Jossey-Bass. Brennan, R. L., & Prediger, D. J. (1981). Coefficient kappa: Some uses, misuses, and alternatives. Educational and Psychological Measurement, 41, 687–699. Brislin, R. W. (1981). Cross-cultural encounters: Face-to-face interaction. Elmsford, NY: Pergamon. Brislin, R. W., Cushner, K., Cherie, C., & Yong, M. (1986). Intercultural interactions: A practical guide. Beverly Hills, CA: Sage. Brown, G. S. (1972). Laws of form. Toronto Ontario, Canada: Bantam. Browne, M. W., & Cudeck, R. (1989). Single sample cross-validation indices for covariance structures. Multivariate Behavioral Research, 24, 445–455. Byrne, B. M. (1989). A primer of LISREL: Basic applications and programming for confirmatory factor analytic models. New York: Springer-Verlag. Cleveland, H., Mangone, G., & Adams, J. (1960). The overseas Americans. New York: McGraw-Hill. DeVellis, R. F. (1991). Scale development. Thousand Oaks, CA: Sage. Emmert, P., & Barker, L. L. (1989). Measurement of communication behavior. New York: Longman. Fleiss, J. I. (1981). Statistical methods for rates and proportions. New York: Wiley. Gao, G., & Gudykunst, W. B. (1990). Uncertainty, anxiety and adaptation. International Journal of Intercultural Relations, 5(4), 301–317. Gardenswartz, L., & Rowe, A. (1993). Managing diversity: A complete desk reference and training guide. Homewood, IL: Business One Irwin. Glaser, B., & Strauss, A. (1967). The discovery of grounded theory. Chicago: Aldine. Gudykunst, W. B. (1989). Culture and the development of interpersonal relationships. In J. A. Anderson (Ed.), Communication yearbook (pp. 315–354). Newbury Park, CA: Sage. Hammer, M. R. (1989). Intercultural communication competence. In M. K. Asante, & W. B. Gudykunst (Eds.), The handbook of international and intercultural communication (pp. 247–260). Newbury Park, CA: Sage. Hammer, M. R., & Bennett, M. J. (1998). The Intercultural Development Inventory (IDI) manual. Portland, OR: The Intercultural Communication Institute. Hammer, M. R. (1999a). Cross-cultural training: the research connection. In S. M. Fowler, & M. G. Mumford (Eds.), The intercultural sourcebook, Vol. 2 (pp. 1–18). Yarmouth, ME: Intercultural Press. Hammer, M. R. (1999b). A measure of intercultural sensitivity: The Intercultural Development Inventory. In S. M. Fowler, & M. G. Mumford (Eds.), The intercultural sourcebook, Vol. 2. (pp. 61–72). Yarmouth, ME: Intercultural Press. Hammer, M. R., Wiseman, R. L., Rasmussen, J. L., & Bruschke, J. C. (1998). A test of anxiety/ uncertainty management theory: The intercultural adaptation context. Communication Quarterly, 46, 309–326. Hawes, F., & Kealey, D. J. (September 1979). Canadians in development: An empirical study of adaptation and effectiveness on overseas assignment. Ottawa, Ontario. Communication Branch Briefing Center, Canadian International Development Agency. Hawes, F., & Kealey, D. J. (1981). An empirical study of Canadian technical assistance. International Journal of Intercultural Relations, 5(3), 239–258. . . Joreskog, K. G., & Sorbom, D. (1984). LISREL-VI user’s guide (3rd ed.). Mooresville, IN: Scientific Software. Kealey, D. J. (1989). A study of cross-cultural effectiveness: Theoretical issues, practical applications. International Journal of Intercultural Relations, 13, 387–428. Kealey, D. J. (1996). The challenge of international personnel selection. In D. Landis, & R. S. Bhagat (Eds.), Handbook of intercultural training (2nd ed.) (pp. 81–105). Thousand Oaks, CA: Sage. ARTICLE IN PRESS M.R. Hammer et al. / International Journal of Intercultural Relations 27 (2003) 421–443 443 Kealey, D. J., & Ruben, B. D. (1983). Cross-cultural personnel selection criteria, issues, and methods. In D. Landis, & R. W. Brislin (Eds.), Handbook of intercultural training, Vol. 1: Issues in theory and design (pp. 155–175). New York: Pergamon. Kelly, G. (1963). A theory of personality: The psychology of personal constructs. New York: Norton. Klineberg, O., & Hull, F. (1979). At a foreign university. New York: Praeger. Landis, D., & Bhagat, R. S. (Eds.) (1996). Handbook of intercultural training (2nd ed.). Thousand Oaks, CA: Sage. Landis, D., & Brislin, R. W. (Eds.) (1983a). Handbook of intercultural training, Vol. I: Issues in theory and design. New York: Pergamon. Landis, D., & Brislin, R. W. (Eds.) (1983b). Handbook of intercultural training, Vol. II: Issues in training methodology. New York: Pergamon. Landis, D., & Brislin, R. W. (Eds.) (1983c). Handbook of intercultural training, Vol. III: Area studies in intercultural training. New York: Pergamon. Maturana, H., & Varela, F. (1987). The tree of knowledge: The biological roots of human understanding. Boston: Shambhala. Neuliep, J. W., & McCroskey, J. C. (1997). The development of a US and generalized Ethnocentrism scale. Communication Research Reports, 14(4), 385–398. Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill. Paige, R. M., Jacobs-Cassuto, M., Yershova, Y., & DeJaeghere, J. (April 1999). Assessing intercultural sensitivity: A validation study of the Hammer and Bennett (1998) Intercultural Development Inventory. Paper presented at the International Academy of Intercultural Research conference, Kent State University, Kent, OH. Parham, T. A. (1989). Cycles of psychological Nigrescence. The Counseling Psychologist, 17(2), 187–226. Pederson, P. (1998). Intercultural sensitivity and the early adolescent. Unpublished doctoral dissertation, University of Minnesota, Minneapolis, MN. Perry, W. B. (1970). Forms of intellectual and ethical development in the college years: A scheme. New York: Holt. Sampson, D. L., & Smith, H. P. (1957). A scale to measure world-minded attitudes. Journal of Social Psychology, 45, 99–106. Shaw, M. E., & Wright, J. M. (1967). Scales for the measurement of attitudes. New York: McGraw-Hill. Stephen, W. G., & Stephen, C. W. (1985). Intergroup anxiety. Journal of Social Issues, 41, 157–176. Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park, CA: Sage. Strahan, R., & Gerbasi, K. C. (1972). Short homogenous version of the Marlowe-Crowne Social Desirability Scale. Journal of Clinical Psychology, 28(2), 191–193. Tower, K. (1990). Intercultural sensitivity survey: Construction and initial validation. Unpublished doctoral dissertation. University of Iowa, Iowa City, IA. Von Foerster, H. (1984). On constructing a reality. In P. Watzlawick (Ed.), The invented reality: Contributions to constructivism (pp. 41–61). New York: Norton. Wiseman, R. L., Hammer, M. R., & Nishida, H. (1989). Predictors of intercultural communication competence. International Journal of Intercultural Relations, 13(3), 349–370.