Strategic Framing: How Leaders Craft the Meaning of Data Use

429295 EPXXXX10.1177/0895904 811429295Park et al.Educational Policy © The Author(s) 2012 Reprints and permission: sagepub.com/journalsPermissions.nav Strategic Framing: How Leaders Craft the Meaning of Data Use for Equity and Learning Educational Policy XX(X) 1–31 © The Author(s) 2012 Reprints and permission: sagepub.com/journalsPermissions.nav DOI: 10.1177/0895904811429295 http://epx.sagepub.com Vicki Park1, Alan J. Daly1, and Alison Wishard Guerra1 Abstract Although there is an emerging body of research that examines data-driven decision making (DDDM) in schools, little attention has been paid to how local leaders strategically frame sensemaking around data use. This exploratory case examines how district and school leaders consciously framed the implementation of DDDM in one urban high school. Leaders strategically constructed diagnostic, motivating, and prognostic frames to promote a culture of using data for continuous improvement. Our findings demonstrate that leaders developed (a) diagnostic frames centered on the need to confront student achievement and opportunity gaps; (b) motivating frames concentrated on school improvement as shared collective responsibility; and (c) prognostic frames focused on making incremental change to sustain reform efforts and the creation of common goals to monitor progress. The findings suggest that framing is an important leadership tactic that needs careful consideration when reforms are introduced and implemented. Keywords implementation, urban schools, qualitative research, educational reform, leadership, organizational behavior, educational policy, school accountability, school capacity, school districts 1 University of California, San Diego, CA, USA Corresponding Author: Vicki Park, Education Studies, University of California, San Diego, Pepper Canyon Hall, 3rd Floor, 9500 Gilman Drive #0070, La Jolla, CA 92093-0070 Email: [email protected] Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 2 Educational Policy XX(X) Introduction In concert with accountability policies, education policy makers and researchers have emphasized the use of data as a key strategy for system improvement. Data-driven decision making (DDDM), which generally means to systematically gather and analyze data to inform decision making (Earl & Katz, 2006; Marsh, Pane, & Hamilton, 2006), has flourished as a pivotal tool for education reform.1 This growing importance is supported by a publishing industry that produces “how-to” books and articles for practitioners on using data (Bernhardt, 2004; Celio & Harvey, 2005); research that focuses on examining district conditions that support the use of data and evidence (Datnow, Park, & Wohlstetter, 2007; Coburn & Talbert, 2006; Honig & Coburn, 2008; Kerr, Marsh, Ikemoto, Darilek, & Barney, 2006; Togneri & Anderson, 2003); and research on school-level implementation and its associated challenges (Diamond & Cooper, 2007; Herman & Gribbons, 2001; Ingram, Louis, & Schroeder, 2004; Lachat & Smith, 2005; Supovitz & Klein, 2003; Young, 2006). Although there is an emerging body of research that examines DDDM in schools (Lachat & Smith, 2005; Marsh et al., 2006), research on its implementation has largely focused on the technical and structural dimensions of data use with less attention paid to how local leaders strategically frame sensemaking around DDDM and attempt to create a culture of continuous improvement. As Ingram et al. (2004) have argued, research on school change and policy implementation tends to overemphasize practices and behaviors, such as data use, and neglects the importance of changing school culture— particularly the process of sensemaking around school reform. These scholars assert that changes in practice will consistently fail without a simultaneous attempt to change tacit beliefs and assumptions held by educators. The culture of schools and districts need to be considered in the implementation of DDDM as they shape how teachers and administrators make sense of and ultimately use data for decision making (Datnow, Park, & Kennedy, 2008; Coburn, Toure, & Yamashita, 2009; Ingram et al., 2004). The purpose of this exploratory case study is to understand how strategic framing of data use by formal leaders mediates the implementation of DDDM. Specifically, we employ frame analysis as a theoretical guide to understand deliberate sensemaking efforts aimed at fostering a culture of productive data use. We examine the types of frames that were created and developed and the degree to which those frames appeared to support shifts in beliefs in one urban high school. By focusing on these strategic frames, we underscore the active role district- and school-level leaders and practitioners play in constructing meaning when implementing data use. Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 3 Park et al. Conceptual Framework We draw on two bodies of literature to understand how formal leaders implement DDDM: sensemaking of school reform and frame analysis. Sensemaking is generally defined as an active and dynamic process by which individuals and groups make meaning of experiences and ideas (Weick, Sutcliffe, & Obstfeld, 2005). Sensemaking is premised on the assumptions that people actively construct knowledge; learning is an interactive process situated in specific contexts; and people’s active interpretation of school improvement matters because it orients their actions (Coburn, 2006). Research in this vein increasingly places local actors at the forefront of implementation efforts as active agents and highlights the process by which they interpret, adapt, or transform policy (Coburn, 2001; Spillane, Reiser, & Reimer, 2002). Such studies have provided key insights into how individuals interpret reform differently based on their roles, backgrounds, and contexts. To date, research on the intersection of school reform and sensemaking has focused on how teachers and administrators interpret school reform in myriad of ways based on their social location (Spillane et al., 2002). However, less attention has been paid to the process by which local actors strategically influence the sensemaking of others on DDDM though the use of framing. Although sensemaking theory has been helpful in pushing the field to consider the importance of meaning-making activities in policy implementation, the theory is limited as it stresses taken-for-granted cognitive processes and tends to downplay other aspects of human relationships, including the dynamics of power and ideology (Weber & Glynn, 2006). Differential access to decision-making positions, as well as resources, means that some social actors have more power to shape social reality. Those in power have more opportunities to leverage and regulate behavior by shaping what is valued or discounted and what is privileged or suppressed (Coburn, 2006; Firestone, Fitz, & Broadfoot, 1999). Thus, educators in position of formal and informal authority can influence how others make meaning of reform efforts. Formal leaders, given their position in the power structure, often have the authority to guide or direct sensemaking processes. Complementing studies on sensemaking but emphasizing efforts to create shared meaning and action, frame analysis provides an analytic architecture to examine how social actors may prime, trigger, or edit sensemaking within the organization. Research on framing has investigated how social actors engage in strategic sensemaking activities to coordinate action, including how problems and solutions are constructed and what motivations are used to persuade people to act (Benford & Snow, 2000). Assuming agency on the Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 4 Educational Policy XX(X) part of social actors, the emphasis is not only on how people interpret reform based on their social location within the frames but also on how the sensemaking process functions to facilitate agreement or collective action (Benford & Snow, 2000; Campbell, 2005). Frame analysis suggests that the degree to which people buy in to the credibility and salience of a movement is dependent on the extent to which three core framing tasks are articulated and shared (Benford & Snow, 2000). The first is diagnostic framing. When involved in diagnostic framing, social actors focus on defining the problem, assigning blame and/or responsibility, and suggest attributions (e.g., good vs. bad). The second framing task is prognostic framing. This type of framing involves an articulation of how the problem may be solved, including strategies for achieving goals. Finally, the third task is motivating framing in which the rationale for action is articulated. Whether frames motivate action or cause shift in beliefs depend on the degree to which they are compelling to individuals (i.e., what frame analysts refer to as resonance). Overall, framing works as a tool to organize experiences by giving them meaning but frame analysts also suggest that it is a contested and negotiated process, mediated by relationships of authority and power. Although the larger institutional context has framed new policies around equity and accountability, how DDDM is framed at the local level and how this framing shapes educators’ sensemaking is largely left unexplored. This is a crucial area that needs to be investigated because how local actors bridge the existing institutional frames and organizational frames on the ground will likely determine the degree of participant buy-in and implementation. Yet, except for a handful of exceptions (e.g., Coburn, 2006; Coburn et al., 2009; Davies, 2002; Grossman, 2010), the framework has not been used to understand educational change, especially at the local level.2 Rarely have previous approaches examined how those in formal positions of power and influence at local levels structure sensemaking on the implementation of DDDM, thereby mediating how individuals define and interpret data use. In exploring this gap, this study contributes to the existing literature on school reform by using frame analysis to explore (a) how leaders construct meaning around the implementation of DDDM, (b) how they frame that meaning for others, and (c) how teachers make sense of data use within those frames. For the purpose of this article, we focus on how those in formal positions of leadership at the district and school levels frame DDDM, although acknowledging that framing tasks can be constructed and undertaken by various actors associated with a social movement. Given the policy context, districts and schools may not focus on using data for continuous improvement but use it only to fulfill policy mandates Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 5 Park et al. (Booher-Jennings, 2005; Diamond & Cooper, 2007). As the emphasis on data is situated within a high-stakes accountability framework, teachers may view the use of data as suspect. In a nationwide study of secondary school teachers, data were perceived as a sanctioning tool used to fit predetermined decisions rather than information used to shape the decision-making process (Ingram et al., 2004). At the district level, Firestone and Gonzalez (2007) suggest that there may be two types of contrasting sensemaking approaches. One approach, which they refer to as an accountability culture, focuses on student test scores, tends to have a short-term time frame, and excludes teacher and principal voices. Data are used mainly to identify problems and monitor compliance. In contrast, a second approach, which they label as the organizational learning culture, emphasizes student learning and instructional improvement, is long term in scope and includes teacher and principals’ voices. The differences in cultures reflect whether districts are engaged in meaningful continuous improvement efforts or are merely chasing numbers to avoid sanctions. How do these different implementation cultures come about regarding data use? What is the role of leaders in shaping these different orientations? And, how do data come into play? School leaders have strong voices in the construction of policy messages: They can shape where and how sensemaking happens, they can frame policy messages and its interpretation, and they can provide material support (Coburn, 2006). The ways in which district and school leaders engage in framing tasks—diagnostic, prognostic, and motivating—will likely shape how teachers make sense of data use and the degree to which teachers believe it as a meaningful strategy for school improvement. Furthermore, how leaders frame DDDM is likely to influence the type of culture that is created around data use. One way they may do so is by engaging in deliberate framing of DDDM as an essential and useful strategy for continuous learning versus framing it as a way to solely meet accountability demands. Method and Data Analysis Our aim in this study was to explore how school and district leaders framed DDDM and the degree of resonance the frames had for teachers. We chose to use case study method (Yin, 2003) for two main reasons: (a) a desire to investigate the content and process of framing on DDDM implementation rather than outcomes and (b) an interest in understanding how local actors construct and reconstruct meaning around the implementation of DDDM. Thus, the unit of study was narrowly focused on how local administrators Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 6 Educational Policy XX(X) Table 1. District and High School Profiles Costa Unified School District (California) Mesa High School 48,700 1,800 1% 30% 1% 54 1% 50% 1% 39% Pacific Islander White (not Hispanic) Free/reduced lunch English language learners 1% 13% 65% 46% 1% 8% 70% 34% Fully credentialed teachers 98% 99% Profile a Total student population Student demographicsb African American (not Hispanic) Asian Filipino Latino/Hispanic Source: California Department of Education. Note: Data are for the 2007-08 school year. a Figures are approximate and rounded to the nearest hundred. b Percentages are approximate and are rounded. and teachers framed and made sense of data use. The goal was to expand the knowledge of the processes of framing of DDDM as it plays out in a given setting and context. Our intent is not to minimize the importance of structures or behaviors but to foreground the often undertheorized process of how local educators engage in changing beliefs and the culture on data use. Description of Sites and Context This exploratory case study drew on data from a larger investigation of the implementation of DDDM for instructional improvement across the United States (Datnow, Park, & Kennedy, 2008).3 Mesa High School is a comprehensive high school and is part of Costa Unified School District (Costa USD), a large urban district located in California (see Table 1 for district and school demographic profiles).4 We chose this site purposively because district and site leaders were focused on creating cultural shifts on data use. Before delving into how leaders framed data use, it is important to understand the broader context of these sites, especially as previous research indicates the Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 7 Park et al. existing school and district capacities influence an organization’s response to accountability data (Diamond & Spillane, 2004). Costa viewed DDDM as a pivotal element of continuous improvement and the district devoted attention, time, and resources to supporting district personnel to use data. This included systemwide interim assessments aligned to state curriculum standards; an accessible web-based data management system to be used by teachers and administrators; biweekly teacher collaboration time to examine and use data; and support staff to aid data use. Leaders also encouraged teachers to use a range of assessments of student learning for formative purposes in planning their instruction. As part of Costa USD, Mesa High School is seen by district administrators as a leader in education reform, especially with respect to DDDM. Academically, Mesa ranks as a top school within the state and received an Academic Performance Index (API) score of 763 (out of 1,000) for the 200708 school year, performing well when compared with its demographically similar counterparts.5 During that year, the school employed 70 full-time faculty members with an average of 13.5 teaching years. Like many high schools, both administrative staff and teachers at Mesa serve in formal leadership roles. Two main leadership bodies direct the school: the administrative staff and the leadership team. Together they establish school-level goals and make day-to-day decisions.6 Data Collection and Analysis The data for this analysis primarily consist of semistructured interviews and focus groups (Creswell, 1998), supplemented by observational data and document review. A total of 27 district and school personnel participated in the study. The research team conducted two rounds of interviews lasting between 1.5 to 2 hr with district personnel, in the spring of 2006 and the fall of 2007 (see appendix for a list of participants, roles, and type of data source). In the fall of 2007, the research team visited Mesa High School to conduct interviews and focus groups with the principal, other on-site administrators, and teachers in addition to observing data-related meetings and classrooms. These interviews lasted between 45 min and 1.5 hr. All interviews were audio-taped and transcribed verbatim. We also gathered and reviewed documents used for implementing decisions based on data (i.e., instructional planning, curricular changes, and policy adjustments). Observations of departmental and leadership meetings related to the use of data were also conducted. Our theoretical framework and goal of understanding the content and process of framing informed our analysis and results. We focused on examining Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 8 Educational Policy XX(X) how leaders strategically engaged in framing tasks in an attempt to produce a shared conceptualization of DDDM and their efforts to shift existing beliefs about data use. Codes and themes were produced based on the core framing tasks outlined in the conceptual framework: diagnostic, motivating, and prognostic. We concentrated on the types of frames those in positions of leadership created, their rationale for doing so, and the degree of resonance it had with other teachers and administrators. Throughout the process, we focused on emerging patterns as well as disconfirming examples and minority viewpoints (Miles & Huberman, 1994). Findings This exploratory case study illustrates how leaders at both the district and school levels created sensemaking frames that persuaded others of the relevance of using data for decision making. Our data suggest that the process of crafting frames and they manner in which they are used are often mutually reinforcing, interdependent, and dynamically build on one another. However, for the purposes of discussion, the frames are presented as three distinct categories. The diagnostic frames that were created centered on the need to confront the student achievement opportunity gap. Part of this framing process included redefining the concept of caring for all students and ensuring academic achievement. In developing motivating frames, those in leadership positions focused on presenting systemwide and student learning improvements as shared collective responsibility. This required that DDDM be framed in ways that supported shifts in beliefs about who owns data and how it should be shared. Finally, the prognostic frames focused on making incremental change to sustain reform efforts and developing common goals to monitor student progress. Diagnostic Framing: Confronting Inequality and the Opportunity Gap For DDDM, the larger accountability context set the staged for some of the frames that were developed at the local level. Costa and Mesa High School’s move toward DDDM resulted from a confluence of factors but was propelled by state and federal accountability policies. The use of data to inform decisions was certainly not new at this stage: District leaders had always collected and analyzed data to make decisions about organizational planning. However, district leaders admitted that they rarely attempted to look at data in a systematic way or in a collaborative context with teachers and administrators. When Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 9 Park et al. California’s accountability system and the No Child Left Behind (NCLB) Act rolled out, district- and site-based leaders decided to view accountability policies as challenges that needed to be tackled by the district as a whole (NCLB, 2001). This was an important diagnostic stance in that accountability policies were going to be treated as an opportunity for growth, not a mere descriptor of the current state of affairs. The superintendent explained, Frankly, when NCLB came in after that [California’s accountability system], we thought that was a good thing too, even though we know there’s problems with it, and we would admit there are some problems, but for us, I think the first thing we did here was embrace all that, in terms of, you know, this is not a bad thing to be held . . . to know where you’re going and to have measures . . . It’s still . . . we still were looking for something that wasn’t just . . . we couldn’t wrangle the numbers, so to speak, that it really would mean something. And so we kind of set some criteria that it had to hold us accountable at a district level, it had to work at a school level, and it had to work at a classroom level, and it had to also work so that it was meaningful to kids and their parents. In support of this point and portraying the depth of the work the district was committed to, the assistant superintendent added, We’ve made the statements that even if the state system goes away, and NCLB goes away, if all that were to go away, this is going to stay in Costa. We will create our own system because this is good and it’s the way our kids get equal access to kids in more affluent areas. We’re their champions. Overall, the district leaders believed that the new accountability system was a positive turn in education reform despite some reservations about how it held schools solely responsible for student results. In starting to diagnose the problem of the achievement gap, Costa’s district leadership took an inquiry stance and reviewed trends in the data. Recall that when leaders engage in developing diagnostic frames, they focus on defining the problem and identifying its attributions. For Costa USD, the problem framing revolved around understanding differences in student outcomes. As a result, conceptualizing the need for DDDM went hand in hand with the need to improve student learning and producing more equitable outcomes. The district leadership asked why some groups of students were Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 10 Educational Policy XX(X) failing if educators believed that all students were capable of achieving high standards. Rather than jumping to solutions or pinpointing the problem, the district leadership decided to frame the importance of taking the time to reflect and carefully examine the data. This orientation reflected the importance of making incremental changes to sustain reform efforts. Beyond student outcome data (e.g., test scores and grade point averages) disaggregated by race/ethnicity and socioeconomic status, the district also focused on data that described “opportunities to learn” (McDonnell, 1995). For example, as they examined course placement data, they found some startling trends and raised key questions within their schools about whether all students had equal access to college preparation courses. The superintendent shared, If you were an Asian student, and you scored at basic, your chances of being in a regular college prep course are about 80 percent. If you’re a Hispanic student and your score is at basic, your chances of being in [a college prep course] were probably less than 40 percent. As the superintendent added, “It’s not like we’ve been ignoring the problem, but until we put data behind it, we weren’t able to [get people to address it].” The examination of various types of data led staff to move away from focusing solely on individual schools or student test scores to reflecting on opportunity and access to quality learning. In diagnostically framing results as a shared and collective responsibility that would be used as a chance to reflect and learn, district and site leaders were initially promoting a culture of inquiry over a more directive approach that would mandate use of data. Redefining the concept of caring. As the district attempted to frame the problem as a learning opportunity gap, they still had to grapple with why such a gap existed in the first place. Leaders had to think through potential responses to low achievement by finding a way to refocus attention away from blaming of students, their families, or the teachers themselves. Part of this process required that educators at all levels of the system confront the culture of low student expectations that pervaded their secondary schools. The diagnostic framing task included an acknowledgement of the district’s low academic rigor and opportunities, especially for students of color and low-income backgrounds. However, topics on student backgrounds in relationship to achievement can be highly charged and littered with pitfalls because any attempts to diagnose a problem includes describing the nature of the problem and attributing blame. Rather than accusing anyone in the system of intentional harm or dereliction of duty, the focus was on using the data as Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 11 Park et al. information to ensure that student placement was more equal. In other words, part of the diagnostic framing task required leaders to strategize about ways to refocus the attention away from personal intentions to actual outcomes. The superintendent noted, Now, [it’s] not mean spirited people, not a horrible system designed to discriminate, but regardless of your intentions, the results were the same. And it’s all about expectations and perception, and that’s where data has become so valuable, it’s the leveler to say, if you are a basic or above, you should be put in A-G [college preparatory] classes. Data highlighting unequal access to college-preparation classes provided a launching pad for the district to have discussions between staff members about their responsibilities and expectations for students. These types of conversations were considered to be very valuable as they provided formal opportunities to examine student data and discuss the fact that the majority of students who were Latino and came from low-income backgrounds were not being successfully served in the district. Rather than assigning personal blame, Costa’s diagnostic framing of achievement as low academic expectations appealed to the idealistic and moral values of educators. The problem, as it was framed, was not due to a lack of compassion or sense of obligation to students. There was already a sense that teachers cared deeply about students’ welfare but high academic standards were often missing, especially for low-income students of color. For example, at Mesa, the principal described the school’s previous culture as such: The part about the nurturing kind of environment and the desire for the kids to be here and the affinity they’ve had for their teachers-, that was here definitely. I think as a school culture it was a very nurturing environment, a lot of respect for the school still by the parents, but the expectations were very low on all levels; behavior, all the way from behavior to academics. There was definitely a sense of-, “but these poor babies.” You know what I mean? “They have such difficult circumstances.” Echoing other veteran staff at the school, Luke, an English teacher, described the previous atmosphere: I saw a lot of kids going through here being encouraged to join the military or to go to community college because after all they certainly Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 12 Educational Policy XX(X) wouldn’t have the grades to go to a college or a university, particularly a university right away. Furthermore, although the professional environment was described as collegial, teachers were not known for engaging in instruction-related discussions with peers or promoting high professional standards. By engaging in diagnostic framing that raised the level of conversation to ideals of equity (i.e., equal opportunity to learn) and quality of teaching and learning, the district clarified the nature of the problem as a need for academic rigor and attempted to create a shared sense of purpose around this issue. Thus, the leadership attempted to redefine the concept of caring as part of their efforts to diagnose the problem. At Mesa, teachers suggested that this framing was creating some shifts in beliefs and practices. Several veteran teachers mentioned an increasing emphasis on high academic standards. Tina, who was part of the administrative leadership team and a teacher at Mesa, described the evolution of the school’s culture: It has in so many ways shifted . . . Its really about the kids engagement, you know student engagement . . . I think here at Mesa one of our strengths is that we’re continually asking the teachers to diagnose each class each year and to not necessarily get caught in that well I’ve been teaching this class for twenty years, it’s the same class. The class itself might be the same but your kids are going to be completely different year to year. She added, “A huge part of our vision is to just never get complacent about who our kids are and to never make assumptions about them either.” Over the past several years, the principal noted, “I’ve seen the staff evolve in that now we see we can also be very caring and challenge our students.” However, as we explain in a later section, establishing high academic standards alone does not produce equitable outcomes, especially when deficit frames about student ability and background are not directly challenged. Motivating Framing: Extending Collective Responsibility and Making Data Safe When leaders engage in developing motivating frames, they focus on providing a rationale for action. Although Costa USD and Mesa High staff were beginning to buy into the idea that low academic rigor was a problem, they Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 13 Park et al. still needed a rationale to view DDDM as a central tool for improving teaching and learning. As frame analysis suggests, diagnosing needs and creating strategies to address problems are not likely to be sufficient to promote buyin: Leaders have to articulate a meaningful rationale for the movement so that teachers and others have a stake in the implementation of DDDM. Merely fulfilling accountability demands or meeting the district’s established goals, although often a typical framing strategy, were not going to be a sufficient motivating frame to encourage staff to actually believe in the relevance of data for decision making. When first introduced to data use, teachers and administrators shared that they feared performance results would be used for political purposes or to unfairly evaluate schools. Tackling such fears was important given that one of the qualities of productive DDDM in schools is the capacity to work collaboratively and transparently (Ingram et al., 2004; Marsh et al., 2006). Motivating frames about data use needed to be explicitly addressed given that teachers perceived many of the accountability mechanisms in NCLB, such as meeting annual yearly progress (AYP), to be high-stakes oriented. School leaders believed that it was important to think about how to steer conversations about students, learning, and instruction so that teachers felt empowered toward proactive improvement rather than a fatalistic lack of agency or the belief that they were engaged in bureaucratic hurdles. The district leadership deliberately set out to present DDM not just as an accountability mechanism but also as a necessary strategy for improving teaching and learning. Leaders took on the task of defining the purpose and motivation for data use around the higher learning and teaching objectives. The answer was not immediately obvious: The district and school leaders wrestled with explaining how data use would ensure equitable and rigorous academic opportunities for all students. Realizing the importance of motivating her staff to move beyond viewing data use as an accountability mechanism, the superintendent described the tenor of a conversation she had with her staff: All of a sudden, once we got through [the goal-setting process, we had to ask] what’s the purpose of these goals anyway, just so that we meet proficiency? Well, why do you want to meet proficiency, just so we stay out of AYP and API jail? No, we really want to do this because this is what it’s really going to take for kids to be able to be successful in truly rigorous college prep courses, you know, and to go on. Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 14 Educational Policy XX(X) To reinforce the point that the district wanted to move past meeting accountability mandates, recent conversations turned to making a framing shift between compliance and commitment. The superintendent argued, “Compliance is doing things right, and commitment is doing the right thing. Compliance is you doing the right thing in your classroom [and] commitment is making sure the right thing is going on in every classroom for every kid.” By attending to these motivating framing themes of equity and collective responsibility, the leadership was attempting to foster a culture of sensemaking where teachers took responsibility for the whole school rather than their individual classrooms. Making data safe. A large part of the motivational framing entailed reshaping perceptions about data analysis and beliefs about who “owns” data. From system-level personnel to school-based leaders, there was an agreement that the sharing and use of data needed to be presented in a nonthreatening manner to nurture a culture of learning and improvement. To encourage collaboration and a sense of collective responsibility, school leaders emphasized the need for data sharing. Sharing data—across all levels of the educational system—is a paradigm shift for educators. For districts like Costa, there was little precedent within their own schools for how data were to be shared and used. The leadership teams argued that data had to be safe to make people feel comfortable with them. That is, teachers needed to feel safe as they delved into data analysis so that they could share instructional practices or ask for help. To do so, leaders evoked medical analogies to promote a frame that perceived data as an instructional tool: no more and no less. The superintendent described the sensemaking frame that they shared with teachers and administrators: So that’s part of how we’re trying to embed this . . . you know, build a culture of data is your friend, just like for doctors, lab reports are not a bad thing, I mean, I wouldn’t want them to go take out my kidney guessing, I’d like to make . . . them to do the right test to make sure they’re taking out the right part. We’re trying to build that kind of culture, which is new to us, for all . . . we’ve never had data to work with in a fashion like this. So I really see that coming along, but it takes a lot of talk. Instead of blaming a teacher or a school for poor performance on the tests, the district leaders focused on examining the data. Part of the rationale for “depersonalizing” data was to motivate teachers through assisting them in making objective assessments about their instructional practices while simultaneously protecting Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 15 Park et al. their sense of self-efficacy. Teachers and administrators mentioned that these types of framing shifts helped them view DDDM practices in a favorable light although it did not completely eliminate emotional responses to data. The principal and other school leaders at Mesa High School had similar attitudes about motivating teachers to use data. Data sharing was encouraged and framed as a nonevaluative tool. Teachers were particularly persuaded to use data within a collaborative learning environment with their peers. Intended to be learning opportunities, these meetings had minimal oversight from administrators. Katie, a science teacher, shared how quickly she adjusted to sharing her data with colleagues: Well in the beginning when I first got here it was really strange to sit there and go to these meetings and take what I thought was something that was kind of personal, my kids’ data, and lay it out there in front of everybody to see. I think it only takes once, like one meeting, one collaboration time to understand that really nobody cares. I think that everybody really wanted to do what was better for the kids and most of them realized that we can sit there and say well little Johnny’s so sweet, little Johnny’s so nice, well that doesn’t tell us squat about little Johnny and how he performs. The principal, in particular, was viewed as an important figure in fostering this evolving attitude toward DDDM. Several teachers described the principal’s role in cultivating a learning attitude toward data: I think Donna [the principal] has done a good job of presenting it in a positive light even though teachers have been resistant in the beginning. I mean she’s offered opportunities that if you struggle, here’s your safety net. If you’ve failed, try again. You know she’s done it in a very non-judgmental way and let people get to their levels. (Candice, Science) People are at different levels and Donna is really good about going here’s where we’re going but kind of letting the teachers have a little more ownership of it than just going, you will be! You know she’s really doing the baby steps and letting people first kind of volunteer, I want to be a part of this. Then it kind of starts to infuse throughout the whole campus and then even those naysayers start to go, oh well you know, check it out. (Deanna, English) Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 16 Educational Policy XX(X) Again, learning from data and from one’s errors was a way to make data use nonthreatening. As a firm believer in using leverage points to persuade and move her staff toward implementation, the principal also acknowledged that different teachers were at different stages of understanding and using data. Tina, the Title I coordinator and a member of the leadership team, affirmed that this attitude was born of experience with implementing change at the school level: Unfortunately, when you jump on that bandwagon and you’re requiring everybody to get to the same place, they’re not there. So you’re almost creating resistance because either fear, you know you have a teacher that says, “Oh my gosh that’s not my teaching style, you want me to interact with the kids?” Immediately that fear comes in. It might be the best strategy in the world but they won’t even give it a try. Instead, the leadership provided multiple opportunities to receive training or multiple points of entry to use data. Leaders at the district and school level acknowledged that to motivate others, resistance to change and emotions mattered. When leaders pay attention to the motivating frame of DDDM implementation, they acknowledge that productive change does not simply evolve out of good ideas or practices. The social and relational aspects of change also need to be addressed. In the case of teachers at Mesa High School, the principal and other leaders at the district recognized that emotional reactions such as fear of evaluation or learning something new may create barriers to DDDM. Without directly confronting this aspect of reform, the credibility and the salience of data use may be undermined. Ultimately, educators in this setting had to be persuaded that an idea is not only good but that its benefits outweigh the costs to themselves and their students. Prognostic Framing: Continuous Improvement and Reflective Practice In addition to crafting diagnostic and motivational frames on data use, leaders also engaged in prognostic framing by laying out strategies for achieving their goals in using data to improve teaching and learning. Frame analysis suggests that for reforms to be salient and credible, leaders must pay attention to developing strategies that address problems in a substantive way. The strategies laid out through prognostic framing at Costa and Mesa centered on the continuous improvement approach with an emphasis on persuading staff Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 17 Park et al. the importance of collecting and using data. Costa already had a sensemaking frame that oriented their perspectives about how change and reform sustainability occurs. In thinking about DDDM, they drew on existing frames that centered on gradual implementation, focusing on gaining buy-in at all levels. With an incremental approach to change, they looked beyond test results and decided to develop multiple measures of accountability, especially since they felt that state and federal accountability systems were too narrow. This was viewed as a critical approach to DDDM implementation as leaders noted it was possible for schools merely to “chase the numbers” without doing what was right for students. Believing that data use cannot be mandated, both district and Mesa’s leaders strategically focused on building a critical mass of staff members who viewed DDDM as valuable and necessary. The assistant superintendent explained, “So we just slowly ramped it up . . . but we tried to build in strategies that don’t cause us to scuttle the mission, so to speak, because of the lack of buy-in that get people comfortable with it, and then just keep opening it up a little bit more.” In addition, gaps evidenced by tests were addressed in a manner that invited help from the district leaders such as, “Tell us what tools we’ve given you. They may not be adequate tools. We need to shore them up.” The district was increasingly framing itself as an important support provider to the schools. At the same time, the district sent clear messages about holding schools accountable for their results. The superintendent stated, “If you’re not thirsty for it, we’re still holding you accountable for the results.” Thus, the district coupled a large degree of support with accountability requirements. Developing goals as reflective practice. Although the accountability system was becoming increasingly clear about the definitions of success, failure, and growth, the process of continuous improvement was a “black box” that schools had to unpack. As part of its strategy, the district’s leadership team decided that having a formalized, districtwide goals would ensure that everyone had a common purpose. Having goals is an important aspect of prognostic framing because they direct attention and resources to a shared vision. District leadership decided early on to focus on developing reasonable, objectively measurable indicators of progress because they believed that generalized goals such as “we want all students to become lifelong learners” would not enable the district to assess whether the goals were being met. They also acknowledged that they needed to rethink the goals for their secondary school students. The superintendent reflected, Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 18 Educational Policy XX(X) I mean [in the past] we were using data and thought we were aligned to the right thing, but were really aligned to the wrong things. We were trying to align to the state standards which makes sense, through about the 6th grade. But then you have to switch your alignment and think, what’s your target? You want kids to get the keys to go to CSU [California State University], ideally UC [University of California] and/or some kind of skilled career. You want them to leave with the key that could open any doors. Well, that then what do they [students] really need to have . . . So that’s the big message in the data: It’s only good . . . if you think carefully about what you are trying to achieve. District leaders were also mindful about the fact that their goals needed to be based on the strengths and needs of their unique context rather than taking something from another district. During this process came the critical realization that the district leadership team was ill equipped to write strong goals.7 Overall, the process of goal development took about 3 years before it was formalized across the system. The goal-setting process was considered invaluable because leaders felt that much of the growth and understanding of how to look at data as well as developing a plan for the future arose out of this process. Thus, not only was the establishment of the goals an important tactic to produce a shared sense of purpose within the district but the process of developing the goals was believed to be a pivotal sensemaking strategy that enabled staff to build the capacity to achieve their goals. As the assistant superintendent shared, We’ve had other districts want to get copies of our goals and just adopt them as their own, but what they don’t realize was all of the growth that took place in the process of building those goals and looking at the data and coming to conclusions—conclusion about what our next steps were, what we were doing well, and what we weren’t doing well, and then looking for the resources to fill in the gaps. Costa’s districtwide goals have remained the same for several years now, although individual schools have also established their own measures of progress tailored to local contexts. As part of the prognostic frame at Mesa High School, the principal and her leadership team have developed two main implementation goals on the use of data. First, the aim is to have teachers share their data and work collaboratively to develop best practices. The school has recently instituted regular departmental and content team meetings to foster reflective practice around data analysis Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 19 Park et al. and instructional planning. The second goal at Mesa is that all teachers use both academic performance and student demographic data to make decisions on differentiating instruction or classroom placement. At the high school level, there is an increasing focus on having students meet college entry requirements. This goal is directly tied to the framing of improving equity and equal opportunity to learn. One method was to redesign courses to prepare all students for college. Instead of basic, college prep, and honors courses, classes were reconfigured to college prep, honors, and advanced placement (AP). Pamela, an English teacher, suggested that restructuring class levels to promote college acceptance and providing open access to AP classes stopped some teachers from using deficit views of students to rationalize poor performance scores. As lowperforming students were supposed to be dispersed across different types of classes, in theory teachers should be able to compare the effectiveness of instructional practices and collaborate. She elaborated on this issue and discussed the change in teachers’ behaviors: There were a couple of teachers who would say, “Well I always have the low kids so of course I’m going to do worse.” But now it’s sort of been all of the low kids have been spread across the board because there aren’t the basic courses anymore, except for one class that’s a read/write aid, but they don’t take the test. They take it within all the same classes in English and in history and in math or whatever. Now it’s more like, “Ok but we have the same kids so the scores should be approximately the same right,” so not too much animosity or whatever anymore. Then the people who used to complain they really look dumb, you know what I mean? So they really stopped. However, it is important to point out that Pamela taught honors-level courses with students who outperformed classes designated as college prep which may have shaded her perceptions. Although the school may have restructured its courses to promote high academic expectations, teachers indicated that they did not experience much change in their student composition within their classrooms. Other teachers’ comments belied Pamela’s belief that class restructuring has led to changed attitudes about students, suggesting that sensemaking frames about equity, student learning, and the motivation for data use remained a contested process. Hayley, another English teacher, talked about how collaborating around data did not work for her: Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 20 Educational Policy XX(X) I mean when we have our collaboration meetings, and we’re looking at our class results, then we’ll look at it and I’m compare my results with another junior teacher’s results and see, ok my students scored really high, yours didn’t score really high, what do I do to teach them? But it’s not even a fair comparison because I teach AP and honors kids. My kids are going to outscore the other college prep students. So it’s not really a fair comparison. It seems a little bit ridiculous for me. For teachers like Hayley, data did not enable collaboration because she believed that test scores were predetermined based on class compositions and students’ ability levels. Other comments reveal how categorization of students by ability levels mediates teacher beliefs about using data. Nancy, an English teacher in the same grade level as Pamela, shared her frustration about using and sharing data: To be honest with you, we don’t have any basic classes. We all have basic kids in our college prep classes so that makes [sharing data] difficult as well. Unless you’re dealing strictly with all honors classes, you’re going to have your kids that are your ‘far below basic’ in your college prep classes. Some of them they just don’t care. Another English teacher viewed equity and excellence as competing rather than complementary goals and had a negative view on class restructuring: You know when I started teaching we had basic English, college prep and honors. Now we have AP, honors and college prep. There’s no more basic English. So the college prep, you know I mean you have to put college prep in quotes there because it’s not really college prep. I mean these kids are probably not even going go to college. Differentiating students by ability was an important distinction that some teachers made in terms of believing that high academic standards were possible for every student. Research on detracking and other equity-oriented reforms indicate that without addressing teachers’ implicit assumptions about student ability levels and their capacity for learning, structural changes will do very little to actually produce equitable outcomes (Lipman, 1997; Oakes, Wells, Jones, & Datnow, 1997). In the case of Mesa, the relabeling of classes resulted in structural tinkering where the names of the course have changed to indicate high goals (e.g., college prep) but students were still clustered by Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 21 Park et al. ability levels. Although the leadership team and majority of the teachers we spoke with bought into the equity framing, preexisting beliefs about student ability and learning were not explicitly tackled. As the restructuring of classes did not fundamentally detrack students, data did not seem to challenge teachers’ implicit deficit frames of students. Teachers’ comments suggest that goals of high academic standards and equal access are likely to be hindered by the organization of classroom such as ability grouping. Thus, although developing common goals had been a useful strategy by the district to mobilize teachers and administrators to focus on using data for instructional improvement to some extent, the sensemaking frames developed by leaders to cultivate DDDM did not necessarily encourage teachers to directly challenge these long-held assumptions about student ability. Continuous monitoring of progress. A large part of the district’s prognostic framing of using data as a continuous improvement tool required a method of frequently evaluating progress. To monitor student performance throughout the year rather than depend on the end-of-the-year state tests to inform decisions, the district created benchmark assessments aligned to the curriculum guides and pacing plans.8 The district’s rationale was to have quarterly assessments that would keep the district on track but would also give schools the flexibility to develop their own school or teacher-created assessments. In addition to the district-required assessments, the teachers at Mesa High School administered their own department-generated midquarter benchmark assessments. These, too, were given in all core subjects and in some elective courses such as art. In addition, teachers frequently used other assessments provided by textbook or curriculum packages. The district also began relying on the PSAT as a data source. The leadership required all 10th graders to take the PSAT, paid for by the district, so that the data could be used for placement into AP classes. As schools become more sophisticated in their data use and decision making, the district’s leadership expects open access or at least expanded opportunities for all students to enroll in AP courses. Furthermore, because the district was increasingly focusing on making sure that students were eligible for the state’s university system, course enrollment was perceived to be an important tool to monitor school progress. Due to the frequent changing of assessments and pacing plans, teachers at Mesa questioned the validity of the district-developed tests. The constant readjustment of the pacing plan and benchmark assessments seemed to have created a level of dissatisfaction about the validity and reliability of using data to make instructional decisions. The pacing plan, in particular, made it difficult for some teachers to buy into the notion that benchmarks reflected actual student learning rather than superficial coverage of the standards. Thus, although similar curriculum guides and pacing plans may help teachers Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 22 Educational Policy XX(X) “stay on the same page,” it may also be counterproductive when it is constantly readjusted. When benchmark assessments are considered unreliable or invalid indicators of teaching practices, data derived from these measurements are unlikely be used to inform instruction. Overall, administrators and teachers suggest the prognostic framing on continuous improvement helped to redirect focus away from merely using data to meet accountability demands, but strategies such as pacing plans and the readjustment of benchmarks also seem to have lessened the relevance of data use for teachers. The district specifically concentrated on the process of establishing goals, monitoring progress, and reevaluating performance continuously. These strategies were meant to build capacity for DDDM and to ensure that continuous improvement became sustainable within schools. The district encouraged schools to use DDDM practices to make substantive changes in their curriculum, instruction, and policies in an effort to improve student learning. Much of this was still a work in progress as the district continues to refine its goals and develops increasingly sophisticated approaches to tackle improvement efforts and build capacity. The core framing tasks undertaken by the system leaders suggest that framing will be an ongoing effort as the district attempts to deepen engagement with DDDM. Conclusion In this study, we examined how district and school site leaders framed data use, focusing on the ways in which they attempted to create opportunities for shared understandings of DDDM implementation. These framing tasks set the conditions for teachers and administrators to construct and reconstruct the purpose and utility of data use. The examples of Costa USD and Mesa shed light on how core framing tasks on DDDM were undertaken by leaders and how that process of framing seem to influence sensemaking and ultimately the implementation of DDDM. Crafting Frames on the Implementation of DDDM Our analysis suggests that diagnostic, motivating, and prognostic frames were used to create a culture of DDDM that focused on using data for continuous improvement. The frames around DDDM that were constructed appear to be dynamic and interacted with existing beliefs and orientations about data use and theory of reform. Leaders in this study recognized that a compliance orientation toward data use would not lead to authentic or sustained engagement with data. Conceptualizing DDDM as a continuous Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 23 Park et al. improvement strategy that directly informs teaching and learning processes appeared to be a strategic sensemaking endeavor that leaders undertook during implementation to persuade teachers that data use was relevant. In terms of diagnostic framing, district leaders had a clear but difficult task of confronting low expectations for their students. Mesa High School was situated in a district where leaders framed underperformance as one of unequal learning opportunity and made a moral and ethical appeal to teachers. The motivating frames highlighted by district and site leaders suggested that the main rationale for data use needs to be continuous improvement rather than a sole focus on meeting accountability demands. Furthermore, emotional responses to change and existing norms about social interaction needed to be addressed. Teachers in this district did not appear to require additional motivation to use data for individual program improvement but perhaps needed a rationale to publicly share data and collaborate. This rationale required paradigm shifts in who “owns” data, how data are shared, and how data are used. Framing in this case appears to have supported the conditions for teachers to share and make meaning of data in a less threatened manner. Finally, the prognostic frames focused on districtwide development of goals and frequent monitoring of student performance. These frames appeared to have mixed results, suggesting goals and monitoring of student outcomes may represent a more contested process. Although framing activities are important, our data suggest that in addition leaders must be mindful about addressing long-standing norms, beliefs, and structure (e.g., about student ability and tracking) that may inhibit inquiry into school-level performance trends. Thus, for framing to be considered credible by those implementing the reform, it also needs to align or slightly stretch existing beliefs and practices. This study suggests implications for understanding elements of leadership and policy implementation. Framing can be conceptualized as a persuading tactic intended to garner and maintain support for causes. How leaders frame the use of data and the conceptual tools they provide are likely to be important in how local educators make sense of DDDM. Our work suggests that the ways in which leaders are intentional about the framing of reform shapes to some degree how others determine whether new ideas are salient and credible. The framing process takes conscious and persistent efforts to engender shared constructs of policy messages. Rather than jumping into implementation of policies, it may behoove leaders to take time to reflect on their existing beliefs and practices, including their theory of change. Framing is a strategic activity that necessitates reflection of existing practices and beliefs, as well as possible solutions embedded within a new or existing theory of Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 24 Educational Policy XX(X) change to articulate a policy message that resonates with implementers. To support DDDM, leaders will likely need to identify a common problem or a cause (i.e., diagnostic framing). To legitimize DDDM, they also may have to develop strategies for action and motivating frames that provide rationale in garnering support from others. If data use is to be a productive strategy in improvement, then leaders and others may need to explicitly define data use and articulate the processes that will produce concrete actions and outcomes. Theoretical and Research Implications The analysis of framing activities in this study builds on research that examines the construction of policy as a social activity (Coburn, 2001; Spillane, Reiser, & Reimer, 2002) where patterns of formal and informal interactions affect the outcomes of implementers’ sensemaking processes. To date, much of this work portrays sensemaking as an intraorganizational function, paying less attention to how district and school site leaders both frame or edit the sensemaking process. These studies also tend to present sensemaking primarily as an unconscious activity where individuals enact policy after negotiating meaning. In addition, the majority of research on DDDM takes for granted the meaning and processes embedded within the model and, as such, rarely investigate how implementers define DDDM. By using frame analysis, our study extends the theoretical approaches to policy implementation by concentrating on intentional sensemaking activities within larger frames. Frame analysis offers analytic purchase on how the meaning of a new reform is consciously created to facilitate its saliency and legitimacy within organizations, such as schools, and in turn, how those enacting the reform make meaning and ultimately implement reform. Our work suggests that implementation is not only a learning process but also a political act where various actors are positioned to frame, direct, support, constrain, or transform practitioners’ sensemaking of reform. The act of framing both foregrounds certain elements of DDDM and in turn backgrounds others. That is, frames have the potential to focus vision and efforts in improving teaching and learning by using data, but these same frames may limit or privilege alternative perspectives. This act of privileging seems to suggest that leading around DDDM is not only about managing technical expertise but also an inherently political process by which certain frames are valued, and as such, shape meaning and potentially outcomes. At the same time, this study illustrates how framing may play an important role in implementation but is insufficient by itself to override long-held Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 25 Park et al. beliefs and practices. As sensemaking theorists have noted, when social actors encounter new reform ideas, they build on preexisting assumptions and capacity to implement policy and practice (Spillane, Reiser, & Reimer, 2002; Weick et al., 2005). Shifting beliefs about data alone is likely inadequate to bring about improvement in teaching and learning when deficit beliefs about student abilities are not confronted and changed. Studies on past reforms found that without critical dialogue offering alternative views, the dominant deficit-model construction of students’ capabilities were reinforced and reproduced (Lipman, 1997; Oakes et al., 1997). Frames about using data to change instructional practices or improving student learning will likely clash with existing assumptions unless these are explicitly examined and leaders are able to successfully articulate another frame that focuses on student strengths. This study also has implications for future research. First, as this study indicates, reform implementation is not merely executing practices and behaviors; it is also about shifting beliefs and attitudes. Understanding how leaders frame the sensemaking of policy messages may provide useful insights about how actors at the social level construct reform ideas and what conditions support or hinder data use for school improvement. It may be useful to apply this analytical strategy to other types of policy implementation to understand the consequences that sensemaking activities, such as framing, may have on beliefs and behaviors at multiple levels of the system. Second, given the limited focus of this exploratory case study, it leaves many questions about DDDM implementation and the process of framing unanswered. The data shared here only captured a snapshot of how sensemaking on data use was framed by those in positions of formal leadership. It would be useful to apply frame analysis to further understand how educators make sense of these frames over time and how this process is negotiated and enacted within multiple contexts and structures (e.g., department level subcultures and informal teacher meetings). Since the study focused on those that had some “formal” leadership role, it would be important to investigate if and how informal leaders framed meaning on data use and how this interacted with frames created by formal leaders. Another limitation is the selection of a district and school self-identified as data-driven and considered high-performing. As Diamond and Spillane (2004) have found, schools considered low-performing viewed data use demoralizing rather than empowering. Comparison studies on how low- and high-performing schools frame DDDM may shed light on how the process varies depending on the school context and the degree to which framing activities can transform existing beliefs. Comparative studies will enable researchers to examine the influence Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 26 Educational Policy XX(X) of leadership framing on educators’ attitudes toward data and whether they differ between high- and low-performing schools. Like most reform efforts, the implementation of DDDM is a dynamic process mediated by how educators at multiple levels frame, make sense of, and enact data use. Reforms efforts that only focus on the technical and structural aspect of educational improvement often neglect the process of learning and sensemaking among actors as well as the larger frames that may influence these processes. In pursuit of educational excellence and equity, policy makers must not forget that schools are ultimately political and social systems where people’s interactions, preexisting knowledge, and assumptions come into play when new polices are introduced. For reform to make a difference, a complicated mix of frames, resources, capacities, and sensemaking have to come together into a meaningful whole. Appendix Costa Unified School District and Mesa High School Participant List Name Lynn Gina Carol Tom Donna Tina Anne Deanna Luke Jim Pamela Hayley Nancy Cole Dean Kate Jane Position and department District Superintendent District Director of Secondary Instruction District Assistant Superintendent of Curriculum & Instruction District Director of Research and Evaluation Principal of Mesa Title I Coordinator & Staff Development Facilitator Teacher—Science Teacher—Art Department Chair Teacher—English Teacher—English, Union Representative Teacher—English Teacher—English Teacher—English Teacher—English, School Leadership Team Teacher—Math Teacher—Math Teacher—Math & English Language Development Department Chair Teacher—Math, School Leadership Team Data source Interview Interview Interview Interview Interview Interview Focus group Interview Interview Interview Interview Interview Interview Focus group Focus group Focus group Focus group (continued) Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 27 Park et al. Appendix (Continued) Name Richard Daniel Frank Kimberly Candice Mason Craig Tim Linda Lee Position and department Teacher—Music Teacher—Science Teacher—Science Teacher—Science Department Chair Teacher—Science, Instructional Coach Teacher—Science, School Leadership Team Teacher—Social Studies Teacher—Social Studies Peer Coach Teacher—Special Education Department Chair Teacher—World Language Data source Focus group Interview Interview Interview Interview Interview Focus group Focus group Focus group Focus group Total: 27 Acknowledgments The authors like to thank the anonymous reviewers, Amanda Datnow, and Christine Cerven for their helpful comments and feedback. Declaration of Conflicting Interests The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. Funding The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article:The grant for this research was provided by NewSchools Venture Fund, with funding originally received from the Bill & Melinda Gates Foundation and the William and Flora Hewlett Foundation. Responsibility for the content of this article remains the authors’ alone. Notes 1. This study draws on Earl and Katz’s (2002) broad definition of what constitutes data. They define data as “summaries that result from collection of information through systematic measurement or observation or analysis about some phenomenon of interest, using quantitative and/or qualitative methods” (p. 1005). Data collected at the school level include demographic and achievement data and also qualitative data such as teachers observation of students or survey of attitudes and perceptions of school stakeholders. 2. Generally, frame analysts have studied groups outside of formal organizations and the process by which large social and political movements garner support for Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 28 3. 4. 5. 6. 7. 8. Educational Policy XX(X) their causes (e.g., the Civil Rights movements). Scholars have begun to notice that social movements research and organizational studies offer intriguing parallels and these two subfields complement one another (Campbell, 2005; McAdams & Scott, 2005). In education, school reform or policy implementation could be considered a social movement with various stakeholders and goals for change, albeit perhaps bounded within an organizational versus an institutional context. The larger project was a multisite case study of data-driven decision making in four urban high schools and systems, funded by NewSchools Venture Fund. We wish to note that the content of this article does not necessarily reflect the positions of NewSchools. The goal was to identify common themes and practices as well as pinpointing implementation differences. Before the data collection occurred, the research team established a list of criteria for site selection. We chose urban systems and high schools that had evidence of implementing DDDM to learn from their experiences with using data. All names are pseudonyms. California assigns each school and district an Academic Performance Index (API) score—a metric summarizing a school’s test results and performance growth. The school had a Similar School Ranking of nine for the 2007-08 school year and has consistently met AYP requirements. The administrative team is composed of the principal, 2 assistant principals, 4 counselors, 10 department chairs representing each academic subject area, and a Title I coordinator who oversees professional development. The leadership team is composed of teacher representatives from various departments. Deciding to be proactive about seeking assistance, the district’s leaders soon thereafter collaborated with an external professional development provider to help them improve on both DDDM and standards-based education. With the aid of an external partner, the district leadership underwent a multiyear process of developing and refining its goals. Benchmarking was considered critical to the DDDM system because it would enable teachers to make instructional adjustments or student remediation throughout the year. All teachers in core subjects were required to administer the quarterly benchmarks except in AP courses. In deciding how frequently to administer the assessments, the district made a strategic decision to go with four benchmarks per year. References Benford, R. D., & Snow, D. A. (2000). Framing processes and social movements: An overview and assessment. Annual Review of Sociology, 26, 11-39. Bernhardt, V. L. (2004). Data analysis for continuous improvements (2nd ed.). New York, NY: Eye on Education. Booher-Jennings, J. (2005). Below the bubble: “Educational triage” and the Texas accountability system. American Educational Research Journal, 42, 231-268. Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 29 Park et al. Campbell, J. L. (2005). Where do we stand? Common mechanisms in organizations and social movement research. In G. F. Davis, D. McAdam, W. R. Scott, & M. N. Zald (Eds.), Social movements and organization theory (pp. 41-72). New York, NY: Cambridge University Press. Celio, M. B., & Harvey, J. (2005). Buried treasure: Developing a management guide from mountains of school data. Seattle, WA: Center for Reinventing Public Education. Coburn, C. E. (2001). Collective sensemaking about reading: How teachers mediate reading policy in their professional communities. Educational Evaluation and Policy Analysis, 23(2), 145-170. Coburn, C. E. (2006). Framing the problem of reading instruction: Using frame analysis to uncover the microprocesses of policy implementation. American Educational Research Journal, 43, 343-379. Coburn, C. E., & Talbert, J. E. (2006). Conceptions of evidence use in school districts: Mapping the terrain. American Journal of Education, 112(4), 469-495. Coburn, C. E., Toure, J., & Yamashita, M. (2009). Evidence, interpretation, and persuasion: Instructional decision making at the district central office. Teachers College Record, 111, 1115-1161. Creswell, J. W. (1998). Qualitative inquiry and research design: Choosing among five traditions. Thousand Oaks, CA: SAGE. Datnow, A., Park, V., Wohlstetter, P. (2007). Achieving with data: How high performance driven school systems use data to improve instruction for elementary school students. Los Angeles, CA: Center on Educational Governance, University of Southern California. Datnow, A., Park, V., & Kennedy, B. (2008). Acting on data: How urban high schools use data to improve instruction. Los Angeles, CA: Center on Educational Governance, University of Southern California. Davies, S. (2002). The paradox of progressive education: A frame analysis. Sociology of Education, 75, 269-286. Diamond, J. B., & Cooper, K. (2007). The uses of testing data in urban elementary schools: Some lessons from Chicago. National Society for the Study of Education Yearbook, 106(1), 241-263. Diamond, J. B., & Spillane, J. P. (2004). High-stakes accountability in urban elementary schools: Challenging or reproducing inequality? Teachers College Record, 106, 1145-1176. Earl, L., & Katz, S. (2002). Leading schools in a data rich world.In K. Leithwood, Pl. Hallinger, G. Furman, P. Gronn, J. MacBeath, B. Mulforld& K. Riley (Eds.) The second international handbook of educational leadership and administration (pp. 1003-1022). Dordrecht, Netherlands: Kluwer. Earl, L., & Katz, S. (2006). Leading schools in a data-rich world: Harnessing data for school improvement. Thousand Oaks, CA: Corwin. Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 30 Educational Policy XX(X) Firestone, W. A., & Gonzalez, R. (2007). Culture and processes affecting data use in school districts. In P. A. Moss (Ed.), Evidence and decision making (pp. 132-154). Chicago, IL: National Society for the Study of Education. Firestone, W. A., Fitz, J., & Broadfoot, P. (1999). Power, learning, and legitimation: Assessment implementation across levels in the United States and the United Kingdom. American Educational Research Journal, 36, 759-793. Grossman, F. D. (2010). Dissent from within: How educational insiders use protest to create policy change. Educational Policy, 24, 655-686. Herman, J., & Gribbons, B. (2001). Lessons learned in using data to support school inquiry and continuous improvement: Final report to the Stuart Foundation. Los Angeles, CA: Center for the Study of Evaluation, University of California, Los Angeles. Honig, M. I., & Coburn, C. E. (2008). Evidence-based decision making in school district central offices: Toward a policy and research agenda. Educational Policy, 22, 578-608. Ingram, D., Louis, K. S., & Schroeder, R. G. (2004). Accountability policies and teacher decision-making: Barriers to the use of data to improve practice. Teachers College Record, 106, 1258-1287. Kerr, K. A., Marsh, J. A., Ikemoto, G. S., Darilek, H., & Barney, H. (2006). Strategies to promote data use for instructional improvement: Actions, outcomes, and lessons from three urban districts. American Journal of Education, 112, 496-520. Lachat, M. A., & Smith, S. (2005). Practices that support data use in urban high schools. Special Issue on Transforming data into knowledge: Applications of data-based decision making to improve instructional practice. Journal of Education Change for Students Placed At-Risk, 10(3), 333-349. Lipman, P. (1997). Restructuring in context: A case study of teacher participation and the dynamics of ideology, race, and power. American Educational Research Journal, 34(1), 3-37. Marsh, J. A., Pane, J. F., & Hamilton, L. S. (2006). Making sense of data-driven decision making in education. Santa Monica, CA: RAND. McAdams, D., & Scott, R. W. (2005). Organizations and movements. In G. F. Davis, D. McAdam, W. R. Scott, & M. N. Zald (Eds.), Social movements and organization theory (pp. 4-40). New York, NY: Cambridge University Press. McDonnell, L.M. (1995). Opportunity to learn as a research concept and a policy instrument. Educational Evaluation and Policy Analysis, 17 (3), 305-322. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis (2nd ed.). Thousand Oaks, CA: SAGE. No Child Left Behind Act of 2001, Public Law 107-110. Oakes, J., Wells, A. S., Jones, M., & Datnow, A. (1997). Detracking: The social construction of ability, cultural politics, and resistance to reform. Teachers College Record, 98, 482-510. Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012 31 Park et al. Spillane, J. P., Reiser, B. J., & Reimer, T. (2002). Policy implementation and cognition: Reframing and refocusing implementation research. Review of Educational Research, 72, 387-431. Supovitz, J. A., & Klein, V. (2003). Mapping a course for improved student learning: How innovative schools systematically use student performance data to guide improvement. Philadelphia, PA: Consortium for Policy Research in Education (CPRE). Togneri, W., & Anderson, S. (2003). Beyond islands of excellence: What districts can do to improve instruction and achievement in all schools. Washington, DC: Learning First Alliance. Weber, K., & Glynn, M. A. (2006). Making sense with institutions: Context, thought and action in Karl Weick’s theory. Organization Studies, 27, 1639-1660. Weick, K. E., Sutcliffe, K. M., & Obstfeld, D. (2005). Organizing and the process of sensemaking. Organization Science, 16, 409-421. Yin, R. (2003). Case study research (3rd ed.). Beverly Hills, CA: SAGE. Young, V. M. (2006). Teacher’s use of data: Loose coupling, agenda setting, and team norms. American Journal of Education,112, 521-548. Bios Vicki Park is an Assistant Project Scientist in Education Studies at the University of California, San Diego and the Director of Research for the Pathways to Postsecondary Education Success: Maximizing Opportunities for Youth in Poverty project. Her research focuses on urban school reform and leadership, policy implementation, and the ways in which class, race, and gender shape educational opportunities for lowincome youth. Alan J. Daly is an Assistant Professor of Education at the University of California, San Diego. In addition to 15 years of public education experience as a teacher, psychologist, and administrator, Alan has also been the Program Director for the Center for Educational Leadership and Effective Schools at the University of California, Santa Barbara. Research interests include leadership, educational policy, district reform, and social network theory. His recent publications include an edited volume entitled, Social Network Theory and Educational Change, published by Harvard Education Press. Alison Wishard Guerra, Assistant Professor of Education Studies, is a member of the faculty of the doctor of education graduate program in teaching and learning at the University of California, San Diego. She has a doctorate in Applied Developmental Studies in Education from UCLA. Her research focuses on social and linguistic development in early childhood, with particular focus on developmental competencies among Latino children from low-income families. Downloaded from epx.sagepub.com at UNIV CALIFORNIA SAN DIEGO on August 31, 2012