When a hazard such as a hurricane threatens, people are forced to make a wide variety of decision... more When a hazard such as a hurricane threatens, people are forced to make a wide variety of decisions, and the information they receive and produce can influence their own and others’ actions. As social media grows more popular, an increasing number of people are using social media platforms to obtain and share information about approaching threats and discuss their interpretations of the threat and their protective decisions. This work aims to improve understanding of natural disasters through social media and provide an annotation scheme to identify themes in user’s social media behavior and facilitate efforts in supervised machine learning. To that end, this work has three contributions: (1) the creation of an annotation scheme to consistently identify hazard-related themes in Twitter, (2) an overview of agreement rates and difficulties in identifying annotation categories, and (3) a public release of both the dataset and guidelines developed from this scheme.
Risks associated with natural hazards such as hurricanes are increasingly communicated on social ... more Risks associated with natural hazards such as hurricanes are increasingly communicated on social media. For hurricane risk communication, visual information products—graphics—generated by meteorologists and scientists at weather agencies portray forecasts and atmospheric conditions and are offered to parsimoniously convey predictions of severe storms. This research considers risk interactivity by examining a particular hurricane graphic which has shown in previous research to have a distinctive diffusion signature: the ‘spaghetti plot’, which contains multiple discrete lines depicting a storm’s possible path. We first analyzed a large dataset of microblog interactions around spaghetti plots between members of the public and authoritative weather sources within the US during the 2017 Atlantic hurricane season. We then conducted interviews with a sample of the weather authorities after preliminary findings sketched the role that experts have in such communications. Findings describe h...
Proceedings of the Sixth International Workshop on Natural Language Processing for Social Media, 2018
A large amount of social media data is generated during natural disasters, and identifying the re... more A large amount of social media data is generated during natural disasters, and identifying the relevant portions of this data is critical for researchers attempting to understand human behavior, the effects of information sources, and preparatory actions undertaken during these events. In order to classify human behavior during hazard events, we employ machine learning for two tasks: identifying hurricane related tweets and classifying user evacuation behavior during hurricanes. We show that feature-based and deep learning methods provide different benefits for tweet classification, and ensemble-based methods using linguistic, temporal, and geospatial features can effectively classify user behavior.
This article investigates the dynamic ways that people communicate, assess, and respond as a weat... more This article investigates the dynamic ways that people communicate, assess, and respond as a weather threat evolves. It uses social media data, which offer unique records of what people convey about their real-world risk contexts. Twitter narratives from 53 people who were in a mandatory evacuation zone in a New York City neighborhood during Hurricane Sandy in 2012 were qualitatively analyzed. The study provides rich insight into the complex, dynamic information behaviors and risk assessments of people at risk, and it illustrates how social media data can be collected, sampled, and analyzed to help provide this understanding. Results show that this sample of people at significant risk attended to forecast information and evacuation orders as well as multiple types of social and environmental cues. Although many tweeted explicitly about the mandatory evacuation order, forecast information was usually referenced only implicitly. Social and environmental cues grew more important as the...
International Journal of Human–Computer Interaction, 2018
Today, disaster events are mobilizing digital volunteers to meet the data needs of those on the g... more Today, disaster events are mobilizing digital volunteers to meet the data needs of those on the ground. One form of this crowd work is Volunteered Geographic Information. This peer-produced spatial data creates the most up-to-date map of the affected region; maintaining the accuracy of these data is therefore a critical task. Accuracy is one aspect of data quality, a relative concept requiring standards to measure against. The field of Geographic Information Sciences has developed standards for this comparison, achieving widespread acceptance. However, the peer production model of spatial data presents new opportunities-and challenges-to traditional methods of quality assessment. Through analysis of the OpenStreetMap database, we show that temporal editing patterns and contributor characteristics can provide additional means of understanding spatial data quality. Drawing upon experiences from Wikipedia, we offer and evaluate three intrinsic quality metrics of peer-produced spatial data to assess the quality of contributions to OpenStreetMap for crisis response.
Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 2016
OpenStreetMap (OSM) is the most widely used volunteer geographic information system. Although it ... more OpenStreetMap (OSM) is the most widely used volunteer geographic information system. Although it is increasingly relied upon during humanitarian response as the most up-todate, accurate, or accessible map of affected areas, the behavior of the mappers who contribute to it is not well understood. In this paper, we explore the work practices and interactions of volunteer mappers operating in the hightempo, high-volume context of disasters. To do this, we built upon and expanded prior network analysis techniques to select high-value portions of the vast OSM data for further qualitative analysis. We then performed detailed content analysis of the identified activity and, where possible, conducted interviews with the participants. This research allowed the identification of seven distinct mapping practices that can be classified according to dimensions of time, space, and interpersonal interaction. Our work represents a baseline for future research about how OSM crisis mapping practices have evolved over time.
2016 49th Hawaii International Conference on System Sciences (HICSS), 2016
An important area of work in big data software engineering involves the design and development of... more An important area of work in big data software engineering involves the design and development of software frameworks for data-intensive systems that perform large-scale data collection and analysis. We report on our work to design and develop a software framework for analyzing the collaborative editing behavior of OpenStreetMap users when working on the task of crisis mapping. Crisis mapping occurs after a disaster or humanitarian crisis and involves the coordination of a distributed set of users who collaboratively work to improve the quality of the map for the impacted area in support of emergency response efforts. Our paper presents the challenges related to the analysis of OpenStreetMap and how our software framework tackles those challenges to enable the efficient processing of gigabytes of OpenStreetMap data. Our framework has already been deployed to analyze crisis mapping efforts in 2015 and has an active development community.
Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 2015
OpenStreetMap (OSM) is a volunteer-driven, globally distributed organization whose members work t... more OpenStreetMap (OSM) is a volunteer-driven, globally distributed organization whose members work to create a common digital map of the world. OSM embraces ideals of open data, and to that end innovates both socially and technically to develop practices and processes for coordinated operation. This paper provides a brief history of OSM and then, through quantitative and qualitative examination of the OSM database and other sites of articulation work, examines organizational growth through the lens of two catastrophes that spurred enormous humanitarian relief responses-the 2010 Haiti Earthquake and the 2013 Typhoon Yolanda. The temporally-and geographically-constrained events scope analysis for what is a rapidly maturing, whole-planet operation. The first disaster identified how OSM could support other organizations responding to the event. However, to achieve this, OSM has had to refine mechanisms of collaboration around map creation, which were tested again in Typhoon Yolanda. The transformation of work between these two events yields insights into the organizational development of large, data-producing online organizations.
Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, 2017
Nepal was struck by two major earthquakes in April and May 2015 which gave rise to much media att... more Nepal was struck by two major earthquakes in April and May 2015 which gave rise to much media attention. Because of photographs' power to influence how people perceive significant events, we investigate how these disasters are represented visually through Twitter-shared images in three ways. First, we compare how geotagged image tweets are distributed vis-à-vis the reported damage, to see if a seemingly "objective" method of representation stands up. Second, with an iteratively developed coding scheme, we examine how images are differently produced and shared within global versus local populations and after each earthquake, with the idea that amplification "collectively instructs" what features of the event are most important. Third, we analyze how images from other locations, disasters, and time periods are appropriated as part of the "story" of the disaster event. We found differences in image popularity, with global twitterers emphasizing recovery and relief efforts in their diffusion of images, and locals emphasizing people suffering and major damage in their sourcing and re-sharing. We also found that globals were more likely to appropriate images, evoking lessons from Sontag about "the pain of others" [39].
The tradeoffs we once worried about pale in comparison to the ones in the newscasts as events unf... more The tradeoffs we once worried about pale in comparison to the ones in the newscasts as events unfolded. Where popular articles once lamented the public impropriety of mobile telephony and its role in hastening life’s already fast pace, we instead heard accounts of calls from frightened passengers on hijacked airliners and doomed office workers in the World Trade Center towers and Pentagon to parents, spouses, and friends. But knowledge of the comforting receipt of those final “I love yous” and the hope that came with apparent calls from the rubble of the buildings must be reconciled with the sad awareness that the same technology had also helped the terrorists communicate and coordinate their efforts. These accounts of the extremes of the use of mobile telephony are now part of the public consciousness; as a result, I expect the divide between the attitudes of users and nonusers to narrow. While everyday struggles to define and adhere to social norms, or socially accepted behavioral...
Proceedings of the ACM on Human-Computer Interaction, 2019
Temporal coordination endures as a central topic in computer-supported cooperative work (CSCW) be... more Temporal coordination endures as a central topic in computer-supported cooperative work (CSCW) because information systems still struggle to adequately support varying representations of time in the context of collaborations that areboth temporally and geographically dispersed. Moreover, the adaptive practices of these broadly dispersed groups are still not well understood. We ask:How do globally distributed teams temporally coordinate to accomplish their work? We examine an extreme case of online temporal coordination: high-tempo information curation about the urgent humanitarian crisis following the 2017 Hurricane Maria landfall in Puerto Rico. Our analysis of synchronous chat transcripts and data artifacts produced by The Standby Task Force reveals how this digital humanitarian group establishes temporal coordination through different shared understandings of time relative to the crisis, the globally distributed work, and the collaborative information technologies. We make four c...
Proceedings of the ACM on Human-Computer Interaction, 2018
Over the past 20 years, the practices of crisis preparedness, response, and recovery have become ... more Over the past 20 years, the practices of crisis preparedness, response, and recovery have become increasingly dependent on information and communication technology (ICT) to accomplish their work. More recently, crisis informatics has developed an analysis of these phenomena from social and computational perspectives. To further to assess the consequences and opportunities of technological developments in the field, we re-interpret the concept of informating, first developed by Zuboff to describe the impacts of technological changes on the workplace during the 1980s. We draw on four contemporary examples of how ICT is changing the way we conceive of and respond to natural hazards to offer a new reading of the concept of informating in the growing field of crisis informatics. We then argue that this concept suggests the adoption of a more critical agenda for crisis informatics research to better respond to contemporary challenges presented by climate change and natural hazards.
Developments in information and communication technology (ICT) have adjusted the opportunities fo... more Developments in information and communication technology (ICT) have adjusted the opportunities for spatial and temporal representations of data, possibly permitting the simultaneous visualization of how different regions and populations are affected during large-scale emergencies and crises. We surveyed 13 crisis-related mashups to derive some high-level design directions to guide the design and testing of next generation crisis support tools. The current web mashups offer a new way of looking at how crises are spatiotemporally ordered. However, since all technology is constrained by limitations of design choice, examining the limits and possibilities of what current design choices afford can inform attributes of what next generation crisis support tools would require.
Opportunities for participation by members of the public are expanding the information arena of d... more Opportunities for participation by members of the public are expanding the information arena of disaster. Social media supports “backchannel ” communications, allowing for wide-scale interaction that can be collectively resourceful, self-policing, and generative of information that is otherwise hard to obtain. Results from our study of information practices by members of the public during the October 2007 Southern California Wildfires suggest that community information resources and other backchannel communications activity enabled by social media are gaining prominence in the disaster arena, despite concern by officials about the legitimacy of information shared through such means. We argue that these emergent uses of social media are pre-cursors of broader future changes to the institutional and organizational arrangements of disaster response.
When a hazard such as a hurricane threatens, people are forced to make a wide variety of decision... more When a hazard such as a hurricane threatens, people are forced to make a wide variety of decisions, and the information they receive and produce can influence their own and others’ actions. As social media grows more popular, an increasing number of people are using social media platforms to obtain and share information about approaching threats and discuss their interpretations of the threat and their protective decisions. This work aims to improve understanding of natural disasters through social media and provide an annotation scheme to identify themes in user’s social media behavior and facilitate efforts in supervised machine learning. To that end, this work has three contributions: (1) the creation of an annotation scheme to consistently identify hazard-related themes in Twitter, (2) an overview of agreement rates and difficulties in identifying annotation categories, and (3) a public release of both the dataset and guidelines developed from this scheme.
Risks associated with natural hazards such as hurricanes are increasingly communicated on social ... more Risks associated with natural hazards such as hurricanes are increasingly communicated on social media. For hurricane risk communication, visual information products—graphics—generated by meteorologists and scientists at weather agencies portray forecasts and atmospheric conditions and are offered to parsimoniously convey predictions of severe storms. This research considers risk interactivity by examining a particular hurricane graphic which has shown in previous research to have a distinctive diffusion signature: the ‘spaghetti plot’, which contains multiple discrete lines depicting a storm’s possible path. We first analyzed a large dataset of microblog interactions around spaghetti plots between members of the public and authoritative weather sources within the US during the 2017 Atlantic hurricane season. We then conducted interviews with a sample of the weather authorities after preliminary findings sketched the role that experts have in such communications. Findings describe h...
Proceedings of the Sixth International Workshop on Natural Language Processing for Social Media, 2018
A large amount of social media data is generated during natural disasters, and identifying the re... more A large amount of social media data is generated during natural disasters, and identifying the relevant portions of this data is critical for researchers attempting to understand human behavior, the effects of information sources, and preparatory actions undertaken during these events. In order to classify human behavior during hazard events, we employ machine learning for two tasks: identifying hurricane related tweets and classifying user evacuation behavior during hurricanes. We show that feature-based and deep learning methods provide different benefits for tweet classification, and ensemble-based methods using linguistic, temporal, and geospatial features can effectively classify user behavior.
This article investigates the dynamic ways that people communicate, assess, and respond as a weat... more This article investigates the dynamic ways that people communicate, assess, and respond as a weather threat evolves. It uses social media data, which offer unique records of what people convey about their real-world risk contexts. Twitter narratives from 53 people who were in a mandatory evacuation zone in a New York City neighborhood during Hurricane Sandy in 2012 were qualitatively analyzed. The study provides rich insight into the complex, dynamic information behaviors and risk assessments of people at risk, and it illustrates how social media data can be collected, sampled, and analyzed to help provide this understanding. Results show that this sample of people at significant risk attended to forecast information and evacuation orders as well as multiple types of social and environmental cues. Although many tweeted explicitly about the mandatory evacuation order, forecast information was usually referenced only implicitly. Social and environmental cues grew more important as the...
International Journal of Human–Computer Interaction, 2018
Today, disaster events are mobilizing digital volunteers to meet the data needs of those on the g... more Today, disaster events are mobilizing digital volunteers to meet the data needs of those on the ground. One form of this crowd work is Volunteered Geographic Information. This peer-produced spatial data creates the most up-to-date map of the affected region; maintaining the accuracy of these data is therefore a critical task. Accuracy is one aspect of data quality, a relative concept requiring standards to measure against. The field of Geographic Information Sciences has developed standards for this comparison, achieving widespread acceptance. However, the peer production model of spatial data presents new opportunities-and challenges-to traditional methods of quality assessment. Through analysis of the OpenStreetMap database, we show that temporal editing patterns and contributor characteristics can provide additional means of understanding spatial data quality. Drawing upon experiences from Wikipedia, we offer and evaluate three intrinsic quality metrics of peer-produced spatial data to assess the quality of contributions to OpenStreetMap for crisis response.
Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 2016
OpenStreetMap (OSM) is the most widely used volunteer geographic information system. Although it ... more OpenStreetMap (OSM) is the most widely used volunteer geographic information system. Although it is increasingly relied upon during humanitarian response as the most up-todate, accurate, or accessible map of affected areas, the behavior of the mappers who contribute to it is not well understood. In this paper, we explore the work practices and interactions of volunteer mappers operating in the hightempo, high-volume context of disasters. To do this, we built upon and expanded prior network analysis techniques to select high-value portions of the vast OSM data for further qualitative analysis. We then performed detailed content analysis of the identified activity and, where possible, conducted interviews with the participants. This research allowed the identification of seven distinct mapping practices that can be classified according to dimensions of time, space, and interpersonal interaction. Our work represents a baseline for future research about how OSM crisis mapping practices have evolved over time.
2016 49th Hawaii International Conference on System Sciences (HICSS), 2016
An important area of work in big data software engineering involves the design and development of... more An important area of work in big data software engineering involves the design and development of software frameworks for data-intensive systems that perform large-scale data collection and analysis. We report on our work to design and develop a software framework for analyzing the collaborative editing behavior of OpenStreetMap users when working on the task of crisis mapping. Crisis mapping occurs after a disaster or humanitarian crisis and involves the coordination of a distributed set of users who collaboratively work to improve the quality of the map for the impacted area in support of emergency response efforts. Our paper presents the challenges related to the analysis of OpenStreetMap and how our software framework tackles those challenges to enable the efficient processing of gigabytes of OpenStreetMap data. Our framework has already been deployed to analyze crisis mapping efforts in 2015 and has an active development community.
Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 2015
OpenStreetMap (OSM) is a volunteer-driven, globally distributed organization whose members work t... more OpenStreetMap (OSM) is a volunteer-driven, globally distributed organization whose members work to create a common digital map of the world. OSM embraces ideals of open data, and to that end innovates both socially and technically to develop practices and processes for coordinated operation. This paper provides a brief history of OSM and then, through quantitative and qualitative examination of the OSM database and other sites of articulation work, examines organizational growth through the lens of two catastrophes that spurred enormous humanitarian relief responses-the 2010 Haiti Earthquake and the 2013 Typhoon Yolanda. The temporally-and geographically-constrained events scope analysis for what is a rapidly maturing, whole-planet operation. The first disaster identified how OSM could support other organizations responding to the event. However, to achieve this, OSM has had to refine mechanisms of collaboration around map creation, which were tested again in Typhoon Yolanda. The transformation of work between these two events yields insights into the organizational development of large, data-producing online organizations.
Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, 2017
Nepal was struck by two major earthquakes in April and May 2015 which gave rise to much media att... more Nepal was struck by two major earthquakes in April and May 2015 which gave rise to much media attention. Because of photographs' power to influence how people perceive significant events, we investigate how these disasters are represented visually through Twitter-shared images in three ways. First, we compare how geotagged image tweets are distributed vis-à-vis the reported damage, to see if a seemingly "objective" method of representation stands up. Second, with an iteratively developed coding scheme, we examine how images are differently produced and shared within global versus local populations and after each earthquake, with the idea that amplification "collectively instructs" what features of the event are most important. Third, we analyze how images from other locations, disasters, and time periods are appropriated as part of the "story" of the disaster event. We found differences in image popularity, with global twitterers emphasizing recovery and relief efforts in their diffusion of images, and locals emphasizing people suffering and major damage in their sourcing and re-sharing. We also found that globals were more likely to appropriate images, evoking lessons from Sontag about "the pain of others" [39].
The tradeoffs we once worried about pale in comparison to the ones in the newscasts as events unf... more The tradeoffs we once worried about pale in comparison to the ones in the newscasts as events unfolded. Where popular articles once lamented the public impropriety of mobile telephony and its role in hastening life’s already fast pace, we instead heard accounts of calls from frightened passengers on hijacked airliners and doomed office workers in the World Trade Center towers and Pentagon to parents, spouses, and friends. But knowledge of the comforting receipt of those final “I love yous” and the hope that came with apparent calls from the rubble of the buildings must be reconciled with the sad awareness that the same technology had also helped the terrorists communicate and coordinate their efforts. These accounts of the extremes of the use of mobile telephony are now part of the public consciousness; as a result, I expect the divide between the attitudes of users and nonusers to narrow. While everyday struggles to define and adhere to social norms, or socially accepted behavioral...
Proceedings of the ACM on Human-Computer Interaction, 2019
Temporal coordination endures as a central topic in computer-supported cooperative work (CSCW) be... more Temporal coordination endures as a central topic in computer-supported cooperative work (CSCW) because information systems still struggle to adequately support varying representations of time in the context of collaborations that areboth temporally and geographically dispersed. Moreover, the adaptive practices of these broadly dispersed groups are still not well understood. We ask:How do globally distributed teams temporally coordinate to accomplish their work? We examine an extreme case of online temporal coordination: high-tempo information curation about the urgent humanitarian crisis following the 2017 Hurricane Maria landfall in Puerto Rico. Our analysis of synchronous chat transcripts and data artifacts produced by The Standby Task Force reveals how this digital humanitarian group establishes temporal coordination through different shared understandings of time relative to the crisis, the globally distributed work, and the collaborative information technologies. We make four c...
Proceedings of the ACM on Human-Computer Interaction, 2018
Over the past 20 years, the practices of crisis preparedness, response, and recovery have become ... more Over the past 20 years, the practices of crisis preparedness, response, and recovery have become increasingly dependent on information and communication technology (ICT) to accomplish their work. More recently, crisis informatics has developed an analysis of these phenomena from social and computational perspectives. To further to assess the consequences and opportunities of technological developments in the field, we re-interpret the concept of informating, first developed by Zuboff to describe the impacts of technological changes on the workplace during the 1980s. We draw on four contemporary examples of how ICT is changing the way we conceive of and respond to natural hazards to offer a new reading of the concept of informating in the growing field of crisis informatics. We then argue that this concept suggests the adoption of a more critical agenda for crisis informatics research to better respond to contemporary challenges presented by climate change and natural hazards.
Developments in information and communication technology (ICT) have adjusted the opportunities fo... more Developments in information and communication technology (ICT) have adjusted the opportunities for spatial and temporal representations of data, possibly permitting the simultaneous visualization of how different regions and populations are affected during large-scale emergencies and crises. We surveyed 13 crisis-related mashups to derive some high-level design directions to guide the design and testing of next generation crisis support tools. The current web mashups offer a new way of looking at how crises are spatiotemporally ordered. However, since all technology is constrained by limitations of design choice, examining the limits and possibilities of what current design choices afford can inform attributes of what next generation crisis support tools would require.
Opportunities for participation by members of the public are expanding the information arena of d... more Opportunities for participation by members of the public are expanding the information arena of disaster. Social media supports “backchannel ” communications, allowing for wide-scale interaction that can be collectively resourceful, self-policing, and generative of information that is otherwise hard to obtain. Results from our study of information practices by members of the public during the October 2007 Southern California Wildfires suggest that community information resources and other backchannel communications activity enabled by social media are gaining prominence in the disaster arena, despite concern by officials about the legitimacy of information shared through such means. We argue that these emergent uses of social media are pre-cursors of broader future changes to the institutional and organizational arrangements of disaster response.
Uploads
Papers by Leysia Palen