Data for Disaster Management: Mind the Gap

Over the last 25 years, the world has seen a rise in the frequency of natural disasters in rich and poor countries alike. Today, there are more people at risk from natural hazards than ever before, with those in developing countries particularly at risk. This essay series is intended to explore measures that have been taken, and could be taken, in order to improve responses to the threat or occurrence of natural disasters in the MENA and Indo-Pacific regions. Read more ...
 

The Big Picture

The number of reported natural disasters has almost doubled since the 1980s.[1] In the Middle East and North Africa, there is even an almost three-fold increase.[2] Today, there are more people vulnerable to natural hazards and affected by natural disasters than ever before, particularly in developing countries. Rapid urbanization, larger coastal settlements, water scarcity, and climate change are among the main interrelated drivers. Climate change, due to human activities such as greenhouse gas emissions and land use, in particular, causes the increase―in frequency and intensity―of hydrometeorological hazards.[3] Scientists have proven this relationship using advanced analytical techniques for several hydrometeorological hazards, such as tropical cyclones in the central Pacific, heavy rainfall in Europe, drought in East Africa, and heat waves in Australia, Asia, and South America. It is evident that a clear-cut distinction between natural and man-made hazards becomes more and more difficult to make, given the aggravation of “existing” natural hazards by the human-caused changes in climate.

Organizations working on disaster management need to find sustainable and scalable ways to mitigate these increased risks and to minimize the loss of lives and livelihoods when a disaster hits. In the pressurized and critical contexts in which they work, funding is severely lacking, creating tremendous strain on the humanitarian aid system. The United Nations, whose most recent humanitarian appeal requested $21.6 billion to meet the needs of 95.4 million people across 40 countries, is facing a staggering 75 percent funding gap.[4] Even though a large portion of this appeal is for complex emergencies, this still leaves a significant amount allocated for natural disasters. Furthermore, more than 50 percent of people affected by natural disasters live in fragile and conflict-affected states.[5] Hence, it is urgently necessary for international humanitarian, as well as national and local governmental and civil society, organizations to do things differently and to innovate. This essay focuses on the role that data plays in disaster management and on data-related innovations that can be beneficial for humanitarian actors and possibly transform the way they operate. 

Data for Disaster Management: Information and Collaboration Gaps

Natural disasters are complex events about which both military and humanitarian responders need accurate, reliable, and timely data in order to respond effectively. This data should cover needs in the areas of crisis impact (baseline, damage and needs, and disaster situation data) and the operational environment (coordination, capacity, service locations, security, and access).[6 A strong match between information needs and available data leads to an enhanced situational awareness, improves sense-making, and enables adequate decision-making when deciding on response activities. This information management process is challenging, given the inherently chaotic and disrupted situation, where decision-makers face compressed timelines and high levels of uncertainty. Indeed, acquiring essential, accurate, and comprehensive data is a highly iterative rather than a linear process where, especially in the immediate aftermath of a disaster, decision-makers invariably face an information gap. This gap is widest in regions with a high data poverty index.[7]

In the case of the 2014 river floods in Bangladesh, the working group on disaster emergency esponse stated right after floods arrived: “Based on the information that was available for review it is difficult to get an overview of the flooding situation across the country because of the quality of information available and because of the differences in the collection, presentation and content of the information. In addition, most of the information is several weeks old.”[8] The heterogeneity in data sets is a consequence of the multitude of procedures and stakeholders involved in a response. During Typhoon Haiyan in the Philippines in 2013, over 100 responding non-governmental organizations (NGOs) and international organizations (IOs) assisted in the on-site emergency operations and completed about 800 assessments.[9] Harmonizing and coordinating the different assessments among organizations is difficult, and interoperability issues in the data sets that come out of the assessments are most commonly unavoidable. Participatory community risk assessments are usually not available when a disaster happens, though they can provide valuable baseline data to responders.

Example of Participatory Community Risk Assessments in India

In addition to the information gap, there is a collaboration gap. All too often, communities are insufficiently involved, whereas they should be considered as the most important stakeholders; Frequently, data is collected “on” them, rather than “with” them. Furthermore, many governments in developing countries have damage and needs assessment processes that are, to some degree, based on verbal communication or paper-based at the lower administrative levels. One of the key findings of the Nethope Report on Information and Communication Technology Usage in the 2010 Pakistan Floods was that three-quarters of the information shared with the Pakistani government was shared through verbal communication (40 percent) or paper (35 percent).[10] During the 2014 Bangladesh floods, the project implementation officers phoned representatives from different wards (the lowest administrative level) and aggregated the data they obtained into a consolidated format for their upazila (a sub-unit of a district). The way the data was aggregated led to a loss of granularity. Moreover, national and local NGOs still have many paper-based processes. Thus, participatory community risk assessments are usually not available when a disaster happens, though they can provide valuable baseline data to responders.[11]

The Data Revolution

The digital age that we have entered holds a lot of potential for narrowing the information and collaboration gaps mentioned above. By the end of 2015, there were more than seven billion mobile cellular subscriptions, corresponding to a worldwide penetration rate of 97% (I.T.U.). Devices, cars, and buildings can form an Internet of Things through the use of sensors, software, and networked connectivity. Hyperconnected people and things create big data at a fast-paced rate, streaming or in batches. This new, big data is often characterized by four V's: volume (scale of data), variety (different forms of data), velocity (rapid generation, streaming data), and veracity (uncertainty of data).[12] It ranges from transactional data, captured data (where one can “opt-in” or “opt-out”), social media, and sensor data up to biological and public records.[13] In fact, digital traces are left behind in almost any transaction, and more and more citizens generate their own online content. The Independent Expert Advisory Group for the U.N. Secretary General on a Data Revolution for Sustainable Development summarizes these developments as follows: “New technologies are leading to an exponential increase in the volume and types of data available, creating unprecedented possibilities for informing and transforming society and protecting the environment. Governments, companies, researchers, and citizen groups are in a ferment of experimentation, innovation, and adaptation to the new world of data, a world in which data are bigger, faster, and more detailed than ever before. This is the data revolution.”[14]

Opportunities and New Players

The data revolution seems to offer great opportunities and challenges, not only for sustainable development but also for humanitarian aid. However, have some of the least developed countries, affected by conflict and disasters, really entered the digital age? Is there not a great difference in access to data between the developed and the developing world, as well as a growing inequality in capabilities, to deal with it? In fact, even in the least developed countries, mobile phone penetration has increased considerably, with penetration rates often between 60 and 80 percent. In many cases, the rate is even higher, given that a mobile phone is, in many cases, shared among family members. Mobile technologies offer various ways to bridge the last mile. Mobile services give often unconnected vulnerable groups access to information and knowledge, empowering them to take action themselves. The mobile phone unlocks SMS-based services, such as access to agricultural information and mobile banking. In cases of flooding, early warning messages can be sent via SMS or voice SMS (to also reach illiterate people); and affected people can text or call specific flood-related hotlines to express their needs and send valuable locally collected data, such as water levels.

In addition, basic smartphones have become increasingly affordable and are the main device for accessing the internet in developing countries. Although internet access through mobile broadband is still lagging considerably behind as compared to 2G mobile, projections are that this gap will narrow in the coming five years. The smartphone allows more extensive bidirectional communication and engagement between affected communities and responding professionals. Community volunteers can use apps to conduct surveys and map their villages. Social media enables affected people to directly voice their needs and concerns and join forces for lobbying and advocacy. Responders have an additional communication channel for engagement with affected people. The increasing affordability of mobile services has made more and more people aware of which services exist, to value and to gain access to them. The phenomenon of digital inclusion has empowered affected communities and, as a consequence, has become an important opportunity and a strategic cornerstone for many donors.[15]

Digital inclusion is an important enabler for improved situational awareness. Humanitarian actors can more easily extend their reach into affected communities through digital channels, either by directly engaging with them or by monitoring and analyzing user-generated content. Microblogs (e.g., Twitter) are widely used in countries such as Indonesia and the Philippines. Analyzing tweets containing disaster-related terms can enhance situation updates, and social network analysis can be used for information management purposes.[16] Social media analysis can also be useful for analyzing public reactions to media reports and unveiling the relative prevalence of topics. However, relying on social media-based feeds for assessing needs is still a bridge too far as, for example, the recent application of social media analysis in the 2015 Nepal earthquake showed.[17] Apart from user-generated content, there is also captured data, such as call detail records. This data can provide additional new insights[18], although until now it has not yet been used in real-time, but mostly for post-event analysis and research.

Existing organizations are transforming and adapting to seize these opportunities and mitigate the involved risks by developing innovative services. Several organizations aim at closing collaboration gaps by promoting and enabling open data and increasing data sharing. The United Nations Office for the Coordination of Humanitarian Affairs (UNOCHA) has developed the Humanitarian Data Exchange (HDX) to aggregate, store, and transform data for the humanitarian community. The World Bank runs the open data for resilience initiative and deploys GeoNode as an open data platform.[19] GeoDASH, the open source geospatial data management and visualization platform in Bangladesh, is an example of such an implementation.[20] Although initiated by the World Bank, it is now completely run by the government. GeoDASH is not specifically aimed at disaster management but does contain several related data sets. U.N. Global Pulse aims at accelerating discovery, development, and scaled adoption of big data innovation for sustainable development and humanitarian action. Social and private enterprises have entered the arena (e.g. Ushahidi, Google Crisis Response, and Microsoft Disaster Response), and Data NGOs (MapAction, Humanity Road, Nethope Crisis Informatics) offer Humanitarian to Humanitarian (H2H) services. Many mobile operators play an important role, such as Telenor and Orange in “digital philanthropy,” opening up their data sets and researching ways to analyze them. An important development is the widespread development of digital volunteers offering their time and expertise in microtasks and micromapping in order to analyze big data, such as social media posts and satellite images. The digital volunteers are active through a large variety of Volunteer and Technical Communities (VTCs), such as Humanitarian OpenStreetMap Team (HOT) and GISCorps, and also through some of the data-NGOs mentioned above. In the Missing Maps initiative, both remote and local volunteers work together to map the most vulnerable places in the developing world.[21] UNOCHA has co-created an umbrella organization that tries to bring all these volunteer organizations together so that they can be more easily activated through the Digital Humanitarian Network (DHN).

In terms of sensor data, satellite imagery has already been important for humanitarian response for a while through The International Charter Space and Major Disasters, where typically large commercial aerospace companies release costly satellite imagery for use by responders quickly after a disaster hits. Cheaper (nano)satellites and UAVs have the potential to democratize earth observation, since they enable more and more organizations and individuals to create openly licensed imagery. However, access to this kind of imagery is still difficult and limited. OpenAerialMap (OAM) addresses this issue by providing a simple way to search, share, and use open imagery.[22]

Challenges

Clearly, the data revolution brings along many challenges. First of all, it might result in the widening of the digital divide between developing and developed countries, as discussed already, but also within a country. A recent study by Pew Research Center showed (for the United States) that those with higher education levels and household income are more likely to use social media. Similarly, only the digitally literate know how to protect their data privacy, given that we are living in an “opt-out” online world.

The second challenge is determining how to use the enormous volumes of data for describing, predicting, diagnosing, or engaging. Twitter has over 400 million tweets a day and five billion crisis-related tweets since 2011. How does one make sense of this enormous haystack of posts? How does one understand the context from which posts are written? On top of this, the social media landscape changes continuously and -although Twitter and Facebook remain in the lead for the time being, different user groups discover and use new social media, with a tendency towards more closed group applications like WhatsApp.[23] Also, there are differences between countries, where, for example, in Russia, V.K. is more popular than Facebook. Social media analysis tools can usually only analyze mainstream social media and a limited set of languages (often in relation to the languages supported by Google Translate). Another challenge is formed by the fact that it is not possible to get access to all content. The “Twitter Streaming A.P.I.” releases only 1 percent of all the tweets produced on Twitter within a certain search window.[24] If one wants access to the full so-called Twitter firehose, then one has to pay. Similarly, satellite imagery, especially at high resolution, is only available at high cost and requires highly specialized expertise and/or artificial intelligence in combination with crowd-sourcing to make sense of it.

The third challenge is avoiding data harm. Data harm is the potential for negative impact on individuals, groups, and organizations when sensitive data gets disclosed, corrupted, or becomes unavailable when needed. What if personable identifiable information (resulting from the social media data or a combination thereof with other data sets) falls into the wrong hands? Or what if financial information about humanitarian response gets falsified, covering up fraud? How does one avoid negative impacts on individuals or certain community groups? One can imagine that damage to earning power or even direct physical damage, especially in case of complex emergencies.

The fourth overarching challenge is augmenting the skill sets of responding organizations to deal with the big data challenges of today and tomorrow. Most NGOs and IOs and some governmental organizations acknowledge the importance of information management as a separate discipline if they want to be able to deal with this new data and information environment adequately. But they are not always sure how to embed it in their operations. Who should they appoint as part-time or full-time information management officers? Should they train data scientists to become humanitarians or the other way around? Should they become social computing organizations themselves with, for example, social media monitoring skills? Or should they develop only a basic social computing understanding in-house so that they can leverage digital volunteering and community initiatives better and/or outsource cost-effectively to data science companies during relief efforts? These limitations impact their ability to change and innovate. As a result, there is a demand for guidance on how to leverage data to the benefit of these organization. More specifically, guidance is required on responsible operational data collection and use, particularly in crisis settings, when life-saving, time-sensitive decisions must be made with imperfect or scarce data. In addition, best practices need to be promoted for collecting, storing, and sharing data in an ethically responsible manner.

Conclusions and Recommendations

The data revolution brings disaster responders a whole new spectrum of data, the full potential of which still must be discovered and exploited. It is beyond doubt that the role that data plays in humanitarian policy and operational response needs to be strengthened and accelerated so that aid can be better targeted and create more impact. Digital technologies are very effective in providing a fast and increased reach to remote and previously unconnected communities and subsequently empowering them. Digital inclusion is essential to overcome the last mile of the information gap, to increase timeliness and data granularity.

The information needs of disaster responders, whether professional responders or responding community members, should be leading in this endeavor. Information needs at local, national and international levels have to be determined so that all available data can be collected, collated, and integrated from multiple stakeholders, for example, to establish a baseline on livelihoods and vulnerabilities beforehand, so that any remaining information gaps can be quickly identified once the disaster unfolds. The multitude of crisis impact assessments pinpoints a collaboration gap (i.e., a lack in harmonization and data-sharing, especially between local and international responding organizations). Similarly, there is often a collaboration gap between well-meant remote digital volunteering initiatives and national and local realities. DHN, as explained before, is the framework between international formal responders (such as U.N. agencies, IFRC, and large NGOs) and digital volunteers; however, there does not seem to be such a framework between national governmental responders and the international community, specifically in terms of data preparedness. Strikingly, we also see a flurry of big data-related initiatives from a variety of actors in the humanitarian ecosystem that are often uncoordinated and fragmented.

Creating strategic partnerships and building relationships between national and international organizations is a protracted process that requires patience and perseverance. This fails to match well with often short-term response activities, where, for example, remote digital volunteers can only devote their free time, usually in periodic bursts. Local ownership is, in the end, key for data initiatives to become sustainable and utilized to their fullest extent and can also lead to a better definition of information needs. Identifying information needs beforehand can also assist in tailoring the assessments further and in minimizing the amount of data to be collected. The Humanitarian Exchange Language, a simple standard for messy data, and geospatial platforms offer ways to make humanitarian data easier to access, share, and use.[25]

For data-poor contexts in developing countries, it will be important to find the right balance between using resources to obtain and analyze big data and using them to digitize local, often paper-based, data collection processes. In order to benefit from new data, new capabilities, such as how to analyze and govern the vast volume of data, have to be developed, both by the local, given the importance of linguistic and cultural analysis, and international community. It will also be essential to balance human and digital proximity. Too much focus on the digital part might lead to a misleading notion of having solid data, whereas a large group of the affected population might not be represented in the dataset. Social media conversations can never yield an in-depth understanding of local sentiments and cultural issues to the extent that face-to-face communications do.

Professional responders, whether international NGOs from the Global North or national and local organizations from the Global South, should develop a coherent data strategy, a digital roadmap of how to include big data into the disaster management cycle and into their internal processes. NDMAs can play a key role in developing such a comprehensive digital strategy in order to benefit to the fullest from the data revolution and are able to develop and build an ecosystem of digitally literate stakeholders essential for disaster preparedness and response in their country.

Altogether, we recommend a strong focus on data preparedness through capacity building, regular multi-institutional mapping of data sets on information needs, standardization, and creating close partnerships to close information and collaboration gaps, especially between the Global South and the Global North.

[1] Centre for Research on the Epidemiology of Disasters Emergency Events Database (EM-DAT), "Disaster Data: A Balanced Perspective," CRED Crunch 41 (2016), http://cred.be/sites/default/files/CredCrunch41.pdf.

[2] "Natural Disasters in the Middle East and North Africa: A Regional Overview," The World Bank, March 6, 2014, accessed July 22, 2016, http://www-wds.worldbank.org/external/default/WDSContentServer/WDSP/IB/….

[3] "Big Data for Climate Change and Disaster Resilience: Realising the Benefits for Developing Countries," Data-Pop Alliance, September 2015, accessed July 22, 2016, http://datapopalliance.org/wp-content/uploads/2015/11/Big-Data-for-Resi….

[4] “Unprecedented US$21.6 billion Needed to Assist 95.4 Million People Across 40 Countries,” United Nations Office for the Coordination of Humanitarian Affairs (UNOCHA), June 22, 2016, accessed July 24, 2016, http://www.unocha.org/top-stories/all-stories/unprecedented-us216-billi….

[5] Katie Harris, David Keen, and Tom Mitchell, "When Disasters and Conflicts Collide: Improving Links between Disaster Resilience and Conflict Prevention," Overseas Development Institute (ODI), February 2013, accessed July 22, 2016, http://www.droughtmanagement.info/literature/ODI_disaster_resilience_co….

[6] Marc van den Homberg, Robert Monné, and Marco Spruit, “Bridging the Information Gap: Mapping Data Sets on Information Needs in the Preparedness and Response Phase,” Tech4Dev UNESCO conference May 2016; "Information Analysis in the First 72 hours," Humanitarian Networks and Practitioners Week Taskforce, accessed August 1, 2016, http://hnpw.org/?speaker=information-analysis-in-the-first-72-hours.

[7] The Data Poverty Index is determined by internet speeds, computer owners, internet users, mobile phone ownership, network coverage, and higher education.

[8] A. Wahed et al., “Flooding in North-Western Bangladesh HCTT Joint Needs Assessment,” Relief Web, September 8, 2014, accessed July 22, 2016, http://reliefweb.int/sites/reliefweb.int/files/resources/0809_NW_Floodi….

[9] M. Van den Homberg et al., “Coordination and Information Management in the Haiyan Response: Observations from the Field,” presented at Humanitarian Technology: Science, Systems and Global Impact 2014, HumTech2014 at MIT, Boston.

[10] "Humanitarianism in the Network Age," United Nations Office for the Coordination of Humanitarian Affairs (UNOCHA), accessed July 24, 2016, https://docs.unocha.org/sites/dms/Documents/WEB%20Humanitarianism%20in%….

[11] Marc van den Homberg and R.M. Neef, “Towards Novel Community-based Collaborative Disaster Management Approaches in the New Information Environment: An NGO Perspective,” International Disaster and Risk Conference proceedings, Davos 2014 and Planet@Risk Global Risk Forum Journal 3, No. 1 (2015).

[12] "The Four V's of Big Data," IBM Big Data and Analytics Hub, accessed July 24, 2016, http://www.ibmbigdatahub.com/infographic/four-vs-big-data.

[13] Katie Whipkey and Andrej Verity, “Guidance for Incorporating Big Data into Humanitarian Operations,” Digital Humanitarians Network, September 2015, accessed July 24, 2016, http://digitalhumanitarians.com/sites/default/files/resource-field_medi….

[14] "A World that Counts: Mobilizing the Data Revolution for Sustainable Development," Independent Expert Advisory Group on a Data Revolution for Sustainable Development, November 2014.

[15] "Digital Inclusion Fact Sheet," USAID, accessed August 1, 2016, https://www.usaid.gov/news-information/fact-sheets/digital-inclusion-fa….

[16] Sheila Bonito, “Data Mining in Twitter, Supporting DRR during Typhoons,” University of the Philippines Open University, Poster at UNISDR conference, 2015.

[17] Sarah E. Vieweg, Carlos Castillo, and Muhammad Imran. “Integrating Social Media Communications into the Rapid Assessment of Sudden Onset Disasters,” Proceedings of SocInfo, 2014; “Lessons Learned: Social Media Monitoring during Humanitarian Crises,” ACAPS, September 21, 2015.

[18] Xin Lu et al., “Unveiling Hidden Migration and Mobility Patterns in Climate Stressed Regions: A Longitudinal Study of Six Million Anonymous Mobile Phone Users in Bangladesh,” Global Environmental Change 38 (2016): 1-7. 

[19] "Open Data for Resilience Initiative (OpenDRI)," Global Facility for Disaster Reduction and Recovery (G.F.D.R.R.), accessed July 24, 2016, https://www.gfdrr.org/opendri.

[20] "GeoDash," Government of Bangladesh, accessed July 24, 2016, http://geodash.gov.bd/.

[21] Examples include the American Red Cross, British Red Cross, Humanitarian OpenStreetMap, and M.S.F.

[22] O.A.M. launched in the summer of 2015 in support of humanitarian response and development mapping projects through the Humanitarian OpenStreetMap Team (HOT). O.A.M. is growing the community of contributors; currently there are thousands of images freely available for use.

[23] Cara McGoogan, "People Sent Three Times as Many Messages on Facebook’s Messenger and WhatsApp in 2015 As They Did Via SMS,” The Telegraph, April 22, 2016, accessed July 24, 2016, http://www.telegraph.co.uk/technology/2016/04/22/end-of-sms-whatsapp-an….

[24] Fred Morstatter et al., “Is the Sample Good Enough? Comparing Data from Twitter’s Streaming API with Twitter’s Firehose,” Association for the Advancement of Artificial Intelligence, 2013, accessed July 24, 2016, http://www.public.asu.edu/~fmorstat/paperpdfs/icwsm2013.pdf; “What Percentage of Tweets Generated During a Crisis Are Relevant for Humanitarian Response?” iRevolutions, November 14, 2012, accessed July 24, 2016, https://irevolutions.org/2012/11/14/percentage-tweets-response/.

[25] "Humanitarian Exchange Language (HXL)," Humanitarian Language Exchange, accessed August 1, 2016, http://hxlstandard.org/.