Observations from Active Standby – Open Data, Mobility and Community Indicators

This week CrisisCommons has continued to remain on “active standby.”  This has meant that we have encouraged volunteers to sign up to provide assistance and we continue to be open for requests from crisis response organizations. We have  focused our efforts on activities which support search and information gathering.  We began the week supporting the UN Office for Coordination of Humanitarian Affairs by gathering Common Operational Datasets. As Japan began their requests for support, there was limited engagement asked of the UN. As a result, CrisisCommons made a decision to continue contributing to the wiki with the aim to provide the UN with a series of recommendations of how volunteer technology communities could contribute open data sources which can support relief agencies – both domestic crisis response agencies such as FEMA’s Emergency Support Functions or international humanitarian relief agencies under the United Nation’s Cluster system. Both have similar needs but different ways of collaborating.  For example, just this week we have recommended the review of the ISAC Common Operational Dataset Guidance to FEMA’s Situational Assessment Planning team for the upcoming National Level Exercise.

One of the most common questions we received this week was, “Why isn’t the response like Haiti?” and “Why aren’t there hundreds of CrisisCamps around the world creating projects?” One of the fundamental differences is that Japan is a country who has a well established emergency response system and that we seek to support operations of crisis response organizations who are operating on the ground. Another has been, there has not been a well established network of CrisisCamps created before the crisis which makes it hard to support community efforts from abroad. With both the New Zealand and North American Blizzard support we had relationships with local Camps who was able to support local efforts. This perhaps is one of the most important lessons learned we observed during Haiti and we endeavor this year to grow capacity in regions like Asia, India, Oceania, Africa, the Middle East and Central Asia. There is much work to do, but we very much want to support local efforts so that volunteer work is aligned with response systems who operate in the best interest of their people.

Another  fundamental difference between Japan and Haiti is that Haiti was and continues to be a developing country. Japan is one of the most technologically advanced nations in the world who themselves sponsor development and capacity building within their own region. With regard to their emergency management capacity, Japan one of the most robust domestic emergency management systems, notwithstanding their focus and investment in community preparedness. People knew what to do if they had warning.  Even with the best of systems, there are always unknowns. There can be capacity challenges. Even the best of systems can be overwhelmed. There can also be great concerns such as the activities surrounding the Fukushima nuclear power plant. To date, the Government of Japan has an operating crisis response system which has asked very little of the international humanitarian relief community.

“The Government of Japan has received offers for assistance from 91 countries, and has accepted assistance from about 15 countries based on assessed needs, which is mostly specialized international urban search and rescue (USAR) teams and medical teams.”
– 3/17 The Chronicle of Philanthropy

When a country has an emergency system, CrisisCommons seeks to support that system and affiliated partners such as NGOs operating in the field. This leads to one of the findings from the CrisisCongress last year is that we must create relationships and build engagement before a crisis occurs to “help the helpers” as well as work with response agencies and organizations after the crisis when they can plan to coordinate. This is what CrisisCamp does best, during non-crisis times camps can bring together communities, foster relationship building and understanding of needs before a disaster. Its an old adage in the crisis management field, “The time to exchange business cards is before a crisis.”

The Value of Open Data Sets

This week, volunteers from around the world have been coordinating through a Skype chat to find open data sets on the internet. This can be from an RSS feed of a local prefecture government’s emergency information to a spreadsheets of mobile charging stations and free phone access at local community centers.  This information can be largely from authoritative sources such as government and academic information or can be completely user-generated using common spreadsheet, document and map based tools.

We especially found that open data sets, which are likely be the most valuable and available during the immediate response, were comprised of infrastructure status information – what have begun to call Community Indicators. First and foremost, people want to share their own personal status that they are okay and to check on loved ones. This is the most paramount need. After connecting with loved ones, people are interested in finding resources such as, Where is a local phone available? What are the conditions of the roads? What stores are open? Where can I find shelter or food? In Japan, this specifically has been a reoccurring theme. We have seen people create spreadsheets that show where people can receive a free meal to where power is cut off. This isn’t crowdsourcing, these are people sharing what they know with their local community. It’s local information for local people. People want to know what’s happening and what they should do.


Volunteers continued throughout the week trying to focus on finding more open data sets and resources. They were joined by a team from the GIS Corps who began to explore prototyping open data layers from the data sets found from the GIS group and the Japan Data Profile. Volunteers quickly ran into simple challenges that there could be a list of resources by name and address in text which really needed to have latitude and longitude to enable that information to be visualized (i.e. placed on a map). Even if they placed on a map, that information isn’t mobile. Yet another challenge. They also ran into the inability to have basic GIS resources of the area such as area boundaries of the local jurisdictions. Again, another, “It would be great to have this before the crisis” resource.



There are a few preliminary observations from this week, but as we look forward to collecting perspectives and input from across the community in the coming weeks through our online survey and community after action activities. Here are a few initial thoughts:

  • Operational datasets need to be found and a gap analysis should be performed before a crisis.
  • “Community Indicator” datasets need to include critical infrastructure and key resource data sets. Owners and operators of CI/KR facilities should make as much data available, in formats which are sharable and ideally mappable.
  • Many local authorities who continue to not use basic information sharing such as RSS.
  • Telecom providers and utilities have limited ability to share their network operational outlook with their customers.
  • Dependance of PDF to share official information and situational reporting from across the response ecosystem. This format, where widely used, does not share well. Information (i.e. data elements) need to flow across the ecosystem where organizations can pull from these resources. When situational awareness information is locked in a PDF, its very difficult to share broadly (even if its open and public).
  • Missing persons data is still a significant challenge. Questions such as “Who is the lead for missing persons data in this crisis?” still come up. Additionally, communities have many languages which are spoken, missing persons information should be able to engage people of many different languages.
  • Even when imagery is open (via the Space Charter), communities can have capacity challenges to fully be able to utilize the resources to the fullest extent of their availability.

Think Different

This week we had a question asked to us by another NGO which focused on finding the status of the communications in the area. Volunteers searched for a map which had all the telecommunications providers which would show the status of the communications infrastructure. This, in a sense, sounds like a simple request. But as volunteers dove into the task, it became a challenge to find information that would give status on each telecommunications company, in a way that could be used with other data. Carrier coverage maps were in PDF, there was no uniform coordination of how each carrier’s status information could be put together with others. To date we are still working it.

In the discussions, we thought about looking at what the NGO really wanted to know.  In particular, they wanted to know where there was an absence of service in within Japan so they could appropriately provide satellite communications in needed areas to their partners.  Ideally what we would have liked to see is that each of the telecommunications carriers could provide some kind of dashboard data feed to provide service indictors such as “Working, Not Working” (and a few gradients inbetween).  What became interesting is that we started looking at the problem in reverse. We had a PDF of the service areas (would be better if those had lat/long, better it not be in a PDF). The public could see that data was flowing from people using social media platforms. What if you had the service areas mapped and layered on that where people were talking using social media. The areas not covered would be those most likely to not have power and/or damage to their cell tower infrastructure. You wouldn’t need the telecommunications companies to provide that information. This became an interesting hypothesis but one that was not executable for a few reasons. We would have to have access to the “firehose” data from social media. This would have to be set up before hand. We would have to have power enough to crunch the data, again more “before the crisis” preparation and finally people would need to turn on their geolocation so that information could be mapped. UCLA was archiving twitter data and said that a majority of the data could not be mapped as is. This reminded us of Craig Fugate’s Waffle House postulate – where he uses indictors such as if the Waffle House is open, they have power and water. This just was a telecommunications application of this theory. If people are talking, communications and data are flowing (may be slow, but something is getting through). How can we find the areas where little via mobile is getting through. Those are likely the most challenged areas.

Through working these options to a conclusion, we learned that it may prove helpful to have some kind of Community Indicator Dashboard, where communications providers and other owners of critical consumer information can share a common status indicator which could be aggregated to give trends analysis.  We saw the data elements of these indicators mapped where during the New Zealand response EW.ORG.NZ volunteers provided a great amount of data provided by local infrastructure providers. For example, the local banks began to update their own feed of where ATMs were up and running.What if critical infrastructure like power, telecommunications and water providers and key resources like major retail could coordinate before the crisis to provide status updates through data feeds to the publics? It’s real time, its edited by the data source. It can be replicated on any mapping platform. By finding that baseline data before the crisis could create a better picture when crowdsourced data is layered to enable local response agencies and humanitarian relief organization to filter and aggregate based on their operational needs.

Not All Data is Open

The initial UN OCHA task prompted us to explore how much “free and open data” could be found that would apply to their operational activities. We found that many of these “Common Operational Datasets” were mostly data from “authoritative sources” such as governments, critical infrastructure and academic institutions have and provide in the public space such as population and transportation data. This week volunteers ran into data that may have been helpful, but it was not open. The private sector and academia own (or have restrictive use) a significant amount of data, especially that of supply chain capacity.  Crisis events are most known for generating vast amounts of user-generated, often unstructured, data that are the eyes and ears of the public. When aggregated together, the user-generated view is often one that reflects immediate concerns and needs of the public. They are sharing what is happening around them. This information it not only people in a personal capacity but those of organizations seeking to provide dialogue and assistance to their communities or customers.  As much as a family is looking to return to their home, a business owner or a supply chain distributor is looking to return to their business. Local information becomes paramount in every crisis.If there is existing data that response organizations know they will need before the crisis, how can collaboration happen to ensure that data is available to those who need it most?

Optimizing for Mobility

One of the findings this week is that many emergency information resources are not optimized or have the capacity during a crisis to optimize their content for mobile. This may be the last mile in crisis content delivery –  the ability to have crisis information on the device that a vast majority of world carries in their pocket – their mobile phone.


When Jan Chipchase gave his 2007 Ted Talk, he spoke of what do people carry when they walk out the door – keys, money and a mobile phone. During a crisis, a mobile phone is the connectivity between the public and emergency information – and most importantly it is the connection between people and their families. So it should be no surprise one of the first user-generated data sets was a spreadsheet with the locations of where people could find “Mobile Charging Stations.” This is the kind of information people want, so much so, they create a list of where people have shared where the stations have been created – either by companies or by individuals who have power and power strips. The image on NHK World of people standing while their cell phone was charged is a powerful reminder that posting on the web is not enough – the content must scale to mobile information delivery.

One of the first teams who wanted to volunteer to help CrisisCommons create a mobile team to support any mobile requirements was the mobile developers from the Appcelerator mobile development community.  We look forward to building a Mobile Community of Interest later this spring to expand how mobile can be better utilized to provide crisis information (both with connectivity and without) to not only help today but to advocate for the optimization of mobile crisis information to (and from) the public.

Data Preparedness

This week we have begun to think of how pre-positioned “before the crisis” data sets and expertise can provide greater capability to local response agencies and organizations as well as to the public so that technology volunteers can support requests or give greater capacity. We identified the following:

  • Before the Crisis: Public repository of pre-identified “Common Operational Datasets” which come from public sources such as census records or land use data
  • Before the Crisis: Pre-planned and exercised community indicator data feeds from critical infrastructure and key resources. These feeds can be created and be operating 24/7 365 days a year. This dashboard would give status updates on power, communications, schools, grocery stores, weather and other local community needs. This would have a mobile distribution.
  • During the Crisis: Collection of crisis-generated operational datasets from community indicator sources as well as authoritative source such as imagery information which are unique to the crisis at hand. These are unforeseen data sets and resources which can provide greater community awareness of crisis information.
  • During the Crisis: Collection of user-generated data sets. This can be citizens reports via social networks. crisis maps, lists and other data sources which citizens create to share what they know with their local community.

Crisis Cloud

Again, this week we have seen the need to provide temporary development space for volunteer technology communities during crisis events. Over the last year, the CrisisCommons Infrastructure Team have been driving the development of not only more steady-state infrastructure support for CrisisCommons but also to provide a needed space during crisis events which may be needed to shore up capacity to for a temporary time period. This could be hosting full PHP instances of Ushahidi maps to providing computational capacity. We all feel very strongly about offering this kind of support to the community. We are currently moving to the Oregon State University’s Open Source Labs and look forward to building more capacity for the community.


This past weekend and next week volunteers in Chile (Santiago) and the United Kingdom (Brighton) are hosting CrisisCamps to support the Japan efforts as well as last Friday’s CrisisCamp in Silicon Valley. Check out what they did in Chile this past weekend from CNN Chile:


Pedro Pablo Fuentes Schuster of Digitales por Chile on CNN Chile

Next Steps

This week we will continue to keep the Skype chat open and volunteers will begin to translate over sections of the  Japan Data Profile. We will continue to keep our volunteer request and project request open for contribution by people wanting to volunteer and by those crisis response organizations or agencies who may want to request specific project. There are many crowd sourced projects is underway from volunteer technology communities. Here are a few who may need more assistance:

There are great resources online (and on the wiki) such as:

Maps, Imagery, Of Interest

For more information visit the wiki page for a further listing of projects.

Working Together

The amazing part of CrisisCommons will always be the people. To the 250 new volunteers who have signed up to be at the ready to help as well as CrisisCamp veterans who consistently stand up to volunteer. We have not received further requests from response agencies to date and have decided to begin to wind down data collection for the Japan Data Profile and to remain ready to assist if a request is issued. Until then, we would like to say a word of thanks to all of our volunteers and people who remained on active standby this week.

  • To all of the volunteers who have been part of the Skype chat coordination supporting the collection of data set – Thank You! In particular, Thanks for bearing with us, keeping focused and for doing the great things you do. We do want to highlight Julie Wolf. We had never met a Skype Coach before, but her dedication and time to helping onboard new volunteers and her infectious enthusiasm and long nights have been greatly valued by us all. To Ted Han, David White and Dan York for their consistent support this week, for their infrastructure expertise and their willingness to not only volunteer this week but all three have joined the Infrastructure Team. We are happy to have you aboard. Thanks again.
  • To the CrisisCommons Infrastructure team, especially Deborah Shaddon and Spike, for weathering the Dreamhost challenges this week and for your constant guidance towards to our infrastructure goals.
  • To Andrej Verity and everyone at the United Nations Office for the Coordination of Humanitarian Affairs for their continued guidance and interest in incorporating volunteer technology communities like CrisisCommons to provide productive support both in steady-state for preparedness and mitigation efforts as well as crisis events. We look forward to working further with you.
  • To Shoreh Elhami, Co-Founder of GISCorps, who offered CrisisCommons a team of GIS volunteers to help collect and visualize datasets. This is the first time we have worked with the GISCorps – and it won’t be the last! To the entire GIS Corps team led by Yoh Kawano at UCLA, thank you for your time this week and the constant engagement.  The open data layer prototype looks really good. Many thanks to all the volunteers who spent time geocoding data!
  • To CrisisCamp veterans Pedro Pablo Fuentes (CrisisCamp Chile) and Jeannie Stamberger (CrisisCamp Silicon Valley) who have created camps in support of the efforts this week. To Heather Leson, Pascal Shuback and David Black for their time providing their experiences to our new volunteer friends and for taking shifts so that others could run out for dinner.
  • To Johnny Diggz and Willow Burgh at Geeks without Bounds. To Willow for her leadership last week to coordinate and keep us all on task and to Diggz for believing in crazy ideas and introducing us to his API experts.
  • To Patrick Meier and Jen Ziemke from Crisismappers and Anahi Iacucci of the Standby Task Force for for their unwavering support and friendship with CrisisCommons. We are grateful for the support of Crisismappers and members of the Standby Task Force, like Humanity Road, who have come from near and far to help collect data sets. We also appreciate the willingness to provide opportunities for translators to assist with the Fletcher School support of the Japan Crisis Map and the Japanese translation of the Ushahidi manual.
  • To FEMA’s Digital Engagement Director, Shayne Adamski for his support in directing us to Jesse Thomas as USAID who then connected us to Interaction. We appreciate your help!
  • To Bob Leipold and that staff at NVOAD for making sure that we are in the loop. Appreciate the inclusion and for us to be a member of the NVOAD Technology Task Force.
  • To Jeff Haynie, Scott and the entire mobile development community at Appcelerator for continuing to be ready to make crisis data mobile! We look forward to working with you in the future to build a mobile community to support community preparedness as well as optimize crisis data during emergencies. Thanks again for reaching out and being great to work with.
  • To Google’s Crisis Response, especially Christaan Adams, for keeping us in the loop of when new imagery is shared. We look forward to working with Google in the future to better understand how technology volunteers can utilize Google’s (and other providers) Crisis Response team’s products.
  • To Microsoft, especially Patrick Svenberg and Claire Bonilla, for offering their support of Microsoft tools to the community. We look forward to talking with you during steady state to see how technology volunteers can better utilize Microsoft crisis response tools and products.
  • To Oregon State University’s Open Source Lab, especially Jeff Sheltren and Lance Albertson, for working us with to have a machine “on the ready” for use by our community. We are looking forward to making OSUOSL our new infrastructure home base and helping grow more resources for HFOSS projects.
  • To CrisisCamps in Silicon Valley, Brighton and Santiago, we thank you for for your community’s support and look forward to posting after action reports from your events to the commons.
  • To NetHope and MapAction for your interest and support of how CrisisCommons may be able to support crisis response organizations.
  • To BBC Click, Voice of America, Germany’s Public Radio, CCWB in DC and the publications who have reached out to learn more about what we are doing. We appreciate your support.

On behalf of the CrisisCommons community, we continue to extend our thoughts and prayers to the people of Japan. Again, we appreciate everyone’s volunteer time, outstanding skills and support over the last week. We will continue to provide updates throughout the week regarding our standby status.

Thanks again to everyone who has pitched in to help.

With gratitude,


Heather Blanchard
Co Founder

About Heather Blanchard

Heather Blanchard is a co-founder of CrisisCommons and CrisisCamp.

Comments are closed.