Quantifying Chaos: How Big Data Has Transformed Natural Disaster Management

In the past few years, deadly weather events have accumulated in the United States like rain in a bucket. As the COVID-19 pandemic raged, wildfires burned across the West and tropical storms battered the coasts. Among the deadly events were Hurricane Ian, which razed communities in Western Florida en route to becoming the third-most destructive weather disaster on record, and the August and Dixie wildfires, which were the two largest in California’s history. There was even a massive winter storm in Texas, of all places, which killed 236 people and left 10 million more without power.

If it feels like there are more natural disasters now than there used to be, that’s because it’s true. Climate change is behind a fivefold increase in the number of weather-related disasters during the last 50 years, according to the World Meteorological Organization (WMO). Between 1970 and 2019, meteorologists have documented approximately 11,000 disasters – many of which showed a “significant human influence.”

“The number of weather, climate, and water extremes are increasing and will become more frequent and severe in many parts of the world as a result of climate change,” explained WMO Secretary-General Professor Petteri Taalas in a statement.

“That means more heatwaves, drought and forest fires such as those we have observed recently in Europe and North America. We have more water vapor in the atmosphere, which is exacerbating extreme rainfall and deadly flooding. The warming of the oceans has affected the frequency and area of existence of the most intense tropical storms.”

That’s the bad news. But there’s good news, too. While the number of disasters has increased by a factor of five, according to WMO, deaths from disasters have decreased by a factor of three due to an increase in early warnings and an improvement in disaster management.

Real-time data deserves a lot of credit. While they can’t prevent natural disasters from occurring, communities that call on the power of big data can predict and contain them.

Indeed, big data analytics has become a cornerstone of emergency management. When it’s broken out and analyzed via cloud computing, information about emergencies that are culled from satellite imagery, drones, social media and sensor networks can enable vital decision-making before, during and after a catastrophe strikes.

Satellites Save Lives from Space

When disasters unfold, rescuers can use real-time satellite imagery to see what’s happening on the ground and find people in need of assistance – on rooftops, for instance, or in the middle of a raging river. Meteorologists, meanwhile, can use satellite images to spot disasters as they’re forming. If they see early signs of a tropical storm or the plumes of smoke belching from an active volcano, they can issue pre-emptive warnings so people can evacuate or seek shelter.

Artificial intelligence can help make sense of views from space. Using deep learning and computer vision, it can identify areas that are vulnerable to extreme weather, recognize everything from ominous cloud formations to survivors, and categorize these heaps of data so weather services and responders can easily follow events in real time.

The Japanese government was among the first to recognize the utility of big data in disaster management. After a series of deadly landslides in 2014, it created a system that uses AI and satellite imagery to predict imminent landslides.

The Japanese system juxtaposes big data from its sensors in heavy rainfall-prone areas with satellite imagery collected by the Japan Aerospace Exploration Agency. AI software continually analyzes the images, then pairs them with sensor data and weather predictions to pinpoint areas at risk of floods and landslides. Local governments can then issue early warnings and, if warranted, issue life-saving evacuation orders.

Drones Survey the Scene

In 2005, Hurricane Katrina devastated parts of Louisiana and Mississippi. Immediately afterward, emergency responders deployed two unmanned aerial vehicles (UAVs) for the first time during a natural disaster to search for trapped survivors in Pearlington, Mississippi.

The first was a 4-foot-long airplane with mounted video and thermal imagery cameras that captured details from as far away as 1,000 feet. The second was a miniature electric helicopter that hovered closer to the ground, taking images of rooftops and even peering into windows.

After just two hours, rescuers determined that no one was trapped and that the floodwaters from the cresting Pearl River posed no additional threat.

In the 16 years since Katrina, drones have gone from experimental to essential and are now smaller, cheaper and more advanced to become a routine part of any search-and-rescue mission.

Case in point: the July 2018 Carr fire, which burned 229,651 acres in northern California and killed eight people. During the fire, officials in Redding, California, deployed drones to map the devastation from the skies and to find areas where the fire was still burning.

Greg Crutsinger of drone data consulting service Scholar Farms was part of the effort. His drones took 360-degree panoramic images at the ground level, creating something akin to Google Street View for emergency responders. Crutsinger now works for private satellite company Planet.

“When I first saw the panoramic images, I was shocked at what they showed,” Crutsinger said. “Being able to pan around and see the destruction on the ground, when I was only used to mapping the area from above, had a big impact on me.”

With a close-up view, Crutsinger discovered that neighborhoods varied in the damage they’d received.

“Some looked post-apocryphal – one where a tornado touched down during the fire was just incinerated,” said Crutsinger, who shared his intelligence with the Redding Police Department, who in turn shared it with evacuated residents to prepare them for what they would see when they returned home to the destruction.  

Social Media Sheds Light

In the chaos of a disaster, what responders need most but often lack is current, accurate information. Social media might be the key to getting it, according to Muhammad Yasin Kabir, a professor of computer science at the Missouri University of Science and Technology.

At the 21st IEEE International Conference on Mobile Data Management in 2020, he and his colleagues presented the prototype for a system that can help coordinate rescue operations by compiling tweets from people affected by disasters.

Called STIMULATE, the cloud-based system uses machine learning to aggregate tweets in a methodical way. When it finds a given keyword, it collects the tweet and filters it into categories that include: rescue needed, DECW (diseased, elderly, children, and pregnant women), water needed, injured, sick, and flood. The system then prioritizes rescue efforts by aggregating different factors, such as weather, GPS location, the type of help needed and the number of rescuers available

By extracting the data of natural disasters from social media posts, tools like STIMULATE can detect and track weather hazards, and determine from whom information is coming – a person affected by the disaster, a government official, the media or an emergency responder.

Some systems can even distinguish fact from rumor so that precious resources can be directed to the people and places that actually need them. Researchers at the University of Virginia and Arizona State University, for example, have a mathematical method to track and aggregate common topics posted to social media during disasters, filtered by location. The technique identifies common and distinct topics, to the point that misinformation – for instance, a post from a person who clearly isn’t affected but purports to be – stands out and can be flagged.

Sensors Sound Early Alarms

While drones and tweets can provide critical information after a disaster, the Internet of Things (IoT) can provide critical information before a disaster.

A compendium of cloud-based sensors located on real-life objects, including everything from streetlights to self-driving vehicles, the IoT collects real-time information about the environment that can be converted into early disaster warnings with the help of machine learning. Water-level monitoring sensors can catch early signs of flooding in rivers or streams, for example, while earth-monitoring sensors near fault lines can report minor rumblings in advance of an earthquake.

For wildfires, the future is “smart firefighting” where a variety of technologies can be leveraged to collect on-the-ground information and generate actionable real-time intelligence that can transform the scene of an inferno.

One company, the Manx Technology Group, is developing sensors that can detect elevated levels of carbon dioxide in the air and pick up on temperature changes from miles away, which could help with detecting wildfires.

“The problem with forest fires is that the forests are usually remote, abandoned and unmanaged areas filled with trees, dry and parching wood, leaves, and so forth that act as a fuel source,” said Ahmad Alkhatib, who studied the issue as part of his Ph.D. work at the University of South Wales.

Sensor equipment can help, said Alkhatib, who reports on ForestWatch, an optical camera sensor system from EnviroVision Solutions. The system uses a tower camera that scans the area for smoke during the day and fire glow during night. It detects smoke from up to 12 miles away.

The sensors that Manx is developing will work from even farther away, according to CEO Joe Hughes.

Some sensors measure air quality to immediately detect dangerous levels of pollution or chemicals, like smoke or radiation. In April 2017, for example, the Environmental Protection Agency (EPA) launched its Wildland Fire Sensors Challenge, which called for competitors to propose a sensor system that would allow firefighters, first responders, and government agencies to continuously and easily measure wildfire smoke exposure. The winning SenSevere system from Sensit Technologies includes a solar-powered battery that can last for three weeks on one charge.

From Storm Clouds to Cloud Computing

To emergency responders, clouds are often harbingers of catastrophe. Satellites, drones, social media and the IoT, however, are using cloud computing to capitalize on big data and advance the world of disaster management. And instead of threatening lives, this kind of cloud is actually saving them, according to Crutsinger.

After his experiences with California wildfires, he created a system that merges mapped images with panoramic views captured using off-the-shelf hardware. To make sense of the data, he uses the Hangar 360 platform designed for industrial data collection.

With the tap of a button on his smartphone, Crutsinger can now deploy a drone to take pictures from many different angles during a disaster. When the drone lands, it transfers photos to his smartphone, and then to the cloud for processing. In less than 20 minutes, the system provides a web link to a 360-degree, high-resolution panoramic of the scene that authorities can consult and share for situational awareness.

“I’ve flown drones through disasters, like the Haiti earthquake, where there is no internet infrastructure,” explained Crutsinger, who in those cases, downloaded data to a physical hard drive. 

“The hard drive is taken to the closest location for processing. [In those instances] disaster relief efforts … are greatly slowed.”

Those missions have made him thankful for the cloud – where information can be uploaded and analyzed quickly enough to not just survey the damage but to actually mitigate it, crucially saving countless lives in the process. 

That might make the next hurricane and wildfire season a little less scary.

This is an update to the original article published February 23, 2022.

Jean Thilmany is a freelance writer living in St. Paul who writes about engineering and technology.

© 2023 Nutanix, Inc. All rights reserved. For additional legal information, please go here.