National Guardsmen assist residents affected by flooding from Hurricane Harvey in 2017. … [+]
The National Guard has its work cut out for it. As first responders to wildfires, floods and other increasingly frequent disasters, National Guardsmen must deploy quickly and precisely where needed.
This is one of the most significant challenges imaginable for the information technology industry: Of the many square miles affected, which school, bridge or neighborhood is most in need of immediate help? Often, lives are at stake.
Alphabet, Google’s parent company, has a factory called X that aims to solve some of the world’s most critical and thorny problems. X has developed a revolutionary solution, using predictive AI in the same way many companies do. Here’s how X did it, and what all business professionals can learn about combating uncertainty and risk with machine learning.
The problem: labeling aerial photos
During and after a severe weather event, drones and manned aircraft collect thousands of aerial photos of affected areas. These images can reveal which buildings and other infrastructure were hit, but only after each image is precisely tagged to indicate the exact location. Unfortunately, the images typically lack this metadata.
Manually tagging photos greatly slows down the National Guard’s response. After an incident, its team typically needs about 12 hours to complete the task. Unfortunately, this process has so far remained manual. It is a difficult task to automate because the photos are taken at different altitudes and at oblique angles.
But this is exactly the kind of problem X was designed to solve: the stakes couldn’t be higher, but it requires a technological breakthrough. X’s initiative to tackle this and other related challenges is called Bellwether, described as “the first prediction engine for Earth and everything on it.”
Sarah Russell, who has led Bellwether since its inception in 2020, explains: “We took on this challenge because we realised that if we could solve it, we would reduce response times to climate disasters and multiply the number of lives saved.”
The solution: Matching photos using machine learning
The technological breakthrough? Matching real photos with artificial ones. Bellwether synthesized a database of simulated reference photos to use as examples. When a real photo matches a photo in the database, it is labeled, so the system knows exactly where and what it is. To synthesize the reference images, X tapped into Google’s wealth of unique geospatial resources, which form the basis of products such as Google Earth and Maps.
It works. Just a few years after Bellwether was created and work began on the solution, the National Guard is already deploying it in testing and plans to use it for this summer’s wildfire season.
With this solution, National Guard team members can immediately look at the hardest-hit areas and know what locations they’re monitoring. They can know which bridges are down. They can ask the areas they’re monitoring, “Show me all the hospitals.” They can give informed answers immediately, eliminating the backlogs that have held them back for years.
ML plays a central role in aerial photography of this type, just as it does in more common enterprise systems. After all, photo matching is exactly the kind of inexact process that ML handles well. No match is guaranteed, because aerial photos are not exact matches. They each come from a unique distance, zoom, and angle, they are potentially obscured by weather conditions, and the landscape they capture has often been affected, sometimes disastrously.
ML eliminates most of the uncertainty by assigning a confidence level to each match. With many photos received, it turns out that enough of them match with a high confidence level, so the system can provide operations staff with visuals covering almost all of the locations involved, even after removing those that did not find a reliable match.
This approach is scalable. “Beyond our deployment to the National Guard, our goal is to make this type of service fundamentally easier to implement for a broader group of disaster responders,” Russell says. “It can be applied to rescue and rebuilding responses to a variety of weather events, including heat waves and tornadoes, for example.”
The universality of predictive AI
Whether it’s shooting for the moon or achieving more typical business goals, ML’s fundamental ability to generate trust levels solves operational challenges universally, across industries. Which customers are likely to buy? Marketing targets them. Which transactions are likely to be fraudulent? Banks block them. Which addresses are likely to receive a delivery tomorrow? UPS plans for them.
This well-known paradigm of driving large-scale operations using machine learning predictions has a name: predictive AI. It’s the practice of systematically filtering out the least confident cases and taking action on the remaining most confident cases.
So how confident is enough? It depends. Each project must determine the best choice of decision threshold based on practical needs. For example, the National Guard needs photos that meet a very high confidence level. In contrast, marketing and fraud detection can afford to target many cases that don’t materialize, which is part of the numbers games that these types of operations inevitably play.
In other words, predictive AI reduces uncertainty by quantifying it. Bellwether is working to extend this prodigious approach to other ways that will also reduce the damage caused by climate disasters, such as predicting where the most lives could be saved—which affected areas should be prioritized for evacuation assistance—and predicting environmental incidents before they happen.
“Machine learning has become the new paradigm in Earth science,” Russell says. “Until recently, for example, hydrology primarily predicted floods using site-specific models. Now, with machine learning, the best models are developed using data collected in different locations: flood behavior on the East Coast of the United States can be used to predict flooding on the West Coast.”
