New University of Chicago Study Tests How Well AI Can Forecast Extreme and Unusual Weather Events

    Researchers at the University of Chicago explored whether AI models can handle extreme weather surprises—events so rare they’ve never appeared in the data used to train them.

    Researchers found that there are key limitations in AI’s ability to forecast rare weather extremes.

    Artificial intelligence is changing the way we forecast the weather, promising faster and more energy-efficient predictions. But can it catch what’s never been seen before? A new study set out to find the answer, testing how well AI performs when the atmosphere breaks the rules.

    AI Forecasts Are Fast, But Not Always Accurate

    Scientists at the University of Chicago, working with NYU and UC Santa Cruz, tested whether cutting-edge AI models could predict the kind of extreme weather events that rarely appear in historical data.

    Their findings, published in the Proceedings of the National Academy of Sciences, revealed that these models struggle when faced with conditions that fall outside the scope of historical data. These “gray swans” are events like once-in-a-2,000-year floods or record-shattering hurricanes that pose serious forecasting challenges because they fall outside the patterns AI models are typically trained to recognize.

    The University of Chicago study revealed that AI models struggle to predict Category 5 hurricanes if similar storms are missing from training data.

    The team trained an AI model called FourCastNet on decades of global weather data, deliberately excluding Category 3 to 5 hurricanes. When tested on real-world conditions that led to a Category 5 storm, the model repeatedly underestimated the storm’s intensity, often predicting it would peak at only Category 2.

    Lead researcher Pedram Hassanzadeh explained that while AI models match supercomputers in forecasting everyday weather, they falter when faced with the unprecedented.

    Learning From the Past—But Only If It Exists

    AI models identify patterns in historical data to predict what’s likely to happen next. This approach works well for routine weather but can struggle to make accurate predictions when rare or extreme events—like the strongest hurricanes—are missing from the training data.

    “The floods caused by Hurricane Harvey in 2017 were considered a once-in-a-2,000-year event, for example,” said Hassanzadeh.

    If events like Harvey aren’t present in the historical record, AI systems may fail to anticipate them.

    Surprisingly, researchers found that the model could still make reasonably accurate predictions if similar events existed elsewhere in the world. For instance, even if Atlantic storm data was removed, the model could forecast Atlantic hurricanes using storm data from the Pacific.

    Still, researchers warn that this workaround doesn’t guarantee accuracy for truly new extremes, especially as climate change drives more unpredictable weather.

    Blending AI with Physics for Better Forecasting

    Traditional weather models rely heavily on physics, using equations to simulate how the atmosphere behaves. AI models like FourCastNet, however, operate more like predictive text: they don’t understand why something happens, only that it’s likely based on previous examples.

    To improve AI’s reliability, the team recommends combining it with physical modeling. One approach, called active learning, involves using AI to help generate synthetic extreme weather examples to train future models. This could teach the AI to better recognize rare but dangerous patterns.

    The takeaway? AI won’t replace traditional forecasts anytime soon, but combining it with physics could help close the gap and improve warnings for extreme weather.

    News reference:

    Baker, E. “University of Chicago analyzes AI’s ability to predict unprecedented weather events”https://www.meteorologicaltechnologyinternational.com/news/numerical-weather-prediction/university-of-chicago-analyzes-ais-ability-to-predict-unprecedented-weather-events.html