Using explainable AI to identify long-term variations in short-term forecast opportunities

How artificial intelligence dissects climate data patterns, unveiling predictable decadal climate states for more accurate forecasts.

Subseasonal refers to a timescale ranging from approximately 2 weeks to 3 months, or the timescale beyond weather but shorter than a season. The subseasonal range serves as a crucial link between weather and climate phenomena (Figure 1), incorporating elements of atmospheric chaos (weather) and ocean memory (climate). However, the complex interplay of these factors renders forecasting within the subseasonal timescale notably challenging, hence its colloquial designation as the “desert of predictability”.

Figure 1. The subseasonal desert of predictability is depicted as the bridge between weather and climate.
Credit. Seasoned Chaos

Forecasts of opportunity

Why persist in traversing the desert of predictability? Within this realm lie high-impact occurrences such as floods, droughts, and heatwaves. Responding to the urgent needs of local management officials, water managers, and stakeholders, there’s a clamor for enhanced research to bolster subseasonal forecasting capabilities, affording longer preparation periods. However, expecting flawless forecasts consistently is unrealistic on such a challenging timescale. Hence, efforts are directed towards identifying predictable states within the climate system, paving the path for forecasts characterised by higher prediction accuracy and confidence, termed forecasts of opportunity

Identifying the occurrence and location of opportunity forecasts hinges on pinpointing the large-scale climate processes that yield higher prediction accuracy. This often materialises when large-scale atmospheric or oceanic conditions are present that provide sources of subseasonal predictability. Notably, the tropics emerge as pivotal hubs of predictability for subseasonal precipitation in midlatitudes via tropical-midlatitude teleconnections or large-scale associations between weather patterns spanning thousands of kilometers. Additionally, the long-term oscillations of the climate system, termed decadal variability, exert a notable influence on the predictability of midlatitude precipitation.

The lingering question concerns the variability of subseasonal predictability from the tropics over long-term decadal timescales. The present research identified decadal (5-10 year) periods with enhanced subseasonal prediction skill alongside intervals of reduced skill due to natural variability in the climate system. This variability was evident in both climate model simulations and real-world data.

Tackling the problem with AI

This study employed an artificial intelligence approach, specifically a subset of machine learning called neural networks, to analyse climate data patterns. Given the data-intensive nature of machine learning, longer climate model simulations have been considered, specifically ten 100-year subsets of the Community Earth System Model Version 2-Large Ensemble (CESM2), amounting to 1,000 years of data. This study uses maps of wintertime (November-March) daily tropical precipitation anomalies (deviations from the average) as input to the neural network to predict the subseasonal precipitation anomaly in three U.S. West Coast regions (Alaska, Pacific Northwest, and California). Specifically, these tropical precipitation maps forecast precipitation in each region 4-5 weeks in advance. For a visual representation of the neural network setup, refer to Figure 2.

Figure 2. Schematic of the neural network setup used to make predictions of precipitation.
Credit. Author

Furthermore, we implement machine learning to gauge prediction confidence, with each output, such as prediction, carrying an associated probability. The outcomes show that prediction accuracy increases with confidence, indicating the networks’ ability to discern predictable time frames within the climate model data. Consequently, we prioritize predictions based on confidence and accuracy to pinpoint confident and correct forecasts—forecasts of opportunity!

Identifying sources of subseasonal predictability

Furthermore, explainable machine learning techniques assist in comprehending the decision-making process of the network. This not only allows us to assess trust in the network and optimise it as needed, but also facilitates leveraging the network for scientific discovery. Explainable artificial intelligence (XAI) methods elucidate how neural networks arrive at their predictions and identify crucial features in the input data. “Explanation/relevance heatmaps” have been generated to highlight significant regions pertinent to the network’s forecasting of opportunities.

The XAI heatmaps help identify the tropical origins of subseasonal predictability for precipitation along the U.S. West Coast. The present forecasts of opportunity highlight the Maritime Continent region, along with the equatorial western and central Pacific, aligning with two primary sources of predictability. These sources include the El Niño Southern Oscillation (ENSO), characterized by fluctuations in sea surface temperatures in the tropical Pacific, and the Madden Julian Oscillation (MJO), a large-scale storm system spanning the tropical Indian and Pacific Oceans. Thus, the improvised networks discern meaningful sources of predictability for mid-latitude subseasonal precipitation forecasts of opportunity.

A case of the “-ilities”: Decadal variability of subseasonal predictability

The XAI analysis enhances confidence in the neural network’s predictions and its ability to identify forecasts of opportunity by identifying relevant sources of predictability. The exploration delved into subseasonal predictability, as quantified by the neural network, revealing fluctuations over decadal timescales and identifying periods of higher or lower skill compared to the overall average. While all predictions showed slight variability on decadal timescales, the most notable findings were the substantial fluctuations in skill for the most confident predictions.

Certain decadal periods exhibited exceptionally high forecast skill levels (>70%), while others displayed skill levels worse than random chance (<50%). The specific low-frequency time intervals exhibit higher predictive skill and forecast confidence, resulting in extended periods of increased predictability. Thus, the decadal variability of subseasonal predictability for midlatitude precipitation has been documented and quantified.

The root cause of this phenomenon, termed the “-ilities,” is attributed to the teleconnection between the tropics and the midlatitudes. Specifically, it’s the slow modulation of this teleconnection that drives periods of both high and low skill. During intervals when anticipated ENSO-like and MJO-like patterns manifest in the midlatitudes, such as increased rainfall in California during an El Niño period, the network exhibits decadal periods of exceptionally high forecast skill with high confidence. 

However, when the expected teleconnection between the tropics and the midlatitudes is not present (e.g., less rainfall in California than normal during an El Niño period), the network remains confident due to learned relationships between the tropics and mid-latitude precipitation but is ultimately incorrect. This discrepancy results in decadal periods characterised by high confidence but low skill, often performing worse than random chance. The decadal variability in subseasonal predictability indicates that while the networks confidently learn relationships between tropical and midlatitude precipitation, these relationships are only reliably present during specific decadal time periods.

Out of the climate model world and into the real world

While deploying neural networks to learn connections between the tropics and subseasonal midlatitude precipitation is compelling, disparities between climate models and real-world data arise. Utilising neural networks trained on climate model data to predict subseasonal precipitation on real-world data (ERA5 reanalysis data from 1959-2021) does result in a slight drop in accuracy for the West Coast regions. However, this discrepancy is expected due to the disparate datasets. Remarkably, prediction skill still exceeds random chance, particularly for opportunity forecasts.

Moreover, skill varies on decadal timescales, mirroring findings from the climate model data, with decadal periods exhibiting nearly 70% accuracy for forecasts of opportunity. The networks have successfully identified genuine and predictable signals within the climate system, facilitating accurate and confident subseasonal precipitation predictions in the real world.

Skillful predictions for real world data by neural networks trained with climate model data reveals not only the utility, but the trust gained from using explainable AI to enhance our understanding of the climate system.

Marybeth Arcodia, Ph.D.

In summary, this study demonstrates that machine learning neural networks can detect predictable decadal climate states, enabling accurate subseasonal forecasts of opportunity in both climate models and real-world data. The utilisation of explainable machine learning facilitates the identification of forecasts of opportunity, sources of predictability, and quantifying decadal variability in subseasonal predictability. Moreover, this approach paves the way for integrating conventional and machine learning methods in advancing climate science to new frontiers.


Journal reference

Arcodia, M. C., Barnes, E. A., Mayer, K. J., Lee, J., Ordonez, A., & Ahn, M. S. (2023). Assessing decadal variability of subseasonal forecasts of opportunity using explainable AI. Environmental Research: Climate2(4), 045002. https://doi.org/10.1088/2752-5295/aced60

Dr Marybeth Arcodia is a Research Scientist at Colorado State University. Her current research explores sources of climate predictability from subseasonal to decadal timescales using explainable artificial intelligence techniques. She also writes for the Seasoned Chaos blog, a subseasonal to seasonal forecasting blog for scientists and non-scientists alike. She received her PhD in Atmospheric Sciences from the University of Miami Rosenstiel School in 2021.