Featured Article: Weather Forecasting Techniques

math

From the roaring waters of Hurricane Harvey to the raging wildfires in California, it is now evident more than ever that we need accurate and reliable weather forecasting techniques to help prepare for and mitigate the impacts of climate-induced phenomena, such as hurricanes and subsequent flooding, heat waves, and strong winds that could spread a limited house fire into an entire forest. So, what makes a good forecasting technique and what are the caveats of the techniques in practice today? These are the questions that this article aims to answer to better reflect how today’s analytics community stands in dealing with weather forecasting challenges.

It is no secret that weather forecasting has been around for ages, rooting back to early civilizations when ancient people tried to predict weather for agriculture and marine transportation, among other purposes. In modern days, however, weather forecasting has evolved from an artistry into a scientific and professional discipline. With the increasing penetration of different forms of media, weather forecasting has become a service that can be commercialized and dispersed. As the nature and purpose of weather forecasting has changed throughout time, techniques for weather prediction have also progressed.

One of the simplest methods of weather forecasting is to use the historical averages of temperature and precipitation in a particular location. Known as the Climatology Method, this approach has many shortcomings, however, and is reliable only if there is little variance in the variables of interest. A more complicated approach is to match the present-day weather condition to a similar weather scenario in the past. A forecast is then generated to align with the area’s previously observed weather progressions. This approach is known as the Analog Method because a similar condition (an analog) in the past serves as the basis for the forecast. This method by itself is not very effective because it is often not possible to find an exact analog for the present-day weather conditions. However, the Analog Method does provide interesting insights that could support the development of more sophisticated prediction models [1].

Numerical weather prediction (NWP) is another class of methods that uses computation, algorithms and equations to provide weather forecasts. With increased data availability and increasingly powerful artificial intelligence (AI) methods, weather forecasting is constantly improving in terms of reliability and accuracy.

Machine learning techniques have long been used to provide forecasts of meteorological indicators, such as precipitation and elevation [2]. One particular method for predicting daily rainfall divides the region of interest into several sub-areas and then models each sub-area using a Radial Basis Function Network (RBFN) based on precipitation data from 100 nearby locations. The RBFN generates a mapping that involves an interpolating function that passes through every data point. In the presence of noise, the interpolating function becomes smoother and is averaged over the noise to provide the best generalization.

Another canonical example of applications for weather forecasting techniques comes from the load forecasting problem. This problem deals with the prediction of power consumption (or “load” in electrical engineering jargon) and is central to power market operations and the reliability of the power grid. This is a particularly interesting application of weather forecasting that brings together several engineering, economic and natural science disciplines. The most common tools in this class of techniques include autoregressive integrated moving average (ARIMA) and partial autocorrelation functions (PACF) [3].

Flood forecasting is another major application of numerical-based methods for prediction of climate-based phenomena. A commonly adopted approach to flood forecasting is the ensemble prediction system (EPS) in which a collection of NWPs is utilized rather than a single deterministic outcome. These NWPs are usually coupled with hydrological models and river discharge predictions implemented though a decision support system. The backbone of this method is a Monte Carlo framework of NPWs that generates multiple weather condition scenarios. This method takes into account the uncertainties involved in the climate patterns and outputs several weather predictions for a given time and target location [4].

Despite major efforts in applying cutting-edge knowledge to the weather forecasting problem, there are still challenges that need to be overcome to provide more reliable and accurate predictions. Specifically, for AI-based models, several issues threaten the credibility of the methods. For one, the structure and design of the machine learning algorithm sometimes does not match the scale and scope of the task. For instance, if daily prediction of temperature is the objective, the recorded climate data available for training and validation of the AI model may not span more than a few decades, a small data set compared to what AI can handle. Additionally, an overly sophisticated AI design for this limited data set increases the chances of overfitting and inaccurate predictions. Another major issue is the lack of a systematic testing for the AI-based techniques as there seems to be no benchmark measure to contrast the output of different prediction models. Yet another challenge is that temporal interdependence in the data hinders immediate application of AI-based tools. In this case, it is often difficult to fully understand and model uncertainties and error patterns, further complicating the development and validation of forecasting tools.

As more theoretical frameworks and computational capabilities become available, weather forecasting methods continue to improve and grow in scale. Further data accumulation also enables improved testing and validation of models. But good enough is never enough. If there is one thing we have learned from the recent hurricanes and wildfires, it is that you cannot be too careful when it comes to weather forecasting.

References

 [1] http://ww2010.atmos.uiuc.edu/(Gh)/guides/mtr/fcst/mth/oth.rxml

 [2] Shrivastava, G., S. Karmakar, M. K. Kowar, and P. Guhathakurta (2012). "Application of artificial neural networks in weather forecasting: a comprehensive literature review." International Journal of Computer Applications, 51(18): 17–29.

 [3] Hippert, H. S., C. E. Pedreira, and R. C. Souza (2001). "Neural networks for short-term load forecasting: a review and evaluation." IEEE Transactions on Power Systems, 16(1): 44–55.

 [4] Cloke, H. L. and F. Pappenberger (2009). "Ensemble flood forecasting: a review." Journal of Hydrology, 375(3): 613–626.