Software Survey: Forecasting 2018

This year’s survey reflects changes in the software market, but it also puts greater emphasis on the requirements of the practicing forecaster.

By Robert Fildes, Oliver Schaer and Ivan Svetunkov

Software Survey Forecasting 2018

Forecasting has long been an essential part of OR/MS and is now making a major contribution to the fast-developing area of business analytics. Forecasts are a necessary input into almost all planning decisions, be they long- or short-term, but they are also needed in performance analysis and revenue optimization. Forecasting methods should, therefore, form an important part of all university or practitioner courses in analytics.

Suitable software is at the heart of the forecasting activity, whether the practitioner works in demand forecasting in the supply chain or as a business or government analyst. Interestingly, the history of forecasting is to a certain extent written by those who have developed user-friendly software packages early in the dissemination of a new method. For example, the early development of Box-Jenkins ARIMA software ensured rapid adoption while the paucity of state space software limited its adoption, until the development of open source software in R (Hyndman and Khandakar, 2008 [3]). But forecasting software is not homogeneous, either in terms of the methods implemented or, more importantly, in terms of the envisaged applications and it is a fast-developing field. OR/MS Today for a number of years has been surveying forecasting software suppliers to offer an overview of the latest features and market trends (most recently, Fry and Mehrotra, 2016 [1]). In this latest survey we attempted to identify all forecasting software providers of packages that included time series forecasting, whether research-oriented software or software designed for operational organizational planning. Of course, only a proportion responded within the deadline and any late submissions will be made available online at:

A core principle of forecasting is to look at past history to understand and predict. The 2016 survey by Fry and Mehrotra included some aspects that have become redundant, for example, flexible data input and output. But this is now straightforward, and we have dropped these issues from the current questionnaire. Fry and Mehrotra also identified a number of trends in the market: “integration,” where the forecasts link into the planning system; “automation,” where the package includes automatic selection from a range of methods; “visualization” for an extended range of graphics; “virtualization,” where the vendor’s offering is delivered through the cloud, or in its more extreme version, the vendor virtually manages the forecasting process from beginning to end. We now see all of these features in current software. The 2018 survey has been redesigned to reflect such changes in the software market, but it also puts greater emphasis on the requirements of the practicing forecaster. Software for forecasting comes in a variety of forms – no single software product includes all the features that a user might want because different users have different requirements. In surveys of practitioners, Excel is usually named as the most commonly used forecasting package (Weller and Crone, 2012 [6]) although it is best thought of as a data handling and analysis program, rather than providing forecasting; its built-in forecasting routines remain limited, even in its latest 2016 incarnation. In addition, its statistical features have weaknesses (see McCullough and Heiser, 2008 [4]).

If Excel and its built-in routines are inadequate, forecasters can attempt to build their own methods using macros. This is not recommended because even simple methods, such as seasonal exponential smoothing, have their pitfalls. Nor is it straightforward to scale up to the number of time series needing forecasting for many applications. Instead, forecasting-specific packages should do the trick, but now the choice is extensive; it can usefully be broken down into a number of different categories that reflect both the specialization for which the package is designed and its focus of application. For example:

General purpose statistical packages. These packages include a wide range of statistical techniques supplemented by forecasting routines. Sometimes there is a more specialized add-in (costing more) that enhances the basic system. Typically, the base general-purpose packages do not include the full range of forecasting methods and the additional features that a more advanced user might require. Market leaders include Minitab, SAS, SPSS, STATA and StatGraphics.

Specialized forecasting software. These may focus on a single technique or a range of methods. They include Autobox (which is an automatic Box-Jenkins ARIMA modeling and forecasting package), ForecastPro (which includes a range of methods), specialist neural network packages (such as Alyuda or Analytic Solver Data Mining), and state space modelling (in STAMP, a part of the OxMetrics range). In addition, there are the seasonal decomposition methods, X-13-ARIMA and Tramo-Seats which are freely available.

A key concept in forecasting, whether for research or in practice, is to keep a hold-out sample distinct from the in-sample fitting period. Secondly, the software should permit easy incorporation of rolling origin forecasting, a critical component of forecast evaluation (see, for example, Ord, Fildes and Kourentzes, 2017 [5], for a full explanation of how these evaluations are best carried out). The revised questionnaire has included a number of queries regarding the calculation of forecast errors-measures. In our experience, these are often poorly executed and yet are critical for the evaluation of forecasting success. For example, the user of ForecastPro can specify a hold-out sample where various error measures are calculated for both fixed and rolling origins. While AutoBox, ForecastPro and Smart are well-validated examples of software, having participated in various published competitions and therefore have been evaluated on standardized tests of forecasting accuracy, others may not conform to the principles of business forecasting for which we have argued (Ord, Fildes, Kourentzes, 2017 [5]). At the very least, all packages should produce forecasts of comparable accuracy to benchmarks available through R – if such comparisons have not been made, buyer beware must be the maxim.

Within the area of specialist software, the particular area of data mining (now often called predictive analytics) deserves extra attention. This software comprises a range of methods suitable for predicting individual consumer behavior. Typically, methods include logistic regression, predictive classification including CHAID and cross-sectional neural nets. The general purpose statistical packages such as SAS, Statgraphics and SPSS include these methods, but sometimes at an additional purchase cost. Weka provides free, well-regarded software: other alternatives are XLMiner, NeuroIntelligence and Neurosolutions (which may be used as Excel add-ins). Their time series capabilities are usually limited.

Econometric packages. While initially developed for econometric modeling, packages such as EViews, Oxmetrics, STATA, RATS and TSP now all include a wide range of regression-based modeling routines that can be easily applied to time series forecasting. They may also include various univariate alternatives. Other econometric packages are more specialized and focus on a limited set of routines, e.g., LINDEP, which includes a range of models suitable for classification.

Forecasting software support packages. These systems are designed to support demand planning and other organizational activities. At their core, these systems often rely on simple smoothing methods, but their strengths are the ability to simultaneously deal with many time series and be integrated (or at least compatible) with other corporate systems such as ERP (Enterprise Resource Planning). Such all-embracing packages include SAP, Oracle, JDA and Vanguard, as well as packages focused on particular sectors of the economy such as retail and health. These packages typically cost many thousands of dollars, and their forecasting components (despite being forecasting packages in part) used to be overly limited. Now some companies such as SAP have included many advanced modeling features in their extended range of program suites including automatic regression modeling; the base suite may, however, still rely on exponential smoothing.

If they are so expensive and often limited, why do companies use them? The answer is straightforward – the packages include data handling facilities that are extremely flexible and meet the needs of most companies working in the supply chain. In particular, they will link to the ERP and financial systems. They also permit multiple-user access. The scale of the forecasting activity they embrace could be tremendous. For example, software to support Walmart’s U.S. stores may have to process more than a billion time series daily (Seaman, 2018 [7]), therefore automatic forecasting and efficient computation is essential. There are many pitfalls in purchasing such packages, not the least of which is to assume that all such packages are well-designed from a forecasting perspective. Some suppliers over-claim and, therefore, benchmarking exercises against standard validated software are critical!

A second aspect is that the design of forecasting support systems should match the organization’s processes. For example, the statistical forecasts are often adjusted based on the expertise of the demand planners. This requires further analysis of the consequent errors and the “forecast value-added” arising from the adjustments.

Finally, there are mathematical and statistical programming languages such as R, Python and MATLAB, that have freely available packages and libraries that support time series analysis, econometrics and forecasting. In addition, SAS and other general-purpose statistical packages also provide programming languages that, like R, can be used to incorporate various statistical and forecasting methods into more complex analyses. The reader is referred to the CRAN time series repositories ( for available packages in R related to time series analysis, modeling and forecasting. Given that R has gained popularity over the past few years, an important criterion for evaluating any software that we ask about in the survey is its flexibility in incorporating R routines seamlessly into its design, thereby increasing its functionality. An advantage of these over commercial packages is that the authors of various innovative models and algorithms often provide open source code for the research and practice communities, ensuring state-of-the-art forecasting solutions. Furthermore, many universities train their graduates using these open source implementations, making them readily usable by the industry. This is a major paradigm shift in forecasting software.

Oliver Schaer, Robert Fildes and Ivan Svetunkov (l-r) of the Centre for Marketing Analytics and Forecasting, Lancaster University, United Kingdom.

Oliver Schaer, Robert Fildes and Ivan Svetunkov (l-r) of the Centre for Marketing Analytics and Forecasting, Lancaster University, United Kingdom.

Two other issues have been explored in this 2018 survey, namely forecast quality measurement and enhanced capabilities (in intermittent demand, new product methods and machine learning, as well as methods that combine the forecasts from a range of different methods). Respondents to the questionnaire were invited to highlight new developments. No consistent themes emerged; big data and machine learning were highlighted as was the need to automate the forecasting process to analyze many data series. Graphical analysis might prove particularly valuable. Readers will be able to see which packages offer these facilities.

A question easily overlooked in surveys like this is how easy the packages are to learn and use but answering would require a full hands-on review. We will note, however, that not all the programs we have used can be regarded as easy for novices (like us). In fact, questionnaire respondents mentioned the need for packages to be usable by nonspecialists and their linkages to dashboards such as Tableau. Overall, the dimensions we have reported on here (see survey data starting on page 48) should give the reader some insight into what is available, although some of the features may be buried deep. They should also supply the practicing forecaster looking to review their software implementation with a checklist of features and a short-list of suppliers to look for in a benchmarking exercise.

Robert Fildes ( is the founding director of the Centre for Marketing Analytics and Forecasting, Lancaster University, United Kingdom. The Centre’s objective is to develop new research on the business forecasting and marketing analytics problems that organizations face and help bridge the gap between research and practice. Oliver Schaer is a doctoral candidate and research associate at the Centre. Ivan Svetunkov is an assistant professor of marketing analytics at the Centre and a developer of several R packages focusing on forecasting and analytics.


  1. Fry, C. and Mehrota, V., Forecasting 2016 (Software Survey), OR/MS Today, June 2016, Vol.43, No. 3. Available online at
  2. Hyndman, R.J., 2017, “Forecasting functions for time series and linear models,” R package version 8.1,
  3. Hyndman, R.J. and Khandakar. Y., 2008, ”Automatic time series forecasting: the forecast package for R,” Journal of Statistical Software, Vol. 26, No. 3, pp. 1-22.
  4. McCullough, B. D. and Heiser, D. A., 2008, “On the accuracy of statistical procedures in Microsoft Excel 2007,” Computational Statistics and Data Analysis, Vol. 52, pp. 4570-4578.
  5. Ord, K., Fildes, R., and Kourentzes, N., 2017, “Principles of Business Forecasting,” 2nd ed, Wessex.
  6. Weller, M., and Crone, S. F., 2012, “Supply Chain Forecasting: Best Practices and Benchmarking Study,” Lancaster Centre for Forecasting White Paper. Available online:
  7. Seaman, B., 2018, “Considerations of a retail forecasting practitioner,” International Journal of Forecasting. Forthcoming.

Visit these Advertiser Sites