Some machine learning models such as Long Short-Term Memory (LSTM) models are of course able to accommodate this, but it’s not standard for all machine learning methods. Additionally, you will need to figure out how to deal with both missing values and outliers. So therein lies the rub – a lot of stuff happening at the level of the individual time series but not a lot of data to be able to separate the signal from the noise.
Complex is not necessarily better than simple in large-scale forecasting
So far, we’ve only considered a single time series. However, when working with demand forecasting, we often work with data at the item/location level, where several thousand different time series are quite common. This means that our entire set of data potentially can be quite huge. In other words, we are entering the domain of “large-scale forecasting”.
The big question then is: Couldn’t we just use machine learning on all that data? In theory, the answer is yes. For practical purposes, it’s not as straightforward as that. The output of our efforts is fed downstream through the planning process, where planning is done for each item/location combination. This requires that forecasts are generated at this level. Obviously, this can be achieved by using machine learning models, but it might require some tweaking, depending on the actual model used. Also, since time series in the current context tend to be short, it’s often a requirement that our modelling setup is updated, once new data becomes available – for instance monthly. This is somewhat different from how many traditional machine learning systems get updated, which is when your model monitoring shows signs of degrading performance.
The above needs to be seen in combination with other requirements that we have for the output for planning purposes. Sales and operations planning is essentially a negotiation process, and to facilitate negotiations, we often want information on what drives the forecast – what are the effects of trend, seasonality, promotions etc. In other words, how do we enable the generation of interpretable output which accommodates different effects of demand drivers for each item/location combination? It’s not enough to state for example that our sales generally have an upward trend.
We need to be able to have different trend information for each item/location combination. Additionally, since part of the subsequent planning most likely involves inventory planning, we need some measurements of forecast uncertainty as input to inventory planning. The traditional time series methods provide confidence intervals for the forecasts we can use. How can we get confidence intervals for our forecasts – for each item/location – if we use machine learning models?
The above points to some technical questions which the use of machine learning in the context of S&OP must be able to answer. However, even in those instances where these technical questions have been answered, evidence has indicated that more complex machine learning models in fact do not produce more accurate forecasts. Some papers show the opposite for smaller problems involving one or just a few time series, but the fact of the matter is that in the domain of large-scale forecasting, more complex models do not yet consistently outperform simpler models in terms of forecast accuracy.
“Simpler” and “yet” are the key
The keywords in that last sentence are “simpler” and “yet”. In this context, “simpler” refers to time series models ranging from a random walk to various ESMs and ARIMA models. If you haven’t previously considered time series methods to support your demand planning activity, the good news is that there are a lot of options to choose from, if you want to pursue this area.
Most planning software vendors have some capabilities in this area, but more specialised vendors such as Autobox, Forecast Pro and SAS also have offerings in this space. If there’s a preference for open source technology, which is the de facto technology standard in machine learning, Rob Hyndman’s Forecast package in R could be an option to pursue. Another open source option could be the Facebook Prophet package (running both in R and Python), which offers a novel approach to time series forecasting. Aspiring as well as experienced data scientists have competencies in either R or Python, making open source tools a natural starting point. A somewhat advanced approach to complement traditional time series methods is to work with hierarchical forecasting – across time, geography, product categories etc.
The obvious benefit is having forecasts that “add up” across the different levels in the hierarchy. The perhaps not so obvious benefit is that you can exploit the fact that the signal-to-noise ratio often is higher at more aggregated levels of the hierarchy and pushes the identified patterns in the forecasts at the aggregated levels down to forecasts at the disaggregated levels that will feed into downstream planning activities.
The word “yet” is important because while machine learning might not be a viable option now, a lot of research, development and experimentation are being done in this area. Given how machine learning has been successful in other business and data domains, the bet is that it will happen in S&OP at some point. Amazon, for instance, already has a commercial offering called DeepAR. However, some evidence suggests that performance is still underwhelming compared to simpler methods.
See for instance this comparison of DeepAR and Facebook Prophet.
There is light at the end of the tunnel – and you can already start to prepare
The most promising results to date have been the results of the M4 competition by e.g. a team from Uber that developed and implemented an approach involving a combination of machine learning and traditional time series methods. The cost, though, was compute time measured in days rather than hours.
So, while we’re not quite there yet in terms of machine learning models consistently performing well in large-scale forecasting scenarios, there is light at the end of the tunnel. Already now, you can start preparing for what the future holds.
The obvious starting point is deciding what forecast accuracy improvements are worth in your organisation. Armed with that information, you can start exploring the options which are already available today. Given the low barrier to entry provided by access to free open source tools and data science competencies familiar with these tools, this makes a good starting point.