Combining SKU-level sales forecasts from models and experts
Introduction
There is abundant literature on the relative performance of model forecasts, expert forecasts and their combination, see Lawrence et al., 2006, Fildes et al., 2009, and the earlier work of Blattberg and Hoch (1990). The most common findings are that expert forecasts can improve on model forecasts and that a linear combination of the model forecast with an expert forecast is often even better. The literature so far mainly considers a few single-product, single-horizon and single-expert cases.
In our present paper we aim to extend the currently available literature by considering various products in various product categories, 12 different forecast horizons and about 50 experts. A main additional feature of our analysis is that we know a few characteristics of these experts and we also observe their behaviour. This allows us to correlate the optimal balance between the model and the experts with their characteristics and their behaviour, which in turn gives guidelines from a managerial perspective, and this is new to the literature.
In this paper we empirically analyze a unique and very large database with model forecasts, expert forecasts and realizations concerning monthly SKU-level sales of a range of pharmaceutical products for a large Netherlands-based firm. At the headquarters office, the model forecasts are automatically created by a statistical package, where the program each month allows for a re-specification of the model and it also re-estimates all parameters each time. The experts, located in local offices in 37 countries, receive these forecasts and, after that, create their forecasts using their own expertise. We will see that expert forecasts often differ from the model forecasts, which is perhaps not unexpected given the fact that the automatic program includes as input only lagged monthly sales values, and that this fact is known to the experts, see Goodwin, 2000, Goodwin, 2002.
The question the firm faces is whether the model forecasts and the expert forecasts can be improved by taking a linear combination of the two. A related question is whether this linear combination should follow an unconditional 50–50% rule, or whether the weights shall depend on the characteristics of the experts.
The literature on combining forecasts in for example Clemen, 1989, Timmermann, 2006 suggests that linear combinations of forecasts may improve on each of its contributors. So the first question we consider in this paper is whether there are optimal weights for each of the experts. And, if so, is that robust across forecast horizons and does it differ across experts?
The second question that we try to answer is whether these optimal weights can be explained by characteristics of the experts. This question is very relevant from a managerial perspective as it facilitates training of experts and also their selection prior to their appointment. Blattberg and Hoch (1990) claim that a 50–50% rule would be best but this claim corresponds with unconditional weights as it is not correlated with experts’ characteristics. Lamont (2002) demonstrated that age (experience) has a positive effect on the quality of an expert, but also that this effect is parabolic. There are also studies like Barber and Odean, 2001, Beyer and Bowden, 1997 which find gender differences in (over-)confidence levels, so perhaps there are also such differences across the relative weights of the experts in the combined forecasts. Finally, the degree of bracketing shall be important for the quality of the combined forecast. Larrick and Soll (2006, p. 112) state that when the rate of bracketing increases, the power of averaging forecasts does too. Their findings were based on experiments, and in the present study we shall seek empirical evidence for this statement based on factual data.
The outline of our paper is as follows. In Section 2 we outline the main features of our unique database. Section 3 deals with the methodology and gives the details of our empirical findings. Section 4 concludes with various implications for managers who need to evaluate the qualities of the experts.
Section snippets
Data
Our data concern a firm that creates model forecasts and which has almost 50 experts allocated in 37 countries1 are allowed to report their own forecasts additional to the model forecasts they receive from the headquarter’s office. Average characteristics of these experts are available. The question the firm has is whether specific combinations of these two sets of forecasts are better
Methodology and results
To address the managerial questions of the firm, which are typical questions any firm would have to manage a range of experts, we aim to compute the optimal value of the weights in a combined forecast. This combined forecast for each expert i given a horizon h is given bywhere we compute the value of ai across all products within an expert-horizon combination. To achieve this aim, we compute the root mean squared prediction error (RMSPE) as
Discussion
Our paper analyzed a very large and unique database with model forecasts and expert forecasts to see if combining these forecasts would be beneficial. Blattberg and Hoch (1990) predicted that unconditional weights of 50–50% would be best. One of the novelties of our study is that we examined if these weights could be predicted by experts’ characteristics and actual behaviour or performance, that is, whether there are perhaps conditional weights.
References (11)
Combining forecasts: A review and annotated bibliography (with discussion)
International Journal of Forecasting
(1989)- et al.
Effective forecasting and judgmental adjustments: An empirical evaluation and strategies for improvement in supply-chain planning
International Journal of Forecasting
(2009) Improving the voluntary integration of statistical forecasts and judgement
International Journal of Forecasting
(2000)Integrating management judgement with statistical methods to improve short-term forecasts
Omega
(2002)Macroeconomic forecasts and microeconomic forecasters
Journal of Economic Behavior & Organization
(2002)
Cited by (35)
Demand forecasting for fashion products: A systematic review
2024, International Journal of ForecastingUse of contextual and model-based information in adjusting promotional forecasts
2023, European Journal of Operational ResearchA machine learning approach for forecasting hierarchical time series
2021, Expert Systems with ApplicationsHierarchical time series forecasting via Support Vector Regression in the European Travel Retail Industry
2019, Expert Systems with ApplicationsBrain imaging and forecasting: Insights from judgmental model selection
2019, Omega (United Kingdom)Judgmental forecast adjustments over different time horizons
2019, Omega (United Kingdom)