Challenges of Automated Forecasting in Retail ISF 2009 Roland Martin, SAF AG Simulation, Analysis and Forecasting Hong Kong, June 2009
Why Computer Automated Ordering (CAO)? Micro-Forecasting (>10k Items per Store per Day) Optimization (Forecast Order; Investment Buying, logistical units, etc.) Non-linear and multi-dimensional setting difficult to evaluate for humans Key goal: Reduce stock but increase availability To cope with mass data in short time and increase return on investment Seite 2 Vertrauliche und urheberrechtlich geschützte Information SAF AG, 2007
Crucial Goals and Requirements for CAO Automation vs. Interactivity Although the automation degree is high (> 98 %) it is crucial that some forecasts are monitored Reasons are for example: Unknown (new) predictors Exceptionally high/low forecasts Management by Exception Priority for different warnings Judgmental Adjustments Often disadvantageous 1,2 Goal is to achieve a high degree of automation (>98 %) 1 Kolassa, Stephan & Schütz, Wolfgang. Judgmental Changes to Retail Sales and Automatic Orders The 28 th Annual International Symposium on Forecasting, 2008 2 Fildes et al. Effective forecasting and judgmental adjustments: An empirical evaluation and strategies for improvement Seite 3 Vertrauliche und urheberrechtlich geschützte Information SAF AG, 2007
Crucial Goals and Requirements for CAO Data Quality Several problems occur even if the data supply is technically excellent Censored Sales Time series Substitution due to out of stocks Lumpiness after out of stocks Incorrect Data (Theft, Spoilage, ) NOSBOS (Not on Shelf but on Stock) Not labeled Pre-Orders in the data Several sources where forecasts can fail due to the data quality Seite 4 Vertrauliche und urheberrechtlich geschützte Information SAF AG, 2007
Crucial Goals and Requirements for CAO Computational Runtime Mass Data Up to 60k Items per Store in Micro-forecasting Forecasting every day Often centralized in the data center Critical Time Window Time restrictions for each store processing Therefore fast decision for forecasting model needed No way to recheck forecasting model Manual adjustments in cases of extreme forecasts (Exception Management) Large number of forecasts restricts the possibilities of model evaluation Seite 5 Vertrauliche und urheberrechtlich geschützte Information SAF AG, 2007
Expectation of Retailers Different Key Performance Indicators (KPI) Stock Out of stock Lost Sales Sophisticated Forecasts Constant Forecasts? Nifty Features for special cases Cannibalization/Halo New Items without Time series Intermittent demand Each and every Forecast is perfect (following which criteria?) Find forecasting models that suit every item at every time Seite 6 Vertrauliche und urheberrechtlich geschützte Information SAF AG, 2007
Expectation of Retailers How do we measure CAO performance? Two possibilities Stock KPIs Forecast accuracy Both alternatives have advantages and disadvantages Stock KPIs Have a direct interpretation in $ Are also interesting as store process KPIs Differentiate between over- and underforecasts Difficult to evaluate clearly Forecast accuracy Is easier to measure and track Does not depend on supply chain or store processes But does not fully effect Retail KPIs Seite 7 Vertrauliche und urheberrechtlich geschützte Information SAF AG, 2007
Expectation of Retailers Constant Forecasts Neither seasonality nor predictor effects Constant point forecast is best result Seite 8 Vertrauliche und urheberrechtlich geschützte Information SAF AG, 2007
Expectation of Retailers Constant Forecasts Changing forecasting model comprises a worse fit But customer is satisfied Seite 9 Vertrauliche und urheberrechtlich geschützte Information SAF AG, 2007
Challenges for Forecasters Find best forecasting method within little time Find rules to engage a lock bar on extreme cases or misuse of features Extrapolation of effects Bad predictor settings (multi-collinearity) Change customer/store processes in order to achieve better results Advertisement Magic Triangle Automation vs. Judgmental Adjustments Establish robustness without losing forecasting power Order quantity determines Retail KPIs not Forecast (accuracy) No Micro-Management (find configurations that suit all items) Single Forecast does not determine the success of a CAO system Incorporate the needs of the Retailers in the Forecasting Process Seite 10 Vertrauliche und urheberrechtlich geschützte Information SAF AG, 2007
Challenges for Forecasters Orders determine Retail KPIs Several forecasting models may lead to the same order amount although they comprise different point forecasts Worse model fit results in a higher safety amount Quantile forecasts may be the same Even if not, logistical rounding may find the same order amount Example: Fcst 1 : 23.4 Fcst 2 : 42.2 LogUnitSize: 50 units In both cases the order will be 50 units Order determines the Retail KPIs not the Forecast Seite 11 Vertrauliche und urheberrechtlich geschützte Information SAF AG, 2007
Challenges for Forecasters Significance of Predictors Too many predictors Senseless predictors (too long) Seite 12 Vertrauliche und urheberrechtlich geschützte Information SAF AG, 2007
Challenges for Forecasters Predictor Effect Determination Unsteady sales values in promotion cause forecasts to fail One occurrence dominates the price predictor effect Next forecast in promotion will probably be too high Seite 13 Vertrauliche und urheberrechtlich geschützte Information SAF AG, 2007
Challenges for Forecasters Extrapolation of Predictor Values Value of the red metric predictor is twice as high in the forecast horizon than in the history predictor effect is extrapolated Using an extrapolation parameter limits the effect Seite 14 Vertrauliche und urheberrechtlich geschützte Information SAF AG, 2007
Challenges for Forecasters Wrong Data Logic Inactive Period is modeled using a zero value for the price predictor Results: Forecast is questionable Seite 15 Vertrauliche und urheberrechtlich geschützte Information SAF AG, 2007
Conclusion Technical challenges: Mass Data Short time to model building Data quality Forecasting challenges: Predictor settings Order determines the Retail KPIs not the Forecast Robust but excellent forecasts Challenge: Large numbers of robust forecast that lead to good Retail KPIs Seite 16 Vertrauliche und urheberrechtlich geschützte Information SAF AG, 2007
forecasting is our success SAF Simulation, Analysis and Forecasting AG High-Tech-Center 2 Bahnstrasse 1 CH-8274 Tägerwilen Tel: +41 (0)71 666 70 00 Fax: +41 (0)71 666 70 10 mailto: contact@saf-ag.com www.saf-ag.com Thank you Seite 17 Vertrauliche und urheberrechtlich geschützte Information SAF AG, 2007