Exponential smoothing: The state of the art – Part II Everette S. Gardner, Jr.

1. Exponential smoothing:The state of the art – Part IIEverette S. Gardner, Jr.

2. Exponential smoothing:The state of the art – Part II • History • Methods • Properties • Method selection • Model-fitting • Inventory control • Conclusions

3. Timeline of Operations Research (Gass, 2002) 1654 Expected value, B. Pascal 1733 Normal distribution, A. de Moivre 1763 Bayes Rule, T. Bayes 1788 Lagrangian multipliers, J. Lagrange 1795 Method of Least Squares, C. Gauss, A. Legendre 1826 Solution of linear equations, C. Gauss 1907 Markov chains, A. Markov 1909 Queuing theory, A. Erlang 1936 The term OR first used in British military applications 1941 Transportation model, F. Hitchcock 1942 U.K. Naval Operational Research, P. Blackett 1943 Neural networks, W. McCulloch, W. Pitts 1944 Game theory, J. von Neumann, O. Morgenstern 1944 Exponential smoothing, R. Brown

4. Exponential smoothing at work “A depth charge has a magnificent laxative effect on a submariner.” Lt. Sheldon H. Kinney, Commander, USS Bronstein (DE 189)

5. Forecast Profiles N A M None Additive Multiplicative N None A Additive DA Damped Additive M Multiplicative DM Damped Multiplicative

6. Damped multiplicative trends (Taylor, 2002) Damping parameter

7. Variations on the standard methods • Multivariate series (Pfefferman & Allen, 1989) • Missing or irregular observations (Wright,1986) • Irregular update intervals (Johnston, 1993) • Planned discontinuities (Williams & Miller, 1999) • Combined level/seasonal component (Snyder & Shami, 2001) • Multiple seasonal cycles (Taylor, 2003) • Fixed drift (Hyndman & Billah, 2003) • Smooth transition exponential smoothing (Taylor, 2004) • Renormalized seasonals (Archibald & Koehler, 2003) • SSOE state-space equivalent methods (Hyndman et al., 2002)

8. Smoothing with a fixed drift (Hyndman & Billah, 2003) • Equivalent to the “Theta method”? (Assimakopoulos and Nikolopoulos, 2000) • How to do it • Set drift equal to half the slope of a regression on time • Then add a fixed drift to simple smoothing, or • Set the trend parameter to zero in Holt’s linear trend • When to do it • Unknown

9. Adaptive simple smoothing (Taylor, 2004) • Smooth transition exponential smoothing (STES) is the only adaptive method to demonstrate credible improved forecast accuracy • The adaptive parameter changes according to a logistic function of the errors • Model-fitting is necessary

10. Renormalization of seasonals • Additive (Lawton, 1998) • Without renormalization • Level and seasonals are biased • Trend and forecasts are unbiased • Renormalization of seasonals alone • Forecasts are biased unless renormalization is done every period • Multiplicative(Archibald & Koehler, 2003) • Competing renormalization methods give forecasts different from each other and from unnormalized forecasts

11. Archibald & Koehler (2003) solution • Additive and multiplicative renormalization equations that give the same forecasts as standard equations • Cumulative renormalization correction factors for those who wish to keep the standard equations

12. Continental Airlines Domestic Yields Model Restarted

13. Standard vs. state-space methods • Trend damping • Standard: Immediate • State-space: Starting at 2 steps ahead • Multiplicative seasonality • Standard: Seasonal component depends on level • State-space: Independent components • Model fitting • Standard: Minimize squared errors • State-space: Minimize squared relative errors if multiplicative errors are assumed.

14. Properties • Equivalent models • Prediction intervals • Robustness

15. Equivalent models • Linear methods • ARIMA • DLS regression • Kernel regression (Gijbels et al.,1999; Taylor, 2004) • MSOE state-space models (Harvey, 1984) • All methods • SSOE state-space models (Ord et al.,1997)

16. Analytical prediction intervals • Options • SSOE models (Hyndman et al., 2005) • Model-free (Chatfield & Yar, 1991) • Empirical evidence • None

17. Empirical prediction intervals • Options • Chebyshev distribution (fitted errors) (Gardner, 1988) • Quantile regression (fitted errors) (Taylor & Bunn, 1999) • Parametric bootstrap (Snyder et al., 2002) • Simulation from assumed model (Bowerman, O’Connell, & Koehler, 2005) • Empirical evidence • Limited, but encouraging

18. Robustness • Many equivalent models for each method (Chatfield et al., 2001; Koehler et al., 2001) • Simple ES performs well in many series that are not ARIMA (0,1,1) (Cogger,1973) • Aggregated series can often be approximated by ARIMA (0,1,1) (Rosanna & Seater, 1995)

19. Robustness (continued) • Exponentially declining weights are robust (Muth, 1960; Satchell & Timmerman, 1995) • Additive seasonal methods are not sensitive to the generating process (Chen,1997) • The damped trend includes numerous special cases (Gardner & McKenzie,1988)

20. Automatic forecasting with the damped additive trend  = .84  = .38  = 1.00

21. Summary of 66 empirical studies,1985-2005 • Seasonal methods rarely used • Damped trend rarely used • Multiplicative trend never used • Little attention to method selection • But exponential smoothing was robust, performing well in at least 58 studies

22. Method selection • Benchmarking • Time series characteristics • Expert systems • Information criteria • Operational benefits • Identification vs. selection

23. Benchmarking in method selection • Methods should be compared to reasonable alternatives • Competing methods should use exactly the same information • Forecast comparisons should be genuinely out of sample

24. Method selection: Time series characteristics • Variances of differences(Gardner & McKenzie,1988) • Seemed a good idea at the time • Discriminant analysis(Shah,1997) • Considered only simple smoothing and a linear trend • Should be tested with an exponential smoothing framework • Regression-based performance index(Meade, 2000) • Considered every feasible time series model • Should be tested with an exponential smoothing framework

25. Method selection: Expert systems • Rule-based forecasting • Original version (Collopy & Armstrong, 1992) • Automatic version (Vokurka et al., 1996) • Streamlined version (Adya et al., 2001) • Other rule-induction systems (Arinze,1994; Flores & Pearce, 2000) • Expert systems are no better than aggregate selection of the damped trend alone (Gardner, 1999)

26. Method selection: AIC Damped trend vs. state-space models selected by AIC: Average of all forecast horizons MAPE Asymmetric MAPE

27. Method selection:Empirical information criteria (EIC) • Strategy: Penalize the likelihood by linear and nonlinear functions of the number of parameters (Billah et al., 2005) • Evaluation: EIC superior to other information criteria, but results are not benchmarked

28. Method selection: Operational benefits • Forecasting determines inventory costs, service levels, and scheduling and staffing efficiency. • Research is limited because a model of the operating system is needed to project performance measures.

29. Method selection: Operational benefits (cont.) • Manufacturing (Adshead & Price, 1987) • Producer of industrial fasteners (£4 million annual sales) • Costs: holding, stockout, overtime • U.S. Navy repair parts (Gardner, 1990) • 50,000 inventory items • Tradeoffs: Backorder delays vs. investment • Savings: $30 million (7%) in investment

30. Average delay in filling backorders

31. Inventory analysis: Packaging materials for snack-food manufacturer Actual Inventory from subjective forecasts Month Target maximum inventory based on damped trend Month Monthly Usage

32. Method selection: Operational benefits (cont.) • Electronics components (Flores et al., 1993) • 967 inventory items • Costs: holding cost vs. margin on lost sales • RAF repair parts (Eaves & Kingsman, 2004) • 11,203 inventory items • Tradeoffs: inventory investment vs. stockouts • Savings: £285 million (14%) in investment

33. Forecasting for inventory control:Cumulative lead-time demand • SSOE models yield standard deviations of cumulative lead-time demand(Snyder et al., 2004) • Differences from traditional expressions (such as ) are significant

34. Standard deviation multipliers, α = 0.30 Lead time

35. Forecasting for inventory control:Cumulative lead-time demand (cont.) • The parametric bootstrap (Snyder et al., 2002) can estimate variances for: • Any seasonal model • Non-normal demands • Intermittent demands • Stochastic lead times

36. Forecasting for inventory control:Intermittent demand • Croston’s method (Croston, 1972) Smoothed nonzero demand Mean demand = Smoothed inter-arrival time • Bias correction (Eaves & Kingsman, 2004; Syntetos & Boylan, 2001, 2005) Mean demand x (1 – α/ 2)

37. Forecasting for inventory control: Intermittent demand (continued) • There is no stochastic model for Croston’s method(Shenstone & Hyndman, 2005) • Many questionable variance expressions in the literature • The state-space model for intermittent series requires a constant mean inter-arrival time (Snyder, 2002) • Why not aggregate the data to eliminate zeroes?

38. Progress in the state of the art, 1985-2005 • Analytical variances are available for most methods through SSOE models. • Robust methods are available for multiplicative trends and adaptive simple smoothing. • Croston’s method has been corrected for bias. • Confusion about renormalization of seasonals has finally been resolved. • There has been little progress in method selection. • Much empirical work remains to be done.

39. Suggestions for research • Refine the state-space framework • Add the damped multiplicative trend • Damp all trends immediately • Test alternative method selection procedures • Validate and compare method selection procedures • Information criteria – Benchmark the EIC • Discriminant analysis • Regression-based performance index

40. Suggestions for research (continued) • Develop guidelines for the following choices: • Damped additive vs. damped multiplicative trend • Fixed vs. adaptive parameters in simple smoothing • Fixed vs. smoothed trend in additive trend model • Standard vs. state-space seasonal components • Additive vs. multiplicative errors • Analytical vs. empirical prediction intervals

41. Conclusion “The challenge for future research is to establish some basis for choosing among these and other approaches to time series forecasting.”(Gardner,1985)