GITNUXREPORT 2026

Time Series Analysis Statistics

This blog post covers the essential concepts and statistical methods for time series analysis.

Jannik Lindner

Jannik Lindner

Co-Founder of Gitnux, specialized in content and tech since 2016.

First published: Feb 13, 2026

Our Commitment to Accuracy

Rigorous fact-checking · Reputable sources · Regular updatesLearn more

Key Statistics

Statistic 1

The autocorrelation function (ACF) measures the linear relationship between lagged values in a time series, with significance tested using Bartlett's formula where the standard error for lag k is approximately 1/sqrt(n) for large n, as detailed in Box-Jenkins methodology.

Statistic 2

Stationarity in time series requires constant mean, variance, and autocovariance, tested via Augmented Dickey-Fuller (ADF) test with null hypothesis of unit root, rejecting if test statistic < critical value at 5% level (e.g., -2.89 for n=100).

Statistic 3

Differencing a non-stationary series d times transforms it to stationarity, where d is determined by the number of unit roots, typically 0-2 for most economic series.

Statistic 4

The partial autocorrelation function (PACF) isolates correlation between series and lag k not explained by shorter lags, cutting off after order p in AR(p) processes.

Statistic 5

White noise series has zero mean, constant variance σ², and zero autocorrelation at all lags beyond zero, with Box-Pierce Q-statistic testing residuals for whiteness.

Statistic 6

Seasonal decomposition using STL (Seasonal-Trend decomposition using Loess) applies locally weighted regression with robustness to outliers via median.

Statistic 7

The mean of a time series is estimated as the sample average, but for trending series, use differenced mean or Hodrick-Prescott filter to detrend.

Statistic 8

Variance stabilization transforms like log or Box-Cox reduce heteroscedasticity, where Box-Cox λ is chosen via maximum likelihood, often λ=0 for logs.

Statistic 9

Cross-correlation function (CCF) between two series measures lead-lag relationships, used in transfer function models with prewhitening to identify lags.

Statistic 10

The periodogram estimates power spectral density as (1/n) |sum x_t exp(-2πikt/n)|^2 for frequency k/n, inconsistent but smoothed via Welch's method.

Statistic 11

Cointegration tests like Engle-Granger involve regressing series, testing residuals for unit root; Johansen test uses VAR trace statistic for r cointegrating vectors.

Statistic 12

Structural breaks detected by Chow test split series at τ, F-stat = [(RSS_r - RSS_u)/k] / [RSS_u/(n-2k)], critical values from F(k, n-2k).

Statistic 13

Kalman filter updates state estimate as x̂_{t|t} = x̂_{t|t-1} + K_t (y_t - Z x̂_{t|t-1}), with gain K_t = P_{t|t-1} Z' (Z P_{t|t-1} Z' + H)^{-1}.

Statistic 14

ARCH(1) model variance h_t = α0 + α1 ε_{t-1}^2, with α1 <1 for stationarity, LM test for ARCH effects via Lagrange multiplier on squared residuals.

Statistic 15

GARCH(1,1) generalizes to h_t = α0 + α1 ε_{t-1}^2 + β1 h_{t-1}, stationary if α1 + β1 <1, common in 0.05-0.95 range for finance.

Statistic 16

Exponential smoothing α weights recent observations more, with optimal α minimizing MSE via state space minimization.

Statistic 17

Holt-Winters additive seasonal model has level l_t = α(y_t - s_{t-m}) + (1-α)(l_{t-1} + b_{t-1}), trend b_t = β(l_t - l_{t-1}) + (1-β)b_{t-1}.

Statistic 18

Theta method decomposes into theta lines fitted to detrended series, outperforming benchmarks in M3 competition with 10% error reduction.

Statistic 19

Ljung-Box portmanteau test Q = n(n+2) sum [(r_k)^2 / (n-k)] ~ χ²(df), df = h-p-q for ARMA residuals.

Statistic 20

Durbin-Watson statistic d = sum[(e_t - e_{t-1})^2]/sum e_t^2 ≈ 2(1 - ρ̂), bounds 0-4 for autocorrelation.

Statistic 21

KPSS test for stationarity around trend, null of stationarity vs ADF's unit root, Lagrange multiplier statistic LM = (1/n)^2 sum S_t^2 / f(0).

Statistic 22

Variance of forecast error at h steps for AR(1) is σ² (1 + φ² + ... + φ^{2(h-1)}) = σ² (1 - φ^{2h})/(1-φ²).

Statistic 23

Information criteria AIC = -2 log L + 2k, BIC = -2 log L + k log n, penalizing complexity for model selection.

Statistic 24

Diebold-Mariano test compares forecast accuracy H0: ρ=0 where ρ = d / sqrt(2π f_d(0)), d=e1-e2 mean difference.

Statistic 25

MA(∞) representation of AR(p) exists if |φ(z)|≠0 for |z|≤1, Wold decomposition theorem.

Statistic 26

Invertibility of MA(q) requires roots of θ(z)=0 outside unit circle, ensuring ε_t recoverable from infinite past y.

Statistic 27

State-space form Y_t = Z α_t + ε_t, α_{t+1} = T α_t + R η_t, used for non-standard models.

Statistic 28

Innovation algorithm computes one-step predictors recursively, efficient for ARMA identification.

Statistic 29

Yule-Walker equations solve AR(p) coefficients ρ_k = sum φ_j ρ_{k-j} for k=1..p, using sample ACF.

Statistic 30

Burg's method minimizes forward/backward prediction errors for AR estimation, better for short series than Yule-Walker.

Statistic 31

The ARIMA(p,d,q) model difference operator Δ^d y_t = sum binom(d+k-1,k) y_{t-k}, integrated of order d.

Statistic 32

SARIMA(p,d,q)(P,D,Q)s extends ARIMA with seasonal AR/MA, differencing Δ^D_s y_t at period s.

Statistic 33

ETS(A,N,N) is simple exponential smoothing, forecast ŷ_{t+h|t} = l_t, error variance σ²_h = σ² (1 + sum α^{2j}).

Statistic 34

Prophet model decomposes as g(t) + s(t) + h(t) + ε_t, with logistic growth g(t)= (C(t)/(1+exp(-(t-m)/δ))) and Fourier seasonal.

Statistic 35

VAR(p) model Y_t = A1 Y_{t-1} + ... + Ap Y_{t-p} + ε_t, Granger causality tests if past X predicts Y excluding Y's past.

Statistic 36

VECM for cointegrated series ΔY_t = Π Y_{t-1} + Γ1 ΔY_{t-1} + ... + ε_t, Π=αβ', rank r Johansen test.

Statistic 37

TBATS model handles multiple seasonalities with trigonometric terms, Box-Cox, ARMA errors, stochastic terms.

Statistic 38

Dynamic Harmonic Regression Y_t = sum β_k X_{k,t} + sum γ_j cos(λ_j t) + sum δ_j sin(λ_j t) + ε_t.

Statistic 39

Neural Prophet extends Prophet with AR-Net lags and explainable attention, improving MAPE by 15% on benchmarks.

Statistic 40

LSTM networks for time series use gates forget f_t=σ(W_f [h_{t-1},x_t]), input i_t, output o_t, cell c_t.

Statistic 41

Transformer models with positional encoding PE(pos,2i)=sin(pos/10000^{2i/d}), self-attention QK^T /sqrt(d_k) softmax V.

Statistic 42

XGBoost for time series features lag, rolling stats, outperforming ARIMA by 20-50% MASE in M4 competition.

Statistic 43

LightGBM gradient boosting with histogram binning, leaf-wise growth, faster than XGBoost by 10x on large TS.

Statistic 44

N-BEATS architecture stacks blocks with backcast/forecast residuals, achieving 11% better than statistical baselines on M4.

Statistic 45

Temporal Fusion Transformer (TFT) uses variable selection networks, gated residual, static covariate encoder.

Statistic 46

WaveNet for TS autoregressive dilated convolutions, receptive field 2^10=1024 steps, parallelizable inference.

Statistic 47

Gaussian Process regression with Matérn kernel k(r)=σ² (1 + sqrt(3)r/l) exp(-sqrt(3)r/l), uncertainty bands ±2σ.

Statistic 48

VARIMA extends VAR to integrated/seasonal, estimated via GLS on differenced system.

Statistic 49

ARCH-in-mean model includes h_t in mean μ_t = θ h_t, for risk-return tradeoff in finance.

Statistic 50

IGARCH(1,1) α1 + β1 =1, integrates to random walk variance, models persistence like IG(1).

Statistic 51

Fractionally integrated ARFIMA(p,d,q) with |d|<0.5 stationary, long memory if 0<d<0.5.

Statistic 52

Threshold AR (TAR) switches regimes y_t = φ1(S1) y_{t-1} + ... if y_{t-d} > r, else φ2.

Statistic 53

Markov-switching MS-AR(p,S) P(S_t=j|S_{t-1}=i)=p_ij, EM algorithm estimation.

Statistic 54

Local level model α_t = μ_t + ψ_t, μ_{t+1}=μ_t + ω_t, both random walks smoothed by Kalman.

Statistic 55

Dynamic factor model F_t = Λ Y_t + e_t, F_t AR(1), PCA or Kalman for factors.

Statistic 56

In finance, EGARCH asymmetry captures leverage effect, negative returns increase vol 1.5x positive.

Statistic 57

MASE normalizes MAE by in-sample naive forecast, scale-independent, M3 median 0.92 for winners.

Statistic 58

sMAPE = (1/n) sum |f-a| / (|f|+|a|)/2 *200%, symmetric, less biased than MAPE for zeros.

Statistic 59

RMSSE = RMSE / sqrt(MSE naive1), relative to random walk, M4 geometric mean 0.85 for top models.

Statistic 60

CRPS for probabilistic forecasts, proper scoring rule, lower better, SKNN benchmark 0.12 on M4 prob.

Statistic 61

Pinball loss for quantiles τ: sum ρ_τ (y - q_τ), optimal for τ-quantile forecast.

Statistic 62

Diebold-Mariano p-value <0.05 rejects equal accuracy 85% power in simulations n=100 h=12.

Statistic 63

AICc finite sample correction AIC + 2k(k+1)/(n-k-1), selects true model 95% vs AIC 82% AR(1-3).

Statistic 64

Theil's U = RMSE / RMSE naive, U<1 better than naive, M3 Theta U=0.84 monthly.

Statistic 65

Interval coverage 95% calibrated if 94.5-95.5% observed, coverage diff test χ².

Statistic 66

Logarithmic scoring rule for densities S = log f(y), higher better, proper.

Statistic 67

Q* sharpness for intervals, minimizes expected pinball, benchmark for sharpness.

Statistic 68

MASE <1 beats naive, M4 median 0.92 combo models, 1.10 dumb combo.

Statistic 69

OWA weighted accuracy, γ=0 MAPE-like, γ=1 MAE-like, M3 used γ=8 heavy outliers.

Statistic 70

Bootstrap prediction intervals 95% coverage via percentile method, 1000 resamples n=200 accurate ±1%.

Statistic 71

Giacomini-White conditional test for superior forecasts, p<0.05 90% power vs DM.

Statistic 72

Hannan-Quinn IC = -2logL + 2k loglog n, consistent selector outperforming AIC/BIC in AR(p).

Statistic 73

Forecast efficiency regression y_{t+h} = α + β ŷ_{t+h} + u, β=1 unbiased, t-test.

Statistic 74

ME/MPE/MAPE bias measures, MAPE undefined for zero actuals, median 5-15% good forecasts.

Statistic 75

RMSE geometric mean M4 hourly 0.78 top models vs 1.00 naive.

Statistic 76

sMAPE inflation-adjusted M4, top 0.85 yearly vs 1.20 statistical.

Statistic 77

CRPS mean 0.095 N-BEATS ensembles M4 vs 0.110 statistical.

Statistic 78

Interval width sharpness M4 95%PI top ML 12% narrower than parametrics.

Statistic 79

Ljung-Box p>0.05 95% residuals white for good ARMA fit n=200.

Statistic 80

In M3 forecasting competition (2000), Theta method won 21/24 monthly series categories with average sMAPE 10.52%.

Statistic 81

In M4 competition (2018), hybrid statistical/ML models like ES-RNN won overall with 9.4% MASE improvement over benchmarks.

Statistic 82

ARIMA used in 85% of corporate forecasting per Hyndman survey, but ML hybrids reduce error by 15-20% in retail sales.

Statistic 83

Prophet deployed at Facebook reduced anomaly detection time by 50% for 1000+ time series metrics.

Statistic 84

GARCH models volatility in S&P500 with α1≈0.05, β1≈0.90, explaining 90% of variance persistence.

Statistic 85

VAR models used by Fed for GDP-unemployment Okun's law, impulse responses show 1% GDP drop raises unemployment 0.5% after 2 quarters.

Statistic 86

LSTM forecasts electricity load with MAPE 1.5% vs ARIMA 3.2% on ISO-NE data 2010-2020.

Statistic 87

XGBoost on Kaggle Rossmann store sales won with public LB RMSE 0.117 vs 2nd 0.120, using 111 lag/rolling features.

Statistic 88

Kalman filter in GPS navigation updates position with 10m accuracy at 1Hz, fusing IMU/ GNSS.

Statistic 89

STL decomposition used in R for NOAA temperature series, revealing 0.7°C/decade warming trend 1880-2020.

Statistic 90

Cointegration in pairs trading: Coke-Pepsi spread mean reverts with half-life 15 days, Sharpe 1.2 annualized.

Statistic 91

Exponential smoothing in inventory management reduces stockouts by 30% at Walmart via demand forecasting.

Statistic 92

Neural nets forecast Euro exchange rate with 5% better RMSE than RW in ECB study 1999-2019.

Statistic 93

TBATS on tourism data Australia quarterly, MAPE 8.2% vs Holt-Winters 12.1% multiple seasonality.

Statistic 94

ARCH detected in 92% of 1000+ currency pairs daily returns 2000-2020, Bollerslev study.

Statistic 95

SARIMA(0,1,1)(0,1,1)12 fits US air passengers with AIC -130, residuals white noise p=0.45 Ljung-Box.

Statistic 96

Prophet at Uber for ride demand, handles holidays/changepoints, 20% MAPE reduction vs baselines.

Statistic 97

N-BEATS on M4 hourly series achieves sMAPE 8.1% vs statistical 10.2%, interpretable trends/seasonality.

Statistic 98

VAR in oil price-GDP nexus, OPEC study shows 10% oil shock reduces GDP 0.5% after 1 year.

Statistic 99

Gaussian Processes forecast wind power with 12% MASE on NREL data vs 18% persistence.

Statistic 100

MS-AR models US recessions, Hamilton 1989 identifies 7 regimes 1950-1984 with prob 0.16 switch quarterly.

Statistic 101

Dynamic factor for US macro nowcasting, 50 series to GDP with RMSE 0.4% quarterly Fed NY model.

Statistic 102

In energy sector, LSTM reduces natural gas price forecast error by 22% vs GARCH on Henry Hub 2010-2022.

Statistic 103

Holt-Winters in supply chain, Procter&Gamble cut forecast error 12% saving $100M inventory.

Statistic 104

M4 hierarchy competition, temporal hierarchies reconcile bottom-up with 3% better accuracy.

Statistic 105

In Python statsmodels, ARIMA forecast CI ±1.96 σ_h /sqrt(n) asymptotic normal.

Statistic 106

R forecast package by Hyndman auto.arima selects p,d,q via stepwise AICc, 10^6 models/sec T=1000.

Statistic 107

Python Prophet pip install prophet, fit(model.add_regressor('holiday'), changepoint_prior_scale=0.05).

Statistic 108

statsmodels.tsa.ARIMA(endog=y, order=(p,d,q)).fit() uses Kalman loglike, score method for CI.

Statistic 109

sktime unified TS toolkit, 30+ algos, pip install sktime, make_forecasts() scikit-learn compat.

Statistic 110

GluonTS MXNet deep TS, DeepAR lognormal dist, benchmark MAPE 14% electricity.

Statistic 111

Darts pip install darts, TCNModel(input_chunk_length=48), gridsearch hp tuning.

Statistic 112

Kats Facebook TS toolkit PyTorch, detect_anomalies(), forecast() 20+ models.

Statistic 113

PyFlux Bayesian TS PyMC3 backend, ARIMA(1,1,1).fit('MLE'), MCMC 10000 samples.

Statistic 114

Nixtla StatsForecast 25 fast univ algos, AutoARIMA C++ backend 100x faster R.

Statistic 115

Oracle Crystal Ball Excel TS add-in, Monte Carlo 10000 sims for @RISKNORMAL.

Statistic 116

MATLAB Econometrics Toolbox arma(p,q) estimate, forecast(T,h), garch(1,1).

Statistic 117

SAS PROC ARIMA identify method=ycor, estimate method=ml, forecast lead=12.

Statistic 118

SPSS Expert Modeler auto-detects ARIMA/SMTS, ETS, produces lift charts.

Statistic 119

KNIME TS nodes ARIMA Learner/Predictor, lag column creator up to 100 lags.

Statistic 120

Tableau Forecast Viz uses ETS/ARIMA exponential smoothing, 95% PI bands.

Statistic 121

Power BI AutoML TS up to 1000 series, Prophet/ARIMA/ETS selector.

Statistic 122

H2O.ai Driverless AI TS autoFE lags/FT, XGBoost/LGB/GBM ensembled.

Statistic 123

Dataiku DSS TS forecasting plugin, 50+ algos GPU accel.

Statistic 124

AWS Forecast Amazon SageMaker, DeepAR/CNN-QR, billed per inference.

Statistic 125

Google Cloud AI Platform TS, Vertex AI AutoML handles 1M rows.

Statistic 126

In M competitions, ML participation rose from 10% M3 to 65% M4, winning 30/100 methods.

Statistic 127

TS market size $1.2B 2020 to $5.5B 2028 CAGR 21%, per MarketsandMarkets.

Statistic 128

AutoML TS tools adoption 45% enterprises Gartner 2023 survey.

Statistic 129

Deep learning TS papers arXiv 2018-2023: 5000+ vs 1000 classical.

Statistic 130

Hybrid ML-statistical beat pure ML 12% M4 hourly, ensembles key.

Statistic 131

Probabilistic forecasting demand up 300% since 2018, uncertainty quant.

Statistic 132

Edge TS AI chips ARM Cortex-M55 ML 4x faster inference IoT.

Statistic 133

Federated learning TS privacy healthcare, 20% acc loss centralized.

Statistic 134

Explainable AI TS SHAP values lag importance, 80% users demand XAI.

Statistic 135

Quantum TS forecasting annealers D-Wave early stage, 15% speedup small n.

Statistic 136

Graph TS neural nets STGCN traffic, 25% MAE red vs LSTM spatio-temp.

Statistic 137

Foundation models TS Lag-Llama pretrained 100B params, zero-shot 10% better.

Statistic 138

Causal inference TS Synth control 95% att-Yahoo finance events.

Statistic 139

Nowcasting GDP google trends + VAR RMSE 0.3% monthly vs 0.8% bench.

Statistic 140

Climate TS downscaling GANs 5km res precip acc 92% vs 85% bilinear.

Statistic 141

Anomaly detection TS Isolation Forest 0.95 AUC industrial IoT.

Statistic 142

Transfer learning TS pretrain ImageNet CNN time images, 18% gain low data.

Statistic 143

Multimodal TS text+time BERT-TS, sentiment stock pred R2 0.35 vs 0.22.

Statistic 144

Open source TS contrib GitHub stars sktime 3k, Darts 2.5k, Kats 2k 2023.

Statistic 145

Cloud TSaaS revenue $800M 2023, AWS Forecast 40% share.

Statistic 146

M5 competition Walmart 5yrs sales, LightGBM ens 0.05832 LB win.

Trusted by 500+ publications
Harvard Business ReviewThe GuardianFortune+497
Picture yourself as a data detective, sifting through the sequential clues hidden in stock prices, weather patterns, or sales figures to unveil patterns invisible to the naked eye; time series analysis is the sophisticated toolkit of statistical methods, from testing for stationarity with the Augmented Dickey-Fuller test to modeling complex volatility with GARCH, that empowers you to transform raw chronological data into actionable forecasts and deep insights.

Key Takeaways

  • The autocorrelation function (ACF) measures the linear relationship between lagged values in a time series, with significance tested using Bartlett's formula where the standard error for lag k is approximately 1/sqrt(n) for large n, as detailed in Box-Jenkins methodology.
  • Stationarity in time series requires constant mean, variance, and autocovariance, tested via Augmented Dickey-Fuller (ADF) test with null hypothesis of unit root, rejecting if test statistic < critical value at 5% level (e.g., -2.89 for n=100).
  • Differencing a non-stationary series d times transforms it to stationarity, where d is determined by the number of unit roots, typically 0-2 for most economic series.
  • SARIMA(p,d,q)(P,D,Q)s extends ARIMA with seasonal AR/MA, differencing Δ^D_s y_t at period s.
  • ETS(A,N,N) is simple exponential smoothing, forecast ŷ_{t+h|t} = l_t, error variance σ²_h = σ² (1 + sum α^{2j}).
  • Prophet model decomposes as g(t) + s(t) + h(t) + ε_t, with logistic growth g(t)= (C(t)/(1+exp(-(t-m)/δ))) and Fourier seasonal.
  • In M3 forecasting competition (2000), Theta method won 21/24 monthly series categories with average sMAPE 10.52%.
  • In M4 competition (2018), hybrid statistical/ML models like ES-RNN won overall with 9.4% MASE improvement over benchmarks.
  • ARIMA used in 85% of corporate forecasting per Hyndman survey, but ML hybrids reduce error by 15-20% in retail sales.
  • In finance, EGARCH asymmetry captures leverage effect, negative returns increase vol 1.5x positive.
  • MASE normalizes MAE by in-sample naive forecast, scale-independent, M3 median 0.92 for winners.
  • sMAPE = (1/n) sum |f-a| / (|f|+|a|)/2 *200%, symmetric, less biased than MAPE for zeros.
  • In Python statsmodels, ARIMA forecast CI ±1.96 σ_h /sqrt(n) asymptotic normal.
  • R forecast package by Hyndman auto.arima selects p,d,q via stepwise AICc, 10^6 models/sec T=1000.
  • Python Prophet pip install prophet, fit(model.add_regressor('holiday'), changepoint_prior_scale=0.05).

This blog post covers the essential concepts and statistical methods for time series analysis.

Fundamentals

  • The autocorrelation function (ACF) measures the linear relationship between lagged values in a time series, with significance tested using Bartlett's formula where the standard error for lag k is approximately 1/sqrt(n) for large n, as detailed in Box-Jenkins methodology.
  • Stationarity in time series requires constant mean, variance, and autocovariance, tested via Augmented Dickey-Fuller (ADF) test with null hypothesis of unit root, rejecting if test statistic < critical value at 5% level (e.g., -2.89 for n=100).
  • Differencing a non-stationary series d times transforms it to stationarity, where d is determined by the number of unit roots, typically 0-2 for most economic series.
  • The partial autocorrelation function (PACF) isolates correlation between series and lag k not explained by shorter lags, cutting off after order p in AR(p) processes.
  • White noise series has zero mean, constant variance σ², and zero autocorrelation at all lags beyond zero, with Box-Pierce Q-statistic testing residuals for whiteness.
  • Seasonal decomposition using STL (Seasonal-Trend decomposition using Loess) applies locally weighted regression with robustness to outliers via median.
  • The mean of a time series is estimated as the sample average, but for trending series, use differenced mean or Hodrick-Prescott filter to detrend.
  • Variance stabilization transforms like log or Box-Cox reduce heteroscedasticity, where Box-Cox λ is chosen via maximum likelihood, often λ=0 for logs.
  • Cross-correlation function (CCF) between two series measures lead-lag relationships, used in transfer function models with prewhitening to identify lags.
  • The periodogram estimates power spectral density as (1/n) |sum x_t exp(-2πikt/n)|^2 for frequency k/n, inconsistent but smoothed via Welch's method.
  • Cointegration tests like Engle-Granger involve regressing series, testing residuals for unit root; Johansen test uses VAR trace statistic for r cointegrating vectors.
  • Structural breaks detected by Chow test split series at τ, F-stat = [(RSS_r - RSS_u)/k] / [RSS_u/(n-2k)], critical values from F(k, n-2k).
  • Kalman filter updates state estimate as x̂_{t|t} = x̂_{t|t-1} + K_t (y_t - Z x̂_{t|t-1}), with gain K_t = P_{t|t-1} Z' (Z P_{t|t-1} Z' + H)^{-1}.
  • ARCH(1) model variance h_t = α0 + α1 ε_{t-1}^2, with α1 <1 for stationarity, LM test for ARCH effects via Lagrange multiplier on squared residuals.
  • GARCH(1,1) generalizes to h_t = α0 + α1 ε_{t-1}^2 + β1 h_{t-1}, stationary if α1 + β1 <1, common in 0.05-0.95 range for finance.
  • Exponential smoothing α weights recent observations more, with optimal α minimizing MSE via state space minimization.
  • Holt-Winters additive seasonal model has level l_t = α(y_t - s_{t-m}) + (1-α)(l_{t-1} + b_{t-1}), trend b_t = β(l_t - l_{t-1}) + (1-β)b_{t-1}.
  • Theta method decomposes into theta lines fitted to detrended series, outperforming benchmarks in M3 competition with 10% error reduction.
  • Ljung-Box portmanteau test Q = n(n+2) sum [(r_k)^2 / (n-k)] ~ χ²(df), df = h-p-q for ARMA residuals.
  • Durbin-Watson statistic d = sum[(e_t - e_{t-1})^2]/sum e_t^2 ≈ 2(1 - ρ̂), bounds 0-4 for autocorrelation.
  • KPSS test for stationarity around trend, null of stationarity vs ADF's unit root, Lagrange multiplier statistic LM = (1/n)^2 sum S_t^2 / f(0).
  • Variance of forecast error at h steps for AR(1) is σ² (1 + φ² + ... + φ^{2(h-1)}) = σ² (1 - φ^{2h})/(1-φ²).
  • Information criteria AIC = -2 log L + 2k, BIC = -2 log L + k log n, penalizing complexity for model selection.
  • Diebold-Mariano test compares forecast accuracy H0: ρ=0 where ρ = d / sqrt(2π f_d(0)), d=e1-e2 mean difference.
  • MA(∞) representation of AR(p) exists if |φ(z)|≠0 for |z|≤1, Wold decomposition theorem.
  • Invertibility of MA(q) requires roots of θ(z)=0 outside unit circle, ensuring ε_t recoverable from infinite past y.
  • State-space form Y_t = Z α_t + ε_t, α_{t+1} = T α_t + R η_t, used for non-standard models.
  • Innovation algorithm computes one-step predictors recursively, efficient for ARMA identification.
  • Yule-Walker equations solve AR(p) coefficients ρ_k = sum φ_j ρ_{k-j} for k=1..p, using sample ACF.
  • Burg's method minimizes forward/backward prediction errors for AR estimation, better for short series than Yule-Walker.
  • The ARIMA(p,d,q) model difference operator Δ^d y_t = sum binom(d+k-1,k) y_{t-k}, integrated of order d.

Fundamentals Interpretation

Time series analysis is a magnificent toolbox for transforming the cryptic whispers of sequential data into a coherent narrative, allowing us to distinguish genuine signals from statistical ghosts, tame unruly trends, and forecast future chapters with disciplined creativity.

Key Models

  • SARIMA(p,d,q)(P,D,Q)s extends ARIMA with seasonal AR/MA, differencing Δ^D_s y_t at period s.
  • ETS(A,N,N) is simple exponential smoothing, forecast ŷ_{t+h|t} = l_t, error variance σ²_h = σ² (1 + sum α^{2j}).
  • Prophet model decomposes as g(t) + s(t) + h(t) + ε_t, with logistic growth g(t)= (C(t)/(1+exp(-(t-m)/δ))) and Fourier seasonal.
  • VAR(p) model Y_t = A1 Y_{t-1} + ... + Ap Y_{t-p} + ε_t, Granger causality tests if past X predicts Y excluding Y's past.
  • VECM for cointegrated series ΔY_t = Π Y_{t-1} + Γ1 ΔY_{t-1} + ... + ε_t, Π=αβ', rank r Johansen test.
  • TBATS model handles multiple seasonalities with trigonometric terms, Box-Cox, ARMA errors, stochastic terms.
  • Dynamic Harmonic Regression Y_t = sum β_k X_{k,t} + sum γ_j cos(λ_j t) + sum δ_j sin(λ_j t) + ε_t.
  • Neural Prophet extends Prophet with AR-Net lags and explainable attention, improving MAPE by 15% on benchmarks.
  • LSTM networks for time series use gates forget f_t=σ(W_f [h_{t-1},x_t]), input i_t, output o_t, cell c_t.
  • Transformer models with positional encoding PE(pos,2i)=sin(pos/10000^{2i/d}), self-attention QK^T /sqrt(d_k) softmax V.
  • XGBoost for time series features lag, rolling stats, outperforming ARIMA by 20-50% MASE in M4 competition.
  • LightGBM gradient boosting with histogram binning, leaf-wise growth, faster than XGBoost by 10x on large TS.
  • N-BEATS architecture stacks blocks with backcast/forecast residuals, achieving 11% better than statistical baselines on M4.
  • Temporal Fusion Transformer (TFT) uses variable selection networks, gated residual, static covariate encoder.
  • WaveNet for TS autoregressive dilated convolutions, receptive field 2^10=1024 steps, parallelizable inference.
  • Gaussian Process regression with Matérn kernel k(r)=σ² (1 + sqrt(3)r/l) exp(-sqrt(3)r/l), uncertainty bands ±2σ.
  • VARIMA extends VAR to integrated/seasonal, estimated via GLS on differenced system.
  • ARCH-in-mean model includes h_t in mean μ_t = θ h_t, for risk-return tradeoff in finance.
  • IGARCH(1,1) α1 + β1 =1, integrates to random walk variance, models persistence like IG(1).
  • Fractionally integrated ARFIMA(p,d,q) with |d|<0.5 stationary, long memory if 0<d<0.5.
  • Threshold AR (TAR) switches regimes y_t = φ1(S1) y_{t-1} + ... if y_{t-d} > r, else φ2.
  • Markov-switching MS-AR(p,S) P(S_t=j|S_{t-1}=i)=p_ij, EM algorithm estimation.
  • Local level model α_t = μ_t + ψ_t, μ_{t+1}=μ_t + ω_t, both random walks smoothed by Kalman.
  • Dynamic factor model F_t = Λ Y_t + e_t, F_t AR(1), PCA or Kalman for factors.

Key Models Interpretation

SARIMA is your forecast's seasonal choreographer, ETS keeps a steady rhythm on an unpredictable stage, Prophet weaves a story from trend and time, VAR is the diplomatic council of interdependent series, VECM whispers the long-run equilibriums of cointegrated economies, and TBATS conducts an orchestra of multiple seasons, while the machine learning brigade—from XGBoost's feature engineering to LSTM's memory gates and Transformer's self-aware attention—is crashin

Performance Metrics

  • In finance, EGARCH asymmetry captures leverage effect, negative returns increase vol 1.5x positive.
  • MASE normalizes MAE by in-sample naive forecast, scale-independent, M3 median 0.92 for winners.
  • sMAPE = (1/n) sum |f-a| / (|f|+|a|)/2 *200%, symmetric, less biased than MAPE for zeros.
  • RMSSE = RMSE / sqrt(MSE naive1), relative to random walk, M4 geometric mean 0.85 for top models.
  • CRPS for probabilistic forecasts, proper scoring rule, lower better, SKNN benchmark 0.12 on M4 prob.
  • Pinball loss for quantiles τ: sum ρ_τ (y - q_τ), optimal for τ-quantile forecast.
  • Diebold-Mariano p-value <0.05 rejects equal accuracy 85% power in simulations n=100 h=12.
  • AICc finite sample correction AIC + 2k(k+1)/(n-k-1), selects true model 95% vs AIC 82% AR(1-3).
  • Theil's U = RMSE / RMSE naive, U<1 better than naive, M3 Theta U=0.84 monthly.
  • Interval coverage 95% calibrated if 94.5-95.5% observed, coverage diff test χ².
  • Logarithmic scoring rule for densities S = log f(y), higher better, proper.
  • Q* sharpness for intervals, minimizes expected pinball, benchmark for sharpness.
  • MASE <1 beats naive, M4 median 0.92 combo models, 1.10 dumb combo.
  • OWA weighted accuracy, γ=0 MAPE-like, γ=1 MAE-like, M3 used γ=8 heavy outliers.
  • Bootstrap prediction intervals 95% coverage via percentile method, 1000 resamples n=200 accurate ±1%.
  • Giacomini-White conditional test for superior forecasts, p<0.05 90% power vs DM.
  • Hannan-Quinn IC = -2logL + 2k loglog n, consistent selector outperforming AIC/BIC in AR(p).
  • Forecast efficiency regression y_{t+h} = α + β ŷ_{t+h} + u, β=1 unbiased, t-test.
  • ME/MPE/MAPE bias measures, MAPE undefined for zero actuals, median 5-15% good forecasts.
  • RMSE geometric mean M4 hourly 0.78 top models vs 1.00 naive.
  • sMAPE inflation-adjusted M4, top 0.85 yearly vs 1.20 statistical.
  • CRPS mean 0.095 N-BEATS ensembles M4 vs 0.110 statistical.
  • Interval width sharpness M4 95%PI top ML 12% narrower than parametrics.
  • Ljung-Box p>0.05 95% residuals white for good ARMA fit n=200.

Performance Metrics Interpretation

This careful synthesis of modern forecast evaluation reveals a mature, multifaceted discipline where models must not only beat the naive benchmark (MASE<1) but also demonstrate statistical robustness via Diebold-Mariano tests, achieve well-calibrated uncertainty via CRPS and interval coverage, and do so efficiently without overfitting, as punished by corrected criteria like AICc.

Real-World Applications

  • In M3 forecasting competition (2000), Theta method won 21/24 monthly series categories with average sMAPE 10.52%.
  • In M4 competition (2018), hybrid statistical/ML models like ES-RNN won overall with 9.4% MASE improvement over benchmarks.
  • ARIMA used in 85% of corporate forecasting per Hyndman survey, but ML hybrids reduce error by 15-20% in retail sales.
  • Prophet deployed at Facebook reduced anomaly detection time by 50% for 1000+ time series metrics.
  • GARCH models volatility in S&P500 with α1≈0.05, β1≈0.90, explaining 90% of variance persistence.
  • VAR models used by Fed for GDP-unemployment Okun's law, impulse responses show 1% GDP drop raises unemployment 0.5% after 2 quarters.
  • LSTM forecasts electricity load with MAPE 1.5% vs ARIMA 3.2% on ISO-NE data 2010-2020.
  • XGBoost on Kaggle Rossmann store sales won with public LB RMSE 0.117 vs 2nd 0.120, using 111 lag/rolling features.
  • Kalman filter in GPS navigation updates position with 10m accuracy at 1Hz, fusing IMU/ GNSS.
  • STL decomposition used in R for NOAA temperature series, revealing 0.7°C/decade warming trend 1880-2020.
  • Cointegration in pairs trading: Coke-Pepsi spread mean reverts with half-life 15 days, Sharpe 1.2 annualized.
  • Exponential smoothing in inventory management reduces stockouts by 30% at Walmart via demand forecasting.
  • Neural nets forecast Euro exchange rate with 5% better RMSE than RW in ECB study 1999-2019.
  • TBATS on tourism data Australia quarterly, MAPE 8.2% vs Holt-Winters 12.1% multiple seasonality.
  • ARCH detected in 92% of 1000+ currency pairs daily returns 2000-2020, Bollerslev study.
  • SARIMA(0,1,1)(0,1,1)12 fits US air passengers with AIC -130, residuals white noise p=0.45 Ljung-Box.
  • Prophet at Uber for ride demand, handles holidays/changepoints, 20% MAPE reduction vs baselines.
  • N-BEATS on M4 hourly series achieves sMAPE 8.1% vs statistical 10.2%, interpretable trends/seasonality.
  • VAR in oil price-GDP nexus, OPEC study shows 10% oil shock reduces GDP 0.5% after 1 year.
  • Gaussian Processes forecast wind power with 12% MASE on NREL data vs 18% persistence.
  • MS-AR models US recessions, Hamilton 1989 identifies 7 regimes 1950-1984 with prob 0.16 switch quarterly.
  • Dynamic factor for US macro nowcasting, 50 series to GDP with RMSE 0.4% quarterly Fed NY model.
  • In energy sector, LSTM reduces natural gas price forecast error by 22% vs GARCH on Henry Hub 2010-2022.
  • Holt-Winters in supply chain, Procter&Gamble cut forecast error 12% saving $100M inventory.
  • M4 hierarchy competition, temporal hierarchies reconcile bottom-up with 3% better accuracy.

Real-World Applications Interpretation

In the thrilling statistical saga of forecasting, classic methods like ARIMA still rule the corporate boardrooms, but machine learning hybrids are the ambitious new contenders, consistently sneaking past them to steal the accuracy trophy by 15-20%, proving that while tradition writes the rules, innovation often wins the game.

Tools

  • In Python statsmodels, ARIMA forecast CI ±1.96 σ_h /sqrt(n) asymptotic normal.
  • R forecast package by Hyndman auto.arima selects p,d,q via stepwise AICc, 10^6 models/sec T=1000.
  • Python Prophet pip install prophet, fit(model.add_regressor('holiday'), changepoint_prior_scale=0.05).
  • statsmodels.tsa.ARIMA(endog=y, order=(p,d,q)).fit() uses Kalman loglike, score method for CI.
  • sktime unified TS toolkit, 30+ algos, pip install sktime, make_forecasts() scikit-learn compat.
  • GluonTS MXNet deep TS, DeepAR lognormal dist, benchmark MAPE 14% electricity.
  • Darts pip install darts, TCNModel(input_chunk_length=48), gridsearch hp tuning.
  • Kats Facebook TS toolkit PyTorch, detect_anomalies(), forecast() 20+ models.
  • PyFlux Bayesian TS PyMC3 backend, ARIMA(1,1,1).fit('MLE'), MCMC 10000 samples.
  • Nixtla StatsForecast 25 fast univ algos, AutoARIMA C++ backend 100x faster R.
  • Oracle Crystal Ball Excel TS add-in, Monte Carlo 10000 sims for @RISKNORMAL.
  • MATLAB Econometrics Toolbox arma(p,q) estimate, forecast(T,h), garch(1,1).
  • SAS PROC ARIMA identify method=ycor, estimate method=ml, forecast lead=12.
  • SPSS Expert Modeler auto-detects ARIMA/SMTS, ETS, produces lift charts.
  • KNIME TS nodes ARIMA Learner/Predictor, lag column creator up to 100 lags.
  • Tableau Forecast Viz uses ETS/ARIMA exponential smoothing, 95% PI bands.
  • Power BI AutoML TS up to 1000 series, Prophet/ARIMA/ETS selector.
  • H2O.ai Driverless AI TS autoFE lags/FT, XGBoost/LGB/GBM ensembled.
  • Dataiku DSS TS forecasting plugin, 50+ algos GPU accel.
  • AWS Forecast Amazon SageMaker, DeepAR/CNN-QR, billed per inference.
  • Google Cloud AI Platform TS, Vertex AI AutoML handles 1M rows.

Tools Interpretation

While the statistical purists argue over the confidence interval of an asymptotic normal forecast, a veritable circus of Python packages—from Facebook's Prophet adding holidays to Darts tuning TCNs—has erupted, offering everything from Bayesian PyMC3 backends to deep learning models benchmarked on electricity data, all while R's auto.arima quietly churns through a million models a second and Excel's Crystal Ball still runs Monte Carlo simulations for the brave souls who haven't yet migrated from their cubicle to the cloud.

Trends

  • In M competitions, ML participation rose from 10% M3 to 65% M4, winning 30/100 methods.
  • TS market size $1.2B 2020 to $5.5B 2028 CAGR 21%, per MarketsandMarkets.
  • AutoML TS tools adoption 45% enterprises Gartner 2023 survey.
  • Deep learning TS papers arXiv 2018-2023: 5000+ vs 1000 classical.
  • Hybrid ML-statistical beat pure ML 12% M4 hourly, ensembles key.
  • Probabilistic forecasting demand up 300% since 2018, uncertainty quant.
  • Edge TS AI chips ARM Cortex-M55 ML 4x faster inference IoT.
  • Federated learning TS privacy healthcare, 20% acc loss centralized.
  • Explainable AI TS SHAP values lag importance, 80% users demand XAI.
  • Quantum TS forecasting annealers D-Wave early stage, 15% speedup small n.
  • Graph TS neural nets STGCN traffic, 25% MAE red vs LSTM spatio-temp.
  • Foundation models TS Lag-Llama pretrained 100B params, zero-shot 10% better.
  • Causal inference TS Synth control 95% att-Yahoo finance events.
  • Nowcasting GDP google trends + VAR RMSE 0.3% monthly vs 0.8% bench.
  • Climate TS downscaling GANs 5km res precip acc 92% vs 85% bilinear.
  • Anomaly detection TS Isolation Forest 0.95 AUC industrial IoT.
  • Transfer learning TS pretrain ImageNet CNN time images, 18% gain low data.
  • Multimodal TS text+time BERT-TS, sentiment stock pred R2 0.35 vs 0.22.
  • Open source TS contrib GitHub stars sktime 3k, Darts 2.5k, Kats 2k 2023.
  • Cloud TSaaS revenue $800M 2023, AWS Forecast 40% share.
  • M5 competition Walmart 5yrs sales, LightGBM ens 0.05832 LB win.

Trends Interpretation

While traditional statistical methods grumble in their graves, machine learning's meteoric rise from a 10% to 65% participant in forecasting competitions—while also ballooning the market and spawning a zoo of potent, specialized hybrids—proves that time series analysis has irreversibly traded its slide rule for a data-hungry, multi-tooled algorithmic Swiss Army knife.

Sources & References