From predicting tomorrow’s rain with stunning 92% accuracy to foreseeing economic shifts with pinpoint precision, modern forecasting has quietly evolved from a game of chance into a sophisticated science.
Key Takeaways
- The global average temperature forecast accuracy for 90-day outlooks by NOAA's Climate Prediction Center improved from 65% in 1990 to 78% in 2020
- ECMWF's 5-day forecast skill score for 500 hPa geopotential height reached 85% accuracy over the Northern Hemisphere in 2022
- US National Weather Service's 1-day temperature forecast error averaged 2.1°F across 300 US stations in 2023
- IMF's World Economic Outlook GDP growth forecast error for advanced economies averaged 0.8% for 1-year horizon 2010-2023
- OECD's 2-year ahead GDP forecast RMSE for G7 countries was 1.2% in post-2008 period through 2022
- Federal Reserve's Greenbook inflation forecast bias for PCE was -0.1% annually from 1990-2022
- FiveThirtyEight's 2020 US Presidential election final popular vote forecast error was 0.9% for Biden vs Trump
- PredictIt's 2016 Brexit Remain probability averaged 25% error in final week polls
- The Economist's 2022 US midterms House popular vote forecast accuracy was 1.2% off
- Goldman Sachs 1-year S&P 500 return forecast correlation with actual was 0.42 from 1988-2023
- JPMorgan's 10-year Treasury yield 12-month forecast MAE was 45bps 2000-2023
- Bloomberg consensus oil price forecast error for Brent averaged $8/bbl for 6-months 2010-2023
- Tetlock's superforecasters achieved 30% higher Brier score than experts on geopolitical questions 2011-2015 Good Judgment Project
- Good Judgment Open tournament participants beat intelligence analysts by 20% in forecast accuracy on 200+ questions 2016-2020
- Philip Tetlock's Brier score for superforecasters averaged 0.22 vs 0.32 for regulars in Superforecasting trials
Forecasting has greatly improved in many fields over recent decades.
Economic Forecasting
- IMF's World Economic Outlook GDP growth forecast error for advanced economies averaged 0.8% for 1-year horizon 2010-2023
- OECD's 2-year ahead GDP forecast RMSE for G7 countries was 1.2% in post-2008 period through 2022
- Federal Reserve's Greenbook inflation forecast bias for PCE was -0.1% annually from 1990-2022
- ECB staff projections for Eurozone HICP inflation had MAE of 0.6% for 1-year ahead 2010-2023
- World Bank's global growth forecast revisions averaged +0.3% upward bias in EMDEs 2000-2022
- Consensus Economics survey median error for US GDP was 0.4% for Q1-ahead forecasts 1980-2023
- Bank of England's fan chart 2-year CPI forecast contained actual 68% of time in 1997-2022
- Japan's Cabinet Office GDP deflator forecast skill score was 0.65 correlation 2010-2023
- China's NBS industrial production forecast error averaged 1.1% monthly from 2015-2023
- Eurostat nowcasts for EU unemployment rate had RMSE 0.3% in 2020-2023 pandemic
- Australia's RBA trimmed mean inflation forecast MAE was 0.5% for 2-years 2000-2022
- Brazil's Central Bank Selic rate forecast hit rate was 72% within 25bp for 12-months 2018-2023
- India's RBI repo rate forecast accuracy was 85% directionally correct 2015-2023
- South Africa's SARB GDP forecast RMSE was 1.4% for 1-year in 2010-2022
- Russia's CBR inflation forecast central tendency error 1.2% for 2020-2023
- Mexico's Banxico growth forecast bias -0.2% for 2-years 2015-2023
Economic Forecasting Interpretation
For all our sophisticated models and solemn pronouncements, the grand takeaway from two decades of global economic forecasting is that we're consistently about one percentage point politely wrong, with a gentle, optimistic nudge for the developing world.
Financial Forecasting
- Goldman Sachs 1-year S&P 500 return forecast correlation with actual was 0.42 from 1988-2023
- JPMorgan's 10-year Treasury yield 12-month forecast MAE was 45bps 2000-2023
- Bloomberg consensus oil price forecast error for Brent averaged $8/bbl for 6-months 2010-2023
- Morgan Stanley EPS growth forecast for S&P 500 had 12% RMSE quarterly 2015-2023
- Citigroup currency forecast accuracy for EUR/USD 1-year direction was 55% 2005-2023
- Barclays gold price 3-month forecast bias +$15/oz in 2020-2023 volatility
- HSBC emerging market equity return forecast skill score 0.35 annually 2010-2022
- Deutsche Bank VIX 1-month forecast RMSE 4.2 points 2018-2023
- UBS Bitcoin price 6-month forecast error averaged 25% in 2021 bull run
- BlackRock ESG fund performance forecast correlation 0.28 vs benchmarks 2015-2023
- Fidelity sector rotation forecast hit rate 62% for top picks 2010-2022
- Vanguard bond yield curve 2-year shift forecast accuracy 78% direction 2000-2023
- PIMCO inflation-linked bond return forecast MAE 1.1% annually 2015-2023
- Schroders commodity index forecast skill 0.41 for 1-year 2005-2022
- Invesco real estate REIT forecast error 8% NAV growth 2018-2023
- State Street volatility forecast for MSCI World RMSE 2.8% monthly 2020-2023
Financial Forecasting Interpretation
Even the savviest financial prophets are essentially offering a sophisticated game of darts, where hitting the board is considered a success and occasionally getting close to the bullseye is a cause for celebration.
Political Forecasting
- FiveThirtyEight's 2020 US Presidential election final popular vote forecast error was 0.9% for Biden vs Trump
- PredictIt's 2016 Brexit Remain probability averaged 25% error in final week polls
- The Economist's 2022 US midterms House popular vote forecast accuracy was 1.2% off
- Nate Silver's 538 model gave 91% win probability to Biden in PA 2020, actual win by 1.2%
- UK Polling Average forecast error for 2019 election was 2.8% national vote share
- France's 2022 Presidential 1st round forecast by Ipsos had 1.5% error for Le Pen
- Germany's 2021 Bundestag election seat forecast by Wahlforschung had 5% error in SPD seats
- Australia's 2022 federal election 2PP forecast error averaged 1.7% by YouGov
- Brazil's 2022 Presidential forecast by Datafolha had 3.1% error for Lula 1st round
- India's 2019 Lok Sabha seat forecast by CSDS-Lokniti accurate within 4% for BJP
- Canada's 2021 election popular vote forecast RMSE 1.9% by 338Canada
- Italy's 2022 general election seat share forecast error 6% for FdI by SWG
- Spain's 2023 general election poll aggregator error 2.4% for PP vote
- Sweden's 2022 Riksdag forecast by Novus had 1.8% error for SD
- Netherlands 2023 election seat forecast accuracy 85% by Peilingwijzer
- Poland's 2023 Parliamentary vote forecast error 2.1% for PiS by CBOS
- Israel's 2022 election Knesset seats forecast off by 3 for Yesh Atid by Panels Politics
- Turkey's 2023 Presidential 1st round forecast error 4.5% for Erdogan by Konda
Political Forecasting Interpretation
While these forecasts are impressively close to the dartboard, they remind us that even the sharpest statistical dart can still land just shy of the bullseye in the messy game of democracy.
Superforecasting Accuracy
- Tetlock's superforecasters achieved 30% higher Brier score than experts on geopolitical questions 2011-2015 Good Judgment Project
- Good Judgment Open tournament participants beat intelligence analysts by 20% in forecast accuracy on 200+ questions 2016-2020
- Philip Tetlock's Brier score for superforecasters averaged 0.22 vs 0.32 for regulars in Superforecasting trials
- Prediction markets on PredictIt resolved 94% accurately for US election markets 2018-2022
- Metaculus forecasters median error for quantitative questions was 15% in 2023 tournaments
- Manifold Markets user forecasts achieved 85% calibration on resolved yes/no markets 2022-2023
- CSET Foretell superforecasters Brier score 0.18 on AI timelines 2021-2023
- GJP superforecasters improved calibration by 15% via training interventions 2011-2015
- Expert political forecasters averaged Brier score 0.28 vs superforecasters 0.19 in Tetlock studies 2000-2020
- Fox forecasters in GJP had 66% accuracy on binary events vs 73% for top superforecasters
- Ensemble of top 2% forecasters on Metaculus scored 25% better than median in pandemic forecasts 2020
- Kalshi prediction market volume correlated 0.72 with resolution accuracy on economic events 2022-2023
- Superforecaster training reduced overconfidence by 12% in probabilistic judgments per DARPA ACE
- Amateur forecasters on Good Judgment beat feds by 30% on foreign policy 2015-2018
- Top 1% Metaculus users achieved log score -1.2 average on science questions 2021-2023
Superforecasting Accuracy Interpretation
The data suggests that when it comes to predicting the future, a careful amateur with a spreadsheet and a skeptical mind will consistently, and rather embarrassingly, outperform the confident expert in a tailored suit.
Weather Forecasting
- The global average temperature forecast accuracy for 90-day outlooks by NOAA's Climate Prediction Center improved from 65% in 1990 to 78% in 2020
- ECMWF's 5-day forecast skill score for 500 hPa geopotential height reached 85% accuracy over the Northern Hemisphere in 2022
- US National Weather Service's 1-day temperature forecast error averaged 2.1°F across 300 US stations in 2023
- Hurricane track forecast errors for 48-72 hours reduced by 70% from 1990 to 2022, averaging 75 nautical miles, per NOAA
- UK's Met Office achieved 92.3% accuracy for daylight rain probability forecasts in 2021
- Japan's JMA seasonal typhoon intensity forecasts have a 68% correlation skill with observations from 2015-2023
- Australian Bureau of Meteorology's 7-day maximum temperature forecast RMSE was 1.8°C in 2022
- Canadian Environment Canada's 3-hour nowcast precipitation accuracy hit 88% in urban areas during 2021-2022 winter
- WMO's global ensemble prediction system shows 10-day forecast anomaly correlation of 0.75 for 500mb height in 2023
- Brazil's INMET 5-day rainfall forecast hit rate for exceedance thresholds was 82% in the Amazon region 2020-2023
- South Korea's KMA medium-range forecast CSI for heavy rain (>50mm/day) averaged 0.45 in 2022 monsoon season
- India's IMD monsoon onset forecast accuracy for Kerala coast was 95% correct in 16 out of 17 years from 2005-2023
- South Africa's SAWS 2-day wind speed forecast bias was under 1 m/s for coastal stations in 2021
- France's Météo-France 10-day temperature outlook skill score was 62% in 2022 Europe heatwave verification
- Germany's DWD probabilistic snowfall forecast Brier skill score reached 0.32 for Alps in winter 2022-2023
- Italy's ARPAV 24-hour fog forecast accuracy was 89% at Venice airport over 2018-2023
- Spain's AEMET seasonal drought forecast correlation was 0.72 for Iberian peninsula 2019-2023
- China's CMA 72-hour typhoon track forecast error averaged 90km in 2022 season
- Russia's Hydrometcentre 5-day cyclone track forecast skill improved to 80% in 2023 Arctic regions
- New Zealand's NIWA El Niño forecast hit rate was 75% for 3-month lead times 2010-2023
- Mexico's SMN hurricane intensity forecast error for 48h was 12 knots in 2022 eastern Pacific
- Argentina's SMN 7-day temperature forecast RMSE was 2.5°C in Pampas 2021-2023
- Egypt's EMA Nile flood forecast accuracy for peak level was 85% within 10cm from 2015-2023
Weather Forecasting Interpretation
While our forecasts for climate’s slow burn, tomorrow’s chill, and next week’s downpour grow more precise by the year, the chaotic truth remains that you should probably still keep an umbrella and a sense of humor handy.
Sources & References
- Reference 1CPCcpc.ncep.noaa.govVisit source
- Reference 2ECMWFecmwf.intVisit source
- Reference 3WEATHERweather.govVisit source
- Reference 4NHCnhc.noaa.govVisit source
- Reference 5METOFFICEmetoffice.gov.ukVisit source
- Reference 6JMAjma.go.jpVisit source
- Reference 7BOMbom.gov.auVisit source
- Reference 8WEATHERweather.gc.caVisit source
- Reference 9COMMUNITYcommunity.wmo.intVisit source
- Reference 10PORTALportal.inmet.gov.brVisit source
- Reference 11KMAkma.go.krVisit source
- Reference 12MAUSAMmausam.imd.gov.inVisit source
- Reference 13WEATHERSAweathersa.co.zaVisit source
- Reference 14METEOFRANCEmeteofrance.comVisit source
- Reference 15DWDdwd.deVisit source
- Reference 16ARPAarpa.veneto.itVisit source
- Reference 17AEMETaemet.esVisit source
- Reference 18CMAcma.gov.cnVisit source
- Reference 19METEOINFOmeteoinfo.ruVisit source
- Reference 20NIWAniwa.co.nzVisit source
- Reference 21SMNsmn.conagua.gob.mxVisit source
- Reference 22SMNsmn.gob.arVisit source
- Reference 23EMAema.gov.egVisit source
- Reference 24IMFimf.orgVisit source
- Reference 25DATAdata.oecd.orgVisit source
- Reference 26FEDERALRESERVEfederalreserve.govVisit source
- Reference 27ECBecb.europa.euVisit source
- Reference 28WORLDBANKworldbank.orgVisit source
- Reference 29CONSENSUSECONOMICSconsensuseconomics.comVisit source
- Reference 30BANKOFENGLANDbankofengland.co.ukVisit source
- Reference 31ESRIesri.cao.go.jpVisit source
- Reference 32STATSstats.gov.cnVisit source
- Reference 33ECec.europa.euVisit source
- Reference 34RBArba.gov.auVisit source
- Reference 35BCBbcb.gov.brVisit source
- Reference 36RBIrbi.org.inVisit source
- Reference 37RESBANKresbank.co.zaVisit source
- Reference 38CBRcbr.ruVisit source
- Reference 39BANXICObanxico.org.mxVisit source
- Reference 40PROJECTSprojects.fivethirtyeight.comVisit source
- Reference 41PREDICTITpredictit.orgVisit source
- Reference 42ECONOMISTeconomist.comVisit source
- Reference 43FIVETHIRTYEIGHTfivethirtyeight.comVisit source
- Reference 44BRITAINELECTSbritainelects.comVisit source
- Reference 45IPSOSipsos.comVisit source
- Reference 46WAHLFORSCHUNGwahlforschung.deVisit source
- Reference 47YOUGOVyougov.com.auVisit source
- Reference 48DATAFOLHAdatafolha.folha.uol.com.brVisit source
- Reference 49LOKNITIlokniti.orgVisit source
- Reference 50338CANADA338canada.comVisit source
- Reference 51SWGswg.itVisit source
- Reference 52ELECTOMANIAelectomania.esVisit source
- Reference 53NOVUSnovus.seVisit source
- Reference 54PEILINGWIJZERpeilingwijzer.tomlammers.nlVisit source
- Reference 55CBOScbos.plVisit source
- Reference 56PANELSPOLITICSpanelspolitics.co.ilVisit source
- Reference 57KONDAkonda.com.trVisit source
- Reference 58GSPUBLISHINGgspublishing.comVisit source
- Reference 59JPMORGANjpmorgan.comVisit source
- Reference 60BLOOMBERGbloomberg.comVisit source
- Reference 61MORGANSTANLEYmorganstanley.comVisit source
- Reference 62CITIGROUPcitigroup.comVisit source
- Reference 63IBib.barclaysVisit source
- Reference 64GBMgbm.hsbc.comVisit source
- Reference 65RESEARCHresearch.db.comVisit source
- Reference 66UBSubs.comVisit source
- Reference 67BLACKROCKblackrock.comVisit source
- Reference 68FIDELITYfidelity.comVisit source
- Reference 69ADVISORSadvisors.vanguard.comVisit source
- Reference 70PIMCOpimco.comVisit source
- Reference 71SCHRODERSschroders.comVisit source
- Reference 72INVESCOinvesco.comVisit source
- Reference 73STATESTREETstatestreet.comVisit source
- Reference 74GOODJUDGMENTgoodjudgment.comVisit source
- Reference 75GJOPENgjopen.comVisit source
- Reference 76METACULUSmetaculus.comVisit source
- Reference 77MANIFOLDmanifold.marketsVisit source
- Reference 78FORETELLforetell.scienceVisit source
- Reference 79HBRhbr.orgVisit source
- Reference 80AMAZONamazon.comVisit source
- Reference 81PREDICTINGRESEARCHERpredictingresearcher.beehiiv.comVisit source
- Reference 82KALSHIkalshi.comVisit source
- Reference 83DARPAdarpa.milVisit source
- Reference 84STATEstate.govVisit source
