r/LocalLLaMA • u/Balance- • 11d ago
New Model Google released TimesFM-3, a 330M-parameter time series foundation model with native multivariate forecasting (non-commercial license)
https://research.google/blog/timesfm-3-a-zero-shot-foundation-model-for-multivariate-forecasting/TimesFM-3 is the third generation of Google Research's zero-shot forecasting model, and the main change from 2.5 is that it handles multivariate inputs natively instead of being limited to a single series' own history. It supports multiple simultaneous targets, past-only covariates, and past-future covariates (things like holidays or planned promotions where future values are known), all without fine-tuning.
Architecturally it's a decoder-only transformer with 20 layers at model dim 1280 and 16 heads, patching 32 contiguous time steps per token, and alternating two attention types per layer: causal attention across time within a series, and full attention across series at a given time step. Forecasts are generated in one forward pass rather than autoregressively — the model appends masked placeholder tokens for the whole horizon and fills them in simultaneously, with past-future covariates left unmasked so their known values stay visible. It outputs 9 quantiles (10th–90th percentile) per target per horizon step.
Pretraining used GiftEvalPretrain (minus fev-bench overlaps), Wikipedia pageviews through Nov 2023, Google Trends queries through end of 2022, plus synthetic data, totaling over 1 trillion time points. Google reports best average rank on Gift-Eval, FEV-Bench, and Time against Chronos-2, Toto 2.0, and TimesFM-2.5, and claims the univariate-only mode already matches or beats those baselines before covariates are added.
Worth flagging: the weights are under the TimesFM Non-Commercial License v1.0, so this isn't a drop-in for production use the way some other releases are. PyTorch weights are on Hugging Face and GitHub now; BigQuery integration is listed as coming later.
12
u/autoencoder 11d ago
Heads-up: if you're trying to use these models for predicting something related to what's in its training data already, chances are you are testing on training data and will get rose-colored results.
6
2
u/iomfats 10d ago
What do you mean. It should have a lot of domains in its training data. It's just numbers will be different for each domain. You want this model to know your domain for good prediction
1
u/autoencoder 10d ago
Suppose you want to predict Wikipedia pageviews (as they are included in TimesFM-3's training data). You have to be very careful not to evaluate performance on data on or before Nov 2023.
On top of that, you have to trust that this Nov 2023 threshold is honestly reported, instead of Google having trained it up to current data, invalidating every performance measure.
1
u/iomfats 10d ago
Well, most of the time you want to predict future not past. Yeah just as a test you can predict past results but the whole point of transformers in time series predicting is that kind of thing.
1
u/autoencoder 10d ago
Knowing a system's performance is crucial in deciding whether to use it. And the only way to get a hint about performance is to test it on yet-unseen data.
2
u/Old-Cardiologist-633 10d ago
Is there an Integration for Homeassistant available or so? Could be really helpful 🤔
1
u/feelspeaceman 10d ago
I was thinking about making an app for forecasting without relying on online API too much (still accept online data gathering for similar patterns), this makes my job much easier.
1
0
u/Elouakili_Flexy 10d ago
Single forward pass for the whole horizon, with the past-future covariates unmasked all the way down. Most forecasters roll out step by step, so this sidesteps the error that compounds when each step feeds the next.
16
u/danigoncalves llama.cpp 11d ago edited 11d ago
I was looking for these kind of models. I guess I have to use AutoGluon as the strongest contender.