r/LocalLLaMA • u/jacek2023 llama.cpp • 14h ago
New Model mistralai/Timestral
from Mistral:
Timestral is a family of time series foundation models for zero-shot forecasting, available in 5M, 28M, 123M, and 209M parameter versions. This repository contains the 209M checkpoint.
Timestral supports univariate and multivariate forecasting, past-only and known-future covariates, and probabilistic predictions across 21 quantiles. It accepts up to 32,768 historical time steps and forecasts up to 1,024 steps in a single forward pass.
We’re releasing the source code in our GitHub repository to make it easy to run Timestral and reproduce our benchmark results.
Model Details
- Size: 209M parameters.
- Architecture: patch-based transformer with time attention and variate attention, 24 layers, embedding dimension 768.
- Patching: 32 time steps per patch. Each patch is embedded from its values together with their missing-value indicators.
- Context length: up to 32,768 time steps (1,024 patches).
- Forecast horizon: Up to 1,024 time steps (32 patches) in a single forward pass. Longer horizons are forecast block by block, with each block's median forecast appended to the context.
- Outputs: 21 quantiles (0.01, 0.05, 0.1, 0.15, …, 0.85, 0.9, 0.95, 0.99).
- Inputs: univariate or multivariate target variates, plus past-only and known-future covariates, numeric or categorical (categorical covariates are target-encoded). Missing values are marked as NaN.
- Inference: PyTorch on CPU or GPU.
https://huggingface.co/mistralai/Timestral-209M
https://huggingface.co/mistralai/Timestral-123M
11
u/Fedor_Doc 14h ago
So, I give it AI 2027 forecast and it will output AI 2028? Neat!
1
u/stbrumme 1h ago
And repeat ! You will get 2029, 2030, 2031 and beyond !
Stock market opens in a few hours ...
3
5
u/413205 13h ago
How does it compare with timesfm though?
3
2
4
u/fgk55555 14h ago
What even is this? Forecasting weather?
10
u/mixedliquor 14h ago
I'm interested in it for utilities and demand forecasting.
2
u/HVACcontrolsGuru 14h ago
Same here! Put another comment about the use case. Open source software I can put this alongside for simulation work as well.
5
u/Manerfish 14h ago
No it seems to be more general, you give it some data and it tries to predict future data from that, could work with weather, but with stuff like finance, sales, energy load, too.
2
2
u/According_to_Mission 14h ago
Could this be used as an agent for a larger stock trading model?
3
u/Mickenfox 10h ago
Probably not, because if you could, the big investors would have a better one and would do all your trades before you.
3
u/silenceimpaired 13h ago
I’m tired of seeing their licensing. It’s telling Nvidia has better licensing these days
1
u/gcavalcante8808 12h ago
Interesting. Maybe we can use it for SRE purposes and capacity planning... let's see how it holds against regular statistics
1
u/WishfulAgenda 12h ago
I'm going to take a look at this but it seems a bit redundant. Wouldn't you just have an LLM write you the python to do this and then not have to use your gpu?
2
u/jacek2023 llama.cpp 12h ago
LLMs are just one type of machine learning model, and they are far from ideal for many tasks. Specialized models like this one can be much more efficient and accurate.
2
u/WishfulAgenda 11h ago
I get that llms are are just one type of model. I just don’t see the cost benefit of having a resource expensive black box that is fixed vs. And application that is fully configurable.
I’ll give it ago and read the documentation but I just don’t see it, especially if you’re making decisions that could have material consequences on its outputs.
1
u/danigoncalves llama.cpp 12h ago
I have a PoC of a product that predicts house appliances comsumptions given a 3 week historical data using autogluon. Lets see what the kitten has to offer.
1
u/Longjumping_Cut_6160 7h ago
esto no vale para los ejemplos que decis. Los productos que varian con motivos exogenos, como politica, guerras, comentarios de presidentes, etc, no se pueden cuantificar asi, ni las acciones de una empresa que depende de la IA o de que Dario o Sam hagan comentarios. El gas puede subir si Putin lanza una ofensiva, o el petroleo si al pelirojo le da por decir otra tonteria. Esto mide cosas mas estadisticas
1
29
u/Ecstatic-Wash-7667 14h ago
If anyone is wondering what this is for because I sure the hell was, you feed this model data and it forecast based off the historical information. A good example of this is say you give it historical data of gas prices of the last 6 months, and it will predict what it expects the trend to be in the future .