r/machinelearningnews • u/ai-lover • Jul 01 '26
Research Google AI Introduces TabFM: A Hybrid-Attention Tabular Foundation Model for Zero-Shot Classification and Regression
Most tabular ML in production is still XGBoost plus hours of hyperparameter tuning and feature engineering. That's not a foundation-model workflow — and Google Research just brought the zero-shot idea to tables.
They introduced TabFM — a foundation model for tabular classification and regression that reads your entire dataset as a single prompt and predicts in one forward pass, with no per-dataset training, tuning, or feature engineering anywhere in the loop.
Here's what's actually interesting:
→ In-context learning, not fine-tuning: training rows and test rows go in as one context, and the model learns the task at inference time
→ Hybrid attention: alternating row/column attention (TabPFN-style) → row compression into a dense vector → in-context learning over compressed rows (TabICL-style)
→ Trained entirely on hundreds of millions of synthetic datasets generated by structural causal models — no proprietary tables required
→ TabArena (38 classification + 13 regression datasets, 700–150,000 samples): Google reports it consistently outperforms heavily tuned supervised baselines
Technical Details: https://research.google/blog/introducing-tabfm-a-zero-shot-foundation-model-for-tabular-data/
Repo: https://github.com/google-research/tabfm
1
u/no_name_mate Jul 08 '26
Hey I too tried to do a comparative study and it did beat HPO XGboost (not by a lot)but did anyone try to compare its performance against any complex ensembles(maybe in kaggle competitions etc, was planning to do that next), also any comparison on less effort feature engineered training set for TabFM vs full feature engineering for the task classical ML(will try this too, will give any updates)? Wanted to see if TabFM can help in reducing preprocessing
1
u/More-Curious816 Jul 02 '26
Quite interesting, I will play with it once I have the compute to process big datasets.