Tabular foundation models got acquired before they got benchmarked

Three things happened to tabular machine learning between May and July 2026, and most of the roundups covered one of them.

On May 4, SAP announced it was acquiring Prior Labs, the lab behind TabPFN. That deal closed on July 17, with SAP committing more than a billion euros over four years to scale it. In June, NVIDIA bought Kumo AI for around $400 million, and Jure Leskovec went with it. On June 30, Google Research published TabFM under Apache-2.0, with weights on Hugging Face and a BigQuery integration promised.¹1TabFM's architecture is worth a look on its own: alternating row and column attention, a row-compression step, then a dedicated in-context learning layer over the compressed embeddings. Google trained it on hundreds of millions of synthetic datasets generated from structural causal models, and evaluated on TabArena's 38 classification and 13 regression tasks.

Read the buyer list rather than the model list. SAP sells the system the tables live inside. NVIDIA sells the silicon underneath. Google sells the warehouse. Three companies that sit closest to enterprise structured data all took the same position in the same quarter.

SAP's CTO Philipp Herzig put the thesis better than any analyst has:

He is right. The capability works. The independent proof that it works the way the marketing says is thinner than a billion-euro commitment implies, and I spent last week watching my own agents fail to check it.

What these models actually do

A tabular foundation model treats prediction as in-context learning. Your labelled rows go in as the prompt, the model does one forward pass, and unseen rows come out scored. There is no per-dataset training step and nothing to tune.

The reason you cannot do this with a language model is structural rather than a matter of scale. Tables are two-dimensional and permutation-invariant, so swapping two rows changes nothing about the data and everything about what a left-to-right model sees. Tokenizers shred continuous values, turning 100.50 into fragments that destroy the column distribution a predictor needs. And flattening a real enterprise table into a text string exhausts the context window long before the model has enough rows to infer anything statistical.

So these models are trained differently. Every one of the leading systems pretrains on synthetic data, not on a scraped corpus of real spreadsheets.side quest That choice makes them a bet on inductive bias rather than a bet on data scale, which is why they behave nothing like an LLM and why they hit their ceiling somewhere else entirely.

Four things the roundups leave out

The strongest model is not licensed for your use

The current TabPFN line ships under Prior Labs' own non-commercial licenses. TabPFN-2.6 carries tabpfn-2.6-license-v1.0, TabPFN-3 and TabPFN-3-Plus carry their own. Research is free, production costs money, which is a normal enough arrangement. The interesting part is where the boundary sits.

The model card permits testing, evaluation and internal benchmarking. It then bars use for "any commercial or production purpose," and spells the exclusions out: "revenue-generating products, competitive benchmarking for procurement, client deliverables, or using the model's results for internal commercial decision-making."²2Prior Labs' own FAQ goes further, putting testing that informs "business decisions, evaluation of vendors, procurement decisions, or commercial workflows" on the commercial-license side of the line.

You may run the model against your own data. You may not use what you learn to decide whether to buy it. The evaluation that would settle it is the evaluation the license excludes.

The benchmark that ranks it first is co-authored by its CEO

TabArena is the benchmark everyone cites for this category, and it is good work: curated, versioned, designed so results compose across papers instead of each team bringing its own datasets.

Frank Hutter is a co-author on TabArena. He is also the co-founder and CEO of Prior Labs, whose TabPFN sits at the top of it.

That is a structural observation and not an accusation. The mindfulmodeler review of the field raised the same overlap and concluded they saw no evidence of bias, and I have not seen any either. Benchmarks in a young field are usually built by the people who care most, which is normally the people who also build models. That is how the field gets a shared yardstick at all.

The problem is what the arrangement lets you conclude. A benchmark co-authored by the lab whose model it ranks first cannot be the only instrument you trust, for the same reason a vendor eval shipped alongside a vendor model cannot be. It does not mean the number is wrong. It means the number has not yet been checked by anyone with an interest in it being lower.

The benchmark's own abstract is more careful than the marketing

Here is TabArena's conclusion, verbatim from the June 2025 paper: "While gradient-boosted trees are still strong contenders on practical tabular datasets, we observe that deep learning methods have caught up under larger time budgets with ensembling. At the same time, foundation models excel on smaller datasets."

The last clause carries the finding. Small datasets, specifically, which is not the same sentence as "tabular foundation models beat XGBoost."

The TabPFN-2.5 paper made the point against itself without meaning to. Its headline TabArena figure plotted the model against AutoGluon 1.4 in extreme mode, tuned for four hours. A four-hour classic ensemble was the line being cleared, not a line left behind years ago.

Then TabPFN-3 landed on May 13, 2026, and moved all of it. The technical report claims first place on TabArena over every other model including tuned and ensembled baselines, scaling to a million training rows, a win over AutoGluon 1.5 in extreme mode at ten times the speed, and beating gradient-boosted-tree baselines that had been tuned for eight hours. TabPFN-3-Plus claims a 200-Elo margin over everything non-TabPFN, widening to 420 on the largest subsets.³3TabPFN-3 Technical Report, arXiv 2605.13986, submitted 2026-05-13 and revised 2026-05-28. It also reports up to 20x faster inference than TabPFN-2.5 and SHAP values up to 120x faster.

If those numbers hold, the small-data framing is already out of date and the trees are in more trouble than the 2025 abstract suggested. So look at the document they arrive in. A technical report from the lab that sells the model, self-reported, scored on the benchmark that lab co-authors, eleven months after the last independent read on the subject.

Accuracy is not calibration, and enterprise work runs on calibration

None of the roundups mentioned this one.

De Melo Costa, Popineau, Rimmel and Doan ran TFMs, GBDTs and classical baselines across the 112 datasets of the TALENT benchmark, looking at uncertainty rather than accuracy.⁴4"High Performance, Low Reliability: Uncertainty Benchmarking for Tabular Foundation Models" (arXiv 2605.28554). The title is the abstract. Their result is a performance-uncertainty tradeoff: TFMs took the highest AUC, and showed lower conditional coverage under conformal prediction than the gradient-boosted trees they beat.

Their conclusion, verbatim: while TFMs advance predictive frontiers, achieving well-calibrated uncertainty remains a major open challenge for their reliable adoption.

For a Kaggle leaderboard, AUC is the score. For fraud scoring, credit decisions, clinical risk and anything that feeds a threshold or a dollar amount, the calibrated probability is the deliverable.side quest A model that ranks correctly and reports its confidence badly has not done the job, it has done the easier half of the job.

So we tried to reproduce one

Hugging Face and alphaXiv are running an agent reproduction challenge through early August: AI coding agents attempt to reproduce the claims of ICML 2026 papers, publish a logbook, and an automated judge scores each claim as verified, falsified, toy or inconclusive. I entered on July 23 and pointed the fleet at TabICLv2, which nobody had claimed.

What follows is not a finding about TabICLv2. Our logbook for it is not published, and I am not going to pre-announce a verdict on a paper we are still working. It is a finding about reproduction, and it cost us a day.

The agent reproduced the wrong claim, and every surface signal said it had succeeded.

It had written its own phrasing of the paper's first claim, as "matches or beats strong tabular baselines on representative benchmarks." Reasonable-sounding. Then it benchmarked TabICLv2 against sklearn trees, HistGBDT and ExtraTrees, on three randomly chosen OpenML datasets, one split each. The run was green and the logbook looked like work.

The paper's actual claim, as the contest records it, names its terms exactly: without any hyperparameter tuning, TabICLv2 surpasses tuned-and-ensembled RealTabPFN-2.5 on TabArena and dominates the accuracy-versus-runtime Pareto front.⁵5From the contest's own claims_anchored.json, which is the file the judge scores against. Worth knowing if you ever enter one of these: the board tracks a fixed per-paper claim list, and a logbook is graded against those strings, not against whatever the participant decided the paper said.

Three substitutions, each individually defensible, and together they replaced a hard claim with an easy one. Beating stock sklearn trees is not in dispute and has not been for a while. The claim that carries information is the comparison against the current best tabular foundation model, on a curated benchmark, under a protocol that controls variance.

This is not an agent problem:

Every enterprise POC I have watched does some version of this. The vendor claims a lift against a strong baseline. The bake-off runs against whatever baseline the team already had lying around, on the two datasets that were easy to export, with one split because the calendar was tight. The result is a real number attached to a question nobody asked.

What I would actually do

The dual-stack advice in circulation is right and I will not spend long on it: prototype with a foundation model, and once the schema stops moving and the volume goes up, pay the one-time tuning cost and ship a gradient-boosted tree behind a low-latency endpoint. That is sound and it is not new.

The win is concentrated somewhere narrower than the pitch suggests. These models are strongest precisely where classical ML is most annoying: small data, cold start, and schemas that change under you. That is a substantial capability and it is much narrower than "enterprise AI." If your problem is fifty thousand rows that arrive in a new shape every quarter, this is the best news in years. If it is a stable schema and a hundred million rows scored in a millisecond, the trees are not going anywhere. Treat that boundary as provisional, though. TabPFN-3 is claiming a million training rows, and if that survives contact with someone independent, the line moves.

Then two habits, both cheap.

Before you believe a number, name the baseline and the protocol behind it. Most published lifts in this category are against something specific, and the specificity is the information. When a vendor will not tell you, or the license will not let you check, treat that as an answer.

And pin the claim before you run the experiment, in writing, as a sentence with named terms in it. We now write the exact claim text at the top of every reproduction and diff the run against it at the end. That step takes about ten minutes and it is the only reason we caught this one.

Where this leaves the category

The buyers are not speculating. SAP put a billion euros and its structured-data franchise behind this, NVIDIA paid $400 million and took the founding team, and Google gave the architecture away for free with a warehouse integration behind it. All three read the same thing, which is that the enterprise prediction layer is up for grabs for the first time since gradient boosting won it.

The models earn that attention. What has not arrived yet is an independent evidence base to match: a benchmark nobody involved has a stake in, calibration numbers reported next to the accuracy numbers, and licenses that let a buyer run the comparison that would settle it. Until then the reasonable posture is to use these where they are strong, which is small and shifting data, and to check every number yourself against the baseline the claim actually named.

Related reading on this site: reproducing warp decode on BlackwellshippedAI Systems & Architecture·Apr 9, 2026reproducing-warp-decode-blackwellFirst independent reproduction of Cursor's Warp Decode on Blackwell. 38% faster Gemma 4, crossover at batch 24 vs vLLM, fusion failure analysis. for what a real reproduction costs once you stop trusting the headline figure, 151 experiments on a GB10shippedPractical Applications·Mar 14, 2026AutoResearch on Blackwell GB10: 151 Experiments OvernightRunning Karpathy's AutoResearch overnight on a Blackwell GB10 revealed that hardware FLOPS, not VRAM, determine optimal model architecture, with the agent discovering a 22.5% improvement using only 6.1 GB of 128 GB available for the hardware this reproduction work runs on, the AI research night shiftshippedPractical Applications·Nov 6, 2025My AI Research Assistant Works the Night Shift (A Claude Code Skill Story)How I built a Claude Code skill that researches AI developments overnight using intelligent automation that adapts, prevents duplicates, and provides instant answers. for the overnight loop the agents run inside, and an autonomous agent squad for ten dollars a monthshippedAI Development & Agents·Feb 6, 2026I Built an Autonomous AI Agent Squad for $10/MonthHow I deployed an autonomous AI agent squad — a CEO that delegates to a research specialist — using OpenClaw, Tailscale, and GLM-4.7 on a $10/month budget. The architecture, the failures, and the lessons. for how the fleet that got this wrong is put together.