Why LLMs Struggle With Tables

How two new foundation models, Google’s TabFM and NVIDIA’s KumoRFM, teach transformers to read structure instead of prose. Every section below has something to poke at, the demos are simulated with canned data but the mechanics they act out are faithful to the papers.

The token soup problem

Paste a table into a chat model and something quietly goes wrong. The model never sees a grid. It sees a stream of tokens, read left to right, exactly the way it reads a sentence. Which column a number sits in, how it relates to the value above it, how two tables join on a key: all of that structure gets flattened away.

Here is a coffee shop’s sales sheet. Flip between how you see it and how a language model sees it, then ask both the same question.

Demo · two ways to read one table

Drink	Size	Price	Sold

Q: How much does the Iced Latte cost?

read the stream, then flip back to the grid and ask again.

A table is not a paragraph. Reading it like one throws away the thing that makes it useful.

That is why the standard advice for enterprise AI has been: LLMs for documents, XGBoost or LightGBM for tables. It works, but it means a trained model, a feature pipeline and a tuning sprint for every new prediction task.

Cells become vectors

TabFM and KumoRFM keep the transformer but change what it sees. No cell is ever spelled out as characters. Each one is converted straight into an embedding: numbers go through Fourier feature encodings so the model understands magnitude and closeness, categories get their own learned vectors.

Demo · tap a cell, see its vector

cell → embedding

Pick a cell above.

tap 3.90, then 4.50. Nearby numbers get similar wave shapes. Categories get distinct blocky ones.

Once every cell is a vector, the model can stop reading left to right and start attending across the grid itself. That is where the two models diverge: TabFM does it for a single table, KumoRFM does it for a whole database.

Inside TabFM

TabFM is Google Research’s tabular foundation model, released June 2026 for zero-shot classification and regression. Give it labelled rows as context plus the rows you want predicted, and it answers in a single forward pass. Internally it works in three moves. Step through them on a table predicting what a customer will order.

Demo · three moves, one prediction

1 · column attention 2 · compress rows 3 · predict in context

Age	Time	Weather	Drink
34	Morning	Cold	Latte
19	Afternoon	Hot	Iced Latte
41	Morning	Cold	Flat White
27	Evening	Mild	Mocha
23	Afternoon	Hot	?
38	Morning	Cold	?

in-context transformer reads the green dots, fills in the amber ones

Four labelled context rows, two rows to predict. Press Start.

Two details worth knowing. First, TabFM was pretrained entirely on hundreds of millions of synthetic tables generated from structural causal models, because real enterprise tables are proprietary and sensitive. Second, the released model has real limits: 10 classes for classification, roughly 500 features, and weights under a non-commercial licence.

No training run

The point of all that machinery is the workflow it replaces. The classic route to a tabular prediction is a modelling sprint. The foundation model route is a prompt. Race them.

Demo · the old way vs the prompt way

Train a custom model

idle

TabFM, in context

idle

To be fair to the old way: a well-tuned gradient boosted model on a big, stable, high-stakes dataset is still hard to beat. The foundation model wins on the long tail, the hundreds of prediction tasks that never get a data scientist because setup costs more than the answer is worth.

Inside KumoRFM

TabFM’s limit is the one every real business hits on day one: the data is never one table. The coffee shop has a Customers table, an Orders table and a Payments table, stitched together by IDs. The traditional fix is to flatten them into one giant feature table before any model can look at them. KumoRFM, built by Kumo AI and acquired by NVIDIA in June 2026, skips the flattening: it treats the whole database as a graph, where every record is a node and every foreign key is an edge.

Demo · one database, three readings

customers

id	name
C1	Sara
C2	Marcus
C3	Priya

orders

id	cust	item
O1	C1	Flat White
O2	C1	Latte
O3	C2	Mocha
O4	C3	Espresso
O5	C2	Iced Latte

payments

id	order	amt
P1	O1	3.60
P2	O2	3.90
P3	O3	4.20
P4	O4	2.80

Twelve tidy records, linked by keys. This is how the business actually stores it.

cust	name	order	item	payment	amt
C1	Sara	O1	Flat White	P1	3.60
C1	Sara	O2	Latte	P2	3.90
C2	Marcus	O3	Mocha	P3	4.20
C3	Priya	O4	Espresso	P4	2.80
C2	Marcus	O5	Iced Latte	NULL	NULL

The old way: join everything into one wide table. Customer details duplicate, unpaid orders punch NULL holes, and this is 12 records. A real warehouse turns this step into days of feature pipelines.

Tap a customer node. KumoRFM gathers that entity's neighbourhood along the foreign keys, hop by hop, as context for a prediction.

flatten it first, then switch to the graph and tap Marcus. Note the unpaid order that broke the flat table is just… a node.

A lighter network handles within-table structure (the same row and column attention idea from TabFM), while a broader one passes information across tables along the edges. In KumoRFM-2 this became a full Relational Graph Transformer. On the RelBench benchmark, the zero-shot model beats both hand-engineered features and supervised deep learning by a few percent on average, and fine-tuning buys another 10 to 30.

No peeking at the future

Enterprise data moves. Every record in KumoRFM’s graph carries a timestamp, so the model can take a snapshot of the database as of any past moment and answer only from what existed then. This kills data leakage, the classic silent failure where a model accidentally trains on information from the future it is trying to predict.

Demo · drag the cutoff

JanFebMarAprMayJun

PREDICT: will Sara order in the 30 days after the cutoff? Everything left of the line is context. The truth lives on the right, and that is the label the model is scored against, never something it gets to read.

Future data is the target, never the context.

Predictive queries

You do not hand KumoRFM a feature pipeline. You hand it a predictive query, written in PQL, a small SQL-flavoured language where the SELECT is about the future. The model parses it, gathers the right temporal subgraph as context, and answers in seconds. Pick one.

Demo · three questions about the future

-- pick a query above

Everything here is simulated with canned numbers. The real system does exactly this shape of work: parse, snapshot, gather context, one forward pass.

Where they fit

First, the one-screen summary:

	Google TabFM	NVIDIA KumoRFM
scope	One table at a time	Whole relational databases
data as	Embedded grid, row and column attention	Temporal graph over foreign keys
you provide	A DataFrame with labelled context rows	A database connection and a predictive query
interface	scikit-learn style API	PQL or natural language
answers	Zero-shot classification and regression	Churn, LTV, fraud, recommendations, forecasts

Neither of these is a plugin that helps a chat model read tables mid-prompt. They are standalone prediction engines, and they slot in upstream of your LLM stack. The strongest pattern pairs them: the tabular model computes the number, the language model explains it. Step through the loop.

Demo · one briefing, two kinds of model

1 · score 2 · store 3 · ask 4 · retrieve docs 5 · retrieve score 6 · write

A nightly job is about to run. Press Next step.

The tabular model does the numeric prediction it was built for. The LLM does the reading, reasoning and writing it was built for. Neither pretends to be the other.

The next unlock in enterprise AI may not be a better chatbot. It may be a model that reads the database the way the database was meant to be read.