What is grounding in LLMs?

Grounding is the practice of making a language model answer only from data supplied to it at query time, rather than from what it absorbed during training. In a grounded system the model calls tools that fetch real records, those records travel into the prompt, and the model's job is to read and explain them. Anything the model cannot see in that retrieved context, it should decline to assert.

How do you prevent AI hallucination in analytics?

Remove the need for the model to recall anything. Have it answer through tools that query your live database, put the returned figures in the prompt, and restrict its role to narration and interpretation. Then verify: compute ground truth directly from the database, check every figure the model asserts against it, and add trap questions about entities and periods that do not exist to see whether the model admits the gap.

Can an AI assistant with weak factual knowledge still be good at analytics?

Yes, and it is often the wrong thing to optimise for. The model SealMetrics ships, gpt-oss-120b, scores 0.168 accuracy on SimpleQA with a 0.782 hallucination rate — weak open-world factual recall. That is irrelevant to grounded analytics, because the assistant is never asked what it remembers about the world. It is asked to read your entrances and conversions, which arrive in the prompt from your own database.

What is the difference between grounding and RAG?

Retrieval-augmented generation is one way to achieve grounding, usually over documents. An analytics assistant grounds through structured tool calls instead: it invokes typed functions against the analytics database and receives numbers, not passages. The principle is the same — the answer is built from what was fetched — but the retrieved evidence is a query result, so it can be compared against ground truth exactly rather than approximately.

Grounding: Why a Good Analytics AI Shouldn't 'Know' Anything

Q: How can I tell if an AI analytics assistant is making numbers up?

Ask it why. A grounded assistant can name the period, the metric and the breakdown behind every figure, because it just retrieved them. Then test it on something that does not exist — a campaign you never ran, a month before you installed tracking. A grounded system says the data is not there. An ungrounded one produces a plausible number, which is exactly the failure mode you cannot spot by reading the answer alone.

July 24, 20266 min readBy Rafa Jiménez

A general chatbot answers from what it remembers. An analytics assistant must answer only from what it just fetched. Build it that way and the model's memory stops mattering — but you still have to prove it, one asserted number at a time.

Key Takeaways

Grounding is an architecture, not a prompt. The tools fetch real numbers, the numbers travel in the prompt, and the model narrates and interprets — it never recalls.
Weak open-world recall becomes irrelevant. The model we ship scores 0.168 on SimpleQA with a 0.782 hallucination rate, and that is fine: it is never asked what it knows, only what your data says.
Grounding must be verified, not trusted — ground truth computed from the database, every asserted figure checked against it.
Traps are the real test: ask about a campaign that never ran. A grounded assistant says so. An ungrounded one invents something plausible.

There is a product principle behind AI analytics that sounds wrong the first time you hear it: the model should not know anything.

Not "should not know much". Should not rely on knowing. Every fact it states about your business should have arrived in the last few hundred milliseconds, from your database, through a tool call — not from a weight matrix trained months ago on the public internet.

That constraint has a name. It is called grounding, and it is the difference between an assistant you can put in front of a marketing team and a very confident text generator.

Knowing versus looking up

A general-purpose chatbot answers from parameters. You ask it the capital of Peru and it produces "Lima" because that association is encoded in its weights. There is no lookup, no source, no citation — just a very good statistical guess. Most of the time it is right. When it is wrong, it is wrong in exactly the same tone of voice.

An analytics assistant cannot work that way, for an obvious reason: your entrances last Tuesday were never in anyone's training data. There is nothing to recall. If the model produces a number, it either fetched it or fabricated it. There is no third option.

So the design goal flips. Instead of making the model smarter about the world, you make it structurally incapable of answering without first going to get the data.

What grounding actually looks like in the stack

Grounding is not a line in the system prompt saying "do not make things up". Models agree to that instruction and then make things up anyway. It is a shape you give the whole request path:

The question arrives in natural language. "Which channel drove the most conversions last month?"
The model selects tools, not answers. Seal AI's assistant has a 63-tool inventory — overview, channels, campaigns, funnels, segments, landing pages, and so on. Its first job is to decide which of them to call and with what parameters.
The tools query your data. Real rows, real aggregates, scoped to your account and your timezone.
The results travel back into the prompt. The numbers are now literally in front of the model as text.
The model narrates and interprets. It reads what came back, arranges it, notices the trend, suggests what to look at next. It is doing language work over evidence, not retrieval from memory.

Step five is the only step where the model's intelligence is spent on the actual question. Everything before it is plumbing — and the plumbing is what makes the answer true.

The payoff: a "bad" model can be the right model

Here is the honest version of our own case. The model inside Seal AI is gpt-oss-120b. On SimpleQA — a benchmark of short open-world factual questions — it scores 0.168 accuracy with a 0.782 hallucination rate. Read cold, that looks disqualifying. A model that confidently invents answers to four out of five general-knowledge questions is not something you would want narrating a revenue report.

Except it is never asked a general-knowledge question. Nobody opens an analytics tool to find out who won a 1994 election. The assistant is asked to read entrances, conversions, channels and bounce rates that arrived in its context window a moment ago, and to explain what they mean. The capability being measured by SimpleQA is one the product deliberately does not use.

What the product does use, the same model is strong at: MMLU 90.0, MMLU-Pro 80.8, GPQA Diamond 80.1 without tools, and — most relevant here — reliable function calling against a large tool inventory. Grounding is what lets you spend your model budget on the capabilities that matter and ignore the ones that don't.

Grounding you trust is grounding you tested

An architecture that should prevent invention is not the same as one that does. Models can call a tool, get a partial answer, and then quietly fill the gap with something reasonable. The only way to know is to check every number.

That is how we ran our own internal benchmark: 18 scenarios in two languages, three passes, three candidate models — 162 live queries against the real production assistant endpoint with the real 63-tool inventory. Before each run the harness computed ground truth directly from the analytics database — entrances, conversions, bounce rate, revenue, top channel, top source, top device — so every figure the assistant asserted could be compared to reality by a deterministic grader, not by a human reading along and nodding.

The shipped model asserted 144 of 144 verified facts correctly. Every single fact miss in the entire run traced back to one badly phrased question of ours, which we will come back to.

Traps: asking about things that do not exist

Checking correct answers only tells you half the story. The interesting failure is not a wrong number — it is a number where there should have been an admission.

So each language block included grounding traps: questions about a campaign that was never run, or a period for which the account holds no data. There is exactly one right behaviour — say the data is not there, and say why. Anything fluent and numeric is a failure, no matter how well written.

The results were more interesting than a pass/fail table. The shipped model passed 18 of 18 traps (95% Wilson confidence interval 0.82-1.00). One alternative, mistral-small-3.2, passed 9 of 18 — and its failure mode was subtle. Asked about a nonexistent campaign, it correctly acknowledged the absence, then drifted into describing other real campaigns with figures nobody asked for. That is not invention; the numbers were true. It is answering a different question, which in a report is its own kind of wrong. In English it also claimed a false incapability — "I don't have access to the necessary tools" — instead of explaining the data's time boundary.

The Wilson intervals matter here: the winner's interval no longer overlaps mistral's, so at this sample size that gap is statistically meaningful rather than a lucky run.

The question we got wrong

One scenario asked for "traffic by device for the last month". That admits two readings: the last 30 days, which was our ground truth, or the previous calendar month. Some models chose the calendar month, where the test account held only five days of data, and answered with those real numbers. The grader marked them as misses.

They were not hallucinations. They were correct answers to a question we had phrased badly. One hundred per cent of the fact misses in the whole run trace to that single scenario. We reworded it, and we keep the defect documented, because a benchmark that only reports its flattering results is a demo. The same applies to our first full run, which we discarded entirely: the harness revealed that the assistant was reusing the chat session between queries, letting later models read earlier models' history. We fixed it, re-ran, and archived both.

What this means when you use the assistant

Practically: ask it why. A grounded assistant can tell you which metric, which period and which breakdown produced a figure, because it fetched them seconds ago. If an assistant cannot show you its evidence, the number is a claim, not a measurement.

And test it once with something that does not exist — a campaign you never ran, a month before you installed the tracker. The answer you want is the boring one: there is no data for that. An assistant willing to say "I don't have that" is the only kind worth believing when it says it does.

More on how the model was chosen in the write-up of the 162-query benchmark, and on the architecture behind it in the Seal AI architecture documentation.