Hallucination & Grounding: Why LLMs Make Things Up

Ask a large language model a question it doesn't actually have the answer to, and more often than not it won't tell you that — it will tell you something, fluently and confidently, and that something will often be wrong. This is "hallucination," and it's one of the few LLM failure modes that's genuinely dangerous, because the output reads exactly as convincingly as a correct answer would. Understanding why it happens, and what "grounding" actually does to reduce it, matters more than memorizing the term.

What hallucination actually means

A hallucination is output that is fluent and confident-sounding but is factually wrong, unsupported by any real source, or inconsistent with the context the model was actually given. The key word is unsupported — the problem isn't that the model made a mistake in reasoning over real evidence, it's that there was no real evidence backing the claim at all, and the model produced one anyway, with the same fluency and certainty it would use for a claim it could actually back up.

Why it happens: there's no "I don't know" signal

A language model is trained to predict the next plausible token given everything that came before it, over an enormous corpus of text. That objective rewards fluency and statistical plausibility — it never explicitly rewards the model for distinguishing "I have a verified fact for this" from "this is a continuation that sounds like something that would follow." Both cases route through the same mechanism: predict the next token that fits the pattern.

That means the model has no built-in confidence gauge that maps cleanly onto truth. When it genuinely "knows" a fact because it appeared consistently across training data, the next-token prediction is well-calibrated. When it doesn't — a rare fact, a made-up citation format, a plausible-looking API that doesn't exist — the model doesn't fall back to silence or a hedge by default. It keeps predicting the most statistically plausible next token, which produces a fluent-sounding guess dressed up exactly like a fact.

Two different failure modes, one label

"Hallucination" gets used as a catch-all, but it's worth separating two distinct failures, because they call for different fixes:

• Intrinsic hallucination — the model contradicts or ignores information that was actually given to it in the prompt or context. You handed it a document that says the refund window is 30 days, and it answers "14 days." This is a failure to faithfully use what was already in front of it.
• Extrinsic hallucination — the model states something as fact that isn't verifiable from the given context or from anything reliably in its training data. This is pure fabrication — an invented citation, a nonexistent function signature, a court case that was never decided.

Intrinsic hallucination is, in a sense, the easier problem: the correct answer was right there, so the fix is about attention and faithfulness to context. Extrinsic hallucination is harder, because there was never a correct answer available to point at — the model simply needed to say it didn't know, and didn't.

Grounding: constrain generation to a verifiable source

Grounding is the general strategy underlying almost every practical mitigation: instead of relying on the model's internal, unverifiable "knowledge" (whatever got absorbed into its weights during training), you constrain it to generate from an explicit source of truth you can actually check — retrieved documents in a RAG pipeline, the result of a database query, the return value of a tool call. The model is still doing next-token prediction, but now the tokens it's predicting from are anchored to something real and inspectable, rather than to a diffuse, unverifiable prior spread across its parameters.

This is why grounding, not better prompting alone, is the foundation of reliable LLM systems: it doesn't change how the model generates text, it changes what the model has available to generate from.

Concrete grounding techniques

In practice, grounding shows up as a small set of composable techniques, usually stacked together:

• Scoped, context-only instructions — the simplest and cheapest fix: explicitly instruct the model to answer only from the provided context and to say "I don't know" when the context doesn't support an answer. It's a prompting-level fix, not an architectural one, but it meaningfully shifts the model away from filling gaps with fabricated content.
• Citation requirements — requiring the model to point to the specific source passage backing each claim. This doesn't stop a determined model from fabricating a citation too, but it makes unsupported claims far easier to spot: a claim with no citation, or one pointing at a passage that doesn't actually say that, is a visible red flag instead of an invisible one.
• Post-generation fact verification — a separate pass, often a second model call, that checks the generated output against the source material before it ever reaches a user. This catches cases where the generation step itself introduced an unsupported claim despite good instructions.

SYSTEM:
Answer ONLY using the context below. Do not use outside
knowledge, even if you are confident it is correct.

If the context does not contain enough information to
answer, respond exactly with: "I don't know based on the
provided information."

For every factual claim, cite the source passage it came
from in square brackets, e.g. [source: doc_3, para_2].

Context:
{retrieved_passages}

Question: {user_question}

Grounding is a mitigation, not a guarantee

It's tempting to treat grounding as a solved problem once it's wired up — retrieval is in place, citations are required, verification runs on every output — but none of that removes the generation step's fundamental nature. The model can still misread a correctly retrieved passage, merge two similar-sounding facts from adjacent chunks, or draw an inference that sounds supported by the source but subtly overstates it. Grounding narrows the space in which hallucination can occur; it doesn't close it.