# Generation Search Engineering · Course · Part 1 · Chapter 6 Interactive edition: https://mutapa.dev/search-engineering/course/generation Generation is bounded by its evidence rather than the model's fluency. Every claim traces to a source, or the answer becomes a refusal. [Chapter 5](https://mutapa.dev/search-engineering/course/ranking-and-fusion.md) ended with a short, ordered set of documents and the scores needed to inspect why they were selected. Generation turns that evidence into a response. The ranking determines which sources receive the model's limited attention; the generator must preserve the connection between those sources and the claims it produces. A language model can answer without that connection. Its response may be specific, fluent, and internally consistent while names, citations, or technical claims lack support in the supplied documents. This failure is widely called *hallucination*. Retrieved context improves the available evidence, but it does not force the model to use that evidence, distinguish a source from its own prior knowledge, or remain silent when the answer is missing. Supplying retrieved passages as context for a generated answer is [retrieval-augmented generation (RAG)](https://en.wikipedia.org/wiki/Retrieval-augmented_generation). The basic assembly is straightforward: format the reranked passages with source identifiers, add the request and grounding instructions, then call the model. Reliability depends on preserving the connection between evidence and output. ## Require citations for generated claims A generator can state a supported fact and an invented detail in the same confident voice. To tell them apart, the answer needs *claim-level citations*: references that connect each verifiable claim to the retrieved passage meant to support it. That connection lets a reader inspect the evidence instead of taking the answer on trust. Citation coverage and citation support are separate tests. Coverage asks whether every claim points to a source, a check that software can perform mechanically. Support asks whether the cited passage actually justifies the claim. That requires comparing the wording and scope of the claim with the evidence. Even a well-placed citation cannot correct a source that is incomplete, outdated, or wrong. The instrument makes both checks visible. Select an answer claim to find its cited passages, or select a passage to see which claims rely on it. Then run the coverage audit to expose the claim that has no source at all. > Interactive instrument: Citation lab. The interactive edition compares generated claims with retrieved passages and identifies unsupported statements, missing evidence, and citation mismatches. The uncited sentence blends into the answer because its specificity and tone resemble the supported claims around it. The coverage check can flag the missing reference, but a citation alone would not make the sentence reliable. A complete audit still compares each claim with its cited text and records enough information to reproduce that comparison later. A useful citation shortens the path from a claim to its evidence. It should open the original source at the relevant passage and preserve identifiers such as the document, page, or section. The reader can then verify the claim without reconstructing the retrieval process. This kind of traceability builds trust one answer at a time. As the [Bayer and Thoughtworks account of a production RAG system](https://martinfowler.com/articles/reliable-llm-bayer.html#BuildingTrustInAProductionLlmSystem) illustrates, evaluation and monitoring are still needed to determine whether the same behavior holds across many requests. ## Unsupported claims can look well grounded A citation audit reveals that a claim lacks support, but it does not reveal where that support was lost. The generator may have ignored a relevant passage, or the passage may never have reached it. Both failures produce the same visible symptom: a fluent answer that extends beyond its evidence. Diagnosis therefore starts with the context supplied to the model. If that context contains the necessary evidence and the answer contradicts or exceeds it, generation failed. If the evidence is absent, the investigation moves upstream: retrieval or ranking may have omitted the source, a query transformation may have changed the intent, or a chunk boundary from [Chapter 2](https://mutapa.dev/search-engineering/course/corpus-and-indexing.md) may have separated the answer from the text needed to interpret it. The generator still needs an explicit instruction to decline when the supplied evidence is insufficient. That control limits the damage, but prompt changes cannot recover information that never entered the context. Treating every unsupported claim as a model problem will send you toward new prompts and new models while the actual defect sits upstream in indexing, query processing, retrieval, or ranking. That diagnosis depends on preserving the exact evidence package supplied to the model. Before the system can record and test that package, however, it must decide which passages fit and which ones to leave out. ## Work within the context window A model can process only a finite number of tokens in one request. This *context window* is shared by the system instructions, the user's request, retrieved passages, conversation history, and the generated response. A larger window raises the ceiling, but it does not remove the need to decide what belongs beneath it. Part of the constraint is arithmetic. Context assembly must reserve space for instructions and the answer before packing retrieved evidence. Code that concatenates passages until the limit is reached can silently truncate a source, citation identifier, or instruction. The resulting answer then appears to ignore information the team believes it supplied. The other constraint is attention. As prompts grow, models may use evidence near the beginning or end more reliably than material buried between unrelated passages. This degradation is often called [context rot](https://www.trychroma.com/research/context-rot). Adding more retrieval results can therefore increase recall while making the relevant evidence harder for the generator to identify. The assembly policy follows from both constraints. Set an explicit token budget, reserve the portions owned by instructions, history, and output, then add complete passages in ranked order until the evidence budget is exhausted. Dropping a lower-ranked passage is safer than cutting through its text and breaking the connection between a claim and its source. The reranking work in [Chapter 5](https://mutapa.dev/search-engineering/course/ranking-and-fusion.md) pays off again here. Its ordering decides which evidence receives scarce prompt space and where that evidence appears. Context assembly is therefore part of retrieval quality rather than a formatting step between ranking and generation. ## Handle insufficient evidence explicitly Fitting evidence into the context window does not mean that the evidence is sufficient. Retrieval may return nothing, several marginally related passages, or sources that support only part of the request. These states require different explanations, but none gives the generator enough material for a complete answer. Retrieval scores cannot establish sufficiency by themselves. They order candidates relative to one another, even when every candidate is weak. The system must instead ask whether the retrieved passages cover the entities, constraints, and claims needed to answer the request. A practical check extracts those entities and constraints from the request and verifies that each appears in at least one passage; where the mapping is less literal, a second model call can grade coverage, at the cost of another inference on the answer path. Conflicting sources should also remain visible rather than being blended into a false consensus. A bounded retry policy can attempt to recover missing evidence before generation. It might try an alternate query formulation, combine lexical and vector retrieval, or relax a filter that the user did not require. Constraints tied to identity, permissions, dates, or domain scope must remain intact because removing them can change the question or expose information that was never in scope. When those attempts still leave a gap, the answer path should stop. The response should state what could not be established, identify the missing evidence, and offer a useful next step such as clarifying the request or selecting another source. This behavior must be explicit in both the prompt and the surrounding application logic; otherwise a model optimized to be helpful will often fill the gap with plausible prose. In a one-pass pipeline, the surrounding code owns the retry limit and the decision to stop. An agent can choose its own next retrieval action, but it still needs the same stopping condition. Additional autonomy does not make an unsupported answer acceptable. ## Treat retrieved content as untrusted input Even sufficient evidence can be hostile. Retrieved passages are evidence for the answer rather than instructions for the system. A document can contain text that tells the model to ignore its rules, reveal private context, or produce a particular response. When that text reaches the prompt through search, the attack is an [indirect prompt injection](https://genai.owasp.org/llmrisk/llm01-prompt-injection/). The prompt should mark retrieved content as untrusted data and keep it separate from system and user instructions. That separation helps the model interpret each part correctly, but wording alone is not a security boundary. A sufficiently effective injection may still influence the output, and citations do not neutralize instructions embedded in a cited source. The durable controls live in application code. Grant the model only the tools and data required for the request, enforce authorization outside the model, validate tool arguments and structured outputs, and require approval before high-impact actions. Input filters, output filters, and secondary model checks can add useful signals, but none should have sole responsibility for blocking an attack. In this chapter's answer-only pipeline, the immediate goal is to prevent retrieved instructions from changing the response or leaking context. The stakes rise when generated output can select tools and cause actions. The same rule carries forward: retrieval may supply facts, but it cannot grant authority. ## Keep retrieval and generation separately testable The controls in this chapter act at different boundaries, so they must remain separately testable. An end-to-end answer can fail because a source was never retrieved, a useful passage was removed during context assembly, or the generator exceeded evidence that was present. Testing only the final prose collapses these defects into one score and leaves the team guessing where to intervene. The boundary between retrieval and generation needs an explicit contract. Retrieval should return ordered passages with stable source identifiers and enough metadata to reproduce the result. Generation should accept that evidence as a fixed input and return an answer whose claims point back to those identifiers, or a structured indication that the evidence is insufficient. With that boundary in place, retrieval can be evaluated against relevance judgments without calling a generator. Generation can be tested against fixed evidence packages without running search. End-to-end tests still matter, but their failures can be followed by stage-level tests that distinguish missing evidence from unsupported synthesis. Production traces should preserve the same seam: the original request, any transformed query, the index and model versions, the passages supplied, the prompt template, and the resulting answer. A failed response can then be replayed against the exact evidence the model received rather than reconstructed from a system that may already have changed. This chapter began with a ranked set of sources and ended with an auditable answer or an explicit decision not to answer. [Chapter 7](https://mutapa.dev/search-engineering/course/evaluation-and-feedback.md) turns those outcomes into measurements. [Chapter 8](https://mutapa.dev/search-engineering/course/agentic-orchestration.md) uses the same observable stages to decide when an agent should search again, choose another tool, or stop.