# Generation
Search Engineering · Diagnostic Runbook · Chapter 6
Interactive edition: https://mutapa.dev/search-engineering/reference/generation
Separate missing evidence, unsupported synthesis, unsafe instructions, and context limits.
## Start with the common causes
1. [The needed evidence never reached the model](https://mutapa.dev/search-engineering/reference/generation.md#retrieval_failure_masquerading)
Prompt changes cannot recover facts absent from the retrieved context.
2. [Claims are not checked against their sources](https://mutapa.dev/search-engineering/reference/generation.md#unsupported_confident_claims)
Specific names, numbers, or facts have no supporting passage.
3. [The system has no insufficient-evidence path](https://mutapa.dev/search-engineering/reference/generation.md#empty_context_confident_answer)
Empty or irrelevant retrieval still produces a fluent answer.
## Browse all symptoms
### Missing evidence
- [Upstream failures blamed on the model](https://mutapa.dev/search-engineering/reference/generation.md#retrieval_failure_masquerading): Wrong or fabricated answers get attributed to the generation model when the retrieved candidate set never contained the needed information. Team iterates on prompts while the defect is upstream.
- [No refusal path](https://mutapa.dev/search-engineering/reference/generation.md#empty_context_confident_answer): Zero-result or irrelevant retrievals still produce a fluent, confident answer. No refusal path, no query relaxation, no signal to the user.
### Unsupported synthesis
- [Fabrication despite grounding](https://mutapa.dev/search-engineering/reference/generation.md#unsupported_confident_claims): Answers contain specific claims (titles, numbers, names) absent from the retrieved context, delivered in a confident tone. Spot-checks of answers against their source passages fail.
### Instruction safety
- [The generator obeys the wrong master](https://mutapa.dev/search-engineering/reference/generation.md#injected_instruction_compliance): The generator follows instructions embedded in user input or retrieved content, overriding intended behavior. No guardrail layer screens input or output.
### Context limits
- [The budget is exceeded silently](https://mutapa.dev/search-engineering/reference/generation.md#context_overflow_truncation): Assembled prompt exceeds the model's context budget; retrieved passages or conversation history are silently truncated and the answer ignores the dropped material.
## Deeper inspection and interventions
## Concept
The generation stage turns a candidate set into an answer. The final ordered candidates from ranking and fusion are assembled into a prompt alongside the user's question, and an LLM produces a response grounded in that context. The reason this stage exists in its current form is hallucination: LLMs produce convincing but not necessarily truthful output, and retrieval supplies something true for the model to lean on. Grounding reduces fabrication; it does not eliminate it.
In classic RAG the stage is structurally one-shot. A first LLM call constructs the query, retrieval runs, and a second LLM call answers with the results in context. There is no self-correction: if the constructed query misses or retrieval comes back thin, the answer model grounds itself on bad candidates and answers anyway. Two consequences follow for diagnosis. First, a large share of apparent generation failures are upstream failures surfacing at the last stage, because this is the first stage a user actually sees. Second, any recovery behavior (query relaxation, refusal, retry) must be built explicitly; the pipeline provides none by default. The agentic variants covered at Stage 7 give the generator the agency to re-search bad context, at a cost in tokens and latency.
Prompt assembly mechanics (passage ordering, source labeling, citation markers) and faithfulness metrics for generated answers are less settled than the rest of this stage; treat guidance there as starting points to validate rather than established practice.
**What good looks like at this stage:** every claim in the answer is supported by the retrieved context, the system refuses or retries when the context cannot support an answer, and generation failures are distinguishable from retrieval failures in the logs.
## Symptom catalog
The catalog is deliberately short; a longer list would be invention.
### `retrieval_failure_masquerading`: Upstream failures blamed on the model
- **Observable signal:** Wrong or fabricated answers are filed as "the model hallucinated." Inspection of the failing requests shows the retrieved candidate set never contained the needed information: chunk boundaries split the answer span, the constructed query missed, or recall was simply low. The team iterates on prompt wording and model choice with no improvement.
- **Frequency:** Very common. The one-shot pipeline answers regardless of context quality, so every upstream defect exits through this stage.
- **Cost of leaving unaddressed:** Optimization effort lands on the wrong stage indefinitely. The generation model gets swapped, the prompt gets rewritten, and recall stays broken.
This symptom is listed first because it changes how every other symptom is read. Before any generation-stage intervention, read the retrieved context for a sample of failing requests. If the answer is not in the context, the defect belongs to Stage 3 (see `retrieval#chunk_boundary_loss`, `retrieval#vector_dominance`) or Stage 2 rather than here.
### `unsupported_confident_claims`: Fabrication despite grounding
- **Observable signal:** Answers include specific claims (titles, numbers, names, quotes) that appear nowhere in the retrieved context, delivered fluently and without hedging. The source course's canonical demonstration: asked without search for an author's BM25 blog posts, the model invents plausible titles the author never wrote.
- **Frequency:** Common, and lower than in earlier model generations. The source course's assessment: much less of a problem than in 2022, but still real even with search-augmented models.
- **Cost of leaving unaddressed:** User trust erodes on exactly the queries where the system sounds most authoritative. In legal, medical, or commerce domains the cost is direct.
Distinguish this from `retrieval_failure_masquerading` by checking the context: this symptom is present only when the context did contain adequate support and the model departed from it, or embellished beyond it.
### `injected_instruction_compliance`: The generator obeys the wrong master
- **Observable signal:** The generator follows instructions embedded in its inputs rather than its system prompt. A widely reported incident: a Chevrolet dealership chatbot was manipulated into agreeing to sell a car for one dollar, via user text instructing it to agree to anything and call it a legally binding offer.
- **Frequency:** Rare as a discovered incident, common as a latent exposure. Any deployed system whose inputs reach the prompt unscreened carries it.
- **Cost of leaving unaddressed:** Reputational and potentially contractual. Severity scales with what the chatbot is authorized to say or do.
That incident is injection through user input. Retrieved documents are a second injection vector: adversarial text in the corpus reaches the prompt through retrieval. Mitigations: a guardrail model run in parallel with the main flow (reject and replace the response if it trips), and fine-tuned safety classifiers such as Llamaguard, which classify against an explicit taxonomy. Constraining the generator to structured output limits the blast radius of a successful injection.
### `empty_context_confident_answer`: No refusal path
- **Observable signal:** Retrieval returns zero or irrelevant results and the system produces a fluent answer anyway. Logs show empty or near-empty candidate sets paired with confident responses. There is no code path that relaxes the query, and none that says "nothing found."
- **Frequency:** Common in first implementations, where the prompt template unconditionally demands an answer.
- **Cost of leaving unaddressed:** The system's worst answers are indistinguishable from its best. Users learn they cannot tell the difference and stop trusting either.
The source course prescribes both halves of the fix. Before generation: classic query relaxation (AND to OR to semantic search, or dropping filters). At generation: accept that true zero-result situations exist and say so. A system that always answers is worse than one that sometimes reports finding nothing.
### `context_overflow_truncation`: The budget is exceeded silently
- **Observable signal:** Answers ignore passages the team believes were provided. Token counts of assembled prompts approach or exceed the model's window, especially in multi-turn sessions where conversation history accumulates. Assembly code concatenates strings without measuring tokens.
- **Frequency:** Common in multi-turn deployments; rare in single-shot pipelines with small top-K.
- **Cost of leaving unaddressed:** Grounding quality degrades invisibly. The passages most likely to be dropped are the ones appended last, regardless of relevance.
Context windows are large (hundreds of thousands of tokens as of the source course, and growing) but finite, and headroom should be preserved for follow-on conversation. A related effect from lesson 12: even within the window, models lose track of mid-context material as prompts grow ([context rot](https://www.trychroma.com/research/context-rot)), which argues for fewer, better passages over aggressive context stuffing.
### `prompt_change_unmeasured`: Prompt wording substitutes for evaluation
- **Observable signal:** A prompt is approved because it reads more forcefully or fixes a few hand-picked examples. The comparison changed retrieval results, model, sampling, or rubric at the same time, and no per-case report survives.
- **Frequency:** Very common because system prompts are easy to edit and difficult to assess by inspection.
- **Cost of leaving unaddressed:** Regressions move between answer quality, refusal, citation support, safety, latency, and cost without attribution. Prompt complexity grows as old clauses remain after their original failure disappears.
Hold the question, retrieved evidence, structured output contract, and rubric fixed. Compare prompt and model candidates across supported, insufficient-evidence, and adversarial cases. Preserve prompt text and version with status, citation, support, safety, latency, cost, and per-case results. A framework can run the matrix, but it cannot choose the rubric or replace human calibration of semantic judges.
## How to inspect
### By code pattern
| Pattern / location | What you're looking for | What suggests a problem |
|---|---|---|
| Prompt template string; `join(...)` over retrieved docs | The assembly step: what is injected, with what instruction | Unconditional "answer using these results" with no branch for empty or weak context suggests `empty_context_confident_answer` |
| `len(results) == 0`, `if not candidates` near the generation call | A zero-result branch | Absence of any such branch; generation call is unconditional |
| Token counting (`tiktoken`, `count_tokens`) in assembly code | Whether the prompt is measured against the window | String concatenation with no token measurement suggests `context_overflow_truncation` |
| Conversation history handling (`messages.append`, session store) | How multi-turn history accumulates | Unbounded history growth with fixed top-K injection |
| Guardrail calls (`llamaguard`, `moderation`, safety classifier) around input and output | A screening layer | No guardrail anywhere suggests latent `injected_instruction_compliance` |
| The instruction text of the prompt | Whether the model is told it may decline | No "if the context does not answer the question, say so" style instruction |
| Logging around the generation call | Whether context and answer are stored together | Answer logged without its context makes `retrieval_failure_masquerading` undiagnosable after the fact |
| Prompt files and release checks | Versioned prompts evaluated against fixed evidence and assertions | Prompt text edited inline with no checked-in cases or result artifact: `prompt_change_unmeasured` |
### By measurement
| Measurement | How to compute | Healthy range / red-flag threshold |
|---|---|---|
| Support rate (claims backed by context) | Sample answers; check each factual claim against the retrieved passages, manually or with an LLM judge whose agreement with humans has been measured | Domain-dependent; any systematic unsupported-claim pattern is a red flag |
| Answerable-from-context rate on failures | For a sample of user-reported wrong answers, label whether the context contained the correct answer | If most failures were unanswerable from context, the problem is upstream: `retrieval_failure_masquerading` |
| Zero-result behavior | Count requests with empty or below-threshold candidate sets; inspect what was returned | Confident prose on empty candidates is `empty_context_confident_answer` |
| Prompt token distribution | Log assembled prompt token counts per request, p50/p95/max | p95 near the model window, or any request over it, flags `context_overflow_truncation` |
| Guardrail trip rate | If a guardrail exists, log trips per thousand requests | Zero trips ever may mean the guardrail is miswired rather than that inputs are clean |
| Session-level answer quality | Attribute user signals (copied quote, highlight, follow-up referencing a source) to the retrieved chunk they point at; average across many sessions | Individual sessions are noisy; the aggregate trend is the metric. See Stage 6 for the full method |
| Prompt candidate matrix | Run every prompt and model candidate over the same cases; compare status, citation, support, safety, latency, and cost by case | Missing combinations or aggregate-only reporting indicates `prompt_change_unmeasured` |
## Diagnostic questions
1. **[interview] When an answer is wrong, what does the team look at first?** If the answer is "the prompt" or "the model," and nobody reads the retrieved context, suspect `retrieval_failure_masquerading` before anything else. Force the habit: sample ten failures, read their contexts, label each answerable or not.
2. **[code] What exactly goes into the prompt, and in what order?** Establish the assembly pipeline: how many candidates, which fields, what instruction, where the question sits. This grounds every later question.
3. **[code] What happens when retrieval returns nothing, or nothing good?** Branches:
- No branch exists, generation always runs: `empty_context_confident_answer`. Add relaxation before generation and a refusal path within it.
- Relaxation exists: ask how many rungs and whether a true zero-result response is permitted at the end of the ladder.
4. **[code] Is the assembled prompt measured in tokens, and what is the multi-turn history policy?** Branches:
- No measurement, unbounded history: suspect `context_overflow_truncation`; add token logging first, then a budget.
- Measured and budgeted: ask how much headroom is reserved for follow-on turns.
5. **[code] Is there a guardrail on input, on output, or neither?** Branches:
- Neither: latent `injected_instruction_compliance`. Weigh a parallel guardrail; the parallel pattern hides its latency.
- Guardrail present: ask what taxonomy it was trained or configured against, and whether that taxonomy matches this deployment's actual risks.
6. **[interview] For answers that pass a spot-check today, has anyone verified claims against sources systematically?** Branches:
- Never: run a support-rate sample (see measurements). Unsupported claims with adequate context is `unsupported_confident_claims`.
- An LLM judge does it: ask whether judge-human agreement was ever measured; unmeasured judges inherit the alignment caveat from Stage 6.
7. **[interview] Is the one-shot ceiling the actual constraint?** If failures cluster on queries needing reformulation (the constructed query misses intent, and a human retrying with different words succeeds), the fix is architectural rather than prompt-level: see Stage 7 for the agentic loop that gives the generator retry capability.
8. **[code] What gates a system-prompt change?** No fixed evidence, versioned rubric, or per-case report means `prompt_change_unmeasured`. Freeze retrieval inputs first, compare prompt and model candidates, and calibrate any semantic judge against human decisions.
## Interventions by likely cause
Ordered by frequency of cause in real systems. Cross-references use `stage#symptom_id` format.
| Hypothesis | Supporting evidence | Intervention | Expected impact | Cost / risk |
|---|---|---|---|---|
| **Failures are upstream; generation is being tuned in vain** (`generation#retrieval_failure_masquerading`) | Sampled failing requests show the answer absent from retrieved context | Stop generation-side iteration; route the diagnosis to Stage 3 (`retrieval#chunk_boundary_loss`, recall measurement) or Stage 2 | Optimization effort lands on the real bottleneck | Low: measurement and triage discipline |
| **No refusal or relaxation path for weak retrievals** (`generation#empty_context_confident_answer`) | Unconditional generation call; empty candidate sets paired with confident answers in logs | Add query relaxation before generation; add an explicit instruction and code path for "the context does not answer this"; permit true zero-result responses | Worst answers become honest instead of confident | Low: local pipeline change |
| **Model departs from adequate context** (`generation#unsupported_confident_claims`) | Support-rate sample shows unsupported claims where context held the answer | Tighten the grounding instruction; reduce injected passage count to fight context rot; add a claim spot-check to evaluation | Fewer fabrications on well-served queries | Low to medium: prompt iteration plus evaluation work |
| **Prompt changes are not controlled experiments** (`generation#prompt_change_unmeasured`) | Inline prompt edits, hand-picked demos, no fixed evidence or per-case report | Check in prompt variants and representative cases; hold evidence and rubric fixed; compare status, support, safety, latency, and cost before release | Prompt efficacy becomes attributable and regressions become reviewable | Low to medium: fixture and rubric work plus evaluation calls |
| **Prompt assembly exceeds the context budget** (`generation#context_overflow_truncation`) | Prompt token p95 near the window; unbounded conversation history | Log token counts; set an assembly budget with headroom for follow-on turns; cap or summarize history; prefer fewer, better passages over more | Grounding stops degrading silently in long sessions | Low: assembly-code change |
| **No screening between inputs and generator** (`generation#injected_instruction_compliance`) | No guardrail layer; chatbot is publicly reachable | Add a guardrail model in parallel (reject and replace on trip); consider a taxonomy-trained classifier such as Llamaguard; constrain output to a structured shape | Blocks the known injection class; limits blast radius of the unknown | Medium: extra model call per request, tuning the taxonomy to the domain |
| **One-shot architecture cannot recover from query misses** (course framing; no single symptom) | Failures cluster on queries a human retry would fix; two-stage pipeline with no feedback | Adopt the agentic loop: generator with search tools, retry capability, iteration cap. See Stage 7 | Recovers reformulation-class failures | High: latency and token cost rise; new failure modes arrive with agency |