# Diagnostic Runbook

Search Engineering · Diagnostic Runbook

Interactive edition: https://mutapa.dev/search-engineering/reference

Start with the failure you can observe. The runbook will route you to the part of the system that most likely owns the cause, then provide checks, measurements, and interventions.

## Find a symptom by visible effect

### Missing results

- [Index and query analysis chains disagree](https://mutapa.dev/search-engineering/reference/corpus-and-indexing.md#analyzer_mismatch): Index-time and query-time analysis chains disagree; terms that verifiably exist in documents never match because they were tokenized or stemmed differently on each side.
- [Aggressive analysis destroys distinctions](https://mutapa.dev/search-engineering/reference/corpus-and-indexing.md#lossy_analysis_chain): Over-aggressive stemming or stopword removal destroys distinctions at index time; exact-token queries (SKUs, names, citations) conflate or vanish, and the loss is unrecoverable at query time.
- [Quantized index with no recovery path](https://mutapa.dev/search-engineering/reference/corpus-and-indexing.md#full_precision_discarded): Index stores only quantized vectors; the over-fetch-and-rerank recovery pattern has nothing to rescore against, capping recall at the compressed ceiling (~60% for binary) with re-embedding as the only exit.
- [Satisfiable natural-language queries return nothing](https://mutapa.dev/search-engineering/reference/query-understanding.md#naive_lexical_zero_result): Multi-token natural-language queries return zero results even though the intent is satisfiable ('top kimchi near Charlotte' as a literal AND query).
- [Compression recall cap, missing over-fetch](https://mutapa.dev/search-engineering/reference/retrieval.md#underfetch_quantized): Quantized vector retrieval returns top-K directly. Recall caps at the quantization recall ceiling (~60% for binary). Team blames the embedding model when the fix is over-fetch + rerank.
- [Answers split across chunks](https://mutapa.dev/search-engineering/reference/retrieval.md#chunk_boundary_loss): Relevant passages exist in the corpus but never get retrieved. Test queries with known correct chunks return surrounding-but-wrong chunks because answer spans cross chunk boundaries.

### Wrong or weak results

- [No operational definition of relevance](https://mutapa.dev/search-engineering/reference/what-is-relevance.md#relevance_undefined): No one can state what a relevant result is for this system in terms a user would recognize; quality debates settle by anecdote or seniority.
- [Relevance treated as an algorithm property](https://mutapa.dev/search-engineering/reference/what-is-relevance.md#missing_domain_signals): Ranking is a single algorithm's output (BM25 or embedding cosine); recency, locality, popularity, and behavior never reach the score. Results are topically right and situationally wrong.
- [The toolbox reduced to its newest shelf](https://mutapa.dev/search-engineering/reference/what-is-relevance.md#embedding_conflation): The organization equates AI-powered search with embeddings plus LLMs; classical-ML options (signals boosting, LTR, semantic knowledge graphs, query understanding) never appear on the roadmap.
- [One context modeled, two missing](https://mutapa.dev/search-engineering/reference/what-is-relevance.md#single_context_system): The system models only one of the three contexts (content, domain, user): plain keyword search, a disconnected knowledge graph, or context-blind collaborative filtering.
- [Relevance by accumulated exception](https://mutapa.dev/search-engineering/reference/what-is-relevance.md#patchwork_rules_sprawl): Hundreds of synonym entries, pinned results, and per-complaint business rules accumulate unowned and untested; in practice the loudest stakeholder defines relevance.
- [One bit per dimension, too few dimensions](https://mutapa.dev/search-engineering/reference/corpus-and-indexing.md#bit_quantization_small_model): Binary quantization applied to a too-small embedding model (~384 dims); vector retrieval quality drops below the plain BM25 baseline.
- [The query is matched as if it were a known item](https://mutapa.dev/search-engineering/reference/query-understanding.md#query_treated_as_document): The system matches the query's tokens or embedding as if the query were a known item. Canonical case: 'iPhone' returns an iPhone 3, most like the token, least like the intent.
- [Head intent, rare phrasing, no signal](https://mutapa.dev/search-engineering/reference/query-understanding.md#tail_query_no_signal): A head intent phrased rarely receives none of the ranking signal, conversion data, or boosts its head-query equivalent enjoys, because signal lookup is exact-string.
- [Multi-sense queries return blended senses](https://mutapa.dev/search-engineering/reference/query-understanding.md#ambiguous_query_mixed_results): Multi-sense queries ('mixer', 'server') return interleaved results from disjoint intents; no disambiguation, no clarification dialogue; dense query embeddings average the senses.
- [Broad intents ranked as if narrow](https://mutapa.dev/search-engineering/reference/query-understanding.md#broad_query_relevance_pressure): Broad queries ('shirts', 'clothing') are ranked as if the user had a narrow target: popularity suppressed, near-duplicate top results, no refinement prompts.
- [Operator tokens stripped as noise](https://mutapa.dev/search-engineering/reference/query-understanding.md#stop_word_misdiagnosis): Operator-like tokens ('near', 'top') are stripped as stop words, discarding location-filter or boost semantics the query understanding layer should have executed.
- [The synonym dial has no good setting](https://mutapa.dev/search-engineering/reference/query-understanding.md#reductionist_synonym_failure): Per-token synonym expansion has no good setting: aggressive drifts from intent, conservative adds no recall; the team cycles through thesaurus edits without converging.
- [Expansion terms come from the wrong distribution](https://mutapa.dev/search-engineering/reference/query-understanding.md#out_of_domain_expansion): Query expansion introduces terms alien to the corpus (out-of-the-box SPLADE expanding 'lasagna' to la, zagna, hotel, festival); precision drops on expanded queries.
- [Positional query-tree rewrites misfire](https://mutapa.dev/search-engineering/reference/query-understanding.md#function_consumes_wrong_neighbor): A semantic function rewrites the query tree positionally and consumes an adjacent token of the wrong type ('near' followed by a non-city), producing a broken or empty filter.
- [The cache serves the wrong meaning](https://mutapa.dev/search-engineering/reference/query-understanding.md#brittle_semantic_cache_false_positive): The embedding-keyed cache serves an interpretation for a query that means something else ('short heel shoes' vs 'long heel shoes' scoring above 0.95 under a generic encoder).
- [Dense retrieval crowds out exact-term needs](https://mutapa.dev/search-engineering/reference/retrieval.md#vector_dominance): Pure dense or dense-heavy hybrid retrieval; exact-term queries (product IDs, names, technical tokens) systematically miss obvious matches; false positives elevated.
- [Naive fusion silently picks a winner](https://mutapa.dev/search-engineering/reference/retrieval.md#score_distribution_mismatch): Hybrid fusion produces erratic results because BM25 (unbounded) and cosine ([-1,1]) live on incompatible scales; one signal silently dominates without normalization.
- [Selective filters destroy HNSW recall](https://mutapa.dev/search-engineering/reference/retrieval.md#hnsw_filter_dominance): Adding selective filters (category, recency, user-scoped) to HNSW retrieval kills recall. Filters apply post-traversal; the graph walk doesn't know about them.
- [Reranker domain doesn't match corpus](https://mutapa.dev/search-engineering/reference/retrieval.md#crossencoder_training_mismatch): Cross-encoder fine-tuned on a different domain (e.g., MS Marco's general web QA) reranks poorly on the target corpus. Lab eval looks fine; production behavior is off.
- [Unbounded scores break term expansion](https://mutapa.dev/search-engineering/reference/retrieval.md#unnormalized_relatedness): Using Elasticsearch/OpenSearch significant-text aggregations or similar unbounded-score primitives for query expansion. Can't threshold reliably across queries because scores don't normalize.
- [The model rediscovers its own label source](https://mutapa.dev/search-engineering/reference/ranking-and-fusion.md#popularity_feature_loop): Feature importance shows popularity dominant and semantic similarity negligible. The model has rediscovered the proxy that generated its own labels.
- [All signals count the same](https://mutapa.dev/search-engineering/reference/ranking-and-fusion.md#flat_signal_weights): Boost values dominated by abundant low-confidence clicks; purchases barely move rankings; negative signals absent entirely. All signal types counted equally.
- [Boosting judged outside its scope](https://mutapa.dev/search-engineering/reference/ranking-and-fusion.md#signals_long_tail_no_op): Team reports 'signals boosting doesn't work'; measurement was on the whole query distribution, or the corpus is transient so per-document signal never accumulates. A structural scope limit rather than a tuning failure.
- [Click data enters the score twice](https://mutapa.dev/search-engineering/reference/ranking-and-fusion.md#signals_features_collision): Signals boost and LTR model disagree; rankings oscillate or the model appears to fight manual boosts. The same click-derived information enters the final score twice.
- [Phase windows chosen by copy-paste](https://mutapa.dev/search-engineering/reference/ranking-and-fusion.md#rerank_depth_mistuned): Relevant candidates sit just below the rerank window and never surface, or latency blows up because an expensive scorer runs over a set a cheaper phase should have narrowed first.

### Stale or unstable results

- [Vectors from a different model than the queries](https://mutapa.dev/search-engineering/reference/corpus-and-indexing.md#stale_embeddings_model_swap): Embedding model swapped or finetuned without re-embedding the corpus; query vectors and document vectors occupy different spaces and similarity scores are noise, often patchily by document age.
- [One intent, many query strings, diluted statistics](https://mutapa.dev/search-engineering/reference/query-understanding.md#intent_fragmentation): The same intent expressed in multiple phrasings ('diet soda', 'sugar-free soda', 'zero calorie soda') accumulates separate, diluted statistics; every model trained on per-query signals underperforms.
- [The entity library falls behind the corpus](https://mutapa.dev/search-engineering/reference/query-understanding.md#entity_taxonomy_drift): The hand-curated entity library falls behind the corpus; queries with newer vocabulary parse as generic terms and lose structured handling.
- [Cluster centroids age out](https://mutapa.dev/search-engineering/reference/retrieval.md#ivf_drift): IVF-indexed retrieval recall degrades over weeks/months on a corpus with significant new content. Cluster centroids become stale; new documents land in suboptimal clusters.
- [Semantic-only ranking ignores time](https://mutapa.dev/search-engineering/reference/retrieval.md#crossencoder_recency_blind): Cross-encoder ranks older content over newer equivalents. Users report stale results. Cross-encoders score semantic relevance only; recency is a non-semantic signal.
- [One user moves a head query](https://mutapa.dev/search-engineering/reference/ranking-and-fusion.md#signal_spam): A product or document climbs a head query's results with signal traceable to one or a few users. No per-user deduplication in the signals aggregation.
- [Query variants split their evidence](https://mutapa.dev/search-engineering/reference/ranking-and-fusion.md#fragmented_signals): Boost tables full of near-duplicate keys ('iPad', 'ipad', 'iPad 2'), each with weak counts; boosting underperforms on queries that are obviously popular. No query normalization in the GROUP BY.
- [Yesterday's winners never age out](https://mutapa.dev/search-engineering/reference/ranking-and-fusion.md#stale_signals): Last season's winners still rank on time-dated queries; users see the old holiday schedule. No half-life decay, or a half-life wrong by orders of magnitude for the domain.

### Unsupported answers

- [Fabrication despite grounding](https://mutapa.dev/search-engineering/reference/generation.md#unsupported_confident_claims): Answers contain specific claims (titles, numbers, names) absent from the retrieved context, delivered in a confident tone. Spot-checks of answers against their source passages fail.
- [Upstream failures blamed on the model](https://mutapa.dev/search-engineering/reference/generation.md#retrieval_failure_masquerading): Wrong or fabricated answers get attributed to the generation model when the retrieved candidate set never contained the needed information. Team iterates on prompts while the defect is upstream.
- [The generator obeys the wrong master](https://mutapa.dev/search-engineering/reference/generation.md#injected_instruction_compliance): The generator follows instructions embedded in user input or retrieved content, overriding intended behavior. No guardrail layer screens input or output.
- [No refusal path](https://mutapa.dev/search-engineering/reference/generation.md#empty_context_confident_answer): Zero-result or irrelevant retrievals still produce a fluent, confident answer. No refusal path, no query relaxation, no signal to the user.
- [The budget is exceeded silently](https://mutapa.dev/search-engineering/reference/generation.md#context_overflow_truncation): Assembled prompt exceeds the model's context budget; retrieved passages or conversation history are silently truncated and the answer ignores the dropped material.
- [Prompt wording substitutes for evaluation](https://mutapa.dev/search-engineering/reference/generation.md#prompt_change_unmeasured): System prompts change without a fixed evidence set, versioned rubric, or per-case comparison. Teams cannot tell whether a prompt improved grounding, refusal behavior, safety, latency, or cost.

### Latency, cost, or capacity

- [Slow responses with no attributable stage](https://mutapa.dev/search-engineering/reference/what-is-relevance.md#response_latency_regression): Responses became slow, but the system records only one total duration. No stage timing, route, model, cache state, retry count, or time-to-first-token dimension can explain the regression.
- [Chunking multiplies the index](https://mutapa.dev/search-engineering/reference/corpus-and-indexing.md#chunk_explosion): Chunking multiplies indexed document count by 100x to 10,000x; vector count, memory, and ANN build cost balloon (50M docs x 2K chunks = 100B vectors).
- [Segment merges rebuild the graph](https://mutapa.dev/search-engineering/reference/corpus-and-indexing.md#hnsw_merge_rebuild_cost): Indexing throughput collapses during segment merges on Lucene-family engines with HNSW fields; the graph is rebuilt on merge while query latency stays healthy, so the backlog grows unseen.
- [LLM-in-the-loop on every query](https://mutapa.dev/search-engineering/reference/query-understanding.md#llm_query_understanding_latency): One or more LLM calls run on every query for rewriting, classification, or extraction, dominating query-path p95 latency and per-query cost, with no cache and no deterministic fast path.
- [Lucene HNSW operational tax](https://mutapa.dev/search-engineering/reference/retrieval.md#hnsw_segment_merge_lag): Indexing throughput drops sharply during segment merges on Lucene-family engines (Elasticsearch, OpenSearch, Solr). The HNSW graph is rebuilt; cost is proportional to graph density.

### Untrustworthy evaluation

- [Nothing to measure against](https://mutapa.dev/search-engineering/reference/what-is-relevance.md#no_labeled_queries): No labeled query set exists anywhere in the system; retrieval and ranking changes ship without measurement against known-good results.
- [Measuring something easier than relevance](https://mutapa.dev/search-engineering/reference/what-is-relevance.md#proxy_metric_optimization): The team optimizes and reports proxies (benchmark scores, engagement, latency, corpus quality) while relevance for actual users goes unmeasured.
- [The system never learns](https://mutapa.dev/search-engineering/reference/what-is-relevance.md#no_feedback_loop): The system never learns from user behavior; the same query returns the same wrong results indefinitely and quality plateaued long ago.
- [Confusion between two distinct recall measurements](https://mutapa.dev/search-engineering/reference/retrieval.md#recall_metric_ambiguity): Quantization 'recall' numbers (e.g., 'binary quantization gives 60% recall') are measured against full-precision ranking rather than user-facing relevance. Confusing the two leads to wrong diagnoses.
- [The ranker trains on its own output](https://mutapa.dev/search-engineering/reference/ranking-and-fusion.md#presentation_bias_echo_chamber): Click-trained ranking keeps promoting the same class of results; new or differently-shaped candidates never gain ground; offline metrics look fine because they are computed on the same biased labels.
- [Judgments inherit the legacy ranker's ordering](https://mutapa.dev/search-engineering/reference/ranking-and-fusion.md#position_bias_labels): Click-derived judgments strongly correlate with the legacy ranker's ordering. Items ranked high by the old system look relevant regardless of merit; CTR was computed over raw impressions.
- [Raw ratios carry no confidence](https://mutapa.dev/search-engineering/reference/ranking-and-fusion.md#sparse_judgment_overconfidence): Rarely-shown documents with one or two lucky clicks outrank consistently-validated ones. Judgment grades of 1.0 sit on a denominator of 1; no prior, no confidence weighting.
- [Features computed at training time rather than click time](https://mutapa.dev/search-engineering/reference/ranking-and-fusion.md#dynamic_feature_leakage): Model performs well offline, poorly online; the gap concentrates in queries where volatile features (price, availability, badges) changed between click time and training time.
- [Train/test split at the wrong level](https://mutapa.dev/search-engineering/reference/ranking-and-fusion.md#query_split_leakage): Suspiciously strong offline metrics that evaporate in A/B tests. Train/test split was done at the document level; query-dependent feature vectors leak across the split.
- [Only clicks are logged](https://mutapa.dev/search-engineering/reference/evaluation-and-feedback.md#missing_results_list): Only clicks are logged; the result lists users actually saw exist nowhere. Skips are invisible and position-bias correction is impossible. Unrecoverable retroactively.
- [Signals keyed on query text](https://mutapa.dev/search-engineering/reference/evaluation-and-feedback.md#query_id_as_text): Signals key on query text (or a hash of it) rather than a per-instance ID; sessions with different result orderings collapse into one record, so actions attribute to results the user never saw.
- [Only the positive channel collected](https://mutapa.dev/search-engineering/reference/evaluation-and-feedback.md#no_negative_signals): Only positive actions are collected; skips, returns, and remove-from-carts never subtract, so rejected results keep whatever boost they earned indefinitely.
- [Sessions cannot be reconstructed](https://mutapa.dev/search-engineering/reference/evaluation-and-feedback.md#session_attribution_gap): Raw signals lack the identifiers (query_id, user_id, timestamps) that post-processing needs to reconstruct sessions; session-scoped judgments and skip mining cannot be computed.
- [The click model drops sessions without clicks](https://mutapa.dev/search-engineering/reference/evaluation-and-feedback.md#zero_click_session_loss): The click model silently drops every zero-click session; where zero-click share is high, most of the traffic contributes no evidence at all.
- [Judgments are raw CTR](https://mutapa.dev/search-engineering/reference/evaluation-and-feedback.md#position_bias_unaccounted): Judgments are raw CTR over impressions; per-rank CTR decays in a power law regardless of content, so whatever the legacy ranker put on top looks relevant.
- [One click, grade 1.0](https://mutapa.dev/search-engineering/reference/evaluation-and-feedback.md#sparse_judgment_overconfidence): Tail (query, doc) pairs with one click on one examine carry grade 1.0 and outrank pairs backed by hundredfold evidence; no prior, no shrinkage.
- [The model learns the ranker that fed it](https://mutapa.dev/search-engineering/reference/evaluation-and-feedback.md#presentation_bias_echo_chamber): The trained ranker reproduces the ranker that generated its training data; whole feature-space regions have zero examples and popularity compounds. Position-bias corrections do not reach this.
- [Labels treated as ground truth](https://mutapa.dev/search-engineering/reference/evaluation-and-feedback.md#judgments_never_ab_tested): Click-model output flows straight into model training; label bugs surface weeks later as inexplicable trained-model behavior instead of failing a cheap reversible test.
- [The window outlives the cause](https://mutapa.dev/search-engineering/reference/evaluation-and-feedback.md#stale_feature_aggregation): Judgments aggregated over a long window reward documents for reasons that no longer hold (was on sale, was in stock, was in the news); aggregation erased the timestamp of the click's cause.
- [Conversions as labels import confounders](https://mutapa.dev/search-engineering/reference/evaluation-and-feedback.md#deep_funnel_label_drift): Conversion-only labels favor cheap, easily purchased items and punish browsing-rich queries; search 'wins' experiments while downstream metrics sag, or the reverse.
- [The judgment key omits user state](https://mutapa.dev/search-engineering/reference/evaluation-and-feedback.md#personalization_query_collapse): Per-query-string judgments conflate distinct intents in personalized systems; the effective query includes user state but the judgment key does not.

### Agent or tool failures

- [Retries without convergence](https://mutapa.dev/search-engineering/reference/agentic-orchestration.md#spaghetti_at_the_wall): Many near-duplicate or semi-random query reformulations per session with no convergence; later queries in a session score no better than earlier ones. Signature of a missing judge-in-the-loop.
- [Self-certified completion](https://mutapa.dev/search-engineering/reference/agentic-orchestration.md#sycophancy_premature_done): The agent declares success and returns confidently regardless of result quality; users see polished prose wrapped around poor results. Signature of a missing outer harness.
- [Missing or theatrical loop bounds](https://mutapa.dev/search-engineering/reference/agentic-orchestration.md#runaway_iteration): Sessions loop until an infrastructure timeout, or every session terminates at the max-iteration cap. Token spend per session has a long tail an order of magnitude above the median.
- [The judge rubber-stamps](https://mutapa.dev/search-engineering/reference/agentic-orchestration.md#evaluator_misalignment): The loop runs, the judge scores, the harness passes outputs, and users still report poor results. Judge-human agreement was never measured on this corpus.
- [The loop pays for an upstream problem](https://mutapa.dev/search-engineering/reference/agentic-orchestration.md#brute_force_masks_weak_tools): Sessions succeed only after many retries; tokens per successful session are high and rising with corpus complexity. The loop is compensating for weak retrieval or ranking tools.
- [The tool hands the agent raw infrastructure](https://mutapa.dev/search-engineering/reference/agentic-orchestration.md#raw_backend_exposed): The agent constructs malformed or subtly wrong backend queries: bad DSL nesting, invented field names, filters that silently match nothing. The tool hands the agent raw infrastructure.
- [The tool fights the training distribution](https://mutapa.dev/search-engineering/reference/agentic-orchestration.md#off_distribution_tool_shape): The agent uses a tool awkwardly or avoids it entirely; the tool's signature has no analog in the web-search-shaped tool-use training distribution.
- [The agent parses its own tools](https://mutapa.dev/search-engineering/reference/agentic-orchestration.md#tool_returns_unstructured_prose): The agent misreads its own tool results: wrong ids passed to follow-up calls, scores misattributed, results conflated across calls. The tool returns free text instead of structured fields.
- [Mid-list options disappear](https://mutapa.dev/search-engineering/reference/agentic-orchestration.md#context_rot_long_list): Selection frequency by list position is U-shaped: the agent picks options from the start and end of a long in-prompt list and forgets the middle. Degrades after a couple hundred entries.
- [One model for every job](https://mutapa.dev/search-engineering/reference/agentic-orchestration.md#frontier_model_overuse): Every LLM call site (classification, judging, extraction, the loop) runs the most expensive model. Cost and latency are high; quality is no better than a mixed fleet would deliver.
- [The route moves cost into failures](https://mutapa.dev/search-engineering/reference/agentic-orchestration.md#model_router_mistuned): A model router exists, but difficult requests fail or retry on the small route while routine requests still reach the large model. Route-specific quality, latency, and cost were never evaluated together.
- [The loop recomputes cache-shaped work](https://mutapa.dev/search-engineering/reference/agentic-orchestration.md#agent_in_hot_query_path): Routine head queries pay for a full agentic loop at query time. Latency is dominated by LLM round trips recomputing work a cache lookup would have produced.

### Access or policy failures

- [Policy runs after candidate generation](https://mutapa.dev/search-engineering/reference/permissioned-retrieval.md#late_filter_leakage): Visible results are filtered, but counts, suggestions, features, or traces still reveal restricted records.
- [Restricted records reach downstream stages](https://mutapa.dev/search-engineering/reference/permissioned-retrieval.md#unauthorized_candidate_entry): A record outside the principal's scope reaches ranking or model context.
- [Cache reuse ignores authorization class](https://mutapa.dev/search-engineering/reference/permissioned-retrieval.md#scope_blind_cache): A cache entry is reused across authorization classes.
- [Secure filtering damages eligible recall](https://mutapa.dev/search-engineering/reference/permissioned-retrieval.md#filtered_ann_recall_collapse): Filtered vector search is secure but misses useful records inside the eligible subset.
- [A protected field already shaped a derivative](https://mutapa.dev/search-engineering/reference/permissioned-retrieval.md#field_mask_after_embedding): A protected field is removed from output after it already influenced an embedding or feature.
- [Audit data copies protected inputs](https://mutapa.dev/search-engineering/reference/permissioned-retrieval.md#restricted_trace_payload): Trace or audit data copies restricted content or raw identity claims.
- [Correlation is mistaken for authority](https://mutapa.dev/search-engineering/reference/policy-through-the-trajectory.md#authority_in_trace_baggage): Unsigned trace baggage is treated as proof of identity, scope, or approval.
- [An earlier permit becomes a transitive grant](https://mutapa.dev/search-engineering/reference/policy-through-the-trajectory.md#upstream_permit_reused): A downstream service accepts an earlier permit without making a current decision.
- [Delegation targets the wrong receiver](https://mutapa.dev/search-engineering/reference/policy-through-the-trajectory.md#wrong_audience_delegation): A delegated credential is replayed at a service it was not issued to.
- [Checkpoint state outlives policy](https://mutapa.dev/search-engineering/reference/policy-through-the-trajectory.md#stale_policy_resume): Checkpointed evidence resumes under an expired credential or older policy.
- [Memory enters another authorization class](https://mutapa.dev/search-engineering/reference/policy-through-the-trajectory.md#memory_scope_crossing): Memory created under one principal or purpose enters another trajectory.
- [The applied rule cannot be reconstructed](https://mutapa.dev/search-engineering/reference/policy-through-the-trajectory.md#policy_version_untracked): A decision cannot be reproduced because its policy version was not recorded.

### Connector failures

- [Indexed authorization data is stale](https://mutapa.dev/search-engineering/reference/connectors-and-trust-boundaries.md#acl_sync_drift): Indexed authorization attributes lag or disagree with their source.
- [Identity context expired before use](https://mutapa.dev/search-engineering/reference/connectors-and-trust-boundaries.md#identity_claim_expired): A connector supplies an expired or revoked identity assertion.
- [Transport succeeds after meaning changes](https://mutapa.dev/search-engineering/reference/connectors-and-trust-boundaries.md#connector_schema_drift): A connector response still parses but no longer means what the consumer expects.
- [A required source fails open](https://mutapa.dev/search-engineering/reference/connectors-and-trust-boundaries.md#unsafe_connector_fallback): A missing required source silently becomes a broader or unsupported completion.
- [Throttling creates more traffic](https://mutapa.dev/search-engineering/reference/connectors-and-trust-boundaries.md#rate_limit_retry_amplification): Parallel branches and retries multiply load after throttling.
- [An unknown effect is repeated](https://mutapa.dev/search-engineering/reference/connectors-and-trust-boundaries.md#ambiguous_write_replay): A timed-out action is repeated before its original commit status is known.
- [An assertion has no attributable source](https://mutapa.dev/search-engineering/reference/connectors-and-trust-boundaries.md#connector_provenance_missing): A policy-relevant assertion cannot be traced to a source version and observation time.

## Browse by system area

- [What Is Relevance?](https://mutapa.dev/search-engineering/reference/what-is-relevance.md): Define what useful results mean before tuning the system that produces them. (9 symptoms)
- [Corpus and Indexing](https://mutapa.dev/search-engineering/reference/corpus-and-indexing.md): Find where ingestion, analysis, chunking, and vector storage hide recoverable evidence. (7 symptoms)
- [Query Understanding](https://mutapa.dev/search-engineering/reference/query-understanding.md): Trace how a short request becomes intent, structure, expansion, and executable search. (13 symptoms)
- [Retrieval](https://mutapa.dev/search-engineering/reference/retrieval.md): Compare lexical and vector retrieval, then locate where candidate generation fails. (11 symptoms)
- [Ranking and Fusion](https://mutapa.dev/search-engineering/reference/ranking-and-fusion.md): Find where labels, behavioral signals, and reranking stages distort the final order. (13 symptoms)
- [Generation](https://mutapa.dev/search-engineering/reference/generation.md): Separate missing evidence, unsupported synthesis, unsafe instructions, and context limits. (6 symptoms)
- [Evaluation and Feedback](https://mutapa.dev/search-engineering/reference/evaluation-and-feedback.md): Audit whether observations, judgments, experiments, and outcomes support trustworthy learning. (12 symptoms)
- [Agentic Orchestration](https://mutapa.dev/search-engineering/reference/agentic-orchestration.md): Inspect loop control, evaluator quality, tool design, context pressure, and serving cost. (12 symptoms)
- [Permissioned Retrieval](https://mutapa.dev/search-engineering/reference/permissioned-retrieval.md): Locate where restricted evidence enters candidates, derivatives, caches, or traces before policy can contain it. (6 symptoms)
- [Policy Through the Trajectory](https://mutapa.dev/search-engineering/reference/policy-through-the-trajectory.md): Trace current identity, target, scope, policy, and validity through branches, memory, caches, and resume. (6 symptoms)
- [Connectors and Trust Boundaries](https://mutapa.dev/search-engineering/reference/connectors-and-trust-boundaries.md): Classify the external boundary, then test provenance, freshness, failure semantics, retries, and effects. (7 symptoms)