Stage 2: Query Understanding

A diagnostic reference. Read while optimizing an existing retrieval-augmented system. Each section serves either a human diagnosing the system or an agent interrogating it.

Concept

Query understanding is everything done to the query before retrieval. The stage receives raw user text and produces whatever retrieval actually consumes: a structured query tree with filters and boosts, an expanded and category-scoped term set, a dense query vector, or a cached interpretation inherited from an equivalent earlier query. Many production systems have no such stage; the raw query goes straight to matching and ranking, and every downstream stage compensates for meaning that was never extracted. The first diagnostic act for this stage is checking whether it exists.

The source course’s governing idea (lesson 10): a search intent is a distribution over the space of documents, and a query is best represented by a bag of documents known to be relevant to it. Engagement logs are the most trustworthy source of the bag; LLM judgment and LLM-generated queries are fallbacks. The mean of the bag’s document vectors is a query vector; the mean cosine from the bag to its centroid is a continuous specificity score; a cosine threshold between two queries’ vectors canonicalizes them to one intent. Every advanced technique at this stage operates on this same primitive: semantic knowledge graph (SKG) traversal subdivides the bag by category, wormhole vectors re-express the bag in a different vector space, semantic caching finds a prior query whose bag was similar, and signals boosting hard-codes the bag for head queries.

The operational skeleton (lesson 11) is a four-phase pipeline: parse (extract known entities, typically via a finite state transducer over an entity library), enrich (SKG traversal for classification, disambiguation, and in-domain expansion; semantic functions that rewrite the query tree deterministically), transform (serialize to engine syntax), and search. A fifth phase, agentic eval, wraps the pipeline in an LLM-judged retry loop and belongs to Stage 7. The lesson explicitly presents this as one of several ways to build the stage: LLMs can substitute at the parse and enrich phases, trading latency, cost, and determinism for flexibility and zero curation.

What good looks like at this stage: the query handed to retrieval encodes the user’s intent, with entities resolved to canonical forms, ambiguous tokens disambiguated by context, expansions drawn from the corpus rather than a generic model’s vocabulary, filters and boosts explicit, and equivalent phrasings recognized as one intent, all within the online path’s latency budget.

Symptom catalog

query_treated_as_document: The query is matched as if it were a known item

• Observable signal: Top results match the query’s surface (its tokens, or its raw embedding) rather than its intent. Lesson 10’s canonical case: “iPhone” on a marketplace returning an iPhone 3. For queries with engagement history, results seeded from the engagement bag diverge sharply from results for the raw query.
• Frequency: Very common. It is the default behavior of both lexical engines and naive dense retrieval, and it persists inside HyDE, which reshapes the query into one document rather than a distribution.
• Cost of leaving unaddressed: Systematic misranking on every query whose intent is a distribution, which is most non-navigational queries. Fixes attempted downstream (reranking, boosts) treat instances, never the class.

intent_fragmentation: One intent, many query strings, diluted statistics

• Observable signal: High-traffic paraphrases of one intent (“diet soda,” “sugar-free soda,” “zero calorie soda,” “soda without sugar”) appear as separate rows in query analytics, each with its own frequency, conversion rate, and learned signals.
• Frequency: Very common in any system keying statistics by query string. Lesson 10 uses exactly this pattern to convince a large retailer to adopt bag-of-documents.
• Cost of leaving unaddressed: Every model trained on per-query signals (ranking, autocomplete, personalization) learns from fragments of the evidence. The damage is diffuse and invisible per query.

tail_query_no_signal: Head intent, rare phrasing, no signal

• Observable signal: “Size 11 men’s shoes” performs well; “hey, do you have any size 11 shoes for men?” performs poorly, despite identical intent. Result quality correlates with query frequency, sharply.
• Frequency: Common. Lesson 10 calls tail queries expressing head intents one of the biggest open opportunities in practice.
• Cost of leaving unaddressed: The long tail of phrasings, a large fraction of traffic in most systems, never benefits from the investment made in head queries (signals, boosts, learning-to-rank features, structured understanding).

ambiguous_query_mixed_results: Multi-sense queries return blended senses

• Observable signal: “Mixer” returns audio equipment interleaved with kitchen appliances; “server” returns DevOps content next to restaurant-tipping content. Where dense retrieval is in play, the query embedding averages the senses and lands between clusters, matching neither well.
• Frequency: Common; guaranteed in any corpus whose vocabulary spans domains.
• Cost of leaving unaddressed: For genuinely ambiguous queries, no single results list is correct. Lesson 10 is blunt that diversifying a results page across mutually exclusive intents fails the user; the response it prescribes is a clarification dialogue (or a context-informed sense pick). Lesson 11’s multi-level SKG traversal supplies the machinery: term to top categories, then term to related terms within each category, with “server” splitting cleanly into DevOps and Travel readings.

broad_query_relevance_pressure: Broad intents ranked as if narrow

• Observable signal: Queries like “shirts” or “clothing” produce near-duplicate top results, minimal variety, and no refinement prompts. Popularity and business signals barely influence ranking on these queries even though the intent leaves room for them.
• Frequency: Common wherever the relevance-vs-popularity blend is a single global constant.
• Cost of leaving unaddressed: Broad queries are where best-sellers, promotions, and faceted refinement earn their keep; suppressing them costs engagement and revenue. The structural fix is a per-query specificity score (mean cosine from the bag to its centroid): high specificity leaves little room for popularity, low specificity invites it. Lesson 10’s empirical claim: high-specificity queries perform better because people know what they want.

naive_lexical_zero_result: Satisfiable natural-language queries return nothing

• Observable signal: Zero-result rate climbs with query token count. Lesson 11’s lab case: “top kimchi near Charlotte” as a literal AND query returns zero results, while the same intent through the pipeline (rating boost, category:Korean expansion, geofilter on Charlotte) returns exactly the right restaurants.
• Frequency: Very common in lexical systems with no query understanding layer.
• Cost of leaving unaddressed: Dead ends on precisely the queries where users state their intent most fully. Users learn to type keyword fragments or leave.

stop_word_misdiagnosis: Operator tokens stripped as noise

• Observable signal: The analyzer’s stop-word list contains tokens like “near,” “top,” or “best.” Queries containing them degrade in a characteristic way: “barbecue near Charlotte” with “near” stopped returns barbecue mentions anywhere, wrong senses included (lesson 11’s example surfaces a festival listing and a country kitchen’s BBQ chicken).
• Frequency: Common; stop-word lists are inherited from defaults and rarely audited against intent.
• Cost of leaving unaddressed: Location filters, popularity boosts, and other operator semantics are silently discarded. These tokens are semantic-function entities, and treating them as noise deletes part of the query’s meaning.

reductionist_synonym_failure: The synonym dial has no good setting

• Observable signal: Repeated thesaurus and expansion-rule edits without convergence. Aggressive per-token expansion drifts from intent; conservative expansion adds no recall. The team describes the tuning as whack-a-mole.
• Frequency: Common in mature lexical systems; this is the endgame of the AND-of-ORs rewrite.
• Cost of leaving unaddressed: Ongoing curation cost with flat quality. Lesson 10’s diagnosis: per-token decomposition assumes attributes and synonyms are independent when they are correlated (“Apple iPhone 16 black” resists decomposition). The holistic alternative keeps the same lexical engine but rewrites as an OR of canonically-equivalent whole queries.

out_of_domain_expansion: Expansion terms come from the wrong distribution

• Observable signal: Printed expansions contain terms that are noise for the corpus, or tokenizer fragments. Lesson 11’s side-by-side: out-of-the-box SPLADE expands “lasagna” to {la, zagna, she, hotel, festival, genre, city} and “barbecue” to a list containing bb and q, while SKG traversal of the same corpus yields category:Italian, {lasagna, alfredo, pasta, italian}.
• Frequency: Common wherever a generic pretrained expansion model (SPLADE without fine-tuning, a general LLM) is applied to a specific domain.
• Cost of leaving unaddressed: Precision loss on expanded queries and erratic recall gains. The in-domain alternative is free by construction: the SKG is computed from the system’s own indexes, so its expansions cannot leave the corpus vocabulary, and the same traversal returns a category scope that a flat token list cannot.

entity_taxonomy_drift: The entity library falls behind the corpus

• Observable signal: Queries containing newer product names, categories, or domain vocabulary produce no entity matches in the parse phase and fall through as generic terms. Sampled zero-entity-match queries contain strings that exist in the corpus’s canonical fields.
• Frequency: Common in any FST-based parsing setup with a hand-curated entity library; lesson 11 acknowledges curation as the labor-intensive part of the pipeline.
• Cost of leaving unaddressed: Structured handling (canonical forms, types, semantic functions) degrades silently as the domain evolves; the pipeline’s value decays without any error surfacing.

function_consumes_wrong_neighbor: Positional query-tree rewrites misfire

• Observable signal: Semantic-function invocations whose consumed neighbor has the wrong type: “near” grabbing a token that is not a place and emitting a broken or empty geofilter. Symptomatically, queries containing function trigger-words show anomalous zero-result or wrong-filter behavior.
• Frequency: Rare relative to the symptoms above, but characteristic of this architecture; it is the parse-phase bug class to audit for.
• Cost of leaving unaddressed: Hard failures on a specific query shape, usually invisible in aggregate metrics because trigger-word queries are a small slice.

brittle_semantic_cache_false_positive: The cache serves the wrong meaning

• Observable signal: Embedding-keyed cache hits where the cached query and the incoming query differ in a way users care about. Lesson 11’s practitioner case: “short heel shoes” and “long heel shoes” scored above 0.95 by a general-purpose embedding model, a catastrophic pair for a fashion marketplace.
• Frequency: Common in first implementations of semantic caching, which default to threshold-only lookup with a generic encoder.
• Cost of leaving unaddressed: Users receive results for a different intent, and the failure is sticky: the wrong interpretation keeps being served from cache. The discipline is precision-first: a 0.9 to 0.95 floor, top-1 retrieval rather than threshold-only, a labeled match/no-match pair set to validate the floor, and a domain-tuned or instruction-tuned encoder where the generic one cannot separate domain-critical distinctions.

llm_query_understanding_latency: LLM-in-the-loop on every query

• Observable signal: Traces show one or more LLM calls (rewriting, classification, entity extraction) inside the synchronous query path for every request. The LLM call dominates p95 latency; per-query cost scales linearly with traffic.
• Frequency: Increasingly common in systems built LLM-first after 2023.
• Cost of leaving unaddressed: Latency and spend on every query, including the large fraction whose understanding is deterministic or repeated. Lesson 11 supplies both mitigations: semantic caching (head-query interpretations inherited by torso and tail paraphrases at embedding-similarity lookup cost) and a deterministic fast path (FST parsing plus SKG enrichment plus semantic functions) with the LLM reserved for queries that defeat it. The source course states the qualitative tradeoff (deterministic, low-latency versus flexible, expensive) without quantifying an end-to-end budget; treat specific numbers as system-specific.

How to inspect

By code pattern

Pattern / location	What you’re looking for	What suggests a problem
Query path entry point: what happens between the request handler and the search call	Whether a query understanding stage exists at all	Raw query string passed straight to the engine; suggests naive_lexical_zero_result, query_treated_as_document
Analyzer / tokenizer config: stopwords, stop_words, stop-word file paths	Operator-like tokens in the stop list	”near”, “top”, “best”, “popular” stopped; suggests stop_word_misdiagnosis
synonyms.txt, synonym graph filters, expansion rule files	Per-token synonym expansion, AND-of-ORs construction	Large hand-maintained thesaurus with churn in git history; suggests reductionist_synonym_failure
SPLADE model IDs, expansion-model loading, LLM expansion prompts	The source of query expansions	Generic pretrained expansion model with no domain fine-tuning; suggests out_of_domain_expansion
Entity library files, tagger collections, FST build scripts	Entity coverage and freshness	Library last regenerated long before the corpus’s latest vocabulary; suggests entity_taxonomy_drift
Semantic-function registrations and their tree-rewrite code	Type checks on consumed neighbors	Positional consumption without a type guard; suggests function_consumes_wrong_neighbor
openai., anthropic., llm, prompt templates inside the query path	Synchronous LLM calls per query	LLM call on every request, no cache in front; suggests llm_query_understanding_latency
Cache layer: Redis key construction for query results/interpretations	String keys vs. embedding keys; the similarity floor	Embedding keys with floor below 0.9, or threshold-only lookup without top-1; suggests brittle_semantic_cache_false_positive
Signals / boost lookup keyed by query	Whether lookup is exact-string or intent-level	Exact-string keying; suggests intent_fragmentation and tail_query_no_signal
Ranking configuration: popularity/business-signal weights	Whether the relevance-vs-popularity blend varies per query	Single global weight, no specificity input; suggests broad_query_relevance_pressure
HyDE prompts or hypothetical-document generation	Where HyDE is applied	HyDE applied to all queries rather than scoped to QA/navigational shapes; suggests query_treated_as_document for distribution-shaped intents

By measurement

Measurement	How to compute	Healthy range / red-flag threshold
Zero-result rate by query token count	Segment the zero-result rate by number of tokens	Rate rising steeply with length indicates naive_lexical_zero_result
Intent-cluster fragmentation	Cluster top-N query embeddings (or bag centroids); count clusters containing multiple high-traffic strings	Many multi-string clusters indicate intent_fragmentation
Result quality by query frequency band	Same relevance evaluation on head, torso, and tail samples	A steep head-to-tail quality cliff indicates tail_query_no_signal
Sense purity of top-K for known-ambiguous tokens	For a hand-built list of ambiguous queries, label top-20 results by sense; measure interleaving	Heavily interleaved senses indicate ambiguous_query_mixed_results
Within-session re-query rate	Fraction of sessions where the user reformulates (“mask” then “N95 mask”)	Rising re-query rate is lesson 10’s primary friction signal for understanding failures; watch alongside scrolling, pagination, and abandonment
Cache false-positive rate	Labeled match/no-match query pairs; measure how many no-match pairs the cache floor admits	Any admitted no-match pair at the production floor is a red flag for brittle_semantic_cache_false_positive
Expansion in-corpus rate	For sampled queries, fraction of expansion terms that occur in the corpus at all	Low in-corpus rate indicates out_of_domain_expansion
Entity-match rate over time	Fraction of queries with at least one entity match, trended	Gradual decline indicates entity_taxonomy_drift
Query-path latency decomposition	Per-component p50/p95 (parse, enrich, LLM calls, cache, engine)	LLM call dominating p95 indicates llm_query_understanding_latency
Specificity distribution	Mean cosine from bag to centroid across bagged queries	Not a health metric per se; its absence from ranking inputs indicates broad_query_relevance_pressure

Diagnostic questions

Triage flow for an agent or engineer assessing this stage. Earlier questions are coarse; branches go deeper when answers are unclear or red-flag.

1. [interview] Is there a query understanding stage at all, or does the raw query go straight to matching and ranking? Lesson 11’s first-question heuristic. If absent, naive_lexical_zero_result and query_treated_as_document are the presumptive findings; everything else in this chapter is a roadmap rather than a diagnosis.

2. [code] What is the query representation handed to retrieval: raw string, expanded terms, structured tree, or vector? Branches:

   • Raw string to a lexical engine: check the stop-word list (stop_word_misdiagnosis) and zero-result rates (naive_lexical_zero_result).
   • Raw embedding to dense retrieval: ask how ambiguous and exact-token queries behave (ambiguous_query_mixed_results; see also retrieval#vector_dominance).
   • Structured tree: continue to entity and function questions.

3. [interview] Are ranking signals and statistics keyed by query string or by intent? Branches:

   • By string: quantify fragmentation (intent_fragmentation) and the head-to-tail quality cliff (tail_query_no_signal).
   • By intent: ask how canonicalization is computed and what threshold was validated.

4. [interview] How are ambiguous tokens handled: embedding average, SKG disambiguation, clarification dialogue, or nothing? “Nothing” plus a cross-domain vocabulary confirms ambiguous_query_mixed_results. Also ask whether broad queries are distinguished from ambiguous ones; conflating them misdirects the fix (clarification suits ambiguity; specificity-governed ranking suits breadth).

5. [code] Where do query expansions come from, and are the scores bounded? Branches:

   • Generic SPLADE or LLM expansion, unexamined: sample expansions and measure the in-corpus rate (out_of_domain_expansion).
   • Hand-maintained synonym lists with endless churn: reductionist_synonym_failure; consider OR-of-whole-queries.
   • Unbounded significant-terms-style scores: see retrieval#unnormalized_relatedness; SKG’s bounded relatedness is the structural fix.

6. [code] What entities and semantic functions are explicit in this domain, and how is the library maintained? Branches:

   • No entity concept: expected if question 1 answered “no stage”; note as roadmap.
   • Hand-curated library: check regeneration cadence against corpus vocabulary (entity_taxonomy_drift) and audit function neighbor-type guards (function_consumes_wrong_neighbor).

7. [code] Where in the query path does an LLM live, and which phases are trusted to it? Branches:

   • LLM on every query, synchronous: measure its latency share (llm_query_understanding_latency); ask what fraction of its outputs a deterministic path would reproduce.
   • LLM only in an eval/retry loop: that is the agentic fifth phase; assess under Stage 7.

8. [code] Does the cache key by string or by embedding? If embedding, what is the floor, is lookup top-1, and has the false-positive rate been measured on labeled pairs? No labeled pair set means the floor is unvalidated; presume brittle_semantic_cache_false_positive until measured.

9. [interview] How does ranking decide the relevance-vs-popularity blend, and does anything per-query inform it? A global constant with no specificity input confirms broad_query_relevance_pressure.

10. [interview] What friction signals are monitored: within-session re-querying, scrolling and pagination depth, abandonment, conversion softening? None monitored means query-understanding regressions are invisible; presentation bias in engagement-derived signals goes undetected. (Measurement detail belongs to Stage 6; the signals originate here.)

Interventions by likely cause

Ordered by frequency of cause. Cross-references to symptom IDs are in stage#symptom_id format.

Hypothesis	Supporting evidence	Intervention	Expected impact	Cost / risk
No query understanding stage exists (query_understanding#naive_lexical_zero_result, query_understanding#query_treated_as_document)	Raw query passed straight to the engine; zero-result rate rises with token count	Introduce a minimal parse-and-enrich pass: entity extraction (FST or LLM structured output) plus in-domain expansion; full four-phase pipeline as the mature form	Large; converts dead-end queries into answered ones and gives every downstream stage a better input	Medium: new online component; start minimal
Statistics keyed by string rather than intent (query_understanding#intent_fragmentation)	Paraphrase clusters visible in query analytics	Canonicalize by intent: bag-of-documents centroids per query, cosine threshold for equivalence, aggregate signals at the intent level	Every signal-consuming model improves; the gain compounds downstream	Low to medium: an offline join and average, plus a re-keying migration
Tail phrasings inherit nothing from head intents (query_understanding#tail_query_no_signal)	Head-to-tail quality cliff	Semantic caching with embedding keys (precision-first: 0.95 floor, top-1, labeled-pair validation); or route tail queries through their canonical intent’s signals	Head-query investment extends across the traffic distribution	Low: Redis supports embedding keys natively; the labeled pair set is the real work
Expansion source is out of domain (query_understanding#out_of_domain_expansion)	Expansion terms absent from the corpus; tokenizer fragments visible	Replace generic expansion with SKG traversal over the system’s own indexes (first-class in Solr; single-hop painless ports for Elasticsearch/OpenSearch); fine-tuned SPLADE is the alternative if a model is required	Clean in-domain expansions plus a category scope in the same traversal	Medium: depends on engine; painless ports are single-hop only
Ambiguous queries answered with blended results (query_understanding#ambiguous_query_mixed_results)	Interleaved senses in top-K for multi-sense tokens	Multi-level SKG traversal for sense inventory and classification; pick a sense from context, or trigger a clarification dialogue when senses are mutually exclusive	Removes a failure class no ranking tweak can fix	Medium: traversal is tens to hundreds of milliseconds; clarification is a product change
Operator tokens treated as stop words (query_understanding#stop_word_misdiagnosis)	“near”/“top”/“best” in the stop list	Register them as semantic-function entities (location filter, rating boost) in the parse phase; remove from the stop list	Restores filter and boost semantics on affected queries	Low: local change, plus function implementation
Ranking blend ignores query breadth (query_understanding#broad_query_relevance_pressure)	Global relevance-vs-popularity constant; broad queries show near-duplicate tops	Compute specificity (mean cosine, bag to centroid); feed it to ranking as the blend governor; train a regression head for unseen queries	Broad queries gain variety and business-signal room; narrow queries stay relevance-pure	Medium: requires bags; regression head needs GPU fine-tuning
Synonym expansion architecture is reductionist (query_understanding#reductionist_synonym_failure)	Thesaurus churn without convergence	Rewrite as OR of canonically-equivalent whole queries (equivalence via bag-centroid cosine); retire per-token expansion incrementally	Ends the tuning treadmill; holistic behavior on the existing lexical engine	Medium: needs the canonicalization layer first
LLM runs synchronously on every query (query_understanding#llm_query_understanding_latency)	LLM dominates p95; cost scales with traffic	Put a semantic cache in front; add a deterministic fast path (FST parse, SKG enrich, semantic functions) and reserve the LLM for queries that defeat it	Latency and cost drop on the cacheable and deterministic majority	Low for the cache; medium for the deterministic path
Cache floor unvalidated or encoder generic (query_understanding#brittle_semantic_cache_false_positive)	No labeled pair set; floor below 0.9; threshold-only lookup	Raise floor to 0.95, switch to top-1, build the match/no-match pair set; fine-tune or switch to an instruction-tuned encoder if domain distinctions still collapse	Wrong-meaning cache hits eliminated at the cost of some hit rate	Low for floor and top-1; encoder fine-tuning is cross-cutting work
Entity library drifted from the corpus (query_understanding#entity_taxonomy_drift)	Declining entity-match rate; corpus vocabulary missing from the library	Regenerate the library from indexed content on a cadence; add misspelling and variant surface forms; monitor match rate	Parse-phase coverage restored and kept	Low: offline regeneration; ongoing ownership needed
Semantic functions lack type guards (query_understanding#function_consumes_wrong_neighbor)	Function invocations consuming wrong-typed neighbors in logs	Add type checks before consumption; fall back to treating the trigger word as a plain term when the guard fails	Removes a hard-failure query shape	Low: localized code fix
HyDE applied to distribution-shaped intents (query_understanding#query_treated_as_document)	HyDE on all queries; e-commerce-style corpus	Scope HyDE to QA and navigational shapes; use bag-of-documents (engagement-derived, or LLM-generated bags where logs are absent) for distribution-shaped intents	Alignment between technique and intent shape	Low: routing change; the bag infrastructure is the larger prerequisite