The indexing work in Chapter 2 prepared the evidence available to retrieval. The online path begins with a different kind of evidence: the words a user types. A query may contain misspellings, aliases, remembered fragments, unstated constraints, or terms whose meaning depends on the domain. It describes the user’s task, but rarely states that task completely.
Query understanding interprets the request before retrieval. It may correct spelling, resolve an alias, identify a constraint, add domain language, or decide that the query should remain unchanged. Any transformation made here reaches every retriever, ranker, and downstream stage, which gives this step both leverage and risk. A useful rewrite exposes evidence the original query left implicit. A poor rewrite changes the task the system is trying to satisfy.
The instrument below compares three queries as typed and after rewriting. For each version, it shows results from BM25, which matches indexed terms, and vector retrieval, which compares semantic representations. Gold titles mark plausible answers to the original request. Select each case and ask whether the rewrite helps both retrieval methods, produces a mixed result, or was unnecessary.
The spelling case shows the clearest gain. The misspelled query gives BM25 only a generic term to match and neither retriever finds Jaws. After correction, both place it first. The vague-memory query behaves differently: both methods already return Groundhog Day first, so rewriting adds complexity without fixing the result.
The implicit-constraint case is less settled. Rewriting “like Alien but for kids” as “family friendly science fiction adventure with creatures” makes the desired attributes explicit, but it does not make every result family friendly or improve both lists consistently. The rewrite is an interpretation of the request, and the corpus may not contain enough evidence to enforce that interpretation. Query understanding therefore needs evaluation just like indexing and ranking.
A short query stands for many results
Lexical and vector retrieval both compare a representation of the query with representations of individual documents. That comparison works when similarity to one document is a good approximation of the user’s goal. The approximation becomes weaker when a short query describes a broad set of acceptable results.
Consider the query “iPhone” on a marketplace. A listing titled simply “iPhone” may resemble the query more closely than listings titled with a model, generation, storage size, or condition. That textual similarity does not establish that the listing is current, available, or appropriate for the shopper. The query provides too little evidence to identify one ideal document, and the set of useful results depends on behavior and context outside the text.
A broader intent can be represented as a distribution over documents. For “iPhone,” that distribution might include the models and accessories that people who issued the query went on to inspect or purchase. A navigational query, where the user wants a particular known page or item, produces a narrow distribution concentrated around one result. Category, recommendation, and preference-shaped queries usually admit a wider range of useful results.
Repeated interactions can make that distribution observable. For a query with enough traffic, collect the documents users engaged with and weight them by the strength of the interaction. The result is a bag of documents, a set of observed results that represents the range and relative frequency of accepted outcomes. Roughly twenty engagements is a practical starting threshold for constructing a bag: with only a few observations, one unusual click can distort the representation. Production systems should validate the minimum against their own traffic and the biases in what users were shown.
Hypothetical Document Embeddings (HyDE) offers another source of evidence when observed interactions are unavailable. A language model generates a hypothetical passage for the query, the system embeds that text, and vector retrieval finds real documents nearby. This can work well when one generated passage reasonably represents the target. A single passage captures less of the variation in a broad or preference-shaped intent, where multiple observed outcomes can represent the acceptable result space more directly.
The appropriate representation therefore depends on the shape of the request and the evidence available. The next section shows how to turn observed documents into a vector the retrieval system can use.
Represent intent from observed behavior
The documents in a bag can stand in for the original query text. Embed each one, give stronger interactions more weight, average the vectors, then normalize the result to unit length. The new vector points toward the part of the embedding space that users chose to explore, allowing retrieval to search from that neighborhood instead of relying only on the few words they typed.
This is a form of relevance feedback drawn from behavior, though the behavior remains imperfect evidence. A purchase may count more than a click, but neither is a direct judgment, and users can only choose from the results they were shown. Pseudo-relevance feedback begins with weaker evidence by treating the first retrieved results as relevant and using them to form another query. Chapter 4 shows how that composition works.
The shape of a bag provides useful signals of its own. A tight cluster suggests a specific intent, while a wider spread leaves ranking more room to use context or popularity. Nearby centroids can reveal different phrasings of the same intent, allowing the system to canonicalize those queries and pool their statistics. These averages work best for preferences expressed across a group of results. A hard constraint such as “for kids” must remain explicit as the query moves through the system.
Parse only when structure helps
A hard constraint needs different treatment from a broad preference, which raises a practical question: how much of the query should be broken into parts? A reductionist approach parses “Apple iPhone 16 black” into brand, product, model, and color. A holistic approach keeps the full query together so relationships among those terms survive.
Substitutability shows what can be lost. An Android phone is a better substitute for an iPhone than a MacBook is, even though the MacBook shares the Apple brand. Yet “I want yellow things” has the opposite shape: shirts, mugs, and cars share no product category, so the color must be handled separately. Useful structure depends on the request rather than one parsing rule applied to every query.
A practical pipeline can keep the whole-query representation while extracting only the structure it can use. It parses known entities, enriches them with domain evidence, then transforms the result into a retrieval request. Step through the phases in the instrument below and watch what each one adds to the request.
"something like Alien but for kids"
- The user names one known film.
- The user also states an audience constraint.
Fallback ruleIf parsing finds no useful structure, pass the original query forward unchanged.
As you move through the instrument, the family-friendly constraint becomes explicit but remains unresolved. The movie corpus has no content-rating field, so a more structured request still cannot enforce it. Parsing can identify known structure with a curated entity library, allowing surface forms such as “Kubrick” and “Stanley Kubrick” to resolve to the same canonical entity, but the evidence required to use that structure must still exist.
Enrichment can then apply semantic functions, small programs triggered by recognized terms whose behavior remains fixed and inspectable. Consider “top kimchi near Charlotte.” A lexical engine may find nothing when asked to match all four tokens because restaurant descriptions do not express location, cuisine, and quality in one text field. The pipeline instead maps “top” to a rating boost, “kimchi” to Korean cuisine and related terms, and “near Charlotte” to a geographic filter. The retriever stays the same, but receives evidence the index can use.
Learn domain language from the index
The related terms for “kimchi” do not have to come from a hand-written synonym list. Chapter 2 introduced the inverted index, which maps terms to documents. Add a forward index that maps each document back to its terms, and the system can move in both directions: from a query term to the documents that contain it, then outward to terms that occur unusually often in those documents.
That term-to-document-to-term structure, together with the categories attached to the documents, forms a semantic knowledge graph. Its value comes from the language of the indexed corpus. Consider a community Q&A corpus with technical and travel categories. In the technical category, “server” leads toward docker, nginx, and ssh. In the travel category, the same token leads toward restaurant, tipping, and VPN. Conditioning the traversal on a category separates meanings that a global synonym list would mix together.
The surrounding phrase can change the evidence just as much as the category. “Airplane” points toward travel, while “airplane crash” points toward science fiction because crashes occur more often in fictional stories than in travel discussions within this corpus. The expansion follows how people use the terms together rather than a fixed definition assigned to either word.
SPLADE, which Chapter 4 introduces, offers a learned alternative. Without domain fine-tuning, its expansion for “lasagna” in a restaurant corpus includes fragments such as “la” and “zagna,” along with “hotel” and “festival.” The index traversal returns pasta, alfredo, and the Italian category instead. Fine-tuning may close that gap, but the index-based method requires no training and keeps every expansion traceable to evidence the system already stores.
Reserve model latency for difficult queries
Learning expansions from the index is useful partly because query understanding happens between a keystroke and the appearance of results. Every operation in this stage consumes time from the same request budget that retrieval and ranking will need later.
The pipeline should therefore begin with the least expensive method that can preserve the request. A deterministic parser can handle known aliases and structured fields, while a bounded index traversal can add domain terms. A language model becomes useful when the query remains ambiguous or requires an interpretation those methods cannot express. Applying that model to every query would add latency and cost even when the original text or a cached interpretation was already sufficient.
Semantic caching reduces that repetition by storing an expensive interpretation under a representation of its intent rather than only the exact query string. A later phrasing of the same request can reuse the work, provided that context and hard constraints still agree. When no cached interpretation fits and the first results expose a mismatch, an agent can inspect the evidence, revise the query, and search again. Chapter 8 develops that loop.
Query understanding ends by handing retrieval a better-specified request, along with any uncertainty it could not resolve. Chapter 4 follows that request into lexical and vector retrieval, where the system must turn the available evidence into a set of candidates.