# Plan and Route Complex Searches

Search Engineering · Course · Part 2 · Chapter 9

Interactive edition: https://mutapa.dev/search-engineering/course/plan-and-route-complex-searches

A complex request is a graph of information dependencies, and progress comes from routing each part to a source that can answer it, then replanning from evidence rather than from motion.

[Chapter 8](https://mutapa.dev/search-engineering/course/agentic-orchestration.md) introduced the smallest useful agentic loop: choose an action, inspect the result, then answer, revise, or stop. That loop can repair one weak search. A request with several information needs introduces a prior decision. Before the agent can choose its next action, it must decide what work exists.

Consider a request over the film corpus: find one film about artificial intelligence and one about memory loss, compare how each treats identity, then recommend the better choice for a discussion group that prefers ambiguity over action. One search is unlikely to recover two well-supported examples and make the comparison. The artificial-intelligence film and the memory-loss film can be found independently, but the comparison depends on both results, and the recommendation depends on the comparison.

A *plan* represents those tasks and dependencies. Each task has an expected result, an eligible tool or source, and a condition that determines whether its evidence is sufficient. The memory-loss branch, written as a task, expects one film record, may use vector search followed by record lookup, and is satisfied only when the full record confirms the theme. The dependencies state which tasks can run now and which must wait.

Planning adds useful autonomy when the work cannot be fixed in advance. It also creates new ways to waste time. An agent can split one request into redundant searches, route a question to a source that cannot answer it, or keep adding branches after the available evidence already covers the goal. A reliable planner must therefore preserve the request's constraints while controlling fan-out, latency, and cost.

## Define what the plan must establish

A planner needs a completion condition before it can decide which steps are necessary. For the film request, completion requires two supported film choices, evidence about identity from each plot, a comparison tied to that evidence, and a recommendation that applies the group's stated preference. “Research the request” does not say when any of those obligations have been met.

The completion condition can be represented as a small set of claims the final answer must support. The two discovery claims can be tested separately: the first film concerns artificial intelligence, and the second concerns memory loss. The comparison claim requires evidence from both records. The recommendation then applies the preference for ambiguity over action to the comparison rather than introducing a new search target.

This structure keeps the plan attached to the user's outcome. Without it, decomposition tends to follow nouns in the request. The agent might search for artificial intelligence, memory, identity, ambiguity, action, discussion groups, and recommendations as seven parallel topics. Most of those searches would discard the relationships that give the request meaning.

The planner should preserve constraints that must be judged together. “A film about memory loss” is one retrieval need. Splitting it into separate searches for films and memory loss would produce candidate sets with no useful join condition. “Prefers ambiguity over action” is a ranking preference for the final comparison rather than a fact to retrieve from the corpus.

Once the goal is expressed as supported claims, decomposition becomes a question of information dependency: which evidence can be obtained independently, and which decision requires an earlier result?

## Decompose around information dependencies

The artificial-intelligence branch and the memory-loss branch can run in parallel because neither needs the other's result. Each branch searches the corpus, inspects its leading candidates, and opens a full record before committing to a film. The comparison waits for both records, while the recommendation waits for the comparison.

That arrangement forms a *directed acyclic graph*, or DAG. Each node is a task, and each directed edge means one task depends on another. The absence of a cycle guarantees the work can finish: a cycle would leave two tasks each waiting on the other's output, while an acyclic graph always has a runnable task until every node completes. Recording dependencies rather than a fixed order exposes both why later tasks must wait and which tasks are ready together.

Press Play below. Watch one weak branch send the system through replan while completed work remains verified. The highlighted phase, tool call, and task states show what changes.

> Interactive instrument: Plan map. The interactive edition steps through a decomposed request and shows which tasks run sequentially, which can run in parallel, and which evidence each task produces.

Parallel execution changes elapsed time only when the branches are independent. If the request instead asked for another film by the director of the artificial-intelligence film, the second search would need the director returned by the first lookup. Structured tasks make that dependency visible and give the harness boundaries where it can validate progress, cache verified results, and reject redundant work.

## Route each task to a source that can answer it

Decomposition determines what must be learned. Routing determines which capability can supply that evidence. The local film system exposes several useful operations: metadata filters for known fields, lexical and vector search over summaries, and record lookup by film identifier. Their contracts describe different evidence.

A metadata filter can enforce year, director, or genre because those fields are stored explicitly. Lexical search can recover distinctive terms such as “artificial intelligence” when the summary uses them. Vector search can recover a concept expressed in different language. Record lookup supplies the complete summary and metadata needed to verify a candidate after discovery.

The planner should choose among those operations from their documented inputs and outputs rather than their names. If a search tool returns only titles and scores, it can discover a candidate but cannot support a thematic comparison. If record lookup requires an identifier, it cannot begin the branch before discovery supplies one.

Work through the routing lab's three tasks in order. Read the evidence need, choose a source, then inspect the route result. The final task requires information outside the connected corpus, so a correct plan must preserve that gap.

> Interactive instrument: Source routing lab. The interactive edition asks you to route subtasks to available tools, then exposes the quality, latency, and authority consequences of each choice.

The unavailable route is part of the result. The agent can state that availability is outside the connected sources, ask permission to use another service, or omit the unsupported claim. Connecting that service would expand what the agent can know while adding requirements for freshness, locale, provenance, authorization, and failure handling.

[Bayer and Thoughtworks' PRINCE system](https://martinfowler.com/articles/reliable-llm-bayer.html#TheResearcherAgent) applies this separation in production. It routes structured questions to Text-to-SQL and unstructured report questions to document retrieval, preserving the source of truth and retrieval behavior for each domain. The agent coordinates those systems instead of flattening them into one context window.

## Route model work by difficulty

Source routing decides where evidence should come from. Model routing decides how much inference capacity a task needs. Constrained classification, extraction into a fixed schema, and a calibrated judge can often use a smaller model. Ambiguous planning, open-ended synthesis, tool selection, and recovery from a failed small-model attempt may justify a larger one.

Start with a deterministic policy over inspectable features such as task type, output constraints, input size, ambiguity, tool requirements, and prior validation failure. Return the route name, chosen model, policy version, reason, token and latency budgets, and fallback with the task result. A model-based router adds its own call, cost, latency, and failure mode; its measured gain must exceed that overhead before it belongs in the hot path.

The smaller route still needs a quality gate. Evaluate both candidates on the same cases, report results by request class, and test the fallback path. Escalation should respond to a concrete signal such as schema failure, low calibrated confidence, missing tool support, or an exceeded input budget. Sending every uncertain request upward saves less than expected, while forcing difficult work downward moves cost savings into retries and corrections.

Routing is therefore a release policy rather than a permanent model ranking. Model behavior, price, and latency can change, so keep model identifiers outside the policy's semantic categories and re-evaluate them under the current workload. [Chapter 12](https://mutapa.dev/search-engineering/course/evaluate-agentic-search.md) closes that loop by comparing route-specific quality, latency, and cost.

## Schedule independent work without multiplying it

Once tasks and routes are known, the harness can run every node whose dependencies are satisfied. Uber's production [Cart Assistant](https://www.uber.com/us/en/blog/uber-cart-assistant/) uses a related pattern: independent item paths execute concurrently, later stages enforce cart-level constraints, and concurrency limits keep the fan-out bounded.

The search planner needs the same accounting. It can cap active branches, tool calls, tokens, and elapsed time, and it can cache results under a normalized query and source so equivalent searches share one execution instead of paying for duplicate reformulations. A required branch remains visibly unresolved if its budget is exhausted. Hiding that gap behind fluent synthesis would turn a resource limit into an unsupported answer.

Multi-agent work follows the same rule. A specialist agent can own one branch when separate context, tools, or expertise make the boundary useful. Agent count alone does not improve the plan. Delegation adds another handoff where constraints or evidence can be lost, and two specialists can still repeat the same search. The orchestrator must supply a scoped task, receive a structured result with provenance, and evaluate whether the handoff advanced the shared completion condition.

## Replan from evidence rather than from motion

A plan is useful because observations can change it. Suppose vector search returns *Ex Machina* for the artificial-intelligence branch and record lookup confirms that its plot centers on a test of a humanoid machine's intelligence and motives. That branch is complete. The memory-loss search might return *Eternal Sunshine of the Spotless Mind*, whose record supports both memory erasure and the characters' unstable sense of their relationship. The planner can now compare the two without issuing more discovery queries.

If the memory branch returns only weak matches, the planner has a specific gap to repair. It can reformulate that branch, change retrievers, or inspect another candidate. It does not need to rerun the completed artificial-intelligence branch. If the record lookup contradicts the search result, the candidate should be rejected and the discovery task reopened rather than passed downstream as verified evidence.

Replanning should respond to a changed state: missing coverage, contradictory evidence, a tool error, or a newly discovered dependency. Repeating searches until one result feels agreeable changes none of those, and it spends budget while narrowing the range of evidence the agent considers.

The harness can make that distinction testable. After each task, it records which completion claim the result supports, the source and timestamp, the remaining gaps, and the budget consumed. The model can propose another action from that state. The harness decides whether the action is valid, redundant, affordable, and permitted.

Stopping becomes another planning decision. The agent should stop when every required claim has adequate evidence, further work is unlikely to change the answer, or a limit requires it to return the unresolved gap. It should ask the user when the missing information is a preference or constraint only the user can supply. Search is appropriate when the missing information belongs to a connected source.

## Keep the plan as an inspectable artifact

The final answer can conceal the path that produced it. A concise recommendation might come from two well-routed searches, or from twelve repeated calls followed by an unsupported guess; reading the answer alone, you cannot tell which. The plan and its execution record preserve the difference.

For each task, the record should include its dependencies, chosen tool and source, inputs, returned evidence, validation result, cost, and status. Plan revisions should remain visible as revisions rather than overwriting the original path. This creates a trajectory that can be debugged and evaluated later.

The record also provides the state needed for recovery. If the comparison fails after both film records have been verified, the system should not search for the films again. It should resume from the completed branches, retain their provenance, and retry only the failed work.

Planning has therefore changed one bounded loop into coordinated search work. The agent can decompose a request, route tasks to sources, run independent branches together, and revise the graph when evidence changes. Those abilities remain useful only if completed work and unresolved gaps survive between actions. [Chapter 10](https://mutapa.dev/search-engineering/course/maintain-state-and-recover.md) follows that state through timeouts, malformed results, retries, and recovery. The routed branches also need a shared policy context that each protected receiver can validate. [Chapter 14](https://mutapa.dev/search-engineering/course/policy-through-the-trajectory.md) carries that context through the graph.