Chapter 8 ended with a bounded loop that could repair one weak search. A request with several information needs creates an earlier problem because the agent cannot choose its next action until it knows what work the answer requires.
Consider this request: find one film about artificial intelligence and one about memory loss, compare how each treats identity, then recommend the better choice for a discussion group that prefers ambiguity over action. The two films can be found independently, but the comparison needs both results, and the recommendation needs the comparison. One search cannot express those relationships or establish when the whole request has been answered.
A plan makes the required work and its dependencies explicit. Each task names the result it should produce, the tools or sources allowed to produce it, and the evidence that would make it complete. That structure lets the agent run independent work together, hold later decisions until their evidence is ready, and repair a weak branch without repeating a successful one. Without those boundaries, planning can multiply searches without bringing the answer any closer.
Define the claims the answer must support
Before choosing any steps, the planner needs a completion condition: a concrete account of what the final answer must establish. The film request requires two supported choices, plot evidence about identity from each film, a comparison grounded in that evidence, and a recommendation that applies the group’s preference. An instruction such as “research the request” gives the agent activity but no defensible stopping point.
The completion condition turns those obligations into claims. One claim says the first film concerns artificial intelligence; another says the second concerns memory loss. The comparison claim cannot be supported until both film records supply evidence about identity, and the recommendation cannot be made until that comparison can be judged against the preference for ambiguity over action.
These relationships keep decomposition attached to the user’s outcome. A planner that merely follows nouns might launch separate searches for artificial intelligence, memory, identity, ambiguity, action, discussion groups, and recommendations. The resulting topics would lose the structure of the request. “A film about memory loss” is one retrieval need, while “ambiguity over action” is a preference to apply after the films have been compared, not another fact to retrieve from the corpus.
With the required claims defined, the planner can ask which evidence is independent and which decision must wait for an earlier result.
Turn the claims into dependencies
The artificial-intelligence and memory-loss claims can be pursued in parallel because neither needs the other’s result. Each branch searches the corpus, inspects its leading candidates, and opens a full record before committing to a film. The comparison remains blocked until both records are verified, and the recommendation remains blocked until the comparison is complete.
This arrangement forms a directed acyclic graph, or DAG. Each node is a task, and each directed edge means one task depends on another. Because the graph contains no cycle, the tasks can be ordered so that every prerequisite appears before the task that needs it. A runnable node can still fail or exhaust its budget, but the graph keeps that failure distinct from a dependency that has not yet been satisfied.
Press Play below. Watch the two discovery branches start together, then follow the weak memory branch through a targeted replan. The highlighted phase, tool call, and task states show that completed work remains verified while only the unresolved claim changes.
Request
Find films about artificial intelligence and memory loss, compare how each treats identity, then recommend one for a group that prefers ambiguity over action.
Plan dependencies
AI branch
Find an AI film
Open Ex Machina
Memory branch
Find a memory film
Open Eternal Sunshine
Compare identity
Recommend one
The trace makes the dependency graph operational. The artificial-intelligence branch stays complete when the first memory search returns weak, action-heavy matches, so the revised query repairs only the open claim. Parallel execution saves elapsed time only because those branches are independent. If the request instead asked for another film by the director of the artificial-intelligence film, the second search would have to wait for the first record to supply the director.
These task boundaries give the harness places to validate progress, cache verified results, and reject redundant work. They do not yet determine which source can produce the evidence each task needs.
Choose sources by the evidence they return
The artificial-intelligence branch begins with discovery. Lexical search can find distinctive plot words when the request overlaps the summary, while vector search can recover the theme when the language differs. Either method may return Ex Machina as a candidate, but a title and score cannot support the later comparison. That claim needs record lookup, whose contract returns the complete summary, metadata, film identifier, and provenance.
Other tasks call for different contracts. A metadata filter can enforce a known year, director, or genre because those fields are stored explicitly. Record lookup cannot begin before discovery provides an identifier, just as search cannot verify evidence it never returns. Those limits require the planner to route by documented inputs and outputs rather than by a tool’s suggestive name.
Work through the routing instrument’s three tasks in order. The first two follow the Ex Machina branch from discovery to verification. The third extends the request by asking where the film streams tonight, which requires current regional information outside the connected corpus. Choose the source that could support each claim, then watch what happens when the right source is unavailable.
Your task
Route three parts of one request. For each task, identify the evidence it needs before choosing a source.
Task 1 of 3
Find a film about artificial intelligence
Evidence needed
A discovery source that can match a theme expressed in the plot.
Choose a source
The unavailable route is part of the result. The agent can state that availability lies outside the connected sources, ask permission to use another service, or omit the unsupported claim. A plot record cannot fill that gap. A regional catalog could, but its answer would be useful only for a specified place and time. The harness would also need to preserve the catalog’s provenance and enforce its authorization and failure rules.
Bayer and Thoughtworks’ PRINCE system applies this separation in production. It routes structured questions to Text-to-SQL and unstructured report questions to document retrieval, preserving the source of truth and retrieval behavior for each domain. The agent coordinates those systems instead of flattening them into one context window.
Match model capacity to the task
After source routing determines where the evidence comes from, model routing determines how much inference capacity each task needs. Extracting named fields from a verified film record into a fixed schema may fit a smaller model. Comparing how two films treat identity, then applying an ambiguous preference, may require a larger one. The distinction belongs to the task, not to the request as a whole.
The harness should begin with a deterministic policy a person can inspect. Task type and required output describe the work. Input size and tool needs rule out routes that cannot perform it, while ambiguity or an earlier validation failure may justify a stronger model. The result should retain enough context to reproduce the choice: the policy version, its reason, the chosen model, and that route’s token and latency limits. If a fallback runs, record that change too. A model-based router adds another call with its own cost, delay, and failure modes, so measured quality gains must justify that overhead before it enters the request path.
A cheaper route still needs a quality gate. Evaluate both model candidates on the same cases, report results by request class, and test the fallback path. Escalation should respond to a failure the harness can observe: invalid schema output, missing tool support, an input beyond the route’s budget, or a validation score below a threshold established on evaluation cases. Escalating every uncertain task gives up much of the expected saving, while forcing difficult work onto the smaller route converts the saving into retries and corrections.
Because behavior, price, and latency can change, model routing must remain a release policy rather than a permanent ranking of models. Keep model identifiers outside the policy’s semantic task categories and re-evaluate the routes under the current workload. Chapter 12 closes that loop by comparing route-specific quality, latency, and cost.
Run ready work once
With dependencies and routes assigned, the harness can start both film-discovery branches together. Their searches are independent, but the comparison still waits for two verified records. Uber’s production Cart Assistant uses a related pattern: independent item paths execute concurrently, later stages enforce cart-level constraints, and concurrency limits keep the branching work bounded.
Parallel work needs limits. The harness caps how many branches can run at once and how many calls, tokens, and seconds the request can spend. It caches each search under its source and the exact normalized inputs that determine the result. An identical call can then reuse its evidence, while a reformulated query remains a new attempt because its changed wording may retrieve different films. If the memory-loss branch uses its budget without finding a supported film, that claim stays unresolved. The final answer must report the gap because polished prose cannot supply the missing evidence.
Those limits still apply when a branch is delegated to a specialist agent. Separate context, tools, or expertise may make that boundary useful, but the handoff creates another place where constraints or evidence can be lost, and two specialists can still repeat the same search. The harness must give each specialist a scoped task, require a structured result with provenance, and accept that result only if it advances one of the shared completion claims. Delegation changes who runs the branch, not what counts as progress.
Let evidence change the plan
Execution changes the graph only when an observation changes what the plan knows. If vector search returns Ex Machina and record lookup confirms that its plot centers on a test of a humanoid machine’s intelligence and motives, the artificial-intelligence claim closes. Weak results for the memory-loss branch leave that claim open, so the planner can reformulate its query, change retrievers, or inspect another candidate without reopening the verified branch.
A later search may return Eternal Sunshine of the Spotless Mind, whose record supports both memory erasure and the characters’ unstable sense of their relationship. That evidence closes the second discovery claim and unlocks the comparison. If lookup instead contradicts the candidate match, validation rejects the candidate and reopens discovery rather than passing unsupported evidence downstream.
Missing coverage, contradictory evidence, a tool error, or a newly discovered dependency can justify replanning because each one changes the recorded state. Repeating searches until a result feels agreeable does not change the requirement, spends budget, and may narrow the evidence the agent considers. After every task, the harness should attach the result to the claim it addressed. The source and timestamp make the evidence traceable, while validation determines whether the claim closes or remains a gap. Recording the consumed budget lets the harness judge the next proposal against what remains. The model may propose another action; the harness decides whether it is valid, redundant, affordable, and permitted.
The opening completion condition also supplies the stopping rule. The agent can stop when every required claim has adequate evidence or when further work is unlikely to change the answer. If a limit leaves a required claim open, the result should name that gap. The agent should ask the user when the missing information is a preference or constraint only that person can supply, and search when the missing evidence belongs to a connected source.
Preserve the plan that actually ran
The final recommendation can conceal how the agent reached it. The same concise answer might come from two well-routed searches or from twelve repeated calls followed by an unsupported guess. The execution record makes those paths distinguishable. Each task remains linked to the work it depended on. Each attempt shows which source received which inputs, what evidence came back, and whether that evidence passed validation. Successful and failed attempts remain in sequence with their costs, so a reviewer can reconstruct the route rather than judging the answer alone.
That record should retain plan revisions rather than overwriting the original path. The request began as a set of claims, became a dependency graph, and changed when evidence closed or reopened its nodes. Keeping those versions creates a trajectory that can be debugged here and evaluated later.
The trajectory also becomes the state needed for recovery. If comparison fails after both film records have been verified, the system should resume from those completed branches, retain their provenance, and retry only the unfinished work. Chapter 10 follows that record through timeouts, malformed results, retries, and recovery.