# Agentic Orchestration

Search Engineering · Course · Part 2 · Chapter 8

Interactive edition: https://mutapa.dev/search-engineering/course/agentic-orchestration

An agent wraps the search system you already run instead of replacing it. Every retry pays for the quality of those tools.

> Interactive instrument: Agent loop. The interactive edition replays a bounded search trajectory one phase at a time, including queries, tool results, evaluation, revision, and the condition that stops the loop.

[Chapter 7](https://mutapa.dev/search-engineering/course/evaluation-and-feedback.md) showed how judgments, behavior, and outcomes reveal when a system should change. Agentic orchestration brings this into one request: after each action, should the system continue, change course, or stop?

The opening instrument replays that process over the same 250 films used throughout the course. Its searches use the index, while its displayed thoughts and reflections are authored explanations of the trace rather than hidden model reasoning. Run the sequence or advance it one phase at a time. The first query returns weak results, inspection changes the search strategy, and a record lookup supplies the evidence needed for the answer.

A one-pass retrieval-augmented generation pipeline retrieves context and answers from one assembled evidence set. *Agentic orchestration* instead allows a model to choose among bounded actions based on intermediate results. The useful unit is the resulting trajectory: the queries issued, tools called, evidence returned, decisions made, and condition that ended the loop.

This chapter opens Part II by adding a control layer around the search system built in Part I, deciding when and how to use its stages again. That control earns its cost when a request is ambiguous or can recover from a weak first attempt. This chapter establishes the smallest useful loop before later chapters add planning, state, recovery, security, and trajectory evaluation.

## Detect when the first attempt failed

A one-pass pipeline can fail cleanly. A search for a family movie about food may return films about cooking that are unsuitable for children. Retrieval and ranking worked, yet the evidence satisfies only part of the request. Failure detection adds the missing comparison: the system checks the results against both constraints, whether the film concerns food and whether it suits a family audience. A result that satisfies only one constraint is not enough. The comparison can reuse the sufficiency check from [Chapter 6](https://mutapa.dev/search-engineering/course/generation.md), asking whether the results cover the request's entities and constraints, and the harness can add cheaper signals: an empty result set, retrieval scores below a floor, or a required field missing from every candidate.

A mismatch can trigger a bounded retry with a revised query or a different tool. If the system cannot resolve the mismatch, it should stop or ask for clarification instead of turning weak evidence into a confident answer. This comparison is the foundation of the loop because it identifies why the first attempt is weak and gives the next action a specific problem to solve.

## Search, inspect, and retry

The control loop turns that mismatch into an action. Because the first search found food-related films but no strong family choice, the model asks for another search rather than answering from incomplete evidence. The *harness*, the application code that manages tools and state, validates the request and runs BM25. It returns the results to the model, which narrows the query from food to chefs and kitchens because the first wording was too broad. The revised search surfaces *Ratatouille*. Rather than answer from the title alone, the model requests a record lookup, and the harness returns its genres and summary, giving the model enough evidence to recommend it as a family film.

Each pass repeats the cycle: choose an action, inspect the evidence, then answer or revise. Query expansion and synonym handling can respond to results while the harness enforces tool schemas, limits, and permitted actions. Tool results show what was retrieved, but they do not prove that the new query improved the result. The loop still needs a quality signal to guide the next action and justify stopping, and evaluation supplies that signal at two boundaries.

## Evaluate inside and outside the loop

The broad food search and the narrower chef query produce different results, but the agent needs a basis for choosing. Inside the loop, a *judge* scores each set against the request, turning reformulation from a guess into a comparison. The judge might use the cross-encoder from [Chapter 5](https://mutapa.dev/search-engineering/course/ranking-and-fusion.md), a learning-to-rank model, or a separate language-model call. Higher scores for chefs and kitchens tell the agent that the narrower query is a better direction.

Once the agent answers, evaluation moves outside the loop. The harness applies signals the agent does not control, then accepts the result, returns feedback for a bounded retry, or stops the process when its limits are reached: a retry ceiling, a token or latency budget, or a judge score that stopped improving between attempts. The judge guides a search the agent is still conducting; the harness decides whether the finished trajectory can pass. Both inherit the discipline from [Chapter 7](https://mutapa.dev/search-engineering/course/evaluation-and-feedback.md), including agreement with human judgments and outcomes.

## Tool design determines agent behavior

When evaluation finds a weak result, the agent can recover only through the actions its tools expose. Tool design therefore shapes the next query, the evidence returned, and the amount of work required before the agent can try again. For the movie task, search accepts keywords and returns titles, summaries, and scores; lookup accepts a film identifier and returns its record. These contracts support discovery and verification without exposing implementation details.

An engine-specific query language would instead make the model manage syntax and domain rules. High-level parameters and fixed lists of valid categories reduce those errors, while stronger retrieval reduces the number of reformulations. That efficiency matters because every retry adds tokens and latency. In informal results reported by one practitioner course, adding an agent improved relevance by roughly 15 percent, sometimes closer to 30, but the gains varied by dataset and cost more time.

## Reuse mature search infrastructure

Every retry pays for weaknesses in its tools, so adding an agent should begin with the search infrastructure you already run in production. A mature engine may already encode the precision, business rules, and feedback the agent would otherwise relearn. Starting instead by chunking every document, generating embeddings, and deploying a new vector database creates a second retrieval system, one that must earn its own relevance judgments, tuning, monitoring, and operational history.

Wrapping the production engine as a tool lets the agent reformulate queries while preserving the ranking from [Chapter 5](https://mutapa.dev/search-engineering/course/ranking-and-fusion.md) and signals from [Chapter 7](https://mutapa.dev/search-engineering/course/evaluation-and-feedback.md). The pattern applies to BM25, vector retrieval, taxonomies, rerankers, and learning-to-rank models. Rather than replacing information retrieval, orchestration gives the agent access to capabilities the search system has already earned.

## Connect session retries to system feedback

A retry repairs one request, but its trajectory can expose a problem shared by many requests. The queries attempted, tools called, judge scores, evidence returned, and final outcome show where the system repeatedly makes users work harder. Within a session, those signals decide whether the agent searches again. Across sessions, the feedback system from [Chapter 7](https://mutapa.dev/search-engineering/course/evaluation-and-feedback.md) aggregates them to reveal recurring failures in queries, tools, retrieval, or ranking.

If food requests repeatedly begin with weak broad results, the team can improve the search tool, retrieval rules, or ranking features. The next session then starts from a stronger system and should require fewer retries. This is a practical step toward the self-learning system introduced in [Chapter 1](https://mutapa.dev/search-engineering/course/what-is-relevance.md): a bounded inner loop corrects the request, while a measured outer loop improves the components used by future requests. Trajectories remain evidence rather than automatic training labels. Teams still need to validate them against human judgments and outcomes, correct for bias, version each change, and test it offline and online before release.

The loop developed here handles one request, one sequence of tool calls, and one decision to continue or stop. More complex requests introduce a new problem: the system must decide what work exists before it can choose the next action. [Chapter 9](https://mutapa.dev/search-engineering/course/plan-and-route-complex-searches.md) extends the loop by decomposing those requests, routing each part to an appropriate tool or agent, and deciding which searches should run in sequence or in parallel. Those tools eventually cross systems the search application does not own. [Chapter 15](https://mutapa.dev/search-engineering/course/connectors-and-trust-boundaries.md) defines the connector contracts that make their results usable.