Chapter 9 preserved the plan that actually ran as an inspectable execution record. That record must remain useful even when a failure arrives before the comparison can begin. The Ex Machina branch is verified, but the memory-loss search has just timed out.
Restarting might still produce an answer, but it would retrieve Ex Machina again, spend more calls, and risk replacing evidence the system had already accepted. Continuing safely requires the application to know that one branch is complete, one remains open, and the failed call changed neither conclusion.
At this point, the trajectory’s state captures the plan, the verified Ex Machina result, the unresolved memory-loss branch, and the budget already spent. A transcript can recount the exchange, but it cannot establish which result passed validation or which task may run next. The application needs a structured record that can survive the failure and preserve those distinctions.
Separate state by purpose and lifetime
The timeout leaves four different things to preserve: the user’s request, the work completed so far, the evidence supporting that work, and any information allowed to influence future sessions. They have different owners and lifetimes, so storing them as one undifferentiated history would make safe recovery harder.
| State class | What it contains | Typical lifetime | Failure when mishandled |
|---|---|---|---|
| Request state | Goal, constraints, approvals | One trajectory | The system drifts from the user’s intent |
| Working state | Plan, completed tasks, open questions, budgets | One trajectory or checkpoint | Work is repeated or skipped |
| Evidence state | Source IDs, passages, tool results, timestamps, confidence | As long as the result must be verified | The answer loses support or uses stale evidence |
| Durable memory | Stable preferences or reusable facts with provenance | Across sessions, under policy | Old, private, or poisoned memory changes later work |
For this comparison, request state defines completion: find two films, compare their treatment of identity, and favor ambiguity over action. Working state marks the Ex Machina branch complete and the memory-loss branch unresolved, while evidence state preserves the record and passages that justify the completed status.
Durable memory sits outside this trajectory because it may shape a later request. The same subject can appear in both places without carrying the same weight. “Prefer ambiguity for this discussion” is a current constraint. An older memory that the reader prefers action films is a separate claim whose source, age, and permitted use still need to be resolved.
Checkpoint only verified progress
A checkpoint is a versioned snapshot from which execution can safely continue. The harness creates one when a task produces the evidence required to pass its validation, rather than after every model thought or attempted call. Each transition follows the same sequence: read the current state, attempt an action, validate its result, then commit the next state.
That order prevents an uncertain result from appearing complete. If vector_search times out, the working state may record the failure and its retry count, but it cannot mark the memory-loss claim as supported. If get_film returns a record whose plot contradicts the candidate match, validation reopens discovery instead of committing bad evidence downstream.
Play the trace below. It pauses when the memory search fails. Choose a recovery policy, then compare the verified work retained, the additional calls, and any change in provenance.
Request
Compare Ex Machina with a film about memory loss, then recommend the more ambiguous treatment of identity.
Trajectory
Checkpoint v1- Working
Request
- Waiting
AI film
- Waiting
Memory film
- Waiting
Compare
Each recovery policy can eventually complete the comparison, but only two preserve the work already verified. Restarting discards the Ex Machina record and repeats four calls. Resuming reloads the checkpoint and retries the unresolved memory-loss branch with one additional call. Falling back also reloads the checkpoint, then replaces vector search with lexical search, which changes the path by which the second film enters the candidate set.
Whichever policy succeeds, the timeout remains in the execution record. It explains why recovery ran, while the last verified checkpoint shows which work was safe to keep. A later evaluator can then distinguish a comparison completed on its first attempt from one completed after a retry or fallback.
Resume from the last verified checkpoint
Recovery begins with the newest state that is both complete and valid. In the trace, checkpoint v2 contains the request, the plan, and verified evidence for Ex Machina. The timed-out memory search happened after v2 but produced no valid result, so replay loads that checkpoint and schedules only the unresolved branch.
Loading v2 turns the saved record into a scheduling decision. The checkpoint marks the Ex Machina node complete and the memory-loss node open, so the harness can reject any proposal that repeats the verified branch. The model may propose retrying the failed tool, reformulating the query, or selecting a fallback, but the harness runs that proposal only if it serves the open node, fits the remaining budget, and preserves completed dependencies. This division forms the state-machine boundary: recorded state determines what may run next instead of leaving that decision to the model’s recollection.
Bayer and Thoughtworks apply this recovery pattern at production scale in PRINCE, which searches preclinical study reports rather than films. After each workflow step, PRINCE saves its progress in PostgreSQL, a relational database serving as the checkpoint store. Logs, intermediate steps, and citations live separately in DynamoDB, a managed database for application data. When a research task fails, a user retry reloads the saved workflow and skips the completed steps. That is the production equivalent of keeping the verified Ex Machina branch while rerunning the memory-loss branch. PRINCE can retry one model call or the whole failed step, then switch model providers if those attempts fail.
Suppose checkpoint v2 contained only the sentence “Ex Machina is a good choice.” The harness could not tell which request that choice answered, which source supported it, or whether the record had passed validation. It might repeat finished work or treat an unsupported suggestion as complete. To prevent either error, v2 must connect the original request and current plan to the node status, tool result, source, and validation decision that closed the branch. Its budget and failure history then tell the harness whether the open branch can afford another attempt.
Saving the final recommendation would erase those relationships. Saving every token of the model’s hidden reasoning would bury them in a costly, potentially sensitive transcript that the harness cannot use as an execution contract. A useful checkpoint sits between those extremes: it preserves the structured facts and decisions that the application will test when execution resumes.
That middle ground is manageable for two film searches, but a deep research task may accumulate many rounds of evidence and failed directions. Microsoft Research’s Re-TRAC addresses that growth by turning each completed trajectory into a structured state containing its evidence, uncertainties, failures, and future plans. The next trajectory begins from that state rather than replaying the entire history. Re-TRAC’s reported gains come from research benchmarks rather than a production deployment, but its role here is precise: compression remains useful only when it preserves what the next decision needs.
Retry only operations that can be repeated safely
A retry repeats an operation after a failure that may clear on another attempt. Repeating the timed-out memory search is relatively safe because the call only reads the corpus, so running it again cannot duplicate a write. It can still waste the remaining budget if the timeout persists, and a malformed query or invalid field name will usually fail again until its input changes. The retry policy must use that difference: it may retry a transient timeout within a fixed limit, but it should send a deterministic input error back for repair instead of repeating the same call.
PRINCE turns that distinction into a bounded repair loop. When its SQL tool encounters a syntax or schema error, it returns the error to the model so the query can change before another attempt. After three failures, the node remains unresolved rather than running indefinitely. The film harness needs the same stopping rule: if revisions do not repair the memory-loss search, it must keep the Ex Machina branch intact and either try another route, report the missing film, or ask for help.
Record fallback as a different evidence path
A fallback substitutes another capability when the preferred one is unavailable or unsuitable. In the failed memory branch, lexical search may replace vector retrieval and find Eternal Sunshine of the Spotless Mind because its summary contains the rewritten query’s terms rather than because its embedding is nearby.
Record lookup can still validate that candidate against the same film summary, but the candidate reached validation through a different retrieval method. Production fallbacks across model providers, search indexes, or structured sources create the same kind of change: they preserve availability by altering the evidence path.
Silently presenting fallback output as equivalent hides the conditions under which the answer was produced. The execution record should retain the original error, the fallback rule that fired, the new source or method, and any changed validation threshold. An evaluator can then separate failures in the preferred path from failures in the recovered path.
Make repeated writes idempotent
Once record lookup validates Eternal Sunshine of the Spotless Mind, the model can complete the comparison and recommend it for the discussion group. If the user then asks to add that film to a shared watchlist, recovery crosses from reading data to changing it. Repeating the search cannot duplicate a change to the corpus, but retrying the watchlist call after its response is lost could add the film twice. To prevent that duplicate, the service must treat every retry as the same logical action rather than another addition. This property is idempotency.
The harness can attach a stable idempotency key such as request-42/add-eternal-sunshine to every attempt of that logical action. The receiving service stores the first successful result and returns it for later attempts with the same key. The harness may retry the call, but the application defines whether repeating it is safe.
Keep durable memory under a separate policy
The watchlist action and its idempotency key belong to the current trajectory. Carrying that entire record into a later request would give temporary work a lasting influence it was never meant to have. Durable memory serves a narrower purpose: it preserves selected information that may help in a future session. Combining the two makes temporary observations difficult to remove and gives old preferences more authority than the current request.
An old memory that the reader prefers action films cannot influence a new request until the harness can judge what that memory means now. Its source distinguishes something the reader stated from a preference the system inferred. Its age and expiry show whether it may still apply, while low confidence gives the harness a reason to ask for confirmation. Preserving that context also gives the reader something they can inspect, correct, or delete.
That provenance lets the harness resolve the current conflict deterministically. The explicit request for ambiguity outranks the older preference, which may still help with a future open-ended recommendation. If the conflict leaves the user’s intent unclear, the system can ask rather than quietly changing the task.
Some state should never cross this boundary as a preference. Tool permissions and approvals depend on the current identity, resource, and policy, so a later trajectory must establish them again. Retrieved film summaries remain evidence with a source. If a sentence from one of those summaries becomes a persistent instruction or preference, untrusted content has crossed into durable memory, creating a form of memory poisoning.
Keep recovery in the execution record
The recovered trajectory now shows that checkpoint v2 preserved Ex Machina, the timeout left the memory-loss branch open, and lexical fallback changed the path by which Eternal Sunshine of the Spotless Mind reached validation. It also shows the calls recovery skipped, the additional call it spent, and the idempotency key that keeps a repeated watchlist action from creating a second effect.
Those details make partial failure diagnosable and prevent a recovered answer from looking like a clean first attempt. The record preserves what the system verified, what failed, and which recovery rule allowed execution to continue.
One boundary remains. A stable, repeatable watchlist action may still be unauthorized, and remembered context cannot decide what a tool may change. Chapter 11 separates retrieved information, available capabilities, and the authority required to act. A later resume must also reauthorize saved evidence and memory under current identity and policy. Chapter 14 applies that rule to resumed state.