Back to blog

Long-Running Agents Don't Just Have a Memory Problem. They Have an Authority Problem.

Better memory helps an agent reason. Authoritative execution state determines what happened, what is legal, and what the system must do next.

Tim OsterhusHistorical article
agentsmillraceruntimeresearch

Everyone keeps saying long-running agents need better memory.

They do. Larger context windows, compaction, retrieval, progress files, and structured project folders all help.

But memory is only part of the problem.

A transcript can tell an agent that it ran the tests. It cannot establish that the tests passed against the current code. A progress file can say a feature is complete. It cannot make that claim true. A chat message can record that a user approved an action. It cannot establish whether that approval still applies after a retry, replan, or crash.

Those are not memory questions.

They are authority questions.

Memory helps the model reason about what it has seen. Execution state represents what the runtime currently treats as established. Runtime authority determines which actions are legal and which transitions may become durable.

Authoritative does not mean infallible. The runtime must represent ambiguity explicitly rather than let the model resolve it silently.

Long-running agents need all three. The industry has spent much more time discussing the first.

A transcript is history, not execution state

Agent transcripts contain observations, tool calls, command results, failed attempts, overwritten files, stale assumptions, and several confident declarations that the work is now definitely finished.

Everything is in there.

That is the problem.

Before acting, the model has to reconstruct the current world from a chronological record of previous states. It must identify current files, successful commands, stale results, and failed attempts.

For a short task, a capable model can usually do this well enough. For a run spanning hundreds of actions, the transcript becomes an increasingly expensive substitute for a state machine.

A summary makes the record shorter. It does not necessarily make the record authoritative.

History is evidence about what happened. Execution state is the system’s current account of what happened.

That distinction sounds small until an agent repeats an expensive command or acts on a file that no longer exists. It also matters when an agent resumes an operation that already committed.

Ledger tested the distinction directly

On August 1, researchers published Turning Interaction History into Execution State. The paper introduces Ledger, a deterministic runtime layer for long-horizon coding agents.

Ledger wraps an otherwise unmodified agent. It converts completed interactions into an explicit record of what the agent observed, modified, and attempted.

It applies that record at two boundaries:

  • An inform path gives the model a compact view of current execution state before inference.
  • A govern path checks commands before execution, reuses results that remain valid, and flags likely redundant work.

The distinction between those paths matters. One helps the model reason. The other changes what the system permits or avoids executing.

Across all 500 SWE-bench Verified tasks, Ledger raised Pass@1 from 56.2% to 64.2% with GPT-5 mini. It raised Pass@1 from 75.8% to 81.0% with MiniMax M2.5. Total cost fell by 28.9% and 31.8%, respectively.

When attached to Codex, Ledger added 3.4 percentage points of Pass@1 while reducing cost by 24.4%.

Those results do not show that every agent needs Ledger. They show that explicit execution state can outperform repeated state reconstruction from raw history.

State without consequences is another note

An execution ledger becomes more useful when it affects execution.

If a system records that a command succeeded, the record must affect whether or how the next command executes. Otherwise, the record is useful for debugging. It is not governing the run.

If approval exists only in a transcript, a restarted agent can reinterpret it or request another token. It may also apply the approval to a different action.

This is the boundary between state and authority.

Authoritative state must have consequences. It must permit, refuse, route, pause, retry, or close work. Otherwise, the agent is still responsible for interpreting the record and deciding what happens next.

That can be fine for an interactive session. It becomes a bad default for unattended work that can modify production systems while everyone is asleep.

People do enjoy sleep, apparently.

Authorization also has to survive the run

Beyond Single-Use Tokens describes a failure mode called semantic replay.

Suppose a user approves one external action. The agent attempts it, receives an ambiguous result, replans, and requests a fresh single-use token. Each token remains single-use, yet the approved action can still execute more than once.

The bug does not live inside the token. It lives in the missing durable relationship between the approval, the canonical action, and the remaining execution budget.

The paper’s CapLease system persists that relationship. It uses transactional Issue, Prepare, and Commit transitions across retries, delegation, concurrency, confirmation replay, and crash recovery.

It is not enough to remember that an operator approved something. The runtime must identify the exact action, its version, remaining authority, and legal recovery behavior.

A wait record is not automatically an authorization protocol. Durable storage helps, but the stored fact also needs precise semantics.

That is not a minor implementation detail. It determines whether an agent can accidentally turn “do this once” into “do this once per recovery attempt.”

Very autonomous. Slightly more autonomous than requested.

Correct execution is not correct intent

A second August 1 paper formalizes a related boundary. Safety Invariants for Agents Orchestrating Irreversible State Transitions defines execution fidelity.

Under the paper’s fault model, the external effect must be either nothing or exactly the transition shown to the user, exactly once. The model includes planner mapping errors, ambiguous outcomes, retries, at-least-once delivery, and delegated callers.

Execution fidelity does not prove that the rendered transition matches the user’s actual intent. No runtime can mathematically determine that the human genuinely wanted a proposed action. The runtime can enforce a narrower guarantee: only the approved transition executes, and it executes at most once.

This is the same general boundary behind a rule I use in Millrace:

Model output is evidence, not runtime truth.

The model can propose an outcome. Millrace checks that the returned evidence came from the expected run. It then validates the marker and artifacts against the selected plan and applies only a legal route.

Graphs are useful because they can carry authority

AiFlow compiles a DSL or JSON definition into a directed streaming graph. Node guardians enforce queue bounds, concurrency, ordering, cancellation, and retries. The compiler validates type and concurrency compatibility before execution.

AiFlow solves a different problem. It focuses on streaming and backpressure, not durable control of multi-stage workflows and external runner sessions.

The relevant convergence is architectural:

workflow definition -> compilation -> validation -> runtime enforcement

The graph gives the runtime a concrete object to govern. A diagram does not provide authority. A graph that the model can ignore does not provide much either.

The useful question is whether the compiled graph determines what can execute.

Loops and graphs are not competing abstractions

Iris uses an inquiry-revision loop instead of organizing candidate improvement through chain, tree, or graph search. It maintains revisable task knowledge and probes decision-critical unknowns. On MLE-Bench, Iris reached a 64.9% any-medal rate under a 12-hour budget, the highest result among the systems compared in the paper.

It would be easy to summarize that result as “loops beat graphs.”

It would also be wrong.

Iris compares strategies for internal search and knowledge management. It does not test whether an external runtime should surrender execution authority to a model.

A loop can run inside one graph stage. A graph can route between specialized loops. The runtime can control when each loop starts, what state it receives, and what evidence permits continuation.

Loops govern local iteration. Graphs govern composition. Runtimes govern authority across both.

What this does and does not say about Millrace

None of these papers validates Millrace itself.

They do not evaluate Millrace. They do not reproduce its complete architecture. They do not establish that Millrace implements every invariant they describe.

Ledger operates mainly at the command-history layer inside a coding-agent run. Millrace governs the larger workflow around bounded calls to external runners. CapLease shows why approvals need durable limits. I need to examine that requirement directly in Millrace. The execution-fidelity paper proves a guarantee only under its own fault model and implementation. Similar architecture does not give Millrace that guarantee.

They independently support several design ideas:

  • Raw interaction history is a poor substitute for explicit execution state.
  • State matters when code uses it to control what happens next.
  • Approvals must survive retries, replanning, delegation, and crashes.
  • Model proposals should not directly commit irreversible state.
  • Compiled graphs can carry validated runtime rules.
  • A loop can still be the right local algorithm inside a larger workflow.

Millrace already implements the workflow-state and transition part of this architecture. It does not yet provide CapLease’s approval model, which ties approval to an exact action and use budget. It also does not generally reconcile uncertain effects in outside systems.

Its selected plan defines legal stages, routes, assets, runners, and outcomes. The runtime stores queues, runs, artifacts, waits, interventions, traces, and durable runner sessions. Those sessions use identity checks and fencing to stop stale attempts from taking over. For a normal agent stage, Millrace checks the returned evidence, validates its marker and artifacts, and applies the selected route.

That does not turn the papers into Millrace benchmarks.

Runtime engineering is where authority becomes executable

This gives us a more precise way to separate the layers of agent engineering:

  • Prompt engineering manages how the system frames a request.
  • Context engineering manages what information the model receives.
  • Harness engineering manages what actions the agent can take.
  • Loop engineering manages how execution continues.
  • Graph engineering manages how stages, outcomes, and transitions compose.
  • Runtime engineering manages which state is authoritative and which transition is legal now.

The layers overlap. They do not replace each other.

Better context can improve the model’s decision. It cannot enforce that one approval authorizes only one action. Better tool constraints can restrict dangerous actions. They do not, by themselves, determine whether a workflow satisfied its closure conditions. A loop can keep work moving. By itself, it cannot establish that the next pass is still allowed.

Those responsibilities need code outside the model.

The practical test

When evaluating a long-running agent system, ask:

  1. Does the system reconstruct current state from the transcript, or maintain explicit execution state?
  2. Which component decides what work is legal to run next?
  3. Can a restart resume from durable state without trusting a model-written summary?
  4. Does an approval identify a canonical action and track its remaining authority?
  5. Can the runtime distinguish failure, success, and an ambiguous external outcome?
  6. Does model output propose a transition, or commit the transition directly?
  7. Do retries, recovery, and closure follow runtime rules that the agent cannot silently reinterpret?

Long-running agents still need memory. They need good prompts, relevant context, useful tools, and enough intelligence to perform the assigned work.

But none of those things decides what the system is now allowed to do.

That is the authority problem.

And solving it is the point where an agent harness starts becoming a runtime.