The Research Is Independently Reconstructing Millrace

I started building Millrace before I had the right language for what it was.

For a while, I called it an orchestration framework. That was technically true in the same way that a gym is technically a room with heavy objects in it. It described something observable without explaining the actual point.

Then loop engineering became a category, and Millrace made more sense. It was built around governed loops. I looked across the public landscape and realized Millrace was solving the next problem: reliably automating the management of multiple loops.

That was when I started calling the layer “Runtime Engineering.”

I ended up being half-wrong, because the next big thing was actually called “Graph Engineering.”

But then I realized I was half-right, because runtime engineering still sat a layer above graphs.

How else are you going to run your graphs, if not through a runtime?

So I kept looking to see if anyone else was converging on the same thesis.

The more interesting evidence did not come from competing products. It came from research papers. Separate groups studying workflows, harnesses, safety, long-horizon agency, routing, and tracing were arriving at individual pieces of the same architecture.

To be precise, none of these papers validates Millrace as a complete product. None even mention it. No broad study has compared the complete architecture (let alone Millrace itself) against every plausible alternative, and no single paper reproduces everything Millrace does.

That qualification sounds like a caveat, but it’s actually the main point of this article.

No single paper independently reconstructs Millrace. The combined literature reconstructs nearly every major decision in its architecture.

The research converged around six principles.

1. Prose is not reliable procedural authority

Agents are very good at producing an output that looks complete. They are much less reliable at following every required step that should have produced it.

SIGIL measured this directly across 30 skills. Prose agents executed 56% of mandated steps with GPT-4o and 68% with GPT-5, even while producing artifacts that passed output checks.

The researchers then compiled those skills into typed executable harnesses. Compiled execution held at 86% across both models. Full procedures were completed 2.3 times as often, using 0.58 times the median tokens.

That comparison matters because it separates two things people commonly blur together: generating an acceptable artifact and executing an acceptable process.

COVENANT attacks the same problem from the workflow side. It treats natural-language workflow instructions as source programs instead of prompt context. The system compiles them into an abstract syntax tree and control-flow graph, then checks proposed actions before committing state or advancing.

In its strongest evaluated configuration, COVENANT raised success from 50.00% to 83.33% and reduced attributed workflow misalignment from 42.50% to 15.83%. Across all 3,000 paired instances, success rose from 43.20% to 73.10%.

Both papers reinforce one of Millrace’s foundational decisions: a hard requirement cannot remain merely a suggestion inside the model’s context.

If an action is mandatory, the system should represent it as structure. If an action is forbidden, the system should not expose it to begin with. If the model is allowed discretion, that discretion should exist inside a bounded decision point.

Prompts can explain the procedure, but they should not be the only things enforcing it.

2. Procedure needs to become an executable artifact

Once prose stops being the authority, something else has to take its place.

The recurring answer is an executable representation of the workflow.

AgentSPEX replaces implicit reactive control flow with typed steps, branches, loops, parallel execution, reusable modules, and explicit state. The workflows execute inside a harness that provides tools, sandboxing, checkpointing, verification, and logging.

CodeSpec reaches a similar conclusion for long-horizon software development. It compiles repository evidence and requirements into executable architecture and behavior specifications that can keep checking the work throughout implementation. On FeatureBench Lite, CodeSpec passed 70.7% of fail-to-pass tests and fully resolved 9 of 30 tasks, outperforming same-backbone baselines on both measures.

Even the field’s vocabulary is moving in this direction. What Makes Prompts a Graph proposes four conditions for prompt graph engineering:

• explicit structure
• separation between structure and prompt content
• executable semantics
• a first-class graph artifact that can be inspected and versioned

While that paper is only a taxonomy, lacking experimental proof, it still formalizes an architectural shift that is already happening in practice.

Millrace takes that same shift all the way to its logical conclusion. A sufficiently specified workflow becomes graph data. The compiler validates it. The runtime executes the legal transitions. Prompts, tools, proof requirements, permissions, and recovery behavior attach to stages instead of floating around as informal expectations.

The workflow stops being something the agent repeatedly interprets from scratch.

It becomes a thing the system can operate.

3. The model should not own traversal

This is the line separating a sophisticated loop from a runtime.

In a normal agent loop, the model does the work and decides what should happen next. It interprets the procedure, chooses a branch, updates its own account of state, and often decides when it is finished.

That is an impressive amount of authority to hand to the component most likely to declare victory confidently. The tests it forgot to run did not fail, after all.

COVENANT draws a cleaner boundary. The model proposes. The controller checks the proposal against the compiled workflow, verifies the required evidence, commits state, and advances the graph.

Agent libOS makes the same boundary explicit at the operating-substrate level. It represents agents as processes with identities, queues, budgets, checkpoints, capabilities, tools, memory, and child processes. Model-visible affordances can evolve, but resource authority changes only through explicit audited runtime primitives.

In its deterministic 27-task benchmark, the prototype completed every task plan while preventing all modeled unauthorized side effects, with a 7.0% conservative false-denial rate. Simple wrapper and sandbox baselines completed the tasks but failed most safety checks.

Bridging Protocol and Production reinforces the boundary from another direction. MCP can connect an agent to tools, but connectivity does not provide identity propagation, execution budgets, structured error semantics, timeout handling, or deterministic recovery. Those responsibilities belong to programmatic infrastructure around the model.

This is the core Millrace split:

Agents do the work. The runtime owns the work lifecycle.

The model can reason, write, inspect, critique, and propose. It should not quietly become the final authority over state transitions merely because it produced a persuasive paragraph about why the next step seems reasonable.

4. Long-running work needs durable external state

You can keep a short task inside one context window. Long-running workflows do not stay short merely because that would be convenient.

They cross context windows. They get interrupted. Machines restart, tools time out, humans take hours to approve something. Another run needs to resume without reconstructing reality from a compacted chat summary written by the same agent that got confused.

Towards Long-Horizon Agents calls reliable long-horizon agency a system property, not a byproduct of a better base model. Its framework explicitly separates intra-context work from cross-context work, where agents must externalize state, checkpoint, hand off, and resume.

Agent libOS supplies concrete runtime primitives for that environment: persistent state, message queues, budgets, checkpoints, restore and fork operations, human approval queues, and auditability.

The United Nations University report on engineering and governing agent harnesses reaches a similar conclusion from a technology-policy perspective. It recommends treating the harness as a distinct runtime object and making state-changing execution interruptible, resumable, and reversible. It also calls for scoped permissions, explicit approval gates, trajectory retention, and recovery procedures.

Again, this report is not experimental validation. It is evidence that runtime governance is becoming legible as its own operational layer.

Millrace stores queues, snapshots, stage results, run artifacts, evidence, and recovery state outside the model. A stage can begin with fresh context because the runtime does not depend on one immortal chat thread remembering everything correctly.

The agent’s context is a work surface.

The database needs to exist elsewhere.

5. Completion is not a final answer

This might be the most important convergence of the lot.

AgentS4D evaluated 6,560 runs across 328 deliberately risk-injected cases and 20 harness-model combinations. It found that 68.0% triggered prespecified unsafe signals. More strikingly, 66.22% of all runs were both unsafe and complete.

In other words, successful task completion did not establish that the execution was safe.

SecRespond found the complementary failure in incident response. Across 23 frontier models and ten compromised environments, no model achieved complete detection and remediation in even one environment. Agents found obvious alerted problems but missed silent intrusions and produced incomplete remediation plans.

Useful work happened, but the material obligations remained unresolved.

Those findings point toward a stronger definition of closure. A runtime should establish separately that:

• the required work was completed
• the permitted path was followed
• actions were authorized
• side effects were correct
• the required evidence exists
• no material obligations or exceptions remain open

AgentTrace supports the observability side of this principle. It reconstructs causal graphs from execution logs and traces failures backward through dependencies. That does not provide prospective control. It does show why graph-structured lineage beats scrolling through a heroic quantity of conversational sludge after something breaks.

Millrace treats closure as a runtime verdict based on evidence and unresolved obligations. An agent saying “done” is an input to that decision, not the decision itself.

6. Evolution needs governance too

Compiled graphs should not be confused with permanently frozen workflows.

Real systems need to improve. Plans change. A failure reveals a missing stage. A different model can reliably handle more work at once. New evidence can justify a new route.

The question is not whether the graph can evolve. The question is who can change it, under what conditions, and what happens before the new version becomes authoritative.

Self-Harness mines execution failures, proposes minimal harness changes, and accepts them only after regression testing. Held-out Terminal-Bench pass rates improved across all three model families. The reported gains were 40.5% to 61.9%, 23.8% to 38.1%, and 42.9% to 57.1%.

MANTA applies bounded inference-time changes to multi-agent topology. It can modify roles, communication links, execution order, information visibility, and validation pathways while preserving the task interface and agent budget. It reported a 74.0 average score, 5.8 percentage points above its strongest baseline.

GraphPlanner adds historical workflow memory to graph-based agent routing. In Phase 2, it reported a 9.3-percentage-point average accuracy gain over the strongest baseline.

These papers do not argue that an executing agent should freely rewrite its own governor whenever it feels inspired. They show that harnesses and topologies can improve when evolution is treated as a separate controlled process.

For Millrace, that suggests a clear pattern. The runtime can allow agents to propose graph amendments, but those amendments should be bounded by policy, compiler-validated, versioned, tested, auditable, and explicitly promoted.

Self-improvement should produce a candidate artifact.

It should not produce a quiet coup.

The combined architecture is the finding

Taken separately, none of these papers validates Millrace.

SIGIL validates compiled procedures, but it is not a persistent workflow runtime. COVENANT validates controller-owned graph traversal, but it does not cover Millrace’s full cross-session lifecycle. Agent libOS validates durable process and capability primitives, but it does not provide Millrace’s compiled cross-stage workflow semantics. AgentS4D validates trajectory-level safety evaluation, but it is a benchmark, not an execution system.

The same boundaries apply to the rest. AgentSPEX is primarily a specification language and harness. CodeSpec focuses on one software-development chain. GraphPlanner optimizes routing. AgentTrace diagnoses failures after execution. SecRespond exposes incomplete closure in one consequential domain.

They are not smaller Millraces.

That is why the combined pattern matters.

Put the findings together and you get:

• compiled control graphs
• runtime-owned traversal
• durable state and restart recovery
• specialized stage harnesses
• capability and approval boundaries
• evidence-controlled commits
• failure and repair loops
• versioned plan evolution
• obligation-complete closure

That list is not a proposal for some future version of Millrace. It is a compact description of the architecture Millrace has been converging toward in public.

No single paper assembles the whole thing. COVENANT, SIGIL, Agent libOS, and AgentS4D collectively come closest, each reconstructing a different load-bearing part.

So the claim is not that academia has proven Millrace is the universally best architecture for every agent workflow. It has not. Normal agents remain better for one-shottable tasks, exploratory work, and workflows that need constant human judgment. There is also no head-to-head study of Millrace against every alternative.

The narrower claim is still substantial:

Independent research programs are documenting the same failure modes Millrace was built around and converging on nearly every major mechanism Millrace uses to address them.

The research is not catching up to the Millrace brand. It is catching up to the shape of the problem.

Millrace just happened to start assembling the full answer early.

A practical runtime test

If you are evaluating a long-running agent system, ask:

• Is the procedure an inspectable executable artifact, or only prose in context?
• Does code own legal state transitions, or does the model decide what happens next?
• Can the system resume from durable state after interruption without trusting a chat summary?
• Are permissions, budgets, approvals, and side effects governed outside the model?
• Does completion require evidence and resolved obligations?
• Can failures enter a defined repair path without silently advancing the workflow?
• Can workflow changes be proposed, tested, versioned, audited, and promoted separately from execution?

If the answer to the first question is no, you probably have a loop, not a graph. Potentially a very good loop.

If the answer to most of these questions is no, you do not have a runtime yet.