4. Solution strategy
Five structural decisions carry the quality goals into the code. Each one is recorded as an architecture decision; this page states the choice and the reasoning, and links onward.
Tokio actors with message passing, no shared state
The runtime core is a set of Tokio actors. Each actor owns its state
exclusively and communicates only through bounded mpsc channels behind a
clonable handle. There is no Arc<Mutex<T>> shared between actors. This makes
it structurally impossible for one actor to mutate another's state, which is how
the "one actor, one responsibility" principle becomes an enforced property
rather than a convention. The trade-off is more message plumbing; the payoff is
the absence of a whole class of async deadlocks and data races.
See the execution model for the engine that runs on top of this.
Inference as a supervised sidecar
Local LLM inference runs in a separate process, not in the daemon: the embedded
llama-server (upstream llama.cpp), which the daemon spawns and speaks to over its
OpenAI-compatible HTTP API. Native tool calling is driven by the model's chat
template (--jinja), not a custom grammar path. Isolating inference keeps a model
crash or an out-of-memory event from taking the runtime down, and lets the runtime
speak to the engine over a narrow interface. Continuous batching decodes several
requests in the same pass, and tracking upstream llama.cpp widens the range of
supported model architectures.
The daemon supervises that process: it polls the liveness of each engine instance and respawns one that died, under a backoff that grows from one second to thirty. A model evicted on purpose is not brought back. See local inference.
A PyO3 bridge with trait-decoupled services
The code a package brings is Python; the runtime is Rust. The bridge is PyO3 with
pyo3-async-runtimes for async interop. The runtime does not hand Python raw
internals: it exposes a set of services (the ctx object) behind Rust traits,
so the contract is decoupled from the implementation and can be mocked for
testing. The type contract names seventeen services, and what a call receives
depends on what it is: a package tool runs with a context that carries no model,
no tool proxy and no memory, and a package workflow adds ctx.llm,
ctx.agents, ctx.workflows and ctx.user to it.
See stack and runtime and the agent contract; ctx is documented
in the SDK reference.
A machine contract for host integration
The runtime is meant to be driven by a host product, so its HTTP surface is a
first-class, stable contract, not an afterthought. The OpenAPI specification is
generated from the code, served by the daemon, and versioned (/api/v1, with
breaking changes reserved for a future /api/v2). Typed host SDKs in TypeScript
and Python are generated from that spec. An integrator drives a real daemon
without reverse-engineering anything.
See host integration. The generated surface is the HTTP API reference; the how-to is Integrate via the driving contract.
Governance lives in the runtime, not in the agent
The step budget is enforced by the runtime around every agent, not implemented by each agent, and it bounds every tool call whichever path it runs on. Recording goes to one register: a tool call a run announces is written, through its run-scoped events, to the hash-chained journal. The autonomy tier reaches an orchestrated run the same way as the budget, through the engine that plans and executes it.
Permission rules and human approval live in the same place. Every tool call an agent makes is decided before it runs by one governance decision, in a conversation, in a task, in a step of a plan and in an item of a batch alike: the organisation policy and a deny rule refuse it, a tool that only reads runs, and anything sensitive runs on a standing consent of the agent or asks a person. An agent is a record and holds no code, so it has no path around that decision. What the decision does not see into is the Python of a package: once a package tool is allowed to run, its code executes in the runtime process with the rights of that process, which is why installing a package is the act of trust. The agent trust model states that boundary. The whole picture is in Cross-cutting concepts and the accountability model.
The governing decisions are the permission model, human in the loop, and secrets and API authentication.