Runtime

The runtime is what turns a Program into something that executes: the host supplies bindings through a RuntimeEnv, Interpreter::load checks everything up front, and Interpreter::run evaluates the program on an input map. This page lists the full surface, plus include_program! for compiling a .dsrs file into a Rust binary.

use dspy_rs::ir::{Budget, Interpreter, RuntimeEnv};

let env = RuntimeEnv::new()
    .bind_model("fast", lm)
    .grant("net:search");

let interp = Interpreter::load(program, env).await?;
let out = interp.run(input, None, Budget::default()).await?;

`RuntimeEnv`

What the host supplies at load: live models, host tool and hole bindings, the sandbox, and the capability grants. Secrets never travel in the artifact; model clients are bound here from host-held keys and env vars.

Method	What it does
`RuntimeEnv::new()`	An empty environment.
`bind_model(name, lm)`	Pre-binds a live model by declared model name (for example `"fast"`). Models not bound here are constructed from their artifact config at load.
`bind_host_tool(name, tool)`	Binds a host tool implementation (a `rig` dyn tool) by tool name. Consulted once at load.
`bind_host_hole(name, f)`	Binds a native implementation for an extern hole by leaf name. The function receives the hole’s resolved input map and returns a JSON value coerced against the hole’s output signature.
`with_sandbox(executor)`	Sets the sandbox that executes holes and sandboxed tools (QuickJS in v1). Required if and only if the program carries sandboxed code.
`grant(cap)`	Grants one capability. The program’s `caps` must be a subset of the grants or the load is refused.
`with_code_mode(config)`	Behind the `code-mode` feature (on by default). When set, every `AgentLoop` presents its non-stop tools as one sandboxed `run_js` tool instead of N JSON tools; the model writes JavaScript that calls them as globals. This is a host presentation choice, not program semantics: the same artifact runs identically either way. See Code Mode.

`Interpreter::load`

Loading front-loads every check; nothing is lazy and nothing waits for call time. The checks run in this order:

Program::validate (the structural graph rules).
program.caps must be a subset of env.grants (no ambient authority).
Every model must be bindable: pre-bound by name, or client-constructible from its config.
Every ToolKind::Host tool name must be bound.
Every sandboxed tool and hole is registered through the full sandbox lifecycle (parse, compile, register). A hole that does not compile fails the load, not the call.

With code mode enabled there is one extra load-time refusal: two non-stop tool names in one loop that mangle to the same JS identifier.

`LoadError` variants

Variant	Plain words
`Invalid`	The program failed validation.
`CapsExceedGrants`	The program asks for capabilities the host did not grant; carries the missing set.
`Model`	A model could not be bound; carries the model name and the reason.
`HostToolUnbound`	A host tool name has no binding in the environment.
`HostHoleUnbound`	An extern hole’s leaf name has no binding in the environment.
`SandboxMissing`	The program carries sandboxed code but the environment has no sandbox executor.
`Register`	A piece of sandboxed code failed to register; carries the location and the underlying error.

`Interpreter::run`

Evaluates the program on an input map, reading parameters through an optional overlay (never mutating the program) and metering spend against a budget.

let out: JsonMap = interp.run(input, overlay, budget).await?;

input is a JSON object of the program’s input fields. It is checked against the program’s external signature: a missing required field or a type mismatch is RunError::Input.
overlay is Option<Arc<Overlay>>. When present, its base must equal the program’s hash or the run fails with RunError::Overlay before anything executes.
budget caps spend for this run.

`Interpreter::run_collecting`

run_collecting(input, overlay, budget) is run with per-leaf metadata: it returns a RunOutput — the same output map plus one LeafOutcome per successful Predict-leaf evaluation, in execution order (ForkJoin branches append in declared branch order). This is the seam Predict<S> uses to reassemble CallMetadata when it executes through the interpreter. Each LeafOutcome carries: name (the program-unique leaf name, the trace span component), raw_response, field_meta (per-field jsonish coercion flags and #[check] results, keyed by canonical field name), usage, model_config_hash, span_id (when a capture scope was active), and — for AgentLoop leaves — tool_calls and tool_executions. Scope rules: Predict and AgentLoop leaves report, Hole leaves do not; only successful evaluations report (a failed Retry attempt leaves no outcome, the succeeding one reports); an agent whose final output came from stop-tool args has empty field_meta; a replay-served leaf reports the recorded raw text and usage with empty field_meta.

`Interpreter::run_conversation`

run_conversation(chat, input, overlay, budget) is the conversation-in/conversation-out entry: one turn with the program’s single leaf over a caller-owned Chat, returning (RunOutput, Chat) — the turn’s output and metadata plus the extended conversation. It exists only for single-leaf programs (one predict or agent node, what Predict<S> compiles to); a multi-node graph is refused with RunError::Input.

// Opening turn: empty chat plus the typed input.
let (out, mut chat) = interp
    .run_conversation(Chat::new(vec![]), Some(input), None, Budget::unlimited())
    .await?;

// Continuation: append a follow-up and send the chat back.
chat.push_message(Message::user("are you sure?"));
let (out, chat) = interp.run_conversation(chat, None, None, Budget::unlimited()).await?;

The chat/input combinations: an empty chat with Some(input) renders the opening turn (system + demos + the formatted input, identical to conversation_opening); a non-empty chat with None is sent as-is; a non-empty chat with Some(input) appends the formatted input as the next user turn. A turn is not a run: each call records one trace span (seq increments per turn) and meters against its own budget. On an agent leaf the turn runs the full tool loop, dispatching tool calls through their bound executors. Replay works turn by turn — a span keys on the full chat sent, so a recorded conversation serves each turn with tool effects baked in. conversation_opening(&input, overlay) renders the opening Chat without calling anything: the same overlay-resolved system + demos (+ agent playbook) + input rendering a run would send. Use it to inspect or edit the first turn before run_conversation. Predict::build_chat/call_and_parse are thin wrappers over these two entries.

Caller-managed tool loops

run_conversation_caller_managed(chat, input, overlay, budget) is the same turn in suspending mode: when the model requests tool calls on an agent leaf, the loop suspends instead of dispatching and returns ConversationTurn::Suspended(ToolSuspension). Execute the calls yourself and feed the results back:

let mut turn = interp
    .run_conversation_caller_managed(Chat::new(vec![]), Some(input), None, Budget::unlimited())
    .await?;
while let ConversationTurn::Suspended(suspension) = turn {
    let results = run_my_tools(suspension.calls()).await; // Vec<String>, one per call
    turn = interp.resume_conversation(suspension, results).await?;
}
let ConversationTurn::Complete { run, chat } = turn else { unreachable!() };

ToolSuspension::calls() is the pending calls in request order; ToolSuspension::chat() is the conversation so far, including the assistant tool-call turn. resume_conversation records one ToolRun event per result (metering the time the suspension was outstanding), pushes one batched tool-result user turn, and continues the loop under the same meters and turn cursor — trace spans, budget metering, and stop-tool semantics are identical to dispatching mode. A stop-tool call completes the turn instead of suspending; a replay scope never suspends (served turns carry every tool effect); Code Mode does not apply, since the caller executes the tools. Dropping a suspension without resuming closes its span as Cancelled. Feed a failed tool’s error text as its result to keep the conversational-repair behavior of dispatching mode.

`Budget`

Run-level spend limits; None means unlimited. Budget::default() and Budget::unlimited() are the same: no limits.

Field	Type	Meaning
`max_lm_calls`	`Option<u32>`	Maximum LM calls for the run. Hard-gated before each call.
`max_tokens`	`Option<u64>`	Token ceiling. Soft: checked against accumulated usage, since usage is only known after a call.
`deadline`	`Option<Instant>`	Wall-clock cutoff. Hard-gated before each call.

An AgentLoop’s per-node budget chains a child meter under the run meter, so node spend also counts against the run.

`RunError` variants

Variant	Plain words
`Lm`	The provider call failed at a leaf.
`Parse`	The response arrived but did not parse as the signature outputs; carries the raw text.
`Tool`	A tool execution failed; carries the tool name and message.
`Hole`	A sandboxed hole failed to execute.
`CapabilityDenied`	Code asked for a capability the run does not permit.
`Budget`	The budget ran out at a leaf.
`Route`	A route port produced a value no arm (and no `else`) matches.
`Cancelled`	The run was cancelled.
`Overlay`	The overlay was minted against a different program hash.
`Input`	The run input was rejected against the program signature.
`Internal`	An interpreter invariant was violated.
`Replay`	A strict replay scope refused this call.

RunError::retryable() is true only for Lm, Parse, Tool, and Hole; those are the errors Retry and Refine may intercept. Budget and CapabilityDenied are never retried.

Ambient overlays

These two functions let #[module] executable functions pick up a candidate without threading it through every call.

with_ambient_overlay(overlay, fut) runs a future with an Arc<Overlay> as the ambient candidate for every #[module] function called on that task. Scoping is task-local: spawned subtasks do not inherit it, and nesting replaces the outer scope. The overlay’s base is checked by Interpreter::run against each module’s program, not here, so one scope can span calls into several modules and only the matching one accepts it.
current_overlay() returns the ambient overlay (Option<Arc<Overlay>>) if a with_ambient_overlay scope is active on this task. #[module]-generated functions read it immediately before Interpreter::run.

`default_lm`

The globally configured LM, used when a module does not name a model. default_lm() returns Option<Arc<LM>>: the model set through configure(...) in the global settings, or None when nothing was configured. Generated #[module] environments use it to bind the default model ref at load.

Embedding programs with `include_program!`

include_program! compiles a .dsrs file into your binary:

dspy_rs::include_program!("programs/qa.dsrs");

let program: &'static dspy_rs::ir::Program = qa::program();

The macro creates a module named after the file stem (qa.dsrs becomes mod qa). The path is resolved relative to your crate’s Cargo.toml directory. The generated module contains:

qa::SOURCE: the embedded text.
qa::program(): the parsed, validated program (panics on a bad file).
qa::try_program(): the same, but returns a Result.
A generated test, so cargo test fails if the file ever becomes invalid.

Validation is layered. Syntax is checked at macro expansion through dsrs-syntax — the shared .dsrs lexer and structural grammar both the macro and the full parser read from — so a malformed file breaks your build. Semantics (types, dataflow, capability rules) are checked by the full parser at first use of program(), and forced at CI time by the generated test — the sqlx-offline analogue. This is the shipping path for programs with host tools or host holes, which dsrs serve cannot bind: embed the file, bind your implementations with bind_host_tool and bind_host_hole, and serve from your own binary.

Get Started

Building Blocks

Programs as Data

Tools & Agents

Evaluation & Optimization

Utilities

`RuntimeEnv`

`Interpreter::load`

`LoadError` variants

`Interpreter::run`

`Interpreter::run_collecting`

`Interpreter::run_conversation`

Caller-managed tool loops

`Budget`

`RunError` variants

Ambient overlays

`default_lm`

Embedding programs with `include_program!`

See also

​RuntimeEnv

​Interpreter::load

​LoadError variants

​Interpreter::run

​Interpreter::run_collecting

​Interpreter::run_conversation

​Caller-managed tool loops

​Budget

​RunError variants

​Ambient overlays

​default_lm

​Embedding programs with include_program!

​See also

`RuntimeEnv`

`Interpreter::load`

`LoadError` variants

`Interpreter::run`

`Interpreter::run_collecting`

`Interpreter::run_conversation`

Caller-managed tool loops

`Budget`

`RunError` variants

Ambient overlays

`default_lm`

Embedding programs with `include_program!`

See also