> ## Documentation Index > Fetch the complete documentation index at: https://dsrs.herumbshandilya.com/llms.txt > Use this file to discover all available pages before exploring further. # Optimizer Engine > Engine, OptimizeTarget, Candidate, Budget, RolloutCache, ScoreMatrix, and the Pareto and outcome types every optimizer shares The optimizer engine is the shared evaluation core under every optimizer. Strategies register candidates and ask the engine to evaluate them against an `OptimizeTarget`; the engine handles candidate binding, rollout fan-out, caching, budget accounting, and score bookkeeping. Every optimizer in DSRs (COPRO, GEPA, MIPROv2, SIMBA, bootstrap, Structural) is a thin strategy over this one core. There is **one** `Engine`. What varies is the target — the lane-erased pair of (thing under optimization, evaluation harness): | Lane | Constructor | Candidate currency | Winner | | --------------------------- | --------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------- | | Module lane | `OptimizeTarget::module(&mut module, &trainset, &metric)` | `Candidate` (name-keyed slots), injected *ambiently* per rollout via `fx::with_params` — never applied by mutation | Installed onto the module through `PredictorInfo::load_state`, once, by `OptimizeTarget::install` | | Program lane (`ir` feature) | `OptimizeTarget::program(&interp, &examples, &metric)` | `ir::Overlay` (or a `Candidate` bound through `fx::Params::bind`), read through at render time | Retrievable as an `Arc` via `OptimizeTarget::winner_overlay`, for `Program::bake` | Because candidate injection is ambient in both lanes — nothing is ever applied to shared state during evaluation — rollouts for *different candidates* share one bounded-concurrency fan-out. All items on this page are exported from the crate root except where a feature gate is noted. ## `OptimizeTarget<'a>` The thing an optimizer optimizes: a module or a program, packaged with its example set (by reference) and metric. | Method | What it does | | ------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `module(module, trainset, metric)` | Module-lane target: a typed `Module + Predictors`, a `&[E]` trainset (`E: ToInput + Serialize`), and a `TypedMetric`. Runs the **naming pass**: every leaf declared via `Predictors` is stamped with its declared name (`PredictorInfo::set_trace_name`), so trace spans, candidate entries, and persistence all address the same names. | | `module_with_valset(module, trainset, valset, metric)` | Same, with an optional validation set. When `Some`, the valset examples become the *leading* columns and the trainset the trailing ones — the layout GEPA's Pareto bookkeeping uses. | | `program(interp, examples, metric)` | Program-lane target: an interpreter-loaded `Program`, labeled `DemoRow` examples, and a `ProgramMetric`. | | `leaves() -> &[LeafInfo]` | The optimizable leaves' read surface, snapshotted at construction: per leaf, `name`, current `instruction`, `default_instruction`, `demos` as flat JSON rows, and `input_fields`/`output_fields` as `(lm name, docs)` pairs. `LeafInfo::schema_for_reflection()` renders the field contract for reflection prompts. | | `num_examples()`, `has_valset()` | Example-set access. | | `val_columns()`, `train_columns()` | The scoring columns (validation prefix, or every example) and the minibatch-sampling pool (trainset suffix, or every example). | | `install(&winner)` | Installs the winning `Candidate` — the **one** mutation of the run. Module lane: merges each slot into the named leaf's state through `PredictorInfo::load_state`. Program lane: binds the winner to an overlay. | | `winner_overlay()` | The installed winner as a bound `Arc` (program lane only). | | `candidate_outputs(indices, &candidate)` | Runs the given examples under the candidate and returns bare output values, no metric and no trace capture — GEPA's best-output collection. | `ProgramMetric` is the JSON-native sibling of `TypedMetric`: loaded programs have no static output type, so the metric scores the interpreter's output `JsonMap` against a labeled `DemoRow`. ```rust theme={null} pub trait ProgramMetric: Send + Sync { async fn evaluate( &self, example: &DemoRow, output: &JsonMap, trace: Option<&Trace>, ) -> Result; } ``` ## `Engine` Owns the candidate registry, the score matrix, the rollout cache, and the budget meter. Strategies register candidates and call the evaluate methods against a target; the engine handles binding, fan-out, caching, accounting, and bookkeeping. | Method | What it does | | ----------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `new(config)` | Builds an engine from an `EngineConfig`. Examples and metric live on the target, not the engine — one engine can serve successive targets (a `Box` pipeline sharing one budget). | | `evaluate_many(target, candidates, subset)` | Evaluates N registered candidates over `subset` example indices (`None` = the target's full set) in **one** bounded-concurrency fan-out — candidate-level parallelism in both lanes. Cached rollouts return their `Eval` with `trace: None` and consume no budget. Returns `BatchEvalOutcome`. | | `evaluate(target, candidate, subset)` | Single-candidate convenience over `evaluate_many`. Returns `EvalOutcome`. | | `evaluate_gated(target, candidate, minibatch, threshold)` | The minibatch gate (the GEPA/SIMBA acceptance pattern): evaluates on `minibatch`; only a minibatch mean strictly greater than `threshold` promotes to a full-set evaluation. Returns `GateOutcome`. | | `register(candidate)` | Registers a module-lane `Candidate`, deduplicating by content hash. Returns its index; a duplicate returns the existing index. | | `register_overlay(overlay)` | Registers a program-lane `ir::Overlay`, deduplicating by `Overlay::hash()`. Returns its index. | | `candidate(i) -> Option<&Candidate>`, `candidate_hash(i)`, `num_candidates()` | Candidate registry access (`candidate` is `None` for an overlay entry). | | `config()`, `spend()`, `matrix()`, `cache()` | State access. | | `pareto()`, `pareto_over(columns)` | Dominance views over the score matrix (all columns, or a subset). | | `budget_allows(n)` | Whether `n` more rollouts fit the remaining budget. | | `charge(metric_calls, lm_calls)` | Charges auxiliary spend the engine did not run itself: reflection LM calls, teacher passes. | | `peak_candidate_concurrency()` | High-water mark of *distinct candidates* with rollouts in flight simultaneously — the parallelism gauge. | The metric runs outside the trace capture scope, so LM-as-judge metrics do not pollute the execution trace. If the uncached portion of a batch does not fit the remaining budget, the engine runs nothing, leaves spend unchanged, and returns `BudgetExhausted`. ## `EngineConfig` | Field | Default | Meaning | | -------------------- | --------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | | `concurrency: usize` | `16` (`DEFAULT_EVAL_CONCURRENCY`) | Rollouts in flight at once within one evaluation batch. | | `budget: Budget` | `Budget::unlimited()` | Hard spend caps; the engine stops cleanly when a batch would not fit. | | `cache_salt: u64` | `0` | Folded into every cache key. Bump it when changing LM sampling settings outside the candidate; sampling params are not part of candidate identity. | ## Candidates A `Candidate` is data: `slots: BTreeMap` mapping leaf name (the `Predictors` contract name) to a partial per-leaf configuration, plus a stable content hash. It is cheap to clone, serializable, and **never applied by mutation**: the engine scopes it ambiently around each rollout (`fx::with_params`); the single mutating step is the caller-driven final `OptimizeTarget::install`. The empty candidate (`Candidate::default()`) is the baseline, the module exactly as it is. | Item | Signature or fields | | --------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `CandidateSlot` | `instruction: Option`, `clear_instruction: bool`, `demos: Option>`. Unset fields leave the leaf's incumbent value untouched; `clear_instruction` explicitly resets to the signature default, winning over any instance override. Demo rows are flat JSON objects, input and output fields merged. | | `Candidate::new()` | The empty candidate. | | `Candidate::with_instruction(name, text)` | Single-predictor instruction candidate, the COPRO and MIPRO case. | | `set_instruction(name, text)`, `clear_instruction(name)`, `set_demos(name, rows)` | Builder-style mutators. `set_demos` with an empty vec clears the demo set. | | `instruction_of(name)`, `demos_of(name)` | Read accessors. | | `is_empty()`, `stable_hash()` | The hash is canonical: identical content hashes identically across processes and map orderings. It is the cache identity. | | `to_params()` | Converts to the ambient-injection currency: name-keyed [`fx::Params`](/docs/components/fx) with explicit clears preserved. `fx::Params::bind(program)` turns the same value into an `ir::Overlay` for the program lane. | `CandidateSlot` (instruction plus demos per leaf name) is a different type from the IR `ir::Overlay`, which maps `ParamId` to `ParamValue` over a compiled `Program`. The program lane consumes the IR type; `Candidate::to_params()` + `Params::bind` is the bridge between them. ## `Budget` and `Spend` `Budget` sets hard caps on evaluation spend; `None` means unlimited. | `Budget` field | Metered as | | --------------------------------- | ------------------------------------------------------------------------------------------------ | | `max_metric_calls: Option` | One per executed rollout. Cache hits do not re-run the metric. | | `max_lm_calls: Option` | One unit per executed rollout, plus auxiliary charges via `charge`. | | `max_tokens: Option` | Checked against recorded token usage; the engine refuses the next batch once the cap is reached. | `Budget::allows(&spend, upcoming_rollouts)` reports whether the batch fits. Zero upcoming rollouts always fit, so cache-only batches never stall. Call and metric caps are enforced prospectively (the batch must fit under the cap); the token cap is retrospective (a batch may overshoot, and the following batch is refused). `Spend` is what the engine has consumed so far: | `Spend` field | Meaning | | --------------------- | -------------------------------------------------------- | | `metric_calls: usize` | Metric evaluations executed. | | `lm_calls: usize` | LM call units: executed rollouts plus auxiliary charges. | | `lm_spans: usize` | Exact `Predict` spans observed across captured traces. | | `cache_hits: usize` | Rollouts served from the cache instead of executed. | | `tokens: LmUsage` | Token totals summed from captured span usage. | This `Budget` is not the IR runtime `Budget` documented on [Runtime](/docs/components/runtime). The runtime type caps one program execution: `max_lm_calls: Option`, `max_tokens`, and a `deadline: Option`, enforced call by call inside the run by a `BudgetMeter` with parent chaining for nested agent loops. The optimizer type caps an entire optimization search across all rollouts: it adds `max_metric_calls`, has no deadline, and is checked per batch before anything runs, with consumption tracked in `Spend`. The program lane runs each rollout under the runtime `Budget::unlimited()` while the optimizer `Budget` governs the batch. ## `RolloutCache` In-memory map from a rollout key to its `Eval` (score plus optional feedback). A candidate re-evaluated on a seen example returns the cached `Eval` with no LM call and no metric call. The key recipe is `(baseline, candidate, example, salt)`, formatted as four 16-digit hex hashes joined by colons: | Component | Module lane | Program lane | | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------- | | `baseline` | Content hash of the `predictors()` state snapshot (`{name → PredictState}`), computed once at target construction. Installing a winner and building a new target yields a new baseline, invalidating stale entries. | `program.meta.program_hash`. | | `candidate` | `Candidate::stable_hash()` | `Overlay::hash()` | | `example` | Content hash of the example | Content hash of the `DemoRow` | | `salt` | `EngineConfig::cache_salt` | `EngineConfig::cache_salt` | Public surface: `get`, `insert`, `len`, `is_empty`. ## Score bookkeeping **`ScoreMatrix`** is a per-instance matrix of candidates (rows, registration order) by examples (columns). Cells are `None` until scored. Methods: `new(columns)`, `candidates()`, `examples()`, `ensure_rows(n)`, `record(candidate, example, score)`, `score(candidate, example)`, `row(candidate)`, `mean(candidate)`, `best_by_mean()`, `pareto()`, `pareto_over(columns)`. The column-restricted view supports GEPA-style setups where train and validation examples share one matrix. **`ParetoView`** is a dominance snapshot computed from the matrix. `best_scores()` gives the best score per viewed column; `wins(candidate)` counts columns the candidate wins or ties on (tolerance `1e-6`); `frontier()` lists candidates winning on at least one column; `statistics()` summarizes. A candidate with zero wins is dominated. GEPA samples parents proportional to their Pareto coverage directly from this view. **`ParetoStatistics`** fields: `num_candidates`, `num_examples_covered`, `avg_coverage: f32`, `max_coverage`, `min_coverage`. A healthy search grows `num_candidates` slowly while `avg_coverage` rises; `num_candidates == 1` means the search has collapsed. ## Outcome types | Type | Shape | | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `RolloutOutcome` | `example: usize`, `eval: Eval`, `trace: Option`. The trace is `None` when the rollout was served from the cache. | | `CandidateEval` | `candidate: usize`, `rollouts: Vec` in request order. `mean()` is the arithmetic mean over the batch (`0.0` when empty); `scores()` collects the raw scores. | | `EvalOutcome` | `Complete(CandidateEval)` or `BudgetExhausted { needed }`. When exhausted, nothing ran and spend is unchanged; `needed` is the uncached rollout count. `completed()` converts to `Option`. | | `BatchEvalOutcome` | `Complete(Vec)`, one per requested candidate in request order, or `BudgetExhausted { needed }`. `completed()` converts to `Option>`. | | `GateOutcome` | `BudgetExhausted { needed }`, `Rejected { minibatch }`, or `Promoted { minibatch, full }`. | ## Rollout mechanics Each program-lane rollout runs `interp.run(input, Some(overlay), Budget::unlimited())` under its own capture scope with `TraceMeta.candidate_hash` set to the candidate's hash and a `program` tag carrying the program hash. Each module-lane rollout scopes the candidate's `fx::Params` ambiently around the whole traced rollout (`rollout_traced`), so every `Predict` leaf binds its own entry at call time. After the metric scores a module-lane rollout, any span-level evals it returns from `TypedMetric::evaluate_spans` are stamped onto the trace's spans; demo-harvesting optimizers prefer these over the rollout score (see [Evaluation](/docs/components/evaluation)). Every pending `(candidate, example)` pair — across all candidates, in both lanes — joins one `buffer_unordered` stream bounded by `EngineConfig::concurrency`. ## See also * [Optimizers](/docs/components/optimizers) * [Evaluation](/docs/components/evaluation) * [Runtime](/docs/components/runtime) for the IR `Budget`, `BudgetMeter`, and `Interpreter::run` * [Traces](/docs/components/traces) for `Trace`, `TraceMeta`, and `Eval` * [Program and nodes](/docs/components/program-and-nodes) for `Program` and `ir::Overlay` * [COPRO](/docs/optimizers/copro), [MIPROv2](/docs/optimizers/miprov2), [GEPA](/docs/optimizers/gepa) * Example: [04-optimize-hotpotqa.rs](https://github.com/krypticmouse/DSRs/blob/main/crates/dspy-rs/examples/04-optimize-hotpotqa.rs) * Example: [08-optimize-mipro.rs](https://github.com/krypticmouse/DSRs/blob/main/crates/dspy-rs/examples/08-optimize-mipro.rs) * Example: [09-gepa-sentiment.rs](https://github.com/krypticmouse/DSRs/blob/main/crates/dspy-rs/examples/09-gepa-sentiment.rs)