> ## Documentation Index > Fetch the complete documentation index at: https://dsrs.herumbshandilya.com/llms.txt > Use this file to discover all available pages before exploring further. # Optimizer Engine > EvalEngine, ProgramEvalEngine, Candidate, Budget, RolloutCache, ScoreMatrix, and the Pareto and outcome types every optimizer shares The optimizer engine is the shared evaluation core under every optimizer. Strategies register candidates and ask the engine to evaluate them; the engine handles overlay application, rollout fan-out, caching, budget accounting, and score bookkeeping. Every optimizer in DSRs (COPRO, GEPA, MIPROv2, bootstrap) is a thin strategy over this core, which exists in two lanes: | Lane | Engine | Candidate type | Parallelism | | --------------------------- | ------------------- | ---------------------------------------------------------------------------- | -------------------------------------------------------- | | Module lane | `EvalEngine` | `Candidate` (named overlays), applied and restored through the mutation seam | Examples within one candidate; candidates are serialized | | Program lane (`ir` feature) | `ProgramEvalEngine` | `ir::Overlay`, read through at render time, never applied | Candidates and examples together in one fan-out | Both lanes share the same `EngineConfig`, `Budget`, `Spend`, `RolloutCache`, `ScoreMatrix`, Pareto bookkeeping, outcome types, and minibatch gate. All items on this page are exported from the crate root except where a feature gate is noted. ## `EvalEngine<'m, E, MT>` Owns the example set, the candidate registry, the score matrix, the rollout cache, and the budget meter. `E` is the trainset [row type](/docs/components/data); construction requires `E: Serialize`. Strategies register candidates and call `evaluate` or `evaluate_gated`; the engine handles application, fan-out, caching, accounting, and bookkeeping. | Method | What it does | | ----------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `new(examples, metric, config)` | Builds an engine over `Vec`, a `&MT` metric, and an `EngineConfig`. Example uids are content hashes of the whole row. | | `evaluate(module, candidate, subset)` | Evaluates one registered candidate over `subset` example indices (`None` means the full set). Applies the overlay, fans out uncached rollouts with bounded concurrency under per-rollout trace capture, restores the module, records scores. Returns `EvalOutcome`. | | `evaluate_gated(module, candidate, minibatch, threshold)` | The GEPA acceptance pattern: evaluates on `minibatch`; only a minibatch mean strictly greater than `threshold` promotes to a full-set evaluation. Returns `GateOutcome`. | | `register(candidate)` | Registers a `Candidate`, deduplicating by content hash. Returns its index; a duplicate returns the existing index. | | `candidate(i)`, `candidate_hash(i)`, `num_candidates()` | Candidate registry access. | | `examples()`, `num_examples()`, `config()`, `spend()`, `matrix()` | State access. | | `pareto()`, `pareto_over(columns)` | Dominance views over the score matrix (all columns, or a subset). | | `budget_allows(n)` | Whether `n` more rollouts fit the remaining budget. | | `charge(metric_calls, lm_calls)` | Charges auxiliary spend the engine did not run itself: reflection LM calls, teacher passes. | | `checkpoint()` | Serializes engine state (example uids, candidates, matrix, cache, spend) to a JSON string, format version 1. | | `resume(examples, metric, config, checkpoint)` | Rebuilds an engine from a checkpoint. Fails if the version or the example set does not match. Completed rollouts are served from the restored cache instead of re-executing. | `evaluate` requires `E: ToInput + Sync`, `M: Module + Facet`, and `MT: TypedMetric`. The metric runs outside the trace capture scope, so LM-as-judge metrics do not pollute the execution trace. **Concurrency model.** Candidates mutate shared predictor state, so the engine serializes candidate application and parallelizes across examples within one candidate (bounded by `EngineConfig::concurrency`). Candidate-level parallelism requires overlays resolved at render time; that is the program lane below. ## `EngineConfig` | Field | Default | Meaning | | -------------------- | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | | `concurrency: usize` | `16` (`DEFAULT_EVAL_CONCURRENCY`) | Rollouts in flight at once within one fan-out. | | `budget: Budget` | `Budget::unlimited()` | Hard spend caps; the engine stops cleanly when a batch would not fit. | | `cache_salt: u64` | `0` | Folded into every cache key. Bump it when changing LM sampling settings outside the candidate; sampling params are not yet part of candidate identity. | ## Candidates and the mutation seam A `Candidate` is data: `overlays: BTreeMap` mapping predictor name to a partial parameter update, plus a stable content hash. The empty candidate (`Candidate::default()`) is the baseline, the module exactly as it is. | Item | Signature or fields | | ------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `Overlay` | `instruction: Option`, `demos: Option>`. `None` leaves the current value untouched. Demo rows are flat JSON objects, input and output fields merged. | | `Candidate::new()` | The empty candidate. | | `Candidate::with_instruction(name, text)` | Single-predictor instruction candidate, the COPRO and MIPRO case. | | `set_instruction(name, text)`, `set_demos(name, rows)` | Builder-style mutators. | | `is_empty()`, `stable_hash()` | The hash is canonical: identical content hashes identically across processes and map orderings. It is the cache and checkpoint identity. | | `CandidateUndo` | Opaque snapshot of pre-overlay `PredictState` for every predictor the candidate touched. | | `apply_candidate(&mut module, &candidate) -> Result` | The one place candidate state is written, through `DynPredictor::apply_update`. If any overlay fails to apply, the overlays applied so far are rolled back before the error returns. | | `restore_candidate(&mut module, undo) -> Result<()>` | Restores the saved state. Attempts every predictor even if one fails, then reports the first error. | This module-lane `Overlay` (instruction plus demos per predictor) is a different type from the IR `ir::Overlay`, which maps `ParamId` to `ParamValue` over a compiled `Program`. The program lane below consumes the IR type. ## `Budget` and `Spend` `Budget` sets hard caps on evaluation spend; `None` means unlimited. | `Budget` field | Metered as | | --------------------------------- | ------------------------------------------------------------------------------------------------ | | `max_metric_calls: Option` | One per executed rollout. Cache hits do not re-run the metric. | | `max_lm_calls: Option` | One unit per executed rollout, plus auxiliary charges via `charge`. | | `max_tokens: Option` | Checked against recorded token usage; the engine refuses the next batch once the cap is reached. | `Budget::allows(&spend, upcoming_rollouts)` reports whether the batch fits. Zero upcoming rollouts always fit, so cache-only batches never stall. Call and metric caps are enforced prospectively (the batch must fit under the cap); the token cap is retrospective (a batch may overshoot, and the following batch is refused). `Spend` is what the engine has consumed so far, serialized into checkpoints: | `Spend` field | Meaning | | --------------------- | -------------------------------------------------------- | | `metric_calls: usize` | Metric evaluations executed. | | `lm_calls: usize` | LM call units: executed rollouts plus auxiliary charges. | | `lm_spans: usize` | Exact `Predict` spans observed across captured traces. | | `cache_hits: usize` | Rollouts served from the cache instead of executed. | | `tokens: LmUsage` | Token totals summed from captured span usage. | This `Budget` is not the IR runtime `Budget` documented on [Runtime](/docs/components/runtime). The runtime type caps one program execution: `max_lm_calls: Option`, `max_tokens`, and a `deadline: Option`, enforced call by call inside the run by a `BudgetMeter` with parent chaining for nested agent loops. The optimizer type caps an entire optimization search across all rollouts: it adds `max_metric_calls`, has no deadline, and is checked per batch before anything runs, with consumption tracked in `Spend`. The program lane runs each rollout under the runtime `Budget::unlimited()` while the optimizer `Budget` governs the batch. ## `RolloutCache` In-memory map from a rollout key to its `Eval` (score plus optional feedback). A candidate re-evaluated on a seen example returns the cached `Eval` with no LM call and no metric call. The cache is serialized into checkpoints, so a resumed run skips completed rollouts. The key recipe is `(baseline, candidate, example, salt)`, formatted as four 16-digit hex hashes joined by colons: | Component | Module lane | Program lane | | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------- | | `baseline` | Content hash of the module's `ModuleState` before the overlay is applied. Permanently installing a winner mid-run changes it, invalidating stale entries. | `program.meta.program_hash`. | | `candidate` | `Candidate::stable_hash()` | `Overlay::hash()` | | `example` | Content hash of the example | Content hash of the `DemoRow` | | `salt` | `EngineConfig::cache_salt` | `EngineConfig::cache_salt` | Public surface: `get`, `insert`, `len`, `is_empty`. ## Score bookkeeping **`ScoreMatrix`** is a per-instance matrix of candidates (rows, registration order) by examples (columns). Cells are `None` until scored. Methods: `new(columns)`, `candidates()`, `examples()`, `ensure_rows(n)`, `record(candidate, example, score)`, `score(candidate, example)`, `row(candidate)`, `mean(candidate)`, `best_by_mean()`, `pareto()`, `pareto_over(columns)`. The column-restricted view supports GEPA-style setups where train and validation examples share one matrix. **`ParetoView`** is a dominance snapshot computed from the matrix. `best_scores()` gives the best score per viewed column; `wins(candidate)` counts columns the candidate wins or ties on (tolerance `1e-6`); `frontier()` lists candidates winning on at least one column; `statistics()` summarizes. A candidate with zero wins is dominated. **`ParetoFrontier`** is a standalone convenience wrapper over the same bookkeeping for callers that track candidate payloads outside an engine; GEPA itself uses the engine's matrix directly. It stores `GEPACandidate` payloads, prunes dominated candidates automatically, and offers `add_candidate(candidate, scores) -> bool`, `sample_proportional_to_coverage()`, `best_by_average()`, `candidates()`, `len()`, `is_empty()`, `statistics()`. **`ParetoStatistics`** fields: `num_candidates`, `num_examples_covered`, `avg_coverage: f32`, `max_coverage`, `min_coverage`. A healthy search grows `num_candidates` slowly while `avg_coverage` rises; `num_candidates == 1` means the search has collapsed. ## Outcome types | Type | Shape | | -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `RolloutOutcome` | `example: usize`, `eval: Eval`, `trace: Option`. The trace is `None` when the rollout was served from the cache. | | `CandidateEval` | `candidate: usize`, `rollouts: Vec` in request order. `mean()` is the arithmetic mean over the batch (`0.0` when empty); `scores()` collects the raw scores. | | `EvalOutcome` | `Complete(CandidateEval)` or `BudgetExhausted { needed }`. When exhausted, nothing ran and spend is unchanged; `needed` is the uncached rollout count. `completed()` converts to `Option`. | | `GateOutcome` | `BudgetExhausted { needed }`, `Rejected { minibatch }`, or `Promoted { minibatch, full }`. | | `ProgramEvalOutcome` | `Complete(Vec)`, one per requested candidate in request order, or `BudgetExhausted { needed }`. `completed()` converts to `Option>`. (`ir` feature) | ## Program lane: `ProgramEvalEngine<'m, MT: ProgramMetric>` Behind the `ir` feature. The IR-native evaluation path: candidate `ir::Overlay`s evaluated over one shared `Arc` through the `Interpreter`, with true candidate-level parallelism. The interpreter reads instructions, demos, and code through the overlay at render time, so there is no mutation, no apply, and no restore. Every uncached `(candidate, example)` pair across all requested candidates joins one bounded-concurrency stream. ```rust theme={null} pub trait ProgramMetric: Send + Sync { async fn evaluate( &self, example: &DemoRow, output: &JsonMap, trace: Option<&Trace>, ) -> Result; } ``` `ProgramMetric` is the JSON-native sibling of `TypedMetric`: loaded programs have no static output type, so the metric scores the interpreter's output `JsonMap` against a labeled `DemoRow`. | Method | What it does | | ----------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `new(examples, metric, config)` | Builds over `Vec`, a `&MT`, and the same `EngineConfig` as the module lane. | | `evaluate_program_candidates(interp, candidates, subset)` | The IR-native entry point: evaluates N registered candidates over `subset` in one fan-out. Each rollout runs `interp.run(input, Some(overlay), Budget::unlimited())` under its own capture scope with `TraceMeta.candidate_hash` set to the overlay hash and a `program` tag carrying the program hash. Returns `ProgramEvalOutcome`. | | `evaluate(interp, candidate, subset)` | Single-candidate convenience; returns the module-lane `EvalOutcome`. | | `evaluate_gated(interp, candidate, minibatch, threshold)` | The same minibatch gate as the module lane; returns `GateOutcome`. | | `register(overlay)` | Registers an `ir::Overlay`, deduplicating by `Overlay::hash()`. Returns its index. | | `candidate(i) -> &Arc`, `candidate_hash(i)`, `num_candidates()` | Registry access. | | `cache()` | The rollout cache; program-lane keys are listed in the table above. | | `peak_candidate_concurrency()` | High-water mark of distinct candidates with rollouts in flight simultaneously. A value of 2 or more is positive evidence that candidate-level parallelism happened; the module lane is structurally pinned to 1. | | `examples()`, `num_examples()`, `config()`, `spend()`, `matrix()`, `pareto()`, `pareto_over()`, `budget_allows()`, `charge()` | Identical to the module lane. | The program lane shares `Spend` accounting, budget gating, and matrix bookkeeping with the module lane, but it does not offer `checkpoint` or `resume`; those exist only on `EvalEngine`. ## See also * [Optimizers](/docs/components/optimizers) * [Evaluation](/docs/components/evaluation) * [Runtime](/docs/components/runtime) for the IR `Budget`, `BudgetMeter`, and `Interpreter::run` * [Traces](/docs/components/traces) for `Trace`, `TraceMeta`, and `Eval` * [Program and nodes](/docs/components/program-and-nodes) for `Program` and `ir::Overlay` * [COPRO](/docs/optimizers/copro), [MIPROv2](/docs/optimizers/miprov2), [GEPA](/docs/optimizers/gepa) * Example: [04-optimize-hotpotqa.rs](https://github.com/krypticmouse/DSRs/blob/main/crates/dspy-rs/examples/04-optimize-hotpotqa.rs) * Example: [08-optimize-mipro.rs](https://github.com/krypticmouse/DSRs/blob/main/crates/dspy-rs/examples/08-optimize-mipro.rs) * Example: [09-gepa-sentiment.rs](https://github.com/krypticmouse/DSRs/blob/main/crates/dspy-rs/examples/09-gepa-sentiment.rs)