Evaluators

These are the Evaluator subclasses used by the workflow components' compute steps. For the plain callable protocols, see Evaluation Classes.

See Writing Evaluation Callbacks and Parallel Evaluation for usage.

ropt.components.evaluators.Evaluator

Bases: ABC

Abstract base class for evaluator components within an optimization workflow.

Subclasses must implement the abstract _eval method, which performs the actual evaluation of variables using an EvaluationBatchContext and returns an EvaluationBatchResult. Callers use eval, which adds the concurrency guard.

Note

Evaluators are not safe for concurrent use. An evaluator raises a RuntimeError if two threads execute its eval method at the same time. Serial reuse is allowed: the same instance may be reused by several compute steps, including on different threads, as long as each call fully completes before the next begins. For parallel workflows use a dispatching evaluator such as ParallelEvaluator, which dispatches tasks to an executor rather than sharing an evaluator across threads. See Optimization Workflows for usage and pitfalls.

__init__

__init__() -> None

Initialize the Evaluator.

eval

eval(
    variables: NDArray[float64],
    context: EvaluationBatchContext,
) -> EvaluationBatchResult

Evaluate objective and constraint functions for given variables.

This follows the EvaluationBatchCallback protocol.

Parameters:

Name	Type	Description	Default
variables	NDArray[float64]	The matrix of variables to evaluate. Each row represents a variable vector.	required
context	EvaluationBatchContext	The evaluation context, providing additional information about the evaluation.	required

Returns:

Type	Description
EvaluationBatchResult	An evaluation results object containing the calculated values.

Raises:

Type	Description
WorkflowError	If another thread is executing this evaluator's eval method at the same time.

ropt.components.evaluators.BatchEvaluator

Bases: Evaluator

An evaluator that defers to a callable callback.

__init__

__init__(*, callback: EvaluationBatchCallback) -> None

Initialize the BatchEvaluator.

Forwards the evaluation to the provided callback, which should implement the EvaluationBatchCallback protocol.

Parameters:

Name	Type	Description	Default
callback	EvaluationBatchCallback	The callback to defer evaluation to.	required

ropt.components.evaluators.FunctionEvaluator

Bases: Evaluator

An evaluator that calls a function.

This Evaluator stores a single function that returns a value for each objective and constraint.

__init__

__init__(
    *,
    function: EvaluationFunctionCallback,
    batch_id_callback: Callable[[], int] | None = None,
) -> None

Initialize the FunctionEvaluator.

Parameters:

Name	Type	Description	Default
function	EvaluationFunctionCallback	The function used for objectives and constraints.	required
batch_id_callback	Callable[[], int] | None	Callable that returns the next batch ID each time it is called.	None

ropt.components.evaluators.ParallelEvaluator

Bases: Evaluator

An evaluator that dispatches tasks to an executor via asyncio.

Submits the rows of the evaluation batch as tasks to the executor's task queue and collects results from a results queue. By default each row is submitted as its own task; the bundle_size constructor argument can be used to group several active evaluations into a single task that the worker executes sequentially.

See Parallel Evaluation for details on how this integrates with the asyncio event loop.

__init__

__init__(
    *,
    function: EvaluationFunctionCallback,
    executor: Executor,
    bundle_size: int = 1,
    batch_id_callback: Callable[[], int] | None = None,
) -> None

Initialize the ParallelEvaluator.

With bundle_size=1 (the default) every active evaluation is sent as its own executor task. Setting bundle_size to an integer > 1 groups up to that many active evaluations into one task that the worker runs sequentially; 0 packs all active evaluations of a batch into a single task.

Parameters:

Name	Type	Description	Default
function	EvaluationFunctionCallback	The function used for objectives and constraints.	required
executor	Executor	The executor to dispatch tasks to.	required
bundle_size	int	Number of active evaluations per executor task.	1
batch_id_callback	Callable[[], int] | None	Callable that returns the next batch ID each time it is called.	None

Raises:

Type	Description
ValueError	If bundle_size is negative.

ropt.components.evaluators.CachedEvaluator

Bases: Evaluator

An evaluator that caches results to avoid redundant computations.

Wraps another evaluator, retrieving previously computed results from EventHandler sources before delegating uncached evaluations.

See Using CachedEvaluator for full details on cache matching, realization name handling, and source management.

__init__

__init__(
    *,
    evaluator: Evaluator,
    sources: Sequence[EventHandler]
    | set[EventHandler]
    | None = None,
    hits_key: str | None = None,
) -> None

Initialize the CachedEvaluator.

The sources argument should be a sequence of EventHandler instances. These handlers are expected to store FunctionResults in their ["results"] attribute.

Parameters:

Name	Type	Description	Default
evaluator	Evaluator	The evaluator to cache.	required
sources	Sequence[EventHandler] | set[EventHandler] | None	EventHandler instances for retrieving cached results.	None
hits_key	str | None	Optional key for storing cache-hits in metadata.	None

eval_cached

eval_cached(
    variables: NDArray[float64],
    evaluator_context: EvaluationBatchContext,
) -> tuple[
    EvaluationBatchResult,
    dict[int, tuple[int, FunctionResults]],
]

Evaluate using cache, returning both results and cache-hit info.

Derived classes can override eval and call this method to access cache-hit information for populating metadata.

Parameters:

Name	Type	Description	Default
variables	NDArray[float64]	Matrix of variables to evaluate.	required
evaluator_context	EvaluationBatchContext	The evaluation context.	required

Returns:

Type	Description
EvaluationBatchResult	An EvaluationBatchResult and the cache hits, keyed by evaluation
dict[int, tuple[int, FunctionResults]]	index, of (realization index, cached FunctionResults).

add_sources

add_sources(
    sources: EventHandler | Sequence[EventHandler],
) -> None

Add one or more EventHandler sources.

Parameters:

Name	Type	Description	Default
sources	EventHandler | Sequence[EventHandler]	EventHandler instances to add as a source.	required

ropt.components.evaluators.BatchIdCounter

A thread-safe counter for generating sequential batch IDs.

Provides a simple default batch_id_callback for evaluators. Each call returns the next integer starting from zero.

Pass the same instance to multiple evaluators to share a single counter across them — useful in nested or parallel optimization setups where all evaluators should produce globally unique batch IDs.

See Writing Evaluation Callbacks for usage details and examples.

__init__

__init__() -> None

Initialize the counter starting at zero.

__call__

__call__() -> int

Return the next batch ID and advance the counter.

ropt.components.evaluators.EvaluationFunctionCallback

Bases: Protocol

Defines the call signature for function callbacks.

A function following this protocol is called once per active row of the evaluation batch, receiving the variable vector for that row together with a EvaluationFunctionContext object that identifies the evaluation.

The function should return a EvaluationFunctionResult object containing the evaluation results.

__call__

__call__(
    variables: NDArray[float64],
    context: EvaluationFunctionContext,
) -> EvaluationFunctionResult

Evaluate objectives and constraints for a single variable vector.

Parameters:

Name	Type	Description	Default
variables	NDArray[float64]	1-D variable vector for this evaluation.	required
context	EvaluationFunctionContext	The EvaluationFunctionContext object identifying the evaluation.	required

Returns:

Type	Description
EvaluationFunctionResult	The evaluation result as a EvaluationFunctionResult object.

ropt.components.evaluators.EvaluationFunctionContext dataclass

Context for a single function evaluation.

Attributes:

Name	Type	Description
realization	int	The realization index.
perturbation	int	The perturbation index (-1 when unperturbed).
batch_id	int	Integer identifying the current evaluation batch.
eval_idx	int	Row index within the batch.
metadata	dict[str, Any] | None	The metadata the run was started with, if any.

ropt.components.evaluators.EvaluationFunctionResult dataclass

Result of a single function evaluation.

Attributes:

Name	Type	Description
objectives	NDArray[float64] | float	The objective values as an array.
constraints	NDArray[float64] | float | None	Optional constraint values as an array.
metadata	dict[str, Any] | None	Optional dictionary containing additional information about the evaluation.