Evaluators

ropt.components.evaluators

Evaluators: the objects a compute step uses to evaluate the model.

Each is an Evaluator subclass. For the plain callable protocols they wrap, see Evaluation Classes. See Writing Evaluation Callbacks and Parallel Evaluation for usage.

ropt.components.evaluators.Evaluator

Bases: ABC

Abstract base class for evaluator components within an optimization workflow.

Subclasses must implement the abstract _eval method, which performs the actual evaluation of variables using an EvaluationBatchContext and returns an EvaluationBatchResult. Callers use eval, which adds the concurrency guard.

Note

Evaluators are not safe for concurrent use. An evaluator raises a WorkflowError if two threads execute its eval method at the same time. Serial reuse is allowed: the same instance may be reused by several compute steps, including on different threads, as long as each call fully completes before the next begins. For parallel workflows use a dispatching evaluator such as ParallelEvaluator, which dispatches tasks to an executor rather than sharing an evaluator across threads. See Optimization Workflows for usage and pitfalls.

__init__

__init__() -> None

Initialize the Evaluator.

eval

eval(
    variables: NDArray[float64],
    context: EvaluationBatchContext,
) -> EvaluationBatchResult

Evaluate objective and constraint functions for given variables.

This follows the EvaluationBatchCallback protocol.

Parameters:

Name	Type	Description	Default
variables	NDArray[float64]	The matrix of variables to evaluate, one vector per row.	required
context	EvaluationBatchContext	The evaluation context.	required

Returns:

Type	Description
EvaluationBatchResult	An evaluation results object containing the calculated values.

Raises:

Type	Description
WorkflowError	If another thread is already executing eval.

ropt.components.evaluators.BatchEvaluator

Bases: Evaluator

An evaluator that defers to a callable callback.

__init__

__init__(*, callback: EvaluationBatchCallback) -> None

Initialize the BatchEvaluator.

Forwards the evaluation to the provided callback, which should implement the EvaluationBatchCallback protocol.

Parameters:

Name	Type	Description	Default
callback	EvaluationBatchCallback	The callback to defer evaluation to.	required

ropt.components.evaluators.FunctionEvaluator

Bases: Evaluator

An evaluator that calls a function once for each variable vector.

The function returns the objective and constraint values for that vector.

__init__

__init__(
    *,
    function: EvaluationFunctionCallback,
    batch_id_callback: Callable[[], int] | None = None,
) -> None

Initialize the FunctionEvaluator.

Parameters:

Name	Type	Description	Default
function	EvaluationFunctionCallback	The function used for objectives and constraints.	required
batch_id_callback	Callable[[], int] | None	Callable that returns the next batch ID each time it is called.	None

ropt.components.evaluators.ParallelEvaluator

Bases: Evaluator

An evaluator that dispatches tasks to an executor via asyncio.

Submits the rows of the evaluation batch as tasks to the executor's task queue and collects results from a results queue. By default each row is submitted as its own task; the bundle_size constructor argument can be used to group several active evaluations into a single task that the worker executes sequentially.

See Parallel Evaluation for details on how this integrates with the asyncio event loop.

__init__

__init__(
    *,
    function: EvaluationFunctionCallback,
    executor: Executor,
    bundle_size: int = 1,
    batch_id_callback: Callable[[], int] | None = None,
) -> None

Initialize the ParallelEvaluator.

With bundle_size=1 (the default) every active evaluation is sent as its own executor task. Setting bundle_size to an integer > 1 groups up to that many active evaluations into one task that the worker runs sequentially; 0 packs all active evaluations of a batch into a single task.

Parameters:

Name	Type	Description	Default
function	EvaluationFunctionCallback	The function used for objectives and constraints.	required
executor	Executor	The executor to dispatch tasks to.	required
bundle_size	int	Number of active evaluations per executor task.	1
batch_id_callback	Callable[[], int] | None	Callable that returns the next batch ID each time it is called.	None

Raises:

Type	Description
ValueError	If bundle_size is negative.

ropt.components.evaluators.BatchIdCounter

A thread-safe counter for generating sequential batch IDs.

Provides a simple default batch_id_callback for evaluators. Each call returns the next integer starting from zero.

Pass the same instance to multiple evaluators to share a single counter across them, so that their batch IDs remain unique.

See Writing Evaluation Callbacks for usage details and examples.

__init__

__init__() -> None

Initialize the counter starting at zero.

__call__

__call__() -> int

Return the next batch ID and advance the counter.

ropt.components.evaluators.EvaluationFunctionCallback

Bases: Protocol

Defines the call signature for function callbacks.

A function following this protocol is called once per active row of the evaluation batch, receiving the variable vector for that row together with an EvaluationFunctionContext object that identifies the evaluation, and returns an EvaluationFunctionResult.

__call__

__call__(
    variables: NDArray[float64],
    context: EvaluationFunctionContext,
) -> EvaluationFunctionResult

Evaluate objectives and constraints for a single variable vector.

Parameters:

Name	Type	Description	Default
variables	NDArray[float64]	1-D variable vector for this evaluation.	required
context	EvaluationFunctionContext	The context identifying the evaluation.	required

Returns:

Type	Description
EvaluationFunctionResult	The objective and constraint values for this variable vector.

ropt.components.evaluators.EvaluationFunctionContext dataclass

Context for a single function evaluation.

Attributes:

Name	Type	Description
realization	int	The realization index.
perturbation	int	The perturbation index (-1 when unperturbed).
batch_id	int	Integer identifying the current evaluation batch.
eval_idx	int	Row index within the batch.
metadata	dict[str, Any] | None	The metadata the run was started with, if any.

ropt.components.evaluators.EvaluationFunctionResult dataclass

Result of a single function evaluation.

Attributes:

Name	Type	Description
objectives	NDArray[float64] | float	The objective values as an array.
constraints	NDArray[float64] | float | None	Optional constraint values as an array.
metadata	dict[str, Any] | None	Optional dictionary with extra information.