Workflow Evaluator Classes

These are the Evaluator subclasses used by the workflow framework's compute steps. For the simple callable protocol used by BasicOptimizer, see Evaluation Classes.

See Writing Evaluation Callbacks and Parallel Evaluation for usage.

ropt.workflow.evaluators.Evaluator

Bases: ABC

Abstract base class for evaluator components within an optimization workflow.

Subclasses must implement the abstract eval method, which is responsible for performing the actual evaluation of variables using an EvaluationBatchContext and returning an EvaluationBatchResult.

Warning

Evaluator instances must not be called concurrently from multiple threads. For parallel workflows use a server-based evaluator such as AsyncEvaluator.

init

__init__() -> None

Initialize the Evaluator.

eval `abstractmethod`

eval(
    variables: NDArray[float64],
    context: EvaluationBatchContext,
) -> EvaluationBatchResult

Evaluate objective and constraint functions for given variables.

This follows the EvaluationBatchCallback protocol.

Parameters:

Name	Type	Description	Default
`variables`	`NDArray[float64]`	The matrix of variables to evaluate. Each row represents a variable vector.	required
`context`	`EvaluationBatchContext`	The evaluation context, providing additional information about the evaluation.	required

Returns:

Type	Description
`EvaluationBatchResult`	An evaluation results object containing the calculated values.

Raises:

Type	Description
`RuntimeError`	If this evaluator is already in use by another thread.

ropt.workflow.evaluators.BatchEvaluator

Bases: Evaluator

An evaluator that defers to a callable callback.

init

__init__(*, callback: EvaluationBatchCallback) -> None

Initialize the BatchEvaluator.

Forwards the evaluation to the provided callback, which should implement the EvaluationBatchCallback protocol.

Parameters:

Name	Type	Description	Default
`callback`	`EvaluationBatchCallback`	The callback to defer evaluation to.	required

eval

eval(
    variables: NDArray[float64],
    context: EvaluationBatchContext,
) -> EvaluationBatchResult

Call the stored callback with the given variables and context.

Parameters:

Name	Type	Description	Default
`variables`	`NDArray[float64]`	Matrix of variables to evaluate (each row is a vector).	required
`context`	`EvaluationBatchContext`	The evaluation context.	required

Returns:

Type	Description
`EvaluationBatchResult`	An `EvaluationBatchResult` with the evaluation results.

ropt.workflow.evaluators.FunctionEvaluator

Bases: Evaluator

An evaluator that calls a function.

This Evaluator stores a single function that returns a value for each objective and constraint.

init

__init__(
    *,
    function: EvaluationFunctionCallback,
    batch_id_callback: Callable[[], int] | None = None,
) -> None

Initialize the FunctionEvaluator.

Parameters:

Name	Type	Description	Default
`function`	`EvaluationFunctionCallback`	The function used for objectives and constraints.	required
`batch_id_callback`	`Callable[[], int] \| None`	Callable that returns the next batch ID each time it is called.	`None`

eval

eval(
    variables: NDArray[float64],
    evaluator_context: EvaluationBatchContext,
) -> EvaluationBatchResult

Evaluate all objective and constraints.

Parameters:

Name	Type	Description	Default
`variables`	`NDArray[float64]`	The matrix of variables to evaluate.	required
`evaluator_context`	`EvaluationBatchContext`	The evaluation context.	required

Returns:

Type	Description
`EvaluationBatchResult`	The result of calling the wrapped evaluator function.

ropt.workflow.evaluators.AsyncEvaluator

Bases: Evaluator

An evaluator that dispatches tasks to a server via asyncio.

Submits the rows of the evaluation batch as tasks to the server's task queue and collects results from a results queue. By default each row is submitted as its own task; the bundle_size constructor argument can be used to group several active evaluations into a single task that the worker executes sequentially.

See Parallel Evaluation for details on how this integrates with the asyncio event loop.

init

__init__(
    *,
    function: EvaluationFunctionCallback,
    server: Server,
    bundle_size: int = 1,
    queue_size: int = 0,
    get_name: NameCallback | None = None,
    batch_id_callback: Callable[[], int] | None = None,
) -> None

Initialize the FunctionEvaluator.

With bundle_size=1 (the default) every active evaluation is sent as its own server task. Setting bundle_size to an integer > 1 groups up to that many active evaluations into one task that the worker runs sequentially; 0 packs all active evaluations of a batch into a single task.

The get_name callback receives the EvaluationFunctionContext objects for every evaluation in a task (a one-element sequence when bundle_size=1) and must return a single task name.

Parameters:

Name	Type	Description	Default
`function`	`EvaluationFunctionCallback`	The function used for objectives and constraints.	required
`server`	`Server`	Optional evaluator server to use.	required
`bundle_size`	`int`	Number of active evaluations per server task.	`1`
`queue_size`	`int`	Maximum size of the result queue.	`0`
`get_name`	`NameCallback \| None`	Optional callable to generate names for tasks.	`None`
`batch_id_callback`	`Callable[[], int] \| None`	Callable that returns the next batch ID each time it is called.	`None`

Raises:

Type	Description
`ValueError`	If `bundle_size` is negative.

eval

eval(
    variables: NDArray[float64],
    evaluator_context: EvaluationBatchContext,
) -> EvaluationBatchResult

Evaluate all objective and constraints.

Parameters:

Name	Type	Description	Default
`variables`	`NDArray[float64]`	The matrix of variables to evaluate.	required
`evaluator_context`	`EvaluationBatchContext`	The evaluation context.	required

Returns:

Type	Description
`EvaluationBatchResult`	The result of calling the wrapped evaluator function.

Raises:

Type	Description
`Abort`	raise if the server is not running.

ropt.workflow.evaluators.CachedEvaluator

Bases: Evaluator

An evaluator that caches results to avoid redundant computations.

Wraps another evaluator, retrieving previously computed results from EventHandler sources before delegating uncached evaluations.

See Optimization Workflows for full details on cache matching, realization name handling, and source management.

init

__init__(
    *,
    evaluator: Evaluator,
    sources: Sequence[EventHandler]
    | set[EventHandler]
    | None = None,
    hits_key: str | None = None,
) -> None

Initialize the CachedEvaluator.

The sources argument should be a sequence of EventHandler instances. These handlers are expected to store FunctionResults in their ["results"] attribute.

Parameters:

Name	Type	Description	Default
`evaluator`	`Evaluator`	The evaluator to cache.	required
`sources`	`Sequence[EventHandler] \| set[EventHandler] \| None`	`EventHandler` instances for retrieving cached results.	`None`
`hits_key`	`str \| None`	Optional key for storing cache-hits in `metadata`.	`None`

eval_cached

eval_cached(
    variables: NDArray[float64],
    evaluator_context: EvaluationBatchContext,
) -> tuple[
    EvaluationBatchResult,
    dict[int, tuple[int, FunctionResults]],
]

Evaluate using cache, returning both results and cache-hit info.

Returns the evaluation results together with a dictionary of cache hits. The dictionary keys are evaluation indices found in cache; values are tuples of (realization index, cached FunctionResults).

Derived classes can override eval and call this method to access cache-hit information for populating metadata.

Note

If realization names are configured, they are used for matching (allowing cache hits across runs). Otherwise realization indices are used.

Parameters:

Name	Type	Description	Default
`variables`	`NDArray[float64]`	Matrix of variables to evaluate.	required
`evaluator_context`	`EvaluationBatchContext`	The evaluation context.	required

Returns:

Type	Description
`tuple[EvaluationBatchResult, dict[int, tuple[int, FunctionResults]]]`	An `EvaluationBatchResult` and the cache hits.

eval

eval(
    variables: NDArray[float64],
    context: EvaluationBatchContext,
) -> EvaluationBatchResult

Evaluate using cache, delegating uncached evaluations.

Parameters:

Name	Type	Description	Default
`variables`	`NDArray[float64]`	Matrix of variables to evaluate.	required
`context`	`EvaluationBatchContext`	The evaluation context.	required

Returns:

Type	Description
`EvaluationBatchResult`	An `EvaluationBatchResult` with calculated or cached values.

add_sources

add_sources(
    sources: EventHandler | Sequence[EventHandler],
) -> None

Add one or more EventHandler sources.

Parameters:

Name	Type	Description	Default
`sources`	`EventHandler \| Sequence[EventHandler]`	`EventHandler` instances to add as a source.	required

ropt.workflow.evaluators.BatchIdCounter

A thread-safe counter for generating sequential batch IDs.

Provides a simple default batch_id_callback for evaluators. Each call returns the next integer starting from zero.

Pass the same instance to multiple evaluators to share a single counter across them — useful in nested or parallel optimization setups where all evaluators should produce globally unique batch IDs.

See Writing Evaluation Callbacks for usage details and examples.

init

__init__() -> None

Initialize the counter starting at zero.

call

__call__() -> int

Return the next batch ID and advance the counter.

ropt.workflow.evaluators.EvaluationFunctionCallback

Bases: Protocol

Defines the call signature for function callbacks.

A function following this protocol is called once per active row of the evaluation batch, receiving the variable vector for that row together with a EvaluationFunctionContext object that identifies the evaluation.

The function should return a EvaluationFunctionResult object containing the evaluation results.

call

__call__(
    variables: NDArray[float64],
    context: EvaluationFunctionContext,
) -> EvaluationFunctionResult

Evaluate objectives and constraints for a single variable vector.

Parameters:

Name	Type	Description	Default
`variables`	`NDArray[float64]`	1-D variable vector for this evaluation.	required
`context`	`EvaluationFunctionContext`	The `EvaluationFunctionContext` object identifying the evaluation.	required

Returns:

Type	Description
`EvaluationFunctionResult`	The evaluation result as a `EvaluationFunctionResult` object.

ropt.workflow.evaluators.EvaluationFunctionContext `dataclass`

Context for a single function evaluation.

Attributes:

Name	Type	Description
`realization`	`int`	The realization index.
`perturbation`	`int`	The perturbation index (`-1` when unperturbed).
`batch_id`	`int`	Integer identifying the current evaluation batch.
`eval_idx`	`int`	Row index within the batch.
`name`	`str \| None`	Optional task name set by the evaluator.

ropt.workflow.evaluators.EvaluationFunctionResult `dataclass`

Result of a single function evaluation.

Attributes:

Name	Type	Description
`objectives`	`NDArray[float64] \| float`	The objective values as an array.
`constraints`	`NDArray[float64] \| float \| None`	Optional constraint values as an array.
`metadata`	`dict[str, Any] \| None`	Optional dictionary containing additional information about the evaluation.

ropt.workflow.evaluators.NameCallback

Bases: Protocol

Defines the call signature for callbacks to get the name of a task.

call

__call__(
    contexts: Sequence[EvaluationFunctionContext],
) -> str

Get the name for a task.

The task may contain a single evaluation or a bundle of several evaluations that the worker runs sequentially. The callback receives the EvaluationFunctionContext objects for every evaluation in the task, in submission order, and should return a single string used as the task name.

Parameters:

Name	Type	Description	Default
`contexts`	`Sequence[EvaluationFunctionContext]`	The contexts for every evaluation in the task.	required

Returns:

Type	Description
`str`	The name of the task.

Workflow Evaluator Classes

ropt.workflow.evaluators.Evaluator

__init__

eval abstractmethod

ropt.workflow.evaluators.BatchEvaluator

__init__

eval

ropt.workflow.evaluators.FunctionEvaluator

__init__

eval

ropt.workflow.evaluators.AsyncEvaluator

__init__

eval

ropt.workflow.evaluators.CachedEvaluator

__init__

eval_cached

eval

add_sources

ropt.workflow.evaluators.BatchIdCounter

__init__

__call__

ropt.workflow.evaluators.EvaluationFunctionCallback

__call__

ropt.workflow.evaluators.EvaluationFunctionContext dataclass

ropt.workflow.evaluators.EvaluationFunctionResult dataclass

ropt.workflow.evaluators.NameCallback

__call__

init

eval `abstractmethod`

init

init

init

init

init

call

call

ropt.workflow.evaluators.EvaluationFunctionContext `dataclass`

ropt.workflow.evaluators.EvaluationFunctionResult `dataclass`

call