Optimization Workflows

Basic Optimization uses BasicOptimizer to drive a single optimization run with a single evaluator. For anything more elaborate — multiple optimizers in sequence, nested optimizations, custom event handling, parallel/async execution — drop down to the workflow framework.

The framework has four concepts:

Concept	Role
ComputeStep	An executable unit of work (run an optimizer, run a single ensemble evaluation, etc.).
EventHandler	A reactive object that observes events emitted by a compute step.
Evaluator	The object a compute step uses to actually evaluate the model.
Executor	Dispatches evaluation tasks to threads, processes, or an HPC cluster.

The first three are covered below. Executors are only relevant for asynchronous and parallel execution and are discussed in Parallel Evaluation.

Compute steps emit EnOptEvent objects at key points during execution — for instance when an evaluation starts or finishes. The most important event is FINISHED_EVALUATION, which carries the generated Results objects. Event handlers are attached to a step and receive its events, allowing them to track, store, or react to results as they arrive.

The EnOptEvent object

Each event is an EnOptEvent dataclass with three fields:

Field	Type	Description
event_type	EnOptEventType	Which lifecycle point triggered the event.
context	EnOptContext	The optimizer context active at the time of the event.
results	tuple[Results, ...]	Result objects (empty tuple when no results apply).

Event types

The EnOptEventType enumeration defines the following event types:

Event type	When it fires
START_OPTIMIZER	Just before an optimization algorithm begins iterating.
FINISHED_OPTIMIZER	Immediately after the optimizer finishes (success or error).
START_EVALUATION	Before evaluating functions (or gradients).
FINISHED_EVALUATION	After evaluation completes — carries results.
START_ENSEMBLE_EVALUATOR	Before an EvaluationStep compute step begins.
FINISHED_ENSEMBLE_EVALUATOR	After an EvaluationStep compute step finishes.

Most event handlers only need to listen for FINISHED_EVALUATION; the other types are useful for logging, progress bars, or custom lifecycle hooks.

A workflow you can read end to end

import numpy as np
from numpy.typing import NDArray

from ropt.context import EnOptContext
from ropt.workflow.compute_steps import OptimizationStep
from ropt.workflow.evaluators import (
    EvaluationFunctionContext,
    EvaluationFunctionResult,
    FunctionEvaluator,
)
from ropt.workflow.event_handlers import ResultsHandler

# 1. Build the configuration.
CONFIG = {
    "variables": {"variable_count": 3, "perturbation_magnitudes": 1e-6},
    "realizations": {"weights": [1.0] * 5},
}

# 2. Define a per-realization evaluation function.
def my_function(
    variables: NDArray[np.float64],
    context: EvaluationFunctionContext,
) -> EvaluationFunctionResult:
    return EvaluationFunctionResult(
        objectives=np.array([(variables - 1.0) @ (variables - 1.0)]),
    )

# 3. Construct an evaluator that calls a per-realization Python function.
evaluator = FunctionEvaluator(function=my_function)

# 4. Build the compute step.
step = OptimizationStep(evaluator=evaluator)

# 5. Attach event handlers.
result_handler = ResultsHandler()  # remember the best
step.add_event_handler(result_handler)

# 6. Run the step.
step.run(
    variables=np.array([0.5, 0.7, 0.9]),
    context=EnOptContext.model_validate(CONFIG),
)

# 7. Read best results from the handlers.
print(f"Optimal variables: {result_handler['results'].evaluations.variables}")

This is a minimal example of optimizing a simple deterministic function. A full runnable example for optimizing the Rosenbrock function with uncertain parameters can be found here: examples/ensemble.py.

Compute steps

Two compute steps ship with ropt:

• OptimizationStep — runs an optimization algorithm.
• EvaluationStep — runs a single ensemble evaluation (no optimizer). For example, useful for evaluating an optimum on a different ensemble, or on a sub-set of realizations.

Both compute steps require an EnOptContext and a variables argument passed to their run(...) method. For OptimizationStep, this is a single 1-D variable vector (the starting point). For EvaluationStep, it may be a single vector or a 2-D matrix where each row is a variable vector to evaluate. An optional metadata dictionary can be attached; if provided, it is included in the Results objects emitted via the FINISHED_EVALUATION event.

Events emitted by OptimizationStep

OptimizationStep executes an optimization algorithm based on the provided context. It iteratively performs function and potentially gradient evaluations, yielding a sequence of FunctionResults and GradientResults objects.

The following events are emitted during execution:

• START_OPTIMIZER: Emitted just before the optimization process begins.
• START_EVALUATION: Emitted immediately before a batch of function or perturbation evaluations is performed.
• FINISHED_EVALUATION: Emitted after an evaluation completes. The event's results field carries the generated Results objects. Event handlers typically listen for this event to process or track optimization progress.
• FINISHED_OPTIMIZER: Emitted after the entire optimization process concludes (successfully, or due to termination conditions or errors).

Events emitted by EvaluationStep

EvaluationStep evaluates a batch of variable vectors. The variables argument can be a single 1-D vector (treated as one row) or a 2-D matrix where each row is a variable vector. The evaluator performs a function evaluation for the full batch and produces a tuple of FunctionResults objects.

The following events are emitted during execution:

• START_ENSEMBLE_EVALUATOR: Emitted before the evaluation process begins.
• START_EVALUATION: Emitted just before the batch evaluation is performed.
• FINISHED_EVALUATION: Emitted after the evaluation completes. The event's results field carries the generated FunctionResults objects. Event handlers typically listen for this event.
• FINISHED_ENSEMBLE_EVALUATOR: Emitted after the entire compute step, including result emission, is finished.

Exit codes

Both run() methods return an ExitCode indicating why the step finished:

Exit code	Meaning
OPTIMIZER_FINISHED	The optimizer terminated normally.
ENSEMBLE_EVALUATOR_FINISHED	The evaluator step completed normally.
TOO_FEW_REALIZATIONS	Too few realizations were evaluated successfully.
MAX_FUNCTIONS_REACHED	Maximum number of function evaluations was reached.
MAX_BATCHES_REACHED	Maximum number of evaluation batches was reached.
USER_ABORT	The optimization was aborted by the user.
ABORT_FROM_ERROR	Aborted due to an error handled elsewhere.

Event handlers

Event handlers are attached to a compute step via its add_event_handler method. Once attached, the handler receives every event the step emits.

Using handlers safely

An event handler is a stateful object that is not safe for concurrent use. There are two ways to drive one, and they are mutually exclusive:

• Attached directly to compute steps. A handler may be attached to several compute steps, and a single instance can accumulate state across them — as long as those steps do not run it concurrently. Serial reuse is fine, even across different threads: each handle_event call must fully complete before the next begins. If two threads execute handle_event at the same time, a RuntimeError is raised.

• Registered with an EventDispatcher. When work runs on several threads at once (for example, ParallelEvaluator with a multi-worker ThreadingExecutor), route events through a dispatcher. It receives events from any thread and delivers them to its handlers one at a time, so a single handler can safely aggregate results produced on many threads. See Event Dispatcher for the pattern.

A handler is owned by either one dispatcher or one-or-more compute steps — never both — and may be registered with at most one dispatcher. Mixing the two, or registering with a second dispatcher, raises a RuntimeError.

Do not share a handler across parallel steps

Never attach the same handler instance to compute steps that may run at the same time on different threads; the moment a second thread executes it while the first is still inside handle_event, a RuntimeError is raised. Give each parallel step its own handler, or route events through an EventDispatcher.

Serial reuse is allowed: the same handler may be reused across steps that run one after another, even on different threads, as long as their calls never overlap.

Pickling

A handler can be pickled before it is first used — for example, when a compute step is shipped to a worker process. A handler that has already processed an event cannot be pickled and raises a RuntimeError.

Reading results is not thread-guarded

Handler state exposed through handler[key] is deliberately not bound to a thread, so results can be read after a run from any thread. Read a handler's stored values only after its producer has finished: after step.run() returns for a directly-attached handler, or after the EventDispatcher has been cancelled and its task group has exited for a handler registered with a dispatcher. Both are synchronization points that make the latest values visible.

Reading a handler's state while it is still processing events on another thread returns a valid object, but possibly a stale one — do not rely on it for the latest result. For live progress during a parallel run, use a CallbackHandler (which is pushed each event) rather than polling another handler's state.

The framework ships four reusable handlers:

Handler	Purpose
ResultsHandler	Keep the best (or last) result. Backs BasicOptimizer.results.
HistoryHandler	Keep every result.
CallbackHandler	Forward selected event types to a user callback.
TableHandler	Append rows to a structured table per result.
EventForwardHandler	Forward events to an EventDispatcher for lock-free dispatch.

Handlers expose their state through dictionary access (handler[key]). By convention, ResultsHandler and HistoryHandler both use the key "results" — e.g. result_handler["results"] or history_handler["results"]. TableHandler uses the table name as key — e.g. table["functions"].

ResultsHandler

ResultsHandler listens for FINISHED_EVALUATION events emitted from within an optimization workflow. It processes the Results objects contained within these events and selects a single FunctionResults object to retain based on defined criteria.

The criteria for selection are:

• what='best' (default): Tracks the result with the lowest weighted objective value encountered so far.
• what='last': Tracks the most recently received valid result.

Optionally, results can be filtered based on constraint violations using the constraint_tolerance parameter. If provided, any result violating constraints beyond this tolerance is ignored.

Additionally, a filter callable can be supplied to apply custom filtering logic. It receives a Results object and should return True to keep the result or False to discard it. Only results that pass both the constraint tolerance check and the custom filter (if provided) are considered for selection.

Tracking logic (comparing 'best' or selecting 'last') operates on the results in the optimizer's domain. However, the final selected result that is made accessible via dictionary access (result_handler["results"]) is transformed to the user's domain (when domain="user", the default).

If the domain type is "user", the result is converted from the optimizer domain to the user domain before being stored.

HistoryHandler

HistoryHandler listens for FINISHED_EVALUATION events emitted by compute steps from within an optimization workflow. It collects all Results objects contained within these events and stores them sequentially in memory.

The accumulated results are stored as a tuple and can be accessed via dictionary access using the key "results" (e.g., history_handler["results"]). Each time new results are received from a valid source, they are appended to this tuple. Initially, history_handler["results"] is None.

If the domain type is "user", the results are converted from the optimizer domain to the user domain before being stored.

CallbackHandler

CallbackHandler listens for events and forwards them to a callback function. It is constructed with a set of event_types to respond to and a single callback. When an event with a matching type arrives, the callback is called with the EnOptEvent.

EventForwardHandler

EventForwardHandler is attached to a compute step and forwards matching events to an EventDispatcher. The dispatcher dispatches them from the asyncio event loop's thread, so handlers registered on the dispatcher require no locking.

See Event Dispatcher for the full pattern.

TableHandler

TableHandler tracks results and stores them in pandas DataFrames.

Tables

Tables can be generated for FunctionResults and GradientResults respectively. Tables are added via the add_table method, which takes a name, a type (either "functions" or "gradients"), a column specification and an optional domain type. The column specification determines which fields of the results are stored in the table and how they are named. The domain type determines whether the results are transformed to the user domain before being stored in the table.

Tables are accessed by their name via dictionary syntax, for example, as handler["evaluations"].

Warning

Tables are generated on the fly from internal data when accessing them in this way. When multiple accesses are needed, it is more efficient to first store them in a variable.

Column specification

Columns are specified by providing a dictionary that maps field names to column titles. The keys denote the names of the fields, using attribute syntax. For instance a functions.objectives key indicates that the result should contain a column with objective values that are found in the objectives field of the functions field of the result. The values corresponding to the keys are used to provide the column names.

For example, passing this dictionary via the columns argument generates a table containing the batch id, the values of all calculated objectives and the vector of variables:

{
    "batch_id": "Batch",
    "functions.objectives": "Objective",
    "evaluations.variables": "Variables",
}

Some fields may result in multiple columns in the DataFrame if their values are vectors or matrices. For example, evaluations.variables will generate a separate column for each variable. The table specification above may generate a pandas DataFrame looking something like this:

    Batch   Objective,0  Variables,v0  Variables,v1  Variables,v2
0       0  1.309826e+02      0.500000      0.900000      1.300000
1       0  4.362553e+12    120.900265     20.698539    -90.578972
...

Here, because the variables are vectors of length 2, there are two variable columns generated. The corresponding column names consist of the column title and the name of the variable vector, separated by a comma. Note that the functions.objectives column also contains a comma followed by a 0 value. This is because functions.objectives is also a vector of values, there just happens to be only one objective. Its index is used instead of a name, because no name was provided in the configuration of the optimization. Fields may even have matrix values, in which case the column names may contain two item names or indices separated by commas.

Changing the column name separator

By default a comma is used to separate fields in the column names if needed. The sep input can be used to provide an alternative separator.

You can exploit this by specifying a newline as the separator and display a nicely formatted table using the tabulate package:

from tabulate import tabulate

print(tabulate(table["functions"], headers="keys", showindex=False))

which will show something like this using multi-line headers:

  Batch         Objective    Variables    Variables     Variables
                        0           v0           v1            v2
-------  ----------------  -----------  -----------  ------------
      0           130.983          0.5          0.9           1.3
...

Default tables

The set_default_tables method can be used to add a set of default tables:

• For function results it generates these tables:
  • "functions": contains a set of values of the calculated functions.
  • "evaluations": contains a set of values for all evaluations.
  • "constraints": contains a set of values for all constraints.
• For gradient results it generates these tables:
  • "gradients": contains a set of values of the calculated gradients.
  • "perturbations": contains a set of values for all perturbations.

Adding columns and retrieving all tables

A single column can be added to an existing table after creation using add_column(table_name, field_name, title).

To retrieve all tables at once as a dictionary mapping names to DataFrames, use get_tables().

Callback functionality

The tables are updated anytime a result is processed. A callback can be registered via set_callback to react each time the tables change. The callback signature is:

def my_callback(event: EnOptEvent) -> None:
    ...

It receives the EnOptEvent that caused the tables to be updated.

Evaluators

Compute steps take an Evaluator instance — not the plain callable accepted by BasicOptimizer. Three synchronous evaluators are provided, plus an asynchronous one described in the next section:

Evaluator	Interface
BatchEvaluator	Batch: f(variables_2d, context) → EvaluationBatchResult.
FunctionEvaluator	Per-row: f(variables_1d, context) → EvaluationFunctionResult.
CachedEvaluator	Wraps another evaluator, caching results by variable vector.
ParallelEvaluator	Parallel evaluation via an Executor — see Parallel Evaluation.

Evaluators are not safe for concurrent use

Like event handlers, an evaluator raises a RuntimeError if two threads execute its eval method at the same time. Serial reuse is allowed: a single evaluator instance may be shared by several compute steps that run one after another, even on different threads (for example, reusing one FunctionEvaluator across nested inner optimizations to keep batch ids counting). Do not share one evaluator across steps that run in parallel; give each parallel step its own evaluator. For the constraints on where each layer of a nested workflow may run, see Nested workflows and process boundaries. Note that the parallelism of ParallelEvaluator happens below eval — it dispatches tasks to an executor, so its own eval is still called on a single thread.

An evaluator can be pickled before it is first used (e.g. when shipped to a worker process), but pickling one that has already run raises a RuntimeError.

BatchEvaluator

BatchEvaluator defers to a callable callback that receives the full 2-D variable matrix and an EvaluationBatchContext, and returns an EvaluationBatchResult. Use this when you need the full batch (e.g. vectorized computation, or an external simulator that accepts all rows at once). The callback has the same signature as the callable accepted by BasicOptimizer.

FunctionEvaluator

FunctionEvaluator stores a single function that returns a value for each objective and constraint. The function is called once per row of the evaluation batch with the variable vector and an EvaluationFunctionContext dataclass exposing realization, perturbation, batch_id, eval_idx, and name. The perturbation value is -1 when the evaluation is not a perturbation (i.e. the unperturbed function evaluation). The name value is the optional task name set by the evaluator (e.g. via ParallelEvaluator's get_name callback) and is None when no name was assigned. The function must return an EvaluationFunctionResult dataclass with:

• objectives: a scalar or 1-D NumPy array of length n_objectives.
• constraints (optional): a scalar or 1-D NumPy array of length n_nonlinear_constraints.
• metadata (optional): a dict[str, Any] whose entries are stored in the metadata field of the returned EvaluationBatchResult.

See Writing Evaluation Callbacks for a worked example.

CachedEvaluator

CachedEvaluator wraps another evaluator with result caching. It retrieves previously computed function results from EventHandler instances specified as sources — typically a HistoryHandler or ResultsHandler. For each variable vector and realization, if a matching cached result is found, the cached objectives and constraints are reused without calling the wrapped evaluator. Only uncached evaluations are forwarded to the underlying evaluator.

Cache matching works as follows: for each requested variable vector and realization, the evaluator searches through the "results" stored by its sources. A match is found when the variables are equal (within floating-point tolerance) and the realization matches. If realization names are configured, they are used for matching (allowing cache hits across different optimization runs with the same realization names). Otherwise, realization indices are used.

If some but not all evaluations are found in cache, the cached ones are marked as inactive and only the missing evaluations are delegated to the wrapped evaluator. The final combined result contains both cached and newly computed values.

Sources can be added dynamically with add_sources().

To record which evaluations were served from cache, pass a hits_key string at construction time. When set, the returned EvaluationBatchResult will contain a boolean NumPy array in its metadata dictionary under that key — True for evaluations that came from the cache, False for those that were freshly computed.

The eval_cached() method is available for derived classes that need access to which evaluations were cache hits — it returns both the EvaluationBatchResult and a dictionary mapping evaluation indices to their cached FunctionResults.

ParallelEvaluator

The evaluators above run each function call sequentially in the current thread. For parallel evaluation — whether via worker threads, separate processes, or an HPC cluster — a parallel evaluator is needed. See Parallel Evaluation for details on ParallelEvaluator and the available executors.

Reusing objectives and constraints

When defining multiple objectives, you may need to reuse the same underlying computation. For example, a total objective could consist of the mean of the realizations plus their standard deviation. Rather than evaluating all realizations twice, compute them once and return the values for both objectives from a single evaluator call.

Where to next

• Parallel Evaluation — run evaluations off-process or on a cluster.
• Using the Workflow Framework — step-by-step example building a workflow from scratch.
• Full example: examples/workflow.py.