Skip to content

Simple API

ropt.simple

The high-level convenience API for running optimizations.

This module builds on the low-level ropt primitives. Import its names directly, for example from ropt.simple import optimize, session. See Running Optimizations for a walkthrough.

Enumerations used in the configuration and results (for example ExitCode and VariableType) are not re-exported here; import them from ropt.enums.

Nothing about a run depends on where it is called from. Where its evaluations happen is decided by the pool it is given with pool=, and which handlers see its results by the handlers= it is given. A session hands out both; a run given no pool evaluates in-process. This holds wherever the run is started from, including a thread you spawn yourself.

Running optimizations

ropt.simple.optimize

optimize(
    config: dict[str, Any],
    x0: ArrayLike,
    function: EvaluationFunction,
    *,
    pool: WorkerPool | None = None,
    handlers: Sequence[EventHandler | SharedHandlers]
    | None = None,
    report: ReportCallback | None = None,
    constraint_tolerance: float = 1e-10,
    metadata: dict[str, Any] | None = None,
) -> OptimizeResult

Run a single optimization.

See Running Optimizations for a walkthrough.

A pool or group that is closed — because it was closed directly, or because its session ended — is refused here with a WorkflowError, as is one carried into a worker process, where it cannot work at all.

Parameters:

Name Type Description Default
config dict[str, Any]

The optimization configuration.

required
x0 ArrayLike

The initial variable vector.

required
function EvaluationFunction

The per-realization evaluation function.

required
pool WorkerPool | None

The pool to evaluate on, from a session factory such as thread_pool. Without one the evaluations run in-process, on the calling thread. A run started from inside an evaluation needs a different pool: the pool it is already running on refuses the work.

None
handlers Sequence[EventHandler | SharedHandlers] | None

Optional result handlers, mixing two kinds. An EventHandler is local: it is claimed for the duration of this run, and may be reused by a later run to accumulate results, but not shared with a concurrent one, and never afterwards with a SharedHandlers group. A group is shared: this run feeds it alongside every other run that lists it.

None
report ReportCallback | None

An optional callback invoked with an EvaluateResult for each function evaluation; return True from it to stop the optimization early with USER_ABORT. Reporting stops there, so results after it in the same batch are not passed on.

None
constraint_tolerance float

The tolerance within which a constraint is considered satisfied. Violations are compared in the domain the optimizer works in, so a scale applies to them as well.

1e-10
metadata dict[str, Any] | None

An optional dictionary attached to every Results this run emits, for example to tag or identify the run. It also reaches function as context.metadata.

None

Returns:

Type Description
OptimizeResult

A OptimizeResult describing the outcome.

ropt.simple.optimize_many

optimize_many(
    config: dict[str, Any] | Sequence[dict[str, Any]],
    x0: ArrayLike,
    function: EvaluationFunction
    | Sequence[EvaluationFunction],
    *,
    pool: WorkerPool | None = None,
    handlers: Sequence[SharedHandlers] | None = None,
    report: ReportCallback
    | Sequence[ReportCallback]
    | None = None,
    limit: int | None = None,
    constraint_tolerance: float = 1e-10,
    metadata: dict[str, Any]
    | Sequence[dict[str, Any]]
    | None = None,
) -> tuple[OptimizeResult, ...]

Run several optimizations concurrently, sharing one pool.

Each of config, x0, and function may be a single value (used for every run) or a sequence (one per run). Sequences set the number of runs and must agree in length; single values are broadcast. A single x0 is a 1-D vector; a per-run sequence of x0s is a 2-D matrix with one vector per row. A sequence that is empty gives no runs, and returns no results.

The runs execute concurrently on driver threads and all evaluate on the same pool, so its workers are shared between them; limit bounds how many run simultaneously. Without a pool the runs still overlap, but each evaluation runs in-process on its own driver thread. See Parallel Execution and Many Runs for a walkthrough, and Failure in one run for what happens when one raises.

Unlike optimize, this takes no local handlers: a local handler belongs to one run at a time and these runs overlap, so handlers= accepts only SharedHandlers groups, which every run feeds together. report=, being local by nature, is the opposite: it is given per run, or broadcast to all of them.

A pool or group that is closed — because it was closed directly, or because its session ended — is refused here with a WorkflowError, as is one carried into a worker process, where it cannot work at all.

Parameters:

Name Type Description Default
config dict[str, Any] | Sequence[dict[str, Any]]

The configuration, or one per run.

required
x0 ArrayLike

The initial variable vector, or one per row.

required
function EvaluationFunction | Sequence[EvaluationFunction]

The evaluation function, or one per run.

required
pool WorkerPool | None

The pool every run evaluates on, from a session factory such as thread_pool. Without one the evaluations run in-process, each on its own driver thread. A run started from inside an evaluation needs a different pool: the pool it is already running on refuses the work.

None
handlers Sequence[SharedHandlers] | None

Optional SharedHandlers groups, fed by every run.

None
report ReportCallback | Sequence[ReportCallback] | None

An optional callback invoked with an EvaluateResult for each function evaluation, either shared by every run or one per run; return True from it to stop that run early with USER_ABORT.

None
limit int | None

The maximum number of runs to execute at once.

None
constraint_tolerance float

The tolerance within which a constraint is considered satisfied. Violations are compared in the domain the optimizer works in, so a scale applies to them as well.

1e-10
metadata dict[str, Any] | Sequence[dict[str, Any]] | None

An optional dictionary attached to every Results a run emits, shared by all runs or given one per run — for example to tag each run with {"run_id": i}. It also reaches each run's function as context.metadata.

None

Returns:

Type Description
tuple[OptimizeResult, ...]

One OptimizeResult per run, in order.

Raises:

Type Description
WorkflowError

If handlers holds a handler that is not in a group.

Evaluating without optimizing

ropt.simple.evaluate

evaluate(
    config: dict[str, Any],
    variables: ArrayLike,
    function: EvaluationFunction,
    *,
    pool: WorkerPool | None = None,
    handlers: Sequence[EventHandler | SharedHandlers]
    | None = None,
    report: ReportCallback | None = None,
    metadata: dict[str, Any] | None = None,
) -> EvaluateResult

Evaluate a single variable vector without optimizing.

Use evaluate_many to evaluate several vectors at once. See Running Optimizations for a walkthrough.

A pool or group that is closed — because it was closed directly, or because its session ended — is refused here with a WorkflowError, as is one carried into a worker process, where it cannot work at all.

Parameters:

Name Type Description Default
config dict[str, Any]

The optimization configuration.

required
variables ArrayLike

The variable vector to evaluate.

required
function EvaluationFunction

The per-realization evaluation function.

required
pool WorkerPool | None

The pool to evaluate on, from a session factory such as thread_pool. Without one the evaluations run in-process, on the calling thread. A run started from inside an evaluation needs a different pool: the pool it is already running on refuses the work.

None
handlers Sequence[EventHandler | SharedHandlers] | None

Optional result handlers, mixing local EventHandler objects with shared SharedHandlers groups, as optimize takes them.

None
report ReportCallback | None

An optional callback invoked with an EvaluateResult for each evaluation. An evaluation is a single batch that has already run by the time the callback sees it, and there is no optimizer loop to interrupt, so unlike on optimize returning True cannot stop anything: it only ends the reporting, and every result is still returned.

None
metadata dict[str, Any] | None

An optional dictionary attached to the emitted Results. It also reaches function as context.metadata.

None

Returns:

Type Description
EvaluateResult

An EvaluateResult for the vector.

Raises:

Type Description
ValueError

If variables is not a single vector.

ropt.simple.evaluate_many

evaluate_many(
    config: dict[str, Any],
    variables: ArrayLike,
    function: EvaluationFunction,
    *,
    pool: WorkerPool | None = None,
    handlers: Sequence[EventHandler | SharedHandlers]
    | None = None,
    report: ReportCallback | None = None,
    metadata: dict[str, Any] | None = None,
) -> tuple[EvaluateResult, ...]

Evaluate a batch of variable vectors without optimizing.

Each row of variables is one variable vector; the results are returned in the same order. See Running Optimizations for a walkthrough.

A pool or group that is closed — because it was closed directly, or because its session ended — is refused here with a WorkflowError, as is one carried into a worker process, where it cannot work at all.

Parameters:

Name Type Description Default
config dict[str, Any]

The optimization configuration.

required
variables ArrayLike

The variable vectors to evaluate, one per row.

required
function EvaluationFunction

The per-realization evaluation function.

required
pool WorkerPool | None

The pool to evaluate on, from a session factory such as thread_pool. Without one the evaluations run in-process, on the calling thread. A run started from inside an evaluation needs a different pool: the pool it is already running on refuses the work.

None
handlers Sequence[EventHandler | SharedHandlers] | None

Optional result handlers, mixing local EventHandler objects with shared SharedHandlers groups, as optimize takes them.

None
report ReportCallback | None

An optional callback invoked with an EvaluateResult for each evaluation. An evaluation is a single batch that has already run by the time the callback sees it, and there is no optimizer loop to interrupt, so unlike on optimize returning True cannot stop anything: it only ends the reporting, and every result is still returned.

None
metadata dict[str, Any] | None

An optional dictionary attached to every emitted Results. It also reaches function as context.metadata.

None

Returns:

Type Description
tuple[EvaluateResult, ...]

One EvaluateResult per input vector.

Raises:

Type Description
ValueError

If variables is not a 2-D matrix.

Sessions and pools

ropt.simple.session

session() -> Session

Open a background session that pools and shared handlers run on.

The session owns one event loop, on a daemon thread, for as long as the block is open. Build pools on it with its factories, and pass them to the runs that should use them:

with session() as s:
    fast = s.thread_pool(workers=8)
    optimize(config, x0, function, pool=fast)

Closing the session releases every pool it created, so most code needs no further cleanup. See Running Optimizations for a walkthrough.

Returns:

Type Description
Session

A context manager owning the session, which binds the

Session

Session itself when used with as.

ropt.simple.Session

An open session, and the factories that build on it.

A session owns one background event loop, on its own daemon thread. Pools need that loop to run on, which is why they are built here rather than constructed on their own. Everything a session hands out is returned to the caller and passed on explicitly; nothing is discovered from the surroundings.

Bind the session with as and call its factories inside the block:

with session() as s:
    fast = s.thread_pool(workers=8)
    optimize(config, x0, function, pool=fast)

Sessions are objects, so opening one inside another is unremarkable: each gets its own loop, and pools from different sessions never interact. A session is single use — once closed it cannot be reopened.

__init__

__init__() -> None

Initialize the session.

__enter__

__enter__() -> Self

Open the session's event loop.

Returns:

Type Description
Self

The session itself.

Raises:

Type Description
WorkflowError

If the session was already opened.

__exit__

__exit__(*_exc: object) -> None

Close the session, releasing every pool it created.

thread_pool

thread_pool(
    *, workers: int = 1, bundle_size: int = 1
) -> WorkerPool

Create a pool that runs evaluations in worker threads.

See Running Optimizations for a walkthrough.

Parameters:

Name Type Description Default
workers int

The number of worker threads.

1
bundle_size int

How many evaluations go to a worker as one task, 0 for the whole batch. See process_pool, where it matters more.

1

Returns:

Type Description
WorkerPool

A pool backed by a thread pool.

process_pool

process_pool(
    *, workers: int = 1, bundle_size: int = 1
) -> WorkerPool

Create a pool that runs evaluations in worker processes.

The evaluation function must be picklable. See Running Optimizations for a walkthrough.

Parameters:

Name Type Description Default
workers int

The number of worker processes.

1
bundle_size int

How many evaluations go to a worker as one task. Every task is transferred to a worker separately, and the evaluations within one run after another, so this is a trade between spreading a batch and the cost of moving it. The default of 1 gives every evaluation its own task, spreading a batch as widely as the workers allow; a larger value groups that many per task; and 0 sends the whole batch as a single task, which suits a pool whose parallelism comes from the runs above it rather than from within a batch.

1

Returns:

Type Description
WorkerPool

A pool backed by a process pool.

local_pool

local_pool(
    *,
    workers: int = 1,
    workdir: Path | str | None = None,
    retries: int = 0,
    bundle_size: int = 1,
) -> WorkerPool

Create a pool that runs each evaluation as a separate local process.

Between a process pool and a cluster: every evaluation gets an interpreter of its own, which can be stopped outright and whose output is captured to a file, but there is no queueing system and nothing to install. This is the local stand-in for hpc_pool: the same job shape, so an evaluation function that works here works there.

Each job is a fresh command rather than a re-import of your script, so the evaluation function must live in a module the job can import, or the ropt[cloudpickle] extra must be installed — which is the recommended way to use this pool.

POSIX only. See Running Optimizations for a walkthrough.

Parameters:

Name Type Description Default
workers int

The maximum number of concurrent local jobs.

1
workdir Path | str | None

The directory holding each evaluation's files. The default is a temporary directory that is removed when the pool closes — unless an evaluation failed, in which case it is kept, with that evaluation's output in it, and its path logged.

None
retries int

Extra polls to wait for a result. The default of 0 is enough: a local job writes its result before it exits.

0
bundle_size int

How many evaluations go to a worker as one task, 0 for the whole batch. See process_pool; each task here is a local process.

1

Returns:

Type Description
WorkerPool

A pool backed by local processes.

hpc_pool

hpc_pool(
    *,
    workers: int = 1,
    cores: int = 1,
    cluster: str | None = None,
    queue: str | None = None,
    workdir: Path | str | None = None,
    config_path: Path | str | None = None,
    template: str | None = None,
    scheduler: str | None = None,
    memory_max: int | str | None = None,
    run_time_max: int | None = None,
    submit_options: dict[str, Any] | None = None,
    retries: int = 30,
    bundle_size: int = 1,
) -> WorkerPool

Create a pool that runs evaluations on an HPC cluster.

Interfaces with a cluster queue (for example Slurm) through pysqa; requires the ropt[hpc] extra. Each evaluation is a job started as its own command, so the evaluation function must live in a module the compute nodes can import, or the ropt[cloudpickle] extra must be installed — which is the recommended way to use this pool. Develop against local_pool first: it has the same shape and the same rule, without a cluster. The cluster is selected from cluster/queue: give a queue to search for its cluster, a cluster to use its default queue, or both to be explicit.

A template is the alternative to all of that: it submits without a configuration, so it cannot be combined with config_path, cluster or queue. See Running on an HPC cluster for the configuration layout and Slurm examples.

Parameters:

Name Type Description Default
workers int

The maximum number of concurrent cluster jobs.

1
cores int

The number of CPUs per job.

1
cluster str | None

The cluster name.

None
queue str | None

The queue name.

None
workdir Path | str | None

The shared-filesystem working directory.

None
config_path Path | str | None

The pysqa configuration directory.

None
template str | None

A submission-script template.

None
scheduler str | None

The queueing system a template is written for.

None
memory_max int | str | None

The memory per job.

None
run_time_max int | None

The run time per job.

None
submit_options dict[str, Any] | None

Extra variables for the submission script.

None
retries int

Number of retries for polling the cluster for results.

30
bundle_size int

How many evaluations go to a worker as one task, 0 for the whole batch. See process_pool; each task here is a cluster job.

1

Returns:

Type Description
WorkerPool

A pool backed by an HPC cluster.

serial_pool

serial_pool() -> WorkerPool

Create a pool that evaluates in-process, on the calling thread.

Identical to the free serial_pool function, except that this pool is closed when the session closes, so runs cannot keep using it afterwards. Prefer the free function for a pool that should outlive any session.

Returns:

Type Description
WorkerPool

A pool without an executor.

shared_handlers

shared_handlers(
    *handler: EventHandler,
    threaded: EventHandler | Sequence[EventHandler] = (),
    report: ReportCallback | None = None,
) -> SharedHandlers

Group result handlers that several runs share.

Pass the group to every run that should feed it, in handlers=. Each handler then sees the results of all those runs, serialized across them, which is what makes accumulating over concurrent runs safe. A run may feed several groups, and mix them with handlers of its own.

A handler joins one group at a time, and a handler that was ever passed to a run as a local handler cannot join a group at all; decide per handler whether it is local or shared. See Running Optimizations for a walkthrough.

Parameters:

Name Type Description Default
handler EventHandler

The result handlers to share, each run on the session's event-loop thread.

()
threaded EventHandler | Sequence[EventHandler]

Handlers (one, or a sequence) to run on a worker thread instead of the loop. This only helps handlers that spend real time in blocking, GIL-releasing I/O (files, databases, network); for in-memory work it gives no benefit under CPython's GIL. See Result Handlers.

()
report ReportCallback | None

An optional callback invoked with an EvaluateResult for each function evaluation across the group's runs. Returning True stops the emitting run early with USER_ABORT if it is an optimization; an evaluation has no optimizer loop to interrupt, so there the return value is ignored.

None

Returns:

Type Description
SharedHandlers

A SharedHandlers group.

ropt.simple.WorkerPool

The workers that run evaluations, and the batch IDs they share.

Created by a session factory such as thread_pool. Every run using the same pool draws its batch IDs from the same counter, so concurrent runs never produce the same batch ID.

A pool lives until its session closes, which releases it. Release it earlier with close, or by using it as a context manager. See Running Optimizations for a walkthrough.

A pool built by serial_pool has no executor and no workers to release; it exists so that runs sharing one batch-ID sequence can say so, whether or not they run in parallel.

__init__

__init__(
    executor: Executor | None = None,
    session: _Session | None = None,
    bundle_size: int = 1,
) -> None

Initialize the pool.

Parameters:

Name Type Description Default
executor Executor | None

The started executor, or None to evaluate in-process.

None
session _Session | None

The session that owns the pool, if it has one.

None
bundle_size int

Evaluations per worker task, 0 for the whole batch.

1

Raises:

Type Description
ValueError

If bundle_size is negative.

closed property

closed: bool

Whether the pool has been released.

Returns:

Type Description
bool

True once the pool, or the session that built it, was closed.

executor property

executor: Executor | None

The executor that runs this pool's evaluations.

Returns:

Type Description
Executor | None

The executor, or None for a serial pool.

batch_ids property

batch_ids: BatchIdCounter

The counter every run on this pool draws its batch IDs from.

Returns:

Type Description
BatchIdCounter

The batch ID counter.

bundle_size property

bundle_size: int

How many evaluations are sent to a worker as one task.

Returns:

Type Description
int

The number of evaluations per task, 0 meaning the whole batch.

close

close() -> None

Release the pool's workers without waiting for the session to close.

See Releasing a pool early for when this matters and the resulting lifecycle.

__enter__

__enter__() -> Self

Enter a block that closes the pool on exit.

Returns:

Type Description
Self

The pool itself.

__exit__

__exit__(*_exc: object) -> None

Close the pool.

ropt.simple.serial_pool

serial_pool() -> WorkerPool

Create a pool that evaluates in-process, on the calling thread.

A serial pool has no workers: it carries only the batch-ID counter that the runs sharing it draw from. Use it to give concurrent runs one batch-ID sequence without running their evaluations in parallel, or as an explicit way to say that a run should evaluate in-process.

It needs no session, and needs no releasing.

Returns:

Type Description
WorkerPool

A pool without an executor.

Offloading work to a pool

ropt.simple.offload

offload(
    work: Callable[[], _T], *, pool: WorkerPool | None = ...
) -> _T
offload(
    work: Sequence[Callable[[], _T]],
    *,
    pool: WorkerPool | None = ...,
) -> tuple[_T, ...]
offload(
    work: Callable[[], _T] | Sequence[Callable[[], _T]],
    *,
    pool: WorkerPool | None = None,
) -> _T | tuple[_T, ...]

Offload one or more callables to a pool.

Pass a single zero-argument callable to run one call and return its result, or a sequence of callables to run them concurrently (they may be entirely different functions) and return a tuple of results in the order of work. Bind arguments with functools.partial.

Without a pool, or with a serial_pool, the callables run inline on the calling thread, one after another. Code that may or may not have a pool to hand therefore needs no fallback: pass whatever it has, including None. The callables must be picklable for a process or HPC pool.

See Running Optimizations for a walkthrough.

A handler in a shared group runs on the pool's own event loop and cannot wait on it; offloading from there raises a WorkflowError. So does a pool that is closed, or one carried into a worker process.

Parameters:

Name Type Description Default
work Callable[[], _T] | Sequence[Callable[[], _T]]

A single zero-argument callable, or a sequence of them.

required
pool WorkerPool | None

The pool to dispatch to, or None to run inline. Work offloaded from inside an evaluation needs a different pool: the pool it is already running on refuses it.

None

Returns:

Type Description
_T | tuple[_T, ...]

The single result, or a tuple of results in the order of work.

Aggregating results across runs

Session.shared_handlers builds the group; the group itself is a SharedHandlers object.

ropt.simple.SharedHandlers

A group of result handlers that several runs share.

Created by shared_handlers and passed to the runs that should feed it. Each handler in the group sees the results of every one of those runs, sequential or concurrent, and the group's dispatcher serializes them, so a handler that accumulates across runs needs no locking of its own.

A group lives until its session closes, which releases it and its handlers. Release it earlier with close, or by using it as a context manager. See Running Optimizations for a walkthrough.

__init__

__init__(
    entries: Sequence[tuple[EventHandler, bool]],
    session: _Session,
) -> None

Initialize the group and start its dispatcher.

Parameters:

Name Type Description Default
entries Sequence[tuple[EventHandler, bool]]

The handlers, each with whether to run it in a thread.

required
session _Session

The session whose event loop the dispatcher runs on.

required

closed property

closed: bool

Whether the group has been released.

Returns:

Type Description
bool

True once the group, or the session that built it, was closed.

attach_to

attach_to(step: ComputeStep[Any]) -> None

Forward the events of a run's compute step to this group's handlers.

A fresh forwarding handler is added per step (one handler cannot serve several steps), carrying only the event types the group's handlers want.

Parameters:

Name Type Description Default
step ComputeStep[Any]

The compute step whose events feed the shared handlers.

required

close

close() -> None

Release the group's handlers without waiting for the session to close.

See Sharing a handler across concurrent runs for when this matters and the resulting lifecycle.

__enter__

__enter__() -> Self

Enter a block that closes the group on exit.

Returns:

Type Description
Self

The group itself.

__exit__

__exit__(*_exc: object) -> None

Close the group.

Result objects

ropt.simple.EvaluateResult dataclass

The values one variable vector produced.

Returned by evaluate, once per input vector by evaluate_many, and handed to a report callback once per function evaluation of a run. See Running Optimizations for a walkthrough.

Attributes:

Name Type Description
variables NDArray[float64] | None

The variable vector these values belong to, or None.

target_objective float | None

The weighted objective, or None where the evaluation produced no valid result.

objectives NDArray[float64] | None

The individual objective values, shape (n_obj,), or None.

constraints NDArray[float64] | None

The constraint values, shape (n_con,), or None when there are no nonlinear constraints or the evaluation produced no valid result.

results FunctionResults | None

The full low-level FunctionResults object, or None. Only an optimization that reached no valid result leaves this empty; an evaluation and a report callback always have one.

ropt.simple.OptimizeResult dataclass

Bases: EvaluateResult

The outcome of a single optimization run.

An optimization ends at one evaluation, the best one it found, so this is that evaluation's EvaluateResult together with the exit code of the run that reached it. See Running Optimizations for a walkthrough.

Attributes:

Name Type Description
variables NDArray[float64] | None

The optimal variable vector, or None if no valid result was found.

target_objective float | None

The weighted objective at the optimum, or None.

objectives NDArray[float64] | None

The individual objective values at the optimum, shape (n_obj,), or None.

constraints NDArray[float64] | None

The nonlinear constraint values at the optimum, or None.

results FunctionResults | None

The full low-level FunctionResults object for the optimum, or None.

exit_code ExitCode

The exit code describing how the optimization terminated.

Callback types

ropt.simple.EvaluationFunction

Bases: Protocol

The call signature for a high-level evaluation function.

__call__

__call__(
    variables: NDArray[float64],
    context: EvaluationFunctionContext,
) -> FunctionValue

Evaluate the objectives and constraints for a single variable vector.

Parameters:

Name Type Description Default
variables NDArray[float64]

The 1-D variable vector for this evaluation.

required
context EvaluationFunctionContext

The context identifying the evaluation.

required

Returns:

Type Description
FunctionValue
FunctionValue

a scalar, or a flat sequence of objectives followed by constraints.

ropt.simple.ReportCallback module-attribute

ReportCallback = Callable[[EvaluateResult], bool | None]

Re-exported for convenience

These names are re-exported from ropt.simple (so simple-API code imports them from one place), but they are the low-level classes and are documented with the components: