Skip to content

Working with Results

ropt exposes the full intermediate and final state of an optimization through Results objects. This page describes the result classes and how to inspect them; see Running Optimizations and Optimization Workflows for how results are produced and delivered to your code.

The result hierarchy

During optimization, function and gradient evaluations generate data that is reported via EnOptEvent objects passed to callbacks.

Each Results object represents the outcome of the calculation for a single variable vector — that is, the objective and gradient values computed at one point in variable space. However, the optimizer may request evaluations at multiple variable vectors in a single batch (for example, multiple perturbations or multiple candidates in a gradient-free method). In that case, the event payload contains a sequence of Results objects, one per variable vector evaluated in that batch.

Two concrete subclasses exist:

Each carries nested ResultField objects:

Result Fields
FunctionResults variables, target_objective, evaluations (FunctionEvaluations), functions (Functions), realizations (Realizations), constraint_info (ConstraintInfo), scaled (ScaledFunctionResults).
GradientResults variables, perturbed_variables, target_gradient, evaluations (GradientEvaluations), gradients (Gradients), scaled (ScaledGradientResults).

What each field holds

FunctionResults fields

  • variables — the variable vector that was evaluated, shape \((n_v,)\).
  • target_objective — the single weighted scalar the optimizer minimizes (0-D array), or None if no aggregate could be formed. Always in the domain the optimizer works in; see Scaling of results.
  • evaluations (FunctionEvaluations) — the raw per-realization values returned by the evaluator:
    • objectives: objective values per realization, shape \((n_r, n_o)\).
    • constraints: constraint values per realization, shape \((n_r, n_c)\) (only present when nonlinear constraints are configured).
    • metadata: optional dict of per-realization metadata arrays, each of shape \((n_r,)\).
  • functions (Functions) — aggregated values derived from the per-realization evaluations (or None if all realizations failed):
    • objectives: individual objective values, shape \((n_o,)\).
    • constraints: individual constraint values, shape \((n_c,)\) (if configured).
  • realizations (Realizations) — ensemble metadata:
    • evaluated_realizations: boolean array indicating which realizations were evaluated, shape \((n_r,)\).
    • objective_weights: per-realization objective weights, shape \((n_o, n_r)\). May change during optimization (for example, when realization filters are active).
    • constraint_weights: per-realization constraint weights, shape \((n_c, n_r)\) (if constraints are configured).

  • constraint_info (ConstraintInfo) — constraint bound information. Present when bounds or constraints are defined. Contains two kinds of data for each constraint type (bound, linear, and nonlinear):

    • Differences: the signed distance between the current value and each bound. For lower bounds, a negative difference means the value is below the bound (violated). For upper bounds, a positive difference means the value is above the bound (violated).
    • Violations: the absolute magnitude of any bound exceedance, or zero when the constraint is satisfied. For example, if a constraint requires \(g(\mathbf{x}) \leq 0\) and the actual value is \(0.5\), the violation is \(0.5\).

    See the ConstraintInfo reference for the full list of fields.

GradientResults fields

  • variables — the unperturbed variable vector, shape \((n_v,)\).
  • perturbed_variables — perturbed variable values, shape \((n_r, n_p, n_v)\).
  • target_gradient — the gradient the optimizer descends, shape \((n_v,)\), or None if estimation failed. Always in the domain the optimizer works in.
  • evaluations (GradientEvaluations) — the raw per-perturbation values returned by the evaluator:
    • perturbed_objectives: objective values for each perturbation, shape \((n_r, n_p, n_o)\).
    • perturbed_constraints: constraint values for each perturbation, shape \((n_r, n_p, n_c)\) (if configured).
    • metadata: optional dict of per-realization/perturbation metadata arrays, each of shape \((n_r, n_p)\).
  • gradients (Gradients) — aggregated gradient values (or None if estimation failed):
    • objectives: per-objective gradients, shape \((n_o, n_v)\).
    • constraints: per-constraint gradients, shape \((n_c, n_v)\) (if configured).
  • realizations (Realizations) — same structure as for FunctionResults (see above).

In the shapes above: \(n_v\) = number of variables, \(n_o\) = number of objectives, \(n_c\) = number of nonlinear constraints, \(n_r\) = number of realizations, \(n_p\) = number of perturbations. All values are NumPy arrays.

Common attributes on all results

Every Results object carries:

  • batch_id: an integer identifying the evaluation batch (potentially generated by the evaluator).
  • metadata: a dictionary of additional information generated during optimization. Not interpreted by ropt — useful for reporting and analysis.
  • names: a mapping from axis name to label tuples. Keys are AxisName values, or the name of a metadata key that defines a user axis. Used to produce labelled multi-index DataFrames when exporting (see Exporting to pandas).

Accessing result data

Common access patterns:

result.variables                   # variable vector evaluated
result.target_objective            # weighted scalar objective
result.functions.objectives        # per-objective values (after weighting)
result.functions.constraints       # per-constraint values

If functions is None, the result represents a request that produced no valid values (for example, all realizations failed). target_objective is None exactly then, so a single guard covers both:

if result.functions is not None:
    print(result.target_objective)

Axes and dimensionality

Much of the data within result objects is multi-dimensional. For example, the objectives field within FunctionEvaluations is a 2-D array where each row is a realization and each column is an objective.

To simplify exporting and reporting, the identity of each dimension is stored as axis metadata on each field. The ResultField base class provides a get_axes class method for retrieving this metadata:

from ropt.results import FunctionEvaluations

FunctionEvaluations.get_axes("objectives")
# (<AxisName.REALIZATION: 'realization'>, <AxisName.OBJECTIVE: 'objective'>)

The AxisName enumeration defines:

Axis name Meaning
VARIABLE Index corresponds to the variable number as defined in VariablesConfig.
OBJECTIVE Index corresponds to the objective number (position in the weights array of ObjectiveFunctionsConfig).
NONLINEAR_CONSTRAINT Index corresponds to the nonlinear constraint number as defined in NonlinearConstraintsConfig.
LINEAR_CONSTRAINT Index corresponds to the linear constraint number as defined in LinearConstraintsConfig.
REALIZATION Index corresponds to the realization number in the ensemble. Present whenever results involve multiple realizations.
PERTURBATION Index corresponds to a perturbation used for gradient estimation. Present in GradientEvaluations where objectives and constraints are reported for each perturbed variable set.

The dimensionality and order of axes for each field are fixed — they are listed in the "Result descriptions" section of each class in the reference.

User-defined axes

Per-realization metadata is the one place where you can add an axis of your own. If the evaluation function returns an array instead of a scalar for a metadata key, that key gains one extra axis, named after the key itself:

return EvaluationFunctionResult(objectives=value, metadata={"residual": residual})

With three realizations and a residual of length three, result.evaluations.metadata["residual"] has shape \((3, 3)\) and axes ("realization", "residual"). Label the new axis by adding an entry to names under the same key:

"names": {"residual": ("x", "y", "z")},

Every realization must return the same number of entries for a key, and a key may not be named after an AxisName value or batch_id; both raise a ValueError. Unlabelled user axes fall back to integer indices, as builtin axes do.

Note

Dimensionality is fixed: even with a single objective, result arrays still include an OBJECTIVE axis of length one.

Scaling of results

Optimization internally works with scaled values: variables are scaled and shifted by their scales and offsets, objective and nonlinear constraint aggregates have their offsets subtracted and are divided by their scales, and objectives marked maximize are negated once they have been combined across realizations.

Every result carries both domains at once, and one rule connects them:

Note

scaled.X is the optimizer's version of X, at the same path. Fields without a scaled counterpart have only one domain.

So result.variables is the variable vector as configured and result.scaled.variables is the same vector as the optimizer proposed it; result.functions.objectives and result.scaled.functions.objectives are the same pair for the aggregates.

Two groups of fields have a single domain:

  • The per-realization values in evaluations are reported exactly as the evaluator returned them. Scales apply to the quantities the optimizer consumes, and the optimizer never sees a single realization, so there is nothing to undo for those fields.
  • target_objective and target_gradient exist only in the domain the optimizer works in. Each is a weighted total over objectives that may differ in both scale and direction, so there is no single factor to undo. The gradient is differentiated with respect to the scaled variables. If you need either in configured terms, combine the objectives yourself using get_objective_scales and the directions on objectives.maximize.

Because the direction is undone when reporting, a combined objective agrees in sign with the per-realization values it summarizes, whether it is an average or a spread.

Metadata

Results carry two independent kinds of metadata, neither interpreted by ropt:

  • Result metadata — the metadata dict on every Results object, identical for every result of a run. It is set once when the run starts: pass a metadata dict to the simple-API optimize / optimize_many / evaluate functions (or to the low-level compute step). Use it to tag or identify a run, for example {"run_id": 7}.
  • Per-realization metadata — the metadata dict on the evaluations field, with one array entry per realization. It is produced by the objective when it returns an EvaluationFunctionResult with a metadata field.

Result metadata is passed to the run and read back from metadata:

result = optimize(config, x0, objective, metadata={"run_id": 7})
result.results.metadata            # {'run_id': 7}

Per-realization metadata is returned by the objective and read back from the evaluations field, with one entry per realization:

def objective(variables, context):
    ...
    return EvaluationFunctionResult(objectives=value, metadata={"shift": shift})


result.results.evaluations.metadata   # {'shift': array([...])}

The full runnable script is examples/simple/metadata.py.

Exporting to pandas

ropt can export results to pandas DataFrames for analysis and reporting. This requires the pandas optional extra (see Installation).

Note

The to_pandas and results_to_pandas functions shown here are the low-level export primitives. Most users do not call them directly: the DataFrameHandler builds and updates these tables automatically as an optimization runs. This section explains what that handler produces under the hood.

The row index and the unstacked column labels come from the names mapping in the configuration. If an axis is not named, its labels fall back to 0-based integer indices. For example, exporting the objectives of a single result without any names gives plain numbers for both the realization and the objective axes:

df = result.to_pandas(["evaluations.objectives"])

                               evaluations.objectives
batch_id realization objective
1        0           0                           2.10
                     1                           0.94
         1           0                           2.35
                     1                           1.02

Adding a names entry replaces those numbers with meaningful labels. The examples below assume the realizations are named "r0"/"r1" and the objectives "val"/"cost".

Exporting selected fields

The to_pandas method on an individual result exports any set of fields, each named by a dotted path from the result:

df = result.to_pandas(["variables", "evaluations.objectives"])

A path may name a field of the result itself ("variables", "target_objective"), a field of one of its sub-objects ("functions.objectives", "scaled.variables"), or an entry of a dict-valued field ("metadata.run.id"). Each path becomes a column of that name. Paths whose value is None, and missing dict keys, are skipped.

By default, every axis of the exported fields becomes a level in a multi-index. For example, objectives in FunctionEvaluations has the axes REALIZATION and OBJECTIVE, so exporting it keeps both in the index — now with the configured names:

df = result.to_pandas(["evaluations.objectives"])

                               evaluations.objectives
batch_id realization objective
1        r0          val                         2.10
                     cost                        0.94
         r1          val                         2.35
                     cost                        1.02

Passing unstack pivots selected axes out of the index and into columns. Here the OBJECTIVE axis is unstacked:

from ropt.enums import AxisName

df = result.to_pandas(
    ["evaluations.objectives"],
    unstack=[AxisName.OBJECTIVE],
)

                     (evaluations.objectives, val)  (evaluations.objectives, cost)
batch_id realization
1        r0                                   2.10                            0.94
         r1                                   2.35                            1.02

The unstacked axis is flattened into the column labels, so each new column is a tuple of the sub-field name and the axis label — here ("objectives", "val") and ("objectives", "cost"). Unstacking more axes adds more elements to these tuples; unstacking every axis leaves a flat table with one row per result.

Aggregating multiple results

results_to_pandas builds on to_pandas to convert a sequence of results into a single DataFrame, one row per result. It automatically unstacks the most common axes (VARIABLE, OBJECTIVE, NONLINEAR_CONSTRAINT) into columns:

from ropt.results import results_to_pandas

df = results_to_pandas(
    all_results,
    fields={"variables"},
    result_type="functions",
)

          (variables, x0)  (variables, x1)  (variables, x2)
batch_id
1                                0.30                         0.42                        -0.11
2                                0.55                         0.48                         0.02
3                                0.61                         0.50                         0.10

Each column is a (field, label) pair, and each row is one result identified by its batch_id. Field names use dot notation for nested sub-fields (for example, variables, target_objective). The result_type argument selects which results to process: "functions" for FunctionResults only, "gradients" for GradientResults only.

Metadata columns

Both kinds of metadata are reached by the same dotted paths, since a path may end in one or more dict keys.

The per-realization metadata attached by the evaluator lives on the evaluations, so it keeps the realization axis. For example, if the objective attached a per-realization shift:

df = result.to_pandas(["evaluations.metadata.shift"])

                     evaluations.metadata.shift
batch_id realization
1        r0                                 0.9
         r1                                 1.1

The run-level result metadata sits directly on the result, so it has no axes and gives one value per result — handy for pulling in a run tag. It may be nested to any depth:

df = results_to_pandas(
    all_results,
    fields={"metadata.run_id", "target_objective"},
    result_type="functions",
)

          target_objective  metadata.run_id
batch_id
1                               1.83                0
2                               0.42                1

Labels and the index

Every axis of an exported field becomes an index level, named after its AxisName value (for example "variable", "realization", "objective"), and batch_id is always prepended so results from different batches stay distinct. The label on each level — and on each unstacked column — comes from the names mapping in the configuration, a dict from axis name to a tuple of labels:

CONFIG = {
    ...
    "names": {
        "variable": ("x0", "x1", "x2"),
        "objective": ("val", "cost"),
    },
}

An axis without a names entry falls back to 0-based integer indices, as in the first example of this section.

Exporting to polars

Every pandas export has a polars counterpart: to_polars and results_to_polars. They accept the same arguments and select and unstack exactly the same fields, so everything in the previous section carries over. This requires the polars optional extra (see Installation).

Polars has no index and its column names must be strings, which leads to the two differences you need to know about.

Index levels become ordinary columns. What pandas puts in the index, polars puts in leading columns of the frame:

df = result.to_polars(["evaluations.objectives"])

┌──────────┬─────────────┬───────────┬────────────────────────┐
│ batch_id ┆ realization ┆ objective ┆ evaluations.objectives │
╞══════════╪═════════════╪═══════════╪════════════════════════╡
│ 1        ┆ r0          ┆ val       ┆ 2.10                   │
│ 1        ┆ r0          ┆ cost      ┆ 0.94                   │
│ 1        ┆ r1          ┆ val       ┆ 2.35                   │
│ 1        ┆ r1          ┆ cost      ┆ 1.02                   │
└──────────┴─────────────┴───────────┴────────────────────────┘

Tuple column labels become joined strings. Where pandas produces the column ("evaluations.objectives", "val"), polars produces "evaluations.objectives,val". The separator is configurable with the sep argument, which defaults to ",":

from ropt.enums import AxisName

df = result.to_polars(
    ["evaluations.objectives"],
    unstack=[AxisName.OBJECTIVE],
)

┌──────────┬─────────────┬────────────────────────────┬─────────────────────────────┐
│ batch_id ┆ realization ┆ evaluations.objectives,val ┆ evaluations.objectives,cost │
╞══════════╪═════════════╪════════════════════════════╪═════════════════════════════╡
│ 1        ┆ r0          ┆ 2.10                       ┆ 0.94                        │
│ 1        ┆ r1          ┆ 2.35                       ┆ 1.02                        │
└──────────┴─────────────┴────────────────────────────┴─────────────────────────────┘

Aggregating a sequence of results works the same way:

from ropt.results import results_to_polars

df = results_to_polars(
    all_results,
    fields={"variables"},
    result_type="functions",
)

┌──────────┬──────────────────────────┬──────────────────────────┬──────────────────────────┐
│ batch_id ┆ variables,x0             ┆ variables,x1             ┆ variables,x2             │
╞══════════╪══════════════════════════╪══════════════════════════╪══════════════════════════╡
│ 1        ┆ 0.30                     ┆ 0.42                     ┆ -0.11                    │
│ 2        ┆ 0.55                     ┆ 0.48                     ┆ 0.02                     │
│ 3        ┆ 0.61                     ┆ 0.50                     ┆ 0.10                     │
└──────────┴──────────────────────────┴──────────────────────────┴──────────────────────────┘

Metadata behaves exactly as described above: per-realization metadata is reachable from to_polars, run-level result metadata only from results_to_polars:

df = results_to_polars(
    all_results,
    fields={"metadata.run_id", "target_objective"},
    result_type="functions",
)

┌──────────┬────────────────────────────┬─────────────────┐
│ batch_id ┆ target_objective ┆ metadata.run_id │
╞══════════╪════════════════════════════╪═════════════════╡
│ 1        ┆ 1.83                       ┆ 0               │
│ 2        ┆ 0.42                       ┆ 1               │
│ 3        ┆ 0.11                       ┆ 2               │
└──────────┴────────────────────────────┴─────────────────┘

Note

Because polars keeps the keys as real columns, it can join fields that vary at different granularities — for example a per-batch gradient with per-perturbation evaluations — by repeating the coarser values across the finer rows. Pandas cannot align such fields and returns them as disjoint blocks of rows padded with missing values instead, so prefer polars when a single table has to mix granularities.

Where to next