Skip to content

Optimization Results

ropt.results

Data classes for storing intermediate optimization results.

During the optimization process, the calculation of functions and gradients generates data that needs to be reported. To facilitate this, new results are passed to callbacks as a sequence of Results objects. These objects can be instances of either the FunctionResults or GradientResults classes, which store the results of function and gradient evaluations, respectively.

Much of the data within these result objects is multi-dimensional. For example, the objectives field, which is part of the nested evaluations object within FunctionResults, is a two-dimensional numpy array. In this array, each column represents a different objective, and each row corresponds to a specific realization number.

To simplify exporting and reporting, the identity of the axes in these multi-dimensional arrays is stored as metadata associated with each field. These fields are derived from the ResultField class, which provides a get_axes class method for retrieving the axes. For instance, for the objectives field, this method would return:

>>> from ropt.results import FunctionEvaluations
>>> FunctionEvaluations.get_axes("objectives")
(<AxisName.REALIZATION: 'realization'>, <AxisName.OBJECTIVE: 'objective'>)
Given that the first axis denotes realizations and the second axis denotes objectives, each row in the array represents the set of objective values for a specific realization. This metadata provides the necessary context for exporting and reporting code to associate each element in the result matrix with its corresponding realization and objective, as specified in the optimizer configuration. The pandas exporting code, for example, utilizes this information to construct a multi-index for the output DataFrame and to transform the multi-dimensional data into multiple columns.

The AxisName enumeration currently defines the following axes:

  • AxisName.OBJECTIVE The index along this axis refers to the objective number as specified in the EnOptConfig configuration.
  • AxisName.NONLINEAR_CONSTRAINT The index along this axis corresponds to the non-linear constraint index defined in the EnOptConfig configuration.
  • AxisName.LINEAR_CONSTRAINT The index along this axis corresponds to the linear constraint index defined in the EnOptConfig configuration.
  • AxisName.VARIABLE The index along this axis refers to the variable number as specified by the EnOptConfig configuration.
  • AxisName.REALIZATION: When results involve an ensemble, this axis represents the different realizations, where the index corresponds to the realization number.
  • AxisName.PERTURBATION For gradient calculations, multiple variable perturbations are used. The objectives and constraints calculated for each perturbation are reported along this axis, which represents the perturbation index.

Refer to the documentation of the individual result classes for the exact dimensions of each result field. The dimensionality of the data and the order of the axes are fixed and listed sequentially for every field.

Note

The dimensionality associated with a result axis is fixed. For instance, even with only a single objective, results containing objective values will still include a AxisName.OBJECTIVE axis of length one.

ropt.results.Results dataclass

Bases: ABC

Abstract base class for storing optimization results.

The Results class serves as a foundation for storing various types of optimization results. It is not intended to be instantiated directly but rather serves as a base for derived classes like FunctionResults and GradientResults, which hold the actual data.

This class provides storage for the following generic information:

  • Batch ID: An optional identifier, potentially generated by the function evaluator, that uniquely identifies a group of function evaluations passed to the evaluator by teh optimizer.
  • Metadata: A dictionary for storing additional information generated during optimization. This metadata can include various primitive values that are not directly interpreted by the optimization code but are useful for reporting and analysis.
  • Names: The optional names attribute is a dictionary that stores the names of the various entities, such as variables, objectives, and constraints. The supported name types are defined in the AxisName enumeration. This information is optional, as it is not strictly necessary for the optimization, but it can be useful for labeling and interpreting results. For instance, when present, it is used to create a multi-index results that are exported as data frames.

The derived classes, FunctionResults and GradientResults, extend this base class with specific attributes for storing function evaluation results and gradient evaluation results, respectively. These derived classes also provide methods for exporting the stored data.

One key method provided by the Results class is to_dataframe, which allows exporting the contents of a specific field, or a subset of its sub-fields, to a pandas DataFrame for further data analysis and reporting.

Attributes:

Name Type Description
batch_id int | None

The ID of the evaluation batch.
metadata dict[str, Any]

A dictionary of metadata.
names dict[str, tuple[str | int, ...]]

Optional names of the various result axes.

transform_from_optimizer abstractmethod 

transform_from_optimizer(
    transforms: OptModelTransforms,
) -> Results

Transform results from the optimizer domain to the user domain.

During optimization, variables, objectives, and constraints are often transformed to a different domain (the optimizer domain) to enhance the performance and stability of the optimization algorithm. The Results objects produced during optimization are initially in the optimizer domain. This method reverses these transformations, mapping the results back to the user-defined domain. The transformations between the user and optimizer domains are defined by the classes in the ropt.transforms module.

For instance, variables might have been scaled and shifted to a range more suitable for the optimizer. This method, using the provided OptModelTransforms object, applies the inverse scaling and shifting to restore the variables to their original scale and offset. Similarly, objectives and constraints are transformed back to the user domain.

These transformations are defined and managed by the OptModelTransforms object, which encapsulates the specific transformations for variables, objectives, and nonlinear constraints.

Parameters:

Name Type Description Default
transforms OptModelTransforms

The transforms to apply.

 required

Returns:

Type Description
Results

A new FunctionResults object with all relevant data transformed
Results

back to the user domain.

to_dataframe 

to_dataframe(
    field_name: str,
    select: Iterable[str],
    unstack: Iterable[AxisName] | None = None,
) -> pd.DataFrame

Export a field to a pandas DataFrame.

Exports the values of a single field to a pandas DataFrame. The field to export is selected by the field_name argument. Typically, such a field contains multiple sub-fields. By default, all sub-fields are exported as columns in the DataFrame, but a subset can be selected using the select argument.

Sub-fields may be multi-dimensional arrays, which are exported in a stacked manner. Using the axis types found in the metadata, the exporter constructs a multi-index labeled with the corresponding names provided via the names field. If names does not contain a key/value pair for the the axis, numerical indices are used. These multi-indices can optionally be unstacked into multiple columns by providing the axis types to unstack via the unstack argument.

The DataFrame Index

The index of the resulting DataFrame may be a multi-index constructed from axis indices or labels. In addition, the batch_id (if not None) is prepended to the index.

Parameters:

Name Type Description Default
field_name str

The field to export.

 required
select Iterable[str]

Select the sub-fields to export. By default, all sub-fields are exported.

 required
unstack Iterable[AxisName] | None

Select axes to unstack. By default, no axes are unstacked.

 None

Raises:

Type Description
NotImplementedError

If the pandas module is not installed.
ValueError

If the field name is incorrect.

Returns:

Type Description
DataFrame

A pandas DataFrame containing the results.
Warning

This function requires the pandas module to be installed.

ropt.results.ResultField dataclass

Base class for fields within Results objects.

The ResultField class serves as a foundation for defining the various data fields that can be stored within Results objects. These fields typically hold multi-dimensional numerical data, such as objective values, constraint values, or gradients.

This class provides a standardized way to:

  • Store metadata about the axes of multi-dimensional arrays.
  • Retrieve the axes associated with a specific field.

Derived classes, such as FunctionEvaluations or Gradients, extend this base class to define specific data structures for different types of optimization results.

get_axes classmethod 

get_axes(name: str) -> tuple[AxisName, ...]

Retrieve the axes associated with a specific field.

Fields within a ResultField object that store multi-dimensional numpy arrays, contain metadata that describes the meaning of each dimension in the array. This method retrieves the axes of a field within a ResultField object from that meta-data, returning a tuple of AxisName][ropt.enums.AxisName] enums.

Parameters:

Name Type Description Default
name str

The name of the field (sub-field) within the ResultField instance or class.

 required

Raises:

Type Description
ValueError

If the provided field name is not recognized.

Returns:

Type Description
tuple[AxisName, ...]

A tuple of AxisName enums, representing the axes of the field.

ropt.results.FunctionResults dataclass

Bases: Results

Stores results related to function evaluations.

The FunctionResults class extends the base Results class to store data specific to function evaluations. This includes:

  1. Evaluations: The results of the function evaluations, including the variable values, objective values, and constraint values for each realization. See FunctionEvaluations.

  2. Realizations: Information about the realizations, such as weights for objectives and constraints, and whether each realization was successful. See Realizations.

  3. Functions: The calculated objective and constraint function values, typically aggregated across realizations. See Functions.

  4. Constraint Info: Details about constraint differences and violations. See ConstraintInfo.

Attributes:

Name Type Description
evaluations FunctionEvaluations

Results of the function evaluations.
realizations Realizations

The calculated parameters of the realizations.
functions Functions | None

The calculated functions.
constraint_info ConstraintInfo | None

Information on constraint differences and violations.

ropt.results.GradientResults dataclass

Bases: Results

Stores results related to gradient evaluations.

The GradientResults class extends the base Results class to store data specific to gradient evaluations. This includes:

  1. Evaluations: The results of the function evaluations for perturbed variables, including the perturbed variable values, objective values, and constraint values for each realization and perturbation. See GradientEvaluations.

  2. Realizations: Information about the realizations, such as weights for objectives and constraints, and whether each realization was successful. See Realizations.

  3. Gradients: The calculated gradients of the objectives and constraints. See Gradients.

Attributes:

Name Type Description
evaluations GradientEvaluations

Results of the function evaluations for perturbed variables.
realizations Realizations

The calculated parameters of the realizations.
gradients Gradients | None

The calculated gradients.

ropt.results.Functions dataclass

Bases: ResultField

Stores the calculated objective and constraint function values.

The Functions class stores the calculated values of the objective and constraint functions. These values are typically derived from the evaluations performed across all realizations, often through a process like averaging. The optimizer may handle multiple objectives and constraints. Multiple objectives are combined into a single weighted sum, which is stored in the weighted_objective field. Multiple constraints are handled individually by the optimizer.

Fields

weighted_objective: The overall objective calculated as a weighted sum over the objectives. This is a single floating point values. It is defined as a numpy array of dimensions 0, hence it has no axes:

  • Shape: \(()\)
  • Axis type: None

objectives: The calculated objective function values. This is a one-dimensional array of floating point values:

  • Shape \((n_o,)\), where:
    • \(n_o\) is the number of objectives.
  • Axis type:

constraints: The calculated constraint function values. This is a one-dimensional array of floating point values:

Attributes:

Name Type Description
weighted_objective NDArray[float64]

The weighted sum of the objective values.
objectives NDArray[float64]

The value of each individual objective.
constraints NDArray[float64] | None

The value of each individual constraint.

ropt.results.Gradients dataclass

Bases: ResultField

Stores the calculated objective and constraint gradients.

The Gradients class stores the calculated gradients of the objective and constraint functions. These gradients are typically derived from function evaluations across all realizations, often through a process like averaging. The optimizer may handle multiple objectives and constraints. Multiple objective gradients are combined into a single weighted sum, which is stored in the weighted_objective field. Multiple constraint gradients are handled individually by the optimizer.

Fields

weighted_objective: The gradient of the weighted objective with respect to each variable:

  • Shape: \((n_v,)\), where:
    • \(n_v\) is the number of variables.
  • Axis type:

objectives: The calculated gradients of each objective with respect to each variable. This is a two-dimensional array of floating point values:

constraints: The calculated gradients of each nonlinear constraint with respect to each variable. This is a two-dimensional array of floating point values:

Attributes:

Name Type Description
weighted_objective NDArray[float64]

The weighted sum of the objective gradients.
objectives NDArray[float64]

The gradient of each individual objective.
constraints NDArray[float64] | None

The gradient of each individual constraint.

ropt.results.FunctionEvaluations dataclass

Bases: ResultField

Stores the results of function evaluations.

The FunctionEvaluations class stores the results of evaluating the objective and constraint functions for a set of variables.

Fields

variables: The vector of variable values at which the functions were evaluated:

  • Shape: \((n_v,)\), where:
    • \(n_v\) is the number of variables.
  • Axis type:

objectives: The calculated objective function values for each realization. This is a two-dimensional array of floating point values where each row corresponds to a realization and each column corresponds to an objective:

constraints: The calculated constraint function values for each realization. Only provided if non-linear constraints are defined. This is a two-dimensional array of floating point values where each row corresponds to a realization and each column corresponds to a constraint:

evaluation_info: Optional metadata associated with each realization, potentially provided by the evaluator. If provided, each value in the info dictionary must be a one-dimensional array of arbitrary type supported by numpy (including objects):

Attributes:

Name Type Description
variables NDArray[float64]

The variable vector:
objectives NDArray[float64]

The objective function values for each realization.
constraints NDArray[float64] | None

The constraint function values for each realization.
evaluation_info dict[str, NDArray[Any]]

Optional metadata for each evaluated realization.

ropt.results.GradientEvaluations dataclass

Bases: ResultField

Stores the results of evaluations for gradient calculations.

The GradientEvaluations class stores the results of evaluating the objective and constraint functions for perturbed variables, which is necessary for gradient calculations.

Fields

variables: The vector of unperturbed variable values:

  • Shape: \((n_v,)\), where:
    • \(n_v\) is the number of variables.
  • Axis type:

perturbed_variables: A three-dimensional array of perturbed variable values for each realization and perturbation:

perturbed_objectives: A three-dimensional array of perturbed calculated objective function values for each realization and perturbation:

perturbed_constraints: A three-dimensional array of perturbed calculated non-linear constraint values for each realization and perturbation:

evaluation_info: Optional metadata associated with each realization, potentially provided by the evaluator. If provided, each value in the info dictionary must be a two-dimensional array of arbitrary type supported by numpy (including objects):

Attributes:

Name Type Description
variables NDArray[float64]

The unperturbed variable vector.
perturbed_variables NDArray[float64]

The perturbed variable values for each realization and perturbation.
perturbed_objectives NDArray[float64]

The objective function values for each realization and perturbation.
perturbed_constraints NDArray[float64] | None

The constraint function values for each realization and perturbation.
evaluation_info dict[str, NDArray[Any]]

Optional metadata for each evaluated realization and perturbation.

ropt.results.Realizations dataclass

Bases: ResultField

Stores information about the realizations.

The Realizations class stores data related to the individual realizations used in the optimization process.

Fields

active_realizations: A boolean array indicating whether each realization's evaluation was evaluated. True indicates that a realization was evaluated:

failed_realizations: A boolean array indicating whether each realization's evaluation was successful. True indicates a failed realization, while False indicates a successful one:

objective_weights: A two-dimensional array of weights used for each objective in each realization:

These weights may change during optimization, depending on the type of objective calculation

constraint_weights: A two-dimensional array of weights used for each constraint in each realization:

These weights may change during optimization, depending on the type of constraint calculation

Attributes:

Name Type Description
active_realizations NDArray[bool_]

Boolean array indicating active realizations.
failed_realizations NDArray[bool_]

Boolean array indicating failed realizations.
objective_weights NDArray[float64] | None

Weights for each objective in each realization.
constraint_weights NDArray[float64] | None

Weights for each constraint in each realization.

ropt.results.ConstraintInfo dataclass

Bases: ResultField

Stores information about constraint differences and violations.

The ConstraintInfo class stores the differences between variable or constraint values and their respective bounds. It also calculates and stores constraint violations. This information is useful for assessing how well the optimization process is satisfying the imposed constraints.

Constraint differences

These represent the difference between a variable or constraint value and its corresponding bound. Whether this difference signifies a violation depends on the bound type:

  • Lower Bounds: A negative difference means the value is below the lower bound, thus violating the constraint.
  • Upper Bounds: A positive difference means the value is above the upper bound, thus violating the constraint.

The class stores the following information on the differences:

Constraint Violations

Constraint violations are calculated based on the constraint differences. If a bound is violated, the violation value is the absolute value of the difference. If the bound is not violated, the violation value is zero.

Fields

The class stores the following information for bound, linear constraint, and non-linear constraint differences and violations as one-dimensional vectors:

  • Differences: bound_lower and bound_upper
  • Violations: bound_violation
  • Shape: \((n_v,)\), where:
    • \(n_v\) is the number of variables.
  • Axis type:
  • Differences: linear_lower and linear_upper
  • Violations: linear_violation
  • Shape: \((n_l,)\), where:
    • \(n_l\) is the number of linear constraints.
  • Axis type:
  • Differences: nonlinear_lower and nonlinear_upper
  • Violations: nonlinear_violation
  • Shape: \((n_c,)\), where:
    • \(n_c\) is the number of non-linear constraints.
  • Axis type:

Attributes:

Name Type Description
bound_lower NDArray[float64] | None

Difference between variables and their lower bounds.
bound_upper NDArray[float64] | None

Difference between variables and their upper bounds.
linear_lower NDArray[float64] | None

Difference between linear constraints and their lower bounds.
linear_upper NDArray[float64] | None

Difference between linear constraints and their upper bounds.
nonlinear_lower NDArray[float64] | None

Difference between nonlinear constraints and their lower bounds.
nonlinear_upper NDArray[float64] | None

Difference between nonlinear constraints and their upper bounds.
bound_violation NDArray[float64] | None

Magnitude of the violation of the variable bounds.
linear_violation NDArray[float64] | None

Magnitude of the violation of the linear constraints.
nonlinear_violation NDArray[float64] | None

Magnitude of the violation of the nonlinear constraints.

ropt.results.results_to_dataframe 

results_to_dataframe(
    results: Sequence[Results],
    fields: set[str],
    result_type: Literal["functions", "gradients"],
) -> pd.DataFrame

Combine a sequence of results into a single pandas DataFrame.

This function aggregates results from multiple FunctionResults or GradientResults objects into a single pandas DataFrame. It is designed to be used with observers that produce results during the optimization process.

The fields argument determines which data fields to include in the DataFrame. These fields can be any of the attributes defined within FunctionResults or GradientResults. Nested fields are specified using dot notation (e.g., evaluations.variables to include the variables field within the evaluations object).

The evaluation_info sub-fields, found within the evaluations fields of functions and gradient results, respectively, are dictionaries. To include specific keys from these dictionaries, use the format evaluations.evaluation_info.key, where key is the name of the desired key.

Many fields may result in multiple columns in the DataFrame. For example, evaluations.variables will generate a separate column for each variable. If available, variable names will be used as column labels. Multi-dimensional fields, such as those with named realizations and objectives, will have column names that are tuples of the corresponding names.

The result_type argument specifies whether to include function evaluation results (functions) or gradient results (gradients).

Parameters:

Name Type Description Default
results Sequence[Results]

A sequence of Results objects to combine.

 required
fields set[str]

The names of the fields to include in the DataFrame.

 required
result_type Literal['functions', 'gradients']

The type of results to include ("functions" or "gradients").

 required

Returns:

Type Description
DataFrame

A pandas DataFrame containing the combined results.