Plugin Manager

ropt.plugins

Extending ropt with plugins.

The ropt.plugins module provides the framework for extending ropt's capabilities through a plugin system. Plugins allow for the integration of custom or third-party components, installed as separate packages.

ropt supports several types of plugins, each addressing a specific aspect of the optimization workflow:

Workflow related plugins:

Workflow related plugins may directly be used by the user to create components when building optimization workflows, generally via these convenience functions:

• create_compute_step: Create compute steps, such as optimizations, to run during workflow execution.
• create_event_handler: Create handlers to process events generated during optimization.
• create_evaluator: Create evaluators to for use by compute steps.

Optimizer related plugins:

These plugins are used to implement specific features of the ensemble optimizer:

• optimizer: Implements optimization algorithms.
• sampler: Generates parameter perturbations, which are used for gradient estimation.
• realization_filter: Selects subsets of ensemble realizations for calculating objectives or constraints.
• function_estimator: Computes final objective function values and gradients from individual realization results.

These plugins are generally not directly created by the user. Instead, the optimization algorithm will find and create the required plugins based on information passed via the optimization configuration.

Plugin Management and Discovery

The PluginManager class is central to the plugin system. It discovers and manages available plugins. Plugins are typically discovered automatically using Python's standard entry points mechanism.

Each plugin type has a corresponding abstract base class that custom plugins must inherit from:

• Workflow plugins: ComputeStepPlugin, EventHandlerPlugin, EvaluatorPlugin
• Optimizer: OptimizerPlugin
• Sampler: SamplerPlugin
• Realization Filter: RealizationFilterPlugin
• Function Estimator: FunctionEstimatorPlugin

Using Plugins

The PluginManager.get_plugin method is used internally by ropt to retrieve the appropriate plugin implementation based on a specified type and method name. The PluginManager.get_plugin_name method can be used to find the name of a plugin that supports a given method.

Plugins can implement multiple named methods. To request a specific method (method-name) from a particular plugin (plugin-name), use the format "plugin-name/method-name". If only a method name is provided, the plugin manager searches through all registered plugins (that allow discovery) for one that supports the method. Using "plugin-name/default" typically selects the primary or default method offered by that plugin, although specifying "default" without a plugin name is not permitted.

Plugins retrieved by the PluginManager.get_plugin method generally implement a create factory method that will be used to instantiate the objects that implement the desired functionality. These objects must inherit from the base class for the corresponding plugin type:

• Workflow related plugins: ComputeStep, EventHandler, Evaluator
• Optimizer: Optimizer
• Sampler: Sampler
• Realization Filter: RealizationFilter
• Function Estimator: FunctionEstimator

The create_evaluator, create_event_handler and create_compute_step convenience functions combine plugin discovery and object creation into a single call.

Pre-installed Plugins Included with ropt

ropt comes bundled with a set of pre-installed plugins:

• Workflow: The built-in default compute step, default event handler and default evaluator plugins, providing components for executing complex optimization workflows.
• Optimizer: The scipy plugin, leveraging algorithms from scipy.optimize, and the ExternalOptimizer plugin, which is used to launch optimizers in separate processes.
• Sampler: The scipy plugin, using distributions from scipy.stats.
• Realization Filter: The default plugin, offering filters based on ranking and for CVaR optimization.
• Function Estimator: The default plugin, supporting objectives based on mean or standard deviation.

PluginManager

Manages the discovery and retrieval of ropt plugins.

The PluginManager is responsible for finding available plugins based on Python's entry points mechanism and providing access to them. It serves as a central registry for different types of plugins used within ropt, such as optimizers, samplers, and workflow components.

Upon initialization, the manager scans for entry points defined under the ropt.plugins.* groups (e.g., ropt.plugins.optimizer). Plugins found this way are loaded and stored internally, categorized by their type.

The primary way to interact with the manager is through the get_plugin method, which retrieves a specific plugin class based on its type and a method name it supports. The get_plugin_name method can be used to find the name of a plugin that supports a given method.

Example: Registering a Custom Optimizer Plugin

To make a custom optimizer plugin available to ropt, you would typically define an entry point in your package's pyproject.toml:

[project.entry-points."ropt.plugins.optimizer"]
my_optimizer = "my_package.my_module:MyOptimizer"

When ropt initializes the PluginManager, it will discover and load MyOptimizer from my_package.my_module, making it accessible via plugin_manager.get_plugin("optimizer", "my_optimizer/some_method") or potentially plugin_manager.get_plugin("optimizer", "some_method") if discovery is allowed and the method is unique.

__init__

__init__() -> None

Initialize the plugin manager.

get_plugin

get_plugin(plugin_type: PluginType, method: str) -> Any

Retrieve a plugin class by its type and a supported method name.

This method finds and returns the class of a plugin that matches the specified plugin_type and supports the given method.

The method argument can be specified in two ways:

1. Explicit Plugin: Use the format "plugin-name/method-name". This directly requests the method-name from the plugin named plugin-name.
2. Implicit Plugin: Provide only the method-name. The manager will search through all registered plugins of the specified plugin_type that allow discovery (see Plugin.allows_discovery). If the method is found in the default plugin of ropt, that plugin is used. Otherwise it returns the first plugin found that supports the method-name.

Parameters:

Name	Type	Description	Default
plugin_type	PluginType	The category of the plugin (e.g., "optimizer", "sampler").	required
method	str	The name of the method the plugin must support, potentially prefixed with the plugin name and a slash (/).	required

Returns:

Type	Description
Any	The plugin class that matches the criteria.

Raises:

Type	Description
ValueError	If no matching plugin is found for the given type and method, or if "default" is used as a method name without specifying a plugin name.

get_plugin_name

get_plugin_name(
    plugin_type: PluginType, method: str
) -> str | None

Return the name of the plugin that supports a given method.

Verifies whether a plugin of the specified plugin_type supports the given method. This is useful for checking availability before attempting to retrieve a plugin with get_plugin.

The method argument can be specified in two ways:

1. Explicit Plugin: "plugin-name/method-name" checks if the specific plugin named plugin-name supports method-name.
2. Implicit Plugin: "method-name" searches through all discoverable plugins of the given plugin_type to see if any support method-name. If the method is found in the default plugin of ropt, that plugin is used. Otherwise it returns the first plugin found that supports the method-name.

Parameters:

Name	Type	Description	Default
plugin_type	PluginType	The category of the plugin (e.g., "optimizer", "sampler").	required
method	str	The name of the method to check, potentially prefixed with the plugin name and a slash (/).	required

Returns:

Type	Description
str | None	The name of a matching plugin supporting the specified method, or None.

PluginType module-attribute

PluginType = Literal[
    "optimizer",
    "sampler",
    "realization_filter",
    "function_estimator",
    "event_handler",
    "compute_step",
    "evaluator",
]

Represents the valid types of plugins supported by ropt.

This type alias defines the string identifiers used to categorize different plugins within the ropt framework. Each identifier corresponds to a specific role in the optimization process:

• "optimizer": Plugins implementing optimization algorithms (OptimizerPlugin).
• "sampler": Plugins for generating parameter samples (SamplerPlugin).
• "realization_filter": Plugins for filtering ensemble realizations (RealizationFilterPlugin).
• "function_estimator": Plugins for estimating objective functions and gradients (FunctionEstimatorPlugin).
• "event_handler": Plugins that create event handlers for processing optimization results (EventHandlerPlugin).
• "compute_step": Plugins that define executable steps within an optimization workflow (ComputeStepPlugin).
• "evaluator": Plugins that define evaluators within an optimization workflow (EvaluatorPlugin).

Plugin

Bases: ABC

Abstract base class for all ropt plugins.

This class serves as the fundamental building block for all plugins within the ropt framework. Any class intended to function as a plugin (e.g., an optimizer, sampler, or event handler) must inherit from this base class.

It defines the core interface that all plugins must adhere to, ensuring consistency and enabling the PluginManager to discover and manage them effectively.

Subclasses must implement the is_supported class method to indicate which named methods (functionalities) they provide. They can optionally override the allows_discovery class method if they should not be automatically selected by the plugin manager when a method name is provided without an explicit plugin name.

is_supported abstractmethod classmethod

is_supported(method: str) -> bool

Verify if this plugin supports a specific named method.

This class method is used by the PluginManager (specifically its get_plugin_name method) to determine if this plugin class provides the functionality associated with the given method name.

Parameters:

Name	Type	Description	Default
method	str	The string identifier of the method to check for support.	required

Returns:

Type	Description
bool	True if the plugin supports the specified method, False otherwise.

allows_discovery classmethod

allows_discovery() -> bool

Determine if the plugin allows implicit discovery by method name.

By default (True), plugins can be found by the PluginManager when a user provides only a method name (without specifying the plugin, e.g., "method-name").

If a plugin should only be used when explicitly named (e.g., "plugin-name/method-name"), it must override this class method to return False.

For instance, the external optimizer plugin acts as a wrapper for other optimizers run in separate processes. It doesn't provide methods directly and must always be explicitly requested, so it overrides this method to return False.

Returns:

Type	Description
bool	True if the plugin can be discovered implicitly by method name.