dair_pll.hyperparameter

Interface for hyperparameter declaration and optimization.

Each experiment run (see SupervisedLearningExperiment) can have its hyperparameters optimized via optuna. By design, each experiment is fully described via a SupervisedLearningExperimentConfig object. This file implements a Hyperparameter class, which can be declared as a member variable of a SupervisedLearningExperimentConfig, or recursively as one of its own dataclass() members.

The following hyperparameters types and priors are supported:

Scalar, a Float or Int which is either uniformly or log-uniformly distributed.

Categorical, a list of float, int, or str types which are selected from uniformly.

class dair_pll.hyperparameter.ScalarT

Templating type hint for Scalars.

alias of TypeVar(‘ScalarT’, int, float)

class dair_pll.hyperparameter.Hyperparameter(value, distribution)[source]

Bases: ABC

Class for declaring and sampling hyperparameters.

Hyperparameters have both a value and a distribution from which vales might be selected.

Declaration of a hyperparameter in a configuration may look like:

@dataclass
class XXXConfig:
    int_par: Int = Int(5, (0, 10))
    float_par: Float = Float(0.1, (1e-4, 1e3), log=True)
    cat_par: Categorical = Categorical('val2', ['val0','val1', 'val2'])

In these cases, the first argument is the default value of the hyperparameter. However, at hyperparameter optimization time, optuna will select hyperparameters from the distribution via the suggest() function. Some hyperparameter types have a default distribution as described in their documentation.

distribution: typing.Sequence[typing.Union[int, float, str]]: Parameters for distribution from which to select value.

value: typing.Union[int, float, str]: Hyperparameter’s current value.

set(value)[source]: Setter for underlying hyperparameter value.

abstract suggest(trial, name)[source]

Suggests a value for the hyperparameter.

This function is abstract as to facilitate specialization for inheriting types.

Parameters:

trial (Trial) – Optuna trial in which parameter is being suggested.
name (str) – Name of hyperparameter.

Return type:

Union[int, float, str]

Returns:

Suggested hyperparameter value.

class dair_pll.hyperparameter.Scalar(value, distribution=None, log=False)[source]

Bases: Hyperparameter, ABC, Generic[ScalarT]

Abstract scalar hyperparameter type.

Defines a uniform or log-uniform distribution over a scalar type, such as integers or real numbers.

The bounds of the distribution can either be specified as a tuple in the distribution attribute, or set as a default based on the provided value in the abstract method default_range().

value: ~ScalarT: Scalar value of hyperparameter.

distribution: typing.Tuple[~ScalarT, ~ScalarT]: Bounds of scalar distribution in format (lower, upper).

log: bool: Whether the distribution is uniform or log-uniform.

abstract default_range(value, log)[source]

Returns default range for Scalar, depending on provided value.

Return type:: Tuple[~ScalarT, ~ScalarT]

class dair_pll.hyperparameter.Int(value, distribution=None, log=False)[source]

Bases: Scalar

Integer scalar hyperparameter.

value: int: Scalar value of hyperparameter.

distribution: typing.Tuple[int, int]: Bounds of scalar distribution in format (lower, upper).

default_range(value, log)[source]

Default bounds for integer hyperparameter.

Returns (max(1, value // RANGE), value * RANGE), where RANGE is 8 in the log-uniform case and 2 otherwise.

Parameters:

value (int) – Default value of hyperparameter.
log (bool) – Whether the distribution is uniform or log-uniform.

Return type:

Tuple[int, int]

Returns:

Default lower/upper bounds.

suggest(trial, name)[source]

Returns suggested (log)-uniform distributed integer.

Return type:: int

class dair_pll.hyperparameter.Float(value, distribution=None, log=False)[source]

Bases: Scalar

Real number (floating-point) scalar hyperparameter.

value: float: Scalar value of hyperparameter.

distribution: typing.Tuple[float, float]: Bounds of scalar distribution in format (lower, upper).

default_range(value, log)[source]

Default bounds for float hyperparameter.

Returns (value / RANGE, value * RANGE), where RANGE is 100 in the log-uniform case and 2 otherwise.

Parameters:

value (float) – Default value of hyperparameter.
log (bool) – Whether the distribution is uniform or log-uniform.

Return type:

Tuple[float, float]

Returns:

Default lower/upper bounds.

suggest(trial, name)[source]

Returns suggested (log)-uniform distributed float.

Return type:: float

class dair_pll.hyperparameter.Categorical(value, distribution)[source]

Bases: Hyperparameter

Categorical hyperparameter.

value: typing.Union[int, float, str]: Hyperparameter’s current value.

distribution: typing.List[typing.Union[int, float, str]]: Parameters for distribution from which to select value.

suggest(trial, name)[source]

Suggests from listed values in distribution uniformly.

Return type:: Union[int, float, str]

dair_pll.hyperparameter.is_dataclass_instance(obj)[source]

Helper function to check if input object is a dataclass().

Return type:: bool

dair_pll.hyperparameter.traverse_config(config, callback, namespace='')[source]

Recursively searches through config and its member dataclass()es, for member Hyperparameter objects.

While traversing the tree, maintains a namespace constructed from a concatenation of the attributes’ names.

When a Hyperparameter ‘h’ under attribute name attr is encountered, this function calls :py:arg:`callback` with inputs (namespace + attr, h).

Parameters:

config (Any) – Configuration dataclass() .
callback (Callable[[str, Hyperparameter], None]) – Callback performed on each Hyperparameter.
namespace (str) – (Optional/internal) prefix for naming hyperparameters.

Return type:

None

dair_pll.hyperparameter.generate_suggestion(config, trial)[source]

Suggests a value all hyperparameters in configuration (but does not set these values).

Recursively searches through config and its member dataclass()es, and generates a suggestion for each contained Hyperparameter.

Parameters:

config – Configuration dataclass() .
trial (Trial) – Optuna trial in which parameters are being suggested.

Return type:

Dict[str, Union[int, float, str]]

Returns:

Suggested hyperparameter value dictionary.

dair_pll.hyperparameter.load_suggestion(config, suggestion)[source]

Fill all hyperparameters in configuration with suggestions.

Recursively searches through config and its member dataclass()es, and sets the values to the suggestion for each contained Hyperparameter.

The suggestion is assumed to be generated by running generate_suggestion() on an identical type to config.

Parameters:

config (Any) – Configuration dataclass() .
suggestion (Dict[str, Union[int, float, str]]) – Suggested hyperparameter set.

Return type:

None

dair_pll.hyperparameter.hyperparameter_values(config)[source]

Lists current values for all hyperparameters in configuration.

Recursively searches through config and its member dataclass()es, and records value for each contained Hyperparameter.

Parameters:: config (Any) – Configuration dataclass() .
Return type:: Dict[str, Union[int, float, str]]
Returns:: Hyperparameter value dictionary.