Tune Search Space API

This section covers the functions you can use to define your search spaces.

Caution

Not all Search Algorithms support all distributions. In particular, tune.sample_from and tune.grid_search are often unsupported. The default Random search and grid search (tune.search.basic_variant.BasicVariantGenerator) supports all distributions.

Tip

Avoid passing large objects as values in the search space, as that will incur a performance overhead. Use tune.with_parameters to pass large objects in or load them inside your trainable from disk (making sure that all nodes have access to the files) or cloud storage. See How can I avoid bottlenecks? for more information.

For a high-level overview, see this example:

config = {
    # Sample a float uniformly between -5.0 and -1.0
    "uniform": tune.uniform(-5, -1),

    # Sample a float uniformly between 3.2 and 5.4,
    # rounding to multiples of 0.2
    "quniform": tune.quniform(3.2, 5.4, 0.2),

    # Sample a float uniformly between 0.0001 and 0.01, while
    # sampling in log space
    "loguniform": tune.loguniform(1e-4, 1e-2),

    # Sample a float uniformly between 0.0001 and 0.1, while
    # sampling in log space and rounding to multiples of 0.00005
    "qloguniform": tune.qloguniform(1e-4, 1e-1, 5e-5),

    # Sample a random float from a normal distribution with
    # mean=10 and sd=2
    "randn": tune.randn(10, 2),

    # Sample a random float from a normal distribution with
    # mean=10 and sd=2, rounding to multiples of 0.2
    "qrandn": tune.qrandn(10, 2, 0.2),

    # Sample a integer uniformly between -9 (inclusive) and 15 (exclusive)
    "randint": tune.randint(-9, 15),

    # Sample a random uniformly between -21 (inclusive) and 12 (inclusive (!))
    # rounding to multiples of 3 (includes 12)
    # if q is 1, then randint is called instead with the upper bound exclusive
    "qrandint": tune.qrandint(-21, 12, 3),

    # Sample a integer uniformly between 1 (inclusive) and 10 (exclusive),
    # while sampling in log space
    "lograndint": tune.lograndint(1, 10),

    # Sample a integer uniformly between 1 (inclusive) and 10 (inclusive (!)),
    # while sampling in log space and rounding to multiples of 2
    # if q is 1, then lograndint is called instead with the upper bound exclusive
    "qlograndint": tune.qlograndint(1, 10, 2),

    # Sample an option uniformly from the specified choices
    "choice": tune.choice(["a", "b", "c"]),

    # Sample from a random function, in this case one that
    # depends on another value from the search space
    "func": tune.sample_from(lambda spec: spec.config.uniform * 0.01),

    # Do a grid search over these values. Every value will be sampled
    # ``num_samples`` times (``num_samples`` is the parameter you pass to ``tune.TuneConfig``,
    # which is taken in by ``Tuner``)
    "grid": tune.grid_search([32, 64, 128])
}

Random Distributions API

tune.uniform	Sample a float value uniformly between lower and upper.
tune.quniform	Sample a quantized float value uniformly between lower and upper.
tune.loguniform	Sugar for sampling in different orders of magnitude.
tune.qloguniform	Sugar for sampling in different orders of magnitude.
tune.randn	Sample a float value normally with mean and sd.
tune.qrandn	Sample a float value normally with mean and sd.
tune.randint	Sample an integer value uniformly between lower and upper.
tune.qrandint	Sample an integer value uniformly between lower and upper.
tune.lograndint	Sample an integer value log-uniformly between lower and upper, with base being the base of logarithm.
tune.qlograndint	Sample an integer value log-uniformly between lower and upper, with base being the base of logarithm.
tune.choice	Sample a categorical value.

Grid Search and Custom Function APIs

tune.grid_search	Specify a grid of values to search over.
tune.sample_from	Specify that tune should sample configuration values from this function.

References

See also Random search and grid search (tune.search.basic_variant.BasicVariantGenerator).