Describe your feature request

allow creating a search algorithm for tune.run given:

• (maybe; see option 1 below) a string defining the search algorithm to use
• (maybe; see option 2 below) the search space configuration

API

there will be two ways to use shim instantiation (option 3 from the design doc):

Option 1

(notice how the search algorithm definition doesn't require redefining the search configuration when initializing HyperOptSearch)

from ray import tune
from ray.tune.param import Float, Integer, Categorical
from ray.tune.param import Grid, Uniform, LogUniform, Normal

space = {
    "lr": Float(1e-4, 1e-1).LogUniform(),
    "num_epochs": Integer(5, 10).Uniform(),
    "temperature": Float().Normal(1, 0.1),
    "batch_size": Categorical([32, 64]).Grid()
}

tune.run(
    trainable,
    space,
    search_alg=HyperOptSearch(metric="mean_loss"))

Option 2

from ray import tune
from ray.tune.param import Float, Integer, Categorical
from ray.tune.param import Grid, Uniform, LogUniform, Normal

space = {
    "lr": Float(1e-4, 1e-1).LogUniform(),
    "num_epochs": Integer(5, 10).Uniform(),
    "temperature": Float().Normal(1, 0.1),
    "batch_size": Categorical([32, 64]).Grid()
}

search_alg = tune.create_searcher("HyperOpt", space,
    metric="mean_loss")

tune.run(
    trainable,
    search_alg=search_alg)

related: #9969, #10401, #10444

CC @richardliaw @krfricke