CODEWITHSUNDEEP
parametershyperopt
MJeremy
MJeremy

Reputation: 1250

Hyperopt: Define parameter which is dependent on other parameter

I am using python package hyperopt and I have a parameter a which requires to be larger than parameter b.

For example, I hope my parameter space is like

from hyperopt import hp

space = {"b": hp.uniform(0, 0.5), "a": hp.uniform(b, 0.5)}

Which, requires a to be at least larger than b, how can I do that?

Thanks in advance

Upvotes: 5

Views: 3724

Answers (2)

hotuagia
hotuagia

Reputation: 46

Maybe my find could help someone. I was using HyperOpt to optimise parameters of a piecewise affine function. Therefore I need nested parameters to properly set x parameters parts:

piecewise function As nested parameters don't work (anymore) for HyperOpt, nor for Optuna... I recommend the pwlf library for all those who have the same problem.

This lib proposes 2 ways to extract the parameters of a piecewise linear function :

  • By providing the number of parts
  • By providing the breakpoint locations

If like me you are interested in finding the breakpoint locations you can provide the number of parts:

import pwlf
import pandas as pd

data = pd.DataFrame.from_dict({
    "x": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
    "y": [0, 4, 6, 7, 8, 9, 10, 7, 3, 1],
})

model = pwlf.PiecewiseLinFit(data.x, data.y)
nb_parts = 3
model.fit(nb_parts)

print(f"x part intersections: {model.fit_breaks}")
# x part intersections: [0. 1.99 6.00 9.]
print(f"linear coefficient of each part: {model.slopes}")
# linear coefficient of each part: [ 4.00,  1.00, -3.00]
print(f"linear bias of each part: {model.intercepts}")
# linear bias of each part: [-7.11e-06,  5.99e+00,  3.00e+01]

Hope it helps!

Upvotes: 0

Eric O. Lebigot
Eric O. Lebigot

Reputation: 94595

A simple option is to use the ability of hyperopt to nest parameters. You can thus define a hyper-parameter space like you want:

space = hp.uniform("a", hp.uniform("b", 0, 0.5), 0.5)

Only "a"'s value is passed to the function that you optimize (because this is the hyper-parameter space), but hyperopt.fmin() will return both parameters.

A similar option, but where the function to be optimized receives both parameters is: 

b_var = hp.uniform("b", 0, 0.5)
space = {"b": b_var, "a": hp.uniform("a", b_var, 0.5)}

Finally, it might be simpler to change a bit the inputs to the optimized function: parameter a can be replaced by a_fraction running between 0 and 1 and interpolating between b and 0.5 (i.e. a_fraction = 0 yields a = b and a_fraction = 1 gives a = 0.5 inside the modified function to be optimized). The parameter space thus has the usual form:

space = {"b": hp.uniform("b", 0, 0.5), "a_fraction": hp.uniform("a_fraction", 0, 1)}

There is an interesting discussion at https://github.com/hyperopt/hyperopt/issues/175#issuecomment-29401501.

Upvotes: 13

Related Questions