I've been hitting an issue where when I define complex conditions that include OrConjunctions, the child hyperparameter is not always defined when the parent passes it's condition. I've broken this down into a simple replicator example:
from ConfigSpace import ConfigurationSpace
from ConfigSpace.hyperparameters import CategoricalHyperparameter
from ConfigSpace.conditions import InCondition, OrConjunction
cs = ConfigurationSpace(seed=1)
hyper_params = {}
hyper_params["hp5"] = CategoricalHyperparameter("hp5", [0,1,2])
hyper_params["hp7"] = CategoricalHyperparameter("hp7", [3,4,5])
hyper_params["hp8"] = CategoricalHyperparameter("hp8", [6,7,8])
for key in hyper_params:
cs.add_hyperparameter(hyper_params[key])
cs.add_condition(InCondition(hyper_params["hp5"], hyper_params["hp8"], [6]))
cs.add_condition(
OrConjunction(
InCondition(hyper_params["hp7"], hyper_params["hp8"], [7]),
InCondition(hyper_params["hp7"], hyper_params["hp5"], [1])))
print(cs)
for cfg in cs.sample_configuration(10):
print(cfg)
print(cfg._vector)
When I run this, I get the following output:
Configuration space object:
Hyperparameters:
hp5, Type: Categorical, Choices: {0, 1, 2}, Default: 0
hp7, Type: Categorical, Choices: {3, 4, 5}, Default: 3
hp8, Type: Categorical, Choices: {6, 7, 8}, Default: 6
Conditions:
(hp7 | hp8 in {7} || hp7 | hp5 in {1})
hp5 | hp8 in {6}
Configuration:
hp8, Value: 7
[ 1. nan nan]
Configuration:
hp5, Value: 2
hp8, Value: 6
[ 0. 2. nan]
Configuration:
hp5, Value: 1
hp7, Value: 4
hp8, Value: 6
[0. 1. 1.]
...
Within these three sample configurations, the following is happening:
- When hp8 == 7, hp7 is not defined despite the conditional
(hp7 | hp8 in {7} || hp7 | hp5 in {1}) which should define hp7 whenever hp8==7 (or hp5==1).
- As expected, when hp8==6, hp5 is defined
- Despite not working when hp8==7, hp7 is defined when hp5==1
I'm relatively new to this package, so it's possible I'm missing something, but as far as I can tell this is not the expected behavior.
Additionally, when I was running my full example and running it through the SMAC optimizer, I would occasionally hit this error, which I think might be related. The hyperparameter that is not specified is also one that was defined as child in a OrConjunction:
File "ConfigSpace/util.pyx", line 212, in get_one_exchange_neighbourhood
File "ConfigSpace/configuration_space.pyx", line 1425, in ConfigSpace.configuration_space.Configuration.is_valid_configuration
File "ConfigSpace/c_util.pyx", line 40, in ConfigSpace.c_util.check_configuration
File "ConfigSpace/c_util.pyx", line 124, in ConfigSpace.c_util.check_configuration
ValueError: Active hyperparameter 'split_3_20_24' not specified!
I've been hitting an issue where when I define complex conditions that include OrConjunctions, the child hyperparameter is not always defined when the parent passes it's condition. I've broken this down into a simple replicator example:
When I run this, I get the following output:
Within these three sample configurations, the following is happening:
(hp7 | hp8 in {7} || hp7 | hp5 in {1})which should define hp7 whenever hp8==7 (or hp5==1).I'm relatively new to this package, so it's possible I'm missing something, but as far as I can tell this is not the expected behavior.
Additionally, when I was running my full example and running it through the SMAC optimizer, I would occasionally hit this error, which I think might be related. The hyperparameter that is not specified is also one that was defined as child in a OrConjunction: