Skip to content

[tune] Add Unit Test for nested PBT + Jenkins #7324

Merged
richardliaw merged 6 commits intoray-project:masterfrom
richardliaw:pbt-nested
Feb 28, 2020
Merged

[tune] Add Unit Test for nested PBT + Jenkins #7324
richardliaw merged 6 commits intoray-project:masterfrom
richardliaw:pbt-nested

Conversation

@richardliaw
Copy link
Copy Markdown
Contributor

@richardliaw richardliaw commented Feb 25, 2020 •
edited
Loading

Why are these changes needed?

Related issue number

Closes #7313.

Checks

@richardliaw richardliaw changed the title [tune] Fix nested PBT [tune] Add Unit Test for nested PBT + Jenkins Feb 25, 2020
richardliaw
richardliaw commented Feb 25, 2020
View reviewed changes
python/ray/tune/tests/test_trial_scheduler.py Outdated
"int_factor": 10,
"id_factor": i
}
trial = _MockTrial(i, hyperparams)
Copy link
Copy Markdown
Contributor Author

@richardliaw richardliaw Feb 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This just makes it so that you can add arbitrary hyperparams in the test suite

@richardliaw richardliaw requested a review from ericl February 25, 2020 23:24
@AmplabJenkins
Copy link
Copy Markdown

AmplabJenkins commented Feb 25, 2020

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/22407/
Test FAILed.

hartikainen
hartikainen approved these changes Feb 26, 2020
View reviewed changes
python/ray/tune/schedulers/pbt.py Outdated
Comment on lines +183 to +184
if not (isinstance(value, list) or callable(value)
or isinstance(value, dict)):
Copy link
Copy Markdown
Contributor

@hartikainen hartikainen Feb 26, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if not (isinstance(value, list) or callable(value)
or isinstance(value, dict)):
if not (isinstance(value, (list, dict)) or callable(value)):

python/ray/tune/tests/test_trial_scheduler.py Outdated
self.assertEqual(type(trials[0].config["int_factor"]), int)
self.assertEqual(trials[0].config["const_factor"], 3)
self.assertEqual(trials[0].config["nest"]["nest_float"], 101.0)
self.assertEqual(type(trials[0].config["nest"]["nest_float"]), float)
Copy link
Copy Markdown
Contributor

@hartikainen hartikainen Feb 26, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
self.assertEqual(type(trials[0].config["nest"]["nest_float"]), float)
self.assertIsInstance(trials[0].config["nest"]["nest_float"], float)

python/ray/tune/tests/test_trial_scheduler.py Outdated
self.assertEqual(trials[0].config["float_factor"], 100.0)
self.assertEqual(type(trials[0].config["float_factor"]), float)
self.assertEqual(trials[0].config["int_factor"], 10)
self.assertEqual(type(trials[0].config["int_factor"]), int)
Copy link
Copy Markdown
Contributor

@hartikainen hartikainen Feb 26, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
self.assertEqual(type(trials[0].config["int_factor"]), int)
self. assertIsInstance(trials[0].config["int_factor"], int)

python/ray/tune/tests/test_trial_scheduler.py Outdated
TrialScheduler.CONTINUE)
self.assertIn(trials[0].restored_checkpoint, ["trial_3", "trial_4"])
self.assertEqual(trials[0].config["float_factor"], 100.0)
self.assertEqual(type(trials[0].config["float_factor"]), float)
Copy link
Copy Markdown
Contributor

@hartikainen hartikainen Feb 26, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
self.assertEqual(type(trials[0].config["float_factor"]), float)
self. assertIsInstance(trials[0].config["float_factor"], float)

@AmplabJenkins
Copy link
Copy Markdown

AmplabJenkins commented Feb 27, 2020

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/22473/
Test FAILed.

@AmplabJenkins
Copy link
Copy Markdown

AmplabJenkins commented Feb 27, 2020

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/22490/
Test FAILed.

@AmplabJenkins
Copy link
Copy Markdown

AmplabJenkins commented Feb 27, 2020

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/22491/
Test FAILed.

richardliaw
richardliaw commented Feb 28, 2020
View reviewed changes
@AmplabJenkins
Copy link
Copy Markdown

AmplabJenkins commented Feb 28, 2020

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/22526/
Test PASSed.

@richardliaw richardliaw merged commit 3fc162f into ray-project:master Feb 28, 2020
@richardliaw richardliaw deleted the pbt-nested branch February 28, 2020 02:17
ffbin pushed a commit to antgroup/ant-ray that referenced this pull request Mar 20, 2020
@richardliaw
[tune] Add Unit Test for nested PBT + Jenkins (ray-project#7324)
1df5a38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hartikainen hartikainen hartikainen approved these changes

@ericl ericl Awaiting requested review from ericl

Assignees

@hartikainen hartikainen

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[tune] PBT hyperparam_mutations does not allow for nested dicts any more

3 participants

@richardliaw @AmplabJenkins @hartikainen