[tune] wrapper function to pass arbitrary objects through the object store to trainables#10679
Merged
richardliaw merged 5 commits intoray-project:masterfrom Sep 11, 2020
Merged
Conversation
richardliaw
approved these changes
Sep 11, 2020
Contributor
|
We will want to fallback to pickle in a future iteration of this. |
1 task
|
Hello! Just wanted to reopen this as I'm currently trying to pass a tf.Dataset object through .with_parameters() and I keep getting the error "tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot convert a Tensor of dtype variant to a NumPy array". I've tried several ways and everything points to this method not being able to handle this object. How would you do this? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
This PR adds a
with_parameterswrapper function to pass arbitrary parameters to Tune trainables. It works similar tofunctools.partial, but it stores parameters in the Ray object store and automatically retrieves and passes them to the trainable.Related issue number
#10305
Also answers numerous requests on Slack on how to pass parameters to trainables.
Checks
scripts/format.shto lint the changes in this PR.