Incorporate compilation environment in hash#5803
Merged
Conversation
jonb377
commented
Nov 15, 2023
Closed
JackCaoG
reviewed
Nov 15, 2023
JackCaoG
reviewed
Nov 15, 2023
Collaborator
JackCaoG
left a comment
There was a problem hiding this comment.
mostly lgtm, minor nits
will-cromar
approved these changes
Dec 1, 2023
yeounoh
reviewed
Dec 4, 2023
| } | ||
| xla_flags.push_back(current_flag); | ||
| } | ||
| // Ensure the flags are sorted so the input order doesn't impact the hash. |
Contributor
There was a problem hiding this comment.
Oh, didn't think of this. Nice catch!
yeounoh
reviewed
Dec 4, 2023
yeounoh
reviewed
Dec 4, 2023
| } | ||
|
|
||
| torch::lazy::hash_t PjRtComputationClient::HashCompilationEnv() { | ||
| // TODO(jonbolin): Incorporate CompileOptions into the hash. These are |
Contributor
There was a problem hiding this comment.
Once we do, we don't need the SPMD mode be part of the hash (virtual device on/off).
eeee59e to
d2190e4
Compare
d2190e4 to
aee73ba
Compare
jonb377
added a commit
that referenced
this pull request
Dec 7, 2023
* Incorporate compilation environment in hash * Add unit test * Incorporate SPMD mode in the hash * Always rehash in env_hash
ManfeiBai
pushed a commit
to ManfeiBai/PyTorchXLA
that referenced
this pull request
Dec 8, 2023
* Incorporate compilation environment in hash * Add unit test * Incorporate SPMD mode in the hash * Always rehash in env_hash
chunnienc
pushed a commit
to chunnienc/xla
that referenced
this pull request
Dec 14, 2023
* Incorporate compilation environment in hash * Add unit test * Incorporate SPMD mode in the hash * Always rehash in env_hash
golechwierowicz
pushed a commit
that referenced
this pull request
Jan 12, 2024
* Incorporate compilation environment in hash * Add unit test * Incorporate SPMD mode in the hash * Always rehash in env_hash
bhavya01
pushed a commit
that referenced
this pull request
Apr 22, 2024
* Incorporate compilation environment in hash * Add unit test * Incorporate SPMD mode in the hash * Always rehash in env_hash
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Include the compilation environment in the graph hash. This will ensure that options which impact the compilation result will change the hash, so the same traced IR graph will result in a different hash when the compilation environment is different.
Currently, the following env vars must be hashed:
XLA_FLAGS,LIBTPU_INIT_ARGS, andTPU_MEGACORE. All env vars which impact the hash must be included inenv_hash.cc, otherwise there can be spurious cache hits!This PR builds on #5800, but it doesn't depend on it and can be landed separately.