Fix problem with union spliting during inlining, add regression test#42347
Merged
dkarrasch merged 4 commits intoJuliaLang:masterfrom Sep 23, 2021
Merged
Fix problem with union spliting during inlining, add regression test#42347dkarrasch merged 4 commits intoJuliaLang:masterfrom
dkarrasch merged 4 commits intoJuliaLang:masterfrom
Conversation
Member
|
Nice first contribution! ;) |
KristofferC
reviewed
Sep 22, 2021
JeffBezanson
approved these changes
Sep 22, 2021
Keno
reviewed
Sep 22, 2021
Keno
approved these changes
Sep 22, 2021
Member
Keno
left a comment
There was a problem hiding this comment.
For tricky compiler cases like these, particularly, when the actual diff is pretty small, I usually like to add a somewhat verbose commit message so that people who come across it later understand how the fix relates to the originally reported issue. Here, I would write something like:
The crash in #42264 showed up when attempting to record the new value for
an SSA rename during unionsplit inlining. Such a crash would only happen when
`compact.idx == 1`, which is an unusual condition, because it implies that no
statement of the original function had yet been processed (which is odd, because
the call being inlined must have been processed by this point). As it turns out, there
is currently exactly one case where this happens. If the inliner sees an `_apply_iterate`
(e.g. from splatting) of something that is not a Tuple or SimpleVector (thus
requiring calls to `iterate` to expand the value during `apply`), and if inference
was able to determine the total length of the resulting iteration, a special case early
inliner, will expand out the splat into explicit iterate calls. E.g. a call like:
%r = tuple((x::Pair)...)
will be expanded out to
%a = iterate(x::Pair)
%b = getfield(%a, 1)
%c = getfield(%a, 2)
%d = iterate(x, %c)
%e = getfield(%d, 1)
%f = getfield(%d, 2)
iterate(x, %f)
%r = tuple(%b, %e)
where the inserted `iterate` calls may themselves be inlined. These newly inserted calls are
"pending nodes" during the actual inlining. Thus, if the original apply call was the first statement
of the function, these nodes would be processed before processing the statements in the function themselves.
In particular, this investigation also shows that `compact.idx`, which is the current location
in the function being inlined into is the wrong value to use for SSA renaming. Rather, we
need to use the SSA value of the just-inserted statement. In the absence of pending nodes,
these are equivalent concepts, but not for pending nodes. Fortunately the IncrementalCompact
iterator provides the old SSA value for just this reason and in fact, non-UnionSplit inlining was
already correct here. Thus, to fix the issue, simply adjust union splitting to work the same way
as a non-UnionSplit inline.
In coming up with the test case, an additional complication is that we currently do not perform this optimization for any calls where the apply call itself was unionsplit. It
is somewhat unusual (which explains why we haven't seen this much) for the apply
to not be union-split, but for the subsequent `iterate` to require union-splitting. In the
problem case, what happened is that for two out of the three union cases, the
`iterate` calls themselves would error, causing the resulting info object to look like
a non-unionsplit apply call, triggering this issue.
Co-authored-by: Keno Fischer <keno@alumni.harvard.edu>
This was referenced Sep 23, 2021
KristofferC
pushed a commit
that referenced
this pull request
Sep 23, 2021
…42347) The crash in #42264 showed up when attempting to record the new value for an SSA rename during unionsplit inlining. Such a crash would only happen when `compact.idx == 1`, which is an unusual condition, because it implies that no statement of the original function had yet been processed (which is odd, because the call being inlined must have been processed by this point). As it turns out, there is currently exactly one case where this happens. If the inliner sees an `_apply_iterate` (e.g. from splatting) of something that is not a Tuple or SimpleVector (thus requiring calls to `iterate` to expand the value during `apply`), and if inference was able to determine the total length of the resulting iteration, a special case early inliner, will expand out the splat into explicit iterate calls. E.g. a call like: %r = tuple((x::Pair)...) will be expanded out to %a = iterate(x::Pair) %b = getfield(%a, 1) %c = getfield(%a, 2) %d = iterate(x, %c) %e = getfield(%d, 1) %f = getfield(%d, 2) iterate(x, %f) %r = tuple(%b, %e) where the inserted `iterate` calls may themselves be inlined. These newly inserted calls are "pending nodes" during the actual inlining. Thus, if the original apply call was the first statement of the function, these nodes would be processed before processing the statements in the function themselves. In particular, this investigation also shows that `compact.idx`, which is the current location in the function being inlined into is the wrong value to use for SSA renaming. Rather, we need to use the SSA value of the just-inserted statement. In the absence of pending nodes, these are equivalent concepts, but not for pending nodes. Fortunately the IncrementalCompact iterator provides the old SSA value for just this reason and in fact, non-UnionSplit inlining was already correct here. Thus, to fix the issue, simply adjust union splitting to work the same way as a non-UnionSplit inline. In coming up with the test case, an additional complication is that we currently do not perform this optimization for any calls where the apply call itself was unionsplit. It is somewhat unusual (which explains why we haven't seen this much) for the apply to not be union-split, but for the subsequent `iterate` to require union-splitting. In the problem case, what happened is that for two out of the three union cases, the `iterate` calls themselves would error, causing the resulting info object to look like a non-unionsplit apply call, triggering this issue. Co-authored-by: Keno Fischer <keno@alumni.harvard.edu> (cherry picked from commit b5f3a99)
KristofferC
pushed a commit
that referenced
this pull request
Sep 28, 2021
…42347) The crash in #42264 showed up when attempting to record the new value for an SSA rename during unionsplit inlining. Such a crash would only happen when `compact.idx == 1`, which is an unusual condition, because it implies that no statement of the original function had yet been processed (which is odd, because the call being inlined must have been processed by this point). As it turns out, there is currently exactly one case where this happens. If the inliner sees an `_apply_iterate` (e.g. from splatting) of something that is not a Tuple or SimpleVector (thus requiring calls to `iterate` to expand the value during `apply`), and if inference was able to determine the total length of the resulting iteration, a special case early inliner, will expand out the splat into explicit iterate calls. E.g. a call like: %r = tuple((x::Pair)...) will be expanded out to %a = iterate(x::Pair) %b = getfield(%a, 1) %c = getfield(%a, 2) %d = iterate(x, %c) %e = getfield(%d, 1) %f = getfield(%d, 2) iterate(x, %f) %r = tuple(%b, %e) where the inserted `iterate` calls may themselves be inlined. These newly inserted calls are "pending nodes" during the actual inlining. Thus, if the original apply call was the first statement of the function, these nodes would be processed before processing the statements in the function themselves. In particular, this investigation also shows that `compact.idx`, which is the current location in the function being inlined into is the wrong value to use for SSA renaming. Rather, we need to use the SSA value of the just-inserted statement. In the absence of pending nodes, these are equivalent concepts, but not for pending nodes. Fortunately the IncrementalCompact iterator provides the old SSA value for just this reason and in fact, non-UnionSplit inlining was already correct here. Thus, to fix the issue, simply adjust union splitting to work the same way as a non-UnionSplit inline. In coming up with the test case, an additional complication is that we currently do not perform this optimization for any calls where the apply call itself was unionsplit. It is somewhat unusual (which explains why we haven't seen this much) for the apply to not be union-split, but for the subsequent `iterate` to require union-splitting. In the problem case, what happened is that for two out of the three union cases, the `iterate` calls themselves would error, causing the resulting info object to look like a non-unionsplit apply call, triggering this issue. Co-authored-by: Keno Fischer <keno@alumni.harvard.edu>
95 tasks
KristofferC
pushed a commit
that referenced
this pull request
Oct 29, 2021
…42347) The crash in #42264 showed up when attempting to record the new value for an SSA rename during unionsplit inlining. Such a crash would only happen when `compact.idx == 1`, which is an unusual condition, because it implies that no statement of the original function had yet been processed (which is odd, because the call being inlined must have been processed by this point). As it turns out, there is currently exactly one case where this happens. If the inliner sees an `_apply_iterate` (e.g. from splatting) of something that is not a Tuple or SimpleVector (thus requiring calls to `iterate` to expand the value during `apply`), and if inference was able to determine the total length of the resulting iteration, a special case early inliner, will expand out the splat into explicit iterate calls. E.g. a call like: %r = tuple((x::Pair)...) will be expanded out to %a = iterate(x::Pair) %b = getfield(%a, 1) %c = getfield(%a, 2) %d = iterate(x, %c) %e = getfield(%d, 1) %f = getfield(%d, 2) iterate(x, %f) %r = tuple(%b, %e) where the inserted `iterate` calls may themselves be inlined. These newly inserted calls are "pending nodes" during the actual inlining. Thus, if the original apply call was the first statement of the function, these nodes would be processed before processing the statements in the function themselves. In particular, this investigation also shows that `compact.idx`, which is the current location in the function being inlined into is the wrong value to use for SSA renaming. Rather, we need to use the SSA value of the just-inserted statement. In the absence of pending nodes, these are equivalent concepts, but not for pending nodes. Fortunately the IncrementalCompact iterator provides the old SSA value for just this reason and in fact, non-UnionSplit inlining was already correct here. Thus, to fix the issue, simply adjust union splitting to work the same way as a non-UnionSplit inline. In coming up with the test case, an additional complication is that we currently do not perform this optimization for any calls where the apply call itself was unionsplit. It is somewhat unusual (which explains why we haven't seen this much) for the apply to not be union-split, but for the subsequent `iterate` to require union-splitting. In the problem case, what happened is that for two out of the three union cases, the `iterate` calls themselves would error, causing the resulting info object to look like a non-unionsplit apply call, triggering this issue. Co-authored-by: Keno Fischer <keno@alumni.harvard.edu> (cherry picked from commit b5f3a99)
KristofferC
pushed a commit
that referenced
this pull request
Oct 29, 2021
…42347) The crash in #42264 showed up when attempting to record the new value for an SSA rename during unionsplit inlining. Such a crash would only happen when `compact.idx == 1`, which is an unusual condition, because it implies that no statement of the original function had yet been processed (which is odd, because the call being inlined must have been processed by this point). As it turns out, there is currently exactly one case where this happens. If the inliner sees an `_apply_iterate` (e.g. from splatting) of something that is not a Tuple or SimpleVector (thus requiring calls to `iterate` to expand the value during `apply`), and if inference was able to determine the total length of the resulting iteration, a special case early inliner, will expand out the splat into explicit iterate calls. E.g. a call like: %r = tuple((x::Pair)...) will be expanded out to %a = iterate(x::Pair) %b = getfield(%a, 1) %c = getfield(%a, 2) %d = iterate(x, %c) %e = getfield(%d, 1) %f = getfield(%d, 2) iterate(x, %f) %r = tuple(%b, %e) where the inserted `iterate` calls may themselves be inlined. These newly inserted calls are "pending nodes" during the actual inlining. Thus, if the original apply call was the first statement of the function, these nodes would be processed before processing the statements in the function themselves. In particular, this investigation also shows that `compact.idx`, which is the current location in the function being inlined into is the wrong value to use for SSA renaming. Rather, we need to use the SSA value of the just-inserted statement. In the absence of pending nodes, these are equivalent concepts, but not for pending nodes. Fortunately the IncrementalCompact iterator provides the old SSA value for just this reason and in fact, non-UnionSplit inlining was already correct here. Thus, to fix the issue, simply adjust union splitting to work the same way as a non-UnionSplit inline. In coming up with the test case, an additional complication is that we currently do not perform this optimization for any calls where the apply call itself was unionsplit. It is somewhat unusual (which explains why we haven't seen this much) for the apply to not be union-split, but for the subsequent `iterate` to require union-splitting. In the problem case, what happened is that for two out of the three union cases, the `iterate` calls themselves would error, causing the resulting info object to look like a non-unionsplit apply call, triggering this issue. Co-authored-by: Keno Fischer <keno@alumni.harvard.edu> (cherry picked from commit b5f3a99)
66 tasks
KristofferC
pushed a commit
that referenced
this pull request
Nov 11, 2021
…42347) The crash in #42264 showed up when attempting to record the new value for an SSA rename during unionsplit inlining. Such a crash would only happen when `compact.idx == 1`, which is an unusual condition, because it implies that no statement of the original function had yet been processed (which is odd, because the call being inlined must have been processed by this point). As it turns out, there is currently exactly one case where this happens. If the inliner sees an `_apply_iterate` (e.g. from splatting) of something that is not a Tuple or SimpleVector (thus requiring calls to `iterate` to expand the value during `apply`), and if inference was able to determine the total length of the resulting iteration, a special case early inliner, will expand out the splat into explicit iterate calls. E.g. a call like: %r = tuple((x::Pair)...) will be expanded out to %a = iterate(x::Pair) %b = getfield(%a, 1) %c = getfield(%a, 2) %d = iterate(x, %c) %e = getfield(%d, 1) %f = getfield(%d, 2) iterate(x, %f) %r = tuple(%b, %e) where the inserted `iterate` calls may themselves be inlined. These newly inserted calls are "pending nodes" during the actual inlining. Thus, if the original apply call was the first statement of the function, these nodes would be processed before processing the statements in the function themselves. In particular, this investigation also shows that `compact.idx`, which is the current location in the function being inlined into is the wrong value to use for SSA renaming. Rather, we need to use the SSA value of the just-inserted statement. In the absence of pending nodes, these are equivalent concepts, but not for pending nodes. Fortunately the IncrementalCompact iterator provides the old SSA value for just this reason and in fact, non-UnionSplit inlining was already correct here. Thus, to fix the issue, simply adjust union splitting to work the same way as a non-UnionSplit inline. In coming up with the test case, an additional complication is that we currently do not perform this optimization for any calls where the apply call itself was unionsplit. It is somewhat unusual (which explains why we haven't seen this much) for the apply to not be union-split, but for the subsequent `iterate` to require union-splitting. In the problem case, what happened is that for two out of the three union cases, the `iterate` calls themselves would error, causing the resulting info object to look like a non-unionsplit apply call, triggering this issue. Co-authored-by: Keno Fischer <keno@alumni.harvard.edu> (cherry picked from commit b5f3a99)
LilithHafner
pushed a commit
to LilithHafner/julia
that referenced
this pull request
Feb 22, 2022
…uliaLang#42347) The crash in JuliaLang#42264 showed up when attempting to record the new value for an SSA rename during unionsplit inlining. Such a crash would only happen when `compact.idx == 1`, which is an unusual condition, because it implies that no statement of the original function had yet been processed (which is odd, because the call being inlined must have been processed by this point). As it turns out, there is currently exactly one case where this happens. If the inliner sees an `_apply_iterate` (e.g. from splatting) of something that is not a Tuple or SimpleVector (thus requiring calls to `iterate` to expand the value during `apply`), and if inference was able to determine the total length of the resulting iteration, a special case early inliner, will expand out the splat into explicit iterate calls. E.g. a call like: %r = tuple((x::Pair)...) will be expanded out to %a = iterate(x::Pair) %b = getfield(%a, 1) %c = getfield(%a, 2) %d = iterate(x, %c) %e = getfield(%d, 1) %f = getfield(%d, 2) iterate(x, %f) %r = tuple(%b, %e) where the inserted `iterate` calls may themselves be inlined. These newly inserted calls are "pending nodes" during the actual inlining. Thus, if the original apply call was the first statement of the function, these nodes would be processed before processing the statements in the function themselves. In particular, this investigation also shows that `compact.idx`, which is the current location in the function being inlined into is the wrong value to use for SSA renaming. Rather, we need to use the SSA value of the just-inserted statement. In the absence of pending nodes, these are equivalent concepts, but not for pending nodes. Fortunately the IncrementalCompact iterator provides the old SSA value for just this reason and in fact, non-UnionSplit inlining was already correct here. Thus, to fix the issue, simply adjust union splitting to work the same way as a non-UnionSplit inline. In coming up with the test case, an additional complication is that we currently do not perform this optimization for any calls where the apply call itself was unionsplit. It is somewhat unusual (which explains why we haven't seen this much) for the apply to not be union-split, but for the subsequent `iterate` to require union-splitting. In the problem case, what happened is that for two out of the three union cases, the `iterate` calls themselves would error, causing the resulting info object to look like a non-unionsplit apply call, triggering this issue. Co-authored-by: Keno Fischer <keno@alumni.harvard.edu>
LilithHafner
pushed a commit
to LilithHafner/julia
that referenced
this pull request
Mar 8, 2022
…uliaLang#42347) The crash in JuliaLang#42264 showed up when attempting to record the new value for an SSA rename during unionsplit inlining. Such a crash would only happen when `compact.idx == 1`, which is an unusual condition, because it implies that no statement of the original function had yet been processed (which is odd, because the call being inlined must have been processed by this point). As it turns out, there is currently exactly one case where this happens. If the inliner sees an `_apply_iterate` (e.g. from splatting) of something that is not a Tuple or SimpleVector (thus requiring calls to `iterate` to expand the value during `apply`), and if inference was able to determine the total length of the resulting iteration, a special case early inliner, will expand out the splat into explicit iterate calls. E.g. a call like: %r = tuple((x::Pair)...) will be expanded out to %a = iterate(x::Pair) %b = getfield(%a, 1) %c = getfield(%a, 2) %d = iterate(x, %c) %e = getfield(%d, 1) %f = getfield(%d, 2) iterate(x, %f) %r = tuple(%b, %e) where the inserted `iterate` calls may themselves be inlined. These newly inserted calls are "pending nodes" during the actual inlining. Thus, if the original apply call was the first statement of the function, these nodes would be processed before processing the statements in the function themselves. In particular, this investigation also shows that `compact.idx`, which is the current location in the function being inlined into is the wrong value to use for SSA renaming. Rather, we need to use the SSA value of the just-inserted statement. In the absence of pending nodes, these are equivalent concepts, but not for pending nodes. Fortunately the IncrementalCompact iterator provides the old SSA value for just this reason and in fact, non-UnionSplit inlining was already correct here. Thus, to fix the issue, simply adjust union splitting to work the same way as a non-UnionSplit inline. In coming up with the test case, an additional complication is that we currently do not perform this optimization for any calls where the apply call itself was unionsplit. It is somewhat unusual (which explains why we haven't seen this much) for the apply to not be union-split, but for the subsequent `iterate` to require union-splitting. In the problem case, what happened is that for two out of the three union cases, the `iterate` calls themselves would error, causing the resulting info object to look like a non-unionsplit apply call, triggering this issue. Co-authored-by: Keno Fischer <keno@alumni.harvard.edu>
staticfloat
pushed a commit
that referenced
this pull request
Dec 23, 2022
…42347) The crash in #42264 showed up when attempting to record the new value for an SSA rename during unionsplit inlining. Such a crash would only happen when `compact.idx == 1`, which is an unusual condition, because it implies that no statement of the original function had yet been processed (which is odd, because the call being inlined must have been processed by this point). As it turns out, there is currently exactly one case where this happens. If the inliner sees an `_apply_iterate` (e.g. from splatting) of something that is not a Tuple or SimpleVector (thus requiring calls to `iterate` to expand the value during `apply`), and if inference was able to determine the total length of the resulting iteration, a special case early inliner, will expand out the splat into explicit iterate calls. E.g. a call like: %r = tuple((x::Pair)...) will be expanded out to %a = iterate(x::Pair) %b = getfield(%a, 1) %c = getfield(%a, 2) %d = iterate(x, %c) %e = getfield(%d, 1) %f = getfield(%d, 2) iterate(x, %f) %r = tuple(%b, %e) where the inserted `iterate` calls may themselves be inlined. These newly inserted calls are "pending nodes" during the actual inlining. Thus, if the original apply call was the first statement of the function, these nodes would be processed before processing the statements in the function themselves. In particular, this investigation also shows that `compact.idx`, which is the current location in the function being inlined into is the wrong value to use for SSA renaming. Rather, we need to use the SSA value of the just-inserted statement. In the absence of pending nodes, these are equivalent concepts, but not for pending nodes. Fortunately the IncrementalCompact iterator provides the old SSA value for just this reason and in fact, non-UnionSplit inlining was already correct here. Thus, to fix the issue, simply adjust union splitting to work the same way as a non-UnionSplit inline. In coming up with the test case, an additional complication is that we currently do not perform this optimization for any calls where the apply call itself was unionsplit. It is somewhat unusual (which explains why we haven't seen this much) for the apply to not be union-split, but for the subsequent `iterate` to require union-splitting. In the problem case, what happened is that for two out of the three union cases, the `iterate` calls themselves would error, causing the resulting info object to look like a non-unionsplit apply call, triggering this issue. Co-authored-by: Keno Fischer <keno@alumni.harvard.edu> (cherry picked from commit b5f3a99)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix for gridap/Gridap.jl#657 and #42264
More detail from @Keno :
"The crash in #42264 showed up when attempting to record the new value for
an SSA rename during unionsplit inlining. Such a crash would only happen when
compact.idx == 1, which is an unusual condition, because it implies that nostatement of the original function had yet been processed (which is odd, because
the call being inlined must have been processed by this point). As it turns out, there
is currently exactly one case where this happens. If the inliner sees an
_apply_iterate(e.g. from splatting) of something that is not a Tuple or SimpleVector (thus
requiring calls to
iterateto expand the value duringapply), and if inferencewas able to determine the total length of the resulting iteration, a special case early
inliner, will expand out the splat into explicit iterate calls. E.g. a call like:
will be expanded out to
where the inserted
iteratecalls may themselves be inlined. These newly inserted calls are"pending nodes" during the actual inlining. Thus, if the original apply call was the first statement
of the function, these nodes would be processed before processing the statements in the function themselves.
In particular, this investigation also shows that
compact.idx, which is the current locationin the function being inlined into is the wrong value to use for SSA renaming. Rather, we
need to use the SSA value of the just-inserted statement. In the absence of pending nodes,
these are equivalent concepts, but not for pending nodes. Fortunately the IncrementalCompact
iterator provides the old SSA value for just this reason and in fact, non-UnionSplit inlining was
already correct here. Thus, to fix the issue, simply adjust union splitting to work the same way
as a non-UnionSplit inline.
In coming up with the test case, an additional complication is that we currently do not perform this optimization for any calls where the apply call itself was unionsplit. It
is somewhat unusual (which explains why we haven't seen this much) for the apply
to not be union-split, but for the subsequent
iterateto require union-splitting. In theproblem case, what happened is that for two out of the three union cases, the
iteratecalls themselves would error, causing the resulting info object to look likea non-unionsplit apply call, triggering this issue."