KAFKA-8731: InMemorySessionStore throws NullPointerException on startup#7132
Conversation
| @Override | ||
| public void remove(final Windowed<Bytes> sessionKey) { | ||
| final ConcurrentNavigableMap<Bytes, ConcurrentNavigableMap<Long, byte[]>> keyMap = endTimeMap.get(sessionKey.window().end()); | ||
| if (keyMap == null) { |
There was a problem hiding this comment.
How could keyMap be null, ie, why would we call remove()for a non-existing session (at least in the DSL)?
Not saying that the fix does not make sense, I am just wondering if we need an additional fix for the DSL?
There was a problem hiding this comment.
Well. Thinking about it twice, during restore maybe? Tombstone are preserved longer in the changelog, hence, for a deleted session we only have the tombstone. On restore, we would never see and "insert" but only the delete.
Maybe worth double checking in the DSL code anyway.
There was a problem hiding this comment.
I assume a tombstone was being restored, where the corresponding put was already cleaned up from the changelog?
There was a problem hiding this comment.
Answered yourself quicker than me. But ok, I'll look around regardless
There was a problem hiding this comment.
I'm fairly confident the only affected case is on the restore path when the changelog contains a tombstone that outlived its corresponding "put" -- during normal processing, if the delete arrives after the key has already been cleaned up it will be dropped as a late record anyways. Assuming we aren't accidentally sending double tombstones anywhere.
|
This bug basically make the in-memory store unusable in practice. We should have a proper ticket for tracking. It's not just a HOTFIX. |
|
Here's a Jira ticket: |
|
Thanks for the ticket! Java8 and Java 11/2.12 passed, Java 11/2.13 failed with known flaky test retest this, please |
mjsax
left a comment
There was a problem hiding this comment.
Thanks for the quick fix! LGTM.
bbejeck
left a comment
There was a problem hiding this comment.
Thanks for the fix @ableegoldman LGTM
|
Java 11 / 2.13 passed. Retest this please. |
|
Merged #7132 into trunk |
…up (#7132) Reviewers: Matthias J. Sax <mjsax@apache.org>, Bill Bejeck <bbejeck@gmail.com>
|
cherry-picked to 2.3 |
* apache-github/2.3: MINOR: Avoid dividing by zero (apache#7143) KAFKA-8731: InMemorySessionStore throws NullPointerException on startup (apache#7132) KAFKA-8715; Fix buggy reliance on state timestamp in static member.id generation (apache#7116) KAFKA-8678; Fix leave group protocol bug in throttling and error response (apache#7101)
…llPointerException on startup (apache#7132) TICKET = KAFKA-8731 LI_DESCRIPTION = EXIT_CRITERIA = HASH [f86f7e2] ORIGINAL_DESCRIPTION = Reviewers: Matthias J. Sax <mjsax@apache.org>, Bill Bejeck <bbejeck@gmail.com> (cherry picked from commit f86f7e2)
Should be cherry-picked to 2.3