octopus: New msgr2 crc and secure modes (msgr2.1)#35720
Merged
yuriw merged 21 commits intoceph:octopusfrom Jul 8, 2020
Merged
Conversation
Provide an iterator-like interface as initializer lists cannot be formed dynamically. Signed-off-by: Ilya Dryomov <idryomov@gmail.com> (cherry picked from commit 1fc5cc2)
It is unused and doesn't make much sense in TxHandler. Signed-off-by: Ilya Dryomov <idryomov@gmail.com> (cherry picked from commit b3e39b1)
OpenSSL supports in-place decryption so we can avoid allocating potentially multi-megabyte and strictly aligned buffer for each decryption operation. ProtocolV2 actually gets the alignment wrong: after read_frame_segment() allocates with cur_rx_desc.alignment, handle_read_frame_segment() effectively replaces that with segment_t::DEFAULT_ALIGNMENT. Signed-off-by: Ilya Dryomov <idryomov@gmail.com> (cherry picked from commit fe97a00)
Signed-off-by: Ilya Dryomov <idryomov@gmail.com> (cherry picked from commit e1d1f61)
Signed-off-by: Ilya Dryomov <idryomov@gmail.com> (cherry picked from commit 6b6d405)
Use it in ProtocolV2.h and later in unit tests. While at it, drop the unused len struct. Signed-off-by: Ilya Dryomov <idryomov@gmail.com> (cherry picked from commit c081f3c)
Start separating frame assembly and disassembly code from frame sending, receiving and handling code, so that assembly and disassembly pieces can be unit tested and hopefully also shared between different messengers (e.g. crimson). This commit factors out the assembly code from Frame. Signed-off-by: Ilya Dryomov <idryomov@gmail.com> (cherry picked from commit 872b125) Conflicts: src/msg/async/frames_v2.h [ commit 1a975fb ("msg/async: fix unnecessary 4 kB allocation in secure mode.") not in octopus ]
Factor out the disassembly code from ProtocolV2 and switch ProtocolV2 to FrameAssembler. Signed-off-by: Ilya Dryomov <idryomov@gmail.com> (cherry picked from commit b9e0cfe) Conflicts: src/msg/async/ProtocolV2.cc [ context: commit d3ec4c01d17 ("msg: Build target 'common' without using namespace in headers") not in octopus ]
l_msgr_recv_bytes calculation was never updated from msgr1. Signed-off-by: Ilya Dryomov <idryomov@gmail.com> (cherry picked from commit dcf30f5)
In preparation for msgr2,1, rename epilogue structs: epilogue_plain_block_t to epilogue_crc_rev0_block_t and epilogue_secure_block_t to epilogue_secure_rev0_block_t (rev0 stands for revision 0). Also, get rid of size constants that just disguise the struct type. Signed-off-by: Ilya Dryomov <idryomov@gmail.com> (cherry picked from commit 712915b)
Clarify that the frame can be aborted at any point after the preamble and the first segment are put on the wire. When that happens, the remaining segments (including the data segment) may be filled with zeros. Signed-off-by: Ilya Dryomov <idryomov@gmail.com> (cherry picked from commit bc835b8)
Implement msgr2.1-crc and msgr2.1-secure modes. Issues with existing msgr2.0-crc and msgr2.0-secure modes and their resolution will be described in doc/dev/msgr2.rst. Signed-off-by: Ilya Dryomov <idryomov@gmail.com> (cherry picked from commit 2966b2a)
Move to a 64-bit counter to avoid wrapping and having to reset the session before the counter repeats. This is in line with NIST Recommendation for GCM [1]: "... this Recommendation suggests, but does not require, that the leading (i.e., leftmost) 32 bits of the IV hold the fixed field; and that the trailing (i.e., rightmost) 64 bits hold the invocation field." See commit bb61e6a ("msg/async/ProtocolV2: avoid AES-GCM nonce reuse vulnerabilities"). [1] https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-38d.pdf Signed-off-by: Ilya Dryomov <idryomov@gmail.com> (cherry picked from commit 7e4388f)
Signed-off-by: Ilya Dryomov <idryomov@gmail.com> (cherry picked from commit 0b3f4c6)
In both msgr2.0 and msgr2.1, segments can be empty. In msgr2.1, epilogue can be empty as well. Handle both by calling the respective handler function directly instead of allocating a buffer::ptr_node for an empty buffer and passing that through READ[_RXBUF]. Signed-off-by: Ilya Dryomov <idryomov@gmail.com> (cherry picked from commit e6e3563)
We aren't interested in peer_required_features anywhere outside _handle_peer_banner_payload() -- once we know there is no mismatch, it's all about peer_supported_features. Signed-off-by: Ilya Dryomov <idryomov@gmail.com> (cherry picked from commit fd5d28f)
Use msgr2.1 if the peer supports it and fall back to msgr2.0 otherwise. Signed-off-by: Ilya Dryomov <idryomov@gmail.com> (cherry picked from commit afc2886)
Signed-off-by: Ilya Dryomov <idryomov@gmail.com> (cherry picked from commit 5eea038)
Contributor
Contributor
rzarzynski
approved these changes
Jun 25, 2020
Member
|
Marking DNM until https://tracker.ceph.com/issues/46180 is fixed. |
Currently it's a mix of hex and dec, making it hard to grep for. Converge on hex to match client_cookie. Signed-off-by: Ilya Dryomov <idryomov@gmail.com> (cherry picked from commit 779a8df)
reuse_connection() can be called on exproto in BANNER_CONNECTING (i.e. without peer_supported_features and with tx/rx_frame_asm set to msgr2.0), but this state isn't carried over. If the donor connection is msgr2.1, this leads to repeated connection faults on crc or auth tag mismatches because we end up assembling 2.0 frames while the peer is expecting 2.1 frames. Fixes: https://tracker.ceph.com/issues/46180 Signed-off-by: Ilya Dryomov <idryomov@gmail.com> (cherry picked from commit baff20a)
Contributor
Author
|
Pushed the fix for https://tracker.ceph.com/issues/46180 (#35816). -s rados/singleton-nomsgr --filter 'all/health-warnings rados' -N 20 / 200: https://pulpito.ceph.com/dis-2020-07-02_20:12:12-rados:singleton-nomsgr-wip-msgr21-octopus-distro-basic-smithi/ |
Contributor
Author
|
@neha-ojha @yuriw Taking DNM off. |
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of #35078 and #35816 (changes from #34927 which got merged via #35078 already in octopus).
Two minor conflicts, documented inline.