Skip to content

octopus: New msgr2 crc and secure modes (msgr2.1)#35720

Merged
yuriw merged 21 commits intoceph:octopusfrom
idryomov:wip-msgr21-octopus
Jul 8, 2020
Merged

octopus: New msgr2 crc and secure modes (msgr2.1)#35720
yuriw merged 21 commits intoceph:octopusfrom
idryomov:wip-msgr21-octopus

Conversation

@idryomov
Copy link
Contributor

@idryomov idryomov commented Jun 23, 2020

Backport of #35078 and #35816 (changes from #34927 which got merged via #35078 already in octopus).

Two minor conflicts, documented inline.

idryomov added 19 commits June 23, 2020 09:21
Provide an iterator-like interface as initializer lists cannot be
formed dynamically.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 1fc5cc2)
Add asserts to avoid bugs like the one fixed in 1a975fb
("msg/async: fix unnecessary 4 kB allocation in secure mode.").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 4894e59)
It is unused and doesn't make much sense in TxHandler.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit b3e39b1)
OpenSSL supports in-place decryption so we can avoid
allocating potentially multi-megabyte and strictly aligned
buffer for each decryption operation.

ProtocolV2 actually gets the alignment wrong: after
read_frame_segment() allocates with cur_rx_desc.alignment,
handle_read_frame_segment() effectively replaces that with
segment_t::DEFAULT_ALIGNMENT.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit fe97a00)
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit e1d1f61)
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 6b6d405)
Use it in ProtocolV2.h and later in unit tests.

While at it, drop the unused len struct.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit c081f3c)
Start separating frame assembly and disassembly code from
frame sending, receiving and handling code, so that assembly
and disassembly pieces can be unit tested and hopefully also
shared between different messengers (e.g. crimson).

This commit factors out the assembly code from Frame.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 872b125)

Conflicts:
	src/msg/async/frames_v2.h [ commit 1a975fb ("msg/async:
	  fix unnecessary 4 kB allocation in secure mode.") not in
	  octopus ]
Factor out the disassembly code from ProtocolV2 and switch
ProtocolV2 to FrameAssembler.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit b9e0cfe)

Conflicts:
	src/msg/async/ProtocolV2.cc [ context: commit d3ec4c01d17
	  ("msg: Build target 'common' without using namespace in
	  headers") not in octopus ]
l_msgr_recv_bytes calculation was never updated from msgr1.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit dcf30f5)
In preparation for msgr2,1, rename epilogue structs:
epilogue_plain_block_t to epilogue_crc_rev0_block_t and
epilogue_secure_block_t to epilogue_secure_rev0_block_t
(rev0 stands for revision 0).

Also, get rid of size constants that just disguise the
struct type.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 712915b)
Clarify that the frame can be aborted at any point after the
preamble and the first segment are put on the wire.  When that
happens, the remaining segments (including the data segment)
may be filled with zeros.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit bc835b8)
Implement msgr2.1-crc and msgr2.1-secure modes.

Issues with existing msgr2.0-crc and msgr2.0-secure modes and
their resolution will be described in doc/dev/msgr2.rst.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 2966b2a)
Move to a 64-bit counter to avoid wrapping and having to reset
the session before the counter repeats.  This is in line with NIST
Recommendation for GCM [1]:

  "... this Recommendation suggests, but does not require, that
  the leading (i.e., leftmost) 32 bits of the IV hold the fixed
  field; and that the trailing (i.e., rightmost) 64 bits hold the
  invocation field."

See commit bb61e6a ("msg/async/ProtocolV2: avoid AES-GCM nonce
reuse vulnerabilities").

[1] https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-38d.pdf

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 7e4388f)
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 0b3f4c6)
In both msgr2.0 and msgr2.1, segments can be empty.  In msgr2.1,
epilogue can be empty as well.

Handle both by calling the respective handler function directly
instead of allocating a buffer::ptr_node for an empty buffer and
passing that through READ[_RXBUF].

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit e6e3563)
We aren't interested in peer_required_features anywhere outside
_handle_peer_banner_payload() -- once we know there is no mismatch,
it's all about peer_supported_features.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit fd5d28f)
Use msgr2.1 if the peer supports it and fall back to msgr2.0
otherwise.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit afc2886)
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 5eea038)
@idryomov idryomov added feature messenger Issues involving one of the Ceph messenger implementations labels Jun 23, 2020
@idryomov idryomov added this to the octopus milestone Jun 23, 2020
@yuriw
Copy link
Contributor

yuriw commented Jun 23, 2020

@yuriw
Copy link
Contributor

yuriw commented Jun 25, 2020

@neha-ojha neha-ojha added the DNM label Jun 26, 2020
@neha-ojha
Copy link
Member

Marking DNM until https://tracker.ceph.com/issues/46180 is fixed.

idryomov added 2 commits July 2, 2020 13:17
Currently it's a mix of hex and dec, making it hard to grep for.
Converge on hex to match client_cookie.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 779a8df)
reuse_connection() can be called on exproto in BANNER_CONNECTING
(i.e. without peer_supported_features and with tx/rx_frame_asm set to
msgr2.0), but this state isn't carried over.  If the donor connection
is msgr2.1, this leads to repeated connection faults on crc or auth tag
mismatches because we end up assembling 2.0 frames while the peer is
expecting 2.1 frames.

Fixes: https://tracker.ceph.com/issues/46180
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit baff20a)
@idryomov
Copy link
Contributor Author

idryomov commented Jul 3, 2020

@idryomov
Copy link
Contributor Author

idryomov commented Jul 3, 2020

@neha-ojha @yuriw Taking DNM off.

@yuriw
Copy link
Contributor

yuriw commented Jul 6, 2020

@yuriw yuriw merged commit db83b8d into ceph:octopus Jul 8, 2020
@idryomov idryomov deleted the wip-msgr21-octopus branch July 15, 2020 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature messenger Issues involving one of the Ceph messenger implementations needs-qa octopus-batch-1 wip-yuri7-testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants