Limited TLS 1.3 Client

Pull Request Dependencies

• The RFC 8448 based test uses the Test::Result consolidation constructor from this pull request. For simplicity the required commit is duplicated here and should be removed after the consolidation constructor is finalized and merged.

Follow-up Pull Requests

The number of open pull requests that are based on this are getting somewhat out of hand. Below is a list to keep track of the dependencies:

• THIS ONE: [TLS 1.3] Limited TLS 1.3 Client Implementation #2922
  • [TLS 1.3] Client Authentication in the main Handshake #2957
  • [TLS 1.3] Session Resumption #2974
  • [TLS 1.3] Fuzz Target for Handshake Message Parsing #2977
  • [TLS 1.3] Post-Quantum Readiness via Hybrid Key Exchange #2983
  • [TLS 1.3] Overhaul TLS::Callbacks #2988
  • [TLS 1.3] Overhaul TLS::Policy #2989
  • INFORMATIVE: [TLS 1.3] Full Integration #2981
  • INFORMATIVE: Post-Quantum TLS experiments based on KEMTLS #2954

Potential Future Work

This is a non-exhaustive list of potential improvements or further development. Note that those points are not tackled by the follow-up PRs. I'll leave this here for future reference.

• Overhaul of TLS_Data_Reader
  • Some of its methods leave room for performance improvements
  • The interface could be easier to use
  • More invariant checks (e.g. assert !has_remaining() on destruction).
  • Also a "sub-reader" concept: i.e. a new ::get_tls_length_value_as_reader() could return a sub-reader that can then be passed into a sub-parser (e.g. Client Hello parsing its extensions)
  • Use it consistently for parsing
• Allow disabling TLS 1.2 at compile time
• std::span to avoid copying
  • C++17 doesn't actually provide it, but a custom implementation should be straight forward
• Separate TLS 1.2 and 1.3 cipher suites
  • TLS 1.3 suites are more focussed
    (some protocol parameters are negotiated differently)
  • the Ciphersuite class provides methods that are strictly not defined for TLS 1.3 (potential cause for bugs)
  • cipher suite handling code becomes more convoluted that necessary
    (e.g. Policy::ciphersuite_list(), generation via tls_suite_info.py)

This PR adds limited TLS 1.3 support. Available functionality:

• functional TLS 1.3 client
  • automatically downgrades to TLS 1.2 if allowed and required by the server
  • usable via Botan-CLI tls_client
• additional extensions as required per RFC 8446:
  • Key_Share
  • Supported_Versions
  • Cookie
  • Signature Algorithms
  • Signature Algorithms Certificate
  • Negotiated Groups
• additional extensions (not strictly required by RFC 8446)
  • Certificate Status Request (OCSP stapling)
  • Record_Size_Limit (RFC 8449)
  • Application Layer Protocol Notification (RFC 7301)
  • Server Name Indication
• Utility and Misc
  • ./botan tls_client --debug to print raw TLS traffic
  • Fixed_Output_RNG can optionally fall back to another RNG when its pool is empty
• Tests integrated or performed
  • New infrastructure has unit tests
  • Integration tests using the test vectors from RFC 8448
  • BoGo tests that do not depend on not-yet-implemented protocol features
    (see here for a list of issues found by BoGo)
  • Handshake and data exchange works using ./botan tls_client with the big players (google.com, cloudflare.com, ...)

Limitations:

• TLS 1.3 support cannot be compiled without enabling TLS 1.2
  (TLS 1.2 can be disabled in the TLS policy though)
• no TLS 1.3 compatible server
• no client authentication (added in: [TLS 1.3] Client Authentication in the main Handshake #2957)
• no session resumption via Tickets (added in: [TLS 1.3] Session Resumption #2974)
• no Pre-Shared-Keys (technical foundation added in: [TLS 1.3] Session Resumption #2974)
• no 0-RTT support (early data) (technical foundation added in: [TLS 1.3] Session Resumption #2974)
• no DTLS
• no fuzzing of the TLS 1.3 implementation (basic fuzz target for parser: [TLS 1.3] Fuzz Target for Handshake Message Parsing #2977)

Implementation Overview

To make this large PR a bit more approachable, we'll provide an overview to the newly introduced components and their responsibilities. Note that this work-in-progress pull request strives towards a usable implementation of "Minimal Viable Product" as outlined in the ToDo-List of this issue comment.

TODO

• server certificate validation
  • OCSP stapling
• Hello Retry Request
• handle CLOSE_NOTIFY properly (difference between TLS 1.2 and 1.3)
• protocol version downgrade
• Key Update
• Key Material Export (RFC 8446 7.5 / RFC 5705)
• review TLS callbacks invocations
• bogo test suite integration
• check occurrences of extensions as indicated in the table in RFC 8446 4.2
• configure ClientHello settings (e.g. ALPN)
• honor the record size limit extension
• Session Resumption (PSK w/o 0-RTT) not part of this PR

Intro

The descriptions mostly refer to the components in the tls13 module, as the TLS 1.2 implementation remains unchanged as far as possible. As initiated in the refactoring by Elektrobit Automotive GmbH, the user-facing interface of TLS::Client and TLS::Server also remains unchanged by means of a Pimpl-pattern.

We nevertheless anticipate some API changes in the public API. Please refer to this issue comment for further details.

Components

https://excalidraw.com/#json=uJjsg_me6QrHsybCbkgxs,Ql21IMba1ZERgxfIiFElKg

Channel

The Channel acts as "composition root" of the entire TLS 1.3 implementation. It implements the library's public APIs via a Pimpl-construction as detailed in this issue comment.
This class implements the client/server agnostic parts of the protocol. Hence, orchestrating the Record_Layer and Handshake_Layer to transform received bytes from the network into handshake messages to be interpreted by the Client (and later Server) implementations or application data to be passed to the using code. Furthermore, it takes care of handling TLS alerts.

Record_Layer

This layer implements the record layer level of the protocol. It parses raw data from the wire to individual records and decrypts protected records using Cipher_State. This corresponds to tasks performed by the Channel (Channel_Impl_12) and the Record_Header class in the 1.2 implementation.
When sending data it add record headers and encryption to the records.

Handshake_Layer

This class takes care of parsing and marshalling of Handshake_Messages as well as equipping them with the appropriate handshake protocol headers. As this class handles the byte-representation of all TLS handshake messages (i.e. before parsing or after serialization), it is responsible for updating the Transcript_Hash_State appropriately.

In TLS 1.2 the task of the Handshake_Layer were mostly implemented in the Handshake_IO class. Note that Handshake_Layer does not perform any handshake state validations. It simply parses/serializes messages as they come in from the wire or the downstream protocol implementation.

Cipher_State

This class implments the Key Schedule mechanism. Most importantly, it derives and holds traffic secrets and provides interfaces for the Record_Layer protect and deprotect records. The Client advances the Cipher_State through the Key Schedule "state machine".

Transcript_Hash_State

This class keeps track of the transcript hash while sending/receiving messages in the Handshake_Layer. Client (and later Server) consult the Transcript_Hash_State for relevant hash-data when updating the Cipher_State appropriately.

Client

The Client's main responsibility continues to lie in process_handshake_message, implemented via individual handle-methods for each specific message type. Hence, the client's TLS state machine is implemented here. During message processing, it updates the state of other components:

• advancing the Key Schedule in Cipher_State
• consult the Transcript_Hash_State
• manage expected next messages in Handshake_Transitions

Handshake_Transitions

This class aids the Client implementation in validating state transitions. It is merely an extraction of the confirm_transition_to and set_expected_next functionality from TLS 1.2's Handshake_State.

Handshake_State

HandshakeState manages the incoming and outgoing messages and keeps a record of them for later reference. Before storing the messages it filters them regarding the Outbound_Message and Inbound_Message variants. Hence, a misuse of Handshake_State will fail to compile (e.g. when a client implementation tries to send a Server_Hello_13). Similarly, the compiler can check that Client implements handle() methods for exactly the relevant handshake messages.

Handshake_Messages

The class hierarchy of Handshake_Messages implements the specifics of individual handshake messages. In the TLS 1.3 implementation we don't rely on the actual polymorphic nature of Handshake_Message but replace it with std::variant constructions to specifically define which handshake message types are expected where.

Each handshake message takes care of "local" protocol semantic checks. I.e. all validations that can be performed without contextual information is happening right in the parsing code. The same holds true for handshake message extensions (e.g. Key_Share, Supported_Groups, ...). Protocol logic that is tied to specific messages or extensions is implemented locally in those classes. For instance, Key_Share implements Diffie-Hellman in its exchange method and Client_Hello_13::retry() implements necessary updates and validations for handling "hello retry requests".

This is a different paradigm compared to the messages in TLS 1.2, where many messages where mere "Data Transfer Objects" without much logic. Hence, messages derive a _12 and _13 sub-class and share the parsing code in the parent class where possible. This is also useful in situations where the interface and usage of the same message type differs between TLS 1.2 and TLS 1.3, but the wire representation is unchanged.