Various changes to reduce header dependencies in TLS#5296
Conversation
0b4373f to
f122bd1
Compare
a583a3c to
d887144
Compare
There was a problem hiding this comment.
Pull request overview
This PR aims to reduce TLS-related header dependencies (and thereby build times) by moving type definitions into more focused headers, replacing includes with forward declarations, and moving some inline/default implementations out-of-line.
Changes:
- Introduce
tls_session_id.hand move TLS session handle/ID types out oftls_session.h. - Reduce includes across TLS headers by using forward declarations and relocating helper APIs (e.g. certificate type conversions).
- Refactor session manager implementations and TLS 1.2 session ticket lifetime handling to reduce header coupling.
Reviewed changes
Copilot reviewed 49 out of 49 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| src/tests/unit_tls.cpp | Adds explicit include needed after dependency reductions |
| src/tests/test_tls_cipher_state.cpp | Adds missing STL include (<map>) |
| src/tests/test_tls.cpp | Adds missing TLS/STL includes after dependency changes |
| src/lib/x509/x509cert.h | Removes Usage_Type definition from this header |
| src/lib/x509/pkix_enums.h | Adds Usage_Type enum definition |
| src/lib/tls/tls_session_manager_stateless.h | Forward declarations + out-of-line default for find_some() |
| src/lib/tls/tls_session_manager_stateless.cpp | Adds find_some() implementation + includes cleanup |
| src/lib/tls/tls_session_manager_noop.h | Moves trivial overrides out-of-line |
| src/lib/tls/tls_session_manager_noop.cpp | Adds out-of-line trivial overrides + includes cleanup |
| src/lib/tls/tls_session_manager_hybrid.h | Moves trivial override out-of-line |
| src/lib/tls/tls_session_manager_hybrid.cpp | Adds out-of-line forwarding implementation + includes cleanup |
| src/lib/tls/tls_session_manager.h | Replaces tls_session.h include with forward decls + tls_session_id.h |
| src/lib/tls/tls_session_id.h | New header for session ID/ticket/handle types |
| src/lib/tls/tls_session.h | Uses new tls_session_id.h, adjusts Session_with_Handle visibility |
| src/lib/tls/tls_policy.h | Replaces heavy includes with tls_algos.h + forward decls |
| src/lib/tls/tls_policy.cpp | Restores concrete includes needed by implementation |
| src/lib/tls/tls_messages.h | Replaces heavy includes with narrower ones + forward decls |
| src/lib/tls/tls_extensions.h | Reduces dependencies; moves certificate-type helpers out; adjusts ctor signatures |
| src/lib/tls/tls_extensions.cpp | Moves required includes to implementation; removes moved helpers; adjusts ctor impls |
| src/lib/tls/tls_channel_impl.h | Adjusts includes for reduced coupling |
| src/lib/tls/tls_channel.h | Removes x509cert include and adds forward decls |
| src/lib/tls/tls_callbacks.h | Replaces x509cert include with pkix_enums to keep Usage_Type available |
| src/lib/tls/tls_callbacks.cpp | Adds include needed after header dependency changes |
| src/lib/tls/tls_algos.h | Adds Certificate_Type enum + helper declarations |
| src/lib/tls/tls_algos.cpp | Adds Certificate_Type helper implementations |
| src/lib/tls/tls13/tls_psk_identity_13.h | Include adjustments |
| src/lib/tls/tls13/tls_messages_13.h | Adds concrete includes + adjusts ctor signature to avoid heavy deps |
| src/lib/tls/tls13/msg_certificate_req_13.cpp | Adds required includes; updates to new ctor signature |
| src/lib/tls/tls12/tls_server_impl_12.cpp | Updates session ticket lifetime handling types/conversions |
| src/lib/tls/tls12/tls_messages_12.h | Refactors some APIs/destructors; changes ticket lifetime hint type |
| src/lib/tls/tls12/tls_handshake_state.cpp | Adds required include after dependency reduction |
| src/lib/tls/tls12/tls_client_impl_12.cpp | Adapts to ticket lifetime hint type change |
| src/lib/tls/tls12/tls_channel_impl_12.h | Removes tls_session.h include |
| src/lib/tls/tls12/msg_session_ticket_12.cpp | Updates ticket lifetime hint serialization/parsing to uint32_t |
| src/lib/tls/tls12/msg_certificate_req_12.cpp | Adds required includes; adds out-of-line destructor |
| src/lib/tls/tls12/msg_certificate_12.cpp | Adds required includes; adds out-of-line destructor + moves count() impl |
| src/lib/tls/tls12/msg_cert_verify_12.cpp | Adds required include after dependency reduction |
| src/lib/tls/sessions_sql/tls_session_manager_sql.h | Adds missing include for SymmetricKey |
| src/lib/tls/sessions_sql/tls_session_manager_sql.cpp | Adds include needed after header changes |
| src/lib/tls/msg_server_hello.cpp | Adds concrete includes needed after dependency reduction |
| src/lib/tls/msg_client_hello.cpp | Adds concrete includes needed after dependency reduction |
| src/lib/tls/info.txt | Exports new header tls_session_id.h |
| src/lib/tls/credentials_manager.h | Replaces heavy includes with forward declarations |
| src/lib/tls/credentials_manager.cpp | Restores concrete includes needed by implementation |
| src/lib/tls/asio/asio_context.cpp | Adds include needed after header dependency changes |
| src/fuzzer/tls_server.cpp | Adds explicit includes to satisfy reduced transitive deps |
| src/fuzzer/tls_client.cpp | Adds explicit includes to satisfy reduced transitive deps |
| src/cli/tls_helpers.h | Adds explicit include needed after dependency changes |
| src/bogo_shim/bogo_shim.cpp | Adds explicit includes needed after dependency changes |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
src/lib/tls/tls12/tls_messages_12.h
Outdated
|
|
||
| private: | ||
| std::chrono::seconds m_ticket_lifetime_hint{}; | ||
| uint32_t m_ticket_lifetime_hint; |
There was a problem hiding this comment.
m_ticket_lifetime_hint is now a plain uint32_t without a default initializer. The constructor New_Session_Ticket_12(Handshake_IO&, Handshake_Hash&) calls io.send(*this) which will serialize the object and read m_ticket_lifetime_hint before it is set, causing undefined behavior. Initialize the member (e.g., uint32_t m_ticket_lifetime_hint{0};) and/or set it explicitly in the constructor.
| uint32_t m_ticket_lifetime_hint; | |
| uint32_t m_ticket_lifetime_hint{0}; |
| pending_state.new_session_ticket(std::make_unique<New_Session_Ticket_12>( | ||
| pending_state.handshake_io(), | ||
| pending_state.hash(), | ||
| handle->ticket().value(), | ||
| static_cast<uint32_t>(policy().session_ticket_lifetime().count()))); |
There was a problem hiding this comment.
policy().session_ticket_lifetime().count() is narrowed to uint32_t without bounds checking. If a policy returns a very large duration, this truncates and could produce an incorrect lifetime hint on the wire. Consider clamping to UINT32_MAX (and ideally rejecting negative values) before casting.
| const uint32_t lifetime = static_cast<uint32_t>(policy().session_ticket_lifetime().count()); | ||
| pending_state.new_session_ticket(std::make_unique<New_Session_Ticket_12>( | ||
| pending_state.handshake_io(), pending_state.hash(), new_handle->ticket().value(), lifetime)); |
There was a problem hiding this comment.
session_ticket_lifetime().count() is cast to uint32_t here as well. To avoid truncation/undefined semantics for very large policy durations, clamp to UINT32_MAX (and handle negative durations if possible) before narrowing.
| #include <botan/mutex.h> | ||
| #include <botan/tls_session.h> | ||
| #include <botan/tls_session_id.h> | ||
| #include <botan/types.h> | ||
|
|
||
| #include <memory> | ||
| #include <optional> | ||
| #include <utility> | ||
| #include <vector> |
There was a problem hiding this comment.
This header now forward-declares Session_with_Handle/Session and returns std::vector<Session_with_Handle> by value (e.g. find(), find_some()). std::vector<T> requires T to be complete in translation units that construct/destroy the vector, so removing <botan/tls_session.h> is likely to break downstream builds. Consider including the header that defines Session_with_Handle (currently tls_session.h) or refactoring the API to avoid returning std::vector<Session_with_Handle> by value from this public header.
| #include <botan/symkey.h> | ||
| #include <botan/tls_alert.h> | ||
| #include <botan/x509cert.h> | ||
|
|
||
| #include <memory> | ||
| #include <optional> | ||
| #include <span> | ||
| #include <string> | ||
| #include <string_view> | ||
| #include <vector> | ||
|
|
||
| namespace Botan { | ||
|
|
||
| class Public_Key; | ||
| class X509_Certificate; | ||
|
|
||
| } // namespace Botan |
There was a problem hiding this comment.
This header now forward-declares X509_Certificate but still exposes std::vector<X509_Certificate> by value in the public API (peer_cert_chain()). std::vector<T> generally requires T to be complete in translation units that construct/destroy it, so removing <botan/x509cert.h> is likely to break downstream builds. Consider keeping the include (or changing the API to avoid std::vector<X509_Certificate> by value).
def05ae to
e37c21f
Compare
reneme
left a comment
There was a problem hiding this comment.
Generally looks good to me. One question regarding runtime vs. compile time efficiency.
|
|
||
| private: | ||
| std::chrono::seconds m_ticket_lifetime_hint{}; | ||
| uint32_t m_ticket_lifetime_hint = 0; |
There was a problem hiding this comment.
It really stinks that <chrono> is so heavy. Ditching ::seconds (and friends) as strong types does reduce the expressiveness of the code. :( ... it's what it is, I guess.
There was a problem hiding this comment.
I know, it's incredibly frustrating when the basic time unit types were so useful as interface types and then the committee decided to throw in so much stuff in to the header that they are no longer viable for that purpose.
IMO C++ is failing badly at both "You don't pay for what you don't use" and "Leave no room for a lower-level language" and the worst part is from all visible signs the committee thinks it's going great.
There was a problem hiding this comment.
#JustUseModules is the answer, I guess...
The way I see it, You don't pay for what you don't use" only accounts for the final binary. And there it delivers in most cases. But compile times are a significant issue if you don't happen to have a 64 core beast in your basement.
There was a problem hiding this comment.
The way I see it, You don't pay for what you don't use" only accounts for the final binary.
That's clearly their interpretation as well. Unfortunately that interpretation assumes developer wait time is free, which is not necessarily accurate.
|
|
||
| private: | ||
| Certificate_Request_13(std::vector<X509_DN> acceptable_CAs, const Policy& policy, Callbacks& callbacks); | ||
| Certificate_Request_13(const std::vector<X509_DN>& acceptable_CAs, const Policy& policy, Callbacks& callbacks); |
There was a problem hiding this comment.
Does it really hurt so much to pull in X509_DN ("pkix_types.h") here? In ::maybe_create this actually uses this lvalue to avoid a copy. Not that it is a particularly hot path in the code, but I'm wondering about the general rule-of-thumb for the trade-off between compile time and run time efficiency.
Similarly in Certificate_Authorities.
There was a problem hiding this comment.
I guess it's a matter of slow cuts. Adding pkix_types.h slows down compilation of tls_messages.h by about 8% (.71s->.76s for GCC, 1.0s->1.07s for Clang) which is not so bad. But tls_messages.h is then included into 40 or 50 TUs. So making it even slightly more expensive balloons out.
Where exactly the line is drawn is certainly open for debate but broadly
-
If something is included into many TUs it should be as light as possible. It doesn't matter if any single particular source file is slow. For example kuznyechik.o is one of the very slowest object files in the entire library to compile, due to fancy
constevaltricks, and do we really need to be doing those, no - but it's just one file and not hurting anything else so who cares. Whereas egconcepts.horloadstor.hbeing even 1ms faster to parse cuts 10s of seconds off the build. Also being common headers they are an ongoing tax; when another source file is added which includes them, the overall compilation cost of these headers grows further. -
If the performance doesn't matter much (for example, if something could cost twice as much and nobody would ever notice) then might as well prefer saving compilation time, since compilation time is a scarce resource of uncertain availability [1] and that resource is shared across the entire codebase.
[1]: In that there is no fixed hard budget but slower compilation time hurts developer velocity and likely as not hurts adoption if it's bad enough to notice.
In this specific case, I'd say: many TUs are affected, it's not really performance critical (only happens during the handshake and the handshake is anyway expensive), so prefer saving compilation time. I guess if we had profiles for the TLS handshake performance [which would be a useful benchmark to have!!] and the extra copy was showing up somehow, that would be worth working on.
As an example going the other way, headers like ct_utils.h and simd_4x32.h are somewhat expensive, about .25-.35 seconds to compile, and used everywhere. But these are run in hot loops that are written expecting values like CT::Mask and SIMD_4x32 will be perfectly inlined, it's difficult to reduce the compilation time meaningfully without drastically impacting performance.
There was a problem hiding this comment.
Thanks for the guidelines. I concur in general terms. Though, still I feel sad to introduce runtime efficiencies like that. But from a gut feeling I also don't think this particular copy would produce measurable runtime slowdowns. On that note, we're starting to use the TLS stack in a system that might actually experience some significant load. Therefore, benchmarks for TLS might come up on my TODO list eventually...
Anyway: with all the header patrol work you've put in, I think it would be really cool to have an automated compile benchmark in CI to catch regressions early and weigh the pros and cons of new API designs early on. (For instance the new loadstor.h is convenient but I do question whether it was really necessary to push the limits that far. 😶)
There was a problem hiding this comment.
I think it would be really cool to have an automated compile benchmark in CI to catch regressions early and weigh the pros and cons of new API designs early on.
Yes I have something specifically like this in mind to include in the nightly run (and can be invoked on demand if a PR seems risky in this way) - Clang's -ftime-trace dumps a lot of useful information.
e37c21f to
f0c730b
Compare
This cuts about 50 CPU-seconds off the time my machine needs to build TLS