Skip to content

Various changes to reduce header dependencies in TLS#5296

Merged
randombit merged 1 commit intomasterfrom
jack/tls-header-patrol
Feb 10, 2026
Merged

Various changes to reduce header dependencies in TLS#5296
randombit merged 1 commit intomasterfrom
jack/tls-header-patrol

Conversation

@randombit
Copy link
Copy Markdown
Owner

This cuts about 50 CPU-seconds off the time my machine needs to build TLS

@randombit randombit force-pushed the jack/tls-header-patrol branch 4 times, most recently from 0b4373f to f122bd1 Compare February 7, 2026 16:57
@coveralls
Copy link
Copy Markdown

coveralls commented Feb 7, 2026

Coverage Status

coverage: 90.075% (+0.003%) from 90.072%
when pulling f0c730b on jack/tls-header-patrol
into 6689463 on master.

@randombit randombit force-pushed the jack/tls-header-patrol branch 2 times, most recently from a583a3c to d887144 Compare February 7, 2026 19:19
@randombit randombit requested a review from Copilot February 7, 2026 19:19
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to reduce TLS-related header dependencies (and thereby build times) by moving type definitions into more focused headers, replacing includes with forward declarations, and moving some inline/default implementations out-of-line.

Changes:

  • Introduce tls_session_id.h and move TLS session handle/ID types out of tls_session.h.
  • Reduce includes across TLS headers by using forward declarations and relocating helper APIs (e.g. certificate type conversions).
  • Refactor session manager implementations and TLS 1.2 session ticket lifetime handling to reduce header coupling.

Reviewed changes

Copilot reviewed 49 out of 49 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
src/tests/unit_tls.cpp Adds explicit include needed after dependency reductions
src/tests/test_tls_cipher_state.cpp Adds missing STL include (<map>)
src/tests/test_tls.cpp Adds missing TLS/STL includes after dependency changes
src/lib/x509/x509cert.h Removes Usage_Type definition from this header
src/lib/x509/pkix_enums.h Adds Usage_Type enum definition
src/lib/tls/tls_session_manager_stateless.h Forward declarations + out-of-line default for find_some()
src/lib/tls/tls_session_manager_stateless.cpp Adds find_some() implementation + includes cleanup
src/lib/tls/tls_session_manager_noop.h Moves trivial overrides out-of-line
src/lib/tls/tls_session_manager_noop.cpp Adds out-of-line trivial overrides + includes cleanup
src/lib/tls/tls_session_manager_hybrid.h Moves trivial override out-of-line
src/lib/tls/tls_session_manager_hybrid.cpp Adds out-of-line forwarding implementation + includes cleanup
src/lib/tls/tls_session_manager.h Replaces tls_session.h include with forward decls + tls_session_id.h
src/lib/tls/tls_session_id.h New header for session ID/ticket/handle types
src/lib/tls/tls_session.h Uses new tls_session_id.h, adjusts Session_with_Handle visibility
src/lib/tls/tls_policy.h Replaces heavy includes with tls_algos.h + forward decls
src/lib/tls/tls_policy.cpp Restores concrete includes needed by implementation
src/lib/tls/tls_messages.h Replaces heavy includes with narrower ones + forward decls
src/lib/tls/tls_extensions.h Reduces dependencies; moves certificate-type helpers out; adjusts ctor signatures
src/lib/tls/tls_extensions.cpp Moves required includes to implementation; removes moved helpers; adjusts ctor impls
src/lib/tls/tls_channel_impl.h Adjusts includes for reduced coupling
src/lib/tls/tls_channel.h Removes x509cert include and adds forward decls
src/lib/tls/tls_callbacks.h Replaces x509cert include with pkix_enums to keep Usage_Type available
src/lib/tls/tls_callbacks.cpp Adds include needed after header dependency changes
src/lib/tls/tls_algos.h Adds Certificate_Type enum + helper declarations
src/lib/tls/tls_algos.cpp Adds Certificate_Type helper implementations
src/lib/tls/tls13/tls_psk_identity_13.h Include adjustments
src/lib/tls/tls13/tls_messages_13.h Adds concrete includes + adjusts ctor signature to avoid heavy deps
src/lib/tls/tls13/msg_certificate_req_13.cpp Adds required includes; updates to new ctor signature
src/lib/tls/tls12/tls_server_impl_12.cpp Updates session ticket lifetime handling types/conversions
src/lib/tls/tls12/tls_messages_12.h Refactors some APIs/destructors; changes ticket lifetime hint type
src/lib/tls/tls12/tls_handshake_state.cpp Adds required include after dependency reduction
src/lib/tls/tls12/tls_client_impl_12.cpp Adapts to ticket lifetime hint type change
src/lib/tls/tls12/tls_channel_impl_12.h Removes tls_session.h include
src/lib/tls/tls12/msg_session_ticket_12.cpp Updates ticket lifetime hint serialization/parsing to uint32_t
src/lib/tls/tls12/msg_certificate_req_12.cpp Adds required includes; adds out-of-line destructor
src/lib/tls/tls12/msg_certificate_12.cpp Adds required includes; adds out-of-line destructor + moves count() impl
src/lib/tls/tls12/msg_cert_verify_12.cpp Adds required include after dependency reduction
src/lib/tls/sessions_sql/tls_session_manager_sql.h Adds missing include for SymmetricKey
src/lib/tls/sessions_sql/tls_session_manager_sql.cpp Adds include needed after header changes
src/lib/tls/msg_server_hello.cpp Adds concrete includes needed after dependency reduction
src/lib/tls/msg_client_hello.cpp Adds concrete includes needed after dependency reduction
src/lib/tls/info.txt Exports new header tls_session_id.h
src/lib/tls/credentials_manager.h Replaces heavy includes with forward declarations
src/lib/tls/credentials_manager.cpp Restores concrete includes needed by implementation
src/lib/tls/asio/asio_context.cpp Adds include needed after header dependency changes
src/fuzzer/tls_server.cpp Adds explicit includes to satisfy reduced transitive deps
src/fuzzer/tls_client.cpp Adds explicit includes to satisfy reduced transitive deps
src/cli/tls_helpers.h Adds explicit include needed after dependency changes
src/bogo_shim/bogo_shim.cpp Adds explicit includes needed after dependency changes

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.


private:
std::chrono::seconds m_ticket_lifetime_hint{};
uint32_t m_ticket_lifetime_hint;
Copy link

Copilot AI Feb 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

m_ticket_lifetime_hint is now a plain uint32_t without a default initializer. The constructor New_Session_Ticket_12(Handshake_IO&, Handshake_Hash&) calls io.send(*this) which will serialize the object and read m_ticket_lifetime_hint before it is set, causing undefined behavior. Initialize the member (e.g., uint32_t m_ticket_lifetime_hint{0};) and/or set it explicitly in the constructor.

Suggested change
uint32_t m_ticket_lifetime_hint;
uint32_t m_ticket_lifetime_hint{0};

Copilot uses AI. Check for mistakes.
Comment on lines +597 to +601
pending_state.new_session_ticket(std::make_unique<New_Session_Ticket_12>(
pending_state.handshake_io(),
pending_state.hash(),
handle->ticket().value(),
static_cast<uint32_t>(policy().session_ticket_lifetime().count())));
Copy link

Copilot AI Feb 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

policy().session_ticket_lifetime().count() is narrowed to uint32_t without bounds checking. If a policy returns a very large duration, this truncates and could produce an incorrect lifetime hint on the wire. Consider clamping to UINT32_MAX (and ideally rejecting negative values) before casting.

Copilot uses AI. Check for mistakes.
Comment on lines +723 to +725
const uint32_t lifetime = static_cast<uint32_t>(policy().session_ticket_lifetime().count());
pending_state.new_session_ticket(std::make_unique<New_Session_Ticket_12>(
pending_state.handshake_io(), pending_state.hash(), new_handle->ticket().value(), lifetime));
Copy link

Copilot AI Feb 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

session_ticket_lifetime().count() is cast to uint32_t here as well. To avoid truncation/undefined semantics for very large policy durations, clamp to UINT32_MAX (and handle negative durations if possible) before narrowing.

Copilot uses AI. Check for mistakes.
Comment on lines 12 to +18
#include <botan/mutex.h>
#include <botan/tls_session.h>
#include <botan/tls_session_id.h>
#include <botan/types.h>

#include <memory>
#include <optional>
#include <utility>
#include <vector>
Copy link

Copilot AI Feb 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This header now forward-declares Session_with_Handle/Session and returns std::vector<Session_with_Handle> by value (e.g. find(), find_some()). std::vector<T> requires T to be complete in translation units that construct/destroy the vector, so removing <botan/tls_session.h> is likely to break downstream builds. Consider including the header that defines Session_with_Handle (currently tls_session.h) or refactoring the API to avoid returning std::vector<Session_with_Handle> by value from this public header.

Copilot uses AI. Check for mistakes.
Comment on lines 14 to +28
#include <botan/symkey.h>
#include <botan/tls_alert.h>
#include <botan/x509cert.h>

#include <memory>
#include <optional>
#include <span>
#include <string>
#include <string_view>
#include <vector>

namespace Botan {

class Public_Key;
class X509_Certificate;

} // namespace Botan
Copy link

Copilot AI Feb 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This header now forward-declares X509_Certificate but still exposes std::vector<X509_Certificate> by value in the public API (peer_cert_chain()). std::vector<T> generally requires T to be complete in translation units that construct/destroy it, so removing <botan/x509cert.h> is likely to break downstream builds. Consider keeping the include (or changing the API to avoid std::vector<X509_Certificate> by value).

Copilot uses AI. Check for mistakes.
@randombit randombit force-pushed the jack/tls-header-patrol branch 7 times, most recently from def05ae to e37c21f Compare February 8, 2026 04:23
Copy link
Copy Markdown
Collaborator

@reneme reneme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good to me. One question regarding runtime vs. compile time efficiency.


private:
std::chrono::seconds m_ticket_lifetime_hint{};
uint32_t m_ticket_lifetime_hint = 0;
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It really stinks that <chrono> is so heavy. Ditching ::seconds (and friends) as strong types does reduce the expressiveness of the code. :( ... it's what it is, I guess.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know, it's incredibly frustrating when the basic time unit types were so useful as interface types and then the committee decided to throw in so much stuff in to the header that they are no longer viable for that purpose.

IMO C++ is failing badly at both "You don't pay for what you don't use" and "Leave no room for a lower-level language" and the worst part is from all visible signs the committee thinks it's going great.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#JustUseModules is the answer, I guess...

The way I see it, You don't pay for what you don't use" only accounts for the final binary. And there it delivers in most cases. But compile times are a significant issue if you don't happen to have a 64 core beast in your basement.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way I see it, You don't pay for what you don't use" only accounts for the final binary.

That's clearly their interpretation as well. Unfortunately that interpretation assumes developer wait time is free, which is not necessarily accurate.


private:
Certificate_Request_13(std::vector<X509_DN> acceptable_CAs, const Policy& policy, Callbacks& callbacks);
Certificate_Request_13(const std::vector<X509_DN>& acceptable_CAs, const Policy& policy, Callbacks& callbacks);
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it really hurt so much to pull in X509_DN ("pkix_types.h") here? In ::maybe_create this actually uses this lvalue to avoid a copy. Not that it is a particularly hot path in the code, but I'm wondering about the general rule-of-thumb for the trade-off between compile time and run time efficiency.

Similarly in Certificate_Authorities.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it's a matter of slow cuts. Adding pkix_types.h slows down compilation of tls_messages.h by about 8% (.71s->.76s for GCC, 1.0s->1.07s for Clang) which is not so bad. But tls_messages.h is then included into 40 or 50 TUs. So making it even slightly more expensive balloons out.

Where exactly the line is drawn is certainly open for debate but broadly

  • If something is included into many TUs it should be as light as possible. It doesn't matter if any single particular source file is slow. For example kuznyechik.o is one of the very slowest object files in the entire library to compile, due to fancy consteval tricks, and do we really need to be doing those, no - but it's just one file and not hurting anything else so who cares. Whereas eg concepts.h or loadstor.h being even 1ms faster to parse cuts 10s of seconds off the build. Also being common headers they are an ongoing tax; when another source file is added which includes them, the overall compilation cost of these headers grows further.

  • If the performance doesn't matter much (for example, if something could cost twice as much and nobody would ever notice) then might as well prefer saving compilation time, since compilation time is a scarce resource of uncertain availability [1] and that resource is shared across the entire codebase.

[1]: In that there is no fixed hard budget but slower compilation time hurts developer velocity and likely as not hurts adoption if it's bad enough to notice.

In this specific case, I'd say: many TUs are affected, it's not really performance critical (only happens during the handshake and the handshake is anyway expensive), so prefer saving compilation time. I guess if we had profiles for the TLS handshake performance [which would be a useful benchmark to have!!] and the extra copy was showing up somehow, that would be worth working on.

As an example going the other way, headers like ct_utils.h and simd_4x32.h are somewhat expensive, about .25-.35 seconds to compile, and used everywhere. But these are run in hot loops that are written expecting values like CT::Mask and SIMD_4x32 will be perfectly inlined, it's difficult to reduce the compilation time meaningfully without drastically impacting performance.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the guidelines. I concur in general terms. Though, still I feel sad to introduce runtime efficiencies like that. But from a gut feeling I also don't think this particular copy would produce measurable runtime slowdowns. On that note, we're starting to use the TLS stack in a system that might actually experience some significant load. Therefore, benchmarks for TLS might come up on my TODO list eventually...

Anyway: with all the header patrol work you've put in, I think it would be really cool to have an automated compile benchmark in CI to catch regressions early and weigh the pros and cons of new API designs early on. (For instance the new loadstor.h is convenient but I do question whether it was really necessary to push the limits that far. 😶)

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be really cool to have an automated compile benchmark in CI to catch regressions early and weigh the pros and cons of new API designs early on.

Yes I have something specifically like this in mind to include in the nightly run (and can be invoked on demand if a PR seems risky in this way) - Clang's -ftime-trace dumps a lot of useful information.

@randombit randombit force-pushed the jack/tls-header-patrol branch from e37c21f to f0c730b Compare February 9, 2026 13:30
@randombit randombit merged commit 1d119e5 into master Feb 10, 2026
46 checks passed
@randombit randombit deleted the jack/tls-header-patrol branch February 10, 2026 01:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants