Skip to content

Conversation

@cpu
Copy link
Member

@cpu cpu commented Oct 30, 2024

This change aligns the Rustls default for a ServerConfig::builder()'s send_tls13_tickets value to match BoringSSL/OpenSSL's default of 2.

Previously we used 4 but this prefers a more limited use-case where the client wants to be able to resume 4 handshakes in parallel in its initial post-full-handshake flight of connections. The overhead of optimizing for this scenario vs using a smaller number of tickets by default also manifests as a performance cost in server-side resumption: the extra 2 tickets cost us 8%/6% (client/server) of a normal resumption, or ~0%/2% of a full handshake. This also manifests when comparing default configurations between TLS libraries due to the high cost of creating new tickets:

We believe a default of 2 strikes a better balance in general - recall each resumed handshake will also replenish the client's tickets. It also avoids folks repeatedly tripping into this during benchmarking (e.g. #1751 (comment), #2167), and aligns us with other parts of the TLS ecosystem on a sensible default. Anyone that has a workload that prefers the old behaviour can adjust the value to 4 or higher manually but we expect this to be niche and not the norm.

@cpu cpu self-assigned this Oct 30, 2024
Copy link
Member

@ctz ctz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also remove the now-unnecessary setting of this in rustls/examples/internal/bench_impl.rs?

@rustls-benchmarking
Copy link

rustls-benchmarking bot commented Oct 30, 2024

Benchmark results

Instruction counts

Significant differences

⚠️ There are significant instruction count differences

Click to expand
Scenario Baseline Candidate Diff Threshold
handshake_tickets_aws_lc_rs_1.3_rsa_chacha_client 30621346 28197729 -2423617 (-7.91%) 0.20%
handshake_session_id_aws_lc_rs_1.3_rsa_chacha_client 30182546 27803966 -2378580 (-7.88%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_chacha_client 30599251 28190719 -2408532 (-7.87%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_chacha_client 30599113 28193672 -2405441 (-7.86%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_chacha_client 30168558 27796986 -2371572 (-7.86%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_chacha_client 30167867 27799517 -2368350 (-7.85%) 0.20%
handshake_session_id_aws_lc_rs_1.3_rsa_aes_client 30218781 27864262 -2354519 (-7.79%) 0.20%
handshake_tickets_aws_lc_rs_1.3_rsa_aes_client 30607895 28228644 -2379251 (-7.77%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_aes_client 30204584 27857416 -2347168 (-7.77%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_aes_client 30204038 27859980 -2344058 (-7.76%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_aes_client 30585610 28221503 -2364107 (-7.73%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_aes_client 30585309 28224358 -2360951 (-7.72%) 0.20%
handshake_tickets_aws_lc_rs_1.3_rsa_chacha_server 32425242 30366907 -2058335 (-6.35%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_chacha_server 32428430 30370183 -2058247 (-6.35%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_chacha_server 32428705 30370624 -2058081 (-6.35%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_aes_server 32431586 30413698 -2017888 (-6.22%) 0.20%
handshake_tickets_aws_lc_rs_1.3_rsa_aes_server 32428160 30410599 -2017561 (-6.22%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_aes_server 32431468 30413833 -2017635 (-6.22%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_chacha_server 30025472 28885373 -1140099 (-3.80%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_chacha_server 30025200 28885432 -1139768 (-3.80%) 0.20%
handshake_session_id_aws_lc_rs_1.3_rsa_chacha_server 30022328 28882723 -1139605 (-3.80%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_aes_server 30075729 28964611 -1111118 (-3.69%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_aes_server 30075517 28964634 -1110883 (-3.69%) 0.20%
handshake_session_id_aws_lc_rs_1.3_rsa_aes_server 30072747 28961979 -1110768 (-3.69%) 0.20%
handshake_tickets_ring_1.3_rsa_chacha_client 42053528 40519430 -1534098 (-3.65%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_chacha_client 42039583 40512177 -1527406 (-3.63%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_chacha_client 42039061 40515436 -1523625 (-3.62%) 0.20%
handshake_tickets_ring_1.3_rsa_aes_client 42095288 40593680 -1501608 (-3.57%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_aes_client 42081361 40586435 -1494926 (-3.55%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_aes_client 42080821 40589686 -1491135 (-3.54%) 0.20%
handshake_session_id_ring_1.3_rsa_chacha_client 41585052 40201653 -1383399 (-3.33%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_chacha_client 41571316 40194667 -1376649 (-3.31%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_chacha_client 41570826 40197608 -1373218 (-3.30%) 0.20%
handshake_session_id_ring_1.3_rsa_aes_client 41654742 40296033 -1358709 (-3.26%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_aes_client 41641024 40289055 -1351969 (-3.25%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_aes_client 41640516 40291988 -1348528 (-3.24%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_chacha_server 43321985 41964559 -1357426 (-3.13%) 0.20%
handshake_tickets_ring_1.3_rsa_chacha_server 43318866 41961608 -1357258 (-3.13%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_chacha_server 43321981 41964668 -1357313 (-3.13%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_aes_server 43381775 42062869 -1318906 (-3.04%) 0.20%
handshake_tickets_ring_1.3_rsa_aes_server 43378656 42059918 -1318738 (-3.04%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_aes_server 43381801 42063008 -1318793 (-3.04%) 0.20%
handshake_session_id_ring_1.3_rsa_chacha_server 42557498 41479898 -1077600 (-2.53%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_chacha_server 42560106 41482491 -1077615 (-2.53%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_chacha_server 42560117 41482569 -1077548 (-2.53%) 0.20%
handshake_session_id_ring_1.3_rsa_aes_server 42646223 41598128 -1048095 (-2.46%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_aes_server 42648831 41600721 -1048110 (-2.46%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_aes_server 42648797 41600799 -1047998 (-2.46%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_server 1211012 1181714 -29298 (-2.42%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_aes_server 1208642 1179467 -29175 (-2.41%) 0.20%
handshake_no_resume_ring_1.3_ecdsap256_aes_server 1644133 1612563 -31570 (-1.92%) 1.05%
handshake_no_resume_ring_1.3_ecdsap256_chacha_server 1641198 1613964 -27234 (-1.66%) 1.05%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_server 2108902 2079520 -29382 (-1.39%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_aes_server 2105692 2076602 -29090 (-1.38%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_aes_server 7594234 7567032 -27202 (-0.36%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_chacha_server 7596412 7569220 -27192 (-0.36%) 0.20%
handshake_no_resume_ring_1.3_rsa_aes_server 11452529 11425323 -27206 (-0.24%) 0.20%
handshake_no_resume_ring_1.3_rsa_chacha_server 11458509 11431303 -27206 (-0.24%) 0.20%

Other differences

Click to expand
Scenario Baseline Candidate Diff Threshold
handshake_no_resume_aws_lc_rs_1.3_rsa_chacha_server 10747990 10675931 -72059 (-0.67%) 1.10%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_client 8568840 8606168 37328 (0.44%) 0.57%
handshake_no_resume_ring_1.3_ecdsap256_aes_client 3627784 3623595 -4189 (-0.12%) 0.46%
handshake_no_resume_aws_lc_rs_1.3_rsa_aes_server 10723601 10715367 -8234 (-0.08%) 0.98%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_aes_client 3084876 3082516 -2360 (-0.08%) 0.22%
handshake_no_resume_ring_1.3_ecdsap256_chacha_client 3625034 3622882 -2152 (-0.06%) 0.48%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_client 3087626 3086982 -644 (-0.02%) 0.28%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_aes_client 8599650 8598311 -1339 (-0.02%) 1.03%
handshake_no_resume_aws_lc_rs_1.2_rsa_aes_server 10471515 10471042 -473 (-0.00%) 1.39%
handshake_no_resume_aws_lc_rs_1.2_rsa_aes_client 1717461 1717476 15 (0.00%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_rsa_chacha_client 1932693 1932686 -7 (-0.00%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_chacha_client 35185133 35185160 27 (0.00%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_aes_client 35183203 35183220 17 (0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_aes_server 46470690 46470681 -9 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_aes_server 46467124 46467116 -8 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_aes_server 46480095 46480087 -8 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_aes_client 58348757 58348748 -9 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.2_rsa_aes_server 46434715 46434708 -7 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_rsa_chacha_server 80554934 80554946 12 (0.00%) 0.20%
transfer_no_resume_ring_1.2_rsa_aes_server 46389591 46389586 -5 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.2_rsa_aes_client 68684448 68684441 -7 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_client 92712624 92712615 -9 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_aes_server 46467806 46467802 -4 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_rsa_chacha_client 92684664 92684671 7 (0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_chacha_client 92682836 92682830 -6 (-0.00%) 0.20%
transfer_no_resume_ring_1.2_rsa_aes_client 58237088 58237091 3 (0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_chacha_server 80540450 80540446 -4 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_server 80641256 80641259 3 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_server 80648884 80648887 3 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_aes_client 58251824 58251826 2 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_aes_client 58256129 58256131 2 (0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_aes_client 58341604 58341602 -2 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_client 92714856 92714858 2 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_aes_server 46460144 46460143 -1 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_aes_client 58254052 58254051 -1 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_chacha_server 80535649 80535648 -1 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_chacha_server 80661851 80661852 1 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_chacha_client 92718859 92718860 1 (0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.2_rsa_aes_client 3873535 3873535 0 (0.00%) 0.20%
handshake_no_resume_ring_1.3_rsa_chacha_client 2662161 2662161 0 (0.00%) 0.20%
handshake_no_resume_ring_1.2_rsa_aes_client 2563544 2563544 0 (0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.2_rsa_aes_client 4215889 4215889 0 (0.00%) 0.20%
handshake_session_id_ring_1.2_rsa_aes_server 4251919 4251919 0 (0.00%) 0.20%
handshake_tickets_ring_1.2_rsa_aes_server 4694983 4694983 0 (0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.2_rsa_aes_server 3892011 3892011 0 (0.00%) 0.20%
transfer_no_resume_ring_1.3_rsa_aes_server 46487101 46487101 0 (0.00%) 0.20%
handshake_no_resume_ring_1.2_rsa_aes_server 11292338 11292338 0 (0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.2_rsa_aes_server 5014085 5014085 0 (0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_chacha_client 92673767 92673767 0 (0.00%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_rsa_aes_client 1925972 1925972 0 (0.00%) 0.20%
transfer_no_resume_ring_1.3_rsa_aes_client 58352502 58352502 0 (0.00%) 0.20%
handshake_tickets_ring_1.2_rsa_aes_client 4497663 4497663 0 (0.00%) 0.20%
handshake_no_resume_ring_1.3_rsa_aes_client 2656296 2656296 0 (0.00%) 0.20%
handshake_session_id_ring_1.2_rsa_aes_client 4236879 4236879 0 (0.00%) 0.20%

Wall-time

Significant differences

⚠️ There are significant wall-time differences

Click to expand
Scenario Baseline Candidate Diff Threshold
handshake_tickets_aws_lc_rs_1.3_ecdsap256_chacha 4.79 ms 4.47 ms ✅ -0.32 ms (-6.67%) 1.37%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_aes 4.80 ms 4.49 ms ✅ -0.31 ms (-6.45%) 1.58%
handshake_tickets_aws_lc_rs_1.3_rsa_chacha 5.48 ms 5.16 ms ✅ -0.32 ms (-5.80%) 1.46%
handshake_tickets_aws_lc_rs_1.3_rsa_aes 5.48 ms 5.16 ms ✅ -0.32 ms (-5.79%) 1.61%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_chacha 5.50 ms 5.18 ms ✅ -0.32 ms (-5.77%) 1.46%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_aes 5.53 ms 5.21 ms ✅ -0.31 ms (-5.69%) 1.30%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_aes 4.48 ms 4.31 ms ✅ -0.18 ms (-3.94%) 1.92%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_chacha 4.45 ms 4.28 ms ✅ -0.17 ms (-3.85%) 1.32%
handshake_session_id_aws_lc_rs_1.3_rsa_chacha 5.14 ms 4.96 ms ✅ -0.18 ms (-3.59%) 1.41%
handshake_session_id_aws_lc_rs_1.3_rsa_aes 5.16 ms 4.98 ms ✅ -0.18 ms (-3.45%) 1.84%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_chacha 5.16 ms 4.98 ms ✅ -0.18 ms (-3.43%) 1.21%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_aes 5.20 ms 5.02 ms ✅ -0.17 ms (-3.33%) 1.56%
handshake_tickets_ring_1.3_ecdsap256_chacha 6.59 ms 6.40 ms ✅ -0.19 ms (-2.93%) 1.00%
handshake_tickets_ring_1.3_ecdsap256_aes 6.63 ms 6.44 ms ✅ -0.18 ms (-2.76%) 1.00%
handshake_tickets_ring_1.3_rsa_chacha 7.08 ms 6.88 ms ✅ -0.19 ms (-2.74%) 1.00%
handshake_tickets_ring_1.3_rsa_aes 7.12 ms 6.93 ms ✅ -0.19 ms (-2.65%) 1.00%
handshake_session_id_ring_1.3_ecdsap256_chacha 6.48 ms 6.33 ms ✅ -0.16 ms (-2.41%) 1.00%
handshake_session_id_ring_1.3_ecdsap256_aes 6.52 ms 6.37 ms ✅ -0.14 ms (-2.17%) 1.00%
handshake_session_id_ring_1.3_rsa_chacha 6.97 ms 6.82 ms ✅ -0.15 ms (-2.17%) 1.00%
handshake_session_id_ring_1.3_rsa_aes 7.01 ms 6.87 ms ✅ -0.14 ms (-2.03%) 1.00%
handshake_tickets_ring_1.3_ecdsap384_chacha 9.67 ms 9.48 ms ✅ -0.19 ms (-2.00%) 1.00%
handshake_tickets_ring_1.3_ecdsap384_aes 9.71 ms 9.52 ms ✅ -0.19 ms (-1.93%) 1.00%
handshake_session_id_ring_1.3_ecdsap384_chacha 9.57 ms 9.41 ms ✅ -0.15 ms (-1.62%) 1.00%
handshake_session_id_ring_1.3_ecdsap384_aes 9.60 ms 9.46 ms ✅ -0.15 ms (-1.51%) 1.00%

Other differences

Click to expand
Scenario Baseline Candidate Diff Threshold
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_chacha 455.32 µs 453.49 µs -1.83 µs (-0.40%) 3.16%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_aes 457.06 µs 455.30 µs -1.76 µs (-0.38%) 2.53%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_aes 5.17 ms 5.15 ms -0.02 ms (-0.37%) 3.77%
handshake_no_resume_ring_1.3_ecdsap256_chacha 499.63 µs 497.84 µs -1.79 µs (-0.36%) 2.38%
transfer_no_resume_aws_lc_rs_1.2_rsa_aes 5.14 ms 5.12 ms -0.01 ms (-0.29%) 3.27%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_aes 4.44 ms 4.43 ms -0.01 ms (-0.25%) 4.44%
transfer_no_resume_ring_1.3_ecdsap256_aes 6.34 ms 6.32 ms -0.02 ms (-0.25%) 3.27%
transfer_no_resume_aws_lc_rs_1.3_rsa_aes 5.11 ms 5.09 ms -0.01 ms (-0.24%) 4.06%
handshake_no_resume_ring_1.3_ecdsap256_aes 502.33 µs 501.26 µs -1.07 µs (-0.21%) 2.73%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_chacha 1.18 ms 1.17 ms -0.00 ms (-0.20%) 1.14%
handshake_no_resume_aws_lc_rs_1.3_rsa_aes 1.12 ms 1.12 ms -0.00 ms (-0.20%) 6.96%
handshake_session_id_aws_lc_rs_1.2_rsa_aes 1.64 ms 1.64 ms 0.00 ms (0.18%) 4.13%
transfer_no_resume_ring_1.3_ecdsap256_chacha 12.95 ms 12.93 ms -0.02 ms (-0.16%) 1.46%
handshake_no_resume_ring_1.3_rsa_chacha 995.00 µs 993.45 µs -1.54 µs (-0.16%) 1.11%
transfer_no_resume_ring_1.3_ecdsap384_aes 9.43 ms 9.42 ms -0.01 ms (-0.15%) 1.96%
handshake_tickets_aws_lc_rs_1.2_rsa_aes 1.81 ms 1.81 ms -0.00 ms (-0.15%) 5.36%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_aes 1.17 ms 1.17 ms -0.00 ms (-0.14%) 1.18%
transfer_no_resume_ring_1.3_rsa_aes 6.83 ms 6.82 ms -0.01 ms (-0.13%) 3.17%
transfer_no_resume_ring_1.2_rsa_aes 6.75 ms 6.75 ms -0.01 ms (-0.11%) 2.53%
transfer_no_resume_ring_1.3_rsa_chacha 13.44 ms 13.43 ms -0.01 ms (-0.10%) 1.41%
handshake_no_resume_ring_1.3_rsa_aes 993.63 µs 992.66 µs -0.97 µs (-0.10%) 1.07%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_chacha 13.63 ms 13.62 ms -0.01 ms (-0.09%) 1.42%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_chacha 12.91 ms 12.90 ms -0.01 ms (-0.08%) 1.76%
handshake_no_resume_aws_lc_rs_1.2_rsa_aes 1.08 ms 1.08 ms 0.00 ms (0.08%) 7.00%
handshake_no_resume_ring_1.3_ecdsap384_aes 3.60 ms 3.59 ms -0.00 ms (-0.07%) 1.00%
transfer_no_resume_ring_1.3_ecdsap384_chacha 16.04 ms 16.03 ms -0.01 ms (-0.07%) 1.23%
handshake_no_resume_ring_1.3_ecdsap384_chacha 3.60 ms 3.59 ms -0.00 ms (-0.07%) 1.00%
handshake_session_id_ring_1.2_rsa_aes 1.52 ms 1.52 ms 0.00 ms (0.07%) 1.00%
transfer_no_resume_aws_lc_rs_1.3_rsa_chacha 13.60 ms 13.59 ms -0.01 ms (-0.06%) 1.50%
handshake_tickets_ring_1.2_rsa_aes 1.60 ms 1.60 ms -0.00 ms (-0.03%) 1.16%
handshake_no_resume_ring_1.2_rsa_aes 989.82 µs 989.57 µs -0.24 µs (-0.02%) 1.00%
handshake_no_resume_aws_lc_rs_1.3_rsa_chacha 1.12 ms 1.12 ms -0.00 ms (-0.02%) 8.34%

Additional information

Historical results

Checkout details:

This change aligns the Rustls default for a `ServerConfig::builder()`'s
`send_tls13_tickets` value to match BoringSSL/OpenSSL's default of 2.

Previously we used 4 but this manifests as a performance gap in
server-side resumption cost when comparing default configurations
between TLS libraries due to the high cost of creating new tickets.

We believe a default of 2 strikes a better balance in general, and
avoids folks repeatedly tripping into false assumptions during
benchmarking. Anyone that has a workload that prefers up-front
server-side cost in favour of reduced client-side latency for batched
connections can tweak the value to 4 or higher manually.
@cpu cpu force-pushed the cpu-drop-default-tickets branch from bdff699 to cf7954a Compare October 30, 2024 16:15
@ctz ctz added the performance_enhancement Pull requests that should improve performance label Oct 30, 2024
@codecov
Copy link

codecov bot commented Oct 30, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.65%. Comparing base (8256c61) to head (cf7954a).
Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2187      +/-   ##
==========================================
- Coverage   94.65%   94.65%   -0.01%     
==========================================
  Files         102      102              
  Lines       23749    23749              
==========================================
- Hits        22480    22479       -1     
- Misses       1269     1270       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@cpu cpu requested a review from djc October 30, 2024 17:13
@djc
Copy link
Member

djc commented Oct 30, 2024

This change aligns the Rustls default for a ServerConfig::builder()'s send_tls13_tickets value to match BoringSSL/OpenSSL's default of 2.

Previously we used 4 but this manifests as a performance gap in server-side resumption cost when comparing default configurations between TLS libraries due to the high cost of creating new tickets.

We believe a default of 2 strikes a better balance in general, and avoids folks repeatedly tripping into this during benchmarking (e.g. #1751 (comment), #2167). Anyone that has a workload that prefers up-front server-side cost in favour of reduced client-side latency for batched connections can tweak the value to 4 or higher manually.

I don't disagree but I want to challenge this. I understand that, all other things being equal, it would be better to match others so that people benchmarking will be less likely to confuse themselves, but I also believe it is more important that we deliver good software to downstream (server) users than that we don't confuse benchmark readers.

As such, doesn't this make rustls worse at resuming sessions in practice? Do we send new tickets in resumed sessions? Why did we pick 4 in the first place? Preferring upfront server-side cost (how much cost are we talking about in practice?) in favor of reduced client-side latency seems like a pretty good trade-off for lots of people... I don't feel like the trade-off is all that crisply articulated in this PR's description.

@ctz
Copy link
Member

ctz commented Oct 31, 2024

As such, doesn't this make rustls worse at resuming sessions in practice?

Yes, but I think in quite limited circumstances; see below.

Do we send new tickets in resumed sessions?

Yes, send_tls13_tickets number of tickets are sent irrespective of the kind of handshake1.

Why did we pick 4 in the first place?

For reference, that commit was d780790 though I didn't record why I specifically chose 4 the default changed from 1 to 4, it was in the context of making TLS1.3 tickets non-reusable.

I think I chose the number 4 on the basis of a typical limit of concurrent connections made by a HTTP1 client to a given host (the actual number varies wildly by browser, with RFC2616 specifying 2 but (eg) Chrome doing 6. However that measure is not relevant for HTTP2, nor HTTP3.

Preferring upfront server-side cost (how much cost are we talking about in practice?)

Seems the benchmarks here are saying the extra 2 tickets cost us 8%/6% (client/server) of a normal resumption, or ~0%/2% of a full handshake.

in favor of reduced client-side latency seems like a pretty good trade-off for lots of people... I don't feel like the trade-off is all that crisply articulated in this PR's description.

The case where 2 is worse than 4 is pretty specific, I think. For the sake of argument, say the number is N:

  1. The client starts with no tickets. It must do a full handshake. It now has N tickets.
  2. the client may now start up to N resumed handshakes in parallel. Once those complete, it has up to N*N tickets (N from each of N handshakes).
  3. the client may now start up to N*N resumed handshakes in parallel. Once those complete, it has N*N*N 2 tickets.

To reiterate, this only limits the number of resumed handshakes a client can make in parallel at any given point. In serial is fine: each time a handshake completes, it replenishes the client's cache by N (having consumed one ticket).

Footnotes

  1. Compare OpenSSL (but not BoringSSL) that sends only one ticket after a resumed handshake, but SSL_CTX_set_num_tickets (defaults to 2) for a full handshake.

  2. Somewhat unrelated, but rustls on the client limits this growth to 8 per host, to avoid this sort of growth pushing items for other hosts out of the session cache.

Copy link
Member

@djc djc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I agree that this makes sense then!

@ctz ctz added this pull request to the merge queue Oct 31, 2024
Merged via the queue into rustls:main with commit 5199db3 Oct 31, 2024
@cpu cpu deleted the cpu-drop-default-tickets branch October 31, 2024 12:21
@cpu
Copy link
Member Author

cpu commented Oct 31, 2024

@ctz Thanks for adding that extra analysis.

@djc Thank you for pushing back 👍 My PR description wasn't doing this change justice. I've lifted some of Ctz's thoughts into the PR description, and tried to reframe it in terms of being a better default in general. The focus shouldn't have been on the benchmarking foibles.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance_enhancement Pull requests that should improve performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants