Skip to content

The performance of nginx drops a lot when using OpenSSL 3.0 #21833

@ShuaiYuan21

Description

@ShuaiYuan21

Some parameters:

Cipher suite: TLS_AES_256_GCM_SHA384
TLS version: TLSv1.3
Kx=EC
Au=EC
ECDH_curve=prime256v1

CPS test result:

OpenSSL 1.1.1 OpenSSL 3.0.9 drop
4295.38 3132.76 27.07%

Flame Graph:

nginx with OpenSSL 1.1.1:
sw_ec_ossl1_sw

nginx with OpenSSL 3.0.9
sw_ec_ossl3_sw

Root Cause

1. EVP_PKEY_derive_set_peer_ex
image

I don't know if this part of the operation is necessary.
But it can be seen that it takes up a lot of CPU resources, so in order to eliminate its impact on performance, we can disable it in the source code.

diff --git a/crypto/evp/exchange.c b/crypto/evp/exchange.c
index d7a4ad142a..58eea73b80 100644
--- a/crypto/evp/exchange.c
+++ b/crypto/evp/exchange.c
@@ -499,7 +499,7 @@ int EVP_PKEY_derive_set_peer_ex(EVP_PKEY_CTX *ctx, EVP_PKEY *peer,

 int EVP_PKEY_derive_set_peer(EVP_PKEY_CTX *ctx, EVP_PKEY *peer)
 {
-    return EVP_PKEY_derive_set_peer_ex(ctx, peer, 1);
+    return EVP_PKEY_derive_set_peer_ex(ctx, peer, 0);
 }

Performance comparison after EVP_PKEY_derive_set_peer_ex disabled

OpenSSL 1.1.1 OpenSSL 3.0.9 drop
4295.38 3747.19 12.76%

The flame graph after EVP_PKEY_derive_set_peer_ex disabled
sw_ec_ossl3_sw_no_set_peer

Now, it looks good, and the performance improved a lot.

Below are based on the flame graph that disabled the EVP_PKEY_derive_set_peer_ex

2. do_sigver_init
This function takes up ~4% of CPU resources.
image

3. EVP_xxx_fetch
These functions take up ~6.3% of CPU resources.
In OpenSSL, these functions are similar to engine_table_select, which consumes fewer CPU cycles.
image

Most of it is a lock operation.
image

4. Some other code changes

Question

  1. If EVP_PKEY_derive_set_peer_ex is not necessary, can we disable it when config the project?
  2. Will the function do_sigver_init and EVP_xxx_fetch be optimized in the future release?

Metadata

Metadata

Assignees

No one assigned

    Labels

    branch: masterApplies to master branchtriaged: featureThe issue/pr requests/adds a featuretriaged: performanceThe issue/pr reports/fixes a performance concern

    Type

    No type

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions