Skip to content

Fix modulewrapper memory leak#3094

Merged
justsmth merged 1 commit intoaws:mainfrom
justsmth:fix-modulewrapper-leak
Mar 12, 2026
Merged

Fix modulewrapper memory leak#3094
justsmth merged 1 commit intoaws:mainfrom
justsmth:fix-modulewrapper-leak

Conversation

@justsmth
Copy link
Copy Markdown
Contributor

@justsmth justsmth commented Mar 12, 2026

Description of changes:

The RSADecryptionPrimitiveCRT function in the ACVP modulewrapper leaks 8 BIGNUMs on every call. RSA_new_private_key_large_e takes const BIGNUM* parameters and copies them internally, but the originals were never freed. RSASignaturePrimitive and RSADecryptionPrimitive had the same issue on their error paths (before RSA_set0_key takes ownership).

Under ASAN, LeakSanitizer reports these leaks to stderr when the modulewrapper exits. Since check_expected.go captures combined stdout+stderr, the leak output gets mixed into the ACVP test result, causing a mismatch against the expected output.

All three functions now use bssl::UniquePtr<BIGNUM> for automatic cleanup.

Testing:

Existing ACVP tests in the FIPS ASAN CI job should now pass.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and the ISC license.

@codecov-commenter
Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.15%. Comparing base (9124e2b) to head (e697aa2).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3094      +/-   ##
==========================================
- Coverage   78.16%   78.15%   -0.02%     
==========================================
  Files         689      689              
  Lines      121628   121627       -1     
  Branches    16982    16982              
==========================================
- Hits        95074    95059      -15     
- Misses      25670    25684      +14     
  Partials      884      884              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@justsmth justsmth marked this pull request as ready for review March 12, 2026 16:05
@justsmth justsmth requested a review from a team as a code owner March 12, 2026 16:05
@justsmth justsmth changed the title [DRAFT] Fix modulewrapper memory leak Fix modulewrapper memory leak Mar 12, 2026
Copy link
Copy Markdown
Contributor

@sgmenda sgmenda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@justsmth justsmth merged commit c21d40d into aws:main Mar 12, 2026
736 of 901 checks passed
@justsmth justsmth deleted the fix-modulewrapper-leak branch March 12, 2026 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants