Skip to content

EaR - Misc fixes found using end-to-end integration testing (#9806) #9859

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
sfc-gh-ahusain merged 1 commit into from
Mar 31, 2023
Merged

EaR - Misc fixes found using end-to-end integration testing (#9806) #9859

sfc-gh-ahusain merged 1 commit into from
Mar 31, 2023

Conversation

@sfc-gh-ahusain
Copy link
Collaborator

@sfc-gh-ahusain sfc-gh-ahusain commented Mar 31, 2023

  • EaR - Misc fixes found using end-to-end integration testing

Description

Major changes proposed includes:

  1. RESTClient filtering of trailing /(s) characters from input URI resource path
  2. Avoid EKP exponential backup given RESTClient supports exponential backoffs retries for all retryable errors.
  3. Memory allocation optimizations: 3.1. BaseCipher key management using Standalone semantics in KMSConnector interface endpoints 3.2. Optimize memcpy while looking encryption-keys in EKP endpoints
  4. Avoid delay while starting EKP, given its criticality during cluster recovery.
  5. Update BlobCipher to handle variable size BaseCipher buffer
  6. Improved logging

Testing

Setup:

  1. External KMS server to supply encryption keys (inhouse)
  2. Create cluster with: cluster_aware & domain_aware config
  • Fix EncryptionOps test

Description

Testing

  • EaR - Misc fixes found using end-to-end integration testing

Description

Major changes:

  1. Cleanup EKP driven exponential backup files.
  2. Update EKP not to use Reformat the go and python bindings #1.

Testing

  • EaR - Misc fixes found using end-to-end integration testing

Description

Address review comments

Testing

  • Fix AES 256 key length value

Description

Testing

  • Address review comments

Description

Testing

(cherry picked from commit 3f6fcad) (#9806)

Replace this text with your description here...

Code-Reviewer Section

The general pull request guidelines can be found here.

Please check each of the following things and check all boxes before accepting a PR.

  • The PR has a description, explaining both the problem and the solution.
  • The description mentions which forms of testing were done and the testing seems reasonable.
  • Every function/class/actor that was touched is reasonably well documented.

For Release-Branches

If this PR is made against a release-branch, please also check the following:

  • This change/bugfix is a cherry-pick from the next younger branch (younger release-branch or main if this is the youngest branch)
  • There is a good reason why this PR needs to go into a release branch and this reason is documented (either in the description above or in a linked GitHub issue)

@sfc-gh-ahusain
EaR - Misc fixes found using end-to-end integration testing (apple#9806)
6524348 
* EaR - Misc fixes found using end-to-end integration testing

Description

Major changes proposed includes:
1. RESTClient filtering of trailing `/`(s) characters from
input URI resource path
2. Avoid EKP exponential backup given RESTClient supports
exponential backoffs retries for all retryable errors.
3. Memory allocation optimizations:
 3.1. BaseCipher key management using Standalone semantics
 in KMSConnector interface endpoints
 3.2. Optimize memcpy while looking encryption-keys in EKP endpoints
4. Avoid delay while starting EKP, given its criticality during
cluster recovery.
5. Update BlobCipher to handle variable size BaseCipher buffer
6. Improved logging

Testing

Setup:
1. External KMS server to supply encryption keys (inhouse)
2. Create cluster with: cluster_aware & domain_aware config

* Fix EncryptionOps test

Description

Testing

* EaR - Misc fixes found using end-to-end integration testing

Description

Major changes:
1. Cleanup EKP driven exponential backup files.
2. Update EKP not to use #1.

Testing

* EaR - Misc fixes found using end-to-end integration testing

Description

Address review comments

Testing

* Fix AES 256 key length value

Description

Testing

* Address review comments

Description

Testing

(cherry picked from commit 3f6fcad)
(apple#9806)
@fdb-windows-ci
Copy link
Collaborator

fdb-windows-ci commented Mar 31, 2023

Doxense CI Report for Windows 10

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Mar 31, 2023

Result of foundationdb-pr-clang on Linux CentOS 7

  • Commit ID: 6524348
  • Duration 0:38:14
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Mar 31, 2023

Result of foundationdb-pr on Linux CentOS 7

  • Commit ID: 6524348
  • Duration 0:50:53
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Contributor

foundationdb-ci commented Mar 31, 2023

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

  • Commit ID: 6524348
  • Duration 2:03:37
  • Result: ❌ FAILED
  • Error: Error while executing command: if $fail_test; then exit 1; fi. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)
  • Cluster Test Logs zip file of the test logs (available for 30 days)

@sfc-gh-ahusain sfc-gh-ahusain merged commit 94cac5b into apple:snowflake/release-71.3 Mar 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sfc-gh-nwijetunga sfc-gh-nwijetunga sfc-gh-nwijetunga approved these changes

@sfc-gh-jslocum sfc-gh-jslocum sfc-gh-jslocum approved these changes

@sfc-gh-akejriwal sfc-gh-akejriwal Awaiting requested review from sfc-gh-akejriwal

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@sfc-gh-ahusain @fdb-windows-ci @foundationdb-ci @sfc-gh-nwijetunga @sfc-gh-jslocum