Skip to content

Unset DisableKeepalive for backfill HTTP client#2137

Merged
Hayden-IO merged 2 commits intosigstore:mainfrom
cmurphy:nokeepalive
Jun 5, 2024
Merged

Unset DisableKeepalive for backfill HTTP client#2137
Hayden-IO merged 2 commits intosigstore:mainfrom
cmurphy:nokeepalive

Conversation

@cmurphy
Copy link
Copy Markdown
Contributor

@cmurphy cmurphy commented Jun 3, 2024

Disabling Keep-Alive, as done by the default transport setting in the hashicorp cleanhttp package, seems to conflict with a network setting between the public good Rekor instances and the bastion and results in GET requests stalling or timing out after processing a few entries. This change adds an option to the rekor client to unset the DisableKeepalive setting and has the backfill script utilize that option. Other uses of the rekor client will see no change.

Summary

Release Note

Documentation

@cmurphy cmurphy requested a review from a team as a code owner June 3, 2024 21:11
@cmurphy cmurphy requested a review from Hayden-IO June 3, 2024 21:11
Hayden-IO
Hayden-IO previously approved these changes Jun 3, 2024
@Hayden-IO Hayden-IO enabled auto-merge (squash) June 3, 2024 21:13
@codecov
Copy link
Copy Markdown

codecov Bot commented Jun 3, 2024

Codecov Report

Attention: Patch coverage is 0% with 5 lines in your changes missing coverage. Please review.

Project coverage is 68.61%. Comparing base (488eb97) to head (33bd958).
Report is 129 commits behind head on main.

Files Patch % Lines
pkg/client/options.go 0.00% 3 Missing ⚠️
pkg/client/rekor_client.go 0.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2137      +/-   ##
==========================================
+ Coverage   66.46%   68.61%   +2.15%     
==========================================
  Files          92       92              
  Lines        9258     7293    -1965     
==========================================
- Hits         6153     5004    -1149     
+ Misses       2359     1545     -814     
+ Partials      746      744       -2     
Flag Coverage Δ
e2etests 50.34% <0.00%> (+2.79%) ⬆️
unittests 48.91% <0.00%> (+1.23%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Disabling Keep-Alive, as done by the default transport setting in the
hashicorp cleanhttp package, seems to conflict with a network setting
between the public good Rekor instances and the bastion and results in
GET requests stalling or timing out after processing a few entries. This
change adds an option to the rekor client to unset the DisableKeepalive
setting and has the backfill script utilize that option. Other uses of
the rekor client will see no change.

Signed-off-by: Colleen Murphy <colleenmurphy@google.com>
auto-merge was automatically disabled June 3, 2024 22:07

Head branch was pushed to by a user without write access

@cmurphy
Copy link
Copy Markdown
Contributor Author

cmurphy commented Jun 3, 2024

@haydentherapper I added another change, this was still hitting a rate limit after 1000 or so requests.

Copy link
Copy Markdown
Contributor

@Hayden-IO Hayden-IO left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SG, we might want to bypass the LB for the backfill (note the playbook is out of date, there is no longer a global rate limit, so you aren't affecting other clients)

Comment thread pkg/client/options.go
RetryCount uint
InsecureTLS bool
Logger interface{}
NoDisableKeepalives bool
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be EnableKeepalives?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The field we're un-setting is DisableKeepalives. EnableKeepalives isn't really correct because it's not enabling Keepalives. The double negative seems more intuitive to me.

Increase the RetryCount setting from the default of 3 up to 10 in order
to avoid giving up too quickly when the script hits the server rate
limit. The retryablehttp client does an exponential backoff, so
increasing the number of tries also increases the amount of time it will
wait in between each try before it eventually succeeds.

Signed-off-by: Colleen Murphy <colleenmurphy@google.com>
@Hayden-IO Hayden-IO merged commit d43e712 into sigstore:main Jun 5, 2024
@github-actions github-actions Bot added this to the v1.2.2 milestone Jun 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants