Skip to content

Update go-getter client options#111

Merged
nywilken merged 5 commits intomainfrom
nywilken/update-go-getter-client
Jun 9, 2022
Merged

Update go-getter client options#111
nywilken merged 5 commits intomainfrom
nywilken/update-go-getter-client

Conversation

@nywilken
Copy link
Copy Markdown
Contributor

@nywilken nywilken commented May 25, 2022

  • Disable reading and writing files through symlinks
  • Add timeouts for GCS and S3 getters

Related to: https://github.com/hashicorp/packer-internal-issues/issues/38 (internal only)

@nywilken nywilken requested a review from a team as a code owner May 25, 2022 01:21
@nywilken nywilken marked this pull request as draft May 25, 2022 01:22
@nywilken nywilken changed the title Update go-getter client options to disable symlinks Update go-getter client options May 25, 2022
@nywilken nywilken marked this pull request as ready for review May 27, 2022 14:56
Copy link
Copy Markdown

@sylviamoss sylviamoss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏼
Since this one is a bigger change that might break existing templates, do you think we should add the changelog entry in this same PR with more information about the changes? Maybe mentioning the bump as well #110

DisableSymlinks: true,
// The order of the Getters in the list may affect the result
// depending if the Request.Src is detected as valid by multiple getters
Getters: []getter.Getter{
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!

@nywilken nywilken force-pushed the nywilken/update-go-getter-client branch from faee09d to f0617fc Compare June 2, 2022 20:17
@nywilken nywilken requested a review from picatz June 2, 2022 20:37
Comment thread multistep/commonsteps/step_download.go Outdated
Netrc: true,
XTerraformGetDisabled: true,
HeadFirstTimeout: 10 * time.Second,
ReadTimeout: 30 * time.Second,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other getter timeouts are 5 minutes, while this one is essentially 30 seconds. Maybe it'd make more sense to have a uniform timeout across the getters, or somehow make these configurable?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea of the uniform timeout. TBH 30 seconds seems really short given the use of go-getter in Packer. I struggled a bit to find good defaults. But ultimately think that going high 30 minutes is a good base line. We can make these configurable for the step so that plugins could override if needed.

That said there is an open issue for improving iso_config to support some addition go-getter Headers, which could also include setting some of these values. Taking a page out of Nomad's book here.

&getter.HgGetter{
Timeout: 5 * time.Minute,
},
new(getter.SmbClientGetter),
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 Should this getter have a timeout? I see this getter only exists in v2, and doesn't have that option currently. Just curious to your thoughts.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah - I think it should. It was not caught in the go-getter updates. I will follow up with a separate PR for that change.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR opened for the SmbClientGetter hashicorp/go-getter#369

@nywilken nywilken force-pushed the nywilken/update-go-getter-client branch from f0617fc to 900cdcf Compare June 6, 2022 16:21
@nywilken nywilken merged commit 73f8475 into main Jun 9, 2022
@nywilken nywilken deleted the nywilken/update-go-getter-client branch June 9, 2022 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants