Add storage class for aws, gcp, and azure#456
Merged
Conversation
Contributor
|
CI appears to be failing on Azure |
Contributor
|
I think this may also tie into #116 as a mechanism to change the storage tier of an object |
Contributor
Author
|
@tustvold Sorry for the delay following up on this. Ive updated azure and the http client. |
mbrobbel
approved these changes
Sep 5, 2025
Contributor
|
@tustvold are you happy with merging this? |
tustvold
approved these changes
Sep 13, 2025
tustvold
left a comment
Contributor
There was a problem hiding this comment.
Looks fine to me, although I haven't tested it
alamb
reviewed
Sep 13, 2025
| /// [Azure](https://learn.microsoft.com/en-us/rest/api/storageservices/set-blob-tier). | ||
| /// `StorageClass` is used as the name for this attribute because 2 of the 3 storage providers | ||
| /// use that name | ||
| StorageClass, |
Contributor
There was a problem hiding this comment.
I verified that adding a new enum variant is not a breaking API change as this enum is marked #[non_exhaustive] ✅
Contributor
|
Thanks everyone! |
Simon-3008-Simon
added a commit
to sap-contributions/arrow-rs-object-store
that referenced
this pull request
Oct 14, 2025
* Improve documentation for http client timeout (apache#390) * chore: fix some clippy 1.89 warnings and ignore some doctests on wasm32 (apache#468) * chore: fix some clippy 1.89 warnings * fix another warning * Skip some doctests for wasm32 * build(deps): bump actions/checkout from 4 to 5 (apache#463) Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 5. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v4...v5) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '5' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Allow "application_credentials" in `impl FromStr for GoogleConfigKey` (apache#467) * build(deps): bump actions/setup-python from 5 to 6 (apache#476) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5 to 6. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v5...v6) --- updated-dependencies: - dependency-name: actions/setup-python dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump actions/setup-node from 4 to 5 (apache#477) Bumps [actions/setup-node](https://github.com/actions/setup-node) from 4 to 5. - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@v4...v5) --- updated-dependencies: - dependency-name: actions/setup-node dependency-version: '5' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump actions/github-script from 7 to 8 (apache#478) Bumps [actions/github-script](https://github.com/actions/github-script) from 7 to 8. - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@v7...v8) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: '8' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(client/retry): include error info in logs when retry occurs (apache#487) On a request retry, it logs an info message stating that an error was encountered and information about the retry process but it hasn't included any details about the error that is causing the retry. This PR updates the logging to include the status if it is a server error and the http error kind if a transport error occurred. While the last error when retries are exhausted is returned up the call stack, the intermediate errors need not be exactly the same. It is helpful to include some minimum information about what error triggered a retry each time it happens. * aws: downgrade credential provider info! log messages to debug! (apache#436) These log messages are very noisy. * Add storage class for aws, gcp, and azure (apache#456) * Add storage class for aws and gcp * Add azure storage class attribute * Update attribute docs * Update http client * Add version 0.12.4 release plan to README (apache#490) * Fix for clippy 1.90 (apache#492) * AWS S3: Support STS endpoint, WebIdentity, RoleArn, RoleSession configuration (apache#480) * Allow setting STS endpoint via env var * Properly use AmazonS3Builder::credentials_from_env for AssumeRoleWithWebIdentity auth flow --------- Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org> * Revert "refactor: remove AWS dynamo integration (apache#407)" (apache#493) This reverts commit 034733f. * Update version to 0.12.4 and add changelog (apache#491) * Update version to 0.12.4 * Update update_changelog.sh script * Update changelog * Last touchups * Update changelog * Reapply "refactor: remove AWS dynamo integration (apache#407)" (apache#494) This reverts commit bebd53b. * Add Content_length header to S3 create_multipart (apache#496) * minor: Fix MSRV CI workflow (apache#502) * ci: Fix MSRV CI workflow * Update .github/workflows/rust.yml * Update .github/workflows/rust.yml --------- Co-authored-by: Matthijs Brobbel <m1brobbel@gmail.com> * All changes to support SAP HDLFS * add hdlfs * FOS-9911: support delta path version 2.0 * FOS-10349: fix object_url (#2) * FOS-10349: fix object_url * FOS-11081: sync code --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org> Co-authored-by: Matthijs Brobbel <m1brobbel@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kyle Barron <kyle@developmentseed.org> Co-authored-by: Phil Bracikowski <13472206+philjb@users.noreply.github.com> Co-authored-by: Alfonso Subiotto Marqués <alfonso.subiotto@polarsignals.com> Co-authored-by: Matthew Turner <me@matthewmturner.com> Co-authored-by: Matt Friede <7852262+Friede80@users.noreply.github.com> Co-authored-by: Yongming Ding <yongmingd@nvidia.com>
Simon-3008-Simon
added a commit
to sap-contributions/arrow-rs-object-store
that referenced
this pull request
Feb 6, 2026
* Improve documentation for http client timeout (apache#390) * chore: fix some clippy 1.89 warnings and ignore some doctests on wasm32 (apache#468) * chore: fix some clippy 1.89 warnings * fix another warning * Skip some doctests for wasm32 * build(deps): bump actions/checkout from 4 to 5 (apache#463) Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 5. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v4...v5) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '5' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Allow "application_credentials" in `impl FromStr for GoogleConfigKey` (apache#467) * build(deps): bump actions/setup-python from 5 to 6 (apache#476) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5 to 6. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v5...v6) --- updated-dependencies: - dependency-name: actions/setup-python dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump actions/setup-node from 4 to 5 (apache#477) Bumps [actions/setup-node](https://github.com/actions/setup-node) from 4 to 5. - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@v4...v5) --- updated-dependencies: - dependency-name: actions/setup-node dependency-version: '5' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump actions/github-script from 7 to 8 (apache#478) Bumps [actions/github-script](https://github.com/actions/github-script) from 7 to 8. - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@v7...v8) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: '8' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(client/retry): include error info in logs when retry occurs (apache#487) On a request retry, it logs an info message stating that an error was encountered and information about the retry process but it hasn't included any details about the error that is causing the retry. This PR updates the logging to include the status if it is a server error and the http error kind if a transport error occurred. While the last error when retries are exhausted is returned up the call stack, the intermediate errors need not be exactly the same. It is helpful to include some minimum information about what error triggered a retry each time it happens. * aws: downgrade credential provider info! log messages to debug! (apache#436) These log messages are very noisy. * Add storage class for aws, gcp, and azure (apache#456) * Add storage class for aws and gcp * Add azure storage class attribute * Update attribute docs * Update http client * Add version 0.12.4 release plan to README (apache#490) * Fix for clippy 1.90 (apache#492) * AWS S3: Support STS endpoint, WebIdentity, RoleArn, RoleSession configuration (apache#480) * Allow setting STS endpoint via env var * Properly use AmazonS3Builder::credentials_from_env for AssumeRoleWithWebIdentity auth flow --------- Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org> * Revert "refactor: remove AWS dynamo integration (apache#407)" (apache#493) This reverts commit 034733f. * Update version to 0.12.4 and add changelog (apache#491) * Update version to 0.12.4 * Update update_changelog.sh script * Update changelog * Last touchups * Update changelog * Reapply "refactor: remove AWS dynamo integration (apache#407)" (apache#494) This reverts commit bebd53b. * Add Content_length header to S3 create_multipart (apache#496) * minor: Fix MSRV CI workflow (apache#502) * ci: Fix MSRV CI workflow * Update .github/workflows/rust.yml * Update .github/workflows/rust.yml --------- Co-authored-by: Matthijs Brobbel <m1brobbel@gmail.com> * All changes to support SAP HDLFS * add hdlfs * FOS-9911: support delta path version 2.0 * FOS-10349: fix object_url (#2) * FOS-10349: fix object_url * FOS-11081: sync code --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org> Co-authored-by: Matthijs Brobbel <m1brobbel@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kyle Barron <kyle@developmentseed.org> Co-authored-by: Phil Bracikowski <13472206+philjb@users.noreply.github.com> Co-authored-by: Alfonso Subiotto Marqués <alfonso.subiotto@polarsignals.com> Co-authored-by: Matthew Turner <me@matthewmturner.com> Co-authored-by: Matt Friede <7852262+Friede80@users.noreply.github.com> Co-authored-by: Yongming Ding <yongmingd@nvidia.com>
This was referenced May 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #330
Rationale for this change
I would like to set the storage class on objects without a custom HTTP client
What changes are included in this PR?
StorageClassvariant toAttributeawsandgcpclientsI started looking into Azure blob tiers but I have never used Azure before and it wasn't immediately obvious to me that it works the same way as AWS and GCP so I did not include anything for that.
Are there any user-facing changes?
A new
Attributevariant is introduced