Skip to content

Enable hierarchical namespace support in cloud-storage-bucket module#3513

Merged
SwarnaBharathiMantena merged 5 commits into
GoogleCloudPlatform:developfrom
SwarnaBharathiMantena:388111344_hierarchicalns_in_cloud_bucket
Jan 10, 2025
Merged

Enable hierarchical namespace support in cloud-storage-bucket module#3513
SwarnaBharathiMantena merged 5 commits into
GoogleCloudPlatform:developfrom
SwarnaBharathiMantena:388111344_hierarchicalns_in_cloud_bucket

Conversation

@SwarnaBharathiMantena

Copy link
Copy Markdown
Contributor

Enable hierarchical namespace support in cloud-storage-bucket module.
Boolean variable "enable_hierarchical_namespace" with default 'false', that links to this setting in the Terraform resource.

Submission Checklist

NOTE: Community submissions can take up to 2 weeks to be reviewed.

Please take the following actions before submitting this pull request.

  • Fork your PR branch from the Toolkit "develop" branch (not main)
  • Test all changes with pre-commit in a local branch #
  • Confirm that "make tests" passes all tests
  • Add or modify unit tests to cover code changes
  • Ensure that unit test coverage remains above 80%
  • Update all applicable documentation
  • Follow Cluster Toolkit Contribution guidelines #

@SwarnaBharathiMantena SwarnaBharathiMantena added the release-key-new-features Added to release notes under the "Key New Features" heading. label Jan 9, 2025

@tpdownes tpdownes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation says this option must be enabled at creation?

https://cloud.google.com/blog/products/storage-data-transfer/understanding-new-cloud-storage-hierarchical-namespace?e=0

Can you document the UX for the following steps:

  1. Create a bucket without HNS?
  2. Add some objects to it manually (outside terraform)
  3. Using terraform, enable HNS on the existing bucket

My hope is that, either the API fails entirely with a clear-ish error message or, at least, the bucket destruction fails because there are existing objects in the bucket that Terraform doesn't know about.

Additionally add some short text to the variable description saying that the option must be set at first creation of the bucket.

@SwarnaBharathiMantena

Copy link
Copy Markdown
Contributor Author

The UX is as follows.
Steps:

  1. Create a bucket without HNS?
  2. Add some objects to it manually (outside terraform)
  3. Using terraform, enable HNS on the existing bucket

On step 3, the existing bucket without HNS got destroyed and was replaced by a new bucket with HNS enabled. The document uploaded on step 2 is lost. No exclusive warning or error observed.

@tpdownes tpdownes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some concerns about the lifecycle of changing this settings, but let's document and prioritize that as follow-up work.

@tpdownes

Copy link
Copy Markdown
Contributor

I have some concerns about the lifecycle of changing this settings, but let's document and prioritize that as follow-up work.

Some further examination suggests that force_destroy was set to true during this test, so the observed behavior is expected. We will follow up to confirm that objects outside Terraform are not destroyed if force_destroy is set to false (default value).

@SwarnaBharathiMantena

SwarnaBharathiMantena commented Jan 10, 2025

Copy link
Copy Markdown
Contributor Author

Tested out the UX when the force_destroy is set to false, and it gave out an error.

Error: exit status 1
Error: Error trying to delete bucket [bucket-name] containing objects without force_destroy set to true

@SwarnaBharathiMantena SwarnaBharathiMantena merged commit e8bf20a into GoogleCloudPlatform:develop Jan 10, 2025
@SwarnaBharathiMantena SwarnaBharathiMantena deleted the 388111344_hierarchicalns_in_cloud_bucket branch February 5, 2025 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-key-new-features Added to release notes under the "Key New Features" heading.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants