Skip to content

Managed Lustre Data Import (Hydration)#4045

Merged
cdunbar13 merged 1 commit into
GoogleCloudPlatform:developfrom
cdunbar13:lustre-hydration
May 7, 2025
Merged

Managed Lustre Data Import (Hydration)#4045
cdunbar13 merged 1 commit into
GoogleCloudPlatform:developfrom
cdunbar13:lustre-hydration

Conversation

@cdunbar13

Copy link
Copy Markdown
Collaborator

This PR adds the capability to import data from GCS bucket upon creation of the Managed Lustre instance.

The bucket is checked for existence prior to running the import command.

The import command has an option for a destination folder which was ignored. This was due to the fact that we're creating a fresh Lustre instance that will not have any data, and the destination folder option requires the specified folder to exist before importing.

Note: This will fail quietly if permissions are not set up correctly. Without the ability to check specific permissions in Terraform this is difficult to do in an automated way, so users are warned in the documentation.

This was tested (using a modified pfs-managed-lustre-vm.yaml blueprint):

  1. Importing data from bucket within the same project
  2. Importing data from another project
  3. No data import
  4. Non-existing bucket

@cdunbar13 cdunbar13 added the release-module-improvements Added to release notes under the "Module Improvements" heading. label May 2, 2025
@cdunbar13 cdunbar13 requested review from a team and samskillman as code owners May 2, 2025 16:51
@cdunbar13 cdunbar13 force-pushed the lustre-hydration branch from 04f761e to 5024e63 Compare May 2, 2025 17:36
@cdunbar13 cdunbar13 assigned harshthakkar01 and unassigned tpdownes May 2, 2025
Comment thread modules/file-system/managed-lustre/main.tf Outdated
Comment thread modules/file-system/managed-lustre/main.tf
@cdunbar13 cdunbar13 force-pushed the lustre-hydration branch from 5024e63 to d718e77 Compare May 7, 2025 20:22
@cdunbar13 cdunbar13 requested a review from harshthakkar01 May 7, 2025 20:23
@cdunbar13 cdunbar13 merged commit 6f1573d into GoogleCloudPlatform:develop May 7, 2025
13 of 65 checks passed
@cdunbar13 cdunbar13 deleted the lustre-hydration branch May 7, 2025 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-module-improvements Added to release notes under the "Module Improvements" heading.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants