Skip to content

Add Managed Lustre to pre-existing-filestore module#3937

Merged
cdunbar13 merged 5 commits into
GoogleCloudPlatform:developfrom
cdunbar13:lustre-pre-existing
Apr 17, 2025
Merged

Add Managed Lustre to pre-existing-filestore module#3937
cdunbar13 merged 5 commits into
GoogleCloudPlatform:developfrom
cdunbar13:lustre-pre-existing

Conversation

@cdunbar13

@cdunbar13 cdunbar13 commented Apr 14, 2025

Copy link
Copy Markdown
Collaborator

Managed Lustre is a new offering on Google Cloud and the initial push to add it to the Toolkit is to add it to the pre-existing-filestore module that takes existing storage infrastructure and mounts it to infrastructure managed by the Toolkit. As of right now, the client drivers for this are only usable on Rocky 8, and Ubuntu 20, 22 images that have a specific kernel version.

Compatibility

  • Rocky 8
  • HPC Rocky 8
  • Ubuntu 20.04
    • kernel version: 5.15.0-1075-gcp
    • recommended image name: ubuntu-2004-focal-v20250130
  • Ubuntu 22.04
    • kernel version: 6.8.0-1021-gcp
    • recommended image name: ubuntu-2204-jammy-v20250128

This was tested (not benchmarked) on each of these OS/images.

@cdunbar13 cdunbar13 added the release-module-improvements Added to release notes under the "Module Improvements" heading. label Apr 14, 2025
@cdunbar13 cdunbar13 requested review from a team and samskillman as code owners April 14, 2025 15:08
@cdunbar13 cdunbar13 mentioned this pull request Apr 15, 2025
Merged
@bernardhan33

bernardhan33 commented Apr 15, 2025 •
edited
Loading

Copy link
Copy Markdown

Drive-by comment (hope you don't mind ;)):

  1. The managed lustre documentation suggests "ubuntu-2004-focal-v20250213" for "Ubuntu 20.04 LTS" family and the description of the PR indicates that the test was done on "ubuntu-2004-focal-v20250130". I just want to confirm that this discrepancy is fine?
  2. A specific configuration step is needed to configure access to managed Lustre instances with --gke-support-enabled but I don't seem to find it in this PR or Adding Managed Lustre to Cluster Toolkit #3950? Did I miss anything?

Thank you!

@cdunbar13

cdunbar13 commented Apr 15, 2025

Copy link
Copy Markdown
Collaborator Author

@bernardhan33

  1. In regards to the image names, when I tested that image actually didn't work as it had a newer kernel than was compatible with the available packages. That may have changed since then.

  2. That has not been included yet. That will be added later as it requires the lnet.conf file modification.

@cdunbar13 cdunbar13 assigned harshthakkar01 and unassigned tpdownes Apr 16, 2025
mr0re1
mr0re1 reviewed Apr 16, 2025
View reviewed changes
Comment thread modules/file-system/pre-existing-network-storage/scripts/install-managed-lustre-client.sh Outdated
mr0re1
mr0re1 previously approved these changes Apr 16, 2025
View reviewed changes

@mr0re1 mr0re1 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than "should drivers be a part of the image instead" question, LGTM.
Let's settle on "image" question offline.

harshthakkar01
harshthakkar01 reviewed Apr 16, 2025
View reviewed changes
Comment thread modules/file-system/pre-existing-network-storage/README.md Outdated
@cdunbar13 cdunbar13 force-pushed the lustre-pre-existing branch from 6094d41 to e67e358 Compare April 16, 2025 20:11
@cdunbar13 cdunbar13 requested a review from samskillman April 17, 2025 13:11
@cdunbar13 cdunbar13 force-pushed the lustre-pre-existing branch from e67e358 to 2685e51 Compare April 17, 2025 13:24
@cdunbar13 cdunbar13 force-pushed the lustre-pre-existing branch 2 times, most recently from bebaab6 to 638c80c Compare April 17, 2025 18:42
@cdunbar13 cdunbar13 force-pushed the lustre-pre-existing branch from 638c80c to 6b251fa Compare April 17, 2025 18:43
@cdunbar13 cdunbar13 merged commit db65389 into GoogleCloudPlatform:develop Apr 17, 2025
@cdunbar13 cdunbar13 deleted the lustre-pre-existing branch April 17, 2025 18:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@harshthakkar01 harshthakkar01 harshthakkar01 approved these changes

@mr0re1 mr0re1 Awaiting requested review from mr0re1

@samskillman samskillman Awaiting requested review from samskillman

Assignees

@samskillman samskillman

@harshthakkar01 harshthakkar01

Labels

release-module-improvements Added to release notes under the "Module Improvements" heading.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@cdunbar13 @bernardhan33 @mr0re1 @samskillman @harshthakkar01 @tpdownes