Description

This pull request enhances the gke-tpu-v6-advanced blueprint by integrating Google Cloud Managed Lustre as an optional storage solution. Large-scale TPU workloads are often bottlenecked by storage I/O, and Managed Lustre addresses this by providing a parallel file system capable of delivering extremely high throughput and low latency, maximizing TPU utilization.

Changes Included

• Added commented-out variables for Lustre configuration to the blueprint vars.
• Included new, commented-out modules for private_service_access, lustre_firewall_rule, managed-lustre, and `lustre-pv.
• Enabled the enable_managed_lustre_csi flag on the GKE cluster module by default.
• Updated the README.md with a comprehensive guide on enabling and using the new Managed Lustre integration.
  Validation:

• Manual end-to-end testing was performed to validate the integration, confirming that a pod could successfully mount and perform I/O operations on the Lustre filesystem.

Submission Checklist

NOTE: Community submissions can take up to 2 weeks to be reviewed.

Please take the following actions before submitting this pull request.

• Fork your PR branch from the Toolkit "develop" branch (not main)
• Test all changes with pre-commit in a local branch #
• Confirm that "make tests" passes all tests
• Add or modify unit tests to cover code changes
• Ensure that unit test coverage remains above 80%
• Update all applicable documentation
• Follow Cluster Toolkit Contribution guidelines #