Skip to content

Default Kueue config for Pathways#5628

Merged
SwarnaBharathiMantena merged 1 commit into
GoogleCloudPlatform:developfrom
scaliby:default-kueue-config
May 19, 2026
Merged

Default Kueue config for Pathways#5628
SwarnaBharathiMantena merged 1 commit into
GoogleCloudPlatform:developfrom
scaliby:default-kueue-config

Conversation

@scaliby

@scaliby scaliby commented May 8, 2026

Copy link
Copy Markdown
Contributor

Objective

Simplify the configuration management for Pathways and remove brittle inline YAML-merging logic.

Changes

  • Removed the complex inline queue-merging logic inside main.tf.
  • Introduced a dedicated, standalone default template file for Pathways (kueue/kueue-configuration-pathways.yaml.tftpl).
  • Refactored main.tf to conditionally load this template when enable_pathways_for_tpus is true and no custom config_path is provided.

This cleanup makes the module logic much easier to maintain and establishes a clean pattern for adding default configurations for other advanced features.

@scaliby scaliby requested a review from a team as a code owner May 8, 2026 07:19
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request streamlines the configuration management for Pathways by replacing complex, inline Terraform logic with a clean, template-based approach. It also introduces support for dynamic slicing and ensures that the underlying Helm controllers are correctly provisioned with appropriate resource defaults when these advanced features are enabled.

Highlights

  • Configuration Refactoring: Removed brittle inline YAML-merging logic in favor of a dedicated template file for Pathways configurations.
  • Template Management: Introduced a new standalone template file 'kueue-configuration-pathways.yaml.tftpl' and updated the logic to conditionally load it.
  • Dynamic Slicing Support: Added support for dynamic slicing configurations and updated Helm values for Kueue and JobSet controllers to include sensible defaults when enabled.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@github-actions github-actions Bot added the external PR from external contributor label May 8, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the Kueue configuration logic within the kubectl-apply module to support dynamic slicing and Pathways, introducing new templates and conditional resource allocation for controllers. Several critical issues were identified: the kueue variable should use type = any per style guidelines to handle complex structures; two referenced template files for dynamic slicing are missing from the PR; the accelerator_type variable is missing from the default template variables; the default resource requests for Kueue and JobSet controllers are potentially too high for standard nodes; and the apiVersion in the new template should be updated to v1beta2 for better compatibility.

Comment thread modules/management/kubectl-apply/main.tf Outdated
Comment thread modules/management/kubectl-apply/main.tf Outdated
Comment thread modules/management/kubectl-apply/main.tf
Comment thread modules/management/kubectl-apply/main.tf Outdated
Comment thread modules/management/kubectl-apply/kueue/kueue-configuration-pathways.yaml.tftpl Outdated
@scaliby scaliby marked this pull request as draft May 8, 2026 09:31
@scaliby scaliby force-pushed the default-kueue-config branch 2 times, most recently from b35030b to 3b24ea4 Compare May 8, 2026 12:51
@scaliby

scaliby commented May 8, 2026

Copy link
Copy Markdown
Contributor Author

/gemini summary

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

This pull request simplifies configuration management for Pathways by replacing complex, inline Terraform logic with a robust, template-based system. It introduces dedicated template files for Pathways and dynamic slicing, refactors the main configuration logic to conditionally load these templates, and updates Helm controller resource defaults to better support these advanced features. These changes improve code maintainability and provide a more flexible configuration interface for users.

Highlights

  • Configuration Refactoring: Removed brittle inline YAML-merging logic in favor of a dedicated template-based approach for Pathways configurations.
  • Template Management: Introduced standalone template files for Pathways and dynamic slicing, improving maintainability and establishing a clean pattern for future configurations.
  • Dynamic Slicing Support: Added support for dynamic slicing and updated Helm controller resource defaults to ensure proper provisioning when advanced features are enabled.
  • Variable Flexibility: Updated the 'kueue' variable type to 'any' to allow for more flexible and user-friendly configuration inputs.
Activity
  • Gemini Code Assist provided a preliminary summary and identified several areas for improvement, including variable typing and template rendering concerns.
  • The author requested a summary update.
  • The pull request includes updates to Terraform modules, variable definitions, and the addition of new configuration templates.

@scaliby scaliby marked this pull request as ready for review May 8, 2026 13:02
@SwarnaBharathiMantena SwarnaBharathiMantena added the release-module-improvements Added to release notes under the "Module Improvements" heading. label May 11, 2026
@SwarnaBharathiMantena SwarnaBharathiMantena self-requested a review May 11, 2026 09:25
@scaliby scaliby force-pushed the default-kueue-config branch from 3b24ea4 to 2943f69 Compare May 11, 2026 10:41
@SwarnaBharathiMantena

Copy link
Copy Markdown
Contributor

/gcbrun

1 similar comment
@SwarnaBharathiMantena

Copy link
Copy Markdown
Contributor

/gcbrun

@SwarnaBharathiMantena SwarnaBharathiMantena left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

The failing tests are either capacity issues or known issues the team is working on.

@scaliby scaliby force-pushed the default-kueue-config branch 2 times, most recently from 40f1676 to c37bfab Compare May 12, 2026 13:55
@SwarnaBharathiMantena

Copy link
Copy Markdown
Contributor

/gcbrun

@SwarnaBharathiMantena SwarnaBharathiMantena self-requested a review May 12, 2026 17:12
@SwarnaBharathiMantena

Copy link
Copy Markdown
Contributor

/gcbrun

Comment thread modules/management/kubectl-apply/main.tf Outdated
Comment thread modules/management/kubectl-apply/variables.tf

@Neelabh94 Neelabh94 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address the comments above.

@scaliby scaliby force-pushed the default-kueue-config branch 2 times, most recently from d7f54a4 to d71a859 Compare May 13, 2026 13:21
@scaliby scaliby force-pushed the default-kueue-config branch 5 times, most recently from de5d7e3 to f7f292b Compare May 13, 2026 14:33
@scaliby scaliby requested review from Neelabh94 and jamOne- and removed request for Neelabh94 May 13, 2026 14:47
Comment thread modules/management/kubectl-apply/main.tf Outdated
Comment thread modules/management/kubectl-apply/main.tf Outdated
@scaliby scaliby force-pushed the default-kueue-config branch 2 times, most recently from 6cb5c96 to d11f84a Compare May 14, 2026 08:27
@scaliby scaliby requested a review from jamOne- May 14, 2026 08:28

@jamOne- jamOne- left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Comment thread modules/management/kubectl-apply/main.tf
@scaliby scaliby force-pushed the default-kueue-config branch from d11f84a to 82e9358 Compare May 14, 2026 12:08
@scaliby

scaliby commented May 14, 2026

Copy link
Copy Markdown
Contributor Author

Please address the comments above.

Done. I also added accelerator_type to blueprints as we discussed.

@Neelabh94

Copy link
Copy Markdown
Contributor

/gcbrun

@Neelabh94 Neelabh94 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@scaliby

scaliby commented May 19, 2026

Copy link
Copy Markdown
Contributor Author

@SwarnaBharathiMantena can we get this merged?

@SwarnaBharathiMantena SwarnaBharathiMantena enabled auto-merge (squash) May 19, 2026 17:10
@SwarnaBharathiMantena SwarnaBharathiMantena merged commit 962b1d3 into GoogleCloudPlatform:develop May 19, 2026
60 of 82 checks passed
kadupoornima pushed a commit to kadupoornima/cluster-toolkit that referenced this pull request May 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external PR from external contributor release-module-improvements Added to release notes under the "Module Improvements" heading.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants