Skip to content

ci: implement conditional sharding logic in integration tests#2692

Merged
anubhav756 merged 4 commits into
googleapis:mainfrom
NirajNandre:ci/shard-integration-tests
Mar 17, 2026
Merged

ci: implement conditional sharding logic in integration tests#2692
anubhav756 merged 4 commits into
googleapis:mainfrom
NirajNandre:ci/shard-integration-tests

Conversation

@NirajNandre

@NirajNandre NirajNandre commented Mar 10, 2026

Copy link
Copy Markdown
Contributor

Description

This PR implements test sharding within the integration suite to address persistent flakiness, resource exhaustion, and long execution times. By enabling selective test execution based on targeted triggers.

Implementation

  • Conditional YAML Logic: Updated .ci/integration.cloudbuild.yaml to wrap each database-specific step in a conditional block that checks the _DB_SHARD variable.
  • Backward Compatibility: Maintained support for the original integration-test-pr trigger by allowing steps to run if _DB_SHARD is unset (empty), ensuring full-suite verification still works.

Should include a concise description of the changes (bug or feature), it's
impact, along with a summary of the solution

PR Checklist

Thank you for opening a Pull Request! Before submitting your PR, there are a
few things you can do to make sure it goes smoothly:

  • Make sure you reviewed
    CONTRIBUTING.md
  • Make sure to open an issue as a
    bug/issue
    before writing your code! That way we can discuss the change, evaluate
    designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)
  • Make sure to add ! if this involve a breaking change

🛠️ Fixes #<issue_number_goes_here>

@NirajNandre NirajNandre requested a review from a team as a code owner March 10, 2026 09:28
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a significant enhancement to the continuous integration pipeline by implementing a conditional test sharding mechanism for the integration test suite. This change allows for more targeted and efficient execution of tests, specifically addressing long-standing issues related to test flakiness, excessive resource utilization, and prolonged build times. By enabling the selective running of database-specific tests, the overall CI process is expected to become more robust and faster.

Highlights

  • Conditional Test Sharding: Implemented logic to conditionally run integration tests based on a _DB_SHARD environment variable.
  • Improved CI Efficiency: Aims to reduce flakiness, resource consumption, and execution time of the integration test suite by allowing selective test execution.
  • Backward Compatibility: Ensures that if the _DB_SHARD variable is not set, the full integration test suite will still execute as before.
Changelog
  • .ci/integration.cloudbuild.yaml
    • Wrapped each database-specific integration test step (e.g., Cloud SQL Postgres, AlloyDB, Bigtable, BigQuery, Dataform, etc.) in a conditional if block.
    • Added logic to check the _DB_SHARD environment variable: if it matches the specific database's shard name or is unset, the test runs; otherwise, it is skipped.
    • Included echo statements to log whether a test is running or being skipped.
    • Modified the Oracle test step to also include the conditional sharding logic, along with its specific setup and coverage checks.
    • Updated the MariaDB test step with conditional sharding logic.
Activity
  • The pull request was created by NirajNandre.
  • The author has confirmed reviewing CONTRIBUTING.md.
  • Several checklist items in the PR description remain unchecked, suggesting ongoing work or pending verification (e.g., opening an issue, ensuring tests/linter pass, code coverage, documentation updates, and breaking change assessment).
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@twishabansal

Copy link
Copy Markdown
Contributor

Instead of adding multiple triggers (one for each db), how about we filter on file changes and run only the affected steps in a single trigger?

A single trigger seems like a better user experience for contributors. They shouldn't have to deal with 10s of triggers. One trigger for each source also requires us to add a new trigger on the introduction of another source. WDYT?

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces test sharding to the integration test suite by adding conditional logic to the Cloud Build configuration. The changes correctly implement the sharding, but introduce a large amount of code duplication by repeating the same if/else block for every test step. I've added a comment suggesting a refactoring using a guard clause pattern. This would make the scripts more readable, reduce nesting, and should be applied across all steps to improve the overall maintainability of this CI configuration.

Note: Security Review is unavailable for this PR.

Comment thread .ci/integration.cloudbuild.yaml Outdated
@anubhav756

Copy link
Copy Markdown
Contributor

Thanks for the inputs!

how about we filter on file changes and run only the affected steps in a single trigger

We did discuss this approach, and have documented the pros and cons here: https://docs.google.com/document/d/1xbbuOKanMOnrwlP5z4pHhNKRBnIzrGFB_kOZ9a3IOOM/edit?tab=t.0#heading=h.blexckjdnwgr

They shouldn't have to deal with 10s of triggers

Could you elaborate a bit more on this? I want to understand what problems could occur for a dev as compared to a single trigger with the entire logic (we can discuss on the doc if that's more convenient).

One trigger for each source also requires us to add a new trigger on the introduction of another source

I assume in case of a single trigger also, we would have to update the logic to accommodate the new source. Do you mean that in case of a single trigger, the dev would be able to add the trigger logic potentially in the same PR as compared to multiple native triggers, where they would have to create the trigger in GCP only after the PR is merged? We can anyways weigh the options in the doc.

@Yuan325 Yuan325 assigned anubhav756 and unassigned Yuan325 Mar 11, 2026
@Yuan325 Yuan325 added the priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. label Mar 11, 2026
@anubhav756

Copy link
Copy Markdown
Contributor

/gcbrun

@NirajNandre NirajNandre force-pushed the ci/shard-integration-tests branch from 7db3145 to 80d4ed5 Compare March 16, 2026 13:17
@anubhav756

Copy link
Copy Markdown
Contributor

/gcbrun

@anubhav756

Copy link
Copy Markdown
Contributor

/gcbrun

@anubhav756

Copy link
Copy Markdown
Contributor

/gcbrun

1 similar comment
@anubhav756

Copy link
Copy Markdown
Contributor

/gcbrun

@anubhav756 anubhav756 merged commit 8deb3d4 into googleapis:main Mar 17, 2026
15 checks passed
NirajNandre added a commit to NirajNandre/genai-toolbox-fork that referenced this pull request Mar 17, 2026
anubhav756 pushed a commit to NirajNandre/genai-toolbox-fork that referenced this pull request Mar 17, 2026
NightStack15 added a commit to NightStack15/googleapis-_-genai-toolbox that referenced this pull request Mar 20, 2026
NirajNandre added a commit to NirajNandre/genai-toolbox-fork that referenced this pull request Mar 24, 2026
…apis#2692)

## Description

This PR implements test sharding within the integration suite to address
persistent flakiness, resource exhaustion, and long execution times. By
enabling selective test execution based on targeted triggers.

### Implementation

- **Conditional YAML Logic**: Updated .ci/integration.cloudbuild.yaml to
wrap each database-specific step in a conditional block that checks the`
_DB_SHARD` variable.
- **Backward Compatibility**: Maintained support for the original
integration-test-pr trigger by allowing steps to run if `_DB_SHARD` is
unset (empty), ensuring full-suite verification still works.

> Should include a concise description of the changes (bug or feature),
it's
> impact, along with a summary of the solution

## PR Checklist

> Thank you for opening a Pull Request! Before submitting your PR, there
are a
> few things you can do to make sure it goes smoothly:

- [x] Make sure you reviewed

[CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md)
- [ ] Make sure to open an issue as a

[bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose)
  before writing your code! That way we can discuss the change, evaluate
  designs, and agree on the general idea
- [ ] Ensure the tests and linter pass
- [ ] Code coverage does not decrease (if any source code was changed)
- [ ] Appropriate docs were updated (if necessary)
- [ ] Make sure to add `!` if this involve a breaking change

🛠️ Fixes #<issue_number_goes_here>
NirajNandre added a commit to NirajNandre/genai-toolbox-fork that referenced this pull request Mar 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants