ci: implement conditional sharding logic in integration tests#2692
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a significant enhancement to the continuous integration pipeline by implementing a conditional test sharding mechanism for the integration test suite. This change allows for more targeted and efficient execution of tests, specifically addressing long-standing issues related to test flakiness, excessive resource utilization, and prolonged build times. By enabling the selective running of database-specific tests, the overall CI process is expected to become more robust and faster. Highlights
Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
|
Instead of adding multiple triggers (one for each db), how about we filter on file changes and run only the affected steps in a single trigger? A single trigger seems like a better user experience for contributors. They shouldn't have to deal with 10s of triggers. One trigger for each source also requires us to add a new trigger on the introduction of another source. WDYT? |
There was a problem hiding this comment.
Code Review
This pull request introduces test sharding to the integration test suite by adding conditional logic to the Cloud Build configuration. The changes correctly implement the sharding, but introduce a large amount of code duplication by repeating the same if/else block for every test step. I've added a comment suggesting a refactoring using a guard clause pattern. This would make the scripts more readable, reduce nesting, and should be applied across all steps to improve the overall maintainability of this CI configuration.
Note: Security Review is unavailable for this PR.
|
Thanks for the inputs!
We did discuss this approach, and have documented the pros and cons here: https://docs.google.com/document/d/1xbbuOKanMOnrwlP5z4pHhNKRBnIzrGFB_kOZ9a3IOOM/edit?tab=t.0#heading=h.blexckjdnwgr
Could you elaborate a bit more on this? I want to understand what problems could occur for a dev as compared to a single trigger with the entire logic (we can discuss on the doc if that's more convenient).
I assume in case of a single trigger also, we would have to update the logic to accommodate the new source. Do you mean that in case of a single trigger, the dev would be able to add the trigger logic potentially in the same PR as compared to multiple native triggers, where they would have to create the trigger in GCP only after the PR is merged? We can anyways weigh the options in the doc. |
|
/gcbrun |
7db3145 to
80d4ed5
Compare
|
/gcbrun |
|
/gcbrun |
|
/gcbrun |
1 similar comment
|
/gcbrun |
…googleapis#2692)" This reverts commit 8deb3d4.
…googleapis#2692)" This reverts commit 8deb3d4.
…apis#2692) ## Description This PR implements test sharding within the integration suite to address persistent flakiness, resource exhaustion, and long execution times. By enabling selective test execution based on targeted triggers. ### Implementation - **Conditional YAML Logic**: Updated .ci/integration.cloudbuild.yaml to wrap each database-specific step in a conditional block that checks the` _DB_SHARD` variable. - **Backward Compatibility**: Maintained support for the original integration-test-pr trigger by allowing steps to run if `_DB_SHARD` is unset (empty), ensuring full-suite verification still works. > Should include a concise description of the changes (bug or feature), it's > impact, along with a summary of the solution ## PR Checklist > Thank you for opening a Pull Request! Before submitting your PR, there are a > few things you can do to make sure it goes smoothly: - [x] Make sure you reviewed [CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md) - [ ] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose) before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea - [ ] Ensure the tests and linter pass - [ ] Code coverage does not decrease (if any source code was changed) - [ ] Appropriate docs were updated (if necessary) - [ ] Make sure to add `!` if this involve a breaking change 🛠️ Fixes #<issue_number_goes_here>
Description
This PR implements test sharding within the integration suite to address persistent flakiness, resource exhaustion, and long execution times. By enabling selective test execution based on targeted triggers.
Implementation
_DB_SHARDvariable._DB_SHARDis unset (empty), ensuring full-suite verification still works.PR Checklist
CONTRIBUTING.md
bug/issue
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
!if this involve a breaking change🛠️ Fixes #<issue_number_goes_here>