ci: Enable parallel test execution#4058
Conversation
Summary of ChangesHello @AbgarSim, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly upgrades the project's testing capabilities by integrating JUnit 5 and enabling parallel test execution. The primary goal is to enhance developer productivity and accelerate feedback loops by allowing tests to run concurrently, thereby reducing the overall test suite execution time. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request introduces parallel test execution for JUnit 5 by adding a junit-platform.properties file. This is a great step towards speeding up the build. My review includes two suggestions to improve the parallel execution configuration. First, I recommend changing the method execution mode to same_thread to prevent potential race conditions in tests that are not thread-safe. Second, I suggest using a dynamic parallelism strategy instead of a fixed one to better adapt to different execution environments. These changes should make the parallel test execution more robust and efficient.
| @@ -0,0 +1,7 @@ | |||
| junit.jupiter.execution.parallel.enabled=true | |||
|
|
|||
| junit.jupiter.execution.parallel.mode.default=concurrent | |||
There was a problem hiding this comment.
Setting junit.jupiter.execution.parallel.mode.default to concurrent will execute test methods within the same test class in parallel. This can introduce flakiness and hard-to-debug race conditions if tests share mutable state via instance fields. A safer default is same_thread, which executes methods within a class sequentially but still allows different test classes to run in parallel. Please ensure that tests are designed to be thread-safe at the method level before using concurrent mode.
junit.jupiter.execution.parallel.mode.default=same_thread
There was a problem hiding this comment.
So testing locally seems that this same_thread is not helping at all with the speed as opposed to concurrent, also running locally a couple of times I don't see any flake failures. Please advice if we can leave as is and change if there are actually flaky tests due to this @lqiu96
There was a problem hiding this comment.
Hmm, I just tried this locally and I think it should be helping:
Without parallel:
[INFO] Reactor Summary for BigQuery Parent 2.57.2-SNAPSHOT:
[INFO]
[INFO] BigQuery Parent .................................... SUCCESS [ 0.762 s]
[INFO] BigQuery ........................................... SUCCESS [02:29 min]
[INFO] Google Cloud BigQuery BOM .......................... SUCCESS [ 0.003 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:31 min
[INFO] Finished at: 2026-01-12T17:22:00-05:00
[INFO] ------------------------------------------------------------------------
Parallel:
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for BigQuery Parent 2.57.2-SNAPSHOT:
[INFO]
[INFO] BigQuery Parent .................................... SUCCESS [ 0.695 s]
[INFO] BigQuery ........................................... SUCCESS [ 28.755 s]
[INFO] Google Cloud BigQuery BOM .......................... SUCCESS [ 0.004 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 29.753 s
[INFO] Finished at: 2026-01-12T17:24:15-05:00
[INFO] ------------------------------------------------------------------------
I think there is quite a big speed up (especially for our GraalVM tests).
There was a problem hiding this comment.
Have you considered this combination?
junit.jupiter.execution.parallel.enabled=true
# Run methods within a class sequentially (avoids shared state issues)
junit.jupiter.execution.parallel.mode.default=same_thread
# Run top-level classes in parallel (provides the speedup)
junit.jupiter.execution.parallel.mode.classes.default=concurrent
There was a problem hiding this comment.
Oh, apologies. I completely missed the original gemini comment. I don't think the gemini suggestion is needed for this PR. From what I've seen, BigQuery tests don't share state and I think we can run everything in parallel.
I've run this quite a few times and I don't see any concerns with concurrent tests. I think we can keep this change as is.
google-cloud-bigquery/src/test/resources/junit-platform.properties
Outdated
Show resolved
Hide resolved
google-cloud-bigquery/src/test/resources/junit-platform.properties
Outdated
Show resolved
Hide resolved
google-cloud-bigquery/src/test/resources/junit-platform.properties
Outdated
Show resolved
Hide resolved
|
/gcbrun |
|
@AbgarSim resolved some flaky tests in a separate PR. If you pull the latest changes, the tests should pass now |
52bb2fb to
6f8191d
Compare
Rebased |
6f8191d to
31a0eff
Compare
|
/gcbrun |
|
Seems like we have a flaky test and some quota issue: I'll re-run this in a bit and look to try and disable this test |
* chore: add release-please config for protobuf-4.x (#4009) * chore: add release-please config for protobuf-4.x * Change release type from java-lts to java-yoshi * feat: Introduce DataFormatOptions to configure the output of BigQuery data types (#4010) * feat: Create DataFormatOptions in BigQuery * feat: Add Builder class for DataFormatOptions * fix: Update existing references of useInt64Timestamp to use DataFormatOption's variant * chore: Fix lint issues * chore: Address PR feedback * chore: Add tests for useInt64Timestamp behavior * chore: Address failing tests and GCA * chore: Remove unused fromPb method * feat: Add timestamp_precision to Field (#4014) * feat: Add timestamp_precision to Field * Apply suggestion from @gemini-code-assist[bot] Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * chore: Address GCA PR feedback * chore: Fix typo * chore: Remove default value * chore: Address PR feedback --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * chore: Use custom timestamp validator for ISO8601 timestamps with more than nanosecond precision (#4017) * chore: Use custom timestamp validator for ISO8601 timestamps with more than nanosecond precision * chore: Rename helper method to validateTimestamp * chore: Address GCA comments * chore: Address last GCA comment * chore: Update to use assert helper method * chore(main): release 2.56.1-SNAPSHOT (#4001) Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> * deps: update dependency com.google.cloud:sdk-platform-java-config to v3.54.2 (#4022) * chore(main): release 2.57.0 (#4021) Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> * test: Clean up resources created in ITs (#4024) * chore: Cleanup initialized resources * chore: Delete created dataset * chore: Use try-with to close stream * chore(deps): update dependency com.google.cloud:google-cloud-bigquerystorage-bom to v3.19.0 (#4025) * chore(main): release 2.57.1-SNAPSHOT (#4023) Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> * chore: Cleanup Intellij test warnings (#4026) * chore: Cleanup initialized resources * chore: Use try-with to close stream * chore: Cleanup Intellij test warnings * chore: Allow for floating point inaccuracies * chore: Add otel delete operation * deps: update actions/upload-artifact action to v6 (#4027) Co-authored-by: Blake Li <blakeli@google.com> * chore: Use assertThrows in tests (#4028) * chore: Cleanup initialized resources * chore: Use try-with to close stream * chore: Cleanup Intellij test warnings * chore: Allow for floating point inaccuracies * chore: Add otel delete operation * chore: Clean up rest of the IT test file * chore: Add GCA feedback * chore(main): release 2.57.1 (#4029) Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> * chore: Update renovate.json (#4031) deps update from bigquerystorage was incorrectly marked as chore(deps). This would cause the upgrade not show up in the release notes and also not trigger release please. This is because the pattern /^com.google.cloud:google-cloud-bigquery is configured in renovate.json, which matches both bigquery and bigquerystorage. However, I believe the original intention is only for bigquery update in samples, not for bigquerystorage update. Fixing the pattern so that the String must ends with bigquery * chore(main): release 2.57.2-SNAPSHOT (#4032) Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> * test: Add integration tests for picosecond support (#4030) * test: Add integration tests for picosecond support * chore: Add a micro -> pico exact timestamp test case * chore: Add additional test cases * chore: Fix test cases with 'Z' * chore: Test if first query has an impact * chore: Remove flaky test for now * chore: Remove testing comment * fix: Job.isDone() uses Job.Status.State if available (#4039) * chore: remove build badges (#4046) b/468377909 * chore: handled race condition in stateless query integration test (#4045) * fix: handled race condition in stateless query integration test The testTableResultJobIdAndQueryId test was failing intermittently on slower networks. The test strictly asserted that Job ID must be null for stateless queries. However, the library correctly falls back to creating a Job ID if the stateless query times out. This change updates the assertion logic to accept either a valid Query ID (stateless success) or a valid Job ID (fallback success). Fixes #4008 * refactor: use XOR assertion for conciseness Applied feedback from code review to use exclusive OR operator for validating JobID/QueryID mutual exclusivity. * fix: apply race condition logic to testStatelessQueries Applied XOR assertion logic to testStatelessQueries. Test was failing on slow networks because they did not account for JOB_CREATION_OPTIONAL falling back to job creation. Fixes #4002 * docs: add comment explaining stateless query fallback behavior * docs: add comment explaining stateless query fallback behavior in testTableResultJobIdAndQueryId() * chore: Ignore unused declared junit-jupiter-engine error in dependencies check (#4048) * chore: Ingore unused junit-jupiter-engine error in dependencies check * chore: Update pom.xml Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * chore: Update comment format for junit-jupiter-engine --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * fix: gracefully handle thread interruption in ConnectionImpl to preve… (#4047) * fix: gracefully handle thread interruption in ConnectionImpl to prevent CI flakes Fixes #3992 * fix: consolidate interrupt checks and handle raw InterruptedException as per review * fix: remove CancelledException check per review feedback * chore(deps): update dependency com.google.cloud:libraries-bom to v26.73.0 (#3922) * docs: Add specific samples for creating and query timestamps (#4051) * docs: Add specific samples for creating and query timestamps * chore: Fix samples checkstyle issue * chore: Address gemini suggestions * chore: Update expiration time for test * chore: Migrate tests to JUnit5 (#4052) * feat: Migrate to JUnit 5 and add parallel test execution * feat: Migrate tests to JUnit5 * chore: Add surefire-junit-platform dep for ITs * chore: Fix broken tests * chore: Upgrade existing integration tests to JUnit 5 syntax and features * chore: Upgrade ITNightlyBigQueryTest to JUnit 5 features and package-private * chore: Make the tests package-private * feat: migrate tests to JUnit 5 assertThrows and static imports * chore: Remove wildcard imports * chore: revert samples to use junit4 * chore: Address code comments * chore: Close connection after test --------- Co-authored-by: AbgarSim <abgar1223@gmail.com> * chore: move Google JDBC driver code (#4050) * test: Use unique table names in tests (#4053) * feat: Migrate to JUnit 5 and add parallel test execution * feat: Migrate tests to JUnit5 * chore: Add surefire-junit-platform dep for ITs * test: Enable parallel tests for JUnit5 * chore: Fix broken tests * chore: Upgrade existing integration tests to JUnit 5 syntax and features * chore: Upgrade ITNightlyBigQueryTest to JUnit 5 features and package-private * chore: Make the tests package-private * feat: migrate tests to JUnit 5 assertThrows and static imports * chore: Remove wildcard imports * chore: revert samples to use junit4 * chore: Address code comments * chore: Close connection after test * chore: Fix flaky tests * chore: Fix flaky tests * chore: Fix tests * chore: Disable cache for query stats * chore: Add unique id to each table * chore: Use unique test table names * chore: Remove parallel test execution * chore: Add comment for cache * chore: Fix broken test --------- Co-authored-by: AbgarSim <abgar1223@gmail.com> * feat:Add JUnit 5 and add parallel test execution (#4058) * chore: cleanup release-please config (#4013) * chore: cleanup release-please config - Remove redundant options already declared at the top level.\n- Remove bumpMinorPreMajor for repositories after the first major release. * chore: format release-please.yml * chore(deps): update dependency com.google.cloud:sdk-platform-java-config to v3.55.1 (#4060) * chore(main): update CODEOWNERS with JDBC ownership (#4063) * chore(deps): update dependency com.google.cloud:google-cloud-bigquerystorage-bom to v3.19.1 (#4036) * chore(deps): update dependency com.google.cloud:google-cloud-bigquerystorage-bom to v3.20.0-rc1 * Update google-cloud-bigquerystorage-bom version --------- Co-authored-by: Diego Marquez <diegomarquezp@google.com> * chore: Enable exponential backoff for retries in tests (#4059) * chore: Enable exponential backoff for retries in tests * chore: Clone table for DML query tests * chore: Use GCA retry settings code suggestion * chore(jdbc): moving helper scripts & simplify dependencies (#4062) * chore: Fix flaky testListTablesWithPartitioning test (#4068) * chore: Fix flaky testListTablesWithPartitioning test * chore: Fix test comment * chore: Fix typo to partitionType * chore(main): release 2.57.2 (#4044) Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> --------- Co-authored-by: Diego Marquez <diegomarquezp@google.com> Co-authored-by: Lawrence Qiu <lawrenceqiu@google.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Abgar Simonean <abgar1223@gmail.com> Co-authored-by: Tomo Suzuki <suztomo@google.com> Co-authored-by: Sivamurugan P <73530366+SivamuruganP@users.noreply.github.com> Co-authored-by: Kirill Logachev <kirl@google.com>
BEGIN_COMMIT_OVERRIDE
ci: Enable parallel test execution
END_COMMIT_OVERRIDE
Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
Fixes #4041
If you write sample code, please follow the samples format.