Skip to content

Fix a few test log spams#3220

Merged
Abacn merged 1 commit intoGoogleCloudPlatform:mainfrom
Abacn:logspanm
Jan 14, 2026
Merged

Fix a few test log spams#3220
Abacn merged 1 commit intoGoogleCloudPlatform:mainfrom
Abacn:logspanm

Conversation

@Abacn
Copy link
Copy Markdown
Contributor

@Abacn Abacn commented Jan 14, 2026 •
edited
Loading

Fix #1800

@gemini-code-assist
Copy link
Copy Markdown

gemini-code-assist bot commented Jan 14, 2026

Summary of Changes

Hello @Abacn, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on improving log clarity and reducing verbosity across the codebase. It addresses two distinct sources of log spam: repetitive status messages during Dataflow job waiting and excessive logging from Kafka clients in test environments. By implementing a one-time log for pending jobs and restricting Kafka client logs to only error messages in specific test contexts, the changes aim to make log output more concise and easier to parse for critical information.

Highlights

  • Reduced repetitive logging: The 'Job still pending' message in AbstractPipelineLauncher will now only be logged once per waitUntilActive call, preventing log spam during long-running Dataflow job waits.
  • Suppressed Kafka client logs in tests: New simplelogger.properties files have been added to several Kafka-related modules, configuring org.apache.kafka.clients logging to ERROR level. This significantly reduces verbose Kafka client output during test execution.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

@Abacn Abacn changed the title Fix a few log spams Fix a few test log spams Jan 14, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Jan 14, 2026 •
edited
Loading

Codecov Report

❌ Patch coverage is 25.00000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 50.86%. Comparing base (cb0b95f) to head (4f2b183).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
...beam/it/gcp/dataflow/AbstractPipelineLauncher.java 25.00% 3 Missing ⚠️
Additional details and impacted files 
@@             Coverage Diff              @@
##               main    #3220      +/-   ##
============================================
+ Coverage     50.84%   50.86%   +0.01%     
- Complexity     5133     5511     +378     
============================================
  Files           976      976              
  Lines         60099    60102       +3     
  Branches       6574     6575       +1     
============================================
+ Hits          30557    30568      +11     
+ Misses        27390    27382       -8     
  Partials       2152     2152
Components Coverage Δ
spanner-templates 71.02% <ø> (+0.04%) ⬆️
spanner-import-export 68.98% <ø> (+0.11%) ⬆️
spanner-live-forward-migration 80.12% <ø> (ø)
spanner-live-reverse-replication 77.77% <ø> (ø)
spanner-bulk-migration 88.44% <ø> (ø)
Files with missing lines Coverage Δ
...beam/it/gcp/dataflow/AbstractPipelineLauncher.java 37.98% <25.00%> (-0.08%) ⬇️

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Abacn
Copy link
Copy Markdown
Contributor Author

Abacn commented Jan 14, 2026

raw logs for Kafka PR integration test reduced from 49 MB to 1.7 MB

@Abacn Abacn marked this pull request as ready for review January 14, 2026 18:47
@Abacn
Copy link
Copy Markdown
Contributor Author

Abacn commented Jan 14, 2026

R: @derrickaw

@Abacn Abacn merged commit 200724c into GoogleCloudPlatform:main Jan 14, 2026
49 of 50 checks passed
@Abacn Abacn deleted the logspanm branch January 14, 2026 19:32
MnkyGns pushed a commit to MnkyGns/DataflowTemplates that referenced this pull request Feb 12, 2026
@Abacn @MnkyGns
Fix a few log spams (GoogleCloudPlatform#3220)
61d6092
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@derrickaw derrickaw derrickaw approved these changes

Assignees

No one assigned

Labels

ignore-for-release size/S

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Bug]: Kafka integration test log spam

2 participants

@Abacn @derrickaw