Skip to content

Fix PostCommit Python job#35709

Merged
damccorm merged 3 commits intomasterfrom
fix-postcommit-python
Jul 30, 2025
Merged

Fix PostCommit Python job#35709
damccorm merged 3 commits intomasterfrom
fix-postcommit-python

Conversation

@Amar3tto
Copy link
Copy Markdown
Collaborator

Fixes #30513
Successful tests: https://github.com/apache/beam/actions/runs/16547689442/job/46814158621


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary of Changes

Hello @Amar3tto, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses an issue with a PostCommit Python job by implementing two distinct fixes. One fix enhances the robustness of an inference example's post-processing logic to accommodate diverse model output formats, preventing potential decoding failures. The second fix resolves a type mismatch in the Debezium I/O connector, ensuring the port parameter is correctly passed as a string.

Highlights

  • Python Inference Example Robustness: Improved the PostProcessor in gemini_text_classification.py to gracefully handle different inference result structures. This includes a direct content extraction path for specific formats and a fallback for the original 'Input: Output:' format, along with an explicit error message for undecodable inference elements, enhancing the example's resilience to varied model outputs.
  • Debezium I/O Connector Port Type Fix: Corrected a type issue in debezium.py by explicitly casting the port parameter to a string (str(port)) during the initialization of the Debezium I/O connector. This ensures compatibility with the underlying connector's requirements, which likely expects the port as a string.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in issue comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request fixes a PostCommit Python job. The changes include:

  1. In gemini_text_classification.py, the PostProcessor is updated to handle different structures of inference results. A critical issue was identified where the code could crash due to unsafe access to the inference object, and a suggestion has been provided to make the logic more robust by adding necessary checks.
  2. In debezium.py, the port argument is explicitly cast to a string to match the expected type in ReadFromDebeziumSchema.

@Amar3tto Amar3tto requested a review from damccorm July 28, 2025 15:30
inference = getattr(element, "inference", None)

if not isinstance(inference, (tuple, list)) or len(inference) < 2:
yield "Can't decode inference for element: " + str(element.example)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the shape of the returned object actually different? Can we update the example to correctly extract the data here instead of just ignoring data?

Copy link
Copy Markdown
Collaborator Author

@Amar3tto Amar3tto Jul 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's a rare case, so I added these checks to prevent failing.
Usually the original logic should work

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, it would be better to rework this, I was going to do it in another PR

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets either rework it in this PR or do something simpler here with a basic try/catch. Ideally, we'd just fix it here though, as it stands this isn't a great example

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@github-actions
Copy link
Copy Markdown
Contributor

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

@Amar3tto Amar3tto requested a review from damccorm July 29, 2025 15:22
Copy link
Copy Markdown
Contributor

@damccorm damccorm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@damccorm damccorm merged commit 5dad7a6 into master Jul 30, 2025
84 of 90 checks passed
@damccorm damccorm deleted the fix-postcommit-python branch July 30, 2025 16:40
Amar3tto added a commit that referenced this pull request Jul 31, 2025
* Fix debezium port type, fix gemini post processor fn

* Add condition

* Use try/catch

(cherry picked from commit 5dad7a6)
Amar3tto added a commit that referenced this pull request Jul 31, 2025
[Cherrypick] Fix PostCommit Python job (#35709)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The PostCommit Python job is flaky

2 participants