Skip to content

test: split github_ghe job into 3 parallel chunks#2517

Merged
theakshaypant merged 1 commit intotektoncd:mainfrom
zakisk:break-ghe-job-into-three
Mar 2, 2026
Merged

test: split github_ghe job into 3 parallel chunks#2517
theakshaypant merged 1 commit intotektoncd:mainfrom
zakisk:break-ghe-job-into-three

Conversation

@zakisk
Copy link
Copy Markdown
Member

@zakisk zakisk commented Mar 2, 2026

Split the single github_ghe CI matrix entry into github_ghe_1, github_ghe_2, and github_ghe_3 to run GHE E2E tests in parallel, matching the existing gitea chunking pattern. The old github_ghe target is kept as a backward compat alias.

Assisted-by: Claude Opus 4.6 (via Claude Code)

📝 Description of the Change

👨🏻‍ Linked Jira

🔗 Linked GitHub Issue

Fixes #

🚀 Type of Change

  • 🐛 Bug fix (fix:)
  • ✨ New feature (feat:)
  • 💥 Breaking change (feat!:, fix!:)
  • 📚 Documentation update (docs:)
  • ⚙️ Chore (chore:)
  • 💅 Refactor (refactor:)
  • 🔧 Enhancement (enhance:)
  • 📦 Dependency update (deps:)

🧪 Testing Strategy

  • Unit tests
  • Integration tests
  • End-to-end tests
  • Manual testing
  • Not Applicable

🤖 AI Assistance

  • I have not used any AI assistance for this PR.
  • I have used AI assistance for this PR.

If you have used AI assistance, please provide the following details:

Which LLM was used?

  • GitHub Copilot
  • ChatGPT (OpenAI)
  • Claude (Anthropic)
  • Cursor
  • Gemini (Google)
  • Other: ____________

Extent of AI Assistance:

  • Documentation and research only
  • Unit tests or E2E tests only
  • Code generation (parts of the code)
  • Full code generation (most of the PR)
  • PR description and comments
  • Commit message(s)

Important

If the majority of the code in this PR was generated by an AI, please add a Co-authored-by trailer to your commit message.
For example:

Co-authored-by: Gemini gemini@google.com
Co-authored-by: ChatGPT noreply@chatgpt.com
Co-authored-by: Claude noreply@anthropic.com
Co-authored-by: Cursor noreply@cursor.com
Co-authored-by: Copilot Copilot@users.noreply.github.com

**💡You can use the script ./hack/add-llm-coauthor.sh to automatically add
these co-author trailers to your commits.

✅ Submitter Checklist

  • 📝 My commit messages are clear, informative, and follow the project's How to write a git commit message guide. The Gitlint linter ensures in CI it's properly validated
  • ✨ I have ensured my commit message prefix (e.g., fix:, feat:) matches the "Type of Change" I selected above.
  • ♽ I have run make test and make lint locally to check for and fix any
    issues. For an efficient workflow, I have considered installing
    pre-commit and running pre-commit install to
    automate these checks.
  • 📖 I have added or updated documentation for any user-facing changes.
  • 🧪 I have added sufficient unit tests for my code changes.
  • 🎁 I have added end-to-end tests where feasible. See README for more details.
  • 🔎 I have addressed any CI test flakiness or provided a clear reason to bypass it.
  • If adding a provider feature, I have filled in the following and updated the provider documentation:
    • GitHub App
    • GitHub Webhook
    • Gitea/Forgejo
    • GitLab
    • Bitbucket Cloud
    • Bitbucket Data Center

Split the single github_ghe CI matrix entry into github_ghe_1,
github_ghe_2, and github_ghe_3 to run GHE E2E tests in parallel,
matching the existing gitea chunking pattern. The old github_ghe
target is kept as a backward compat alias.

Signed-off-by: Zaki Shaikh <zashaikh@redhat.com>
Assisted-by: Claude Opus 4.6 (via Claude Code)
@zakisk zakisk requested a review from chmouel March 2, 2026 07:40
@gemini-code-assist
Copy link
Copy Markdown

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the efficiency of the CI pipeline by introducing parallel execution for GitHub Enterprise end-to-end tests. By dividing the previously monolithic 'github_ghe' job into three concurrent segments, the overall feedback loop for GHE-related changes will be substantially reduced, leading to faster development cycles and improved resource utilization within the CI system.

Highlights

  • Parallelized GHE E2E Tests: The single 'github_ghe' CI job has been split into three parallel jobs ('github_ghe_1', 'github_ghe_2', 'github_ghe_3') to accelerate GitHub Enterprise end-to-end test execution.
  • Backward Compatibility: The original 'github_ghe' target is retained as an alias to ensure compatibility with existing workflows, alongside 'github_second_controller'.
  • Test Chunking Logic: New logic was introduced to calculate and distribute GHE tests into three distinct chunks, mirroring the existing Gitea test parallelization pattern.
Changelog
  • hack/gh-workflow-ci.sh
    • Introduced a new array 'ghe_tests' to specifically filter and store GitHub Enterprise E2E tests.
    • Modified the conditional check for 'github_tests' to correctly exclude the new 'github_ghe*' targets.
    • Added logic to calculate 'ghe_chunk_size' and 'ghe_remainder' for distributing GHE tests into three parts.
    • Updated the 'case' statement to handle the new 'github_ghe_1', 'github_ghe_2', and 'github_ghe_3' targets, each printing its respective chunk of GHE tests.
    • Modified the 'github_second_controller' case to also act as a backward-compatible alias for 'github_ghe', printing all GHE tests.
    • Updated the list of supported targets and backward compatibility aliases in the error message.
    • Modified the 'print_tests' loop to iterate over the new 'github_ghe_1', 'github_ghe_2', 'github_ghe_3' targets.
Ignored Files
  • Ignored by pattern: .github/workflows/** (1)
    • .github/workflows/e2e.yaml
Activity
  • No human activity has occurred on this pull request yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@pipelines-as-code
Copy link
Copy Markdown

🔍 PR Lint Feedback

Note: This automated check helps ensure your PR follows our contribution guidelines.

⚠️ Items that need attention:

🎫 Jira reference

Add a Jira reference in the description using one of the following formats:

  • https://issues.redhat.com/browse/SRVKP-<number>

If no SRVKP ticket exists yet, link a GitHub issue instead (e.g., Fixes #123).
Minor housekeeping PRs without Jira coverage can skip this after confirming with reviewers.


ℹ️ Next Steps

  • Review and address the items above
  • Push new commits to update this PR
  • This comment will be automatically updated when issues are resolved
🔧 Admin Tools (click to expand)

Automated Issue/Ticket Creation:

  • /issue-create - Generate a GitHub issue from this PR content using AI
  • /jira-create - Create a SRVKP Jira ticket from this PR content using AI

⚠️ Important: Always review and edit generated content before finalizing tickets/issues.
The AI-generated content should be used as a starting point and may need adjustments.

These commands are available to maintainers and will post the generated content as PR comments for review.

🤖 This feedback was generated automatically by the PR CI system

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request splits the github_ghe E2E tests into three parallel chunks, which is a good improvement for CI performance. The implementation follows the existing pattern for gitea tests. I've left a few comments with suggestions to improve consistency and fix a potential bug related to how test lists are generated. Overall, the changes look good.


local -a ghe_tests=()
if [[ "${target}" == github_ghe* ]]; then
mapfile -t ghe_tests < <(echo "${all_tests}" | grep -iP 'GithubGHE' 2>/dev/null | grep -ivP 'Concurrency' 2>/dev/null | sort 2>/dev/null)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

With set -o pipefail active, this line will cause the script to exit if grep doesn't find any matching tests, because grep will exit with status 1. You should handle this case to avoid unexpected script failures. Appending || true after the grep pipeline is a common way to solve this.

This issue is pre-existing and also affects the creation of gitea_tests and github_tests.

Suggested change
mapfile -t ghe_tests < <(echo "${all_tests}" | grep -iP 'GithubGHE' 2>/dev/null | grep -ivP 'Concurrency' 2>/dev/null | sort 2>/dev/null)
mapfile -t ghe_tests < <(echo "${all_tests}" | grep -iP 'GithubGHE' 2>/dev/null | grep -ivP 'Concurrency' 2>/dev/null || true | sort 2>/dev/null)

fi

local -a ghe_tests=()
if [[ "${target}" == github_ghe* ]]; then
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The ghe_tests array should also be populated for the github_second_controller target, as it's an alias for running GHE tests. This will allow you to reuse the ghe_tests array for the alias case, improving consistency and avoiding redundant computations. See my other comment on how to use this.

Suggested change
if [[ "${target}" == github_ghe* ]]; then
if [[ "${target}" == github_ghe* || "${target}" == "github_second_controller" ]]; then

# pull_request_target runs the workflow YAML from main which still sends old
# target names.
github_second_controller | github_ghe)
printf '%s\n' "${all_tests}" | grep -iP 'GithubGHE' | grep -ivP 'Concurrency'
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

After modifying the condition to populate ghe_tests for github_second_controller (as per my other suggestion), you can simplify this to use the ghe_tests array directly. This avoids re-running grep and ensures the output is sorted, which is consistent with the new chunked targets.

Suggested change
printf '%s\n' "${all_tests}" | grep -iP 'GithubGHE' | grep -ivP 'Concurrency'
printf '%s\n' "${ghe_tests[@]}"

@theakshaypant theakshaypant merged commit c057f26 into tektoncd:main Mar 2, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants