feat: Implement GraphQL batch fetching for .tekton files by chmouel · Pull Request #2423 · tektoncd/pipelines-as-code

chmouel · 2026-01-28T17:30:37Z

This commit introduces the use of GitHub's GraphQL API for fetching multiple .tekton directory files simultaneously. Previously, each file was fetched individually using REST API calls.

The new implementation leverages GraphQL batching to retrieve all necessary YAML files in a single request, significantly reducing the number of API calls and improving performance. This change also necessitates new test cases to cover the GraphQL client functionality and ensures compatibility with GitHub Enterprise instances.

📝 Description of the Change

👨🏻‍ Linked Jira

https://redhat.atlassian.net/browse/SRVKP-11470

🔗 Linked GitHub Issue

Fixes #

🚀 Type of Change

🧪 Testing Strategy

🤖 AI Assistance

I have not used any AI assistance for this PR.
I have used AI assistance for this PR.

If you have used AI assistance, please provide the following details:

Which LLM was used?

Extent of AI Assistance:

Documentation and research only
Unit tests or E2E tests only
Code generation (parts of the code)
Full code generation (most of the PR)
PR description and comments
Commit message(s)

Important

If the majority of the code in this PR was generated by an AI, please add a Co-authored-by trailer to your commit message.
For example:

Co-authored-by: Gemini gemini@google.com
Co-authored-by: ChatGPT noreply@chatgpt.com
Co-authored-by: Claude noreply@anthropic.com
Co-authored-by: Cursor noreply@cursor.com
Co-authored-by: Copilot Copilot@users.noreply.github.com

**💡You can use the script ./hack/add-llm-coauthor.sh to automatically add
these co-author trailers to your commits.

✅ Submitter Checklist

gemini-code-assist · 2026-01-28T17:30:57Z

Summary of Changes

Hello @chmouel, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a significant optimization to how .tekton files are retrieved from GitHub repositories. By switching from numerous individual REST API calls to a single, batched GraphQL query, the system can now fetch all necessary YAML files much more efficiently. This change aims to enhance overall performance and reduce the load on the GitHub API, particularly for repositories with many .tekton configuration files.

Highlights

GraphQL Batch Fetching: Implemented GitHub's GraphQL API to batch fetch multiple .tekton directory files simultaneously, replacing individual REST API calls.
Performance Improvement: Significantly reduced the number of GitHub API calls required to retrieve .tekton files, leading to improved performance and efficiency.
New GraphQL Client: Introduced a new graphQLClient with logic for constructing GraphQL queries, handling requests, and dynamically determining the correct GraphQL endpoint for both public GitHub and GitHub Enterprise instances.
Enhanced Test Coverage: Added comprehensive unit tests for the new GraphQL client functionality, including tests for batch fetching, error handling, and endpoint construction. Existing tests were updated to reflect the reduced API call counts.
Test Infrastructure Update: Modified the test helper SetupGitTree to include a mock GraphQL API handler, enabling realistic testing of the new GraphQL fetching mechanism.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

pipelines-as-code · 2026-01-28T17:31:00Z

🔍 PR Lint Feedback

Note: This automated check helps ensure your PR follows our contribution guidelines.

⚠️ Items that need attention:

🎫 Jira reference

Add a Jira reference in the description using one of the following formats:

https://issues.redhat.com/browse/SRVKP-<number>

If no SRVKP ticket exists yet, link a GitHub issue instead (e.g., Fixes #123).
Minor housekeeping PRs without Jira coverage can skip this after confirming with reviewers.

🤖 AI attribution

The following commits lack an explicit AI attribution footer:

434d449 feat: Implement GraphQL batch fetching for .tekton files

If no AI assistance was used for a commit, you can ignore this warning.
Otherwise add an Assisted-by: or Co-authored-by: footer referencing the AI used.

ℹ️ Next Steps

Review and address the items above
Push new commits to update this PR
This comment will be automatically updated when issues are resolved

🔧 Admin Tools (click to expand)

Automated Issue/Ticket Creation:

/issue-create - Generate a GitHub issue from this PR content using AI
/jira-create - Create a SRVKP Jira ticket from this PR content using AI

⚠️ Important: Always review and edit generated content before finalizing tickets/issues.
The AI-generated content should be used as a starting point and may need adjustments.

These commands are available to maintainers and will post the generated content as PR comments for review.

_{🤖 This feedback was generated automatically by the PR CI system}

chmouel · 2026-01-28T17:33:11Z

using a library for graphql library was actually more code than doing json dump and string, i tried to make it as compact as possible after running via LLM

gemini-code-assist

Code Review

This pull request introduces a significant performance improvement by replacing multiple REST API calls with a single batched GraphQL query to fetch .tekton files. However, a potential GraphQL injection vulnerability was identified in the construction of the GraphQL query where the ref variable is not properly escaped. This should be addressed by ensuring all variables inserted into the query string are correctly escaped for GraphQL string literals. Additionally, a critical bug in a test mock prevents compilation, and some Go 1.22-specific syntax in tests might affect compatibility with older Go versions.

pkg/test/github/github.go

pkg/provider/github/graphql.go

pkg/provider/github/graphql_test.go

chmouel · 2026-01-28T17:32:11Z

pkg/test/github/github.go

+		assert.NilError(t, err)
+
+		// Register handler once with all collected files (only if we have files)
+		if len(allFiles) > 0 {


this is complicated just to get tests done but i am not sure how to simplify it

Copilot

Pull request overview

This PR implements GraphQL batch fetching for .tekton directory files to replace individual REST API calls, reducing API request count and improving performance. The change introduces a new GraphQL client that fetches multiple YAML files in a single request using GitHub's GraphQL API with query aliases.

Changes:

Introduced GraphQL batch fetching mechanism that reduces API calls from N+2 to 3 (where N is the number of YAML files)
Added comprehensive GraphQL client with batching support (max 50 files per batch)
Updated test infrastructure to mock GraphQL endpoints and validate the new functionality

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
pkg/provider/github/graphql.go	New GraphQL client implementation with endpoint construction, query building, and batch fetching logic
pkg/provider/github/github.go	Modified `concatAllYamlFiles` to use GraphQL batch fetching instead of individual REST API calls
pkg/provider/github/graphql_test.go	Unit tests for GraphQL endpoint construction, query building, and batch fetching
pkg/provider/github/github_test.go	Updated existing tests to reflect reduced API call counts and added GraphQL-specific integration tests
pkg/test/github/github.go	Enhanced test helper with GraphQL endpoint mocking and query parsing logic

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

pkg/provider/github/graphql.go

pkg/test/github/github.go

pkg/provider/github/github.go

pkg/test/github/github.go

pkg/provider/github/github_test.go

pkg/test/github/github.go

pkg/provider/github/graphql.go

chmouel · 2026-03-09T10:35:57Z

Follow-up since last review

Fixed GraphQL request construction to go through go-github's request builder, which preserves the standard GitHub headers (User-Agent, X-GitHub-Api-Version, etc.) on batch requests.
Fixed default_branch batch reads so they resolve the branch head once and reuse that exact commit SHA for both tree listing and the subsequent GraphQL file fetch. That keeps the directory listing and file contents pinned to the same revision.

pkg/provider/github/github_test.go

pkg/provider/github/github.go

zakisk · 2026-03-25T06:43:30Z

we you take look on above comment we can final this

chmouel · 2026-03-31T07:24:20Z

Thanks @theakshaypant for catching the redundant logging and the rate limit header suggestion. Both have been implemented.

chmouel · 2026-03-31T07:24:26Z

@zakisk

I've addressed your concerns:

Test consistency: Consolidated the 3 GraphQL test functions into a single table-driven test (TestGetTektonDirGraphQL) following project patterns. No more underscores in test names.
GraphQL client storage: Implemented it. The client is now cached in the Provider struct and reused across calls, ready for future GraphQL operations.
Rate limit visibility: Added rate limit headers reading and logging from GraphQL responses.

The metrics remain as a request counter (consistent with other providers), but the rate limit headers are logged separately to show actual consumption.

chmouel · 2026-03-31T10:35:21Z

I want to merge this after we do a release so we can dogfood it properly so let's wait a bit

This commit introduces the use of GitHub's GraphQL API for fetching multiple `.tekton` directory files simultaneously. Previously, each file was fetched individually using REST API calls. The new implementation leverages GraphQL batching to retrieve all necessary YAML files in a single request, significantly reducing the number of API calls and improving performance. This change also necessitates new test cases to cover the GraphQL client functionality and ensures compatibility with GitHub Enterprise instances. Signed-off-by: Chmouel Boudjnah <chmouel@redhat.com>

chmouel · 2026-04-01T13:58:04Z

/retest

chmouel · 2026-04-01T13:58:39Z

/test

chmouel · 2026-04-01T14:04:04Z

/test go-testing

Critical bug fix: Merged PRs were being incorrectly detected as "closed", causing tickets to transition to "In Progress" instead of "On QA". Problem: - pr.GetMerged() boolean is not reliably populated when listing PRs - Merged PRs have state="closed", so they were caught by the closed check - Example: tektoncd/pipelines-as-code#2423 detected as "closed" not "merged" Solution: - Check pr.MergedAt timestamp first (always set for merged PRs) - Only treat as closed if state="closed" AND MergedAt is nil/zero - More reliable detection using timestamp instead of boolean field Impact: - Merged PRs now correctly transition tickets to "On QA" - Closed (unmerged) PRs still correctly transition to "In Progress" Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

pipelines-as-code bot added feature New feature or request github optimization labels Jan 28, 2026

gemini-code-assist bot reviewed Jan 28, 2026

View reviewed changes

chmouel force-pushed the graphql branch 2 times, most recently from b9946cd to dd0fa67 Compare January 29, 2026 09:11

chmouel commented Feb 3, 2026

View reviewed changes

chmouel force-pushed the graphql branch 2 times, most recently from dd0fa67 to f891612 Compare February 16, 2026 15:07

chmouel marked this pull request as ready for review February 17, 2026 19:17

Copilot AI review requested due to automatic review settings February 17, 2026 19:17

Copilot started reviewing on behalf of chmouel February 17, 2026 19:18 View session

Copilot AI reviewed Feb 17, 2026

View reviewed changes

zakisk reviewed Feb 19, 2026

View reviewed changes

pkg/provider/github/graphql.go Show resolved Hide resolved

chmouel force-pushed the graphql branch 5 times, most recently from 9be2762 to d0e3668 Compare February 26, 2026 10:28

chmouel linked an issue Feb 26, 2026 that may be closed by this pull request

Optimise downloading multiple .tekton files with GraphQL #2403

Closed

chmouel mentioned this pull request Feb 26, 2026

Optimise downloading multiple .tekton files with GraphQL #2403

Closed

chmouel force-pushed the graphql branch from 64170d5 to d0e3668 Compare March 3, 2026 22:11

theakshaypant reviewed Mar 4, 2026

View reviewed changes

pkg/provider/github/graphql.go Outdated Show resolved Hide resolved

pkg/provider/github/graphql.go Show resolved Hide resolved

chmouel force-pushed the graphql branch from 21630dc to 1b12a64 Compare March 9, 2026 10:03

chmouel force-pushed the graphql branch from 1b12a64 to 434d449 Compare March 9, 2026 11:26

zakisk reviewed Mar 10, 2026

View reviewed changes

pkg/provider/github/github_test.go Outdated Show resolved Hide resolved

zakisk reviewed Mar 10, 2026

View reviewed changes

pkg/provider/github/github.go Outdated Show resolved Hide resolved

tekton-robot added the needs-rebase label Mar 25, 2026

chmouel force-pushed the graphql branch from 434d449 to 5d357b2 Compare March 31, 2026 06:59

tekton-robot removed the needs-rebase label Mar 31, 2026

chmouel force-pushed the graphql branch from 5d357b2 to 96cea5c Compare March 31, 2026 07:32

zakisk approved these changes Mar 31, 2026

View reviewed changes

chmouel force-pushed the graphql branch from f98db7e to 69faa7a Compare April 1, 2026 12:05

Merge branch 'main' into graphql

3fea3db

chmouel merged commit b8ed140 into tektoncd:main Apr 1, 2026
13 checks passed

chmouel deleted the graphql branch April 1, 2026 14:18

divyansh42 mentioned this pull request Apr 2, 2026

fix(github): detect merged PRs using MergedAt field openshift-pipelines/skipjira#9

Merged

Conversation

chmouel commented Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📝 Description of the Change

👨🏻‍ Linked Jira

🔗 Linked GitHub Issue

🚀 Type of Change

🧪 Testing Strategy

🤖 AI Assistance

✅ Submitter Checklist

Uh oh!

gemini-code-assist bot commented Jan 28, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

pipelines-as-code bot commented Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔍 PR Lint Feedback

⚠️ Items that need attention:

🎫 Jira reference

🤖 AI attribution

ℹ️ Next Steps

Uh oh!

chmouel commented Jan 28, 2026

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chmouel Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chmouel commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zakisk commented Mar 25, 2026

Uh oh!

chmouel commented Mar 31, 2026

Uh oh!

chmouel commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chmouel commented Mar 31, 2026

Uh oh!

chmouel commented Apr 1, 2026

Uh oh!

chmouel commented Apr 1, 2026

Uh oh!

chmouel commented Apr 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

chmouel commented Jan 28, 2026 •

edited

Loading

pipelines-as-code bot commented Jan 28, 2026 •

edited

Loading

chmouel commented Mar 9, 2026 •

edited

Loading

chmouel commented Mar 31, 2026 •

edited

Loading