test: use dynamic projects and smee for GitLab E2E#2524
Conversation
🔍 PR Lint Feedback
|
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request enhances the GitLab E2E tests by introducing dynamic project creation and webhook routing through gosmee/smee. This resolves issues with parallel test execution and improves reliability in Kind-based CI environments. Additionally, it includes a cleanup script for stale projects and fixes title assertions in several tests. Highlights
Ignored Files
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Pull request overview
Refactors the GitLab end-to-end (E2E) test harness to avoid shared/static GitLab projects and to support Kind-based CI by creating per-run projects and routing webhooks through smee/gosmee (matching the existing Gitea pattern).
Changes:
- Introduce a unified GitLab E2E lifecycle helper (
TestMR) that creates a fresh GitLab project + webhook, creates the Repository CRD, optionally opens an MR, and returns a cleanup function. - Replace the static
TEST_GITLAB_PROJECT_IDsetup with dynamic project creation under a group (TEST_GITLAB_GROUP) and a smee URL (TEST_GITLAB_SMEEURL), including YAML config support and CI workflow updates to generate per-run smee channels. - Add a GitLab project garbage-collector script and update documentation/examples for the new GitLab E2E requirements.
Reviewed changes
Copilot reviewed 17 out of 17 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
test/pkg/options/options.go |
Removes the now-obsolete static GitLab ProjectID from E2E options. |
test/pkg/gitlab/test.go |
Adds TestOpts + TestMR lifecycle helper to standardize GitLab E2E setup/teardown. |
test/pkg/gitlab/setup.go |
Updates required env vars and teardown logic for dynamic GitLab projects. |
test/pkg/gitlab/scm.go |
Adds CreateGitLabProject helper to create projects and attach webhooks. |
test/pkg/gitlab/crd.go |
Refactors GitLab CRD creation to consume TestOpts (dynamic URL/settings/incomings). |
test/pkg/configfile/config.go |
Extends YAML-based config to support GitLab group and smee_url. |
test/pkg/configfile/config_test.go |
Updates config tests and expected env vars for the new GitLab config shape. |
test/gitlab_push_gitops_command_test.go |
Migrates GitLab GitOps push/tag tests to use the new TestMR lifecycle. |
test/gitlab_oktotest_thread_reply_test.go |
Migrates thread-reply MR test to TestMR. |
test/gitlab_merge_request_test.go |
Migrates GitLab MR test suite to TestMR and updates assertions/flows accordingly. |
test/gitlab_incoming_webhook_test.go |
Migrates incoming webhook tests to TestMR and uses dynamic namespaces/projects. |
test/gitlab_delete_tag_event_test.go |
Migrates delete-tag test to TestMR. |
test/e2e-config.yaml.example |
Updates example E2E config to use GitLab group + smee URL (no static project ID). |
test/README.md |
Documents manual GitLab E2E setup with smee/gosmee and cleanup guidance. |
hack/gh-workflow-ci.sh |
Sets GitLab group in CI and scrubs GitLab smee URL in collected logs. |
hack/cleanup-gitlab-projects.py |
Adds a script to garbage-collect stale GitLab test projects. |
.github/workflows/e2e.yaml |
Generates a unique smee channel for GitLab E2E and runs gosmee to forward webhooks. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Code Review
This pull request significantly refactors the GitLab E2E tests to use dynamic project creation and gosmee for webhook forwarding, aligning them with the existing Gitea test patterns. This change addresses previous issues with static project IDs and direct webhook delivery, which caused conflicts and unreliable results in CI environments. The introduction of a unified TestMR helper function and TestOpts struct centralizes test setup and teardown logic, greatly improving maintainability and reducing code duplication across individual GitLab tests. Documentation has been updated to reflect these changes, including instructions for manual GitLab test setup and a new cleanup script for stale projects. Overall, this is a well-executed refactoring that enhances the robustness and reliability of the GitLab E2E test suite.
Note: Security Review is unavailable for this PR.
9cc79d1 to
3b7c0c4
Compare
GitLab E2E tests previously relied on a static pre-created project
(TEST_GITLAB_PROJECT_ID) and direct webhook delivery to the controller
URL. This broke in Kind-based CI environments where the controller runs
inside the cluster and is not reachable from the external GitLab
instance (gitlab.pipelinesascode.com).
This was causing issue when multiple e2e runs were executed in parallel,
as they would all share the same project and webhook, leading to
conflicts and unreliable test results when testing comments, commit
statuses, and merge request events.
Refactor GitLab E2E tests to dynamically create and delete projects per
test run, and route webhooks through gosmee/smee — the same pattern
already used by Gitea tests. Each test now creates a fresh GitLab
project with a webhook pointing to a smee channel, and gosmee forwards
payloads to the in-cluster controller.
Key changes:
- CreateGitLabProject() now accepts hookURL and webhookSecret as
parameters instead of reading TEST_EL_URL from the environment,
mirroring CreateGiteaRepo() in test/pkg/gitea/scm.go.
- TestMR() reads TEST_GITLAB_SMEEURL and passes it as the webhook
target when creating projects. TearDown() deletes the project on
cleanup.
- CI workflow generates a unique smee channel for the gitlab_bitbucket
matrix job and runs a gosmee client to forward webhooks, matching
the existing Gitea gosmee setup.
- Add SmeeURL field to GitLabConfig in the YAML-based test
configuration, with corresponding test coverage.
- Add hack/cleanup-gitlab-projects.py to garbage-collect stale test
projects older than 7 days (dry-run by default, --force to delete).
- Fix title assertions in TestGitlabIssueGitopsComment,
TestGitlabConsistentCommitStatusOnMR, and
TestGitlabMergeRequestCelPrefix. These tests expected custom titles
but the repository status always reflects the commit message set by
TestMR ("Committing files from test on ..."). The mismatch was
latent in the refactoring and surfaced when running the full suite.
- Document manual GitLab test setup in test/README.md including smee
channel generation, required env vars, and project cleanup.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Chmouel Boudjnah <chmouel@redhat.com>
3b7c0c4 to
8c396ea
Compare
GitLab E2E tests previously relied on a static pre-created project (TEST_GITLAB_PROJECT_ID) and direct webhook delivery to the controller URL. This broke in Kind-based CI environments where the controller runs inside the cluster and is not reachable from the external GitLab instance (gitlab.pipelinesascode.com).
This was causing issue when multiple e2e runs were executed in parallel, as they would all share the same project and webhook, leading to conflicts and unreliable test results when testing comments, commit statuses, and merge request events.
Refactor GitLab E2E tests to dynamically create and delete projects per test run, and route webhooks through gosmee/smee — the same pattern already used by Gitea tests. Each test now creates a fresh GitLab project with a webhook pointing to a smee channel, and gosmee forwards payloads to the in-cluster controller.
Key changes:
CreateGitLabProject() now accepts hookURL and webhookSecret as parameters instead of reading TEST_EL_URL from the environment, mirroring CreateGiteaRepo() in test/pkg/gitea/scm.go.
TestMR() reads TEST_GITLAB_SMEEURL and passes it as the webhook target when creating projects. TearDown() deletes the project on cleanup.
CI workflow generates a unique smee channel for the gitlab_bitbucket matrix job and runs a gosmee client to forward webhooks, matching the existing Gitea gosmee setup.
Add SmeeURL field to GitLabConfig in the YAML-based test configuration, with corresponding test coverage.
Add hack/cleanup-gitlab-projects.py to garbage-collect stale test projects older than 7 days (dry-run by default, --force to delete).
Fix title assertions in TestGitlabIssueGitopsComment, TestGitlabConsistentCommitStatusOnMR, and TestGitlabMergeRequestCelPrefix. These tests expected custom titles but the repository status always reflects the commit message set by TestMR ("Committing files from test on ..."). The mismatch was latent in the refactoring and surfaced when running the full suite.
Document manual GitLab test setup in test/README.md including smee channel generation, required env vars, and project cleanup.
📝 Description of the Change
👨🏻 Linked Jira
🔗 Linked GitHub Issue
Fixes #
🚀 Type of Change
fix:)feat:)feat!:,fix!:)docs:)style:)refactor:)perf:)test:)build:)ci:)chore:)revert:)deps:)🧪 Testing Strategy
🤖 AI Assistance
If you have used AI assistance, please provide the following details:
Which LLM was used?
Extent of AI Assistance:
Important
If the majority of the code in this PR was generated by an AI, please add a
Co-authored-bytrailer to your commit message.For example:
Co-authored-by: Gemini gemini@google.com
Co-authored-by: ChatGPT noreply@chatgpt.com
Co-authored-by: Claude noreply@anthropic.com
Co-authored-by: Cursor noreply@cursor.com
Co-authored-by: Copilot Copilot@users.noreply.github.com
**💡You can use the script
./hack/add-llm-coauthor.shto automatically addthese co-author trailers to your commits.
✅ Submitter Checklist
fix:,feat:) matches the "Type of Change" I selected above.make testandmake lintlocally to check for and fix any issues. For an efficient workflow, I have considered installing pre-commit and runningpre-commit installto automate these checks.