Skip to content

Upgrade ingress-nginx in E2E tests#7792

Merged
cert-manager-prow[bot] merged 2 commits intocert-manager:masterfrom
wallrj:ingress-nginx-exact-path
Jun 11, 2025
Merged

Upgrade ingress-nginx in E2E tests#7792
cert-manager-prow[bot] merged 2 commits intocert-manager:masterfrom
wallrj:ingress-nginx-exact-path

Conversation

@wallrj
Copy link
Copy Markdown
Member

@wallrj wallrj commented Jun 10, 2025

Upgrade ingress-nginx to allow us to recreate #7791 in our E2E tests.

ingress-nginx v4.12 enabled strict-validate-path-type by default:

/kind cleanup

Use the latest version of `ingress-nginx` in E2E tests to ensure compatibility

@cert-manager-prow cert-manager-prow bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note-none Denotes a PR that doesn't merit a release note. kind/bug Categorizes issue or PR as related to a bug. dco-signoff: yes Indicates that all commits in the pull request have the valid DCO sign-off message. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jun 10, 2025
@rouke-broersma
Copy link
Copy Markdown
Contributor

You need to have config.strict-validate-path-type set to true for paths to be denied by admission I think

@wallrj
Copy link
Copy Markdown
Member Author

wallrj commented Jun 10, 2025

You need to have config.strict-validate-path-type set to true for paths to be denied by admission I think

config.strict-validate-path-type was enabled by default in 4.12:

cert-manager E2E tests still use 4.10:

@wallrj
Copy link
Copy Markdown
Member Author

wallrj commented Jun 10, 2025

Upgraded to ingress-nginx 4.12 and now I'm seeing the error:

  • cert-manager controller

E0610 20:30:59.371117 1 controller.go:157] "re-queuing item due to error processing" err="admission webhook "validate.nginx.ingress.kubernetes.io" denied the request: ingress contains invalid paths: path /.well-known/acme-challenge/9Ea_73wVjeFLyLPgxdOz8Y6hq0rEboLLNwUGWoreD-g cannot be used with pathType Exact" logger="cert-manager.controller"

  • ingress-nginx controller

E0610 20:31:39.376497 11 main.go:96] "invalid ingress configuration" err="ingress contains invalid paths: path /.well-known/acme-challenge/9Ea_73wVjeFLyLPgxdOz8Y6hq0rEboLLNwUGWoreD-g cannot be used with pathType Exact" ingress="e2e-tests-certificates-qfdff/cm-acme-http-solver-7gl2p"

@wallrj wallrj changed the title WIP: Enable ingress-nginx admission webhook WIP: Upgrade ingress-nginx in E2E tests Jun 10, 2025
--set controller.service.type=ClusterIP \
--set controller.config.no-tls-redirect-locations= \
--set admissionWebhooks.enabled=false \
--set admissionWebhooks.enabled=true \
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This had been disabled years ago due to an incompatibility with an old version of Kubernetes:

IMAGE_projectcontour_amd64 := ghcr.io/projectcontour/contour:v1.29.1@sha256:bb7af851ac5832c315e0863d12ed583cee54c495d58a206f1d0897647505ed70

IMAGE_ingressnginx_arm64 := registry.k8s.io/ingress-nginx/controller:v1.10.1@sha256:624d1a22b56a52fc4b8e330bef968cd77d49c6eeb36166f20036d50782307341
IMAGE_ingressnginx_arm64 := registry.k8s.io/ingress-nginx/controller:v1.12.3@sha256:800048a4cdf4ad487a17f56d22ec6be7a34248fc18900d945bc869fee4ccb2f7
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

crane manifest registry.k8s.io/ingress-nginx/controller:v1.12.3
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:aadad8e26329d345dea3a69b8deb9f3c52899a97cbaf7e702b8dfbeae3082c15",
      "size": 3153,
      "platform": {
        "architecture": "amd64",
        "os": "linux"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:005b02efca0f909488b970e398f27258db2f293868e539530b4c9e5daecc4784",
      "size": 3153,
      "platform": {
        "architecture": "arm",
        "os": "linux",
        "variant": "v7"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:800048a4cdf4ad487a17f56d22ec6be7a34248fc18900d945bc869fee4ccb2f7",
      "size": 3153,
      "platform": {
        "architecture": "arm64",
        "os": "linux"
      }
    }
...
}

@cert-manager-prow cert-manager-prow bot added area/acme Indicates a PR directly modifies the ACME Issuer code area/acme/http01 Indicates a PR modifies ACME HTTP01 provider code size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jun 10, 2025
@wallrj
Copy link
Copy Markdown
Member Author

wallrj commented Jun 10, 2025

@wallrj: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-cert-manager-master-e2e-v1-33 ed9d60e link true /test pull-cert-manager-master-e2e-v1-33
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

As expected, the Ingress related E2E tests failed after I upgraded to ingress-nginx v4.12, which has strict-validate-path turned on by default:

image

Then after cherry-picking @sspreitzer 's patch from #7793 , the E2E tests once again passed.

@wallrj wallrj force-pushed the ingress-nginx-exact-path branch from ebadc35 to 0b4ea0f Compare June 10, 2025 21:50
@cert-manager-prow cert-manager-prow bot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. release-note-none Denotes a PR that doesn't merit a release note. labels Jun 10, 2025
@wallrj wallrj changed the title WIP: Upgrade ingress-nginx in E2E tests Upgrade ingress-nginx in E2E tests Jun 10, 2025
@cert-manager-prow cert-manager-prow bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 10, 2025
@inteon
Copy link
Copy Markdown
Member

inteon commented Jun 11, 2025

/approve
/lgtm

@cert-manager-prow cert-manager-prow bot added the lgtm Indicates that a PR is ready to be merged. label Jun 11, 2025
@cert-manager-prow
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: inteon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@cert-manager-prow cert-manager-prow bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 11, 2025
@cert-manager-prow cert-manager-prow bot merged commit b80ca21 into cert-manager:master Jun 11, 2025
6 checks passed
@wallrj
Copy link
Copy Markdown
Member Author

wallrj commented Jun 17, 2025

/cherry-pick release-1.18

@cert-manager-bot
Copy link
Copy Markdown
Contributor

@wallrj: new pull request created: #7807

Details

In response to this:

/cherry-pick release-1.18

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@wallrj
Copy link
Copy Markdown
Member Author

wallrj commented Jun 17, 2025

/cherry-pick release-1.17

@cert-manager-bot
Copy link
Copy Markdown
Contributor

@wallrj: new pull request created: #7808

Details

In response to this:

/cherry-pick release-1.17

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

alexlebens pushed a commit to alexlebens/infrastructure that referenced this pull request Oct 8, 2025
This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [cert-manager](https://cert-manager.io) ([source](https://github.com/cert-manager/cert-manager)) | minor | `v1.18.2` -> `v1.19.0` |

---

### Release Notes

<details>
<summary>cert-manager/cert-manager (cert-manager)</summary>

### [`v1.19.0`](https://github.com/cert-manager/cert-manager/releases/tag/v1.19.0)

[Compare Source](cert-manager/cert-manager@v1.18.2...v1.19.0)

cert-manager is the easiest way to automatically manage certificates in Kubernetes and OpenShift clusters.

This release focuses on expanding platform compatibility, improving deployment flexibility, enhancing observability, and addressing key reliability issues.

> 📖  Read the full release notes at cert-manager.io: <https://cert-manager.io/docs/releases/release-notes/release-notes-1.19>

Changes since `v1.18.0`:

#### Feature

- Add IPv6 rules to the default network policy ([#&#8203;7726](cert-manager/cert-manager#7726), [@&#8203;jcpunk](https://github.com/jcpunk))
- Add `global.nodeSelector` to helm chart to allow for a single `nodeSelector` to be set across all services. ([#&#8203;7818](cert-manager/cert-manager#7818), [@&#8203;StingRayZA](https://github.com/StingRayZA))
- Add a feature gate to default to Ingress `pathType` `Exact` in ACME HTTP01 Ingress challenge solvers. ([#&#8203;7795](cert-manager/cert-manager#7795), [@&#8203;sspreitzer](https://github.com/sspreitzer))
- Add generated `applyconfigurations` allowing clients to make type-safe server-side apply requests for cert-manager resources. ([#&#8203;7866](cert-manager/cert-manager#7866), [@&#8203;erikgb](https://github.com/erikgb))
- Added API defaults to issuer references group (cert-manager.io) and kind (Issuer). ([#&#8203;7414](cert-manager/cert-manager#7414), [@&#8203;erikgb](https://github.com/erikgb))
- Added `certmanager_certificate_challenge_status` Prometheus metric. ([#&#8203;7736](cert-manager/cert-manager#7736), [@&#8203;hjoshi123](https://github.com/hjoshi123))
- Added `protocol` field for `rfc2136` DNS01 provider ([#&#8203;7881](cert-manager/cert-manager#7881), [@&#8203;hjoshi123](https://github.com/hjoshi123))
- Added experimental field `hostUsers` flag to all pods. Not set by default. ([#&#8203;7973](cert-manager/cert-manager#7973), [@&#8203;hjoshi123](https://github.com/hjoshi123))
- Support configurable resource requests and limits for ACME HTTP01 solver pods through ClusterIssuer and Issuer specifications, allowing granular resource management that overrides global `--acme-http01-solver-resource-*` settings. ([#&#8203;7972](cert-manager/cert-manager#7972), [@&#8203;lunarwhite](https://github.com/lunarwhite))
- The `CAInjectorMerging` feature has been promoted to BETA and is now enabled by default ([#&#8203;8017](cert-manager/cert-manager#8017), [@&#8203;ThatsMrTalbot](https://github.com/ThatsMrTalbot))
- The controller, webhook and ca-injector now log their version and git commit on startup for easier debugging and support. ([#&#8203;8072](cert-manager/cert-manager#8072), [@&#8203;prasad89](https://github.com/prasad89))
- Updated `certificate` metrics to the collector approach. ([#&#8203;7856](cert-manager/cert-manager#7856), [@&#8203;hjoshi123](https://github.com/hjoshi123))

#### Bug or Regression

- ACME: Increased challenge authorization timeout to 2 minutes to fix `error waiting for authorization` ([#&#8203;7796](cert-manager/cert-manager#7796), [@&#8203;hjoshi123](https://github.com/hjoshi123))
- BUGFIX: permitted URI domains were incorrectly used to set the excluded URI domains in the CSR's name constraints ([#&#8203;7816](cert-manager/cert-manager#7816), [@&#8203;kinolaev](https://github.com/kinolaev))
- Enforced ACME HTTP-01 solver validation to properly reject configurations when multiple ingress options (`class`, `ingressClassName`, `name`) are specified simultaneously ([#&#8203;8021](cert-manager/cert-manager#8021), [@&#8203;lunarwhite](https://github.com/lunarwhite))
- Increase maximum sizes of PEM certificates and chains which can be parsed in cert-manager, to handle leaf certificates with large numbers of DNS names or other identities ([#&#8203;7961](cert-manager/cert-manager#7961), [@&#8203;SgtCoDFish](https://github.com/SgtCoDFish))
- Reverted adding the `global.rbac.disableHTTPChallengesRole` Helm option. ([#&#8203;7836](cert-manager/cert-manager#7836), [@&#8203;inteon](https://github.com/inteon))
- This change removes the `path` label of core ACME client metrics and will require users to update their monitoring dashboards and alerting rules if using those metrics. ([#&#8203;8109](cert-manager/cert-manager#8109), [@&#8203;mladen-rusev-cyberark](https://github.com/mladen-rusev-cyberark))
- Use the latest version of `ingress-nginx` in E2E tests to ensure compatibility ([#&#8203;7792](cert-manager/cert-manager#7792), [@&#8203;wallrj](https://github.com/wallrj))

#### Other (Cleanup or Flake)

- Helm: Fix naming template of `tokenrequest` RoleBinding resource to improve consistency ([#&#8203;7761](cert-manager/cert-manager#7761), [@&#8203;lunarwhite](https://github.com/lunarwhite))
- Improve error messages when certificates, CRLs or private keys fail admission due to malformed or missing PEM data ([#&#8203;7928](cert-manager/cert-manager#7928), [@&#8203;SgtCoDFish](https://github.com/SgtCoDFish))
- Major upgrade of Akamai SDK. NOTE: The new version has not been fully tested end-to-end due to the lack of cloud infrastructure. ([#&#8203;8003](cert-manager/cert-manager#8003), [@&#8203;hjoshi123](https://github.com/hjoshi123))
- Update kind images to include the Kubernetes 1.33 node image ([#&#8203;7786](cert-manager/cert-manager#7786), [@&#8203;wallrj](https://github.com/wallrj))
- Use `maps.Copy` for cleaner map handling ([#&#8203;8092](cert-manager/cert-manager#8092), [@&#8203;quantpoet](https://github.com/quantpoet))
- Vault: Migrate Vault E2E add-on tests from deprecated `vault-client-go` to the new `vault/api` client. ([#&#8203;8059](cert-manager/cert-manager#8059), [@&#8203;armagankaratosun](https://github.com/armagankaratosun))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MS4xMzUuNCIsInVwZGF0ZWRJblZlciI6IjQxLjEzNS40IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJjaGFydCJdfQ==-->

Reviewed-on: https://gitea.alexlebens.dev/alexlebens/infrastructure/pulls/1711
Co-authored-by: Renovate Bot <renovate-bot@alexlebens.net>
Co-committed-by: Renovate Bot <renovate-bot@alexlebens.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. area/acme/http01 Indicates a PR modifies ACME HTTP01 provider code area/acme Indicates a PR directly modifies the ACME Issuer code dco-signoff: yes Indicates that all commits in the pull request have the valid DCO sign-off message. kind/bug Categorizes issue or PR as related to a bug. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants