Skip to content

Improve E2E test debuggability in CI #4077

@JAORMX

Description

@JAORMX

Summary

When E2E tests fail in CI, we lack the information needed to diagnose root causes. A recent api-clients failure returned HTTP 500 instead of 201 during workload creation, but we couldn't determine why because:

  1. Response bodies aren't logged on failure — tests assert status codes but don't dump the response body when assertions fail
  2. thv serve logs aren't captured — the API server's stdout/stderr isn't saved as a CI artifact
  3. Ginkgo report files aren't generated — the upload step consistently finds no files at test/e2e/ginkgo-report.xml

Proposed improvements

1. Log response bodies on assertion failure

Test helpers like createWorkload, registerClient, etc. should log the response body when the status code doesn't match expectations. Example pattern:

resp := createWorkload(apiServer, workloadReq)
body, _ := io.ReadAll(resp.Body)
resp.Body.Close()
Expect(resp.StatusCode).To(Equal(http.StatusCreated),
    "Unexpected status code. Response body: %s", string(body))

2. Capture thv serve logs as CI artifacts

The E2E test runner or helpers should redirect thv serve stderr/stdout to a log file, and the CI workflow should upload it alongside test results.

3. Fix Ginkgo report generation

The workflow expects test/e2e/ginkgo-report.xml and test/e2e/junit-report.xml but they're never created. Either:

  • Pass --junit-report=junit-report.xml to ginkgo in run_tests.sh
  • Or remove the upload step if reports aren't needed

4. Consider adding --vv output on failure

When a test fails, having verbose Ginkgo output (--vv) would show the full step timeline, making it easier to trace what happened.

Context

Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions