perf: reduce CI integration test wall time by ~50% by Copilot · Pull Request #1399 · github/gh-aw-firewall

Copilot · 2026-03-23T15:12:11Z

Integration tests take 37 min critical path / 106 min aggregate across 4 jobs. Root causes: 10s default docker compose down grace period per test (×237 tests), unbalanced job sizes, and unnecessary job dependencies.

`docker compose down -t 1` (P0, ~30 min aggregate saved)

Containers' main processes have already exited by the time stopContainers() runs. The default 10s SIGTERM grace period is pure waste.

// before
await execa('docker', ['compose', 'down', '-v'], { cwd: workDir, ... });
// after
await execa('docker', ['compose', 'down', '-v', '-t', '1'], { cwd: workDir, ... });

Split Domain & Network job (P1, ~17 min off critical path)

test-domain-network (77 tests, 37 min) → two parallel jobs:

Domain Tests: blocked-domains, dns-servers, empty-domains, wildcard-patterns
Network Tests: ipv6, localhost-access, network-security

Remove chroot job dependency (P4, ~4 min saved)

test-chroot-package-managers had needs: test-chroot-languages but installs its own runtimes via setup actions. Removed the unnecessary serial dependency.

Add `skip-pull` tests to CI (P5)

skip-pull.test.ts was not matched by any job's --testPathPatterns and is now included in the Container & Ops Tests job.

Note: Three other previously unmatched test files (ghes-auto-populate, gh-host-injection, workdir-tmpfs-hiding) have pre-existing failures and were not added — they need to be fixed separately before being included in CI.

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

- Add `-t 1` to `docker compose down` to reduce container shutdown grace period from 10s to 1s (~30 min aggregate CI savings) - Split Domain & Network Tests into separate Domain Tests and Network Tests jobs to reduce critical path (~17 min savings) - Remove unnecessary `needs: test-chroot-languages` dependency from chroot package-managers job (~4 min savings) - Add missing tests (gh-host-injection, ghes-auto-populate, skip-pull, workdir-tmpfs-hiding) to CI jobs Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com> Agent-Logs-Url: https://github.com/github/gh-aw-firewall/sessions/48965f04-d66b-4ef4-914d-f4ecd980ed20

github-actions · 2026-03-23T16:08:15Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	86.11%	86.35%	📈 +0.24%
Statements	86.03%	86.27%	📈 +0.24%
Functions	86.13%	86.41%	📈 +0.28%
Branches	79.31%	79.59%	📈 +0.28%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
`src/docker-manager.ts`	87.7% → 88.5% (+0.88%)	87.0% → 87.9% (+0.90%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

github-actions · 2026-03-23T16:14:39Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	passed	✅ PASS
Node.js	execa	✅	passed	✅ PASS
Node.js	p-limit	✅	passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #1399 · ◷

Mossaka · 2026-03-23T17:53:20Z

@copilot please fix the test failures from https://github.com/github/gh-aw-firewall/actions/runs/23444805387/job/68214288261?pr=1399 and https://github.com/github/gh-aw-firewall/actions/runs/23444805387/job/68214288292?pr=1399

Remove ghes-auto-populate, gh-host-injection, and workdir-tmpfs-hiding from CI test patterns. These tests have pre-existing failures and were never running in CI before. They need to be fixed separately before being added to CI. Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com> Agent-Logs-Url: https://github.com/github/gh-aw-firewall/sessions/9e55fbef-edba-4844-a9fa-b2176ac4cc53

Copilot · 2026-03-23T17:55:59Z

@copilot please fix the test failures from https://github.com/github/gh-aw-firewall/actions/runs/23444805387/job/68214288261?pr=1399 and https://github.com/github/gh-aw-firewall/action...

Removed ghes-auto-populate, gh-host-injection, and workdir-tmpfs-hiding from the CI test patterns in 1aa7f15. These 3 test files have pre-existing failures (they were never running in CI before this PR) — e.g. ghes-auto-populate can't find workDir for Squid log assertions, workdir-tmpfs-hiding expects tmpfs overlay behavior not configured in CI, and gh-host-injection has an --env-all propagation issue. The skip-pull tests pass and remain in the Container & Ops job.

Copilot

Pull request overview

This PR speeds up the repository’s GitHub Actions integration test suite by reducing per-test Docker Compose teardown time, improving parallelism by splitting a long-running job, and removing an unnecessary job dependency.

Changes:

Reduce docker compose down grace period to 1s during test cleanup to avoid repeated 10s waits.
Split the previous Domain & Network integration job into two parallel jobs (Domain Tests / Network Tests).
Include skip-pull integration tests in CI and remove an unnecessary needs dependency in chroot workflows.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File	Description
`src/docker-manager.ts`	Speeds up container teardown by adding `docker compose down -t 1`.
`src/docker-manager.test.ts`	Updates the unit test expectation to match the new compose down args.
`.github/workflows/test-integration-suite.yml`	Splits domain/network tests into two jobs and adds `skip-pull` to patterns.
`.github/workflows/test-chroot.yml`	Removes a serial dependency so package manager tests can run in parallel.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

github-actions · 2026-03-23T19:03:17Z

🔮 The oracle records this smoke rite.
PR titles (merged):

Propagate $GITHUB_PATH into chroot PATH for setup-* action support
feat: increase default agent memory limit to 6GB and enable swap
GitHub MCP: ✅ | safeinputs-gh CLI: ❌ | Playwright: ✅ | Tavily: ❌
File write: ✅ | Bash cat verify: ✅ | Discussion interaction: ❌ | Build (npm ci && npm run build): ✅
Overall status: FAIL

🔮 The oracle has spoken through Smoke Codex

Warning

⚠️ Firewall blocked 2 domains

The following domains were blocked by the firewall during workflow execution:

ab.chatgpt.com
registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "ab.chatgpt.com"
    - "registry.npmjs.org"

See Network Configuration for more information.

Initial plan

8c5c530

Copilot AI assigned Copilot and Mossaka Mar 23, 2026

Copilot started work on behalf of Mossaka March 23, 2026 15:12 View session

Copilot AI linked an issue Mar 23, 2026 that may be closed by this pull request

perf: integration tests taking 37+ minutes in CI (Domain & Network job) #1376

Closed

Copilot AI changed the title ~~[WIP] Optimize integration test durations in CI~~ perf: reduce CI integration test wall time by ~50% Mar 23, 2026

Copilot AI requested a review from Mossaka March 23, 2026 15:19

Copilot finished work on behalf of Mossaka March 23, 2026 15:19

github-actions bot added the build-test label Mar 23, 2026

This comment has been minimized.

Sign in to view

Copilot started work on behalf of Mossaka March 23, 2026 17:53 View session

Copilot finished work on behalf of Mossaka March 23, 2026 17:56

Mossaka marked this pull request as ready for review March 23, 2026 18:54

Copilot AI review requested due to automatic review settings March 23, 2026 18:54

Copilot started reviewing on behalf of Mossaka March 23, 2026 18:55 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

Mossaka merged commit d3f6c86 into main Mar 23, 2026
63 checks passed

Mossaka deleted the copilot/optimize-ci-integration-tests branch March 23, 2026 19:18

github-actions bot mentioned this pull request Mar 24, 2026

fix: write apiKeyHelper to ~/.claude/settings.json for Claude Code v2.1.81+ #1414

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

perf: reduce CI integration test wall time by ~50%#1399

perf: reduce CI integration test wall time by ~50%#1399
Mossaka merged 3 commits intomainfrom
copilot/optimize-ci-integration-tests

Copilot AI commented Mar 23, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 23, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 23, 2026

Uh oh!

This comment has been minimized.

Mossaka commented Mar 23, 2026

Uh oh!

Copilot AI commented Mar 23, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

github-actions bot commented Mar 23, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Copilot AI commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

docker compose down -t 1 (P0, ~30 min aggregate saved)

Split Domain & Network job (P1, ~17 min off critical path)

Remove chroot job dependency (P4, ~4 min saved)

Add skip-pull tests to CI (P5)

Uh oh!

github-actions bot commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Coverage Check Passed

Overall Coverage

Uh oh!

github-actions bot commented Mar 23, 2026

🏗️ Build Test Suite Results

Uh oh!

This comment has been minimized.

Mossaka commented Mar 23, 2026

Uh oh!

Copilot AI commented Mar 23, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

github-actions bot commented Mar 23, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Mar 23, 2026 •

edited

Loading

`docker compose down -t 1` (P0, ~30 min aggregate saved)

Add `skip-pull` tests to CI (P5)

github-actions bot commented Mar 23, 2026 •

edited

Loading