Skip to content

[test] Deflake test/integration/invalid-custom-routes/test/index.test.ts #88416

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
eps1lon merged 1 commit into from
Jan 12, 2026
Merged

[test] Deflake test/integration/invalid-custom-routes/test/index.test.ts #88416

eps1lon merged 1 commit into from
Jan 12, 2026
+178 −148

Conversation

@eps1lon
Copy link
Member

@eps1lon eps1lon commented Jan 12, 2026 •
edited
Loading

More and more checks get deferred since we want "ready" to be a signal for "ready to use" not "ready and correct". Tests need to account for that.

This PR extracts launching the app out of getStderr so that we can actually retry reading stderr without attempting to launch a new app.
Some tests already retried getStderr. This PR just expands the automatic retry to every test. Just without restarting the app and instead reusing the existing instance.

Fixes

Expected substring: "destination is missing for route {"source":"/hello"}"
Received string: ""

-- https://github.com/vercel/next.js/actions/runs/20915455078/job/60087861981

GPT-5.1-Codex-Max prompts

This test should always launch the app. getStderr should just read stderr and then we need to retry every stderr assertion like "should error when empty headers array is present on header item" does

Remove writeConfigAndStart and expectStderr helpers and inline their impl instead.

@nextjs-bot nextjs-bot added created-by: Next.js team PRs by the Next.js team. tests labels Jan 12, 2026
@nextjs-bot
Copy link
Collaborator

nextjs-bot commented Jan 12, 2026

Stats from current PR

✅ No significant changes detected

📊 All Metrics
📖 Metrics Glossary

Dev Server Metrics:

  • Listen = TCP port starts accepting connections
  • First Request = HTTP server returns successful response
  • Cold = Fresh build (no cache)
  • Warm = With cached build artifacts

Build Metrics:

  • Fresh = Clean build (no .next directory)
  • Cached = With existing .next directory

Change Thresholds:

  • Time: Changes < 50ms AND < 10%, OR < 2% are insignificant
  • Size: Changes < 1KB AND < 1% are insignificant
  • All other changes are flagged to catch regressions

⚡ Dev Server

Metric Canary PR Change Trend
Cold (Listen) 455ms 456ms ✓ ▁█▁▁▁
Cold (Ready in log) 443ms 442ms ✓ █▂▇▇▆
Cold (First Request) 834ms 827ms ✓ █▂▇██
Warm (Listen) 456ms 456ms ✓ ▆▃▆▃▃
Warm (Ready in log) 443ms 441ms ✓ ▁█▁▁▄
Warm (First Request) 349ms 348ms ✓ ▆██▁▁
📦 Dev Server (Webpack) (Legacy)

📦 Dev Server (Webpack)

Metric Canary PR Change Trend
Cold (Listen) 507ms 506ms ✓ █████
Cold (Ready in log) 480ms 479ms ✓ ▁▂█▅▄
Cold (First Request) 1.980s 1.999s ✓ ▄▄█▄▄
Warm (Listen) 507ms 506ms ✓ ▅▅▅▅▁
Warm (Ready in log) 481ms 480ms ✓ ▇▁▅▅▅
Warm (First Request) 2.063s 1.988s ✓ ▆▄█▅▅

⚡ Production Builds

Metric Canary PR Change Trend
Fresh Build 4.158s 4.197s ✓ ▁█▁▁▂
Cached Build 4.197s 4.214s ✓ ▁█▁▂▂
📦 Production Builds (Webpack) (Legacy)

📦 Production Builds (Webpack)

Metric Canary PR Change Trend
Fresh Build 15.621s 15.711s ✓ ▂▂█▁▂
Cached Build 15.799s 15.730s ✓ ▂▁█▁▂
node_modules Size 457 MB 457 MB ✓ █████
📦 Bundle Sizes

Bundle Sizes

⚡ Turbopack

Client

Main Bundles: **430 kB** → **430 kB** ✅ -29 B

82 files with content-based hashes (individual files not comparable between builds)

Server

Middleware
Canary PR Change
middleware-b..fest.js gzip 788 B 788 B ✓
Total 788 B 788 B ✓
Build Details
Build Manifests
Canary PR Change
_buildManifest.js gzip 451 B 450 B ✓
Total 451 B 450 B ✅ -1 B

📦 Webpack

Client

Main Bundles
Canary PR Change
2086.HASH.js gzip 169 B N/A -
2161-HASH.js gzip 5.41 kB N/A -
2747-HASH.js gzip 4.48 kB N/A -
4322-HASH.js gzip 52.8 kB N/A -
ec793fe8-HASH.js gzip 62.3 kB N/A -
framework-HASH.js gzip 59.8 kB 59.8 kB ✓
main-app-HASH.js gzip 251 B 253 B ✓
main-HASH.js gzip 38.6 kB 39 kB ✓
webpack-HASH.js gzip 1.68 kB 1.68 kB ✓
1596.HASH.js gzip N/A 169 B -
2658-HASH.js gzip N/A 52.7 kB -
6349-HASH.js gzip N/A 4.46 kB -
7019-HASH.js gzip N/A 5.43 kB -
b17a3386-HASH.js gzip N/A 62.3 kB -
Total 225 kB 226 kB ⚠️ +243 B
Polyfills
Canary PR Change
polyfills-HASH.js gzip 39.4 kB 39.4 kB ✓
Total 39.4 kB 39.4 kB ✓
Pages
Canary PR Change
_app-HASH.js gzip 194 B 193 B ✓
_error-HASH.js gzip 182 B 182 B ✓
css-HASH.js gzip 336 B 335 B ✓
dynamic-HASH.js gzip 1.8 kB 1.8 kB ✓
edge-ssr-HASH.js gzip 256 B 256 B ✓
head-HASH.js gzip 352 B 349 B ✓
hooks-HASH.js gzip 385 B 384 B ✓
image-HASH.js gzip 580 B 580 B ✓
index-HASH.js gzip 259 B 258 B ✓
link-HASH.js gzip 2.5 kB 2.51 kB ✓
routerDirect..HASH.js gzip 319 B 317 B ✓
script-HASH.js gzip 385 B 387 B ✓
withRouter-HASH.js gzip 316 B 315 B ✓
1afbb74e6ecf..834.css gzip 106 B 106 B ✓
Total 7.97 kB 7.96 kB ✅ -8 B

Server

Edge SSR
Canary PR Change
edge-ssr.js gzip 125 kB 125 kB ✓
page.js gzip 239 kB 240 kB ✓
Total 364 kB 364 kB ⚠️ +281 B
Middleware
Canary PR Change
middleware-b..fest.js gzip 652 B 651 B ✓
middleware-r..fest.js gzip 155 B 156 B ✓
middleware.js gzip 32.7 kB 33 kB 🔴 +354 B (+1%)
edge-runtime..pack.js gzip 842 B 842 B ✓
Total 34.3 kB 34.7 kB ⚠️ +354 B
Build Details
Build Manifests
Canary PR Change
_buildManifest.js gzip 738 B 738 B ✓
Total 738 B 738 B ✓
Build Cache
Canary PR Change
0.pack gzip 3.62 MB 3.63 MB 🔴 +9.08 kB (+0%)
index.pack gzip 99.5 kB 99.6 kB ✓
index.pack.old gzip 99.8 kB 97.5 kB 🟢 2.28 kB (-2%)
Total 3.82 MB 3.82 MB ⚠️ +6.9 kB

🔄 Shared (bundler-independent)

Runtimes
Canary PR Change
app-page-exp...dev.js gzip 303 kB 303 kB ✓
app-page-exp..prod.js gzip 158 kB 158 kB ✓
app-page-tur...dev.js gzip 303 kB 303 kB ✓
app-page-tur..prod.js gzip 158 kB 158 kB ✓
app-page-tur...dev.js gzip 299 kB 299 kB ✓
app-page-tur..prod.js gzip 156 kB 156 kB ✓
app-page.run...dev.js gzip 300 kB 300 kB ✓
app-page.run..prod.js gzip 156 kB 156 kB ✓
app-route-ex...dev.js gzip 68.2 kB 68.2 kB ✓
app-route-ex..prod.js gzip 46.9 kB 46.9 kB ✓
app-route-tu...dev.js gzip 68.2 kB 68.2 kB ✓
app-route-tu..prod.js gzip 46.9 kB 46.9 kB ✓
app-route-tu...dev.js gzip 67.8 kB 67.8 kB ✓
app-route-tu..prod.js gzip 46.7 kB 46.7 kB ✓
app-route.ru...dev.js gzip 67.8 kB 67.8 kB ✓
app-route.ru..prod.js gzip 46.7 kB 46.7 kB ✓
dist_client_...dev.js gzip 324 B 324 B ✓
dist_client_...dev.js gzip 326 B 326 B ✓
dist_client_...dev.js gzip 318 B 318 B ✓
dist_client_...dev.js gzip 317 B 317 B ✓
pages-api-tu...dev.js gzip 41.1 kB 41.1 kB ✓
pages-api-tu..prod.js gzip 31.2 kB 31.2 kB ✓
pages-api.ru...dev.js gzip 41.1 kB 41.1 kB ✓
pages-api.ru..prod.js gzip 31.2 kB 31.2 kB ✓
pages-turbo....dev.js gzip 50.8 kB 50.8 kB ✓
pages-turbo...prod.js gzip 38.2 kB 38.2 kB ✓
pages.runtim...dev.js gzip 50.8 kB 50.8 kB ✓
pages.runtim..prod.js gzip 38.2 kB 38.2 kB ✓
server.runti..prod.js gzip 62 kB 62 kB ✓
Total 2.68 MB 2.68 MB ✓

@eps1lon eps1lon marked this pull request as ready for review January 12, 2026 12:04
@eps1lon eps1lon requested a review from unstubbable January 12, 2026 12:05
@eps1lon eps1lon enabled auto-merge (squash) January 12, 2026 12:05
@eps1lon eps1lon merged commit f814b8b into canary Jan 12, 2026
288 of 290 checks passed
@eps1lon eps1lon deleted the sebbie/01-12-_test_deflake_test/integration/invalid-custom-routes/test/index.test.ts branch January 12, 2026 12:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@unstubbable unstubbable unstubbable approved these changes

Assignees

No one assigned

Labels

created-by: Next.js team PRs by the Next.js team. tests

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@eps1lon @nextjs-bot @unstubbable