Skip to content

Create a new compaction mechanism to ensure the TaskCache functions properly#89075

Closed
lukesandberg wants to merge 2 commits intofamily_size_configurationfrom
collision_tolerant_compaction
Closed

Create a new compaction mechanism to ensure the TaskCache functions properly#89075
lukesandberg wants to merge 2 commits intofamily_size_configurationfrom
collision_tolerant_compaction

Conversation

@lukesandberg
Copy link
Contributor

@lukesandberg lukesandberg commented Jan 27, 2026

When compacting TaskCache data, dedup by key and value. Ensure SSTs break sorting ties by value so that merging works correctly.

Copy link
Contributor Author

lukesandberg commented Jan 27, 2026

@lukesandberg lukesandberg changed the base branch from persistent_hash to graphite-base/89075 January 27, 2026 00:39
@lukesandberg lukesandberg force-pushed the collision_tolerant_compaction branch from dd53faa to 7d26e47 Compare January 27, 2026 00:39
@lukesandberg lukesandberg changed the base branch from graphite-base/89075 to family_size_configuration January 27, 2026 00:40
@codspeed-hq
Copy link

codspeed-hq bot commented Jan 27, 2026

Merging this PR will not alter performance

✅ 17 untouched benchmarks
⏩ 3 skipped benchmarks1


Comparing collision_tolerant_compaction (b044b58) with family_size_configuration (49f013c)

Open in CodSpeed

Footnotes

  1. 3 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@lukesandberg lukesandberg force-pushed the family_size_configuration branch from 9ae341f to fd272b9 Compare January 27, 2026 00:48
@lukesandberg lukesandberg force-pushed the collision_tolerant_compaction branch from 7d26e47 to 1e4f1a8 Compare January 27, 2026 00:48
@lukesandberg lukesandberg force-pushed the family_size_configuration branch from fd272b9 to 4daa326 Compare January 27, 2026 00:59
@lukesandberg lukesandberg force-pushed the collision_tolerant_compaction branch from 1e4f1a8 to d8f3fce Compare January 27, 2026 00:59
@lukesandberg lukesandberg force-pushed the family_size_configuration branch from 4daa326 to ec1dad5 Compare January 27, 2026 01:03
@lukesandberg lukesandberg force-pushed the collision_tolerant_compaction branch from d8f3fce to 8e911a5 Compare January 27, 2026 01:03
@lukesandberg lukesandberg force-pushed the family_size_configuration branch from ec1dad5 to f502c31 Compare January 27, 2026 01:06
@lukesandberg lukesandberg force-pushed the collision_tolerant_compaction branch 2 times, most recently from 26a17ab to fdb39f1 Compare January 29, 2026 19:32
@lukesandberg lukesandberg force-pushed the family_size_configuration branch from f502c31 to cc8a6aa Compare January 29, 2026 19:32
@nextjs-bot
Copy link
Collaborator

nextjs-bot commented Jan 29, 2026

Failing test suites

Commit: a0ba566 | About building and testing Next.js

pnpm test packages/next/src/server/app-render/action-handler.test.ts (job)

Expand output

● Test suite failed to run

Jest encountered an unexpected token

Jest failed to parse a file. This happens e.g. when your code or its dependencies use non-standard JavaScript syntax, or when Jest is not configured to support such syntax.

Out of the box Jest supports Babel, which will be used to transform your files into valid JS based on your Babel configuration.

By default "node_modules" folder is ignored by transformers.

Here's what you can do:
 • If you are trying to use ECMAScript Modules, see https://jestjs.io/docs/ecmascript-modules for how to enable it.
 • If you are trying to use TypeScript, see https://jestjs.io/docs/getting-started#using-typescript
 • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
 • If you need a custom transformation specify a "transform" option in your config.
 • If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.

You'll find more details and examples of these config options in the docs:
https://jestjs.io/docs/configuration
For information about custom transformations, see:
https://jestjs.io/docs/code-transformation

Details:

C:\actions-runner\_work\next.js\next.js\packages\next\src\server\app-render\action-handler.ts:1
({"Object.<anonymous>":function(module,exports,require,__dirname,__filename,jest){import { RSC_HEADER, RSC_CONTENT_TYPE_HEADER, NEXT_ROUTER_STATE_TREE_HEADER, ACTION_HEADER, NEXT_ACTION_NOT_FOUND_HEADER, NEXT_ROUTER_PREFETCH_HEADER, NEXT_ROUTER_SEGMENT_PREFETCH_HEADER, NEXT_URL, NEXT_ACTION_REVALIDATED_HEADER } from '../../client/components/app-router-headers';
                                                                                  ^^^^^^

SyntaxError: Cannot use import statement outside a module

  3 | describe('parseHostHeader', () => {
  4 |   it('should return correct host', () => {
> 5 |     expect(parseHostHeader({})).toBe(undefined)
    |                        ^
  6 |
  7 |     expect(
  8 |       parseHostHeader({

  at Runtime.createScriptFromCode (../node_modules/.pnpm/jest-runtime@29.7.0/node_modules/jest-runtime/build/index.js:1505:14)
  at Object.<anonymous> (../packages/next/src/server/app-render/action-handler.test.ts:5:24)

@nextjs-bot
Copy link
Collaborator

nextjs-bot commented Jan 30, 2026

Stats from current PR

✅ No significant changes detected

📊 All Metrics
📖 Metrics Glossary

Dev Server Metrics:

  • Listen = TCP port starts accepting connections
  • First Request = HTTP server returns successful response
  • Cold = Fresh build (no cache)
  • Warm = With cached build artifacts

Build Metrics:

  • Fresh = Clean build (no .next directory)
  • Cached = With existing .next directory

Change Thresholds:

  • Time: Changes < 50ms AND < 10%, OR < 2% are insignificant
  • Size: Changes < 1KB AND < 1% are insignificant
  • All other changes are flagged to catch regressions

⚡ Dev Server

Metric Canary PR Change Trend
Cold (Listen) 455ms 456ms ▁▁▁▁▁
Cold (Ready in log) 451ms 452ms ▁▂▂▁▁
Cold (First Request) 863ms 861ms ▁▄▃▃▃
Warm (Listen) 455ms 456ms ▁▁▁▁▁
Warm (Ready in log) 451ms 451ms ▁▁▁▁▁
Warm (First Request) 362ms 360ms ▁▁▁▁▁
📦 Dev Server (Webpack) (Legacy)

📦 Dev Server (Webpack)

Metric Canary PR Change Trend
Cold (Listen) 458ms 506ms ▁▁▅▁▁
Cold (Ready in log) 461ms 462ms ▃▃▅▂▄
Cold (First Request) 1.985s 1.922s ▂▂▆▂▂
Warm (Listen) 457ms 458ms ▁▁▄▁▁
Warm (Ready in log) 459ms 461ms ▃▃▅▂▄
Warm (First Request) 1.969s 1.956s ▂▂▅▁▂

⚡ Production Builds

Metric Canary PR Change Trend
Fresh Build 4.177s 4.201s ▁▁▁▁▁
Cached Build 4.190s 4.170s ▁▁▁▁▁
📦 Production Builds (Webpack) (Legacy)

📦 Production Builds (Webpack)

Metric Canary PR Change Trend
Fresh Build 14.497s 14.540s ▁▁▅▁▁
Cached Build 14.619s 14.563s ▁▁▅▁▁
node_modules Size 467 MB 467 MB ▁▁▁▁▁
📦 Bundle Sizes

Bundle Sizes

⚡ Turbopack

Client

Main Bundles: **437 kB** → **437 kB** ✅ -1 B

81 files with content-based hashes (individual files not comparable between builds)

Server

Middleware
Canary PR Change
middleware-b..fest.js gzip 765 B 767 B
Total 765 B 767 B ⚠️ +2 B
Build Details
Build Manifests
Canary PR Change
_buildManifest.js gzip 451 B 450 B
Total 451 B 450 B ✅ -1 B

📦 Webpack

Client

Main Bundles
Canary PR Change
5528-HASH.js gzip 5.47 kB N/A -
6280-HASH.js gzip 56.9 kB N/A -
6335.HASH.js gzip 169 B N/A -
912-HASH.js gzip 4.53 kB N/A -
e8aec2e4-HASH.js gzip 62.5 kB N/A -
framework-HASH.js gzip 59.7 kB 59.7 kB
main-app-HASH.js gzip 256 B 255 B
main-HASH.js gzip 39.1 kB 39.1 kB
webpack-HASH.js gzip 1.68 kB 1.68 kB
262-HASH.js gzip N/A 4.52 kB -
2889.HASH.js gzip N/A 169 B -
5602-HASH.js gzip N/A 5.48 kB -
6948ada0-HASH.js gzip N/A 62.5 kB -
9544-HASH.js gzip N/A 57.5 kB -
Total 230 kB 231 kB ⚠️ +616 B
Polyfills
Canary PR Change
polyfills-HASH.js gzip 39.4 kB 39.4 kB
Total 39.4 kB 39.4 kB
Pages
Canary PR Change
_app-HASH.js gzip 194 B 194 B
_error-HASH.js gzip 183 B 180 B 🟢 3 B (-2%)
css-HASH.js gzip 331 B 330 B
dynamic-HASH.js gzip 1.81 kB 1.81 kB
edge-ssr-HASH.js gzip 256 B 256 B
head-HASH.js gzip 351 B 352 B
hooks-HASH.js gzip 384 B 383 B
image-HASH.js gzip 580 B 581 B
index-HASH.js gzip 260 B 260 B
link-HASH.js gzip 2.49 kB 2.49 kB
routerDirect..HASH.js gzip 320 B 319 B
script-HASH.js gzip 386 B 386 B
withRouter-HASH.js gzip 315 B 315 B
1afbb74e6ecf..834.css gzip 106 B 106 B
Total 7.97 kB 7.97 kB ✅ -1 B

Server

Edge SSR
Canary PR Change
edge-ssr.js gzip 126 kB 126 kB
page.js gzip 249 kB 249 kB
Total 375 kB 375 kB ⚠️ +473 B
Middleware
Canary PR Change
middleware-b..fest.js gzip 615 B 616 B
middleware-r..fest.js gzip 156 B 155 B
middleware.js gzip 33.1 kB 33.2 kB
edge-runtime..pack.js gzip 842 B 842 B
Total 34.7 kB 34.8 kB ⚠️ +37 B
Build Details
Build Manifests
Canary PR Change
_buildManifest.js gzip 732 B 736 B
Total 732 B 736 B ⚠️ +4 B
Build Cache
Canary PR Change
0.pack gzip 3.85 MB 3.85 MB 🔴 +8.06 kB (+0%)
index.pack gzip 102 kB 103 kB
index.pack.old gzip 104 kB 104 kB
Total 4.05 MB 4.06 MB ⚠️ +9.13 kB

🔄 Shared (bundler-independent)

Runtimes
Canary PR Change
app-page-exp...dev.js gzip 315 kB 315 kB
app-page-exp..prod.js gzip 167 kB 167 kB
app-page-tur...dev.js gzip 315 kB 315 kB
app-page-tur..prod.js gzip 167 kB 167 kB
app-page-tur...dev.js gzip 312 kB 312 kB
app-page-tur..prod.js gzip 165 kB 165 kB
app-page.run...dev.js gzip 312 kB 312 kB
app-page.run..prod.js gzip 165 kB 165 kB
app-route-ex...dev.js gzip 70.5 kB 70.5 kB
app-route-ex..prod.js gzip 49 kB 49 kB
app-route-tu...dev.js gzip 70.5 kB 70.5 kB
app-route-tu..prod.js gzip 49 kB 49 kB
app-route-tu...dev.js gzip 70.1 kB 70.1 kB
app-route-tu..prod.js gzip 48.8 kB 48.8 kB
app-route.ru...dev.js gzip 70.1 kB 70.1 kB
app-route.ru..prod.js gzip 48.7 kB 48.7 kB
dist_client_...dev.js gzip 324 B 324 B
dist_client_...dev.js gzip 326 B 326 B
dist_client_...dev.js gzip 318 B 318 B
dist_client_...dev.js gzip 317 B 317 B
pages-api-tu...dev.js gzip 43.2 kB 43.2 kB
pages-api-tu..prod.js gzip 32.9 kB 32.9 kB
pages-api.ru...dev.js gzip 43.1 kB 43.1 kB
pages-api.ru..prod.js gzip 32.8 kB 32.8 kB
pages-turbo....dev.js gzip 52.5 kB 52.5 kB
pages-turbo...prod.js gzip 39.4 kB 39.4 kB
pages.runtim...dev.js gzip 52.5 kB 52.5 kB
pages.runtim..prod.js gzip 39.3 kB 39.3 kB
server.runti..prod.js gzip 62.7 kB 62.7 kB
Total 2.8 MB 2.8 MB

Copy link
Member

@sokra sokra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That MUST be part of #88904, otherwise it's incorrect and would corrupt the cache during compaction.

But I think we shouldn't have a family config for that, but encode this in a bit of the value type. This way you could also mix set values with normal values. See my comment on 88904.

@lukesandberg lukesandberg force-pushed the collision_tolerant_compaction branch from c34eeaa to b044b58 Compare February 8, 2026 19:17
@lukesandberg lukesandberg force-pushed the family_size_configuration branch from d482259 to 49f013c Compare February 8, 2026 19:17
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 26, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

created-by: Turbopack team PRs by the Turbopack team. locked Turbopack Related to Turbopack with Next.js.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants