Skip to content

backport: feat: implement LRU cache with invocation ID scoping for minimal mode response cache#89129

Merged
ztanner merged 2 commits intonext-15-5from
ztanner/backport-88509-2
Jan 28, 2026
Merged

backport: feat: implement LRU cache with invocation ID scoping for minimal mode response cache#89129
ztanner merged 2 commits intonext-15-5from
ztanner/backport-88509-2

Conversation

@ztanner
Copy link
Member

@ztanner ztanner commented Jan 27, 2026

… response cache (#88509)

## Summary

Implements an LRU cache with compound keys for the minimal mode response
cache to improve cache hit rates during parallel revalidation scenarios.

**Problem**: The previous single-entry cache (`previousCacheItem`) keyed
by pathname caused cache collisions when multiple concurrent invocations
(e.g., during ISR revalidation) accessed the same pathname. Each
invocation would overwrite the previous entry, leading to cache misses
and redundant work.

**Solution**: An LRU cache using compound keys (`pathname +
invocationID`) that allows multiple invocations to cache entries for the
same pathname independently:

```
Cache Key Structure
─────────────────────
/blog/post-1\0inv-abc  →  {entry, expiresAt}
/blog/post-1\0inv-def  →  {entry, expiresAt}
/blog/post-1\0__ttl__  →  {entry, expiresAt}  (TTL fallback)
/api/data\0inv-ghi     →  {entry, expiresAt}
```

### Cache Key Strategy

- **With `x-invocation-id` header**: Entries are keyed by invocation ID
for exact-match lookups (always a cache hit if the entry exists)
- **Without header (TTL fallback)**: Entries use a `__ttl__` sentinel
key and validate via expiration timestamp

### Configuration via Environment Variables

Cache sizing can be tuned via environment variables (using
`NEXT_PRIVATE_*` prefix for infrastructure-level settings):

| Environment Variable | Default | Description |
|---------------------|---------|-------------|
| `NEXT_PRIVATE_RESPONSE_CACHE_MAX_SIZE` | 150 | Max entries in the LRU
cache |
| `NEXT_PRIVATE_RESPONSE_CACHE_TTL` | 10000 | TTL in ms for cache
entries (fallback validation) |

### LRU Cache Enhancement

Added an optional `onEvict` callback to `LRUCache` that fires when
entries are evicted due to capacity limits. This enables tracking
evicted invocation IDs for warning detection without introducing
timer-based cleanup.

### Eviction Warnings

When a cache entry is evicted and later accessed by the same invocation,
a warning is logged suggesting to increase
`NEXT_PRIVATE_RESPONSE_CACHE_MAX_SIZE`. This helps developers tune cache
sizes for their workload.

### Additional Changes

- Renamed header from `x-vercel-id` to `x-invocation-id` for clarity
- Added `withInvocationId()` test helper for cache testing

## Test Plan

- Existing response cache tests pass with updated header name
- Unit tests for `LRUCache` including `onEvict` callback behavior
- Updated standalone mode tests to use `withInvocationId()` helper
@ztanner ztanner force-pushed the ztanner/backport-88509-2 branch from e89b7b1 to dfa9b15 Compare January 27, 2026 22:17
@nextjs-bot
Copy link
Collaborator

nextjs-bot commented Jan 27, 2026

Failing test suites

Commit: 5aa761f | About building and testing Next.js

pnpm test packages/next-codemod/transforms/__tests__/app-dir-runtime-config-experimental-edge.test.js (job)

  • app-dir-runtime-config-experimental-edge > transforms correctly using "app-dir-runtime-config-experimental-edge/basic" data (DD)
Expand output

● app-dir-runtime-config-experimental-edge › transforms correctly using "app-dir-runtime-config-experimental-edge/basic" data

expect(received).toEqual(expected) // deep equality

- Expected  - 3
+ Received  + 3

- export const runtime = "edge";
+ export const runtime = "edge";
- export default function Page() {
+ export default function Page() {
-   return <div>hello world</div>;
+   return <div>hello world</div>;
  }

  at runInlineTest (../node_modules/.pnpm/jscodeshift@17.0.0_@babel+preset-env@7.26.9_@babel+core@7.26.10_/node_modules/jscodeshift/dist/testUtils.js:49:18)
  at runTest (../node_modules/.pnpm/jscodeshift@17.0.0_@babel+preset-env@7.26.9_@babel+core@7.26.10_/node_modules/jscodeshift/dist/testUtils.js:98:3)
  at Object.<anonymous> (../node_modules/.pnpm/jscodeshift@17.0.0_@babel+preset-env@7.26.9_@babel+core@7.26.10_/node_modules/jscodeshift/dist/testUtils.js:115:7)

We originally prefixed the TTL sentinel with the null-byte separator to
avoid collisions with real invocation IDs, but that makes
`extractInvocationID` return "ttl" instead of undefined for TTL-mode
keys.

This switches to a reserved magic string sentinel so TTL-mode entries
are correctly treated as “no invocation ID”.
@ztanner ztanner force-pushed the ztanner/backport-88509-2 branch from f37e072 to 5aa761f Compare January 28, 2026 01:22
@ztanner ztanner marked this pull request as ready for review January 28, 2026 02:10
@ztanner ztanner requested a review from ijjk January 28, 2026 02:10
@ztanner ztanner merged commit aff987d into next-15-5 Jan 28, 2026
124 of 129 checks passed
@ztanner ztanner deleted the ztanner/backport-88509-2 branch January 28, 2026 02:32
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 11, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants