Skip to content

feat: dmmf streaming with an E2E test#29377

Merged
jacek-prisma merged 16 commits intomainfrom
fix/dmmf-v8-string-limit-fallback-test
Mar 25, 2026
Merged

feat: dmmf streaming with an E2E test#29377
jacek-prisma merged 16 commits intomainfrom
fix/dmmf-v8-string-limit-fallback-test

Conversation

@jacek-prisma
Copy link
Copy Markdown
Contributor

@jacek-prisma jacek-prisma commented Mar 24, 2026

Cleaned up version of #29137 with an E2E test

Summary by CodeRabbit

  • Tests

    • Added an end-to-end test and supporting fixture to validate generation and handling of extremely large schemas (512MB+).
  • New Features

    • Introduced a streaming DMMF parsing path with an automatic buffered fallback to handle very large schema outputs more reliably and reduce memory-related failures.
    • Improved error handling and diagnostics for DMMF generation/parsing failures.
  • Chores

    • Added an optional streaming JSON parser dependency.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Mar 24, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Adds a new e2e fixture that generates a very large Prisma schema and runs generation steps; updates internals to add a buffered, chunked DMMF retrieval fallback using a streaming JSON parser when V8 string-size errors occur, and adds the new dependency.

Changes

Cohort / File(s) Summary
Large Schema Generation E2E Test
packages/client/tests/e2e/large-schema-generation/_steps.ts, packages/client/tests/e2e/large-schema-generation/package.json, packages/client/tests/e2e/large-schema-generation/prisma.config.ts, packages/client/tests/e2e/large-schema-generation/prisma/schema.prisma, packages/client/tests/e2e/large-schema-generation/readme.md, packages/client/tests/e2e/large-schema-generation/src/generate-schema.ts, packages/client/tests/e2e/large-schema-generation/tsconfig.json
Adds a new E2E fixture that generates a huge schema (script creates 7,200 models), provides package/tsconfig/prisma config and placeholder schema, and step definitions that run generation and prisma generate while asserting the buffered-DMMF fallback log appears.
Internals: DMMF retrieval & deps
packages/internals/package.json, packages/internals/src/engine-commands/getDmmf.ts
Adds @streamparser/json (dependency and peer, marked optional) and implements a buffered DMMF fallback: detect V8 ERR_STRING_TOO_LONG, read WASM buffer in 16MB chunks, parse with a streaming JSON parser, ensure buffer cleanup, and centralize mapping of WASM panics/errors.
🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main changes: implementing DMMF streaming functionality and adding an E2E test to verify it works with large schemas.
Docstring Coverage ✅ Passed Docstring coverage is 80.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/dmmf-v8-string-limit-fallback-test

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/client/tests/e2e/large-schema-generation/_steps.ts`:
- Around line 10-12: The test currently only runs "pnpm prisma generate" and
asserts success; change the test (the async function assigned to test in
_steps.ts) to force and verify the fallback/streaming path by running the
generate command with a deterministic toggle and checking output: set an env var
or flag (e.g., PRISMA_FORCE_FALLBACK=1 or --enable-fallback) when invoking the
command, capture stdout/stderr from the child process, and assert that it
contains a known marker string (for example "Using query engine fallback" or the
streaming-specific log/metric). Alternatively enable debug metrics (e.g.,
--debug-metrics) and assert the expected metric/log line is present so the test
proves the fallback/streaming path was exercised rather than just exit success.

In `@packages/client/tests/e2e/large-schema-generation/package.json`:
- Around line 6-8: The package.json for the large-schema-generation e2e fixture
is missing a pinned `@prisma/client-runtime-utils` entry; update the
"dependencies" block to include "@prisma/client-runtime-utils" pointing to the
same local tarball (e.g. "/tmp/prisma-client-0.0.0.tgz") alongside
"@prisma/client" so installs remain hermetic and version-aligned for this
fixture.

In `@packages/client/tests/e2e/large-schema-generation/readme.md`:
- Around line 3-4: Update the test description string that currently reads ">
512GB DMMF" to use the correct unit reflecting the V8 string limit (e.g., ">
512MB DMMF" or "hundreds of MB") so the scope accurately describes the target;
locate the phrase `> 512GB DMMF` in the README entry and replace it with the
corrected unit/phrase to avoid misleading expectations.

In `@packages/client/tests/e2e/large-schema-generation/src/generate-schema.ts`:
- Around line 26-31: The test currently hardcodes targetModelCount (8000) and
uses startIdx/endIdx without asserting the generated schema is actually large
enough to hit the fallback path; add an explicit guard/assert after
computing/choosing targetModelCount (and before running the code under test)
that verifies the produced schema/input size or model count meets the minimum
required threshold (e.g., assert modelCount >= targetModelCount or assert
generatedDmmfSizeBytes > MIN_DMMF_BYTES) and fail the test early with a clear
message if not; update the variables referenced (targetModelCount, startIdx,
endIdx) to be used in the assertion so the test deterministically enforces the
large-input path.

In `@packages/internals/package.json`:
- Around line 83-85: Move `@streamparser/json` out of peerDependencies and into
dependencies in package.json so it is installed at runtime (getDmmf.ts uses
require('@streamparser/json') directly). Edit package.json to remove
"@streamparser/json" from the "peerDependencies" object and add it to
"dependencies" with the same version ("0.0.22"), update the lockfile/restore
installs, and run the test/build to ensure getDmmf.ts's runtime require succeeds
for large schemas.

In `@packages/internals/src/engine-commands/getDmmf.ts`:
- Around line 63-69: Update the docblock text to use the repo convention "Wasm"
instead of "WASM" wherever it appears in the comment describing the buffered API
(referencing get_dmmf_buffered(), DmmfBuffer, .len(), .read_chunk(), .free());
edit the docblock lines that currently read "WASM" to "Wasm" so the description
matches the project's WebAssembly abbreviation convention.
- Around line 58-60: Change isV8StringLimitError to accept error: unknown and
implement a safe type guard: first check error instanceof Error (or that it's a
non-null object) before reading properties, e.g. if (!(error instanceof Error))
return false; then safely access (error as any).code === 'ERR_STRING_TOO_LONG'
and return that boolean; also update the doc comment above the function to use
"Wasm API" instead of "WASM API".

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: bd416ce6-b97b-4838-85b1-b94415f23f7d

📥 Commits

Reviewing files that changed from the base of the PR and between 32fb24b and 6d28494.

📒 Files selected for processing (9)
  • packages/client/tests/e2e/large-schema-generation/_steps.ts
  • packages/client/tests/e2e/large-schema-generation/package.json
  • packages/client/tests/e2e/large-schema-generation/prisma.config.ts
  • packages/client/tests/e2e/large-schema-generation/prisma/schema.prisma
  • packages/client/tests/e2e/large-schema-generation/readme.md
  • packages/client/tests/e2e/large-schema-generation/src/generate-schema.ts
  • packages/client/tests/e2e/large-schema-generation/tsconfig.json
  • packages/internals/package.json
  • packages/internals/src/engine-commands/getDmmf.ts

Comment thread packages/client/tests/e2e/large-schema-generation/_steps.ts
Comment thread packages/client/tests/e2e/large-schema-generation/package.json
Comment thread packages/client/tests/e2e/large-schema-generation/readme.md Outdated
Comment thread packages/client/tests/e2e/large-schema-generation/src/generate-schema.ts Outdated
Comment thread packages/internals/package.json Outdated
Comment thread packages/internals/src/engine-commands/getDmmf.ts Outdated
Comment thread packages/internals/src/engine-commands/getDmmf.ts
@jacek-prisma jacek-prisma changed the title feat: dmmf 8 streaming with an E2E test feat: dmmf streaming with an E2E test Mar 24, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 24, 2026

🟢 All good, this PR is no longer uses custom engine.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
packages/internals/src/engine-commands/getDmmf.ts (1)

63-69: ⚠️ Potential issue | 🟡 Minor

Update the docblock to match the implementation.

This function no longer does any string decoding, and the new comment still says WASM. Please describe the Uint8Array + streaming-parser path and switch to the repo's Wasm spelling.

✏️ Suggested wording
- * Read DMMF from the handle-based buffered WASM API in chunks and parse via
- * chunked string decoding. This bypasses V8's string length limit by reading
+ * Read DMMF from the handle-based buffered Wasm API in chunks and parse it
+ * incrementally with a streaming JSON parser. This bypasses V8's string limit by reading
  * the DMMF as Uint8Array chunks from a caller-owned DmmfBuffer handle.

As per coding guidelines, **/*.{ts,tsx,js,jsx,md}: Use the correct WebAssembly abbreviation: 'Wasm', not 'WASM'.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/internals/src/engine-commands/getDmmf.ts` around lines 63 - 69,
Update the top docblock for getDmmf (the comment above the function that starts
"Read DMMF from the handle-based buffered WASM API...") to accurately describe
the current implementation: replace "WASM" with the repo-preferred "Wasm",
remove any mention of chunked string decoding, and state that the function reads
Uint8Array chunks from a DmmfBuffer handle and feeds them into a streaming
parser (using .len(), .read_chunk(), and .free()) to build the DMMF; keep
references to the required get_dmmf_buffered() from `@prisma/prisma-schema-wasm`
and the DmmfBuffer API (.len, .read_chunk, .free) so the docblock matches the
code path used by getDmmf.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/internals/src/engine-commands/getDmmf.ts`:
- Around line 91-96: The current bare catch around require('@streamparser/json')
swallows all errors; change it to catch the thrown error (e.g., catch (err)) and
only throw the custom "install" Error when err.code indicates the module is
missing (MODULE_NOT_FOUND / ERR_MODULE_NOT_FOUND); for any other error rethrow
the original err so initialization/compatibility issues in `@streamparser/json`
surface. Update the block that sets JSONParser (the
require('@streamparser/json') call and its catch) accordingly, preserving the
existing custom message for the missing-module case.

---

Duplicate comments:
In `@packages/internals/src/engine-commands/getDmmf.ts`:
- Around line 63-69: Update the top docblock for getDmmf (the comment above the
function that starts "Read DMMF from the handle-based buffered WASM API...") to
accurately describe the current implementation: replace "WASM" with the
repo-preferred "Wasm", remove any mention of chunked string decoding, and state
that the function reads Uint8Array chunks from a DmmfBuffer handle and feeds
them into a streaming parser (using .len(), .read_chunk(), and .free()) to build
the DMMF; keep references to the required get_dmmf_buffered() from
`@prisma/prisma-schema-wasm` and the DmmfBuffer API (.len, .read_chunk, .free) so
the docblock matches the code path used by getDmmf.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: c3b2f8f7-26aa-4e27-a550-86528a1c83e2

📥 Commits

Reviewing files that changed from the base of the PR and between 6d28494 and e78a795.

📒 Files selected for processing (6)
  • packages/client/tests/e2e/large-schema-generation/_steps.ts
  • packages/client/tests/e2e/large-schema-generation/readme.md
  • packages/client/tests/e2e/large-schema-generation/src/generate-schema.ts
  • packages/internals/helpers/build.ts
  • packages/internals/package.json
  • packages/internals/src/engine-commands/getDmmf.ts

Comment thread packages/internals/src/engine-commands/getDmmf.ts Outdated
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (2)
packages/internals/src/engine-commands/getDmmf.ts (1)

92-97: ⚠️ Potential issue | 🟡 Minor

Don't treat every require() failure as "not installed".

This still swallows initialization / compatibility errors inside @streamparser/json and replaces them with a misleading install hint. Only translate MODULE_NOT_FOUND / ERR_MODULE_NOT_FOUND for @streamparser/json; rethrow everything else.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/internals/src/engine-commands/getDmmf.ts` around lines 92 - 97, The
catch block around the dynamic require of `@streamparser/json` (where JSONParser
is assigned) currently treats any error as "not installed"; change it to only
convert the error into the install hint when the caught error is a
module-not-found for '@streamparser/json' (e.g. error.code ===
'MODULE_NOT_FOUND' || error.code === 'ERR_MODULE_NOT_FOUND' and/or the message
references '@streamparser/json'), and rethrow the original error for any other
failure (so initialization/compatibility errors from the package surface instead
of being masked).
packages/internals/package.json (1)

80-93: ⚠️ Potential issue | 🟠 Major

Remove @streamparser/json from dependencies—it should remain peer-only.

The package is listed in both dependencies and peerDependencies, which defeats the optional peer contract. Per design, @streamparser/json is only needed for users handling very large schemas (>~536MB DMMF) that hit V8's string length limit. It should remain declared only as an optional peer dependency, not as a direct dependency shipped to all consumers.

Suggested fix
   "dependencies": {
     "@prisma/config": "workspace:*",
     "@prisma/debug": "workspace:*",
     "@prisma/dmmf": "workspace:*",
     "@prisma/driver-adapter-utils": "workspace:*",
     "@prisma/engines": "workspace:*",
     "@prisma/fetch-engine": "workspace:*",
     "@prisma/generator": "workspace:*",
     "@prisma/generator-helper": "workspace:*",
     "@prisma/get-platform": "workspace:*",
     "@prisma/prisma-schema-wasm": "7.5.0-15.280c870be64f457428992c43c1f6d557fab6e29e",
     "@prisma/schema-engine-wasm": "7.5.0-15.280c870be64f457428992c43c1f6d557fab6e29e",
     "@prisma/schema-files-loader": "workspace:*",
-    "@streamparser/json": "^0.0.22",
     "arg": "5.0.2",
     "prompts": "2.4.2"
   },
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/internals/package.json` around lines 80 - 93, Remove
"@streamparser/json" from the "dependencies" section and ensure it remains only
in "peerDependencies" and "peerDependenciesMeta" (marked optional); specifically
edit the package.json to delete the "@streamparser/json" entry under
"dependencies" while leaving the existing "@streamparser/json" entry under
"peerDependencies" and its corresponding optional flag in "peerDependenciesMeta"
intact, then regenerate the lockfile (npm/yarn/pnpm install) so the dependency
graph reflects this change.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/internals/src/engine-commands/getDmmf.ts`:
- Around line 191-200: The buffered fallback catch in getDMMFBuffered needs to
detect Wasm panics and preserve the panic payload instead of always wrapping
everything in a GetDmmfError: inside the catch for getDMMFBuffered, call
isWasmPanic(bufferedError) first and if true construct/throw a RustPanic
containing the original panic payload (and call debugErrorType with type
'wasm-panic' or similar) so the real panic is propagated; otherwise treat it as
a normal error but avoid unsafe casting by checking if bufferedError instanceof
Error to read message (fallback to String(bufferedError) if not) when creating
the GetDmmfError and calling debugErrorType. Ensure references to
getDMMFBuffered, isWasmPanic, RustPanic, GetDmmfError, and debugErrorType are
used to locate and implement the fix.

---

Duplicate comments:
In `@packages/internals/package.json`:
- Around line 80-93: Remove "@streamparser/json" from the "dependencies" section
and ensure it remains only in "peerDependencies" and "peerDependenciesMeta"
(marked optional); specifically edit the package.json to delete the
"@streamparser/json" entry under "dependencies" while leaving the existing
"@streamparser/json" entry under "peerDependencies" and its corresponding
optional flag in "peerDependenciesMeta" intact, then regenerate the lockfile
(npm/yarn/pnpm install) so the dependency graph reflects this change.

In `@packages/internals/src/engine-commands/getDmmf.ts`:
- Around line 92-97: The catch block around the dynamic require of
`@streamparser/json` (where JSONParser is assigned) currently treats any error as
"not installed"; change it to only convert the error into the install hint when
the caught error is a module-not-found for '@streamparser/json' (e.g. error.code
=== 'MODULE_NOT_FOUND' || error.code === 'ERR_MODULE_NOT_FOUND' and/or the
message references '@streamparser/json'), and rethrow the original error for any
other failure (so initialization/compatibility errors from the package surface
instead of being masked).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: b66fb948-f5de-46a1-a40d-75b29826679d

📥 Commits

Reviewing files that changed from the base of the PR and between e78a795 and 73ffc78.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (2)
  • packages/internals/package.json
  • packages/internals/src/engine-commands/getDmmf.ts

Comment thread packages/internals/src/engine-commands/getDmmf.ts
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (3)
packages/internals/src/engine-commands/getDmmf.ts (3)

64-64: ⚠️ Potential issue | 🟡 Minor

Use Wasm (not WASM) in this docblock.

Line 64 uses WASM. Please rename it to Wasm to match repo conventions.

As per coding guidelines, **/*.{ts,tsx,js,jsx,md}: Use the correct WebAssembly abbreviation: 'Wasm', not 'WASM'.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/internals/src/engine-commands/getDmmf.ts` at line 64, The docblock
in getDmmf.ts uses the incorrect WebAssembly abbreviation "WASM"; update the doc
comment text (the line containing "WASM") to use "Wasm" instead so it matches
project conventions and coding guidelines—search for the docblock around the
getDmmf function or the comment starting "Read DMMF from the handle-based
buffered WASM API" and replace "WASM" with "Wasm".

93-98: ⚠️ Potential issue | 🟠 Major

Do not swallow non-module errors from require('@streamparser/json').

Line 95 uses a bare catch, so initialization/compatibility failures inside @streamparser/json are reported as “not installed”. Only map true module-not-found cases to the install message; rethrow everything else.

Proposed fix
     let JSONParser: typeof import('@streamparser/json').JSONParser

     try {
       JSONParser = (require('@streamparser/json') as typeof import('@streamparser/json')).JSONParser
-    } catch {
+    } catch (error) {
+      const requireError = error as NodeJS.ErrnoException
+      const missingModule =
+        (requireError.code === 'MODULE_NOT_FOUND' || requireError.code === 'ERR_MODULE_NOT_FOUND') &&
+        requireError.message?.includes('@streamparser/json')
+      if (!missingModule) {
+        throw error
+      }
       throw new Error(
         'Streaming JSON parser required for DMMF >= 500MB but `@streamparser/json` is not installed. Try adding it as a dependency.',
       )
     }
#!/bin/bash
set -euo pipefail

echo "Inspect dynamic require block:"
sed -n '90,101p' packages/internals/src/engine-commands/getDmmf.ts | cat -n

echo
echo "Check whether MODULE_NOT_FOUND / ERR_MODULE_NOT_FOUND are discriminated in this file:"
rg -n "MODULE_NOT_FOUND|ERR_MODULE_NOT_FOUND|@streamparser/json|catch\\s*\\(" packages/internals/src/engine-commands/getDmmf.ts -C2

Expected result: explicit code-based discrimination for missing module before throwing the custom install error.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/internals/src/engine-commands/getDmmf.ts` around lines 93 - 98, The
current try/catch around requiring '@streamparser/json' swallows all errors;
change the catch in the JSONParser import block so it only maps a genuine
"module not found" error to the install message and rethrows any other errors:
in the try that assigns JSONParser from require('@streamparser/json'), inspect
the caught error (e.g., err.code === 'MODULE_NOT_FOUND' || err.code ===
'ERR_MODULE_NOT_FOUND') and additionally verify the missing-module message
references '@streamparser/json' before throwing the custom install Error; for
any other err, rethrow it unchanged so initialization/compatibility failures
surface.

195-201: ⚠️ Potential issue | 🟠 Major

Preserve WasmPanic even when it is not an Error instance.

At Line 196, panic mapping is gated by error instanceof Error. If WasmPanic is a non-Error thrown object, it falls through to the generic unknown-error branch and loses panic payload.

Proposed fix
     } catch (error) {
-      if (error instanceof Error) {
+      if (isWasmPanic(error) || error instanceof Error) {
         debugErrorType({ type: 'wasm-error' as const, reason: '(get-dmmf-buffered wasm)', error })
         throw mapWasmPanicToGetDmmfError(error, '(get-dmmf-buffered wasm)')
       }

       throw new GetDmmfError({
#!/bin/bash
set -euo pipefail

echo "Inspect buffered fallback catch block:"
sed -n '188,207p' packages/internals/src/engine-commands/getDmmf.ts | cat -n

echo
echo "Inspect WasmPanic type/guard definitions:"
rg -n "export (type|interface) WasmPanic|function isWasmPanic|export function isWasmPanic" packages/internals/src/panic.ts -C3

Expected result: catch path should preserve WasmPanic regardless of instanceof Error.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/internals/src/engine-commands/getDmmf.ts` around lines 195 - 201,
The catch block currently only treats thrown Wasm panics when error instanceof
Error; update it to detect and preserve WasmPanic regardless of being an Error
by using the WasmPanic type guard (isWasmPanic) or equivalent check, then call
debugErrorType({ type: 'wasm-error', reason: '(get-dmmf-buffered wasm)', error
}) and throw mapWasmPanicToGetDmmfError(error, '(get-dmmf-buffered wasm)') for
WasmPanic values; leave the existing fallback that throws new GetDmmfError for
genuinely unknown/non-panic values. Ensure you reference debugErrorType,
mapWasmPanicToGetDmmfError, isWasmPanic, WasmPanic, and GetDmmfError when
locating the code to change.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/internals/src/engine-commands/getDmmf.ts`:
- Around line 226-237: mapWasmPanicToGetDmmfError currently hardcodes the panic
request label to '@prisma/prisma-schema-wasm get_dmmf', which misattributes
buffered-fallback panics; change mapWasmPanicToGetDmmfError signature to accept
a requestLabel (string) parameter and use that value when constructing the
RustPanic (replace the hardcoded request argument), then update its callers to
pass either '@prisma/prisma-schema-wasm get_dmmf' or '@prisma/prisma-schema-wasm
get_dmmf_buffered' as appropriate so diagnostics correctly indicate the source
path.

---

Duplicate comments:
In `@packages/internals/src/engine-commands/getDmmf.ts`:
- Line 64: The docblock in getDmmf.ts uses the incorrect WebAssembly
abbreviation "WASM"; update the doc comment text (the line containing "WASM") to
use "Wasm" instead so it matches project conventions and coding
guidelines—search for the docblock around the getDmmf function or the comment
starting "Read DMMF from the handle-based buffered WASM API" and replace "WASM"
with "Wasm".
- Around line 93-98: The current try/catch around requiring '@streamparser/json'
swallows all errors; change the catch in the JSONParser import block so it only
maps a genuine "module not found" error to the install message and rethrows any
other errors: in the try that assigns JSONParser from
require('@streamparser/json'), inspect the caught error (e.g., err.code ===
'MODULE_NOT_FOUND' || err.code === 'ERR_MODULE_NOT_FOUND') and additionally
verify the missing-module message references '@streamparser/json' before
throwing the custom install Error; for any other err, rethrow it unchanged so
initialization/compatibility failures surface.
- Around line 195-201: The catch block currently only treats thrown Wasm panics
when error instanceof Error; update it to detect and preserve WasmPanic
regardless of being an Error by using the WasmPanic type guard (isWasmPanic) or
equivalent check, then call debugErrorType({ type: 'wasm-error', reason:
'(get-dmmf-buffered wasm)', error }) and throw mapWasmPanicToGetDmmfError(error,
'(get-dmmf-buffered wasm)') for WasmPanic values; leave the existing fallback
that throws new GetDmmfError for genuinely unknown/non-panic values. Ensure you
reference debugErrorType, mapWasmPanicToGetDmmfError, isWasmPanic, WasmPanic,
and GetDmmfError when locating the code to change.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 822c4b7b-85b3-4136-9427-c6407cc278bc

📥 Commits

Reviewing files that changed from the base of the PR and between 73ffc78 and 04c30b1.

📒 Files selected for processing (1)
  • packages/internals/src/engine-commands/getDmmf.ts

Comment thread packages/internals/src/engine-commands/getDmmf.ts
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 24, 2026

size-limit report 📦

Path Size
packages/client/runtime/index-browser.js 2.29 KB (0%)
packages/client/runtime/index-browser.d.ts 3.37 KB (0%)
packages/cli/build/index.js 2.51 MB (+0.99% 🔺)
packages/client/prisma-client-0.0.0.tgz 26.81 MB (-0.01% 🔽)
packages/cli/prisma-0.0.0.tgz 13.51 MB (+0.04% 🔺)
packages/bundle-size/da-workers-libsql/output.tgz 1.33 MB (0%)
packages/bundle-size/da-workers-neon/output.tgz 1.39 MB (0%)
packages/bundle-size/da-workers-pg/output.tgz 1.38 MB (0%)
packages/bundle-size/da-workers-planetscale/output.tgz 1.33 MB (0%)
packages/bundle-size/da-workers-d1/output.tgz 1.3 MB (0%)

chris-tophers and others added 11 commits March 25, 2026 02:19
When prisma generate processes schemas that produce DMMF larger than
~536MB, the existing get_dmmf() WASM call fails with V8's hard-coded
string length limit (0x1fffffe8 characters). This adds automatic
fallback to the buffered DMMF API (get_dmmf_buffered + read_dmmf_chunk)
which returns data as chunked Uint8Array, bypassing the V8 string limit.

The fallback is transparent — it only activates when the V8 string limit
error is detected, so there is no behavior change for schemas that work
with the existing API.

Companion to: prisma/prisma-engines#5757
Fixes: #29111
Array.join('') on decoded string chunks also hits V8's string limit
for DMMF > 536MB. For large schemas, use @streamparser/json to process
Uint8Array chunks directly without creating a single large string.

Tested with a very large DMMF (>536MB) — streaming
parse completes in ~41s.
- Use instanceof RangeError (not Error) for V8 string limit detection
- Add typeof guard for buffered WASM exports with clear error message
- Extract 500MB threshold to STRING_JOIN_LIMIT named constant
- Cast bufferedError as Error in debugErrorType call
- Add @streamparser/json dependency to package.json
- Fix onValue callback types for @streamparser/json API
Update the TypeScript fallback code to use the new handle-based API from
prisma-engines#5757. Instead of three separate free functions
(get_dmmf_buffered, read_dmmf_chunk, free_dmmf_buffer) with implicit
global state, the API now returns a DmmfBuffer handle:

  const buffer = prismaSchemaWasm.get_dmmf_buffered(params)
  buffer.len()                    // total byte count
  buffer.read_chunk(offset, len)  // Uint8Array chunk
  buffer.free()                   // release WASM memory

Also addresses remaining CodeRabbit review items:
- Add parser.end() after final write() to finalize @streamparser/json
- Wrap require('@streamparser/json') with descriptive error on missing dep
@jacek-prisma jacek-prisma force-pushed the fix/dmmf-v8-string-limit-fallback-test branch from 9742339 to a96cd2c Compare March 24, 2026 19:19
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/client/tests/e2e/large-schema-generation/prisma.config.ts`:
- Around line 3-7: Add a short documentation comment above the exported Prisma
config to explain the fixture's purpose; specifically, add a docblock describing
that the default export is a test fixture Prisma configuration (using
defineConfig) pointing to an in-repo SQLite file (url: 'file:./dev.db') so
readers know this file is used for e2e large-schema generation tests and should
not be modified.

In `@packages/client/tests/e2e/large-schema-generation/readme.md`:
- Around line 3-4: Update the README wording to narrowly describe what the
fixture actually exercises: state that it validates that `prisma generate` can
complete for extremely large schemas and triggers the buffered DMMF fallback,
and remove claims about Prisma Client runtime behavior or performance;
specifically edit the README text that currently asserts handling of "memory
issues, performance degradation, or other problems" to only mention successful
generation and fallback behavior for large DMMF.

In `@packages/client/tests/e2e/large-schema-generation/src/generate-schema.ts`:
- Line 11: Remove or rewrite the low-signal inline comments that merely restate
the next line (the comment above the basic schema configuration, the one
adjacent to the TTL value 7200, and the similar short comments near the other
schema fields reported in the review); either delete those comments or replace
them with brief rationale explaining why specific choices were made (e.g., why
TTL is 7200) so that comments answer "why" not "what" for the schema generation
code (look for comments around the basic schema configuration block and the
schema field assignments referenced in the review).
- Around line 74-76: The file uses a CommonJS runtime check (require.main ===
module) which is fragile in ESM; replace that check with an ESM-safe entry point
comparison that uses import.meta.url and fileURLToPath to compare against
process.argv[1]. Update the script's entry guard where generateLargeSchema() is
invoked so it calls generateLargeSchema() only when
fileURLToPath(import.meta.url) equals process.argv[1], and import fileURLToPath
from 'url' as needed to perform the comparison.

In `@packages/internals/package.json`:
- Around line 80-94: Remove the direct dependency entry for "@streamparser/json"
from the "dependencies" section of package.json so it is only declared as an
optional peer dependency; leave the "peerDependencies" and
"peerDependenciesMeta" entries as-is (optional true). This matches the
conditional require() usage in getDmmf.ts (the code path that tries to
require("@streamparser/json") and handles the error) and ensures consumers opt
into installing the package rather than being forced by a direct dependency.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 22fd4dfa-ea68-44d2-969f-08c031fc536b

📥 Commits

Reviewing files that changed from the base of the PR and between 04c30b1 and a96cd2c.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (9)
  • packages/client/tests/e2e/large-schema-generation/_steps.ts
  • packages/client/tests/e2e/large-schema-generation/package.json
  • packages/client/tests/e2e/large-schema-generation/prisma.config.ts
  • packages/client/tests/e2e/large-schema-generation/prisma/schema.prisma
  • packages/client/tests/e2e/large-schema-generation/readme.md
  • packages/client/tests/e2e/large-schema-generation/src/generate-schema.ts
  • packages/client/tests/e2e/large-schema-generation/tsconfig.json
  • packages/internals/package.json
  • packages/internals/src/engine-commands/getDmmf.ts

Comment thread packages/client/tests/e2e/large-schema-generation/prisma.config.ts
Comment thread packages/client/tests/e2e/large-schema-generation/readme.md Outdated
Comment thread packages/client/tests/e2e/large-schema-generation/src/generate-schema.ts Outdated
Comment thread packages/internals/package.json Outdated
Comment thread packages/internals/package.json Outdated
@jacek-prisma jacek-prisma requested a review from aqrln March 25, 2026 12:13
@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented Mar 25, 2026

Merging this PR will not alter performance

✅ 17 untouched benchmarks
⏩ 30 skipped benchmarks1


Comparing fix/dmmf-v8-string-limit-fallback-test (b9cd953) with main (4131568)

Open in CodSpeed

Footnotes

  1. 30 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Comment thread packages/internals/src/engine-commands/getDmmf.ts
Comment thread packages/internals/src/engine-commands/getDmmf.ts Outdated
@jacek-prisma jacek-prisma requested a review from aqrln March 25, 2026 14:34
@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label Mar 25, 2026
@aqrln aqrln added this to the 7.6.0 milestone Mar 25, 2026
@jacek-prisma jacek-prisma merged commit 30f0af6 into main Mar 25, 2026
252 of 253 checks passed
@jacek-prisma jacek-prisma deleted the fix/dmmf-v8-string-limit-fallback-test branch March 25, 2026 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lgtm This PR has been approved by a maintainer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants