Skip to content

[lexical-code-shiki][nextjs-code-shiki] Bug Fix: Externalize shiki dependencies in the published bundle#8514

Merged
etrepum merged 7 commits into
facebook:mainfrom
etrepum:claude/fix-shiki-bundling-ncqxt
May 14, 2026
Merged

[lexical-code-shiki][nextjs-code-shiki] Bug Fix: Externalize shiki dependencies in the published bundle#8514
etrepum merged 7 commits into
facebook:mainfrom
etrepum:claude/fix-shiki-bundling-ncqxt

Conversation

@etrepum

@etrepum etrepum commented May 14, 2026

Copy link
Copy Markdown
Collaborator

Description

@lexical/code-shiki declares shiki, @shikijs/core, @shikijs/engine-javascript, @shikijs/langs, and @shikijs/themes as runtime dependencies, but scripts/build.mjs was inlining all of them into the published bundle. The dev artifact packages/lexical-code-shiki/dist/LexicalCodeShiki.mjs shipped a private copy of the shiki sources at ~9.8 MB even though the same packages would be installed from npm via the declared dependencies. Apps that depend on @lexical/code-shiki therefore paid the shiki cost twice and the in-bundle copy could drift from the npm copy.

This PR adds shiki and @shikijs to the thirdPartyExternals list in scripts/build.mjs so Rollup marks shiki/* and @shikijs/* external for the published bundle. The output now imports them from node_modules at runtime, including the dynamic @shikijs/langs/<lang> and @shikijs/themes/<theme> imports shiki uses for on-demand language/theme loading.

The previous treeshake: 'recommended' carve-out for @lexical/code-shiki was a workaround for an oniguruma-to-es feature-detection bug that only surfaced because the engine was being inlined. With shiki externalized that workaround is no longer relevant, so the per-package treeshake selector is simplified back to the default 'smallest' for everything except @lexical/code-prism (which still inlines prismjs as before).

Shiki is kept as a regular dependency rather than promoted to peerDependencies. This matches the existing repo convention — peers are reserved for ecosystem singletons whose duplication actively breaks things (react, react-dom, yjs); other internal third-party deps (@floating-ui/react, react-error-boundary, prismjs) stay as regular dependencies. Shiki's only mutable state is the createHighlighterCoreSync(...) instance owned by @lexical/code-shiki itself, so two on-disk copies of shiki would not silently mis-render.

To prevent this from regressing again, the existing lexical-esm-nextjs release-artifact fixture is moved to a top-level example at examples/nextjs-code-shiki (@lexical/nextjs-code-shiki-example) and rewritten to use the Lexical extension system end-to-end:

  • The editor is built from a single root extension whose dependencies are RichTextExtension, HistoryExtension, AutoFocusExtension, and a small example-owned CodeShikiDemoExtension. No legacy LexicalComposer / *Plugin wrappers.
  • CodeShikiDemoExtension pulls in CodeShikiExtension as a dependency, uses $initialEditorState to seed the document from getCodeLanguageOptions(), and from its register hook awaits loadCodeLanguage('python') to exercise shiki's dynamic @shikijs/langs/<lang> import path. On resolve it inserts a bold Loaded: python node. The async path is gated by an ssr config field so it only runs in the browser.
  • The example renders SSR-friendly output via a small SSRContentEditable component that uses withDOM from @lexical/headless/dom to render the seeded editor state into HTML on the server (via a throwaway root element and innerHTML), injects it with dangerouslySetInnerHTML + suppressHydrationWarning, and rebinds the real DOM root with a ref on the client.
  • The unused ToolbarPlugin, TreeViewPlugin, public/icons/* SVG set, and tailwindcss/postcss/autoprefixer config/devDeps are removed. styles.css trims to just the rules the simplified editor uses.

prepare-release.test.mjs globs both examples/*/package.json and scripts/__tests__/integration/fixtures/*/package.json, so moving the fixture under examples/ keeps the same install-tarball/next build/Playwright flow without changes to the test harness. The Astro fixture is left in scripts/__tests__/integration/fixtures/lexical-esm-astro-react, so @lexical/code (Prism path) continues to have release-tarball coverage too.

Bundle size impact for @lexical/code-shiki:

Build Before After
LexicalCodeShiki.mjs (dev) ~9.8 MB ~18 KB
LexicalCodeShiki.js (dev) ~9.8 MB ~18 KB
LexicalCodeShiki.{mjs,js} (prod) inlined ~6 KB

Closes #8515

Test plan

Before

packages/lexical-code-shiki/dist/LexicalCodeShiki.mjs is ~9.8 MB and contains inlined shiki source (ShikiError, clone, oniguruma-to-es feature detection, etc.) instead of import statements for the declared dependencies.

After

  • node scripts/build.mjs produces ~18 KB LexicalCodeShiki.{mjs,js} whose only shiki references are import { ... } from '@shikijs/core', from '@shikijs/engine-javascript', from 'shiki/langs', and from 'shiki/themes'.
  • pnpm run clean && node scripts/build.mjs --prod produces ~6 KB minified bundles with the same external imports.
  • pnpm run test-unit — 3086 passed, 1 skipped.
  • pnpm exec vitest --project scripts-unit --no-watch — 588 passed.
  • pnpm run tsc — clean.
  • pnpm run lint and pnpm exec prettier --check — clean.
  • @lexical/code-prism bundle is unchanged (still inlines prismjs with treeshake: false).
  • prepare-release.test.mjs will exercise the new examples/nextjs-code-shiki example: install the @lexical/code-shiki-0.44.0.tgz (and friends) tarballs, run next build (which both validates that shiki / @shikijs/* resolve as external dependencies under Next.js / webpack, and renders the seeded HTML on the server via withDOM), then run npm run test (Playwright) against next start. Assertions:
    • Registered:.*typescript — synchronous bundledLanguagesInfo list resolved through the published bundle, rendered in both the SSR HTML and post-hydration.
    • Loaded: python — the client-side dynamic import('@shikijs/langs/python') inside shiki resolved at runtime, confirming that @shikijs/langs stayed external in the published bundle rather than getting inlined by Rollup.

…ublished bundle

`shiki`, `@shikijs/core`, `@shikijs/engine-javascript`, etc. are declared as
runtime dependencies of `@lexical/code-shiki` but were still being inlined
by Rollup, producing a ~9.8 MB bundle that shipped its own copy of the
shiki sources. Treat `shiki` and `@shikijs/*` as third-party externals so
the published bundle imports them from node_modules and shrinks to ~18 KB
(prod ~6 KB). The previous `treeshake: 'recommended'` workaround for the
inlined `oniguruma-to-es` feature detection is no longer needed and is
reverted to `'smallest'`.
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 14, 2026
@vercel

vercel Bot commented May 14, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
lexical Ready Ready Preview, Comment May 14, 2026 5:11pm
lexical-playground Ready Ready Preview, Comment May 14, 2026 5:11pm

Request Review

…@lexical/code-shiki

Switch the lexical-esm-nextjs integration fixture from `@lexical/code`
(which transitively exercises Prism) to `@lexical/code-shiki`, and load
a non-default language (`python`) via `loadCodeLanguage` to exercise
the dynamic `@shikijs/langs/<lang>` import path through Next.js. This
gives the `prepare-release` integration tests release-tarball coverage
for the shiki bundle, which would have caught a regression like
shiki being inlined or its dynamic imports being inadvertently
externalized without a corresponding declared dependency. The Astro
fixture is left on the Prism path so both highlighters remain
covered by release-artifact tests.
… a top-level example and convert to extensions

Move scripts/__tests__/integration/fixtures/lexical-esm-nextjs to
examples/nextjs-code-shiki and rename the package to
@lexical/nextjs-code-shiki-example. The prepare-release integration
test still picks it up automatically: prepare-release.test.mjs globs
both `examples/*/package.json` and
`scripts/__tests__/integration/fixtures/*/package.json`, so the same
release-tarball install/build/Playwright flow now runs against the
example.

While moving, port the example to the Lexical extension system:

- Drop the legacy LexicalComposer/RichTextPlugin/HistoryPlugin/
  AutoFocusPlugin wiring and the custom CodeHighlightingPlugin
  useEffect. The editor is now built from a single root extension
  whose dependencies are RichTextExtension, HistoryExtension,
  AutoFocusExtension, and a small example-owned
  CodeShikiDemoExtension.
- CodeShikiDemoExtension uses $initialEditorState to seed the
  "Registered: ..." text from getCodeLanguageOptions() and a
  register() hook to await loadCodeLanguage('python'), appending
  "Loaded: python" once the dynamic @shikijs/langs/python import
  resolves. This is the load-bearing assertion that confirms
  @shikijs/* are external in the published @lexical/code-shiki bundle
  rather than inlined by Rollup.
- Remove the unused ToolbarPlugin, TreeViewPlugin, public/icons SVG
  set, and tailwind/postcss/autoprefixer config (none of which were
  actually used). Trim styles.css to just the rules the simplified
  editor still references.
- Update the page title to "Lexical Next.js Code Shiki Example" and
  update the Playwright assertion accordingly.

The Astro fixture is left in scripts/__tests__/integration/fixtures so
@lexical/code (Prism path) keeps release-artifact coverage too.
@etrepum etrepum changed the title [lexical-code-shiki] Bug Fix: Externalize shiki dependencies in the published bundle [lexical-code-shiki][nextjs-code-shiki] Bug Fix: Externalize shiki dependencies in the published bundle May 14, 2026
claude and others added 2 commits May 14, 2026 17:10
…: assertion

The SSR commit changed CodeShikiDemoExtension to prepend the bold
`Loaded: python` node via `$getRoot().selectStart().insertNodes(...)`,
so the first .editor-paragraph's text content now starts with
`Loaded: python\nRegistered:...` instead of `Registered:...`. Drop
the `^` anchor on the regex so the assertion stays a substring test
for `Registered:.*typescript` regardless of insertion order.
@etrepum etrepum added this pull request to the merge queue May 14, 2026
Merged via the queue into facebook:main with commit 428e5c1 May 14, 2026
69 of 73 checks passed
@etrepum etrepum deleted the claude/fix-shiki-bundling-ncqxt branch May 19, 2026 17:54
@etrepum etrepum mentioned this pull request May 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. extended-tests Run extended e2e tests on a PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: @lexical/code-shiki includes every Shiki import causing avoidable bundle size inflation

3 participants