feat: add markdown processor abstraction#16848
Conversation
🦋 Changeset detectedLatest commit: 62f090e The changes in this PR will be included in the next version bump. This PR includes changesets to release 418 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
| try { | ||
| const decodedImagePath = JSON.parse(imagePath.replaceAll('"', '"')); | ||
| const decodedImagePath = JSON.parse( | ||
| imagePath.replace(/&(?:#x22|quot);/g, '"').replace(/&(?:#x27|apos);/g, "'"), |
There was a problem hiding this comment.
Remark (somewhat unusually) uses numerical entities instead of named ones, other Markdown processors are more likely to use named entities.
| * falls back to its built-in handling for the known `unified` processor name. | ||
| * Third-party processors should provide this to enable MDX support. | ||
| */ | ||
| createMdxRenderer?( |
There was a problem hiding this comment.
MDX code in core, looks weird, but it's more likely than you think. Ever since we added MDX Astro has had some sort of MDX-specific layer somewhere. At this point, I find it doubtful we'd ever be dropping MDX support and so I figured this was the best API
(and down the line I'd like for the first party processors to also use that API, but it's a bigger conversation of making MDX fully built-in, ha)
ArmandPhilippot
left a comment
There was a problem hiding this comment.
I left a few nits for the configuration reference, and I think the changeset should be a bit more verbose both to hype the feature and to show that Sätteri is also available! If we don't tell users Sätteri exists, they will not try it. 😅
…or<T> Replace `MarkdownProcessorEntry` + `UnifiedProcessorDescriptor` with a single generic `MarkdownProcessor<TOptions>` interface, and rename the renderer to `MarkdownRenderer` so the two concepts have separate nouns. Move the MDX renderer types (`MdxRenderer`, `MdxRendererOptions`, `MdxRenderResult`) into `@astrojs/internal-helpers/markdown` so they can be referenced by processor packages without astro re-entering the dep graph. Also relaxes the over-narrow `AstroMetadata` shape (`scripts: never[]`, `propagation: 'none'`, …) so the unified MDX pipeline's compiler-produced metadata fits the same type the satteri pipeline produces.
Co-authored-by: Erika <3019731+Princesseuh@users.noreply.github.com>
Co-authored-by: Armand Philippot <git@armand.philippot.eu>
|
| 📦 Package | 📏 Size |
|---|---|
| @napi-rs/wasm-runtime@1.1.4 | 6.3 MB |
| @bruits/satteri-linux-x64-gnu@0.6.3 | 3.1 MB |
| @emnapi/core@1.9.1 | 1.9 MB |
| @emnapi/runtime@1.9.1 | 424.1 kB |
| @emnapi/wasi-threads@1.2.0 | 227.6 kB |
| satteri@0.6.3 | 218.5 kB |
Total size change: 12.1 MB
This PR contains the following updates: | Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) | |---|---|---|---| | [astro](https://astro.build) ([source](https://github.com/withastro/astro/tree/HEAD/packages/astro)) | [`6.3.8` → `6.4.2`](https://renovatebot.com/diffs/npm/astro/6.3.8/6.4.2) |  |  | --- ### Release Notes <details> <summary>withastro/astro (astro)</summary> ### [`v6.4.2`](https://github.com/withastro/astro/blob/HEAD/packages/astro/CHANGELOG.md#642) ##### Patch Changes - [#​16889](withastro/astro#16889) [`b94bcfd`](withastro/astro@b94bcfd) Thanks [@​Princesseuh](https://github.com/Princesseuh)! - Fixes a `plugins is not iterable` crash when using a pre-6.0 `@astrojs/mdx` alongside integrations (e.g. Starlight) that set `markdown.remarkPlugins`, `markdown.rehypePlugins`, or `markdown.remarkRehype`. - [#​16878](withastro/astro#16878) [`b9f6bb9`](withastro/astro@b9f6bb9) Thanks [@​fkatsuhiro](https://github.com/fkatsuhiro)! - Fixes an issue where on-demand (SSR) dynamic routes would return 404 when a prerendered dynamic route with the same URL pattern was sorted first alphabetically. In production builds with `@astrojs/node` adapter, if `[a_prebuild].astro` (prerender=true) came before `[b_ssr].astro` alphabetically, requests to URLs not in the prerendered route's static paths would 404 instead of falling through to the SSR route. The fix adds fallthrough logic so that when a prerendered dynamic route matches but can't serve the request, Astro tries subsequent matching routes. ### [`v6.4.1`](https://github.com/withastro/astro/blob/HEAD/packages/astro/CHANGELOG.md#641) ##### Patch Changes - [#​16883](withastro/astro#16883) [`eeb064c`](withastro/astro@eeb064c) Thanks [@​Princesseuh](https://github.com/Princesseuh)! - Restores the `astro/jsx/rehype.js` entry point so that older versions of `@astrojs/mdx` continue to work when used with Astro 6.x. This entry point will be removed in Astro 7.0. ### [`v6.4.0`](https://github.com/withastro/astro/blob/HEAD/packages/astro/CHANGELOG.md#640) [Compare Source](https://github.com/withastro/astro/compare/astro@6.3.8...astro@6.4.0) ##### Minor Changes - [#​16468](withastro/astro#16468) [`4cff3a1`](withastro/astro@4cff3a1) Thanks [@​matthewp](https://github.com/matthewp)! - Adds a new `preserveBuildServerDir` adapter feature Adapters can now set `preserveBuildServerDir: true` in their adapter features to keep the `dist/server/` directory structure for static builds, mirroring the existing `preserveBuildClientDir` option. This is useful for adapters that require a consistent `dist/client/` and `dist/server/` layout regardless of build output type. ```js setAdapter({ name: 'my-adapter', adapterFeatures: { buildOutput, preserveBuildClientDir: true, preserveBuildServerDir: true, }, }); ``` - [#​16848](withastro/astro#16848) [`f732f3c`](withastro/astro@f732f3c) Thanks [@​Princesseuh](https://github.com/Princesseuh)! - Adds a new `markdown.processor` configuration option, allowing you to choose an alternative Markdown processor. Websites with many Markdown/MDX files tend to be slow to build because the unified ecosystem (e.g., remark, rehype) is slow to process. This feature introduces the ability to replace this part of the build pipeline with another processor. The default processor is `unified()`. This means that existing configurations remain unchanged and your remark/rehype plugins continue to work. ```js // astro.config.mjs import { defineConfig } from 'astro/config'; import { unified } from '@​astrojs/markdown-remark'; import remarkToc from 'remark-toc'; export default defineConfig({ markdown: { processor: unified({ remarkPlugins: [remarkToc], }), }, }); ``` In addition to this new configuration option, Astro provides a new alternative processor based on Rust: [Sätteri](https://satteri.bruits.org/). You can choose to use it now by installing `@astrojs/markdown-satteri`, importing the `satteri()` processor, and adapting your existing configuration: ```js // astro.config.mjs import { defineConfig } from 'astro/config'; import { satteri } from '@​astrojs/markdown-satteri'; export default defineConfig({ markdown: { processor: satteri({ features: { directive: true }, }), }, }); ``` This processor does not support the remark and rehype plugins. This means you may need to convert them to [MDAST or HAST plugins](https://satteri.bruits.org/docs/plugins/) to retain your current functionality. The existing top-level `markdown.remarkPlugins`, `markdown.rehypePlugins`, `markdown.remarkRehype`, `markdown.gfm`, and `markdown.smartypants` options still work, but are now deprecated and will be removed in a future major update. The matching `remarkPlugins`, `rehypePlugins`, and `remarkRehype` options on the MDX integration are also deprecated for the same reason. To anticipate their removal, move them onto `unified({...})` (or your preferred plugin processor) : ```diff // astro.config.mjs import { defineConfig } from 'astro/config'; import remarkToc from 'remark-toc'; import rehypeSlug from 'rehype-slug'; + import { unified } from '@​astrojs/markdown-remark'; export default defineConfig({ markdown: { + processor: unified({ + remarkPlugins: [remarkToc], + rehypePlugins: [rehypeSlug], + remarkRehype: true, + gfm: true, + smartypants: true, + }), - remarkPlugins: [remarkToc], - rehypePlugins: [rehypeSlug], - remarkRehype: true, - gfm: true, - smartypants: true, }, }); ``` For more information on enabling and using this feature in your project, see our [Markdown guide](https://docs.astro.build/en/guides/markdown-content/). To give feedback on this new Rust processor, see the [Native Markdown / MDX parsing and processing RFC](withastro/roadmap#1364). ##### Patch Changes - [#​16468](withastro/astro#16468) [`4cff3a1`](withastro/astro@4cff3a1) Thanks [@​matthewp](https://github.com/matthewp)! - Skips the static preview server when an adapter provides its own `previewEntrypoint`, allowing the adapter to handle both static and dynamic routes - [#​16811](withastro/astro#16811) [`e0e26db`](withastro/astro@e0e26db) Thanks [@​matthewp](https://github.com/matthewp)! - Fixes `X-Forwarded-Host` and `X-Forwarded-Proto` headers being ignored when set in a custom `src/app.ts` fetch handler before creating `FetchState` - [#​16468](withastro/astro#16468) [`4cff3a1`](withastro/astro@4cff3a1) Thanks [@​matthewp](https://github.com/matthewp)! - Fixes the static preview server to respect `preserveBuildClientDir`, serving files from `build.client` instead of `outDir` when the adapter requires it - [#​16770](withastro/astro#16770) [`1e2aa11`](withastro/astro@1e2aa11) Thanks [@​matthewp](https://github.com/matthewp)! - Fixes a race condition where the Vite dep optimizer could lose React dependencies in dev mode when using Astro Actions - [#​16468](withastro/astro#16468) [`4cff3a1`](withastro/astro@4cff3a1) Thanks [@​matthewp](https://github.com/matthewp)! - Exempts internal routes (e.g. server islands) from `getStaticPaths()` validation, fixing server island rendering on static sites - [#​16468](withastro/astro#16468) [`4cff3a1`](withastro/astro@4cff3a1) Thanks [@​matthewp](https://github.com/matthewp)! - Fixes preview for static sites that contain non-prerendered routes. Previously, the preview command ignored SSR routes discovered during route scanning and always used the static preview server. - Updated dependencies \[[`f732f3c`](withastro/astro@f732f3c), [`f732f3c`](withastro/astro@f732f3c)]: - [@​astrojs/internal-helpers](https://github.com/astrojs/internal-helpers)@​0.10.0 - [@​astrojs/markdown-remark](https://github.com/astrojs/markdown-remark)@​7.2.0 </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yMDQuMSIsInVwZGF0ZWRJblZlciI6IjQzLjIwNS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->
Changes
The
markdown.processorpart of withastro/roadmap#1364, please see the RFC for details.Testing
Updated tests that relied on specific remark plugins and what not. Added tests using a custom processor to make sure it works from e2e
Docs
withastro/docs#13919