Docs: Add client-side media processing documentation#75895
Docs: Add client-side media processing documentation#75895adamsilverstein wants to merge 22 commits into
Conversation
Document the upcoming capability detection areas including credentialless iframe support, device memory, network conditions, and Web Worker checks. Notes that browsers without credentialless support will have client-side media disabled by default, with a server-side filter opt-in.
Architecture explanation, developer how-to guide, editor filters reference, and make/core announcement draft for the client-side media processing feature shipping in WP 7.0.
- Fix wasm-vips links to point to canonical kleisauke/wasm-vips repo - Add wp_ prefix to filter name in feature-detection docblock - Use wp.apiFetch instead of hardcoded /wp-json/ path in sideload example Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Document the finalize REST endpoint, editor.media.imageQuality JS filter, and server-side hook compatibility added in PR #74913 across architecture, how-to, and Make Core post docs. Also fixes a typo ("eprocessing") in the Make Core post.
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
- Replace COEP/COOP cross-origin isolation with Document-Isolation-Policy (Chromium 137+ only; Firefox/Safari not supported) - Remove outdated "not yet wired through" note on imageQuality hook - Add client_side_supported_mime_types PHP filter documentation - Add replace_file parameter to sideload endpoint docs - Update browser compatibility table and troubleshooting section - Update feature detection table to match current implementation
Adopt trunk's feature-detection docblock listing — it accurately reflects implemented checks (hardware concurrency). Drop the credentialless-iframe bullet since no such check exists in the code.
|
Flaky tests detected in 47c6a81. 🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/26815437455
|
The feature shipping target moved from 7.0 to 7.1 (per #76756). Bring the architecture explanation and how-to guide up to date with the post-7.0-cycle state of the code: - Reposition introductions around 7.1 with 7.0 as the groundwork cycle. - HEIC/HEIF: document the canvas-based fallback path (createImageBitmap, HTMLImageElement+OffscreenCanvas, HEIC container parsing + WebCodecs VideoDecoder), JPEG companion file, and isHeicCanvasSupported() gate. - AVIF: note the wp_prevent_unsupported_mime_type_uploads bypass when generate_sub_sizes=false (#76371). - Batch thumbnail generation via image.copyMemory() / thumbnailImage() (#76979). - Sub-size deduplication via image_size: string|string[] on the sideload route (#77036). - Single VIPS instance via promise caching (#76780). - Build-output trimming: remove vips-jxl.wasm (#76639), skip non-min worker (#76615), skip WASM-inlined sourcemaps (#75993). - COI: <img> excluded from cross-origin attribute injection (#76618). - Loosened feature-detection thresholds: 2 cores, 3g (#76616). - Post-saving lock fix: Save Draft now respects the lock (#76973). - convert_format declared as boolean on sideload route (#77565). - Fix incorrect REST index settings list (only image_sizes and image_size_threshold are exposed). - Remove references to client_side_supported_mime_types filter (PR #76549 was closed unmerged); state the supported MIME set is fixed at CLIENT_SIDE_SUPPORTED_MIME_TYPES. Refs #75111.
The feature ships in 7.1 instead of 7.0 (per #76756). Reframe the post as a 7.1 announcement with a dedicated 'What's new in 7.1' section listing HEIC support, end-to-end AVIF uploads, batch thumbnail generation, sub-size deduplication, single VIPS instance, save lock fix, loosened device requirements, <img> COI exclusion, smaller build output, and the convert_format boolean coercion fix. Each item is linked to its merged PR. Move the original 7.0 cycle deliverables under 'What changed during 7.0' so the historical context stays available for the announcement audience. Update technical overview to use the actual function name (gutenberg_is_client_side_media_processing_enabled), reflect that DIP replaces COEP/COOP, and remove the inaccurate note that the imageQuality hook isn't wired up. Drop the make-believe client_side_supported_mime_types filter; PR #76549 was closed unmerged. State that the supported MIME set is fixed at CLIENT_SIDE_SUPPORTED_MIME_TYPES. Update browser compatibility table to reflect the DIP-based gating (Chromium 137+, no Firefox/Safari) and add a feature detection thresholds section.
The handbook generator derives slugs from filename basenames, so docs/how-to-guides/client-side-media.md and docs/explanations/architecture/client-side-media.md collided on the slug "client-side-media" and broke docs:gen in CI. Rename the architecture file to client-side-media-architecture.md and update all internal cross-references plus the public-URL references in the Make/Core draft post.
Run docs:build to add manifest entries for the new how-to guide and architecture explanation pages, fixing the check-local-changes CI step.
Adds a Frequently Asked Questions section addressing common public misconceptions raised on issue #76756 — bandwidth-vs-CPU/RAM framing, the 'never trust the client' security objection, fallback behavior, hook compatibility, format conversion, and browser support.
|
@adamsilverstein this can be removed from the 7.0 project board, right? |
yes, moved this and all other client-side media tickets that weren't already done to the 7.1 board. |
Merge the 7.0 and 7.1 sections of the Make Core post into a single "What's included" summary so first-time readers (who only see the feature in 7.1) aren't asked to track which version landed what. Replace inaccurate "Faster uploads" framing with "Faster downloads" benefit — the client uploads the original plus every sub-size, so total bytes go up; the actual win is libvips' better compression producing smaller files for site visitors. Add a Mermaid flow diagram in the architecture doc showing both client- and server-side paths converging at wp_generate_attachment_metadata. Add a HEIC browser-support note on first mention (Chromium on macOS and Windows-with-HEVC, Safari on macOS; Firefox unsupported), and correct the WebCodecs HEVC fallback to say Chromium and cover Windows via the Microsoft HEVC Video Extension. Strip out "how we got here" PR-link references throughout the architecture doc — sub-size dedup, <img> CORS exclusion, convert_format boolean, image_size array, post-save lock — these were process artifacts, not what the feature does today. Drop the WASM bundle size figures and the CPU threshold tuning history for the same reason. Tighten the WASM module loading bullets.
…client-side media docs The filter actually fires twice during a client-side upload — once with context 'create' during the initial upload (WP core's wp_generate_attachment_metadata() runs even when intermediate_image_sizes_advanced is filtered to empty) and again with 'update' at finalize. Earlier wording suggested the filter was bypassed initially and only ran at finalize, which is wrong. The double-fire pattern matches how WordPress already handles big-image uploads on the server, where sub-size generation is deferred and triggers a second 'update' pass. Frame the contract that way across all three docs and tell plugin authors to be idempotent — there's nothing new for them to handle that they haven't already accommodated for big-image uploads. Other corrections in the same pass: - Soften the "security model is unchanged" FAQ in the make/core post: AVIF uploads from the client-side path do bypass wp_prevent_unsupported_mime_type_uploads when generate_sub_sizes is false, which is a deliberate exception worth surfacing. - Fix HEVC fallback wording in the how-to guide (Chrome 107+ on macOS → Chromium 107+ via the platform codec, covering Windows HEVC Video Extension as well). - Note Safari's HEIC canvas fallback in browser-support tables across all three docs (architecture and how-to guide both said flat "Not supported"). - Add packages/vips/src/worker.ts to the source-files list — it's the actual entry point loader.ts dynamically imports. - Align image quality figure to "0–1, default 0.82" everywhere (architecture doc had "82%"). - Drop residual historical PR link in the how-to guide troubleshooting table (#76973 Save Draft fix). - Drop the "graduated from a Gutenberg experiment" version-history framing from the architecture doc and make/core post. - Drop the historical 7.0 iteration link from the make/core post related-tracking-issues list.
…ype set Add a "See also" footer to the architecture doc pointing to the how-to guide and the editor-filters reference, so plugin authors who land on the architecture page have an obvious next step toward actionable guidance. Add a callout in the editor-filters reference noting that CLIENT_SIDE_SUPPORTED_MIME_TYPES is fixed and not exposed via a filter. Plugin authors looking for a "supported types" filter will land in the filter reference first; documenting the absence prevents a hunt that the make/core post and how-to guide already mention but the reference itself didn't.
- Split the long lede sentence in the make/core post explaining the finalize 'update' pass into two sentences for readability. - Cross-reference the security-FAQ MIME-bypass detail from the AVIF Key benefits bullet so readers connecting "accepts AVIF" to "how" can find the explanation in one click. - Update the architecture upload-flow Mermaid label to "wp_generate_attachment_metadata context: 'update'" so attentive readers don't infer the filter only fires at finalize. - Move the dense parenthetical in the how-to guide finalize section into a trailing sentence, keeping the cross-link to the server-side compatibility section.
…ision Reflect four client-side media PRs merged after these docs were last revised, keeping the architecture and how-to guides in sync with current behavior: - #76707: document the disabled libvips operation cache (Cache.max(0)) and the 50-operation worker recycling that bound WASM linear-memory growth, plus the globalThis.__vipsDebug hook for capturing the suppressed vips output. - #74903: document the sideload endpoint's image-dimension validation and its 400 error responses (rest_upload_dimension_mismatch and friends). - #78359: note that the finalize endpoint returns the refreshed attachment, which the editor uses to keep the block URL (and front-end srcset) in sync. - #78038: clarify that sub-size filenames derive from the original basename and only the scaled full-size copy carries the -scaled suffix.
Add a Requirements section near the top of the how-to guide covering browser support, required browser capabilities, and device/network thresholds (>2GB RAM, >=2 cores, faster-than-2g). The system thresholds were previously only discoverable by scanning the troubleshooting table. Remove the now-redundant trailing Browser compatibility section, which the new section subsumes.
Co-authored-by: Aaron Jorbin <aaronjorbin@users.noreply.github.com>
- Note automatic retry with exponential backoff for resilient uploads (#76765) - Reframe cross-origin isolation as a capability (SharedArrayBuffer) extenders can use, not an implementation detail - Clarify that Firefox/Safari are unsupported for the full pipeline, with a pointer to the compatibility section - Add publication-date browser-version context and a Chrome Platform Status link for tracking Document-Isolation-Policy support - Fix the Safari compatibility row wording and stray parenthesis - Merge the duplicated 'Feature detection thresholds' and 'Known limitations' sections into one - Explain why the AVIF MIME-check exception carries minimal security risk
|
Thanks for the thorough review @aaronjorbin and @westonruter. I pushed 47c6a81 addressing the feedback on the make/core draft:
|
Summary
Adds comprehensive documentation for the Client-Side Media Processing feature shipping in WordPress 7.1. The underlying infrastructure landed during the 7.0 cycle when the feature graduated from a Gutenberg experiment; the 7.1 release adds HEIC support, AVIF end-to-end uploads, performance improvements, and finishes hardening before the public release.
docs/explanations/architecture/client-side-media-architecture.md): How the system works — the three-package split (@wordpress/upload-media,@wordpress/vips,@wordpress/media-utils), upload queue and operation chaining, WebAssembly/libvips pipeline, Cross-Origin Isolation and SharedArrayBuffer requirements, browser feature detection, and graceful fallback to server-side processing.docs/how-to-guides/client-side-media.md): Practical guide for plugin/theme developers — disabling the feature, customizing image processing via existing WordPress filters, handling compatibility with metadata hooks, and debugging.docs/reference-guides/filters/editor-filters.md): New "Client-side media processing" section documenting thewp_client_side_media_processing_enabledPHP filter, thegenerate_sub_sizesandconvert_formatREST API parameters, and the existing core filters that the client-side pipeline respects (big_image_size_threshold,image_editor_output_format,image_save_progressive,wp_image_maybe_exif_rotate).docs/contributors/client-side-media-make-core-post.md): Announcement post draft for make.wordpress.org/core, including a FAQ covering the bandwidth-vs-CPU framing, the security model, browser fallback behavior, hook compatibility, and the Firefox/Safari exclusion rationale.docs/manifest.jsonanddocs/toc.jsonregenerated to surface the new pages in the handbook.Closes #75111
Test plan