Skip to content

canvas: Add vello backend#36821

Merged
sagudev merged 3 commits intoservo:mainfrom
sagudev:vello_backend2
Jul 26, 2025
Merged

canvas: Add vello backend#36821
sagudev merged 3 commits intoservo:mainfrom
sagudev:vello_backend2

Conversation

@sagudev
Copy link
Member

@sagudev sagudev commented May 3, 2025

Add vello backend by implementing Backend traits in canvas crate (so this lives in canvas_paint_thread - embedded process). Current implementation uses normal wgpu, so we block on GPU work. Vello backend is gated behind vello feature and dom_canvas_vello_enabled pref.

Feature-wise this backend is on on par with raqote (sometimes better sometimes worse), but performance wise it's worse.

Known vello problems:

Known servo problems

  • bad performance due to blocking on GPU work
    • some get/put intensive tests TIMEOUT
  • proper shadow support (non-blocker as we already are living without it now)
    • support for rect shadow is there but unimplemented currently as that's the state in raqote

Testing: mach try vello will run normal WPT (with raqote) + vello_canvas subsuite that runs only on /html/canvas/element. All subsuite expectations are stored separately.
Fixes: #36823
Fixes: #35230

@sagudev sagudev added the A-content/canvas 2d canvas API label May 3, 2025
@sagudev

This comment was marked as outdated.

@gterzian
Copy link
Member

gterzian commented May 3, 2025

How does that sound?

Thanks for looking into this.

For my understanding, it is correct that you are now relying on Vello's internal use of wgpu? How does that interact, if at all, with our own use of wgpu-core in components/webgpu?

My initial proposal in zullip was to rather explore a different architecture, one that would still be using our own components/webgpu infrastructure.

As I remember the Vello codebase from a year ago, there was an interface for some sort of rendering backend, which then would use wgpu, but in theory we could implement our own(perhaps using our WebGPU dom objects), and then run vello directly in script and send commands to components/webgpu. The main benefits, in theory, were re-using same infra as our WebGPU implementation, and also the ability to remove the canvas thread. Perhaps it could also remove the " uses normal wgpu, so we block on GPU work" part.

Anyway, if you are making progress with this I think it's a good idea simply to remove the raquote dependency, although longer term perhaps it could be replaced by something close to my proposal?

@sagudev
Copy link
Member Author

sagudev commented May 3, 2025

Anyway, if you are making progress with this I think it's a good idea simply to remove the raquote dependency, although longer term perhaps it could be replaced by something close to my proposal?

That is eventual plan, but vello is missing some stuff.

My initial proposal in zullip was to rather explore a different architecture, one that would still be using our own components/webgpu infrastructure.
As I remember the Vello codebase from a year ago, there was an interface for some sort of rendering backend, which then would use wgpu, but in theory we could implement our own(perhaps using our WebGPU dom objects), and then run vello directly in script and send commands to components/webgpu. The main benefits, in theory, were re-using same infra as our WebGPU implementation, and also the ability to remove the canvas thread. Perhaps it could also remove the " uses normal wgpu, so we block on GPU work" part.

We actually just need to write custom wgpu backend that uses our webgpu objects (sends IPC messages to webgpu thread). Anyway, vello started going into different direction, exploring more hybrid (because current vello GPU requirements are high in terms of features due to compute shaders) and cpu only approaches (the last one is only as fallback). Given this new circumstances I think this makes more sense (although I would be interested in exploring for vello to still live in content process). But I have an idea how we can still use our webgpu thread but keeping vello out of content process (canvas_paint_thread would send IPC to webgpu). Why? To better support hybrid.

@sagudev
Copy link
Member Author

sagudev commented May 3, 2025

For my understanding, it is correct that you are now relying on Vello's internal use of wgpu?

Yes.

How does that interact, if at all, with our own use of wgpu-core in components/webgpu?

I have yet to check it, but I think it's all good (we run from different wgpu Instances) although we are currently using different wgpu versions which could mask the problem. But we need to force vulkan (llvmpipe) in CI or else it will use GLES which fights with servo's uses of GL (we might have same problem with GLES in webgpu).

@nicoburns
Copy link
Contributor

That is eventual plan, but vello is missing some stuff

What is missing?

although we are currently using different wgpu versions which could mask the problem

A Vello release with WGPU is planned within the next couple of weeks :) (and I'm pretty sure it will work just fine with the same version of WGPU).

@sagudev
Copy link
Member Author

sagudev commented May 3, 2025

That is eventual plan, but vello is missing some stuff

What is missing?

Filter, but this is known in vello: linebender/vello#476

although we are currently using different wgpu versions which could mask the problem

A Vello release with WGPU is planned within the next couple of weeks :) (and I'm pretty sure it will work just fine with the same version of WGPU).

Yeah, the only potencial problems can be statics in wgpu or some other global states that are shared between wgpu instances (or servo in case of GL). But that's probably just me being paranoid (although keep in mind that typical wgpu users do not have multiple instances across multiple threads).

@mrobinson
Copy link
Member

Sorry that I haven't gotten a chance to look at this yet. I hope to do so in the next couple days. This is very exciting! I would be in favor of removing the raqote backend even if some thing in Vello are missing. I think putting all of our effort behind a single canvas is a useful side-effect of removing support for raqote.

@sagudev
Copy link
Member Author

sagudev commented May 12, 2025

I would be in favor of removing the raqote backend even if some thing in Vello are missing. I think putting all of our effort behind a single canvas is a useful side-effect of removing support for raqote.

Currently, I think we should keep both for now:

  1. vello backend isn't complete yet, with the biggest problem being: canvas: Add vello backend #36821 (comment) (that I will prefer to defer to follow up as it will raise complexity significantly).
  2. Performance is currently pretty bad. Reasons for this are twofolded:
    1. We are blocking on rendering
    2. raquote updates drawtarget as soon as new command is issued, thus effectively making it async (we do not block content process on completion most of the time), while for vello we just record commands and do actual drawing when requested.

Also here are current test results: https://github.com/sagudev/servo/actions/runs/14968535017

EDIT:
3. Vello requires WebGPU capable graphic which is higher requirement than current gles2 capable device.

@sagudev sagudev force-pushed the vello_backend2 branch 2 times, most recently from 0131bdc to b906b78 Compare May 15, 2025 08:45
github-merge-queue bot pushed a commit that referenced this pull request Jun 4, 2025
I fell into trap of over-generalization in
#36793, but
#36821 showed `Cow<[u8]>` is all we
need (to reuse existing vec alloc or pass on a slice).

Testing: There are WPT tests, but it's just refactor so rust keeps us
safe.
Split of #36821

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>
github-merge-queue bot pushed a commit that referenced this pull request Jun 5, 2025
This is also required by spec:
https://html.spec.whatwg.org/multipage/canvas.html#ensure-there-is-a-subpath
and if we not ensure it vello will trigger debug asserts. Enforcing this
at `PathBuilderRef` is the lowest possible level that avoids misuse (and
avoids IPC messages if we were to do this in content process) while
still keeping it from backend.

Testing: There are tests in WPT
Split of #36821
try run: https://github.com/sagudev/servo/actions/runs/15449044694

---------

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>
@servo-wpt-sync
Copy link
Collaborator

🤖 Opened new upstream WPT pull request (web-platform-tests/wpt#52971) with upstreamable changes.

@servo-wpt-sync
Copy link
Collaborator

📝 Transplanted new upstreamable changes to existing upstream WPT pull request (web-platform-tests/wpt#52971).

github-merge-queue bot pushed a commit that referenced this pull request Jun 5, 2025
@sagudev sagudev marked this pull request as ready for review July 7, 2025 13:43
github-merge-queue bot pushed a commit that referenced this pull request Jul 12, 2025
This PR removes existing path(segment) abstractions in favor of
`kurbo::BezPath`, well actually wrapped `kurbo::BezPath`, to ensure
building of valid paths. This allows us better Path2D building in script
and doing all validation and segmentation there and also allows us
remove blocking is_point_in_path on Path2D as we can now do this in
script. Current path is still done on canvas thread side as it will be
harder to move to script (will be done as a follow up), but it now uses
this new path abstraction.

Using kurbo also allows us to ditch our manual svgpath parser with the
one provided by kurbo.

Same code is stolen from: #36821.

Testing: Existing WPT tests
Fixes: #37904

wpt run: https://github.com/sagudev/servo/actions/runs/16172191716

---------

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>
@sagudev sagudev force-pushed the vello_backend2 branch 3 times, most recently from 5d4d313 to 2870106 Compare July 14, 2025 14:26
@sagudev sagudev requested a review from mrobinson July 15, 2025 13:21
@sagudev sagudev force-pushed the vello_backend2 branch 5 times, most recently from 94bc0b0 to e6df2f0 Compare July 22, 2025 13:51
@sagudev sagudev force-pushed the vello_backend2 branch 2 times, most recently from 8e4d0de to 1c4cfbd Compare July 25, 2025 14:43
@sagudev
Copy link
Member Author

sagudev commented Jul 25, 2025

So many rebases to solve merge conflicts, most of which are actually my fault.

Copy link
Member

@jdm jdm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm impressed! The new backend is really readable. You've done great work to improve all the canvas code for this change.

sagudev added 3 commits July 26, 2025 06:10
Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>
Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>
Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>
@sagudev
Copy link
Member Author

sagudev commented Jul 26, 2025

Heh, another rebase.

I'm impressed! The new backend is really readable. You've done great work to improve all the canvas code for this change.

Yeah, that was the goal. Initial backend code was much larger and complicated. I was also able to upstream euclid <-> kurbo From/Into in kurbo, so that also helped.

@sagudev sagudev added this pull request to the merge queue Jul 26, 2025
Merged via the queue into servo:main with commit d678901 Jul 26, 2025
21 checks passed
@sagudev sagudev deleted the vello_backend2 branch July 26, 2025 05:24
@TimvdLippe
Copy link
Contributor

Congrats on landing this!

minghuaw pushed a commit to minghuaw/servo that referenced this pull request Aug 1, 2025
Add vello backend by implementing Backend traits in canvas crate (so
this lives in canvas_paint_thread - embedded process). Current
implementation uses normal wgpu, so we block on GPU work. Vello backend
is gated behind `vello` feature and `dom_canvas_vello_enabled` pref.

Feature-wise this backend is on on par with raqote (sometimes better
sometimes worse), but performance wise it's worse.

## Known vello problems:

- image roundtrip does not work (fixed in
linebender/vello#974)
- linebender/vello#1066 (fixed)
- clip layers are not working properly:
linebender/vello#1061
  - `/html/canvas/element/pixel-manipulation/2d.imageData.put.*`
  - `/html/canvas/element/path-objects/2d.path.clip.intersect.html`
- linebender/vello#1056
-
`/html/canvas/element/fill-and-stroke-styles/2d.gradient.interpolate.coloralpha.html`
- `kurbo::Cap::Butt` is defect (only visible with big lineWidth)
linebender/vello#1063
  - `/html/canvas/element/line-styles/2d.line.cross.html`
  - `/html/canvas/element/line-styles/2d.line.miter.acute.html`
- other lack of strong correct problems
(linebender/vello#1063 (comment)):
  - `/html/canvas/element/path-objects/2d.path.rect.selfintersect.html`
- There is currently no way to do put image properly in vello as we
would need to ignore all clips and other stuff (we try to work around
this on best effort basis)
linebender/vello#1088
  - `/html/canvas/element/pixel-manipulation/2d.imageData.put.*`
- precision problems
  - `/html/canvas/element/path-objects/2d.path.stroke.scale2.html`
  - `/html/canvas/element/path-objects/2d.path.arc.scale.1.html`

## Known servo problems

- bad performance due to blocking on GPU work
  - some get/put intensive tests `TIMEOUT`
- proper shadow support (non-blocker as we already are living without it
now)
- support for rect shadow is there but unimplemented currently as that's
the state in raqote

Testing: `mach try vello` will run normal WPT (with raqote) +
vello_canvas subsuite that runs only on `/html/canvas/element`. All
subsuite expectations are stored separately.
Fixes: servo#36823
Fixes: servo#35230

---------

Signed-off-by: sagudev <16504129+sagudev@users.noreply.github.com>
@sagudev sagudev self-assigned this Aug 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-content/canvas 2d canvas API

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support running vello in CI Add Vello as alternative 2d canvas backend

7 participants