Skip to content

ggml: WebGPU backend host improvements and style fixing#14978

Merged
reeselevine merged 9 commits intoggml-org:masterfrom
reeselevine:master
Aug 4, 2025
Merged

ggml: WebGPU backend host improvements and style fixing#14978
reeselevine merged 9 commits intoggml-org:masterfrom
reeselevine:master

Conversation

@reeselevine
Copy link
Collaborator

@reeselevine reeselevine commented Jul 30, 2025

This PR adds a few improvements to the host-side setup for WebGPU, which should make it easier to add more operations/improve performance:

  • Creates a pool of parameter buffers which operations can pull from, allowing more in-flight operations.
  • Batches command submissions
  • Some refactoring of the WebGPU setup code to avoid repeating a bunch of code for new operations
  • Uses clang-format to format the WebGPU file, since this wasn't done before (sorry).

@github-actions github-actions bot added the ggml changes relating to the ggml tensor library for machine learning label Jul 30, 2025
@reeselevine
Copy link
Collaborator Author

reeselevine commented Jul 30, 2025

Looks like one test is failing, I'll investigate that.

guokoni

This comment was marked as spam.

Sign in to view

@guokoni

This comment was marked as spam.

Sign in to view
@reeselevine reeselevine mentioned this pull request Aug 1, 2025
Merged
@github-actions github-actions bot added the devops improvements to build systems and github actions label Aug 1, 2025
@reeselevine
Copy link
Collaborator Author

reeselevine commented Aug 1, 2025

A couple updates:

  • The CI in Dawn's upstream repository (https://github.com/google/dawn/actions/workflows/ci.yml) have been failing for a few weeks now, which causes issues for the CI here since it was relying on pulling their artifacts. So I created my own release of Dawn with the binaries from the last successful build here: https://github.com/reeselevine/dawn/releases/tag/v1.0.0, and am using that in the WebGPU CI here now.
  • I'm not 100% sure why the WebGPU Ubuntu CI test was failing in earlier commits on this PR, as I could not reproduce it locally. My best guess is that there was something broken in the interaction between Dawn's thread-safe implementation (which they call ImplicitDeviceSynchronization) and the simulated LLVMpipe Vulkan backend used by the CI here. Right now, it seems like combining the successful build from the previous Dawn commits and the updated code in this PR is stable, the CI runs successfully locally and has succeeded in a few successive commits.

Going forward, I'll roll my own releases of Dawn when necessary, and once the Dawn folks are able to get their CI working again. Otherwise the code here is ready for review/merging.

@CISC @ggerganov

ggerganov
ggerganov approved these changes Aug 2, 2025
View reviewed changes
Copy link
Member

@ggerganov ggerganov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going forward, I'll roll my own releases of Dawn when necessary, and once the Dawn folks are able to get their CI working again.

Sounds good 👍

@reeselevine reeselevine merged commit 587d011 into ggml-org:master Aug 4, 2025
47 checks passed
@CISC
Copy link
Collaborator

CISC commented Aug 4, 2025

@reeselevine It looks like the CI is stalling after a crash due to missing SET_ROWS support.

@reeselevine
Copy link
Collaborator Author

reeselevine commented Aug 4, 2025

@CISC yeah I noticed that after it was merged 😞. Simplest solution is to disable set rows for now, I have a commit ready: reeselevine@ae8edbf

But I also am working on adding support for SET_ROWS. If I get that done today I'll open a PR with support for it, otherwise I'll open a quick PR with that fix.

@ngxson ngxson mentioned this pull request Sep 5, 2025
Draft
blime4 referenced this pull request in blime4/llama.cpp Feb 5, 2026
@reeselevine
ggml: WebGPU backend host improvements and style fixing (#14978)
037d143 
* Add parameter buffer pool, batching of submissions, refactor command building/submission

* Add header for linux builds

* Free staged parameter buffers at once

* Format with clang-format

* Fix thread-safe implementation

* Use device implicit synchronization

* Update workflow to use custom release

* Remove testing branch workflow
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ggerganov ggerganov ggerganov approved these changes

+1 more reviewer

@guokoni guokoni guokoni left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

devops improvements to build systems and github actions ggml changes relating to the ggml tensor library for machine learning

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@reeselevine @guokoni @CISC @ggerganov