Skip to content

sycl: unify unary kernels with a generic implementation and enable wide operator support#17213

Merged
CISC merged 15 commits intoggml-org:masterfrom
shani-f:feature/sycl-unary-generic
Nov 15, 2025
Merged

sycl: unify unary kernels with a generic implementation and enable wide operator support#17213
CISC merged 15 commits intoggml-org:masterfrom
shani-f:feature/sycl-unary-generic

Conversation

@shani-f
Copy link
Contributor

@shani-f shani-f commented Nov 12, 2025

Summary

Adds a generic unary implementation for the SYCL backend, allowing many unary operators to share a single optimized execution path.
The implementation matches the behavior of the existing CPU unary kernels.

Changes

  • Added ggml_sycl_op_unary generic function
  • Updated unary dispatch in element_wise.cpp
  • Removed per-op SYCL kernels (ABS, SGN, NEG, STEP, etc.)
  • Updated documentation in:
    • docs/ops.md
    • docs/ops/SYCL.csv

Implementation

  • One templated kernel handles all unary ops
  • Supports 4-D tensors and non-contiguous views
  • Supports F16 and F32 data types
  • Uses dispatch_ggml_sycl_op_unary with parallel_for
  • Eliminates duplicated indexing logic across operators

Supported Ops

  • ABS
  • SGN
  • NEG
  • STEP
  • RELU
  • HARDSIGMOID
  • TANH
  • GELU
  • SILU
  • SIGMOID
  • HARDSWISH
  • GELU_QUICK
  • GELU_ERF
  • EXP
  • ELU

Testing

  • All supported unary ops pass test-backend-ops
  • Verified correctness on contiguous + non-contiguous tensors
  • Matches CPU results

Performance

  • Single optimized unary path for all ops
  • Reduced kernel count and maintenance complexity
  • Same SYCL scheduling style as existing ops

Compatibility

  • Works on OpenCL and Level Zero devices
  • No changes required for CPU fallback
  • Follows SYCL backend design conventions

@github-actions github-actions bot added documentation Improvements or additions to documentation ggml changes relating to the ggml tensor library for machine learning SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language labels Nov 12, 2025
@DajanaV DajanaV mentioned this pull request Nov 12, 2025
Closed
@shani-f
Copy link
Contributor Author

shani-f commented Nov 12, 2025

Hello @CISC @NeoZhangJianyu,
The ggml-ci-x64-cpu-low-perf test failed, but it’s CPU-only and unrelated to my SYCL changes.
What’s the next step toward merging?
Thanks!

NeoZhangJianyu
NeoZhangJianyu approved these changes Nov 13, 2025
View reviewed changes
Copy link
Collaborator

@NeoZhangJianyu NeoZhangJianyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's good job!
Thank you!

@NeoZhangJianyu
Copy link
Collaborator

NeoZhangJianyu commented Nov 13, 2025

Hello @CISC @NeoZhangJianyu, The ggml-ci-x64-cpu-low-perf test failed, but it’s CPU-only and unrelated to my SYCL changes. What’s the next step toward merging? Thanks!

We could merge directly after the conflicts are fixed.

@NeoZhangJianyu
Copy link
Collaborator

NeoZhangJianyu commented Nov 14, 2025

@shani-f
Could you fix the conflicts?

Thank you!

@shani-f
Copy link
Contributor Author

shani-f commented Nov 14, 2025

Hello @NeoZhangJianyu,
Yesterday I resolved all conflicts and the branch was fully up to date.
The new conflicts came from the latest merge into master, and I’ve resolved those as well.
Thanks!

CISC
CISC reviewed Nov 14, 2025
View reviewed changes
@DajanaV DajanaV mentioned this pull request Nov 15, 2025
Open
CISC
CISC reviewed Nov 15, 2025
View reviewed changes
@CISC
Copy link
Member

CISC commented Nov 15, 2025

Regen the CSV, at least TOPK_MOE should no longer be there.

@shani-f
Copy link
Contributor Author

shani-f commented Nov 15, 2025 via email •
edited
Loading

@CISC
Copy link
Member

CISC commented Nov 15, 2025

Is everything correct and finalized now?

I really did mean that you should regen it, there are more ops in the CSV that would be removed if you did.

@shani-f
Copy link
Contributor Author

shani-f commented Nov 15, 2025

Hello @CISC,
I tried to regenerate the CSV, but I couldn’t find any mechanism in the current LLAMA/ggml version that actually rebuilds SYCL.csv automatically.
If you want, I can reset both ops.md and SYCL.csv to their upstream versions and leave them as-is.

Let me know what you prefer.
Thanks!

@CISC
Copy link
Member

CISC commented Nov 15, 2025

I tried to regenerate the CSV, but I couldn’t find any mechanism in the current LLAMA/ggml version that actually rebuilds SYCL.csv automatically.

As explained here, simply run test-backend-ops support --output csv from your build/bin folder.

llama.cpp/docs/ops.md

Line 7 in 9ee5bdc

1. Run `test-backend-ops support --output csv` with your backend name and redirect output to a csv file in `docs/ops/` (e.g., `docs/ops/CUDA.csv`)

@shani-f
Copy link
Contributor Author

shani-f commented Nov 15, 2025

Hello @CISC,
I really hope it’s okay now.
Thank you so much for your help!

@CISC
Copy link
Member

CISC commented Nov 15, 2025

Hello @CISC, I really hope it’s okay now. Thank you so much for your help!

Yes, perfect, thank you! :)

@CISC CISC merged commit 72bd732 into ggml-org:master Nov 15, 2025
72 of 73 checks passed
@NeoZhangJianyu
Copy link
Collaborator

NeoZhangJianyu commented Nov 17, 2025

@shani-f
I find there are 4 OPs' status are chagned from support to unsupport。
But I test them with latest version: commit 2376b77 (HEAD -> master, tag: b7083
All are supported (some are not full supported).

Could you confirm it?

./build/bin/test-backend-ops  -o  GROUP_NORM_MUL_ADD 
./build/bin/test-backend-ops  -o NORM_MUL_ADD
./build/bin/test-backend-ops  -o RMS_NORM_MUL_ADD 
 ./build/bin/test-backend-ops  -o SOFTCAP

I think the method "Run test-backend-ops support --output csv with your backend name and redirect output to a csv file in docs/ops/ (e.g., docs/ops/CUDA.csv) " is with some issues.

We can't use above method to create CSV file and overwrite the existed CSV file directly.
It will lead to this issue.

Recommend to test and merge CSV manually.

Thank you!

@CISC
Copy link
Member

CISC commented Nov 17, 2025 •
edited
Loading

@NeoZhangJianyu No, this is correct, those "ops" are fusion and are now filtered from the list.

Edit: If at some point test-backend-ops becomes capable of detecting that fusion happens they can be added back in.

@NeoZhangJianyu
Copy link
Collaborator

NeoZhangJianyu commented Nov 18, 2025

Why other backends support them if those ops are fusion in the MD?

run ./build/bin/test-backend-ops -o GROUP_NORM_MUL_ADD , we will see GROUP_NORM_MUL_ADD cases are passed.
But in the ops.md, GROUP_NORM_MUL_ADD status is 'unsupported'.

If user want to run a new LLM including GROUP_NORM_MUL_ADD by llama.cpp, SYCL backend won't be ignored due to it's not supported shown by ops.md.
It will give people wrong info.

@CISC
Copy link
Member

CISC commented Nov 18, 2025

Why other backends support them if those ops are fusion in the MD?

It's outdated information, it will disappear for all other backends as well.

Every backend that has all the basic ops in the fusion passed these tests before, now they are filtered.

Anico2 added a commit to Anico2/llama.cpp that referenced this pull request Jan 15, 2026
@Anico2
sycl : unify unary kernels with a generic implementation and enable w…
bdc1b3c 
…ide operator support (ggml-org#17213)

* SYCL: add generic unary op implementation for multiple ops (ABS/SGN/…); unify non-contiguous access

* SYCL: update documentation and sycl.csv to reflect new unary op support

* update ops.md after syncing SYCL.csv changes

* Fix SYCL.csv merge conflict

* Update ops.md after fixing SYCL.csv conflicts

* Fix SYCL.csv tail after merge conflict and regenerate ops.md

* Fix line endings and final newline in SYCL.csv

* Remove TOPK_MOE entries from SYCL.csv as requested

* Update ops.md after removing TOPK_MOE from SYCL.csv

* Regenerated SYCL.csv and synced ops.md with upstream

* Update ops.md using create_ops_docs.py
blime4 referenced this pull request in blime4/llama.cpp Feb 5, 2026
@shani-f
sycl : unify unary kernels with a generic implementation and enable w…
94b0723 
…ide operator support (#17213)

* SYCL: add generic unary op implementation for multiple ops (ABS/SGN/…); unify non-contiguous access

* SYCL: update documentation and sycl.csv to reflect new unary op support

* update ops.md after syncing SYCL.csv changes

* Fix SYCL.csv merge conflict

* Update ops.md after fixing SYCL.csv conflicts

* Fix SYCL.csv tail after merge conflict and regenerate ops.md

* Fix line endings and final newline in SYCL.csv

* Remove TOPK_MOE entries from SYCL.csv as requested

* Update ops.md after removing TOPK_MOE from SYCL.csv

* Regenerated SYCL.csv and synced ops.md with upstream

* Update ops.md using create_ops_docs.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@CISC CISC CISC approved these changes

@NeoZhangJianyu NeoZhangJianyu NeoZhangJianyu approved these changes

Assignees

No one assigned

Labels

documentation Improvements or additions to documentation ggml changes relating to the ggml tensor library for machine learning SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@shani-f @NeoZhangJianyu @CISC