Skip to content

CANN: Implement GLU ops#14884

Merged
hipudding merged 1 commit intoggml-org:masterfrom
hipudding:glu
Jul 26, 2025
Merged

CANN: Implement GLU ops#14884
hipudding merged 1 commit intoggml-org:masterfrom
hipudding:glu

Conversation

@hipudding
Copy link
Collaborator

@hipudding hipudding commented Jul 26, 2025

Implement REGLU, GEGLU, SWIGLU ops according to #14158

Make sure to read the contributing guidelines before submitting a PR

@github-actions github-actions bot added ggml changes relating to the ggml tensor library for machine learning Ascend NPU issues specific to Ascend NPUs labels Jul 26, 2025
@hipudding
CANN: Implement GLU ops
c72d871 
Implement REGLU, GEGLU, SWIGLU ops according to ggml-org#14158
@hipudding
Copy link
Collaborator Author

hipudding commented Jul 26, 2025

Testing 2 devices
Backend 1/2: CANN0
Device description: Ascend910B4
Device memory: 30196 MB (29819 MB free)
REGLU(type=f16,ne_a=[128,2,2,2],v=0,swapped=0): OK
REGLU(type=f16,ne_a=[5,7,11,13],v=0,swapped=0): OK
REGLU(type=f16,ne_a=[128,2,2,2],v=0,swapped=1): OK
REGLU(type=f16,ne_a=[5,7,11,13],v=0,swapped=1): OK
REGLU(type=f16,ne_a=[128,2,2,2],v=0,split): OK
REGLU(type=f16,ne_a=[5,7,11,13],v=0,split): OK
REGLU(type=f16,ne_a=[128,2,2,2],v=1,swapped=0): OK
REGLU(type=f16,ne_a=[5,7,11,13],v=1,swapped=0): OK
REGLU(type=f16,ne_a=[128,2,2,2],v=1,swapped=1): OK
REGLU(type=f16,ne_a=[5,7,11,13],v=1,swapped=1): OK
REGLU(type=f16,ne_a=[128,2,2,2],v=1,split): OK
REGLU(type=f16,ne_a=[5,7,11,13],v=1,split): OK
REGLU(type=f32,ne_a=[128,2,2,2],v=0,swapped=0): OK
REGLU(type=f32,ne_a=[5,7,11,13],v=0,swapped=0): OK
REGLU(type=f32,ne_a=[128,2,2,2],v=0,swapped=1): OK
REGLU(type=f32,ne_a=[5,7,11,13],v=0,swapped=1): OK
REGLU(type=f32,ne_a=[128,2,2,2],v=0,split): OK
REGLU(type=f32,ne_a=[5,7,11,13],v=0,split): OK
REGLU(type=f32,ne_a=[128,2,2,2],v=1,swapped=0): OK
REGLU(type=f32,ne_a=[5,7,11,13],v=1,swapped=0): OK
REGLU(type=f32,ne_a=[128,2,2,2],v=1,swapped=1): OK
REGLU(type=f32,ne_a=[5,7,11,13],v=1,swapped=1): OK
REGLU(type=f32,ne_a=[128,2,2,2],v=1,split): OK
REGLU(type=f32,ne_a=[5,7,11,13],v=1,split): OK
GEGLU(type=f16,ne_a=[128,2,2,2],v=0,swapped=0): OK
GEGLU(type=f16,ne_a=[5,7,11,13],v=0,swapped=0): OK
GEGLU(type=f16,ne_a=[128,2,2,2],v=0,swapped=1): OK
GEGLU(type=f16,ne_a=[5,7,11,13],v=0,swapped=1): OK
GEGLU(type=f16,ne_a=[128,2,2,2],v=0,split): OK
GEGLU(type=f16,ne_a=[5,7,11,13],v=0,split): OK
GEGLU(type=f16,ne_a=[128,2,2,2],v=1,swapped=0): OK
GEGLU(type=f16,ne_a=[5,7,11,13],v=1,swapped=0): OK
GEGLU(type=f16,ne_a=[128,2,2,2],v=1,swapped=1): OK
GEGLU(type=f16,ne_a=[5,7,11,13],v=1,swapped=1): OK
GEGLU(type=f16,ne_a=[128,2,2,2],v=1,split): OK
GEGLU(type=f16,ne_a=[5,7,11,13],v=1,split): OK
GEGLU(type=f32,ne_a=[128,2,2,2],v=0,swapped=0): OK
GEGLU(type=f32,ne_a=[5,7,11,13],v=0,swapped=0): OK
GEGLU(type=f32,ne_a=[128,2,2,2],v=0,swapped=1): OK
GEGLU(type=f32,ne_a=[5,7,11,13],v=0,swapped=1): OK
GEGLU(type=f32,ne_a=[128,2,2,2],v=0,split): OK
GEGLU(type=f32,ne_a=[5,7,11,13],v=0,split): OK
GEGLU(type=f32,ne_a=[128,2,2,2],v=1,swapped=0): OK
GEGLU(type=f32,ne_a=[5,7,11,13],v=1,swapped=0): OK
GEGLU(type=f32,ne_a=[128,2,2,2],v=1,swapped=1): OK
GEGLU(type=f32,ne_a=[5,7,11,13],v=1,swapped=1): OK
GEGLU(type=f32,ne_a=[128,2,2,2],v=1,split): OK
GEGLU(type=f32,ne_a=[5,7,11,13],v=1,split): OK
SWIGLU(type=f16,ne_a=[128,2,2,2],v=0,swapped=0): OK
SWIGLU(type=f16,ne_a=[5,7,11,13],v=0,swapped=0): OK
SWIGLU(type=f16,ne_a=[128,2,2,2],v=0,swapped=1): OK
SWIGLU(type=f16,ne_a=[5,7,11,13],v=0,swapped=1): OK
SWIGLU(type=f16,ne_a=[128,2,2,2],v=0,split): OK
SWIGLU(type=f16,ne_a=[5,7,11,13],v=0,split): OK
SWIGLU(type=f16,ne_a=[128,2,2,2],v=1,swapped=0): OK
SWIGLU(type=f16,ne_a=[5,7,11,13],v=1,swapped=0): OK
SWIGLU(type=f16,ne_a=[128,2,2,2],v=1,swapped=1): OK
SWIGLU(type=f16,ne_a=[5,7,11,13],v=1,swapped=1): OK
SWIGLU(type=f16,ne_a=[128,2,2,2],v=1,split): OK
SWIGLU(type=f16,ne_a=[5,7,11,13],v=1,split): OK
SWIGLU(type=f32,ne_a=[128,2,2,2],v=0,swapped=0): OK
SWIGLU(type=f32,ne_a=[5,7,11,13],v=0,swapped=0): OK
SWIGLU(type=f32,ne_a=[128,2,2,2],v=0,swapped=1): OK
SWIGLU(type=f32,ne_a=[5,7,11,13],v=0,swapped=1): OK
SWIGLU(type=f32,ne_a=[128,2,2,2],v=0,split): OK
SWIGLU(type=f32,ne_a=[5,7,11,13],v=0,split): OK
SWIGLU(type=f32,ne_a=[128,2,2,2],v=1,swapped=0): OK
SWIGLU(type=f32,ne_a=[5,7,11,13],v=1,swapped=0): OK
SWIGLU(type=f32,ne_a=[128,2,2,2],v=1,swapped=1): OK
SWIGLU(type=f32,ne_a=[5,7,11,13],v=1,swapped=1): OK
SWIGLU(type=f32,ne_a=[128,2,2,2],v=1,split): OK
SWIGLU(type=f32,ne_a=[5,7,11,13],v=1,split): OK
GEGLU_ERF(type=f16,ne_a=[128,2,2,2],v=0,swapped=0): OK
GEGLU_ERF(type=f16,ne_a=[5,7,11,13],v=0,swapped=0): OK
GEGLU_ERF(type=f16,ne_a=[128,2,2,2],v=0,swapped=1): OK
GEGLU_ERF(type=f16,ne_a=[5,7,11,13],v=0,swapped=1): OK
GEGLU_ERF(type=f16,ne_a=[128,2,2,2],v=0,split): OK
GEGLU_ERF(type=f16,ne_a=[5,7,11,13],v=0,split): OK
GEGLU_ERF(type=f16,ne_a=[128,2,2,2],v=1,swapped=0): OK
GEGLU_ERF(type=f16,ne_a=[5,7,11,13],v=1,swapped=0): OK
GEGLU_ERF(type=f16,ne_a=[128,2,2,2],v=1,swapped=1): OK
GEGLU_ERF(type=f16,ne_a=[5,7,11,13],v=1,swapped=1): OK
GEGLU_ERF(type=f16,ne_a=[128,2,2,2],v=1,split): OK
GEGLU_ERF(type=f16,ne_a=[5,7,11,13],v=1,split): OK
GEGLU_ERF(type=f32,ne_a=[128,2,2,2],v=0,swapped=0): OK
GEGLU_ERF(type=f32,ne_a=[5,7,11,13],v=0,swapped=0): OK
GEGLU_ERF(type=f32,ne_a=[128,2,2,2],v=0,swapped=1): OK
GEGLU_ERF(type=f32,ne_a=[5,7,11,13],v=0,swapped=1): OK
GEGLU_ERF(type=f32,ne_a=[128,2,2,2],v=0,split): OK
GEGLU_ERF(type=f32,ne_a=[5,7,11,13],v=0,split): OK
GEGLU_ERF(type=f32,ne_a=[128,2,2,2],v=1,swapped=0): OK
GEGLU_ERF(type=f32,ne_a=[5,7,11,13],v=1,swapped=0): OK
GEGLU_ERF(type=f32,ne_a=[128,2,2,2],v=1,swapped=1): OK
GEGLU_ERF(type=f32,ne_a=[5,7,11,13],v=1,swapped=1): OK
GEGLU_ERF(type=f32,ne_a=[128,2,2,2],v=1,split): OK
GEGLU_ERF(type=f32,ne_a=[5,7,11,13],v=1,split): OK
GEGLU_QUICK(type=f16,ne_a=[128,2,2,2],v=0,swapped=0): OK
GEGLU_QUICK(type=f16,ne_a=[5,7,11,13],v=0,swapped=0): OK
GEGLU_QUICK(type=f16,ne_a=[128,2,2,2],v=0,swapped=1): OK
GEGLU_QUICK(type=f16,ne_a=[5,7,11,13],v=0,swapped=1): OK
GEGLU_QUICK(type=f16,ne_a=[128,2,2,2],v=0,split): OK
GEGLU_QUICK(type=f16,ne_a=[5,7,11,13],v=0,split): OK
GEGLU_QUICK(type=f16,ne_a=[128,2,2,2],v=1,swapped=0): OK
GEGLU_QUICK(type=f16,ne_a=[5,7,11,13],v=1,swapped=0): OK
GEGLU_QUICK(type=f16,ne_a=[128,2,2,2],v=1,swapped=1): OK
GEGLU_QUICK(type=f16,ne_a=[5,7,11,13],v=1,swapped=1): OK
GEGLU_QUICK(type=f16,ne_a=[128,2,2,2],v=1,split): OK
GEGLU_QUICK(type=f16,ne_a=[5,7,11,13],v=1,split): OK
GEGLU_QUICK(type=f32,ne_a=[128,2,2,2],v=0,swapped=0): OK
GEGLU_QUICK(type=f32,ne_a=[5,7,11,13],v=0,swapped=0): OK
GEGLU_QUICK(type=f32,ne_a=[128,2,2,2],v=0,swapped=1): OK
GEGLU_QUICK(type=f32,ne_a=[5,7,11,13],v=0,swapped=1): OK
GEGLU_QUICK(type=f32,ne_a=[128,2,2,2],v=0,split): OK
GEGLU_QUICK(type=f32,ne_a=[5,7,11,13],v=0,split): OK
GEGLU_QUICK(type=f32,ne_a=[128,2,2,2],v=1,swapped=0): OK
GEGLU_QUICK(type=f32,ne_a=[5,7,11,13],v=1,swapped=0): OK
GEGLU_QUICK(type=f32,ne_a=[128,2,2,2],v=1,swapped=1): OK
GEGLU_QUICK(type=f32,ne_a=[5,7,11,13],v=1,swapped=1): OK
GEGLU_QUICK(type=f32,ne_a=[128,2,2,2],v=1,split): OK
GEGLU_QUICK(type=f32,ne_a=[5,7,11,13],v=1,split): OK
Backend CANN0: OK
Backend 2/2: CPU
Skipping
2/2 backends passed

@noemotiovon
Copy link
Collaborator

noemotiovon commented Jul 26, 2025

LGTM!

@hipudding hipudding requested review from CISC, ggerganov and slaren July 26, 2025 09:12
@hipudding hipudding self-assigned this Jul 26, 2025
@CISC
Copy link
Collaborator

CISC commented Jul 26, 2025

LGTM, but I can't test.

@CISC CISC removed their request for review July 26, 2025 09:16
@hipudding hipudding merged commit 11dd5a4 into ggml-org:master Jul 26, 2025
48 checks passed
blime4 referenced this pull request in blime4/llama.cpp Feb 5, 2026
@hipudding
CANN: Implement GLU ops (#14884)
48e3a3c 
Implement REGLU, GEGLU, SWIGLU ops according to #14158
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@slaren slaren slaren approved these changes

@ggerganov ggerganov Awaiting requested review from ggerganov

Assignees

@hipudding hipudding

Labels

Ascend NPU issues specific to Ascend NPUs ggml changes relating to the ggml tensor library for machine learning

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@hipudding @noemotiovon @CISC @slaren