Skip to content

Simplify the SIMD code#324

Merged
ggerganov merged 2 commits intomasterfrom
simd
Dec 24, 2022
Merged

Simplify the SIMD code#324
ggerganov merged 2 commits intomasterfrom
simd

Conversation

@ggerganov
Copy link
Copy Markdown
Member

@ggerganov ggerganov commented Dec 23, 2022

This is an attempt to simplify and unify the SIMD implementation for various architectures.
The idea is similar to #95 (comment)

We define a common subset of C macros which map to specific intrinsics based on the current architecture:

#define GGML_F32_VEC        ...
#define GGML_F32_VEC_ZERO   ...
#define GGML_F32_VEC_SET1   ...
#define GGML_F32_VEC_LOAD   ...
#define GGML_F32_VEC_STORE  ...
#define GGML_F32_VEC_FMA    ...
#define GGML_F32_VEC_ADD    ...
#define GGML_F32_VEC_MUL    ...
#define GGML_F32_VEC_REDUCE ...

#define GGML_F16_VEC        ...
#define GGML_F16_VEC_ZERO   ...
#define GGML_F16_VEC_SET1   ...
#define GGML_F16_VEC_LOAD   ...
#define GGML_F16_VEC_STORE  ...
#define GGML_F16_VEC_FMA    ...
#define GGML_F16_VEC_ADD    ...
#define GGML_F16_VEC_MUL    ...
#define GGML_F16_VEC_REDUCE ...

We then implement the SIMD functions using only these macros.
Refactored functions:

Supported and tested architectures:

  • Arm64
  • ARMv7
  • x86
  • WASM
  • ppc64le

Still not sure if this would be easier to maintain, but I will give it a try

@ggerganov ggerganov force-pushed the simd branch 3 times, most recently from 9e726a3 to bad040f Compare December 23, 2022 16:47
@ggerganov ggerganov marked this pull request as ready for review December 23, 2022 16:52
@ggerganov ggerganov merged commit ac521a5 into master Dec 24, 2022
@ggerganov ggerganov deleted the simd branch December 24, 2022 08:22
anandijain pushed a commit to anandijain/whisper.cpp that referenced this pull request Apr 28, 2023
* ggml : simplify the SIMD code

* ggml : generic reduce for all register sizes + comments
jacobwu-b pushed a commit to jacobwu-b/Transcriptify-by-whisper.cpp that referenced this pull request Oct 24, 2023
* ggml : simplify the SIMD code

* ggml : generic reduce for all register sizes + comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant