Benchmark results

# Encoder

Collection of [bench](https://github.com/ggerganov/whisper.cpp/tree/master/examples/bench) results for various platforms and devices.
If you want to submit info about your device, simply run the [bench](https://github.com/ggerganov/whisper.cpp/tree/master/examples/bench) tool or the [extra/bench-all.sh](https://github.com/ggerganov/whisper.cpp/blob/master/extra/bench-all.sh) and report the results in the comments below.

*Suggestions for better summary of the results are welcome*

| CPU | OS | Config | Model | Th | Load | Enc. | Commit |
| --- | -- | ------ | ----- | ------- | --------- | ----------- | ------ |
| MacBook M1 Pro | MacOS 13.0.1 |  NEON BLAS | tiny | 8 | 71 | 102 | 206fc93 |
| MacBook M1 Pro | MacOS 13.0.1 |  NEON BLAS | base | 8 | 96 | 220 | 206fc93 |
| MacBook M1 Pro | MacOS 13.0.1 |  NEON BLAS | small | 8 | 233 | 685 | 206fc93 |
| MacBook M1 Pro | MacOS 13.0.1 |  NEON BLAS | medium | 8 | 603 | 1928 | 206fc93 |
| MacBook M1 Pro | MacOS 13.0.1 |  NEON BLAS | large | 8 | 1158 | 3350 | 206fc93 |
| --- |
| MacBook M1 Pro | MacOS 13.0.1 |  NEON BLAS | small | 1 | 251 | 2605 | 206fc93 |
| MacBook M1 Pro | MacOS 13.0.1 |  NEON BLAS | small | 4 | 255 | 884 | 206fc93 |
| --- |
| Mac Mini M1 | MacOS |  NEON BLAS | tiny | 4 | 62 | 194 | fcf515d |
| Mac Mini M1 | MacOS |  NEON BLAS | base | 4 | 81 | 380 | fcf515d |
| Mac Mini M1 | MacOS |  NEON BLAS | small | 4 | 204 | 1249 | fcf515d |
| Mac Mini M1 | MacOS |  NEON BLAS | medium | 4 | 876 | 3980 | fcf515d |
| Mac Mini M1 | MacOS |  NEON BLAS | large | 4 | 1876 | 7979 | fcf515d |
| --- |
| Ryzen 9 3900X | Ubuntu 20.04 |  AVX2 | tiny | 8 | 107 | 422 | fcf515d |
| Ryzen 9 3900X | Ubuntu 20.04 |  AVX2 | base | 8 | 137 | 880 | fcf515d |
| Ryzen 9 3900X | Ubuntu 20.04 |  AVX2 | small | 8 | 280 | 2874 | fcf515d |
| Ryzen 9 3900X | Ubuntu 20.04 |  AVX2 | medium | 8 | 692 | 9610 | fcf515d |
| Ryzen 9 3900X | Ubuntu 20.04 |  AVX2 | large | 8 | 1317 | 16917 | fcf515d |
| --- |
| Ryzen 9 3900X | Ubuntu 20.04 |  AVX2 BLAS | tiny | 4 | 120 | 780 | fcf515d |
| Ryzen 9 3900X | Ubuntu 20.04 |  AVX2 BLAS | base | 4 | 151 | 1173 | fcf515d |
| Ryzen 9 3900X | Ubuntu 20.04 |  AVX2 BLAS | small | 4 | 289 | 3062 | fcf515d |
| Ryzen 9 3900X | Ubuntu 20.04 |  AVX2 BLAS | medium | 4 | 711 | 9175 | fcf515d |
| Ryzen 9 3900X | Ubuntu 20.04 |  AVX2 BLAS | large | 4 | 1282 | 16050 | fcf515d |
| --- |
| Ryzen 9 5950X | Ubuntu 22.04 |  AVX2 | tiny | 8 | 135 | 197 | fcf515d |
| Ryzen 9 5950X | Ubuntu 22.04 |  AVX2 | base | 8 | 176 | 421 | fcf515d |
| Ryzen 9 5950X | Ubuntu 22.04 |  AVX2 | small | 8 | 357 | 1393 | fcf515d |
| Ryzen 9 5950X | Ubuntu 22.04 |  AVX2 | medium | 8 | 855 | 4404 | fcf515d |
| Ryzen 9 5950X | Ubuntu 22.04 |  AVX2 | large | 8 | 1576 | 8118 | fcf515d |
| --- |
| Raspberry Pi 4 | |  NEON | tiny | 4 | 1436 | 13839 | fcf515d |
| Raspberry Pi 4 | |  NEON | base | 4 | 1894 | 30552 | fcf515d |
| --- |
| iPhone 13 Mini | iOS 16.0 |  NEON BLAS | base | 4 | 97 | 1091 | fcf515d |
| --- |
| MacBook M1 Pro | Vivaldi |  WASM | tiny | 8 | 133 | 3785 | fcf515d |
| MacBook M1 Pro | Vivaldi |  WASM | base | 8 | 172 | 8253 | fcf515d |
| --- |
| MacBook M1 Pro | Chrome |  WASM | tiny | 8 | 134 | 3776 | fcf515d |
| MacBook M1 Pro | Chrome |  WASM | base | 8 | 168 | 8200 | fcf515d |
| --- |
| MacBook M1 Pro | Firefox |  WASM | tiny | 8 | 137 | 2626 | fcf515d |
| MacBook M1 Pro | Firefox |  WASM | base | 8 | 183 | 6226 | fcf515d |

# memcpy

### MacBook M1 Pro

```
./bench -w 1 -t 1
memcpy: 37.59 GB/s
```

### Ryzen 9 5950X

```
./bench -w 1 -t 1
memcpy: 16.74 GB/s
```

# ggml_mul_mat

### MacBook M1 Pro

```
./bench -w 2 -t 1
ggml_mul_mat:    64 x    64: F16    330.6 GFLOPS (128 runs) / F32    466.0 GFLOPS (128 runs)
ggml_mul_mat:   128 x   128: F16    737.5 GFLOPS (128 runs) / F32    838.9 GFLOPS (128 runs)
ggml_mul_mat:   256 x   256: F16    938.6 GFLOPS (128 runs) / F32   1062.3 GFLOPS (128 runs)
ggml_mul_mat:   512 x   512: F16   1312.5 GFLOPS (128 runs) / F32   1835.5 GFLOPS (128 runs)
ggml_mul_mat:  1024 x  1024: F16   1765.1 GFLOPS (128 runs) / F32   2041.4 GFLOPS (128 runs)
ggml_mul_mat:  2048 x  2048: F16   1784.3 GFLOPS (104 runs) / F32   1859.2 GFLOPS (109 runs)
ggml_mul_mat:  4096 x  4096: F16   1855.1 GFLOPS ( 14 runs) / F32   1873.3 GFLOPS ( 14 runs)
``` 

### Ryzen 9 5950X

```
WHISPER_OPENBLAS=1 make -j bench && ./bench -w 2 -t 1
ggml_mul_mat:    64 x    64: F16     56.3 GFLOPS (128 runs) / F32     70.2 GFLOPS (128 runs)
ggml_mul_mat:   128 x   128: F16     47.8 GFLOPS (128 runs) / F32     67.0 GFLOPS (128 runs)
ggml_mul_mat:   256 x   256: F16    185.1 GFLOPS (128 runs) / F32    332.7 GFLOPS (128 runs)
ggml_mul_mat:   512 x   512: F16    386.4 GFLOPS (128 runs) / F32    658.6 GFLOPS (128 runs)
ggml_mul_mat:  1024 x  1024: F16    636.2 GFLOPS (128 runs) / F32   1012.0 GFLOPS (128 runs)
ggml_mul_mat:  2048 x  2048: F16    950.9 GFLOPS ( 56 runs) / F32   1296.8 GFLOPS ( 76 runs)
ggml_mul_mat:  4096 x  4096: F16   1168.6 GFLOPS (  9 runs) / F32   1403.1 GFLOPS ( 11 runs)
```

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Benchmark results #89

Encoder

memcpy

MacBook M1 Pro

Ryzen 9 5950X

ggml_mul_mat

MacBook M1 Pro

Ryzen 9 5950X

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

CPU	OS	Config	Model	Th	Load	Enc.	Commit
MacBook M1 Pro	MacOS 13.0.1	NEON BLAS	tiny	8	71	102	`206fc93`
MacBook M1 Pro	MacOS 13.0.1	NEON BLAS	base	8	96	220	`206fc93`
MacBook M1 Pro	MacOS 13.0.1	NEON BLAS	small	8	233	685	`206fc93`
MacBook M1 Pro	MacOS 13.0.1	NEON BLAS	medium	8	603	1928	`206fc93`
MacBook M1 Pro	MacOS 13.0.1	NEON BLAS	large	8	1158	3350	`206fc93`
---
MacBook M1 Pro	MacOS 13.0.1	NEON BLAS	small	1	251	2605	`206fc93`
MacBook M1 Pro	MacOS 13.0.1	NEON BLAS	small	4	255	884	`206fc93`
---
Mac Mini M1	MacOS	NEON BLAS	tiny	4	62	194	`fcf515d`
Mac Mini M1	MacOS	NEON BLAS	base	4	81	380	`fcf515d`
Mac Mini M1	MacOS	NEON BLAS	small	4	204	1249	`fcf515d`
Mac Mini M1	MacOS	NEON BLAS	medium	4	876	3980	`fcf515d`
Mac Mini M1	MacOS	NEON BLAS	large	4	1876	7979	`fcf515d`
---
Ryzen 9 3900X	Ubuntu 20.04	AVX2	tiny	8	107	422	`fcf515d`
Ryzen 9 3900X	Ubuntu 20.04	AVX2	base	8	137	880	`fcf515d`
Ryzen 9 3900X	Ubuntu 20.04	AVX2	small	8	280	2874	`fcf515d`
Ryzen 9 3900X	Ubuntu 20.04	AVX2	medium	8	692	9610	`fcf515d`
Ryzen 9 3900X	Ubuntu 20.04	AVX2	large	8	1317	16917	`fcf515d`
---
Ryzen 9 3900X	Ubuntu 20.04	AVX2 BLAS	tiny	4	120	780	`fcf515d`
Ryzen 9 3900X	Ubuntu 20.04	AVX2 BLAS	base	4	151	1173	`fcf515d`
Ryzen 9 3900X	Ubuntu 20.04	AVX2 BLAS	small	4	289	3062	`fcf515d`
Ryzen 9 3900X	Ubuntu 20.04	AVX2 BLAS	medium	4	711	9175	`fcf515d`
Ryzen 9 3900X	Ubuntu 20.04	AVX2 BLAS	large	4	1282	16050	`fcf515d`
---
Ryzen 9 5950X	Ubuntu 22.04	AVX2	tiny	8	135	197	`fcf515d`
Ryzen 9 5950X	Ubuntu 22.04	AVX2	base	8	176	421	`fcf515d`
Ryzen 9 5950X	Ubuntu 22.04	AVX2	small	8	357	1393	`fcf515d`
Ryzen 9 5950X	Ubuntu 22.04	AVX2	medium	8	855	4404	`fcf515d`
Ryzen 9 5950X	Ubuntu 22.04	AVX2	large	8	1576	8118	`fcf515d`
---
Raspberry Pi 4		NEON	tiny	4	1436	13839	`fcf515d`
Raspberry Pi 4		NEON	base	4	1894	30552	`fcf515d`
---
iPhone 13 Mini	iOS 16.0	NEON BLAS	base	4	97	1091	`fcf515d`
---
MacBook M1 Pro	Vivaldi	WASM	tiny	8	133	3785	`fcf515d`
MacBook M1 Pro	Vivaldi	WASM	base	8	172	8253	`fcf515d`
---
MacBook M1 Pro	Chrome	WASM	tiny	8	134	3776	`fcf515d`
MacBook M1 Pro	Chrome	WASM	base	8	168	8200	`fcf515d`
---
MacBook M1 Pro	Firefox	WASM	tiny	8	137	2626	`fcf515d`
MacBook M1 Pro	Firefox	WASM	base	8	183	6226	`fcf515d`

Benchmark results #89

Description

Encoder

memcpy

MacBook M1 Pro

Ryzen 9 5950X

ggml_mul_mat

MacBook M1 Pro

Ryzen 9 5950X

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions