Name and Version

$ ./build/bin/llama-cli --version
version: 8329 (fbaa95b)
built with GNU 11.4.0 for Linux aarch64

Operating systems

Linux, Other? (Please let us know in description)

GGML backends

CPU

Hardware

Graviton-3 / Graviton-4 / Google Tensor G5 / MediaTek Dimensity 9500

Models

Qwen2-VL 2B Q4_0 with Q8_0 quantized mmproj
Qwen2 1.5B-Instruct Q4_0
Qwen2 1.5B-Instruct Q8_0
Qwen2 1.5B-Instruct F16

Problem description & steps to reproduce

When building llama.cpp for Arm64 hardware with SVE or SVE2, running llama-cli on either Linux or Android produces seemingly random ASCII characters as output instead of legible text.
This seems to happen across various devices on both Linux and Android, with the unifying factor being an Arm64 CPU with SVE enabled. This happens both with KleidiAI enabled and disabled.

Steps to reproduce:

• Build llama.cpp with the ggml CPU backend:

Configure:

Linux:
cmake -DGGML_METAL=OFF -DGGML_BLAS=OFF -DGGML_ACCELERATE=OFF -DGGML_OPENMP=OFF -DGGML_OPENCL=OFF -DGGML_VULKAN=OFF -DGGML_KOMPUTE=OFF -DGGML_HIP=OFF -DGGML_CUDA=OFF -DGGML_MUSA=OFF -DGGML_BUILD_EXAMPLES=OFF -DLLAMA_BUILD_TESTS=ON -DCMAKE_BUILD_TYPE=Release -DGGML_LLAMAFILE=OFF -DLLAMA_CURL=OFF -DBUILD_SHARED_LIBS=OFF -B build
Android:
cmake -DCMAKE_TOOLCHAIN_FILE=$NDK_PATH/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-28 -DGGML_OPENMP=OFF -DGGML_LLAMAFILE=OFF -DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON -DBUILD_SHARED_LIBS=ON -B build

Build:

cmake --build build --config Release -j

• Run llama-cli -m "path/to/model.gguf" -p "Hello!" (or with any prompt, or in interactive mode and type the prompt)

First Bad Commit

a0c2b20

Relevant log output

build      : b8329-fbaa95bc
model      : Qwen2-1.5B-Instruct.Q4_0.gguf
modalities : text

available commands:
  /exit or Ctrl+C     stop or exit
  /regen              regenerate the last response
  /clear              clear the chat history
  /read               add a text file


> Tell me a joke!

+8+2A9.;&>D(5EC;0!;=3#++$@1$!8!.,G#F59,)+66DEED!E8&&';H,/E3,')-@+"(/<;B1B3+!F),'D