Skip to content

Misc. bug: Docker version of llama.cpp doesn't support CUDA 12.8 anymore #21429

@StableLlama

Description

@StableLlama

Name and Version

I just run:

$  docker pull ghcr.io/ggml-org/llama.cpp:server-cuda
...
Digest: sha256:0d60155f9cbd5118d02568d90f505638259d85f6f1cc4ac98d0f1002001e1f7a

Trying to run that container is now failing with:

$  docker run --gpus all ghcr.io/ggml-org/llama.cpp:server-cuda
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running prestart hook #0: exit status 1, stdout: , stderr: Auto-detected mode as 'legacy'
nvidia-container-cli: requirement error: unsatisfied condition: cuda>=12.9, please update your driver to a newer version, or use an earlier cuda container: unknown

The version of yesterday was running fine on my system (nvidia Driver Version: 570.211.01 CUDA Version: 12.8)

It was mentioned nowhere that the minimum required CUDA version was raised to 12.9
And looking at cloud providers like vast I see many instances running with CUDA less than 12.9, so those older versions are still highly relevant and a raise of the minimum requirements should be considered only very carefully.

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

llama-server

Command line

`docker run --gpus all ghcr.io/ggml-org/llama.cpp:server-cuda`

Problem description & steps to reproduce

It doesn't start and shows a minimum CUDA version >= 12.9 error

First Bad Commit

No response

Relevant log output

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    CUDARelated to the CUDA backendbugSomething isn't workingdevopsimprovements to build systems and github actions

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions