Skip to content

Qwen Omni divide by zero error (with solution) #17125

@LostRuins

Description

@LostRuins

Hello @ngxson @ggerganov ,

When using Qwen Omni 3B, there is a chance to get a divide by zero error which will crash the program.

llama.cpp/tools/mtmd/clip.cpp

Lines 2664 to 2666 in b8595b1

if (is_vision) {
get_u32(KEY_IMAGE_SIZE, hparams.image_size);
get_u32(KEY_PATCH_SIZE, hparams.patch_size);

The problem is that when you load the mmproj for qwen omni, there is a vision and audio clip context being created, However, hparams.patch_size is only populated when is_vision is true.

Then for the clip audio context, when clip_graph is constructed

patch_size(hparams.patch_size),
n_patches_x(img.nx / patch_size),
n_patches_y(img.ny / patch_size),

we have patch_size(hparams.patch_size) followed by n_patches_x(img.nx / patch_size), depending on init patch_size can be 0 when initializing the audio context, leading to divide by zero error and crash.

Solution:
Make KEY_PATCH_SIZE optional and always read that hparam regardless of vision or not.
LostRuins@6cce98e

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions