Skip to content

feat: Add support for scanning GGUF models from OCI registries#4335

Merged
spiffcs merged 67 commits intomainfrom
4184-pt2-oci-model-support
Feb 9, 2026
Merged

feat: Add support for scanning GGUF models from OCI registries#4335
spiffcs merged 67 commits intomainfrom
4184-pt2-oci-model-support

Conversation

@spiffcs
Copy link
Contributor

@spiffcs spiffcs commented Nov 5, 2025

Summary

This PR follows up on #4279 by adding support for a new source type.

With this change users can do the following: syft -o json docker.io/ai/qwen3-vl | jq .:

They'll get an SBOM with a single package showing the gguf model and details for the model pulled from https://hub.docker.com/u/ai

Example of metadata extracted

 "metadata": {
        "modelFormat": "gguf",
        "modelName": "Qwen3-Vl-8B-Instruct",
        "modelVersion": "unknown",
        "hash": "321c13d3e93151b5",
        "license": "apache-2.0",
        "ggufVersion": 3,
        "architecture": "qwen3vl",
        "quantization": "Q4_K_M",
        "parameters": 8190735360,
        "tensorCount": 399,
        "header": {
          "general.base_model.0.name": "Qwen3 VL 8B Instruct",
          "general.base_model.0.organization": "Qwen",
          "general.base_model.0.repo_url": "https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct",
          "general.base_model.count": 1,
          "general.basename": "Qwen3-Vl-8B-Instruct",
          "general.file_type": 15,
          "general.finetune": "Instruct",
          "general.quantization_version": 2,
          "general.quantized_by": "Unsloth",
          "general.repo_url": "https://huggingface.co/unsloth",
          "general.size_label": "8B",
          "general.tags": {
            "type": 8,
            "len": 2,
            "startOffset": 741,
            "size": 41
          },
          "general.type": "model",
          "quantize.imatrix.chunks_count": 694,
          "quantize.imatrix.dataset": "unsloth_calibration_Qwen3-VL-8B-Instruct.txt",
          "quantize.imatrix.entries_count": 252,
          "quantize.imatrix.file": "Qwen3-VL-8B-Instruct-GGUF/imatrix_unsloth.gguf",
          "qwen3vl.attention.head_count": 32,
          "qwen3vl.attention.head_count_kv": 8,
          "qwen3vl.attention.key_length": 128,
          "qwen3vl.attention.layer_norm_rms_epsilon": 0.000001,
          "qwen3vl.attention.value_length": 128,
          "qwen3vl.block_count": 36,
          "qwen3vl.context_length": 262144,
          "qwen3vl.embedding_length": 4096,
          "qwen3vl.feed_forward_length": 12288,
          "qwen3vl.n_deepstack_layers": 3,
          "qwen3vl.rope.dimension_sections": {
            "type": 5,
            "len": 4,
            "startOffset": 1268,
            "size": 16
          },
          "qwen3vl.rope.freq_base": 5000000,
         "tokenizer.ggml.add_bos_token": false,
          "tokenizer.ggml.bos_token_id": 151643,
          "tokenizer.ggml.eos_token_id": 151645,
          "tokenizer.ggml.merges": {
            "type": 8,
            "len": 151387,
            "startOffset": 3197544,
            "size": 2731548
          },

This PR adds support for scanning OCI model artifacts (specifically GGUF-based AI models) stored in OCI registries. The
implementation enables Syft to extract SBOM information from AI models without downloading the entire model file—only
the GGUF headers are fetched using a lazy reader that closes after we've read all the header bytes.

Key changes:

  • New ocimodelsource package implementing a source provider for OCI model artifacts
  • Registry client that fetches only GGUF headers (up to 8MB) instead of full model files
  • New OCIMediaTypeResolver interface and ContainerImageModel file resolver for media type-based file discovery
  • Extended generic cataloger with WithParserByMediaType() to support cataloging by OCI media type
  • Post-processor for GGUF cataloger to merge metadata found from multi-layer model artifacts

OCI Model Source

This new source type enables scanning AI models stored as OCI artifacts in container registries (like Docker Hub or other OCI registries).

How it works:

  1. Detection: Identifies model artifacts by checking if the manifest's config media type starts with
    application/vnd.docker.ai.model.config.*
  2. Selective fetching: Instead of downloading entire GGUF layers (often 4-70GB), it uses HTTP range requests to fetch
    only the first 8MB containing the GGUF header metadata. The download is terminated early by closing the reader after
    reading the header bytes.
  3. Temporary storage: Headers are written to temp files, keyed by the layer digest, and cleaned up when the source is
    closed.
  4. File resolver: A new ContainerImageModel resolver implements both file.Resolver and file.OCIMediaTypeResolver,
    allowing catalogers to discover layers by OCI media type (application/vnd.docker.ai.gguf.v3) rather than filesystem
    paths.
  5. Provider integration: Registered as the oci-model provider, positioned after the regular pull providers in the source
    provider chain—so normal container images are handled first, and this kicks in only for artifacts with model-specific
    media types. It also makes it so that for syft's current MOST common cases (non ai containers) we're not adding extra work.

Keys Goals:

  • Scanning a 70GB LLaMA model takes seconds instead of minutes
  • No disk space needed for full model storage
  • Works with Docker's AI model packaging spec and Ollama registries

@spiffcs spiffcs force-pushed the 4184-pt2-oci-model-support branch from 5853129 to 8031957 Compare November 13, 2025 06:19
Base automatically changed from 4184-gguf-parser to main November 13, 2025 22:43
@spiffcs spiffcs changed the title wip: 4184 pt2 oci model support feat: 4184 ai oci model support Dec 19, 2025
Copy link
Contributor

@wagoodman wagoodman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should take a pass to unexport whatever doesn't need to be in the public API

@spiffcs
Copy link
Contributor Author

spiffcs commented Dec 19, 2025

First big change to make here is how the Provider and Source interact. Provider should be thin and call Source, Source should do most of the work, provider should return the source.Source answer.

@spiffcs spiffcs force-pushed the 4184-pt2-oci-model-support branch from 4fc836b to a4ef861 Compare December 19, 2025 18:06
wagoodman and others added 5 commits January 9, 2026 12:56
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
@spiffcs spiffcs force-pushed the 4184-pt2-oci-model-support branch from 5c6a10e to 9026b8a Compare January 9, 2026 18:15
@github-actions github-actions bot added the json-schema Changes the json schema label Jan 9, 2026
spiffcs and others added 14 commits January 9, 2026 13:20
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
…into 4184-pt2-oci-model-support

* '4184-pt2-oci-model-support' of github.com:anchore/syft:
  fix generation to account for self referential types
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
…into 4184-pt2-oci-model-support

* '4184-pt2-oci-model-support' of github.com:anchore/syft:
  orient pkg metadata names by display name not type name

Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Signed-off-by: Christopher Phillips <32073428+spiffcs@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
…l-support

* 'main' of github.com:anchore/syft:
  chore(deps): bump the actions-minor-patch group across 2 directories with 3 updates (#4568)
  chore(deps): bump the go-minor-patch group with 6 updates (#4567)
  chore(deps): update tools to latest versions (#4565)
  chore(deps): bump github.com/spdx/tools-golang (#4557)
  ci: enable zizmor to fail PRs (#4556)
  Chore new slack action (#4553)
  chore(deps): update anchore dependencies (#4552)
  chore(deps): update tools to latest versions (#4551)
  chore(deps): update tools to latest versions (#4545)
  chore(deps): update tools to latest versions (#4542)
  chore(deps): bump the go-minor-patch group with 4 updates (#4543)
  chore(deps): bump anchore/sbom-action (#4544)
@spiffcs spiffcs added the blocked Progress is being stopped by something label Feb 3, 2026
@wagoodman wagoodman changed the title feat: 4184 ai oci model support feat: Add support for scanning GGUF models from OCI registries Feb 9, 2026
@spiffcs spiffcs merged commit 2c5e193 into main Feb 9, 2026
11 checks passed
@github-project-automation github-project-automation bot moved this from In Review to Done in OSS Feb 9, 2026
@spiffcs spiffcs deleted the 4184-pt2-oci-model-support branch February 9, 2026 21:05
@spiffcs spiffcs removed the blocked Progress is being stopped by something label Feb 9, 2026
@willmurphyscode willmurphyscode added the enhancement New feature or request label Feb 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request json-schema Changes the json schema

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

3 participants