Skip to content

[GGUF] metadata serialization into Uint8Arrray#1740

Merged
mishig25 merged 9 commits intomainfrom
serialize_ggufmetadata
Sep 11, 2025
Merged

[GGUF] metadata serialization into Uint8Arrray#1740
mishig25 merged 9 commits intomainfrom
serialize_ggufmetadata

Conversation

@mishig25
Copy link
Collaborator

@mishig25 mishig25 commented Sep 10, 2025

Description

This PR introduces function that serializes GGUF metadata/header into Uint8Arrray so that I can use #1718 to update gguf metadata on hf.co

  • serializeTypedMetadata() - Serialize GGUF metadata to binary format
  • serializeGgufHeader() - Create complete GGUF headers with metadata + tensor info + alignment
  • Enhanced gguf() function - Now returns littleEndian property for endianness detection

Usage example

// Edit first kB of file
await commit({
  repo,
  accessToken: "hf_...",
  operations: [{
    type: "edit",
    originalContent: new Blob(original gguf header),
    edits: [{
      start: 0,
      end: 1000,
      content: new Blob(serializeGgufHeader(new gguf header with updated metadata))
    }]
  }]
})

@mishig25 mishig25 changed the base branch from serialize_gguf_metadata to main September 10, 2025 10:26
@mishig25 mishig25 marked this pull request as ready for review September 10, 2025 10:33
@mishig25 mishig25 requested a review from coyotte508 September 10, 2025 10:33
@mishig25 mishig25 changed the title [GGUF] metadata serialization [GGUF] metadata serialization into Blob Sep 10, 2025
Copy link
Member

@coyotte508 coyotte508 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we want to return Uint8Array and let user do Uint8Arrray => Blob conversion (or even the commit function)

not sure though

@mishig25 mishig25 force-pushed the serialize_ggufmetadata branch from d65989a to 54dc2e5 Compare September 10, 2025 10:46
…variable and unnecessary whitespace. This cleanup improves code readability without altering functionality.
@mishig25 mishig25 changed the title [GGUF] metadata serialization into Blob [GGUF] metadata serialization into Uint8Arrray Sep 10, 2025
@mishig25
Copy link
Collaborator Author

mishig25 commented Sep 10, 2025

pushed df2d0a9 , for serialize functions to return Uint8Arrray, instead of Blob

Comment on lines +485 to +487
expect(result).toHaveProperty("littleEndian");
expect(typeof result.littleEndian).toBe("boolean");
// Most modern GGUF files should be little-endian
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
expect(result).toHaveProperty("littleEndian");
expect(typeof result.littleEndian).toBe("boolean");
// Most modern GGUF files should be little-endian

keep things a bit more compact no?

Copy link
Member

@julien-c julien-c left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a few things but lgtm

const version = typedMetadata.version.value;

// Start with GGUF magic number: "GGUF"
const magicNumber = new Uint8Array([0x47, 0x47, 0x55, 0x46]);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reuse ggufMagicNumber already in the file

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

handled in bc620b9

const version = typedMetadata.version.value;

// Start with GGUF magic number: "GGUF"
const magicNumber = new Uint8Array([0x47, 0x47, 0x55, 0x46]);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

handled in bc620b9

Comment on lines +768 to +773
export function serializeGgufHeader(
typedMetadata: GGUFTypedMetadata,
tensorInfos: GGUFTensorInfo[],
options: {
/**
* Whether to use little endian byte order
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm why is this function so duplicated / not calling serializeTypedMetadata? let's try to reuse it

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

handled in ff80f28

* @param options - Serialization options
* @returns A Uint8Array containing the serialized GGUF header data
*/
export function serializeTypedMetadata(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
export function serializeTypedMetadata(
export function serializeGgufMetadata(

suggestion: more simply lilke this

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

handled in 4eb5e95

@mishig25 mishig25 merged commit 9df67cd into main Sep 11, 2025
5 checks passed
@mishig25 mishig25 deleted the serialize_ggufmetadata branch September 11, 2025 08:47
mishig25 added a commit that referenced this pull request Sep 19, 2025
# `buildGgufHeader`: GGUF header reconstruction with proper alignment

## 🎯 Why `serializeGgufMetadata`
#1740 isn't enough

**`serializeGgufMetadata`** only handles metadata serialization but
**lacks critical functionality**:

- ❌ **No tensor info preservation** - loses original tensor definitions
- ❌ **No alignment calculation** - creates invalid memory layout  
- ❌ **Incomplete headers** - generates metadata-only files
- ❌ **Framework incompatible** - resulting files can't be loaded

## 🚀 `buildGgufHeader`: Complete solution

**`buildGgufHeader`** is a **comprehensive function** that uses
`serializeGgufMetadata` internally but adds essential missing pieces:

```typescript
export async function buildGgufHeader(originalFileBlob, updatedMetadata, options) {
  // 1. ✅ Use serializeGgufMetadata for metadata
  const newHeaderBytes = serializeGgufMetadata(updatedMetadata, {
    littleEndian: options.littleEndian,
    alignment,
  });

  // 2. ✅ Preserve original tensor info (missing in serializeGgufMetadata)
  const originalTensorInfoBlob = originalFileBlob.slice(tensorInfoStart, tensorInfoEnd);

  // 3. ✅ Calculate proper alignment (missing in serializeGgufMetadata)  
  const prePadLenNew = kvEndOffset + tensorInfoSize;
  const GGML_PAD = (x, n) => (x + n - 1) & ~(n - 1);
  const targetTensorDataOffset = GGML_PAD(prePadLenNew, alignment);
  const padLen = targetTensorDataOffset - prePadLenNew;

  // 4. ✅ Reconstruct complete, valid header
  return new Blob([
    newHeaderBytes.slice(0, kvEndOffset),  // Metadata from serializeGgufMetadata
    originalTensorInfoBlob,                // Preserved tensor info
    new Uint8Array(padLen)                 // Correct alignment padding
  ]);
}
```

## 📊 Comparison

| Function | Metadata | Tensor Info | Alignment | Complete Header |
|----------|----------|-------------|-----------|-----------------|
| `serializeGgufMetadata` | ✅ | ❌ | ❌ | ❌ |
| `buildGgufHeader` | ✅ | ✅ | ✅ | ✅ |

## 🎯 Key Innovation

**`buildGgufHeader` = `serializeGgufMetadata` + tensor preservation +
alignment calculation**

This creates **production-ready GGUF files** that maintain framework
compatibility while enabling metadata updates.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants