This release includes significant improvements in performance and critical fixes in both uploading and downloading large files. It is strongly recommended to upgrade to this release.

Note: v1.4.2 replaces v1.4.1 which had a packaging issue (version bump was merged after the release was built). The code is identical.

New Features and Improvements

• Upload optimization avoids unnecessary duplicate SHA-256 computation (@XciD in #679 and #705)
• Download buffer memory is now scaled by active downloads (@hoytak in #666)

Bug Fixes and Enhancements

• Resolve potential deadlock / stall on large file downloads (@XciD in #698)
• Shard Upload extended timeout (was silently timing out on large uploads previously) (@rajatarya in #685)
• Improvements to monitoring and simulation/testing (@seanses in #617, @hoytak in #681)

What's Changed

• Scale download buffer memory limit by number of active downloads by @hoytak in #666
• Feature to monitor client process system usage by @seanses in #617
• Streaming data writer by @hoytak in #656
• Fix command injection in release workflow (CVE) by @XciD in #677
• feat: accept pre-computed SHA-256 in upload_files() by @XciD in #678
• XetSession API by @seanses in #657
• Naming clarification: A Xorb is a data object, CAS is the remote server. by @hoytak in #680
• Fix for incorrect error propagation on truncated download stream. by @hoytak in #683
• Simulation interface for LocalTestServer: supports deletion, direct access, data dumps, etc. by @hoytak in #681
• Rework simulation pipeline for adaptive concurrency and connection resiliency. by @hoytak in #648
• feat: add skip_sha256 option to SingleFileCleaner by @XciD in #679
• fix: prevent download stall on large file reconstruction by @XciD in #698
• fix: per-request timeout for shard uploads (XET-885) by @rajatarya in #685
• version bump to 1.4.0 by @rajatarya in #699
• Code reorganization towards release of xet cargo package by @hoytak in #693
• Record API changes in api_changes/updates__.md by @hoytak in #689
• Rework the interface for session task to get result from registered upload by @seanses in #690
• feat: expose skip_sha256 parameter in Python upload API by @XciD in #705
• version bump to 1.4.1 by @rajatarya in #707
• chore: version bump to 1.4.2 by @XciD in #712

Full Changelog: v1.3.2...v1.4.2

This release includes significant improvements in performance and critical fixes in both uploading and downloading large files. It is strongly recommended to upgrade to this release.

✨ New Features and Improvements

• Upload optimization avoids unnecessary duplicate SHA-256 computation (@XciD in #679 and #705)
• Download buffer memory is now scaled by active downloads (@hoytak in #666)

🐛 Bug Fixes and Enhancements

• Resolve potential deadlock / stall on large file downloads (@XciD in #698)
• Shard Upload extended timeout (was silently timing out on large uploads previously) (@rajatarya in #685)
• Improvements to monitoring and simulation/testing (@seanses in #617, @hoytak in #681)

What's Changed

• Scale download buffer memory limit by number of active downloads by @hoytak in #666
• Feature to monitor client process system usage by @seanses in #617
• Streaming data writer by @hoytak in #656
• Fix command injection in release workflow (CVE) by @XciD in #677
• feat: accept pre-computed SHA-256 in upload_files() by @XciD in #678
• XetSession API by @seanses in #657
• Naming clarification: A Xorb is a data object, CAS is the remote server. by @hoytak in #680
• Fix for incorrect error propagation on truncated download stream. by @hoytak in #683
• Simulation interface for LocalTestServer: supports deletion, direct access, data dumps, etc. by @hoytak in #681
• Rework simulation pipeline for adaptive concurrency and connection resiliency. by @hoytak in #648
• feat: add skip_sha256 option to SingleFileCleaner by @XciD in #679
• fix: prevent download stall on large file reconstruction by @XciD in #698
• fix: per-request timeout for shard uploads (XET-885) by @rajatarya in #685
• version bump to 1.4.0 by @rajatarya in #699
• Code reorganization towards release of xet cargo package by @hoytak in #693
• Record API changes in api_changes/updates__.md by @hoytak in #689
• Rework the interface for session task to get result from registered upload by @seanses in #690
• feat: expose skip_sha256 parameter in Python upload API by @XciD in #705

Full Changelog: v1.3.2...v1.4.1

🐛 Bug Fixes and Enhancements

• Up to 4x speedup on downloads when downloading multiple files at once by @Hugoch in #668

What's Changed

• Bump time from 0.3.44 to 0.3.47 in /hf_xet_wasm by @dependabot[bot] in #654
• fix: wrap TrackingProgressUpdater in AggregatingProgressUpdater by @Hugoch in #668
• Fixes for intermittent test failures on windows. by @hoytak in #669
• hf-xet 1.3.2 version bump by @rajatarya in #671

Full Changelog: v1.3.1...v1.3.2

What's Changed

• Add Windows ARM64 build support by @rajatarya in #662
• Version bump for hf-xet 1.3.1 release by @rajatarya in #665

Full Changelog: v1.3.0...v1.3.1

Adaptive concurrency control

hf-xet uploads and downloads now intelligently control concurrency, make the best use of bandwidth while keeping reasonable memory usage.

Bug fixes and enhancements

• Better Ctrl-C detection and handling on Windows
• Unix Socket Proxy
• Better testing

New Contributors

• @Wrufesh made their first contribution in #584
• @salmanmkc made their first contribution in #600

Full Changelog: v1.2.0...v1.3.0

What's Changed

1. Update “git xet install” command to also run “git lfs install” if git-lfs not configured yet, this streamlines the “brew install git-xet” workflow.
2. Remove “homebrew/openssl@3” dependency from git-xet macOS build from Github Actions.
3. Enabled adaptive concurrency for upload

Full Changelog: git-xet-v0.2.0...git-xet-v0.2.1

1. Extends support to Windows platform and SSH remote URL, marking all three major platforms supported with both HTTP and SSH remote URL.
2. Add “git xet track” command to replace “git lfs track” to unify branding.
3. Upload performance improvement: avoid re-computation of SHA256 in git-xet and uses the value passed in from git-lfs.

Full Changelog: git-xet-v0.1.0...git-xet-v0.2.0

✨ New Features and Improvements

• New file-based logging system to support enhanced diagnostics and debugging (by @hoytak in #502)
• SOCKS5 Proxy support (by @SuperKenVery in #474)
• Support for Free-threaded Python 3.13 and 3.14 (by @rajatarya @seanses in #524)
• Improved performance by disabling disk-based chunk cache by default (by @rajatarya in #535)
• Updated rust edition to 2024, upgrade rustc to 1.89 (by @seanses in #494)

🐛 Bug Fixes and Enhancements

• Improved documentation for some crates (by @assafvayner in #492)
• Improved user-configurable constant handling (by @hoytak in #493)
• Bug fixes to diagnostic scripts and build workflows
• Remove Xet protocol docs as they live elsewhere now (see https://huggingface.co/docs/xet/index)

What's Changed

• Adding README to few crates for documentation by @assafvayner in #492
• Rename Threadpool class name to XetRuntime to reflect usage by @hoytak in #491
• Improved user-configurable constant handling by @hoytak in #493
• upgrade rust edition to 2024; upgrade rustc to 1.89 by @seanses in #494
• MacOS diag scripts by @rajatarya in #497
• Allow Duration and byte sizes in constants for easier use. by @hoytak in #495
• Build and release git-xet by @seanses in #499
• Fix git xet release bug by @seanses in #504
• Better support "xet-write-token" API authorization model and LFS Batch Api change by @seanses in #498
• Fix git-xet release bug by @seanses in #505
• git-xet install script for Linux & macOS by @seanses in #503
• Convert status code to error for get_cas_jwt by @seanses in #509
• Cache rust build in actions by @seanses in #513
• hashing and chunking example tools by @assafvayner in #496
• Enable socks5 proxy support by @SuperKenVery in #474
• spec draft by @assafvayner in #422
• Added lazy evaluation functionality to error printer. by @hoytak in #510
• move spec to docs by @assafvayner in #515
• integrate docs debugging by @assafvayner in #516
• rm all docs files by @assafvayner in #517
• git-xet Windows installer and code signing by @seanses in #519
• Fix python 314 compat by @Qubitium in #520
• Upgrade macos-13 to macos-15-intel due to closing down by @seanses in #521
• openapi spec and Makefile for it by @assafvayner in #518
• fix: explicitly specify main branch for hub client in migration utility by @sirahd in #522
• Create README.md for Git-Xet by @seanses in #529
• Logging to directory + log file management; default to log directory for hf_xet by @hoytak in #502
• Fix breaking build changes due to git safety checks. by @hoytak in #530
• Adding python-version 3.13t and 3.14t to builds by @rajatarya in #524
• Add fallback if unable to get git commit by @rajatarya in #531
• Fix typos by @omahs in #508
• Fix clippy issues with new rust version. by @hoytak in #533
• Disable DiskCache in hf_xet, continue to use it in git_xet by @rajatarya in #535
• Improved logging for cas_client crate by @hoytak in #537
• Test suite for directory logging functionality by @hoytak in #536
• fix: small typo by @crStiv in #534
• version bump to 1.2.0 for release by @rajatarya in #539
• Update Python classifiers by @rajatarya in #540

New Contributors

• @SuperKenVery made their first contribution in #474
• @Qubitium made their first contribution in #520
• @omahs made their first contribution in #508
• @crStiv made their first contribution in #534

Full Changelog: v1.1.10...v1.2.0

Git-Xet is a Git LFS custom transfer agent that implements upload and download of files using the Xet protocol. Follow your regular workflow to git lfs track ... & git add ... & git commit ... & git push and your files are uploaded to Hugging Face repos automatically using the Xet protocol. Enjoy the dedupe!

Installation

1. Make sure you have git and git-lfs installed and configured correctly.
2. For Linux (amd64 & aarch64) and macOS (amd64 & aarch64), run the following in your terminal to install and configure git-xet (requires curl and unzip):

curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/huggingface/xet-core/refs/heads/main/git_xet/install.sh | sh

3. For Windows (amd64), either

• download git-xet-windows-installer-x86_64.zip and run the msi file after unzip, or
• download git-xet-windows-x86_64.zip and place git-xet.exe under a PATH directory, and run git-xet install in a terminal.

How It Works

Git-Xet works by registering itself as a custom transfer agent to Git LFS by name "xet". On "git push", "git fetch" or "git pull", git-lfs negotiates with the remote server to determine the transfer agent to use. During this process, git-lfs sends to the server all locally registered agent names in the Batch API request, and the server replies with exactly one agent name in the response. Should "xet" be picked, git-lfs delegates the uploading or downloading operation to Git-Xet through a sequential protocol.

For more details regarding Git LFS custom transfer agent protocol, see https://github.com/git-lfs/git-lfs/blob/main/docs/api/batch.md and https://github.com/git-lfs/git-lfs/blob/main/docs/custom-transfers.md.

🔧 Improvements & Tools:

• Comprehensive Diagnostic Scripts - New debugging tools for Linux and Windows
• Network Reliability Enhancements - Better retry logic for I/O errors
• Simplified DNS resolution to run in Kubernetes environments
• CAS API Path Modernization - Updated to use plural nouns following REST conventions

🐛 Bug Fixes:

• Chunker Boundary Triggering Fix - Fixed deduplication consistency issues
• WASM First Chunk Dedup Handling - Improved client-side control
• Data Type Safety Enhancements - Standardized on u64 for cross-platform compatibility

What's Changed

• Add input params to Run name in GH Workflow UI by @rajatarya in #478
• Thin wasm: do not automatically set is_dedup to true for first chunk by @coyotte508 in #481
• update api paths to use plural nouns by @assafvayner in #482
• Rename xet_threadpool to xet_runtime to reflect usage by @hoytak in #484
• use u64 rather than usize in file hashing paths by @assafvayner in #485
• Git-Xet: LFS custom transfer agent with Xet protocol by @seanses in #425
• Drop "GaiResolverWithAbsolute" by @seanses in #486
• Fix wheel upload for linux for dev/alpha/beta tags by @hoytak in #379
• Adding retry for unhandled io errors when sending requests by @jgodlew in #468
• Updated chunker to eliminate spurious boundary triggering. by @hoytak in #487
• Diagnostic Scripts + README changes by @rajatarya in #489
• hf_xet 1.1.10 by @assafvayner in #490

Full Changelog: v1.1.9...v1.1.10