feat: add --batch-size flag to push refs in batches#173
Merged
Conversation
Add support for pushing refs in smaller batches to avoid server-side limits and timeouts when syncing large repositories with many tags/branches. - Add --batch-size flag (default 0 = no batching, original behavior) - Add References() method to GitRepository interface - Implement collectRefs() and pushRefsInBatches() helpers - Add MinBatchSize validation (must be 0 or >= 10) This addresses issues where repositories with 1000+ refs fail to sync to GHES with 'command error on refs/heads/<branch>: failed' errors.
There was a problem hiding this comment.
Pull request overview
This PR adds a --batch-size flag to the push and sync commands to enable pushing Git refs in smaller batches, helping avoid server-side limits when syncing repositories with many branches and tags.
Changes:
- Added
--batch-sizeflag with validation (0 for no batching, or minimum 10) - Implemented batching logic to collect and push refs in configurable batch sizes
- Added comprehensive unit tests for the new functionality
- Pinned tool dependencies to specific versions for Go 1.21 compatibility
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/push.go | Added batch-size flag, validation, collectRefs() function, and pushRefsInBatches() function to handle batched pushing |
| src/push_test.go | Added comprehensive tests for batch size validation, ref collection, and batched pushing functionality with mock implementations |
| src/git.go | Extended GitRepository interface with References() method and added (unused) RefInfo struct |
| src/git_test.go | Added tests verifying mock implementations satisfy the GitRepository and GitRemote interfaces |
| script/bootstrap | Pinned tool dependencies from @master to specific versions (v0.16.0, v1.6.0) for Go 1.21 compatibility |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
cd1a3b1 to
854c013
Compare
- Remove unused RefInfo struct from git.go - Remove redundant pushedAny variable tracking in pushRefsInBatches - Remove incomplete TestPushRefsInBatches_PartialUpToDate test (already covered by existing test case)
luketomlinson
approved these changes
Jan 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds a
--batch-sizeflag to thepushandsynccommands that allows pushing refs in smaller batches instead of all at once. This helps avoid server-side limits when syncing large repositories with many branches and tags.Problem
When syncing repositories with a large number of refs (branches + tags), the push operation can fail due to server-side limits on the number of refs that can be pushed in a single operation.
Solution
--batch-sizeflag (default: 0, meaning no batching for backward compatibility)+refs/heads/main:refs/heads/maininstead of wildcardsChanges
References()method toGitRepositoryinterface--batch-sizeflag with validation (must be 0 or >= 10)collectRefs()function to gather branch and tag refspushRefsInBatches()function to push refs in smaller batchesUsage
Testing