[v6] plumbing: format/idxfile, Fix version and fanout checks#1936
Merged
pjbgf merged 4 commits intogo-git:mainfrom Mar 28, 2026
Merged
[v6] plumbing: format/idxfile, Fix version and fanout checks#1936pjbgf merged 4 commits intogo-git:mainfrom
pjbgf merged 4 commits intogo-git:mainfrom
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Tightens .idx/.rev parsing and LazyIndex initialization validation to reject unsupported versions and detect corrupted fanout / reverse-index data, with expanded tests and fuzz coverage.
Changes:
- Enforce strict idx version matching and validate fanout monotonicity during decode/init.
- Rework
LazyIndex.EntriesByOffset()to stream offset order via.revrather than allocating + sorting, and update tests accordingly. - Improve error handling in packfile parsing callbacks and expand error-path tests/fuzzing for idx/rev decoding.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
plumbing/format/revfile/decoder_test.go |
Extends revfile decode tests to assert emitted positions on error paths. |
plumbing/format/packfile/parser.go |
Stops ignoring observer/storage errors during parse and returns contextual errors. |
plumbing/format/idxfile/lazy_index_test.go |
Adds init error tests; updates EntriesByOffset test to close iterator and assert ordering. |
plumbing/format/idxfile/lazy_index.go |
Adds fanout + rev range validation; changes EntriesByOffset to stream via .rev; adds iterator type. |
plumbing/format/idxfile/idxfile_test.go |
Ensures iterator returned by EntriesByOffset is closed in tests. |
plumbing/format/idxfile/idxfile.go |
Adds MemoryIndex guidance and checks write errors when building hashes. |
plumbing/format/idxfile/fuzz_test.go |
Adds decoder fuzz target seeded with real fixtures and large fixture data. |
plumbing/format/idxfile/decoder_test.go |
Replaces temp-file checksum test with table-driven decode error tests + helpers. |
plumbing/format/idxfile/decoder.go |
Enforces strict version match, validates fanout monotonicity, and adjusts offset64 counting type. |
Comments suppressed due to low confidence (1)
plumbing/format/idxfile/fuzz_test.go:94
fixture.Idx()returns a reader that should be closed (as done in other tests). Here it's passed directly toio.ReadAllwithout closing, which can leak file descriptors during fuzz runs. Assign it to a variable anddefer Close()(ort.Cleanup) after reading.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: Paulo Gomes <paulo@entire.io>
Signed-off-by: Paulo Gomes <paulo@entire.io>
Signed-off-by: Paulo Gomes <paulo@entire.io>
Signed-off-by: Paulo Gomes <paulo@entire.io>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 8 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
plumbing/format/idxfile/decoder.go:178
readOffsetsnow declareso64cntasint64, but uses it directly as the length inmake([]byte, o64cnt*8), which won’t compile becausemakerequires anintlength. Either keepo64cntasint, or cast with an explicit bounds check (and return an error ifo64cnt*8would overflowint).
func readOffsets(idx *MemoryIndex, r io.Reader) error {
var o64cnt int64
for k := range fanout {
if pos := idx.FanoutMapping[k]; pos != noMapping {
if _, err := io.ReadFull(r, idx.Offset32[pos]); err != nil {
return err
}
for p := 0; p < len(idx.Offset32[pos]); p += 4 {
if idx.Offset32[pos][p]&(byte(1)<<7) > 0 {
o64cnt++
}
}
}
}
if o64cnt > 0 {
idx.Offset64 = make([]byte, o64cnt*8)
if _, err := io.ReadFull(r, idx.Offset64); err != nil {
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tightens idxfile decoding validation by enforcing strict version matching and detecting corrupted fanout tables.