Skip to content

[v6] plumbing: format/idxfile, Fix version and fanout checks#1936

Merged
pjbgf merged 4 commits intogo-git:mainfrom
pjbgf:idx
Mar 28, 2026
Merged

[v6] plumbing: format/idxfile, Fix version and fanout checks#1936
pjbgf merged 4 commits intogo-git:mainfrom
pjbgf:idx

Conversation

@pjbgf
Copy link
Copy Markdown
Member

@pjbgf pjbgf commented Mar 28, 2026

Tightens idxfile decoding validation by enforcing strict version matching and detecting corrupted fanout tables.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Tightens .idx/.rev parsing and LazyIndex initialization validation to reject unsupported versions and detect corrupted fanout / reverse-index data, with expanded tests and fuzz coverage.

Changes:

  • Enforce strict idx version matching and validate fanout monotonicity during decode/init.
  • Rework LazyIndex.EntriesByOffset() to stream offset order via .rev rather than allocating + sorting, and update tests accordingly.
  • Improve error handling in packfile parsing callbacks and expand error-path tests/fuzzing for idx/rev decoding.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
plumbing/format/revfile/decoder_test.go Extends revfile decode tests to assert emitted positions on error paths.
plumbing/format/packfile/parser.go Stops ignoring observer/storage errors during parse and returns contextual errors.
plumbing/format/idxfile/lazy_index_test.go Adds init error tests; updates EntriesByOffset test to close iterator and assert ordering.
plumbing/format/idxfile/lazy_index.go Adds fanout + rev range validation; changes EntriesByOffset to stream via .rev; adds iterator type.
plumbing/format/idxfile/idxfile_test.go Ensures iterator returned by EntriesByOffset is closed in tests.
plumbing/format/idxfile/idxfile.go Adds MemoryIndex guidance and checks write errors when building hashes.
plumbing/format/idxfile/fuzz_test.go Adds decoder fuzz target seeded with real fixtures and large fixture data.
plumbing/format/idxfile/decoder_test.go Replaces temp-file checksum test with table-driven decode error tests + helpers.
plumbing/format/idxfile/decoder.go Enforces strict version match, validates fanout monotonicity, and adjusts offset64 counting type.
Comments suppressed due to low confidence (1)

plumbing/format/idxfile/fuzz_test.go:94

  • fixture.Idx() returns a reader that should be closed (as done in other tests). Here it's passed directly to io.ReadAll without closing, which can leak file descriptors during fuzz runs. Assign it to a variable and defer Close() (or t.Cleanup) after reading.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread plumbing/format/packfile/parser.go Outdated
Comment thread plumbing/format/idxfile/decoder.go
Comment thread plumbing/format/idxfile/lazy_index.go
Comment thread plumbing/format/idxfile/lazy_index.go
pjbgf added 4 commits March 28, 2026 14:26
Signed-off-by: Paulo Gomes <paulo@entire.io>
Signed-off-by: Paulo Gomes <paulo@entire.io>
Signed-off-by: Paulo Gomes <paulo@entire.io>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 8 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

plumbing/format/idxfile/decoder.go:178

  • readOffsets now declares o64cnt as int64, but uses it directly as the length in make([]byte, o64cnt*8), which won’t compile because make requires an int length. Either keep o64cnt as int, or cast with an explicit bounds check (and return an error if o64cnt*8 would overflow int).
func readOffsets(idx *MemoryIndex, r io.Reader) error {
	var o64cnt int64
	for k := range fanout {
		if pos := idx.FanoutMapping[k]; pos != noMapping {
			if _, err := io.ReadFull(r, idx.Offset32[pos]); err != nil {
				return err
			}

			for p := 0; p < len(idx.Offset32[pos]); p += 4 {
				if idx.Offset32[pos][p]&(byte(1)<<7) > 0 {
					o64cnt++
				}
			}
		}
	}

	if o64cnt > 0 {
		idx.Offset64 = make([]byte, o64cnt*8)
		if _, err := io.ReadFull(r, idx.Offset64); err != nil {

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@pjbgf pjbgf merged commit 5e903a3 into go-git:main Mar 28, 2026
27 of 28 checks passed
@pjbgf pjbgf deleted the idx branch March 28, 2026 14:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants