Skip to content

Sha 256 support #281

@Byron

Description

@Byron

Even though the foundation is set, it needs another push to actually make it work with different kinds of hashes.

Tasks

  • remove hash-type specific methods from git-hash and replace them with parametric usage of git_hash::Kind
  • all code assuming hashes of len 20 should receive this value as parameter instead. This is what git does for the old index and pack file formats.
  • a way to pass --object-hash information to the gix CLI
  • Remove default sha1 feature from gix-hash crate and deal with the fallout
    • Add forwarding features to every plumbing crate that depends on it directly or indirectly
    • adjust tests to set sha1 by default
    • configure docs.rs to enable sha1
    • CI should execute script that runs cargo check -p gix-<name> to show it's failing with an error message
    • adjust all other jobs
    • let gix chose SHA1 as default
  • remove SHA1 mention from git-features feature toggles
  • parameterize hash len when decoding non-blob objects (see this for an example)
  • understand and implement pack idx V3. - see if git actually implements this, and maybe decide that gitoxide won't handle the transition period, is either one has or another.
  • add new Sha256 enum variant, consider putting it behind a feature flag, and add a hasher for it as well.
  • general tests for reading refs and objects of different len
  • tests for writing and reading objects of different len, maybe even write a conversion program which transforms an entire repo and double-checks with git-fsck
  • when cloning, check the uninmplemented!() invocation to configure the repo for expecting a different hash

Crates Completed

  • gix-commitgraph - minor testing, run tests with multiple hash kinds

Tasks

(generated by Codex, needs individual review and adjustment)

Based on issue #281 and current repo state, here is an effort-ordered crate list.

  • gix-hash (small, foundational): remove default sha1, keep explicit sha1/sha256 features, and keep no-default build/tests green; watch out for Cargo feature additivity and clear compile errors when no hash is selected.
  • gix-refspec (small): add explicit SHA256 parse tests for object-hash-looking refspecs; watch out for logic that implicitly treats 40-hex as “an object hash”.
  • gix-filter + gix-worktree-stream (small to medium): add explicit dual-hash tests for filter/stream paths; watch out for protocol/hash-kind encoding paths that currently default to SHA1 semantics.
  • gix-blame + gix-diff (medium): remove hardcoded Kind::Sha1.null() sentinels where possible and add SHA256 fixtures; watch out for sentinel IDs that must stay “impossible/non-matching” rather than repo-real IDs.
  • gix-object (medium): parameterize non-blob/tree decode by hash length and add SHA256 object decode/encode tests; watch out for the tree parser hotspot that still splits at 20 bytes (TODO(SHA256)).
  • gix-ref (medium): add explicit refs/reflog read-write tests for both hash lengths; watch out for many 40-char and SHA1-null expectations in tests.
  • gix-index (medium-high): extend index/extension roundtrip tests for SHA256; watch out for offset/checksum math tied to len_in_bytes() and trailing checksum region assumptions.
  • gix-protocol + gix-transport (medium-high): add object-format=sha256 negotiation tests end-to-end; watch out for code paths that currently accept/fallback only to SHA1.
  • gix-odb (high): add explicit loose/packed read-write and prefix lookup tests under SHA256; watch out for SHA1-shaped helpers (e.g., 20-byte helpers and 40-hex prefix assumptions).
  • gix-pack (very high): full SHA256 coverage for pack data/index/multi-index read-write and verification; watch out for SHA1-only assertions and format/version assumptions in pack/index code.
  • gix (very high): implement and test clone/fetch repo reconfiguration when remote hash kind differs; watch out for the current clone unimplemented!() on object-hash mismatch.
  • double-check for TODO(SHA256) or similar, and for gix_hash::Kind::Sha1::null() and similar

Implementation ideas

  • make sure once Sha256 is added as ObjectId variant, that it's behind a feature toggle to allow builds that opt-out of SHA256 support to not unnecessarily use more memory than needed. Maybe there are alternatives to this, too.
  • One way to do that with approximately zero overhead would be to such functions generic on the object ID, using a trait that has a method to get the type. Then object IDs with a known type return a constant from that method, and object IDs with a runtime dispatched type return the value of that enum.

    • @joshtriplett - taken verbatim as I'd barely be able to improve on it when paraphrasing. In short, have a trait for oid or allow efficient conversions to oid (it's just a slice, so that should work for specifically sized types as well especially if these were provided by git-hash.

Notes

  • find ways to use the existing highly-parallel pack traversal (along with integration of loose-objects) to build an inverse-ref table to quickly traverse objects bottom-up to change the hash used along with all references, while being fast. This ties into being able to build new packs quickly, ideally even with delta-compression (the latter then has to be re-created as most objects actually change) - re-using deltas for blobs is the only way.
    • The existing traversal can mutate data in the tree, which is enough to decode the object and keep direct references for later.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-tracking-issueAn issue to track to track the progress of multiple PRs or issueshelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions