Skip to content

feat(importer) Implement Rabin Fingerprinting #56

@whyrusleeping

Description

@whyrusleeping

Our current file chunking scheme is just splitting on blocks of a certain size. This is bad for a variety of reasons. @jbenet and I think it will be best to use rabin fingerprinting as our default method for file chunking. The downside is that, other than a brief description on wikipedia, there isnt much to go on for the algorithm itself, so its gonna be a bit of fun figuring this out.

Metadata

Metadata

Assignees

Labels

kind/enhancementA net-new feature or improvement to an existing feature

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions