Skip to content

use an order-preserving, scan-friendly database key encoding #4567

@erikgrinaker

Description

@erikgrinaker

The BlockStore currently uses this encoding for e.g. block metadata keys:

[]byte(fmt.Sprintf("H:%v", height))

This uses alphabetical ordering instead of numerical ordering, such that e.g. block 10 is ordered between block 1 and block 100, not between 9 and 11. This makes it impossible to do efficient range scans - if we e.g. want to prune all blocks between 0 and 1000000 we have to explicitly test for the existence of each and every one rather than simply scan the keys that actually exist. Since this obviously does not scale, one must resort to e.g. short-circuiting a reverse iteration on the first missing key, which is not robust.

The encoding should instead use the big-endian binary representation of the number, or some other order-preserving encoding.

Metadata

Metadata

Assignees

Labels

T:breakingType: Breaking ChangeT:encodingType: Amino, ProtoBufT:jankType Jank! Non-urgent but still high-impact fixes.T:perfType: Performance

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions