Skip to content

Do not use base58 encoded keys in the metabase#1529

Merged
fyrchik merged 1 commit intonspcc-dev:masterfrom
fyrchik:metabase-base58
Sep 15, 2022
Merged

Do not use base58 encoded keys in the metabase#1529
fyrchik merged 1 commit intonspcc-dev:masterfrom
fyrchik:metabase-base58

Conversation

@fyrchik
Copy link
Contributor

@fyrchik fyrchik commented Jun 16, 2022

Close #1482 .

name                        old time/op    new time/op    delta
Select/string_equal-8         8.87µs ± 5%    2.92µs ± 7%  -67.05%  (p=0.000 n=10+10)
Select/string_not_equal-8     4.95ms ±11%    1.34ms ± 9%  -73.04%  (p=0.000 n=10+10)
Select/common_prefix-8        55.5µs ± 6%    15.5µs ± 9%  -72.08%  (p=0.000 n=9+10)
Select/unknown-8              3.32µs ± 9%    0.85µs ± 8%  -74.46%  (p=0.000 n=10+10)
Put/parallel-8                1.27ms ±12%    1.13ms ±19%  -11.07%  (p=0.009 n=10+10)
Put/sequential-8              4.55ms ±14%    4.32ms ±12%     ~     (p=0.190 n=10+10)
ListWithCursor/1_item-8       4.96µs ±16%    3.25µs ± 7%  -34.32%  (p=0.000 n=10+10)
ListWithCursor/10_items-8     15.4µs ±14%     6.5µs ± 8%  -57.47%  (p=0.000 n=10+10)
ListWithCursor/100_items-8     119µs ± 8%      37µs ± 4%  -69.08%  (p=0.000 n=9+8)
Get/1_objects-8               8.46µs ±15%    3.96µs ±17%  -53.15%  (p=0.000 n=9+10)
Get/10_objects-8              92.8µs ±12%    41.8µs ±12%  -54.98%  (p=0.000 n=10+9)
Get/100_objects-8              944µs ±12%     415µs ± 8%  -56.05%  (p=0.000 n=10+9)

name                        old alloc/op   new alloc/op   delta
Put/parallel-8                 133kB ±17%     135kB ±10%     ~     (p=0.631 n=10+10)
Put/sequential-8               174kB ± 4%     186kB ± 4%   +6.74%  (p=0.000 n=10+9)
ListWithCursor/1_item-8       1.91kB ± 0%    1.49kB ± 0%  -21.80%  (p=0.000 n=10+10)
ListWithCursor/10_items-8     5.50kB ± 0%    2.77kB ± 0%  -49.59%  (p=0.002 n=8+10)
ListWithCursor/100_items-8    41.5kB ± 0%    15.7kB ± 0%  -62.20%  (p=0.000 n=10+10)
Get/1_objects-8               2.83kB ± 0%    2.16kB ± 0%  -23.73%  (p=0.000 n=9+10)
Get/10_objects-8              29.6kB ± 2%    22.7kB ± 0%  -23.16%  (p=0.000 n=10+10)
Get/100_objects-8              295kB ± 1%     227kB ± 0%  -23.20%  (p=0.000 n=10+10)

name                        old allocs/op  new allocs/op  delta
Put/parallel-8                   478 ± 1%       343 ± 2%  -28.18%  (p=0.000 n=10+10)
Put/sequential-8                 811 ± 1%       670 ± 1%  -17.37%  (p=0.000 n=10+10)
ListWithCursor/1_item-8         34.0 ± 0%      27.0 ± 0%  -20.59%  (p=0.000 n=10+10)
ListWithCursor/10_items-8        100 ± 0%        51 ± 0%  -49.00%  (p=0.000 n=10+10)
ListWithCursor/100_items-8       758 ± 0%       290 ± 0%  -61.74%  (p=0.000 n=10+10)
Get/1_objects-8                 48.0 ± 0%      36.0 ± 0%  -25.00%  (p=0.000 n=9+10)
Get/10_objects-8                 506 ± 1%       389 ± 0%  -23.09%  (p=0.000 n=10+10)
Get/100_objects-8              5.07k ± 1%     3.89k ± 0%  -23.24%  (p=0.000 n=10+10)

@codecov
Copy link

codecov bot commented Jun 16, 2022 •
edited
Loading

Codecov Report

Merging #1529 (126254b) into master (d6fef68) will increase coverage by 0.08%.
The diff coverage is 90.00%.

@@            Coverage Diff             @@
##           master    #1529      +/-   ##
==========================================
+ Coverage   33.58%   33.67%   +0.08%     
==========================================
  Files         352      352              
  Lines       23637    23682      +45     
==========================================
+ Hits         7938     7974      +36     
- Misses      15040    15045       +5     
- Partials      659      663       +4
Impacted Files Coverage Δ
pkg/local_object_storage/metabase/shard_id.go 28.57% <ø> (ø)
pkg/local_object_storage/metabase/version.go 88.88% <ø> (ø)
pkg/local_object_storage/metabase/counter.go 73.97% <50.00%> (+0.36%) ⬆️
pkg/local_object_storage/metabase/iterators.go 65.85% <50.00%> (-1.24%) ⬇️
pkg/local_object_storage/metabase/db.go 69.04% <60.00%> (-1.84%) ⬇️
pkg/local_object_storage/metabase/list.go 95.40% <80.00%> (-2.48%) ⬇️
pkg/local_object_storage/metabase/delete.go 78.40% <85.00%> (+0.37%) ⬆️
pkg/local_object_storage/metabase/util.go 87.83% <88.46%> (-3.83%) ⬇️
pkg/local_object_storage/metabase/select.go 78.79% <96.15%> (+1.60%) ⬆️
pkg/local_object_storage/metabase/containers.go 92.98% <100.00%> (-0.13%) ⬇️
... and 10 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@fyrchik fyrchik mentioned this pull request Jun 21, 2022
Merged
@fyrchik
Copy link
Contributor Author

fyrchik commented Jun 24, 2022

Ready for review, will remove draft mark after the release.

@cthulhu-rider
Copy link
Contributor

cthulhu-rider commented Sep 6, 2022

@fyrchik results look nice, can we finalize the PR?

@fyrchik
Copy link
Contributor Author

fyrchik commented Sep 9, 2022

New benchmarks -with the current master:

name                        old time/op    new time/op    delta
Get/1_objects-8               11.9µs ± 7%     4.4µs ± 3%  -62.94%  (p=0.000 n=8+9)
Get/10_objects-8               120µs ± 7%      56µs ±19%  -53.61%  (p=0.000 n=10+10)
Get/100_objects-8             1.23ms ± 6%    0.57ms ± 7%  -53.58%  (p=0.000 n=10+10)
ListWithCursor/1_item-8       5.04µs ± 8%    4.17µs ± 9%  -17.31%  (p=0.000 n=10+10)
ListWithCursor/10_items-8     16.1µs ±11%     8.1µs ± 3%  -49.63%  (p=0.000 n=10+8)
ListWithCursor/100_items-8     117µs ± 6%      46µs ± 7%  -60.51%  (p=0.000 n=10+10)
Put/parallel-8                1.56ms ± 8%    1.37ms ±28%  -12.06%  (p=0.002 n=10+10)
Put/sequential-8              5.03ms ± 6%    4.72ms ±14%     ~     (p=0.068 n=8+10)
Select/string_equal-8         11.5µs ± 3%     3.7µs ±10%  -67.63%  (p=0.000 n=9+10)
Select/string_not_equal-8     7.52ms ± 1%    1.82ms ± 3%  -75.82%  (p=0.000 n=8+7)
Select/common_prefix-8        85.9µs ± 3%    20.7µs ± 7%  -75.91%  (p=0.000 n=9+10)
Select/unknown-8              3.34µs ± 6%    1.05µs ± 9%  -68.51%  (p=0.000 n=10+9)

name                        old alloc/op   new alloc/op   delta
Get/1_objects-8               3.32kB ± 0%    1.86kB ± 0%  -43.86%  (p=0.000 n=9+10)
Get/10_objects-8              34.9kB ± 1%    23.2kB ± 1%  -33.56%  (p=0.000 n=9+10)
Get/100_objects-8              351kB ± 0%     234kB ± 0%  -33.26%  (p=0.000 n=8+10)
ListWithCursor/1_item-8       1.91kB ± 0%    1.49kB ± 0%  -21.80%  (p=0.000 n=10+10)
ListWithCursor/10_items-8     5.50kB ± 0%    2.77kB ± 0%  -49.59%  (p=0.000 n=10+10)
ListWithCursor/100_items-8    41.5kB ± 0%    15.7kB ± 0%  -62.20%  (p=0.000 n=7+10)
Put/parallel-8                 118kB ± 6%     131kB ± 9%  +11.17%  (p=0.000 n=9+10)
Put/sequential-8               171kB ± 1%     184kB ± 2%   +7.83%  (p=0.000 n=10+10)

name                        old allocs/op  new allocs/op  delta
Get/1_objects-8                 57.0 ± 0%      36.0 ± 0%  -36.84%  (p=0.000 n=9+10)
Get/10_objects-8                 606 ± 1%       408 ± 0%  -32.58%  (p=0.000 n=10+10)
Get/100_objects-8              6.10k ± 0%     4.09k ± 0%  -32.95%  (p=0.000 n=8+10)
ListWithCursor/1_item-8         34.0 ± 0%      27.0 ± 0%  -20.59%  (p=0.000 n=10+10)
ListWithCursor/10_items-8        100 ± 0%        51 ± 0%  -49.00%  (p=0.000 n=10+10)
ListWithCursor/100_items-8       758 ± 0%       290 ± 0%  -61.74%  (p=0.000 n=10+10)
Put/parallel-8                   490 ± 1%       335 ± 1%  -31.62%  (p=0.000 n=10+10)
Put/sequential-8                 833 ± 0%       672 ± 1%  -19.38%  (p=0.000 n=10+10)

cthulhu-rider
cthulhu-rider previously approved these changes Sep 12, 2022
View reviewed changes
@fyrchik
[nspcc-dev#1482] metabase: Encode database keys in binary
126254b 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
@fyrchik
Copy link
Contributor Author

fyrchik commented Sep 15, 2022

Rebased on master

@fyrchik fyrchik merged commit ae1dab2 into nspcc-dev:master Sep 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cthulhu-rider cthulhu-rider cthulhu-rider approved these changes

@carpawell carpawell Awaiting requested review from carpawell carpawell is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Do not use base58 strings as keys in the metabase

2 participants

@fyrchik @cthulhu-rider