Skip to content

Feat/Logical object counters#1771

Merged
cthulhu-rider merged 7 commits intonspcc-dev:masterfrom
carpawell:feat/logical-object-counters
Sep 13, 2022
Merged

Feat/Logical object counters#1771
cthulhu-rider merged 7 commits intonspcc-dev:masterfrom
carpawell:feat/logical-object-counters

Conversation

@carpawell
Copy link
Member

@carpawell carpawell commented Sep 8, 2022

Closes #1658.

# HELP neofs_node_object_counter Objects counters per shards
# TYPE neofs_node_object_counter gauge
neofs_node_object_counter{shard="H3hDLxLhK78d7Gd4izmsMM",type="logic"} 3
neofs_node_object_counter{shard="H3hDLxLhK78d7Gd4izmsMM",type="phy"} 4
neofs_node_object_counter{shard="RDUb4UwA3rvMbnqEjPZWA8",type="logic"} 1
neofs_node_object_counter{shard="RDUb4UwA3rvMbnqEjPZWA8",type="phy"} 1

Will add tests for high-level components. UPD: done.

@carpawell carpawell self-assigned this Sep 9, 2022
@carpawell carpawell added enhancement Improving existing functionality neofs-storage Storage node application issues labels Sep 9, 2022
@codecov
Copy link

codecov bot commented Sep 9, 2022

Codecov Report

Merging #1771 (498c797) into master (4afb928) will increase coverage by 0.19%.
The diff coverage is 64.44%.

@@            Coverage Diff             @@
##           master    #1771      +/-   ##
==========================================
+ Coverage   33.38%   33.58%   +0.19%     
==========================================
  Files         352      352              
  Lines       23499    23637     +138     
==========================================
+ Hits         7845     7938      +93     
- Misses      15004    15040      +36     
- Partials      650      659       +9     
Impacted Files Coverage Δ
pkg/local_object_storage/engine/metrics.go 0.00% <ø> (ø)
pkg/local_object_storage/engine/shards.go 66.29% <0.00%> (ø)
pkg/local_object_storage/shard/gc.go 23.64% <0.00%> (-0.36%) ⬇️
pkg/local_object_storage/metabase/control.go 77.77% <20.00%> (-4.58%) ⬇️
pkg/local_object_storage/metabase/put.go 73.57% <40.00%> (-0.44%) ⬇️
pkg/local_object_storage/shard/control.go 76.08% <42.85%> (-1.95%) ⬇️
pkg/local_object_storage/metabase/counter.go 73.61% <72.09%> (-0.31%) ⬇️
pkg/local_object_storage/metabase/delete.go 78.03% <75.00%> (+1.11%) ⬆️
pkg/local_object_storage/metabase/inhume.go 85.54% <86.66%> (+1.54%) ⬆️
pkg/local_object_storage/shard/delete.go 69.76% <100.00%> (+0.71%) ⬆️
... and 7 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@carpawell carpawell marked this pull request as ready for review September 9, 2022 11:43
It will allow to use labeled object counters, e.g.: "phy" for all
the physically stored objects, "logic" for only available objects. Also, it
will allow to add typed (regular, TS, SG, LOCK) counters if needed.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
It now supports typed object counter metrics.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
@carpawell carpawell requested a review from fyrchik September 12, 2022 17:37
cthulhu-rider
cthulhu-rider previously approved these changes Sep 13, 2022
Do not treat objects with expiration as expired by default.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
- Meta now supports (and requires) inc/dec labeled counters
- The new logic counter is incremented on `Put` operations and is
decremented on `Inhume` and `Delete` operations that are performed on
_stored_ objects only
- Allow force counters sync. "Force" mode should be used on metabase resync
and should not be used on a regular meta start

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
cthulhu-rider
cthulhu-rider previously approved these changes Sep 13, 2022
@carpawell
Copy link
Member Author

Also, added tests on the shard level.

Use meta's operation results to change the metrics. Support typed object
counters.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Do not perform operations that produce unused results in `syncCounter`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
@cthulhu-rider cthulhu-rider merged commit 532c41c into nspcc-dev:master Sep 13, 2022
@carpawell carpawell deleted the feat/logical-object-counters branch September 13, 2022 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Improving existing functionality neofs-storage Storage node application issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add metric with object count for the node and for each shard.

3 participants