Skip to content

Feat/Persistent session storage#1255

Merged
alexvanin merged 8 commits intonspcc-dev:masterfrom
carpawell:feat/persistent-sessions
Mar 29, 2022
Merged

Feat/Persistent session storage#1255
alexvanin merged 8 commits intonspcc-dev:masterfrom
carpawell:feat/persistent-sessions

Conversation

@carpawell
Copy link
Member

@carpawell carpawell commented Mar 20, 2022

Closes #1189. Not sure if node admin should be able to configure encryption.

Please, double-check the encryption approach.

@codecov
Copy link

codecov bot commented Mar 20, 2022

Codecov Report

Merging #1255 (04ff4f8) into master (5c52796) will increase coverage by 0.15%.
The diff coverage is 45.95%.

❗ Current head 04ff4f8 differs from pull request most recent head b475feb. Consider uploading reports for the commit b475feb to get more accurate results

@@            Coverage Diff             @@
##           master    #1255      +/-   ##
==========================================
+ Coverage   35.87%   36.03%   +0.15%     
==========================================
  Files         296      301       +5     
  Lines       18518    18778     +260     
==========================================
+ Hits         6644     6766     +122     
- Misses      11375    11484     +109     
- Partials      499      528      +29     
Impacted Files Coverage Δ
cmd/neofs-cli/modules/acl/extended/create.go 52.14% <0.00%> (-1.15%) ⬇️
cmd/neofs-cli/modules/object.go 15.42% <0.00%> (-0.53%) ⬇️
cmd/neofs-cli/modules/util.go 25.27% <0.00%> (-0.48%) ⬇️
pkg/innerring/processors/netmap/process_cleanup.go 0.00% <0.00%> (ø)
pkg/innerring/processors/netmap/process_peers.go 0.00% <0.00%> (ø)
pkg/network/address.go 90.90% <ø> (ø)
pkg/services/object/acl/acl.go 24.00% <0.00%> (-0.80%) ⬇️
pkg/services/object/acl/v2/errors.go 0.00% <0.00%> (ø)
pkg/services/object/acl/v2/service.go 0.00% <0.00%> (ø)
pkg/local_object_storage/metabase/db.go 76.81% <33.33%> (-6.80%) ⬇️
... and 14 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5c52796...b475feb. Read the comment docs.

Move in-memory session storage to the separate directory of `storage`. It is
done for future support of different kind of session storages.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
@carpawell carpawell self-assigned this Mar 21, 2022
@carpawell carpawell added enhancement Improving existing functionality neofs-storage Storage node application issues labels Mar 21, 2022
@carpawell carpawell marked this pull request as ready for review March 21, 2022 15:20
Comment on lines +48 to +67
// nil value is a hallmark
// of the nested buckets
if v == nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have anything other that sub-buckets in the root bucket?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in fact, no, but the idea that a node could potentially use storage that has not been created by it haunts me

Comment on lines +157 to +159
// This test was added to fix bolt's behaviour since the persistent
// storage uses cursor and there is an issue about `cursor.Delete`
// method: https://github.com/etcd-io/bbolt/issues/146.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, unexpected. As I understand, this is not fixed yet?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is an old issue (origins to the original archived repo) and I could reproduce key skipping when did such test in the one tx (updating and iterating in one Update call), so technically it is still an issue (at least there is no detailed behavior description about such use-cases)

however, code written in that PR works fine, but I've decided to add that test to know when bbolt's maintainers have done something breaking about that issue

Comment on lines +70 to +83
rawKey, err := x509.MarshalECPrivateKey(cfg.privateKey)
if err != nil {
return nil, fmt.Errorf("could not marshal provided private key: %w", err)
}

// tagOffset is a constant offset for
// tags when marshalling ECDSA key in
// ASN.1 DER form
const tagOffset = 7

// using first 32 bytes from
// the marshalled private key
// as a secret
c, err := aes.NewCipher(rawKey[tagOffset : tagOffset+32])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/golang/go/blob/master/src/crypto/x509/sec1.go#L58

rawKey := make([]byte, (key.Curve.Params().N.BitLen()+7)/8)
cfg.privateKey.D.FillBytes(rawKey)

Also works for every private key type, not just 32-byte one.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks better, does the same, thanks

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Add `WithEncryption` option that passes ECDSA key to the persistent session
storage. It uses 32 bytes from marshalled ECDSA key in ASN.1 DER from in
AES-256 algorithm encryption in Galois/Counter Mode.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>

data := []byte("nice encryption, awesome tests")

encryptedData, err := ts.encrypt(data)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Compare that encrypted and decrypted data are not the same.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Use persistent storage usage in the node if it was configured so.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
@alexvanin alexvanin merged commit 6ec104d into nspcc-dev:master Mar 29, 2022
@carpawell carpawell deleted the feat/persistent-sessions branch March 29, 2022 07:07
cthulhu-rider pushed a commit to cthulhu-rider/neofs-node that referenced this pull request Sep 13, 2022
…hod's docs

Method never returns `PersistentStatePathDefault` value, and this is
correct.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
cthulhu-rider pushed a commit to cthulhu-rider/neofs-node that referenced this pull request Sep 13, 2022
…efault

In current implementation storage node app uses in-memory session
storage if `persistent_sessions.path` value is empty/missing in the
config.

Clarify default behavior in `config/example/node.yaml`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
Move in-memory session storage to the separate directory of `storage`. It is
done for future support of different kind of session storages.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
Add `WithEncryption` option that passes ECDSA key to the persistent session
storage. It uses 32 bytes from marshalled ECDSA key in ASN.1 DER from in
AES-256 algorithm encryption in Galois/Counter Mode.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
Use persistent storage usage in the node if it was configured so.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Improving existing functionality neofs-storage Storage node application issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support persistent sessions

3 participants