Nucypher-Core part 2 (more ABI changes) by fjarri · Pull Request #2809 · nucypher/nucypher

fjarri · 2021-10-14T03:50:01Z

Breaking changes:

Instead of just signing the interface in NodeMetadata, sign the whole metadata. As a part of that, interface_signature and timestamp attributes were removed. timestamp is now a metadata attribute, since it's recorded when the metadata is created.
hrac, publisher_verifying_key, and _public_signature are no longer bundled with EncryptedTreasureMap.
publisher_verifying_key is added to ReencryptionRequest.
MessageKit does not sign anything now, it's the user's responsibility.
Added EncryptedKeyFrag, a separate versioned entity hosting an encrypted AuthorizedKeyFrag.
Added AuthorizedTreasureMap, a separate versioned entity hosting an "unverifed" treasure map (similarly to how the pair AuthorizedKeyFrag-VerifiedKeyFrag works.
Using the whole keyfrag to create a signature in AuthorizedKeyFrag, not just its checksum.
The brand of AuthorizedKeyFrag changed from AKF_ to AKFr
Normalized the usage of VariableLengthBytestring. Now every class knows how to deserialize itself from the stream, instead of relying on the higher level class to provide it with a bytestring of correct length.
Denoting optional fields with a flag instead of a VariableLengthBytestring.

In particular, fixes #2743:

MessageKit is just plain capsule + ciphertext
AuthorizedKeyFrag (which is a plaintext for EncryptedKeyFrag) has its main payload (kfrag) + HRAC signed by the publisher
AuthorizedTreasureMap (which is a plaintext for EncryptedTreasureMap) has its main payload (treasure map) + recipient key signed by the publisher
Removed alice_verifying_key from ReencryptionRequest (as a part of What is the purpose of sending verifying keys in ReencryptionRequest? #2812)

Other changes:

Removed some unused imports
Removed passing Alice's signing power to Enrico in Enrico.from_alice(), since Enrico doesn't sign anything anymore.
Fixed an argument name in Ursula._decrypt_kfrag() - the verifying key passed to it is supposed to be Publisher's
UnbondedWorker is not mysteriously caught and ignored in verify_node(), but instead propagated to be caught in learn_from_teacher_node() like the rest of exceptions.
Fixed treasure map caching in Bob
Moved ReencryptionResponse verification from RetrievalClient._request_reencryption() to ReencryptionResponse.verify()
Normalized parameter names and order in protocol object constructors.
Using http.HTTPStatus enum instead of hardcoded status codes.

For reviewers:

Need to decide on What is the purpose of sending verifying keys in ReencryptionRequest? #2812, as a result publisher_verifying_key may be removed from TreasureMap, or a cross-check added to see if it matches the HRAC
See the TODO in AuthorizedKeyFrag.verify() - should we have a forced verification method for KeyFrag in Umbral?
At the moment HRAC is not saved in AuthorizedKeyFrag, and the recipient key is not saved in AuthorizedTreasureMap. The externally passed value is used for verification. Another option is to store them in the respective objects (still passing an external value, of course) - this creates some redundancy, but may simplify error checking. A parallel can be drawn also with how capsules are signed in ReencryptionResponse, but not included in it. Thoughts?
Make a trivial versioned plaintext class for MessageKit, to enable possible updates?
Rename MessageKit to EncryptedMessage for the uniformity with kfrags and treasure maps?

vepkenez

I can't speak to some of the implications but I love the continuing advancements in simplicity.
🥇

nucypher/core.py

nucypher/utilities/porter/control/specifications/porter_schema.py

tests/unit/test_porter.py

nucypher/utilities/porter/control/specifications/porter_schema.py

Instead: - Add it as a parameter to `retrieve_cfrags()` - Add it to `ReencryptionRequest` - Remove `EncryptedTreasureMap._public_signature` and `hrac`

fjarri · 2021-10-30T19:38:44Z

The last commit attempts to normalize the VariableLengthBytestring usage in protocol objects (this was the remaining TODO item). The problem was that variable size objects and lists were treated differently in different classes. Some classes were relying on the higher-level class to supply a bytestring of correct length to from_bytes(), and just assumed that all that remained is the variable length part. Sometimes a class actually wrapped the variable-length part in VariableLengthBytestring. Sometimes it was both.

So in that commit the guidelines were:

Every class knows how to deserialize itself from the given bytestring, returning the unused remainder
Versioned was changed to support that, adding take() and requiring deserialization handlers return the object and the remainder of the bytestring
Whenever a list occurs in a class, it is wrapped in VariableLengthBytestring. This is done for both lists of types with constant sizes and lists of types with variable sizes, for uniformity (an alternative for constant sized types would be specifying the number of items). Since all objects know how to deserialize themselves from the stream, it is not necessary to wrap every element of the list in VariableLengthBytestring - we just take() until we're out of bytes.

The disadvantage of the take() approach is that you can't just skip a variable-sized field and deserialize it later (lazily) - you have to deserialize everything. I think that for protocol objects it is not a problem.

One small change to be added is a simpler support for optional fields - we just need a byte flag instead of wrapping them in a VariableLengthBytestring.

nucypher/network/server.py

…geKit

…reasureMap similarly to KeyFrag

Now it's the user's responsibility

…crypting_key Also change recipient_key to policy_encrypting_key in MessageKit.author(), now that MessageKit is only used for reencryptable messages.

…core constructors

Enrico doesn't sign anything anymore.

Allow for every Versioned type to be able to deserialize itself from the bytestring.

…string

KPrasch

Already left all the comments and questions I have here. LGTM. I don't know if it really counts as a "part 3" but let's move onto the next PR for any follow-ups. As discussed the tasks mentioned in the "for reviewers" section need to be resolved before the next major release.

fjarri · 2021-11-02T20:55:30Z

The TODOs were filed as issues:

See the TODO in AuthorizedKeyFrag.verify() - should we have a forced verification method for KeyFrag in Umbral?

Filed nucypher/rust-umbral#73

At the moment HRAC is not saved in AuthorizedKeyFrag, and the recipient key is not saved in AuthorizedTreasureMap. The externally passed value is used for verification. Another option is to store them in the respective objects (still passing an external value, of course) - this creates some redundancy, but may simplify error checking. A parallel can be drawn also with how capsules are signed in ReencryptionResponse, but not included in it. Thoughts?

Filed #2816

Make a trivial versioned plaintext class for MessageKit, to enable possible updates?

Filed #2817

Rename MessageKit to EncryptedMessage for the uniformity with kfrags and treasure maps?

Filed #2818

fjarri added a commit to fjarri/nucypher that referenced this pull request Oct 14, 2021

Add a newsfragment for PR nucypher#2809

8f80b06

fjarri force-pushed the core-part2 branch 2 times, most recently from 3e96c10 to 8e601b4 Compare October 14, 2021 05:18

fjarri mentioned this pull request Oct 15, 2021

Extract protocol classes into nucypher-core #2802

Merged

fjarri added a commit to fjarri/nucypher that referenced this pull request Oct 16, 2021

Add a newsfragment for PR nucypher#2809

cead60c

fjarri force-pushed the core-part2 branch 12 times, most recently from 950baf4 to 1b465ea Compare October 20, 2021 23:45

fjarri requested review from KPrasch, cygnusv, derekpierre, jMyles, piotr-roslaniec and vepkenez October 20, 2021 23:51

fjarri force-pushed the core-part2 branch from 4b65fe1 to f575aad Compare October 21, 2021 04:47

fjarri added a commit to fjarri/nucypher that referenced this pull request Oct 21, 2021

Add a newsfragment for PR nucypher#2809

4b60c8a

fjarri force-pushed the core-part2 branch 2 times, most recently from ce6cf4d to 3e1eafc Compare October 21, 2021 05:03

vepkenez approved these changes Oct 21, 2021

View reviewed changes

derekpierre reviewed Oct 21, 2021

View reviewed changes

piotr-roslaniec mentioned this pull request Oct 22, 2021

Update protocol in accordance with nucypher-core nucypher/taco-web#15

Closed

fjarri added 3 commits October 29, 2021 15:06

Stop bundling publisher_verifying_key with EncryptedTreasureMap

4bb6c82

Instead: - Add it as a parameter to `retrieve_cfrags()` - Add it to `ReencryptionRequest` - Remove `EncryptedTreasureMap._public_signature` and `hrac`

Remove hrac from EncryptedTreasureMap

238a2cf

Remove unused properties of PolicyMessageKit

60dc147

fjarri force-pushed the core-part2 branch 2 times, most recently from 6687ef3 to 848d7e3 Compare October 30, 2021 19:25

fjarri marked this pull request as ready for review October 30, 2021 23:03

derekpierre reviewed Nov 1, 2021

View reviewed changes

nucypher/network/server.py Outdated Show resolved Hide resolved

fjarri added 12 commits November 1, 2021 13:31

Make a separately versioned EncryptedKeyFrag instead of using a Messa…

af40467

…geKit

Don't use Message kit in EncryptedTreasureMap and extract AuthorizedT…

21cf2ba

…reasureMap similarly to KeyFrag

Remove signing from MessageKit

004aa31

Now it's the user's responsibility

Move ReencryptionResponse verification to core.py

c0cf12e

Remove alice_verifying_key from ReencryptionRequest

70b2a1d

Making names uniform throughout the codebase: policy_key -> policy_en…

fe02c30

…crypting_key Also change recipient_key to policy_encrypting_key in MessageKit.author(), now that MessageKit is only used for reencryptable messages.

Move signer to the first position in the parameter list in various …

7197d85

…core constructors

Use constants from http module for HTTP status codes

06e17e8

Remove Alice's SigningPower from Enrico in from_alice()

3c347c8

Enrico doesn't sign anything anymore.

Fix an undefined variable usage

1114891

Normalize the usage of VariableLengthBytestrings

944d337

Allow for every Versioned type to be able to deserialize itself from the bytestring.

Use a simple flag to denote optionality instead of VariableLengthByte…

c8a1d40

…string

fjarri force-pushed the core-part2 branch from 345c8cd to c8a1d40 Compare November 1, 2021 22:55

KPrasch approved these changes Nov 2, 2021

View reviewed changes

KPrasch merged commit c0fc5eb into nucypher:main Nov 2, 2021

fjarri deleted the core-part2 branch November 2, 2021 20:55

fjarri mentioned this pull request Nov 2, 2021

Formalize node protocol #2418

Closed

KPrasch mentioned this pull request Jan 22, 2022

Use nucypher/rust-python #2849

Merged

This was referenced Mar 25, 2022

Better Arrangement ID. #180

Closed

Ursula's HTTPResponse for consider_arrangement #234

Closed

Find a way to serialize a TreasureMap without encrypting it for Bob #1326

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Nucypher-Core part 2 (more ABI changes)#2809

Nucypher-Core part 2 (more ABI changes)#2809
KPrasch merged 21 commits intonucypher:mainfrom
fjarri:core-part2

fjarri commented Oct 14, 2021 •

edited

Loading

Uh oh!

vepkenez left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fjarri commented Oct 30, 2021

Uh oh!

Uh oh!

KPrasch left a comment •

edited

Loading

Uh oh!

fjarri commented Nov 2, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

fjarri commented Oct 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vepkenez left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fjarri commented Oct 30, 2021

Uh oh!

Uh oh!

KPrasch left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fjarri commented Nov 2, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

fjarri commented Oct 14, 2021 •

edited

Loading

KPrasch left a comment •

edited

Loading