Refactor Specifications/Serializers by vepkenez · Pull Request #1555 · nucypher/nucypher

vepkenez · 2020-01-05T20:06:11Z

refactors code around input and output specifications and serializers
removes 100+ lines of controllers/serializers code
adds new declarative format for io specification using py-marshmallow
adds improved validation for different datatypes
additional extensibility to validate keys, treasure maps, message kits etc.
allows for specifications to indicate type which enables better connect native integration as well as improved compatibility with REST/OPTIONS interfaces
required by [WIP] NuCypher ConnectNative Firefox Extension #1299

derekpierre

@vepkenez - marshmellow definitely provides some specification clarity.

Some questions/comments for you.

derekpierre · 2020-01-06T20:24:21Z

nucypher/characters/control/interfaces.py

              n: int,
              expiration: maya.MayaDT,
              value: int = None,
+              first_period_reward: int = None,


I believe @vzotova got rid of this parameter in #1459 - Perhaps it is best to rebase your PR on that branch?

Yes, I think this parameter got removed

^ @vepkenez (https://github.com/nucypher/nucypher/pull/1555/files#r363467150)

#1459 is already merged into matster, so if this PR is currently based over the latest, and this variable is unused within this function it can be safely removed.

nucypher/characters/control/interfaces.py

nucypher/characters/control/specifications/__init__.py

nucypher/characters/control/specifications/alice.py

nucypher/characters/control/specifications/fields/messagekit.py

nucypher/characters/control/specifications/fields/treasuremap.py

nucypher/characters/control/specifications/enrico.py

nucypher/characters/control/specifications/bob.py

nucypher/characters/control/specifications/fields/messagekit.py

mswilkison · 2020-01-09T00:13:35Z

nucypher/characters/control/controllers.py

+    def get_serializer(self, interface_name: str):
+        return self.specification.get_serializer(inteface_name)
+
+    def _perform_action(self, action: str, request: dict) -> dict:


nucypher/characters/control/controllers.py

nucypher/characters/control/specifications/exceptions.py

nucypher/characters/control/specifications/fields/label.py

nucypher/characters/control/specifications/fields/messagekit.py

tests/characters/test_specifications.py

nucypher/characters/control/controllers.py

nucypher/characters/control/specifications/fields/datetime.py

nucypher/characters/control/specifications/bob.py

derekpierre

Unsure how I feel about the CLI integration - a bit of indirection - need to think a bit more about it.

Seems like we need some more tests...?

nucypher/characters/control/controllers.py

derekpierre · 2020-01-14T20:07:47Z

nucypher/characters/control/specifications/fields/treasuremap.py

+    def _deserialize(self, value, attr, data, **kwargs):
+        return b64decode(value)
+
+    def _validate(self, value):


Can validate here simply be (?)

try: treasure_map = TreasureMap.from_bytes(value): return True except: return False

I'm unsure whether the verify kwarg for from_bytes can remain as True (default) or needs to be set to False.

Yes, I'd love it if that worked but unfortunately importing nucypher.policy.collections.Treasuremap causes a circular import since Bob is imported into that same file... which I think should maybe be a TODO but maybe there is some reason I'm not aware of.

vepkenez · 2020-01-14T20:49:56Z

Unsure how I feel about the CLI integration - a bit of indirection - need to think a bit more about it.

I agree @derekpierre. It's not ideal. It was better before the cleanup-options PR. I think in the long run, the path is going to be to remove some of the complexity and have all of the CLI options be specified through similar schemas so it will actually be very simple.

so instead of

alice_verifying_key = fields.Key(
         load_only=True, required=True,
         click=click.option(
             '--alice-verifying-key',
             help="Alice's verifying key as a hexadecimal string",
             required=True, type=click.STRING,))

we'll have

alice_verifying_key = fields.Key(load_only=True, required=True, help="Alice's verifying key as a hexadecimal string")

And then the click stuff would be generated from that.
Because of the recent changes that happened concurrently to the work in this PR, I just folded it in in a way that would provide exactly matching results.

@KPrasch and I are planning to expand this concept to move most of the ad-hoc code that lives in the cli files into interfaces that can then be accessed through any entry point... but later.

… on a WIP variant of #1555. For that history, see: d8aebfd

…merged.

handle TypeError in invalid RPC input

fix double encode

address RFCs

…ilaizers with schema modeling (Needs adiitional follow up).

…ication API (successful policy creation by RPC controller)

fix rpc_control_blockchain test

remove specifications references remove bad idea

rename 'connect' to 'connect_cli'

rebuilt reqs

fix scoping issue

Co-Authored-By: Derek Pierre <derek.pierre@gmail.com> Co-Authored-By: Derek Pierre <derek.pierre@gmail.com>

…e cleanup.

derekpierre

🎸

derekpierre · 2020-02-06T14:12:17Z

I agree with @cygnusv that some commits can probably be squashed for a cleaner history.

vepkenez added the API label Jan 5, 2020

vepkenez requested review from KPrasch, cygnusv, derekpierre, fjarri, jMyles, michwill and mswilkison January 5, 2020 20:10

vepkenez added Bob 👨‍💼 Effects the "Bob" development area Alice 👩 Effects the "Alice" development area labels Jan 5, 2020

vepkenez changed the title ~~Refactor Specifications~~ [WIP] Refactor Specifications Jan 5, 2020

vepkenez mentioned this pull request Jan 6, 2020

[WIP] NuCypher ConnectNative Firefox Extension #1299

Closed

derekpierre reviewed Jan 6, 2020

View reviewed changes

vepkenez force-pushed the refactor-specifications branch from 7a84aa8 to 8c0600e Compare January 7, 2020 20:01

vepkenez changed the title ~~[WIP] Refactor Specifications~~ Refactor Specifications/Serializers Jan 8, 2020

mswilkison reviewed Jan 9, 2020

View reviewed changes

derekpierre reviewed Jan 9, 2020

View reviewed changes

nucypher/characters/control/specifications/bob.py Show resolved Hide resolved

vepkenez force-pushed the refactor-specifications branch 4 times, most recently from 761105a to e540f89 Compare January 12, 2020 20:26

derekpierre reviewed Jan 14, 2020

View reviewed changes

vepkenez changed the title ~~Refactor Specifications/Serializers~~ [WIP] Refactor Specifications/Serializers Jan 17, 2020

jMyles mentioned this pull request Jan 28, 2020

[EPIC] Birdcage: A PR for people who want to pretend to like Bob. #1330

Merged

jMyles added a commit that referenced this pull request Jan 28, 2020

Beginning of alternate timeline for birdcage. It was previously based…

7eeb533

… on a WIP variant of #1555. For that history, see: d8aebfd

jMyles added a commit that referenced this pull request Jan 28, 2020

Taking interfaces back to master until #1555 or something like it is …

723d349

…merged.

jMyles added a commit that referenced this pull request Jan 28, 2020

Taking interfaces back to master until #1555 or something like it is …

b6e6fce

…merged.

KPrasch force-pushed the refactor-specifications branch from 6d59811 to e449341 Compare January 31, 2020 20:06

vepkenez and others added 23 commits February 6, 2020 03:20

fix failing test on invalid rpc input

c9f0d6c

handle TypeError in invalid RPC input

adds cleartext field

f570146

fix messagekit serializer

a439ad8

implement cleartext field

77d2a93

fix double encode

deserialize input to encrypt

4042f45

address RFCs

Aiming to unify command specifications, controllers, validations, ser…

d34e6df

…ilaizers with schema modeling (Needs adiitional follow up).

Deprecates character control decorator in favor of new command specif…

763e295

…ication API (successful policy creation by RPC controller)

click integration

38f2ef0

Passing RPCController tests

71a7fc7

Implements new input scheme API on Webcontrollers.

e70ba7e

prototype depcrecation of options flags on cli

d347428

move nucypher command decorator to CharacterPublicInterface

44ce4b4

fix rpc_control_blockchain test

incorporate changes from option-cleanup

08dab01

remove specifications mappings

75a84ff

remove specifications references remove bad idea

finish implementing bob, alice, enrico

f381d92

rename 'connect' to 'connect_cli'

suggestions from derekpierre and mswilkison

ef1f374

rebuilt reqs

define controller's place in the world

a3b5a02

only some characters have an interface

2ead183

Uses named web controller routing over interface injection.

785f988

fix scoping issue

better exception handling

abb2cb5

handle kwargs based requests

fcb9763

Co-Authored-By: Derek Pierre <derek.pierre@gmail.com> Co-Authored-By: Derek Pierre <derek.pierre@gmail.com>

Accomodate the usage of --rate for granting via LCI specs; Post rebas…

a3ee748

…e cleanup.

field validation, tests, squashed

d656142

vepkenez force-pushed the refactor-specifications branch from 3c5d1a8 to d656142 Compare February 6, 2020 11:25

derekpierre approved these changes Feb 6, 2020

View reviewed changes

address @cygnusv suggestions

0a0761e

KPrasch force-pushed the refactor-specifications branch from 7531d5e to 0a0761e Compare February 6, 2020 22:24

KPrasch merged commit 481af33 into nucypher:master Feb 6, 2020

jMyles mentioned this pull request May 24, 2021

Super MarioBob 2 #2711

Closed

Conversation

vepkenez commented Jan 5, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

derekpierre left a comment

Choose a reason for hiding this comment

Uh oh!

derekpierre Jan 6, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

michwill Jan 9, 2020

Choose a reason for hiding this comment

Uh oh!

derekpierre Feb 3, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KPrasch Feb 5, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mswilkison Jan 9, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

derekpierre left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

derekpierre Jan 14, 2020

Choose a reason for hiding this comment

Uh oh!

vepkenez Jan 14, 2020

Choose a reason for hiding this comment

Uh oh!

vepkenez commented Jan 14, 2020

Uh oh!

derekpierre left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

derekpierre commented Feb 6, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

vepkenez commented Jan 5, 2020 •

edited

Loading

derekpierre Jan 6, 2020 •

edited

Loading

derekpierre Feb 3, 2020 •

edited

Loading

derekpierre left a comment •

edited

Loading