Skip to content

[EPIC] Porter MVP - "Infura for NuCypher"#2664

Merged
KPrasch merged 69 commits intomainfrom
porter
Aug 4, 2021
Merged

[EPIC] Porter MVP - "Infura for NuCypher"#2664
KPrasch merged 69 commits intomainfrom
porter

Conversation

@derekpierre
Copy link
Copy Markdown
Member

@derekpierre derekpierre commented Apr 26, 2021

porter_review_Logo2017_860x250-1-1

("Porter" is also a Canadian airline based out of Toronto - https://blog.flyporter.com/)

CHAMPION: @derekpierre

Goal

Simplifies and abstracts the complexities surrounding the nucypher protocol. It aims to negate the need for running the python client by applications to interact with the network - think web and mobile applications. It would leverage rust-umbral and associated JS bindings to achieve cross-platform functionality via the browser, without needing to rewrite the entire python client.

web app <--> "js-nucypher" <--> porter <--> network

Related to https://github.com/nucypher/productdev/issues/7.

MVP Milestone: https://github.com/nucypher/nucypher/milestone/18

High-level areas of focus for MVP:

PR Ramp Order

@derekpierre derekpierre added API Protocol Protocol design Web Webpages labels Apr 26, 2021
@derekpierre derekpierre changed the title [EPIC] Porter - "Infura for NuCypher" [EPIC] Porter MVP - "Infura for NuCypher" May 20, 2021
@derekpierre derekpierre closed this Jun 7, 2021
@derekpierre derekpierre reopened this Jun 7, 2021
@derekpierre derekpierre self-assigned this Jul 5, 2021
@derekpierre derekpierre added this to the Porter v1 (MVP) milestone Jul 5, 2021
@derekpierre derekpierre force-pushed the porter branch 2 times, most recently from daeb398 to 76e7b03 Compare July 12, 2021 23:20
derekpierre added a commit that referenced this pull request Jul 14, 2021
)

DEFAULT_PAGINATION_SIZE: int = 30 # TODO: Use dynamic pagination size (see #1424)
DEFAULT_STAKER_PAGINATION_SIZE_LIGHT_NODE: int = int(os.environ.get(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pattern worries me. First, the pagination size feels like a more high-level setting; whoever calls StakingEscrowAgent should worry about setting it. Second, the low-level class's behavior is influenced by an envvar, which may lead to some hard to diagnose problems. The only other place where we do that is ContractEventsThrottler.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The caller of StakingEscrowAgent can still proactively specify a pagination_size see get_all_active_stakers() which is the only place a pagination size is needed/used. If the user doesn't specify a pagination_size (i.e. None), only then is the default used.

We were already using a default value for light nodes in prior code because they act differently than regular nodes. However, the number was fixed and could not be modified meaning that if we were wrong on the value it couldn't be modified in the field without changing the code. To me, this was the best compromise between allowing the user to specify a pagination size, if not use a default, if the default failed then it could be modified proactively.

The goal here is not to have the env var ever set, but rather have it act as a fallback just in case, for some reason, our beliefs about what the pagination size should be are incorrect.

Regarding "hard to diagnose" problems, for extra help, we log when the default pagination size is used and its value. see L324.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't mind the default, or that there's an envvar for the default, it just seems that this specific spot is too deep in the call stack to depend on external environment. I admit I can't think of a good solution at the moment.

Regarding "hard to diagnose" problems, for extra help, we log when the default pagination size is used and its value.

It helps, but it's still a surprising side effect.

…eated and used for tests. exec_work_order tests now actually use /reencrypt ursulas endpoints.
…r /exec_work_order. Documentation cleanup: use version 6.0.0 in output, 401 error status code.
… about url encoding in docs.

Add unit test for Base64BytesRepresentation field.
Cleanup imports.
Allow query parameters to be passed in the URL
Porter implementation of `/exec_work_order` (associated with TMapConKFrags for `/reencrypt` endpoint)
Copy link
Copy Markdown
Contributor

@piotr-roslaniec piotr-roslaniec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed some of the PRs I previously missed out on. LGTM 🦝

Copy link
Copy Markdown
Member

@cygnusv cygnusv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked individual PRs, and ran porter via CLI and docker. This is an awesome work! 💪

@KPrasch
Copy link
Copy Markdown
Member

KPrasch commented Aug 4, 2021

@derekpierre
Copy link
Copy Markdown
Member Author

@derekpierre - What do you make of the CI failure? https://app.circleci.com/pipelines/github/nucypher/nucypher/8212/workflows/73bacae3-e320-4803-a63e-14ab170a8210/jobs/139336

@KPrasch, it was introduced by the latest rebase which excluded a needed code change - should be fixed now. Thanks for checking!

Copy link
Copy Markdown
Member

@KPrasch KPrasch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stellar effort here @derekpierre 👏🏻

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Protocol Protocol design Web Webpages

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants