Superceeded by new plan: #4593
Name
Ephesus
Brand
Goals
1. Creator Payouts: Enable council to pay creators at scale.
2. YPP: Enable Atlas apps to use Youtube-Synch infrastructure.
3. Liberated: Transition to Liberated stage by dropping Sudo.
4. Blacklisting: Self-imposed governance blacklisting.
5. Runtime bugfix: membership.update_accounts weight calculation.
6. Upgrade: Execute a safe, well tested and legitimate runtime upgrade.
Scope
This release bundles together goals that
- require both on- and off-chain changes
- are relatively unrelated
The former may possibly be suboptimal, as it requires the council and community to understand changes which do not really require any collective agreement or legitimacy. The latter is required for efficiency, and that is likely to continue to be the case in the future, as there are non-trivial fixed costs, in time, testing and communication, for doing any runtime change at all, hence batching a few different changes becomes economical.
The work has been ongoing for a while, hence a lot of what is included in this plan is just the tail end of finalisation, quality-assurance, deployment and delivery estimation, and also for communicating clearly with the rest of the community.
Overview
This is the first post-mainnet network, and it tries to bring online two major areas of work (1,2), shed a final constraint of the launch process (3) and also deliver a low hanging governance win (4). It involves some new challenges not present in prior testnets, specifically a
- much higher bar needed for testing of new runtime features, as faults or failures are much more costly.
- need to interact with the live governance process of the DAO itself, and making sure benefits and risks are understood, and outcome is legitimate.
In what follows, a brief summary of the motivation and scope of work in each goal is provided.
1. Creator Payouts: Allow council to pay creators at scale.
Creators provide one of the most valuable resources to the platform, their creative energy, audience and content. Access to these are scarce, and thus the DAO must be able to compensate them, just like any other role in the system, such as council members, workers or validators. Unlike any of these other roles, there can be a very large number of creators that need to be paid in production and at scale, and this comes into conflict with the limited processing resources of the blockchain.
For this reason, a scaleable pull based payout system has been introduced in the runtime, where the council is able to issue at scale payment to an unlimited number of recipients, funded out of the council budget - via the content working group budget. Initial designs can be found described here #2365. Briefly stated, the council uploads a Merkle commitment over a larger dataset which explains who has earned what money for what reason. The dataset can be generated, and verified, using new tools in the CLI. The dataset itself will live in the council bag stored by the storage system, and council members will need to upload this dataset using the CLI. In the future Pioneer perhaps can do this as part of proposal creation flow. How exactly the council should determine what to pay for, at what time, as well as defining abuse and sanctions, is something that is left unspecified. We have simply assumed that the council will somehow make this determination, and use the tools and features to put it into action.
Importantly, the creator payments to be launched as part of YPP in Gleev does not plan to use council financing in any way, hence the council must determine how it will use this capability for it's own purposes going forward. Importantly the nuts and bolts of operating this functionality is not that easy, hence documentation will be provided in the form of a tutorial (#4536). Creators can, through applications like Atlas, use proofs about what they have earned, produced from this dataset, to cash out money they are owed into their channel account. There has been complementary work to add support for executing and viewing the history of such creator payouts in Atlas as well.
2. YPP: Enable Atlas apps to use Youtube-Synch infrastructure.
Most existing creators will already have a channel on YouTube, and this channel may have a considerable back-catalogue. Moreover, active creators will continue to publish to their YouTube channel for the foreseeable future, even if they are willing to experiment with Joystream. Youtube-Synch is a new backend service that application operators can use to authentication YouTube creators and replicate their presence there onto Joystream automatically, both upon signup and on an ongoing basis. This will have the effect of reducing the cost of trying and participating on Joystream for these creators. This service could in principle be used with any application, but Atlas has also been updated to have a native integration which makes it particularly easy for Atlas application operators to onboard YouTube creators, for example by doing the authentication through a nice signup flow, and conveying rewards associated with participation. Participation terms will be up to each application to determine, but Gleev (the Jsgenesis operator application) will be following this scheme Joystream/gleev#7. Importantly, each application instance will operate and it's own infrastructure, which means a dedicated server must continuously run the scanning and uploading process - as well as pay the fees, and hold the authentication token information for YouTube for each user. The infrastructure is able to publish videos under the channel of a given creator, because the creator has to make the application a so-called collaborator, which is an on-chain role in channels that allow third parties to work to contribute to channel on behalf of the owner.
The current plan is to not deploy the application attribution system, defined here #4307 and here #4158, which means that the council will not be able to determine through which application instance a given video was published or youtube-synched, this is unfortunate, as it is harder for the council to support apps that do a good job with onboarding.
Applications can directly subsidise creators into joining their YPP programs by just paying them directly, and a payments feature has been added to member remarks extrinsic to enable this (#4507). Operators will be provided an updated guide on how one runs the new infrastructure Joystream/atlas#3628, and also tooling to administer the rewards to be paid in accordance with their policy (Joystream/youtube-synch#126).
3. Liberated: Transition to Liberated stage by dropping Sudo.
The rollout of the mainnet was staged, as explained here. We are currently in the Supervised stage, hence all of the functionality is online, however the Sudo actions are still available, despite not having been used. This last remnant of the launch will be removed as part of this network. Apart from some cleanup of integration tests which depended on this pallet (#4401), this will not have any implications.
4. Blacklisting: Self-imposed governance blacklisting
There are many early backers, FMs and Jsgenesis affiliates who are subject to vesting, and they may want to be able to provide a credible signal that they will not use their stake to influence governance, but while retaining their $JOY stake which is locked (#2927). For this reason, a new runtime level feature has been added which allows an account to self-impose an irreversible ban on their own account, which when combined with vesting, provides a strong assurance of non-interference. There will be a very minor tweak to Pioneer in order to prevent such accounts from attempting to vote, but other than that this has no direct implications
5. Runtime bugfix: membership.update_accounts weight calculation
Described here #4535.
6. Upgrade: Execute a safe, well tested and legitimate runtime upgrade.
There has not been a community driven runtime upgrade so far on the Joystream mainnet, but in the future there are likely to be many attempts. Upgrades are both valuable and risky, where risks primarily come from the possibility of altering the runtime in some way which is
- harmful to the objectives of the community, either by intention or malintent.
- deemed as illegitimate by some material contingent, in terms of the process.
The only way to manage these risks is to have well documented and clear rules, generally accepted as legitimate, for the process of how runtime upgrades are conducted. At this time no such rules exist, and unfortunately since there is no constitution, there is no place for people to easily identify, review or suggest changes to any candidate rules.
As a result, some initial rules will be proposed as the as the best way to get this runtime upgrade into production (#4541), and also some tooling and documentation will have to be developed to allow everyone to verify a proposed upgrade (#4554).
Quality Assurance
Versioning
| Software |
Maintainer |
Current Version |
Ephesus Version |
| Runtime Spec Version |
Ignazio |
1001 |
2000 |
| Node |
Mokhtar |
? |
8.2.0 |
| Atlas |
Artem |
1.2.2 |
1.3.0 |
| Gleev |
Artem |
1.0.0 |
1.1.0 |
| Pioneer |
Theo |
TBD. |
TBD. |
| QN |
Zeeshan |
0.1.0 |
1.0.0 |
| Argus |
Leszek/Zeeshan |
0.2.0 |
0.3.0 |
| Colossus |
Mokhtar |
3.0.0 |
3.1.0 |
| Hydra |
Zeeshan |
4.0.0-alpha.9 |
4.0.0-alpha.10 |
| CLI |
Zeeshan |
0.10.0 |
0.11.0 |
| YoutubeSync |
Zeeshan |
- |
1.0.0 |
| Orion |
Leszek |
1.0.0 |
2.0.0 |
ETA
Milestone for when upgrade can be proposed for council is: 15th February 2023
Blockchain
This will be a runtime upgrade, without any migration.
Asana Timeline
Monorepo
We are working on branch ephesus, which is based on rhodes branch, which itself is just olympia
https://github.com/Joystream/joystream/tree/ephesus
Release Manager
Bedeho
Regular Meetings
We start regular weekly release calls on Mondays 11am CET.
Before each meeting make sure that
- all work you are doing is reflected in issues or PRs with the
ephesus label
- if you have completed work, then make sure issue for that work is closed.
┆Issue is synchronized with this Asana task by Unito
Superceeded by new plan: #4593
Name
EphesusBrand
Goals
1.
Creator Payouts: Enable council to pay creators at scale.2.
YPP: Enable Atlas apps to use Youtube-Synch infrastructure.3.
Liberated: Transition to Liberated stage by dropping Sudo.4.
Blacklisting: Self-imposed governance blacklisting.5.
Runtime bugfix: membership.update_accounts weight calculation.6.
Upgrade: Execute a safe, well tested and legitimate runtime upgrade.Scope
This release bundles together goals that
The former may possibly be suboptimal, as it requires the council and community to understand changes which do not really require any collective agreement or legitimacy. The latter is required for efficiency, and that is likely to continue to be the case in the future, as there are non-trivial fixed costs, in time, testing and communication, for doing any runtime change at all, hence batching a few different changes becomes economical.
The work has been ongoing for a while, hence a lot of what is included in this plan is just the tail end of finalisation, quality-assurance, deployment and delivery estimation, and also for communicating clearly with the rest of the community.
Overview
This is the first post-mainnet network, and it tries to bring online two major areas of work (1,2), shed a final constraint of the launch process (3) and also deliver a low hanging governance win (4). It involves some new challenges not present in prior testnets, specifically a
In what follows, a brief summary of the motivation and scope of work in each goal is provided.
1.
Creator Payouts: Allow council to pay creators at scale.Creators provide one of the most valuable resources to the platform, their creative energy, audience and content. Access to these are scarce, and thus the DAO must be able to compensate them, just like any other role in the system, such as council members, workers or validators. Unlike any of these other roles, there can be a very large number of creators that need to be paid in production and at scale, and this comes into conflict with the limited processing resources of the blockchain.
For this reason, a scaleable pull based payout system has been introduced in the runtime, where the council is able to issue at scale payment to an unlimited number of recipients, funded out of the council budget - via the content working group budget. Initial designs can be found described here #2365. Briefly stated, the council uploads a Merkle commitment over a larger dataset which explains who has earned what money for what reason. The dataset can be generated, and verified, using new tools in the CLI. The dataset itself will live in the council bag stored by the storage system, and council members will need to upload this dataset using the CLI. In the future Pioneer perhaps can do this as part of proposal creation flow. How exactly the council should determine what to pay for, at what time, as well as defining abuse and sanctions, is something that is left unspecified. We have simply assumed that the council will somehow make this determination, and use the tools and features to put it into action.
Importantly, the creator payments to be launched as part of YPP in Gleev does not plan to use council financing in any way, hence the council must determine how it will use this capability for it's own purposes going forward. Importantly the nuts and bolts of operating this functionality is not that easy, hence documentation will be provided in the form of a tutorial (#4536). Creators can, through applications like Atlas, use proofs about what they have earned, produced from this dataset, to cash out money they are owed into their channel account. There has been complementary work to add support for executing and viewing the history of such creator payouts in Atlas as well.
2.
YPP: Enable Atlas apps to use Youtube-Synch infrastructure.Most existing creators will already have a channel on YouTube, and this channel may have a considerable back-catalogue. Moreover, active creators will continue to publish to their YouTube channel for the foreseeable future, even if they are willing to experiment with Joystream. Youtube-Synch is a new backend service that application operators can use to authentication YouTube creators and replicate their presence there onto Joystream automatically, both upon signup and on an ongoing basis. This will have the effect of reducing the cost of trying and participating on Joystream for these creators. This service could in principle be used with any application, but Atlas has also been updated to have a native integration which makes it particularly easy for Atlas application operators to onboard YouTube creators, for example by doing the authentication through a nice signup flow, and conveying rewards associated with participation. Participation terms will be up to each application to determine, but
Gleev(the Jsgenesis operator application) will be following this scheme Joystream/gleev#7. Importantly, each application instance will operate and it's own infrastructure, which means a dedicated server must continuously run the scanning and uploading process - as well as pay the fees, and hold the authentication token information for YouTube for each user. The infrastructure is able to publish videos under the channel of a given creator, because the creator has to make the application a so-called collaborator, which is an on-chain role in channels that allow third parties to work to contribute to channel on behalf of the owner.The current plan is to not deploy the application attribution system, defined here #4307 and here #4158, which means that the council will not be able to determine through which application instance a given video was published or youtube-synched, this is unfortunate, as it is harder for the council to support apps that do a good job with onboarding.
Applications can directly subsidise creators into joining their YPP programs by just paying them directly, and a payments feature has been added to member remarks extrinsic to enable this (#4507). Operators will be provided an updated guide on how one runs the new infrastructure Joystream/atlas#3628, and also tooling to administer the rewards to be paid in accordance with their policy (Joystream/youtube-synch#126).
3.
Liberated: Transition to Liberated stage by dropping Sudo.The rollout of the mainnet was staged, as explained here. We are currently in the
Supervisedstage, hence all of the functionality is online, however theSudoactions are still available, despite not having been used. This last remnant of the launch will be removed as part of this network. Apart from some cleanup of integration tests which depended on this pallet (#4401), this will not have any implications.4.
Blacklisting: Self-imposed governance blacklistingThere are many early backers, FMs and Jsgenesis affiliates who are subject to vesting, and they may want to be able to provide a credible signal that they will not use their stake to influence governance, but while retaining their $JOY stake which is locked (#2927). For this reason, a new runtime level feature has been added which allows an account to self-impose an irreversible ban on their own account, which when combined with vesting, provides a strong assurance of non-interference. There will be a very minor tweak to Pioneer in order to prevent such accounts from attempting to vote, but other than that this has no direct implications
5.
Runtime bugfix: membership.update_accounts weight calculationDescribed here #4535.
6.
Upgrade: Execute a safe, well tested and legitimate runtime upgrade.There has not been a community driven runtime upgrade so far on the Joystream mainnet, but in the future there are likely to be many attempts. Upgrades are both valuable and risky, where risks primarily come from the possibility of altering the runtime in some way which is
The only way to manage these risks is to have well documented and clear rules, generally accepted as legitimate, for the process of how runtime upgrades are conducted. At this time no such rules exist, and unfortunately since there is no constitution, there is no place for people to easily identify, review or suggest changes to any candidate rules.
As a result, some initial rules will be proposed as the as the best way to get this runtime upgrade into production (#4541), and also some tooling and documentation will have to be developed to allow everyone to verify a proposed upgrade (#4554).
Quality Assurance
EphesusQA plan pioneer#4098Versioning
EphesusVersion100120008.2.01.2.21.3.01.0.01.1.00.1.01.0.00.2.00.3.03.0.03.1.04.0.0-alpha.94.0.0-alpha.100.10.00.11.01.0.01.0.02.0.0ETA
Milestone for when upgrade can be proposed for council is: 15th February 2023
Blockchain
This will be a runtime upgrade, without any migration.
Asana Timeline
Monorepo
We are working on branch
ephesus, which is based onrhodesbranch, which itself is justolympiahttps://github.com/Joystream/joystream/tree/ephesus
Release Manager
Bedeho
Regular Meetings
We start regular weekly release calls on Mondays 11am CET.
Before each meeting make sure that
ephesuslabel┆Issue is synchronized with this Asana task by Unito