Motivation

The Joystream platform will have a multitude of working groups, already now we know we need groups for the

• storage and distribution
• content
• forum
• memberships
• builders
• query nodes
• growth & marketing & support

Recently the first version of the content working group was completed

https://github.com/Joystream/substrate-runtime-joystream/tree/13dae643ccffd3d6216705c7c139954f67cd887d/src/content_working_group

However, this implementation took a great deal of time, not least because of the formidable size and complexity involved, where the core - yet still incomplete - module took more than 2.6K lines of code, and the extremely light test coverage took more than 2K lines of code. This is too heavy a burden for each module.

Content Working Group Analysis

If we look at all the public calls, both dispatchable and not, they can be categorised into four groups as follows.

Out of place domain business logic

Here we had

• create_channel
• transfer_channel_ownership
• update_channel_as_owner
• update_channel_as_curation_actor

These are basically intimately tied to the business logic of the domain, the content directory, and not with the working group itself. The only reason they were located here had to do with artifactual limitations of the current state of the permissions and versioned object store. No working group should have these sorts of calls, they should be in a domain module.

Hiring management

Here we had

• add_curator_opening
• accept_curator_applications
• begin_curator_applicant_review
• fill_curator_opening
• withdraw_curator_application
• terminate_curator_application
• apply_on_curator_opening

These are a very thin layer over the hiring module and membership module which had to do with hiring curators. An additional important job was also to bind an opening policy (OpeningPolicyCommitment) to the opening when it was created, in order to lock in a set of free parameters in the hiring flow that the hiring module itself did not lock in during the lifetime of a hiring. These had to be implemented in some way, as the hiring module itself does not have any extrinsics or events, as its a utility module. However, what is quite clear, is that almost nothing in these is very specific to the content directory working group. The same logic would apply to any working group.

Role lifecycle management

Here we had

• update_curator_role_account
• update_curator_reward_account
• leave_curator_role
• terminate_curator_role
• set_lead
• unset_lead
• unstaking

These are practical administration involved in the lifecycle of a curator, and many calls are missing still, around updating the actual reward, or intitating slashing, or even contesting a slashing, etc. Again, these are needed in any working group.

Permissions module service interface

Here we had

• account_can_act_as_principal

This is a lookup method servicing the permissions module, and is in fact the only thing so far which is unlikely tied to the working group, as it attempts to address the question of what group member can do what at a given.

Conclusion

Form the run through above its very clear that there is a wide scope of functionality which will need to be reproduced across many working groups, that is not already present in any other reusable form. The question is then what the right abstraction is to make it much easier to write new working groups.

Proposal

This proposal attempts to give a high level specification for how to reduce the cost and complexity involved in developing new working groups based on the analysis above.

Modules

For each subsystem, the following set of modules are involved

• Domain module: A module that holds the core functionality of the subsystem, and has a built in thin view of the different roles, solely for the purpose of authenticating extrinsics from the given agent. The role sets, and associated authentication accounts and status (enabled/disabled) are updated via non-dispatchable module calls which abstract the rest of our runtime governance. Should ideally be a highly reusable in other runtimes.
• Bureaucracy module (NEW): A new module which tries to hold a projection of the purely bureaucratic dimensions of any working group, hence it hires the basic functionality tied to staking, slashing, rewards, hiring, terminating, leaving, membership binding and so on.
• Working group module: A module that handles role actions specific to the core functionality of the subsystem.
• Infrastructure modules: Background modules that provide general services, such as staking, hiring, recurring rewards, membership, etc.

Bureaucracy module

This is the only entirely new module in this proposal. It provides a general implementation of many of the generic role related activities seen in the analysis above.

Instance module

One important property of this module is that it is an instance module, meaning that we have multiple instances with separate storage, as described here

https://substrate.dev/recipes/storage/instantiable.html#instantiable-pallets

In our case, we will have one instance per working group. This has multiple benefits:

• Single event handler: The module needs to notify other modules, most importantyl the working group, of certain important events, such as a new actor entering the group through hiring. This can now easily be done via runtime trait where only a single handler needs to know, unlike for example in the shared staking module, each event handler had to check if the event was truly for them.
• Safety: The state of different working groups is separated, meaning there is no room for mixups, where the resources of one group inadvertently is corrupted by another.
• Light weight migration: Since the storage state is per group, it will be much smaller, meaning that migrations can be separated, hence it will be less trouble with the size of the storage, or even having to upgrade all working groups at once.

One disadvantage would be if we wanted to have the ability to deploy new working groups dynamically, without runtime upgrades, then this would be a limitation. It is however unclear if we ever want this, and if we do, we may end up putting this all on a contract abstraction anyway, where a new bureaucracy module could be deployed each time.

Improvements

Other improvements will also be made over the analogous implementation in the content working group.

• Arbitrary org-chart: We will allow an arbitrary structured actor hierarchy, rather than the forced single level in the content working group. Each actor is under the control of the actor that hired them, or optionally an actor can be transferred to be below another later. This control involves setting reward rates, slashing, terminating, etc. It also involves setting permissions, such as whether this actor can hire etc. Lastly it also involves creating a token mint for a child, and transferring minting capacity from your own budget. Control is transitive, so not only the direct parent, but any parent can exercise this control. If an actor with children removed, then the children become parents of the parent of this actor. Actors without a parent are only controlled by non-dispatchables, which can be called by the council. They are in-effect sort of leads.
• Lead hireable: Leads can be hired through normal hiring flow, initiated by root origin.
• Complete life cycle management: Currently, lots of basic management is not present, such as slashing, changing rewards, etc. This will be introduced.
• Remove registration requirement with membership module: Needless complexity caused by role entry gating in Simplify membership module  #115
• Move binding to membership out of that module and into this one: Move membership to role binding out in Simplify membership module  #115

It is probably wise to start with a normal 1-level org structure to begin with, and then when this entire proposal has been sorted out, the more complex org-chart can be attempted.

Case Study: The forum

Here we consider an example case, the forum subsystem, of how the different modules can worth together in a hypothetical runtime following this proposals. This diagram is a good starting point.

Based on this, we can try to walk through the call-dynamics of a few different cases.

Lead joins working group

1. The council passes a proposal that creates an opening in the relevant bureaucracy module instance, by invoking the right calls on Bureuacracy<T> trait.
2. A set of applicants send extrinsics to buraucracy module instance. Each time, the Trait::membership trait is used to authenticate and bind member to application.
3. The council passes another proposal to begin review via Bureuacracy<T>.
4. The council passes another proposal to fill position via Bureuacracy<T>, and this filling is signalled through Trait::eventSink.
5. The forum_working_group handles the signal via its implementation of BureaycracyEventSink, by adding the new actors to its local representation of the working group. This means the call must include sufficient information about where this actor is located in the bureaucracy for the module to understand whether it is the lead or just a normal moderator.
   If it is the lead, the one should make appropriate updates to form module, so it knows that there is a new moderator at the root level category.

User joins system

1. A new account establishes a new membership by sending an extrinsic to buy_membership in membership module.
2. The module signals this event through Trait::eventSink.
3. The forum_working_group will receive it through MembershipEventSink implementation, and it will create new use profile in the forum by calling via Trait::forum, and also remember the binding between the forum user and the membership.