Skip to content

Creator payouts implementation in Joystream.js package#3182

Closed
zeeshanakram3 wants to merge 35 commits intoJoystream:carthagefrom
zeeshanakram3:joystream.js
Closed

Creator payouts implementation in Joystream.js package#3182
zeeshanakram3 wants to merge 35 commits intoJoystream:carthagefrom
zeeshanakram3:joystream.js

Conversation

@zeeshanakram3
Copy link
Copy Markdown
Contributor

@zeeshanakram3 zeeshanakram3 commented Feb 7, 2022

The code for this implementation exists in the joystreamjs package at the root level of monorepo.
The implementation assumption is that the payout payload would be fetched from some remote over HTTP

The following CLI commands have been implemented for creator payout management

  • content:updateCreatorPayoutCommitment Update creator payout payload commitment hash in the runtime storage.
  • content:creatorPayoutRecord Show payout information for creator given a channel id.
  • content:claimCreatorPayoutReward Claim creator payout reward for a given channel id.
  • content:creatorPayoutPayloadHeader Get header from serialized payload file.
  • content:creatorPayoutRecordAtByteOffset Get creator payout record from serialized payload file at given byte.
  • content:generateCreatorPayoutPayload Create serialized creator payout payload from JSON input.
  • content:generateCreatorPayoutMerkleRoot Generate merkle root for creator payout payload passed as JSON input.
  • The PR contains the documentation of these commands

┆Issue is synchronized with this Asana task by Unito

@vercel
Copy link
Copy Markdown

vercel bot commented Feb 7, 2022

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/joystream/pioneer-testnet/3ofQGYHKEnJuyVwVymVxfa1qTW46
✅ Preview: Canceled

[Deployment for 7aea9f8 canceled]

@bedeho
Copy link
Copy Markdown
Member

bedeho commented Feb 7, 2022

  1. I think its better to turn the entire format into one message, otherwise (se)deserializing this properly will be more cumbersome.
  2. What is purpose of NonLeafNodeMerkleBranchByteOffset message?
  3. CreatorPayoutBody::merkle_branches_of_non_leaf_nodes field ?

@zeeshanakram3
Copy link
Copy Markdown
Contributor Author

  1. Do you mean both header and body should be the single message? Also, doesn't Protobuf provide out-of-the-box functionality to (se)deserializing different message properties?

For 2 & 3, please read the detailed answer:

If the payout vector is[c_0||p_0||m_0,...,c_7||p_7||m_7] , then the payload body according to proposed format would be:

c_0||p_0||proof_0||d_0 ... c_7||p_7||proof_7||d_7        

Now suppose we need to prove the membership of payout record for channel id c_2(shown green in the diagram), to prove that and claim a reward from runtime we need to pass the following args to the extrinsic

pub fn claim_channel_reward(
           ...
           proof: Vec<ProofElement<T>>, //  [`proof_3`, `proof_01`, `proof_4567`] (shown blue in the diagram)
           item: PullPayment<T>, // { c_2, p_2, m_2 }
       ) -> DispatchResult {

image

Problem

The payload body does not have all the Merkle branches required to construct the membership proof. For this specific example, proof_01 & proof_4567 are missing from the payload body.
To construct proof we need Merkle branches of both leaf and non-leaf(see diagram for differentiation) nodes,

Solution

Include the remaining Merkle branches in the payload body

Why need CreatorPayoutBody::merkle_branches_of_non_leaf_nodes?

merkle_branches_of_non_leaf_nodes is needed to construct the complete Merkle path required to prove the membership of payout entry(c_i||p_i||m_i) for channel id c_i.

Why need NonLeafNodeMerkleBranchByteOffset?

It is required to index into Merkle branch (of the non-leaf node) at a given byte offset in the payload body to construct Merkle path

@zeeshanakram3
Copy link
Copy Markdown
Contributor Author

zeeshanakram3 commented Feb 8, 2022

Update

a merkle branch is a full sequence of hashes that make up the required proof

As you said in the discussion, another way to solve the problem is (and I assume this is what you actually meant):

to change the structure of payout record, considering the above example, the payout record for channel id c_2 in the payload body would be:

c_2||p_2||proof_3||proof_01||proof_4567||d_2

Schema Changes

The payout record for single channel would be:

  message CreatorPayout {
    required uint32 channel_id = 1;  // c_i
    required float cumulative_payout_owed = 2; // p_i
   
    enum Side {
        Left = 0;
        Right = 1;
    }

    message MerkleBranch {
        required bytes merkle_branch = 1;
        required Side side = 2;
    }

    repeated MerkleBranch merkle_branches = 3; 
    required string payout_rationale = 4; // d_i
  }

It will be the responsibility of someone who actually uploads the raw payload to provide the correct sequence of hashes that make up the membership proof. And, not the responsibility of client software to calculate the Merkle path and then fetch relevant branches/hashes/nodes that make up the required proof

@zeeshanakram3 zeeshanakram3 marked this pull request as draft February 9, 2022 07:36
@zeeshanakram3 zeeshanakram3 marked this pull request as ready for review February 10, 2022 09:48
offset?: number
): Promise<CreatorPayoutPayload.Header | CreatorPayoutPayload.Body.CreatorPayout> {
// HTTP Url of remote host where payload is stored
let url = process.env.CREATOR_PAYOUT_DATA_URL as string
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the browser env, we won't be able to define this at build time since this URL can be dynamic. So it need to provided via some config/parameter instead

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. In fact, that was discussed earlier too, I had planned to change that, Now I will prioritize it.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just making a note, not a priority at all from my perspective, realistically, the earliest time for using this in atlas is like 2 months

@zeeshanakram3 zeeshanakram3 changed the base branch from olympia to rhodes March 18, 2022 09:19
@bedeho bedeho changed the base branch from rhodes to ephesus April 28, 2022 10:30
Lezek123 added 3 commits May 26, 2022 12:48
…-integration

Project token->Content pallet integration
…rs-member-id

Creator token transfers: Validate dst member_id
@vercel
Copy link
Copy Markdown

vercel bot commented Jun 2, 2022

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Updated
pioneer-testnet ⬜️ Ignored (Inspect) Sep 25, 2022 at 0:34AM (UTC)

@zeeshanakram3 zeeshanakram3 changed the base branch from ephesus to carthage September 27, 2022 08:56
@zeeshanakram3
Copy link
Copy Markdown
Contributor Author

superseded by #4334

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants