Skip to content

Remove SubstrateEventEntity.extrinsicArgs#510

Merged
zeeshanakram3 merged 1 commit intoJoystream:masterfrom
Lezek123:remove-event-tx-args
Oct 12, 2022
Merged

Remove SubstrateEventEntity.extrinsicArgs#510
zeeshanakram3 merged 1 commit intoJoystream:masterfrom
Lezek123:remove-event-tx-args

Conversation

@Lezek123
Copy link
Copy Markdown

Removes unused SubstrateEventEntity.extrinsicArgs field, which had terrible impact on performance.

Example:
If I currently use utility.batch to create 1000 members, each of the thousands of event entities that result from this extrinsic will include a serialized version of a huge utility.batch call args. This is extermely bad case of redundancy, not only does it take enormous amount of storage, but it also takes ~10 minutes for the indexer to process such transaction.

…abase redundancy issue by remov

affects: @joystream/hydra-indexer-gateway, @joystream/hydra-indexer
@bedeho
Copy link
Copy Markdown
Member

bedeho commented Oct 11, 2022

Should this be in scope for Carthage ?

@Lezek123
Copy link
Copy Markdown
Author

I think that's a good idea, it will be a significant optimalization, fixing many issues with long query node indexing / processing times.

Copy link
Copy Markdown
Contributor

@zeeshanakram3 zeeshanakram3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the mappings, when you get the event context, it returns all the info related to the event/extrinsic, since now extrinsics args info is not being stored indexer DB, shouldn't the extrinsics args property also be removed from EventContext since the args are no longer available.

@Lezek123
Copy link
Copy Markdown
Author

From the mappings, when you get the event context, it returns all the info related to the event/extrinsic, since now extrinsics args info is not being stored indexer DB, shouldn't the extrinsics args property also be removed from EventContext since the args are no longer available.

When events are queried from indexer by the processor, this query is used:

  substrateEventsAfter(where: { ${eventsFilter}${extrinsicsFilter} blockNumber_gt: ${block_gt}, blockNumber_lte: ${block_lte} }, ${idFilter} limit: ${limit}) {
    id
    name 
    method 
    params {
      name
      type
      value
    }
    indexInBlock 
    blockNumber
    blockTimestamp
    extrinsic {
      method
      section
      versionInfo
      signer
      args
      signature
      hash
      tip
    }
  }

So instead of coming from event.extrinsicArgs, the extrinsic arguments come from event.extrinsic.args.
The event.extrinsicArgs was just a redundant, unused field.

@zeeshanakram3
Copy link
Copy Markdown
Contributor

So instead of coming from event.extrinsicArgs, the extrinsic arguments come from event.extrinsic.args. The event.extrinsicArgs was just a redundant, unused field.

Ah ok then, LGTM

@zeeshanakram3 zeeshanakram3 merged commit 0397c9d into Joystream:master Oct 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants