Skip to content

Metadata: Retain a subset of metadata pallets#879

Merged
jsdw merged 32 commits intomasterfrom
lexnv/stirp_metadata
Apr 4, 2023
Merged

Metadata: Retain a subset of metadata pallets#879
jsdw merged 32 commits intomasterfrom
lexnv/stirp_metadata

Conversation

@lexnv
Copy link
Collaborator

@lexnv lexnv commented Mar 23, 2023 •
edited by jsdw
Loading

This PR strips the metadata to include only some provided pallets.

Metadata size with single pallet

The No Generics means that the extrinsic types were stripped of the generic type parameters.
Those generics where not needed and include nested type IDs that are unnecessarily increasing
the final metadata size. (Relies on paritytech/scale-info#176).

No Generics With Generics Pallet
244K 244K Alliance.scale
240K 240K AllianceMotion.scale
8,0K 240K AssetTxPayment.scale
32K 244K Assets.scale
12K 12K Auctions.scale
8,0K 240K AuthorityDiscovery.scale
8,0K 240K Authorship.scale
16K 244K Babe.scale
16K 244K Balances.scale
16K 244K Bounties.scale
16K 240K ChildBounties.scale
12K 12K Claims.scale
16K 16K Configuration.scale
28K 252K Contracts.scale
16K 240K ConvictionVoting.scale
16K 240K CoreFellowship.scale
240K 240K Council.scale
16K 16K Crowdloan.scale
248K 248K Democracy.scale
8,0K 8,0K Dmp.scale
20K 248K ElectionProviderMultiPhase.scale
16K 244K Elections.scale
12K 240K FastUnstake.scale
8,0K 240K Glutton.scale
12K 240K Grandpa.scale
8,0K 240K Historical.scale
16K 16K Hrmp.scale
20K 244K Identity.scale
12K 240K ImOnline.scale
12K 240K Indices.scale
8,0K 8,0K Initializer.scale
240K 240K Lottery.scale
12K 240K MessageQueue.scale
8,0K 240K Mmr.scale
240K 240K Multisig.scale
44K 252K Nfts.scale
20K 248K Nis.scale
28K 248K NominationPools.scale
8,0K 240K Offences.scale
12K 12K ParaInclusion.scale
12K 12K ParaInherent.scale
12K 12K ParaScheduler.scale
8,0K 8,0K ParaSessionInfo.scale
8,0K 8,0K ParachainsOrigin.scale
20K 20K Paras.scale
12K 12K ParasDisputes.scale
8,0K 8,0K ParasShared.scale
16K 16K PhragmenElection.scale
8,0K 240K Pov.scale
12K 240K Preimage.scale
240K 240K Proxy.scale
8,0K 240K RandomnessCollectiveFlip.scale
12K 240K RankedCollective.scale
248K 248K RankedPolls.scale
244K 244K Recovery.scale
248K 248K Referenda.scale
12K 12K Registrar.scale
8,0K 240K Remark.scale
8,0K 240K RootTesting.scale
12K 240K Salary.scale
240K 240K Scheduler.scale
12K 240K Session.scale
12K 12K Slots.scale
20K 244K Society.scale
36K 252K Staking.scale
12K 240K StateTrieMigration.scale
240K 240K Sudo.scale
312K 312K System.scale
240K 240K TechnicalCommittee.scale
12K 240K TechnicalMembership.scale
8,0K 240K Timestamp.scale
16K 240K Tips.scale
8,0K 240K TransactionPayment.scale
12K 240K TransactionStorage.scale
12K 240K Treasury.scale
12K 12K Ump.scale
28K 244K Uniques.scale
240K 240K Utility.scale
12K 240K Vesting.scale
12K 244K VoterList.scale
240K 240K Whitelist.scale
36K 36K XcmPallet.scale
468K 468K polkadot_metadata.scale

Closes #629.

// CC @paritytech/subxt-team

lexnv added 2 commits March 23, 2023 16:35
@lexnv
Update cargo.lock to use scale-info v2.4.0
4dbbd16 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
metadata: Retain only a subset of the metadata
99934de 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv lexnv self-assigned this Mar 23, 2023
jsdw
jsdw reviewed Mar 24, 2023
View reviewed changes
jsdw
jsdw reviewed Mar 24, 2023
View reviewed changes
lexnv added 12 commits March 24, 2023 16:22
@lexnv
codegen: Generate top level Event
3f0f55f 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
metadata: Only retain DispatchError
704c512 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
metadata: Export just the retain method
d49c801 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
cli: Retain pallets
5ebab47 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
metadata: Do not include extrinsic metadata
47e9163 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
retain: Fix clippy
19eec1f 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
test-runtime: Generate per pallet metadata and rs file
725a2e5 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
ui-tests: Check per metadata generated files
8515b1e 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
Revert "test-runtime: Generate per pallet metadata and rs file"
a8d368b 
This reverts commit 725a2e5.

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
Merge remote-tracking branch 'origin/master' into lexnv/stirp_metadata
64c316b
@lexnv
ui-tests: Adjust path to metadata file
36bfe9e 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
ui-tests: Change drop order to keep PalletMetadata around
e90920c 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
jsdw
jsdw reviewed Mar 28, 2023
View reviewed changes
jsdw
jsdw reviewed Mar 28, 2023
View reviewed changes
jsdw
jsdw reviewed Mar 28, 2023
View reviewed changes
jsdw
jsdw reviewed Mar 28, 2023
View reviewed changes
jsdw
jsdw reviewed Mar 28, 2023
View reviewed changes
jsdw
jsdw reviewed Mar 28, 2023
View reviewed changes
jsdw
jsdw reviewed Mar 28, 2023
View reviewed changes
lexnv and others added 3 commits March 28, 2023 13:14
@lexnv @jsdw
Update metadata/src/retain.rs
396cc7c 
Co-authored-by: James Wilson <james@jsdw.me>
@lexnv
Address feedback
cc1809f 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
retain: Keep extrinsic type
5a3c81d 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
jsdw
jsdw reviewed Mar 28, 2023
View reviewed changes
lexnv added 2 commits March 28, 2023 13:39
@lexnv
cli: Introduce MetadataSource
4a71546 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
cli: Use MetadataSource helper
a663236 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
jsdw
jsdw reviewed Mar 29, 2023
View reviewed changes
lexnv added 2 commits March 29, 2023 18:03
@lexnv
retain: Do not include generic type params in retained metadata
01ff1b5 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
Adjust subxt to scale-info v2.5.0
e28bb8a 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv lexnv mentioned this pull request Mar 29, 2023
Merged
@jsdw
Copy link
Collaborator

jsdw commented Mar 29, 2023

Awesome; looks like removing those generic types had a really notable impact! (allets like Sudo and MultiSig I guess depend on RuntimeCall for the inner stuff which depends on types from all other pallets, so it makes sense that they are still big :))

lexnv added 4 commits March 29, 2023 23:08
@lexnv
Update scaleinfo to v2.5.0
2dee176 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
Remove deprecated fn
13cbbd6 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
testing: Fix clippy
115fbfb 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
benches: Use inner fields of scale info
6284f4a 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
jsdw
jsdw approved these changes Mar 31, 2023
View reviewed changes
Copy link
Collaborator

@jsdw jsdw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great; happy to see this merge :)

@niklasad1 niklasad1 self-requested a review March 31, 2023 15:28
niklasad1
niklasad1 reviewed Apr 4, 2023
View reviewed changes
niklasad1
niklasad1 reviewed Apr 4, 2023
View reviewed changes
niklasad1
niklasad1 reviewed Apr 4, 2023
View reviewed changes
niklasad1
niklasad1 reviewed Apr 4, 2023
View reviewed changes
@jsdw jsdw self-requested a review April 4, 2023 11:08
@jsdw
Copy link
Collaborator

jsdw commented Apr 4, 2023

Instead of trying to strip type params out of the extrinsic type (whose effect is primarily to remove the reference to the RuntimeCall type), I now strip the RuntimeCall type variants to only include those for pallets we want to keep. This keeps the size fairly uniformly small across all pallets now (below based on artifacts/polkadot.metadata):

pallet size
System 64233
Scheduler 13027
Preimage 9102
Babe 14074
Timestamp 7879
Indices 10815
Balances 14605
TransactionPayment 8060
Authorship 8461
Staking 35015
Offences 7783
Historical 6555
Session 10574
Grandpa 11876
ImOnline 10598
AuthorityDiscovery 6563
Democracy 23583
Council 16705
TechnicalCommittee 16749
PhragmenElection 15828
TechnicalMembership 8984
Treasury 11710
Claims 11536
Vesting 11809
Utility 13546
Identity 21004
Proxy 16162
Multisig 16092
Bounties 13829
ChildBounties 14950
Tips 13978
ElectionProviderMultiPhase 18513
VoterList 12627
NominationPools 23233
FastUnstake 10442
ParachainsOrigin 6561
Configuration 14857
ParasShared 7399
ParaInclusion 12255
ParaInherent 11830
ParaScheduler 9923
Paras 17175
Initializer 8226
Dmp 7453
Ump 11275
Hrmp 15863
ParaSessionInfo 8246
ParasDisputes 9899
Registrar 12484
Slots 9715
Auctions 11495
Crowdloan 14214
XcmPallet 36657

niklasad1
niklasad1 reviewed Apr 4, 2023
View reviewed changes
niklasad1
niklasad1 reviewed Apr 4, 2023
View reviewed changes
niklasad1
niklasad1 reviewed Apr 4, 2023
View reviewed changes
@jsdw jsdw merged commit 8a7c172 into master Apr 4, 2023
@jsdw jsdw deleted the lexnv/stirp_metadata branch April 4, 2023 15:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jsdw jsdw Awaiting requested review from jsdw

1 more reviewer

@niklasad1 niklasad1 niklasad1 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@lexnv lexnv

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

CLI Tool: Strip metadata to only include certain pallets

3 participants

@lexnv @jsdw @niklasad1