Metadata V15: Expose API to fetch metadata for version by lexnv · Pull Request #13287 · paritytech/substrate

lexnv · 2023-01-31T17:26:12Z

This PR extends the Metadata API to fetch the metadata at a given provided version.

It lays the foundation for keeping the metadata V14 around and exposing the V15 for early adopters before making a major breaking change.

The runtime API comes with 2 new functions:

metadata_at_version - Fetch the metadata at a given version
metadata_versions - List the supported versions by metadata_at_version

At the current moment, substrate only supports V14.
Soon, the V15 will be exposed as unstable under the u32::max value.

Users could then fetch the v15 metadata, allowing for an appropriate time to migrate their use cases.

Part of: #12939.

polkadot companion: paritytech/polkadot#6655
cumulus companion: paritytech/cumulus#2155

CC: @paritytech/tools-team

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

lexnv · 2023-02-02T17:15:03Z

bot rebase

…d_at_versions

paritytech-processbot · 2023-02-02T17:15:12Z

Rebased

lexnv · 2023-02-07T14:02:56Z

bot rebase

…d_at_versions

paritytech-processbot · 2023-02-07T14:03:10Z

Rebased

frame/support/test/tests/pallet.rs

jsdw · 2023-02-08T11:01:45Z

primitives/api/src/lib.rs

+		/// Returns the supported versions of metadata.
+		///
+		/// This can be used to call `metadata_at_version`.


Suggested change

/// Returns the supported versions of metadata.

///

/// This can be used to call `metadata_at_version`.

/// Returns the available metadata versions.

Nit; Just because "This can be used to call.." felt a bit unclear :)

I would keep "supported". Feels more clear on what this is about.

jsdw

I'm not an expert in this area but it looks good to me!

frame/support/test/tests/pallet.rs

frame/support/procedural/src/construct_runtime/expand/metadata.rs

bkchr

Looks good, but we should forward the selection of the supported metadata to the metadata crate. See my comments for more information.

bkchr · 2023-02-13T08:39:28Z

primitives/api/src/lib.rs

+		/// Returns the supported versions of metadata.
+		///
+		/// This can be used to call `metadata_at_version`.


I would keep "supported". Feels more clear on what this is about.

primitives/api/src/lib.rs

frame/support/procedural/src/construct_runtime/expand/metadata.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

…d_at_versions

lexnv · 2023-03-15T16:58:33Z

bot merge

paritytech-processbot · 2023-03-15T16:58:42Z

Error: "Check reviews" status is not passing for paritytech/polkadot#6655

lexnv · 2023-03-15T17:00:11Z

bot merge

paritytech-processbot · 2023-03-15T17:00:20Z

Error: "Check reviews" status is not passing for paritytech/polkadot#6655

bkchr · 2023-03-15T17:47:24Z

Cc @jacogr

lexnv · 2023-03-15T17:49:18Z

bot merge

) * impl_runtime_apis: Generate getters for `metadata_at` functions Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * runtime: Implement new `Metadata` runtime trait Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * runtime: Move `metadata_at` functions to construct_runtime macro Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * contruct_runtime: Use `OpaqueMetadata` from hidden imports Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Adjust testing Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/tests: Add tests for the new API Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/tests: Adjust metdata naming Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Expose `metadata-v14` feature flag Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Expose metadata only under feature flags Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Expose v14 metadata by default Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Expose metadata feature for testing Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Test metadata under different feature flags Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update primitives/api/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update primitives/api/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * client/tests: Adjust testing to reflect trait Metadata change Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/metadata-ir: Add intermediate representation types for metadata Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/metadata-ir: Convert metadata to V14 Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/metadata-ir: Add API to convert metadata to multiple versions Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/metadata-ir: Expose V14 under feature flag Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Adjust to metadata IR Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: More adjustments Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Guard v14 details under feature flag Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Adjust testing Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * CI: Ensure `quick-benchmarks` uses `metadata-v14` Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Use `metadata-v14` for benchmarks Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Adjust cargo fmt Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * kitchensink-runtime: Add feature flag for `metadata-v14` Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support/test: Adjust testing Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support/test: Check crates locally Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Activate metadata-v14 for pallets Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Remove metadata-v14 feature flag Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/metadata_ir: Move `api.rs` to `mod.rs` Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Handle latest metadata conversion via IR Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/tests: Add constant for metadata version 14 Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support/test: Fix merge conflict Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update frame/support/Cargo.toml Co-authored-by: Bastian Köcher <git@kchr.de> * Update frame/support/src/metadata_ir/mod.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update frame/support/test/Cargo.toml Co-authored-by: Bastian Köcher <git@kchr.de> * Update primitives/api/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * frame/metadata: Collect pallet documentation for MetadataIR Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/tests: Check pallet documentation is propagated to MetadataIR Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Improve documentation Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> --------- Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Co-authored-by: parity-processbot <> Co-authored-by: Bastian Köcher <git@kchr.de>

Polkadot-Forum · 2023-05-10T15:30:54Z

This pull request has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/stablising-v15-metadata/2819/1

) * impl_runtime_apis: Generate getters for `metadata_at` functions Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * runtime: Implement new `Metadata` runtime trait Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * runtime: Move `metadata_at` functions to construct_runtime macro Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * contruct_runtime: Use `OpaqueMetadata` from hidden imports Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Adjust testing Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/tests: Add tests for the new API Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/tests: Adjust metdata naming Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Expose `metadata-v14` feature flag Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Expose metadata only under feature flags Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Expose v14 metadata by default Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Expose metadata feature for testing Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Test metadata under different feature flags Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update primitives/api/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update primitives/api/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * client/tests: Adjust testing to reflect trait Metadata change Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/metadata-ir: Add intermediate representation types for metadata Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/metadata-ir: Convert metadata to V14 Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/metadata-ir: Add API to convert metadata to multiple versions Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/metadata-ir: Expose V14 under feature flag Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Adjust to metadata IR Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: More adjustments Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Guard v14 details under feature flag Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Adjust testing Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * CI: Ensure `quick-benchmarks` uses `metadata-v14` Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Use `metadata-v14` for benchmarks Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Adjust cargo fmt Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * kitchensink-runtime: Add feature flag for `metadata-v14` Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support/test: Adjust testing Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support/test: Check crates locally Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Activate metadata-v14 for pallets Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Remove metadata-v14 feature flag Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/metadata_ir: Move `api.rs` to `mod.rs` Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Handle latest metadata conversion via IR Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/tests: Add constant for metadata version 14 Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support/test: Fix merge conflict Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update frame/support/Cargo.toml Co-authored-by: Bastian Köcher <git@kchr.de> * Update frame/support/src/metadata_ir/mod.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update frame/support/test/Cargo.toml Co-authored-by: Bastian Köcher <git@kchr.de> * Update primitives/api/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * frame/metadata: Collect pallet documentation for MetadataIR Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/tests: Check pallet documentation is propagated to MetadataIR Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Improve documentation Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> --------- Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Co-authored-by: parity-processbot <> Co-authored-by: Bastian Köcher <git@kchr.de>

lexnv added 6 commits January 30, 2023 16:46

impl_runtime_apis: Generate getters for metadata_at functions

a730fe1

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

runtime: Implement new Metadata runtime trait

bc1fed4

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

runtime: Move metadata_at functions to construct_runtime macro

46681dd

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

contruct_runtime: Use OpaqueMetadata from hidden imports

d03dd7d

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

Adjust testing

441b7ad

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

frame/tests: Add tests for the new API

555ccfb

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

lexnv self-assigned this Jan 31, 2023

github-actions bot added the A0-please_review Pull request needs code review. label Jan 31, 2023

lexnv added B7-runtimenoteworthy C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit labels Jan 31, 2023

This was referenced Jan 31, 2023

Companion for #13287 paritytech/polkadot#6655

Merged

Companion for #13287 paritytech/cumulus#2155

Merged

Merge remote-tracking branch 'origin/master' into lexnv/md15_expose_m…

b43038f

…d_at_versions

lexnv mentioned this pull request Feb 2, 2023

Metadata V15: Add Runtime API metadata #13302

Merged

4 tasks

Merge remote-tracking branch 'origin/master' into lexnv/md15_expose_m…

f484197

…d_at_versions

jsdw reviewed Feb 8, 2023

View reviewed changes

frame/support/test/tests/pallet.rs Outdated Show resolved Hide resolved

jsdw reviewed Feb 8, 2023

View reviewed changes

jsdw approved these changes Feb 8, 2023

View reviewed changes

niklasad1 reviewed Feb 13, 2023

View reviewed changes

frame/support/test/tests/pallet.rs Show resolved Hide resolved

niklasad1 reviewed Feb 13, 2023

View reviewed changes

frame/support/procedural/src/construct_runtime/expand/metadata.rs Outdated Show resolved Hide resolved

bkchr suggested changes Feb 13, 2023

View reviewed changes

the-right-joyce added B1-note_worthy Changes should be noted in the release notes T1-runtime This PR/Issue is related to the topic “runtime”. and removed B7-runtimenoteworthy labels Feb 13, 2023

jsdw self-requested a review February 13, 2023 12:30

lexnv and others added 8 commits March 15, 2023 09:28

Update frame/support/Cargo.toml

81c91ac

Co-authored-by: Bastian Köcher <git@kchr.de>

Update frame/support/src/metadata_ir/mod.rs

98fab89

Co-authored-by: Bastian Köcher <git@kchr.de>

Update frame/support/test/Cargo.toml

31d3483

Co-authored-by: Bastian Köcher <git@kchr.de>

Update primitives/api/src/lib.rs

63b3109

Co-authored-by: Bastian Köcher <git@kchr.de>

frame/metadata: Collect pallet documentation for MetadataIR

db4f553

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

frame/tests: Check pallet documentation is propagated to MetadataIR

a38ca53

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

frame/support: Improve documentation

6208434

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

Merge remote-tracking branch 'origin/master' into lexnv/md15_expose_m…

9065690

…d_at_versions

paritytech-processbot bot merged commit bd52212 into master Mar 15, 2023

paritytech-processbot bot deleted the lexnv/md15_expose_md_at_versions branch March 15, 2023 17:49

jacogr mentioned this pull request Mar 15, 2023

Cater for runtime Metadata_* v2 polkadot-js/api#5545

Merged

haerdib mentioned this pull request Mar 23, 2023

Investigate: can the tip type be detected via metadata or Runtime? scs/substrate-api-client#344

Closed

jasl mentioned this pull request May 11, 2023

Upgrade to Polkadot v0.9.42 Phala-Network/phala-blockchain#1263

Merged

github-actions bot mentioned this pull request Jun 15, 2023

Update substrate/polkadot/cumulus from v0.9.40 to v0.9.43 moonbeam-foundation/moonbeam#2354

Closed

valentinfernandez1 mentioned this pull request Jun 23, 2023

Upgrade to 0.9.42 paritytech/trappist#240

Closed

7 tasks

kacperzuk-neti mentioned this pull request Jun 23, 2023

Polkadot v0.9.43 liberland/liberland_substrate#295

Merged

15 tasks

lexnv mentioned this pull request Jun 29, 2023

Stabilize V15 Metadata #14481

Merged

aurexav added a commit to darwinia-network/darwinia that referenced this pull request Jul 6, 2023

Companion of paritytech/substrate#13287

81b1efb

ToufeeqP mentioned this pull request Aug 31, 2023

Upgrade to polkadot-v1.0 availproject/avail#195

Merged

dmitrylavrenov mentioned this pull request Nov 14, 2023

Revisit custom RPCs in view of Metadata V15 humanode-network/humanode#833

Open

Imod7 mentioned this pull request Mar 6, 2024

feat: add version query param in transaction/material endpoint paritytech/substrate-api-sidecar#1410

Closed

Conversation

lexnv commented Jan 31, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lexnv commented Feb 2, 2023

Uh oh!

paritytech-processbot bot commented Feb 2, 2023

Uh oh!

lexnv commented Feb 7, 2023

Uh oh!

paritytech-processbot bot commented Feb 7, 2023

Uh oh!

Uh oh!

jsdw Feb 8, 2023

Choose a reason for hiding this comment

Uh oh!

bkchr Feb 13, 2023

Choose a reason for hiding this comment

Uh oh!

jsdw left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

bkchr left a comment

Choose a reason for hiding this comment

Uh oh!

bkchr Feb 13, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

lexnv commented Mar 15, 2023

Uh oh!

paritytech-processbot bot commented Mar 15, 2023

Uh oh!

lexnv commented Mar 15, 2023

Uh oh!

paritytech-processbot bot commented Mar 15, 2023

Uh oh!

bkchr commented Mar 15, 2023

Uh oh!

lexnv commented Mar 15, 2023

Uh oh!

Polkadot-Forum commented May 10, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

lexnv commented Jan 31, 2023 •

edited

Loading