try-runtime-cli: `execute-block` & `create-snapshot` tests by Szegoo · Pull Request #14343 · paritytech/substrate

Szegoo · 2023-06-11T15:55:49Z

This PR adds tests for the following try-runtime commands:

execute-block
create-snapshot

Part of: #13796

Kusama address: DfqY6XQUSETTszBQ1juocTcG9iiDoXhvq1CoVadBSUqTGJS

Szegoo · 2023-06-12T06:48:41Z

@liamaharon Could you review this?

liamaharon

Looking good, thanks for this!

utils/frame/try-runtime/cli/tests/create_snapshot.rs

utils/frame/try-runtime/cli/tests/follow_chain.rs

utils/frame/try-runtime/cli/tests/execute_block.rs

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

utils/frame/try-runtime/cli/Cargo.toml

utils/frame/try-runtime/cli/tests/create_snapshot.rs

liamaharon

LGTM

utils/frame/try-runtime/cli/tests/create_snapshot.rs

utils/frame/try-runtime/cli/tests/execute_block.rs

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

kianenigma · 2023-06-16T10:55:51Z

utils/frame/try-runtime/cli/tests/create_snapshot.rs

+				Command::new(cargo_bin("substrate"))
+					.stdout(process::Stdio::piped())
+					.stderr(process::Stdio::piped())
+					.args(&["try-runtime", "--runtime=existing"])


FYI, the need for --runtime for this subcommand was a fundamental mistake that I added at some point @liamaharon 🙈

wdyt about existing always being a default?

utils/frame/try-runtime/cli/tests/create_snapshot.rs

kianenigma · 2023-06-16T10:57:19Z

utils/frame/try-runtime/cli/tests/execute_block.rs

+					.stderr(process::Stdio::piped())
+					.args(&["try-runtime", "--runtime=existing"])
+					.args(&["execute-block"])
+					.args(&["live", format!("--uri={}", ws_url).as_str()])


This will only execute the latest block. Would be interesting if you also think about incorporating --at, if reasonably possible.

Same applies to the other one as well.

Sure, I will do that.

test-utils/cli/src/lib.rs

kianenigma · 2023-06-20T12:42:55Z

@liamaharon Don't merge this yet :P I am working on incorporating --at into the tests.

Please let us know once ready for final review.

Szegoo · 2023-06-21T09:55:29Z

@liamaharon I realised that the problem I encountered was not in the block_hash function. It seems like there isn't an issue, but I am a bit confused.

In the execute_block function why don't we execute the block we specified with the --at option? The code will do execute_block for the next block.

substrate/utils/frame/try-runtime/cli/src/commands/execute_block.rs

Lines 99 to 107 in 6e0059a

    
           let executor = build_executor::<HostFns>(&shared); 
        
           let ext = command.state.into_ext::<Block, HostFns>(&shared, &executor, None, true).await?; 
        
           // get the block number associated with this block. 
        
           let block_ws_uri = command.block_ws_uri::<Block>(); 
        
           let rpc = ws_client(&block_ws_uri).await?; 
        
           let next_hash = next_hash_of::<Block>(&rpc, ext.block_hash).await?; 
        
           log::info!(target: LOG_TARGET, "fetching next block: {:?} ", next_hash);

liamaharon · 2023-06-21T10:27:59Z

Hmm.. Intuitively I would also expect execute_block to re-execute the block at rather than at + 1. @kianenigma do you remember if this behavior is intentional?

utils/frame/try-runtime/cli/tests/create_snapshot.rs

ggwpez · 2023-06-21T10:43:01Z

utils/frame/try-runtime/cli/tests/create_snapshot.rs

+	#[tokio::test]
+	async fn create_snapshot_works() {
+		// Build substrate so binaries used in the test use the latest code.
+		common::build_substrate(&["--features=try-runtime"]);


I now understood why we have to build again. Just very annoying and slow…
Maybe we can somehow inherit the build args in this function, so that it re-uses the cargo cache.

That would be great. It does take quite some time to run all of these tests.

Yea that would be a follow up as well; debug that build_substrate function and check if we can make it re-use the already existing build artifacts by using the same compile args that were used for the test.

utils/frame/try-runtime/cli/tests/create_snapshot.rs

utils/frame/try-runtime/cli/tests/execute_block.rs

ggwpez · 2023-06-21T10:49:46Z

frame/executive/src/lib.rs


+		frame_support::log::info!(
+			target: LOG_TARGET,
+			"try-runtime: Block #{:?} successfully executed",


Just some ideas, we could do stuff here like:

State trace (output of all modified storage keys)

Print all executed calls + Events

Consumed weight

Block size etc

Probably most of it in JSON, so we can render it eventually (like chopsticks already does for state changes).

Should this be done in this PR?

No. Just some ideas for the future 😄

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

ggwpez · 2023-06-21T13:22:32Z

Needs to merge master to make the CI happy.

ggwpez · 2023-06-24T09:35:37Z

Going to merge this. We can investigate afterwards if this --at needs a fix. This way we already have some tests.

ggwpez · 2023-06-24T09:36:03Z

bot merge

paritytech-processbot · 2023-06-24T09:36:10Z

Waiting for commit status.

…h#14343) * execute-block test * test create-snapshot * oops * Update utils/frame/try-runtime/cli/tests/create_snapshot.rs Co-authored-by: Liam Aharon <liam.aharon@hotmail.com> * Update utils/frame/try-runtime/cli/tests/create_snapshot.rs Co-authored-by: Liam Aharon <liam.aharon@hotmail.com> * Update utils/frame/try-runtime/cli/tests/create_snapshot.rs Co-authored-by: Liam Aharon <liam.aharon@hotmail.com> * remove snapshot * execute block: new log * use prefix & make tempfile a dev dependencie * Update utils/frame/try-runtime/cli/tests/execute_block.rs Co-authored-by: Liam Aharon <liam.aharon@hotmail.com> * Update utils/frame/try-runtime/cli/tests/create_snapshot.rs Co-authored-by: Liam Aharon <liam.aharon@hotmail.com> * ".git/.scripts/commands/fmt/fmt.sh" * --at option in execute-block test * fixes & use --at option in create-snapshot test * hmm * fmt * remove nonsense * Update utils/frame/try-runtime/cli/tests/create_snapshot.rs Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update utils/frame/try-runtime/cli/tests/execute_block.rs Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * remove unnecessary test modules * try to load snapshot file * fix --------- Co-authored-by: Liam Aharon <liam.aharon@hotmail.com> Co-authored-by: command-bot <> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

kianenigma · 2023-08-14T05:57:58Z

/tip small

substrate-tip-bot · 2023-08-14T05:58:35Z

@kianenigma A small (2 KSM) tip was successfully submitted for @Szegoo (DfqY6XQUSETTszBQ1juocTcG9iiDoXhvq1CoVadBSUqTGJS on kusama).

https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Fkusama-rpc.polkadot.io#/referenda

execute-block test

4bb74f2

Szegoo changed the title ~~execute-block & create-snapshot tests~~ try-runtime-cli: execute-block & create-snapshot tests Jun 11, 2023

Szegoo added 2 commits June 12, 2023 08:45

test create-snapshot

cb9720c

oops

2644c25

Szegoo marked this pull request as ready for review June 12, 2023 06:47

liamaharon self-requested a review June 12, 2023 08:04

liamaharon reviewed Jun 14, 2023

View reviewed changes

Szegoo and others added 5 commits June 15, 2023 09:04

Update utils/frame/try-runtime/cli/tests/create_snapshot.rs

64ed955

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

Update utils/frame/try-runtime/cli/tests/create_snapshot.rs

aa7e6a7

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

Update utils/frame/try-runtime/cli/tests/create_snapshot.rs

2989dfc

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

remove snapshot

af1cfde

execute block: new log

c1d5b80

Szegoo requested review from a team and liamaharon June 15, 2023 08:07

liamaharon reviewed Jun 15, 2023

View reviewed changes

utils/frame/try-runtime/cli/Cargo.toml Outdated Show resolved Hide resolved

utils/frame/try-runtime/cli/tests/create_snapshot.rs Outdated Show resolved Hide resolved

paritytech-ci requested a review from a team June 15, 2023 09:57

use prefix & make tempfile a dev dependencie

c3a964d

liamaharon approved these changes Jun 16, 2023

View reviewed changes

utils/frame/try-runtime/cli/tests/create_snapshot.rs Outdated Show resolved Hide resolved

utils/frame/try-runtime/cli/tests/execute_block.rs Outdated Show resolved Hide resolved

Szegoo and others added 2 commits June 16, 2023 12:39

Update utils/frame/try-runtime/cli/tests/execute_block.rs

e7a54ab

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

Update utils/frame/try-runtime/cli/tests/create_snapshot.rs

d3a9ca9

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

kianenigma reviewed Jun 16, 2023

View reviewed changes

utils/frame/try-runtime/cli/tests/create_snapshot.rs Outdated Show resolved Hide resolved

kianenigma approved these changes Jun 16, 2023

View reviewed changes

Szegoo added 2 commits June 17, 2023 16:04

fixes & use --at option in create-snapshot test

84a27de

hmm

7581859

Szegoo commented Jun 17, 2023

View reviewed changes

test-utils/cli/src/lib.rs Show resolved Hide resolved

Szegoo added 2 commits June 17, 2023 16:27

fmt

8fc182a

remove nonsense

b7c157b

kianenigma added A3-in_progress Pull request is in progress. No review needed at this stage. and removed A0-please_review Pull request needs code review. labels Jun 20, 2023

ggwpez approved these changes Jun 21, 2023

View reviewed changes

Szegoo and others added 3 commits June 21, 2023 13:30

Update utils/frame/try-runtime/cli/tests/create_snapshot.rs

f1c77b4

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Update utils/frame/try-runtime/cli/tests/execute_block.rs

08b881c

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

remove unnecessary test modules

e78e7e4

Szegoo and others added 3 commits June 21, 2023 16:11

try to load snapshot file

4e3e29b

Merge branch 'paritytech:master' into try-runtime-tests

2c689d5

fix

daab1cc

ggwpez added A0-please_review Pull request needs code review. and removed A3-in_progress Pull request is in progress. No review needed at this stage. labels Jun 24, 2023

paritytech-processbot bot merged commit 0d09a26 into paritytech:master Jun 24, 2023

ggwpez mentioned this pull request Aug 24, 2023

Check if try-runtime execute-block respects --at paritytech/try-runtime-cli#26

Closed

kianenigma mentioned this pull request Aug 14, 2023

try-runtime-cli docs and tests #13796

Closed

6 tasks

crystalin mentioned this pull request Oct 20, 2023

Update substrate/polkadot/cumulus from v0.9.43 to v1.1.0 moonbeam-foundation/moonbeam#2535

Closed

Conversation

Szegoo commented Jun 11, 2023 • edited by juangirini Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Szegoo commented Jun 12, 2023

Uh oh!

liamaharon left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

liamaharon left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

liamaharon Jun 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kianenigma commented Jun 20, 2023

Uh oh!

Szegoo commented Jun 21, 2023

Uh oh!

liamaharon commented Jun 21, 2023

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ggwpez Jun 21, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ggwpez commented Jun 21, 2023

Uh oh!

ggwpez commented Jun 24, 2023

Uh oh!

ggwpez commented Jun 24, 2023

Uh oh!

paritytech-processbot bot commented Jun 24, 2023

Uh oh!

kianenigma commented Aug 14, 2023

Uh oh!

substrate-tip-bot bot commented Aug 14, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Szegoo commented Jun 11, 2023 •

edited by juangirini

Loading

liamaharon Jun 16, 2023 •

edited

Loading

ggwpez Jun 21, 2023 •

edited

Loading