Skip to content

Graph simulator init#196

Merged
aramikm merged 11 commits intomainfrom
graph_simulation
May 2, 2024
Merged

Graph simulator init#196
aramikm merged 11 commits intomainfrom
graph_simulation

Conversation

@aramikm
Copy link
Copy Markdown
Collaborator

@aramikm aramikm commented May 1, 2024

Goal

The goal of this PR is to allow us to do some large scale verification and stress testing.

Closes #197

Discussion

  • Precomputed state with 20_000 users is uploaded here
  • Refer to ReadMe for more info

Computed Scenarios

  • Key rotation for random pages in private Follows
  • Key rotation for random pages in private Friendship

@aramikm aramikm marked this pull request as ready for review May 1, 2024 21:58
Comment on lines +13 to +14
For being able to run the 20k users example the **maxPageId** on _frequency.json_ file should be changed to at least **64**.
We will update this value after it was updated un Frequency chain first.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the other pending PR we're only bumping it to 32... should we tweak this test scenario?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also i updated to 64 in frequency.json and test failed, maybe we need to update in all configs or any specific config? from testing perspective

Should import data: InvalidPageId(17)
thread '' panicked at simulator/src/scenarios/common.rs:41:18:
Should import data: InvalidPageId(17)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

based on what is mentioned in the Readme you should only update frequency.json. I will add that you will need ot build after so that it rans the latest version

Copy link
Copy Markdown
Collaborator Author

@aramikm aramikm May 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JoeCap08055 If we would like to test any private friendship scenarios we need at least 64 but since we don't have any use-cases on-chain, it doesn't make sense to set the chain value to 64 for now.

use dsnp_graph_core::api::api_types::*;
use rand::{prelude::SliceRandom, thread_rng};

pub fn execute_key_rotation_private_follow(state: &mut GlobalState, env: Environment) {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, this makes me think... I know we talked about a future enhancement to support a key rotation top-level method in the API, but I don't think we ever created an issue for it...

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have force_recalculate_graphs that does key rotation on all pages also but currently we only do key rotation lazily whenever a page is modified.

Copy link
Copy Markdown
Collaborator

@JoeCap08055 JoeCap08055 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When running without the 20000 users pre-loaded data, I see lots of messages about

thread '<unnamed>' panicked at simulator/src/init.rs:145:18:
error exporting updates with 6099 connections: GraphIsFull

When running with the 20000 pre-load file, I see the same errors, but the process finishes pretty quickly.

Both runs end with the same result:

make: *** [run-sim] Error 101

Steps taken:

  • Edit frequency.json to change max page id to 64
  • Put 20000_simulation_state.bin in project root
  • Run make build-sim ; make run-sim

Copy link
Copy Markdown
Contributor

@saraswatpuneet saraswatpuneet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left some nit

  • ran fine, i had to update maxPageId in all 3 configs but not sure which is right one
  • we can update the doc to reflect the above

@JoeCap08055
Copy link
Copy Markdown
Collaborator

When running without the 20000 users pre-loaded data, I see lots of messages about

thread '<unnamed>' panicked at simulator/src/init.rs:145:18:
error exporting updates with 6099 connections: GraphIsFull

When running with the 20000 pre-load file, I see the same errors, but the process finishes pretty quickly.

Both runs end with the same result:

make: *** [run-sim] Error 101

Steps taken:

  • Edit frequency.json to change max page id to 64
  • Put 20000_simulation_state.bin in project root
  • Run make build-sim ; make run-sim

Found my error; re: @saraswatpuneet 's comment about building the SDK first. I had a stale build from an older branch.
Simulation runs successfully now.

Copy link
Copy Markdown
Collaborator

@JoeCap08055 JoeCap08055 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Built SDK and ran simulation
    lgtm

@aramikm aramikm merged commit 10aba19 into main May 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Simulator] Allow running system wide graph simulation

3 participants