Skip to content

New console command: random seeder#458

Merged
josecelano merged 7 commits into
torrust:developfrom
josecelano:453-load-test
Feb 6, 2024
Merged

New console command: random seeder#458
josecelano merged 7 commits into
torrust:developfrom
josecelano:453-load-test

Conversation

@josecelano

@josecelano josecelano commented Feb 6, 2024

Copy link
Copy Markdown
Member

It will upload random torrents to the Index. For testing purposes. It could be used to seed E2E test environments too.

  • Reorganize console mod to include more mods.
  • Move API client from testing to production.
  • Create scaffolding for a new console command: seeder(with clap and anyhow)
  • Use the API client to seed a live env with random torrents.

The console command is:

cargo run --bin seeder -- --api-base-url <API_BASE_URL> --number-of-torrents <NUMBER_OF_TORRENTS> --user <USER> --password <PASSWORD> --interval <INTERVAL>

For example:

cargo run --bin seeder -- --api-base-url "localhost:3001" --number-of-torrents 1000 --user admin --password 12345678 --interval 0

That command would upload 1000 random torrents to the Index using the user account admin with password 123456 and wait 1 second between uploads.

Output:

$ cargo run --bin seeder -- --api-base-url "localhost:3001" --number-of-torrents    3 --user admin --password 12345678 --interval 5
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `target/debug/seeder --api-base-url 'localhost:3001' --number-of-torrents 3 --user admin --password 12345678 --interval 5`
2024-02-06T17:12:23.325055202+00:00 [seeder][INFO] Trying to login with username: admin ...
2024-02-06T17:12:23.666824220+00:00 [seeder][INFO] Logged as admin with account: admin 
2024-02-06T17:12:23.666870390+00:00 [seeder][INFO] Uploading 3 random torrents to the Torrust Index ...
2024-02-06T17:12:23.666875190+00:00 [seeder][INFO] Uploading torrent #1 ...
2024-02-06T17:12:23.666886180+00:00 [seeder][INFO] Uploading torrent with uuid: c6a4752a-d27c-4ba9-a367-cf41c9f22d4a ...
2024-02-06T17:12:23.735547434+00:00 [seeder][INFO] Uploaded torrent: {"torrent_id":54,"info_hash":"7e0a8e0461bd2fd46b55c95da95721c836ae9349"}
2024-02-06T17:12:28.735666148+00:00 [seeder][INFO] Uploading torrent #2 ...
2024-02-06T17:12:28.735738597+00:00 [seeder][INFO] Uploading torrent with uuid: a383519c-0018-43ec-82df-10044c61d46d ...
2024-02-06T17:12:28.793988941+00:00 [seeder][INFO] Uploaded torrent: {"torrent_id":55,"info_hash":"12f0f9edd2494d56a36c13b833aee37d1a0bb84d"}
2024-02-06T17:12:33.794118199+00:00 [seeder][INFO] Uploading torrent #3 ...
2024-02-06T17:12:33.794209398+00:00 [seeder][INFO] Uploading torrent with uuid: bc32812c-aab2-40b7-999e-d58c4070655c ...
2024-02-06T17:12:33.874166798+00:00 [seeder][INFO] Uploaded torrent: {"torrent_id":56,"info_hash":"4b9f6e1b2e4ddcb25b292ba0cb8561e2daa5b243"}

```console
src/console/
├── commands
│   ├── mod.rs
│   └── tracker_statistics_importer
│       ├── app.rs
│       └── mod.rs
├── cronjobs
│   ├── mod.rs
│   └── tracker_statistics_importer.rs
└── mod.rs
```

- We will create a new command to seed the Index with random torrents.
- This new strcuture is similar to what we have in the Tracker.
@josecelano josecelano added Testing Checking Torrust - System Admin - Enjoyable to Install and Setup our Software labels Feb 6, 2024
@josecelano josecelano added this to the v3.0.0 milestone Feb 6, 2024
@josecelano josecelano linked an issue Feb 6, 2024 that may be closed by this pull request
@josecelano josecelano requested a review from da2ce7 February 6, 2024 12:13
Added `server` subfolder. We will add a client folder.
@josecelano

Copy link
Copy Markdown
Member Author

Hi @da2ce7 I'm going to clone the API client from testing code to production. I would keep the same code in tests. I do not want to couple tests to production code. In the future, we could:

  • Extract a common package that both the server and the client depend on with common types (requests and responses). We can call it torrust-index-api-contract.
  • Extract a server package (torrust-api-server) that depends on the torrust-index-api-contract package.
  • Extract a client package (torrust-api-client) that depends on the torrust-index-api-contract package.
  • Tests in the server can use the torrust-api-client package.

We can refactor this in the next milestone.

It will be use in a new console command (seeder).
New console command to upload torrent to the Index remotely by using the
API client.

```console
cargo run --bin seeder -- --api-base-url <API_BASE_URL> --number-of-torrents <NUMBER_OF_TORRENTS> --user <USER> --password <PASSWORD> --interval <INTERVAL>
```

For example:

```console
cargo run --bin seeder -- --api-base-url "localhost:3001" --number-of-torrents 1000 --user admin --password 12345678 --interval 0
```

That command would upload 1000 random torrents to the Index using the user
account `admin` with password `123456` and waiting `1` second between uploads.
@josecelano josecelano marked this pull request as ready for review February 6, 2024 17:21
@josecelano

Copy link
Copy Markdown
Member Author

ACK 324fac7

@josecelano josecelano merged commit 663bd6e into torrust:develop Feb 6, 2024
@josecelano josecelano deleted the 453-load-test branch February 6, 2024 17:45
@josecelano josecelano self-assigned this Feb 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

- System Admin - Enjoyable to Install and Setup our Software Testing Checking Torrust

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

New console command to upload random torrents to the Index: seeder

1 participant