Skip to content

Migration progress in percentage#4725

Merged
clemahieu merged 2 commits intonanocurrency:developfrom
RickiNano:database-migration-percentage
Sep 25, 2024
Merged

Migration progress in percentage#4725
clemahieu merged 2 commits intonanocurrency:developfrom
RickiNano:database-migration-percentage

Conversation

@RickiNano
Copy link
Copy Markdown
Contributor

@RickiNano RickiNano commented Sep 10, 2024

Each step in the LMDB -> RocksDb migration now shows the progress in percentage completion.
This pr also adds a missing check for rep_weights count on completion.

Here is an example of the output log after a migration of the current live ledger. I have cut the example down to reduce size.

[2024-09-08 13:27:51.448] [ledger] [info] Migrating LMDB database to RocksDB. This will take a while...
[2024-09-08 13:27:51.700] [ledger] [info] Step 1 of 7: Converting 201939233 entries from blocks table
[2024-09-08 13:28:06.071] [ledger] [info] 5000000 blocks converted (2%)
[2024-09-08 13:28:26.194] [ledger] [info] 10000000 blocks converted (4%)
[2024-09-08 13:28:43.688] [ledger] [info] 15000000 blocks converted (7%)
[2024-09-08 13:29:36.411] [ledger] [info] 20000000 blocks converted (9%)
[2024-09-08 13:31:02.810] [ledger] [info] 25000000 blocks converted (12%)
[2024-09-08 13:31:55.081] [ledger] [info] 30000000 blocks converted (14%)
............
[2024-09-08 14:05:12.277] [ledger] [info] 185000000 blocks converted (91%)
[2024-09-08 14:06:40.593] [ledger] [info] 190000000 blocks converted (94%)
[2024-09-08 14:07:43.115] [ledger] [info] 195000000 blocks converted (96%)
[2024-09-08 14:08:34.332] [ledger] [info] 200000000 blocks converted (99%)
[2024-09-08 14:09:15.069] [ledger] [info] 201939233 entries converted (100%)
[2024-09-08 14:09:15.071] [ledger] [info] Step 2 of 7: Converting 27226087 entries from pending table
[2024-09-08 14:09:15.754] [ledger] [info] 500000 entries converted (1%)
[2024-09-08 14:09:16.513] [ledger] [info] 1000000 entries converted (3%)
[2024-09-08 14:09:17.422] [ledger] [info] 1500000 entries converted (5%)
[2024-09-08 14:09:18.744] [ledger] [info] 2000000 entries converted (7%)
[2024-09-08 14:09:34.113] [ledger] [info] 2500000 entries converted (9%)
[2024-09-08 14:09:35.856] [ledger] [info] 3000000 entries converted (11%)
............
[2024-09-08 14:10:54.568] [ledger] [info] 24500000 entries converted (89%)
[2024-09-08 14:10:57.789] [ledger] [info] 25000000 entries converted (91%)
[2024-09-08 14:11:02.119] [ledger] [info] 25500000 entries converted (93%)
[2024-09-08 14:11:05.754] [ledger] [info] 26000000 entries converted (95%)
[2024-09-08 14:11:11.586] [ledger] [info] 26500000 entries converted (97%)
[2024-09-08 14:11:13.163] [ledger] [info] 27000000 entries converted (99%)
[2024-09-08 14:11:14.047] [ledger] [info] 27226087 entries converted (100%)
[2024-09-08 14:11:14.047] [ledger] [info] Step 3 of 7: Converting 36913275 entries from confirmation_height table
[2024-09-08 14:11:14.656] [ledger] [info] 500000 entries converted (1%)
[2024-09-08 14:11:15.168] [ledger] [info] 1000000 entries converted (2%)
[2024-09-08 14:11:15.630] [ledger] [info] 1500000 entries converted (4%)
[2024-09-08 14:11:16.093] [ledger] [info] 2000000 entries converted (5%)
[2024-09-08 14:11:16.632] [ledger] [info] 2500000 entries converted (6%)
............
[2024-09-08 14:12:58.828] [ledger] [info] 35000000 entries converted (94%)
[2024-09-08 14:12:59.383] [ledger] [info] 35500000 entries converted (96%)
[2024-09-08 14:12:59.678] [ledger] [info] 36000000 entries converted (97%)
[2024-09-08 14:13:15.473] [ledger] [info] 36500000 entries converted (98%)
[2024-09-08 14:13:17.734] [ledger] [info] 36913275 entries converted (100%)
[2024-09-08 14:13:17.734] [ledger] [info] Step 4 of 7: Converting 36913275 entries from accounts table
[2024-09-08 14:13:18.010] [ledger] [info] 500000 entries converted (1%)
[2024-09-08 14:13:18.549] [ledger] [info] 1000000 entries converted (2%)
[2024-09-08 14:13:19.270] [ledger] [info] 1500000 entries converted (4%)
[2024-09-08 14:13:20.118] [ledger] [info] 2000000 entries converted (5%)
[2024-09-08 14:13:20.964] [ledger] [info] 2500000 entries converted (6%)
[2024-09-08 14:13:21.551] [ledger] [info] 3000000 entries converted (8%)
............
[2024-09-08 14:17:30.146] [ledger] [info] 35500000 entries converted (96%)
[2024-09-08 14:17:31.476] [ledger] [info] 36000000 entries converted (97%)
[2024-09-08 14:17:33.273] [ledger] [info] 36500000 entries converted (98%)
[2024-09-08 14:17:34.482] [ledger] [info] 36913275 entries converted (100%)
[2024-09-08 14:17:34.483] [ledger] [info] Step 5 of 7: Converting 5538654 entries from rep_weights table
[2024-09-08 14:17:35.524] [ledger] [info] 500000 entries converted (9%)
[2024-09-08 14:17:36.697] [ledger] [info] 1000000 entries converted (18%)
[2024-09-08 14:17:38.295] [ledger] [info] 1500000 entries converted (27%)
[2024-09-08 14:17:40.180] [ledger] [info] 2000000 entries converted (36%)
[2024-09-08 14:17:41.463] [ledger] [info] 2500000 entries converted (45%)
[2024-09-08 14:17:42.710] [ledger] [info] 3000000 entries converted (54%)
[2024-09-08 14:17:43.672] [ledger] [info] 3500000 entries converted (63%)
[2024-09-08 14:17:46.968] [ledger] [info] 4000000 entries converted (72%)
[2024-09-08 14:17:52.017] [ledger] [info] 4500000 entries converted (81%)
[2024-09-08 14:17:58.306] [ledger] [info] 5000000 entries converted (90%)
[2024-09-08 14:18:01.618] [ledger] [info] 5500000 entries converted (99%)
[2024-09-08 14:18:02.008] [ledger] [info] 5538654 entries converted (100%)
[2024-09-08 14:18:02.008] [ledger] [info] Step 6 of 7: Converting 0 entries from pruned table
[2024-09-08 14:18:02.019] [ledger] [info] 0 entries converted (100%)
[2024-09-08 14:18:02.019] [ledger] [info] Step 7 of 7: Converting 43278268 entries from final_votes table
[2024-09-08 14:18:02.432] [ledger] [info] 500000 entries converted (1%)
[2024-09-08 14:18:03.066] [ledger] [info] 1000000 entries converted (2%)
[2024-09-08 14:18:03.529] [ledger] [info] 1500000 entries converted (3%)
[2024-09-08 14:18:04.145] [ledger] [info] 2000000 entries converted (4%)
[2024-09-08 14:18:04.758] [ledger] [info] 2500000 entries converted (5%)
............
[2024-09-08 14:21:06.247] [ledger] [info] 41000000 entries converted (94%)
[2024-09-08 14:21:19.742] [ledger] [info] 41500000 entries converted (95%)
[2024-09-08 14:21:20.616] [ledger] [info] 42000000 entries converted (97%)
[2024-09-08 14:21:22.728] [ledger] [info] 42500000 entries converted (98%)
[2024-09-08 14:21:23.769] [ledger] [info] 43000000 entries converted (99%)
[2024-09-08 14:21:24.657] [ledger] [info] 43278268 entries converted (100%)
[2024-09-08 14:21:24.657] [ledger] [info] Finalizing migration...
[2024-09-08 14:21:26.886] [ledger] [info] Migration completed. Make sure to enable RocksDb in the config file under [node.rocksdb]
[2024-09-08 14:21:26.886] [ledger] [info] After confirming correct node operation, the data.ldb file can be deleted if no longer required

@clemahieu clemahieu merged commit 0cfb0a7 into nanocurrency:develop Sep 25, 2024
@qwahzi qwahzi added this to the V28 milestone Sep 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@clemahieu clemahieu clemahieu approved these changes

Assignees

No one assigned

Labels

None yet

Projects

Nano Roadmap
Status: Merged / V28.0

Milestone

V28

Development

Successfully merging this pull request may close these issues.

3 participants

@RickiNano @clemahieu @qwahzi