ci: add mainnet sync test (first 1000 blocks)#1460
Merged
Conversation
Adds +sync-mainnet-1000 / +sync-mainnet-1000-snapshot earthly targets that verify a fresh midnight-node can sync the first 1000 blocks of Midnight Mainnet against a captive postgres preloaded with a minimal cardano-db-sync snapshot. The snapshot generator (scripts/sync-test/build-snapshot.sh) runs against a host-local db-sync and dumps just enough rows for the cnight-observation, committee-selection, federated-authority and bridge data sources to return identical results to the producer's full db-sync over the cardano window that Midnight blocks 1..1000 reference. Server-side TEMP TABLEs materialise the consumed-in-window NIGHT producer set once, then every COPY filters off it -- without that, the cnight inherent's byte-exact equality check fails on block #1 and best stays at #0 forever. The runner (scripts/sync-test/run-sync.sh) spins up a postgres:17.4-alpine container with self-signed TLS (matching midnight-node's PgSslMode::Require), loads the snapshot, runs midnight-node with CFG_PRESET=mainnet, and tails its logs until 'best: #N >= SYNC_UNTIL' (default 1000) or it stalls. Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
The cnight observation data source runs four parallel cardano queries (get_registrations / get_deregistrations / get_asset_creates / get_asset_spends). Each spawns parallel workers that allocate shared memory segments via dsm_impl_posix. Docker's default /dev/shm is 64MB, which is too small even for one worker pair on the ma_tx_out scan, so the queries fail with 'could not resize shared memory segment ... No space left on device' and block import for Midnight block #1. Bump --shm-size=1g on the postgres receiver. Also gitignore the snapshot file produced by build-snapshot.sh so it's not accidentally committed. Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
…lder Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
… -> 21MB Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
ba03cb9 to
56783ca
Compare
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
gilescope
approved these changes
May 6, 2026
gilescope
left a comment
Contributor
There was a problem hiding this comment.
An integration test that is fast!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Adds a sync-test for the first 1000 blocks of mainnet. Uses a snapshot of Cardano mainnet containing only the mainnet data we need (6.8 MB)
Added:
Example: https://github.com/midnightntwrk/midnight-node/actions/runs/25375174559/job/74411117476?pr=1460
🗹 TODO before merging
📌 Submission Checklist
git commit -s) for the DCO🧪 Testing Evidence
Please describe any additional testing aside from CI:
🔱 Fork Strategy
Links