Adds +sync-mainnet-1000 / +sync-mainnet-1000-snapshot earthly targets that
verify a fresh midnight-node can sync the first 1000 blocks of Midnight
Mainnet against a captive postgres preloaded with a minimal cardano-db-sync
snapshot.

The snapshot generator (scripts/sync-test/build-snapshot.sh) runs against a
host-local db-sync and dumps just enough rows for the cnight-observation,
committee-selection, federated-authority and bridge data sources to return
identical results to the producer's full db-sync over the cardano window
that Midnight blocks 1..1000 reference. Server-side TEMP TABLEs materialise
the consumed-in-window NIGHT producer set once, then every COPY filters off
it -- without that, the cnight inherent's byte-exact equality check fails
on block #1 and best stays at #0 forever.

The runner (scripts/sync-test/run-sync.sh) spins up a postgres:17.4-alpine
container with self-signed TLS (matching midnight-node's PgSslMode::Require),
loads the snapshot, runs midnight-node with CFG_PRESET=mainnet, and tails
its logs until 'best: #N >= SYNC_UNTIL' (default 1000) or it stalls.

Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>

The cnight observation data source runs four parallel cardano queries
(get_registrations / get_deregistrations / get_asset_creates /
get_asset_spends). Each spawns parallel workers that allocate shared
memory segments via dsm_impl_posix. Docker's default /dev/shm is 64MB,
which is too small even for one worker pair on the ma_tx_out scan, so
the queries fail with 'could not resize shared memory segment ... No
space left on device' and block import for Midnight block #1.

Bump --shm-size=1g on the postgres receiver. Also gitignore the
snapshot file produced by build-snapshot.sh so it's not accidentally
committed.

Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>

* feat: add earthly target to sync first 1000 blocks of mainnet

Adds +sync-mainnet-1000 / +sync-mainnet-1000-snapshot earthly targets that
verify a fresh midnight-node can sync the first 1000 blocks of Midnight
Mainnet against a captive postgres preloaded with a minimal cardano-db-sync
snapshot.

The snapshot generator (scripts/sync-test/build-snapshot.sh) runs against a
host-local db-sync and dumps just enough rows for the cnight-observation,
committee-selection, federated-authority and bridge data sources to return
identical results to the producer's full db-sync over the cardano window
that Midnight blocks 1..1000 reference. Server-side TEMP TABLEs materialise
the consumed-in-window NIGHT producer set once, then every COPY filters off
it -- without that, the cnight inherent's byte-exact equality check fails
on block midnightntwrk#1 and best stays at #0 forever.

The runner (scripts/sync-test/run-sync.sh) spins up a postgres:17.4-alpine
container with self-signed TLS (matching midnight-node's PgSslMode::Require),
loads the snapshot, runs midnight-node with CFG_PRESET=mainnet, and tails
its logs until 'best: #N >= SYNC_UNTIL' (default 1000) or it stalls.

Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>

* fix(sync-test): postgres needs --shm-size=1g for parallel cnight queries

The cnight observation data source runs four parallel cardano queries
(get_registrations / get_deregistrations / get_asset_creates /
get_asset_spends). Each spawns parallel workers that allocate shared
memory segments via dsm_impl_posix. Docker's default /dev/shm is 64MB,
which is too small even for one worker pair on the ma_tx_out scan, so
the queries fail with 'could not resize shared memory segment ... No
space left on device' and block import for Midnight block midnightntwrk#1.

Bump --shm-size=1g on the postgres receiver. Also gitignore the
snapshot file produced by build-snapshot.sh so it's not accidentally
committed.

Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>

* chore: fix comments + postgres URL for snapshot building

Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>

* docs: add comments describing each address and policy in snapshot builder

Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>

* build: save snapshot into static/ after building

Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>

* feat: tighten min/max cardano blocks in snapshot - size reduced, 36MB -> 21MB

Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>

* feat: restrict snapshot data to known addresses: 21MB -> 7.1MB

Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>

* feat: reduce snapshot size by 200MB by nulling unused block data

Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>

* chore: commit snapshot

Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>

* ci: add mainnet sync test to CI

Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>

* ci: print logs for mainnet sync

Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>

---------

Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>

Run the local-env stack + verify-finality inside earthly's nested daemon and
confirm all 5 validators finalize block #1, reaching them via in-sandbox
localhost. End-to-end ~6.5 min on a cold nested daemon (public-dep cold-pull
included) — cold-pull does not dominate; no pull-cache mitigation needed.

Fixes host-parity gaps the first-cut +local-env-ci was missing (each a real
bring-up failure when absent; the host LOCALLY path gets them from direnv
.envrc + the worktree):
  - WORKDIR /work + COPY res            (midnight-setup mounts ../../../../res/)
  - COPY midnight-reserve-contracts     (contract-compiler mounts the submodule)
  - MIDNIGHT_RESERVE_CONTRACTS_PATH=... (else compose volume spec is :/contracts)
  - ARCHITECTURE=linux/$USERARCH        (compose platform: wants linux/amd64)

Adds +local-env-ci-localimg: a local measurement harness that injects the 4
private images via docker save→load (no GHCR auth needed), used to prove P1 on
a box without registry access; dumps midnight-setup/contract-compiler logs on
bring-up failure.

Assisted-by: Claude:claude-opus-4-8

…etns

Adds +local-env-e2e-ci-localimg: brings the stack up inside earthly's nested
daemon and runs the full e2e suite against it from within the same netns —
26 passed / 0 failed / 1 ignored, all 5 nodes finalized #1.

URL discovery resolved: e2e selects endpoints by cargo feature (config.rs).
In-netns we use --no-default-features --features local → 127.0.0.1:9933/1337
(compose publishes both), so the local-ci 172.17.0.1 docker0-gateway hack is
unnecessary — the test reaches the stack via sandbox-localhost.

Build notes (FROM +prep, Amazon Linux glibc + rust + repo, so cargo test runs
in-place with the correct CARGO_MANIFEST_DIR for runtime-values):
  - earthly 0.8 injects dockerd → WITH DOCKER works off a non-dind base, but it
    injects only the docker CLI, not the compose-v2 plugin (dind:alpine bundles
    it) → install it to /usr/local/lib/docker/cli-plugins/
  - +prep has no node/npm → dnf install nodejs npm
  - same res/ + reserve-contracts submodule + env parity as +local-env-ci

Assisted-by: Claude:claude-opus-4-8

Adds +local-env-full-ci-localimg: stack bring-up → verify-finality → e2e suite
→ toolkit multi-dest E2E, all in ONE nested-dockerd stack (the WITH DOCKER
stack can't persist across separate earthly invocations, so the flip from the
serialized host job must be a single consolidated target). Proven green
locally: 5/5 nodes finalized #1, e2e 26 passed/0 failed/1 ignored, toolkit
no-hang send + verify all pass (~26 min).

This is the local-image proof (docker save→load) of the shape +local-env-ci
will take in CI (registry --pull).

Assisted-by: Claude:claude-opus-4-8