Skip to content

feat(CI): add v2 remote sync verification test job#1499

Merged
TheLastCicada merged 7 commits into
v2-rc2from
feat/v2-remote-sync-tests
Feb 19, 2026
Merged

feat(CI): add v2 remote sync verification test job#1499
TheLastCicada merged 7 commits into
v2-rc2from
feat/v2-remote-sync-tests

Conversation

@TheLastCicada

Copy link
Copy Markdown
Contributor

Summary

  • Adds a new independent parallel CI job test-v2-remote-sync that verifies data synced from a remote participant CADT instance
  • The participant instance (cadt-participant1-testing) has been populated with v2 test data including 3 projects with full entity chains (methodology, program, locations, validations, verifications, issuances, units, estimations, ratings, co-benefits, stakeholders, labels, AEF T1-T5)
  • The test data includes 2 generations: initial creation followed by edits (project description update, unit status changed to Retired) and deletes (1 estimation and 1 co-benefit removed)
  • The sync job imports the remote org, waits for datalayer sync, then verifies exact field matches for all entity types, including that edited fields reflect Gen 2 changes and deleted records are absent
  • No faucet or org creation needed since subscribing to remote stores is free; mirrors are disabled

Data on participant instance

  • Org UID: 3b5608bf3456586dfe6e2353785b93d863db578b2d961bc2e1d142196924c91a
  • 3 projects: V2-GTD001 (Japanese Coastal Blue Carbon), V2-GTD002 (Moroccan Geothermal, edited), V2-GTD003 (Brazilian Agroforestry)
  • 3 units: V2-GTD001-UNIT-001 (Held), V2-GTD002-UNIT-001 (Retired after edit), V2-GTD003-UNIT-001 (Held)
  • 21 entity types verified with exact field matches

Test plan

  • CI job runs in parallel with existing jobs (no needs: dependency)
  • Verify the sync test correctly imports the remote v2 org
  • Verify sync completes and all record counts match
  • Verify exact field values for all entity types including edited and deleted records

Add independent parallel CI job that imports a remote participant's v2
organization, waits for datalayer sync, and verifies all synced data
matches expected values exactly. Tests create, edit, and delete
operations by checking 3 projects, 3 units, and all related entities
(methodology, program, locations, validations, verifications, issuances,
estimations, ratings, co-benefits, stakeholders, labels, AEF T1-T5).
No faucet or org creation needed since subscribing is free.
The sync test job was attempting to import the remote V2 organization
immediately after the CADT health check, but the wallet was still
syncing. Add a wallet sync polling step (up to 20 min) before the
CADT API readiness check and org import.
- Set DATALAYER_FILE_SERVER_URL to null instead of localhost to fully
  prevent mirror creation (AUTO_MIRROR_EXTERNAL_STORES=false alone
  does not prevent mirrors created during store subscription)
- Reorder startup: wait for wallet sync BEFORE starting CADT so
  background tasks don't fail continuously on wallet sync errors
- Pre-subscribe to org store via chia RPC and wait for datalayer to
  sync it before calling the CADT import API
- Add wallet re-sync wait after subscriptions since new DL subs
  cause wallet to re-derive puzzle hashes
- Add retry loop for org import since importOrganization returns
  early (without creating DB record) if the store is not yet synced
The governance node has separate store IDs for V1 and V2:
- V1: 29fe490b... (main store with only a 'v1' key)
- V2: 4212425c... (main store with a 'v2' key)

The sync test was using the V1 governance body ID for the V2 config,
so GovernanceV2.sync() could never find the v2 sub-store and the
orgList/pickList never synced.
The V2 API returns arrays directly, not {data: [...]} objects like V1.
The jq selector '.data[] // .[]' fails on arrays because '.data' on
an array is an error, not null. Use a conditional expression that
handles both formats.
The wallet may report synced=true at the Chia RPC level while still
being unavailable or not fully ready for datalayer store transactions
at the CADT layer. This is especially likely when a real
GOVERNANCE_LOOKUP_ID is configured, since CADT subscribes to the
governance store immediately on startup, triggering additional wallet
activity.

Three-layered fix:
- Add waitForWalletReadyForTransactions helper that polls /v1/health
  for 3 consecutive successes before proceeding
- Call this helper in the upgrade test's before() hook to confirm
  wallet stability before attempting org creation
- Add outer retry loop (3 attempts, 90s settle delay) around the
  entire create + waitForV1OrganizationReady cycle so that a
  PENDING-org-disappeared failure (wallet not fully synced for
  transactions) retries the whole creation rather than failing
- Extend CI wait step to poll CADT /v1/health after chia rpc
  reports synced=true, requiring 3 consecutive 200s before tests run
@TheLastCicada TheLastCicada merged commit ccd2c5e into v2-rc2 Feb 19, 2026
18 of 19 checks passed
@TheLastCicada TheLastCicada deleted the feat/v2-remote-sync-tests branch February 19, 2026 02:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant