feat(CI): add v2 remote sync verification test job#1499
Merged
Conversation
Add independent parallel CI job that imports a remote participant's v2 organization, waits for datalayer sync, and verifies all synced data matches expected values exactly. Tests create, edit, and delete operations by checking 3 projects, 3 units, and all related entities (methodology, program, locations, validations, verifications, issuances, estimations, ratings, co-benefits, stakeholders, labels, AEF T1-T5). No faucet or org creation needed since subscribing is free.
The sync test job was attempting to import the remote V2 organization immediately after the CADT health check, but the wallet was still syncing. Add a wallet sync polling step (up to 20 min) before the CADT API readiness check and org import.
- Set DATALAYER_FILE_SERVER_URL to null instead of localhost to fully prevent mirror creation (AUTO_MIRROR_EXTERNAL_STORES=false alone does not prevent mirrors created during store subscription) - Reorder startup: wait for wallet sync BEFORE starting CADT so background tasks don't fail continuously on wallet sync errors - Pre-subscribe to org store via chia RPC and wait for datalayer to sync it before calling the CADT import API - Add wallet re-sync wait after subscriptions since new DL subs cause wallet to re-derive puzzle hashes - Add retry loop for org import since importOrganization returns early (without creating DB record) if the store is not yet synced
The governance node has separate store IDs for V1 and V2: - V1: 29fe490b... (main store with only a 'v1' key) - V2: 4212425c... (main store with a 'v2' key) The sync test was using the V1 governance body ID for the V2 config, so GovernanceV2.sync() could never find the v2 sub-store and the orgList/pickList never synced.
The V2 API returns arrays directly, not {data: [...]} objects like V1.
The jq selector '.data[] // .[]' fails on arrays because '.data' on
an array is an error, not null. Use a conditional expression that
handles both formats.
The wallet may report synced=true at the Chia RPC level while still being unavailable or not fully ready for datalayer store transactions at the CADT layer. This is especially likely when a real GOVERNANCE_LOOKUP_ID is configured, since CADT subscribes to the governance store immediately on startup, triggering additional wallet activity. Three-layered fix: - Add waitForWalletReadyForTransactions helper that polls /v1/health for 3 consecutive successes before proceeding - Call this helper in the upgrade test's before() hook to confirm wallet stability before attempting org creation - Add outer retry loop (3 attempts, 90s settle delay) around the entire create + waitForV1OrganizationReady cycle so that a PENDING-org-disappeared failure (wallet not fully synced for transactions) retries the whole creation rather than failing - Extend CI wait step to poll CADT /v1/health after chia rpc reports synced=true, requiring 3 consecutive 200s before tests run
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
test-v2-remote-syncthat verifies data synced from a remote participant CADT instancecadt-participant1-testing) has been populated with v2 test data including 3 projects with full entity chains (methodology, program, locations, validations, verifications, issuances, units, estimations, ratings, co-benefits, stakeholders, labels, AEF T1-T5)Data on participant instance
3b5608bf3456586dfe6e2353785b93d863db578b2d961bc2e1d142196924c91aTest plan
needs:dependency)