IMPORT currently triggers an async INSPECT job after completion. That job runs a general index consistency check between primary and secondary indexes. While that validation is useful, what we really want is a check that verifies the completeness of the import -- specifically, that no rows were missed.
Instead of the existing index consistency check, IMPORT should trigger a new INSPECT check that compares the imported table’s row count against the expected count. This approach works because the table is offline during IMPORT, and the INSPECT job runs with an AS OF timestamp taken from before the table is made public again.
Implementation notes
- No new SQL is required, since IMPORT starts the INSPECT job directly.
- Add a new check type to the INSPECT job spec and invoke it from IMPORT.
- Ensure an AS OF timestamp is always set when this check runs.
- Add a new INSPECT error type for row count mismatches.
Jira issue: CRDB-55448
Epic CRDB-58692
IMPORT currently triggers an async INSPECT job after completion. That job runs a general index consistency check between primary and secondary indexes. While that validation is useful, what we really want is a check that verifies the completeness of the import -- specifically, that no rows were missed.
Instead of the existing index consistency check, IMPORT should trigger a new INSPECT check that compares the imported table’s row count against the expected count. This approach works because the table is offline during IMPORT, and the INSPECT job runs with an AS OF timestamp taken from before the table is made public again.
Implementation notes
Jira issue: CRDB-55448
Epic CRDB-58692