Skip to content

CLI based smoke tests are broken but still pass #1392

@SuperFluffy

Description

@SuperFluffy

The smoke-cli tests running CI are broken but still pass CI.

The run-smoke-cli just recipe creates a temp file and passes it to astria-cli, but the latter then refuses to run because a file at that path already exists. Subsequently, the submission step also fails. However, CI ultimately still ends up passing, because the checks are wrong because checks < max_checks ? checks++ (so checks can be at most max_checks), but failure is only established if checks > max_cheecks, which never happens.

Code showing temp file error:

astria/charts/deploy.just

Lines 397 to 410 in 10b196a

withdrawals_dst="$(mktemp)"
# uncomment this line if you want to inspect `withdrawals_dst`
trap "rm -f ${withdrawals_dst@Q}" EXIT
echo "passing ${withdrawals_dst} to astria-cli"
astria-cli bridge collect-withdrawals \
--rollup-endpoint {{eth_ws_url}} \
--contract-address {{evm_contract_address}} \
--from-rollup-height 1 \
--to-rollup-height $CURRENT_BLOCK \
--sequencer-asset-to-withdraw nria \
--bridge-address {{sequencer_bridge_address}} \
--output "${withdrawals_dst}"

Code showing bad success condition error:

astria/charts/deploy.just

Lines 419 to 435 in 10b196a

while [ $CHECKS -lt $MAX_CHECKS ]; do
CHECKS=$((CHECKS+1))
BALANCE=$(astria-cli sequencer account balance astria17w0adeg64ky0daxwd2ugyuneellmjgnxl39504 --sequencer-url {{sequencer_rpc_url}} | awk '/nria/{print $(NF-1)}')
echo "Check $CHECKS, Balance: $BALANCE, Expected: $EXPECTED_BALANCE"
if [ "$BALANCE" == "$EXPECTED_BALANCE" ]; then
echo "Bridge Out Sequencer success"
break
else
sleep 1
fi
done
if [ $CHECKS -gt $MAX_CHECKS ]; then
echo "Bridge Out Sequencer failure"
exit 1
fi
exit 0

Error message:

passing /tmp/tmp.ia1n4HU1IV to astria-cli
  2024-08-22T10:54:28.153755Z ERROR astria_cli::commands::bridge::collect: error: failed to open specified file for writing
    at crates/astria-cli/src/commands/bridge/collect.rs:294
    in astria_cli::commands::bridge::collect::open_output with target: /tmp/tmp.ia1n4HU1IV

failed to open output for writing

Caused by:
   0: failed to open specified file for writing
   1: File exists (os error 17)

Location:
    /home/runner/actions-runner/_work/astria/astria/crates/astria-cli/src/commands/bridge/collect.rs:300:10
failed reading actions from file: /tmp/tmp.ia1n4HU1IV

Caused by:
   0: failed deserializing file contents height-to-sequencer-actions serde object
   1: EOF while parsing a value at line 1 column 0

Location:
    /home/runner/actions-runner/_work/astria/astria/crates/astria-cli/src/commands/bridge/submit.rs:103:10

┆Issue Number: ENG-738

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions