Skip to content

Optimize PR test workflow: Parallelize smoke and sequential tests#31

Merged
oskarszoon merged 1 commit into
bsv-blockchain:mainfrom
oskarszoon:feature/long-tests
Oct 21, 2025
Merged

Optimize PR test workflow: Parallelize smoke and sequential tests#31
oskarszoon merged 1 commit into
bsv-blockchain:mainfrom
oskarszoon:feature/long-tests

Conversation

@oskarszoon

Copy link
Copy Markdown
Contributor

Summary

This PR optimizes the PR smoke test workflow by splitting smoke tests and sequential tests into parallel jobs, reducing total test time from ~18.5 minutes to ~5-6 minutes (70% faster).

Changes

  1. Split workflow into 4 parallel jobs:

    • smoketest: Basic functionality checks (~2.5 min)
    • sequential-sqlite: Sequential tests with SQLite backend (~5 min)
    • sequential-postgres: Sequential tests with Postgres backend (~5 min)
    • sequential-aerospike: Sequential tests with Aerospike backend (~5 min)
  2. Enhanced test/scripts/run_tests_sequentially.sh:

    • Added --db parameter to filter tests by database backend
    • Enables running tests for specific databases (sqlite, postgres, aerospike)
    • Case-insensitive matching for test function names
  3. New Makefile targets:

    • sequentialtest-sqlite: Run only SQLite sequential tests
    • sequentialtest-postgres: Run only Postgres sequential tests
    • sequentialtest-aerospike: Run only Aerospike sequential tests

Performance Impact

Before:

  • Smoke tests: 2.5 minutes (serial)
  • Sequential tests: 16 minutes (serial, all 3 DBs)
  • Total: 18.5 minutes

After:

  • All 4 jobs run in parallel
  • Total: ~5-6 minutes (duration of longest job)
  • Improvement: 70% faster 🚀

Additional Benefits

  • Faster feedback: Smoke test failures visible in 2.5 min instead of 18.5 min
  • Better debugging: Separate logs per database backend
  • Clearer failures: Easy to identify which DB backend has issues
  • Better resource utilization: Parallel execution on teranode-runner

Test Plan

  • ✅ Script syntax validated
  • ✅ Makefile targets verified
  • ✅ YAML workflow syntax validated
  • ✅ Filter logic tested with all database variants
  • 🔄 Will be validated by this PR's workflow run

Related Issue

Addresses slow PR test workflow execution times.

@oskarszoon

Copy link
Copy Markdown
Contributor Author

/claude-review

@claude

claude Bot commented Oct 21, 2025

Copy link
Copy Markdown
Contributor

Claude finished @oskarszoon's task —— View job


Code Review Complete ✅

This PR successfully optimizes the PR smoke test workflow with a clean parallelization strategy. The changes are well-structured and should achieve the claimed 70% reduction in test execution time.

Strengths

Workflow Design (.github/workflows/teranode_pr_smoketests.yaml:17-153)

  • Clean separation into 4 parallel jobs with no dependencies
  • Proper artifact upload with if: always() ensures results are captured even on failure
  • Consistent structure across all jobs reduces maintenance burden
  • Docker Hub login conditional correctly handles fork PRs

Shell Script Enhancement (test/scripts/run_tests_sequentially.sh:6-23)

  • Case-insensitive matching (tr '[:upper:]' '[:lower:]') is robust for test name variations
  • Substring matching (=~) correctly handles test names like TestLongestChainSQLite
  • Error handling with set -euo pipefail prevents silent failures
  • Clear usage message for invalid options

Makefile Targets (Makefile:166-179)

  • New targets follow existing conventions
  • Proper output directory creation and log capture
  • Consistent naming pattern (sequentialtest-<db>)

Minor Observations

Resource Utilization: Since all 4 jobs run on teranode-runner, ensure the runner has sufficient resources (CPU/memory/disk) to handle:

  • 4 parallel Go test processes
  • Multiple TestContainers (Postgres, Aerospike) launched simultaneously
  • Docker image pulls

If runner capacity is limited, consider adding a max-parallel limit at the workflow level to prevent resource exhaustion.

Future Improvement: The script filters tests by substring matching on function names (e.g., sqlite, postgres, aerospike). This works well with the current naming convention (TestDoubleSpendSQLite), but consider documenting this convention to prevent future tests from accidentally being skipped if they don't follow the pattern.

Recommendation

Approved - This is a solid optimization that delivers significant value with minimal risk. The implementation is clean, follows best practices, and the parallelization approach is sound given that each test suite uses isolated containers.


View Job RunView Branch

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - Clean parallelization strategy that achieves significant performance improvement with minimal risk.

@sonarqubecloud

Copy link
Copy Markdown

@oskarszoon oskarszoon merged commit e7c795e into bsv-blockchain:main Oct 21, 2025
8 checks passed
@oskarszoon oskarszoon self-assigned this Oct 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants