Skip to content

optimize ci workflow#3755

Merged
jolestar merged 4 commits into
mainfrom
optimize_ci
Nov 19, 2025
Merged

optimize ci workflow#3755
jolestar merged 4 commits into
mainfrom
optimize_ci

Conversation

@jolestar

Copy link
Copy Markdown
Contributor

Summary

Summary about this PR

  • Closes #issue

@vercel

vercel Bot commented Nov 18, 2025

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
rooch-portal-v2.1 Ready Ready Preview Comment Nov 19, 2025 0:36am
test-portal Ready Ready Preview Comment Nov 19, 2025 0:36am
1 Skipped Deployment
Project Deployment Preview Comments Updated (UTC)
rooch Ignored Ignored Preview Nov 19, 2025 0:36am

@github-actions

github-actions Bot commented Nov 18, 2025

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

OpenSSF Scorecard

PackageVersionScoreDetails
actions/Swatinem/rust-cache 2.*.* 🟢 6.9
Details
CheckScoreReason
Code-Review🟢 7Found 5/7 approved changesets -- score normalized to 7
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Packaging⚠️ -1packaging workflow not detected
Maintained🟢 1024 commit(s) and 2 issue activity found in the last 90 days -- score normalized to 10
Binary-Artifacts🟢 10no binaries found in the repo
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Token-Permissions🟢 10GitHub workflow tokens follow principle of least privilege
Security-Policy⚠️ 0security policy file not detected
Fuzzing⚠️ 0project is not fuzzed
License🟢 10license file detected
Pinned-Dependencies🟢 10all dependencies are pinned
Signed-Releases⚠️ -1no releases found
Branch-Protection⚠️ 0branch protection not enabled on development/release branches
Vulnerabilities🟢 91 existing vulnerabilities detected
SAST🟢 6SAST tool is not run on all commits -- score normalized to 6
actions/actions/cache 4.*.* 🟢 6.7
Details
CheckScoreReason
Code-Review🟢 10all changesets reviewed
Maintained🟢 43 commit(s) and 2 issue activity found in the last 90 days -- score normalized to 4
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Packaging⚠️ -1packaging workflow not detected
Binary-Artifacts🟢 10no binaries found in the repo
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
License🟢 10license file detected
Pinned-Dependencies🟢 3dependency not pinned by hash detected -- score normalized to 3
Fuzzing⚠️ 0project is not fuzzed
Signed-Releases⚠️ -1no releases found
Security-Policy🟢 9security policy file detected
Branch-Protection⚠️ -1internal error: error during GetBranch(releases/v1): error during branchesHandler.query: internal error: githubv4.Query: Resource not accessible by integration
SAST🟢 10SAST tool is run on all commits
Vulnerabilities🟢 91 existing vulnerabilities detected
actions/actions/checkout 4.*.* 🟢 6.2
Details
CheckScoreReason
Maintained⚠️ 12 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 1
Code-Review🟢 10all changesets reviewed
Binary-Artifacts🟢 10no binaries found in the repo
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Fuzzing⚠️ 0project is not fuzzed
License🟢 10license file detected
Signed-Releases⚠️ -1no releases found
Packaging⚠️ -1packaging workflow not detected
Security-Policy🟢 9security policy file detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Branch-Protection⚠️ -1internal error: error during GetBranch(releases/v5): error during branchesHandler.query: internal error: githubv4.Query: Resource not accessible by integration
Pinned-Dependencies🟢 3dependency not pinned by hash detected -- score normalized to 3
Vulnerabilities🟢 91 existing vulnerabilities detected
SAST🟢 8SAST tool detected but not run on all commits
actions/dtolnay/rust-toolchain stable 🟢 6.1
Details
CheckScoreReason
Maintained🟢 1015 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 10
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Binary-Artifacts🟢 10no binaries found in the repo
Packaging⚠️ -1packaging workflow not detected
Code-Review⚠️ 0Found 1/17 approved changesets -- score normalized to 0
Token-Permissions🟢 10GitHub workflow tokens follow principle of least privilege
Pinned-Dependencies⚠️ 0dependency not pinned by hash detected -- score normalized to 0
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Vulnerabilities🟢 100 existing vulnerabilities detected
Fuzzing⚠️ 0project is not fuzzed
License🟢 10license file detected
Signed-Releases⚠️ -1no releases found
Security-Policy🟢 3security policy file detected
Branch-Protection⚠️ -1internal error: error during branchesHandler.setup: internal error: some github tokens can't read classic branch protection rules: https://github.com/ossf/scorecard-action/blob/main/docs/authentication/fine-grained-auth-token.md
SAST⚠️ 0SAST tool is not run on all commits -- score normalized to 0
actions/actions/checkout 4.*.* 🟢 6.2
Details
CheckScoreReason
Maintained⚠️ 12 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 1
Code-Review🟢 10all changesets reviewed
Binary-Artifacts🟢 10no binaries found in the repo
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Fuzzing⚠️ 0project is not fuzzed
License🟢 10license file detected
Signed-Releases⚠️ -1no releases found
Packaging⚠️ -1packaging workflow not detected
Security-Policy🟢 9security policy file detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Branch-Protection⚠️ -1internal error: error during GetBranch(releases/v5): error during branchesHandler.query: internal error: githubv4.Query: Resource not accessible by integration
Pinned-Dependencies🟢 3dependency not pinned by hash detected -- score normalized to 3
Vulnerabilities🟢 91 existing vulnerabilities detected
SAST🟢 8SAST tool detected but not run on all commits
actions/dorny/paths-filter 3.*.* 🟢 3
Details
CheckScoreReason
Maintained⚠️ 00 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0
Binary-Artifacts🟢 10no binaries found in the repo
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Packaging⚠️ -1packaging workflow not detected
Code-Review⚠️ 2Found 5/18 approved changesets -- score normalized to 2
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Pinned-Dependencies⚠️ 0dependency not pinned by hash detected -- score normalized to 0
License🟢 10license file detected
Fuzzing⚠️ 0project is not fuzzed
Security-Policy⚠️ 0security policy file not detected
Signed-Releases⚠️ -1no releases found
Branch-Protection⚠️ -1internal error: error during branchesHandler.setup: internal error: some github tokens can't read classic branch protection rules: https://github.com/ossf/scorecard-action/blob/main/docs/authentication/fine-grained-auth-token.md
SAST⚠️ 0SAST tool is not run on all commits -- score normalized to 0
Vulnerabilities⚠️ 014 existing vulnerabilities detected

Scanned Files

  • .github/workflows/check_build_test.yml
  • .github/workflows/check_licenses.yml
  • .github/workflows/check_move_constant_errors.yml
  • .github/workflows/cross_platform_check.yml
  • .github/workflows/linux_macos_release.yml
  • .github/workflows/quick_checks.yml
  • .github/workflows/windows_release.yml

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR optimizes the CI workflow infrastructure by consolidating and reorganizing GitHub Actions workflows to improve efficiency and reduce redundancy.

  • Consolidates standalone check workflows (licenses, Move constant errors) into a unified quick_checks.yml workflow
  • Removes separate platform-specific release workflows in favor of a new cross_platform_check.yml for build validation
  • Optimizes the main build/test workflow to use pre-built binaries and better caching strategies
  • Adds label-based conditional execution for Docker builds on pull requests

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
.github/workflows/windows_release.yml Removed - Windows release workflow deleted as part of consolidation
.github/workflows/linux_macos_release.yml Removed - Linux/macOS release workflow deleted as part of consolidation
.github/workflows/check_move_constant_errors.yml Removed - Consolidated into quick_checks.yml
.github/workflows/check_licenses.yml Removed - Consolidated into quick_checks.yml
.github/workflows/quick_checks.yml New workflow combining license checks, formatting, and Move constant error validation with conditional execution based on file changes
.github/workflows/cross_platform_check.yml New workflow for cross-platform build validation across Linux, macOS, and Windows
.github/workflows/docker_build.yml Added label-based conditional execution to allow PR-triggered Docker builds only when labeled
.github/workflows/check_build_test.yml Optimized to use optci profile builds, Rust cache, and pre-built binaries instead of rebuilding
.github/workflows/cancel.yml Updated comments to note missing workflow IDs for new workflows

Comment thread .github/workflows/check_build_test.yml
Comment thread .github/workflows/cancel.yml
Comment thread .github/workflows/cross_platform_check.yml
Comment thread .github/workflows/check_build_test.yml
Comment thread .github/workflows/cross_platform_check.yml
Comment thread .github/workflows/check_build_test.yml Outdated
Comment thread .github/workflows/docker_build.yml Outdated
Comment thread .github/workflows/docker_build.yml Outdated
Comment thread .github/workflows/check_build_test.yml Outdated
Comment thread .github/workflows/quick_checks.yml Outdated
@jolestar jolestar merged commit a36fc7b into main Nov 19, 2025
19 checks passed
@jolestar jolestar deleted the optimize_ci branch November 19, 2025 13:59
jolestar added a commit that referenced this pull request Nov 21, 2025
This commit addresses the SDK test timeout issues caused by port conflicts
and incomplete process cleanup.

## Problem
- Web SDK tests frequently timeout when trying to start Rooch server
- Port 6767/6768 conflicts when multiple tests run or old processes remain
- No timeout mechanism for server startup, leading to indefinite hangs
- Process cleanup was incomplete, leaving zombie processes

## Solution
1. **Dynamic Port Allocation**
   - Added isPortAvailable() function to check port availability
   - Modified loadRoochEnv() to check if requested port is available
   - Falls back to dynamic port (0) if requested port is occupied
   - Ensures each test gets a unique port

2. **Timeout Mechanism**
   - Added 30-second timeout to roochAsyncCommand()
   - Provides clear error messages when startup fails
   - Includes captured output in error for debugging

3. **Improved Process Cleanup**
   - Enhanced cleanEnv() to use SIGTERM first, then SIGKILL
   - Added fallback port-based cleanup using lsof
   - Cross-platform support (macOS/Linux/Windows)
   - Better logging for troubleshooting

4. **Better Logging**
   - Added debug logs for port allocation and process lifecycle
   - Logs PID and port for each server instance
   - Helps diagnose issues in CI environment

## Testing
- Verified multiple servers can start concurrently without conflicts
- Confirmed process cleanup properly frees ports
- Tested timeout mechanism with unavailable ports

Fixes #3755
jolestar added a commit that referenced this pull request Nov 22, 2025
This commit addresses the SDK test timeout issues caused by port conflicts
and incomplete process cleanup.

## Problem
- Web SDK tests frequently timeout when trying to start Rooch server
- Port 6767/6768 conflicts when multiple tests run or old processes remain
- No timeout mechanism for server startup, leading to indefinite hangs
- Process cleanup was incomplete, leaving zombie processes

## Solution
1. **Dynamic Port Allocation**
   - Added isPortAvailable() function to check port availability
   - Modified loadRoochEnv() to check if requested port is available
   - Falls back to dynamic port (0) if requested port is occupied
   - Ensures each test gets a unique port

2. **Timeout Mechanism**
   - Added 30-second timeout to roochAsyncCommand()
   - Provides clear error messages when startup fails
   - Includes captured output in error for debugging

3. **Improved Process Cleanup**
   - Enhanced cleanEnv() to use SIGTERM first, then SIGKILL
   - Added fallback port-based cleanup using lsof
   - Cross-platform support (macOS/Linux/Windows)
   - Better logging for troubleshooting

4. **Better Logging**
   - Added debug logs for port allocation and process lifecycle
   - Logs PID and port for each server instance
   - Helps diagnose issues in CI environment

## Testing
- Verified multiple servers can start concurrently without conflicts
- Confirmed process cleanup properly frees ports
- Tested timeout mechanism with unavailable ports

Fixes #3755
jolestar added a commit that referenced this pull request Nov 22, 2025
…e tests

## Problem
- Previous approach used hardcoded ports (6767) causing conflicts
- Manual server management via npm scripts was error-prone
- Tests couldn't run concurrently due to port conflicts
- Platform-specific cleanup commands (lsof/netstat)

## Solution: Global Setup with Dynamic Port Allocation

### 1. Global Setup (globalSetup.ts)
- Starts single shared Rooch server for all e2e tests
- Uses dynamic port allocation (0 = auto-assign)
- Sets VITE_FULLNODE_URL environment variable
- Automatic cleanup via Vitest teardown

### 2. Simplified npm Scripts
- Removed: prepare:e2e, stop:e2e, wait-on dependency
- Now: just 'vitest run e2e'
- No manual server management needed

### 3. TestBox Integration
- Each test's TestBox now configures rooch CLI to use global server
- Fixed cmdPublishPackage to replace 'default' with actual address
- Improved JSON parsing to handle mixed output (logs + JSON)
- Enhanced error handling with detailed logging

### 4. Better Error Handling
- Capture both stdout and stderr from execSync
- Robust JSON extraction from mixed output
- Clear error messages for debugging

## Benefits
✅ No port conflicts (dynamic allocation)
✅ Faster tests (shared server, no repeated startup)
✅ Cross-platform (no lsof/netstat needed)
✅ Simpler scripts (3 files changed, 4 scripts removed)
✅ Better reliability (automatic cleanup)
✅ CI compatible (no workflow changes needed)

## Test Results
- All 13 test files pass (32 tests)
- Duration: ~89s
- Dynamic port confirmed: http://127.0.0.1:64271

Fixes #3755
jolestar added a commit that referenced this pull request Nov 22, 2025
…3794)

* fix(sdk): improve web SDK test port management and process cleanup

This commit addresses the SDK test timeout issues caused by port conflicts
and incomplete process cleanup.

## Problem
- Web SDK tests frequently timeout when trying to start Rooch server
- Port 6767/6768 conflicts when multiple tests run or old processes remain
- No timeout mechanism for server startup, leading to indefinite hangs
- Process cleanup was incomplete, leaving zombie processes

## Solution
1. **Dynamic Port Allocation**
   - Added isPortAvailable() function to check port availability
   - Modified loadRoochEnv() to check if requested port is available
   - Falls back to dynamic port (0) if requested port is occupied
   - Ensures each test gets a unique port

2. **Timeout Mechanism**
   - Added 30-second timeout to roochAsyncCommand()
   - Provides clear error messages when startup fails
   - Includes captured output in error for debugging

3. **Improved Process Cleanup**
   - Enhanced cleanEnv() to use SIGTERM first, then SIGKILL
   - Added fallback port-based cleanup using lsof
   - Cross-platform support (macOS/Linux/Windows)
   - Better logging for troubleshooting

4. **Better Logging**
   - Added debug logs for port allocation and process lifecycle
   - Logs PID and port for each server instance
   - Helps diagnose issues in CI environment

## Testing
- Verified multiple servers can start concurrently without conflicts
- Confirmed process cleanup properly frees ports
- Tested timeout mechanism with unavailable ports

Fixes #3755

* fix(sdk): address code review comments

- Fix Windows batch command syntax: use %%a instead of %a
- Make cleanup synchronous to avoid race conditions
- Remove redundant execSync require (use top-level import)
- Simplify cleanup logic by removing async setTimeout

Changes based on Copilot review feedback.

* feat(sdk): implement global setup with dynamic port allocation for e2e tests

## Problem
- Previous approach used hardcoded ports (6767) causing conflicts
- Manual server management via npm scripts was error-prone
- Tests couldn't run concurrently due to port conflicts
- Platform-specific cleanup commands (lsof/netstat)

## Solution: Global Setup with Dynamic Port Allocation

### 1. Global Setup (globalSetup.ts)
- Starts single shared Rooch server for all e2e tests
- Uses dynamic port allocation (0 = auto-assign)
- Sets VITE_FULLNODE_URL environment variable
- Automatic cleanup via Vitest teardown

### 2. Simplified npm Scripts
- Removed: prepare:e2e, stop:e2e, wait-on dependency
- Now: just 'vitest run e2e'
- No manual server management needed

### 3. TestBox Integration
- Each test's TestBox now configures rooch CLI to use global server
- Fixed cmdPublishPackage to replace 'default' with actual address
- Improved JSON parsing to handle mixed output (logs + JSON)
- Enhanced error handling with detailed logging

### 4. Better Error Handling
- Capture both stdout and stderr from execSync
- Robust JSON extraction from mixed output
- Clear error messages for debugging

## Benefits
✅ No port conflicts (dynamic allocation)
✅ Faster tests (shared server, no repeated startup)
✅ Cross-platform (no lsof/netstat needed)
✅ Simpler scripts (3 files changed, 4 scripts removed)
✅ Better reliability (automatic cleanup)
✅ CI compatible (no workflow changes needed)

## Test Results
- All 13 test files pass (32 tests)
- Duration: ~89s
- Dynamic port confirmed: http://127.0.0.1:64271

Fixes #3755

* fix(sdk): remove TOCTOU race condition in port availability check

## Problem
The isPortAvailable() function had a Time-of-Check-Time-of-Use (TOCTOU)
race condition:

1. Check if port is available ✓
2. Close test server, releasing the port ⚠️
3. **Another process could steal the port here** 💥
4. Try to start Rooch server on that port ✗

This could cause:
- Server startup failures
- 30-second timeout instead of fast failure
- Flaky tests in concurrent environments

## Solution
Remove the isPortAvailable() check entirely and use a simpler strategy:

- For port 0, 6767, or 6768: Always use dynamic allocation
- For other ports: Trust the caller, let server fail fast if unavailable

Benefits:
✅ No race condition
✅ Simpler code (removed isPortAvailable function)
✅ Faster failure (server bind error vs 30s timeout)
✅ Common ports (6767/6768) always use dynamic allocation

## Trade-offs
- Custom ports (not 0/6767/6768) won't be pre-checked
- Server will fail at bind time instead of during port check
- This is actually better: fail-fast with clear error message

* refactor(sdk): remove unnecessary address replacement in cmdPublishPackage

## Reason
The rooch CLI already supports 'default' keyword in named addresses:
--named-addresses alice=0x1234,bob=default

The 'default' keyword is automatically resolved to the active account
address by the rooch command itself.

## Changes
- Removed manual address replacement logic (await defaultCmdAddress())
- Simplified cmdPublishPackage to directly use options.namedAddresses
- Added comment explaining that rooch CLI handles 'default' keyword

## Benefits
✅ Simpler code (removed 2 lines)
✅ Faster execution (no async address lookup)
✅ Consistent with rooch CLI behavior
✅ Less error-prone (let CLI handle address resolution)

## Test Results
All 13 test files pass (32 tests) - Duration: 89.66s

* security(sdk): use execFileSync instead of execSync to prevent shell injection

## Security Issue
Using execSync with shell: true can lead to command injection vulnerabilities
if user input is not properly sanitized.

## Solution
Refactored buildRoochCommand to return structured command information:
- cmd: executable path
- args: array of arguments
- env: environment variables

Updated roochCommand to use execFileSync instead of execSync:
- execFileSync doesn't use shell, preventing injection attacks
- Arguments are passed as array, not concatenated strings
- More secure and explicit command execution

Updated roochAsyncCommand to use spawn without shell:
- spawn(cmd, args, { env }) instead of spawn(command, { shell: true })
- Same security benefits as execFileSync

## Benefits
✅ Prevents shell injection vulnerabilities
✅ More explicit argument handling
✅ Proper environment variable passing
✅ Cleaner separation of concerns

## Test Results
All 13 test files pass (32 tests) - Duration: 90.03s

Addresses GitHub Advanced Security code scanning alert

* fixup
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants