Add a GitHub workflow to Newton against the latest nightly Warp version#1088
Conversation
Signed-off-by: Eric Shi <ershi@nvidia.com>
📝 WalkthroughWalkthroughIntroduces a new GitHub Actions workflow file that implements scheduled nightly testing with Warp language. The workflow detects updates to Warp in the dependency lock file, provisions an EC2 GPU runner if updates are found, executes tests on the provisioned infrastructure, and cleans up the runner. Changes
Sequence DiagramsequenceDiagram
actor Scheduler as GitHub Scheduler
participant Workflow as Workflow Orchestrator
participant CheckJob as check-warp-update
participant StartJob as start-runner
participant TestJob as nightly-warp-tests
participant StopJob as stop-runner
participant AWS as AWS EC2
Scheduler->>Workflow: Trigger scheduled workflow
Workflow->>CheckJob: Run nightly check
CheckJob->>CheckJob: Check uv.lock for Warp update
activate CheckJob
alt Warp Updated
CheckJob->>CheckJob: Set warp-updated=true
else No Update
CheckJob->>CheckJob: Set warp-updated=false
end
deactivate CheckJob
alt warp-updated == true
Workflow->>StartJob: Provision runner
StartJob->>AWS: Launch EC2 GPU instance
AWS->>StartJob: Instance ready
Workflow->>TestJob: Run tests
TestJob->>TestJob: Install dependencies<br/>Run test suite
else No Updates
Note over Workflow: Skip test execution
end
Workflow->>StopJob: Stop runner (always)
StopJob->>AWS: Terminate EC2 instance
AWS->>StopJob: Instance stopped
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes
Possibly related PRs
Suggested labels
Suggested reviewers
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
.github/workflows/scheduled_nightly_warp_tests.yml (2)
122-123: Specify an explicit Python version for reproducibility.The
uv python installstep does not pin a Python version, which could lead to inconsistencies across runs if the system default changes or if the EC2 AMI ships different versions. Explicitly specify a version to match your project's Python requirements.Apply this diff to pin the Python version:
- name: Set up Python - run: uv python install + run: uv python install 3.12Adjust
3.12to match your project's minimum supported Python version.
12-19: Workflow-level environment variables are well-organized.Defining AWS region, instance type, and security settings at the workflow level makes the configuration maintainable and reusable. Consider documenting the
AWS_SECURITY_GROUP_IDSand hardcoded subnet (line 98) in a comment explaining their purpose, since these are environment-specific IDs that may need updating if infrastructure changes.Add a brief comment before the
env:section explaining the AWS resource IDs:# AWS environment variables—update these if infrastructure or security groups change env: AWS_REGION: us-east-2 ...
📜 Review details
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
.github/workflows/scheduled_nightly_warp_tests.yml(1 hunks)
🧰 Additional context used
🧠 Learnings (3)
📓 Common learnings
Learnt from: shi-eric
Repo: newton-physics/newton PR: 879
File: .gitlab-ci.yml:125-145
Timestamp: 2025-10-04T06:44:52.713Z
Learning: In the Newton project, the "linux-x86_64 test warp nightly" GitLab CI job intentionally runs on every pipeline (not limited to `.test_common` rules) to detect Warp nightly integration issues early, since Warp nightly releases are unpredictable. This design is acceptable because `allow_failure: true` prevents blocking the pipeline.
Learnt from: nvlukasz
Repo: newton-physics/newton PR: 519
File: newton/_src/solvers/featherstone/kernels.py:75-75
Timestamp: 2025-08-12T18:04:06.577Z
Learning: The Newton physics framework requires nightly Warp builds, which means compatibility concerns with older stable Warp versions (like missing functions such as wp.spatial_adjoint) are not relevant for this project.
Learnt from: shi-eric
Repo: newton-physics/newton PR: 521
File: newton/examples/example_cloth_hanging.py:36-36
Timestamp: 2025-08-12T05:17:34.423Z
Learning: The Newton migration guide (docs/migration.rst) is specifically for documenting how to migrate existing warp.sim functionality to Newton equivalents. New Newton-only features that didn't exist in warp.sim do not need migration documentation.
📚 Learning: 2025-10-04T06:44:52.713Z
Learnt from: shi-eric
Repo: newton-physics/newton PR: 879
File: .gitlab-ci.yml:125-145
Timestamp: 2025-10-04T06:44:52.713Z
Learning: In the Newton project, the "linux-x86_64 test warp nightly" GitLab CI job intentionally runs on every pipeline (not limited to `.test_common` rules) to detect Warp nightly integration issues early, since Warp nightly releases are unpredictable. This design is acceptable because `allow_failure: true` prevents blocking the pipeline.
Applied to files:
.github/workflows/scheduled_nightly_warp_tests.yml
📚 Learning: 2025-08-12T18:04:06.577Z
Learnt from: nvlukasz
Repo: newton-physics/newton PR: 519
File: newton/_src/solvers/featherstone/kernels.py:75-75
Timestamp: 2025-08-12T18:04:06.577Z
Learning: The Newton physics framework requires nightly Warp builds, which means compatibility concerns with older stable Warp versions (like missing functions such as wp.spatial_adjoint) are not relevant for this project.
Applied to files:
.github/workflows/scheduled_nightly_warp_tests.yml
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: Run GPU Benchmarks / Run GPU Benchmarks on AWS EC2
- GitHub Check: Run GPU Tests / Run GPU Unit Tests on AWS EC2
🔇 Additional comments (6)
.github/workflows/scheduled_nightly_warp_tests.yml (6)
21-25: Workflow scheduling and triggers are well configured.The schedule (daily at 8 AM UTC / midnight PST) with manual workflow_dispatch fallback is a solid pattern for detecting integration issues with nightly Warp builds while remaining testable.
27-56: Cost optimization strategy is sound.The
check-warp-updatejob efficiently detects new Warp nightly builds on a standard runner before provisioning expensive EC2 GPU infrastructure. The conditional output (line 32) gates downstream jobs, which aligns well with the PR objective to include cost-saving optimizations.
125-126: Lock file is updated twice; verify this is intentional.Line 43 updates the lock file in
check-warp-update, and line 126 updates it again innightly-warp-tests. While this ensures the latest nightly is used at test time, it defeats the purpose of the check-update job's cost optimization (since the lock file change will be committed to the runner, not persisted). Clarify whether this redundancy is intentional for test freshness or if one update suffices.
138-162: Cleanup logic correctly ensures EC2 runner is always stopped.The
stop-runnerjob usesif: always()with a check forneeds.start-runner.result != 'skipped'to guarantee cleanup even if tests fail. This prevents resource leaks and cost overruns. The job dependencies (line 144-146) correctly wait for both upstream jobs before stopping.
70-105: AWS credentials and EC2 runner provisioning are correctly structured.The workflow uses OIDC for AWS credential configuration (line 71) rather than static secrets, which is a security best practice. The EC2 runner provisioning with resource tags (lines 100-105) aids in cost tracking and resource management. The AMI lookup with error handling (lines 84-89) provides reasonable failure detection.
92-92: Confirm your team's versioning and maintenance strategy for the pinnedmachulav/ec2-github-runneraction.The latest release is v2.4.2, published September 24, 2025. The pinned commit hash (
a6dbcefcf8a31a861f5e078bb153ed332130c512) was not matched to a specific released version via public API queries, so the age and changes between the current pin and the latest release cannot be confirmed without repository access.Verify:
- Which released version the pinned commit corresponds to
- Whether the approximately 50-day gap since v2.4.2's release presents any material feature or security improvements your team requires
- That your team has a documented process to periodically audit this dependency
No known security advisories exist for this action at this time.
…on (newton-physics#1088) Signed-off-by: Eric Shi <ershi@nvidia.com>
…on (newton-physics#1088) Signed-off-by: Eric Shi <ershi@nvidia.com>
Description
As part of the efforts to move all our CI/CD over to GitHub, we need to add a job that does the same thing as the job we ran on the internal GitLab instance to test Newton against the most recent nightly Warp build.
There are also some cost-saving optimizations to skip the scheduled job if Newton is already testing with the latest nightly build of Warp.
Newton Migration Guide
Please ensure the migration guide for warp.sim users is up-to-date with the changes made in this PR.
docs/migration.rstis up-to dateBefore your PR is "Ready for review"
newton/tests/test_examples.py)pre-commit run -aSummary by CodeRabbit