Add example to run physx policy with mujoco solver#364
Conversation
WalkthroughA new example script demonstrates controlling the Anymal C robot in the Newton simulation environment using a pretrained PhysX walking policy. The script sets up the simulation, handles robot state initialization, computes observations for the policy, runs control and simulation steps, and optionally renders the results. Key functions and a main class manage the flow. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant Example
participant Policy
participant Simulator
participant Renderer
User->>Example: Initialize (stage_path, headless, etc.)
Example->>Simulator: Build model, load URDF, set initial state
Example->>Policy: Load pretrained policy
loop For each frame
Example->>Policy: Compute observation
Policy-->>Example: Return actions
Example->>Simulator: Apply actions, step simulation
alt Rendering enabled
Example->>Renderer: Render current state
end
end
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📥 CommitsReviewing files that changed from the base of the PR and between e2c5687d77cc7f666f22a7a576c3576840a1ad5a and 3d15740. 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
✨ Finishing Touches
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (3)
newton/examples/example_anymal_c_walk_physx_policy.py (3)
41-45: Fix docstring formatting.The docstring has formatting issues with missing spaces and line breaks.
-def quat_rotate_inverse(q: torch.Tensor, v: torch.Tensor) -> torch.Tensor: - """Rotate a vector by the inverse of a quaternion along the last dimension of q and v. Args: - q: The quaternion in (x, y, z, w). Shape is (..., 4). - v: The vector in (x, y, z). Shape is (..., 3). Returns: - The rotated vector in (x, y, z). Shape is (..., 3). - """ +def quat_rotate_inverse(q: torch.Tensor, v: torch.Tensor) -> torch.Tensor: + """Rotate a vector by the inverse of a quaternion along the last dimension of q and v. + + Args: + q: The quaternion in (x, y, z, w). Shape is (..., 4). + v: The vector in (x, y, z). Shape is (..., 3). + + Returns: + The rotated vector in (x, y, z). Shape is (..., 3). + """
200-203: Consider implementing the suggested optimization.The inline comment correctly identifies an optimization opportunity. Instead of launching
apply_mjc_control_kernelat every simulation step, you could update positions and velocity targets only at the policy inference rate.Would you like me to help implement this optimization by directly updating
self.solver.mjw_data.ctrl[0]and avoiding unnecessary kernel launches?
236-236: Remove whitespace from blank line.- example = Example(stage_path=args.stage_path, headless=args.headless) - + example = Example(stage_path=args.stage_path, headless=args.headless) + # Download the policy from the newton-assets repository
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📥 Commits
Reviewing files that changed from the base of the PR and between abf2c41 and bef51f5612b89e55a21aedf1f6e29602959a50e7.
📒 Files selected for processing (1)
newton/examples/example_anymal_c_walk_physx_policy.py(1 hunks)
🧰 Additional context used
🪛 Ruff (0.11.9)
newton/examples/example_anymal_c_walk_physx_policy.py
26-26: os imported but unused
Remove unused import: os
(F401)
236-236: Blank line contains whitespace
Remove whitespace from blank line
(W293)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: run-asv-benchmarks
- GitHub Check: run-newton-tests / newton-unittests (ubuntu-latest)
- GitHub Check: run-newton-tests / newton-unittests (windows-latest)
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (1)
newton/examples/example_anymal_c_walk_physx_policy.py (1)
26-26: Remove unused import.The
osmodule is imported but never used in the code.-import os - import torch
🧹 Nitpick comments (3)
newton/examples/example_anymal_c_walk_physx_policy.py (3)
39-45: Fix docstring formatting.The docstring has formatting issues - the Args and Returns sections are not properly structured.
@torch.jit.script def quat_rotate_inverse(q: torch.Tensor, v: torch.Tensor) -> torch.Tensor: - """Rotate a vector by the inverse of a quaternion along the last dimension of q and v. Args: - q: The quaternion in (x, y, z, w). Shape is (..., 4). - v: The vector in (x, y, z). Shape is (..., 3). Returns: - The rotated vector in (x, y, z). Shape is (..., 3). + """Rotate a vector by the inverse of a quaternion along the last dimension of q and v. + + Args: + q: The quaternion in (x, y, z, w). Shape is (..., 4). + v: The vector in (x, y, z). Shape is (..., 3). + + Returns: + The rotated vector in (x, y, z). Shape is (..., 3). """
200-202: Consider implementing the suggested performance optimization.The inline comment mentions a valid performance optimization: avoiding repeated kernel launches by updating control targets only at the policy frequency rather than every simulation step.
Would you like me to help implement this optimization as mentioned in the PR objectives?
236-236: Remove whitespace from blank line.The blank line contains unnecessary whitespace.
example = Example(stage_path=args.stage_path, headless=args.headless) - + # Download the policy from the newton-assets repository
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📥 Commits
Reviewing files that changed from the base of the PR and between bef51f5612b89e55a21aedf1f6e29602959a50e7 and bb6468922d97976344861559cee54e5aa0ffce94.
📒 Files selected for processing (1)
newton/examples/example_anymal_c_walk_physx_policy.py(1 hunks)
🧰 Additional context used
🪛 Ruff (0.11.9)
newton/examples/example_anymal_c_walk_physx_policy.py
26-26: os imported but unused
Remove unused import: os
(F401)
236-236: Blank line contains whitespace
Remove whitespace from blank line
(W293)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: run-newton-tests / newton-unittests (ubuntu-latest)
- GitHub Check: run-newton-tests / newton-unittests (windows-latest)
- GitHub Check: run-asv-benchmarks
🔇 Additional comments (4)
newton/examples/example_anymal_c_walk_physx_policy.py (4)
58-72: LGTM - Well-structured observation computation.The observation computation function correctly extracts and transforms robot state data, handles coordinate frame transformations, and efficiently rearranges joint indices between Lab and MuJoCo conventions.
166-175: Excellent CUDA graph optimization implementation.The conditional CUDA graph setup provides significant performance benefits when available, with proper fallback to direct simulation calls.
224-230: LGTM - Proper kebab-case argument formatting.The argument parsing correctly uses kebab-case (
--stage-path,--num-frames) as noted in previous reviews.
237-249: LGTM - Proper asset management and tensor initialization.The code correctly downloads the policy asset, loads the TorchScript model, and initializes the necessary tensors for simulation state tracking.
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
This reverts commit 31c61c8adeff3708bc9118d38d0a158d93573e56. Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Description
Adding anymal example to run pretrained physx policy.
There is also a suggestion that we should avoid launching apply_mjc_control_kernel each step. Positions and velocity targets are updated at the rate of the outer control loop (policy inference) and there is actually no need to copy this each step.
Newton Migration Guide
Please ensure the migration guide for warp.sim users is up-to-date with the changes made in this MR.
docs/migration.rstis up-to dateBefore your PR is "Ready for review"
pre-commit run -aSummary by CodeRabbit