Skip to content

Add example to run physx policy with mujoco solver#364

Merged
eric-heiden merged 16 commits into
newton-physics:mainfrom
jvonmuralt:test
Jul 12, 2025
Merged

Add example to run physx policy with mujoco solver#364
eric-heiden merged 16 commits into
newton-physics:mainfrom
jvonmuralt:test

Conversation

@jvonmuralt

@jvonmuralt jvonmuralt commented Jul 9, 2025

Copy link
Copy Markdown
Member

Description

Adding anymal example to run pretrained physx policy.
There is also a suggestion that we should avoid launching apply_mjc_control_kernel each step. Positions and velocity targets are updated at the rate of the outer control loop (policy inference) and there is actually no need to copy this each step.

Newton Migration Guide

Please ensure the migration guide for warp.sim users is up-to-date with the changes made in this MR.

  • The migration guide in docs/migration.rst is up-to date

Before your PR is "Ready for review"

  • All commits are signed-off to indicate that your contribution adheres to the Developer Certificate of Origin requirements
  • I understand that GitHub does not perform any GPU testing of this pull request
  • Necessary tests have been added
  • Documentation is up-to-date
  • Code passes formatting and linting checks with pre-commit run -a

Summary by CodeRabbit

  • New Features
    • Added an example demonstrating control of the Anymal C robot with a pretrained walking policy in a Newton physics simulation.
    • Includes simulation stepping, policy inference, and optional rendering with configurable device and headless mode.
    • Automatically downloads necessary assets and saves rendered output when enabled.

@jvonmuralt jvonmuralt changed the title Test Add exmaple to run physx policy with mujoco solver Jul 9, 2025
@jvonmuralt jvonmuralt changed the title Add exmaple to run physx policy with mujoco solver Add example to run physx policy with mujoco solver Jul 9, 2025
@jvonmuralt jvonmuralt linked an issue Jul 9, 2025 that may be closed by this pull request
@jvonmuralt jvonmuralt marked this pull request as ready for review July 9, 2025 13:38
Comment thread newton/examples/example_anymal_c_walk_physx_policy.py Outdated
@coderabbitai

coderabbitai Bot commented Jul 11, 2025

Copy link
Copy Markdown
Contributor

Walkthrough

A new example script demonstrates controlling the Anymal C robot in the Newton simulation environment using a pretrained PhysX walking policy. The script sets up the simulation, handles robot state initialization, computes observations for the policy, runs control and simulation steps, and optionally renders the results. Key functions and a main class manage the flow.

Changes

File(s) Change Summary
newton/examples/example_anymal_c_walk_physx_policy.py Added a new example script demonstrating Anymal C control using a pretrained PhysX walking policy with Newton simulation, including quaternion inverse rotation, observation computation, policy inference, simulation stepping, and optional rendering.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant Example
    participant Policy
    participant Simulator
    participant Renderer

    User->>Example: Initialize (stage_path, headless, etc.)
    Example->>Simulator: Build model, load URDF, set initial state
    Example->>Policy: Load pretrained policy
    loop For each frame
        Example->>Policy: Compute observation
        Policy-->>Example: Return actions
        Example->>Simulator: Apply actions, step simulation
        alt Rendering enabled
            Example->>Renderer: Render current state
        end
    end
Loading

Poem

Hopping through code with a robot in stride,
Newton and PhysX now walk side by side.
A policy learned, a bunny’s delight—
Anymal C marches through day and night.
With quaternions spun and torques applied,
This rabbit approves: let’s take it for a ride! 🐇


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e2c5687d77cc7f666f22a7a576c3576840a1ad5a and 3d15740.

📒 Files selected for processing (1)
  • newton/examples/example_anymal_c_walk_physx_policy.py (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • newton/examples/example_anymal_c_walk_physx_policy.py
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: run-asv-benchmarks
  • GitHub Check: run-newton-tests / newton-unittests (ubuntu-latest)
  • GitHub Check: run-newton-tests / newton-unittests (windows-latest)
✨ Finishing Touches
  • 📝 Generate Docstrings

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (3)
newton/examples/example_anymal_c_walk_physx_policy.py (3)

41-45: Fix docstring formatting.

The docstring has formatting issues with missing spaces and line breaks.

-def quat_rotate_inverse(q: torch.Tensor, v: torch.Tensor) -> torch.Tensor:
-    """Rotate a vector by the inverse of a quaternion along the last dimension of q and v.    Args:
-    q: The quaternion in (x, y, z, w). Shape is (..., 4).
-    v: The vector in (x, y, z). Shape is (..., 3).    Returns:
-    The rotated vector in (x, y, z). Shape is (..., 3).
-    """
+def quat_rotate_inverse(q: torch.Tensor, v: torch.Tensor) -> torch.Tensor:
+    """Rotate a vector by the inverse of a quaternion along the last dimension of q and v.
+    
+    Args:
+        q: The quaternion in (x, y, z, w). Shape is (..., 4).
+        v: The vector in (x, y, z). Shape is (..., 3).
+    
+    Returns:
+        The rotated vector in (x, y, z). Shape is (..., 3).
+    """

200-203: Consider implementing the suggested optimization.

The inline comment correctly identifies an optimization opportunity. Instead of launching apply_mjc_control_kernel at every simulation step, you could update positions and velocity targets only at the policy inference rate.

Would you like me to help implement this optimization by directly updating self.solver.mjw_data.ctrl[0] and avoiding unnecessary kernel launches?


236-236: Remove whitespace from blank line.

-        example = Example(stage_path=args.stage_path, headless=args.headless)
-        
+        example = Example(stage_path=args.stage_path, headless=args.headless)
+
         # Download the policy from the newton-assets repository
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between abf2c41 and bef51f5612b89e55a21aedf1f6e29602959a50e7.

📒 Files selected for processing (1)
  • newton/examples/example_anymal_c_walk_physx_policy.py (1 hunks)
🧰 Additional context used
🪛 Ruff (0.11.9)
newton/examples/example_anymal_c_walk_physx_policy.py

26-26: os imported but unused

Remove unused import: os

(F401)


236-236: Blank line contains whitespace

Remove whitespace from blank line

(W293)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: run-asv-benchmarks
  • GitHub Check: run-newton-tests / newton-unittests (ubuntu-latest)
  • GitHub Check: run-newton-tests / newton-unittests (windows-latest)

Comment thread newton/examples/example_anymal_c_walk_physx_policy.py Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (1)
newton/examples/example_anymal_c_walk_physx_policy.py (1)

26-26: Remove unused import.

The os module is imported but never used in the code.

-import os
-
 import torch
🧹 Nitpick comments (3)
newton/examples/example_anymal_c_walk_physx_policy.py (3)

39-45: Fix docstring formatting.

The docstring has formatting issues - the Args and Returns sections are not properly structured.

 @torch.jit.script
 def quat_rotate_inverse(q: torch.Tensor, v: torch.Tensor) -> torch.Tensor:
-    """Rotate a vector by the inverse of a quaternion along the last dimension of q and v.    Args:
-    q: The quaternion in (x, y, z, w). Shape is (..., 4).
-    v: The vector in (x, y, z). Shape is (..., 3).    Returns:
-    The rotated vector in (x, y, z). Shape is (..., 3).
+    """Rotate a vector by the inverse of a quaternion along the last dimension of q and v.
+    
+    Args:
+        q: The quaternion in (x, y, z, w). Shape is (..., 4).
+        v: The vector in (x, y, z). Shape is (..., 3).
+    
+    Returns:
+        The rotated vector in (x, y, z). Shape is (..., 3).
     """

200-202: Consider implementing the suggested performance optimization.

The inline comment mentions a valid performance optimization: avoiding repeated kernel launches by updating control targets only at the policy frequency rather than every simulation step.

Would you like me to help implement this optimization as mentioned in the PR objectives?


236-236: Remove whitespace from blank line.

The blank line contains unnecessary whitespace.

         example = Example(stage_path=args.stage_path, headless=args.headless)
-        
+
         # Download the policy from the newton-assets repository
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between bef51f5612b89e55a21aedf1f6e29602959a50e7 and bb6468922d97976344861559cee54e5aa0ffce94.

📒 Files selected for processing (1)
  • newton/examples/example_anymal_c_walk_physx_policy.py (1 hunks)
🧰 Additional context used
🪛 Ruff (0.11.9)
newton/examples/example_anymal_c_walk_physx_policy.py

26-26: os imported but unused

Remove unused import: os

(F401)


236-236: Blank line contains whitespace

Remove whitespace from blank line

(W293)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: run-newton-tests / newton-unittests (ubuntu-latest)
  • GitHub Check: run-newton-tests / newton-unittests (windows-latest)
  • GitHub Check: run-asv-benchmarks
🔇 Additional comments (4)
newton/examples/example_anymal_c_walk_physx_policy.py (4)

58-72: LGTM - Well-structured observation computation.

The observation computation function correctly extracts and transforms robot state data, handles coordinate frame transformations, and efficiently rearranges joint indices between Lab and MuJoCo conventions.


166-175: Excellent CUDA graph optimization implementation.

The conditional CUDA graph setup provides significant performance benefits when available, with proper fallback to direct simulation calls.


224-230: LGTM - Proper kebab-case argument formatting.

The argument parsing correctly uses kebab-case (--stage-path, --num-frames) as noted in previous reviews.


237-249: LGTM - Proper asset management and tensor initialization.

The code correctly downloads the policy asset, loads the TorchScript model, and initializes the necessary tensors for simulation state tracking.

Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
This reverts commit 31c61c8adeff3708bc9118d38d0a158d93573e56.

Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
@eric-heiden eric-heiden enabled auto-merge (squash) July 12, 2025 00:37
@eric-heiden eric-heiden merged commit 11f31bc into newton-physics:main Jul 12, 2025
7 checks passed
jvonmuralt added a commit to jvonmuralt/newton that referenced this pull request Jul 14, 2025
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
eric-heiden pushed a commit to eric-heiden/newton that referenced this pull request Jan 28, 2026
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
@coderabbitai coderabbitai Bot mentioned this pull request Feb 9, 2026
4 tasks
mmacklin pushed a commit to mmacklin/newton that referenced this pull request Apr 7, 2026
Signed-off-by: Julia Prozorova <jprozorova@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create benchmarking scripts in Newton for KPI

3 participants