Update walk on sand example#631
Conversation
Signed-off-by: Gilles Daviet <gdaviet@nvidia.com>
Signed-off-by: Gilles Daviet <gdaviet@nvidia.com>
Signed-off-by: Gilles Daviet <gdaviet@nvidia.com>
Signed-off-by: Gilles Daviet <gdaviet@nvidia.com>
📝 WalkthroughWalkthroughREADME links updated; internal numeric fields in the implicit MPM solver cast to explicit float/vec types; new Any-mal C sand-walking example registered and rewritten to use a viewer, MuJoCo robot solver, implicit MPM sand, and a TorchScript policy with CUDA-graph support; removed Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor User
participant Viewer
participant Example
participant Assets as AssetStore
participant Policy as TorchScriptPolicy
participant MuJoCo as SolverMuJoCo
participant MPM as SolverImplicitMPM
User->>Viewer: launch runner
Viewer->>Example: init(viewer, options)
Example->>Assets: download policy
Assets-->>Example: policy path
Example->>Policy: load TorchScript
Example->>MuJoCo: build robot & set initial state
Example->>MPM: build sand domain (voxel_size, friction, ...)
Note over Example,Policy: optional CUDA graph capture
loop per frame
Viewer->>Example: step()
Example->>Example: compute_obs(state)
Example->>Policy: forward(obs)
Policy-->>Example: actions
Example->>MuJoCo: apply_control(actions remapped)
alt CUDA graph available
Example->>MuJoCo: launch captured graph
else
Example->>MuJoCo: simulate step
end
Example->>MPM: simulate step
Example-->>Viewer: render/log_state
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested reviewers
Tip 🔌 Remote MCP (Model Context Protocol) integration is now available!Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats. 📜 Recent review detailsConfiguration used: Path: .coderabbit.yml Review profile: CHILL Plan: Pro 💡 Knowledge Base configuration:
You can enable these sources in your CodeRabbit configuration. 📒 Files selected for processing (1)
🧰 Additional context used🧠 Learnings (1)📚 Learning: 2025-08-25T11:07:47.779ZApplied to files:
🧬 Code graph analysis (1)newton/examples/mpm/example_anymal_c_walk_on_sand.py (1)
🔇 Additional comments (17)
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
Status, Documentation and Community
|
Signed-off-by: Gilles Daviet <gdaviet@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (3)
newton/examples/mpm/example_anymal_c_walk_on_sand.py (3)
41-57: Docstring formatting + optional quaternion normalization for numerical robustness.
- The docstring is squashed; reformat for readability.
- If inputs may drift from unit quaternions, normalize q to avoid subtle frame drift. TorchScript-friendly normalization shown below.
@torch.jit.script def quat_rotate_inverse(q: torch.Tensor, v: torch.Tensor) -> torch.Tensor: - """Rotate a vector by the inverse of a quaternion along the last dimension of q and v. Args: - q: The quaternion in (x, y, z, w). Shape is (..., 4). - v: The vector in (x, y, z). Shape is (..., 3). Returns: - The rotated vector in (x, y, z). Shape is (..., 3). - """ - q_w = q[..., 3] # w component is at index 3 for XYZW format + """ + Rotate vectors by the inverse of a quaternion (last dim of q/v). + Args: + q: Quaternion in XYZW order, shape (..., 4). + v: Vectors to rotate, shape (..., 3). + Returns: + Rotated vectors in body frame, shape (..., 3). + """ + # Optional: normalize to guard against drift + denom = torch.sqrt(torch.clamp(torch.sum(q * q, dim=-1, keepdim=True), min=1.0e-12)) + q = q / denom + + q_w = q[..., 3] # w component is at index 3 for XYZW format q_vec = q[..., :3] # xyz components are at indices 0, 1, 2 a = v * (2.0 * q_w**2 - 1.0).unsqueeze(-1) b = torch.cross(q_vec, v, dim=-1) * q_w.unsqueeze(-1) * 2.0 # for two-dimensional tensors, bmm is faster than einsum if q_vec.dim() == 2: c = q_vec * torch.bmm(q_vec.view(q.shape[0], 1, 3), v.view(q.shape[0], 3, 1)).squeeze(-1) * 2.0 else: c = q_vec * torch.einsum("...i,...i->...", q_vec, v).unsqueeze(-1) * 2.0 return a - b + c
275-283: Use index_select instead of gather for action reordering.
index_selectis simpler and avoids constructing a broadcasted index tensor.- self.rearranged_act = torch.gather(self.act, 1, self.mujoco_to_lab_indices.unsqueeze(0)) + self.rearranged_act = torch.index_select(self.act, 1, self.mujoco_to_lab_indices)
350-361: Fix redundant prod and clarify particle mass computation.
cell_volumeis already a scalar;np.prod(cell_volume)is redundant. Also, name the variable to reflect it’s a mass proxy.- cell_volume = np.prod(cell_size) + cell_volume = np.prod(cell_size) @@ - volume = np.prod(cell_volume) * packing_fraction + particle_volume = cell_volume * packing_fraction @@ - builder.particle_mass = np.full(points.shape[0], volume) + builder.particle_mass = np.full(points.shape[0], particle_volume)Note: this sets each particle’s “mass” proportional to cell volume; if you intend physical mass, multiply by a material density.
📜 Review details
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
⛔ Files ignored due to path filters (1)
docs/images/examples/example_anymal_c_walk_on_sand.jpgis excluded by!**/*.jpg
📒 Files selected for processing (5)
README.md(2 hunks)newton/_src/solvers/implicit_mpm/solver_implicit_mpm.py(1 hunks)newton/examples/__init__.py(2 hunks)newton/examples/mpm/example_anymal_c_walk_on_sand.py(5 hunks)newton/examples/mpm/example_mpm_granular.py(0 hunks)
💤 Files with no reviewable changes (1)
- newton/examples/mpm/example_mpm_granular.py
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-08-19T12:27:30.629Z
Learnt from: preist-nvidia
PR: newton-physics/newton#579
File: newton/examples/example_mujoco.py:350-354
Timestamp: 2025-08-19T12:27:30.629Z
Learning: In Newton examples, there's a distinction between solver parameters and Example class attributes. The Example class can have its own use_mujoco attribute for controlling example-level behavior (like CUDA graphs, rendering logic), while the solver uses use_mujoco_cpu for backend selection. These serve different purposes and should not be conflated during API renames.
Applied to files:
newton/examples/mpm/example_anymal_c_walk_on_sand.py
🧬 Code graph analysis (1)
newton/examples/mpm/example_anymal_c_walk_on_sand.py (4)
newton/_src/solvers/implicit_mpm/solver_implicit_mpm.py (1)
setup_collider(1135-1196)newton/_src/sim/model.py (2)
state(440-474)control(476-509)newton/_src/sim/state.py (3)
State(21-112)requires_grad(78-84)clear_forces(64-75)newton/examples/__init__.py (3)
create_parser(100-126)init(129-170)run(36-44)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: run-newton-tests / newton-unittests (windows-latest)
- GitHub Check: run-newton-tests / newton-unittests (ubuntu-latest)
🔇 Additional comments (4)
newton/_src/solvers/implicit_mpm/solver_implicit_mpm.py (1)
1071-1086: Type coercions to Python float/wp.vec3f are correct; keep everything consistently float32 into Warp.Casting density, friction, thresholds, and voxel size to Python float and yield_stresses to wp.vec3f aligns better with Warp kernel signatures and avoids dtype ambiguity when passing scalars into kernels/structs. Looks good.
Two light checks:
- Ensure downstream callers don’t rely on double precision for these fields; Warp’s float maps to float32.
- Division
self.yield_stresses / self.densityproduces a vec3f, which is compatible withscaled_yield_stress.fill_(dtype vec3). Confirm at runtime that no implicit host<->device transfer is triggered there.If useful, validate no dtype promotions by printing dtypes once at startup:
print(type(self.density), type(self.friction_coeff), type(self.voxel_size), type(self.tolerance)) print(type(self.max_fraction), type(self.yield_stresses))newton/examples/__init__.py (2)
54-57: String reflow is fine; no behavior change.Reformatting the deprecation warning string using parentheses preserves message content and avoids a long literal. No action needed.
189-190: CLI mapping added for anymal_c_walk_on_sand – nice. Verify import path works.The short name correctly maps to
newton.examples.mpm.example_anymal_c_walk_on_sand. As a quick smoke test:
python -m newton.examples anymal_c_walk_on_sand --viewer null --num-frames 1This should import the module and run one frame without opening a window.
newton/examples/mpm/example_anymal_c_walk_on_sand.py (1)
254-257: Good separation: CUDA graph for robot, regular step for sand.Capturing only the rigid simulation and leaving MPM uncaptured matches current Warp graph support. Nice organization of control→simulate_robot→simulate_sand sequencing.
Also applies to: 258-264, 299-308
Signed-off-by: Gilles Daviet <gdaviet@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
newton/examples/mpm/example_anymal_c_walk_on_sand.py (1)
59-76: Great fix: observations now use zero-copy Warp→Torch views.Switching to
wp.to_torch(state.joint_q/joint_qd)eliminates host transfers and ensures tensors stay on the active device. This addresses the earlier feedback about unsafetorch.tensor(...)conversions.
🧹 Nitpick comments (3)
newton/examples/mpm/example_anymal_c_walk_on_sand.py (3)
40-56: Tidy up docstring; optionally clarify unit-quaternion assumption.
- The docstring lines include inline “Args/Returns” on the same line; minor formatting polish helps readability.
- The math assumes
qis unit-length. If that’s guaranteed upstream, mention it in the docstring; otherwise consider normalizing or asserting near-unit length to avoid drift.Example minimal change:
- """Rotate a vector by the inverse of a quaternion along the last dimension of q and v. Args: - q: The quaternion in (x, y, z, w). Shape is (..., 4). - v: The vector in (x, y, z). Shape is (..., 3). Returns: - The rotated vector in (x, y, z). Shape is (..., 3). - """ + """Rotate vector(s) by the inverse of a quaternion. + + Args: + q: Quaternion(s) in (x, y, z, w) format, shape (..., 4). Assumes unit length. + v: Vector(s), shape (..., 3). + Returns: + Rotated vector(s), shape (..., 3). + """
233-235: Avoid redundant asset download; reuse the earlier asset_path.You already have
asset_path = newton.utils.download_asset("anybotics_anymal_c")above. Reuse it to constructpolicy_pathto save an extra lookup.-policy_asset_path = newton.utils.download_asset("anybotics_anymal_c") -policy_path = str(policy_asset_path / "rl_policies" / "anymal_walking_policy_physx.pt") +policy_path = str(asset_path / "rl_policies" / "anymal_walking_policy_physx.pt")
260-266: Confirm CUDA-graph compatibility of viewer-driven forces.
viewer.apply_forces(self.state_0)is insidesimulate_robot()which is captured. If the viewer updates forces interactively each frame on the CPU side, ensure the captured kernels read the updated device buffers on eachwp.capture_launch(and that no CPU-side logic inapply_forcesis required per frame).If forces don’t update under capture, we can gate the call and inject a pre-step hook:
- self.viewer.apply_forces(self.state_0) + # During graph replay, ensure forces are already staged in device buffers. + if self.graph is None: + self.viewer.apply_forces(self.state_0)And before launching the graph each frame (after we’ve updated control but before physics):
def step(self): # compute control before graph/step self.apply_control() + # Stage any interactive forces to device before graph replay + if self.graph: + self.viewer.stage_forces(self.state_0) # implement as a no-op if not neededOnly adopt if you observe stale forces during CUDA graph replay.
Also applies to: 285-293, 299-306
📜 Review details
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
README.md(2 hunks)newton/examples/mpm/example_anymal_c_walk_on_sand.py(5 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- README.md
🧰 Additional context used
🧠 Learnings (2)
📚 Learning: 2025-08-19T12:27:30.629Z
Learnt from: preist-nvidia
PR: newton-physics/newton#579
File: newton/examples/example_mujoco.py:350-354
Timestamp: 2025-08-19T12:27:30.629Z
Learning: In Newton examples, there's a distinction between solver parameters and Example class attributes. The Example class can have its own use_mujoco attribute for controlling example-level behavior (like CUDA graphs, rendering logic), while the solver uses use_mujoco_cpu for backend selection. These serve different purposes and should not be conflated during API renames.
Applied to files:
newton/examples/mpm/example_anymal_c_walk_on_sand.py
📚 Learning: 2025-08-25T11:07:47.779Z
Learnt from: gdaviet
PR: newton-physics/newton#631
File: newton/examples/mpm/example_anymal_c_walk_on_sand.py:164-169
Timestamp: 2025-08-25T11:07:47.779Z
Learning: In Newton's MuJoCo solver implementation, setting TARGET_POSITION mode on floating base DOFs (first 6 DOFs) does not overconstrain the base - the solver handles this appropriately and maintains expected behavior for floating robots.
Applied to files:
newton/examples/mpm/example_anymal_c_walk_on_sand.py
🧬 Code graph analysis (1)
newton/examples/mpm/example_anymal_c_walk_on_sand.py (3)
newton/_src/sim/model.py (2)
state(440-474)control(476-509)newton/_src/sim/state.py (3)
State(21-112)requires_grad(78-84)clear_forces(64-75)newton/examples/__init__.py (3)
create_parser(100-126)init(129-170)run(36-44)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: run-newton-tests / newton-unittests (windows-latest)
- GitHub Check: run-newton-tests / newton-unittests (ubuntu-latest)
🔇 Additional comments (4)
newton/examples/mpm/example_anymal_c_walk_on_sand.py (4)
164-168: Setting TARGET_POSITION on all DOFs is acceptable per MuJoCo backend behavior.Based on the retrieved learning for this repo, applying
TARGET_POSITIONto the six floating-base DOFs is handled gracefully by the MuJoCo solver and does not overconstrain the base. No change requested here.
255-259: Viewer integration and unified runner usage look solid.Setting the model on the viewer and enabling particle visualization aligns with the new examples API. Capturing immediately after init is a good default.
201-216: Solver setup is consistent with the refactor goals.
- Robot on MuJoCo, sand on implicit MPM with dynamic grid options and tolerance—looks correct.
grid_paddingwhendynamic_grid=Falseis a nice touch.
390-420: CLI integration via the unified runner is clean.
- Extends the common parser appropriately and routes through
newton.examples.init/run.- GPU device check is clear; error path exits early.
Signed-off-by: Gilles Daviet <gdaviet@nvidia.com>
|
Hey Gilles, if you follow these instructions then we don't need to manually trigger the GPU workflows for your pull request every time it's updated:
|
|
Looks like The example gallery will also need to be slightly tweaked to match the new conventions after #632 goes in today:
|
|
I'm integrating this PR into #626 so we can get this merged today. |
Description
_physx_policyversionEXAMPLES-GUIDE.mdBefore your PR is "Ready for review"
newton/tests/test_examples.py)pre-commit run -aSummary by CodeRabbit
New Features
Documentation
Bug Fixes
Refactor