Skip to content

Fixed MJC shape mapping#514

Merged
Milad-Rakhsha-NV merged 2 commits into
newton-physics:mainfrom
Milad-Rakhsha-NV:milad/fix-shape-mapping
Aug 8, 2025
Merged

Fixed MJC shape mapping#514
Milad-Rakhsha-NV merged 2 commits into
newton-physics:mainfrom
Milad-Rakhsha-NV:milad/fix-shape-mapping

Conversation

@Milad-Rakhsha-NV

@Milad-Rakhsha-NV Milad-Rakhsha-NV commented Aug 8, 2025

Copy link
Copy Markdown
Member

Currently all geoms in all environments of MJC map to the geoms in the first environment in Newton. This is a wrong behavior and prevents material shape randomization.

Description

Newton Migration Guide

Please ensure the migration guide for warp.sim users is up-to-date with the changes made in this MR.

  • The migration guide in docs/migration.rst is up-to date

Before your PR is "Ready for review"

  • All commits are signed-off to indicate that your contribution adheres to the Developer Certificate of Origin requirements
  • Necessary tests have been added and new examples are tested (see newton/tests/test_examples.py)
  • Documentation is up-to-date
  • Code passes formatting and linting checks with pre-commit run -a

Summary by CodeRabbit

  • New Features

    • Enhanced support for multiple environments in MuJoCoWarp by making shape-to-geom index mappings environment-aware.
  • Bug Fixes

    • Updated array indexing in tests and core logic to correctly handle environment-specific mappings for shape and geometry indices.

Signed-off-by: Milad Rakhsha <mrakhsha@nvidia.com>
@coderabbitai

coderabbitai Bot commented Aug 8, 2025

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

The changes update the MuJoCoWarp solver to support multiple environments by making shape-to-geom index mappings environment-aware. This involves converting relevant arrays and dictionary mappings from 1D to 2D, indexed by both environment and geometry indices, and updating kernel argument types and test code to reflect the new structure.

Changes

Cohort / File(s) Change Summary
MuJoCo solver multi-env mapping
newton/solvers/mujoco/solver_mujoco.py
Refactored shape-to-geom index mapping from 1D to 2D arrays to support multiple environments. Updated kernel argument types, mapping construction, and assignment logic to use environment and geometry indices. Adjusted internal data structures and array initializations for environment-aware indexing.
Test updates for multi-env mapping
newton/tests/test_mujoco_solver.py
Updated test cases to use 2D indexing ([world_idx, geom_idx]) when accessing the shape-to-geom index mapping, aligning with the new environment-aware structure. No changes to test logic or public interfaces.

Sequence Diagram(s)

sequenceDiagram
    participant Test as Test Case
    participant Solver as MuJoCoSolver
    participant Kernel as update_geom_properties_kernel

    Test->>Solver: convert_to_mjc(model, ...)
    Solver->>Solver: Build full_shape_mapping with (env_idx, geom_idx)
    Solver->>Solver: Initialize to_newton_shape_array[env_idx, geom_idx]
    Solver->>Solver: Set model.to_newton_shape_index as 2D array
    Solver->>Kernel: Call update_geom_properties_kernel(..., to_newton_shape_index)
    Kernel->>Kernel: Access to_newton_shape_index[worldid, geom_idx]
    Note right of Kernel: Properties updated per environment
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~15 minutes

Possibly related PRs

Suggested reviewers

  • eric-heiden

Note

🔌 MCP (Model Context Protocol) integration is now available in Early Access!

Pro users can now connect to remote MCP servers under the Integrations page to get reviews and chat conversations that understand additional development context.

✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai generate unit tests to generate unit tests for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🔭 Outside diff range comments (3)
newton/solvers/mujoco/solver_mujoco.py (1)

2530-2536: update_geom_properties: use mjw_model’s world dimension, not model.num_envs

When envs aren’t separated, mjw_model holds nworld=1 fields. Launching kernels with dim=(model.num_envs, ...) will cause out-of-bounds writes to mjw_model arrays. Use the world dimension from mjw_model’s field shapes.

Apply this diff:

-        num_geoms = self.mj_model.ngeom
-        num_worlds = self.model.num_envs  # why is there no 'self.mjw_model.nworld'?
+        num_geoms = self.mj_model.ngeom
+        # Use the actual world dimension of the mjw model arrays.
+        # This is 1 when environments are not separated.
+        num_worlds = int(self.mjw_model.geom_pos.shape[0])
newton/tests/test_mujoco_solver.py (2)

688-696: Add an invariant check to catch the original cross-env mapping bug.

Right now, “expected” values are derived from the same mapping under test, which could mask regressions (e.g., all worlds mapping to env 0 shapes). Assert that, for each world, geom→shape indices differ from world 0 by a constant offset across geoms.

Apply this diff inside the loop after computing shape_idx to ensure the per-env offset is consistent (skip unmapped geoms):

 for world_idx in range(self.model.num_envs):
     for geom_idx in range(num_geoms):
-        shape_idx = to_newton_shape_index[world_idx, geom_idx]
+        shape_idx = to_newton_shape_index[world_idx, geom_idx]
+        # Invariant: for replicated envs, mapping should shift by a constant per-env offset
+        if world_idx > 0:
+            base_shape_idx = to_newton_shape_index[0, geom_idx]
+            if base_shape_idx >= 0 and shape_idx >= 0:
+                if env_offsets[world_idx] is None:
+                    env_offsets[world_idx] = int(shape_idx - base_shape_idx)
+                else:
+                    self.assertEqual(
+                        int(shape_idx - base_shape_idx),
+                        env_offsets[world_idx],
+                        f"Geom→shape mapping offset must be constant within world {world_idx}",
+                    )
         if shape_idx < 0:  # No mapping for this geom
             continue

Additionally, define env_offsets before the loop:

# Place right after `tested_count = 0`
env_offsets = [None] * self.model.num_envs

893-901: Repeat the invariant check in the update test to guard against regressions.

Ensure we don’t silently reintroduce the “all envs map to env 0” bug when updating properties.

Apply this diff inside the loop after computing shape_idx:

 for world_idx in range(self.model.num_envs):
     for geom_idx in range(num_geoms):
-        shape_idx = to_newton_shape_index[world_idx, geom_idx]
+        shape_idx = to_newton_shape_index[world_idx, geom_idx]
+        # Invariant: ensure a constant per-env offset relative to world 0
+        if world_idx > 0:
+            base_shape_idx = to_newton_shape_index[0, geom_idx]
+            if base_shape_idx >= 0 and shape_idx >= 0:
+                if env_offsets[world_idx] is None:
+                    env_offsets[world_idx] = int(shape_idx - base_shape_idx)
+                else:
+                    self.assertEqual(
+                        int(shape_idx - base_shape_idx),
+                        env_offsets[world_idx],
+                        f"Geom→shape mapping offset must be constant within world {world_idx}",
+                    )
         if shape_idx < 0:  # No mapping
             continue

And define env_offsets before the loop:

# Place right before the `for world_idx` loop
env_offsets = [None] * self.model.num_envs
🧹 Nitpick comments (1)
newton/solvers/mujoco/solver_mujoco.py (1)

992-1009: Migration note: public field shape change

to_newton_shape_index is now 2D. Please update the migration guide and any downstream docs/usages accordingly.

I can draft a short migration snippet (old vs new indexing) for warp.sim users if helpful.

📜 Review details

Configuration used: .coderabbit.yml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between a390cda and 245c882.

📒 Files selected for processing (2)
  • newton/solvers/mujoco/solver_mujoco.py (4 hunks)
  • newton/tests/test_mujoco_solver.py (2 hunks)
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: adenzler-nvidia
PR: newton-physics/newton#479
File: newton/solvers/mujoco/solver_mujoco.py:2147-2149
Timestamp: 2025-07-28T15:04:54.360Z
Learning: The ls_parallel option is specific to MuJoCo Warp and is not available in the pure MuJoCo mjOption structure. It should only be set on mjw_model.opt, not mj_model.opt.
📚 Learning: 2025-07-15T21:00:03.709Z
Learnt from: dylanturpin
PR: newton-physics/newton#394
File: newton/tests/test_ik.py:119-119
Timestamp: 2025-07-15T21:00:03.709Z
Learning: In the Newton physics engine codebase, it's acceptable to test private API functions (those prefixed with underscore) in unit and component tests, as the maintainers consider this appropriate for testing internal functionality.

Applied to files:

  • newton/tests/test_mujoco_solver.py
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: Run GPU Benchmarks (Pull Request)
  • GitHub Check: Run GPU Unit Tests on AWS EC2 (Pull Request)
  • GitHub Check: run-newton-tests / newton-unittests (ubuntu-latest)
  • GitHub Check: run-newton-tests / newton-unittests (windows-latest)
🔇 Additional comments (6)
newton/solvers/mujoco/solver_mujoco.py (3)

992-1009: 2D to_newton_shape_index: signature + indexing look correct

Switch to wp.array2d and (worldid, geom_idx) access is consistent with the new env-aware design. No issues spotted here.


2251-2261: Env-aware full_shape_mapping expansion is correct

Mapping each global Newton shape index to its (env_idx, mj_geom_idx) is the right approach when separating envs to worlds.


2262-2263: Fallback mapping when not separating envs

Mapping to (0, geom_idx) keeps a single-world MuJoCo model consistent. Good.

newton/tests/test_mujoco_solver.py (3)

690-693: Correct: switch to env-aware indexing for geom→shape mapping.

Using to_newton_shape_index[world_idx, geom_idx] aligns tests with the new 2D mapping in the solver.


895-899: Correct: env-aware indexing for dynamic updates as well.

Mirroring the conversion test, this change ensures updates validate per-env mappings.


662-669: No single-dimension indexing of to_newton_shape_index remains
A project-wide search (rg -n 'to_newton_shape_index\[[^]]+\]' | gawk …) returned no instances of one-dimensional subscripting without a comma. No further changes are needed.

Comment thread newton/solvers/mujoco/solver_mujoco.py
Comment thread newton/solvers/mujoco/solver_mujoco.py
@preist-nvidia preist-nvidia requested a review from vreutskyy August 8, 2025 06:55

@vreutskyy vreutskyy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@shi-eric

shi-eric commented Aug 8, 2025

Copy link
Copy Markdown
Member

I was curious about the massive perf hit (20x slowdown) to simulation.bench_mujoco.FastCartpole.time_simulate. I thought it was a fluke from a bad EC2 system but I ran it again and got the same result. Can you take a closer look?

@Milad-Rakhsha-NV

Copy link
Copy Markdown
Member Author

Thanks Eric yes I was also curious what is going on there
That said, the mujoco solver initialization now is expected to take a bit longer (should not be 19x though) since we need full shape mapping, meaning instead of reusing the mapping for the first environment, we need to store mapping for all 256 envs

@shi-eric

shi-eric commented Aug 8, 2025

Copy link
Copy Markdown
Member

Thanks Eric yes I was also curious what is going on there That said, the mujoco solver initialization now is expected to take a bit longer (should not be 19x though) since we need full shape mapping, meaning instead of reusing the mapping for the first environment, we need to store mapping for all 256 envs

I ran what should be the equivalent example_mujoco.py command example_mujoco.py --stage_path None --robot cartpole --num-frames 50 --num-envs 256 --headless --actuation random --random-init and didn't see your pull request changes taking any longer, so it's probably some issue with the benchmark that I should look into.

@shi-eric

shi-eric commented Aug 8, 2025

Copy link
Copy Markdown
Member

Thanks Eric yes I was also curious what is going on there That said, the mujoco solver initialization now is expected to take a bit longer (should not be 19x though) since we need full shape mapping, meaning instead of reusing the mapping for the first environment, we need to store mapping for all 256 envs

I couldn't reproduce it on my local machine either when running asv (asv continuous --launch-method spawn --interleave-rounds --append-samples --no-only-changed -b Fast a390cdaf794c853e524cd0f66592e0a849b92425 b1dc22e96dbfc899c93e7e26ef65a482fd779820) so I think you should ignore the benchmark failure and merge it if it's ready

@Milad-Rakhsha-NV

Copy link
Copy Markdown
Member Author

Awesome, thanks for helping!

@Milad-Rakhsha-NV Milad-Rakhsha-NV enabled auto-merge (squash) August 8, 2025 17:41
@Milad-Rakhsha-NV Milad-Rakhsha-NV merged commit 57fe555 into newton-physics:main Aug 8, 2025
10 of 11 checks passed
@coderabbitai coderabbitai Bot mentioned this pull request Aug 22, 2025
4 tasks
@coderabbitai coderabbitai Bot mentioned this pull request Dec 8, 2025
1 task
eric-heiden pushed a commit to eric-heiden/newton that referenced this pull request Jan 28, 2026
Signed-off-by: Milad Rakhsha <mrakhsha@nvidia.com>
mmacklin pushed a commit to mmacklin/newton that referenced this pull request Apr 7, 2026
Signed-off-by: Milad Rakhsha <mrakhsha@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants