Fix contact shape mapping#620
Conversation
Signed-off-by: camevor <camevor@nvidia.com>
📝 WalkthroughWalkthroughSimplifies MuJoCo solver contact and geometry mapping by removing contact_geom_mapping, mj_geoms, shape_map, and selected_bodies_set. Deletes create_newton_contact_geom_mapping. update_contacts now passes to_newton_shape_index into convert_mjw_contact_to_warp_kernel. Geometry creation uses body.add_geom(**geom_params) directly. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor Sim as SolverMuJoCo
participant MJ as MuJoCo Model
participant K as convert_mjw_contact_to_warp_kernel
participant W as Warp/Newton Shapes
Sim->>MJ: Gather contacts
note over Sim,MJ: Existing contact data from MuJoCo
rect rgba(200,230,255,0.3)
note right of Sim: Mapping source changed
Sim->>K: update_contacts(..., to_newton_shape_index, ...)
end
K->>W: Convert MJ contacts -> Warp using to_newton_shape_index
K-->>Sim: Converted contacts
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Tip 🔌 Remote MCP (Model Context Protocol) integration is now available!Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats. ✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
Status, Documentation and Community
|
Signed-off-by: camevor <camevor@nvidia.com>
Signed-off-by: camevor <camevor@nvidia.com>
091bcba to
147993a
Compare
adenzler-nvidia
left a comment
There was a problem hiding this comment.
Thanks for the fix and cleanup! Verified that this fixes training in issacLab. Let's add a ticket and follow-up on some regression tests next week.
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
newton/_src/solvers/mujoco/solver_mujoco.py (2)
1006-1028: Static-shape mapping is overwritten across worlds; world-id becomes ambiguous and there’s a write race.For shapes with
shape_group < 0(static, shared across all envs),full_shape_mapping[global_shape_idx]is written once per world with a differentenv_idx, so the last writer wins. Later,convert_newton_contacts_to_mjwarp_kernelinfersworldidfrom this mapping, which can point to the wrong world for contacts in other envs. Also, multiple threads write the same index concurrently.Fix: store
worldid = -1for static shapes (so the contact code can derive the world from the other shape), write the static entry once (e.g.,env_idx == 0), but still fillreverse_shape_mappingfor every world. This also removes the write race.Apply this diff:
def update_incoming_shape_xform_kernel( @@ - template_shape_idx = geom_to_shape_idx[geom_idx] + template_shape_idx = geom_to_shape_idx[geom_idx] if template_shape_idx < 0: return - if shape_group[template_shape_idx] < 0: - # this is a static shape that is used in all environments - global_shape_idx = template_shape_idx - else: - global_shape_idx = env_idx * shape_range_len + template_shape_idx - full_shape_mapping[global_shape_idx] = wp.vec2i(env_idx, geom_idx) - reverse_shape_mapping[env_idx, geom_idx] = global_shape_idx - # Update incoming shape transforms - # compute the difference between the original shape transform - # and the transform after applying the joint child transform - # and the transform MuJoCo does on mesh geoms - original_tf = shape_transform[global_shape_idx] - mjc_p = geom_pos[geom_idx] - q = geom_quat[geom_idx] - # convert quat from wxyz to xyzw - mjc_q = wp.quat(q[1], q[2], q[3], q[0]) - mjc_tf = wp.transform(mjc_p, mjc_q) - shape_incoming_xform[global_shape_idx] = mjc_tf * wp.transform_inverse(original_tf) + if shape_group[template_shape_idx] < 0: + # static shape shared across envs + global_shape_idx = template_shape_idx + # reverse map must exist for each env/geom pair + reverse_shape_mapping[env_idx, geom_idx] = global_shape_idx + # write the forward mapping and incoming xform once to avoid a race + if env_idx != 0: + return + # use worldid = -1 so downstream chooses the other shape's world + full_shape_mapping[global_shape_idx] = wp.vec2i(-1, geom_idx) + else: + # per-env replicated shape + global_shape_idx = env_idx * shape_range_len + template_shape_idx + reverse_shape_mapping[env_idx, geom_idx] = global_shape_idx + full_shape_mapping[global_shape_idx] = wp.vec2i(env_idx, geom_idx) + # Update incoming shape transforms (computed from env0 geom arrays) + original_tf = shape_transform[global_shape_idx] + mjc_p = geom_pos[geom_idx] + q = geom_quat[geom_idx] + mjc_q = wp.quat(q[1], q[2], q[3], q[0]) # wxyz -> xyzw + mjc_tf = wp.transform(mjc_p, mjc_q) + shape_incoming_xform[global_shape_idx] = mjc_tf * wp.transform_inverse(original_tf)
273-315: Guard against negative shape indices when building geoms in contact conversion.
shape_a/shape_bare conditionally checked for bodies, butto_mjc_geom_index[shape_*]is still indexed unconditionally. If a sentinel like -1 ever occurs, we index out of range.Apply this diff:
- world_geom_a = to_mjc_geom_index[shape_a] - world_geom_b = to_mjc_geom_index[shape_b] + world_geom_a = wp.vec2i(-1, -1) + world_geom_b = wp.vec2i(-1, -1) + if shape_a >= 0: + world_geom_a = to_mjc_geom_index[shape_a] + if shape_b >= 0: + world_geom_b = to_mjc_geom_index[shape_b]
🧹 Nitpick comments (2)
newton/_src/solvers/mujoco/solver_mujoco.py (2)
511-552: Rename kernel parameter for clarity and consistency.
convert_mjw_contact_to_warp_kernelstill names the first parametercontact_geom_mapping, but you now passto_newton_shape_index. Renaming improves readability and reduces future confusion.Apply this diff:
@wp.kernel def convert_mjw_contact_to_warp_kernel( # inputs - contact_geom_mapping: wp.array2d(dtype=wp.int32), + to_newton_shape_index: wp.array2d(dtype=wp.int32), pyramidal_cone: bool, @@ - for i in range(2): - pair[i] = contact_geom_mapping[worldid, geoms_mjw[i]] + for i in range(2): + pair[i] = to_newton_shape_index[worldid, geoms_mjw[i]]
2014-2017: Storing geom names and resolving indices post-compile — sensible decoupling.Writing
body.add_geom(**geom_params)and deferring tomj_name2idavoids relying on speculative indices. The subsequent name→id resolution (Lines 2274-2284) is robust given the unique naming schemef"{geom_type_name[stype]}_{shape}".Optionally assert uniqueness at creation time to catch accidental collisions early:
- name = f"{geom_type_name[stype]}_{shape}" + name = f"{geom_type_name[stype]}_{shape}" + # Defensive: ensure name is unique + if mujoco.mj_name2id(spec, mujoco.mjtObj.mjOBJ_GEOM, name) >= 0: + raise ValueError(f"Duplicate geom name detected: {name}")
📜 Review details
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
newton/_src/solvers/mujoco/solver_mujoco.py(2 hunks)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Run GPU Benchmarks (Pull Request)
- GitHub Check: Run GPU Unit Tests on AWS EC2 (Pull Request)
- GitHub Check: run-newton-tests / newton-unittests (windows-latest)
- GitHub Check: run-newton-tests / newton-unittests (ubuntu-latest)
🔇 Additional comments (3)
newton/_src/solvers/mujoco/solver_mujoco.py (3)
1617-1636: Switched to to_newton_shape_index for contact mapping — correct direction.Passing
self.to_newton_shape_indexintoconvert_mjw_contact_to_warp_kernelaligns update_contacts with the new post-compile mapping. Shapes: the kernel expects a 2D int32 map [world, geom] → shape, which matches howto_newton_shape_indexis constructed at Lines 2295-2321. No functional concerns here.
1006-1028: Assumption: template → global shape indexing via env-stride may be invalid if indices aren’t block-strided by group.The
global_shape_idx = env_idx * shape_range_len + template_shape_idxformula assumes per-env shapes occupy contiguous blocks with identical intra-env ordering. If shape indices are interleaved or not strictly block-strided, reverse mapping becomes incorrect.Consider asserting this precondition right after computing
selected_shapes(Line 1937) inconvert_to_mjc. Example check (debug-only):@@ - self.selected_shapes = wp.array(selected_shapes, dtype=wp.int32, device=model.device) + self.selected_shapes = wp.array(selected_shapes, dtype=wp.int32, device=model.device) + # Debug: verify per-env shapes are block-strided and consistent + if separate_envs_to_worlds: + non_neg = np.where(shape_group >= 0)[0] + if len(non_neg): + first_group = np.min(shape_group[non_neg]) + per_env = np.where(shape_group == first_group)[0] + stride = len(per_env) + for env_idx in range(1, model.num_envs): + g = first_group + env_idx + env_shapes = np.where(shape_group == g)[0] + if len(env_shapes) != stride: + raise AssertionError("Per-env shape counts differ; mapping by fixed stride is invalid") + # Optional: verify relative ordering matches template indicesIf this assertion fails in real models, we should pass an explicit “template index → per-env global index” lookup into
update_incoming_shape_xform_kernelinstead of deriving it arithmetically.
2539-2570: Use the actual number of worlds from mjwarp data when launching geom-updates.
update_geom_propertiesusesnum_worlds = self.model.num_envs. Ifseparate_envs_to_worlds=False,put_data(..., nworld=1)creates 2D geom arrays with first dim 1, butself.model.num_envscould be >1, leading to OOB writes.Consider deriving
num_worldsfrom the target arrays or fromself.mjw_data.nworld:- num_worlds = self.model.num_envs # why is there no 'self.mjw_model.nworld'? + num_worlds = int(self.mjw_data.nworld) if hasattr(self, "mjw_data") else self.model.num_envsIf
mjw_dataisn’t available (CPU path), ensure this path isn’t invoked or adjust accordingly.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
newton/_src/solvers/mujoco/solver_mujoco.py (2)
1617-1630: Contact mapping switch is correct; add precondition checks and align kernel arg name.Good change passing
to_newton_shape_indexinto the contact conversion. Two follow-ups:
- Add a guard to fail fast if the mapping wasn’t initialized (e.g., when constructing the solver with pre-existing mjw_model/mjw_data).
- Minor naming nit: the kernel’s parameter is still called
contact_geom_mapping, which now conceptually representsto_newton_shape_index. Renaming improves readability.Apply the precondition check close to the launch:
wp.launch( convert_mjw_contact_to_warp_kernel, dim=mj_data.nconmax, inputs=[ - self.to_newton_shape_index, + # Preconditions + # Ensure mapping exists and matches current MjModel ngeom. + # This is especially important if the solver was constructed + # with pre-existing mjw_model/mjw_data and skipped convert_to_mjc(). + (None if self.to_newton_shape_index is not None else (_ for _ in ()).throw( + RuntimeError("to_newton_shape_index is None; mapping not initialized. Call convert_to_mjc or provide mapping.") + )), + (None if self.to_newton_shape_index is not None and + self.to_newton_shape_index.shape[1] == self.mj_model.ngeom else (_ for _ in ()).throw( + ValueError(f"to_newton_shape_index shape mismatch; expected second dim {self.mj_model.ngeom}") + )), + self.to_newton_shape_index, self.mjw_model.opt.cone == int(self.mujoco.mjtCone.mjCONE_PYRAMIDAL), mj_data.ncon, mj_contact.frame, mj_contact.dim, mj_contact.geom, mj_contact.efc_address, mj_contact.worldid, mj_data.efc.force, ],If you prefer a rename for clarity (outside this hunk), consider updating the kernel signature and its uses:
- def convert_mjw_contact_to_warp_kernel( - contact_geom_mapping: wp.array2d(dtype=wp.int32), + def convert_mjw_contact_to_warp_kernel( + to_newton_shape_index: wp.array2d(dtype=wp.int32), pyramidal_cone: bool, ... ) @@ - for i in range(2): - pair[i] = contact_geom_mapping[worldid, geoms_mjw[i]] + for i in range(2): + pair[i] = to_newton_shape_index[worldid, geoms_mjw[i]]
1979-1984: Avoid in-place mutation ofshape_sizewhen zero-filling.
size = shape_size[shape]returns a view; zero-filling will mutatemodel.shape_scaleunexpectedly. Copy before patching zeros.- size = shape_size[shape] + # copy to avoid mutating model.shape_scale in-place + size = np.array(shape_size[shape], copy=True) if np.any(size > 0.0): # duplicate nonzero entries at places where size is 0 nonzero = size[size > 0.0][0] size[size == 0.0] = nonzero geom_params["size"] = size
📜 Review details
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
newton/_src/solvers/mujoco/solver_mujoco.py(2 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-08-19T12:27:30.629Z
Learnt from: preist-nvidia
PR: newton-physics/newton#579
File: newton/examples/example_mujoco.py:350-354
Timestamp: 2025-08-19T12:27:30.629Z
Learning: In Newton examples, Example.use_mujoco is a high-level attribute that controls whether to use MuJoCo solver vs other solvers (like XPBD), while SolverMuJoCo.use_mujoco_cpu controls the backend within MuJoCo (CPU vs Warp). These are separate concepts serving different purposes - the PR rename only applies to the solver parameter, not the Example class attributes.
Applied to files:
newton/_src/solvers/mujoco/solver_mujoco.py
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Run GPU Benchmarks (Pull Request)
- GitHub Check: Run GPU Unit Tests on AWS EC2 (Pull Request)
- GitHub Check: run-newton-tests / newton-unittests (windows-latest)
- GitHub Check: run-newton-tests / newton-unittests (ubuntu-latest)
🔇 Additional comments (1)
newton/_src/solvers/mujoco/solver_mujoco.py (1)
2014-2017: Recording geoms by name and resolving indices post-compile: solid improvement.Storing
shape -> geom_nameat creation and resolving to indices post-compile removes fragility around assumed ordering and fixes the contact shape mapping issue targeted by this PR. The approach fits well with MuJoCo’s name lookup and your later two-way mapping builds.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
newton/_src/solvers/mujoco/solver_mujoco.py (2)
2014-2017: Name-based geom bookkeeping is safer; mapping computed post-compile is correct.Using
body.add_geom(**geom_params)and storing the geom name, then resolving indices withmj_name2idpost-compile, avoids brittle index assumptions during spec construction. The uniqueness guarantees above make this robust.Minor nit:
- The local
shape_mappingnow holds names during spec-build and indices afterwards; consider renaming the temporary (e.g.,shape_to_geom_name) to reduce cognitive load while reading this function. No functional change.
1616-1629: Good fix: use the canonical geom→shape map in contacts.Switching the first kernel arg to
self.to_newton_shape_indexaligns contact-pair reporting with the mapping used elsewhere (built inupdate_incoming_shape_xform_kernel). Shapes/dtypes match (int32, [nworld, ngeom]), so this resolves the incorrect contact sensor mapping.✅ Verified: no leftover references to the old mapping helpers in solver_mujoco.py. Searches for
create_newton_contact_geom_mappingandmj_geomsreturned no matches, andcontact_geom_mappingonly appears in the kernel’s parameter list and its use sites as intended. Allshape_maphits are confined to the USD-import utility (import_usd.py) and are unrelated to this change.Two small follow-ups:
- Consider renaming the kernel’s parameter from
contact_geom_mappingtogeom_to_shape_indexfor clarity, since it now directly receivesto_newton_shape_index. This is a naming-only change.- Optional guard: if any
pair[i]ends up-1, you may want to skip/zero that contact to avoid downstream consumers tripping on invalid shape ids (rare, but can occur with visual-only or filtered geoms if assumptions drift).
📜 Review details
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
newton/_src/solvers/mujoco/solver_mujoco.py(2 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-08-19T12:27:30.629Z
Learnt from: preist-nvidia
PR: newton-physics/newton#579
File: newton/examples/example_mujoco.py:350-354
Timestamp: 2025-08-19T12:27:30.629Z
Learning: In Newton examples, Example.use_mujoco is a high-level attribute that controls whether to use MuJoCo solver vs other solvers (like XPBD), while SolverMuJoCo.use_mujoco_cpu controls the backend within MuJoCo (CPU vs Warp). These are separate concepts serving different purposes - the PR rename only applies to the solver parameter, not the Example class attributes.
Applied to files:
newton/_src/solvers/mujoco/solver_mujoco.py
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Run GPU Benchmarks (Pull Request)
- GitHub Check: Run GPU Unit Tests on AWS EC2 (Pull Request)
- GitHub Check: run-newton-tests / newton-unittests (windows-latest)
- GitHub Check: run-newton-tests / newton-unittests (ubuntu-latest)
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Thanks for the quick review! Created #621 for the testing. |
Signed-off-by: camevor <camevor@nvidia.com>
Signed-off-by: camevor <camevor@nvidia.com>
Description
This PR removes the no-longer correct contact shape introduced for the contact sensors in favor of the mapping used elsewhere.
Before your PR is "Ready for review"
newton/tests/test_examples.py)pre-commit run -aSummary by CodeRabbit