Skip to content

Adds newton engine#4761

Merged
ooctipus merged 1 commit into
isaac-sim:developfrom
ooctipus:feature/develop/add_newton_engine
Mar 2, 2026
Merged

Adds newton engine#4761
ooctipus merged 1 commit into
isaac-sim:developfrom
ooctipus:feature/develop/add_newton_engine

Conversation

@ooctipus

Copy link
Copy Markdown
Collaborator

Description

This PR adds newton engine to isaaclab together with backend agnostic syntax
this pr happens after lazy import
Fixes # (issue)

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (existing functionality will not work without user modification)
  • Documentation update

Screenshots

Please attach before and after screenshots of the change if applicable.

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

@github-actions github-actions Bot added documentation Improvements or additions to documentation enhancement New feature or request asset New asset feature or request isaac-mimic Related to Isaac Mimic team labels Feb 27, 2026
@ooctipus ooctipus force-pushed the feature/develop/add_newton_engine branch from a4d008d to 9dbaa03 Compare March 1, 2026 10:36
@github-actions github-actions Bot added the isaac-lab Related to Isaac Lab team label Mar 1, 2026
@ooctipus ooctipus marked this pull request as ready for review March 1, 2026 10:49
@greptile-apps

greptile-apps Bot commented Mar 1, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds the Newton physics engine as an alternative backend to IsaacLab, implementing backend-agnostic abstractions throughout the core framework. The implementation includes a full-featured NewtonManager with support for three solver types (MuJoCo-Warp, XPBD, Featherstone), CUDA graph optimization, and contact sensor integration.

Key changes:

  • Implemented NewtonManager physics backend with multi-solver support and CUDA graph capture
  • Refactored callback systems to use physics manager events instead of omni.timeline (backend-agnostic)
  • Made backend detection case-insensitive across asset_base, sensor_base, and interactive_scene
  • Implemented newton_replicate() for USD prototype replication into Newton worlds
  • Renamed NewtonContactSensorCfg to ContactSensorCfg for consistency
  • Pinned mujoco-warp==3.5.0 and warp-lang==1.12.0rc2 dependencies

Issues found:

  • Critical: backend_utils._get_backend() will crash if SimulationContext.instance() returns None, despite docstring claiming fallback behavior
  • Minor: Unused variables in newton_replicate.py (newton_world_to_env_id dict, device parameter)

Confidence Score: 3/5

  • This PR introduces significant backend functionality with one critical bug that will cause crashes
  • Score reflects substantial new functionality (~1000 LOC) that is generally well-implemented, but contains a critical null-check bug in backend_utils._get_backend() that contradicts the documented fallback behavior. The Newton manager implementation appears solid with proper solver support, CUDA graphs, and contact handling. Minor cleanup needed for unused variables.
  • Pay close attention to source/isaaclab/isaaclab/utils/backend_utils.py - the _get_backend() method needs a null check before accessing SimulationContext.instance().physics_manager

Important Files Changed

Filename Overview
source/isaaclab/isaaclab/utils/backend_utils.py Added backend detection logic, but missing null check causes crash when SimulationContext not initialized
source/isaaclab/isaaclab/managers/manager_base.py Migrated from omni.timeline to physics manager callbacks with weakref safety wrapper
source/isaaclab/isaaclab/sensors/sensor_base.py Migrated to backend-agnostic callbacks, removed PhysX-specific imports and error handling
source/isaaclab_newton/isaaclab_newton/physics/newton_manager.py Full Newton physics manager implementation with multi-solver support, CUDA graphs, and contact sensors
source/isaaclab_newton/isaaclab_newton/cloner/newton_replicate.py Implemented USD prototype replication into Newton worlds, has unused dict and parameter
source/isaaclab_newton/isaaclab_newton/sensors/contact_sensor/contact_sensor.py Adapted contact sensor for Newton backend with inline mask creation and enum compatibility handling

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Application Start] --> B[SimulationContext.initialize]
    B --> C{Physics Backend?}
    C -->|PhysX| D[PhysxManager]
    C -->|Newton| E[NewtonManager]
    
    E --> F[NewtonManager.initialize]
    F --> G[Configure solver type]
    G -->|MuJoCo-Warp| H[SolverMuJoCo]
    G -->|XPBD| I[SolverXPBD]
    G -->|Featherstone| J[SolverFeatherstone]
    
    H & I & J --> K[start_simulation]
    K --> L[Dispatch MODEL_INIT event]
    L --> M[Finalize ModelBuilder]
    M --> N[Dispatch PHYSICS_READY event]
    N --> O{CUDA enabled?}
    O -->|Yes| P[Initialize CUDA graph]
    O -->|No| Q[Skip graphing]
    
    P & Q --> R[Ready for step loop]
    
    D --> S[PhysX initialization path]
    S --> R
    
    R --> T[FactoryBase._get_backend]
    T --> U{SimulationContext exists?}
    U -->|No| V[CRASH - Missing null check]
    U -->|Yes| W[Return backend name]
    
    style V fill:#f66
    style E fill:#6f6
    style H fill:#6f6
    style I fill:#6f6
    style J fill:#6f6
Loading

Last reviewed commit: 4777b98

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

20 files reviewed, 3 comments

Edit Code Review Agent Settings | Greptile

Comment on lines +40 to +52
"""Return active backend name for this factory.

Falls back to ``"physx"`` for backward compatibility when no simulation
context is initialized yet.
"""
# Import lazily to avoid import cycles at module load time.
manager_name = SimulationContext.instance().physics_manager.__name__.lower()
if "newton" in manager_name:
return "newton"
if "physx" in manager_name:
return "physx"
else:
raise ValueError(f"Unknown physics manager: {manager_name}")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docstring claims fallback to "physx" when no simulation context exists, but code doesn't implement this. If SimulationContext.instance() returns None (which is possible per the type signature), line 46 will raise AttributeError.

Suggested change
"""Return active backend name for this factory.
Falls back to ``"physx"`` for backward compatibility when no simulation
context is initialized yet.
"""
# Import lazily to avoid import cycles at module load time.
manager_name = SimulationContext.instance().physics_manager.__name__.lower()
if "newton" in manager_name:
return "newton"
if "physx" in manager_name:
return "physx"
else:
raise ValueError(f"Unknown physics manager: {manager_name}")
@classmethod
def _get_backend(cls, *args, **kwargs) -> str:
"""Return active backend name for this factory.
Falls back to ``"physx"`` for backward compatibility when no simulation
context is initialized yet.
"""
# Import lazily to avoid import cycles at module load time.
sim_ctx = SimulationContext.instance()
if sim_ctx is None or sim_ctx.physics_manager is None:
return "physx" # Backwards compatibility
manager_name = sim_ctx.physics_manager.__name__.lower()
if "newton" in manager_name:
return "newton"
if "physx" in manager_name:
return "physx"
else:
raise ValueError(f"Unknown physics manager: {manager_name}")

Comment on lines +60 to +64
newton_world_to_env_id = {}
for col, env_id in enumerate(env_ids.tolist()):
# begin a new world context (Newton assigns world ID = col)
builder.begin_world()
newton_world_to_env_id[col] = env_id

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

newton_world_to_env_id dict is created but never used. Can be removed.

@greptile-apps

greptile-apps Bot commented Mar 1, 2026

Copy link
Copy Markdown
Contributor
Additional Comments (1)

source/isaaclab_newton/isaaclab_newton/cloner/newton_replicate.py
device parameter is not used in the function body.

@ooctipus ooctipus force-pushed the feature/develop/add_newton_engine branch 5 times, most recently from 4880abc to c4563a2 Compare March 2, 2026 04:24
@ooctipus ooctipus force-pushed the feature/develop/add_newton_engine branch from c4563a2 to fe96fcd Compare March 2, 2026 04:50
@ooctipus ooctipus merged commit 77a17ab into isaac-sim:develop Mar 2, 2026
8 of 10 checks passed
daniela-hase pushed a commit to daniela-hase/IsaacLab that referenced this pull request Mar 30, 2026
# Description

This PR adds newton engine to isaaclab together with backend agnostic
syntax
this pr happens after lazy import
Fixes # (issue)

<!-- As a practice, it is recommended to open an issue to have
discussions on the proposed pull request.
This makes it easier for the community to keep track of what is being
developed or added, and if a given feature
is demanded by more than one party. -->

## Type of change

<!-- As you go through the list, delete the ones that are not
applicable. -->

- Bug fix (non-breaking change which fixes an issue)
- New feature (non-breaking change which adds functionality)
- Breaking change (existing functionality will not work without user
modification)
- Documentation update

## Screenshots

Please attach before and after screenshots of the change if applicable.

<!--
Example:

| Before | After |
| ------ | ----- |
| _gif/png before_ | _gif/png after_ |

To upload images to a PR -- simply drag and drop an image while in edit
mode and it should upload the image directly. You can then paste that
source into the above before/after sections.
-->

## Checklist

- [ ] I have read and understood the [contribution
guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html)
- [ ] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [ ] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

<!--
As you go through the checklist above, you can mark something as done by
putting an x character in it

For example,
- [x] I have done this task
- [ ] I have not done this task
-->
AntoineRichard added a commit that referenced this pull request Apr 17, 2026
## Summary

Fix incorrect attribute name `contact_margin` → `gap` on Newton
`ShapeConfig` in `NewtonManager.create_builder()`.

Newton PR #1732 renamed `contact_margin` to `gap` (broad-phase AABB
expansion). The IsaacLab code was never updated, creating a dead
attribute that Python silently accepted. The intended 1 cm default shape
gap was never applied.

## Newton PR #1732 rename mapping

| Old field | New field | Semantics |
|-----------|-----------|-----------|
| `thickness` | `margin` | Shape surface extension (affects contact
forces) |
| `contact_margin` | **`gap`** | Broad-phase AABB expansion (detection
range only) |
| `rigid_contact_margin` | `rigid_gap` | Builder-level default gap
(already 0.1) |

## Timeline

| Date | Newton | IsaacLab | `contact_margin` valid? |
|------|--------|----------|------------------------|
| Feb 19 | pin: `51ce35e` (has `contact_margin`) | #4646 adds
`contact_margin = 0.01` | Yes |
| Feb 24 | **PR #1732 renames `contact_margin` → `gap`** | — | — |
| Mar 2 | pin updated to `v0.2.3` (after rename) | #4761 keeps
`contact_margin` | **No — broken** |
| Mar 9 | — | #4883 removes the line | Removed |
| Apr 13 | — | #4822 re-adds `contact_margin` | **No — still broken** |

## Ablation: gap vs margin

We conducted an ablation study to understand the impact. Note: `margin`
(shape surface
extension) is a different field from `gap` (broad-phase range). The
original code intended
to set `gap`, but setting `margin` also has a significant positive
effect on training for
rough terrain locomotion.

| Setting | `gap` | `margin` | Go1 Reward (300 iter) | Effect |
|---------|-------|----------|----------------------|--------|
| Dead field (baseline) | 0.1 (default) | 0.0 | ~1.0 | — |
| `gap=0.01` only | 0.01 | 0.0 | 0.66 | No training improvement |
| `margin=0.01` only | 0.1 (default) | 0.01 | 18.77 | Major improvement
|
| `gap=0.01` + `margin=0.01` | 0.01 | 0.01 | 16.54 | Similar to
margin-only |

**This PR fixes the correct field migration** (`contact_margin` →
`gap`). The `margin` setting
for rough terrain contact quality will be addressed separately in the
rough terrain env PR
(#5248) via a new `default_shape_margin` config field on `NewtonCfg`.

## Test plan

- [x] Verified `contact_margin` is not a field on `ShapeConfig` (Newton
1.1.0.dev0)
- [x] Verified `gap` is the correct replacement field per Newton PR
#1732
- [x] Confirmed by camevor (Newton developer)
- [x] Ablation study: gap alone doesn't change training, so existing
tests should pass

Co-authored-by: Antoine RICHARD <antoiner@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

asset New asset feature or request documentation Improvements or additions to documentation enhancement New feature or request isaac-lab Related to Isaac Lab team isaac-mimic Related to Isaac Mimic team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants