Skip to content

[Fix] Register Warp task variants in benchmark and environment scripts#5784

Merged
kellyguo11 merged 1 commit into
isaac-sim:developfrom
hujc7:jichuanh/benchmark-warp-tasks-import
May 27, 2026
Merged

[Fix] Register Warp task variants in benchmark and environment scripts#5784
kellyguo11 merged 1 commit into
isaac-sim:developfrom
hujc7:jichuanh/benchmark-warp-tasks-import

Conversation

@hujc7

@hujc7 hujc7 commented May 26, 2026

Copy link
Copy Markdown
Collaborator

1. Summary

  • Benchmark scripts and standalone environment helpers failed with gymnasium.error.NameNotFound: Environment 'Isaac-Cartpole-Direct-Warp' doesn't exist when invoked with any -Warp-v0 task. Docs at source/overview/core-concepts/physical-backends/newton/warp-environments.rst advertise the very command that fails.
  • Root cause: the affected scripts import isaaclab_tasks only; the -Warp-v0 variants are registered in the optional isaaclab_tasks_experimental extension. The RL training/play scripts already conditionally import it; these did not.
  • Fix: add the same with contextlib.suppress(ImportError): import isaaclab_tasks_experimental block after the existing import isaaclab_tasks. Seven scripts, 39 insertions total.

2. Scripts changed

File Before After
scripts/benchmarks/benchmark_rsl_rl.py imports isaaclab_tasks only + conditional isaaclab_tasks_experimental
scripts/benchmarks/benchmark_non_rl.py imports isaaclab_tasks only + conditional isaaclab_tasks_experimental
scripts/benchmarks/benchmark_rlgames.py imports isaaclab_tasks only + conditional isaaclab_tasks_experimental
scripts/environments/list_envs.py imports isaaclab_tasks only + conditional isaaclab_tasks_experimental
scripts/environments/random_agent.py imports isaaclab_tasks only + conditional isaaclab_tasks_experimental
scripts/environments/zero_agent.py imports isaaclab_tasks only + conditional isaaclab_tasks_experimental
scripts/environments/export_IODescriptors.py imports isaaclab_tasks only + conditional isaaclab_tasks_experimental

All eight scripts/reinforcement_learning/{rl_games,rsl_rl,sb3,skrl}/{train,play}.py scripts already follow this pattern; the affected scripts are the remaining gap.

3. Verification

  • Pre-commit clean on all 8 changed files (ruff, ruff-format, codespell, RST hooks).
  • Pattern mirrors scripts/reinforcement_learning/rsl_rl/train.py:56-58, which is the canonical placement.
  • contextlib.suppress(ImportError) keeps the scripts working when the experimental extension isn't installed (e.g. minimal IsaacLab installs).

4. Cherry-pick target

The customer-visible bug is on release/3.0.0-beta2 per the NVBug. After this lands on develop, cherry-pick onto that release branch so QA can verify on the same build.

Refs NVBug 6224916

Three benchmark scripts (benchmark_rsl_rl, benchmark_non_rl,
benchmark_rlgames) and four environment scripts (list_envs,
random_agent, zero_agent, export_IODescriptors) import
isaaclab_tasks but not isaaclab_tasks_experimental, so the -Warp-v0
task variants never register in gymnasium. Running any of them with
e.g. --task Isaac-Cartpole-Direct-Warp-v0 fails with
gymnasium.error.NameNotFound.

Add the same conditional import the RL launch scripts already use:

    import isaaclab_tasks
    with contextlib.suppress(ImportError):
        import isaaclab_tasks_experimental

The suppress block keeps the scripts working when the experimental
extension isn't installed.

Refs NVBug 6224916
@github-actions github-actions Bot added the isaac-lab Related to Isaac Lab team label May 26, 2026

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary

This is a clean and well-documented bug fix. The PR correctly addresses the missing task registration for -Warp-v0 variants in benchmark and environment scripts.

✅ What looks good

  • Follows established patterns: The conditional import pattern using contextlib.suppress(ImportError) matches exactly what the RL training scripts (e.g., scripts/reinforcement_learning/rsl_rl/train.py:56-58) already use. Good consistency.

  • Graceful degradation: When isaaclab_tasks_experimental is not installed, the scripts continue to work for non-Warp tasks. This maintains backward compatibility with minimal Isaac Lab installs.

  • Import placement: The experimental import is correctly placed after import isaaclab_tasks and before any from isaaclab_tasks.utils import ... statements, ensuring task registration happens before any task lookups.

  • Documentation: The commit message and PR description are thorough — root cause, fix rationale, verification steps, and cherry-pick guidance are all clearly documented.

  • Changelog fragment: Properly added with correct RST formatting.

📝 Minor observations

  1. PLACEHOLDER comment placement inconsistency: In scripts/environments/export_IODescriptors.py, the # PLACEHOLDER: Extension template comment appears after the new import block (line 37), while in the benchmark scripts it appears before (e.g., benchmark_non_rl.py line 88). This is cosmetic but worth noting for future consistency.

  2. Broken Links CI check: The CI shows a failing "Check for Broken Links" job. This appears unrelated to the changes in this PR (no documentation links were modified), but worth confirming it's a pre-existing issue.

Verdict

The fix is correct and follows established conventions. The code is ready to merge once CI completes (assuming the broken links issue is unrelated).

@hujc7 hujc7 marked this pull request as ready for review May 27, 2026 07:34
@hujc7 hujc7 requested review from kellyguo11 and ooctipus as code owners May 27, 2026 07:34
@greptile-apps

greptile-apps Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds the missing isaaclab_tasks_experimental conditional import to seven benchmark and environment scripts, mirroring the pattern already present in the RL training/play scripts. Without this, any invocation with a -Warp-v0 task variant fails with gymnasium.error.NameNotFound because Warp environment variants are only registered by the experimental extension.

  • Adds import contextlib and a with contextlib.suppress(ImportError): import isaaclab_tasks_experimental block to scripts/benchmarks/benchmark_{non_rl,rlgames,rsl_rl}.py and scripts/environments/{list_envs,random_agent,zero_agent,export_IODescriptors}.py.
  • Includes a changelog fragment documenting the fix for downstream release notes.
  • The three environment scripts (random_agent.py, zero_agent.py, export_IODescriptors.py) add the new block without the canonical # PLACEHOLDER: Extension template (do not remove this comment) prefix, leaving that comment orphaned at its original position further down each file.

Confidence Score: 4/5

The fix is straightforward and functionally correct; the only gap is a cosmetic comment placement inconsistency in three of the seven changed files.

All seven scripts now correctly acquire the experimental task registry when available, with the ImportError safely suppressed for minimal installs. Three environment scripts omit the PLACEHOLDER comment that the canonical pattern and the four other changed files include immediately above the suppress block; the comment is left orphaned further down each file.

scripts/environments/random_agent.py, scripts/environments/zero_agent.py, scripts/environments/export_IODescriptors.py — the PLACEHOLDER comment needs to be moved above the new import block to match the canonical pattern.

Important Files Changed

Filename Overview
scripts/benchmarks/benchmark_non_rl.py Adds import contextlib and the canonical PLACEHOLDER+suppress block after import isaaclab_tasks; correctly mirrors the RL training script pattern.
scripts/benchmarks/benchmark_rlgames.py Same canonical PLACEHOLDER+suppress import added correctly; no issues.
scripts/benchmarks/benchmark_rsl_rl.py Same canonical PLACEHOLDER+suppress import added correctly; no issues.
scripts/environments/list_envs.py Adds the PLACEHOLDER+suppress block correctly; no issues.
scripts/environments/random_agent.py Suppress block added without the PLACEHOLDER comment, leaving the existing PLACEHOLDER comment orphaned at line 39 after the import and other code.
scripts/environments/zero_agent.py Same PLACEHOLDER comment inconsistency as random_agent.py; suppress block added without moving or repeating the PLACEHOLDER comment.
scripts/environments/export_IODescriptors.py Same PLACEHOLDER comment inconsistency; suppress block added without the canonical comment prefix, and the PLACEHOLDER comment remains at its original later position.
source/isaaclab_tasks/changelog.d/jichuanh-benchmark-warp-tasks-import.rst New changelog fragment; concise and accurately describes the fix.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Script Startup] --> B[import isaaclab_tasks]
    B --> C{contextlib.suppress block}
    C -->|ImportError silenced| D[isaaclab_tasks_experimental unavailable\nMinimal install — Warp tasks not registered]
    C -->|Success| E[isaaclab_tasks_experimental imported\nWarp -Warp-v0 task variants registered]
    D --> F[gym.make task]
    E --> F
    F -->|Warp task + no experimental| G[NameNotFound error — FIXED by this PR]
    F -->|Standard task OR Warp + experimental| H[Environment created successfully]
Loading

Reviews (1): Last reviewed commit: "[Fix] Register Warp task variants in ben..." | Re-trigger Greptile

Comment on lines 15 to 19
import isaaclab_tasks # noqa: F401

with contextlib.suppress(ImportError):
import isaaclab_tasks_experimental # noqa: F401
from isaaclab_tasks.utils import (

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The experimental import block is added without the # PLACEHOLDER: Extension template (do not remove this comment) prefix that the canonical pattern requires (see scripts/reinforcement_learning/rsl_rl/train.py:56-58). The comment was already present further down the file (line 39) and now sits orphaned after the import it was meant to mark. The same issue exists in zero_agent.py and export_IODescriptors.py.

Suggested change
import isaaclab_tasks # noqa: F401
with contextlib.suppress(ImportError):
import isaaclab_tasks_experimental # noqa: F401
from isaaclab_tasks.utils import (
import isaaclab_tasks # noqa: F401
# PLACEHOLDER: Extension template (do not remove this comment)
with contextlib.suppress(ImportError):
import isaaclab_tasks_experimental # noqa: F401
from isaaclab_tasks.utils import (

Comment on lines 15 to 19
import isaaclab_tasks # noqa: F401

with contextlib.suppress(ImportError):
import isaaclab_tasks_experimental # noqa: F401
from isaaclab_tasks.utils import (

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Same missing # PLACEHOLDER: Extension template (do not remove this comment) prefix as in random_agent.py. The comment currently lives orphaned at line 39 after the block it should introduce.

Suggested change
import isaaclab_tasks # noqa: F401
with contextlib.suppress(ImportError):
import isaaclab_tasks_experimental # noqa: F401
from isaaclab_tasks.utils import (
import isaaclab_tasks # noqa: F401
# PLACEHOLDER: Extension template (do not remove this comment)
with contextlib.suppress(ImportError):
import isaaclab_tasks_experimental # noqa: F401
from isaaclab_tasks.utils import (

Comment on lines 35 to 39
import isaaclab_tasks # noqa: F401

with contextlib.suppress(ImportError):
import isaaclab_tasks_experimental # noqa: F401
from isaaclab_tasks.utils import parse_env_cfg

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Same missing # PLACEHOLDER: Extension template (do not remove this comment) prefix. The comment remains at its original position after parse_env_cfg, separated from the block it's meant to annotate.

Suggested change
import isaaclab_tasks # noqa: F401
with contextlib.suppress(ImportError):
import isaaclab_tasks_experimental # noqa: F401
from isaaclab_tasks.utils import parse_env_cfg
import isaaclab_tasks # noqa: F401
# PLACEHOLDER: Extension template (do not remove this comment)
with contextlib.suppress(ImportError):
import isaaclab_tasks_experimental # noqa: F401
from isaaclab_tasks.utils import parse_env_cfg

@kellyguo11 kellyguo11 merged commit e001e19 into isaac-sim:develop May 27, 2026
38 of 40 checks passed
kellyguo11 pushed a commit to kellyguo11/IsaacLab-public that referenced this pull request May 27, 2026
isaac-sim#5784)

## 1. Summary

- Benchmark scripts and standalone environment helpers failed with
``gymnasium.error.NameNotFound: Environment 'Isaac-Cartpole-Direct-Warp'
doesn't exist`` when invoked with any ``-Warp-v0`` task. Docs at
``source/overview/core-concepts/physical-backends/newton/warp-environments.rst``
advertise the very command that fails.
- Root cause: the affected scripts import ``isaaclab_tasks`` only; the
``-Warp-v0`` variants are registered in the optional
``isaaclab_tasks_experimental`` extension. The RL training/play scripts
already conditionally import it; these did not.
- Fix: add the same ``with contextlib.suppress(ImportError): import
isaaclab_tasks_experimental`` block after the existing ``import
isaaclab_tasks``. Seven scripts, 39 insertions total.

## 2. Scripts changed

| File | Before | After |
|---|---|---|
| ``scripts/benchmarks/benchmark_rsl_rl.py`` | imports
``isaaclab_tasks`` only | + conditional ``isaaclab_tasks_experimental``
|
| ``scripts/benchmarks/benchmark_non_rl.py`` | imports
``isaaclab_tasks`` only | + conditional ``isaaclab_tasks_experimental``
|
| ``scripts/benchmarks/benchmark_rlgames.py`` | imports
``isaaclab_tasks`` only | + conditional ``isaaclab_tasks_experimental``
|
| ``scripts/environments/list_envs.py`` | imports ``isaaclab_tasks``
only | + conditional ``isaaclab_tasks_experimental`` |
| ``scripts/environments/random_agent.py`` | imports ``isaaclab_tasks``
only | + conditional ``isaaclab_tasks_experimental`` |
| ``scripts/environments/zero_agent.py`` | imports ``isaaclab_tasks``
only | + conditional ``isaaclab_tasks_experimental`` |
| ``scripts/environments/export_IODescriptors.py`` | imports
``isaaclab_tasks`` only | + conditional ``isaaclab_tasks_experimental``
|

All eight
``scripts/reinforcement_learning/{rl_games,rsl_rl,sb3,skrl}/{train,play}.py``
scripts already follow this pattern; the affected scripts are the
remaining gap.

## 3. Verification

- Pre-commit clean on all 8 changed files (ruff, ruff-format, codespell,
RST hooks).
- Pattern mirrors
``scripts/reinforcement_learning/rsl_rl/train.py:56-58``, which is the
canonical placement.
- ``contextlib.suppress(ImportError)`` keeps the scripts working when
the experimental extension isn't installed (e.g. minimal IsaacLab
installs).

## 4. Cherry-pick target

The customer-visible bug is on ``release/3.0.0-beta2`` per the NVBug.
After this lands on ``develop``, cherry-pick onto that release branch so
QA can verify on the same build.

Refs NVBug 6224916
kellyguo11 added a commit that referenced this pull request May 28, 2026
… locomanip quaternion, deformable demo, IK, benchmark scripts (#5816)

# Description

Cherry pick following PRs from develop:

- #5779 
- #5760
- #5791
- #5810 
- #5811
- #5812 
- #5754 
- #5802 
- #5803 
- #5782
- #5804 
- #5744 
- #5775 
- #5784

---------

Signed-off-by: Kelly Guo <kellyg@nvidia.com>
Co-authored-by: John <jaybdub@users.noreply.github.com>
Co-authored-by: myurasov-nv <168484206+myurasov-nv@users.noreply.github.com>
Co-authored-by: Antoine RICHARD <antoiner@nvidia.com>
Co-authored-by: Mike Yan Michelis <46975745+mmichelis@users.noreply.github.com>
Co-authored-by: hujc <jichuanh@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants