Skip to content

[Docs] Expand Isaac Teleop XR performance optimization guide#5643

Merged
kellyguo11 merged 4 commits into
isaac-sim:developfrom
rwiltz:rwiltz/xr-perf-optimization-docs
May 16, 2026
Merged

[Docs] Expand Isaac Teleop XR performance optimization guide#5643
kellyguo11 merged 4 commits into
isaac-sim:developfrom
rwiltz:rwiltz/xr-perf-optimization-docs

Conversation

@rwiltz

@rwiltz rwiltz commented May 15, 2026

Copy link
Copy Markdown
Contributor

Description

Expands the Optimize XR Performance section of the Isaac Teleop feature guide
with the most common levers users reach for when XR teleop cannot sustain the
headset's display rate -- particularly on lower-spec GPUs or in heavy scenes.
What changed:

  • RTX - Minimal renderer: new dropdown explaining when to use it, how to
    enable it from the viewport renderer dropdown, the recommended Render
    Settings
    (Minimal Shading Mode = Diffuse/Glossy/Emission), and the
    current DistantLight-only lighting limitation, with a snippet showing how
    to swap a DomeLight for a DistantLight.
  • XR Resolution Multiplier slider: new dropdown describing the
    XR -> Advanced Settings -> Render Resolution slider for trading image
    sharpness for GPU headroom (0.8 as a sensible starting point).
  • Physics / render time step: refreshed to focus on Quest 3 / Pico 4 Ultra
    (90 Hz), explain what sim.render_interval actually controls, and call out
    the sim.dt stability/performance trade-off.
  • Removed the Try running physics on CPU dropdown -- this is already the
    default for these workflows.

Fixes # (issue)

Type of change

  • Documentation update

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

@rwiltz rwiltz requested a review from hougantc-nvda May 15, 2026 17:23
@github-actions github-actions Bot added documentation Improvements or additions to documentation isaac-lab Related to Isaac Lab team labels May 15, 2026

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Isaac Lab Review Bot

PR #5643 - [Docs] Expand Isaac Teleop XR performance optimization guide

Summary

This is a well-crafted documentation PR that significantly improves the XR performance optimization guide. The additions provide actionable guidance for users struggling with frame rate issues on lower-spec hardware.

✅ What Looks Good

  1. Clear technical explanations: The relationship between sim.dt, render_interval, and actual frame rates is well documented with practical recommendations.

  2. Modern hardware focus: Updated to reference Quest 3 and Pico 4 Ultra (90 Hz) rather than older devices.

  3. Complete RTX Minimal guidance: The new section thoroughly covers:

    • When to use it (lower-spec GPUs, complex scenes)
    • How to enable it (viewport dropdown + Render Settings)
    • The DistantLight-only limitation with a working code snippet
  4. Resolution multiplier documentation: Clear explanation of the tradeoff with a sensible starting point (0.8).

  5. Good decision to remove the "physics on CPU" dropdown since it is already the default.

  6. Proper assets: New screenshots follow the existing _static/teleop/ convention with descriptive names.

  7. Changelog fragment follows the expected format and references the appropriate section.

📋 Minor Item to Verify

  • The changelog fragment references :ref:\isaac-teleop-performance`. Please confirm this label exists in the RST file (likely around line 950 of isaac_teleop.rst`). If missing, the docs build will produce a warning.

Verdict

Looks good to merge once the docs CI completes successfully. Great work on making XR performance optimization more accessible! 🎮


🔄 Update (2026-05-15)

New commits add two more performance optimization dropdowns:

  1. Configure retargeting execution - Documents RetargetingExecutionConfig with clear sync vs pipelined mode guidance and DeadlinePacingConfig tuning.

  2. Check CloudXR frame pacing - Explains pacer behavior and provides actionable diagnostics via --cloudxr_env.

Both additions maintain the same high documentation quality. No issues found. ✅


🔄 Update (2026-05-16)

Latest SHA reviewed: 802206d321507cba2c3084867ea70887fe052c7b

Incremental Changes Since Last Review

Changes since 73d64c5:

  • Removed docs/isaaclab-docs.tar.gz — binary archive cleanup ✅
  • Removed docs/teleop_migration_2.3_to_3.0.md — migration guide removed (683 lines)

The core documentation improvements remain unchanged. The file removals appear to be intentional cleanup (the migration guide was likely a working document that does not belong in this docs PR, and the tarball was generated/transient).

No new issues found. PR remains good to merge. ✅

@greptile-apps

greptile-apps Bot commented May 15, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR expands the Optimize XR Performance section of the Isaac Teleop feature guide with five new dropdowns covering the RTX - Minimal renderer, the XR resolution-multiplier slider, sim.dt/render_interval trade-offs, RetargetingExecutionConfig sync/pipelined modes, and CloudXR frame-pacing diagnostics. Three annotated screenshot assets accompany the renderer and resolution sections.

  • RTX - Minimal renderer dropdown explains when to use it, how to enable it from the viewport, recommended Render Settings, and the DistantLight-only limitation with a Python swap snippet.
  • XR Resolution Multiplier and Retargeting execution dropdowns provide slider guidance and RetargetingExecutionConfig/DeadlinePacingConfig configuration patterns.
  • Physics/render time step section is refreshed for Quest 3 / Pico 4 Ultra (90 Hz) and the old CPU-physics dropdown is removed (already the default).

Confidence Score: 5/5

Documentation-only change with no runtime code modifications; safe to merge.

All changes are prose, reStructuredText markup, and screenshot assets. The code snippets in the new dropdowns match the actual API in isaac_teleop_cfg.py and the script CLI flags. The only finding is a minor documentation consistency gap in the retargeting snippet.

docs/source/features/isaac_teleop.rst — the retargeting execution snippet warrants a quick look for import completeness.

Important Files Changed

Filename Overview
docs/source/features/isaac_teleop.rst Core doc change — adds five new performance-optimization dropdowns; retargeting snippet is missing imports and context for where the assignment belongs
source/isaaclab_teleop/changelog.d/rwiltz-docs-rtx-minimal-renderer.rst Changelog fragment accurately summarising the new documentation sections; no issues
docs/source/_static/teleop/recommended-render-select.jpg New screenshot asset for the RTX - Minimal renderer dropdown selection
docs/source/_static/teleop/recommended-render-settings.jpg New screenshot asset for the Render Settings / Minimal Shading Mode panel
docs/source/_static/teleop/xr-resolution-slider.jpg New screenshot asset for the XR Resolution Multiplier slider

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A([XR teleop cannot sustain headset display rate]) --> B{Already tuned\nsim.dt & render_interval?}
    B -- No --> C[Set sim.dt = 1/90\nset render_interval = 2]
    C --> D{Still dropping\nframes?}
    B -- Yes --> D
    D -- No --> Z([Done])
    D -- Yes --> E[Switch viewport to\nRTX - Minimal renderer]
    E --> F[Set Minimal Shading Mode\n= Diffuse/Glossy/Emission]
    F --> G{Using DomeLight?}
    G -- Yes --> H[Swap/supplement with\nDistantLight]
    G -- No --> I{Still dropping\nframes?}
    H --> I
    I -- No --> Z
    I -- Yes --> J[Lower XR Resolution\nMultiplier e.g. 0.8]
    J --> K{Heavy retargeting?}
    K -- Python-light --> L[Use RetargetingExecutionConfig\nmode=sync]
    K -- Native/heavy --> M[Use RetargetingExecutionConfig\nmode=pipelined, tune safety_margin_s]
    L --> N{Frame-time\nspikes visible?}
    M --> N
    N -- Yes --> O[Override CloudXR frame pacer\nvia custom .env + --cloudxr_env]
    N -- No --> Z
    O --> Z
Loading

Reviews (2): Last reviewed commit: "Remove accidentally committed docs tarba..." | Re-trigger Greptile

Comment thread docs/source/features/isaac_teleop.rst
Comment thread docs/source/features/isaac_teleop.rst

@hougantc-nvda hougantc-nvda left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@isaaclab-review-bot

Copy link
Copy Markdown

🔄 Isaac Lab Review Bot - Incremental Review Update

Reviewing new commits: 92524e973d64c5

Summary of New Changes

This incremental update includes:

  1. Image format optimization: Converted three documentation screenshots from PNG to JPG format (recommended-render-select, recommended-render-settings, xr-resolution-slider) - reduces file sizes while maintaining visual clarity.

  2. Migration guide addition: Added comprehensive docs/teleop_migration_2.3_to_3.0.md (683 lines) - a detailed, machine-actionable guide for migrating custom teleop recording scripts and G1-based environment configs from Isaac Lab 2.3 to 3.0 with Isaac Teleop. This is excellent documentation that covers:

    • CLI argument changes (--enable_pinocchio removal, --teleop_device default change)
    • CloudXR CLI arguments
    • IsaacTeleop stack detection
    • Three-way device path branching
    • Context manager wrapping for simulation loop
    • Quaternion convention changes (WXYZ → XYZW)
    • Pink IK config updates
    • ProxyArray access patterns
  3. Changelog enhancement: Updated changelog.d/rwiltz-docs-rtx-minimal-renderer.rst to mention RetargetingExecutionConfig (sync vs pipelined modes), DeadlinePacingConfig.safety_margin_s, and CloudXR frame-pacing diagnostic.

  4. Documentation tarball: Added docs/isaaclab-docs.tar.gz for distribution.

Previous Feedback Status

The previous suggestion regarding the DistantLight code example (adding surrounding class skeleton context) remains applicable to the unchanged portion of isaac_teleop.rst.

Overall Assessment

✅ The incremental changes are well-structured and improve the documentation quality. The migration guide is particularly valuable for users upgrading their teleop implementations.


Automated review by Isaac Lab Review Bot

@rwiltz

rwiltz commented May 16, 2026

Copy link
Copy Markdown
Contributor Author

@greptile

@kellyguo11 kellyguo11 merged commit b541035 into isaac-sim:develop May 16, 2026
33 of 34 checks passed
matthewtrepte pushed a commit to matthewtrepte/IsaacLab that referenced this pull request May 18, 2026
…im#5643)

# Description

Expands the **Optimize XR Performance** section of the Isaac Teleop
feature guide
with the most common levers users reach for when XR teleop cannot
sustain the
headset's display rate -- particularly on lower-spec GPUs or in heavy
scenes.
What changed:
- **RTX - Minimal renderer**: new dropdown explaining when to use it,
how to
enable it from the viewport renderer dropdown, the recommended **Render
Settings** (**Minimal Shading Mode = Diffuse/Glossy/Emission**), and the
current `DistantLight`-only lighting limitation, with a snippet showing
how
  to swap a `DomeLight` for a `DistantLight`.
- **XR Resolution Multiplier slider**: new dropdown describing the
**XR -> Advanced Settings -> Render Resolution** slider for trading
image
  sharpness for GPU headroom (`0.8` as a sensible starting point).
- **Physics / render time step**: refreshed to focus on Quest 3 / Pico 4
Ultra
(90 Hz), explain what `sim.render_interval` actually controls, and call
out
  the `sim.dt` stability/performance trade-off.
- Removed the **Try running physics on CPU** dropdown -- this is already
the
  default for these workflows.

Fixes # (issue)

## Type of change

- Documentation update

## Checklist

- [x] I have read and understood the [contribution
guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html)
- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants