Update Visualizer Integration Tests + Some Visualizer Fixes#5733
Conversation
There was a problem hiding this comment.
🔍 Isaac Lab Code Review
Thank you for this substantial PR improving the visualizer integration tests! I've conducted a thorough review focusing on test quality, Isaac Lab conventions, and potential issues.
✅ What's Working Well
-
Comprehensive Test Refactoring: The new
test_visualizer_integration.pyis significantly more robust than the previous implementation, with better structure and more thorough coverage of play/pause/resume cycles. -
Multi-Backend Testing: The parameterized tests covering both PhysX and Newton MJWarp backends are well-designed and follow pytest best practices.
-
Debug Infrastructure: The debug image capture system (
_WRITE_VIS_DEBUG_FRAMES,_save_visualizer_debug_image) is a valuable addition for diagnosing flaky tests. -
Clean Config Reorganization (new in 6055e74): The
VisualizerCfgfields are now logically grouped with section comments (Primary interactive camera, Tiled camera settings, Partial visualization, Markers, Live Plots, Internal). This improves readability significantly. -
Improved UI Naming: The Newton visualizer's ImageLogger controls renamed to "Tiled Camera View" with a dedicated
_draw_tiled_camera_view_controlsmethod provides a clearer user experience. -
Proper Test Separation (new in 6055e74): Splitting tiled camera tests into dedicated
test_*_tiled_camera_motionfunctions is a much cleaner architecture than mixing assertions in the main motion tests.
✅ All Issues Resolved (dc7e30f)
The latest commit addresses all previously flagged items:
-
_WRITE_VIS_DEBUG_FRAMES = False— ✅ Now set toFalseby default invisualizer_integration_utils.py -
Pyglet headless configuration — ✅ Handled automatically via
NewtonVisualizerCfg(headless=True)which triggersruntime_headlessdetection innewton_visualizer.py -
Rerun import guard — ✅ Now uses deferred imports inside
_get_visualizer_cfg()with properfrom isaaclab_visualizers.rerun import ...pattern
📋 Changes in This Update (395ee63 → dc7e30f)
| File | Change |
|---|---|
visualizer_integration_utils.py |
✅ New shared module with all test helpers; _WRITE_VIS_DEBUG_FRAMES = False |
test_visualizer_integration_{newton,physx}.py |
✅ Backend-specific test entry points |
test_visualizer_tiled_integration_{newton,physx}.py |
✅ Dedicated tiled camera test entry points |
newton_visualizer.py |
✅ Cross-platform headless detection (sys.platform not in (\"win32\", \"darwin\")) |
test_visualizer_cartpole_integration.py |
🗑️ Removed (replaced by new modular structure) |
🏁 Summary
All blockers resolved! The PR is ready for merge.
- Set
_WRITE_VIS_DEBUG_FRAMES = False - Pyglet headless handled via cfg
- Deferred rerun imports
Excellent refactoring work — the new modular test structure with separate backend/tiled test files is much cleaner.
📝 Update Review (dc7e30f)
This update includes additional substantial changes beyond the visualizer test improvements:
✅ New Changes Reviewed
1. Newton Sensors Stale Data Fix (Issue #4970) — Excellent fix!
- Added timestamp guards to all Newton sensor kernels (
ContactSensor,Imu,Pva,JointWrenchSensor) - Prevents reading stale pre-reset physics data when
scene.reset(env_ids)is called without a subsequent physics step - Comprehensive regression tests added for each sensor type
- PhysX sensors also updated with matching timestamp guards
2. OVRTX Renderer Cleanup
- Removed legacy OVRTX 0.2.x code paths (good cleanup for forward compatibility)
- Removed
_IS_OVRTX_0_3_0_OR_NEWERchecks and legacy kernels - Simplified
create_scene_partition_attributes()signature
3. PhysX Scene-Data Rigid Body View Fix
- Changed from wildcard patterns to exact prim paths
- Avoids spurious warnings for assets with joint prims sharing body names
4. PPISP Camera Discovery Fix
- Fixed
auto_camera_ppisp_cfg()to continue scanning when generated RenderProducts lack PPISP child - Added test case for this edge case
5. Kitless Launch Handling
- Fixed explicit
--viz nonehandling - Properly detects
visualizer_explicit=Truewithvisualizer=Nonecase
6. Visualizer Test Refactoring
- Tests properly split into separate files per physics backend
- Tiled camera tests in dedicated files
- Shared utilities in
visualizer_integration_utils.py
7. Documentation Updates
- Removed "experimental" statement from
isaaclab_ovpackage docs - Added comprehensive changelogs for all fixes
✅ Code Quality
All changes follow Isaac Lab conventions:
- Proper changelog entries in
changelog.d/ - Clear docstrings referencing issue numbers
- Comprehensive test coverage for all fixes
- Clean separation of concerns
No new issues identified. The PR continues to be ready for merge.
Automated review by Isaac Lab Review Bot | Updated for commit dc7e30f
📝 Update Review (90c5a6e)
The latest commits add a substantial pyproject.toml migration across the entire Isaac Lab codebase. This is a clean modernization effort.
✅ Migration Changes Reviewed
1. Build System Migration
- All
setup.pyfiles removed across 18+ subpackages - Package metadata moved to
pyproject.toml[project]sections - Build requirements simplified: removed
tomldependency from build-system - Uses
setuptools>=70.0,<82.0.0consistently
2. Runtime Dependency Updates
- Replaced third-party
tomlwith stdlibtomllib(Python 3.11+) SimulationContextand tests now usetomllibfor reading.kitpreset files__version__now usesimportlib.metadata.version()pattern consistently
3. Test Dependencies Refactored
- Test packages (
pytest,pytest-mock,junitparser,flatdict,flaky,coverage>=7.6.1) moved from base deps to[test]optional extra - Docker images updated to install
pytestexplicitly - CI action (
.github/actions/run-tests/action.yml) now installs test deps before running
4. Other Fixes
- PhysX Rigid Body View: Restored wildcard patterns (
/World/envs/env_*) for scene-data rigid body view — fixes Newton visualizers not updating live PhysX transforms - License Exception: Updated
typing_extensionslicense fromPSF-2.0toPython Software Foundation License - Changelog CLI: Added version sync check between
extension.tomlandpyproject.toml
5. Minor Code Cleanup
install.pyreferences updated fromsetup.pytopyproject.toml- Observation manager test fixed for whitespace sensitivity in table rendering
- Benchmark utils updated to use
pyproject.tomlas repo marker - isaaclab_rl extra names normalized to PEP 685 hyphenated forms (
rsl-rl,rl-games)
✅ Quality Assessment
All changes follow Isaac Lab conventions:
- Proper changelog entries in
changelog.d/ - Consistent patterns applied across all packages
- Good use of
importlib.metadatafor version detection withPackageNotFoundErrorfallback - Tests updated to match new structure
No issues found. This is a well-executed modernization.
Automated review by Isaac Lab Review Bot | Updated for commit 90c5a6e
📝 Update Review (5b41c70)
Documentation-only update adding known issue documentation for Newton + Digit robots.
✅ Changes Reviewed
1. Newton Closed-Loop Articulation Limitation
- Added detailed known issue to
docs/source/refs/issues.rstexplaining why Digit robots don't work on Newton - The root cause (duplicate link slots in
ArticulationViewfor bodies that are children of multiple joints) is well-documented - Workaround provided: use
physxpreset until upstream fix lands
2. Environment Table Updates
- Updated
docs/source/overview/environments.rstto removenewton_mjwarpfrom supported physics presets for Digit environments - Affects:
Isaac-Velocity-Flat-Digit-v0,Isaac-Velocity-Rough-Digit-v0,Isaac-Tracking-LocoManip-Digit-v0
3. Changelog Skip Marker
- Added
changelog.d/octi-docs-digit-newton-known-limitation.skipappropriately for docs-only change
✅ Quality Assessment
- Documentation is accurate and technically detailed
- Follows Isaac Lab documentation conventions
- Properly uses changelog skip marker for docs-only changes
No issues found. PR remains ready for merge.
Automated review by Isaac Lab Review Bot | Updated for commit 5b41c70
Greptile SummaryThis PR replaces the old
Confidence Score: 3/5Two test-file issues need fixing before merging: the debug PNG write flag is enabled by default and the pyglet headless guard was dropped, both of which can cause unexpected CI failures. The debug flag being left as True will write PNG files on every CI run and adds an implicit Pillow requirement; the missing pyglet headless setup is a regression from the deleted file that the original comment explicitly called out as required for headless Newton ViewerGL. Both issues are in the test path, not production code, but can cause CI runs to fail or behave unexpectedly. source/isaaclab_visualizers/test/test_visualizer_integration.py needs the debug flag corrected and the pyglet headless setup restored; source/isaaclab/isaaclab/scene/scene_data_provider.py warrants a second look at how non-env-path cameras are handled in _walk_camera_prims. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["KitVisualizer._setup_camera_sensor_view()"] --> B{uses_camera_sensor_view?}
B -- No --> Z[return]
B -- Yes --> C{cameras_enabled?}
C -- No --> ERR[raise RuntimeError]
C -- Yes --> D[Create / resolve camera sensor]
D --> E{runtime_headless?}
E -- No --> F[_setup_camera_image_window]
E -- Yes --> G[Log: running without image panel]
H["KitVisualizer._update_camera_image_panel()"] --> I{camera_sensor is None?}
I -- Yes --> Z2[return]
I -- No --> J[Update owned camera poses]
J --> K[camera_sensor.update]
K --> L{camera_image_provider is None?}
L -- Yes --> Z3[return]
L -- No --> M[camera_rgb_batch → upload to panel]
Reviews (1): Last reviewed commit: "sync" | Re-trigger Greptile |
| "isaaclab.sim.simulation_context", | ||
| ) | ||
|
|
||
| _WRITE_VIS_DEBUG_FRAMES = True |
There was a problem hiding this comment.
Debug flag enabled in test code — contradicts PR description
The PR description states that debug image capture is "disabled by default," but _WRITE_VIS_DEBUG_FRAMES = True means every CI run will write PNG files to logs/viz_integration_captures. This also introduces a hard runtime dependency on PIL (imported inside _save_visualizer_debug_image); if Pillow is not in the test dependencies, every test that reaches frame-capture paths will raise an ImportError.
| _WRITE_VIS_DEBUG_FRAMES = True | |
| _WRITE_VIS_DEBUG_FRAMES = False | |
| """Whether to emit visualizer debug PNGs during integration tests.""" |
| from __future__ import annotations | ||
|
|
||
| import warp as wp | ||
|
|
||
| from isaaclab.app import AppLauncher | ||
|
|
||
| # launch Kit app | ||
| simulation_app = AppLauncher(headless=True, enable_cameras=True).app |
There was a problem hiding this comment.
Missing
pyglet headless configuration required for Newton ViewerGL
The deleted test_visualizer_cartpole_integration.py contained an explicit comment explaining this requirement: "Pyglet must use HeadlessWindow (EGL) before pyglet.window is imported so Newton ViewerGL can construct without an X11 display." The new file omits import pyglet; pyglet.options["headless"] = True before AppLauncher. On headless CI machines without X11/Wayland, pyglet may fall back to a crashing backend at window-construction time, causing Newton visualizer tests to fail with a display error.
| from __future__ import annotations | |
| import warp as wp | |
| from isaaclab.app import AppLauncher | |
| # launch Kit app | |
| simulation_app = AppLauncher(headless=True, enable_cameras=True).app | |
| from __future__ import annotations | |
| import warp as wp | |
| # Pyglet must use HeadlessWindow (EGL) before ``pyglet.window`` is imported so Newton | |
| # ViewerGL can construct without an X11 display (matches ``headless=True`` on NewtonVisualizerCfg). | |
| import pyglet | |
| pyglet.options["headless"] = True | |
| from isaaclab.app import AppLauncher | |
| # launch Kit app | |
| simulation_app = AppLauncher(headless=True, enable_cameras=True).app |
| if visualizer_kind == "rerun": | ||
| __import__("newton") | ||
| from isaaclab_visualizers.rerun import RerunVisualizer, RerunVisualizerCfg | ||
|
|
||
| web_port, grpc_port = _allocate_rerun_test_ports(host="127.0.0.1") |
There was a problem hiding this comment.
Missing
__import__("rerun") guard inconsistent with viser handling
For viser (line 255) the code guards with __import__("viser") before the lazy import, allowing a clean skip if the package is absent. The __import__("rerun") guard present in the deleted file was removed, so a missing rerun package now produces an ImportError from the from isaaclab_visualizers.rerun import ... line rather than a clean skip signal.
| Args: | ||
| stage: USD stage to traverse, or ``None``. | ||
|
|
||
| Returns: | ||
| Dictionary with keys ``order`` (template prim paths using ``env_%d``), | ||
| ``positions``, ``orientations`` (per-camera, per-env, with ``None`` for | ||
| absent envs), and ``num_envs``. Returns ``None`` when ``stage`` is ``None``. | ||
| """ | ||
| if stage is None: | ||
| return None | ||
|
|
There was a problem hiding this comment.
Non-env-path cameras silently assigned
world_id=0
In _walk_camera_prims, world_id is reset to 0 at the top of each iteration. Any camera prim whose path does not match _ENV_PATH_RE (e.g. /World/Camera) gets world_id=0, and the if world_id > num_envs: num_envs = world_id branch updates num_envs from -1 to 0. After num_envs += 1 the result is 1, so that global camera is placed into per_world_pos[0] — indistinguishable from an env_0 camera. Callers iterating the result by environment index will silently receive a global camera pose as if it belongs to env_0.
🤖 Incremental Review Update (395ee63)Changes since last review (004a12c): Summary of New ChangesThis update continues the CI/test infrastructure improvements with the following key changes: 1. New
|
8db166f to
742b5aa
Compare
26f8d6d to
c4af93a
Compare
…m#5733) <!-- Thank you for your interest in sending a pull request. Please make sure to check the contribution guidelines. Link: https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html 💡 Please try to keep PRs small and focused. Large PRs are harder to review and merge. --> Improve the visualizer integration test - Unskip all tests - Add retry loop for pause/replay step to reduce flakyness - Separate out visualizer tests into separate files and kit processes - Fix camera pose of Kit Viz + Newton_MJWarp case - Add debug image capture mode (disabled by default) - Add simple tiled image check Other visualizer fixes - Pin pyarrow to 22.0.0 to fix Rerun on Windows - Make tiled camera view instructions in the visualizer docs more explicit - Fix Newton Pyglet issue on Windows - Change tiled camera panel in Newton Viewer from "Image Logger (1)" to "Tiled Visualizer Camera" <!-- As a practice, it is recommended to open an issue to have discussions on the proposed pull request. This makes it easier for the community to keep track of what is being developed or added, and if a given feature is demanded by more than one party. --> <!-- As you go through the list, delete the ones that are not applicable. --> - Bug fix (non-breaking change which fixes an issue) - New feature (non-breaking change which adds functionality) Please attach before and after screenshots of the change if applicable. <!-- Example: | Before | After | | ------ | ----- | | _gif/png before_ | _gif/png after_ | To upload images to a PR -- simply drag and drop an image while in edit mode and it should upload the image directly. You can then paste that source into the above before/after sections. --> - [ ] I have read and understood the [contribution guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html) - [ ] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format` - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have updated the changelog and the corresponding version in the extension's `config/extension.toml` file - [ ] I have added my name to the `CONTRIBUTORS.md` or my name already exists there <!-- As you go through the checklist above, you can mark something as done by putting an x character in it For example, - [x] I have done this task - [ ] I have not done this task -->
… teleop/mimic dependency (#5829) # Description Cherry pick bug fix PRs from develop: - #5821 - #5820 - #5733 - #5824 --------- Signed-off-by: peterd-NV <peterd@nvidia.com> Co-authored-by: peterd-NV <peterd@nvidia.com> Co-authored-by: ooctipus <zhengyuz@nvidia.com> Co-authored-by: matthewtrepte <mtrepte@nvidia.com>
Description
Improve the visualizer integration test
Other visualizer fixes
Type of change
Screenshots
Please attach before and after screenshots of the change if applicable.
Checklist
pre-commitchecks with./isaaclab.sh --formatconfig/extension.tomlfileCONTRIBUTORS.mdor my name already exists there