Skip to content

Adds multirotor vision-based navigation task and acceleration, velocity and position controllers #3895

Merged
kellyguo11 merged 523 commits into
isaac-sim:developfrom
ntnu-arl:feature/drone
May 8, 2026
Merged

Adds multirotor vision-based navigation task and acceleration, velocity and position controllers #3895
kellyguo11 merged 523 commits into
isaac-sim:developfrom
ntnu-arl:feature/drone

Conversation

@Zwoelf12

Copy link
Copy Markdown
Contributor

Description

This PR extends 3760 by introducing a navigation tasks for the ARL robot. The PR adds a confined cluttered environment, adds acceleration, velocity and position controllers + configs, extends the MDP and RL configs and adds a Variational Auto Encoder to generate image latents for observations. The PR depends on the MultiMeshRayCasterCamera introduced in PR 3298 (currently not merged in IsaacLab).

Changes

Type of Change

  • New feature (non-breaking change which adds functionality)
  • Documentation update (added docs/comments where applicable)

Files changed (high-level summary)

  • New files added:
    • source/isaaclab_tasks/isaaclab_tasks/manager_based/drone_arl/navigation/* (new task code and config, obstacle scene code and config)
    • source/isaaclab_tasks/isaaclab_tasks/manager_based/drone_arl/mdp/vae_model.pt
    • source/isaaclab/isaaclab/controllers/lee_acceleration_control_cfg.py
    • source/isaaclab/isaaclab/controllers/lee_acceleration_control.py
    • source/isaaclab/isaaclab/controllers/lee_velocity_control_cfg.py
    • source/isaaclab/isaaclab/controllers/lee_velocity_control.py
    • source/isaaclab/isaaclab/controllers/lee_position_control_cfg.py
    • ource/isaaclab/isaaclab/controllers/lee_position_control.py
  • Modified:
    • source/isaaclab_tasks/isaaclab_tasks/manager_based/drone_arl/mdp/* (added navigation specifics)
    • source/isaaclab/isaaclab/envs/mdp/actions/actions_cfg.py (added new action config)
    • source/isaaclab/isaaclab/envs/mdp/actions/thrust_actions.py (introduced new navigation action handling controller application)
  • Total diff (branch vs main, includes also unmerged changes of PR 3760 and PR 3298): 74 files changed, 8029 insertions, 88 deletions

Dependencies

  • The new drone task references standard repo-internal packages and Isaac Sim; no external pip packages required beyond the repo standard.

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

@kellyguo11 kellyguo11 moved this from In review to In progress in Isaac Lab Apr 17, 2026
@pascal-roth

Copy link
Copy Markdown
Collaborator

@AntoineRichard @kellyguo11 moved the pt file to nuclues, lets get that PR merged

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Isaac Lab Review Bot

Summary

This is a follow-up review after new commits. The previous critical findings have been addressed.

Previous Concerns Status

Addressed: NavigationAction.reset() inverted conditional — now correctly uses if env_ids is None: env_ids = slice(None) after calling parent reset.

Addressed: ImageLatentObservation.__call__ now uses .clone() on the sensor data to avoid mutating the shared buffer.

Addressed: ImageLatentObservation.__init__ now correctly uses bool(cfg.params.get(...)) instead of tuple assignment.

Addressed: lee_attitude_control.py docstring now correctly documents [thrust, roll, pitch, yaw_rate] matching the implementation.

Addressed: lee_attitude_control_cfg.py now includes class_type: type = LeeAttController.

Implementation Verdict

Ship it — All critical bugs have been fixed. The remaining items from the previous review (improvement suggestions about variable naming, hover edge-case guards, and max_inclination_angle_rad being unused in some configs) are minor and don't block merge.

CI Status

Most CI checks still pending. Recommend waiting for isaaclab_contrib and isaaclab_tasks test suites to pass before merge.

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Isaac Lab Review Bot

Summary

This is a follow-up review. All previous critical concerns have been addressed, and no new issues are introduced in the latest commits.

Previous Concerns Status

✅ All previously identified critical bugs (inverted conditional, missing .clone(), incorrect param parsing, docstring mismatch, missing class_type) have been fixed.

Implementation Verdict

Ship it — The code is correct and ready for merge.

CI Status

CI labeler passed. Recommend confirming isaaclab_contrib and isaaclab_tasks test suites pass before final merge.

Comment thread docs/source/overview/environments.rst Outdated
Comment thread docs/source/overview/environments.rst Outdated

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Isaac Lab Review Bot

Summary

This is a follow-up review. The previous review gave a "Ship it" verdict, and the latest commit (4c0a281) contains no substantive code changes that would alter that assessment.

Implementation Verdict

Ship it — No new issues introduced. The code remains correct and ready for merge.

CI Status

CI labeler passed. All previous concerns remain addressed.

@kellyguo11

Copy link
Copy Markdown
Contributor

seeing some doc failures

/home/runner/work/IsaacLab/IsaacLab/docs/source/overview/environments.rst:754: ERROR: Error parsing content block for the "list-table" directive: uniform two-level bullet list expected, but row 53 does not contain the same number of items as row 1 (4 vs 5).

@kellyguo11 kellyguo11 merged commit 1336acd into isaac-sim:develop May 8, 2026
32 of 34 checks passed
@github-project-automation github-project-automation Bot moved this from In progress to Done in Isaac Lab May 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

asset New asset feature or request documentation Improvements or additions to documentation enhancement New feature or request infrastructure isaac-lab Related to Isaac Lab team

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

7 participants