Add semantic segmentation tutorial#876
Conversation
feac7fb to
bdcfb78
Compare
SteveMacenski
left a comment
There was a problem hiding this comment.
Really love this, thank you! ❤️
…fig example (ros-navigation#874) Signed-off-by: EricoMeger <ericomeger9@gmail.com> Signed-off-by: pepisg <pedro.gonzalez@eia.edu.co>
* Add docs Signed-off-by: Tony Najjar <tony.najjar@dexory.com> * revert Signed-off-by: Tony Najjar <tony.najjar@dexory.com> * . Signed-off-by: Tony Najjar <tony.najjar@dexory.com> * add console Signed-off-by: Tony Najjar <tony.najjar@dexory.com> --------- Signed-off-by: Tony Najjar <tony.najjar@dexory.com> Signed-off-by: pepisg <pedro.gonzalez@eia.edu.co>
…gation#871) * Update velocity polygon linear_min/max Description Update collision monitor tutorial for holonomic robot Signed-off-by: Jaerak Son <sjr9017@naver.com> * [collision_monitor] Update velocity polygon angles and refine documentation - Updated `direction_start_angle` and `direction_end_angle` parameters in YAML to match the reference diagram. - Refined `using_collision_monitor.rst` based on review feedback (removed redundant non-negative description). Signed-off-by: Jaerak Son <sjr9017@naver.com> --------- Signed-off-by: Jaerak Son <sjr9017@naver.com> Signed-off-by: pepisg <pedro.gonzalez@eia.edu.co>
Signed-off-by: pepisg <pedro.gonzalez@eia.edu.co>
Signed-off-by: pepisg <pedro.gonzalez@eia.edu.co>
Signed-off-by: pepisg <pedro.gonzalez@eia.edu.co>
Signed-off-by: pepisg <pedro.gonzalez@eia.edu.co>
Signed-off-by: pepisg <pedro.gonzalez@eia.edu.co>
Co-authored-by: Steve Macenski <stevenmacenski@gmail.com> Signed-off-by: Pedro Alejandro González <71234974+pepisg@users.noreply.github.com> Signed-off-by: pepisg <pedro.gonzalez@eia.edu.co>
Co-authored-by: Steve Macenski <stevenmacenski@gmail.com> Signed-off-by: Pedro Alejandro González <71234974+pepisg@users.noreply.github.com> Signed-off-by: pepisg <pedro.gonzalez@eia.edu.co>
Co-authored-by: Steve Macenski <stevenmacenski@gmail.com> Signed-off-by: Pedro Alejandro González <71234974+pepisg@users.noreply.github.com> Signed-off-by: pepisg <pedro.gonzalez@eia.edu.co>
98b19d5 to
9a78479
Compare
SteveMacenski
left a comment
There was a problem hiding this comment.
It looks like you have pulled in other changes from other commits? Can you resolve this :-)
I again very much appreciate your support and help in documentation and leaving breadcrumbs for the next person to do things like GPS navigation and semantic segmentation. If you have anything else in mind, I'd love to hear it :-)
tutorials/index.rst
Outdated
|
|
||
| docs/navigation2_on_real_turtlebot3.rst | ||
| docs/navigation2_with_slam.rst | ||
| docs/navigation2_with_semantic_segmentation.rst |
There was a problem hiding this comment.
Can you move this down next to the isaac tutorial? These tutorials are in no way really ordered, but I did try at least to have the most basic at top first
There was a problem hiding this comment.
PS @AJedancov for the doc updates, maybe it would be good to use those grids/tiles for the tutorials page instead of just a list!
| Overview | ||
| ======== | ||
|
|
||
| This tutorial demonstrates how to use semantic segmentation in costmaps with stereo cameras, using a custom `semantic_segmentation_layer plugin <https://github.com/ros-navigation/navigation2_tutorials/tree/master/nav2_semantic_segmentation_layer>`_ and a pre-trained segmentation model that works on Gazebo's Baylands world. It was written by Pedro Gonzalez at `robot.com <https://robot.com/>`_. |
Co-authored-by: Steve Macenski <stevenmacenski@gmail.com> Signed-off-by: Pedro Alejandro González <71234974+pepisg@users.noreply.github.com>
…mantic-segmentation
I guess GPS could be added to the tutorial. The baylands world aligns with the real location in the bay area 😄 : https://maps.app.goo.gl/4jwUKYkMKdbiSW5s8. Thought It would be better to keep the tutorial simple having localization as a given, but i guess we could always point back the gps tutorial. Off the top of my head, this would probably make us:
I could do that in another PR It could be interesting to extend also with monocular depth estimation. I've seen a couple wrappers of depth anything 3, not sure though how they would work on simulation images, and I think that would probably be GPU only due to the model overhead, but I guess it could be tested out. Finally I'd love to personally work on making a neural network + costmap layer that produces traversability on the ground plane end to end (monocular camera -> local costmap -> layer to mostly propagate odometry). Though this will probably take time and experimentation. It could also be interest to try to build a demo/layer using pre-trained occupancy networks, but that also could be quite GPU expensive. If you know about folks with similar interests we could connect through slack. Thanks a lot for your review and your interest on getting this into nav2! hope people find it useful. |
Goes together with ros-navigation/navigation2_tutorials#136