mentors: @felixvd @mamoll @v4hn

This issue will bundle all relevant information about the GSoC project. A Discord channel is also available to discuss on this project. I will try to add pertinent information from Discord to this issue. This year, GSoC is a 10-week commitment of ~17 hours per week.

Motivation and goals

This GSoC project goal is to benchmark complete tests throughout the entire MoveIt pipeline and eventually the MoveIt Task Constructor (MTC). This will be done through a benchmark suite. Unit tests will also be benchmarked.

• Compare the impact of changes we consider integrating into MoveIt.
• Find regressions between MoveIt versions.
• Isolated tests or benchmarks don't necessarily reflect the entire MoveIt pipeline.
• Some algorithms do well in a particular environment but fall short in other contexts.

Planning

Phase 1: Exploration and benchmarks (June)

• Create multiple tests with a few scenes (empty scene, solid primitives, meshes, cluttered) with some joint/pose goals, path constraints, FCL and Bullet, planners (OMPL, CHOMP, STOMP) using the moveit_ros_benchmarks package.
• Pick and place benchmark using MTC.
• Analyze tests and decide whether we want to use only self contained tests or use and expand the moveit_ros_benchmarks when benchmarking planners throughout the entire MoveIt stack.
• Add an Amazon Picking Challenge (APC) scenario and do benchmarks on it.

Phase 2: Creation of the benchmark suite

• Implement the benchmark suite to run tests.
• Aggregate and plot results through Python.
• Add more tests and scenes.

MoveIt Benchmark Suite

It was agreed that the benchmark suite should be a separate repository. So there will be no pull-requests to MoveIt.

• Moveit Benchmark Suite
• Scene parser

Documentation

• Benchmark Tutorial
• Benchmark Visualizer
• Planner Arena

Links

• Proposal
• GSoC log
• Discord #gsoc2021