Skip to content

Hotfix: Add backwards compatibility for old scene serialization format#2986

Merged
v4hn merged 5 commits intomoveit:masterfrom
wxmerkt:hotfix/backwards-compatibility-for-old-scene-format
Dec 10, 2021
Merged

Hotfix: Add backwards compatibility for old scene serialization format#2986
v4hn merged 5 commits intomoveit:masterfrom
wxmerkt:hotfix/backwards-compatibility-for-old-scene-format

Conversation

@wxmerkt
Copy link
Copy Markdown
Contributor

@wxmerkt wxmerkt commented Dec 8, 2021

This PR adds backwards compatibility for the old scene serialization format to PlanningScene following #2037, cf. the regression report for Noetic in #2985. It first checks which version of the syntax is provided (by checking the line following the start of a new object - previously the number of shapes, now the translational component of the pose) and deactivates the subframe-poses if using the old format, as well as maintains compatibility by resetting the pose to identity.

@felixvd @rhaschke Can you please review this PR, backport it to noetic-devel, and release? :-)

I've added unittests for the old format as well. Detail on changes/logic in the individual commits.

Resolves #2985

…e format

The serialization format for the .scene files changed in
moveit#2037. This commits a
testcase using the old scene format. It will fail and a subsequent
commit to introduce backwards compatibility to the scene-file parsing
will make it pass.
…e version format

This commit adds a mechanism for detecting the version of the scene file
format to enable the loadGeometryFromStream method to read old version
scene files without having to migrate them. To detect the version of the
scene format, we use the content of the line following the start of an
object: In the old version of the format, this specified the number of
shapes in the object (a single int). In the new version of the format,
it is the translational part of the pose of the object (i.e. three
double values separated by spaces). To detect the format, we check for
the number of spaces after trimming the string.
@codecov
Copy link
Copy Markdown

codecov bot commented Dec 8, 2021

Codecov Report

Merging #2986 (a41d0ff) into master (32a1883) will increase coverage by 0.03%.
The diff coverage is 73.92%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2986      +/-   ##
==========================================
+ Coverage   61.57%   61.59%   +0.03%     
==========================================
  Files         370      370              
  Lines       33151    33162      +11     
==========================================
+ Hits        20408    20424      +16     
+ Misses      12743    12738       -5     
Impacted Files Coverage Δ
moveit_core/planning_scene/src/planning_scene.cpp 64.33% <73.92%> (+0.25%) ⬆️
...g_scene_interface/src/planning_scene_interface.cpp 51.10% <0.00%> (-1.09%) ⬇️
..._interface/src/detail/constrained_goal_sampler.cpp 76.48% <0.00%> (+1.97%) ⬆️
.../ompl_interface/src/detail/constrained_sampler.cpp 60.98% <0.00%> (+17.08%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 32a1883...a41d0ff. Read the comment docs.

Copy link
Copy Markdown
Contributor

@rhaschke rhaschke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this contribution. I slightly simplified the code. Otherwise looks good to me.
Well done providing unit tests!

@rhaschke rhaschke added the awaits 2nd review one maintainer approved this request label Dec 8, 2021
@wxmerkt
Copy link
Copy Markdown
Contributor Author

wxmerkt commented Dec 9, 2021

Fantastic, thank you very much for the swift review and improvements @rhaschke :-)

Copy link
Copy Markdown
Contributor

@simonschmeisser simonschmeisser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd still highly welcome a PR for adding a sane format and I'm somewhat anxious about rebasing #2854 onto this ...

@wxmerkt
Copy link
Copy Markdown
Contributor Author

wxmerkt commented Dec 9, 2021

I'd still highly welcome a PR for adding a sane format and I'm somewhat anxious about rebasing #2854 onto this ...

Thanks for pointing this out - taking a glance at #2854 it appears to change the serialization format as well - having the unit test should ensure that in the future backwards compatibility can be maintained more easily while still enabling new features to be added - the format has been released and people in the field are relying on it (if it requires changing a unit test to pass, it shouldn't be a patch release)

Edit: Taking a closer look at #2854 - the changes appear to be backwards compatible by only appending to the end of an object and having logic to skip parts if unsupported / gracefully continue with the "current-new" format (not quite TLV, but still forwards compatible) - the issue fixed here was changing the format at the beginning of an object.

@simonschmeisser thank you for your review. Looking forward to getting the fix merged and released to Noetic :-).

@v4hn
Copy link
Copy Markdown
Contributor

v4hn commented Dec 10, 2021

Looks good to me. Really sorry for breaking the format.
@simonschmeisser If we want to change the format again in the future it would definitely make sense to add a version number at the beginning of the file and patch older versions to skip the version tag.
Or we just directly go for a different format that does not asciify floating values anymore. 😬
The current way to detect the file version works but is definitely overhead...

@v4hn v4hn merged commit 9d78be4 into moveit:master Dec 10, 2021
@rhaschke
Copy link
Copy Markdown
Contributor

I strongly support migrating the file format to yaml. This still means asciifying floats, though. Do you have a binary format in mind, @v4hn?

@v4hn
Copy link
Copy Markdown
Contributor

v4hn commented Dec 10, 2021

migrating the file format to yaml

If you got a hammer, ... 😄

Do you have a binary format in mind?

Not really, but do you foresee any problem with defining one ourselves?

  • version tag at start of file
  • define (little endian) byte order for binary numbers
  • add number of elements in front of each list
  • dump pretty much the same data we store now.

I don't see the need to provide backward compatibility for files generated with a more current version of MoveIt, and that's the main downside I see for this style in contrast to yaml.

If we can keep parsers for old versions around somehow, we might even use the binary serialization of the moveit_msgs::PlanningScene message, but that might be a rather stupid idea.

@rhaschke
Copy link
Copy Markdown
Contributor

I think, having a human-readable and -editable format is required. I wouldn't go for a binary format. Actually, I'm surprised that you really had something like this in mind. We don't use binary formats anywhere else...
Actually, I was thinking about yaml-based message serialization.

@v4hn
Copy link
Copy Markdown
Contributor

v4hn commented Dec 10, 2021 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaits 2nd review one maintainer approved this request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scene format breakage between 1.1.5 and 1.1.6 in released distro (Noetic)

4 participants