Skip to content

Reduce spurious unittest failures due to randomness#1381

Merged
rhaschke merged 1 commit intomoveit:masterfrom
ubi-agni:fix-flaky-tests
Mar 7, 2019
Merged

Reduce spurious unittest failures due to randomness#1381
rhaschke merged 1 commit intomoveit:masterfrom
ubi-agni:fix-flaky-tests

Conversation

@rhaschke
Copy link
Copy Markdown
Contributor

@rhaschke rhaschke commented Mar 6, 2019

Kinematics unittests use random sampling, which might (randomly) cause the tests to fail.
This fix reduces randomness by using a fixed seed to sample robot states.

Additionally, the "searchIKWithCallback" test should resample dismissed samples as pointed out
in moveit/moveit_kinematics_tests#16.

Kinematics unittests use random sampling, which might (randomly) cause the tests to fail.
This fix reduces randomness by using a fixed seed to sample robot states.

Additionally, the "searchIKWithCallback" test should resample dismissed samples as pointed out
in moveit/moveit_kinematics_tests#16.
Copy link
Copy Markdown
Member

@henningkayser henningkayser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought finding rare failures is the point of testing with randomness.
I agree, the sample size is more than high enough here to use deterministic numbers.

Just curious, why are some of the tests disabled?

@rhaschke
Copy link
Copy Markdown
Contributor Author

rhaschke commented Mar 7, 2019

I thought finding rare failures is the point of testing with randomness.

No. We are just lazy and don't want to provide numerous samples manually.
A unit test should validate that code that was working at a certain time, continues to work in the future.

@rhaschke rhaschke merged commit d411fe3 into moveit:master Mar 7, 2019
@rhaschke rhaschke deleted the fix-flaky-tests branch March 7, 2019 13:28
mayman99 pushed a commit to mayman99/moveit that referenced this pull request Mar 9, 2019
Kinematics unit tests use random sampling, which might (randomly) cause the tests to fail (unfortunately sampling many failing samples).
This fix reduces randomness by using a fixed seed to sample robot states.

Additionally, the "searchIKWithCallback" test should resample dismissed samples as pointed out
in moveit/moveit_kinematics_tests#16.
sjahr pushed a commit to sjahr/moveit that referenced this pull request Jun 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants