Skip to content

[Servo] CI simplification#1556

Merged
tylerjw merged 12 commits intomoveit:mainfrom
AndyZe:andyz/disable_flaky_test
Sep 29, 2022
Merged

[Servo] CI simplification#1556
tylerjw merged 12 commits intomoveit:mainfrom
AndyZe:andyz/disable_flaky_test

Conversation

@AndyZe
Copy link
Copy Markdown
Member

@AndyZe AndyZe commented Sep 7, 2022

  • Delete the flaky, timing-based singularity integration test
  • Add a new, simple, reliable unit test for singularities
  • Delete the "basic integration tests" that were redundant with some other tests
  • Use rclcpp::WallRate everywhere, since rclcpp::Rate can move backward in time

Fixes #1499

@AndyZe AndyZe marked this pull request as draft September 7, 2022 02:40
@AndyZe AndyZe force-pushed the andyz/disable_flaky_test branch from 42f5494 to cd920b2 Compare September 7, 2022 03:07
@AndyZe AndyZe changed the title [Servo] Increase test timeout [Servo] More test adjustments Sep 7, 2022
@codecov
Copy link
Copy Markdown

codecov bot commented Sep 7, 2022

Codecov Report

Base: 51.10% // Head: 51.10% // Increases project coverage by +0.01% 🎉

Coverage data is based on head (322739a) compared to base (7fec51b).
Patch coverage: 88.47% of modified lines in pull request are covered.

❗ Current head 322739a differs from pull request most recent head 0357848. Consider uploading reports for the commit 0357848 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1556      +/-   ##
==========================================
+ Coverage   51.10%   51.10%   +0.01%     
==========================================
  Files         380      381       +1     
  Lines       31796    31790       -6     
==========================================
- Hits        16246    16243       -3     
+ Misses      15550    15547       -3     
Impacted Files Coverage Δ
.../moveit_servo/include/moveit_servo/pose_tracking.h 100.00% <ø> (ø)
moveit_ros/moveit_servo/src/utilities.cpp 87.24% <87.24%> (ø)
moveit_ros/moveit_servo/src/servo_calcs.cpp 66.45% <100.00%> (-1.79%) ⬇️
...dl_kinematics_plugin/src/kdl_kinematics_plugin.cpp 76.43% <0.00%> (+1.08%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@AndyZe AndyZe force-pushed the andyz/disable_flaky_test branch 2 times, most recently from 88ed9cb to 75d84ea Compare September 7, 2022 17:37
@AndyZe AndyZe marked this pull request as ready for review September 7, 2022 17:37
@AndyZe AndyZe force-pushed the andyz/disable_flaky_test branch from 75d84ea to db86299 Compare September 7, 2022 17:37
@AndyZe AndyZe force-pushed the andyz/disable_flaky_test branch from 2675031 to c2fe14f Compare September 8, 2022 13:22
@AndyZe AndyZe changed the title [Servo] More test adjustments [Servo] CI simplification Sep 8, 2022
@AndyZe AndyZe force-pushed the andyz/disable_flaky_test branch from 9a46508 to 7fe804f Compare September 16, 2022 16:56
Copy link
Copy Markdown
Member

@henningkayser henningkayser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with these changes overall, but deleting flaky tests is really not a good solution, it only helps with hiding and forgetting bugs. Please only comment the tests so that CI succeeds.

@moveit moveit deleted a comment from mergify bot Sep 20, 2022
@AndyZe
Copy link
Copy Markdown
Member Author

AndyZe commented Sep 20, 2022

I understand your high-level concern but this real-time control loop is inherently tied to timing, so it might not be suitable for CI. That's why I made a unit test to replace the integration test.

For example, I would say that this integration test code is a little questionable:

  while (!sawTrackedStatus() && iterations++ < test_parameters_->timeout_iterations)
  {
    auto msg = std::make_unique<geometry_msgs::msg::TwistStamped>();
    msg->header.stamp = node_->now();
    msg->twist.linear.x = 0.8;
    pub_twist_cmd_->publish(std::move(msg));
    publish_loop_rate.sleep();
  }
  // Test that we didn't timeout
  EXPECT_LT(iterations, test_parameters_->timeout_iterations);

Nevertheless, I think the steady_clock change might make this test more reliable so we can try restoring the integration test now and see how it goes.

@AndyZe AndyZe force-pushed the andyz/disable_flaky_test branch from 7fe804f to 999c8b2 Compare September 20, 2022 17:10
@tylerjw tylerjw force-pushed the andyz/disable_flaky_test branch from 322739a to 3322f19 Compare September 28, 2022 18:42
Copy link
Copy Markdown
Member

@tylerjw tylerjw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merging because I hope this resolves the servo flaky test.

@tylerjw tylerjw merged commit f24bbc0 into moveit:main Sep 29, 2022
AndyZe pushed a commit to AndyZe/moveit2 that referenced this pull request Feb 28, 2023
AndyZe added a commit to AndyZe/moveit2 that referenced this pull request Mar 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI: Servo singularity test is flaky

3 participants