Skip to content

temp-colcon-cache-branch-5#6

Closed
ruffsl wants to merge 18 commits intomainfrom
temp-colcon-cache-branch-5
Closed

temp-colcon-cache-branch-5#6
ruffsl wants to merge 18 commits intomainfrom
temp-colcon-cache-branch-5

Conversation

@ruffsl
Copy link
Owner

@ruffsl ruffsl commented May 11, 2021

Oh yea, check this out. I've got colcon-cache working to help cache the build and test verbs. This sample PR only modifies the nav2_gazebo_spawner package, that doesn't have any other packages that depend upon it, so when restoring the cache and falling back to the cache provided from the main branch, the CI will only attempt to rebuild and retest packages that have changed (or still haven't yet passed at building and testing from before).

CI job from main branch (29m 0s):

CI job for this PR branch (5m 39s):

Build Workspace | /opt/overlay_ws 5s

Details
#!/bin/bash -eo pipefail
colcon cache lock

BUILD_UNFINISHED=$(
  colcon list \
    --names-only \
    --packages-skip-build-finished \
  | xargs)
BUILD_FAILED=$(
  colcon list \
    --names-only \
    --packages-select-build-failed \
  | xargs)
BUILD_VOID=$(
  colcon list \
    --names-only \
    --packages-select-cache-void \
    --packages-respective-cache-verb build \
  | xargs)

. /opt/underlay_ws/install/setup.sh
rm -rf log
colcon build \
  --packages-above \
    $BUILD_UNFINISHED \
    $BUILD_FAILED \
    $BUILD_VOID \
  --mixin ${OVERLAY_MIXINS}

Starting >>> nav2_common
�]0;colcon cache [0/31 done] [1 ongoing]�Finished <<< nav2_common [0.01s]
�]0;colcon cache [1/31 done] [0 ongoing]�Starting >>> nav_2d_msgs
�]0;colcon cache [1/31 done] [1 ongoing]�Finished <<< nav_2d_msgs [0.01s]
�]0;colcon cache [2/31 done] [0 ongoing]�Starting >>> nav2_gazebo_spawner
�]0;colcon cache [2/31 done] [1 ongoing]�Finished <<< nav2_gazebo_spawner [0.01s]
�]0;colcon cache [3/31 done] [0 ongoing]�Starting >>> nav2_msgs
�]0;colcon cache [3/31 done] [1 ongoing]�Finished <<< nav2_msgs [0.01s]
�]0;colcon cache [4/31 done] [0 ongoing]�Starting >>> nav2_voxel_grid
�]0;colcon cache [4/31 done] [1 ongoing]�Finished <<< nav2_voxel_grid [0.03s]
�]0;colcon cache [5/31 done] [0 ongoing]�Starting >>> dwb_msgs
�]0;colcon cache [5/31 done] [1 ongoing]�Finished <<< dwb_msgs [0.02s]
�]0;colcon cache [6/31 done] [0 ongoing]�Starting >>> nav2_util
�]0;colcon cache [6/31 done] [1 ongoing]�Finished <<< nav2_util [0.01s]
�]0;colcon cache [7/31 done] [0 ongoing]�Starting >>> nav_2d_utils
�]0;colcon cache [7/31 done] [1 ongoing]�Finished <<< nav_2d_utils [0.02s]
�]0;colcon cache [8/31 done] [0 ongoing]�Starting >>> nav2_behavior_tree
�]0;colcon cache [8/31 done] [1 ongoing]�Finished <<< nav2_behavior_tree [0.02s]
�]0;colcon cache [9/31 done] [0 ongoing]�Starting >>> nav2_lifecycle_manager
�]0;colcon cache [9/31 done] [1 ongoing]�Finished <<< nav2_lifecycle_manager [0.02s]
�]0;colcon cache [10/31 done] [0 ongoing]�Starting >>> nav2_map_server
�]0;colcon cache [10/31 done] [1 ongoing]�Finished <<< nav2_map_server [0.02s]
�]0;colcon cache [11/31 done] [0 ongoing]�Starting >>> nav2_amcl
�]0;colcon cache [11/31 done] [1 ongoing]�Finished <<< nav2_amcl [0.01s]
�]0;colcon cache [12/31 done] [0 ongoing]�Starting >>> nav2_costmap_2d
�]0;colcon cache [12/31 done] [1 ongoing]�Finished <<< nav2_costmap_2d [0.02s]
�]0;colcon cache [13/31 done] [0 ongoing]�Starting >>> nav2_rviz_plugins
�]0;colcon cache [13/31 done] [1 ongoing]�Finished <<< nav2_rviz_plugins [0.02s]
�]0;colcon cache [14/31 done] [0 ongoing]�Starting >>> nav2_core
�]0;colcon cache [14/31 done] [1 ongoing]�Finished <<< nav2_core [0.01s]
�]0;colcon cache [15/31 done] [0 ongoing]�Starting >>> costmap_queue
�]0;colcon cache [15/31 done] [1 ongoing]�Finished <<< costmap_queue [0.02s]
�]0;colcon cache [16/31 done] [0 ongoing]�Starting >>> dwb_core
�]0;colcon cache [16/31 done] [1 ongoing]�Finished <<< dwb_core [0.02s]
�]0;colcon cache [17/31 done] [0 ongoing]�Starting >>> nav2_bt_navigator
�]0;colcon cache [17/31 done] [1 ongoing]�Finished <<< nav2_bt_navigator [0.04s]
�]0;colcon cache [18/31 done] [0 ongoing]�Starting >>> nav2_controller
�]0;colcon cache [18/31 done] [1 ongoing]�Finished <<< nav2_controller [0.03s]
�]0;colcon cache [19/31 done] [0 ongoing]�Starting >>> nav2_navfn_planner
�]0;colcon cache [19/31 done] [1 ongoing]�Finished <<< nav2_navfn_planner [0.02s]
�]0;colcon cache [20/31 done] [0 ongoing]�Starting >>> nav2_planner
�]0;colcon cache [20/31 done] [1 ongoing]�Finished <<< nav2_planner [0.02s]
�]0;colcon cache [21/31 done] [0 ongoing]�Starting >>> nav2_recoveries
�]0;colcon cache [21/31 done] [1 ongoing]�Finished <<< nav2_recoveries [0.04s]
�]0;colcon cache [22/31 done] [0 ongoing]�Starting >>> nav2_regulated_pure_pursuit_controller
�]0;colcon cache [22/31 done] [1 ongoing]�Finished <<< nav2_regulated_pure_pursuit_controller [0.02s]
�]0;colcon cache [23/31 done] [0 ongoing]�Starting >>> nav2_smac_planner
�]0;colcon cache [23/31 done] [1 ongoing]�Finished <<< nav2_smac_planner [0.02s]
�]0;colcon cache [24/31 done] [0 ongoing]�Starting >>> nav2_waypoint_follower
�]0;colcon cache [24/31 done] [1 ongoing]�Finished <<< nav2_waypoint_follower [0.04s]
�]0;colcon cache [25/31 done] [0 ongoing]�Starting >>> dwb_critics
�]0;colcon cache [25/31 done] [1 ongoing]�Finished <<< dwb_critics [0.03s]
�]0;colcon cache [26/31 done] [0 ongoing]�Starting >>> dwb_plugins
�]0;colcon cache [26/31 done] [1 ongoing]�Finished <<< dwb_plugins [0.02s]
�]0;colcon cache [27/31 done] [0 ongoing]�Starting >>> nav2_dwb_controller
�]0;colcon cache [27/31 done] [1 ongoing]�Finished <<< nav2_dwb_controller [0.05s]
�]0;colcon cache [28/31 done] [0 ongoing]�Starting >>> navigation2
�]0;colcon cache [28/31 done] [1 ongoing]�Finished <<< navigation2 [0.05s]
�]0;colcon cache [29/31 done] [0 ongoing]�Starting >>> nav2_bringup
�]0;colcon cache [29/31 done] [1 ongoing]�Finished <<< nav2_bringup [0.08s]
�]0;colcon cache [30/31 done] [0 ongoing]�Starting >>> nav2_system_tests
�]0;colcon cache [30/31 done] [1 ongoing]�Finished <<< nav2_system_tests [0.31s]
�]0;colcon cache [31/31 done] [0 ongoing]�
Summary: 31 packages finished [1.50s]
[connext_cmake_module] Warning: The location at which Connext was found when the workspace was built [[/opt/rti.com/rti_connext_dds-5.3.1]] does not point to a valid directory, and the NDDSHOME environment variable has not been set. Support for Connext will not be available.
Starting >>> nav2_gazebo_spawner
�]0;colcon build [0/1 done] [1 ongoing]�Finished <<< nav2_gazebo_spawner [0.78s]
�]0;colcon build [1/1 done] [0 ongoing]�
Summary: 1 package finished [1.08s]
CircleCI received exit code 0

Test Workspace | /opt/overlay_ws 4s

Details
#!/bin/bash -eo pipefail
TEST_FAILURES=$(
  colcon list \
    --names-only \
    --packages-select-test-failures \
  | xargs)
TEST_VOID=$(
  colcon list \
    --names-only \
    --packages-select-cache-void \
    --packages-respective-cache-verb test \
  | xargs)

. install/setup.sh
TEST_PACKAGES=$(
  colcon list \
    --names-only \
    --packages-above \
      $TEST_FAILURES \
      $TEST_VOID \
  | circleci tests split \
    --split-by=timings \
    --timings-type=classname \
    --show-counts \
  | xargs)
set -o xtrace
colcon test \
  --packages-select ${TEST_PACKAGES} \
  --retest-until-pass ${RETEST_UNTIL_PASS} \
  --ctest-args --test-regex "test_.*"
colcon test-result \
  --verbose

[connext_cmake_module] Warning: The location at which Connext was found when the workspace was built [[/opt/rti.com/rti_connext_dds-5.3.1]] does not point to a valid directory, and the NDDSHOME environment variable has not been set. Support for Connext will not be available.
Read 1 lines of classname(s)
Error reading historical timing data: file does not exist
Requested weighting by historical based timing, but they are not present. Falling back to weighting by name.
Bucket 0: assigning 1 classname(s), total weight 1
+ colcon test --packages-select nav2_gazebo_spawner --retest-until-pass 2 --ctest-args --test-regex 'test_.*'
Starting >>> nav2_gazebo_spawner
�]0;colcon test [0/1 done] [1 ongoing]�Finished <<< nav2_gazebo_spawner [1.01s]
�]0;colcon test [1/1 done] [0 ongoing]�
Summary: 1 package finished [1.36s]
+ colcon test-result --verbose
Summary: 2274 tests, 0 errors, 0 failures, 0 skipped
CircleCI received exit code 0

cc @SteveMacenski

@ruffsl ruffsl force-pushed the main branch 2 times, most recently from a11f6b2 to 577b401 Compare May 23, 2021 03:11
@ruffsl ruffsl closed this May 24, 2021
@ruffsl ruffsl deleted the temp-colcon-cache-branch-5 branch May 24, 2021 22:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant