As discussed in colcon/colcon-core#213 , I think it would be useful to add a selection argument to filter packages by those were it's source code has changed with respect to what was used in the previous workspace build. This could be achieved by augmenting the verbs to audit the source folder of a package when specified, and later comparing the source tree with the previous audit file when determining selecting conditions. It could also be version control aware to ignore specific files/folders.

An example workflow could be as follows:

# setup workspace
mkdir -p ~/ws/src && cd ~/ws
vcs import src < ros2.repos

# build workspace with audit
colcon build --audit

# make source changes 
touch src/foo/foo.txt

# gather list of changed packages
PKGS_CHANGED=$(colcon list --packages-skip-source-unchanged | xarg)

# rebuild only changed packages and above
colcon clean --build --install --packages-above $PKGS_CHANGED
colcon build --audit --packages-above $PKGS_CHANGED --cmake-clean-cache
colcon test  --packages-above $PKGS_CHANGED

# rinse and repeat
touch src/bar/bar.txt
PKGS_CHANGED=$(colcon list --packages-skip-source-unchanged | xarg)
colcon clean --build --install --packages-above $PKGS_CHANGED
colcon build --audit --packages-above $PKGS_CHANGED --cmake-clean-cache
colcon test  --packages-above $PKGS_CHANGED
...

So far, I'm thinking of adding two arguments, one that can be used with existing verbs to audit the source and store the audit file to the package build folder for later use. The second would be used to filter packages by investigating deltas between the audit file and current package's source tree.

Before starting a PR, I'd like to iterate on a few things:

Terminology

I've used the term audit as similar defined in https://github.com/jessek/hashdeep CLI, but perhaps this argument should be made more specific? e.g. --audit-packages?

For the filter argument, given the AND logic of all conditions, I see there is a pattern for having an inclusive and exclusive conditions for setting selected = False assignment. E.g. packages_select_test_failures vs. packages_skip_test_passed. In a similar style, perhaps we could use the names:

• packages_select_audit_changed
• packages_skip_audit_changed

or

• packages_select_audit_changed
• packages_skip_audit_unchanged

I guess it was intentional by design not to make select/skip_test purly symmetric, as packages_skip_test_passed includes both failed and untested packages, while packages_select_test_failures only includes failed packages. What would be the appropriate dualism here?

Extension

Taking a look at the colcon-package-selection extensions, I think adding the feature here might be most appropriate for code reuse. However, if we wanted to add dependencies like gitpython to be version control aware for ignoring files, should we then seek to break this out into its own extension?

Persisting Audits

When the build verb is specified to audit, should the audit file only be over-written to the build folder upon a successful build? I was thinking if the audit file was always updated, regardless of the success of the build, then what would be the cleanest order of commands to rebuild packages matching (audit-changed OR build-not-passed)?

What does it mean to the filter aurments when the audit file does not exist? Should unchanged only be interpreted by the existence of a colcon_audit that matches the audit at runtime, similar to how a package build is only considered as passed if the colcon_build.rc exists and only contains 0? This might be a consideration for the asymmetric duality in select/skip arguments.