The script files in a ROS workspace are used to setup the environment in a shell to use the artifacts of the workspace. This for example includes environment variables like PATH but also shell specific functionalities.

The goals of the environment setup are:

• Ability to extend the current environment, does not overwrite/undo any existing environment modifications.
• Avoid duplicate items in environment variables, to enable sourcing a workspace again after e.g. having added packages to it.
• Provide shell specific features like completion.
• Each package must contain the scripts (or at least the information) to setup the environment it needs since packages must be composable on the file system level (e.g. when packaging them into Debian packages).

Current status:

• ament_cmake and colcon generate a set of scripts which source each other in a hierarchy.
• The prefix level scripts determine the topological order of the packages in the workspace (using a Python module located in the root of the workspace).
• This script then source the package level script of each package in topological order.
• When sourcing a non-primary shell (like bash) first the primary shell scripts (in this case sh) are being sourced and afterwards the bash specific scripts are sourced to add bash specific functionality.

Runtime:

• For a workspace with e.g. ~100 packages sourcing the prefix level script results in roughly ~300-500 script files to be sourced. This is due to the fact that each package contributes a few environment hooks which are stored in separate files.
• Each environment hook which wants to contribute a value to an environment variable like PATH must first check if the to-be-added value is already in the environment variable (to satisfy the previously stated goal to avoid duplicates).

Performance:

• For larger workspaces (> 100 packages) the time to source the prefix level script can be significant.
  • On a system with a fast IO this might be "only" 1-2 seconds, on e.g. a lower end embedded system the time might even be in the order of dozens of seconds.
• Surprisingly the large number of sourced files isn't the significant part.
• The check performed for each value to be added to an environment variable is the most significant part. Especially when doing an isolated installation the length of each environment variable increases linear with the number of packages in the workspace and with it the effort to determine if a value is already part of an environment variable also gets more expensive with each package added.

Proposal:

• Since the runtime cost stems from the fact that the check-for-duplicates is currently being performed in the shell I propose to move that logic into the existing Python module.
• Early testing indicates that the performance improvements is higher than an order of magnitude for large workspaces and also scales better when adding more packages to a workspace.

This proposal will address #573.

Implementation details:

• In order to move the logic into Python the intended behavior change of an environment hook must be "machine readable". See generate .dsv files beside known environment hooks ament/ament_cmake#187 for details on a proposal for how to store that information (in a semicolon separated file beside the existing environment hook).
• Until now the prefix level Python module only determined the topological order and its logic was independent from the actual shell being used. With this proposal it not only has to perform the logic to determine the intended environment changes but also communicate the results to the used shell which then needs to apply them. Therefore the Python module needs to generate shell specific code which can then be executed by the shell script to perform the environment changes efficiently. See perform environment calculation in Python colcon/colcon-core#209 which shows how to customize the Python module for each shell.