Skip to content

Cache CI environment so it doesn't need to be rebuilt each time #458

@DaniBodor

Description

@DaniBodor

From this blog post.

Reuse

If you must run the CI, the best way to reduce its impact is limit the actual work that needs to be done.

With the cache action, you can cache dependencies and build outputs to make your workflows faster and thus more efficient. Maybe you need to compile some dependency, download and pre-compute some data, or set up your python environment via pip. These typically do not change much from run to run, so try to cache these where possible.

Caching your Python environment

Just the installation of the dependencies of some Python code via pip can be quite significant. Some libraries just seem to pull in an endless stream of dependencies. So, why don't we cache our entire Python environment?

Below is a snippet that we find effective in our workflows for Python code.

As the cache key, we use a combination of the Python directory name (this includes the version) in combination with the hash of the pyproject.toml, setup.cfg, or requirements.txt file. Whenever these get updated, the cache gets invalidated and regenerated.

This means we can also safely skip the pip install step if we hit the cache. Depending on the number of dependencies, this virtually eliminates the setup time of your workflow.

# tests.yaml
jobs:
  test:
    steps:
      ...
      
      - uses: actions/cache@v3
        id: cache-python-env
        with:
          path: ${{ env.pythonLocation }}
          key: ${{ env.pythonLocation }}-${{ hashFiles('pyproject.toml') }}

      - name: Install dependencies
        if: steps.cache-python-env.outputs.cache-hit != 'true'
        run: |
          python -m pip install -e .[develop]

Metadata

Metadata

Assignees

No one assigned

    Labels

    CIcontinuous integrationstaleissue not touched from too much time

    Type

    No type

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions