Problems

There have been multiple reported issues related to compression of caches. The issue we are looking to solve:

• Cross-OS compatibility: Currently on windows cache uses a different compression algorithm (gzip) as compared to on linux | mac (zstd). This leads to different versions for caches created on different platforms. Therefore caches created on windows might not be recoverable on linux | mac. For more details on cache version see this.

Proposal

We are looking to solve both the problems as follows::

• Change the default tar used on windows runners to GNUtar. This is already suggested as a workaround for people in these problems. Same tooling will ensure that cache can be reused across all three OSes.
• Fallback to BSDtar with zstd on windows. BSDtar is already present on Windows runners by default but it does not use zstd due to the issue of compression hanging with large caches. In our testing, we found that performing archiving and compression as separate processes (instead of calling tar --use-compress-program) does not have the hang problem for caches of size up to 2GB.

Reasoning to choose GNUtar over BSDtar as default

BSDtar has some implementation problems. That’s the reason it stopped being used in MacOS for our action. For more details see actions/toolkit#552.

Related issues which should get fixed with this proposal

We have consolidated issues related to the above problems here. Feel free to provide feedback regarding these in this issue itself.

• Cache misses due to path-based versioning and its normalization #330
• Support multiple compression algorithms when restoring cache toolkit#543
• zstd compression hanging on Windows with large files #301
• Actions/cache breaks symlink on Windows runners #1010
• Caching problem - unpacking tar #198 (Should be fixed with use of GNUtar)
• Cache action shouldn't use first tar on the path #125 (Won’t fix)
• Allow choosing cache compression algorithm explicitly #389 (Not fixing as of today)