Store collector data in own cache, emit only hashes to PHPStan#319
Merged
Store collector data in own cache, emit only hashes to PHPStan#319
Conversation
Member
Author
|
Result cache size measurements:
|
Member
Author
|
shipmonk core: First Full Run
Reuse Cache Run
phpstan-src: First Full Run
Reuse Cache Run
|
Member
Author
|
dcd folder stats:
Size distribution:
|
PHPStan's result cache becomes huge (up to 1GB) because DCD collectors emit many JSON-serialized usage strings. This slows down every PHPStan startup, even for partial analysis where DCD is disabled. Introduce UsageCacheStorage that stores full collector data in %tmpDir%/dcd/ using content-addressable files. Collectors now emit only md5 hashes through PHPStan's mechanism, dramatically shrinking the result cache. Re-introduce class-boundary batching (flush at ClassMethodsNode) to further reduce the number of emits. GC of orphaned cache files runs automatically after each full analysis. Co-Authored-By: Claude Code
Content-addressable storage means identical hashes always produce identical content, so concurrent writes to the same file are harmless. Co-Authored-By: Claude Code
Co-Authored-By: Claude Code
68684e3 to
747cd39
Compare
Split cache files into 256 subdirectories using the first 2 hex chars of the hash as the folder name (e.g. dcd/ab/cdef1234....dat). Co-Authored-By: Claude Code
When shipmonkDeadCode.cache.useOwnCache is true (default), collector data is stored in DCD's own cache and only hashes are emitted to PHPStan. When false, serialized data is emitted directly through PHPStan's collector mechanism (original behavior). Co-Authored-By: Claude Code
Co-Authored-By: Claude Code
When switching from offloadCollectorData: true to false, the leftover cache directory should still be cleaned up on the next full analysis. Co-Authored-By: Claude Code
Co-Authored-By: Claude Code
Co-Authored-By: Claude Code
Better names since no file I/O happens when offloadCollectorData is disabled — pack/unpack reflects the serialization semantics. Co-Authored-By: Claude Code
Co-Authored-By: Claude Code
Contributor
|
nice! |
staabm
reviewed
Mar 30, 2026
Contributor
|
Awesome. Should this be documented? The readme could maybe also instruct how this should be cached on CI? |
Member
Author
I dont think so, it works seamlessly for official documented CI setup |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PHPStan's result cache becomes huge (up to 1GB for huge projects) because DCD collectors emit many JSON-serialized usage strings.
Introduce
UsageCacheStoragethat stores full collector data in%tmpDir%/dcd/using content-addressable files. Collectors now emit only md5 hashes through PHPStan's mechanism, dramatically shrinking the result cache. Re-introduce class-boundary batching (flush atClassMethodsNode) to further reduce the number of emits. GC of orphaned cache files runs automatically after each full analysis.Inspired by my idea in phpstan/phpstan#14074 (comment)