Make symbolizer read-time #4564
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
✅ Meticulous spotted zero visual differences across 451 screens tested: view results. Expected differences? Click here. Last updated for commit 3301b57. This comment will update as new commits are pushed. |
3546264 to
878f5bc
Compare
878f5bc to
c5f8350
Compare
metalmatze
reviewed
Apr 29, 2024
| // in memory due to high cardinality of the dictionaries. | ||
| Encoding: schemapb.StorageLayout_ENCODING_PLAIN_UNSPECIFIED, | ||
| Type: schemapb.StorageLayout_TYPE_STRING, | ||
| Encoding: schemapb.StorageLayout_ENCODING_RLE_DICTIONARY, |
Member
There was a problem hiding this comment.
This encoding change means that previous parquet files aren't properly read anymore, right? So, basically, do people need to start over for this new storage layout?
c5f8350 to
5f570cc
Compare
b022e16 to
946ec47
Compare
Storing stacktraces in frostdb has the advantage that the metastore does not have to be persistent, and instead can be re-built at any point in time. This patch is the start of moving to a model, where the metastore is ephemeral and is only a read-through cache for symbolization, that happens at query time, as opposed to all the time asynchronously.
946ec47 to
3301b57
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This improves the symbolization story in various ways:
This overall results in significantly less resources used for symbolization while delivering the same experience as before.