Initial tag query performance improvements#848
Conversation
Use strings as keys until both non-string ids are used everywhere
|
Very interesting, how does it impact memory usage for the map? |
|
The strings all reference the strings that each Edit: That being said, changing IDs to binary value types instead of strings/pointers is a worthwhile goal. |
|
Interesting. Actually at the time of writing that, as you already mentioned, my idea was that the next step would be to change everything that's still using strings into numeric types and then we won't need to convert between strings and numeric types anymore. I did not expect this temporary solution of converting forth and back is going to be that expensive, so I think your changes make sense that way. |
|
thanks @shanson7 |
The basic idea here is that the string representation of an id is already in use by the defById map. During the course of indexing and querying, we are repeatedly doing fmt calls to translate between the two types. I think the original motivation behind using Id over string was to reduce pointer usage for the GC but this results in many allocations during a tag query. Here is my comparison with the existing benchmark (1m benchtime):
Improvements across the board (especially in allocations) except for indexing, which is pretty close.
Once a numeric Id representation is used all over metrictank, these two maps will still use the same key, but it will be even more efficient!