Remove Lucene split packages#82132
Conversation
This PR proposes to fix the Lucene split package issue in LazySoftDeletesDirectoryReaderWrapper.
| } | ||
|
|
||
| private static CacheKey getCacheKey(DirectoryReader reader, String field) throws IOException { | ||
| SoftDeletesDirectoryReaderWrapper wrapper = new SoftDeletesDirectoryReaderWrapper(reader, field); |
There was a problem hiding this comment.
Hm, I realized there's an issue if reader.getReaderCacheHelper() is null. I can easily work around the null pointer here, but I need to somehow get a key inside the leaf and codec readers.
| } | ||
|
|
||
| static CacheKey createCacheKey(DirectoryReader in) { | ||
| try (DirectoryReader clone = DirectoryReader.open(in.directory())) { |
There was a problem hiding this comment.
This feels like too much of a hack to bring into ES. It's effectively opening up the index multiple times now whenever a search hits a shard in the frozen tier (one extra time per lucene index, and then once per segment in the Lucene index) . Let's rather work towards exposing the right abstraction in Lucene.
There was a problem hiding this comment.
Yeah I agree, it's a total hack. I couldn't figure out a better way to get to a unique CacheKey. Let's try to get a change in Lucene.
|
Pinging @elastic/es-core-infra (Team:Core/Infra) |
|
Hi @grcevski, I've created a changelog YAML for you. |
|
@elasticmachine run Check labels |
|
Hi @grcevski, I've updated the changelog YAML for you. |
This PR proposes to fix the Lucene split package issue in LazySoftDeletesDirectoryReaderWrapper.
I did two things to make this work, one is a hack until we perhaps make a change in Lucene:
If we manage to get the DelegatingCacheHelper implementation as a public implementation in Lucene, then we get much better implementation.
closes #81981