-
Notifications
You must be signed in to change notification settings - Fork 25.8k
Use mmapfs by default #16983
Description
ES today has a default directory implementation which switches based on file extension, only using mmapfs for files that Lucene does heavy random access on (terms dict, doc values) and niofs otherwise.
But unfortunately, for compound files (.cfs), which I suspect are often the majority of segments, especially for large shards, it still uses niofs, hurting performance when we do random access on the sub-files within those.
ES used to have issues with segment explosion, which I think contributed to wanting to reduce the mmap count, but we've fixed those, and cfs is now enabled by default, so I think the performance hit of using the inferior niofs on compound file segments is the wrong tradeoff.
We could alternatively keep the current "switching" directory, but for .cfs extension use mmapfs.