Skip to content

Use mmapfs by default #16983

@mikemccand

Description

@mikemccand

ES today has a default directory implementation which switches based on file extension, only using mmapfs for files that Lucene does heavy random access on (terms dict, doc values) and niofs otherwise.

But unfortunately, for compound files (.cfs), which I suspect are often the majority of segments, especially for large shards, it still uses niofs, hurting performance when we do random access on the sub-files within those.

ES used to have issues with segment explosion, which I think contributed to wanting to reduce the mmap count, but we've fixed those, and cfs is now enabled by default, so I think the performance hit of using the inferior niofs on compound file segments is the wrong tradeoff.

We could alternatively keep the current "switching" directory, but for .cfs extension use mmapfs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions