Lucene is adding support for loading terms index offheap using mapped files. Elasticsearch should expose settings to specify list of fields that can be initialized offheap. That will prevent JVM OOM issues for many users by loading index for big fields like "_id" offheap. For example:
<index_name>/_settings --data
'{
"settings":{
"index.lazily_load_fst_fields_offheap": ["_id"]
}
}'
Is the best way to pass it though IndexWriterConfig to the underlying lucene DirectoryReader here? Does this setting needs to be passed from other places in the code?
Lucene is adding support for loading terms index offheap using mapped files. Elasticsearch should expose settings to specify list of fields that can be initialized offheap. That will prevent JVM OOM issues for many users by loading index for big fields like "_id" offheap. For example:
<index_name>/_settings --data
'{
"settings":{
"index.lazily_load_fst_fields_offheap": ["_id"]
}
}'
Is the best way to pass it though IndexWriterConfig to the underlying lucene DirectoryReader here? Does this setting needs to be passed from other places in the code?