This is a meta issue to track some of the changes in ES that we probably want to backport.
Some of these probably will have a long tail and pull in many other changes.

On-heap / off-heap

• Use reader attributes to control term dict memory useage elastic/elasticsearch#42838 Use reader attributes to control term dict memory useage
• Move the terms index of _id off-heap. elastic/elasticsearch#52405 Move the terms index of _id off-heap

MetaData storage / ClusterState

• Reduce number of writes needed for metadata updates elastic/elasticsearch#48701 Reduce the number of writes needed for metadata updates
• Use Lucene exclusively for metadata storage elastic/elasticsearch#50144 Use Lucene exclusively for metadata storage
• More logging for slow cluster state application elastic/elasticsearch#45007 More logging for slow cluster state application
• Warn on slow metadata persistence elastic/elasticsearch#47005 Warn on slow metadata persistence

Engine

• Allow realtime get to read from translog elastic/elasticsearch#48843 Allow realtime get to read from translog
• Fix ref count handling in Engine.failEngine elastic/elasticsearch#48639 Fix ref count handling in Engine.failEngine
• Set Lucene version upon index creation. elastic/elasticsearch#36038 Set Lucene version upon index creation

Translog

• elastic/elasticsearch@41d3eb3
• elastic/elasticsearch@2b8c7c5
• elastic/elasticsearch@6531369
• elastic/elasticsearch@972b81f
• elastic/elasticsearch@b034d1e (requires replication/recovery changes)

Replication

• Advance checkpoints only after persisting ops elastic/elasticsearch#43205 Advance checkpoints only after persisting ops
• Ignore Lucene index in peer recovery if translog corrupted elastic/elasticsearch#49114 Ignore Lucene index in peer recovery if translog corrupted
• Replicate write actions before fsyncing them elastic/elasticsearch#49746 Replicate write actions before fsyncing them

Recovery (soft-deletes)

• Make soft-deletes mandatory elastic/elasticsearch#50775 Make soft-deletes mandatory
• Use index for peer recovery instead of translog elastic/elasticsearch#45136 Use index for peer recovery instead of translog
• Ignore translog retention policy if soft-deletes enabled elastic/elasticsearch#45473 Ignore translog retention policy if soft-deletes enabled
• elastic/elasticsearch@09b46c8 Goodbye and thank you synced flush
• Deprecated support for delaying state recovery pending master nodes elastic/elasticsearch#53646 Deprecated support for delaying state recovery pending master nodes
• Make peer recovery clean files step async elastic/elasticsearch#43787 Make peer recovery clean files step async
• Only retain reasonable history for peer recoveries elastic/elasticsearch#45208 Only retain reasonable history for peer recoveries
• Account soft deletes for committed segments elastic/elasticsearch#43126 Account soft deletes for committed segments
• Do not use soft-deletes to resolve indexing strategy elastic/elasticsearch#43336 Do not use soft-deletes to resolve indexing strategy
• Retry failed peer recovery due to transient errors elastic/elasticsearch#55353 Retry failed peer recovery due to transient errors
• Avoid copying file chunks in peer covery elastic/elasticsearch#56072 Avoid copying file chunks in peer covery

Snapshots

• Add CoolDown Period to S3 Repository elastic/elasticsearch#51074 Add CoolDown Period to S3 Repository
• Cleanup Old index-N Blobs in Repository Cleanup elastic/elasticsearch#49862 Cleanup Old index-N Blobs in Repository Cleanup
• Remove Snapshot INIT Step elastic/elasticsearch#55918 Remove Snapshot INIT Step
• More Efficient Snapshot State Handling elastic/elasticsearch#56669 More Efficient Snapshot State Handling
• Fix Snapshot Abort Not Waiting for Data Nodes elastic/elasticsearch#58214

Other

• Optimize sort on long field elastic/elasticsearch#48804 Optimize sort on long field
• Defer reroute when starting shards elastic/elasticsearch#44433 Defer reroute when starting shards
• Improve handshake failure messages elastic/elasticsearch#44485 Improve handshake failure messages
• Check shard limit after applying index templates elastic/elasticsearch#44619 Check shard limit after applying index templates