For helping users to reduce duplicates in Elasticsearch, Beats should have some opt-in support to configure the document ID. If the field @metadata.id is set, Beats will already use said ID as document ID. But this feature is still well hidden, because we currently have no support for setting this field.

If the document ID is set, Beats will publish using the op_type create.

• Add support to read IDs from existing JSON logs Add support to set the document ID in the filebeat json reader #5844
• fingerprint processor Fingerprint processor #14205
• processor adding a randomized UUID like ID (for example for metrics data) Add ID processor #14524