You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Elastic support matrix indicates that the latest Filebeat 7.x version works with all 7.x versions of Elasticsearch. This is an assumption I'm making based on the table pictured below. There is a "Compatibility with Beats" table but it does not include Elasticsearch or Kibana columns.
This is not true for all Filebeat modules because there are specific Elasticsearch features utilized in module ingest node pipelines that require newer ES versions. For example:
The set processors copy_from feature. Used to copy objects.
The convert processor's type: ip. Used to ensure strings are valid IPs for ip mapping fields.
registered_domain is new in 7.13.
network_direction is new in 7.12.
There are several other features where Filebeat automatically rewrites the ingest node pipeline to create a compatible pipeline for the current ES version (possibly with a reduced feature set).
The append processors allow_duplicates.
community_id is new in 7.12
The set processor's ignore_empty.
uri_parts is new in 7.12
user_agent requires ecs: true in ES [6.7, 7.0).
How do we want to handle modules that require specific Elasticsearch versions and how do we indicate this to users in the support matrix?
It might just be to say a particular module requires ES 7.N, documented it, add a runtime check, and then put a footnote in the support matrix that says certain Filebeat modules have additional version constraints.
Or indicate that Filebeat requires a specific ES version like Filebeat 7.13 requires at least ES 7.12 (FB works with previous ES minor). This would mean that modules can be developed using more recent ES features and it's easy to test with one previous ES version.
Manually fix every module pipeline that uses new features like copy_from or convert with type: ip. With this we never get to take advantage of new features in ES.
The Elastic support matrix indicates that the latest Filebeat 7.x version works with all 7.x versions of Elasticsearch. This is an assumption I'm making based on the table pictured below. There is a "Compatibility with Beats" table but it does not include Elasticsearch or Kibana columns.
This is not true for all Filebeat modules because there are specific Elasticsearch features utilized in module ingest node pipelines that require newer ES versions. For example:
setprocessorscopy_fromfeature. Used to copy objects.convertprocessor'stype: ip. Used to ensure strings are valid IPs foripmapping fields.registered_domainis new in 7.13.network_directionis new in 7.12.There are several other features where Filebeat automatically rewrites the ingest node pipeline to create a compatible pipeline for the current ES version (possibly with a reduced feature set).
appendprocessorsallow_duplicates.community_idis new in 7.12setprocessor'signore_empty.uri_partsis new in 7.12user_agentrequiresecs: truein ES [6.7, 7.0).How do we want to handle modules that require specific Elasticsearch versions and how do we indicate this to users in the support matrix?
copy_fromorconvertwithtype: ip. With this we never get to take advantage of new features in ES.