In the current modules today we allow users to reference variable that will be replaced when the modules are loaded into Filebeat. Like this:

var:
  - name: paths
    default:
      - /var/log/mysql/error.log*
      - /var/log/mysqld.log*
    os.darwin:
      - /usr/local/var/mysql/{{.builtin.hostname}}.{{.builtin.domain}}.err*
    os.windows:
      - "c:/programdata/MySQL/MySQL Server*/error.log*"

In the above example, we have two placeholder values.

• {{.builtin.hostname}}
• {{.builtin.domain}}

In issue #279 we discussed where this replacement could happen and we have decided that it make sense to allow it to be executed in the Agent with a few hard rules in place.

• Define a limited set of accessible values.
• Share this set of values with the package-registry to do some package validation.
• If the we try to reference invalid or non-existing values we should fail hard and early in the validation chain. (before sending it to the process)

Possible Values

We should keep the list of exposed values short and follow ECS naming here. Here as an early list of values we could expose.

• Version of the Agent.
• Platform (Windows, Darwin, linux)
• Architecture.
• Hostname
• Domain

Syntax

The syntax used in the current module definition is the one from the golang template syntax. This means the YAML is read as a Golang template and parsed again as a ucfg configuration. I think we should not go that route:

• Because the golang templates brings more than just values (loops, conditions, etc)
• This could allow users to changes the YAML keys using templates snippets.

Instead, I see two possible options:

1. Using the current field reference syntax.

We could use a superset of the field reference syntax in beats, this would mean that we have to implement a custom resolver from go-ucfg. This will effectively limit to only replacing values in the YAML values and nothing else.

${.agent.version}
${.agent.host}
or
${var:agent.host}

Usage:

paths:
      - /usr/local/var/mysql/${.agent.hostname}.${.agent.domain}}.err*

2. Use a handlebar-like library

Implements or use a golang's Handlebar library with only support variable replacement and no logical operator. From what I see, the majority of the template engine defines more than just values replacement.

Where this is used?

Currently, the only module I see that uses it is the MySQL module in filebeat.

winterfell~/go/src/github.com/elastic/beats(master|✔) % ag 'builtin' --yaml
filebeat/module/mysql/slowlog/manifest.yml
7:      - /var/lib/mysql/{{.builtin.hostname}}-slow.log
9:      - /usr/local/var/mysql/{{.builtin.hostname}}-slow.log*

filebeat/module/mysql/error/manifest.yml
9:      - /usr/local/var/mysql/{{.builtin.hostname}}.{{.builtin.domain}}.err*

Required decisions

• List of fields
• Corresponding ECS field.
• Syntax to use for referencing the values.