Skip to content

[Filebeat] Refactor autodiscover default input settings handling#12193

Merged
exekias merged 5 commits intoelastic:masterfrom
exekias:autodiscover-default-config
May 24, 2019
Merged

[Filebeat] Refactor autodiscover default input settings handling#12193
exekias merged 5 commits intoelastic:masterfrom
exekias:autodiscover-default-config

Conversation

@exekias
Copy link
Copy Markdown
Contributor

@exekias exekias commented May 16, 2019 •
edited
Loading

Autodiscover hints default config disable (unreleased) was done at the hints
level, while it only makes sense for certain configs builder (logs).

This change moves this logic to the logs builder and improves how
users can define default settings for logs, ie:

Enable hints but disable log retrieval by default, only containers with the co.elastic.logs/enabled: true annotation will be retrieved:

filebeat.autodiscover:
  providers:
    - type: docker # or kubernetes
      hints.enabled: true
      hints.default_config.enabled: false

Logs are read by default for all containers, using the given input
settings. Containers with co.elastic.logs/enabled: false annotation
will be ignored:

filebeat.autodiscover:
  providers:
    - type: kubernetes # or docker
      hints.enabled: true
      hints.default_config:
        type: container
        paths:
          - /var/log/containers/*${data.container.id}.log

@exekias exekias added in progress Pull request is currently in progress. libbeat needs tests needs_docs containers Related to containers use case Team:Integrations Label for the Integrations team v7.2.0 labels May 16, 2019
@exekias exekias requested a review from a team as a code owner May 16, 2019 19:04
@exekias
Copy link
Copy Markdown
Contributor Author

exekias commented May 16, 2019 •
edited
Loading

@alakahakai this changes a bit the default setting you added, would love your feedback here

@exekias
Copy link
Copy Markdown
Contributor Author

exekias commented May 16, 2019

@vjsamuel FYI ^

@vjsamuel
Copy link
Copy Markdown
Contributor

vjsamuel commented May 16, 2019

thanks I need to remember this :)

@alakahakai
Copy link
Copy Markdown

alakahakai commented May 18, 2019

@exekias The changes related to disabling default config LGTM

@exekias exekias force-pushed the autodiscover-default-config branch from a3659eb to a77e419 Compare May 20, 2019 14:47
@exekias exekias changed the title Move default config logic to the builder [Filebeat] Refactor autodiscover default input settings handling May 20, 2019
Move default config logic to the builder
489b6f1 
Autodiscover hints default config (unreleased) was done at the hints
level, while it only makes sense for certain configs builder (logs).

This change moves this logic to the `logs` builder and improves how
users can define default settings for logs, ie:

Enable hints but disable log retrieval by default:

```
filebeat.autodiscover:
  providers:
    - type: docker # or kubernetes
      hints.enabled: true
      hints.default_config.enabled: false
```

Only containers with the `co.elastic.logs/enabled: true` annotation
will be retrieved.

```
filebeat.autodiscover:
  providers:
    - type: kubernetes # or docker
      hints.enabled: true
      hints.default_config:
        type: container
        paths:
          - /var/log/containers/*${data.container.id}.log
```

Logs are read by default for all containers, using the given input
settings. Containers with `co.elastic.logs/enabled: false` annotation
will be ignored.
@exekias exekias force-pushed the autodiscover-default-config branch from a77e419 to 489b6f1 Compare May 20, 2019 15:01
@exekias exekias added the needs_backport PR is waiting to be backported to other branches. label May 22, 2019
jsoriano
jsoriano reviewed May 23, 2019
View reviewed changes
Copy link
Copy Markdown
Member

@jsoriano jsoriano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, please update the changelog to replace the change previously added

jsoriano
jsoriano approved these changes May 24, 2019
View reviewed changes
Copy link
Copy Markdown
Member

@jsoriano jsoriano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for these changes!

@exekias exekias merged commit e5fd309 into elastic:master May 24, 2019
@exekias exekias mentioned this pull request May 24, 2019
Merged
exekias pushed a commit to exekias/beats that referenced this pull request May 24, 2019
[Filebeat] Refactor autodiscover default input settings handling (ela…
1b04806 
…stic#12193)

Autodiscover hints default config (unreleased) was done at the hints
level, while it only makes sense for certain configs builder (logs).

This change moves this logic to the `logs` builder and improves how
users can define default settings for logs, ie:

Enable hints but disable log retrieval by default:

```
filebeat.autodiscover:
  providers:
    - type: docker # or kubernetes
      hints.enabled: true
      hints.default_config.enabled: false
```

Only containers with the `co.elastic.logs/enabled: true` annotation
will be retrieved.

```
filebeat.autodiscover:
  providers:
    - type: kubernetes # or docker
      hints.enabled: true
      hints.default_config:
        type: container
        paths:
          - /var/log/containers/*${data.container.id}.log
```

Logs are read by default for all containers, using the given input
settings. Containers with `co.elastic.logs/enabled: false` annotation
will be ignored.

(cherry picked from commit e5fd309)
@exekias exekias removed the needs_backport PR is waiting to be backported to other branches. label May 24, 2019
exekias pushed a commit that referenced this pull request May 24, 2019
[Filebeat] Refactor autodiscover default input settings handling (#12193
4eefd49 
) (#12272)

Autodiscover hints default config (unreleased) was done at the hints
level, while it only makes sense for certain configs builder (logs).

This change moves this logic to the `logs` builder and improves how
users can define default settings for logs, ie:

Enable hints but disable log retrieval by default:

```
filebeat.autodiscover:
  providers:
    - type: docker # or kubernetes
      hints.enabled: true
      hints.default_config.enabled: false
```

Only containers with the `co.elastic.logs/enabled: true` annotation
will be retrieved.

```
filebeat.autodiscover:
  providers:
    - type: kubernetes # or docker
      hints.enabled: true
      hints.default_config:
        type: container
        paths:
          - /var/log/containers/*${data.container.id}.log
```

Logs are read by default for all containers, using the given input
settings. Containers with `co.elastic.logs/enabled: false` annotation
will be ignored.

(cherry picked from commit e5fd309)
@exekias exekias mentioned this pull request May 28, 2019
Closed
@exekias exekias mentioned this pull request Jun 4, 2019
Merged
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
[Filebeat] Refactor autodiscover default input settings handling (ela…
3691a6c 
…stic#12193) (elastic#12272)

Autodiscover hints default config (unreleased) was done at the hints
level, while it only makes sense for certain configs builder (logs).

This change moves this logic to the `logs` builder and improves how
users can define default settings for logs, ie:

Enable hints but disable log retrieval by default:

```
filebeat.autodiscover:
  providers:
    - type: docker # or kubernetes
      hints.enabled: true
      hints.default_config.enabled: false
```

Only containers with the `co.elastic.logs/enabled: true` annotation
will be retrieved.

```
filebeat.autodiscover:
  providers:
    - type: kubernetes # or docker
      hints.enabled: true
      hints.default_config:
        type: container
        paths:
          - /var/log/containers/*${data.container.id}.log
```

Logs are read by default for all containers, using the given input
settings. Containers with `co.elastic.logs/enabled: false` annotation
will be ignored.

(cherry picked from commit 0d24fa2)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jsoriano jsoriano jsoriano approved these changes

Assignees

@exekias exekias

Labels

containers Related to containers use case in progress Pull request is currently in progress. libbeat review Team:Integrations Label for the Integrations team v7.2.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@exekias @vjsamuel @alakahakai @jsoriano @andresrc