[Filebeat] Document netflow internal_networks and set default by andrewkroh · Pull Request #24110 · elastic/beats

andrewkroh · 2021-02-18T14:58:09Z

What does this PR do?

Documentation for the internal_networks option of the Netflow input and module was missing.
Also the module's manifest did not declare the option so if it was not set in the module config
it would cause an error.

I did not see where a default was set for the netflow input's internal_networks option so I set that
to private to keep the old behavior before this was configurable.

Fixes #24094

Why is it important?

Existing users of the netflow module are broken after upgrade unless they change their config.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

Fixes [filebeat][netflow] filebeat does't start when internal_networks variable isn't defined in config file #24094

elasticmachine · 2021-02-18T14:58:11Z

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

Documentation for the `internal_networks` option of the Netflow input and module was missing. Also the module's manifest did not declare the option so if it was not set in the module config it would cause an error. I did not see where a default was set for the netflow input's internal_networks option so I set that to `private` to keep the old behavior before this was configurable. Fixes elastic#24094

andrewstucki · 2021-02-18T15:13:08Z

@andrewkroh wow, not sure how we didn't catch this previously, but I checked over the previous changes again and we also look like we have an issue with the configuration. In filebeat/module/netflow/log/config/netflow.yml:

{{if .internal_networks}}
internal_hosts:
{{range .internal_networks}}
- '{{ . }}'
{{end}}
{{end}}

notice the internal_hosts v. internal_networks. When the input has:

 InternalNetworks []string `config:"internal_networks"`

Any chance you could add a fix for that here too? Otherwise it's not configurable at all 😞

andrewstucki

LGTM

elasticmachine · 2021-02-18T15:25:08Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Started by user Andrew Stucki
Start Time: 2021-02-19T22:19:43.973+0000
Duration: 48 min 34 sec
Commit: d10a383

Test stats 🧪

Test	Results
Failed	0
Passed	13056
Skipped	2215
Total	15271

Trends 🧪

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	13056
Skipped	2215
Total	15271

* upstream/master: [Elastic Agent] Fix docker entrypoint for elastic-agent. (elastic#24155) [PACKAGING] Push docker images with the architecture in the version (elastic#24121) [Agent] Add agent standalone manifests for system module & Pod's log collection (elastic#23938) indicator type url is in upper case (elastic#24152) [Filebeat] Document netflow internal_networks and set default (elastic#24110) [Filebeat] Adding fixes to the TI module (elastic#24133) [Enhancement] Add RotateOnStartup feature flag for file output (elastic#19347) [Ingest Manager] Fix: Successfully installed and enrolled agent running standalone (elastic#24128) Set Elastic licence type for APM server Beats update job (elastic#24122) Add logrotation section on Running Filebeat on k8s (elastic#24120) [CI] Run if manual UI (elastic#24116) [CI] enable x-pack/heartbeat in the CI (elastic#23873)

…dows-7 * upstream/master: Remove OSS reference for kibana and elasticsearch (elastic#24164) Skip flaky TestActions on MacOSx (elastic#23966) [Filebeat][AWS] Fix vpcflow pipeline exception: Cannot invoke "Object.getClass()" because "receiver" is null (elastic#24167) [Elastic Agent] Fix docker entrypoint for elastic-agent. (elastic#24155) [PACKAGING] Push docker images with the architecture in the version (elastic#24121) [Agent] Add agent standalone manifests for system module & Pod's log collection (elastic#23938) indicator type url is in upper case (elastic#24152) [Filebeat] Document netflow internal_networks and set default (elastic#24110) [Filebeat] Adding fixes to the TI module (elastic#24133) [Enhancement] Add RotateOnStartup feature flag for file output (elastic#19347) [Ingest Manager] Fix: Successfully installed and enrolled agent running standalone (elastic#24128) Set Elastic licence type for APM server Beats update job (elastic#24122) Add logrotation section on Running Filebeat on k8s (elastic#24120) [CI] Run if manual UI (elastic#24116) [CI] enable x-pack/heartbeat in the CI (elastic#23873) chore: comment out the E2E (elastic#24109) chore: add-backport-next (elastic#24098) Adjust the position of the architecture name in Dockerlogbeat tarball (elastic#24095) Update dependencies for M1 support in System (elastic#24019)

…-arm * upstream/master: (24 commits) Add example input autodsicover config (elastic#24157) Empty configuration options generate `<no value>` string for azure-eventhub input (elastic#24156) Remove OSS reference for kibana and elasticsearch (elastic#24164) Skip flaky TestActions on MacOSx (elastic#23966) [Filebeat][AWS] Fix vpcflow pipeline exception: Cannot invoke "Object.getClass()" because "receiver" is null (elastic#24167) [Elastic Agent] Fix docker entrypoint for elastic-agent. (elastic#24155) [PACKAGING] Push docker images with the architecture in the version (elastic#24121) [Agent] Add agent standalone manifests for system module & Pod's log collection (elastic#23938) indicator type url is in upper case (elastic#24152) [Filebeat] Document netflow internal_networks and set default (elastic#24110) [Filebeat] Adding fixes to the TI module (elastic#24133) [Enhancement] Add RotateOnStartup feature flag for file output (elastic#19347) [Ingest Manager] Fix: Successfully installed and enrolled agent running standalone (elastic#24128) Set Elastic licence type for APM server Beats update job (elastic#24122) Add logrotation section on Running Filebeat on k8s (elastic#24120) [CI] Run if manual UI (elastic#24116) [CI] enable x-pack/heartbeat in the CI (elastic#23873) chore: comment out the E2E (elastic#24109) chore: add-backport-next (elastic#24098) Adjust the position of the architecture name in Dockerlogbeat tarball (elastic#24095) ...

…c#24110) Documentation for the `internal_networks` option of the Netflow input and module was missing. Also the module's manifest did not declare the option so if it was not set in the module config it would cause an error. I did not see where a default was set for the netflow input's internal_networks option so I set that to `private` to keep the old behavior before this was configurable. Fixes elastic#24094 (cherry picked from commit 3ca53aa)

#24278) Documentation for the `internal_networks` option of the Netflow input and module was missing. Also the module's manifest did not declare the option so if it was not set in the module config it would cause an error. I did not see where a default was set for the netflow input's internal_networks option so I set that to `private` to keep the old behavior before this was configurable. Fixes #24094 (cherry picked from commit 3ca53aa)

…c#24110) Documentation for the `internal_networks` option of the Netflow input and module was missing. Also the module's manifest did not declare the option so if it was not set in the module config it would cause an error. I did not see where a default was set for the netflow input's internal_networks option so I set that to `private` to keep the old behavior before this was configurable. Fixes elastic#24094 (cherry picked from commit 3ca53aa)

#24279) Documentation for the `internal_networks` option of the Netflow input and module was missing. Also the module's manifest did not declare the option so if it was not set in the module config it would cause an error. I did not see where a default was set for the netflow input's internal_networks option so I set that to `private` to keep the old behavior before this was configurable. Fixes #24094 (cherry picked from commit 3ca53aa)

#24280) Documentation for the `internal_networks` option of the Netflow input and module was missing. Also the module's manifest did not declare the option so if it was not set in the module config it would cause an error. I did not see where a default was set for the netflow input's internal_networks option so I set that to `private` to keep the old behavior before this was configurable. Fixes #24094 (cherry picked from commit 3ca53aa)

…c#24110) (elastic#24280) Documentation for the `internal_networks` option of the Netflow input and module was missing. Also the module's manifest did not declare the option so if it was not set in the module config it would cause an error. I did not see where a default was set for the netflow input's internal_networks option so I set that to `private` to keep the old behavior before this was configurable. Fixes elastic#24094 (cherry picked from commit 563e964)

andrewkroh added bug Filebeat Filebeat Team:Security-External Integrations labels Feb 18, 2021

botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Feb 18, 2021

andrewkroh requested a review from andrewstucki February 18, 2021 14:58

andrewkroh force-pushed the bugfix/fb/internal-networks branch from 780da09 to 15fa438 Compare February 18, 2021 14:59

andrewkroh force-pushed the bugfix/fb/internal-networks branch from 15fa438 to ebf144f Compare February 18, 2021 15:00

andrewkroh added the needs_backport PR is waiting to be backported to other branches. label Feb 18, 2021

andrewkroh mentioned this pull request Feb 18, 2021

[filebeat][netflow] filebeat does't start when internal_networks variable isn't defined in config file #24094

Closed

Fix module config file

64f4b58

andrewstucki approved these changes Feb 18, 2021

View reviewed changes

Generate docs

d10a383

andrewkroh merged commit 3ca53aa into elastic:master Feb 20, 2021

andrewkroh mentioned this pull request Mar 1, 2021

Cherry-pick #24110 to 7.x: [Filebeat] Document netflow internal_networks and set default #24278

Merged

6 tasks

andrewkroh added v7.13.0 and removed needs_backport PR is waiting to be backported to other branches. labels Mar 1, 2021

andrewkroh mentioned this pull request Mar 1, 2021

Cherry-pick #24110 to 7.11: [Filebeat] Document netflow internal_networks and set default #24279

Merged

6 tasks

andrewkroh added the v7.11.2 label Mar 1, 2021

This was referenced Mar 1, 2021

Locality values mismatch between Netflow input and ingest pipeline #24272

Closed

Cherry-pick #24110 to 7.12: [Filebeat] Document netflow internal_networks and set default #24280

Merged

andrewkroh added the v7.12.0 label Mar 1, 2021

andrewkroh removed the v7.11.2 label Mar 17, 2021

andrewkroh added v7.12.1 v7.12.0 and removed v7.12.0 v7.12.1 labels Apr 8, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Filebeat] Document netflow internal_networks and set default#24110

[Filebeat] Document netflow internal_networks and set default#24110
andrewkroh merged 3 commits intoelastic:masterfrom
andrewkroh:bugfix/fb/internal-networks

andrewkroh commented Feb 18, 2021

Uh oh!

elasticmachine commented Feb 18, 2021

Uh oh!

andrewstucki commented Feb 18, 2021 •

edited

Loading

Uh oh!

andrewstucki left a comment

Uh oh!

elasticmachine commented Feb 18, 2021 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

Trends 🧪

Test stats 🧪

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

andrewkroh commented Feb 18, 2021

What does this PR do?

Why is it important?

Checklist

Related issues

Uh oh!

elasticmachine commented Feb 18, 2021

Uh oh!

andrewstucki commented Feb 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andrewstucki left a comment

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Feb 18, 2021 • edited by jenkins-beats-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

Trends 🧪

💚 Flaky test report

Test stats 🧪

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

andrewstucki commented Feb 18, 2021 •

edited

Loading

elasticmachine commented Feb 18, 2021 •

edited by jenkins-beats-ci bot

Loading