[Auditbeat] Change module configuration from "metricsets" to "datasets" by cwurm · Pull Request #10018 · elastic/beats

cwurm · 2019-01-11T14:11:41Z

This changes the Auditbeat system module configuration to use "datasets" rather than "metricsets" to configure its sub-modules:

- module: system
  datasets:
    - host
    - process
    [...]

It simply overwrites the MetricSets field in the default Metricbeat configuration with the value of the datasets configuration.

@andrewkroh We talked about the difficulty of making changes to go-ucfg or to the Metricbeat code. Turns out there is no need to change anything under metricbeat/, so I think this might be ok? What do you think? I think in the long term we might still want to change to a fileset-like configuration, but for now this would avoid having a mix of "dataset" and "metricset" in anything user-facing, i.e. documentation or configuration.

Follow-up:

Change documentation and other places where "metricset(s)" is used

elasticmachine · 2019-01-11T14:11:43Z

Pinging @elastic/secops

x-pack/auditbeat/module/system/system.go

ruflin · 2019-01-11T15:51:51Z

So far I kept fileset and metricset around as we still had it in the config and I thought we keep it (for example #9922). Could you share a bit more background on this change? I assume for auditbeat it would mean both options work?

cwurm · 2019-01-11T16:49:30Z

The data Auditbeat reports are not “metrics”. “Audit events” is probably the most accurate term. So it would be weird if its sub-modules would be called “metricsets” in the user-facing configuration and documentation. It leaks its Metricbeat roots that the user should not see. “Datasets” is the more generic term that we arrived at in the SecOps team. Metricbeat and Filebeat can keep their terms, this is just for Auditbeat. Since with this change the code would overwrite the MetricSets field with the contents of `datasets`, only `datasets` will work for Auditbeat.

…

On Fri, 11 Jan 2019 at 15:51 Nicolas Ruflin ***@***.***> wrote: So far I kept fileset and metricset around as we still had it in the config and I thought we keep it (for example #9922 <#9922>). Could you share a bit more background on this change? I assume for auditbeat it would mean both options work? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#10018 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAqZRfc3mWk25WnfK_BkqpKgaYHS6yPBks5vCLMbgaJpZM4Z7hz1> .

ruflin · 2019-01-14T07:25:47Z

This is then a breaking change I assume and all users have to change their config for 7? This needs a changelog entry then.

One other thing I worry about is that we have afterwards 2 different "values" of dataset. On the event side the dataset is {module}.{metricset}, on the config side it's just what metricset was previously. Do you see this as confusing or ok?

cwurm · 2019-01-14T10:58:39Z

This is then a breaking change I assume and all users have to change their config for 7?

Auditbeat doesn't have any metricsets at the moment, so if we can get this change in for 6.6 we would not have a breaking change.

One other thing I worry about is that we have afterwards 2 different "values" of dataset. On the event side the dataset is {module}.{metricset}, on the config side it's just what metricset was previously. Do you see this as confusing or ok?

Personally, I think this is ok, and actually preferable to the situation where it's called dataset in the event, but metricset in the configuration.

ruflin · 2019-01-14T12:54:29Z

Ok, SGTM for the dataset change.

tsg

Code change LGTM. I'd wait for a review from @andrewkroh as well, to ACK the approach.

andrewkroh

LGTM.

…s" (elastic#10018) Changes the Auditbeat system module configuration to use "datasets" rather than "metricsets" to configure its sub-modules: ``` - module: system datasets: - host - process [...] ``` (cherry picked from commit e42ccb4)

…om "metricsets" to "datasets" (#10050) Cherry-pick of PR #10018 to 6.6 branch. Original message: Changes the Auditbeat system module configuration to use "datasets" rather than "metricsets" to configure its sub-modules: ``` - module: system datasets: - host - process [...] ```

…om "metricsets" to "datasets" (#10051) Cherry-pick of PR #10018 to 6.x branch. Original message: Changes the Auditbeat system module configuration to use "datasets" rather than "metricsets" to configure its sub-modules: ``` - module: system datasets: - host - process [...] ```

Since elastic/beats#10018, the Auditbeat System module uses `datasets` instead of `metricsets` to configure its sub-modules. This reflects that change and fixes the [current CI failures](https://beats-ci.elastic.co/job/elastic+beats-tester+master/242/) for beats-tester on Windows.

Follow up to #10018. Changes the Auditbeat system module documentation to say dataset rather than metricset in all places. No Go code changes.

Follow up to elastic#10018. Changes the Auditbeat system module documentation to say dataset rather than metricset in all places. No Go code changes. (cherry picked from commit 0a4a349) (cherry picked from commit 2fa9541ee595153a393b3adf2e33a44f23b7a0f6)

…" in docs (#10106) Cherry-pick of PR #10101 to 6.6 branch. Original message: Follow up to #10018. Changes the Auditbeat system module documentation to say `dataset` rather than `metricset` in all places. No Go code changes.

…" in docs (#10105) Cherry-pick of PR #10101 to 6.x branch. Original message: Follow up to #10018. Changes the Auditbeat system module documentation to say `dataset` rather than `metricset` in all places. No Go code changes.

…tion from "metricsets" to "datasets" (elastic#10050) Cherry-pick of PR elastic#10018 to 6.6 branch. Original message: Changes the Auditbeat system module configuration to use "datasets" rather than "metricsets" to configure its sub-modules: ``` - module: system datasets: - host - process [...] ```

…dataset" in docs (elastic#10106) Cherry-pick of PR elastic#10101 to 6.6 branch. Original message: Follow up to elastic#10018. Changes the Auditbeat system module documentation to say `dataset` rather than `metricset` in all places. No Go code changes.

Christoph Wurm added 3 commits January 11, 2019 13:46

Override metricsets with datasets for system module.

776e853

Fix system tests.

5e0f8a9

Comments

0ccdb70

cwurm added review needs_backport PR is waiting to be backported to other branches. Auditbeat SecOps labels Jan 11, 2019

cwurm requested a review from andrewkroh January 11, 2019 14:11

cwurm requested review from a team as code owners January 11, 2019 14:11

houndci-bot reviewed Jan 11, 2019

View reviewed changes

x-pack/auditbeat/module/system/system.go Show resolved Hide resolved

x-pack/auditbeat/module/system/system.go Show resolved Hide resolved

tsg approved these changes Jan 14, 2019

View reviewed changes

andrewkroh approved these changes Jan 14, 2019

View reviewed changes

cwurm merged commit e42ccb4 into elastic:master Jan 14, 2019

cwurm mentioned this pull request Jan 14, 2019

[Auditbeat] Cherry-pick #10018 to 6.6: Change module configuration from "metricsets" to "datasets" #10050

Merged

cwurm added v6.6.0 and removed needs_backport PR is waiting to be backported to other branches. labels Jan 14, 2019

cwurm mentioned this pull request Jan 14, 2019

[Auditbeat] Cherry-pick #10018 to 6.x: Change module configuration from "metricsets" to "datasets" #10051

Merged

cwurm added the v6.7.0 label Jan 14, 2019

cwurm mentioned this pull request Jan 15, 2019

Auditbeat System module: Change metricsets to datasets elastic/beats-tester#107

Merged

This was referenced Jan 16, 2019

[Auditbeat] Rename "metricset" to "dataset" in docs #10101

Merged

[Auditbeat] System module 6.6 #8725

Closed

cwurm pushed a commit that referenced this pull request Jan 16, 2019

[Auditbeat] Rename "metricset" to "dataset" in docs (#10101)

0a4a349

Follow up to #10018. Changes the Auditbeat system module documentation to say dataset rather than metricset in all places. No Go code changes.

cwurm mentioned this pull request Jan 16, 2019

[Auditbeat] Cherry-pick #10101 to 6.x: Rename "metricset" to "dataset" in docs #10105

Merged

cwurm mentioned this pull request Jan 16, 2019

[Auditbeat] Cherry-pick #10101 to 6.6: Rename "metricset" to "dataset" in docs #10106

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Auditbeat] Change module configuration from "metricsets" to "datasets"#10018

[Auditbeat] Change module configuration from "metricsets" to "datasets"#10018
cwurm merged 3 commits intoelastic:masterfrom
cwurm:system_config_datasets

cwurm commented Jan 11, 2019

Uh oh!

elasticmachine commented Jan 11, 2019

Uh oh!

Uh oh!

Uh oh!

ruflin commented Jan 11, 2019

Uh oh!

cwurm commented Jan 11, 2019 via email •

edited

Loading

Uh oh!

ruflin commented Jan 14, 2019 •

edited

Loading

Uh oh!

cwurm commented Jan 14, 2019

Uh oh!

ruflin commented Jan 14, 2019

Uh oh!

tsg left a comment

Uh oh!

andrewkroh left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

cwurm commented Jan 11, 2019

Uh oh!

elasticmachine commented Jan 11, 2019

Uh oh!

Uh oh!

Uh oh!

ruflin commented Jan 11, 2019

Uh oh!

cwurm commented Jan 11, 2019 via email • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ruflin commented Jan 14, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cwurm commented Jan 14, 2019

Uh oh!

ruflin commented Jan 14, 2019

Uh oh!

tsg left a comment

Choose a reason for hiding this comment

Uh oh!

andrewkroh left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

cwurm commented Jan 11, 2019 via email •

edited

Loading

ruflin commented Jan 14, 2019 •

edited

Loading