Skip to content

Add nodes to filebeat-kubernetes.yaml ClusterRole - fixes #24051#24052

Merged
ChrsMark merged 3 commits intoelastic:masterfrom
jeffspahr:master
Feb 17, 2021
Merged

Add nodes to filebeat-kubernetes.yaml ClusterRole - fixes #24051#24052
ChrsMark merged 3 commits intoelastic:masterfrom
jeffspahr:master

Conversation

@jeffspahr
Copy link
Copy Markdown
Contributor

@jeffspahr jeffspahr commented Feb 16, 2021 •
edited
Loading

What does this PR do?

Adds nodes to the fileabeat ClusterRole.
Fixes #24051

Why is it important?

Filebeat will throw an error and be unable to list nodes without this.

Checklist

  • My code follows the style guidelines of this project
    - [ ] I have commented my code, particularly in hard-to-understand areas
    - [ ] I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
    - [ ] I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

How to test this PR locally

Deploy Filebeat from the latest manifest in the docs:
https://www.elastic.co/guide/en/beats/filebeat/7.11/running-on-kubernetes.html
curl -L -O https://raw.githubusercontent.com/elastic/beats/7.11/deploy/kubernetes/filebeat-kubernetes.yaml

Use the autodiscover config by doing the following:
# To enable hints based autodiscover, remove filebeat.inputs configuration and uncomment this:

Filebeat starts up and is unable to list nodes:

E0215 03:45:32.109053 7 reflector.go:127] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: Failed to watch *v1.Node: failed to list *v1.Node: nodes "k3s-01a.spahr.dev" is forbidden: User "system:serviceaccount:bourbontracker:filebeat" cannot list resource "nodes" in API group "" at the cluster scope
E0215 03:45:33.243209 7 reflector.go:127] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: Failed to watch *v1.Node: failed to list *v1.Node: nodes "k3s-01a.spahr.dev" is forbidden: User "system:serviceaccount:bourbontracker:filebeat" cannot list resource "nodes" in API group "" at the cluster scope

This error goes away after adding nodes to the ClusterRole.

Related issues

Use cases

Screenshots

Logs

Logs without the change:

E0215 03:45:32.109053 7 reflector.go:127] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: Failed to watch *v1.Node: failed to list *v1.Node: nodes "k3s-01a.spahr.dev" is forbidden: User "system:serviceaccount:bourbontracker:filebeat" cannot list resource "nodes" in API group "" at the cluster scope
E0215 03:45:33.243209 7 reflector.go:127] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: Failed to watch *v1.Node: failed to list *v1.Node: nodes "k3s-01a.spahr.dev" is forbidden: User "system:serviceaccount:bourbontracker:filebeat" cannot list resource "nodes" in API group "" at the cluster scope

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Feb 16, 2021
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Feb 16, 2021 •
edited by jenkins-beats-ci bot
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: jsoriano commented: /test

  • Start Time: 2021-02-16T19:18:04.139+0000

  • Duration: 60 min 40 sec

  • Commit: 6e57685

Trends 🧪

Image of Build Times

❕ Flaky test report

No test was executed to be analysed.

@andresrc andresrc added the Team:Integrations Label for the Integrations team label Feb 16, 2021
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Feb 16, 2021

Pinging @elastic/integrations (Team:Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Feb 16, 2021
@ChrsMark ChrsMark self-assigned this Feb 16, 2021
@ChrsMark ChrsMark self-requested a review February 16, 2021 09:00
@ChrsMark
Copy link
Copy Markdown
Member

ChrsMark commented Feb 16, 2021

Hey! Thanks for working on this! You will need to update https://github.com/elastic/beats/blob/master/deploy/kubernetes/filebeat/filebeat-role.yaml too in order to have a happy/green CI.

@jeffspahr
Copy link
Copy Markdown
Contributor Author

jeffspahr commented Feb 16, 2021

Thanks for catching that @ChrsMark! I just added that in a new commit.

@jsoriano
Copy link
Copy Markdown
Member

jsoriano commented Feb 16, 2021

/test

ChrsMark
ChrsMark approved these changes Feb 17, 2021
View reviewed changes
Copy link
Copy Markdown
Member

@ChrsMark ChrsMark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! Thanks for fixing this @jeffspahr!

@ChrsMark ChrsMark merged commit 410d900 into elastic:master Feb 17, 2021
@ChrsMark ChrsMark mentioned this pull request Feb 17, 2021
Merged
3 tasks
ChrsMark pushed a commit to ChrsMark/beats that referenced this pull request Feb 17, 2021
@jeffspahr @ChrsMark
Add nodes to filebeat-kubernetes.yaml ClusterRole - fixes elastic#24051…
34e671f 
… (elastic#24052)

(cherry picked from commit 410d900)
@ChrsMark ChrsMark mentioned this pull request Feb 17, 2021
Merged
3 tasks
ChrsMark pushed a commit to ChrsMark/beats that referenced this pull request Feb 17, 2021
@jeffspahr @ChrsMark
Add nodes to filebeat-kubernetes.yaml ClusterRole - fixes elastic#24051…
e7d8967 
… (elastic#24052)

(cherry picked from commit 410d900)
v1v added a commit to v1v/beats that referenced this pull request Feb 17, 2021
@v1v
Merge remote-tracking branch 'upstream/master' into feature/packaging…
e737d7e 
…-arm

* upstream/master:
  [CI] install docker-compose with retry (elastic#24069)
  Add nodes to filebeat-kubernetes.yaml ClusterRole - fixes elastic#24051 (elastic#24052)
  updating manifest files for filebeat threatintel module (elastic#24074)
  Add Zeek Signatures (elastic#23772)
  Update Beats to ECS 1.8.0 (elastic#23465)
  Support running Docker logging plugin on ARM64 (elastic#24034)
  Fix ec2 metricset fields.yml and add integration test (elastic#23726)
  Only build targz and zip versions of Beats if PACKAGES is set in agent (elastic#24060)
  [Filebeat] Add field definitions for known Netflow/IPFIX vendor fields (elastic#23773)
  [Elastic Agent] Enroll with Fleet Server (elastic#23865)
  [Filebeat] Convert logstash logEvent.action objects to strings (elastic#23944)
  [Ingest Management] Fix reloading of log level for services (elastic#24055)
  Add Agent standalone k8s manifest (elastic#23679)
ChrsMark added a commit that referenced this pull request Feb 17, 2021
@ChrsMark @jeffspahr
Add nodes to filebeat-kubernetes.yaml ClusterRole - fixes #24051 (#24052
340760f 
) (#24080)

(cherry picked from commit 410d900)

Co-authored-by: Jeff Spahr <spahrj@gmail.com>
v1v added a commit to v1v/beats that referenced this pull request Feb 17, 2021
@v1v
Merge remote-tracking branch 'upstream/master' into feature/retry-win…
e977e49 
…dows-7

* upstream/master: (332 commits)
  Use ECS v1.8.0 (elastic#24086)
  Add support for postgresql csv logs (elastic#23334)
  [Heartbeat] Refactor config system (elastic#23467)
  [CI] install docker-compose with retry (elastic#24069)
  Add nodes to filebeat-kubernetes.yaml ClusterRole - fixes elastic#24051 (elastic#24052)
  updating manifest files for filebeat threatintel module (elastic#24074)
  Add Zeek Signatures (elastic#23772)
  Update Beats to ECS 1.8.0 (elastic#23465)
  Support running Docker logging plugin on ARM64 (elastic#24034)
  Fix ec2 metricset fields.yml and add integration test (elastic#23726)
  Only build targz and zip versions of Beats if PACKAGES is set in agent (elastic#24060)
  [Filebeat] Add field definitions for known Netflow/IPFIX vendor fields (elastic#23773)
  [Elastic Agent] Enroll with Fleet Server (elastic#23865)
  [Filebeat] Convert logstash logEvent.action objects to strings (elastic#23944)
  [Ingest Management] Fix reloading of log level for services (elastic#24055)
  Add Agent standalone k8s manifest (elastic#23679)
  [Metricbeat][Kubernetes] Extend state_node with more conditions (elastic#23905)
  [CI] googleStorageUploadExt step (elastic#24048)
  Check fields are documented for aws metricsets (elastic#23887)
  Update go-concert to 0.1.0 (elastic#23770)
  ...
ChrsMark added a commit that referenced this pull request Feb 18, 2021
@ChrsMark
Cherry-pick #24052 to 7.11: Add nodes to filebeat-kubernetes.yaml Clu…
3996720 
…sterRole - fixes #24051 (#24081)
@maggieghamry
Copy link
Copy Markdown

maggieghamry commented Apr 23, 2021

@ChrsMark @jeffspahr is there a similar PR to fix this same issue in 7.11 for Metricbeat as well? (Same issue is occurring there)

@maggieghamry maggieghamry mentioned this pull request Apr 23, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ChrsMark ChrsMark ChrsMark approved these changes

Assignees

@ChrsMark ChrsMark

Labels

Team:Integrations Label for the Integrations team v7.11.1 v7.12.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Filebeat kubernetes manifest is missing nodes in the ClusterRole

6 participants

@jeffspahr @elasticmachine @ChrsMark @jsoriano @maggieghamry @andresrc