Skip to content

Add support for ephemeral containers in elastic agent dynamic provider#27707

Merged
MichaelKatsoulis merged 4 commits intoelastic:masterfrom
MichaelKatsoulis:support-ephemeral-container-k8s-provider
Sep 7, 2021
Merged

Add support for ephemeral containers in elastic agent dynamic provider#27707
MichaelKatsoulis merged 4 commits intoelastic:masterfrom
MichaelKatsoulis:support-ephemeral-container-k8s-provider

Conversation

@MichaelKatsoulis
Copy link
Copy Markdown
Contributor

@MichaelKatsoulis MichaelKatsoulis commented Sep 2, 2021 •
edited
Loading

What does this PR do?

This PR adds support for ephemeral containers in elastic agent dynamic kubernetes provider.

Why is it important?

Ephemeral containers that are part of a pod can be monitored and be part of the published event.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

  1. Create a kind cluster with "feature-gates": "EphemeralContainers=true". example kind config (
    kind-ephemeral.yaml.zip)
  2. Create en example pod
cat <<EOF | kubectl create -f - 
apiVersion: v1
kind: Pod
metadata:
  name: example-pod
spec:
  containers:
  - image: mhausenblas/simpleservice:0.5.0
    name: app
EOF
  1. Attach an ephemeral container to it kubectl debug -it example-pod --image=busybox --target=app
  2. Run agent with pod resource enabled
providers.kubernetes:
  scope: cluster
  cleanup_timeout: 360s
  resources:
    pod:
       enabled: true
  1. Run inspect command to check what is the compiled configuration (use proper path for the elastic-agent.yml config file :
    elastic-agent -c /etc/agent.yml -e inspect output -o default
  2. Verify in the output that mapping is produced for the ephemeral container
processors:
    - add_fields:
        fields:
          container:
            id: a8d7f67674c848c27e71b7a97f3f82d9492080308a72153458a15df7485904bf
            image: busybox
            name: debugger-pbjkz
            runtime: containerd
          namespace: default
          pod:
            ip: 10.244.0.5
            labels: {}
            name: example-pod
            uid: d7a97d03-3baa-457c-b9e2-a8e80b831c23
        target: kubernetes

Related issues

@MichaelKatsoulis MichaelKatsoulis added enhancement backport-v7.16.0 Automated backport with mergify labels Sep 2, 2021
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Sep 2, 2021
@MichaelKatsoulis MichaelKatsoulis added the Team:Integrations Label for the Integrations team label Sep 2, 2021
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Sep 2, 2021

Pinging @elastic/integrations (Team:Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Sep 2, 2021
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Sep 2, 2021 •
edited by jenkins-beats-ci bot
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-09-07T09:50:59.329+0000

  • Duration: 82 min 49 sec

  • Commit: 094f149

Test stats 🧪

Test Results
Failed 0
Passed 7036
Skipped 16
Total 7052

Trends 🧪

Image of Build Times

Image of Tests

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 7036
Skipped 16
Total 7052

@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Sep 2, 2021

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b support-ephemeral-container-k8s-provider upstream/support-ephemeral-container-k8s-provider
git merge upstream/master
git push upstream support-ephemeral-container-k8s-provider

ChrsMark
ChrsMark approved these changes Sep 7, 2021
View reviewed changes
Copy link
Copy Markdown
Member

@ChrsMark ChrsMark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! We will also cover this with e2e in elastic/e2e-testing#1090

@MichaelKatsoulis MichaelKatsoulis merged commit 8afb80e into elastic:master Sep 7, 2021
mergify bot pushed a commit that referenced this pull request Sep 7, 2021
@MichaelKatsoulis
Add support for ephemeral containers in elastic agent dynamic provider (
d2915ea 
#27707)

* Add support for ephemeral containers in elastic agent dynamic provider

(cherry picked from commit 8afb80e)
@mergify mergify bot mentioned this pull request Sep 7, 2021
Merged
MichaelKatsoulis added a commit that referenced this pull request Sep 7, 2021
@mergify @MichaelKatsoulis
Add support for ephemeral containers in elastic agent dynamic provider (
f5dceab 
#27707) (#27772)

* Add support for ephemeral containers in elastic agent dynamic provider

(cherry picked from commit 8afb80e)

Co-authored-by: Michael Katsoulis <michaelkatsoulis88@gmail.com>
mdelapenya added a commit to mdelapenya/beats that referenced this pull request Sep 9, 2021
@mdelapenya
Merge branch 'master' into 27608-fix-elastic-agent-build
16ae989 
* master: (39 commits)
  [Heartbeat] Move JSON tests from python->go (elastic#27816)
  docs: simplify permissions for Dockerfile COPY (elastic#27754)
  Osquerybeat: Fix osquery logger plugin severy levels mapping (elastic#27789)
  [Filebeat] Update compatibility function to remove processor description on ES < 7.9.0 (elastic#27774)
  warn log entry and no validation failure when both queue_url and buck… (elastic#27612)
  libbeat/cmd/instance: ensure test config file has appropriate permissions (elastic#27178)
  [Heartbeat] Add httpcommon options to ZipURL (elastic#27699)
  Add a header round tripper option to httpcommon (elastic#27509)
  [Elastic Agent] Add validation to ensure certificate paths are absolute. (elastic#27779)
  Rename dashboards according to module.yml files for master (elastic#27749)
  Refactor vagrantfile, add scripts for provisioning with docker/kind (elastic#27726)
  Accept syslog dates with leading 0 (elastic#27775)
  [Filebeat] Add timezone config option to decode_cef and syslog input (elastic#27727)
  [Filebeat] Threatintel compatibility updates (elastic#27323)
  Add support for ephemeral containers in elastic agent dynamic provider (elastic#27707)
  [Filebeat] Integration tests in CI for AWS-S3 input (elastic#27491)
  Fix flakyness of TestFilestreamEmptyLine (elastic#27705)
  [Filebeat] kafka v2 using parsers (elastic#27335)
  Update Kafka version parsing / supported range (elastic#27720)
  Update Sarama to 1.29.1 (elastic#27717)
  ...
Icedroid pushed a commit to Icedroid/beats that referenced this pull request Nov 1, 2021
@MichaelKatsoulis
Add support for ephemeral containers in elastic agent dynamic provider (
e17cb69 
elastic#27707)

* Add support for ephemeral containers in elastic agent dynamic provider
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ChrsMark ChrsMark ChrsMark approved these changes

@exekias exekias Awaiting requested review from exekias

Assignees

No one assigned

Labels

backport-v7.16.0 Automated backport with mergify enhancement Team:Integrations Label for the Integrations team

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Agent] Support ephemeral containers in k8s provider

3 participants

@MichaelKatsoulis @elasticmachine @ChrsMark