Skip to content

Make k8sattributesprocessor available in Agent running in OTel mode#4893

Merged
michalpristas merged 3 commits intoelastic:mainfrom
ChrsMark:add_filestorage_extention_otel_component
Jun 11, 2024
Merged

Make k8sattributesprocessor available in Agent running in OTel mode#4893
michalpristas merged 3 commits intoelastic:mainfrom
ChrsMark:add_filestorage_extention_otel_component

Conversation

@ChrsMark
Copy link
Member

@ChrsMark ChrsMark commented Jun 10, 2024 •
edited
Loading

What does this PR do?

This PR makes the k8sattributesprocessor available to users running Elastic Agent in otel mode.

Why is it important?

To allow users of Elastic Agent running as as OpenTelemetry Collector to configure pipelines containing the k8sattributesprocessor.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in ./changelog/fragments using the changelog tool
  • I have added an integration test or an E2E test

How to test this PR locally

  1. Prepare the binaryDEV=true EXTERNAL=true SNAPSHOT=true PLATFORMS=linux/amd64 PACKAGES=docker mage -v package
  2. build the image cd build/package/elastic-agent/elastic-agent-linux-amd64.docker/docker-build && docker build -t agent-otel-dev .
  3. Load to local cluster kind load docker-image agent-otel-dev:latest
  4. prepare the ES secrets
kubectl create secret generic elastic-secret \
     --from-literal=es_endpoint='https://test-otel-blablabla.es.us-central1.gcp.cloud.es.io/' \                 
     --from-literal=es_password='asdfgp' \     
     --from-literal=es_user='elastic'
  1. specify the image:
image: agent-otel-dev:latest
imagePullPolicy: Never
  1. Deploy the manifest from https://github.com/elastic/opentelemetry-dev/pull/258 : k apply -f elastic-otel-collector.yaml
  2. Verify the output:
2024-06-10T13:17:25.677Z	info	ResourceLog #0
Resource SchemaURL: 
Resource attributes:
     -> k8s.namespace.name: Str(kube-system)
     -> k8s.pod.name: Str(kindnet-6lgrl)
     -> k8s.container.restart_count: Str(0)
     -> k8s.pod.uid: Str(3844b30e-67ee-4bcc-98ac-8f2196852939)
     -> k8s.container.name: Str(kindnet-cni)
     -> k8s.daemonset.name: Str(kindnet)
     -> k8s.node.name: Str(kind-control-plane)
     -> k8s.pod.start_time: Str(2024-06-10T13:09:03Z)
ScopeLogs #0
ScopeLogs SchemaURL: 
InstrumentationScope  
LogRecord #0
ObservedTimestamp: 2024-06-10 13:17:25.420729149 +0000 UTC
Timestamp: 2024-06-10 13:17:25.300601954 +0000 UTC
SeverityText: 
SeverityNumber: Unspecified(0)
Body: Str(I0610 13:17:25.300466       1 main.go:223] Handling node with IPs: map[172.21.0.2:{}])
Attributes:
     -> logtag: Str(F)
     -> log.file.path: Str(/var/log/pods/kube-system_kindnet-6lgrl_3844b30e-67ee-4bcc-98ac-8f2196852939/kindnet-cni/0.log)
     -> log.iostream: Str(stderr)
     -> time: Str(2024-06-10T13:17:25.300601954Z)
Trace ID: 
Span ID: 
Flags: 0
LogRecord #1
ObservedTimestamp: 2024-06-10 13:17:25.420987445 +0000 UTC
Timestamp: 2024-06-10 13:17:25.300633177 +0000 UTC
SeverityText: 
SeverityNumber: Unspecified(0)
Body: Str(I0610 13:17:25.300494       1 main.go:227] handling current node)
Attributes:
     -> log.iostream: Str(stderr)
     -> logtag: Str(F)
     -> log.file.path: Str(/var/log/pods/kube-system_kindnet-6lgrl_3844b30e-67ee-4bcc-98ac-8f2196852939/kindnet-cni/0.log)
     -> time: Str(2024-06-10T13:17:25.300633177Z)
Trace ID: 
Span ID: 
Flags: 0
	{"kind": "exporter", "data_type": "logs", "name": "debug"}

@mergify
Copy link
Contributor

mergify bot commented Jun 10, 2024

This pull request does not have a backport label. Could you fix it @ChrsMark? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v./d./d./d is the label to automatically backport to the 8./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

@ChrsMark ChrsMark force-pushed the add_filestorage_extention_otel_component branch from fa14718 to f6dc899 Compare June 10, 2024 12:44
@ChrsMark ChrsMark marked this pull request as ready for review June 10, 2024 13:42
@ChrsMark ChrsMark requested a review from a team as a code owner June 10, 2024 13:42
@pierrehilbert pierrehilbert added the Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team label Jun 10, 2024
@elasticmachine
Copy link
Contributor

elasticmachine commented Jun 10, 2024

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

@ChrsMark
Add k8sattributesprocessor component
920a942 
Signed-off-by: ChrsMark <chrismarkou92@gmail.com>
@ChrsMark ChrsMark force-pushed the add_filestorage_extention_otel_component branch from f6dc899 to 920a942 Compare June 10, 2024 14:49
@elastic-sonarqube
Copy link

elastic-sonarqube bot commented Jun 11, 2024

Quality Gate passed Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

michalpristas
michalpristas approved these changes Jun 11, 2024
View reviewed changes
Copy link
Contributor

@michalpristas michalpristas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@michalpristas michalpristas merged commit 2e09c33 into elastic:main Jun 11, 2024
@ChrsMark ChrsMark mentioned this pull request Jun 12, 2024
Merged
@michalpristas michalpristas added backport-v8.14.0 Automated backport with mergify and removed backport-skip labels Jun 21, 2024
mergify bot pushed a commit that referenced this pull request Jun 21, 2024
@ChrsMark @mergify
Add k8sattributesprocessor component (#4893)
2ff344d 
Signed-off-by: ChrsMark <chrismarkou92@gmail.com>
Co-authored-by: Michal Pristas <michal.pristas@gmail.com>
(cherry picked from commit 2e09c33)

# Conflicts:
#	go.mod
#	go.sum
#	internal/pkg/otel/README.md
#	internal/pkg/otel/components.go
michalpristas pushed a commit that referenced this pull request Jun 21, 2024
@mergify
[8.14](backport #4893) Make k8sattributesprocessor available in Age…
c7a9d27 
…nt running in OTel mode (#4981)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@michalpristas michalpristas michalpristas approved these changes

@andrzej-stencel andrzej-stencel Awaiting requested review from andrzej-stencel andrzej-stencel is a code owner automatically assigned from elastic/elastic-agent-control-plane

Assignees

@ChrsMark ChrsMark

Labels

backport-v8.14.0 Automated backport with mergify Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@ChrsMark @elasticmachine @michalpristas @pierrehilbert