Skip to content

obsservice: add logs_service proto for OTLP collector#93733

Merged
craig[bot] merged 2 commits intocockroachdb:masterfrom
aadityasondhi:aaditya/obsservice-OTLP
Jan 23, 2023
Merged

obsservice: add logs_service proto for OTLP collector#93733
craig[bot] merged 2 commits intocockroachdb:masterfrom
aadityasondhi:aaditya/obsservice-OTLP

Conversation

@aadityasondhi
Copy link
Copy Markdown
Contributor

@aadityasondhi aadityasondhi commented Dec 15, 2022

This patch adds the required protos for implementing the logs_service for the OTLP collector. This will allow cockroach to export events to an OTEL collector in the form of OTEL-compatible logs using the LogsServiceClient. Additionally, obsservice can ingest these logs as a LogsServiceServer when used in place of an OTEL collector.

Epic: CRDB-16791

Release note: None

@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@aadityasondhi aadityasondhi force-pushed the aaditya/obsservice-OTLP branch from 5b9326e to 4a92b37 Compare December 15, 2022 22:50
@aadityasondhi aadityasondhi marked this pull request as ready for review December 19, 2022 18:38
@aadityasondhi aadityasondhi requested a review from a team December 19, 2022 18:38
@aadityasondhi aadityasondhi requested a review from a team as a code owner December 19, 2022 18:38
@aadityasondhi aadityasondhi requested review from andreimatei and removed request for a team December 19, 2022 18:38
Copy link
Copy Markdown
Collaborator

@dhartunian dhartunian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @andreimatei)

Copy link
Copy Markdown
Contributor

@andreimatei andreimatei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @aadityasondhi)


pkg/obsservice/obspb/update-opentelemetry-proto.sh line 83 at r1 (raw file):

# into:
# option go_package = "v1";
find $DEST_DIR -type f -name "*.proto" -exec sed -i '' -e "s/option go_package = \"go.opentelemetry.io\/proto\/otlp\/.*\/v1\"/option go_package = \"v1\"/" {} + ;

why did this change?
I don't see -i taking an argument in the man page.

Copy link
Copy Markdown
Contributor Author

@aadityasondhi aadityasondhi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @andreimatei)


pkg/obsservice/obspb/update-opentelemetry-proto.sh line 83 at r1 (raw file):
It errored on my local machine with:

./update-opentelemetry-proto.sh
Cloning opentelemetry-proto in /var/folders/sy/cstk63m54r992k2zybwh1_p80000gq/T/tmp.UX5yJRME.
Making a backup copy of opentelemetry-proto in /var/folders/sy/cstk63m54r992k2zybwh1_p80000gq/T/tmp.UVc0KC7t.
Copying protos.
Editing protos.
sed: 1: "opentelemetry-proto/com ...": invalid command code o
Deleted temp working directory /var/folders/sy/cstk63m54r992k2zybwh1_p80000gq/T/tmp.UX5yJRME

Did some digging and found: https://stackoverflow.com/questions/19456518/error-when-using-sed-with-find-command-on-os-x-invalid-command-code

"On the OSX version of sed, the -i option expects an extension argument so your command is actually parsed as the extension argument and the file path is interpreted as the command code.
Try adding the -e argument explicitly and giving '' as argument to -i"

I am assuming your machine is running Linux?

Copy link
Copy Markdown
Contributor Author

@aadityasondhi aadityasondhi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @andreimatei)


pkg/obsservice/obspb/update-opentelemetry-proto.sh line 83 at r1 (raw file):

Previously, aadityasondhi (Aaditya Sondhi) wrote…

It errored on my local machine with:

./update-opentelemetry-proto.sh
Cloning opentelemetry-proto in /var/folders/sy/cstk63m54r992k2zybwh1_p80000gq/T/tmp.UX5yJRME.
Making a backup copy of opentelemetry-proto in /var/folders/sy/cstk63m54r992k2zybwh1_p80000gq/T/tmp.UVc0KC7t.
Copying protos.
Editing protos.
sed: 1: "opentelemetry-proto/com ...": invalid command code o
Deleted temp working directory /var/folders/sy/cstk63m54r992k2zybwh1_p80000gq/T/tmp.UX5yJRME

Did some digging and found: https://stackoverflow.com/questions/19456518/error-when-using-sed-with-find-command-on-os-x-invalid-command-code

"On the OSX version of sed, the -i option expects an extension argument so your command is actually parsed as the extension argument and the file path is interpreted as the command code.
Try adding the -e argument explicitly and giving '' as argument to -i"

I am assuming your machine is running Linux?

I wonder if this breaks the Linux version since -i doesn't take an argument. I can add a condition based on the OS in the script to make it work for both.

@aadityasondhi aadityasondhi force-pushed the aaditya/obsservice-OTLP branch from 4a92b37 to b361cf0 Compare January 9, 2023 18:31
Copy link
Copy Markdown
Contributor Author

@aadityasondhi aadityasondhi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @andreimatei)


pkg/obsservice/obspb/update-opentelemetry-proto.sh line 83 at r1 (raw file):

Previously, aadityasondhi (Aaditya Sondhi) wrote…

I wonder if this breaks the Linux version since -i doesn't take an argument. I can add a condition based on the OS in the script to make it work for both.

@andreimatei Fixed it to make it work with both OS. Let me know what you think.

Copy link
Copy Markdown
Contributor

@andreimatei andreimatei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @aadityasondhi)


pkg/obsservice/obspb/update-opentelemetry-proto.sh line 83 at r1 (raw file):

Previously, aadityasondhi (Aaditya Sondhi) wrote…

@andreimatei Fixed it to make it work with both OS. Let me know what you think.

damn BSDs

Let's find something that works across both of them, otherwise there's too many lines
I think this should work:

sed -i.bak -e ...
rm *.bak

And please extract this in a separate commit.

This patch adds the required protos for implementing the logs_service
for the OTLP collector. This will allow cockroach to export events to an
OTEL collector in the form of OTEL-compatible logs using the
`LogsServiceClient`. Additionally, obsservice can ingest these logs as a
`LogsServiceServer` when used in place of an OTEL collector.

Epic: CRDB-16791

Release note: None
The OSX implementation of sed did not support the -i flag without
parameters. This change makes the script compatible with both the linux
and OSX implementation of sed.

Release note: None
@aadityasondhi aadityasondhi force-pushed the aaditya/obsservice-OTLP branch from b361cf0 to c16b213 Compare January 19, 2023 16:03
Copy link
Copy Markdown
Contributor Author

@aadityasondhi aadityasondhi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @andreimatei)


pkg/obsservice/obspb/update-opentelemetry-proto.sh line 83 at r1 (raw file):

Previously, andreimatei (Andrei Matei) wrote…

damn BSDs

Let's find something that works across both of them, otherwise there's too many lines
I think this should work:

sed -i.bak -e ...
rm *.bak

And please extract this in a separate commit.

Done.

Copy link
Copy Markdown
Contributor

@andreimatei andreimatei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@aadityasondhi
Copy link
Copy Markdown
Contributor Author

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Jan 23, 2023

Build succeeded:

@craig craig bot merged commit b673ada into cockroachdb:master Jan 23, 2023
@aadityasondhi aadityasondhi deleted the aaditya/obsservice-OTLP branch January 30, 2023 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants