Skip to content

feat: support for both HTTP and gRPC protocols in meter/logging/tracing#1490

Merged
tolgaozen merged 3 commits intoPermify:masterfrom
vikashsprem:feat/support-grpc
Aug 24, 2024
Merged

feat: support for both HTTP and gRPC protocols in meter/logging/tracing#1490
tolgaozen merged 3 commits intoPermify:masterfrom
vikashsprem:feat/support-grpc

Conversation

@vikashsprem
Copy link
Copy Markdown

@vikashsprem vikashsprem commented Aug 24, 2024

/claim #1090
/close #1090

Add support for both HTTP and gRPC protocols in meter/logging/tracing in meter/logging/tracing exports

Summary by CodeRabbit

  • New Features

    • Introduced a new Protocol configuration field for log, tracing, and metrics exporters, allowing users to specify communication protocols (HTTP or gRPC) for enhanced data transmission.
    • Enhanced exporter creation functions to accept the protocol parameter, increasing flexibility in configuring log, tracer, and meter exporters.
  • Bug Fixes

    • Improved error handling for unsupported protocol inputs in OTLP exporter functions, ensuring clearer feedback to users.
  • Documentation

    • Updated documentation to reflect changes made to configuration options and exporter functionalities.

@vercel
Copy link
Copy Markdown

vercel bot commented Aug 24, 2024

Someone is attempting to deploy a commit to the permify Team on Vercel.

A member of the Team first needs to authorize it.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Aug 24, 2024

Caution

Review failed

The pull request is closed.

Walkthrough

The changes introduce a new Protocol field to the configuration of log, trace, and meter exporters, allowing the specification of communication protocols such as HTTP or gRPC. This enhancement modifies various factory and initialization functions across multiple files to accept and properly handle this new parameter, making the exporters more flexible in their configuration.

Changes

Files Change Summary
internal/config/config.go Added Protocol field to LogExporter, Tracer, and Meter types for specifying the communication protocol.
pkg/cmd/config.go Introduced new flags for log-protocol, tracer-protocol, and meter-protocol to set communication protocols, and integrated these settings into the configuration output.
pkg/cmd/serve.go Updated serve function to include protocol parameter for logger, tracer, and meter initialization.
pkg/telemetry/logexporters/factory.go, Modified ExporterFactory to add protocol parameter for log exporter creation, adjusting logic to pass the protocol during initialization.
pkg/telemetry/logexporters/otlp.go, Enhanced NewOTLP function with protocol parameter, adding logic to handle HTTP and gRPC cases via a switch statement.
pkg/telemetry/meterexporters/factory.go, Updated ExporterFactory for meter exporters to include protocol parameter and adjust logic accordingly.
pkg/telemetry/meterexporters/otlp.go, Modified NewOTLP function to accept protocol parameter, providing support for both HTTP and gRPC protocols in its implementation.
pkg/telemetry/tracerexporters/factory.go, Updated ExporterFactory for tracer exporters to include protocol parameter and pass it dynamically instead of using hardcoded values.
pkg/telemetry/tracerexporters/otlp.go Enhanced NewOTLP function with protocol parameter, improving flexibility in exporter creation based on specified protocol.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant ServeFunction
    participant ExporterFactory
    participant NewOTLP

    User->>ServeFunction: Call with protocol
    ServeFunction->>ExporterFactory: Create LogExporter with protocol
    ExporterFactory->>NewOTLP: Create using protocol
    NewOTLP-->>ExporterFactory: Return configured exporter
    ExporterFactory-->>ServeFunction: Return exporter
    ServeFunction-->>User: Return success
Loading

Assessment against linked issues

Objective Addressed Explanation
Support gRPC for OTLP exporters (issue #1090) βœ…
Expand configuration with protocol option βœ…
Enable dynamic protocol selection in exporters βœ…

Poem

πŸ‡ Hopping through code, what a delight,
With gRPC support, the future is bright!
Protocols now dance in the config so fair,
Logs, traces, and metrics, all handled with care.
A leap for flexibility, a jump full of cheer,
Here's to our exporters, let's give a loud cheer! πŸŽ‰


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@algora-pbc
Copy link
Copy Markdown

algora-pbc bot commented Aug 24, 2024

πŸ’΅ To receive payouts, sign up on Algora, link your Github account and connect with Stripe.

@vikashsprem vikashsprem changed the title Add support for both HTTP and gRPC protocols in meter/logging/tracing [feat]: support for both HTTP and gRPC protocols in meter/logging/tracing Aug 24, 2024
@vikashsprem vikashsprem changed the title [feat]: support for both HTTP and gRPC protocols in meter/logging/tracing feat: support for both HTTP and gRPC protocols in meter/logging/tracing Aug 24, 2024
@vikashsprem
Copy link
Copy Markdown
Author

@tolgaozen Please review this PR. Thank you.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 09ab66e and 63ca8d9.

Files selected for processing (8)
  • internal/config/config.go (3 hunks)
  • pkg/cmd/serve.go (3 hunks)
  • pkg/telemetry/logexporters/factory.go (1 hunks)
  • pkg/telemetry/logexporters/otlp.go (1 hunks)
  • pkg/telemetry/meterexporters/factory.go (1 hunks)
  • pkg/telemetry/meterexporters/otlp.go (1 hunks)
  • pkg/telemetry/tracerexporters/factory.go (1 hunks)
  • pkg/telemetry/tracerexporters/otlp.go (1 hunks)
Additional comments not posted (15)
pkg/telemetry/logexporters/otlp.go (3)

15-36: LGTM: HTTP protocol implementation.

The HTTP case is well-implemented, correctly handling URL path and insecure connections.


38-54: LGTM: gRPC protocol implementation.

The gRPC case is well-implemented, correctly handling insecure connections.


56-58: LGTM: Default case for unsupported protocols.

The default case correctly handles unsupported protocols by returning an error.

pkg/telemetry/tracerexporters/otlp.go (3)

16-38: LGTM: HTTP protocol implementation.

The HTTP case is well-implemented, correctly handling headers, URL path, and insecure connections.


40-60: LGTM: gRPC protocol implementation.

The gRPC case is well-implemented, correctly handling headers and secure connections.


62-64: LGTM: Default case for unsupported protocols.

The default case correctly handles unsupported protocols by returning an error.

pkg/telemetry/meterexporters/otlp.go (3)

16-39: LGTM: HTTP protocol implementation.

The HTTP case is well-implemented, correctly handling headers, URL path, and insecure connections.


41-58: LGTM: gRPC protocol implementation.

The gRPC case is well-implemented, correctly handling headers and secure connections.


60-61: LGTM: Default case for unsupported protocols.

The default case correctly handles unsupported protocols by returning an error.

internal/config/config.go (3)

98-98: LGTM! Protocol field added to LogExporter.

The addition of the Protocol field to the LogExporter struct is clear and consistent with the intended functionality.


109-109: LGTM! Protocol field added to Tracer.

The addition of the Protocol field to the Tracer struct is consistent and well-documented.


121-121: LGTM! Protocol field added to Meter.

The addition of the Protocol field to the Meter struct is consistent and well-documented.

pkg/cmd/serve.go (3)

203-203: LGTM! Protocol field used in logger configuration.

The use of the Protocol field in the logger configuration is consistent and aligns with the intended functionality.


297-297: LGTM! Protocol field used in tracer configuration.

The use of the Protocol field in the tracer configuration is consistent and aligns with the intended functionality.


349-349: LGTM! Protocol field used in meter configuration.

The use of the Protocol field in the meter configuration is consistent and aligns with the intended functionality.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 09ab66e and 63ca8d9.

Files selected for processing (8)
  • internal/config/config.go (3 hunks)
  • pkg/cmd/serve.go (3 hunks)
  • pkg/telemetry/logexporters/factory.go (1 hunks)
  • pkg/telemetry/logexporters/otlp.go (1 hunks)
  • pkg/telemetry/meterexporters/factory.go (1 hunks)
  • pkg/telemetry/meterexporters/otlp.go (1 hunks)
  • pkg/telemetry/tracerexporters/factory.go (1 hunks)
  • pkg/telemetry/tracerexporters/otlp.go (1 hunks)
Additional comments not posted (9)
pkg/telemetry/logexporters/otlp.go (1)

13-58: LGTM! But verify the function usage in the codebase.

The changes are well-structured and enhance the flexibility of the NewOTLP function by supporting both HTTP and gRPC protocols. The error handling for unsupported protocols is a good addition.

Ensure that all function calls to NewOTLP correctly pass the new protocol parameter.

Run the following script to verify the function usage:

Verification successful

Function Usage Verified: All Calls to NewOTLP Pass the Protocol Parameter Correctly

The function NewOTLP is used in various parts of the codebase, and all instances correctly pass the protocol parameter as required. The calls specify the protocol explicitly, ensuring the function operates as intended.

  • pkg/telemetry/meterexporters/factory.go
  • pkg/telemetry/tracerexporters/factory.go
  • pkg/telemetry/logexporters/factory.go
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify all function calls to `NewOTLP` match the new signature.

# Test: Search for the function usage. Expect: Only occurrences of the new signature.
rg --type go -A 5 $'NewOTLP'

Length of output: 4634

pkg/telemetry/tracerexporters/otlp.go (1)

14-64: LGTM! But verify the function usage in the codebase.

The changes are consistent and enhance the flexibility of the NewOTLP function by supporting both HTTP and gRPC protocols. The error handling for unsupported protocols is a good addition.

Ensure that all function calls to NewOTLP correctly pass the new protocol parameter.

Run the following script to verify the function usage:

Verification successful

Function Usage Verified: All NewOTLP Calls Updated with Protocol Parameter

The NewOTLP function calls across the codebase correctly pass the new protocol parameter, ensuring consistency with the updated function signature.

  • pkg/telemetry/tracerexporters/factory.go: Calls NewOTLP with the protocol parameter.
  • pkg/telemetry/meterexporters/factory.go: Calls NewOTLP with the protocol parameter.
  • pkg/telemetry/logexporters/factory.go: Calls NewOTLP with the protocol parameter.
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify all function calls to `NewOTLP` match the new signature.

# Test: Search for the function usage. Expect: Only occurrences of the new signature.
rg --type go -A 5 $'NewOTLP'

Length of output: 4634

pkg/telemetry/meterexporters/otlp.go (1)

14-61: LGTM! But verify the function usage in the codebase.

The changes are consistent and enhance the flexibility of the NewOTLP function by supporting both HTTP and gRPC protocols. The error handling for unsupported protocols is a good addition.

Ensure that all function calls to NewOTLP correctly pass the new protocol parameter.

Run the following script to verify the function usage:

Verification successful

Function usage verified successfully. All calls to the NewOTLP function correctly pass the protocol parameter as intended. The changes are consistent across the codebase.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify all function calls to `NewOTLP` match the new signature.

# Test: Search for the function usage. Expect: Only occurrences of the new signature.
rg --type go -A 5 $'NewOTLP'

Length of output: 4634

internal/config/config.go (3)

98-98: Addition of Protocol field in Log struct.

The Protocol field has been added to allow specification of the communication protocol for the log exporter. This change aligns with the PR objectives to support both HTTP and gRPC protocols.


109-109: Addition of Protocol field in Tracer struct.

The Protocol field has been added to allow specification of the communication protocol for the tracing exporter. This change aligns with the PR objectives to support both HTTP and gRPC protocols.


121-121: Addition of Protocol field in Meter struct.

The Protocol field has been added to allow specification of the communication protocol for the metrics exporter. This change aligns with the PR objectives to support both HTTP and gRPC protocols.

pkg/cmd/serve.go (3)

203-203: Addition of Protocol parameter in logger initialization.

The Protocol parameter has been added to the logexporters.ExporterFactory call, enhancing the flexibility of the logging configuration by allowing different protocols.


297-297: Addition of Protocol parameter in tracer initialization.

The Protocol parameter has been added to the tracerexporters.ExporterFactory call, enhancing the flexibility of the tracing configuration by allowing different protocols.


349-349: Addition of Protocol parameter in meter initialization.

The Protocol parameter has been added to the meterexporters.ExporterFactory call, enhancing the flexibility of the metering configuration by allowing different protocols.

Comment on lines +10 to +15
func ExporterFactory(name, endpoint string, insecure bool, urlpath string, headers map[string]string, protocol string) (metric.Exporter, error) {
switch name {
case "otlp", "otlp-http":
return NewOTLP(endpoint, insecure, urlpath, headers)
return NewOTLP(endpoint, insecure, urlpath, headers, "http")
case "otlp-grpc":
return NewOTLPGrpc(endpoint, insecure, headers)
return NewOTLP(endpoint, insecure, urlpath, headers, "grpc")
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Utilize the protocol parameter directly.

The protocol parameter is added to the function signature but is not used in the logic. Consider using the protocol parameter directly instead of hardcoding the values.

Apply this diff to use the protocol parameter:

 switch name {
 case "otlp", "otlp-http":
-	 return NewOTLP(endpoint, insecure, urlpath, headers, "http")
 case "otlp-grpc":
-	 return NewOTLP(endpoint, insecure, urlpath, headers, "grpc")
+	 return NewOTLP(endpoint, insecure, urlpath, headers, protocol)

Committable suggestion was skipped due to low confidence.

Comment on lines +10 to +15
func ExporterFactory(name, endpoint string, insecure bool, urlpath string, headers map[string]string, protocol string) (*otlplogs.Exporter, error) {
switch name {
case "otlp", "otlp-http":
return NewOTLP(endpoint, insecure, urlpath, headers)
return NewOTLP(endpoint, insecure, urlpath, headers, "http")
case "otlp-grpc":
return nil, fmt.Errorf("%s log exporter is unsupported", name)
return NewOTLP(endpoint, insecure, urlpath, headers, "grpc")
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Utilize the protocol parameter directly.

The protocol parameter is added to the function signature but is not used in the logic. Consider using the protocol parameter directly instead of hardcoding the values.

Apply this diff to use the protocol parameter:

 switch name {
 case "otlp", "otlp-http":
-	 return NewOTLP(endpoint, insecure, urlpath, headers, "http")
 case "otlp-grpc":
-	 return NewOTLP(endpoint, insecure, urlpath, headers, "grpc")
+	 return NewOTLP(endpoint, insecure, urlpath, headers, protocol)

Committable suggestion was skipped due to low confidence.

Comment on lines +10 to +19
func ExporterFactory(name, url string, insecure bool, urlpath string, headers map[string]string, protocol string) (trace.SpanExporter, error) {
switch name {
case "zipkin":
return NewZipkin(url)
case "jaeger":
return NewJaegar(url)
case "otlp", "otlp-http":
return NewOTLP(url, insecure, urlpath, headers)
return NewOTLP(url, insecure, urlpath, headers, "http")
case "otlp-grpc":
return NewOTLPGrpc(url, insecure, headers)
return NewOTLP(url, insecure, urlpath, headers, "grpc")
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Utilize the protocol parameter directly.

The protocol parameter is added to the function signature but is not used in the logic. Consider using the protocol parameter directly instead of hardcoding the values.

Apply this diff to use the protocol parameter:

 case "otlp", "otlp-http":
-	 return NewOTLP(url, insecure, urlpath, headers, "http")
 case "otlp-grpc":
-	 return NewOTLP(url, insecure, urlpath, headers, "grpc")
+	 return NewOTLP(url, insecure, urlpath, headers, protocol)

Committable suggestion was skipped due to low confidence.

@vikashsprem
Copy link
Copy Markdown
Author

vikashsprem commented Aug 24, 2024

Now I have fixed the code inconsistency reviewed by coderabbitai

@vikashsprem
Copy link
Copy Markdown
Author

vikashsprem commented Aug 24, 2024

@tolgaozen , Please review and let me know if any changes are needed.

@tolgaozen tolgaozen merged commit 5624dc6 into Permify:master Aug 24, 2024
@tolgaozen
Copy link
Copy Markdown
Member

hello @vikashsprem , thank you for your contribution πŸš€

@vikashsprem vikashsprem deleted the feat/support-grpc branch August 24, 2024 18:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support exporting OTLP over gRPC

2 participants