Skip to content

OTLP Source unified endpoint for logs, traces and metrics#5677

Merged
kkondaka merged 27 commits intoopensearch-project:mainfrom
ps48:otel-telemetry-source
Jun 26, 2025
Merged

OTLP Source unified endpoint for logs, traces and metrics#5677
kkondaka merged 27 commits intoopensearch-project:mainfrom
ps48:otel-telemetry-source

Conversation

@ps48
Copy link
Copy Markdown
Member

@ps48 ps48 commented May 4, 2025

Description

OTLP Source unifies the three existing source OTel-logs,traces and metrics sources. This provides ease of use for a single endpoint for the three sources separated by path. It re-uses most of the existing APIs and services including the latest http-common library for GRPCserver creation.

Screenshot 2025-05-06 at 11 03 06 AM

Major decision points made in the PR:

  1. Port is kept as 21893, logically made sense and 21890/1/2 are taken up by logs, traces and metrics.
  2. The paths are separated out in the config for each type of telemetry and is made optional.
    3. The codec not configurable to the older format. My argument here is we need to move the new uses to the new OTel format. Updated to the below based on comments
  3. The codec is now configurable defaulting to otel, can be configured to opensearch format if required by the users.

Note: All the above decisions can be changed if, needed.

Adding some sample configuration on usage from README.md here:

otel-telemetry-pipeline:
  source:
    otlp:
      ssl: false
  route:
    - logs: 'getEventType() == "LOG"'
    - traces: 'getEventType() == "TRACE"'
    - metrics: 'getEventType() == "METRIC"'
  sink:
    - pipeline:
        name: "logs-pipeline"
        routes:
          - "logs"
    - pipeline:
        name: "traces-pipeline"
        routes:
          - "traces"
    - pipeline:
        name: "metrics-pipeline"
        routes:
          - "metrics"

Issues Resolved

#5596

Check List

  • New functionality includes testing.
  • New functionality has a documentation issue. Please link to it in this PR.
    • New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Copy Markdown
Collaborator

@KarstenSchnitter KarstenSchnitter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, thanks for providing this draft. I only added some comments based on a very brief check on the code.


// If the path is null for any of the sources, we will not transform the path
// and use the default path
if (oTelTraceSourcePath != null && oTelMetricsSourcePath != null && oTelLogsSourcePath != null) {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this check not performed separately on each source path?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to check the feasibility of this other approach. Not sure how one service can be of GRPC and other on custom HTTP endpoint.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@KarstenSchnitter This actually made more sense, once I looked into this further. I have changed my approach to accommodate this 😄

@ps48 ps48 changed the title OTLP Source OTLP Source unified endpoint for logs, traces and metrics May 5, 2025
@ps48
Copy link
Copy Markdown
Member Author

ps48 commented May 6, 2025

not sure why build is failing here:

> Task :data-prepper-core:integrationTest

PipelinesWithAcksIT > one_pipeline_three_sinks_multiple_records() FAILED
    java.lang.NullPointerException at PipelinesWithAcksIT.java:217

Below is from the local run:

❯ ./gradlew :data-prepper-core:integrationTest

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to https://docs.gradle.org/8.8/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.

BUILD SUCCESSFUL in 2m 57s
62 actionable tasks: 1 executed, 61 up-to-date

@ps48 ps48 marked this pull request as ready for review May 6, 2025 18:18
Copy link
Copy Markdown
Collaborator

@KarstenSchnitter KarstenSchnitter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking up my suggestion. I would still like to see support for the old "opensearch" format very much.

LOG.warn("Creating http source without SSL/TLS. This is not secure.");
LOG.warn("In order to set up TLS for the http source, go here: https://github.com/opensearch-project/data-prepper/tree/main/data-prepper-plugins/http-source#ssl");
LOG.warn(
"In order to set up TLS for the http source, go here: https://github.com/opensearch-project/data-prepper/tree/main/data-prepper-plugins/http-source#ssl");
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error message can be misleading. We are in the otel-source not the http source, right?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, @Davidding4718 @dlvenable Let me know your thoughts on this as the original authors. I can do a minor PR for this.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created the PR here: #5689

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. Might cause merge conflicts though. Be careful, not to take it out again.

import java.util.function.Function;


public class CreateServer {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not so sure about the class name. To me this class looks like a factory of some kind and I would name it as such. But I am also fine with the name it got right now.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Karsten,
The CreateServer abstaction was added in last week by this PR #5653. I can rename it ServerFactory or something similar, but I feel this can be out of scope for this PR.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I have seen when the change was introduced. I would appreciate a renaming at some point in time, but I am fine with not doing it in this PR.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree with Karsten, need better name

username: my-user
password: my_s3cr3t
```

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add an example with mTLS?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@KarstenSchnitter can you please explain what's needed in the example. I haven't dived deep into the Data prepper and OTel Source auth yet. Is this a feature request or just an example?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's say you have a setup were an OTel Collector is used as a local gateway. It is configured to forward data to Data Prepper. To establish trust and proper authentication/authorisation mTLS is used. Would be nice to have this setup at least the Data Prepper part as an example.

@ps48 ps48 mentioned this pull request May 7, 2025
7 tasks
@ps48 ps48 force-pushed the otel-telemetry-source branch from c540ae3 to c0c721d Compare May 8, 2025 19:25
@ps48
Copy link
Copy Markdown
Member Author

ps48 commented May 9, 2025

Thanks for taking up my suggestion. I would still like to see support for the old "opensearch" format very much.

Added support for Old formats here: 37b71ef

Copy link
Copy Markdown
Collaborator

@KarstenSchnitter KarstenSchnitter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR and addressing my comments. Looks good to me now.

import java.util.function.Function;


public class CreateServer {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I have seen when the change was introduced. I would appreciate a renaming at some point in time, but I am fine with not doing it in this PR.

LOG.warn("Creating http source without SSL/TLS. This is not secure.");
LOG.warn("In order to set up TLS for the http source, go here: https://github.com/opensearch-project/data-prepper/tree/main/data-prepper-plugins/http-source#ssl");
LOG.warn(
"In order to set up TLS for the http source, go here: https://github.com/opensearch-project/data-prepper/tree/main/data-prepper-plugins/http-source#ssl");
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. Might cause merge conflicts though. Be careful, not to take it out again.

username: my-user
password: my_s3cr3t
```

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's say you have a setup were an OTel Collector is used as a local gateway. It is configured to forward data to Data Prepper. To establish trust and proper authentication/authorisation mTLS is used. Would be nice to have this setup at least the Data Prepper part as an example.

@ps48
Copy link
Copy Markdown
Member Author

ps48 commented May 30, 2025

@KarstenSchnitter, will need your approval here again, after resolving the conflicts from main. @dlvenable & @kkondaka will need your help for the second review and approval.
Thanks!

import java.util.function.Function;


public class CreateServer {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree with Karsten, need better name


import org.opensearch.dataprepper.plugins.server.ServerConfiguration;

public class ConvertConfiguration {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This class name and method name is also little confusing. Do we really need a class for this?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The usage of this class was introduced in this PR: #5653. all other otel source adhere to this. I'd consider changing this as out of scope for this PR as we need to change this for all the Otel sources

otlpSourceConfig.getTracesOutputFormat() == OTelOutputFormat.OPENSEARCH ? new OTelProtoOpensearchCodec.OTelProtoDecoder() : new OTelProtoStandardCodec.OTelProtoDecoder(),
buffer, pluginMetrics);

ServerConfiguration serverConfiguration = ConvertConfiguration.convertConfiguration(otlpSourceConfig);
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it should just be a private method in this class instead of a separate class.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above. This was introduced in all Otel source with this PR #5653. We can refactor all of them later

static final String METRICS_PATH = "metrics_path";
static final String TRACES_PATH = "traces_path";
static final String SSL = "ssl";
static final String LOGS_OUTPUT_FORMAT = "logs_output_format";
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we just have one output format?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a product choice. Form what I thought from Karsten's comment is; some customers might want to use Traces with older opensearch format and logs with OpenSearch format. Hence, kept them separate.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have an output_format to support simple use cases? Then we could also offer the other three for more complicated ones.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, updated. Thanks for the input @kkondaka @dlvenable

@JsonProperty(PORT)
private int port = DEFAULT_PORT;

@JsonProperty(LOGS_PATH)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here. Should just take ONE prefix path and append FIXED logs, traces and metrics path to it?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my understanding these should be kept separate, cause even in the OTLP HTTP exporter they support separate paths for each type of signal. https://github.com/open-telemetry/opentelemetry-collector/tree/v0.87.0/exporter/otlphttpexporter

Copy link
Copy Markdown
Collaborator

@KarstenSchnitter KarstenSchnitter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some minor remarks. But it needs additional review fromother maintainers.

Comment on lines +36 to +37
testImplementation 'org.junit.jupiter:junit-jupiter-api:5.8.1'
testImplementation 'org.mockito:mockito-core:4.0.0'
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the versions be inherited from the parent?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, I removed them 707889f

ps48 added 5 commits June 19, 2025 11:41
Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
ps48 added 6 commits June 19, 2025 11:42
Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
@ps48 ps48 force-pushed the otel-telemetry-source branch from ba0057f to 1b75321 Compare June 19, 2025 18:42
ps48 added 2 commits June 19, 2025 11:44
Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
static final String METRICS_PATH = "metrics_path";
static final String TRACES_PATH = "traces_path";
static final String SSL = "ssl";
static final String LOGS_OUTPUT_FORMAT = "logs_output_format";
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have an output_format to support simple use cases? Then we could also offer the other three for more complicated ones.

static final String UNAUTHENTICATED_HEALTH_CHECK = "unauthenticated_health_check";

@JsonProperty(REQUEST_TIMEOUT)
private int requestTimeoutInMillis = DEFAULT_REQUEST_TIMEOUT_MS;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have this take a Data Prepper Duration instead. This is our new way to handle timeouts.

Here is an example:

@JsonProperty("flush_interval")
@DurationMin(seconds = 60)
@DurationMax(seconds = 3600)
private Duration flushInterval = Duration.ofSeconds(DEFAULT_FLUSH_INTERVAL);

ps48 added 2 commits June 25, 2025 07:25
Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
Copy link
Copy Markdown
Member

@dlvenable dlvenable left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you fixed a few comments have resolved those. There are still some about using Duration and also about having a single output_format.

ps48 added 2 commits June 25, 2025 10:10
Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
@ps48
Copy link
Copy Markdown
Member Author

ps48 commented Jun 25, 2025

@dlvenable Have updated the PR with output_format and Duration changes. Would be great if you can take a look now 😄
Thanks!


public OTelOutputFormat getLogsOutputFormat() {
return logsOutputFormat;
return logsOutputFormat != OTelOutputFormat.OTEL ? logsOutputFormat : outputFormat;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should be unit tested.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Thanks!

Let's improve this code some though:

return logsOutputFormat != null ? logsOutputFormat : outputFormat;

And change the default value of logsOutputFormat to null.

Do the same for the others.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree this should be the right way, updated it here: c119aa8


public OTelOutputFormat getLogsOutputFormat() {
return logsOutputFormat;
return logsOutputFormat != OTelOutputFormat.OTEL ? logsOutputFormat : outputFormat;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Thanks!

Let's improve this code some though:

return logsOutputFormat != null ? logsOutputFormat : outputFormat;

And change the default value of logsOutputFormat to null.

Do the same for the others.

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
@kkondaka kkondaka merged commit 5ad289d into opensearch-project:main Jun 26, 2025
69 of 74 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jun 26, 2025
* init single endpoint

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* add reflection service

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update getmetadata expression to support eventType

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* format files

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update config and add health check

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* add http support and update config options

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* reset Otel trace source

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* add tests for healthcheck, auth, requests and config

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* add tests for grpc

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update retry tests

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update plugin name from otel-telemetry-source to otlp

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update readme with usage details

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* reset Otel trace source changes

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* revert GetMetadataExpressionFunction and OTelTraceSource changes

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* remove example

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update source to use http-common server

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* added back retry tests, use http-common health check

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update readme with authentication details

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* fix checkstyle issues

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* add support for OpenSearch formats & update readme

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* remove dupe in settings

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* remove junit and mockito

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update config fields and move certs

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update tests with new config options

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* use data prepper duration and add generic output_format

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update timeouts to duration

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update the output format defaults to null

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

---------

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
(cherry picked from commit 5ad289d)
kkondaka pushed a commit that referenced this pull request Jun 26, 2025
)

* init single endpoint



* add reflection service



* update getmetadata expression to support eventType



* format files



* update config and add health check



* add http support and update config options



* reset Otel trace source



* add tests for healthcheck, auth, requests and config



* add tests for grpc



* update retry tests



* update plugin name from otel-telemetry-source to otlp



* update readme with usage details



* reset Otel trace source changes



* revert GetMetadataExpressionFunction and OTelTraceSource changes



* remove example



* update source to use http-common server



* added back retry tests, use http-common health check



* update readme with authentication details



* fix checkstyle issues



* add support for OpenSearch formats & update readme



* remove dupe in settings



* remove junit and mockito



* update config fields and move certs



* update tests with new config options



* use data prepper duration and add generic output_format



* update timeouts to duration



* update the output format defaults to null



---------


(cherry picked from commit 5ad289d)

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
Co-authored-by: Shenoy Pratik <sgguruda@amazon.com>
JonahCalvo pushed a commit to JonahCalvo/os-data-prepper that referenced this pull request Jul 17, 2025
…-project#5677)

* init single endpoint

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* add reflection service

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update getmetadata expression to support eventType

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* format files

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update config and add health check

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* add http support and update config options

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* reset Otel trace source

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* add tests for healthcheck, auth, requests and config

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* add tests for grpc

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update retry tests

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update plugin name from otel-telemetry-source to otlp

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update readme with usage details

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* reset Otel trace source changes

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* revert GetMetadataExpressionFunction and OTelTraceSource changes

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* remove example

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update source to use http-common server

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* added back retry tests, use http-common health check

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update readme with authentication details

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* fix checkstyle issues

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* add support for OpenSearch formats & update readme

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* remove dupe in settings

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* remove junit and mockito

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update config fields and move certs

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update tests with new config options

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* use data prepper duration and add generic output_format

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update timeouts to duration

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update the output format defaults to null

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

---------

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
Signed-off-by: Jonah Calvo <caljonah@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants