Skip to content

Add Elastic Streams support to TCP and UDP integrations#15917

Merged
taylor-swanson merged 4 commits intoelastic:mainfrom
jamesagarside:claude/add-elastic-streams-toggle-011CUz4yJBgvexEvgNaBpFsC
Dec 2, 2025
Merged

Add Elastic Streams support to TCP and UDP integrations#15917
taylor-swanson merged 4 commits intoelastic:mainfrom
jamesagarside:claude/add-elastic-streams-toggle-011CUz4yJBgvexEvgNaBpFsC

Conversation

@jamesagarside
Copy link
Contributor

This change adds the "Use the logs data stream" toggle to both the Custom TCP Logs and Custom UDP Logs integrations, enabling Elastic Streams functionality that was previously only available in the Custom Logs (Filestream) integration.

Changes:

  • Added use_logs_stream boolean configuration variable to TCP and UDP manifests
  • Updated TCP and UDP input templates to conditionally use index: logs when enabled
  • Bumped TCP version from 2.0.1 to 2.1.0
  • Bumped UDP version from 2.2.1 to 2.3.0
  • Updated changelogs for both integrations

When enabled, this feature sends all ingested data to the "logs" data stream instead of the configured dataset. The "Write to logs streams" option must be enabled in the output settings for this to work.

Proposed commit message

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

@jamesagarside jamesagarside requested a review from a team as a code owner November 10, 2025 11:16
@cla-checker-service
Copy link

cla-checker-service bot commented Nov 10, 2025

💚 CLA has been signed

@jamesagarside jamesagarside force-pushed the claude/add-elastic-streams-toggle-011CUz4yJBgvexEvgNaBpFsC branch from d810a30 to 99c363b Compare November 10, 2025 11:20
@andrewkroh andrewkroh added Integration:udp Custom UDP Logs Integration:tcp Custom TCP Logs Team:Integration-Experience Security Integrations Integration Experience [elastic/integration-experience] labels Nov 10, 2025
@elasticmachine
Copy link

Pinging @elastic/integration-experience (Team:Integration-Experience)

@andrewkroh andrewkroh added the needs CLA User must sign the Elastic Contributor License before review. label Nov 10, 2025
@jamesagarside jamesagarside force-pushed the claude/add-elastic-streams-toggle-011CUz4yJBgvexEvgNaBpFsC branch from 65dcf44 to 9138a2b Compare November 10, 2025 22:29
@andrewkroh andrewkroh removed the needs CLA User must sign the Elastic Contributor License before review. label Nov 11, 2025
@jamesagarside jamesagarside force-pushed the claude/add-elastic-streams-toggle-011CUz4yJBgvexEvgNaBpFsC branch 4 times, most recently from d66c121 to 3b0be96 Compare November 12, 2025 18:10
jamesagarside added a commit to jamesagarside/integrations that referenced this pull request Nov 12, 2025
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
jamesagarside and others added 3 commits November 12, 2025 18:14
- Add use_logs_stream configuration option to enable sending data to the unified "logs" data stream
- Update tcp.yml.hbs and udp.yml.hbs templates to conditionally route to logs index
- Bump TCP version to 2.1.0 and UDP version to 2.3.0
- Follows the same pattern as filestream package

This feature allows users to send all ingested data to the "logs" data stream instead of custom datasets when the "Write to logs streams" option is enabled in output settings.

Note: System tests for logs-stream variants will fail due to a known elastic-package bug being fixed in elastic/elastic-package#3037. Policy tests pass, confirming the configuration is correct.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
The elastic-package linter requires changelog links to have valid PR numbers.
Removed the 'link' field from changelog entries until a PR is created.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@jamesagarside jamesagarside force-pushed the claude/add-elastic-streams-toggle-011CUz4yJBgvexEvgNaBpFsC branch from 01de3e9 to 652fe1f Compare November 12, 2025 18:14
@jamesagarside
Copy link
Contributor Author

@andrewkroh looking like we're G2G if I could get a review please.

@andrewkroh andrewkroh added the enhancement New feature or request label Nov 25, 2025
Changed version constraint from "^8.13.0 || ^9.0.0" to "^9.2.0" for both
TCP and UDP packages since Elastic Streams was introduced in 9.2.0.

Also updated the use_logs_stream description to clarify the 9.2.0 requirement.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@jamesagarside jamesagarside force-pushed the claude/add-elastic-streams-toggle-011CUz4yJBgvexEvgNaBpFsC branch from 9f3551a to 1d4db6b Compare December 1, 2025 20:03
@elasticmachine
Copy link

💚 Build Succeeded

History

@taylor-swanson taylor-swanson merged commit b2777a1 into elastic:main Dec 2, 2025
7 checks passed
@elastic-vault-github-plugin-prod

Package tcp - 2.1.0 containing this change is available at https://epr.elastic.co/package/tcp/2.1.0/

@elastic-vault-github-plugin-prod

Package udp - 2.3.0 containing this change is available at https://epr.elastic.co/package/udp/2.3.0/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Integration:tcp Custom TCP Logs Integration:udp Custom UDP Logs Team:Integration-Experience Security Integrations Integration Experience [elastic/integration-experience]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants