Skip to content

Backport PR #17125 to 8.x: entrypoint: avoid polluting stdout#17136

Merged
donoghuc merged 1 commit into8.xfrom
backport_17125_8.x
Feb 21, 2025
Merged

Backport PR #17125 to 8.x: entrypoint: avoid polluting stdout#17136
donoghuc merged 1 commit into8.xfrom
backport_17125_8.x

Conversation

@github-actions
Copy link
Copy Markdown
Contributor

Backport PR #17125 to 8.x branch, original message:


Release notes

[rn:skip]

What does this PR do?

routes output from setup-related bits in the entrypoint scripts to stderr, so that stdout can include only the output of the actual program.

Why is it important/What is the impact to the user?

This change makes it possible to use the stdout of entrypoints like bin/logstash-plugin without the setup-related pollution:

Using system java: /Users/rye/.jenv/shims/java

For example, with #17124 addition of --no-expand, and given an allowlist containing the names of plugins to keep,

comm -1 -2 <(cat allowlist | sort) <(bin/logstash-plugin list --no-expand | sort) | xargs bin/logstash-plugin remove 

The above is not possible if the stdout of bin/logstash-plugin invocation is littered with info about the JVM.

Checklist

  • My code follows the style guidelines of this project
  • [ ] I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] I have made corresponding change to the default configuration files (and/or docker env variables)
  • [ ] I have added tests that prove my fix is effective or that my feature works

Author's Checklist

  • manual windows validation (are acceptance tests enough?)

How to test this PR locally

observe that the side-channel bits are on STDERR and can be redirected away

bin/logstash-plugin list --verbose 2>/dev/null

Logs

With patch, the info about which java is being used is on STDERR:

╭─{ rye@perhaps:~/src/elastic/logstash@main (pluginmanager-list-simple ✘) }
╰─● bin/logstash-plugin list | awk '{print "[PRGOUT]:" $0}' 
Using system java: /Users/rye/.jenv/shims/java
[PRGOUT]:logstash-codec-avro
[PRGOUT]:logstash-codec-cef
[PRGOUT]:logstash-codec-collectd
[PRGOUT]:logstash-codec-dots
[PRGOUT]:logstash-codec-edn
[PRGOUT]:logstash-codec-edn_lines
[PRGOUT]:logstash-codec-es_bulk
[PRGOUT]:logstash-codec-fluent
[PRGOUT]:logstash-codec-graphite
[PRGOUT]:logstash-codec-json
[PRGOUT]:logstash-codec-json_lines
[PRGOUT]:logstash-codec-line
[PRGOUT]:logstash-codec-msgpack
[PRGOUT]:logstash-codec-multiline
[PRGOUT]:logstash-codec-netflow
[PRGOUT]:logstash-codec-plain
[PRGOUT]:logstash-codec-rubydebug
[PRGOUT]:logstash-filter-aggregate
[PRGOUT]:logstash-filter-anonymize
[PRGOUT]:logstash-filter-cidr
[PRGOUT]:logstash-filter-clone
[PRGOUT]:logstash-filter-csv
[PRGOUT]:logstash-filter-date
[PRGOUT]:logstash-filter-de_dot
[PRGOUT]:logstash-filter-dissect
[PRGOUT]:logstash-filter-dns
[PRGOUT]:logstash-filter-drop
[PRGOUT]:logstash-filter-elastic_integration
[PRGOUT]:logstash-filter-elasticsearch
[PRGOUT]:logstash-filter-fingerprint
[PRGOUT]:logstash-filter-geoip
[PRGOUT]:logstash-filter-grok
[PRGOUT]:logstash-filter-http
[PRGOUT]:logstash-filter-json
[PRGOUT]:logstash-filter-kv
[PRGOUT]:logstash-filter-memcached
[PRGOUT]:logstash-filter-metrics
[PRGOUT]:logstash-filter-mutate
[PRGOUT]:logstash-filter-prune
[PRGOUT]:logstash-filter-ruby
[PRGOUT]:logstash-filter-sleep
[PRGOUT]:logstash-filter-split
[PRGOUT]:logstash-filter-syslog_pri
[PRGOUT]:logstash-filter-throttle
[PRGOUT]:logstash-filter-translate
[PRGOUT]:logstash-filter-truncate
[PRGOUT]:logstash-filter-urldecode
[PRGOUT]:logstash-filter-useragent
[PRGOUT]:logstash-filter-uuid
[PRGOUT]:logstash-filter-xml
[PRGOUT]:logstash-input-azure_event_hubs
[PRGOUT]:logstash-input-beats
[PRGOUT]: └── logstash-input-elastic_agent (alias)
[PRGOUT]:logstash-input-couchdb_changes
[PRGOUT]:logstash-input-dead_letter_queue
[PRGOUT]:logstash-input-elastic_serverless_forwarder
[PRGOUT]:logstash-input-elasticsearch
[PRGOUT]:logstash-input-exec
[PRGOUT]:logstash-input-file
[PRGOUT]:logstash-input-ganglia
[PRGOUT]:logstash-input-gelf
[PRGOUT]:logstash-input-generator
[PRGOUT]:logstash-input-graphite
[PRGOUT]:logstash-input-heartbeat
[PRGOUT]:logstash-input-http
[PRGOUT]:logstash-input-http_poller
[PRGOUT]:logstash-input-jms
[PRGOUT]:logstash-input-pipe
[PRGOUT]:logstash-input-redis
[PRGOUT]:logstash-input-stdin
[PRGOUT]:logstash-input-syslog
[PRGOUT]:logstash-input-tcp
[PRGOUT]:logstash-input-twitter
[PRGOUT]:logstash-input-udp
[PRGOUT]:logstash-input-unix
[PRGOUT]:logstash-integration-aws
[PRGOUT]: ├── logstash-codec-cloudfront
[PRGOUT]: ├── logstash-codec-cloudtrail
[PRGOUT]: ├── logstash-input-cloudwatch
[PRGOUT]: ├── logstash-input-s3
[PRGOUT]: ├── logstash-input-sqs
[PRGOUT]: ├── logstash-output-cloudwatch
[PRGOUT]: ├── logstash-output-s3
[PRGOUT]: ├── logstash-output-sns
[PRGOUT]: └── logstash-output-sqs
[PRGOUT]:logstash-integration-jdbc
[PRGOUT]: ├── logstash-input-jdbc
[PRGOUT]: ├── logstash-filter-jdbc_streaming
[PRGOUT]: └── logstash-filter-jdbc_static
[PRGOUT]:logstash-integration-kafka
[PRGOUT]: ├── logstash-input-kafka
[PRGOUT]: └── logstash-output-kafka
[PRGOUT]:logstash-integration-logstash
[PRGOUT]: ├── logstash-input-logstash
[PRGOUT]: └── logstash-output-logstash
[PRGOUT]:logstash-integration-rabbitmq
[PRGOUT]: ├── logstash-input-rabbitmq
[PRGOUT]: └── logstash-output-rabbitmq
[PRGOUT]:logstash-integration-snmp
[PRGOUT]: ├── logstash-input-snmp
[PRGOUT]: └── logstash-input-snmptrap
[PRGOUT]:logstash-output-csv
[PRGOUT]:logstash-output-elasticsearch
[PRGOUT]:logstash-output-email
[PRGOUT]:logstash-output-file
[PRGOUT]:logstash-output-graphite
[PRGOUT]:logstash-output-http
[PRGOUT]:logstash-output-lumberjack
[PRGOUT]:logstash-output-nagios
[PRGOUT]:logstash-output-null
[PRGOUT]:logstash-output-pipe
[PRGOUT]:logstash-output-redis
[PRGOUT]:logstash-output-stdout
[PRGOUT]:logstash-output-tcp
[PRGOUT]:logstash-output-udp
[PRGOUT]:logstash-output-webhdfs
[PRGOUT]:logstash-patterns-core
[success]                                                                                                                                                                                                                                                                                                                                                                                                                        

Without patch, the info about which java is being used is on STDOUT:

╭─{ rye@perhaps:~/src/elastic/logstash@main (pluginmanager-list-simple ✘) }
╰─● bin/logstash-plugin list | awk '{print "[PRGOUT]:" $0}'
[PRGOUT]:Using system java: /Users/rye/.jenv/shims/java
[PRGOUT]:logstash-codec-avro
[PRGOUT]:logstash-codec-cef
[PRGOUT]:logstash-codec-collectd
[PRGOUT]:logstash-codec-dots
[PRGOUT]:logstash-codec-edn
[PRGOUT]:logstash-codec-edn_lines
[PRGOUT]:logstash-codec-es_bulk
[PRGOUT]:logstash-codec-fluent
[PRGOUT]:logstash-codec-graphite
[PRGOUT]:logstash-codec-json
[PRGOUT]:logstash-codec-json_lines
[PRGOUT]:logstash-codec-line
[PRGOUT]:logstash-codec-msgpack
[PRGOUT]:logstash-codec-multiline
[PRGOUT]:logstash-codec-netflow
[PRGOUT]:logstash-codec-plain
[PRGOUT]:logstash-codec-rubydebug
[PRGOUT]:logstash-filter-aggregate
[PRGOUT]:logstash-filter-anonymize
[PRGOUT]:logstash-filter-cidr
[PRGOUT]:logstash-filter-clone
[PRGOUT]:logstash-filter-csv
[PRGOUT]:logstash-filter-date
[PRGOUT]:logstash-filter-de_dot
[PRGOUT]:logstash-filter-dissect
[PRGOUT]:logstash-filter-dns
[PRGOUT]:logstash-filter-drop
[PRGOUT]:logstash-filter-elastic_integration
[PRGOUT]:logstash-filter-elasticsearch
[PRGOUT]:logstash-filter-fingerprint
[PRGOUT]:logstash-filter-geoip
[PRGOUT]:logstash-filter-grok
[PRGOUT]:logstash-filter-http
[PRGOUT]:logstash-filter-json
[PRGOUT]:logstash-filter-kv
[PRGOUT]:logstash-filter-memcached
[PRGOUT]:logstash-filter-metrics
[PRGOUT]:logstash-filter-mutate
[PRGOUT]:logstash-filter-prune
[PRGOUT]:logstash-filter-ruby
[PRGOUT]:logstash-filter-sleep
[PRGOUT]:logstash-filter-split
[PRGOUT]:logstash-filter-syslog_pri
[PRGOUT]:logstash-filter-throttle
[PRGOUT]:logstash-filter-translate
[PRGOUT]:logstash-filter-truncate
[PRGOUT]:logstash-filter-urldecode
[PRGOUT]:logstash-filter-useragent
[PRGOUT]:logstash-filter-uuid
[PRGOUT]:logstash-filter-xml
[PRGOUT]:logstash-input-azure_event_hubs
[PRGOUT]:logstash-input-beats
[PRGOUT]: └── logstash-input-elastic_agent (alias)
[PRGOUT]:logstash-input-couchdb_changes
[PRGOUT]:logstash-input-dead_letter_queue
[PRGOUT]:logstash-input-elastic_serverless_forwarder
[PRGOUT]:logstash-input-elasticsearch
[PRGOUT]:logstash-input-exec
[PRGOUT]:logstash-input-file
[PRGOUT]:logstash-input-ganglia
[PRGOUT]:logstash-input-gelf
[PRGOUT]:logstash-input-generator
[PRGOUT]:logstash-input-graphite
[PRGOUT]:logstash-input-heartbeat
[PRGOUT]:logstash-input-http
[PRGOUT]:logstash-input-http_poller
[PRGOUT]:logstash-input-jms
[PRGOUT]:logstash-input-pipe
[PRGOUT]:logstash-input-redis
[PRGOUT]:logstash-input-stdin
[PRGOUT]:logstash-input-syslog
[PRGOUT]:logstash-input-tcp
[PRGOUT]:logstash-input-twitter
[PRGOUT]:logstash-input-udp
[PRGOUT]:logstash-input-unix
[PRGOUT]:logstash-integration-aws
[PRGOUT]: ├── logstash-codec-cloudfront
[PRGOUT]: ├── logstash-codec-cloudtrail
[PRGOUT]: ├── logstash-input-cloudwatch
[PRGOUT]: ├── logstash-input-s3
[PRGOUT]: ├── logstash-input-sqs
[PRGOUT]: ├── logstash-output-cloudwatch
[PRGOUT]: ├── logstash-output-s3
[PRGOUT]: ├── logstash-output-sns
[PRGOUT]: └── logstash-output-sqs
[PRGOUT]:logstash-integration-jdbc
[PRGOUT]: ├── logstash-input-jdbc
[PRGOUT]: ├── logstash-filter-jdbc_streaming
[PRGOUT]: └── logstash-filter-jdbc_static
[PRGOUT]:logstash-integration-kafka
[PRGOUT]: ├── logstash-input-kafka
[PRGOUT]: └── logstash-output-kafka
[PRGOUT]:logstash-integration-logstash
[PRGOUT]: ├── logstash-input-logstash
[PRGOUT]: └── logstash-output-logstash
[PRGOUT]:logstash-integration-rabbitmq
[PRGOUT]: ├── logstash-input-rabbitmq
[PRGOUT]: └── logstash-output-rabbitmq
[PRGOUT]:logstash-integration-snmp
[PRGOUT]: ├── logstash-input-snmp
[PRGOUT]: └── logstash-input-snmptrap
[PRGOUT]:logstash-output-csv
[PRGOUT]:logstash-output-elasticsearch
[PRGOUT]:logstash-output-email
[PRGOUT]:logstash-output-file
[PRGOUT]:logstash-output-graphite
[PRGOUT]:logstash-output-http
[PRGOUT]:logstash-output-lumberjack
[PRGOUT]:logstash-output-nagios
[PRGOUT]:logstash-output-null
[PRGOUT]:logstash-output-pipe
[PRGOUT]:logstash-output-redis
[PRGOUT]:logstash-output-stdout
[PRGOUT]:logstash-output-tcp
[PRGOUT]:logstash-output-udp
[PRGOUT]:logstash-output-webhdfs
[PRGOUT]:logstash-patterns-core
[success]

routes output from setup-related functions to stderr, so that stdout can
include only the output of the actual program.

(cherry picked from commit 91258c3)
@elastic-sonarqube
Copy link
Copy Markdown

Quality Gate passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

@elasticmachine
Copy link
Copy Markdown

💛 Build succeeded, but was flaky

Failed CI Steps

Copy link
Copy Markdown
Contributor

@mashhurs mashhurs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM~

@donoghuc donoghuc merged commit 74aeea6 into 8.x Feb 21, 2025
@donoghuc donoghuc deleted the backport_17125_8.x branch February 21, 2025 20:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants