Skip to content

Backport PR #16456 to 8.15: Fixes the issue where LS wipes out all quotes from docker env variables.#16458

Merged
mashhurs merged 1 commit into8.15from
backport_16456_8.15
Sep 17, 2024
Merged

Backport PR #16456 to 8.15: Fixes the issue where LS wipes out all quotes from docker env variables.#16458
mashhurs merged 1 commit into8.15from
backport_16456_8.15

Conversation

@github-actions
Copy link
Copy Markdown
Contributor

@github-actions github-actions bot commented Sep 17, 2024

Backport PR #16456 to 8.15 branch, original message:

Release notes

  • Fixes the regression introduced in 8.15. The regression was Logstash wiped out single and double quotes from CONFIG_STRING. The issue happens when running Logstash on docker with CONFIG_STRING and the CONFIG_STRING contains quotes (for example hosts=>"my.es.host")

What does this PR do?

Fixes the issue where LS wipes out all quotes from docker env variables. This is an issue when running LS on docker with CONFIG_STRING, needs to keep quotes with env variable.

Why is it important/What is the impact to the user?

Unblocks the user who faced upgrade issue from 8.15.0 or 8.15.1

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files (and/or docker env variables)
  • I have added tests that prove my fix is effective or that my feature works

Author's Checklist

  • [ ]

How to test this PR locally

  • run ./gradlew clean bootstrap assemble installDefaultGems && rake artifact:docker command to build the artifact on your local. The artifact will be 9.0.0-SNAPSHOT

  • make sure docker engine is running

  • create docker-compose.yml file in your testing path and fill it with following config:

    services:
logstash:
  image: docker.elastic.co/logstash/logstash:9.0.0-SNAPSHOT
  container_name: logstash
  ports:
    - "9600:9600"
  environment:
    - MONITORING_ENABLED=false
    - XPACK_MONITORING_ENABLED=false
    - CONFIG_STRING='input {
        stdin { }
        beats { port => 5040 }
      }
      output {
        elasticsearch {
          hosts => ["https://es:9200"]
          data_stream => "true"
          data_stream_type => "logs"
          user => "removed"
          password => "removed"
        }
        stdout { }
      }'

  • run docker compose up to check if Logstash spins up without issue

Note that, you can replace CONFIG_STRING='' single quote with double or remove it. You can also place ${VAR} in the CONFIG_STRING (example ${USER} will result in setting user). This PR makes the same user experience with 8.13.0 where we started adding env2yaml/substitution ${VAR} changes.
Without this change, we get following error:

logstash  | [2024-09-16T21:01:29,703][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [A-Za-z0-9_-], [ \\t\\r\\n], \"#\", \"{\", [A-Za-z0-9_], \",\", \"]\" at line 1, column 84 (byte 84) after output { elasticsearch { hosts => [https", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:239:in `initialize'", "org/logstash/execution/AbstractPipelineExt.java:173:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:48:in `initialize'", "org/jruby/RubyClass.java:949:in `new'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:49:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:386:in `block in converge_state'"]}

You can validate this issue by changing defined docker-compose.yml image to image: docker.elastic.co/logstash/logstash:8.15.1

How to run docker acceptance tests?

  • run ./gradlew clean bootstrap assemble installDefaultGems && rake artifact:docker command to build the artifact on your local. The artifact will be 9.0.0-SNAPSHOT
  • make sure docker engine is running
  • go to qa folder and run run bundle install && bundle exec rspec docker/spec/full/container_spec.rb
# logs
Failed (
expected true
     got false
). Sleeping for 2.0
    has an Ubuntu 20.04 base image

Finished in 2 minutes 14.2 seconds (files took 0.13976 seconds to load)
8 examples, 0 failures

Related issues

Use cases

  • Users who are deploying LS using docker-compose.yml
  • User who are using config file with ${VAR} defined in it

Screenshots

Logs

➜  docker-tests docker compose up
[+] Running 1/0
 ✔ Container logstash  Recreated                                                                                                                                                    0.0s 
Attaching to logstash
logstash  | 2024/09/16 21:03:01 Setting 'config.string' from environment.
logstash  | 2024/09/16 21:03:01 Setting 'xpack.monitoring.enabled' from environment.
logstash  | Using bundled JDK: /usr/share/logstash/jdk
logstash  | Replacing `${CONFIG_STRING}` with actual value
logstash  | Placeholder value: "input { stdin { } beats { port => 5040 } } output { elasticsearch { hosts => [\"https://es:9200\"] data_stream => \"true\" data_stream_type => \"logs\" user => \"removed\" password => \"removed\" } stdout { } }"
logstash  | refined_value-1: "input { stdin { } beats { port => 5040 } } output { elasticsearch { hosts => [\"https://es:9200\"] data_stream => \"true\" data_stream_type => \"logs\" user => \"removed\" password => \"removed\" } stdout { } }"
logstash  | refined_value-2: "input { stdin { } beats { port => 5040 } } output { elasticsearch { hosts => [\"https://es:9200\"] data_stream => \"true\" data_stream_type => \"logs\" user => \"removed\" password => \"removed\" } stdout { } }"
logstash  | Replacing `${XPACK_MONITORING_ENABLED}` with actual value
logstash  | Placeholder value: "false"
logstash  | refined_value-1: "false"
logstash  | refined_value-2: "false"
logstash  | Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties
logstash  | [2024-09-16T21:03:07,387][WARN ][deprecation.logstash.settings] The setting `http.host` is a deprecated alias for `api.http.host` and will be removed in a future release of Logstash. Please use api.http.host instead
logstash  | [2024-09-16T21:03:07,393][INFO ][logstash.runner          ] Log4j configuration path used is: /usr/share/logstash/config/log4j2.properties
logstash  | [2024-09-16T21:03:07,394][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"9.0.0", "jruby.version"=>"jruby 9.4.8.0 (3.1.4) 2024-07-02 4d41e55a67 OpenJDK 64-Bit Server VM 21.0.4+7-LTS on 21.0.4+7-LTS +indy +jit [aarch64-linux]"}
logstash  | [2024-09-16T21:03:07,395][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Dlogstash.jackson.stream-read-constraints.max-string-length=200000000, -Dlogstash.jackson.stream-read-constraints.max-number-length=10000, -Dls.cgroup.cpuacct.path.override=/, -Dls.cgroup.cpu.path.override=/, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED, -Dio.netty.allocator.maxOrder=11]
logstash  | [2024-09-16T21:03:07,396][INFO ][logstash.runner          ] Jackson default value override `logstash.jackson.stream-read-constraints.max-string-length` configured to `200000000`
logstash  | [2024-09-16T21:03:07,396][INFO ][logstash.runner          ] Jackson default value override `logstash.jackson.stream-read-constraints.max-number-length` configured to `10000`
logstash  | [2024-09-16T21:03:07,399][INFO ][logstash.settings        ] Creating directory {:setting=>"path.queue", :path=>"/usr/share/logstash/data/queue"}
logstash  | [2024-09-16T21:03:07,400][INFO ][logstash.settings        ] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/usr/share/logstash/data/dead_letter_queue"}
logstash  | [2024-09-16T21:03:07,471][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
logstash  | [2024-09-16T21:03:07,479][INFO ][logstash.agent           ] No persistent UUID file found. Generating new UUID {:uuid=>"be7ba1fa-b92d-48d4-af13-eb628303d572", :path=>"/usr/share/logstash/data/uuid"}
logstash  | [2024-09-16T21:03:07,694][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
logstash  | [2024-09-16T21:03:07,802][INFO ][org.reflections.Reflections] Reflections took 54 ms to scan 1 urls, producing 139 keys and 484 values
logstash  | [2024-09-16T21:03:08,005][INFO ][logstash.javapipeline    ] Pipeline `main` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
logstash  | [2024-09-16T21:03:08,010][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["https://es:9200"]}
logstash  | [2024-09-16T21:03:08,051][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://removed:xxxxxx@es:9200/]}}
logstash  | [2024-09-16T21:03:08,079][INFO ][logstash.outputs.elasticsearch][main] Failed to perform request {:message=>"es: Name or service not known", :exception=>Manticore::ResolutionFailure, :cause=>#<Java::JavaNet::UnknownHostException: es: Name or service not known>}
logstash  | [2024-09-16T21:03:08,080][WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://removed:xxxxxx@es:9200/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :message=>"Elasticsearch Unreachable: [https://es:9200/][Manticore::ResolutionFailure] es: Name or service not known"}
logstash  | [2024-09-16T21:03:08,088][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>10, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>1250, "pipeline.sources"=>["config string"], :thread=>"#<Thread:0x55ae05b9 /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
logstash  | [2024-09-16T21:03:08,479][INFO ][logstash.javapipeline    ][main] Pipeline Java execution initialization time {"seconds"=>0.39}
logstash  | [2024-09-16T21:03:08,482][INFO ][logstash.inputs.beats    ][main] Starting input listener {:address=>"0.0.0.0:5040"}
logstash  | [2024-09-16T21:03:08,496][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
logstash  | [2024-09-16T21:03:08,509][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
logstash  | [2024-09-16T21:03:08,530][INFO ][org.logstash.beats.Server][main][cf137c1a24220ee2df85a28ed886d5d3a188863995701d0bb62ab4c1159bfc30] Starting server on port: 5040

@mashhurs @logstashmachine
Fixes the issue where LS wipes out all quotes from docker env variabl…
09dd407 
…es. (#16456)

* Fixes the issue where LS wipes out all quotes from docker env variables. This is an issue when running LS on docker with CONFIG_STRING, needs to keep quotes with env variable.

* Add a docker acceptance integration test.

(cherry picked from commit 7c64c73)
@elastic-sonarqube
Copy link
Copy Markdown

elastic-sonarqube bot commented Sep 17, 2024

Quality Gate passed Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Sep 17, 2024

💚 Build Succeeded

mashhurs
mashhurs approved these changes Sep 17, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@mashhurs mashhurs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm~

@mashhurs mashhurs merged commit 7a9f41a into 8.15 Sep 17, 2024
@mashhurs mashhurs deleted the backport_16456_8.15 branch September 17, 2024 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mashhurs mashhurs mashhurs approved these changes

Assignees

No one assigned

Labels

backport v8.15.2

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@elasticmachine @mashhurs @jsvd