Pipeline Plugin directives historically could be either quoted or bareword, meaning that the following were equivalent:
filter {
kv {
"source" => "message"
}
}
filter {
kv {
source => "message"
}
}
But under the Java Execution Engine, quoted directives cause an Java::JavaLang::IllegalStateException during startup that crashes the process.
╭─{ yaauie@castrovel:~/src/elastic/logstash (✘ 7.x) }
╰─● bin/logstash -e 'input { stdin {} } filter { kv { "source" => "message" } } output { stdout {} }'
[snip]
[2019-10-15T19:58:25,603][ERROR][logstash.filters.kv ] Unknown setting '"source"' for kv
[2019-10-15T19:58:25,618][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: (ConfigurationError) Something is wrong with your configuration.", :backtrace=>["org.logstash.config.ir.CompiledPipeline.<init>(CompiledPipeline.java:100)", "org.logstash.execution.JavaBasePipelineExt.initialize(JavaBasePipelineExt.java:60)", "org.logstash.execution.JavaBasePipelineExt$INVOKER$i$1$0$initialize.call(JavaBasePipelineExt$INVOKER$i$1$0$initialize.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:837)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1156)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuperSplatArgs(IRRuntimeHelpers.java:1143)", "org.jruby.ir.targets.InstanceSuperInvokeSite.invoke(InstanceSuperInvokeSite.java:39)", "Users.yaauie.src.elastic.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$initialize$0(/Users/yaauie/src/elastic/logstash/logstash-core/lib/logstash/java_pipeline.rb:26)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:86)", "org.jruby.RubyClass.newInstance(RubyClass.java:915)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "Users.yaauie.src.elastic.logstash.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0(/Users/yaauie/src/elastic/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:36)", "Users.yaauie.src.elastic.logstash.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0$__VARARGS__(/Users/yaauie/src/elastic/logstash/logstash-core/lib/logstash/pipeline_action/create.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "Users.yaauie.src.elastic.logstash.logstash_minus_core.lib.logstash.agent.RUBY$block$converge_state$2(/Users/yaauie/src/elastic/logstash/logstash-core/lib/logstash/agent.rb:326)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:136)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:77)", "org.jruby.runtime.Block.call(Block.java:129)", "org.jruby.RubyProc.call(RubyProc.java:295)", "org.jruby.RubyProc.call(RubyProc.java:274)", "org.jruby.RubyProc.call(RubyProc.java:270)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)", "java.lang.Thread.run(Thread.java:748)"]}
warning: thread "Converge PipelineAction::Create<main>" terminated with exception (report_on_exception is true):
LogStash::Error: Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create<main>`
create at org/logstash/execution/ConvergeResultExt.java:109
add at org/logstash/execution/ConvergeResultExt.java:37
converge_state at /Users/yaauie/src/elastic/logstash/logstash-core/lib/logstash/agent.rb:339
[2019-10-15T19:58:25,628][ERROR][logstash.agent ] An exception happened when converging configuration {:exception=>LogStash::Error, :message=>"Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create<main>`", :backtrace=>["org/logstash/execution/ConvergeResultExt.java:109:in `create'", "org/logstash/execution/ConvergeResultExt.java:37:in `add'", "/Users/yaauie/src/elastic/logstash/logstash-core/lib/logstash/agent.rb:339:in `block in converge_state'"]}
[2019-10-15T19:58:25,657][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<LogStash::Error: Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create<main>`>, :backtrace=>["org/logstash/execution/ConvergeResultExt.java:109:in `create'", "org/logstash/execution/ConvergeResultExt.java:37:in `add'", "/Users/yaauie/src/elastic/logstash/logstash-core/lib/logstash/agent.rb:339:in `block in converge_state'"]}
[2019-10-15T19:58:25,724][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
bin/logstash -e 68.04s user 2.17s system 405% cpu 17.312 total
[error: 1 (17.000s)]
Disabling Java Execution and falling back to the Ruby Execution Engine, we encounter no such failure:
╭─{ yaauie@castrovel:~/src/elastic/logstash (✘ 7.x) }
╰─● bin/logstash --java-execution=false -e 'input { stdin {} } filter { kv { "source" => "message" } } output { stdout {} }'
[snip]
[2019-10-15T19:57:48,814][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.sources"=>["config string"]}
The stdin plugin is now waiting for input:
[2019-10-15T19:57:48,969][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0xd4c7e3d sleep>"}
[2019-10-15T19:57:49,005][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2019-10-15T19:57:49,327][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
^C[2019-10-15T19:57:58,895][WARN ][logstash.runner ] SIGINT received. Shutting down.
[2019-10-15T19:57:59,042][INFO ][logstash.pipeline ][main] Pipeline has terminated {:pipeline_id=>"main", :thread=>"#<Thread:0xd4c7e3d run>"}
[2019-10-15T19:57:59,103][INFO ][logstash.runner ] Logstash shut down.
bin/logstash --java-execution=false -e 78.73s user 2.75s system 266% cpu 30.519 total
[success (31.000s)]
Pipeline Plugin directives historically could be either quoted or bareword, meaning that the following were equivalent:
But under the Java Execution Engine, quoted directives cause an
Java::JavaLang::IllegalStateExceptionduring startup that crashes the process.Disabling Java Execution and falling back to the Ruby Execution Engine, we encounter no such failure: