Skip to content

Filebeat sometimes does not shut down when only one input is configured and it is non-existent #45034

@ycombinator

Description

@ycombinator

Version

main / 9.1.0-SNAPSHOT

Operating System

Ubuntu Linux 24.10 ARM64 (but I suspect this bug is OS-agnostic)

Steps to Reproduce:

  1. Create a Filebeat configuration with exactly one input and make sure that no such input exists in Filebeat.
cat <<EOF > filebeat-test.yml
filebeat.inputs:
- type: nonexistent
  id: id-none
  enabled: true

output.console:
  enabled: true
EOF
  1. Run Filebeat with this configuration. You may have to repeat this step a few times to hit the bug.
./filebeat -c ./filebeat-test.yml -e

Observed behavior

Filebeat sometimes exits with an error indicating that the input type is invalid.

...
{"log.level":"info","@timestamp":"2025-06-25T10:33:20.485-0700","log.logger":"crawler","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.(*crawler).Start","file.name":"beater/crawler.go","file.line":72},"message":"Loading Inputs: 1","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-06-25T10:33:20.485-0700","log.logger":"crawler","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.(*crawler).Stop","file.name":"beater/crawler.go","file.line":156},"message":"Stopping Crawler","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-06-25T10:33:20.485-0700","log.logger":"crawler","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.(*crawler).Stop","file.name":"beater/crawler.go","file.line":166},"message":"Stopping 0 inputs","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-06-25T10:33:20.485-0700","log.logger":"crawler","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.(*crawler).Stop","file.name":"beater/crawler.go","file.line":186},"message":"Crawler stopped","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-06-25T10:33:20.485-0700","log.logger":"registrar","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/registrar.(*Registrar).Stop","file.name":"registrar/registrar.go","file.line":126},"message":"Stopping Registrar","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-06-25T10:33:20.485-0700","log.logger":"registrar","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/registrar.(*Registrar).Run","file.name":"registrar/registrar.go","file.line":162},"message":"Ending Registrar","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-06-25T10:33:20.485-0700","log.logger":"registrar","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/registrar.(*Registrar).Stop","file.name":"registrar/registrar.go","file.line":131},"message":"Registrar stopped","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-06-25T10:33:20.488-0700","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/monitoring/report/log.(*reporter).logTotals","file.name":"log/log.go","file.line":200},"message":"Total metrics","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"cpu":{"id":".lxc","stats":{"periods":0,"throttled":{"ns":0,"periods":0}}},"memory":{"id":".lxc","mem":{"usage":{"bytes":191672320}}}},"cpu":{"system":{"ticks":30,"time":{"ms":30}},"total":{"ticks":90,"time":{"ms":90},"value":90},"user":{"ticks":60,"time":{"ms":60}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":8},"info":{"ephemeral_id":"7b898a9b-112e-43f4-b0ed-4c1081fe067b","name":"filebeat","uptime":{"ms":21},"version":"9.1.0"},"memstats":{"gc_next":45950370,"memory_alloc":22462864,"memory_sys":49894664,"memory_total":36586408,"rss":139886592},"runtime":{"goroutines":10}},"filebeat":{"events":{"active":0,"added":0,"done":0},"harvester":{"closed":0,"open_files":0,"running":0,"skipped":0,"started":0},"input":{"log":{"files":{"renamed":0,"truncated":0}}}},"libbeat":{"config":{"module":{"running":0,"starts":0,"stops":0},"reloads":0,"scans":0},"output":{"batches":{"split":0},"events":{"acked":0,"active":0,"batches":0,"dead_letter":0,"dropped":0,"duplicates":0,"failed":0,"toomany":0,"total":0},"read":{"bytes":0,"errors":0},"type":"console","write":{"bytes":0,"errors":0,"latency":{"histogram":{"count":0,"max":0,"mean":0,"median":0,"min":0,"p75":0,"p95":0,"p99":0,"p999":0,"stddev":0}}}},"pipeline":{"clients":0,"events":{"active":0,"dropped":0,"failed":0,"filtered":0,"published":0,"retry":0,"total":0},"queue":{"acked":0,"added":{"bytes":0,"events":0},"consumed":{"bytes":0,"events":0},"filled":{"bytes":0,"events":0,"pct":0},"max_bytes":0,"max_events":3200,"removed":{"bytes":0,"events":0}}}},"registrar":{"states":{"cleanup":0,"current":0,"update":0},"writes":{"fail":0,"success":0,"total":0}},"system":{"cpu":{"cores":12},"load":{"1":0,"15":0.06,"5":0.03,"norm":{"1":0,"15":0.005,"5":0.0025}}}},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2025-06-25T10:33:20.488-0700","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/monitoring/report/log.(*reporter).logTotals","file.name":"log/log.go","file.line":201},"message":"Uptime: 16.738388ms","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-06-25T10:33:20.488-0700","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/monitoring/report/log.(*reporter).snapshotLoop","file.name":"log/log.go","file.line":168},"message":"Stopping metrics logging.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-06-25T10:33:20.488-0700","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).launch","file.name":"instance/beat.go","file.line":542},"message":"filebeat stopped.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2025-06-25T10:33:20.488-0700","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.handleError","file.name":"instance/beat.go","file.line":1355},"message":"Exiting: Failed to start crawler: starting input failed: error while initializing input: Error creating input. No such input type exist: 'nonexistent'","service.name":"filebeat","ecs.version":"1.6.0"}
Exiting: Failed to start crawler: starting input failed: error while initializing input: Error creating input. No such input type exist: 'nonexistent'

But at other times, Filebeat simply hangs while trying to shut itself down.

...
{"log.level":"info","@timestamp":"2025-06-25T10:30:30.701-0700","log.logger":"crawler","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.(*crawler).Start","file.name":"beater/crawler.go","file.line":72},"message":"Loading Inputs: 1","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-06-25T10:30:30.701-0700","log.logger":"crawler","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.(*crawler).Stop","file.name":"beater/crawler.go","file.line":156},"message":"Stopping Crawler","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-06-25T10:30:30.701-0700","log.logger":"crawler","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.(*crawler).Stop","file.name":"beater/crawler.go","file.line":166},"message":"Stopping 0 inputs","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-06-25T10:30:30.701-0700","log.logger":"crawler","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.(*crawler).Stop","file.name":"beater/crawler.go","file.line":186},"message":"Crawler stopped","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-06-25T10:30:30.701-0700","log.logger":"registrar","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/registrar.(*Registrar).Stop","file.name":"registrar/registrar.go","file.line":126},"message":"Stopping Registrar","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-06-25T10:30:30.701-0700","log.logger":"registrar","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/registrar.(*Registrar).Run","file.name":"registrar/registrar.go","file.line":162},"message":"Ending Registrar","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-06-25T10:30:30.701-0700","log.logger":"registrar","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/registrar.(*Registrar).Stop","file.name":"registrar/registrar.go","file.line":131},"message":"Registrar stopped","service.name":"filebeat","ecs.version":"1.6.0"}

With some debugging, it looks like Filebeat gets stuck on this line while trying to shut itself down:

defer stateStore.Close()

Expected behavior

Filebeat always exits with an error indicating that the input type is invalid.

Metadata

Metadata

Assignees

Type

No fields configured for Bug.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions