Skip to content

Stops sending logs after harvester error #12988

@glenjamin

Description

@glenjamin

While slurping logs from a nomad install, we found that filebeat would stop sending logs, and tailing the log showed no errors.

Looking further up the logs we do see a single error line, but then the process keeps running and doesn't ship any logs.

Log sample:

2019-07-19T18:17:48.295Z        ERROR   log/input.go:484        Harvester could not be started on new file: /opt/nomad/alloc/ba9e1994-fd5e-8e64-55fb-4afa2e1a1894/alloc/logs/.asldjkasldjaslkdjsalkdjasdlkjasd.stdout.fifo, Err: error setting up harvester: Harvester setup failed. Unexpected file opening error: Tried to open non regular file: "prw-------" .asldjkasldjaslkdjsalkdjasdlkjasd.stdout.fifo

The affected directory looks like this

-rw-r--r-- 1 root   root    13970 Jul 19 19:13 asdfghjklasdfghjklsdfghjk.stderr.0
prw------- 1 root   root        0 Jul 19 19:13 .asdfghjklasdfghjklsdfghjk.stderr.fifo|
-rw-r--r-- 1 root   root       30 Jul 19 19:12 asdfghjklasdfghjklsdfghjk.stdout.0
prw------- 1 root   root        0 Jul 19 19:12 .asdfghjklasdfghjklsdfghjk.stdout.fifo|

For confirmed bugs, please report:

  • Version: 7.2.0
  • Operating System: Ubuntu 18.04
  • Discuss Forum URL: n/a
  • Steps to Reproduce: have the glob match some named pipes

Metadata

Metadata

Assignees

No one assigned

    Labels

    FilebeatFilebeatStalledneeds_teamIndicates that the issue/PR needs a Team:* label

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions