Skip to content

Filestream input locks files in Windows #29113

@Sergi-GC

Description

@Sergi-GC
  • Version: v7.13.1 and v7.15.1
  • Operating System: Windows
  • Description:

When using the Filestream input in Windows, Filebeat locks the files it's monitoring and prevents other processes from renaming or moving the files.

This has been reproduced in Windows Server 2016, Windows 10 Enterprise and in Windows 10 Home environment.

Error in Windows Server 2016:
filebeat_filstream_locked_error

Error in Windows 10 Home:
windows-10-test

This issue does not reproduce in MacOS.
And also does not happen with the Log input.

  • Steps to reproduce:

Case 1: Commenting out the close inactive settings for both inputs

Settings:

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - C:\logs\this_file_is_not_locked.log

  clean_removed: true
  close_renamed: true
 #close_inactive: 1m
  scan_frequency: 10s

- type: filestream
  enabled: true
  paths:
    - C:\logs\this_file_is_locked.log

  clean_removed: true
  close.on_state_change.removed: true
  close.on_state_change.renamed: true
  #close.on_state_change.inactive: 1m
  prospector.scanner.check_interval: 10s

Results:

  • The file this_file_is_not_locked can be renamed/deleted/moved any time since the moment Filebeat is started.
  • The file this_file_is_locked cannot be renamed/deleted/moved since the moment Filebeat is started.

Case 2: Setting close inactive settings to 1min

Settings:

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - C:\logs\this_file_is_not_locked.log

  clean_removed: true
  close_renamed: true
  close_inactive: 1m
  scan_frequency: 10s

- type: filestream
  enabled: true
  paths:
    - C:\logs\this_file_is_locked.log
   
  clean_removed: true
  close.on_state_change.removed: true
  close.on_state_change.renamed: true
  close.on_state_change.inactive: 1m
  prospector.scanner.check_interval: 10s

Results:

  • The file this_file_is_not_locked can be renamed/deleted/moved any time since the moment Filebeat is started.
  • The file this_file_is_locked cannot be renamed/deleted/moved for about 1m since the moment Filebeat is started. After 1 min the file can be renamed/deleted/moved.

Case 3: Commenting out all the close/clean settings

Settings:

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - C:\logs\this_file_is_not_locked.log

  #clean_removed: true
  #close_renamed: true
  #close_inactive: 1m
  scan_frequency: 10s

- type: filestream
  enabled: true
  paths:
    - C:\logs\this_file_is_locked.log
   
  #clean_removed: true
  #close.on_state_change.removed: true
  #close.on_state_change.renamed: true
  #close.on_state_change.inactive: 1m
  prospector.scanner.check_interval: 10s

Results:

  • The file this_file_is_not_locked can be renamed/deleted/moved any time since the moment Filebeat is started.

  • The file this_file_is_locked cannot be renamed/deleted/moved for about 1m since the moment Filebeat is started.

  • Logs from each scenario:
    filebeat_case_1.log
    filebeat_case_2.log
    filebeat_case_3.log

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions