Skip to content

Input execd - process are not closed when Telegraf (service) stops #7876

@Trovalo

Description

@Trovalo

Relevant telegraf.conf:

[[inputs.execd]]

   command = [
       'powershell',
       '-File',
       'C:\Monitoring\telegraf\Configuration\input_scripts\xesmarttarget\xe_qmonitor.ps1',
       '-ServerName',
       'SQLCSRV04\SQL2017'
   ]

  signal = "none"
  restart_delay = "1m"
  data_format = "influx"
  
[[inputs.execd]]

  command = [
      'powershell',
      '-File',
      'C:\Monitoring\telegraf\Configuration\input_scripts\xesmarttarget\xe_qmonitor.ps1',
      '-ServerName',
      'SQLCSRV04\ITEC'
  ]

  signal = "none"
  restart_delay = "1m"
  data_format = "influx"

System info:

  • Telegraf Nightly (last build)
  • Windows Server 2016

Steps to reproduce:

  1. run Telegraf as a service (on windows)
  2. use the execd plugin to start one or more processes
  3. stop the telegraf service

You will see that the started processes are still up and running.

The same does not happen when running telegraf as a console command (cmd), if the cmd command is stopped using ctrl+c all the started processes are also closed

Expected behavior:

If the telegraf service is stopped (on windows), the processes started by execd are closed

Actual behavior:

when the telegraf service is stopped (on windows) the execd started processes are not closed

Additional info:

Here are the logs generated by telegraf
Running as a service

2020-07-22T14:10:02Z I! Loaded inputs: exec execd execd internal sqlserver sqlserver
2020-07-22T14:10:02Z I! Loaded aggregators: 
2020-07-22T14:10:02Z I! Loaded processors: strings converter strings
2020-07-22T14:10:02Z I! Loaded outputs: influxdb
2020-07-22T14:10:02Z I! Tags enabled: company=quantumdatis_demo host=QDSRVMONITOR
2020-07-22T14:10:02Z I! [agent] Config: Interval:15s, Quiet:false, Hostname:"QDSRVMONITOR", Flush Interval:10s
2020-07-22T14:10:02Z D! [agent] Initializing plugins
2020-07-22T14:10:02Z D! [agent] Connecting outputs
2020-07-22T14:10:02Z D! [agent] Attempting connection to [outputs.influxdb]
2020-07-22T14:10:02Z D! [agent] Successfully connected to outputs.influxdb
2020-07-22T14:10:02Z D! [agent] Starting service inputs
2020-07-22T14:10:02Z I! [inputs.execd] Starting process: C:\Program Files\XESmartTarget\XESmartTarget.exe [--File C:\Monitoring\telegraf\Configuration\input_scripts\xesmarttarget\telegraf.json --NoLogo --Quiet --GlobalVariables ServerName=SQLCSRV04\SQL2017]
2020-07-22T14:10:02Z I! [inputs.execd] Starting process: powershell [-File C:\Monitoring\telegraf\Configuration\input_scripts\xesmarttarget\xe_qmonitor.ps1 -ServerName SQLCSRV04\ITEC]
2020-07-22T14:10:12Z D! [outputs.influxdb] Buffer fullness: 0 / 15000 metrics
2020-07-22T14:10:22Z D! [outputs.influxdb] Wrote batch of 1230 metrics in 313.4507ms
2020-07-22T14:10:22Z D! [outputs.influxdb] Buffer fullness: 0 / 15000 metrics
2020-07-22T14:10:30Z D! [agent] Stopping service inputs

Running as a cmd command

2020-07-22T14:11:36Z I! [inputs.execd] Starting process: powershell [-File C:\Monitoring\telegraf\Configuration\input_scripts\xesmarttarget\xe_qmonitor.ps1 -ServerName SQLCSRV04\ITEC]
2020-07-22T14:11:47Z D! [outputs.influxdb] Wrote batch of 1229 metrics in 292.0335ms
2020-07-22T14:11:47Z D! [outputs.influxdb] Buffer fullness: 0 / 15000 metrics
2020-07-22T14:11:54Z D! [agent] Stopping service inputs
2020-07-22T14:11:57Z D! [outputs.influxdb] Buffer fullness: 0 / 15000 metrics
2020-07-22T14:11:59Z I! [inputs.execd] Process C:\Program Files\XESmartTarget\XESmartTarget.exe shut down
2020-07-22T14:12:04Z E! [inputs.execd] Error in plugin: read |0: file already closed
2020-07-22T14:12:04Z E! [inputs.execd] Error in plugin: error reading stderr: read |0: file already closed
2020-07-22T14:12:04Z I! [inputs.execd] Process C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe shut down
2020-07-22T14:12:04Z D! [agent] Input channel closed
2020-07-22T14:12:04Z D! [agent] Processor channel closed
2020-07-22T14:12:04Z D! [agent] Processor channel closed
2020-07-22T14:12:04Z D! [agent] Processor channel closed
2020-07-22T14:12:04Z I! [agent] Hang on, flushing any cached metrics before shutdown
2020-07-22T14:12:04Z D! [outputs.influxdb] Buffer fullness: 0 / 15000 metrics
2020-07-22T14:12:04Z D! [agent] Stopped Successfully

As you can see in the logs, the "service log" last message is D! [agent] Stopping service inputs, meanwhile in the "cmd command log" there are additional messages confirming that the child processes have been closed

Metadata

Metadata

Assignees

Labels

area/execdIssues related to execd or plugins that would be better suited to be used through execdplatform/windows

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions