Skip to content

Possible thread leak at the SSI server (v5.6.2) #2468

@iagaponenko

Description

@iagaponenko

When stress-testing Qserv (which relies upon SSI), crashes were observed at the server process during thread creation. The following message was posted by the application log file:

terminate called after throwing an instance of 'std::system_error'
  what():  Resource temporarily unavailable

The crash was observed in a non-XROOTD/SSI code. However, further examination of the core file revealed that XROTD had 28525 theads out of 29206 total threads created by the process. All XROOTD threads were in the following state at the time of the crash:

XrdSysSemaphore::Wait

The problem has already been reported to @abh3, who pointed me out to the following documentation: https://xrootd.web.cern.ch/doc/dev57/xrd_config.htm#_Toc171719950 and suggested to configure the server to limit the number of threads.

However, the documentation says that the default limit assumed by XROOTD/SSI is:

Defaults
     xrd.sched mint 8 maxt 2048 avlt 512 idle 780

And this is far lower than the number of threads observed. Hence this request. I suspect that it's either that the documentation is incorrect or the implementation has a bug, since it may not enforce the thread limit as specified in the documentation.

Configuring the server explicitly with the above-mentioned default values DID affect the application - it seemed to stop exceeding the specified thread limit.

Metadata

Metadata

Assignees

Labels

LSSTVera Rubin Observatory — http://lsst.org

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions