Skip to content

Logging leaks file handles leading to FD exhaustion for long running scripts #12882

@elpollouk

Description

@elpollouk

Describe the bug
Each call to get_default_cli().invoke(...) will leak a file handle due to logging.

To Reproduce
Run this script under Linux:

from azure.cli.core import get_default_cli
import os
import tempfile

for _ in range(10):
    with tempfile.TemporaryFile('w+t') as temp:
        get_default_cli().invoke(["account", "show"], out_file=temp)

open_fds = os.listdir(f"/proc/{os.getpid()}/fd")
print("Leaked file handles:")
for fd in open_fds:
    try:
        target = os.readlink(f"/proc/{os.getpid()}/fd/{fd}")
        print(f"  {fd} - {target}")
    except:
        pass

Expected behavior
When the invoke() command returns, all internal resources specific to the child process are released.

Environment summary
Untunu 18.04
Python 3.6.9
azure-cli 2.3.1
azure-cli-core 2.3.1

Additional context
The problem is due to azlogging.py not cleaning up the logging.FileHandler resource allocation on line 101

I did try updating end_cmd_metadata_logging to clean up the file handler, but this method is not called if the az process errors, e.g. a problem with the user's credentials. This is because exception handling bypasses most clean up code.

Metadata

Metadata

Labels

CoreCLI core infrastructure

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions