Skip to content

Conversation

@rauanargyn
Copy link

@rauanargyn rauanargyn commented May 16, 2020

This PR aims to fix the issue with "multiprocessing.shared_memory.SharedMemory" class registering open shared memory with resource_tracker, causing shared memory clean up after any of the connecting processes exit. The idea is not to use the resource_tracker at all and let the user be responsible for shared memory clean up. This allows shared memory to persist even if either the creating or any other reading processes have long exited.

https://bugs.python.org/issue39959

shm.py

from multiprocessing import shared_memory
import struct
import sys

def connect_shm(name):
    shm_r = shared_memory.SharedMemory(name)
    print("Attached to shared memory")
    print(struct.unpack("i", bytes(shm_r.buf[0:4])))

    shm_r.close()

def create_shm(name, spin=False):
    shm_c = shared_memory.SharedMemory(name=name, create=True, size=4096*5)

    print("Created shared memory")

    buf = shm_c.buf

    buf[0:4] = struct.pack("i", 4096)

    shm_c.close()

if __name__ == "__main__":
    mode = sys.argv[1]

    name = "test"

    if mode == "-create":
        create_shm(name, True)
    elif mode == "-attach":
        connect_shm(name)
python3.8 shm.py -create
python3.7 shm.py -attach
  • Before this change

resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown

  • After this change

Created shared memory

Attached to shared memory
(4096,)

https://bugs.python.org/issue39959

@rauanargyn rauanargyn requested a review from tiran as a code owner May 16, 2020 21:19
@the-knights-who-say-ni
Copy link

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept this contribution by verifying everyone involved has signed the PSF contributor agreement (CLA).

Recognized GitHub username

We couldn't find a bugs.python.org (b.p.o) account corresponding to the following GitHub usernames:

@rauanargyn

This might be simply due to a missing "GitHub Name" entry in one's b.p.o account settings. This is necessary for legal reasons before we can look at this contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

You can check yourself to see if the CLA has been received.

Thanks again for the contribution, we look forward to reviewing it!

@rauanargyn
Copy link
Author

Will be adjusting tests

@rauanargyn rauanargyn closed this May 17, 2020
@rauanargyn rauanargyn deleted the fix-issues-39959 branch May 17, 2020 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants