Skip to content

segfault when "viewing" closed mmap object #9537

@ahaldane

Description

@ahaldane

This crashes in python2 and python3 with a segfault:

import mmap

# create test file
a = arange(10., dtype='f4')
a.tofile('test')

# opens it as a memmap
with open('test', 'r+b') as f:
      mm = mmap.mmap(f.fileno(),0)

#view the mmap, then close it
x = np.frombuffer(mm, dtype='f4')
mm.close()
print(x)  # segfault

In frombuffer mumpy uses the PEP3118 interface to the mmap to get a pointer to the data, so it seems we are using the interface incorrectly somewhere.

It seems to me that the "correct" behavior may be for it to be impossible to close the memmap while pointers to it exist; this is the behavior for memoryviews of mmaps. That is, the line mm.close() shoud raise an error BufferError: cannot close exported pointers, as in currently happens in python3:

>>> import mmap
>>> with open('test', 'r+b') as f:
>>>     mm = mmap.mmap(f.fileno(),0)
>>> mv = memoryview(mm)
>>> mm.close()
BufferError: cannot close exported pointers exist

however note also that in python2 we cannot create the memoryview (mv) because of an error TypeError: cannot make memory view because object does not have the buffer interface.

Note how mmap_buffer_getbuf (link) in the mmap module takes care to keep track of any buffer access to the mmap (the exports counter). Evidently we are somehow bypassing that in numpy.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions