Skip to content

Concurrent read segfault #844

@ArnaudLevaufre

Description

@ArnaudLevaufre

Hello

When openning a netcdf file in read mode more than one time in a ThreadPool results in a segfault. When using a process pool there is no issue and the concurrent read works fine. It can be an issue in a web environment where the server is using threads to handle its clients and two or more clients make a request that need to read the same netcdf file.

I have a simple python script and a dataset that reproduces the segfault:
netcdfSegfault.zip

The python script provided in the zip:

import netCDF4
from multiprocessing.pool import ThreadPool

def read_netcdf(path):
    ieast = 1750
    iwest = 1760
    inorth = 1380
    isouth = 1370

    with netCDF4.Dataset(path, 'r') as ncf:
        return ncf['/Depth/ndepths'][:][iwest:ieast, isouth:inorth]


if __name__ == "__main__":
    print("Netcdf4 version: %s" % netCDF4.__version__)
    path = "./max_depth_ndepth_quonops.nc"
    with ThreadPool(2) as p:
        print(p.map(read_netcdf, [path for i in range(2)]))

Will output the following when executed.

Netcdf4 version: 1.4.0
Segmentation fault (core dumped)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions