Skip to content

batch and movie calling grdblend can lead to instability #5763

@PaulWessel

Description

@PaulWessel

Description of the problem

This is another problem specific to batch and movie: When working with tiled remote datasets the collection of tiles is sent to grdblend via a GMT_Call_Module call. As the blending proceeds there are places where temporary grids are created, such as

/tmp/grdblend_resampled_53553_5.nc

where 53553 was the process PID and 5 is a running number (starting at 0). The problem seems to be (again, hard to know for sure and to debug these issues) that some of the other concurrent processes launched by movie or batch may be trying to create the same files, and suddenly we see these types of messages:

grdblend [ERROR]: File /tmp/grdblend_resampled_53553_5.nc was not found

Clearly, one process thought it was done with its temporary file but another process had the same name and now there is chaos. Running movie or batch with -x2 means we only launch one job at the time and no such error appears. The more cores, the more likely the instability.

A relative simple solution would be not to just use the PID to make the temporary file but use the standard mktemp function. That way we get unique files on the file system and not just in some process.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions