-
Notifications
You must be signed in to change notification settings - Fork 408
Description
Description of the problem
This is another problem specific to batch and movie: When working with tiled remote datasets the collection of tiles is sent to grdblend via a GMT_Call_Module call. As the blending proceeds there are places where temporary grids are created, such as
/tmp/grdblend_resampled_53553_5.nc
where 53553 was the process PID and 5 is a running number (starting at 0). The problem seems to be (again, hard to know for sure and to debug these issues) that some of the other concurrent processes launched by movie or batch may be trying to create the same files, and suddenly we see these types of messages:
grdblend [ERROR]: File /tmp/grdblend_resampled_53553_5.nc was not found
Clearly, one process thought it was done with its temporary file but another process had the same name and now there is chaos. Running movie or batch with -x2 means we only launch one job at the time and no such error appears. The more cores, the more likely the instability.
A relative simple solution would be not to just use the PID to make the temporary file but use the standard mktemp function. That way we get unique files on the file system and not just in some process.