Skip to content

Suppress batch default handling of moving products out #6932

@PaulWessel

Description

@PaulWessel

Description of the desired feature

The batch module runs jobs in parallel and may produce products with unique names, similar to that of movie. When a single job completes it tries to move any named products up to the top directory, which for my case may give scripts that end with

mv -f ${BATCH_NAME}.* /Users/pwessel/Dropbox/UH/ACTIVE/PROJECTS/OXFORD2022/EMPERORS

However, not every possibly batch scenario will produce single products that are called ${BATCH_NAME}.* (here, EFLX3_1000.*). In my case, batch is given a file via -T with 3 columns of parameters (two densities and an elastic thickness) and the output of the script is a single grid that has those parameters in the file name, since flex_2510_2930_15.0k.grd is much more helpful to me (I can see the densities and the elastic thickness I used via the file name) than EFLX3_1433.grd, where I would have to run grdinfo to learn which solution this is. Thus, there are clearly cases where the single number naming is not helpful (whereas it is always used for movies).

I will think more about this but some initial thoughts are listed here:

  1. We could add an option -C that does not issue those move commands and then the user is on the hook to do something specific in the main script. In my case I write directly to an outside final directory, so no need to move.
  2. Alternatively, we add an option that takes a format statement and then BATCH_NAME and others are formed from the template and the various columns in the -Tfile. In my case, that would mean I pass something like -Fflex_%4.0lf_%4.0lf_%4.1lfk and then this template is used with the three items. If the file had trailing texts etc then it would use %s instead.

I think the second one is getting a bit too complicated and will experiment with the first for now, but I do like the power of the 2nd option.

Maybe @Esteban82 has thoughts on this?

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions