Ability to resume a failed operation using checkpoint files #432
Conversation
aca9892 to
ddabf10
Compare
dba1080 to
a4021e5
Compare
| if (!manager.isEmpty()) { | ||
| writeCheckpointFile(manager); | ||
| LOGGER.info("Checkpoints for the current operation were written to {}.", CHECKPOINT_CSV); | ||
| } |
There was a problem hiding this comment.
Does it make sense to add another line in the log with something like the following, so that the user won't have to look up in the manual on how to leverage this checkpoint file?
LOGGER.info("To resume the current operation, add the following command line flag --dsbulk.log.checkpoint.file={} .", CHECKPOINT_CSV);
There was a problem hiding this comment.
Yes, good idea. I also think we should skip this message completely if the operation completed successfully, wdyt? It doesn't make any sense to resume a completed operation.
There was a problem hiding this comment.
Agreed. If the job didn't get interrupted, it will be confusing to see this message.
weideng1
left a comment
There was a problem hiding this comment.
I've tested the new resume feature with a failed job and I was able to get it to pick up where it left off. LGTM
Fixes #431.