Skip to content

commands/{pre-,}push: allow missing objects#2078

Closed
ttaylorr wants to merge 4 commits intomasterfrom
allow-missing-objects
Closed

commands/{pre-,}push: allow missing objects#2078
ttaylorr wants to merge 4 commits intomasterfrom
allow-missing-objects

Conversation

@ttaylorr
Copy link
Contributor

This pull request teaches the git lfs push & git lfs pre-push commands to allow missing objects by default, and enforce that they exist with the --strict-mode flag.

From #2063:

The commands should still return with a non-zero exit code, but at least the other objects have a chance to be uploaded.

I think printing a warning and exiting cleanly is OK in this scenario, where --strict-mode is not given, because the command did what it was supposed to do given the expectations set by the caller.

Finally, double check that upload retries reset the progress meter properly.

I verified that this is what happens, but I'm unsure if it's required that I q.meter.Pause() and q.meter.Start() in order to log to stderr. I think that it is, but perhaps not? (cc @technoweenie)

Lastly, I think that this was originally targeted for v2.0.2 (from #2063), but perhaps it should just ship in 2.1.0 since --strict-mode is a new flag and this is technically a breaking change?


/cc @git-lfs/core #2063
Closes: #2063

@ttaylorr ttaylorr added this to the v2.1.0 milestone Mar 24, 2017
@ttaylorr ttaylorr requested a review from technoweenie March 24, 2017 02:39
@ttaylorr ttaylorr force-pushed the allow-missing-objects branch 2 times, most recently from 55cc4bc to 451ffd3 Compare March 24, 2017 03:10
@technoweenie
Copy link
Contributor

I don't think we need --strict-mode. The push commands should always be strict. They should just follow through the entire upload, uploading everything they can, before exiting with an error. By adding --strict-mode, the unsafe behavior (ignoring missing objects) is default.

@ttaylorr
Copy link
Contributor Author

I don't think we need --strict-mode. The push commands should always be strict. They should just follow through the entire upload, uploading everything they can, before exiting with an error. By adding --strict-mode, the unsafe behavior (ignoring missing objects) is default.

Good point, what do you think about changing --strict-mode to --allow-missing and leaving it to false by default?

@ttaylorr
Copy link
Contributor Author

Re-reading this, I just saw:

They should just follow through the entire upload, uploading everything they can, before exiting with an error.

I think keeping in --allow-missing is a safe idea, so long as we continue to let the transfer queue operate until it's processed all objects and Wait() has been called, only then can we exit with a non-zero code if --allow-missing is not given.

@ttaylorr
Copy link
Contributor Author

Third take -- let's remove --strict-mode (and the idea of --allow-missing) all together, and still allow lfs push and lfs pre-push to upload all objects that it can and exit in a dirty state if any of the objects in the pushset were missing.

This will allow

  • All of the LFS objects that do exist to travel to the remote as early as possible
  • A user to see all of the missing objects in one push instead of getting one error at a time
  • A user to "fix" the missing object later, and have a smaller push size when they complete the fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants