Skip to content

[HOTFIX] Delete corrupt loose objects during LooseObjectsStep#1110

Merged
derrickstolee merged 12 commits intomicrosoft:releases/shippedfrom
derrickstolee:loose-check
May 9, 2019
Merged

[HOTFIX] Delete corrupt loose objects during LooseObjectsStep#1110
derrickstolee merged 12 commits intomicrosoft:releases/shippedfrom
derrickstolee:loose-check

Conversation

@derrickstolee
Copy link
Contributor

@derrickstolee derrickstolee commented May 7, 2019

These are the same commits as #1094 and #1116, now targeting releases/shipped.

If we have a corrupt loose object, the git pack-objects command will fail as it cannot put that object into a pack-file. This stops all future iterations of the LooseObjectsStep from succeeding, so users will be stuck in a bad situation.

  • Refactor WriteLooseObjectIds() to rely on a new LooseObjectsBatch() method.

  • On failure to create a pack, check all objects in the batch (using LooseObjectsBatch() and libgit2) and delete those that fail.

  • Start disposing the repo in the dehydrate verb for safety.

  • Create a functional test that verifies this behavior. Move the LooseObjectStepTests to have a new enlistment for each test, since we are messing with the object store. Part of the issue is that .NET doesn't allow us to move files that are marked as read-only, which becomes problematic as tests create pack-indexes and other files.

Resolves #1079

We will need to enumerate a batch of loose objects independently
of writing them to a stream. Pull out the loop and consume it as
an enumerator instead.
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
@derrickstolee derrickstolee added affects: performance affects: live-site Improving our ability to diagnose and fix the product labels May 7, 2019
@derrickstolee derrickstolee added this to the M152 milestone May 7, 2019
@derrickstolee derrickstolee requested review from jeschu1 and jrbriggs May 7, 2019 15:22
It appears that sometimes we are getting an extra loose object
after some of the steps, and it is probably because we are deleting
pack-files from the object store so we download a "missing" commit
or something.

Use ShouldBeAtLeast() to demonstrate that the loose objects we
expect to remain are still there.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
@derrickstolee derrickstolee merged commit 7880de1 into microsoft:releases/shipped May 9, 2019
@jrbriggs jrbriggs modified the milestones: M152, M153, M151 May 23, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

affects: live-site Improving our ability to diagnose and fix the product affects: performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants