Skip to content

Retry fetching repo from template#7080

Merged
vilmibm merged 3 commits intocli:trunkfrom
heaths:issue7055
Mar 29, 2023
Merged

Retry fetching repo from template#7080
vilmibm merged 3 commits intocli:trunkfrom
heaths:issue7055

Conversation

@heaths
Copy link
Contributor

@heaths heaths commented Mar 3, 2023

Fixes #7055

@heaths
Copy link
Contributor Author

heaths commented Mar 3, 2023

/cc @samcoe with whom I've been discussing #7055, which this bug will resolve.

@samcoe samcoe self-assigned this Mar 5, 2023
@heaths
Copy link
Contributor Author

heaths commented Mar 13, 2023

@samcoe this doesn't actually work, hence still a draft. I feel the code is in better shape and I've added tests. But while git fetch would fail, git clone doesn't for some reason: it just warns that you cloned an empty repository. I could go back to the same commands that failed before, but this is really feeling like a backend bug. We can fix the CLI, but anyone else creating from a template then immediately cloning will hit this, too. Seems the cloneTemplateRepository mutation shouldn't return until the backend clone is actually complete.

@samcoe
Copy link
Contributor

samcoe commented Mar 14, 2023

@heaths That is annoying behavior from git. The code looks good for what it's worth. That is exactly what I was envisioning. Does git exit with a different status code when it warns that an empty repo was cloned? If so, we could interpret that and then try doing a fetch. In that case the clone would not retry but the fetch could.

This is definitely a bit of confusing behavior from the backend during the cloneTemplateRepository mutation but I believe it stems from the fact that setting up the repo on the backend is an async process. As the API doesn't know when the async process will be completed it decides not to wait and just returns back the response.

@heaths
Copy link
Contributor Author

heaths commented Mar 14, 2023

@samcoe no, exit code is still 0.

Does GraphQL - or GitHub's implementation of it - make any stipulations for long-running operations (LROs)? I'm on the REST API Stewardship Board for Azure and we have some guidelines for REST APIs like so: https://github.com/microsoft/api-guidelines/blob/vNext/azure/Guidelines.md#long-running-operations--jobs

I realize this is more academic, but for any LROs like that having a status monitor can be useful for customers to monitor for when a resource is available: perhaps the original resource for, say, a PUT/PATCH; or, a separate resource for a POST.

A quick search on "graphql long running operation" yields no specs, but some interesting ideas using either a status monitor or returning an interface / union. Perhaps something to consider. Again, though, I realize such a change would be breaking. Just food for thought, I guess.

For the problem at hand, checking stdout (or stderr? I didn't look where it went) for that warning would be too fragile. For whatever reason fetch failed, maybe instead of a git pull we could do what git pull effectively entails: a git fetch + git merge. That said, the behavior where git fetch fails but git pull doesn't is particularly peculiar. Perhaps git pull doesn't follow the exact same code path and does a "best effort". Unless this behavior was documented - or perhaps a way to make sure either fails in this case - I'd hate to take a dependency on it. I'll do some digging into that, though. Maybe there's a switch that could make git pull fail similarly.

@samcoe
Copy link
Contributor

samcoe commented Mar 15, 2023

@heaths That is unfortunate that the exit code is still 0, I agree that parsing stdout to see if the repo cloned was empty to too brittle. Let me know how the investigation goes on using git pull instead of git fetch. Perhaps that is the path forward.

I am unaware of any stipulations for long-running operations that GitHub has for the GraphQL API. I also wouldn't be the best person to ask even if there was one. I can ask internally if you would like. It would be nice if there was one though.

@heaths
Copy link
Contributor Author

heaths commented Mar 15, 2023

That'd be great if you want to have them reach out over email or Teams. I have a lot of experience in LROs and work with some others with even more. Not for GraphQL, mind you, but we could help with any REST discussions and some of that might translate to some ideas for GraphQL.

@heaths heaths marked this pull request as ready for review March 29, 2023 14:43
@heaths heaths requested a review from a team as a code owner March 29, 2023 14:43
@heaths heaths requested review from vilmibm and removed request for a team March 29, 2023 14:43
@heaths
Copy link
Contributor Author

heaths commented Mar 29, 2023

@samcoe this is ready. I found that if I specify the branch explicitly, it will fail the clone if the branch is not fully set up. I tried several times against a real template repo and got it to fail a couple of times. The updated test reflects that. I capture stderr and only emit that if it's not the expected error, which shows in the test. I attempted a few different ways to "inject" stderr into the stubs but, without significant overhaul of the internal runner, was unsuccessful; however, with these changes I did confirm in a live test that I didn't see "fatal: Remote branch main not found in upstream origin" when a retry happened (set GH_DEBUG=1).

@cliAutomation cliAutomation added the external pull request originating outside of the CLI core team label Mar 29, 2023
Copy link
Contributor

@mislav mislav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great; thank you! Just one point about the change in functionality

@heaths
Copy link
Contributor Author

heaths commented Mar 29, 2023

GitHub Actions seem to be experiencing trouble.

@vilmibm vilmibm merged commit 72f64cc into cli:trunk Mar 29, 2023
@heaths heaths deleted the issue7055 branch March 29, 2023 23:56
Minhanhac

This comment was marked as spam.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external pull request originating outside of the CLI core team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Regression] Cannot create and clone repo from template repo

6 participants