Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

graphqlbackend: Support perforce changelists in repo and commits API#51291

Merged
indradhanush merged 10 commits into
mainfrom
ig/perforce-changelists-api
May 4, 2023
Merged

graphqlbackend: Support perforce changelists in repo and commits API#51291
indradhanush merged 10 commits into
mainfrom
ig/perforce-changelists-api

Conversation

@indradhanush

@indradhanush indradhanush commented May 1, 2023

Copy link
Copy Markdown
Contributor

The changes in this PR were extracted from #51195 and then iterated upon once it was clear they need to be in a separate PR.

Why

We want to show changelist IDs for perforce depots in the UI instead of commit SHAs which is meaningless for a perforce user.

What

In this PR, we introduce some growing changes to the repository and commits APIs. They are:

  • Added a boolean isPerforceDepot enum sourceType to the Repositories API to differentiate between a Perforce depot converted to a git repo vs a native git repo
  • Added a new nullable type perforceChangelist to the Commits API
  • If a repo was converted from a perforce depot, we parse the Body of of the commit object to return the original changelist message as the Subject
  • If a repo was converted from a perforce depot, the Body API will now return nil because we do not want to show the metadata that was added to the commit's body during the conversion of the perforce changelist to a git commit

Test plan

  • Added unit tests
  • Tested with UI updates (not part of this PR)

@cla-bot cla-bot Bot added the cla-signed label May 1, 2023
@indradhanush indradhanush force-pushed the ig/perforce-changelists-api branch 3 times, most recently from bd8c3c6 to b90ebe8 Compare May 1, 2023 12:11
@indradhanush indradhanush marked this pull request as ready for review May 1, 2023 12:15
@indradhanush indradhanush requested review from a team and peterguy May 1, 2023 12:15
@indradhanush indradhanush self-assigned this May 1, 2023
Comment thread cmd/frontend/graphqlbackend/schema.graphql Outdated
@indradhanush indradhanush force-pushed the ig/perforce-changelists-api branch from b90ebe8 to 529c8d0 Compare May 3, 2023 10:46
@indradhanush indradhanush requested a review from keegancsmith May 3, 2023 10:48
@sourcegraph-bot

sourcegraph-bot commented May 3, 2023

Copy link
Copy Markdown
Contributor

📖 Storybook live preview

Comment thread cmd/frontend/graphqlbackend/git_commit.go Outdated
Comment thread cmd/frontend/graphqlbackend/repository.go Outdated
//
// For depots converted with git-p4, this special handling is NOT required.
if r.repoResolver.IsPerforceDepot() && strings.HasPrefix(commit.Message.Body(), "[p4-fusion") {
subject, err := parseP4FusionCommitSubject(commit.Message.Subject())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the graphqlbackend package is quite a behemoth already. I recommend new code which contains business logic to go somewhere else and then the resolver code is very very lightweight. Can you update this PR to put the P4 parsing coding and the other non-graphql specific p4 code somewhere else?

It also feels a bit weird to update just this graphql resolver to massage the subject. But I don't have a better suggestion, so I guess lets do this. But it almost feels like in some other layer you want to translate git commit messages into a different format. To me it feels like that should just be done in the p4-fusion layer? IE all you are doing is translating the message here, why not just write the mssage you want in p4-fusion? That does sound a bit more hectic to do though.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both git p4 and p4-fusion add Perforce information to the git commit message when they convert Perforce changelists to git commits. That information has traditionally simply been displayed to the user in the UI, but now we want to hide that information and show the user only the original commit message from Perforce. However, the extra information is important because it contains the Changelist Id, which we want to display in the UI instead of commit hashes, and also use in URLs as we go forward with these changes. We are hesitant to change p4-fusion and git p4 (we don't own those tools, we currently need that changelist info, and there are clients with already-synced depots, so they have this commit message format), so we need to parse the commit message (and subject, for ones generated by p4-fusion) at some point along the way to the UI; there might be a better place to do it, but I'm not sure we can push it any deeper than this.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I agree with that point. I still think p4 specific business logic like this should not live inside the graphql resolver for a git commit subject. It feels alright that we inject something here though that looks like this. IE I am mostly taking issue with strings.HasPrefix(commit.Message.Body(), "[p4-fusion") { living here. It feels like the impact on this function should be mostly a single function call which works something like

if subject, ok = maybeTransformP4Subject(ctx, r.repoResolver, commit.Message); ok {
  return subject
}

Or alternatively based on slack discussions the design of this is gonna change a bit and the impl of subject resolver will live in something p4 specific.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, pushing that code you mentioned into a function would be prettier and more portable.

@sashaostrikov sashaostrikov left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Left a couple of comments.

Comment thread cmd/frontend/graphqlbackend/git_commit_test.go
Comment thread cmd/frontend/graphqlbackend/git_commit_test.go
Comment thread cmd/frontend/graphqlbackend/perforce_changelist_test.go Outdated
"""
Returns true if this git-repo was converted from a perforce depot.
"""
isPerforceDepot: Boolean!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember this being something @mrnugget was worried about happening (similarly to adding additional isX isY stuff to the Repository and other types)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One idea we were thinking of having as an adopted pattern in the packages side of things was moving to union types across the graphql API, but thatd be a much bigger endeavor (for which we had a staffing proposal under the packages project, unfortunately not going ahead this quarter :/ )

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this concern. Should probably use the same suggestion I used here https://github.com/sourcegraph/sourcegraph/pull/51291#discussion_r1182834582

Is your suggestion something along the lines of my second suggestion? IE rely on ... on PerforceRepo {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is your suggestion something along the lines of my second suggestion? IE rely on ... on PerforceRepo {

I think along that line of thought, yea. In the client code you can switch on __typename and render differently (or different components entirely) based on that value to e.g. include or exclude stuff based on if its e.g. a PerforceRepo, PackageRepo, GitRepo etc

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was originally the idea we had come up with in a chat between Philipp Spiess, Ryan Slade, Peter and myself. I had planned for it to be a subproject under this quarters packages project and for the idea to be picked up then by other non-git ecosystems, but alas no staffing.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In our case, this might not be a trivial change though, since you'd probably need to change the "main case" (git repository) in all providers. But maybe there's a way around that and make it gradual, i.e. define a interface PerforceDepot and then implement ToPerforceDepot() on the type you have here and return new type when it's a perforce depot.

My only concern is if this is the right time to be implementing this as an interface, since at the root of it all its still a git repo that we're presenting as a perforce depot by only tweaking the commit message and the commit ref.

It would have made sense if we were to introduce perforce depots as first class citizens and then this change might have been a better time to do it then. What do you think?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's the same thing as what you already have: an enum and a field that's only valid if the enum has a certain value. Except that the interfaces hide that from you and are more GraphQL-native. With your solution you'd need to query this:

query {
  repo(name: "foo/bar") {
    # [...]
    sourceType
    perforceChangelist
  }
}

Then in your client-side application code you'd have to do this:

if (repo.sourceType === sourceType.PERFORCE_DEPOT) {
  // use changelist
}

With an interface you'd do this (I think, again, you'd need to confirm this):

query {
  repo(name: "foo/bar") {
    # [...]
    ... on PerforceDepot {
      perforceChangelist
    }
  }
}

And you'd get a PerforceDepot type in your client-side code.

What I'm unsure about is whether repo needs to be an interface for that or not. If it has to be an interface, then it's a lot of work to change, I think.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay - let me give this a try and come back with what I find out.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think in order to do that it would have to become an interface. I think further down the line it would be very nice to have a interface Repository and then two types type Depot and type GitRepository that will host non-common fields to have a proper abstraction. I don't know when we want to go more first class here, as long as it's only the 2 or 3 fields here that are depot specific, I think we can live with that for now, but we should account for a more complex migration of the API further down the line.

@indradhanush indradhanush May 4, 2023

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary based on quick chat on Slack

Here's the ideal scenario:

  1. Define a new interface (say Source) with the common fields between a perforce depot and a git repo
  2. Update repo to implement this Source interface and add fields like stars , fork etc
  3. Create a new type that also implements Source and adds the perforceChangelist on top of it

But this is likely going to be a big change (especially in the client) and out of scope of this PR but should be on our minds to get around to eventually.

For the time being - the approach of enum should be good to go ahead with.

@sourcegraph-buildkite

sourcegraph-buildkite commented May 4, 2023

Copy link
Copy Markdown
Collaborator

Bundle size report 📦

Initial size Total size Async size Modules
0.01% (+0.19 kb) 0.00% (+0.19 kb) 0.00% (0.00 kb) 0.00% (0)

Look at the Statoscope report for a full comparison between the commits ba38eea and e08c73f or learn more.

Open explanation
  • Initial size is the size of the initial bundle (the one that is loaded when you open the page)
  • Total size is the size of the initial bundle + all the async loaded chunks
  • Async size is the size of all the async loaded chunks
  • Modules is the number of modules in the initial bundle

// If parsing this commit message fails for some reason, log the reason and fall-through
// to return the the original git-commit's subject instead of a hard failure or an empty
// subject.
r.logger.Error("failed to parse p4 fusio commit subject", log.Error(err))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
r.logger.Error("failed to parse p4 fusio commit subject", log.Error(err))
r.logger.Error("failed to parse p4 fusion commit subject", log.Error(err))

@keegancsmith keegancsmith left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

backend LGTM. some comments inline, I'll leave it to the people who had the further slack discussion on API design to approve the graphql schema changes.

//
// For depots converted with git-p4, this special handling is NOT required.
if r.repoResolver.IsPerforceDepot() && strings.HasPrefix(commit.Message.Body(), "[p4-fusion") {
subject, err := parseP4FusionCommitSubject(commit.Message.Subject())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I agree with that point. I still think p4 specific business logic like this should not live inside the graphql resolver for a git commit subject. It feels alright that we inject something here though that looks like this. IE I am mostly taking issue with strings.HasPrefix(commit.Message.Body(), "[p4-fusion") { living here. It feels like the impact on this function should be mostly a single function call which works something like

if subject, ok = maybeTransformP4Subject(ctx, r.repoResolver, commit.Message); ok {
  return subject
}

Or alternatively based on slack discussions the design of this is gonna change a bit and the impl of subject resolver will live in something p4 specific.

return nil, errors.Wrap(err, "failed to retrieve innerRepo")
}

if r.innerRepo.ExternalRepo.ServiceType == extsvc.TypePerforce {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use the returned repo above, don't directly access innerRepo

@eseliger eseliger left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GQL schema change LGTM

@indradhanush indradhanush merged commit e09a769 into main May 4, 2023
@indradhanush indradhanush deleted the ig/perforce-changelists-api branch May 4, 2023 17:18
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants