Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

batches: support distinct GH App installs for commit signing across namespaces #53212

Merged
courier-new merged 27 commits into
batches/commit-signingfrom
kr/multi-installs
Jun 13, 2023
Merged

batches: support distinct GH App installs for commit signing across namespaces #53212
courier-new merged 27 commits into
batches/commit-signingfrom
kr/multi-installs

Conversation

@courier-new

@courier-new courier-new commented Jun 9, 2023

Copy link
Copy Markdown
Contributor

Closes https://github.com/sourcegraph/sourcegraph/issues/53280.

Adds support for installing a Batches domain GitHub App on multiple accounts and then locating the appropriate installation by repo namespace == account name in order to authenticate the ChangesetSource for duplicating and signing commits.

Walkthrough

I'm the master of producing chunky PRs lately so here's another code walkthrough to ease the burden of reviewing this. Remember that more than half the files changed are just for stupid logger plumbing and nothing else. 🙈 We'll go back to front again.

new migrations - [code]

  • The first migration adds additional metadata about a GitHub App Installation to the github_app_installs table. It also creates an index based on the account_login, since that's how we'll look up the right install for a given commit signing interaction.
  • The second migration adds a unique constraint on for app_id and installation_id. This enables us to easily update records with the same SQL INSERT statement we use to add a new one.

github_apps/store - [code]

  • Install now takes a fully formed ghtypes.GitHubAppInstallation as an input and performs an upsert (insert or update) of the record.
  • GetLatestInstallID has been replaced with the more relevant GetInstallID, which looks up installations by app ID and account/namespace name and returns the installation ID of the latest record that matches.
  • BulkInstall was removed because it was easier to just loop and upsert individual records.
  • SyncInstallations is a new method that performs the logic that was originally in the installation backfill worker. This allows us to tap into it in another place. The logic has been enhanced slight as well:
    • We now delete all Installations for an App if that App was not reachable on GitHub.
    • We now add (upsert) records for all Installations returned from the remote request, including existing ones. It upserts them with all the additional metadata.

worker/installation_backfill - [code]

  • The worker routine has been simplified to mostly just preparing a GitHub client and then calling store.SyncInstallations. The test has largely been migrated to the store package tests instead.

batches/sources - [code]

  • withGitHubAppAuthenticator now takes account as an additional argument and uses it to look up the installation for both the right app ID and account name when creating the installation authenticator.

batches.graphql - [code]

  • I replaced our GitHubAppConfiguration type with the same GitHubApp type from the GH Apps GraphQL schema, since we're increasingly needing more and more properties of the App in the Batches space and it was beginning to feel redundant. Now the commitSigningConfiguration field will resolve to a full GitHubApp rather than our shallow copy of one.

anything in frontend/internal/batches

  • Almost everything here is literally me just passing a logger or maybe an EnterpriseDB around. It was extremely painful to write, but hopefully less painful to skim. 🙂
  • The only exception is code_host.go, where we now nest a githubapp.NewGitHubAppResolver for the commit signing configuration.

graphqlbackend.graphqlbackend.go - [code]

  • Because the Batches GraphQL Schema now depends on the GitHub Apps Schema also being available, I updated NewSchemaWithBatchChangesResolver to provide both as resolvers, and updated NewSchema to accept a slice of resolvers instead of just one. This had a bit of a ripple effect in that I had to update everyone who calls this method to call it with a slice now. 😅

githubappauth/resolver.go - [code]

  • Originally, to resolve the Installations field on a GitHubApp, this resolver would requery GitHub itself for the Installations every time. Now we take the Installations we have stored in the DB to resolve this field faster.
  • We also opportunistically kick of store.SyncInstallations whenever Installations is called, so the set of Installations for the GitHubApp will be updated immediately after if they are out-of-date. This has the added benefit of providing us a way to trigger a "manual sync" of GitHub App Installations if needed.
  • This sync process is offloaded to a goroutine so as not to block the GraphQL request from completing.
  • Resolving Installations is also now guarded behind a sync.Once mechanism to ensure the function is only executed once for a given GraphQL request.

githubappauth/middleware.go [code]

  • The /setup handler will now fetch the whole GitHub App Installation before saving it in the DB, to ensure we can populate all the necessary metadata.

GitHubAppPage - [code]

  • This page now has a configurable header breadcrumb.
  • Rather than showing the warning banner about being un-editable if it's rendered for a Batches App, now it shows just the parts of the page that are relevant to Batch Changes, namely, the basic App details and the list of Installations registered on Sourcegraph.
  • This page is now accessible for Batches domain Apps at /site-admin/batch-changes/github-apps/:id.

Demo

Prior to this PR, we would just pretend that whatever GitHub App installation we had would work for any repositories on that code host. That would mean that publishing changesets on repositories from separate orgs wouldn't just not produce signed commits, it would actually just outright fail to publish.

In the demo, I start out with a new GitHub App, which I only install to my personal user account namespace. I test that when I publish two changesets, one to a repo in my user namespace and one to a repo in the sourcegraph-testing namespace, they both succeed to publish. In this case, the sourcegraph-testing one just falls back on normal batch change user credentials, yielding an unsigned commit but one which is authored by me. However, the courier-new one is able to authenticate with as the GH App installation for that account and duplicate the commit, yielding the signed one!

Then, I make the GH App publish and install it for sourcegraph-testing. I modify the branch name for my batch spec to trigger new changesets to be published and publish the two again. This time, both succeed and have signed commits.

Screen.Recording.2023-06-09.at.3.56.31.PM.mov

Test plan

Updated unit tests where applicable. Manual testing of the whole flow.

@courier-new courier-new added batch-changes Issues related to Batch Changes batches-signed-commit labels Jun 9, 2023
@courier-new courier-new self-assigned this Jun 9, 2023
@cla-bot cla-bot Bot added the cla-signed label Jun 9, 2023
Base automatically changed from kr/better-remove-modal to kr/ux-create-app-2 June 9, 2023 23:18
@courier-new courier-new requested a review from a team June 10, 2023 04:06
@courier-new courier-new marked this pull request as ready for review June 10, 2023 04:06
@sourcegraph-bot

sourcegraph-bot commented Jun 10, 2023

Copy link
Copy Markdown
Contributor

Codenotify: Notifying subscribers in CODENOTIFY files for diff 8af181a...41f3297.

Notify File(s)
@BolajiOlajide client/web/src/enterprise/batches/settings/BatchChangesSettingsArea.story.tsx
client/web/src/enterprise/batches/settings/BatchChangesSiteConfigSettingsPage.story.tsx
client/web/src/enterprise/batches/settings/CommitSigningIntegrationNode.tsx
client/web/src/enterprise/batches/settings/backend.ts
@efritz enterprise/cmd/worker/internal/githubapps/worker/BUILD.bazel
enterprise/cmd/worker/internal/githubapps/worker/installation_backfill.go
enterprise/cmd/worker/internal/githubapps/worker/installation_backfill_test.go
@eseliger client/web/src/enterprise/batches/settings/BatchChangesSettingsArea.story.tsx
client/web/src/enterprise/batches/settings/BatchChangesSiteConfigSettingsPage.story.tsx
client/web/src/enterprise/batches/settings/CommitSigningIntegrationNode.tsx
client/web/src/enterprise/batches/settings/backend.ts
cmd/frontend/graphqlbackend/batches.go
enterprise/internal/batches/sources/sources.go
enterprise/internal/batches/sources/sources_test.go
@unknwon enterprise/cmd/frontend/internal/auth/githubappauth/middleware.go
enterprise/cmd/frontend/internal/auth/githubappauth/resolver.go
enterprise/cmd/frontend/internal/auth/githubappauth/resolver_test.go

@sourcegraph-bot

sourcegraph-bot commented Jun 10, 2023

Copy link
Copy Markdown
Contributor

📖 Storybook live preview

Base automatically changed from kr/ux-create-app-2 to batches/commit-signing June 12, 2023 15:29
@courier-new courier-new requested a review from pjlast June 12, 2023 15:38

@st0nebreaker st0nebreaker left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great! Thank you Kelli! I manually tested and see it all working 😄

I am noticing with multiple installs, the 2nd nodes "View in GitHub" link is broken. Do you see this on your end?

Screen.Recording.2023-06-12.at.3.30.28.PM.mov

Comment thread cmd/frontend/graphqlbackend/batches.graphql
Comment thread enterprise/internal/github_apps/store/store.go Outdated
Comment thread enterprise/cmd/frontend/internal/auth/githubappauth/resolver.go

@pjlast pjlast left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not much to add, just see comment on table constraint 🙏

Comment thread enterprise/internal/github_apps/store/store_test.go Outdated
Comment thread migrations/frontend/squashed.sql
@courier-new

Copy link
Copy Markdown
Contributor Author

I am noticing with multiple installs, the 2nd nodes "View in GitHub" link is broken. Do you see this on your end?

Oh hmm, I don't know how thoroughly I looked at that. I love your test org by the way. 😂 I'll look into that. Is the account that you're signed into on GitHub when you opened those links an owner of that org? It might 404 if your account isn't allowed to administer that installation.

@courier-new courier-new merged commit 44829f6 into batches/commit-signing Jun 13, 2023
@courier-new courier-new deleted the kr/multi-installs branch June 13, 2023 17:12
@st0nebreaker

st0nebreaker commented Jun 13, 2023

Copy link
Copy Markdown
Contributor

Oh hmm, I don't know how thoroughly I looked at that. I love your test org by the way. 😂 I'll look into that. Is the account that you're signed into on GitHub when you opened those links an owner of that org? It might 404 if your account isn't allowed to administer that installation.

Haha thank yoou ♀️

Oh I'm not the owner of that org, I believe it allowed me to sign a commit through that org though with this setup. Let me test again, I have to step away but will test through this again when I'm back. Even so, we should maybe add more messaging around this edge case, right?

@courier-new

courier-new commented Jun 13, 2023

Copy link
Copy Markdown
Contributor Author

Oh I'm not the owner of that org, I believe it allowed me to sign a commit through that org though with this setup.

Yeah that would be expected to me. A normal user should still be able authenticate with the installation, they just won't be able to administer it (and hence might not be able to see the install page for it on GitHub)

Even so, we should maybe add more messaging around this edge case, right?

Probably! But we won't be able to know if the current user is an org owner or not. We have this note on the "Creation" page about installing apps on orgs: "Only organization owners can register GitHub Apps."

Maybe I could add a note like that at the top of the "Installations" list? Like "Only organization owners can manage installations for organizations."?

@st0nebreaker

Copy link
Copy Markdown
Contributor

Maybe I could add a note like that at the top of the "Installations" list? Like "Only organization owners can manage installations for organizations."?

That sounds like a great solution to me! I just tested again (video below) with 3 orgs, one I'm just a member of, not an owner. The org settings aren't available, but everything else from SG batches commit signing works well. I think this disclaimer would be 👌🏻

Screen.Recording.2023-06-13.at.4.50.48.PM.mov

@courier-new

Copy link
Copy Markdown
Contributor Author

Great, thank you for testing and confirming that. 😄 I'll follow up on that.

coury-clark referenced this pull request Jun 20, 2023
… installations (#53790)

@st0nebraker [pointed
out](https://github.com/sourcegraph/sourcegraph/pull/53212#pullrequestreview-1475310777)
in a previous PR that the "View on GitHub" link for GitHub App
installations only works if you are able to administer that
installation, e.g. that you are an org owner for the org where it's
installed, if it's on an org, or that you are the user. If we had
this information, we could just hide the button for installations the
user can't actually view, but since we don't, we thought it
could help to add a note instead.

## Test plan

Just an in-product docs change. Verified the text appears on the GitHub
App page.

<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->
 <br> Backport be1ff32 from #53611

Co-authored-by: Kelli Rockwell <kelli@sourcegraph.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

batch-changes Issues related to Batch Changes cla-signed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants