Skip to content

Commit 55820ee

Browse files
theakshaypanttheakshaypant
committed
feat(github): add glob pattern support for token scope repos
Add support for glob patterns when specifying repositories for GitHub App token scoping in both global and repository-level configurations. Users can now use patterns like "myorg/*" to match multiple repos instead of listing each one explicitly. For global config (secret-github-app-scope-extra-repos), glob patterns are expanded by listing all repositories where the GitHub App is installed. Results are cached to avoid repeated API calls when multiple patterns are specified. For repository-level config (github_app_token_scope_repos), glob patterns are matched against Repository CRDs in the same namespace, maintaining the existing namespace isolation requirement. Both exact matches and glob patterns can be combined. Invalid glob patterns are validated early and return clear error messages. Jira: https://issues.redhat.com/browse/SRVKP-10030 Signed-off-by: Akshay Pant <akshay.akshaypant@gmail.com>
1 parent 680e99e commit 55820ee

5 files changed

Lines changed: 241 additions & 21 deletions

File tree

‎docs/content/docs/guide/repositorycrd.md‎

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -240,6 +240,7 @@ This setting enables the scoping of the GitHub token to private and public repos
240240
### Scoping the GitHub token using Global configuration
241241

242242
You can use the global configuration to use as a list of repositories used from any Repository CR in any namespace.
243+
You can specify repositories using exact names or glob patterns (e.g., `myorg/*` to match all repositories under an organization where the app is installed).
243244

244245
To set the global configuration, in the `pipelines-as-code` configmap, set the `secret-github-app-scope-extra-repos` key, as in the following example:
245246

@@ -250,13 +251,14 @@ To set the global configuration, in the `pipelines-as-code` configmap, set the `
250251
name: pipelines-as-code
251252
namespace: pipelines-as-code
252253
data:
253-
secret-github-app-scope-extra-repos: "owner2/project2, owner3/project3"
254+
secret-github-app-scope-extra-repos: "owner2/project2, owner3/*"
254255
```
255256

256257
### Scoping the GitHub token using Repository level configuration
257258

258259
You can use the `Repository` custom resource to scope the generated GitHub token to a list of repositories.
259260
The repositories can be public or private, but must reside in the same namespace as the repository with which the `Repository` resource is associated.
261+
You can specify repositories using exact names or glob patterns (e.g., `myorg/*` to match all repositories under an organization which have the GitHub app installed).
260262

261263
Set the `github_app_token_scope_repos` spec configuration within the `Repository` custom resource, as in the following example:
262264

@@ -271,18 +273,18 @@ Set the `github_app_token_scope_repos` spec configuration within the `Repository
271273
settings:
272274
github_app_token_scope_repos:
273275
- "owner/project"
274-
- "owner1/project1"
276+
- "owner1/*"
275277
```
276278

277279
In this example, the `Repository` custom resource is associated with the `linda/project` repository in the `test-repo` namespace.
278-
The scope of the generated GitHub token is extended to the `owner/project` and `owner1/project1` repositories, as well as the `linda/project` repository. These repositories must exist under the `test-repo` namespace.
280+
The scope of the generated GitHub token is extended to the `owner/project` repository, all repositories matching `owner1/*`, as well as the `linda/project` repository. These repositories must exist under the `test-repo` namespace.
279281

280282
**Note:**
281283

282-
If any of the repositories do not exist in the namespace, the scoping of the GitHub token fails with an error message as in the following example:
284+
If any of the repositories or patterns do not match any repository in the namespace, the scoping of the GitHub token fails with an error message as in the following example:
283285

284286
```console
285-
failed to scope GitHub token as repo owner1/project1 does not exist in namespace test-repo
287+
failed to scope GitHub token as repo with pattern owner1/project1 does not exist in namespace test-repo
286288
```
287289

288290
### Combining global and repository level configuration
@@ -335,10 +337,10 @@ creation of the GitHub token fails with the following error message:
335337
```
336338

337339
- If the scoping of the GitHub token to the repositories set in global or repository level configuration fails for any reason,
338-
the CI process does not run. This includes cases where the same repository is listed in the global or repository level configuration,
340+
the CI process does not run. This includes cases where the same repository is listed (or matched) in the global or repository level configuration,
339341
and the scoping fails for the repository level configuration because the repository is not in the same namespace as the `Repository` custom resource.
340342

341-
In the following example, the `owner5/project5` repository is listed in both the global configuration and in the repository level configuration:
343+
In the following example, the `owner5/project5` repository is listed in the global configuration and matches the pattern in the repository level configuration:
342344

343345
```yaml
344346
apiVersion: v1
@@ -360,11 +362,11 @@ and the scoping fails for the repository level configuration because the reposit
360362
url: "https://github.com/linda/project"
361363
settings:
362364
github_app_token_scope_repos:
363-
- "owner5/project5"
365+
- "owner5/*"
364366
```
365367

366-
In this example, if the `owner5/project5` repository is not under the `test-repo` namespace, scoping of the GitHub token fails with the following error message:
368+
In this example, if the `owner5/project5` repository (or any other repository satisfying the specified pattern of `owner5/*`) is not under the `test-repo` namespace, scoping of the GitHub token fails with the following error message:
367369

368370
```yaml
369-
failed to scope GitHub token as repo owner5/project5 does not exist in namespace test-repo
371+
failed to scope GitHub token as repo with pattern owner5/* does not exist in namespace test-repo
370372
```

‎pkg/provider/github/github.go‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ import (
1111
"sync"
1212
"time"
1313

14+
"github.com/gobwas/glob"
1415
"github.com/google/go-github/v74/github"
1516
"github.com/jonboulle/clockwork"
1617
"github.com/openshift-pipelines/pipelines-as-code/pkg/apis/pipelinesascode/keys"
@@ -636,7 +637,17 @@ func ListRepos(ctx context.Context, v *Provider) ([]string, error) {
636637
}
637638

638639
func (v *Provider) CreateToken(ctx context.Context, repository []string, event *info.Event) (string, error) {
640+
var appReposCache []*github.Repository
641+
639642
for _, r := range repository {
643+
// Check if this is a glob pattern
644+
if strings.ContainsAny(r, "*?[") {
645+
if err := v.expandGlobAndAddRepoIDs(ctx, r, &appReposCache); err != nil {
646+
v.Logger.Warn("failed to expand glob pattern %q: %v", r, err)
647+
}
648+
continue
649+
}
650+
640651
split := strings.Split(r, "/")
641652
infoData, _, err := wrapAPI(v, "get_repository", func() (*github.Repository, *github.Response, error) {
642653
return v.Client().Repositories.Get(ctx, split[0], split[1])
@@ -655,6 +666,52 @@ func (v *Provider) CreateToken(ctx context.Context, repository []string, event *
655666
return token, nil
656667
}
657668

669+
func (v *Provider) expandGlobAndAddRepoIDs(ctx context.Context, repoPattern string, cache *[]*github.Repository) error {
670+
// We can skip error check here as all the glob compilation has been checked
671+
// before this method is called.
672+
reposToScope, _ := glob.Compile(repoPattern)
673+
674+
if *cache == nil {
675+
repos, err := v.listAppRepos(ctx)
676+
if err != nil {
677+
return err
678+
}
679+
*cache = repos
680+
}
681+
682+
for _, repo := range *cache {
683+
repoFullName := repo.GetFullName()
684+
if reposToScope.Match(repoFullName) {
685+
v.RepositoryIDs = uniqueRepositoryID(v.RepositoryIDs, repo.GetID())
686+
}
687+
}
688+
689+
return nil
690+
}
691+
692+
func (v *Provider) listAppRepos(ctx context.Context) ([]*github.Repository, error) {
693+
var allRepos []*github.Repository
694+
695+
opt := &github.ListOptions{PerPage: v.PaginedNumber}
696+
for {
697+
repoList, resp, err := wrapAPI(v, "list_app_repos", func() (*github.ListRepositories, *github.Response, error) {
698+
return v.Client().Apps.ListRepos(ctx, opt)
699+
})
700+
if err != nil {
701+
return nil, fmt.Errorf("failed to list app repos: %w", err)
702+
}
703+
704+
allRepos = append(allRepos, repoList.Repositories...)
705+
706+
if resp.NextPage == 0 {
707+
break
708+
}
709+
opt.Page = resp.NextPage
710+
}
711+
712+
return allRepos, nil
713+
}
714+
658715
func uniqueRepositoryID(repoIDs []int64, id int64) []int64 {
659716
r := repoIDs
660717
m := make(map[int64]bool)

‎pkg/provider/github/scope.go‎

Lines changed: 29 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ import (
77
"net/url"
88
"strings"
99

10+
"github.com/gobwas/glob"
1011
"github.com/openshift-pipelines/pipelines-as-code/pkg/apis/pipelinesascode/v1alpha1"
1112
"github.com/openshift-pipelines/pipelines-as-code/pkg/events"
1213
"github.com/openshift-pipelines/pipelines-as-code/pkg/params"
@@ -24,7 +25,7 @@ func ScopeTokenToListOfRepos(ctx context.Context, vcx provider.Interface, pacInf
2425
listRepos bool
2526
token string
2627
)
27-
listURLs := map[string]string{}
28+
listURLs := []string{}
2829
repoListToScopeToken := []string{}
2930

3031
// This is a Global config to provide list of repos to scope token
@@ -34,6 +35,14 @@ func ScopeTokenToListOfRepos(ctx context.Context, vcx provider.Interface, pacInf
3435
if configValueS == "" {
3536
continue
3637
}
38+
// May or may not be a glob
39+
if _, err := glob.Compile(configValueS); err != nil {
40+
msg := fmt.Sprintf("invalid repo glob specified %s", configValueS)
41+
eventEmitter.EmitMessage(nil, zap.ErrorLevel, "InvalidRepoGlobSpecified", msg)
42+
43+
return "", errors.New(msg)
44+
}
45+
3746
repoListToScopeToken = append(repoListToScopeToken, configValueS)
3847
}
3948
listRepos = true
@@ -50,15 +59,30 @@ func ScopeTokenToListOfRepos(ctx context.Context, vcx provider.Interface, pacInf
5059
if err != nil {
5160
return "", err
5261
}
53-
listURLs[splitData[1]+"/"+splitData[2]] = splitData[1] + "/" + splitData[2]
62+
listURLs = append(listURLs, splitData[1]+"/"+splitData[2])
5463
}
5564
for i := range repo.Spec.Settings.GithubAppTokenScopeRepos {
56-
if _, ok := listURLs[repo.Spec.Settings.GithubAppTokenScopeRepos[i]]; !ok {
57-
msg := fmt.Sprintf("failed to scope GitHub token as repo %s does not exist in namespace %s", repo.Spec.Settings.GithubAppTokenScopeRepos[i], ns)
65+
// May or may not be a glob
66+
repoToScope, err := glob.Compile(repo.Spec.Settings.GithubAppTokenScopeRepos[i])
67+
if err != nil {
68+
msg := fmt.Sprintf("invalid repo glob specified %s", repo.Spec.Settings.GithubAppTokenScopeRepos[i])
69+
eventEmitter.EmitMessage(nil, zap.ErrorLevel, "InvalidRepoGlobSpecified", msg)
70+
71+
return "", errors.New(msg)
72+
}
73+
globMatchFound := false
74+
// Match glob to repos list.
75+
for _, repoKey := range listURLs {
76+
if repoToScope.Match(repoKey) {
77+
repoListToScopeToken = append(repoListToScopeToken, repoKey)
78+
globMatchFound = true
79+
}
80+
}
81+
if !globMatchFound {
82+
msg := fmt.Sprintf("failed to scope GitHub token as repo with pattern %s does not exist in namespace %s", repo.Spec.Settings.GithubAppTokenScopeRepos[i], ns)
5883
eventEmitter.EmitMessage(nil, zap.ErrorLevel, "RepoDoesNotExistInNamespace", msg)
5984
return "", errors.New(msg)
6085
}
61-
repoListToScopeToken = append(repoListToScopeToken, repo.Spec.Settings.GithubAppTokenScopeRepos[i])
6286
}
6387
// When the global configuration is not set then check for secret-github-app-token-scoped key for the repo level configuration
6488
if pacInfo.SecretGHAppRepoScoped && pacInfo.SecretGhAppTokenScopedExtraRepos == "" {

‎pkg/provider/github/scope_test.go‎

Lines changed: 139 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,75 @@ func TestScopeTokenToListOfRepos(t *testing.T) {
7171
URL: privateRepo,
7272
},
7373
}
74+
75+
// Additional repos for glob pattern testing
76+
repoDataGlobAndExact := &v1alpha1.Repository{
77+
TypeMeta: metav1.TypeMeta{},
78+
ObjectMeta: metav1.ObjectMeta{
79+
Name: "publicrepo-glob-and-exact",
80+
Namespace: testNamespace.Name,
81+
},
82+
Spec: v1alpha1.RepositorySpec{
83+
URL: repoFromWhichEventComes,
84+
Settings: &v1alpha1.Settings{
85+
GithubAppTokenScopeRepos: []string{"owner1/repo1", "owner2/*"},
86+
},
87+
},
88+
}
89+
90+
repoDataInvalidGlob := &v1alpha1.Repository{
91+
TypeMeta: metav1.TypeMeta{},
92+
ObjectMeta: metav1.ObjectMeta{
93+
Name: "publicrepo-invalid-glob",
94+
Namespace: testNamespace.Name,
95+
},
96+
Spec: v1alpha1.RepositorySpec{
97+
URL: repoFromWhichEventComes,
98+
Settings: &v1alpha1.Settings{
99+
GithubAppTokenScopeRepos: []string{"owner2/[invalid"},
100+
},
101+
},
102+
}
103+
104+
repoDataGlobNoMatch := &v1alpha1.Repository{
105+
TypeMeta: metav1.TypeMeta{},
106+
ObjectMeta: metav1.ObjectMeta{
107+
Name: "publicrepo-glob-nomatch",
108+
Namespace: testNamespace.Name,
109+
},
110+
Spec: v1alpha1.RepositorySpec{
111+
URL: repoFromWhichEventComes,
112+
Settings: &v1alpha1.Settings{
113+
GithubAppTokenScopeRepos: []string{"nonexistent/*"},
114+
},
115+
},
116+
}
117+
118+
// Additional private repos for glob testing
119+
privateRepo2 := "https://org.com/owner2/repo3"
120+
repoData2 := &v1alpha1.Repository{
121+
TypeMeta: metav1.TypeMeta{},
122+
ObjectMeta: metav1.ObjectMeta{
123+
Name: "privaterepo2",
124+
Namespace: testNamespace.Name,
125+
},
126+
Spec: v1alpha1.RepositorySpec{
127+
URL: privateRepo2,
128+
},
129+
}
130+
131+
privateRepo3 := "https://org.com/owner1/repo1"
132+
repoData3 := &v1alpha1.Repository{
133+
TypeMeta: metav1.TypeMeta{},
134+
ObjectMeta: metav1.ObjectMeta{
135+
Name: "privaterepo3",
136+
Namespace: testNamespace.Name,
137+
},
138+
Spec: v1alpha1.RepositorySpec{
139+
URL: privateRepo3,
140+
},
141+
}
142+
74143
tests := []struct {
75144
name string
76145
tData testclient.Data
@@ -129,7 +198,7 @@ func TestScopeTokenToListOfRepos(t *testing.T) {
129198
},
130199
repository: repoData,
131200
repoListsByGlobalConf: "",
132-
wantError: "failed to scope GitHub token as repo owner2/repo2 does not exist in namespace pipelinesascode",
201+
wantError: "failed to scope GitHub token as repo with pattern owner2/repo2 does not exist in namespace pipelinesascode",
133202
wantToken: "",
134203
},
135204
{
@@ -178,6 +247,69 @@ func TestScopeTokenToListOfRepos(t *testing.T) {
178247
wantToken: "123abcdfrf",
179248
repositoryID: []int64{789, 10112, 112233},
180249
},
250+
{
251+
name: "repos are listed using both glob pattern and exact match",
252+
tData: testclient.Data{
253+
Namespaces: []*corev1.Namespace{testNamespace},
254+
Secret: []*corev1.Secret{validSecret},
255+
Repositories: []*v1alpha1.Repository{
256+
repoDataGlobAndExact, repoData1, repoData2, repoData3,
257+
},
258+
},
259+
repository: repoDataGlobAndExact,
260+
repoListsByGlobalConf: "",
261+
wantError: "",
262+
wantToken: "123abcdfrf",
263+
repositoryID: []int64{789, 10112, 112233, 445566},
264+
},
265+
{
266+
name: "invalid glob pattern returns error",
267+
tData: testclient.Data{
268+
Namespaces: []*corev1.Namespace{testNamespace},
269+
Secret: []*corev1.Secret{validSecret},
270+
Repositories: []*v1alpha1.Repository{
271+
repoDataInvalidGlob, repoData1,
272+
},
273+
},
274+
repository: repoDataInvalidGlob,
275+
repoListsByGlobalConf: "",
276+
wantError: "invalid repo glob specified",
277+
wantToken: "",
278+
},
279+
{
280+
name: "glob pattern that matches no repos returns error",
281+
tData: testclient.Data{
282+
Namespaces: []*corev1.Namespace{testNamespace},
283+
Secret: []*corev1.Secret{validSecret},
284+
Repositories: []*v1alpha1.Repository{
285+
repoDataGlobNoMatch, repoData1,
286+
},
287+
},
288+
repository: repoDataGlobNoMatch,
289+
repoListsByGlobalConf: "",
290+
wantError: "failed to scope GitHub token as repo with pattern nonexistent/* does not exist in namespace pipelinesascode",
291+
wantToken: "",
292+
},
293+
{
294+
name: "invalid glob pattern in global config returns error",
295+
tData: testclient.Data{
296+
Namespaces: []*corev1.Namespace{testNamespace},
297+
Secret: []*corev1.Secret{validSecret},
298+
},
299+
repository: &v1alpha1.Repository{
300+
TypeMeta: metav1.TypeMeta{},
301+
ObjectMeta: metav1.ObjectMeta{
302+
Name: "publicrepo",
303+
Namespace: testNamespace.Name,
304+
},
305+
Spec: v1alpha1.RepositorySpec{
306+
URL: repoFromWhichEventComes,
307+
},
308+
},
309+
repoListsByGlobalConf: "owner1/[invalid",
310+
wantError: "invalid repo glob specified",
311+
wantToken: "",
312+
},
181313
}
182314
for _, tt := range tests {
183315
t.Run(tt.name, func(t *testing.T) {
@@ -232,7 +364,7 @@ func TestScopeTokenToListOfRepos(t *testing.T) {
232364
pacInfo: pacInfo,
233365
}
234366

235-
extraRepoInstallIDs := map[string]string{"owner/repo": "789", "owner1/repo1": "10112", "owner2/repo2": "112233"}
367+
extraRepoInstallIDs := map[string]string{"owner/repo": "789", "owner1/repo1": "10112", "owner2/repo2": "112233", "owner2/repo3": "445566"}
236368
for v := range extraRepoInstallIDs {
237369
split := strings.Split(v, "/")
238370
mux.HandleFunc(fmt.Sprintf("/repos/%s/%s", split[0], split[1]), func(w http.ResponseWriter, _ *http.Request) {
@@ -243,8 +375,11 @@ func TestScopeTokenToListOfRepos(t *testing.T) {
243375
eventEmitter := events.NewEventEmitter(run.Clients.Kube, logger)
244376
token, err := ScopeTokenToListOfRepos(ctx, gvcs, pacInfo, tt.repository, run, info, eventEmitter, logger)
245377
assert.Equal(t, token, tt.wantToken)
246-
if err != nil {
247-
assert.Equal(t, err.Error(), tt.wantError)
378+
if tt.wantError != "" {
379+
assert.Assert(t, err != nil, "expected error but got none")
380+
assert.Assert(t, strings.Contains(err.Error(), tt.wantError), "error %q should contain %q", err.Error(), tt.wantError)
381+
} else {
382+
assert.NilError(t, err)
248383
}
249384
assert.Equal(t, len(gvcs.RepositoryIDs), len(tt.repositoryID))
250385
})

0 commit comments

Comments
 (0)