Skip to content

Commit 926d278

Browse files
yevgenypatsdisq
yevgenypats
and
disq
authored
feat: Move AWS to use built-in transformations (no codegen) (#6337)
Similar to #6276 moving transformation to be part of our SDK cloudquery/plugin-sdk#564 This is related to cloudquery/plugin-sdk#573 Few notes on column changes: most of changes are just additional columns and this should be fine as we did too much transformation. we want to keep the code simple, focus on extraction and provide and easy way of doing "dumb" one level transformation via reflection. The additional column are because we have in some places a duplicate where we add our own `arn` so it can be consistent but this is our decision and we still want to be as close as possible to the API so some duplicates is fine (and it makes the code simple and consistent). breaking changes in `aws_docdb_pending_maintenance_actions ` those apparently never worked and those columns where always empty (they didn't have mock tests so we didn't catch those) Co-authored-by: Kemal <223029+disq@users.noreply.github.com> Co-authored-by: Kemal Hadimli <disq@users.noreply.github.com>
1 parent 779907c commit 926d278

778 files changed

Lines changed: 1823 additions & 37018 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎plugins/source/aws/Makefile‎

Lines changed: 1 addition & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -13,13 +13,8 @@ install-tools:
1313
lint:
1414
golangci-lint run --config ../../.golangci.yml
1515

16-
.PHONY: gen-code
17-
gen-code:
18-
grep -rl '// Code generated by codegen; DO NOT EDIT.' resources/services/* | xargs rm
19-
go run codegen/main.go
20-
2116
.PHONY: gen-mocks
22-
gen-mocks: gen-code
17+
gen-mocks:
2318
go install github.com/golang/mock/mockgen@v1.6.0
2419
rm -rf ./client/mocks/*
2520
go generate ./client/...

‎plugins/source/aws/client/helpers.go‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ import (
1313

1414
"github.com/aws/aws-sdk-go-v2/aws/arn"
1515
"github.com/aws/smithy-go"
16+
"github.com/cloudquery/plugin-sdk/codegen"
1617
"github.com/cloudquery/plugin-sdk/schema"
1718
)
1819

@@ -396,3 +397,31 @@ func Sleep(ctx context.Context, dur time.Duration) error {
396397
return nil
397398
}
398399
}
400+
401+
func CreateTrimPrefixTransformer(prefixes ...string) func(field reflect.StructField) (string, error) {
402+
return func(field reflect.StructField) (string, error) {
403+
name, err := codegen.DefaultNameTransformer(field)
404+
if err != nil {
405+
return "", err
406+
}
407+
for _, v := range prefixes {
408+
if strings.HasPrefix(name, v) {
409+
return name[len(v):], nil
410+
}
411+
}
412+
return name, nil
413+
}
414+
}
415+
416+
func CreateReplaceTransformer(replace map[string]string) func(field reflect.StructField) (string, error) {
417+
return func(field reflect.StructField) (string, error) {
418+
name, err := codegen.DefaultNameTransformer(field)
419+
if err != nil {
420+
return "", err
421+
}
422+
for k, v := range replace {
423+
name = strings.ReplaceAll(name, k, v)
424+
}
425+
return name, nil
426+
}
427+
}

‎plugins/source/aws/codegen/main.go‎

Lines changed: 0 additions & 132 deletions
This file was deleted.

‎plugins/source/aws/codegen/recipes/accessanalyzer.go‎

Lines changed: 0 additions & 70 deletions
This file was deleted.

‎plugins/source/aws/codegen/recipes/account.go‎

Lines changed: 0 additions & 46 deletions
This file was deleted.

‎plugins/source/aws/codegen/recipes/acm.go‎

Lines changed: 0 additions & 40 deletions
This file was deleted.

‎plugins/source/aws/codegen/recipes/amp.go‎

Lines changed: 0 additions & 52 deletions
This file was deleted.

0 commit comments

Comments
 (0)