Skip to content

fix: Fix Transform hang in CLI sync#2001

Merged
marianogappa merged 4 commits intomainfrom
mariano/wrap-errors-in-status
Dec 12, 2024
Merged

fix: Fix Transform hang in CLI sync#2001
marianogappa merged 4 commits intomainfrom
mariano/wrap-errors-in-status

Conversation

@marianogappa
Copy link
Copy Markdown
Contributor

@marianogappa marianogappa commented Dec 11, 2024

When CLI runs sync_v3 and a transformer is in the spec, if the transformer errors at transform time, the CLI can hang forever.

This fix makes sure that the .Transform() function always returns after an error.

Tested 20 times:

Loading spec(s) from cmd/testdata/transformer-errors.yml
Starting sync for: test (cloudquery/test@v4.7.0) -> [test (cloudquery/test@v2.7.0)]
Error: failed to sync v3 source test: rpc error: code = Internal desc = failing at the transformer stage according to spec requirements
exit status 1
Loading spec(s) from cmd/testdata/transformer-errors.yml
Starting sync for: test (cloudquery/test@v4.7.0) -> [test (cloudquery/test@v2.7.0)]
Error: failed to sync v3 source test: rpc error: code = Internal desc = failing at the transformer stage according to spec requirements
exit status 1
Loading spec(s) from cmd/testdata/transformer-errors.yml
Starting sync for: test (cloudquery/test@v4.7.0) -> [test (cloudquery/test@v2.7.0)]
Error: failed to sync v3 source test: rpc error: code = Internal desc = failing at the transformer stage according to spec requirements
exit status 1
Loading spec(s) from cmd/testdata/transformer-errors.yml
Starting sync for: test (cloudquery/test@v4.7.0) -> [test (cloudquery/test@v2.7.0)]
Error: failed to sync v3 source test: rpc error: code = Internal desc = failing at the transformer stage according to spec requirements
exit status 1
Loading spec(s) from cmd/testdata/transformer-errors.yml
Starting sync for: test (cloudquery/test@v4.7.0) -> [test (cloudquery/test@v2.7.0)]
Error: failed to sync v3 source test: rpc error: code = Internal desc = failing at the transformer stage according to spec requirements
exit status 1
Loading spec(s) from cmd/testdata/transformer-errors.yml
Starting sync for: test (cloudquery/test@v4.7.0) -> [test (cloudquery/test@v2.7.0)]
Error: failed to sync v3 source test: rpc error: code = Internal desc = failing at the transformer stage according to spec requirements
exit status 1
Loading spec(s) from cmd/testdata/transformer-errors.yml
Starting sync for: test (cloudquery/test@v4.7.0) -> [test (cloudquery/test@v2.7.0)]
Error: failed to sync v3 source test: EOF
exit status 1
Loading spec(s) from cmd/testdata/transformer-errors.yml
Starting sync for: test (cloudquery/test@v4.7.0) -> [test (cloudquery/test@v2.7.0)]
Error: failed to sync v3 source test: rpc error: code = Internal desc = failing at the transformer stage according to spec requirements
exit status 1
Loading spec(s) from cmd/testdata/transformer-errors.yml
Starting sync for: test (cloudquery/test@v4.7.0) -> [test (cloudquery/test@v2.7.0)]
Error: failed to sync v3 source test: rpc error: code = Internal desc = failing at the transformer stage according to spec requirements
exit status 1
Loading spec(s) from cmd/testdata/transformer-errors.yml
Starting sync for: test (cloudquery/test@v4.7.0) -> [test (cloudquery/test@v2.7.0)]
Error: failed to sync v3 source test: rpc error: code = Internal desc = failing at the transformer stage according to spec requirements
exit status 1
Loading spec(s) from cmd/testdata/transformer-errors.yml
Starting sync for: test (cloudquery/test@v4.7.0) -> [test (cloudquery/test@v2.7.0)]
Error: failed to sync v3 source test: rpc error: code = Internal desc = failing at the transformer stage according to spec requirements
exit status 1
Loading spec(s) from cmd/testdata/transformer-errors.yml
Starting sync for: test (cloudquery/test@v4.7.0) -> [test (cloudquery/test@v2.7.0)]
Error: failed to sync v3 source test: rpc error: code = Internal desc = failing at the transformer stage according to spec requirements
exit status 1
Loading spec(s) from cmd/testdata/transformer-errors.yml
Starting sync for: test (cloudquery/test@v4.7.0) -> [test (cloudquery/test@v2.7.0)]
Error: failed to sync v3 source test: rpc error: code = Internal desc = failing at the transformer stage according to spec requirements
exit status 1
Loading spec(s) from cmd/testdata/transformer-errors.yml
Starting sync for: test (cloudquery/test@v4.7.0) -> [test (cloudquery/test@v2.7.0)]
Error: failed to sync v3 source test: rpc error: code = Internal desc = failing at the transformer stage according to spec requirements
exit status 1
Loading spec(s) from cmd/testdata/transformer-errors.yml
Starting sync for: test (cloudquery/test@v4.7.0) -> [test (cloudquery/test@v2.7.0)]
Error: failed to sync v3 source test: rpc error: code = Internal desc = failing at the transformer stage according to spec requirements
exit status 1
Loading spec(s) from cmd/testdata/transformer-errors.yml
Starting sync for: test (cloudquery/test@v4.7.0) -> [test (cloudquery/test@v2.7.0)]
Error: failed to sync v3 source test: rpc error: code = Internal desc = failing at the transformer stage according to spec requirements
exit status 1
Loading spec(s) from cmd/testdata/transformer-errors.yml
Starting sync for: test (cloudquery/test@v4.7.0) -> [test (cloudquery/test@v2.7.0)]
Error: failed to sync v3 source test: rpc error: code = Internal desc = failing at the transformer stage according to spec requirements
exit status 1
Loading spec(s) from cmd/testdata/transformer-errors.yml
Starting sync for: test (cloudquery/test@v4.7.0) -> [test (cloudquery/test@v2.7.0)]
Error: failed to sync v3 source test: rpc error: code = Internal desc = failing at the transformer stage according to spec requirements
exit status 1
Loading spec(s) from cmd/testdata/transformer-errors.yml
Starting sync for: test (cloudquery/test@v4.7.0) -> [test (cloudquery/test@v2.7.0)]
Error: failed to sync v3 source test: rpc error: code = Internal desc = failing at the transformer stage according to spec requirements
exit status 1
Loading spec(s) from cmd/testdata/transformer-errors.yml
Starting sync for: test (cloudquery/test@v4.7.0) -> [test (cloudquery/test@v2.7.0)]
Error: failed to sync v3 source test: rpc error: code = Internal desc = failing at the transformer stage according to spec requirements
exit status 1

@marianogappa marianogappa requested review from a team and erezrokah December 11, 2024 13:16
@github-actions github-actions bot added the fix label Dec 11, 2024
erezrokah
erezrokah approved these changes Dec 11, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@erezrokah erezrokah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, had a few questions mostly to clarify how everything works

@marianogappa
Copy link
Copy Markdown
Contributor Author

marianogappa commented Dec 12, 2024

Looks promising on CLI tests

Screenshot 2024-12-12 at 13 07 36

erezrokah
erezrokah approved these changes Dec 12, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@erezrokah erezrokah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 🚀

@marianogappa marianogappa merged commit 0474b9f into main Dec 12, 2024
@marianogappa marianogappa deleted the mariano/wrap-errors-in-status branch December 12, 2024 11:18
@cq-bot cq-bot mentioned this pull request Dec 12, 2024
Merged
kodiakhq bot pushed a commit that referenced this pull request Dec 12, 2024
@cq-bot
chore(main): Release v4.71.1 (#2002)
759d173 
🤖 I have created a release *beep* *boop*
---


## [4.71.1](v4.71.0...v4.71.1) (2024-12-12)


### Bug Fixes

* Fix Transform hang in CLI sync ([#2001](#2001)) ([0474b9f](0474b9f))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@erezrokah erezrokah erezrokah approved these changes

Assignees

No one assigned

Labels

fix

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@marianogappa @erezrokah