fix: Ignore env variables in comments#625
Merged
kodiakhq[bot] merged 2 commits intomainfrom Jan 26, 2023
Merged
Conversation
⏱️ Benchmark resultsComparing with 281e706
|
erezrokah
approved these changes
Jan 26, 2023
| return nil, fmt.Errorf("%s and %s are reserved words in CloudQuery config", openPlaceholder, closePlaceholder) | ||
| } | ||
|
|
||
| // replace placeholder variables with valid yaml, otherwise it cannot be parsed |
Member
There was a problem hiding this comment.
GitHub wrote their own YAML parser. We should not do it https://github.com/cloudquery/plugin-sdk/issues/480#issuecomment-1404694684
candiduslynx
approved these changes
Jan 26, 2023
yevgenypats
approved these changes
Jan 26, 2023
kodiakhq bot
pushed a commit
that referenced
this pull request
Jan 26, 2023
🤖 I have created a release *beep* *boop* --- ## [1.30.0](v1.29.0...v1.30.0) (2023-01-26) ### Features * **destination:** Filter the duplicate primary keys prior to writing batch ([#629](#629)) ([505709e](505709e)), closes [#627](#627) ### Bug Fixes * Ignore env variables in comments ([#625](#625)) ([08bace8](08bace8)) * Only call `newExecutionClient` if needed ([#630](#630)) ([ece947f](ece947f)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This changes the yaml config parsing code to ignore comments entirely. The solution used here is relatively short and clean, but it required some unfortunate hacking because the curly brackets we use for env variable placeholders are not valid yaml, and so any attempt to parse the yaml with them still in place fails (unless we were to write our own parser).
Simple regex replacement of comments would also be prone to issues, because it wouldn't be aware of the context (maybe the
#is inside a string, for example). This solution replaces the variables with temporary placeholders, converts each yaml section to json and back to yaml to strip comments, and then replaces the temporary placeholders again before continuing as before.