Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: cloudquery/plugin-sdk
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 281e706
Choose a base ref
...
head repository: cloudquery/plugin-sdk
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: 521d78c
Choose a head ref
  • 4 commits
  • 9 files changed
  • 4 contributors

Commits on Jan 26, 2023

  1. fix: Ignore env variables in comments (#625) 

    - Fixes #613

This changes the yaml config parsing code to ignore comments entirely. The solution used here is relatively short and clean, but it required some unfortunate hacking because the curly brackets we use for env variable placeholders are not valid yaml, and so any attempt to parse the yaml with them still in place fails (unless we were to write our own parser). 

Simple regex replacement of comments would also be prone to issues, because it wouldn't be aware of the context (maybe the `#` is inside a string, for example). This solution replaces the variables with temporary placeholders, converts each yaml section to json and back to yaml to strip comments, and then replaces the temporary placeholders again before continuing as before.
    @hermanschaaf
    hermanschaaf authored Jan 26, 2023
    1 Configuration menu
    Copy the full SHA
    08bace8 View commit details
    Browse the repository at this point in the history

  2. fix: Only call newExecutionClient if needed (#630) 

    
This is more of hot fix/patch to #610.
#610 had an unwanted side effect on the migrate command, as now we init the plugin before running it.
initializing the plugin is expensive, requires credentials and does a whole lot of validations, for example:
https://github.com/cloudquery/cloudquery/blob/a06139efad9a2e8de14f186a223222c9d52ce58a/plugins/source/aws/client/client.go#L383
https://github.com/cloudquery/cloudquery/blob/a06139efad9a2e8de14f186a223222c9d52ce58a/plugins/source/azure/client/client.go#L124

Those seem unnecessary for tables migration (unless one uses dynamic tables).

---
    @erezrokah
    erezrokah authored Jan 26, 2023
    1 Configuration menu
    Copy the full SHA
    ece947f View commit details
    Browse the repository at this point in the history

  3. feat(destination): Filter the duplicate primary keys prior to writing… 

    … batch (#629)

Implements #627
    @candiduslynx
    candiduslynx authored Jan 26, 2023
    1 Configuration menu
    Copy the full SHA
    505709e View commit details
    Browse the repository at this point in the history

  4. chore(main): Release v1.30.0 (#628) 

    🤖 I have created a release *beep* *boop*
---


## [1.30.0](v1.29.0...v1.30.0) (2023-01-26)


### Features

* **destination:** Filter the duplicate primary keys prior to writing batch ([#629](#629)) ([505709e](505709e)), closes [#627](#627)


### Bug Fixes

* Ignore env variables in comments ([#625](#625)) ([08bace8](08bace8))
* Only call `newExecutionClient` if needed ([#630](#630)) ([ece947f](ece947f))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
    @cq-bot
    cq-bot authored Jan 26, 2023
    1 Configuration menu
    Copy the full SHA
    521d78c View commit details
    Browse the repository at this point in the history
Loading