Skip to content

Conversation

@marianogappa
Copy link
Contributor

@marianogappa marianogappa commented Aug 8, 2024

StateClients currently don't work in docker-based plugins because they cannot connect to destination sockets.

This CLI change detects that a source has:

  • the Docker registry setting
  • the backend_options set

And in this case it makes the following config changes:

  • Forces destinations to use TCP instead of sockets
  • Adds the host.docker.internal extra host to the docker create configuration
  • Sniffs the source's variables and replaces instances of localhost with this special hostname

As a result, normal syncs continue to work exactly the same as before, but syncs that suffered from this config problem now work seamlessly with the new settings:

Screenshot 2024-08-08 at 16 33 32
Screenshot 2024-08-08 at 16 36 32

(Note that I'm using an unreleased pb-go replace)

@marianogappa marianogappa requested review from a team and erezrokah and removed request for a team August 8, 2024 15:41
@cq-bot cq-bot added the area/cli label Aug 8, 2024
cli/cmd/sync.go Outdated
managedplugin.WithTeamName(teamName),
managedplugin.WithLicenseFile(licenseFile),
}
if isThereADockerSourcePluginWithBackend {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we do this only for destinations that are used as state?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also a follow PR can be to update the logic to consider if the OS is Windows per #18039

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made the following changes:

  • UseTCP is more strict; only enabling it when the destination is used as state
  • I propagated the state that a source shouldReplaceLocalhost all the way to ReplaceVariables. Before, it was replacing if a source had a RegistryDocker, which is not ideal because it doesn't necessarily need to use state. I don't think it's a bug because if it doesn't use a backend it probably doesn't need any replacing, but it's more correct now.
  • Added tests where possible.

Regarding the Windows edge case, I'd fix it separately (in another PR, as you say)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I re-ran the syncs in the OP; they still work)

@marianogappa marianogappa requested a review from erezrokah August 9, 2024 10:00
@marianogappa marianogappa added the automerge Automatically merge once required checks pass label Aug 9, 2024
@kodiakhq kodiakhq bot merged commit 2d542da into main Aug 9, 2024
@kodiakhq kodiakhq bot deleted the mariano/enable-stateclient-under-docker branch August 9, 2024 13:51
kodiakhq bot pushed a commit that referenced this pull request Aug 13, 2024
🤖 I have created a release *beep* *boop*
---


## [6.4.0](cli-v6.3.0...cli-v6.4.0) (2024-08-13)


### Features

* Enable StateClient usage under docker plugins. ([#18880](#18880)) ([2d542da](2d542da))
* Enable user to override `invocation_id` ([#18878](#18878)) ([3af7f5e](3af7f5e))


### Bug Fixes

* **deps:** Update module github.com/cloudquery/cloudquery-api-go to v1.12.7 ([#18865](#18865)) ([32a17d8](32a17d8))
* **deps:** Update module github.com/cloudquery/plugin-sdk/v4 to v4.59.0 ([#18881](#18881)) ([8f7667f](8f7667f))
* **deps:** Update module github.com/cloudquery/plugin-sdk/v4 to v4.60.0 ([#18922](#18922)) ([7626636](7626636))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/cli automerge Automatically merge once required checks pass

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants