Skip to content

feat: Add Metrics and improve scheduler with DFS#318

Merged
yevgenypats merged 13 commits intomainfrom
feat/metrics_and_scheduling
Oct 30, 2022
Merged

feat: Add Metrics and improve scheduler with DFS#318
yevgenypats merged 13 commits intomainfrom
feat/metrics_and_scheduling

Conversation

@yevgenypats
Copy link
Copy Markdown
Contributor

Trying to split this PR: #298 into smaller bits.

Metrics:

  • This introduce Metrics as stateful struct as it should be (instead of prop drilling) + GetMetrics Protobuf API

Scheduler:

  • I believed I've hit a bug/deadlock in our current scheduler and also for metrics to work I needed to move the scheduler from schema to plugins so I added that in the same PR.

Currently the user will specify only one variable concurrency and the scheduler will decide on how to split it between levels. For simplicity I kept it the same way as before with concurrency for only the first level.

Concurrent DFS will make sure there are no deadlocks and memory is always kept at O(goroutines) and o(h) (where h is height).

Copy link
Copy Markdown
Member

@erezrokah erezrokah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good and nothing blocking.

Added a few comments but I don't think any are blocking

EndTime time.Time
}

func (s *TableClientMetrics) Equal(other *TableClientMetrics) bool {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason not to use https://pkg.go.dev/gotest.tools/assert#DeepEqual in tests? Or https://github.com/google/go-cmp?

If we need the Equal signature we can wrap a call to go-cmp

Same for the other Equal.

If the reason is to skip StartTime and EndTime looks like we can do it via google/go-cmp#143 (comment)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DeepEqual is super slow and can have unpredictable results depending on what you are trying to achieve so we want to define equal for every type. DeepEqual is usually used for testing purposes when you don't control or have access to some external struct.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My concern is that someone adds a field to the struct and then forgets to add it to the Equal.
i.e. We'll have to maintain this implementation. Not sure the performance hit is something that can really slow us down.

Where is the Equal function used?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we use it in tests but what it the issue of maintaining this and using a library that is used to compare structs which are not in control of the author?

If you look at SourceMetrics the Equal function is not that simple comparing the map but it ensure it will work and I can also add tests to that to ensure we don't forget to update that.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's only used in the tests I would say we don't need it as a part of the struct and just do the equality in the test (via library or a helper function).

Again this is not blocking for the PR, just seems there's already existing code we can use to compare structs in tests, so we don't need to re-implement it

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to what @erezrokah said; not a blocker, but I'd vote for go-cmp in tests rather than maintain equality operators if we don't need them in our (non-test) code.

yevgenypats and others added 7 commits October 30, 2022 15:21
Co-authored-by: Erez Rokah <erezrokah@users.noreply.github.com>
Co-authored-by: Erez Rokah <erezrokah@users.noreply.github.com>
Co-authored-by: Erez Rokah <erezrokah@users.noreply.github.com>
@yevgenypats yevgenypats merged commit 2d7a83b into main Oct 30, 2022
@yevgenypats yevgenypats deleted the feat/metrics_and_scheduling branch October 30, 2022 14:12
kodiakhq bot pushed a commit that referenced this pull request Oct 30, 2022
🤖 I have created a release *beep* *boop*
---


## [0.13.15](v0.13.14...v0.13.15) (2022-10-30)


### Features

* Add Metrics and improve scheduler with DFS ([#318](#318)) ([2d7a83b](2d7a83b))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
kodiakhq bot pushed a commit to cloudquery/cloudquery that referenced this pull request Oct 30, 2022
yevgenypats added a commit to cloudquery/cloudquery that referenced this pull request Oct 30, 2022
Copy link
Copy Markdown
Contributor

@hermanschaaf hermanschaaf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I read through, and it all looks good to me; the concurrency also looks good, though I'd like to tweak it a bit further once we have benchmarks in place in the coming weeks 👍

Comment on lines +57 to +58
defer wg.Done()
defer p.tableSem.Release(1)
Copy link
Copy Markdown
Contributor

@hermanschaaf hermanschaaf Oct 31, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think the ordering of the defers need to be swapped here to match the order of the Acquire and wg.Add operations (since defers get executed in reverse order)

p.metrics.initWithClients(table, clients)
for _, client := range clients {
client := client
if err := p.tableSem.Acquire(ctx, 1); err != nil {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a passing comment, but since the semaphore is acquired inside the clients loop, technically the tableSem is more like a tableClientSem - so if you have multiple accounts and set table concurrency to 1, only one account will be resolved at a time. Not necessarily a bad thing, but maybe just worth keeping in mind / documenting.

yevgenypats added a commit that referenced this pull request Oct 31, 2022
This add a type system to CloudQuery SDK.

This is mandatory to support multiple destinations.

Also, this found quite a few bugs where we were sending random stuff
over the wire without any validation apart from maybe when we were
hitting the database and then failing batches all together.

In CloudQuery type system I used heavily https://github.com/jackc/pgtype
and kept the license and the copyright in it's own package `cqtype`.

This is a continue of this PR
#298 where I split it into
this one and #318

Co-authored-by: Erez Rokah <erezrokah@users.noreply.github.com>
Co-authored-by: Herman Schaaf <hermanschaaf@gmail.com>
yevgenypats added a commit to cloudquery/cloudquery that referenced this pull request Oct 31, 2022
yevgenypats added a commit to cloudquery/cloudquery that referenced this pull request Oct 31, 2022
yevgenypats added a commit to cloudquery/cloudquery that referenced this pull request Nov 1, 2022
This is instead of #3176 

SDK PRs:

cloudquery/plugin-sdk#318
cloudquery/plugin-sdk#320

Previous related CloudQuery PRs:

#3286

Co-authored-by: Herman Schaaf <hermanschaaf@gmail.com>
Co-authored-by: Erez Rokah <erezrokah@users.noreply.github.com>
yevgenypats added a commit to cloudquery/cloudquery that referenced this pull request Nov 1, 2022
This is instead of #3176 

SDK PRs:

cloudquery/plugin-sdk#318
cloudquery/plugin-sdk#320

Previous related CloudQuery PRs:

#3286

Co-authored-by: Herman Schaaf <hermanschaaf@gmail.com>
Co-authored-by: Erez Rokah <erezrokah@users.noreply.github.com>
daniel-garcia pushed a commit to infobloxopen/ibcq-source-k8s that referenced this pull request Feb 24, 2026
daniel-garcia pushed a commit to infobloxopen/ibcq-source-k8s that referenced this pull request Feb 24, 2026
This is instead of cloudquery/cloudquery#3176 

SDK PRs:

cloudquery/plugin-sdk#318
cloudquery/plugin-sdk#320

Previous related CloudQuery PRs:

cloudquery/cloudquery#3286

Co-authored-by: Herman Schaaf <hermanschaaf@gmail.com>
Co-authored-by: Erez Rokah <erezrokah@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants