Skip to content

fix: Error on incorrect table configuration#237

Merged
kodiakhq[bot] merged 9 commits intomainfrom
error-on-incorrect-tables
Oct 7, 2022
Merged

fix: Error on incorrect table configuration#237
kodiakhq[bot] merged 9 commits intomainfrom
error-on-incorrect-tables

Conversation

@hermanschaaf
Copy link
Copy Markdown
Contributor

@hermanschaaf hermanschaaf commented Oct 4, 2022

This adds checks for a whole range of misconfigurations that could happen in the tables config.

Some of the main cases handled:

  • invalid table name
  • wildcard in skip_tables section
  • child table specified without its parent table
  • list of tables is empty

Closes cloudquery/cloudquery#2312

@github-actions github-actions bot added fix and removed fix labels Oct 4, 2022
@hermanschaaf hermanschaaf changed the title fix: Error on incorrect tables fix: Error on incorrect table configuration Oct 4, 2022
@github-actions github-actions bot added fix and removed fix labels Oct 4, 2022
@hermanschaaf hermanschaaf marked this pull request as ready for review October 4, 2022 11:40
Copy link
Copy Markdown
Contributor

@yevgenypats yevgenypats left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. few nits.

}

// return an error if child table is included, but not its parent table
selectedTables := map[string]bool{}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I think we should just disallow child tables (at least for now until someone will ask for it.)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before we disallow child tables, we should indicate what is and isn't a child table in the docs. (Not hard, I can do this in a separate PR). But let's at least allow it to be specified for now, as long as you also specify the parent, otherwise you'll get some surprising behavior.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, if child table is specified I'll automatically fetch parent tables if we are at it already.


logger.Debug().Interface("tables", tableNames).Msg("got table names")

for _, table := range p.tables {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we already validates tables, let's just use tableName and do range _, table := range tableName so the validation logic is in one place

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tableNames is of type []string, while p.tables is of type schema.Tables. We can't range over tableNames, since need the schema.Table here. We could add a lookup for the schema.Table object? I played around with it, but it doesn't seem better

Copy link
Copy Markdown
Contributor

@yevgenypats yevgenypats left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good

@yevgenypats
Copy link
Copy Markdown
Contributor

@hermanschaaf can you solve the conflict and we can merge?

@kodiakhq kodiakhq bot merged commit 6ad75f5 into main Oct 7, 2022
@kodiakhq kodiakhq bot deleted the error-on-incorrect-tables branch October 7, 2022 07:46
kodiakhq bot pushed a commit that referenced this pull request Oct 7, 2022
🤖 I have created a release *beep* *boop*
---


## [0.12.8](v0.12.7...v0.12.8) (2022-10-07)


### Bug Fixes

* Error on incorrect table configuration ([#237](#237)) ([6ad75f5](6ad75f5))
* Exit gracefully on context cancelled ([#252](#252)) ([b4df92e](b4df92e))
* Progressbar should go into stdout ([#250](#250)) ([b8bcdad](b8bcdad))
* Recover panic in table resolver and object resolver flows ([#257](#257)) ([04dba02](04dba02))
* Stop if PreResourceResolver fails ([#251](#251)) ([ee83f8f](ee83f8f))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: Source config with non-existent tables should report an error or warning

2 participants