Skip to content

feat: Policy Snapshot Testing#449

Merged
bbernays merged 125 commits intocloudquery:mainfrom
bbernays:Test-Runner
Feb 21, 2022
Merged

feat: Policy Snapshot Testing#449
bbernays merged 125 commits intocloudquery:mainfrom
bbernays:Test-Runner

Conversation

@bbernays
Copy link
Copy Markdown
Collaborator

@bbernays bbernays commented Jan 28, 2022

PR Overview:

  • adds new command: cloudquery snapshot <selector> <output-directory>
    example: cloudquery snapshot aws//foundational_security/ec2/EC2.18 database-data

  • adds new command running policy tests:

cloudquery policy test <path/to/policy/dir> <path/to/snapshout/outputdir>

Example invocation:

go run main.go policy test .cq/policies/github.com/cloudquery-policies/aws ./dest

Output:

  • stores 2 files in the following directory structure
    <output-directory>/query-<CONTROL-ID>/tests/<UUID>
    - pg-dump.sql stores all relevant tables and views including data
    - data.json this stores the output of control against the dumped data

Known issues:

  • Panic occurs if DSN is a blank string
  • pg_dump- exports views as tables... This adds complications for migrations in the future as any tooling we might have will operate at the table level rather than the newly derived table from the view
  • Logging statements are missing or wrong
    - When SQL queries returns no data output is \N (postgres syntax for null)

To Do:

  • Parse DSN to replace hard coded credentials
  • Respect 2nd arg that defines snapshot directory for storage

This pr is blocked by this pr: https://github.com/cloudquery/cloudquery/pull/449/files

roneli
roneli previously requested changes Feb 13, 2022
Copy link
Copy Markdown
Contributor

@roneli roneli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some comments around design, the whole PR is good, I think we can move some bits here and there, had a few suggestions.

Copy link
Copy Markdown
Member

@disq disq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few nits

@bbernays bbernays dismissed stale reviews from roneli and yevgenypats February 21, 2022 14:48

All comments addressed (either implemented or deferred)

@bbernays bbernays merged commit c6c24ed into cloudquery:main Feb 21, 2022
@bbernays bbernays deleted the Test-Runner branch February 21, 2022 14:57
TinLe pushed a commit to TinLe/cloudquery that referenced this pull request Feb 22, 2022
* upstream/main:
  feat: Policy Snapshot Testing (cloudquery#449)
  fix: Remove enable_partial_fetch and support for migrationless providers (cloudquery#495)
  fix: Upgrade schema in history mode (cloudquery#494)
  fix: Don't report unmanaged provider diags to sentry (cloudquery#492)
  feat: Modules v2 support (cloudquery#489)
  chore: Synced local '.github/dependabot.yml' with remote 'dependabot.yml' (cloudquery#490)
  chore: Remove '--debug' flag from issue template (cloudquery#488)
  fix: Don't return an error if encountering an empty subpolicy (cloudquery#486)
  fix: Fetch summary SQL state 54000 (cloudquery#487)
erezrokah pushed a commit that referenced this pull request Aug 14, 2022
erezrokah pushed a commit that referenced this pull request Aug 14, 2022
Synced local file(s) with [cloudquery/.github](https://github.com/cloudquery/.github).





---

This PR was created automatically by the [repo-file-sync-action](https://github.com/BetaHuhn/repo-file-sync-action) workflow run [#2774614062](https://github.com/cloudquery/.github/actions/runs/2774614062)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants