Skip to content

fix: Fix date parsing in Credential Report CSVs (v2)#1902

Merged
kodiakhq[bot] merged 7 commits intomainfrom
fix-credential-reports
Sep 21, 2022
Merged

fix: Fix date parsing in Credential Report CSVs (v2)#1902
kodiakhq[bot] merged 7 commits intomainfrom
fix-credential-reports

Conversation

@hermanschaaf
Copy link
Copy Markdown
Contributor

This fixes a couple of issues with credential reports:

  • Credential Report CSVs can have values like N/A and not_supported, which previously failed to parse as dates, but now will result in nil columns
  • Some columns were strings when they should have been dates

This was quite hard to fix, as it required creation of a custom type to the unmarshal CSV date, as well as a custom resolver to convert this to a normal timestamp. I also added a custom test to test different scenarios, including one where we expect certain columns to be nil, but want to check that date parsing still works.

@hermanschaaf hermanschaaf requested review from a team and bbernays and removed request for a team September 20, 2022 15:44
@hermanschaaf hermanschaaf requested a review from disq September 20, 2022 15:46
Copy link
Copy Markdown
Member

@disq disq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@hermanschaaf hermanschaaf added the automerge Automatically merge once required checks pass label Sep 20, 2022
@kodiakhq kodiakhq bot merged commit 00f71ed into main Sep 21, 2022
@kodiakhq kodiakhq bot deleted the fix-credential-reports branch September 21, 2022 08:19
kodiakhq bot pushed a commit that referenced this pull request Sep 21, 2022
🤖 I have created a release *beep* *boop*
---


## [0.14.0-pre.0](plugins/source/aws-v0.13.24-pre.0...plugins/source/aws/v0.14.0-pre.0) (2022-09-21)


### ⚠ BREAKING CHANGES

* Migrate AWS plugin to v2 (#1774)

### Features

* Add AWS SSM Parameters resource ([#1222](#1222)) ([3fde704](3fde704))
* Add cq-gen config for apigateway ([#1541](#1541)) ([15bb1a6](15bb1a6))
* Add cq-gen hcl and a new field to sqs queues ([#1453](#1453)) ([a9584fd](a9584fd))
* Add elasticache resources ([#1327](#1327)) ([1e5ef30](1e5ef30))
* Add fsx data repo associations ([#1280](#1280)) ([8b02ce3](8b02ce3))
* Add fsx data repo tasks ([#1279](#1279)) ([e5774fa](e5774fa))
* Add fsx filesystems ([#1277](#1277)) ([2c46e9e](2c46e9e))
* Add fsx snapshots ([#1278](#1278)) ([750d878](750d878))
* Add fsx storage virtual machines ([#1296](#1296)) ([b4f335d](b4f335d))
* Add fsx volumes ([#1322](#1322)) ([9031692](9031692))
* Add IAM policy tags ([#1433](#1433)) ([70d8365](70d8365))
* Add tags for Glue Databases ([#1326](#1326)) ([2e083e7](2e083e7))
* Add transfer servers ([#1284](#1284)) ([a3bf2bf](a3bf2bf))
* Add website, docs and blog to our main repo ([#1159](#1159)) ([dd69948](dd69948))
* AWS v2: Comment out max_retries and max_backoff in example config ([#1836](#1836)) ([388b7ec](388b7ec))
* AWS v2: Split users and credential report users into separate resources ([#1835](#1835)) ([d5b772e](d5b772e))
* Extend sns subscription data ([#1424](#1424)) ([63887e7](63887e7))
* Implement EC2 Key Pairs ([#1403](#1403)) ([#1325](#1325)) ([b9d5b74](b9d5b74))
* Migrate AWS plugin to v2 ([#1774](#1774)) ([ab4483e](ab4483e))
* Remove global region ([#1883](#1883)) ([99e7089](99e7089))


### Bug Fixes

* **aws:** Nil pointer in config unmarshal ([#1859](#1859)) ([64ea261](64ea261))
* **deps:** Update github.com/gocarina/gocsv digest to 71f3a5c ([#1660](#1660)) ([cf26fcc](cf26fcc))
* **deps:** Update golang.org/x/sync digest to 7fc1605 ([#1652](#1652)) ([daafae1](daafae1))
* **deps:** Update module github.com/cloudquery/cq-gen to v0.0.10 ([#1474](#1474)) ([b142e13](b142e13))
* **deps:** Update module github.com/cloudquery/cq-gen to v0.0.11 ([#1491](#1491)) ([5140bef](5140bef))
* **deps:** Update module github.com/cloudquery/cq-gen to v0.0.12 ([#1503](#1503)) ([a740719](a740719))
* **deps:** Update module github.com/cloudquery/cq-gen to v0.0.9 ([#1286](#1286)) ([67ac422](67ac422))
* **deps:** Update module github.com/cloudquery/plugin-sdk to v0.6.2 ([#1838](#1838)) ([5b16c59](5b16c59))
* **deps:** Update module github.com/cloudquery/plugin-sdk to v0.6.3 ([#1858](#1858)) ([9e3ace7](9e3ace7))
* **deps:** Update module github.com/cloudquery/plugin-sdk to v0.6.4 ([#1862](#1862)) ([5d141cf](5d141cf))
* **deps:** Update module github.com/cloudquery/plugin-sdk to v0.7.1 ([#1865](#1865)) ([474bb70](474bb70))
* **deps:** Update module github.com/cloudquery/plugin-sdk to v0.7.2 ([#1872](#1872)) ([49ed26d](49ed26d))
* **deps:** Update module github.com/cloudquery/plugin-sdk to v0.7.3 ([#1886](#1886)) ([7435d59](7435d59))
* **deps:** Update module github.com/cloudquery/plugin-sdk to v0.7.4 ([#1889](#1889)) ([63a5362](63a5362))
* Docs and trigger release ([2028f79](2028f79))
* **ec2:** Add ARN to key pair ([#1339](#1339)) ([189e2c0](189e2c0))
* ECS Tags ([#1515](#1515)) ([e076217](e076217))
* Fix date parsing in Credential Report CSVs (v2) ([#1902](#1902)) ([00f71ed](00f71ed))
* Handle SNS Subscriptions in Pending State ([#1705](#1705)) ([6412087](6412087))
* IAM roles ([#1566](#1566)) ([09ef545](09ef545))
* IAM Users ([#1567](#1567)) ([1e319da](1e319da))
* IAM users PK should be account_id + id, not ARN ([#1876](#1876)) ([20096aa](20096aa))
* Regenerate iam users + some fixes ([#1888](#1888)) ([0361ff7](0361ff7))
* Small fixes to example.yml and typos in cloudtrail errors ([#1877](#1877)) ([c2754fa](c2754fa))
* Tags in users.go ([#1708](#1708)) ([ce34eed](ce34eed))
* Update endpoints ([#1273](#1273)) ([186a840](186a840))
* Update endpoints ([#1432](#1432)) ([4a3c861](4a3c861))
* Update endpoints ([#1514](#1514)) ([eafea83](eafea83))
* Update endpoints ([#1539](#1539)) ([d63fc1d](d63fc1d))
* Update endpoints ([#1563](#1563)) ([373ae23](373ae23))
* Update endpoints ([#1607](#1607)) ([29e5910](29e5910))
* Update endpoints ([#1627](#1627)) ([1ec2fef](1ec2fef))
* Update endpoints ([#1703](#1703)) ([b001114](b001114))
* Update endpoints ([#1709](#1709)) ([739c188](739c188))
* Update endpoints ([#1830](#1830)) ([bf05794](bf05794))
* Update endpoints ([#1880](#1880)) ([a64f5aa](a64f5aa))
* Update endpoints ([#1904](#1904)) ([af84989](af84989))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automerge Automatically merge once required checks pass

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants