-
Notifications
You must be signed in to change notification settings - Fork 4
Comparing changes
Open a pull request
base repository: cloudquery/filetypes
base: v3.1.6
head repository: cloudquery/filetypes
compare: v3.2.0
- 7 commits
- 18 files changed
- 3 contributors
Commits on Jun 7, 2023
-
feat: Separate file parts (#154)
This is 100% compatible with the 'old ways' because it still [implements](https://github.com/cloudquery/filetypes/pull/154/files#diff-0965f1fa88ac40fcfc62a7eb27c953cf287c8d9a10e80f5d6e84aeb165322594) `WriteTableBatchFile` with unchanged signature. `Read` is not refactored due to the interface differences between parquet and the other types.
Configuration menu - View commit details
-
Copy full SHA for 3798fab - Browse repository at this point
Copy the full SHA 3798fabView commit details
Commits on Jun 8, 2023
-
Configuration menu - View commit details
-
Copy full SHA for a25f427 - Browse repository at this point
Copy the full SHA a25f427View commit details
Commits on Jun 13, 2023
-
fix(deps): Update module github.com/cloudquery/plugin-sdk/v3 to v3.10…
….5 (#201) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [github.com/cloudquery/plugin-sdk/v3](https://togithub.com/cloudquery/plugin-sdk) | require | patch | `v3.10.4` -> `v3.10.5` | --- ### Release Notes <details> <summary>cloudquery/plugin-sdk</summary> ### [`v3.10.5`](https://togithub.com/cloudquery/plugin-sdk/releases/tag/v3.10.5) [Compare Source](https://togithub.com/cloudquery/plugin-sdk/compare/v3.10.4...v3.10.5) ##### Bug Fixes - **deps:** Update github.com/cloudquery/arrow/go/v13 digest to [`0f7bd3b`](https://togithub.com/cloudquery/plugin-sdk/commit/0f7bd3b) ([#​961](https://togithub.com/cloudquery/plugin-sdk/issues/961)) ([21f3b68](https://togithub.com/cloudquery/plugin-sdk/commit/21f3b68d45d79e9e726cbe395044a3560145003d)) - **deps:** Update github.com/cloudquery/arrow/go/v13 digest to [`6b7fa9c`](https://togithub.com/cloudquery/plugin-sdk/commit/6b7fa9c) ([#​962](https://togithub.com/cloudquery/plugin-sdk/issues/962)) ([78eecf2](https://togithub.com/cloudquery/plugin-sdk/commit/78eecf2f4cf4027c7c37c8297ff7845debf49fd8)) - **deps:** Update github.com/cloudquery/arrow/go/v13 digest to [`71dfe94`](https://togithub.com/cloudquery/plugin-sdk/commit/71dfe94) ([#​953](https://togithub.com/cloudquery/plugin-sdk/issues/953)) ([b48ae1a](https://togithub.com/cloudquery/plugin-sdk/commit/b48ae1a546a5f4e73e88793da9afda3b91f3ba08)) - **deps:** Update github.com/cloudquery/arrow/go/v13 digest to [`7f6aaff`](https://togithub.com/cloudquery/plugin-sdk/commit/7f6aaff) ([#​963](https://togithub.com/cloudquery/plugin-sdk/issues/963)) ([8c7acdd](https://togithub.com/cloudquery/plugin-sdk/commit/8c7acdd63318bdc270b6fd1141db14148f7ba68c)) - **deps:** Update github.com/cloudquery/arrow/go/v13 digest to [`8f72077`](https://togithub.com/cloudquery/plugin-sdk/commit/8f72077) ([#​958](https://togithub.com/cloudquery/plugin-sdk/issues/958)) ([6f6c993](https://togithub.com/cloudquery/plugin-sdk/commit/6f6c9936e24f9460c253a297df44415dd4eef64f)) - **deps:** Update github.com/cloudquery/arrow/go/v13 digest to [`90670b8`](https://togithub.com/cloudquery/plugin-sdk/commit/90670b8) ([#​955](https://togithub.com/cloudquery/plugin-sdk/issues/955)) ([047ab30](https://togithub.com/cloudquery/plugin-sdk/commit/047ab3066139a00ad665b8ff766bd7f57e70803f)) - **deps:** Update github.com/cloudquery/arrow/go/v13 digest to [`b359e74`](https://togithub.com/cloudquery/plugin-sdk/commit/b359e74) ([#​960](https://togithub.com/cloudquery/plugin-sdk/issues/960)) ([7e95e7d](https://togithub.com/cloudquery/plugin-sdk/commit/7e95e7dbadd1b12772eadc8607f267310ca5583e)) - **deps:** Update github.com/cloudquery/arrow/go/v13 digest to [`d8eacf8`](https://togithub.com/cloudquery/plugin-sdk/commit/d8eacf8) ([#​966](https://togithub.com/cloudquery/plugin-sdk/issues/966)) ([2d32679](https://togithub.com/cloudquery/plugin-sdk/commit/2d3267979a6c66c7fd89a284e2d369b10338a7af)) - **deps:** Update github.com/cloudquery/arrow/go/v13 digest to [`e258cfb`](https://togithub.com/cloudquery/plugin-sdk/commit/e258cfb) ([#​957](https://togithub.com/cloudquery/plugin-sdk/issues/957)) ([df842e0](https://togithub.com/cloudquery/plugin-sdk/commit/df842e01437f51b835f83683001c4fb15fc36b7a)) - **transformers:** Ability to transform `any` with TypeTransformer ([#​956](https://togithub.com/cloudquery/plugin-sdk/issues/956)) ([c989c28](https://togithub.com/cloudquery/plugin-sdk/commit/c989c288ab2fa6f34f6dd71ed7a8fc4597db085e)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS4xMTUuMiIsInVwZGF0ZWRJblZlciI6IjM1LjExNS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->
Configuration menu - View commit details
-
Copy full SHA for 0c8519c - Browse repository at this point
Copy the full SHA 0c8519cView commit details -
fix(deps): Update module github.com/cloudquery/plugin-sdk/v3 to v3.10…
….6 (#202) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [github.com/cloudquery/plugin-sdk/v3](https://togithub.com/cloudquery/plugin-sdk) | require | patch | `v3.10.5` -> `v3.10.6` | --- ### Release Notes <details> <summary>cloudquery/plugin-sdk</summary> ### [`v3.10.6`](https://togithub.com/cloudquery/plugin-sdk/releases/tag/v3.10.6) [Compare Source](https://togithub.com/cloudquery/plugin-sdk/compare/v3.10.5...v3.10.6) ##### Bug Fixes - Don't write last batch in managed writer if the context was canceled ([#​964](https://togithub.com/cloudquery/plugin-sdk/issues/964)) ([8027e62](https://togithub.com/cloudquery/plugin-sdk/commit/8027e62b66c0acf799d795a479e072f86a6dc205)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS4xMTUuMiIsInVwZGF0ZWRJblZlciI6IjM1LjExNS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->
Configuration menu - View commit details
-
Copy full SHA for a94a865 - Browse repository at this point
Copy the full SHA a94a865View commit details
Commits on Jun 14, 2023
-
fix: Use buffered writing mode in parquet (#203)
Closes #137 #### Benchmarks Performed by the following command in `parquet` dir: ```sh go test \ -test.run=BenchmarkWrite \ -test.bench=BenchmarkWrite \ -test.count 10 -test.benchmem -test.benchtime 10000x ``` <details><summary>Before this update</summary> ``` goos: darwin goarch: arm64 pkg: github.com/cloudquery/filetypes/v3/parquet BenchmarkWrite-10 10000 4628263 ns/op 5796480 B/op 44245 allocs/op BenchmarkWrite-10 10000 4480788 ns/op 5796474 B/op 44245 allocs/op BenchmarkWrite-10 10000 4591783 ns/op 5796471 B/op 44245 allocs/op BenchmarkWrite-10 10000 4610580 ns/op 5796477 B/op 44245 allocs/op BenchmarkWrite-10 10000 4524806 ns/op 5796473 B/op 44245 allocs/op BenchmarkWrite-10 10000 4557667 ns/op 5796466 B/op 44245 allocs/op BenchmarkWrite-10 10000 4869530 ns/op 5796476 B/op 44245 allocs/op BenchmarkWrite-10 10000 4894571 ns/op 5796474 B/op 44245 allocs/op BenchmarkWrite-10 10000 4700499 ns/op 5796468 B/op 44245 allocs/op BenchmarkWrite-10 10000 4793868 ns/op 5796473 B/op 44245 allocs/op PASS ok github.com/cloudquery/filetypes/v3/parquet 539.889s ``` </details> <details><summary>After this update</summary> ``` goos: darwin goarch: arm64 pkg: github.com/cloudquery/filetypes/v3/parquet BenchmarkWrite-10 10000 923740 ns/op 1146573 B/op 15695 allocs/op BenchmarkWrite-10 10000 970047 ns/op 1146193 B/op 15695 allocs/op BenchmarkWrite-10 10000 920979 ns/op 1146542 B/op 15695 allocs/op BenchmarkWrite-10 10000 923738 ns/op 1146486 B/op 15695 allocs/op BenchmarkWrite-10 10000 918581 ns/op 1146055 B/op 15694 allocs/op BenchmarkWrite-10 10000 906547 ns/op 1146690 B/op 15695 allocs/op BenchmarkWrite-10 10000 912946 ns/op 1146381 B/op 15695 allocs/op BenchmarkWrite-10 10000 921024 ns/op 1146378 B/op 15695 allocs/op BenchmarkWrite-10 10000 905637 ns/op 1146371 B/op 15695 allocs/op BenchmarkWrite-10 10000 919410 ns/op 1146494 B/op 15695 allocs/op PASS ok github.com/cloudquery/filetypes/v3/parquet 158.831s ``` </details>
Configuration menu - View commit details
-
Copy full SHA for 5ad67f3 - Browse repository at this point
Copy the full SHA 5ad67f3View commit details -
fix(deps): Update github.com/cloudquery/arrow/go/v13 digest to 4d76231 (
#198) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [github.com/cloudquery/arrow/go/v13](https://togithub.com/cloudquery/arrow) | replace | digest | `88d5dc2` -> `4d76231` | --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS4xMTIuMSIsInVwZGF0ZWRJblZlciI6IjM1LjExNy4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->
Configuration menu - View commit details
-
Copy full SHA for aafa961 - Browse repository at this point
Copy the full SHA aafa961View commit details -
chore(main): Release v3.2.0 (#199)
🤖 I have created a release *beep* *boop* --- ## [3.2.0](v3.1.6...v3.2.0) (2023-06-14) ### Features * Separate file parts ([#154](#154)) ([3798fab](3798fab)) ### Bug Fixes * **deps:** Update github.com/cloudquery/arrow/go/v13 digest to 4d76231 ([#198](#198)) ([aafa961](aafa961)) * **deps:** Update module github.com/cloudquery/plugin-sdk/v3 to v3.10.5 ([#201](#201)) ([0c8519c](0c8519c)) * **deps:** Update module github.com/cloudquery/plugin-sdk/v3 to v3.10.6 ([#202](#202)) ([a94a865](a94a865)) * Use buffered writing mode in parquet ([#203](#203)) ([5ad67f3](5ad67f3)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Configuration menu - View commit details
-
Copy full SHA for 032b258 - Browse repository at this point
Copy the full SHA 032b258View commit details
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff v3.1.6...v3.2.0