Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: cloudquery/filetypes
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v3.1.6
Choose a base ref
...
head repository: cloudquery/filetypes
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v3.2.0
Choose a head ref
  • 7 commits
  • 18 files changed
  • 3 contributors

Commits on Jun 7, 2023

  1. feat: Separate file parts (#154)

    This is 100% compatible with the 'old ways' because it still [implements](https://github.com/cloudquery/filetypes/pull/154/files#diff-0965f1fa88ac40fcfc62a7eb27c953cf287c8d9a10e80f5d6e84aeb165322594) `WriteTableBatchFile` with unchanged signature.
    
    
    `Read` is not refactored due to the interface differences between parquet and the other types.
    disq authored Jun 7, 2023
    Configuration menu
    Copy the full SHA
    3798fab View commit details
    Browse the repository at this point in the history

Commits on Jun 8, 2023

  1. Configuration menu
    Copy the full SHA
    a25f427 View commit details
    Browse the repository at this point in the history

Commits on Jun 13, 2023

  1. fix(deps): Update module github.com/cloudquery/plugin-sdk/v3 to v3.10…

    ….5 (#201)
    
    This PR contains the following updates:
    
    | Package | Type | Update | Change |
    |---|---|---|---|
    | [github.com/cloudquery/plugin-sdk/v3](https://togithub.com/cloudquery/plugin-sdk) | require | patch | `v3.10.4` -> `v3.10.5` |
    
    ---
    
    ### Release Notes
    
    <details>
    <summary>cloudquery/plugin-sdk</summary>
    
    ### [`v3.10.5`](https://togithub.com/cloudquery/plugin-sdk/releases/tag/v3.10.5)
    
    [Compare Source](https://togithub.com/cloudquery/plugin-sdk/compare/v3.10.4...v3.10.5)
    
    ##### Bug Fixes
    
    -   **deps:** Update github.com/cloudquery/arrow/go/v13 digest to [`0f7bd3b`](https://togithub.com/cloudquery/plugin-sdk/commit/0f7bd3b) ([#&#8203;961](https://togithub.com/cloudquery/plugin-sdk/issues/961)) ([21f3b68](https://togithub.com/cloudquery/plugin-sdk/commit/21f3b68d45d79e9e726cbe395044a3560145003d))
    -   **deps:** Update github.com/cloudquery/arrow/go/v13 digest to [`6b7fa9c`](https://togithub.com/cloudquery/plugin-sdk/commit/6b7fa9c) ([#&#8203;962](https://togithub.com/cloudquery/plugin-sdk/issues/962)) ([78eecf2](https://togithub.com/cloudquery/plugin-sdk/commit/78eecf2f4cf4027c7c37c8297ff7845debf49fd8))
    -   **deps:** Update github.com/cloudquery/arrow/go/v13 digest to [`71dfe94`](https://togithub.com/cloudquery/plugin-sdk/commit/71dfe94) ([#&#8203;953](https://togithub.com/cloudquery/plugin-sdk/issues/953)) ([b48ae1a](https://togithub.com/cloudquery/plugin-sdk/commit/b48ae1a546a5f4e73e88793da9afda3b91f3ba08))
    -   **deps:** Update github.com/cloudquery/arrow/go/v13 digest to [`7f6aaff`](https://togithub.com/cloudquery/plugin-sdk/commit/7f6aaff) ([#&#8203;963](https://togithub.com/cloudquery/plugin-sdk/issues/963)) ([8c7acdd](https://togithub.com/cloudquery/plugin-sdk/commit/8c7acdd63318bdc270b6fd1141db14148f7ba68c))
    -   **deps:** Update github.com/cloudquery/arrow/go/v13 digest to [`8f72077`](https://togithub.com/cloudquery/plugin-sdk/commit/8f72077) ([#&#8203;958](https://togithub.com/cloudquery/plugin-sdk/issues/958)) ([6f6c993](https://togithub.com/cloudquery/plugin-sdk/commit/6f6c9936e24f9460c253a297df44415dd4eef64f))
    -   **deps:** Update github.com/cloudquery/arrow/go/v13 digest to [`90670b8`](https://togithub.com/cloudquery/plugin-sdk/commit/90670b8) ([#&#8203;955](https://togithub.com/cloudquery/plugin-sdk/issues/955)) ([047ab30](https://togithub.com/cloudquery/plugin-sdk/commit/047ab3066139a00ad665b8ff766bd7f57e70803f))
    -   **deps:** Update github.com/cloudquery/arrow/go/v13 digest to [`b359e74`](https://togithub.com/cloudquery/plugin-sdk/commit/b359e74) ([#&#8203;960](https://togithub.com/cloudquery/plugin-sdk/issues/960)) ([7e95e7d](https://togithub.com/cloudquery/plugin-sdk/commit/7e95e7dbadd1b12772eadc8607f267310ca5583e))
    -   **deps:** Update github.com/cloudquery/arrow/go/v13 digest to [`d8eacf8`](https://togithub.com/cloudquery/plugin-sdk/commit/d8eacf8) ([#&#8203;966](https://togithub.com/cloudquery/plugin-sdk/issues/966)) ([2d32679](https://togithub.com/cloudquery/plugin-sdk/commit/2d3267979a6c66c7fd89a284e2d369b10338a7af))
    -   **deps:** Update github.com/cloudquery/arrow/go/v13 digest to [`e258cfb`](https://togithub.com/cloudquery/plugin-sdk/commit/e258cfb) ([#&#8203;957](https://togithub.com/cloudquery/plugin-sdk/issues/957)) ([df842e0](https://togithub.com/cloudquery/plugin-sdk/commit/df842e01437f51b835f83683001c4fb15fc36b7a))
    -   **transformers:** Ability to transform `any` with TypeTransformer ([#&#8203;956](https://togithub.com/cloudquery/plugin-sdk/issues/956)) ([c989c28](https://togithub.com/cloudquery/plugin-sdk/commit/c989c288ab2fa6f34f6dd71ed7a8fc4597db085e))
    
    </details>
    
    ---
    
    ### Configuration
    
    📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
    
    🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.
    
    ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
    
    🔕 **Ignore**: Close this PR and you won't be reminded about this update again.
    
    ---
    
     - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box
    
    ---
    
    This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate).
    <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS4xMTUuMiIsInVwZGF0ZWRJblZlciI6IjM1LjExNS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->
    cq-bot authored Jun 13, 2023
    Configuration menu
    Copy the full SHA
    0c8519c View commit details
    Browse the repository at this point in the history
  2. fix(deps): Update module github.com/cloudquery/plugin-sdk/v3 to v3.10…

    ….6 (#202)
    
    This PR contains the following updates:
    
    | Package | Type | Update | Change |
    |---|---|---|---|
    | [github.com/cloudquery/plugin-sdk/v3](https://togithub.com/cloudquery/plugin-sdk) | require | patch | `v3.10.5` -> `v3.10.6` |
    
    ---
    
    ### Release Notes
    
    <details>
    <summary>cloudquery/plugin-sdk</summary>
    
    ### [`v3.10.6`](https://togithub.com/cloudquery/plugin-sdk/releases/tag/v3.10.6)
    
    [Compare Source](https://togithub.com/cloudquery/plugin-sdk/compare/v3.10.5...v3.10.6)
    
    ##### Bug Fixes
    
    -   Don't write last batch in managed writer if the context was canceled ([#&#8203;964](https://togithub.com/cloudquery/plugin-sdk/issues/964)) ([8027e62](https://togithub.com/cloudquery/plugin-sdk/commit/8027e62b66c0acf799d795a479e072f86a6dc205))
    
    </details>
    
    ---
    
    ### Configuration
    
    📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
    
    🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.
    
    ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
    
    🔕 **Ignore**: Close this PR and you won't be reminded about this update again.
    
    ---
    
     - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box
    
    ---
    
    This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate).
    <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS4xMTUuMiIsInVwZGF0ZWRJblZlciI6IjM1LjExNS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->
    cq-bot authored Jun 13, 2023
    Configuration menu
    Copy the full SHA
    a94a865 View commit details
    Browse the repository at this point in the history

Commits on Jun 14, 2023

  1. fix: Use buffered writing mode in parquet (#203)

    Closes #137 
    
    #### Benchmarks
    Performed by the following command in `parquet` dir:
    ```sh
    go test \
      -test.run=BenchmarkWrite \
      -test.bench=BenchmarkWrite \
    -test.count 10 -test.benchmem -test.benchtime 10000x
    ```
    
    <details><summary>Before this update</summary>
    
    ```
    goos: darwin
    goarch: arm64
    pkg: github.com/cloudquery/filetypes/v3/parquet
    BenchmarkWrite-10          10000           4628263 ns/op         5796480 B/op      44245 allocs/op
    BenchmarkWrite-10          10000           4480788 ns/op         5796474 B/op      44245 allocs/op
    BenchmarkWrite-10          10000           4591783 ns/op         5796471 B/op      44245 allocs/op
    BenchmarkWrite-10          10000           4610580 ns/op         5796477 B/op      44245 allocs/op
    BenchmarkWrite-10          10000           4524806 ns/op         5796473 B/op      44245 allocs/op
    BenchmarkWrite-10          10000           4557667 ns/op         5796466 B/op      44245 allocs/op
    BenchmarkWrite-10          10000           4869530 ns/op         5796476 B/op      44245 allocs/op
    BenchmarkWrite-10          10000           4894571 ns/op         5796474 B/op      44245 allocs/op
    BenchmarkWrite-10          10000           4700499 ns/op         5796468 B/op      44245 allocs/op
    BenchmarkWrite-10          10000           4793868 ns/op         5796473 B/op      44245 allocs/op
    PASS
    ok      github.com/cloudquery/filetypes/v3/parquet      539.889s
    ```
    
    </details> 
    
    <details><summary>After this update</summary>
    
    ```
    goos: darwin
    goarch: arm64
    pkg: github.com/cloudquery/filetypes/v3/parquet
    BenchmarkWrite-10          10000            923740 ns/op         1146573 B/op      15695 allocs/op
    BenchmarkWrite-10          10000            970047 ns/op         1146193 B/op      15695 allocs/op
    BenchmarkWrite-10          10000            920979 ns/op         1146542 B/op      15695 allocs/op
    BenchmarkWrite-10          10000            923738 ns/op         1146486 B/op      15695 allocs/op
    BenchmarkWrite-10          10000            918581 ns/op         1146055 B/op      15694 allocs/op
    BenchmarkWrite-10          10000            906547 ns/op         1146690 B/op      15695 allocs/op
    BenchmarkWrite-10          10000            912946 ns/op         1146381 B/op      15695 allocs/op
    BenchmarkWrite-10          10000            921024 ns/op         1146378 B/op      15695 allocs/op
    BenchmarkWrite-10          10000            905637 ns/op         1146371 B/op      15695 allocs/op
    BenchmarkWrite-10          10000            919410 ns/op         1146494 B/op      15695 allocs/op
    PASS
    ok      github.com/cloudquery/filetypes/v3/parquet      158.831s
    ```
    
    </details>
    candiduslynx authored Jun 14, 2023
    Configuration menu
    Copy the full SHA
    5ad67f3 View commit details
    Browse the repository at this point in the history
  2. fix(deps): Update github.com/cloudquery/arrow/go/v13 digest to 4d76231 (

    #198)
    
    This PR contains the following updates:
    
    | Package | Type | Update | Change |
    |---|---|---|---|
    | [github.com/cloudquery/arrow/go/v13](https://togithub.com/cloudquery/arrow) | replace | digest | `88d5dc2` -> `4d76231` |
    
    ---
    
    ### Configuration
    
    📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
    
    🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.
    
    ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
    
    🔕 **Ignore**: Close this PR and you won't be reminded about this update again.
    
    ---
    
     - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box
    
    ---
    
    This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate).
    <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS4xMTIuMSIsInVwZGF0ZWRJblZlciI6IjM1LjExNy4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->
    cq-bot authored Jun 14, 2023
    Configuration menu
    Copy the full SHA
    aafa961 View commit details
    Browse the repository at this point in the history
  3. chore(main): Release v3.2.0 (#199)

    🤖 I have created a release *beep* *boop*
    ---
    
    
    ## [3.2.0](v3.1.6...v3.2.0) (2023-06-14)
    
    
    ### Features
    
    * Separate file parts ([#154](#154)) ([3798fab](3798fab))
    
    
    ### Bug Fixes
    
    * **deps:** Update github.com/cloudquery/arrow/go/v13 digest to 4d76231 ([#198](#198)) ([aafa961](aafa961))
    * **deps:** Update module github.com/cloudquery/plugin-sdk/v3 to v3.10.5 ([#201](#201)) ([0c8519c](0c8519c))
    * **deps:** Update module github.com/cloudquery/plugin-sdk/v3 to v3.10.6 ([#202](#202)) ([a94a865](a94a865))
    * Use buffered writing mode in parquet ([#203](#203)) ([5ad67f3](5ad67f3))
    
    ---
    This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
    cq-bot authored Jun 14, 2023
    Configuration menu
    Copy the full SHA
    032b258 View commit details
    Browse the repository at this point in the history
Loading