Skip to content

fix(destinations): Log correct size of batch#588

Merged
kodiakhq[bot] merged 2 commits intocloudquery:mainfrom
erezrokah:fix/log_correct_number_of_batch
Jan 9, 2023
Merged

fix(destinations): Log correct size of batch#588
kodiakhq[bot] merged 2 commits intocloudquery:mainfrom
erezrokah:fix/log_correct_number_of_batch

Conversation

@erezrokah
Copy link
Copy Markdown
Member

@erezrokah erezrokah commented Jan 9, 2023

Summary

If we exceed BatchSizeBytes the batch size can be lower than the one in the spec


Use the following steps to ensure your PR is ready to be reviewed

  • Read the contribution guidelines 🧑‍🎓
  • Run go fmt to format your code 🖊
  • Lint your changes via golangci-lint run 🚨 (install golangci-lint here)
  • Update or add tests 🧪
  • Ensure the status checks below are successful ✅

@github-actions github-actions bot added fix and removed fix labels Jan 9, 2023
@github-actions
Copy link
Copy Markdown

github-actions bot commented Jan 9, 2023

⏱️ Benchmark results

Comparing with 3b07885

  • DefaultConcurrencyDFS-2 resources/s: 11,475 ⬆️ 6.00% increase vs. 3b07885
  • DefaultConcurrencyRoundRobin-2 resources/s: 11,068 ⬇️ 11.28% decrease vs. 3b07885
  • Glob-2 ns/op: 145.4 ⬇️ 35.35% decrease vs. 3b07885
  • TablesWithChildrenDFS-2 resources/s: 28,858 ⬆️ 3.83% increase vs. 3b07885
  • TablesWithChildrenRoundRobin-2 resources/s: 31,571 ⬆️ 11.49% increase vs. 3b07885
  • TablesWithRateLimitingDFS-2 resources/s: 28.34 ⬆️ 0.25% increase vs. 3b07885
  • TablesWithRateLimitingRoundRobin-2 resources/s: 823.8 ⬆️ 0.69% increase vs. 3b07885
  • BufferedScanner-2 ns/op: 10.11 ⬇️ 20.47% decrease vs. 3b07885
  • LogReader-2 ns/op: 30.59 ⬇️ 21.35% decrease vs. 3b07885

@kodiakhq kodiakhq bot merged commit 9cebafe into cloudquery:main Jan 9, 2023
kodiakhq bot pushed a commit that referenced this pull request Jan 9, 2023
🤖 I have created a release *beep* *boop*
---


## [1.24.0](v1.23.0...v1.24.0) (2023-01-09)


### Features

* Add local backend for storing cursor state ([#569](#569)) ([3b07885](3b07885))
* Remove codegen ([#589](#589)) ([1c5943a](1c5943a))


### Bug Fixes

* **destinations:** Log correct size of batch ([#588](#588)) ([9cebafe](9cebafe))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants