Skip to content

fix(destinations): Set done even if no resources to flush#537

Merged
kodiakhq[bot] merged 1 commit intocloudquery:mainfrom
erezrokah:fix/deadlock_managed_writer
Dec 28, 2022
Merged

fix(destinations): Set done even if no resources to flush#537
kodiakhq[bot] merged 1 commit intocloudquery:mainfrom
erezrokah:fix/deadlock_managed_writer

Conversation

@erezrokah
Copy link
Copy Markdown
Member

@erezrokah erezrokah commented Dec 28, 2022

Summary

Related to cloudquery/cloudquery#6070. This fixes a deadlock in the managed writer when we wait for the flush to finish indefinitely.
The cause is that the resources can already be flushed by another case in the switch, so there might not be resources to flush when we hit the flush case.
The fix is to always pass done to the channel


Use the following steps to ensure your PR is ready to be reviewed

  • Read the contribution guidelines 🧑‍🎓
  • Run go fmt to format your code 🖊
  • Lint your changes via golangci-lint run 🚨 (install golangci-lint here)
  • Update or add tests 🧪
  • Ensure the status checks below are successful ✅

@github-actions
Copy link
Copy Markdown

⏱️ Benchmark results

Comparing with 5ffad03

  • DefaultConcurrency-2 resources/s: 11,186 ⬇️ 5.27% decrease vs. 5ffad03
  • Glob-2 ns/op: 207.5 ⬆️ 27.90% increase vs. 5ffad03
  • TablesWithChildrenDefaultConcurrency-2 resources/s: 27,703 ⬇️ 10.78% decrease vs. 5ffad03
  • BufferedScanner-2 ns/op: 12.69 ⬆️ 26.32% increase vs. 5ffad03
  • LogReader-2 ns/op: 38.1 ⬆️ 19.90% increase vs. 5ffad03

@github-actions github-actions bot added fix and removed fix labels Dec 28, 2022
@kodiakhq kodiakhq bot merged commit 02eca6d into cloudquery:main Dec 28, 2022
@erezrokah erezrokah deleted the fix/deadlock_managed_writer branch December 28, 2022 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants