Skip to content

Conversation

@gruuya
Copy link
Contributor

@gruuya gruuya commented Aug 2, 2024

Also disable all analyzer and optimizer rules, to avoid the stack overflow issue when applying a large number of sync calls during flushing.

Finally, make DataSyncConfig::max_syncs_per_url default to 100 now; it also works for 200 too but that incurs a performance penalty, so we should strive to get inter-sync dataframe squashing as a follow-up.

@gruuya gruuya requested a review from mildbyte August 2, 2024 08:23
@gruuya gruuya merged commit ccacfe0 into main Aug 2, 2024
@gruuya gruuya deleted the increase-sync-len-threshold branch August 2, 2024 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants