Skip to content

Fix some memory/storage issues in CI#15795

Merged
Mytherin merged 1 commit intoduckdb:v1.2-histrionicusfrom
lnkuiper:test_fixes
Jan 20, 2025
Merged

Fix some memory/storage issues in CI#15795
Mytherin merged 1 commit intoduckdb:v1.2-histrionicusfrom
lnkuiper:test_fixes

Conversation

@lnkuiper
Copy link
Collaborator

These tests

test/sql/json/test_json_copy_tpch.test_slow -- 1
test/sql/aggregate/group/test_group_by_parallel.test_slow -- 2
test/sql/parallelism/intraquery/depth_first_evaluation.test_slow -- 3

Were giving memory/storage issues in CI.

For (1.), I've reduced the TPC-H scale factor to 0.1 instead of 1. We're still testing what we want to test, so this shouldn't be an issue (TPC-H files are really large when exported to JSON).

For (2.), the setting PRAGMA verify_external; was triggering a lot of partitioning and spilling to test the external grouped aggregation behavior. It was spilling more than necessary because the HT size calculation was off a bit, especially when this setting was enabled.

For (3.), a slight increase to the memory limit in the test was enough to make it work again. We're still testing the behavior we want to.

@lnkuiper lnkuiper changed the base branch from main to v1.2-histrionicus January 20, 2025 11:53
@Mytherin Mytherin merged commit 5d49c57 into duckdb:v1.2-histrionicus Jan 20, 2025
46 checks passed
@Mytherin
Copy link
Collaborator

Thanks!

@lnkuiper lnkuiper deleted the test_fixes branch April 14, 2025 09:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants