Skip to content

Fix memory tracking for input_format_parallel_parsing#12672

Merged
alexey-milovidov merged 5 commits intoClickHouse:masterfrom
azat:input_format_parallel_parsing-memory-tracking-fix
Jul 24, 2020
Merged

Fix memory tracking for input_format_parallel_parsing#12672
alexey-milovidov merged 5 commits intoClickHouse:masterfrom
azat:input_format_parallel_parsing-memory-tracking-fix

Conversation

@azat
Copy link
Copy Markdown
Member

@azat azat commented Jul 22, 2020

Changelog category (leave one):

  • Bug Fix

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Fix memory tracking for input_format_parallel_parsing (by attaching thread to group)

This PR includes:

  • small cleanup (moves some bits into module part) - separate patch
  • attach/detach thread to group - separate patch
  • integration test (since this is the only test that can cover this, because it can be reproduced only with limit for total memory tracking)

Fixes: #12583
input_format_parallel_parsing introduced in: #6553

Details

HEAD:

  • 65e0dab3dac1f466864e0d849d7ac1fa7fff05ba

@robot-clickhouse robot-clickhouse added the pr-bugfix Pull request with bugfix, not backported by default label Jul 22, 2020
@azat
Copy link
Copy Markdown
Member Author

azat commented Jul 22, 2020

Functional stateless tests (address) — fail: 20

Fails in upstream too

@azat
Copy link
Copy Markdown
Member Author

azat commented Jul 22, 2020

Will rebase against pre #12418

@azat azat force-pushed the input_format_parallel_parsing-memory-tracking-fix branch from 65e0dab to 91c1e2f Compare July 22, 2020 22:04
@azat
Copy link
Copy Markdown
Member Author

azat commented Jul 22, 2020

Will rebase against pre #12418

This won't help I guess, since the PR is applied on top of the upstream/master, so first the original issue should be addressed

@azat azat force-pushed the input_format_parallel_parsing-memory-tracking-fix branch from 91c1e2f to a5885b4 Compare July 23, 2020 08:31
@alexey-milovidov alexey-milovidov self-assigned this Jul 23, 2020
@alexey-milovidov
Copy link
Copy Markdown
Member

LGTM, but what's about integration test?

@azat
Copy link
Copy Markdown
Member Author

azat commented Jul 23, 2020

LGTM, but what's about integration test?

Does related, fixed (pops up after removing std::bind)

@alexey-milovidov alexey-milovidov merged commit 54a021b into ClickHouse:master Jul 24, 2020
@azat azat deleted the input_format_parallel_parsing-memory-tracking-fix branch July 24, 2020 17:47
robot-clickhouse pushed a commit that referenced this pull request Jul 27, 2020
robot-clickhouse pushed a commit that referenced this pull request Jul 27, 2020
robot-clickhouse pushed a commit that referenced this pull request Jul 27, 2020
alexey-milovidov added a commit that referenced this pull request Jul 27, 2020
Backport #12672 to 20.5: Fix memory tracking for input_format_parallel_parsing
alexey-milovidov added a commit that referenced this pull request Jul 27, 2020
Backport #12672 to 20.6: Fix memory tracking for input_format_parallel_parsing
alexey-milovidov added a commit that referenced this pull request Jul 27, 2020
Backport #12672 to 20.4: Fix memory tracking for input_format_parallel_parsing
robot-clickhouse pushed a commit that referenced this pull request Jul 27, 2020
alexey-milovidov added a commit that referenced this pull request Jul 27, 2020
Backport #12672 to 20.3: Fix memory tracking for input_format_parallel_parsing
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-bugfix Pull request with bugfix, not backported by default

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Seesawing in MemoryTracking metric causing >20.3 to fail some writes and reads, unable to identify cause.

3 participants