Unrevert "Refactor how IInputFormat deals with reading many files at once" by al13n321 · Pull Request #82949 · ClickHouse/ClickHouse

al13n321 · 2025-06-30T19:01:25Z

Changelog category (leave one):

Not for changelog (changelog entry is not required)

#80931 (which was reverted by #82934) + fix for https://github.com/ClickHouse/clickhouse-private/issues/33107 (second commit; the added test reproduces it).

Closes https://github.com/ClickHouse/clickhouse-private/issues/33107

…es at once"" This reverts commit 1f28e8b.

clickhouse-gh · 2025-06-30T19:01:51Z

Workflow [PR], commit [06e0dfe]

Summary: ❌

job_name	test_name	status
Stateless tests (amd_debug, distributed plan, s3 storage)		failure
	02443_detach_attach_partition	FAIL
Stress test (amd_debug)		failure
	Found signal in gdb.log	FAIL

al13n321 · 2025-07-03T17:24:27Z

Test failures look unrlated:

WARNING: ThreadSanitizer: thread leak (pid=3824) #83063
https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=82949&sha=251df787a5b37e6b7473dbb6cf6657c314b520eb&name_0=PR&name_1=AST%20fuzzer%20%28amd_ubsan%29 has a null pointer on field in in Connection::receivePacket(). I guess it was called after disconnect() somehow. I couldn't immediately figure out what normally prevents Connection methods from being called after disconnect().

al13n321 · 2025-07-05T00:34:46Z

Stateless tests:

mc: <ERROR> Unable to list folder. Requested path `/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/clickminio` not found

I don't understand how this PR could possibly cause this. But it happened on two commits in a row, and I don't see it on other PRs.

al13n321 · 2025-07-08T03:46:27Z

AST fuzzer unrelated: Logical Error: LIMIT WITH TIES without ORDER BY #71756
00804_test_alter_compression_codecs unrelated: no-msan in 00804_test_alter_compression_codecs #83421

al13n321 · 2025-07-11T17:17:13Z

Failed 02561_null_as_default_more_formats and 00900_long_parquet seem related, but idk how to investigate. They don't fail on master. In both cases it looks like clickhouse-client INSERT ... FORMAT PARQUET got SIGABRT. ~~We don't have logs or stack traces from the client in tests, so I've no idea which assert (?) it is~~ I don't understand why there's no stack trace in clickhouse-test output, clickhouse-client is supposed to print it to stderr on abort, and clickhouse-test is supposed to dump stderr on test failure. I couldn't reproduce it locally (ran the two tests 600 times in asan build).

al13n321 · 2025-07-11T20:22:26Z

Now fasttest failed with this (repeating till timeout):

2025.07.11 21:28:49.559192 [ 218734 ] {} <Error> DatabaseCatalog: Cannot drop table test_qddg4zoa.no_insertable (00000000-0000-0000-0000-000000000001). Will retry later.: std::exception. Code: 1001, type: std::__1::filesystem::filesystem_error, e.what() = filesystem error: in remove_all: Permission denied ["/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/ci/tmp/run_r0/store/000/00000000-0000-0000-0000-000000000001/"], Stack trace (when copying this message, always include the lines below):

I couldn't find any mention of this error in issues or slack. I don't see how it can be related to this PR.

Is this PR cursed?

al13n321 · 2025-07-12T01:59:39Z

02233_HTTP_ranged unrelated: #82844

…-fast-usage Fix flaky tests with parquet by partially reverting #82949

Fix rebase issue: - 20250728 ClickHouse#84180 - 20250713 ClickHouse#82949 - 20250703 ClickHouse#82934 - 20250626 ClickHouse#80931 - 20250604 ClickHouse#79649 - 20250502 ClickHouse#79180 - 20250416 ClickHouse#78485 - 20250306 ClickHouse#76662 Co-authored-by: liuneng1994 <neng.liu@kyligence.com>

Fix rebase issue: - 20250806 ClickHouse#84821 - 20250804 ClickHouse#83997 - 20250728 ClickHouse#84180 - 20250713 ClickHouse#82949 - 20250703 ClickHouse#82934 - 20250626 ClickHouse#80931 - 20250604 ClickHouse#79649 - 20250502 ClickHouse#79180 - 20250416 ClickHouse#78485 - 20250306 ClickHouse#76662 Co-authored-by: liuneng1994 <neng.liu@kyligence.com>

Unrevert "Refactor how IInputFormat deals with reading many files at once"

Fix rebase issue: - 20250728 ClickHouse#84180 - 20250713 ClickHouse#82949 - 20250703 ClickHouse#82934 - 20250626 ClickHouse#80931 - 20250604 ClickHouse#79649 - 20250502 ClickHouse#79180 - 20250416 ClickHouse#78485 - 20250306 ClickHouse#76662 Co-authored-by: liuneng1994 <neng.liu@kyligence.com>

Fix rebase issue: - 20250728 ClickHouse#84180 - 20250713 ClickHouse#82949 - 20250703 ClickHouse#82934 - 20250626 ClickHouse#80931 - 20250604 ClickHouse#79649 - 20250502 ClickHouse#79180 - 20250416 ClickHouse#78485 - 20250306 ClickHouse#76662 Co-authored-by: liuneng1994 <neng.liu@kyligence.com> (cherry picked from commit c68fb33)

Fix rebase issue: - 20250728 ClickHouse#84180 - 20250713 ClickHouse#82949 - 20250703 ClickHouse#82934 - 20250626 ClickHouse#80931 - 20250604 ClickHouse#79649 - 20250502 ClickHouse#79180 - 20250416 ClickHouse#78485 - 20250306 ClickHouse#76662 Co-authored-by: liuneng1994 <neng.liu@kyligence.com> (cherry picked from commit c68fb33) (cherry picked from commit 12ab7a6)

al13n321 added 2 commits June 30, 2025 16:53

Revert "Revert "Refactor how IInputFormat deals with reading many fil…

77fbb7b

…es at once"" This reverts commit 1f28e8b.

Fix double initialization

1165849

clickhouse-gh bot added the pr-not-for-changelog This PR should not be mentioned in the changelog label Jun 30, 2025

al13n321 added 2 commits June 30, 2025 19:03

Comment

6e10333

Merge remote-tracking branch 'origin/master' into shp2

88c8bc0

jkartseva self-assigned this Jul 1, 2025

al13n321 added 2 commits July 2, 2025 02:17

Unrelated fix

0673378

Merge remote-tracking branch 'origin/master' into shp2

251df78

jkartseva approved these changes Jul 2, 2025

View reviewed changes

alesapin mentioned this pull request Jul 2, 2025

Implement reading from Iceberg data files using field-id #83065

Closed

al13n321 added 4 commits July 3, 2025 19:10

Merge remote-tracking branch 'origin/master' into shp2

e0b1fe5

Fix unrelated build error

d24820f

Merge remote-tracking branch 'origin/master' into shp2

0e19c52

Merge remote-tracking branch 'origin/master' into shp2

5199cd8

Merge remote-tracking branch 'origin/master' into shp2

f8663c5

al13n321 enabled auto-merge July 8, 2025 03:46

al13n321 added 2 commits July 8, 2025 07:01

Merge remote-tracking branch 'origin/master' into shp2

9b9683f

Merge remote-tracking branch 'origin/master' into shp2

63e7535

Merge remote-tracking branch 'origin/master' into shp2

314fc37

Merge remote-tracking branch 'origin/master' into shp2

9247cbc

Merge remote-tracking branch 'origin/master' into shp2

06e0dfe

al13n321 added this pull request to the merge queue Jul 12, 2025

Merged via the queue into master with commit 67fc087 Jul 12, 2025
118 of 123 checks passed

github-merge-queue bot pushed a commit that referenced this pull request Jul 22, 2025

Merge pull request #84180 from jkartseva/revert-thread-pool-cb-runner…

9d200fa

…-fast-usage Fix flaky tests with parquet by partially reverting #82949

zvonand pushed a commit to Altinity/ClickHouse that referenced this pull request Aug 31, 2025

Merge pull request ClickHouse#82949 from ClickHouse/shp2

36caa6c

Unrevert "Refactor how IInputFormat deals with reading many files at once"

zvonand pushed a commit to Altinity/ClickHouse that referenced this pull request Sep 2, 2025

Merge pull request ClickHouse#82949 from ClickHouse/shp2

fe97330

Unrevert "Refactor how IInputFormat deals with reading many files at once"

zvonand pushed a commit to Altinity/ClickHouse that referenced this pull request Sep 3, 2025

Merge pull request ClickHouse#82949 from ClickHouse/shp2

f3e7171

Unrevert "Refactor how IInputFormat deals with reading many files at once"

zvonand mentioned this pull request Sep 3, 2025

? Antalya 25.6.5 Backport of 83653: Read iceberg data files by field ids Altinity/ClickHouse#999

Closed

13 tasks

zvonand pushed a commit to Altinity/ClickHouse that referenced this pull request Sep 8, 2025

Merge pull request ClickHouse#82949 from ClickHouse/shp2

e803bc1

Unrevert "Refactor how IInputFormat deals with reading many files at once"

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unrevert "Refactor how IInputFormat deals with reading many files at once"#82949

Unrevert "Refactor how IInputFormat deals with reading many files at once"#82949
al13n321 merged 16 commits intomasterfrom
shp2

al13n321 commented Jun 30, 2025 •

edited

Loading

Uh oh!

clickhouse-gh bot commented Jun 30, 2025 •

edited

Loading

Uh oh!

al13n321 commented Jul 3, 2025

Uh oh!

al13n321 commented Jul 5, 2025 •

edited

Loading

Uh oh!

al13n321 commented Jul 8, 2025

Uh oh!

al13n321 commented Jul 11, 2025 •

edited

Loading

Uh oh!

al13n321 commented Jul 11, 2025

Uh oh!

al13n321 commented Jul 12, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

al13n321 commented Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog category (leave one):

Uh oh!

clickhouse-gh bot commented Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

al13n321 commented Jul 3, 2025

Uh oh!

al13n321 commented Jul 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

al13n321 commented Jul 8, 2025

Uh oh!

al13n321 commented Jul 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

al13n321 commented Jul 11, 2025

Uh oh!

al13n321 commented Jul 12, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

al13n321 commented Jun 30, 2025 •

edited

Loading

clickhouse-gh bot commented Jun 30, 2025 •

edited

Loading

al13n321 commented Jul 5, 2025 •

edited

Loading

al13n321 commented Jul 11, 2025 •

edited

Loading