Not found column when combining row policies, prewhere and additional_table_filters #85834
Description
Company or project name
No response
Describe what's wrong
The logic in PlannerJoinTree-add_filter pushes "extra" filters to where_filters once the PrewhreInfo has already been filled.
When building the query plan, the initial PrewhereInfo object gets executed against the header block and ends up removing the input columns. Later, when adding the where, if the condition relied on the same columns, they'll not be present. Hence the exception.
Does it reproduce on the most recent release?
Yes
How to reproduce
Repro based on tests/queries/0_stateless/00950_default_prewhere.sql
arthur :) DROP TABLE IF EXISTS test_generic_events_all;
CREATE TABLE test_generic_events_all (APIKey UInt8, SessionType UInt8) ENGINE = MergeTree() PARTITION BY APIKey ORDER BY tuple();
INSERT INTO test_generic_events_all VALUES( 42, 42 );
ALTER TABLE test_generic_events_all ADD COLUMN OperatingSystem UInt64 DEFAULT 42;
DROP TABLE IF EXISTS test_generic_events_all
Query id: 2e48f52f-8c4d-4516-98e6-e6eca3844387
Ok.
0 rows in set. Elapsed: 0.004 sec.
CREATE TABLE test_generic_events_all
(
`APIKey` UInt8,
`SessionType` UInt8
)
ENGINE = MergeTree
PARTITION BY APIKey
ORDER BY tuple()
Query id: 07ef956f-ff0d-4685-b4ff-4fa469a982b3
Ok.
0 rows in set. Elapsed: 0.032 sec.
INSERT INTO test_generic_events_all FORMAT Values
Query id: 62000080-8f4f-440f-99f8-b8f3d171182b
Ok.
1 row in set. Elapsed: 0.010 sec.
ALTER TABLE test_generic_events_all
(ADD COLUMN `OperatingSystem` UInt64 DEFAULT 42)
Query id: 874411e5-7398-4f8d-a888-11313ed3c54d
Ok.
0 rows in set. Elapsed: 0.017 sec.
arthur :) CREATE ROW POLICY rp ON test_generic_events_all USING APIKey>35 TO CURRENT_USER;
CREATE ROW POLICY rp ON test_generic_events_all FOR SELECT USING APIKey > 35 TO CURRENT_USER
Query id: 7a7567a2-f3c2-4f4d-a6af-44764f333750
Ok.
0 rows in set. Elapsed: 0.005 sec.
arthur :) SELECT OperatingSystem FROM test_generic_events_all PREWHERE APIKey = 42 SETTINGS additional_table_filters = {'test_generic_events_all': 'APIKey > 40'};
SELECT OperatingSystem
FROM test_generic_events_all
PREWHERE APIKey = 42
SETTINGS additional_table_filters = {'test_generic_events_all':'APIKey > 40'}
Query id: 4e66da7d-8e79-4e6f-893f-ded230330b98
Elapsed: 0.009 sec.
Received exception from server (version 25.8.1):
Code: 10. DB::Exception: Received from localhost:9000. DB::Exception: Not found column APIKey: in block OperatingSystem UInt64 UInt64(size = 0). (NOT_FOUND_COLUMN_IN_BLOCK)
arthur :)
Expected behavior
Query succeeds and filters are properly applied
Error message and/or stacktrace
Received exception from server (version 25.8.1):
Code: 10. DB::Exception: Received from localhost:9000. DB::Exception: Not found column APIKey: in block OperatingSystem UInt64 UInt64(size = 0). (NOT_FOUND_COLUMN_IN_BLOCK)
Additional context
No response