Can't get the right logical plan after optimizer #3421
Description
Describe the bug
After the type coercion moved to logical optimizer.
The order of optimizer rule is the filter_push_down is before the TypeCoercion, but we can't get the right logical plan
For example
❯ \d test
+---------------+--------------+------------+-------------+-----------------------------+-------------+
| table_catalog | table_schema | table_name | column_name | data_type | is_nullable |
+---------------+--------------+------------+-------------+-----------------------------+-------------+
| datafusion | public | test | 0 | Timestamp(Nanosecond, None) | YES |
| datafusion | public | test | 1 | Date32 | YES |
| datafusion | public | test | 2 | Int64 | YES |
| datafusion | public | test | 3 | Int64 | YES |
| datafusion | public | test | 7 | Decimal128(9, 0) | YES |
| datafusion | public | test | 8 | Float32 | YES |
| datafusion | public | test | 11 | Utf8 | YES |
| datafusion | public | test | 16 | Utf8 | YES |
| datafusion | public | test | 100000 | Int64 | YES |
| datafusion | public | test | 100001 | Int64 | YES |
| datafusion | public | test | 100002 | Int64 | YES |
+---------------+--------------+------------+-------------+-----------------------------+-------------+
❯ explain select "1" from test where "8" = 1;
+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| plan_type | plan |
+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| logical_plan | Projection: #test.1 |
| | Filter: #test.8 = CAST(Int64(1) AS Float32) |
| | TableScan: test projection=[1, 8], partial_filters=[#test.8 = Int64(1)] |
❯ explain verbose select "1" from test where "8" = 1;
+-------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| plan_type | plan |
+-------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| initial_logical_plan | Projection: #test.1 |
| | Filter: #test.8 = Int64(1) |
| | TableScan: test |
| logical_plan after simplify_expressions | SAME TEXT AS ABOVE |
| logical_plan after pre_cast_lit_in_comparison | SAME TEXT AS ABOVE |
| logical_plan after decorrelate_where_exists | SAME TEXT AS ABOVE |
| logical_plan after decorrelate_where_in | SAME TEXT AS ABOVE |
| logical_plan after scalar_subquery_to_join | SAME TEXT AS ABOVE |
| logical_plan after subquery_filter_to_join | SAME TEXT AS ABOVE |
| logical_plan after eliminate_filter | SAME TEXT AS ABOVE |
| logical_plan after common_sub_expression_eliminate | SAME TEXT AS ABOVE |
| logical_plan after eliminate_limit | SAME TEXT AS ABOVE |
| logical_plan after projection_push_down | Projection: #test.1 |
| | Filter: #test.8 = Int64(1) |
| | TableScan: test projection=[1, 8] |
| logical_plan after rewrite_disjunctive_predicate | SAME TEXT AS ABOVE |
| logical_plan after reduce_outer_join | SAME TEXT AS ABOVE |
| logical_plan after filter_push_down | Projection: #test.1 |
| | Filter: #test.8 = Int64(1) |
| | TableScan: test projection=[1, 8], partial_filters=[#test.8 = Int64(1)] |
| logical_plan after TypeCoercion | Projection: #test.1 |
| | Filter: #test.8 = CAST(Int64(1) AS Float32) |
| | TableScan: test projection=[1, 8], partial_filters=[#test.8 = Int64(1)]
The partial_filters was not casted to the right type, it will break the pruning for the parquet.
Why we assign this order for the optimizer rule?
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
Additional context
Add any other context about the problem here.