Skip to content

CNF/Constraint optimizer in new analyzer#47617

Merged
alexey-milovidov merged 17 commits intomasterfrom
cnf-new-analyzer
Apr 2, 2023
Merged

CNF/Constraint optimizer in new analyzer#47617
alexey-milovidov merged 17 commits intomasterfrom
cnf-new-analyzer

Conversation

@antonio2368
Copy link
Copy Markdown
Member

@antonio2368 antonio2368 commented Mar 15, 2023

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Add CNF/constraint optimizer in new analyzer.

  • AddIndexConstraintsOptimizer
  • Substitute column optimizer
  • try to make it not crash

Documentation entry for user-facing changes

  • Documentation is written (mandatory for new features)

Information about CI checks: https://clickhouse.com/docs/en/development/continuous-integration/

@robot-ch-test-poll4 robot-ch-test-poll4 added the pr-improvement Pull request with some product improvements label Mar 15, 2023
@antonio2368 antonio2368 force-pushed the cnf-new-analyzer branch 2 times, most recently from 4a82c02 to dbf0ee9 Compare March 16, 2023 09:26
@antonio2368 antonio2368 marked this pull request as ready for review March 17, 2023 13:47
void ConvertQueryToCNFPass::run(QueryTreeNodePtr query_tree_node, ContextPtr context)
{
const auto & settings = context->getSettingsRef();
if (!settings.convert_query_to_cnf)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check better should be done in context of every QueryNode

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why? don't we use the same context for all of them?

};

optimize_filter(query_node->getWhere());
optimize_filter(query_node->getPrewhere());
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't do it for HAVING clause?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the original implementation, it was done only for WHERE clause so I just went with that.
HAVING is applied to aggregate values, while constraints are defined for each table, i.e. we can't have constraints for aggregate.
But I can actually transform it to CNF so I will add it.

Copy link
Copy Markdown
Contributor

@UnamedRus UnamedRus Aug 2, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, it should work for any (ok, except steps after aggregation) part of query, for example in SELECT part as well.
#33544


auto & new_function_node = current->as<FunctionNode &>();
function_node->getArguments().getNodes() = std::move(new_function_node.getArguments().getNodes());
function_node->resolveAsFunction(function_resolver);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not *function_node = std::move(new_function_node)?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definition of implicit copy assignment operator for 'IQueryTreeNode' is deprecated because it has a user-declared destructor (clang -Wdeprecated-copy-with-dtor)

{
return node_with_hash.hash == rhs.node_with_hash.hash
? negative < rhs.negative
: node_with_hash.hash < rhs.node_with_hash.hash;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting statement

{
transformAtoms([&](const AtomicFormula & atom)
{
static const std::unordered_map<std::string, std::string> inverse_relations = {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems that this map must be the same as in pushNotIntoFunction

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we want only a subset - function starting with not and comparisons that are not less or equals

@alexey-milovidov alexey-milovidov merged commit 5ebf668 into master Apr 2, 2023
@alexey-milovidov alexey-milovidov deleted the cnf-new-analyzer branch April 2, 2023 17:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-improvement Pull request with some product improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants