What problem does this PR solve?

Issue Number: ref #5294

What is changed and how it works?

• optimize agg with one string key
  • type: str-bin, str-bin-padding
  • struct AggregationMethodOneKeyStringNoCache
• optimize agg with two keys
  • type: number-64-bytes, str-bin, str-bin-padding
  • struct AggregationMethodFastPathTwoKeysNoCache
• AggregationMethodOneKeyStringNoCache and AggregationMethodFastPathTwoKeysNoCache will generate empty data for string column because such column will be discarded.

  • tiflash/dbms/src/Columns/IColumn.h

    Lines 160 to 173 in 84c2650

    	/** Deserializes a value that was serialized using IColumn::serializeValueIntoArena method.
    	* Returns pointer to the position after the read data.
    	* Note:
    	* 1. For string columns with collation it is actually impossible to restore the value.
    	* In this case, the column restored by this function is discarded, the compiler will add a
    	* special aggregate function(any) as the group by column's output. For example:
    	* select string_column, count(*) from table group by string_column
    	* if string_column has collation information, the query will be rewrite to
    	* select any(string_column), count(*) from table group by string_column
    	* 2. The input parameter `collator` does not work well for complex columns(column tuple),
    	* but it is only used by TiDB , which does not support complex columns, so just ignore
    	* the complex column will be ok.
    	*/
    	virtual const char * deserializeAndInsertFromArena(const char * pos, std::shared_ptr<TiDB::ITiDBCollator> collator = nullptr) = 0;

  • tiflash/dbms/src/Flash/Coprocessor/DAGExpressionAnalyzer.cpp

    Lines 272 to 306 in 84c2650

    	if (collator != nullptr)
    	{
    	/// if the column is a string with collation info, the `sort_key` of the column is used during
    	/// aggregation, but we can not reconstruct the origin column by `sort_key`, so add an extra
    	/// extra aggregation function any(group_by_column) here as the output of the group by column
    	const String & agg_func_name = "any";
    	AggregateDescription aggregate;
    	DataTypes types(1);
    	aggregate.argument_names.resize(1);
    	types[0] = type;
    	aggregate.argument_names[0] = name;
    	String func_string = genFuncString(agg_func_name, aggregate.argument_names);
    	bool duplicate = false;
    	for (const auto & pre_agg : aggregate_descriptions)
    	{
    	if (pre_agg.column_name == func_string)
    	{
    	aggregated_columns.emplace_back(func_string, pre_agg.function->getReturnType());
    	duplicate = true;
    	break;
    	}
    	}
    	if (duplicate)
    	continue;
    	aggregate.column_name = func_string;
    	aggregate.parameters = Array();
    	aggregate.function = AggregateFunctionFactory::instance().get(agg_func_name, types, {}, 0, false);
    	aggregate.function->setCollator(getCollatorFromExpr(expr));
    	aggregate_descriptions.push_back(aggregate);
    	DataTypePtr result_type = aggregate.function->getReturnType();
    	// this is a temp result since implicit cast maybe added on these aggregated_columns
    	aggregated_columns.emplace_back(func_string, result_type);
    	}

• optimize ColumnString::deserializeAndInsertFromArena by same way above.

We may need JIT for further optimization about AGG/JOIN/SET later.

Benchmark

ENV

• TiFlash x 1
• limit cpu up to 2000%
• original commit: 49ca973
• ClickBench

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

None