Skip to content

In Power BI, filters break with non-Latin character values. #449

@luckyfs

Description

@luckyfs

Prerequisites:

  1. Clickhouse on-premises, Ubuntu (project database or sandbox play.clickhouse.com (select * from actors))
  2. Power BI Desktop, Windows
  3. Clickhouse ODBC driver (Unicode)
  4. Clickhouse native driver for Power BI
  5. DirectQuery mode for the Power BI report

Steps to Reproduce:

  1. Open Power BI Desktop
  2. Connect to the Clickhouse database using the native connector
  3. Create a report that calculates a count of rows where the filter parameter is an attribute containing Unicode values (Cyrillic, Chinese, and others).
  4. Apply a filter with a Unicode value

Actual Result: Power BI Desktop throws an error: "OLE DB or ODBC error: [Expression.Error] We couldn't fold the expression to the data source. Please try a simpler expression."

Expected Result: The report successfully calculates the count of rows.

Log Analysis Findings:

  1. The Clickhouse native connector log does not register this error. It seems that the query does not reach the connector.
  2. The Power BI Desktop log shows that the query is generated. In the query, Unicode characters are encoded (example from the log, see the WHERE block): SELECT [t2].[SEGMENT_RENNA_ID] AS [c52],[t2].[SEGMENT_RENNA_TXT] AS [c53],[t2].[SEGMENT_RENNA_ID] AS [o0]#(lf)FROM [DICT_DIM_SKUMATCH] AS [t2]#(lf))#(lf) AS [t0]#(lf)WHERE #(lf)(#(lf)[c53] = N'\u041C\u0420\u0416'#(lf))#(lf)#(lf)GROUP BY [c52],[o0]#(lf)ORDER BY #(lf)
  3. ODBC trace logs show that when Unicode values are used, question marks appear, indicating that the ODBC driver cannot correctly interpret Unicode values.

Please help us resolve this issue, as this error essentially blocks the use of Power BI. I would like to remind you that 99% of reports in Power BI are created in Power BI Desktop and subsequently published to the server.

I have attached logs at all levels, screenshots, and a video of the system's behavior.

Some articles suggest checking the same connection to excel, but this is a completely different case, since excel always downloads all the data first, after which the user uses excel to filter the data. In other words, there are no direct requests from excel to the clickhouse database
Microsoft.Mashup.Container.NetFX45.22512.2024-08-15T15-48-23-346359.log
msmdsrv.26716.2024-08-15T15-48-23-324476.log
logs.txt
SQL.LOG
Microsoft.Mashup.Container.NetFX45.16788.2024-08-15T15-48-23-399625.log
Microsoft.Mashup.Container.NetFX45.29208.2024-08-15T15-48-23-339172.log
PBIDesktop.26112.2024-08-15T15-48-16-135078.log

Video
Screenshot_1

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions