What happens?

We've had some significant performance slow down in moving from DuckDB 1.2.2 to 1.3.2. I'm working my way through our various queries to see if I can produce simple bug reports.

To Reproduce

This query:

SELECT
  t2.Trade_Type , 
  MAX(t2.encoded_buffer_data) as encoded_buffer_data
FROM transactions_anon t2 
WHERE
    (t2.Book = 318)
    AND
    ( t2.Transaction_Day <= 2460882 )
    AND
    (NOT t2.tombstone )
    AND
    (t2.version <= 2410330 AND (COALESCE(t2.next_version, 9223372036854775807) > 2410330)) 
AND  t2.error IS NULL
GROUP BY t2.Trade_Type

Is three to four times slower from 1.3+ onwards. It's still slow on the nightly, which I tried today (8 Aug).

If I remove the AND t2.error IS NULL it's back to the original 1.2.2 speed. Also if I move that is null check to MAX(t2.encoded_buffer_data) FILTER (WHERE t2.error IS NULL) ... then it's also 1.2.2 speed.

I've attached the json profiles for the original query against 1.2.2 and 1.3.2.

I've also attached an anonymised version of the database (thanks @soerenwolfers) which exhibits the problem.

profile-1.2.2.json
profile-1.3.2.json
bug.db.zip

OS:

aarch64

DuckDB Version:

1.3.2

DuckDB Client:

CLI

Hardware:

Apple M4 Max 128GB RAM

Full Name:

David Corcoran

Affiliation:

Topaz Technology Ltd

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have tested with a nightly build

Did you include all relevant data sets for reproducing the issue?

Yes

Did you include all code required to reproduce the issue?

• Yes, I have

Did you include all relevant configuration (e.g., CPU architecture, Python version, Linux distribution) to reproduce the issue?

• Yes, I have