Skip to content

Expand upstream SQL test allowlist#710

Merged
peter-leonov-ch merged 1 commit into
mainfrom
copilot/add-more-sql-tests
May 9, 2026
Merged

Expand upstream SQL test allowlist#710
peter-leonov-ch merged 1 commit into
mainfrom
copilot/add-more-sql-tests

Conversation

Copilot AI commented May 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Continues #708 by growing tests/clickhouse-test-runner/upstream-allowlist.txt from 1 to 36 upstream ClickHouse stateless SQL tests.

Each added test was verified to pass through the harness on both backends (client, http) and both server matrix versions (head, latest). Coverage spans:

  • Basic SELECT, parser numerics, reinterpretAsString
  • Array functions, ARRAY JOIN, nested columns, arrayEnumerateUniq
  • GROUP BY / WITH TOTALS / HAVING, DISTINCT in subqueries
  • FixedString conversions, higher-order function constants, argMin/argMax
  • IN / set, ORDER BY descending on (Fixed)String
  • ANY/ALL LEFT/INNER joins on numbers, strings, mixed types

Checklist

  • A human-readable description of the changes was provided to include in CHANGELOG

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@codecov

codecov Bot commented May 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@peter-leonov-ch peter-leonov-ch marked this pull request as ready for review May 9, 2026 10:07
@peter-leonov-ch peter-leonov-ch self-requested a review as a code owner May 9, 2026 10:07
Copilot AI review requested due to automatic review settings May 9, 2026 10:07
@peter-leonov-ch peter-leonov-ch requested a review from mshustov as a code owner May 9, 2026 10:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Expands the upstream ClickHouse stateless SQL test allowlist used by tests/clickhouse-test-runner to increase CI/harness coverage across more query patterns and functions.

Changes:

  • Added 35 upstream stateless SQL test names to tests/clickhouse-test-runner/upstream-allowlist.txt.
  • Kept the existing allowlisted test (01428_hash_set_nan_key) while growing the curated set to 36 total tests.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@peter-leonov-ch peter-leonov-ch merged commit d4c2efe into main May 9, 2026
68 checks passed
@peter-leonov-ch peter-leonov-ch deleted the copilot/add-more-sql-tests branch May 9, 2026 10:20
peter-leonov-ch added a commit that referenced this pull request May 9, 2026
Continues #708 / #710 by growing
`tests/clickhouse-test-runner/upstream-allowlist.txt` from 36 → 424
entries through the Node.js harness, validated on both `client` and
`http` backends.

- [x] Set up upstream ClickHouse sparse checkout, install Python deps,
build the test runner, start ClickHouse via docker-compose
- [x] Verify the existing 36-test allowlist passes through the harness
on both `client` and `http` backends
- [x] Build the per-batch driver (add 100, run both backends with
`--no-stateful --no-long`, keep passers, drop failures and skips)
- [x] Run batches 1-6 (600 candidates evaluated; 342 passed on both
`client` and `http` backends and were kept; 257 failures and 1 skip
dropped); allowlist 36 → 378
- [x] Fix CI failures from prior commit: drop 54 entries from the
allowlist that either failed in the CI matrix (`{client,http} ×
{head,latest}`) or that act as **prefix matches** for failing tests
(clickhouse-test treats positional args as substring/prefix matches, so
e.g. `00396_uuid` was unintentionally pulling in the failing
`00396_uuid_v7`). Net: allowlist now 424 entries (+388 vs the pre-#710
baseline of 36)
- [x] Document the harness purpose and the batch-add strategy (including
the prefix-match gotcha) in `AGENTS.md` so future agent runs have the
context
- [x] PR ready for review

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: peter-leonov-ch <209667683+peter-leonov-ch@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants