Expand upstream SQL test allowlist#710
Merged
Merged
Conversation
Agent-Logs-Url: https://github.com/ClickHouse/clickhouse-js/sessions/c7ab734d-0285-4f2a-9a36-ec8891982784 Co-authored-by: peter-leonov-ch <209667683+peter-leonov-ch@users.noreply.github.com>
Copilot created this pull request from a session on behalf of
peter-leonov-ch
May 9, 2026 09:47
View session
|
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Contributor
There was a problem hiding this comment.
Pull request overview
Expands the upstream ClickHouse stateless SQL test allowlist used by tests/clickhouse-test-runner to increase CI/harness coverage across more query patterns and functions.
Changes:
- Added 35 upstream stateless SQL test names to
tests/clickhouse-test-runner/upstream-allowlist.txt. - Kept the existing allowlisted test (
01428_hash_set_nan_key) while growing the curated set to 36 total tests.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
7 tasks
peter-leonov-ch
added a commit
that referenced
this pull request
May 9, 2026
Continues #708 / #710 by growing `tests/clickhouse-test-runner/upstream-allowlist.txt` from 36 → 424 entries through the Node.js harness, validated on both `client` and `http` backends. - [x] Set up upstream ClickHouse sparse checkout, install Python deps, build the test runner, start ClickHouse via docker-compose - [x] Verify the existing 36-test allowlist passes through the harness on both `client` and `http` backends - [x] Build the per-batch driver (add 100, run both backends with `--no-stateful --no-long`, keep passers, drop failures and skips) - [x] Run batches 1-6 (600 candidates evaluated; 342 passed on both `client` and `http` backends and were kept; 257 failures and 1 skip dropped); allowlist 36 → 378 - [x] Fix CI failures from prior commit: drop 54 entries from the allowlist that either failed in the CI matrix (`{client,http} × {head,latest}`) or that act as **prefix matches** for failing tests (clickhouse-test treats positional args as substring/prefix matches, so e.g. `00396_uuid` was unintentionally pulling in the failing `00396_uuid_v7`). Net: allowlist now 424 entries (+388 vs the pre-#710 baseline of 36) - [x] Document the harness purpose and the batch-add strategy (including the prefix-match gotcha) in `AGENTS.md` so future agent runs have the context - [x] PR ready for review --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: peter-leonov-ch <209667683+peter-leonov-ch@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Continues #708 by growing
tests/clickhouse-test-runner/upstream-allowlist.txtfrom 1 to 36 upstream ClickHouse stateless SQL tests.Each added test was verified to pass through the harness on both backends (
client,http) and both server matrix versions (head,latest). Coverage spans:SELECT, parser numerics,reinterpretAsStringARRAY JOIN, nested columns,arrayEnumerateUniqGROUP BY/WITH TOTALS/HAVING,DISTINCTin subqueriesFixedStringconversions, higher-order function constants,argMin/argMaxIN/ set, ORDER BY descending on (Fixed)StringANY/ALLLEFT/INNERjoins on numbers, strings, mixed typesChecklist