Fix CI: drop unrecognized `--print-time` flag from upstream-sql-tests by Copilot · Pull Request #708 · ClickHouse/clickhouse-js

Copilot · 2026-05-08T18:45:15Z

Fix CI errors from #707.

Remove --print-time from workflow + README example (it isn't a valid tests/clickhouse-test flag)
Remove unused clickhouse_version workflow_dispatch input from upstream-sql-tests.yml
Remove unused UPSTREAM_REF env var from upstream-sql-tests.yml
Install jinja2 in upstream-sql-tests.yml
Fix MaxListenersExceededWarning in client backend by replacing pipeline(result.stream, process.stdout, { end: false }) with for await (const chunk of result.stream) so no error/close listeners accumulate on stdout
Keep --trace-warnings enabled in tests/clickhouse-test-runner/bin/clickhouse so any future Node warnings include their stack traces in CI logs

Agent-Logs-Url: https://github.com/ClickHouse/clickhouse-js/sessions/082ff17d-204a-4d87-8da8-e9a080a4a54b Co-authored-by: peter-leonov-ch <209667683+peter-leonov-ch@users.noreply.github.com>

Agent-Logs-Url: https://github.com/ClickHouse/clickhouse-js/sessions/28a5d7f8-37bc-4cb2-81d4-573656dcd9c8 Co-authored-by: peter-leonov-ch <209667683+peter-leonov-ch@users.noreply.github.com>

CLAassistant · 2026-05-08T18:45:23Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ peter-leonov-ch
❌ Copilot
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

Copilot

Pull request overview

This PR fixes the failing upstream ClickHouse SQL test workflow by removing the unsupported --print-time flag, and (via included #707 commits) adds the upstream SQL test harness + CI workflow that runs a curated subset of upstream tests against @clickhouse/client.

Changes:

Remove the invalid --print-time argument from the upstream test invocation and documentation examples.
Add a runner helper script + allowlist file to run a curated set of upstream tests/clickhouse-test cases locally.
Add an upstream-sql-tests GitHub Actions workflow (matrix over backend impl + ClickHouse image tag) and document the harness in contributor docs.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
`tests/clickhouse-test-runner/upstream-allowlist.txt`	Adds curated list of upstream tests to run.
`tests/clickhouse-test-runner/scripts/run-upstream-tests.sh`	Adds helper script to build/run the harness against upstream checkout with allowlist + env defaults.
`tests/clickhouse-test-runner/README.md`	Updates usage docs and examples (removes `--print-time`).
`tests/clickhouse-test-runner/.gitignore`	Ignores `.upstream/` directory used for upstream sparse checkout.
`CONTRIBUTING.md`	Adds pointer to the upstream SQL test harness docs.
`AGENTS.md`	Adds guidance/links for maintaining the upstream SQL test harness.
`.github/workflows/upstream-sql-tests.yml`	Adds CI workflow to run upstream SQL tests matrix and upload artifacts.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

codecov · 2026-05-08T18:56:37Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Agent-Logs-Url: https://github.com/ClickHouse/clickhouse-js/sessions/c4b5a2ac-1a65-4042-b329-b06304df0542 Co-authored-by: peter-leonov-ch <209667683+peter-leonov-ch@users.noreply.github.com>

Agent-Logs-Url: https://github.com/ClickHouse/clickhouse-js/sessions/5d5935b7-23d3-40a7-8e48-e3b18b7663cc Co-authored-by: peter-leonov-ch <209667683+peter-leonov-ch@users.noreply.github.com>

peter-leonov-ch · 2026-05-08T19:15:36Z

@copilot fix this warning from CI: "WARNING: jinja2 not installed! Template tests will be skipped."

…ts warning Agent-Logs-Url: https://github.com/ClickHouse/clickhouse-js/sessions/cc1e7775-1f5b-4741-af43-506e5799a6e5 Co-authored-by: peter-leonov-ch <209667683+peter-leonov-ch@users.noreply.github.com>

Copilot · 2026-05-08T19:16:36Z

@copilot fix this warning from CI: "WARNING: jinja2 not installed! Template tests will be skipped."

Done in f1647cf — added a pip install jinja2 step before the test run so upstream clickhouse-test no longer skips template tests.

peter-leonov-ch · 2026-05-08T19:22:14Z

@copilot

Fix this error from CI by first adding --trace-warnings and then waiting for CI to finish

Reason: having stderror:  
(node:6532) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 error listeners added to [SyncWriteStream]. MaxListeners is 10. Use emitter.setMaxListeners() to increase limit
(Use `node --trace-warnings ...` to show where the warning was created)
(node:6532) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 close listeners added to [SyncWriteStream]. MaxListeners is 10. Use emitter.setMaxListeners() to increase limit

…gnose CI MaxListenersExceededWarning Agent-Logs-Url: https://github.com/ClickHouse/clickhouse-js/sessions/a4242f5c-e468-42b8-94fc-88f9b528e41c Co-authored-by: peter-leonov-ch <209667683+peter-leonov-ch@users.noreply.github.com>

Copilot · 2026-05-08T19:23:14Z

@copilot

Fix this error from CI by first adding --trace-warnings and then waiting for CI to finish
...

Added --trace-warnings to the node invocation in tests/clickhouse-test-runner/bin/clickhouse in 6a7580a. Waiting for CI to surface the stack trace for the MaxListenersExceededWarning so I can pinpoint the leak.

…iteration over result.stream Agent-Logs-Url: https://github.com/ClickHouse/clickhouse-js/sessions/523648aa-0fff-488b-ba63-722a358c2bfe Co-authored-by: peter-leonov-ch <209667683+peter-leonov-ch@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Agent-Logs-Url: https://github.com/ClickHouse/clickhouse-js/sessions/6de9c4aa-1cd3-455c-a954-e78d7a3dce1b Co-authored-by: peter-leonov-ch <209667683+peter-leonov-ch@users.noreply.github.com>

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

## Summary Continues #708 by growing `tests/clickhouse-test-runner/upstream-allowlist.txt` from 1 to 36 upstream ClickHouse stateless SQL tests. Each added test was verified to pass through the harness on both backends (`client`, `http`) and both server matrix versions (`head`, `latest`). Coverage spans: - Basic `SELECT`, parser numerics, `reinterpretAsString` - Array functions, `ARRAY JOIN`, nested columns, `arrayEnumerateUniq` - `GROUP BY` / `WITH TOTALS` / `HAVING`, `DISTINCT` in subqueries - `FixedString` conversions, higher-order function constants, `argMin`/`argMax` - `IN` / set, ORDER BY descending on (Fixed)String - `ANY`/`ALL` `LEFT`/`INNER` joins on numbers, strings, mixed types ## Checklist - [x] A human-readable description of the changes was provided to include in CHANGELOG Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: peter-leonov-ch <209667683+peter-leonov-ch@users.noreply.github.com>

Continues #708 / #710 by growing `tests/clickhouse-test-runner/upstream-allowlist.txt` from 36 → 424 entries through the Node.js harness, validated on both `client` and `http` backends. - [x] Set up upstream ClickHouse sparse checkout, install Python deps, build the test runner, start ClickHouse via docker-compose - [x] Verify the existing 36-test allowlist passes through the harness on both `client` and `http` backends - [x] Build the per-batch driver (add 100, run both backends with `--no-stateful --no-long`, keep passers, drop failures and skips) - [x] Run batches 1-6 (600 candidates evaluated; 342 passed on both `client` and `http` backends and were kept; 257 failures and 1 skip dropped); allowlist 36 → 378 - [x] Fix CI failures from prior commit: drop 54 entries from the allowlist that either failed in the CI matrix (`{client,http} × {head,latest}`) or that act as **prefix matches** for failing tests (clickhouse-test treats positional args as substring/prefix matches, so e.g. `00396_uuid` was unintentionally pulling in the failing `00396_uuid_v7`). Net: allowlist now 424 entries (+388 vs the pre-#710 baseline of 36) - [x] Document the harness purpose and the batch-add strategy (including the prefix-match gotcha) in `AGENTS.md` so future agent runs have the context - [x] PR ready for review --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: peter-leonov-ch <209667683+peter-leonov-ch@users.noreply.github.com>

Copilot AI and others added 4 commits May 8, 2026 13:48

Add helper script + allowlist for upstream test runner harness

e9cb6a7

Agent-Logs-Url: https://github.com/ClickHouse/clickhouse-js/sessions/082ff17d-204a-4d87-8da8-e9a080a4a54b Co-authored-by: peter-leonov-ch <209667683+peter-leonov-ch@users.noreply.github.com>

Add upstream-sql-tests GitHub Actions workflow

9fe5ea2

Agent-Logs-Url: https://github.com/ClickHouse/clickhouse-js/sessions/082ff17d-204a-4d87-8da8-e9a080a4a54b Co-authored-by: peter-leonov-ch <209667683+peter-leonov-ch@users.noreply.github.com>

Document upstream harness; trim whitespace in allowlist parser

3750d58

Agent-Logs-Url: https://github.com/ClickHouse/clickhouse-js/sessions/082ff17d-204a-4d87-8da8-e9a080a4a54b Co-authored-by: peter-leonov-ch <209667683+peter-leonov-ch@users.noreply.github.com>

Remove --print-time flag (not supported by upstream clickhouse-test)

165d23f

Agent-Logs-Url: https://github.com/ClickHouse/clickhouse-js/sessions/28a5d7f8-37bc-4cb2-81d4-573656dcd9c8 Co-authored-by: peter-leonov-ch <209667683+peter-leonov-ch@users.noreply.github.com>

Copilot AI assigned Copilot and peter-leonov-ch May 8, 2026

Copilot created this pull request from a session on behalf of peter-leonov-ch May 8, 2026 18:45 View session

peter-leonov-ch mentioned this pull request May 8, 2026

Run upstream ClickHouse SQL test suite against @clickhouse/client #707

Closed

1 task

peter-leonov-ch marked this pull request as ready for review May 8, 2026 18:52

Copilot AI review requested due to automatic review settings May 8, 2026 18:52

peter-leonov-ch requested review from mshustov and peter-leonov-ch as code owners May 8, 2026 18:52

Copilot started reviewing on behalf of peter-leonov-ch May 8, 2026 18:52 View session

peter-leonov-ch approved these changes May 8, 2026

View reviewed changes

Copilot AI reviewed May 8, 2026

View reviewed changes

Comment thread tests/clickhouse-test-runner/README.md Outdated

Comment thread .github/workflows/upstream-sql-tests.yml Outdated

Comment thread .github/workflows/upstream-sql-tests.yml Outdated

Comment thread .github/workflows/upstream-sql-tests.yml Outdated

peter-leonov-ch and others added 2 commits May 8, 2026 21:08

Potential fix for pull request finding

66aecaa

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

34edd02

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot started work on behalf of peter-leonov-ch May 8, 2026 19:10 View session

Remove unused clickhouse_version workflow_dispatch input

5649b80

Agent-Logs-Url: https://github.com/ClickHouse/clickhouse-js/sessions/c4b5a2ac-1a65-4042-b329-b06304df0542 Co-authored-by: peter-leonov-ch <209667683+peter-leonov-ch@users.noreply.github.com>

Copilot finished work on behalf of peter-leonov-ch May 8, 2026 19:12

Copilot AI requested a review from peter-leonov-ch May 8, 2026 19:12

Copilot started work on behalf of peter-leonov-ch May 8, 2026 19:12 View session

Remove unused UPSTREAM_REF env from upstream-sql-tests workflow

3a11bfc

Agent-Logs-Url: https://github.com/ClickHouse/clickhouse-js/sessions/5d5935b7-23d3-40a7-8e48-e3b18b7663cc Co-authored-by: peter-leonov-ch <209667683+peter-leonov-ch@users.noreply.github.com>

Copilot finished work on behalf of peter-leonov-ch May 8, 2026 19:13

Copilot started work on behalf of peter-leonov-ch May 8, 2026 19:15 View session

Install jinja2 in upstream-sql-tests workflow to silence template tes…

f1647cf

…ts warning Agent-Logs-Url: https://github.com/ClickHouse/clickhouse-js/sessions/cc1e7775-1f5b-4741-af43-506e5799a6e5 Co-authored-by: peter-leonov-ch <209667683+peter-leonov-ch@users.noreply.github.com>

github-advanced-security AI found potential problems May 8, 2026

View reviewed changes

Comment thread .github/workflows/upstream-sql-tests.yml Dismissed

Comment thread .github/workflows/upstream-sql-tests.yml Dismissed

Copilot finished work on behalf of peter-leonov-ch May 8, 2026 19:17

Copilot started work on behalf of peter-leonov-ch May 8, 2026 19:22 View session

Copilot finished work on behalf of peter-leonov-ch May 8, 2026 19:23

Copilot started work on behalf of peter-leonov-ch May 8, 2026 19:26 View session

Copilot finished work on behalf of peter-leonov-ch May 8, 2026 19:29

peter-leonov-ch approved these changes May 9, 2026

View reviewed changes

peter-leonov-ch requested a review from Copilot May 9, 2026 09:26

Copilot started reviewing on behalf of peter-leonov-ch May 9, 2026 09:27 View session

Copilot started work on behalf of peter-leonov-ch May 9, 2026 09:29 View session

Copilot AI reviewed May 9, 2026

View reviewed changes

Comment thread tests/clickhouse-test-runner/src/backends/client.ts

Comment thread tests/clickhouse-test-runner/scripts/run-upstream-tests.sh

Re-add --trace-warnings to clickhouse-test-runner wrapper per request

a941d15

Agent-Logs-Url: https://github.com/ClickHouse/clickhouse-js/sessions/6de9c4aa-1cd3-455c-a954-e78d7a3dce1b Co-authored-by: peter-leonov-ch <209667683+peter-leonov-ch@users.noreply.github.com>

Copilot finished work on behalf of peter-leonov-ch May 9, 2026 09:30

Copilot AI requested a review from peter-leonov-ch May 9, 2026 09:30

Potential fix for pull request finding

60b7d39

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

peter-leonov-ch merged commit 4949fbd into main May 9, 2026
63 checks passed

peter-leonov-ch deleted the copilot/fix-ci-errors-from-pr-707 branch May 9, 2026 09:36

Copilot stopped work on behalf of peter-leonov-ch due to an error May 9, 2026 09:36
Pull request is not open

Copilot AI mentioned this pull request May 9, 2026

Expand upstream SQL test allowlist #710

Merged

1 task

Copilot AI mentioned this pull request May 9, 2026

Expand upstream SQL test allowlist (+342 tests, batches 1–6) #712

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix CI: drop unrecognized `--print-time` flag from upstream-sql-tests#708

Fix CI: drop unrecognized `--print-time` flag from upstream-sql-tests#708
peter-leonov-ch merged 13 commits into
mainfrom
copilot/fix-ci-errors-from-pr-707

Copilot AI commented May 8, 2026 •

edited

Loading

Uh oh!

CLAassistant commented May 8, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented May 8, 2026

Uh oh!

peter-leonov-ch commented May 8, 2026

Uh oh!

Copilot AI commented May 8, 2026

Uh oh!

Uh oh!

Uh oh!

peter-leonov-ch commented May 8, 2026

Uh oh!

Copilot AI commented May 8, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

Copilot AI commented May 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CLAassistant commented May 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented May 8, 2026

Codecov Report

Uh oh!

peter-leonov-ch commented May 8, 2026

Uh oh!

Copilot AI commented May 8, 2026

Uh oh!

Uh oh!

Uh oh!

peter-leonov-ch commented May 8, 2026

Uh oh!

Copilot AI commented May 8, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Copilot AI commented May 8, 2026 •

edited

Loading

CLAassistant commented May 8, 2026 •

edited

Loading