Merged
Conversation
Before, when a column had the BYTE_ARRAY type, anyquery would still try to interpret it at UTF-8, leading to a Replacement Character. To fix this, Anyquery now make sure a []byte slice is returned rather than a string. I couldn't find any other way to fix this as parquet-go/parquet-go returns a string even for a parquet BYTE_ARRAY
As stated in the SQLite docs, https://www.sqlite.org/json1.html#jptr:~:text=The%20SQLite%20version%20of%20json_extract()%20only%20returns%20JSON SQLite json_extract differs from the MySQL one, which means an additionnal json_unquote isn't needed. Because the parser is originally for MySQL, I've modified the yacc file to remove json_unquote
Now, Anyquery only queries the column requested by the user to ClickHouse. This speeds up queries because ClickHouse doesn't have to read all columns (while being columnar-based).
When you do a SELECT count(*), SQLite does not request any column, resulting in an empty SELECT for the remote database, which isn't supported by most SQL query engines. To work around this, we arbitrarily add the first column to the SELECT
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
There was a problem hiding this comment.
Pull Request Overview
This PR implements support for querying DuckDB and Cassandra databases within Anyquery, addresses performance optimizations for ClickHouse, and includes various website documentation updates for the 0.4.3 release.
- Adds new database connectors for DuckDB and Cassandra with full schema introspection and querying capabilities
- Optimizes ClickHouse queries to only fetch requested columns instead of all columns
- Updates website documentation with new database guides and styling improvements
Reviewed Changes
Copilot reviewed 18 out of 22 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
module/cassandra.go |
Implements complete Cassandra database module with connection pooling, schema mapping, and CQL query optimization |
module/clickhouse.go |
Optimizes column selection to only fetch columns used in queries, improving performance |
module/db_helper.go |
Adds fallback logic to ensure at least one column is selected for COUNT(*) queries |
namespace/namespace.go |
Registers new Cassandra module and updates supported connections list |
namespace/connections.go |
Implements Cassandra connection registration with table discovery and filtering |
website/src/content/docs/docs/database/cassandra.md |
Comprehensive documentation for Cassandra integration |
website/src/content/docs/docs/database/scylladb.md |
Documentation for ScyllaDB (Cassandra-compatible) support |
go.mod |
Updates dependencies including new Cassandra driver |
Files not reviewed (1)
- website/pnpm-lock.yaml: Language not supported
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request merges the changes for the
0.4.3release. It fixes issues #51 and #52.Enhancement