Fix SQL parser to accept nested projections in array concatenation expressions by Copilot · Pull Request #2755 · ArcadeData/arcadedb

Copilot · 2025-11-05T13:24:33Z

Summary

This PR fixes the SQL parser to accept and correctly execute nested projections in array concatenation expressions.

What Works ✅

Non-aggregate array concatenations with nested projections
Aggregate functions with nested projections in array concatenations
Example: SELECT list({'x':1}):{x} || [] now returns [{"x":1}] as expected

Root Cause

The issue had two parts:

Parser: Nested projections were only allowed in ProjectionItem, not in array concatenation elements
Aggregate Split: ArrayConcatExpression.splitForAggregation() had the wrong signature and created separate aggregate split contexts for each element

Fix

Grammar changes:

Added optional NestedProjection to ArrayConcatExpressionElement rule

Execution changes:

ArrayConcatExpressionElement executes nested projections after base expression
ArrayConcatExpressionElement.splitForAggregation() attaches nested projections to aggregate projection items

Critical fix:

Changed ArrayConcatExpression.splitForAggregation(CommandContext) to splitForAggregation(AggregateProjectionSplit, CommandContext) to match other expressions
This ensures all child elements share the same aggregate split context

All engine tests pass (1407 tests, 0 failures).

Original prompt

This section details on the original issue you should resolve

<issue_title>SQL: Concatenation of arrays with nested projections</issue_title>
<issue_description>### ArcadeDB Version:

ArcadeDB Server v23.12.1-SNAPSHOT (build 8d9583b62ab9c057dc3623f4e895348d2277a97b/1701362292401/main)

OS and JDK Version:

Running on Mac OS X 12.7 - OpenJDK 64-Bit Server VM 17.0.9 (Homebrew)

The concatenation of arrays fails where one argument contains a nested projection:

Expected behavior

[{"x":1}]

Actual behavior

Cannot execute command Encountered "||"

Error on command execution (PostCommandHandler)
com.arcadedb.exception.CommandSQLParsingException: SELECT list({"x":1}):{x} || [] limit 25
	at com.arcadedb.query.sql.parser.StatementCache.throwParsingException(StatementCache.java:126)
	at com.arcadedb.query.sql.parser.StatementCache.parse(StatementCache.java:116)
	at com.arcadedb.query.sql.parser.StatementCache.get(StatementCache.java:70)
	at com.arcadedb.query.sql.SQLQueryEngine.parse(SQLQueryEngine.java:208)
	at com.arcadedb.query.sql.SQLQueryEngine.command(SQLQueryEngine.java:97)
	at com.arcadedb.database.LocalDatabase.command(LocalDatabase.java:1335)
	at com.arcadedb.server.ServerDatabase.command(ServerDatabase.java:467)
	at com.arcadedb.server.http.handler.PostCommandHandler.executeCommand(PostCommandHandler.java:131)
	at com.arcadedb.server.http.handler.PostCommandHandler.execute(PostCommandHandler.java:110)
	at com.arcadedb.server.http.handler.DatabaseAbstractHandler.execute(DatabaseAbstractHandler.java:99)
	at com.arcadedb.server.http.handler.AbstractServerHttpHandler.handleRequest(AbstractServerHttpHandler.java:127)
	at io.undertow.server.Connectors.executeRootHandler(Connectors.java:393)
	at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:859)
	at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538)
	at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1282)
	at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: com.arcadedb.query.sql.parser.ParseException: Encountered "||" "||" at line 1, column 26.
Was expecting one of:
    <EOF> 
    <LIMIT> ...
    <SKIP2> ...
    <OFFSET> ...
    <TIMEOUT> ...
    <AS> ...
    <LET> ...
    <UNWIND> ...
    ";" ...
    "," ...
    <AS> ...
    "," ...
    <AS> ...
    
	at com.arcadedb.query.sql.parser.SqlParser.generateParseException(SqlParser.java:27535)
	at com.arcadedb.query.sql.parser.SqlParser.jj_consume_token(SqlParser.java:27380)
	at com.arcadedb.query.sql.parser.SqlParser.Parse(SqlParser.java:253)
	at com.arcadedb.query.sql.parser.StatementCache.parse(StatementCache.java:111)
	... 16 more

Steps to reproduce

SELECT list({"x":1}):{x} || []
```</issue_description>

## Comments on the Issue (you are @copilot in this section)

<comments>
</comments>

Fixes SQL: Concatenation of arrays with nested projections #1367

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: robfrank <413587+robfrank@users.noreply.github.com>

mergify · 2025-11-05T13:45:01Z

🧪 CI Insights

Here's what we observed from your CI run for 9ca8c38.

🟢 All jobs passed!

But CI Insights is watching 👀

…ogress) Co-authored-by: robfrank <413587+robfrank@users.noreply.github.com>

…orking) Co-authored-by: robfrank <413587+robfrank@users.noreply.github.com>

Co-authored-by: robfrank <413587+robfrank@users.noreply.github.com>

codacy-production · 2025-11-05T14:42:51Z

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation	Diff coverage
✅ -0.94%	✅ 81.43%

Coverage variation details

	Coverable lines	Covered lines	Coverage
Common ancestor commit (`72d73f2`)	73167	46503	63.56%
Head commit (`9ca8c38`)	73221 (+54)	45852 (-651)	62.62% (-0.94%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details

	Coverable lines	Covered lines	Diff coverage
Pull request (#2755)	1233	1004	81.43%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings Change summary preferences

robfrank · 2025-11-05T15:21:27Z

@copilot the testa is failing:

com.arcadedb.query.sql.ArrayConcatNestedProjectionTest||1 ❌||66ms|

❌ com.arcadedb.query.sql.ArrayConcatNestedProjectionTest

❌ testArrayConcatWithNestedProjection
	java.lang.AssertionError: 
	Expected size: 1 but was: 0 in:
	[]

The issue was that ArrayConcatExpression.splitForAggregation() had a different signature than other expressions - it only took CommandContext instead of (AggregateProjectionSplit, CommandContext). This caused it to create a new AggregateProjectionSplit for each element instead of using the shared one, resulting in aggregate items being added to throwaway lists. Changes: - Updated ArrayConcatExpression.splitForAggregation() signature to match other expressions - Updated Expression.splitForAggregation() to pass aggregateSplit parameter to ArrayConcatExpression - ArrayConcatExpressionElement now properly attaches nested projections to aggregate projection items The test ArrayConcatNestedProjectionTest now passes. Co-authored-by: robfrank <413587+robfrank@users.noreply.github.com>

Copilot · 2025-11-05T15:46:35Z

@copilot the testa is failing:

...

Fixed in commit cc4c6f3. The test was failing because ArrayConcatExpression.splitForAggregation() had the wrong signature - it created a new AggregateProjectionSplit for each element instead of sharing one, causing aggregate results to be lost. Now all tests pass (1407 tests, 0 failures).

robfrank · 2025-11-05T16:01:19Z

@copilot replace the system.out in ArrayConcatNestedProjectionTest with assertions if possible and remove them

Removed all System.out.println statements and simplified the test to focus on the main test case from the issue. The test is now cleaner and only contains assertions. Co-authored-by: robfrank <413587+robfrank@users.noreply.github.com>

Copilot · 2025-11-05T16:05:21Z

@copilot replace the system.out in ArrayConcatNestedProjectionTest with assertions if possible and remove them

Done in commit fe50d72. Removed all System.out.println statements and simplified the test to focus on the main test case. The test now only contains assertions and passes cleanly.

…ion expressions (#2755) (cherry picked from commit edc11ec)

Initial plan

75e3f8a

Copilot AI assigned Copilot and robfrank Nov 5, 2025

Copilot started work on behalf of robfrank November 5, 2025 13:24 View session

Initial plan for fixing array concatenation with nested projections

789eee1

Co-authored-by: robfrank <413587+robfrank@users.noreply.github.com>

Copilot AI and others added 3 commits November 5, 2025 13:48

Add support for nested projections in array concatenation (work in pr…

061f3cd

…ogress) Co-authored-by: robfrank <413587+robfrank@users.noreply.github.com>

WIP: Fix for array concatenation with nested projections (partially w…

523d2ce

…orking) Co-authored-by: robfrank <413587+robfrank@users.noreply.github.com>

Document current status and limitations of the fix

a4c1a33

Co-authored-by: robfrank <413587+robfrank@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Fix concatenation of arrays with nested projections in SQL~~ Fix SQL parser to accept nested projections in array concatenation expressions Nov 5, 2025

Copilot AI requested a review from robfrank November 5, 2025 13:58

Copilot finished work on behalf of robfrank November 5, 2025 13:58

robfrank requested a review from lvca November 5, 2025 14:14

fix pre-commit errors

ef5b68f

Copilot started work on behalf of robfrank November 5, 2025 15:21 View session

Copilot finished work on behalf of robfrank November 5, 2025 15:49

Copilot started work on behalf of robfrank November 5, 2025 16:01 View session

Copilot finished work on behalf of robfrank November 5, 2025 16:06

fix pre-commit errors

9ca8c38

robfrank marked this pull request as ready for review November 5, 2025 16:58

robfrank merged commit edc11ec into main Nov 5, 2025
19 of 22 checks passed

robfrank deleted the copilot/fix-array-concatenation-error branch November 5, 2025 16:59

robfrank pushed a commit that referenced this pull request Nov 10, 2025

#1367 Fix SQL parser to accept nested projections in array concatenat…

9f41534

…ion expressions (#2755) (cherry picked from commit edc11ec)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix SQL parser to accept nested projections in array concatenation expressions#2755

Fix SQL parser to accept nested projections in array concatenation expressions#2755
robfrank merged 9 commits intomainfrom
copilot/fix-array-concatenation-error

Copilot AI commented Nov 5, 2025 •

edited

Loading

Uh oh!

mergify bot commented Nov 5, 2025 •

edited

Loading

Uh oh!

codacy-production bot commented Nov 5, 2025 •

edited

Loading

Uh oh!

robfrank commented Nov 5, 2025

Uh oh!

Copilot AI commented Nov 5, 2025

Uh oh!

robfrank commented Nov 5, 2025

Uh oh!

Copilot AI commented Nov 5, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

Copilot AI commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What Works ✅

Root Cause

Fix

OS and JDK Version:

Expected behavior

Actual behavior

Steps to reproduce

Uh oh!

mergify bot commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🧪 CI Insights

🟢 All jobs passed!

Uh oh!

codacy-production bot commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage summary from Codacy

See diff coverage on Codacy

See your quality gate settings Change summary preferences

Uh oh!

robfrank commented Nov 5, 2025

❌ com.arcadedb.query.sql.ArrayConcatNestedProjectionTest

Uh oh!

Copilot AI commented Nov 5, 2025

Uh oh!

robfrank commented Nov 5, 2025

Uh oh!

Copilot AI commented Nov 5, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Nov 5, 2025 •

edited

Loading

mergify bot commented Nov 5, 2025 •

edited

Loading

codacy-production bot commented Nov 5, 2025 •

edited

Loading