Map List<T> operations to PostgreSQL arrays#541
Map List<T> operations to PostgreSQL arrays#541austindrenski wants to merge 4 commits intonpgsql:devfrom
Conversation
758f808 to
bcbfab6
Compare
bcbfab6 to
07cdf5c
Compare
07cdf5c to
f57ea42
Compare
2f3c10d to
917c184
Compare
917c184 to
8d101b1
Compare
fcacb79 to
fd4ef6d
Compare
8c5bbad to
fe624da
Compare
d30c512 to
ea3f89f
Compare
|
@roji I've split out some of the unrelated changes and rebased. The last review commented on changes to the query pipeline, but I think most of those should be reverted here. This PR still doesn't address multidimensional arrays, but given the current size of it, I'd prefer to punt multidimensional support (as well as |
2fd689b to
9d77bf3
Compare
There was a problem hiding this comment.
I finally got back to this and give it another round. We need to hurry if we really want to get this into 2.2.
Just a reminder to always keep in mind to avoid adding stuff for completeness's sake, and wait until (several) users request things from us - I think some features introduced in this PR aren't very likely to be used etc.
src/EFCore.PG/Query/ExpressionTranslators/Internal/NpgsqlArrayFragmentTranslator.cs
Show resolved
Hide resolved
src/EFCore.PG/Query/ExpressionTranslators/Internal/NpgsqlArrayMemberTranslator.cs
Outdated
Show resolved
Hide resolved
src/EFCore.PG/Query/ExpressionTranslators/Internal/NpgsqlArrayMemberTranslator.cs
Outdated
Show resolved
Hide resolved
| /// A translated expression or null if no translation is supported. | ||
| /// </returns> | ||
| [CanBeNull] | ||
| Expression ArrayInstanceHandler([NotNull] MemberExpression expression) |
There was a problem hiding this comment.
Is Array.Length really given as a MemberExpression in the expression tree? Unless I'm mistaken it has its own NodeType. This is why I had to have special support for it in NpgsqlSqlTranslatingExpressionVisitor.
src/EFCore.PG/Query/ExpressionTranslators/Internal/NpgsqlArrayMemberTranslator.cs
Outdated
Show resolved
Hide resolved
src/EFCore.PG/Query/ExpressionVisitors/NpgsqlSqlTranslatingExpressionVisitor.cs
Outdated
Show resolved
Hide resolved
| protected override Expression VisitBinary(BinaryExpression expression) | ||
| { | ||
| if (expression.NodeType == ExpressionType.ArrayIndex) | ||
| var left = Visit(expression.Left); |
There was a problem hiding this comment.
The point of the previous code was to immediately delegate to base.VisitBinary() in all cases, unless when the expression's node type is ArrayIndex. In the new code you call Visit() on Left and Right, and then it gets called again if the expression isn't ArrayIndex or Index. This could be bad for performance and may potentially have other side-effects - when we're changing the query pipeline I'd rather be as little invasive as we can.
So I'd start with a switch on the NodeType, which immediately returns base.VisitBinary() for anything but what we handle here.
There was a problem hiding this comment.
I've reworked this into a switch to cut down on the possibility of wasted visits. How does it look now?
src/EFCore.PG/Query/ExpressionVisitors/NpgsqlSqlTranslatingExpressionVisitor.cs
Show resolved
Hide resolved
| { | ||
| switch (expression) | ||
| { | ||
| case ArrayAnyAllExpression e: |
There was a problem hiding this comment.
I'm confused... NpgsqlArrayFragmentTranslator now constructs and returns ArrayAnyAllExpressions, are they expected to find their way back here in the SqlTranslatingExpressionVisitor? Is this because we assume the fragment translators execute before this method? Either way, the whole point of having the FragmentTranslator is to cleanly encapsulate all the array logic in one place, but this seems to spread it back around...
The situation seems to be similar with the other custom expression we have, but I may be misunderstanding completely.
There was a problem hiding this comment.
I based this on the implementation in SqlTranslatingExpressionVisitor which does similar work on NodeType.Extension expressions defined upstream.
The fragment translators always run first, so the idea is to package up a bunch of complicated nodes into one custom expression that we can easily identify and handle later on in the normal translator workflow.
It does spread around some logic, but this is still advantageous for the NpgsqlSqlTranslatingExpressionVisitor, because we're just performing normal expression visitor operations on ArrayAnyAllExpression (e.g. visit members, return original or modified).
Note we do have one remaining piece of complicated ArrayAnyAllExpression logic outside the fragment translator, which involves translating a ContainsResultOperator. I want to refactor that into a fragment translator too, but I'd like to deal with that in a later PR.
|
@roji Thanks for the review. I'm (slowly) working my way through it, and hope to have it addressed within the next day or two. |
9d77bf3 to
c6ac0e0
Compare
cd548cd to
b978f59
Compare
TODO: - Investigate `Array.Length` as member vs node. - Decide on whether to defer to client eval for pre-9.4 versions. - Recheck/revert logic in `NpgsqlSqlTranslatingExpressionVisitor` to make sure the query pipeline is modified as minimally as possible.
b978f59 to
75293ec
Compare
|
@roji I've finished going through that latest review. Some notable improvements based on your comments:
I would like to get this incorporated during the preview cycles for 3.0, but I don't think there's any real need to merge it before then, so no need to rush on the re-review. |
|
@austindrenski I just re-noticed this... It's really unfortunately we didn't get this merged before the new query pipeline work. Do you have time to rebase this to the latest version and do the porting? |
|
Any movement on getting this into the library? It would be good to not have to worry about arrays T[] and just use List in the code base I'm working on. |
|
Closing as there has been no comment from @austindrenski for a while, and this PR would probably need to be redone anyway after the query pipeline redo of 3.0. The tracking issue here is #395. |

This is a subset of the work from #431 refactored without
IQueryOptimizerorNpgsqlExistsToAnyRewritingExpressionVisitorper #460.Compatibility
=<><><=>=@><@&&||array_append(anyarray, anyelement)||array_cat(anyarray, anyarray)||array_dims(anyarray)array_fill(anyelement, int[] [, int[]])array_length(anyarray, int)array_lower(anyarray, int)array_ndims(anyarray)array_position(anyarray, anyelement [, int])array_positions(anyarray, anyelement)array_prepend(anyelement, anyarray)||array_remove(anyarray, anyelement)array_replace(anyarray, anyelement, anyelement)array_to_string (anyarray, text)array_to_string(anyarray, text [, text])array_upper(anyarray, int)string_to_array (text, text)string_to_array(text, text [, text])Postponed
array_agg(expression)cardinality(anyarray)unnest(anyarray)unnest(anyarray, anyarray [, ...])Related
#395
#431
#460
#542
#543
#544
#545
#549
#551
#552