Skip to content

[ES|QL] Remove EsqlDataTypes#111089

Merged
ioanatia merged 18 commits intoelastic:mainfrom
ioanatia:esql_data_types_part_2
Jul 24, 2024
Merged

[ES|QL] Remove EsqlDataTypes#111089
ioanatia merged 18 commits intoelastic:mainfrom
ioanatia:esql_data_types_part_2

Conversation

@ioanatia
Copy link
Copy Markdown
Member

follows up on #111016 (comment) and moves areCompatible to DataType.

furthermore it completely removes EsqlDataTypes which I hope won't be too disruptive 🙈

@ioanatia ioanatia added >non-issue Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) :Analytics/ES|QL AKA ESQL v8.16.0 labels Jul 19, 2024
@ioanatia ioanatia requested review from bpintea and not-napoleon July 19, 2024 11:19
@ioanatia ioanatia marked this pull request as ready for review July 19, 2024 11:19
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

Copy link
Copy Markdown
Member

@nik9000 nik9000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome. @not-napoleon will probably want to see it too.

Copy link
Copy Markdown
Member

@luigidellaquila luigidellaquila left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thank you very much @ioanatia!

I left only one comment about a method name that doesn't really match its logic.

@ioanatia ioanatia requested a review from luigidellaquila July 19, 2024 13:46
Copy link
Copy Markdown
Member

@luigidellaquila luigidellaquila left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Copy link
Copy Markdown
Contributor

@astefan astefan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've looked especially at the change in the Analyzer regarding isPrimitive method call and when we removed UNSUPPORTED from the isPrimitive method and why.

Without testing if everything is green again, we should keep isPrimitive with the original implementation (no UNSUPPORTED data type there: return t != OBJECT && t != NESTED).
The "other" isPrimitive method (which is sad it exists) should be renamed to isPrimitiveAndSupported and it should have the same implementation as of now (meaning return isPrimitive(t) && t != UNSUPPORTED). They should both live in DataType class.
Thanks.

Copy link
Copy Markdown
Member

@not-napoleon not-napoleon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I've been meaning to do this refactor for a while, thank you for getting to it!

@not-napoleon
Copy link
Copy Markdown
Member

@astefan I don't think we need both isPrimitive and isPrimitiveAndSupported. It looks to me like the only usage of the version of isPrimitive currently in DataType (which is the one that returns false for UNSUPPORTED) is this line:

 return isPrimitive(from) && isPrimitive(to) && converterFor(from, to) != null;

Which occurs in EsqlDataTypeConverter#canConvert() and DataTypeConverter#canConvert() (another good target for merging classes). The very next clause, converterFor(from, to) != null will be false for UNSUPPORTED as well, so in practice the behavior of isPrimitive for that value doesn't matter.

Given that, I think we can adopt the EsqlDataTypes version of isPrimitive with no change to current behavior, and that removes one bit of complexity. I do think it's worth doing some archeology at some point to figure out if we intended to allow UNSUPPORTED as a primitive type in the Analyzer, it's not at all clear to me why that's correct.

More generally, I am concerned that these functions using a != assert incorrectly opts in new data types. Like, is it correct that isPrimitive(PARTIAL_AGG) == true? I don't think that's something we should address in this PR though, as it would be a behavior change.

@ioanatia
Copy link
Copy Markdown
Member Author

More generally, I am concerned that these functions using a != assert incorrectly opts in new data types. Like, is it correct that isPrimitive(PARTIAL_AGG) == true? I don't think that's something we should address in this PR though, as it would be a behavior change.

Thank you all for the feedback!
I went with @astefan's suggestion in the end - I don't have enough context on how isPrimitive is being used and I was aiming to keep this PR as a simple refactor rather than introduce any behavior change.
@not-napoleon @astefan maybe one of you can follow up on the comment from @not-napoleon after we merge this PR.

Copy link
Copy Markdown
Contributor

@bpintea bpintea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also think this is the right way to merge the classes.

Like, is it correct that isPrimitive(PARTIAL_AGG) == true?

Probably not, but I'd think this would be a isPrimitive() concern. There is a need to exclude UNSUPPORTED privitives.

Otoh, I think I'd have called the function isSupportedPrimitive() (since it's not isPrimitiveAndNotUnsupported()), but that's a nit, AFAI'm concerned it can be merged as is.

Copy link
Copy Markdown
Contributor

@astefan astefan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks

@ioanatia ioanatia merged commit 5ce265a into elastic:main Jul 24, 2024
weizijun added a commit to weizijun/elasticsearch that referenced this pull request Jul 25, 2024
* main: (39 commits)
  Update README.asciidoc (elastic#111244)
  ESQL: INLINESTATS (elastic#109583)
  ESQL: Document a little of `DataType` (elastic#111250)
  Relax assertions in segment level field stats (elastic#111243)
  LogsDB data generator - support nested object field (elastic#111206)
  Validate `Authorization` header in Azure test fixture (elastic#111242)
  Fixing HistoryStoreTests.testPut() and testStoreWithHideSecrets() (elastic#111246)
  [ESQL] Remove Named Expcted Types map from testing infrastructure  (elastic#111213)
  Change visibility of createWriter to allow tests from a different package to override it (elastic#111234)
  [ES|QL] Remove EsqlDataTypes (elastic#111089)
  Mute org.elasticsearch.repositories.azure.AzureBlobContainerRetriesTests testReadNonexistentBlobThrowsNoSuchFileException elastic#111233
  Abstract codec lookup by name, to make CodecService extensible (elastic#111007)
  Add HTTPS support to `AzureHttpFixture` (elastic#111228)
  Unmuting tests related to free_context action being processed in ESSingleNodeTestCase (elastic#111224)
  Upgrade Azure SDK (elastic#111225)
  Collapse transport versions for 8.14.0 (elastic#111199)
  Make sure contender uses logs templates (elastic#111183)
  unmute HistogramPercentileAggregationTests.testBoxplotHistogram (elastic#111223)
  Refactor Quality Assurance test infrastructure (elastic#111195)
  Mute org.elasticsearch.xpack.restart.FullClusterRestartIT testDisableFieldNameField {cluster=UPGRADED} elastic#111222
  ...

# Conflicts:
#	server/src/main/java/org/elasticsearch/TransportVersions.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Analytics/ES|QL AKA ESQL >non-issue Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v8.16.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants