Skip to content

Add GET_FORMAT Function To OpenSearch SQL Plugin#1299

Merged
MaxKsyunz merged 3 commits intoopensearch-project:mainfrom
Bit-Quill:integ-add-get_format-function
Feb 15, 2023
Merged

Add GET_FORMAT Function To OpenSearch SQL Plugin#1299
MaxKsyunz merged 3 commits intoopensearch-project:mainfrom
Bit-Quill:integ-add-get_format-function

Conversation

@GabeFernandez310
Copy link
Copy Markdown
Contributor

@GabeFernandez310 GabeFernandez310 commented Jan 27, 2023

Signed-off-by: GabeFernandez310 Gabriel.Fernandez@improving.com

Description

Adds the get_format function to the OpenSearch SQL Plugin. It's implementation is aligned with the MySQL Documentation.

Note: This currently only supports a USA format for DATE, TIME, TIMESTAMP and DATETIME. Support and testing for other formats will be completed in follow-up work

Examples:

opensearchsql> SELECT get_format(DATE, 'USA');
fetched rows / total rows = 1/1
+---------------------------+
| get_format(DATE, 'USA')   |
|---------------------------|
| %m.%d.%Y                  |
+---------------------------+

opensearchsql> SELECT DATE_FORMAT('2003-10-03',GET_FORMAT(DATE,'USA'));
fetched rows / total rows = 1/1
+----------------------------------------------------+
| DATE_FORMAT('2003-10-03',GET_FORMAT(DATE,'USA'))   |
|----------------------------------------------------|
| 10.03.2003                                         |
+----------------------------------------------------+

opensearchsql> SELECT DATE_FORMAT('2003-10-03', null);
fetched rows / total rows = 1/1
+-----------------------------------+
| DATE_FORMAT('2003-10-03', null)   |
|-----------------------------------|
| null                              |
+-----------------------------------+

Issues Resolved

#722

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@GabeFernandez310 GabeFernandez310 requested a review from a team as a code owner January 27, 2023 20:24
@Yury-Fridlyand Yury-Fridlyand added enhancement New feature or request backport 2.x labels Jan 27, 2023
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Jan 27, 2023 •
edited
Loading

Codecov Report

Merging #1299 (1d052f8) into main (4b3cdbb) will increase coverage by 0.00%.
The diff coverage is 100.00%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff            @@
##               main    #1299   +/-   ##
=========================================
  Coverage     98.36%   98.37%           
- Complexity     3643     3649    +6     
=========================================
  Files           343      343           
  Lines          9017     9041   +24     
  Branches        585      585           
=========================================
+ Hits           8870     8894   +24     
  Misses          142      142           
  Partials          5        5
Flag Coverage Δ
sql-engine 98.37% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...c/main/java/org/opensearch/sql/expression/DSL.java 100.00% <100.00%> (ø)
...arch/sql/expression/datetime/DateTimeFunction.java 100.00% <100.00%> (ø)
...h/sql/expression/function/BuiltinFunctionName.java 100.00% <100.00%> (ø)
...pensearch/sql/sql/parser/AstExpressionBuilder.java 100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@acarbonetto acarbonetto mentioned this pull request Jan 31, 2023
Closed
penghuo
penghuo reviewed Jan 31, 2023
View reviewed changes
core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java Outdated
* @return ExprValue..
*/
private ExprValue exprGetFormat(ExprValue type, ExprValue format) {
String key = type.stringValue() + "-" + format.stringValue();
Copy link
Copy Markdown
Collaborator

@penghuo penghuo Jan 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does https://guava.dev/releases/19.0/api/docs/com/google/common/collect/Table.html solve the problem without concat the string value?

Copy link
Copy Markdown
Contributor Author

@GabeFernandez310 GabeFernandez310 Feb 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed here, and here

penghuo
penghuo reviewed Jan 31, 2023
View reviewed changes
docs/user/dql/functions.rst Outdated

Argument type: TYPE, STRING
TYPE must be one of the following tokens: [DATE, TIME, DATETIME, TIMESTAMP].
STRING must be one of the following tokens: ["USA", "JIS", "ISO", "EUR", "INTERNAL"] (" can be replaced by ').
Copy link
Copy Markdown
Collaborator

@penghuo penghuo Jan 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only "USA" support for now, right?

Copy link
Copy Markdown
Contributor Author

@GabeFernandez310 GabeFernandez310 Jan 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes. I will adjust the documentation for now and update it once the other formats are added.

Copy link
Copy Markdown
Contributor Author

@GabeFernandez310 GabeFernandez310 Feb 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed here

@GabeFernandez310
Added Implementation and Testing
f97eea4 
Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>
@GabeFernandez310 GabeFernandez310 force-pushed the integ-add-get_format-function branch from d4aacb4 to f97eea4 Compare January 31, 2023 23:36
@GabeFernandez310
Copy link
Copy Markdown
Contributor Author

GabeFernandez310 commented Feb 1, 2023

I had to rebase while addressing PR comments. There appears to be a failing test which exists on the main branch as well.

dai-chen
dai-chen reviewed Feb 1, 2023
View reviewed changes
Copy link
Copy Markdown
Collaborator

@dai-chen dai-chen left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May I ask where this comes from? Is it part of deprecate legacy engine or other issue? Because I didn't find it in our codebase.

@GabeFernandez310
Copy link
Copy Markdown
Contributor Author

GabeFernandez310 commented Feb 1, 2023

May I ask where this comes from? Is it part of deprecate legacy engine or other issue? Because I didn't find it in our codebase.

The get_format function was listed along with others in issue #722 to support date and time functions.

@Yury-Fridlyand
Copy link
Copy Markdown
Collaborator

Yury-Fridlyand commented Feb 1, 2023

It is one of MySQL functions which we're going to support.

@GabeFernandez310 GabeFernandez310 marked this pull request as draft February 2, 2023 18:14
@Yury-Fridlyand Yury-Fridlyand mentioned this pull request Feb 6, 2023
Closed
87 tasks
@GabeFernandez310 GabeFernandez310 marked this pull request as ready for review February 9, 2023 17:52
@dai-chen
Copy link
Copy Markdown
Collaborator

dai-chen commented Feb 10, 2023 •
edited
Loading

@GabeFernandez310 Shall we merge IT fix from main before merging this?

@MaxKsyunz MaxKsyunz merged commit 72547f4 into opensearch-project:main Feb 15, 2023
@MaxKsyunz MaxKsyunz deleted the integ-add-get_format-function branch February 15, 2023 02:37
@opensearch-trigger-bot opensearch-trigger-bot bot mentioned this pull request Feb 15, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@penghuo penghuo penghuo left review comments

@dai-chen dai-chen dai-chen approved these changes

+1 more reviewer

@Yury-Fridlyand Yury-Fridlyand Yury-Fridlyand approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

backport 2.x enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@GabeFernandez310 @codecov-commenter @Yury-Fridlyand @dai-chen @penghuo @MaxKsyunz