Skip to content

Conversation

@samuelcolvin
Copy link
Contributor

@samuelcolvin samuelcolvin commented Aug 15, 2024

Which issue does this PR close?

NA

Rationale for this change

Support latest features of sqlparser-rs.

What changes are included in this PR?

Uprev sqlparser-rs to v0.50.0

Are these changes tested?

No new tests needed AFAIK.

Are there any user-facing changes?

Some new SQL syntax supported.

@github-actions github-actions bot added the sql SQL Planner label Aug 15, 2024
Value::SingleQuotedString(s) => Some(s.to_string()),
Value::DollarQuotedString(s) => Some(s.to_string()),
Value::Number(_, _) | Value::Boolean(_) => Some(value.to_string()),
Value::UnicodeStringLiteral(s) => Some(s.to_string()),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really understand why value_to_string supports (returns Some(String)) for some types of string, but not others, so I'm not clear whether Value::UnicodeStringLiteral should return Some or None?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect this was not carefully designed and likely is not an intentional oversight

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think what you have here is ok

@samuelcolvin
Copy link
Contributor Author

Waiting for decision on apache/datafusion-sqlparser-rs#1382.

@alamb
Copy link
Contributor

alamb commented Aug 15, 2024

BTW I double checked with this branch that you can still use id as a column name without double quotes

$ git status
On branch sqlparser-main
nothing to commit, working tree clean
~/Software/datafusion/datafusion-cli$ cargo run
...
> create table foo(id int) as values (1);
0 row(s) fetched.
Elapsed 0.030 seconds.

> select id from foo;
+----+
| id |
+----+
| 1  |
+----+
1 row(s) fetched.
Elapsed 0.007 seconds.

>

@samuelcolvin samuelcolvin changed the title Support HEAD of sqlparser-rs main sqlparser-rs v0.50.0 Aug 16, 2024
@samuelcolvin samuelcolvin marked this pull request as ready for review August 16, 2024 11:12
@alamb alamb changed the title sqlparser-rs v0.50.0 Update to sqlparser-rs v0.50.0 Aug 16, 2024
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @samuelcolvin -- this looks great to me. I had some style suggestions, but they aren't required from my perspective to merge this PR

self.parse_value(value, planner_context.prepare_param_data_types())
}
SQLExpr::Extract { field, expr } => {
SQLExpr::Extract { field, expr, .. } => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recommend explicitly listing out the other fields here and returning a NotImplemented error if there is some new syntax that DataFusion doesn't support.

Otherwise what we have seen a few times in the past is that the parser accepts some new syntax element but DataFusion just igores it silently

SQLDataType::Bytea => Ok(DataType::Binary),
SQLDataType::Interval => Ok(DataType::Interval(IntervalUnit::MonthDayNano)),
SQLDataType::Struct(fields) => {
SQLDataType::Struct(fields, _) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likewise here I think it would be good to name the field like

Suggested change
SQLDataType::Struct(fields, _) => {
SQLDataType::Struct(fields, _bracket_type) => {

So it is easier to understand by reading just the code what is being ignored

describe_alias: DescribeAlias::Describe, // only parse 'DESCRIBE table_name' and not 'EXPLAIN table_name'
hive_format: _,
table_name,
..
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

per comment above, I recommend keeping the fields named explcitly

Value::SingleQuotedString(s) => Some(s.to_string()),
Value::DollarQuotedString(s) => Some(s.to_string()),
Value::Number(_, _) | Value::Boolean(_) => Some(value.to_string()),
Value::UnicodeStringLiteral(s) => Some(s.to_string()),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think what you have here is ok

2020

query error
query R
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥳 🦜

@alamb alamb merged commit 7fa7689 into apache:main Aug 17, 2024
@alamb
Copy link
Contributor

alamb commented Aug 17, 2024

Thanks again @samuelcolvin

@samuelcolvin samuelcolvin deleted the sqlparser-main branch August 17, 2024 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sql SQL Planner sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants