Add json->sql deserialisation and execution.#6919
Merged
Mytherin merged 16 commits intoduckdb:masterfrom Apr 7, 2023
Merged
Conversation
…ore advanced queries but basics work
Mytherin
reviewed
Apr 4, 2023
Collaborator
Mytherin
left a comment
There was a problem hiding this comment.
Thanks for the PR! Looks good. Some minor comments below.
Member
Author
|
yo-yo @Mytherin |
Collaborator
|
Thanks! LGTM. Next up replacing the existing (de)serialize methods nad unifying all the (de)serialize infrastructure :) |
Wang-Ji20
reviewed
Jul 18, 2023
| void JsonDeserializer::ThrowTypeError(yyjson_val *val, const char *expected) { | ||
| auto actual = yyjson_get_type_desc(val); | ||
| auto &parent = Current(); | ||
| if (yyjson_is_obj(parent.val)) { |
Contributor
There was a problem hiding this comment.
These two branches don't throw anything, and the string msg is discarded right away. Is that a bug?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a continuation of #6647 and adds three new functions:
json_deserialize_sql(JSON): convert a json serialized sql select statement back into an (equivalent) sql statement string.json_execute_serialized_sql(JSON): execute a json serialized sql select statement directly as a table function - basically as if it were a subquery.PRAGMA json_execute_serialized_sql(VARCHAR): execute a json serialized sql select statement as if it were a top level statement through a pragma instead. Basically deserialises the given json back into a sql string which is then executed normally.I have tested this by hacking in a json ser/de roundtrip in the DeserializedStatementVerifier, and it passes the entire duckdb unit test suite, so I am reasonably confident that this implementation is correct and serializes e.g. all the types, expressions and values correctly, but I will add a proper deserialization/serialization verification pass when I add the binary FormatDe/Serializer in core duckdb.