add special handling for sqlite dbs with last, first, select, length#17415
add special handling for sqlite dbs with last, first, select, length#17415fdncred merged 6 commits intonushell:mainfrom
last, first, select, length#17415Conversation
|
Very, very nice. Hate to get greedy, but would it make sense to extend this to |
|
what would be the equivalent sqlite sql for slice? |
|
|
|
Probably not worth the extra effort. And yes, |
It handles them by ignoring the step element, so it can be translated into |
|
ignoring the step element sounds like a bug |
|
😰 hope this goes well, let's give 'er a try. |
(total tangent, since I agree we shouldn't worry about it for this PR) Ah yeah, I missed that until @fdncred mentioned it today in Open Mic. Just in case you ever need it, though, the following works, and is similar to what I use in ls | select ...(0..4..10 | into list)Using the use std-rfc/tables *
ls | select slices 0..4..10The only "gotcha" is that Note that ls | select slices 0..2..6 7..8 |
This PR is a follow-on of these PRs to hopefully finish the last fixing of the sqlite pushdown. The biggest part of this fix is creating mini-sqlite-column-projection code for when sqlite column names are aliased in rust code like `command_line as command`. It's not a fully sql parser but it looks for column aliasing so it knows how to select the data properly. I've tried to comment the code verbosely to help us all remember what it's doing. - #17415 - #17645 - #17680 - #17668 I also took the opportunity to refactor the push down calls from filter commands so that they're easier to understand by calling a new pipeline function that does the same thing but in a little cleaner way. There's also a bunch of new tests around `open some_sqlite_file.db`. Before this PR you would get this with `history | select command` because `command` is an alias to `command_line`. ```nushell ❯ history | select command Error: nu::shell::error × Failed to execute query ╭─[repl_entry #1:1:11] 1 │ history | select command · ───┬── · ╰── no such column: command in SELECT command FROM [history] at offset 7 ╰──── ``` After, it just works (tm) ## Release notes summary - What our users need to know Refactor sqlite push down and apply to (nearly) all filter commands. ## Tasks after submitting N/A
This PR introduces some pushdown optimizations with
last,first,select, andlengthwhen used with thehistorycommand and sqlite databases.Before
After
Note the improvement of
history | lastgoing from 90,000us to 219us.Release notes summary - What our users need to know
This PR introduces some pushdown optimizations with
last,first,select, andlengthwhen used with thehistorycommand and sqlite databases.Tasks after submitting
N/A