Skip to content

Feature url build_query accepts records with lists of strings#14073

Merged
fdncred merged 4 commits intonushell:mainfrom
adaschma:feature-build-query-lists
Oct 22, 2024
Merged

Feature url build_query accepts records with lists of strings#14073
fdncred merged 4 commits intonushell:mainfrom
adaschma:feature-build-query-lists

Conversation

@adaschma
Copy link
Copy Markdown
Contributor

Description

Swagger supports lists (a.k.a arrays) in query parameters:

https://swagger.io/docs/specification/v3_0/serialization/

It supports three different styles:

  • explode=true
  • spaceDelimited
  • pipeDelimited

With explode=true being the default and hence most common. It is the hardest to use inside of nushell, as the others are just a string join away. This commit adds lists with the explode=true format.

User-Facing Changes

Before:

: {a[]: [one two three], b: four} | url build-query                                                                          
Error: nu::shell::unsupported_input                                                                                          
                                                                                                                         
  × Unsupported input                                                                                                        
   ╭─[entry #33:1:1]                                                                                                         
 1 │ {a[]: [one two three], b: four} | url build-query                                                                       
   · ───────────────┬───────────────   ───────┬───────                                                                       
   ·                │                         ╰── Expected a record with string values                                       
   ·                ╰── value originates from here                                                                           
   ╰────

After:

: {a[]: [one two three], b: four} | url build-query                                                                          
a%5B%5D=one&a%5B%5D=two&a%5B%5D=three&b=four

Despite reading CONTRIBUTING.md I didn't get approval before making the change. My judgment is that this doesn't qualify as being "change something significantly".

Tests + Formatting

I added the Example instance for the automatic tests. I couldn't figure out how to add an Example for the error case, so I did that with manual testing. E.g.:

: {a[]: [one two [three]], b: four} | url build-query                                                                        

Error: nu::shell::unsupported_input                                                                                          

                                                                                                                             

  × Unsupported input                                                                                                        

   ╭─[entry #3:1:1]                                                                                                          

 1 │ {a[]: [one two [three]], b: four} | url build-query                                                                     

   · ────────────────┬────────────────   ───────┬───────                                                                     

   ·                 │                          ╰── Expected a record with list of string values                             

   ·                 ╰── value originates from here                                                                          

   ╰────

    : {a[]: [one two 3hr], b: four} | url build-query                                                                            

Error: nu::shell::unsupported_input                                                                                          

                                                                                                                             

  × Unsupported input                                                                                                        

   ╭─[entry #4:1:1]                                                                                                          

 1 │ {a[]: [one two 3hr], b: four} | url build-query                                                                         

   · ──────────────┬──────────────   ───────┬───────                                                                         

   ·               │                        ╰── Expected a record with list of string values                                 

   ·               ╰── value originates from here                                                                            

   ╰──── 

I ran the four cargo commands on my local machine. I had to run the tests with:

LANG=C and -j 1 and even then I got one failure:

thread 'commands::umkdir::mkdir_umask_permission' panicked at crates/nu-command/tests/commands/umkdir.rs:148:9:              
assertion `left == right` failed: Most *nix systems have 0o00022 as the umask. So directory permission should be 0o40755 = 0o

40777 & (!0o00022)
left: 16893
right: 16877

but this isn't related to this change (I seem to not be running most *nix system; and don't have a lot of RAM for the number of cores). The other three cargo commands didn't have errors or warnings.

After Submitting

I will add the new example to the documentation.

Open questions / possible future work

Things I noticed, and would like to mention and am open to adding, but don't think I am deep enough in nushell to do them pro-actively.

Add an argument for the other query parameter list styles

I don't know how frequent they are and I currently don't need them, so following KISS I didn't add them.

long input_span marked

In e.g.:

    : {a[]: [one two 3hr], b: four} | url build-query                                                                            

Error: nu::shell::unsupported_input                                                                                          

                                                                                                                             

  × Unsupported input                                                                                                        

   ╭─[entry #4:1:1]                                                                                                          

 1 │ {a[]: [one two 3hr], b: four} | url build-query                                                                         

   · ──────────────┬──────────────   ───────┬───────                                                                         

   ·               │                        ╰── Expected a record with list of string values                                 

   ·               ╰── value originates from here                                                                            

   ╰──── 

the entire record is marked as input_span instead of just the "3hr" that is causing the problem. Changing that would be trivial, but I'm not deep enough into nushell to understand all the consequences of changing that.

Error message says string values despite accepting numbers etc.

The error message said it only accepted strings despite accepting numbers etc. (anything it can coerce into string). I couldn't find a good wording myself and that was how it was before. I simply added a "list of strings".

Swagger supports lists (a.k.a arrays) in query parameters:

  https://swagger.io/docs/specification/v3_0/serialization/

It supports three different styles:

  - explode=true
  - spaceDelimited
  - pipeDelimited

With explode=true being the default and hence most common. It is also
the hardest to use inside nushell, as the others are just a `string
join` away. This commit adds lists with the explode=true format.
instead of `match ... { Ok(s) => Ok(s), _ => Err(...) }`. Reduces
nesting.
Copy link
Copy Markdown
Member

@sholderbach sholderbach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Nushell side of things looks good to me.
Not super familiar if there are any limitations or best practices for this style of query string we should take into account.

@sholderbach sholderbach added the A:networking All about our `http` and `url` commands and everything going accross the network. label Oct 17, 2024
"the a[] may be a bit confusing here. If you read it it might imply you
need to append it to be able to use the list form but it will work with
any key. (even if the consuming API may have some ideas about what it
permits there)"

Thanks: shoulderbach
adaschma added a commit to adaschma/nushell.github.io that referenced this pull request Oct 17, 2024
@adaschma
Copy link
Copy Markdown
Contributor Author

After Submitting

I will add the new example to the documentation.

I was going to add the example to nushell.github.io. But after reading its CONTRIBUTING.md my understanding is that no manual change is needed:

While most documentation is updated via commits to this repository, an important exception is the help documentation for all commands. These pages are generated automatically from the internal help in each command's .rs file. Please see the main Nushell repo and submit pull requests for changes to command documentation there.

Please, correct me if I'm wrong.

@sholderbach
Copy link
Copy Markdown
Member

Thanks for thinking about the documentation

If the documentation change is just part of the command page, you don't have to do anything. If it makes sense to add something to the book or cookbook feel free to do so.

@fdncred fdncred merged commit 04fed82 into nushell:main Oct 22, 2024
@fdncred
Copy link
Copy Markdown
Contributor

fdncred commented Oct 22, 2024

Thanks

@github-actions github-actions bot added this to the v0.100.0 milestone Oct 22, 2024
fdncred pushed a commit that referenced this pull request Oct 29, 2024
…join` and `url build-query` (#14173)

Addresses one of the points in #14162

# Description

Factors out part of the `url::build_query::to_url` function into a
separate function `url::query::record_to_qs()`, which is then used in
both `url::build_query` and `url::join`.

# User-Facing Changes

Like with `url build-query` (after #14073), `url join` will allow list
values in `params` and behavior of two commands will be same.

```nushell
> {a: ["one", "two"], b: "three"} | url build-query
"a=one&a=two&b=three"

> {scheme: "http", host: "host", params: {a: ["one", "two"], b: "three"}} | url join 
"http://host?a=one&a=two&b=three"
```

# Tests + Formatting

Added an example to `url join` for the new behavior.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A:networking All about our `http` and `url` commands and everything going accross the network.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants