Skip to content

allow records to have type annotations#8914

Merged
fdncred merged 3 commits intonushell:mainfrom
1Kinoti:record_shape
Apr 26, 2023
Merged

allow records to have type annotations#8914
fdncred merged 3 commits intonushell:mainfrom
1Kinoti:record_shape

Conversation

@1Kinoti
Copy link
Copy Markdown
Contributor

@1Kinoti 1Kinoti commented Apr 17, 2023

Description

follow up to #8529
cleaned up version of #8892

  • the original syntax is okay
def okay [rec: record] {}
  • you can now add type annotations for fields if you know them before hand
def okay [rec: record<name: string>] {}
  • you can specify multiple fields
def okay [person: record<name: string age: int>] {}

# an optional comma is allowed
def okay [person: record<name: string, age: int>] {}
  • if annotations are specified, any use of the command will be type checked against the specified type
def unwrap [result: record<ok: bool, value: any>] {}

unwrap {ok: 2, value: "value"}

# errors with

Error: nu::parser::type_mismatch

  × Type mismatch.
   ╭─[entry #4:1:1]
 1  unwrap {ok: 2, value: "value"}
   ·         ───────┬─────
   ·                    ╰── expected record<ok: bool, value: any>, found record<ok: int, value: string>
   ╰────

here the error is in the ok field, since any is coerced into any type
as a result unwrap {ok: true, value: "value"} is okay

  • the key must be a string, either quoted or unquoted
def err [rec: record<{}: list>] {}

# errors with
Error:
  × `record` type annotations key not string
   ╭─[entry #7:1:1]
 1  def unwrap [result: record<{}: bool, value: any>] {}
   ·                            ─┬
   ·                             ╰── must be a string
   ╰────
  • a key doesn't have to have a type in which case it is assumed to be any
def okay [person: record<name age>] {}

def okay [person: record<name: string age>] {}
  • however, if you put a colon, you have to specify a type
def err [person: record<name: >] {}

# errors with
Error: nu::parser::parse_mismatch

  × Parse mismatch during operation.
   ╭─[entry #12:1:1]
 1  def unwrap [res: record<name: >] { $res }
   ·                             
   ·                             ╰── expected type after colon
   ╰────

User-Facing Changes

[BREAKING CHANGES]

  • this change adds a field to SyntaxShape::Record so any plugins that used it will have to update and include the field. though if you are unsure of the type the record expects, SyntaxShape::Record(vec![]) will suffice

@sophiajt
Copy link
Copy Markdown
Contributor

sophiajt commented Apr 17, 2023

We were talking about this in a recent design meeting, and we decided to move away from the <T> generic style. Instead, we'd list off the types in a name:value style, closer to a call:

table(name: string, age: int)

or

record(name: string, id: string)

or

list(int)

We talked a bit about making the last one [int] but I think decided to go with the readable names for now.

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 17, 2023

Codecov Report

Merging #8914 (5131248) into main (a1b7261) will decrease coverage by 0.34%.
The diff coverage is 86.62%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #8914      +/-   ##
==========================================
- Coverage   68.57%   68.24%   -0.34%     
==========================================
  Files         636      636              
  Lines      101772   101881     +109     
==========================================
- Hits        69788    69526     -262     
- Misses      31984    32355     +371     
Impacted Files Coverage Δ
crates/nu-protocol/src/ty.rs 80.50% <72.22%> (-0.62%) ⬇️
crates/nu-parser/src/parser.rs 85.84% <85.47%> (+0.31%) ⬆️
crates/nu-parser/src/type_check.rs 30.72% <92.30%> (+1.40%) ⬆️
crates/nu-cmd-lang/src/core_commands/error_make.rs 62.98% <100.00%> (+0.83%) ⬆️
crates/nu-command/src/env/load_env.rs 81.11% <100.00%> (ø)
crates/nu-protocol/src/syntax_shape.rs 83.96% <100.00%> (+2.85%) ⬆️

... and 3 files with indirect coverage changes

@1Kinoti
Copy link
Copy Markdown
Contributor Author

1Kinoti commented Apr 17, 2023

i think this is easy to fix! let me do that

@sophiajt
Copy link
Copy Markdown
Contributor

Nice, I'm glad it cleans up some code, too.

@bobhy
Copy link
Copy Markdown
Contributor

bobhy commented Apr 18, 2023

Wrapping these shape decls for table and record in parens seems odd compared to def <name> [ <arglist> ]. Why should these delimiters be different?
But rather than change the parens to square brackets on shape decls, I'd go the other way and use parens in def arg decls. I've always thought the square brackets in this context were non-industry-standard. I get that it's evocative of a list, but still...
Is it just too late to change?

@amtoine
Copy link
Copy Markdown
Member

amtoine commented Apr 18, 2023

can't wait to see these new annotations in nushell @1Kinoti 🤩

@sophiajt
Copy link
Copy Markdown
Contributor

Wrapping these shape decls for table and record in parens seems odd compared to def <name> [ <arglist> ]. Why should these delimiters be different? But rather than change the parens to square brackets on shape decls, I'd go the other way and use parens in def arg decls. I've always thought the square brackets in this context were non-industry-standard. I get that it's evocative of a list, but still... Is it just too late to change?

Not sure what you're asking, but if you are asking about reusing delimiters, you wouldn't want the same delimiter for both the argument list and the type parameters to the types. Readability improves by using two separate ones[foo: Record[name: string, id: int]] is much harder to read than [foo: Record(name: string, id: int)]

@bobhy
Copy link
Copy Markdown
Contributor

bobhy commented Apr 18, 2023

Let me take the broader question up on Discord, it's not a problem with this PR specifically. It's great for Nu to be able to annotate record and table shapes!

@fdncred
Copy link
Copy Markdown
Contributor

fdncred commented Apr 19, 2023

I remember having this discussion about this change but what I don't remember is why we changed from <> to (). Seems like list<int> is more intuitive than list(int) since the parentheses makes it looks like list() is a function with the int parameter.

Also, since we allow optional parentheses in custom command definition, like:

def ok (rec: record(name: string)) { $rec }

i'm not sure that's more readable than this below. It kind of feels the same.

def ok [rec: record[name: string]] { $rec }

But

def ok (rec: record<name: string>) { $rec }

and

def ok [rec: record<name: string>] { $rec }

both still read as rec is the parameter with a type of record<name: string>.

I'm not willing to die on this hill, but I think the "alligators" <> are more intuitive than the parentheses, in this use case.

Update: One big thing that I forgot to mention was that we've been considering adding function calling that looks like command(param). So, this is the main reason why I'm in the list<int> camp.

@sholderbach sholderbach added the A:type-system Problems or features related to nushell's type system label Apr 19, 2023
@amtoine
Copy link
Copy Markdown
Member

amtoine commented Apr 19, 2023

I'm not willing to die on this hill, but I think the "alligators" <> are more intuitive than the parentheses, in this use case.

i agree with this 👍
maybe more: "i think the () are less intuitive and look like function calls" 😋

@bobhy
Copy link
Copy Markdown
Contributor

bobhy commented Apr 19, 2023

I think the reason we're going back and forth on this is that we lack an agreed overarching model for the language.
Do we want NU to "just look right" as a thing unto itself? or should it "remind users of" some other language, and, if so, which (language or languages)?
For now, I say, ship it! We have a working feature (modulo the final syntax), and the strong possibility that resolution of the syntax question will require more wide-spread changes when it comes. Might as well get the benefit of the (stronger typing) behavior in the meantime.

@fdncred fdncred marked this pull request as draft April 20, 2023 13:18
@sophiajt
Copy link
Copy Markdown
Contributor

sophiajt commented Apr 22, 2023

Just to 👍 what @fdncred said - after we talked about it in a recent design meeting, we realised that that () for the type parameter would look too confusing if folks used the alternate parameter syntax def foo (x: list(int)) { ... }

We've been talking about also adding an alternative call syntax like foo([1, 2, 3]) so that might encourage more folks to use the () syntax.

As a result, seems like <> is our best bet for the type parameter syntax for now.

Sorry about going back and forth.

@amtoine
Copy link
Copy Markdown
Member

amtoine commented Apr 22, 2023

and using <> back might have the side-effect of making this PR not a breaking-change 👍

@1Kinoti
Copy link
Copy Markdown
Contributor Author

1Kinoti commented Apr 22, 2023

Just to +1 what @fdncred said - after we talked about it in a recent design meeting, we realised that that () for the type parameter would look too confusing if folks used the alternate parameter syntax def foo (x: list(int)) { ... }

We've been talking about also adding an alternative call syntax like foo([1, 2, 3]) so that might encourage more folks to use the () syntax.

As a result, seems like <> is our best bet for the type parameter syntax for now.

Sorry about going back and forth.

no problem @jntrnr , let me do that

@1Kinoti 1Kinoti marked this pull request as ready for review April 22, 2023 20:13
@fdncred
Copy link
Copy Markdown
Contributor

fdncred commented Apr 22, 2023

Thanks @1Kinoti. We're close to a release (April 25), and we have a 2 day freeze before that day, so we probably won't land this until after that. I'd like to have more than a day to test it. Thanks again for all your support and PRs!!!

@1Kinoti
Copy link
Copy Markdown
Contributor Author

1Kinoti commented Apr 22, 2023

Thanks @1Kinoti. We're close to a release (April 25), and we have a 2 day freeze before that day, so we probably won't land this until after that. I'd like to have more than a day to test it. Thanks again for all your support and PRs!!!

you're welcome @fdncred! after the release it is

@fdncred fdncred merged commit 77ca73f into nushell:main Apr 26, 2023
@1Kinoti 1Kinoti deleted the record_shape branch April 26, 2023 13:29
@fdncred
Copy link
Copy Markdown
Contributor

fdncred commented Apr 26, 2023

@1Kinoti Thanks again. I'm loving this change!

@amtoine
Copy link
Copy Markdown
Member

amtoine commented Apr 26, 2023

can't wait to play with them 🤩

sholderbach pushed a commit that referenced this pull request Jul 7, 2023
# Description

follow up to #8529 and #8914

this works very similarly to record annotations, only difference being
that

```sh
table<name: string>
      ^^^^  ^^^^^^
      |     | 
      |     represents the type of the items in that column
      |
      represents the column name
```
more info on the syntax can be found
[here](#8914 (comment))

# User-Facing Changes

**[BREAKING CHANGE]**
this change adds a field to `SyntaxShape::Table` so any plugins that
used it will have to update and include the field. though if you are
unsure of the type the table expects, `SyntaxShape::Table(vec![])` will
suffice
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A:type-system Problems or features related to nushell's type system status:wait-until-after-nushell-release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants