as a user, state.key() and the ensuing concept of "public keys"* feel like a complex implementation detail that shouldn't be exposed to me. why not just accept a string directly? i'm guessing it's because you're using the span of the state.key() call, but in that case why not just use the span of the state() call?

*though there isn't really an explanation for that concept to begin with. i'm guessing that it's like the difference between new symbols (private) and registered symbols (public) in javascript?

At first glance, I'm with @cAttte here. Could you expand on the motivation for the new API surface?

Could you expand on the motivation for the new API surface?

From the discussion in #2425, it seemed to me like state(init) with a totally argument-independent key would be too unexpected, so I thought making the argument-independence explicit in state(init, key: state.key()) would make the API more intuitive.

I'm not 100% satisfied with the way I implemented state(init) (i.e. the third point above), since there's still wonky behavior like (1, 2).map(state) being two different states but (1, 1).map(state) being two copies of the same one, but I don't really have a better idea either. Maybe documenting this well enough and telling people to use string-based private keys if they ever encounter such a collision is good enough. Or maybe this single-argument state call should just get removed and we could go back to the previous syntax of only having state(key, init), but still keeping the new feature of private string keys (which is kind of the main feature of this PR).

If we go down that route, maybe the API should just be state(key: str, init, #[named] public: bool) instead.

On the other hand, that'd go back to being a pretty small change and wouldn't really address users' confusion over why there has to be a string key in addition to giving it a variable name.

That just gave me another idea: What about doing it like I just described, but also making key named and just defaulting to "". Most simple usages of state are just assigning it to a variable once and for those, the span is unique enough, so the string isn't needed to disambiguate and can just be an arbitrary value (in this case the empty string).

At least I assume that's most usages, maybe the actual data contradicts me and this doesn't make much sense...

the new feature of private string keys (which is kind of the main feature of this PR)

i'm sorry, but you still haven't explained what that is or why a user would want that!

Think of it as a combination of both the anonymous, span-based keys and the string keys we have now.

They are "private" because creating a state with the same string key but at a different source code location will always produce a different state.
The advantage of this over just having anonymous states that don't have any string key at all is that you can do stuff like for i in range(n) { let state-i = state(..., key: state.key(str(i), public: false)) }, where the states would have been the same without the explicit key.

regarding the idea of using the state's initial value as a component of the key/hash: it makes sense when we think of the typical "static state" use case, where you have something like #let my-state = state(5) and you're done. in this scenario, we statically know every state's initial value so it's easy to assume that e.g. two span-sharing states generally won't share initial values, right?

but the whole point of still allowing string keys is to aid in the "dynamic state" use case: here a user may have something like (a, b).map(state) where a and b are not statically known. in this scenario, the user might naively test their function with a ≠ b without yet realizing that if ever a = b, their function will silently break in a way that is impossible to debug.

this is why i think it's better to force the user to go "all or nothing": either provide no key, which should break every time (here is where the compiler hint comes into play), or provide a key, which should never break (assuming reasonable keys). all of this, of course, on top of the new span-based key approach, as spans don't pose this problem.

regarding private keys: i think i understand now. a private key is a span+string combo (which in my opinion is the ideal solution), while a public key is a plain string that universally identifies this state, in the same way that state currently works. right?

i think public keys are really bad and shouldn't exist. if you wanna share state between modules, just pass the state object around. stringly-typed global state names are not a proper way of communicating between modules. edit: for example, if i ever decide to change a state name but forget to update it in other files, everything silently breaks.

regarding private keys: i think i understand now. a private key is a span+string combo (which in my opinion is the ideal solution), while a public key is a plain string that universally identifies this state, in the same way that state currently works. right?

Yes, exactly.

i think public keys are really bad and shouldn't exist. if you wanna share state between modules, just pass the state object around. stringly-typed global state names are not a proper way of communicating between modules.

I feel like there might be some situations where it could be useful, but I also can't come up with anything concrete. Either way, it might be good to keep it temporarily (maybe as deprecated if people agree that it's bad) to give users time to update.

just realized, counter() would have to be updated as well. i can think of 3 approaches:

1. keep the current key: str or selector parameter, and just make it optional: reasonable, but removes the natural semblance with state() which is bad for the user as (a) it suggests that they are two disparate mechanisms, and (b) it compels them to provide a key, when generally they won't need to after this change.
2. split the signature into a positional but optional target: selector parameter, and another named key: str parameter: this is a little strange, particularly because these two parameters are disjoint; a targeted counter with a key makes no sense (the target is the key); the ad-hoc argument parser would have to error here.
3. split the signature into two named parameters, target: selector and key: str: again, the argument parser would have to error when both parameters are specified, but i think this is more intuitive when both params are named, because there is a sort of "symmetry". also, target: selector is reminiscent of outline's signature, which i think is another nice symmetry.

1. #let counter(key?)
2. #let counter(target?, key: none)
3. #let counter(target: none, key: none)

the simplest option, #​1, is ugly. #​3 sounded very nice at first, but it incurs a wide-reaching breaking change: every cute counter(heading) would have to change to counter(target: heading). bad! to avoid this, we could go with #​2, which isn't very intuitive.

I don't understand your issue with number 1, that seems perfectly reasonable to me.
Also, considering the direction counters will probably go in (see the discussion in #5716), this is even an argument to keep state.key around and also use it in counters (Tho maybe it being called state.key would be a bit confusing, but I don't have a better idea either). That way a future multi-key counter could look like counter(heading, state.key("theorem", public: false)) and in particular the key could be created once (#let k = state.key("theorem", public: false)) and e.g. passed to both counter(heading, k, ...).get() and counter(k).step().

Just the counter could be enough. Just letting it be let c = counter("theorem"); counter(heading, c) and passing an existing counter to the constructor would just extract its keys.

#let k = state.key("theorem", public: false)

reified "private keys" are still a bad idea, even if less bad than bare string identifiers: they add an unnecessary layer of indirection. i agree that if you wanna refer to the counter, just refer to the counter. (not to a "pointer" to the counter).

I don't understand your issue with number 1

(thanks for the #​5716 link!) the symmetry is lost between state(key: "...") and counter(key: "..."). this is unintuitive and it "normalizes" the usage of keys in counters, which i think should be discouraged after this change.

let c = counter("theorem")

which leads me to this: i don't understand why we're still talking about counter keys. a user should just be able to do let theorems = counter(), where explicit counter keys are only for the rare dynamic counters. right?

i guess that helps me understand the use case for public keys: it's more ergonomic to do counter("theorem") than to carry around a theorems; this is similar to custom figure kinds, which are string-based. but i don't think the trade-off is worth it, and either way it wouldn't be consistent with state() keys (which will be private).

I think at least having a private key type is actually a technical necessity, since StateUpdateElem has a public key field.
(And no, that can't be made #[internal], I tried, but internal fields can't be used with selectors so state.get() would stop working.)

I reworked everything based on the feedback here.
Now state is either called as state(init) or as state(key, init) and the key is always private. I also added a warning in a call like state(init) if init is a string.

Similarly, counter keys are now also private and you can call counter() as a shorthand for counter("").

I'm not sure if I used all the special typst documentation syntax correctly, so some feedback on that would be appreciated.

I think this isn't working as expected anymore. This:

#let a = state("hi", none)
#let b = state("hi", none)

#a.update("Hello")

#context b.get()

gives

Hello

I think it's because the select_where! selector discards the span when calling into_value().

This is not the main obstacle though. From my point of view, there's two things:

• Deciding what exactly we want (the hard part). I think something like this here could be workable, but I am not yet certain whether this is the way to go.
• Making sure that it's it's not immediately breaking. IIUC, this PR changes the behaviour of the state constructor in a very breaking way (replacing key with init value). If we decide on a breaking change, there needs to be a good migration path and warnings first.

Making sure that it's it's not immediately breaking.

The whole point of this PR is already breaking, so if people will have to look into how state changed anyway, we might as well go with the new behavior anyway. I even already added a warning for this, so I'm kinda doubtful of whether it's really worth it to roll this out in two phases.

Note that the argument order of state(key, init) isn't changed, only state(arg) went from state(arg, none) to state("", arg).

I disagree that the point of this is to be breaking. The point is to enable anonymous states. It might be that we come to the conclusion that the interface we want is breaking and then we have to gauge the impact and see how to deal with that, but there also certainly possible design that are not breaking.

I meant "the point is breaking" as in "the point (making state keys anonymous) is breaking", not that being breaking is literally the point. But alright, I can see how it wouldn't have to be breaking.
Personally, I can't think of a good non-breaking API tho and I also can't think of a use case for the old key behavior. (But I'm also not going to die on this hill.)

My concern is more breakage of packages than a good use case of the old behaviour. I'm not sure how widespread breakage would be, but I could imagine that a lot of packages stop working immediately if we aren't careful. I'd rather have those be warnings instead initially.

How about the following: I can change this code to still store spans with the keys like now, but not actually use them yet, other than emitting a warning when the same key is used with a different span. Then, once that's released, we change it for the next version to actually use the spans for the keys.

That'd be the most careful, non-breaking thing that comes to mind right now.

I haven't made up my mind yet on how to proceed here. Unfortunately, I can't prioritize this now. Let's revisit this after 0.14.

Currently, I'm not confident in this change and, for now, I'd rather not make it. I expect some change to happen at some point, but it will need some more time. I expect that this topic will resurface in relation to custom types where similar span-related things might be needed for type equality. This will hopefully allow me to make up my mind and make things consistent. It might also be a good opportunity to batch up breaking changes with appropriate migration paths and/or behavior.