feat(marshal): passable to quasi-quoted Justin expr#2799
Conversation
c06a938 to
165a775
Compare
165a775 to
abbd85b
Compare
dckc
left a comment
There was a problem hiding this comment.
some notes on docs
The code looks plausible. I'd have to study it (and the surrounding context) quite a bit more to be confident that it's correct.
dckc
left a comment
There was a problem hiding this comment.
I'm satisfied.
I leave it to you to judge whether to wait for a review from @kriskowal
032533a to
e515fc5
Compare
| // The example below is the `patt1` test case from `qp-on-pattern.test.js`. | ||
| // Please co-maintain the following doc-comment and that test module. | ||
| /** | ||
| * `qp` for quote passable as a quasi-quoted Justin expression. |
There was a problem hiding this comment.
Now that this landed, I'd expect it to appear in the list of function on https://endojs.github.io/endo/modules/_endo_marshal.html . But I don't see it. Anybody know why not?
My understanding is that Looking at In that case, |
|
That's a good idea. I actually tried to do something similar and gave up because, for some reason, it did not occur to me to modify If I do it, it'll be in PR separate from #2801 . Or, if anyone else wants to try, I'll be happy to review. |
Yeah a |
Closes: #XXXX Refs: #2799 (comment) ## Description At #2799 (comment) @dckc suggested > I suggest including it in the comment. I intended to. But somehow I resolved it before actually doing this, causing me to miss it when evaluating if I should just merge-and-squash #2799. This PR corrects that omission. Minor internal comment only. It should not have any other implications.
Closes: #XXXX Refs: Agoric/agoric-sdk#5401 ## Description We often need to see the deep contents of a Passable. Indeed, this was kicked off by a slack thread about wanting to quote ```js M.splitRecord( harden({ assetKind: M.or('nat', 'set', 'copySet', 'copyBag'), decimalPlaces: M.and(M.gte(-100), M.lte(100)), }), ) ``` This PR provides a `qp` function meaning "quote passable". You can use it where you'd use `q`. It produces a quasi-quoted Justin expression of the form in which the passable would be passed. For example, the `qp` of the above pattern is ```js `makeTagged("match:splitRecord", [ { assetKind: makeTagged("match:or", [ "nat", "set", "copySet", "copyBag", ]), decimalPlaces: makeTagged("match:and", [ makeTagged("match:gte", -100), makeTagged("match:lte", 100), ]), }, ])` ``` ### Security Considerations Unlike `q`, `qp` does not lazily postpone rendering the expression into Justin. `q` can do this easily because it is inside the `'ses'` assert.js module and can use the rights amplification available only there. `qp` depends on `'@endo/marshal'` which is several levels up from `'ses'`. Rather than expose these rights amplification powers (or several other kludges that failed), I just made `qp` non-lazy. Thus there are no security implications. ### Scaling Considerations Because `qp` is not lazy, simply constructing an error with `Fail` or `makeError(X...)` will pay all the cost of rendering to Justin, whether or not that error is ever used. ### Documentation Considerations Because Justin is a subset of HardenedJS, the Justin expressions produced by `qp` can be used in documentation if needed. Though see Compatibility Considerations below. ### Testing Considerations - [ ] Only minimal tests written for `qp`. We should also write a test that loops through all the `jsonJustinPairs`, just as other Justin tests do. ### Compatibility Considerations Using Justin with symbols exposes in the rendered string how Justin currently represents passable symbols. That changes in #2793. Whichever of these gets merged first, the other will have to reconcile merge conflicts. And any saved Justin strings containing that old Justin representation will need to be updated or marked stale. ### Upgrade Considerations Other than the Compatibility Considerations above, none. - [x] Update `NEWS.md` for user-facing changes.
Closes: #XXXX Refs: #2799 (comment) ## Description At #2799 (comment) @dckc suggested > I suggest including it in the comment. I intended to. But somehow I resolved it before actually doing this, causing me to miss it when evaluating if I should just merge-and-squash #2799. This PR corrects that omission. Minor internal comment only. It should not have any other implications.
Closes: #XXXX Refs: #2799 ## Description #2799 added a new `q`-like quoter named `qp`, for "quote passable", that renders any value into Justin source text. This was specifically motivated for displaying patterns within error messages in a readable and informative manner. The `@endo/patterns` package itself formulates many error messages to explain why a specimen did not match a pattern. Some of these message include quoted forms of nested patterns. But those were still quotes with `q`, leading @dckc to report still seeing error messages like ``` Error { message: '... - Must match one of ["[match:splitRecord]","[match:splitRecord]"]', } ``` This PR changes the quoting of nested errors to use `qp` instead, so a `[match:splitRecord]` might instead render as ```js `makeTagged("match:splitRecord", [ { assetKind: makeTagged("match:or", [ "nat", "set", "copySet", "copyBag", ]), decimalPlaces: makeTagged("match:and", [ makeTagged("match:gte", -100), makeTagged("match:lte", 100), ]), }, ])` ``` This is more verbose. But error messages are designed to carry the critical clues to help find a bug. For nested patterns, this extra information is worth the verbosity. ### Security Considerations We were already quoting nested patterns with `q`, unredacting them, under the assumption that specimens may carry secrets worth protecting from callers, but patterns typically do not. Switching these from `q` to `qp` is consistent with that assumption, but actually exposes info that `q` happened not to expose. When unredacting happened not to expose secrets it said were ok to expose, it is easy for developers not to notice the secrets that could have been shown but weren't. After this PR, these unredacted secrets will be exposed to callers. ### Scaling Considerations Where `q` is lazy, `qp` is eager, rendering its argument up front into Justin even if the containing error is never created or thrown. Thus, we must take care to only use `qp` when we're already committed to creating and throwing the error. I believe all such changes in this PR do so, and so do not add overhead to the happy path. ### Documentation Considerations I don't know what documentation we provide explaining the interpretation of error messages. If we have any explain pattern match error messages, this PR might need us to change those explanations. ### Testing Considerations Using `qp` to formulate the error messages definitely makes it more awkward to test is the error message is as expected. You can see this in the tests this PR needed to change. Fortunately, as shown in these error message tests, you can use `${q(qp(...))}` to formulate the golden in the test. ### Compatibility Considerations Because this PR changes the error messages, as pointed out above, golden tests of error message text may no longer match, ***causing old working tests to start failing***. However, this is not ***technically*** a compat break, because we take the stance that the content of error messages is not normative, so the error messages can become more informative over time with a version bump. Thus, I use `:` rather than `!` to recommend that this PR not cause a version bump. (See #2868 (review) below) ### Upgrade Considerations None.
Closes: #XXXX
Refs: Agoric/agoric-sdk#5401
Description
We often need to see the deep contents of a Passable. Indeed, this was kicked off by a slack thread about wanting to quote
This PR provides a
qpfunction meaning "quote passable". You can use it where you'd useq. It produces a quasi-quoted Justin expression of the form in which the passable would be passed. For example, theqpof the above pattern isSecurity Considerations
Unlike
q,qpdoes not lazily postpone rendering the expression into Justin.qcan do this easily because it is inside the'ses'assert.js module and can use the rights amplification available only there.qpdepends on'@endo/marshal'which is several levels up from'ses'. Rather than expose these rights amplification powers (or several other kludges that failed), I just madeqpnon-lazy. Thus there are no security implications.Scaling Considerations
Because
qpis not lazy, simply constructing an error withFailormakeError(X...)will pay all the cost of rendering to Justin, whether or not that error is ever used.Documentation Considerations
Because Justin is a subset of HardenedJS, the Justin expressions produced by
qpcan be used in documentation if needed. Though see Compatibility Considerations below.Testing Considerations
qp. We should also write a test that loops through all thejsonJustinPairs, just as other Justin tests do.Compatibility Considerations
Using Justin with symbols exposes in the rendered string how Justin currently represents passable symbols. That changes in #2793. Whichever of these gets merged first, the other will have to reconcile merge conflicts. And any saved Justin strings containing that old Justin representation will need to be updated or marked stale.
Upgrade Considerations
Other than the Compatibility Considerations above, none.
NEWS.mdfor user-facing changes.