feat: support M.raw() in method guards#1831
Conversation
369921f to
afc3670
Compare
afc3670 to
cb4a38d
Compare
erights
left a comment
There was a problem hiding this comment.
Looks great! But the klass: '...' issue must first be fixed. See the AwaitArgGuard as an example guide.
| interfaceName: M.string(), | ||
| methodGuards: M.recordOf(M.string(), MethodGuardShape), | ||
| sloppy: M.boolean(), | ||
| defaultGuards: M.or('never', 'passable', 'raw'), |
There was a problem hiding this comment.
Since this property is new, consider whether it should be moved into the optional part of this splitRecord pattern. But if you are confident we don't need to, then don't.
There was a problem hiding this comment.
Thanks, I reintroduced sloppy and moved defaultGuards down to remain compatible.
There was a problem hiding this comment.
Actually, you introduced defaultGuards in the mandatory section. If we get away with that, then we know this new code never encounters any InterfaceGuardPayloads created by code before this PR. That's great! If we believe that, then I think you can simply delete the optional sloppy: below. Otherwise, I think both sloppy: and defaultGuards: need to go into the optional section.
There was a problem hiding this comment.
I think we need both sloppy and defaultGuards in the optional section.
cb4a38d to
fe8ad36
Compare
M.rawValue() in method guardsM.raw() in method guards
fe8ad36 to
7ba22ea
Compare
2f707d0 to
263254a
Compare
|
All comments are addressed, and CI is passing. Ready for rereview! |
263254a to
186711b
Compare
gibson042
left a comment
There was a problem hiding this comment.
Even reading the linked issues, I'm not clear on the motivation for introducing this non-Passable "raw" argument/return and unknown-method functionality, and am wondering if it might be better to instead establish a convention for e.g. conveying mutable objects via thunks and/or introduce specific configuration for supporting unknown methods. Can you explain the need?
packages/exo/src/exo-makers.js
Outdated
| * }> | undefined} interfaceGuard | ||
| * @param {I} init | ||
| * @param {M & ThisType<{ self: M, state: ReturnType<I> }>} methods | ||
| * @param {M & ThisType<{ self: Facet<M>, state: ReturnType<I> }>} methods |
There was a problem hiding this comment.
https://github.com/Agoric/agoric-sdk/blob/master/packages/SwingSet/docs/virtual-objects.md draws a distinction between an exo object (therein a "VDO") and a facet, so self: Facet<M> seems wrong. I think the "Facet" type should be renamed accordingly, perhaps to something like "Guarded"?
packages/exo/src/exo-tools.js
Outdated
| const defendSyncArgs = (syncArgs, redactConfig, label = undefined) => { | ||
| const { | ||
| declaredLen, | ||
| hasRestArgGuard, | ||
| paramsPattern, | ||
| rawRestArgs, | ||
| redactedIndices, | ||
| } = redactConfig; |
There was a problem hiding this comment.
Clarifying suggestion (but note that at this point I am not clear on the specifics of what "redaction" is doing here at all):
| const defendSyncArgs = (syncArgs, redactConfig, label = undefined) => { | |
| const { | |
| declaredLen, | |
| hasRestArgGuard, | |
| paramsPattern, | |
| rawRestArgs, | |
| redactedIndices, | |
| } = redactConfig; | |
| const defendSyncArgs = (syncArgs, redactionConfig, label = undefined) => { | |
| const { | |
| declaredLen, | |
| hasRestArgGuard, | |
| paramsPattern, | |
| rawRestArgs, | |
| redactedIndices, | |
| } = redactionConfig; |
packages/exo/src/exo-tools.js
Outdated
| let redactedArgs = syncArgs; | ||
| if (rawRestArgs) { | ||
| // Don't harden the rest args. | ||
| redactedArgs = syncArgs.slice(0, Math.min(syncArgs.length, declaredLen)); |
There was a problem hiding this comment.
I don't think this Math.min is doing anything; slice already clamps its second argument to the length of the array.
| redactedArgs = syncArgs.slice(0, Math.min(syncArgs.length, declaredLen)); | |
| redactedArgs = syncArgs.slice(0, declaredLen); |
packages/exo/src/exo-tools.js
Outdated
| redactedArgs[i] = REDACTED_RAW_ARG; | ||
| } | ||
|
|
||
| mustMatch(harden(redactedArgs), paramsPattern, label); |
There was a problem hiding this comment.
The redactedArgs name seems a little misleading, implying e.g. "those arguments that have been redacted". Maybe instead matchableArgs?
Also, I question the value of completely ignoring raw rest args here—it's probably better to preserve their presence:
// Use syncArgs if possible, but copy it when necessary to implement redactions.
let matchableArgs = syncArgs;
if (rawRestArgs && syncArgs.length > declaredLen) {
const restLen = syncArgs.length - declaredLen;
const redactedRest = Array(restLen).fill(REDACTED_RAW_ARG);
matchableArgs = [...syncArgs.slice(0, declaredLen), ...redactedRest];
} else if (redactedIndices.length > 0 && redactedIndices[0] < syncArgs.length) {
matchableArgs = [...syncArgs];
}
for (const i of redactedIndices) {
if (i >= matchableArgs.length) {
break;
}
matchableArgs[i] = REDACTED_RAW_ARG;
}
mustMatch(harden(redactedArgs), paramsPattern, label);
packages/exo/src/exo-tools.js
Outdated
| let rawRestArgGuard = restArgGuard; | ||
| let rawRestArgs = false; | ||
| if (isRawGuard(rawRestArgGuard)) { | ||
| rawRestArgGuard = undefined; |
There was a problem hiding this comment.
| rawRestArgGuard = undefined; | |
| rawRestArgGuard = REDACTED_RAW_ARG; |
There was a problem hiding this comment.
This needed some finessing, after replacing undefined with the only pattern that could possibly work: M.arrayOf(REDACTED_RAW_ARG).
There was a problem hiding this comment.
Ah, better yet then:
| rawRestArgGuard = undefined; | |
| matchableRestArgGuard = M.any(); |
There was a problem hiding this comment.
If we're going through the effort to create rest arguments that are each REDACTED_RAW_ARG, I'd rather do the stricter arrayOf pattern. Without being strict, I see no compelling reason to do anything other than just drop raw rest arguments rather than doing this whole match-but-bypass dance.
packages/exo/src/exo-tools.js
Outdated
| const defendSyncMethod = (method, methodGuardPayload, label) => { | ||
| const { returnGuard } = methodGuardPayload; | ||
| const isRawReturn = isRawGuard(returnGuard); | ||
| const redactConfig = redactRawArgs(methodGuardPayload); |
There was a problem hiding this comment.
| const redactConfig = redactRawArgs(methodGuardPayload); | |
| const redactionConfig = toRedactionConfig(methodGuardPayload); |
packages/patterns/src/types.js
Outdated
| * @property {() => RawGuard} raw | ||
| * In parameter position, pass this argument through without any hardening or checking. | ||
| * In rest position, pass the rest of the arguments through without any hardening or checking. | ||
| * In return position, return the result without any hardening or ßchecking. |
There was a problem hiding this comment.
| * In return position, return the result without any hardening or ßchecking. | |
| * In return position, return the result without any hardening or checking. |
Exos used to be tightly coupled to
We already had the unknown-method functionality called
I want to be able to create exo objects or facets whose methods can be just as flexible as a vanilla JS function, but have that be an opt-out so that the Exo model of guards and patterns is the default behaviour.
Funny you should mention that. Thunks are non-passable, too!
The specific configuration is to explicitly specify
I don't want to make breaking design changes to contracts or libraries that I'm converting to be Exo-objects. Especially when so much of their behaviour fits into the Exo-object model already. |
packages/exo/src/exo-makers.js
Outdated
| // Used in typing. | ||
| GET_INTERFACE_GUARD; |
There was a problem hiding this comment.
What does this no-effect statement do?
| /** @type {KitContext<ReturnType<I>,F>} */ | ||
| const context = { state, facets: {} }; | ||
| /** @type {{ state: ReturnType<I>, facets: unknown }} */ | ||
| const context = { state, facets: null }; |
There was a problem hiding this comment.
Why is null here correct? The original
| const context = { state, facets: null }; | |
| const context = { state, facets: {} }; |
seems more correct.
There was a problem hiding this comment.
I intended for the null to make it explicit that facets is not yet initialized. I could have used undefined instead, but I think {} implies that the identity will be kept intact (which it isn't). The code below replaces it with the actual facets object.
erights
left a comment
There was a problem hiding this comment.
LGTM!
But especially curious about facets: null. Am I missing something?
I think that just pushes the question back a level... what is the underlying motivation for non-Passable promise resolution?
Are they? import "@endo/init";
import { Far } from "@endo/far";
import { M, matches } from "@endo/patterns";
const fn = Far("fn", () => "returned");
fn();
// => "returned"
matches(fn, M.remotable());
// => true
Ah, this seems like it's heading in the direction I care about. What is being converted to exo, what is the benefit, and how do they currently interact with non-Passable values? |
packages/patterns/src/types.js
Outdated
| */ | ||
|
|
||
| /** | ||
| * @typedef {'never' | 'passable' | 'raw'} DefaultGuardType |
There was a problem hiding this comment.
Why allow "never"? I think we would normally use undefined to indicate an unchanged default, and certainly not a word that could be misinterpreted as meaning something like "even less than raw".
| * @typedef {'never' | 'passable' | 'raw'} DefaultGuardType | |
| * @typedef {undefined | 'passable' | 'raw'} DefaultGuardType |
| const LABEL_INSTANCES = DEBUG.split(',').includes('label-instances'); | ||
|
|
||
| /** | ||
| * @template {{}} T |
There was a problem hiding this comment.
Clarity/repository consistency nit:
| * @template {{}} T | |
| * @template {object} T |
(details)
$ patt='[{]([{][}]|object)[}]'
$ git grep -liE "$patt" packages/ | xargs grep --no-filename -oE "$patt" | sort | uniq -c
9 {{}}
187 {object}There was a problem hiding this comment.
They are not the same. Object.create(proto) is a type error unless proto extends {}.
|
|
||
| /** @typedef {AwaitArgGuard | Pattern} ArgGuard */ | ||
| /** | ||
| * @typedef {{}} RawGuardPayload |
There was a problem hiding this comment.
| * @typedef {{}} RawGuardPayload | |
| * @typedef {object} RawGuardPayload |
There was a problem hiding this comment.
This time, I'm specifying that RawGuardPayload is the empty object, not just anything of type object.
packages/exo/src/exo-tools.js
Outdated
| let rawRestArgGuard = restArgGuard; | ||
| let rawRestArgs = false; | ||
| if (isRawGuard(rawRestArgGuard)) { | ||
| rawRestArgGuard = undefined; |
There was a problem hiding this comment.
Ah, better yet then:
| rawRestArgGuard = undefined; | |
| matchableRestArgGuard = M.any(); |
| * interfaceName: string, | ||
| * methodGuards: any, | ||
| * options: {sloppy: true}) => InterfaceGuard<Record<PropertyKey, MethodGuard>> | ||
| * options: {defaultGuards?: 'passable' | 'raw', sloppy?: true }) => InterfaceGuard<any> |
There was a problem hiding this comment.
With the above suggestion to replace "never" with undefined, this could also be simplified.
| * options: {defaultGuards?: 'passable' | 'raw', sloppy?: true }) => InterfaceGuard<any> | |
| * options: {defaultGuards?: DefaultGuardType, sloppy?: true }) => InterfaceGuard<any> |
There was a problem hiding this comment.
Can't really simplify, because it has special meaning as a function overload.
packages/patterns/src/types.js
Outdated
| * In parameter position, guard a parameter by awaiting it. Can only be used in | ||
| * parameter position of an `M.callWhen`. | ||
| * `M.await(M.nat())`, for example, with `await` the corresponding argument, | ||
| * check that the fulfillment of the `await` satisfies the `M.nat()` | ||
| * pattern, and only then proceed to call the raw method with that fulfillment. | ||
| * If the argument already passes the `M.nat()` pattern, then the result of | ||
| * `await`ing it will still pass, and the `M.callWhen` will still delay the | ||
| * raw method call to a future turn. | ||
| * If the argument is a promise that rejects rather than fulfills, or if its | ||
| * fulfillment does not satisfy the nested pattern, then the call is rejected | ||
| * without ever calling the raw method. |
There was a problem hiding this comment.
👍 👍 for adding this.
| * In parameter position, guard a parameter by awaiting it. Can only be used in | |
| * parameter position of an `M.callWhen`. | |
| * `M.await(M.nat())`, for example, with `await` the corresponding argument, | |
| * check that the fulfillment of the `await` satisfies the `M.nat()` | |
| * pattern, and only then proceed to call the raw method with that fulfillment. | |
| * If the argument already passes the `M.nat()` pattern, then the result of | |
| * `await`ing it will still pass, and the `M.callWhen` will still delay the | |
| * raw method call to a future turn. | |
| * If the argument is a promise that rejects rather than fulfills, or if its | |
| * fulfillment does not satisfy the nested pattern, then the call is rejected | |
| * without ever calling the raw method. | |
| * Guard a positional parameter in `M.callWhen`, awaiting it and matching its | |
| * fulfillment against the provided pattern. | |
| * For example, `M.callWhen(M.await(M.nat())).returns()` will await the first | |
| * argument, check that its fulfillment satisfies `M.nat()`, and only then call | |
| * the guarded method with that fulfillment. If the argument is a non-promise | |
| * value that already satisfies `M.nat()`, then the result of `await`ing it will | |
| * still pass, and `M.callWhen` will still delay the guarded method call to a | |
| * future turn. | |
| * If the argument is a promise that rejects rather than fulfills, or if its | |
| * fulfillment does not satisfy the nested pattern, then the call is rejected | |
| * without ever invoking the guarded method. |
There was a problem hiding this comment.
Credit to @erights for the addition. I just cherry-picked it. 😊
packages/patterns/src/types.js
Outdated
| * @property {(rArgGuard: Pattern) => MethodGuardMaker2} rest | ||
| * @property {(returnGuard?: Pattern) => MethodGuard} returns | ||
| * @property {(restArgGuard: SyncValueGuard) => MethodGuardMaker2} rest | ||
| * If the rest argument guard is not `M.raw()`, all rest arguments are | ||
| * automatically hardened and must be Passable. | ||
| * @property {(returnGuard?: SyncValueGuard) => MethodGuard} returns | ||
| * If the return guard is not `M.raw()`, the return value is automatically | ||
| * hardened and must be Passable. |
There was a problem hiding this comment.
Is it possible to define these types in a way that doesn't require so much repetition?
/**
* @typedef {MethodGuardOptMaker & MethodGuardRestMaker & MethodGuardFinalizer} MethodGuardMaker
* A method name and parameter/return signature like:
* ```js
* foo(a, b, c = d, ...e) => f
* ```
* should be guarded by something like:
* ```js
* {
* ...otherMethodGuards,
* foo: M.call(AShape, BShape).optional(CShape).rest(EShape).returns(FShape),
* }
* ```
*/
/**
* @typedef {object} MethodGuardOptMaker
* @property {(...optArgGuards: ArgGuard[]) => (MethodGuardRestMaker & MethodGuardFinalizer)} optional
* Optional arguments not guarded with `M.raw()` are automatically hardened and
* must be Passable.
*/
/**
* @typedef {object} MethodGuardRestMaker
* @property {(restArgGuard: SyncValueGuard) => MethodGuardFinalizer} rest
* If the rest argument guard is not `M.raw()`, all rest arguments are
* automatically hardened and must be Passable.
*/
/**
* @typedef {object} MethodGuardFinalizer
* @property {(returnGuard?: SyncValueGuard) => MethodGuard} returns
* If the return guard is not `M.raw()`, the return value is automatically
* hardened and must be Passable.
*/There was a problem hiding this comment.
Thanks, I like that! Usually, my DRY factor is 2, but I am more lenient sometimes when I haven't seen the best way to factor.
Oh, I misspoke when I said "thunks are non-passable". I meant to say "passable thunks are not consistently supported in SwingSet and other random parts of our system, so I wouldn't count on using them for anything important until they have had significantly more testing."
IBC/Network vats and Pegasus are being made durable and upgradable. I'm trying to do a minimal implementation of upgrade-tolerant promise-like remotables, and that's the last in a long line of trying to do promise-like remotables and bashing my head into the ceiling imposed by the passable restrictions. This PR is the result of trying to raise that ceiling so that I can use Zones and Exo-objects for durability as advertised. |
7c1fc7a to
a0b3a5c
Compare
Refs: #1831, Agoric/agoric-sdk#11931 ## Description We wish to visibly deprecate the `sloppy` option of `@endo/patterns` interface guards. To avoid an explosion of `@typedef` blocks, we migrate the `@endo/patterns` types to TypeScript and then add the minimally required `@deprecated` tags. ### Security Considerations N/A ### Scaling Considerations N/A ### Documentation Considerations N/A ### Testing Considerations This modifies the implementation of the `@endo/patterns` types, but should not modify them. Since the number of lines changed is larged, we should attempt to convince ourselves that the types are in fact the same. ### Compatibility Considerations N/A ### Upgrade Considerations Deprecates but does not remove the `sloppy` option. `NEWS.md` is updated.
closes: #1725 closes: #1772
refs: #1724
Description
Complete the #1724 experiment, as well as using the
defaultGuardsoption (originally discussed in #1725 (comment)) instead of boolean flags forsloppyandraw.Security Considerations
The new
M.raw()guard is only honoured by arguments and return values indefendSyncMethod, so it should not leak elsewhere as a pattern.Scaling Considerations
Some additional overhead in redacting raw arguments (in
defendSyncArgs), but work is minimized by calculating relevant data structures while wrapping methods before any exos are created (indefendSyncMethod).Documentation Considerations
Testing Considerations
Upgrade Considerations
Backward compatibility is implemented for interfaceGuards created using the legacy
sloppyparameter.