feat: represent new accounts as AccountDeltas#1896
Conversation
| // if the account is new, insert the storage map entries into the advice provider. | ||
| if partial_native_acc.is_new() { | ||
| for storage_map in partial_native_acc.storage().maps() { | ||
| let map_entries = storage_map | ||
| .entries() | ||
| .flat_map(|(key, value)| { | ||
| value.as_elements().iter().chain(key.as_elements().iter()).copied() | ||
| }) | ||
| .collect(); | ||
| // TODO: Should we use the sequential hash instead? | ||
| inputs.add_map_entry(storage_map.root(), map_entries); | ||
| } | ||
| } |
There was a problem hiding this comment.
Thinking more about this, there are some better solutions:
- Using the sequential hash would be optimal, but we somehow need to get that hash into
account_delta::add_storage_map_entries_to_delta, which would have to be either provided as part of the prologue or pushed onto the advice stack by an event. This would require an extra field to track this in the host, which feels a bit unclean. - Alternatively, we could also use the hash of the root and do the same in
add_storage_map_entries_to_delta. This would probably be the simplest approach. - We could also have
account_delta::add_storage_map_entries_to_deltaemit an event and the host pushes the data to the advice stack in response. The host would need to be changed to containPartialAccountinstead ofAccountHeader, which means we need to clone the partial account whenever we construct a tx host.
I think I will change it to the second one, as that seems like the minimal fix, unless anyone has other preferences.
There was a problem hiding this comment.
Is there something fundamentally wrong with the current approach?
There was a problem hiding this comment.
I'm not sure if overwriting the SMT's root in the advice provider is fine or not. I don't know if the smt module sets it somewhere. This is a general problem with the advice map. You don't really know how others are using it. That's why we have this soft rule of "data in the advice map should have its hash as the key", but it's a bit unclean here.
There was a problem hiding this comment.
but as we're using roots as keys, which result from cryptographic hashes, we shouldn't have to worry about collisions, right?
There was a problem hiding this comment.
so if that's the only concern, I think the current approach works well 👍🏼
There was a problem hiding this comment.
I think option 2 is not that much different from the current approach - so, I'd keep the code as is.
But I do agree that not relying on STORAGE_MAP_ROOT |-> [MAP ENTRIES] entry in the advice map would be a bit better, and maybe the 3rd option is the right way to do it. Let's create an issue to potentially come back to this in the future.
mmagician
left a comment
There was a problem hiding this comment.
LGTM with a few minor questions, but nothing blocking
| // Add the removed fee to the post fee delta to get the pre-fee delta, against which | ||
| // the delta commitment needs to be validated. | ||
| post_fee_account_delta.vault_mut().add_asset(self.fee.into()).map_err(|err| { | ||
| ProvenTransactionError::AccountDeltaCommitmentMismatch(Box::from(err)) | ||
| })?; | ||
|
|
||
| let expected_commitment = self.account_update.account_delta_commitment; | ||
| let actual_commitment = post_fee_account_delta.to_commitment(); |
There was a problem hiding this comment.
I've looked into the call sites of validate() a bit, and aside from tests (and deserialization*) it seems to be used only in ProvenTransacationBuilder::build().
And build(), in turn, is only meaningfully used within LocalTransactionProver::build_proven_transaction, where we have the following logic:
// Compute the commitment of the pre-fee delta, which goes into the proven transaction,
// since it is the output of the transaction and so is needed for proof verification.
let pre_fee_delta_commitment: Word = pre_fee_account_delta.to_commitment();
let builder = ProvenTransactionBuilder::new(
...,
pre_fee_delta_commitment,
...
)...;
// The full transaction delta is the pre fee delta with the fee asset removed.
let mut post_fee_account_delta = pre_fee_account_delta;
post_fee_account_delta
.vault_mut()
.remove_asset(Asset::from(tx_outputs.fee))
.map_err(TransactionProverError::RemoveFeeAssetFromDelta)?;So it seems that fee manipulation within validate basically replicates that same logic.
*I haven't thought too much whether this should be done for deserialization, i.e. in what scenarios this would happen.
bobbinth
left a comment
There was a problem hiding this comment.
Looks good! Thank you! I left some comments inline. The main ones are:
- A comment about potentially including code root into account delta commitment computation.
- A comment about potential vulnerability in how we build storage map deltas for initial storage maps.
| // if the account is new, insert the storage map entries into the advice provider. | ||
| if partial_native_acc.is_new() { | ||
| for storage_map in partial_native_acc.storage().maps() { | ||
| let map_entries = storage_map | ||
| .entries() | ||
| .flat_map(|(key, value)| { | ||
| value.as_elements().iter().chain(key.as_elements().iter()).copied() | ||
| }) | ||
| .collect(); | ||
| // TODO: Should we use the sequential hash instead? | ||
| inputs.add_map_entry(storage_map.root(), map_entries); | ||
| } | ||
| } |
There was a problem hiding this comment.
I think option 2 is not that much different from the current approach - so, I'd keep the code as is.
But I do agree that not relying on STORAGE_MAP_ROOT |-> [MAP ENTRIES] entry in the advice map would be a bit better, and maybe the 3rd option is the right way to do it. Let's create an issue to potentially come back to this in the future.
crates/miden-lib/asm/kernels/transaction/lib/account_delta.masm
Outdated
Show resolved
Hide resolved
| #! Inserts the entries of the provided storage map root into the account delta. | ||
| #! | ||
| #! These entries must be present in the advice provider with the map root as the key. | ||
| #! For each entry, the initial value in the delta is set to the empty word so the delta for this | ||
| #! map is computed as if the map had initially been empty. | ||
| #! | ||
| #! The procedure proves inclusion of the map entries in the account's storage map at the provided | ||
| #! index. Note that the advice map entry could contain multiple occurrences of a key-value pair | ||
| #! A -> B, but this procedure has no way of detecting this condition. | ||
| #! Such entries would naturally be deduplicated by the nature of the storage map or link map | ||
| #! (used to track the storage map delta), and so are not problematic for overall correctness. | ||
| #! | ||
| #! Inputs: | ||
| #! Operand stack: [MAP_ROOT, slot_idx] | ||
| #! Advice map: { MAP_ROOT: [MAP_ENTRIES] } | ||
| #! Outputs: [] | ||
| proc.add_storage_map_entries_to_delta |
There was a problem hiding this comment.
Thinking more about this, I wonder if this approach is completely secure. IIUC, here we check that the set of entries provided via the advice map is present in the the storage map with the specified root. However, we do not check that this set of entries is contains all entries in the storage map.
For example, let's say a storage map has 2 entries: [KEY1 |-> VAL1, KEY2 |-> VAL2] but the advice map contains an entry MAP_ROOT |-> [VAL2, KEY2]. As far as I can tell, this procedure will still pass and build the delta as if the map only contained KEY2 |-> VAL2 entry.
I'm not sure of the exact way to exploit this - but it feels a bit dangerous. Maybe the approach could be:
- Start with an empty storage map
- Read key value pairs from the advice map (as we do now)
- Insert these key-value pairs into the map.
Essentially, for new accounts, we'd be running a kind of a "deploy" procedure where we'd start with an empty account, and then insert the values into the storage (this should naturally update the delta, I think). Then, we'd check that the resulting commitment matches the initial account commitment, and only then we'd call mem_copy_initial_storage_slots.
There was a problem hiding this comment.
Thanks for pointing that out, I agree that's a problem with the current approach.
I changed it to roughly how you described it, with a minor difference. The initial slots are still copied before all entries are inserted, because while iterating the storage, we need to know what the type of each slot is, and so that must be done on initialized memory.
I'm also not sure if you meant comparing the storage commitment against the initial one or each storage map's root against the committed one, but I went with the latter. Both should give us the same validation, but the per-map assertion can help more with debugging (e.g. to figure out which one of multiple maps are incorrect), but that's a minor thing. It also means we don't have to recompute the storage commitment.
bobbinth
left a comment
There was a problem hiding this comment.
Thank you! Looks good. I left a few more comments inline.
bobbinth
left a comment
There was a problem hiding this comment.
Looks good! Thank you! I left just a few small comments inline.
Represent new accounts as
AccountDeltas instead of as fullAccountobjects.The main change is how 1) the delta tracker in the tx host is initialized and 2) how the in-kernel delta is initialized:
EMPTY_WORD -> NEW_VALUEEMPTY_WORD -> NEW_VALUEcompute_commitmenttime.This initialization is necessary because for new accounts, the full state is required, so that public observers of transactions can turn the resulting account delta into an
Account, onto which any subsequentAccountDeltacan be applied.Other Changes:
Account, while a partial state delta can only be applied to an existingAccount.AccountUpdateDetailsnow.ProvenTransaction::validate.TxAccountUpdateto that type itself, instead of bloating theProvenTransaction::validatefunction. We were actually only enforcing the empty transaction check for transactions against public accounts instead of all accounts, which was likely a result of the largevalidatefunction, so I think this is an improvement.TxAccountUpdatevalidity as part of construction instead of afterward.See also #1879 (comment) for some background on this PR.
Open questions:
closes #1879