Consensus changes for SIP-19#17508
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
3 Ignored Deployments
|
akichidis
left a comment
There was a problem hiding this comment.
Adding a few comments for now - will continue reviewing
consensus/core/src/transaction.rs
Outdated
There was a problem hiding this comment.
Providing this here as alternative: we could probably use the sender to send a vec![transaction_guard] and let the TransactionConsumer pretty much do what it was doing before without any significant modifications apart from caching now multiple transactions instead of only one if we get over allowed size. Then a this point we just need to return/wait on the returned inlucluded_in_block_ack . I think the optimisation of using one ack sender makes sense, but I feel that it complicates a bit things on the TransactionConsumer side.
There was a problem hiding this comment.
Thanks for pointing this out!
I guess in that case we need to spawn a separated task to wait on multiple oneshot channels (one for each TransactionGuard), then after all channels have been signalled, the task eventually signals the oneshot channel returned by submit_no_wait. I am not quite familiar with the runtime so not sure how much burden will this create.
Or we could actually change the API to remove the 'no wait' semantic - since it is already async.
I can proceed either way if people are okay with it.
There was a problem hiding this comment.
I think we can use something like futures::future::join_all() to await on all oneshot channels without spawning a new task. But I'm not sure if it is an improvement over using only one ack callback. Maybe keep the current logic and we can iterate on this part in future? Wdyt @akichidis?
There was a problem hiding this comment.
Yeah anything like futures::future::join_all() would do here , even a simple loop, as it's all or nothing. @mwtian it's not a performance improvement but rather , to my view, a complexity improvement. I felt that with the current approach there is lots for break up/ management in the TransactionConsumer which I would avoid. Just to be clear, not a deal breaker as solution, but that would be my preference.
consensus/core/src/transaction.rs
Outdated
There was a problem hiding this comment.
I think we can use something like futures::future::join_all() to await on all oneshot channels without spawning a new task. But I'm not sure if it is an improvement over using only one ack callback. Maybe keep the current logic and we can iterate on this part in future? Wdyt @akichidis?
|
@shio-coder is attempting to deploy a commit to the Mysten Labs Team on Vercel. A member of the Team first needs to authorize it. |
8b5c3da to
02d13a8
Compare
akichidis
left a comment
There was a problem hiding this comment.
Thanks for the changes @shio-coder! A few more comments from my side
narwhal/worker/src/batch_maker.rs
Outdated
There was a problem hiding this comment.
Just to reconfirm, based on the SIP-19 Note that the Soft Bundle API does not provide a strict ordering guarantee. However, it does provide an extremely high probability of being ordered correctly. , the intention is best effort to fit multiple transactions in the same batch, right ?
Also how do we intend to measure the successful inclusion of the soft bundle transactions in same batch/block? I assume the appearance in checkpoints would be one way if lower level metrics don't make much sense
There was a problem hiding this comment.
the intention is best effort to fit multiple transactions in the same batch
That's right! The intention is to include the txs as a bundle if feasible to do so. The pre-existing constraints (e.g. batch size limit) are there for a good reason, so that should come first. It is alright for a Soft Bundle to fail.
I assume the appearance in checkpoints would be one way if lower level metrics don't make much sense
Exactly. It seems natural for the client to monitor the bundle it submitted, by looking at whether transactions are ordered as specified in the eventual checkpoint. If the bundles are failing frequently, one can tell that something must be wrong.
02d13a8 to
631751b
Compare
## Description This is a follow up change after PR #17508 , for [SIP-19](https://github.com/sui-foundation/sips/blob/main/sips/sip-19.md). The changeset focuses on sui-core, particularly authority server, to add a new API `SoftBundleCertifiedTransactionsV3` that allows submission of a Soft Bundle through gRPC. The API accepts a list of `CertifiedTransaction` for input, then returns either nothing (should `wait_for_effects` be `false`) or a list of `HandleCertificateResponseV3` containing the execution result. The API is named `_v3` because it reuses `HandleCertificateResponseV3`. Besides introducing a new feature flag and a protocol config field, a local node config field `enable_soft_bundle` is also added, to allow a validator node to turn off soft bundle support without having to upgrade the entire network. This can be useful should the feature causes undesired effect. ## Test plan The feature is not exactly user-facing and do not break anything. --- ## Release notes - [x] Protocol: New feature flag and protocol config fields will be introduced. - [x] Nodes (Validators and Full nodes): A new API `SoftBundleCertifiedTransactionsV3` will be added. - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: --------- Co-authored-by: Mark Logan <mark@mystenlabs.com>
## Description This is a follow up change after PR #17508 and #17868 , for SIP-19. The change adds a unit test for soft bundle failure modes, as well as enable soft bundle feature flag in devnet and testnet. ## Test plan The feature is not exactly user-facing and do not break anything. --- ## Release notes - [x] Protocol: Enable soft bundle on Protocol Version 50 in devnet and testnet. - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: --------- Co-authored-by: Mark Logan <103447440+mystenmark@users.noreply.github.com>
Description
Please refer to this SIP draft:
sui-foundation/sips#19
The PR focuses only on consensus related changes, covering both Narwhal and Mysticeti.
We will proceed to work on the remaining parts once this gets merged.
The PR includes a commit from @mwtian that changes
TransactionConsumerfor ack handling in a bundle context.Special thanks to @mwtian for taking time to discussions and authoring the change above!
Test Plan
The feature is not exactly user-facing and do not break anything.
Type of Change (Check all that apply)