feat(tpu-v2): provide swap protocol versioning#2324
Conversation
onur-ozkan
left a comment
There was a problem hiding this comment.
Is this blocking anything currently? These changes look like a hacky workaround rather than a being proper solution. It would be great if we can avoid this change.
| fn legacy_swap_version() -> u32 { 1 } | ||
|
|
||
| fn is_legacy_swap_version(swap_version: &u32) -> bool { *swap_version == 1 } |
There was a problem hiding this comment.
| fn legacy_swap_version() -> u32 { 1 } | |
| fn is_legacy_swap_version(swap_version: &u32) -> bool { *swap_version == 1 } | |
| const fn legacy_swap_version() -> u32 { 1 } | |
| fn is_legacy_swap_version(swap_version: &u32) -> bool { *swap_version == legacy_swap_version() } |
We need swap versioning to do a legacy fallback on order matching |
| /// In the future alls users will be using TPU V2 by default without "use_trading_proto_v2" configuration. | ||
| pub fn use_trading_proto_v2(mut self, use_trading_proto_v2: bool) -> Self { | ||
| if !use_trading_proto_v2 { | ||
| self.swap_version = 1; |
There was a problem hiding this comment.
we should use a const like LEGACY_SWAP_VERSION (indicating current version this code supports)
There was a problem hiding this comment.
Current version this code/repo supports is this const SWAP_VERSION.
If use_trading_proto_v2:false it means user didnt enable TPU
reused const fn 53d4b40
we need const function for #[serde(default = "legacy_swap_version")], so no need to create one more const variable.
UPD: I refactored this function a bit according to Onur notes, so please check newer commits
| pub fn use_trading_proto_v2(mut self, use_trading_proto_v2: bool) -> Self { | ||
| if !use_trading_proto_v2 { | ||
| self.swap_version = legacy_swap_version(); | ||
| } | ||
| self | ||
| } |
There was a problem hiding this comment.
Why not this?
| pub fn use_trading_proto_v2(mut self, use_trading_proto_v2: bool) -> Self { | |
| if !use_trading_proto_v2 { | |
| self.swap_version = legacy_swap_version(); | |
| } | |
| self | |
| } | |
| pub fn use_trading_proto_v2(mut self) -> Self { | |
| self.swap_version = legacy_swap_version(); | |
| self | |
| } |
or, at least rename it to something like maybe_**
There was a problem hiding this comment.
u suggested to change version to legacy protocol calling use_trading_proto_v2
please re read doc comment
/// When a new [TakerOrderBuilder::new] is created, it sets [SWAP_VERSION] by default.
/// However, if user has not specified in the config to use TPU V2,
/// the TakerOrderBuilder's swap_version is changed to legacy.
/// In the future alls users will be using TPU V2 by default without "use_trading_proto_v2" configuration.
pub fn use_trading_proto_v2(mut self, use_trading_proto_v2: bool) -> Self {There was a problem hiding this comment.
What I suggest is, either remove this boolean from the function and check it on the caller side or declare that in the function name with "maybe" prefix.
There was a problem hiding this comment.
What I suggest is, either remove this boolean from the function and check it on the caller side
I dont get it, on the caller side it is where? Right now it looks like, you call use_trading_proto_v2 and set legacy
pub fn use_trading_proto_v2(mut self) -> Self {
self.swap_version = legacy_swap_version();
self
}
There was a problem hiding this comment.
I didn't pay attention to the logic, the usage was just unclear. Looking at the logic, it's even more unclear.. Function is called use_trading_proto_v2 but it downgrades version to legacy if given boolean is false.
Why not just this:
pub fn use_trading_proto_legacy(mut self) -> Self {
self.swap_version = legacy_swap_version();
self
}and call it only when use_trading_proto_v2 is false?
There was a problem hiding this comment.
the thing is that use_trading_proto_v2 initially is a config boolean flag, so use_trading_proto_legacy is supposed to be a question
ok I refactored function, now its purpose is only to set legacy protocol a62f56a
| pub fn use_trading_proto_v2(mut self, use_trading_proto_v2: bool) -> Self { | ||
| if !use_trading_proto_v2 { | ||
| self.swap_version = legacy_swap_version(); | ||
| } | ||
| self |
| let alice_swap_v = maker_match.request.swap_version; | ||
|
|
||
| // Start legacy swap if taker uses legacy protocol (version 1) or if conditions for trading_proto_v2 aren't met. | ||
| if alice_swap_v == 1u32 || !ctx.use_trading_proto_v2() { |
There was a problem hiding this comment.
| if alice_swap_v == 1u32 || !ctx.use_trading_proto_v2() { | |
| if alice_swap_v == legacy_swap_version() || !ctx.use_trading_proto_v2() { |
| let bob_swap_v = taker_match.reserved.swap_version; | ||
|
|
||
| // Start legacy swap if maker uses legacy protocol (version 1) or if conditions for trading_proto_v2 aren't met. | ||
| if bob_swap_v == 1u32 || !ctx.use_trading_proto_v2() { |
There was a problem hiding this comment.
| if bob_swap_v == 1u32 || !ctx.use_trading_proto_v2() { | |
| if bob_swap_v == legacy_swap_version() || !ctx.use_trading_proto_v2() { |
|
|
||
| wasm_bindgen_test_configure!(run_in_browser); | ||
|
|
||
| const LEGACY_SWAP_V: u32 = 1; |
There was a problem hiding this comment.
legacy_swap_version can replace this need
| #[serde(default = "legacy_swap_version")] | ||
| #[serde(skip_serializing_if = "is_legacy_swap_version")] | ||
| pub swap_version: u32, |
There was a problem hiding this comment.
It's worth to create a separate type (or even a separate module for handling everyting related to versioning for swaps) for swap versioning, e.g., something like:
struct SwapVersion {
#[serde(default = "legacy_swap_version")]
#[serde(skip_serializing_if = "is_legacy_swap_version")]
swap_version: u32,
}which can help us to avoid this serialization rules applied all around the codebase.
I think this is a non-blocker improvement and I am okay to approve this PR without it. If you don't want to do it in this PR, can you leave a TODO note on one of them?
There was a problem hiding this comment.
I like this idea!
it will be safer to work with version field wrapped into structure with all annotations which we need, then adding manually version with annotations to p2p messages structures.
Im going to do it in the current pr to not refactoring everything in next PR.
There was a problem hiding this comment.
It's worth to create a separate type (or even a separate module for handling everyting related to versioning for swaps) for swap versioning, e.g., something like:
struct SwapVersion { #[serde(default = "legacy_swap_version")] #[serde(skip_serializing_if = "is_legacy_swap_version")] swap_version: u32, }which can help us to avoid this serialization rules applied all around the codebase.
I tested such wrapper, deserialisation and serialisation dont work as we would like if you dont provide serde annotation above the pub swap_version: SwapVersion, field.
https://github.com/laruh/benchmarks_and_tests/blob/merge-candidate/serde_example/src/tests.rs
So we still have to add annotations, but I think that looks better:

And everything is in separate module

I will push changes for review
onur-ozkan
left a comment
There was a problem hiding this comment.
Thanks!
LGTM other than 2 minor notes which are not blockers.
| impl SwapVersion { | ||
| pub(crate) fn is_legacy(&self) -> bool { self.version == legacy_swap_version() } |
There was a problem hiding this comment.
| impl SwapVersion { | |
| pub(crate) fn is_legacy(&self) -> bool { self.version == legacy_swap_version() } | |
| impl SwapVersion { | |
| pub(crate) const fn is_legacy(&self) -> bool { self.version == legacy_swap_version() } |
There was a problem hiding this comment.
Hmm why we can use const here if actual self.version value will be known at runtime?
I mean I can compile it, I thought it wont work. Is it bcz version has a primitive type?
There was a problem hiding this comment.
version size is known at compile time.
|
|
||
| #[derive(Copy, Clone, Debug, Deserialize, Eq, PartialEq, Serialize)] | ||
| pub struct SwapVersion { | ||
| pub version: u32, |
There was a problem hiding this comment.
Not a blocker but u32 doesn't seem necessary for swap versioning, u8 should be more than enough.
There was a problem hiding this comment.
I think it wont be a problem to move to u32, so we can start with u8
bab63f1
|
@onur-ozkan I noticed if user update the app having old swap entries in db, there could be error when we try to get such data, as structs expect version field Well, changes are about TPU (not in production yet), but I think if migration 13 adds swap_version column, then it should also handle setting old swaps to version 1. |
|
@laruh you should add this in a new migration 14. Existing DBs that ran migration 13 will not run it again. |
The current pr adds 13 migration. Dev doesn have it |
ah my bad. only checked the commit diff u linked :) |
updated my_swaps table migration, new swap_version field provided in Maker/TakerSwapStorage
de60aae to
fe1bfc3
Compare
|
Since the base branch was changed, I force-pushed an updated branch (cherry picked feature commits) with a clean git history to resolve conflicts |
ooh, I deleted the base branch after merging to dev, sorry about that. No need for another review from @onur-ozkan as I will give it a review before merging. |
Its ok, I have old original branches locally and can push them if someone needs it |
| /// Adds Swap Protocol version column to `my_swaps` table | ||
| pub const ADD_SWAP_VERSION_FIELD: &str = "ALTER TABLE my_swaps ADD COLUMN swap_version INTEGER;"; | ||
| /// Sets default value for `swap_version` to `1` for existing rows | ||
| pub const SET_DEFAULT_SWAP_VERSION: &str = "UPDATE my_swaps SET swap_version = 1 WHERE swap_version IS NULL;"; |
There was a problem hiding this comment.
About naming vars:
I think SET_DEFAULT_SWAP_VERSION here actually means not 'default' but 'existing'.
Plus, there is also a comment in the code:
/// When a new [TakerOrderBuilder::new] is created, it sets [SWAP_VERSION] by default.
So I guess, SWAP_VERSION is in fact SWAP_VERSION_DEFAULT
dimxy
left a comment
There was a problem hiding this comment.
Apart from my SWAP_VERSION nit,
LGTM.
(PS. I believe this is an interim fix for version selection mechanism, until we work out some general approach for it, to solve the old nodes compatibility problem)
shamardy
left a comment
There was a problem hiding this comment.
Great work! All these comments are non-blockers but good to have.
| ]; | ||
|
|
||
| /// Adds Swap Protocol version column to `my_swaps` table | ||
| pub const ADD_SWAP_VERSION_FIELD: &str = "ALTER TABLE my_swaps ADD COLUMN swap_version INTEGER;"; |
There was a problem hiding this comment.
I think it would be useful to add the version to stats_swaps table as well. It should be done in another PR though as this is should be merged soon.
There was a problem hiding this comment.
I just noticed that currently we don't need the swap version in my_swaps table as we have swap_type https://github.com/KomodoPlatform/komodo-defi-framework/blob/731e605fc2b53b9f4afa9e13e3682c5be7ed9cf6/mm2src/mm2_main/src/database/my_swaps.rs#L89 https://github.com/KomodoPlatform/komodo-defi-framework/blob/39515a9f3ea1089bb462e99c8cafb1049a920dbd/mm2src/mm2_main/src/lp_swap.rs#L152-L154 but version can be different from type in the future, for instance MAKER_SWAP_V2_TYPE which s TPU Maker can have multiple versions but one type. Just thought to mention this and the redundancy we have.
There was a problem hiding this comment.
I just noticed that currently we don't need the swap version in my_swaps table as we have swap_type
From what I see these TYPE consts were provided to separate legacy and swap_v2 taker/maker data formats in database. I think it is something different from swap protocol version.
Well may be in practice we will see, will taker/maker types be different from swap protocol version or not.
ps: This field is useful for MySwapForRpc, as it can be retrieved from both get_taker_swap_data_for_rpc and get_maker_swap_data_for_rpc. Having a ready swap_version field simplifies the process of determining the swap version.
There was a problem hiding this comment.
I understand, that's why I said currently :)
What I meant was that we could have converted this type to the version without adding a new column for now, that's all. But I like having version explicitly and resolving this comment.
There was a problem hiding this comment.
Leaving it only for stats_swaps #2324 (comment)
Inline serde annotations.
* dev: fix(derive_key_from_path): check length of current_key_material (#2356) chore(release): bump mm2 version to 2.4.0-beta (#2346) fix(tests): add additional testnet sepolia nodes to test code (#2358) fix(swaps): maintain legacy compatibility for negotiation messages (#2353) refactor(SwapOps): impl defaults for protocol specific swapops fns (#2354) feat(tpu-v2): provide swap protocol versioning (#2324) feat(wallet): add change mnemonic password rpc (#2317) fix(tpu-v2): fix tpu-v2 wait for payment spend and extract secret (#2261) feat(tendermint): unstaking/undelegation (#2330) fix(utxo-withdraw): get hw ctx only when `PrivKeyPolicy` is trezor (#2333) feat(event-streaming): API-driven subscription management (#2172) fix(hash-types): remove panic, enforce fixed-size arrays (#2279) fix(ARRR): store unconfirmed change output (#2276) feat(tendermint): staking/delegation (#2322) chore(deps): `timed-map` migration (#2247) fix(mem-leak): `running_swap` never shrinks (#2301) chore(dep-bump): libp2p (#2326) refactor(build script): rewrite the main build script (#2319)
* dev: fix(derive_key_from_path): check length of current_key_material (#2356) chore(release): bump mm2 version to 2.4.0-beta (#2346) fix(tests): add additional testnet sepolia nodes to test code (#2358) fix(swaps): maintain legacy compatibility for negotiation messages (#2353) refactor(SwapOps): impl defaults for protocol specific swapops fns (#2354) feat(tpu-v2): provide swap protocol versioning (#2324) feat(wallet): add change mnemonic password rpc (#2317) fix(tpu-v2): fix tpu-v2 wait for payment spend and extract secret (#2261) feat(tendermint): unstaking/undelegation (#2330) fix(utxo-withdraw): get hw ctx only when `PrivKeyPolicy` is trezor (#2333) feat(event-streaming): API-driven subscription management (#2172) fix(hash-types): remove panic, enforce fixed-size arrays (#2279) fix(ARRR): store unconfirmed change output (#2276) feat(tendermint): staking/delegation (#2322) chore(deps): `timed-map` migration (#2247) fix(mem-leak): `running_swap` never shrinks (#2301) chore(dep-bump): libp2p (#2326) refactor(build script): rewrite the main build script (#2319)
* dev: (24 commits) fix(eth-tpu): remove state from funding validation (GLEECBTC#2334) improvement(rpc-server): rpc server dynamic port allocation (GLEECBTC#2342) fix(tests): fix or ignore unstable tests (GLEECBTC#2365) fix(fs): make `filter_files_by_extension` return only files (GLEECBTC#2364) fix(derive_key_from_path): check length of current_key_material (GLEECBTC#2356) chore(release): bump mm2 version to 2.4.0-beta (GLEECBTC#2346) fix(tests): add additional testnet sepolia nodes to test code (GLEECBTC#2358) fix(swaps): maintain legacy compatibility for negotiation messages (GLEECBTC#2353) refactor(SwapOps): impl defaults for protocol specific swapops fns (GLEECBTC#2354) feat(tpu-v2): provide swap protocol versioning (GLEECBTC#2324) feat(wallet): add change mnemonic password rpc (GLEECBTC#2317) fix(tpu-v2): fix tpu-v2 wait for payment spend and extract secret (GLEECBTC#2261) feat(tendermint): unstaking/undelegation (GLEECBTC#2330) fix(utxo-withdraw): get hw ctx only when `PrivKeyPolicy` is trezor (GLEECBTC#2333) feat(event-streaming): API-driven subscription management (GLEECBTC#2172) fix(hash-types): remove panic, enforce fixed-size arrays (GLEECBTC#2279) fix(ARRR): store unconfirmed change output (GLEECBTC#2276) feat(tendermint): staking/delegation (GLEECBTC#2322) chore(deps): `timed-map` migration (GLEECBTC#2247) fix(mem-leak): `running_swap` never shrinks (GLEECBTC#2301) ...
* dev: feat(rpc): add is_success field to legacy MySwapStatusResponse (#2371) fix(key-derivation): use stored Argon2 parameters instead of default values (#2360) fix(tests): stabilize `tendermint_coin::test_claim_staking_rewards` (#2373) improvement(RPCs): group staking rpcs under a namespace (#2372) feat(tendermint): claim delegation rewards (#2351) fix(eth-tpu): remove state from funding validation (#2334) improvement(rpc-server): rpc server dynamic port allocation (#2342) fix(tests): fix or ignore unstable tests (#2365) fix(fs): make `filter_files_by_extension` return only files (#2364) fix(derive_key_from_path): check length of current_key_material (#2356) chore(release): bump mm2 version to 2.4.0-beta (#2346) fix(tests): add additional testnet sepolia nodes to test code (#2358) fix(swaps): maintain legacy compatibility for negotiation messages (#2353) refactor(SwapOps): impl defaults for protocol specific swapops fns (#2354) feat(tpu-v2): provide swap protocol versioning (#2324) feat(wallet): add change mnemonic password rpc (#2317)
ref issue #2337
ordermatch structs which got swap_version field:
swap structs. if swap started swap_version will be saved in db:
swap version 2 is tpu v2 protocol
legacy swap protocol is 1 version
When Taker or Maker build order, the value
const SWAP_VERSION: u32 = 2;is used for order Builder.However in
create_maker_order/lp_auto_buyif user didnt usectx.use_trading_proto_v2()in config we set legacy protocol version to the order.