fix(indexeddb): fix IDB cursor.continue_() call after drop#2028
fix(indexeddb): fix IDB cursor.continue_() call after drop#2028
Conversation
Signed-off-by: borngraced <samuelonoja970@gmail.com>
|
@borngraced please fix wasm lint and PR name lint (like "allow first result using IDB cursor") |
Signed-off-by: borngraced <samuelonoja970@gmail.com>
done |
Signed-off-by: borngraced <samuelonoja970@gmail.com>
Signed-off-by: borngraced <samuelonoja970@gmail.com>
Signed-off-by: borngraced <samuelonoja970@gmail.com>
onur-ozkan
left a comment
There was a problem hiding this comment.
Great work!
At the end, it would be super nice to have unit tests for fn next and fn first in mm2src/mm2_db/src/indexed_db/drivers/cursor/cursor.rs module.
| DbCursorEvent::NextItem { | ||
| result_tx, | ||
| first_result_only, | ||
| } => { | ||
| result_tx.send(cursor.next(first_result_only).await).ok(); |
There was a problem hiding this comment.
Instead of making NextItem complicated, what do you think adding a new event called FirstItem?
| } | ||
|
|
||
| pub(crate) async fn next(&mut self) -> CursorResult<Option<(ItemId, Json)>> { | ||
| pub(crate) async fn next(&mut self, first_result_only: bool) -> CursorResult<Option<(ItemId, Json)>> { |
There was a problem hiding this comment.
I believe this function should have consistent behavior without relying on conditional flags. Could we possibly create a new function called fn first and retain the existing function as is? The idea is to separate the common parts, which can then be called from both this function and fn first.
There was a problem hiding this comment.
thanks done this and it improved the code greatly!
|
Will duplicate note in the PR. Also there is a problem with |
I actually wanted to ask whether it solves the issue with cursor only when we need the first found item. I assume a more general fix is preferred because this one looks like a temporary workaround. |
It turned out that whole problem was in obtaining the first item. In CursorDriver::next method. UPD: previously when we were trying to get one item by next(), the process continued in loop |
Signed-off-by: borngraced <samuelonoja970@gmail.com>
pub(crate) async fn next(&mut self) -> CursorResult<Option<(ItemId, Json)>> {
loop {
...
match cursor_action {
CursorAction::Continue => cursor.continue_().map_to_mm(|e| CursorError::AdvanceError {
description: stringify_js_error(&e),
})?,
CursorAction::ContinueWithValue(next_value) => {
cursor
.continue_with_key(&next_value)
.map_to_mm(|e| CursorError::AdvanceError {
description: stringify_js_error(&e),
})?
},
// Don't advance the cursor.
// Here we set the `stopped` flag so we return `Ok(None)` at the next iteration immediately.
// This is required because `item_action` can be `CollectItemAction::Include`,
// and at this iteration we will return `Ok(Some)`.
CursorAction::Stop => self.stopped = true,
};
match item_action {
CursorItemAction::Include => return Ok(Some(item.into_pair())),
// Try to fetch the next item.
CursorItemAction::Skip => (),
}
}
}
/// if getting the first result is what we want
pub(crate) async fn first(&mut self) -> CursorResult<Option<(ItemId, Json)>> {
let event = match self.cursor_item_rx.next().await {
Some(event) => event,
None => return Ok(None),
};
let _cursor_event = event.map_to_mm(|e| CursorError::ErrorOpeningCursor {
description: stringify_js_error(&e),
})?;
let cursor = match cursor_from_request(&self.cursor_request)? {
Some(cursor) => cursor,
// No more items.
None => return Ok(None),
};
let (_, js_value) = match (cursor.key(), cursor.value()) {
(Ok(key), Ok(js_value)) => (key, js_value),
// No more items.
_ => return Ok(None),
};
let item: InternalItem =
deserialize_from_js(js_value).map_to_mm(|e| CursorError::ErrorDeserializingItem(e.to_string()))?;
Ok(Some(item.into_pair()))
}
```
|
|
@laruh i improved the impl. so you can test it again. thanks |
|
@borngraced recheck clippy or fmt please, CI fails |
not an issue with my pr? https://github.com/KomodoPlatform/komodo-defi-framework/actions/runs/7166781305/job/19511514652?pr=2028#step:3:8 |
Hmm lets rerun to check UPD: yeah, its not your branch issue. locally its fine. |
Signed-off-by: borngraced <samuelonoja970@gmail.com>
After many hours of debugging I found another fix which introduces a method called full explanation: #2028 (comment) |
|
@borngraced You can cherry pick the last commit from |
@borngraced Thanks for the great find and fix. Can we use |
We can use indexedb curosor.advance to achieve similar behavior as and something like this for limit const limit = 10;
let count = 0;
if cursor && count < limit {
cursor.continue();
count++;
}but I'll research and explore other options for sure |
shamardy
left a comment
There was a problem hiding this comment.
Thank you for the fixes! A few more comments :)
| // Cursor returns values from the lowest to highest key indexes. | ||
| // But we need to get the most highest height, so reverse the cursor direction. | ||
| .reverse() | ||
| .where_(condition) |
There was a problem hiding this comment.
What would happen if there are more than one object/record that satisfies the condition? I see that we return the first object, are we sure we won't get Uncaught Error: closure invoked recursively or after being dropped. This is part of the reason I wanted where and limit, so we can get more than one object that satisfies the condition, if we want this in other cases (other than the case here where we want only the first object).
P.S. If we don't get the error when there are more than one object that satisfies the condition, then the inclusion of limit and refactoring of where to return more than one object if they exist can be done in a different PR since this PR purpose is to fix the error only.
There was a problem hiding this comment.
hmm this seems like another enhancement that's not related to this PR.
This PR only handle case for retrieving the first item that meets a condition(if specified) which solves js exception error: Uncaught Error: closure invoked recursively or after being But for sure this PR indeed fixed that and also optimizes calls to indexeddb.
I will work on the later issue during next print.
shamardy
left a comment
There was a problem hiding this comment.
LGTM but for a few nits! Please fix the first comment so that I can finally merge this PR.
* evm-hd-wallet: (27 commits) Fix todo comments Fix HDAddressOps::Address trait bounds fix(indexeddb): fix IDB cursor.continue_() call after drop (GLEECBTC#2028) security bump for `h2` (GLEECBTC#2062) fix(makerbot): allow more than one prices url in makerbot (GLEECBTC#2027) fix(wasm worker env): refactor direct usage of `window` (GLEECBTC#1953) feat(nft): nft abi in withdraw_nft RPC, clear_nft_db RPC (GLEECBTC#2039) refactor(utxo): refactor utxo output script creation (GLEECBTC#1960) feat(ETH): balance event streaming for ETH (GLEECBTC#2041) chore(release): bump mm2 version to 2.1.0-beta (GLEECBTC#2044) feat(trezor): add segwit support for withdraw with trezor (GLEECBTC#1984) chore(config): remove vscode launchjson (GLEECBTC#2040) feat(trading-proto-upgrade): wasm DB, kickstart, refund states, v2 RPCs (GLEECBTC#2015) feat(UTXO): balance event streaming for Electrum clients (GLEECBTC#2013) feat(tx): add new sign_raw_transaction rpc for UTXO and EVM coins (GLEECBTC#1930) fix(p2p): handle encode_and_sign errors (GLEECBTC#2038) chore(release): add changelog entries for v2.0.0-beta (GLEECBTC#2037) chore(network): write network information to stdout (GLEECBTC#2034) fix(price_endpoints): add cached url (GLEECBTC#2032) deps(network): sync with upstream yamux (GLEECBTC#2030) ...
* dev: feat(zcoin): ARRR WASM implementation (GLEECBTC#1957) feat(trading-proto-upgrade): locked amounts, kmd burn and other impl (GLEECBTC#2046) fix(indexeddb): set stop on success cursor condition (GLEECBTC#2067) feat(config): add `max_concurrent_connections` to mm2 config (GLEECBTC#2063) feat(stats_swaps): add gui/mm_version in stats db (GLEECBTC#2061) fix(indexeddb): fix IDB cursor.continue_() call after drop (GLEECBTC#2028) security bump for `h2` (GLEECBTC#2062) fix(makerbot): allow more than one prices url in makerbot (GLEECBTC#2027) fix(wasm worker env): refactor direct usage of `window` (GLEECBTC#1953) feat(nft): nft abi in withdraw_nft RPC, clear_nft_db RPC (GLEECBTC#2039) refactor(utxo): refactor utxo output script creation (GLEECBTC#1960) feat(ETH): balance event streaming for ETH (GLEECBTC#2041) chore(release): bump mm2 version to 2.1.0-beta (GLEECBTC#2044) feat(trezor): add segwit support for withdraw with trezor (GLEECBTC#1984)


If we try to get first result from
indexeddb cursorlike this https://github.com/KomodoPlatform/komodo-defi-framework/blob/272f46b00778e5aacdc89ab1288891cfe5185e02/mm2src/coins/z_coin/storage/blockdb/blockdb_idb_storage.rs#L90-L98 this error is being logged on the browserUncaught Error: closure invoked recursively or after being droppedon the web becauseCursorDrivernext loop is still looping due to this https://github.com/KomodoPlatform/komodo-defi-framework/blob/272f46b00778e5aacdc89ab1288891cfe5185e02/mm2src/mm2_db/src/indexed_db/drivers/cursor/cursor.rs#L286-L302 after getting the first value we requested andCursoris dropped as we callednext()just once here .I found a fix which introduces a method called
where_that takes a closure. The closure returns a boolean, allowing you to control when a loop should stop. This approach works reliably in various situations.Example - Simple Condition
When calling the
nextmethod once, you can easily obtain the first available resultwithout needing to continue the cursor using a simple closure like
where_first():Example - Complex Condition
For more complex conditions, such as deserializing an IndexedDB data, handling errors, and checking a specific condition, you can use a closure like the one below:
related: #2019