Skip to content

feat: relay snapshot DB reconcile for stale order status#67

Merged
arkanoider merged 3 commits into
mainfrom
feat/relay-order-db-reconcile
May 11, 2026
Merged

feat: relay snapshot DB reconcile for stale order status#67
arkanoider merged 3 commits into
mainfrom
feat/relay-order-db-reconcile

Conversation

@arkanoider

@arkanoider arkanoider commented May 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

Fixes local SQLite orders rows staying non-terminal (e.g. pending) when Mostro already published expired / canceled (etc.) nostr order events while Mostrix was closed. The Orders tab reconcile previously fetched pending-only, so the latest terminal state from the relay was dropped before it could touch the DB.

What changed

  • Single relay fetch per tick: aggregate latest SmallOrder per order id (no status filter), then:
    • run terminal status reconcile against local rows (should_apply_status_transition + Order::update_status);
    • fill the in-memory book with pending + currency filter only (unchanged UX).
  • Live nostr order events: reconcile terminal statuses before the pending-filtered UI update so DB stays aligned without waiting for the 30s poll.
  • Startup: one run_relay_order_db_reconcile_once when relays are reachable, before user order chat hydration.
  • Thread SqlitePool through start_fetch_scheduler / spawn_fetch_scheduler_loops (main + key reload + scheduler reload + reconnect paths).
  • Naming/doc: replace misleading create_seven_days_filter with create_mostro_list_fetch_filter and document that there is no since—only author + kind + MOSTRO_LIST_FETCH_EVENT_LIMIT (50).

Limitations

Relay list snapshots are still capped by MOSTRO_LIST_FETCH_EVENT_LIMIT; very old terminal updates might not appear in the fetched window until a targeted query exists.

Breaking changes

  • Public re-export: create_seven_days_filter removed from mostrix::util; use create_mostro_list_fetch_filter and/or MOSTRO_LIST_FETCH_EVENT_LIMIT.

Summary by CodeRabbit

  • New Features

    • Automatic reconciliation to keep local order statuses synced with relay state, including startup and targeted background reconciliation.
    • Background scheduler now reliably receives DB context for its tasks.
  • Refactor

    • Order fetch/parsing and reconciliation logic reorganized for latest-per-id aggregation and targeted relay queries.
  • Bug Fixes

    • Fetching capped at 500 events with strict author/kind filtering (removed 7-day window).
  • Documentation

    • Coding standards example updated to show new exports and limits.
  • Tests

    • Added reconciliation tests for terminal-status updates.

Review Change Stack

When listings expire or cancel on the daemon while Mostrix is offline, DB rows could stay non-terminal and skew My Trades. Fetch full relay snapshots (not pending-only), merge latest per order id, update local rows for terminal statuses via should_apply_status_transition, and keep the Orders tab on pending+currency filtered subset.

Also rename create_seven_days_filter to create_mostro_list_fetch_filter with accurate docs and MOSTRO_LIST_FETCH_EVENT_LIMIT (no since window was ever applied).

Co-authored-by: Cursor <cursoragent@cursor.com>
@coderabbitai

coderabbitai Bot commented May 11, 2026

Copy link
Copy Markdown
Contributor

Walkthrough

Threads SqlitePool into the fetch scheduler, replaces seven-day filters with an event-limit filter, adds aggregation/fetch helpers, introduces a relay-to-DB reconciliation module (batch + targeted ticks), updates startup/UI wiring to run reconciliation, and adds tests and a models helper for targeted IDs.

Changes

Order Reconciliation & Scheduler

Layer / File(s) Summary
Filter & Data Shape
src/util/filters.rs, src/util/mod.rs, docs/CODING_STANDARDS.md
Replaces create_seven_days_filter with create_mostro_list_fetch_filter and MOSTRO_LIST_FETCH_EVENT_LIMIT = 500; updates create_filter dispatch and util re-exports; docs example updated.
Order Aggregation & Fetch Helpers
src/util/order_utils/helper.rs
Adds aggregate_latest_orders_by_id(), fetch_mostro_order_events(), pending_orders_for_book(), fetch_small_order_by_id_from_relay(), widens is_terminal_trade_status visibility; parse/get flows now reuse these helpers and tests extended.
Module Declaration & Re-exports
src/util/order_utils/mod.rs
Adds relay_order_db_reconcile submodule and publicly re-exports reconciliation helpers and constants.
Relay-to-DB Order Reconciliation
src/util/order_utils/relay_order_db_reconcile.rs
New module with reconcile_terminal_order_statuses_from_relay(), run_relay_order_db_reconcile_once(), run_targeted_relay_order_db_reconcile_tick(), reconcile_one_order_if_terminal(), TARGETED_RELAY_RECONCILE_MAX_PER_TICK, and async tests validating DB updates.
Fetch Scheduler Pool Integration
src/util/order_utils/fetch_scheduler.rs
start_fetch_scheduler and spawn_fetch_scheduler_loops accept SqlitePool; periodic tick now fetches relay events, aggregates latest-by-id, reconciles terminal statuses via the pool, computes pending orders, and replaces in-memory orders; live-notification path reconciles per-order before applying updates.
Startup & UI Handler Wiring
src/main.rs, src/ui/key_handler/async_tasks.rs
Passes pool to scheduler at startup, conditionally runs run_relay_order_db_reconcile_once/run_targeted_relay_order_db_reconcile_tick when relays are reachable, and updates UI handlers to pass the DB pool into scheduler spawns.
Models: Targeted Reconcile IDs
src/models.rs
Adds Order::list_ids_for_targeted_relay_reconcile(pool) to select non-terminal orders with persisted trade keys for targeted reconciliation.

Sequence Diagram

sequenceDiagram
  participant Client as Relay Client
  participant Scheduler as Fetch Scheduler
  participant Helper as Order Helpers
  participant Reconcile as Reconciliation Module
  participant DB as SQLite DB

  Scheduler->>Helper: fetch_mostro_order_events(client, mostro_pubkey)
  Helper->>Client: query events (create_mostro_list_fetch_filter)
  Client-->>Helper: event batch
  Helper-->>Scheduler: aggregated latest_by_id map
  Scheduler->>Reconcile: reconcile_terminal_order_statuses_from_relay(pool, latest_map)
  Reconcile->>DB: load/update orders by id
  DB-->>Reconcile: update results
  Reconcile-->>Scheduler: Ok(())
  Scheduler->>Helper: pending_orders_for_book(latest_map)
  Helper-->>Scheduler: Vec<SmallOrder>
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • MostroP2P/mostrix#21: Related edits to util filter API and re-exports (create_seven_days_filter → create_mostro_list_fetch_filter).
  • MostroP2P/mostrix#22: Related changes in order_utils helper parsing and fetch flows.
  • MostroP2P/mostrix#24: Related fetch scheduler and start_fetch_scheduler/spawn_fetch_scheduler_loops changes.

Suggested reviewers

  • grunch
  • AndreaDiazCorreia
  • Catrya

Poem

I hopped through code with a tiny scroll,
Threaded a pool to the scheduler's soul.
Relay events met DB light with grace,
Orders found their rightful place.
🐇🥕 Cheers for reconciled rows and tidy trace!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main objective: implementing relay snapshot database reconciliation for stale order statuses, which is the core feature of this changeset.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/relay-order-db-reconcile

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
src/util/order_utils/helper.rs (1)

320-329: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Prefer terminal updates when two order events share the same second.

Line 325 only replaces the stored order when new_ts > old_ts. Because created_at is second-granularity here, a pending event and a later expired/canceled event emitted in the same second will keep whichever one was seen first. That can leave the snapshot non-terminal and bypass the DB reconcile this PR is adding.

🔧 Localized fix
         latest_by_id
             .entry(order_id)
             .and_modify(|existing| {
                 let new_ts = order.created_at.unwrap_or(0);
                 let old_ts = existing.created_at.unwrap_or(0);
-                if new_ts > old_ts {
+                let prefer_new = new_ts > old_ts
+                    || (new_ts == old_ts
+                        && matches!(order.status, Some(status) if is_terminal_trade_status(status))
+                        && !matches!(existing.status, Some(status) if is_terminal_trade_status(status)));
+                if prefer_new {
                     *existing = order.clone();
                 }
             })
             .or_insert(order);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/util/order_utils/helper.rs` around lines 320 - 329, The current update in
latest_by_id only replaces on new_ts > old_ts, which leaves earlier-seen
non-terminal events when two events share created_at second; change the logic
inside the and_modify closure for latest_by_id (the order_id / existing / order
handling) to also prefer terminal events when timestamps are equal: replace the
condition with one that updates when new_ts > old_ts OR (new_ts == old_ts AND
is_terminal(&order) && !is_terminal(existing)); add or reuse a small
is_terminal(Order) helper if needed to detect terminal states (expired/canceled)
so terminal events win ties even within the same second.
src/ui/key_handler/async_tasks.rs (1)

215-229: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Await relay→DB reconcile before re-hydrating active-order DM state.

This respawns the scheduler and then immediately calls hydrate_startup_active_order_dm_state(pool), but the terminal-status reconcile now runs in the background inside spawn_fetch_scheduler_loops. After an offline period, these reload paths can still hydrate chats for orders that the relay already expired/canceled because the DB has not been reconciled yet. Startup was fixed by awaiting run_relay_order_db_reconcile_once; apply_pending_fetch_scheduler_reload and reload_runtime_session_after_reconnect need the same sequencing.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/ui/key_handler/async_tasks.rs` around lines 215 - 229, Ensure the
relay→DB reconciliation completes before rehydrating active-order DM state: call
and await run_relay_order_db_reconcile_once(...) and only after it finishes
invoke hydrate_startup_active_order_dm_state(pool).apply the same sequencing
change to apply_pending_fetch_scheduler_reload and
reload_runtime_session_after_reconnect so they await
run_relay_order_db_reconcile_once(...) before calling any hydrate or reload
logic; reference spawn_fetch_scheduler_loops,
hydrate_startup_active_order_dm_state, run_relay_order_db_reconcile_once,
apply_pending_fetch_scheduler_reload, and reload_runtime_session_after_reconnect
to locate the spots to reorder.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/util/order_utils/relay_order_db_reconcile.rs`:
- Around line 210-213: The test currently only ensures
reconcile_terminal_order_statuses_from_relay(&pool,
&relay_latest).await.unwrap() doesn't panic; add a postcondition that queries
the DB via the same test pool to assert no rows were inserted or updated (e.g.,
fetch count or fetch orders by relay id and assert it equals 0 or equals the
original pre-reconcile state). Use the existing pool handle and relay_latest
identifiers in your assertion (call the repository/select helper used elsewhere
in tests or run a SELECT COUNT(*) on the orders table for relay_latest entries)
immediately after the reconcile call to verify no changes occurred.
- Around line 51-53: The current use of Order::get_by_id(pool,
&order_id.to_string()).await silently ignores all errors; change it to match the
Result so you only treat RowNotFound as the benign case and surface or log any
other DB errors. Specifically, replace the let Ok(...) else { return } pattern
by matching the result of Order::get_by_id, keep the early return for
RowNotFound, but for other Err(e) either return an Err/propagate the error or
call the reconciler logger (include e) so non-RowNotFound failures are not
swallowed; refer to Order::get_by_id, pool, order_id and the RowNotFound variant
when implementing this handling.
- Around line 78-82: The in-memory SQLite tests use async fn test_pool() which
currently calls SqlitePool::connect("sqlite::memory:") causing multiple pooled
connections to see different databases; change test_pool to create the pool with
SqlitePoolOptions::new().max_connections(1).connect("sqlite::memory:").await
(preserving the expect message) so the pool is limited to a single connection
and the tests (which call test_pool) will see the same in-memory DB.

---

Outside diff comments:
In `@src/ui/key_handler/async_tasks.rs`:
- Around line 215-229: Ensure the relay→DB reconciliation completes before
rehydrating active-order DM state: call and await
run_relay_order_db_reconcile_once(...) and only after it finishes invoke
hydrate_startup_active_order_dm_state(pool).apply the same sequencing change to
apply_pending_fetch_scheduler_reload and reload_runtime_session_after_reconnect
so they await run_relay_order_db_reconcile_once(...) before calling any hydrate
or reload logic; reference spawn_fetch_scheduler_loops,
hydrate_startup_active_order_dm_state, run_relay_order_db_reconcile_once,
apply_pending_fetch_scheduler_reload, and reload_runtime_session_after_reconnect
to locate the spots to reorder.

In `@src/util/order_utils/helper.rs`:
- Around line 320-329: The current update in latest_by_id only replaces on
new_ts > old_ts, which leaves earlier-seen non-terminal events when two events
share created_at second; change the logic inside the and_modify closure for
latest_by_id (the order_id / existing / order handling) to also prefer terminal
events when timestamps are equal: replace the condition with one that updates
when new_ts > old_ts OR (new_ts == old_ts AND is_terminal(&order) &&
!is_terminal(existing)); add or reuse a small is_terminal(Order) helper if
needed to detect terminal states (expired/canceled) so terminal events win ties
even within the same second.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: e479ff5e-6ff5-41bd-86c5-aac620dcb43a

📥 Commits

Reviewing files that changed from the base of the PR and between 3504f48 and ccabec6.

📒 Files selected for processing (9)
  • docs/CODING_STANDARDS.md
  • src/main.rs
  • src/ui/key_handler/async_tasks.rs
  • src/util/filters.rs
  • src/util/mod.rs
  • src/util/order_utils/fetch_scheduler.rs
  • src/util/order_utils/helper.rs
  • src/util/order_utils/mod.rs
  • src/util/order_utils/relay_order_db_reconcile.rs

Comment thread src/util/order_utils/relay_order_db_reconcile.rs Outdated
Comment thread src/util/order_utils/relay_order_db_reconcile.rs
Comment thread src/util/order_utils/relay_order_db_reconcile.rs

@mostronatorcoder mostronatorcoder Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the PR carefully, and I do see a blocker in the current reconciliation design.

Blocker

The new relay-to-DB reconcile path fixes the previous pending-only bug, but it now depends entirely on the global capped snapshot returned by create_mostro_list_fetch_filter().

That filter no longer uses a time window and instead applies only:

  • author
  • kind
  • MOSTRO_LIST_FETCH_EVENT_LIMIT = 500

Then both startup reconcile and periodic reconcile do:

  • fetch latest global order events
  • aggregate latest event per order id
  • reconcile local DB rows from that bounded snapshot

The problem is that this makes local DB correctness for a given stale order depend on whether that order’s terminal event happens to be inside the relay’s most recent 500 order events overall.

So the PR fixes one class of stale-status bug, but it also means another perfectly valid stale local row can now remain unreconciled indefinitely simply because enough unrelated order traffic pushed its terminal event out of the global window.

I know the PR description mentions this as a limitation, but in this case it is not just a limitation of presentation. It directly affects the correctness of the new reconciliation mechanism, because the feature claim is specifically about aligning stale local rows with relay truth.

I think this needs a more targeted fetch path for the local rows that are actually candidates for reconciliation, instead of relying on a generic global capped list snapshot.

For example, reconciling local non-terminal rows by querying specific order identifiers (or another targeted relay lookup strategy) would avoid this false-negative window.

Because this can silently leave the DB in the exact stale state the PR is trying to solve, I am marking it as request changes.

@mostronatorcoder

Copy link
Copy Markdown
Contributor

I reviewed the PR in detail and left a formal request changes review.

My main concern is not the specific RowNotFound point here, but the broader reconciliation strategy introduced by this PR.

Right now the local DB reconcile depends on a global capped relay snapshot (MOSTRO_LIST_FETCH_EVENT_LIMIT = 500) instead of a targeted lookup for the local non-terminal rows that actually need reconciliation.

That means a stale local order can still remain stale indefinitely if its terminal relay event falls outside the latest global window due to unrelated traffic.

So even though this PR improves the previous pending-only behavior, I do not think the current reconcile design is strong enough yet for the correctness claim it is making.

@arkanoider

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented May 11, 2026

Copy link
Copy Markdown
Contributor
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@mostronatorcoder mostronatorcoder Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed the latest changes.

The main blocker I raised is addressed now.

The PR no longer relies only on the global capped relay snapshot for DB correctness. It keeps the snapshot-based reconcile as a cheap broad pass, but adds a targeted per-order relay reconcile for local non-terminal rows, with bounded work per tick and round-robin progression.

That hybrid approach is the right tradeoff here:

  • global snapshot for normal list refresh
  • targeted fetches for eventual DB correctness on locally-stale trades

I also checked the updated selection logic for targeted candidates and the startup/scheduler wiring, and I do not see a new blocker in the revised design.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/util/order_utils/relay_order_db_reconcile.rs (1)

23-31: 💤 Low value

Consider Ok(()) removal — function is infallible.

reconcile_terminal_order_statuses_from_relay always returns Ok(()) because reconcile_one_order_if_terminal is ()-typed (errors are logged internally). The Result<()> signature is misleading at call sites: callers ?-propagate something that can't fail. Either make this ()-returning or have the inner function propagate errors meaningfully (e.g., aggregate and surface counts of failures).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/util/order_utils/relay_order_db_reconcile.rs` around lines 23 - 31, The
function reconcile_terminal_order_statuses_from_relay currently returns
Result<()> but never fails because reconcile_one_order_if_terminal returns ()
and logs errors internally; change reconcile_terminal_order_statuses_from_relay
to be infallible by updating its signature to return () (remove Result), delete
the final Ok(()) and keep the for-loop calling reconcile_one_order_if_terminal,
and then update any callers that currently use ? to stop propagating a Result
(or handle accordingly). If you prefer surfacing failures instead, alternatively
make reconcile_one_order_if_terminal return Result<()> and have
reconcile_terminal_order_statuses_from_relay collect/propagate errors (e.g.,
aggregate failure count) so the Result return becomes meaningful.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/util/order_utils/relay_order_db_reconcile.rs`:
- Around line 23-31: The function reconcile_terminal_order_statuses_from_relay
currently returns Result<()> but never fails because
reconcile_one_order_if_terminal returns () and logs errors internally; change
reconcile_terminal_order_statuses_from_relay to be infallible by updating its
signature to return () (remove Result), delete the final Ok(()) and keep the
for-loop calling reconcile_one_order_if_terminal, and then update any callers
that currently use ? to stop propagating a Result (or handle accordingly). If
you prefer surfacing failures instead, alternatively make
reconcile_one_order_if_terminal return Result<()> and have
reconcile_terminal_order_statuses_from_relay collect/propagate errors (e.g.,
aggregate failure count) so the Result return becomes meaningful.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 5e215d0e-c34c-45ca-8bf6-94ff5849a06f

📥 Commits

Reviewing files that changed from the base of the PR and between 1a585ab and 860f247.

📒 Files selected for processing (6)
  • src/main.rs
  • src/models.rs
  • src/util/order_utils/fetch_scheduler.rs
  • src/util/order_utils/helper.rs
  • src/util/order_utils/mod.rs
  • src/util/order_utils/relay_order_db_reconcile.rs
🚧 Files skipped from review as they are similar to previous changes (2)
  • src/util/order_utils/mod.rs
  • src/util/order_utils/fetch_scheduler.rs

@arkanoider arkanoider merged commit c496f0c into main May 11, 2026
11 checks passed
@arkanoider arkanoider deleted the feat/relay-order-db-reconcile branch May 11, 2026 20:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant