Add cache for multi asset id to avoid excessive joins#934
Merged
Conversation
Contributor
gilescope
reviewed
Mar 18, 2026
gilescope
approved these changes
Mar 18, 2026
LGLO
approved these changes
Mar 18, 2026
LGLO
left a comment
Contributor
There was a problem hiding this comment.
I've checked at some random range of blocks and the speed up was around 10% but still I think it is worth it.
Added complexity is limited and it pays off in multiple queries.
gilescope
pushed a commit
that referenced
this pull request
Apr 8, 2026
gilescope
added a commit
that referenced
this pull request
Apr 20, 2026
* Add cache for multi asset id to avoid excessive joins * change file * fix npm audit * Apply suggestion from @gilescope --------- Co-authored-by: Squirrel <giles.cope@shielded.io> Signed-off-by: Giles Cope <gilescope@gmail.com>
gilescope
added a commit
that referenced
this pull request
Apr 20, 2026
…y_hash Needed because cherry-pick landed in a branch whose .sqlx prepared cache does not include this query (cached key changed after trailing-whitespace normalisation). Using the runtime form avoids the compile-time DB check entirely and matches the pattern used elsewhere in #934. Signed-off-by: Giles Cope <gilescope@gmail.com>
This was referenced Apr 20, 2026
Closed
m2ux
added a commit
that referenced
this pull request
Apr 23, 2026
…934) Signed-off-by: Mike Clay <mike.clay@shielded.io>
m2ux
added a commit
that referenced
this pull request
Apr 23, 2026
…934) Signed-off-by: Mike Clay <mike.clay@shielded.io>
gilescope
added a commit
that referenced
this pull request
Apr 27, 2026
…y_hash Needed because cherry-pick landed in a branch whose .sqlx prepared cache does not include this query (cached key changed after trailing-whitespace normalisation). Using the runtime form avoids the compile-time DB check entirely and matches the pattern used elsewhere in #934. Signed-off-by: Giles Cope <gilescope@gmail.com>
github-merge-queue Bot
pushed a commit
that referenced
this pull request
Apr 27, 2026
* Increase performance of cNight db-sync queries. First query for coarse bounds of tx, tx_out and ma_tx_out tables and use them in the 'business' queries to reduce size of tables before joins are made. Signed-off-by: Giles Cope <gilescope@gmail.com> * Add tx_in indexes as well Signed-off-by: Giles Cope <gilescope@gmail.com> * refactor: use sqlx::query_as! macro for cNight observation queries Convert the runtime query_as form back to the compile-time macro form for all queries in cnight_observation.rs. This restores static query verification against the db-sync schema via the .sqlx prepared cache. Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com> Signed-off-by: Giles Cope <gilescope@gmail.com> * chore: replace sqlx::query_as! macro with runtime form in get_block_by_hash Needed because cherry-pick landed in a branch whose .sqlx prepared cache does not include this query (cached key changed after trailing-whitespace normalisation). Using the runtime form avoids the compile-time DB check entirely and matches the pattern used elsewhere in #934. Signed-off-by: Giles Cope <gilescope@gmail.com> * feat: add changefile Signed-off-by: Giles Cope <gilescope@gmail.com> * fix: should be debug not warn Signed-off-by: Giles Cope <gilescope@gmail.com> * refactor: introduce PagedQuery to consolidate cNight observation query args Replaces the seven repeated arguments (start, end, limit, offset, low_bound, high_bound + per-call locals) on the four cNight observation query helpers with a single `PagedQuery<'a>` struct. Squashed from PR #1365 fixup commits b9497e8 + b146dc2 + b1ffce6 (originally landed as "fix: cargo clippy" / "Apply suggestion"). Also drops the now-unused `QueryBounds` import in the data_source module. Signed-off-by: Giles Cope <gilescope@gmail.com> * fix: clippy needless borrow in get_block_by_hash bind Signed-off-by: Giles Cope <gilescope@gmail.com> * fix: fixed compilation for try-runtime feature (#1427) Signed-off-by: Tomasz Bartos <tomasz.bartos@shielded.io> Signed-off-by: Giles Cope <gilescope@gmail.com> --------- Signed-off-by: Giles Cope <gilescope@gmail.com> Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com> Signed-off-by: Tomasz Bartos <tomasz.bartos@shielded.io> Co-authored-by: Lech Głowiak <lech.glowiak@iohk.io> Co-authored-by: Lech Głowiak <lech.glowiak@shielded.io> Co-authored-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com> Co-authored-by: Klapeyron <11329616+Klapeyron@users.noreply.github.com>
15 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.








Overview
Cache multi_asset.id to avoid excessive joins:
Make the database do less work by pre-caching join data that is fixed. Perf testing has confirmed this improves the query times under load.
Adds an in-memory cache for
multi_asset.idlookups, replacing repeatedJOIN multi_assetin db-sync queries with a single cached lookup per (policy, name) pair. This eliminates themulti_assetjoin from registration, deregistration, asset create/spend, and candidate token queries, reducing query complexity and improving observation performance.🗹 TODO before merging
📌 Submission Checklist
🧪 Testing Evidence
Please describe any additional testing aside from CI:
🔱 Fork Strategy
Links