upstream of 0.22.5 changes#1429
Merged
Merged
Conversation
ozgb
approved these changes
Apr 27, 2026
First query for coarse bounds of tx, tx_out and ma_tx_out tables and use them in the 'business' queries to reduce size of tables before joins are made. Signed-off-by: Giles Cope <gilescope@gmail.com>
Signed-off-by: Giles Cope <gilescope@gmail.com>
Convert the runtime query_as form back to the compile-time macro form for all queries in cnight_observation.rs. This restores static query verification against the db-sync schema via the .sqlx prepared cache. Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com> Signed-off-by: Giles Cope <gilescope@gmail.com>
…y_hash Needed because cherry-pick landed in a branch whose .sqlx prepared cache does not include this query (cached key changed after trailing-whitespace normalisation). Using the runtime form avoids the compile-time DB check entirely and matches the pattern used elsewhere in #934. Signed-off-by: Giles Cope <gilescope@gmail.com>
Signed-off-by: Giles Cope <gilescope@gmail.com>
Signed-off-by: Giles Cope <gilescope@gmail.com>
…y args Replaces the seven repeated arguments (start, end, limit, offset, low_bound, high_bound + per-call locals) on the four cNight observation query helpers with a single `PagedQuery<'a>` struct. Squashed from PR #1365 fixup commits b9497e8 + b146dc2 + b1ffce6 (originally landed as "fix: cargo clippy" / "Apply suggestion"). Also drops the now-unused `QueryBounds` import in the data_source module. Signed-off-by: Giles Cope <gilescope@gmail.com>
Signed-off-by: Giles Cope <gilescope@gmail.com>
Signed-off-by: Tomasz Bartos <tomasz.bartos@shielded.io> Signed-off-by: Giles Cope <gilescope@gmail.com>
ef9460d to
763912a
Compare
Contributor
Author
|
fixed unverified commits. |
cosmir17
approved these changes
Apr 27, 2026
15 tasks
gilescope
added a commit
that referenced
this pull request
Apr 27, 2026
upstream of 0.22.5 changes (#1429)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Backports the four cNIGHT db-sync observation perf commits from PR #1365 (which landed on
release/node-0.22.5) ontomain. The companion #934 (multi-asset cache) is already onmainas17ad5e8b, so this PR contains only the perf delta.Net effect: significantly lower postgres CPU / query latency during cNIGHT observation, no behavioural change.
Commits
tx/tx_out/ma_tx_outid bounds for the block-range window, then constrains the four observation queries (registration / deregistration / asset-create / asset-spend) by primary-key range. Postgres can prune rows before the expensive joins. IntroducesPagedQueryandQueryBounds.tx_inindexes as well — adds thetx.idbounding to thetx_in-keyed queries (spend / deregistration) so they benefit from the same pruning.sqlx::query_as!macro for cNight observation queries — restores compile-time query verification against the db-sync schema via the.sqlxprepared cache.sqlx::query_as!with runtime form inget_block_by_hash— needed because the cherry-pick landed in a branch whose.sqlxcache key changed after trailing-whitespace normalisation; runtime form avoids the compile-time DB check.log::warn!("Bounds:…")from the perf commit down todebug!.Supersedes / overlaps
feat/fiwcakym-upstream-node-0.22.5-rc.2— Upstream node-0.22.5-rc.2 changes, 516 files, automated bulk upstream). Upstream node-0.22.5-rc.2 changes #1377 contains the same four perf commits as a subset. This PR is the targeted minimal backport so the perf wins can land independently of the bigger upstream sweep — once merged, the corresponding commits in Upstream node-0.22.5-rc.2 changes #1377 will become no-ops on rebase.ozgb-reduce-cnight-query-capacity— feat: reduce data queried by 16x). Both touchprimitives/mainchain-follower/src/data_source/cnight_observation.rs. Whichever lands second will need a small rebase; feat: reduce data queried by 16x #1367 changes thetx_capacity * 64overestimate factor, which is on a line near (but not the same as) the bounds-query block introduced here.Resolution note
The cherry-pick of the Increase performance commit conflicted on
data_source/cnight_observation.rs:mainhas thestart_sub_query_timermetrics infrastructure (added afterrelease/node-0.22.5branched), so the new bounds queries were wrapped in matchingcnight_get_low_bounds/cnight_get_high_boundstimers for consistency with the rest of that function.🗹 TODO before merging
📌 Submission Checklist
git commit -s) for the DCOchanges/changed/cnight-query-coarse-bounds.md)🧪 Testing Evidence
Observed reduced db-sync query times on a live cNIGHT observation workload (originally measured against
release/node-0.22.5on PR #1365).🔱 Fork Strategy
Links
17ad5e8b)#1429