Skip to content

improve execution data related query performance#599

Merged
pk910 merged 2 commits intomasterfrom
pk910/address-page-performance
Mar 5, 2026
Merged

improve execution data related query performance#599
pk910 merged 2 commits intomasterfrom
pk910/address-page-performance

Conversation

@pk910
Copy link
Copy Markdown
Member

@pk910 pk910 commented Mar 4, 2026

Summary

Fixes a PostgreSQL compatibility bug and optimizes address page database queries for accounts with large transaction counts.

Bug Fix

  • Fixed missing subquery aliases in GetElTransactionsByAccountIDCombined and GetElTokenTransfersByAccountIDCombined that caused subquery in FROM must have an alias errors on PostgreSQL (introduced in 62847ad for SQLite compatibility)

Performance Improvements

Measured via EXPLAIN ANALYZE against a production-scale database (14M transactions, 11.5M token transfers, 56.9M internal transactions):

Query Before After Improvement
Token transfer count (38k transfers) 22.8s 21ms 1,086x
Token transfer count (2.7M transfers) 724ms 135ms 5.4x
Internal tx count (14.5M internal txs) 780ms 83ms 9.4x
Transaction count (5M txs) 244ms 41ms 6x
Tab visibility: internal txs 21ms (full count) <0.1ms (EXISTS) 210x
Tab visibility: withdrawals CTE scan <0.1ms (EXISTS) instant

Changes

  • Count queries: Replaced UNION ALL count patterns with separate index-only scans per direction (from_id / to_id counted independently). Slightly overcounts self-referencing entries but is acceptable since counts are already capped at 100k.
  • Tab visibility checks: Added HasElTransactionsInternalByAccount and HasElWithdrawalsByAccountID using SELECT EXISTS(...) instead of running full count queries just to check if rows exist.
  • Withdrawal query: Replaced CTE-based count+data approach with separate count and data queries, consistent with other table patterns.
  • New indexes on el_token_transfers: Composite indexes (from_id, token_type) and (to_id, token_type) to enable index-only scans for token transfer count queries filtered by token type.
  • New indexes on el_transactions_internal: Single-column indexes (from_id) and (to_id) for efficient count and EXISTS queries. These are smaller than the existing composite indexes and reduce heap fetches from 100k+ to <200 for index-only scans.

All queries verified on both PostgreSQL and SQLite.

PostgreSQL tuning notes

Analysis revealed that shared_buffers = 128MB on a 37GB database causes severe cold-cache performance issues (e.g., index-only scans hitting disk for every heap fetch). Recommended:

  • Increase shared_buffers to at least 2-4GB
  • Ensure autovacuum runs frequently to maintain visibility maps (reduces heap fetches)
  • Set random_page_cost = 1.1 if using SSD storage

@pk910 pk910 merged commit 9f8d120 into master Mar 5, 2026
4 checks passed
@pk910 pk910 deleted the pk910/address-page-performance branch March 5, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants