-
-
Notifications
You must be signed in to change notification settings - Fork 278
perf: added indexes for efficient queries in a multi-tenant setup, issue #2370 #2371
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf: added indexes for efficient queries in a multi-tenant setup, issue #2370 #2371
Conversation
… efficient queries in a multi-tenant setup, issue Permify#2370
WalkthroughAdds a Goose SQL migration that creates two concurrent multi-tenant composite indexes and drops an older index; the down migration reverses these changes. Migration uses NO TRANSACTION and IF [NOT] EXISTS/IF EXISTS clauses. Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~3 minutes Possibly related issues
Poem
✨ Finishing Touches🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
🧹 Nitpick comments (3)
internal/storage/postgres/migrations/20250912101325_add_multi_tenant_indexes.sql (3)
3-3: Index choice aligns with query pattern; consider DESC for clarity (optional) and verify duplicate functional indexesThe composite index (tenant_id, id) matches MAX(id) per tenant and supports backward scans. Adding DESC on id is optional (planner can scan backward regardless) but can better communicate intent.
Also, IF NOT EXISTS prevents name collisions but won’t avoid creating a duplicate functional index with a different name. Please verify no equivalent index already exists.
Apply this small clarity-oriented tweak if you prefer:
-CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_transactions_tenant_id ON transactions (tenant_id, id); +CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_transactions_tenant_id ON transactions (tenant_id, id DESC);To check for existing equivalent indexes by definition (not just by name), run these queries against your DB:
-- Transactions: list existing indexes that start with (tenant_id, id) SELECT i.relname AS index_name, ix.indisvalid, pg_get_indexdef(i.oid) AS indexdef FROM pg_class t JOIN pg_index ix ON ix.indrelid = t.oid JOIN pg_class i ON i.oid = ix.indexrelid WHERE t.relname = 'transactions' AND ix.indisready AND pg_get_indexdef(i.oid) ILIKE '%(tenant_id, id%'; -- Ensure none of the returned indexdefs are functionally identical.
4-4: Good match for “latest schema version per tenant”; optional DESC for intent clarity and null handling checkThis supports SELECT … WHERE tenant_id = $1 ORDER BY version DESC LIMIT 1 via index-only backward scan. Adding DESC on version is optional (planner can scan backward); consider it to better reflect the access pattern. Verify version is NOT NULL (or acceptable null ordering).
You may opt for:
-CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_schema_tenant_version ON schema_definitions (tenant_id, version); +CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_schema_tenant_version ON schema_definitions (tenant_id, version DESC);Confirm there isn’t already a functionally identical index:
SELECT i.relname AS index_name, ix.indisvalid, pg_get_indexdef(i.oid) AS indexdef FROM pg_class t JOIN pg_index ix ON ix.indrelid = t.oid JOIN pg_class i ON i.oid = ix.indexrelid WHERE t.relname = 'schema_definitions' AND ix.indisready AND pg_get_indexdef(i.oid) ILIKE '%(tenant_id, version%';
3-4: Consider schema-qualifying table names to avoid search_path surprisesIf your goose runtime relies on search_path, unqualified names can target the wrong schema in some environments. Schema-qualify if your project standards require it (e.g., public.transactions). If search_path is pinned in migrations, you can ignore this.
Please confirm whether migrations enforce a fixed search_path. If not, consider qualifying the table names in this migration and future ones.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
internal/storage/postgres/migrations/20250912101325_add_multi_tenant_indexes.sql(1 hunks)
🔇 Additional comments (2)
internal/storage/postgres/migrations/20250912101325_add_multi_tenant_indexes.sql (2)
1-1: Correct use of NO TRANSACTION for concurrent index opsRequired for CREATE/DROP INDEX CONCURRENTLY. This avoids transaction-block errors during migration.
6-8: Down migration is correct and safe for concurrent environmentsDROP INDEX CONCURRENTLY with IF EXISTS matches the Up semantics and respects NO TRANSACTION. Looks good.
|
Hello @zhurs , thank you for your contribution. Could you please remove the following index in the same migration file, as it will no longer be needed? The index is:
|
|
Yes, sure. The query plan after removing the index I’ve also checked that all queries include the predicate |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
🧹 Nitpick comments (1)
internal/storage/postgres/migrations/20250912101325_add_multi_tenant_indexes.sql (1)
2-5: Optional: embed rationale as SQL comments for future maintainers.Short inline comments help future readers understand why these exact indexes exist and why the legacy one was dropped.
Apply this diff to add succinct rationale:
-- +goose Up +-- Rationale: Speed up per-tenant queries +-- - transactions: allow index-only backward scan for MAX(id) WHERE tenant_id=? +-- - schema_definitions: allow index-only backward scan for latest version per tenant CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_transactions_tenant_id ON transactions (tenant_id, id); CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_schema_tenant_version ON schema_definitions (tenant_id, version); -DROP INDEX CONCURRENTLY IF EXISTS idx_schema_version; +-- Drop legacy version-only index; composite (tenant_id, version) covers call-sites +DROP INDEX CONCURRENTLY IF EXISTS idx_schema_version;
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
internal/storage/postgres/migrations/20250912101325_add_multi_tenant_indexes.sql(1 hunks)
🔇 Additional comments (5)
internal/storage/postgres/migrations/20250912101325_add_multi_tenant_indexes.sql (5)
1-1: Correct use of NO TRANSACTION for concurrent index ops.CREATE/DROP INDEX CONCURRENTLY cannot run inside a transaction; the goose directive ensures both Up/Down run outside a txn. Good.
3-4: Composite indexes align with the workload; plan-friendly for backward scans.
- (tenant_id, id) supports MAX(id) per-tenant via backward index scan without heap fetches.
- (tenant_id, version) supports ORDER BY version DESC LIMIT 1 per-tenant via backward index scan.
No change requested.
8-10: Down migration order is safe and considerate.Recreating idx_schema_version before dropping the new composite indexes avoids a performance cliff during rollback. Good use of CONCURRENTLY + IF [NOT] EXISTS/IF EXISTS.
5-5: No version-only schema_definitions queries detected
A repository-wide search surfaced only migration files touchingidx_schema_version. No application code or raw SQL strings queryschema_definitionsbyversionwithout also filtering ontenant_id. Dropping the old single-column index is safe.
1-10: Postgres CONCURRENTLY + IF NOT EXISTS compatibility: verified
- internal/storage/postgres/utils/version.go enforces a minimum supported Postgres version of 13.8.
- CI tests default to Postgres 14 (gc_test.go) and docs mention Postgres 15.
- Existing migrations already use
CREATE/DROP INDEX CONCURRENTLY IF [NOT] EXISTSwithout issues.No changes required.
|
Great, thank you! I’m merging it now, and it will be live in the next release |
Details are in issue #2370
Here are results without and with indexes:
Summary by CodeRabbit