Skip to content

Conversation

@zhurs
Copy link

@zhurs zhurs commented Aug 12, 2025

Details are in issue #2370

Here are results without and with indexes:

explain (analyze on, buffers on)
select MAX(id) from transactions where tenant_id = 't2';
/*
 Result  (cost=1103.95..1103.96 rows=1 width=8) (actual time=14.476..14.478 rows=1 loops=1)
  Buffers: shared hit=1636
  InitPlan 1 (returns $0)
    ->  Limit  (cost=0.29..1103.95 rows=1 width=8) (actual time=14.469..14.471 rows=1 loops=1)
          Buffers: shared hit=1636
          ->  Index Scan Backward using pk_transaction on transactions  (cost=0.29..5518.59 rows=5 width=8) (actual time=14.467..14.467 rows=1 loops=1)
                Index Cond: (id IS NOT NULL)
                Filter: ((tenant_id)::text = 't2'::text)
                Rows Removed by Filter: 110298
                Buffers: shared hit=1636
Planning Time: 0.126 ms
Execution Time: 14.507 ms
 */

CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_transactions_tenant_id ON transactions (tenant_id, id);

explain (analyze on, buffers on)
select MAX(id) from transactions where tenant_id = 't2';

/*
 Result  (cost=4.39..4.40 rows=1 width=8) (actual time=0.053..0.054 rows=1 loops=1)
  Buffers: shared hit=1 read=3
  InitPlan 1 (returns $0)
    ->  Limit  (cost=0.42..4.39 rows=1 width=8) (actual time=0.050..0.050 rows=1 loops=1)
          Buffers: shared hit=1 read=3
          ->  Index Only Scan Backward using idx_transactions_tenant_id on transactions  (cost=0.42..20.29 rows=5 width=8) (actual time=0.048..0.049 rows=1 loops=1)
                Index Cond: ((tenant_id = 't2'::text) AND (id IS NOT NULL))
                Heap Fetches: 0
                Buffers: shared hit=1 read=3
Planning:
  Buffers: shared hit=19 read=1
Planning Time: 0.405 ms
Execution Time: 0.077 ms
 */
explain (analyze on, buffers on)
select version from schema_definitions where tenant_id = 't12' order by version DESC limit 1;
/*
 Limit  (cost=20.65..20.65 rows=1 width=21) (actual time=0.134..0.135 rows=1 loops=1)
  Buffers: shared hit=63
  ->  Sort  (cost=20.65..20.67 rows=10 width=21) (actual time=0.132..0.133 rows=1 loops=1)
        Sort Key: version DESC
        Sort Method: top-N heapsort  Memory: 25kB
        Buffers: shared hit=63
        ->  Index Only Scan using pk_schema_definition on schema_definitions  (cost=0.43..20.60 rows=10 width=21) (actual time=0.062..0.084 rows=150 loops=1)
              Index Cond: (tenant_id = 't12'::text)
              Heap Fetches: 0
              Buffers: shared hit=63
Planning Time: 0.159 ms
Execution Time: 0.198 ms
 */

CREATE INDEX idx_schema_tenant_version ON schema_definitions (tenant_id, version);

explain (analyze on, buffers on)
select version from schema_definitions where tenant_id = 't12' order by version DESC limit 1;
/*
 Limit  (cost=0.43..0.68 rows=1 width=21) (actual time=0.048..0.049 rows=1 loops=1)
  Buffers: shared hit=4
  ->  Index Only Scan Backward using idx_schema_tenant_version on schema_definitions  (cost=0.43..4.73 rows=17 width=21) (actual time=0.046..0.046 rows=1 loops=1)
        Index Cond: (tenant_id = 't12'::text)
        Heap Fetches: 0
        Buffers: shared hit=4
Planning Time: 0.227 ms
Execution Time: 0.074 ms
 */

Summary by CodeRabbit

  • Chores
    • Database migration improves query performance in multi-tenant environments, speeding transaction and schema-related operations. Designed for safe, zero-downtime rollout with rollback support. Users should see faster page loads and smoother navigation under high tenant loads. No configuration changes or action required.

@coderabbitai
Copy link

coderabbitai bot commented Aug 12, 2025

Walkthrough

Adds a Goose SQL migration that creates two concurrent multi-tenant composite indexes and drops an older index; the down migration reverses these changes. Migration uses NO TRANSACTION and IF [NOT] EXISTS/IF EXISTS clauses.

Changes

Cohort / File(s) Summary
Postgres migration
internal/storage/postgres/migrations/20250912101325_add_multi_tenant_indexes.sql
Adds a NO TRANSACTION Goose migration: Up creates idx_transactions_tenant_id on transactions(tenant_id, id) and idx_schema_tenant_version on schema_definitions(tenant_id, version) using CREATE INDEX CONCURRENTLY IF NOT EXISTS; drops idx_schema_version with DROP INDEX CONCURRENTLY IF EXISTS. Down recreates idx_schema_version and drops the two new indexes concurrently if they exist.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related issues

Poem

I’m a rabbit in the code-lined glen,
I plant two indexes, then hop again.
Tenants find their rows with grace,
Versions fall into place.
CONCURRENTLY I dig, no fright—queries nap light. 🐇✨

✨ Finishing Touches
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (3)
internal/storage/postgres/migrations/20250912101325_add_multi_tenant_indexes.sql (3)

3-3: Index choice aligns with query pattern; consider DESC for clarity (optional) and verify duplicate functional indexes

The composite index (tenant_id, id) matches MAX(id) per tenant and supports backward scans. Adding DESC on id is optional (planner can scan backward regardless) but can better communicate intent.

Also, IF NOT EXISTS prevents name collisions but won’t avoid creating a duplicate functional index with a different name. Please verify no equivalent index already exists.

Apply this small clarity-oriented tweak if you prefer:

-CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_transactions_tenant_id ON transactions (tenant_id, id);
+CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_transactions_tenant_id ON transactions (tenant_id, id DESC);

To check for existing equivalent indexes by definition (not just by name), run these queries against your DB:

-- Transactions: list existing indexes that start with (tenant_id, id)
SELECT i.relname AS index_name, ix.indisvalid, pg_get_indexdef(i.oid) AS indexdef
FROM pg_class t
JOIN pg_index ix ON ix.indrelid = t.oid
JOIN pg_class i ON i.oid = ix.indexrelid
WHERE t.relname = 'transactions'
  AND ix.indisready
  AND pg_get_indexdef(i.oid) ILIKE '%(tenant_id, id%';

-- Ensure none of the returned indexdefs are functionally identical.

4-4: Good match for “latest schema version per tenant”; optional DESC for intent clarity and null handling check

This supports SELECT … WHERE tenant_id = $1 ORDER BY version DESC LIMIT 1 via index-only backward scan. Adding DESC on version is optional (planner can scan backward); consider it to better reflect the access pattern. Verify version is NOT NULL (or acceptable null ordering).

You may opt for:

-CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_schema_tenant_version ON schema_definitions (tenant_id, version);
+CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_schema_tenant_version ON schema_definitions (tenant_id, version DESC);

Confirm there isn’t already a functionally identical index:

SELECT i.relname AS index_name, ix.indisvalid, pg_get_indexdef(i.oid) AS indexdef
FROM pg_class t
JOIN pg_index ix ON ix.indrelid = t.oid
JOIN pg_class i ON i.oid = ix.indexrelid
WHERE t.relname = 'schema_definitions'
  AND ix.indisready
  AND pg_get_indexdef(i.oid) ILIKE '%(tenant_id, version%';

3-4: Consider schema-qualifying table names to avoid search_path surprises

If your goose runtime relies on search_path, unqualified names can target the wrong schema in some environments. Schema-qualify if your project standards require it (e.g., public.transactions). If search_path is pinned in migrations, you can ignore this.

Please confirm whether migrations enforce a fixed search_path. If not, consider qualifying the table names in this migration and future ones.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 8accccd and 3810bf0.

📒 Files selected for processing (1)
  • internal/storage/postgres/migrations/20250912101325_add_multi_tenant_indexes.sql (1 hunks)
🔇 Additional comments (2)
internal/storage/postgres/migrations/20250912101325_add_multi_tenant_indexes.sql (2)

1-1: Correct use of NO TRANSACTION for concurrent index ops

Required for CREATE/DROP INDEX CONCURRENTLY. This avoids transaction-block errors during migration.


6-8: Down migration is correct and safe for concurrent environments

DROP INDEX CONCURRENTLY with IF EXISTS matches the Up semantics and respects NO TRANSACTION. Looks good.

@tolgaozen
Copy link
Member

Hello @zhurs , thank you for your contribution. Could you please remove the following index in the same migration file, as it will no longer be needed? The index is:

CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_schema_version ON schema_definitions (version);
Also, could you share the test result after removing it?

@zhurs
Copy link
Author

zhurs commented Aug 12, 2025

Yes, sure.
I intentionally placed the CREATE statement at the start of the Down section to ensure that there is always at least one index on version.

The query plan after removing the index idx_schema_version:

Limit  (cost=0.43..0.68 rows=1 width=21) (actual time=0.061..0.062 rows=1 loops=1)
  Buffers: shared hit=4
  ->  Index Only Scan Backward using idx_schema_tenant_version on schema_definitions  (cost=0.43..4.73 rows=17 width=21) (actual time=0.060..0.060 rows=1 loops=1)
        Index Cond: (tenant_id = 't12'::text)
        Heap Fetches: 0
        Buffers: shared hit=4
Planning Time: 0.165 ms
Execution Time: 0.086 ms

I’ve also checked that all queries include the predicate tenant_id=?, so the new index should cover all use cases.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
internal/storage/postgres/migrations/20250912101325_add_multi_tenant_indexes.sql (1)

2-5: Optional: embed rationale as SQL comments for future maintainers.

Short inline comments help future readers understand why these exact indexes exist and why the legacy one was dropped.

Apply this diff to add succinct rationale:

 -- +goose Up
+-- Rationale: Speed up per-tenant queries
+-- - transactions: allow index-only backward scan for MAX(id) WHERE tenant_id=?
+-- - schema_definitions: allow index-only backward scan for latest version per tenant
 CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_transactions_tenant_id ON transactions (tenant_id, id);
 CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_schema_tenant_version ON schema_definitions (tenant_id, version);
-DROP INDEX CONCURRENTLY IF EXISTS idx_schema_version;
+-- Drop legacy version-only index; composite (tenant_id, version) covers call-sites
+DROP INDEX CONCURRENTLY IF EXISTS idx_schema_version;
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 3810bf0 and 8610910.

📒 Files selected for processing (1)
  • internal/storage/postgres/migrations/20250912101325_add_multi_tenant_indexes.sql (1 hunks)
🔇 Additional comments (5)
internal/storage/postgres/migrations/20250912101325_add_multi_tenant_indexes.sql (5)

1-1: Correct use of NO TRANSACTION for concurrent index ops.

CREATE/DROP INDEX CONCURRENTLY cannot run inside a transaction; the goose directive ensures both Up/Down run outside a txn. Good.


3-4: Composite indexes align with the workload; plan-friendly for backward scans.

  • (tenant_id, id) supports MAX(id) per-tenant via backward index scan without heap fetches.
  • (tenant_id, version) supports ORDER BY version DESC LIMIT 1 per-tenant via backward index scan.
    No change requested.

8-10: Down migration order is safe and considerate.

Recreating idx_schema_version before dropping the new composite indexes avoids a performance cliff during rollback. Good use of CONCURRENTLY + IF [NOT] EXISTS/IF EXISTS.


5-5: No version-only schema_definitions queries detected
A repository-wide search surfaced only migration files touching idx_schema_version. No application code or raw SQL strings query schema_definitions by version without also filtering on tenant_id. Dropping the old single-column index is safe.


1-10: Postgres CONCURRENTLY + IF NOT EXISTS compatibility: verified

  • internal/storage/postgres/utils/version.go enforces a minimum supported Postgres version of 13.8.
  • CI tests default to Postgres 14 (gc_test.go) and docs mention Postgres 15.
  • Existing migrations already use CREATE/DROP INDEX CONCURRENTLY IF [NOT] EXISTS without issues.

No changes required.

@tolgaozen
Copy link
Member

Great, thank you! I’m merging it now, and it will be live in the next release

@tolgaozen tolgaozen merged commit 4e4a12d into Permify:master Aug 12, 2025
7 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants