Skip to content

Add database performance indexes with comprehensive testing#618

Closed
vzwjustin wants to merge 1 commit into
coleam00:mainfrom
vzwjustin:main
Closed

Add database performance indexes with comprehensive testing#618
vzwjustin wants to merge 1 commit into
coleam00:mainfrom
vzwjustin:main

Conversation

@vzwjustin

Copy link
Copy Markdown

✅ Database Migration Implementation:

  • Added 2 optimized performance indexes for frequently queried fields
  • idx_archon_tasks_project_status: Tasks by project + status
  • idx_archon_crawled_pages_source_chunk: Pages by source + chunk (covers source-only queries)
  • Dropped redundant idx_archon_crawled_pages_source_id index to reduce overhead

✅ Production Safety Features:

  • Uses CONCURRENTLY for zero-downtime deployment and index removal
  • Uses IF NOT EXISTS/IF EXISTS for proper duplicate handling
  • Only adds missing indexes, removes redundant ones

✅ Comprehensive Testing Completed:

  • Set up local Supabase test environment
  • Created full Archon schema in test database
  • Successfully executed migration script
  • Verified all indexes created correctly
  • Confirmed Archon server starts and runs with new indexes
  • Validated database connectivity and service initialization

Performance improvements ready for production deployment.

✅ Database Migration Implementation:
- Added 2 optimized performance indexes for frequently queried fields
- idx_archon_tasks_project_status: Tasks by project + status
- idx_archon_crawled_pages_source_chunk: Pages by source + chunk (covers source-only queries)
- Dropped redundant idx_archon_crawled_pages_source_id index to reduce overhead

✅ Production Safety Features:
- Uses CONCURRENTLY for zero-downtime deployment and index removal
- Uses IF NOT EXISTS/IF EXISTS for proper duplicate handling
- Only adds missing indexes, removes redundant ones

✅ Comprehensive Testing Completed:
- Set up local Supabase test environment
- Created full Archon schema in test database
- Successfully executed migration script
- Verified all indexes created correctly
- Confirmed Archon server starts and runs with new indexes
- Validated database connectivity and service initialization

Performance improvements ready for production deployment.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 7, 2025

Copy link
Copy Markdown

Warning

Rate limit exceeded

@spotty118 has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 6 minutes and 27 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 012d2c5 and 906fd6f.

📒 Files selected for processing (1)
  • migration/add_performance_indexes.sql (1 hunks)
✨ Finishing Touches
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai

coderabbitai Bot commented Sep 7, 2025

Copy link
Copy Markdown
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

1 similar comment
@coderabbitai

coderabbitai Bot commented Sep 7, 2025

Copy link
Copy Markdown
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@vzwjustin

Copy link
Copy Markdown
Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Sep 7, 2025

Copy link
Copy Markdown
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@leex279

leex279 commented Sep 8, 2025

Copy link
Copy Markdown
Collaborator

I am no expert on this, but the whole indexing stuff increases the storage by a lot (according to claude by 3x). For production usecases its fine, but for testing and a lot of people running it locally with supabase cloud, this will lead into running into the storage limit for the free tier. so maybe we should move the whole indexing into the new migration sql file you created here and the user can decide themself to use it or not.

@coleam00 what do you think? Am I wrong with my assumption?

@Wirasm

Wirasm commented Sep 12, 2025

Copy link
Copy Markdown
Collaborator

I will Close this based on @leex279 Comment

Thank you for the contribution @spotty118

@Wirasm Wirasm closed this Sep 12, 2025
POWERFULMOVES added a commit to POWERFULMOVES/PMOVES-Archon that referenced this pull request Feb 12, 2026
…ion (coleam00#618)

* docs(monitoring): Add comprehensive monitoring integration documentation

This adds centralized documentation for the production monitoring stack
including:
- Prometheus metrics collection (port 9090)
- Grafana dashboards (port 3002)
- Loki log aggregation (port 3100)
- Promtail container log collection
- Blackbox exporter (port 9115)
- Startup commands and health checks
- Integration with PMOVES services via labeled metrics

Related docs:
- pmoves/docs/DOCKER_COMPOSE_ENVIRONMENT_LOADING.md
- pmoves/docs/DOCKER_COMPOSE_NETWORKING_GUIDE.md

Refs: #monitoring

* docs: Add universal agent context patterns and BoTZ skills marketplace

Integrate "digital production controls on analog signals" pattern for all
PMOVES.AI agents (Claude Code CLI, Agent Zero, Archon, BoTZ, custom).

**Agent Context Patterns (Tier System):**
- Tier 1: Always load (main PMOVES.AI context)
- Tier 2: On-demand load (major subsystems)
- Tier 3: Conditional load (integration workspaces)
- Tier 4: Explicit load only (nested components)

**New Documentation:**
- pmoves/docs/AGENT_CONTEXT_PATTERNS.md - Universal context hierarchy
- pmoves/docs/BOTZ_SKILLS_MARKETPLACE.md - 50+ skills catalog
- pmoves/docs/CLAUDE_CONTEXT_AUDIT.md - 51 worktrees, 31 CLAUDE.md files

**Updates:**
- .claude/CLAUDE.md - Added context loading strategy section
- docs/PMOVES-submodule-list.md - PMOVES-supabase entry added

**Key Pattern:** Context flows from top-level authority down (main → submodule → component),
never sideways or up. This prevents conflicts while maintaining flexibility.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Codex Agent <codex-agent@example.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
coleam00 pushed a commit that referenced this pull request Apr 7, 2026
Unlocks: hook lifecycle events (#618), startup() pre-warm,
rate_limit_event handling, is_error bug fix on error results.

All existing features (effort, thinking, maxBudgetUsd, total_cost_usd,
task events) were already available at resolved v0.2.74 but this ensures
we get the latest fixes and hook_started/progress/response types.
Tyone88 pushed a commit to Tyone88/Archon that referenced this pull request Apr 16, 2026
Unlocks: hook lifecycle events (coleam00#618), startup() pre-warm,
rate_limit_event handling, is_error bug fix on error results.

All existing features (effort, thinking, maxBudgetUsd, total_cost_usd,
task events) were already available at resolved v0.2.74 but this ensures
we get the latest fixes and hook_started/progress/response types.
joaobmonteiro pushed a commit to joaobmonteiro/Archon that referenced this pull request Apr 26, 2026
Unlocks: hook lifecycle events (coleam00#618), startup() pre-warm,
rate_limit_event handling, is_error bug fix on error results.

All existing features (effort, thinking, maxBudgetUsd, total_cost_usd,
task events) were already available at resolved v0.2.74 but this ensures
we get the latest fixes and hook_started/progress/response types.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants