Skip to content

docs(benchmarks): capture 2026-05-15 remaining-benches results — postgres backend#78

Merged
jphein merged 1 commit into
mainfrom
docs/2026-05-15-remaining-benches
May 15, 2026
Merged

docs(benchmarks): capture 2026-05-15 remaining-benches results — postgres backend#78
jphein merged 1 commit into
mainfrom
docs/2026-05-15-remaining-benches

Conversation

@jphein

@jphein jphein commented May 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

Captures the 5 of 9 mempalace benchmark suites that hadn't been re-run after the postgres cutover. Result: 64 / 64 passed in 1h 24m.

Suite Status
`test_ingest_bench` passed
`test_knowledge_graph_bench` passed
`test_layers_bench` passed
`test_mcp_bench` passed
`test_memory_profile` passed
`test_chromadb_stress` skipped — chromadb-specific, we're on postgres

Already captured 2026-05-14: `test_search_bench`, `test_recall_threshold`, `test_palace_boost` (`docs/benchmarks/2026-05-14-search-bench-hybrid-cutover.json`).

Headlines from the JSON

  • Ingest at 100 files: 6.5 files/s, 15.3 drawers/s, peak RSS 389 MB (Δ 25.5 MB).
  • Knowledge graph stats: latency stable from 500 → 5000 scale points.
  • Layer-1 unbounded fetch: RSS growth sub-linear through 5000.
  • MCP tool-status repeated calls: no leak signal.
  • Search heap top allocators: consistent with prior chromadb-era numbers — no regression on the hot path.

Postgres backend (`disks:5433`, 273k-drawer `mempalace_2026_05_13` database) used as TEST_POSTGRES_DSN.

Test plan

  • All 64 tests pass at `--bench-scale=small`.
  • No FAILED/ERROR lines in run.log.
  • Result JSON validates as object with 7 top-level keys (ingest, chunking, kg, layers, mcp, memory_profile, heap).

🤖 Generated with Claude Code

…gres backend

5 of 9 mempalace benchmark suites that had not run after the postgres
cutover, re-run against ``disks:5433`` (273k drawers, mempalace_2026_05_13).

Suites: ``test_ingest_bench``, ``test_knowledge_graph_bench``,
``test_layers_bench``, ``test_mcp_bench``, ``test_memory_profile``.
Skipped: ``test_chromadb_stress`` (chromadb-specific, we're on postgres).
Already captured separately: ``test_search_bench``, ``test_recall_threshold``,
``test_palace_boost`` (2026-05-14).

Result: **64 / 64 passed in 1h 24m 25s** at ``--bench-scale=small``.
Headlines from the JSON:

* ingest 100-file batch: 6.5 files/s, 15.3 drawers/s, peak RSS 389 MB
  (Δ 25.5 MB)
* knowledge-graph stats latency stable from 500 → 5000 scale points
* layer-1 unbounded fetch RSS growth stays sub-linear through 5000
* MCP tool-status repeated-call RSS profile clean (no leak signal)
* heap top allocators on search consistent with prior chromadb-era
  numbers — no regression on the hot path

Probe results from a previous bench run (search/recall/boost) are at
``docs/benchmarks/2026-05-14-search-bench-hybrid-cutover.json``.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 15, 2026 07:07

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@jphein jphein merged commit 6ecefa7 into main May 15, 2026
3 of 4 checks passed
@jphein jphein deleted the docs/2026-05-15-remaining-benches branch May 15, 2026 07:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants