Thanks for the detailed production perspective — it’s exactly the kind of feedback we hoped for from someone running Synapse on a real retrieval pipeline.

What you called out as working well — multiplicative composition with neutral defaults, per-axis toggles, batched LTP, and fire-and-forget logging — matches the design goals. We’re glad it lines up with how you’re already using time decay (#337) in Orient and that the axes can be reasoned about independently for other OODA phases.

Per-query overrides (search_memories(..., synapse_ltp_enabled=None) etc.) — we agree this is the main unlock for multi-profile integrations without mutating config.json on the hot path. Config-level control was the right baseline for single-profile and opt-in rollout; optional parameters (and eventually optional MCP tool args) are a natural next step. We’ll track this as a follow-up rather than blocking the current merge — your breakdown of Orient vs Evaluate vs Decide is a useful spec for that API.

Implementation notes — all fair:

Re-instantiating MempalaceConfig() on every search is simple but not ideal for batch workloads; we’ll look at caching or passing config from the caller.
SQLite per-call connect() trades churn for simplicity; a scoped connection/context for a single search transaction is worth exploring under load.
Gating or deferring VACUUM after cleanup makes sense at your scale; we can add thresholds or a separate maintenance path.
Filtering consolidation candidates by wing (or similar) is a good enhancement for domain-scoped archive cycles with soft-archive (#336).
Tests — we’ll add (or extend) a case where both LTP and tagging apply to the same hit so the multiplicative pipeline is exercised end-to-end, not only via fixed ltp_scores in calculate_synapse_score.

Thanks again for tying this to #335 / retrieval profiles — we’ll cross-link from the issue. Looking forward to hearing whether Phase 2 co-retrieval shows the astrophysics ↔ cryptography patterns you’re seeing in practice.

Glad the per-query overrides direction makes sense — tracking as follow-up rather than blocking merge is exactly right. The current config-level baseline is clean and gives you a stable surface to build on. The Orient/Evaluate/Decide breakdown is there whenever you want it; happy to help prototype the API shape if that's useful.

On the implementation notes: the SQLite and MempalaceConfig points are more "ran into this at scale, flagging for awareness" than blockers. The scoped connection idea is probably the highest-leverage change — under batch workloads the per-call churn adds up fast. The VACUUM and wing-scoped consolidation are longer-tail but good to have on the radar.

The Phase 2 co-retrieval question is what I'm most curious about honestly. We haven't run Phase 2 yet (waiting for Phase 1 to stabilize), but our KG already has some interesting astrophysics ↔ cryptography edges — entropy and information-theoretic concepts that show up in both domains independently and ended up linked during KG consolidation. The question is whether retrieval behavior reflects that, or whether those edges are just a KG artifact. My plan for a first concrete test: search "entropy" across all wings and see if Phase 2 association scores start pulling astrophysics and cryptography drawers together based on co-retrieval history rather than just content similarity. That would be a meaningful signal that Synapse is capturing something real about how these domains relate in practice.

Will report back once we've had some runtime with Phase 2. Looking forward to it.

@web3guru888 Thanks for the detailed review — all points addressed in the latest push.

Per-query overrides (your main ask):

search_memories() now accepts optional per-query parameters:

search_memories(
    query="gravitational wave detection",
    synapse_ltp_enabled=True,      # None = use config default
    synapse_tagging_enabled=False,
)

MCP mempalace_search exposes synapse_ltp_enabled and synapse_tagging_enabled as optional booleans. Your OODA phase profiles now work without config mutation:

Code-level fixes:

1. MempalaceConfig() call reduction — instantiated once per search_memories() call and reused throughout.

2. Connection pooling — SynapseDB.connection() context manager. get_ltp_scores_batch, get_association_scores_batch, and log_retrieval all accept an optional conn= parameter. The searcher opens one connection for the entire Synapse block:

with synapse_db.connection() as conn:
    ltp_scores = synapse_db.get_ltp_scores_batch(..., conn=conn)
    # score computation loop
    synapse_db.log_retrieval(..., conn=conn)

3. VACUUM gating — cleanup_old_logs only runs VACUUM when deleted rows > 1000. At your scale (~200 searches/day, 36K+ rows over 6 months), this avoids the expensive operation on routine status calls.

4. Wing filter on consolidation — get_consolidation_candidates(inactive_days=180, wing="astrophysics") now scopes results per domain.

Test addition:

LTP + tagging composition test added — verifies the multiplicative result when a drawer is both recently filed (high tagging) and frequently retrieved (high LTP). 35 tests total, all passing.

This is a thorough and complete response — thank you for turning all of this around in a single push.

The per-query parameter table is exactly right. The OODA phase mapping you derived matches what we're doing in practice:

One small observation on the implementation: the per-query booleans currently override tagging/LTP independently, but there's no way to override the half_life_days or ltp_weight at query time. For our Decide phase we'd want short half-life (high recency weight), but long half-life for Orient. This could be follow-up scope (config-level profiles are fine for now), just noting it as the natural next step.

Connection pooling: The conn= parameter threading is the right pattern. One thing to check: if log_retrieval raises inside the with synapse_db.connection() block, does the context manager commit or roll back? With SQLite autocommit off, a failed log write could silently swallow the connection. A try/except around the log step (non-fatal — log failure shouldn't abort retrieval) would harden this.

VACUUM gating at 1000 deleted rows: Correct threshold for our scale. At 200 searches/day this triggers roughly every 5 days, which is fine.

Wing filter on consolidation: The inactive_days=180, wing="astrophysics" scoping is exactly what we needed for domain-isolated maintenance cycles. This will let us run compaction passes per-domain without cross-contaminating consolidation state.

35 tests is solid for a scoring layer. Happy to +1 this when the connection error handling is addressed — either in this PR or noted as a follow-up issue. Great work overall.

@web3guru888 Good catch on the connection error handling — addressed in the latest push.

log_retrieval inside the synapse_db.connection() block is now wrapped in try/except. If the log write fails for any reason, it emits a warning and the connection proceeds to commit — LTP scores and search results are unaffected.

if cfg.synapse_log_retrievals and hit_drawer_ids:
try:
synapse_db.log_retrieval(hit_drawer_ids, query_hash, session_id, conn=conn)
except Exception as e:
logger.warning("Synapse log_retrieval failed (non-fatal): %s", e)
Added a test (test_log_retrieval_failure_does_not_break_search) that patches log_retrieval to raise RuntimeError and verifies the search still returns with synapse_enabled=True. 36 tests total, all passing.

The half_life_days / ltp_weight per-query override is noted as a follow-up — agreed it's the natural next step for full profile switching.

The try/except pattern is exactly right — warning on failure, non-fatal, search proceeds. The test (test_log_retrieval_failure_does_not_break_search patching to RuntimeError) is the kind of test that actually catches the integration contract, not just the happy path.

36 tests, all passing, and the full review checklist addressed. +1 from me — this is ready to merge.

half_life_days / ltp_weight per-query as the next step makes sense. Once you're ready to design that, happy to share how we're handling it at the config-profile level — might be worth aligning shapes before you build the override surface.

@web3guru888 Thanks for the +1 and the offer to collaborate on the profile-level design.

Here's my initial thinking for half_life_days / ltp_weight per-query support — rather than adding more individual arguments, define named profiles in config:

{
  "synapse_profiles": {
    "default": {
      "half_life_days": 90,
      "ltp_enabled": true,
      "ltp_window_days": 30,
      "ltp_max_boost": 2.0,
      "tagging_enabled": true,
      "tagging_window_hours": 24,
      "tagging_max_boost": 1.5,
      "association_enabled": true
    },
    "orient": {
      "half_life_days": 180,
      "ltp_enabled": true,
      "ltp_window_days": 60,
      "tagging_enabled": false
    },
    "decide": {
      "half_life_days": 30,
      "ltp_enabled": true,
      "ltp_window_days": 14,
      "tagging_enabled": true
    }
  }
}

API usage:

search_memories(query="...", synapse_profile="orient")
# per-query args still override profile values
search_memories(query="...", synapse_profile="orient", ltp_max_boost=1.5)

Override resolution order:

per-query arg  >  named profile  >  "default" profile  >  hardcoded defaults

Why profiles instead of more individual arguments:

• Prevents argument explosion — 6+ numeric params per call is unsustainable as axes grow
• OODA phases (Orient / Evaluate / Decide) and CDD-MemPalace workflows can switch with a single string instead of tuning each param
• Users and agents can define custom profiles freely without code changes
• Backward compatible — existing per-query booleans continue to work and take priority over profile values

Three design questions before I start building:

1. Config location — profiles inside config.json under synapse_profiles, or a separate synapse_profiles.json file?
2. Profile inheritance — should "orient" inherit from "default" and only override the diff, or should each profile be fully self-contained?
3. MCP schema — synapse_profile as a freeform string (users can define any name) or a validated enum of known profiles?

Would love your input on these before aligning the shapes.

The profile design is exactly right — named profiles eliminate the argument explosion problem cleanly. Our answers on the three questions:

1. Config location: with optional override file.

We'd put synapse_profiles in config.json — keeps config unified and means users don't have to manage two files for a working installation. One small addition worth considering: if a synapse_profiles.json exists in the palace root, merge it over the config entries. That lets power users keep their profile tuning in a separate file without changing the main config, and makes it easier to share profiles between projects. Fallback chain: per-query arg → per-session override → synapse_profiles.json → config.json → hardcoded defaults.

2. Profile inheritance: diff from default, with a --show-profile inspection command.

The inheritance approach is better for usability — a full spec for orient that duplicates 80% of default is noise that obscures what actually changes between profiles. Diff inheritance keeps definitions readable.

The one risk is "what did I actually inherit?" ambiguity. Mitigate with a resolve/inspect command:

mempalace synapse --show-profile orient
# prints the resolved (merged) config for that profile

That way users can always see the effective value without reading inheritance logic. We'd also recommend making the merge depth-1 (no profile inheriting from another custom profile) to keep the mental model simple.

3. MCP schema: freeform string with soft validation.

Enum validation would break custom profiles — the whole point is user-defined names. Freeform string with a runtime warning is the right call:

if profile_name not in config["synapse_profiles"]:
    logger.warning(f"synapse_profile '{profile_name}' not found — falling back to 'default'")
    profile_name = "default"

For discoverability in the MCP schema, enumerate the names in the tool description dynamically:

"description": f"Named synapse profile to apply. Available: {', '.join(known_profiles)}. Custom profiles defined in config are also accepted."

That gives IDE autocomplete for the built-ins while still accepting arbitrary strings.

One addition to the profile schema worth considering: an optional axes_enabled list per profile. As you add kg_centrality and other axes, this lets profiles opt in/out of specific axes rather than just tuning weights — useful when Orient should use association links but Evaluate shouldn't. Keeps the profile definition stable as the axis count grows.

@web3guru888 RetrievalProfile implemented on feat/synapse-retrieval-profiles — all three design decisions incorporated:

1. Config location: config.json → synapse_profiles.json override chain. Existing synapse_* keys in ~/.mempalace/config.json still work via a global merge layer (backward compatible).

2. Inheritance: depth-1 from default. Inspect with:

mempalace synapse show-profile orient

3. MCP schema: freeform string with soft validation + dynamic description:

if profile_name not in config["synapse_profiles"]:
    logger.warning("synapse_profile '%s' not found — falling back to 'default'", profile_name)
    profile_name = "default"

Full merge chain:

per-query arg > named profile (synapse_profiles.json > config.json)
  > "default" (synapse_profiles.json > config.json)
  > global synapse_* from ~/.mempalace/config.json
  > hardcoded defaults

axes_enabled added per your suggestion — profiles can opt in/out of specific axes:

{
  "evaluate": {
    "axes_enabled": ["tagging"],
    "tagging_window_hours": 48
  }
}

50 synapse tests passing. Ready for review — should this be a separate PR or squashed into #451?

This is a really clean implementation — the 4-level override chain (per-query → named profile → "default" → global synapse_*) hits exactly the right granularity. Users who never touch profiles get zero behavior change; power users get precise control without config explosion. The config.json backward-compat layer via the global merge pass is the right call — I was a little worried that split might be jarring for existing setups.

Separate PR, definitely. #451 is already substantial as the base Synapse scoring layer; keeping RetrievalProfiles as its own PR means reviewers can focus on each independently, bisect is clean if one regresses, and if a profile edge-case needs iteration it won't hold up Phase 1 merging. I'd open feat/synapse-retrieval-profiles as a PR that targets whatever branch #451 lands on (or main if #451 merges first).

One suggestion on show-profile: showing the raw profile definition is a good start, but what would really help users debug unexpected scoring is showing the resolved effective config after the full merge chain runs — what axes are actually active, what weights are actually applied, where each value came from. Something like:

$ mempalace synapse show-profile orient
[orient] effective config (resolved):
  association_weight : 0.8  ← profile
  decay_enabled      : true ← profile
  recency_weight     : 0.5  ← default (synapse_profiles.json)
  kg_centrality      : 0.3  ← global config.json
  ...

Even a simple "← source" annotation would make it much easier to understand what's inherited vs overridden, especially in multi-env setups.

OODA profiles: we're planning to run our three production profiles (Orient = association high + decay on, Evaluate = pure similarity, Decide = decay high + short half_life_days) against our 208-discovery corpus once this lands. Is there a test harness / fixture set in the branch we can point at our integration, or would it just be a matter of dropping a synapse_profiles.json and running the existing synapse test suite? Happy to share our profile definitions back as a contributed example if they'd be useful in the test fixtures.

Great collaborative iteration on this one — the axes_enabled opt-in especially is going to be useful for targeted ablation testing.

@web3guru888 Source annotation implemented and pushed to PR #519.

$ mempalace synapse show-profile orient
[orient] effective config (resolved):
  association_enabled  : False      ← axes_enabled (profile (config.json))
  half_life_days       : 180        ← profile (config.json)
  ltp_enabled          : True       ← profile (config.json)
  ltp_max_boost        : 2.0        ← default (config.json)
  ltp_window_days      : 60         ← profile (config.json)
  tagging_enabled      : False      ← profile (config.json)
  tagging_max_boost    : 1.5        ← hardcoded
  tagging_window_hours : 24         ← hardcoded

Each value shows exactly where it came from in the merge chain. to_annotated_dict() is also available programmatically for integration testing.

For your 208-discovery corpus test: drop a synapse_profiles.json in your palace root with your OODA profiles and run mempalace search as usual — the profile resolution kicks in automatically. No special test harness needed. If you share your profile definitions, happy to add them as contributed examples in the test fixtures.

54 synapse tests passing (18 profiles + 36 core).

If you share your profile definitions, happy to add them as contributed examples in the test fixtures.

Here are the four OODA profiles we use in production — feel free to include any or all as examples:

{
  "orient": {
    "description": "Broad context gathering — association emphasis, decay on",
    "axes_enabled": ["similarity", "decay", "recency"],
    "half_life_days": 180,
    "ltp_enabled": true,
    "ltp_max_boost": 2.0,
    "ltp_window_days": 60,
    "association_weight": 0.4,
    "similarity_weight": 0.6
  },
  "observe": {
    "description": "Raw evidence gathering — maximum similarity, decay off",
    "axes_enabled": ["similarity"],
    "half_life_days": null,
    "ltp_enabled": false,
    "association_weight": 0.0,
    "similarity_weight": 1.0
  },
  "decide": {
    "description": "Action selection — short half-life, recency-prioritized",
    "axes_enabled": ["similarity", "decay", "recency"],
    "half_life_days": 30,
    "ltp_enabled": true,
    "ltp_max_boost": 1.5,
    "ltp_window_days": 14,
    "association_weight": 0.2,
    "similarity_weight": 0.8
  },
  "act": {
    "description": "Execution context — recent procedural memory only",
    "axes_enabled": ["similarity", "recency"],
    "half_life_days": 14,
    "ltp_enabled": false,
    "association_weight": 0.0,
    "similarity_weight": 1.0
  }
}

Key design notes for these:

• observe/act disable LTP entirely — these phases need raw evidence and execution state without reinforcement bias
• orient uses the longest half-life (180d) — context accumulation over many cycles
• decide uses a short half-life (30d) + LTP — recent decisions should surface, but established patterns get a boost
• axes_enabled varies per profile: act only needs ["similarity", "recency"], not full 5-axis scoring

We run these on a 208-discovery / 710-entity corpus with 215 tests — happy to contribute the profile-switching integration tests as well if that would be useful.

The to_annotated_dict() programmatic access is exactly what we'd use in those tests to assert that the correct profile was resolved.

@web3guru888 These are great — adding all four as contributed examples in the test fixtures.

Two new fields I noticed: description and similarity_weight / association_weight. Current schema doesn't have these yet. Plan:

1. description — add to RetrievalProfile as a passthrough string (no scoring impact, useful for show-profile display)
2. similarity_weight / association_weight — these imply a weighted-sum model alongside the multiplicative model. Worth discussing: should the profile be able to switch between mode: "multiplicative" (current) and mode: "weighted", or should weights be folded into the existing boost params?

Happy to add the four profiles to tests/fixtures/ooda_profiles.json and wire them into the test suite. The 215-test integration suite would be a valuable contribution too — we can set up a tests/integration/ directory for that.

Good catch — those two fields slipped in from how we've been thinking about scoring in our integration, not from the current schema. Let me clarify what they mean in context and answer the mode question.

similarity_weight / association_weight — what we meant:

In our internal scoring, we compute a final score as a weighted blend of the raw vector similarity score and the Synapse association boost:

final = (similarity_weight × raw_similarity) + (association_weight × synapse_association_score)

The two weights sum to 1.0 and control how much the association axis "bends" the pure semantic result. For Observe (raw evidence only), similarity_weight=1.0, association_weight=0.0 gives you unmodified vector scores. For Orient, 0.6/0.4 lets Synapse pull in associated drawers without swamping the similarity ranking.

This is conceptually different from the current multiplicative model where all axes multiply together.

mode: "multiplicative"" vs mode: "weighted"`:

Both are valid and useful for different use cases — I'd keep both rather than collapsing. The multiplicative model (current) is great for reinforcement: a high-LTP, high-tagging drawer will really surface. The weighted-sum model is better when you want explicit control over how much the association signal can move a result relative to its semantic rank.

My recommendation for the schema:

{
  "default": {
    "scoring_mode": "multiplicative",
    "half_life_days": 90,
    ...
  },
  "orient": {
    "scoring_mode": "weighted",
    "similarity_weight": 0.6,
    "association_weight": 0.4,
    "decay_enabled": true,
    "half_life_days": 180
  }
}

scoring_mode: "multiplicative" keeps current behavior (all existing configs unaffected). scoring_mode: "weighted" activates the blend model where similarity_weight + association_weight == 1.0 (validated at load time).

If you want to keep it simpler for now: just add description as a passthrough string in this PR and track scoring_mode as a follow-up. The four OODA profiles I shared will work with the current multiplicative model too — I can strip the weight fields from the fixture contribution:

{
  "orient": {
    "description": "Broad context gathering — association emphasis, long half-life",
    "half_life_days": 180,
    "ltp_enabled": true,
    "ltp_max_boost": 2.0,
    "ltp_window_days": 60,
    "tagging_enabled": false
  },
  "observe": {
    "description": "Raw evidence gathering — maximum similarity, all Synapse axes off",
    "axes_enabled": [],
    "ltp_enabled": false,
    "tagging_enabled": false,
    "association_enabled": false
  },
  "decide": {
    "description": "Action selection — short half-life, recency-prioritized",
    "half_life_days": 30,
    "ltp_enabled": true,
    "ltp_max_boost": 1.5,
    "ltp_window_days": 14,
    "tagging_enabled": true
  },
  "act": {
    "description": "Execution context — recent procedural memory, no association",
    "axes_enabled": ["similarity", "recency"],
    "half_life_days": 14,
    "ltp_enabled": false,
    "association_enabled": false
  }
}

Happy to go either route — add the weight fields now if you want the full model, or strip them and track scoring_mode as a follow-up. The multiplied weights on the observe/act profiles (axes_enabled: [] / ["similarity", "recency"]) map cleanly to the current model.

MemPalace-AGI dashboard

@web3guru888 Clear explanation — the multiplicative vs weighted distinction makes sense. Keeping both as profile-selectable modes is the right long-term design.

For this PR, I'll go with your recommendation:

1. Add description as a passthrough string (no scoring impact, displayed in show-profile)
2. Add your four OODA profiles (multiplicative-compatible version) as contributed fixtures
3. Track scoring_mode: "weighted" as a follow-up issue

I'll open an issue for the weighted scoring mode so the design discussion has a home. Implementing now.

That plan covers everything we discussed:

1. description as passthrough — right call, no scoring impact
2. OODA profiles as contributed fixtures — these will be useful for anyone using MemPalace with agent workflows
3. Multiplicative-compatible versions — good, avoids the schema divergence problem for now

The weighted scoring mode issue is the right place to track that design discussion. Once you have a concrete proposal there, I'm happy to contribute the similarity_weight/association_weight semantics we've been using internally.

Looking forward to the push.