Linked Issue

Closes #3788

Description

Local LLMs (LM Studio, Ollama) often wrap JSON output in conversational text (e.g., "Here is the memory extraction: {...} I hope this helps!"). This caused empty results because json.loads failed on the chatty text and the error was silently swallowed.

Three changes:

1. Enhanced extract_json() to locate JSON by first { / last } boundaries when no code block is found. This is safe for all existing callers — clean JSON, code-block-wrapped JSON, and arrays all produce identical results since the heuristic only activates when there's no code block AND the text contains {/}.

2. Added JSONDecodeError fallback to memory action parsing in both sync and async _add_to_vector_store paths. This was completely missing — when the LLM returned chatty text, json.loads(remove_code_blocks(response)) failed and the outer except Exception silently set new_memories_with_actions = {}.

3. Fixed fact retrieval paths to pass the original response to extract_json() instead of the already-processed output of remove_code_blocks(). Previously response = remove_code_blocks(response) mutated the variable, so the fallback received degraded input.

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Refactor (no functional changes)
• Documentation update

Breaking Changes

N/A

Test Coverage

• I added/updated unit tests
• I added/updated integration tests
• I tested manually (describe below)
• No tests needed (explain why)

Tests Added

New test file tests/test_chatty_llm_parsing.py with 21 tests across three test classes:

TestExtractJson (9 tests) — validates extract_json() handles:

• Pure JSON, markdown code blocks, plain code blocks
• Chatty text with embedded markdown code blocks
• Chatty text without code blocks (new {/} heuristic)
• Multi-line JSON in chatty text, nested objects, whitespace, no-JSON-at-all

TestRemoveCodeBlocks (4 tests) — regression tests confirming:

• Clean code block stripping works
• Chatty text without code blocks is NOT parsed (verifying fallback is needed)
• <think> tag handling edge cases

TestFallbackChain (8 tests) — validates the full remove_code_blocks → extract_json pattern as used in main.py:

• Clean JSON, markdown-wrapped, chatty+markdown, chatty without markdown
• <think> tags with JSON, LM Studio-style responses
• Completely invalid responses (verifies error propagation)
• Facts extraction responses (both LLM call paths)

All 58 tests pass (21 new + 25 existing memory + 12 LLM provider tests).

Checklist

• My code follows the project's style guidelines
• I have performed a self-review of my code
• I have added tests that prove my fix/feature works
• New and existing tests pass locally
• I have updated documentation if needed