fix: use appendSystemPrompt for JSON mode instructions for Claude Code#889
fix: use appendSystemPrompt for JSON mode instructions for Claude Code#889ben-vargas wants to merge 3 commits into
Conversation
… provider - Move JSON formatting instructions from user prompt to appendSystemPrompt - Return jsonModeInstruction from message converter instead of appending to prompt - Update both doGenerate and doStream methods to handle JSON instructions properly - This should improve JSON parsing reliability by using proper system prompts
|
Converting to draft until someone who is having parsing issues tests to confirm if (A) the parsing issues are improved (there may need to be some parsing cleanup too), and (B) testing to ensure it doesn’t make things worse. |
|
Hi @ben-vargas - I'd love to test this as its a persistent issue for me. |
|
Thanks for your work on this @ben-vargas - here's what I found. To be honest, I am quite novice. I'm not completely certain I applied your new code correctly. If you have any recommendations on how to test - please let me know!
Let me know if you need more details about my process or the errors! Edit - the error codes seem to change from parse-to-parse. Here is another one when trying again parsing for 30 tasks: ⠹ Parsing PRD and generating tasks... |
Ported critical JSON parsing improvements from ai-sdk-provider-claude-code to enhance reliability and performance of JSON extraction in the custom SDK. Key improvements: - Replace basic regex patterns with jsonc-parser for robust parsing - Implement O(n) bracket-matching algorithm vs previous O(n²) approach - Add support for JSON with comments (JSONC) and trailing commas - Better handling of markdown code fences and variable declarations - Efficient fallback strategy for malformed JSON (limited to 1000 chars) - Return formatted JSON via JSON.stringify for consistency Performance gains: - Large valid JSON: <100ms (previously could timeout) - JSON with trailing garbage: <200ms with 10KB of noise - Deeply nested malformed JSON: <500ms (prevents hanging) Changes: - Install jsonc-parser dependency - Rewrite json-extractor.js with optimized implementation - Add comprehensive test suite with 26 tests covering edge cases - Verify compatibility with existing language-model.js usage This addresses potential issues with Claude's varied response formats and significantly improves performance for large or malformed JSON responses.
Previously, abort event listeners were added to AbortSignal without being
removed, potentially causing memory leaks when reusing the same signal
across multiple API calls. This was a critical issue that could lead to
resource exhaustion in production environments.
Changes implemented:
- Add proper cleanup of abort event listeners in finally blocks
- Use { once: true } option for automatic listener removal
- Add cancel handler for stream to ensure cleanup on cancellation
- Ensure listeners are removed even when errors occur
The fix applies to both doGenerate() and doStream() methods:
- Store abort listener reference to enable removal
- Add event listener with { once: true } for auto-cleanup
- Remove listener in finally block as defensive measure
- Add stream cancel handler to clean up on early termination
Added comprehensive test coverage:
- Verify listeners use { once: true } option
- Confirm cleanup in finally blocks
- Test cleanup on error conditions
- Validate stream cancellation cleanup
- Ensure no listener accumulation with signal reuse
This prevents memory leaks and follows best practices for event listener
management, matching the fix implemented in ai-sdk-provider-claude-code
(commit 4017a76).
|
|
@dhabedank - Thanks for testing. I'm not sure if that patch method would have applied properly or not. I typically just use something like I'm trying to break it now by using more than 10 tasks. I asked Claude Code to create a PRD (crm.prd) for a CRM and then I parsed this out with
I wonder if you can give this a try? I had already committed those two commits above with additional updates from my ai-sdk-provider-claude-code repo that hadn't yet been implemented in the custom-sdk built into task-master. So at the moment I don't know if any of these changes helped resolved the issue, or if I'm still just not testing it right to get it to break. I guess I can switch back to the main release and test the crm.prd again to see... |
|
No, switching to the main npm release parsed successfully too. I'm still not confident I can even replicate the parsing issue(s) everyone is seeing. I guess I need an actual PRD from someone that is failing (or obfuscated or made-up one similar to what I'm doing), but that is confirmed failing along with the exact parse-prd flags being used. |
|
@ben-vargas Thank you for your ongoing efforts to resolve this issue. I am still running into the same issue with master and this PR. Some background:
Here is the output from running
|
|
Thanks for your information contribution @nwentling5, and for taking the time to test. Indeed, while this PR has some improvements to parsing logic and use of system prompt input for instructing JSON output rather than appending to user prompt; it still doesn't make progress towards stopping claude code's seemingly arbitrary decision to cutoff output after 4000, 8000, 10000, 12000, 16000 characters. |
|
@ben-vargas Actually, here is my output. I thought I was on your PR, but npm was pointing to the wrong one.
Yeah. Very odd that we can't set that on the Claude side and get it to stick. Are there any other commands or settings you want me to try on my end? |
|
Excellent research and root cause was done by @LinLL in this issue report on our repo #913 and reported to claude code here anthropics/claude-code#2904 |
|
Can some of you experiencing this issue more frequently/consistently checkout #920 and test if you get better results? |
|
There are worthwhile fixes in here, but ultimately they were just guesses at the underlying fix for the discussion herein. I'm closing this PR for now as I believe the root cause on the truncation issues was found and addressed in #920 - these enhancements can be done later on an updated branch, possibly as part of a refactor to use ai-sdk-provider-claude-code as a dependency rather than an internal custom-sdk implementation. |
Fix: Use appendSystemPrompt for JSON mode instructions in Claude Code provider
Summary
This PR fixes an issue where JSON formatting instructions were being appended directly to the user's prompt in the Claude Code provider. The instructions are now properly sent through the
appendSystemPromptoption, which should improve JSON parsing reliability and maintain cleaner separation between user content and system instructions.Problem
When using the Claude Code provider in JSON mode (
object-json), the system was appending JSON formatting instructions directly to the user's prompt. This approach had several issues:Solution
The fix moves JSON formatting instructions from the user prompt to the
appendSystemPromptparameter:Modified
message-converter.js:finalPromptjsonModeInstructioncontaining the formatting rulesUpdated
language-model.js:doGenerateanddoStreammethods now extractjsonModeInstructionappendSystemPromptwhen in object-json modeappendSystemPromptalready exists (concatenate with newlines)Changes
src/ai-providers/custom-sdk/claude-code/message-converter.js: Return JSON instructions separately instead of appending to promptsrc/ai-providers/custom-sdk/claude-code/language-model.js: Use appendSystemPrompt for JSON instructions in both generate and stream methodsTesting
Impact
This change should improve:
Notes
This is a non-breaking change that maintains backward compatibility while improving the implementation of JSON mode in the Claude Code provider.