fix: strengthen prd.json schema instructions to prevent AI hallucination#142
Conversation
|
@anthonymq is attempting to deploy a commit to the plgeek Team on Vercel. A member of the Team first needs to authorize it. |
WalkthroughThe PR strengthens JSON schema validation guidance across documentation and code by mandating a flat root-level structure with "name" field, explicitly prohibiting invalid patterns like "prd" wrapper objects and "tasks" arrays, and providing comprehensive schema requirements and anti-patterns to prevent AI hallucination of invalid schemas. Changes
Estimated Code Review Effort🎯 2 (Simple) | ⏱️ ~12 minutes Possibly Related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 markdownlint-cli2 (0.18.1)skills/ralph-tui-create-json/SKILL.mdmarkdownlint-cli2 v0.18.1 (markdownlint v0.38.0) Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Fixes subsy#141 The AI agent was generating invalid prd.json files with nested structures (e.g., wrapping everything in a 'prd' object) instead of the flat schema expected by the JSON tracker. Changes: - PrdChatApp.tsx: Added explicit WRONG examples and stronger DO NOT USE guidance in the skillPrompt - ralph-tui-create-json/SKILL.md: Added comprehensive anti-patterns section with visual ❌ WRONG and ✅ CORRECT examples The validation in json.ts already correctly rejects these invalid schemas, but this fix prevents the AI from generating them in the first place.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #142 +/- ##
=======================================
Coverage 44.11% 44.11%
=======================================
Files 62 62
Lines 15716 15716
=======================================
Hits 6933 6933
Misses 8783 8783 🚀 New features to boost your workflow:
|
311d19f to
0258339
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
skills/ralph-tui-create-json/SKILL.md (1)
300-356: Fix field name inconsistency in example.The example uses
"project"field (line 303), but the schema definition requires"name"field (line 56, line 12). This inconsistency contradicts the updated schema and could confuse users or the AI agent.📝 Proposed fix
{ - "project": "my-app", + "name": "my-app", "branchName": "ralph/task-priority", "description": "Add priority levels to tasks",
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
@anthonymq thanks - nice clean PR - good enhancement! |
…cination fix: strengthen prd.json schema instructions to prevent AI hallucination
Summary
Fixes #141
The AI agent was generating invalid
prd.jsonfiles with nested structures (e.g., wrapping everything in a"prd"object) instead of the flat schema expected by the JSON tracker.Changes
PrdChatApp.tsx: Added explicit WRONG examples and stronger "DO NOT USE" guidance in the skillPromptralph-tui-create-json/SKILL.md: Added comprehensive anti-patterns section with visual ❌ WRONG and ✅ CORRECT examplesProblem
When using
ralph-tui create-prdwith chat mode, the AI agent would sometimes generate:{ "prd": { "metadata": {...}, "overview": {...}, ... } }Instead of the correct flat structure:
{ "name": "Feature Name", "userStories": [...] }Solution
The validation in
json.tsalready correctly rejects these invalid schemas, but this fix prevents the AI from generating them in the first place by:Testing
bun run typecheckpassesbun run buildsucceedsSummary by CodeRabbit
Documentation
✏️ Tip: You can customize this high-level summary in your review settings.