fix(prompt): Prevent model from reverting successful changes#9253
Conversation
The prompt for the CLI agent is updated to prevent the model from reverting its own changes after successful verification. - The "Proactiveness" section is updated to instruct the model to consider all created files, especially tests, as permanent artifacts. - A new "Finalize" step is added to the workflow, explicitly instructing the model not to remove or revert any changes or created files after verification passes.
Summary of ChangesHello @galz10, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request enhances the reliability of the Gemini CLI agent by modifying its core prompt to ensure that successfully verified changes and created artifacts are permanently retained. Previously, the agent sometimes reverted its own work, such as deleting test files, after task completion. The updated prompt clarifies the expectation for persistence, preventing unintended data loss and improving the agent's overall consistency. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request updates the core prompt to prevent the AI model from reverting its own successful changes. The changes include making the "Proactiveness" section more explicit about treating created files as permanent artifacts, and adding a new "Finalize" step to the workflow that instructs the model not to undo any work after successful verification. The wording in the "Plan" step has also been improved for clarity. These changes are well-thought-out and directly address the stated problem by providing clearer and more explicit instructions to the model. The modifications should effectively prevent the undesirable behavior of the model deleting its own work.
|
Size Change: +301 B (0%) Total Size: 17.4 MB ℹ️ View Unchanged
|
TLDR
This pull request updates the core prompt for the Gemini CLI agent to prevent it from reverting its own changes after successful verification. It was observed that the model would sometimes undo its own work, such as deleting test files, after completing a task. These changes ensure that all work is preserved once it has been successfully verified.
Dive Deeper
The previous prompt was ambiguous about the final state of the work. The model, in an attempt to be "clean", would sometimes revert changes or delete files it created, especially test files. This behavior is undesirable as it destroys valuable work.
The changes in this PR address this issue by:
These changes make the prompt clearer and more explicit about the desired behavior, ensuring that the model preserves all successful work.
Reviewer Test Plan
To test this change, a reviewer can:
For example, a good prompt would be: "add a test file for the function
fooinfoo.tsand write a simple test case for it". After the model is done, the test file should remain in the file system.Testing Matrix
Linked issues / bugs