Implement bot that performs time-series metric analysis and suggests repo management improvements#25945
Conversation
…thout package.json scripts
…e brain instructions
|
Size Change: -4 B (0%) Total Size: 33.9 MB
ℹ️ View Unchanged
|
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces an autonomous agent designed to improve repository health by analyzing time-series metrics and proactively suggesting management improvements. By separating concerns into deterministic data collection and agentic reasoning, the system can identify bottlenecks—such as high PR latency or maintainer workload imbalances—and propose targeted, validated interventions. The implementation emphasizes security and human-in-the-loop oversight, ensuring that all proposed changes are technically critiqued and require manual approval before being applied to the repository. Highlights
Ignored Files
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request introduces a metrics history synchronization and rolling window update mechanism for the Gemini CLI bot. It adds a sync.ts script to download historical metrics from previous CI runs and updates metrics/index.ts to calculate 7-day and 30-day deltas for tracked metrics. My review identified several critical issues: missing imports for execFileSync in multiple files, vulnerabilities to command injection due to unsafe shell execution, and an incorrect rolling window size in the metrics update logic that contradicts the intended 5000-row retention policy.
…repo management improvements (google-gemini#25945)
…repo management improvements (google-gemini#25945)
Summary
Implements an agent that runs nightly and analyzes the metrics time series and sends PRs to optimize them. For example:
Upon observing high per-maintainer issue load, the bot may send a pull request implementing an automated triage process. Maintainers can leave comments to collaborate with and leave feedback for the bot until the PR is completed. The bot learns from and remembers this user feedback, allowing it to adapt to the repo maintainers expectations.
Once the PR is completed, the bot remembers this as well as any relevant user feedback in its memory and uses this to focus subsequent efforts.
Why did you choose this approach
This repository needs improvements made to its policies and/or automation to help aid maintainers in keeping a steady throughput of work items, PRs etc. All of these goals can be easily validated via metrics.
A human can spend several days working through these metrics and produce a plan with targeted improvements to the existing automation, however, this is a static set of mitigations, which solve only the cases the human thinks of, and cannot react to new trends in the repository. An agent, however, is able to monitor metrics on some cadence, discover anomalies, recommend new policies, monitor them in the days following their check in, and then recommend follow up actions to improve their effectiveness.
Examples:
Demo run
In an example run, the agent noticed that the 1000 item issue count metric was unchanged across two runs, and it deduced that it is inaccurate. It sent this PR to correct the issue: https://github.com/google-gemini/gemini-cli/pull/26071/changes
Security
For security purposes, the agent runs in an isolated GitHub actions workflow stage with no direct credential or repo write access. It is solely responsible for investigating and recommending changes to the user in the form of a git patch which is uploaded, and applied in a subsequent "publish" stage, which has the write access necessary to perform repo updates. It can also push updates and comment on its own PRs but deterministic policies disallow interactions with others PRs.
Human approval is required for all repo changes.
Memory
Memory between runs is stored in transient GitHub actions artifacts in a lessons-learned.md file. In a lot of ways, this agent acts as a sort of "unrolled loop" of a persistent agent. Each session is a brand new session, but seeded with memory and data accumulated from past runs.
Future Considerations
Building a system like this that periodically introspects on and suggests improvements to key metrics opens the door to the notion of agent directly and proactively contributing to the Gemini CLI codebase. For now, we'll scope this to engineering system improvements but one can imagine growing this into a fully fledged AI contributor which makes product improvements, proactively fixes bugs based on their impact, etc.