fix(bench): predictor DB-localization rule (Runtime gap)#2788
Conversation
Greptile code reviewThis repo uses Greptile for automated review. Before merge, aim for Confidence Score: 5/5 with zero unresolved review threads — see CONTRIBUTING.md. Run a review — add a PR comment with: Give it ~5-10 minutes (sometimes longer) for results, then fix feedback and re-trigger until you reach Confidence Score: 5/5. Optional: automate with the greploop skill. |
|
@greptile review |
Greptile SummaryAdds a "Database-localization rule" block to the predictor system prompt in
Confidence Score: 5/5Safe to merge — the change is purely additive prompt text with no modifications to parsing, validation, or any runtime code path. All changes are confined to the static string returned by _build_system_prompt(). No control-flow, schema validation, scoring logic, or external calls are touched. The new rule block follows the exact structural pattern of existing rules (namespace-scope, performance-fault), and the MySQL tiebreaker is grounded in the triage data cited in the PR description. No files require special attention. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[LLM receives alert + investigation summary] --> B{Investigation summary present?}
B -- Yes --> C[Treat as AUTHORITATIVE for rank-1]
B -- No --> D[Reason from alert alone]
C --> E{Root cause type?}
D --> E
E -- namespace_* token --> F["fault_object = namespace/X - Scope Rule"]
E -- DB symptoms named --> G{Which DB service?}
E -- Performance anomaly --> H["fault_object = service with highest saturation / latency - Performance-fault Rule"]
E -- Other --> I[Standard ranking]
G -- tsdb-mysql / MySQL error --> J["fault_object = app/tsdb-mysql - DB-localization Rule"]
G -- redis-cart / Redis error --> K["fault_object = app/redis-cart - DB-localization Rule"]
J --> L{MySQL root_cause?}
L -- port mismatch / refused on non-3306 --> M[mysql_invalid_port]
L -- access denied / auth failed --> N[mysql_invalid_credentials]
L -- too many connections / pool at limit --> O[db_connection_exhaustion]
L -- Uncertain --> M
K --> P{Redis root_cause?}
P -- NOAUTH / WRONGPASS / no requirepass --> Q[missing_secret_binding]
P -- READONLY replica --> R[db_readonly_mode]
P -- max clients reached / maxclients --> S[db_connection_exhaustion]
Reviews (3): Last reviewed commit: "added redis" | Re-trigger Greptile |
|
@greptile review |
|
🧠 @YauhenBichel opened a PR. Maintainers feared them. CI genuflected. It merged. 🚨
👋 Join us on Discord - OpenSRE : hang out, contribute, or hunt for features and issues. Everyone's welcome. |
Fixes #2074
Describe the changes you have made in this PR -
Add a Database-localization rule to the predictor system prompt to fix the dominant Runtime-stratum failure pattern surfaced by the loss-mode triage on
dev-2026-06-09T13:10:13Z. When DB symptoms appear (MySQL connection refused, port mismatch, auth failure, pool exhaustion), the predictor now localizes onto the DB service (app/tsdb-mysql,app/redis-cart) instead of the upstream caller that surfaces the failure.Root cause from the triage
Runtime loss-mode breakdown on the post-vocab-fix full-N (n=423 cells × 3 modes):
Within OBJECT_MISS,
tsdb-mysqlaccounted for 37 of 135 cells (27%). In 20 of those, the predictor pickedapp/ts-inside-payment-service(the immediate DB caller). Within OBJECT_HIT_RC_MISS, the dominant pattern wasmysql_invalid_port → db_connection_exhaustion(25 cells) — the same DB-localization problem at the root_cause level (LLM reaches for the generic exhaustion bucket rather than the specific port-misconfig cause).Combined: 62 of 233 Runtime losses (27%) are a form of DB-localization error.
Code Understanding and AI Usage
Did you use AI assistance (ChatGPT, Claude, Copilot, etc.) to write any part of this code?
If you used AI assistance:
Explain your implementation approach:
tests/benchmarks/cloudopsbench/predictor/llm_call.py_build_system_prompt(). Symmetric framing to the existing namespace-scope rule (faults live at their origin level).The rule has three parts:
fault_objectMUST be the DB service, not the upstream caller. Explicitly calls out thets-inside-payment-servicesubstitution as the wrong-localization pattern.mysql_invalid_port(port mismatch, "connection refused" on non-3306),mysql_invalid_credentials(auth failure), anddb_connection_exhaustion(actual pool saturation). Targets the 25-cellmysql_invalid_port → db_connection_exhaustionconfusion.mysql_invalid_portanddb_connection_exhaustion, prefer port — exhaustion is the over-fired generic bucket on this corpus.Checklist before requesting a review
Note: Please check Allow edits from maintainers if you would like us to assist in the PR.