Skip to content

fix(bench): break circular import in predictor/__init__.py#2802

Merged
YauhenBichel merged 3 commits into
mainfrom
fix/2074-bench-break-circular-import
Jun 12, 2026
Merged

fix(bench): break circular import in predictor/__init__.py#2802
YauhenBichel merged 3 commits into
mainfrom
fix/2074-bench-break-circular-import

Conversation

@YauhenBichel

Copy link
Copy Markdown
Collaborator

Fixes #2074

Describe the changes you have made in this PR -

tests/benchmarks/cloudopsbench/predictor/__init__.py was eagerly
re-exporting two names from
predictor.investigation_handoff. After the recent vocabulary
refactor, the import chain became:

predictor/init.py
→ investigation_handoff
→ scoring._taxonomy_for_root_cause
→ vocabulary
→ (back through scoring) → circular

This PR replaces the eager re-export with a PEP 562 __getattr__ lazy
loader, so apply_investigation_handoff and
align_predictions_to_investigation are resolved on first access
instead of at module load time. The cycle never executes.


Code Understanding and AI Usage

Did you use AI assistance (ChatGPT, Claude, Copilot, etc.) to write any part of this code?

  • No, I wrote all the code myself
  • Yes, I used AI assistance (continue below)

If you used AI assistance:

  • I have reviewed every single line of the AI-generated code
  • I can explain the purpose and logic of each function/component I added
  • I have tested edge cases and understand how the code handles them
  • I have modified the AI output to follow this project's coding standards and conventions

Explain your implementation approach:


Checklist before requesting a review

  • I have added proper PR title and linked to the issue
  • I have performed a self-review of my code
  • I can explain the purpose of every function, class, and logic block I added
  • I understand why my changes work and have tested them thoroughly
  • I have considered potential edge cases and how my code handles them
  • If it is a core feature, I have added thorough tests
  • My code follows the project's style guidelines and conventions

Note: Please check Allow edits from maintainers if you would like us to assist in the PR.

@github-actions

Copy link
Copy Markdown
Contributor

Greptile code review

This repo uses Greptile for automated review. Before merge, aim for Confidence Score: 5/5 with zero unresolved review threads — see CONTRIBUTING.md.

Run a review — add a PR comment with:

@greptile review

Give it ~5-10 minutes (sometimes longer) for results, then fix feedback and re-trigger until you reach Confidence Score: 5/5.

Optional: automate with the greploop skill.

@YauhenBichel

Copy link
Copy Markdown
Collaborator Author

@greptile review

@greptile-apps

greptile-apps Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR resolves a circular import in predictor/__init__.py by restructuring the module graph: vocabulary constants are extracted to a new leaf module (closed_vocabulary.py), the taxonomy_for_root_cause / infer_fault_object helpers are moved to a new taxonomy.py that both scoring and investigation_handoff can import without touching predictor/__init__.py, and a PEP 562 __getattr__ lazy loader is added as an additional guard for the two investigation-handoff re-exports. A separate L0 investigation-quality metrics panel is also added to the benchmark reporting output.

  • Structural fix (taxonomy.py + closed_vocabulary.py): eliminates the cycle at the dependency graph level by giving scoring and investigation_handoff a shared, package-agnostic import target; predictor/__init__.py, scoring.py, and investigation_handoff.py all updated to use the new paths.
  • Lazy loader (predictor/__init__.py): PEP 562 __getattr__ defers loading of investigation_handoff until first access; __all__ is kept intact so from predictor import * still resolves the two names correctly via getattr.
  • L0 reporting panel (reporting.py + test_reporting.py): adds markdown and HTML rendering for investigation-native metrics (investigation_a1, translation_loss, etc.) behind a presence check, with a companion test.

Confidence Score: 5/5

Safe to merge — the circular-import cycle is broken at the structural level, all callers are updated, and backward compatibility is preserved through re-export shims and the lazy loader.

The dependency graph refactor is correct and complete: closed_vocabulary.py and taxonomy.py are genuine leaf modules with no back-edges into predictor/, every direct caller of the old private functions is updated, and the vocabulary.py shim keeps the existing import surface intact. The __getattr__ lazy loader is a well-placed additional guard. No logic was changed in the scoring or prediction paths.

No files require special attention beyond the already-noted caching behaviour in predictor/__init__.py.

Important Files Changed

Filename Overview
tests/benchmarks/cloudopsbench/predictor/init.py Replaced eager re-export of investigation-handoff names with PEP 562 getattr lazy loader; the circular-import guard is functional but getattr does not pin resolved values into globals(), so attribute access through the module object will re-execute the import machinery on every call.
tests/benchmarks/cloudopsbench/taxonomy.py New module housing taxonomy_for_root_cause and infer_fault_object (moved from scoring.py); cleanly severs the circular dependency by giving scoring and investigation_handoff a shared import target that doesn't load predictor/init.py.
tests/benchmarks/cloudopsbench/closed_vocabulary.py New leaf module holding all closed-vocabulary constants; no imports from the bench subpackage, so it is safe to import from anywhere without triggering cycles.
tests/benchmarks/cloudopsbench/scoring.py Dropped local definitions of _infer_fault_object and _taxonomy_for_root_cause; now imports them from taxonomy.py. Two import blocks from the same module could be merged (style).
tests/benchmarks/cloudopsbench/predictor/investigation_handoff.py Switched from scoring._taxonomy_for_root_cause to taxonomy.taxonomy_for_root_cause; removes the direct dependency on scoring that was the other leg of the cycle.
tests/benchmarks/cloudopsbench/predictor/vocabulary.py Converted from a self-contained constants file to a thin re-export shim over closed_vocabulary.py; backward-compatible for all existing callers.
tests/benchmarks/_framework/reporting.py Added _render_l0_investigation_markdown and _render_l0_investigation_html panels and wired them into the existing render pipeline; follows the established pattern for optional metric panels.
tests/benchmarks/_framework/tests/test_reporting.py Added a test verifying the L0 investigation panel appears in markdown output when the relevant metrics are present in cell data.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    subgraph Before["Before (circular)"]
        direction TB
        A1["predictor/__init__.py"] -->|eager import| B1["investigation_handoff.py"]
        B1 -->|from scoring import _taxonomy_for_root_cause| C1["scoring.py"]
        C1 -->|from predictor.vocabulary import ...| D1["predictor/vocabulary.py"]
        D1 -->|triggers parent package init| A1
        style A1 fill:#f88,stroke:#c00
        style D1 fill:#f88,stroke:#c00
    end
    subgraph After["After (acyclic)"]
        direction TB
        A2["predictor/__init__.py"] -->|eager imports| E2["llm_call / snapping / rerank / vocabulary"]
        A2 -->|__getattr__ lazy| B2["investigation_handoff.py"]
        B2 -->|from taxonomy import taxonomy_for_root_cause| T2["taxonomy.py"]
        C2["scoring.py"] -->|from taxonomy import ...| T2
        E2 -->|from closed_vocabulary import ...| CV["closed_vocabulary.py"]
        T2 -->|from closed_vocabulary import ...| CV
        style CV fill:#8f8,stroke:#060
        style T2 fill:#8f8,stroke:#060
    end
Loading

Reviews (3): Last reviewed commit: "fixed security issues" | Re-trigger Greptile

Comment on lines +109 to +123
def __getattr__(name: str) -> Any:
"""Lazy-load investigation handoff so ``scoring`` can import ``vocabulary``
without a circular import (handoff imports ``_taxonomy_for_root_cause`` from
``scoring``)."""
if name in _INVESTIGATION_HANDOFF_EXPORTS:
from tests.benchmarks.cloudopsbench.predictor.investigation_handoff import (
align_predictions_to_investigation,
apply_investigation_handoff,
)

return {
"align_predictions_to_investigation": align_predictions_to_investigation,
"apply_investigation_handoff": apply_investigation_handoff,
}[name]
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The __getattr__ implementation does not cache the resolved symbol on the module's __dict__. PEP 562 only intercepts attribute lookups that miss __dict__, so from predictor import apply_investigation_handoff works (result is stored in the caller's namespace and __getattr__ is never called again for that caller), but any caller that accesses the symbol via direct module attribute access — e.g. predictor.apply_investigation_handoff(...) in a loop — will call __getattr__ on every access, constructing the dict each time. Pinning the resolved value into globals() (which is module.__dict__) after the first resolution is the standard PEP 562 pattern and eliminates the overhead.

Suggested change
def __getattr__(name: str) -> Any:
"""Lazy-load investigation handoff so ``scoring`` can import ``vocabulary``
without a circular import (handoff imports ``_taxonomy_for_root_cause`` from
``scoring``)."""
if name in _INVESTIGATION_HANDOFF_EXPORTS:
from tests.benchmarks.cloudopsbench.predictor.investigation_handoff import (
align_predictions_to_investigation,
apply_investigation_handoff,
)
return {
"align_predictions_to_investigation": align_predictions_to_investigation,
"apply_investigation_handoff": apply_investigation_handoff,
}[name]
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
def __getattr__(name: str) -> Any:
"""Lazy-load investigation handoff so ``scoring`` can import ``vocabulary``
without a circular import (handoff imports ``_taxonomy_for_root_cause`` from
``scoring``)."""
if name in _INVESTIGATION_HANDOFF_EXPORTS:
from tests.benchmarks.cloudopsbench.predictor.investigation_handoff import (
align_predictions_to_investigation,
apply_investigation_handoff,
)
_attrs = {
"align_predictions_to_investigation": align_predictions_to_investigation,
"apply_investigation_handoff": apply_investigation_handoff,
}
# Cache each resolved symbol so subsequent direct module-attribute
# accesses (e.g. ``predictor.apply_investigation_handoff(...)``) skip
# __getattr__ entirely — standard PEP 562 pattern.
globals().update(_attrs)
return _attrs[name]
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

@YauhenBichel

Copy link
Copy Markdown
Collaborator Author

@greptile review

Comment thread tests/benchmarks/cloudopsbench/closed_vocabulary.py Fixed
Comment thread tests/benchmarks/cloudopsbench/closed_vocabulary.py Fixed
Comment thread tests/benchmarks/cloudopsbench/closed_vocabulary.py Fixed
Comment thread tests/benchmarks/cloudopsbench/closed_vocabulary.py Fixed
Comment thread tests/benchmarks/cloudopsbench/closed_vocabulary.py Fixed
Comment thread tests/benchmarks/cloudopsbench/taxonomy.py Fixed
Comment thread tests/benchmarks/cloudopsbench/taxonomy.py Fixed
Comment thread tests/benchmarks/cloudopsbench/closed_vocabulary.py Fixed
Comment thread tests/benchmarks/cloudopsbench/closed_vocabulary.py Fixed
Comment thread tests/benchmarks/cloudopsbench/closed_vocabulary.py Fixed
Comment thread tests/benchmarks/cloudopsbench/closed_vocabulary.py Fixed
Comment thread tests/benchmarks/cloudopsbench/closed_vocabulary.py Fixed
Comment thread tests/benchmarks/cloudopsbench/taxonomy.py Fixed
Comment thread tests/benchmarks/cloudopsbench/taxonomy.py Fixed
@YauhenBichel YauhenBichel marked this pull request as ready for review June 12, 2026 13:58
@YauhenBichel YauhenBichel merged commit e86af16 into main Jun 12, 2026
16 checks passed
@YauhenBichel YauhenBichel deleted the fix/2074-bench-break-circular-import branch June 12, 2026 13:58
@github-actions

Copy link
Copy Markdown
Contributor

💼 Interviewer: describe a time you shipped something impactful.

@YauhenBichel: points at this PR

Interviewer: you're hired. 🤝


👋 Join us on Discord - OpenSRE : hang out, contribute, or hunt for features and issues. Everyone's welcome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Benchmark opensre+LLM vs LLM-alone (Cloudopsbench)

2 participants