refactor(ci): embed e2e advisor with Pi SDK#3504
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (3)
✅ Files skipped from review due to trivial changes (1)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughWalkthroughMigrate the E2E advisor from a Pi-specific CLI to an SDK-driven analyzer: add an SDK devDependency, add a TypeScript analyzer CLI that builds schema-driven prompts, normalizes JSON results, and writes artifacts; update the workflow to install and run the SDK-based analyzer; refresh docs, comment tooling, and artifact/summary paths. ChangesE2E Advisor SDK Migration
sequenceDiagram
participant GH as GitHub Actions
participant Analyzer as analyze.mts
participant Git as Git
participant SDK as Advisor SDK
participant Provider as AI Provider
participant Artifacts as Artifacts
GH->>Analyzer: workflow dispatch / env
Analyzer->>Git: compute changed files & diff
Git-->>Analyzer: files + diff
Analyzer->>Artifacts: write prompt artifact
alt skip (E2E_ADVISOR_RUN_ANALYSIS=0 or no creds)
Analyzer->>Artifacts: write unavailable summary/result
Analyzer->>GH: exit 0
else run analysis
Analyzer->>SDK: prepare in-memory config, select model
SDK->>Provider: send prompt, enforce read-only tools
Provider-->>SDK: response + tool events
SDK-->>Analyzer: captured output (capped, timeout-bounded)
Analyzer->>Analyzer: extract JSON, normalize AdvisorResult
Analyzer->>Artifacts: write raw/result/final/summary
Analyzer->>GH: publish summary, post PR comment
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.github/workflows/e2e-advisor.yaml (1)
185-189:⚠️ Potential issue | 🟠 Major | ⚡ Quick winGuard trusted analyzer entrypoint existence before execution.
Line 185 invokes
analyze.mtsunconditionally from the trustedmaincheckout, which is causingMODULE_NOT_FOUNDin CI and cascading artifact/comment failures.💡 Suggested fix
- name: Run E2E recommendation advisor id: analysis continue-on-error: true env: @@ run: | cd "$ADVISOR_WORKDIR" + if [ ! -f "$ADVISOR_DIR/tools/e2e-advisor/analyze.mts" ]; then + mkdir -p "$GITHUB_WORKSPACE/artifacts/e2e-advisor" + printf '# E2E Recommendation Advisor\n\nSkipped: trusted main checkout does not yet contain tools/e2e-advisor/analyze.mts.\n' > "$GITHUB_WORKSPACE/artifacts/e2e-advisor/e2e-advisor-summary.md" + exit 0 + fi node --experimental-strip-types "$ADVISOR_DIR/tools/e2e-advisor/analyze.mts" \ --base "$BASE_REF" \ --head "$HEAD_REF" \ --schema "$ADVISOR_DIR/tools/e2e-advisor/schema.json" \ --out-dir "$GITHUB_WORKSPACE/artifacts/e2e-advisor"🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/e2e-advisor.yaml around lines 185 - 189, Guard the trusted analyzer entrypoint before invoking node: check that "$ADVISOR_DIR/tools/e2e-advisor/analyze.mts" exists and is readable and only run the node command if it does; if the file is missing, log a clear message and skip or exit gracefully so CI doesn't show MODULE_NOT_FOUND and cascade failures. Update the step that runs analyze.mts to perform this existence check (refer to analyze.mts and the ADVISOR_DIR variable and the node invocation that writes to "$GITHUB_WORKSPACE/artifacts/e2e-advisor") and ensure downstream steps handle the skipped case appropriately.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In @.github/workflows/e2e-advisor.yaml:
- Around line 185-189: Guard the trusted analyzer entrypoint before invoking
node: check that "$ADVISOR_DIR/tools/e2e-advisor/analyze.mts" exists and is
readable and only run the node command if it does; if the file is missing, log a
clear message and skip or exit gracefully so CI doesn't show MODULE_NOT_FOUND
and cascade failures. Update the step that runs analyze.mts to perform this
existence check (refer to analyze.mts and the ADVISOR_DIR variable and the node
invocation that writes to "$GITHUB_WORKSPACE/artifacts/e2e-advisor") and ensure
downstream steps handle the skipped case appropriately.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: c00fde76-cd5d-4365-acca-ee660299208b
⛔ Files ignored due to path filters (1)
package-lock.jsonis excluded by!**/package-lock.json
📒 Files selected for processing (7)
.github/workflows/e2e-advisor.yamlpackage.jsontools/e2e-advisor/README.mdtools/e2e-advisor/analyze.mtstools/e2e-advisor/comment.mtstools/e2e-advisor/models.template.jsontools/e2e-advisor/pi-analyze.mjs
💤 Files with no reviewable changes (1)
- tools/e2e-advisor/pi-analyze.mjs
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tools/e2e-advisor/analyze.mts`:
- Around line 349-354: The truncation banner can be lost because
raw.prepend(...) may be immediately trimmed away if raw is already full; change
the pattern to ensure space for the banner before inserting it (e.g., compute
banner = `<...truncated...>` then call a method that trims the buffer from the
opposite end or explicitly remove enough bytes from the end/front to make room,
and only then prepend), or alternatively append the banner to the end of the
buffer instead of prepending so trimming won't remove it. Update the occurrences
that use raw.prepend(...) (and the related text.droppedBytes logic) at the
locations shown (the prepend calls around the variables raw and text) to first
reserve room for the banner or append the banner so the truncation marker cannot
be immediately discarded.
- Around line 104-105: Temporary advisor configDir (variable configDir) and
files like auth.json/models.json and persisted session data must be removed
after the run to avoid leaving API keys on disk; add deterministic cleanup that
removes configDir (use fs.promises.rm or fs.rm with recursive:true, force:true)
in all exit scenarios by invoking deletion in a finally block around the main
async flow and/or registering handlers for process exit signals (exit, SIGINT,
uncaughtException) so configDir is always deleted whether the run succeeds or
errors; ensure deletion occurs after any code that reads/writes those files (the
blocks that write auth.json/models.json and persist session data) to avoid race
conditions.
- Around line 131-137: The credential check (hasLikelyAdvisorCredential()) and
the model materialization (prepareAdvisorConfig()) are inconsistent: preflight
allows provider-specific creds (ANTHROPIC_API_KEY, OPENAI_API_KEY, Bedrock etc.)
but prepareAdvisorConfig() only writes models.json when E2E_ADVISOR_API_KEY is
present, causing "Could not find configured advisor model" later. Fix by making
the flows consistent: either (A) tighten hasLikelyAdvisorCredential() to require
E2E_ADVISOR_API_KEY only, or (B) update prepareAdvisorConfig() to detect
provider-specific credentials (use the same detection logic as
hasLikelyAdvisorCredential()), resolve provider with E2E_ADVISOR_PROVIDER ||
defaultProvider detection, call defaultModelForProvider(provider) and write a
non-empty models.json for that provider even when E2E_ADVISOR_API_KEY is not
set; ensure code paths that reference modelPattern (E2E_ADVISOR_MODEL ||
defaultModelForProvider(provider)) match this logic so prepareAdvisorConfig()
materializes the same model names.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 8f0c5f45-3b31-40d0-b730-9eb1a4fa5f94
📒 Files selected for processing (1)
tools/e2e-advisor/analyze.mts
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.github/workflows/e2e-advisor.yaml (1)
88-93:⚠️ Potential issue | 🟠 Major | ⚡ Quick winDon't switch the trusted entrypoint to
analyze.mtsyet.This job executes the analyzer from the separate
ref: maincheckout, not from the PR branch.maindoes not containtools/e2e-advisor/analyze.mtsyet, so the step hard-fails withMODULE_NOT_FOUNDbefore any advisor artifacts are produced. Please keep the trusted run path on the existing analyzer untilmainhas the new file, or add the same.mts/legacy fallback pattern you already use fordispatch.mtsandcomment.mts.Also applies to: 157-175
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/e2e-advisor.yaml around lines 88 - 93, The workflow step that checks out the trusted advisor code and then runs analyze.mts is switching the trusted run to a file that doesn't exist on main and causes MODULE_NOT_FOUND; restore the existing trusted analyzer path (keep running the analyzer from the PR checkout) or add the same fallback logic used for dispatch.mts and comment.mts so analyze resolution tries the legacy path first then analyze.mts (mirror the pattern used around dispatch.mts/comment.mts); update the checkout/step that references analyze.mts and the corresponding resolution logic in the workflow (also apply the same change to the other occurrence referenced in lines 157-175) so the job does not hard-fail when main lacks tools/e2e-advisor/analyze.mts.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In @.github/workflows/e2e-advisor.yaml:
- Around line 88-93: The workflow step that checks out the trusted advisor code
and then runs analyze.mts is switching the trusted run to a file that doesn't
exist on main and causes MODULE_NOT_FOUND; restore the existing trusted analyzer
path (keep running the analyzer from the PR checkout) or add the same fallback
logic used for dispatch.mts and comment.mts so analyze resolution tries the
legacy path first then analyze.mts (mirror the pattern used around
dispatch.mts/comment.mts); update the checkout/step that references analyze.mts
and the corresponding resolution logic in the workflow (also apply the same
change to the other occurrence referenced in lines 157-175) so the job does not
hard-fail when main lacks tools/e2e-advisor/analyze.mts.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 1c50e94f-63ad-43d8-94eb-7ec3a930cfc0
📒 Files selected for processing (3)
.github/workflows/e2e-advisor.yamltools/e2e-advisor/README.mdtools/e2e-advisor/analyze.mts
✅ Files skipped from review due to trivial changes (1)
- tools/e2e-advisor/README.md
Summary
Refactors the E2E advisor from a spawned Pi CLI process into an embedded Pi SDK session. This keeps the recommendation behavior and artifact contract while simplifying process handling and adding an exported HTML session artifact.
Changes
tools/e2e-advisor/pi-analyze.mjswith TypeScripttools/e2e-advisor/analyze.mtsusing the Pi SDK directly.@earendil-works/pi-coding-agent, run the TypeScript analyzer, and upload renamed advisor artifacts includinge2e-advisor-session.html.pi-*wording while keeping the existingPI_E2E_ADVISOR_API_KEYGitHub secret mapped into the analyzer environment.Type of Change
Verification
npx prek run --all-filespassesnpm testpassesmake docsbuilds without warnings (doc changes only)Note:
npm testwas run, but the existing installer-integration testskips onboarding when shared host preflight detects Docker is missingfails locally with status1instead of0; targeted rerun reproduces the same unrelated failure.Signed-off-by: Carlos Villela cvillela@nvidia.com
Summary by CodeRabbit
Chores
Documentation