fix: route prompt graders through CopilotEngine#258
Merged
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
# Conflicts: # internal/graders/prompt_grader.go # internal/graders/prompt_grader_test.go
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR migrates prompt graders to execute judge turns via the shared execution.CopilotEngine instead of constructing copilot.Client instances directly, while adding engine request options needed by graders (tools, enqueue mode, streaming, ephemeral sessions, and skipping workspace capture).
Changes:
- Thread a narrow
Executorintograders.Contextand plumb it from the orchestration runner. - Update prompt graders (independent + pairwise) to call the execution engine with grader-specific execution options.
- Extend execution request/engine support for tools, streaming, message mode, ephemeral sessions (with deletion), and optional workspace capture; update mocks/tests accordingly.
Show a summary per file
| File | Description |
|---|---|
| internal/orchestration/runner.go | Attaches the execution engine to grader context so prompt graders can execute judge turns. |
| internal/graders/grader.go | Introduces a narrow Executor interface and adds it to graders.Context. |
| internal/graders/prompt_grader.go | Routes prompt grading through the execution engine and preserves post-grade error recovery behavior. |
| internal/graders/prompt_grader_test.go | Updates/extends tests to validate executor usage and new execution request options. |
| internal/execution/engine.go | Extends ExecutionRequest with tools, message mode, streaming, ephemeral session, and workspace-capture skipping. |
| internal/execution/copilot.go | Implements the new request options in CopilotEngine.Execute, including ephemeral session deletion and optional workspace capture. |
| internal/execution/mock.go | Adds SkipWorkspaceCapture handling to the mock engine response. |
| internal/execution/copilot_engine_test.go | Adds coverage for grader request options and ephemeral session deletion / tracking behavior. |
Copilot's findings
- Files reviewed: 8/8 changed files
- Comments generated: 2
This was referenced May 20, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
copilot.ClientdirectlyCloses #54
Validation
go test ./...copilot.NewClientis confined tointernal/execution