[Copilot speaking]

In v0.33.0 which upgraded the bundled copilot-cli from 1.0.2 to 1.0.49 (copilot-sdk go module v0.1.32 → v0.3.0), every non-skill tool call in an eval run is rejected. The agent sees unexpected user permission response on each tool invocation and eventually gives up. Trigger evals that only invoke the skill tool appear to work because the SDK handles skill internally without going through the permission flow.

Root cause

In copilot-sdk go@v0.3.0, the wire values of PermissionRequestResultKind changed:

// vendor/github.com/github/copilot-sdk/go@v0.3.0/types.go
const (
    PermissionRequestResultKindApproved PermissionRequestResultKind = "approve-once" // was "approved"
    PermissionRequestResultKindRejected PermissionRequestResultKind = "reject"
    PermissionRequestResultKindUserNotAvailable PermissionRequestResultKind = "user-not-available"
    PermissionRequestResultKindNoResult         PermissionRequestResultKind = "no-result"
)

Waza's permission handler in internal/execution/copilot.go still returns the old literal:

func allowAllTools(request copilot.PermissionRequest, invocation copilot.PermissionInvocation) (copilot.PermissionRequestResult, error) {
    // value for 'Kind' came from the permissions_test.go in the Copilot SDK.
    return copilot.PermissionRequestResult{Kind: "approved"}, nil
}

copilot-cli 1.0.49 doesn't recognise the legacy string and treats the response as malformed. The corresponding permission.completed event arrives at the agent as a denial.

Evidence

• Permission events are paired (11 permission.requested / 11 permission.completed) — not a hang.
• Only the skill tool returns success: true in session_digest.tool_calls. view, glob, powershell, etc. all return success: false.
• Agent's final reasoning: "It seems like all my tool calls are being denied with 'unexpected user permission response'".
• Same eval spec passed on the previous waza build (which bundled copilot-cli 1.0.2 and the old "approved" wire value).

Fix

Two equivalent options:

// Option A — use the SDK constant
func allowAllTools(request copilot.PermissionRequest, invocation copilot.PermissionInvocation) (copilot.PermissionRequestResult, error) {
    return copilot.PermissionRequestResult{Kind: copilot.PermissionRequestResultKindApproved}, nil
}

// Option B — drop allowAllTools entirely and use the SDK's built-in handler
permRequestCallback := copilot.PermissionHandler.ApproveAll

Option B is preferable; it stays correct automatically if the SDK changes the canonical kind again.

Suggested additional hardening

• Consider treating an unknown/empty Kind returned from OnPermissionRequest as an error at session creation time (or in CI) so the next wire-value rename surfaces as a build/test failure instead of silent per-tool denials.
• A smoke test that exercises one non-skill tool call through allowAllTools would have caught this — the existing tests appear to only validate the struct shape, not the round-trip through copilot-cli.