You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
echo "🎭 This is a stub - not sending to Slack"
if [ -f "$GH_AW_AGENT_OUTPUT" ]; then
MESSAGE=$(cat "$GH_AW_AGENT_OUTPUT" | jq -r '.items[] | select(.type == "send_slack_message") | .message')
echo "Would send to Slack: $MESSAGE"
{
echo "### 📨 Slack Message Stub"
echo "**Message:** $MESSAGE"
echo ""
echo "> ℹ️ This is a stub for testing purposes. No actual Slack message is sent."
} >> "$GITHUB_STEP_SUMMARY"
else
echo "No agent output found"
fi
append-only-comments
footer
run-started
run-success
run-failure
true
> 📰 *BREAKING: Report filed by [{workflow_name}]({run_url})*{effective_tokens_suffix}{history_link}
📰 BREAKING: [{workflow_name}]({run_url}) is now investigating this {event_type}. Sources say the story is developing...
📰 VERDICT: [{workflow_name}]({run_url}) has concluded. All systems operational. This is a developing story. 🎤
📰 DEVELOPING STORY: [{workflow_name}]({run_url}) reports {status}. Our correspondents are investigating the incident...
timeout-minutes
15
strict
false
Smoke Test: Copilot Engine Validation
IMPORTANT: Keep all outputs extremely short and concise. Use single-line responses where possible. No verbose explanations.
Test Requirements
GitHub MCP Testing: Review the last 2 merged pull requests in ${{ github.repository }}
MCP Scripts GH CLI Testing: Use the mcpscripts-gh tool to query 2 pull requests from ${{ github.repository }} (use args: "pr list --repo ${{ github.repository }} --limit 2 --json number,title,author")
Serena MCP Testing:
Use the Serena MCP server tool activate_project to initialize the workspace at ${{ github.workspace }} and verify it succeeds (do NOT use bash to run go commands - use Serena's MCP tools)
After initialization, use the find_symbol tool to search for symbols (find which tool to call) and verify that at least 3 symbols are found in the results
Playwright Testing: Use the playwright tools to navigate to https://github.com and verify the page title contains "GitHub" (do NOT try to install playwright - use the provided MCP tools)
Web Fetch Testing: Use the web-fetch tool to fetch https://github.com and verify the response contains "GitHub" (do NOT use bash or playwright for this test - use the web-fetch tool directly)
File Writing Testing: Create a test file /tmp/gh-aw/agent/smoke-test-copilot-${{ github.run_id }}.txt with content "Smoke test passed for Copilot at $(date)" (create the directory if it doesn't exist)
Bash Tool Testing: Execute bash commands to verify file creation was successful (use cat to read the file back)
Discussion Interaction Testing:
Use the github-discussion-query mcp-script tool with params: limit=1, jq=".[0]" to get the latest discussion from ${{ github.repository }}
Extract the discussion number from the result (e.g., if the result is {"number": 123, "title": "...", ...}, extract 123)
Use the add_comment tool with discussion_number: <extracted_number> to add a fun, playful comment stating that the smoke test agent was here
Build gh-aw: Run GOCACHE=/tmp/go-cache GOMODCACHE=/tmp/go-mod make build to verify the agent can successfully build the gh-aw project (both caches must be set to /tmp because the default cache locations are not writable). If the command fails, mark this test as ❌ and report the failure.
Upload gh-aw binary as artifact: After a successful build, use bash to copy the ./gh-aw binary into the staging directory (mkdir -p $RUNNER_TEMP/gh-aw/safeoutputs/upload-artifacts && cp ./gh-aw $RUNNER_TEMP/gh-aw/safeoutputs/upload-artifacts/gh-aw), then call the upload_artifact safe-output tool with path: "gh-aw", retention_days: 1, and skip_archive: true. The upload_artifact tool is available and configured in this workflow run — use it directly, do NOT use missing_tool for it. Mark this test as ❌ if the build in step 9 failed.
Discussion Creation Testing: Use the create_discussion safe-output tool to create a discussion in the announcements category titled "copilot was here" with the label "ai-generated"
Workflow Dispatch Testing: Use the dispatch_workflow safe output tool to trigger the haiku-printer workflow with a haiku as the message input. Create an original, creative haiku about software testing or automation.
PR Review Testing: Review the diff of the current pull request. Leave 1-2 inline create_pull_request_review_comment comments on specific lines, then call submit_pull_request_review with a brief body summarizing your review and event COMMENT. To test reply_to_pull_request_review_comment: use the pull_request_read tool (with method: "get_review_comments" and pullNumber: ${{ github.event.pull_request.number }}) to fetch the PR's existing review comments, then reply to the most recent one using reply_to_pull_request_review_comment with its actual numeric id as the comment_id. Note: create_pull_request_review_comment does not return a comment_id — you must fetch existing comment IDs from the GitHub API. If the PR has no existing review comments, skip the reply sub-test.
Output
Create an issue with a summary of the smoke test run:
Use a temporary ID (e.g. aw_smoke1) for the issue so you can reference it later
Run URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
Timestamp
Pull request author and assignees
Set Issue Type: Use the set_issue_type safe-output tool with issue_number: "aw_smoke1" (the temporary ID from step 1) and issue_type: "Bug" to set the type of the just-created smoke test issue.
Only if this workflow was triggered by a pull_request event: Use the add_comment tool to add a very brief comment (max 5-10 lines) to the triggering pull request (omit the item_number parameter to auto-target the triggering PR) with:
PR titles only (no descriptions)
✅ or ❌ for each test result
Overall status: PASS or FAIL
Mention the pull request author and any assignees
Use the add_comment tool to add a fun and creative comment to the latest discussion (using the discussion_number you extracted in step 8) - be playful and entertaining in your comment
Use the send_slack_message tool to send a brief summary message (e.g., "Smoke test ${{ github.run_id }}: All tests passed! ✅")
If all tests pass and this workflow was triggered by a pull_request event:
Use the add_labels safe-output tool to add the label smoke-copilot to the pull request (omit the item_number parameter to auto-target the triggering PR)
Use the remove_labels safe-output tool to remove the label smoke from the pull request (omit the item_number parameter to auto-target the triggering PR)
Important: If no action is needed after completing your analysis, you MUST call the noop safe-output tool with a brief explanation. Failing to call any safe-output tool is the most common cause of safe-output workflow failures.
{"noop": {"message": "No action needed: [brief explanation of what was analyzed and why]"}}