Summary

• Date: 2026-04-03
• Pages visited: /gh-aw/ (Home), /gh-aw/setup/quick-start/, /gh-aw/setup/cli/
• Overall impression: As a new user, the docs have a polished, professional landing page and a clear getting-started flow. The Quick Start guide is well-structured with numbered steps, estimated time, and a walkthrough video. However, a few friction points exist for complete beginners — especially around jargon, the search not working in dev mode, and the add-wizard command path being opaque.

⚠️ Note: Playwright visual screenshots were unavailable in this environment (network isolation between agent and Playwright containers). All analysis was performed via curl + Python text extraction.

🔴 Critical Issues Found

1. Search is non-functional in dev mode

The search bar shows a modal with the message: "Search is only available in production builds. Try building and previewing the site to test it out locally."

As a new user landing on the docs and wanting to search for "how to install" or "Copilot setup", the search button does nothing — without an obvious explanation visible until you click it. This is especially jarring for users who instinctively reach for search first.

Impact: Blocks discoverability for new users trying to find specific help.
Recommendation: Show the "dev only" warning directly on the disabled search button, or add a tooltip so users aren't confused by a silent click.

🟡 Confusing Areas

2. "frontmatter" jargon used without link in Step 4

In Step 4 of the Quick Start, the guide says:

"If you have changed the frontmatter (the YAML configuration block between --- markers at the top of the file)..."

The word is used with a brief inline definition, which is good. However, it is not linked to the Frontmatter reference page (which exists at /gh-aw/reference/frontmatter/). A curious beginner wanting to learn more about what YAML options are available has to manually navigate the sidebar to find it.

Recommendation: Link "frontmatter" to /gh-aw/reference/frontmatter/ in Step 4.

3. add-wizard command uses an opaque workflow path

Step 2 instructs the user to run:

gh aw add-wizard githubnext/agentics/daily-repo-status

The argument githubnext/agentics/daily-repo-status is not explained. As a beginner I have no idea:

• What githubnext/agentics is (an org? a repo? a registry?)
• Whether I can use other values here
• How I'd find other available workflows to add

Recommendation: Add a sentence like: "This installs a workflow from GitHub's workflow registry (githubnext/agentics) — a curated collection of pre-built agentic workflows."

4. "COPILOT_GITHUB_TOKEN is distinct from GITHUB_TOKEN" — not immediately obvious why

Step 2 mentions requiring a COPILOT_GITHUB_TOKEN that is "distinct from the default GITHUB_TOKEN" and links to the Authentication page. For a new user, it's not immediately clear why two tokens are needed or what makes the Copilot token special. The distinction is important for security reasons but gets lost in the step.

Recommendation: Add a one-line TL;DR in Step 2, e.g.: "This is needed because GitHub Copilot requires a Personal Access Token or GitHub App token — the default workflow token doesn't include Copilot API access."

5. CLI page is overwhelming for beginners

The CLI Commands page lists every command in exhaustive detail — init, add-wizard, add, compile, list, run, status, logs, audit, health checks, enable, disable, remove, update, upgrade, mcp, pr, transfer, mcp-server, domains, version, completion, project, hash-frontmatter, and more. There is a "Most Common Commands" table at the top (✅ good), but the sheer volume of content overwhelms a beginner looking for "what do I actually need to use?"

Recommendation: Consider splitting the CLI page into a "Getting Started commands" section and an "Advanced / Reference" section. The Most Common Commands table is a good start but could be more prominent (e.g., sticky or in a callout box).

6. No mention of what happens if GitHub Actions is not enabled

The Prerequisites section says "GitHub Actions enabled — Check in Settings → Actions" but doesn't explain what to do if it's not enabled or what the consequence is. A new user who's never used GitHub Actions may not know where to look.

Recommendation: Add a link to GitHub's docs on enabling Actions, or a brief note: "If Actions is disabled, navigate to Settings → Actions → General → Allow all actions."

🟢 What Worked Well

1. Clear tagline on the homepage: "Repository automation, running the coding agents you know and love, with strong guardrails in GitHub Actions." — immediately communicates the value proposition.

2. Estimated time on Quick Start: "Estimated time: 10 minutes" is a confidence booster for new users.

3. Walkthrough video in Step 1: A real MP4 demo video (69MB, properly served) embedded in the Quick Start is excellent for visual learners.

4. Prerequisites clearly listed: The prerequisites section covers AI Account, GitHub Repo, Actions, CLI, and OS. Comprehensive and easy to scan.

5. Inline explanation of jargon: When "frontmatter" is used, it's followed by (the YAML configuration block between --- markers at the top of the file) — good inline glossary pattern.

6. No broken links: All 80+ internal links on the Quick Start page return HTTP 200. Zero 404s on navigable content.

7. "Most Common Commands" table: The CLI page leads with a clear summary table of the 7 most-used commands. This is a good UX pattern.

8. Early development warning is visible: The caution note about the project being in early development is present on the homepage, which sets appropriate expectations.

9. Navigation structure: The sidebar is well-organized into Setup, Guides, Patterns, Reference, and Troubleshooting — logical for a technical audience.

Recommendations

Quick Wins 🏃

1. Link "frontmatter" in Quick Start Step 4 to /gh-aw/reference/frontmatter/
2. Explain the githubnext/agentics/daily-repo-status path with one sentence about the workflow registry
3. Add tooltip/badge to search button explaining it's disabled in dev mode
4. Add a one-liner on why COPILOT_GITHUB_TOKEN ≠ GITHUB_TOKEN in Step 2

Medium-term Improvements 🛠️

5. Split CLI Commands page into "Beginner Commands" and "Full Reference" sections
6. Add GitHub Actions enable link in Prerequisites
7. Add a visual "how it works" diagram on the homepage between the tagline and the feature list — the text description is good but a simple flowchart would help complete beginners understand the compile → Actions → result cycle

Longer-term 📚

8. A "What is an agentic workflow?" explainer for users who haven't encountered the term before — a 2-paragraph plain-English intro before the technical details

References: §23947396770

AI generated by Documentation Noob Tester · history

• expires on Apr 4, 2026, 1:28 PM UTC