Emphasize mandatory dogfooding/user testing for CLI changes (#1760)

Copilot · nathanjmcdougall · web-flow · commit d6f8ba762baf · 2026-04-04T08:37:44.000+13:00
* Initial plan * Emphasize mandatory dogfooding/user testing for CLI changes Add a "Generally Important Instruction" in AGENTS.md requiring dogfooding and user testing for any CLI command changes. Strengthen usethis-cli-modify skill (v1.2 → v1.3) with: - Bold mandatory labels on dogfooding/user testing procedure steps - New "Dogfooding and user testing are mandatory" section explaining why unit tests alone are insufficient - Split "Adding, removing, or renaming" into separate bullets for clearer discoverability when adding new commands Agent-Logs-Url: https://github.com/usethis-python/usethis-python/sessions/fd68fe02-245e-4df0-b122-5a5a38112d9d Co-authored-by: nathanjmcdougall <18602289+nathanjmcdougall@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: nathanjmcdougall <18602289+nathanjmcdougall@users.noreply.github.com>
diff --git a/.agents/skills/usethis-cli-modify/SKILL.md b/.agents/skills/usethis-cli-modify/SKILL.md
@@ -4,7 +4,7 @@ description: Modify the usethis CLI layer (commands, options, help text) and kee
 compatibility: usethis, Python, typer, markdown
 license: MIT
 metadata:
-  version: "1.2"
+  version: "1.3"
 ---
 
 # Modifying the CLI
@@ -14,18 +14,28 @@ metadata:
 1. Make your changes to the CLI layer (command functions, options, help text, or app registration).
 2. Update the CLI documentation to reflect every user-facing change.
 3. Run the affected interface-level tests.
-4. Use the `usethis-cli-dogfood` skill to validate the command against the real repo.
-5. Use the `usethis-cli-user-test` skill to verify the happy path in a fresh temporary project.
+4. **Dogfood the command** — use the `usethis-cli-dogfood` skill to run the command against this repository. This step is mandatory.
+5. **User-test the command** — use the `usethis-cli-user-test` skill to verify the happy path in a fresh temporary project. This step is mandatory.
 
 ## When this skill applies
 
 Use this skill whenever you modify anything under the `_ui` package, including:
 
-- Adding, removing, or renaming a command
+- Adding a new command or subcommand
+- Removing or renaming a command
 - Changing a command's options, arguments, or defaults
 - Changing help text or command descriptions
 - Modifying how commands are registered on the app
 
+## Dogfooding and user testing are mandatory
+
+Steps 4 and 5 are not optional — they are how you verify that your changes actually work for real users. Unit tests alone are not sufficient because they exercise code paths in isolation and cannot catch problems that arise from real project structures, real dependency resolution, or real file system interactions.
+
+- **Dogfooding** (step 4) catches edge cases from a complex real-world project with existing configuration, dependencies, and non-trivial structure.
+- **User testing** (step 5) catches issues with initial setup, missing files, or assumptions about pre-existing configuration that only a mature project would have.
+
+If either test reveals a problem, fix the code, write a regression test, and re-run both tests before considering the change complete.
+
 ## Update the CLI documentation
 
 **Every user-facing CLI change requires a documentation update.** The documentation is manually written and is not auto-generated from the code, so it will not update itself.
diff --git a/AGENTS.md b/AGENTS.md
@@ -483,6 +483,7 @@ External skills can be installed if they are not present — see the `usethis-sk
 
 - ALWAYS check the [Function Reference](#function-reference) section above before implementing any utility logic — mature, tested functions already exist for common operations such as reading dependencies, detecting tools, and printing console output.
 - ALWAYS use possibly relevant agent skills when they are available. Eagerly use skills, if in doubt, assume a skill is relevant.
+- ALWAYS dogfood and user-test CLI changes. After implementing or modifying any CLI command, use the `usethis-cli-dogfood` skill to run the command against this repo and the `usethis-cli-user-test` skill to verify the happy path in a fresh project. Both are mandatory — never skip them, even for seemingly simple changes. Use the `usethis-cli-modify` skill for guidance on the full CLI change workflow.
 - ALWAYS use the `usethis-skills-modify` skill when modifying any agent skill (`SKILL.md` file). Do not edit skill files without it — it enforces version bumping, scope checking, and content quality guidelines. Similarly, ALWAYS use `usethis-skills-create` when creating a new skill.
 - ALWAYS use `find-skills` to research new skill capabilities if there are difficult tasks, tasks in an unfamiliar domain, if you believe there is a lack of clarity or direction around precisely how to proceed, or if you get stuck or find something surprisingly challenging. When using this skill, please be sure to use the `usethis-skills-external-install` skill when deciding to install a new external skill.
 - ALWAYS consider the `usethis-python-test-full-coverage` to be relevant: if your task involves
