Overview

Create dt-method-08-deep.instructions.md — the on-demand deep instruction file for Method 8: Test and Validate. Loaded explicitly by the coach via read_file when advanced testing expertise is needed. The method-tier file covers test design basics, data analysis patterns, and iteration triggers; this deep file adds advanced multi-variate test design, small-sample statistical analysis, iteration trigger frameworks, deep bias mitigation strategies, and manufacturing-specific testing contexts.

Target File

.github/instructions/dt-method-08-deep.instructions.md

Frontmatter

---
description: 'Deep expertise for Method 8: Test and Validate — advanced test design, small-sample analysis, iteration triggers, and bias mitigation'
applyTo: ''
---

Note: applyTo is empty — this file is loaded on-demand by the coach agent, not auto-loaded by glob.

Required Content

Advanced Test Design (Supports Hat 1: Test Designer)

Beyond basic observational testing:

• Multi-variate testing: Testing multiple variables simultaneously — when it's appropriate (mature prototypes) vs. when to isolate (early validation)
• Contextual inquiry protocols: Testing within the actual work environment, not lab conditions — capturing environmental factors that affect usability
• Diary studies for extended testing: When single-session testing isn't enough — longitudinal approaches for workflow integration testing
• Expert review integration: Combining user testing with expert heuristic evaluation — when each approach adds unique signal
• Accessibility testing patterns: Testing with diverse abilities, assistive technologies, and accessibility standards compliance

Small-Sample Data Analysis (Supports Hat 2: Data Analyst)

Rigorous analysis with typical DT sample sizes (5-15 participants):

• Pattern recognition over statistics: When to use qualitative pattern analysis vs. statistical methods — threshold guidance for sample sizes
• Severity-frequency matrix: Classifying findings by how severe the impact is crossed with how frequently it occurs — prioritization framework for small-N findings
• Triangulation techniques: Combining behavioral observation, verbal feedback, and task completion data — when findings converge vs. diverge
• Saturation detection: Recognizing when additional testing participants are yielding diminishing new insights — a practical stopping rule

Iteration Trigger Frameworks (Supports Hat 3: Iteration Strategist)

Making principled decisions about when to iterate:

• Severity-based routing: Critical findings → immediate iteration; moderate → batch iteration; minor → backlog for future
• Assumption validation scoring: Tracking which initial assumptions were confirmed, challenged, or invalidated — and the iteration implications of each
• Pivot vs. persevere framework: When testing reveals fundamental concept issues vs. refinable execution issues — decision criteria for each path
• Iteration scope management: Determining the right "blast radius" for changes — micro-tweaks vs. significant redesign based on testing signal strength

Deep Bias Mitigation

Extended strategies beyond basic awareness:

• Confirmation bias countermeasures: Structured approaches for seeking disconfirming evidence — devil's advocate protocols, pre-registration of expected outcomes
• Sunk-cost awareness: Recognizing when investment in a prototype direction creates resistance to pivoting despite testing evidence
• Social desirability mitigation: Techniques for reducing participant tendency to give positive feedback — indirect questioning, behavioral vs. stated preference analysis
• Observer effect management: Minimizing the impact of being watched on participant behavior — remote testing, embedded observation, think-aloud timing
• Anchoring bias in analysis: Avoiding letting the first few participants' feedback anchor interpretation of subsequent data

Manufacturing Testing Contexts

From DT4HVE manufacturing expertise:

• Shift-based testing constraints: Testing across multiple shifts with different operators, fatigue levels, and handoff patterns
• Safety-critical testing boundaries: What can be tested with real operators vs. what requires simulation — risk assessment for prototype testing
• Noisy/distraction-rich environments: Testing in actual factory conditions where noise, interruptions, and multitasking are constant
• Multi-role testing: Testing the same prototype with operators, supervisors, maintenance staff, and safety officers — each role has different validation criteria

Token Budget

Target: ~2,000-3,000 tokens (on-demand tier)

How to Build This File

This is an .instructions.md file — use the prompt-builder agent (not task-implementor) for the authoring phase. The prompt-builder includes built-in Prompt Quality Criteria validation and sandbox testing specific to AI artifacts (.instructions.md, .prompt.md, .agent.md, SKILL.md).

Workflow: /task-research → /task-plan → /prompt-build → /task-review
Between each phase, run /clear to reset context.

Phase 1: Research

Source Material:

1. design-thinking-for-hve-capabilities/guidance/08-test-and-validate.md
2. .github/instructions/dt-method-08-testing.instructions.md (already-built method-tier file)

The DT4HVE guidance file lives in the DT4HVE repository. If you don't have local access, ask the user to provide it or use read_file if the repo is cloned nearby.

Steps:

1. Read both source materials above.
2. Read .github/instructions/prompt-builder.instructions.md for authoring standards.
3. Read any existing dt-method-*-deep instruction files for structural precedent.
4. Gather content on advanced test design, small-sample analysis, iteration triggers, bias mitigation, and manufacturing testing contexts.

Starter prompt:

/task-research

Research for dt-method-08-deep.instructions.md (on-demand deep file)

Read the DT4HVE source material at design-thinking-for-hve-capabilities/guidance/08-test-and-validate.md AND the already-built method-tier file at .github/instructions/dt-method-08-testing.instructions.md. Extract advanced/deep-dive content that goes BEYOND the basic method-tier coverage:
- Advanced test design — multi-variate testing, contextual inquiry, diary studies, expert review, accessibility testing
- Small-sample analysis — pattern recognition with 5-15 participants, severity-frequency matrix, triangulation, saturation detection
- Iteration trigger frameworks — severity-based routing, assumption validation scoring, pivot vs. persevere, scope management
- Deep bias mitigation — confirmation, sunk-cost, social desirability, observer effect, anchoring bias
- Manufacturing testing contexts from DT4HVE domain expertise

Also read .github/instructions/prompt-builder.instructions.md for authoring standards and any existing dt-method-*-deep.instructions.md files for structural precedent.

Output: research summary from Phase 1 above

Phase 2: Plan

Steps:

1. Review the research output from Phase 1.
2. Plan the deep instruction file structure — advanced test design, small-sample analysis, iteration triggers, bias mitigation, manufacturing testing.
3. Define section ordering, token allocation, and confirm empty applyTo.

Starter prompt:

/task-plan

Plan for dt-method-08-deep.instructions.md (on-demand deep file)

Using the Phase 1 research output, plan the deep instruction file:

- Advanced test design section — multi-variate testing, contextual inquiry, diary studies, expert review, accessibility testing
- Small-sample data analysis — pattern recognition vs. statistics, severity-frequency matrix, triangulation, saturation detection
- Iteration trigger frameworks — severity routing, assumption scoring, pivot vs. persevere decision criteria
- Deep bias mitigation strategies — confirmation, sunk-cost, social desirability, observer effect, anchoring
- Manufacturing testing contexts — shift-based, safety-critical, noisy environments, multi-role testing
- On-demand loading structure — empty applyTo, loaded via read_file by the coach
- Content must clearly go beyond what the method-tier file already covers
- Section ordering and token budget allocation (~2,000-3,000 tokens)

Output: plan at .copilot-tracking/plans/{date}-dt-method-08-deep-plan.md

Phase 3: Build

Steps:

1. Review the plan from Phase 2.
2. Author the instruction file using /prompt-build.
3. Content supports three hat roles — organize material so the coach can quickly find relevant advanced content.

Starter prompt:

/prompt-build file=.github/instructions/dt-method-08-deep.instructions.md

Build using the plan at .copilot-tracking/plans/{date}-dt-method-08-deep-plan.md.

This is an on-demand deep instruction file for Method 8: Test and Validate. Key authoring notes:
- applyTo is EMPTY — this file is loaded on-demand by the coach, not auto-loaded by glob
- Content provides advanced/deep-dive material beyond the basic method-tier file
- Advanced test design with multi-variate testing, contextual inquiry, diary studies
- Small-sample analysis with pattern recognition, severity-frequency matrix, saturation detection
- Iteration trigger frameworks with severity routing, assumption scoring, pivot vs. persevere
- Deep bias mitigation strategies with structured countermeasures for five bias types
- Manufacturing testing contexts — shift-based, safety-critical, noisy, multi-role
- Writing style: guidance over commands — deep reference material, not procedural steps
- Token budget: ~2,000-3,000 tokens

Phase 4: Review

Steps:

1. Review the built file against prompt-builder standards and the issue requirements.
2. Validate hat coverage, test design depth, bias mitigation quality, manufacturing context, and prompt-builder compliance.

Starter prompt:

/task-review

Review: .github/instructions/dt-method-08-deep.instructions.md

Validate against:
- prompt-builder.instructions.md authoring standards
- Hat coverage — advanced material supports Test Designer, Data Analyst, and Iteration Strategist roles
- Test design depth — advanced multi-variate and contextual approaches, not just basic test plans
- Small-sample analysis quality — practical techniques for 5-15 participant studies
- Bias mitigation depth — structured countermeasures, not just awareness
- Manufacturing testing contexts — practical factory-floor testing constraints
- Empty applyTo in frontmatter (on-demand loading)
- Writing style: guidance over commands
- Token budget: ~2,000-3,000 tokens
- Structural consistency with other deep-tier instruction files

After Review

• Pass: Mark complete.
• Iterate: Address review findings, rebuild, re-review.
• Escalate: If blocked by missing DT4HVE source material or architectural questions, raise to the user.

Authoring Standards

Follow .github/instructions/prompt-builder.instructions.md:

• Empty applyTo: since this is on-demand content
• Writing style: guidance over commands
• Organized by hat affinity to help the coach locate relevant content quickly

Success Criteria

• File created at .github/instructions/dt-method-08-deep.instructions.md
• Frontmatter has empty applyTo: (on-demand loading)
• Advanced test design with multi-variate testing, contextual inquiry, diary studies, expert review, accessibility
• Small-sample data analysis with severity-frequency matrix and saturation detection
• Iteration trigger frameworks with severity routing and pivot vs. persevere criteria
• Deep bias mitigation with structured countermeasures for five bias types
• Manufacturing testing contexts including shift-based, safety-critical, noisy/distraction-rich, multi-role
• Token count within ~2,000-3,000 target
• Passes task-reviewer validation against prompt-builder standards
• Each prompt, instructions, or agent file registered in collections/design-thinking.collection.yml with path and kind fields
• Each prompt, instructions, or agent file registered in collections/hve-core-all.collection.yml with path and kind fields
• npm run plugin:generate succeeds after collection manifest updates