Skip to content

bug(showcase): document-processing and data-analysis cards skip AskUserQuestion #21

@Astro-Han

Description

@Astro-Han

Symptom

On the three Show Case cards (writing / document-processing / data-analysis), clicking a card should trigger the bundled skill's Step 1 "Clarify" and invoke AskUserQuestion to gather inputs before acting.

Observed (local dev, bun --cwd packages/desktop-electron dev):

  • writing-assistant: asks questions reliably
  • document-processing: often skips questions, proceeds with assumed defaults
  • data-analysis: often skips questions, proceeds with assumed defaults

Root cause (hypothesis)

Not a code-level bug. Verified in this session that all three skills have structurally identical prompts:

  • skills/writing-assistant/SKILL.md
  • skills/document-processing/SKILL.md
  • skills/data-analysis/SKILL.md

Each has the same <GATE> block, same 3-step workflow, same Step 1 Clarify section phrased as "ask the user ..." in natural language. None names AskUserQuestion by tool name.

The divergence is LLM-level non-determinism: writing tasks are information-sparse (the model has nothing to default to → must ask), while document and analysis cards arrive with strong implicit framing (Excel / CSV / Word file types) that lets the model confidently skip clarification and proceed directly.

Invocation path (for reference)

  • Card click → session-new-view.tsx start() calls sdk.client.session.create({ skill: name }) then sdk.client.session.command(buildSkillSessionCommandInput(...)) with command: name
  • session.ts routes POST /:sessionID/command to SessionPrompt.command()
  • command/index.ts registers skills as commands with the SKILL.md body as the template (no hints, no $ARGUMENTS substitution)

Suggested fixes (ordered by cost)

  1. Strengthen Step 1 in document/analysis skills: explicitly name the tool. Change "ask the user ..." to "YOU MUST call the AskUserQuestion tool to ask ... before doing anything else". Do not ship without this line; the tool-name mention materially changes behavior. Cost: edit 2 SKILL.md files.
  2. Front-load required-inputs list: add a "Required inputs" bullet list at the top of each skill and make Step 1 say "If any required input is missing, call AskUserQuestion to collect it". This removes the "can infer from file type" shortcut. Cost: edit 2 SKILL.md files.
  3. (If 1+2 still flaky) Add an explicit gate/guard rail in command/index.ts that prepends a tool-directive preamble to skill templates so the skill prompt becomes: [preamble: you MUST call AskUserQuestion first unless all X, Y, Z are provided] + <skill body>.

Out of scope

Acceptance

Clicking document-processing or data-analysis card should reliably (N=5 trials) invoke AskUserQuestion before any tool call against files.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium priorityappApplication behavior and product flowsbugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions