Skip to content

scoring: context-dependent anti-trigger risk assessment #78

Description

@spboyer

Problem

Waza's anti-trigger scoring uses blanket discouragement — any DO NOT USE FOR: gets the same warning. Sensei v1.3.0 uses context-dependent risk based on skill set size:

Context Risk Level
1-5 skills, clear domain boundaries Low — anti-triggers are fine
5-15 skills, some overlap Moderate — keywords start competing
15+ skills, overlapping domains High — keyword contamination is measurable

Expected Behavior

Update the anti-trigger scoring in internal/scoring/scoring.go to:

  • Accept a skill count parameter (or detect from context)
  • Scale the warning severity based on skill set size
  • Low risk → informational note
  • High risk → strong warning about keyword contamination
  • Reference the kvenkatrajan empirical evidence

References

  • spboyer/sensei references/scoring.md — check 4 (anti-trigger detection)
  • kvenkatrajan gist on 24 Azure skills keyword contamination
  • Anthropic skills repo: 4/5 use positive-only routing

Acceptance Criteria

  • Anti-trigger risk scales with skill set size
  • Low/Moderate/High risk levels
  • Tests for each risk level
  • internal/scoring/scoring.go updated

Metadata

Metadata

Assignees

No one assigned

    Labels

    go:needs-researchNeeds investigationsensei-parityParity with spboyer/sensei scoringsquadSquad triage inbox — Lead will assign to a membersquad:basherAssigned to Basher (Tester / QA)squad:linusAssigned to Linus (Backend Developer)

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions