Skip to content

Feature Request: Include instructions.md files during test execution #239

Description

@ccarrizosaAS

Summary

It would be very useful if Waza included *.instructions.md files as part of the context when executing tests. I've tried to include them as part of the fixture files but unfortunately seems that are ignored by the AI or not understood as instruction files even though I've tried to copy to the .github/instructions/ folder.

Motivation

Many projects use *.instructions.md files to provide additional guidance for contributors, agents, or automation systems. These files often contain:

  • Project-specific conventions
  • Environment setup details
  • Test execution expectations
  • Special constraints or workflows
  • Repository-specific behaviors

Currently, when tests are executed, this contextual information may not be available, which can lead to:

  • Incorrect assumptions during execution
  • Inconsistent behavior across repositories
  • Reduced reproducibility
  • Missing project-specific requirements

Including the *.instructions.md files to the test, gives a more realistic measurement about the agent performance on a real scenario.

Proposed Behavior

Add a new configuration parameter to eval.yml or task.yml files to include provided instruction files to the test case execution environment.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions