Summary
It would be very useful if Waza included *.instructions.md files as part of the context when executing tests. I've tried to include them as part of the fixture files but unfortunately seems that are ignored by the AI or not understood as instruction files even though I've tried to copy to the .github/instructions/ folder.
Motivation
Many projects use *.instructions.md files to provide additional guidance for contributors, agents, or automation systems. These files often contain:
- Project-specific conventions
- Environment setup details
- Test execution expectations
- Special constraints or workflows
- Repository-specific behaviors
Currently, when tests are executed, this contextual information may not be available, which can lead to:
- Incorrect assumptions during execution
- Inconsistent behavior across repositories
- Reduced reproducibility
- Missing project-specific requirements
Including the *.instructions.md files to the test, gives a more realistic measurement about the agent performance on a real scenario.
Proposed Behavior
Add a new configuration parameter to eval.yml or task.yml files to include provided instruction files to the test case execution environment.
Summary
It would be very useful if Waza included
*.instructions.mdfiles as part of the context when executing tests. I've tried to include them as part of the fixture files but unfortunately seems that are ignored by the AI or not understood as instruction files even though I've tried to copy to the.github/instructions/folder.Motivation
Many projects use
*.instructions.mdfiles to provide additional guidance for contributors, agents, or automation systems. These files often contain:Currently, when tests are executed, this contextual information may not be available, which can lead to:
Including the
*.instructions.mdfiles to the test, gives a more realistic measurement about the agent performance on a real scenario.Proposed Behavior
Add a new configuration parameter to
eval.ymlortask.ymlfiles to include provided instruction files to the test case execution environment.