Migrated from spboyer/waza#387
Summary
Design how eval.yaml can reference remote graders via Go-module-style paths.
Concept
graders:
- ref: github.com/waza-evals/fact@v1.0
- ref: github.com/myorg/custom-grader@v2.1
config:
threshold: 0.8
Design Questions
- How does
waza get resolve and cache remote graders?
- Version pinning semantics (exact, range, latest)?
- Offline/air-gapped fallback from local cache?
- How to handle transitive dependencies (grader A depends on dataset B)?
- Authentication for private repos?
Summary
Design how eval.yaml can reference remote graders via Go-module-style paths.
Concept
Design Questions
waza getresolve and cache remote graders?Parent: #385