The Ruff for AGENTS.md — deterministic quality scores for the instruction files that drive your AI. Same input, same score, on every machine.

That last badge is Schliff scoring this repo's own AGENTS.md — live, right now: 91.6 · A.

Your AI instruction files silently degrade — and nothing catches it. AGENTS.md is read by Cursor, Codex, Copilot, and Claude Code — one rotting file now quietly degrades four tools. A trigger phrase rots, an edge case slips, the file balloons past its token budget. No error, no red test — just agents that quietly get worse.

Schliff scores AGENTS.md — and the rest of the family (SKILL.md, CLAUDE.md, .cursorrules, system prompts) — against an explicit, versioned rubric. No LLM judge in the critical path. No network. No randomness. A rule engine you can read, pin, and gate CI on.

pip install schliff
schliff score AGENTS.md   # or any SKILL.md / CLAUDE.md / .cursorrules
schliff demo              # no instruction file handy? score a built-in bad one

This is the real, current output of schliff score AGENTS.md on this repo's own AGENTS.md — clone and run it yourself:

schliff v8.5.0

  structure             █████████░   90/100  great
  operational_coverage  ██████████  100/100  perfect
  efficiency            ████████░░   78/100  good
  composability         ████░░░░░░   45/100  poor
  clarity               ██████████  100/100  perfect

  Structural Score  ██████████████████░░  91.6/100  [A]

  Tokens: 837 / 3,000 (ok)
  Format: agents.md (normalized)

No model produced that number. Run it on another laptop and you get 91.6 again. A score you can't reproduce isn't a measurement — it's a vibe.

Every number in this README comes from released schliff==8.5.0 (pip install schliff==8.5.0 to reproduce byte-for-byte). No install? Paste your file into the playground — same engine, with AGENTS.md and SKILL.md tabs.

The SKILL.md for ShieldClaw — a real prompt-injection-defense skill, now archived — is Schliff's reproducible before/after. The fixtures ship in docs/case-studies/shieldclaw/; every number below is the current engine's output, reproducible with schliff score. (For the 27.9 row, score a copy of SKILL-before.md outside that directory — in place, the engine auto-discovers the sibling eval suite.)

Two separate effects, and Schliff refuses to conflate them. Adding the eval suite lifted the measurement ceiling (27.9 → 83.7) — that is coverage, not quality. The quality delta is 83.7 → 93.8, driven by composability 20 → 86 and efficiency 60 → 83 after adding scope boundaries, an I/O contract, and handoffs — structural gaps a linter can't see, caught as a number that was too low.

A second field run, against an external repo (hydra, measured on released schliff==8.4.0, 2026-07-03): 71.0 [C] → 76.5 [B] — edges 82→100, composability 56→81, fix merged upstream (Zandereins/hydra#34). Efficiency deliberately stayed at 47: a ~14k-token file against a 1,000-token budget was an informed decline, not a blind chase of the number. (External repo — not re-runnable from these fixtures.)

Most "AI quality" tools ask another LLM how good your prompt feels — a different answer every run. That makes the score non-reproducible (re-run it, get a different number), un-auditable (the rubric lives in a hidden prompt), and trivially gameable (write for the judge, not the user). You can't gate a release on a number that drifts. Schliff computes how good the file measurably is — the same answer every run.

Deterministic means reproducible and auditable — it does not automatically mean the number is right. Schliff's claim is narrower and checkable: the rubric is open source, every scorer is readable, the weights are a dict, and the case studies above show the score moving with real fixes. If you disagree with the rubric, you can read it and file an issue — you can't do that with a judge prompt.

Config linters tell you whether the file is valid — a list of pass/fail rules. Schliff tells you how good it is — one graded 0–100 score you can gate, diff across commits, and rank.

• Reproducible. The headline composite is computed from a canonical, versioned weight registry. Calibration is off by default, so verify, badge, and the leaderboard return the same score on your laptop and in CI.
• Auditable. Every dimension is a readable scorer in scripts/scoring/. The weights are a dict you can open. There is no hidden judge prompt.
• Anti-gaming, precisely scoped. A dedicated guard layer (guards.py) detects and floors padding, junk fences, platitude farms, and keyword stuffing — worthless text cannot outrank operational text. A plausible lie about your repo is out of reach of any static scorer (see What the score does not measure).
• Zero core dependencies. Core Schliff is stdlib-only and runs on Python ≥ 3.10. (Optional [evolve] / [judge] extras pull in LLM clients for an opt-in smoke-test only — never for scoring.)

Because the number is stable, it does real work: gate pull requests on it in CI, or diff and compare it across commits with the CLI.

An optional LLM judge exists for exploratory work, but it is never part of the deterministic score. The number you gate on is rule-based, end to end.

Full methodology: docs/SCORING.md.

For the SKILL.md family, Schliff runs 8 scorers per file. 7 of them form the headline composite; security (always computed) and runtime (opt-in) are reported as separate signals so a security warning never silently inflates or deflates your quality grade.

The seven headline weights are renormalized to sum to 1.0 — that is the canonical basis.

Schliff does not quietly renormalize across whatever you happened to measure. Unmeasured dimensions contribute 0 and stay in the denominator — so coverage gaps lower your ceiling instead of quietly disappearing. Your score ceiling equals your measurement coverage. Measure 4 of the 7 headline dimensions and your maximum possible score is capped accordingly, with an explicit warning:

ℹ Scored 4/7 dimensions — the score can't exceed 42% until the rest
  are measured. Run /schliff:init to add an eval suite and score:
  triggers, quality, edges.

S ≥ 95 · A ≥ 85 · B ≥ 75 · C ≥ 65 · D ≥ 50 · E ≥ 35 · F < 35

• Structure, not truth. Schliff cannot verify that a documented command exists or runs. A syntactically plausible fabrication — invented-but-real-looking commands in well-formed sections — scores in the S range. This is a documented, test-pinned limit (test_known_limit_plausible_fabrication_scores_high, spec §11).
• Not agent behavior. A high score doesn't prove your agent gets better — validity evidence today is case-study-level (the two dated before/afters above), not benchmark-level.
• Token counts are estimates. stdlib len//4, not a tokenizer.
• Coverage is on you. triggers/quality/edges need an eval suite (/schliff:init) or the ceiling warning caps the score. AGENTS.md has no such gap.
• Informed declines are valid. A low dimension can be a deliberate tradeoff (hydra left efficiency at 47 rather than gut a 14k-token file).

A high Schliff score is necessary, not sufficient.

One engine, five instruction-file formats — each with its own token budget and scorer set:

Format is auto-detected; override with --format (skill, claude, cursor, agents, system-prompt).

Published on the GitHub Marketplace as AGENTS.md Lint (Schliff).

Gate pull requests on instruction-file quality. The action defaults to your repo-root AGENTS.md and posts a scored comment on every PR:

# .github/workflows/agents-lint.yml
name: AGENTS.md Lint
on: [pull_request]
jobs:
  score:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: Zandereins/schliff@v1
        with:
          minimum-score: '75'   # optional: fail the PR below this score

By default it scores AGENTS.md at the repo root; set skill-path: to lint a SKILL.md, CLAUDE.md, or .cursorrules instead. One caveat: the Action installs the latest released engine from PyPI, so after a release its scores can lead an older pinned install; pin it with schliff-version: '8.5.0' in the with: block if you need byte-stable gates.

Prefer not to depend on a third-party action? The dependency-light equivalent:

      - run: pip install schliff
      - run: schliff verify AGENTS.md --min-score 75

schliff verify exits non-zero when the score falls short and works for every supported format — the minimum is scaled by measurement coverage, so SKILL.md files without an eval suite aren't auto-failed. Requires schliff ≥ 8.5.0 for AGENTS.md: older engines scored it under the SKILL profile (#101).

Show your repo's AGENTS.md quality — no setup, no CI, no account:

![AGENTS.md quality](https://img.shields.io/endpoint?url=https%3A%2F%2Fschliff-playground.vercel.app%2Fapi%2Fbadge%3Frepo%3DOWNER%2FREPO)

Replace OWNER/REPO with your repository. The badge is scored live from your AGENTS.md at HEAD; GitHub's image cache (camo) may delay refreshes. Public repos only.

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/Zandereins/schliff
    rev: v8.5.0
    hooks:
      - id: schliff-verify
        args: ['--min-score', '75']

The hook fires on SKILL.md files (its files filter); gate AGENTS.md with the Action or schliff verify AGENTS.md in CI.

schliff <command> [path] [options]

Beyond grading, Schliff can apply fixes. The improvement engine measures first, then fixes (not the other way around):

1. Score the file across all dimensions.
2. Generate deterministic patch gradients for the weakest dimensions.
3. Apply the safe, rule-based patches automatically — ~32% of suggested fixes apply deterministically through the apply gate (confidence=high, single-edit), as measured by the canonical script measure_patch_ratio.py — re-run it to check. The rest are handed to an optional LLM.
4. Re-score and keep the change only if the score improved — otherwise revert.
5. Stop on plateau detection or when the target is reached.

It also carries cross-session episodic memory (episodic_store.py), so improvement runs learn from prior attempts instead of repeating them. Drive it from Claude Code with /schliff:auto, or use schliff evolve directly. This is an optional convenience layer — the deterministic score is the product.

pip install "schliff[evolve,judge]"  # optional LLM extras for this layer only

LLM extras power this optional layer only; they are never used for scoring.

The full methodology — scorer internals, the full-denominator composite, the anti-gaming guards, and the calibration model — lives in docs/SCORING.md.

scripts/
├── cli.py                  # CLI entrypoint
├── scoring/
│   ├── registry.py         # canonical weights, scorer lists, headline exclusions
│   ├── composite.py        # full-denominator composite model
│   ├── formats.py          # format detection + token budgets
│   ├── guards.py           # anti-gaming detection
│   └── structure.py · triggers.py · quality.py · edges.py · …
├── text_gradient.py        # deterministic patch gradients (apply gate)
├── episodic_store.py       # cross-session episodic memory
└── measure_patch_ratio.py  # canonical source for the patch-ratio claim

• Docs: docs/SCORING.md
• Playground: schliff-playground.vercel.app — paste a SKILL.md or AGENTS.md, get a live score (or schliff demo in the CLI). The playground reports the structural score for SKILL.md — the CLI's full 7-dim composite can differ; AGENTS.md has no such gap.
• Leaderboard: schliff-leaderboard.vercel.app
• Case studies: docs/case-studies/

MIT © Franz Paul