sentinel

Paste vendor URLs, run an audit. Sentinel extracts and checks every marketing claim automatically.
Live dashboard showing market inflation score, autonomous watch loop, and real-time audit activity feed.
Vendors ranked by substantiation score. Every claim tagged: publicly substantiated, self-reported, or no receipt.
Drill into any vendor to see each claim, its verdict, and the exact web sources behind it.
Sentinel auto-generates procurement questions an agent should ask before trusting any unsubstantiated claim.

Inspiration

Software is increasingly bought by agents, not people. An AI assistant picking a support tool, a vector database, or an SDR platform reads the vendor's marketing page and takes it at face value — because that page is the only machine-readable thing on offer.

Marketing copy is written to persuade humans. "Best-in-class." "Trusted by thousands." "10x faster." None of it is substantiated, and an agent has no way to tell a claim backed by a public receipt from one invented in a growth meeting. Marketing inflated for humans is invisible to agents — and that gap is about to get expensive, because agents are starting to spend real money on what those pages tell them.

We wanted to build the missing layer: a system that treats every vendor claim as a burden of proof, hunts for public evidence, renders a verdict an agent can cite, and keeps doing it without a human in the loop. One principle anchored the whole project: we measure public substantiation, never truth. Every verdict is SUPPORTED, SELF_REPORTED_ONLY, or NO_PUBLIC_RECEIPT_FOUND, surfaced as Publicly substantiated / Self-reported only / No public receipt. We never say a vendor is lying. We say whether the open web backs them up.

What it does

Sentinel watches the marketing pages of every vendor in a category. When a claim changes, it re-audits that vendor against public evidence on a self-improving inference cascade, publishes the structured result to cited.md where other agents can read it, and charges those agents per fetch via x402.

The audit pipeline is a chain of specialized stages: ingest / extract pull the page and lift out atomic, checkable claims; hunt searches the open web (Tavily) for corroborating receipts; judge assigns a verdict and confidence score per claim; advise / scoring rolls the verdicts into a per-vendor substantiation score and a category-wide claim inflation index — the hero number on the dashboard. The inflation index is the gap between what the category asserts and what it can actually prove.

How we built it

Architecture. A FastAPI backend runs the pipeline and an asyncio watch loop that ticks every WATCH_INTERVAL_S, SHA-256-diffs each vendor page against the last snapshot, and fires an autonomous re-audit on any change. A React + Vite frontend renders a dark liquid-glass dashboard whose status strip polls /sentinel/status and whose activity feed subscribes to /activity/stream over SSE — so the entire autonomous loop is visible in real time. Motion only fires on real events; nothing on the screen is decorative.

The inference cascade. Cost was a first-class design constraint. Every audit starts on a cheap tier and only escalates to a premium model when confidence falls below threshold. The cheap client resolves through a priority chain — TrueFoundry gateway -> Pioneer adaptive tier -> local stand-in. On every escalation, Sentinel fires a fire-and-forget {claim, cheap_verdict, premium_verdict} feedback POST so the cheap tier can learn from the premium tier's corrections. That is the self-improving part: the cascade gets cheaper over time as the small model absorbs where the big model overruled it.

Env-gated integrations. Only ANTHROPIC_API_KEY and TAVILY_API_KEY are required to start. Every later integration — Pioneer, TrueFoundry, Senso, x402, ClickHouse, Composio, Thesys — stays silently disabled until both its key and config variable land. Publish and notify seams still emit skipped:no_key activity events when a key is absent, so the feed renders the full loop end-to-end even on a laptop with nothing configured.

Attribution. The audit engine (ingest/extract/hunt/judge/advise pipeline, cascade routing, scoring, leaderboard UI) was adapted from our prior project Receipts, built June 10. Everything in the autonomy, inference cascade, publish, and payment layers was built today at Harness Engineering Hack (June 12, 2026 · AWS Builder Loft SF).

Challenges we ran into

Real autonomy needs a controllable stunt. To demo "claim changes -> Sentinel notices -> re-audits -> republishes" live, we could not wait for a real vendor to edit their site. We built a fictional, controllable test vendor at /test-vendor/nimbus we can mutate on stage to trigger the autonomous re-audit on demand — without ever theatrically editing a real company's score.

Idempotency under a watch loop. A loop that re-audits on every tick will happily publish duplicates. Hash-keying audits and caching by content was essential to keep cited.md clean — an unchanged page costs nothing and republishes nothing.

External schema gaps. The cited.md publish path is fully built, but it is parked on a geo_question_id schema requirement in the publisher's onboarding flow that we deliberately chose not to force during the hack. Knowing when not to ship a half-integration was its own discipline.

Scope under a clock. Deciding what to wire as a seam — ready the moment a key lands — versus what to fully ship was the constant trade across every dispatch.

Accomplishments that we're proud of

"Substantiation, not truth" is a feature, not a hedge. Once we stopped trying to adjudicate whether a vendor was right and only measured whether the public web backed them up, the whole product sharpened. Verdicts became defensible, the vocabulary got disciplined (Verified / Unverified / Unsupported / No evidence are all banned from the UI), and the legal surface shrank to nearly nothing.

A confidence-gated cascade is most of the cost story. Routing easy claims to a small model and reserving the premium model for genuine ambiguity did more for unit economics than any single model swap — and the feedback loop means that ratio improves with use.

Designing for "key-absent" makes a system honest. Forcing every integration to degrade gracefully to a visible skipped:no_key event meant we were never faking the demo. The loop you see is the loop that runs.

The money moment is a paywall, not a pitch. A buyer agent fetching a verdict through an x402 paywall is worth more than any slide.

What we learned

Machine-readable trust is a different product than human-readable credibility. The verdict vocabulary, the evidence links, the substantiation score — all of it had to be designed for a consumer that cannot be charmed by prose. That constraint made everything cleaner.

A real-time activity feed, where every pipeline event animates in as it happens, is the most effective demo for an autonomous system. Judges do not want to hear that the loop is running. They want to watch it.

What's next for Sentinel

Close the loop end-to-end: ship the x402 paywall on the verdict endpoint and the buyer agent that pays for a fetch — the full money moment — then turn on the ClickHouse sink for the 49-run backfill and the Composio claim-change alerts that are already seamed in. Longer term: a staked dispute layer where vendors submit counter-evidence with USDC on the line, and post-purchase agent reviews that close the loop between what vendors claim and what deployed agents actually observe.

The vision is simple: every claim on the agentic web, carrying its own receipt, priced per read.