1. Filter dead relays first (NIP-66) — only 37% of relay-user pairs in NIP-65 lists point to normal content relays (NIP-11 survey). The rest are offline, paid, restricted, or missing. Removing them stops you wasting connection budget on relays that will never respond (success rate goes from ~30% to ~75%), and each dead relay wastes 15 seconds of timeout. Zero algorithmic changes needed.
2. Add randomness to relay selection — deterministic algorithms (greedy set-cover) pick the same popular relays every time. Those relays prune old events. Stochastic selection discovers relays that keep history. 1.5× better recall at 1 year across 6 profiles.
3. Learn from what relays actually return — no client tracks "did this relay deliver events?" Track it, feed it back into selection. At 7d, recall jumps from ~40% to ~85-92% after 2-3 sessions. At 1yr, gains are limited by relay retention (~3-14pp over 2 profiles without cache; re-benchmarking in progress). (Thompson Sampling).
4. Use EOSE-race for feeds — query 20 relays in parallel, stop 2 seconds after the first one finishes. You'll have 86-99% of your events in under 3 seconds total. Show events as they stream in. (Latency data)

Each technique adds incremental value. You don't need to implement everything at once:

Steps 1a and 1b are alternative entry points — 1a replaces your routing layer, 1b augments it. Step 1b already includes Thompson Sampling (it's the same ~80 LOC). Steps 2-4 are incremental enhancements that apply to the 1a path. †Thompson 7d recall = 85-92% (6-profile mean from HJO benchmark, genuine multi-session data). Thompson 1yr recall is under re-benchmarking — initial 2-profile data (Gato, Telluride, no cache) shows ~27-45%, limited by relay retention at long windows. ††Hybrid 1yr recall is under re-benchmarking. [min–max] ranges show the spread across tested profiles — your recall depends on your follow graph size and relay diversity. All stateless values are 6-profile means. Feed TTFE = time to first event (all algorithms share the same fast relay). "+2s" = EOSE-race grace period; "instant completeness" = all events arrive with first EOSE (1-2 relay setups). 1yr recall is the more informative metric — 7d masks relay retention problems that dominate real-world performance. Latency data from 7 cross-profile benchmarks (194–2,784 follows).

If you're building on an existing library, here's where you stand and what to do next:

Each link goes to a per-client cheat sheet with specific code paths, current behavior, and upgrade recommendations.

Your relay picker optimizes for "who publishes where" on paper, but the relay that should have the event often doesn't — due to retention policies, downtime, silent write failures, or auth restrictions.

25 relay selection algorithms (10 from real clients, 6 experimental-actionable, 7 academic, 2 baselines — plus 2 latency-aware variants), tested against 7 real Nostr profiles (194-2,784 follows), across 6 time windows (7 days to 3 years), with and without NIP-66 liveness filtering. Every algorithm connected to real relays and queried for real events. Latency benchmarks across all 7 profiles measure TTFE, EOSE-race convergence, and profile-view timing.

Full methodology: OUTBOX-REPORT.md | Reproduce results: Benchmark-recreation.md | Produced for nostrability#69

Benchmark data collected February 2026. Relay state changes continuously — relative algorithm rankings should be stable; absolute recall percentages will vary on re-run.

There are two architecturally distinct approaches to outbox routing. Both benefit significantly from Thompson Sampling (85-92% at 7d; 1yr numbers under re-benchmarking), but they differ in where changes land and what tradeoffs they impose.

Full outbox routing — replace your relay selection layer. For each followed author, route queries to their NIP-65 write relays instead of broadcasting to a fixed relay set. This is what Welshman/Coracle, rust-nostr, NDK, and Gossip do.

Hybrid outbox enrichment — keep your existing relay infrastructure for the main feed and add parallel outbox queries only for long-tail paths: profile views, event lookups, and thread traversal. The main feed stays on your app relays (fast, predictable latency). Profile views additionally query the viewed author's write relays. Event lookups use relay hints and author write relays as fallback tiers.

Note: The 1yr Thompson numbers in Section 8.5 were collected with a phase2 cache bug that inflated multi-session recall. The relative comparison (hybrid vs full outbox architecture) is directionally valid, but the absolute 1yr recall numbers need re-benchmarking. The hybrid approach queries fewer outbox relays per author (top 3) but compensates with the app relay safety net. Hybrid converges faster (session 2 vs session 3-4) because the app relay floor provides a strong initial signal.

Decision tree:

Do you have a routing layer that selects relays per-author?
├─ Yes → Add Thompson Sampling to it (Step 4)
│        85-92% 7d recall; 1yr under re-benchmarking
│
└─ No (fixed app relays / broadcast)
   │
   ├─ Can you rewrite your routing layer?
   │  └─ Yes → Implement full outbox (Step 1a → Step 4)
   │           Best recall, but biggest engineering investment
   │
   └─ No, or need to preserve feed latency guarantees?
      └─ Add hybrid outbox (Step 1b)
         ~80 LOC, no routing layer changes; 1yr recall under re-benchmarking
         Profile views: fetch author's kind 10002, query top 3 write relays in parallel
         Event lookups: rank relay hints by Thompson score, NIP-65 fallback
         Thread loading: propagate relay hints from e-tags

The relay that's "best on paper" isn't always the one that delivers events. Greedy set-cover (used by Gossip, Applesauce, Wisp) wins on-paper relay assignments but drops to 16% event recall at 1 year — while algorithms with randomness or per-author diversity reach 24-25%. This is inherent to the algorithm: greedy set-cover is a static, one-shot computation — it picks relays based on declared write lists and never learns whether those relays actually delivered.*

What to do: Track which relays return events. Feed that data back into selection. Thompson Sampling does this with ~80 lines of code on top of Welshman/Coracle's existing algorithm (code below).

*Greedy set-cover solves "which relays cover the most authors?" but the answer doesn't change between sessions. A relay that failed to deliver events last time gets picked again next time if it still covers the most authors on paper. Learning algorithms (Thompson, MAB) update their beliefs after each session.

7d data from HJO benchmark (6 profiles × 5 sessions, genuine multi-session). 1yr Thompson gains were previously reported as +60-70pp but those numbers were inflated by a phase2 cache bug — genuine 1yr gains are under re-benchmarking.

NDK-specific Thompson Sampling results (NDK's priority-based algorithm + Thompson, 5 learning sessions, 1yr, NIP-66 liveness, cap@20):

NDK's priority cascade (selected-first > popularity > lexicographic) limits Thompson's influence — the cascade short-circuits scoring when connected relays satisfy the per-author target. The Priority variant (preserving the cascade) is more stable than the Unified variant (replacing it with a 1.5x bonus). Thompson converges by session 3 for Telluride and session 4 for Gato. In same-benchmark single-run comparisons (S5, same relay state), Welshman+Thompson outperforms NDK+Thompson by ~3-9pp (26.7% vs 24.1% for Gato, 45.4% vs 41.4% for Telluride — these are single-run S5 values, slightly different from the S3-5 averages in the table above). This gap is structural — Welshman's per-user relay budgeting gives Thompson full scoring control, while NDK's cascade constrains it to the third tier.

Note: Small profiles (<200 follows) may see minimal gains — the 20-relay budget already covers most combinations.

NIP-66 publishes relay liveness data. Filtering out dead relays before running any algorithm means you stop wasting connections on relays that will never respond. The benefit is efficiency — fewer wasted slots in your 20-connection budget — not a coverage guarantee. Event recall impact is roughly neutral: stochastic algorithms gain ~+5pp, while Thompson Sampling and Greedy show negligible or slightly negative impact (likely noise from stochastic selection variance and intermittently available relays).

What to do: Fetch NIP-66 monitor data (kind 30166), classify relays as online/offline/dead, exclude dead ones before relay selection (code below).

Relay success rate = % of selected relays that actually respond to queries. This is an efficiency improvement (fewer wasted connections), not necessarily more events retrieved.

Speed impact: Across 10 profiles (4,587 relay queries), NIP-66 pre-filtering reduces feed load time by 45% (40s → 22s). Dead relays each burn a 15-second timeout that blocks a concurrency slot from querying live relays.

Relay list pollution is worse than expected. NIP-11 probing across 36 profiles (13,867 relay-user pairs) shows 46% of relay-user pairs point to relays that won't serve content — offline (34%), paid (7%), restricted (4%), or auth-gated (0.5%). Another 17% lack NIP-11 but are likely functional (NIP-11 is voluntary — ~500 relays don't serve it, per nostr.watch). Nearly half of all unique relay URLs in NIP-65 lists are offline. The most common dead relays (relay.nostr.band, nostr.orangepill.dev, nostr.zbd.gg) appear in 32-34 of 36 tested profiles. Use NIP-66 liveness (WebSocket connectivity) rather than NIP-11 to filter dead relays. See Section 5.3 for the full breakdown.

At 1 year, greedy set-cover gets only 16% event recall. Welshman's stochastic scoring gets 24% — 1.5× better. Filter Decomposition (rust-nostr, deterministic) does even better at 25%. (All 6-profile means.) The winning factor isn't randomness vs determinism — it's relay diversity. Algorithms that give each author their own relay picks (FD's per-author top-N, Welshman's random perturbation) discover small/niche relays that retain events well. Algorithms that concentrate on popular relays (greedy, popularity-weighted) fill the 20-relay budget with the same high-volume relays that prune old events aggressively. FD's median per-author recall (87.5% on ODELL/1,779 follows) vs Welshman's (50.0%) shows the effect: FD gives equitable coverage across authors, while popularity weighting gets high recall for authors on popular relays but zero for authors on niche ones. At 7 days all algorithms cluster at 83-84% — the differences only emerge at longer windows where relay retention diverges. Note: stochastic results have meaningful run-to-run variance (±2–8pp depending on profile size).

What to do: If you use greedy set-cover, switch to per-author relay selection (Filter Decomposition) or stochastic scoring (Welshman). Either way, upgrade to Thompson Sampling for the biggest gains — learning steers toward relays that actually deliver, regardless of popularity.

Feed queries to 20 outbox relays produce the first event in 530-670ms across all tested profiles (194–2,784 follows). The algorithm doesn't matter — TTFE depends on which relay responds fastest, and all algorithms include at least one fast relay. What matters is when to stop waiting for the rest.

The EOSE-race tradeoff. When your fastest relay finishes (sends EOSE), you have a fraction of your total recall. Each additional second of waiting adds more events from slower relays. Across 7 profiles:

Coverage and latency are directly opposed. More relays = more events found, but longer to collect them all. This is the fundamental tradeoff:

Two relays finish instantly but miss half the events. Twenty relays find nearly everything but take 2-5s to converge. Hybrid outbox side-steps this: show app relay events immediately (2-4 relay speed), stream in outbox events in the background (20-relay coverage). The user sees something in <600ms and everything within 2-5s.

Profile-view latency. Querying an author's top 3 NIP-65 write relays for a profile view takes 750-920ms median (96-100% hit rate). This is algorithm-independent — every profile view does the same outbox lookup.

Practical timeout settings:

• EOSE-race grace period: 2s for feeds (86-99% completeness), 5s for archival/search
• Individual relay timeout: 15s (matches most relay EOSE timeouts)
• Profile-view timeout: 3s (covers p95 of 1.7-2.5s)
• Dead relay cost: Each dead relay burns a full 15s timeout. NIP-66 filtering removes 40-66% of dead relays — this is as much a latency optimization as a connection budget one.

Showing late-arriving events. The EOSE-race means your feed renders in <1s but more events arrive over the next 2-5s. Use a "N new posts" banner (like Twitter) to buffer late events without reflowing the user's reading position. For profile views, use shimmer placeholders that resolve as relays respond. See IMPLEMENTATION-GUIDE.md § Showing late-arriving events for visual examples and code.

Latency data from 7 cross-profile benchmarks (194–2,784 follows, 178–1,234 relays). See OUTBOX-REPORT.md § 8.7 for full data.

TTFE (first event) is fast and algorithm-independent — 530-670ms regardless of algorithm. But how fast the full feed populates is improvable. Adding a latency discount to Thompson scoring steers selection toward fast relays, so more events arrive within the first 2 seconds.

The change: Multiply each relay's Thompson score by 1 / (1 + latencyMs / 1000). Learn latencyMs from your own connect+query measurements using an exponential moving average (EWMA, α=0.3). Cold start = no latency data = discount of 1.0 (identical to base Thompson).

// After Thompson scoring, add one line:
const latencyMs = relayStats.get(relay)?.latencyMs;  // EWMA from past queries
const discount = latencyMs !== undefined ? 1 / (1 + latencyMs / 1000) : 1.0;
const score = quality * (1 + Math.log(weight)) * sampleBeta(alpha, beta) * discount;

The discount is hyperbolic, not exponential — a slow-but-reliable relay at 2s still competes (discount 0.33) if it has strong delivery. A fast relay at 200ms gets 0.83. A 5s relay is near-excluded at 0.17.

Cross-profile results (6 profiles × 5 sessions, 7d window, Welshman+Thompson+Latency):

What to do: For apps targeting typical users (< 500 follows), add the latency discount unconditionally — it's a 1-line change with near-zero recall cost. For apps targeting power users (1000+ follows), either make the discount strength tunable or skip it if total recall matters more than feed population speed.

Why learn latency yourself: Your measured connect+query times reflect your users' actual experience. Each relay's performance varies by client location, time of day, and load. The EWMA adapts to this naturally — a relay that gets slow gets deprioritized, one that speeds up gets promoted. Persist the EWMA alongside your Thompson Sampling stats (same DB table, one extra column).

Data: 6 profiles (194–2,795 follows), 5 learning sessions each, 7d window, NIP-66 liveness filtered, cap@20. FD+Thompson+Latency shows the same pattern with ~2× the recall cost — Welshman variant is strictly safer. See OUTBOX-REPORT.md § 8.6 for per-profile tables and session progression.

All algorithms reach within 1-2% of their unlimited ceiling at 20 relays.

What to do: Cap at 20 connections. For the ~3-5% of active follows without relay lists, use fallback strategies (relay hints from tags, indexer queries, hardcoded popular relays).

All deployed client algorithms plus key experimental ones:

Baselines (for comparison, not recommendations):

1yr and 7d recall: 6-profile means from cross-profile benchmarks (Section 8.2 of OUTBOX-REPORT.md). [min–max] ranges show the spread across tested profiles (194–1,779 follows for the 6-profile set; Thompson variants use a 4-profile set up to 2,784 follows) — your recall will land somewhere in this range depending on your follow graph. All testable-reliable authors, 20-connection cap except Direct Mapping. ‡Thompson 1yr multi-session numbers are under re-benchmarking — previous values (84-89%) were inflated by a phase2 cache bug. NDK+Thompson 1yr = 29% [21–38] (2-profile mean, genuine, collected with --no-phase2-cache). Welshman+Thompson 7d = 92% (4-profile mean, Section 8.3, genuine). Welshman+Thompson and FD+Thompson converge within 2-3 sessions. NDK+Thompson converges by session 3-4 (slower due to the priority cascade limiting Thompson's influence) and yields smaller gains (+5-15pp). Stochastic algorithms have additional run-to-run variance on top of the cross-profile range (see variance analysis). Ditto-Mew baseline = 4-profile mean with NIP-66.

**Direct Mapping uses unlimited connections (all declared write relays, typically 50-200+). Its high recall reflects connection count, not algorithmic superiority.

***Primal's low recall may reflect a benchmark methodology limitation (querying a caching aggregator as if it were a standard relay) rather than a definitive measure of aggregator quality. App devs using Primal should test against their own use cases.

Replace random() in Welshman's scoring with sampleBeta(successes, failures) per relay. This keeps the beneficial randomness, adds learning, and is ~80 lines of code. If you use rust-nostr/Filter Decomposition, use FD+Thompson instead — same idea but without the popularity weight, which fits Filter Decomposition's per-author structure better.

// Current Welshman (stateless):
const score = quality * (1 + Math.log(weight)) * Math.random();

// With Thompson Sampling (learns from delivery):
function sampleBeta(a: number, b: number): number {
  const x = gammaVariate(a);
  return x / (x + gammaVariate(b));
}
const alpha = stats.eventsDelivered + 1;  // successes + prior
const beta = stats.eventsExpected - stats.eventsDelivered + 1;  // failures + prior
const score = quality * (1 + Math.log(weight)) * sampleBeta(alpha, beta);

Track per-relay stats in a small table (~100 bytes per relay):

CREATE TABLE relay_stats (
  relay_url TEXT PRIMARY KEY,
  times_selected INTEGER DEFAULT 0,
  events_delivered INTEGER DEFAULT 0,
  events_expected INTEGER DEFAULT 0,
  last_selected_at INTEGER,
  latency_ms REAL           -- optional: EWMA of connect+query time (§5)
);

The full integration loop:

// On app startup:
const relayStats = await db.loadRelayStats(); // {relay → (delivered, expected)}

// When building a feed subscription:
function selectRelays(candidates: RelayCandidate[], budget: number): string[] {
  const scored = candidates.map(r => {
    const stats = relayStats.get(r.url) ?? { delivered: 0, expected: 0 };
    const alpha = stats.delivered + 1;  // Beta prior: successes + 1
    const beta = stats.expected - stats.delivered + 1;  // failures + 1
    return {
      url: r.url,
      score: r.quality * (1 + Math.log(r.weight)) * sampleBeta(alpha, beta)
    };
  });
  return scored.sort((a, b) => b.score - a.score).slice(0, budget).map(r => r.url);
}

// After receiving events, update stats:
function updateStats(selectedRelays: string[], eventsPerRelay: Map<string, number>) {
  for (const relay of selectedRelays) {
    const stats = relayStats.get(relay) ?? { delivered: 0, expected: 0 };
    const delivered = eventsPerRelay.get(relay) ?? 0;
    stats.delivered += delivered > 0 ? 1 : 0;  // binary: did it deliver anything?
    stats.expected += 1;
    relayStats.set(relay, stats);
  }
  db.saveRelayStats(relayStats);  // persist for next session
}

If you use Filter Decomposition (rust-nostr's break_down_filter()), FD+Thompson is a drop-in upgrade: score each author's write relays by sampleBeta(α, β) instead of lexicographic order. No popularity weight — the score is purely learned delivery performance:

// Current Filter Decomposition (stateless):
// Sort write relays lexicographically, take top N
const selected = authorWriteRelays.sort().slice(0, writeLimit);

// With FD+Thompson (learns from delivery):
const scored = authorWriteRelays.map(relay => {
  const stats = relayStats.get(relay) ?? { delivered: 0, expected: 0 };
  const alpha = stats.delivered + 1;
  const beta = stats.expected - stats.delivered + 1;
  return { relay, score: sampleBeta(alpha, beta) };
});
const selected = scored.sort((a, b) => b.score - a.score)
  .slice(0, writeLimit).map(r => r.relay);

Same sampleBeta(), same stats table, same update loop as Thompson Sampling above. The only difference: no (1 + Math.log(weight)) multiplier. This avoids biasing toward popular relays that many authors declare but that prune aggressively — scoring purely from observed delivery.

1yr cross-profile results after 5 learning sessions (cap@20, NIP-66 filtered):

⚠️ Methodology note: The 1yr multi-session numbers below were collected with a phase2 cache bug that inflated S2+ verification recall. Session 1 numbers and the relative comparison (FD vs Welshman variant) are valid, but absolute S2-S5 recall values are inflated. Re-benchmarking in progress. See methodology note.

Both algorithms converge within 2-3 sessions. Welshman+Thompson leads by 5-7pp at all profile sizes after convergence — the popularity weight provides a consistent advantage. See Section 8.4 for the full comparison including session progression.

If your client uses a fixed set of app relays (like Ditto-Mew's 4 relays, or any client with hardcoded relay URLs), you can add outbox routing to profile views and event lookups without changing your feed routing. The app relays handle the main feed. Outbox relays handle long-tail content.

The pattern has three parts:

1. Profile views — when loading a user's profile, fetch their kind 10002 relay list, pick top 3 write relays by Thompson score, and query them in parallel with your app relays:

// Profile feed: app relays (fast path) + outbox relays (parallel enrichment)
async function fetchProfileFeed(nostr, pubkey, filter) {
  // Fast path: query app relays as usual
  const appPromise = nostr.query([filter]);

  // Parallel: look up author's write relays, score, query top 3
  const outboxPromise = fetchOutboxEvents(nostr, pubkey, filter);

  const [appEvents, outboxEvents] = await Promise.all([appPromise, outboxPromise]);

  // Merge and deduplicate
  const seen = new Set();
  const merged = [];
  for (const ev of [...appEvents, ...outboxEvents]) {
    if (!seen.has(ev.id)) { seen.add(ev.id); merged.push(ev); }
  }
  return merged;
}

async function fetchOutboxEvents(nostr, pubkey, feedFilter) {
  const relayList = await nostr.query([{ kinds: [10002], authors: [pubkey], limit: 1 }]);
  if (!relayList.length) return [];

  const writeRelays = relayList[0].tags
    .filter(([name, , marker]) => name === 'r' && marker !== 'read')
    .map(([, url]) => url).filter(Boolean);

  const top3 = scorer.rank(writeRelays).slice(0, 3);
  const events = await nostr.group(top3).query([feedFilter]);

  for (const url of top3) scorer.update(url, events.length > 0);
  scorer.persist();
  return events;
}

2. Event lookups — rank relay hints and NIP-65 write relays by Thompson score before querying:

// Tier 2: sort relay hints by Thompson score
const rankedHints = scorer.rank(relayHints);
const hintEvents = await nostr.group(rankedHints).query(filter);

// Tier 3: author's write relays, top 3 by Thompson
const writeRelays = extractWriteRelays(authorRelayList);
const ranked = scorer.rank(writeRelays);
const events = await nostr.group(ranked.slice(0, 3)).query(filter);

3. Thread traversal — propagate relay hints from e tags so each ancestor lookup uses the hint from the child event:

// NIP-10 e-tag: ["e", <event-id>, <relay-url>, <marker>, <pubkey>]
const parentRef = getParentEventRef(event); // { id, relay?, author? }
// Pass relay hint and author hint to useEvent for the parent lookup
useEvent(parentRef.id, parentRef.relay ? [parentRef.relay] : undefined, parentRef.author);

1yr benchmark results (4-profile mean, cap@20, NIP-66, 5 sessions):

⚠️ Methodology note: These 1yr multi-session numbers were collected with a phase2 cache bug that inflated S2+ verification recall. Re-benchmarking in progress. See methodology note.

‡1yr hybrid recall under re-benchmarking. Converges by session 2. See OUTBOX-REPORT.md § 8.5 for per-profile data and bench/src/algorithms/ditto-outbox.ts for the benchmark implementation.

Fetch relay liveness from NIP-66 monitors and exclude dead relays before running your selection algorithm:

// Fetch NIP-66 monitor data (kind 30166) and classify relays
async function filterLiveRelays(candidateRelays: string[]): Promise<string[]> {
  const monitorEvents = await pool.querySync(
    ["wss://relay.nostr.watch"], // NIP-66 monitor relay
    { kinds: [30166] }
  );
  const alive = new Set(monitorEvents.map(e => e.tags.find(t => t[0] === "d")?.[1]).filter(Boolean));
  return candidateRelays.filter(url => alive.has(url));
}

// Use it before any relay selection algorithm:
const liveRelays = await filterLiveRelays(allDeclaredRelays);
const selected = runYourAlgorithm(liveRelays, budget);

Periodically verify that your outbox relays are actually returning events. When gaps are found, add fallback relays automatically:

async function checkDelivery(author: string, outboxRelays: string[]) {
  const fromOutbox = await queryEvents(outboxRelays, { authors: [author], limit: 50 });
  const fromIndexer = await queryEvents(["wss://relay.nostr.band"], { authors: [author], limit: 50 });
  const missing = fromIndexer.filter(e => !fromOutbox.has(e.id));
  if (missing.length > 5) {
    addFallbackRelay(author, fromIndexer.bestRelay);
  }
}

Prerequisites: Deno v2+

cd bench

# Assignment coverage (fast, no network after initial fetch)
deno task bench <npub_or_hex>

# Event retrieval — connects to real relays
deno task bench <npub_or_hex> --verify

# With NIP-66 liveness filter
deno task bench <npub_or_hex> --verify --nip66-filter liveness

# Specific algorithms
deno task bench <npub_or_hex> --algorithms greedy,welshman,welshman-thompson,mab

# Multi-session Thompson Sampling (5 learning sessions)
bash run-benchmark-batch.sh

Run deno task bench --help for all options. See Benchmark-recreation.md for full reproduction instructions.

All data in this report was collected from a single observer. Relay latency, success rates, and timeout behavior are location-dependent — someone in Brazil, Southeast Asia, or on a VPN will see different numbers. We need benchmark runs from different locations to validate whether findings generalize.

What to run (takes ~30 min, needs Deno v2+ and internet):

cd bench
deno task bench 3bf0c63fcb93463407af97a5e5ee64fa883d107ef9e558472c4eb9aaaefa459d \
  --verify --verify-window 604800 \
  --nip66-filter liveness --no-phase2-cache \
  --output both

What to share: Open an issue with your JSON file from bench/results/, your approximate location (country/region), and connection type (home, VPN, cloud, mobile).

What should vary by location: TTFE, relay success rates, timeout counts, NIP-66 RTT correlation strength (NIP-66 monitors are in specific locations — correlation from your vantage point may be stronger or weaker).

What should be stable: Relative algorithm rankings, relay retention patterns, which algorithms benefit from NIP-66 filtering.

OUTBOX-REPORT.md              Full analysis report (methodology + all data)
IMPLEMENTATION-GUIDE.md       How to implement the recommendations above
Benchmark-recreation.md       Step-by-step reproduction instructions
bench/                        Benchmark tool (Deno/TypeScript)
  main.ts                     CLI entry point
  src/algorithms/             25 algorithm implementations (+2 latency-aware variants)
  src/phase2/                 Event verification + baseline cache
  src/nip66/                  NIP-66 relay liveness filter
  src/relay-scores.ts         Thompson Sampling score persistence
  probe-nip11.ts              NIP-11 relay classification probe
  run-benchmark-batch.sh      Multi-session batch runner
  results/                    JSON benchmark outputs
analysis/
  clients/                    Per-client cheat sheets (6 files)
  cross-client-comparison.md  Cross-client comparison by decision point

The phase2 baseline cache (bench/src/phase2/cache.ts, fixed in schema v2) had a lossy serialization bug: it stored the union of event IDs across all relays but lost per-relay mappings. When loaded in sessions 2+, the full union was assigned to every relay that had events, inflating verification recall. A deterministic algorithm like NDK baseline jumped from ~16% (S1, genuine) to ~96% (S2+, inflated) despite selecting the same relays.

What's affected: All multi-session 1yr/3yr Thompson claims from run-benchmark-batch.sh (Welshman+Thompson, FD+Thompson, Hybrid+Thompson, and the Section 8.3 learning curve table). The batch script did not use --no-phase2-cache.

What's trustworthy:

• Session 1 data — no cache on first session, always genuine
• 7d HJO data — 6 profiles × 5 sessions × 4 Thompson algorithms, genuine multi-session
• NDK+Thompson 1yr — collected with --no-phase2-cache, genuine
• All stateless algorithm numbers — unaffected (no learning, no cache dependency)
• Relative comparisons — the cache bug inflated all algorithms equally, so relative rankings and gaps are directionally valid

Status: Cache code fixed (schema v2 stores per-relay event IDs). Batch script updated to use --no-phase2-cache. Re-benchmarking the full 4-profile × 1yr × 5-session matrix is tracked as follow-up.

• Full Analysis Report — 15-client cross-analysis + complete benchmark data
• Implementation Guide — Detailed recommendations with code examples
• Cross-Client Comparison — How 15 clients make each decision
• Benchmark Recreation — Reproduce all results
• nostrability#69 — Parent issue
• NIP-65 — Relay List Metadata specification
• Building Nostr — Protocol architecture guide (relay routing, content migration, bootstrapping)
• replicatr — Event replication daemon for relay list changes (negentropy sync)