Edge billing cutover#349
Merged
Merged
Conversation
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
motatoes
approved these changes
Jun 3, 2026
This was referenced Jun 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Edge-first Pro billing (shadow-mode, behind flags)
What & why
Moves Pro billing off per-cell Postgres onto the global edge (D1 + Workers), so cells become
stateless-for-billing and billing is a single global concern. Free billing already runs on the
edge; this ports the Pro path (scale-event → Stripe meter pipeline).
Everything runs in shadow mode behind flags — no change to what anyone is billed until an
explicit, documented cutover. The cell stays authoritative throughout the soak.
What's included
usage_tickerstampsmemory_mb/cpu_countonusage_tick.events-ingest) — writes pro ticks to D1usage_samples; resolves planauthoritatively from D1, not the stale cell-PG value on the envelope.
billing-rollup, new cron Worker) — aggregatesusage_samples→usage_meter_events→ Stripe meter events.billing_mode-aware (legacy per-tier / unifiedflat), idempotent (deterministic ids + Stripe identifier).
SHADOWgates the send;BILL_FROMgates the cutover boundary.the subscription itself: all catalog prices + $30 credit, idempotent. The price catalog is
published to D1
billing_pricesby a one-shot, globalcmd/ensure-products.PRO_BILLING_AUTHORITY=cell|edge(CP) gates the three cell billers(
usage_reporter/allocator/sender) and requires cap-tokens for creates (no edge bypass).usage-paritychecker (CP) +/internal/usage-parity(edge) diff edge vs cellGB-seconds during the soak.
effectivePlanresolves cap-token → D1org-policy→ cell-PG.Flags (all default to no-op / shadow)
SHADOWtruePRO_BILLING_AUTHORITYcelledgeBILL_FROM0OPENSANDBOX_USAGE_PARITY_URL<edge-host>/internal/usage-parity)Data flow
worker ticks → events-ingest → D1 usage_samples → billing-rollup → usage_meter_events → Stripe; parity checker diffs edge vs cellsandbox_scale_events.Stripe webhook
The webhook terminates on the edge (
<edge-host>/webhooks/stripe; prod is alreadyhttps://app.opencomputer.dev/webhooks/stripe). Keep a single endpoint (the edge) — the cellalso has a
/webhooks/striperoute but must NOT be registered, or you'd double-provision.The edge handler needs these events subscribed:
checkout.session.completed— required; the setup-checkout completion drivesprovisioning +
mark-pro.customer.subscription.deleted— required; downgrade →mark-free.customer.subscription.created— optional (redundantmark-pro); safe to include.All other event types are ack'd and ignored.
Validated end-to-end on dev (Stripe test mode)
PRO_BILLING_AUTHORITYboth modes (billers start/stop; direct API-key create → 401 in edge mode).BILL_FROMboundary (below-T consumed-not-billed, ≥T billed).Deploy & rollout safety
No deploy-ordering dependency between worker/CP/edge (each is backward- and forward-compatible:
old ticks lack dims →
usage_sampleswritten as0; new ticks are a superset old consumersignore). All money-moving paths default to off. Per component:
events-ingest,billing-rollup— no-op on deploy. Gated bySHADOW=true+PRO_BILLING_AUTHORITY=cell. Live-but-harmless effects: shadow dataaccumulates in D1 (no cleanup yet), and plan is now read authoritatively from D1 (correctness,
not an amount change).
api-edge— NOT a no-op on deploy. Prod Stripe already points at the edge, sodeploying adds
provisionProSubscriptionto the live webhook. Hard rule:api-edgedeploy in an env may need theSandboxWsGatewayDO migration-ledgerreconciliation (delete-class), independent of the above.
Cutover sequence (per environment, after parity is clean ≥24h)
Stripe Dashboard;
ensure-productshas published the catalog to that env's D1billing_prices;the edge
/webhooks/stripeis the single registered endpoint with the events above.CAPACITY_ALLOCATOR_SETTLE=1m, restart CP, wait forthe allocator to settle + the sender to flush — confirm no
pendingbillable_eventsand nounbilled buckets
< T.BILL_FROM=T+SHADOW=falseon billing-rollup, deploy → edge bills[T, ..).PRO_BILLING_AUTHORITY=edge, restart → cell billers stop, cap-tokens required.across the seam (
/runon billing-rollup echoescfg.billFromfor confirmation).SHADOW=true,PRO_BILLING_AUTHORITY=cell) + restart. Already-sentStripe meter events can't be un-sent — a post-cutover over-bill needs manual credit adjustments,
which is why the shadow soak exists. Don't shorten it.