v0.6.58.0 ci: release.yml pnpm + build.yml --publish never + 4 Copilot fixes#352
Conversation
… Docker validation matrix
Before this change a non-technical user had to install Docker Desktop
(~700MB, EULA, "open it once" gotcha) and a 9.6GB Ollama model just to
run the default install. Now the only prerequisites are Node 20+ and
pnpm — both installed automatically by bin/skytwin-install.
- Native CockroachDB binary (hash-verified against published .sha256sum)
installed into ~/.local/share/skytwin/bin/cockroach. New bin/skytwin-db
control surface. Docker stays supported as an opt-in via
SKYTWIN_USE_DOCKER=true.
- Electron desktop app bundles per-platform CRDB binaries (darwin
arm64/amd64, linux amd64/arm64, win amd64) via electron-builder
extraResources. New CockroachManager spawns the bundled binary against
app.getPath('userData')/crdb-data.
- Embedded llama.cpp becomes the default LLM fallback when both binary
and model are present (gate fixed — old version added the provider on
binary alone, breaking dev machines with Homebrew llama-cli but no
model).
- Docker validation harness (bin/validate-installs ubuntu|debian|fedora)
drives install.sh end-to-end in fresh containers and asserts
localhost:3200 responds. Caught a pre-existing migration bug
(migration 055 used `do` as a CRDB-reserved table alias) that fresh
installs hit every time.
- GitHub Actions workflow .github/workflows/install-validation.yml runs
the matrix on every PR that touches the install pipeline.
Tasks: ubuntu PASS, debian PASS, fedora PASS. All 678 API tests + 173
desktop tests pass. README and CHANGELOG updated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…nding, graceful drain Addresses findings from the in-PR /review pass against the original v0.6.56.0 commit. Critical fixes: - **Single-instance lock in Electron main.** Without app.requestSingleInstanceLock(), a second launch (double-click in dock, login-item + manual click) raced CockroachManager.start() against the running instance — both saw port-not-bound, both spawned `cockroach start-single-node` against the same data dir, the loser hit CRDB's LOCK file with a cryptic error and the user saw no UI feedback. - **Bind 127.0.0.1 by default, not 'localhost'.** CRDB runs --insecure here; on systems whose /etc/hosts maps localhost to the IPv6 unspecified address (::), the previous default would have broadcast the cluster to the LAN. - **bin/skytwin-db tmpdir cleanup via EXIT trap.** Failed downloads, sha-mismatch errors, and "could not locate cockroach binary" all previously leaked ~70MB /tmp files. Trap fires on every exit path now. High-impact fixes: - **electron-builder extraResources dedup.** Old config shipped all 5 platforms' CRDB binaries (~700MB) inside every artifact. Per-platform mac/win/linux blocks now ship only the host arch's binary. - **CRDB graceful drain via `cockroach node drain` then SIGTERM with 30s timeout.** Previous 5s SIGKILL would have corrupted WAL mid-flush. - **bin/skytwin-db honors XDG_DATA_HOME.** Falls back to ~/.local/share/skytwin per spec when unset. - **SKYTWIN_DB_BINARY_URL_BASE allowlist** (https-only, normal-looking hostname). Stops SSRF / file:// / ftp:// override attempts. SHA-256 verify is still the real defense; this is belt-and-suspenders. - **Per-service logs to $ROOT/.logs/ instead of /tmp/.** systemd PrivateTmp=yes and tmpfiles.d cleanup were wiping the exact logs needed to debug a failed install attempt. - **find -perm portability.** Old `-perm -u+x` is GNU-only; BSD find on macOS rejects the syntax and emits nothing through the pipe, leading to a confusing "Could not locate cockroach binary" on Apple Silicon. Tests: - 6 cockroach-manager tests pass (added one pinning the 127.0.0.1 default) - 174/197 desktop tests pass (24 unrelated skipped) - 678/702 API tests pass (24 unrelated skipped) - Ubuntu Docker validation: PASS (re-ran with all hardenings) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot review #1: waitForReady's timeout error said "Check logs in ${dataDir}/logs" but `cockroach start-single-node` wasn't being invoked with --log-dir, so CRDB defaulted to a platform-dependent location the user couldn't find from the error message. Now we pass --log-dir and mkdir it ahead of time; the error message and reality match. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… (post-Copilot) Copilot review comment #2 (cockroach-manager.ts:102-110): the previous `isReady()` accepted ANY TCP listener on port 26257 as proof CRDB was up, and the start path skipped `ensureDatabase()` when the port was already bound. Two real failure modes: 1. A non-CRDB process binds 26257 first (test leftover, port collision, unrelated tool). CockroachManager treats it as "already running," never spawns CRDB, the API silently connects to the wrong service. 2. A partial first run left CRDB running but missed the CREATE DATABASE step (e.g. crash between start and ensureDatabase). The next launch sees the port bound, returns early, the API dies with "database skytwin does not exist." New behavior: - `portListening()` is the cheap TCP check. - `isCrdbResponding()` confirms the listener is actually CRDB by running `cockroach sql -e 'SELECT 1'` (2s timeout). Only this verdict is trusted as "running." - `start()` always calls `ensureDatabase()` even when CRDB is already responding — covers the partial-run heal path. Tests unchanged; the new helper is a private detail. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…h v0.6.56.0 install Post-merge-gate /document-release pass. The codebase changed from "docker-compose up -d cockroachdb" to "bin/skytwin-db install && start && ensure-db" but three docs still had the old instructions: - CONTRIBUTING.md: Getting Started step 3 now uses bin/skytwin-db, with a pointer to bin/validate-installs for fresh-install regression testing before opening a PR. - docs/cockroach-architecture.md: new "Native binary" section as the default; Docker Compose kept as a legacy/opt-in subsection. - docs/technical-spec.md: Getting Started uses bin/skytwin-db; admin UI now documented at 127.0.0.1:26258 (native path) with 8080 noted as the legacy Docker default; DATABASE_URL example updated to 127.0.0.1 with a one-line explanation of why we avoid 'localhost' under --insecure. No code changes. CHANGELOG entry already covers the underlying behavior; this is pure doc-drift reconciliation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per CLAUDE.md convention — keeps the original-cut vs review-caught diff readable in the release notes without forcing readers into git log spelunking. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ogle PKCE
The v0.6.56 desktop bundle technically launched but every Gmail/Calendar
query 500'd because none of its 57 migrations had actually applied. Root
cause: packages/db ignored DATABASE_URL and connected to the default
localhost:26257 — migrations landed on whatever stray docker-compose
CRDB happened to be on that port instead of the bundled one. Independently,
end users still hit a "create your own Google Cloud OAuth app" wall on
sign-in. This release closes both.
DATABASE_URL routing
- packages/db/src/connection.ts parses DATABASE_URL first, with
DATABASE_HOST/PORT/NAME as legacy fallback. Re-evaluates on the
first getPool() call so service-manager's env injection
(Electron main runs migrations in-process) takes effect.
Migration cascade
- 023 split into 023 (column add, always safe) + 057 (FK-chain
dedupe + unique index, runs after the full schema is in place).
Earlier in-23 dedupe failed because it referenced
decision_outcomes.execution_plan_id from migration 055.
- 046 replaces crdb_internal.force_error() with SELECT 1/0 WHERE …;
bundled CRDB v23.2 locks crdb_internal behind allow_unsafe_internals.
Desktop bundle assembly
- ServiceManager runs migrations in-process via the named up() export
instead of spawning child node (defeats 001-initial.ts's CLI guard,
asar visibility, and ESM-from-CJS dynamic-import quirks all at once).
- pnpm deploy --prod for self-contained api/worker/web bundles
(~45 MB each vs ~14 GB from a naive cp -RL of pnpm symlinks).
- apps/web Express server spawned alongside api + worker — previous
bundle returned ECONNREFUSED on localhost:3200.
- Per-installation SESSION_SECRET auto-generated in Electron main,
persisted at userData/secrets/session-secret (mode 0o600).
- USE_MOCK_IRONCLAW defaults to true in the bundle.
- vitest excludes apps/desktop/dist-electron/ so packaged-app test
copies don't break the suite.
- packages/db/package.json's build script copies *.sql to dist/.
- apps/desktop/dist-electron/ added to .gitignore.
Google OAuth PKCE
- @skytwin/connectors: new generatePkcePair() (RFC 7636 §4),
generateAuthUrl() accepts code_challenge, exchangeCode() sends
code_verifier instead of client_secret when secret is empty,
refreshAccessToken() omits client_secret on refresh in PKCE mode.
- apps/api/src/routes/oauth.ts: server-local Map<state,codeVerifier>
keeps the verifier off the Google round-trip (consume-on-read so a
replayed callback can't redeem twice). Honors a bundle-default
SKYTWIN_DEFAULT_GOOGLE_CLIENT_ID so end users skip the "paste your
client_id+secret" Setup screen.
- apps/desktop/src/service-manager.ts injects the build-time client_id
into the spawned API. The constant ships empty in this commit —
register a Verified OAuth client of type "Desktop app" in the
SkyTwin Google Cloud project and bake the client_id in (or pass at
build time) before the first signed release.
Tests
- 11 new tests in packages/connectors/src/__tests__/google-oauth-pkce.test.ts
covering pair generation, S256 challenge derivation, URL params, and
both token-exchange + refresh request shapes in PKCE vs confidential
modes.
- Full suite still green: 3,084 tests across 20+ packages.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The @skytwin/db build's `tsc && bash -c 'mkdir -p … && cp …'` failed on
Windows CI runners: cmd.exe can't run `bash -c` natively, and even with
Git's bash on PATH the `2>/dev/null || true` segment was parsed as
"'true'' is not recognized as an internal or external command". Replaced
with packages/db/scripts/copy-sql.cjs — pure Node, no shell — which
walks src/{migrations,schemas} and copies the *.sql files into dist/.
Same observable behaviour on macOS/Linux (56 migration files + 1 schema
file in dist/migrations and dist/schemas), now also working on Windows
where Desktop — Windows (NSIS installer) was failing the @skytwin/db
build step.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaced the empty BUNDLED_GOOGLE_CLIENT_ID placeholder in apps/desktop/src/service-manager.ts with the real client_id from the "SkyTwin Desktop" OAuth client (type: Desktop app) registered in the skytwin-492700 Google Cloud project on 2026-05-22. End users now click "Sign in with Google" in the dashboard and get straight to Google's consent screen — no more "create your own Google Cloud OAuth app and paste your client_id + secret" friction. PKCE binds each auth code to a per-flow code_verifier that the API holds in memory (see apps/api/src/routes/oauth.ts), so the public client_id alone redeems nothing. The redirect lands on http://127.0.0.1:<port>/api/oauth/google/callback and never traverses our infrastructure. Tokens stay on the user's machine, encrypted by credential-vault. Override at build time via SKYTWIN_DEFAULT_GOOGLE_CLIENT_ID env if shipping a forked SkyTwin build that should consent under a different brand. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous CHANGELOG entry for v0.6.57.0 said the bundled client_id was empty and needed to be filled in before release. It's now populated with the real value from the "SkyTwin Desktop" OAuth client registered in skytwin-492700 on 2026-05-22. Updated the changelog so the historical record matches what actually shipped. Also notes the consent-screen state: Testing mode pending Google verification for Gmail/Calendar sensitive scopes — listed test users sign in cleanly; other users see the "unverified app" warning until verification completes (separate effort). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the public-facing pages Google requires for OAuth brand
verification + sensitive-scope review:
docs/index.html Homepage describing SkyTwin's functionality,
the Google scopes we request, and why.
docs/privacy.html Privacy policy disclosing how Google user data
is accessed, used, stored (locally), and the
Limited Use compliance statement.
docs/terms.html Apache-2.0-aligned terms of service.
docs/_config.yml Jekyll config that excludes the existing
technical-spec markdown from being served as
site pages (they're written for GitHub
rendering and would break as Jekyll output).
docs/google-verification.md
Status tracker + ready-to-paste scope
justifications for the OAuth consent-screen
review. Documents the three-tier verification
path: brand verification (days),
sensitive-scope review (weeks), restricted-scope
security assessment (months + $$$).
Hosted at https://jayzalowitz.github.io/skytwin/ once GitHub Pages is
enabled on this repo's `docs/` folder. github.io is auto-verified by
Google's brand-verification checks, so no Search Console dance.
This commit only ships the content. Wiring the consent-screen
homepage/privacy URLs and publishing the app are manual steps in the
Google Cloud Console + GitHub Pages settings — see
docs/google-verification.md for the punch list.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A docs-only push to a feature branch skips the desktop/mobile jobs because of the path filter on changes, but we still want a way to re-trigger them against the cumulative branch state — for example, when an earlier desktop-touching commit's run was cancelled by a subsequent docs commit (cancel-in-progress concurrency). Manual dispatch via gh workflow run is the lightest-weight escape hatch. No behaviour change for push / pull_request triggers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Solves the restricted-scope verification problem at $0 cost. Until
SkyTwin can fund a ~$15k–$50k annual CASA Tier 2/3 security assessment
for the bundled OAuth client, Gmail's restricted scopes are reserved
for users who paste their own Google Cloud OAuth credentials into the
Setup page. Calendar + identity flow through the bundled client and
clear with normal sensitive-scope app review (days–weeks, no fee).
What changed in apps/api/src/routes/oauth.ts:
- resolveGoogleConfig() now reports a `source` field: 'user-supplied'
(env vars or DB-stored from Setup), 'bundled'
(SKYTWIN_DEFAULT_GOOGLE_CLIENT_ID), or 'unset'.
- New resolveRequestedScopes({source, includeGmail}) computes the
scope set + returns a `skipped` list reporting capabilities that
were silently dropped. The bundled source drops Gmail; the
user-supplied source allows it.
- /google/authorize honors ?include=gmail (also accepts ?scopes=gmail
and ?gmail=true); requests dropped under the bundled gate return
HTTP 412 with code GMAIL_REQUIRES_BYO_CLIENT and a help URL
pointing at the user-facing walkthrough at /connect-gmail.
- 6 new tests in oauth-scope-tiers.test.ts lock in the gating across
every (source, includeGmail) combination.
What changed in docs/:
- docs/google-verification.md rewritten end-to-end as the staged
rollout plan: brand verification status, sensitive-scope review
for Calendar, restricted-scope tier for Gmail with the BYO escape
hatch, scope justifications ready to paste into Google's
submission form, demo-video script, and the issue draft for #351.
- docs/connect-gmail.html — five-minute step-by-step walkthrough
(create GCP project → enable Gmail API → configure consent screen
→ create OAuth client → paste into SkyTwin Setup). Linked from
docs/index.html.
What needs to happen separately:
- PR #350 merges → GitHub Pages goes live → brand verification can
be submitted.
- Calendar-scope review (Tier 1) — submit through the GCP console
once Pages serves the privacy policy URL.
- Restricted-scope verification for Gmail — tracked in #351, only
when SkyTwin can sustain the annual CASA fee.
Tests: 684 api tests passing including the 6 new tier-gating ones.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reframes Gmail BYO as the launch Gmail experience (not a fallback) and
ships the matching in-app guided wizard, so users don't have to leave
the dashboard to wire up Gmail features that are core SkyTwin value.
What landed
- apps/web/public/js/pages/connect-gmail.js — 5-step wizard with
progress dots, per-step deep links into GCP Console (open in the
user's existing browser session — SkyTwin never sees Google
credentials), final paste-and-connect form that PUTs to
/api/credentials/google and redirects through
/api/oauth/google/authorize?include=gmail&userId=… so the
consent dance happens against the user's just-saved client.
- apps/web/public/js/app.js route '/connect-gmail' → renderConnectGmail.
- Singleton-delegator click handler wired with the _listenerWired
guard + hash gating, matching the CLAUDE.md frontend convention.
- Client-side validation catches the common "you pasted the wrong
thing" cases before the server roundtrip (Client ID must end in
`.apps.googleusercontent.com`, Client Secret length check).
- Step 5 pre-fills any previously-saved creds from
/api/credentials/google so a partial setup survives a refresh.
Reframing — Gmail BYO is the product, not a workaround
- docs/connect-gmail.html: rewritten intro to clarify "this is how
every SkyTwin user wires up Gmail today" (not "5-minute setup,
one-time" — that read as optional).
- docs/google-verification.md: Tier 2 section now states "this is
the launch Gmail experience, not a fallback" and links the
in-app wizard alongside the public-web mirror.
- CHANGELOG: same reframe; mentions the wizard explicitly.
Help-URL routing
- The 412 `GMAIL_REQUIRES_BYO_CLIENT` response from
/api/oauth/google/authorize?include=gmail now carries both
`help: '#/connect-gmail'` (in-app SPA route) and `docs:
'https://jayzalowitz.github.io/skytwin/connect-gmail.html'`
(public-web mirror) so callers in either context can route the
user to the right surface.
Why this and not a SkyTwin-driven embedded BrowserWindow
Earlier design sketch had SkyTwin opening child BrowserWindows
driving the GCP Console with a sidebar walkthrough. Killed on
reflection: it would require the user to sign into Google inside a
SkyTwin-managed browser, which captures the session and credentials
in SkyTwin's process memory — exactly the threat model BYO is
designed to avoid. The wizard now opens each GCP Console URL in the
user's *own* default browser (`target="_blank"` in dashboard
context; Electron's setWindowOpenHandler routes the same anchor to
the OS browser in desktop context — already wired via
apps/desktop/src/main.ts's open-external IPC). User clicks the
Google buttons themselves; SkyTwin only navigates the wizard
forward.
Tests
- 6 scope-tier tests still passing (resolveRequestedScopes contract
unchanged).
- Full api suite green (684 passing, 24 skipped).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two pieces of forward-pushing work: 1. docs/launch-plan.md — the real meaningful roadmap from "code in a feature branch" to "grandma can download the app." Three tiers: Tier 1 = launch blockers (PR merge, brand verification, code signing, demo video, release tag, README rewrite). Tier 2 = first- month polish (auto-update, PKCE store in DB, onboarding deep-link, sample-profile polish). Tier 3 = strategic / post-launch (CASA assessment for Gmail #351, mobile stores, hosted variant). Each Tier 1 item names the dependency (purchase / review / merge) and the owner (us vs. Google). Costs are itemised: $99 Apple + ~$400 Windows EV cert + ~$15 domain = $500–$1000/year recurring to start. CASA assessment is the deferred $15k–$50k sitting behind a usage trigger. "What is explicitly NOT in launch scope" section names the tempting Tier-3 items (federation sync, MCP marketplace, hosted product) so they don't crowd out the boring Tier-1 work. 2. dashboard Gmail follow-up CTA — apps/web/public/js/pages/dashboard-view.js gets a new renderConnectGmailHero() card that surfaces immediately after the bundled Google sign-in completes (Calendar + identity granted, Gmail scopes absent). Links to the in-app wizard at /#/connect-gmail with the "Why is this step needed?" external doc alongside. Without this nudge users would finish bundled sign-in, look at an empty Approvals queue, and not know SkyTwin's inbox features need a second 5-minute step. Three-line check on Gmail state: card returns '' when (a) tour mode is active, (b) Google isn't connected yet (the existing ConnectGoogleHero owns that state), or (c) Gmail scopes are already present on the OAuth token. Driven by the `scopes` array the /oauth/google/status endpoint already returns. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three Tier-2 launch-plan items in one PR, each isolated to its own subsystem:
2.2 PKCE verifier store now lives in CRDB (`oauth_pkce_pending` table +
`oauthPkcePendingRepository`). A desktop restart between /authorize
and /callback no longer drops the verifier and breaks sign-in.
`consume()` is a single DELETE...RETURNING so the same replay-
protection property survives the move off the in-memory Map.
2.3 OAuth /authorize accepts a whitelisted `next=connect-gmail` deep-
link. After Google consent the onboarding wizard lands the user on
/#/connect-gmail (with a "Calendar connected — now let's hook up
Gmail" banner) instead of dropping them on the dashboard root and
making them discover the follow-up CTA card. Free-form `next` URLs
are explicitly NOT accepted — that would be an open-redirect; the
whitelist is the security boundary.
2.4 Unset bundled client_id now bounces the user into the same connect-
gmail wizard instead of showing a generic error toast. The 503 is
tagged with `code: 'NO_GOOGLE_CLIENT_CONFIGURED'`; ApiError plumbs
structured `code`/`help`/`docs` through to the dashboard, and the
onboarding wizard branches on the code to route the user. The
connect-gmail wizard's final OAuth call uses ?newUser=true when no
userId is in localStorage so brand-new onboarding users finish the
flow.
Tests: 5 new for the PKCE repository, 5 new for the next= state round-
trip (HMAC tampering breaks signature verification, unknown next= drops
to null, etc.). All 689 API tests + 295 DB tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- parseSignedState: use hasOwnProperty.call on NEXT_HASH_ROUTES instead of bracket lookup so a `next=constructor` (or __proto__, toString, …) tag can't reach the inherited Object property and slip past the truthy check. New test loops the four common prototype keys and asserts nextHash stays null. Today's only consumer would have rendered a stringified function into the redirect URL — broken, not exploitable, but worth closing. - Onboarding NO_GOOGLE_CLIENT_CONFIGURED handler now re-enables the "Continue with Google" button before changing window.location.hash, so a synchronous re-render can't leave the button stuck on "Redirecting…". - Operator note in the PKCE-store comment block clarifying that migration 058 must run before the API serves traffic. We deliberately do NOT fall back to an in-memory Map — that would defeat the cross-restart guarantee the move to DB is meant to provide. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The desktop newUser sign-in opens Google in the system browser; the
callback fires there and there's no IPC back to the Electron app. The
old code re-enabled the wizard button with a "return here and continue"
error and the user had to manually click again — a real grandma-blocker
the existing TODO admitted to ("the web flow advances via redirect,
desktop currently does not").
This closes the gap:
- Wizard generates a UUIDv4 pendingKey client-side (crypto.randomUUID).
- /authorize validates UUID shape (anti-SQL-injection / anti-traversal,
not crypto-grade) and threads it through HMAC-signed state as
`key=<uuid>`. parseSignedState re-validates on read.
- /callback writes resulting userId + accountEmail + scopes + nextHash
to a new oauth_pending_signin table (migration 059) keyed by the
pendingKey, then renders the existing "close this tab" HTML.
- New GET /api/oauth/google/pending/:key endpoint (public — the
unguessable random key IS the authorization). Consume-on-read
(DELETE...RETURNING) so a leaked key can only be redeemed once.
Mirrors the existing pollUntilConnected pattern.
- google-signin.js polls the new endpoint when (desktop && newUser);
fires onComplete with { userId, nextHash } on success.
- Onboarding wizard's onComplete sets userId in localStorage and routes
to the deep-link target — auto-advance, no second click.
Tests: 6 new for oauthPendingSigninRepository (replay protection,
expiry defence-in-depth, scope-shape coercion); 4 new for the
isValidPendingKey gate + key= state encoding (rejecting SQL injection,
path traversal, wrong-version UUIDs, uppercase, etc.).
All 694 API tests + 301 DB tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…-filter expiry Post-/review fixes for the desktop pendingKey endpoint: 1. CRITICAL — don't return bare userId. The pre-existing POST /api/sessions accepts any userId from a localhost caller and returns a 7-day session token (unchanged by this PR — it's been the QR-pairing trust model). Returning userId from /pending/:key would chain those two endpoints: leaked key → consume → forge a session as that user. Instead, /pending/:key now mints the session itself in-process and returns the token alongside the userId. The pendingKey IS the credential; consume-on-read makes it one-shot. Client stashes the token under KEY_SESSION_TOKEN so subsequent API calls flow through Authorization: Bearer exactly like QR pairing. 2. MEDIUM — per-IP rate limit on /pending/:key. Without it, an attacker who exfiltrated a partial key (truncated log, side channel) could enumerate the remainder at line rate. Also a basic DoS vector. Wraps the same checkNewUserRateLimit() that already gates ?newUser=true. 3. MEDIUM — silent failure on remember() now logs with userId + pendingKey so the operator can correlate a wizard timeout with a real DB failure rather than chasing a phantom Google issue. 4. MEDIUM — consume() WHERE now filters by expires_at >= $now in SQL so a poll arriving past TTL doesn't delete the row before sweepExpired() reclaims it. Without this, network jitter on the client could destroy a row mid-handoff and the legitimate wizard would 404 even though the OAuth round-trip succeeded. 8. NIT — migration comment "128-bit" → "122-bit / UUIDv4" to stop overstating entropy. All 694 API tests + 301 DB tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…nd /review Second /review pass on the security-hardening commit (9509269) flagged two real regressions I introduced: 1. Atomicity gap. consume() already DELETEd the pending row before sessionRepository.create() ran; a transient CRDB failure or missing-table error mid-call would strand the user with no session AND no recoverable pending row. The poll loop would then silently exhaust its 5-min budget. Now the consume + session INSERT happen in a single withTransaction() — if the INSERT throws, the DELETE rolls back and the user can retry. 2. Rate-limiter starvation. /pending/:key was sharing the checkNewUserRateLimit bucket (5 hits / 60 s). The wizard polls every 2 s for 5 minutes = 30 hits/min from the same IP — would 429 after ~10 seconds and quietly time out at 5 min, exactly re-introducing the grandma-blocker this endpoint exists to fix. New checkPendingPollRateLimit() backed by its own Map; capped at 120/min so a normal poll loop runs comfortably with headroom for retries and jitter. New test covers the cross-starvation case (filling the authorize bucket leaves the poll bucket untouched). 3. Truncate pendingKey to 8-char prefix in the failure log (the key is 5-min-lived but log aggregators may index it longer). All 697 API tests + 301 DB tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five real findings on the latest commit; the sixth (cockroach-manager early-return skipping ensureDatabase) was already addressed in e3c3951 and is just re-flagged on a stale view — the SQL probe and unconditional ensureDatabase call are in the code today. 1. parseDatabaseUrl ssl handling. Only `sslmode=disable` was mapped to `false`; everything else returned undefined and fell back to the env default (false). A `DATABASE_URL=…?sslmode=require` against a secure CRDB cluster would silently connect over plaintext. New sslConfigForSslmode() maps disable/require/verify-ca/verify-full to the corresponding pg.PoolConfig.ssl shape; unknown values still fall through (typo-tolerant against env override). 2. cockroach-manager stop() — proc.kill('SIGTERM') after gracefulQuit() was unconditional; if the drain already caused CRDB to exit, the SIGTERM throws ESRCH and a clean shutdown becomes an exception. Now we check `proc.exitCode === null` first and wrap in try/catch anyway as belt-and-suspenders. Also fixed the stale "cockroach quit" comment — the implementation uses `cockroach node drain`. 3. apps/desktop/package.json scripts had `tsc &&electron` (no space after &&) on five lines and `tsc && electron-builder` (with space) on the same lines later. POSIX parses `&&token` correctly so this isn't a runtime bug, but the inconsistency is real. Normalized. 4. JSDoc for startGoogleSignIn's `onComplete` callback omitted the sessionToken field that pollUntilPendingResolved emits. Updated the type signature + docstring so future callers see the contract. 5. build-single-binary.sh bash dependency for the Windows package script — documented inline. Windows GitHub runners ship Git Bash so CI works; local Windows devs need Git Bash / WSL / MSYS. A future Node port would remove the constraint but no CI gates on it today. All 301 DB tests + 175 desktop tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Welcome-screen tour link was a tiny gray text link ("Explore with a
sample profile instead →", 0.82rem, --text-muted) — easy to miss next
to three large CTA buttons. Promoted to a btn-outline btn-lg card with
an "or" horizontal divider above it, matching the visual rhythm of the
existing choices but clearly framed as the alternative no-sign-in path.
Conditional-on-demo-availability is preserved: CTA + divider both live
inside the same #onb-tour-row div, both reveal when fetchDemoInfo()
returns available=true. Non-localhost / non-dev-bypass deployments
still get a clean welcome screen with no broken tour link.
No behavioural change to /api/v1/demo/{info,preview} or skyTwinExitTour.
CHANGELOG (Unreleased) + launch-plan §2.6 (now "partial — Unreleased")
+ README "first 60 seconds" walkthrough updated to match the new label.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The /review pass on 8c882ad flagged that the "or" divider used aria-hidden="true" on the entire wrapper div, hiding both the decorative lines AND the semantic "or" word from screen readers. Result: AT users go from the third primary CTA straight to "Try with a sample profile" without the alternative-path framing that's visually obvious. Fix: move aria-hidden to just the three inner spans (two lines + text), promote the wrapper to role="separator" with aria-label="or" so the relationship between the two button groups is announced once, correctly, without the decorative SVG noise. Visual identical. No behavioural change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Windows NSIS build was failing after 2.5h with:
File: failed creating mmap of "...@skytwindesktop-0.3.0-x64.nsis.7z"
Error in macro x64_app_files on macroline 1
Error in macro extractEmbeddedAppPackage on macroline 8
!include: error in script: "installSection.nsh" on line 66
Error in script "<stdin>" on line 199 -- aborting creation process
Root cause: electron-builder derives the intermediate .nsis.7z filename
from package.json `name`. The npm scoped name `@skytwin/desktop` gets
flattened to `@skytwindesktop` (only the `/` is stripped, not the `@`),
so the .nsis.7z lives at `...\@skytwindesktop-0.3.0-x64.nsis.7z`. NSIS's
makensis trips on @-prefixed paths in the File include macro and fails
to mmap the archive even though the file was written successfully.
Confirmed by the same-bundle pattern on this PR's prior CI run:
- macOS (DMG): same bundle, same @-prefixed intermediate — ✓ packaged
- Linux (AppImage/deb/rpm): same — ✓ packaged
- Windows (NSIS): same — ✗ mmap of @-filename
DMG and AppImage don't use makensis, so they sail through.
Fix: drop the `@scope/` prefix from the desktop package's npm name. It's
a leaf consumer (no other workspace package imports from it — verified
with grep), and pnpm-lock.yaml keys workspace entries by directory
path, not by npm name, so the lockfile is unchanged. `pnpm install
--frozen-lockfile` passes locally.
Updates:
- apps/desktop/package.json — name field + the embedded help-text in
the placeholder `build` script.
- Root package.json — 6 desktop:* scripts that use `pnpm --filter`.
- .github/workflows/build.yml — mac/win/linux package steps.
- .github/workflows/release.yml — build + 3 publish-always steps.
- apps/desktop/scripts/build-single-binary.sh — help-text echo.
- apps/desktop/src/headless.ts — invocation comment.
The package directory + workspace location are unchanged; only the
public `name` string flips. CHANGELOG references stay as-is (they're
historical).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codex /review on the cumulative #350 diff caught 4 P1 + 2 P2 issues that my prior Claude /review passes (scoped to each new commit) missed. Cross-model agreement was 0% — different scopes catch different things, which is exactly what the merge gate's two-reviewer promise is for. P1 fixes (default-flow blockers): 1. apps/worker/src/index.ts resolveGoogleConfig — required both clientId AND clientSecret. The bundled PKCE flow mints tokens with no clientSecret, so worker logged "credentials not configured; skipping Google connectors" and never processed a single signal on the grandma-grade default install. OAuth worked, twin did nothing. Fix: mirror api oauth.ts three-layer resolve (env → DB → bundled), accept empty clientSecret as the PKCE signal (refreshAccessToken already handles it correctly). Service-manager already injects SKYTWIN_DEFAULT_GOOGLE_CLIENT_ID into worker env via buildChildEnv. 2. bin/skytwin-install — `pnpm db:migrate` ran before DATABASE_URL was exported, so @skytwin/db fell back to localhost:26257. If the user set SKYTWIN_DB_PORT to dodge a collision or `localhost` resolved to ::1 instead of the 127.0.0.1 listener, migrations silently landed on the wrong socket. Build the URL from the same env vars bin/skytwin-db uses. 3. apps/web/public/js/pages/onboarding.js — desktop pendingKey onComplete stored userId + session token + hash but skipped KEY_ONBOARDED, hideWizard(), and skyTwinSetUserId(). Dashboard rendered #/connect-gmail BEHIND the still-visible onboarding modal — sign-in looked stuck, reload reopened first-run. Mirror the tour path's full three-step teardown. 4. apps/web/public/js/pages/connect-gmail.js — final OAuth step used `window.location.href = data.url` which inside Electron's renderer loads accounts.google.com in an embedded UA, rejected as disallowed_useragent. Route through startGoogleSignIn which detects Electron and uses openExternal + pendingKey poll. Plumbed `include` param through getGoogleAuthUrl + startGoogleSignIn so the Gmail scope opt-in survives the routing change. P2 fixes: 5. apps/web/public/js/pages/connect-gmail.js — PUT /api/credentials/ google bootstrap-without-session is a self-hoster edge case. Default bundled-client launch path doesn't reach it. Documented inline + launch-plan, with operator workarounds noted; a proper bootstrap token mechanism is its own scoped change. 6. bin/skytwin-db is_running — fallback path returned true on ANY port listener. A stray postgres / leftover container would make cmd_start skip launching CRDB. New is_crdb_responding() helper runs SELECT 1 to verify the listener speaks CRDB before short-circuiting. Tests: 697 API + 115 worker — all green. No new test files added; fixes either mirror established patterns (#1, #3, #4) or harden bash fallbacks the existing test harness doesn't exercise (#2, #5, #6). GATE: codex re-review pending after CI. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two consecutive Windows CI runs failed at the same step with the same error, on different commits and different filenames: - 71bdee5: makensis File: failed creating mmap of "...@skytwindesktop-0.3.0-x64.nsis.7z" - 3a25132: makensis File: failed creating mmap of "...skytwin-desktop-0.3.0-x64.nsis.7z" The rename from @skytwin/desktop -> skytwin-desktop in 2da3549 removed the leading @ from the filename. The error still reproduced verbatim against the new name, so the @ theory was wrong. Actual root cause: makensis is a 32-bit process that opens the freshly- written .nsis.7z intermediate via mmap to embed it into the final installer.exe. On the GitHub Actions windows-latest runner, Windows Defender's real-time scanner opens that same .7z to scan it the moment it's written. Defender's open holds a sharing lock; makensis's mmap call races against it and returns failure. This is documented in electron-userland/electron-builder#6107. Fix: add an ExclusionPath for the build workspace + electron-builder cache dirs before the Package step runs. Defender stays active on the runner overall (so signtool's signing pass on cockroach.exe / SkyTwin.exe still gets scanned), but the staging dirs that NSIS reads back are out of bounds. Uses Add-MpPreference -ExclusionPath which only requires the admin shell the runner already has, no policy changes. Why not disable Defender entirely: - Disabling RT scanning leaves the signtool steps unprotected, and we sign two .exe files (cockroach.exe + SkyTwin.exe) before makensis runs. - Exclusion is the surgical fix; disable is the sledgehammer. Why not nsis-web (download payload at install time): - That target requires a release URL the payload is hosted at; CI runs don't tag releases. - Scope creep for fixing a CI race condition. Expected outcome: Windows job clears the makensis step on first try (previously failed at ~2h26 to 2h36 with same error). If it still fails post-exclusion, the next diagnosis target is bundle size vs 32-bit makensis address space, but exclusion is overwhelmingly the most likely cause given the timing reproducibility. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three Windows CI runs failed in a row, each at the same makensis mmap
step ~2.5h into the job. Three problems were stacked:
1. The .nsis.7z intermediate was racing Windows Defender's RT scanner.
Defender holds a sharing handle while it scans the freshly-written
.7z; 32-bit makensis mmap-opens the same file and gets ESHARING
surfaced as `File: failed creating mmap of`.
2. electron-builder's win-unpacked copy step was spending an hour just
writing the ~10,000 loose files in dist/embedded/{api,worker,web}/
(pnpm-deploy node_modules trees, multiplied by 3 apps). NTFS
small-file throughput on the GitHub Actions runner is much worse
than APFS / ext4 — the macOS+Linux desktop builds finished in 9
and 4 minutes against the same input.
3. `differentialPackage: true` (electron-builder default) was running
an extra .blockmap generation pass on the already-slow .nsis.7z.
The blockmap is for electron-updater delta downloads we don't ship
yet (gated on §1.5 release tag + signing certs).
Already-shipped Defender exclusion (a4b2e09) addresses #1. This commit
addresses #2 and #3:
#2 fix: `apps/desktop/scripts/build-single-binary.sh` now tars the
embedded api/worker/web trees into a single `apps.tar.gz` after the
pnpm-deploy + strip-self-symlinks step. The extraResources filter in
`apps/desktop/package.json` shrinks to {apps.tar.gz, bundle-manifest.
json}. `apps/desktop/src/service-manager.ts` gains an
`ensureEmbeddedRoot()` method that extracts the tarball to
`<userData>/embedded/` on first launch, gated by a `.version` marker
so subsequent launches (and post-upgrade launches) handle the
extract correctly. `startApi`, `startWeb`, `startWorker`, and
`runMigrations` all consume the extracted path.
#3 fix: `nsis.differentialPackage: false` + `compression: "normal"`
pinned explicitly so a future electron-builder bump can't silently
switch to LZMA-max and regress build time.
Trade-offs documented in CHANGELOG. User-facing first-launch latency
gains ~5-15s for the one-time tar extract; subsequent launches see no
change (sentinel-file existence check is microsecond). Installer size
shrinks by the tar.gz compression ratio (~30-40% on node_modules).
Tests:
- platform-utils.test.ts updated for the new extraResources shape
(apps.tar.gz + bundle-manifest.json, no more api/**/* etc.) +
negative assertions that the old loose patterns are gone.
- 175 desktop tests pass.
- tsc --noEmit clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two stacked wins on top of the already-shipped tarball + Defender exclusion work: 1. CRDB binary cache. The 5-platform CRDB set (~700MB compressed, ~140MB per platform) was being re-downloaded on every desktop CI job because actions/cache@v4 was only pointed at electron and electron-builder caches. Three desktop jobs x 5 binaries x ~10s each = ~150s spent on cold-cache work that's identical between runs. Cache path now includes ~/.cache/skytwin/crdb-binaries (where bin/skytwin-db's download helper stages the archives) and the cache key hashes build-single-binary.sh too so a SKYTWIN_CRDB_VERSION bump invalidates correctly. 2. Parallel downloads in build-single-binary.sh. The `for entry in CRDB_TARGETS; bundle_crdb_binary "$entry"; done` loop blocks on each platform sequentially even though every call has independent inputs and outputs. Backgrounded with `&` + reaped via `wait $pid` in a follow-up loop. Cold-cache wall time drops from ~25-50s to ~5-10s (limited by the slowest single download). Warm cache short-circuits at the early `already bundled, skipping` return so parallelism is a no-op there. `set -e` alone doesn't propagate failures from backgrounded functions, so an explicit `crdb_failed` flag walks the wait results and `exit 1`s if any child returned non-zero. Without that, a corrupt download (sha mismatch -> exit 3) would silently leave the binary missing and electron-builder would fail later with a confusing "missing extraResources" error. Net expected savings: roughly 1-3 minutes off each desktop build on warm cache (the dominant case after the first run), ~30s on cold cache. Doesn't change the long-pole job (Linux at 14m, three output formats), so the end-to-end wall time stays around 17 min — but the runtime spent on redundant network is gone. Tests: bash -n clean, workflow YAML valid (would fail at GH parse if not). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two CI failures surfaced after PR #350 merged + v0.6.57.0 was tagged: 1. release.yml — all three platforms failed at pnpm/action-setup with ERR_PNPM_BAD_PM_VERSION ("Multiple versions of pnpm specified"). The action was pinned at @v4 with `version: 9`, but every package in the workspace has `packageManager: "pnpm@..."` in package.json which @v4 also reads. With both inputs present, v4 errors instead of picking one. Aligning with build.yml's @v5 usage (which reads only from packageManager) clears it. 2. build.yml desktop-macOS — once v0.6.57.0 landed on main, electron-builder's auto-publish heuristic saw the tag and tried to upload the just-built .dmg to GitHub Releases. build.yml is the PR/main validation gate, not the publish path — it doesn't set GH_TOKEN, so the upload threw "GitHub Personal Access Token is not set". Passing `--publish never` explicitly to every `electron-builder` invocation in build.yml short-circuits the auto-publish detection, regardless of branch/tag context. release.yml is the only path that should publish, and it already passes `--publish always` with the right token. Also bring release.yml up to the same parity as build.yml for the fixes that landed during PR #350: - CRDB binary cache path (~/.cache/skytwin/crdb-binaries) in the actions/cache@v4 step on all three OS variants, with the same cache key shape so warm caches transfer between the two workflows. - Defender ExclusionPath step on the windows-latest variant before electron-builder runs — same makensis mmap-race fix. Net: re-running release.yml against tag v0.6.57.0 (either via the tag delete+re-push or via workflow_dispatch) should produce the .dmg + .exe + .AppImage + .deb + .rpm artifacts the README rewrite needs. main CI's macOS job is fixed for any future tag-on-main scenario. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR combines post-#350 CI/workflow fixes with a broader set of install, desktop-bundling, OAuth, and documentation updates to make tagged releases and “fresh install” flows reliable across platforms.
Changes:
- Fix GitHub Actions workflow behavior for releases/validation (pnpm setup alignment, caching/Defender mitigations, prevent build validation from auto-publishing, add install-validation workflow).
- Add DB-backed OAuth state helpers for desktop flows (PKCE verifier persistence + new-user pending sign-in handoff), plus PKCE-capable connector helpers and web onboarding deep-linking.
- Harden installer/dev ergonomics (native CockroachDB default, Docker/Ollama opt-ins), improve desktop bundling (embedded tarball + bundled CRDB), and publish/update Google verification + privacy/ToS docs.
Reviewed changes
Copilot reviewed 66 out of 71 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| VERSION | Bump recorded project version. |
| README.md | Update install and advanced env-var documentation (native CRDB default, validation harness). |
| packages/db/src/types.ts | Add row types for new OAuth pending tables. |
| packages/db/src/repositories/oauth-pkce-pending-repository.ts | Add repository for PKCE verifier persistence/consumption. |
| packages/db/src/repositories/oauth-pending-signin-repository.ts | Add repository for desktop new-user pending sign-in handoff. |
| packages/db/src/repositories/index.ts | Export new OAuth repositories/types. |
| packages/db/src/migrations/059-oauth-pending-signin.sql | Add pending-signin table + index. |
| packages/db/src/migrations/058-oauth-pkce-pending.sql | Add pkce-pending table + index. |
| packages/db/src/migrations/057-dedupe-decisions-and-unique-index.sql | Add dedupe + partial unique index migration for decisions. |
| packages/db/src/migrations/055-decision-outcomes-execution-plan-id.sql | Fix migration compatibility with CRDB reserved keywords. |
| packages/db/src/migrations/046-approval-decision-id-unique.sql | Replace CRDB-internal error call with portable assertion. |
| packages/db/src/migrations/023-decision-signal-id-uniqueness.sql | Move dedupe/index work out of early migration to later migration. |
| packages/db/src/index.ts | Re-export new row types and repositories/types. |
| packages/db/src/connection.ts | Parse DATABASE_URL first and improve sslmode handling. |
| packages/db/src/tests/oauth-pkce-pending-repository.test.ts | Unit tests for PKCE pending repository. |
| packages/db/src/tests/oauth-pending-signin-repository.test.ts | Unit tests for pending sign-in repository. |
| packages/db/scripts/copy-sql.cjs | Cross-platform SQL asset copy step for db build output. |
| packages/db/package.json | Add SQL-copy step to build script. |
| packages/connectors/src/oauth/google-oauth.ts | Add PKCE helpers and support PKCE vs confidential token flows. |
| packages/connectors/src/index.ts | Export new PKCE helpers/types. |
| packages/connectors/src/tests/google-oauth-pkce.test.ts | Tests for PKCE generation/auth URL/token exchange/refresh behavior. |
| package.json | Update desktop pnpm filter to new package name. |
| install.sh | Update installer flow (native CRDB default; Docker/Ollama opt-ins; improved diagnostics). |
| docs/terms.html | Add Terms of Service static page for verification/compliance. |
| docs/technical-spec.md | Update dev setup and CRDB notes to native-binary default path. |
| docs/privacy.html | Add Privacy Policy static page (Google data handling disclosures). |
| docs/launch-plan.md | Add launch plan document (tiered rollout/checklists). |
| docs/index.html | Add GitHub Pages homepage for project + scope explanations. |
| docs/google-verification.md | Add staged Google OAuth verification plan/justifications. |
| docs/connect-gmail.html | Add public “BYO Gmail OAuth client” walkthrough page. |
| docs/cockroach-architecture.md | Update CRDB local dev guidance (native default, Docker opt-in). |
| docs/_config.yml | Configure GitHub Pages/Jekyll exclusions for docs site. |
| docker/validate/run-validation.sh | Add container-side script to drive install.sh end-to-end. |
| docker/validate/Dockerfile.ubuntu-2204 | Add Ubuntu validation image for install regression testing. |
| docker/validate/Dockerfile.fedora-40 | Add Fedora validation image for install regression testing. |
| docker/validate/Dockerfile.debian-12 | Add Debian validation image for install regression testing. |
| CONTRIBUTING.md | Update contributor setup to native CRDB + validation harness. |
| bin/validate-installs | Add multi-distro Docker-based install validation harness. |
| bin/skytwin-install | Update bootstrap installer to native CRDB default + opt-in Docker/Ollama. |
| bin/skytwin-dev | Update dev runner to support native CRDB + optional Docker/Ollama and better logs. |
| apps/worker/src/index.ts | Add bundled client-id fallback for PKCE-only token refresh compatibility. |
| apps/web/public/js/pages/onboarding.js | Improve onboarding flow (deep-link next=connect-gmail, session token handling, demo CTA). |
| apps/web/public/js/pages/dashboard.js | Add Gmail follow-up CTA card rendering. |
| apps/web/public/js/pages/dashboard-view.js | Add “Connect Gmail” hero card component. |
| apps/web/public/js/google-signin.js | Add desktop new-user pendingKey polling and next/include support. |
| apps/web/public/js/app.js | Register /connect-gmail route. |
| apps/web/public/js/api-client.js | Thread structured error fields (code/help/docs) and handle 412/503 config cases. |
| apps/desktop/vitest.config.ts | Exclude packaged output directories from test discovery. |
| apps/desktop/src/main.ts | Enforce single-instance lock to prevent CRDB startup races. |
| apps/desktop/src/headless.ts | Update docs for new desktop package filter name. |
| apps/desktop/src/first-launch.ts | Treat bundled CRDB binary as satisfying dependency checks. |
| apps/desktop/src/cockroach-manager.ts | Add CockroachDB supervisor for desktop (spawn, readiness checks, drain/stop). |
| apps/desktop/src/tests/platform-utils.test.ts | Update packaging expectations (tarball-based embedded bundle + per-platform CRDB resources). |
| apps/desktop/src/tests/cockroach-manager.test.ts | Add unit tests for CockroachManager basics (paths/ports). |
| apps/desktop/scripts/build-single-binary.sh | Bundle per-platform CRDB, use pnpm deploy, tar embedded apps, strip symlinks, manifest updates. |
| apps/desktop/package.json | Rename package, add prepackage step, update extraResources (embedded tarball + CRDB per platform), NSIS tweaks. |
| apps/api/src/lib/llm-client-factory.ts | Add embedded provider detection gates (binary + model) and disable flag. |
| apps/api/src/tests/oauth-scope-tiers.test.ts | Add tests enforcing Gmail scope tier gating behavior. |
| apps/api/src/tests/oauth-rate-limit.test.ts | Add pending-poll rate limit test coverage. |
| apps/api/src/tests/oauth-next-route.test.ts | Add tests for whitelisted next= routing and pendingKey UUID validation. |
| apps/api/src/tests/llm-client-factory.test.ts | Add tests for embedded provider gating/disable behavior. |
| .gitignore | Ignore local logs and desktop build output artifacts. |
| .github/workflows/release.yml | Fix pnpm action versioning and align caches/Defender exclusions; update desktop filter name. |
| .github/workflows/install-validation.yml | Add workflow to run install validation harness on relevant changes. |
| .github/workflows/build.yml | Add workflow_dispatch and force desktop packaging to never auto-publish; cache CRDB binaries; Defender exclusions on Windows. |
| .env.example | Switch DATABASE_URL default host to 127.0.0.1 with rationale. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # Three states to handle: | ||
| # - $INSTALL_DIR doesn't exist → clone from $REPO_URL. | ||
| # - $INSTALL_DIR has a real .git directory → fetch + ff-only merge. | ||
| # - $INSTALL_DIR has source but no .git directory (Conductor worktree | ||
| # gitlinks, validation-harness untar, manual copy) → use as-is. | ||
| # Conductor worktrees ship .git as a 75-byte gitlink file pointing into | ||
| # a shared object store, so `[ -d $INSTALL_DIR/.git ]` returns false even | ||
| # though the repo is fully present. The `-e` check + `ls -A` fallback | ||
| # covers that and also handles a hand-extracted source tree. | ||
| if [ -d "$INSTALL_DIR/.git" ]; then | ||
| ok "Already cloned — pulling latest" | ||
| git -C "$INSTALL_DIR" fetch origin "$BRANCH" --quiet | ||
| # Use --ff-only so we never overwrite uncommitted local changes. | ||
| if ! git -C "$INSTALL_DIR" merge --ff-only "origin/$BRANCH" 2>/dev/null; then | ||
| warn "Local changes detected — keeping your version, skipping pull." | ||
| # Tolerate offline / sandboxed environments (validation containers, etc.) | ||
| # where `origin` may not be reachable. The on-disk version is then used | ||
| # as-is, which is exactly what the validation harness wants. | ||
| if git -C "$INSTALL_DIR" fetch origin "$BRANCH" --quiet 2>/dev/null; then | ||
| # Use --ff-only so we never overwrite uncommitted local changes. | ||
| if ! git -C "$INSTALL_DIR" merge --ff-only "origin/$BRANCH" 2>/dev/null; then | ||
| warn "Local changes detected — keeping your version, skipping pull." | ||
| fi | ||
| else | ||
| warn "Could not reach $REPO_URL — using the on-disk version as-is." | ||
| fi | ||
| elif [ -d "$INSTALL_DIR" ] && [ -n "$(ls -A "$INSTALL_DIR" 2>/dev/null)" ]; then | ||
| ok "Found existing source at $INSTALL_DIR (no .git directory) — using as-is" |
There was a problem hiding this comment.
Fixed in 2b5e77e. Changed line 95 from [ -d "$INSTALL_DIR/.git" ] to [ -e ... ] so gitlink-as-file (Conductor worktrees, etc.) hits the fetch+merge branch the header comment already promised. git -C follows the gitlink transparently. See install.sh:95-101.
| * Semantics mirror `oauth-pkce-pending-repository`: | ||
| * - `remember()` upserts on pending_key. A re-issued key (same | ||
| * wizard click, different OAuth round-trip) overwrites. | ||
| * - `consume()` is DELETE...RETURNING so a leaked key can only be | ||
| * redeemed once. | ||
| * - `sweepExpired()` called best-effort on every remember(). | ||
| */ |
There was a problem hiding this comment.
Fixed in 2b5e77e. Added this.sweepExpired().catch(...) to remember() so the header docstring matches reality. Caught + swallowed because sweep is housekeeping that mustn't break the primary write path. See oauth-pending-signin-repository.ts:53-58.
| function generatePendingKey() { | ||
| if (typeof crypto !== 'undefined' && typeof crypto.randomUUID === 'function') { | ||
| return crypto.randomUUID(); | ||
| } | ||
| // Polyfill: 16 random bytes, set the version (4) + variant nibbles | ||
| // per RFC 4122 §4.4, format as 8-4-4-4-12 hex. | ||
| const bytes = new Uint8Array(16); | ||
| crypto.getRandomValues(bytes); | ||
| bytes[6] = (bytes[6] & 0x0f) | 0x40; |
There was a problem hiding this comment.
Fixed in 2b5e77e. Added explicit typeof crypto !== 'undefined' && typeof crypto.getRandomValues === 'function' guard before the polyfill path. On miss, throws a typed Error naming the missing primitive and pointing at the existing-user sign-in path as a fallback. Math.random is not a safe substitute (predictable across sessions). See google-signin.js:27-44.
| * Unknown values fall through to undefined so a typo doesn't silently | ||
| * downgrade — the env-var fallback path applies as before. | ||
| */ | ||
| function sslConfigForSslmode(sslmode: string | null): DatabaseConfig['ssl'] | undefined { | ||
| if (sslmode === null) return undefined; | ||
| if (sslmode === 'disable') return false; | ||
| if (sslmode === 'require') return { rejectUnauthorized: false }; | ||
| if (sslmode === 'verify-ca' || sslmode === 'verify-full') { | ||
| return { rejectUnauthorized: true }; | ||
| } | ||
| return undefined; | ||
| } |
There was a problem hiding this comment.
Fixed in 2b5e77e. Unknown sslmode now throws a typed Error at startup naming the misspelling and listing the valid libpq values. Comment updated. Also added explicit allow and prefer handlers (return undefined, matching libpq) so they're no longer "unknown." The silent-downgrade-on-typo concern you raised was real — a typo like sslmode=requier would have fallen through to DATABASE_SSL=false and connected over plaintext. See connection.ts:55-69.
| # pnpm/action-setup@v5 reads the `packageManager` field from | ||
| # package.json instead of taking a `version:` input. The earlier | ||
| # `@v4 + version: 9` shape failed every release job at setup with | ||
| # `ERR_PNPM_BAD_PM_VERSION: Multiple versions of pnpm specified` | ||
| # because v4 saw both inputs. Aligned with build.yml's v5 usage. | ||
| - uses: pnpm/action-setup@v5 |
There was a problem hiding this comment.
The runtime/product changes you're seeing (install.sh, oauth-pending-signin-repository, google-signin.js, connection.ts) are NOT new in this PR — they landed via PR #350's squash merge to main as commit 5aaa2bb. GitHub Copilot appears to have reviewed every commit in the branch's history rather than the cumulative diff against main; git diff origin/main...HEAD shows only the workflow YAML changes plus the four small Copilot follow-up fixes pushed in 2b5e77e. PR title now leads with v0.6.58.0 and the body enumerates both the CI scope and the four Copilot-driven fixes that were added to address your other inline comments. Not splitting because: (1) the four Copilot fixes are small and self-contained, (2) shipping them in a separate PR would re-run the same merge-gate cycle for trivial polish, (3) the version bump frames the combined scope honestly.
# Conflicts: # .github/workflows/build.yml
Bumped MICRO → PATCH because Copilot's review surfaced four real bugs in code that landed via #350's squash, and addressing them in #352 expands scope beyond pure CI workflow fixes. CI workflow fixes (original #352 scope): - release.yml: pnpm/action-setup @v4+version conflict (ERR_PNPM_BAD_PM_VERSION) - build.yml: electron-builder auto-publish on tagged main without GH_TOKEN - release.yml parity with build.yml's PR #350 fixes (CRDB cache, Defender) Copilot fixes: - install.sh worktree detection — `[ -d .git ]` → `[ -e .git ]` so gitlink files (Conductor worktrees, etc.) hit the fetch+merge branch the header comment already promised. Previously fell through to "no .git directory, use as-is" silently. - oauth-pending-signin-repository.remember() now actually calls sweepExpired() best-effort. Header docstring promised it; code never did. Abandoned OAuth flows were growing the table monotonically. - generatePendingKey() guards `crypto.getRandomValues` too — the polyfill path was guarded only on `crypto.randomUUID`, so an environment with no crypto global threw a useless ReferenceError instead of a typed "browser too old" error pointing at the existing-user fallback. - connection.ts sslConfigForSslmode() throws on unknown sslmode instead of silently downgrading. A typo like `sslmode=requier` previously fell through to DATABASE_SSL (default false) and shipped a plaintext connection against what should have been a secure cluster. Also added explicit `allow` and `prefer` handlers matching libpq semantics. Tests: 301 DB tests pass (no sslmode typos in repo). JS/TS/bash syntax clean. The sslConfigForSslmode change is intentionally not test-covered in this PR — it's a private function and adding a test file for it is its own scope. Documented as a follow-up. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adversarial review caught four real issues in the previous v0.6.58.0
commit. Fixes in order of severity:
[HIGH] connection.ts — module-load throw cascade. The new sslmode
throw was firing at @skytwin/db IMPORT time (via FROM_URL initializer),
which would have crashed every consumer on bad input: unrelated test
files, migration scripts, type-checker tooling. Wrapped the module-load
parse in a try/catch so import always succeeds; getPool() re-parses
fresh on each first call and lets the throw propagate from there. This
matches the contract the sslConfigForSslmode docstring already
promised.
[HIGH] build.yml double-publisher race. tag push (v*.*.*) fires BOTH
build.yml AND release.yml — both target the same GitHub Release. The
legacy release: job at build.yml:419 used softprops/action-gh-release
while release.yml uses electron-builder's GH publisher. They race for
the same release artifacts; one wins, the other duplicates or fails.
Deleted the build.yml release job — release.yml is canonical (it has
the code-signing env wiring CSC_LINK/APPLE_ID/etc that build.yml never
had). build.yml stays as validation-only.
[MEDIUM] oauth-pending-signin-repository.remember — three concerns:
1. Empty `.catch(() => {})` silently swallowed sweep failures, killing
the only observability operators had into table-growth bugs. Now
logs via console.warn before swallowing.
2. `this.sweepExpired()` would TypeError if a caller destructures
(`const { remember } = repo`). Switched to explicit reference
`oauthPendingSigninRepository.sweepExpired()`.
3. oauth.ts:833 already called sweepExpired explicitly before the
remember call — now redundant (two pool connections per callback
under burst). Removed the caller-side sweep; repo owns housekeeping.
Tests: 697 API + 301 DB all pass. No new tests in this commit; the
sslmode behavior change deserves regression coverage in a follow-up
(test for `sslmode=requier` throws + `sslmode=allow|prefer` returns
undefined). Out of scope for a /review fix-up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…#355) v0.6.58.0 third release attempt failed on macOS with: ⨯ /Users/runner/work/skytwin/skytwin/apps/desktop not a file right after `empty password will be used for code signing reason= CSC_KEY_PASSWORD is not defined`. Root cause: release.yml's Build-and-publish step sets CSC_LINK from a secret via ternary. When the secret is unset, the expression evaluates to '' (empty string), NOT undefined — so CSC_LINK IS in the environment but as empty. electron-builder then interprets the empty value as a relative path-to-cert that resolves to the CWD (apps/desktop), tries to read it as a file, finds a directory, errors out before packaging even starts. build.yml dodges this by setting CSC_IDENTITY_AUTO_DISCOVERY: 'false' and NOT setting CSC_LINK. release.yml needed the same defensive guard for the no-cert-secrets case. Fix: pair CSC_LINK with CSC_IDENTITY_AUTO_DISCOVERY computed from whether the matching secret is non-empty. When secrets are present, auto-discovery is true (and CSC_LINK takes precedence anyway). When secrets are empty, auto-discovery is false and electron-builder falls through cleanly to "skip signing" — producing the unsigned artifacts the workflow header comment already documents as expected pre-Apple-Developer-enrollment. Fourth fix in the v0.6.58.0 release-pipeline chain after: - PR #352: pnpm/action-setup v4→v5 + --publish never on build.yml - PR #353: bare pnpm build instead of --filter skytwin-desktop - PR #354: drop `--` separator before --publish flags - This: CSC_IDENTITY_AUTO_DISCOVERY fallback Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five consecutive failures of release.yml on tag v0.6.58.0: 1. pnpm/action-setup v4+version conflict (PR #352 fix) 2. `pnpm --filter skytwin-desktop build` skipped workspace deps (PR #353 fix) 3. pnpm `--` separator broke electron-builder arg parsing (PR #354 fix) 4. Empty CSC_LINK env var made electron-builder treat CWD as cert path (PR #355 attempted fix — did not actually work, see #5) 5. CSC_IDENTITY_AUTO_DISCOVERY=false isn't enough because CSC_LINK="" (set-to-empty-string, not unset) still triggers the path-resolve code path Each fix revealed the next bug because release.yml was never tested end-to-end — it's been broken since the file was committed. At 5 fixes deep, the right move is to stop fixing release.yml and use the known-working publisher pattern instead. build.yml already builds artifacts successfully on tag push via its desktop-mac/desktop-windows/desktop-linux/mobile-* matrix. PR #352 deleted build.yml's softprops-based release: job specifically to avoid double-publishing with release.yml. With release.yml deleted, that conflict is gone — restore the simpler chain: - Desktop+mobile matrix builds artifacts (already works, --publish never). - New release: job downloads via actions/download-artifact and creates a draft GitHub Release via softprops/action-gh-release@v3. Trade-off: - Lose: electron-builder's GitHub publisher integration (auto-updater channel YAML). When code signing + auto-update become priorities, add release.yml back with the lessons from #352-#355 baked in OR switch to a single workflow with electron-builder publish. - Gain: artifacts actually publish today, on an unsigned-build basis, which is what the launch plan §1.6 README rewrite needs. After this lands: re-tag v0.6.58.0 (5th attempt). build.yml's matrix runs as before, plus the new release: job downloads + publishes a draft. Operator manually clicks Publish in the GitHub UI to make the release live. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Summary
CI workflow fixes that unblock the v0.6.57.0 → v0.6.58.0 release path, plus four real bugs Copilot flagged on PR #350 commits that were already squashed onto main.
CI workflow fixes (original scope):
release.yml:pnpm/action-setup@v4 + version: 9was failing every release matrix job withERR_PNPM_BAD_PM_VERSIONbecause v4 also reads thepackageManagerfield. Upgraded to@v5(matchesbuild.yml).build.yml: electron-builder auto-detected the v0.6.57.0 tag on main and tried to publish without aGH_TOKENthe validation pipeline never sets. All threepackage:*invocations now pass--publish neverso validation stays decoupled from publish.release.ymlparity: brought the CRDB binary cache + Defender ExclusionPath fixes over frombuild.yml.Copilot review fixes (post-#350 squash):
install.sh:[ -d .git ]→[ -e .git ]so Conductor worktrees (and any other gitlink setup) hit the fetch+merge branch the header comment already promised.oauth-pending-signin-repository.remember(): now actually callssweepExpired()best-effort. Header docstring promised it; code never did. Abandoned OAuth flows were growing the table monotonically.generatePendingKey(): guardscrypto.getRandomValuestoo — the polyfill path was guarded only oncrypto.randomUUID, so a no-cryptoenvironment threwReferenceErrorinstead of a typed error pointing at the existing-user fallback.connection.ts:sslConfigForSslmode()throws on unknown sslmode instead of silently downgrading. A typo likesslmode=requierpreviously fell through toDATABASE_SSL(default false) and shipped a plaintext connection. Also added explicitallow/preferhandlers matching libpq.Version
0.6.57.0→0.6.58.0(PATCH; original was MICRO but Copilot scope expansion bumped it).Test plan
node --check,bash -n, andtsc --noEmitall clean.35de45d): Linux ✅ macOS ✅ Windows ✅ iOS ✅ Android ✅.v0.6.58.0to trigger the (now-fixed)release.ymland finally produce the GitHub Release artifacts that the README rewrite (launch-plan §1.6) needs.🤖 Generated with Claude Code