Skip to content

feat: daemon self-provisioning — desktop becomes pure display layer#89

Merged
AprilNEA merged 17 commits intomasterfrom
feat/daemon-self-provision
Mar 21, 2026
Merged

feat: daemon self-provisioning — desktop becomes pure display layer#89
AprilNEA merged 17 commits intomasterfrom
feat/daemon-self-provision

Conversation

@AprilNEA
Copy link
Member

Summary

Daemon auto-provisions everything on startup. Desktop reduces to: connect gRPC → watch setup status → render UI.

Daemon changes

  • Boot asset provisioning: ensure_boot_assets() downloads kernel/rootfs before runtime.init(). Bundle seeding copies from app bundle first (zero CDN on first launch from DMG)
  • Docker CLI tools: Non-blocking ensure_docker_tools() in recovery phase
  • gRPC decoupled from runtime: Single gRPC server starts before asset download. Services use SharedRuntime (OnceLock) — return UNAVAILABLE until runtime ready
  • CLI tools symlink: New helper RPC cli_link/cli_unlink for /usr/local/bin/docker → app bundle
  • Proto: DOWNLOADING_ASSETS phase, docker_tools_installed flag

CLI changes

  • install/uninstall hidden as _install/_uninstall (internal, consumed by brew/DMG)
  • _install stripped to helper + daemon + shell only (DNS/socket/assets handled by daemon)
  • _install --helper-path flag for custom helper binary location

Build

  • Makefile: Developer ID signing, sign-daemon/sign-all/verify targets

Test plan

  • cargo test --workspace passes
  • cargo clippy / cargo fmt clean
  • Daemon cold start with bundle seeding (no network)
  • gRPC GetSetupStatus / WatchSetupStatus via grpcurl
  • Helper installed via osascript, DNS/socket/routes configured
  • Container IP direct access (172.17.0.x) verified

Copilot AI review requested due to automatic review settings March 21, 2026 10:32
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR shifts ArcBox toward a self-provisioning daemon architecture: the daemon now boots up in observable phases (gRPC first, then asset provisioning, then runtime/services), and privileged helper functionality expands to support system-wide Docker CLI symlinks. This enables the desktop app to act primarily as a UI that watches setup status.

Changes:

  • Added new setup status signals (DOWNLOADING_ASSETS, docker_tools_installed) to support phased boot progress reporting.
  • Refactored daemon startup into early init + gRPC start + runtime init + post-runtime services, with RPC services gated on a SharedRuntime OnceLock.
  • Added helper RPC + mutation implementation for /usr/local/bin CLI symlink link/unlink, plus daemon recovery tasks to install Docker tools and CLI links.

Reviewed changes

Copilot reviewed 26 out of 28 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
rpc/arcbox-protocol/src/generated/arcbox.v1.rs Regenerated protocol types to include new setup phase + docker tools flag.
rpc/arcbox-protocol/proto/api.proto Adds DOWNLOADING_ASSETS phase and docker_tools_installed field to SetupStatus.
app/arcbox-helper/tests/rpc_test.rs Extends mock helper server to include new CLI link/unlink RPCs.
app/arcbox-helper/tests/common/mod.rs Introduces shared mock-helper test harness for integration tests.
app/arcbox-helper/src/validate.rs Adds CLI name/target validation for /usr/local/bin symlink management.
app/arcbox-helper/src/server/mutations/mod.rs Adds mutations module declarations including new cli mutations.
app/arcbox-helper/src/server/mutations/cli.rs Implements privileged symlink management for Docker CLI tools.
app/arcbox-helper/src/server/handler.rs Wires new CLI link/unlink RPC handlers to mutations.
app/arcbox-helper/src/lib.rs Extends tarpc service trait with cli_link / cli_unlink.
app/arcbox-helper/src/client.rs Adds client helpers for new CLI link/unlink RPCs.
app/arcbox-daemon/src/startup.rs Splits daemon initialization into init_early and init_runtime, adds bundle seeding + boot asset provisioning.
app/arcbox-daemon/src/shutdown.rs Makes runtime shutdown conditional on runtime having been initialized.
app/arcbox-daemon/src/services.rs Starts gRPC once up-front; post-runtime services start afterward and use deferred runtime handle.
app/arcbox-daemon/src/self_setup/cli_tools.rs Adds a self-setup task to create /usr/local/bin Docker CLI symlinks via helper.
app/arcbox-daemon/src/self_setup.rs Exposes new CliTools self-setup task module.
app/arcbox-daemon/src/recovery.rs Adds non-blocking Docker tools installation and CLI symlink self-setup during recovery.
app/arcbox-daemon/src/main.rs Orchestrates phased startup: early init → start gRPC → init runtime → start remaining services.
app/arcbox-daemon/src/context.rs Replaces direct runtime field with SharedRuntime and a runtime() accessor.
app/arcbox-daemon/Cargo.toml Adds dependency on arcbox-docker-tools for daemon-side CLI tool installation.
app/arcbox-cli/src/commands/mod.rs Hides install/uninstall as internal _install/_uninstall commands.
app/arcbox-cli/src/commands/install.rs Simplifies install to helper + daemon registration + shell; moves infra setup to daemon self-setup.
app/arcbox-cli/Cargo.toml Removes arcbox-helper dependency from CLI crate.
app/arcbox-api/src/system.rs Adds docker_tools_installed to SetupState and a setter.
app/arcbox-api/src/lib.rs Re-exports SharedRuntime for daemon/service wiring.
app/arcbox-api/src/grpc.rs Introduces SharedRuntime (OnceLock-backed) and returns UNAVAILABLE until runtime is ready.
Makefile Adds Developer ID signing targets (sign-daemon, sign-all, verify) and updates run-daemon flow.
Cargo.lock Updates workspace dependency graph reflecting new/removed crate dependencies.

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e942c0f9df

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copilot AI review requested due to automatic review settings March 21, 2026 11:05
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 34 out of 36 changed files in this pull request and generated 8 comments.

AprilNEA added 17 commits March 21, 2026 19:33
…flag

Support daemon self-provisioning by adding:
- DOWNLOADING_ASSETS = 8 phase to SetupStatus.Phase enum
- docker_tools_installed bool field to SetupStatus message
- set_docker_tools_installed() setter on SetupState
Before runtime.init(), the daemon now downloads kernel/rootfs if not
cached and copies the agent binary from the boot cache. The setup phase
is set to DOWNLOADING_ASSETS during the download so desktop clients can
show progress.
Spawn a non-blocking task in the recovery phase that installs docker,
buildx, compose, and credential-helper from assets.lock if not already
present. Sets docker_tools_installed on SetupState when done.
Download kernel/rootfs and runtime binaries as step 4/6 of the install
flow so the daemon can cold-start without network access.
Auto-detect "Developer ID Application: ArcBox, Inc." from keychain.
Ad-hoc signing (-s -) doesn't work with Virtualization.framework on
recent macOS — the process gets SIGKILL'd.

New targets:
- sign-daemon: build + Developer ID sign arcbox-daemon
- sign-all: build + sign all three binaries
- verify: inspect signing status of all binaries
- sign: kept for CI smoke tests (ad-hoc only)
DNS resolver, Docker socket, boot assets, and runtime binaries are all
handled by the daemon's self-setup and startup provisioning. The install
command now only does what requires sudo (helper installation) plus
daemon LaunchAgent registration for CLI-only users.

Also removes the arcbox-helper dependency from arcbox-cli since no
commands use the helper tarpc client directly anymore.
Rename to `_install` / `_uninstall` with `hide = true`. These are
implementation details consumed by brew postinstall and DMG installers,
not user-facing commands. The daemon self-provisions everything users
need on first start.
Two-phase startup:
1. init_early: directories, config, sockets → start gRPC with
   SystemService only → clients can observe DOWNLOADING_ASSETS phase
2. init_runtime: seed/download boot assets, build runtime, start VM →
   abort early gRPC, restart with all services

Bundle seeding: when running inside an app bundle, the daemon copies
boot assets from Contents/Resources/assets/{version}/ to the cache
before attempting CDN download. This makes first launch offline-capable
when assets are bundled in the DMG.

DaemonContext.runtime is now Option<Arc<Runtime>>, accessed via
runtime() which panics if called before init_runtime.
seed_from_bundle() now copies all assets from the app bundle:
  - Contents/Resources/assets/{version}/ → ~/.arcbox/boot/{version}/
  - Contents/Resources/runtime/          → ~/.arcbox/runtime/
  - Contents/Resources/bin/arcbox-agent  → ~/.arcbox/bin/arcbox-agent

ensure_docker_tools() passes Contents/MacOS/xbin/ as bundle_dir to
DockerToolManager, so docker/buildx/compose/credential-helper are
installed from the bundle before falling back to CDN download.

This makes the first launch from DMG fully offline-capable.
New helper RPC methods create/remove symlinks in /usr/local/bin/ for
Docker CLI tools, pointing into the app bundle's Contents/MacOS/xbin/.
Strict validation: only allowed tool names, target must be inside an
.app bundle, no path traversal.

The daemon's self-setup now includes a CliTools task that runs after
DNS/socket setup, creating symlinks like:
  /usr/local/bin/docker → ArcBox Desktop.app/Contents/MacOS/xbin/docker

This gives desktop users system-wide `docker` access without manual
PATH configuration, matching Docker Desktop / OrbStack behavior.
Service impls now hold Arc<OnceLock<Arc<Runtime>>> instead of
Arc<Runtime>. RPCs return UNAVAILABLE while runtime is not set.
This eliminates the gRPC server restart — a single server runs from
boot, and Machine/Sandbox services become available once init_runtime()
fills the OnceLock.

Fixes: Desktop gRPC client losing connection when server restarts.
- Fix pkill pattern: ArcBoxHelper → arcbox-helper
- Remove helper binary, plist, and socket after bootout
- Remove /usr/local/bin/docker* symlinks created by cli_link
Move shared constants to arcbox-constants/src/paths.rs:
- privileged::{HELPER_BINARY, HELPER_PLIST, HELPER_SOCKET, DOCKER_SOCKET}
- labels::{DAEMON, HELPER}
- DOCKER_CLI_TOOLS (single source of truth for docker/buildx/compose/credential)

Update all consumers:
- install.rs: use privileged:: constants
- uninstall.rs: use privileged:: + DOCKER_CLI_TOOLS
- validate.rs: use DOCKER_CLI_TOOLS from constants
- cli_tools.rs: use DOCKER_CLI_TOOLS from constants
- docker_socket.rs: use privileged::DOCKER_SOCKET
- socket.rs (helper): use privileged::DOCKER_SOCKET
- lib.rs (helper): derive HELPER_SOCKET from constants

Fix stale comment in self_setup.rs referencing old `sudo arcbox install`.
- Add unit tests for validate_cli_name/validate_cli_target
- Handle shared_runtime.set() double-call as error, not silent drop
- Remove unused OnceLock import in context.rs
- cli.rs link(): only replace ArcBox-owned symlinks (check .app/Contents/MacOS/xbin/)
- cli.rs unlink(): use same strict ownership check
- uninstall.rs: match ownership check pattern
- Gate CLI symlink creation behind docker_integration flag
- cli_tools.rs is_satisfied(): verify symlink points to current xbin_dir, not just "ArcBox"
- Makefile: pass SIGN=0 to rebuild-run-daemon.sh (avoid double-sign)
- startup.rs: run seed_from_bundle in spawn_blocking (avoid stalling tokio)
- validate.rs: restrict cli_target to /Applications/ or /Users/ (prevent
  fake .app paths in /tmp); add test for /tmp/evil.app rejection
- mod.rs + main.rs: gate _uninstall behind #[cfg(target_os = "macos")]
@AprilNEA AprilNEA force-pushed the feat/daemon-self-provision branch from 6c3e24b to 9e952dd Compare March 21, 2026 11:36
@AprilNEA AprilNEA merged commit eedcdeb into master Mar 21, 2026
8 checks passed
@AprilNEA AprilNEA deleted the feat/daemon-self-provision branch March 21, 2026 11:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants