feat: add standalone mode for Docker-free local development#2012
Merged
feat: add standalone mode for Docker-free local development#2012
Conversation
Pull Request Test Coverage Report for Build 5a0db6db-bca6-478c-bbbd-cd9fc16e1472Details
💛 - Coveralls |
…pment Add a new `astro dev standalone` command that runs Airflow locally without Docker, using `airflow standalone` and `uv` for dependency management. This provides a dramatically faster dev loop for Airflow 3 projects. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
d95fbe6 to
1e42a1e
Compare
for more information, see https://pre-commit.ci
Standalone mode now works with both Airflow 2 (runtime 4.0.0+) and Airflow 3 (runtime 3.x). The health check endpoint, image registry, and settings version are determined dynamically based on the detected Airflow major version. Changes: - Accept airflowMajor "2" or "3" (was "3" only) - Health check: /health + webserver (AF2) vs /api/v2/monitor/health + api-server (AF3) - Image registry: quay.io/astronomer/astro-runtime (AF2) vs astrocrpublic.azurecr.io/runtime (AF3) - Settings version passed dynamically (was hardcoded 3) - Kill/reset cleans up both AF2 and AF3 credential files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
for more information, see https://pre-commit.ci
Default `astro dev standalone` to background mode — the CLI starts the airflow process, writes a PID file, waits for the health check, prints status, and returns. A `--foreground` flag preserves the previous stream-to-terminal behaviour. New subcommands: - `astro dev standalone stop` — SIGTERM the process group, clean up PID file - `astro dev standalone logs [-f]` — dump or tail the log file Also wires `reset` to stop a running process before cleaning up files. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
9683aac to
2593b32
Compare
Contributor
|
@jlaneve I know this mirrors the airflow standalone command but would an shorter alias be useful too? Edit: no longer relevant now that it's a flag / config setting. |
…install - Remove airflowMajor field and AF2 code paths (standalone is AF3-only) - Replace Docker-based constraint extraction with HTTP fetch from pip.astronomer.io/runtime-constraints - Implement 2-step install: first install airflow with full constraints, then install user requirements with only airflow/task-sdk version locks - Add parsePackageVersionFromConstraints helper for task-sdk version - Remove runtimeImageName, execDockerRun, constraintsFileInImage - Simplify healthEndpoint to always return AF3 endpoint Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
b1a21b5 to
e91a7ad
Compare
…an project root - Fix CDN base URLs: constraintsBaseURL → cdn.astronomer.io/runtime-constraints - Add freezeBaseURL (cdn.astronomer.io/runtime-freeze) for full 191-package list - getConstraints() now fetches and caches both constraints + freeze files; returns freeze path for use as pip -c arg in step 1 install - Move "already running" check to top of Start() before any install work - Add ensureCredentials() to seed passwords file with admin:admin on first run - Add readCredentials() to display username/password in startup output - Redirect AIRFLOW_HOME → .astro/standalone/ so airflow.cfg, airflow.db, and logs/ all live there instead of cluttering the project root - Set AIRFLOW__CORE__SIMPLE_AUTH_MANAGER_PASSWORDS_FILE to .astro/standalone/ - Update Kill() to clean up .venv/ and .astro/standalone/ only - Update tests: freeze file routing in fetch mock, AIRFLOW_HOME assertions, new TestStandaloneEnsureCredentials, TestStandaloneReadCredentials tests Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add Standalone.Build() stub (errStandaloneNotSupported) to satisfy the updated ContainerHandler interface which gained a Build method in main - Resolve conflict in cmd/airflow_test.go: keep both TestAirflowStandalone and new TestAirflowBuild from main, preserving all subtests from both Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Rename the command from 'standalone' to 'local' for a more intuitive UX. The internal Go types (Standalone, StandaloneHandlerInit) are unchanged. - cmd/airflow.go: Use:"standalone" → Use:"local", all function/var names - cmd/airflow_hooks.go: EnsureStandaloneRuntime → EnsureLocalRuntime - cmd/airflow_test.go: update all test names and assertions - airflow/standalone.go: add Build() stub (satisfies updated ContainerHandler) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
astro dev standalone for Docker-free local developmentastro dev local for Docker-free local development
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Parse the optional -python-X.Y suffix from runtime image tags (e.g. 3.1-12-python-3.11) instead of hardcoding Python 3.12. Falls back to 3.12 (the default for all Runtime 3.x images) when no suffix is present. Cache filenames now include the Python version to avoid stale lookups when switching between Python versions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…lution - Fix standaloneExecAirflowCommand using non-existent .venv/bin/bash; use system bash instead (venv PATH is already set in env) - Fix .env file overriding standalone-critical vars (AIRFLOW_HOME, etc.); .env is now applied first, then critical settings override on top - Add 3-tier Python version resolution: Dockerfile tag → runtime JSON → fallback 3.12 - Add DefaultPythonVersion/PythonVersions fields to RuntimeVersionMetadata - Add GetDefaultPythonVersion() to look up Python version from updates.astronomer.io Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…s subcommand - Remove redundant "already running" check in startBackground (Start already guards it) - Strip matching quotes from .env values to match Docker Compose behavior - Add --port flag and config.CFG.APIServerPort fallback for custom webserver port - Add `astro dev local ps` subcommand to check process status - Make readCredentials deterministic by preferring the admin user Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove comments that restate the code, duplicate doc comments from elsewhere, or are stale context from incremental development. Keep numbered steps in Start() and comments that explain non-obvious "why". Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
config.CFG.APIServerPort.GetString() returns "0" (not "") when the port is not configured in .astro/config.yaml. This caused the health check to hit localhost:0 and time out. Add explicit check for "0" in webserverPort(). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Settings import runs airflow CLI commands (connections list, variables list) that require the database to be initialized. Previously settings were applied before `airflow standalone` was started, causing the DB to not exist yet. Move applySettings to after the health check passes in both foreground and background modes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the separate `astro dev local` command tree with a unified
mode-based approach. A `dev.mode` project config ("docker" or
"standalone") controls which backend is used, with `--standalone` and
`--docker` persistent flags for per-call overrides. All existing
commands (start, stop, logs, ps, kill, restart) now work in both modes.
- Add `dev.mode` config with "docker" default and validator
- Add `--standalone`/`--docker` mutually exclusive persistent flags
- Add resolveDevMode/resolveHandlerInit for mode-based handler dispatch
- Update all pre-run hooks to skip Docker in standalone mode
- Add --foreground/-f and --port/-p flags to start and restart
- Remove `astro dev local` command tree entirely
- Update user-facing strings from `astro dev local` to `astro dev`
- Update command descriptions to be mode-agnostic
- build/upgrade-test always use Docker regardless of mode
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the separate `astro dev local` command tree with a unified
mode-based approach. A `dev.mode` project config ("docker" or
"standalone") controls which backend is used, with `--standalone` and
`--docker` persistent flags for per-call overrides. All existing
commands (start, stop, logs, ps, kill, restart) now work in both modes.
- Add `dev.mode` config with "docker" default and validator
- Add `--standalone`/`--docker` mutually exclusive persistent flags
- Add resolveDevMode/resolveHandlerInit for mode-based handler dispatch
- Update all pre-run hooks to skip Docker in standalone mode
- Add --foreground/-f and --port/-p flags to start and restart
- Remove `astro dev local` command tree entirely
- Update user-facing strings from `astro dev local` to `astro dev`
- Update command descriptions to be mode-agnostic
- build/upgrade-test always use Docker regardless of mode
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
astro dev local for Docker-free local developmentInstead of scattering isStandaloneMode() checks in every hook, assign a noOpContainerRuntime in standalone mode so containerRuntime is never nil. Downstream hooks work unchanged — the no-op methods just return nil. Eliminates a nil-pointer landmine for future contributors. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add a custom usage template that splits flags into Common, Docker Mode, and Standalone Mode sections using cobra flag annotations. Applied to the start and restart commands which have mode-specific flags. Other commands keep the default template. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…, parse, settings) Implement 6 ContainerHandler methods on the Standalone struct that previously returned errStandaloneNotSupported: - Run: executes arbitrary commands in the venv environment - Bash: opens an interactive shell with venv env (AIRFLOW_HOME, PATH) - ImportSettings: imports connections/variables/pools from settings file - ExportSettings: exports to settings file or .env file - Pytest: runs pytest on DAGs with file path resolution and args passthrough - Parse: validates DAGs by running the default integrity test Add ensureVenv() helper to validate the venv exists before running commands, and resolveInEnvPath() to look up binaries in the env's PATH (needed because exec.Command resolves using the parent process's PATH, not cmd.Env). Update error messages for Build, ComposeExport, and UpgradeTest to be more descriptive about why they're unavailable in standalone mode. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Map the existing --scheduler, --triggerer, --api-server, --webserver, and --dag-processor flags to standalone log prefixes so users can filter log output by component. The --webserver flag maps to api-server since AF3 standalone has no separate webserver process. When no flags are passed, all lines are shown (including standalone orchestrator startup messages). When any component filter is active, only matching lines appear. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… mode dispatch Pre-flight check dials localhost:<port> before starting to prevent silently connecting to the wrong Airflow instance. Also routes build and upgrade-test commands through resolveHandlerInit() so they properly error in standalone mode, and fixes the custom port env var to use the AF3 config key (AIRFLOW__API__PORT). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
jeremybeard
reviewed
Feb 25, 2026
Move standalone mode implementation and tests to standalone.go and standalone_test.go to better reflect their contents. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Floating tags like "3.1" (without the -Z patch) are now resolved to the latest pinned version (e.g., "3.1-12") via the runtime versions JSON from updates.astronomer.io. Unrecognizable tags (custom images) get a clear error explaining that a pinned runtime image is required. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Match Docker mode behavior by setting SIMPLE_AUTH_MANAGER_ALL_ADMINS=True instead of requiring login with seeded credentials. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Match Docker image build log behavior — hide uv install output by default and only show it when --verbose / debug logging is active. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
# Conflicts: # config/config.go # config/types.go
The LocalExecutor communicates with the api-server via the execution API. Without this, custom ports cause Connection refused errors because the executor defaults to localhost:8080 while the api-server is elsewhere. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ions in standalone mode Add an experimental notice when starting standalone mode. Fix a goroutine leak in startForeground where the signal handler blocked forever on normal exit. Replace concrete type assertions (containerHandler.(*Standalone)) with a SetStartOpts interface method so the cmd layer no longer depends on the Standalone concrete type. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Python's _scproxy calls SCDynamicStoreCopyProxies which is not fork-safe. When Airflow's LocalExecutor forks, this can spin at 100% CPU indefinitely. Setting NO_PROXY=* tells Python to skip _scproxy entirely. We only do this when no proxy is configured (env vars, .env file, or macOS system settings) so corporate proxy users aren't affected. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
jeremybeard
reviewed
Mar 2, 2026
…s struct Fold the 8 positional Start parameters and the separate SetStartOpts call into a single StartOptions struct, reducing the interface surface and eliminating the two-step setup pattern. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
aadd5b9 to
b7d7123
Compare
jeremybeard
approved these changes
Mar 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Adds a standalone mode for running Airflow locally without Docker, unified into the existing
astro devcommand tree via mode-based dispatch.How it works
dev.modeproject config ("docker"or"standalone") controls which backend is used--standalone/--dockerpersistent flags allow per-call overridesstart,stop,logs,ps,kill,restart) work in both modesrun,bash,pytest,parse,object import,object exportall work in standalone modelogssupports component filtering (--scheduler,--api-server, etc.) in both modesbuild,upgrade-test, andcompose-exportreturn clear "not available" errors in standalone modeKey changes
dev.modeconfig with"docker"default, validated to accept only"docker"or"standalone"--standalone/--dockermutually exclusive persistent flags on thedevroot commandresolveDevMode()resolves mode (flag → config priority),resolveHandlerInit()returns the right handlerConfigureContainerRuntimeassigns anoOpContainerRuntimein standalone mode, so downstream hooks (EnsureRuntime,SetRuntimeIfExists,KillPreRunHook,KillPostRunHook) work without nil-pointer guards or mode checks--foreground/-fand--port/-pflags added for standalone modestartandrestartuse a custom usage template that organizes flags into Common / Docker Mode / Standalone Mode sectionsastro dev logscomponent flags (--scheduler,--api-server,--triggerer,--dag-processor,--webserver) filter standalone log lines by prefix.--webservermaps toapi-serversince AF3 standalone has no separate webserver. Unfiltered shows all lines including standalone orchestrator messages.airflow standalonedoesn't crash when the port is taken (only the api-server subprocess fails), so without this check the health check would pass against whichever service already occupies the portAIRFLOW__API__PORT(AF3 config key) instead of the deprecatedAIRFLOW__WEBSERVER__WEB_SERVER_PORT, reads from both--portflag andapi-server.portconfigbuildandupgrade-testnow route throughresolveHandlerInit()instead of hardcodedcontainerHandlerInit, so they properly error in standalone mode instead of silently running Docker operationsastro dev localcommand tree is replaced by the unified approachStandalone command implementations
astro dev run <cmd>airflow dags list)astro dev bashAIRFLOW_HOME,PATHset)astro dev object importairflow_settings.yamlvia venvastro dev object export.envvia venvastro dev pytestastro dev parseastro dev logsastro dev buildastro dev compose-exportastro dev upgrade-testUsage
Setting the mode via config:
Using per-command flags:
Standalone-specific commands:
What the flags look like
Known limitation
astro dev run --standalonedoes not work because theruncommand usesDisableFlagParsing(pre-existing cobra behavior to pass args through to airflow). Useastro config set dev.mode standaloneinstead, which works correctly for all commands includingrun.Test plan
TestStandaloneStart_PortInUse)astro config set api-server.portworks correctlybuild,upgrade-test,compose-exportshow proper errors in standalone mode🤖 Generated with Claude Code