Skip to content

Use local docker registry to push and pull app images#1355

Merged
djmb merged 38 commits intobasecamp:mainfrom
npezza93:local-docker-registry
Oct 8, 2025
Merged

Use local docker registry to push and pull app images#1355
djmb merged 38 commits intobasecamp:mainfrom
npezza93:local-docker-registry

Conversation

@npezza93
Copy link
Contributor

Allow applications to be deployed without needing to set up a repository in a remote Docker registry.

If the registry server starts with localhost, Kamal will start a local docker registry on that port and push the app image to it.

Then when pulling the image onto the servers, we use net-ssh to forward the that port from the app server to the deployment server.

This is branched off @djmb 's initial work. I have this working on a simple app but curious to open this up to try out to find issues and possibly optimize. Saw an overall speed up of around 10-15 seconds.

To get this to work update your deploy.yml files registry section to look something like:

registry:
  server: "localhost:5555"

@npezza93 npezza93 force-pushed the local-docker-registry branch from c217a8b to f24e74c Compare January 14, 2025 19:39
def setup
combine \
docker(:start, "kamal-docker-registry"),
docker(:run, "--detach", "-p", "127.0.0.1:#{local_port}:5000", "--name", "kamal-docker-registry", "registry:2"),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Publishing the port on your local machine will make it available in the same network segment (see moby/moby#45610). That's why I didn't go any further with this at the time.

I think we can mitigate this though by generating a password for the registry when we boot it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm looks like a pr was merged a few days ago to resolve that. Perhaps in the next release of buildkit it'll be fixed for us?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh awesome! This will be great 🎉

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this is now released: https://docs.docker.com/engine/release-notes/28/

@npezza93 npezza93 force-pushed the local-docker-registry branch 3 times, most recently from f3dc9f3 to 3c21394 Compare January 20, 2025 16:03
Copy link
Contributor

@ShPakvel ShPakvel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again for working on it ❤️ 🚀

pull_on_hosts(KAMAL.hosts - first_hosts)
else
pull_on_hosts(KAMAL.hosts)
Kamal::Cli::PortForwarding.new(KAMAL.hosts, KAMAL.config.registry.local_port).forward do
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you guys for working on this feature! 🙏🏼 I was thinking to try it myself 😄

Minor 2 cents suggestion:

  • This line read like forward ports always. While there is condition inside of class implementation. Even with local registry by default it kind of confusing in case of remote registry usage.
  • And the condition in the class if KAMAL.config.registry.local? is the only part that coupled to registry port. Extracting that condition will make class more general purpose PortForwarding to use for any other service port forwarding.

So how about, instead of this line here (and condition in the class), to do something like following with direct condition?

    def pull_on_hosts(hosts, forward_registry_port: KAMAL.config.registry.local?)
      if forward_registry_port
        return Kamal::Cli::PortForwarding.new(hosts, KAMAL.config.registry.local_port).forward do
          pull_on_hosts(hosts, forward_registry_port: false)
        end
      end

      ...
    end

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @ShPakvel, I like the suggestion around making the PortForwarding class usable elsewhere and not tying it to if the registry is local. Didnt really follow the code suggestion but made some edits to make the class usable elsewhere. Let me know what you think

djmb and others added 12 commits February 10, 2025 10:21
Allow applications to be deployed without needing to set up a repository
in a remote Docker registry.

If the registry server starts with `localhost`, Kamal will start a local
docker registry on that port and push the app image to it.

Then when pulling the image onto the servers, we use net-ssh to forward
the that port from the app server to the deployment server.
@npezza93 npezza93 force-pushed the local-docker-registry branch from ace00d7 to a106b81 Compare February 10, 2025 15:21
@npezza93 npezza93 force-pushed the local-docker-registry branch from 3c5dc0b to 48f56b3 Compare May 13, 2025 20:41
@driveton
Copy link

I'm a solodeveloper and I was using Cloudflare tunnels to connect to a local registry but this is much better. Thank you @npezza93

I wonder if can use the same configuration for local and remote registry to avoid "works on machine". Changes here: https://github.com/driveton/kamal/tree/local-registry-branch

  • Fail if there's no image set. Remove this so there's no magic here.
def image
    name = raw_config&.image.presence
    name ||= raw_config&.service if registry.local?

    name
end
  • We can keep login/logout from registry and add setup/remove.
➜   git:(main) ✗ bundle exec kamal registry help 
Commands:
  kamal registry help [COMMAND]  # Describe subcommands or one specific subcommand
  kamal registry login           # Log in to registry locally and remotely
  kamal registry logout          # Log out of remote registry locally and remotely
  kamal registry remove          # Remove local registry
  kamal registry setup           # Setup local registry

  • We can have a local: true/false and a registry_port in deploy.yml and/or have a deploy.localhost.yml and secrets-localhost.
def local?
    registry_config["local"] == true
end

def registry_port ## Instead of local_port
   registry_config.fetch("registry_port", 5000)
end
  • Fail if username/password are not, same as remote registry even if they are not used.
registry:
  server: localhost:5000
  username: localuser
  password:
    - KAMAL_REGISTRY_PASSWORD
  registry_port: 5000 ## Similar to app_port
  local: true
  • Forward ports explicitly only when the registry is local
def pull(forward_registry_port: KAMAL.config.registry.local?)
    login_to_registry_remotely

    if forward_registry_port
      return Kamal::Cli::PortForwarding.new(KAMAL.app_hosts, KAMAL.config.registry.registry_port).forward do
        pull(forward_registry_port: false)
      end
    end

    if (first_hosts = mirror_hosts).any?
      # Pull on a single host per mirror first to seed them
      say "Pulling image on #{first_hosts.join(", ")} to seed the #{"mirror".pluralize(first_hosts.count)}...", :magenta
      pull_on_hosts(first_hosts)
      say "Pulling image on remaining hosts...", :magenta
      pull_on_hosts(KAMAL.app_hosts - first_hosts)
    else
      pull_on_hosts(KAMAL.app_hosts)
    end
  end

@driveton
Copy link

Can we make this work when running Kamal inside the devcontainer?

@npezza93
Copy link
Contributor Author

Can we make this work when running Kamal inside the devcontainer?

I have no need for such a feature but you are more than welcome to open a pr against this and I can include it

@driveton
Copy link

Sure, it's kind of tricky though. I'm not sure how to solve it.

My understanding is that Kamal, at least for Rails, is "best used" inside the devcontainer

  • If we create the local registry in the host, we need to the app devcontainer where kamal is running to see the local registry in the host.
  • If we create the local registry as a service the devcontainer, we need to tunnel the remote sever to the registry running in the devcontainer (in this scenario you don't need to setup the registry, just add it as service in the compose file).

Docker expects https so for any of those scenarios to work, we need to "hack" to accept http or to install certificates which is cumbersome.

@airblade
Copy link
Contributor

It would be great to merge this. Currently I use ghcr.io and it is randomly totally unusable due to docker pull slowing to a crawl. I wish I could just push my images directly to my server from my local machine.

@khaled-badenjki
Copy link

This would be a great feature to have. what is blocking this MR to be merged? is any help needed?

@djmb djmb merged commit 7638229 into basecamp:main Oct 8, 2025
1 of 7 checks passed
@npezza93 npezza93 deleted the local-docker-registry branch October 8, 2025 13:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants