I thought I’d give NixOS a spin after chatting with Jeff N. I had a great experience, I went from zero knowledge to a working web server in under two hours. And that includes lots of time writing this blog post, reading Mastodon, etc. And it was fun too!
I got this done mostly by asking Claude to tell me what to do: who has patience to read docs? It’s fun to learn something new this way. Also using this Wiki page and this baby steps tutorial. Julia’s notes are good too.
Update: I did this all as a Proxmox VM. There’s also notes on installing NixOS in a Proxmox LXC container.
Some basic concepts
I’m still trying to work my head around what Nix does. My impression is it’s similar to a Dockerfile or a puppet/ansible script, it’s a config file that creates a working Linux environment for stuff to run in. The difference is Nix has fancy design so everything’s managed, rollbacks are possible, etc.
NixOS is not Nix. Nix is the package manager, NixOS is an operating system built using the Nix package manager. Nix is like apt, NixOS is like Ubuntu.
The master configuration for NixOS and all the things running in it (like a web server) is in /etc/nixos/configuration.nix and has its own syntax. Almost the whole system is configured here. Nix folks have written little shims to take configuration directives from this file and write them into whatever weirdo format the ancient Unix utility might expect. (I hear the sendmail.cf generator is sentient but no one can understand what it’s saying.) You edit configuration.nix, check it into git, etc. You use nixos-rebuild switch to tell the system to implement the changes.
I’m looking at NixOS for a whole system image but there’s also a way to use Nix for your personal user environment that’s broadly similar.
There’s an effort to make bit-for-bit reproducible builds for NixOS which has nice affordances for security. But that’s not the only reason to use NixOS, more a happy side effect.
Install experience
First attempt for an install was with the minimal NixOS ISO (still 1GB!). But they aren’t kidding about minimal, this thing has you running fdisk yourself and leaves you with an unconfigured system. The graphical installer (3.5GBish) partitions a disk and gives you a basic working system when it’s done, so that’s easier.
After the default install the system reboots. 2.7GB is consumed on the hard drive and there’s 89 packages (not counting documentation packages). There’s about 10 processes running other than the kernel and all its threads. They all run out of /nix/store. They include
- systemd running journald, udevd, oomd, timesyncd, logind
- dbus-daemon
- nsncd: a shim for name service (this one?)
- Networkmanager
That’s a remarkably small system.
/bin and /usr/bin are nearly empty. There’s about 10 directories in my $PATH but much of what I’d expect to find is in /run/current-system/sw/. All the binaries there are symlinks into /nix/store. They’ve been dynamically linked with libraries like libc referencing paths in /nix/store.
NixOS installs a fairly complete Linux environment, at least it has less, bash, rsync, ssh. No sshd, python, or vim though. So why 2.7GB? As near as I can tell there’s a lot of stuff in /nix/store that’s not in my current environment, like a whole install of python3.11. Those binaries seem to work if I run them directly but I’m guessing I’m supposed to configure the NixOS system to officially use them if I want them.
The /nix/store can apparently get very large but there’s ways to manage it. I’m not sure if there’s an easy way to say “prune everything off the disk I don’t actually use” to get the install size down to 100MB or so.
Installing some packages in NixOS
First up, add an ssh server so I can log in easily. That’s as simple as editing configuration.nix to uncomment the line services.openssh.enable = true and then running nixos-rebuild switch to make it happen. The switch takes about 6 seconds. Various docs reference having to configure the firewall or enable password login, I didn’t have to do either with the NixOS GUI install’s defaults. (There’s some magic that allows ssh through the firewall if you enable it.)
I tried a rollback to remove ssh again. Easy as running nixos-rebuild switch --rollback. Apparently nix is keeping a history of what it’s done that it can undo. The rollback kills the sshd so I can’t log in anymore. My active sshd was still running, so I guess it doesn’t kill everything. (That implies many things about just how much state Nix will alter. Files, not memory, is my guess.)
Adding other shell utilities just requires listing them in environment.systemPackages. (Or maybe in a user environment, don’t understand that yet.) Whitespace delimited list. I did “joe git zip unzip” for now. And another rebuild switch and now I have those binaries available.
Final project was a hello world web server. I had Claude do this part for me.
# Enable Caddy web server
services.caddy = {
enable = true;
configFile = pkgs.writeText "Caddyfile" ''
:80 {
root * /var/www
file_server
}
'';
};
# Create a simple HTML file
system.activationScripts.createWebRoot = ''
mkdir -p /var/www
echo "<html><body><h1>Hello World from NixOS!</h1></body></html>" > /var/www/index.html
'';
# Allow HTTP traffic
networking.firewall.allowedTCPPorts = [ 80 ];
That all just worked, thanks Claude! The firewall change was necessary: without it Caddy was running but unreachable outside this machine. iptables -L confirms that there is a firewall running (the default for NixOS), the chain nixos-fw seems most relevant. It’s letting in only ssh and http and ICMP ping.
Things to learn about
- Updating packages from network
- Scripting creating a new NixOS system from the environment file
- Per-user Nix config vs system-wide
- Nix state tracking, how rollbacks work
- Logging
- Recommendations for running actual production services in NixOS
- Pruning disk usage
- The ecosystem of Nix packages
- nix-shell
- WTF a flake is
Thoughts
This experiment was a good experience. Over the years I’ve tried a bunch of ways to automate setting up Linux system environments and gotten very frustrated every time. NixOS I had up and running in an hour and I feel like I could use this for real.
What would I use it for? Not sure, need to think more. I’ve been considering rolling my own NAS, a setup with NFS and Samba and syncthing and restic. I was just going to hand configure it but maybe NixOS would be a nice way to do that in a more managed way. As a VM in Proxmox I guess? Or maybe just an LXC container. Or just as a Nix thing in a general purpose VM, not sure if Nix gives me the separation I’d otherwise use Proxmox for.
I’ve run into some usability warts. nix search does not work as the config file suggests, you have to do some complicated set of flags for “experimental features” (this is apparently fixable but is a bad experience for a newbie!) The configuration that the NixOS installer created throws various warnings when you build with it. But stuff mostly seems to work.
The NixOS docs seem pretty good. I haven’t read them much but Claude has and that works great for queries. When I’ve looked stuff up myself there’s clear explanations for what I’m trying to learn.
I love learning new stuff like this. Claude was a big help.
Nelson's log 