the personal website of mat ess mat ess mat@mat.services https://www.mat.services https://www.mat.services/image/favicon.svg © 2022-2023 mat ess Zola 2022-10-17T00:00:00+00:00 https://www.mat.services/atom.xml 2022-10-16T00:00:00+00:00 2022-10-16T00:00:00+00:00 https://www.mat.services/posts/now-on-netlify/

a short update regarding me packing up my static site and moving from fly.io to netlify

]]> <p>a short update today, and one that (unfortunately) obsoletes a lot of the work I put into <a href="https://www.mat.services/posts/static-site-with-nix-and-caddy/">my last post on static site hosting</a>: this site is now hosted on netlify! after <a href="https://www.mat.services/posts/command-line-flake-arguments/">my most recent article</a> ran into some accessibility issues for people outside of my fly.io deployment region, I decided to minimize the moving parts involved in keeping the site running. i moved my static content over to <a rel="noopener noreferrer" target="_blank" href="https://www.netlify.com/">netlify</a>, a well-liked PaaS for deploying webapps and static sites (they also offer other things I don't care about, such as &quot;Serverless&quot; and FaaS offerings). <a rel="noopener noreferrer" target="_blank" href="https://docs.netlify.com/configure-builds/file-based-configuration/">netlify's configuration file</a> supports all the same headers and redirects options that I was using in caddy, so the transition was extremely smooth.</p> <p>fly.io is still running my goatcounter analytics instance, as well as my gitea instance, and I don't intend to stop using it for my other cloud deployment needs. ultimately, i didn't want to debug what could have been going wrong with the server on fly.io, nor did i want to think hard about scaling up into multiple regions or similar strategies. netlify seems like a good fit for straightforward and performant static site hosting.</p> <p>my lingering quibble with netlify, along with many of the hosted PaaS and CI/CD solutions out there, is one of interoperability. in order to make use of continuous deployment functionality, virtually every one of these tools or platforms requires source code to be hosted on Github or Gitlab. this seems pretty arbitrary to me: couldn't most platforms just make use of git and webhooks directly? support could be easily extended then beyond Github and Gitlab to self managed git hosting instances, such as Gitea.</p> <p>am I missing something obvious here?</p> 2022-10-10T00:00:00+00:00 2022-10-10T00:00:00+00:00 https://www.mat.services/posts/command-line-flake-arguments/

a tutorial on 'breaking' the hermeticity of nix flakes by adding convenient command line flags

]]> <p><a rel="noopener noreferrer" target="_blank" href="https://serokell.io/blog/practical-nix-flakes">Nix flakes</a> are very useful, but the feature of a <a rel="noopener noreferrer" target="_blank" href="https://bazel.build/basics/hermeticity">fully hermetic build</a> also means that they carry with them a certain degree of inflexibility. <a rel="noopener noreferrer" target="_blank" href="https://github.com/NixOS/nix/issues/2861#issuecomment-891521971">Users have asked for a mechanism to parameterize flakes</a>, but there seems to be no interest from the Nix maintainers in adding such a feature.</p> <blockquote> <p>While many readers will be very familiar with the concept of a function and the idea of parameterizing some piece of code or data, others might appreciate a bit of background. <a href="https://www.mat.services/posts/command-line-flake-arguments/#appendix">Take a look at the appendix for a tangential explainer of some fundamental ideas referenced throughout this post.</a></p> </blockquote> <div id=jumpback></div> <p>From outside of a flake's Nix expression, the flake is a black box, a mapping from its input sources to the artifacts that it outputs. There is no built-in way to pass an arbitrary value, a boolean flag or string, from the command line in order to override a deeply nested configuration embedded in your flake. In most cases, this is fine, and you can work around many apparent needs for a one off override by building separate outputs for different purposes. Your flakes will probably come out better designed if you can use them strictly as intended.</p> <p>A real-life use case where a command line override would come in handy comes from my own <a rel="noopener noreferrer" target="_blank" href="https://daiderd.com/nix-darwin/">nix-darwin</a> configuration, where I have my <a rel="noopener noreferrer" target="_blank" href="https://brew.sh/">homebrew</a> formulas and casks specified. I like to let Nix manage installing and upgrading these packages, but unfortunately the process to check for updates eats up a bit of time. When I'm testing a new change to some other part of my nix-darwin setup, I don't like to waste time repeatedly checking for homebrew updates. I could manually disable homebrew updates, but then I'd have to remember to flip it back on. Wouldn't it be nice to have two commands? Something like:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> rebuild the full system flake, when we want to perform homebrew updates </span><span>darwin-rebuild switch --flake </span><span style="color:#ececec;">&lt;</span><span>flake-path</span><span style="color:#ececec;">&gt; </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> rebuild without homebrew, for testing changes more quickly </span><span>darwin-rebuild switch --flake </span><span style="color:#ececec;">&lt;</span><span>flake-path</span><span style="color:#ececec;">&gt; </span><span>--no-homebrew </span></code></pre> <h2 id="ab-using-nix-flake-inputs-to-pass-overrides">(Ab)using Nix flake inputs to pass overrides</h2> <p>As mentioned, flakes are roughly equivalent to pure functions from their (aptly named) inputs to their outputs. Typically, our inputs are Nix expressions or source code that we want to use in our flake, but they can be used to smuggle in any old source code tree we want. What if we parameterized our flake with a special source input that we just use to control some behavior of the build?</p> <p>While Nix has no generalized mechanism for command line overrides to flakes, there is a mechanism for <em>overriding flake inputs</em>: combine that with an input that we use for the special purpose of gating a feature or setting a value, and we have what we want.</p> <h3 id="command-line-flags-for-flakes">Command line flags for flakes</h3> <p>I have totally lost track of my initial encounter with this method, but at some point I came across the <a rel="noopener noreferrer" target="_blank" href="https://github.com/boolean-option"><code>boolean-option</code></a> Github account that includes two repositories of interest: <code>true</code> and <code>false</code>. Upon inspection, each repo includes a trivial flake with a single output <code>value</code>, a boolean Nix value that matches the name of the repo. You can add a command line flag with a desired default value like so:</p> <pre data-lang="nix" style="background-color:#393939;color:#dedede;" class="language-nix "><code class="language-nix" data-lang="nix"><span>{ </span><span> </span><span style="color:#fed6af;">inputs </span><span style="color:#ececec;">= </span><span>{ </span><span> </span><span style="color:#fed6af;">my-flag</span><span>.</span><span style="color:#fed6af;">url </span><span style="color:#ececec;">= </span><span style="color:#d68686;">&quot;github:boolean-option/true&quot;</span><span>; </span><span> }; </span><span> </span><span style="color:#fed6af;">outputs </span><span style="color:#ececec;">= </span><span>{ self</span><span style="color:#ececec;">, </span><span>my-flag }: { </span><span> </span><span style="color:#fed6af;">message </span><span style="color:#ececec;">= </span><span style="color:#fed6af;">if </span><span>my-flag</span><span style="color:#ececec;">.</span><span>value </span><span> </span><span style="color:#fed6af;">then </span><span style="color:#d68686;">&quot;it&#39;s good!&quot; </span><span> </span><span style="color:#fed6af;">else </span><span style="color:#d68686;">&quot;it&#39;s bad..&quot;</span><span>; </span><span> }; </span><span>} </span></code></pre> <p>Now you can build this flake and control the output message from the command line:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>nix eval --raw </span><span style="color:#d6d6d680;">&#39;</span><span style="color:#d68686;">.#message</span><span style="color:#d6d6d680;">&#39; </span><span style="color:#d6d6d680;">&quot;</span><span style="color:#d68686;">it&#39;s good!</span><span style="color:#d6d6d680;">&quot; </span><span> </span><span>nix eval --override-input my-flag github:boolean-option/false --raw </span><span style="color:#d6d6d680;">&#39;</span><span style="color:#d68686;">.#message</span><span style="color:#d6d6d680;">&#39; </span><span style="color:#d6d6d680;">&quot;</span><span style="color:#d68686;">it&#39;s bad..</span><span style="color:#d6d6d680;">&quot; </span></code></pre> <p>So far, every flake-oriented Nix command I have tried supports the <code>override-input</code> flag, so this is a pretty reliable mechanism for passing in overrides where needed.</p> <p><a rel="noopener noreferrer" target="_blank" href="https://git.mat.services/mat/dotfiles.nix/src/branch/main/flake.nix#L16-L17">Take a look</a> <a rel="noopener noreferrer" target="_blank" href="https://git.mat.services/mat/dotfiles.nix/src/branch/main/flake.nix#L256">at how I use this</a> <a rel="noopener noreferrer" target="_blank" href="https://git.mat.services/mat/dotfiles.nix/src/branch/main/darwin/homebrew.nix#L31">in my own system flake</a>.</p> <h2 id="appendix">Appendix</h2> <p><a href="https://www.mat.services/posts/command-line-flake-arguments/#jumpback">Go back to the article</a></p> <h3 id="functions">Functions</h3> <p>In mathematics, a function is a mapping from some domain to a range. This can be expressed another way by saying that a function is a relationship between the elements of two sets, the inputs (or the domain) and the outputs (or the range). Take this simple function for example:</p> <pre style="background-color:#393939;color:#dedede;"><code><span>f(x) = x * 2 </span></code></pre> <p>The mapping can be expressed explicitly to reveal the two sets:</p> <table><thead><tr><th>Input / Domain / <code>x</code></th><th>Output / Range / <code>f(x)</code></th></tr></thead><tbody> <tr><td>...</td><td>...</td></tr> <tr><td>-1</td><td>-2</td></tr> <tr><td>0</td><td>0</td></tr> <tr><td>1</td><td>2</td></tr> <tr><td>...</td><td>...</td></tr> </tbody></table> <p>Functions in programs bear some similarities to mathematical functions, and many mathematical functions can be expressed simply and directly in most programming languages:</p> <pre data-lang="python" style="background-color:#393939;color:#dedede;" class="language-python "><code class="language-python" data-lang="python"><span style="color:#fed6af;">def </span><span style="color:#fffd87;">f</span><span>(x): </span><span> </span><span style="color:#fed6af;">return </span><span>x </span><span style="color:#ececec;">* </span><span style="font-weight:bold;color:#87d6d5;">2 </span></code></pre> <p>Most programming languages, however, do not enforce many (or any) of the laws governing mathematical functions. Functions which depend on global variables, mutable state, and side effects like user input are referred to as <em>impure functions</em>:</p> <pre data-lang="python" style="background-color:#393939;color:#dedede;" class="language-python "><code class="language-python" data-lang="python"><span>history </span><span style="color:#ececec;">= </span><span>[] </span><span> </span><span style="color:#fed6af;">def </span><span style="color:#fffd87;">impure</span><span>(): </span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> global variables </span><span> </span><span style="color:#fffb9d;">global </span><span>x </span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> user input </span><span> y </span><span style="color:#ececec;">= </span><span style="color:#fffd87;">input</span><span>(</span><span style="color:#d6d6d680;">&quot;</span><span style="color:#d68686;">give me y!</span><span style="color:#d6d6d680;">&quot;</span><span>) </span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> mutable state </span><span> history.append((x, y)) </span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> user output, also a side effect </span><span> </span><span style="color:#fffd87;">print</span><span>(x </span><span style="color:#ececec;">+ </span><span>y) </span></code></pre> <h3 id="builds-as-functions">Builds as Functions</h3> <p>In many ways, the process of building software can be compared to a function. In the simplest case, the method of building a particular piece of software could be viewed as a function:</p> <pre data-lang="python" style="background-color:#393939;color:#dedede;" class="language-python "><code class="language-python" data-lang="python"><span style="color:#fed6af;">def </span><span style="color:#fffd87;">build</span><span>(source) </span><span> software </span><span style="color:#ececec;">= </span><span style="color:#fffd87;">compile</span><span>(source) </span><span> </span><span style="color:#fed6af;">return </span><span>software </span></code></pre> <p>For many build systems, the source inputs are taken from the local working directory, and additional parameters can be passed from the command line or environment. For example, <code>make</code> allows you to override variables in the <code>Makefile</code> by passing them along with the rule name:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>make clone BRANCH=dev </span></code></pre> <p><code>docker</code> allows argument and environment variable overrides to be passed to <code>Dockerfile</code> builds, container runs, and <code>docker-compose</code> stacks:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>docker[-compose] build --build-arg HTTP_PROXY=http://123.123.123.123:123 </span><span>docker[-compose] run --env DOMAIN_NAME=foo.bar </span></code></pre> <h3 id="nix-flakes-as-pure-functions">Nix flakes as Pure Functions</h3> <p>If software builds are functions, then Nix flakes are pure functions. Indeed, the structure of a Nix flake is this:</p> <pre data-lang="nix" style="background-color:#393939;color:#dedede;" class="language-nix "><code class="language-nix" data-lang="nix"><span>{ </span><span> </span><span style="color:#fed6af;">description </span><span style="color:#ececec;">= </span><span style="color:#d68686;">&quot;Because functions are confusing without names and explanations&quot;</span><span>; </span><span> </span><span style="color:#fed6af;">inputs </span><span style="color:#ececec;">= </span><span>{ </span><span> </span><span style="color:#fed6af;">builder</span><span>.</span><span style="color:#fed6af;">url </span><span style="color:#ececec;">= </span><span style="color:#d68686;">&quot;...&quot;</span><span>; </span><span> </span><span style="color:#fed6af;">source</span><span>.</span><span style="color:#fed6af;">url </span><span style="color:#ececec;">= </span><span style="color:#d68686;">&quot;...&quot;</span><span>; </span><span> }; </span><span> </span><span style="color:#fed6af;">outputs </span><span style="color:#ececec;">= </span><span>{ self</span><span style="color:#ececec;">, </span><span>builder</span><span style="color:#ececec;">, </span><span>source }: { </span><span> </span><span style="color:#fed6af;">software </span><span style="color:#ececec;">= </span><span>builder</span><span style="color:#ececec;">.</span><span>build source; </span><span> }; </span><span>} </span></code></pre> <p>Which is not too far off from this:</p> <pre data-lang="python" style="background-color:#393939;color:#dedede;" class="language-python "><code class="language-python" data-lang="python"><span style="color:#fed6af;">def </span><span style="color:#fffd87;">flake</span><span>(builder, source): </span><span> </span><span style="color:#fed6af;">return </span><span>builder.build(source) </span></code></pre> <p>Nix flakes are truly pure, and Nix code running in the context of a flake is unable to access things like <code>builtins.currentTime</code>, which would be a side effect.</p> <h3 id="parameters-and-arguments">Parameters and Arguments</h3> <p><em>Parameters</em> and <strong>arguments</strong> are closely related—indeed, they are often confused or conflated entirely. I like to distinguish between them in terms of <em>abstract</em> inputs and <strong>concrete</strong> inputs.</p> <p>Consider our same function from above: <code>x</code> stands in for any given value that our function might receive, without representing a specific value. <code>x</code> is <em>abstract</em>, and also the <em>parameter</em> of our function.</p> <p>On the other hand, specific values of our domain, like -1, 0, and 1 (and actually many other numbers) are <strong>concrete</strong> values that can be supplied to the function to receive a specific output. Any specific value that is passed to our function is an <strong>argument</strong>.</p> <h3 id="parameterizing-a-build">Parameterizing a Build</h3> <p>Now let's tie it all together.</p> <ul> <li>Builds are a bit like functions: <code>build(source) -&gt; software</code></li> <li>Most builds are impure, and let you do whatever you want: access the filesystem and network, check the time of day, flip a coin: <code>build(source, anythingElse) -&gt; software</code></li> <li>Nix flakes are pure, meaning we can't depend on things besides our explicitly specified inputs.</li> </ul> <p>For an &quot;impure&quot; regular build, parameterizing is easy. Define a new variable or argument and pass it at the command line:</p> <pre style="background-color:#393939;color:#dedede;"><code><span>make install PREFIX=$HOME </span><span>docker build --build-arg VERSION v1.2 </span></code></pre> <p>With Nix flakes, we have to parameterize our build with an additional source input. This is definitely a bit of a hack, but it can be very useful when you need it!</p> <p><a href="https://www.mat.services/posts/command-line-flake-arguments/#jumpback">Go back to the article</a></p> <h2 id="acknowledgements">Acknowledgements</h2> <ul> <li>Thank you to <a rel="noopener noreferrer" target="_blank" href="https://github.com/rrbutani">Rahul Butani</a>, the creator of the <code>boolean-option</code> Github account!</li> <li>Thank you to Hollis Druhet, my official copy editor, who has been unofficially helping out with blog edits up until now.</li> </ul> 2022-09-15T00:00:00+00:00 2022-09-15T00:00:00+00:00 https://www.mat.services/posts/cellular-automata-toys/

a short announcement post for a few interactive cellular automata toys written in lua

]]> <p>after reading <a rel="noopener noreferrer" target="_blank" href="https://healeycodes.com/virtual-ants">Andrew Healey's blog post about Langton's Ant</a>, i found myself inspired to reimplement some of his work. i've read numerous rave reviews of löve2d for writing simple toys, so i decided to give that a shot here as well. Langton's Ant went well enough that i also implemented Life and Wireworld.</p> <p><a rel="noopener noreferrer" target="_blank" href="https://git.mat.services/mat/love-cellular-automata">check the toys out here; there are instructions for running with löve2d directly or using nix</a>. the code is hosted on a personal instance of gitea, so if you want to collaborate or share feedback, you'll have to request an account or shoot me an email.</p> 2022-09-04T00:00:00+00:00 2022-10-17T00:00:00+00:00 https://www.mat.services/posts/static-site-with-nix-and-caddy/

a post from mat's blog.

<p><strong>UPDATE</strong> most of the information on this page is no longer accurate for my blog, <a href="https://www.mat.services/posts/now-on-netlify/">which has since moved to netlify</a>. it should still work, or get you close, but i recommend checking out <a rel="noopener noreferrer" target="_blank" href="https://github.com/LutrisEng/nix-fly-template"><code>nix-fly-template</code></a>.</p> <h2 id="motivation">Motivation</h2> <p>So, <a rel="noopener noreferrer" target="_blank" href="https://sfconservancy.org/GiveUpGitHub/">you've ditched Github and friends</a>, <a href="https://www.mat.services/posts/gitea-on-fly-io/">set up your own Gitea instance</a>, and there's just one (read: at least one) thing left for you to take care of—that snazzy static site you had up on Github Pages.</p> <h3 id="pages-and-static-site-hosting">Pages and Static Site Hosting</h3> <p>Github Pages, and the identically named product from Gitlab, have been a part of my personal site workflow for about as long as I have been maintaining a personal site. I can remember how empowering it felt when I finally realized how much utility Github had packed into the Pages product. I went through a gleeful week of churning out Vue.js templates, starry-eyed with how easy Pages made the deployment process.</p> <p>Fast forward to today, and my migration away from Github and Gitlab has me once again pondering the question of an easy-to-use static site host. The tech industry has left me spoiled for choice here. Aside from Pages products from Github and Gitlab, the major cloud infrastructure providers all have their own take on static site hosting. More specialized companies like Netlify, Vercel, and Render offer a lot to hobbyist developers in terms of static hosting resources, with some going even further to provide Serverless-style &quot;Functions-as-a-Service&quot; products. </p> <p>Some of these options were ruled out for me by requiring source code to be hosted on Github and friends, while others were much more heavyweight than what my requirements demanded. I was also motivated by an urge to consolidate: I already had projects running on Fly.io, as well as some important configuration and infrastructure on DigitalOcean, which made me hesitant to bring another third party into the mix.</p> <p>As you may have guessed from the title, my ultimate decision was to take inspiration from <a rel="noopener noreferrer" target="_blank" href="https://fly.io/docs/getting-started/static/">a Fly.io tutorial document</a> explaining how to deploy a very simple vanilla HTML site on Fly using a Go-powered webserver. I'm going to expand on their design a bit by introducing two major components: <a rel="noopener noreferrer" target="_blank" href="https://nixos.org">Nix</a>, to give us the power to build our site with whatever static site generator (or other build process) we want to use; and <a rel="noopener noreferrer" target="_blank" href="https://caddyserver.com">Caddy</a>, to give us a more flexible and extensible platform for actually serving the content.</p> <p>For the purposes of this article, I'll assume you already have your static site ready to go. Whether you're writing pure HTML by hand, or using a cutting-edge Javascript framework that renders down to static resources, you'll be able to package it up with Nix and serve it with Caddy, all hosted on Fly.io.</p> <h2 id="before-you-deploy-nix">Before You Deploy: Nix</h2> <p>Much like static site hosts, static site <strong>generators</strong> are everywhere, and it seems like one of those things where people have particularly strong opinions on what workflow fits them best. Some people eschew a &quot;generator&quot; entirely and chain more purpose-built tools together to achieve the perfect bespoke website output. Whatever your personal choice is for generating the content for your static site, it helps to have a sort of &quot;universal entry point&quot; to actually kick off the build process. Github and Gitlab each have their own custom Continuous Integration/Continuous Delivery systems where the build for the site can be configured. <a rel="noopener noreferrer" target="_blank" href="https://en.wikipedia.org/wiki/Make_(software)#Makefile">Makefiles</a> can declare a set of commands to run in a generic way, but they don't help us pull in dependencies from the outside. <a rel="noopener noreferrer" target="_blank" href="https://docs.docker.com/engine/reference/builder/">Dockerfiles</a> are another generic build specification that also make it easy to fetch any buildtime dependencies we need. Nix, a declarative package manager, is another option that we can use to specify our build and dependencies, without necessarily requiring us to actually make a container (although we can if we want!). Nix also offers more guarantees in terms of reproducibility and hermeticity than Docker. I already use Nix flakes for virtually all of my personal work, so that's how we'll specify our build step today.</p> <p>Nix is a very powerful tool, but it can be intimidating to use, and I've encountered many people saying that the documentation is too sparse to be useful. I'll keep the Nix details minimal, and try to include enough explanation for what we're doing that even unfamiliar readers can follow along, but I would encourage anyone who wants some more background on Nix and Nix flakes to check out <a rel="noopener noreferrer" target="_blank" href="https://xeiaso.net/blog/nix-flakes-1-2022-02-21">these</a> <a rel="noopener noreferrer" target="_blank" href="https://xeiaso.net/blog/nix-flakes-2-2022-02-27">posts</a> from Xe Iaso's blog.</p> <p>If you don't already have Nix installed, we'll start with that:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> macOS </span><span>sh </span><span style="color:#ececec;">&lt;</span><span>(curl -L https://nixos.org/nix/install) </span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Linux multi-user install (recommended by Nix) </span><span>sh </span><span style="color:#ececec;">&lt;</span><span>(curl -L https://nixos.org/nix/install) --daemon </span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Linux single-user install (required when using SELinux) </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> also Windows via WSL2 </span><span>sh </span><span style="color:#ececec;">&lt;</span><span>(curl -L https://nixos.org/nix/install) --no-daemon </span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Docker </span><span>docker run -it nixos/nix </span></code></pre> <p>Next up, we'll enable Nix flakes as a feature, since they are still disabled by default. You can do this the easy way, by editing one of <code>~/.config/nix/nix.conf</code> or <code>/etc/nix/nix.conf</code> (if you're using a multi-user install, you'll also need to restart <code>nix-daemon</code>):</p> <pre data-lang="conf" style="background-color:#393939;color:#dedede;" class="language-conf "><code class="language-conf" data-lang="conf"><span>experimental-features </span><span style="color:#ececec;">=</span><span> nix</span><span style="color:#ececec;">-</span><span>command flakes </span></code></pre> <p>If you enjoy pain, you can do this the hard way by remembering to type this at the beginning of all your Nix commands, or setting an alias:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> one off </span><span>nix --experimental-features </span><span style="color:#d6d6d680;">&#39;</span><span style="color:#d68686;">nix-command flakes</span><span style="color:#d6d6d680;">&#39;</span><span> build </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> sort of consistent but also not really </span><span style="color:#fffd87;">alias nix</span><span style="color:#ececec;">=</span><span style="color:#d68686;">nix </span><span>--experimental-features </span><span style="color:#d6d6d680;">&#39;</span><span style="color:#d68686;">nix-command flakes</span><span style="color:#d6d6d680;">&#39; </span></code></pre> <p>Now we can mosey on over to our site source. You might already be managing the source with Git, but if not, let's do that now:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#fffd87;">cd</span><span> site </span><span>git init </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Be careful not to commit anything secret! Git will keep a record of it </span><span>git add important/stuff but/no/secrets </span><span>git commit -m </span><span style="color:#d6d6d680;">&quot;</span><span style="color:#d68686;">TODO: pithy quip about starting a new endeavor</span><span style="color:#d6d6d680;">&quot; </span></code></pre> <p>Nix flakes require you to be using some form of source control that Nix understands, which means (to my understanding) either Git or Mercurial. Everything tracked by source control will end up going into the Nix store, so be doubly sure that you haven't committed any secrets, tokens, or manifestos that you don't want leaking out.</p> <h3 id="flake-nix-entry-point"><code>flake.nix</code>: Entry Point</h3> <p>We finally have all the foundations in place to put our site's source into a flake. We can get a fresh flake by using the default template:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>nix flake init </span></code></pre> <p>This will leave us with the following:</p> <pre data-lang="nix" style="background-color:#393939;color:#dedede;" class="language-nix "><code class="language-nix" data-lang="nix"><span>{ </span><span> </span><span style="color:#fed6af;">description </span><span style="color:#ececec;">= </span><span style="color:#d68686;">&quot;A very basic flake&quot;</span><span>; </span><span> </span><span style="color:#fed6af;">outputs </span><span style="color:#ececec;">= </span><span>{ self</span><span style="color:#ececec;">, </span><span>nixpkgs }: { </span><span> </span><span style="color:#fed6af;">packages</span><span>.</span><span style="color:#fed6af;">x86_64-linux</span><span>.</span><span style="color:#fed6af;">hello </span><span style="color:#ececec;">= </span><span>nixpkgs</span><span style="color:#ececec;">.</span><span>legacyPackages</span><span style="color:#ececec;">.</span><span>x86_64-linux</span><span style="color:#ececec;">.</span><span>hello; </span><span> </span><span style="color:#fed6af;">defaultPackage</span><span>.</span><span style="color:#fed6af;">x86_64-linux </span><span style="color:#ececec;">= </span><span>self</span><span style="color:#ececec;">.</span><span>packages</span><span style="color:#ececec;">.</span><span>x86_64-linux</span><span style="color:#ececec;">.</span><span>hello; </span><span> }; </span><span>} </span></code></pre> <p>Hmm. Well, that's a VERY basic flake. It's also seemingly out of date, as the latest advice I've seen recommends <code>packages.${system}.default</code> over <code>defaultPackage.${system}</code>. Let's scrap that and pull in a template from the <a rel="noopener noreferrer" target="_blank" href="https://flake.parts"><code>flake-parts</code></a> (<a rel="noopener noreferrer" target="_blank" href="https://github.com/hercules-ci/flake-parts">Github</a>), a flake that bills itself as the <em>&quot;Core of a distributed framework for writing Nix flakes&quot;</em>. </p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>rm flake.nix </span><span>nix flake init --template github:hercules-ci/flake-parts </span></code></pre> <p>In my experience, <code>flake-parts</code> is an extremely helpful tool that provides some Nix functions that drastically reduce the amount of boilerplate you need for a working flake:</p> <pre data-lang="nix" style="background-color:#393939;color:#dedede;" class="language-nix "><code class="language-nix" data-lang="nix"><span>{ </span><span> </span><span style="color:#fed6af;">description </span><span style="color:#ececec;">= </span><span style="color:#d68686;">&quot;Description for the project&quot;</span><span>; </span><span> </span><span> </span><span style="color:#fed6af;">inputs </span><span style="color:#ececec;">= </span><span>{ </span><span> </span><span style="color:#fed6af;">flake-parts</span><span>.</span><span style="color:#fed6af;">inputs</span><span>.</span><span style="color:#fed6af;">nixpkgs</span><span>.</span><span style="color:#fed6af;">follows </span><span style="color:#ececec;">= </span><span style="color:#d68686;">&quot;nixpkgs&quot;</span><span>; </span><span> </span><span style="color:#fed6af;">nixpkgs</span><span>.</span><span style="color:#fed6af;">url </span><span style="color:#ececec;">= </span><span style="color:#d68686;">&quot;github:NixOS/nixpkgs/nixos-unstable&quot;</span><span>; </span><span> }; </span><span> </span><span> </span><span style="color:#fed6af;">outputs </span><span style="color:#ececec;">= </span><span>{ self</span><span style="color:#ececec;">, </span><span>flake-parts</span><span style="color:#ececec;">, ... </span><span>}: </span><span> flake-parts</span><span style="color:#ececec;">.</span><span>lib</span><span style="color:#ececec;">.</span><span>mkFlake { </span><span style="color:#fed6af;">inherit self</span><span>; } { </span><span> </span><span style="color:#fed6af;">imports </span><span style="color:#ececec;">= </span><span>[ </span><span> </span><span style="color:#87ae86;"># To import a flake module </span><span> </span><span style="color:#87ae86;"># 1. Add foo to inputs </span><span> </span><span style="color:#87ae86;"># 2. Add foo as a parameter to the outputs function </span><span> </span><span style="color:#87ae86;"># 3. Add here: foo.flakeModule </span><span> </span><span> ]; </span><span> </span><span style="color:#fed6af;">systems </span><span style="color:#ececec;">= </span><span>[ </span><span style="color:#d68686;">&quot;x86_64-linux&quot; &quot;aarch64-darwin&quot; </span><span>]; </span><span> </span><span style="color:#fed6af;">perSystem </span><span style="color:#ececec;">= </span><span>{ config</span><span style="color:#ececec;">, </span><span>self&#39;</span><span style="color:#ececec;">, </span><span>inputs&#39;</span><span style="color:#ececec;">, </span><span>pkgs</span><span style="color:#ececec;">, </span><span>system</span><span style="color:#ececec;">, ... </span><span>}: { </span><span> </span><span style="color:#87ae86;"># Per-system attributes can be defined here. The self&#39; and inputs&#39; </span><span> </span><span style="color:#87ae86;"># module parameters provide easy access to attributes of the same </span><span> </span><span style="color:#87ae86;"># system. </span><span> </span><span> </span><span style="color:#87ae86;"># Equivalent to inputs&#39;.nixpkgs.legacyPackages.hello; </span><span> </span><span style="color:#fed6af;">packages</span><span>.</span><span style="color:#fed6af;">default </span><span style="color:#ececec;">= </span><span>pkgs</span><span style="color:#ececec;">.</span><span>hello; </span><span> }; </span><span> </span><span style="color:#fed6af;">flake </span><span style="color:#ececec;">= </span><span>{ </span><span> </span><span style="color:#87ae86;"># The usual flake attributes can be defined here, including system- </span><span> </span><span style="color:#87ae86;"># agnostic ones like nixosModule and system-enumerating ones, although </span><span> </span><span style="color:#87ae86;"># those are more easily expressed in perSystem. </span><span> </span><span> }; </span><span> }; </span><span>} </span></code></pre> <p>All we're going to need is <code>systems</code> and <code>perSystem</code>, so let's clean up the template to look something like this:</p> <pre data-lang="nix" style="background-color:#393939;color:#dedede;" class="language-nix "><code class="language-nix" data-lang="nix"><span>{ </span><span> </span><span style="color:#fed6af;">description </span><span style="color:#ececec;">= </span><span style="color:#d68686;">&quot;Statically generated site&quot;</span><span>; </span><span> </span><span> </span><span style="color:#fed6af;">inputs </span><span style="color:#ececec;">= </span><span>{ </span><span> </span><span style="color:#fed6af;">nixpkgs</span><span>.</span><span style="color:#fed6af;">url </span><span style="color:#ececec;">= </span><span style="color:#d68686;">&quot;github:NixOS/nixpkgs/nixos-unstable&quot;</span><span>; </span><span> </span><span> </span><span style="color:#fed6af;">flake-parts</span><span>.</span><span style="color:#fed6af;">url </span><span style="color:#ececec;">= </span><span style="color:#d68686;">&quot;github:hercules-ci/flake-parts&quot;</span><span>; </span><span> </span><span style="color:#fed6af;">flake-parts</span><span>.</span><span style="color:#fed6af;">inputs</span><span>.</span><span style="color:#fed6af;">nixpkgs</span><span>.</span><span style="color:#fed6af;">follows </span><span style="color:#ececec;">= </span><span style="color:#d68686;">&quot;nixpkgs&quot;</span><span>; </span><span> }; </span><span> </span><span> </span><span style="color:#fed6af;">outputs </span><span style="color:#ececec;">= </span><span>{ self</span><span style="color:#ececec;">, </span><span>flake-parts</span><span style="color:#ececec;">, ... </span><span>}: </span><span> flake-parts</span><span style="color:#ececec;">.</span><span>lib</span><span style="color:#ececec;">.</span><span>mkFlake { </span><span style="color:#fed6af;">inherit self</span><span>; } { </span><span> </span><span style="color:#87ae86;"># modify these as needed if you&#39;re using a different system </span><span> </span><span style="color:#fed6af;">systems </span><span style="color:#ececec;">= </span><span>[ </span><span style="color:#d68686;">&quot;x86_64-linux&quot; &quot;aarch64-darwin&quot; </span><span>]; </span><span> </span><span style="color:#fed6af;">perSystem </span><span style="color:#ececec;">= </span><span>{ config</span><span style="color:#ececec;">, </span><span>self&#39;</span><span style="color:#ececec;">, </span><span>inputs&#39;</span><span style="color:#ececec;">, </span><span>pkgs</span><span style="color:#ececec;">, </span><span>system</span><span style="color:#ececec;">, ... </span><span>}: { </span><span> </span><span style="color:#fed6af;">devShells</span><span>.</span><span style="color:#fed6af;">default </span><span style="color:#ececec;">= </span><span style="color:#fffd87;">import </span><span style="color:#d68686;">./shell.nix </span><span>{ </span><span style="color:#fed6af;">inherit pkgs</span><span>; }; </span><span> </span><span style="color:#fed6af;">packages</span><span>.</span><span style="color:#fed6af;">default </span><span style="color:#ececec;">= </span><span>pkgs</span><span style="color:#ececec;">.</span><span>callPackage </span><span style="color:#d68686;">./site.nix </span><span>{}; </span><span> }; </span><span> }; </span><span>} </span></code></pre> <p>This won't build right away. First we're going to have to add the two <code>.nix</code> files we used.</p> <h3 id="shell-nix-reproducible-development-environment"><code>shell.nix</code>: Reproducible Development Environment</h3> <p><code>shell.nix</code> is a common feature of many Nix builds, and typically specifies the packages that are used for developing and iterating on a project. If you haven't used Nix before, you likely have the dependencies and tools for your static site installed using your system package manager, or using a language-specific build tool. By switching to <code>shell.nix</code>, you can decouple the project development environment from your local system, akin to using a Docker container for development. If you don't want or need any special tools to build your site, you can mostly ignore this file, but here's what this might look like for a simple static site:</p> <pre data-lang="nix" style="background-color:#393939;color:#dedede;" class="language-nix "><code class="language-nix" data-lang="nix"><span>{ pkgs </span><span style="color:#ececec;">? </span><span style="color:#fffd87;">import </span><span style="color:#d68686;">&lt;nixpkgs&gt; </span><span>}: </span><span>pkgs</span><span style="color:#ececec;">.</span><span>mkShell { </span><span> </span><span style="color:#fed6af;">buildInputs </span><span style="color:#ececec;">= </span><span>[ </span><span> </span><span style="color:#87ae86;"># build your static site with one of these </span><span> pkgs</span><span style="color:#ececec;">.</span><span>hugo </span><span> </span><span style="color:#87ae86;"># or pkgs.zola or pkgs.jekyll </span><span> </span><span> </span><span style="color:#87ae86;"># deploy with fly </span><span> pkgs</span><span style="color:#ececec;">.</span><span>flyctl </span><span> ]; </span><span>} </span></code></pre> <h3 id="site-nix-reproducible-site-build"><code>site.nix</code>: Reproducible Site Build</h3> <p>We'll write <code>site.nix</code> as a function from packages in <code>nixpkgs</code> to a derivation, which will let us call it more conveniently with <code>pkgs.callPackage</code>, as we did above in <code>flake.nix</code>. Here's an example of a site build with <code>hugo</code>:</p> <pre data-lang="nix" style="background-color:#393939;color:#dedede;" class="language-nix "><code class="language-nix" data-lang="nix"><span>{ stdenv</span><span style="color:#ececec;">, </span><span>hugo</span><span style="color:#ececec;">, </span><span>scour }: </span><span>stdenv</span><span style="color:#ececec;">.</span><span>mkDerivation { </span><span> </span><span style="color:#fed6af;">name </span><span style="color:#ececec;">= </span><span style="color:#d68686;">&quot;static-site&quot;</span><span>; </span><span> </span><span style="color:#fed6af;">src </span><span style="color:#ececec;">= </span><span style="color:#d68686;">./.</span><span>; </span><span> </span><span style="color:#fed6af;">nativeBuildInputs </span><span style="color:#ececec;">= </span><span>[ </span><span> </span><span style="color:#87ae86;"># specify site build dependencies here </span><span> hugo </span><span> </span><span style="color:#87ae86;"># optimize SVGs </span><span> scour </span><span> ]; </span><span> </span><span style="color:#fed6af;">buildPhase </span><span style="color:#ececec;">= </span><span style="color:#d68686;">&#39;&#39; </span><span style="color:#d68686;"> # prepare and build the site </span><span style="color:#d68686;"> scour -i favicon-original.svg -o favicon.svg </span><span style="color:#d68686;"> hugo -D </span><span style="color:#d68686;"> &#39;&#39;</span><span>; </span><span> </span><span style="color:#fed6af;">installPhase </span><span style="color:#ececec;">= </span><span style="color:#d68686;">&#39;&#39; </span><span style="color:#d68686;"> # install the Hugo output </span><span style="color:#d68686;"> cp -r public $out </span><span style="color:#d68686;"> &#39;&#39;</span><span>; </span><span>} </span></code></pre> <p>If we need to build any other auxiliary outputs, like Docker images (hint!), we can add them here. For now, let's just save our progress:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>git add flake.nix shell.nix site.nix </span><span>nix flake lock </span><span>git add flake.lock </span><span>git commit -m </span><span style="color:#d6d6d680;">&quot;</span><span style="color:#d68686;">Initialize Nix flake</span><span style="color:#d6d6d680;">&quot; </span></code></pre> <p>Now we can test that our flake works with <code>nix build</code>:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>nix build </span><span>ls result </span><span>index.html main.css ... </span></code></pre> <h2 id="serving-caddy">Serving: Caddy</h2> <p>We can reliably build our site, but now we need a way to serve it on the <a rel="noopener noreferrer" target="_blank" href="https://xkcd.com/181/">blagoblag</a>. Let's use Caddy! The syntax is marginally less arcane than Apache or Nginx, and it has cool features like HTTPS-by-default!</p> <p>Sadly, the first thing we're going to have to do in our <code>Caddyfile</code> is turn that off:</p> <pre data-lang="Caddyfile" style="background-color:#393939;color:#dedede;" class="language-Caddyfile "><code class="language-Caddyfile" data-lang="Caddyfile"><span style="color:#87ae86;"># fly.io handles https for us </span><span>{ </span><span> auto_https off </span><span>} </span><span> </span><span>:8080 { </span><span> </span><span style="color:#fffd87;">root</span><span> * </span><span style="font-weight:bold;color:#d6d6ae;">{$SITE_ROOT} </span><span> </span><span style="color:#fffd87;">encode</span><span> gzip </span><span> </span><span style="color:#fffd87;">file_server </span><span> </span><span> </span><span style="color:#87ae86;"># redirect to your custom 404 page </span><span> </span><span style="color:#fffd87;">handle_errors</span><span> { </span><span> @404 { </span><span> </span><span style="color:#fed6af;">expression</span><span> {http.error.status_code} == 404 </span><span> } </span><span> </span><span style="color:#fed6af;">rewrite</span><span> @404 /404.html </span><span> </span><span style="color:#fed6af;">file_server </span><span> } </span><span>} </span></code></pre> <p>You can test this by adding <code>caddy</code> to your <code>shell.nix</code> file or else installing it locally, and running something like this:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>nix build </span><span>env SITE_ROOT=result caddy validate </span><span>2022/08/27 19:40:48.724 WARN http automatic HTTPS is completely disabled for server {</span><span style="color:#d6d6d680;">&quot;</span><span style="color:#d68686;">server_name</span><span style="color:#d6d6d680;">&quot;</span><span>: </span><span style="color:#d6d6d680;">&quot;</span><span style="color:#d68686;">srv0</span><span style="color:#d6d6d680;">&quot;</span><span>} </span><span>Valid configuration </span><span>env SITE_ROOT=result caddy run </span></code></pre> <p>You should be able to browse to your site at <code>127.0.0.1:8080</code> and load it, although some resources may load improperly or not at all if they are expected to be accessed at a particular hostname. When we deploy to Fly, however, everything should be working. Add the Caddyfile to git:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>git add Caddyfile </span><span>git commit -m </span><span style="color:#d6d6d680;">&quot;</span><span style="color:#d68686;">Add Caddyfile</span><span style="color:#d6d6d680;">&quot; </span></code></pre> <h2 id="deploying-fly-io">Deploying: Fly.io</h2> <p>We can scaffold a new Fly app in the usual way:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>flyctl launch \ </span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> something unique, doesn&#39;t matter if you&#39;re going to use a custom domain </span><span> --name seals-meander-daringly </span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> region where the app runs, don&#39;t supply this option if you want to interactively choose a region \ </span><span> --region ewr \ </span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> don&#39;t immediately deploy, we need to edit our fly.toml first \ </span><span> --no-deploy </span><span> </span><span>git add fly.toml </span><span>git commit -m </span><span style="color:#d6d6d680;">&quot;</span><span style="color:#d68686;">Initialize Fly app</span><span style="color:#d6d6d680;">&quot; </span></code></pre> <p>We need to find some way for Fly to build a VM from our application and Caddyfile. Fly supports Dockerfiles, so let's just go ahead and start with that:</p> <pre data-lang="Dockerfile" style="background-color:#393939;color:#dedede;" class="language-Dockerfile "><code class="language-Dockerfile" data-lang="Dockerfile"><span style="color:#fed6af;">FROM</span><span> nixos/nix:latest </span><span> </span><span style="color:#fed6af;">WORKDIR </span><span>/code </span><span style="color:#fed6af;">ADD </span><span>. /code </span><span style="color:#fed6af;">RUN </span><span>nix \ </span><span> --extra-experimental-features nix-command \ </span><span> --extra-experimental-features flakes \ </span><span> build </span><span> </span><span style="color:#fed6af;">FROM</span><span> caddy:latest </span><span> </span><span style="color:#fed6af;">COPY</span><span> Caddyfile /etc/caddy/Caddyfile </span><span style="color:#fed6af;">COPY</span><span> --from=0 /code/result /var/www </span><span style="color:#fed6af;">ENV </span><span>SITE_ROOT=/var/www </span><span style="color:#fed6af;">RUN </span><span>caddy </span></code></pre> <p>We can use a multi-stage build to run the Nix build first, then copy that into a container with the Caddyfile and run that. We're set to deploy!</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>flyctl deploy </span></code></pre> <p>Now we can set up a custom domain if we want:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>flyctl ips list </span><span>VERSION IP TYPE REGION CREATED AT </span><span>v4 1.2.3.4 public global 2022-08-09T02:19:27Z </span><span>v6 aaaa:bbbb:1::a:cccc public global 2022-08-09T02:19:29Z </span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> add DNS A and AAAA records for the above addresses </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> e.g., using doctl for digitalocean </span><span>doctl compute domain records create \ </span><span> --record-type A \ </span><span> --record-name my </span><span> --record-data 1.2.3.4 \ </span><span> static.site </span><span> </span><span>doctl compute domain records create\ </span><span> --record-type AAAA \ </span><span> --record-name my </span><span> --record-data aaaa:bbbb:1::a:cccc \ </span><span> static.site </span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> get a certificate </span><span>flyctl certs add my.static.site </span></code></pre> <p>A small addition to the Caddyfile will be helpful, as well:</p> <pre data-lang="Caddyfile" style="background-color:#393939;color:#dedede;" class="language-Caddyfile "><code class="language-Caddyfile" data-lang="Caddyfile"><span>http://seals-meander-daringly.fly.dev { </span><span> </span><span style="color:#fffd87;">redir</span><span> https://my.static.site </span><span>} </span></code></pre> <p>Don't forget to <code>flyctl deploy</code> again!</p> <h2 id="bonus-round-configure-everything-in-nix">Bonus Round: Configure Everything in Nix</h2> <p>&quot;But mat!&quot; you're shouting in anguish, &quot;Why do we have to write a crummy Dockerfile to build our software? You said several sections ago that Nix could be the universal entry point, and that it was better than Docker for some important sounding reasons!&quot;</p> <p>I know. What's more is, you're entirely right. We DON'T have to settle for a Dockerfile! Nix has some tooling available to build our Docker images for us, and we can plug that right into our Fly.io application.</p> <p>Let's add a <code>container.nix</code> file:</p> <pre data-lang="nix" style="background-color:#393939;color:#dedede;" class="language-nix "><code class="language-nix" data-lang="nix"><span>{ dockerTools</span><span style="color:#ececec;">, </span><span>caddy</span><span style="color:#ececec;">, </span><span>site }: </span><span> dockerTools</span><span style="color:#ececec;">.</span><span>buildLayeredImage { </span><span> </span><span style="color:#fed6af;">name </span><span style="color:#ececec;">= </span><span style="color:#d68686;">&quot;static-site&quot;</span><span>; </span><span> </span><span style="color:#fed6af;">tag </span><span style="color:#ececec;">= </span><span style="color:#d68686;">&quot;2022-08-28&quot;</span><span>; </span><span> </span><span> </span><span style="color:#fed6af;">config </span><span style="color:#ececec;">= </span><span>{ </span><span> </span><span style="color:#fed6af;">Cmd </span><span style="color:#ececec;">= </span><span>[ </span><span style="color:#d68686;">&quot;</span><span style="color:#898989;">${</span><span style="color:#d68686;">caddy</span><span style="color:#898989;">}</span><span style="color:#d68686;">/bin/caddy&quot; &quot;run&quot; &quot;-config&quot; &quot;</span><span style="color:#898989;">${</span><span style="color:#d68686;">./Caddyfile</span><span style="color:#898989;">}</span><span style="color:#d68686;">&quot; </span><span>]; </span><span> </span><span style="color:#fed6af;">Env </span><span style="color:#ececec;">= </span><span>[ </span><span> </span><span style="color:#d68686;">&quot;SITE_ROOT=</span><span style="color:#898989;">${</span><span style="color:#d68686;">site</span><span style="color:#898989;">}</span><span style="color:#d68686;">&quot; </span><span> ]; </span><span> }; </span><span> } </span></code></pre> <p>Add it to source control:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>git add container.nix </span><span>git commit -m </span><span style="color:#d6d6d680;">&quot;</span><span style="color:#d68686;">Add Docker image build</span><span style="color:#d6d6d680;">&quot; </span></code></pre> <p>And then refer to that in our <code>flake.nix</code>:</p> <pre data-lang="nix" style="background-color:#393939;color:#dedede;" class="language-nix "><code class="language-nix" data-lang="nix"><span>{ </span><span> </span><span style="color:#87ae86;"># ... </span><span> </span><span style="color:#fed6af;">outputs </span><span style="color:#ececec;">= </span><span>{ self</span><span style="color:#ececec;">, </span><span>flake-parts</span><span style="color:#ececec;">, ... </span><span>}: </span><span> flake-parts</span><span style="color:#ececec;">.</span><span>lib</span><span style="color:#ececec;">.</span><span>mkFlake { </span><span style="color:#fed6af;">inherit self</span><span>; } { </span><span> </span><span style="color:#87ae86;"># ... </span><span> </span><span style="color:#fed6af;">perSystem </span><span style="color:#ececec;">= </span><span>{ config</span><span style="color:#ececec;">, </span><span>self&#39;</span><span style="color:#ececec;">, </span><span>inputs&#39;</span><span style="color:#ececec;">, </span><span>pkgs</span><span style="color:#ececec;">, </span><span>system</span><span style="color:#ececec;">, ... </span><span>}: { </span><span> </span><span style="color:#87ae86;"># ... </span><span> </span><span style="color:#fed6af;">packages</span><span>.</span><span style="color:#fed6af;">container </span><span style="color:#ececec;">= </span><span>pkgs</span><span style="color:#ececec;">.</span><span>callPackage </span><span style="color:#d68686;">./container.nix </span><span>{ </span><span> </span><span style="color:#fed6af;">site </span><span style="color:#ececec;">= </span><span>config</span><span style="color:#ececec;">.</span><span>packages</span><span style="color:#ececec;">.</span><span>default; </span><span> }; </span><span> }; </span><span> }; </span><span>} </span></code></pre> <p>You can test that the build works like so, including running it if you have a local Docker installation:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>nix build .#container </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> if you have a working Docker installation </span><span>docker load </span><span style="color:#ececec;">&lt;</span><span> result </span><span>docker run -itp 8080:8080 static-site:2022-08-28 </span><span>curl http://</span><span style="color:#fed6af;">[</span><span>::</span><span style="color:#fed6af;">]</span><span>:8080 </span></code></pre> <p>Some people, myself included, don't care to run Docker engine on their Macbooks, so the easiest way for us to test this would be to package up the deployment step into a script and run it on a Linux host with Docker, such as in a container on a CI/CD server.</p> <p>Let's add another <code>callPackage</code> friendly Nix file:</p> <pre data-lang="nix" style="background-color:#393939;color:#dedede;" class="language-nix "><code class="language-nix" data-lang="nix"><span>{ lib</span><span style="color:#ececec;">, </span><span>docker</span><span style="color:#ececec;">, </span><span>flyctl</span><span style="color:#ececec;">, </span><span>formats</span><span style="color:#ececec;">, </span><span>writeShellScriptBin</span><span style="color:#ececec;">, </span><span>dockerImage }: </span><span>writeShellScriptBin </span><span style="color:#d68686;">&quot;deploy&quot; &#39;&#39; </span><span style="color:#d68686;"> set -euxo pipefail </span><span style="color:#d68686;"> export PATH=&quot;</span><span style="color:#898989;">${</span><span style="color:#d68686;">lib</span><span style="color:#ececec;">.</span><span style="color:#d68686;">makeBinPath [(docker</span><span style="color:#ececec;">.</span><span style="color:#d68686;">override { </span><span style="color:#fed6af;">clientOnly </span><span style="color:#ececec;">= </span><span style="font-weight:bold;color:#d6d6ae;">true</span><span style="color:#d68686;">; }) flyctl]</span><span style="color:#898989;">}</span><span style="color:#d68686;">:$PATH&quot; </span><span style="color:#d68686;"> archive=</span><span style="color:#898989;">${</span><span style="color:#d68686;">dockerImage</span><span style="color:#898989;">} </span><span style="color:#d68686;"> image=$(docker load &lt; $archive | awk &#39;{ print $3; }&#39;) </span><span style="color:#d68686;"> flyctl deploy -i $image </span><span style="color:#d68686;">&#39;&#39; </span></code></pre> <p>And plug it into the flake:</p> <pre data-lang="nix" style="background-color:#393939;color:#dedede;" class="language-nix "><code class="language-nix" data-lang="nix"><span>{ </span><span> </span><span style="color:#87ae86;"># ... </span><span> </span><span style="color:#fed6af;">outputs </span><span style="color:#ececec;">= </span><span>{ self</span><span style="color:#ececec;">, </span><span>flake-parts</span><span style="color:#ececec;">, ... </span><span>}: </span><span> flake-parts</span><span style="color:#ececec;">.</span><span>lib</span><span style="color:#ececec;">.</span><span>mkFlake { </span><span style="color:#fed6af;">inherit self</span><span>; } { </span><span> </span><span style="color:#87ae86;"># ... </span><span> </span><span style="color:#fed6af;">perSystem </span><span style="color:#ececec;">= </span><span>{ config</span><span style="color:#ececec;">, </span><span>self&#39;</span><span style="color:#ececec;">, </span><span>inputs&#39;</span><span style="color:#ececec;">, </span><span>pkgs</span><span style="color:#ececec;">, </span><span>system</span><span style="color:#ececec;">, ... </span><span>}: { </span><span> </span><span style="color:#87ae86;"># ... </span><span> </span><span style="color:#fed6af;">apps</span><span>.</span><span style="color:#fed6af;">deploy </span><span style="color:#ececec;">= </span><span>pkgs</span><span style="color:#ececec;">.</span><span>callPackage </span><span style="color:#d68686;">./deploy.nix </span><span>{ </span><span> </span><span style="color:#fed6af;">dockerImage </span><span style="color:#ececec;">= </span><span>config</span><span style="color:#ececec;">.</span><span>packages</span><span style="color:#ececec;">.</span><span>container; </span><span> }; </span><span> }; </span><span> }; </span><span>} </span></code></pre> <p>Now you should be able to use a command like this on a Docker-friendly host, and your site will be up and running before long:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>nix run .#deploy </span></code></pre> <h3 id="future-directions">Future Directions</h3> <p>There is just a touch of boilerplate left in <code>flake.nix</code>, required to thread the correct nixpkgs context through (<code>callPackage</code>, passing <code>site</code> and <code>dockerImage</code> explicitly). We could easily package that up in a <code>flake-module.nix</code> and add it to the <code>imports</code> argument of <code>flake-parts.lib.mkFlake</code>. I'll probably try to make another blog post out of the process of packaging up the Nix glue, so keep your eyes peeled!</p> <p>Thanks for reading!</p> <h2 id="acknowledgements">Acknowledgements</h2> <ul> <li><a rel="noopener noreferrer" target="_blank" href="https://github.com/LutrisEng/nix-fly-template">Thanks to Lutris, Inc. for their <code>nix-fly-template</code>, which was very influential in the writing of this post.</a></li> <li>Thanks to Hollis Druhet for reading the draft of this post and offering feedback!</li> </ul> 2022-08-07T00:00:00+00:00 2022-09-04T00:00:00+00:00 https://www.mat.services/posts/gitea-on-fly-io/

a post from mat's blog.

<h2 id="setting-the-scene">Setting the Scene</h2> <p>Inspired by the <a rel="noopener noreferrer" target="_blank" href="https://sfconservancy.org/GiveUpGitHub/">Give Up Github campaign</a>, I recently decided I wanted to spin up my own instance of <a rel="noopener noreferrer" target="_blank" href="https://gitea.io/">Gitea</a>. There are free (as in beer), free (as in freedom), public instances of Gitea and other FOSS-leaning code forges, but self-hosted Gitea struck me as a nice way to take even a bit more ownership over my own code.</p> <p>I maintain a small PC running in my home as a server for a few services running via Proxmox, but I am really dissatisfied with my workflows for managing that box lately (picture SSHing into LXC containers and manually editing systemd configurations... yuck). I recently read a couple of different articles singing the praises of <a rel="noopener noreferrer" target="_blank" href="https://fly.io/">Fly.io</a>, a platform as a service (PaaS) that replicates most of the good parts of the classic Heroku developer experience. Further enticed by their ample free tier, I took the plunge and created a Fly.io account.</p> <h2 id="getting-started">Getting Started</h2> <p>First things first: in order to interact with Fly.io, we primarily use the <code>flyctl</code> command line tool. It's available from a variety of sources:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Nix </span><span>nix-shell --packages flyctl </span><span>nix-env --install --attr nixpkgs.flyctl </span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> macOS </span><span>brew install flyctl </span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Linux + non-Homebrew users </span><span>curl -L https://fly.io/install.sh </span><span style="color:#ececec;">| </span><span>sh </span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Windows users </span><span>iwr https://fly.io/install.ps1 -useb </span><span style="color:#ececec;">| </span><span>iex </span></code></pre> <p><code>flyctl</code> can handle signing up for a Fly.io account if you haven't done that, otherwise you can use it to sign in:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Opens your browser to set up an account </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> This step requires a credit card, in the event you exceed the free tier limits </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> See https://fly.io/docs/about/pricing/ for details </span><span>flyctl auth signup </span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Opens your browser to sign in </span><span>flyctl auth login </span></code></pre> <h2 id="configuring-apps-with-flyctl">Configuring Apps with <code>flyctl</code></h2> <p>In order to avoid even the appearance of disorganization, we'll start off with a git repository for tracking our Fly.io configurations:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>mkdir fly-apps</span><span style="color:#ececec;">; </span><span style="color:#fffd87;">cd</span><span> fly-apps </span><span>git init </span><span>mkdir gitea</span><span style="color:#ececec;">; </span><span style="color:#fffd87;">cd</span><span> gitea </span></code></pre> <p>Next up, we create our app configuration and register it with Fly.io. <code>flyctl</code> takes care of this for us in a single command, <code>flyctl launch</code>. The command will prompt you interactively for some input, but here we'll just pass some flags directly:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>flyctl launch \ </span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> use the official Gitea docker image \ </span><span> --image gitea/gitea:latest \ </span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> give our instance a unique name, this will be used to generate a development hostname like gitea-mat-services.fly.dev \ </span><span> --name gitea-mat-services \ </span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> region where the app runs, don&#39;t supply this option if you want to interactively choose a region \ </span><span> --region ewr \ </span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> don&#39;t immediately deploy, we need to edit our fly.toml first \ </span><span> --no-deploy </span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> don&#39;t forget to commit! </span><span>git add fly.toml </span><span>git commit -m </span><span style="color:#d6d6d680;">&quot;</span><span style="color:#d68686;">Add default generated configuration</span><span style="color:#d6d6d680;">&quot; </span></code></pre> <p>We will also need a volume for persisting git repositories:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Create a 3GB volume </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Note: this will consume your entire free tier allotment for volume storage! </span><span>flyctl volumes create gitea_data </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Alternatively, create a smaller volume </span><span>flyctl volumes create gitea_data --size 1 </span></code></pre> <p>Now our app is ready to deploy, but first we're going to make some changes to the <code>fly.toml</code> configuration that was generated for us. <a rel="noopener noreferrer" target="_blank" href="https://blog.gitea.io/2022/04/running-gitea-on-fly.io/">Gitea has their own post on Fly.io which includes a sample configuration</a>, and we can reference that to end up with something like the following:</p> <pre data-lang="diff" style="background-color:#393939;color:#dedede;" class="language-diff "><code class="language-diff" data-lang="diff"><span style="color:#ffffff;">&lt;snip&gt; </span><span style="color:#ffffff;"> </span><span style="color:#ffffff;">[env] </span><span style="color:#60b38a;">+ GITEA____APP_NAME = &quot;git.mat.services: Gitea for me&quot; </span><span style="color:#60b38a;">+ GITEA__database__DB_TYPE = &quot;sqlite3&quot; </span><span style="color:#60b38a;">+ GITEA__database__PATH = &quot;/data/gitea/gitea.db&quot; </span><span style="color:#60b38a;">+ GITEA__server__DOMAIN = &quot;gitea-on-fly.fly.dev&quot; </span><span style="color:#60b38a;">+ GITEA__server__SSH_DOMAIN = &quot;gitea-on-fly.fly.dev&quot; </span><span style="color:#60b38a;">+ GITEA__server__ROOT_URL = &quot;https://gitea-on-fly.fly.dev&quot; </span><span style="color:#60b38a;">+ GITEA__security__INSTALL_LOCK = &quot;true&quot; # Don&#39;t show installer </span><span style="color:#60b38a;">+ # GITEA__service__DISABLE_REGISTRATION = &quot;true&quot; # TODO: uncomment once you have created your first user </span><span style="color:#ffffff;"> </span><span style="color:#60b38a;">+ [mounts] </span><span style="color:#60b38a;">+ destination = &quot;/data&quot; </span><span style="color:#60b38a;">+ source = &quot;gitea_data&quot; </span><span style="color:#ffffff;"> </span><span style="color:#ffffff;">&lt;snip&gt; </span><span style="color:#ffffff;"> </span><span style="color:#60b38a;">+ # ssh traffic </span><span style="color:#60b38a;">+ [[services]] </span><span style="color:#60b38a;">+ internal_port = 22 </span><span style="color:#60b38a;">+ protocol = &quot;tcp&quot; </span><span style="color:#60b38a;">+ [[services.ports]] </span><span style="color:#60b38a;">+ port = 22 </span><span style="color:#ffffff;"> </span><span style="color:#ffffff;">[[services]] </span><span style="color:#ffffff;">&lt;snip&gt; </span><span style="color:#ca7172;">- internal_port = 8080 </span><span style="color:#60b38a;">+ internal_port = 3000 </span><span style="color:#ffffff;">&lt;snip&gt; </span></code></pre> <p>Don't forget to save your work!</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>git add fly.toml </span><span>git commit -m </span><span style="color:#d6d6d680;">&quot;</span><span style="color:#d68686;">Add Gitea specific configuration</span><span style="color:#d6d6d680;">&quot; </span></code></pre> <h2 id="liftoff">Liftoff!</h2> <p>The stars are aligned. The witching hour has arrived. Let's deploy our app:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>flyctl deploy </span></code></pre> <p>After <code>flyctl</code> does its thing, your Gitea instance should be up and running at <code>https://&lt;your-app-name&gt;.fly.dev</code>! For the extremely lazy, <code>flyctl</code> will do the work for you:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>flyctl open </span></code></pre> <p>Now you can register your account! If you want to keep your Gitea instance private, uncomment the last line in the <code>env</code> section and rerun <code>flyctl deploy</code> after registering.</p> <h2 id="bonus-round-configuring-your-custom-domain-with-fly-io">Bonus Round: Configuring Your Custom Domain with Fly.io</h2> <p>Some, like myself, will not be satisfied accessing their Gitea instances with a <code>.fly.dev</code> URL. Thankfully, Fly.io makes it a breeze to configure a secure custom domain with the help of <a rel="noopener noreferrer" target="_blank" href="https://letsencrypt.org/">LetsEncrypt</a>.</p> <p>Fly.io and <code>flyctl</code> seem to offer tooling for managing your domains and DNS records entirely within the Fly.io system, but I personally didn't explore this route, as my own domains and DNS records are managed elsewhere. Whatever you use to manage your own domains and DNS, you will need to create two records for your domain name corresponding with the IPv4 and IPv6 addresses of the application:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>flyctl ips list </span><span>TYPE ADDRESS REGION CREATED AT </span><span>v4 1.2.3.4 global 2022-07-03T04:39:18Z </span><span>v6 dead:beef:1::a:a global 2022-07-03T04:39:19Z </span></code></pre> <p>Create an A record for the v4 address, and an AAAA record for the v6 address. With those records in place, <code>flyctl</code> can automatically perform the necessary setup with LetsEncrypt to secure your domain:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>flyctl certs add git.mat.services </span><span>flyctl open </span></code></pre> <p>Now it's time to revel in your new code hosting powers.</p> <h3 id="the-first-push-or-256mb-just-isn-t-enough-sometimes">The First Push (or, 256MB just isn't enough sometimes)</h3> <p>With everything now set up, let's go ahead and kick the tires on our new Gitea installation. We can push our <code>fly-apps</code> repository to Gitea as a test. Create a new blank repository on Gitea with an appropriate name, then add that as a remote for your local repo:</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>git remote add origin git@git.mat.services:mat/fly-apps.git </span></code></pre> <p>And now it's as easy as one simple command!</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>git push --set-upstream origin main </span><span>Enumerating objects: 3, done. </span><span>Counting objects: 100</span><span style="color:#fed6af;">%</span><span> (3/3), done. </span><span>Writing objects: 100</span><span style="color:#fed6af;">%</span><span> (3/3), 195 bytes </span><span style="color:#ececec;">| </span><span>195.00 KiB/s, done. </span><span>Total 3 (delta 0), reused 0 (delta 0), pack-reused 0 </span><span>remote: </span><span>remote: Gitea: Internal Server Error </span><span>Gitea: Internal error </span><span>To git.mat.services:mat/fly-apps.git </span><span> </span><span style="color:#ececec;">! </span><span>[remote rejected] main -</span><span style="color:#ececec;">&gt;</span><span> main (pre-receive hook declined) </span><span>error: failed to push some refs to </span><span style="color:#d6d6d680;">&#39;</span><span style="color:#d68686;">git.mat.services:mat/fly-apps.git</span><span style="color:#d6d6d680;">&#39; </span></code></pre> <p>Uh. Hm. I just got an email saying a Fly.io instance ran out of memory and crashed. Let's peek at our Fly.io dashboards:</p> <figure> <img alt="Fly.io memory dashboard" src=fly-io-memory-dashboard.webp /> <figcaption><p>Fly.io memory dashboard</p></figcaption> </figure> <p>That doesn't look so great. It seems like Gitea idles just under the amount of memory we have with a default instance size, and operations like <code>git push</code> can bump it over the threshold to an out-of-memory error. Let's check out the &quot;Scale&quot; section of the dashboard, and increase the memory allotment for this VM:</p> <figure> <img alt="Fly.io VM scaling interface" src=fly-io-scale-vm.webp /> <figcaption><p>Fly.io VM scaling interface</p></figcaption> </figure> <p>You can also use <code>flyctl</code> to scale your app!</p> <pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>flyctl scale memory 512 </span></code></pre> <p>I have been running my Gitea install on a 512MB instance since the first day I started using it, which seems to be plenty of headroom for personal use. If you open up your Gitea installation to the public and it starts to get popular, you might end up needing to scale up further.</p> <p>It's worth noting that scaling up to 512 MB means you will start accruing a Fly.io balance, but so far this has never been more than a $2 monthly bill for me. If that's too steep for you, consider that Fly.io doesn't (at the time of this writing) charge for monthly bills below $5, or else check out some of the competing PaaS options out there.</p> <h2 id="acknowledgements">Acknowledgements</h2> <ul> <li><a rel="noopener noreferrer" target="_blank" href="https://xeiaso.net/blog/fly.io-heroku-replacement">Thank you to Xe Iaso for xer blog post on Fly.io that inspired me to try it in the first place!</a></li> <li><a rel="noopener noreferrer" target="_blank" href="https://blog.gitea.io/2022/04/running-gitea-on-fly.io/">Thank you to techknowlogick for their instructive post on running Gitea on Fly.io!</a></li> </ul>