Tony Finn - Personal Blog

Jujutsu (jj), a git compatible VCS

2024-10-14T00:00:00+00:00

It's quite a common opinion that git (while a big improvement on what came before) still has plenty of rough edges, particularly with regards to the user interface. At the same time, there's also a significant barrier to entry for a new version control system with all the tooling that's built up around git over the years, particularly if you want to try it in your workplace without migrating the entire company.

jj</a> is one of the latest round of git-compatible version control systems which allows you to have a better experience locally, without having to abandon everything that depends on git. I've been trying it out now for about two months, and this post shares some of my thoughts.

Building Arch Packages With Nix

2023-01-23T00:00:00+00:00

In recent months I've been building more and more of my packages with Nix. However, most of my personal systems are still running Arch. While NixOS is making headways there and is probably the eventual destination for all my systems, I'm not interested in migrating all my personal infrastructure all at once.

Instead, what I really want to do is having my Nix tooling output Arch packages that I can install on those systems in this transitional period. After some experimentation, I got it working for one of my packages, this blog, and this post details the steps it took.

Runnable Flakes (Nix From First Principles: Flake Edition #9)

2022-11-10T00:00:00+00:00

In the last two parts, I covered using flakes for two different types of output: packages</a> and developer environments</a>. This time I'll cover a third type of output - runnable commands. These are what power the nix run</code> command that you've seen a few times throughout the series, and now you will find out how to get this functionality for your own flakes.



Flakes and Developer Environments (Nix From First Principles: Flake Edition #8)
2022-11-07T00:00:00+00:00
While packages are the most common type of item you'll find contained in flakes,
another type of item is a development shell</em>. This is an environment that you can
enter with the nix develop</code> command. By default, each package will provide a development
shell which is the build environment used to build it. By running just nix develop</code>
without specifying which shell is wanted, nix</code> will put you in a bash</code> shell with the
environment from the default package.</p>
However, sometimes you may wish to set up additional tools in a development environment,
for example you may have scripts written in Python and want to have Python available as
for a developer to use, yet you do not want to include Python as a dependency in your built
package. Or you might want to set some environment variables so that logging is set to
debug mode in your development environment.</p>
You can customise the nix develop</code> environment with the devShells</code> attribute set in a
flake.…
</p>


What about flakes then? (Nix From First Principles: Flake Edition #7)
2022-11-06T00:00:00+00:00
One of the big new concepts in Nix is the flake</code>, which is a new standard
format for nix projects to declare all their outputs. At the beginning of
this series, I mentioned them as being one of the major components of modern
Nix, and now this series has introduced enough of a foundation it's time
to explain flakes themselves.</p>
What is a flake?</h2>

Why the name 'flake'?</h3>
Nix's logo is a snow flake. Nix itself means snow in Latin. So Nix flakes
are a pun on snow flakes.</p>
</aside>
A flake</code> is a standard format for describing a collection of Nix resources.
These resources can be packages, of the type described in previous posts,
intended to be used to install some software to your system. There's also a
number of other types of resources that can be exposed by the flake which
we'll cover in later posts. Some examples of these are:</p>

developer environment descriptions</li>
modules for configuring NixOS systems</li>
runnable commands</li>
</ul>
By standardising a format to list all these outputs (compared to the ad-hoc
files this series has been using until now), Nix allows a number of higher level
tools to operate on these resources, like the nix profile install</code> and nix run</code>
commands that have been shown in earlier parts.</p>
Finally, the development of the flake based tooling allowed the Nix team
to resolve some pain points with the older generation of tooling, such
as making the process of recording versions for reproducibility more ergonomic with
lock files, or making better ways to identify newer versions of the same
package to allow tools to manage updates.</p>


Nixpkgs - How Not to Reinvent the Wheel (Nix From First Principles: Flake Edition #6)
2022-10-12T00:00:00+00:00
In the last section I discussed creating your first derivation, which
allows you to make a first nix package. As you might have noticed, the
default execution environment is incredibly barebones, to the point that
you needed to include such fundamental tools as chmod</code> and cp</code>. If that
process had to be repeated by every Nix user, it would be very inconvenient.
Luckily, there is nixpkgs, which provides a number of packages that the
community has already built, along with the standard environment which
includes a number of tools to use in building your own. You may remember
installing some of these packages in part 3</a>.</p>


Overview (Nix From First Principles: Flake Edition #1)
2022-10-05T00:00:00+00:00
One technology that I'd been hearing more and more over the last two years has been
Nix, in its various forms. The big draws I've heard about include declarative,
portable, cross lang dependency specification, and the ability to have a config file
declare the entire state of a system without storing huge docker container artifacts
for every permutation.</p>
Actually learning Nix though felt much harder than it needed to be. The biggest problem
is that the documentation you need is often fragmented among the different subprojects,
and also the transition that Nix is in at the moment from the current standard methods
of default.nix</code>, configuration.nix</code> and shell.nix</code> to a new world of flakes</code>.</p>
As an outsider looking in, the flake world felt kind of in limbo at the moment
where anyone I talked to indicated that flakes fix so many of the problems with
current nix, but also they're still officially experimental so the official documentation
declines to mention flakes.</p>
After spending a fair amount of time understanding Nix for myself, I've decided to write my own guide
which explains it from scratch in a world using only the new Nix features,
with the nix</code> CLI and flakes for the use cases I want from Nix. Credit
to ianthehenry</a> for his guide on Nix focused on the "old world" approach, and Xe Iaso</a>
for explaining flakes. The information included in this guide
is largely from synthesising their blogs and the official documentation.</p>
This first post will describe the major parts of the Nix ecosystem.</p>


Installation (Nix From First Principles: Flake Edition #2)
2022-10-05T00:00:00+00:00
Nix can be installed on Linux or Mac systems. If you're using the Nix based Linux
distribution, NixOS, then you'll already have it installed. Otherwise the official
nix website provides the most up to date installation steps</a>, which
at the time of writing are</p>
Linux:</p>
sh</span> <(</span>curl</span> -L https://nixos.org/nix/install) --daemon</span></span></code></pre>
Mac:</p>
sh</span> <(</span>curl</span> -L https://nixos.org/nix/install)</span></span></code></pre>
Windows (run in WSL):</p>
sh</span> <(</span>curl</span> -L https://nixos.org/nix/install) --no-daemon</span></span></code></pre>


Nix Package Manager CLI (Nix From First Principles: Flake Edition #3)
2022-10-05T00:00:00+00:00
The foundational part of Nix is the Nix package manager. This is a tool that
allows you to install packages. The "first layer" of Nix usage is to use this
tool to install packages. The Nix package manager, similar to projects like
os-rpmtree (used in Fedora Silverblue) or snapper (used in snap based Linux distros),
will allow you to easily revert package installs or updates if something goes wrong.</p>
This post covers the basics of using the nix package manager at the command
line - while most users use nix through more advanced workflows, the basics
explained here will be useful in understanding those workflows, as well as
for users who want to dip their toes into Nix by treating it like a more
traditional package manager to start out.</p>


Just Enough Nixlang (Nix From First Principles: Flake Edition #4)
2022-10-05T00:00:00+00:00
This is a crash course in Nix's language, which I'll also sometimes call Nixlang
from here on, to avoid confusing it with the Nix package manager. For more detailed
instructions see the manual</a>. The Nix language is used heavily
within Nix, for defining packages, environments, runnable commands and library
functions.</p>
Nix's language is expression oriented. This means everything, including up to
the level of an entire program, has an output value. It will be most familiar
if you have experimented with functional langages such as Haskell, but it is
less complicated than those languages, so you don't need to have mastered one
to start.</p>
To start experimenting with it, you can run the nix repl</code>
command which will let you type in Nix expressions and see the output.</p>


Derivations (Nix From First Principles: Flake Edition #5)
2022-10-05T00:00:00+00:00
So, time to explain the nix derivation. A Nix derivation</code> is a special
type of set</code> created from the derivation</code> function which describes
how to obtain and build a package. The Nix package manger will then evaluate
this derivation, and use the result to copy the built package into the Nix
store, which is effectively a folder with all locally Nix built or installed
packages in it. The subfolders are hashes derived from the inputs to the
expression that built the stored packages.</p>


Preserve - Jellyfin Music Client
2021-01-31T00:00:00+00:00

In the last few months, I've moved away from using Spotify and back to using my own music collection, largely
driven by tracks which I include in my playlists disappearing off the service due to licensing issues. Moving fully
back to local media has left to issues keeping my library in sync across devices, so I've begun using Jellyfin,
an open source media server to store my collection and make it available to all my devices. However, the default
web interface in Jellyfin did not provide the same efficiency I liked from Clementine, my music player I used locally.
Much like foobar2000, Clementine has a very efficient interface for the way in which I tend to listen to music - i.e.
rapidly creating a playlist of disparate tracks via search, listening to it for a few days or weeks, then making
a new collection. The same is not true for Jellyfin's web interface (and wasn't true for Spotify either). So I built my
own client to replicate that experience.</p>
Preserve</h2>
</div>
Preserve</a> is a web based media player frontend to a jellyfin server. It aims to
replicate the experience of using players like foobar2000 or Clementine with its two panel design allowing you to quickly
search tracks and assemble/order a playlist how you like, including full keyboard controls. It's fully open source, under GPLv2+.</p>
Since it is a jellyfin client, you will first need to host
your music using a Jellyfin server, but once that is done you can use Preserve in your browser at preserveplayer.com</a>
or install a desktop client from the Preserve Gitlab releases</a>.</p>


    
        
        
        
    </picture>
</a>
Preserve v0.5.3</figcaption>
</figure>


Home Server Part 1, Requirements and Hardware
2019-09-07T00:00:00+00:00
Having recently built a new gaming desktop, I still had enough components that
weren't reused to decide what to do with them. I'd also been meaning to build
some sort of server setup for a while, so it proved a good opportunity to
convert my old desktop into a server. In the coming weeks, I'll post a couple
more posts detailing my software setup but for now I'll go over my goals and
hardware.</p>
Requirements</h2>
So what did I need a home server for anyway?</p>
File sync</h3>
I've used a number of solutions for this over the years. Dropbox had been my
goto but late last year they dropped support for filesystems other than
ext4</a>
which was a wake up call to investigate other options. While they've since
relented, it was during this time I discovered
Syncthing</a> which took my interest. As someone who
likes to have control over their data, its nature as a P2P tool appealed to
me. This meant that my data no longer had to leave servers I control for sync.</p>
This was especially relevant because of my setup for password management. I've
never been entirely happy entrusting my passwords to a third party service
like LastPass, but wanted the benefits of a password manager. This meant that
I used a keepass database in Dropbox for syncing passwords between devices, on
the grounds that someone would have to both breach Dropbox and break KeePass's
encryption to obtain my passwords. But I still wasn't entirely happy having my
passwords on a cloud service. In addition, the Android experience just was not
there, as the best I could do was open the dropbox copy in KeePassDroid which
was a one way export and occasionally expired meaning I'd have to log in again
to access the passwords again. It also ruled out using a password manager
password on Dropbox itself.</p>
Syncthing ended up being a very useful tool to replace Dropbox, and I'd tried
it out with a couple of use cases. I moved my password database sync to there,
and my RPG rulebook PDFs. However, there was one area it fell down, and that
was syncing between my desktop and my tablet for files which both modified,
such as my password database.</p>
Because it was P2P and had no persistent host, to sync my password database
required my tablet to be awake and my desktop to be on at the same time, which
was a rare occurence. As such my password database ended up getting out of
sync and having merge conflicts which need to be resolved. I tried using my
phone as a always-on sync host for that folder which worked a bit, but it
would often take a while to sync because of the phone being asleep or killing
the syncthing app when low on resources.</p>
This meant that I needed an always-on host to ensure that all devices had
somewhere to sync that would keep changes from getting too divergent. I could
just run syncthing on my VPS, but that puts me back with the Dropbox scenario
of having to entrust someone else (linode) with my password database. And as
my VPS was internet facing, the risk of compromise is much higher.</p>
Home streaming</h3>
I have a variety of content I've ripped from physical media or stripped DRM,
as well as from sources like Bandcamp artists that simply aren't on streaming
services over the years. Sometimes I would like to watch or listen this away
from my desktop such as on my tablet which doesn't have the capacity to store
much. This meant when I wanted to watch some movies in bed for the evening
that were not available on any of my subscribed streaming services, I had to
copy them manually onto my device each time. Which usually meant I didn't
watch them over whatever was on Netflix at the time.</p>
Backups</h3>
My backup strategy over the years has been varying levels of existent. Most
recently it has consisted of putting code into private gitlab repos and other
highly important files into Dropbox. This protected me against hard drive
failure for the covered content, but there was other content (Twitch stream
archives, aforementioned ripped movies) which was just too large to
economically store in Dropbox. Also as people would be rushing to mention on a
similar post, Dropbox is not really a backup - it's online and if compromised,
someone could encrypt my Dropbox files with ransomware in the event of a
breach and the client would dutifully overwrite my local copies.</p>
The other issue was that I wanted something easier to restore. While I could
get back to a working system without anything too major lost in the event of a
hard drive failure or similar, when actually put to the test it was a painful
experience. While I did get back to a working system, the time spent restoring
applications, their configs, games with mods etc. left a lot to be desired.
All this content could be pulled back from the internet, but I wanted
something a bit more "restore from backup, be back like nothing happened".</p>
Fun</h3>
While this whole project may not be most people's idea of fun, but it'd let me
play around with technologies that I didn't often use, which I always find
interesting.</p>
Hardware Setup</h2>
As previously mentioned, I had built a new gaming desktop so most of the
hardware for this actually are parts from my previous desktop. So my system
composed of the i5-4670k, 16gb of DDR3 RAM and gigabyte z87 motherboard used
in my old system. In addition, when it was my gaming system, the storage setup
was a bit of a mess. It had started out simply enough with a 256GB SSD + 2TB
HDD, but as it was serving part time duty as a file server already, it had
grown to 2 SSDs (a 256 gb 840 Pro and 1TB 850 Pro). 2x2TB HDDs and 2x4TB HDDs
(All WD blue drives). Most of these were formatted with NTFS as that worked
best for drives shared between Linux and Windows. Also, when I upgraded my GPU
to a GTX 1080, I actually had to remove most of the drive bays to fit the GPU,
with the result that one of the SSDs was just loose in the case.</p>
As it turns out, having a wall of HDDs in your gaming desktop is not great for
airflow.</p>
So the plan was the new gaming desktop is much simpler from a storage
perspective. It has a new 1TB 970 Pro SSD for the Windows/boot drive, as
that's where X-Plane 11 lives and is the most IO-heavy use, and the 1TB 850
Pro also went to the new build to be my Linux root drive.</p>
Everything else stayed in the old build. In addition, moving the gtx1080 to my
new system let me put the drive trays back in. So now the system has the 840
Pro SSD as the boot/OS drive, and 2x2TB + 2x4TB of hard drives for
storage.</p>
While it had been used as my gaming desktop, the i5-4670k had been overclocked
up to 4.3ghz, but as part of this setup that was reversed for reasons of heat
and power consumption - the server uses weren't demanding enough to require
the overclock to stay, as this had mostly been done for the purposes of
running X-Plane 11 at an acceptable framerate.</p>


Screenshot testing with Rust
2018-12-18T00:00:00+00:00
Intro</h2>
This is a follow on from my previous blog post about capturing screenshots with Rust + OpenGL. While the previous article
most revolved around the process of actually capturing the rendered pixel data for my OpenGL application, this time around
I will focus on the process of actually validating the screenshots.</p>


Capturing screenshots with Rust + OpenGL
2018-12-15T00:00:00+00:00
Intro</h2>
One ongoing area of side projects for me has been game development. It's an area which
I've often found interesting, and while the only games I've completed so far have
been small games for Ludum Dare with the deadline hanging over, I've often had a few
larger side projects running around.</p>
In general for these side projects, I tend to write a lot of the lower level code myself,
mostly because it's something that I enjoy learning about. Part of this means that I can't
just rely on the framework or engine rendering my game correctly, and instead need some way
to test it. Unit tests clearly aren't enough here, while they can verify that I'm
calling the functions I expect, they really can't verify that this results in something
cohesive being rendered to the screen.</p>
Inspired by tools that exist in my day job of web development like Percy</a>,
I decided to take the approach of comparing screenshots taken to reference screenshots.</p>
For some reason, I was under the impression that Factorio</a> had
used a similar approach and detailed it in one of their FFF blogs, but when researching for this blog
post, I was unable to track this down. They certainly perform integration testing</a>
but it appears screenshots are not involved.</p>

Tony Finn - Personal Blog

Jujutsu (jj), a git compatible VCS

Building Arch Packages With Nix

Runnable Flakes (Nix From First Principles: Flake Edition #9)

Flakes and Developer Environments (Nix From First Principles: Flake Edition #8)

What about flakes then? (Nix From First Principles: Flake Edition #7)

Nixpkgs - How Not to Reinvent the Wheel (Nix From First Principles: Flake Edition #6)

Overview (Nix From First Principles: Flake Edition #1)

Installation (Nix From First Principles: Flake Edition #2)

Nix Package Manager CLI (Nix From First Principles: Flake Edition #3)

Just Enough Nixlang (Nix From First Principles: Flake Edition #4)

Derivations (Nix From First Principles: Flake Edition #5)

Preserve - Jellyfin Music Client

Home Server Part 1, Requirements and Hardware

Requirements</h2> So what did I need a home server for anyway?</p>

Fun</h3> While this whole project may not be most people's idea of fun, but it'd let me play around with technologies that I didn't often use, which I always find interesting.</p>

Screenshot testing with Rust

Capturing screenshots with Rust + OpenGL

Requirements</h2>
So what did I need a home server for anyway?</p>

Fun</h3>
While this whole project may not be most people's idea of fun, but it'd let me play around with technologies that I didn't often use, which I always find interesting.</p>