DEV Community: Ruairí O'Brien

My Dev Tool List 2025

Ruairí O'Brien — Tue, 02 Dec 2025 20:34:34 +0000

TL;DR

On Mac:

brew install --cask nikitabobko/tap/aerospace

brew install atuin chezmoi gh fzf eza bat ripgrep starship git-delta fd tmux stern mise jj hl

On Linux with apt:

sudo apt install -y \
  fzf \
  fd-find \
  ripgrep \
  bat \
  tmux


# eza (modern ls)
sudo apt install -y gpg
sudo mkdir -p /etc/apt/keyrings
wget -qO- https://raw.githubusercontent.com/eza-community/eza/main/deb.asc | sudo gpg --dearmor -o /etc/apt/keyrings/gierens.gpg
echo "deb [signed-by=/etc/apt/keyrings/gierens.gpg] http://deb.gierens.de stable main" | sudo tee /etc/apt/sources.list.d/gierens.list
sudo apt update && sudo apt install -y eza


# delta (git pager)
DELTA_VERSION="0.18.2"
wget https://github.com/dandavison/delta/releases/download/${DELTA_VERSION}/git-delta_${DELTA_VERSION}_amd64.deb
sudo dpkg -i git-delta_${DELTA_VERSION}_amd64.deb
rm git-delta_${DELTA_VERSION}_amd64.deb

# mise (version manager)
curl https://mise.run | sh

# jj (Jujutsu)
curl -LsSf https://github.com/jj-vcs/jj/releases/latest/download/jj-x86_64-unknown-linux-musl.tar.gz | tar xzf - -C ~/.local/bin jj

# gh (GitHub CLI)
curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | sudo dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | sudo tee /etc/apt/sources.list.d/github-cli.list
sudo apt update && sudo apt install -y gh

# Setup chezmoi and pull your dotfiles
sh -c "$(curl -fsLS get.chezmoi.io)" -- init --apply YOUR_GITHUB_USERNAME

AeroSpace window manager for macOS

It angers me how much time I wasted dragging things around the screen before. If you use a mac, give it a try.

brew install --cask nikitabobko/tap/aerospace

Atuin modern shell history

Sync your shell history across devices. Only useful if you're working on multiple machines, which I do and this is extremely useful.

brew install atuin
echo 'eval "$(atuin init zsh)"' >> ~/.zshrc

Atuin also comes with a pretty cool desktop app which I love the idea of but have yet to do anything useful with.

Chezmoi dotfile and config sync

I regularly work on at least 3 different machines. This makes everything way easier to keep in sync and is a lot better than the custom git setup I made a few times.

brew install chezmoi

chezmoi init

chezmoi add ~/.bashrc # or .zshrc or whatever

GitHub CLI

Obviously, if you use GitHub. Makes everything way easier.

brew install gh

Example, creating a new repo:

gh repo create cool-repo --public --source=. --remote=origin

fzf fuzzy finder cli

When you've got bad memory and can't type, like me, this is a very useful tool.

brew install fzf

Then you can use Ctrl-r to search.

eza modern ls replacement

A common theme is I like colours and icons, particularly in the terminal. eza is a modern replacement for ls with all that built in.

brew install eza

bat a cat clone with syntax highlighting and git integration

Cat a file and see colours. Also shows git changes inline.

brew install bat

I alias cat to bat. You can do this with atuin's dotfile sync if you're using that:

atuin dotfiles alias add cat bat

ripgrep fast grep replacement

Something so impressively fast you can't help but use it.

brew install ripgrep

starship cross-shell prompt

I just like the colours and things. Spent too much time configuring it maybe but I like it.

brew install starship

git-delta git pager with syntax highlighting and side-by-side diffs

If you use git diff a lot, this will save your eyes.

brew install git-delta

Update your ~/.gitconfig:

[core]
    pager = delta

[interactive]
    diffFilter = delta --color-only

[delta]
    navigate = true    # use n and N to move between diff sections

    # delta detects terminal colors automatically; set one of these to disable auto-detection
    # dark = true
    # light = true

[merge]
    conflictStyle = zdiff3

fd find replacement

I have never remembered the correct arguments for find. This is a modern replacement that I like.

brew install fd

e.g. from the docs:

> cd /etc
> fd '^x.*rc$'
X11/xinit/xinitrc
X11/xinit/xserverrc

tmux terminal multiplexer

It took me way too long to get on the tmux train. What was I even doing before?

brew install tmux

# Start a session in a directory
tmux new-session -d -s my-session -c ~/some-dir

# Attach to session
tmux attach -t my-session

Leave ssh sessions, or anything really, running in the background and come back to them later.

stern multi-pod log tailing for Kubernetes

The amount of times I've opened a bunch of terminal windows trying to tail logs in a multi instance setup. This is much easier than any of the hacky solutions I came up with before.

brew install stern

mise version manager for CLI tools

I was a chef briefly in a past life so the name speaks to me. This is a tool like nvm, pyenv etc. but supports a collection of CLI tools.

brew install mise

Example installing python 3.14 globally:

mise use --global python@3.14

jj a version control system that works with git

This is the most recent addition for me. I don't know why I like it yet. I just know I do.

brew install jj

Setting it up in an existing git repo:

jj git init --colocate

hl high-performance log viewer

No more getting wrekt by huge log files.

brew install hl

hl /path/to/huge/logfile.log

# or many log files
hl /path/to/logs/*.log

lazyvim Neovim config

I'm forever trying to use vim and then giving up, falling back to VSCode. I think this one is starting to stick.

Coding Agents

Of course. Wrote a detailed post about how I'm using those:

Ruairí O'Brien

Nov 30 '25

A Gaggle of Agents

#ai #programming #agents

Comments 2

12 min read

A Gaggle of Agents

Ruairí O'Brien — Sun, 30 Nov 2025 18:56:23 +0000

There's a lot of hype around coding agents and the discourse can be annoying.

I've always been into new shiny tools though. I'm forever messing with editors, IDEs, CLIs and anything else that seemed useful. Many of which no longer exist.

In college we learned to code in assembly. C seemed very simple by comparison. Then object oriented languages and scripting seemed quaint. IDEs with refactoring and autocomplete made coding much easier. Tools like React and the ability to install npm packages made hacking a UI together somewhat less miserable and then before long somehow much more miserable. I had to take time to learn each new thing before it became useful but I enjoyed the learning. Now it's LLMs and agents. They're different but it's a similar feeling.

It started with GitHub copilot and fancy autocomplete. Then web chat interfaces, copying and pasting code back and forth like StackOverflow. Now we've got coding agents.

Agents are a useful tool because you can give them work that you would otherwise have to do yourself and they'll just go and try to do it, with varying degrees of success. Using techniques I discuss here, I try to increase the frequency of success.

Using the term "agents" in this context, I'm thinking about instances of a coding agent in one context window. A single LLM thread, primed with some context and with access to a set of tools.

Agents are limited by their context window so you can only do so much before they start to go a bit crazy. Most LLMs are currently Autoregressive so performance varies depending on what's in the context at the time. At least for now, it makes sense to optimize running many small agent sessions over one long one and therefore crafting a workflow that is good for that.

Here's my list of characteristics for a good coding agent setup:

A good context primer (AGENTS.md) so each new agent starts well.
Context primers for different types of work, most coding agents now support custom sub-agents for this.
A way for agents to track and remember work.
The ability to run multiple agents in parallel.
A good set of tools for the agents to use.
Sensible defaults for things like permitted commands.

For my own stuff, I use Claude Code. For work I use GitHub Copilot. In this post I'll focus on Claude Code and will likely follow up with a Copilot version.

Example Project Setup

My choice of project here might not be ideal and unless you do iOS development, you may not be able to repeat the steps on your own machine (which is fine) but it does illustrate something about coding agents. I'm not an iOS developer. This is a project I've wanted to do for years but would probably never have bothered with, until now and until coding agents made it easier.

I made a windows phone game when I was in college, showing incredible foresight into where that technology was heading.

I want to port the game over to iOS and I want to learn about native iOS development using Swift in the process.

How I setup the project is not important, you could use any codebase here instead but here's what I did anyway:

Create a new game in XCode.
Set SpriteKit as the Game Technology.
Set Swift as the language.
Selected include iOS Application and include macOS Application (including macOS because why not)

All the code for the sample project is up here https://github.com/ruarfff/Nathaniel

After creating the project I setup a .gitignore file for XCode projects.

I'm using the jj VCS here, which I recommend looking into for this kind of setup, but you can use a standard git commit/git push flow instead if you prefer.

Setup jj in the repo:

jj git init --colocate

Create a GitHub repo for the project using the GitHub CLI:

gh repo create Nathaniel --public --source=. --remote=origin

Then commit the gitignore:

# Describe the working change, like git commit:
jj describe -m "Add gitignore"

# Put all changes on main:
jj bookmark set main -r @

#Push changes to the GitHub repo:
jj git push

# Or if creating a new branch which I needed to do first time here:
jj git push --allow-new

I do this 3 step, describe, bookmark, push process all the time. There might be some easier way but this is the habit I've formed.

At this point, there are 2 commits. The initial commit from XCode and the gitignore commit.

Now those commits are up on GitHub here and here.

Then I took my old Windows Phone app code and put it into a directory called Legacy.

I hit a warning trying to commit that using jj. Some game assets are large so I needed to configure max size:

jj config set --repo snapshot.max-new-file-size 2049292

Then:

jj describe -m "Add old windows phone 7 game code."
jj bookmark set main -r @
jj git push

Code ended up here https://github.com/ruarfff/Nathaniel/tree/main/Legacy

Hopefully that gives you an idea of the code we're working with. Now it's time to prepare the agent coding configurations.

Coding Agent Setup with Task Management and Memory

There is a tool called beads https://github.com/steveyegge/beads that is an interesting concept for managing agent work. Agents can use it to create and track issues. It has a graph structure to help an agent navigate historical tasks without loading everything into context unnecessarily.

There are other similar tools like GitHub spec-kit that track things in yaml files and provide tools to help agents manage work with scripts, custom commands etc. There will likely be many more options over time but right now, beads is the one I find most intuitive and with the lowest cognitive load.

The beads repo has a good setup guide but here's what I did:

# Install beads
brew tap steveyegge/beads
brew install bd

# Set it up in the project
bd init

This gets beads ready by creating some files in the repo, some you commit, some you don't. Beads provides a .gitignore.

jj describe -m "Add initial beads setup."
jj bookmark set main -r @
jj git push

AGENTS.md

I am using claude code so, in the project directory I run claude. I run /model in the prompt input to make sure I'm on the latest and greatest, Opus 4.5 at time of writing. Then I run /init to allow claude to set things up for itself.

Claude will generate a CLAUDE.md file.

By default, claude code uses a CLAUDE.md file. I prefer using AGENTS.md since it appears to be an emerging standard and just makes things a little easier if I want to use other coding agents in the same project.

# Create an AGENTS.md file

touch AGENTS.md

One option to make it use AGENTS.md is to symlink, e.g. ln -s AGENTS.md CLAUDE.md

What I decided to do is copy the contents of CLAUDE.md into AGENTS.md and update CLAUDE.md to:

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.


Read instructions from AGENTS.md:

@AGENTS.md

@AGENTS.md should essentially template in the contents so claude will use what's in there.

Next I asked claude to run bd onboard (I ran claude and said "please run bd onboard")

This updated the AGENTS.md file. It also created a copilot instructions file which I decided to delete.

jj describe -m "Add AGENTS.md and run bd onboard."
jj bookmark set main -r @
jj git push

That results in this commit.

Coming up with a Plan

Usually working on a project I'd have a good idea of what I want to do and how to do it. This time I really don't have a clue. Claude, and most other coding agents, have the concept of sub agents. One that Claude comes with is a plan agent.

I run Claude, hit Shift + Tab twice, and give it this prompt (Claude is very forgiving of my typos and bad spelling):

Familiarize yourself wiht this codebase. The goal of the project is to port an old C#, XNA based windows phone 7
game to Swift + SpriteKit to run on iOS and Mac. We will need to port all the code plus migrate the assets and
figure out a game engine. We'll also need to figure out a new approach for the tiled level editor. Please think hard
on this, build out a roadmap and use beads to create epics and tasks.

The planning agent spins up multiple agents to get through the task faster:

I then get prompted to give some feedback and make some decisions:

I get a summary of the decisions:

Then the agent gets to work creating a roadmap in beads:

Because I really don't know what to do here, i.e. I've never used Swift, SpriteKit and don't know how to go about porting the game, I'm letting the Claude plan agent figure out most of the details for me. Usually I'd provide a lot more direction and be more interactive, creating tasks step by step. Sometimes I'd create a beads task directly but usually I'd tell the agent what I want to happen and have it create the tasks for me.

Something I haven't shown here so far is Claude asking me for permissions to do stuff. I do have some things I always allow and others I prefer to be prompted for, just to feel like I have some control. See https://code.claude.com/docs/en/settings for how to manage these settings.

At this point, Claude created a plan but I wasn't paying attention and it went off and started working on it too.

It ran through some of the changes and they did not work. I decided to commit anyway since the plan is in there:

jj describe -m "Add plan and initial content pipeline"
jj bookmark set main -r @
jj git push

Resulting in this commit

The app was failing to load saying it couldn't load maps. I toggled planning mode (Shift + Tab twice) and asked Claude to come up with a plan to fix it.

Claude went and got some tasks ready in beads. At this point I want to start a new context window. Always worth clearing the context when there's a pause.

I type /clear and hit Shift + Tab to toggle 'accept edits' on.

So with a fresh context I prompt:

Please review tasks in beads and pick up the highest priority issue.

This is essentially my workflow. Plan, prioritize, execute. Get the agent to do as much of the work as possible.

Custom Agents

The plan agent (Shift + Tab twice) is pretty cool. It's an example of a custom agent.

Custom agents have some nice features, like their own context window and the ability to prime them with instructions that you can run repeatedly. You can also curate the tools they have access to.

A fair comment might be that custom agents are almost always overkill and that's probably true but I found two use cases I liked for this project.

There's the review agent, literally the example in the docs, so I won't go into detail on that.

The other agent I find really useful is the game tester agent. An agent that runs the game and makes sure it's more or less working. I will explain how I set that up.

When you run Claude, you can execute an /agents command to see the list of agents you have.

You can see the agents I created here and also the 'Create new agent' option. If you select that, you will be guided through the process of setting an agent up.

To create the iOS tester agent, I ran that, saying I wanted to create an agent for testing an iOS game project. I knew I'd need some tools for it to use so I install the iOS Simulator MCP tool:

claude mcp add ios-simulator -- npx -y ios-simulator-mcp --env IOS_SIMULATOR_MCP_DEFAULT_OUTPUT_DIR=~/dev/Nathaniel/test-artifacts

I also created a document with a test plan to help guide the agent https://github.com/ruarfff/Nathaniel/blob/main/docs/ios-test-plan.md. I could probably make this even more advanced using skills but this is good enough for now.

Here's the agent definition I ended up with https://github.com/ruarfff/Nathaniel/blob/main/.claude/agents/ios-tester.md.

I can run the agent just by telling Claude to use the agent by name:

Running Agents In Parallel

You don't want to be waiting around for agents to finish. You probably want to be able to kick off multiple tasks and check in on them later.

Git has the concept of workspaces. So does jj. I'll use jj here of course but git workspaces work in a similar fashion.

Here's a simple example. I see two tasks that should be safe enough to do in parallel.

Create Weapon protocol and Gun (Nathaniel-0gc)
Port A* pathfinding (Nathaniel-6wt)

I'll create two workspaces:

jj workspace add ../nathaniel-agent1
jj workspace add ../nathaniel-agent2

This creates two directories at the same level as the working directory with copies of the code.

Prepare agent 1:

cd ../nathaniel-agent1
jj new main -m "Agent 1: Create Weapon protocol and Gun (Nathaniel-0gc)"

Prepare Agent 2:

cd ../nathaniel-agent2
jj new main -m "Agent 2: Port A* pathfinding (Nathaniel-6wt)"

cd ../Nathaniel # back to main

Setup the current change and see how the workspaces look:

 jj describe -m "Implement game mechanics"

❯ jj log
@  onvtmuyl ruairi 2025-11-29 15:52:10 default@ 9280b0d0
│  Implement game mechanics
│ ○  puxmrlmu ruairi 2025-11-29 15:50:31 nathaniel-agent2@ e946ae92
├─╯  (empty) Agent 2: Port A* pathfinding (Nathaniel-6wt)
│ ○  ooxnksvr ruairi 2025-11-29 15:50:08 nathaniel-agent1@ e1912bb8
├─╯  (empty) Agent 1: Create Weapon protocol and Gun (Nathaniel-0gc)
◆  nonoyurv ruairi 2025-11-29 15:39:35 main git_head() 30c1d06c
│  Fix game loading - game actually renders and character control works.

Create a tmux session for agent 1:

tmux new-session -d -s agent1 -c ~/dev/nathaniel-agent1

Create a tmux session for agent 2:

tmux new-session -d -s agent2 -c ~/dev/nathaniel-agent2

Optionally, create one for your main workspace where you'll do the merge:

tmux new-session -d -s main -c ~/dev/Nathaniel

Then:

tmux attach -t agent1
claude

Prompt it to run the task Create Weapon protocol and Gun (Nathaniel-0gc).

tmux attach -t agent2
claude

Prompt it to run the task Port A* pathfinding (Nathaniel-6wt).

tmux attach -t main
claude

I can detach (usually Ctrl-b d) and go about other things, checking in on them occasionally.

Once they're done:

tmux attach -t main

jj log

Now I can manually rebase, e.g.

jj rebase -s bbbb -d aaaa

But really I'd just tell Claude to look after rebasing for me and handle any conflicts.

Then I can get rid of the workspaces:

jj workspace forget nathaniel-agent1
jj workspace forget nathaniel-agent2

And shut down the tmux sessions.

tmux kill-session -t agent1
tmux kill-session -t agent2
tmux kill-session -t main

Or kill all at once:

tmux kill-server

Another approach is Claude Code Cloud. I won't go into detail on how to do that here since it doesn't work well with this example project but it's a feature available in Claude that I love. You can do similar stuff with ChatGPT and GitHub Copilot. The agent basically has a dev machine and you can ask it to work on tasks there.

Being out for a walk, chatting to the agent having it open PRs for you is quite an interesting experience.

Conclusion

It feels to me there's just an infinite amount of work to do and even with busy little agents doing a bunch of it, there will always be more to do. I currently find the setup described here nice for getting a little more done in an enjoyable way.

I don't know what the future holds. It feels like Star Trek, talking to "Computer" to get things done. Progress should be good for everyone, but sometimes it feels like everything is being driven by psychotic ultra capitalist tech bros salivating at having us all live in poverty while AI does every job and just pours money into their pockets, to buy bigger yachts, bunkers or whatever. Maybe that's where we're headed but I don't want to believe that.

I find coding agents really useful. I feel optimistic. I was able to revive my old college game, something I'd probably never have bothered with otherwise and I have so many other things I'm looking forward to trying. Hopefully this post gave you some ideas too. Good luck out there.

P.S. This post might be about AI but I typed it all out myself in my own words. If I didn't take the time to write it, you shouldn't take the time to read it and I really appreciate if you did.

Getting started with Mastodon

Ruairí O'Brien — Sun, 18 Jun 2023 14:33:02 +0000

If you go to a social media site like Twitter, you set up an account and just start using it. Mastodon is like Twitter in a lot of ways but very different in how it works. One key difference is Mastodon is a protocol for building social media sites rather than being the site.

Because Mastodon is a protocol, there are many sites using it. It is possible to interact with any of them using a Mastodon client. They are all compatible with eachother because of the protocol. This setup has become known as the Fediverse. It's even possible for you to set up your own Mastodon server or build your own client.

For many of us looking for a way to get started and explore, this is a little bit confusing.

If you're not looking to build a Mastodon server or a client, it is possible to get started with a few simple steps:

pick a server to join
create an account
pick a client

Pick a server to join

There are many options, and this can get a bit weird because each server can have a different scope, rules and moderation. Some of the core servers are an excellent place to start.

When I set up my account I went with mastodon.online. Another popular option is mastodon.social. These are owned by the Mastodon gGmbH non-profit. There are other options which you can view here.

The most noticeable impact of your choice at the start is this is where your account will live. Your profile URL will have the server name in it. For example, my profile URL is https://mastodon.online/@ruarfffbecause I went with mastodon.online.

You will also be subject to the rules of the server you join. This may become quite important to you, depending on who's running it. As any social media site has rules and can suspend or remove accounts for various reasons, the same is true here. The server you join may have rules you don't like or even cease to exist if the owner decides to remove it. Fortunately, it is possible to migrate your account to another server if you need to.

Pick a client

Because Mastodon is a protocol, anyone can build a client for it. The client can turn the Fediverse into a more conventional social media experience. They can aggregate various servers, allowing you to follow users on different servers and interact with them.

If you are a desktop user, consider Elk. This gives a similar experience to Twitter if that's what you're after.

You set up an account on Elk using your Mastodon profile. If you haven't already set up a Mastodon account on a server, Elk will guide you through that process.

On mobile, there are a few options.

For Android, one option is Tusky.

For iOS, there's Ivory but it's unfortunately not free. Ivory is the one I'm using and it's pretty good.

There's a lot more you can do but this is all you need to get started. Not sure I'll ever do it but it's pretty cool you can go and build your own Mastodon server and client if you want to.

Why, in big companies, is it easy to do bad things and hard to do good things? (with many memes)

Ruairí O'Brien — Thu, 08 Jun 2023 19:51:29 +0000

Often, the bigger the company, the bigger the problems and the harder it is to make changes to solve them.

This is all in the context of working in software development.

Most companies are trying to adopt technology and "transform". As hard as things are at large technology companies, they're so much worse at older companies. In these, engineering is a cost center.

Making positive change is hard but making things worse is somehow far easier.

This is not to be defeatist. You can always make things better. Working in Software is still one of the best jobs around. It's no harm to talk about the issues every so often though.

It's easy to introduce more bureaucracy but hard to reduce toil

You may have encountered Westrum's organisational model.

Many large companies trend towards Bureaucratic. In Mark Schwartz's book The (Delicate) Art of Bureaucracy, there are some fantastic explanations of this. Bureaucracy is not inherently bad but it goes against all good software development principles.

Making things better is often the simple act of removing something or making a thing easier to do. In bureaucratic organisations, adding a new painful process is pretty easy. Removing one is not.

For a software developer trying to do good work, bureaucratic organisations can be the toughest to deal with. The company may be great to work for. You may really like your team but the constant struggle to improve the most basic things can be tough.

In a bureaucratic organisation solutions like ticketing systems often take hold. Improving DevX and automation would be far more beneficial but difficult. Buying some COTS to cover over structural issues is easy. My deep dislike for how that turns out inspired me to create ticketculturecon.

So many people, every day, waste so much time doing busy work that never means anything and that's very sad.

Small iterations are hard, long projects are easy

Agile is almost a bad word in tech these days. A lot of companies claim to be doing it while fighting very hard against some of the most basic principles. For example, doing things in small iterations.

The book Project to Product does a good job of explaining this. Large companies still do everything in projects. Projects have a specific timeline, budget, definition of done. Good software development doesn't work like that.

A product needs constant tending until it's no longer needed. It's rare for large companies to understand this.

There are always exceptions but in general software development is iterative. Even software development for spaceships is iterative. Iterative doesn't mean broken until done. It means we deliver a small slice of value, as fast as we can, so we learn and make better decisions for the next iteration.

At SpaceX they applause when a test launch explodes after takeoff. They do this because they have learned something, not because the rocket exploded exactly as planned. Companies would benefit from optimising for good decision making and learning. Not fortune telling.

There are many bad side effects of not putting in the hard work and providing a good vision and strategy. One is many people end up trying to predict how long things will take and keeping track of what's going on.

Meta work is easy, valuable work is hard

A good friend of mine refers to a large amount of activities that go on in big corps as Meta Work. These are the activities that don't produce any real value. They do however produce a lot of information and require a lot of people to manage them. Those people in turn need to ask a lot of time from those attempting to do valuable work by getting them to attend meetings, report progress, come up with wild guesses for how long things will take and so on.

I am not trying to be cruel. There are so many great people involved in planning, reporting and such. Nobody actually wants to be doing meta work. That's what the incentives have us do. Focusing on building out backlogs is such a waste but so easy at big companies. Having a vision and focusing on the most important thing right now to move in the right direction is hard.

Architecting systems for autonomy is hard.

Building trust is hard.

Technical Strategy is hard, productivity theatre is easy

A clear strategy is powerful. I only fully appreciated this after reading the book Good Strategy Bad Strategy.

It's easy to make plans but hard to build a good strategy. A plan is not a strategy.

It's easy to do a reorg but not easy to get the organisation behind a technical strategy.

Often organising people is leadership's greatest lever. Lack of resolve under political pressure squanders it.

Inventing rules that don't help anything is easy, prioritising what's valuable is hard.

Coming up with a backlog of ideas is easy, true innovation is hard.

Putting in good engineering work and fixing a mess is hard. Massively overcomplicating everything to avoid working on fixing things is easy.

Software engineering is hard. Not just shipping new features and closing tickets but really engineering solid solutions.

Convincing everyone of the need to make improvements is hard. Being a hero and sacrificing to maintain the status quo is easy.

Conclusion

It's so hard to convey but there's a massive corpus on what works and it feels like a minority of people in tech are aware of it. Just some examples:

Accelerate, the most scientific resource we have on what works and what doesn't when it comes to building software.
Modern Software Engineering, the best summary of all the good stuff, backed by all kinds of data.
The DevOps Handbook, there's literally a handbook.

There are so many good books and resources out there. Many people have worked in excellent technical organisations and know what works.

Doing what's right should be easy but it rarely is. Exercise is hard. Good diet is hard. Interesting problems are hard. It just seems to be how this crazy world works.

There are many reasons why big companies get into a place where doing good is harder than it needs to be.

Ultimately, we're all just human and everyone is trying to do their best. Hopefully you'll make some good change today and not just publish a rant full of memes.

Getting a ChatGPT powered, Python API, production ready

Ruairí O'Brien — Fri, 02 Jun 2023 10:21:07 +0000

In the last post we created a simple app powered by ChatGPT.

Now we will look at ways to productionise the application.

Getting an application working and deploying it from your machine is a good start. You will quickly hit limitations working this way. We will set things up so that whatever is on the main branch on GitHub deploys to production.

Another issue we have is we depend on the ChatGPT API, and it is slow! We will introduce another component to our system to cache ChatGPT responses. Our system will look like this.

Getting your idea deployed and usable is only the beginning. We need tools and techniques to observe and operate our application in production. We also need ways to keep our code quality high so we're confident in deploying often.

We usually automate tests early and design our system to be easy to test with fast feedback. It's a big topic, though. Testing prompt-based applications is interesting. There's also a lot to observability. We'll save looking at those for the next post in this series.

Deployment Pipeline

One of the first things I do when starting a new project is set up a deployment pipeline. It's never been easier to set one up, and it's something that only gets harder the longer you wait.

Deploying to production early and often is so important. You may not need convincing, but if you would like more context on why, check out this great post: HOW MUCH IS YOUR FEAR OF CONTINUOUS DEPLOYMENT COSTING YOU? Also consider reading more of Charity Majors' blog. Another excellent resource is Modern Software Engineering by Dave Farley.

We can use GitHub actions for our pipeline. This is free for public repositories. We will be following the recommendations from the fly.io docs.

The steps will be the same for both the UI and the backend. We're only setting up continuous deployment in this part. We'll update the pipeline to run some tests in the next post.

Do these steps in both the UI and API. You must have followed the steps in part 1 for deploying with fly.io.

cd into the repository and run fly tokens create deploy.
Copy the output.
Go to the repository settings in GitHub and select 'Secrets and variable' -> 'Actions'.
Click the button 'New repository secret'
Call the secret FLY_API_TOKEN and paste the token you got from running fly tokens create deploy, into the Secret input.
Back in the repository on your local machine, create a workflow file: mkdir -p .github/workflows/ && touch .github/workflows/fly.yml.

Put the following content in the .github/workflows/fly.yml file.

name: Fly Deploy
on:
  push:
    branches:
      - main
jobs:
  deploy:
    name: Deploy app
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: superfly/flyctl-actions/setup-flyctl@master
      - run: flyctl deploy --remote-only
        env:
          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}

This workflow does the same thing we've been doing from our local machine with the flyctl CLI. It's using the token we configured as opposed to how we did fly auth login locally, but otherwise, it's the same.

Push the changes up git add . && git commit -m 'Add deployment workflow' && git push.

Give it a few minutes and you should hopefully see the pipeline run.

Now every time a push to main occurs, the latest code will deploy.

Caching ChatGPT Responses

If you have followed along, creating the app step by step, you may have noticed the API is very slow.

ChatGPT takes a long time to respond. There are many ways to go about optimising and tweaking performance. In a lot of cases though, a simple cache will work.

You may have heard the "two hard things" saying:

There are only two hard things in Computer Science: cache invalidation and naming things.

-- Phil Karlton

Caching can be challenging, but most of the internet uses it. If you keep it very simple, caching is a powerful tool. I've rarely had a real-world use case of an application which had to do time-consuming computation or work across network boundaries where I didn't use caching and where it didn't help.

Often an in-memory cache is enough to start with. We can't do that running in fly.io as our application gets shut down when not in use. Luckily we can spin up a free Redis instance. We will go through that now.

Fly.io provides Redis using Upstash.

cd into your local API repository.

Create a Redis instance:

flyctl redis create

After creation, you should be presented with a connection URL which you must copy. If you need to look it up again you can run the following:

flyctl redis list

You should see your instance and an ID for it. Copy the ID and run:

fly redis status <the ID you copied>

You will see some output, similar to the following, which includes a 'Private URL'.

Redis
  ID             = aaV829vaMVDGbi5
  Name           = late-waterfall-1133
  Plan           = Free
  Primary Region = mad
  Read Regions   = ams
  Private URL     = redis://password@fly-magical-javelin-30042.upstash.io

In that example you would copy redis://password@fly-magical-javelin-30042.upstash.io

We need to store the connection URL as a secret in our application.

flyctl secrets set REDIS_URL="the URL you copied"

Demonstrating using the previous example:

flyctl secrets set REDIS_URL="redis://password@fly-magical-javelin-30042.upstash.io"

Now we need to update our API to use Redis for caching.

We will use a handy Python package called fastapi-cache2.

poetry add fastapi-cache2

Back in our app/main.py file we need to import some cache modules.

from fastapi_cache import FastAPICache
from fastapi_cache.backends.redis import RedisBackend
from fastapi_cache.decorator import cache

We're using FastAPICache to initialize the cache. We're going to configure a Redis backend (we could but won't use in-memory or some other storage backend instead).

The cache decorator adds an annotation we can add to any function. The decorator caches the output of that function, keyed to the inputs.

When introducing a new component like this, we face choices and tradeoffs again. If our app depends on Redis, should anyone who wants to run it now have Redis running locally? Should we make the application behave differently when running locally and only connect to Redis in production?

The choice is yours, but I'll give my recommendation. When running locally, I try to be as like production as possible. Docker Compose is suitable for this.

Add a docker-compose.yml file.

touch docker-compose.yml

Add the following content to that file:

services:
  app:
    build:
      context: ./
      dockerfile: ./Dockerfile
    depends_on:
      - redis
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY:-UNDEFINED}
      - NYTIMES_API_KEY=${NYTIMES_API_KEY:-UNDEFINED}
    volumes:
      - ./:/usr/app
      - /usr/app/.venv/ # This stops local .venv getting mounted
    ports:
      - "8080:8080"
    command: ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080", "--reload"]

  redis:
    image: redis
    ports:
      - '6379:6379'
    volumes:
      - redis:/data

volumes:
  redis:
    driver: local

This sets up a Redis instance we can now connect to. Your application running in its container will see the redis server on its network with a hostmane of redis.

Add a startup hook to app/main.py:

@app.on_event("startup")
async def startup():
    REDIS_URL = os.getenv("REDIS_URL", "redis://redis")
    redis = aioredis.from_url(REDIS_URL)
    FastAPICache.init(RedisBackend(redis), prefix="fastapi-cache")

We can now cache our API call with the following annotation:

@cache(namespace="test", expire=21600)

namespace can be any string to namespace these cache entries.

expire is how long in seconds we want to cache this entry for. 21600 is 6 hours.

The full main file:

import os

from fastapi import FastAPI, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from fastapi_cache import FastAPICache
from fastapi_cache.backends.redis import RedisBackend
from fastapi_cache.decorator import cache
from redis import asyncio as aioredis

from .nytimes_client import get_top_stories
from .story_formatter import format_stories_to_string
from .summariser import summarise_news_stories

app = FastAPI()

origins = [
    "http://localhost:8080",
    "http://127.0.0.1:5500/",  # Live server default
    "https://summer-ui.fly.dev/",
]

app.add_middleware(
    CORSMiddleware,
    allow_origins=[origins],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)


@app.get("/")
def index():
    return {"msg": "Welcome to the News App"}


@app.get("/news")
@cache(namespace="test", expire=21600)
def news():
    summary = ""
    images = []
    try:
        stories = get_top_stories()
        for story in stories:
            images.extend(story["multimedia"])
        summary = summarise_news_stories(format_stories_to_string(stories))
        print(summary)
        images = list(
            filter(lambda image: image["format"] == "Large Thumbnail", images)
        )
    except Exception as e:
        print(e)
        raise HTTPException(
            status_code=500, detail="Apologies, something bad happened :("
        )
    return {"summary": summary, "images": images}


@app.on_event("startup")
async def startup():
    REDIS_URL = os.getenv("REDIS_URL", "redis://redis")
    redis = aioredis.from_url(REDIS_URL)
    FastAPICache.init(RedisBackend(redis), prefix="fastapi-cache")

Now run:

docker compose up

You should be able to load the application on http://localhost:8080/news.

We exposed the redis server on port 6379 which means you can connect to it on localhost:6379.

I used a VSCode database plugin:

With this cache setup, our API will only hit the ChatGPT API, at most, once every 6 hours. The site will be faster, and we'll spend less money on ChatGPT calls.

You might be thinking, what about the unfortunate user who opens the app after the cache expires? There are many options for improving performance here. We could, for example, run a scheduled job that refreshes the cache for us. If you find this problem interesting and want to learn some techniques, check out the book Designing Data-Intensive Applications.

Conclusion

Now our application is automatically deploying. We've also improved performance and reduced costs using caching.

How do we know what happens in our application after it's deployed? How do we test it? We'll cover all that in the next post coming soon.

Thank you for reading!

Building and deploying a web API powered by ChatGPT

Ruairí O'Brien — Wed, 24 May 2023 22:32:42 +0000

ChatGPT is an interesting tool. It makes certain things that used to be hard to do pretty easy. In some cases, it introduces new concepts you may not have thought to do before.

OpenAI provides some sdks to help build on top of ChatGPT.

In this post, we'll take a simple idea implemented using ChatGPT and deploy it. In a follow-up post, we'll look at how to operate the application in production. We will also look at how to overcome some of the challenges of using ChatGPT in a live application.

Please note: In these posts we end up signing up to various services. All but OpenAI have free plans. OpenAI has a free trial but does cost money when the free trial expires.

Our Sample App

In the DeeplearningAI Prompt Engineering Course, there is a section on summarising content.

This is an interesting function and one that would be quite tricky to do without using an LLM or some form of NLP.

In our sample application, we will use the NYTimes API to pull in the top stories for today. We will get ChatGPT to summarise the stories and we will show that summary.

The end result is available to view at: https://summer-ui.fly.dev/

The sample code for this is available in 2 repos:

Feel free to ignore the UI code if you wish as it's only there to display the data pulled from our API.

Tools

Our API is built using FastAPI which is a Python web framework.

We can use the NYTimes API to get some news and the OpenAI API to run ChatGPT.

We can deploy to fly.io.

For the UI we'll use vanilla JavaScript, CSS and HTML.

GitHub will make an appearance, with a section on building a deployment pipeline using GitHub actions. That will be in part 2.

We'll also end up using Python, Docker and Git.

Setup

Unfortunately, this full setup requires and lot of signing up for stuff and getting API Keys.

I hope you'll bear with me.

If you want a secure way to store these API keys locally, check out my post on that.

Get an NYTimes API Key

For this, follow the NYTimes setup guide.

In short:

Go here, click Create Account (unless you already have one).
Once logged in go to 'My Apps' and add a new app.
Once you select the new app, you should see an API key.

Copy that API key and make it available in an environment variable called NYTIMES_API_KEY.

Not sure what I mean by set an environment variable? Check out this post.

Get a ChatGPT API Key

Go to the OpenAI API page.
Go to API Keys.

Once you have your key, add it to an environment variable called OPENAI_API_KEY.

Note: OpenAI's API is a paid service. You will get a free trial when you sign up. If, like me, you signed up ages ago, forgot about it and have lost your free trial, you will need to add a payment method. Everything I did while testing this stuff cost about 5 cents but I added 5 euros to my account. Hopefully, you have the free trial.

Fly.io

If you want to make something available on the internet you need to deploy it somewhere. There are a vast number of options. Fly.io is one that is currently free. You can also setup a free database and even a free redis cache.

Creating the App

This is a step-by-step guide. You could skip this whole section, pull down the sample code and run it. If you would like a deeper explanation of all the bits though, read on.

FastAPI App

We're going to use Python for the backend API. FastAPI is a nice framework. Both OpenAI and NYTimes provide Python packages for interacting with their APIs.

This, again, is a step with many options. The following is one way to go about doing it.

Create a directory for your project. I called my app Summer because it sounds like summarise ¯\_(ツ)_/¯

Call yours whatever you wish but replace summer in the various examples.

I like to use a directory, under my home directory, called dev and put my repos in there.

Note: All these commands assume you're using a reasonable shell (not Cmd.exe).

mkdir ~/dev/summer-api
cd ~/dev/summer-api

I'm using the -api suffix here because I want to keep my backend and frontent in two different repositories.

Initialize git so we can keep our changes in source control.

git init

You must have Python installed on your system. I'm currently using Python 3.11 but most versions from 3.8 on should be fine. Consider using pyenv to manage your Python installations if you don't already.

Install poetry:

pip install poetry

We'll use poetry as it simplifies dependency management and makes it easier to run virtual environments.

Setup our python project with poetry:

poetry init

You will be asked for various inputs. Most of them are obvious. The main thing is to match the dependnecies.

Package name [summer-api]:
Version [0.1.0]:
Description []:  Summarising the news!
Author [Your name here, n to skip]:  Your name
License []:  MIT
Compatible Python versions [^3.11]:

Would you like to define your main dependencies interactively? (yes/no) [yes]

You can add dependencies during the init or just skip that and paste the dependencies listed below into the pyproject.toml file resulting in this:

[tool.poetry]
name = "summer-api"
version = "0.1.0"
description = "Summarising the news!"
authors = ["Your name"]
license = "MIT"
readme = "README.md"
packages = [{include = "summer_api"}]

[tool.poetry.dependencies]
python = "^3.11"
fastapi = "^0.95.2"
uvicorn = {extras = ["standard"], version = "^0.22.0"}
pynytimes = "^0.10.0"
openai = "^0.27.7"
fastapi-cache2 = "^0.2.1"
redis = "^4.5.5"

[tool.poetry.group.dev.dependencies]
httpx = "^0.24.1"
pytest = "^7.3.1"
black = "^23.3.0"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Feel free to update the versions although they will not be guaranteed to work with the code in this post.

Make sure to run poetry install if you manually update the pyproject.yml file.

Create a directory for our source code.

mkdir app

Create a main file.

touch app/__init__.py
touch app/main.py

Open app/main.py in an editor and enter the following:

from fastapi import FastAPI

app = FastAPI()


@app.get("/")
def index():
    return {"msg": "Welcome to the News App"}

Run the app:

poetry run uvicorn app.main:app --reload

You should see output like this:

INFO:     Will watch for changes in these directories: ['~/dev/summer-api']
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [29115] using WatchFiles
INFO:     Started server process [29144]
INFO:     Waiting for application startup.
INFO:     Application startup complete.

You can go to http://localhost:8000/ in your browser or hit it with curl e.g.

curl http://localhost:8000
{"msg":"Welcome to the News App"}

Get top stories from NYTimes

Create a client:

touch app/nytimes_client.py

In that file, setup a call to get today's top stories:

import os

from pynytimes import NYTAPI
# This is the API Key we setup and added to the env earlier
API_KEY = os.getenv("NYTIMES_API_KEY", "")
nyt = NYTAPI(API_KEY, parse_dates=True)


def get_top_stories():
    return nyt.top_stories()

To test it out, let's create and endpoint that simply returns the top stories.

In our app/main.py file import our new module:

from .nytimes_client import get_top_stories

Add a new endpoint:

@app.get("/news")
def news():
    return get_top_stories()

Run the app and test it (if the app was already running, the --reload flag will have made it restart so you don't have to run it again).

curl http://localhost:8000/news

If you have jq installed you can use it to make the output look nicer.

curl http://localhost:8000/news | jq

You should get a pretty big output here with a list of stories and lots of content.

Summarise the news

Now we'll introduce ChatGPT.

Create a new file:

touch app/summariser.py

The code in this file is explained using inline comments.

import os

import openai

# You will need to get your API key from https://platform.openai.com/account/api-keys
openai.api_key = os.getenv("OPENAI_API_KEY")


# Got this function from this amazing course https://www.deeplearning.ai/short-courses/chatgpt-prompt-engineering-for-developers/
def get_completion(prompt, model="gpt-3.5-turbo"):
    messages = [{"role": "user", "content": prompt}]
    response = openai.ChatCompletion.create(
        model=model,
        messages=messages,
        temperature=0,  # this is the degree of randomness of the model's output
    )
    return response.choices[0].message["content"]


def summarise_news_stories(stories):
    """
    Takes in a list of news stories and prompts ChatGPT to
    generate a short summary of each story based on the provided
    Section, Subsection, Title, and Abstract.
    The function returns the summary generated by ChatGPT.

    Args:
    - stories (str): A list of news stories to be summarised.
    Each story should be a block of text with the following format
    where each part is on a newline:

    Section: the section
    Subsection: the subsection
    Title: the title
    Abstract: the Abstract

    The values for 'Subsection' and 'Abstract' can be empty
    strings if not applicable.

    Returns:
    - summary (str): A string containing the summary generated by ChatGPT
    for all the news stories.
    """

    print("Beginning summary")
    prompt = f"""
    Your task is to generate a short summary of a series of
    news stories given the Section, Subsection, Title and
    Abstract, of each story.

    The sections are after the 'Section:'.
    The subsections are after the 'Subsection:'. The subsections can be empty.

    The title is after the 'Title:'. The abstract is after the 'Abstract:'.
    The abstract can be empty.

    Summarise the stories below, delimited by triple backticks,
    in the style of an AI trying to convey information to humans.
    Please use paragraphs where appropriate and use at most 800 words.

    Stories: ```{stories}```
    """

    return get_completion(prompt)

Formatting the stories

While we could probably have gotten ChatGPT to figure out the JSON from the NYTimes API, it's no harm to reduce the amount of stuff we send to it so let's format the stories to send to ChatGPT.

Create another file:

touch app/story_formatter.py

Add a function to that file to format the stories so we only send the content we need:

def format_stories_to_string(stories):
    stories_string = ""
    for story in stories:
        title = story["title"]
        abstract = story["abstract"]
        section = story["section"]
        subsection = story["subsection"]
        stories_string += f"""
        Section: {section}
        Subsection: {subsection}
        Title: {title}
        Abstract: {abstract}
        """
    return stories_string

The news endpoint

Now we can put everything together. Update our app/main.py file to look like the following:

import os

from fastapi import FastAPI, HTTPException

from .nytimes_client import get_top_stories
from .story_formatter import format_stories_to_string
from .summariser import summarise_news_stories

app = FastAPI()


@app.get("/")
def index():
    return {"msg": "Welcome to the News App"}


@app.get("/news")
def news():
    summary = ""
    images = []
    try:
        stories = get_top_stories()
        for story in stories:
            images.extend(story["multimedia"])
        summary = summarise_news_stories(format_stories_to_string(stories))
        images = list(
            filter(lambda image: image["format"] == "Large Thumbnail", images)
        )
    except Exception as e:
        print(e)
        raise HTTPException(
            status_code=500, detail="Apologies, something bad happened :("
        )
    return {"summary": summary, "images": images}

You can now test it out but note, the call to ChatGPT takes a very long time. We will look at how to deal with that issue in part 2 of this series.

curl http://localhost:8000/news

This should output a summary of the news as well as a list of all the related images.

There's another step we need before we setup a UI for this. Our API expects any calls in the browser to come from the same URL as our application. This is a security mechanism called Cross-Origin Resource Sharing (CORS). We need to update our API to allow requests from our UI.

To make that work, in our main.py add and configure the CORS middleware:

from fastapi.middleware.cors import CORSMiddleware

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

Note: for allow_origins=["*"], you may want to be very specific once you know the URL your UI is at e.g. allow_origins=["http://localhost:8080"],. Using the wildcard here to reduce the chances of things going wrong while following these steps.

Setup our UI

This is totally optional so we will quickly go through the steps.

mkdir ~/dev/summer-ui
cd  ~/dev/summer-ui
git init
mkdir app
touch app/index.html
touch app/script.js
touch app/style.css

In the app/index.html file:

<!DOCTYPE html>
<html>
<head>
    <title>Today's News</title>
    <link rel="stylesheet" type="text/css" href="styles.css">
    <meta http-equiv="Content-Security-Policy" content="upgrade-insecure-requests" />
</head>
<body>
    <main>
        <div id="loader">
            <h1>This should be quick, but ever so often it takes a long time...</h1>
            <div class="lds-circle"><div></div></div>
        </div>
        <div id="news-container" class="container hidden">
            <h1>Today's News</h1>
            <div class="date" id="date"></div>
            <p id="summary">
            </p>
        </div>
        <div id="images" class="img-container">
        </div>
    </main>
    <script src="script.js"></script>
</body>
</html>

I am not good at CSS so I used ChatGPT to create some of this. I prompted it with:

Can you make some CSS that shows a bunch of images, defined in html img elements, and displays them like a bunch of physical photographs that have been spread out on a table?

It output some of the following which I have modified in places.

In the app/style.css file:

body {
  margin: 0;
  padding: 0;
  font-family: Arial, sans-serif;
  color: #333;
  line-height: 1.6;
  background: rgb(238, 174, 202);
  background: radial-gradient(
    circle,
    rgba(238, 174, 202, 1) 0%,
    rgba(148, 187, 233, 1) 100%
  );
}

.container {
  width: 80%;
  max-width: 800px;
  margin: 0 auto;
  padding: 20px;
  background-color: #fff;
  box-shadow: 0px 0px 20px rgba(0, 0, 0, 0.1);
  padding-bottom: 4rem;
}

h1 {
  font-size: 2em;
  margin-bottom: 0.5em;
  color: #212121;
  text-align: center;
}

p {
  font-size: 1em;
  text-align: justify;
}

/* Media queries for responsive design */
@media (max-width: 768px) {
  /* Tablets */
  .container {
    width: 90%;
  }

  h1 {
    font-size: 1.75em;
  }

  p {
    font-size: 0.9em;
  }
}

@media (max-width: 480px) {
  /* Phones */
  .container {
    width: 95%;
  }

  h1 {
    font-size: 1.5em;
  }

  p {
    font-size: 0.85em;
  }
}

.img-container {
  display: flex;
  flex-wrap: wrap;
  justify-content: center;
}

.photo {
  position: relative;
  margin: -15px; /* negative margin makes the images overlap */
  transform: rotate(-10deg); /* starting angle */
  box-shadow: 0 0 10px rgba(0, 0, 0, 0.5);
  transition: transform 0.5s;
  z-index: 1; /* ensure that images can stack on top of each other */
}

.photo:hover {
  transform: rotate(0deg); /* reset to straight when hovered */
}

.photo img {
  max-width: 150px;
  border-radius: 10px;
}

.date {
  text-transform: uppercase;
  text-align: center;
}

#loader {
  padding-top: 20%;
  text-align: center;
}

.lds-circle {
  display: inline-block;
  transform: translateZ(1px);
}
.lds-circle > div {
  display: inline-block;
  width: 64px;
  height: 64px;
  margin: 8px;
  border-radius: 50%;
  background: #fff;
  animation: lds-circle 2.4s cubic-bezier(0, 0.2, 0.8, 1) infinite;
}
@keyframes lds-circle {
  0%,
  100% {
    animation-timing-function: cubic-bezier(0.5, 0, 1, 0.5);
  }
  0% {
    transform: rotateY(0deg);
  }
  50% {
    transform: rotateY(1800deg);
    animation-timing-function: cubic-bezier(0, 0.5, 0.5, 1);
  }
  100% {
    transform: rotateY(3600deg);
  }
}

.hidden {
  display: none;
}

In the app/script.js file:

window.onload = function () {
  fetch("https://summer-api.fly.dev/news/")
    .then((response) => response.json())
    .then((data) => {
      document.getElementById("loader").style.display = "none";
      document.getElementById("news-container").style.display = "block";

      const summaryElement = document.getElementById("summary");
      const imagesElement = document.getElementById("images");

      summaryElement.innerHTML = data.summary.split("\n").join("<br>");

      data.images.forEach((image, index) => {
        const img = document.createElement("img");
        img.className = "photo";
        img.src = image.url;
        img.alt = image.caption;
        img.style.width = "150px";
        img.style.height = "150px";

        if (index % 3 === 0) {
          img.style.gridColumnEnd = "span 2";
          img.style.gridRowEnd = "span 2";
        }

        imagesElement.appendChild(img);
      });
    })
    .catch((err) => console.error(err.message));

  function formatDate(date) {
    const options = {
      weekday: "long",
      month: "long",
      day: "numeric",
      year: "numeric",
    };
    return date.toLocaleDateString("en-US", options);
  }

  const today = new Date();
  document.getElementById("date").textContent = formatDate(today);
};

Note: make sure to replace the URL in that script with whatever URL your API is at

Now you can serve that directory e.g. using the Live Server in VSCode or using some other utility e.g. serve.

All going well you'll see a page like this:

Deploying to fly.io

Let's start with the backend.

First, install flyctl https://fly.io/docs/hands-on/install-flyctl/

fly auth login

Go to our API directory:

cd ~/dev/summer-api

The easiest way to deploy a service is by using a Dockerfile to define the environment. If you can build and run it locally it'll probably work when deployed.

Create a Dockerfile.

touch Dockerfile

With this content:

FROM python:3.11 as builder
WORKDIR /tmp

RUN pip install poetry

COPY ./pyproject.toml ./poetry.lock /tmp/

RUN poetry export -f requirements.txt --output requirements.txt --without-hashes

FROM python:3.11

WORKDIR /usr/app

COPY --from=builder /tmp/requirements.txt ./requirements.txt

RUN pip install --no-cache-dir --upgrade -r ./requirements.txt

COPY . .

EXPOSE 8080

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080"]

Test it out:

docker build . -t summer-api
docker run -p 8080:8080 summer-api

You should be able to hit your application at http://localhost:8080.

Fly knows how to deploy a Dockerfile.

fly launch

That will guide you through setting up your app.

We also need to give it our API keys.

flyctl secrets set NYTIMES_API_KEY=${NYTIMES_API_KEY}
flyctl secrets set OPENAI_API_KEY=${OPENAI_API_KEY}

All going well your API should be up an running. Mine for example is here: https://summer-api.fly.dev/

You will have a unique URL for your app.

For the UI we can do something similar.

cd ~/dev/summer-ui

First, update the script in the UI so it points to the new URL for your API.

So this bit:

window.onload = function () {
  fetch("https://summer-api.fly.dev/news/")

There, you should replace "https://summer-api.fly.dev/news/" with your URL.

Then create a Dockerfile:

touch Dockerfile

Add the following content to the Dockerfile:

FROM pierrezemb/gostatic
COPY ./app/ /srv/http/

EXPOSE 8043

8043 happens to be the port that image uses by default.

Run fly:

fly launch

Fly launch creates a file in your repo called fly.toml.

I hit an issue where I needed to update that file to the following in my UI because it defaulted to the wrong port:

app = "summer-ui"
primary_region = "cdg"

[http_service]
  internal_port = 8043
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0

You may or may not need this. Make sure to update the app and primary_region to the correct values for your app.

Improvements

This post is already huge so I will put all the improvements into their own post.

In the next post we'll look at:

setting up a continuous delivery pipeline
use redis to cache ChatGPT responses and speed up our site

Thank you for reading!

Understanding python async with FastAPI

Ruairí O'Brien — Tue, 06 Sep 2022 23:11:50 +0000

Writing asynchronous code in python is quite powerful and can perform pretty well if you use something like uvloop:

uvloop makes asyncio fast. In fact, it is at least 2x faster than nodejs, gevent, as well as any other Python asynchronous framework. The performance of uvloop-based asyncio is close to that of Go programs.

Writing asynchronous code in python is also pretty easy to mess up.

This post is an attempt to understand async python a little better and make it easier to spot the common mistakes.

There's a repo if you'd like to have the code examples: https://github.com/ruarfff/understanding-python-async

In this post we will look at:

async/concurrent vs parallel
interesting things to know about async in python
async with FastAPI
what do do if something you use doesn't support async

Coding along

Grab the sample code:

git clone git@github.com:ruarfff/understanding-python-async.git && cd understanding-python-async
You should also have curl installed

You can use docker (recommended) or setup your own python environment.

With docker

make build

Without docker

If you're not using docker, make sure to have python >= 3.8 installed.

You should be using a unix like shell e.g. Linux, Mac, WSL.

pyenv

pyenv install 3.10.6

pyenv global 3.10.6

Setup a virtual environment and run all commands in this context

python -m venv ./.venv

source .venv/bin/activate

Install requirements pip install -r requirements.txt

In the examples below, run the python scripts directly instead of using make.

Async Vs Parallel

Parallel and asynchronous are not the same thing. If you have a good mental model for async vs parallel feel free to skip this section.

When I was in college I wrote a mobile game. I had to learn about the game loop.

Within the loop, I had code to:

process inputs
update the game state based on reactions to inputs and existing algorithms
render the game

It was all perfectly synchronous so I couldn't have anything take too long or the game would become unplayable. I became obsessed with improving performance but I always hit limits. I was attempting to render at 30FPS (frames per second meaning the loop must carry out all processing at least 30 times per second). Every time I coded up a new game agent or increased the number of agents on the screen, the FPS would suffer and I'd go mad trying to figure out how to optimise performance again. Then one day, I discovered the joy and pain of async programming.

Mobile games ran on a single processor but using threading, you could simulate parallelism. It was, in fact, asynchronous but you didn't really have to think too hard about scheduling the threads. You hand that job over to the operating system. It did make things a lot more complicated and harder to understand though and that's the major tradeoff.

This might be fairly inaccurate but I like to think of asyncio (and its Cython-based friend uvloop) as a sort of game loop. It's not running in parallel but the loop will keep looping if you don't block it. If you do block it, you mess the whole thing up.

Where that mental model breaks down is when we think about CPU bound VS IO-bound.

Asynchronous code makes sense when there's a lot of waiting or the code is IO-Bound. It's somewhat more intuitive. Like when you are coding. Consider an example flow:

Task - get some code to production

you write some code
run and test the code
commit and push the code
watch the build pipeline
get distracted and start chatting on slack/teams
go for lunch
remember you had a build pipeline running
test the deployment
done

There are steps you don't wait for e.g. watching the build pipeline. This is asynchronous and frees you up to do other things. It's not necessarily more efficient but it's better than sitting there waiting for the build pipeline to finish. You got other tasks done while also getting the code to production.

A build pipeline has steps that can be run in parallel e.g. testing and linting. This can be done in parallel if we give each step all its own resources and a dedicated machine. We could do all the steps asynchronously on one large machine but we can use many smaller, cheaper machines to do the work in parallel.

You can think of async as one resource doing many things at once and figuring out how not to wait for each thing to be finished. Parallel processing is multiple resources each working on separate tasks or parts of a task.

Throughout the rest of this post, we will be focusing on the asynchronous model. When writing python apps with a framework like FastAPI, we are generally doing IO-Bound work e.g. downstream requests, reading databases, files etc. The asynchronous model is most useful in this case.

We will focus primarily on the patterns for using coroutines and async/await.

Example of async code using asyncio

import asyncio
import datetime
import time

from colorama import init
from colorama import Fore, Back, Style


async def main():

    print(Back.GREEN + Fore.WHITE + "Starting synchronous work.", flush=True)

    start_time = datetime.datetime.now()

    synchronous_work()
    synchronous_work()

    end_time = datetime.datetime.now() - start_time

    print(
        Back.GREEN
        + Fore.WHITE
        + f"Ending synchronous work. Total time: {end_time.total_seconds(): .2} sec.",
        flush=True,
    )

    print(Back.MAGENTA + Fore.WHITE + "Starting awaiting async functions.", flush=True)

    start_time = datetime.datetime.now()

    await async_work(Back.LIGHTYELLOW_EX + Fore.WHITE)
    await async_work(Back.LIGHTRED_EX + Fore.WHITE)

    end_time = datetime.datetime.now() - start_time

    print(
        Back.MAGENTA
        + Fore.WHITE
        + f"Ending awaiting async functions. Total time: {end_time.total_seconds(): .2} sec.",
        flush=True,
    )

    print(Back.LIGHTBLUE_EX + Fore.WHITE + "Starting running async.", flush=True)

    start_time = datetime.datetime.now()

    tasks = [
        async_work(Back.LIGHTYELLOW_EX + Fore.WHITE),
        async_work(Back.LIGHTRED_EX + Fore.WHITE),
    ]

    await asyncio.gather(*tasks)

    end_time = datetime.datetime.now() - start_time

    print(
        Back.LIGHTBLUE_EX
        + Fore.WHITE
        + f"Ending running async. Total time: {end_time.total_seconds(): .2} sec.",
        flush=True,
    )

    print(Style.RESET_ALL)


def synchronous_work():
    print(Back.CYAN + Fore.WHITE + "Pretending to wait.")
    time.sleep(2)


async def async_work(colour: str = Fore.WHITE):
    print(colour + "Pretending to wait async.")
    await asyncio.sleep(2)


if __name__ == "__main__":
    init()
    asyncio.run(main())

Run it:

make async-example

This contrived example shows a simple case where asynchronous code got things done faster than blocking code.

Running 2 tasks that each take 2 seconds asynchronously allows them to be finished in 2 seconds.

If you await 2 async functions in sequence, it's sort of like it was done synchronously so if you really want multiple steps to run asynchronously you can use utilities such as the asyncio.gather function in that example.

It's Async all the way down

You must use async all the way down. If you block at any point within an async function, you block everything.

We will explore this more in the FastAPI section but for now, it's important to be aware that this:

async def synchronous_work_in_async_function():
    time.sleep(2)

Has very different effects to this:

async def async_work():
    await asyncio.sleep(2)

The GIL

The python GIL (global interpreter lock) can lead to some issues when writing async code. For the most part, you'll probably never notice if you're writing IO-bound apps but for CPU bound, it may become an issue.

The GIL prevents race conditions and ensures thread safety.

All threads in python share the same memory space. The GIL helps ensure thread safety for multiple threads interacting with the same variable.

An example of where this might be important. Python uses a ref count to determine if something should be garbage collected.

import sys


def main():
    a_thing = "a thing"
    print(sys.getrefcount(a_thing))
    another_thing = a_thing
    print(sys.getrefcount(another_thing))
    del a_thing
    print(sys.getrefcount(another_thing))


if __name__ == "__main__":
    main()

If multiple threads are updating references, you might get an accidental garbage collection or a memory leak. The lock is there to ensure this doesn't happen and garbage collection is done safely.

The downside is this can affect performance.

In hindsight, the GIL is not ideal, since it prevents multithreaded CPython programs from taking full advantage of multiprocessor systems in certain situations.

There are ways around this. Some libraries aren't effected by the GIL e.g. Numpy..

When writing a web app with FastAPI, using async/await and a library like uvicorn, the GIL is less of an issue as we are primarily IO bound.

Async FastAPI

FastAPI is quite fast, unless you make it not fast :)

How does that happen?

The async docs for FastAPI are really good. We won't repeat much from them here but instead look at some examples.

A "hello world" FastAPI app looks like this:

from fastapi import FastAPI

app = FastAPI()


@app.get("/")
async def root():
    return {"message": "Hello World"}

When we start up our FastAPI service, it will handle requests asynchronously on the eventloop. One worker should be able to handle many requests in this scenario.

Let's see what that looks like. If you have the examples repo you can run ./test-endpoints.sh to see all these outputs or feel free to follow along step by step here.

Create a hello world sample like the following:

touch src/fastapi_example.py

from fastapi import FastAPI
import time
import datetime
import asyncio

from colorama import init
from colorama import Fore, Back, Style

app = FastAPI()


@app.get("/")
async def root():
    start_time = datetime.datetime.now()
    print(
        Back.GREEN + Fore.WHITE + f"Hello world started at: {start_time} sec.",
        flush=True,
    )
    return {"message": "Hello World"}

We can hit the same endpoint multiple times simultaneously with curl.

Create a text file like this:

touch test_urls.txt

Add this content:

url = "http://127.0.0.1:8000/"
url = "http://127.0.0.1:8000/"
url = "http://127.0.0.1:8000/"
url = "http://127.0.0.1:8000/"

Start your FastAPI app:

uvicorn src.fastapi_example:app --reload

In another console, run the test script:

./test-endpoints.sh

You should see output similar to the following:

{"message":"Hello World"}{"message":"Hello World"}{"message":"Hello World"}{"message":"Hello World"}{"message":"So slow"}{"message":"So slow"}

In the application logs:

That was perhaps too fast to prove anything.

Update your endpoint with await asyncio.sleep(4):

@app.get("/")
async def root():
    start_time = datetime.datetime.now()
    print(
        Back.GREEN + Fore.WHITE + f"Hello world started at: {start_time} sec.",
        flush=True,
    )
    await asyncio.sleep(4)
    return {"message": "Hello World"}

Run the test again:

Despite each call not responding for 4 seconds, they all kicked off at the same time. FastAPI is handling requests asynchronously.

Now to demonstrate where things can go wrong, let's add another endpoint:

@app.get("/slow")
async def slow():
    start_time = datetime.datetime.now()
    print(
        Back.GREEN + Fore.WHITE + f"Slow started at: {start_time} sec.",
        flush=True,
    )
    time.sleep(4)
    return {"message": "So slow"}

Here's we're using the blocking time.sleep instead of the non blocking asyncio.sleep.

Use the curl_configs/test_urls_slow.txt file:

url = "http://127.0.0.1:8000/slow"
url = "http://127.0.0.1:8000/slow"
url = "http://127.0.0.1:8000/"
url = "http://127.0.0.1:8000/"
url = "http://127.0.0.1:8000/"
url = "http://127.0.0.1:8000/"

The order is important. We want the slow ones to kick off first.

Update the curl command to allow 6 concurrent requests:

curl --parallel --parallel-immediate --parallel-max 6 --config curl_configs/test_urls_slow.txt

Oops! We just blocked all requests from being processed.

The main takeaway from this is, you just can't mix asynchronous and synchronous functionality in a FastAPI application this way.

Consider this next example:

@app.get("/sync-slow")
def sync_slow():
    start_time = datetime.datetime.now()
    print(
        Back.LIGHTRED_EX + Fore.WHITE + f"Sync slow started at: {start_time} sec.",
        flush=True,
    )
    time.sleep(4)
    return {"message": "So slow"}

Notice, we are not specifying async on the function definition.

Use the curl_configs/test_urls_sync_slow.txt file:

url = "http://127.0.0.1:8000/sync-slow"
url = "http://127.0.0.1:8000/sync-slow"
url = "http://127.0.0.1:8000/"
url = "http://127.0.0.1:8000/"
url = "http://127.0.0.1:8000/"
url = "http://127.0.0.1:8000/"

`curl --parallel --parallel-immediate --parallel-max 6 --config curl_configs/test_urls_sync_slow.txt`

If you don't specify your endpoint as being async i.e. as a coroutine, FastAPI will look after it for you instead as it will assume this route contains blocking calls whereas if you specify async, it's up to you to make sure it really is asynchronous.

Rules of thumb:

if all your code uses coroutines i.e. all use async/await, use async when defining your route
if any of your code is blocking e.g. using requests, some non aio ORM etc. do not use async when specifying the route

If you use requests or standard SQLAlchemy anywhere in a route, you will block the execution.

Consider alternatives like:

Pubsub

FastAPI is great at managing async within the context of the framework e.g. in a route. It cannot account for any code you write that may be running on some schedule or handling incoming events. Especially if you're using the same event loop.

To illustrate, let's add some pubsub:

pip install --upgrade google-cloud-pubsub

If you don't want to use docker and would like to follow along please install the gcloud cli and run the following to start a pubsub emulator:

gcloud beta emulators pubsub start --project=example-project

$(gcloud beta emulators pubsub env-init)

If you are using docker compose, the emulator is already setup for you in the sample code.

We are doing this so we can see what happens when our FastAPI application listens to messages out side of standard web requests.

Update our FastAPI app to handle messages:

Import the pubsub lib:

from google.cloud import pubsub_v1

Setup a subscription:

def create_standard_subscription() -> None:
    project_id = "example-project"
    topic_id = "testing"
    subscription_id = "testing-subscription"

    publisher = pubsub_v1.PublisherClient()

    subscriber = pubsub_v1.SubscriberClient()
    subscription_path = subscriber.subscription_path(project_id, subscription_id)
    topic_path = publisher.topic_path(project_id, topic_id)
    try:
        subscriber.create_subscription(
            request={"name": subscription_path, "topic": topic_path}
        )
    except:
        print("Subscription already exists")

    def callback(message: pubsub_v1.subscriber.message.Message) -> None:
        start_time = datetime.datetime.now()
        if message.data.decode("utf-8") == "non-blocking":
            print(
                Back.BLUE + Fore.WHITE + f"Starting async message at: {start_time}.",
                flush=True,
            )
            message.ack()
        elif message.data.decode("utf-8") == "blocking":
            print(
                Back.LIGHTMAGENTA_EX
                + Fore.WHITE
                + f"Starting sync message at: {start_time}.",
                flush=True,
            )
            time.sleep(5)
            message.ack()

    subscriber.subscribe(subscription_path, callback=callback)
    print(f"Listening for messages on {subscription_path}..\n")


@app.on_event("startup")
async def startup_event():
    create_standard_subscription()

Configure a pubsub client to send messages e.g.

touch src/pubsub_client.py

With the following content:

from google.cloud import pubsub_v1 as pubsub
import itertools


def main():
    publisher = pubsub.PublisherClient()

    project_id = "example-project"
    topic_id = "testing"

    publisher = pubsub.PublisherClient()
    topic_path = publisher.topic_path(project_id, topic_id)

    try:
        topic = publisher.create_topic(request={"name": topic_path})
        print(f"Created topic: {topic.name}")
    except:
        print("Topic already exists")

    print("Run 5 non-blocking")
    for _ in itertools.repeat(None, 5):
        future = publisher.publish(topic_path, b"non-blocking")
        print(future.result())

    print("Run 5 blocking")
    for _ in itertools.repeat(None, 5):
        future = publisher.publish(topic_path, b"blocking")
        print(future.result())


if __name__ == "__main__":
    main()

We are going to send some messages to our app and see how it responds in different scenarios.

Start the app:

make up

Run the client in another session:

make pubsub

Or if you are not using docker:

python src/pubsub_client.py

You should see something like:

Created topic: projects/example-project/topics/testing
Run 5 non-blocking
1
2
3
4
5
Run 5 blocking
6
7
8
9
10

You should see output int he app console like this:

All the messages were processed asynchronously. The pubsub is using a callback and the eventloop used by FastAPI is not effected.

Where async + pubsub can go wrong

You may be tempted to use something like gcloud-aio-pubsub with your fastapi app. You can use something like nest-asyncio to hook into the fastapi eventloop. This will allow you to use the same eventloop for your pubsub client and let you use standard async await with event handling.

You should avoid this however. It is very easy to add a blocking call into an event handler. This will block the event loop and prevent any other events and web requests from being processed thus holding up your whole application.

It is advisable to keep it simple. Let fastapi deal with the complexity of asyncio and use a background thread for pubsub.

Useful links

How to only run a job on a pull request in CircleCI

Ruairí O'Brien — Tue, 27 Jul 2021 22:00:52 +0000

I wanted a thing to only happen when a pull request is opened. I also wanted to do some cleanup when the pull request is closed. In my last place we used GitHub actions and this was super easy.
Now I am using CircleCI and this wasn't so easy.

In this post we will look at how to only run a job on a pull request in CircleCI. There is one major caveat. We also need a way to trigger the job on a pull request. We will look at how to do this with the CircleCI web api.

Conditionally run a job

There are a few options you can use to only run a job on a pull request in CircleCI. There is the option to
only ever build on a pull request but this is all or nothing
i.e. you can never run a build on a branch without opening a pull request.

Another option is, within a job, you can inspect the environment variables to see if there is a pull request number like so:

if [ "${CIRCLE_PULL_REQUEST##*/}" != "" ];then
    echo "Is a pull request"
fi

This is OK but it would be nice to conditionally run a whole job instead. It is not possible to read environment variables when the pipeline is loaded. It is only possible when a job is run.
To work around this we can use the circleci/continuation orb.
If you are trying this out, make sure to update your project settings in Advanced Settings -> Enable dynamic config using setup workflows.

CircleCI expects all your configuration in one file called .circleci/config.yml. The continuation orb takes over as the entry point giving you access to the environment variables and then runs the pipeline using whatever configuration you tell it to.
It's a little bit weird but it works.

This is an example of using the continuation orb to conditionally run a job only on a pull request.

.circleci/config.yml

setup: true
version: 2.1
orbs:
  continuation: circleci/continuation@0.2.0
workflows:
  setup:
    jobs:
      - continuation/continue:
          configuration_path: ".circleci/main.yml"
          parameters: /home/circleci/params.json
          pre-steps:
            - run:
                command: |
                  if [ -z "${CIRCLE_PULL_REQUEST##*/}" ]
                  then
                    IS_PR=false
                  else
                    IS_PR=true
                  fi
                  echo '{ "is_pr": '$IS_PR' }' >> /home/circleci/params.json

Note, we mentioning PR here but you could do more or less anything to configure your pipeline there. /home/circleci/params.json is written to and specified with parameters: /home/circleci/params.json.

.circleci/main.yml

version: 2.1

parameters:
  is_pr:
    type: boolean
    default: false

jobs:
  do_something:
    docker:
      - image: cimg/base:2021.04
    steps:
        - run:
            name: something
            command: echo 'You get the picture'

workflows:
  version: 2

   whence-pr:
    when: << pipeline.parameters.is_pr >>
    jobs:
      - do_something:
            name: something

We called the file main.yml here but it could be any file. You just need to specify it in the parameter called configuration_path. This post also shows another way to generate the configuration on the fly.

Now we have passed the is_pr parameter to the pipeline. We can conditionally run things using when: << pipeline.parameters.is_pr >>.

There is one major issue with this approach. Our build may have run before a PR (pull request) was ever opened. Opening a PR will not trigger a build in CircleCI.

Triggering CircleCI pipeline when a pull request is opened

First thing you must do is grab a CircleCi API token. A personal API token will do for this example.

You can trigger a pipeline run like so:

SCM=github
ORG=your-org-here
PROJECT=your-project-here
CIRCLE_BRANCH=a-derived-branch

curl -X POST \
    -H "Circle-Token: ${CIRCLE_TOKEN}" \
    -H 'Content-Type: application/json' \
    -H 'Accept: application/json' \
    -d "{\"branch\":\"${CIRCLE_BRANCH}\"}" \
    https://circleci.com/api/v2/project/${SCM}/${ORG}/${PROJECT}/pipeline

Hopefully it's clear what values you need to change there. How you will run this bit depends on what tools you have available to you. I was using GitHub and even though we use CircleCI, there are enough free GitHub Action minutes for me to setup an action like this:

.github/workflows/pr.yml

name: Trigger Build on PR

on:
  pull_request:
    types: [opened, reopened]

jobs:
  trigger-build:
    runs-on: ubuntu-latest

    steps:
      - name: Trigger CircleCI
        env:
          CIRCLE_BRANCH: ${{ github.head_ref }}
          CIRCLE_TOKEN: ${{ secrets.CIRCLE_TOKEN }}
          ORG: your-org-here
          PROJECT: your-project-here
        run: |
          curl -X POST \
          -H "Circle-Token: ${CIRCLE_TOKEN}" \
          -H 'Content-Type: application/json' \
          -H 'Accept: application/json' \
          -d "{\"branch\":\"${CIRCLE_BRANCH}\"}" \
          https://circleci.com/api/v2/project/github/${ORG}/${PROJECT}/pipeline

This feels like an incredible hack but it works.

A side note on doing something when a PR is merged

This has nothing to do with CircleCI but if you happen to have access to GitHub actions this might be useful.

.github/workflows/pr-closed.yml

name: On PR Closed

on:
  pull_request:
    types: [closed]

jobs:
  on-pr-closed:
    runs-on: ubuntu-latest
    steps:
      - name: Print PR number
        env:
          PR_NUMBER: ${{ github.event.number }}
        run: |
          echo "${PR_NUMBER}"

Check the Mood of a Slack Channel using Machine Learning

Ruairí O'Brien — Wed, 10 Feb 2021 12:58:08 +0000

Sentiment

a thought, opinion, or idea based on a feeling about a situation, or a way of thinking about something

Sentiment analysis, in the context of this article, is the use of natural language processing to quantify the general opinion or mood of some text. We will keep it simple and only look at determining if some text is positive or negative. Positive being generally complimentary in nature. Negative being generally critical in nature. To add some level of (perhaps impractical) practicality we will look at analysing text from a Slack channel.

In this article we will:

Download chat history from a public slack channel (which is optional as you can really use any text)
Use some python tools to process and inspect the chat data
Use a pre-trained model to classify sentences from the chat as positive or negative
Train our own custom model with some training data
Test our own model and use it on the slack data
Prepare a docker image with our model so we can deploy and use it as a service

If you just want to look at some code, this GitHub repository has everything. The jupyter notebook in it also includes a lot of the instructions from here. In this article we will look at all the steps in excruciating detail so we really understand how it all works.

Setup

The easiest thing to do it the clone this GitHub repository. It is using pipenv to manage dependencies so if you follow the Readme and you have a recent version of python as well as pipenv installed, you should be good to go.

git clone git@github.com:ruarfff/slack-sentiment.git

cd slack-sentiment

pipenv sync --dev

pipenv run notebook

The example is mostly run in a jupyter notebook. Of course feel free to use your own setup instead and copy code from here as needed. If you don't want to use pipenv you can also install all the dependencies listed in the Pipfile using pip directly.

e.g.

pip install jupyter slack-sdk nltk numpy pandas matplotlib

Get Data from a Slack Channel

I know I don't need to say this but obviously be careful and understand the sensitivity of your slack data. I ran this against a slack channel with friends. I used slack as I thought it was a nice example for taking data from an arbitrary source and processing it. It was also a fun example to use.

I won't go into too much detail here as the slack documentation is good but here's a brief explanation of how to get the chat history for a channel.

You will need to get the ID for the channel and a token to make a request to the slack api.

To get the slack channel ID:

There is probably an easier way but the one I use is to right click the channel name in the channel menu and select "Copy link". The link will look something like https://your-org.slack.com/archives/C12345. The channel ID is the last piece. That's C12345 in the example URL.

To get a token:

Setup a bot user following these instructions https://api.slack.com/bot-users.
The OAuth scope must allow channels:history. Add the bot user to the channel you wish to read messages from. The channel should be public. I have not tested any of this with private channels.

The next section assumes you have 2 environment variables:

SLACK_SENTIMENT_BOT_TOKEN - The bot token we setup previously.

SLACK_SENTIMENT_CHANNEL_ID - The channel ID we retrieved previously.

If you are not familiar with how to setup environment variables, feel free to just replace those values with hardcoded values but obviously be careful not to accidentally push your token up to Github or something like that :)
Incidentally, I do have a post on managing secrets while developing if you're interested.

The following python example gets all the slack messages for a channel and prints them:

# Get a timestamp for 7 days ago in a weird way. I need to lean more python. 
# This is only used if we want to limit how far back in time we go for fetching slack messages.
from datetime import timedelta, datetime, date
today = date.today()
week_ago = today - timedelta(days=7)
week_ago_timestamp = (datetime.combine(week_ago, datetime.min.time()) - datetime(1970, 1, 1)) / timedelta(seconds=1)

import os
# Import WebClient from Python SDK (github.com/slackapi/python-slack-sdk)
from slack_sdk import WebClient
from slack_sdk.errors import SlackApiError
import json
from time import sleep

token = os.environ.get("SLACK_SENTIMENT_BOT_TOKEN")
channel_id = os.environ.get("SLACK_SENTIMENT_CHANNEL_ID")
page_size = 100

client = WebClient(token=token)

conversation_history = []
has_more = True
cursor = None

try:
    while has_more:
        has_more = False
        result = client.conversations_history(
            channel=channel_id, 
            limit=page_size,
            cursor=cursor
            # If you wanted to limit to the last 7 days uncomment the next line and put a , after cursor on the previous line
            # oldest = week_ago_timestamp
        )        
        conversation_history.extend(result["messages"])        
        has_more = result['has_more']
        if result['response_metadata'] is not None:
            print('Still fetching')
            cursor = result['response_metadata']['next_cursor']
            sleep(0.5) # Avoid being rate limited 
    print('Done!')
except SlackApiError as e:
    print("Error getting conversations: {}".format(e))

print(conversation_history)

Prepare the Data

In data engineering a huge amount of time is spent organising and cleaning data. Slack is just one example of a wide variety of extremely versatile data sources. In the following example we will gently dip our toes into that kind of work. Let's take our slack messages and clean them up a bit. Let's also break them up into sentences and words.

I will just drop our main data cleaning function in here and then go through it line by line.

import nltk
from nltk.tokenize import word_tokenize, sent_tokenize
import itertools

def get_test_data():
    all_text = [message['text'] for message in conversation_history if 'text' in message]
    tokenized_text = [sent_tokenize(text) for text in all_text]
    sentences = list(itertools.chain(*tokenized_text))
    tokenized_words = [word_tokenize(sentence) for sentence in sentences]
    words = list(itertools.chain(*tokenized_words))
    return (sentences, words)

main_sentences, main_words = get_test_data()

Line by line

all_text = [message['text'] for message in conversation_hissory if 'text' in message]

The response from the Slack API stores the text from the slack conversation in a field called text. We don't want any of the other information like the user who posted it or the time it was posted. That line iterates over each response and extracts the text to a new list.

tokenized_text = [sent_tokenize(text) for text in all_text]

Iterate over all the text and use NLTK's sentence tokeniser to split the text up into sentences.

sentences = list(itertools.chain(*tokenized_text))

The previous line actually gave us a list of lists since sent_tokenize returns a list. This line flattens that to a single list.

tokenized_words = [word_tokenize(sentence) for sentence in sentences]

This line used NLTK's word tokeniser to split each slack message up into words.

words = list(itertools.chain(*tokenized_words))

word_tokenize returns a list so we ended up with a list of lists. This line flattens that to a single list.

return (sentences, words)

Here we are returning a tuple containing the sentences and words. In this case, a tuple is just a handy way to return multiple things from a function without going to the bother of creating a new data type for it.

main_sentences, main_words = get_test_data()

We then assign the values from the returned tuple to 2 new variables. We will use these variables again throughout the rest of this example.

Use a Pre-trained Model

We will be training our own model and we will look at how to deploy our own trained model soon. First it could be useful to look at an example of using a pre-trained model. We will use NLTK's SentimentIntensityAnalyzer.

You can import and setup the model like so:

from nltk import download
import matplotlib.pyplot as plt
from nltk.sentiment import SentimentIntensityAnalyzer

# The SentimentIntensityAnalyzer model needs us to pull down the vader_lexicon
download('vader_lexicon')

sia = SentimentIntensityAnalyzer()

Then you can take one of the sentences from our slack "corpus" and pass it in:

sia.polarity_scores(main_sentences[0])["compound"]

That will return a number. If the number is greater than 0 then we say the sentence has a positive sentiment.

Here's an example that checks each sentence and draws a pie chart showing the ratio of positive vs negative.

num_pos = 0
num_neg = 0

def is_positive(sentence: str) -> bool:
    return sia.polarity_scores(sentence)["compound"] > 0

for s in main_sentences:
    if is_positive(s):
        num_pos +=1
    else:
        num_neg += 1

labels = 'Positive', 'Negative'
sizes = [num_pos, num_neg]

fig1, ax1 = plt.subplots()
ax1.pie(sizes, labels=labels, autopct='%1.1f%%',
        shadow=True, startangle=90)
ax1.axis('equal')

plt.show()

Example output:

This was just to show an example but it might also demonstrate the importance of the next step. The results we got from the data in its current state may not have been very accurate. Let's see what we can do to clean up the data and reduce the noise in there.

Clean the Data

From this point on we won't be using the sentences from our dataset anymore. We will focus on the individual words instead. Let's create a frequency distribution graph of the words. This is also to demonstrate how many of them are more or less useless.

from nltk.probability import FreqDist

fdist = FreqDist(main_words)

fdist.plot(30,cumulative=False)
plt.show()

If you run that, you probably see a lot of words like "I", "if", "me" etc. Also maybe some punctuations. These are not too useful for our analysis.

Let's look at a helper function to clean that up.

When we start training our own model we will be using a method where we store each word along with its frequency and train a model using those features (the words and frequency) along with labels (positive and negative).

We will need 2 functions. One to clean our data, removing noise such as meaningless words, punctuation etc. Then we need a function to convert a stream of words into a hashmap with words and number of occurences.

# Assumes you previously had from nltk import download
download('stopwords')
download('names')

from nltk.corpus import stopwords, names
from string import punctuation

# A large set of names. We lowercase here as we will lowercase our own data first too
name_words = set([n.lower() for n in names.words()])
# A set of stopwords provided by NLTK (if, as etc.)
stop_words = set(stopwords.words("english"))

def clean_words(words):
    return [w for w in [w.lower() for w in words if w.isalpha()] if w not in stop_words and w not in name_words and w not in punctuation]

Let's split clean_words and look at each bit.

w for w in [w.lower() for w in words if w.isalpha()]

Go through all the words in the words array which is passed as an argument to our function. Check if a word is alphanumeric (no special characters we can't use). Then extract that out to a new list.

if w not in stop_words and w not in name_words and w not in punctuation

This filters our list. We filter out all those stop words. We filter out any names and punctuations. This gives us a cleaner set to work with.

You may be wondering what we are doing here? How can we analyse the data without context? We are going to use a pretty basic model. We will more or less be taking bags of words with labels and training a model with them. Then we will feed other bags of words to the model and have it make a guess at the sentiment. It doesn't know anything about actual sentences. It mostly goes off of word frequency.

To help us shape our data even more for later use we need one more function.

def word_counts(words):
    counts = {}
    for word in words:
        counts[word] = counts.get(word, 0) + 1
    return counts

This takes our words and puts them into a HashMap where the key is the word and the value is the number of times that word has shown up in our data.

Now we can get the data ready.

main_words = clean_words(main_words)
main_words_counts = word_counts(main_words)

You can create a new distribution graph and see if it looks better.

fdist = FreqDist(main_words)
fdist.plot(30,cumulative=False)
plt.show()

Training Our Own Model

Now we will train a model so we can call ourselves AI scientists. We will use something called a Naive Bayes classifier. NLTK provides one for us to use. Our main job here really is to provide a lot of labelled data to train the model and the library does the rest. Over time as we become more familiar with the data we might figure out clever ways of modifying it to improve our model. We will look briefly at that concept but will not go into it in any depth here.

In the sample repo for this there are some data files. These can also be downloaded from the Sentiment Labelled Sentences Data Set. We will also use some data provided by NLTK.

First we load the files from the repo using the pandas library.

import pandas as pd

amazon = pd.read_csv('sentiment_labelled_sentences/amazon_cells_labelled.txt', names=['review', 'sentiment'], sep='\t') 
imdb = pd.read_csv('sentiment_labelled_sentences/imdb_labelled.txt', names=['review', 'sentiment'], sep='\t') 
yelp = pd.read_csv('sentiment_labelled_sentences/yelp_labelled.txt', names=['review', 'sentiment'], sep='\t')

Those files must be located in a folder called sentiment_labelled_sentences in the same directory as the python script if using the code as is.

Now we download the movie review corpus provided by NLTK.

# Assumes you previously had from nltk import download
download('movie_reviews')

The next step is to take all that data and split it into 2 lists. One list of negative reviews. The other of positive reviews.

reviews = [*amazon['review'].values, *imdb['review'].values, *yelp['review'].values]
sentiment = [*amazon['sentiment'].values, *imdb['sentiment'].values, *yelp['sentiment'].values]

positive_reviews = []
negative_reviews = []

from nltk.corpus import movie_reviews

# Process the corpus data which is split into files with IDs of pos and neg
for f in movie_reviews.fileids('pos'):
    positive_reviews.append((word_counts(clean_words(movie_reviews.words(f))), 'pos'))
for f in movie_reviews.fileids('neg'):
    negative_reviews.append((word_counts(clean_words(movie_reviews.words(f))), 'neg'))

# Process the data we extracted from files into reviews and sentiment lists with matching indexes  
for i, r in enumerate(reviews):
    review_words = word_tokenize(r)
    if sentiment[i] == 1:
        positive_reviews.append((word_counts(clean_words(review_words)), 'pos'))
    else:
        negative_reviews.append((word_counts(clean_words(review_words)), 'neg'))

Now that we have all that data prepared we can train a model.

from nltk.classify import NaiveBayesClassifier

# We will use 80% of the data set for training and 20% for testing
split_pct = .80

pos_split = int(len(positive_reviews)*split_pct)
neg_split = int(len(positive_reviews)*split_pct)

train_set = positive_reviews[:pos_split] + negative_reviews[:neg_split]
test_set = positive_reviews[pos_split:] + negative_reviews[neg_split:]

model = NaiveBayesClassifier.train(train_set)

Here we split the data into training and testing sets:

split_pct = .80

pos_split = int(len(positive_reviews)*split_pct)
neg_split = int(len(positive_reviews)*split_pct)

train_set = positive_reviews[:pos_split] + negative_reviews[:neg_split]
test_set = positive_reviews[pos_split:] + negative_reviews[neg_split:]

It's hard to know what the optimal split is but I have
read some discussion indicating 80/20 might be a good start.

Once we have our data read, the actual training of the model is simply calling one function provided by NLTK:

model = NaiveBayesClassifier.train(train_set)

At this point we can start using the model to classify all that data we prepared from slack.

model.classify(main_words_counts)

It is a good idea for us to use our test data and determine the accuracy of the model first though.

from nltk.classify.util import accuracy

print(100 * accuracy(model, test_set))

When I checked the model it was coming in between 65% and 70% accurate. At this point there's some work to do to improve the model. I am not going to go into that now except to provide some basic pointers. Perhaps in a follow up article I will get into it. For now let's look at some things you can do to see where things need improving.

We can gather all the guesses that went wrong from the test data.

errors = []
for (data, label) in test_set:
    guess = model.classify(data)
    if guess != label:
        errors.append((label, guess, data))

print(errors[0])

Looking at the places where guesses went wrong, maybe we can modify the data to improve the training. Maybe take variations on a word and ad that to the training set.

We can see which features our model believes are most informative.

model.show_most_informative_features()

By getting a better understanding of how the model learns, we improve our ability to know how to train it. It's difficult but also interesting.

Deploying a Model

Now we can save the model. You could push this model up somewhere and pull it in to some other useful location. For an example of how to deploy this model in a web service using docker please see the Readme for this project.

You can imagine some process whereby you train a model at some interval or on some event and push the model to a location from which it gets pulled down for use in another application or service.

Write the model to a file:

import pickle

model_file = open('.models/sentiment_classifier.pickle','wb')
pickle.dump(model, model_file)
model_file.close()

For a full example of how to then use that model in a web service please make sure to look at the Example Repository.

A quick example of how if might work follows:

model_file = open('.models/sentiment_classifier.pickle', 'rb')
model = pickle.load(model_file)
model_file.close()

print(100 * accuracy(model, test_set))
model.show_most_informative_features()
model.classify(main_words_counts)

That's it. Hopefully this has been useful. Thank you for reading.

References

I wrote this post as I was studying material from the following places:

Training and Deploying an ML Model as a Microservice

Text Analytics for Beginners using NLTK

Sentiment Analysis: First Steps With Python's NLTK Library

Training an ML Model for Sentiment Analysis in Python

Day 31 - Next Permutation

Ruairí O'Brien — Sun, 31 Jan 2021 16:57:24 +0000

The Problem

Implement next permutation, which rearranges numbers into the lexicographically next greater permutation of numbers.

If such an arrangement is not possible, it must rearrange it as the lowest possible order (i.e., sorted in ascending order).

The replacement must be in place and use only constant extra memory.

Example 1:

Input: nums = [1,2,3]
Output: [1,3,2]

Example 2:

Input: nums = [3,2,1]
Output: [1,2,3]

Example 3:

Input: nums = [1,1,5]
Output: [1,5,1]

Example 4:

Input: nums = [1]
Output: [1]

Constraints:

1 <= nums.length <= 100
0 <= nums[i] <= 100

Tests

import pytest
from .Day31_NextPermutation import Solution

s = Solution()


@pytest.mark.parametrize(
    "nums,expected",
    [
        ([1, 2, 3], [1, 3, 2]),
        ([3, 2, 1], [1, 2, 3]),
        ([1, 1, 5], [1, 5, 1]),
        ([1], [1]),
    ],
)
def test_next_permutation(nums, expected):
    s.nextPermutation(nums)

    assert nums == expected

Solution

from typing import List


class Solution:
    def nextPermutation(self, nums: List[int]) -> None:
        """
        Do not return anything, modify nums in-place instead.
        """
        i = len(nums) - 2
        while i >= 0 and nums[i + 1] <= nums[i]:
            i -= 1

        if i >= 0:
            j = len(nums) - 1
            while j >= 0 and nums[j] <= nums[i]:
                j -= 1
            nums[i], nums[j] = nums[j], nums[i]

        k = len(nums) - 1
        while i < k:
            i += 1
            nums[i], nums[k] = nums[k], nums[i]
            k -= 1

Analysis

Day 30 - Minimize Deviation in Array

Ruairí O'Brien — Sun, 31 Jan 2021 16:52:10 +0000

The Problem

You are given an array nums of n positive integers.

You can perform two types of operations on any element of the array any number of times:

If the element is even, divide it by 2.
For example, if the array is [1,2,3,4], then you can do this operation on the last element, and the array will be [1,2,3,2].
If the element is odd, multiply it by 2.
For example, if the array is [1,2,3,4], then you can do this operation on the first element, and the array will be [2,2,3,4].

The deviation of the array is the maximum difference between any two elements in the array.

Return the minimum deviation the array can have after performing some number of operations.

Example 1:

Input: nums = [1,2,3,4]
Output: 1
Explanation: You can transform the array to [1,2,3,2], then to [2,2,3,2], then the deviation will be 3 - 2 = 1.

Example 2:

Input: nums = [4,1,5,20,3]
Output: 3
Explanation: You can transform the array after two operations to [4,2,5,5,3], then the deviation will be 5 - 2 = 3.

Example 3:

Input: nums = [2,10,8]
Output: 3

Constraints:

n == nums.length
2 <= n <= 105
1 <= nums[i] <= 109

Tests

import pytest
from .Day30_MinimizeDeviationInArray import Solution

s = Solution()


@pytest.mark.parametrize(
    "nums,expected",
    [
        ([1, 2, 3, 4], 1),
        ([4, 1, 5, 20, 3], 3),
        ([2, 10, 8], 3),
    ],
)
def test_minimum_deviation(nums, expected):
    assert s.minimumDeviation(nums) == expected

Solution

from typing import List
import heapq
import math


class Solution:
    def minimumDeviation(self, nums: List[int]) -> int:
        vals = []
        minimum = math.inf
        minimum_deviation = math.inf

        for n in nums:
            if n % 2 == 0:
                vals.append(-n)
                minimum = min(minimum, n)   
            else:
                evened = n * 2
                vals.append(-evened)
                minimum = min(minimum, evened)

        heapq.heapify(vals)

        while vals:
            e = -heapq.heappop(vals)
            minimum_deviation = min(minimum_deviation, e - minimum)

            if e % 2 != 0:
                return minimum_deviation
            e = e // 2
            minimum = min(minimum, e)
            heapq.heappush(vals, -e)

        return minimum_deviation

Analysis

Day 29 - Vertical Order Traversal of a Binary Tree

Ruairí O'Brien — Sun, 31 Jan 2021 16:41:30 +0000

Given the root of a binary tree, calculate the vertical order traversal of the binary tree.

For each node at position (row, col), its left and right children will be at positions (row + 1, col - 1) and (row + 1, col + 1) respectively. The root of the tree is at (0, 0).

The vertical order traversal of a binary tree is a list of top-to-bottom orderings for each column index starting from the leftmost column and ending on the rightmost column. There may be multiple nodes in the same row and same column. In such a case, sort these nodes by their values.

Return the vertical order traversal of the binary tree.

Example 1:

Input: root = [3,9,20,null,null,15,7]
Output: [[9],[3,15],[20],[7]]
Explanation:
Column -1: Only node 9 is in this column.
Column 0: Nodes 3 and 15 are in this column in that order from top to bottom.
Column 1: Only node 20 is in this column.
Column 2: Only node 7 is in this column.

Example 2:

Input: root = [1,2,3,4,5,6,7]
Output: [[4],[2],[1,5,6],[3],[7]]
Explanation:
Column -2: Only node 4 is in this column.
Column -1: Only node 2 is in this column.
Column 0: Nodes 1, 5, and 6 are in this column.
          1 is at the top, so it comes first.
          5 and 6 are at the same position (2, 0), so we order them by their value, 5 before 6.
Column 1: Only node 3 is in this column.
Column 2: Only node 7 is in this column.

Example 3:

Input: root = [1,2,3,4,6,5,7]
Output: [[4],[2],[1,5,6],[3],[7]]
Explanation:
This case is the exact same as example 2, but with nodes 5 and 6 swapped.
Note that the solution remains the same since 5 and 6 are in the same location and should be ordered by their values.

Solution

from collections import deque, OrderedDict
from typing import List
from .util import TreeNode


class TravesalItem:
    def __init__(self, node, row, col):
        self.node = node
        self.row = row
        self.col = col


class Solution:
    def verticalTraversal(self, root: TreeNode) -> List[List[int]]:
        grid_items = []
        traversal = {}

        traversal_queue = deque([TravesalItem(root, 0, 0)])
        while traversal_queue:
            ti = traversal_queue.popleft()
            if ti.node is not None:
                grid_items.append((ti.col, ti.row, ti.node.val))
                traversal_queue.append(
                    TravesalItem(ti.node.left, ti.row + 1, ti.col - 1)
                )
                traversal_queue.append(
                    TravesalItem(ti.node.right, ti.row + 1, ti.col + 1)
                )

        for col, row, val in sorted(grid_items):
            if col not in traversal:
                traversal[col] = []
            traversal[col].append(val)

        return traversal.values()