Docker

I've been curious about integrating AI agents into my workflow recently, and so I started looking at how this could be done using my current equipment. Data sovereignty is important to me so sending all my data to train a remote AI agent doesn't appeal. I was expecting to need to buy a new gaming rig with a couple of high end graphics cards in it, but after some research I found that this wasn't the case.

I found a system called llama.cpp, which is an efficient LLM engine written in C++. The idea behind llama.cpp is that you can host small, efficient AI agents without having to throw thousands at equipment to get them running. As I have a Raspberry PI model 5, with 16GB of RAM in my office I thought this was a good candidate to get running.

Whilst setting up docker on my local development machine the other day I encountered a permission problem. After installing docker I found that I had this permission problem that meant I couldn't run docker using my local user accounts. I was therefore forced to run docker as sudo, which I didn't want to do every time.

This is the error I was getting.

$ docker run hello-world
docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: 
Post http://%2Fvar%2Frun%2Fdocker.sock/v1.40/containers/create: dial unix /var/run/docker.sock: connect: 
permission denied.
See 'docker run --help'.

It turns out that docker have a section on their website dedicated to sorting out this issue. The steps involved in correcting this problem are as follows.