Kbravh's Blog

Interactive Pixel Spritesheet Animation with CSS

Sat, 10 May 2025 21:57:15 GMT

I've dabbled in pixel art here and there (see Fungus February), and recently I had an idea to draw a pixelized lever to use for a light/dark mode toggle. However, I wanted to try to handle the animation and interactivity entirely with CSS.

Spritesheet setup

First, I started out by creating a spritesheet of all six frames of my animation.

By using a spritesheet, we can handle our entire animation using a single asset. Each frame is 32x32 pixels. We can display our image with a regular <img> tag, then wrap it in a <div>, set its height to the size of a single frame, and cut off any overflow.

html

<div class="frame">
  <img
    class="lever"
    src="lever.png"
    role="presentation"
  />
</div>

css

.frame {
  height: 32px;
  width: 32px;
  overflow: hidden;

  .lever {
    width: 32px;
  }
}

This is rather small, so let's scale it up We'll use a custom CSS property --pixel-size as our scaling factor. Also, since we're scaling up such a small image, we'll need to add a rule to tell the browser to keep things pixelated: image-rendering: pixelated;. Without this, the browser would try to smooth out our image while enlarging it, giving us blurry pixel art.

css

: root {
  --scale: 4;
  --frame-size: calc(var(--scale) * 32px);
}

.frame {
  height: var(--frame-size);
  width: var(--frame-size);
  overflow: hidden;
  image-rendering: pixelated;

  .lever {
    width: var(--frame-size);
  }
}

Set image-rendering to pixelated auto

User interaction

Now that we have our image and frame set up, we need a trigger to kick off the animation. Since we need to allow for user interaction and need a boolean state, a checkbox will work perfectly. Let's set up a checkbox adjacent to our lever frame. We can put a wrapper around them both, then use some absolute positioning to place the checkbox above the image.

html

<div class="wrapper">
  <input
    type="checkbox"
    class="checkbox"
  />
  <div class="frame">
    <img
      class="lever"
      src="lever.png"
      role="presentation"
    />
  </div>
</div>

css

:root {
  --scale: 4;
  --frame-size: calc(var(--scale) * 32px);
}

.wrapper {
  position: relative;
  width: var(--frame-size);

  &:has(.checkbox:focus-visible) {
    outline: 3px solid deepskyblue;
    outline-offset: 3px;
  }
}

.checkbox {
  position: absolute;
  inset: 0;
  opacity: 0;
  cursor: pointer;
  z-index: 1;
}

.frame {
  height: var(--frame-size);
  width: var(--frame-size);
  overflow: hidden;
  image-rendering: pixelated;

  .lever {
    width: var(--frame-size);
  }
}

Checkbox opacity

Create the animation

Finally, we have everything we need to set up our animation! Whenever the checkbox has the checked state, we'll shift our spritesheet down so that it sits on the last frame. We'll need a few pieces to make this work well.

Set up a new property for the spritesheet's Y offset.
Change the value of that property when the checkbox is checked.
Add a transition property so that the shift doesn't happen immediately.
Add a translate property to the lever spritesheet so it follows the offset we defined.

1. Offset property

First, let's create a CSS custom property for our spritesheet offset. Instead of just defining it on the :root, though, we're going to use the @property at-rule. Defining it this way lets the browser know that our property is a length, which means it can be animated. We can also set our initial offset to 0px.

@property --lever-offset {
  syntax: '<length>';
  initial-value: 0px;
  inherits: false;
}

2. Update property when checked

With our offset property defined, let's update it based on the checkbox's state. By default, our offset is already 0px, so we don't need to worry about the unchecked state. When our checkbox is checked, then, let's set the offset to go all the way to the end of the spritesheet. We have six frames total, but five that need to be animated to, so we can multiply five by the size of our frame. We'll also make it negative so that the spritesheet moves up the page and not down.

.checkbox {
  ... &:checked ~ .frame .lever {
    /* This comes out to -640px */
    --lever-offset: calc(-5 * var(--frame-size));
  }
}

3. Transition our property

Now, let's take advantage of the fact that our property can be animated/transitioned. On our <img>, we're going to set a transition property. Normally, we'd set this to watch a certain property, like transition: transform ..., transition: opacity ..., or even transition: all .... But we can tell the browser to explicitly use our property:

.lever {
  width: var(--frame-size);
  transition: --lever-offset 500ms linear;
}

This means that whenever the checkbox is checked, instead of flipping our value from 0px immediately to -640px, the browser will actually interpolate linearly from 0 to -640 over 500ms.

4. Move the spritesheet

Now that our offset property is defined and is being updated, let's consume it. We're going to translate our image by the offset amount. The translate property takes the X axis translation first, so we'll pass 0 then our variable.

.lever {
  width: var(--frame-size);
  translate: 0 var(--lever-offset);
  transition: --lever-offset 500ms linear;
}

From here on out, the levers you see are interactive! Give them a click to see them in action.

There's obviously something wrong with this animation; there's another constraint we need to add. As you can see from the demo above, animating this now causes our spritesheet to smoothly glide from one end to another. What we need instead is to jump from one frame to the next. We can do this with the new CSS round function. This function lets us pass in our current value and a "rounding interval", which we can treat like a step. So we'll pass in our image offset as the value and the size of our frame as the rounding interval.

.lever {
  width: var(--frame-size);
  translate: 0 round(var(--lever-offset), var(--frame-size));
  transition: --lever-offset 500ms linear;
}

This is actually the first way I approached this animation!

transition: --lever-offset 500ms steps(5);

Open for details on what went wrong:

The steps function seems to split our offset from 0 to -640 into five equal pieces and jump from frame to frame the same way.

256 -> 384 -> 512 -> 640

There's a slight bug that occurs, though, and it reveals a bit about how the steps function actually works under the hood. If the user clicks the checkbox and quickly clicks it again before the animation is complete, the animation will reverse, _but_ it won't stick to these steps!

I've slowed down the animation below; click to start it, then click again to interrupt and reverse it.

So, it seems like the step function is actually just running a linear interpolation behind the scenes, and each time 1/5 of the total amount passes (128 in this case), it emits an update. If a user were to reverse the animation when the internal linear function were at 300, for example, the step function would start counting from there! It would emit an update at 300, 172, 44, and 0. This means that instead of seeing our nice, clean frames, our image now starts jumping in the middle of frames.

By taking over this calculation ourselves with round, we have more control over what we actually want to happen. Also, an additional benefit is that we don't have to keep our interpolation linear! We could swap to an easing function and make certain frames stay longer than others.

Conclusion

Our animation is now complete! We have an interactive pixel art spritesheet animation coded entirely in CSS.

html

<div class="wrapper">
  <input
    type="checkbox"
    class="checkbox"
  />
  <div class="frame">
    <img
      class="lever"
      src="lever.png"
      role="presentation"
    />
  </div>
</div>

css

:root {
  --scale: 4;
  --frame-size: calc(var(--scale) * 32px);
}

@property --lever-offset {
  syntax: '<length>';
  initial-value: 0px;
  inherits: false;
}

.wrapper {
  position: relative;

  &:has(.checkbox:focus-visible) {
    outline: 3px solid deepskyblue;
    outline-offset: 3px;
  }
}

.checkbox {
  position: absolute;
  inset: 0;
  opacity: 0;
  cursor: pointer;
  z-index: 1;

  &:checked ~ .frame .lever {
    --lever-offset: calc(-5 * var(--frame-size));
  }
}

.frame {
  height: var(--frame-size);
  width: var(--frame-size);
  overflow: hidden;
  image-rendering: pixelated;

  .lever {
    width: var(--frame-size);
    translate: 0 round(var(--lever-offset), var(--frame-size));
    transition: --lever-offset 500ms linear;
  }
}

Fungus February

Fri, 03 May 2024 02:02:53 GMT

Bolstered by the fun I had during SeptemBIT 2021 (yes, I know it's been over 2 years), I decided to try my hand at some pixel art again. This time, though, inspired by T. Kingfisher's What Moves the Dead and What Feasts at Night, I fell down a rabbit hole of interesting fungi. What a rabbit hole it was!

So while I debated on whether to upload these for Fungus February or Mushroom March, I realized I had missed Amanita April and was already standing on the front porch of Morel May. But I digress; please enjoy these pixel art fungi 🍄

Amanita muscaria

Known as fly amanita, this is the classic mushroom most widely recognized from Super Mario Bros.

Amanita caesarea

Caesar's amanita, found primarily along Roman roads in Europe.

Geoglossum difforme

The earthtongue, so named because it literally looks like a black tongue sticking out of the ground.

Mycena rosea

The rosy bonnet, a slightly bioluminescent fungus cap.

Mycena haematopus

The bleeding bonnet. This mushroom legitimately looks like it's bleeding when you prick it. Quite offputting, but also pretty metal.

Craterellus cornucopioides

The horn of plenty, or black trumpet, is an edible fungus often found in North America.

Sarcoscypha coccinea

The scarlet elf cup is very common in Russia, where it is added to food for its bright color.

Details

All of these drawings were done in Pixel Studio on an Android device. Each was done on a 16x16 pixel canvas. I wasn't too particular about the color pallet, but I did try to keep it to about 2-6 colors on each drawing.

Create an interface between an LLM and a set of tools

Tue, 16 May 2023 13:59:16 GMT

In order to handle some scheduling tasks with the help of a large language model, we can create a set of tools that the LLM can use to process data. LangChain presents this pattern as Agents and Tools.

Instead of using those, for now I've just defined some of my own functions so that I can call the functions without needing to serialize all of the parameters. I'll send a system message to the GPT model letting it know which tools it has available to it.

You are a scheduling assistant who works with users to find the best times for meetings by comparing their schedules with the schedules of other guests. However, you will not speak directly with the user; you will instead communicate with the user using function calls. You will also have access to functions for determining the calendar information and schedule overlaps.

Whenever you respond to any incoming data, you will not respond with regular text. Instead, you will respond only with a JSON object of the following type:

/**
* A response. Includes the name of the function to call, the required parameters for that function, and the reasoning behind calling that function.
*/
type Response = {
  function: string,
  params: any[],
  reason: string,
}

Do not improvise. Here are the function types and their descriptions, along with any other relevant types:

/**
 * Sends a message to the user and receive their response.
 */
type CommunicateWithUserFunc = (
  message: string
) => { user: User; timestamp: string; response: string };

type User = {
  name: string;
  email: string;
};

/**
 * Returns a list of all users and their email addresses.
 */
type GetUsersFunc = () => User[];

/**
 * A calendar event. Has a start and end time in ISO format. Has a title, and the email of the event owner.
 */
type Event = {
  startTime: string;
  endTime: string;
  title: string;
  owner: string;
};

/**
 * A potential meeting time.
 */
type MeetingTime = {
  startTime: string;
  endTime: string;
};

/**
 * Takes in a list of users, and returns a list of potential meeting times.
 */
type FindMeetingTimeFunc = ({
  users,
  duration = 30,
  startDate,
  endDate,
}: {
  users: User[];
  duration: number;
  startDate?: string | Date;
  endDate?: string | Date;
}) => MeetingTime[];

Now that the model knows the tools it has at hand, we can test it out by sending a user message.

USER

{
  user: { name: "Karey Higuera", email: "karey@email.com" },
  timestamp: "2023-05-11T17:09:59-0400",
  response:
    "Hey there! I need to schedule a meeting with Kishore for this week.",
};

ASSISTANT

{
  "function": "GetUsersFunc",
  "params": [],
  "reason": "Retrieve the list of all users and their email addresses."
}

USER

[
  {name: 'Karey Higuera', email: 'karey@email.com'},
  {name: 'Kishore Mecal', email: 'kishore@email.com'},
  {name: 'Kabel Lemonk', email: 'kabel@email.com'},
  {name: 'Artair Umero', email: 'artair@email.com'}
]

ASSISTANT

{
  "function": "FindMeetingTimeFunc",
  "params": [{
    "users": [
      {"name": "Karey Higuera", "email": "karey@email.com"},
      {"name": "Kishore Mecal", "email": "kishore@email.com"}
    ],
    "duration": 60,
    "startDate": "2023-05-15",
    "endDate": "2023-05-19"
  }],
  "reason": "Find potential meeting times for Karey Higuera and Kishore Mecal with a duration of 60 minutes between May 15th and May 19th."
}

USER

[
  {
    startTime: '2023-05-16T05:00:00.000Z',
    endTime: '2023-05-16T06:00:00.000Z'
  },
  {
    startTime: '2023-05-17T08:00:00.000Z',
    endTime: '2023-05-17T09:00:00.000Z'
  },
  {
    startTime: '2023-05-19T19:00:00.000Z',
    endTime: '2023-05-19T20:00:00.000Z'
  }
]

ASSISTANT

{
  "function": "CommunicateWithUserFunc",
  "params": [
    "I have found some potential meeting times for you and Kishore. Here are the options:\n\n1. May 16th, 5:00 AM - 6:00 AM\n2. May 17th, 8:00 AM - 9:00 AM\n3. May 19th, 7:00 PM - 8:00 PM\n\nPlease let me know which option works best for you."
  ],
  "reason": "Provide the user with the potential meeting times and ask for their preferred option."
}

USER

{
  user: {
    name: "Karey Higuera",
    email: "karey@email.com",
  },
  timestamp: "2023-05-11T19:33:02.879Z",
  response: "Can we actually do a 30 minute meeting? I don't need a whole hour",
};

ASSISTANT

{
  "function": "FindMeetingTimeFunc",
  "params": [{
    "users": [
      {"name": "Karey Higuera", "email": "karey@email.com"},
      {"name": "Kishore Mecal", "email": "kishore@email.com"}
    ],
    "duration": 30,
    "startDate": "2023-05-15",
    "endDate": "2023-05-19"
  }],
  "reason": "Find potential meeting times for Karey Higuera and Kishore Mecal with a duration of 30 minutes between May 15th and May 19th."
}

USER

[
  {
    startTime: "2023-05-16T05:00:00.000Z",
    endTime: "2023-05-16T05:30:00.000Z",
  },
  {
    startTime: "2023-05-17T08:00:00.000Z",
    endTime: "2023-05-17T08:30:00.000Z",
  },
  {
    startTime: "2023-05-19T19:00:00.000Z",
    endTime: "2023-05-19T19:30:00.000Z",
  },
]

ASSISTANT

{
  "function": "CommunicateWithUserFunc",
  "params": [
    "Great! I have found some potential meeting times for a 30-minute meeting between you and Kishore. Here are the options:\n\n1. May 16th, 5:00 AM - 5:30 AM\n2. May 17th, 8:00 AM - 8:30 AM\n3. May 19th, 7:00 PM - 7:30 PM\n\nPlease let me know which option works best for you."
  ],
  "reason": "Provide the user with the updated potential meeting times for a 30-minute meeting and ask for their preferred option."
}

From note Sample calendaring conversation with GPT-3.5

Create a custom LlamaIndex loader for Google Calendar

Tue, 16 May 2023 13:56:12 GMT

The published GoogleCalendarReader only accesses events from the 'primary' calendar; that is, the calendar of the user whose credentials are used.

We need the calendar events from any users that we specify, so I've written a new loader that allows for that.

"""Google Calendar data loader. Pulls in events from the Google Calendar API for specified users."""

import datetime
import os
from typing import Any, List, Optional, Union

from llama_index.readers.base import BaseReader
from llama_index.readers.schema.base import Document

SCOPES = ["https://www.googleapis.com/auth/calendar.readonly"]


class GoogleCalendarReader(BaseReader):
    """Google Calendar data loader.

    Pulls in events from the Google Calendar API for specified users.

    """

    def load_data(
        self,
        users: Optional[List[str]] = ["primary"],
        number_of_results: Optional[int] = 100,
        start_date: Optional[Union[str, datetime.date]] = None,
        end_date: Optional[Union[str, datetime.date]] = None,
    ) -> List[Document]:
        """Load data from users' calendars.

        Args:
            users (Optional[List[str]]): a list of the users' email addresses to load data for. Defaults to 'primary' (own calendar)
            number_of_results (Optional[int]): the number of events to return. Defaults to 100.
            start_date (Optional[Union[str, datetime.date]]): the start date to return events from. Defaults to today.
            end_date (Optional[Union[str, datetime.date]]): the end date to return events from. Defaults to 7 days from start_date.
        """

        from googleapiclient.discovery import build

        credentials = self._get_credentials()
        service = build("calendar", "v3", credentials=credentials)

        if start_date is None:
            start_date = datetime.date.today()
        elif isinstance(start_date, str):
            start_date = datetime.date.fromisoformat(start_date)

        if end_date is None:
            end_date = start_date + datetime.timedelta(days=7)
        elif isinstance(end_date, str):
            end_date = datetime.date.fromisoformat(end_date)

        start_datetime = datetime.datetime.combine(start_date, datetime.time.min)
        start_datetime_utc = start_datetime.strftime("%Y-%m-%dT%H:%M:%S.%fZ")
        end_datetime = datetime.datetime.combine(end_date, datetime.time.max)
        end_datetime_utc = end_datetime.strftime("%Y-%m-%dT%H:%M:%S.%fZ")

        events = []
        for user in users:
            events_result = (
                service.events()
                .list(
                    calendarId=user,
                    timeMin=start_datetime_utc,
                    timeMax=end_datetime_utc,
                    maxResults=number_of_results,
                    singleEvents=True,
                    orderBy="startTime",
                )
                .execute()
            )
            events += events_result.get("items", [])

        if not events:
            print("No upcoming events found.")
            return []

        results = []

        for event in events:
            if "dateTime" in event["start"]:
                start = event["start"]["dateTime"]
            else:
                start = event["start"]["date"]

            if "dateTime" in event["end"]:
                end = event["end"]["dateTime"]
            else:
                end = event["end"]["date"]

            title = event.get("title", None)
            summary = event.get("summary", None)
            event_string = f"Status: {event['status']}, "
            if title is not None:
                event_string += f"Title: {title}, "
            if summary is not None:
                event_string += f"Summary: {summary}, "
            event_string += f"Start time: {start}, "
            event_string += f"End time: {end}, "

            organizer = event.get("organizer", {})
            display_name = organizer.get("displayName", "N/A")
            email = organizer.get("email", "N/A")
            if display_name == "N/A":
                event_string += f"Organizer: {email}"
            else:
                event_string += f"Organizer: {display_name} ({email})"

            results.append(Document(event_string, event["id"]))

        return results

    def _get_credentials(self) -> Any:
        """Get valid user credentials from storage.

        The file token.json stores the user's access and refresh tokens, and is
        created automatically when the authorization flow completes for the first
        time.

        Returns:
            Credentials, the obtained credential.
        """
        from google.auth.transport.requests import Request
        from google.oauth2.credentials import Credentials
        from google_auth_oauthlib.flow import InstalledAppFlow

        creds = None
        if os.path.exists("token.json"):
            creds = Credentials.from_authorized_user_file("token.json", SCOPES)
        # If there are no (valid) credentials available, let the user log in.
        if not creds or not creds.valid:
            if creds and creds.expired and creds.refresh_token:
                creds.refresh(Request())
            else:
                flow = InstalledAppFlow.from_client_secrets_file(
                    "credentials.json", SCOPES
                )
                creds = flow.run_local_server(port=0)
            # Save the credentials for the next run
            with open("token.json", "w") as token:
                token.write(creds.to_json())

        return creds


if __name__ == "__main__":
    reader = GoogleCalendarReader()
    print(reader.load_data())

Enable logging with LlamaIndex

Thu, 11 May 2023 18:29:02 GMT

As of version 0.6.0, the llama_index library no longer automatically logs runtime information and details about its processes. However, you can enable logging by adding the following to the beginning of your script:

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

This will show the token counts for embedding and LLM usage.

The level can be changed to logging.DEBUG for even more details, such as the actual data it's processing.

Configure LlamaIndex to read from Google Calendar

Tue, 09 May 2023 20:37:15 GMT

LlamaIndex can read from the Google Calendar API, but to do so, you must first configure a credentials file.

To generate the credentials file, first navigate to this guide on creating credentials. To read from the Calendar, we'll need an OAuth client ID (see relevant heading). Click the Go to Credentials button and navigate to the Credentials dashboard.

If you don't yet have an account, sign up.

If you don't yet have a service account, make one.

Next, navigate to the Google API Library, search for the Google Calendar API, and click Enable.

Return to the Credentials dashboard, click Create Credentials, and choose OAuth client ID. Fill out the required details on the first screen (OAuth consent screen).

Next, you'll need to add a scope to read from the calendar. Click Add or remove scopes and add the following scope from the list:

.../auth/calendar.readonly

Return to the Credentials dashboard, click Create Credentials, and click OAuth client ID. Under Application type, choose Desktop app, add a name, then click Create.

On the pop-up window that appears, click Download json and save your credentials file as credentials.json. This credentials file should live alongside the python script you'll use.

The script

Place the following script in a python file alongside your credentials file:

# llama_index 0.6.5
from llama_index import GPTVectorStoreIndex, download_loader

# Setup data loader
GoogleCalendarReader = download_loader('GoogleCalendarReader')
loader = GoogleCalendarReader()

# load data
documents = loader.load_data()
index = GPTVectorStoreIndex.from_documents(documents)

# query model
query_engine = index.as_query_engine()
response = query_engine.query('<your question>')
print(response)

When you run the application for the first time, a browser window will open asking you to consent to API access.

Once you consent, the script will run and answer your question.

Keep in mind that the default model for LlamaIndex is text-davinci-003. See Change the model used by LlamaIndex. The model is not aware of the current date.

Access calendar events from other users' calendars

The default LlamaIndex Google Calendar loader only loads calendar events for the authenticated user. In order to access calendar events for other users as well, I've written my own Google Calendar loader.

Problems using a Vector store for calendar events

The basic index used in LlamaIndex examples is the GPTSimpleVectorIndex. However, a vector store is most useful for creating a cloud of ideas and doing text analysis. It is not a good data structure for handling calendaring events.

Here are all the LlamaIndex index types.

A better index type for calendar events is probably the List index. The pros to this are that the LLM can examine all of the events to come to a conclusion. The cons are that we need to pass ALL the events through to the model, and this takes a while and uses up a lot of tokens. Running a test with this method took over a minute for a response when comparing only two users' calendars.

It seems like it would be better to handle calendaring logic in code itself, and have the LLM be the conversationalist/middle-man between the user and the code.

Change the model used by LlamaIndex

Tue, 09 May 2023 20:32:43 GMT

By default, LlamaIndex uses the text-davinci-003 model. For improved results, you can change to a different model using these instructions.

We are currently using OpenAI models, but LlamaIndex can be connected to any LLM that the LangChain project supports.

Here, I am going to change from text-davinci-003 to gpt-3.5-turbo.

`gpt-3.5-turbo` is technically a chat model, not a text model. While LlamaIndex's abstraction still works, this is something to keep in mind as things are different under the hood.

We can take our script from Configure LlamaIndex to read from Google Calendar as an example.

We'll need to pull in some new dependencies that allow us to define the model we'd like to use (namely LLMPredictor and ServiceContext from llama_index, and ChatOpenAI from langchain.

from llama_index import GPTVectorStoreIndex, download_loader, LLMPredictor, ServiceContext
from langchain.chat_models import ChatOpenAI

Now, we can define the LLM (large language model) we would like to use, the create a service context with that LLM.

> Whenever we define the LLM we'd like to use, we can set it's temperature. Temperature is a representation of how "random" or "creative" the model will be. Since we don't necessarily want the model to be creative and want it to base its conclusion simply on the data we provide, we set it to 0.

# define LLM
llm_predictor = LLMPredictor(llm=ChatOpenAI(temperature=0, model_name="gpt-3.5-turbo"))
service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor)

Finally, we pass that service context to the vector index when we create it.

# Setup data loader
GoogleCalendarReader = download_loader('GoogleCalendarReader')
loader = GoogleCalendarReader()

# load data
documents = loader.load_data()
index = GPTVectorStoreIndex.from_documents(documents, service_context=service_context)

# query model
query_engine = index.as_query_engine()
response = query_engine.query('Today is May 9th, 2023. When is my next meeting with Austin?')
print(response)

Now, any queries to the index will use the specified model.

What is LlamaIndex?

Tue, 09 May 2023 20:31:04 GMT

LlamaIndex is an open-source project built on top of LangChain that provides a way to give context to large language models so that they can give more informed responses.

How does it work?

LlamaIndex comes with various data loaders that can be found on the Llama Hub. After loading data in from an API, documents, or any other data source, that data is split into smaller chunks. Then, the smaller chunks are loaded into a vector database, also known as an embedding.

Vector databases are multi-dimensional arrays that allow a large language model to see how closely words are related. Using a vector database allows us to provide more context and give "memory" to a LLM.

Once LlamaIndex has a vector database built, you can query the LLM. LlamaIndex acts as a broker and uses a receiver method to find relevant data from the vector database, which it then sends along to the LLM so that it can answer the query with context. Read more about LangChain indices and receivers.

Photo restoration using Lama-cleaner

Thu, 04 May 2023 14:35:36 GMT

I've just heard about this open-source tool called Lama-cleaner. Based on inpainting and Stable diffusion, one of the main tools used in AI image generation, this tool lets you easily erase blemishes and objects from old photos.

I've manually attempted to erase these cracks and tears before, but it was long and hard work without satisfactory results. However, Lama-cleaner lets you simply draw over the detail, and it removes it after only a few seconds.

Take a look at my first two attempts:

This photo had a bend and tear across the middle, along with some stains on the edges.

This photo had actually lifted, separated from the backing, and cracked across the entire image.

Inkstrike

Wed, 03 May 2023 17:48:36 GMT

Wand, rare (requires attunement)

"Once wielded by the intrepid journalist Jay, Inkstrike serves as a reminder that the pen can indeed be mightier than the sword. Using the quill's arcane power, he uncovered hidden truths and delivered truly hard-hitting journalism." - The Lore of the Inkweaver

The wielder can use an action to shoot a magical ink projectile with a range of 30 feet.
The base damage of the projectile is (INT)d4! (exploding dice ¹).
On each use, roll a d20. If the wielder rolls a 1, the ink from the quill gets in their eyes, and they become blinded for one minute.

Footnotes

This means that if you roll the highest value of a dice, the dice "explodes" and you roll again. For example, if I roll a d6 and get a 6, I can roll again and sum the values. This continues ad infinitum. ↩

The Seamstress

Mon, 01 May 2023 15:53:40 GMT

A card from the Tarnish deck, a Tarot-like deck of cards for a homebrew Dungeons and Dragons campaign I'm working on.

Information

You turn the card over and see The Seamstress. Suddenly, your vision goes black and you awake in an old, wooden room filled to the brim with fabrics, buttons, quilts, and all manner of threads and needles. The Seamstress is leaning over a large tapestry, pulling a thread tight. She looks up and smiles when she sees you and beckons you over, motioning for you to look at her work. The images on the tapestry shift and move as you look at them. A large, grey whale surfaces and sprays red water. Waves ripple out and become musical notes and smoke that swirl above a large bonfire as hooded figures dance in circles. As you peer deeply into the tapestry, the Seamstress suddenly reaches out and shoves her hand into your chest. You feel a sharp pinch as she withdraws her hand, pulling out a heartstring. "With your help, I'll finally finish this story". She dangles the thread above the tapestry and dips it in, and the fabric greedily absorbs it. The images begin to swirl before you, when suddenly the tapestry leaps off the table and wraps itself around you, smothering you into blackness. You suddenly awake standing back where you were as you drew the card, and not a second has passed. The card crumbles to ash in your hand.

Effect

You suddenly know with a surety that you will die in 7 days. It is as unavoidable as the rising and setting of the sun itself, and you know without a doubt that your death is a surety. However, you feel some sort of deep connection to the tapestry, to this island, and to this world. You will gain a level each day until you die.

ASCII Text in ChatGPT with a virtual machine

Mon, 17 Apr 2023 15:00:57 GMT

ChatGPT and other systems built on large language models are not skilled at understanding ASCII text and ASCII art, the practice of using unicode symbols to draw larger pictures. AI Weirdness - What Does This Say is the most recent article I stumbled upon in which the author shows how poorly these models generate and read ASCII text.

In Engraved - Building a Virtual Machine inside ChatGPT, an article published late last year, the author was able to have ChatGPT pretend to be a linux machine, and the conversation acted much more like a user interacting with a Linux command line. I decided to experiment to see if using command line tools for generating ASCII art and text in this "virtual machine" mode work any better than ChatGPT in conversation mode.

For this experiment, I'll be using the relatively new GPT-4 model.

ASCII text with vanilla ChatGPT

First, let's get a baseline performance with ChatGPT and ask it to turn the word "soup" into ASCII text.

I'll give it a 3/10; it got the first letter right and didn't create any nonsense shapes. It wasn't, however, able to correcly spell soup. Let's give it a few more tries.

Overall, I'm impressed that it was able to consistently produce text in general and not mangled garbage, but I am disappointed with the results.

ASCII text with LinuxGPT and `figlet`

Next, let's start a new session and invite ChatGPT to enter this virtual machine mode.

We're in! Next, let's "install" the CLI tool that we'll be using to create our ASCII text, figlet.

It's so interesting to see ChatGPT pretend to be a Linux terminal so well. Now that we have our figlet tool installed, let's try to use it to spell out "soup" again.

ChatGPT knows what figlet is, how it works, and what it's supposed to do! Unfortunately, though, we can't trick ChatGPT into a better performance.

As a last ditch attempt, what if we try an arguably more common word like "cat"?

Hey look, it worked! It took the liberty of capitalizing it, but I'll count this as a win. What if we try "dog"?

Okay, it definitely seems like we might have discovered a bias in the model 😆.

ASCII text with LinuxGPT and `cowsay`

The last thing I want to try is using cowsay, a tool for making an ASCII cow say things (I know, very on the nose). Let's install cowsay.

And now let's have our cow say "soup":

Nice! We've got a success. Here's a regular Linux terminal output for comparison:

This makes sense; ChatGPT isn't having to come up with any new art here. It can simply copy the cow and replace the speech bubble with the plain text provided. I'm sure we can break this, though. First, let's give it a lengthy chunk of text.

Wow, it handled it like a champ! Here's the regular terminal output for comparison.

Let's try something else. cowsay provides some options for customization, and even (despite its name) provides some other animals than just this cow. We'll hit it with some customization options: cowsay -e TT -T U Arma virumque cano, Troiae qui primus ab oris Italiam will have eyes as "TT" and the tongue as "U".

And there we see the failure immediately. ChatGPT didn't know where to put the eyes or tongue, so it just tossed them into a random place in the generated image. It should have looked like this:

ChatGPT still has a ways to go

Overall, these large language models are extremely impressive and useful, but they definitely have their limitations! Have fun experimenting 🤖

Improving the DnD Constitution attribute

Tue, 03 Jan 2023 04:32:02 GMT

In Dungeons and Dragons, the Constitution attribute is the only attribute that has no related skills. There is only a saving throw, usually used for resisting poison or drunkenness. This means that characters end up dedicating valuable points to an attribute that is rarely used. What are some ways to make Constitution more than just a measure of a character's health points? How can we give players more opportunities to use Constitution in meaningful ways during gameplay?

New skills

The most obvious option that comes to mind is creating new skills that depend on Constitution. Adding more skills to Constitution can open the game up to richer, more nuanced play and story-telling. This can allow the players to think more deeply about how some of these attributes can be used.

Let's take a look at some new potential Constitution skills.

Endurance

Anytime a player character needs to perform some physical feat, they almost always roll a Strength check. However, longer-lasting activities that rely more on stamina than brute strength are a great candidate for a Constitution check. Take, for example, the following activities:

long-distance running
long-distance swimming
holding breath
maintaining strength without food and drink
resistance to exhaustion (e.g., doctors working long shifts or soldiers executing long operations without sleep)

If a character has gone an extended period without a long rest, you could have them roll an Endurance check to see if they resist gaining a level of exhaustion a while longer.

The Endurance skill could also come into play for chases (see DMG, p. 242). Instead of rolling Constitution saving throws, players can roll Endurance checks to determine whether they can dash again without taking exhaustion levels.

**Update 1-14-2023**

I began playing Dungeons and Dragons about 6 months ago, so I'm very new to the TTRPG scene. I showed this article to a friend, who immediately pointed me to the Endurance feat from Dnd 3.5e🤦‍♂ It's reassuring, at the very least, to know that other people have thought about these same problems.

Iron Gut

This one might not have as many obvious uses, but it fits cleanly into Constitution. This could apply to activities like:

competition eating or drinking
surviving with limited rations
eating dangerous food

A character with proficiency in Iron Gut could consume less palatable food or drink without becoming sick. This would be useful in survival situations in the wilderness.

Recovery

Constitution is essential to establishing a character's overall health. Shouldn't it also affect how quickly they can recover from injuries or exhaustion?

A potential idea would be to alter the healing mechanic to make healing be more or less effective based on a Recovery skill roll. When a character attempts to regain health (via potion, healing spell, hit dice, &c.), they roll for Recovery. A roll of 7 or below heals for half the amount, 8-16 heals regularly, and 17 or above offers twice the amount of health.

Players could also keep track of more granular details of injuries that their characters sustain and the time it takes those to heal. If a character badly injures their arm in a fall, they may need to roll Recovery checks each morning until they are able to perform regular activities again.

Hardiness

With higher Constitution, a character might be able to withstand the effects of extreme environments. In the Dungeon Master's Guide (DMG p.110), some extreme weather conditions are mentioned, such as Extreme Heat, Extreme Cold, and Frigid Water. These conditions cause players to roll Constitution saving throws to avoid ill effects such as exhaustion.

Instead of rolling Constitution saving throws, the players could roll a Hardiness check to resist the heat or cold. This could even be applied to combat, allowing the player to shrug off some fire or ice damage from a spell.

Resilience

A skill like Resilience could help the character resist physical damage. If a character suffers fall damage, a Resilience roll could determine if that fall damage is lessened or increased from the baseline.

It could even be used in combat. A player could roll a Resilience check to attempt to shrug off the effects of a critical hit; if they beat a DC 20, then they just take regular damage instead.

Be careful with this one! Getting a critical hit feel awesome as a player, but having that immediately disregarded because an NPC rolls a good Resilience check could be disheartening. Don't make the game miserable, and be careful about combat balance.

Isn't this just complicating things?

"Wait!" You might exclaim. "Most of the situations that could come up in-game can just be handled with a saving throw!" If we follow that thread, though, we could just start reducing all other skill checks to saving throws as well.

Instead of telling your player to "make a Sleight of Hand check to pickpocket the guard", you could reduce this to "make a DEX save to see if you can pull your hand back with the key before he notices you."
Instead of saying "make an Athletics check to see if you can climb this wall", you could reduce it to "make a STR saving throw to see if you can climb the wall before your strength gives out."

I'd also argue that there's a reason we have proficiencies and expertises in the other attributes. Let's take Charisma for example: a Bard might have very good Charisma with expertise in Performance. Having a high Charisma attribute means he will inherently be better than average at all CHA checks, but being good at Performance doesn't automatically make someone good at Intimidation.

The same logic should apply to Constitution. Someone with good CON would probably be better on average at holding their breath than someone with low CON, but that doesn't mean they're equally good at holding their breath as they are at long-distance running or toughing out extremely cold weather. Introducing new skills can make Constitution a richer element to explore in games.

Give it a shot!

In the end, the Constitution attribute in Dungeons and Dragons has the potential to be much more than just a measure of a character's health points. By introducing new skills like Endurance, Iron Gut, Recovery, Hardiness, and Resilience, players can make use of Constitution in a variety of situations and add depth to their characters. These skills not only add new gameplay elements, but also allow players to role-play their characters in more interesting and realistic ways. Whether you're a seasoned D&D player looking to add new dimensions to your game or a beginner seeking to fully utilize your character's attributes, these new Constitution skills are worth considering. Give them a try in your next game and see how they enhance your play experience!

Retrieve the timezone from a JavaScript date

Tue, 29 Nov 2022 00:34:47 GMT

If you have a date object and want to display the time to the user, it can be useful to display the timezone as well. The only way to show the timezone normally is to display the full date in a long format.

const date = new Date('2022-11-29T19:00:00-05:00');
date.toLocaleString('en-US', {
  timeZoneName: 'short',
});
// 11/29/2022, 7:00:00 PM EST

However, we're not allowed to pass other options like dateStyle if we use timeZoneName. So if we want to format the date ourselves, we can use the Intl.DateTimeFormat() formatter to get the date back in an array of pieces. The last piece will be our time zone.

const date = new Date('2022-11-29T19:00:00-05:00');

const dateParts = new Intl.DateTimeFormat('en-US', {
  timeZoneName: 'short',
}).formatToParts(date);
/** dateParts
   {
     [
      { type: 'month', value: '11' },
      { type: 'literal', value: '/' },
      { type: 'day', value: '29' },
      { type: 'literal', value: '/' },
      { type: 'year', value: '2022' },
      { type: 'literal', value: ', ' },
      { type: 'timeZoneName', value: 'EST' }
    ]
   }
*/

const timeZone = dateParts.pop().value
console.log(timeZone)
// EST

You can also get some other formats by changing timeZoneName:

{
  "short": "EST",
  "long": "Eastern Standard Time",
  "shortOffset": "GMT-5",
  "longOffset": "GMT-5:00",
  "shortGeneric": "ET",
  "longGeneric": "Eastern Time"
}

Making a state machine in TypeScript

Fri, 25 Nov 2022 23:13:01 GMT

State machines are a nifty programming algorithm to control application flow. In it's simplest form, a state machine has a set of states that it can exist in and a set of actions that can occur in each of those states.

A great example is a traffic light. In a standard traffic light, the three states are green, yellow, and red. There is one action, change. When the light is in the green state and receives the change action, it will always go into the yellow state. There is a clear order that can't be violated.

The state machine knows the flow of these states and their actions, and it will never allow your program to get into an incorrect state (as long as your states and actions are defined correctly!). This gives you much cleaner and predictable code; instead of needing to check a bunch of conditionals like if (!error && !loading && response), you can just check the state using if (state === 'ready').

If you plan on using many state machines that could have nested logic, I recommend reaching for a strong library like xstate. But if you just need a one-off, simple machine, you can make one yourself!

For this example, we'll make a state machine to run a robotic fortune-teller. Think of this like one of those Zoltar fortune teller machines you might see at an old carnival; the user can push a button, and the robot will tell them either a good or bad fortune.

Though this is a silly example, think about how this model can be applied in real applications. For example, instead of getting back a good or bad fortune from a robot, you could get back a response or an error from an API!

Modeling our states

First, we'll need to clearly define the states and actions. It's a great help to draw this out visually.

The robot will first greet the user and invite them to push the button to receive a fortune. Whenever the user pushes the button, the robot will take a moment to look into his crystal ball to find a fortune. Once the robot receives the fortune back, it will display the fortune to the user. Since it could be a good or bad fortune, we can display those differently! The bad fortune will flash scary lights and play spooky music, while a good fortune will play happy music. Regardless of the type of fortune the user gets back, they can push the button again to get another fortune.

Now that we have our robot behavior drawn out, let's rewrite our states and actions to be more program-focused instead of behavior focused.

We first show our greet state. When the user pushes the button, we'll send a request to whatever service or man-behind-the-curtain we're using to generate the fortunes, and we'll show our waiting state. Once we get a response back, we can show the fortune based on whether it's good or bad. Once the response is shown, the another response can be sent, which puts our robot back in the waiting state.

With our states and actions clearly defined, we can finally start writing some code! While we could manage all of this with strings, we'll be using enums to help prevent any accidental mispellings or errors.

enum MachineStates {
  greet = 'greet',
  waiting = 'waiting',
  show_bad = 'show_bad',
  show_good = 'show_good',
}

enum MachineActions {
  send_request = 'send_request',
  receive_bad = 'receive_bad',
  receive_good = 'receive_good',
}

type MachineState = keyof typeof MachineStates
type MachineAction = keyof typeof MachineActions

We've also created two types based on our enums. Using the keyof typeof keywords lets us pull all the keys of the enums, which is essentially equivalent to the following:

type MachineState = 'greet' | 'waiting' | 'show_bad' | 'show_good'
type MachineAction = 'send_request' | 'receive_bad' | 'receive_good'

However, we get the added benefit of these types automatically updating any time we alter our enums.

Building the state machine

Since we have our actions and states defined, we need to lay out the same model that we had in our diagram. Our machine needs to know which actions are allowed from which states, and what to do when it receives a certain action. This is just going to be a JSON object, so let's mock out what it might look like so that we can then define a type for it.

const stateMachine = {
  'greet': {
    'send_request': 'waiting'
  },
  'waiting': {
    'receive_bad': 'show_bad',
    'receive_good': 'show_good'
  }
  // ...
}

We can see that each key in our state machine needs to be one of our possible states. We'll look at the first state, greet, as an example. When we're in the greet state, the only action we can take is send_request. When that action occurs, it takes us to the waiting state. So we have the top level keys being possible states, second level keys being possible actions, and our values being the resulting state. Let's make a type for this.

type StateMachine = {
  [state in MachineStates]: {
    [action in MachineActions]?: MachineState
  }
}

For our top level keys, we're essentially looping through all of the states in our MachineStates enum. That object's keys come from a loop of all of our MachineActions, which point to a single resulting MachineState.

Nota bene: we've included a ? after the machine actions. This means that all of our states are required to be filled out, but each state doesn't have to accept every action.

With our type defined, let's fill out our actual machine.

const stateMachine: StateMachine = {
  [MachineStates.greet]: {
    [MachineActions.send_request]: MachineStates.waiting
  },
  [MachineStates.waiting]: {
    [MachineActions.receive_bad]: MachineStates.show_bad,
    [MachineActions.receive_good]: MachineStates.show_good,
  },
  [MachineStates.show_bad]: {
    [MachineActions.send_request]: MachineStates.waiting,
  },
  [MachineStates.show_good]: {
    [MachineActions.send_request]: MachineStates.waiting,
  }
}

Great! Now that we have our machine modeled and built, we need a way to utilize the machine.

Querying the state machine

We're going to make a function that accepts the current state and the action that occurs then uses the state machine to tell us the new state we should be in.

const getNextState = (
  state: MachineState,
  action: MachineAction
) => {
  const newState = stateMachine[state][action];
  return newState;
}

Our function just has to lookup the new state we should be in using our JSON object! We can even have it throw an error if we try to perform an action that's impossible from our current state.

const getNextState = (
  state: MachineState,
  action: MachineAction
) => {
  const newState = stateMachine[state][action];
    if (!newState) {
    throw new Error(`Action ${action} is not a valid step from state ${state}`);
  }
  return newState;
}

One of the nicest things about this machine is that it's stateless. The machine isn't keeping track of what state we're in; it just tells you how to respond based on a certain state and action. This means that the state machine will be really easy to test 😎

Using the state machine in an application

Fantastic! Now that we have our machine built and know how to use it, we can plug it into an application. Instead of actually controlling a robot, though, it'll be easier to demonstrate how it would be used in a React component.

We'll start off with a barebones functional component.

const FortuneTeller = () => {
  return (
    <div>
      <h1>Fortune Teller</h1>
    </div>
  )
}

Since our state machine is stateless, we'll need to keep track of our state in this component using the useState hook. We can initialize the state to the greet state. We'll also eventually need a variable for our fortune, so we'll create it now too.

const FortuneTeller = () => {
  const [state, setState] = useState<MachineState>(MachineStates.greet)
  const [fortune, setFortune] = useState<string>('');
  return (
    <div>
      <h1>Fortune Teller</h1>
    </div>
  )
}

And now, one of the coolest parts about using a state machine: since we have our states defined, we know all the states that our frontend UI can be in! We can build them out without needing to worry about complicated conditionals.

const FortuneTeller = () => {
  const [state, setState] = useState<MachineState>(MachineStates.greet)
  return (
    <div>
      <h1>Fortune Teller</h1>
      {state === MachineStates.greet && (
        <p>Press the button to get your fortune!</p>
    )}
      {state === MachineStates.waiting && (
      <p>Waiting for fortune...</p>
    )}
      {state === MachineStates.show_bad && (
        <div>
          <p>Uh oh! You got a bad fortune!</p>
          <p>{fortune}</p>
        </div>
      )}
      {state === MachineStates.show_good && (
        <div>
          <p>Nice! You got a good fortune!</p>
          <p>{fortune}</p>
        </div>
      )}
    </div>
  )
}

We just add a check to see which state we're in, and display the corresponding UI. Next, we can add our button in so the user can interact and request a fortune.

const FortuneTeller = () => {
  const [state, setState] = useState<MachineState>(MachineStates.greet)
  return (
    <div>
      <h1>Fortune Teller</h1>
      {state === MachineStates.greet && <p>Press the button to get your fortune!</p>}
      {state === MachineStates.waiting && <p>Waiting for fortune...</p>}
      {state === MachineStates.show_bad && (
        <div>
          <p>Uh oh! You got a bad fortune!</p>
          <p>{fortune}</p>
        </div>
      )}
      {state === MachineStates.show_good && (
        <div>
          <p>Nice! You got a good fortune!</p>
          <p>{fortune}</p>
        </div>
      )}
      <button
        onClick={() => requestFortune()}
      >
        Get a fortune!
      </button>
    </div>
  )
}

We need to be careful, though. What happens if the user clicks to get a fortune, but then clicks again while we're in the waiting state? We know from our state machine that the state waiting can't accept the action send_request!

We have two options:

don't display the button if we're in the waiting state
disable the button if we're in the waiting state

Both of these options are valid! You can choose which fits best with your project.

{state !== MachineStates.waiting && (
  <button
    onClick={() => requestFortune()}
  >
  Get a fortune!
  </button>
)}
// OR
<button
  onClick={() => requestFortune()}
  disabled={state === MachineStates.waiting}
>
  Get a fortune!
</button>

Great! Now the last step is to implement the function that actually requests a fortune, requestFortune():

const requestFortune = async (): Promise<void> => {
    setState(state => getNextState(state, MachineActions.send_request))
    const [isGoodFortune, fortuneResponse] = await seeFuture();
    setState(state => getNextState(
      state,
      isGoodFortune ?
        MachineActions.receive_good :
        MachineActions.receive_bad
      )
    )
    setFortune(fortuneResponse)
  }

When we begin this function call, we pass in our current state and the action that we're taking, send_request. Look at that! We don't have to worry about what state we want to go to because the state machine takes care of that for us. We just have to tell it what state we're in and what action we want to do.

We make our call to our fortune service seeFuture() and await the response (hey, seeing the future isn't easy!), then send off our next action depending on if the fortune was good or bad. Finally, we update the fortune so it can be displayed to the user.

Nota bene: whenever we update the state using the useState setter function, we have to use the function version. Otherwise, React might try to batch your updates together and skip a step in your machine, which will cause your machine to throw an error.

// ✅ Do this
setState(state => getNewState(state, MachineActions.send_request))

// 🚫 Don't do this!
setState(getNewState(state, MachineActions.send_request))

Go forth and machine!

And there we go! You now have a homebrew state machine to manage your application and write more predictable, maintainable code. Have fun experimenting!

Typeguards and Closures in TypeScript

Thu, 01 Sep 2022 04:00:00 GMT

A closure is when a function calls a variable that is not included in its own scope. For example:

const name = 'kbravh'

const logName = () => {
  console.log(name)
}

If the logName() function is passed around anywhere else, it will still have access to the name variable, even though it's out of scope! JavaScript does us the favor of making sure the function always has access to it.

When we use this closure pattern, though, TypeScript alerts us to a potential issue. If TypeScript cannot ensure that a variable won't change in the closure, it will show a warning. Take the following example:

type Settings = {
  username: string,
  nickname?: string
}

const settings: Settings = {
  username: 'kbravh',
  nickname: 'coolbro'
}

We define a Settings type with an optional property nickname, then we set up a settings object that uses that field. Next, we define a function showNickname that requires a string as a parameter.

const showNickname = (nickname: string) => {
  console.log(nickname);
}

In order to be able to call our function without problems, we then set up a typeguard to make sure that nickname is defined. So when we call showNickname, we don't have any issues!

if (!settings.nickname) {
  throw new Error('Nickname necessary!')
}

showNickname(settings.nickname);

Instead of immediately showing the user's nickname though, let's only show it once the user has clicked somewhere. So we'll set up an event listener that calls our function and passes in the nickname.

if (!settings.nickname) {
  throw new Error('Nickname necessary!')
}

window.addEventListener('click', () => showNickname(settings.nickname))
// ❌ Argument of type 'string | undefined' is not assignable to parameter of type 'string'.

Uh oh! TypeScript isn't happy about that and gives us a type error, saying that the nickname we're passing in might be undefined. But how? We set up our typeguard!

Though it's not immediately apparent, TypeScript is actually correct here. TypeScript is letting you know that even though nickname is defined now, it might not be by the time the closure is actually executed. Before the event listener fires, there might be a line of code that sets nickname back to undefined.

if (!settings.nickname) {
  throw new Error('Nickname necessary!')
}

window.addEventListener('click', () => showNickname(settings.nickname))
// ❌ Argument of type 'string | undefined' is not assignable to parameter of type 'string'.

settings.nickname = undefined

We need to make sure that the code is safe and that nickname is defined when we need it. There are a couple of options for doing so!

Solutions

Move the typeguard

If we move our typeguard into the closure, we can make sure that it's called right before the function executes.

window.addEventListener('click', () => {
  if (!settings.nickname) {
    throw new Error('Nickname necessary!')
  }
  showNickname(settings.nickname)
  // No TypeScript error! 👍
})

We could also move the typeguard into showNickname function itself, or even provide a default value.

const showNickname = (nickname?: string) => {
  if (!nickname) {
    throw new Error('Nickname necessary!')
  }
  console.log(nickname)
}

// or...

const showNickname = (nickname: string = 'Pal') => {
  console.log(nickname)
}

Extract nickname into a `const`

If we can let TypeScript know that the value won't change, it will be able to rest easy.

// Extract nickname into its own const
const nickname = settings.nickname

if (!nickname) {
  throw new Error('Nickname necessary!')
}

window.addEventListener('click', () => showNickname(settings.nickname))
// No TypeScript error! 👍

"But wait!" You might exclaim, "wasn't our settings object defined with const? Why didn't that work?"

Whenever we instantiate an object with const, it just lets TypeScript know that the entire variable cannot be reassigned. The object's properties can still be changed, though!

const settings: Settings = {
  username: 'kbravh'
}

settings = {
  username: 'newuser'
}
// ❌ Cannot assign to 'settings' because it is a constant.

settings.username = 'newuser'
// No TypeScript error

Unfortunately, using Object.freeze() to lock our object properties does not resolve the TypeScript error.

const settings: Settings = {
  username: 'kbravh',
  nickname: 'coolbro'
}

Object.freeze(settings)

// or...

const settings = Object.freeze<Settings>({
  username: 'kbravh',
  nickname: 'coolbro'
})

window.addEventListener('click', () => showNickname(settings.nickname))
// ❌ Argument of type 'string | undefined' is not assignable to parameter of type 'string'.

Finding the coordinates of an element in a 2D array

Tue, 24 May 2022 05:00:00 GMT

While working on a coding challenge that @cassidoo sent out in her latest newsletter, I needed to find the coordinates of a particular number in a two-dimensional array of numbers. Here's a quick method for doing so!

Finding the coordinates using loops

Let's assume that we have a 3x3 array of numbers, and we want to find the location of the number '5'. I've arbitrarily assigned the axes x and y here.

const grid = [
---> y
|  [8,6,7],
|  [5,3,0],
v  [9,1,2]
x
]

While it may be a simple approach, looping through our array is a straightforward way to find our target element. Check out the function below:

const find2dCoordinates = <T>(
  grid: T[][],
  target: T
): {x: number; y: number} => {
  let coords = {x: -1, y: -1};
  grid.every((row, i) =>
    row.every((item, j) => {
      if (item === target) {
        coords = {x: i, y: j};
        return false;
      }
      return true;
    })
  );
  return coords;
};

We first define a set of coordinates: ts~let coords = {x: -1, y: -1}; since we know we'll never have negative coordinates with an array, we can know that we ran into an error if the returned object has negative values.

Next, we're going to loop through each item in each row. Since we want the indices as well as the items themselves, we use the ts~.every() function instead of ts~for...of loops. We then check if each item is equal to our target. If you had an array of objects instead of numbers, you'd want to use some other equality check here.

Now, when we find a match, we store the coordinates in our ts~coords variable. But we have a slight problem! Since we're not using ts~for...of loops, we can't use a break to jump out of our loops early. 😕 Have no fear, because we planned for this 😎 By using ts~.every() instead of ts~.forEach(), we have an escape hatch. The ts~.every() function will evaluate your condition against each item in an array until it finishes the array, at which point it returns ts~true, or until it finds an item that fails, at which point it returns ts~false. So we return ts~true ourselves until we find the item we're looking for, then we return ts~false, which causes ts~every() to jump out early, and which bubbles up the the other ts~.every() function as well. Success!

What about a flattened array?

Sometimes, we may have a two-dimensional array that has been flattened into one dimension. If you've ever worked with PNG images before, this is how their pixel information is stored. So our grid from earlier would be ts~[8,6,7,5,3,0,9,1,2]. How can we find our coordinates now?

First, let's find the index of the number we're looking for. Then, if we know the original length of each row, we can calculate the coordinates it would have had!

export const find2dCoordinatesFlattened = <T>(
  grid: T[],
  target: T,
  rowLength: number
): {x: number; y: number} => {
  const position = grid.findIndex(item => item === target);
  return {
    x: Math.floor(position / rowLength),
    y: position % rowLength,
  };
};

We first use the ts~.findIndex() function to find the index of our target item in the one dimensional array. Then, we use two tricks to find the x and y coordinates. By dividing the index we found by the original row length and rounding down, we can find which row the item belongs on. Then, if we use the modulo operator on the index with the row length, we can see which column it belongs to.

Final thoughts

This was a fun little project, and I recommend you take a shot at the puzzle from the newsletter I mentioned in the intro!

Also, you might have asked yourself if flattening the array ourselves and using the second method might be faster than the first method of loops. Check out these benchmarks results comparing both methods.

First, I ran it on a very small 2x3 array.

Pretty comparable scores! But then I ran it against a 50x10 array.

Yikes 😬 Flattening it ourselves falls way behind. This makes sense, though, as we have to manipulate the entire array first to flatten it, while the loops can simply dive right in and jump out early when they find the result.

HTML Entities from the Twitter v2 API

Sat, 26 Feb 2022 05:00:00 GMT

While working on my Tweet to Markdown projects, I realized that the Twitter API was sending back the text of the tweets with some characters encoded as HTML entities.

What are HTML entities?

HTML entities are short codes used to represent characters that could otherwise be interpreted as HTML code. They begin with an & symbol and end with a semi-colon. For example, if you wanted to write a less-than sign "<", you would need to use the code <. Otherwise, the HTML parser might think that it's the beginning of an HTML tag like the first character in <div>.

If you are working with React and JSX, there is an eslint rule that will warn you if you don't use an HTML entity when you should.

More information regarding HTML entities can be found in the MDN docs, and a full (albeit overwhelming) list of HTML entities can be found as part of the HTML specification.

How to clean up HTML entites

Thankfully, there are plenty of libraries in each programming language that can take care of parsing HTML entities back into their respective symbols. For Node.js, I use the aptly named html-entities library. For example, this is how I would handle retrieving a tweet from the Twitter API.

import { decode } from 'html-entities'
import {default as Axios} from 'axios'

const bearer = `<bearer-token>`
const tweetId = '<tweet-id>'

const tweet = await Axios({
    method: 'GET',
    url: `https://api.twitter.com/2/tweets/${id}`,
    headers: {Authorization: `Bearer ${bearer}`},
  })
    .then(response => response.data)

const decodedText = decode(tweet.data.text)

Once we fetch the tweet from the API, we just pass the text to the js~decode() function, and we'll get back our tweet with the symbols as we expect them.

console.log(tweet.data.text)
// This tweet has an ampersand (&amp;).

console.log(decode(tweet.data.text))
// This tweet has an ampersand (&).

Why would Twitter encode the characters?

I haven't found a documented reason as to why Twitter encodes their tweet text with HTML entities. My best guess is that it's to protect their users from potential mistakes or code injection. If I pull a tweet from the API and embed it in a div using the innerHTML property, any HTML code could be injected there. By encoding all HTML entities in the tweet text, this becomes harmless and removes a potential attack vector.

Encoding QR codes in Braille characters

Sun, 30 Jan 2022 05:00:00 GMT

I stumbled upon an article from Terence Eden in which he and a friend contemplated a method for sending QR codes via a text-only medium such as Twitter. They landed on sending the QR code as black and white emoji squares, like so:

The benefit of sending the QR code this way is that it is scannable via a phone immediately! But there are some downsides:

Here, I was able to adjust the line and character spacing with CSS to remove as much space between the emoji as possible. This is not possible on a 3rd party platform, which would interfere with scanning the code.
The minimum size of a QR code is a 21x21 block, or 441 characters. This is much too big for the current tweet limit of 280 characters. You'd have to use a Micro QR code (starting at 11x11) or an Aztec code (starting at 15x15).

However, if we concede to not being able to immediately scan the QR code when sending it via text, it opens up other avenues for encoding it. What came to mind for me was using Braille unicode characters! Each Braille character is a 2x4 grid, which means that we can compress a QR code down by a factor of 8.

Terminal window

⡏⡭⠭⡍⣟⢫⠛⡏⡭⠭⡍⡇
⡇⠧⠤⠇⡗⡊⡒⡇⠧⠤⠇⡇
⡯⢭⣭⡯⢉⡑⠡⢹⣙⡙⢙⡇
⡷⠽⠭⠬⡅⣨⡽⣪⢃⡐⢏⡇
⡇⡏⠉⡇⡏⣤⠡⢪⢃⡔⣻⡇
⠧⠭⠭⠥⠧⠼⠽⠮⠤⠵⠽⠇

Caveats

Now, in a perfect world, we'd get to use that 8-times compression to its fullest extent. Unfortunately, Braille characters count as 2 characters in Twitter's character count. New lines also each count as a character.

How big of a Braille QR code can we fit then with these limitations? With some quick back of the napkin math, our upper limit can be expressed by $2(x^2) + x = 280$, which comes out to a little over 11. This means that at most, we can tweet an 11x11 Braille grid, giving us a maximum size of a 22x22 QR code. This is better than the emojis, but still rather limited in terms of data density.

For a more in-depth dive into encoding lots of data into a tweet, check out the work done by qntm and the library base2048.

Converting a QR code to Braille characters

To get started, let's take a look at how Braille characters are handled in unicode.

Each Braille character is made up of 8 dots, with each dot assigned a position 1-8. If a dot is present, it's represented as a 1, otherwise a 0. This data is then stored in a binary string, from position 8 down to 1. For example, the character ⢃ has dots in spots 1, 2, and 8. So the binary string would be 10000011.

To get the unicode character from the binary string, we first convert the binary to a hex value, then add an offset of 0x2800. Working with hex and binary values in JavaScript requires casting values back and forth from integers to strings in order to do the base conversions.

const hex = parseInt(parseInt(binary, 2).toString(16), 16);
const address = hex + 0x2800;

Once we have our unicode address, we can get our Braille character.

const character = String.fromCodePoint(address);

We can wrap this up into a function:

const binaryToCharacter = binary =>
  String.fromCodePoint(parseInt(parseInt(binary, 2).toString(16), 16) + 0x2800);

Encode the QR code

So if we have a QR code as an two-dimensional array of 1s and 0s, 1s being white and 0s being black, we can loop through the array and convert each 2x4 block into a Braille character.

It is very likely that part of some of the blocks will fall outside of the QR code. We can just set any of these as 0s.

const qr = [
  [1,1,1,1,1,1,1,1],
  [1,0,0,0,1,1,0,1],
  ...
]

const braille = []

  for (let r = 0; r < qr.length; r += 4) {
    const row = []
    for (let c = 0; c < qr[r].length; c += 2) {
      const binary =
        `${qr[r + 3]?.[c + 1] ?? 0}` +  // 8
        `${qr[r + 3]?.[c] ?? 0}` +      // 7
        `${qr[r + 2]?.[c + 1] ?? 0}` +  // 6
        `${qr[r + 1]?.[c + 1] ?? 0}` +  // 5
        `${qr[r]?.[c + 1] ?? 0}` +      // 4
        `${qr[r + 2]?.[c] ?? 0}` +      // 3
        `${qr[r + 1]?.[c] ?? 0}` +      // 2
        `${qr[r]?.[c] ?? 0}`            // 1

      const characterCode = binaryToCharacter(binary)
      row.push(characterCode)
    }
    braille.push(row)
  }

We now have a two-dimensional array of Braille characters! We can print this to the console as follows:

braille.map(row => row.join('')).join('\n');

Decode the Braille QR code

So now that we've encoded our QR code as Braille and sent it over the wire, we can decode it to get back our original code.

We'll split the string back up into a 2d array, create an empty 2d array that's twice as wide and 4 times as long, then expand each Braille character into the 8 spots it corresponds to.

// Split Braille string into 2d array
const message = `
⡏⡭⠭⡍⣟⢫⠛⡏⡭⠭⡍⡇
⡇⠧⠤⠇⡗⡊⡒⡇⠧⠤⠇⡇
⡯⢭⣭⡯⢉⡑⠡⢹⣙⡙⢙⡇
⡷⠽⠭⠬⡅⣨⡽⣪⢃⡐⢏⡇
⡇⡏⠉⡇⡏⣤⠡⢪⢃⡔⣻⡇
⠧⠭⠭⠥⠧⠼⠽⠮⠤⠵⠽⠇
`;
const braille = message
  .split('\n') // split into rows
  .map(row => row.split('')) // split each row into characters
  .filter(row => row.length); // remove any empty rows

Next, we create our new empty 2d array that will be the size of our original QR code. We'll set all the values to 0 initially; QR codes don't have any black borders, so this will allow us to later clean up any of those extra rows or columns we may have added when the Braille ran over the edges.

let qr = new Array(braille.length * 4)
  .fill(0)
  .map(() => new Array(braille[0].length * 2).fill(0));

Now, we expand each Braille character into it's original 8 spots.

// a helpful function to get our binary string back
const characterToBinary = character =>
  (character.codePointAt(0) - 0x2800).toString(2).padStart(8, '0');

braille.forEach((row, rowIndex) => {
  row.forEach((character, charIndex) => {
    const r = rowIndex * 4;
    const c = charIndex * 2;
    const binary = characterToBinary(character);
    qr[r][c] = +binary[7];
    qr[r + 1][c] = +binary[6];
    qr[r + 2][c] = +binary[5];
    qr[r][c + 1] = +binary[4];
    qr[r + 1][c + 1] = +binary[3];
    qr[r + 2][c + 1] = +binary[2];
    qr[r + 3][c] = +binary[1];
    qr[r + 3][c + 1] = +binary[0];
  });
});

Finally, we can clean up any empty rows and columns. By using flatMap, we can return less elements than were originally in the array in case we get rid of any rows.

qr = qr.flatMap(row => {
  let newRow = row
    .join('')
    .replaceAll(/0*$/g, '')
    .split('')
    .map(num => +num);
  return newRow.length ? [newRow] : [];
});

And there we are! Back to our 2d QR array of 1s and 0s. While this is entirely useless, it's a fun application of some handy unicode characters!

SeptemBIT 2021

Sat, 30 Oct 2021 04:00:00 GMT

The pixel artist @Saultoons hosts a yearly pixel art challenge on Twitter each September. He provides a color palette and daily prompts. I've been wanting to learn to draw for a while, and I figured that pixel art would be a nice way to jump in! I didn't make it all the way through the month, but I'm rather proud of what I was able to accomplish.

Check out the art I was able to produce!

weapon

shield

clothing

map

potion

bag

torch

pet

bard

tankard

food

NPC

mushroom

slime

fairy

portal

rat

skeleton

goblin

orc

eye

trap

chest

treasure

boss key

warrior

Tools

As a beginner, I found Asesprite to be very approachable and easy to jump into. It's available for all operating systems, which was a huge plus. When I was away from my computer and wanted to draw on my phone, I used the Android app Pixel Studio.

Cracking a Captcha with Tesseract.js

Wed, 12 May 2021 05:00:00 GMT

I wanted to automate some tasks on a site that I frequently use, but every now and then the site presents a captcha. I figured that with a little cleanup of the captcha images, I'd be able to use OCR (Optical Character Recognition) to read the captchas and provide the correct answer.

The Problem with Captchas and OCR

The captcha images on the site consist of a string of five characters with various colored lines drawn through. For example, look at these three captchas.

The letters aren't distorted and they're on a nice white background, so we shouldn't have any problem dropping them straight into an OCR program, right? Well, not quite. The problem is that these little lines are very effective at thwarting OCR engines because the majority of those programs rely on edge detection to identify each letter. Tesseract.js is one of the most popular OCR libraries for JavaScript. Let's visit their site and try our captcha on their demo.

Instead of reading FCWVw, Tesseract produced ECYA w-; this is definitely not what we're looking for. We're going to have to clean up this image so that it doesn't confuse Tesseract.

Cleaning up the Captchas

We need to get rid of these lines if we want Tesseract.js to stand a fair chance at reading our image. Thankfully, there's an interesting attribute of these captchas that will help us: all of the characters are a solid color, and each line is a different color. These colors change every time a new captcha is produced.

So, if we break our png down pixel by pixel and count how many of each color pixel appears, we can find:

Which color appears most often (the white background)
Which color appears second most often (our characters)
The rest of the colors (the lines)

Identifying the characters

Let's pull in a png library that will help us analyze the image and then break down our color count.

const fs = require('fs');
const PNG = require('pngjs').PNG;

// open image
const captcha = fs.readFileSync(path);
const pngImage = PNG.sync.read(captcha);
let {data, height, width} = pngImage;

// create a dictionary to keep track of our pixel counts
let colorOccurrences = {};

for (let y = 0; y < height; y++) {
  // rows
  for (let x = 0; x < width; x++) {
    // columns
    /**
     * Each pixel is a set of 4 values:
     * Red, Green, Blue, Alpha (transparency)
     */
    let index = (width * y + x) * 4;

    // create a string of the R-G-B color values
    let color = `${data[index]}-${data[index + 1]}-${data[index + 2]}`;
    // we can ignore white since it will always be the background
    if (color !== '255-255-255') {
      // increase the count by 1 (or set it to 1 if the color wasn't there yet)
      colorOccurrences[color] = (colorOccurrences[color] || 0) + 1;
    }
  }
}

We first open our image as a file and pass it in to the PNG library, which gives us a pngImage object. This object holds all the information about our image, but we're only concerned with 3 parts: the height, the width, and the data.

The data is a 1-dimensional array of all the color information that makes up our image. Each pixel is represented by 4 values from 0-255: Red, Green, Blue, and Alpha (the transparency). So for each round of the loop, we need to multiply our index by 4 to jump to the next pixel.

For example, let's look at a 2x2 (4 pixel) image.

The data array for this image would be:

/* R     G     B     A */
[
  252,  215,  64,   255, // pixel 1 (0,0)
  64,   252,  89,   255, // pixel 2 (0,1)
  0,    180,  239,  255, // pixel 3 (1,0)
  126,  64,   252,  255  // pixel 4 (1,1)
]

Now that we have our data, we can loop through each pixel and grab the RGB values (we don't need the alpha value). We'll store the RGB value as a string like R-G-B to use it as a key in our colorOccurrences object, then keep a count of how may pixels of each color occur. We'll just ignore the white pixels since it's the background color.

Finally, we can find our color that appeared most frequently. This will correspond to the color of our characters.

// grab all of the colors in the pattern [R-G-B, # of occurrences]
let colors = Object.entries(colorOccurrences);
// find the color that occurred most
let highestColor = colors.reduce((highColor, currentColor) => {
  if (highColor[1] > currentColor[1]) {
    return highColor;
  } else {
    return currentColor;
  }
});
// grab just the R-G-B as an array, we don't need the number of occurrences
let highestColorRGB = highestColor[0].split('-');

Now that we know what color our characters are, we can preserve those while removing the unnecessary noise from the picture.

Removing the lines

Since we know which colors we want to keep (the white background and whatever color our characters are), we can loop back through our image and repaint any pixel we see that doesn't match.

for (let y = 0; y < height; y++) {
  // rows
  for (let x = 0; x < width; x++) {
    // columns
    let index = (width * y + x) * 4;

    // grab the RGB values of the current pixel
    let RGB = [data[index], data[index + 1], data[index + 2]];

    // ignore white pixels so we don't alter the background
    if (RGB[0] === 255 && RGB[1] === 255 && RGB[2] === 255) continue;

    /**
     * We need to be a little forgiving when checking the colors.
     * Sometimes individual pixels are only 1-3 points of R, G, or B away,
     * especially on the edges of the characters.
     */
    // find how far each pixel color channel is from the color of the characters
    let [red, green, blue] = [
      Math.abs(highestColorRGB[0] - RGB[0]),
      Math.abs(highestColorRGB[1] - RGB[1]),
      Math.abs(highestColorRGB[2] - RGB[2]),
    ];

    // if any color channel is more than 3 points away
    if (red > 3 || green > 3 || blue > 3) {
      // paint the pixel...
    }
  }
}

We set up a loop again, row by row and column by column, and grab the RGB values of the current pixel. We'll ignore any white pixel because we want to leave the background alone. Then, we check to see if the current pixel's color matches the color of the characters.

We have to leave a little leeway for each color check; sometimes there's a discrepancy of 1-3 points on each color channel between adjacent pixels, especially around the edges of the characters. So the main color might be 10-30-59, but one of the pixels on the character might be 11-29-57. We'll let these close pixels slide by unscathed. If we see that any color channel is more than 3 points off, we'll paint the pixel to get rid of the line. But what color do we paint the pixel?

The first option that comes to mind is to just erase the lines by painting each pixel white like the background.

// if any color channel is more than 3 points away
if (red > 3 || green > 3 || blue > 3) {
  data[index] = 255;
  data[index + 1] = 255;
  data[index + 2] = 255;
}

This will produce the following image.

Not too bad! Let's see what happens when we pass this to Tesseract.

Uh oh. 😕 This time, Tesseract read VAN FCW\Vw. While the captcha looks a lot cleaner to us, we've actually created a lot of new edges which confuses Tesseract. Let's take a different approach.

Instead of painting the pixels white and leaving holes in our characters, we can try to fill in the gaps instead. The simplest thing we can do is just paint our current pixel the same color as the one above it.

// if any color channel is more than 3 points away
if (red > 3 || green > 3 || blue > 3) {
  // Grab the pixel that is one row up (y-1)
  let aboveIndex = (width * (y - 1) + x) * 4;

  // Paint our pixel to match the pixel above
  data[index] = data[aboveIndex];
  data[index + 1] = data[aboveIndex + 1];
  data[index + 2] = data[aboveIndex + 2];
}

With this new technique, we get:

We still get some odd drips and artifacts, but this looks a lot better! Let's see what Tesseract thinks about our new version.

A perfect success! By eliminating extra edges and solidifying our characters, we've cleaned up the captcha enough to give Tesseract a fighting chance.

Future Improvements

I had a little over 800 of the captchas saved, so I decided to run both the white-line method and the full-character method against all of them to see the difference in their effectiveness.

The full-character method had a 69% success rate, compared to a 26% success rate for the white-line method. Based on the data I gathered from these tests, it's clear that Tesseract.js makes some common misreads:

8 as g
9 as o
Q as O
J as ]
J as 1
q as g

As a future improvement, we could provide our own training data to Tesseract.js based on the specific font used in these captchas! That would alleviate the confusion that occurs between these similar characters.

Conclusion

By cleaning up the lines crossing through our captcha, we were able to use the Tesseract.js OCR library to decode the captchas and allow for a bit more automation. And who doesn't love some more automation in their life?

Here's the full code:

const fs = require('fs');
const PNG = require('pngjs').PNG;

// open image
const captcha = fs.readFileSync(path);
const pngImage = PNG.sync.read(captcha);
let {data, height, width} = pngImage;

// create a dictionary to keep track of our pixel counts
let colorOccurrences = {};

for (let y = 0; y < height; y++) {
  // rows
  for (let x = 0; x < width; x++) {
    // columns
    /**
     * Each pixel is a set of 4 values:
     * Red, Green, Blue, Alpha (transparency)
     */
    let index = (width * y + x) * 4;

    // create a string of the R-G-B color values
    let color = `${data[index]}-${data[index + 1]}-${data[index + 2]}`;
    // we can ignore white since it will always be the background
    if (color !== '255-255-255') {
      // increase the count by 1 (or set it to 1 if the color wasn't there yet)
      colorOccurrences[color] = (colorOccurrences[color] || 0) + 1;
    }
  }
}

// grab all of the colors in the pattern [R-G-B, # of occurrences]
let colors = Object.entries(colorOccurrences);
// find the color that occurred most
let highestColor = colors.reduce((highColor, currentColor) => {
  if (highColor[1] > currentColor[1]) {
    return highColor;
  } else {
    return currentColor;
  }
});
// grab just the R-G-B as an array, we don't need the number of occurrences
let highestColorRGB = highestColor[0].split('-');

for (let y = 0; y < height; y++) {
  // rows
  for (let x = 0; x < width; x++) {
    // columns
    let index = (width * y + x) * 4;

    // grab the RGB values of the current pixel
    let RGB = [data[index], data[index + 1], data[index + 2]];

    // ignore white pixels so we don't alter the background
    if (RGB[0] === 255 && RGB[1] === 255 && RGB[2] === 255) continue;

    /**
     * We need to be a little forgiving when checking the colors.
     * Sometimes individual pixels are only 1-3 points of R, G, or B away,
     * especially on the edges of the characters.
     */
    // find how far each pixel color channel is from the color of the characters
    let [red, green, blue] = [
      Math.abs(highestColorRGB[0] - RGB[0]),
      Math.abs(highestColorRGB[1] - RGB[1]),
      Math.abs(highestColorRGB[2] - RGB[2]),
    ];

    // if any color channel is more than 3 points away
    if (red > 3 || green > 3 || blue > 3) {
      // if any color channel is more than 3 points away
      if (red > 3 || green > 3 || blue > 3) {
        // Grab the pixel that is one row up (y-1)
        let aboveIndex = (width * (y - 1) + x) * 4;

        // Paint our pixel to match the pixel above
        data[index] = data[aboveIndex];
        data[index + 1] = data[aboveIndex + 1];
        data[index + 2] = data[aboveIndex + 2];
      }
    }
  }
}

// save new image
const imageBuffer = PNG.sync.write(pngImage);
fs.writeFileSync(`${path.replace('.png', '')}-clean.png`, imageBuffer);

A Study in Poisson - Naive Dart Throwing

Tue, 04 May 2021 05:00:00 GMT

What is the Poisson Disc Process?

In many programs, we need a way to generate randomly distributed data, also known as noise. One of the most well-known random data signals is called white noise. White noise generation is very simple and performant but often leads to close clusters of points. Upon observing the generated data, these clusters can be quite noticeable and distracting.

An alternative to white noise is blue noise, a type of random data that avoids the clustering phenomenon by adding a simple constraint: any new random point must maintain a minimum distance from all other points. This prevents clusters from forming while still maintaining a random distribution. The poisson disc process is a popular algorithm used to create blue noise.

Poisson Disc Algorithms

There are various algorithms that have been developed for creating poisson disc distributions. In this series, we'll be looking at the following:

Naive Dart Throwing
Scalloped Sector (Dunbar and Humphrey)
Heirarchical Dart Throwing (Cline, Egbert, and White)
Fast Poisson Disc Sampling (Bridson)
Modified Bridson (Roberts)

We'll also take a look at using Spectral Analysis to determine the quality of the blue noise we produce.

Naive Dart Throwing

The most basic method of generating a poisson disc distribution is known as dart throwing. A random coordinate is picked (as if you had just randomly thrown a dart at a dartboard), and then that point is compared with all other points on the graph to see if it maintains a certain minimum distance from them. If the new point crosses that distance threshold with any other point, it is disqualified and a new point is generated.

Replay Animation

There are a couple of problems with this "naive" approach.

The process will get slower and slower as more points are added
There are plenty of points that are checked that are nowhere near our point in question
We do not know when/if we have a maximal set of points (points that completely fill the graph)

Algorithm Inefficiency

The issue of slowness is a fundamental flaw of this algorithm. In Big O notation, this algorithm has a running time of O(n²). That means that as the number of points increases, the worst-case time to run the algorithm increases exponentially.

One of the major causes of this inefficiency is that the algorithm has to compare our new point to every single point that exists so far, even if they're nowhere nearby. Visually, we can clearly see which points are close enough to be of concern. The dart-throwing algorithm, though, only has a list of all points that exist. It would need some sort of secondary data structure to keep track of point locations relative to each other, rather than just the order they happened to be created.

Another issue is that new points are created entirely at random. If there were a way to keep track of any empty space available, we could limit our point creation to those areas.

Algorithm Convergence

With dart throwing, there isn't a clean-cut way to know if our data set is full and without gaps, but we can try to approximate. In order to approach a maximal set of data points with dart throwing, we can set an error threshold.

We let the algorithm try to create new random data points. Each time it fails, it will count the number of consecutive errors that have occurred. If we successfully place a new point, we'll reset our error count. But if the number of consecutive errors reaches the threshold, the algorithm will stop execution.

A low threshold will leave more gaps in our graph but will execute more quickly. The higher we raise our threshold, the closer we come to a maximal data set, but at the cost of execution time.

Feel free to edit the parameters for the following dart throwing implementation and see how the data generation changes. The error threshold bar at the bottom of the field will show how close the algorithm is to meeting the error limit and ending execution.

Replay

Error Threshold: 100

Minimum distance between points: 20

Point size: 2

Note: Minimum distance is the edge-to-edge gap between points. If you increase point size significantly, you may want to also increase the minimum distance for a more visually pleasing distribution.

Conclusion

The naive dart throwing algorithm for generating blue noise is a good first step for understanding blue noise generation, but due to its potential for long execution time, especially with large data sets, it would be a good idea to use one of the newer, more efficient algorithms that we'll be studying.

In the next part of this series we'll take a look at the scalloped sector algorithm, one of the earlier improvements made to this process back in 2006. I look forward to seeing you there!

The Difference Between apt and apt-get

Wed, 27 Jan 2021 05:00:00 GMT

Debian shipped with the apt command in 2014 to package the most common functionality of apt-get and apt-cache into one useful package while leaving the more obscure commands behind.

apt also provides sensible defaults, like showing the download progress bar and showing the number of packages that can be updated.

See this FOSS article for more information.

Creating a CircleCI Orb

Mon, 05 Oct 2020 05:00:00 GMT

A guide for creating a CircleCI orb with the newly released CircleCI Orb Development Kit.

Introduction to Orbs

CircleCI is a continuous integration tool that allows you to run packages of code called orbs against your code every time a commit is made to your repository, or every time a pull request is created or merged. This allows you to perform automated testing or other tasks to help maintain the quality of your projects.

CircleCI has just released a new tool, the Orb Development Kit, to simplify and streamline the process of creating new orbs. To demonstrate how to use the kit, we'll be creating a simple orb for cowsay, a CLI tool that has a cow say a message in your console.

Terminal window

❯ cowsay CircleCI is great!
 ____________________
< CircleCI is great! >
 --------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Though this particular cowsay orb is silly and a bit simple, creating a new orb that positively impacts developers and helps solve a challenge is a great way that you can contribute to Hacktoberfest! See the CircleCI Hacktoberfest page for more details. Use this article as a guide and go out there and create something truly awesome.

Setup for Making an Orb

In order to use the new development kit, we need to set up the CircleCI CLI. This command line tool provides a lot of handy tools that we can use for interacting with CircleCI and our pipelines.

In order to be able to create an orb, you must first enable it in your Organization Settings. Navigate to the CircleCI Dashboard and on the menu on the left hand side of the screen, click Organization Settings.

Click on the Security tab and choose yes under Allow Uncertified Orbs. This will give you permission to publish orbs.

We will also need to create a GitHub repository for our new orb. You can head to the new repository page by clicking this fancy link: github.new. Go ahead and leave the repository empty (don't add a README, license, or .gitignore file). Once you create it, take note of the git URL as the CLI will ask for it soon!

Scaffolding Out the Orb

With the new orb development kit, much of the initial setup is taken care of for us as we only need to answer a few questions. Make sure that you are on a recent version of the CLI (at least v0.1.10211) to be able to use this new command. Run circleci version to see your current version, and circleci update to pull down a newer version if necessary.

To get started with the toolkit, run the command circleci orb init <path to new orb>. The path can be a new folder, and the tool will create it. For ours, let's create an orb named cowsay-orb, so our command will be circleci orb init cowsay-orb.

The toolkit will walk us through the steps to initialize and build out the orb. Follow the instructions as presented in the CLI; you will usually except the default options. See here a screenshot of my log:

Note that once you provide the GitHub link to the tool, it will create an initial commit for you locally and display the message An initial commit has been created - please run 'git push origin master' to publish your first commit!. Open a new terminal (don't cancel the toolkit process that's still running!), navigate to your new orb directory, and push the commit to GitHub.

Terminal window

cd cowsay-orb
git push origin master

Once you push the commit, return the toolkit and hit enter on Done. The toolkit will wrap up it's work and give you a URL to view your new orb's build pipeline! (Note: at the time of writing, the provided URL is incorrect. See the issue) You can view your pipeline at https://circleci.com/gh/<namespace>/<orb-name>.

The pipeline is set up to require your manual action to actually publish versions of your orb. Head to the pipeline for your orb, and you'll see a purple message that says On Hold.

Click on the hold-for-dev-publish step that is next to the purple pause icon and click Approve to give the pipeline permission to publish the first test copy of your orb.

After a short time, the pipeline will finish running two jobs and your first development copy of the orb is now published!

Now, on to our actual development.

Orb Layout

Open up your new orb directory in your code editor and ensure that you are not on the main branch. The toolkit should place you on the branch alpha automatically. You will always want to do your development on a separate branch like alpha, then merge that branch into your main branch later.

Let's take a look at the folder structure that was created for us.

.
├── .circleci
│   ├── config.yml
│   └── README.md
├── .git
│   └── ...
├── src
│   ├── commands
│   │   ├── greet.yml
│   │   └── README.md
│   ├── examples
│   │   ├── example.yml
│   │   └── README.md
│   ├── executors
│   │   ├── default.yml
│   │   └── README.md
│   ├── jobs
│   │   ├── hello.yml
│   │   └── README.md
│   ├── scripts
│   │   ├── greet.sh
│   │   └── README.md
│   ├── tests
│   │   ├── greet.bats
│   │   └── README.md
│   ├── @orb.yml
│   └── README.md
├── .gitignore
├── CHANGELOG.md
├── LICENSE
└── README.md

The .circleci directory contains administrative resources for the orb itself. We'll look at it in a bit more detail later on.

The .git directory provides some templates for GitHub issues and pull requests which you can customize if you like, but we won't discuss it in this article.

The src directory contains all of the different subdirectories that we'll be working in for this project. An orb can be made up of commands, examples, jobs, scripts, executors, and tests.

Commands are small, reusable pieces of functionality.
Examples are sample scripts that you provide to the user showing how to use your orb.
Jobs are longer processes made up of steps that can combine multiple commands to complete a task.
Scripts are pieces of code in other languages that can be used by your commands to accomplish more than what you could easily express in yaml, the language that the orbs are written in.
Executors are custom definitions of what type of "machine" or image you want your orb to be run on.
Tests and test resources can be placed in the test folder and executed by your integration tests.

For this simple orb that we are creating, we will not be defining jobs, scripts, executors, or separate tests, so those folders can safely be deleted. If you want to configure any of those yourself, look at the README provided in each folder, the examples provided, or the pages I have linked in the list above.

Open up the @orb.yml file located in the src folder. This is the main starting point for your orb. Whenever you publish the orb, the toolkit automatically takes each of the .yml files in your project and packs them together into one file. In this main @orb.yml file, you can define details about your orb and import any other orbs you might need.

Let's look at our fields here:

version - the configuration file version that we are using. This will be 2.1, the latest.
description - should be a short and sweet snippet about what our orb will do.
home_url - a reference to the home page of the tool we're building an orb for.
display_url - a reference to our GitHub repository for this orb.
orbs - any other orbs that we'll pull in to our project.

Our @orb.yml will look like this:

version: 2.1

description: >
  A simple CircleCI Orb to bring some beef to your logs.

display:
  home_url: 'https://en.wikipedia.org/wiki/Cowsay'
  source_url: 'https://github.com/kbravh/cowsay-orb'

orbs:
  node: circleci/node@4.0.1

We're now ready to develop our orb functionality!

Create Our Orb's Command

Open up the commands directory. We can open the greet.yml to learn what a command should look like.

⚠️ Note: From here on out, we'll be writing YAML, which is indent dependent and is very picky (especially with the YAML linter that is built into the toolkit). If you run into errors, please check the following:

Ensure your indentations are correct.
There shouldn't be any whitespace at the end of lines. The Trailing Spaces VS Code extension is very helpful here.
There must be a blank line at the end of every YAML file.

description: >
  This command echos "Hello World" using file inclusion.
# What will this command do?
# Descriptions should be short, simple, and clear.
parameters:
  to:
    type: string
    default: 'World'
    description: 'Hello to whom?'
steps:
  - run:
      environment:
        PARAM_TO: <<parameters.to>>
      name: Hello Greeting
      command: <<include(scripts/greet.sh)>>

The description should succinctly describe what this command will do.
The parameters section defines arguments that we can pass to our orb to let the user customize what it will do.
The steps are the individual commands that will be run. Notice that any parameters that we define above can be referenced using double angle brackets << parameters.<name> >> and the parameter's name using dot notation.

Let's change the name of this file to speak.yml. The names of the files in the command folder will become the actual commands available for our orb. We can now build out our own functionality. We'll inform the user that this speak command will have a cow say a command in the logs.

description: >
  This command will have a cow say a message in your logs.

Next, let's provide a parameter for the message the cow will say. The name of the parameter is the first line written under parameters. We can define a type (string, boolean, enum) and also define a default value in case the user does not pass one. If defaults are not provided, the job will fail for any missing parameters.

parameters:
  message:
    type: string
  default: "Mooo!"
  description: "What will the cow say?"

Finally, let's the define the steps of this command itself. The run operator will let us run a command as if we were typing in the terminal. We can provide a name for the command, and then pass the command itself. We'll need to run the command npx cowsay to fetch the cowsay package, and then we pass in the message from our parameters. By passing the -q flag to npx, we can have it be quiet and not print any of its own output, like download times.

steps:
  - run:
      name: Let the cow speak
      command: npx -q cowsay << parameters.message >>

That's all there is to defining a small command! So our speak.yml file will come together to look like this:

description: >
  This command will have a cow say a message in your logs.
parameters:
  message:
    type: string
    default: 'Mooo!'
    description: 'What will the cow say?'
steps:
  - run:
      name: Let the cow speak
      command: npx -q cowsay << parameters.message >>

Define an Integration test

Now that we have our command written, let's create an integration test. This will serve to make sure our orb command runs correctly, and we can also use this as the basis for our example.

Navigate to the .circleci folder and open config.yml. The toolkit uses this configuration to publish development and production versions of our orb. We can clean up this file a bit to accommodate our orb. Under the orbs heading, let's add the node orb.

orbs:
  node: circleci/node@4.0.1

As we haven't included any Bash scripts, we can remove the bats and shellcheck orbs. This means we should also head down to the workflows section and remove the shellcheck/check and the bats/run commands. Since we no longer have these jobs, we should also remove them from the requires list under the hold-for-dev-publish job.

Now, let's define our integration test. Under the jobs heading (which should be around line 22), remove everything beneath the integration-test-1 command. We'll add our own configuration here. We need to tell the job what environment to execute the job in, so set the executor to node/default, an executor defined in the node orb that comes with Node.js installed. We need Node.js to run the npx cowsay command. Then, define the steps we want the job to run. For us, this will just be our cowsay/speak command. Don't forget to pass in a message!

jobs:
  # Define one or more jobs which will utilize your orb's commands and parameters to validate your changes.
  integration-test-1:
    executor: node/default
    steps:
      - cowsay/speak:
          message: 'Howdy!'

This integration test is very basic and just ensures that our command can be run without throwing any errors or crashing the pipeline. See the orb Testing Methodologies page for more information about testing.

⚠️ Note: Since the only published version of our orb that exists right now doesn't have our speak command that we created, this integration test will fail! To get around that for now, let's comment out the call to run integration-test-1 under the integration-test_deploy job, and comment out the two requires lines under orb-tools/dev-promote-prod-from-commit-subject. We can re-enable it once we successfully run the pipeline. Our code sections will now appear like this:

integration-test_deploy:
    when: << pipeline.parameters.run-integration-tests >>
    jobs:
      # Run any integration tests defined within the `jobs` key.
      # - integration-test-1

...

- orb-tools/dev-promote-prod-from-commit-subject:
    orb-name: kbravh/cowsay-orb
    context: orb-publishing
    add-pr-comment: false
    fail-if-semver-not-indicated: true
    publish-version-tag: false
    # requires:
    #   - integration-test-1
    filters:
    branches:
      only: master

Provide an Example

We can finally wrap up one of the last steps: writing an example for our users. This will resemble our integration test.

Open up the examples folder; you can delete the example.yml. Create a new file congratulate.yml, and let's start building out our example.

We can include a description just like in other files, so let's describe what will happen in this test.

description: >
  Congratulate the user on the pipeline passing.

Next comes the usage tag. This is where we'll actually build out everything that will appear as the example for the user. We'll include the same version number 2.1, and we'll list the orbs that we'll use. The development version of our orb is automatically tagged as dev:alpha by the toolkit, which is why we include it like that here. Once you have a published version of your orb, you can update this example with the actual version.

usage:
  version: 2.1
  orbs:
    cowsay: kbravh/cowsay@dev:alpha
    node: circleci/node@4.0.1

Finally, we'll build out the job we would need to run. We can call it congratulate. We define our executor just as we did in the integration test, and call our cowsay/speak command with a custom message.

jobs:
  congratulate:
    executor: node/default
    steps:
      - cowsay/speak:
          message: 'Your pipeline looks great!'

We're finally ready to try out our orb!

Testing and Publishing the Orb

Before we push our orb to the pipeline for testing, we can do a quick local validation using the circleci orb validate tool in the CLI. This command will verify that our orb is formatted correctly. However, since our orb is currently split into many different files, we'll need to pack it together before validating. Both of these steps can be done using these two commands in your terminal.

Terminal window

circleci orb pack src/ > orb.yml
circleci orb validate orb.yml

The pack command squashes our various YAML files down into one consolidated orb file. Normally this command would just print the result out to the terminal, so we instead direct it into a file orb.yml. Then, our validate command checks our formatting.

Having any issues with YAML formatting? Try dropping your YAML files into this YAML linter to catch syntax errors.

If you see the message Orb at 'orb.yml' is valid. then you are ready to publish!

Publishing a development version is rather straightforward: you just need to push your changes to GitHub on your alpha branch!

Once you push your changes, head over to the pipeline at https://circleci.com/gh/<namespace>/<orb-name>. You'll have to click the On Hold button again to pass the job through. Once the pipeline finishes, you'll have a new dev version with your commands!

You can now re-enable your integration tests that we disabled above and push the code to GitHub once more. This will now run your integration test and publish another dev version. You can open the pipeline details and see the result of your integration test once it passes.

Once the pipeline finishes, you are finally ready to publish a live, production version of your orb!

Publish a Production Orb

To start the process of upgrading your dev orb to a production version, you'll need to head over to GitHub and create a pull request from your alpha branch to your default branch. You have to include a special [semver:<increment>] tag in the title of your pull request. This is how the toolkit will know what version to assign to your orb when it is published. Check the following table to know what increment to include.

Increment	Description
major	Issue a 1.0.0 incremented release
minor	Issue a x.1.0 incremented release
patch	Issue a x.x.1 incremented release
skip	Do not issue a release

If we wanted our first version of our cowsay orb to be 0.0.1, we'd include [semver:patch] in the title. However, let's put [semver:major] so that our first version is 1.0.0.

After clicking Create pull request, you'll see a window below with a status. This window will show the status of the alpha pipeline, which should indicate that all the checks have passed. If everything is ready, you'll have the option to Merge pull request. It is recommended to click the small arrow next to the Merge button and choose to Squash and merge, as it will preserve your pull request title.

You'll be asked to confirm the merge. Make sure your semver tag is present still in the title! By merging in this pull request to your default branch, a final pipeline run will be started. Follow our same On Hold ritual from before and allow the pipeline to finish running.

Once the pipeline finishes, your orb should now be live! Visit it at https://circleci.com/developer/orbs/orb/<namespace>/<orb-name>. Note: If you try to search for your orb at the Orb Registry and it can't be found, try waiting a little while before searching again. It seems to take a bit for new orbs to be indexed by the search engine.

Congratulations on making an orb! Hopefully this guide has been helpful. Go out there and do something awesome!

Extra information

Want to make an orb private? Orbs are public by design, but you can unlist an orb so that it won't show up in the Registry. Learn how to Unlist an orb.

Avoid Losing Errors in a Bash Pipe

Sun, 04 Oct 2020 05:00:00 GMT

Have you ever wanted to string together some bash commands using pipes, but weren't sure how to not lose the error codes in the process?

tl;dr Add set -o pipefail to the beginning of your script!

For the CircleCI Rome Orb that I created, I was seeing very odd output to the pipeline logs for the commands. This was due to the way that the Rome linter streams its output. Each message was appearing one letter per line instead of showing the full message.

In order to alleviate this, I wanted to pipe my linting command through to cat to be displayed all at once instead of being streamed in character by character.

npx rome check | cat

Are you curious as to why we can't use echo here? The echo command only takes input from the arguments passed to it directly, not from stdin. If we used echo here, the console would only output a blank line!

However, just doing this piping would cause a problem in our script execution; if linting failed, the error code would just disappear! The problem stems from the fact that Bash does not always act like higher level programming languages. If a command fails, Bash will continue execution like normal. For a linting command, we want to get back our exit code so we know if linting failed without losing it in the pipe.

Luckily, there's a handy option for this.

set -o pipefail

By running set -o pipefail in our script before our pipe command, bash will finish executing the entire pipe and throw the error code of the right-most error that occurs, if any. So if npx rome check fails, that error code will still be thrown even though cat succeeds.

If you want to get some more in depth information and see some other good options to set for your bash scripts, check out vaneyckt's great article.

Switch Audio Outputs with a Hotkey

Fri, 11 Sep 2020 05:00:00 GMT

This script will cycle to the next available audio output device. It can be tied to a hotkey to easily be triggered. This is handy, for example, for swapping between speakers and headphones.

This script will work on systems running PulseAudio or Pipewire services.

Tools/commands used in this script

declare - declares a new variable, allows for setting 'types'
pacmd - provides information about PulseAudio audio devices
pactl - provides information about PipeWire audio devices
notify-send - sends pop-up notifications to user (may need to be installed with sudo apt install libnotify-bin)
- Icons -i can be found in the following locations:
  - /usr/share/icons/gnome/32x32
  - /usr/share/notify-osd/icons/
grep - the ever useful search command
- Here, used with Character Classes
sed - the inline editing tool
mapfile - the recommended way to populate arrays from grep results
Run through ShellCheck to verify script.

Complete Script

#!/bin/bash

# Check which sound server is running
if pgrep pulseaudio >/dev/null; then
  sound_server="pulseaudio"
elif pgrep pipewire >/dev/null; then
  sound_server="pipewire"
else
  echo "Neither PulseAudio nor PipeWire is running."
  exit 1
fi

# Grab a count of how many audio sinks we have
if [[ "$sound_server" == "pulseaudio" ]]; then
  sink_count=$(pacmd list-sinks | grep -c "index:[[:space:]][[:digit:]]")
  # Create an array of the actual sink IDs
  sinks=()
  mapfile -t sinks < <(pacmd list-sinks | grep 'index:[[:space:]][[:digit:]]' | sed -n -e 's/.*index:[[:space:]]\([[:digit:]]\)/\1/p')
  # Get the ID of the active sink
  active_sink=$(pacmd list-sinks | sed -n -e 's/[[:space:]]*\*[[:space:]]index:[[:space:]]\([[:digit:]]\)/\1/p')

elif [[ "$sound_server" == "pipewire" ]]; then
  sink_count=$(pactl list sinks | grep -c "Sink #[[:digit:]]")
  # Create an array of the actual sink IDs
  sinks=()
  mapfile -t sinks < <(pactl list sinks | grep 'Sink #[[:digit:]]' | sed -n -e 's/.*Sink #\([[:digit:]]\)/\1/p')
  # Get the ID of the active sink
  active_sink_name=$(pactl info | grep 'Default Sink:' | sed -n -e 's/.*Default Sink:[[:space:]]\+\(.*\)/\1/p')
  active_sink=$(pactl list sinks | grep -B 2 "$active_sink_name" | sed -n -e 's/Sink #\([[:digit:]]\)/\1/p' | head -n 1)
fi

# Get the ID of the last sink in the array
final_sink=${sinks[$((sink_count - 1))]}

# Find the index of the active sink
for index in "${!sinks[@]}"; do
  if [[ "${sinks[$index]}" == "$active_sink" ]]; then
    active_sink_index=$index
  fi
done

# Default to the first sink in the list
next_sink=${sinks[0]}
next_sink_index=0

# If we're not at the end of the list, move up the list
if [[ $active_sink -ne $final_sink ]]; then
  next_sink_index=$((active_sink_index + 1))
  next_sink=${sinks[$next_sink_index]}
fi

#change the default sink
if [[ "$sound_server" == "pulseaudio" ]]; then
  pacmd "set-default-sink ${next_sink}"
elif [[ "$sound_server" == "pipewire" ]]; then
  # Get the name of the next sink
  next_sink_name=$(pactl list sinks | grep -C 2 "Sink #$next_sink" | sed -n -e 's/.*Name:[[:space:]]\+\(.*\)/\1/p' | head -n 1)
  pactl set-default-sink "$next_sink_name"
fi

#move all inputs to the new sink
if [[ "$sound_server" == "pulseaudio" ]]; then
  for app in $(pacmd list-sink-inputs | sed -n -e 's/index:[[:space:]]\([[:digit:]]\)/\1/p'); do
    pacmd "move-sink-input $app $next_sink"
  done
elif [[ "$sound_server" == "pipewire" ]]; then
  for app in $(pactl list sink-inputs | sed -n -e 's/.*Sink Input #\([[:digit:]]\)/\1/p'); do
    pactl "move-sink-input $app $next_sink"
  done
fi

# Create a list of the sink descriptions
sink_descriptions=()
if [[ "$sound_server" == "pulseaudio" ]]; then
  mapfile -t sink_descriptions < <(pacmd list-sinks | sed -n -e 's/.*alsa.name[[:space:]]=[[:space:]]"\(.*\)"/\1/p')
elif [[ "$sound_server" == "pipewire" ]]; then
  mapfile -t sink_descriptions < <(pactl list sinks | sed -n -e 's/.*Description:[[:space:]]\+\(.*\)/\1/p')
fi

# Find the index that matches our new active sink
for sink_index in "${!sink_descriptions[@]}"; do
  if [[ "$sink_index" == "$next_sink_index" ]]; then
    notify-send -i audio-volume-high "Sound output switched to ${sink_descriptions[$sink_index]}"
    exit
  fi
done

Install

Create the file audio-device-switch.sh, paste in the script, and place it in /usr/local/bin.
Make the script executable: sudo chmod 755 /usr/local/bin/audio-device-switch.sh.
Open the Keyboard Shortcuts settings page, add a new shortcut, tell it to execute audio-device-switch.sh, and set up your shortcut!
Install the notify-send library if you want to see a popup notification when the audio device switches: sudo apt install libnotify-bin.

Customizations

Feel free to modify this script and make it your own. Some ideas for customization:

Different icon in the notification

Line 83 of the script calls notify-send with the -i flag which defines which icon is displayed. Stock icons are found in:

/usr/share/icons/gnome/32x32
/usr/share/notify-osd/icons/

Acknowledgements

This is a more modern, robust rewrite of tsvetan's solution on the Ubuntu forums.

Versions and Aliases in AWS Lambda

Tue, 09 Jun 2020 05:00:00 GMT

Have you ever wanted to make a change to a Lambda function that's running in your live environment, but you've been worried that you'd make a mistake and have some down time? Or worse, have you made the change and brought down your site? 😬 AWS Lambda provides a way to create versions of a function in order to test functionality in lower environments before moving it into higher environments, all without the risk of affecting your deployed code.

Creating a version

First, create a new function or choose a function that you've already created. At the top of the page under the Actions dropdown, choose to publish a new version of your function.

You can add a description here to your version. This is essentially saving a snapshot of your function as it is now (like a git commit).

You'll notice that the window says it is creating a new version from js~$LATEST. This is the tag that AWS gives to your current working state. Any changes you make to your function are saved in the js~$LATEST version even if you don't publish an explicit version yourself.

Once we publish our new version, we'll be taken to this new copy of our lambda function. We can tell what version we're on by the number appended to the function name. We can return to our js~$LATEST function version by using the Version dropdown or the breadcrumb trail at the top left.

We can't make changes to or publish a new version of this snapshot, so let's return to our js~$LATEST version and make a change to our code! We'll publish it as version 2.

There we go! We now have multiple versions of our functions that we can use for testing in different environments. These different versions can be referenced by using the ARN or function name with the version number appended like so: js~AliasAndVersioning:2.

But what if we publish a third version and want a CloudWatch rule or an S3 trigger to point at our new version? With version numbers, we'd have to go and update the resource information with the new number each time. That's where aliases come in to save the day.

The Power of Aliases

Aliases are essentially nicknames or tags that we can point at certain versions of our function. When we create a new version of our function, we can simply move our alias to point there instead. Let's see it in action.

Creating a Production alias

Under the Actions dropdown, choose to create an alias. Here we can set the name, a description, and which version we'll point the alias at. Let's set js~PROD (Production) at version 1 that we created.

The weighted alias section at the bottom allows you to divert a percentage of your traffic to different versions of your function, like for A/B testing.

When we save this alias, we'll see a screen very similar to what we saw when we created a new version, but the alias name is now appended to the function name instead of the version number!

Creating a Development alias

Let's now create a DEV (development) alias. Instead of pointing this at a version number, we can point it directly at js~$LATEST to always reflect our latest changes.

Now we have our aliases for our different development environments! So when I reference this function while setting up any other resources, I can reference the alias of the function like js~AliasAndVersioning:DEV.

Update alias reference

Let's say that we've finished testing version 2 of our function in the development environment and we're ready to use it in Production. If we have our references set up to point to the alias, it's as easy as just adjusting the alias definition.

Let's choose our js~PROD alias from the Alias dropdown so that we can edit its settings. In the Alias configuration box at the bottom, click the Edit button.

From the version dropdown on the page, we can now choose version 2 and save our changes.

Production will now be using our newer version! 🎉

Final Thoughts

Congratulations! You'll now be a pro working with versions and aliases in AWS Lambda. Your work mates will be in awe of your awesome devops skills, and your clients will be well pleased knowing their sites will be more resilient and much less likely to have down time.

Adjust AWS Cron Events for Daylight Savings Time

Tue, 21 Apr 2020 05:00:00 GMT

In order to lower costs for unused resources in AWS (which add up way faster than you think), we spin-down servers in our non-production environments every night during non-working hours and bring them back up in the morning. This entire process is handled by a single lambda function that is kicked off by some cron triggers in CloudWatch Event rules.

We have the environments set to sleep every weekday between 7pm and 7am, so they'll be back up in time for the first developers who get in at 7:30am. They also sleep all weekend since we don't do any development on Saturday or Sunday. Things were working flawlessly! That is, until Daylight Savings Time hit.

The Problem

For those who may not be familiar with Daylight Savings Time (DST), it's the practice of setting the clocks forward one hour in spring and back one hour in fall to better take advantage of the daylight hours. Unfortunately, since the cron triggers in AWS are set up on UTC time, the DST time change caused our spin-up function to run an hour late.

/* Original CRON triggers when not in DST */
0 13 ? * MON-FRI * // spin-up at 13:00 UTC (7am CST)
0 1 ? * TUE-SAT * // spin-down at 1:00 UTC (7pm CST)

Curious about how cron events work? Cron expressions in AWS are made up of 6 slots. From left to right:

Minutes (0-59)
Hours (0-23)
Day of the Month (0-31)
Month (1-12 or JAN-DEC)
Day of the Week (1-7 or SUN-SAT)
Year (1970-2199)

An asterisk can stand in as a wildcard in any slot.

I adjusted our cron events to account for the time change as follows:

/* New CRON triggers during DST */
0 12 ? * MON-FRI * // spin-up at 12:00 GMT (7am CST)
0 0 ? * TUE-SAT * // spin-down at 0:00 GMT (7pm CST)

This would work, but it was a short term fix. I wanted a longer term solution so we wouldn't run into this problem again in November when the time changed back.

The Plan

It occurred to me that I could have the very same spin-up/spin-down function adjust its own cron triggers based on whether or not we were in Daylight Savings Time. To do this, I would just need to add a little bit of time logic.

First, I needed to get the current time local time and determine whether or not it was Daylight Savings Time. Thankfully, Python has some very useful date functionality that would make this easy to do.

from datetime import datetime
import pytz

utc_time = datetime.utcnow().replace(tzinfo=pytz.utc)
cst_timezone = pytz.timezone('US/Central')
cst_time = utc_time.astimezone(cst_timezone)

We import the py~datetime library, the main Python library for working with any date or time objects, and py~pytz, a library for accurately handling timezones. We create py~utc_time and set it equal to the current time in UTC with py~datetime.utcnow(). Note that we have to manually replace the timezone information on the native py~datetime object with the py~utc timezone from py~pytz.

Next, we create a central time timezone object py~cst_timezone and pull that information from the py~pytz library. Finally, we convert our UTC time to CST time using the function py~astimezone(), which takes in a timezone object as a parameter.

So now that we have our current time in the CST timezone, we can determine if it's Daylight Savings Time.

# This will show the offset in seconds for DST ⏰
# If it's zero, we're not in DST
is_dst = cst_time.tzinfo._dst.seconds != 0

If we dig down into our current time object py~cst_time, we can get more information about the timezone. By accessing our timezone information (py~tzinfo), then our DST information (py~_dst), we can find how many py~seconds of an offset is currently on our time. If we are in DST, the offset will not be 0 (it will be 3600 seconds, really, since time gets shifted an hour).

The Triggers

Now that we know if it's Daylight Savings Time or not, we can adjust our cron triggers. First, let's define what our expressions will be based on if we're in DST. During DST, we'll want the earlier triggers.

if is_dst:
  start_schedule = 'cron(0 12 ? * * *)'
  stop_schedule = 'cron(0 0 ? * * *)'
else:
  start_schedule = 'cron(0 13 ? * * *)'
  stop_schedule = 'cron(0 1 ? * * *)'
# Curious why these cron expressions don't have the weekdays anymore? Read on!

Next, we need to actually update the rules in CloudWatch.

  import boto3
  client = boto3.client('events')

  START_RULE = "start_rule"
  STOP_RULE = "stop_rule"

  start_settings = client.list_targets_by_rule(Rule=START_RULE)
  stop_settings = client.list_targets_by_rule(Rule=STOP_RULE)

We create our py~boto3 client to be able to access our AWS resources. Assuming our rules are named py~"start_rule" and py~"stop_rule", we can then fetch information about these rules from CloudWatch. Here, we are using the function py~list_targets_by_rule() to get back the targets these rules have (that would be this function).

# Remove targets from current rules
client.remove_targets(
  Rule=START_RULE,
  Ids=[(start_settings['Targets'][0]['Id'])]
)
client.remove_targets(
  Rule=STOP_RULE,
  Ids=[(stop_settings['Targets'][0]['Id'])]
)
# Delete rules
client.delete_rule(Name=START_RULE)
client.delete_rule(Name=STOP_RULE)

Technically here we aren't updating the cron triggers, but rather replacing them. In order to delete a cron event trigger in AWS, you must first remove its targets by passing in an array of IDs. We'll give it the information we got from our previous call. We only have one target, so we pull the first (and only) item from our settings that we queried like py~start_settings['Targets'][0]['Id'] . Once we delete our targets, we can delete our rules.

# Add new rules
client.put_rule(
  Name=START_RULE,
  ScheduleExpression=start_schedule,
  State='ENABLED',
  Description="Automatic trigger for the Spinup-Spindown function."
)
client.put_rule(
  Name=STOP_RULE,
  ScheduleExpression=stop_schedule,
  State='ENABLED',
  Description="Automatic trigger for the Spinup-Spindown function"
)

Now, we can add our new cron rules in. We'll set the same name as before and set our new cron expressions that we defined as py~start_schedule and py~stop_schedule. We enable the rule, and add our description.

# Add targets
client.put_targets(
  Rule=START_RULE,
  Targets=[
    {
      'Id': start_settings['Targets'][0]['Id'],
      'Arn':start_settings['Targets'][0]['Arn'],
      'Input': start_settings['Targets'][0]['Input']
    }
  ]
)
client.put_targets(
  Rule=STOP_RULE,
  Targets=[
    {
      'Id': stop_settings['Targets'][0]['Id'],
      'Arn':stop_settings['Targets'][0]['Arn'],
      'Input': stop_settings['Targets'][0]['Input']
    }
  ]
)

Finally, we add our targets back in. We can do this by pulling all the information out of our settings objects. Our updated cron rules are in place!

Final Adjustments

So now our function can update its own cron triggers, awesome! But wait; what happens on the days that the time change actually happens? Won't our function run an hour late or an hour early on that one particular day? It will indeed, but we come out lucky here.

If you remember from the cron expressions we had at the beginning, we only ran this function Monday through Friday because we don't want our environments up on the weekend. In the United States, DST begins the 2nd Sunday in March and ends on the 1st Sunday in November. So our 1 hour shift would happen on a weekend, and it would never affect us!

Like I said though, our orignal cron expressions don't run on weekends, so we'll need to change our cron expressions slightly so they will. If you notice from the code snippet earlier, we already did this:

if is_dst:
  start_schedule = 'cron(0 12 ? * * *)'
  stop_schedule = 'cron(0 0 ? * * *)'
else:
  start_schedule = 'cron(0 13 ? * * *)'
  stop_schedule = 'cron(0 1 ? * * *)'

These cron expressions will now run every day of the week, instead of just Monday through Friday.

Now that our function runs every day, we'll need to stop it from doing the spin-up and spin-down on weekends with a bit more time logic.

weekday = cst_time.weekday()
#in the python calendar, 5 and 6 correspond to Saturday and Sunday
if weekday == 5 or weekday == 6:
  return {
      'statusCode': 200,
      'message': 'Today is a not a weekday, so the spin-up/spin-down process will not proceed.'
  }
else:
  begin_startstop_process()

We take our py~cst_time object and pull the day of the week out using py~weekday(). We then check whether it's a Saturday or a Sunday, and if so, we return a statusCode and message and stop execution. Otherwise we can get things going!

Final Thoughts

So there we go, we can now automatically adjust our cron triggers in AWS so that Daylight Savings Time won't affect us again! However, there are definitely some future improvements we could make. For example, you might ask the following questions:

Why do we adjust the rules every single day even when the rules will only change twice a year?

What if we have a function that needs to run on weekends that we don't want to be late twice a year?

This could be resolved pretty simply by extracting all of this functionality into a separate function. Using cron expressions, we could also set this function to only run on Sundays in March and November when the time changes would happen! Let's wrap this all up (or checkout the gist!):

/* CRON triggers for DST function */
0 12 8-14 MAR SUN * // 13:00 UTC (6am CST) for the second Sunday in March
0 0 1-7 NOV SUN * // 1:00 UTC (6pm CST) for the first Sunday in November

from datetime import datetime
import pytz
import boto3

# define our CloudEvents client
client = boto3.client('events')

# Let's grab our current time in GMT and convert it to local time (US Central)
utc_time = datetime.utcnow().replace(tzinfo=pytz.utc)
cst_timezone = pytz.timezone('US/Central')
cst_time = utc_time.astimezone(cst_timezone)

# This will show the offset in seconds for DST ⏰
# If it's zero, we're not in DST
is_dst = cst_time.tzinfo._dst.seconds != 0

# Determine our new cron expressions for a function running everyday at 7am and 7pm CST
if is_dst:
  start_schedule = 'cron(0 12 ? * * *)'
  stop_schedule = 'cron(0 0 ? * * *)'
else:
  start_schedule = 'cron(0 13 ? * * *)'
  stop_schedule = 'cron(0 1 ? * * *)'

START_RULE = "start_rule"
STOP_RULE = "stop_rule"

# Fetch the targets of our current cron rules
start_settings = client.list_targets_by_rule(Rule=START_RULE)
stop_settings = client.list_targets_by_rule(Rule=STOP_RULE)

# Remove targets from current rules
client.remove_targets(
  Rule=START_RULE,
  Ids=[(start_settings['Targets'][0]['Id'])]
)
client.remove_targets(
  Rule=STOP_RULE,
  Ids=[(stop_settings['Targets'][0]['Id'])]
)

# Delete rules
client.delete_rule(Name=START_RULE)
client.delete_rule(Name=STOP_RULE)

# Add new rules
client.put_rule(
  Name=START_RULE,
  ScheduleExpression=start_schedule,
  State='ENABLED',
  Description="Automatic trigger for the function."
)
client.put_rule(
  Name=STOP_RULE,
  ScheduleExpression=stop_schedule,
  State='ENABLED',
  Description="Automatic trigger for the function"
)

# Add back our targets
client.put_targets(
  Rule=START_RULE,
  Targets=[
    {
      'Id': start_settings['Targets'][0]['Id'],
      'Arn':start_settings['Targets'][0]['Arn'],
      'Input': start_settings['Targets'][0]['Input']
    }
  ]
)
client.put_targets(
  Rule=STOP_RULE,
  Targets=[
    {
      'Id': stop_settings['Targets'][0]['Id'],
      'Arn':stop_settings['Targets'][0]['Arn'],
      'Input': stop_settings['Targets'][0]['Input']
    }
  ]
)

Thanks for sticking around and reading. Hopefully this will help you out with your work in AWS!

Want to dig a bit deeper and learn a bit more? Here are some useful links:

Daylight Savings Time in the US | Wikipedia

Schedule Expressions for Rules | Amazon Web Services

Quick and Simple Cron Expression Editor | crontab guru

Error Handling in API Gateway with AWS Lambda and Node.js

Mon, 10 Feb 2020 05:00:00 GMT

For a recent project, I set up a lambda function that would hydrate a template using data posted to it through API Gateway. The template name would be specified in the request, which means I needed to send back some errors if anything went wrong, such as the template not being found.

These errors are handled differently in async and non-async functions. We will go over those in detail below, then look at how to catch those errors in the Gateway and return the correct API response.

Setup

We can prepare two test events in the Lambda console to trigger errors and successful runs. Create a new lambda function using Node.js 12. You can leave the default execution role. Once the function is created, you will see options at the top right of the page. Choose to configure test events.

Create two test events using the code snippets below.

// Success
{
  "trigger": true
}

// Failure
{
  "trigger": false
}

We can now manually trigger successes and failures very easily using our test events. Let's set up our functions.

Async Lambda Function Errors

Async functions allow for errors and results to be returned using throw and return, respectively. The following sample function demonstrates this functionality.

exports.handler = async event => {
  let result = await new Promise((resolve, reject) => {
    if (event.trigger) {
      resolve(`Success!`)
    } else {
      throw `Failure!`
      // reject(`Failure!`) This will also bubble up as an error!
    }
  })
  return result
}

Alternatively, you can return a Promise from an async function, and the lambda function will automatically handle resolve and reject for responses and errors.

exports.handler = async event => {
  return new Promise((resolve, reject) => {
    if (event.trigger) {
      resolve(`Success!`)
    } else {
      reject(`Failure!`)
    }
  })
}

Non-Async Lambda Function Errors

Handling responses and errors in non-async functions comes down to context and callback, two parameters passed in to the handler functions. The original way to return a response or error was using context.succeed() and context.fail(). This, although still functional, was superseded by callback() once AWS added support for Node 4.3.2. These are implemented as follows:

Context Method

exports.handler = (event, context) => {
  if (event.trigger) {
    /* outdated but included for compatibility */
    context.succeed(`Success!`)
  } else {
    /* outdated but included for compatibility */
    context.fail(`Failure!`)
  }
}

Callback Method

exports.handler = (event, context, callback) => {
  if (event.trigger) {
    /* the first parameter is for errors, so we pass null for a success */
    callback(null, `Success!`)
  } else {
    callback(`Failure!`)
  }
}

API Gateway Error Mapping

Now that we have our errors propagating correctly from our Lambda functions, we can catch those errors and map them to HTTP responses. First, create a new REST API. We can leave default settings and add a name and description. Next, we'll add a POST method to it by choosing Add Method from the Actions menu, then choosing POST.

Once we click the check mark, we'll see a setup screen. We can choose any of our Lambda functions for the integration (leave Lambda proxy integration unchecked!).

Once the method is created, we'll see a screen that will show us the flow of our method.

First, we'll choose the bottom-left box that says Method Response. Here we can add a response (an HTTP response code) that our method will be able to send back. Add 400 as a response and click the check box to save changes.

Click Method Execution at the top of the screen to return to the previous overview.

We'll now be able to add the actual mapping of the error we throw in the lambda function to the 400 HTTP code we added! Click on the bottom-right box, the Integration Response. When we add an integration response here, we'll be presented with three boxes.

The first box will be a regular expression to match the error code we throw back. Let's set our regular expression to check for our error "Failure!". We'll need to add a wildcard in front as the error will be inside of a JSON object, and will thus have other characters in front of it. regex~.*Failure!.*

API Gateway won't run these checks against successful responses! If your lambda doesn't throw an error, it will pass through as a 200 response. You can gain a bit more control over this with Lambda Proxy Integration.

We can now choose the 400 error code from the dropdown and save our changes.

Test It Out

That's it for the setup! We can now test out our error mapping. On the overview screen, we see the leftmost box marked Test.

On the test page, we are given a box to provide a test body. We can use the same bodies that we used for the lambda function test cases. Add the failure request body and click Test. Your result will pop up on the right side of the screen.

Congrats! You've successfully set up error mapping in API Gateway.

Extra: Custom Error Objects

What if we want to send back something more than just a string as an error? We might want to send back more information such as a stack trace, a list of multiple errors, or a JSON object. Luckily we can do this without too much more effort!

First, let's edit our Lambda function to send a JSON object instead of just our regex~Failure! string by itself (make sure to still return this string somewhere in your response as that's how we're catching the error with our regular expression).

exports.handler = (event, context, callback) => {
  if (event.trigger) {
    /* the first parameter is for errors, so we pass null for a success */
    callback(null, `Success!`)
  } else {
    // callback(`Failure!`)

    // We have to stringify our object or we'll just get back "[object Object]"
    callback(JSON.stringify({
      error: `Failure!`,
      detail: `The request has failed due to the trigger passed in`
    }))
  }
}

If we test this, we see our error coming through! However, it would nice if we could send this back in JSON format instead of leaving it as a string. This is very easy to do with a mapping template.

Let's head back to the integration response page where we set up our regular expression. If we open up our failure mapping, we can see that there is a mapping templates section at the bottom of the page. We'll choose to add a template; type in the default MIME type already shown (html~application/json) and click the checkbox.

Next, we'll add the actual mapping template. There's a ton that you can do with these mapping templates (see the link at the end of the article), but we'll keep it extremely simple in this article. Add the following template:

$input.path('$.errorMessage')

This tells API Gateway to look at the errorMessage key on the incoming input (where our stringified JSON object is) and pull that object out. Save your changes, and you can test again.

Congratulations, you've got it! You can now pass your custom Lambda errors through API Gateway to the client!

Want to dig a bit deeper and learn a bit more? Here are some useful links:

Handle Lambda Errors in API Gateway | Amazon Web Services

AWS Lambda Function Handler in Node.js| Amazon Web Services

Create Models and Mapping Templates for Request and Response Mappings | Amazon Web Services

Node.js 4.3.2 Runtime Now Available on Lambda | Amazon Web Services

Kbravh's Blog

Interactive Pixel Spritesheet Animation with CSS

Spritesheet setup

User interaction

Create the animation

1. Offset property

2. Update property when checked

3. Transition our property

4. Move the spritesheet

Conclusion

Fungus February

Amanita muscaria

Amanita caesarea

Geoglossum difforme

Mycena rosea

Mycena haematopus

Craterellus cornucopioides

Sarcoscypha coccinea

Details

Create an interface between an LLM and a set of tools

USER

ASSISTANT

USER

ASSISTANT

USER

ASSISTANT

USER

ASSISTANT

USER

ASSISTANT

Create a custom LlamaIndex loader for Google Calendar

Enable logging with LlamaIndex

Configure LlamaIndex to read from Google Calendar

The script

Access calendar events from other users' calendars

Problems using a Vector store for calendar events

Change the model used by LlamaIndex

What is LlamaIndex?

How does it work?

Photo restoration using Lama-cleaner

Inkstrike

Footnotes

The Seamstress

Information

Effect

ASCII Text in ChatGPT with a virtual machine

ASCII text with vanilla ChatGPT

ASCII text with LinuxGPT and figlet

ASCII text with LinuxGPT and cowsay

ChatGPT still has a ways to go

Improving the DnD Constitution attribute

New skills

Endurance

Iron Gut

Recovery

Hardiness

Resilience

Isn't this just complicating things?

Give it a shot!

Retrieve the timezone from a JavaScript date

Making a state machine in TypeScript

Modeling our states

Building the state machine

Querying the state machine

Using the state machine in an application

Go forth and machine!

Typeguards and Closures in TypeScript

Solutions

Move the typeguard

Extract nickname into a const

Finding the coordinates of an element in a 2D array

Finding the coordinates using loops

What about a flattened array?

Final thoughts

HTML Entities from the Twitter v2 API

What are HTML entities?

How to clean up HTML entites

Why would Twitter encode the characters?

Encoding QR codes in Braille characters

Caveats

ASCII text with LinuxGPT and `figlet`

ASCII text with LinuxGPT and `cowsay`

Extract nickname into a `const`