Carmalou.JS

Parsing PNGs with Node, Part 4

Thu, 04 Sep 2025 00:00:00 +0000

We left off the last post looping over each pixel, wahoo! So this final post will focus on ✨filter types✨.

Here’s the image I’m working against:

And here’s the final output of this post.

In the last post, we were splitting the scanline into individual rows and looping over each byte in the row. Conveniently enough, the sample image has a 1:1 pixel to byte ratio, so we don’t need to do anything special to group bytes together to make a more complex pixel, or split them up. But keep in mind that won’t always be the case.

You might’ve noticed in the last method we worked on, if a row was 150 pixels wide, we were always adding one additional byte to the row. That first byte of the row was added to a variable called filterByte. (You can refresh your memory here.)

So what is a filter byte? The concept is so clever! Let’s circle back to the baby blue rbg value I referenced in the last post: 137, 207, 240.

Converted to a byte array, that single pixel looks like this:

[ 10001001, 11001111, 11110000 ]

Of course, this is just a single pixel, think about an entire image. We already talked about how unwieldy this could get, but consider how often a single color gets repeated in an image. Look at the sample grayscale image we’re using - there’s a lot of the same colors. Think about how much space a file could save if it could use one complete pixel value as a reference for other pixels - that is what Filter Types do. 🤯

There are 5 different values for a filter type:

0: No filter is applied. The values for this row are absolute values
1: Sub filtering - performs a look behind to the pixel directly to its left
2: Up filtering - performs a “look up” to the pixel directly above it
3: Average filtering - takes an average of the neighboring pixels
4: Paeth filtering - using neighboring pixels, Paeth is predictive in choosing the value closest to the computed value

Let’s go through these one at a time. In the previous post, we left off looping over the row, but now let’s assume we’re passing the entire row to a function where the looping is performed.

// note that this function will assume that currentRow DOES NOT include filterByte
+ function applyPixelFilter(filterType, bitDepth, currentRow, previousRow) {
+   switch(filterType) {
+     case 0: 
+     case 1:
+     case 2:
+     case 3:
+     case 4:
+   }
+ }

Since a filter type of 0 means the pixels are already in their absolute form, we’ll start with filter type 1. As mentioned above, filter type 1 is performing a look-behind to it’s left-hand neighbor. The first row in a PNG might use this filter type, since it doesn’t have any neighbors above it for reference.

When the look-behind is performed, the current pixel doesn’t just assume the value of the previous pixel - instead the value is the combined total of the left-hand pixel and the current pixel, up to 255. Note that this number wraps around, so any overflow beyond 255 is the actual value used.

  /* ... */

  switch(filterType) {
    /* ... */

    case 1:
+   let unfilteredPixels = []

+   currentRow.forEach((value, i) => {
+     if (i === 0) {
+       unfilteredPixels.push(value)
+       return
+     }

      // perform look-behind
+     unfilteredPixels.push(
+       (value + (unfilteredPixels[unfilteredPixels.length - 1] || 0)) % 256
+     )
+   })

+   return unfilteredPixels

    /* ... */
  }

In our first case (as we will in every case) we loop over the row. (Note that you need to check for the bitDepth to make sure that a pixel really is a single byte, as this assumes. I left that out to keep the code more approachable.)

Within the loop, we are assuming that the first byte value is an absolute value, which is safe for a look-behind, because that pixel has no left-hand neighbor to reference. Moving on, we are using the last value from our unfilteredPixels array as our reference. It’s important to note that as we are looping over the row, each previous pixel was also referencing its left-hand neighbor. In order to calculate the correct value for the current pixel, we need to look at the absolute value of the previous pixel, which we receive after filtering, not the provided value. This will be true for all filter types.

Filter type 2 is very similar, but instead of a look-behind, it performs a “look-up”. Using previousRow as our reference, we will combine the current pixel’s value and the absolute value from the same space in the previous row.

  /* ... */

  switch(filterType) {
    /* ... */

    case 2:
+     let unfilteredPixels = []

+     currentRow.forEach((value, i) => {
        const b = previousRow?.[i] || 0

+       unfilteredPixels.push((value + b) % 256)
+     })

+     return unfilteredPixels

    /* ... */
  }

Filter type 3 is a little bit more complicated, but it builds on types 1 and 2. Instead of adding the absolute value of a neighboring pixel, we take the average of two neighboring pixels - left and above - and add their average to the current pixel’s value.

  /* ... */

  switch(filterType) {
    /* ... */

    case 3:
+   let unfilteredPixels = []

+   currentRow.forEach((value, i) => {
+     if (i === 0) {
+       const b = previousRow[i] || 0
+       const avg = Math.floor(b / 2)

+       unfilteredPixels.push((value + avg) % 256)

+       return
+     }

+     const a = unfilteredPixels[unfilteredPixels.length - 1] || 0
+     const b = previousRow[i] || 0

+     const avg = Math.floor((a + b) / 2)

+     unfilteredPixels.push((value + avg) % 256)
+   })

+   return unfilteredPixels

    /* ... */
  }

Similar to case 1, we know that the first pixel in this row will not have a left-hand neighbor, so we take the average of the neighborhood above, and add that value to the current pixel. Remember that we are always taking absolute values after the filtering has been applied.

Moving on in the loop, we take each reference value, and find their average. That average is then applied to the current pixel’s value.

Lastly, we have Paeth filtering, which is the most complicated. It uses three reference pixels: left, top, and top-left. It might look like this, where D is the current pixel:

C  B
A  D

Let’s look at a Paeth function in isolation:

// paeth takes in each of our reference values
function paeth(a, b, c) {
  // first find P -- that is your comparison
  const p = a + b - c

  // now that we have p as a comparison, we can find which initial value is closest to p
  // take the absolute value of the difference between p and the current value
  // compare that difference to the previous difference
  // if the new difference is lesser -- use it
  const leastDiffObj = [a, b, c].reduce(
    (prev, curr) => {
      const difference = Math.abs(p - curr)

      if (prev.difference === null) {
        return {
          value: curr,
          difference,
        }
      }

      const diffObj =
        difference < prev.difference ? { value: curr, difference } : prev

      return diffObj
    },
    {
      value: a,
      difference: null,
    }
  )

  return leastDiffObj.value
}

We use p as a comparision value for the rest of the reference values. p takes on the value of a+b-c. We loop over a, b, and c to find the value that is furthest from p, but note that we want the outcome to be less than p. So given p=30, a=20, b=20, and c=65, even though c is furthest from p, a would still win because it is a lesser value.

Now that we’ve completed the paeth function, we can apply its result.

  /* ... */

  switch(filterType) {
    /* ... */

    case 4:
+   let unfilteredPixels = []

+   currentRow.forEach((value, i) => {
+     if (i === 0) {
+       const b = previousRow[i]

+       unfilteredPixels.push((value + b) % 256)

+       return
+     }

+     const a = unfilteredPixels[unfilteredPixels.length - 1] || 0
+     const b = previousRow[i] || 0
+     const c = previousRow[i - 1] || 0

+     let difference = paeth(a, b, c)

+     unfilteredPixels.push((value + difference) % 256)
+   })

+   return unfilteredPixels

    /* ... */
  }

Similar to cases 1 and 3, if we are on the first index of a row, that index has neither a left-hand neighbor, or a top-left neighbor, so b wins by default. Continuing through the loop, we use the paeth function to determine the winning value, and then apply that value to the current pixel, exactly like the rest of the filter types. (Note: the value being applied as a result of the paeth function is the absolute value of the winning pixel, not the difference between that value and p.)

Now that we’ve made it to the end of our filter type method, we have a multidimensional array of 8-bit numbers, representing grayscale pixels! 🙌 What do we do with it? The sky’s the limit!

If you’d like to see your progress thus far you could:

paste the result into the terminal
use a PNG library to rewrite your pixels into a new PNG (or write your own 👀)
use the pixels to create a .pgm file (that’s what I did here)

I learned a lot in this project, and I really enjoyed recounting the journey through these posts. The project was a challenge, but it was a blast! I hope you found these posts helpful and informative, and I really appreciate you for reading them. Thank you!

If you enjoyed this post, please check back soon. I’m still tinkering with PNGs, and this time I am implementing different image resizing algorithms and posting about my progress along the way.

Parsing PNGs with Node, Part 3

Wed, 03 Sep 2025 00:00:00 +0000

We left off the last post with the image metadata and raw bytes for the pixels.

Here’s the image I’m working against:

And here’s the final output of this post.

The IHDR chunk fully parsed provides this information:

{
  "width": 150,
  "height": 200,
  "bitDepth": 8,
  "colorType": 0,
  "compression": 0,
  "filter": 0,
  "interlace": 0
}

Our new function will take in the metadata and raw pixels like so:

const colorChannelMatrix = {
  0: 1,
  2: 3,
  3: 1,
  4: 2,
  6: 4,
}

function parsePixels(signature, pixels) {
  const { height, width, colorType, bitDepth } = signature

  // the product of the colorType and bitDepth determines the bits per pixel
  let bitsPerPixel = colorChannelMatrix[colorType] * bitDepth

  // using the derived bits per pixel * width, find the bits per row / scanline
  let bitsPerRow = bitsPerPixel * width

  // finally divide that by 8 to find the bytes per row
  let bytesPerRow = Math.ceil(bitsPerRow / 8)
}

Before we get too far into parsing the pixels, let’s go over the image metadata, the raw pixels, and what it all signifies.

The pixels parameter is a Buffer array, but let’s think about what that means. The buffer is a just a blob of pixels, and just like the chunks of the pixels, nothing signifies the beginning or end of a row. So how do we know when one row of pixels ends and another begins? Let’s get into it!

Let’s start by considering the difference between a single value versus a single pixel. Consider RBG values, such as a baby blue 137, 207, 240.

In order to apply that blue, we need all three of the numbers in the correct order. After all 240, 137, 207 is compromised of the same values, but is a completely different color!

It’s probably obvious that the individual values are 137, 207 and 240, but consider that the binary representation of this color looks like this:

[ 1, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0 ]

And that’s just a single pixel! We talked before about how the byte array of pixels is just a flat blob of bits. Take a high definition photo, it might be 1920 x 1080 pixels or ~2.1 million pixels. Think about that compared to the single pixel array up above, because that’s a lot of data without an obvious place to start or stop! It’s easy to see how getting off by 1 or 2 bits could really change the outcome of our image.

Fortunately, PNGs provide us with everything we need to pull pixels out of this haystack of 0s and 1s. To derive the number of bits in a single pixel, we combine bitDepth and colorType. There are 5 different color types, but for the purposes of this blog post, we’ll be focusing on grayscale. (Read more about different color types and their associated bit depths in the spec.)

A Color Type will only support certain Bit Depths. Grayscale is represented by two color type codes: 0 and 4. 0 indicates grayscale without transparency (alpha), and 4 indicates grayscale with transparency. Color type 4 supports bit depths of 8 and 16, while Color Type 0 supports all bit depths (1, 2, 4, 8, and 16).

You might be thinking, “ok, that’s interesting, but what does it mean?” Essentially, your colorType will tell you how many individual values make up a pixel, and your bitDepth will tell you how many bits make up an individual value.

So using my sample photo as an example:

let bitsPerPixel = colorChannelMatrix[colorType] * bitDepth // 1 * 8

For my image, the color type is 0, which indicates a single numerical value represents each pixel. The bit depth is 8, which indicates we need to parse 8 bits at a time to find the value for each pixel. Now we will use width to determine how many bits are in a row, and then convert that to bytes.

let bitsPerRow = bitsPerPixel * width // 8 * 150

let bytesPerRow = Math.ceil(bitsPerRow / 8) // 1200 / 8

Remember, while the sample image allocates 8 bits per pixel, or exactly one byte, it’s possible for a pixel to be larger or even smaller than a single byte. That’s why we first calculate the pixel size in bits and then convert to bytes.

Now that we know how many bytes are in a row, let’s continue:

function parsePixels(signature, pixels) {
  const { height, width, colorType, bitDepth } = signature

  // the product of the colorType and bitDepth determines the bits per pixel
  let bitsPerPixel = colorChannelMatrix[colorType] * bitDepth

  // using the derived bits per pixel * width, find the bits per row / scanline
  let bitsPerRow = bitsPerPixel * width

  // finally divide that by 8 to find the bytes per row
  let bytesPerRow = Math.ceil(bitsPerRow / 8)

+ const numScanlines = height
+ let currentScanline = 0
+ const scanlineLength = bytesPerRow + 1 // add one for the filterByte

  // a matrix containing pixel values per row
+ const pixelMap = []

+ while (currentScanline < numScanlines) { /* ... */}
}

This sets us up to loop over the pixels buffer array, knowing where a given row of pixels ends, and how many totals rows to expect. scanlineLength is the number of bytes per row plus one. This is because the first pixel of a row is allocated for the filter type. This was my favorite part of PNGs! I think filter types are so clever, and deserve their own post, so we’ll cover them next.

Let’s continue on the loop:

function parsePixels(signature, pixels) {
  const { height, width, colorType, bitDepth } = signature

  // the product of the colorType and bitDepth determines the bits per pixel
  let bitsPerPixel = colorChannelMatrix[colorType] * bitDepth

  // using the derived bits per pixel * width, find the bits per row / scanline
  let bitsPerRow = bitsPerPixel * width

  // finally divide that by 8 to find the bytes per row
  let bytesPerRow = Math.ceil(bitsPerRow / 8)

  const numScanlines = height
  let currentScanline = 0
  const scanlineLength = bytesPerRow + 1 // add one for the filterByte

  // a matrix containing pixel values per row
  const pixelMap = []

  while (currentScanline < numScanlines) {
    // keep reference to initial offset
+   let startingOffset = currentScanline

    // increment row offset
+   currentScanline++

    // slice the scanline to get the current row's values
+   let row = pixels.subarray(startingOffset, startingOffset + scanlineLength)
+   let filteredRow = []

    // find the filter type and set the offset
+   let filterType = row[0];

    // set the offset to 1 to skip the filter type value
+   let offset = 1;

    // loop over the row itself
+   while (offset < row.length) {
+     let i = offset;

      // log out the current pixel value
+     console.log(row[i])

+     offset++;
+   }

    return {
      signature,
      pixelMap,
    }
  }
}

Let’s recap. At the beginning of this post we had a flat byte array and the PNG’s signature. We used the signature to derive the number of bits per pixel, and how many individual numbers make up the pixel. We also determined how many pixels are in a row, and how many rows make up the image. With all of this information, we’re now looping over the image accessing individual pixel values - wow!

Are we done? If we take these pixel values and dump them into another file, will we be duplicating the PNG? Maybe, but probably not. There’s one more step, and that’s to apply the filter type. I’ll cover that in the next post.

Parsing PNGs with Node, Part 2

Thu, 07 Aug 2025 00:00:00 +0000

We left off the last post having successfully parsed the IHDR chunk, revealing metadata about the image.

Here’s the image I’m working against:

And here’s the final output of this post.

The IHDR chunk fully parsed provides this information:

{
  "width": 150,
  "height": 200,
  "bitDepth": 8,
  "colorType": 0,
  "compression": 0,
  "filter": 0,
  "interlace": 0
}

Now that we have the image metadata, let’s parse the rest of the image!

We’ll start by adding to our loop:

function parsePng(data) {
  let width;
  let height;
  let bitDepth;
  let colorType;
  let compression;
  let filter;
  let interlace;

  // this will hold the pixel bytes as we pull them from the image
+ let idatBuffers = []

  // first handle for the signature
  let i = 8;
+ let futureBufLength = 0

  // loop over the data
  while (i < data.length) {
    // find the length of the current chunk
    let chunkLength = data.readUInt32BE(i);

    // find the type of the current chunk
    let chunkName = data.toString("ascii", i + 4, i + 8);

+   let chunkData = data.subarray(i + 8, i + 8 + chunkLength)

    // access IHDR
    if (chunkName == "IHDR") {
      // create an offset to set the position at the beginning of the data section
      let ihdr = i + 8;

      // parse the header data
      width = data.readUInt32BE(ihdr);
      height = data.readUInt32BE(ihdr + 4);
      bitDepth = data[ihdr + 8];
      colorType = data[ihdr + 9];
      compression = data[ihdr + 10];
      filter = data[ihdr + 11];
      interlace = data[ihdr + 12];
    }

    // get each chunk of pixels
+   if (chunkName == 'IDAT') {
+     idatBuffers.push(chunkData)
+     futureBufLength += chunkLength
+   }

    // update the offset
    i = i + chunkLength + 12;
  }

  // return the header data
  return {
    width,
    height,
    bitDepth,
    colorType,
    compression,
    filter,
    interlace,
  };
}

In the last post, we talked about how there are no spaces between chunks, so updating the offset will bring us to the beginning of the next chunk. For us, the next chunk we care about is the IDAT chunk.

There can be multiple image data (IDAT) chunks, and these chunks take on the structure of any other chunk. Their first four bytes will contain the length of the data section. The next four will indicate the chunk type. After the type, we’re finally accessing the actual data, phew!

Within our IDAT if-block, it’s pretty straightforward. We use chunkData, which is a Buffer array containing only the data, and we push that into idatBuffers which will be our reference to our pixels.

Now that we have our pixels in an array, we’re ready for the next step.

const zlib = require('zlib')

function parsePng(data) {
  let width
  let height
  let bitDepth
  let colorType
  let compression
  let filter
  let interlace

  // this will hold the pixel bytes as we pull them from the image
  let idatBuffers = []

  // loop over the data
  // first handle for the signature
  let i = 8
  let futureBufLength = 0

  while (i < data.length) {
    let chunkLength = data.readUInt32BE(i)
    let chunkName = data.toString('ascii', i + 4, i + 8)
    let chunkData = data.subarray(i + 8, i + 8 + chunkLength)

    // get and set the signature
    if (chunkName == 'IHDR') {
      let ihdr = i + 8
      width = data.readUInt32BE(ihdr)
      height = data.readUInt32BE(ihdr + 4)
      bitDepth = data[ihdr + 8]
      colorType = data[ihdr + 9]
      compression = data[ihdr + 10]
      filter = data[ihdr + 11]
      interlace = data[ihdr + 12]
    }

    // get each chunk of pixels
    if (chunkName == 'IDAT') {
      idatBuffers.push(chunkData)
      futureBufLength += chunkLength
    }

    // signifies the end of the file
    // since we are all the way through the file, return the signature and pixels
+   if (chunkName == 'IEND') {
+     const pixel = Buffer.concat(idatBuffers, futureBufLength)

+     try {
+       let decompressed = zlib.inflateSync(pixel)

+       return {
+         signature: {
+           width,
+           height,
+           bitDepth,
+           colorType,
+           compression,
+           filter,
+           interlace,
+         },
+         pixelBytes: decompressed,
+       }
+     } catch (err) {
+       console.error('Failed to decompress:', err)
+       return
+     }
    }

    i = i + chunkLength + 12
  }
}

The last chunk of a valid PNG will always be the IEND chunk. Its data section will be empty, so once we find this chunk, we know we’re at the end of our file stream.

A few things happen here:

First, we create a new Buffer array with all of our pixel bytes (note that even though the idatBuffer array was a nested array, the pixel array will be flat).

That’s not the important part though - the most important part of this new set of code is here: zlib.inflateSync(pixel). PNGs compress their pixel data using zlib compression. You can find more information about this here in the spec.

We’re using the Node zlib library to “inflate” or decompress the pixel data. I plan to manually implement the algorithm in the future, but for now, we’ll just use the library.

Now that we have the image metadata (signature) and the raw pixels – we’re ready to get started! Check out the next blog post where we finally start decoding these pixels!

Parsing PNGs with Node, Part 1

Tue, 05 Aug 2025 00:00:00 +0000

This blog post is the first in a series about image manipulation and parsing. This project started because I was curious how I might go about manually resizing an image. I’m also experimenting with manual image generation as a part of this project.

If you’d like to follow along as we go, the repo for the project is here. The PNG spec is here.

Here is the image I used:

libpng also has a comprehensive suite of images to use in tests.

Let’s jump in.

The question: What do the internals of an image actually look like?

It started with curiosity around this question: “What actually is an image? What does it look like under the hood?” Obviously different image formats will look different interally, so I had to pick one to start with, and I chose PNGs. Why PNGs? I didn’t have any particular reasoning behind selecting PNGs as my test case. Since I work on a Mac, I thought being able to generate my own PNG samples via screenshots might make this easier. That turned out not to be the case, but more on that later.

I knew there was only one to find out – crack open the file and look inside.

PNG Internals

While there are many different chunk types defined for a PNG, only three are absolutely required for a valid PNG. The IHDR chunk, at least one IDAT chunk (although there could be multiple, depending on the size of the image), and lastly the IEND chunk.

Because the goal of my project was not to write a brand new PNG viewer (although, that could happen in a future post!), I focused only on the absolutely necessary parts. As such, this post will not cover chunk types outside of IHDR, IDAT, and IEND.

Note: If you’d like to learn more, all chunk types are covered in the spec.

IHDR Values and Structure

PNGs are made up of multiple different “chunks”. In order to be a valid PNG, the first chunk must be the IHDR chunk. The Image Header chunk (IHDR) contains, you guessed it, information about the image. It is always 13 bytes and contains the following information, which will always be in this order:

Width (in pixels)
Height (in pixels)
Bit depth
Color type
Compression method
Filter method
Interlace method

I’ll explain more about these in the next post. For now, just know that since the image is a flat array of bytes, we need all of this information to derive information such as: where a row of pixels starts and stops, how many bits or bytes make up a single pixel, how many color channels are used, and more. For now, we’ll focus on parsing out this information.

Parsing the IHDR

Here is an example from my project to parse the IHDR chunk.

First, we need to handle for the PNG signature. The signature is always 8 bytes (bytes, not bits), and contains the same decimal values. These values indicate the file type is a PNG. Since we know we’re parsing a PNG, we’ll just skip the signature.

Our method will take the buffer response from reading the file as it’s parameter.

function parsePng(data) {
  // the variables we are looking for from the IHDR chunk
  let width;
  let height;
  let bitDepth;
  let colorType;
  let compression;
  let filter;
  let interlace;

  // first handle for the signature
  let i = 8;
}

So far, it’s pretty straightforward. The data parameter is a buffer (an array of bytes). After we instantiate the variables we plan to parse out of IHDR, we create our offset, which skips the first 8 bytes. Next we would start looping over the buffer, but before we get too far into that, let’s talk about chunk structure.

In the first section, I mentioned that the PNG buffer is a flat array of bytes - that means there are no delimiters to signal the end of a line or the beginning of the next line. There’s also no delimiter between chunks. In fact, there are no spaces or empty bytes at all between any of the sections of a PNG. However, it’s possible to derive when you’re at the end of a chunk, based on the structure.

A chunk is made up of four parts:

Length - the first 4 bytes of a chunk indicate the length of the data section. It’s important to note that this length is not the length of the current chunk. It is only representative of the data within the chunk.
Type - the next 4 bytes tell you the type (name) of the current chunk
Data - this is the part we are really after! This is the only part of a chunk with a dynamic length. All other sections are 4 bytes long.
CRC - the final 4 bytes of a chunk. It’s essentially a checksum to validate your offsets

Every single chunk type will follow this structure and order, including IHDR and IDAT chunks.

Now that we know a little more about what we’re looking at, let’s continue:

function parsePng(data) {
  let width;
  let height;
  let bitDepth;
  let colorType;
  let compression;
  let filter;
  let interlace;

  // first handle for the signature
  let i = 8;

  // loop over the data
+  while (i < data.length) {
    // find the length of the current chunk
+    let chunkDataLength = data.readUInt32BE(i);

    // update the offset
+    i = i + chunkLength + 12;
+  }
}

We’ve added the loop and the first building blocks we need. In the loop, we first find the length of the data section, and then we use that to update the loop’s offset. We take the current position in the buffer, and add the data length (because this is the part of a chunk whose length is dynamic) + 12 bytes, to cover the rest of the chunk. Remember, there are no spaces or empty bytes between chunks, so this offset will take us to the very first byte of the following chunk.

Now we’re ready to parse the IHDR chunk:

function parsePng(data) {
  let width;
  let height;
  let bitDepth;
  let colorType;
  let compression;
  let filter;
  let interlace;

  // first handle for the signature
  let i = 8;

  // loop over the data
  while (i < data.length) {
    // find the length of the current chunk
    let chunkLength = data.readUInt32BE(i);

    // find the type of the current chunk
+   let chunkName = data.toString("ascii", i + 4, i + 8);

    // access IHDR
+   if (chunkName == "IHDR") {
      // create an offset to set the position at the beginning of the data section
+     let ihdr = i + 8;

      // parse the header data
+     width = data.readUInt32BE(ihdr);
+     height = data.readUInt32BE(ihdr + 4);
+     bitDepth = data[ihdr + 8];
+     colorType = data[ihdr + 9];
+     compression = data[ihdr + 10];
+     filter = data[ihdr + 11];
+     interlace = data[ihdr + 12];
+   }

    // update the offset
    i = i + chunkLength + 12;
  }

  // return the header data
+ return {
+   width,
+   height,
+   bitDepth,
+   colorType,
+   compression,
+   filter,
+   interlace,
+ };
}

You might notice a difference between the way we treat the first value within a chunk (length) and the next value (type). The length is always 32-bit integer, whereas the chunk type is always 4 ASCII characters. Because these are different data types, we must access their values differently.

The next bit of code is very straightforward. We identify the IHDR chunk and set the header values. Notice again that the first two values are treated differently than the next five. Because those first two initial values are 32-bit integers, we need to parse all four bytes together to find the correct value for each.

The final five values are all single byte values, so they can be directly accessed.

The image header for the test image I used looked like this:

{
  "width": 150,
  "height": 200,
  "bitDepth": 8,
  "colorType": 0,
  "compression": 0,
  "filter": 0,
  "interlace": 0
}

Now that we have the image header information, we’re ready to get into the actual pixel values of the PNG! I’ll cover that in the next blog post.

Migrate Data In Postgres

Wed, 23 Apr 2025 00:00:00 +0000

Lately I’ve been doing a lot more data-driven work, including data analysis. From this work, I’ve needed to import and export a lot of data. Here is my best quick-and-dirty way to copy a table from one Postgres table to another. I frequently use this method to refresh my local environment with production data.

Of course, if you’re using a tool like pgAdmin, you can export data at the click of a button, but depending on what you plan to do with the data, that may not be what you want to do. Using the pgAdmin export will include all of the record IDs, so if you’re using this data to augment another database (or in my case, refresh), this might not be what you want. But if that doesn’t matter, using a built-in import/export tool is definitely the easiest solution.

Unfortunately this usually doesn’t work for me, so this is what I do instead:

Export the prod data into a CSV

\copy (select columnA, columnB, columnC from tablename) to 'absolute/path/plus/file.csv' csv DELIMITER ',';

Make sure you include the absolute filepath, otherwise you’ll have a lot of trouble finding your csv. (This is the absolute path to your local machine.)

You can include any query above, so if you need to filter or sort in a certain way, adjust your query accordingly.

Drop the data from my local database and reset the sequence

Make triple-sure you’re in the correct environment before you do this!!

truncate tablename;

select nextval('sequencenameforprimarykey');
alter sequence sequencename restart with 1;

select nextval('sequencenameforprimarykey');
select count(*) from tablename

After you truncate your table and reset your sequence, you can select each and see that you’re ready to start fresh with your prod data.

Insert data from the CSV

\copy table_name(columnA, columnB, columnC) from 'absolute/path/plus/file.csv' csv DELIMITER ',';

Notice that the above command is basically just the inverse of the first \copy command in step 1.

Depending on how much data you’re copying over, this step might take a while. At the end, you’ll see something like COPY 137 in your terminal, which indicates the copy is complete and 137 records were copied into your table.

I’ll usually use one more select count(*) from tablename to see the total number of records in the table, as a sanity check.

A few gotchas to watch for:

Permission denied errors

If you used \COPY without root permissions, try again with \copy. The \COPY command has a slightly different signature and requires root access, but \copy will work regardless.

“Cannot insert {type} into column”

An issue of incorrect data to column type. When this happens to me, it usually means I forgot to specify the columns in my \copy to command, or I specified them in the wrong order. If you are going to ignore the IDs when you move data from one place to another – which is a good idea – you must specify the columns in your \copy to command!

“Missing data for column”

When this happens to me, it usually means I forgot to specify the delimiter to be a comma in my \copy to command. Check your CSV file and ensure that you’re using the correct delimiter for your data.

“Duplicate key value violates unique constraint”

Most likely cause is that you copied your entire table, including ids, but forgot to drop your secondary table’s data and/or reset the sequence.

And that’s about it! At this point, I’ve copied production data into a local or sandbox environment where its safe to tinker with or even delete data. Just be careful and always, always, always make sure you’re not in production when tinkering!

Happy SQL-ing!

Shutting down my startup: The End of Neptune Divorce

Sun, 16 Oct 2022 00:00:00 +0000

It’s sort of hard to know where to start writing this blog post, especially since I’ve never blogged about this project before. I think the idea of “#BuildInPublic” where you share your progress along the way is neat, and its great for marketing, but, funny enough, its also a lot of work! When you’re trying to build a business - marketing, coding, networking, selling, and all that - it’s tough to find the time to craft a cute tweet about where you’re at.

I’ll be honest here and add that, just like with all of social media, I think people share the happy bits, and skim over the hard parts. That mentality always made me a little hesitant to share, because my experience with Neptune was a lot more hard parts than happy parts.

Tl;dr: Neptune Divorce is shutting down. The rest of this post is a chronicle of the journey I took building Neptune, and any nuggets of knowledge I might’ve gained along the way.

Neptune: the beginning

I first tweeted about Neptune Divorce a little over a year ago when we had our launch on Twitter. My business partner and I had been slowly working on it for a long time. For me, it all started years ago when I went through a divorce myself and I couldn’t believe that there wasn’t some sort of online service to handle all of this for you. I thought at the time that I had some sort of brilliant revolutionary idea, but, spoiler alert, today I am not so sure.

Neptune, in a nutshell, was a way for someone to go online, answer a few questions, and get divorce paperwork. The entire process could be completed in less than an hour. I knew from my own experience how difficult it was to find someone both affordable and reliable to prepare divorce paperwork. And, of course, the last thing you want to think about when your life has been turned upside down is whether or not the person who is supposed to help you is actually just bullshitting you.

I thought this beautiful little app could alleviate at least some of the stress a person goes through when they are getting divorced. We wanted to make it as easy as possible, so creating an account was free, and using the app was free. You paid at the end, once you were ready to actually get your documents. My partner and I hoped that if we left payment until the end, we would be able to build a trust relationship with our customers.

Neptune germinated in my brain for a few years before anything came to fruition. I talked endlessly about the concept, built a small prototype, and generally thought I had a winning idea. But, at the same time, I was too scared to launch a product like this without a business partner, and more specifically a legal business partner. Finding the right business partner was tricky for me. As a woman in tech, I was wary of partnering with the right person who was going to listen to me and let me do what I do best. Almost every day, I deal with dudes in tech who don’t take me seriously, so the very last thing I wanted was something like that in my own business. I’m thankful to say that connecting with my business partner was likely the best thing to come out of this whole adventure. She’s an absolute beast in her field, and at the same time, she recognizes that I am an expert in mine. We complement each other, and working with her has been an absolute blast.

Cheerleaders, pitches, and accelerators

One person I want to call out (in a positive way) is Kristin Garcia. She is truly a gem, and was our biggest cheerleader along the way. Kristin and I met through Techlahoma, and I can’t say enough kind things about her. When I shared this project with her, I was still very shy about talking about it, but Kristin saw the potential before I did.

Kristin pushed us to do two really important things: complete a pitch competition and apply for an accelerator. Even though Neptune is at an end, those were really cool things to be a part of.

The pitch competition

Kristin reached out about a pitch competition in Tulsa. The only downside was that we didn’t have a pitch deck, and the competition was only days away. Once we knew we would be pitching, we spent every free second practicing and working on our pitch deck.

The competition was specifically for female founders and, let me say, there’s nothing like being in a room with a bunch of supportive women doing really amazing things. Every business there deserved to take first place, and even though there were only two prizes, all of these ladies remained so supportive of each other. I don’t have words for how amazing it was. It will always be one of my favorite experiences of the whole Neptune rollercoaster.

Unfortunately since we had so little notice, Harmonniey, my business partner, wasn’t able to join. This meant I had to pitch, which is not really what I wanted my role to be in the business. I would’ve been happy to stay behind my laptop building, and let Harmonniey do all the talking. But if we wanted to participate in the competition, it was going to have to be me.

On the day of the competition, my sister and I hopped in the car and drove nearly 2 hours to Tulsa. At this point, I was still really nervous sharing Neptune, so I practiced my pitch all the way there. As I sat in the audience listening to all these incredible women pitching their businesses, it was clear that all of us were all trying to use our skills to make positive change in the world.

After the pitches were complete, the judges disappeared to discuss who would win the prizes, and the founders had time to chat with the audience. I had so many people approach me to congratulate me and tell me how incredibly needed this product was. Everyone had a story about a friend, aunt, brother, or cousin who had the hardest time navigating the divorce process and how this app would’ve been so helpful for them. One woman told me that Neptune would be “generation-changing.” I still think about that sometimes. It was so much validation and exactly at a time when we needed it. We had done a soft-launch the previous September, and since neither myself or Harmonniey really knew how to market, nothing had happened. I tweeted about it once, and then got a few positive responses from friends, but nothing more. Here we were in March, a full six months later, with no customers yet, but hearing this positive feedback made me sure that we would get customers, if we could only be patient.

When the judges returned to announce the winners, they said what everyone in the room was thinking - it was an incredibly hard decision. Everyone who presented had a great idea, and they wished that they could give everyone a prize. They announced first place, and then they announced second place - no Neptune. Of course, I was disappointed, but it was hard to be too disappointed after such great feedback from the crowd, and when the winners were so deserving. But then - I’ll never forget this - completely out of left field, they announced a surprise third place prize and it was for Neptune. Talk about validation - the crowd telling me how great an idea this was, and then the judges saying that they added an extra prize and it was going to Neptune?! Surely, surely this idea would work. All we needed was time. (As a side note, having my sister there with me for that was amazing. I’m not sure I would’ve even believed it had she not been there too.)

The accelerator

Not long after the pitch competition, Kristin struck again! She knew of an accelerator and she thought we should apply. I’ll admit, I wasn’t sure. Long before Harmonniey had come onboard, back when Neptune was little more than an idea and a prototype, I had applied to an accelerator as a solo founder, and I didn’t get in. That stuck with me, and because of it, I was pretty sure we wouldn’t get in. But with Kristin’s encouragement, we did but Kristin really encouraged us to try, and so Harmonniey and I took the leap. We got in!

The accelerator was intense. I already thought about Neptune all the time, but this turned it up to 11. Each week we would meet with different mentors - business owners, marketers, venture capitalists, and all kinds of people each with varying disciplines. And each week we would explain that, no, we didn’t have customers, but it was a solid idea, and we knew it could work. Each week, we asked the same question: how do you market something like this? It isn’t a purse, it isn’t an entertainment app. Getting divorced is a very private, very painful thing, Even if you’re the person initiating, it is still turning your whole life upside-down. How do you sell something like that? Targeted ads? Well, most people don’t hop on social media and share their exciting divorce news. Even though we kept telling people we didn’t have customers yet, we tried to remain upbeat. And our mentors were mostly upbeat as well. Most of them were excited about the idea and they thought it was really innovative and needed.

The accelerator was very focused on getting a business funded through VCs, and even though Harmonniey and I weren’t sure about taking on funding, working on a pitch was a big part of the cirriculum. We leaned in and practiced our pitch, and tried to gloss over the fact that we didn’t have any paying customers. After all, the numbers appeared to be on our side. The divorce industry in the U.S. is a billion dollar a year industry. Oklahoma, our homebase, has the second highest divorce rate in the country. If we could capture as low as 5 percent of the Oklahoma market, that would be a sustainable business. If we could duplicate that in other states with high populations, like Texas, we were talking about millions of dollars per year.

As practice leading up to the big pitch night at the end of the accelerator, we had “Investor week” where we practice-pitched to investors. This coincided with a vacation I had planned, so I took my laptop on vacation and pitched investors from my hotel room, and tried to figure out new and different ways to spin a lack of customers. Investor week went surprisingly well considering our customer issues. And even better, during investor week, Harmonniey and I finally pulled the trigger on paying for Google ads. And if that wasn’t enough, during this vacation, I learned that we had our first two real sign ups on our website. Let me tell you - I was elated. I texted Harmonniey at 11pm to tell her the great news. It was really happening! Our hard work was finally paying off, and Neptune was going to work. I could feel it.

Believing in the mission

Any article from Insider or Forbes will tell you that you have to be your business’ biggest cheerleader. Of course that is true, but at the same time, that’s not really my style. I’m naturally a very cautious person. Some people have even called me pessimistic. And for a long time, that carried over to Neptune as well. I believed it could work, but I wasn’t willing to bet the house on it. But at some point in all of this, I started drinking my own kool-aid and believing we had something really amazing on our hands. And part of me still believes that’s true.

We had a socially-minded app, when it feels like most apps coming out of Silicon Valley are putting different kinds of mustaches on an image and saying that’s “changing the world.” We were two women building a tech company in Oklahoma, one of the most conservative states in the nation. And, over time, we came to the realization that our app wasn’t just about making the divorce process easier, but an access-to-justice app. We started to really focus on the idea that our app was about making the justice system more accessible to individuals with lower incomes. And I really, really wish that had turned out to be the case. I think it would’ve been amazing to achieve something like that, but unfortunately the numbers got in the way.

Making the call to use ads

Of course, through our time at the accelerator we had spoken to lots of different business owners, and a resounding piece of advice we got was don’t use ads. Don’t pay for ads until you have to. And, to an extent, I do think this is a solid piece of advice. Ads are very expensive, and there are many free ways to market a business. Social media is an obvious one, but even simpler than that, just finding potential customers and talking to them. Yes, it’s uncomfortable, but it’s free.

Marketing divorce, though, that is tricky. People don’t want to tell you they’re getting a divorce. Harmonniey and I talked about this every day for months - how do we market something that’s focused on one of the worst things a person can go through? Everything we read seemed to be focused on physical products, a specific niche within a business, or fun consumer-based apps. Every week we asked our different mentors about how to market something like this, and we scoured the internet searching for tips on something like we were building, and found almost nothing.

After a while ads seemed like our only option. We went into it accepting that if we didn’t have any customers after 60 days, we would shut down ads and reassess. We initially tried running ads ourselves and found it very difficult. We weren’t getting the results we wanted, so we hired someone to help us. At this point, we were spending a significant amount of money each month on the business. We both knew this wasn’t something we could justify forever, but we both really believed that if we could only reach the right people, we would start getting paying customers, which would justify paying for the ads.

Pitch Night

As if it were some sort of teen sports movie, it had all led up to this: Pitch Night! Pitch night was open to the public and it was the place to showcase all we had learned and done during the accelerator.

Harmonniey had encouraged me to do the pitch, since it was my initial idea, and my story behind the app. I agreed with her, but in my perfect world I would live behind the scenes and just build the app. Talking about Neptune meant talking about my divorce, and I always felt weird about that. But I understood what Harmonniey meant, and I practiced the pitch diligently.

Even though we had some sign ups on our app, the ads had only been running a few weeks at this point, so we didn’t have any customers from ads yet. And unfortunately, none of the sign ups we did have had turned into paying customers. Just hours before the pitch, I received an email from a user I had been corresponding with where she shared that she had gone another direction due to the app being too buggy. It was a devastating blow just before the pitch. We resolved the bug really quickly, but it was too late for that customer.

It was tough to get back in the mindset before the pitch, but with encouragement from Harmonniey and other friends from the accelerator, the pitch went really well. Lots of different people, including a handful of my friends, attended the pitch and we got more validation about how necessary something like this is. But at the same time, one person asked a haunting question: “with the numbers you described, if the app is as easy to use as you say, why isn’t everyone using it?” I felt dread even before they had finished asking the question, because I had been asking myself the same thing for months, and I couldn’t come up with an answer.

Going downhill

After the accelerator, we kept going with as much enthusiasm as we could, but it became more and more difficult without any customers. We had far more money going out than we were seeing coming in. After a few months like that, it was difficult to imagine that a few orders would come in and make up for the deficit, but still we pushed on.

I made the call to lets ads go on for longer than planned because I really believed if we just had more time something would change. But after four months - twice as long as we had initially agreed - it was clear that Neptune wasn’t going to work. We had the data to show that people were in fact making it to our site, and some people were even signing up. But something was happening after sign up. People would simply drop off and never log in again. Any business advice you read about user acquistion tells you to reach out to your users actively for feedback. Give your product away for free in exchange for a 20 minute call for feedback, talk about your product all you can and accept all feedback.

But when the product is divorce paperwork, its difficult to reach out and ask someone to share that experience in the name of feedback. And that’s assuming anyone responds at all. It’s already difficult to get customer feedback on surveys, but can you imagine if someone asked you to please share how your divorce went?

Shutting down

After four months of ads, the numbers were clear. Of the traffic that made it to the site, we did have some user conversions. But of those users, almost none actually used the app, and fewer still completed the process. No one completed the process and went on to pay for the completed paperwork and instructions. Four months of paying for ads, a year in business, and thousands of hours building, talking, thinking, and living Neptune, I had to accept that it just wasn’t going to work. Late last month, Harmonniey and I met at her office, exchanged very sad glances, and agreed that it was over.

What does it all mean?

It’s difficult to know why users were dropping off after sign up, but we have some ideas. One of those ideas is that we have a trust issue with our customers. Obviously the divorce process is a very emotional time, whether you’re initiating it or not. One thing we heard a few times was “how does a person know this site is legit?” Of course we included all over the site that it was designed and created by an attorney, but trust is about more than the idea that the paperwork is correct. Going to court and filing a divorce petition is overwhelming. The trust relationship a person has with their attorney is about trusting that the attorney is writing up the correct paperwork, and it’s also about how that attorney makes their client feel. Can an app replicate that sense of a security? I think some apps can for some people, but its clear to me now that Neptune did not.

Another difficult item was pricing. There are businesses that provide this service at a much lower cost. We thought we could charge more for our service since it was created by an attorney, however a few times we got feedback from non-customers that the app was too expensive compared to our competitors. (A few times we even got feedback that it should be free!) This creates problems with the sustainability of our business. I wouldn’t say that Harmonniey and I were in this to make millions (although we wouldn’t have turned it down!), but we also didn’t want to offer it for free. After all, it’s a single-use product (you would hope anyway), so any time you drop the price, that dramatically affects your overall earnings. There are a limited amount of divorces we could hope to capture, so it’s not as though we would be able to make it up on volume either.

Lastly, it takes a long time for a person to go from thinking about getting a divorce to being ready to actually file. We believe it’s very likely that a lot of the users we saw sign up will end up filing, but how long will it take? I’ve read estimates that say people take an average of four years to make the leap from thinking about divorce and deciding to do it, to actually filing and getting divorced. When we thought about that, it became even more difficult to keep going. We had ongoing costs each month for infrastructure alone, not even considering the ads we were having to pay for in order to get these sign ups. If we assume generously it takes a year for a person to sign up for our app and then convert to a paying customer, we have lost money on that customer without even accounting for ad spend. It was an untenable situation.

Couldn’t you just leave it going?

We did consider just turning off ads and leaving the app running. With ads turned off, it costs very little to actually run the site. I’m fairly certain we could go into the settings on our infrastructure provider and save even more money, but I also think it’s more painful to leave it going if it isn’t going to do anything. Another consideration I’ve had to make is that if I am doing this, that means I am not doing something else. The human brain can really only work on one thing at a time. Even when we believe we’re multitasking, our brains are really just rapidly switching back and forth between the two tasks, giving the illusion of doing two things at once. Of course in actuality, that means that neither task is being done as well as it could be.

So taking everything above into consideration, the emotional cost of watching Neptune slowly fail, versus pulling the plug now based on the data we have. The fact that working on this means not working on something else that could have more potential, and of course the money being spent on something that has no returns. The decision was difficult, of course, but it also became the obvious decision after a while.

What I learned

Do your research first

It’s probably not a huge surprise, but everyone who says talk to potential customers before you build is right! I kept telling myself that Neptune was different, and these potential customers would be too hard to reach and likely wouldn’t even want to talk about something so personal anyway. All that might be true, but today if I had an idea like that, I would think very carefully about how viable the idea is if I can’t talk to anyone who might purchase it before building.

It doesn’t have to be expensive to run an app

It’s actually quite inexpensive infrastructurally-speaking to run an app. Our app was less than $30/month, and it could’ve even been cheaper. We avoided AWS or anything that felt like it was going to take too much time to set up or stay on top of. Our main focus was finding a solution that would allow us to run our code as hands-off as possible. That meant managed databases and something with a simple interface. We decided to go with Render, and it was a good choice. I’ll be remaining with Render for future projects.

Finding the right balance is hard

Through this experience, I can absolutely understand why people want to quit their jobs and go full time on a project. You cannot overestimate the time commitment of building something new and then convincing people that it is worth parting with their money. I definitely understand the dominant advice about not quitting your job until you’ve replaced, or even doubled, that income. But it’s also very exhausting to work what quickly becomes 2+ jobs. And if you add in a partner or children into the mix, it’s a compounding issue.

I listened to a podcast where a married couple had built a business, and the husband/father remarked at one point that it was tough because, no matter what, you would always feel guilty. You might feel guilty because you’re spending time with kiddos, but there’s biz work to be done. But then, if you’re working on the business, you’ll feel guilty that you aren’t spending that time with your children. I found this to be exceedingly true for myself. My oldest child likes to play “Mother hen” where they try to care for everyone around them, and that includes me. On a few different occasions, I would be working late and they would come into my office to chide me for being up on the computer so late. Even in a situation where I thought I had found a balance - spend dinner and hang out time with kids, and then work after bedtime - it was still difficult to feel good about the “balancing act”.

Pitching your idea is harder than you think

Speaking and presenting at technical events and conferences is not at all the same as speaking and pitching a business idea. This one took me a long time to learn. I had been talking to my friends about Neptune for years, so I was very used to talking about the pros and cons of different tech stacks, or where to host the project, or how to manage the document generation piece. But when you’re pitching a business, no one cares about that at much. Truthfully, people mostly care about if you can make money. After that, they care about how much money you can make, and then how much revenue that turns out to be after costs. Your tech might rank fifth on the list of things people care about. This really bummed me out! I can’t say I was exclusively interested in the tech, but it ranked very high on my list, and it was something I was very comfortable talking about.

Data, data, data

People will often tell you what they think you want to hear in a conversation. If you ask someone if they think something is a good idea or if their organization would be open to partnering, most people will say yes because they don’t want to hurt your feelings. I definitely understand that instinct, but as a founder who was trying to figure out the viability of a product, it makes things really hard! If you are considering the viability of an idea, I would encourage you to keep this in mind.

On the other hand, some people seem to think that the best way they can help you is to tear your idea to pieces. Some people will be really mean about it! Try not to let those people get inside your head. I’ve heard advice before along the lines of: “don’t ever listen to people who aren’t doing the work.” Obviously this advice shouldn’t be taken to an extreme, but listen to your own intuition. If it feels like this person doesn’t know what they are talking about, or if they are just being mean for the sake of being mean - listen to that feeling.

But above all, listen to the data. Numbers don’t lie. If you’re doing the marketing and you’re doing the sales, and talking to potential customers, but not getting any bites? Give it a reasonable amount of time, and then reassess. Just because an idea isn’t working right away doesn’t mean it won’t work. But as I described above, an idea “working” is very relative. One person using Neptune would’ve been really amazing, but it wouldn’t have made us all of our money back. You have to have your own definition of what works, and how long your idea can take to work, and then stick to it. Take it from me, it really really hurts to continually give something more time, but then it still doesn’t work.

What happens next?

I didn’t even realize how much of myself I had tied up in this project until we officially made the call shut it down. From our conversations, I know Harmonniey has ideas for what we can do next, but if I’m being honest, I don’t know what is next for me. Failing is incredibly hard, and having gone through it with something so important to me, its really hard to sign up for that possibility again. I think in all likelihood, we will try again on a new idea, but at this moment in time it’s a little hard to consider.

I wish I had a better note to end this on. All I can say is that if you made it to the end of this post, I really appreciate you for reading. If you have any questions or want to chat, please find me on Twitter.

Accounta-Billie Part 2: The first weekend

Tue, 20 Oct 2020 00:00:00 +0000

Note: This blog post references an app I no longer maintain. I let the domain go as well. I left the post up because it was a fun little project, and I enjoyed building it, as well as writing about it.

As I write this, I have scheduled Accounta-Billie to launch a beta this Friday, 10-23-2020. Looking back I have been working on it for about 3 months, which makes me chuckle now because my initial goal was to finish version 1 in a weekend(!). Three months as a side project is honestly pretty respectable in my opinion, especially since it is actually launching Friday, versus many, many other projects of mine that have been started but have never seen the light of day.

You might be wondering why I gave in to the hype of a “weekend startup” and made that audacious goal in the first place. Firstly, I guess part of me really likes hack-a-thons. I don’t really participate in them outside of my home, but I do really like getting loads of soda and chips and staying up super late hacking away and watching Silicon Valley.

via GIPHY

But beyond that, I also really believe in taking an idea from start to finish in the shortest amount of time possible. If you have an idea and you aren’t sure if it is viable, doable, whatever, its a smart idea to answer that question as soon as possible. If this is something you’re working on in your spare time, you don’t have a ton of it (that’s why it’s ‘spare’) and so it’s important to make it count. And it’s really easy to lose momentum on something that previous really excited you, especially if you’re super busy (like say, the world is in the middle of a pandemic, you’re working your full time job while doing virtual school with kids, and also generally trying to maintain some sanity). So in the interests of all of those things, I personally am a big believer in getting a project from start to finish in the shortest amount of time possible.

Of course, like all things in software and life, there are about a million caveats to that. The first caveat is that it has to be reasonable. To that end, you probably aren’t going to build Facebook in a weekend. This is where I made my first mistake.

via GIPHY

So here’s the thing: while I believe really strongly in taking an idea from start to finish as quickly as possible, it has to be reasonable and that means it has to be small. Take your idea for an app that does all the things and start trimming. When you think it’s reasonable, trim some more, and then trim even more. You know that rule where you 3x your estimate at work? Do that here too.

So if you could “probably” finish this thing in a weekend? It’s too big. Could you finish it on a Saturday before dinner? That’s the right size! There are going to be things you haven’t thought of, and styling is going to take longer than you anticipated, and so on. By getting your idea down to a single feature (or two, if you’re Wonder Woman) you’re setting yourself up for success. And if you are the unicorn everyone talks about who has perfect estimates? Well you either have some free time this weekend, or you can start on the next feature. Win-win!

So now that I’ve preached the good word of keeping MVPs (minimum viable products) as small as possible, let me regale you with all the ways I did not take my own advice.

via GIPHY

The original feature list included account creation (both in-app and social login), group creation and management, multiple calendar views, multiple permissions for views (a user deciding who can see a specific task), filters for viewing only certain group member’s tasks, include both a desktop and mobile view, and make it a PWA.

Phew, that is a lot of features. In hindsight, it’s easy for me to see that was never going to happen over a weekend, but I dream big. In the moment, it’s a much harder balancing act to both maintain the excitement you feel over a new project, as well as realistically determine what you can accomplish. I wish I had good advice for how to balance those two things, but the best I can really offer is to try to choose a project that you’ll use yourself. For me, that is Accounta-Billie, and knowing that it’s going to solve a problem I have is encouraging enough to keep me working on it.

I started off the project pretty well doing lots and lots of planning. I had everything broken down (I’ll write another post detailing how I handled the project management.), and I think that also helped keep me excited and focused. As long as I’m talking about planning, I want to mention Phil Schatz’s awesome project for adding extra functionality to a GitHub project board. It’s a wonderful and helpful project!

Over the first weekend, while I did not complete the first set of requirements, and honestly I still haven’t. A few of the features won’t be included in the beta, but will be added once the app moves to production. I ran into some troubles - especially with auth! It felt really good to get auth to completion, as well as deploying a project even though its incomplete. That’s right! It was deployed that first weekend, even without groups!

via GIPHY

Unfortunately auth did take a lot longer than I had originally intended, and so I wasn’t able to create multiple calendar views. I also didn’t finish the filters or different permissions layers. Ultimately I spent probably half of my working time on auth, which really makes my auth win that much greater!

For my next project or features, I know a lot better how to plan and what to plan so that I can actually complete whatever I’m working on in a weekend. Let me know what you thought of the post, and please go check out Accounta-Billie and sign up for the beta!

Hit me up on Twitter!

Accounta-Billie Part 1: Introducing Accounta-Billie

Fri, 28 Aug 2020 00:00:00 +0000

I wanted to take some time to write about a project I’ve been working on called Accounta-Billie. Accounta-Billie is a productivity and goals app. The concept is simple: you and your friends make up a group within the app. When one person in the group creates a task, such as “complete Accounta-Billie V1”, the whole group can see it. Once you’ve made this commitment to complete your task, your friends can help hold you accountable, and you can help hold your friends accountable to their goals as well.

It may seem counter-intuitive to try to make big goals (like building an app) during this time of COVID. I’ve seen tweets about how much people have “gotten done” during this time, and other tweets imploring people to “use this time wisely”. I find that kind of gross. Life is so much more than just checking items off a checklist. During a time like this, I really think we should just let everyone be. There’s no need to use this time to complete all of your life’s accomplishments if you’re having trouble right now - or any time for that matter! Life is too short to spend your time doing anything that you don’t want to be doing.

If you’re wondering how I got from the previous paragraph to “Let’s build an accountability app,” let me tell you a bit about my own quarantine experience. I started quarantine thinking, like a lot of people, that this wouldn’t last too long. I can remember thinking back in March that I only had to get to the summer, and then my kids would be in daycare, and I could I get some work done and also have my sanity back. And then when summer came and cases were worse, we made the decision to keep them home. I told myself that I only needed to get through the summer, and once school started back up, everything would go back to normal. Since then, the tough decision was made to keep my kids home and do virtual school. And even since that decision was made, my kids’ school district has decided not to have in-person classes at all.

During this never-ending (and also somehow always changing) quarantine, I adopted some pretty unhealthy habits, with the idea that I was just coping with a short-term situation. I justified it, telling myself that it was only until, only until, only until…. But the habits weren’t the kinds of habits that even made you feel better. Sure, they might have helped in the short term, but, if anything, they just compounded the depressed feelings even more in the long term. So once we decided to keep my kids out of school, I knew something had to change. In that moment, I suddenly snapped out of my short term thinking and started to accept this situation as a long term sort of “new normal.”

I wanted these new habits to start small, so my goal was to begin exercising regularly (of course, I didn’t know then what “regularly” meant to me), in hopes that exercise would help lift my spirits. I mentioned this to my sister, and she was really excited about the idea of us being accountability buddies. This made a lot of sense, both to help us each stick with our goals, but also because right now, we can’t see each other. We hoped that this would help keep us in contact.

But we couldn’t find an app that was quite right. She suggested the typical fitness apps, but they didn’t allow you to share tasks. I suggested a spreadsheet, but she said she probably wouldn’t use anything that she couldn’t access on her phone (she also called me lame for suggesting a spreadsheet). Since we couldn’t agree on what to use, naturally, the next step was to begin building my own app. I’ll detail everything over the next few blog posts, so stay tuned!

I hope you’ll check out Accounta-Billie. I’m looking for people for the beta, which is launching really soon. If you head on over to accountabillie.com, you can sign up there. I would love for you to use it, check it out, and then to get your honest feedback. Check back soon for my post about my first weekend working on Accounta-Billie and finally conquering auth.

Let me know what you think on Twitter!

How to manage navigation in Xamarin

Mon, 20 Apr 2020 00:00:00 +0000

As someone who loves JavaScript as much as I do, I never expected to be working in Xamarin. But that’s where I’ve been since I started my awesome new job! Xamarin brings some interesting challenges. You might think it’s just regular old C#, but since Xamarin uses MVVM architecture versus MVC architecture, I found a few things very tricky to manage. In this blog post, we’ll talk about how I managed navigation in a master detail app.

So in this app we used a master-detail app, which included a hamburger menu. The master property contained the hamburger menu, while the detail contained whichever partial view page was meant to be the current content. The code behind included a built-in Navigation property. You can use it like this:

await Navigation.PushAsync(new MyPage());

According to Xamarin docs, most or all of our app’s logic should be moved to the ViewModel. You can access the navigational properties via App.Current.MainPage.Navigation and push pages to the stack, as well as remove pages. However, I ran into a problem where the hamburger menu was disappearing when I would use App.Current.MainPage.Navigation. I realized I needed to find a way to actually update the Detail property itself. Unfortunately the Detail property doesn’t seem to be easily accessible. So I added a new constructor to the MainPage like this:

// MainPage.xaml.cs
// MainPage constructor
public MainPage(string page, params object[] args) {
    Type myPage = Type.GetType(page);
    Detail = new NavigationPage((Page)Activator.CreateInstance(myPage, args));
}

// FlowerPage.xaml.cs
// call MainPage page and pass in page, and any parameters
string type = typeof(FlowerDetailPage).AssemblyQualifiedName;
List<string> params = new List<string>();
params.Add("I am param");

App.Current.MainPage.Navigation.PushAsync(new MainPage(), params.ToArray());

This works well enough, however it seems a little yucky to keep adding new MainPages to the navigation stack. It seems like it would be better to directly update the Detail property, but how can we access it?

We ended up creating a navigational client with a method like this:

// NavigationClientManager.cs
public static async Task SetDetail(Type page, param object[] args) {
    MainPage mpage = ((MainPage)((NavigationPage)App.Current.MainPage).CurrentPage);
    Page detailPage = (Page)Activator.CreateInstance(page, args);
    mpage.Detail.Navigation.PushAsync(detailPage);
}

// FlowerPage.xaml.cs
// set the detail page
List<string> params = new List<string>();
params.Add("I am param");

await NavigationClientManager.SetDetail(typeof(FlowerDetailPage), params.ToArray());

This allows the ViewModel to simply call the SetDetail method, and avoid all of the casting of types in the nav client. It really cleans up the ViewModels and makes it very clear what is going on.

If you have another way of managing navigation in a master-detail app, I would love to hear it. Let me know what you think!

Shout out to my friend Jacob Danks who helped me come to this solution. Thanks Jacob!

Caches: Caches.match

Wed, 01 May 2019 00:00:00 +0000

Arguably, the biggest part of service workers is being able to fetch data off of a user’s machine, rather than making trips across the network. There’s a variety of reasons reaching over the network isn’t ideal – lack of connection, intermittent connection, speed of the network. Fortunately service workers help with all of these things.

In this post, we’ll look at caches.match – the function to fetch data from a service worker.

There are two matching methods within the Cache API – caches.match and cache.matchAll.

While caches.match requires at least one parameter, cache.matchAll has no required parameters. First, let’s dig into cache.matchAll.

cache.matchAll can only be run on a single cache. It has two optional parameters – request and options.

(async function() {
	var cache = await caches.open(cacheName);
	var rez = await cache.matchAll(request);
}())

If the cache contained a key-value pair matching that request, the associated response would be returned. If there were no match, rez will be undefined.

The steps look like this:

Check that the request exists as a key in the list

If it does, add it to a response list

Return list of responses

_{Source: Service worker spec}

You can call cache.matchAll without a request, and it will return all the responses in the specified cache.

So what’s the difference between caches.match and cache.matchAll?

caches.match must be run with a request parameter. Additionally it’ll check over each and every cache in the app. It might look like this:

(async function() {
	var rez = await caches.match(request);
}())

Unfortunately checking over each cache can result in it being a little slow, especially if there are multiple caches created for an app. It can be better to specify a cache and search it specifically, or even only have only a single cache. That would look really similar to our code up above with matchAll.

(async function() {
	var cache = await caches.open(cacheName);
	var rez = await cache.match(request);
}())