Hollerith you need Google

IBM 129 Data Punch
Test Card
A bitbucket ... for punched out bits :)

Inspiration

We noticed a large number of diseases and illnesses affect the elderly. Unfortunately, the elderly do not have access to the latest technology, or are often times stuck in the last century with regards to knowing how to use technology. We decided to build an interface for Google Assistant that those stuck in the 20th century could use without speaking and otherwise increasing their chances of infecting others.

What it does

We encode questions for the Google Assistant via IBM Punch Cards, and provide CV technology to read in cards using only a flatbed scanner and no other special hardware. When cards are read in, Google's Cloud Text-to-Speech allows us to interact with the Google Assistant on our phones to get Google Cards in response to our questions.

How I built it

I used my IBM 129 Card Data Recorder from 1973 to punch IBM Cards from Druckwerke Reichenbach (who unfortunately last made cards in the 80s). IBM 026, 029, and 024 card punches will also work, as will card duplicators and punches. You can even use the IBM Port-a-Punch for this! My 129 has a LED column counter, fibre optic card reader and duplicator, and 7kb of wire wrapped SLT memory, which made punching out sample cards for the demo and project easier than on a 029 or 026 machine.

Cards were read in using a HP M426fdw scanner at 300dpi, although any scanner will work for this. Afterwards, we used openCV to align images and extract punched holes using a pixel-averaging algorithm, which works quite reliably in reading in cards.

Having extracted the text, I queried Google's Text-to-Speech Cloud API to get a reliable way to interact with a listening Google Voice Assistant. We used the Wavenet model, which is quite realistic and handles queries well.

To simulate real life, we used a Nokia 8 phone running Android 9, which is a rather generic Android phone and shows the capability of our punch to speech translator.

We didn't have time to make a streamlined web app, so the video demo requires us to manually trigger each step, but writing the app would be trivial with more time.

Challenges I ran into

I acquired the IBM Punch in a nonoperative state, so fixing a 45 year old machine was rather challenging. However, it is fully operational now, and punches, duplicates, and verifies without issue.