Matt Dodge

Building a Wordle Bot In Under 100 Lines of Python

2022-01-26T15:14:00+00:00

If you don’t know what Wordle is, get out from under your rock and check it out. It’s a simple game at surface level but does end up needing some strategy to master it. It didn’t take more than a couple of rounds of playing it before my curiosity piqued and I felt the need to build a bot to play the game and learn more about it. Let’s get going!

The Objective

Build a Python program that plays Wordle. It should make guesses of words and try and figure out the ultimate answer word in as few guesses as possible.

The Basics

The basics of the game are pretty straightforward. We need to be able to keep a list of valid words, make guesses, collect responses, and update our word list based on the data in those responses. This high-level Python code can be the baseline of our solving application.

def solve():
    valid_words = ALL_WORDS
    while True:
        guess = make_guess(valid_words)
        print("Guess: " + guess.upper())
        result = collect_result()
        if result == CORRECT:
            print("I won!")
            break
        valid_words = update_valid_words(valid_words, guess, result)

We really only have 3 methods to build here with this simple structure. The first is make_guess. This method should take a list of valid words and guess a word out of that list. To start, let’s just choose randomly. We’ll make this smarter later. For now we just want to get the basic gameplay in place.

import random

def make_guess(valid_words):
    """ Given a list of valid words, make a guess as to the next word """
    return random.choice(valid_words)

The collect_result method just needs to collect some input from the user about the guess. The user of our program will input the guess into the Wordle website and that will tell us what is right and wrong. The user needs to input those values into our program. We’ll use 3 different characters to denote the 3 possible results for each letter:

_ = Miss - this letter does not appear in the word
? = Wrong Spot - this letter is in the word but not in this location
! = Correct - this letter appears in the word in this spot

Some straightforward Python user input collection along with some regex matching for data validation gets us on our way.

import re

def collect_result():
    """ Collect the result of our guess from the user """
    result = input("What's the result? (_/?/!) ")
    match = re.match(r'^[!?_]{5}', result)
    if not match:
        print("Invalid response string, try again")
        return collect_result()
    return result

Now we need to update our valid words based on our guess and the result, that’s what the update_valid_words method is for. We’ll start with a trivial implementation of this that isn’t very performant but should get the job done. We’ll iterate through all remaining valid words and see if they would have yielded the same result had we guessed our guess. In order to do this, we’ll need a new method called get_result that will return what the result for a guess would be if we knew the answer.

def get_result(guess, answer):
    """ Given a guess and a known answer, return what the result would be """
    result = ""
    for pos, ch_guess, ch_answer in zip(range(5), guess, answer):
        if ch_guess == ch_answer:
            result += "!"
        elif ch_guess not in answer:
            result += "_"
        else:
            result += "?"
    return result

def update_valid_words(valid_words, guess, result):
    """ Return a list of valid words based on previously valid words and a new guess/result """
    return [word for word in valid_words if get_result(guess, word) == result]

There are a few issues with this, specifically around words with duplicate letters (e.g. ROBOT, SPEED, etc). But we’ll sort those out later. At this point we have enough to play the game! We can put all of these methods together and play. Let’s see how our game plays with a target word of SNAKE.

$ python solve_wordle.py
Guess: WHISK
What's the result? (_/?/!) ___??
Guess: DUSKY
What's the result? (_/?/!) __?!_
Guess: SPOKE
What's the result? (_/?/!) !__!!
Guess: STAKE
What's the result? (_/?/!) !_!!!
Guess: SNAKE
What's the result? (_/?/!) !!!!!
I won!

It worked! In 5 guesses even, not too shabby considering the simple approach for a lot of those methods. Since we’re relying on a random choice here this will likely be different each time too, 5 seemed to be the median solve amount for SNAKE given this algorithm. Let’s see if we can do better.

Smarter Guesses

A good Wordle player is one who can guess a good word. Astute analysis, am I right? To start we just randomly picked a word from our remaining words, but that’s not very smart. What makes a good guess though?

My initial theory was that a good guess would be one that had the most common letters in it. This is a fairly safe assumption, especially early on in the guessing. With many words left we want to learn as much about them all as we can, so guessing common letters is a natural instinct. Surely a first guess of “RATED” would likely teach us more about the word than a guess of “JAZZY” would. Let’s enhance our make_guess method to employ this strategy.

To do this, the first step is to figure out which letters are the most frequent. We should only consider valid words for these character counts too, no sense counting words we know are not correct. Python’s built-in Counter object can help with this.

After we get the counts of each character in the remaining words, we want to find the words that have the most of these characters. We also don’t want to count duplicates here either. Remember, our goal is to find out about as many letters as possible. So while E may be the most common letter in words, it doesn’t make sense to guess EERIE. We don’t want to waste those other letter slots.

from collections import Counter

def make_guess(valid_words):
    # Given a list of valid words, make a guess as to the next word

    # Figure out the counts first

    counts = Counter()
    for word in valid_words:
        counts.update(word)

    # Now score the words based on their counts
    # This will be a list of tuples (score, word)
    word_scores = []
    for word in valid_words:
        unique_chars = set(word)
        word_score = sum([counts.get(c) for c in unique_chars])
        word_scores.append((word_score, word))

    # Sort the list and pick the highest score
    return sorted(word_scores, reverse=True)[0][1]

Not a terribly complicated method. We can probably make a few enhancements, like ignore letters that we already know are in the word, but this should do for now. Let’s play again against the word SNAKE and see if we do better.

$ python solve_wordle.py
Guess: LATER
What's the result? (_/?/!) _?_?_
Guess: SHADE
What's the result? (_/?/!) !_!_!
Guess: SUAVE
What's the result? (_/?/!) !_!_!
Guess: SPACE
What's the result? (_/?/!) !_!_!
Guess: SNAKE
What's the result? (_/?/!) !!!!!
I won!

We still got the word but it still took us 5 tries. Also, since our guessing algorithm is deterministic now, this will be the result every single time we play, at least for a target word of SNAKE.

One interesting observation here is that this algorithm thinks the best possible starting word is LATER (or any of its anagrams i.e., ALTER, ALERT). If you’re looking for a good starting word, these might be good ones to try.

This algorithm will get some words in just a few tries, but 5 tries doesn’t cut it for me. I can do that on my own, I want my computer to be smarter than me. Let’s do better.

Put This Machine To Work

Linear-time frequency counting isn’t much of an algorithm to write home about. We might even be able to roughly eyeball that ourselves as humans. If we think about what a good guess is, it is one that will tell us the most information based on the response. In other words, it is the one that will yield, on average, the smallest next list of valid words.

We have a computer in front of us, we can simulate this! For each valid word remaining, what would happen if we guessed it? We can run each valid word as a guess against every other valid word as an answer and see what the resulting word list would be. This comes out to an O(N^2) algorithm against a relatively small problem space of just a few thousand words. Let’s go!

def make_guess(valid_words):
    """ Given a list of valid words, make a guess as to the next word """
    guess = None
    lowest_worst_score = len(valid_words) + 1
    lowest_total_score = len(valid_words) ** 2
    for possible_guess in valid_words:
        worst_score = 0
        total_score = 0
        for possible_answer in valid_words:
            # reuse our old get_result method from before, nice
            result = get_result(possible_guess, possible_answer)
            num_new_valid_words = len(update_valid_words(valid_words, possible_guess, result))
            worst_score = max(num_new_valid_words, worst_score)
            total_score += num_new_valid_words

        # If this possble guess has a lower total/average number of words remaining, use it instead
        if worst_score < lowest_worst_score:
            guess = possible_guess
            lowest_worst_score = worst_score
            lowest_total_score = total_score

        # If it's a tie in average words remaining, choose the one that doesn't have the worst case
        elif worst_score == lowest_worst_score and total_score < lowest_total_score:
            guess = possible_guess
            lowest_worst_score = worst_score
            lowest_total_score = total_score
    return guess

This doesn’t look too bad, basically we’re just seeing how many words will be remaining after every possible guess and every possible answer. It will definitely be slower, but I think it will be smarter too. However, there’s a small gotcha here. We thought our algorithm would run in O(N^2) time, which isn’t too bad. But our rather simple implementation of update_valid_words comes back to bite us here. That method actually has its own O(N) runtime, so really this guess method will now run in O(N^3). It’s still a small data set, but cubic runtime gets big quickly.

Rather than trying to optimize either of those methods, let’s just make this strategy the one that runs once we’re down to less than 500 words in our valid word list. We can keep the frequency counting algorithm for longer word lists but once we’re closer to the end we’ll do the deep search.

def make_guess(valid_words):
    if len(valid_words) < 500:
        return make_guess_exhaustive(valid_words)
    else:
        return make_guess_frequency(valid_words)

Time to try it out against the word SNAKE again!

$ python solve_wordle.py
Guess: LATER
What's the result? (_/?/!) _?_?_
Guess: SHADE
What's the result? (_/?/!) !_!_!
Guess: SNAKE
What's the result? (_/?/!) !!!!!
I won!

Nice! Only 3 guesses! We might be getting a bit lucky here though. If we know the S, A, and E are in the right places then SNAKE, SUAVE, and SPACE should all be roughly equal. But this algorithm should still be more performant in the general case. Can we test that theory though?

The Hardest Wordle Word

At this point I got a bit curious, how would this bot do against other wordle words? I had spot-checked a few of the recent games and the bot was consistently beating my mere human brain, exactly what I wanted! Would this be the case for any word? Was there a theoretical maximum number of guesses a bot with this strategy would need to guess a word?

I decided to run a simulator, much like we do for the exhaustive guess algorithm. For every word in the word list, run the bot and see how many guesses it would take to guess it. The code for this wasn’t really challenging, but it’s a bit beside the point so I won’t share it here.

As expected, most of the words were solved in 2 or 3 guesses. The occasional 4 would slip in there too. But there was one word that took 8, yes 8, guesses to get. Without further ado, I present to you the hardest possible Wordle word:

CATCH

Catch? Really? Why is that hard? Pretty easy word with pretty common letters. What’s going on here? Let’s play with our bot and see what happens.

$ python solve_wordle.py
Guess: LATER
What's the result? (_/?/!) _!!__
Guess: FATTY
What's the result? (_/?/!) _!!?_
Guess: HATCH
What's the result? (_/?/!) _!!!!
Guess: BATCH
What's the result? (_/?/!) _!!!!
Guess: MATCH
What's the result? (_/?/!) _!!!!
Guess: WATCH
What's the result? (_/?/!) _!!!!
Guess: PATCH
What's the result? (_/?/!) _!!!!
Guess: CATCH
What's the result? (_/?/!) !!!!!
I won!

Interesting! The bot quickly got to the _ATCH suffix, but then was stuck spinning its wheels trying to guess the first letter. Because it didn’t have that much information, it was really only able to guess one letter at a time and hope that it was right. If we could somehow learn about more than one letter here we could probably solve it faster.

And there it is, a truly marvelous realization after I looked at this for a while:

It is sometimes advantageous to guess a word that you know is wrong in order to make a better guess on your next word.

The rules just state the guess has to be a real word, not a word that is valid given the current set of known information. In other words, if we have 6 words that end in _ATCH left in our list, don’t just guess them one by one. Instead, guess another word that isn’t in the remaining valid word list to try and get as much info as possible.

What does this look like in code? It’s surprisingly easy. Instead of iterating through valid words looking for possible guesses, we’ll iterate through all words. This will have an obvious performance hit, so we’ll only run this when we’re down to 50 words or less.

Here’s the relevant part of our make_guess_exhaustive method now

def make_guess_exhaustive(valid_words):
    guessable_words = ALL_WORDS if len(valid_words) < 50 else valid_words
    for possible_guess in guessable_words:
        # rest of the method goes here...

Now what happens if we run against CATCH?

$ python solve_wordle.py
Guess: LATER
What's the result? (_/?/!) _!!__
Guess: CRIMP
What's the result? (_/?/!) !____
Guess: CATCH
What's the result? (_/?/!) !!!!!
I won!

Look at that! After finding out that the A and T were correct, the bot guesses CRIMP with zero intention of winning but instead gathering information. As a result, it knows which letter is the first letter and is able to guess CATCH correctly on the 3rd try. Much better than the 8th guess like before. I call that a win!

Wrap Up and Other Enhancements

This was a fun little experiment and weekend project. I think it also demonstrates how an idea can start small and with some pretty simple pseudo-code and eventually turn into a rather complex and impactful algorithm. The code is still concise, but it ends up getting the job done.

There were a few gotchas I discovered along the way while building this that I didn’t get into too much here, for the sake of keeping the article focused. The final code, shared below, includes these enhancements and realizations though.

Multiple Letters - Wordle is kind of weird (smart?) about how it handles multiple letters in a word. If the answer is ROUND and your guess is ROBOT, the first O would be green (right spot) but the second O would be blank, rather than yellow. This is a bit of a gotcha and needs to be accounted for in the code
Concept of “knowns” - The update_valid_words method’s algorithm and performance wasn’t satisfactory to me. I ended up implementing a concept of “knowns” about the answer word. These are basically things that we know about the word, where letters can and can’t be, what letters are missing, etc. In the final code you’ll see these knowns references flying around.
Word list - If you’ve played Wordle you will notice that most words are pretty common. Most of the examples here only referenced a common word list (2,314 5-letter words). The list of 5-letter words in my /usr/share/dict/words file was 9,330, a much larger list.

To check out the finished code for this project, head on over to the mattdodge/wordle_bot GitHub repo. Feel free to fork, report issues, and/or submit PRs there.

The ultimate file ended up being over 100 lines of code, but that’s mainly because it includes the enhancements and other features I mentioned here.

I hope you enjoyed this post and this journey of making a fun little bot for a fun little game. Of course, don’t use this bot to cheat, that’s not cool. But learning and tinkering with Python for real-world projects like this is always fun - happy Wordle-ing!

The Hardest and Easiest New York Times Crossword Puzzles of 2021

2022-01-01T15:14:00+00:00

2021 is officially behind us. Let’s take a look back at the 365 daily New York Times crossword puzzles from the year and see which ones were the hardest and which ones were the easiest. Using the crowd-sourced solve times from hundreds of solvers at XW Stats, I was able to rank the puzzles in terms of difficulty. I’ll describe the method below, but for you with short attention spans here are those rankings, along with links to the puzzles at XWordInfo.

Hardest Puzzles of 2021

Here are the 5 most difficult/slowest puzzles of 2021, based on comparing solve times to a solver’s average solve time on that day of the week.

Friday, Dec 10 - Joe DiPietro, Themeless (68.1% harder)
Friday, Mar 19 - Kameron Austin Collins, Themeless (60.3% harder)
Wednesday, Oct 6 - Jules Markey, Pigeon Holes (49.9% harder)
Tuesday, June 8 - Christopher Adams, Added H (49.4% harder)
Sunday, May 16 - Joe DiPietro, A Shot in the Dark (48.2% harder)

Joe DiPietro, a veteran constructor with hundreds of published puzzles, appears twice on the list. His Friday December 10 puzzle had some tricky crosses and difficult cluing. 19% of XW Stats solvers needed to cheat on that puzzle, compared to the 10% average cheat rate on Fridays. Jules Markey clearly caught the solvers off-guard with the rare Wednesday rebus puzzle.

Easiest Puzzles of 2021

Here are the 5 easiest/fastest puzzles of 2021, based on comparing solve times to a solver’s average solve time on that day of the week.

Thursday, Nov 18 - Ori Brian, First Second Middle… (46.2% easier)
Thursday, Sep 16 - Kevin Patterson, Silver Lining (45.8% easier)
Friday, Oct 29 - Aimee Lucido, Themeless (43.69% easier)
Sunday, Dec 5 - Chase Dittrich and Jeff Chen, Come Again? (43.67% easier)
Wednesday, Oct 27 - Johanna Fenimore, SNL (43.5% easier)

A couple of Thursdays lead the easy list this year. Neither involved any kind of rebus or typical Thursday trickery which probably led to their faster than average solve times. Johanna Fenimore’s inclusion of several famous Saturday Night Live quotes also seemed to be a crowd favorite.

Calculation Method

To compute these lists, I define a puzzle’s difficulty to be how much longer a solver takes to solve that puzzle compared to his/her average on that day of the week. Let’s get into some specifics.

The first concept to understand that drives this calculation is something I call “Adjusted Solve Time.” You can read more about Adjusted Solve Time on the XW Stats help page, but it is basically an adjusted time it takes to solve a puzzle that takes into account any squares that were checked or revealed. This metric adds a time penalty for checking and revealing squares so that puzzles that require “cheating” show up as harder than those that do not.

Once I have a solver’s adjusted solve time, the goal is to get their “average solve time” too. This value is intended to represent an individual solver’s average time to solve a puzzle on that day of the week. Monday puzzles are obviously easier than Thursdays, so a solver will have different averages for each. But solvers also sometimes get better at puzzles over time. It is relatively safe to assume that a solver, especially a newer one, is probably solving puzzles faster at the end of the year than they were at the beginning of the year. To account for this, the average solve time metric uses a 10 week rolling average for each day of the week. In other words, a solver’s average solve time on a random Tuesday is the average of their solve times on the previous 10 Tuesday puzzles before that.

Now I can compare a solver’s adjusted solve time for a given puzzle and compare it to their “expected” adjusted solve time (their 10 day average) for that puzzle. I can do this for every solver and for every puzzle throughout the year, then attempt to aggregate this data together. There are some different ways to do this, but I ended up settling on taking the median percentage difference of adjusted solve times for each puzzle. Median makes more sense than mean, as the latter can be heavily skewed by outliers or strange performances. I opted for median over a different percentile (e.g., 75th percentile) because I believed it would best capture the experience of both faster and slower solvers.

So, putting this all together gives me percentages of how much slower/harder or faster/easier a puzzle was. When I say that Joe DiPietro’s puzzle was 68.1% harder, that means that the median solver took 68.1% longer to solve that puzzle than they did the 10 previous Friday puzzles before that. This includes the time penalties for checking/revealing.

Raw Data

If you’d like to conduct some of your own analysis, I’ve provided some of the raw data here. This data has been normalized and anonymized.

Crossword Difficulty 2021 Full Data.csv

Here is a brief description of each field in the CSV:

Puzzle Date - the date the puzzle was published
Constructor - the puzzle’s constructor
Day of Week - the day of the week the puzzle was constructed
Cheat Pct - the percentage of solvers that checked or revealed a square in the puzzle
Adjusted Mean - the average/mean of all adjusted solve times compared to average adjusted solve time
Adjusted Median - the median of all adjusted solve times compared to average adjusted solve time
Adjusted P75 - the 75th percentile of all adjusted solve times compared to average adjusted solve time
Adjusted P90 - the 90th percentile of all adjusted solve times compared to average adjusted solve time
Actual Mean - the average/mean of all non-adjusted solve times compared to average solve time
Actual Median - the median of all non-adjusted solve times compared to average solve time
Actual P75 - the 75th percentile of all non-adjusted solve times compared to average solve time
Actual P90 - the 90th percentile of all non-adjusted solve times compared to average solve time

FiveThirtyEight Riddler - Prolog to the Rescue!

2019-02-22T15:14:00+00:00

One of my weekly routines is to solve the FiveThirtyEight Riddler puzzle. If you’ve never done them before, they are a set of weekly riddles that tend to involve math, logic, and statistics. Sometimes, though, they make you think. That’s silly, thinking is what computers are for!

This week’s puzzle is a perfect example of a puzzle that can be solved with one of my favorite programming languages, Prolog. I believe that in order to be a good software engineer or computer scientist you must be versed in many different programming languages. There simply isn’t one “best” programming language, they come in all shapes and sizes and some are better than others for a certain problem.

One rather niche programming language is Prolog. If you’ve never seen it or worked with it before I highly recommend checking out some tutorials and getting familiar with the basics. Essentially, it’s a language that allows you to define a set of rules within a problem space and then find the answers, if they exist, that satisfy all of those rules.

In this post, I’ll walk through how we can use Prolog to quickly and succinctly arrive at an answer for this week’s Riddler puzzle. I’ll be using SWI-Prolog for this, but I believe it will work in any flavor of Prolog that you choose.

Problem Statement

First, let’s take a look at this week’s puzzle description:

We are marooned on an island that has the following curious property: Everyone over a certain age lies all the time. More specifically, there is an age limit L — a positive integer — and all islanders who are younger than age L only tell the truth, while islanders who are at least L years old only tell lies.

We are greeted by five islanders who make the following statements:

A: “B is more than 20 years old.”

B: “C is more than 18 years old.”

C: “D is less than 22 years old.”

D: “E is not 17 years old.”

E: “A is more than 21 years old.”

A: “D is more than 16 years old.”

B: “E is less than 20 years old.”

C: “A is 19 years old.”

D: “B is 20 years old.”

E: “C is less than 18 years old.”

What is L? And what did we just learn about the ages of the islanders?

So basically we have a group of 5 islanders with certain ages. Some of them are liars and some are not, and they are giving us facts about the other islanders’ ages. These “facts” make great rules in a Prolog program. Let’s get to work.

Starting Small

We’ll start off with a very small subset of this puzzle. I’m going to show some Prolog syntax below. If you’ve never seen it before it may look a little strange, I encourage you to read through this wiki tutorial first, it will give you the basics.

I’m going to start with some simple claims about peoples ages.

% Claim that a Person is older than Age
claim_older(Person, Age) :- Person > Age.

% Claim that a Person is younger than Age
claim_younger(Person, Age) :- Person < Age.

This sets up 2 types of “claims” that we can make about an islander’s age. A claim that a person is older than an age (claim_older) is true only if that person’s age is greater than the specified age. A similar rule is declared for younger. We can load this Prolog program into our interpreter and start making queries against it.

Note: We need to specify some “bounds” for the person’s age, otherwise Prolog won’t know how to assign someone’s age and will try infinite possibilities. This is the purpose of the between fact in our query.

First, we’ll bound A’s age between 10 and 40 (reasonable boundaries given the problem statement). Then we’ll specify a claim that A is older than 15 and a claim that A is younger than 18. Prolog will tell us possibilities for how old A is.

?- between(10, 40, A), claim_older(A, 15), claim_younger(A, 18).
A = 16 ;
A = 17 ;
false.

Prolog has determined that A is either 16 or 17 years old - that makes sense! When querying in the interpreter, you can press ; to see more possible results. In this case, after we hit the 17 result there were no more valid options, so Prolog returned false.

What happens if we give it an impossible claim like A is older than 18 and younger than 15?

?- between(10, 40, A), claim_older(A, 18), claim_younger(A, 15).
false.

We can even specify claims about multiple people - this time the query is broken up into multiple lines to make it easier to read:

?- between(10, 40, A),
   between(10, 40, B),
   claim_older(A, 15),
   claim_younger(A, 17),
   claim_older(B, 35).
A = 16,
B = 36 ;
A = 16,
B = 37 ;
A = 16,
B = 38 ;
A = 16,
B = 39 ;
A = 16,
B = 40 ;
false.

Prolog has spit out all of the possible pairs of ages that are valid for this scenario, neat!

The Invention of Lying

In our first runs we were able to make claims but we always assumed those claims were true. Per our problem statement, claims are only true if the claimant is under a certain age. Let’s define some additional rules for when a claim is made but the claimant is lying.

% A claimant is a liar if they are at least a certain age
% We'll hard code this to 20 years old for now
liar(Claimant) :- Claimant >= 20.

% We make two declarations for an older claim, one from a lying
% claimant and one from one telling the truth
claim_older(Person, Age, Claimant) :-
    liar(Claimant),
    Person =< Age.
claim_older(Person, Age, Claimant) :-
    not(liar(Claimant)),
    Person > Age.

claim_younger(Person, Age, Claimant) :-
    liar(Claimant),
    Person >= Age.
claim_younger(Person, Age, Claimant) :-
    not(liar(Claimant)),
    Person < Age.

run(A, B) :-
    between(10, 40, A),
    between(10, 40, B),
    claim_older(A, 18, B),
    claim_younger(A, 15, B),
    claim_younger(B, 21, A).

There are a few things to notice about our program now.

We are hard-coding the age of lying to 20 for now, we’ll variablize this soon.
We’ve added the ability for a claim to have a Claimant as well - this is the person, potentially a liar, making the claim.
We’ve added a second set of rules for each claim, now with one that supports a liar. Notice the inverted comparison operators in each set of claims. When Prolog looks through its rules it will find one that matches. We can create an “or” condition by specifying multiple possibilities for each claim - one for a lying claimant and one for one who tells the truth.
We have added our claims and bounds to a run rule, this saves us some repetitive typing in the interpreter. Now we can just query run(A, B) and see the results. Let’s do that now!

?- run(A, B).
A = 15,
B = 20 ;
A = 16,
B = 20 ;
A = 17,
B = 20 ;
A = 18,
B = 20 ;
false.

Prolog has given us some pairs of ages that are valid given the rules we provided. The possibilities boil down to 15 <= A <= 18, and B = 20. If we look back to our original claims we basically had B claiming that A is older than 18 and younger than 15. Since that is impossible, Prolog determined that B must be a liar. But since liars are only at least 20 years old (since we hard coded that rule), B must be at least 20. However, we also had A claiming that B was younger than 21. Since we know A is between 15 and 18 (since B was a liar), we know A must be telling the truth (younger than 20). So B indeed is younger than 21 and at least 20, or, in other words, B is 20.

Sexier Ranges

Bet that’s the first time you’ve seen those two words next to each other.

Prolog is great at giving us multiple solutions to the problem. But we’re going to be working a lot with ranges of ages. For the previous example, it would have been great if rather than iterating through every possible set of ages Prolog could have told us the range of ages that were possible in one go. Fortunately, Prolog provides a way to define “constraints” for values using something called Constraint Logic Programming. Let’s adapt our previous code to use constraints instead now.

:- use_module(library(clpfd)).

liar(Claimant) :- Claimant #>= 20.
not_liar(Claimant) :- Claimant #< 20.

claim_older(Person, Age, Claimant) :-
    liar(Claimant),
    Person #=< Age.
claim_older(Person, Age, Claimant) :-
    not_liar(Claimant),
    Person #> Age.

claim_younger(Person, Age, Claimant) :-
    liar(Claimant),
    Person #>= Age.
claim_younger(Person, Age, Claimant) :-
    not_liar(Claimant),
    Person #< Age.

run(A, B) :-
    A in 10..40,
    B in 10..40,
    claim_older(A, 18, B),
    claim_younger(A, 15, B),
    claim_younger(B, 21, A).

Check out what’s new

We import the clpfd module at the top of our program rules now, to get access to the constraint logic predicates.
We prefix our comparison operators with the # symbol. So it’s Person #< Age now instead of just Person < Age. These operators are documented here if you’d like to read more about them.
We added a not_liar rule instead of using not(liar). This is most likely due to my lack of complete understanding of Prolog, but I kept having trouble inverting the CLP constraint operators so I wrote it out explicitly.
Use A in 10..40 instead of between(10, 40, A) in our run rule.

Now check out our output when we run — much more readable!

?- run(A, B).
B = 20,
A in 15..18 ;
false.

Variable Liars

Up until now, we’ve been assuming that everyone over 20 is a liar. In reality, the age of that lying threshold is something we need to solve for as well. We’ll do this by making another variable for the lying age and pass that through with our claims.

:- use_module(library(clpfd)).

liar(Claimant, LieThreshold) :- Claimant #>= LieThreshold.
not_liar(Claimant, LieThreshold) :- Claimant #< LieThreshold.

claim_older(Person, Age, Claimant, LieThreshold) :- 
	liar(Claimant, LieThreshold),
	Person #=< Age.
claim_older(Person, Age, Claimant, LieThreshold) :- 
	not_liar(Claimant, LieThreshold),
	Person #> Age.

claim_younger(Person, Age, Claimant, LieThreshold) :- 
	liar(Claimant, LieThreshold),
	Person #>= Age.
claim_younger(Person, Age, Claimant, LieThreshold) :- 
	not_liar(Claimant, LieThreshold),
	Person #< Age.

run(A, B, L) :-
	A in 10..40,
	B in 10..40,
	L in 10..40,
	claim_older(A, 18, B, L),
	claim_younger(A, 15, B, L),
	claim_younger(B, 21, A, L).

We’ve pretty much just added an argument to each of our rules that will keep track of the lying age threshold. We can test this by running again with our known threshold of 20.

?- run(A, B, 20).
B = 20,
A in 15..18 ;
false.

Cool, same result, that makes sense! But what if we want a variable lie threshold?

?- run(A, B, L).
A in 15..18,
A#>=L,
L in 10..18,
B in 21..40 ;

A in 15..18,
A#=<L+ -1,
L in 16..20,
B#>=L,
B#>=L,
B in 16..20 ;
false.

This syntax is a little weirder but it makes sense if you parse it out. I won’t go into too much detail (it won’t really matter when we plug in our actual claims) but here’s the gist. With a completely variable lie threshold (well, between 10 and 40 per our constraint) there are two solution possibilities. We know B is a liar because of the impossible claim. But what if A is a liar too.

Tying it all Together

We’ve got a pretty slick claiming system in a rather succinct format (imagine writing this code in another language!), now it’s just time to fill out the actual claims from the problem statement and see what our solution is! (obviously, spoilers ahead…)

:- use_module(library(clpfd)).

liar(Claimant, LieThreshold) :- Claimant #>= LieThreshold.
not_liar(Claimant, LieThreshold) :- Claimant #< LieThreshold.

claim_older(Person, Age, Claimant, LieThreshold) :- 
	liar(Claimant, LieThreshold),
	Person #=< Age.
claim_older(Person, Age, Claimant, LieThreshold) :- 
	not_liar(Claimant, LieThreshold),
	Person #> Age.

claim_younger(Person, Age, Claimant, LieThreshold) :- 
	liar(Claimant, LieThreshold),
	Person #>= Age.
claim_younger(Person, Age, Claimant, LieThreshold) :- 
	not_liar(Claimant, LieThreshold),
	Person #< Age.

claim_equal(Person, Age, Claimant, LieThreshold) :-
	liar(Claimant, LieThreshold),
	Person #\= Age.
claim_equal(Person, Age, Claimant, LieThreshold) :-
	not_liar(Claimant, LieThreshold),
	Person #= Age.
	
claim_not_equal(Person, Age, Claimant, LieThreshold) :-
	liar(Claimant, LieThreshold),
	Person #= Age.
claim_not_equal(Person, Age, Claimant, LieThreshold) :-
	not_liar(Claimant, LieThreshold),
	Person #\= Age.

run(Islanders, L) :-
	Islanders ins 10..40,
	L in 10..40,
	[A, B, C, D, E] = Islanders,
	claim_older(B, 20, A, L),
	claim_older(D, 16, A, L),
	claim_older(C, 18, B, L),
	claim_younger(E, 20, B, L),
	claim_younger(D, 22, C, L),
	claim_equal(A, 19, C, L),
	claim_not_equal(E, 17, D, L),
	claim_equal(B, 20, D, L),
	claim_older(A, 21, E, L),
	claim_younger(C, 18, E, L).

A few quick things to note about the changes here:

I’ve converted the run arguments to a list of islanders now, rather than explicitly saying A, B, etc. Also, notice the ins syntax for a list rather than the in syntax for a single variable.
I’ve added claims for “equal” and “not equal” ages. I probably could have gotten away with re-writing D’s claim that E’s age is not 17 in a different way, but I figured I’d be explicit.
I’ve loaded in the 10 claims from the problem statement.

And now if we run it, with our ages and lying threshold as variables, we will see our solution:

?- run([A, B, C, D, E], L).
A = L, L = 19,
B = 20,
C = 18,
D in 10..16,
E in 20..40 ;
false.

There you have it! The age threshold is 19, making A (age 19), B (age 20) and E (age >= 20) all liars. C (age 18) and D (age <= 16) are truth tellers.

I hope this post has at least minimally encouraged you to go explore some other programming languages or to scrounge up some interest in Prolog. If you’re curious or want to explore more, the completed and commented code from this tutorial is available on a GitHub gist here. Also feel free to reach out to me with any questions, suggestions, or Prolog tips!

Creating and Publishing My First Crossword

2019-02-19T15:14:00+00:00

My latest hobby is solving crossword puzzles. I’ve been doing the New York Times puzzle daily for the last 8 months or so and absolutely love it. I am constantly impressed and amazed by what the creators of these crosswords can do — build a complete and connected puzzle while still integrating humor, trivia, creativity, and puns.

I’ve been so impressed, in fact, that I decided to give it a go myself. What goes into building — colloquially referred to as “constructing” — a crossword puzzle? Can I do it myself? Can I end up constructing a puzzle that makes its way into the New York Times, the gold standard of crosswords? Let’s find out!

Puzzle Construction Goals

My first step in the process was to set up some guidelines. After completing enough crosswords I learned things that I liked seeing in puzzles and things that I didn’t.

No “double whammies” - I never knew what these were called but I would see them occasionally. I call it a double whammy when two clues cross that involve somewhat obscure “trivia” that you either know or you don’t. I’ve since learned this is referred to as a “Natick” in the crossword community. They can be annoying when you’re solving and I wanted my puzzle to avoid them.
No “auto-fill” help - Probably the most obvious challenge in constructing a crossword is getting all of the letters (known as the “fill”) to line up and create words in both directions. Perhaps unsurprisingly, there are computer programs that can help build puzzles that have clean fill. I wanted to avoid using these and slog through it the old fashioned way.
Clean fill - This is a bit harder to explicitly quantify but I really wanted my puzzle to not have anything too obscure or lucky. After doing a few puzzles you realize that there are many clues that end up being somewhat “convenient” for the constructor. I wanted to make sure every clue of my puzzle was there deliberately and not just there because I got lucky that LEV happens to be the currency of Bulgaria, for example.
Have fun - I did this to have fun and to learn - I am not trying to make a career out of crossword constructing. I needed to remind myself of this goal several times throughout the process when I started to get frustrated.

Lastly, before I get into too many details, this post is going to detail my process of constructing an entire puzzle. There will obviously be “spoilers” ahead. If you’d like to solve my puzzle first, you can find it here, in several formats: [AcrossLite] [PDF] [Solution]

The Theme

Most NYT crossword puzzles are centered around some type of “theme.” These themes range from clues that contain colors to Denzel Washington movie quotes and even to “rebuses” - boxes that can contain more than one letter or a non-letter. Virtually all constructors agree that the first step to building a themed puzzle is to come up with your theme idea and the theme clues. Theme clues tend to be the longest answers in a puzzle and thus are the hardest to place, so you want to get them out of the way first.

I actually started down this constructing journey while I was sitting on the beach during my Babymoon in Hawaii. And being a California boy, a beach theme seemed appropriate. Looking around for inspiration, I saw a happy little kiddo building a sand castle and wondered whether I could incorporate that into my puzzle. SANDCASTLE is 10 letters long, a reasonable length for a theme clue, and also has some alternate meanings. SAND can be a verb, and CASTLE can refer to a chess piece. Maybe I could make this work.

The beach where the construction inspiration struck

Theme clues need to be consistent across the puzzle, so I started trying to come up with other two-word beach-related clues that could have alternate meanings. I ended up jotting down 5 total theme clues, knowing I would probably only use 3 or 4, that met these criteria.

SANDCASTLE (10 letters) - could also refer to sanding down a chess piece
CATCHRAYS (9 letters) - could also refer to catching baseballs from Tampa Bay Rays players
OPENUMBRELLAS (13 letters) - could also refer to opening up umbrella insurance policies
SURFBOARDS (10 letters) - could also refer to surfing the web and browsing message boards
COASTLINE (9 letters) - could also refer to coasting on a bike on a straight line

I also may need to have a “theme revealer” in my puzzle. A theme revealer is a clue that tends to be towards the bottom right of a puzzle that ties together all of the theme entries. For mine, a simple revealer of BEACH (5) (“Locale of the clues of this puzzle”) would probably suffice.

I also thought it’d be fun to incorporate BABYMOON (8 letters) into my puzzle if I could make it work, given the fun-ness of the term and its relevance to my puzzle. We’ll see…

The Grid

Now that I had my theme clues it was time to start laying out the structure of the puzzle. The grid basically refers to the placement of the black squares in the crossword puzzle. That ends up defining word lengths for every answer and where the theme answers will go as well.

I didn’t think this would be that difficult of a task but it ended up being one of the hardest parts of the entire process. What you have to know before doing this is that there are a few simple rules about a “valid” crossword grid:

Diagonal Symmetry - All (or almost all) crosswords are symmetric across the diagonal - this means if you put a black square in the top-left corner, one has to go in the bottom-right corner as well.
No 1 or 2 letter words - The minimum word length for an answer is 3 letters - seems obvious, right? This ends up being trickier than you’d think because a lot of times you want to terminate a word going across but that causes a 1 or 2 letter word going down.
No more than 78 words - Tossing black squares all over the place makes a bunch of short words, that’s a no-no.
Grid connectivity - The entire grid must be connected, you can’t close off one section from another.

Tools

This seems like a good time to discuss the tools I used for constructing this sucker. I started off drawing by hand. That lasted about 5 minutes before I realized how many times I’d be erasing. I then moved on to Google Sheets but was getting frustrated with formatting and dealing with symmetry. I then found a cool free tool online called Phil. This kept track of symmetry for me and would also let me fill my clues in later. I ended up using Phil for most of the grid construction phase but ultimately sprung for CrossFire - a full-featured app that has a bunch of features including auto-fill (which I won’t use—see rule #2 above!), word counts, and output formats. If you’re serious about constructing I recommend using a comprehensive tool like CrossFire or Crossword Compiler - it makes your life so much easier.

Due to the symmetric constraints, it’s imperative to have symmetric theme entries too. Fortunately, two sets of my theme entries had the same number of letters. SANDCASTLE and SURFBOARDS both with 10, CATCHRAYS and COASTLINE both with 9.

Biting off more than I could chew

I also had a 5th theme entry of OPENUMBRELLAS which had 13 letters and no partner. I started off trying to include this, but since there’s no clue to use symmetrically, it had to go in the center of my grid. However, since it’s 13 letters, and the grid is 15 letters wide, that forces 1 black square on each side of it.

This was already looking weird with those two black squares. I tried to cram my other theme entries into the puzzle but it got crowded quickly and it was quite difficult (at least for a rookie like me) to create a good grid. I ended up getting them all to kind of fit and adhere to the rules with the following setup.

I was so excited once it all actually fit and validated but then I quickly started noticing a lot of problems. First of all, I had two 15-letter words I would going to have to work in there now (3-D and 11-D). Not only did that seem difficult, but your theme entries should also typically be the longest words in your puzzle. There are also some other patterns that would undoubtedly prove to be problematic later — the 6-letter word ending in U at 5-D — two 10-letter words that will have to fit between a 15-letter and a 7-letter word at 2-D and 31-D. Plus it was just kind of an ugly grid overall.

I spent at least 3 hours trying to work this grid — it was way more difficult than I imagined it would be. As painful as it was, I ended up scrapping it and starting over, this time not expecting to get my OPENUMBRELLAS clue in the puzzle.

Round 2

Starting over on the grid felt like a step backward but it was much smoother the second time through. At first, I was really trying to work a BEACH revealer in there too, but wasn’t having much luck. Adding any kind of 5 letter word into a grid where your theme answers are 9 and 10 letters long ended up being a big challenge. Just like the umbrella clue, I abandoned the idea of trying to get BEACH in there…for now at least…

I did end up moving my theme answers to be more on top of each other—this prevented me from having a lot of long words which I figured would make my life easier in the future. After about 30 minutes I ended up with this grid that I think will work well enough.

Grid is complete!

The Fill

Now that the grid was ready came the “fun” part—completing the rest of the letters of the puzzle so that everything is a word. Constructors call this the “fill” because it fills in the rest of the puzzle. There are a few rules about the fill, of course. I have my personal goals like no Naticks and common knowledge only. There is also no profanity allowed and no duplicate clues either.

This was the step that I assumed would be the hardest part of the entire process. I was quite correct in that assumption. Within the first few minutes I realized I was in for quite the challenge.

The longest remaining words in my puzzle are the two 10-letter words at 11-D and 28-D. I figured I’d start there so I started tackling the left-center and bottom-left sections first. I didn’t really know exactly where to start so I just looked up 10 letter words starting with A (since it goes under COASTLINE). There is, unsurprisingly, a lot of them.

I tried to choose words that were fun or would let me be creative with the clues. My method was pretty much to try to plug a word in, fill in around it until I got stuck, and then start over. This took a while and got pretty demoralizing. I was also only doing this by hand right now, no tools or word lists involved yet. I needed a better strategy.

Word Lists

I would often encounter a situation like the following: I need a 5 letter word that ends in A, starts with a vowel, and has a consonant as the 2nd to last letter. Fortunately, my knowledge of grep, a simple command-line tool to parse text with regular expressions, and a local dictionary file with all of the English words helped a bit. The problem with this approach though was that I would ignore any proper nouns (OBAMA), abbreviations (ASPCA), or compound words (UPONA). This is quite a large list to be ignoring. I needed to find a list of all possible words for a crossword - to Google I went!

It turned out I wasn’t the only constructor with this problem. There’s actually a whole economy of fellow crossword constructors selling their word lists. They even “score” the word lists so you can prioritize better/more fun words over other less common words. I wasn’t ready to spring for a premium word list quite yet, but it did make me realize that if you wanted to be a serious constructor, the word list you work off of is probably the most important tool in your toolbox.

If you want to use any of these scored word lists though then you’ll need some robust crossword construction software. As I mentioned earlier, I ended up downloading a Mac app called CrossFire. They offer a free trial that includes a word list as well as some cool lookup features. It ended up working great for me so I shelled out the 50 smackers to get the full version which lets you save progress and work for more than 1 hour.

I also stumbled across an amazing website called XWordInfo which contains a premium word list maintained by legendary constructor Jeff Chen. His clue lookup tool along with my dictionary grepping and CrossFire word list would get me on my way.

Fill Progress

I ended up getting the left-center and bottom-left sections filled in, working off the (kind of lame) word ASSOCIATED, so I moved on to the bottom-center. Then I got completely stuck. I tried and tried and tried to work clues in there but couldn’t ever get anything to fit.

I had sunk hours into getting that bottom left section to work, so going back and redoing it sounded dreadful, but I didn’t think I had any other choice.

After erasing and trying out some other options I had a bit of a breakthrough. I had been relying pretty heavily on words that existed in my word lists and in previous puzzles. There is no rule that says a word has to have existed before though—creativity is allowed! I ended up using YEAHDUH (7) going across the top of the bottom-left section. This is an NYT debut word and a fun one that could be clued with something like “Sarcastic response from a teenager.” It wasn’t easy, but after about 30-40 more minutes I had a fill on the bottom-center that worked with the whole left side. Success!

I also had a realization midway through working that bottom section. I had left out my BEACH revealer clue but there was a way for me to work in a related revealer at 34-D with something like BEACHBALL or BEACHBAGS. I ended up going with BEACHBODS, a fun clue that is also an NYT debut. I could clue with something like “Physiques seen at the locale of the themed clues of this puzzle.” Things were starting to come together!

I’m not sure if it was coincidence or if I was just getting better at navigating the word lists, but the rest of the puzzle filled in a lot smoother than the bottom left. I would never describe it as easy, but I didn’t have any major regressions or problems. After a few more hours of grepping, Googling, searching, filling, and crying, I had a complete fill.

The Clues

The last step in the crossword construction process is writing the clues. This is a pretty fun step and is where the constructor’s creativity really gets to shine. In order to properly write the clues though, you have to know what level of difficulty you are expecting your puzzle to be. For example, the word APPLE could be clued as “iPhone Maker” for an easy puzzle but clued as “Jobs creation” for a harder puzzle.

To decide what difficulty you want your puzzle to be it’s important to understand the general levels of difficulty of the publisher you are submitting to. For the New York Times, 15x15 themed puzzles (like mine) run on Monday through Thursday, with Mondays being the easiest and Thursdays being the hardest. Thursday puzzles also tend to involve a bit more trickery in the theme with things like rebuses or missing letters. You definitely get a feel for it the longer you do the crosswords, but this theme that I had constructed felt like a Monday or Tuesday theme. Ultimately it’s up to the editors which day a puzzle will run, but I would try to clue my puzzle to be a hard-Monday to an easy-Tuesday difficulty.

Realistically when you’re working on the fill you are coming up with some clue ideas in your head as you go. Therefore, this process went pretty quickly. Like the other steps though, there are a few rules about the clues. Again the obvious ones apply such as no profanity or obscure knowledge. The most interesting rule though is that none of your clues can have any of the words from your puzzle or other clues in them. In other words, a word may not appear more than once in a puzzle or its clues.

Looking up how answers have been clued in previous puzzles was extremely helpful for this step. Almost too helpful honestly. With tools like XWordInfo, I could see every way that a word had been clued in the past. This ended up making me realize that previous constructors and editors had found much more interesting ways to write clues than I had. I tried to avoid copying old clues as much as I could and rather used them as inspiration for an interesting way to write a clue.

For example, seeing EAR clued as “It may pop on a plane” led me to try to use the fact that an ear can pop in my clue. I ended up going with “Poppable body part.”

Writing the clues was really fun. Especially letting them sit for a few days and sleeping on some of them for a while. I won’t list all of them out here but here are a few of my favorites:

“Anti-liquid agcy.” (TSA)
“Sub-par” (BIRDIE)
“Like kangaroos and platypuses” (AUSTRALIAN - a (surprising) NYT debut in the Shortz-era)
“Source of sin” (EDEN)
“Kids take them down from Ladders” (CHUTES)

Final Adjustments

I sent the completed puzzle to a few friends and family members who do crosswords on a regular basis. They gave me great feedback about the sections they liked and the clues that they thought were a stretch. One area, in particular, stood out in many of the responses - the bottom-left. Specifically the crossing of NENA (German pop star) and ELEVE (French word for student). I think I knew deep down that this was a Natick and violated my rule of no double-whammies. This section had given me so much trouble that I tried to ignore it, but after a few people brought it up, I knew I had to change it. Another painful erasure of clues, but it ended up not being too tricky to fix and not impacting other sections too much.

I also received some common feedback that the clue for INNY (Concave navel) was actually spelled INNIE. Another one that I knew was a bit of a stretch but couldn’t find a better fill over in that section. I ended up re-cluing INNY as “In New York” with “What Buff. and Brkln. have in common.” This is still a stretch and not the best clue, but it’s better than a misspelling.

After my last tweaks, I was left with my final puzzle:

All done!

Publishing

Now that the clues were written and the puzzle was finished up it was time to submit for publishing! The CrossFire app makes it super easy to conform to the NYT submission guidelines, which (for some reason) still require hard-copy submissions. I printed out my PDF puzzle and mailed it over, along with a huge sense of accomplishment. Then the waiting game began.

Due to most of the publishers’ rules, once a puzzle is accepted the publisher owns the full rights to it. As a result, you can’t just submit your puzzle to as many publishers as you want and see which one replies first. I waited for a few months and eventually got back the following reply from the NYT editing crew, via email.

Thanks for sending us your SURFBOARDS 15x. This theme idea was nice, but the puns weren’t exciting Will quite enough to say yes. Sorry! We appreciated the chance to take a look here, though.

Drats! They didn’t like it. After reading some other message board posts from people who have published many puzzles, I learned that this response basically meant the theme wasn’t interesting enough for them, so there’s pretty much no chance of fixing it. Starting over is a better option than repairing.

After the NYT rejection came through I also submitted the puzzle to another publisher, the Universal Crossword. Their editor, David Steinberg, got back to me more quickly (about 6 weeks later) with a similar response:

Anyway, what an impressive first construction! I especially got a chuckle out of the reparsed SURFBOARDS and COASTLINE. The other two don’t grab me quite as much, though—in SAND CASTLE, I wish the meaning of “sand” changed as well; as for CATCH RAYS, the clue feels a little stretchy to me. While I’m afraid this one is a near miss for me, I hope you’ll try again with another puzzle!

Womp! 0 for 2. At this point, I started to come to terms with the fact that this puzzle wouldn’t make it into a major publication. However, the silver lining is that I still own the rights to it and can share the details with you all!

Lessons Learned

What a fun process this was! While I didn’t officially time it, the full construction process took several weeks. I’d estimate about 40 or 50 hours total. And I now have even more appreciation and respect for the people who do this for a living. Here are a few takeaways about what goes into constructing a puzzle:

It’s all about the word list - Having a thorough and complete word list is so so important. Especially if you want to be churning out puzzles at a decent pace. I can’t imagine coming up with a good, clean fill without one.
The grid is tough - Building the grid seems like an easy task but it is quite challenging. A bad grid can also set you up for a lot of trouble and headache down the road.
Undoing is ok - It was so painful to undo a bunch of work or erase a completed section, especially when said section took hours to begin with. But whenever I did undo a section, the next time through was a lot easier and I ended up being happy with it in the end.
Trust your gut - If a clue or word feels iffy or like a stretch, it probably is.
Take notes - I took tons of notes on different sections, wrote down which words worked, which words almost worked, why words didn’t work, etc. That ended up being very useful when working on a tough section.

I’m already working on my next puzzle and can’t wait to share it with you all when it’s done — hopefully this time it will be an NYT crossword link!

Interfaces for the Internet of Things

2018-11-07T15:14:00+00:00

It’s undeniable that we are entering a new era of computing. Whether you call it the Internet of Things, the Third Wave, the Fourth Wave, distributed computing, whatever it is, the technology landscape and the machines around us are starting to look different.

Plenty has been written about this shift but one aspect that is often overlooked is how interfaces evolve as we move into new technical eras. An interface, for the purposes of this post, refers to how humans interact with machines. You use different interfaces every day, whether you realize it or not. As the hardware and software that surrounds us change to embrace this new generation, new interfaces will be born.

The Past

Early computers did not have monitors that displayed the output of the machines. Instead, users would enter commands into a terminal, known as a teletype, and the results would get printed out on paper. Yes, real paper. Eventually, as computers became more mainstream, new interfaces like blinking lights, the monitor, and the mouse were born to allow humans to better interact with machines.

An early interface for computers – a Teletype Model 33

The 1990s brought a new era to the personal computer space, one where actions taken on a machine often depended on the state of a different machine across the network. This era introduced concepts like the World Wide Web and it undoubtedly changed the world of computing as we know it. It also meant that our interface for machines had to change. We needed a way to visualize and easily digest the state of a different machine and, as a result, the web browser was born.

Eventually, we entered the next generation of computing — mobile. This led to a plethora of new interfaces from touch screens to responsive design to fingerprint sensors that allow us to interact with these tiny machines that we carry in our pockets.

The Present

Which brings us to today. We are in a new time. One where devices are more powerful than ever and contain their own application logic. One where there are so many connected things talking to each other that we, as humans, aren’t able to keep track of all of the connections. One where we expect things to “just work together” without any effort and somehow be secure as well. One where connectivity and reliability are (often incorrectly) assumed.

The environment of technology around us is extremely complex. And, like always, the development of the infrastructure has predated the development of our interfaces. There are, however, a handful of different interfaces that are starting to emerge, ready to tackle this massive challenge and become the interfaces of the future.

Voice

Voice is now becoming a commonplace interface thanks to an elegant convergence of several different technologies. Natural language processing is finally starting to understand what we’re trying to say with pretty high accuracy. Text-to-speech algorithms kind of actually sound like humans now and not Microsoft Sam robots. And there are more microphones around us than ever before (this one’s a bit more creepy than elegant I suppose) with mobile phones and Amazon Echos.

As a result of this, virtual assistants are popping up everywhere. The obvious ones are Siri, OK Google (or Hey Google…or Google Assistant…or whatever they call it now), and Alexa (maybe also Cortana, does anyone actually use that?).

There are less obvious virtual assistants too. Bank of America has an assistant, Erica, that lets you interact with your bank account using voice. At this point, we’ve all probably interacted with a phone tree virtual assistant that (supposedly) helps us get in touch with the right customer service representative.

Not every voice interface is a virtual assistant though. Cable TV remotes can now be controlled (with impressive accuracy I have found) with your voice. Identity verification on some banking phone calls can now be done by repeating a phrase with your voice. Turn-by-turn navigation and you car stereo are controlled by voice.

While some of these technologies are still very limited in terms of their function and reliability, they do indicate a trend toward more voice interfaces in the future.

Low-Code

While voice is becoming a great way to interact with a set of partially pre-determined functions, it doesn’t excel when any variability is introduced into the system. You can’t tell Alexa to “run the sprinklers for 15 minutes if it reaches 90 degrees” for example. Even if you could, that would presumably only be because a developer had programmed that “recipe” into the Alexa dialect.

However, users continue to demand more control over their devices. They want to explore and utilize their creativity. They need an interface to do that. And they don’t want to have to write code or know programming to make it happen. We’re seeing control over data and devices expand into a realm of users who are technical but don’t program. This is where low-code or no-code interfaces are starting to shine.

Software solutions like IFTTT and Node-RED allow general consumers and technical hobbyists to connect their different services, apps, and devices together in creative and intelligent ways, all without writing any code. This puts users in control of their own world and not always reliant on what functionality already exists inside of an app or product.

A screenshot from Node-RED — allowing workflows to be built without writing any code

On the enterprise side, products like the niolabs System Designer and Mendix allow companies to define complex applications that power their businesses (disclaimer: I work at niolabs). These tools, which don’t require software developers to operate them, are opening up “intelligence” and “logic” to a much broader audience. This will almost certainly shape the way teams within companies are structured in the future.

Operating early computers meant working in the terminal and understanding various shell commands, something that the layman was not able to do. Eventually, through newer interfaces and better software, computers became accessible to the masses. The same will be true for distributed systems, which are quickly becoming more accessible to non-developers.

Proximity Wireless

While this progress is exciting, many of the newer interfaces don’t seem poised to gracefully handle the immense scale that will come with this new computing era. Launching an app or talking to your virtual assistant to deploy hundreds or thousands of devices is unreasonable. This is where we need devices to do a little bit of self-negotiation and pairing without too much human involvement.

The fundamental idea of this type of interface is if two things are supposed to connect with each other, they are both expecting a connection, and they are close by, then they should just connect. The only human requirement should be to put the system into a “discoverable” or ready-to-pair state.

One of the earliest examples I can think of this is the Wi-Fi Protected Setup feature that exists on most modern wireless routers. According to Wikipedia, this was introduced in 2006. In case you aren’t familiar, the way it works is that in order to join a new wireless network you push a button on the router to automatically connect your device to it within a short period of time. This technology relied on the assumption that if you had physical access to the router then you could probably be trusted.

There are many more wireless-proximity-based interfaces that are in use today. Bluetooth pairing is one that almost everyone has used at this point. Put at least one of the devices in “pairing mode” and another device can connect to it. NFC and close-range RFID allow us to make payments and unlock doors. Even scanning QR codes is a form of “visual” proximity wireless technology.

When talking about large numbers of devices or broad audiences of non-technical users, device interaction will have to be somewhat automatic and not require too much human involvement. The growth of some of these wireless technologies may be an indicator that this interface is just right for that type of problem.

Up until now, humans have interacted with computers one machine at a time. When you type on your keyboard or tap on your phone you are, for the most part, controlling one and only one machine. However, we are quickly becoming in need of interfaces that allow us to interact with systems — multiple devices and machines that work together. It’s difficult to say for sure what these interfaces will be — some combination of the above, voice, low-code, and/or proximity wireless, seem like reasonable expectations at the moment. What’s certain, however, is that they will be novel and different compared to the past.

The Two Questions You Should Never Ask Anyone

2017-10-20T20:56:25+00:00

Ok, there are probably more than two questions that fit into this category. How much money do you make? What’s that thing on your face? Did you have to kill anyone in Iraq? But these are two I’ve had recent personal experiences with that may not be as obvious, so I thought I’d share.

So, without further ado, the two questions you should never ask anyone are:

How is the job hunt going?

When are you going to have kids?

The first one was totally my bad. I recently asked this to a friend of mine who had been unemployed for some time and who I knew was sending out resumes. As soon as I said it I regretted it. I know this person pretty well, and if there was good news, they would have told me. Instead, I had to make them remind me of the fact that they were still in the midst of the dreaded job hunt. What a great friend I am!

It’s obvious in hindsight, but a simple rule applies here—don’t ask a question that you know doesn’t have a good answer.

The second question is related, but in this case, the tables are turned for me. This is the question I really want to talk about here.

When are you going to have kids?

First of all, not everyone wants to have children. And they likely don’t have a strong desire to constantly defend such a personal decision. Then there is the whole other group of people who want to, but can’t.

My wife recently shared on social media the struggles we’ve been having with infertility over the last couple years. Throughout that time, I cannot tell you how many times we’ve been asked that seemingly innocent question.

Believe me, no one dealing with infertility actually thinks this question is in any way intended to be hurtful. Sometimes it comes from family members who are just excited to have another baby around. Sometimes it’s from colleagues who are just asking the typical small-talk questions they always ask a semi-newly-married couple. Regardless, we know the question is not meant to harm.

The problem is it sometimes does.

It’s not so much that it saddens you or angers you to hear that question, it’s more that it reminds you that you are in an awful situation and then forces you to decide whether you want to address that situation or not.

Do I really want to launch into the whole explanation right now? I could flip the awkward situation back around on the question-asker.

Not sure, we’ve been trying for two years. Dozens of medications don’t seem to be working, any suggestions?

While that likely achieves the objective of preventing this person from asking the question again, it’s probably not the most delicate way to respond.

Instead, I often just end up avoiding the conflict altogether and responding with something ambiguous like the following:

Oh, I don’t know, one of these days, we’re starting to talk about it.

The problem with a response like that is that it doesn’t really address the problem. And I think the root of the problem is simple—people don’t talk about infertility.

Let’s talk about it

Almost everyone who has seriously battled infertility has probably asked themselves something along the lines of “why is this happening to me.” It feels like such a lonely and hidden experience. AND I AM A GUY! I can’t even begin to imagine how it feels for a woman experiencing it when our society has put so much unfair pressure on women to raise families.

The thing is, it’s not a lonely experience. Tons of people deal with it and, while every journey is different, there are a lot of others out there who can relate to your grief. There are countless different estimates on the number (1 in 4 couples, 1 in 6, etc.) but whatever the actual number is one thing is certain: someone you know, maybe even someone you consider yourself close to, has struggled with infertility.

I know this because as my wife and I started to gradually open up and share what we’ve been experiencing, people often responded that they had a similar experience or knew someone who had. Given so many people around us that can relate, why does it still feel so shameful to share it? The perfectly-manicured lives we tend to portray on social media probably don’t help. Neither do Hollywood films where quick (and often inconvenient) pregnancies are a common trope.

This lack of awareness around the topic can cause otherwise well-intended questions to have unfortunate consequences. Hopefully this post can do its small part in raising awareness about a difficult and all-too-common subject.

A Statistical Look at Liar’s Dice

2017-10-09T20:56:25+00:00

Liar’s dice is a great game. Easy to learn, difficult to master. A good balance of luck and strategy. If you don’t know the game, I encourage you to check it out, it’s great for a group.

In our last session, we encountered an interesting situation when we got down to the final two. One player had all 6 of his remaining dice while his opponent was down to one. Of course this led to much discussion and side betting. Mainly, we set out (read: argued) to answer the following questions:

Can the player with all six dice devise an optimal strategy to guarantee a victory?
If not, what are the odds for the player with only one dice?

And just like that, a bet was born. A bet not even involving the two players actually playing the game.

The Strategy

Before we get to the bet, we needed to agree on what the strategy of the player with all 6 dice would do. We pretty quickly arrived that, in order to avoid ever “bluffing,” the person with 6 should just call the highest thing they can possibly call and hope that the opponent with 1 die won’t be able to beat it. So, given dice 1, 1, 3, 4, 4, 5, he should call four 4s (1s are wild).

The Bettors

After taking minimal game theory courses in college, I tend to assume there is always some kind of strange optimal strategy. I was unable to find said strategy but I of course immediately started trying. My goal was to find some consistent strategy (one that did not allow for variable bluffing, despite it being a key component of the game) and then calculate the odds of how often it would win. A statistical (and thus, sure fire, right?) approach to the problem.

On the other side of this bet was my friend Kurt. Kurt, a long time card-player, doesn’t think about the math right away. Instead, he rapidly starts running through thousands of iterations in his head of how the game might play out. His intuition about odds and seeing the game so many times allows him to determine how often each player will win.

The Bet

My quick math showed me that the person with all 6 dice would only lose if the opponent had the same number they called, or had a 1 (wild card). Given 1, 1, 3, 4, 4, 5 and a call of four 4s, the underdog only wins when he has a 4 or a 1, thus allowing him to call five 4s and win. Rolling a 1 and another specific roll will happen 1 in 3 times and it needs to happen 6 times in a row in order to lose. In other words, only 1 in 729 times will the underdog win.

Kurt did some other kind of gambler’s voodoo magic in his head and boldly exclaimed “if you give me 1 dice against your 6 15 times in a row, I bet you I win one.” Me having just calculated the odds with 100% certainty (maybe not…) saw an opportunity to strike. I replied “you’re on” so quickly that Kurt second-guessed himself based on my confidence. I balked a bit at his request for 2:1 odds (he’s already getting odds by running it 15 times) but eventually gave in, thinking I was still a statistical lock.

6 dice vs 1 die. 15 times in a row.

Time to play.

The Result

The first few turns went exactly according to plan. I’d call something like three 5s, Kurt has a 3, game over.

For the most part, the first 10 games went great. Kurt rolled a couple 1s in there but would pretty much lose on the next turn every time. But then, somewhere around the 10th game, something happened.

I rolled 1 1 3 4 4 5 and, per the strategy, called four 4s, the most I could safely call. Kurt then comes back with four 5s, having rolled his own 5.

Oh, crap.

I lost, and he didn’t roll a duplicate or a 1. I ended up winning that game anyways, but he did get me down to three dice. I went through the next few games no problem and won the bet. But my interest had been piqued. My assumptions, and thus my odds, weren’t correct. I must find out what they actually are.

Simulation

Now realizing the problem is possibly more complicated than simple statistics can solve, I set out to write a simple simulation script that could run many many trials to get an estimate of the odds. It also seemed to be a good opportunity to test out Jupyter Notebooks, something I had been meaning to do for a while. All of the code and iPython notebooks for this project is available on GitHub.

Building out the dice randomization, general gameplay flow, and my simple rule-based-no-bluffing strategy wasn’t too tricky. The real variable in this situation is the strategy that Kurt should use.

Run 1 - The One-Upper

As a sanity check, I wanted to check my original odds and give Kurt a basic strategy. Whatever I bid, Kurt bids one more of that and hopes that he has a 1 or whatever number I called. Not smart, but a good starting point.

def get_kurts_bid(my_dice, num_dice_matt_has, matts_bid=None):  
    if matts_bid is None:
        # If Matt has not bid yet start off with 1 of my dice
        return (1, my_dice)
    else:
        # Return one more of what Matt said
        return (matts_bid[0] + 1, matts_bid[1])

Simulation Complete
-------------------
Num Simulations: 10,000,000
Took:            247.9 seconds
Kurt Wins:       13,742
Win Percent:     0.14%
Wins 1 in 727.70 times

Sure enough, 10 million simulations show that my odds of 1 in 729 were likely right. It also, perhaps more importantly, shows that the code is working. Time to start optimizing.

Run 2 - The Good Guesser

Maybe Kurt can do better by trying to trap Matt on the initial bid. Rather than letting Matt drive the action, Kurt can try and guess the number that Matt would have called anyway. An initial guess of three of whatever die Kurt rolls seems like a good guess.

def get_kurts_bid(my_dice, num_dice_matt_has, matts_bid=None):
    if my_dice == 1:
        if matts_bid is None:
            return (1, 2) 
        else:
            return (matts_bid[0] + 1, matts_bid[1])
    else:
        if matts_bid is None:
            return (3, my_dice) 
        else:
            return (matts_bid[0] + 1, matts_bid[1])

Simulation Complete
-------------------
Num Simulations: 10,000,000
Took:            243.0 seconds
Kurt Wins:       14,365
Win Percent:     0.14%
Wins 1 in 696.14 times

Not much improvement, dropped to 1 in 696 times for Kurt to win. This actually makes sense, mostly because of gameplay. Once Kurt wins the first turn, Matt gets to go first in subsequent turns. So really this strategy only affects the very first turn of the game.

Run 3 - The End Gamer

Since the previous strategy only affected the first turn, let’s see if Kurt can be smarter towards the end of the game, if he can get there.

When it gets down to Matt having two or less dice Kurt wants to guess two of whatever his dice is. The hope is Matt rolls 1, 3 and Kurt rolls 4. Matt guesses two 3s, Kurt comes back with two 4s and wins.

def get_kurts_bid(my_dice, num_dice_matt_has, matts_bid=None):
    # When Matt has 2 or less dice, guess 2 of whatever I have if I can, otherwise guess 3 of them
    if num_dice_matt_has <= 2:
        if (2, my_dice) > matts_bid:
            return (2, my_dice)
        return (3, my_dice)
    if my_dice == 1:
        if matts_bid is None:
            return (1, 2) 
        else:
            return (matts_bid[0] + 1, matts_bid[1])
    else:
        if matts_bid is None:
            return (3, my_dice) 
        else:
            return (matts_bid[0] + 1, matts_bid[1])

Simulation Complete
-------------------
Num Simulations: 10,000,000
Took:            243.5 seconds
Kurt Wins:       14,831
Win Percent:     0.15%
Wins 1 in 674.26 times

A little better, but we’re still not doing that much damage. Kurt is only getting to the final two dice about 1 in 80 times anyway. Any strategy we apply there doesn’t do too much to help overall.

Run 4 - The Finger Crosser

Let’s apply the situation that happened during the real game. As a refresher, Matt rolled 1 1 3 4 4 5 and called four 4s. Then Kurt then came back with four 5s, having rolled his own 5. So Kurt’s strategy is to guess the same quantity that Matt bids, but of the number he has instead. The hope is that Matt has only a few natural rolls of his number and the rest 1s.

def get_kurts_bid(my_dice, num_dice_matt_has, matts_bid=None):
    # When Matt has 2 or less dice, guess 2 of whatever I have if I can, otherwise guess 3 of them
    if num_dice_matt_has <= 2:
        if (2, my_dice) > matts_bid:
            return (2, my_dice)
        return (3, my_dice)
    if my_dice == 1:
        if matts_bid is None:
            return (1, 2) 
        else:
            return (matts_bid[0] + 1, matts_bid[1])
    else:
        if matts_bid is None:
            return (3, my_dice)  # Might as well try and trap him
        elif (matts_bid[0], my_dice) > matts_bid:
            return (matts_bid[0], my_dice)  # Hoping for [1, 1, 3] against [4]
        else:
            return (matts_bid[0] + 1, my_dice) # Hope he has all 1s?

Simulation Complete
-------------------
Num Simulations: 10,000,000
Took:            294.1 seconds
Kurt Wins:       89,389
Win Percent:     0.89%
Wins 1 in 111.87 times

Ooooh, progress. Kurt wins 1 in 112 times now. This situation of having all 1s and one natural actually happens a lot, especially towards the end of the game when there are fewer dice.

In the last case, if Matt calls three 4s and I have a 3, for example, Kurt would call four 3s. From what I can tell, this only happens when Matt rolls all 1s, so probably not a whole lot.

Run 5 - The Adjuster

Upon thinking more about the last strategy, I realize it’s essentially the same as the original end game strategy. Actually, it’s an even better end game strategy! The previous end game strategy of guessing 2 of whatever Kurt has (if possible) doesn’t account for a Matt roll of 3, 4 and a Kurt roll of 5. Matt guesses one 4, Kurt should guess one 5 but would previously guess two 5s.

Let’s apply the general game strategy to the end game also.

def get_kurts_bid(my_dice, num_dice_matt_has, matts_bid=None):
    if my_dice == 1:
        if matts_bid is None:
            return (1, 2) 
        else:
            return (matts_bid[0] + 1, matts_bid[1])
    else:
        if matts_bid is None:
            return (3, my_dice)  # Might as well try and trap him
        elif (matts_bid[0], my_dice) > matts_bid:
            return (matts_bid[0], my_dice)  # Hoping for [1, 1, 3] against [4]
        else:
            return (matts_bid[0] + 1, my_dice) # Hope he has all 1s

Simulation Complete
-------------------
Num Simulations: 10,000,000
Took:            301.5 seconds
Kurt Wins:       354,633
Win Percent:     3.55%
Wins 1 in 28.20 times

Whaaa!! Kurt wins this way 1 in 28 times!

It kind of makes sense maybe. Kurt just needs to get through a couple turns by rolling a 1 or the same as me and then has to count on me rolling 1s. It definitely happens, and it turns out it happens about 3.5% of the time.

Outcome

So, 15 runs at 2:1 odds are actually looking pretty good right now.

Even more importantly, it shows that Kurt’s intuition was nearly spot on. This is truly fascinating to me, just by drawing on past experiences and seeing so many dice rolls, Kurt was able to almost exactly predict the odds of this complex scenario.

At least I still won the bet.

Tweet-2-RSS has been shut down

2013-10-03T09:56:25+00:00

A few months back I created a service that would allow people to get tweets or other content from the Twitter API in RSS or XML format. It ended up growing a moderate amount and amassed over 2,500 users. Well, some frustrating and disappointing news came through yesterday; the Twitter API team has suspended the Tweet-2-RSS application for violation of the Terms of Service. They did so without any warning and it was certainly a surprise to me. Here is the relevant quote from their API Terms (Section I.4):

You will not attempt or encourage others to sell, rent, lease, sublicense, redistribute, or syndicate access to the Twitter API or Twitter Content to any third party without prior written approval from Twitter. If you provide downloadable datasets of Twitter Content or an API that returns Twitter Content, you may only return IDs (including tweet IDs and user IDs).

They consider returning the body of the tweet in any form “re-syndication” of their content. Their “solution” is to return a tweet ID and have the end user make their own API call to obtain the actual tweet. Obviously this is a horrible solution for current users as it would require you to register your own Application and make a custom API call. If you could do that, you wouldn’t be using this service at all.

There are dozens of uses for getting RSS/XML Twitter data; use in existing RSS readers, XML-only plugins, and more. Unfortunately, it seems that the Twitter API team doesn’t recognize them or care. Their lowly customer service rep who was on my case specifically called it out (and even tried to get me to rat out some similar services):

Services which provide XML/RSS output of Twitter Content are disallowed per our API Terms of Service, unless they only return IDs. If you have encountered other services which engage in this activity please let us know and we will investigate them for compliance with our Rules.

I “politely” responded that I would not snitch on fellow developers who are simply trying to make the web a better place and that they could do their own homework to find them.

To the users of Tweet-2-RSS, I apologize for the sudden suspension. It came as a surprise to me as well. Even though Twitter can’t seem to do some simple Googling, there are other services out there that will accomplish the same thing. However, be warned that they are most likely in violation of the same terms. For now, Tweet-2-RSS will just return some empty XML results so your plugins won’t “break” or throw exceptions. It will just look like there is no content. To those of you who donated and helped out with the hosting costs: I greatly appreciate the support and help. I will be refunding your donations considering the service is no longer operating. Wait for an email from me.

This isn’t the end of the road though. I understand the need for a service like this and I am currently working with similar RSS services to come up with a workaround. It may take a few weeks but I’m confident that a solution will arise, it may just be a hacky one. If you are interested in being involved let me know via email or a comment below.

Realtime Stock Analyst Opinions

2013-07-24T16:11:33+00:00

Two words: WolframAlpha. Or is that one word? I don’t know, but what I do know is that it is a lifesaver for any engineering student these days. Unfortunately for me, it only really existed during my senior year of college, which happened to be the year I got the highest GPA. Coincidence? Probably.

Until now, I had really only used WolframAlpha for its scientific and mathematical solving abilities. But it can do so much more! Featured in this post will be a snippet of its financial knowledge. Head on over and try searching for your favorite stock symbol. You will notice a plethora of information from realtime trade data to price projections. You will also notice that they provide a breakdown of analyst opinions on the stock from buy to sell. Wouldn’t it be cool if we could see this breakdown for all stocks in the market and how they compare to one another? I’m glad you asked.

Queue the WolframAlpha API. Thanks to that API, a realtime backend provided by Firebase, and some nifty visuals courtesy of HighCharts, I was able to put together a basic visual of the overall opinion of all 500 of the stocks on the S&P 500. So go check it out! More on how it works below, but first a screenshot!

The way it works is rather simple, on my command it takes the current list of the 500 S&P 500 stock companies (courtesy of Wikipedia) and one-by-one (in parallel of course) asks WolframAlpha for the analyst opinions of the stock. It collects and parses that information, then pushes it up to a Firebase. Firebase is nice and will keep all of this data for me and even notify existing connections if any of the data has changed. So if you happen to be looking at the chart when the analyst opinions get updated, the chart will get updated too. The whole app is written in Node and hosted on AppFog, another first for me. I have to say, their deployment and management process is really straight-forward.

It was a fun little project, I got to experiment with AppFog, and I think it’s actually a somewhat useful and meaningful visual. Hope you enjoy! Check it out here: http://stockynode.aws.af.cm; you can leave feedback in the comments below.

24 : A childhood game taken to the next level

2013-07-19T15:27:49+00:00

Remember that math game 24 from middle school? It’s the one where you are given 4 numbers and you have to do basic math operations (+,-,*,/) to make the 4 numbers equal 24. For example, if I had the numbers 2,3,4,5 I could get to 24 by doing 2*(5+4+3) or maybe by doing 4*(5+3-2). You get the idea.

So the game itself is pretty simple and for some reason I find myself playing it with the numbers on a clock or credit card or something else when I am bored. And I was finding that it was normally pretty easy to find a solution given any set of numbers. That made me start to ask a couple of questions.

Exactly how many solutions exist for solving 24, given a set of 4 integers each between 1 and 9?
Is there a reason 24 was chosen? Does it have more solutions than another number, say 20?

The first step was to figure out exactly how many sets of 4 numbers there were. We need unique sets here, the set [1,3,3,7] is fundamentally equivalent to [3,1,3,7] for the purposes of the game. Sadly, this took me longer than it should have, my combinatorics must be rusty. But by using the Multiset Coefficient (basically, combinations with replacement) I used the following formula to determine that there are only 495 unique sets. Much less than I expected given that there are 6,561 total sets of numbers (9^4).

(n+k−1)!
--------
k!(n−1)!

The next step is to bust out the IDE and get to coding. Fortunately, this part didn’t take as long as the math (my coding isn’t rusty yet, phew). Here is a quick little Python 24-solver (really an N-solver…we’ll get there) I threw together.

def solve(n, nums, solveStr='x'):
    ''' Play 24 with nums 1,3,4,6 like so:  
        >>> solve(24, [1,3,4,6])
        (6/(1-(3/4)))                       '''
    n = float(n)
    solved = False

    if len(nums) == 1: 
        return (solveStr.replace('x', str(int(n))) if n == float(nums[0]) else False)

    for num in [float(x) for x in nums]:
        possibles = [   ( n-num , '+', False),
                        ( num-n , '-', True),
                        ( n+num , '-', False),
                        ( n*num , '/', False),
                        ( (n/num if num != 0 else 100000) , '*', True),
                        ( (num/n if n != 0 else 100000) , '/', True)]

        wo = [x for x in nums]
        wo.remove(num)

        for possible in possibles:
            if possible[2]:
                solved = solved or solve(possible[0], wo, solveStr.replace('x', '(' + str(int(num)) + possible[1] + 'x)'))
            else:
                solved = solved or solve(possible[0], wo, solveStr.replace('x', '(x' + possible[1] + str(int(num)) + ')'))

    return solved

So now I’ve got a solver and a list of 495 unique sets, time to find out how many of those actually have a solution for 24. Ready? Drumroll….

381 out of 495 have a solution!

381?! Wayyyy more than I was guessing. No wonder this game is so easy, pick 4 numbers and you pretty much have a 3 out of 4 shot that there is a solution. So on to part 2, is there possibly an easier or more difficult number to use as a target? I ran it for target solutions from 1 to 100, and the winner (meaning the number with the most solutions) is….more drumroll…

2- with 479 solutions out of 495 possible

2 has the most, followed up by the rest of the single-digit numbers interestingly. 24 came in 17th place. 94 and 97 tied for last with 40 solutions each (remember, only checking from 1-100). Here is the full list as a gist for your viewing pleasure.

One more fun thing; there are actually 2 (and only 2!) sets of 4 that can produce solutions for a target of 314. Here they are, can you come up with the formulas to get there?

1 5 7 9
5 6 8 8