Earlier today I got involved in a discussion on Twitter with noted internet profiles Michael Bolton, James Bach and Woody Zuill. It all started when I read an interesting tweet by Woody, immediately followed by a reply from James:

I backtracked the discussion to this tweet by Michael:

After quite a few tweets we ended up here:

You can catch up on the details by reading them all from top down.

First of all, if it’s not already obvious from my tweets, let me make it clear that I’m not an expert on either testing, AI or philosophy. For me the the discussion was all about if it’s possible to write programs that do more than just verify compliance with given rules (“checking”), i.e. programs that somehow performs “tests” and thus finds problems outside what has been specified in the checks. Second, I mention that I probably joined the discussion in a late stage since I had not seen the citations of the two Collins books that Michael mentions, but after reading the tweets again, not being on a crowded subway, I noticed that I had simply missed them. I apologize for my sloppiness.

Anyway, the thing that got me going was the statement by Michael that “Testing is more than checking”. I agree that it seems like a perfectly logical statement, and at the current state of things I do agree that machines or programs are not capable of the level of testing that man can do. But is it impossible?

Somewhere in the middle of the discussion I compared the human brain to a machine that has been programmed by evolution (and of course what you have learned during life, but even that ability is a result of evolution, so: evolution). My intention with this was to illustrate that if machines/programs are not capable of testing (but rather just perform a certain amount of checks or validations) then neither are we, “the human machines”. Or are we? If we are, then there must be a a limit where a high enough number of checks/validations becomes the “act of testing”. At least if we assume that there is nothing magical about about the brain that can not, sooner or later, be copied by man-made components.

To proceed, we must ask ourselves where this limit lies. Consider a rat in a labyrinth – is it testing its way out or to find the food, or is it just performing a series of checks? A fly trying to get out through a window? A virus trying pass a cell membrane? When does testing cease and mere checking take over? At what level are the programs we write today? Are we even close to producing anything living in nature? If not, when, if ever, will we get there?

That was all I wanted to discuss. I did not intend to insult anyone or appear as dismissive. I would love to continue discussing comprehension vs. representation and so forth, but it’s already way past bedtime.