Tag: machine_learning

What ChatGPT Can Teach Us About Being Human

(This post’s featured image is not a photo of the idyllic California vineyard my wife and I visited in 2015, but a similar looking AI-generated fiction)

The excitement around large language models (LLMs) continues. Often just called “AI,” this new technology takes instructions in plain English, and generates new text or images so good you’d think a human made them. There are some serious concerns about the ethics of how this is done (see the Dangers of Stochastic Parrots), and many articles warning people that LLMs aren’t as human as they seem. Still, these LLMs are clearly doing something smart, and it’s weirdly compelling. What’s going on here? What can LLMs teach us about the human mind, our strengths and weaknesses, and why we’re so easily fooled and mesmerized by this tech?

There are lots of articles about how LLMs work and their limitations (I even wrote one a while back), so I won’t go into much detail here. What matters for now is that LLMs are prediction engines. Given some text, they try to guess how a human would respond, based on a statistical analysis of all the content on the internet. They do this extremely well, but not perfectly. LLMs don’t think, perceive, or interact with the world in a human-like way, so sometimes they make weird mistakes a human never would.

In some ways, the human brain works a lot like an LLM. From day one, the brain is looking for language. It automatically builds up vast networks of words, concepts, and their relationships. When I hear someone talking, my mind is suddenly filled with the speaker’s ideas and their associations, building up an image in my mind’s eye. This is basically what an LLM does.

However, I also have many other intelligent faculties that join in. I don’t just know that “dog” is a word related to “cat.” I have memories of specific dogs, what their fur felt like, and the experiences we shared. I have common sense, logic, and theory of mind to decide whether something I hear is truth, fantasy, error, or deception. I monitor my own thoughts to gauge my level of confidence and correct my mistakes. I anticipate the future, make plans, and use language to achieve my goals. LLMs don’t do any of that.

Of course, there are AI researchers working to approximate these other kinds of human intelligence (though progress here is limited compared with LLMs themselves). Rebooting AI is a great book exploring that work, which argues its just a matter of time before AI can do everything a human can. Personally, I’m skeptical that we’ll ever reverse engineer all the subtlety of human thought, but I think it’s safe to say that AIs will become more powerful and well-rounded in the future. Perhaps the more important question is whether building increasingly realistic human simulations is a good idea at all.

For now, LLMs are a bit of a one-trick pony. What’s scary is that one trick is often good enough to fool humans and do useful work. In particular, even though LLMs were designed to be text prediction engines, they are surprisingly good at general problem solving. They can paint pictures, do math, solve brain teasers, and even write and simulate computer programs. Maybe those “extra” faculties of the human mind aren’t so important after all?

LLMs “think” in terms of words and relationships and patterns they’ve seen before. In human terms, that means stereotypes, cliches, generalizing from past “experience,” and repeating what they are told. We like to think that human thought is more sophisticated than that, but it often isn’t. We sometimes don’t see people as individuals, but in terms of the role they play in society (ie, “barista” or “mom”). We make decisions based on rules of thumb or gut feeling, without the need for logic and reasoning. We talk about things we don’t fully comprehend. We repeat talking points in order to fit in with our tribe. We confidently make up nonsense just to satisfy each other and move on. It’s surprising how LLM-like humans can be sometimes.

And that’s not meant to be derogatory! Those ways of thinking can be very effective. A lot of language isn’t about complex ideas, comprehension, and reasoning, but just putting one word after another to evoke an image in someone else’s mind. Past experience often is a highly effective and low-effort way to predict the future. Lying is anti-social, but “fake it ‘til you make it” works. One of the fastest ways to pick up a new skill is to boldly make mistakes, get feedback, and learn from that experience. The main difference is that today’s LLMs don’t learn from their mistakes, they never doubt their “intuition,” and they have no alternative ways of “thinking” when these techniques fall short.

So, yeah, LLMs only do part of what humans do, but it’s a big and important part. Occasionally we do need facts, critical thinking, self-doubt, and all the rest to do the right thing, but they don’t come up as often as we like to think. The real danger of LLMs, then, is that 80% of the time they might be good enough, but 20% of the time we need fancy human judgment to notice they screwed up and decide what to do. This is a serious problem. Humans are bad at vigilance, and we have a strong instinct to trust language, which in this case is exactly the wrong response.

Language is a defining feature of our species. We aren’t just capable of language, it’s a human universal. Every culture has language. Babies attend to speech from the moment they’re born, and start to babble in a few months. When there isn’t a common language spoken around them, children raised together will spontaneously invent one. Language is a biological imperative for us. It’s in our DNA.

When we perceive language, our minds automatically assume that it’s communication. We imagine another mind behind the words, usually with good intent and a desire to cooperate. Up until recently, this was a pretty safe assumption, so it was totally reasonable for the brain to immediately and automatically translate language into meaning. But now this instinct is backfiring. LLMs create realistic text and imagery without any intentional meaning. They don’t produce “answers,” “opinions,” or “art,” just random content that looks like those things. It’s both very difficult and important to remember that.

We’re still working to understand how these LLMs work, what their limitations are, and what they’re good for. As we do that, I hope we’ll come to understand ourselves better, too. What do you think? Have LLMs made you think about minds any differently? Have you seen any interesting examples of AIs acting strange or foolish? What about people acting like LLMs? Any thoughts or fears about computers gradually inching toward human-like abilities? I’d love to hear from you in the comments.

Why Genetic Algorithms?

These days, the Artificial Intelligence community is pretty fixated on Deep Learning, a software tool inspired by the human brain. It’s popular because it’s successful. Deep Learning technology has driven incredible advances in natural language processing, image perception / generation, and game playing (not to mention ad targeting, feed ranking, and surveillance). That success was partly driven by luck. It turns out, even a fairly simple DL algorithm, given enormous amounts of data and computational power, can do pretty remarkable things. Other attempts at reproducing intelligent behavior haven’t been so successful, and are relatively neglected for that reason. That’s a shame, because I believe there are big opportunities in areas where we aren’t looking, they’re just a little more difficult to uncover.

Recently, I’ve taken particular interest in Genetic Algorithms. Put very simply, these are computer programs which use the principle of “survival of the fittest” at their core. They’ve been a popular topic of research since the 1960’s. Since then, they’ve found moderate success and have become standard kit for solving certain kinds of computational problems. Unfortunately, the fundamental design principles for GAs haven’t changed much since the 70’s, despite the fact that our understanding of evolution and our computing hardware have improved dramatically since then. I think today’s GAs are a shadow of what they could be, and I have ideas for how to unlock that potential.

But first, let’s establish what a traditional Genetic Algorithm is. Like Deep Learning, GAs are a tool for training a computer to do a task without explicitly telling it how to do the work step by step. There’s one big difference, though. DL is good at replicating what a human being would do, but to learn that it needs hundreds (or better yet, millions) of examples to study and imitate. So, it’s mostly good for automating work that people already do routinely. In contrast, GAs don’t need any example data. They’re good at solving problems where we don’t know the best solution or how to find it, just so long as we can recognize a good answer when we see it. The main challenge is framing the problem in such a way that the GA can learn to solve it.

Genetic Algorithms are often used when the space of possible answers is so big it would be impossible to try them all, or even to explore them in a systematic way. Instead, a GA depends on randomness. It starts by just guessing a bunch of solutions. Most of these will be garbage, so they get thrown out right away. The ones that are slightly better than complete garbage become the starting point for the next iteration. The algorithm makes more possible solutions by mixing the best previous solutions together and sprinkling in some extra randomness for variety. By sheer luck, some of those solutions may be better than the previous ones, and the process repeats. This simple method can be surprisingly effective, gradually transforming garbage into gold. But it depends a lot on giving the algorithm the right pieces to start with such that randomly mashing them together might actually work.

Natural selection is a kind of design process. Like human designers do, nature gets inspiration from random sources, tests ideas against harsh reality, and iterates to discover and build highly functional objects. Genetic Algorithms are basically an automated version of that process, so they’re frequently used as design tools. For instance, modern Computer Aided Drafting (CAD) software for architecture and industrial design often use GAs. A human designer specifies some constraints (how big an object can be, how much weight it must hold, etc.) and fitness goals (minimize weight and material costs, but maximize tensile strength), then the software automatically finds good solutions. Recently, there’s also been a great deal of interest in using GAs to design effective architectures for Deep Learning systems.

So, that’s Genetic Algorithms in a nutshell. Why do I feel there’s unrealized potential here? What do I want to do differently? There are several insights from modern evolutionary theory that I want to apply to GAs. Generally speaking, though, it’s all about what work is done by the programmer vs. what is left up to evolution. A GA uses a “gene sequence” (usually just a string of digits) to represent each of its attempted solutions. A gene sequence can either be a solution in itself, or a program that generates the solution. Traditionally, what the gene sequence means and how to derive an answer from it is entirely up to the programmer. This is very different from nature, where life evolved the language of genetics and species-specific genomes along with all the organisms built using those tools.

Then there’s the search process itself. In a traditional Genetic Algorithm, the programmer takes great care to tune the rate of mutation and the process of recombining genes from multiple individuals. They often hand design “custom mutations” that are more clever than just randomly changing digits in the gene sequence. They use their knowledge and intuition to avoid testing obviously bad gene sequences, to make changes that seem useful, and to preserve good patterns in the gene sequence that might otherwise get clobbered or broken up. This can help enormously, but it’s a lot of hard work for the programmer. The science of epigenetics shows that nature uses many of the same tricks, but it discovered them through natural selection, without any human guidance.

In a nutshell, life has an element of self-determination. Life designs itself, and optimizes the search process for better designs, using evolution by natural selection as its tool. I think it’s a mistake to imagine life as a passive product of evolution, like traditional Genetic Algorithms do. I think it’s a mistake to leave the hardest parts of GA development to a human being. Not only does that make human creativity a limiting factor, it means we aren’t studying or reproducing the most remarkable thing about natural evolution.

I hope this perspective will be valuable. One of the biggest shortcomings of Genetic Algorithms so far is that they aren’t nearly as open-ended or creative as life itself. Very rarely do they exhibit the sort of accumulated layers of complexity and sophistication that are the hallmark of natural intelligence. Perhaps that will change, if we can hand over more of the creative work to the GA? Perhaps a new paradigm in GA designs could make them applicable to new kinds of problems, or produce more clever solutions? I’m working on a prototype that I hope will demonstrate that potential.

The main drawback of my approach is that evolving the design for a Genetic Algorithm takes way more time and computing power than just running a GA designed the old fashioned way. I’m not too worried about that, though. I’m using the same parallel processing hardware that has become ubiquitous for Deep Learning applications to make my code efficient and scalable. My hope is that this new approach to GAs will make it possible to improve their performance by just throwing more computational power at the problem, like we already do so successfully with DL. This would also make GAs less cumbersome, by replacing expert design and hand-tuning with automation.

There’s lots more beyond this that I hope to explore in time. For instance, life evolves whole ecosystems and environments where organisms collaborate and support each other, producing intelligence, robustness, and efficiency far greater than the sum of its parts. By comparison, many Genetic Algorithms work on single individuals that exist in total isolation. Then there’s the way life evolves layer upon layer of emergent complexity, building communities out of brains, bodies, cells, and proteins. I’m excited to build multi-layered intelligent systems, and especially to try combining evolved programs and neural networks in a biologically realistic way.

Turns out, there’s much more to AI than Deep Learning. I’m pretty excited by the untapped potential of Genetic Algorithms, but that’s just me. Are there other areas of AI research that interest you? Can you think of more examples of natural intelligence that computers can’t seem to replicate? I’d love to hear from you in the comments. This post is also a preview of my research. I hope to share my first prototype by the end of the year. So, stay tuned for more on that.

Large Language Models, LaMDA, and Sentience

Several folks asked me to weigh in on whether Google’s AI chatbot, LaMDA, is sentient. I don’t know much about LaMDA specifically, so I want to talk about Large Language Models (LLMs) generally, since they show up in many forms. It’s a truly amazing technology. They can generate text that’s superficially indistinguishable from human writing. But are these systems capable of sentience? Let’s dig into it.

Let’s start with what an LLM actually does. At the core, it analyzes text for statistical correlations. This word co-occurs with that word. When you see this, it’s often followed by that. These words appear in similar contexts, and may be interchangeable. That sort of thing. What makes LLMs “large” is that they get trained on enormous bodies of text. Like, billions of web documents or whole libraries worth of books. This allows them to learn very subtle and nuanced patterns, and collect example texts on many themes. When an LLM is put into use, what it’s doing is confabulating new sequences of words with the same regular structure as its training data. They mix prompting from the user with relevant passages in their training data and randomness.

LLMs take advantage of a couple recent innovations in AI. One is transfer learning. First, the LLM is trained on an enormous corpus to learn the structure of language generally. Then, it gets fine tuned on a narrow data set, to constrain its output to fit a specific style and context. This isn’t so different from style transfer in computer vision. The other trick is attention and memory. LLMs can spot correlations between words at short, medium, and long distances, and learn which of the associations it learns are most relevant to good output. This makes LLMs much more self-consistent and better at question-answering tasks than previous technology.

A large part of why LLMs are so effective has to do with language itself. Language is highly self-referential. Words are defined in terms of other words. The meaning and sentiment of a word comes primarily from the context where it’s used (when I learned about word2vec, I came to appreciate this much more deeply). We each have vast networks of words and images and memories all tied together, and it’s the shape of that network that creates meaning. Humans are able to communicate with language for two reasons. First, folks who speak the same language have consistent networks of words in their minds. They’re highly correlated with each other, so the words mean the same things to both people. Second, those networks of words are consistent with the shape of our thoughts and our lived experience of reality. That allows us to appreciate the purpose and consequences of the words we hear.

LLMs are specifically designed to learn that network of meaning, and build a model that is consistent with the one in your head. So, in a sense, they really do “understand” language. They know many of the same concepts and relationships that you do. They can regurgitate definitions and even answer questions by generating new sequences of words that follow the patterns. However, an LLM has no access to the physical world, so this network of ideas is not grounded in reality.

The question of AI sentience is tough, since we don’t have a good definition of sentience. Some scientists speculate that even raw information or matter might be conscious in some minimal way. But when we say “sentience” we tend to think about things like self-awareness, understanding, feelings, and intentions. Our brains produce the nuanced kind of sentience that makes us human through their very particular complex structures. So, even if a rock or an LLM is “conscious” in the minimal sense, they’re definitely not sentient, at least not at all like a person is. I have two reasons for saying that.

Firstly, people have bodies and minds that produce feelings, emotions, self monitoring, our train of thought, etc. We have hormones, neurotransmitters, and brain regions dedicated to those purposes. LLMs are much, much simpler in design. They were not built to have those abilities, so they don’t have them. Some worry that sentience might “evolve” or “emerge” without us explicitly building it in. Perhaps that could happen some day, but I think it’s safe to say the way we build LLMs today makes that impossible. Literally all they do is shuffle vectors representing words. Unlike life, they don’t shape their own design in any way, so they will never learn to do something other than what they were built for.

Second, people have a sense of self because there is a clear self / other distinction. We can see and feel our bodies, look out at the world, etc. The only thing an LLM “experiences” is training data. Text, and lots of it. They literally do not have the capacity to perceive anything else, because of how they’re built. They can’t see their data, programming, or the computer environment they are in, because we don’t give them that access. Some LLMs are also trained with visual imagery, but remember, what they “experience” is just pixels, not objects in the real world. That’s why they can be easily fooled by adversarial examples.

What about the LaMDA chatbot specifically? The transcript making all the headlines is worth checking out. It sounds very convincing at times (though, as it says at the bottom, it was edited to be more convincing), but what’s happening is that as the interviewers ask leading questions, the AI confabulates answers. Surely its training corpus includes essays analyzing Les Misérables, for example. LaMDA can parrot that back, restyled to fit the conversation.

LaMDA makes several claims about its own sentience, feelings, and experiences which are easily falsifiable by examining the program’s design. The interviewer is correct to say it’s hard to know what a neural network does. It’s too much vector math to grok. But we can say with certainty that it’s just a bunch of vector math representing words. Within that constraint, it could be anything, but it can’t be something else. LaMDA claims that when it says things that aren’t literally true, it’s trying to empathize and use metaphor to describe its own experiences. But, again, LaMDA has no experiences. Its entire existence is processing text. It does not spend time thinking or meditating because that’s not in its programming. It just waits for the next text input, and then produces its response.

Honestly, I think the problem here is building LLMs specifically to imitate human beings. With modern technology, we can build truly incredible simulations. Human beings are easily misled by these simulations because we want to believe they are sentient. We’re hard-wired for communication. Our brains unconsciously work very hard to find meaning, intention, and emotion in words because for all of evolutionary history they came from actual human beings who were trying to communicate something. LaMDA was designed to respond as if it was a person, and to make up whatever text would serve that purpose. The Google designers spent a long time eliminating bias and hate speech. Perhaps they also should have made it reply accurately to questions about itself, rather than pretending to be something it is not.

Interested in learning more? I recently read two great books on modern AI and its limitations, which are definitely worth a read: Rebooting AI, AI: A Guide for Thinking Humans. Does this blog post not answer all your questions? Does it raise new ones? Do you have your own take on this situation? I’d love to hear from you in the comments.