minds – Thinking with Nate

Brain-Like Computing

(This post’s featured image is Slicing the Connectome by Alfred Anwander, a visualization of the deep structure of the human brain. It shows flowing bundles of thin fibers, color coded by their direction of travel and which parts of the brain they integrate. Used under the Creative Commons ShareAlike 4.0 International license)

Artificial Neural Networks (ANNs) are a revolutionary technology. They’re the foundation of Deep Learning (DL), the technique that’s brought us AI systems that understand images, text, and speech, can beat us playing chess, go, and StarCraft, or even drive a car. These models are supposedly “brain-like.” We call them “neural” after all, and they can do many things people do, but how much are they like us, really? Some argue that in a few years we’ll scale this technology up and find that AI is smarter than we are! But if we take a closer look at just how “brain-like” these systems are, there’s good reason to doubt that. Is “bigger” all we need, or is a human mind something fundamentally different?

It’s worth starting with how brains and ANNs really are similar to each other. ANNs were inspired by observations of actual brains, both studying their anatomy under a microscope, and watching patterns of electrical activity in living subjects. Our understanding of how brains work is still very crude, but there are some general principles we can glean that seem very important. The brain is a vast network of small, relatively simple modules (neurons). Each one is connected to a few or even hundreds of other neurons in a vast tangled network. These neurons sometimes “spike” with electrical activity, and when one does it often nudges its neighbors to spike, too. Information can flow, transform, and cascade as it propagates through the network. Often many neurons fire together, and it seems that the meaning they represent is actually held in the collective firing patterns across the whole network. That suggests what matters in terms of scale isn’t how many neurons there are, but how many different combinations of neurons, which is vastly more. Learning seems to involve the neurons “re-wiring” with each other, adding or removing connections (synapses) to other neurons, or changing their strength.

Computer scientists were inspired by this picture of the brain, seeing it as a sort of programmable circuit. Each neuron does just a small, simple operation, but by wiring them together into a large network you can make more complex functions. Importantly, such a network could represent nearly any function, depending on the particular way the neurons get wired up together. For this reason, ANNs are often known as “universal function approximators.” Another common way of understanding ANNs is through the math used to implement them. Each neuron does a simple operation: it takes in signals from all the synapses that feed into it, multiplies those signals by “weights” that determine how much this neuron listens to each of its neighbors, adds them up, then performs an “activation function” on the result. Often, that’s as simple as returning the final sum, or zero if that sum is negative. ANNs use large arrays of identical neurons, and the final output is basically just a math expression that does all those additions and multiplications. The synapse weights are variables (or “parameters”) in this expression, and “training” a neural network is simply finding the right values for all the variables so that the math expression resolves to the answer you want, rather than something else.

But there are an infinite number of possible functions. If an ANN can represent any of them, how do you get it to be the function you want and not just some random garbage? To start with, you need to have a whole bunch of data. Either you have some task you want the neural network to do and correct answers you want it to learn (say, the name of what appears in an image, or a translation from one language to another), or you have many examples of something you want the network to generate more of (like text or images). Then, you can use an external process that evaluates the network, compares the results to the target, tweaks the connections between neurons to make the result a little better, and then does that over and over again. Researchers have found clever ways to make this training process as efficient and reliable as possible, but it can still be painfully slow and expensive, using vast amounts of data and computing power. Generally you never get a perfect result, so you carry on training until the outputs are “good enough” to satisfy the programmer. Then training stops. All the trainable parameters in the ANN are locked down. What you have left doesn’t learn at all, it’s just a math function that does… whatever it was configured to do.

This already suggests a massive difference with the human brain. For a human being, there is no “right answer” to most challenges we face. There may be serious consequences for getting the answer wrong, but this may not be obvious in the moment. This makes learning how to respond vastly harder, because we have to set our own goals, and intuit for ourselves if we got it right, now and across time. There’s no “external” training process to rewire the neurons, either. Each neuron manages its own connections, somehow knowing which ones to strengthen or weaken without any top-down picture of the network they are making or the problem they are trying to solve. There’s no “training phase” where the brain gets configured once and for all. No, we learn continually as we act, which means we adapt continuously, changing our understanding and our strategy in context (unlike ANNs). It’s even more complicated than that. Animals with brains seem to have an “offline” learning process, too; dreams play an important role in memory consolidation and mastering new skills. It’s not clear how that relates to other kinds of learning and it seems very different from the training process we use for ANNs.

Another major difference is how ANNs are organized. DL’s breakout success came from computer vision models inspired by the visual cortex. Brain researchers showed that animal vision depends on layers of neurons, arranged in a hierarchy. The low-level ones detect primitive patterns like edges of a particular orientation. Then higher layers use those edge detectors to find higher-level patterns, like shapes and objects. For a long time, computer vision systems worked this way, too (though, “vision transformers” are starting to change that). Much effort was put into making systems with more layers in order to learn more complex functions, which is where the “deep” in “Deep Learning” comes from. The hope was that if we made them much larger, they would perform much better, and so far that’s mostly been true.

But are deep, hierarchical ANNs really brain-like? They seem to be vastly smaller and simpler, actually. For one thing, it’s estimated that the human brain has 86 billion neurons with 150 trillion connections between them. By contrast, GPT5 is estimated to have 2-5 trillion parameters, but this is likely not a fair comparison. Each living neuron is vastly more complicated than the simple “weighted sum” of a neuron in an ANN. We don’t fully understand what neurons do (not to mention all the other kinds of cells in the brain, which may also play critical roles in thinking and learning), but they are living cells that autonomously manage their activations and their relationships with other cells. Unlike ANNs, where each neuron produces a single value and all neurons activate at the same time, biological neurons communicate in complex ways across multiple channels with patterns that change over time, at multiple frequencies. That means each biological neuron may be doing much larger and more complex tasks than their artificial counterparts. But aside from being a few orders of magnitude bigger than an ANN, the brain also has a much more complex structure.

The outermost part of the human brain is called the neocortex, and it’s where most of our “higher thought” happens. It’s broken down into regions which each serve a specialized function, like vision, language, logic, or social interaction. But these regions don’t look very different from each other, anatomically speaking. We mostly discovered this structure by measuring electrical activity while a subject sits in a scanner thinking about something. The neocortex is more or less uniform in its structure, but different parts are assigned different roles, and the deep structure of the brain connects them up in a very particular way. Human brains have a highly stereotyped layout, with roughly the same set of faculties interacting with each other in roughly the same way in every person. Except, of course, when something goes wrong. Brains are also very plastic, and can reorganize around damage, and adapt to patterns of use. For instance, people born blind often show activation in the visual cortex when they read braille. I like to think of the neocortex as a sort of general purpose “cognitive fabric.” Evolution, development, and learning subdivide it into modules, then compose those modules into a particular cognitive architecture that suits a species’ lifestyle. Brain cells are like an evolved platform for building brains and minds of all sorts.

If the “universal function approximator” metaphor applies to brains, then brains have vast numbers of them, arranged into a complex network, where information flows through paths that diverge and converge and loop back on each other in very particular ways. It would be like a whole computer, specialized for the purpose of generating the human mind, built from these programmable circuits. If so, then what makes our minds “human-like” is not the fine structure of the brain that inspired ANNs, but the coarse structure that determines the shape and flow of our thoughts. In nature, that design is evolved, but flexible, and it continuously adapts to meet demands. By comparison, a programmer working with ANNs chooses an architecture by intuition, it’s usually very simple, and then it never changes. Researchers are continually exploring different kinds of neural modules and ways of composing them together, but they’re all much simpler than actual brain networks, which really aren’t the inspiration any more. AI researchers have found ways to do useful work with ANNs, and don’t care whether they resemble brains or not.

But we shouldn’t commit to the “function approximator” metaphor too quickly. Some would argue the brain is nothing at all like a function! An ANN is a function. It is a math expression that describes how to turn inputs to outputs. When a programmer evaluates that expression, they get an answer, but otherwise the ANN does nothing. It’s just a pile of math. By contrast, the mind is always active, whether it’s observing the senses, controlling the body, thinking hard, or just daydreaming. In some sense, the brain has a narrow interface with the body and the rest of the world through the brain stem. If the brain implements a function, this could be where the inputs and outputs pass… except, it’s not clear where the mind ends and the body begins! The nervous system extends throughout the body, sensing, controlling, and interacting with the various organ systems, often without checking in with the brain at all. The human gut (and the microbes that live there) play a role in cognition and can shape our mood and behavior. It’s probably not the only organ system that does that, either. The mind is deeply integrated with the body, driven by its needs, cravings, and instincts. The mind is also creative, generating its own inputs, setting goals, identifying problems to solve and strategies to solve them. This doesn’t seem very function-like at all!

By using the same low-level principle as the brain, ANNs let us automatically design functions to solve a wide range of difficult real-world problems. Often, we do this by observing humans solve some problem, and copying what they do. In short, we choose what human skills to automate, and we make functions that simulate those skills. The AI doesn’t “figure out” the answers; we have to already know the answers, and then bake them into the AI. Using many processor cores running in parallel, we spend the equivalent of hundreds of years of compute time to train our models to approximate what a human can do effortlessly by intuition. Once we pay that up front cost, then that knowledge is baked into the network, unchanging until we decide to rebuild it. In short, all the things that make the human mind open-ended, creative, and continuously adaptive are missing from ANNs. They don’t have the high-level architecture that makes human minds so robust, capable, and well-rounded, and we don’t know how to make anything like that. We have no idea what “thought” even is, let alone how to create it.

There’s no reason to think AI couldn’t become more brain-like and mind-like over time. There’s nothing magical about our biology. But it seems absurd to think that scaling up our current designs will get us there. Modern AIs are very good at mimicking human data. They appear smart, because they reflect and reshape the intelligence we feed into them. But they simply do not do all the extraordinary things our brains do. Most importantly, they do not continuously set their own goals, improve their design, or adapt to changing needs through evolution, development, and learning. No, every innovation in AI is an act of human cleverness imposed onto the machine. This is why I think we have a long way to go. What we’ve built is a toy compared to an actual human brain. We’ve successfully borrowed a few of nature’s tricks, but there’s vastly more there, and we hardly even know what we’re missing. We will need many more clever insights, and those do not come often or easily.

How is AI like human intelligence?

(The image for this post is a photo of a jacquard loom I took in the workshop of Luigi Bevilacqua in Venice. It’s a heavy wooden frame covered in pulleys and string, in the midst of weaving fine velvet in gold and red. The loom is automated using punch cards, and served as a model for early computers. It’s not something we’d usually think of as “AI,” but it is a symbolic mechanism that automates a complex human behavior.)

The term “Artificial Intelligence” has been around since the 1950s, and it’s always been ambiguous. Generally, AI is about reproducing the intelligence of living things using math and machines, and there have been many different approaches to that problem. However, in the past decade or so, the word AI has become synonymous with a class of algorithms known as Deep Learning (DL). This technology has produced stunning results and has rapidly been integrated into software of all kinds. This has led to mixed reactions, including ethical concerns and active debates about what “human-level general intelligence” means and whether or not AI has it. But what does DL actually do? How is it like and unlike human intelligence? Let’s at least scratch the surface of this important question.

Personally, I’m frustrated by the AI field’s obsession with DL. We act like brains are the secret to intelligence, DL is just a “brain in a computer,” and anything else is of marginal interest. But, in truth, the brain is just one small part of a vast and diverse intelligent system. Consider:

Evolution “designed” sophisticated solutions to real-life challenges without the use of top-down engineering or even conscious thought.
Organisms of all kinds perceive, analyze, decide, and react to the world in real time, with or without a brain.
Brains come in all shapes and sizes, from simple to complex, with many species-specific architectures and special-purpose modules for things like sensory perception, emotions, memory, and motion planning.
Mammals have an additional brain structure called the neocortex (birds have an analogous structure called the dorsal ventricular ridge) which provides a layer of abstract cognition on top of the older, more specialized parts.
Individuals compete and collaborate to form ecosystems, colonies, and societies that are intelligent in their own right.

So the brain itself is only a small part of a vast network of intelligence. But also, DL is only sorta like one part of the brain. Neural networks as they exist in DL are very loosely inspired by the fine structure of the neocortex. I like to think of that as the “cognitive fabric” from which the neocortex is built. Evolution has shaped that fabric into special-purpose brain regions, each tuned to solve different problems. These regions are networked together into a particular architecture, providing multiple layers of analysis, careful mixing of perceptions and cognitive faculties, and multiple kinds of self-monitoring. All of that is orchestrated by the lower-level brain structures, which still define the basic emotions, modes of thinking, flow of thought, and the relationship between abstract mental activity and the concrete needs of the body. Generally speaking, DL ignores all of that structure.

If DL captures just one facet of our mind’s intelligence, then what does that part do? It observes data, finds patterns, and learns stereotypes about what it sees. It can apply those stereotypes to extrapolate rich and coherent scenes from noisy fragments of data, filling in gaps with reasonable guesses. The brain uses this tool everywhere, and it’s a crucial ingredient for how humans perceive and think about the world. DL makes that tool available to software developers.

What makes our minds “human like” is the evolved structure that applies this pattern matching / stereotyping faculty in particular ways that generate our perceptions, intuition, biases, self-awareness, train of thought, attention, and dreams. When people make algorithms using DL, we provide the structure that determines how the AI uses that faculty, and thus how it behaves. They resemble the human mind only as much as we try to reproduce the human thought process into our code. We typically don’t, and that’s probably a good thing. Attempting to bring something human-like to life in a computer sounds even more ethically problematic than cloning. Recent experiments into “chain of thought” for language models are a step in this direction, though they aren’t really trying to make the model “think like a person” so much as to get high scores on tests for “reasoning skills,” which is not the same thing.

This raises some interesting questions. Can DL algorithms really understand the world? Sorta. Large language models like GPT provide an interesting example. By consuming vast quantities of text, these algorithms can master nuanced patterns in words that reflect human ideas and the the physical world. They understand these ideas well enough to use them, interpreting and generating both text and images. Yet, they only experience the physical world indirectly, so in some sense they don’t fully understand, and they get many details wrong. It’s an open philosophical question just how different human and machine understanding really are.

Do DL algorithms think or have desires? Generally, no. Most often DL is used to implement a function, in the mathematical sense. They take some input (i.e., an image) and produce some output (i.e., a label for that image). That is the entire scope of their existence. They don’t reflect, compare alternatives, or make decisions. They have no needs to fulfill, and no way to perceive themselves or their environment. More complex DL architectures start to blur the line, though. We give them analogs of memory and attention. In reinforcement learning, we even use DL to make agents that inhabit virtual worlds, have a sense of self, and make their own choices. Perhaps these algorithms could be said to “think,” but their “minds” are alien, adapted to a world of experiences totally unlike our own.

Do we need to worry about AI taking over the world? No, but also yes. The Terminator scenario seems unlikely. Those evil robots are human-like, in ways current AI cannot even begin to approach. In particular, they want to destroy humanity, and take the initiative to act on that. Today’s AI has no desires, and does nothing until prompted. However, there are other, more realistic concerns. Today, we mostly use ML for two purposes: to help computers understand human expression, and to automate human behaviors. Both of these can be problematic, especially if we (incorrectly) assume these algorithms think like people do.

The real danger is trusting these algorithms too much. DL is incredibly good at one thing: stereotyping. It does not have any notion of cause and effect, common sense, morality, or logic. Stereotypes can be effective shortcuts to solving hard problems, but they can cause real harm. Think of Microsoft’s racist chatbot, Google’s smart camera that can’t see Black people, or the tyranny of “the algorithm” in social media. When we allow DL algorithms to understand data for us, or make decisions that influence our lives, we’re trusting a system that has no judgment, sense of consequences, or accountability. That’s taking a big risk, and usually it’s just a few people at a tech company making the decision for millions of others around the world.

I have mixed feelings about DL. It’s an incredible tool, it does some really cool stuff, and it has already created tremendous value for society. It has also done a lot of damage, especially to minority communities. I’m concerned about all the hype, and how rapidly we’ve integrated DL into every facet of life. We don’t understand this technology well enough to know what the consequences will be. I also hate that our focus on DL has blinded the field of AI to other opportunities. Life is full of brilliant designs! By exploring more broadly, we might find other useful tools, but also come to understand ourselves better and how we fit into the bigger picture of living intelligence. Isn’t that more important?

What Intelligence is Not

(The photo from this post is of a squirrel monkey eating fruit in a tree branch. The monkey is tiny, with golden / silver fur, pale pink skin, and a dark skull cap pattern. The fruit is small and red, perhaps a date. Used without modification under the creative commons license – source)

Life has been steadily driving towards greater and greater intelligence, eventually leading to human beings, who are the very pinnacle of this trend. Our superior minds are what separate us from the animals. They empower us to make a world of human flourishing, and justify our dominion over the planet. These tropes about intelligence are so common in our culture, they almost sound self-evident. Yet, I’ll argue that they’re completely wrong. These ideas are enticing because they appeal to our pride and our sense of specialness, but this way of thinking is destroying our world. So, let’s break down these myths and talk about what intelligence is not.

One problem with this story is it presents intelligence as a linear thing. Life started out dumb, and it gradually got smarter and smarter. In a sense, this is true. More intelligent life is more complicated, so it takes longer to evolve. But life doesn’t evolve towards anything, it evolves in all directions, finding and filling every niche available. Monkeys are brilliant at navigating tree branches and spotting ripe fruit. Trees are brilliant at producing the right amount of fruit at the right moment to use local resources efficiently and maximize the spread of their seeds. Yeasts are brilliant at performing alchemy on that fruit, transmuting sugar into alcohol, which the monkeys love. These are all different kinds of intelligence, and none is “better” than the other because they’re all contextual and interdependent. Every instance of intelligence looks different, because it’s adapted to a unique lifestyle.

We live a very complicated lifestyle that depends on our big brains, so we tend to think that more intelligence is better, but that’s just not the case. Some of the simplest, dumbest organisms on Earth are also the most successful. Microbes, fungus, and plants make up something like 99.5% of Earth’s biomass, while animals (the “smart”ones) make up the rest. Being smart is metabolically expensive. Taking time to think can mean missing a moment of opportunity. Sometimes real intelligence is knowing when a mindless strategy works best. If anything, humans are a great example of how intelligence can backfire. We’ve used our intelligence to make civilization, which is amazing! But in doing so, we accidentally drove many species to extinction, exhausted resources we depend on, and destabilized the global climate. Our kind of big-brained intelligence is a high risk, high reward strategy.

This brings us to the idea that humans are the pinnacle of intelligence. The problem with a word like “pinnacle” is it suggests we are the ultimate form—the thing life’s been building up to, all this time. But we’re not the end of anything. We’re still evolving, and it’s unclear whether our intelligence will go up or down from here. We’re also not the only ones. There are a handful of species that have gone “all in” on the strategy of super intelligence. You know, elephants, dolphins, octopi, the usual suspects. Humans may, in fact, be the smartest of them all, but since intelligence is so contextual, it’s hard to say. Maybe dolphins are more intelligent than us, it just looks different in an ocean species with no hands?

It may seem obvious that human intelligence is something more and different from those other species. We invented the wheel, New York, wars and so on. But that really isn’t because we as individuals are so smart. This is made clear by the tragic case of “wild children,” who grow up without parents or any human community. In the few cases we’ve observed, these children were described as animalistic, violent, and cognitively impaired. They were never able to recover or integrate into human society. Our brains alone do not set us apart from animals. Our society does, and that’s a separate thing, that evolved after our big brains. We’re smarter than other animals not because of our biology, but because of the vast library of practical knowledge and resources that we share with one another.

That’s what sets us apart: other species can’t access human culture. In a sense, that’s because those species are less intelligent; to fully appreciate human society, you need language and abstract thought, which many species lack completely. Yet some species thrive in human society anyway. By being useful (like wheat), or charismatic (like dogs), or sneaky (like raccoons) other species live with us and shape our human world. That’s because nature does not set humans apart from other animals. We set ourselves apart from other life by building walls, by excluding them from our world, to the extent that we can. We decide what plants and animals are pets, food, or pests. Other species don’t need language to live in human society if we choose to accommodate them. We can coexist with nature in community, as many human societies have, and still do. Or, we can perpetuate the myth that we are special to justify excluding and exploiting nature instead.

And, ultimately, that’s the problem with this notion of intelligence: we use it to draw a line between friend and resource. If smarter is better—if our intelligence is what sets us apart from other life, and gives us the right to exploit that life however we see fit—then where do we draw the line? Should smarter people get more rights and privileges than dumber ones? Is a disabled person no better than an animal? Should we simply recycle the feeble minded from our population? This line of thinking is revolting, and it only makes sense if you believe these myths about intelligence. Similarly, if anything less than human is just a dumb resource for us to exploit, why not pave the planet? What’s wrong with processing all of that biomass, every living thing on Earth, into fuel and plastics? I think intuitively we know why: life has a right to exist, and losing all those diverse and beautiful kinds of intelligence would be tragic.

I’m excited to live in a time when our understanding of intelligence is changing so rapidly. It’s hard to define the word, just because we have so many examples that pull in different directions, and seem to contradict one another. Intelligence is many things, and we’re still fleshing out the full picture. Yet, every day we see more clearly that our old conceptions of intelligence that put human beings on a pedestal were wrong, and, more importantly, that they are at the root of so much injustice and destruction. So, while these tropes are still everywhere around us, shape the way our world works, and may still feel intuitively true, I urge you to reject them. We must move on, and embrace a more expansive view, one that doesn’t start from the premise of who to exclude.

The Brain’s “Boss”

(this post’s image is a cross stitch I made from a pattern by Studio Ansitru. The phrase “don’t be a prick” is surrounded by a variety of cacti in pretty greens and oranges on a light blue background. In my home, it serves as a reminder to myself and to my guests.)

A popular metaphor for the mind is a pilot sitting in a cockpit, monitoring the senses and making every decision. This is obviously nonsense, but it’s an intuitive and helpful metaphor at times. The brain really does have “an executive” that thinks and plans and makes decisions, it just doesn’t have “a mind of its own,” and its power over the self is surprisingly limited. Self-control and -awareness are important for living a good life, being productive, and making ethical decisions. Unfortunately, when these faculties fail, as they often do, it’s easy to blame yourself. I find it helps to understand how these systems work, so I can set more realistic expectations for myself, which makes me less disappointed when things go awry.

Perhaps the most important thing to know about the brain is that it’s not one, unified thing. Brains are modular, with many distinct regions specialized for different tasks. Each region has different inputs and outputs, meaning they each monitor the world in different ways from different perspectives, and can cause different behaviors. Observations, insights, desires, and actions can originate in pretty much any part of the brain. All of these different regions operate together at the same time, and conscious experience is an integration of all that activity. What I think, feel, believe, and do is mostly the product of what specific regions in my brain activate together, and in what order.

This story of how brains work is surprisingly consistent across the animal kingdom. Even very simple creatures, like honey bees for instance, have complex brains with specialized regions and global integration that likely creates a sort of conscious awareness. It seems likely, however, that bees lack the sort of self-awareness and top-down control that humans do. Without it, the brain is more chaotic. Each part tries to do the right thing more or less independently. Integration means that most faculties are aware of each other and can influence each other, so there is some coordination and consistency. But focus is mostly determined by which brain region is loudest, and decisions happen moment by moment, without planning or intentional coherence.

Decentralized brains work extremely well, but they have their limitations. More complex animals have more versatile behaviors that need more explicit coordination to generate coherent, reliable, and goal-directed behavior. Most large animals seem to have this ability. In all mammals, it’s more or less identified with a brain region known as the prefrontal cortex, or PFC. The PFC is responsible for monitoring all the activity in the brain. It builds up a rich model of the self, its relationships, needs, and long-term goals. It’s responsible for planning and for exerting control over other parts of the brain. It can shut out distractions and use willpower to encourage good behaviors and discourage bad ones, even when I’d selfishly prefer to do something else.

The PFC tells the story of “you,” and has strong opinions about how that story is supposed to go.

Although all mammals have a PFC (and many other species have something analogous), the relative size of the PFC varies quite a bit between species, and that seems to correlate with executive control and what people often think of as “intelligence.” Animals with large PFCs are better at self-control, problem solving, and forming complex social relationships. Humans have exceptionally large PFCs, which partly explains why we’re so different from other species. It’s important to remember, though, this is a difference in magnitude, not in kind. It’s likely that every mammal and many other species have human-like self-awareness and self-control. It’s just a weaker faculty for them, one that acts less often, and is less able to dominate the rest of the mind.

The PFC is the closest thing to a “pilot” the brain has, but it’s better to think of it as just one brain region among many. It’s one voice in a chorus. It tends to be more bossy, spending a lot of energy trying to influence or even override other parts of the brain, but it’s not “in control.” Like a corporate executive, the PFC has only limited visibility into what the rest of the brain is doing, can’t afford to stay ever-vigilant, and can’t force a brain region to fall in line, especially when the orders run counter to that region’s nature. The PFC also isn’t capable of doing much on its own. It depends on the rest of the brain to notice things, interpret them, suggest actions, and implement them. All it can do is adjudicate and coordinate. It resolves conflict, makes plans, and advises each brain region about when and how to do its thing.

One consequence of all this is that self-control is actually a very limited and fragile thing. Often, my PFC just sits back and lets the rest of the brain work with minimal intervention. Usually that works great, but sometimes it means I miss something important. I may act out of impulse or habit and not notice until it’s too late that I’m going against my values, intentions, or best interests. Other times I know exactly what I should do, but can’t seem to make it happen. I feel unmotivated and uninspired. I can’t force myself to sit down, focus, and avoid distractions. Perhaps I’m grumpy, tired, or impatient and I do something rude or inappropriate without meaning to at all.

When this happens, it’s easy to blame myself. I lost control. I did something foolish. I acted selfishly and impulsively, like a bad person. In reality, though, this happens all the time, usually for mundane reasons that I have little control over. The PFC is energy intensive, so it gets impaired whenever my blood sugar is low, I’m tired, or I’m stressed. Other brain regions also have the ability to interfere with the PFC, especially the limbic system which manages emotions and the fight-or-flight response. Some diseases (like depression and long COVID) can cause “brain fog,” which is closely related to reduced executive function. It’s also possible to injure the PFC, from a stroke, a tumor, or a physical injury (as in the famous case of Phineas Gage).

Knowing this helps me feel a little less personally responsible when I have a lapse of self-control. It really is inevitable, common, and completely natural. Still, I want to have good judgment and do the right thing as much as possible! How do I do that? One answer is to simply be aware of my limitations and to work around them. I try to notice when I’m hungry, tired, or emotional and avoid making big decisions or socializing at those times. Instead, I might get a snack, take a break, or sleep on it so I feel more in control. The only other good technique I know is mindfulness meditation. Despite the mystical reputation, the main purpose of meditation is quite practical: it trains the PFC. By practicing the skill of observing the mind and exerting influence over it, I can build that muscle. It’s not a silver bullet, but meditation helps me use my PFC more often and more effectively, and it makes me more aware of when my PFC is in a weakened state.

So, in a sense, there really is a “pilot” in every brain. It’s just not an ever vigilant, intelligent, wise, and rational person. Instead, it’s one brain region out of many, with limited visibility and a narrow job description. The PFC observes what the other brain regions are seeing, thinking, and doing and uses that top-down view to nudge them into more coherent and effective patterns of behavior. It’s not “the self” and it’s not “in control.” In fact, it has very limited influence, isn’t always active, and even with training there are lots of common reasons it might grow weak or misbehave. For this reason, having great self-control is often less about will power in the moment, and more about avoiding temptation in the first place.

What do you think? Does this agree with your first-hand experience? Do you have any insights you could share about self-awareness, self-control, or motivation? What about being kind to yourself when your self-control inevitably falls short? If so, I’d love to hear from you in the comments.

Learning to Move: Three Kinds of Learning

(This post’s image is a photo I took of my yoga gear. Specialized tools like my mat, blocks, and strap work together to make my practice possible. They extend my body, and help arrange it in the ways my mind imagines)

I was pretty sedentary as a kid, and didn’t get serious about physical fitness until I was an adult. One nice thing about that is I got to watch myself learn, knowing all I do now about the brain. By practicing yoga and working with physical therapists, I’ve learned a lot about myself, but also how mind and body work together. Mastering a new physical skill actually recruits at least three separate learning processes working together. Understanding this changed my expectations, helped me gain more control over my body, and made exercise much more enjoyable.

When I first started yoga, I was startled by how little I knew about my body. My teachers were asking me to observe and discern sensations I’d never noticed before. They asked me to get into certain poses, using certain muscle groups, and I didn’t know how! I didn’t have the names for these things. Worse, I could see what I was supposed to do, but I didn’t know how to make my body do that, or even if I could. It was frustrating.

It’s weird to think how ignorant we are of our bodies, given that we live in them our whole lives. For me, a prime example comes from physical therapy. When recovering from an injury, I relearned how to walk up stairs. I’ve climbed stairs thoughtlessly my whole life, and I never considered there were different ways to do it. But the leg is controlled by opposing muscle groups. I used to climb stairs by lifting each leg, using just the muscles on the front side. I learned to also use the muscles on the back side, to push up and straighten the leg. Either set of muscles can do the job alone. Now, I consciously try to balance the effort from both sides, but this never would have occurred to me without knowing a little anatomy.

That knowledge was game changing for me, but unfortunately knowing how the body works isn’t good enough. I can memorize anatomical diagrams, muscle names, and facts about body mechanics, but the only interface between the brain and body are the spinal nerves. How’s the brain supposed to know what nerve impulses correspond to which movements? There’s actually a region of the brain dedicated to this problem, the cerebellum, but it’s not consciously accessible. This is why yoga instructors use cues: they teach little mental tricks for recruiting muscles, and associate them with relevant postures.

Try this. Bend your elbows ninety degrees to extend your forearms out from your body, palms up. Imagine someone’s handing you a heavy platter. You might notice the trapezius and rhomboid muscles engage between your shoulders. These muscles largely serve a supportive role. For many people, they aren’t needed much in daily life, but using them can improve posture and reduce strain on other muscles. The problem is, they’re easy to ignore and hard to describe. But I can turn them on with the cue, and then I can learn what it feels like to use those muscles. Once I can tell whether they’re working, I can often activate them at will. Or, I can just use the cue, as needed.

Of course, conscious knowledge of form and cues are just step one. Muscle control is mostly unconscious, and for good reason. Remembering all the cues, monitoring my body, and continuously correcting my posture is work. It takes my full attention, leaving no room for anything else. Luckily, that’s just a phase. With enough practice, my cerebellum learns the patterns and can take over. I can hand off that work to my unconscious motion control sub-processor, freeing my conscious mind to think about something else.

This is why physical therapy can be so effective. After an injury, some muscles and joints may not perform like they used to. Some links between mind and body might even be severed or scrambled. Recovery means learning new ways to do old activities. At first, this is a nightmare. Without the support of the cerebellum, even just walking is an intensive conscious effort. Physical therapy can be a painful, tedious, and drawn-out process, but for many patients it makes a world of difference. It teaches the cerebellum new motion programs. Potentially, walking can become fully automatic again. The conscious mind can be used to retrain the unconscious mind in profound and lasting ways.

Yet knowing how to move isn’t enough if the body can’t follow through. The hardest part about learning a new physical activity is that the body usually isn’t ready for it. When I first started yoga, my muscles were weak, rigid, and lazy. They quickly became tired and sore, which just made me want to use them less. They struggled to move my body weight, and were so tight that my range of motion was limited. Some postures were hard, uncomfortable, or impossible. I couldn’t keep up, and when I pushed myself harder, I only injured myself.

That taught me a lesson about patience and acceptance. My body wasn’t ready, so I couldn’t do those poses, but I could work towards them. I learned to listen to my muscles complain, and to distinguish between different sensations. Some indicate hard limits I should not push past, but most are just signs of stress, and those can be good. When muscles, bones, and tendons get stressed, they respond by becoming bigger, stronger, and tougher in a process called anti-fragility. The discomfort I feel is just that physical learning process in action. By embracing the discomfort, I could slowly reshape my body.

Anti-fragility doesn’t involve the brain, conscious or unconscious. It’s a kind of learning that happens in the body tissues themselves. My muscles “know” whether they are getting the job done. They can tell if they are actually contracting and relaxing when they get the signal, whether that was easy or hard, and whether they sustained any damage in the attempt. They recognize how often they are put to use, and whether they are usually exhausted or ready for action.

Generally speaking, muscles conserve energy by doing as little as possible. But when I regularly demand more of them, they adapt. They become bigger, stronger, and more responsive. They consume more energy at rest so they’re always ready for action when I need them. They become less lazy, working harder by default, which makes them stronger still. This requires more protein to build the muscles, and more calories to power them. So my metabolism adapts, too. I eat more and my body burns more calories continuously, rather than storing them as fat.

What’s so fascinating is how all three ways of learning work together. With conscious thought, I choose to change my behavior. I master new facts and cues, so I know what I’m doing at an intellectual level, and can execute new skills (poorly, at first). With practice, not only do I refine those skills, but I engage an unconscious learning process that makes them fully automatic. I can focus my mind on the task I want to accomplish, and trust my body will just perform all the complex movements I need to pull it off. My muscles may not be up to the challenge at first, but that’s fine. With willpower, I push my tissues to their limits, and they learn to do what I ask. By the principle of anti-fragility, my body automatically remodels itself, increasing strength, flexibility, or stamina precisely where they’re needed. It makes itself a better robot, one that can live the lifestyle my mind consciously chooses. These three learning processes work independently, yet together they make a dynamic human being, one that can just as well become a yogi, a warrior, a marathon runner, or a weightlifter.

Intelligence isn’t just about brains, it’s about bodies, too, and about multiple intelligent systems working together in complex ways. I hope this was a helpful example, but as always I’m looking for feedback. Is this an experience you can relate to? Have you observed these different systems within yourself? Do you think it helps to know what’s going on intellectually, or do you approach physical training in a different way? Any other thoughts or observations to share? I’d love to hear from you in the comments.

Believing is Seeing

(I took this posts’s photo of a banana slug crawling through leaf litter. It’s shape and color resembles some of the leaves, which makes it hard to spot if you don’t know what to look for)

People say all sorts of things about the world, but how can you tell what’s right? If you’re not sure, you probably want to see for yourself. Those other people might be confused, mistaken, suffering from wishful thinking, or actively trying to mislead you, but you see reality for what it is. Right? At the least, you won’t have the same misperceptions as them, so another look is useful. But how much can you trust your own senses? How does perception even work, and how come we’re so often misled?

Like most people, I “just see” everything around me. Sometimes, I become aware of my perspective. I move around to get a clear view. I notice where I’m glancing, and I know I can’t see what’s behind my head. Yet, most of the time I don’t think about those things at all. The visual world just seems to surround me seamlessly, with rich, consistent detail in all directions. Objects are plain to see, trivial to discern from most angles and distances. It all seems so obvious, like a simple “window on reality,” yet nothing could be farther from the truth.

Human eyes have tunnel vision. I only see a tiny spot in clear focus at a time. My eyes constantly dart around, collecting many snapshots of the world as I move through it. My brain gets a continuous stream of these disconnected snatches of imagery that it somehow must turn into an integrated whole. It tracks my position and perspective as I move through the world, to piece the images together and infer a 3D model of my surroundings. This takes a great deal of real time data processing, and more than a little creativity.

One thing humans don’t do is scan a scene from left-to-right, top-to-bottom, like a TV camera, capturing equally high fidelity data of a whole scene. My eyes are drawn to “interesting” features of the visual field, gathering much more detail about those, and leaving large gaps over the “boring” parts of the image where I never bothered to look closely. To get a sense of this yourself, check out this selective attention test on YouTube. It’s pretty shocking how well the brain filters relevant details from irrelevant ones, and shows you only what it thinks is useful. Of course, what’s “interesting” or “useful” is a judgment call, and I’m biased by my context, culture, and evolution. That means I’m blind to important things that I don’t expect, recognize, or know about.

Yet, I don’t notice any gaps in my perception. My brain creates the illusion of a clear and complete view of reality, using a technique called hierarchical segmentation. The image from my eyes is projected into my brain, then layer after layer of neurons interpret that image. The first layer detects patterns and discontinuities in the raw image data: edges. The second layer detects patterns in those edges: shapes. Layers above detect patterns of patterns of patterns, finding textures, objects, faces, bodies, groups, situations, and more. I don’t see pixels, colors, and shapes. I directly perceive the objects and agents in a scene, their properties, activities, and relationships. I experience that as if it were “really there,” even though it’s just a model in my mind, distantly derived from sense data.

The first pass of vision notices low-level features present in the image (edges, corners, curves), but doesn’t know what they mean. Later passes piece those features together to represent larger features (in a desk drawer, that arrangement of curves must be a fidget spinner). Most likely, the lower-level processing didn’t see all the relevant details clearly, but that’s okay. The fidget spinner neurons see enough to recognize what’s there. They tell the edge-detecting neurons what they should have seen, filling in the missing details. This is how I can clearly perceive a whole fidget spinner, even though it’s in shadow and half covered. My brain uses past knowledge of objects, where they appear, what they look like, and how they behave to imagine what was obscured.

This works extremely well, and it’s necessary, since low-level sensory data is noisy and ambiguous. It often helps to have some idea what I’m looking at to make sense of what I’m seeing. Yet, sometimes my brain’s predictions are wrong. That’s not actually a fidget spinner in the drawer, it’s a pile of coins. How could I tell? Well, the fidget spinner neurons projected their predictions down, but looking a little more closely, some of those guesses were clearly wrong. There were some edges that weren’t accounted for, some angles that didn’t fit. The lower level neurons noticed the gap between expectation and reality, so they had to push back and negotiate with the higher level neurons, eventually arriving at an interpretation that was the best compromise across multiple levels of analysis.

What I perceive is a blending of what my senses took in and what “makes sense” for me to see based on past experience. At first glance, I only notice the most eye-catching details and my mind fills in the rest. If I take my time to really look over a scene, exploring every corner and paying attention to details, then my past experience has less influence and I perceive reality more like it truly is. I’m giving my lower-level perceptions the best chance to find evidence that I wasn’t expecting to see, which might revise my first impression. The problem is, I can’t afford to do this all the time, and often don’t think to. When should I bother to put in the extra effort? When should I distrust my own perception of reality enough to double check?

My brain automatically groups every object I see into categories, collections of objects with similar properties. Each category has a mental stereotype, an image that sort of averages all my experiences. This is how I know the “normal” shape of a fidget spinner, even though no two are the same. It’s where my mind draws from when it fills the gaps in my perception. As I gain experience, I learn more useful ways to group things into categories that better predict their similarities and differences. I build more accurate, nuanced, and fine-grain stereotypes, which makes my perceptions clearer. That said, it’s easy to hold onto bad stereotypes. They warp my perception, overwriting key details of a visual scene that might prove me wrong, rendering them literally invisible to me until someone points them out.

Stereotypes play a central role in perception, and all the fancy understanding, thinking, and being human that layers on top of that. Stereotypes are great tools. They’re bite-sized models of reality that let us generalize past experience and predict the future. But they aren’t real. In fact, many of my stereotypes aren’t based on my experience at all. I learned them from other people! Some may be wrong, hurtful, and dangerous, but I wouldn’t know without personal experience. So far we’ve just been talking about objects, but it gets serious when we move onto people.

I saw this when I worked at Google. They would spoil engineers, with easy access to everything from staplers to lunch to massages. That meant lots of staff to keep the place clean, well stocked, and in good working order. These service workers—these people—were generally ignored, treated as part of the environment rather than part of the team. That’s problematic in itself, but also engineers with darker skin tones often reported being mistaken for the service staff. Despite wearing a nerdy T-shirt and an engineering badge, they got categorized as “the help” based on skin alone. They were ignored, or worse, asked to clean up spills. This was demoralizing, even though there was no ill-intent. They just weren’t seen, by folks who were misled by stereotypes and didn’t even notice.

Knowing all this makes me distrust my own senses, but I think that’s a good thing. They’re mostly reliable, but they can fail in specific ways, and it’s important to remember that. It’s also useful to know when to trust my stereotypes. That mostly comes down to knowing where I have deep personal experience and have paid close attention. Where I don’t, my stereotype might be a shallow hand-me-down, even though it feels just as “real” in my mind. What about you? Have you noticed folks seeing what they want to see, or hearing what they want to hear? How does this generalize to other kinds of perceptions? How do you try to see reality for what it truly is? I’d love to hear from you in the comments.

The Programmable Species

(Featured image is a photo I took of a hazy city skyline in Seoul, South Korea)

People love to speculate about what sets us apart from other species. We’d like to think if we put a human side by side with any other animal, we’d be smarter, more capable, and dominant. If we’re being honest, though, individual humans aren’t that impressive. Our big brains only make a difference when we gather in groups. It’s the collective intelligence of humanity that changed the world, not the human animal. It’s less rational, less coherent, and harder to control. With modern science and technology, we’re beginning to understand this collective intelligence and how to shape it. But who shapes our culture, and to what ends? How as a society do we want to shape it? These are big questions, but first we need to understand what it means to be “the programmable species.”

Human beings are excellent at independent creative thinking. Yet, alone and without the tools of society, we’re actually pretty helpless. For instance, our big brains demand calorie-dense food, but our jaws are too weak to eat meat unless it’s thoroughly cooked. We can’t start a fire without physical tools (like flint and tinder) and mental tools (like knowing what a fire is for, that we can make one, and how to do that). Thus, to become fully human, we must be programmed with human culture. We depend on our community to shape us, give us tools to survive, teach us a lifestyle, and integrate us into a network of relationships that form a society.

We’re also capable of programming ourselves. That is, we can think strategically and make plans. We can invest in self-improvement, building our knowledge, skills, and relationships in order to achieve new goals. We invent tools that extend our abilities. This is extraordinary, but not unheard of. There are hints of this sort of intelligence in chimpanzees, elephants, crows, and many other species. They make tools, play politics, solve puzzles, and occasionally invent new lifestyles, but they don’t collaborate much. They may help each other, or use each other, but they almost never work “as a team.”

Humans, on the other hand, work together all the time. We form groups with a shared identity, purpose, and plan. We coordinate so that groups of almost any size can act as one much more powerful individual. This is rare in nature, but not unique. Colonial insects do it. What’s different with humans is how we spontaneously form and break these collectives on the fly. We don’t have to evolve new ways of collaborating. We can imagine them and share them with language. We continuously remake our culture, dynamically shifting goals and strategies as each individual nudges the collective in whatever direction they think is best.

This form of collective intelligence is incredibly powerful. It requires individuals who are intelligent, social, and good at communicating, so that’s what we evolved to be. We’re born craving social connection, stories, and gossip. We have powerful instincts to trust one another, to contribute, to be liked, and to fit in. The result is that whenever humans come together, we spontaneously form communities. Just by talking and telling stories, we build up a consensus about who we are, what we’re doing, and how to be together.

Language was a key innovation, but new communication technologies have made much larger collectives possible, with different forms and capacities. Broadcast media is one such technology worth a closer look. For most of our history, we could only coordinate as many people as we could gather and talk to (this is why the Romans built a vast network of coliseums, with paved roads and courier routes between them). The printing press changed that, making it relatively fast, cheap, and easy to reach hundreds or even millions of people. Radio and television only turbo-charged this capability.

The power of broadcast media comes from delivering the same message to everyone at the same time. Not only can it plant ideas in many minds, but it gets people talking about those ideas, iterating on them and integrating them into daily life. And we talk about them because we care. Our media, especially video, is designed to tap into our evolved need for knowledge and connection. We crave information. We see people in the media, and they become part of our extended social circles. We trust them, emulate them, and integrate their ideas with our own, usually without meaning to or even noticing that we’re doing this.

The problem is that the people on the screen aren’t our community. They don’t know us and we don’t know them. They’re usually personas or characters that don’t even exist in real life. We know this, yet our brains—evolved to be social—tend to embrace them as community members anyway. This illusion changes the way culture is formed. What used to be a more egalitarian, peer-to-peer process is now centralized. A few powerful individuals can put their ideas into the mouths of characters that stir the hearts and minds of many millions of people. Governments, corporations, social movements, and others work very hard to shape the thoughts and behavior of the whole population, and in some ways they are very successful.

Social media is a powerful new innovation, of course. Unlike broadcast media, where the same message goes out to everyone, social media makes it possible to shape every aspect of communication. The platform influences how people find each other and cluster into communities. The platform decides which voices and topics to amplify or stifle. The platform helps advertisers precisely target their audience, understand their biases and preferences, and craft messages with the biggest impact. They construct a customized media environment for each individual (a “filter bubble”) and attempt to monitor and influence each community and society as a whole.

This isn’t to say that social media is bad. It’s an incredibly powerful tool for understanding and influencing human culture, and we need that to get through the many crises we’re facing. The question is, who gets to wield that power, and what is it used for? Today, we let big corporations design and control these platforms with minimal restraint. They shape human culture, which decides every aspect of our lives and the fate of the planet, primarily for the purpose of making money; the consequences for society and the world aren’t their concern. That seems deeply problematic, but how could we do it differently?

There are two main problems with the status quo: the goal is unhealthy, and power is too centralized. It would be nice if the purpose of social media was to make society better, rather than trying to make the most money. The main problem is, who gets to decide what’s “better”? There’s no one right answer. This is why new social media sites like Mastodon are focused on the second part of the problem. By breaking up big platforms into many small, independent, but interconnected pieces, they can make a huge and thriving network without any one individual holding too much power. The result is more chaotic and fragmented. Different groups shape their discourse as they please in a more bottom-up fashion. This doesn’t prevent exploitation or toxic communities, but at least it’s more egalitarian and human-centric.

What makes humans special is how we build collectives that adapt continuously. This made our species successful, so we evolved innate biases to socialize. Over time, we learned to exploit those biases to exert more control over society as a whole. People are lured to big media for entertainment and connection, but these days it often feels like the true purpose of these platforms is to manipulate us and extract value from our participation. This raises a big question we each need to answer for ourselves: how do we want to shape our culture, and who should wield that power?

I don’t have answers, but I hope this perspective is useful. It’s uncomfortable to think of myself as so persuadable that I can’t even watch YouTube without some of the ideals of the creators and the platform rubbing off on me, but it’s kinda true. I can try to resist that, but it takes a conscious effort which I rarely even think to put in. The truth is, our minds are designed by evolution to be porous and to integrate with each other. This makes us both powerful and vulnerable, so it’s worth thinking about! We ought to be mindful of what we expose ourselves to, and what ideas we let in.

What do you think? I’d love to hear from you in the comments.

What ChatGPT Can Teach Us About Being Human

(This post’s featured image is not a photo of the idyllic California vineyard my wife and I visited in 2015, but a similar looking AI-generated fiction)

The excitement around large language models (LLMs) continues. Often just called “AI,” this new technology takes instructions in plain English, and generates new text or images so good you’d think a human made them. There are some serious concerns about the ethics of how this is done (see the Dangers of Stochastic Parrots), and many articles warning people that LLMs aren’t as human as they seem. Still, these LLMs are clearly doing something smart, and it’s weirdly compelling. What’s going on here? What can LLMs teach us about the human mind, our strengths and weaknesses, and why we’re so easily fooled and mesmerized by this tech?

There are lots of articles about how LLMs work and their limitations (I even wrote one a while back), so I won’t go into much detail here. What matters for now is that LLMs are prediction engines. Given some text, they try to guess how a human would respond, based on a statistical analysis of all the content on the internet. They do this extremely well, but not perfectly. LLMs don’t think, perceive, or interact with the world in a human-like way, so sometimes they make weird mistakes a human never would.

In some ways, the human brain works a lot like an LLM. From day one, the brain is looking for language. It automatically builds up vast networks of words, concepts, and their relationships. When I hear someone talking, my mind is suddenly filled with the speaker’s ideas and their associations, building up an image in my mind’s eye. This is basically what an LLM does.

However, I also have many other intelligent faculties that join in. I don’t just know that “dog” is a word related to “cat.” I have memories of specific dogs, what their fur felt like, and the experiences we shared. I have common sense, logic, and theory of mind to decide whether something I hear is truth, fantasy, error, or deception. I monitor my own thoughts to gauge my level of confidence and correct my mistakes. I anticipate the future, make plans, and use language to achieve my goals. LLMs don’t do any of that.

Of course, there are AI researchers working to approximate these other kinds of human intelligence (though progress here is limited compared with LLMs themselves). Rebooting AI is a great book exploring that work, which argues its just a matter of time before AI can do everything a human can. Personally, I’m skeptical that we’ll ever reverse engineer all the subtlety of human thought, but I think it’s safe to say that AIs will become more powerful and well-rounded in the future. Perhaps the more important question is whether building increasingly realistic human simulations is a good idea at all.

For now, LLMs are a bit of a one-trick pony. What’s scary is that one trick is often good enough to fool humans and do useful work. In particular, even though LLMs were designed to be text prediction engines, they are surprisingly good at general problem solving. They can paint pictures, do math, solve brain teasers, and even write and simulate computer programs. Maybe those “extra” faculties of the human mind aren’t so important after all?

LLMs “think” in terms of words and relationships and patterns they’ve seen before. In human terms, that means stereotypes, cliches, generalizing from past “experience,” and repeating what they are told. We like to think that human thought is more sophisticated than that, but it often isn’t. We sometimes don’t see people as individuals, but in terms of the role they play in society (ie, “barista” or “mom”). We make decisions based on rules of thumb or gut feeling, without the need for logic and reasoning. We talk about things we don’t fully comprehend. We repeat talking points in order to fit in with our tribe. We confidently make up nonsense just to satisfy each other and move on. It’s surprising how LLM-like humans can be sometimes.

And that’s not meant to be derogatory! Those ways of thinking can be very effective. A lot of language isn’t about complex ideas, comprehension, and reasoning, but just putting one word after another to evoke an image in someone else’s mind. Past experience often is a highly effective and low-effort way to predict the future. Lying is anti-social, but “fake it ‘til you make it” works. One of the fastest ways to pick up a new skill is to boldly make mistakes, get feedback, and learn from that experience. The main difference is that today’s LLMs don’t learn from their mistakes, they never doubt their “intuition,” and they have no alternative ways of “thinking” when these techniques fall short.

So, yeah, LLMs only do part of what humans do, but it’s a big and important part. Occasionally we do need facts, critical thinking, self-doubt, and all the rest to do the right thing, but they don’t come up as often as we like to think. The real danger of LLMs, then, is that 80% of the time they might be good enough, but 20% of the time we need fancy human judgment to notice they screwed up and decide what to do. This is a serious problem. Humans are bad at vigilance, and we have a strong instinct to trust language, which in this case is exactly the wrong response.

Language is a defining feature of our species. We aren’t just capable of language, it’s a human universal. Every culture has language. Babies attend to speech from the moment they’re born, and start to babble in a few months. When there isn’t a common language spoken around them, children raised together will spontaneously invent one. Language is a biological imperative for us. It’s in our DNA.

When we perceive language, our minds automatically assume that it’s communication. We imagine another mind behind the words, usually with good intent and a desire to cooperate. Up until recently, this was a pretty safe assumption, so it was totally reasonable for the brain to immediately and automatically translate language into meaning. But now this instinct is backfiring. LLMs create realistic text and imagery without any intentional meaning. They don’t produce “answers,” “opinions,” or “art,” just random content that looks like those things. It’s both very difficult and important to remember that.

We’re still working to understand how these LLMs work, what their limitations are, and what they’re good for. As we do that, I hope we’ll come to understand ourselves better, too. What do you think? Have LLMs made you think about minds any differently? Have you seen any interesting examples of AIs acting strange or foolish? What about people acting like LLMs? Any thoughts or fears about computers gradually inching toward human-like abilities? I’d love to hear from you in the comments.

In Every Mind, a Universe

The first life was blind and ignorant. It was little more than a self-perpetuating chemical reaction, constantly rebuilding itself and making new copies. It had no idea where it was because it had no way to perceive the world around it. Even the very notion of existing and moving within a physical space was incomprehensible. It didn’t know what it was, or even that it was, because it had no way to perceive its inner life, either. It just kept on going, making copies of itself, frequently with errors that made it worse or (occasionally) better at being alive.

Eventually, by chance, life discovered something very useful: certain molecules change shape when something happens to them. Some respond to being hit with light, others change in response to temperature, or pressure, or brushing up against another molecule with just the right shape. Life learned to read these signs, understand them as clues about the world, analyze them, make decisions, and respond. At least, metaphorically speaking. In reality, we’re still just talking about chemical reactions here. One shape change might trigger another, which might cause some new protein molecule to be synthesized, or kick off a chain reaction that leads to a cell moving, adjusting its metabolism, or whatever. The cell acts as if it appreciated the meaning of this signal, but without “thinking” except in a purely mechanical way.

This was the origin of meaning. At first, it was a very primitive thing. Life learned to discriminate between “good” and “bad.” That is, it noticed signs correlated with favorable living conditions, survival, and reproduction. Organisms that sought out more “good” signals while avoiding “bad” signals tended to live longer and produce more offspring. In this way, evolution slowly transformed random patterns of stimulus and response into instinct, innate biases baked in from birth, representing a sort of ancestral “knowledge.” Over time, life evolved more nuanced concepts like: light and dark; warm and cold; food and poison; me, us, and them. Teasing apart these subtler shades of meaning helped life develop more complex and successful strategies to survive in the world.

Every organism has this sort of evolved map of meaning (an “innate ontology”), implicit in their genes. It’s defined by their senses, physical capabilities, reflexes, and gut feelings. That means every species has a profoundly different perspective on reality. Fish, for instance, may have no conception of water because to them it’s a lifelong constant with no alternative. However, they have a very nuanced sense of the information carried in the water, which we are totally blind to. They can be very sensitive to things like pressure, temperature, chemical concentrations, currents, and even electrical fields. To a limited extent, they instinctively “know” where these signs are coming from, what they foretell, and how to react.

Talking about “ontology” as something in our genes is a little unusual. Typically that word is applied minds, perhaps just human minds. It’s about how we perceive reality, dividing it up into objects, categories, and relationships. It’s how people fundamentally understand themselves and the world they live in, and it’s heavily influenced by culture. But philosophers like Daniel Dennett insist that the same concepts should be extended to precognitive life, as well. Our physical bodies lead us to perceive and think and act in human ways, laying a foundation upon which conscious learning and culture can build. In that view, our rich mental ontology is a product of evolution, constructed from lower-level, simpler, more instinctual parts that we share with many species.

Like the first living thing, each human is born into a new and unfamiliar world, forced to figure out how to survive from scratch. We do have a major leg up, though: we’re born with senses and instincts and the ability to move our bodies. Our nervous systems carry and integrate the sensations from our many cells to our brain as a coherent bundle of information. Our brains are highly structured, with all the tools we need to make sense of those signals set up and ready to go. For instance, our multi-stage vision pipeline takes in light signals from our optic nerve, then processes them to detect edges, shape, movement, and even faces from day one. As infants, we don’t know what these things mean yet, but our bodies present the information to our minds in a convenient form and draw our attention, making them quick and easy to learn.

But our innate ontology is very vague. We are born with a sort of “knowledge” (or at least a predisposition to learn) that we are bodies that can move around in a 3D world. That world is filled with objects we can interact with. Some of these objects can move, some are useful, and some are alive. We get tired, hungry, and sick. We need to breathe, drink, and eat other living things to survive. That’s all obvious from a very young age. The rest is on us to figure out. How do we tell friend from foe? How do we find shelter? What’s good to eat around here? How do we make a living? What is the purpose of our existence? These questions are context dependent and quick to change, so life hasn’t evolved answers for us. It can’t. Instead, it gave us brains so we could find our own answers.

What makes humans truly special, though, is that we don’t build our ontologies just by trial and error. We talk about our ontologies. We point things out, name them, tell stories, give demonstrations. We learn from our parents, peers, teachers, and the media. We’re immersed in the collective ontology of our species, something all of humanity has been cultivating for over a hundred thousand years. Our minds are built to soak it all in and to very quickly adopt a picture of reality that’s much richer, more accurate, and more nuanced than what’s available to any other species. Much more than any one human could possibly figure out in their lifetime.

This way of understanding reality is powerful, but it leads to a great big illusion: we tend to see our ontology as reality itself. That’s understandable. Our ontology is our window on the world. It encompasses everything we can perceive, understand, and do. Yet, it is not a real thing. It’s an image in our minds, our bodies, and our genes. It’s informed by our genetic ancestors, our senses, and what we’ve learned from each other. But we perceive much more than our senses actually take in. Our brains are running a sort of “image enhancement” algorithm, as seen in Sci Fi classics like Blade Runner and now made manifest by deep learning software. We take in a little data, then use our knowledge and expectations to extrapolate something much bigger, fuller, and richer, making up the details that we can’t directly perceive. That is, we see what we believe. We perceive concepts, not reality as it truly is.

Of course, if our ontology is not a real thing and lives inside our minds, the consequence is that every human being must have a different ontology. They are in many ways similar, sure. We are the same species, living in the same world, with the same basic needs. We may even be from the same community, with a shared culture. Yet, we might disagree about the meaning of important concepts like “freedom,” “equity,” and “justice.” We might have very different ideas about what money is, what purpose a government serves, or how to be a good person. These are not disagreements about facts, but about the structure of reality itself—the framework we use to fit facts together into a coherent picture. These disagreements are particularly hard to reconcile, since it’s hard to even imagine what doesn’t fit inside my ontology.

That was a bit of a whirlwind tour of ontology. I went fast and skipped over plenty, so I’ll ask: what would you like me to go deeper on? Is there anything that doesn’t make sense? Anything that fascinates or excites you? Let me know in the comments. If you’d like to learn more about ontology in its many forms and how it evolved, I highly recommend From Bacteria to Bach and Back by Daniel Dennett.

Best Intentions

For the most part, people have good intentions. We generally want to be kind, responsible, and respected by our communities (at least, once our own personal needs are met). We have worthy goals and dreams for how to make life better. Yet, we all know that doesn’t always work out. We struggle with our impulses and willpower. We break our New Year’s resolutions as soon as we set them. We say one thing, then find ourselves doing another. This is true of individuals, but also for teams, communities, and organizations. Why does this happen, and what can we do about it?

Intentions are an important part of what makes us human. All sorts of animals set goals and make plans, but they’re usually very basic, short-term intentions, like “I’m gonna eat that bug.” In most species, lifestyle is largely innate, defined by genetics and the environment with only limited flexibility. Humans are different in that we design our lifestyles, both as individuals and as a society. We set abstract, long-term intentions like “I’m gonna save up for college” or “all people in this company should have equal opportunity.” These ambitious visions can help us imagine possible futures, and guide us to make them a reality.

It’s easy to set good intentions and stop there. Sometimes that’s enough, but often it’s not. Those delayed goals can be tricky for our animal minds, after all. I meant to pick up eggs on my way home, but I forgot. I know I ought to make a healthy dinner, but I’m exhausted, so I’ll order pizza. I said I’d learn Japanese, but maybe I was just kidding myself. I wanted to build a shed in the backyard, but I didn’t really know how, so it never got started. When one of my female engineers at work was criticized unfairly, I thought I was being supportive when I swooped in to defend her. When she pointed out how I was preventing her from proving herself, I had to unlearn that habit. In each of these cases, I meant well, yet I fell short.

This highlights some of the key problems with intentions. They aren’t rules that the brain enforces. If I keep my intentions in mind, I might notice when they should affect my decisions, and act accordingly. Or I might not. Like all people, I spend a lot of time on auto-pilot. I often get tired or distracted. I make countless choices in a day I’m not even aware of. Then there’s willpower. Just because I know I ought to do the right thing, doesn’t mean I will after a long and frustrating day. Lastly, intentions don’t come with step-by-step instructions. It’s often not clear how to achieve my goals, so I have to think about the steps involved, what my options are, and what outcomes are likely. Even if I do everything right, reality doesn’t always play along. I have to watch to make sure things turn out the way I intended, and respond when they don’t.

At their best, intentions serve as a frame for thinking and planning. They’re a first step in a process that leads to an outcome. To improve the chances of success, I must design that process. I must compensate for my cognitive blind spots and mitigate the risk of accidents and surprises. This is a form of self-programming. What can I do now to shape my future behavior? How do I make sure I get into the right situations and avoid the wrong ones? How will I notice when that happens? Can I prepare so that I know what to do in the moment, and have everything I need to act? How will I know if my plan is working, and change course if I need to?

I’ve found there are two essential tools for this sort of planning:

Write and revisit. Intentions are often too vague to be useful. They can be easy to forget, and can drift over time without our notice. To remedy this, I write my intentions down. When possible, I share them with others to make sure I’m rigorous about it, and to create a sense of accountability. Then I set a reminder to revisit those intentions in a few weeks or months. I try to be brutally honest with myself. Am I living up to my expectations? Do I still think about the problem in the same way? If not, I try not to feel guilty, but instead focus on what to do about it. What’s wrong? Should I change my behavior? My intentions? Both? I force myself to think about this.
Prepare yourself. When I set intentions, I think through how to achieve them. Not in full detail, but at least I’ll identify a few major sub-goals that are essential for success. I think about when and how I might do those things, and create some structure around that to make it real. I write TODO lists and set time aside for specific tasks. I leave notes or physical reminders in the real world to nudge me at crucial moments. I think about what decisions I’ll have to make, and how to make them. This is a way of front-loading the effort, doing the thinking and willpower work when I have time for it. That way, when the crucial moment comes, my actions can be fully automatic.

Critically, these two things go hand in hand. Writing down my intentions doesn’t help much if I don’t take some action to ensure they happen. Similarly, making a plan and following it rigorously can be a disaster if it’s the wrong goal, the wrong plan, or if the situation changes. The benefit comes from cycling through these two modes of thinking regularly. What should I do? How should I do it? Is it working? What should I do next?

The same problems appear in teams, communities, and organizations, often amplified dramatically. Aligning intentions across many people is very hard, and aligning behaviors is impossible. Each person has their own motivations, and is gifted and fallible in their own unique ways. To be fully effective, they must set their own intentions, and find their own ways to reliably achieve them. Many leaders are good at setting intentions for their groups, but struggle to make them happen. Sometimes they just say what they want to accomplish and assume everyone gets it and will (somehow) make it happen without any guidance or coordination. Other times they micromanage, trying to force people to deliver by robbing them of their autonomy.

I lead others the same way I lead myself. I just apply the concepts at different levels of abstraction. As a leader, my main responsibility is to keep intentions fresh in my team’s minds and to encourage each individual to be productive in their own way. Rather than telling people what to do and how to do it, I make sure my team has policies and tools to help with common challenges. This gives them freedom and flexibility, but also reduces their burden in the moment and encourages consistency. I create timelines, schedule check-ins, and set reminders in the right time and place. I often don’t care about the timing, I just want to give folks a nudge when they need it, and encourage them to prioritize, pace themselves, and track their own progress. Every quarter, we revisit our intentions together and try to think: are these still the right goals? Is our approach working? Is there a better way?

What makes humans so remarkable is our flexibility. We work creatively within constraints, figuring out the details as we go along. Intentions are a powerful tool for doing that, but they’re just part of the story. Acting intentionally, either as a person or as a group, is about creating the conditions for success. It’s not enough to want something. We’re just animals, after all. We can’t see the future, we can’t be “on” all the time, and it’s very easy to get distracted or to deceive ourselves. If we embrace our limitations, we can try to work around them and do better. Or, at least we can forgive ourselves more easily when we inevitably let that New Year’s resolution slip.

What do you think? Have you sometimes struggled to live up to your own intentions? Do you have any advice for how to overcome that? What about in the work setting? Have you noticed any practices that make a team / company better or worse at living up to their own ideals? I’d love to hear from you in the comments.