Before you read this, note that this post is discussing a subset of AI tools. Specifically, large language models, which are a type of generative AI. It’s important to distinguish this from the umbrella term of AI, which includes many other types of models and tools that do not share the quirks and limitations discussed below.

The Many Truths of Toddlers

To live with a toddler is to be reminded daily, hourly, minutely, of the simple fact that multiple things can be true at the same time. You can deeply want to color with only the blue crayon AND only the red crayon AND only the orange crayon at the same time. You can be excited about going to play soccer AND devastated by the idea of putting on pants.

Life is full of layers.

Sometimes it’s not about multiple truths, but rather the rapid change between individual truths. A deep abhorrence to the very idea of peanut butter rapidly replaced by an existential need for peanut butter on a spoon.

It’s not that my son isn’t a reliable reporter. Quite the opposite; he is very truthful and has the communication skills required to very clearly answer questions about a wide variety of topics. It’s that he very reliably reports what is true specifically for him, not yet feeling a need to match his truth with anyone else’s.

You could say that toddlers are non-deterministic. That means that the same input does not always lead to the same output, i.e. if you ask the same question many times, you can get many answers. Some perhaps are more common than others, but nevertheless you do not carry the expectations of 2+2=4 into a conversation with a toddler.

While this is inspiring on many levels - if only we were all so clear in communicating our wants and needs - there are also good reasons why we see the integration of others truths as a critical part of development. No matter how true it may be that I want to drive on the left side of the road and am individually capable of doing so safely, it’s incompatible with driving on public roads in North America.

Shifting Our Sources of Truth

As adults these are the sorts of truths we tend to focus on. The truths that we share with others and as a result take on a definition of reality. They are a requirement for functional society, and in numerous ways our lives and livelihoods depend on it.

It’s no secret that we’ve watched the cohesion of these shared truths fragment in recent years. There’s no doubt that technology has played a role. While it’s popular to believe that more and faster communication results in a faster emergence of truth, that’s more ideology than reality. In fact, more and faster communication also fans the flames of conspiracy theories. As Yuval Noah Harari pointed out in Nexus, there’s an argument to be made that one consequence of the printing press was the spread of a conspiracy theory that triggered witch trials1.

In our current context we have a new plot twist - personalized media that automatically tailors itself to your consumption preferences. You may think “impossible, I HATE what I am reading. It absolutely makes me sick!” Fair enough, but note that I said consumption preferences. You may think that you are flooded with information that is against your ideology, but the truth is that it’s all optimized for what you will spend time reading. We are sucked in by what creates the strongest emotional reaction, which is often negative.

And of course, it’s all backed by the presentation of facts. How the facts are legitimized varies, but in a certain sense it’s also irrelevant. Each of us has an internal mechanism that helps us to identify what we believe constitutes this shared reality of objective truth. And whatever your preferred source of information is, it passes your check.

The Era of Search

We have many options for how to double check claimed facts. Still going decades strong, Snopes and similar cites work to identify falsehoods that have gained popularity online. Health information can be checked with a doctor. Investment advice can be discussed with a financial advisor. But “can be checked with a [insert professional]” is not the same thing as actually doing so.

Since the rise of the internet and the belief in unfettered access to information, we have been doing our checking online. Google and other search engines became the first stop, before asking a professional a question. After all, perhaps there was a clear and simple answer to be found if you just looked for it. Sometimes this worked, but as the emphasis moved to providing answers based on the intention of your search it became harder to know if you were getting accurate information or if you were getting information that fed the bias baked into your question. You do not see the same websites nor the same information when you search “are vaccines safe” vs “why are vaccines dangerous”.

But when you run the same search twice, you do see the same information. Only over time do the search results significantly change, with that change representing changes in what information people have found most useful. With traditional search engines, when two people search the same phrase at the same time, they see the same results. Generative AI breaks that consistency.

The Plot Twist of LLMs

That is because, like toddlers, the models underlying these tools are non-deterministic; they too do not give the same answer each time you ask the same question. Like toddlers, each answer is delivered with complete authority and conveyance of conviction. Unlike toddlers, the summaries drafted by large language models (LLMs) are extremely compelling to adults. While we know that the durability of a toddler answer comes with a metaphorical asterisk, we lack the same skepticism towards LLM-generated summaries.

Why do the answers from LLMs vary? A few reasons. LLMs are trained on massive data sets that include vast swaths of the internet, including conflicting information on virtually every topic. While most of the time it may cite information that would be considered factual, some of the time it will not. And then there are so-called hallucinations, which are when the model fabricates the answer. It’s important to understand that, unlike other software bugs that can be eliminated by refining the code, both sourcing variable material and hallucinations are an intrinsic feature of LLMs.

These effects are compounded by so-called sycophantic behavior. I hate the term sycophancy2; using it in this context hides manipulation behind a word that few people were previously familiar with. So let’s call it what it is: models were made to compliment you so that you would spend more time with them. It’s manipulative and results in asinine feedback3.

Sycophancy in LLMs means that when you think you are receiving unbiased feedback, you are in fact being given blind compliments. This is bad for knowledge, improving work output, and downright dangerous depending on your mental and emotional state4.

Perhaps tools could be built that layer on top of the models to provide some safeguard, but the product releases from the leading providers have focused on changes that will increase the amount of time people spend on the platform. Very little mention has been made on safeguarding unsafe responses, and what little changes have been made are largely from lawsuits. Newest reports are talking about the ability to predict shopping purchases. A money maker, no doubt, but brings us no closer to addressing these fundamental issues.

What Happens When We Rely on Non-Deterministic Truths

Imagine, for a moment, that you and a friend both go to read about the Peruvian Andes in preparation for your upcoming vacation. You’re wanting to learn more than just about the tourism opportunities, and you are very excited about this trip, so you read an encyclopedia article. Your friend does the same, and soon you begin to discuss excitedly what you read.

Except, nothing you are saying matches what your friend is saying. Did they even read the actual article? A few sentences later and you’re convinced that they either are lying and read something completely unrelated, or that they are completely fabricating the weirdest details. Are they messing with you on purpose or are they going crazy?

Now imagine that you are instead talking to a stranger. Without the trust of friendship, you jump more quickly to confusion and then to suspicion. What is this persons deal? They are adamant that truth is on their side but that is clearly impossible. You can’t talk to a person like this, they need a family member to intervene!

In fact, you are actually lucky if you have these interactions, because they would give you an opportunity to see the inconsistencies. More commonly, we rely on one friend to tell us about what they researched for our vacation and happily set off on the itinerary they provide. That seems to be what happened to a few folks who nearly got themselves lost in the Andes searching for a mountain that doesn’t exist, thanks to a very compelling recommendation by ChatGPT5.

How to Spot an Expert Faker

When I talk about this with friends, they express surprise and then comfort themselves by saying something along the lines of “that’s never happened to me.” Unfortunately, they are wrong. It’s not happened to them in a way that they’ve noticed. Because if you have used AI for factual reference more than a handful of times, this has happened to you.

The challenge is that it’s nearly impossible to detect if it’s a topic outside of your area of expertise. But that’s the catch - we look up the things that are outside of our area of expertise. It leaves us vulnerable to the most dangerous form of ignorance, which is when we think we know something that we don’t know.

There is a silver lining, which is that it’s very easy to identify these errors when you’re within your area of expertise. Each time you look something up, go ahead and enter an equal number of prompts about something you know a lot about. Read the responses carefully.

Most of the responses will be spot on, then suddenly you’ll come across one that you find yourself reading twice. Grammatically it’s correct, and the vocabulary looks correct but if that’s all someone were to read they would completely misunderstand the concept. It feels almost subversive to read, and once seen, it cannot be unseen.

That is the immunity we need.