AI is not just interesting per se, it also enlightens how animal and human intelligence works. AI can give blatantly absurd answers. These “HALLUCINATIONS” persist despite post-training efforts (such as providing extensive human feedback to an AI’s responses before it is released to the public). The OpenAI authors of Why language models hallucinate examined ten major AI benchmarks, including those used by Google, OpenAI and also the top leaderboards that rank AI models. This revealed that nine benchmarks use binary grading systems that award zero points for AIs expressing uncertainty.
This creates what the authors term an “epidemic” of penalising honest responses. When an AI system says “I don’t know”, it receives the same score as giving completely wrong information. The optimal strategy to score highly under such evaluation becomes clear to the AI system: always guess, never acknowledge uncertainty [1].
The question is what is uncertainty? It’s a statement A (which could be a fact or a logical deduction) such that NON A could be true too. Life is full of these.
Human beings contemplate all the times a set of lines (that is consequences) incorporating A… And then a set of lines (that is consequences) incorporating NON A… Then they read between the lines, or the consequences and conclusions the lines bring: they INTER LEGERE… INTELLIGENTLY DISCRIMINATE, and take a gamble. Intelligence is indeed this capacity to read between the lines. If an intelligent animal did not have the capacity to DECIDE between A and NON A when the logic at hand does not allow it to do so, it would not be intelligent.
Say a bird gets in a bush: it could go out that way, A, or the other, NON A. What is dinner going to do? A or NON A? The predator does not know, it needs to choose between A and NON A. If it did not choose, acknowledging uncertainty, in a systemic fashion, it would starve to death.
Two days ago I made a mountain hike/run in muddy, wet, snowy and solitary conditions. Often I couldn’t follow the path: it was a torrent, I had to go through tall plants with broad leaves often half a foot across, making it impossible to be really sure of the ground. Other times I was completely off paths, negotiating steep terrain, looking for a path which existed only on maps. During that outing, over several hours, I had to make several decisions per second, all of them uncertain to a decree. Many of them, guessing wrong, could have meant another accident like the one I suffered a few months ago (when a rock broke under running impact). So I had to decide what reality was on the order of 20,000 times… No decision would have meant dying in the mountains of exposure.
In other words, for biological intelligence, TOO MUCH UNCERTAINTY MEANS DEATH. Too much uncertainty brings the problem of Buridan’s ass.
Should two courses be judged equal, then the will cannot break the deadlock, all it can do is to suspend judgement until the circumstances change, and the right course of action is clear.
— Jean Buridan, c. 1340
Later writers satirised this view in terms of an ass which, confronted by both food and water, must necessarily die of both hunger and thirst while pondering a decision.
Many thinkers have pondered this problem. And it has a solution: because NOT taking a decision one will die, advanced animal psychobiology has evolved IRRATIONAL OVERDRIVE of equivalent outcomes: confronted to apparently equivalent solutions, a brain will choose one over the other, in a timely manner… no logic needed. There is a connection there with fascism [2].
This is why users of AI do not like to be presented with uncertain conclusions: they prefer erroneous certainty, which is actionable, to honest uncertainty, which is not.
The same will extend to any normal user of human leadership: this is why authoritarian rule is preferred.
In the case of AI, holding both A and NON A doubles the logic, hence the energy spending. computational economics. Uncertainty-aware language models require significantly more computation than today’s guessing and decisive approach, as they must evaluate multiple possible responses and estimate confidence levels. For a system processing billions of queries daily, this brings unbearably higher operational costs.
***
The necessity of having to decide has to do with an n-N problem: the analyzing system has only n available configurations, which is often much less than reality out there which has a much larger number of possible configurations, N… By abuse of language, we will call these potential sets of configurations n and N!
To make n an approximation of N requires to decide which approximations one can get away with. This applies to sets of neurological networks known as “brains”, which have to constantly approximate N with n. In other words, guessing, not to say lying, is an intrinsic part of wisdom. Intelligence creates, between the lines what those lines do not have, a better approximation of reality, but that’s all it is.
Patrice Ayme
[1] At OpenAI, we’re working hard to make AI systems more useful and reliable. Even as language models become more capable, one challenge remains stubbornly hard to fully solve: hallucinations. By this we mean instances where a model confidently generates an answer that isn’t true. Our new research paper (opens in a new window)
argues that language models hallucinate because standard training and evaluation procedures reward guessing over acknowledging uncertainty.
***
[2] Fascism, per se, is simply the instinct of making out of many minds, just one. It’s a form of abstraction: out of a set, the many, is made a point, the one. It is SPATIAL ABSTRACTION. Yes, fascism as spatial abstraction.
In the matter of hallucinations, the abstraction is temporal. Out of a span of time, a decision point needs to be reached. That faculty, to make decisions in an instant, is actually presented by Hitler himself as a great advantage of having a tyrant as the leader of a country .
Political cartoon c. 1900, showing the United States Congress as Buridan’s ass (in the two hay piles version), hesitating between a Panama route or a Nicaragua route for an Atlantic–Pacific canal.
Patrice Ayme's Thoughts 