Legends of Forecasting

What do the British television legend, Mystic Meg, a German octopus from Oberhausen, and American poker champion, Amarillo Slim, have in common? All three made their names forecasting the future — but each did so in a radically different way, and in their differences lies a useful lesson for investors.

Mystic Meg, familiar to millions through her National Lottery predictions and weekly horoscope columns, belonged to an ancient tradition: that of the oracle. Like the priestesses of Delphi, her forecasts were rich in imagery but light on commitment. Prophecies were carefully constructed to seem profound while offering enough flexibility to be interpreted as correct whatever the outcome. When Croesus asked the Oracle if he should wage war, she answered that if he crossed the river, a great empire would fall - omitting to mention that it might be his own. Many modern forecasts, especially in investment research, follow the same pattern: hedged language, broad scenarios, and enough vagueness to ensure that any result can later be described as success.

Paul the Octopus, meanwhile, took a very different approach. Without knowledge, models, or even much deliberation, Paul chose winners in the 2010 World Cup by selecting mussels from two different boxes. His streak of correct guesses dazzled a world hungry for magic, but there was no skill involved. His "forecasting" was pure heuristic: instinctive, non-systematic, and random. Yet because we are wired to seek patterns and meaning, we were quick to attribute expertise to what was, in fact, sheer chance. In markets too, happy accidents are often mistaken for skill, and runs of luck for deep insight. If the short-term success of stocks with names conveniently close to fads and crazes is any guide, we can't dismiss how many people may look to Paul as their forecasting inspiration.

Amarillo Slim, by contrast, represents a very different spirit: the disciplined gambler. Famous for betting on anything, from horse races to the velocity of a fly on a windowpane, Slim's genius lay not in predicting certainties but in finding small edges wherever they appeared. He understood odds, psychology, and timing, and knew that success came not from being always right, but from sizing bets correctly when probabilities tilted in his favour. His approach was far closer to the spirit of good forecasting: conditional, probabilistic, and always aware of the margin between luck and leverage.

Between them, they offer a vivid map of the forecasting landscape: the mystic, the random, and the rational. To understand how we can forecast better - and avoid the traps that even skilled investors fall into - we must first be clear about what forecasting really is, and what it is not.

Why bother at all?

'Predictions are hard, particularly of the future' is old wisdom. Certainly, analysts don't seem very good at it - the 'hit rate' for the average equity analyst is only around 50%. That is to say, half of the stocks they say will go up don't, and vice versa. If forecasting seems so prone to failure, why we bother at all?

Well, think of old ships navigating by the stars. Stars, as we know, are massive balls of boiling gas billions of miles away. They don't tell us where to go - they just give us a series of reference points that help us navigate. Even flawed forecasts perform a vital function: they help structure action in a world that demands choices before certainty is available. As Tetlock and Gardner have shown, even the best forecasters aren't clairvoyant. Their skill lies not in seeing the future clearly, but in refining probabilities better than chance and updating beliefs more quickly than others when the world shifts. A world without forecasts would be a world paralysed by indecision, hostage to events rather than prepared to meet them.

Forecasting is rarely just about estimating an objective outcome; it is also about estimating how others will perceive and respond to that outcome. In investment markets particularly, it is often not the truth that matters most, but the consensus about the truth. Keynes’s famous beauty contest analogy captures the problem neatly: it is not about picking the prettiest face, but about picking the face that the majority will find prettiest. In buoyant markets, perception tends to outweigh fundamentals; in reflexive systems, beliefs shape prices, which in turn reinforce beliefs. Thus, forecasting becomes a double exercise: what is likely to happen, and how likely is the crowd to anticipate it correctly, early, or at all?

Good forecasting, then, is not the search for certainty but the management of uncertainty. It is about holding beliefs lightly, and updating them as new information arrives. This idea is not new. It has a formal structure in probability theory known as Bayesian reasoning, which offers a disciplined way of adjusting our forecasts over time. Before we can forecast better, we need to understand how this process of updating really works.

Say the Met Office think there's a 60% chance of rain today. It's looking pretty grey at 10am so you start thinking that might be closer to 70%, positively threatening by midday so you revise up to 90% - but the wind picks up, and the clouds start clearing with blue sky appearing in the distance, so you shift down to 20%. Perhaps it ends up not raining at all, and as you approach the stroke of midnight your confidence grows into, eventually, certainty. That process, taking the initial forecast and then layering new information on top to revise the probability, is the essence of Bayesian reasoning.

Forecasting the Forecasters

Bayesian thinking matters in financial markets because conditions are never static. Each piece of information - a quarterly earnings release, a policy announcement, a shift in commodity prices - should not simply confirm or overturn our views, but refine them. The point is not to be "right" once, but to adjust probabilities dynamically as reality unfolds. Success comes less from early conviction than from flexible interpretation: holding beliefs lightly enough to revise when new evidence emerges, but firmly enough to act when the odds are in our favour.

Yet in markets, forecasting is never just about fundamentals. It is also about forecasting how others will react. Markets are reflexive systems, where perception feeds back into price. An earnings miss may not move a stock if expectations were already low; a political shock might cause panic far beyond its immediate economic effect. The crowd’s reaction, not the underlying event, often drives returns. Good forecasting, then, is not just double-layered - it’s recursive: what will happen, what will people believe has happened, and how will they behave as a result? Anticipating this loop is the real task.

One of the quiet dangers in forecasting is misplaced certainty. As we refine our views and narrow our expected range of outcomes, our models become more fragile. In Bayesian terms, we move from high entropy, where many outcomes are possible, to low entropy, where few are. But when beliefs become too concentrated, small shocks can cause disproportionate damage. In markets, as in life, it is often not uncertainty that kills you, but overconfidence. Most failures are not failures of prediction; they are failures of imagination.

If forecasting is a discipline of managing uncertainty, not eliminating it, then the best forecasters are not those who make the fewest mistakes, but those who respond most skilfully when reality shifts. They treat forecasts not as promises but as scaffolding - structures to be revised, rebuilt, or abandoned as new evidence demands. Conviction is allowed to grow, but only in proportion to the strength and quality of information, and always with an awareness that the ground may yet move beneath them.

In markets, this mindset is not just philosophical, it’s practical. Renaissance Technologies, the most successful quantitative hedge fund in history, built its success on the ruthless updating of signals; discarding strategies the moment their predictive power decayed, regardless of theoretical beauty. Stan Druckenmiller, too, has spoken of the importance of rapidly shifting positioning when regimes change, even when it means abandoning deeply held views.

Common to all of them is the understanding that forecasting is not a one-shot prediction but an evolving map. Good forecasters maintain scenario trees, not single narratives. They track their confidence levels explicitly, not just their directional calls. They anticipate not only what might happen, but how others might misread what happens - and what opportunities that misreading might create. Above all, they cultivate a deep intellectual humility, paired with the operational discipline to act decisively when the odds shift in their favour.

Forecasting, then, is not the pursuit of certainty, but the practice of steering through uncertainty with discipline and care. We cannot command the tides or chart a perfect course through the unknown, but we can prepare ourselves to move with the currents when they shift. Most forecasting, like Mystic Meg's riddles or Paul the Octopus’s guesses, offers little more than comfort or coincidence. True forecasting demands the spirit of Amarillo Slim: a willingness to place small, calculated bets on partial knowledge, and the humility to adapt when the odds change. The best investors are not those who claim to see the future clearly, but those who revise their maps steadily as the world unfolds, and act decisively when the probabilities favour them.