Machine learning

One year ago, in our article “Can We Finally Use ChatGPT as a Quantitative Analyst?”, we explored the feasibility of leveraging ChatGPT for quantitative analysis. Since then, a lot has changed: newer models are now available (from OpenAI and also other vendors), and the ecosystem around AI-assisted analysis has evolved significantly. Back then, we encountered numerous challenges, ranging from model hallucinations and faulty code generation to excessive overfitting. In this article, we revisit these issues to assess what has improved and what remains unresolved, with the goal of finally answering whether we can use LLMs to assist with quantitative analysis tasks.

Everyone is excited about the potential of large language models (LLMs) to assist with forecasting, research, and countless day-to-day tasks. However, as their use expands into sensitive areas like financial prediction, serious concerns are emerging—particularly around memory leaks. In the recent paper “The Memorization Problem: Can We Trust LLMs’ Economic Forecasts?”, the authors highlight a key issue: when LLMs are tested on historical data within their training window, their high accuracy may not reflect real forecasting ability, but rather memorization of past outcomes. This undermines the reliability of backtests and creates a false sense of predictive power.

When using machine learning to forecast global equity returns, it’s tempting to focus on the raw prediction—whether some stock market is expected to go up or down. But our research shows that the real value lies elsewhere. What matters most isn’t the level or direction of the machine learning model’s forecast but how much it differs from a simple, price-based benchmark—such as a naive moving average signal. When that gap is wide, it often reveals hidden mispricings. In other words, it’s not about whether the ML model predicts positive or negative returns but whether its view disagrees sharply with what a basic trend-following model would suggest. Those moments of disagreement offer the most compelling opportunities for tactical country allocation.

In two of our previous articles, we explored the idea of using artificial intelligence to backtest trading strategies. Since then, AI has continued to develop, with tools like ChatGPT evolving from simple Q&A assistants into more complex tools that may aid in developing and testing investment strategies—at least, according to some of the more optimistic voices in the field. Over a year has passed since our first experiments, and with all the current hype around the usefulness of large language models (LLMs), we believe it’s the right time to critically revisit this topic. Therefore, our goal is to evaluate how well today’s AI models can perform as quasi-junior quantitative analysts—highlighting not only the promising use cases but also the limitations that still remain.

Building machine learning models for trading is full of nuances, and one important but often overlooked question is: what exactly should we try to predict—the direction of the next market move or the actual value of the asset’s return? A recent paper by Cheng, Shang, and Zhao, titled “Direction is More Important than Speed” offers a clear and practical answer. Their research shows that focusing on direction—simply whether returns will be positive or negative—leads to better model accuracy and, more importantly, stronger real-world investment performance. This is especially true when using machine learning methods, where predicting the direction allows models to better capture downside risks and build more effective trading strategies. For anyone using ML in finance, this paper makes a strong case that predicting where the market is headed is often more valuable than predicting how far it will go.

Can machine learning models better predict stock returns if they are tailored to specific industries, or is a one-size-fits-all (generalist) approach sufficient? This question lies at the heart of a recent research paper by Matthias Hanauer, Amar Soebhag, Marc Stam, and Tobias Hoogteijling. Their findings suggest that the optimal solution lies somewhere in between: a “Hybrid” machine learning model that is aware of industry structures but still trained on the full cross-section of stocks offers the best performance.