Xiaohong Chen

Excited to share that Mahmudur Rahman and I got our paper "ColdNet: Neural Causal Inference Under Extreme Imbalance and Sparsity" accepted at the WSDM Workshop on Benchmarking Causal Models (CausalBench'26). This paper introduces a novel solution for estimating individual treatment effects under three co-occurring challenges - extreme class imbalance (0.39% treatment rate), 99%+ cold start cases(no history), and 97%+ sparse outcomes, achieving strong results in production at billion-scale. I had the pleasure of attending the conference in Boise, Idaho this past February and presenting this work in person. Big thanks to CausalBench'26 Program Chairs and organizers for hosting and seamless communication throughout each process, as well as Amazon Science for supporting this piece of work.

54

1 Comment

I've just read about the research that explores using LLMs as tools for reinforcement learning (RL) tasks, where an agent needs to search for information and learn from it to make decisions. I'm sharing it because maybe you'll also find it interesting. Normally, RL requires the agent to interact with external search engines, which can be costly and time-consuming (https://lnkd.in/g4Jc82xq). The researchers wanted to see if LLMs, which already have a lot of general knowledge, could perform well on these tasks without needing to rely on external tools. Here’s how they did it: - Self-Search: The first part of the experiment involved testing how well LLMs can perform search-like tasks on their own. This is done by asking the model questions in a structured way, which helps the LLM search its own knowledge base. The LLMs performed well on benchmarks, like answering questions, without needing to pull data from an outside search engine. - Self-Search RL (SSRL): To improve the model’s search ability, they introduced Self-Search RL (SSRL), which rewards the LLM for how well it uses its internal knowledge. This way, the LLM gets better over time at refining its responses through practice, instead of having to get info from external sources. This makes the model more self-sufficient. Here's what they discovered: - Many LLMs already have enough general knowledge to perform well on certain tasks. - SSRL can reduce "hallucinations" by encouraging more accurate use of internal knowledge. - Models trained with SSRL start working even better with external search without additional efforts. #AI #MachineLearning #DeepLearning #LLM #Search #ReinforcementLearning #TechInnovation #ArtificialIntelligence

3

We (w/ Shansan Gong, Ruixiang ZHANG, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong) released a family of 7B diffusion language models, DiffuCoder, that specializes on code generation, with a focus on understanding and improving masked diffusion models. A core analysis of DiffuCoder is the autoregressiveness (AR-ness) score, a novel metric that quantifies the causal patterns in decoding, revealing how diffusion models break from strict left-to-right generation for more flexible, non-linear code planning. Recent advances in autoregressive (AR) models dominate code generation, but diffusion-based LLMs (dLLMs) like DiffuCoder offer a promising alternative, especially for complex programming tasks. DiffuCoder explores how these models decode differently—showing less global AR-ness in code tasks compared to math—and how temperature affects both token selection and generation order, unlike traditional AR models. We also introduce coupled-GRPO, a post-training RL method with a coupled-sampling scheme, to reduce performance drops during accelerated decoding, boosting parallelism and efficiency. We use a self-improvement pipeline that leverages AR-ness analysis, coupled-GRPO optimization, and evaluation on benchmarks like AceCode-89k to refine decoding strategies. This approach enables DiffuCoder to navigate diverse code generation pathways and enhance performance with modest computational overhead. Looking ahead, we aim to further leverage Reinforcement Learning to steer code generation through these decoding patterns, with the discrete nature of AR-ness scores providing a foundation for search-based strategies—ideal for the sparse rewards of optimizing complex code structures. Check out our full paper and code for a deeper dive! Paper: https://lnkd.in/gVWU3BDJ Code: https://lnkd.in/gmXTZ_6n Models: https://lnkd.in/gTcKCDr9 #MachineLearning #AI #CodeGeneration #DiffusionModels #NLP

220

5 Comments

Just arXived: MUSTAFAR – Promoting Unstructured Sparsity for KV Cache Pruning in LLM Inference How do we shrink the memory-hogging key-value cache in large language models—without fine-tuning or accuracy degredation? Meet MUSTAFAR, our new end-to-end system that: 🔍 Prunes at the finest granularity with per-token magnitude-based unstructured sparsity: up to 70% pruning on both Key & Value caches with negligible quality drop. Notably, we can prune the Value cache (something structured methods can’t do) to very high sparsity! 📦 Compresses the resulting sparse cache into a bitmap-based format, packing the cache down to 45% of its original size. ⚡️ Accelerates decode-time attention via a custom sparse attention kernel, boosting tokens/sec throughput by up to 2.23× over dense inference. 🛠️ Integrates with orthogonal methods like token eviction and quantization for even richer compression. 👉 Check out the preprint & code: Paper: https://lnkd.in/eXeA99EZ Code & Kernel: https://lnkd.in/eZ9fyQP9 Huge thanks to my co-authors Helya Hosseini, Ramyad Hadidi and advisor Bahar Asgari for her guidance and insights 🚀

85

2 Comments

Most existing #lean4 datasets contain only correct proofs. Models learn error correction with RL, that's expensive. With UW Math AI lab we release a dataset of 260k erroneous Lean proofs with - compiler feedback - reasoning trace - corrected proof Improvements in Error Correction: - Goedel 8B: 2x - Kimina 8B: 3x Paper: https://lnkd.in/gaybt4bd

117

2 Comments

Excited to share our new paper published in Human Factors: “Inferring Hidden Attentional States in Driving: A Bayesian Approach to Modeling Distraction and Secondary Task Engagement.” This work was led by Lekhapriya Dheeraj Kashyap as part of her PhD research, and I’m grateful to have collaborated on it. Many real-world systems face the same challenge: the most important states are hidden. In driving, we can observe signals such as speed, eye movements, or pupil dilation — but not the driver’s true attentional state. In this work, we develop a Bayesian decision framework based on a Partially Observable Semi-Markov Decision Process (POSMDP) to infer latent attentional states and model how drivers allocate attention between competing tasks. The model detects distraction earlier than common heuristic rules and reveals substantial heterogeneity in attention strategies. Beyond driving safety, problems like this appear in many AI systems where human states are latent and decisions unfold sequentially. Great collaboration with Lekhapriya Kashyap, Yanling Chang, Maryam Zahabi, and Alfredo Garcia. 📄 Paper: https://lnkd.in/g23ubzht

9

Just read a great blog post by Sebastian Raschka on the KV cache in LLMs - https://lnkd.in/g538VzvB and had to share a summary of my key takeaways! For anyone working with or interested in Large Language Models, understanding the KV cache is a must for optimizing inference. This technique is all about efficiency. Instead of re-calculating the key and value vectors for the entire input sequence with every new token, the KV cache cleverly stores these and reuses them. The result is a significant boost in text generation speed. The article also dives into the practical side with a code implementation and discusses useful optimizations like pre-allocating memory and using a sliding window to manage memory. If you're looking to get into the nitty-gritty of how LLMs work under the hood, I highly recommend checking out the full post! #LLMs #MachineLearning #AI #DeepLearning #InferenceOptimization

12

Excited to share that the ADAPT group (https://lnkd.in/gV4hZmS8) at UIUC will be presenting three papers at the upcoming OOPSLA 2025 conference next week. Here are some brief details of the papers. 𝐆𝐀𝐋𝐀: 𝐀 𝐇𝐢𝐠𝐡 𝐏𝐞𝐫𝐟𝐨𝐫𝐦𝐚𝐧𝐜𝐞 𝐆𝐫𝐚𝐩𝐡 𝐍𝐞𝐮𝐫𝐚𝐥 𝐍𝐞𝐭𝐰𝐨𝐫𝐤 𝐀𝐜𝐜𝐞𝐥𝐞𝐫𝐚𝐭𝐢𝐨𝐧 𝐋𝐀𝐧𝐠𝐮𝐚𝐠𝐞 𝐚𝐧𝐝 𝐂𝐨𝐦𝐩𝐢𝐥𝐞𝐫:  Most GNN acceleration systems either perform intra-operator optimizations (e.g., TACO, SparseTIR) or inter-operator optimizations (e.g., Graphiler), missing the synergistic benefits of performing both simultaneously. In this work, we propose a language and a compiler with two complementary IRs to effectively track and transform GNN code to perform both classes of optimizations (and more!) to achieve, on average, 2.55x geometric speedup across a wide range of GNNs, graphs, and systems. This work is led by the PhD student Damitha Sandeepa Lenadora 📄Paper: https://lnkd.in/gAPpm4P7 𝐒𝐏𝐋𝐀𝐓: 𝐀 𝐅𝐫𝐚𝐦𝐞𝐰𝐨𝐫𝐤 𝐟𝐨𝐫 𝐎𝐩𝐭𝐢𝐦𝐢𝐬𝐞𝐝 𝐆𝐏𝐔 𝐂𝐨𝐝𝐞-𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐨𝐧 𝐟𝐨𝐫 𝐒𝐏𝐚𝐫𝐬𝐞 𝐫𝐞𝐠𝐮𝐋𝐚𝐫 𝐀𝐓𝐭𝐞𝐧𝐭𝐢𝐨𝐧: Transformer models are increasingly using sparsified attention mechanisms (e.g. Mistral) to handle large context lengths. In spite of their importance, we noticed that there does not exist general and high-performance code generation support for diverse sparse attention patterns used by different models. Aggravating this fact is that most sparse attention workloads exhibit moderate sparsity (20%-50% dense) and hence most sparse compilers and libraries are ill-suited to optimize these workloads. To address this gap, we introduce a new sparse format, ACSR, and an associated GPU code generation algorithm, SPLAT, that accelerates a diverse set of sparse attention workloads with a geometric speedup of over 2x. This work is led by PhD student Ahan Gupta 📄 Paper: https://lnkd.in/gzkdmuCW 𝐀𝐮𝐭𝐨𝐦𝐚𝐭𝐞𝐝 𝐕𝐞𝐫𝐢𝐟𝐢𝐜𝐚𝐭𝐢𝐨𝐧 𝐨𝐟 𝐒𝐨𝐮𝐧𝐝𝐧𝐞𝐬𝐬 𝐨𝐟 𝐃𝐍𝐍 𝐂𝐞𝐫𝐭𝐢𝐟𝐢𝐞𝐫𝐬: Abstract interpretation-based neural network verification techniques are popular in the literature. However, they are usually implemented in general tensor programming languages and their soundness is manually proven on paper. We for the first time introduce a new verification system that can automatically prove the soundness of verifiers specified using the NN certifier specification language we built earlier, known as ConstraintFlow. This allows rapid iteration of DNN certifier specifications with push-button feedback about the soundness. This work is led by PhD student Avaljot Singh, who is co-advised by Gagandeep Singh. 📄 Paper: https://lnkd.in/gpscbtKF Hoping to see you all at OOPSLA 2025 (https://lnkd.in/gH4j5ix7?) next week!

43

1 Comment

Reinforcement Pre-Training (Microsoft, June 2025) Paper: https://lnkd.in/eY4PCpUi Abstract: "In this work, we introduce Reinforcement Pre-Training (RPT) as a new scaling paradigm for large language models and reinforcement learning (RL). Specifically, we reframe next-token prediction as a reasoning task trained using RL, where it receives verifiable rewards for correctly predicting the next token for a given context. RPT offers a scalable method to leverage vast amounts of text data for general-purpose RL, rather than relying on domain-specific annotated answers. By incentivizing the capability of next-token reasoning, RPT significantly improves the language modeling accuracy of predicting the next tokens. Moreover, RPT provides a strong pre-trained foundation for further reinforcement fine-tuning. The scaling curves show that increased training compute consistently improves the next-token prediction accuracy. The results position RPT as an effective and promising scaling paradigm to advance language model pre-training."

7

1 Comment

DFT is slow. So i trained a JAX GNN to approximate crystal stability in < 1 hour. For a generative materials pipeline, I needed a fast way to score candidate crystals without running DFT every time. So I built a physics-informed Graph Neural Network (Combining Haiku + Jraph) that predicts formation energy from structure. Heres the trick: I didn't make the model relearn chemistry. Instead, I fed elemental properties (electronegativity, radius, ionization energy, etc/) as node features so It could focus on what matters: geometry + bonding topology. What I got ✅ Converged in ~8 epochs on a single T4 ✅ Test MAE ~0.072 eV/atom ✅ Clean parity plot (tight diagonal) Next step? This model will be used as a fast evaluator inside a SE(3) diffusion model for novel crystal generation. #AI4Science #JAX #GraphNeuralNetworks #MaterialsScience

14

Introducing Reinforcement Pre-training (RPT), a new scaling paradigm for LLM and RL. The key idea is very simple – we reframe next-token prediction as a reasoning task and use RLVR as the protocol to pre-train LLMs with web-scale text data. RPT exhibits superior zero-shot performance on various downstream tasks and provides a strong pre-trained foundation for further (reinforcement) fine-tuning. RPT is also data and sample efficient as it conducts RL learning and reasoning at token level. We believe RPT will be a new scaling dimension for LLM and AI in addition to traditional language model pre-training scaling and test-time scaling. The current results are very promising but preliminary. There will be huge potentials with RPT – a very profound and interesting question is “what if an LLM can (nearly) perfectly predict / reason the next tokens?” as we observed new scaling laws (i.e., the second curve) on token accuracy vs. compute with RPT. Additionally, RL is set to be the paramount learning mechanism for future AI. Consequently, scaling RL (with RPT) will be profoundly critical for reaching the next level of AI. // RPT is one of the most significant ideas in LLM and AI that deserves substantial investment in terms of both compute and talent. I am eager to witness more exciting results to come. // It is not about the size of the cherry and the cake. The cake is the cherry, and the cherry is the cake :) https://lnkd.in/gysKJWcQ

647

10 Comments

Positional encoding and persistent homology have been powerful tools for graph neural networks. This very recent ICML paper combines the two methodologies to leverage both algorithm strengths. It's an interesting new direction for TDA + neural network architectures. https://lnkd.in/eskZZfsW

92

2 Comments

🚀 Smarter, Faster RAG: A Policy-Optimized, Memory-Efficient Framework for Test-Time Reasoning 📄 New arXiv print 🔗 Scaling Test-Time Inference with Policy-Optimized, Dynamic Retrieval-Augmented Generation via KV Caching and Decoding https://lnkd.in/dErdTDf7 🔍 The Problem Retrieval-Augmented Generation (RAG) boosts LLMs with external knowledge but suffers from: ❌ Static retrieval schedules ❌ Distractor-induced hallucinations ❌ Linear memory growth in KV caching ❌ Inefficient test-time compute without reward alignment. 🧠 Our Framework: PORAG + ATLAS + CRITIC We propose a three-component architecture that dynamically optimizes retrieval, rewards factual and high-quality outputs, and reduces compute/memory costs — all without altering the pretrained base model at inference. 1️⃣ PORAG: Policy-Optimized Retrieval-Augmented Generation • Built on Group Relative Policy Optimization (GRPO) • Fine-tunes with QLoRA adapters, guided by two specialized reward heads: – Retrieval Fidelity (factual correctness) – Response Quality (fluency, coherence, utility) • Enables single-shot decoding with KL-regularized policy updates for stability 2️⃣ ATLAS: Adaptive Token-Layer Attention Scoring • Detects when to retrieve via Multi-Layer Attention Gradient (MLAG) • Builds what to retrieve via Layerwise Representation Pooling (LRP) • Leverages hybrid attention-embedding relevance, with entropy-aware scaling to manage retrieval load 3️⃣ CRITIC: Cache Reduction via Importance-based Token Inclusion Criteria • Selectively retains high-impact tokens in KV cache • Hybrid importance signal: attention strength, entropy spread, gradient sensitivity • Reduces memory and latency for long-context generation, with adaptive compression ⚙️ Exhaustive Test-Time Inference Scaling — No Fine-Tuning Required Our framework integrates advanced inference-time scaling techniques that allow LLMs to trade off compute, latency, and answer quality — dynamically and intelligently. These include: ✔️ Entropy-Guided Decoding — invoke deeper reasoning under uncertainty ✔️ Monte Carlo Tree Search (MCTS) — strategic decoding via multi-branch simulation ✔️ Mixture of Agents — task-routed decoding using a pool of specialized policies ✔️ Lookahead Decoding — speculative multi-token generation and verification These methods are plugged into our test-time control module, which uses token-level entropy, compute budget, and reward models to dynamically select the optimal strategy for each input. 🧠 Conclusion Our framework enables scalable, retrieval-aware LLM inference by combining reward-optimized generation, introspective retrieval, and memory-efficient decoding. It is: ✔️ Transformer-agnostic ✔️ Plug-and-play for SLMs ✔️ Ready for enterprise-scale, compute-sensitive deployments 📌 Full paper on arXiv: https://lnkd.in/dErdTDf7 🤝 With Akash Das, SHIVAM GUPTA, and Venkataramana Runkana #LLMInference #TestTimeScaling

17

Recently the Muon optimizer has shown great promise in LLM pre-training. By performing gradient matrix orthogonalization, Muon can effectively avoid/migitate gradient norm explosions or vanishing. In collaboraiton with Prof. Sijia Liu, recently my students Ruijie Zhang, Yequan Zhao, Ziyue Liu, Zhengyang Wang, Dongyang Li and Yupeng Su developed and implemented Teon, a tensorized version of Muon, to improve LLM pre-training efficiency & accuracy. The key idea is to stack multiple gradient matrices together and perform mode-i orthogonalization of the resulting higher-order tensor, which can effectively explore the correlation among the gradient matrices. When tested on GPT-2 and LlaMA models, Teon shows consistent improvement over the conventional Muon baselines. They also provided theory to explain why "tensorization" can offer benefits, and how to maximize the benefit during the tensorization process. Paper link: https://lnkd.in/gpq-kxmW

123

Our colleagues at Lawrence Livermore National Laboratory and ETH Zürich have introduced SPADA, "Spatial Dataflow Architecture Programming Language." This programming language provides precise control over data placement, dataflow patterns, and asynchronous operations while abstracting architecture-specific design. As part of this work, the team implemented a compiler targeting CSL and the Cerebras SDK, and demonstrated that SPADA can serve as both a high-level programming interface and an intermediate representation for DSLs, enabling developers to express complex parallel patterns for the Cerebras architecture. Tal Ben-Nun #IAmCerebras https://lnkd.in/gkUnru3G

60

1 Comment

Our latest paper has been accepted at AAAI 2026. We introduce Structural Alignment, a reinforcement learning approach that aligns large language models with human-like discourse structures, enabling more coherent and well-organized long-form text generation. Our paper, “Structural Alignment: Aligning Large Language Models with Structural Information,” is available here: https://lnkd.in/gx3ZRTZu

49

Looking for a quick overview of Agentic RL for LLMs? Sharing a recent blog post with (interactive) flashcard-style summaries of techniques from "The Landscape of Agentic Reinforcement Learning for LLMs: A Survey (Zhang et al., 2026)," covering planning, tool-use, memory, self-improvement, and reasoning. Link in comments!

14

1 Comment