Xiao Ma

We (w/ Shansan Gong, Ruixiang ZHANG, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong) released a family of 7B diffusion language models, DiffuCoder, that specializes on code generation, with a focus on understanding and improving masked diffusion models. A core analysis of DiffuCoder is the autoregressiveness (AR-ness) score, a novel metric that quantifies the causal patterns in decoding, revealing how diffusion models break from strict left-to-right generation for more flexible, non-linear code planning. Recent advances in autoregressive (AR) models dominate code generation, but diffusion-based LLMs (dLLMs) like DiffuCoder offer a promising alternative, especially for complex programming tasks. DiffuCoder explores how these models decode differently—showing less global AR-ness in code tasks compared to math—and how temperature affects both token selection and generation order, unlike traditional AR models. We also introduce coupled-GRPO, a post-training RL method with a coupled-sampling scheme, to reduce performance drops during accelerated decoding, boosting parallelism and efficiency. We use a self-improvement pipeline that leverages AR-ness analysis, coupled-GRPO optimization, and evaluation on benchmarks like AceCode-89k to refine decoding strategies. This approach enables DiffuCoder to navigate diverse code generation pathways and enhance performance with modest computational overhead. Looking ahead, we aim to further leverage Reinforcement Learning to steer code generation through these decoding patterns, with the discrete nature of AR-ness scores providing a foundation for search-based strategies—ideal for the sparse rewards of optimizing complex code structures. Check out our full paper and code for a deeper dive! Paper: https://lnkd.in/gVWU3BDJ Code: https://lnkd.in/gmXTZ_6n Models: https://lnkd.in/gTcKCDr9 #MachineLearning #AI #CodeGeneration #DiffusionModels #NLP

220

5 Comments

Longfeng Wu is an applied scientist intern at Amazon in Sunnyvale, CA, where she has been focusing on developing large-scale generative recommendation models based on semantic identifiers (IDs) aimed at fundamentally transforming how recommendations are generated and delivered. A Ph.D. student in computer science, Longfeng is advised by Dawei Zhou at the Sanghani Center for Artificial Intelligence and Data Analytics.

60

3 Comments

We’re excited to share our #NeurIPS2025 work: LayerIF: Estimating Layer Quality in Large Language Models Using Influence Functions! 📌 Our key idea: LLMs vary widely in how well different layers are trained. Instead of relying on heuristics or weight-based signals, LayerIF uses training-data influence scores to measure how each layer contributes to validation performance. This provides a data-centric, model-agnostic assessment of layer quality. 📈 We find: • Improved expert allocation for LoRA-MoE architectures • Data-driven layer sparsity allocation for pruning • A simple, generalizable pipeline for layer-wise diagnostics • Strong alignment between our influence-based signals and downstream performance If you’re working on LLM training, pruning, interpretability, or MoE systems, we’d love to chat. 📍 Come meet us at NeurIPS San Diego tomorrow at 4:30 PM during the poster session. 🔗 Paper link: https://lnkd.in/gVR284ws

61

2 Comments

Anyscale has contributed Ray to the PyTorch Foundation, joining PyTorch, vLLM, and DeepSpeed under open governance to advance scalable, distributed AI infrastructure. Janakiram MSV (Forbes) explains how this governance model reduces fragmentation and supports transparent collaboration across the AI ecosystem in this article: 🔗 https://lnkd.in/dwy4HpTC #PyTorchFoundation #Ray #OpenSourceAI #AIInfrastructure #DistributedAI

137

1 Comment

I came across an incredible paper from Tsinghua University’s Duan Ran and team, which just won Best Paper at STOC 2025. They’ve cracked the code: a deterministic SSSP algorithm for directed graphs that beats Dijkstra’s classic O(m+nlog⁡n) time by using a clever mix of bounded relaxations (like Bellman-Ford) and frontier clustering to avoid full sorting, achieving O(m⋅log⁡2/3n). What does that mean for us in practical terms? 1. Scalability in action: Huge, sparse graphs—like large road networks, logistics simulations, or network topologies—could be processed much more quickly. 2. Simplified approach: No need for heavy priority queues. Less overhead, less complexity. 3. Better real-time performance: Great for environments where quick adjustments matter, such as navigation apps, real-time analytics, or distributed systems. 4. Updated thinking: Even a “solved” classical problem can evolve—proof that fresh thinking matters. The research is still early-stage and theoretical, but the implications for graph-heavy applications are exciting. Looking forward to seeing this inspire next-gen pathfinding tools and libraries. #SoftwareEngineering #Algorithms #GraphTheory #Innovation #STOC2025

6

Kyunghyun Cho shares his expertise on neural machine translation and deep learning. His contributions to sequence modeling have fundamentally shaped modern NLP approaches. https://lnkd.in/gBvWqsTq #NeurIPS2025 #NeurIPSanDiego

115

Looking for a quick overview of Agentic RL for LLMs? Sharing a recent blog post with (interactive) flashcard-style summaries of techniques from "The Landscape of Agentic Reinforcement Learning for LLMs: A Survey (Zhang et al., 2026)," covering planning, tool-use, memory, self-improvement, and reasoning. Link in comments!

14

1 Comment

📢 LLMs can reason. But can they act, adapt, and collaborate over time? Researchers from UIUC, with collaborators from Meta, Amazon, and Google DeepMind, just released a deep survey on Agentic Reasoning for Large Language Models — and it reframes how we should think about “reasoning” altogether. The core shift: This work argues that strong benchmarks ≠ real intelligence. Instead of one-shot reasoning, agentic reasoning treats LLMs as interactive agents that: - Plan over long horizons - Use tools and external systems - Learn from feedback and memory - Coordinate with other agents They organize agentic reasoning into three layers: - Foundational → planning, tool use, search - Self-evolving → reflection, memory, continual improvement - Collective → multi-agent collaboration and role coordination Why this matters right now: - If you’re building or working with agents, this explains why: - Prompt tricks alone don’t scale to real-world tasks - Memory and feedback loops are becoming first-class design primitives - Multi-agent systems are less about “more agents” and more about reasoning structure It also clearly separates: - In-context reasoning (orchestration at inference time) vs Post-training reasoning (RL and fine-tuning to internalize behaviors) That distinction alone changes how you design agent systems. Feels like a roadmap for where agentic AI is actually headed — not just what demos look cool on Twitter. #AgenticAI #LLMResearch #AIEngineering #Meta #Amazon #Google

37

Most existing #lean4 datasets contain only correct proofs. Models learn error correction with RL, that's expensive. With UW Math AI lab we release a dataset of 260k erroneous Lean proofs with - compiler feedback - reasoning trace - corrected proof Improvements in Error Correction: - Goedel 8B: 2x - Kimina 8B: 3x Paper: https://lnkd.in/gaybt4bd

117

2 Comments

A team from Tsinghua University has shattered the long-standing speed barrier in shortest path algorithms by designing a new solution that beats Dijkstra’s classical approach—without relying on sorting. Traditionally, Dijkstra’s algorithm finds shortest paths by repeatedly sorting nodes by distance, which imposes a fundamental speed bottleneck linked to sorting time. The new algorithm cleverly sidesteps this restriction by clustering boundary nodes and using selective exploration, reducing the number of candidates at each step and avoiding costly sorts. Their breakthrough also adapts Bellman-Ford techniques for directed graphs, combining randomized and deterministic innovations. This lets them solve shortest path problems faster than ever before—even for arbitrary graph weights—a feat considered impossible for 40 years. Their work won Best Paper at STOC and is set to reshape how developers approach pathfinding and graph optimisations. #algorithm #programming #computerscienc

4

👉 My Key Takeaways from Chip Huyen's Recent Interview on Lenny's Podcast Chip Huyen is the author of the widely recognized "AI Engineering: Building Applications with Foundation Models". Link to the podcast: https://lnkd.in/dxZ-tFWX 💡 Importance of post-training.  Pre-training gives you raw capabilities (next token prediction on massive data), but post-training is what makes the model actually usable. SFT on high-quality examples + RLHF. Fine-tuning should be your last resort, not first. Most problems can be solved with better prompts, better data, or RAG. 💡 Evals.  You can't improve what you can't measure. Need multiple types: unit tests (does this specific prompt work?), integration tests (does the whole pipeline work?), regression tests (did we break something?), and user feedback loops. The hardest part isn't writing evals; it's maintaining them as your product evolves. 💡AI products.  Reliability and UX matter more than models. Most AI product failures aren't about bad models: they're about reliability (API limits, latency spikes, poor monitoring) and UX (users don't understand how to use it, doesn't fit workflow). Building reliable platforms and talking to users constantly beats chasing SOTA models. Most insights come from watching users, not from benchmarks. 💡How to improve AI-powered apps.  What people think improves apps: staying current on AI news, chasing newest agentic framework, obsessing over vector database choice, constantly evaluating model benchmarks, fine-tuning models. What actually improves apps: talking to users, building reliable platforms, preparing better data, optimizing end-to-end workflows, writing better prompts. Better prompt engineering beats switching models 90% of the time. A well-crafted system prompt, clear instructions, good examples (few-shot), and proper output formatting can transform a mediocre experience into a great one. 💡 Advice for builders.  Start with user problem, not with cool AI technique. Use the simplest solution that works (often that's a good prompt, not a fine-tuned model). Build evals early. Focus on end-to-end experience. Don't fine-tune unless you've exhausted everything else. Don't treat AI as deterministic (it's not, you need to handle variability). Don't ignore data quality (garbage in, garbage out).

1

1 Comment

Releasing the Memory-QA Benchmark from EMNLP 2025 🚀 Excited to announce the official release of the benchmark dataset for our paper, "Memory-QA: Answering Recall Questions Based on Multimodal Memories"! I presented this work as an oral presentation at EMNLP 2025 in Suzhou last November. Since then, we’ve been working hard to make the resources available to the community. A huge shoutout to my co-first author, Hongda Jiang, for his incredible effort in getting this release over the finish line. 🔗 Code & Data: https://lnkd.in/eGrXX7Uz 📄 Paper: https://lnkd.in/enUXSbag ⚠️ A Note on Data Availability: For those diving into the repo, please note there are some small differences between the open-source release and the dataset used in the paper due to licensing restrictions: - Training Data: While we cannot release the specific QA pairs and memory entries used for training, we have provided all the training images. We also included detailed instructions and prompts in the paper so you can generate your own training data. - Test Sets: The test-l set is available (with a small portion of samples excluded), but the test-s set remains private. We hope this benchmark helps propel further research into multimodal memory QA and recall!

35

Excited to share that the ADAPT group (https://lnkd.in/gV4hZmS8) at UIUC will be presenting three papers at the upcoming OOPSLA 2025 conference next week. Here are some brief details of the papers. 𝐆𝐀𝐋𝐀: 𝐀 𝐇𝐢𝐠𝐡 𝐏𝐞𝐫𝐟𝐨𝐫𝐦𝐚𝐧𝐜𝐞 𝐆𝐫𝐚𝐩𝐡 𝐍𝐞𝐮𝐫𝐚𝐥 𝐍𝐞𝐭𝐰𝐨𝐫𝐤 𝐀𝐜𝐜𝐞𝐥𝐞𝐫𝐚𝐭𝐢𝐨𝐧 𝐋𝐀𝐧𝐠𝐮𝐚𝐠𝐞 𝐚𝐧𝐝 𝐂𝐨𝐦𝐩𝐢𝐥𝐞𝐫:  Most GNN acceleration systems either perform intra-operator optimizations (e.g., TACO, SparseTIR) or inter-operator optimizations (e.g., Graphiler), missing the synergistic benefits of performing both simultaneously. In this work, we propose a language and a compiler with two complementary IRs to effectively track and transform GNN code to perform both classes of optimizations (and more!) to achieve, on average, 2.55x geometric speedup across a wide range of GNNs, graphs, and systems. This work is led by the PhD student Damitha Sandeepa Lenadora 📄Paper: https://lnkd.in/gAPpm4P7 𝐒𝐏𝐋𝐀𝐓: 𝐀 𝐅𝐫𝐚𝐦𝐞𝐰𝐨𝐫𝐤 𝐟𝐨𝐫 𝐎𝐩𝐭𝐢𝐦𝐢𝐬𝐞𝐝 𝐆𝐏𝐔 𝐂𝐨𝐝𝐞-𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐨𝐧 𝐟𝐨𝐫 𝐒𝐏𝐚𝐫𝐬𝐞 𝐫𝐞𝐠𝐮𝐋𝐚𝐫 𝐀𝐓𝐭𝐞𝐧𝐭𝐢𝐨𝐧: Transformer models are increasingly using sparsified attention mechanisms (e.g. Mistral) to handle large context lengths. In spite of their importance, we noticed that there does not exist general and high-performance code generation support for diverse sparse attention patterns used by different models. Aggravating this fact is that most sparse attention workloads exhibit moderate sparsity (20%-50% dense) and hence most sparse compilers and libraries are ill-suited to optimize these workloads. To address this gap, we introduce a new sparse format, ACSR, and an associated GPU code generation algorithm, SPLAT, that accelerates a diverse set of sparse attention workloads with a geometric speedup of over 2x. This work is led by PhD student Ahan Gupta 📄 Paper: https://lnkd.in/gzkdmuCW 𝐀𝐮𝐭𝐨𝐦𝐚𝐭𝐞𝐝 𝐕𝐞𝐫𝐢𝐟𝐢𝐜𝐚𝐭𝐢𝐨𝐧 𝐨𝐟 𝐒𝐨𝐮𝐧𝐝𝐧𝐞𝐬𝐬 𝐨𝐟 𝐃𝐍𝐍 𝐂𝐞𝐫𝐭𝐢𝐟𝐢𝐞𝐫𝐬: Abstract interpretation-based neural network verification techniques are popular in the literature. However, they are usually implemented in general tensor programming languages and their soundness is manually proven on paper. We for the first time introduce a new verification system that can automatically prove the soundness of verifiers specified using the NN certifier specification language we built earlier, known as ConstraintFlow. This allows rapid iteration of DNN certifier specifications with push-button feedback about the soundness. This work is led by PhD student Avaljot Singh, who is co-advised by Gagandeep Singh. 📄 Paper: https://lnkd.in/gpscbtKF Hoping to see you all at OOPSLA 2025 (https://lnkd.in/gH4j5ix7?) next week!

43

1 Comment

Excited to share that our paper, "Uncovering the Deceptive Tactics of Stalkerware," has been accepted at ACM ASIACCS 2026! Stalkerware apps are designed to be invisible, and that invisibility turns out to be deeply engineered. Our work takes a large-scale longitudinal look at the Android stalkerware ecosystem, analyzing how these apps systematically under-disclose their surveillance capabilities, how a distinct class of apps within the ecosystem functions solely as installation guides for stalkerware, and how developer evasion strategies evolve in response to each new Android privacy update. A huge thank you to my co-authors Wenxuan Bao and Vincent Bindschaedler for making this work possible. I also want to thank the Coalition Against Stalkerware for their support in this work. Looking forward to presenting it and engaging with the community in Bangalore! #ASIACCS #MobileSecurity #AndroidSecurity #Privacy

39

Just came across a breakthrough from researchers at Tsinghua University—they’ve developed a new shortest-path algorithm that can actually beat Dijkstra’s classic algorithm in certain cases. What’s new? Their method runs in O(m · log^(2/3) n) time, making it asymptotically faster than Dijkstra’s O(m + n log n) for graphs with non-negative weights. Why it matters: This could reshape fields like: Network routing Logistics & supply chain AI pathfinding Chip design Big data systems It’s a strong reminder that even “solved” problems in computer science can still spark surprising innovations. 📄 Read the full paper here: https://lnkd.in/gYwfcqCR Also I am preparing a vlog diving deeper into this algorithm— I’ll be posting that soon!

14

2 Comments

💡 Standing Out in Tech: Lessons from Lam Research’s Career Webinar by Justin Lai, PMP Lai and Audrey Choi Last week I attended a webinar where Audrey Choi, a former undergraduate student at UC Berkeley obtained a Master’s-level position at Lam Research.   The webinar offered practical advice for students and early-career professionals interested in the semiconductor and tech industry. I came away with insights that I’m excited to apply as I continue developing my skills and projects. Some highlights: ⚡ Projects speak louder than degrees – hands-on experience can really make your resume stand out. ⚡ Be versatile – having multiple complementary skills helps you stand out from the crowd. ⚡ Network strategically – talking to professionals, seeking feedback, and making genuine connections can enhance both your resume and your career path. ⚡ Rejection is part of growth – keep applying, improve your materials, and show continuous development. Events like this are an excellent way to gain practical career advice, learn from experienced professionals, and connect with people shaping the industry. I’m looking forward to putting these insights into practice and continuing to grow in my career! #LamResearch #Semiconductors #CareerAdvice #Networking #ProfessionalGrowth #ProjectsMatter #SkillDevelopment #EarlyCareer

10

3 Comments