PyTorch

Only 3 days remain to join our speaker lineup in Shanghai! 🎤 We are looking for session presentations, panels, and lightning talks for KubeCon + CloudNativeCon + OpenInfra Summit + PyTorch Conference China. If you have been working on technical breakthroughs or real-world use cases, the community wants to hear from you. The portal closes in 72 hours. Get your proposal in today: https://bit.ly/40QIjb2 And don't forget early bird rates end in 1 week! Save now: https://bit.ly/3NG8hv1 #KubeCon #CloudNativeCon #OpenInfraSummit #PyTorchCon #Kubernetes #K8s #CloudNative #OpenInfra #PyTorch #PyTorchFoundation #FutureOfAI #AI #GenAI #MachineLearning #ML #DeepLearning #OpenSource #OpenSourceSoftware #OpenSourceDevelopment #OpenSourceCommunity #OSS #CNCF #CloudNativeComputingFoundation #events #linux #software #CallForProposals #CallForPapers #CFP #CallForSpeakers

🎉 vLLM v0.20.0 is out — 752 commits from 320 contributors, 123 of them new to the project. A few things worth calling out beyond the changelog: 🚀 A platform baseline shift, not just another release. v0.20.0 moves to CUDA 13.0.2 / PyTorch 2.11 / Python 3.14 / Transformers v5 — a deliberately breaking change. The reason: PyTorch 2.11 is the first release to ship a default aarch64 CUDA wheel on PyPI. That means GB200 / Grace-Blackwell users now get working CUDA from a plain `pip install torch` — no `--extra-index-url`, no nightly, no custom build. If you're standing up a GB200 or GB300 box for inference, this is the version cliff to be aware of. 🙏 Thanks to the PyTorch team for shipping it. ⚙️ Engine & performance work: ⚡ FA4 re-enabled as the default MLA prefill backend (head-dim 512 + paged-KV on SM90+) 🗜️ TurboQuant 2-bit KV cache — 4× capacity, with FA3/FA4 prefill support 🧩 Model Runner V2: full CUDA graph for Eagle prefill, multiple prompt logprobs, stale-token accuracy fix 🛠️ Initial vLLM IR skeleton + rms_norm op — foundation for future kernel work 🧪 Online quantization frontend (FP8 / MXFP8 consolidated) 📈 Batch-invariant fused RMS norm: 2.1% E2E latency improvement 🖥️ Hardware breadth this release: 🟢 NVIDIA: Jetson Thor (sm_110), MXFP4 W4A4 CUTLASS MoE on SM100, TRTLLM-GEN NVFP4 MoE 🔴 AMD: Zen CPU via zentorch, MORI EP, AITER MLA + Eagle3, RDNA 3.5/4 device IDs 🔵 Intel: XPU with MXFP8/MXFP4 quant and FP8 KV cache 🆕 New & updated models: DeepSeek V4, Hunyuan v3 preview, Granite 4.1 Vision, EXAONE-4.5, Phi-4-reasoning-vision. 🐋 DeepSeek V4 specifically — v0.20.0 rolls in everything that's landed since the Day-0 docker image: 🔥 MegaMoE — DeepGEMM MegaMoE integration, a mega kernel from DeepSeek. Opt in with `--moe-backend deep_gemm_mega_moe` (Blackwell only). 🔧 MTP > 1 crash on Hopper — fixed 🔧 Indexer top_k correctness fix 🔧 Shared-experts SwiGLU clipping — fixes an accuracy regression on reasoning requests 🔧 Engine hang on long-context inputs — fixed 🔧 Chat template now renders tool arguments correctly If you've been running V4 on the Day-0 image, this is the upgrade to take. 📖 DeepSeek V4 recipe: https://lnkd.in/gR_jJesr 📝 Full release notes: https://lnkd.in/gKMqNs9d Huge thanks to the 320 contributors who shipped this release. 🙌

Need to cut LLM training checkpoint costs? Training LLMs requires periodic checkpoints, full snapshots of model weights, optimizer states, and gradients saved to storage so training can resume after interruptions. At scale, these checkpoints become massive. NVIDIA nvCOMP is a GPU-accelerated lossless compression library that compresses the checkpoint before it leaves GPU memory, no roundtrip – no extra data movement. Developers can easily integrate high-throughput compression directly into their Python workflows (such as PyTorch or TensorFlow). 🔗 Read the full post: https://lnkd.in/gsgFqH_x #PyTorch #OpenSourceAI #AI #Inference #Innovation

Want to train LLMs on longer contexts without re-engineering your entire systems stack? Introducing AutoSP — the first compiler-based solution that automatically optimizes LLM training for long contexts. Under the hood, AutoSP applies a series of compiler passes that trigger sequence parallelism, paired with a curated activation-checkpointing scheme tailored for long-context training. It's integrated directly into DeepSpeed, so enabling long-context training is just a config change away. No more rewiring your stack to push context lengths. Read the blog to learn more 🖇️ https://lnkd.in/gu46-dey ✍ Ahan Gupta, Zhihao W., Neel Dani, Masahiro Tanaka, Tunji Ruwase, Minjia Zhang #PyTorch #DeepSpeed #AutoSP #OpenSourceAI

My latest podcast is an interview from the recent PyTorch event in Paris. In it I speak with Mark Collier, executive director of the PyTorch Foundation and announcements from the event. If you use LLMs and AI tools, the chances are the model it uses were trained with PyTorch, so this is foundational stuff! Enjoy below or search for "Tech Lounge" wherever you get podcasts. https://lnkd.in/d-upHkSp

The countdown has begun: One week left to submit your talk! ⏳ The Call for Proposals for KubeCon + CloudNativeCon + OpenInfra Summit + PyTorch Conference China, 8-9 September in Shanghai, closes on 3 May. This is your opportunity to share your expertise with a global audience across tracks spanning developer experience, ML systems, infrastructure innovation, and open source leadership. Don't miss the chance to help shape the program for this massive cross-community event. Submit by 3 May: https://bit.ly/40QIjb2 Not planning to speak? Register by 6 May + save! https://bit.ly/3NG8hv1 #KubeCon #CloudNativeCon #OpenInfraSummit #PyTorchCon #Kubernetes #K8s #CloudNative #OpenInfra #PyTorch #PyTorchFoundation #FutureOfAI #AI #GenAI #MachineLearning #ML #DeepLearning #OpenSource #OpenSourceSoftware #OpenSourceDevelopment #OpenSourceCommunity #OSS #CNCF #CloudNativeComputingFoundation #events #linux #software #CallForProposals #CallForPapers #CFP #CallForSpeakers

The 2026 PyTorch Docathon is a community event where contributors help improve the tutorials, guides, examples, and documentation that support the PyTorch ecosystem. Running May 5 to May 19, the Docathon includes beginner-friendly issues, skill-based tasks, and community support for both new and experienced contributors. It’s a practical way to contribute to PyTorch while helping make the project easier to learn and use. RSVP for updates and event details: https://lnkd.in/gXVN2TyM #PyTorch #PyTorchDocathon #OpenSourceAI

IBM Research introduced the RITS Platform to provide its research community with shared access to model inference endpoints. vLLM is at the heart of the RITS Platform and has been critical to democratizing access to the latest LLMs as they release. “The vLLM community is vibrant and responsive, and with collaborative expertise, we are able to do great things both upstream and internally by leveraging and contributing to this groundbreaking project. vLLM has been critical to democratizing access to our research community to the latest and greatest LLMs as they release.” – Priya Nagpurkar, Vice President, AI Platform, IBM Research Read our latest case study: 🔗 https://lnkd.in/gVjCnBDm #vLLM #PyTorch #OpenSourceAI

Mark Collier, Executive Director of PyTorch Foundation and General Manager of AI & Infrastructure at The Linux Foundation, opened PyTorch Conference Europe 2026 with a keynote on how the open source intelligence stack advances through coordination across hardware, software, and open source communities. He shared updates on our growth, our expanding set of hosted projects, and why open collaboration remains essential to building AI infrastructure at global scale. 🎥 Hear from Mark: https://lnkd.in/gHmMTHRw #PyTorchCon #PyTorch #vLLM #DeepSpeed #Ray #Helion #Safetensors #ExecuTorch #OpenSourceAI