Revant Kumar

It’s interesting to see the shift around reinforcement learning — it’s no longer about whether RL is the right approach, but how we can train it more efficiently and at scale. I’m expecting to see a lot more domain-specific applications pop up, both in public and private (enterprise) spaces. https://lnkd.in/gjc3RdcP

18

1 Comment

What I'm Reading This Week 📚 ⚡ Diffusion Coming to LLMs https://lnkd.in/gMrhRsvj Lots of announcements at Google IO this week. One that caught my eye is Gemini's new Diffusion model—applying image diffusion algorithms to text generation for faster speeds. This could be a game changer for latency-sensitive applications like IDE auto-completion and real-time code editing. 🔄 Real-Time Collaboration Architecture https://lnkd.in/ghwrBAxT Matthew Weidner delivers a brilliant deep dive into synchronizing state between central servers and multiple clients. If you've ever wondered how Google Docs manages concurrent edits without chaos, this is your answer. His exploration of operational transformation and server reconciliation patterns gives me new appreciation for the complexity behind seamless collaboration. 📈 Economic Reality Check https://lnkd.in/gfTf2TG9 Thomas Klitgaard from the NY Fed provides a down-to-earth explanation of why the US consistently runs trade deficits. While the current administration's trade war rhetoric continues, one thing is clear: rebalancing trade won't happen without painful adjustments for American consumers. The macroeconomic forces at play are more complex than political soundbites suggest. 🎯 Algebraic Effects Explained https://lnkd.in/g8zeFzWX A fascinating introduction to algebraic effects—essentially resumable exceptions that enable elegant control flow. While this cutting-edge programming language feature might be hard to wrap your head around, the Ante team does an excellent job explaining when and why you'd need such abstractions. 🤖 Google's Coding Agent https://jules.google/ Google launches Jules, their take on cloud-hosted agentic coding loops. Between AI-powered IDEs, local agents like Claude Code, and these cloud solutions, we're clearly still figuring out the right form factor for agentic programming. The overloaded "Codex" branding from OpenAI isn't helping clarity either, but the underlying trend toward autonomous code generation is undeniable. What's your take on the current state of AI coding assistants? Are you finding them more helpful or distracting in your development workflow? #TechArchitecture #AICodeGeneration #DistributedSystems #EconomicPolicy #FutureOfProgramming #AgenticDevelopment #DeveloperTools

15

Is Seattle a better place to hire for early-stage AI companies than San Francisco? Kyle Corbitt, Founder & CEO at OpenPipe, thinks it might be. In this week’s episode of (AI) People, we talk about what it’s like building an AI infra company outside the Bay Area, and why OpenPipe’s Seattle location gives them a hiring edge: “There’s a smaller pool, sure, but less noise, fewer lookalike startups, and just enough ex-Big Tech engineers looking for something more meaningful.” We also cover: – Fine-tuning custom models and agents – The hidden costs of RAG pipelines – Why OpenPipe isn’t trying to be the everything store for LLMs ▶️ Watch on YouTube 🎙️ Listen on Spotify Links below #AI #Hiring #Startups #LLM #FineTuning #AIInfra #Founders

9

2 Comments

⚡ Exciting to see our Block-Max Pruning (BMP) technique in Infinity, an open-source AI-native database designed for LLM applications! In their latest VLDB paper, “Balancing the Blend: An Experimental Analysis of Trade-offs in Hybrid Search”, Hai Jin, Yingfeng Zhang, and co-authors present a rigorous evaluation of hybrid search architectures — combining full-text, sparse, dense, and tensor retrieval. To support efficient sparse vector search at scale, they’ve integrated BMP into Infinity’s SVS engine — a nice validation of our work on fast, top-k lexical retrieval. 🔗 BMP paper: https://lnkd.in/dsc33hGc 🔗 BMP code: https://lnkd.in/dxBxv225 🔗 Infinity: https://lnkd.in/ddRK5mbr 🔗 Hybrid Search paper: https://lnkd.in/dfBuDXmt Great to see ideas from traditional IR continuing to shape the next generation of retrieval infrastructure!

31

Weekly arXiv Scan: 3 Papers that caught my eye (and my Agent’s) My custom Gemini x OpenClaw agent scanned recent papers on arXiv this week in Audio, NLP, and GenAI. It filtered for novelty and engineering rigor. I filtered for practical application and "human" impact. Here are the winners that made the cut during the baby nap time :) 1. Scaling Open Discrete Audio Foundation Models (SODA) https://lnkd.in/eXYnjfpd The Agent's TL;DR: Introduces SODA, a 4B parameter native audio model trained on interleaved semantic, acoustic, and text tokens, establishing new scaling laws for non-cascaded speech tasks. My Take: Human conversation relies on tone, pauses, and emotion. Those are the things that get stripped away when we convert Speech-to-Text for an LLM. By processing audio tokens natively, we aren't just processing data; we are preserving the humanity of the interaction. Native audio-in architecture is what we need for empathetic AI. 2. Reverso: Efficient Time Series Foundation Models https://lnkd.in/eyAgsN_h The Agent's TL;DR: Proposes a hybrid architecture interleaving linear RNNs and long convolutions that matches Transformer zero-shot forecasting performance while being 100x smaller in parameter count. My Take: Time-series analysis and prediction might be the problem where the sledgehammer solution is an overkill. A model that is 100x smaller is a model one can actually use for the personal gains (like some extra signals for investments? maybe?). It’s faster to train, cheaper to serve, and easier to debug. 3. A Generative-First Neural Audio Autoencoder https://lnkd.in/eeFtDpkB The Agent's TL;DR: Proposes an architecture with a massive 3360x temporal downsampling factor, allowing 60 seconds of audio to be represented by fewer than 800 tokens for extremely fast generation. My Take: High fidelity is great, but latency is the user experience killer. Compressing a minute of audio into <800 tokens is a massive engineering win. This reduces the context burden on the LLM significantly, meaning faster, cheaper, and more responsive voice agents. This is a great step towards the feeling of a snappy real-time conversations. The trend this week is “Efficiency over Excess”, finding architectures that do the same work with a fraction of the compute. #NLP #AudioAI #MachineLearning #HumanInTheLoop #arXivScan

39

8 Comments

Why is Redis So Fast and Efficient (despite being single-threaded)? 🔹 In-Memory Storage Redis keeps all data in RAM where access times are measured in nanoseconds, not milliseconds. 🔹 Single-Threaded Event Loop Redis processes commands on a single thread, avoiding locks, race conditions, and context switches. Thanks to I/O multiplexing, it efficiently handles thousands of concurrent client connections using an event loop. 🔹 Optimized Data Structures Redis provides specialized implementations of lists, sets, sorted sets and hashes optimized for both performance and memory efficiency. 🔹 I/O Efficiency Redis uses a lightweight text-based protocol called RESP to handle network I/O, and supports pipelining, allowing clients to send multiple commands in a single request. 🔹 Server-Side Scripting Redis's Lua scripting engine lets you execute complex multi-step operations atomically on the server, eliminating network roundtrips. Read the full article here: https://lnkd.in/gJeSU-pa Subscribe for more such articles.

484

28 Comments

Want to train LLM to reason right on your mac? Check out my article on Reinforcement Fine-Tuning LLMs with GRPO on Apple silicon using MLX framework. I remembered vividly when DeepSeek-R1 burst onto the scene this year, it dominated discussions over Chinese reunion dinners, AI stocks crashed, and friends I knew (including me) were busy asking it to reason their Bazi. A key idea introduced along with Deepseek R1 was Group Relative Policy Optimisation (GRPO), which helps to incentivise reasoning capability in LLMs via reinforcement learning. Since then, there have been courses such as Predibase's course on implementing GRPO, and improvements to the algorithm such as Mistral AI's Magistral LLM which was trained on a more efficient GRPO algorithm. Following the course by Predibase hosted on DeepLearning.AI, I have written an article where I explain step by step each component of GRPO equation, with a worked example to illustrate the algorithm. If you have on hand an Apple silicon computer, I have also included a Jupyter notebook which implements GRPO using Apple's MLX framework. Credits to Awni Hannun for creating the base notebook for LoRA Fine-Tuning with MLX LM. Full disclosure that I created the article and the notebook with the assistance of Google Gemini. If you noticed any errors with either the article or the notebook, please let me know or file a pull request.

889

39 Comments

Meta has taken another clear step in how seriously it is treating AI infrastructure. With the launch of Meta Compute, the company is planning data center capacity at gigawatt-plus scale to support long-term AI demand. As Mark Zuckerberg put it: “Meta is planning to build tens of gigawatts this decade, and hundreds of gigawatts or more over time.” This goes well beyond a single site or region. Planning at this level means power, land, grid access, and specialist talent become the main constraints, not technology. What stands out is the shift in mindset. AI compute is no longer a supporting function. It is being treated as core infrastructure. It will be interesting to see how quickly utilities, local markets, and workforces can scale to match this level of ambition.

13

1 Comment

NVIDIA's DGX Spark started shipping this week. $3,999 for a petaflop on your desk. Research labs currently skip experiments because justifying cloud spend takes three approvals. Startups burn runway on compute before knowing if ideas work. Teams prototype in the cloud, then find their models behave differently in production. Run 200B parameter models locally. Fine-tune 70B models without cloud dependency. Link two units for 405B parameters. The development cycle tightens considerably. The GB10 Grace Blackwell chip is purpose-built for this—MediaTek collaborated on the CPU side, 128GB of memory sits between the CPU and GPU without the usual bottleneck. Different from retrofitting gaming hardware for AI workloads. Training frontier models still requires massive infrastructure. Production inference at scale needs proper compute. But prototyping, testing, fine-tuning—that friction just dropped significantly. Useful ideas die when the barrier to experiment is too high. The path from concept to test matters. $4,000 isn't nothing, but it's orders of magnitude less than building a cluster. NVIDIA's framing this as "democratizing AI"—partially marketing, partially true. The number of teams who can now iterate at this level just expanded considerably.

77

4 Comments