Craig McLuckie

Two of AWS’ Sr. Principal Engineers, Nicholas Matsakis and Marc Bowes, take us inside the development of Aurora DSQL in a fascinating new blog. The post shows how the team scaled write operations without two-phase commit, overcame garbage collection hurdles, and ultimately embraced Rust for both data and control planes. What I find particularly interesting is how the team's journey with Rust evolved. They started cautiously, using it for a single component, and ended up rewriting almost everything in Rust. It's a reminder that sometimes the hard choices - like adopting a new language - can unlock tremendous value. The full post is well worth your time. It's not just about database internals, but also about team dynamics, a culture of learning (and questioning established approaches), and making difficult architectural decisions. Read it here: https://lnkd.in/eMhxBu9e Now, go build!

1,045

16 Comments

Excited to partner with AMZN. We’re co-building a new Stateful Runtime product for agentic applications on Bedrock powered by OpenAI models, expanding Frontier via AWS, scaling with 2GW of Trainium compute, and creating custom models for Amazon apps. https://lnkd.in/gKrRwiiY

393

9 Comments

NEXTGEN INFRA STARTUPS MUST PLAY OFFENSE Historically, software infrastructure / middleware startups have played DEFENSE well. They have built deep, defensible technology moats over a period of time. The founders have been deeply technical, have sold to other developers, and have been content to not have a strong public-facing brand. However, this playbook (and archetype) must change. Infra founders and startups must play OFFENSE and become more dynamic and front footed. Why? First, foundation models are adding more and more infastructural components and nibbling away at what was traditionally considered infrastructure. No company can now comfortably sit and own a part of the stack. It will inevitably erode. Second, end customers are deciding or opining on what infrastructure to use. Guess what? Customers of infra products are no longer just career engineers but ordinary people (including VCs!) Infrastructure now has a PLG motion. Having a customer-facing brand is no longer optional. Finally, it’s possible (or even probable) that your company might need to fundamentally change its business model, and not be an infrastructure company any more. And in this crucible moment, you need to have the muscle and the ability to seize the opportunity and go for it. Consider OpenAI launching ChatGPT or Vercel launching v0. They could never have managed it with a legacy infrastructure mindset. Check out Groq’s homepage - it looks more like a vibecoding startup than a chip company. To summarize, infrastructure startups need to evolve. They must start thinking about the end customers, become much more public-facing with their brand, and develop the muscle to build customer-facing (not just developer facing) products. The archetype of infrastructure founders is changing. They cannot just be technical. That’s tablestakes. They must build a deep understanding of their customers’ customers, be superb at product development, marketing, storytelling and brand building, and be ready to move up the stack at any time. In other words, they must be able to play OFFENSE. That’s what greatness means in infra companies today.

146

8 Comments

There's been some talk around storage unification and zero-copy lakehouse integrations recently, and I wanted to better define these terms as well as look at ways we should evaluate different design decisions in this space. So I’ve published a new post: A Conceptual Model for Storage Unification Storage unification isn't about “one storage format to rule them all,” but virtualizing them into a unified logical model. In this post, I lay out a conceptual framework for thinking about storage unification (aka data virtualization). I explore: 🔹 Internal vs. shared tiering (and why shared tiering is so tricky). 🔹 When materialization makes more sense. 🔹 Client vs. server-side stitching. 🔹 Lifecycle and schema management across evolving formats. 🔹 Seven key considerations for evaluating different approaches. This is the groundwork for a set of posts where I’ll dig into how some real-world systems deliver storage unification, primarily about unifying real-time and historical data. https://lnkd.in/dBiS2kxJ

45

Since recently joining the Scale AI team as SVP of Engineering, I’ve been inspired by how deeply the team understands what it takes to operationalize AI at enterprise scale. Over the past several years, Scale has helped some of the world’s largest enterprises integrate AI across their most complex workflows. Today marks the next chapter in that journey. We’re excited to open-source Agentex, the agentic infrastructure layer in Scale GenAI Platform, built to enable enterprises to manage secure and reliable enterprise AI. We believe Agentex will become the standard layer for hosting and orchestrating AI agents by enabling developers to build freely while giving enterprises the control and reliability they need for mission-critical systems. Starting today, it’s open-sourced and available to everyone. Learn more about how Agentex powers enterprise AI workflows and try it out yourself: https://lnkd.in/gP7kQsMW

474

40 Comments

Have you heard of the llm-d project? It improves LLM performance and drives down costs of inference serving on Kubernetes. It's based around a few key projects including vLLM and kgateway. If you want a deep dive into how it works with a detailed step-by-step "packet flow" through llmd, take a look at this blog I wrote recently. 👉 https://lnkd.in/g2mbEh3m

44

3 Comments

LLMs are powerful, but they are stateless machines. They don’t carry memory or intent, they work with the context you provide at that moment. The same model can feel brilliant in one scenario and completely off in another. The difference is not the model, but the context and use case we design around it. The technology is already here. The real opportunity lies in how we shape it to solve real problems.

104

11 Comments

A super cool part of Modal's internals is how we manage compute capacity. We use a wide range of compute providers in something like 80 regions and have to balance supply and demand continuously. Optimizing cloud costs is a fun problem that can be framed as a linear program. Basically you want to minimize the total spend on compute, subject to finding room for all the current workloads on Modal. We continuously scrape prices across thousands of combinations of instance types and regions, and re-solve this optimization problem every minute. We launch and terminate new workers all over the world every few seconds, 24/7, fully automatically, as a result of these solutions. I often think of multi-tenancy (or resource pooling) as a "free lunch". By aggregating lots of highly variable workloads, we end up with a fairly smooth demand curve. This together with our resource solver lets us run workers at extremely high GPU utilization. Together with our ability to spin up containers very fast, this is how we deliver on the "serverless" promise at high scale. Read the blog post: https://lnkd.in/ejZdY6yJ

203

4 Comments

As I'm leaving AWS re:Invent in Vegas, I’m reminded of a counterintuitive truth: competing with the public clouds is often really good business VCs love to ask, “Why wouldn’t the incumbent do this?” But in markets this large (and evolving this quickly) that’s rarely the right question. The real questions are: Who can assemble the best team, get to market fastest and with the most focus? And where are there opportunities to partner with the biggest players rather than compete? We’ve seen this pattern repeat across the ecosystem: -People assumed Vercel and AWS were destined to compete. Today, they’re strategic partners with CEO Guillermo Rauch in the keynote -Snowflake vs. Amazon Redshift. Now, AWS is one of Snowflake’s strongest distribution channels -In Datadog’s early pitches, the concern was, “If it’s just monitoring AWS, why not use CloudWatch?” Fast forward to now, and Datadog is now $50B+ public company We’re going to see the same dynamics play out with OpenAI and Anthropic. These are becoming foundational platforms. But that doesn’t preclude incredible companies from being built on, next to, and even around them. Some will look competitive. Many will end up as partners. And the biggest outcomes often start in those gray zones

217

26 Comments

Great piece by Emilia David on the LangChain open source integration ecosystem, the power of LangGraph, and our newly released LangGraph Platform Helps put everything you see us working in perspective Link: https://lnkd.in/ghajS7zx

100

5 Comments