When code is cheap, performance is expensive.

Supercomputing for Artificial Intelligence is a practical, systems-oriented guide to understanding what really happens when modern AI models run at scale.

This book is not about writing code faster.

It is about understanding what happens when that code runs — on GPUs, across nodes, under real resource constraints.

In an era where AI tools can generate entire training pipelines in minutes, the real engineering challenge has shifted: performance, scalability, efficiency, and informed trade-offs. HPC for AI is about judgment, not recipes. This book is written for that moment of transition, where generating code is easy, but understanding systems is hard again.

What this book is about

Supercomputing for Artificial Intelligence provides a rigorous yet hands-on introduction to High Performance Computing as it applies to modern AI workloads, with an explicit focus on execution behavior, performance, and scalability.

The focus is explicitly on training, not inference.

Readers are guided from foundational supercomputing concepts to the efficient and scalable training of deep learning models on real supercomputing platforms.

The book integrates system-level reasoning across:

• computer architecture and modern GPU systems
• parallel and distributed execution models
• deep learning frameworks (TensorFlow and PyTorch)
• performance analysis and scalability metrics
• reproducible experiments on production-grade infrastructures

Rather than presenting isolated techniques, the book is structured as a learning path whose technical culmination is the ability to reason about and execute large-scale AI training workloads.

Why this book exists (and why now)

AI-assisted coding tools are changing how software is written. They are not changing the fundamental physics of computation.

Generating code is becoming trivial. Understanding bottlenecks, overheads, scaling limits, and cost–performance trade-offs is not. This book addresses that gap. As the marginal cost of writing code collapses, performance becomes the scarce resource. The goal of this book is to help readers recognize, measure, and reason about that scarcity.

It is written for readers who want to:

• understand why performance behaves the way it does
• measure instead of guess
• scale only when it makes sense
• avoid mistaking “more GPUs” for “better systems”
• develop engineering judgment beyond code-level correctness

In short: to develop engineering judgment in AI systems.

Technical scope and structure

The primary technical focus of the book is the training of AI models on high-performance computing systems. Later chapters guide the reader through:

• efficient single-node training
• data parallelism and distributed training
• scalability analysis and diminishing returns
• end-to-end workflows for modern deep learning models

This includes preparation for training contemporary Large Language Models on distributed GPU-based supercomputers.

Earlier chapters cover:

• supercomputing fundamentals
• system architecture
• software environments and schedulers
• classical parallel programming models

These sections can also be read independently as a rigorous introduction to High Performance Computing.

Topics such as inference optimization, deployment, and edge execution are introduced only where needed for system-level context.

Used in real courses, on real supercomputers

This book is currently used as core reference material in master’s-level courses on supercomputing and artificial intelligence, including the HPC for AI course  (MEI master, FIB-UPC), where students are explicitly trained to reason about performance and scalability, not just to run code.

All examples and experiments are designed to run on real supercomputing platforms, not simplified toy environments. Companion code and fully reproducible experiments are provided.

Open HTML edition (January 2026)

The second edition of Supercomputing for Artificial Intelligence will be published openly in HTML format at the end of January 2026, once the current publishing agreement concludes.

This open edition will make the full content freely accessible to students, researchers, and practitioners worldwide.

Get the book – The book is now fully open

The complete second edition of Supercomputing for Artificial Intelligence is now freely available online in HTML format.

This is not a preview or selected chapters — it is the full 800+ page book, openly accessible for students, researchers, and practitioners.

You can read it here:

👉 https://jorditorresbcn.github.io/supercomputing-for-ai-book/

The open version reflects the most recent edition, refined after intensive classroom use on real supercomputing platforms.

The book is also available in:

• Digital and print editions via Amazon (USA and Spain editions)
• Companion code and experiments on GitHub Repository