The pace of technological innovation is blazing fast. Cutting edge breakthroughs like AI, VR, autonomous vehicles, and quantum computing all require processing huge amounts of data at sub-millisecond speeds. Our computing infrastructure desperately needs to keep up. Legacy architectures are buckling under immense workloads. Even the latest multi-core CPUs, GPUs, FPGAs, and NICs have limited I/O connectivity for linking add-in cards and peripherals. This spells trouble for scaling next-gen systems. Enter high performance PCIe switches to save the day!
What is a PCIe Switch?
A PCIe switch acts as a multiplier for PCIe lanes. It‘s an intelligent I/O intermediary that sits between your host system and endpoints. By expanding native PCIe connectivity of a platform, a single switch can enable attaching numerous bandwidth-hungry devices. Whether you need vast amounts of GPUs, SSDs, or FPGAs for your application, a switch provides the flexible I/O backbone to make it possible.
Internal Anatomy
Internally, a PCIe switch has three core ingredients:
- Upstream port(s): Connect to CPU for base PCIe lanes
- Downstream ports: Increased number of ports for endpoints
- Switching logic: Manages dynamic routing of traffic
This combination delivers transparent PCIe multiplication. The switching logic receives requests, determines which downstream port should handle it based on address mapping, and pushes it out the correct port. Devices see no difference versus being directly wired to native CPU PCIe. It‘s plug-and-play expansion.
Two Primary Archetypes
There are two main switch topologies – fanout and fabric. Both expand connectivity, just in slightly different fashions:
Fanout: Simple one upstream port to many downstream ports. Direct paths to endpoints. Easy to deploy but supports only one host system.
Fabric: Mesh design allows multi-host architectures. More complex routing but enables sharing endpoints across systems.
Topology varies based on application needs. Do you need massive I/O for one host, or resource pooling across many? That shapes architecture choices.
Why Are PCIe Switches Crucial?
We live in an era of accelerators. Workloads are diversifying. An ASIC handles AI inference differently than a GPU crunches financial risk models or database analytics. We need specialty processing suited to applications.
However, most accelerators weren‘t designed for linking together. They focus on their workload rather than collaboration. This is where switches save the day – they act as the literal switching fabric enabling accelerators to interoperate efficiently.
Instead of re-architecting every accelerator to directly communicate, systems utilize a switching intermediary. Much easier!
This is a vastly superior approach compared to old shared bus architectures. Shared parallel buses bottlenecked performance since devices contend for bandwidth. Switched designs keep traffic isolated for non-blocking throughput to each accelerator.
In short, PCIe switches enable composing tailored hardware blocks into versatile architecture instead of relying on monolithic general purpose systems. Specialization is the future!
Diving Into Use Cases
There‘s no shortage of applications that rely on the connectivity of PCIe switches:
Machine Learning and AI Training
Deep neural network training workloads necessitate data parallelism across multiple GPUs to converge models. The largest AI supercomputers compose thousands of accelerator cards. For example, NVIDIA‘s Selene cluster links 560 DGX A100 nodes with 8x A100 GPUs per node. We‘re talking over 4,480 high speed GPUs!
There‘s simply no way to directly cable that many accelerators to hosts at scale. The solution is using PCIe switches as the backbone for modular computing clusters. Each chassis mounts a PCIe switch to compose multiple A100 GPUs that fan out over fast networking to nodes. This flexible composability fuels incredible AI innovations.
Accelerated Database Analytics
Real-time database analytics on vast datasets also leverages PCIE switching to harness GPU parallelism. Queries that would take minutes on CPU hardware finish in seconds with a GPU boost. To scale this out, database servers utilize PCIe switches to mount many GPUs or FPGAs per system.
Some architectures even disaggregate GPU pools across PCIe fabric networks. Any server can tap into shared accelerator resources which massively improves efficiency. This would never be possible without the connectivity of high-radix PCIe switches.
Software Defined Networking (SDN)
SDN and network function virtualization (NFV) are vital technologies that underpin cloud infrastructure. They involve replacing dedicated routing hardware with virtual network functions running on commodity servers. However, processing packets in software at line rate speeds requires I/O acceleration.
This is where FPGAs programmed as network accelerators shine. PCIe switches play a key role interconnecting arrays of FPGAs to maximize throughput. By composing purpose-built network function accelerators, switches enable impressive software-defined networking performance.
Expanding NVMe Storage
Modern flash-based storage relies on the phenomenal bandwidth of PCI Express and NVMe. But even the highest core count CPUs still offer a limited number of native PCIe lanes. Direct attaching lots of NVMe drives for blazing fast IO performance quickly saturates lanes.
This is where PCIe switches save the day yet again! With fanout or fabric switching, we can massively overcome CPU PCIe lane limitations to enable rack-scale NVMe-oF solid state storage pools disaggregated across the network. The switching fabric is the key enabler linking vast racks of NVMe drives.
Diving Into Architectural Considerations
There are several key factors to weigh when architecting systems leveraging PCIe switches:
Topology Tradeoffs
As discussed earlier, fanout and fabric topologies each have pros and cons. Fanout is simpler for one host while fabric allows resource pooling across systems. Depending on needs you choose one vs the other.
However, we can also blend designs. For example, we could configure a system where groups of accelerators fanout to individual hosts, and those hosts in turn share endpoints over a PCIe fabric. Hierarchical switching is powerful!
Generational Bandwidth
With each PCIe generation, lane bandwidth doubles:
| Version | Lane Bandwidth |
|---|---|
| PCIe 3.0 | ~1GB/s |
| PCIe 4.0 | ~2GB/s |
| PCIe 5.0 | ~4GB/s |
| PCIe 6.0 | ~8GB/s |
Latest generation switches boost total throughput. However, mismatches anywhere along the path limit speeds. Using a Gen3 switch to connect Gen4 GPU to Gen 5 CPU wastes potential. Plan carefully!
Port Counts and Bifurcation
More switch ports means we can attach more endpoints. For example, a 16-port PCIe 4.0 switch with each port allocated 4 lanes yields 64 downstream lanes. Routed intelligently by the switch, this lets us compose many accelerators.
Additionally, bifurcation modes dynamically split ports to better match endpoint needs. An 8-lane port could instead be used as two 4-lane ports. This optimizes matching downstream device bandwidth demands with appropriate port widths.
Getting port counts and bifurcation modes right lets us dial in configurations to perfectly suit workloads. Planning ahead for future bandwidth needs also prevents costly switch upgrades down the road.
What Does the Future Hold?
PCIe continues pushing performance boundaries with speeds doubling nearly every two years. We will soon have PCIe 6.0 delivering a mind-blowing ~64GB/s per lane! Switches will allow scaling incredibly fast storage and accelerator pools far beyond what CPUs natively offer.
Even more interesting is the evolution of new interconnects riding on top of PCIe infrastructure – most notably Compute Express Link (CXL). CXL promises cache and memory coherence between CPUs, GPUs, FPGAs and other endpoints. This could enable advanced composable architectures by pooling resources.
For example, PCIe switches coupled with CXL might let GPUs directly access disaggregated networked storage without traversing the host CPU and suffering NUMA effects. Or provide shared virtual memory across heterogeneous accelerators. Exciting innovations ahead!
The Crucial Role of PCIe Switching
PCI Express switches play a pivotal role interconnecting diverse accelerators and I/O to compose cutting edge architectures. They overcome legacy limitations of CPUs only offering small PCIe port counts while delivering tremendous bandwidth scalability.
Looking at challenges of crunching exponentially growing datasets across AI, blockchain, IoT, quantum computing and more – it‘s impossible to rely on monolithic general compute. We desperately need specialty processing tailored to workloads, and crucially a way to link these technologies together. PCIe switching provides the answer!
Expect to see PCIe switches embraced everywhere as the secret weapon driving orders of magnitude acceleration for next-gen infrastructure. Buckle up, the future of composable computing built on switch fabrics looks blindingly fast!
