Introduction

Artificial Intelligence (AI) and very large (hyperscale) datacenters are each not new concepts individually. What is new is how they are now fuelling each other in an unprecedented growth cycle. The data center (DC) industry is currently going through a period of fundamental structural change, driven by the rapid transition from traditional cloud DCs towards AI DCs. Pivotal to this transformation and a key enabler is fiber densification – a dramatic, structural increase in the volume of optical fiber deployed inside AI data centers.

The implications of fiber densification are far-reaching, impacting everything from the:

• physical layout of the DC 
• the design and production of very small form factor connectors and transceivers
• the strategies of the handful of global, vertically integrated optical fiber manufacturers 
• large scale investments in research and development of next-generation optical technologies.

What Is Driving the Surge in AI Data Center Fiber Deployment?

Cloud DCs vs. AI DCs: A Fundamental Redesign

The architecture of modern DCs is undergoing a fundamental redesign because of the unique demands of artificial intelligence workloads. While traditional cloud computing relied on a relatively predictable growth curve in network bandwidth, the advent of AI has necessitated a phase-change in infrastructure topology.  This shift is not simply an incremental upgrade in technology but a total redesign of DC connectivity, from the rack to the DC hall and campus.

The difference in how AI workloads are processed compared to how traditional cloud workloads are processed is the main driver of the increase in fiber volume we are currently experiencing. 

Why do AI DCs require so much more fiber?

CPU vs GPU

In a standard cloud environment, tasks like web hosting, database management, and file storage are typically processed by Central Processing Units (CPUs) in a hierarchical fashion via access, aggregation and core layers. Data primarily moves between an external end-user and the server – a pattern known as ‘North-South’ traffic: North-bound = data leaving the DC for external destinations; South-bound = data entering the DC from an external source.

North-South traffic demands robust external gateways, but relatively modest internal interconnects.

But this network flow is not what occurs with the ‘all-to-all’ communication required for AI processing with Graphics Processing Unit (GPU) or accelerators.  

AI DCs use iterative, compute-heavy processes where tens of thousands of GPUs must work in tandem as a single logical entity or brain. Interconnecting and synchronizing so many GPUs requires high-performance parallel processing. The network must support the constant synchronization of the AI model parameters and the timely exchange of vast amounts of mathematical modelling data, creating this large volume of internal traffic. This  traffic flow is described as ‘East-West’ traffic, where the flow of data is wholly internal, moving laterally or sideways within the DC – from server to server, or rack to rack. 

This flow now accounts for the large majority of data movement within current day AI DCs. The associated architecture optimizes performance but is very fiber intensive as the number of East-West connections scales with the number of nodes or GPUs deployed.

Where Does Most Fiber Get Deployed in an AI Data Center?

Front End Network vs Back End Network

In a modern AI DC, the network architecture is bifurcated into two distinct layers: the frontend network (FENW) and the backend network (BENW). 

The frontend network functions as the ‘interface’ to the datacenter, handling external user requests, task scheduling, and management traffic. This layer typically utilizes traditional Ethernet-based architecture with speeds reaching 400G. 

The backend network, however, is where the highest fiber densification occurs. Dedicated exclusively to GPU-to-GPU communication, the BENW must provide ultra-high bandwidth, very low-level latency, and a lossless environment to ensure that expensive GPU resources do not sit idle while waiting for data packets.

^{The Real Cost of Network Errors in AI Training}

The scale of East-West traffic is orders of magnitude larger than the traffic moving North-South, in and out of the datacenter.  A single training run on a one trillion-parameter model requires the continuous movement of petabytes of data across the backend fabric. Any link error or packet loss in this network is not good; it can force the entire training batch to be restarted, leading to significant financial losses and delays in model development. Consequently, AI DC operators deploy specialized fabrics such as InfiniBand or high-performance Ethernet for the BENW; and these require ultra-high bandwidth fiber fabrics that use far more fiber than legacy CPU-based networks (or FENW’s) to meet the necessary AI performance metrics.

^{Fiber densification in the AI era – Key Takeaways:} 

• The surge in fiber deployment in AI DCs comes down to a fundamental shift in how AI related data moves, how processors communicate and how computing is now structured in the AI era. 

• East-West (backend/GPU-to-GPU) traffic now dominates data movement inside AI DCs, adding hundreds, even thousands of additional fiber connections per rack.

• To keep the GPU’s, or accelerators, performing optimally they need to be inter-connected using fiber. The combination of these various factors has resulted in unprecedented demand for fiber cable and connectivity. 

A result of the unprecedented demand for fiber is that it is increasingly viewed as a scarce strategic capital asset; multi-billion-dollar, multi-year fiber supply agreements are now being locked in by major data center operators.

^{Frequently Asked Questions}

This article is part of the STL NeuEra series, exploring the optical and digital infrastructure powering the AI revolution. For more insights on fiber densification, AI data center design, and next-generation connectivity, follow the STL Neuera series.