Structured Cabling for On-Premise AI

The growth of Artificial Intelligence (AI) continues to drive the evolution of data center computing. While cloud-based AI solutions dominate today, a notable shift towards on-premise AI deployment is emerging. Enterprises often start with cloud solutions to reduce upfront costs and transition to on-premise systems as their needs become obvious. This shift is driven by:

Panduit had the opportunity to work with a large law firm that serves as an example of utilizing a customized on-premise solution to meet Enterprise AI needs with minimal AI hardware. Common to other businesses, the firm planned to train the AI on its internal documents. They sought a system that could identify all legal briefs related to ongoing cases and provide a comprehensive summary. Instead of having multiple law clerks searching for days, the AI system provided the data within minutes. By training on their database, they ended up needing four H100 servers, which fit into their existing data center without any significant alterations.

A planning challenge that Enterprise businesses face is the high-speed cable connecting the AI servers and switches. AI networks require up to eight times more fiber, and the cable density in racks and pathways is significantly higher, complicating installation, testing, and labeling. In response, the industry developed helpful, structured cabling designed for AI. The top Cloud providers recognized the benefits of structured cabling over 40 years ago and use it for ease of installation, scalability, and long-term reliability – advantages that are exponentially more important for AI.

AI Structured Cabling Benefits

For more information, read the Panduit White Paper: Impact of Structured Cabling on AI Network Performance.

AI systems use MPO fiber cabling in a Rail topology rather than the typical Leaf-Spine topology used in data centers. The Rail Optimized system minimizes the number of hops between servers to achieve the lowest possible latency. Figure 1 shows a simplified example showing the differences between a traditional network and a Rail Optimized network.

Figure 1

Where Rail Optimized networks use more fibers, simplified routing, and reduced congestion are achieved with structured cable.

Effective high-density AI systems use multi-fiber trunks that consolidate multiple cables into single-jacket assemblies, reducing pathway congestion by up to 75%, as shown in Fig. 2.

Figure 2

High-density cabling from Leaf switches to Spine switches is often found in separate rows and needs these trunks. Fig. 3 shows a switch-to-switch routing for a Nvidia DGX H200 or B200 SuperPOD (32 servers) that uses trunks and simplifies patching.

Figure 3

Structured cabling speeds up installation for rack and stacked AI cabinets. Most AI systems are built and evaluated by an integrator, and they ship pre-configured cabinets to the data center, where they roll racks into place and connect cables. Cloud providers follow this process to speed up setup and get equipment operational and revenue-generating more quickly. Structured cabling offers the possibility of having an Overhead Distribution Rack (ODR) suspended from the pathways, allowing or the installation and assessment of rack-to-rack horizontal cabling ahead of time. Short fiber patch cords connect from the server to the ODR after the racks arrive.

Fig. 4 and Fig. 5 illustrate an ODR and its application in a SuperPOD.

Figure 4

Figure 5

Due to the soaring prices of AI servers and the rapid pace of new AI software, businesses choose structured cabling for its scalability in AI systems. With its separation between patch cords and horizontal cabling, moving and adding cables is simpler.

The data center landscape is undergoing rapid change. AI needs high-quality, easy-to-install infrastructure, and businesses can learn from Cloud providers to simplify their own AI networks. Structured cabling systems, such as those used by the Cloud, are essential for a robust and reliable on-premise network. Panduit is ready to help you on this journey.

For more information visit Panduit.com.