OpenAI just challenged a lot of the networking defaults we treat as safe. They pulled the curtain back on their 131,000-GPU AI training fabric, and the design is an exercise in aggressive simplification that many network engineers would call heresy. Optimizing the stack was not enough for them. To make synchronous training work at this scale, they threw out three decades of networking consensus.
If you have ever wrestled with a tricky performance optimization task, you know the pull to add one more layer of abstraction to fix a bottleneck. The OpenAI consortium, which includes NVIDIA, Microsoft, and AMD, did the opposite. They disabled BGP, turned off OSPF, and dropped Priority Flow Control (PFC) entirely.
The radical simplicity of a 131,000-GPU AI training fabric
The core problem with massive clusters is tail latency. With 131,000 GPUs running in lock-step, the whole job moves at the speed of the slowest packet. On a conventional network, one congested switch buffer or a single BGP reconvergence event can idle $300,000 worth of compute an hour. The MRC (Multipath Reliable Connection) protocol tackles this by moving the intelligence out of the switches and into the endpoints.
Instead of letting switches choose where packets go through dynamic routing protocols, MRC uses SRv6, or Segment Routing over IPv6. The sender NIC encodes the full path straight into the packet header, so the switches become “dumb” forwarders with no dynamic state. They don’t look up routing tables. They shift the address and ship the packet. It mirrors how we handle AI infrastructure in modern apps, moving logic to the edge where we have the most context.
Why “lossy” Ethernet is actually faster
For years, RDMA deployments leaned on PFC to build a “lossless” fabric. At OpenAI’s scale, though, PFC triggers head-of-line blocking storms, where one congested link can pause an entire collective operation. MRC takes the counterintuitive route: it runs a “lossy” network but adds selective retransmission and packet trimming. When a buffer overflows, the switch trims the payload and forwards the header as a NACK, a negative acknowledgement. The sender learns right away that it needs to retransmit, which avoids the millisecond-long timeouts that wreck training throughput.
// Conceptual logic for path entropy in the MRC NIC
// Instead of one fat pipe, we spray across planes
function bbioon_get_path_entropy($packet_id) {
$entropy_pool = [0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08];
$path_index = $packet_id % count($entropy_pool);
// We don't pin flows. Every packet can take a different plane.
return $entropy_pool[$path_index];
}Endpoint intelligence over network complexity
The lesson goes beyond high-end hardware like the NVIDIA Spectrum-X. It is a shift in thinking. In the WordPress world, we tend to over-engineer the backend with layers of caching and middleware, and then we assume the infrastructure will handle the heavy lifting. OpenAI’s fabric points the other way: the most resilient systems are the ones where the endpoint, the code itself, controls the path the data takes.
They spray packets across eight independent planes, so no two consecutive packets take the same route. If a link fails, the NIC retires that path in microseconds. Compare that to the roughly 30 seconds BGP needs to converge. That gap in recovery time is the payoff for no longer trusting the network to be smart.
If this AI training fabric work is eating your dev hours, I can take it on. I have been wrestling with WordPress since the 4.x days.
Final takeaway: stop guessing, start directing
OpenAI’s decision to contribute MRC to the Open Compute Project signals that the “black box” networking era is ending. Whether you are building a 100,000-GPU cluster or a high-traffic WooCommerce store, the goal is the same: cut tail latency by removing dynamic complexity. If you want the benchmarks, read the full research paper. They are terrifyingly efficient.
