From Mega-Campus to Mesh: Why AI Infrastructure Is Going Distributed
For three years, AI infrastructure has run on one assumption – the bigger the better. Bigger campuses, which consume more power and resources to handle larger compute workloads. Lacking investment diversification while creating an energy draw large enough to rival whole stats. That assumption is now cracking under its own weight. Power, land, and permitting simply can't keep pace with how fast AI labs want to move. Out of that pressure, a fundamental shift is already underway - away from a handful of massive, centralized campuses, toward a mesh of smaller, interconnected sites working together as if they were one massive machine. Here's why the mega-campus era is hitting its limit, where the cracks are already showing, and what's rising to replace it.
Why "Scale-Up" Hit a Wall
The magnitude of the demand problem cannot be overestimated. Industry estimates (Data Center World 2026) suggest that the potential new AI-driven power demand in the U.S. will be about 200 gigawatts by the end of the decade — just as about 104 gigawatts of existing power capacity is expected to retire. In aggregate, the country has to close a more than 300 gigawatt gap in less than a decade. That gap is not being closed by a few big campuses looking for another multi-gigawatt grid interconnection. These interconnection queues are already longer than what utilities can clear and actual completion rates for projects in queue are still low. In other words, the single-mega-site model is running into a hard physical ceiling well before a capital ceiling.
What "Distributed" Actually Means
The emerging answer isn’t just “build smaller” but “build networks of sites that work as a single computing resource. The clearest example is playing out at the frontier. Google has spread its TPU infrastructure across regional clusters – Ohio and Iowa/Nebraska sites, for instance – that are becoming increasingly tied together to pool power and compute across locations, rather than concentrating it in a single building. Instead of trying to pack that scale into one place, Microsoft and OpenAI have unveiled similarly ambitious plans to link many ultra-large campuses into a single multi-gigawatt training system, rather than trying to fit that scale into one location.
Success hinges on infrastructure that was once an afterthought: data center interconnect (DCI). With high-capacity fiber links between sites, the same real-time, high-bandwidth traffic can now move between buildings as once stayed within the confines of a single building. Now compute and datasets move across sites as the workloads demand. Google’s own network team reports its global backbone saw roughly a 10x increase in WAN traffic from 2020 to 2025, mostly due to this type of cross-site distribution of AI workloads. Training runs are also increasingly structured around this reality, with synchronous and asynchronous coordination strategies to keep GPUs scattered across geography working in step.
The Inefficiencies Nobody's Fully Solved
Distributing compute across sites solves the "one site can't get enough power" problem, but it introduces a different set of inefficiencies that the industry is still working through.
The first is a measurement problem. Power Usage Effectiveness (PUE) is the traditional yardstick for datacenter efficiency. This measures how well a facility delivers power to its compute - it also says nothing about what that compute actually produces. I.e. a site with excellent PUE running poorly utilized inference workloads is still wasting energy. "Tokens per watt" is emerging as the more honest metric, but it's not yet standardized or widely reported.
The second is a reliability tax. Because AI training load is far more variable and critical than typical grid demand, operators running on-site generation for backup or supplemental power often have to overbuild capacity by 30% to 70% relative to actual average demand. The result – a significant amount of stranded, underused capacity built purely for resilience.
The third is physics. Distance between sites isn't free. Recent modeling of synchronous per-step data parallel training found that the useful range for splitting a training job across two clusters sits roughly between 10 and 100 kilometers. Once we go beyond this range, fiber latency starts eating into the compute-communication overlap that makes distributed training viable at all. But this says nothing about asynchronous training methods (built specifically to tolerate cross-continent latency), or about inference and independent job scheduling, where the per-request latency budget is orders of magnitude more forgiving. The distinction matters and is the difference between a hard physical ceiling and a routing problem.
Finally, there's coordination fragility. Running compute across independently sited, independently powered facilities introduces stragglers (slow nodes dragging down a whole training run), fault tolerance headaches, and even silent data corruption. These errors don't crash a job but quietly degrade its output. None of these problems exist in the same form on a single unified site. They are the direct cost of going distributed.
The Hurdles Slowing the Transition
Beyond the technical inefficiencies, a handful of very physical bottlenecks are shaping how fast and where this distributed infrastructure can be built.
The most immediate constraint is supply of equipment. Specifically transformers, switchgear, and batteries which makes up less than 10% of a datacenter's construction cost, but also are now the real pacing item for the entire build. A multi-billion-dollar campus can could sit idle for months waiting on a single transformer order. The U.S. has relied heavily on imports to mitigate this issue. Canada, Mexico, and South Korea now are major suppliers, while transformer imports from China alone grew more than fivefold between 2022 and 2025. And this is doesn’t even take in to account the volatile geopolitical backdrop influencing trade at the moment. (It's worth noting that widely circulated claims this year that "half of all 2026 U.S. datacenter capacity" would be delayed or canceled have been challenged by infrastructure analysts as an overreach of the underlying data. The supply chain strain is real, but the most dramatic framings of it deserve some skepticism.)
Permitting and local opposition add another level of friction. Community concerns about water use, noise and grid strain have scuttled or delayed data center projects worth tens of billions of dollars over the past two years. The larger the request, the greater the friction tends to be. A single request for multiple gigawatts of new grid capacity is subject to much more scrutiny, and takes much longer to get through, than a number of smaller, easier-to-site requests.
Quietly, fiber availability has become a siting constraint in its own right. GPU clusters have very bursty all-to-all communication patterns that require much more bandwidth than what metro fiber rings were originally designed to support, so operators are verifying fiber routes before signing power contracts.
Where the Solutions Are Heading
None of this implies we should abandon large sale projects. It suggest getting smarter about how the scale is assembled. A few advances are gaining real traction:
Faster-to-deploy, distributed power, sited closer to where compute actually needs to sit rather than concentrated behind one enormous interconnection request. This reduces both queue time and local opposition by shrinking the size of any single ask.
Efficiency gains at the silicon layer, from custom accelerators to more efficient general-purpose chips, squeezing more useful compute out of every megawatt.
Diversify supply chains for equipment such as transformers, switchgear, and batteries.
Coordination and orchestration software that can manage a portfolio of smaller, geographically separate sites as if it were a single compute resource. This directly addresses the straggler, fault-tolerance, and utilization problems described above, which today mostly get solved with brute-force overprovisioning.
Next-generation optical interconnect, including early standardization efforts around optical compute interconnect and coherent fiber technology.
The Shape of Things to Come
The big picture is becoming obvious even though all the details are still being fleshed out; the industry won't fill the "power gap" with fewer and larger campuses. The industry will fill the power gap with many smaller, optimally placed, and better connected facilities, which will be connected via fiber and software to create a facility that acts as if it is a single large computer. While mega-campuses will continue to be built, they will not be the sole or even primary model on the table. Compute has never been the bottleneck. The bottleneck has always been the idea that compute has to live in one location to act as one system; and this idea is currently in the process of being torn down gigawatt-by-gigawatt.
References
Data Center Knowledge — "Data Center World 2026: Real Estate, On-site Power Speed AI Buildout" — https://www.datacenterknowledge.com/energy-power-supply/data-center-world-2026-real-estate-distributed-power-reshape-ai-buildout
Data Center Knowledge — "Data Center World 2026: AI Pushes Infrastructure to New Limits" — https://www.datacenterknowledge.com/build-design/data-center-world-2026-ai-pushes-infrastructure-to-new-limits
SemiAnalysis — "Multi-Datacenter Training: OpenAI's Ambitious Plan To Beat Google's Infrastructure" — https://newsletter.semianalysis.com/p/multi-datacenter-training-openais
SemiAnalysis — "Stop Saying Half of 2026 US Datacenter Capacity Is Canceled" — https://newsletter.semianalysis.com/p/stop-saying-half-of-2026-us-datacenter
Google Cloud Blog — "Data center and global networks built for AI era" — https://cloud.google.com/blog/products/networking/data-center-and-global-networks-built-for-ai-era
Tom's Hardware — "Half of planned US data center builds have been delayed or canceled..." — https://www.tomshardware.com/tech-industry/artificial-intelligence/half-of-planned-us-data-center-builds-have-been-delayed-or-canceled-growth-limited-by-shortages-of-power-infrastructure-and-parts-from-china-the-ai-build-out-flips-the-breakers
AI 2027 Tracker — "Massive datacenter buildouts continue" (citing IEA 2026 energy and AI analysis, Data Center Watch) — https://ai2027-tracker.com/predictions/datacenter-buildout/
arXiv — Papavasileiou et al., "Modeling the Impact of Fiber Latency on Compute-Communication Overlap in Geo-Distributed Multi-Datacenter AI Training" — https://arxiv.org/abs/2605.19169
TechPlusTrends — "AI Data Center Power Requirements 2026: The Grid-to-Chip Guide" — https://techplustrends.com/ai-data-center-power-requirements-2026-guide/
DEV Community — "Fiber Scarcity Now Dictates AI Data Center Locations" — https://dev.to/lse-group-corporation/fiber-scarcity-now-dictates-ai-data-center-locations-5e97
Author's note: The author is involved in ventures developing distributed power and orchestration solutions for AI infrastructure.
Comments