NVIDIA's Vera Rubin is the first AI system that makes "run frontier AI inside our own building" a line item a mid-size company can actually price. It is also the most over-claimed piece of hardware in the industry right now. This is the version with the numbers attached.

What Vera Rubin actually is

Vera Rubin is not one chip. It is a rack-scale system, and the shipping-era product is called Vera Rubin NVL72: 72 Rubin GPUs, 36 Vera CPUs, 18 compute trays, and 9 switch trays, all liquid-cooled, in a single cabinet. NVIDIA rates it at 3.6 EFLOPS of sparse NVFP4 inference and 2.52 EFLOPS of dense NVFP4 training, with 20.7 TB of HBM4 memory and 1,400 TB/s of aggregate memory bandwidth. [1] [2]

Before we go further, one naming trap that trips up almost every article written about this platform.

NVIDIA's March 2025 GTC keynote put a system called NVL144 on the roadmap for the second half of 2026, and Rubin Ultra NVL576 for the second half of 2027. [3] The 2026 commercial material instead centers on NVL72. NVIDIA has never publicly explained that shift, so you should be suspicious of anyone who tells you NVL144 was "renamed" or "cancelled" — nobody outside NVIDIA knows, and there is a separate NVL144 CPX product that makes the shorthand actively misleading. [4] If you are budgeting for a deployment today, the system you can buy is the NVL72.

Inside the rack

The building block is a Vera Rubin Superchip: one Vera CPU plus two Rubin GPUs. A compute tray holds two Superchips. Eighteen of those trays, plus nine switch trays, make the rack.

Each Rubin GPU is itself two chips. NVIDIA's architecture disclosure describes a package containing two reticle-limited compute dies joined by NV-HBI, totaling 336 billion transistors, 224 streaming multiprocessors, and 896 Tensor Cores. [5] Each GPU carries 288 GB of HBM4 at 19.2 TB/s, with 3.0 TB/s of NVLink to its neighbours. [2]

Those two figures used to be different, and it is worth understanding why before you read anyone else's spec sheet.

In September 2026, NVIDIA quietly revised the shipping specifications for Rubin. The preliminary numbers — the ones in the GTC decks, in the architecture blog posts, and in most articles still being published — were 22 TB/s of HBM4 bandwidth and 3.6 TB/s of NVLink per GPU. The shipping numbers are 19.2 TB/s and 3.0 TB/s. At rack scale that means 1,400 TB/s of aggregate memory bandwidth instead of a planned 1,580, and 216 TB/s of NVLink instead of 260. Compute is unchanged at 3.6 EFLOPS sparse NVFP4 inference and 2.52 EFLOPS dense training. Scale-out networking went up, from 28.8 to 32.4 TB/s. [43]

NVIDIA has not explained the reduction. The most likely cause is HBM4 supply: the 10.75 Gbps/pin rate originally targeted sat well above the HBM4 specification, and holding it across a production ramp may not have been achievable. Whatever the reason, the consequence for a buyer is simple. Anyone quoting 22 TB/s is quoting a withdrawn number, and a proposal built on it overstates memory bandwidth by about 11%.

The Vera CPU is where the "Vera" comes from, and it is easy to skip past. It is an NVIDIA-custom Arm v9.2-compatible part with 88 "Olympus" cores, 176 threads, up to 1.2 TB/s of LPDDR5X bandwidth, and 1.8 TB/s of coherent NVLink-C2C linking it to its two GPUs. [6] That coherence is the point. The CPU and GPUs share one address space, which is why a rack can act like one machine instead of 72 networked servers.

Then there is the copper. NVIDIA describes the rack's rear spine as four cartridges carrying 5,000 cables and more than two miles of copper. That is what it takes to give every GPU 3 TB/s of NVLink 6 to its neighbors, 216 TB/s across the rack.

100%

Schematic, not to scale. Tray and package layouts follow NVIDIA's published descriptions; slot proportions are illustrative. Power, weight, and physical dimensions for a Vera Rubin rack are not publicly specified — get the selected OEM's facility engineering package before approving a site.

What one rack can compute

Peak numbers are the easiest thing in this article to misread, so here is the honest split.

3.6 EFLOPS is sparse NVFP4 inference, the most aggressive precision-and-sparsity setting on the datasheet. It is a capability ceiling, not a throughput promise. Real serving throughput depends on the model, the context length, the concurrency, the batch size, the serving stack, and how much power you let the rack draw.

What that ceiling buys in practice is the ability to hold something genuinely large in one coherent domain. 20.7 TB of fast memory in one rack means a trillion-parameter-class model fits with room for long context and the key-value cache that makes long conversations expensive everywhere else. This is the real change: not that Rubin is faster at a given task, but that the class of model you can run without a network hop is now much bigger.

NVIDIA also ships a DGX Rubin NVL8 — eight Rubin GPUs in a node-scale system with Intel Xeon 6776P CPUs. [7] It matters for our audience because it is the on-ramp: not every company needs 72 GPUs, and not every building can host them. It also took the heaviest revision of the family, losing 300 GB of HBM4 (2.3 TB down to 2.0 TB) and dropping memory bandwidth from 176 to 130 TB/s and NVLink from 28.8 to 24 TB/s. [43] Oracle plans to offer it as HGX Rubin NVL8 in an x86 configuration with two 64-core Intel Xeon 6 CPUs, 4 TB of DDR5, and 6.4 Tbps of RDMA networking. [44]

Idle. Press Run to send a prompt through the rack.

Illustrative. Prefill and decode are the two phases of transformer inference: prefill processes the whole prompt at once and is compute-bound, decode generates one token at a time and is memory-bandwidth-bound. That is exactly why a rack with 1,400 TB/s of aggregate memory bandwidth helps long-context and high-concurrency work, and why the same rack can be fast on one workload and unremarkable on another.

The power story, which is the real story

This is where the numbers get interesting, and where nearly every article gets it wrong. There is no single NVIDIA rack wattage. What exists is a set of figures that answer different questions. Click any bar below to see what it actually is.

Select a bar to see what the number really measures.

Rack IT load 227 kW

Electricity price 9.77 ¢/kWh

Assumed PUE 1.20

Annual IT energy1.99 GWhrack load only

Annual electricity$194,278IT load only

With cooling overhead$233,134at the PUE above

Electricity per GPU-hour$0.3172 GPUs, all year

Defaults reflect ASUS's published MaxP envelope for its Vera Rubin NVL72 AI Pod (227 kW) and the U.S. industrial electricity average for July 2026 (9.77 ¢/kWh). PUE is an assumption, not a Rubin specification. Constant full-load is assumed; real workload profiles are lumpier. Excludes demand charges, capacity charges, taxes, local tariffs, cooling infrastructure capex, support, and financing. The per-GPU-hour figure is electricity only — compare it with the surveyed three-year Rubin rental rate of over $8.50 per GPU-hour, and remember that rental includes the hardware.

For context, HPE lists GB200 NVL72 at 132 kW nominal with a peak near 192 kW, and GB300 NVL72 lands around 135 kW with peaks reported between 142 and 155 kW depending on whose reference architecture you read. [12] [13] So a Vera Rubin rack is not a leap into a new order of magnitude in draw. It is roughly a Blackwell-class electrical footprint that does far more useful work per watt.

What is new is the plumbing. Vera Rubin is 100% liquid-cooled with a 45°C (113°F) inlet design point, which is hot enough that warm-water loops can reject heat with far less chiller work than the cold-water systems of a decade ago. And NVIDIA's forward architecture for the megawatt-class racks that follow moves the facility to 800 VDC distribution, converting 13.8 kVAC to 800 VDC centrally and stepping down to 12 VDC at the rack, which cuts current, copper, and conversion stages. [15] The OCP-era MGX 1.2 specification adds rack-scale blind-mate liquid cooling and a liquid-cooled 5,000A busbar, which is what a 250 kW rack needs in place of the copper busbars a 130 kW rack could get away with. [45]

Where these racks live today

Here is the honest status as of 4 November 2026, sorted by how strong the evidence is. Click any entry to expand it.

The headline change since this post was first drafted: Vera Rubin is no longer a roadmap you have to take on faith. Oracle made Vera Rubin NVL72 and Vera CPUs generally available on OCI on 27 October 2026, with plans to expand OCI Superclusters to multi-gigawatt scale. [44] Google Cloud announced A5X instances built on Vera Rubin NVL72 with general availability in November 2026, scaling toward as many as 960,000 Rubin GPUs across multiple sites and expanding into Europe. [46] Two of the three largest clouds will rent you a Vera Rubin rack by the hour within weeks of publication, which matters more than any benchmark in this article: it means you can measure the platform on your own workload before you commit a building to it.

Read that list in order and the shape of the moment has changed since this post was first drafted. In mid-October it was a roadmap with one provider-reported production case and two hyperscalers still in labs. By the start of November it was generally available on Oracle, weeks from general availability on Google Cloud, shipping in containerised form from early 2027, and ramping across five clouds. The silicon is real, the ordering path is real, and the gigawatt announcements are still mostly intentions. If you are planning a 2026 or 2027 private deployment, you are buying into a platform in its first production year — which is exactly what you should expect to pay a premium for, and exactly why the cloud availability now matters: you can rent one by the hour and measure it before you commit a building to it.

How much should you trust the performance claims?

Four different kinds of number get quoted as "Rubin performance," and they are not interchangeable.

The one claim I would hold at arm's length is NVIDIA's up to 30× AgentX versus GB300 at 160 TPS/user, which was explicitly pending independent review when it was published. Treat it as a hypothesis with a marketing department behind it.

None of this supports the sentence "Rubin is 10× faster than Blackwell." There is also no like-for-like public Rubin-versus-H200 performance-per-watt test with matched precision, workload, and serving stack, so anyone quoting that multiplier is extrapolating.

Why private AI, and where the usual argument goes wrong

Most of the writing on this subject gets the compliance story backwards, and I want to correct it because the corrected version is more useful.

The popular claim: regulated companies legally cannot use public cloud inference.

What the rules actually say. HIPAA explicitly permits electronic protected health information to be processed in the cloud, provided there is a compliant business associate agreement and the entity meets its other duties — a cloud provider handling encrypted ePHI is a business associate. [29] GDPR does not ban cloud inference; when personal data goes to an organization outside the EEA, Chapter V transfer conditions apply on top of the usual legal basis, minimization, security, and processor-contract obligations. [30] ITAR has a limited exception for unclassified technical data protected by specified end-to-end encryption under 22 CFR §120.54, conditional on encryption strength and country and key controls. [31] And banks are not forbidden from cloud AI; they remain accountable for third-party risk — security, subcontractors, data access and retention and destruction, auditability, continuity, and regulatory access. [32]

So the constraint is not a wall. It is a stack of conditions: where the data physically sits, who can be compelled to produce it, which subprocessors touch it, how long it is retained, whether you can audit the environment, and whether the contract survives a subpoena or a change in provider terms.

That is the fork where a rack matters. The question stops being "is cloud AI legal" and becomes "can we accept this provider's terms, subprocessors, retention policy, and jurisdiction — for this workload?" For a customer-support summarizer, the answer is usually yes, and you should use the cloud, because it is cheaper and better. For the model you fine-tuned on twenty years of claims data, or the underwriting logic that is the company, or client matter files under privilege, the answer changes. ABA Formal Opinion 512 puts lawyers on the hook for analyzing confidentiality risks and vendor terms, and notes that self-learning tools can require informed consent before client information is entered at all. [33]

At the extreme end, the answer is not "put it in a public endpoint." It is "bring the capability into the accredited environment." On 1 May 2026 the Department of War announced that NVIDIA and seven other AI companies would deploy capabilities into classified IL6 and IL7 networks. [34] That is the same instinct as an on-prem rack, taken to its logical conclusion.

The survey data points the same direction without proving causation. Dell found 82% of 500 IT leaders preferred on-premises or hybrid data management. [35] Cisco found 90% of more than 2,600 security and privacy professionals believed locally stored data is inherently safer — while 91% thought global providers protect data better than their own organizations do. [36] Read together, those two numbers describe what companies actually want: control over the data path, not a claim of superior security.

And to be clear about the limit of the argument: on-premises is not compliant by default. Putting a rack in your building does not create access control, audit logging, model governance, data minimization, incident response, or export-control analysis. It relocates the problem; it does not solve it.

The math for a mid-size company

Here is the arithmetic, with every assumption visible, so you can substitute your own.

Capital. NVIDIA publishes no list price. Public reporting puts a quoted Vera Rubin NVL72 system in the $5M–$7M range, and a separately reported Morgan Stanley model lands at $7,803,148 per rack. [37] [38] Treat both as preliminary budget anchors with unclear bundle scope, not invoices.

Power. Using ASUS's published envelope of 187–227 kW for its NVL72 system, and the U.S. industrial electricity average of 9.77 cents/kWh from July 2026:

Assumption	Arithmetic	Result
IT load, MaxQ	187 kW × 8,760 h	1,638,120 kWh/year
IT load, MaxP	227 kW × 8,760 h	1,988,520 kWh/year
Electricity at MaxQ	1,638,120 × $0.0977	$160,044/year
Electricity at MaxP	1,988,520 × $0.0977	$194,278/year
Same, at an assumed PUE of 1.20	× 1.20	$192,053–$233,134/year

That is a constant-full-load illustration. Real workload profiles are lumpier, and it excludes demand charges, capacity charges, and local tariffs.

Space. CBRE's Q1 2026 asking rents for 250–500 kW requirements were $190–$235 per kW-month in Northern Virginia and $200–$230 in Chicago. [40] A 250 kW reservation in Northern Virginia is roughly $570,000–$705,000 per year before electricity. If you own the building, the analogous burden is power and cooling infrastructure capex rather than rent.

The own-versus-rent break-even. SemiAnalysis surveyed three-year Rubin rentals at over $8.50 per GPU-hour. [27] At full use, 72 GPUs for 8,760 hours is more than $5.36 million per year. Against that comparator, buying breaks even at roughly 43%–59% utilization on the stated assumptions, rising to about 64% if the hardware costs $7.8 million. That calculation covers identical Rubin hardware rented versus owned. It says nothing about substituting a proprietary API model, because you cannot self-host GPT, Claude, or Gemini, and no public matched rack-capacity result exists.

One thing that changed in late October makes this decision easier to de-risk. With Vera Rubin NVL72 now generally available on OCI and arriving on Google Cloud, you can rent the exact hardware you are considering and benchmark your own workload on it before you commit capital. The break-even above is a calculation; a measured utilization curve from a rented rack is evidence. If you are anywhere near the 43–64% band, that measurement is the cheapest part of the project.

The bottom line in plain language. A Vera Rubin NVL72 is a data-center project, not a server purchase: 72 GPUs, 36 CPUs, direct liquid cooling, and a 187–227 kW envelope in one cabinet. At 227 kW continuous, the IT load alone is about 1.99 GWh a year, roughly $194,000 at the U.S. average industrial rate, before facility overhead, demand charges, cooling infrastructure, support, and financing. The trade is control of the data path and predictable high-volume local capacity, and it only pays if you can keep the system genuinely busy.

What this doesn't change

I am not going to pretend the rack solves the problem it appears to solve.

Utilization is the whole game. A rack at 25% utilization is an expensive space heater with good marketing. The economics only work against a sustained portfolio of compatible workloads — internal assistants, document workflows, contact-center QA, code models, domain agents — not one pilot.

You still need the software and the people. A rack needs a serving stack, an inference gateway, identity and key management, segmentation, observability, chargeback, capacity scheduling, patching, incident response, and vendor support. There is no authoritative public FTE-per-rack benchmark, and the BLS median wage for computer network support specialists — $76,220 — is a broad proxy for a job that is really platform engineering. [41]

The facility work comes before the purchase order — though NVIDIA is now attacking that problem directly. NVIDIA publishes a 45°C inlet design point and a 100%-liquid-cooling statement. It does not publish universal Rubin rack dimensions, loaded weight, floor loading, clearances, connector and input-voltage detail, cable plans, coolant flow, or pressure drop. [42] Do not substitute GB200 rack data for Rubin data. Get the selected OEM's installation and facility-engineering package before you approve a site.

What has changed is that the deployment path is becoming standardised rather than bespoke. NVIDIA contributed its Vera Rubin rack and compute tray designs to the Open Compute Project, and the resulting MGX 1.2 specification gives system builders a common target for blind-mate liquid cooling, rack-scale interconnect, and power interfaces. [45] Supermicro has already shipped against it: a 256-CPU, 100% liquid-cooled Vera CPU rack for agentic orchestration, tool calling and sandboxed execution, deployable alongside an NVL72 or on its own. [48] Alongside it, NVIDIA, AMD, Google and Meta published the Open Data Center Specification v0.7 for facility design, and Google, Meta, Microsoft and NVIDIA published a Grid Resilience Specification describing how large AI loads should behave during utility disturbances — staying connected and restoring to near-full load in under two seconds. For a regulated buyer that last document matters: it is the beginning of an answer to the utility-interconnection question that stalls more private deployments than the hardware does.

And the containerised DSX Pod shortens the physical build itself, if NVIDIA's six-month-to-14-day claim holds in practice. It packs seven Vera Rubin racks into a shipping container of roughly 28,000 kg with all cabling and liquid cooling assembled at the factory, designed to travel on normal roads, with deliveries expected from early 2027. [47] Treat the timeline as design intent until someone you trust has one running.

Benchmarks are workload engineering. A 36-GPU MLPerf result, a DeepSeek-R1 single-turn test, and a Kimi K2 projection are each useful and none of them yields a universal 72-GPU tokens-per-second figure. Measure your own prompt and output distributions, context lengths, concurrency, latency SLO, precision, model, retrieval and tool-call behavior, and power boundary on the stack you intend to run.

And cloud is still the right answer for most workloads. The convergence of local compute and frontier capability does not make owning hardware correct by default. It makes the question worth asking per workload, which is a different and better thing.

Where we come in

Deciding which AI runs in the cloud, which runs on your own metal, and how to design the boundary between them is the work we do at Grid Theory. That includes the parts that are easy to skip: the data-path map, the control and audit story your risk team will actually accept, the utilization plan that makes the hardware math close, and the governance that on-premises deployment does not give you for free.

If you are weighing a private deployment — a full rack, an NVL8-class node, or a hybrid split — book a discovery call and we will map it to your workloads.

One Rack, 72 GPUs: What NVIDIA's Vera Rubin Actually Does for Private AI