Microsoft has unveiled the Maia 300, an in-house AI accelerator carrying 288 GB of HBM4 memory and 18 TB/s of memory bandwidth.

The chip extends Microsoft's Maia line of custom accelerators deployed alongside Nvidia and AMD GPUs in Azure data centers. With 288 GB of HBM4, the Maia 300 sits at the top of the memory-capacity tier for accelerators shipping in 2025-2026, where most high-end competitors carry between 192 GB and 288 GB. The 18 TB/s figure exceeds the published per-device bandwidth of currently shipping accelerators — Nvidia's B200 at 8 TB/s and AMD's MI300X at 5.3 TB/s — by 2-3x.

What does HBM4 bring to the Maia platform?

HBM4 is the fourth generation of the high-bandwidth memory standard maintained by JEDEC. The transition from HBM3e raises per-stack capacity from 24 GB to 32-36 GB and lifts per-pin data rates from 6.4-9.6 Gbps to 11-16 Gbps. Aggregate package bandwidth typically climbs 50-70% over the previous generation.

Microsoft has not published the stack count or per-pin transfer rate of the Maia 300's HBM4 subsystem. Eight stacks of 36 GB would reach 288 GB; alternative splits such as twelve stacks of 24 GB or nine stacks of 32 GB are also plausible.

Eight-high stacks at 32 Gbps per pin over a 1024-bit interface would yield approximately 16-18 TB/s aggregate, consistent with the disclosed figure.

Memory bandwidth frequently bottlenecks large model training and inference before raw FLOPs do. The transformer attention mechanism scales memory traffic with sequence length and parameter count, and a chip moving 18 TB/s can keep larger working sets resident in memory without recomputation.

For training runs already saturating HBM3e at 8 TB/s, a jump to 18 TB/s cuts step time proportionally; for inference fleets, it lifts the number of concurrent sessions each device handles.

How does 288 GB reshape model residency?

The 288 GB figure determines which models run on a single device without spilling to CPU memory or NVMe storage.

A dense LLM in FP16 occupies roughly 2 bytes per parameter; with KV-cache and activation headroom, a 288 GB device can host models on the order of 100-130 billion parameters fully in HBM.

Mixture-of-experts configurations stretch that further, since only active experts must be resident at any moment.

Lower-precision weight formats — FP8, FP4, and emerging NVFP4 — would raise resident model size by 2-4x.

Microsoft has not confirmed which formats the Maia 300 supports natively, nor whether it ships tensor-core paths optimized for FP4 inference.

Why is Microsoft building its own silicon?

Azure's hyperscale economics depend on the cost and availability of accelerators.

By designing its own chips, Microsoft aims to reduce per-token inference cost, hedge against allocation cycles on third-party GPUs, and tune silicon for its specific software stack, including the ONNX runtime and Triton compiler paths.

Cloud margins, not peak training throughput alone, will determine whether Microsoft scales Maia deployments further. Custom silicon must clear an internal return-on-investment threshold measured against buying equivalent GPUs at market price.

The first Maia generation appeared in Azure in 2023 as a 105-billion-transistor test vehicle; the Maia 300 represents the second commercial step.

What does this mean for the broader accelerator market?

Nvidia still controls the bulk of the AI accelerator market. AMD, Intel, and the hyperscalers — Google with TPU, Amazon with Trainium, Microsoft with Maia — compete for the remainder.

Each generation of HBM tightens the performance frontier; HBM4-class devices shipping this year and next set a new bandwidth ceiling that compilers and kernels will need to target.

Microsoft's HBM4 supplier — among SK Hynix, Samsung, and Micron, all of which have announced HBM4 sampling — will influence unit cost. Each HBM4 die carries 24-36 GB of capacity; the eight to twelve stacks per package that the 288 GB figure implies drive significant share-of-bill on the finished accelerator.

If Microsoft prices Maia 300 aggressively and integrates it tightly with Azure networking and orchestration, the chip could capture internal workloads now running on Nvidia hardware. External Azure customers are unlikely to access Maia silicon directly; Microsoft has historically restricted custom accelerators to first-party services such as Copilot and Bing ranking.

Volume timing, foundry partner, and per-rack power draw remain undisclosed. Microsoft has not set a public ship date for Maia 300.