Huawei's OceanStor M900 targets a 60-microsecond NPU-to-SSD path while serving as a PB-scale shared key-value cache for AI inference. The vendor positions the new platform as context memory storage spanning its Ascend compute ecosystem.

KV cache refers to the key-value tensors a transformer model generates during attention computation. Caching them avoids recomputation across tokens, but per-accelerator caching scales with model depth, sequence length, and user concurrency. A shared or distributed KV cache moves that state to a pool outside the accelerator, freeing on-board HBM at the cost of cross-tier latency.

What the 60-microsecond figure implies

A 60-μs round trip from NPU to SSD runs an order of magnitude tighter than typical networked NVMe paths. Those paths generally land between 200 μs and 1 ms, depending on protocol, network hops, and queue depth. Local HBM access completes in well under 1 μs.

The M900 places Huawei's OceanStor in an operating window closer to CXL-attached memory pools and certain in-memory fabrics, but with flash rather than DRAM as the backing medium at scale.

The launch material does not name the interconnect — NVMe-oF, RoCE, or a proprietary fabric. It leaves the SSD media type, controller topology, and wear-leveling strategy undisclosed.

KV-cache traffic differs from conventional block or object IO. Entries churn at token-generation rates, with bursts during prefill and steadier traffic during decode.

Why shared KV cache matters for inference economics

PB-scale shared cache has direct capex implications for inference operators. A 70B-parameter model serving 8K-token contexts with FP16 KV state occupies tens of gigabytes per concurrent user. A fleet handling tens of thousands of sessions at high cache hit rates can consume hundreds of terabytes.

Whether that footprint lives in per-accelerator HBM, pooled CXL memory, or shared flash determines capex per active session.

The M900 fits Huawei's broader vertical stack integrating its Ascend NPUs with custom storage and networking. The vendor has not confirmed external commercial availability dates, list pricing, or per-rack capacity figures.

What remains unverified

The "~60 μs" figure is a vendor claim from launch material. Independent benchmark data, third-party testing under mixed KV-cache workloads, and TCO comparisons with DRAM-based alternatives are not yet published.

KV-cache hit ratios depend on traffic locality, batching policy, and eviction strategy. None of these are standardized across vendors.

For operators evaluating inference-at-scale storage, the metric worth tracking is sustained latency under load. Peak numbers from vendor demos translate poorly to production. Whether the OceanStor M900 holds its 60-μs figure across a populated rack, sustained write pressure, and concurrent prefill/decode traffic will determine its position against CXL memory expanders and direct-attached flash topologies already shipping.