NVIDIA
November 13, 2026 · Read the original
The post describes serving GLM-5.3 NVFP4, a quantized mixture-of-experts model with 753 billion parameters and 40 billion activated per token, across two NVIDIA DGX Stations connected over ConnectX-8 RDMA. The checkpoint requires about 464 GB of GPU memory, exceeding the 252 GB of HBM3e on a single GB300 GPU, so tensor parallelism (TP=2) splits the model across the two GPUs and Station A returns answers through one OpenAI-compatible API. vLLM launches the distributed service with multi-token prediction, FP8 KV-cache storage, and prefix caching. AIPerf benchmarks on the SPEED-Bench Coding dataset show output throughput peaking at 607.8 tokens per second at concurrency 16 and falling to 385.0 tokens per second at concurrency 32, with mean request latency of 453.5 seconds.
Written from the article itself at capture time and checked against it before publication: every number and every name in it comes from the source. It is our summary, not the lab’s wording — the original is linked above.
The announcement says what the lab claims. The record says what was measured, by whom, and whether anyone reproduced it. Read both before believing either.