NVIDIA publishes Muse Spark Open deployment paths—and the throughput claim needs context
The GB300-focused release pairs high headline throughput with narrower documented deployment results.
In this article
What the speed claim actually describes
NVIDIA’s launch post claims up to 50,000 tokens per second per GPU and 600 per user for Muse Spark Open on GB300 NVL72. The figure caption identifies four Blackwell GPUs with speculative decoding, rather than the whole rack. These vendor figures have different denominators: aggregate throughput per GPU and generation speed per user. Neither alone establishes how long an application request will take.
The rack provides the infrastructure
The GB300 NVL72 specification lists 72 Blackwell Ultra GPUs, 36 Grace CPUs, 20 TB of GPU memory and 130 TB/s of NVLink bandwidth in a liquid-cooled rack. These specifications describe the platform’s capacity, not the number of GPUs used by every deployment. For a team considering its own installation, the practical question is therefore which subset and serving setup a published result uses, rather than whether the model merely supports this hardware family.
The documented recipe is a separate benchmark
Dynamo’s development documentation reports a four-GPU GB300 SGLang run at concurrency 16: 635.8 output tokens per second in total, or 158.95 per GPU. Its agentic trace has approximately 64,000 input tokens, 400 output tokens and 90% cache reuse. Experimental vLLM and TensorRT-LLM results cover only the first 500 requests and establish functional operation; they are not comparable full-trace runs. These documented results are not a reproduction of the launch post’s headline. Keep workload, cache state, concurrency and output-token denominator attached when comparing measurements.
