3,000 Tokens/Sec Locally: Testing the ZGX Fury

3,000 Tokens/Sec Locally: Testing the ZGX Fury

More

Summary

Sam Witteveen takes a look at the HP ZGX Fury, HP’s version of Nvidia’s DGX Station, and tests how it handles local AI workloads for a whole team. Where the DGX Spark and ZGX Nano sit on a desk for one user, the Fury is built to serve many people at once.

The machine pairs Nvidia’s GB300 Blackwell Ultra chip with a 72-core Grace CPU, delivering 20 petaflops of AI compute at FP4. It has 748 GB of memory shared between CPU and GPU, including 252 GB of high-bandwidth memory at 7.1 TB/s, compared with 1.8 TB/s on the RTX Pro 6000. It also has two 400 Gbit/s network ports, liquid cooling, and rack-mount support.

HP adds its own software, including the ZRT command-line tool, which downloads a model from Hugging Face, configures it for vLLM, and serves an endpoint with commands like zrt serve. The Fury is also the first GB300 station certified for Red Hat Enterprise Linux, and it ships with a quantized Nemotron 3 Super tuned by Infinity AI that is said to double concurrent users and throughput.

In testing with many simultaneous streams, the machine passes 3,000 tokens per second in aggregate. A RAG test with 16 concurrent users exceeds 1,300 tokens per second, and with 64 users each stream still runs at roughly 80 to 90 tokens per second. The video is sponsored by HP.


📺 Source: Sam Witteveen · Published October 30, 2026
🏷️ Format: Review

1 Item

Channels