{"id":349500,"date":"2026-01-03T06:50:22","date_gmt":"2026-01-03T06:50:22","guid":{"rendered":"https:\/\/www.newsbeep.com\/uk\/349500\/"},"modified":"2026-01-03T06:50:22","modified_gmt":"2026-01-03T06:50:22","slug":"building-our-office-storage-for-the-nvidia-gb10-agent-ai-cluster-page-2-of-4","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/uk\/349500\/","title":{"rendered":"Building Our Office Storage for the NVIDIA GB10 Agent AI Cluster – Page 2 of 4"},"content":{"rendered":"

Going Fast(er) with an All-Flash Local NAS for our AI Agent Machines<\/p>\n

As a quick recap, a big reason to use this particular QNAP TS-h1290FX NAS was that it supports high-end U.2 SSDs, like the Solidigm D5-P5336. It also has an onboard NVIDIA ConnectX-6 dual port 25GbE NIC. From a noise perspective, it is one of, if not the only, system in this class that has this level of performance, but is also quiet enough we can have it in a studio. For any organization that is running lots of small AI agent machines in an office setting, there is a huge benefit to not having a big and loud system.<\/p>\n

\"QNAP<\/a>QNAP TS H1290FX Rear Fan 1<\/p>\n

Inside, the system uses an AMD EPYC processor, but as an exciting point, it also has various PCIe Gen4 card slots.<\/p>\n

\"QNAP<\/a>QNAP TS H1290FX Inside 9<\/p>\n

We also reviewed the\u00a0QNAP QXG-25G2SF-CX6<\/a>. We are using an extra dual port 25GbE card with NVIDIA ConnectX-6 onboard to give us a total of 4x 25GbE ports, and that is our base configuration. 100GbE total is roughly a PCIe Gen5 x4 SSD worth of speed. Since we are using the 30.72TB Solidigm D5-P5336 SSDs which are PCIe Gen4 x4, it gives us roughly twice the interface bandwidth of the SSDs.<\/p>\n

\"QNAP<\/a>QNAP QXG 25G2SF CX6 Front Angled 1<\/p>\n

What is more, we found that we could push the cards to actually run full 25GbE speeds using our high-end Keysight CyPerf setup, even when running a mix of real-world application traffic, not just generic packets.<\/p>\n

\"QNAP<\/a>QNAP QXG 25G2SF CX6 Keysight Cyperf Performance 25G 512 Users Education<\/p>\n

Our original plan for this (and we should note it works) was to use a pair of low-power edge switches like the QNAP QSW-M7308R-4X<\/a>. These have four 100GbE ports and eight 25GbE ports.<\/p>\n

\"QNAP<\/a>QNAP QSW M7308R 4X Front 1<\/p>\n

Key to setting up performant AI cluster storage, even on smaller nodes, is to get RDMA and RoCEv2 working. For that, you need features like Priority Flow Control, or PFC.<\/p>\n

\"QNAP<\/a>QNAP QSW M7308R 4X QSS Pro System Port Management PFC<\/p>\n

ECN or Explicit Congestion Notification was another feature that this particular switch has, that we need for RoCE v2 networking.<\/p>\n

\"QNAP<\/a>QNAP QSW M7308R 4X QSS Pro QoS ECN<\/p>\n

This actually worked decently well. We had four 25GbE ports connected between the NAS and the switch. Another four of the QSFP28 ports for connecting the NVIDIA GB10 systems like the\u00a0Dell Pro Max with GB10<\/a> and\u00a0NVIDIA DGX Spark<\/a>. The GB10 networking is really funky as we went over in detail in The NVIDIA GB10 ConnectX-7 200GbE Networking is Really Different<\/a>. We could use a second switch for 100GbE RDMA networking between the GB10\u2019s for an East-West GPU communication network, and then the original switch for the more North-South storage network. There is a lot of wisdom to setting this as a single network, but this setup allowed us to also hook up four additional systems using 25GbE adapters to the storage network, and just use that as mostly a storage back-end network to hold models.<\/p>\n

\"NVIDIA<\/a>NVIDIA GB10 ConnectX 7 Ibdev2netdev<\/p>\n

To be clear here, we were very happy with this, $2000 of switches, a few DACs and low-cost optics, and we got great performance to access our 360TB pool of storage. Then we started to get the itch to do a bit more with our setup.<\/p>\n","protected":false},"excerpt":{"rendered":"Going Fast(er) with an All-Flash Local NAS for our AI Agent Machines As a quick recap, a big…\n","protected":false},"author":2,"featured_media":349501,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[554,10560,304,733,4308,134958,134959,3352,134960,134961,86,56,54,55],"class_list":{"0":"post-349500","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-amd","10":"tag-apple","11":"tag-artificial-intelligence","12":"tag-artificialintelligence","13":"tag-gb10","14":"tag-mikrotik","15":"tag-nvidia","16":"tag-qnap","17":"tag-solidigm","18":"tag-technology","19":"tag-uk","20":"tag-united-kingdom","21":"tag-unitedkingdom"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/349500","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/comments?post=349500"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/349500\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media\/349501"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media?parent=349500"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/categories?post=349500"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/tags?post=349500"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}