{"id":808278,"date":"2026-10-20T05:10:12","date_gmt":"2026-10-20T05:10:12","guid":{"rendered":"https:\/\/www.newsbeep.com\/uk\/808278\/"},"modified":"2026-10-20T05:10:12","modified_gmt":"2026-10-20T05:10:12","slug":"amd-ryzen-ai-halo-review-local-ai-powerhouse-with-128-gb-of-memory-local-llms","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/uk\/808278\/","title":{"rendered":"AMD Ryzen AI Halo Review – Local-AI Powerhouse with 128 GB of Memory – Local LLMs"},"content":{"rendered":"

\tLocal LLMs: Today’s ModelsSo what do you actually run on it? As mentioned before, open-source AI models come in a huge range of sizes. I’d group them into these four tiers:Small (1 to 8 billion parameters): Llama 3.1 8B, Qwen3-8B, Phi, Gemma. About 2-5 GB at 4-bit. Fast even on a phone or a cheap GPU, and fine for chat and summarizing, though not the smartest.Middle (14 to 32 billion parameters): Qwen3-32B, Gemma-27B. Clearly smarter, and still small enough (about 8-20 GB) to fit a single high-end GPU.Large (70-120 billion parameters): Llama 3.3 70B, gpt-oss-120b. 40-64 GB even at 4-bit, so a 32 GB card can’t hold them and this box starts to matter.Frontier (200 billion parameters and up): Qwen3-235B, DeepSeek-V3 (671B), Llama 405B. The biggest open models, close to the cloud services. At 85-400 GB they need a workstation, or a 128 GB box like the Ryzen AI Halo in this review can fit some smaller versions.Models also come in two types, and the difference decides how fast they run:Dense: uses all of its parameters for every word. A 70B dense model reads all 70 billion weights each time, which is fairly slow.Mixture-of-Experts (MoE): uses only a small part of itself per word. gpt-oss-120b uses about 5 of its 120 billion parameters per word, roughly 4%; Qwen3-235B about 22 of its 235, roughly 9%, so it reads far less memory per word than its size suggests.That is why a 235B MoE can run faster than a dense 70B, and why this box can run the huge models at a speed you can actually use. Here is how fast a few of today’s models run on the Halo, with the NVIDIA DGX Spark alongside:\"\"\/
\nThey all run, and they all run fast enough to use, and both boxes are fairly similar in their performance, with NVIDIA’s DGX Spark having a small advantage.<\/p>\n

One number the chart above doesn’t show is time to first token\u2014how long you wait before the first word appears. If you’ve ever used an AI chatbot, you’ve noticed that “thinking” step and probably wondered why this is taking so long.<\/p>\n

\"\"\/
\nWe tested this first step, called prefill, which is where the model reads through your prompt, and unlike reply generation it leans on raw compute rather than memory bandwidth. Compute is exactly where the Halo is weakest, so here the DGX Spark’s lead is larger than generation speed, where both are pretty much tied.<\/p>\n","protected":false},"excerpt":{"rendered":"Local LLMs: Today’s ModelsSo what do you actually run on it? As mentioned before, open-source AI models come…\n","protected":false},"author":2,"featured_media":808279,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[554,733,4308,86,56,54,55],"class_list":["post-808278","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-technology","tag-uk","tag-united-kingdom","tag-unitedkingdom"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/808278","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/comments?post=808278"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/808278\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media\/808279"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media?parent=808278"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/categories?post=808278"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/tags?post=808278"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}