{"id":530125,"date":"2026-03-11T23:11:13","date_gmt":"2026-03-11T23:11:13","guid":{"rendered":"https:\/\/www.newsbeep.com\/ca\/530125\/"},"modified":"2026-03-11T23:11:13","modified_gmt":"2026-03-11T23:11:13","slug":"introducing-nemotron-3-super-an-open-hybrid-mamba-transformer-moe-for-agentic-reasoning","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/ca\/530125\/","title":{"rendered":"Introducing Nemotron 3 Super: An Open Hybrid Mamba-Transformer MoE for Agentic Reasoning"},"content":{"rendered":"

Agentic AI systems need models with the specialized depth to solve dense technical problems autonomously. They must excel at reasoning<\/a>, coding, and long-context analysis, while remaining efficient enough to run continuously at scale.\u00a0<\/p>\n

Multi-agent systems<\/a> generate up to 15x the tokens<\/a> of standard chats, re-sending history, tool outputs, and reasoning steps at every turn. Over long tasks, this \u201ccontext explosion\u201d causes goal drift, where agents gradually lose alignment with the original objective. And using massive reasoning models for every sub-task\u2014the \u201cthinking tax\u201d\u2014makes multi-agent applications too expensive and sluggish for practical use.<\/p>\n

Today, we are releasing Nemotron 3 Super<\/a> to address these limitations. The new Super model is a 120B total, 12B active-parameter model that delivers maximum compute efficiency and accuracy for complex multi-agent applications such as software development and cybersecurity triaging. This model follows the introduction of Nemotron 3 Nano<\/a> in December.<\/p>\n

Super addresses the \u201cthinking tax\u201d with its hybrid mixture-of-experts (MoE<\/a>) architecture. It delivers over 5x throughput than the previous Nemotron Super. This model tackles the \u201ccontext explosion\u201d with a native 1M-token context window that gives agents long-term memory for aligned, high-accuracy reasoning. The model is fully open with open weights, datasets, and recipes so developers can easily customize, optimize, and deploy it on their own infrastructure<\/a>.<\/p>\n

What makes Nemotron 3 Super different<\/a><\/p>\n

Nemotron 3 Super isn\u2019t just a bigger Nano. It introduces architectural innovations that allow the model to mitigate some of the typical efficiency-accuracy tradeoffs for high-capacity reasoning models:<\/p>\n

Latent MoE that calls 4x as many expert specialists for the same inference cost, by compressing tokens before they reach the experts.<\/p>\n

Multi-token prediction (MTP) that predicts multiple future tokens in one forward pass, dramatically reducing generation time for long sequences and enabling built-in speculative decoding.<\/p>\n

Hybrid Mamba-Transformer backbone integrating Mamba layers for sequence efficiency with Transformer layers for precision reasoning, delivering higher throughput with 4x improved memory and compute efficiency.<\/p>\n

Native NVFP4 pretraining optimized for NVIDIA Blackwell, significantly cutting memory requirements and speeding up inference by 4x on NVIDIA B200 compared to FP8 on NVIDIA H100, while maintaining accuracy.<\/p>\n

Multi-environment reinforcement-learning (RL) post-trained with RL<\/a> across 21 environment configurations using NVIDIA NeMo Gym<\/a> and NVIDIA NeMo RL<\/a>, trained with more than 1.2 million environment rollouts.<\/p>\n

These advantages come together to create a model that is well suited for long-running autonomous agents. On PinchBench<\/a>\u2014a new benchmark for determining how well LLM models perform as the brain of an OpenClaw agent\u2014Nemotron 3 Super scores 85.6% across the full test suite, making it the best open model in its class.<\/p>\n

See it in action<\/a><\/p>\n

If you want to go hands on with Nemotron 3 Super, follow the tutorial video below. This will walk you through how to use the model from build.nvidia.com<\/a> to OpenCode.<\/p>\n

Video 1. A tutorial walkthrough of Nemotron 3 Super<\/p>\n

Diving deep into the architecture<\/a><\/p>\n

Hybrid Mamba-Transformer MoE backbone<\/a><\/p>\n

Super builds on the same hybrid philosophy as Nano but at a fundamentally different scale. The backbone interleaves three layer types:<\/p>\n

Mamba-2 layers handle the majority of sequence processing. State space models (SSMs) provide linear-time complexity with respect to sequence length, which is what makes the 1M-token context window practical rather than theoretical. When an agent needs to reason over an entire codebase, a long conversation history, or a stack of retrieved documents, Mamba layers keep the memory footprint manageable.<\/p>\n

Transformer attention layers are interleaved at key depths. Pure SSMs can struggle with precise associative recall\u2014the kind of task where you need to find one specific fact buried in a long context. The attention layers preserve this capability, ensuring that Super maintains high-fidelity retrieval even when the \u201cneedle\u201d sits in the middle of a haystack of conflicting information.<\/p>\n

MoE layers scale effective parameter count without the cost of dense computation. Only a subset of experts activates per token, keeping latency low and throughput high\u2014critical when many agents are running concurrently in a shared deployment.<\/p>\n

\"ArchitectureFigure 1. A layer pattern diagram showing repeating blocks of Mamba-2\/MoE pairs interleaved with attention layers<\/p>\n

Latent MoE<\/a><\/p>\n

Standard MoE architectures route tokens directly from the model\u2019s full hidden dimension to the experts. As models grow, this routing layer becomes a bottleneck\u2014it increases compute costs and limits how many experts you can practically deploy.<\/p>\n

Super introduces latent MoE: Before routing decisions are made, token embeddings are projected into a compressed, low-rank latent space. Expert computation happens in this smaller dimension, and results are projected back to the full model dimension afterward.<\/p>\n

Why this matters in practice:<\/p>\n

More experts, same cost. By compressing tokens before they reach the experts, latent MoE enables the model to consult 4x as many experts for the exact same computational cost as running one.<\/p>\n

Finer-grained specialization. With more experts available, the model can afford highly specialized routing\u2014for example, activating distinct experts for Python syntax versus SQL logic\u2014that are only activated when strictly necessary. This granularity is especially valuable in agentic settings where a single conversation may span tool calls, code generation, data analysis, and conversational reasoning within a few turns.<\/p>\n

\"AFigure 2. Side-by-side comparison of standard MoE vs. latent MoE architectures<\/p>\n

Multi-token prediction (MTP)<\/a><\/p>\n

Standard language models<\/a> are trained to predict one token at a time\u2014a fundamentally myopic objective. Super is trained with MTP, where specialized prediction heads forecast several future tokens simultaneously from each position.<\/p>\n

This has two concrete benefits:<\/p>\n

Stronger reasoning during training. Predicting multiple future tokens forces the model to internalize longer-range structure and logical dependencies. Rather than learning to guess plausible next words, the model must learn to anticipate coherent sequences. This produces measurable gains on chain-of-thought tasks where each step must follow logically from the last.<\/p>\n

Built-in speculative decoding at inference. By predicting multiple future tokens simultaneously in one forward pass, MTP dramatically reduces the time required to generate long sequences. The MTP heads provide draft predictions that can be verified in parallel, enabling up to 3x wall-clock speedups for structured generation tasks like code and tool calls\u2014without requiring a separate draft model.<\/p>\n

Both benefits stem from the same design decision. Unlike architectures that train independent prediction heads per offset, Super uses a shared-weight design across all MTP heads. This keeps the parameter overhead minimal while improving training stability\u2014the heads learn to agree on coherent continuations rather than diverging into offset-specific shortcuts. The same weight sharing also makes the speculative drafts more consistent at longer draft lengths, which is where independently trained heads typically degrade.<\/p>\n

Native NVFP4 pretraining<\/a><\/p>\n

Most quantized models start as full-precision and get compressed after training, which inevitably introduces accuracy loss. Super takes a different approach: The majority of floating-point multiply-accumulate operations during pretraining run in NVFP4,<\/a> the NVIDIA 4-bit floating-point format. Optimized for Blackwell, this significantly cuts memory requirements and speeds up inference compared to FP8, while maintaining accuracy.<\/p>\n

Training natively in reduced precision means the model learns to be accurate within the constraints of 4-bit arithmetic from the very first gradient update. The result is a model that is mathematically stable and accurate despite running on a significantly reduced memory footprint.<\/p>\n

How we trained Nemotron 3 Super<\/a><\/p>\n

Nemotron 3 Super is trained in three sequential phases, each building on the last. Pretraining<\/a> establishes broad world knowledge and language understanding at scale. Supervised fine-tuning shapes the model\u2019s behavior across the task types it will encounter in deployment. Reinforcement learning then refines that behavior against verifiable outcomes across diverse agentic environments.<\/p>\n

Pretraining<\/a><\/p>\n

Super is pretrained on 25 trillion tokens using NVFP4, the NVIDIA 4-bit floating-point format optimized for NVIDIA Blackwell. Rather than quantizing a full-precision model after the fact, Super trains natively in reduced precision from the first gradient update\u2014meaning the model learns to be accurate within the constraints of 4-bit arithmetic throughout pretraining, not just at inference. The pretraining corpus spans 10 trillion unique curated tokens, with the model seeing 25 trillion total tokens across the run, including additional compute focused on reasoning and coding.<\/p>\n

Supervised fine-tuning<\/a><\/p>\n

Before reinforcement learning, Super undergoes supervised fine-tuning on about 7 million SFT samples. They\u2019re drawn from a broader post-training corpus of 40 million samples, which cover reasoning, instruction following, coding, safety, and multi-step agent tasks. This stage establishes the behavioral foundation that RL then refines. The model learns the format and structure of correct responses across task types, giving the subsequent RL phase a stable starting point rather than optimizing from a raw pretrained checkpoint.<\/p>\n

Multi-environment reinforcement learning<\/a><\/p>\n

To align Super with real agentic behavior, the model is post-trained using reinforcement learning across diverse environments in NeMo Gym,<\/a> the NVIDIA open source library for building and scaling RL training environments. These environments evaluate the model\u2019s ability to perform sequences of actions\u2014generating correct tool calls, writing functional code, producing multi-part plans that satisfy verifiable criteria\u2014not just providing satisfying single-turn responses. These trajectories form the core training data to run reinforcement learning at scale with the NeMo RL<\/a> open library.<\/p>\n

This trajectory-based reinforcement produces a model that behaves reliably under multi-step workflows, reduces reasoning drift, and handles the kinds of structured operations common in agentic pipelines.<\/p>\n

Benchmarking Nemotron 3 Super<\/a><\/p>\n

Nemotron 3 Super achieves leading accuracy across a number of important agentic benchmarks while maintaining incredible throughput.<\/p>\n

\"AFigure 3. A chart comparing Nemotron 3 Super accuracy on key benchmarks against similarly sized open models.<\/p>\n

The \u201cSuper + Nano\u201d deployment pattern<\/a><\/p>\n

Nemotron 3 Nano is an excellent choice for achieving high accuracy in executing targeted, individual steps within an agentic workflow. However, when multi-agent applications escalate to complex, multi-step activities, they require a high-capacity model for superior planning and reasoning.\u00a0 Think of a computer use agent that needs to make decisions between different modalities of tools in order to, say, create a presentation with 10 high-quality slides.<\/p>\n

Nemotron 3 Super is ideal in this use. For instance, in software development, simple merge requests can be addressed by Nemotron 3 Nano while complex coding tasks that require deeper understanding of the code base can be handled by Nemotron 3 Super. And expert-level coding tasks can be addressed by proprietary models.<\/p>\n

Building with Super\u2019s open resources<\/a><\/p>\n

Nemotron 3 Super<\/a> is fully open\u2014weights, datasets, and recipes\u2014so developers can easily customize, optimize, and deploy the model on their own infrastructure for maximum privacy and security.<\/p>\n

Model weights<\/a><\/p>\n

Full parameter checkpoints for Nemotron 3 Super are available on Hugging Face<\/a> and through NVIDIA NIM<\/a>. The NVIDIA Nemotron Open Model License<\/a> gives enterprises the flexibility to maintain data control and deploy anywhere.<\/p>\n

End-to-end training and evaluation recipes<\/a><\/p>\n

We are releasing the complete training and evaluation recipe for Nemotron 3 Super, covering the full pipeline from pretraining through alignment. This enables developers to reproduce Super\u2019s training, adapt the recipe for domain-specific variants, or use it as a starting point for their own hybrid architecture research.<\/p>\n

Deployment cookbooks<\/a><\/p>\n

We\u2019ve built ready-to-use cookbooks<\/a> for major inference engines, each with configuration templates, performance tuning guidance, and reference scripts:<\/p>\n

vLLM Cookbook<\/a>: High-throughput continuous batching and streaming for Super.<\/p>\n

SGLang Cookbook<\/a>: Fast, lightweight inference optimized for multi-agent tool-calling workloads.<\/p>\n

NVIDIA TensorRT LLM Cookbook<\/a>: Fully optimized TensorRT LLM engines with latent MoE kernels for production-grade, low-latency deployment.<\/p>\n

Fine-tuning cookbooks<\/a><\/p>\n

Explore our Nemotron 3 Super customization cookbooks to efficiently fine-tune for your domain (LoRA\/SFT) or advance its agentic reasoning capabilities (GRPO\/DAPO):<\/p>\n

Open datasets<\/a><\/p>\n

Nemotron 3 Super is built on a fully open, end-to-end data pipeline that spans pretraining, post-training, and interactive reinforcement learning\u2014giving developers reproducible building blocks for agentic AI.<\/p>\n

Pretraining corpora<\/a>: 10 trillion curated tokens, trained over 25 trillion total seen tokens, plus an additional 10 billion tokens focused on reasoning and 15 million coding problems. All aggressively deduplicated and quality-filtered to maximize signal-to-noise.<\/p>\n

Post-training datasets<\/a>: 40 million new supervised and alignment samples, covering reasoning, instruction following, coding, safety, and multi-step agent tasks across supervised fine-tuning, preference data, and RL trajectories (about 7 million used directly for SFT)<\/p>\n

RL tasks and environments<\/a>: Interactive RL across 21 environment configurations and 37 datasets (~10 of which are being released) including software engineer-style agent training and tool-augmented search\/planning tasks\u2014moving beyond static text into dynamic, verifiable execution workflows and generating ~1.2 million environment rollouts during training.<\/p>\n

Open training and evaluation infrastructure<\/a><\/p>\n

NVIDIA publishes development techniques and tools, giving researchers and enterprises the flexibility to customize Nemotron 3 Super or build their own reasoning models. All recipes integrate with the Nemotron GitHub repository, NeMo Gym<\/a>, NeMo RL<\/a>, NVIDIA NeMo Data Designer<\/a>, NVIDIA NeMo Curator<\/a>, and NVIDIA NeMo Evaluator<\/a>\u2014providing a complete, reproducible pipeline from data to deployment.<\/p>\n

All Nemotron models are released with an open evaluation approach, including a published evaluation recipe<\/a> that enables anyone to rerun and inspect the full evaluation pipeline from Nemotron 3 Super.<\/p>\n

Get started<\/a><\/p>\n

Nemotron 3 Super is live now. Available across leading inference platforms and packaged as NVIDIA NIM, Super can run anywhere from the workstation to the cloud. Try it on Perplexity<\/a> with a Pro subscription or through API, OpenRouter<\/a>, or build.nvidia.com<\/a>.<\/p>\n

Download the weights from Hugging Face<\/a>, launch an optimized instance through NVIDIA NIM, fine-tune with Unsloth<\/a>, or start with the cookbooks to get running in minutes.<\/p>\n

Super is also available through Baseten<\/a>, Cloudflare<\/a>, CoreWeave<\/a>, DeepInfra<\/a>, Fireworks AI,<\/a> FriendliAI, <\/a>Inference.net<\/a>, Lightning AI<\/a>, Modal,<\/a> Nebius<\/a>, and Together AI<\/a>.<\/p>\n

Check out our GitHub repository<\/a> which has getting started instructions for platforms like OpenCode, OpenHands, and OpenClaw.<\/p>\n

For the full technical details, read the Nemotron 3 Super technical report<\/a>.<\/p>\n

Stay up-to-date on NVIDIA Nemotron<\/a> by subscribing to NVIDIA news<\/a> and following NVIDIA AI on LinkedIn<\/a>, X<\/a>, Discord<\/a>, and YouTube<\/a>. Visit the Nemotron developer page<\/a> for resources to get started. Explore open Nemotron models and datasets on Hugging Face<\/a> and Blueprints<\/a> on build.nvidia.com<\/a>. And engage with Nemotron livestreams<\/a>, tutorials<\/a>, and the developer community on the NVIDIA forum<\/a> and Discord<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"Agentic AI systems need models with the specialized depth to solve dense technical problems autonomously. They must excel…\n","protected":false},"author":2,"featured_media":530126,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[62,276,277,49,48,61],"class_list":{"0":"post-530125","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-ca","12":"tag-canada","13":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts\/530125","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/comments?post=530125"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts\/530125\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/media\/530126"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/media?parent=530125"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/categories?post=530125"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/tags?post=530125"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}