Read Llama-Nemotron paper.

Llama-Nemotron: Efficient Reasoning Models

The world of large language models (LLMs) is constantly evolving, with models becoming increasingly capable of complex tasks. A significant recent advancement has been the introduction of “reasoning models,” which can engage in deep thinking processes like chain-of-thought, self-verification, and reflection to solve challenging problems. However, achieving state-of-the-art reasoning often comes with a cost: these models can be computationally expensive and slow to run during inference.

This paper has recently introduced the Llama-Nemotron series of models, an open family of heterogeneous reasoning models designed to deliver exceptional reasoning capabilities while prioritizing inference efficiency. This family includes three sizes: Nano (8B), Super (49B), and Ultra (253B). A key feature is their open license for enterprise use, contributing significantly to the open-source community.

Why Reasoning and Efficiency Together?

Reasoning capabilities are becoming increasingly crucial for model intelligence and for building complex systems like agentic pipelines. Models that can “think deeply” are necessary for tackling tasks like PhD-level STEM questions and competition-level math problems. However, inference efficiency is no longer just a deployment detail; it’s a core limiting factor for overall model performance and the viability of these advanced applications.

The Llama-Nemotron models directly address this by aiming to maximize inference efficiency as a primary optimization objective. They also introduce a dynamic reasoning toggle (using the system prompt “detailed thinking on/off”), allowing users to switch between standard chat and detailed reasoning modes during inference. This ensures that computational resources are used judiciously and that the response style is appropriate for the task, without needing separate models.

The Llama-Nemotron Family: Sizes and Capabilities

The series consists of three distinct models:

  • LN-Nano (8B): The smallest model in the family, demonstrating that structured reasoning can be effectively transferred to compact models.

  • LN-Super (49B): A mid-sized model based on Llama 3.3-70B-Instruct, optimized for efficient performance on a single NVIDIA H100 GPU. It performs competitively with other models in its weight class.

  • LN-Ultra (253B): The flagship model, based on Llama 3.1-405B-Instruct. It is optimized to run efficiently on a full 8xH100 node and achieves state-of-the-art performance among open models across a wide range of reasoning and non-reasoning benchmarks.

According to Artificial Analysis, an independent benchmarking company, LN-Ultra was the most “intelligent” open model as of April 2025.

Crafting Efficiency: The Puzzle Framework

A major innovation behind the Llama-Nemotron models’ efficiency is the use of the Puzzle framework. This Neural Architecture Search (NAS) framework transforms large language models into hardware-efficient variants under real-world deployment constraints.

Here’s how Puzzle works:

  • Crafting “Puzzle Pieces”: It uses block-wise local distillation to create a library of alternative transformer blocks for each layer of the base model. These alternative blocks approximate the original function but have improved computational properties (like lower latency, memory, or higher throughput). Block variants include Attention removal (reducing compute and KV-cache memory) and Variable FFN dimensions (enabling compression). These variants introduce an explicit trade-off between computational cost and model accuracy.

  • Assembling the Puzzle Architecture: A mixed-integer programming (MIP) solver is used to select one block per layer from the library. This assembly process optimizes for quality while meeting specific constraints like throughput, latency, or memory budget. This allows targeting precise points on the accuracy-efficiency Pareto frontier.

For LN-Ultra, an additional compression technique called FFN Fusion is applied after Puzzle removes some attention layers. This technique identifies consecutive FFN blocks and replaces them with fewer, wider layers that can be executed in parallel, reducing sequential depth and improving inference latency, especially on multi-GPU setups.

These optimizations translate into tangible gains. For example, LN-Super achieves a 5x throughput speedup over Llama 3.3-70B-Instruct on a single H100 GPU at a large batch size. LN-Ultra is optimized for an 8xH100 node, achieving a 1.71x latency improvement over Llama 3.1-405B-Instruct. During RL training, FP8 inference generation yielded a 1.8x generation speedup against BF16 for LN-Ultra.

The Multi-Stage Training Journey

The Llama-Nemotron models are built through a rigorous five-stage training process:

  • Inference Optimization: Neural Architecture Search (NAS) using the Puzzle framework and FFN Fusion are applied based on Llama 3 models.
  • Recovery Training: Knowledge distillation and continued pretraining are used to recover quality loss from the architectural changes and improve inter-block compatibility.
  • Supervised Fine-Tuning (SFT): The models are fine-tuned on a mix of standard instruction data and carefully curated reasoning traces. This stage is crucial for distilling reasoning behavior from powerful teacher models like DeepSeek-R1 and for establishing control over the “detailed thinking on/off” toggle. Training involves token-level cross-entropy loss and mixing reasoning and non-reasoning data.
  • Large-Scale Reinforcement Learning (RL): Applied particularly to LN-Ultra, this stage focuses on improving capabilities in complex mathematics and STEM. RL is essential for enabling the student model (LN-Ultra) to surpass the capabilities of its teacher (DeepSeek-R1). This involved using the GRPO algorithm, accuracy rewards, and format rewards for thinking tags. A curriculum training strategy was found to be helpful for stabilizing training and achieving higher accuracy by progressively increasing the difficulty of samples.
  • Alignment Phase: A final short phase focuses on instruction following and human preference using RL techniques like RLOO and iterative online RPO.

High-Quality Synthetic Data

A core component of the training process is the Llama-Nemotron-Post-Training-Dataset. This open-sourced, carefully curated dataset was used during the SFT and RL stages. It targets key capabilities such as mathematical reasoning, coding, science, and instruction following. Synthetic responses are generated by various open-source models and filtered for quality, correctness, and complexity. j For training the reasoning toggle, paired data is constructed where prompts have both a reasoning response (“detailed thinking on”) and a non-reasoning response (“detailed thinking off”).

Evaluation Results

The Llama-Nemotron models were evaluated across a range of reasoning and non-reasoning benchmarks, including AIME, GPQA-Diamond, LiveCodeBench, MATH500, IFEval, BFCL V2 Live, and Arena-Hard. Evaluations were performed at 32k context length to accommodate long reasoning traces.

  • LN-Ultra stands out, matching or outperforming all existing open-weight models on these benchmarks. It achieves state-of-the-art performance on GPQA-Diamond among open models, demonstrating the effectiveness of the large-scale RL training. LN-Ultra consistently outperforms DeepSeek-R1 and Llama-3.1-405B in both accuracy and efficiency on GPQA-Diamond.

  • LN-Super performs competitively in its weight class. In reasoning-on mode, it outperforms distilled models like DeepSeek-R1-Distilled-Llama-70B. In reasoning-off mode, it performs comparably to its base model, Llama-3.3-70B. It also achieved an impressive Arena Hard score, beating some proprietary models.

  • LN-Nano demonstrates strong performance on reasoning benchmarks despite its small size, highlighting the success of the SFT pipeline and curated data. Beyond reasoning and chat, the models were also evaluated on JudgeBench, an out-of-distribution task evaluating LLM-as-a-Judge capabilities. LN-Ultra emerged as the best open-source model on this benchmark, significantly surpassing DeepSeek-R1, and LN-Super also outperformed other models, showing strong generalization abilities.

Conclusion

The Llama-Nemotron series represents a significant contribution to the open-source AI landscape. By combining efficient architecture design through the Puzzle framework with a multi-stage training process involving supervised fine-tuning and large-scale reinforcement learning, this paper has created a family of models that are both highly capable in reasoning and efficient for inference. The release of the model weights, training data, and code under a permissive license further supports open research and development in reasoning models.

The paper underscores that while SFT is effective for transferring reasoning from a strong teacher, large-scale, curriculum-driven RL is essential to push capabilities beyond the teacher’s performance. It also highlights the need for multiple post-training stages to produce a well-rounded, all-around model.

The Llama-Nemotron models offer a powerful combination of performance, efficiency, and openness, making them a valuable resource for developers and researchers working on the next generation of AI applications.