SoTA Feed — Every open-weights release from the labs that matter

Ad: Read SoTA Feed without this slot — ad-free site plus a personal ad-free feed URL $3/month

Nemotron-Labs-Sigma-distill-3B-nfe2

Nov 9, 2026 · NVIDIA · license: other · view on Hugging Face ↗
8.5 GB · 4.2B dense

Nemotron-Labs-Sigma-distill-3B-nfe2

Model Overview

Description

Nemotron-Labs-Sigma-distill-3B-nfe2 is a 3 billion-parameter continuous diffusion language model distilled for fast few-step generation. It was produced with Phased Distribution Matching Distillation (Phased DMD) from a multiscale SUSReg-regularized 3B teacher of the same Nemotron-Labs-Sigma architecture family (MS-SUSReg-3B). This is the same MS-SUSReg lineage as the released Nemotron-Labs-Sigma-3B base model.

The student generates a 32-token block in exactly **2 denoising function evaluations (NFE)** plus one final readout, using the native fresh_x0 sampler with self-conditioning carry. The number of denoising steps is fixed by training: this variant is trained for NFE=2 only, and the bundled inference code intentionally does not expose an arbitrary step override (other step counts would evaluate untrained phase endpoints). Sibling variants are published for NFE = 2, 4, 8 and 16 per model size.

This model is ready for commercial or non-commercial use.

License/Terms of Use

Use of this model is governed by the NVIDIA Nemotron Open Model License.

Deployment Geography

Global

Use Case

Researchers and developers who need fast, few-step text generation from diffusion language models: latency-sensitive generation research, speculative and parallel decoding studies, and deployment-oriented experimentation. The model runs offline on the user's own hardware; NVIDIA hosts no service.

Release Date

References

Model Architecture

Input

Output

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

Software Integration

Run the model on a CUDA-capable NVIDIA GPU in bfloat16. CPU inference is not supported: the diffusion sampler is numerically degraded on CPU and produces unusable output.

The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.

Model Version(s)

Environment

torch>=2.4
transformers>=4.55

A CUDA-capable NVIDIA GPU is required (bfloat16); CPU inference is not supported.

Generate with This Model

import torch
from transformers import AutoModel, AutoTokenizer

repo = "nvidia/Nemotron-Labs-Sigma-distill-3B-nfe2"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModel.from_pretrained(repo, trust_remote_code=True)
model = model.cuda().to(torch.bfloat16).eval()

prompt = "def fibonacci(n):"
input_ids = tokenizer(prompt, return_tensors="pt").input_ids.cuda()

# steps is fixed to the native NFE (2) by the checkpoint; do not override.
out_ids, nfe = model.generate(
    input_ids,
    max_new_tokens=256,
    stop_strings=["\nclass", "\ndef"],  # optional block-level stopping
    tokenizer=tokenizer,
)
print(tokenizer.decode(out_ids[0, input_ids.shape[1]:], skip_special_tokens=True))
print(f"[Number of Function Evaluations (NFE)={nfe}]")

Evaluation Results

Accuracy in percent, reported as **mean +/- 2 standard errors over 10 generation seeds** (42-51) at the model's native NFE, CFG 1, score temperature 1, 32-token blocks. Source: the release handoff's verified multi-seed statistics.

BenchmarkScore (%)
GSM8K (CoT)74.42 +/- 0.46
MATH (Minerva)49.71 +/- 0.30
HumanEval32.38 +/- 1.34
HumanEval+30.55 +/- 0.86
MBPP20.78 +/- 1.00
MBPP+30.87 +/- 0.79
Six-task mean39.78 +/- 0.57

Ethical Considerations

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

For more detailed information on ethical considerations for this model, please see the Model Card++ Bias, Explainability, Safety & Security, and Privacy Subcards.

Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns here.

Citations

@article{nemotronlabssigma2026,
  title       = {Large Language Continuous Diffusion Models},
  author      = {Zhihan Yang and Wei Guo and Jean-Marie Lemercier and Simon Welker and
                 Mohammad Mahdi Kamani and Yonggan Fu and Sajad Norouzi and Julius Berner and
                 Tomas Geffner and Karsten Kreis and Yongxin Chen and Molei Tao and
                 John Thickstun and Pavlo Molchanov and Ante Juki{\'c} and Arash Vahdat and
                 Morteza Mardani},
  journal     = {arXiv preprint arXiv:2610.02665},
  year        = {2026},
  url         = {https://arxiv.org/abs/2610.02665}
}

← all releases