SoTA Feed — Every open-weights release from the labs that matter

Ad: Read SoTA Feed without this slot — ad-free site plus a personal ad-free feed URL $3/month

Qwen3.8-Flash-Next-NVFP4-QAD

Nov 17, 2026 · NVIDIA · license: other · view on Hugging Face ↗
130 GB · MoE: 180B total, ≈62B (≈44 GB) active · NVFP4

Qwen3.8-Flash-Next-NVFP4-QAD

This is an experimental mixed-precision QAD checkpoint of Qwen3.8-Flash-Next, exported after 100 optimizer updates using NVIDIA Model Optimizer and NeMo AutoModel with FSDP2. It contains the full multimodal model, including the frozen vision encoder, PLE n-gram embedding, and MTP module. Weight files occupy approximately 129.7 GB (120.8 GiB).

**This checkpoint uses a more aggressive quantization recipe than Qwen3.8-Flash-Next-NVFP4.** In addition to routed experts, it quantizes shared experts to NVFP4 and selected GDN, QSA, and indexer projections to MXFP8. The repository name's -QAD suffix identifies the training method; it does not mean the original NVFP4 recipe was kept unchanged. Task accuracy gains from QAD are not statistically established.

Weight shards were consolidated for distribution. Every source file was verified against its export SHA256, and consolidated files passed a complete readback. Tensor names, shapes, dtypes, and stored values are unchanged.

Quantization recipe

ComponentStored precisionQAD treatment
Main LM routed experts, 48 MoE groupsNVFP4, block size 16Trained
Main LM shared experts, 144 linear matricesNVFP4, block size 16Trained
GDN projections, 180 linear matricesMXFP8, block size 32Trained
QSA q/k/v/o projections, 48 linear matricesMXFP8, block size 32Trained
QSA indexer projections, 12 linear matricesMXFP8, block size 32Frozen
PLE n-gram embeddingPer-tensor FP8 E4M3, BF16 scaleFrozen
MTP routed expertsFP8 E4M3, 128 x 128 block scalesFrozen
MTP shared experts and attentionBF16Frozen
Vision encoder, routers, gates, convolutions, output head, other excluded tensorsOriginal precisionFrozen

The main model uses native NVFP4 and MXFP8 computation in the validated serving configuration. Selected excluded operations retain their original FP32 contracts. The KV cache was BF16 during evaluation; KV-cache quantization is not part of this recipe. See hf_quant_config.json, config.json, and quantization_recipe.json for machine-readable metadata.

The main LM was quantized from the original BF16 model. The preserved MTP and PLE state derives from the released NVIDIA mixed-precision checkpoint. QAD did not further quantize or train MTP. All benchmark measurements below disabled MTP.

QAD training

The student used fake-quantized computation with floating master weights and a frozen BF16 teacher. Training used text inputs; the vision encoder stayed frozen.

SettingValue
Completed optimizer updates100
Maximum training sequence length16,384 tokens
Global batch size512 sequences
Consumed sequence examples51,200, including any repeated samples
Microbatch size1 sequence per rank
FSDP2 world size64
OptimizerAdam, betas (0.9, 0.95), epsilon 1e-8, weight decay 0
Peak / minimum learning rate1e-5 / 1e-6
Warmup / scheduler horizon50 / 500 updates; this export stops at update 100

16,384 is a sequence-length ceiling. It does not mean every example contained 16,384 non-padding tokens. The exact training data files are not distributed in this repository. This is an inference checkpoint; optimizer state is not included.

Export validation compared native training-reference and exported-model log probabilities at input lengths 256, 4,096, 8,192, and 16,384. Across 28,924 scored positions, mean absolute log-probability error was 1.21e-6, maximum error was 1.00e-5, and top-1 agreement was 99.25%. This establishes export parity within the tested limits, not parity with BF16 model accuracy.

Experimental evaluation

These are internal experimental measurements on a 0–100 scale. They are not the scores from the original NVIDIA model card, whose evaluation protocols differ. The PTQ column is the corresponding aggressive recipe before QAD.

Validation scope: checkpoint identity, native quantization kernels, score coverage, and task-specific grading were checked. Historical full-run request retry, timeout, and output-termination accounting is incomplete, particularly for Terminal-Bench. The numbers are descriptive; unresolved limit effects prevent a clean attribution of differences to quantization or training.

Benchmark / metricAggressive PTQQAD update 100
GPQA-Diamond accuracy91.3591.60
IFBench instruction-following score79.6079.93
SciCode subtask success49.4851.11
AA-LCR accuracy77.6377.13
MMMU-Pro accuracy76.6577.34
Terminal-Bench 2.1 native pass@1, qualified85.3985.96

SciCode uses eight operational repeats for QAD and the **second eight-repeat PTQ cohort**. The first PTQ cohort scored 52.00, and pooling all 16 PTQ repeats gives 50.74. Repeats used the same configured request seed and are not independent seed trials. Against the second cohort, the QAD difference is +1.63 percentage points with a conditional whole-problem bootstrap 95% interval of [-0.66, +3.97]. This cohort variation precludes a claim of reliable recovery.

Terminal-Bench retains all 712 outcomes per checkpoint. Protocol-defined runtime failures were retained as zero scores: 13/712 for PTQ and 4/712 for QAD. The task-cluster bootstrap 95% interval for QAD minus PTQ is [-1.97, +3.09] percentage points. Historical allocation-wide timeout/retry and output-cap rates remain unknown; later two-task logging diagnostics do not retroactively fill those gaps.

No statistically established task-accuracy improvement over PTQ is claimed. SciCode subtask success must not be compared with published main-problem scores.

Serving

Validated hardware was NVIDIA GB300, using native Blackwell kernels, vLLM 0.30.0, four-way tensor parallelism, and expert parallelism. This recipe additionally needs the compatibility helpers in serving/: narrow MXFP8 output padding/cropping and the canonical sparse-indexer policy used for training/export validation. The helpers check the ModelOpt runtime source hash and refuse an unvalidated build.

Follow serving/README.md for the tested settings and compatibility requirements. Generic transformers.from_pretrained() loading and arbitrary vLLM versions have not been validated for this mixed format. The model configuration preserves the base model's context settings; QAD export parity was checked through 16K, and task-serving configurations used up to 262,144. This release does not establish accuracy at the base model's maximum context.

MTP is preserved for completeness. MTP-enabled accuracy, acceptance rates, and performance were not validated for this QAD checkpoint. With standard exact speculative rejection sampling, draft quantization primarily changes acceptance and speed; approximate verification can change the output distribution.

Architecture and intended use

The upstream model combines a vision encoder, Gated DeltaNet, Qwen Sparse Attention, MoE layers, gated residual streams, and n-gram embeddings. Inputs may contain text, images, or video, and outputs are text. Refer to the base model card for its architecture, training information, intended uses, and limitations. This release is intended for quantization and QAD research and deployment validation.

License, attribution, and limitations

Use is governed by the NVIDIA Open Model License and the upstream Qwen Community License 1.0. Qwen is the upstream model developer. This checkpoint does not replace or waive upstream terms. The serving helper code carries the separate license in serving/LICENSE.

The model can generate incorrect, biased, toxic, or otherwise unsuitable content. Evaluate it on the intended application, including multimodal inputs and long contexts, before deployment. Quantization and short QAD training do not remove the base model's limitations.

References: NVIDIA Model Optimizer, NeMo AutoModel, released NVFP4 model.

← all releases