NVIDIA, United States · Released October 15, 2026Nemotron 3.5 Super VL 120B-A12B at a glance
- Made by
- NVIDIA, United States
- Released
- October 15, 2026
- Parameters
- 124.5B, 12B active
- Context window
- 1M tokens
Nemotron 3.5 Super VL 120B-A12B VRAM Requirements
NVIDIA, United States · Released October 15, 2026New
Nemotron 3.5 Super VL 120B-A12B has 124.5B parameters (12B active per token). In BF16 its weights alone take 226 GiB; quantized to 4-bit, about 75.6 GiB. Each 32K-token request adds 0.25 GiB of KV cache. Cheapest way to run it today: 4 × GeForce RTX 3090 (24 GB) at $0.484 per hour.
Size your own setup in the calculator →VRAM by precision and context length
Weights plus the KV cache of one request (BF16 cache) and the model's fixed recurrent state, in GiB. Add 1 to 3 GiB for the inference engine itself.
| Weights format | Weights only | + 8K context | + 32K context | + 128K context | + 1M context |
|---|---|---|---|---|---|
| NVFP4 (native) | 64.4 | 64.6 | 64.8 | 65.5 | 72.5 |
| BF16 / FP16 | 226 | 227 | 227 | 227 | 234 |
| FP8 / INT8 | 114 | 114 | 115 | 115 | 122 |
| 4-bit (GGUF Q4_K_M) | 75.6 | 75.8 | 76.0 | 76.8 | 83.8 |
How many GPUs to run Nemotron 3.5 Super VL 120B-A12B
Fewest GPUs that fit the weights plus one 8K-token request. Data-center GPUs use vLLM (90% of memory usable), the others llama.cpp. Several GPUs split the model with tensor or pipeline parallelism.
| GPU | NVFP4 (native) | BF16 / FP16 | 8-bit | 4-bit |
|---|---|---|---|---|
| GeForce RTX 4090 24 GB | Not on this GPU | 16 (several machines) | 6 | 4 |
| GeForce RTX 5090 32 GB | Not on this GPU | 8 | 4 | 3 |
| RTX PRO 6000 Blackwell 96 GB | Not on this GPU | 3 | 2 | 1 |
| Mac M5 Max 128 GB 96 GB usable | Not on this GPU | 3 | 2 | 1 |
| Ryzen AI Max+ 395 128 GB 96 GB usable | Not on this GPU | 3 | 2 | 1 |
| DGX Spark (GB10) 128 GB | Not on this GPU | 2 | 1 | 1 |
| NVIDIA H100 80 GB | Not on this GPU | 4 | 2 | 2 |
| NVIDIA H200 141 GB | Not on this GPU | 2 | 1 | 1 |
| NVIDIA B200 (HGX) 180 GB | 1 | 2 | 1 | 1 |
8-bit: FP8 with vLLM (INT8 on GPUs without FP8), Q8_0 with llama.cpp. 4-bit: AWQ / GPTQ with vLLM, Q4_K_M with llama.cpp, both counted at the Q4_K_M size. Not on this GPU: the official NVFP4 weights do not run there (llama.cpp needs a GGUF version, and vLLM needs a GPU with FP4 support).
Cheapest way to run Nemotron 3.5 Super VL 120B-A12B today
For each GPU: the fewest cards that fit Nemotron 3.5 Super VL 120B-A12B with one 8K-token request, the most faithful weight format at that count, and the cheapest on-demand price (2026-10-20).
| Setup | Per hour | Weights | Provider | Per month | Rent |
|---|---|---|---|---|---|
| 4 × GeForce RTX 3090 24 GB | $0.484 | Q4_K_M | Vast.ai | $353.32 | Rent 4 × GeForce RTX 3090 on Vast.ai (opens in a new tab) → |
| 6 × RTX A4000 16 GB | $0.528 | Q4_K_M | Vast.ai | $385.44 | Rent 6 × RTX A4000 on Vast.ai (opens in a new tab) → |
| 2 × RTX A6000 48 GB | $0.672 | Q4_K_M | Vast.ai | $490.56 | Rent 2 × RTX A6000 on Vast.ai (opens in a new tab) → |
| 4 × RTX A5000 24 GB | $0.752 | Q4_K_M | Vast.ai | $548.96 | Rent 4 × RTX A5000 on Vast.ai (opens in a new tab) → |
| 4 × RTX PRO 4000 Blackwell 24 GB | $1.02 | Q4_K_M | Vast.ai | $744.60 | Rent 4 × RTX PRO 4000 Blackwell on Vast.ai (opens in a new tab) → |
| 2 × RTX 6000 Ada 48 GB | $1.07 | Q4_K_M | Vast.ai | $779.64 | Rent 2 × RTX 6000 Ada on Vast.ai (opens in a new tab) → |
| 3 × NVIDIA A40 48 GB | $1.20 | 4-bit | Vast.ai | $878.19 | Rent 3 × NVIDIA A40 on Vast.ai (opens in a new tab) → |
| 6 × GeForce RTX 5080 16 GB | $1.21 | Q4_K_M | Vast.ai | $880.38 | Rent 6 × GeForce RTX 5080 on Vast.ai (opens in a new tab) → |
On-demand prices from AWS, Google Cloud, Azure, RunPod, Vast.ai, Oracle Cloud, OVHcloud, Scaleway, Verda, Akash and UpCloud, checked every hour. Serving many users needs more KV cache, so more memory: size it in the calculator. Some links are affiliate links: StudioTV may earn a commission, at no extra cost to you.
AWS, Google Cloud and UpCloud prices: Spare Cores (opens in a new tab), CC BY-SA 4.0 (opens in a new tab), shared under the same license.
KV cache: how memory grows with context
Architecture: KV on 8 of 88 layers (GQA, 2 KV × 128) · 40 Mamba-2 layers · 40 MoE layers · C-RADIO vision encoder. Its KV cache is 11.0× smaller than if every layer used full attention, because only 8 of its 88 layers keep a KV cache; the others hold a small fixed-size state. Only 12B parameters are active per token, but all 124B must sit in memory: VRAM depends on the total, speed on the active part.
| Context per request | BF16 cache | FP8 cache |
|---|---|---|
| 8K tokens | 0.06 GiB | 0.03 GiB |
| 32K tokens | 0.25 GiB | 0.13 GiB |
| 128K tokens | 1.00 GiB | 0.50 GiB |
| 1M tokens | 8.00 GiB | 4.00 GiB |
Plus 162 MiB of recurrent state per request, whatever the context.
Nemotron 3.5 Super VL 120B-A12B VRAM FAQ
How much VRAM does Nemotron 3.5 Super VL 120B-A12B need?
In BF16 the weights alone take 226 GiB (243 GB). In FP8 that is 114 GiB, and about 75.6 GiB with 4-bit quantization (Q4_K_M); the official NVFP4 checkpoint is 64.4 GiB. Each request then adds KV cache: 0.06 GiB at 8K tokens and 0.25 GiB at 32K (BF16 cache).
Can Nemotron 3.5 Super VL 120B-A12B run on a single RTX 4090 (24 GB)?
No. Even in 4-bit it needs 4 RTX 4090s. In 4-bit it fits on a single RTX PRO 6000 Blackwell (96 GB).
Can Nemotron 3.5 Super VL 120B-A12B run on an RTX 5090 (32 GB)?
No. Even in 4-bit it needs 3 RTX 5090s.
Can Nemotron 3.5 Super VL 120B-A12B run on a Mac?
Yes. In 4-bit (Q4_K_M with llama.cpp) it needs a Mac with 128 GB of unified memory or more, of which macOS gives the GPU about 96 GB, for one request with an 8K-token context. In 8-bit (Q8_0) it needs 192 GB or more.
Can Nemotron 3.5 Super VL 120B-A12B run on a DGX Spark or a Ryzen AI Max+ 395 (128 GB)?
Yes. On a DGX Spark (128 GB) it fits in Q8_0 (114 GiB with an 8K-token context); on a Ryzen AI Max+ 395 with 128 GB, whose GPU gets up to 96 GB, it fits in Q4_K_M (75.8 GiB with an 8K-token context), with llama.cpp and one request.
What is the cheapest way to run Nemotron 3.5 Super VL 120B-A12B?
On 2026-10-20, the cheapest on-demand setup is 4 × GeForce RTX 3090 (24 GB) with Q4_K_M weights on Vast.ai, at $0.484 per hour (about $353.32 per month). Next: 6 × RTX A4000 (16 GB) with Q4_K_M weights on Vast.ai, at $0.528 per hour. Sized for one 8K-token request; prices are checked every hour.
How many H100 GPUs does Nemotron 3.5 Super VL 120B-A12B need?
With one request and an 8K-token context: 4 in BF16, 2 in FP8 and 2 in 4-bit (vLLM, 90% of memory usable). Serving many users at once needs more memory for their KV caches: the calculator sizes that for you.
Nemotron 3.5 Super VL 120B-A12B VRAM badge
For a model card or a README: the 4-bit size and the smallest GPU it fits on, linked to this page. Data as JSON: /api/models/nemotron-3-5-super-vl-120b-a12b.json.
[](https://studiotvai.com/vram-requirements/nemotron-3-5-super-vl-120b-a12b)Other models
Same family (hybrid mamba): Nemotron 3 Nano 30B-A3B · Nemotron 3 Super 120B-A12B · Nemotron 3.5 Bolt 74B-A9B · Nemotron 3 Nano 4B
All 131 other models
Llama 3.1 8B · Llama 3 8B · Llama 3.2 3B · Gemma 2 9B · Gemma 3 4B · LLaVA 1.6 Mistral 7B · Mistral Small 3.2 24B · Qwen3 32B · Llama 3.3 70B · Llama 3.1 405B · Hy-Memory 30B-A3B · Xing4.0 29B-A4B · Qwen3.5 4B · MiniCPM-V-4.7 1B · Qwen3.5 9B · Qwen3.8 27B · Qwen3.6 35B-A3B · Qwen3.5 122B-A10B · Qwen3.8-Flash-Next · LFM2.5 8B-A1B · LFM2 24B-A2B · Gemma 4 12B · Gemma 4 31B · Gemma 4 26B-A4B · gpt-oss 20B · gpt-oss 120B · Llama 4 Scout 17B-16E · Llama 4 Maverick 17B-128E · Kolibri-1 · Nemotron Labs Dory 1.0 4B · Step 3.5 Flash · Muse Glimmer 30B · Nemotron 3 Nano 30B-A3B · Nemotron 3 Super 120B-A12B · Nemotron 3.5 Bolt 74B-A9B · MiniMax M2.7 · DeepSeek R1 · Kimi K2.6 · GLM-5.3 · GLM-5.3-Flash · DeepSeek V4 Flash · DeepSeek V4 Pro · DeepSeek V4.1 Flash · MiMo-V2.6 Pro · MiMo-V2.6 Flash · Muse Spark Open · Beam 1.0 501B-A23B · Kimi K3 · Mistral Large 4 · Qwen3-Coder-Next 80B-A3B · Qwen3-Coder 30B-A3B · Qwen3-Coder 480B-A35B · DeepSeek V4 Flash Vision Exp · Qwen3.8 2.4T-A95B · Ornith 1.5 397B · Ornith 1.5 35B-A3B · Ornith 1.5 9B · Atria Dawn Preview · Intern S2 397B · Clef · Clef Flash · D1 3B · Humanizer · JEV 27B VL · Mellum2.1 12B-A2.5B · LightOnOCR-3 4B · Spark-X2.5 4B · Agnes-3.0-Qwen · Youtu-Parsing-Omni · Agnes-3.0-Flash · Mistral 7B v0.3 · Mistral Small 3.1 24B · Mistral Nemo · Ministral 8B · DeepSeek V3 · DeepSeek R1 Qwen3 8B · DeepSeek R1 Distill Qwen 32B · DeepSeek V3.1 · DeepSeek R1 Distill Qwen 14B · DeepSeek R1 Distill Qwen 7B · DeepSeek R1 Distill Llama 8B · Qwen3 4B · Qwen3 8B · Qwen2.5 7B · Qwen3 14B · Qwen2.5 3B · Qwen3.6 27B · Qwen3.5 27B · Qwen2.5-Coder 7B · Qwen2.5 32B · Qwen2.5-Coder 14B · Qwen3.5 35B-A3B · Qwen3 30B-A3B · Qwen2.5 14B · Qwen2.5-Coder 32B · Kimi K2.7 Code · Kimi K2.5 · Kimi K2 · GLM-5.2 · GLM-5.1 · GLM-5 · MiniMax M2.5 · MiniMax M2 · MiMo-V2.5 · Ornith-1.0 397B · Qwen2.5-Coder 3B · Qwen3 235B-A22B · Qwen3.5 397B-A17B · Gemma 4 E4B · Gemma 4 E2B · Nemotron 3 Nano 4B · Nemotron 3.5 Lightning 30B-A3B · MiniCPM-V-4.7 35B-A3B · LFM2.5 24B-A2B · GLM-4.7 · Qwen3-VL 8B · Qwen2.5-VL 7B · Qwen3-VL 4B · Qwen2.5-VL 3B · Qwen2.5-VL 32B · Qwen2-VL 7B · GLM-4.5-Air · GLM-4.5 · phi-4 · Phi-3.5-mini · Phi-4-mini · Phi-3-mini-128k · Unlimited-OCR · Ling-3.1-flash · Step 5 Preview · Hy3
Estimates, not guarantees: computed from the official config.json with the same engine as the LLM VRAM Calculator. Real usage depends on your engine version and settings.