LLM VRAM Calculator

How much VRAM and how many GPUs your LLM needs, with the real KV cache of today's architectures. Pick a model or paste any Hugging Face link.

Trending on Hugging Face

Estimates, not guarantees. Real usage depends on your engine and settings: keep some headroom and confirm with nvidia-smi. Confirm with nvidia-smi.

Estimator

Humanizer

InferencevLLM1 × 80 GB

Your setup

Pick a model, the context and your GPUs.

Model

Pick a model

Stephen Yu (independent)

12.0B parameters · 256K context · Apache 2.0

48 layers · 8 full attention (1 KV × 512) · 40 sliding-window layers (1024 tokens, 8 KV × 256)

Open the Humanizer model card

Workload

Weights

Requests

Context per request (tokens)

Concurrent requests

KV cache

Hardware

GPU

Memory per GPU (GB)

Number of GPUs

Serving engine · vLLM

Engine

Usable memory (%)

Overhead / GPU (GiB)

Reserves gpu_memory_utilization (0.90 by default) of each GPU, then fills what is left after weights and activations with paged KV cache.

How it is computed

weights = params × bits ÷ 8 ÷ (TP × PP)

KV = Σ layer groups · layers × width × cached tokens × bytes

cached = context · min(context, window) · context ÷ ratio

width = 2 × KV heads × head dim (GQA) · latent + RoPE (MLA)

(KV + state) × sequences + overhead ≤ memory × usable %

Assumes a modern engine with a paged KV cache that keeps only the window on sliding-window layers. Real usage can differ by a few GiB.

Your answer

How many GPUs, how fast, and what it costs.

To run it you need

Fits

1 × NVIDIA H100 · 80 GB

Humanizer, BF16 / FP16 weights, 4 concurrent requests of 32K tokens. 80 GB of GPU memory in total.

From $2.27/h to rent, on Vast.aiCompare 4 providers ↓

Speed & cost

Per request ≈ 96 tok/s

Total, 4 requests≈ 385 tok/s

First token, 32K prompt ≈ 2.3 s

Your cost per 1M tokens $1.64

No API price for this model on OpenRouter.

Rough estimate (±30–50%) of generation at full context, from memory bandwidth (3.35 TB/s per GPU) and vLLM efficiency; first token from compute (989 TFLOPS). Speculative decoding is not modelled. API prices: OpenRouter, per generated token.

VRAM per GPU

Weights22.3 GiB
KV cache3.25 GiB
Runtime overhead3.00 GiB

Needed28.5 GiB

Usable 71.7 GiB

Headroom43.2 GiB

Capacity

Max requests at 32K57

Max context with 4 requests256K (model max)

KV per token 26.0 KiB

KV per request 832.0 MiB

Treating every layer as full attention would give 12.0 GiB per request, 14.8× more than this architecture actually caches.

Compare with your real usage

Paste the startup log of vLLM or llama.cpp (llama-server, Ollama, LM Studio). It is read in your browser: nothing is sent.

Better quality or a cheaper GPU?

Pick a more faithful weight format (it may need more GPUs) or a cheaper card. Click an option to apply it.

Does it fit? Weights × context on 1 × NVIDIA H100

Weights	4K	8K	32K	128K	256K
BF16 / FP16
FP8
INT4 GPTQ / AWQ

Longest context that fits

BF16 / FP16 to INT4 GPTQ / AWQ up to 256K

✓ fits · ~ tight · ✕ does not fit. Context per request, 4 concurrent requests, vLLM. Click a cell to apply it. Tap a format to use it at its longest context.

Quantization on NVIDIA H100

1 GPU

Fewer bits per weight, fewer GPUs, at some quality cost. The KV cache is set separately.

Cheapest GPUs that fit in one server

Cheapest on-demand price among RunPod, Verda, Vast.ai, Microsoft Azure, checked every hour. Weight format as selected above.

Other models that fit on 1 × NVIDIA H100

Largest first, each with the most faithful weight format that fits 32K tokens × 4 requests.

Rent and deploy

Today's NVIDIA H100 prices per provider, then the command to launch it.

NVIDIA H100 rental prices

Provider	Per GPU-hour	1 GPU, per month	Rent
Vast.ai	$2.27	$1,656.37	Rent on Vast.ai (opens in a new tab) →
RunPod	$3.99	$2,912.70	Rent on RunPod (opens in a new tab) →
Verda	$4.01	$2,925.11	Rent on Verda (opens in a new tab) →
Microsoft Azure	$12.29	$8,971.70	Rent on Microsoft Azure (opens in a new tab) →

Loading the daily history…

On-demand, per GPU, checked every hour (last check 2026-10-11 19:08 UTC). Monthly = 730 hours. NVIDIA H100 price page

Deploy it

vllm serve jialinyyzz/humanizer \
  --tensor-parallel-size 1 \
  --max-model-len 32768 \
  --max-num-seqs 4 \
  --gpu-memory-utilization 0.90 \
  --dtype bfloat16

Rent 1 × NVIDIA H100 on Vast.ai $2.27/h

Some rental links are affiliate links: StudioTV may earn a commission, at no extra cost to you.

Fits1 × NVIDIA H100$2.27/h

New & trending open models

Lab countries from the registry of this site, completed with Epoch AI (opens in a new tab) (CC BY 4.0).

Use it from your AI assistant or your code

Ask Claude, ChatGPT or any MCP client whether a model fits on your GPU: it calls this calculator and answers with the same numbers, for the listed models and any Hugging Face model. No key, no sign-up.

MCP server (Streamable HTTP): https://studiotvai.com/api/mcp
Tools: estimate_vram, models_that_fit, gpu_prices

Claude Code: claude mcp add --transport http studiotv-vram https://studiotvai.com/api/mcp

The same data as JSON: /api/models.json (every model, with the link to its file), /api/gpus.json, and one file per model such as /api/models/llama-3-3-70b.json. Each model page also has a VRAM badge to paste in a model card.

VRAM requirements by model

GPU memory for the weights alone, in GiB. Each model page details the KV cache by context length and how many GPUs of each type it takes.

Model	Parameters	BF16	FP8	4-bit
Dense
Llama 3.1 8B VRAM	8.0B	15.0	8.5	4.6
Mistral Small 3.2 24B VRAM	24B	44.7	23.6	13.4
Qwen3 32B VRAM	33B	61.0	32.0	18.6
Llama 3.3 70B VRAM	71B	131	67.7	39.9
Llama 3.1 405B VRAM	406B	756	382	229
Youtu-Parsing-Omni VRAMNew	5.3B	9.9	5.6	2.8
Mistral 7B-v0.3 VRAM	7.2B	13.5	7.0	4.1
Mistral-Small-3.1 24B VRAM	24B	44.7	23.6	13.4
Mistral-Nemo VRAM	12B	22.8	12.7	7.0
DeepSeek-R1-Qwen3 8B VRAM	8.2B	15.3	8.8	4.7
DeepSeek-R1-Distill-Qwen 32B VRAM	33B	61.0	32.0	18.6
DeepSeek-R1-Distill-Qwen 14B VRAM	15B	27.5	15.2	8.5
DeepSeek-R1-Distill-Qwen 7B VRAM	7.6B	14.2	8.1	4.4
DeepSeek-R1-Distill-Llama 8B VRAM	8.0B	15.0	8.5	4.6
Qwen3 4B VRAM	4.0B	7.5	4.1	2.3
Qwen3 8B VRAM	8.2B	15.3	8.8	4.7
Qwen2.5 7B VRAM	7.6B	14.2	8.1	4.4
Qwen3 14B VRAM	15B	27.5	15.2	8.5
Qwen2.5 3B VRAM	3.1B	5.7	3.2	1.8
Qwen2.5 32B VRAM	33B	61.0	32.0	18.6
Qwen2.5 14B VRAM	15B	27.5	15.2	8.5
Hybrid linear attention
Qwen3.5 4B VRAM	4.7B	8.5	4.8	2.5
Qwen3.5 9B VRAM	9.7B	17.5	10.7	5.2
Qwen3.8 27B VRAM	28B	51.0	27.8	15.4
Qwen3.6 35B-A3B VRAM	36B (3B active)	65.4	33.6	19.6
Qwen3.5 122B-A10B VRAM	125B (10B active)	228	116	68.8
Qwen3.8-Flash-Next VRAM	180B (6B active)	330	166	105
LFM2.5 8B-A1B VRAM	8.5B (1.5B active)	15.8	8.1	4.8
LFM2 24B-A2B VRAM	24B (2.3B active)	44.4	22.3	13.4
Ornith 1.5 35B-A3B VRAM	36B (3B active)	65.4	33.6	19.6
Ornith 1.5 9B VRAM	9.7B	17.5	10.7	5.2
Clef VRAMNew	27B	51.0	27.8	15.4
Clef Flash VRAMNew	9.4B	17.5	10.7	5.2
D1 3B VRAMNew	3.1B	5.8	3.4	1.6
JEV 27B VL VRAMNew	28B	51.0	27.8	15.4
LightOnOCR-3 4B VRAMNew	4.5B	8.5	4.8	2.5
Agnes-3.0-Qwen VRAMNew	180B (7.8B active)	330	166	105
Agnes-3.0-Flash VRAMNew	33B	60.8	32.8	18.4
Qwen3.6 27B VRAM	28B	51.0	27.8	15.4
Qwen3.5 27B VRAM	28B	51.0	27.8	15.4
Qwen3.5 35B-A3B VRAM	36B (3B active)	65.4	33.6	19.6
Sliding-window attention
Gemma 4 12B VRAM	12B	22.3	12.1	6.9
Gemma 4 31B VRAM	31B	58.2	30.4	17.5
Gemma 4 26B-A4B VRAM	26B (4B active)	48.1	24.7	15.4
gpt-oss 20B VRAM	21B (3.6B active)	38.9	20.6	10.8
gpt-oss 120B VRAM	117B (5.1B active)	218	110	58.3
Llama 4 Scout 17B-16E VRAM	109B (17B active)	202	103	60.8
Llama 4 Maverick 17B-128E VRAM	402B (17B active)	748	376	226
Kolibri-1 VRAMNew	78B (3.46B active)	145	73.3	44.0
Humanizer VRAMNew	12B	22.3	13.0	6.9
Mellum2.1 12B-A2.5B VRAMNew	12B (2.5B active)	22.6	11.7	7.4
Spark-X2.5 4B VRAM	4.1B	7.7	4.1	2.4
Ministral 8B VRAM	8.0B	14.9	8.5	4.6
Hybrid Mamba
Nemotron 3 Nano 30B-A3B VRAM	32B (3B active)	58.8	30.1	22.0
Nemotron 3 Super 120B-A12B VRAM	124B (12B active)	225	113	75.5
Large MoE
MiniMax M2.7 VRAM	229B (10B active)	426	214	129
DeepSeek R1 VRAM	684B (37B active)	1250	627	377
Kimi K2.6 VRAM	1027B (32B active)	1913	959	577
GLM-5.3 VRAM	753B (40.3B active)	1385	694	418
GLM-5.3-Flash VRAM	321B (18B active)	585	294	176
DeepSeek V4 Flash VRAM	291B (13B active)	530	266	160
DeepSeek V4 Pro VRAM	1599B (49B active)	2930	1467	885
DeepSeek V4.1 Flash VRAM	763B (16B active)	1395	699	421
MiMo-V2.6 Pro VRAMNew	1024B (42B active)	1904	954	574
MiMo-V2.6 Flash VRAMNew	311B (15B active)	577	290	174
Kimi K3 VRAM	2780B (104B active)	5178	2591	1563
Mistral Large 4 VRAMNew	1052B (49B active)	1959	981	591
DeepSeek V4 Flash Vision Exp VRAM	305B (15B active)	530	266	160
Qwen3.8 2.4T-A95B VRAM	2446B (95B active)	4507	2257	1361
Ornith 1.5 397B VRAM	403B (18B active)	739	371	223
Atria Dawn Preview VRAMNew	753B (42B active)	1385	694	418
Intern S2 397B VRAMNew	403B (18B active)	739	371	223
DeepSeek-V3 VRAM	685B (40B active)	1250	627	378
DeepSeek-V3.1 VRAM	685B (40B active)	1250	627	378
Kimi-K2.7-Code VRAM	1027B (33B active)	1913	959	577
Kimi-K2.5 VRAM	1027B (33B active)	1913	959	577
Kimi-K2 VRAM	1026B (33B active)	1912	958	577
GLM-5.2 VRAM	753B (42B active)	1385	694	418
GLM-5.1 VRAM	754B (42B active)	1386	695	418
GLM-5 VRAM	754B (42B active)	1386	695	418
MiniMax-M2.5 VRAM	229B (11B active)	426	214	129
MiniMax-M2 VRAM	229B (11B active)	426	214	129
MiMo-V2.5 VRAM	311B (17B active)	577	290	174
Ornith-1.0 397B VRAM	397B (11B active)	739	371	223
Coding models
Qwen3-Coder-Next 80B-A3B VRAM	80B (3B active)	148	74.8	44.9
Qwen3-Coder 30B-A3B VRAM	31B (3.3B active)	56.9	29.0	17.2
Qwen3-Coder 480B-A35B VRAM	480B (35B active)	894	449	270
Qwen2.5-Coder 7B VRAM	7.6B	14.2	8.1	4.4
Qwen2.5-Coder 14B VRAM	15B	27.5	15.2	8.5
Qwen2.5-Coder 32B VRAM	33B	61.0	32.0	18.6
Mixture of experts
Qwen3 30B-A3B VRAM	31B (3B active)	56.9	29.0	17.2

LLM VRAM FAQ

How much VRAM do I need to run a 70B model?

Take Llama 3.3 70B. Its weights need 131 GiB in BF16 (2 × H100 80 GB), 67.7 GiB in FP8 and about 39.9 GiB in 4-bit (1 × H100, or 2 × RTX 4090). Then add the KV cache of each request: 2.5 GiB at 8K tokens and 10.0 GiB at 32K.

Can I run an LLM on a 24 GB GPU like the RTX 4090?

Yes, models up to about 35B parameters fit in 4-bit with an 8K-token context: Llama 3.1 8B, Mistral Small 3.2 24B, Qwen3 32B, Qwen3.5 4B, Qwen3.5 9B, Qwen3.8 27B, Qwen3.6 35B-A3B, LFM2.5 8B-A1B, LFM2 24B-A2B, Gemma 4 12B, Gemma 4 31B, Gemma 4 26B-A4B, gpt-oss 20B, Qwen3-Coder 30B-A3B, Ornith 1.5 35B-A3B, Ornith 1.5 9B, Clef, Clef Flash, D1 3B, Humanizer, JEV 27B VL, Mellum2.1 12B-A2.5B, LightOnOCR-3 4B, Spark-X2.5 4B, Youtu-Parsing-Omni, Agnes-3.0-Flash, Mistral 7B-v0.3, Mistral-Small-3.1 24B, Mistral-Nemo, Ministral 8B, DeepSeek-R1-Qwen3 8B, DeepSeek-R1-Distill-Qwen 32B, DeepSeek-R1-Distill-Qwen 14B, DeepSeek-R1-Distill-Qwen 7B, DeepSeek-R1-Distill-Llama 8B, Qwen3 4B, Qwen3 8B, Qwen2.5 7B, Qwen3 14B, Qwen2.5 3B, Qwen3.6 27B, Qwen3.5 27B, Qwen2.5-Coder 7B, Qwen2.5 32B, Qwen2.5-Coder 14B, Qwen3.5 35B-A3B, Qwen3 30B-A3B, Qwen2.5 14B, Qwen2.5-Coder 32B. Larger models need several GPUs or a card with more memory.

How many GPUs do I need to run an LLM?

Each GPU must hold its share of the weights plus the KV cache of every concurrent request, under the memory the serving engine allows itself (90% by default in vLLM). The calculator tries 1, 2, 4, 8 GPUs and more, splitting the model with tensor parallelism inside a server and pipeline parallelism across servers, and returns the fewest that fit.

What drives LLM VRAM usage?

Four things decide how much memory a model needs once it is serving requests.

Weights are fixed: parameters × bits. “4-bit” formats really cost 4.2 to 4.9 bits once scales are counted.
KV cache grows with context × concurrent sequences; it decides how many users a GPU can serve.
Tensor parallelism splits GQA heads but never below one head per GPU, and replicates MLA latents.
Serving engines leave memory free on purpose: vLLM uses 90% of each GPU by default.

Why is VRAM higher than the model size on disk?

The file holds only the weights. At serving time you also pay for the KV cache, which grows with context × concurrent sequences, runtime buffers such as CUDA graphs and activation workspace, and the share of memory the engine leaves free on purpose.

Why are most LLM VRAM formulas wrong?

The classic estimate, 2 × layers × KV heads × head dim × bytes per token, assumes every layer attends to the whole context. Most models released since 2025 break that rule:

Hybrid linear attention (Qwen 3.5–3.8, Nemotron 3, Kimi K3): only 1 layer in 4, or fewer, keeps a KV cache; the rest hold a fixed-size state.
Sliding windows (Gemma 4, gpt-oss): most layers keep only the last 128 to 1,024 tokens.
MLA (GLM-5.3, Kimi K3): one 576-wide compressed latent per token instead of keys and values.
Compressed attention (DeepSeek V4): a 128-token window plus the rest of the sequence compressed 4× or 128×.

Why does the KV cache differ so much between models of the same size?

2026 architectures cache very different amounts per token. Qwen 3.5+ keeps a KV cache on only 1 layer in 4, Gemma 4 and gpt-oss limit most layers to a short sliding window, MLA models store one compressed latent instead of keys and values, and DeepSeek V4 compresses the sequence itself. At 128K context, Llama 3.3 70B needs 40 GiB of KV per sequence; Qwen3.8 27B needs 8 GiB.

Can quantization make a model fit on fewer GPUs?

Yes. Weights take parameters × bits per weight, so going from BF16 to 8 bits halves them and 4-bit formats cut them by about 3.5×. As a rule of thumb, 8-bit is near lossless, 5 to 6-bit loses very little and 4-bit loses a little more, depending on the model. Set a fixed number of GPUs and the calculator shows the most accurate format that fits.

Does quantizing the weights shrink the KV cache?

No. The weight format (FP8, NVFP4, GGUF Q4_K_M…) and the KV cache precision are set independently. An FP8 KV cache halves the cache whatever the weights are stored in, at a small accuracy cost.

How much memory does fine-tuning take?

Full fine-tuning with AdamW in mixed precision costs about 16 bytes per parameter before activations: BF16 weights and gradients, an FP32 master copy and two FP32 optimizer moments. LoRA keeps the frozen model at 2 bytes per parameter, QLoRA at about 0.52. Activations then scale with sequence length × batch, not with parameter count.