LG AI Research, South Korea · Released July 29, 2026
K-EXAONE 2.0 750B-A37B has 749.4B parameters (37B active per token). In BF16 its weights alone take 1395 GiB; quantized to 4-bit, about 421 GiB. Each 32K-token request adds 2.54 GiB of KV cache. Cheapest way to run it today: 7 × NVIDIA A100 80GB at $4.41 per hour. It calls tools and has a thinking mode you can switch off.
Size your own setup in the calculator →Weights plus the KV cache of one request (BF16 cache), in GiB. Add 1 to 3 GiB for the inference engine itself.
| Weights format | Weights only | + 8K context | + 32K context | + 128K context | + 256K context |
|---|---|---|---|---|---|
| BF16 / FP16 | 1395 | 1395 | 1397 | 1405 | 1415 |
| FP8 / INT8 | 699 | 700 | 702 | 709 | 719 |
| 4-bit (GGUF Q4_K_M) | 421 | 422 | 424 | 431 | 441 |
Exact size of each quantization in DevQuasar/NOT_WOKRKING-LGAI-EXAONE.K-EXAONE-2.0-750B-A37B-GGUF (opens in a new tab), and the smallest GPU that runs it with llama.cpp, Ollama or LM Studio and an 8K-token context.
| Quantization | File size | Smallest GPU |
|---|---|---|
| Q2_K | 254 GiB | More than 96 GB: see the calculator |
Read every day from Hugging Face. Split files are added together; vision projectors are left out.
Fewest GPUs that fit the weights plus one 8K-token request. Data-center GPUs use vLLM (90% of memory usable), the others llama.cpp. Several GPUs split the model with tensor or pipeline parallelism.
| GPU | BF16 / FP16 | 8-bit | 4-bit |
|---|---|---|---|
| GeForce RTX 4090 24 GB | 96 (several machines) | 48 (several machines) | 24 (several machines) |
| GeForce RTX 5090 32 GB | 64 (several machines) | 32 (several machines) | 16 (several machines) |
| RTX PRO 6000 Blackwell 96 GB | 16 (several machines) | 8 | 5 |
| Mac M5 Max 128 GB 96 GB usable | 16 (several machines) | 8 | 5 |
| Ryzen AI Max+ 395 128 GB 96 GB usable | 16 (several machines) | 8 | 5 |
| DGX Spark (GB10) 128 GB | 16 (several machines) | 6 | 4 |
| NVIDIA H100 80 GB | 24 (several servers) | 16 (several servers) | 7 |
| NVIDIA H200 141 GB | 16 (several servers) | 6 | 4 |
| NVIDIA B200 (HGX) 180 GB | 16 (several servers) | 5 | 3 |
8-bit: FP8 with vLLM (INT8 on GPUs without FP8), Q8_0 with llama.cpp. 4-bit: AWQ / GPTQ with vLLM, Q4_K_M with llama.cpp, both counted at the Q4_K_M size.
For each GPU: the fewest cards that fit K-EXAONE 2.0 750B-A37B with one 8K-token request, the most faithful weight format at that count, and the cheapest on-demand price (2026-10-30).
| Setup | Per hour | Weights | Provider | Per month | Rent |
|---|---|---|---|---|---|
| 7 × NVIDIA A100 80GB | $4.41 | 4-bit | Vast.ai | $3,219.30 | Rent 7 × NVIDIA A100 80GB on Vast.ai (opens in a new tab) → |
| 5 × RTX PRO 6000 Blackwell 96 GB | $6.67 | Q4_K_M | Vast.ai | $4,869.10 | Rent 5 × RTX PRO 6000 Blackwell on Vast.ai (opens in a new tab) → |
| 3 × AMD MI300X 192 GB | $7.17 | 4-bit | RunPod | $5,234.10 | Rent 3 × AMD MI300X on RunPod (opens in a new tab) → |
| 7 × NVIDIA H100 80 GB | $14.95 | 4-bit | Vast.ai | $10,914.96 | Rent 7 × NVIDIA H100 on Vast.ai (opens in a new tab) → |
| 6 × NVIDIA H100 NVL 94 GB | $15.82 | 4-bit | Vast.ai | $11,550.06 | Rent 6 × NVIDIA H100 NVL on Vast.ai (opens in a new tab) → |
| 2 × NVIDIA B300 / GB300 288 GB | $16.70 | 4-bit | Scaleway | $12,191.00 | Rent 2 × NVIDIA B300 / GB300 on Scaleway (opens in a new tab) → |
| 2 × AMD MI355X / MI350X 288 GB | $17.20 | 4-bit | Oracle Cloud | $12,556.00 | Rent 2 × AMD MI355X / MI350X on Oracle Cloud (opens in a new tab) → |
| 4 × NVIDIA H200 141 GB | $20.22 | 4-bit | Vast.ai | $14,757.68 | Rent 4 × NVIDIA H200 on Vast.ai (opens in a new tab) → |
On-demand prices from AWS, Google Cloud, Azure, RunPod, Vast.ai, Oracle Cloud, OVHcloud, Scaleway, Verda, Akash and UpCloud, checked every hour. Serving many users needs more KV cache, so more memory: size it in the calculator. Some links are affiliate links: StudioTV may earn a commission, at no extra cost to you.
AWS, Google Cloud and UpCloud prices: Spare Cores (opens in a new tab), CC BY-SA 4.0 (opens in a new tab), shared under the same license.
Sampling defaults from the official generation_config.json (opens in a new tab) of LGAI-EXAONE/K-EXAONE-2.0-750B-A37B.
| Setting | Value |
|---|---|
| Temperature | 1 |
| Top-p | 0.95 |
vLLM and SGLang apply these defaults unless a request sets its own. With llama.cpp, pass them yourself: --temp 1 --top-p 0.95
Reasoning. Thinking mode you can switch on or off. With vLLM or SGLang, set it per request: "chat_template_kwargs": {"enable_thinking": false} turns it off, true on. Reasoning tokens take context and KV cache like any other token: leave room for them.
Architecture: MoE 256 experts + 1 shared · 20 global layers + 58 sliding-window layers (57 × 128 tokens, 1 × 4096), GQA 8 KV × 128. Its KV cache is 3.8× smaller than if every layer used full attention, because most layers only keep a short window of recent tokens. Only 37B parameters are active per token, but all 749B must sit in memory: VRAM depends on the total, speed on the active part.
| Context per request | BF16 cache | FP8 cache |
|---|---|---|
| 8K tokens | 0.67 GiB | 0.33 GiB |
| 32K tokens | 2.54 GiB | 1.27 GiB |
| 128K tokens | 10.0 GiB | 5.02 GiB |
| 256K tokens | 20.0 GiB | 10.0 GiB |
In BF16 the weights alone take 1395 GiB (1498 GB). In FP8 that is 699 GiB, and about 421 GiB with 4-bit quantization (Q4_K_M). Each request then adds KV cache: 0.67 GiB at 8K tokens and 2.54 GiB at 32K (BF16 cache).
No. Even in 4-bit it needs 24 RTX 4090s, spread over several machines. In 4-bit it needs 7 H100 GPUs.
No. Even in 4-bit it needs 16 RTX 5090s, spread over several machines.
No single Mac holds it, even in 4-bit: it needs 422 GiB with an 8K-token context, and the largest Mac (512 GB) gives its GPU about 384 GB.
No. Even in 4-bit it needs 4 DGX Spark units or 5 Ryzen AI Max+ 395 machines (128 GB), linked over the network.
On 2026-10-30, the cheapest on-demand setup is 7 × NVIDIA A100 80GB with 4-bit weights on Vast.ai, at $4.41 per hour (about $3,219.30 per month). Next: 5 × RTX PRO 6000 Blackwell (96 GB) with Q4_K_M weights on Vast.ai, at $6.67 per hour. Sized for one 8K-token request; prices are checked every hour.
With one request and an 8K-token context: 24 in BF16, 16 in FP8 and 7 in 4-bit (vLLM, 90% of memory usable). Above 8 GPUs, that means several servers. Serving many users at once needs more memory for their KV caches: the calculator sizes that for you.
Yes, K-EXAONE 2.0 750B-A37B supports tool (function) calling: it can ask your application to run a function and use the result.
Thinking mode you can switch on or off. With vLLM or SGLang, set it per request with "chat_template_kwargs": {"enable_thinking": false} (or true). Reasoning tokens count toward the context and the KV cache.
The official defaults are temperature 1, top-p 0.95.
For a model card or a README: the 4-bit size and the smallest GPU it fits on, linked to this page. Data as JSON: /api/models/k-exaone-2-0.json.
LG AI Research, South Korea · Released July 29, 2026[](https://studiotvai.com/vram-requirements/k-exaone-2-0)Same family (large moe): MiniMax M3.1 Flash · MiniMax M3 · MiniMax M2.7 · DeepSeek R1 · Kimi K2.6 · GLM-5.3 · Atria Dawn · Atria Dawn Preview · Hy4 Preview · Inkling · GLM-5.3-Flash · DeepSeek V4 Flash · DeepSeek V4 Pro · DeepSeek V4.1 Flash · MiMo-V2.6 Pro · MiMo-V2.6 Flash · Muse Spark Open · Beam 1.0 501B-A23B · Kimi K3 · Mistral Large 4 · Mistral Large 3 · Mistral Small 4 119B · Leanstral 1.5 119B-A6B · Mixtral 8x22B · DeepSeek V4 Flash Vision Exp · Qwen3.8 2.4T-A95B · Ornith 1.5 397B · Intern S2 397B · DeepSeek V3 · DeepSeek V3.1 · Kimi K2.7 Code · Kimi K2.5 · Kimi K2 · GLM-5.2 · GLM-5.1 · GLM-5 · MiniMax M2.5 · MiniMax M2 · MiMo-V2.5 · Ornith-1.0 397B · Qwen3 235B-A22B · Qwen3.5 397B-A17B · GLM-4.7 · GLM-4.5 · Ling-3.1-flash · Step 5 Preview · Hy3
Llama 3.1 8B · Llama 3 8B · Llama 3.2 3B · Nanbeige4.3 33B · Granite 4.2 30B · Granite 4.2 8B · Granite 4.2 3B · MiniCPM5 2B · MiniCPM5 1B · Ministral 3 14B · Ministral 3 8B · Ministral 3 3B · Devstral 2 123B · Devstral Small 2 24B · Mistral Medium 3.5 128B · Magistral Small 1.2 24B · Mistral Small 3 24B · Mistral Large 2411 123B · Pixtral Large 124B · Codestral 22B · Mistral Small 2409 22B · Pixtral 12B · Nemotron 3.5 Parse · Gemma 2 9B · Gemma 3 4B · LLaVA 1.6 Mistral 7B · Mistral Small 3.2 24B · Qwen3 32B · Llama 3.3 70B · Llama 3.1 405B · Hy-Memory 30B-A3B · Kimi VL A3B Instruct · Mixtral 8x7B · Unlimited-OCR · Unlimited-OCR 2 · Xing4.0 29B-A4B · Qwen3.5 4B · MiniCPM-V-4.7 1B · Raon-1 mini preview · Qwen3.5 9B · Qwen3.8 27B · Qwen3.6 35B-A3B · Qwen3.5 122B-A10B · Qwen3.8-Flash-Next · LFM3 1.7B · LFM3-VL 2B · d1 2B · LFM2.5 8B-A1B · LFM2 24B-A2B · Gemma 4 12B · Gemma 4 31B · Gemma 4 26B-A4B · gpt-oss 20B · gpt-oss 120B · Llama 4 Scout 17B-16E · Llama 4 Maverick 17B-128E · Kolibri-1 · Recursive Flint · Nemotron Labs Dory 1.0 4B · Step 3.5 Flash · Step 3.7 Flash · Laguna S 2.1 · Laguna XS 2.1 · Command A+ · Inkling Small · Muse Glimmer 30B · Nemotron 3 Nano 30B-A3B · Nemotron 3 Super 120B-A12B · Nemotron 3 Ultra 550B-A55B · Mamba-Codestral 7B · Nemotron 3.5 Super VL 120B-A12B · Nemotron 3.5 Bolt 74B-A9B · MiniMax M3.1 Flash · MiniMax M3 · MiniMax M2.7 · DeepSeek R1 · Kimi K2.6 · GLM-5.3 · Atria Dawn · Atria Dawn Preview · Hy4 Preview · Inkling · GLM-5.3-Flash · DeepSeek V4 Flash · DeepSeek V4 Pro · DeepSeek V4.1 Flash · MiMo-V2.6 Pro · MiMo-V2.6 Flash · Muse Spark Open · Beam 1.0 501B-A23B · Kimi K3 · Mistral Large 4 · Mistral Large 3 · Mistral Small 4 119B · Leanstral 1.5 119B-A6B · Mixtral 8x22B · Qwen3-Coder-Next 80B-A3B · Qwen3-Coder 30B-A3B · Qwen3-Coder 480B-A35B · DeepSeek V4 Flash Vision Exp · Qwen3.8 2.4T-A95B · Ornith 1.5 397B · Ornith 1.5 35B-A3B · Ornith 1.5 9B · Intern S2 397B · Clef · Clef Flash · d1 3B · Humanizer · JEV 27B VL · Mellum2.1 12B-A2.5B · LightOnOCR-3 4B · Spark-X2.5 4B · Agnes-3.0-Qwen · Youtu-Parsing-Omni · Agnes-3.0-Flash · Mistral 7B v0.3 · Mistral Small 3.1 24B · Mistral Nemo · Ministral 8B · DeepSeek V3 · DeepSeek R1 Qwen3 8B · DeepSeek R1 Distill Qwen 32B · DeepSeek V3.1 · DeepSeek R1 Distill Qwen 14B · DeepSeek R1 Distill Qwen 7B · DeepSeek R1 Distill Llama 8B · Qwen3 4B · Qwen3 8B · Qwen2.5 7B · Qwen3 14B · Qwen2.5 3B · Qwen3.6 27B · Qwen3.5 27B · Qwen2.5-Coder 7B · Qwen2.5 32B · Qwen2.5-Coder 14B · Qwen3.5 35B-A3B · Qwen3 30B-A3B · Qwen2.5 14B · Qwen2.5-Coder 32B · Kimi K2.7 Code · Kimi K2.5 · Kimi K2 · GLM-5.2 · GLM-5.1 · GLM-5 · MiniMax M2.5 · MiniMax M2 · MiMo-V2.5 · Ornith-1.0 397B · Qwen2.5-Coder 3B · Qwen3 235B-A22B · Qwen3.5 397B-A17B · Gemma 4 E4B · Gemma 4 E2B · Nemotron 3 Nano 4B · Nemotron 3.5 Lightning 30B-A3B · MiniCPM-V-4.7 35B-A3B · LFM2.5 24B-A2B · GLM-4.7 · Qwen3-VL 8B · Qwen2.5-VL 7B · Qwen3-VL 4B · Qwen2.5-VL 3B · Qwen2.5-VL 32B · Qwen2-VL 7B · GLM-4.5-Air · GLM-4.5 · phi-4 · Phi-3.5-mini · Phi-4-mini · Phi-3-mini-128k · Ling-3.1-flash · Step 5 Preview · Hy3 · Ornith-1.5 122B · Spark-X2.6 4B · Sage-A1.5 · MLExpert · Agents-A1 4B · Ling-3.0-flash-Sante · MiniCPM-OCR-1.0 · WeVisOne-VL 4B · Granite 4.1 3B
Estimates, not guarantees: computed from the official config.json with the same engine as the LLM VRAM Calculator. Real usage depends on your engine version and settings.