Alibaba, China · Released August 8, 2026Qwen3.8 2.4T-A95B at a glance
- Made by
- Alibaba, China
- Released
- August 8, 2026
- Parameters
- 2446.2B, 95B active
- Context window
- 256K tokens
Qwen3.8 2.4T-A95B VRAM Requirements
Alibaba, China · Released August 8, 2026
Qwen3.8 2.4T-A95B has 2446.2B parameters (95B active per token). In BF16 its weights alone take 4507 GiB; quantized to 4-bit, about 1361 GiB. Each 32K-token request adds 2.88 GiB of KV cache. Cheapest way to run it today: 6 × AMD MI355X / MI350X (288 GB) at $32.94 per hour.
Size your own setup in the calculator →VRAM by precision and context length
Weights plus the KV cache of one request (BF16 cache) and the model's fixed recurrent state, in GiB. Add 1 to 3 GiB for the inference engine itself.
| Weights format | Weights only | + 8K context | + 32K context | + 128K context | + 256K context |
|---|---|---|---|---|---|
| BF16 / FP16 | 4507 | 4509 | 4511 | 4519 | 4531 |
| FP8 / INT8 | 2257 | 2259 | 2261 | 2269 | 2281 |
| 4-bit (GGUF Q4_K_M) | 1361 | 1362 | 1364 | 1373 | 1385 |
How many GPUs to run Qwen3.8 2.4T-A95B
Fewest GPUs that fit the weights plus one 8K-token request. Data-center GPUs use vLLM (90% of memory usable), the others llama.cpp. Several GPUs split the model with tensor or pipeline parallelism.
| GPU | BF16 / FP16 | FP8 / INT8 | 4-bit (GGUF Q4_K_M) |
|---|---|---|---|
| GeForce RTX 4090 24 GB | > 128 | 128 (several servers) | 96 (several servers) |
| GeForce RTX 5090 32 GB | > 128 | 96 (several servers) | 64 (several servers) |
| RTX PRO 6000 Blackwell 96 GB | 64 (several servers) | 32 (several servers) | 16 (several servers) |
| Mac M5 Max 128 GB 96 GB usable | 64 (several servers) | 32 (several servers) | 16 (several servers) |
| NVIDIA H100 80 GB | 96 (several servers) | 48 (several servers) | 24 (several servers) |
| NVIDIA H200 141 GB | 48 (several servers) | 24 (several servers) | 16 (several servers) |
| NVIDIA B200 (HGX) 180 GB | 32 (several servers) | 16 (several servers) | 16 (several servers) |
Cheapest way to run Qwen3.8 2.4T-A95B today
For each GPU: the fewest cards that fit Qwen3.8 2.4T-A95B with one 8K-token request, the most faithful weight format at that count, and the cheapest on-demand price (2026-10-10).
| Setup | Per hour | Weights | Provider | Per month | Rent |
|---|---|---|---|---|---|
| 6 × AMD MI355X / MI350X 288 GB | $32.94 | 4-bit (GGUF Q4_K_M) | RunPod | $24,046.20 | Rent 6 × AMD MI355X / MI350X on RunPod (opens in a new tab) → |
| 6 × NVIDIA B300 / GB300 288 GB | $53.94 | 4-bit (GGUF Q4_K_M) | RunPod | $39,376.20 | Rent 6 × NVIDIA B300 / GB300 on RunPod (opens in a new tab) → |
On-demand prices from RunPod, Vast.ai, Verda and Azure, checked every hour. Serving many users needs more KV cache, so more memory: size it in the calculator. Some links are affiliate links: StudioTV may earn a commission, at no extra cost to you.
KV cache: how memory grows with context
Architecture: 92 layers · 23 full attention (4 KV × 256) · 69 linear attention. Its KV cache is 4.0× smaller than if every layer used full attention, because only 23 of its 92 layers keep a KV cache; the others hold a small fixed-size state. Only 95B parameters are active per token, but all 2446B must sit in memory: VRAM depends on the total, speed on the active part.
| Context per request | BF16 cache | FP8 cache |
|---|---|---|
| 8K tokens | 0.72 GiB | 0.36 GiB |
| 32K tokens | 2.88 GiB | 1.44 GiB |
| 128K tokens | 11.5 GiB | 5.75 GiB |
| 256K tokens | 23.0 GiB | 11.5 GiB |
Plus 560 MiB of recurrent state per request, whatever the context.
Qwen3.8 2.4T-A95B VRAM FAQ
How much VRAM does Qwen3.8 2.4T-A95B need?
In BF16 the weights alone take 4507 GiB (4840 GB). In FP8 that is 2257 GiB, and about 1361 GiB with 4-bit quantization (Q4_K_M). Each request then adds KV cache: 0.72 GiB at 8K tokens and 2.88 GiB at 32K (BF16 cache).
Can Qwen3.8 2.4T-A95B run on a single RTX 4090 (24 GB)?
No. Even in 4-bit it needs 96 (several servers) RTX 4090s. In 4-bit it needs 24 (several servers) H100 GPUs.
What is the cheapest way to run Qwen3.8 2.4T-A95B?
On 2026-10-10, the cheapest on-demand setup is 6 × AMD MI355X / MI350X (288 GB) with 4-bit (GGUF Q4_K_M) weights on RunPod, at $32.94 per hour (about $24,046.20 per month). Next: 6 × NVIDIA B300 / GB300 (288 GB) with 4-bit (GGUF Q4_K_M) weights on RunPod, at $53.94 per hour. Sized for one 8K-token request; prices are checked every hour.
How many H100 GPUs does Qwen3.8 2.4T-A95B need?
With one request and an 8K-token context: 96 (several servers) in BF16, 48 (several servers) in FP8 and 24 (several servers) in 4-bit (vLLM, 90% of memory usable). Serving many users at once needs more memory for their KV caches: the calculator sizes that for you.
Qwen3.8 2.4T-A95B VRAM badge
For a model card or a README: the 4-bit size and the smallest GPU it fits on, linked to this page. Data as JSON: /api/models/qwen3-8-2-4t-a95b.json.
[](https://studiotvai.com/vram-requirements/qwen3-8-2-4t-a95b)Other models
Same family (large moe): MiniMax M2.7 · DeepSeek R1 · Kimi K2.6 · GLM-5.3 · GLM-5.3-Flash · DeepSeek V4 Flash · DeepSeek V4 Pro · DeepSeek V4.1 Flash · MiMo-V2.6 Pro · MiMo-V2.6 Flash · Kimi K3 · Mistral Large 4 · DeepSeek V4 Flash Vision Exp · Ornith 1.5 397B · Atria Dawn Preview · Intern S2 397B
All models: Llama 3.1 8B · Mistral Small 3.2 24B · Qwen3 32B · Llama 3.3 70B · Llama 3.1 405B · Qwen3.5 4B · Qwen3.5 9B · Qwen3.8 27B · Qwen3.6 35B-A3B · Qwen3.5 122B-A10B · Qwen3.8-Flash-Next · LFM2.5 8B-A1B · LFM2 24B-A2B · Gemma 4 12B · Gemma 4 31B · Gemma 4 26B-A4B · gpt-oss 20B · gpt-oss 120B · Llama 4 Scout 17B-16E · Llama 4 Maverick 17B-128E · Kolibri-1 · Nemotron 3 Nano 30B-A3B · Nemotron 3 Super 120B-A12B · MiniMax M2.7 · DeepSeek R1 · Kimi K2.6 · GLM-5.3 · GLM-5.3-Flash · DeepSeek V4 Flash · DeepSeek V4 Pro · DeepSeek V4.1 Flash · MiMo-V2.6 Pro · MiMo-V2.6 Flash · Kimi K3 · Mistral Large 4 · Qwen3-Coder-Next 80B-A3B · Qwen3-Coder 30B-A3B · Qwen3-Coder 480B-A35B · DeepSeek V4 Flash Vision Exp · Ornith 1.5 397B · Ornith 1.5 35B-A3B · Ornith 1.5 9B · Atria Dawn Preview · Intern S2 397B · Clef · Clef Flash · D1 3B · Humanizer · JEV 27B VL · Mellum2.1 12B-A2.5B · LightOnOCR-3 4B · Spark-X2.5 4B · Agnes-3.0-Qwen
Estimates, not guarantees: computed from the official config.json with the same engine as the LLM VRAM Calculator. Real usage depends on your engine version and settings.