Alibaba, China · Released September 17, 2024
Qwen2.5 3B has 3.09B parameters. In BF16 its weights alone take 5.75 GiB; quantized to 4-bit, about 1.80 GiB. Each 32K-token request adds 1.13 GiB of KV cache. Cheapest way to run it today: 1 × GeForce RTX 3090 (24 GB) at $0.143 per hour.
Size your own setup in the calculator →KV cache matches the GGUF header of Qwen/Qwen2.5-3B-Instruct-GGUF (opens in a new tab) (what llama.cpp allocates), October 11, 2026
Weights plus the KV cache of one request (BF16 cache), in GiB. Add 1 to 3 GiB for the inference engine itself.
| Weights format | Weights only | + 8K context | + 32K context |
|---|---|---|---|
| BF16 / FP16 | 5.75 | 6.03 | 6.87 |
| FP8 / INT8 | 3.16 | 3.45 | 4.29 |
| 4-bit (GGUF Q4_K_M) | 1.80 | 2.08 | 2.92 |
Exact size of each quantization in Qwen/Qwen2.5-3B-Instruct-GGUF (opens in a new tab), and the smallest GPU that runs it with llama.cpp, Ollama or LM Studio and an 8K-token context.
| Quantization | File size | Smallest GPU |
|---|---|---|
| Q8_0 | 3.37 GiB | RTX 4060 (8 GB) |
| Q6_K | 2.60 GiB | RTX 4060 (8 GB) |
| Q5_K_M | 2.27 GiB | RTX 4060 (8 GB) |
| Q5_0 | 2.22 GiB | RTX 4060 (8 GB) |
| Q4_K_M | 1.96 GiB | RTX 4060 (8 GB) |
| Q4_0 | 1.86 GiB | RTX 4060 (8 GB) |
| Q3_K_M | 1.61 GiB | RTX 4060 (8 GB) |
| Q2_K | 1.28 GiB | RTX 4060 (8 GB) |
Read every day from Hugging Face. Split files are added together; vision projectors are left out.
Fewest GPUs that fit the weights plus one 8K-token request. Data-center GPUs use vLLM (90% of memory usable), the others llama.cpp. Several GPUs split the model with tensor or pipeline parallelism.
| GPU | BF16 / FP16 | 8-bit | 4-bit |
|---|---|---|---|
| GeForce RTX 4090 24 GB | 1 | 1 | 1 |
| GeForce RTX 5090 32 GB | 1 | 1 | 1 |
| RTX PRO 6000 Blackwell 96 GB | 1 | 1 | 1 |
| Mac M5 Max 128 GB 96 GB usable | 1 | 1 | 1 |
| NVIDIA H100 80 GB | 1 | 1 | 1 |
| NVIDIA H200 141 GB | 1 | 1 | 1 |
| NVIDIA B200 (HGX) 180 GB | 1 | 1 | 1 |
8-bit: FP8 with vLLM (INT8 on GPUs without FP8), Q8_0 with llama.cpp. 4-bit: AWQ / GPTQ with vLLM, Q4_K_M with llama.cpp, both counted at the Q4_K_M size.
For each GPU: the fewest cards that fit Qwen2.5 3B with one 8K-token request, the most faithful weight format at that count, and the cheapest on-demand price (2026-10-11).
| Setup | Per hour | Weights | Provider | Per month | Rent |
|---|---|---|---|---|---|
| 1 × GeForce RTX 3090 24 GB | $0.143 | BF16 | Vast.ai | $104.39 | Rent 1 × GeForce RTX 3090 on Vast.ai (opens in a new tab) → |
| 1 × NVIDIA L4 24 GB | $0.268 | BF16 | Vast.ai | $195.64 | Rent 1 × NVIDIA L4 on Vast.ai (opens in a new tab) → |
| 1 × GeForce RTX 5080 16 GB | $0.268 | BF16 | Vast.ai | $195.64 | Rent 1 × GeForce RTX 5080 on Vast.ai (opens in a new tab) → |
| 1 × GeForce RTX 4090 24 GB | $0.356 | BF16 | Vast.ai | $259.88 | Rent 1 × GeForce RTX 4090 on Vast.ai (opens in a new tab) → |
| 1 × NVIDIA A100 40GB | $0.427 | BF16 | Vast.ai | $311.71 | Rent 1 × NVIDIA A100 40GB on Vast.ai (opens in a new tab) → |
| 1 × GeForce RTX 5090 32 GB | $0.431 | BF16 | Vast.ai | $314.63 | Rent 1 × GeForce RTX 5090 on Vast.ai (opens in a new tab) → |
| 1 × RTX 6000 Ada 48 GB | $0.539 | BF16 | Vast.ai | $393.47 | Rent 1 × RTX 6000 Ada on Vast.ai (opens in a new tab) → |
| 1 × NVIDIA L40S 48 GB | $0.800 | BF16 | Vast.ai | $584.00 | Rent 1 × NVIDIA L40S on Vast.ai (opens in a new tab) → |
On-demand prices from RunPod, Vast.ai, Verda and Azure, checked every hour. Serving many users needs more KV cache, so more memory: size it in the calculator. Some links are affiliate links: StudioTV may earn a commission, at no extra cost to you.
Architecture: 36 layers · 36 full attention (2 KV × 128). Every layer keeps keys and values for every token (grouped-query attention), so the cache grows linearly with context.
| Context per request | BF16 cache | FP8 cache |
|---|---|---|
| 8K tokens | 0.28 GiB | 0.14 GiB |
| 32K tokens | 1.13 GiB | 0.56 GiB |
In BF16 the weights alone take 5.75 GiB (6.2 GB). In FP8 that is 3.16 GiB, and about 1.80 GiB with 4-bit quantization (Q4_K_M). Each request then adds KV cache: 0.28 GiB at 8K tokens and 1.13 GiB at 32K (BF16 cache).
Yes. In BF16 it fits on one RTX 4090 with an 8K-token context (6.03 GiB including the cache, with llama.cpp).
On 2026-10-11, the cheapest on-demand setup is 1 × GeForce RTX 3090 (24 GB) with BF16 weights on Vast.ai, at $0.143 per hour (about $104.39 per month). Next: 1 × NVIDIA L4 (24 GB) with BF16 weights on Vast.ai, at $0.268 per hour. Sized for one 8K-token request; prices are checked every hour.
With one request and an 8K-token context: 1 in BF16, 1 in FP8 and 1 in 4-bit (vLLM, 90% of memory usable). Serving many users at once needs more memory for their KV caches: the calculator sizes that for you.
For a model card or a README: the 4-bit size and the smallest GPU it fits on, linked to this page. Data as JSON: /api/models/qwen2-5-3b.json.
Alibaba, China · Released September 17, 2024[](https://studiotvai.com/vram-requirements/qwen2-5-3b)Same family (dense): Llama 3.1 8B · Mistral Small 3.2 24B · Qwen3 32B · Llama 3.3 70B · Llama 3.1 405B · Youtu-Parsing-Omni · Mistral 7B-v0.3 · Mistral-Small-3.1 24B · Mistral-Nemo · DeepSeek-R1-Qwen3 8B · DeepSeek-R1-Distill-Qwen 32B · DeepSeek-R1-Distill-Qwen 14B · DeepSeek-R1-Distill-Qwen 7B · DeepSeek-R1-Distill-Llama 8B · Qwen3 4B · Qwen3 8B · Qwen2.5 7B · Qwen3 14B · Qwen2.5 32B · Qwen2.5 14B
All models: Llama 3.1 8B · Mistral Small 3.2 24B · Qwen3 32B · Llama 3.3 70B · Llama 3.1 405B · Qwen3.5 4B · Qwen3.5 9B · Qwen3.8 27B · Qwen3.6 35B-A3B · Qwen3.5 122B-A10B · Qwen3.8-Flash-Next · LFM2.5 8B-A1B · LFM2 24B-A2B · Gemma 4 12B · Gemma 4 31B · Gemma 4 26B-A4B · gpt-oss 20B · gpt-oss 120B · Llama 4 Scout 17B-16E · Llama 4 Maverick 17B-128E · Kolibri-1 · Nemotron 3 Nano 30B-A3B · Nemotron 3 Super 120B-A12B · MiniMax M2.7 · DeepSeek R1 · Kimi K2.6 · GLM-5.3 · GLM-5.3-Flash · DeepSeek V4 Flash · DeepSeek V4 Pro · DeepSeek V4.1 Flash · MiMo-V2.6 Pro · MiMo-V2.6 Flash · Kimi K3 · Mistral Large 4 · Qwen3-Coder-Next 80B-A3B · Qwen3-Coder 30B-A3B · Qwen3-Coder 480B-A35B · DeepSeek V4 Flash Vision Exp · Qwen3.8 2.4T-A95B · Ornith 1.5 397B · Ornith 1.5 35B-A3B · Ornith 1.5 9B · Atria Dawn Preview · Intern S2 397B · Clef · Clef Flash · D1 3B · Humanizer · JEV 27B VL · Mellum2.1 12B-A2.5B · LightOnOCR-3 4B · Spark-X2.5 4B · Agnes-3.0-Qwen · Youtu-Parsing-Omni · Agnes-3.0-Flash · Mistral 7B-v0.3 · Mistral-Small-3.1 24B · Mistral-Nemo · Ministral 8B · DeepSeek-V3 · DeepSeek-R1-Qwen3 8B · DeepSeek-R1-Distill-Qwen 32B · DeepSeek-V3.1 · DeepSeek-R1-Distill-Qwen 14B · DeepSeek-R1-Distill-Qwen 7B · DeepSeek-R1-Distill-Llama 8B · Qwen3 4B · Qwen3 8B · Qwen2.5 7B · Qwen3 14B · Qwen3.6 27B · Qwen3.5 27B · Qwen2.5-Coder 7B · Qwen2.5 32B · Qwen2.5-Coder 14B · Qwen3.5 35B-A3B · Qwen3 30B-A3B · Qwen2.5 14B · Qwen2.5-Coder 32B · Kimi-K2.7-Code · Kimi-K2.5 · Kimi-K2 · GLM-5.2 · GLM-5.1 · GLM-5 · MiniMax-M2.5 · MiniMax-M2 · MiMo-V2.5 · Ornith-1.0 397B
Estimates, not guarantees: computed from the official config.json with the same engine as the LLM VRAM Calculator. Real usage depends on your engine version and settings.