Which LLMs can run on a single NVIDIA L40?
With 4-bit weights and an 8K-token context: Ornith 1.5 35B-A3B, Qwen3.5 35B-A3B, Qwen3.6 35B-A3B, MiniCPM-V-4.7 35B-A3B, Agnes-3.0-Flash, DeepSeek R1 Distill Qwen 32B, Qwen2.5 32B, Qwen2.5-Coder 32B, Qwen3 32B, Nemotron 3 Nano 30B-A3B, Nemotron 3.5 Lightning 30B-A3B, Gemma 4 31B, Qwen3 30B-A3B, Qwen3-Coder 30B-A3B, JEV 27B VL, Qwen3.6 27B, Qwen3.5 27B, Qwen3.8 27B, Clef, Gemma 4 26B-A4B, Mistral Small 3.1 24B, Mistral Small 3.2 24B, LFM2.5 24B-A2B, LFM2 24B-A2B, gpt-oss 20B, DeepSeek R1 Distill Qwen 14B, Qwen2.5-Coder 14B, Qwen2.5 14B, Qwen3 14B, Mistral Nemo, Mellum2.1 12B-A2.5B, Gemma 4 12B, Humanizer, Ornith 1.5 9B, Qwen3.5 9B, Clef Flash, LFM2.5 8B-A1B, DeepSeek R1 Qwen3 8B, Qwen3 8B, DeepSeek R1 Distill Llama 8B, Llama 3.1 8B, Ministral 8B, Gemma 4 E4B, DeepSeek R1 Distill Qwen 7B, Qwen2.5 7B, Qwen2.5-Coder 7B, Mistral 7B v0.3, Youtu-Parsing-Omni, Gemma 4 E2B, Qwen3.5 4B, LightOnOCR-3 4B, Spark-X2.5 4B, Qwen3 4B, Nemotron 3 Nano 4B, D1 3B, Qwen2.5 3B, Qwen2.5-Coder 3B. Larger models need several GPUs; the LLM VRAM calculator sizes them.