Mistral Large 2411 123B VRAM Requirements

Mistral Large 2411 123B at a glance

Made by

Mistral AI, France

Released

November 18, 2024

Parameters

122.6B

Context window

128K tokens

License

Mrl license

Reads

Text only

Languages

English, French, German, Spanish, Italian, Portuguese and 4 more

On Hugging Face

288 likes, 11K downloads last month (opens in a new tab)

Runs with

vLLM, SGLang, llama.cpp (Ollama, LM Studio), MLX (Mac)

Checked against

KV cache matches the GGUF header of bartowski/Mistral-Large-Instruct-2411-GGUF (opens in a new tab) (what llama.cpp allocates), October 30, 2026

VRAM by precision and context length

Weights plus the KV cache of one request (BF16 cache), in GiB. Add 1 to 3 GiB for the inference engine itself.

Mistral Large 2411 123B GGUF files

Exact size of each quantization in bartowski/Mistral-Large-Instruct-2411-GGUF (opens in a new tab), and the smallest GPU that runs it with llama.cpp, Ollama or LM Studio and an 8K-token context.

Read every day from Hugging Face. Split files are added together; vision projectors are left out.

How many GPUs to run Mistral Large 2411 123B

Fewest GPUs that fit the weights plus one 8K-token request. Data-center GPUs use vLLM (90% of memory usable), the others llama.cpp. Several GPUs split the model with tensor or pipeline parallelism.

8-bit: FP8 with vLLM (INT8 on GPUs without FP8), Q8_0 with llama.cpp. 4-bit: AWQ / GPTQ with vLLM, Q4_K_M with llama.cpp, both counted at the Q4_K_M size.

Cheapest way to run Mistral Large 2411 123B today

For each GPU: the fewest cards that fit Mistral Large 2411 123B with one 8K-token request, the most faithful weight format at that count, and the cheapest on-demand price (2026-10-30).

On-demand prices from AWS, Google Cloud, Azure, RunPod, Vast.ai, Oracle Cloud, OVHcloud, Scaleway, Verda, Akash and UpCloud, checked every hour. Serving many users needs more KV cache, so more memory: size it in the calculator. Some links are affiliate links: StudioTV may earn a commission, at no extra cost to you.

AWS, Google Cloud and UpCloud prices: Spare Cores (opens in a new tab), CC BY-SA 4.0 (opens in a new tab), shared under the same license.

KV cache: how memory grows with context

Architecture: 88 layers · GQA, 8 KV heads × 128. Every layer keeps keys and values for every token (grouped-query attention), so the cache grows linearly with context.

Mistral Large 2411 123B VRAM badge

For a model card or a README: the 4-bit size and the smallest GPU it fits on, linked to this page. Data as JSON: /api/models/mistral-large-2411.json.

[![Mistral Large 2411 123B VRAM](https://studiotvai.com/badge/mistral-large-2411.svg)](https://studiotvai.com/vram-requirements/mistral-large-2411)

Other models

Same family (dense): Llama 3.1 8B · Llama 3 8B · Llama 3.2 3B · Nanbeige4.3 33B · Granite 4.2 30B · Granite 4.2 8B · Granite 4.2 3B · MiniCPM5 2B · MiniCPM5 1B · Ministral 3 14B · Ministral 3 8B · Ministral 3 3B · Devstral 2 123B · Devstral Small 2 24B · Mistral Medium 3.5 128B · Magistral Small 1.2 24B · Mistral Small 3 24B · Pixtral Large 124B · Codestral 22B · Mistral Small 2409 22B · Pixtral 12B · Nemotron 3.5 Parse · LLaVA 1.6 Mistral 7B · Mistral Small 3.2 24B · Qwen3 32B · Llama 3.3 70B · Llama 3.1 405B · Youtu-Parsing-Omni · Mistral 7B v0.3 · Mistral Small 3.1 24B · Mistral Nemo · DeepSeek R1 Qwen3 8B · DeepSeek R1 Distill Qwen 32B · DeepSeek R1 Distill Qwen 14B · DeepSeek R1 Distill Qwen 7B · DeepSeek R1 Distill Llama 8B · Qwen3 4B · Qwen3 8B · Qwen2.5 7B · Qwen3 14B · Qwen2.5 3B · Qwen2.5 32B · Qwen2.5 14B · Qwen3-VL 8B · Qwen2.5-VL 7B · Qwen3-VL 4B · Qwen2.5-VL 3B · Qwen2.5-VL 32B · Qwen2-VL 7B · phi-4 · Phi-3.5-mini · Phi-4-mini · Phi-3-mini-128k · WeVisOne-VL 4B · Granite 4.1 3B

Estimates, not guarantees: computed from the official config.json with the same engine as the LLM VRAM Calculator. Real usage depends on your engine version and settings.