Microsoft, United States · Released November 18, 2026New
Aion-1.0 2.3B has 2.29B parameters. In BF16 its weights alone take 4.26 GiB; quantized to 4-bit, about 1.37 GiB. Each 32K-token request adds 1.04 GiB of KV cache. Cheapest way to run it today: 1 × RTX A4000 (16 GB) at $0.088 per hour. It calls tools.
Read every day from Hugging Face. Split files are added together; vision projectors are left out.
How many GPUs to run Aion-1.0 2.3B
Fewest GPUs that fit the weights plus one 8K-token request. Data-center GPUs use vLLM (90% of memory usable), the others llama.cpp. Several GPUs split the model with tensor or pipeline parallelism.
GPU
BF16 / FP16
8-bit
4-bit
GeForce RTX 4090 24 GB
1
1
1
GeForce RTX 5090 32 GB
1
1
1
RTX PRO 6000 Blackwell 96 GB
1
1
1
Mac M5 Max 128 GB 96 GB usable
1
1
1
Ryzen AI Max+ 395 128 GB 96 GB usable
1
1
1
DGX Spark (GB10) 128 GB
1
1
1
NVIDIA H100 80 GB
1
1
1
NVIDIA H200 141 GB
1
1
1
NVIDIA B200 (HGX) 180 GB
1
1
1
8-bit: FP8 with vLLM (INT8 on GPUs without FP8), Q8_0 with llama.cpp. 4-bit: AWQ / GPTQ with vLLM, Q4_K_M with llama.cpp, both counted at the Q4_K_M size.
Cheapest way to run Aion-1.0 2.3B today
For each GPU: the fewest cards that fit Aion-1.0 2.3B with one 8K-token request, the most faithful weight format at that count, and the cheapest on-demand price (2026-11-22).
On-demand prices from AWS, Google Cloud, Azure, RunPod, Vast.ai, Oracle Cloud, OVHcloud, Scaleway, Verda, Akash and UpCloud, checked every hour. Serving many users needs more KV cache, so more memory: size it in the calculator. Some links are affiliate links: StudioTV may earn a commission, at no extra cost to you.
Architecture: 28 layers · 7 full attention (8 KV × 128) · 21 sliding-window layers (2048 tokens, 8 KV × 128). Its KV cache is 3.4× smaller than if every layer used full attention, because most layers only keep a short window of recent tokens.
Context per request
BF16 cache
FP8 cache
8K tokens
0.38 GiB
0.19 GiB
32K tokens
1.04 GiB
0.52 GiB
Aion-1.0 2.3B VRAM FAQ
How much VRAM does Aion-1.0 2.3B need?
In BF16 the weights alone take 4.26 GiB (4.6 GB). In FP8 that is 2.51 GiB, and about 1.37 GiB with 4-bit quantization (Q4_K_M). Each request then adds KV cache: 0.38 GiB at 8K tokens and 1.04 GiB at 32K (BF16 cache).
Can Aion-1.0 2.3B run on a single RTX 4090 (24 GB)?
Yes. In BF16 it fits on one RTX 4090 with an 8K-token context (4.65 GiB including the cache, with llama.cpp).
Can Aion-1.0 2.3B run on an RTX 5090 (32 GB)?
Yes. In BF16 (4.65 GiB with an 8K-token context) it fits on one RTX 5090, with llama.cpp.
Can Aion-1.0 2.3B run on a Mac?
Yes. In 4-bit (Q4_K_M with llama.cpp) it needs a Mac with 8 GB of unified memory or more, of which macOS gives the GPU about 5 GB, for one request with an 8K-token context. The same Mac also holds it in 8-bit (Q8_0).
Can Aion-1.0 2.3B run on a DGX Spark or a Ryzen AI Max+ 395 (128 GB)?
Yes. In BF16 (4.65 GiB with an 8K-token context) it fits on a DGX Spark (128 GB) and on a Ryzen AI Max+ 395 with 128 GB, whose GPU gets up to 96 GB, with llama.cpp and one request.
What is the cheapest way to run Aion-1.0 2.3B?
On 2026-11-22, the cheapest on-demand setup is 1 × RTX A4000 (16 GB) with BF16 weights on Vast.ai, at $0.088 per hour (about $64.24 per month). Next: 1 × NVIDIA L4 (24 GB) with BF16 weights on Vast.ai, at $0.104 per hour. Sized for one 8K-token request; prices are checked every hour.
How many H100 GPUs does Aion-1.0 2.3B need?
With one request and an 8K-token context: 1 in BF16, 1 in FP8 and 1 in 4-bit (vLLM, 90% of memory usable). Serving many users at once needs more memory for their KV caches: the calculator sizes that for you.
Does Aion-1.0 2.3B support tool calling?
Yes, Aion-1.0 2.3B supports tool (function) calling: it can ask your application to run a function and use the result.
Aion-1.0 2.3B VRAM badge
For a model card or a README: the 4-bit size and the smallest GPU it fits on, linked to this page. Data as JSON: /api/models/aion-1-0-2-3b.json.
Estimates, not guarantees: computed from the official config.json with the same engine as the LLM VRAM Calculator. Real usage depends on your engine version and settings.