Can I run this LLM?
Can a NVIDIA RTX 4090 (24GB) run GLM-4.5 Air (MoE)?
GLM · MIT · estimated for a 24.0 GB machine at 4K context.
GLM-4.5 Air: residency and offload on an RTX 4090
Content reviewed
Smallest listed pack · Q2_K
- Weights
- 45.3 GB
- KV cache
- 0.8 GB
- Runtime reserve
- 0.8 GB
- Total requirement
- 46.9 GB
- Listed hardware memory
- 24.0 GB
- Usable model budget
- 21.6 GB
Estimated at 4,096 tokens, llama.cpp, FP16 K/V cache, one stream and one GPU. Total = weights + cache + runtime reserve. The usable budget assumes 90% of device memory. Budget and reserve are site assumptions, not measured free memory. Actual peak allocation and runtime support can differ.
Active parameters are not the download
The publisher describes GLM-4.5 Air as a 106B-total, 12B-active mixture-of-experts model, with thinking and non-thinking modes. The active count concerns work per token, not resident weight storage. Its architecture headline differs from this catalog’s GGUF accounting; the breakdown uses listed pack sizes where available rather than multiplying the active count to infer memory.
Plan host memory before offloading
Compare the smallest listed pack shown here with the usable single-GPU budget. If it exceeds that budget, selecting an even smaller context cannot remove the weight allocation. llama.cpp supports CPU/GPU placement controls, but an offloaded run needs enough host RAM for the remainder and other allocations. PCIe transfers and CPU work can reduce speed; no fixed rate is promised here.
A smaller model can be the simpler setup
GLM-4.7 Flash is a smaller GLM comparison with version-specific serving instructions. Qwen3 14B’s publisher names local applications including llama.cpp and MLX-LM. These links use this RTX 4090 and identical short-context settings. They show memory estimates, not equal task quality or measured performance.
Smaller models to compare
Same-context catalog estimates; speed is unmeasured and task quality can differ.
See the calculation methodology, or contribute a measured run with setup details.
No — too big for this machine.
GLM-4.5 Air (MoE) needs more memory than the 24.0 GB on a NVIDIA RTX 4090 (24GB), even at its smallest quantization.
Arena Text Overall · style-controlled
1,373Local pack untestedglm-4.5-air · published configuration reference · local quantizations not tested
Arena Text Overall (style-controlled), exact published entry glm-4.5-air, 2026-10-08: reference rating 1,373.1. Published confidence bounds 1,368.9–1,377.4. 30,658 votes. Published glm-4.5-air configuration reference. Reasoning mode is not disclosed by the Arena dataset. Tested precision, runtime and sampling settings are not disclosed in this dataset. Your selected local quantization has not been measured by this Arena entry. No quantization quality adjustment is applied.
Snapshot published 2026-10-08 · snapshot imported 2026-10-11 UTC · validated last-good data · manual refresh
Arena leaderboard dataset · Arena / lmarena-ai, Arena Leaderboard Dataset (CC BY 4.0). Selected Text Overall style-controlled rows; field names adapted, ratings unchanged. · dataset CC BY 4.0 · revision e16f90232986
Too big for your machine — run it in the cloud
Needs 1–2× 80 GB GPU (A100 / H100) · about $1.50–4/hr
- RunPod — Per-second GPU pods — bring your own GGUF, spin up on demand.
- Vast.ai — Cheap GPU marketplace — rent by the hour, bring your own GGUF.
- Vultr — Cloud GPU VMs by the hour — full root access, run any stack.
Affiliate links — we may earn a commission if you sign up, at no cost to you. Prices are indicative and vary by provider and region.
Ran GLM-4.5 Air (MoE) on a NVIDIA RTX 4090 (24GB)? Add your real numbers →