Can I run this LLM?

Can a NVIDIA L40S (48GB) run GLM-4.5 Air (MoE)?

GLM · MIT · estimated for a 48.0 GB machine at 4K context.

Yes — it runs great.

Best fit: GLM-4.5 Air (MoE) at Q2_K · 35.9 GB · 99–149 tok/s on a NVIDIA L40S (48GB).

Every quantization on this machine

Quant	Size	Speed	Fit
Q2_K	35.9 GB	99–149 tok/s	Runs great
Q3_K_M	47.6 GB	83–124 tok/s	Won't fit
Q4_K_M	67.0 GB	65–98 tok/s	Won't fit
Q5_K_M	78.7 GB	58–86 tok/s	Won't fit
Q6_K	91.1 GB	51–77 tok/s	Won't fit
Q8_0	117.4 GB	42–63 tok/s	Won't fit
FP16	220.9 GB	24–36 tok/s	Won't fit

Ran GLM-4.5 Air (MoE) on a NVIDIA L40S (48GB)? Add your real numbers →