Can I run this LLM?

Can a NVIDIA RTX 6000 Ada (48GB) run GLM-4.5 Air (MoE)?

GLM · MIT · estimated for a 48.0 GB machine at 4K context.

Yes — it runs great.

Best fit: GLM-4.5 Air (MoE) at Q2_K · 35.9 GB · 107–161 tok/s on a NVIDIA RTX 6000 Ada (48GB).

Every quantization on this machine

Quant	Size	Speed	Fit
Q2_K	35.9 GB	107–161 tok/s	Runs great
Q3_K_M	47.6 GB	90–135 tok/s	Won't fit
Q4_K_M	67.0 GB	71–106 tok/s	Won't fit
Q5_K_M	78.7 GB	63–94 tok/s	Won't fit
Q6_K	91.1 GB	56–84 tok/s	Won't fit
Q8_0	117.4 GB	46–69 tok/s	Won't fit
FP16	220.9 GB	27–40 tok/s	Won't fit

Ran GLM-4.5 Air (MoE) on a NVIDIA RTX 6000 Ada (48GB)? Add your real numbers →