Can I run this LLM?

Can a AMD RX 9070 (16GB) run GLM-4.7 Flash (MoE)?

GLM · MIT · estimated for a 16.0 GB machine at 4K context.

Yes — it runs great.

Best fit: GLM-4.7 Flash (MoE) at Q2_K · 11.3 GB · 167–250 tok/s on a AMD RX 9070 (16GB).

Every quantization on this machine

Quant	Size	Speed	Fit
Q2_K	11.3 GB	167–250 tok/s	Runs great
Q3_K_M	14.6 GB	150–225 tok/s	Won't fit
Q4_K_M	18.3 GB	134–201 tok/s	Won't fit
Q5_K_M	21.4 GB	123–185 tok/s	Won't fit
Q6_K	24.7 GB	114–171 tok/s	Won't fit
Q8_0	31.8 GB	97–146 tok/s	Won't fit
FP16	59.9 GB	62–93 tok/s	Won't fit

Ran GLM-4.7 Flash (MoE) on a AMD RX 9070 (16GB)? Add your real numbers →