Question 1

Can a NVIDIA RTX 5070 Ti (16GB) run Gemma 4 31B?

Accepted Answer

Just about — Gemma 4 31B runs at Q2_K (11.8 GB) at about 45–67 tok/s on a NVIDIA RTX 5070 Ti (16GB).

Question 2

How much memory does Gemma 4 31B need?

Accepted Answer

Gemma 4 31B needs about 13.7 GB at 4K context at its smallest quantization. A NVIDIA RTX 5070 Ti (16GB) has 16.0 GB.

Quant	Size	Speed	Fit
Q2_K	11.8 GB	45–67 tok/s	Tight / slow
Q3_K_M	14.7 GB	37–56 tok/s	Won't fit
Q4_K_M	18.3 GB	31–47 tok/s	Won't fit
Q5_K_M	21.7 GB	27–40 tok/s	Won't fit
Q6_K	25.2 GB	23–35 tok/s	Won't fit
Q8_0	32.6 GB	19–28 tok/s	Won't fit
FP16	61.4 GB	10–15 tok/s	Won't fit