Question 1

Can a NVIDIA RTX 5060 Ti (16GB) run Granite 4.2 30B?

Accepted Answer

Yes — Granite 4.2 30B runs at Q2_K (10.9 GB) at about 26–39 tok/s on a NVIDIA RTX 5060 Ti (16GB).

Question 2

How much memory does Granite 4.2 30B need?

Accepted Answer

Granite 4.2 30B needs about 12.7 GB at 4K context at its smallest quantization. A NVIDIA RTX 5060 Ti (16GB) has 16.0 GB.

Quant	Size	Speed	Fit
Q2_K	10.9 GB	26–39 tok/s	Runs great
Q3_K_M	14.1 GB	21–31 tok/s	Won't fit
Q4_K_M	17.7 GB	17–25 tok/s	Won't fit
Q5_K_M	20.8 GB	15–22 tok/s	Won't fit
Q6_K	24.0 GB	13–19 tok/s	Won't fit
Q8_0	31.1 GB	10–15 tok/s	Won't fit
FP16	58.6 GB	5–8 tok/s	Won't fit