Question 1

Can a NVIDIA RTX 3060 Laptop (6GB) run Granite 4.2 8B?

Accepted Answer

Just about — Granite 4.2 8B runs at Q2_K (3.4 GB) at about 52–78 tok/s on a NVIDIA RTX 3060 Laptop (6GB).

Question 2

How much memory does Granite 4.2 8B need?

Accepted Answer

Granite 4.2 8B needs about 4.9 GB at 4K context at its smallest quantization. A NVIDIA RTX 3060 Laptop (6GB) has 6.0 GB.

Quant	Size	Speed	Fit
Q2_K	3.4 GB	52–78 tok/s	Tight / slow
Q3_K_M	4.4 GB	44–65 tok/s	Won't fit
Q4_K_M	5.4 GB	37–56 tok/s	Won't fit
Q5_K_M	6.3 GB	33–49 tok/s	Won't fit
Q6_K	7.2 GB	29–43 tok/s	Won't fit
Q8_0	9.4 GB	23–35 tok/s	Won't fit
FP16	17.6 GB	13–20 tok/s	Won't fit