Question 1

Can a AMD RX 9070 XT (16GB) run DeepSeek-R1-Distill 32B?

Accepted Answer

Just about — DeepSeek-R1-Distill 32B runs at Q2_K (12.3 GB) at about 32–49 tok/s on a AMD RX 9070 XT (16GB).

Question 2

How much memory does DeepSeek-R1-Distill 32B need?

Accepted Answer

DeepSeek-R1-Distill 32B needs about 14.2 GB at 4K context at its smallest quantization. A AMD RX 9070 XT (16GB) has 16.0 GB.

Quant	Size	Speed	Fit
Q2_K	12.3 GB	32–49 tok/s	Tight / slow
Q3_K_M	15.9 GB	26–39 tok/s	Won't fit
Q4_K_M	19.9 GB	21–32 tok/s	Won't fit
Q5_K_M	23.3 GB	19–28 tok/s	Won't fit
Q6_K	26.9 GB	16–24 tok/s	Won't fit
Q8_0	34.8 GB	13–19 tok/s	Won't fit
FP16	65.5 GB	7–11 tok/s	Won't fit