Can I run this LLM?
Can your computer run Mellum2.1 12B A2.5B Thinking (MoE)?
Mellum · Apache 2.0
Memory & fit
Needs about 9.0 GB of memory at its smallest quantization (4K context). It runs on 73 of 88 tracked machines.
Pack & allocation notes: KV estimates assume a 1,024-token sliding window on 21 of 28 layers; actual runtime allocation may be higher. MXFP4_MOE is not listed. The FP16 label uses BF16 weights.