newsReddit r/LocalLLaMATrust 58 · CommunityPublished 4d agoLive · 4d ago
Qwen3.6-27B UD Q3 with kv at q8 is quite amazing for simple proof of concepts
Preface, technology is not my industry, but I am a very passionate poor man. So much so that I discovered 'AI' - ChatGPT in the beginning of 2025. So go easy on me, I only try. I kind of understand MOE vs. Dense models, MOEs are much forgiving when it comes to running as there are only X amount of experts activated at inference, if i understand correctly, where in dense model every parameter is activated so depending on the model size the software pushes
