newsReddit r/LocalLLaMATrust 58 · CommunityPublished 1mo agoLive · 1mo ago
Going from single GPU to dual GPU is nice but not in the way I expected
I was expecting what when doubling my VRAM from 24gb to 2x24gb I'd use higher quants with more context, and thus get smarter LLMs, but that's not what it ended up happening. At least for coding, I found that the difference in quality from, say, qwen 27B UD-Q4-XL to a Q6 or Q8 is rather small. Instead, at least for coding, the way I'm getting advantage of my extra power is parallelism. Instead of getting a smarter LLM, I am using qwen 27B with
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 46%NexusGPU/tensor-fusion →
- PossiblePossibly related (embedding) · 46%Efficient PEFT Methods with Adaptive Checkpointing for Vision Models and VLMs on Resource Constrained Consumer-GPUs →
- PossiblePossibly related (embedding) · 46%aivrar/multi-turboquant →
- PossiblePossibly related (embedding) · 50%notwitcheer/llm-bench-rig →
- PossiblePossibly related (embedding) · 55%helasaoudi/llm-inspector →
Covers (incoming)
paperGPU Parallelization Strategies for Forward and Backward Propagation in Shallow Neural Networks: A CUDA-Based Comparative StudyrepoNexusGPU/tensor-fusionpaperEfficient PEFT Methods with Adaptive Checkpointing for Vision Models and VLMs on Resource Constrained Consumer-GPUsrepoaivrar/multi-turboquantreponotwitcheer/llm-bench-rigrepohelasaoudi/llm-inspector
Related across the graph
paperEfficient PEFT Methods with Adaptive Checkpointing for Vision Models and VLMs on Resource Constrained Consumer-GPUsreponotwitcheer/llm-bench-rigrepoNexusGPU/tensor-fusionpaperGPU Parallelization Strategies for Forward and Backward Propagation in Shallow Neural Networks: A CUDA-Based Comparative Studyrepohelasaoudi/llm-inspectorrepoaivrar/multi-turboquant