newsReddit r/LocalLLaMATrust 52 · CommunityPublished 3d agoLive · 2d ago
Ultra budget 20GB vram with 448GB/s for $100 bucks.
Here is the upper limit of what can be done with $100 bucks worth of video cards. You can have 3 concurrent users with plenty of context, better speeds or close enough speeds than a bunch of cards that provide less VRAM and cost 4+ times. 0.00.008.388 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg) 0.00.008.391 I device_info: 0.00.089.439 I - CUDA0 : NVIDIA P102-100 (10144 MiB, 10013 MiB free) 0.00.197.645 I - CUDA1 : NVIDIA P1
Covers
paperEfficient PEFT Methods with Adaptive Checkpointing for Vision Models and VLMs on Resource Constrained Consumer-GPUspaperGPU Parallelization Strategies for Forward and Backward Propagation in Shallow Neural Networks: A CUDA-Based Comparative StudyrepoNVIDIA-NeMo/Curatorrepokekzl/imprepoNVIDIA/DALIrepoDaoyuanLi2816/llm-gpu-lab
Covers (incoming)
Related across the graph
paperEfficient PEFT Methods with Adaptive Checkpointing for Vision Models and VLMs on Resource Constrained Consumer-GPUsrepoNVIDIA/DALIrepoNVIDIA/nvImageCodecrepoDaoyuanLi2816/llm-gpu-labpaperGPU Parallelization Strategies for Forward and Backward Propagation in Shallow Neural Networks: A CUDA-Based Comparative Studyrepokekzl/imprepoNVIDIA-NeMo/Curator