Read original ↗
newsReddit r/LocalLLaMATrust 52 · CommunityPublished 4d agoLive · 3d ago

Single RTX 5090: Qwen3.8-27B NVFP4 at a real 262K context in vLLM — 77 tok/s short-context, 64.7 tok/s at 128K

This is the Qwen3.8-27B setup I actually use every day on one RTX 5090. I wanted to write it down with enough detail that another 5090 owner can reproduce it instead of guessing which memory knobs I used. The short version: the full 262,144-token window fits together with vision, FP8 KV, prefix caching, tool calling, and a normal KDE desktop. Decode is 77.2 tok/s after a 1K prompt and 64.7 tok/s with 128K already resident. A successful 262,000-tok

Why these links exist

Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.

Covers

Related across the graph