Read original ↗

repoGitHubTrust 82 · PrimaryPublished 4d agoLive · 4d ago

bitsandbytes-foundation/bitsandbytes

Accessible large language models via k-bit quantization for PyTorch.

Lineage graph

Paper → model → repo connections mined from source citations (Tier-1 exact match).

Implements

paper$\text{Log}_\text{b}$Quant: Quantizing Language Models in Logarithmic Space paperGSRQ: Gain-Shape Residual Quantization for Sub-1-bit KV Cache paperRaBitQCache: Rotated Binary Quantization for KVCache in Long Context LLM Inference

Related to

toolQuantBench glossary_termTransformer

Implements (incoming)

paperBiSCo-LLM: Lookup-Free Binary Spherical Coding for Extreme Low-Bit Large Language Model Compression paperThe Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs

Related across the graph

paperBiSCo-LLM: Lookup-Free Binary Spherical Coding for Extreme Low-Bit Large Language Model Compression glossary_termTransformer paperThe Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs paperGSRQ: Gain-Shape Residual Quantization for Sub-1-bit KV Cache paperRaBitQCache: Rotated Binary Quantization for KVCache in Long Context LLM Inference toolQuantBench paper$\text{Log}_\text{b}$Quant: Quantizing Language Models in Logarithmic Space

Topics

llm machine-learning pytorch qlora quantization