Attention

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

SGLang is a high-performance serving framework for large language models and multimodal models.

A concise but complete full-attention transformer with a set of promising experimental features from various papers

Build a modern LLM from scratch. Every line commented. Explained like we are five.

Learn LLM internals step by step - from tokenization to attention to inference optimization.

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

Explorations into training LLMs to use clinical calculators from patient history, using open sourced models. Will start with Wells' Criteria

The official repository of FlashSinkhorn [ICML 2026 Oral]

Implementation of Danijar's latest iteration for his Dreamer line of work

A hands-on course for building modern LLMs from scratch in PyTorch, with 26 runnable Jupyter Notebooks covering tokenizers, attention, MoE, RLHF, inference, eva…

Implementation of Mimic-Video, Video-Action Models for SOTA Generalizable Robot Control Beyond VLAs

Experimental playground for benchmarking language model (LM) architectures, layers, and tricks on smaller datasets. Designed for flexible experimentation and ex…

Pre-built wheels that erase Flash Attention 3 installation headaches.

Visual knowledge bank for understanding large language models, with 180 concept cards from tokenization to deployment.

Monitor Valorant gameplay through an external overlay to track player stats and match data in real time.

Monitor Valorant gameplay through an external overlay to track player stats and match data in real time.

Emotion text classification using Llama3-8b with LoRA and FlashAttention. Based on LLaMA-Factory.

Official code for the Manning book on structural LLM optimization: depth/width pruning, knowledge distillation, and attention optimization, runnable on free Col…

Valorant 2026: AI-Powered Crosshair & Cloud DNS Overlay for Next-Gen Esports

Implementation of Poly-attention, a higher-order self-attention proposed by Chakrabarti et al. of Columbia

Can LLMs Predict Their Own Failures? Self-Awareness via Internal Circuits

End-to-end pipeline for seeing how LLMs actually process your prompts. Capture attention across every layer, render heatmaps and cooking curves, compare variant…

Efficient multi-token attribution for reasoning language models — Python package, CLI, and HTML token traces

LLM Kernel Library for Rust

qMLX: Custom inference engine for Qwen 3.5 122B on Apple Silicon, extending MLX with hybrid attention support, SSD-backed KV cache, and RYS layer duplication fo…

Native Windows builds of vLLM 0.25.1 with CUDA 12.8, Python 3.13, Multi-TurboQuant, and experimental CPU/NVMe KV-cache offload.

Training-free KV cache compression via E8 lattice VQ. 2-bit KV that preserves retrieval (30/30 NIAH vs TurboQuant 0/30). Calibration-free, 9 architectures valid…

Unified KV cache compression for LLM inference — TurboQuant, IsoQuant, PlanarQuant, TriAttention. 10 methods, GPU-validated, multi-GPU planner. Compress KV cach…

Deep learning framework for LLMs (Llama/Gemma/Qwen) on CPU, Apple MLX, Metal, CUDA. Load PyTorch/ONNX/TF/GGUF with zero conversion. PyTorch alternative with nat…

A linear-cost attention variant that holds quality past a million tokens.

Implementations of many attention variants, benchmarked.