repoGitHubTrust 82 · PrimaryPublished 3d agoLive · yesterday
marcelroed/gigatoken
Language model tokenization at GB/s
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 70%GigaToken: ~1000x faster Language model tokenization →
