glossary termAngestromTrust 60Published 2mo agoLive · 2mo ago
Token
A chunk of text (often a word-piece) that a model reads and predicts.
A chunk of text (often a word-piece) that a model reads and predicts. A chunk of text (often a word-piece) that a model reads and predicts.
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- LinkedLinked via unknownSparse attention at million-token context →
- LinkedLinked via unknownSpeculative decoding with draft models →
- LinkedLinked via unknownWhich tokens does a hybrid model predict better? →
- PossiblePossibly related (embedding) · 49%mm7894215/TokenTracker →
- PossiblePossibly related (embedding) · 58%Tokens Define Model Cost and Context Limits - Let's Data Science →
- PossiblePossibly related (embedding) · 50%VasiHemanth/tokentelemetry →
- PossiblePossibly related (embedding) · 48%Anthropic's extravagant tokenizer complicates AI pricing →
- PossiblePossibly related (embedding) · 51%Tokens, vectors, contexts and costs - Blocks & Files →
Related to
Covers (incoming)
newsWhich tokens does a hybrid model predict better?newsTokens Define Model Cost and Context Limits - Let's Data SciencenewsAnthropic's extravagant tokenizer complicates AI pricingnewsTokens, vectors, contexts and costs - Blocks & FilesnewsTokenizer Expansion: Upgrading a Model's Tokenizer in Place - LFM2.5-8B-A1B
Related to (incoming)
Related across the graph
newsAnthropic's extravagant tokenizer complicates AI pricingnewsWhich tokens does a hybrid model predict better?newsTokens Define Model Cost and Context Limits - Let's Data SciencerepoVasiHemanth/tokentelemetryrepogeorg-jung/FastBertTokenizerpaperSparse attention at million-token contextrepomm7894215/TokenTrackernewsTokens, vectors, contexts and costs - Blocks & FilesnewsTokenizer Expansion: Upgrading a Model's Tokenizer in Place - LFM2.5-8B-A1BpaperSpeculative decoding with draft models