Interactive labWASM · runs in browser
Self-attention from scratch
The mechanism behind every Transformer, in ~20 lines of NumPy — see exactly how query/key/value vectors turn into attention weights.
numpy
Python notebookClick Run to execute
The mechanism behind every Transformer, in ~20 lines of NumPy — see exactly how query/key/value vectors turn into attention weights.