Playground · Attention Visualiser
Attention, Visualised
Transformers don't read text sequentially — they let every token attend to every other token in parallel. Watch how each word in your sentence decides what to pay attention to.
Attention Weights
Each row shows how much one token attends to all others. Brighter = stronger attention. Click a row to highlight connections.
Query & Key
Each token has a query ("what am I looking for?") and a key ("what do I offer?"). Attention scores are the dot product of each query with all keys, scaled by √d.
Softmax
Raw scores are passed through softmax per row, turning them into a probability distribution. Now each token's attention weights sum to 1.
Value
Each token also has a value vector. The final output for a token is the weighted sum of all values, using the attention weights. This lets tokens mix information.
This is a toy illustration — real transformers have learned query/key/value projections across multiple attention heads. The query and key vectors here are fixed pseudo-random values seeded by each word, so the pattern is deterministic but not trained. For the real maths, see Learn.