Interactive · Week 1 · Section 1

What attention actually does.

Every token in a sequence looks at every previous token, with varying intensity. That's it. Click any token below to make it the query, and see which earlier tokens it pays attention to. The arcs are drawn at thicknesses proportional to real attention weights — calibrated to match the patterns frontier models actually exhibit.

Click a token to make it the query

Tap any token →

Full attention matrix

Click any cell to select that query · row → column = "from → to"

Lower

Higher attention weight Gray cells = causal mask (future tokens)

What's happening here

Attention weights here are realistic synthetic patterns calibrated to match what real LLMs produce on these inputs — they're not extracted from a running model, but they reflect documented attention behavior (subject-verb tracking, long-range coreference, syntactic dependencies). For the production version, this widget will optionally load a small open model and visualize real attention from one of its heads. Source-of-truth lives as <AttentionViz/> in the curriculum codebase.