ConeheadAI · leftover lesson
SNK3-003 · Curriculum How-to
Self-attention in plain English — land at 34:20 (with context)
L0–L2 · Curious–Capable (bootstrap — system rebalances) · ~5m
Look for this insight
Self-attention means each word/token can look at other words in the same message to decide what matters — the core move behind modern chat models. You do not need the full lecture to use that idea.
Lesson hook
Karpathy's GPT lecture is long; we won't dump an hour on you. Three minutes that will make sense. Before the clip: modern chat models break text into tokens (word pieces). Self-attention is how each token gathers clues from other tokens in the same prompt — why 'bank' means river vs money from nearby words. At 34:20 he lands on that idea. Click once; watch ~3 minutes. Tool skill: know why context in the prompt matters when you use chat AI.
Do this now
Read the 2-sentence context on this card first (tokens + look-at-neighbors). Then click the snack (opens at 34:20). Watch until ~37:20. Completion MC: Self-attention is mainly about (A) encrypting the prompt (B) letting each token gather info from other tokens in the sequence (C) deleting half the model (D) checking your Gmail. Answer: B. Write one workplace example where nearby words change meaning (≤12 words).
Completion check: mc_choice_B
Staff review Sign in