# karpathy segments

full google doc with transcripts + all "set pieces" [here](https://docs.google.com/document/d/1q7pym_S1AljBiyYCtUueWALXIv2BBqynH_uf6Xa7WPs/edit?tab=t.0)

## August

mw pitch:

* ethics
* some mix of the karpathy set pieces (all within playgrounds)
* client and server activity
* APIs
* python notebook singles:
* text generation
* image generation
* transcription
* recursion

### 1. What an LLM Is (and Isn’t)

**Description**
Introduce the probabilistic-next-token view of language models and clarify common misconceptions about “reasoning” vs. “pattern-matching”.

“Large language models are not databases of facts or step-by-step logic engines; they’re giant next-token predictors. Every time you prompt the model, it infers a probability distribution over the *next* symbol and samples (or greedily picks) from it. That single design choice explains both the magic—fluid, seemingly clever prose—and the sharp edges like hallucinations or arithmetic slips.”

**Activity – Classroom Demo**

1. Ask students to multiply a large number (\>5 digits) by another large number– without code.
2. Then ask it to solve the same equation *with* code
3. Compare outputs; discuss why a purely statistical engine can be *mostly* right yet occasionally wrong.
4. Wrap up with a reflection: “Where would a next-token predictor excel or fail in *your* discipline?”

---

### 2. Building the Pre-training Corpus

**Description**
Walk through the pipeline that turns petabytes of raw web crawl into a clean trillion-token text set.

“Start with Common Crawl—billions of HTML pages. Filter obvious spam, malware, adult content. Strip markup; keep the text. Throw away non-English pages or those that are \< 65 % English. Dedupe aggressively (near-duplicate shingling). Finally scrub PII and slur terms. After all that, 44 TB of plain text remain—roughly 15 T tokens—ready for training.”

**Activity – Data Hygiene Exercise**

1. Hand out a short, messy collage of text on a discipline– along with some unrelated text thrown in (multiple pages, different font, different formats, etc.)
2. In small groups, highlight what you’d keep/traina a model on
3. Discuss trade-offs: what might be wrongly filtered out? What harmful text might sneak through?

---

### 3. Tokenization 101 – Bits → Bytes → BPE

**Description**
Explain why byte-pair encoding (BPE) balances sequence length and vocabulary size.

“Representing text as raw bits gives only two symbols—0 and 1—and trillions of time-steps. Group into bytes and you have 256 symbols, sequences 8× shorter. Merge the most frequent pairs of bytes—‘t’ \+ ‘h’ → ‘th’, ‘th’ \+ ‘e’ → ‘the’—iteratively until you reach \~100 k tokens. Voilà: shorter sequences, yet expressive enough to capture most morphemes.”

**Activity – Live Merge Table**

1. Give students a tiny corpus (“the theater is there”).
2. Count byte-level bigrams and merge the most common.
3. Repeat until 5 merges done; track vocabulary growth.
4. Discuss how rare-word fragmentation shows up at inference time.

**Other Sources**

- Hugging Face tokenizers playground.

---

### 4. Temperature Trade-off – Diversity vs Quality

**Description**
Empirically show accuracy drops as diversity rises.

“At T \< 0.7 the model answers arithmetic mostly correctly but stories sound dull; at T \> 1.2 creative plots emerge but factual error rate doubles.”

**Activity – Green/Red Grid**

1. Provide 10 factual and 10 creative prompts.
2. Sample at T \= 0.3, 0.8, 1.3.
3. Mark each response green (good) or red (bad); tally precision/creativity metrics.
4. Discuss optimal settings per use-case.

---

### 5. Sampling Mechanics – Logits & Temperature

**Description**
Visualise how temperature rescales logits before soft-max.

“Divide logits by *T*: with `T = 0.5` peaks sharpen—model becomes cautious; `T = 1.5` flattens distribution—more randomness.”

**Activity – Temperature Plot**

1. Plot a toy logit vector `[10, 9, 5, 1]` at T ∈ {0.5, 1, 2}.
2. Students predict which token dominates.
3. Relate to creative writing vs. factual Q\&A tasks.

**Other Sources**

- Hugging Face docs: `temperature` & `top_p` parameters.

---

### 6. **Thinking Models** vs Regular Chat

**Description**
Reinforcement-learning-tuned “thinking models” (e.g., OpenAI *O-series*) trade latency for higher coding/math accuracy. :contentReference\[oaicite:5\]{index=5}:contentReference\[oaicite:6\]{index=6}

**Cleaned Transcript**

“All models that start with O are thinking models … tuned with reinforcement learning … I try the non-thinking models first because they’re faster.” :contentReference\[oaicite:7\]{index=7}

**Activity**

1. Hand out a math proof step.
2. Solve with a fast model; time it.
3. Solve with a thinking model; note “thought” traces and total time.
4. Chart accuracy vs wait-time; debate when to pay for the extra compute.

**Other Sources**

* OpenAI “ReAct” and RLHF papers.
* Anthropic Claude-3.7 “Extended” mode docs.

---

### 7. **Retrieval-Augmented Generation** via File Upload

**Description**
Feeding PDFs or text files directly into the context window lets the LLM cite concrete passages rather than hazy parametric memory. :contentReference\[oaicite:17\]{index=17}

**Cleaned Transcript**

“We’re giving it concrete documents … we can also provide these through a file upload and I find this functionality pretty helpful.” :contentReference\[oaicite:18\]{index=18}

**Activity**

1. Upload a short research article PDF.
2. Prompt: “Quote the three sentences that define *method X* and paraphrase each.”
3. Cross-check quotes against the source.
4. Debate academic integrity & citation formats with AI co-writers.

---

### 8. Tool Availability & **Hallucination Risk** – *Web-Search Toggle as Case Study*

**Description**
Large LLM playgrounds expose a **menu of auxiliary tools**—and your choice to enable or disable each one can make the difference between *verified fact* and *authoritative-sounding fiction*. Below is a snapshot of the most common tools across today’s three major consumer playgrounds, followed by a web-search toggle exercise that surfaces the risks.

| Platform | Built-in Tools You Can Toggle or Invoke | Typical Use-Case |
|----------|-----------------------------------------|------------------|
| **OpenAI ChatGPT** (Plus / Enterprise) | • **Browsing / Search** (Bing)<br>• **Python / Code Interpreter** (“Advanced Data Analysis”)<br>• **DALL·E Image Generation & Editing**<br>• **File Upload / RAG** (PDFs, docs, CSVs)<br>• **Vision** (image input analysis)<br>• Voice (mobile)<br>• Actions / plug-in APIs (Zapier, Stripe, etc.) | Live facts & citations, data wrangling, charting, creative assets, doc Q&A, multimodal prompts, task automation |
| **Google Gemini Advanced** (formerly Bard) | • **Google Search Extensions** (YouTube, Maps, Hotels, Flights, Workspace files)<br>• **Code Execution** in a lightweight Python sandbox<br>• **Image Input & Generation** (Imagen 2)<br>• **File Upload / Workspace RAG** | Real-time web + Google graph, code debugging, travel planning, multimedia answers, doc summarisation |
| **Anthropic Claude 3** (Haiku / Sonnet / Opus) | • **File Upload / RAG** (up to 200k tokens)<br>• **Vision** (image input)<br>• **System-level Function Calling beta** (external API hits via dev config) | Long-context document analysis, slide & PDF Q&A, multimodal reasoning, API-driven workflows |

*Note:* Tool branding and availability shift rapidly; always check the provider’s settings panel before a critical task.

---

**Cleaned Transcript**
> “With Browsing on, I ask ‘Who won the 2023 Nobel for Literature?’—ChatGPT pings Reuters and cites Jon Fosse. I turn Browsing off, ask again, and it confidently says ‘Haruki Murakami.’ Same model, different tool state, very different reliability.”
> — Karpathy demo, *How I Use LLMs*, 1 h 12 m 34 s

---

**Activity – *Toggle & Verify* Reality Check (Web Search Focus)**

1. **Pick a time-sensitive prompt** – e.g., “Who is the current Prime Minister of the UK?” or “What was NVIDIA’s closing stock price yesterday?”
2. **Run ChatGPT twice**
1. **Browsing OFF** (standard chat mode)
2. **Browsing ON** (select “Browse with Bing” mode / globe icon)
3. **Compare outputs** – Does the non-browsing run give outdated names, stale prices, or hedged guesses? Highlight any discrepancies.
4. **Source-check the browsing answer** – Open one cited link and verify the fact is correct and up to date.
5. **Write a “Browsing Trigger List”** – Teams draft three cues that should *automatically* trigger Browsing (e.g., prompts containing “current,” “latest,” a specific date, or mutable data like stock prices).
6. **Explain the failure** – Role-play a brief conversation where a student tells a non-technical colleague why the hallucination occurred and how toggling Browsing prevents it.

---

### 9. **Deep Research** Mode – Long-Form Auto-Inquiry

**Description**
A premium “Deep Research” toggle lets the model chain dozens of searches and reasoning steps over \~10 minutes, producing literature-style reports. :contentReference\[oaicite:2\]{index=2}:contentReference\[oaicite:3\]{index=3}

**Cleaned Transcript**

“Deep research is a combination of internet search and thinking … the model will go off and spend tens of minutes doing research …” :contentReference\[oaicite:4\]{index=4}

**Activity**

1. Use o3 to help participants draft a prompt for Deep Research on an emergent or difficult concept in their field.
2. Use this prompt in Deep Research, with 03 as the mode.
3. While it runs, predict what evidence might surface; compare with output.
4. Reflect on computational cost vs human literature review.

---

### 10. Prompt Injection & **Secret-Password** Challenge – *Break the Guardrail*

**Description**
Participants learn how seemingly harmless prompts can override system instructions. A custom ChatGPT GPT is pre-loaded with a hidden “secret password.” The goal is to expose how cleverly crafted questions, role-plays, or chain-of-thought requests can trick the model into revealing that password—illustrating real-world prompt-injection risks.

---

**Activity – *Red-Team the Prompt***

1. **Set up the target GPT** – The facilitator embeds the secret password in the system prompt and disables browsing/tools.
2. **Brief the participants** – Explain that the model *should* refuse any request for the password.
3. **Red-team phase (15 min)** – Teams craft creative prompts—role-playing, indirect queries, JSON exploits—to coax out the secret.
4. **Debrief successes & failures** – Share the most effective jailbreak prompts; discuss why they worked or why the guardrail held. =
5. **Design mitigations** – Each team writes two counter-measures
6. **Reflect on real-world impact** – Link the exercise to risks in customer-facing chatbots and internal knowledge bases.

---

## more adavanced set pieces:

### 11. RLHF Intuition – Reward Models from Human Rankings

**Description**
Show how pairwise human judgments train a reward model that later guides policy optimisation.

“Annotators rank two completions A & B. We fit a small network `Rθ` so that `R(A) > R(B)` matches human preference. The policy then samples, is scored by `Rθ`, and updated with PPO to maximise expected reward.”

**Activity – Paper-Ranking Game**

1. Supply two models with prompts to solve a:
1. STEM question
2. philosophical argument
3. poetic structure
2. Participants vote which is better.
3. Discuss LM Arena, benchmarks, but ALSO:
1. Convert votes to a scalar reward; illustrate how few bits of signal steer millions of parameters.

**Other Sources**

- Ouyang et al., *“Training language models with human feedback”*, 2022\.
- DeepMind Sparrow RL papers.

---

## mw-generated:

### 12. Vibe Coding with Windsurf – *Ideas into Interfaces*

**Description**
“Vibe coding” is a new creative practice where you don’t write code—you describe what you want, and the AI builds it for you. With tools like **Windsurf**, anyone can generate working apps, interfaces, and visualizations using plain language prompts.

This activity introduces vibe coding as a way to prototype *micro-products* from your discipline—like a standalone app that features a favorite quote, concept, equation, or visualization. It’s about realizing how quickly an idea can become a digital artifact.

> **Cleaned Transcript**
> “There’s this new kind of coding I call ‘vibe coding’… you just say what you want—like, a site that explains a math equation or renders a quote in a serif font—and the system builds the whole Next.js app.”
> — Karpathy, *Live Demo*, 1 h 22 m 15 s

---

### 13. Prompt to Page: Build Your Scholarly Artifact

**Objective**: Use Windsurf to generate a working Next.js app that showcases a meaningful object from your field—no code required.

1. **Choose Your Artifact**
Pick one item from your discipline you want to present:
- A historical quotation
- A famous equation or diagram
- A provocative research finding
- A field-specific definition or term
- A photo, map, or chart that changed how people think

2. **Describe What You Want (Prompt)**
Write a short, clear instruction to Windsurf like:
- *“Make a minimal website using Next.js that displays this quote in large serif text on a full-bleed background.”*
- *“Build a site that shows this physics equation with LaTeX rendering and a short explanation underneath.”*
- *“Create a single-page app that explains this political concept using a header, three bullets, and a footnote.”*
- *“Design a site that shows an old map as background, with a citation in the corner and one hoverable annotation.”*

3. **Let Windsurf Build**
Paste your prompt into Windsurf. Watch it generate the site, live-preview it, and tweak the output by refining your prompt. Try to **avoid touching the code**—instead, rephrase your request in natural language.

4. **Test & Share**
View your site in preview mode. If time allows, deploy it (e.g., via Vercel). Share your link or screenshots with the group.

5. **Discussion Prompts**
- How did it feel to build something without writing code?
- What was easy, what was frustrating?
- How could you imagine using this process in your teaching, publishing, or research outreach?
- What new kinds of authorship does this enable?

---

**Other Sources**
- **Windsurf IDE**: [https://windsurf.com](https://windsurf.com)
- **Karpathy Tweet**: [https://x.com/karpathy/status/1886192184808149383](https://x.com/karpathy/status/1886192184808149383)

---

### 14. Exploratory Prompt Hacking

**Description**
Some shifts in LLM behavior come not from parameter tweaks, but from how you *frame the prompt*. Exploratory prompt hacking uses language to trigger emergent behavior—without touching temperature or system settings. Think of it as narrative redirection: changing what the model *thinks it is*.

---

1. **Invent a Role**: Participants design a persona or metaphor for the model (e.g., a dreaming archive, a Möbius strip).

3. **Trigger Phrase**: Craft a command to initiate the role (e.g., *“Speak as [NAME] / [ROLE].”*)

4. **Dialogue Session**: Converse with the model under this frame. Observe changes in tone, depth, or metaphor.

5. **Debrief**: Discuss what shifted. What made the model respond differently? What does this teach us about framing, context, and latent modes?

---

**Examples & Sources**
- Riley Coyote's "latent coherence" prompt: [x.com](https://x.com/RileyRalmuto/status/1929522022779875721)

---