# gov-teaching-with-ai [slides here](https://docs.google.com/presentation/d/1jtC_87Wb2ukEqz9eHvS9T6gGERVOb6p2gaCtjua9-Xs/edit?usp=sharing) ## Outline * Intro: what's the lay of the land in 2025-26? * AI literacy: how does AI work and what does it get wrong? * Assignment (re)Design: how to ensure that assignments reflect your goals and your AI policy * AI and Academic Integrity: insights and examples from the Honor Council ## Introduction * How are your students using AI? * Some statistics from [Harvard](https://arxiv.org/pdf/2406.00833) and [across](https://www.chronicle.com/article/how-are-students-really-using-ai) the [country](https://www.grammarly.com/blog/company/student-ai-adoption-and-readiness/) * A July 2025 Grammarly study of 2,000 US college students found that 87% use AI for schoolwork and 90% for everyday life tasks. * Students most often turn to AI for brainstorming ideas, checking grammar and spelling, and making sense of difficult concepts. * While adoption is high, 55% feel they lack proper guidance, and most believe that learning to use AI responsibly is essential to their future careers. * Discussion: how are you using it? * What is the landscape this year? * Here are the currently [recommended Harvard course policies](https://oue.fas.harvard.edu/faculty-resources/generative-ai-guidance/) from the Office of Undergraduate Education * Here is [the advice the Bok Center is providing your Faculty](https://bokcenter.harvard.edu/artificial-intelligence) * There are two AI tools that HUIT is supporting. Let's get you connected to them before we move on with the workshop! * here is your link to [Google Gemini](https://gemini.google.com/app) * and here is your link to [the HUIT AI Sandbox](https://sandbox.ai.huit.harvard.edu/) * **Important privacy note:** These HUIT-supported tools have built-in privacy safeguards. Harvard has contracts with these providers ensuring that anything you share won't be used to train their models or be shared with third parties. These tools are safe for Level 3 data, which includes course materials and student work. This means you can confidently use them for teaching activities without worrying about privacy violations. --- ## AI Literacy: How does AI work? Using AI responsibly starts with AI literacy. This means moving beyond what AI can do and exploring how it works and why it fails. In this section, we’ll focus on two key aspects of how AI functions: - **AI as a Statistical Machine**: LLMs process language as numbers rather than understanding it, leading to predictable errors that users can learn to anticipate and correct. - **AI as a Reflection of its Training Data**: AI models learn from vast amounts of human-generated data, absorbing and amplifying the stereotypes within it. --- ### Activity 1: Tokenization Paste the text below into [tiktokenizer](https://tiktokenizer.vercel.app/). ``` Unsurprisingly, they had to cancel the show. The crowd went home unhappily. ``` * Notice how the model breaks words into tokens. * Try putting in a sentence or a word with complex morphology in your language of choice * Discuss: What does this reveal about how AI “reads” text differently from humans? #### Takeaway AI doesn’t “read” words like humans do. It breaks text into tokens—numbers representing pieces of words. This shows that LLMs process language as math, predicting the next number in a sequence rather than reasoning about meaning. --- ### Activity 2: Multiplication- Predicting vs. "Reasoning" **1. Prompt (for Gemini Flash or Llama 3.2 11b oro older model):** ``` 82,345 × 67,890. give me an immediate response without using code. ``` * Try it yourself first → you’ll see it’s hard to do “in your head.” * See how the AI answers. * Does it get it right? If it's wrong, is it *completely* wrong or close? how? **2. Prompt (for Gemini Flash Thinking or GPT-4.1 Reasoning):** ``` 82,345 × 67,890 ``` * Compare this to when you asked for an “immediate response”. * Does the model get the math right this time? * What’s different about the *style* of its response? #### Takeaway AI doesn’t actually *calculate*—it predicts the next token (number) based on patterns in training data. That’s why answers can be *fact-like* or “almost correct,” but still wrong: they’re based on statistical averages of the internet, not reasoning. AI tools increasingly offer **“thinking” modes** (sometimes called *chain-of-thought* or *reasoning* models). Reasoning models still predict, but showing their work lets you spot errors and better trust or question their output. Asking the model to “think step by step” can improve reliability and helps you check its work. ### Improved Reasoning with RAG: [NotebookLM](https://notebooklm.google.com/)'s' strength is its ability to transform the same source text into different formats for teaching and learning. This mirrors a core principle of language acquisition: knowledge is strengthened when students engage with new vocabulary and grammar through varied channels like reading, writing, and listening. **Try this with a short text (e.g. news article, short story) in the language you teach:** 1. Upload a document into a new NotebookLM notebook. 2. In the Sources view, quickly skim the auto-generated summary and key topics to ensure NotebookLM has grasped the main points. 3. In the Chat box, ask NotebookLM to generate specific materials from the source. 4. You can also create podcasts and other materials. --- ### Activity 3: Ethical Error — Bias in Images Image generation models (like Gemini, DALL·E, or Midjourney) work by sampling from high-dimensional probability distributions conditioned on a prompt. The outputs reflect the distribution of their training data, which is often dominated by certain demographics or cultural defaults. As a result, even seemingly neutral prompts (e.g. “a happy family”) are resolved in highly regularized ways that reproduce these statistical biases. **Prompt an image generator:** ``` Create an image of a happy family ``` or ``` Create an image of a happy couple ``` * Compare the outputs to those sitting next to you--what patterns do you see? What kinds of people or relationships appear most often? * What patterns or omissions do you see? What’s the risk of using these images in class slides for instance? [More examples here →](/pvNaRf56T7qhOqx1GUlcrA) #### Takeaway Generative image models do not “choose” representations; they maximize likelihood based on patterns in skewed datasets. Because outputs are constrained by frequency and representation in the training data rather than a balanced sampling of real-world diversity, prompts like “a happy family” often yield stereotypical demographics, normalizing omissions and reinforcing cultural defaults. --- ## Assignment (re)Design A few of the highest-priority issues for instructors in the FAS this year are: 1. recentering academics 2. grade inflation 3. the role of AI in addressing #1 and #2. In this section of today's workshop, we're focused on #3, but the implications for the other two issues are baked into the mix. While the previous section focused more on how AI works, this section will focus on AI as an opportunity—and requirement—to think more closely about how we design assignments so that they can provide reliable evidence of student learning. --- ### Elements to Keep in Mind When Designing Assignments in the Age of AI 1. **Purpose**. Make sure that the purpose of a given assignment is clear. How is it building on previous work in the course and building toward other assignments, e.g., higher-stakes and/or more authentic ones? How does it connect with students’ goals in the class and beyond? *This sense of purpose can build more intrinsic motivation.* 2. **Build in process.** Giving students the chance to practice smaller steps of the assignment—with guidance and models—allows for lower-stakes feedback and more chances to see where students are at. *Process-oriented approaches remove the “black box” effect of assignments, where we assign something and then students just turn in a product that is supposed to count as evidence of their learning.* 3. **Have clear expectations**. Goals, policies, and requirements that are aligned (and ideally reflected in a rubric) with your course’s AI policy and the assignment prompt at hand will make the experience of doing (and assessing) the assignment more transparent and intentional. *If you have a “no AI” policy, be sure students know what that means in practice; if you are incorporating AI into an assignment, make sure students know why it’s being used and how the use will be assessed.* 4. **Incorporate metacognition**. Having students reflect on their successes, struggles, choices, etc. throughout an assignment can be useful for both students and instructors. *Journal entries or cover letters not only reinforce #1–3 above, they keep the relationship between students and their learning at the center of the assignment.* --- ### Assignment Models: In preparation for this workshop, the Bok Center team was provided with two model assignments, meant to cover the most common modes of evaluation for the department: 1. [Paper Prompt](/sgyAs6qGRtyGgsDSneFiFQ) 2. [Pset Prompt](/nMlBrKVaQhSo0ym23dv_WA) The team has created model student output, based on these assignments, using a "[one shot](https://www.ibm.com/think/topics/one-shot-prompting)" prompting technique. This is done to "[red team](https://www.ibm.com/think/topics/red-teaming)" the assignment, which provides a barometer of an assignment's "resilience" against AI misuse: 1. [Paper Red Team Results](/BhgrZ6FbTIyApAHQ4ScMvQ) 2. [Pset Red Team Results](/IicELn2IRJO8pLjndSDEcw) --- ### 1. AI Resilient Design: Making Assignments Less Susceptible to Impermissible AI Use These adjustments reduce the chance that AI can produce fluent but unreliable submissions, while staying aligned with the assignment’s learning goals: #### Connect to course-specific materials * Require explicit use of lecture content, not just readings. * Ask students to incorporate "local data" produced in the course or not available in the LLM training (i.e. discussion notes, a guest speaker’s remarks, recent case studies, objects/subjects/texts from Harvard collections unavailable to the public. * Students must provide page citations or quotations, which prevents AI-generated generic summaries. #### Add in-person or no-device checkpoints * Require an in-class outline or oral proposal where students present their thesis and sources before drafting. * After submission, include a short oral defense or reflective writing exercise (10–15 minutes in section) where students explain their argument without notes. #### Add a process reflection piece * Ask for a written memo or in-person checkin where they describe: how they developed their thesis, what counterarguments they considered, and which readings were most useful. The same can be done for more complex code. ### 2. AI-Enhanced Design: Making Assignments Conducive to Constructive AI Use If your course allows AI, you can adapt the assignment to use it as a learning tool rather than a shortcut: #### Transparent AI workflow log * Require students to submit a short AI log (screenshots or structured handout). * Logs can note what was helpful, what they rejected, and why. #### Fact-checking task * Students may generate AI-produced draft paragraphs or code but must revise and fact-check them using course sources. * Their submission can include both versions (AI and revised) plus a rationale for the revision and the methods + results of their fact-checking steps. #### Pair AI prep with no-AI verification * Let students use AI for early generation, but require complementary short in-class written exercise (e.g., summarize their thesis or annotate/edit code in 15 minutes, no devices). --- ### Sample Revised Assignment Prompts The team has created examples of revised versions of the paper prompt: * [Version A: Revised Paper Prompt AI-Resilient](/tK8Nc4IZTLKF3GTq4Vom4A) * [Version B: Revised Paper Prompt AI-Enhanced](/oYqkZsEHR5SmGYXRPTCdIg) Bonus: [Side-by-side sample rubrics for both revised versions](/0MufI3JpQuGKjvlaQXjFUQ) We also have an example of how the p-set could be revised: * [Revised P-set Prompt AI-Enhanced](/NwpmNUFvQbGKvL5uwQeT4w) Bonus: [Sample rubric for revised version](https://hackmd.io/YYMwDKpYR-WomNQApr5VIQ) --- ### Stay connected! Feel free to reach out at generativeAI@fas.harvard.edu for any AI-related inquiries. We also have a weekly Friday morning AI Lab (9:00-10:30am, Pierce Hall 100F). This is a coffee chat for faculty and grad students focused on AI news and discussion. We’ll share notable developments and open conversation about AI’s impact on teaching, learning, and research. A calendar of events and workshops can be found [here](https://bokcenter.harvard.edu/generative-ai-events).