# Midterm II Extra Credit: ChatGPT as an Instructional Tool #### CS181 #### Spring 2023 ![](https://i.imgur.com/HMKpGFK.png) ## Overview In this extra-credit assignment, you will be exploring the capabilities of ChatGPT as an instructional tool in coding and content heavy CS courses. The final deliverable will be a scientific report written in conference style LaTex (see Conference Style Paper template at the end of this [slide deck](https://docs.google.com/presentation/d/1EQIupyrH7z2sUUH99CMQQJc9JzV8reTzQ6RYOeWQ1tE/edit?usp=sharing)). **Due Date:** Last day of Spring classes. ### Getting Started Open a free account with OpenAI, then go to [`https://chat.openai.com/chat`](https://chat.openai.com/chat) to access the ChatGPT interface. Orient yourself to using the ChatGPT interface, for example, see [ChatGPT tutorial: ChatGPT - A guide on how to use OpenAI's new ChatGPT](https://lablab.ai/t/chatgpt-guide). Experiment with using ChatGPT in coding and debugging, see [ChatGPT tutorial: How to easily improve your coding skills with ChatGPT](https://lablab.ai/t/chatgpt-tutorial-how-to-easily-improve-your-coding-skills-with-chatgpt). ## The Research Project For the rest of the semester, you will experiement with using ChatGPT to support your learning in CS181. You may work individually or in teams of two. While you are not required to engage with the technical underpinning behind the ChatGPT model, you are expected to be scientifically rigorous in your experiments and analyses. ### The Deliverable Your final deliverable will be a write-up summarizing related works, your experiments and your analyses, formatted as a conference-style paper. Your write-up must include: 1. **Introduction** - *Problem Statement* - What is the main question(s) you are investigating (e.g. "Can ChatGPT be used as an instructional tool for large coding-heavy classes, by automating low-level coding support like debugging?") - *Problem Impact* - Why is this an interesting or important question? If you can answer this question, then so what? 3. **Related Works** - Have similar or related questions been asked and answered in prior works? - Are there existing studies that can help you build your experiments and conduct your analyses? 5. **Experiments** - *Hypothesis* - clearly state what you think will happen (e.g. "I think using ChatGPT to debug will take longer than debugging by myself.") - *Experiment Design* - clearly and thoroughly describe how your experiments can help reliably and rigorously test your hypothesis (e.g. "I will do N number of randomized debugging tasks using ChatGPT and N number of randomized debugging tasks, of equal difficulty, by myself."). - *Evaluation Metrics* - clearly define what you are measuring during your experiments and how these measurements will help you test your hypothesis (e.g. "I will measure how long it takes for me to find and fix the bug.") 7. **Results & Analysis** - *Comparative Summary and Synthesis of Results* - summarize and synthesize your results in insightful ways, for example, comparing and contrasing different experimental conditions, highlighting trends and differences, etc. - *Hypothesis Evaluation* - given your experimental results, is there evidence for or against your initial hypotheses? - *Hypothesis Revision* - given your experimental results, can you revise your initial hypotheses accordingly? 9. **Broader Impact Analysis** In what ways can your findings be significant in a broader socio-technical context (see Guide to Performing Broader Impact Analysis on Canvas). 10. **Optional** For students interested in research, you are encouraged to consider additional questions, for example: - How exactly is ChatGPT trained? - What are some known issues that existing works have noted about using ChatGPT in education? Do they affect the usage of ChatGPT in CS181? - You have the option of fine-tuning ChatGPT on CS181 content (re-training ChatGPT on CS181 docs) and then have ChatGPT summarize technical content for you. Are ChatGPT summaries reliable (e.g. factual, complete, pedagogical etc)? Might we use ChatGPT as personalized instructional support for CS181? ### References 0. ***(How to Conduct Research in ML)*** [Fundamental Skills in Research](https://docs.google.com/presentation/d/1EQIupyrH7z2sUUH99CMQQJc9JzV8reTzQ6RYOeWQ1tE/edit?usp=sharing) 1. ***(The Underlying Language Model, GPT3)*** [Language Models are Few-Shot Learners](https://arxiv.org/abs/2005.14165v4) 2. ***(Fine Tuning with Human Feedback and RL, InstructGPT)*** [Training language models to follow instructions with human feedback](https://arxiv.org/abs/2203.02155) 3. ***(ChatGPT3)*** [Introducing ChatGPT](https://openai.com/blog/chatgpt)