# Early-Term Feedback Summary ### CS181 ### Spring 2023 ![](https://i.imgur.com/xDR9VQd.png) ## Design and Use of Course Components The following questions are about what is the difference between the deisgn of class, sections, OHs; how to utilize each appropriately; and what is the pedagogical purpose behind the design of each course component. 0. **Question.** What is ML (or this class) about? That is, we talk about conceptual understanding in-class, step-through math derivations in sections, practice coding on the homeworks. So what's more important -- concept, math or coding? **Answer:** ML and CS181 is about all three things: - ***Strong conceptual understanding of models:*** it's not just about heuristically knowing how each model work, but about knowing exactly how models compare, concretely when they should be used, precisely when and how they might fail and how they can be actionably improved. Conceptual understanding is informed by math but must be built up independently -- that is, *math mastery does not imply conceptual mastery*. - ***Solid grasp of some of the math behind models:*** it's not possible to master *all* the math behind our ML models, but we have identified a fundamental set of math skills that will help you ground your conceptual understanding. Math mastery is informed by conceptual understanding, but must be built up independently -- that is, *conceptual understanding does not imply math fluency*. - ***Solid grasp of the computational aspects of ML model implementation:*** In practice, you'd rarely build your model from scratch (e.g. from `numpy`). However, it is nonetheless useful that we all have the experience of building models from scratch -- this struggle (and it is a struggle for all of us) teaches us visceral lessons about why and where computing with ML models is hard (and can potentially fail). Computational familiarity with ML models must be built up independently -- that is, *conceptual and mathematical understanding does not imply computational understanding*. We know it's a big ask -- for you to engage with all three types of learning ***equally*** -- and we've built in lots of flexibility in the course structure to support your learning. You are always welcome to reach for support in any of the above asks! 1. **Question.** Why don't we derive math during class meetings? What am I suppose to get out of class meetings? Shouldn't instructors be demonstrating the derivations on the "Math Behind the Slides" instead of TFs? What's the instructor good for? **Answer:** Class meetings are for a large audience of diverse interest and backgrounds, and will be of *general interest*. They are designed to: 1. challenge your understanding understanding 2. connect different concepts from different parts of the course 3. connect theory to practice 4. show you how topics in CS181 come up in ML state-of-the-art 5. clarify really tricky math theory (not derivations) Sections and OHs (including instructor OH) are for a smaller audience and have a much narrower focus. They are designed to: 1. clarify notation and step-through derivations slowly 2. refresh on background materials if needed (e.g. reviewing the definition of derivatives, integrals, bias etc) 3. look at specific parts of the PSETs 4. do more practice problems 5. provide individualized support **The Take-Away:** Find the in-person course components that works for you (you don't need to find value in all components)! **What's your TF uniquely good for:** - **(Required and background math)** your TFs are super awesome at all the math that is required for the course (stuff on Midterm Skills Check), they can matrix complete the square with the best of them! - **(Required and background coding)** your TFs are super great at coding up all the models you'll be asked to implement! They are ninja's with `numpy`, `scipy` and `pytorch` etc. - **(Relatability)** Your TFs get you. They were students in CS181 years ago and know intuitively what you might be struggling with. **What's your instructor uniquely good for:** - **(Fancy math)** your instructor can also complete a square and would be more than happy to sit with you through any derivation. But your instructor is *uniquely* qualified to show you: - the machine learning significance in the math you're doing - connect different pieces of math to give you a different perspective on ML - show you what kinds of math we're not covering but is important for deeper understanding During class meetings, your instructor will be focusing on delivering "fancy math". - **(Experience in Machine Learning Research and Practice)** What's most valuable about your instructor is not that they know lots of math (math is well covered in many textbooks), it's that they have deep experience in applying the math to derive new math, new insights and new solutions to problems. Unfortunately, this type of knowledge (model interpretation, comparison, evaluation, critique/revision) is not well covered in textbooks. **The Take-Away:** Get to know the range of expertise on the the staff, really take advantage of the human resources available in this class and maximize what you get out of the class for yourself! 4. **Question:** I'm struggling to derive value from in-class exercises -- I find sitting in class reading through the notebook by myself frustrating/unproductive/lonely. What is the point of in-class exercises? **Answer:** In-calss exercises are designed to: - Take the burden off of coding from scratch, so that you can focus on analysis. - Challenge and deepen your conceptual understanding (with "tricky" scenarios) - Provide context for discussion with peers and your instructor - Provide code examples and templates of ML pipeline that you can use beyond CS181 In-class exercises are **not** designed to: - Be read line-by-line code-wise - Teach you to use api's during class (the api for most packages are complex and anti-intuitive and takes time to master) - Be done alone (though we recognize and respect that many people prefer to work alone) **The Take-Away:** You get the most out of in-class exercises if you focus on the analysis rather than the code and actively discuss your analysis with peers and your instructor! Alternatively, for students who prefer non-discussion based learning, you might prefer to attend a non-discussion based section rather than attend class. 3. **Question:** I understand that it's important to evaluate and interpret your models, isn't it repititve to talk about model evaluation and interpretation every lecture? I worry that we are not covering enough "technical content" by focusing on model interpretation, evaluation and ethics, and that this will affect my future career in ML. I'm uncomfortable with the vague open-ended nature of model interpretation (for example, problem 4 on HW's) and don't feel like they should be emphasized in CS courses. **Answer:** Model interpretation, evaluation and ethics is super technical! Interpretable ML, ML fairness are extremely technically challenge, active and deep subfields of machine learning! Unfortunately, unlike deriving the expression for the MLE of probabilistic linear regression, there are no "right answers" in interpretability and fairness (there are a huge number of ways to "explain" a model, and how to interpret a model varies drastically depending on the model and the context). However, just because there is no "right answer" it doesn't mean that all answers are equally good -- it is an active area of research comparing different mathematical defintions of interpretability and fairness. Since interpretability, fairness and ethics are new academic concepts for most students, we are starting off gently and will be providing more structure and rigorous lenses for these questions as the semester progresses. 5. **Question:** When I watch the videos, read the math notes and go to class I feel like I understand the material, but when I go to section I feel like there's a lot more I'm suppose to have learned! What am I missing? Sometimes the sections have an overwhelming amount of math, am I suppose to memorize all of it? I find the sections repetitive (they just summarizing the video and the math notes). Do I have to go? **Answer:** Machine learning involves arbitrary amounts of math (there is no math ceiling in machine learning, there is always more to know and you're always better off knowing more math). However, the math we require of you is limited and well-defined! We will try to be very clear what math is required for you to replication or understand (but not replicate), and what math is optional. That is, not all the math in the notes, textbook or sections are required math. You can also check for yourself what math is needed and what math is optional by frequently referring to the [Midterm Skills Check](https://hackmd.io/kgggEfujRsewF8XJ3qtsuA) and [Midterm Practic Problems](https://hackmd.io/kgggEfujRsewF8XJ3qtsuA). If you feel like you have a good grasp of the math you need, then there is no need to go to section unless you want to. 5. **Question:** There are a lot of options in the course, I have choice paralysis and/or FOMO. What should I do to find a rhythm that works for me? I dislike discussion-based learning in the flipped-classroom, what should I do? I need more discussion-based learning, how can I get more one-on-one time? **Answer:** Because we know how diverse you are in terms of your interests, background and learning styles, we have designed ways to individualize the course for you. You don't need to engage with every single aspect of the course to maximize your learning! For example... **For the discussion-based learner:** - attend class meetings with a friend group and discuss the in-class exercise with them, as well as talk to your instructor - find some discussion-based sections (not all sections are discussion based) - go to OHs, especially instructor OHs to engage in free-form discussion **For the non-discussion-based learner:** - watch the videos for content - read the math notes yourself first - go to a lecture style section where the TF talks through the math notes (not all sections are lecture style) - don't attend class-meetings, sections, OH that are discussion-based ## Expectations for Students The following questions are about what level of engagement and effort we are expecting from students. 1. **Question.** There's a lot of math in the notes, how deeply do I need to know the math (i.e. do I need to be able to derived everything from scratch)? I worry that I don't know the math well enough to practice machine learning, how do I know if I know enough? **Answer:** We have put together the [Midterm Skills Check](https://hackmd.io/kgggEfujRsewF8XJ3qtsuA) and [Midterm Practice Problems](https://hackmd.io/kgggEfujRsewF8XJ3qtsuA) to help guide your studying. You should refer frequently to these documents to prioritize skills you work on. 2. **Question.** Can you describe the expectations for three types of students: (1) student who want to put in minimal effort (2) student who wants to put in average effort (3) student who want to maximimally utilize course opportunities? **Answer:** In the following, we give some examples of different levels of engagement a student might choose. Note that we always reward more engagement (through the extra credit), but we don't penalize non-engagement with optional course content (we're happy to base your final grade just on your HW, midterm and practical points)! **Minimum Effort** - Watch most of the lecture videos/read slides - Read "Math Behind Slides" for high-level understanding, not doing line-by-line math checks - Attend a lecture or a section (to satisfy attendance) each week - Start the PSET early enough to get it done - Read textbook, notes or internet when doing PSETs - Ask questions on Ed or in OH when stuck on PSETs - Review the Midterm Skills Check and Midterm Practice Problems before midterms - Skim notes, textbook to study for midterms - Start studying for the midterm early enough to do ok **Average Effort** - Watch most of the lecture videos/read slides -- *writing down questions* - Read "Math Behind Slides" for high-level understanding, *doing some but not all the line-by-line math checks* -- *writing down questions* - Attend a lecture or a section (to satisfy attendance) each week -- *actively trying to get questions answered* - *Look at the PSET when released and make a plan for completion* - Read textbook, notes or internet when doing PSETs - Ask questions on Ed or in OH when stuck on PSETs - *Review the Midterm Skills Check and Midterm Practice Problems at the end of each week to make sure learning is on track* - *Review notes, textbook carefully to study for midterms* - *Attend midterm review sections* - *Start studying for the midterms at least two weeks in advance* **Fully Utilizing** - Watch most of the lecture videos/read slides -- writing down questions - Read "Math Behind Slides" for high-level understanding, doing some but not all the line-by-line math checks -- writing down questions - *Engage with the "Optional" materials to peek at deeper treatments of the topics* - *Regularly attend lectures -- actively trying to get questions answered, actively trying to challenge your conceptual understanding through in-class exercises* - *Regularly attend your favorite Section and/or instructor math OH to nail down math details* - *Regularly attend Beyond sections and instructor OH to relate course content to applications beyond CS181* - *Regularly seek to deepen your understanding through extended discussion with course humans* - Look at the PSET when released and make a plan for completion - Read textbook, notes or internet when doing PSETs - *Proactively attend OHs and monitor Ed when completing PSETs* - Review the Midterm Skills Check and Midterm Practice Problems at the end of every week to make sure learning is on track - Review notes, textbook to study for midterms - Attend midterm review sections - Start studying for the midterms at least two weeks in advance ## Request for Additional Support The following questions are about additional support for the course. 1. **Question:** Instead of doing summaries of the videos, can we cover that content in a different perspective or cover different content? **Answer:** I'm always happy to adjust the "summary" part of class meetings to what folks want to focus on. But again, going through derivations on the math notes take a long time (and is not broadly useful to the class), so we will save that for section and OH. 3. **Question:** Can we have more virtual sections and OHs? **Answer:** Note that all instructor OHs are already hybrid (there's always a Zoom option). Skyler's OH are on Zoom. We'll talk about making more OH hybrid. Unfortunately sections will unlikely be hybrid as that requires complex negotiation between writing on the board vs writing on the computer. 5. **Question:** Can we have more opportunities to step through math? Can we see fancier math? **Answer:** There are now two additional instructor OHs each week dedicated to math. 6. **Question:** Can we have a centralized place where we can find all the readings and resources? **Answer:** Yes, we'll try have everything on the "Module" section on Canvas. 7. **Question:** Can we make the HWs harder? **Answer:** We are happy to provide more opportunities to engage with ML: [Midterm Practice Problems](https://hackmd.io/kgggEfujRsewF8XJ3qtsuA). But the majority of students find the current HW setting to be sufficiently challenging, as such we will not increase the difficulty of HWs. 9. **Question:** Can we have more practice problems? **Answer:** Yes! We've put together a *template* for you to generate many additional practice problems: [Midterm Practice Problems]. We have also included links for you to practice coding. 11. **Question:** Can we have more opportunities to check our conceptual understanding? **Answer:** Yes! We will be importing concept quizzes for each lecture into Canvas for focus to check their conceptual understanding. 13. **Question:** Can you help me find a PSET or discussion group? **Answer:** Yes! Reach out in person or via Ed and we will help connect you to other students in the class! 15. **Question:** Can you make the PSETs, notes, textbooks etc typo-free? **Answer:** The staff is generating, updating and managing a large amount of content. Each piece of content is usually reviewed by multpile sets of eyes and we try to make what we put out as typo-free as possible. However, entirely typo-free will be impossible. In terms of not having typos affect your HW performance -- we welcome you to check your understanding, formulae, facts in OH or on Ed before you sit down to seriously implement. We also encourage you to post on Ed or come to OH whenever you encounter a weird error or issue on the HWs so that you can receive timely help!