# Talk: The Future of Teaching

Watch this on YouTube if you want: https://www.youtube.com/watch?v=S9Jor12Cxdc

## Basics

### Objectives of this talk
- Reflect on some of the less-obvious innovations of CodeRefinery
- Motivate what I think everyone *should* be doing
- Discuss the future of teaching in CodeRefinery and Aalto Scientific Computing

### Disclaimer

- This is focused on practical, hands-on teaching. If your teaching is more like "lecture" or "info dump" you will be less interested.
- These are my (rkdarst's) opinions, formed over years of teaching in-person and online.
- I will sometimes make absolute statements for dramatic effect. I do actually realize everything is a balance.

<br><br>
## The role of teaching?

### Three scales of teaching

- This is my random thought
- Can we make three levels of teaching
- One-on-one (or small group) **mentoring**
- "**Interactive** courses": ~20-30 people
- **MOOC-like**: 100 or more. (subclass: asynchronous recordings and text material)
- Technology and practices has allowed the last to become much better than before
- I think that the first and last are squeezing out the use case of the middle one.

### Learning styles
- For example,
- Some people want to **passively watch** courses to see a big picture, but don't want to get hands-on.
- Some people want to **take courses and be engaged** (for example to keep focused).
- Some want **social/one-on-one** mentoring and interaction.
- Some will prefer material (reading, videos) to follow along **later at their own pace**.
- Not everyone *needs* to be active during a course (online they don't take a seat from others)
- Some may not have time or energy to be fully engaged now
- **All of these are valid and the rest of this talk's points help support all at once.**
- **On-demand as I need it** seems to be more common

### Role of mentoring
- We can't have "teaching" and "doing" separate concepts
- After a course, people should know a follow-up
- We need good mentor networks right where people work (research groups) to support people
- Support services, RSE


<br><br><br><br><br><br><br><br>
## What is teaching?

### Old: alone, a hero
Individual skills are most emportant (but don't forget the invisible helpers!)
<a title="Japanese Ministry of Defense, CC BY 4.0 &lt;https://creativecommons.org/licenses/by/4.0&gt;, via Wikimedia Commons" href="https://commons.wikimedia.org/wiki/File:Misa_Matsushima_MODPressRelease.jpg"><img width="75%" alt="Misa Matsushima MODPressRelease" src="https://upload.wikimedia.org/wikipedia/commons/5/54/Misa_Matsushima_MODPressRelease.jpg"></a>

<br><br><br><br><br><br><br><br>
### Carpentries: small team for a workshop
Two people can do a task better and more safely than one.
<a title="Alex Pereslavtsev (GFDL 1.2 &lt;http://www.gnu.org/licenses/old-licenses/fdl-1.2.html&gt; or GFDL 1.2 &lt;http://www.gnu.org/licenses/old-licenses/fdl-1.2.html&gt;), via Wikimedia Commons" href="https://commons.wikimedia.org/wiki/File:Transaero_777_landing_at_Sharm-el-Sheikh_Pereslavtsev.jpg"><img width="75%" alt="Transaero 777 landing at Sharm-el-Sheikh Pereslavtsev" src="https://upload.wikimedia.org/wikipedia/commons/thumb/c/ca/Transaero_777_landing_at_Sharm-el-Sheikh_Pereslavtsev.jpg/512px-Transaero_777_landing_at_Sharm-el-Sheikh_Pereslavtsev.jpg"></a>

<br><br><br><br><br><br><br><br>
### CodeRefninery worshop: primary flight control
A whole crew working together puts on a large production
<a title="U.S. Navy photo by Mass Communication Specialist Seaman Dean M. Cates, Public domain, via Wikimedia Commons" href="https://commons.wikimedia.org/wiki/File:US_Navy_120207-N-RG587-065_Sailors_watch_from_primary_flight_control_as_an_F-A-18C_Hornet_launches_from_the_flight_deck_aboard_the_Nimitz-class_air.jpg"><img width="75%" alt="US Navy 120207-N-RG587-065 Sailors watch from primary flight control as an F-A-18C Hornet launches from the flight deck aboard the Nimitz-class air" src="https://upload.wikimedia.org/wikipedia/commons/thumb/4/43/US_Navy_120207-N-RG587-065_Sailors_watch_from_primary_flight_control_as_an_F-A-18C_Hornet_launches_from_the_flight_deck_aboard_the_Nimitz-class_air.jpg/512px-US_Navy_120207-N-RG587-065_Sailors_watch_from_primary_flight_control_as_an_F-A-18C_Hornet_launches_from_the_flight_deck_aboard_the_Nimitz-class_air.jpg"></a>

<br><br><br><br><br><br><br><br>
### Workshop series: a fleet
(image not included)

<br><br>
## CodeRefinery and Aalto Scientific Computing innovations

Audience feedback (summer 2021, course using most of these techniques, several hundred participants). It really works:

> I attended several "top" conferences/workshops/seminars as well as
videolectures this past year in their virtual implementations, and this
event is easily the best out of all of them when it comes down to
presentations and audience participation!

<br><br>
### Online

- Everyone was forced to go online in 2020.
- I think we were more successful than most.
- Many (but not all) other topics you will see below are made possible by being online.

<br><br>
### Co-teaching

- rkdarst: "I am done with teaching alone"
- Every lesson should be arranged with two co-teachers: not alternating in sections but teaching as discussing between people.
- In a large class (or even medium), you don't get learners interacting with you.
- Instead, provide your own interaction.
- It makes it like a show and provides meaningful engagement
- It takes some getting used to, but in the end reduces workload (you can help each other live).
- Easier to onboard new instructors
- **Teaching together = sharing our own skills together**, also about infra and support
- Our manual (link below) gives some hints on how this can work in practice.

Link: https://coderefinery.github.io/manuals/team-teaching/

<br><br>
### HackMD for communication

- HackMD is a simultaneously editable document
- Any other option could work too
- Don't expect learners to speak up - let them write up!
- Multiple ways to answer
- You can answer by voice
- You can also answer by writing
- Don't use linear chat: too disorganized
- You can get as many helpers answering as you need
- You post it afterwards, there is a permanent record.
- **Include in your screenshare**, so learners will know it's active.

Learner info: https://coderefinery.github.io/manuals/hackmd-mechanics/
Helper info: https://coderefinery.github.io/manuals/hackmd-helper/

Question: what if you want to show a webpage opening a link in your materials?
A: I don't quite understand this question, but I can put links and open in web browser as needed.

<br><br>
### Get many helpers

- The above mean it is *very* easy for helpers to join
- For a single session
- In spare time during their main job
- Across borders
- You start getting massive courses with many helpers
- Lower cognitive load per person
- Needs good coordination! But it's not so hard - it becomes natural after you experience it once.

<br><br>
### Portrait screen sharing

- Do not share a FullHD screen if you expect learners to have space to follow along
- Share a vertical screen
- 840x1080 encodes to youtube at pixel-perfect resolution
- **Just share vertically by default**, unless there is a strong reason otherwise.
- It takes some getting used to but we have few long-term complains
- Keep your screen sharing simple!
- You have half the screen available for HackMD/instructor chat/etc.
- Question: how to share the vertical screen in Zoom?
- under "advanced screen sharing": share portion of the screen


<br><br>
### Teams and breakout rooms
- Even 20 people is too much to form a community
- Solution: **Teams**
- Pre-assign learners to teams of 5-6
- One "exercise leader" per team
- Each team gets a breakout room
- **Teams the same across the whole course**
- We then do heavy recruitment for these Exercise Leaders
- How can we motivate them more?
- This is labor and coordination-intensive, but has an amazing effect
- Bring your own breakout room
- If learners and helpers already know each other the effect is even better
- More lasting effect for the research group
- Example: CodeRefineries, ~100 learners, ~20 teams

Exercise leader training: https://coderefinery.github.io/manuals/helper-intro/

<br><br>
### Livestreaming

- Zoom is better than it was 1.5 years ago...
- ... but it is still a major limiter in online teaching
- Limited meeting size
- Risk of trolls, joining must be resticted
- Learners hear all instructor talk
- Learners can say things and be in recordings
- ... but streaming services have the ability to reach millions of people
- Since we have HackMD for communication, and co-teaching to provide interaction, **why do we need to be in the same meeting as the learners?**
- Livestreaming provides a solution
- We use OBS (Open broadcaster software)
- **In the future I emphasize livestreaming and big audiences**

Read more: https://coderefinery.github.io/manuals/coderefinery-mooc/

<br><br>
### CodeRefinery MOOC strategy
- Private Zoom for instructors
- Zoom captured and livestreamed
- Learners follow livestream by
- themselves
- in small teams
- in other courses
- Interaction is via HackMD
- Instructor Zoom is private, allows planning and resting during breaks.
- This takes getting used to but people have liked it
- Questions:
- how do instructors know whether it is going well in the teams/exercises?
- What would be the first step when people would move from "basic" powerpoint Zoom lectures into your model Richard?
- [RB answering so answer may not be "right"]: in my opinion this model is a good step as soon as we want to record or stream while still having interaction

Information flow:

<br><br>
### Disributed registration / local watching parties

- Breakout rooms can be organized **without main coordinators knowing**: scale independently
- So livestreaming can expand and somehow keep this personal touch.

<br><br>
### Clarify and separate roles

- Isn't the above hard to do? Yes, but we can divide it into distinct roles
- Our roles
- **Co-instructors** - Just teach, don't worry about other things. It's actually easier than other strategies
- **Host** - people management
- **Director** - tech management
- **Helpers** - spare hands, asking and answering questions. Can join instructor room and speak up.
- **HackMD manager** - keep HackMD formatted well
- **Learners**
- Instructors just teach, no other tasks
- Some roles can be combined but be careful.
- Most people seem to think a massive course is easier, because we divide the tasks up.
- Varied roles and co-teaching makes it easy to onboard people

Link: role descriptions in manuals: https://coderefinery.github.io/manuals/

<br><br>
### Recordings (+ made available immediately)

- Recordings allow
- Learners to review later, at their own pace
- Catch up if they miss one day
- Go along with our free-flowing reading material
- (I don't consider the main point to be "people taking the course later" but that certainly does happen)
- Recordings allow us to "compete with ourselves": instead of coming to class some fraction of people might instead be happy with watching later. This is not bad.
- Recordings can only be done if learner privacy is **guarenteed**.
- If recordings not done within 24 hours, they will likely not be done.
- Most useful same-day, so learners can catch up missed days
- No time for careful review to remove learner voice or video.
- How to
- use MOOC strategy (no learners in instructor room = guaranteed privacy)
- OBS provides high-quality recordings
- Livestream services provide instant recordings.
- ffmpeg-editlist provides quick editing: https://github.com/coderefinery/ffmpeg-editlist

Example: ASC YouTube playlists: https://www.youtube.com/channel/UCNErdFO1_GzSkDx0bLKWXOA/playlists

<br><br>
### Free-flowing materials

- Often our teaching is free-flowing text (like this), not slides
- Benefit: easy to use for self-study
- Benefit: It is easier for learners to scroll up and down
- Include text headings / link anchors
- It helps make it easy to use it as reference while doing hands-on
- But I am still figuring out the best balance for other types of talks
- sphinx-minipres turns a normal webpage into a slideshow (still under testing, not used much)

Example: this talk (not that it's ideal)
Example: CodeRefinery: https://coderefinery.github.io/git-intro/

<br><br>
### Define your audience well
- Set expectations well, or a large number of people will be disappointed
- Clearly define what prerequisites are
- But don't gatekeep
- Clarify what people can get if they don't meet prerequisites
- Accept passive learners (lurkers): people can watch without being active, online they don't take a seat from someone else
- When you do something advanced, clearly indicate
- "Don't follow along now, this is a demo", "most people will watch this as a demo", "this is advanced"

Example: https://aaltoscicomp.github.io/python-for-scicomp/#who-is-the-course-for

<br><br>
### Have a good schedule
- Since it is online, peole can easily drop by for one session
- Once you define your audience well, you define which sessions are relevant to which audience
- Make it easy for people to drop by for the parts they are interested in.

<br><br>
### Backwards lesson design

- There is a major trap
- We teach what **we** would do
- Not what a **new learner** would do
- Always backwards-design teaching material based on the final result.
- What will the person be able to do (~3 things)
- What will they remember? (~3 things)
- Minimize unneeded dependencies and complexities

Link: https://teachtogether.tech/en/index.html#s:process

<br><br>
### Installation help
- Installation instructions can make or break a workshop
- Defense in depth:
- Minimize dependencies
- Have "installation verification" steps
- Video demo of installation and/or verification
- Installation help session
- Tested on different operating systems
- Online allows more opportunity to have advance installation help/verification sessions.
- Equally, test the lesson on other operating systems! And for leaners who don't have all your advanced tools.

Example (CodeRefinery): https://coderefinery.github.io/installation/
Example (Python for SciComp): https://aaltoscicomp.github.io/python-for-scicomp/installation/

<br><br>
### Good use of screensharing
- At any point, screenshare should always answer the question "Where are we? What should I be doing now? When do I need to be back?"
- During breaks: `Break until xx:10`
- Exercises: links, what to accomplish and focus on (and what is *not* expected), end time.
- Example: `Exercises until xx:45, then brief discussion before a break. Everyone should be able to finish ex 1-2. 3-4 are harder and don't worry if you don't finish them.`
- During follow-along work: are there clear titles and section headings so that people can find the place?
- Include HackMD when needed, so learners know it is active
- Screenshare so that it's easy to process videos later
- Section heads are visible
- Point cursor to what you are discussing
- This also helps learners to follow.


<br><br>
### Design public-first
- Design for re-use from day one
- It's not worth being "private until it is ready"
- Simpler access management
- When teaching: times in generic format, like "xx:10"
- Open science and all

<br><br>
### Breaks
- Not negotiable
- Minimum 10 minutes
- Don't combine with exercise sessions or other optional material
- Design lesson around breaks, not breaks around the lesson
- We tend insist on "10 minutes, somewhere between xx:50 and xx:10"
- Make sure staff can take breaks, too.

<br><br>
### Feedback - real-time
- Old style: "I need to hear from learners to know if they are following and see learners faces so I can know if they get it"
- How many actually speak up?
- Do faces really tell you much?
- New style:
- People can write problems in the HackMD
- They don't just write problems, but also real-time questions (since helpers might answer them)
- Your *real* feedback comes from your co-instructor and team of helpers who will actively, not just passively, give you constant feedback
- I (rkdarst) feel much better teaching with this system.

<br><br>
### Feedback - afterwards
- Always ask for feedback
- immediate
- anonymous
- Asking for voice answers won't get you what you need.
- We end with two key questions:
- Name one good thing
- Name one thing to be improved
- Post-workshop surveys if possible?
- We should also **give each other feedback as instructors**

Example: https://coderefinery.github.io/2021-05-10-workshop/questions/day6/#feedback-for-the-workshop

<br><br>
## Other minor recommendations
- Start the course 15-30 minutes early, always plan on staying 15-30 minutes late
- Before: icebreakers, (co-instructor) chat
- After: Discussion
- Consider this for other meetings in general
- Demonstrate how to interact and ask questions.
- Helpers seed questions on HackMD
- Keep un-redistributable information out of the stream (so that you can post the video without editing it out)
- ... and plenty more found in our manuals (and instructor training): https://coderefinery.github.io/manuals/

<br><br>
## Open questions
- Is forcing people to come in-person useful just because it forces us to

<br><br>
## To do for the future

There is plenty of stuff we still need to work on

### More partnerships
- Now that we can scale, why not work together?
- We can provide more specialist courses
- Join CodeRefinery: https://coderefinery.org
- CodeRefinery standardized workshop templates to make things easier?

### Reduce workload on coordinators
- Registration and instructor coordinator have a huge load
- They should each probably only coordinate one course per semester (unless this is their primary job)
- It seems that this will be our limiting factor for teaching

### Instructor training (CodeRefinery instructor training)

- My vision: Not teaching dedicated teachers, but technical staff who need to teach as part of their job
- Our strategies aren't that hard (except possibly the director), but do take some getting used to.

### Make streaming simpler
- The tech setup can be a bit complex
- But I think it could be simplified a lot:
- One laptop with external monitor for only broadcasting
- Other instructors need just one computer
- Hackthon early January for this

*rkdarst's streaming setup during Python for Scientific Computing. This looks fun but is just excessive:*

### Instructor mentoring process
- A better way to give feedback to each other

### Subtitles
- Somehow, subtitles have become a proxy for accessibility (even though the effect I see is "release fewer videos").
- We are far more accessible than other courses: instead of being limited to those who can follow along live, it is put on YouTube and gets automatic subtitles. Possible to review later.
- Can we make this better?

## Conclusions

- I consider in-person teaching is almost obsolete (← exageration)
- The old way of online teaching (zoom) is not good enough
- Teaching alone is wasting resources (and some good grant possibilities)
- The following are very important:
- Co-teaching
- Hackmd
- Portrait screen sharing
- Recording
- By working together and using these strategies, we can
- Reach more people
- Have more interactivity
- Easier by taking better advantage of a wide variety of helpers
- Wider variety of specialized courses
- Have more fun

<br><br>
## What do we do now?
1. Common space for discussion. Work in public by default (CR chat).
2. All lesson materials FAIR
3. All lesson materials able to accept contributions with low effort (PRs)
4. Train the Trainer (or maybe more like "try it out session")
5. Some co-teaching in existing workshops
6. Workshops coordinated together
7. Whole training plan synced

## See also

- CodeRefinery manuals: https://coderefinery.github.io/manuals/
- Demo of livestream teaching, with explanations: https://www.youtube.com/watch?v=WjmttAniZX8

## Q&A

- Ask questions like this
- Discuss like this
- The hands-on course format you present requires around 4 people, that is many. When does it break-even? How large a course is needed to make it worth it?
- rkdarst: strategies can be mixed-and-matched to do it with fewer people. 4 is plenty but fewer could be possible (minimum is two co-instructors)
- rkdarst: Also consider that people can drop by without disrupting other work: it's much easier to get four people.
- rkdarst: but my guess is it's worth it with 50-60 learners.
- What do you think about the role of the written material in these kinds of courses? Should it work as stand-alone materials or should it be specially created so that it works with pair teaching?
- rkdarst: I am generally biased towards "slightly better standalone, which is slightly worse for teaching". After some time you get better at balancing these, I think.
- These courses don't facilitate giving credits, do they? How do you see this could be made possible?
- rkdarst: Presence vs engagement fallacy (even in a lecture hall, does it mean people are paying attention?)
- rkdarst: Assignment of credits gets pushed to monitoring engagement in the local breakout rooms - there is no real change here
- rkdarst: I have thought of things like random real-time surveys to make sure livestream attendees are *present*. Then the usual "do the homework, write reflections".
- rkdarst: This is something to develop later in the future
- What would be the first step in moving from "basic" Powerpoint Zoom lectures to your model Richard? Start with incorporating HackMD into lectures, for example?
- [CodeRefinery manual](https://coderefinery.github.io/manuals/online/) with different levels of online training.
- I didn't get the 5-6 people per team, but 4 instructors for 60 participants combo. What happens in breakout rooms?
- The 4-people didn't include the helpers in break-out rooms. It takes 4 to have a single excercise room (one *expert helper*). Then one more *excercise leader* for each break-out room. The *leaders* get a training.
- (Discussion about hybrid courses)
- rkdarst: My current feeling is we should be "online-first", and the local interaction is provided by smaller groups distributed around locations. I don't think the benefits of hybrid are worth the costs of not being able to do the above.
- rkdarst: Some things like co-teaching/HackMD provide a level of engagement that exceeds traditional lectures (of course these aren't exclusive with teaching in a classroom, but I think it is harder. And experience is less good when you are limited to one laptop).
- (what about no-shows in large courses?)
- rkdarst: When it is online, who cares? We don't lose a seat for someone else.
- rkdarst: People are busy, maybe they can't attend each week. Maybe people want to drop by for just a few sessions, or try it out the first day. Or mostly follow along the videos. I think these are all valid cases and we should say they are OK.
- rkdarst: I would optimize for number of attendees, not attendance rate. I'd rather 3x the number of people register and 1/2 sign up, than 1x register and 100% attend.
- (Postprocessing videos takes a lot of time)
- rkdarst: ffmpeg-editlist makes this quite seamless (rkdarst started it): you make a YAML file defining the cut points, then run it asynchronously. It's really fast.
- rkdarst: my (rkdarst's) biggest barrier is reviewing it for privacy (learners appearing) - this is why I am so strict on this separation. Beyond that, learners seem to strongly prefer something good enough done fast, as opposed to perfect done never.
- rkdarst: https://github.com/coderefinery/ffmpeg-editlist
- (Aren't you overly-concerned about the privacy aspect?)
- rkdarst: possibly. Yes, we could say "if you say anything, then it will be recorded and published". But to me I'm not fully happy with that, it seems like we are speaking from a position of power. What about mistakes? It causes quiet people to stay more quiet. Better to separate the instructors from the learners/helpers and protect privacy and give them the freedom to speak.