# 2022 Online QuantEcon Workshops Report

**Authors:** Matthew McKay, Thomas J. Sargent and John Stachurski

**Date:** October 2022

QuantEcon ran two online workshops on computational economics during the summer of 2022, one to students based in Africa (July 2022) and another to students in India (August 2022). In addition to education, we aimed to build partnerships and gain experience in online content delivery, with a focus on laying foundations for activities described in the Schmidt Futures India summer course proposal.

## Overview

The workshops were centered on 30 pre-recorded videos and accompanying Jupyter notebooks released via Google Classroom. We invited students to watch, listen, and participate by working through problems and writing code associated with each session. Live components were added to provide feedback and enhance interactivity.

### Africa Workshop

Quantecon organized [this workshop](https://quantecon.github.io/ASE_ENSEA_workshop/) in conjunction with The African School of Economics (ASE), the Institute of Statistics and Applied Economics, and Ecole Nationale Supérieure de Statistique et d’Economie Appliquée (ENSEA). The workshop ran from from July 11 to July 22. It attracted 214 registrations, 52\% from ASE (Benin), 7\% from ASE (CIV), and 30\% from ANSEA (CIV). The remainder of registrations were from a variety of institutions across Africa. Approximately 67\% of participants were economics majors. 72\% were Masters level students and above.

Before the workshop, we conducted a Google poll in which we asked prospective students to tell us about their backgrounds in economics, various parts of mathematics and statistics, and computer programming, if any. A majority of the participants reported an understanding of multivariate calculus, linear algebra, and statistics. 30\% of participants reported a modest to high level of experience in computer programming in Python.

In the second week of our Africa workshop, we provided a live lecture via Google Meet. Positive feedback on this lecture encouraged us to put more emphasis on adding live components to our August India workshop.

### India Workshop

Quantecon designed [this workshop](https://quantecon.github.io/indian_summer_workshop/) in conjunction with the Department of Economics at Ashoka University, the Delhi School of Economics, and the Indian Statistical Institute. It was offered to a wide range of participants across multiple Universities across India running 08th-12th and 15th-19th of August.

The workshop attracted 1,325 registrations. We conducted a Google poll beforehand and found that approximately 93\% of participants were economics majors and 82\% were Masters level students and above. A majority of the participants reported an understanding of multivariate calculus, linear algebra, and statistics. 10\% of participants reported a modest to high level of experience in computer programming in Python.

Prior to the workshop, we selected and trained 10 teaching assistants (TAs) from the group of registered participants. During the workshop, the team of TAs provided 20 tutorial sessions each afternoon and evening. Each tutorial contained exercises (distributed through Jupyter notebooks) directly related to the material presented that day. Assistance from TAs was essential given the scale of the India workshop, enabling us to offer a highly interactive experience for participants. TAs themselves reported a strongly positive learning experience.

## Infrastructure

This section reports learning outcomes for our team regarding infrastructure.

### Integration Issues

In order to simplify workshop delivery machinery, we sought a single provider of services to deliver the workshop. We selected the Google suite of products (Google Classroom and Google Colab) because of its support for hosted Jupyter notebooks via Google Colaboratory and available integration with notebooks posted on Google Classroom.

Google Classroom provided a reasonable quality platform to deliver content. Nevertheless, having to use multiple services to communicate and organise the workshops caused a relatively high manual workload component and was error prone. Limited integration of the various services caused a range of problems. Using individual services also causes some confusion on the user end when students end up receiving messages about the classes from multiple systems.

The use of Google Groups to send messages to students led to a poor user experience. Mailing list support was switched to Mailchimp, which was able to produce more professional, easier to read emails with rich HTML support. It also allowed us the scale to reach all those registered for the India Workshop.

While the Google product suite was able to support 20 daily tutorial sessions, setup of each session required manual processing and a separate registration process was required for workshop participants via Google Forms.

In the future we will aim for a more integrated platform for running these workshops, where a hosted information webpage for the upcoming course is integrated with registrations and mailing list support.

### Registration and Authentication

We ran into conflicts with universities already using Google products. Google Classroom could not be used by some workshop participants via their Google-managed university login details because Google limits cross-domain logins by default. While QuantEcon enabled external access to our QuantEcon organisation hosted Google Classroom, Google requires both systems to enable cross-domain logins, resulting in many students having to sign up to a private Google gmail account to access materials.

In the future we aim to offer a variety of login options including OAuth for GitHub, Google and/or locally managed logins to enable students a range of options when signing up to attend a workshop to prevent reliance on any single login system.

### Automatic grading

One key feature we lacked for the delivery of these workshops was automatic grading functionality. While Google Classroom enables the inclusion of quizzes, polls, and assignments, it lacked any automatic grading for computational content through Jupyter notebooks. We considered [otter-grader](https://otter-grader.readthedocs.io/en/latest/) but encountered difficulties with the need to collect notebooks from workshop participants and then match the notebooks to an ID for processing on Google Classroom. We also considered the use of [nbgrader](https://nbgrader.readthedocs.io/en/stable/) but found it unsuitable given the need to be hosted in a Jupyter Hub environment.

In 2023, we will prioritize automatic grading for future workshops. Unfortunately, most current offerings lack the ability to integrate with a computational kernel. Of the projects we evaluated that support Jupyter notebooks, the deployment environment and/or lack of integration imposed large overhead costs. There is significant scope to develop a new tool to provide automatic grading functionality. Such a tool could also be developed to make use of the most modern tooling including web assembly (WASM) to run supported programming language kernels in the browser.

### Video Content

While Youtube paired well with Google Classroom, it required students to watch a video and then either open the lecture in another window or else come back to the written lecture material after watching the video. Having the ability to embed video into a workshop or lecture page could substantially improve student experience. It could also change how videos support material. Making small focused videos that teach a concept or idea may better target student learning and support the written material in a more collaborative way.

### Improved Interactivity

The Google Classroom product builds in a very basic form of messaging that is largely focused on class-wide messages rather than discussions on a topic. For the workshops in Africa and India we setup discourse to have an avenue whereby students could discuss the lecture material and pose questions for the teaching staff. However, we found engagement to be low, partly because it was not integrated with material delivery. A learning management system (LMS) with in-built messaging support could improve engagement from students if they can easily hold online discussions, either based on a lecture page, or in a broad discussion forum.

## Timing

During the India workshop, feedback on tutorials was generally very positive but some reported lower attendance rates than expected, given the signup rate on Google Classroom. By questioning students and TAs, we found that Covid-related shedule changes had led to conflict between our workshop and exams in some schools. We have noted the critical importance of minimizing scheduling conflicts going forward.

In general, participants had limited amounts of time each day to commit to the workshop series. New designs based on less content per day should be considered. If platforms are more automated we could also make the workshops longer to accomodate a slower pace.