---
# System prepended metadata

title: 'Module 2: Syllabus Planning'
tags: [RDS_Module_2]

---

# Module 2: Syllabus Planning

###### tags: `RDS_Module_2`

[toc]

## TODOs:

- Course dataset
- Explore Compas - How clean is it? Do we need to introduce artificial missing values etc?
- EDA: To what extent do we need to do EDA before cleaning?
    - Look at each column independently
- Separate datasets used in taught vs. hand-on sessions probably preferrable to challenge attendees in hands-on sessions
- Examples of issues with raw data: e.g. scenarios where there are issue in collection
- Anything else to include in taught elements or exclude?
- How to store cleaned data? e.g. tabular in dataframe?
- Data protection/"personal data", what to look out for?
  - making data public/private

## Course Objectives and Outcomes

The main objectives of the proposed course are the following: 
•	Teach attendees how to use research data science (RDS) methods in an interdisciplinary research environment.
•	Move beyond DS principles offered by existing courses, towards the *craft* of doing RDS: A hands-on, practical understanding, focused on collaboration, reproducibility and openness. 
•	Provide exposure to a diverse set of real-world RDS projects and demonstrate the decision-making process used to choose the right method and tools for each setting and in each project step. 
•	Embed data ethics, diversity and inclusion awareness into the attendees' approach to all stages of a RDS project, providing multiple examples.

The expected outcomes are the following:
•	Attendees will understand fundamental RDS methods and tools and know when/how to apply them to their PhD/postdoctoral research in order to draw data-driven insights or create data-driven tools. 
•	Attendees will be familiar with the stages of an RDS project, from scoping and data exploration to deployment and will become aware of the challenges of tackling ambiguous real-world problems.
•	Attendees will be able to recognise power imbalances, bias and diversity issues in their technical work and in their ways of working and challenge them.
•	Combined with our existing RSE courses, attendees will have the necessary tools to apply for junior level RSE/RDS positions.


## Module Plan

Module 2: Handling data and deployment
- Sub-module A (Taught) - with tailored data/examples for the material:
  - Data wrangling, cleaning, provenance, testing
  - Handling missing data, different types of data (e.g. text, datetimes)
  - Data access
    - Basic Databases (SQL, accessing from Python)
    - APIs 
  - EDI: Why raw data are not raw
    - sampling bias
    - collection error
    - badly designed capture/missed variables (e.g. missing q in survey, leading q)
    - subjectivity/inconsistent labels
  - For all the above: Focus on which action is appropriate for each dataset or situation.
  - Basic deployment (Docker) and reproducibility.
    - version control, requirements.txt, virtual environments...
- Sub-module B (Hands-on project) - with COMPAS/equivalent dataset used throughout course:
  - Attendees will need to access a (fake) NHS data set from an Azure database and figure out how to pre-process it to prepare the data for later use. They will need to discuss and decode various complexities (e.g. missing/ambiguous values, bias in data collection, data privacy and sensitivity).
  - Attendees will need to work in teams, reviewing each other's code and all contributing to a common code base and will have to use a simple Docker container for reproducibility.

## Similar MOOCs / Refs

- John's Hopkins Data Science Specialization - Course 3 Data Cleaning: https://www.coursera.org/learn/data-cleaning?specialization=jhu-data-science. Includes
  - reading data
  - summarize/sort
  - reshaping/merging
  - regex
  - dates 

- Data Feminism: https://mutabit.com/repos.fossil/datafem/uv/datafem.pdf

- Turing way course: https://docs.google.com/document/d/1vnLZ1LpHCUHyESvnTFHob_ObF4OHnWSw/edit

- IBM - Data Analysis with Python: https://www.coursera.org/learn/data-analysis-with-python?specialization=applied-data-science#syllabus. Includes:
  - Importing Data
  - Data Wrangling
  - EDA
  - Model Development

- Databricks - Applied Data Science for Data Analysts: https://www.coursera.org/learn/applied-data-science-for-data-analysts?specialization=data-science-with-databricks-for-data-analysts#syllabus. Includes:
  - Feature Engineering and Selection:
    - feature eng.
    - missing values
    - data types 


# Syllabus

## Getting Data (https://hackmd.io/7rron7vtTLK_uKx_5A_PrQ)

### Where to find data?

Popular online sources of data. Often domain specific.

E.g.
 - ONS
 - data.gov.uk
 - kaggle
 - [UCI Machine Learning Repository](https://archive.ics.uci.edu/ml/index.php)
 - [World Bank](https://microdata.worldbank.org/index.php/home)
 - sklearn, tensorflow datasets, pytorch datasets (ready to go w/ framework)
 - GitHub
 - [Zenodo](https://zenodo.org/)
 - https://datasetsearch.research.google.com/
 - [UK Data Service](https://www.ukdataservice.ac.uk/)

or:

- How to find data if it's not available in a common repository?
- Creating/collecting your own dataset.

### Legality

#### Licenses

Ensure that license for your chosen dataset fits use.

- Common license types
  - example permissive
  - example non permissive
- (GitHub) default (no license)
- Public domain? (TODO: need to find out what's allowed here)

#### GDPR

High level summary. Redirect to specific training.

- What is **personal** data?
- Protected characteristics
- Resonsibilities: collection, storage, right to be removed, 
- Roles (data protection officers etc.)

#### Commercial agreements

Abide by any agreement made with partners

### Pandas intro


### Data sources/querying data

- check what's covered in RSE course

#### Databases

- SQL
  - Common flavours: PostgreSQL, SQLite, MySQL...
- (NoSQL)
  - Elastic, MongoDB... 

Where:
- Local vs hosted

Getting data:

- Querying  (select, join...)
- An example with a python package (pandas, sqlite3, ...)

TODO: some broad guide of which type of DB to choose (find a blog to link to?)

#### APIs

- https://github.com/public-apis/public-apis
- web api example

#### Scraping data

light touch: point towrads popular packages like [Scrapy](https://scrapy.org/)

- legality

#### Tabular Data

- csv example
- (xls...)

#### Structured Data

- json
- (yaml, xml)

#### Other

- Image, text documents, video, spatial, binary

### Security


#### Controlling access 

### What's the Data For?

- Covered by Module 1?

What task/question are we trying to answer with the data?
How to choose a dataset based on the task/question

### Bias

- Types of bias to be aware of in data

### Linking datasets

Mention idea but cover in Data Wrangling

## Types of Data

- Data types
- Non tabular data (image, audio, video, spatial, documents)
- Temporal data


## Wrangling and cleaning data

- Data consistency (e.g. consistent datatypes)
- Missing data
- Regex
- Binning
- Date processing
- Privacy/Anonymisation
- *Should* data be used?
- badly designed capture/missed variables (e.g. missing q in survey, leading q)
- subjectivity/inconsistent labels
- Wrangling text/image data?

## to be sorted:

- Saving/publishing data
- Virtual environments, dependencies etc. (? maybe module 1 or intro to hands-on exercise)



# Chris Burr Ehtics Course Meeting (4/5/2021)

- [Course Proposal](https://thealanturininstitute.sharepoint.com/:w:/s/dp/Ed3YNc6MCBdCnOh4M9tYFN8BXmUxhZ8qGKTopKDIe2sAQg?e=PnNnKM&wdLOR=c77BAAB4D-6E64-0D4D-81BC-2230D6B645E6)
- Collaborate between ethics & RDS courses?
- COMPAS: Maybe problematic (controversy/sensitivity of topic may distract from core content of course). Consider alternatives + synthetic data?

- Syllabus
  - 3 independent modules:
    - ethics & governance 
    - responsible research (prob greatest overlap with RDS course)
    - public communication  

- Resp research & Innovation:
  - ~10 lectures (of ~1 hour each) + exercises
  - No pre-requisite to know Python. Should know general statistics/data analysis.  
  - Qualitative assessment (exercises to think through problems, not exercises with fixed solutions)
  - explore previous examples (cambridge analytica etc.)
  - AI lifecycle
    - DESIGN 
      - project planning
      - problem formulation
      - data extraction/procurement
      - data analysis
      - preprocessing
    - DEVELOP
      - model selection
      - model training
      - model testing & validation
      - model reporting
    - DEPLOY 
      - model implemenetation
      - user training
      - system use and monitoring
      - model updating/deprovisioning
  - Planning exercise of exploratory analysis looking at biases in a Jupyter notebook, and others. 
  - Interest in creating synthetic dataset with desired properties (e.g. bias in missing data...)
  - https://catalogofbias.org/
  - https://alan-turing-institute.github.io/rrp-selfassessment/introduction.html
    -  Bias self-assessment (examples of types of bias in healthcare and questions you might ask) 
  - https://cdeiuk.github.io/bias-mitigation/
    - How they created a synthetic dataset:
      - https://github.com/CDEIUK/bias-mitigation
      -  See `notebooks/notebooks/recruiting/preprocessing.ipynb`

- Particularly interested in healthcare, criminal justice, and hiring examples (in that order of preference).
- Have a fairly large budget, including for graphic design.
- No requirement for single dataset used throughout the code - probably want something for exploratory data analysis, and something for reporting.
- See their course as theory, ours as practice, so two should complement each other well.
- Design workshops in June - we could go to observe if we like

